Evaluating forecast accuracy for Error Correction constraints and Intercept Correction

Richard Eidestedt & Stefan Ekberg

Supervisor: Johan Lyhagen

Bachelor Thesis Department of Statistics Uppsala University Fall 2012

Abstract
This paper examines the forecast accuracy of an unrestricted Vector Autoregressive (VAR) model for GDP, relative to a comparable Vector Error Correction (VEC) model that recognizes that the data is characterized by co-integration. In addition, an alternative forecast method, Intercept Correction (IC), is considered for further comparison. Recursive out-of-sample forecasts are generated for both models and forecast techniques. The generated forecasts for each model are objectively evaluated by a selection of evaluation measures and equal accuracy tests. The result shows that the VEC models consistently outperform the VAR models. Further, IC enhances the forecast accuracy when applied to the VEC model, while there is no such indication when applied to the VAR model. For certain forecast horizons there is a significant difference in forecast ability between the VEC IC model compared to the VAR model. Keywords: Forecast Accuracy, Vector Error Correction, Vector Autoregressive, Co-integration, Intercept Correction and Diebold-Mariano test

Contents
1 Introduction .........................................................................................................................................1 1.1 Previous research ............................................................................................................................2 2 Methodology and Data ........................................................................................................................4 2.1 Data ................................................................................................................................................4 2.2 Methodology ..................................................................................................................................5 3 Theoretical framework .......................................................................................................................7 3.1 Vector Autoregressive models .......................................................................................................7 3.2 Vector Error Correction models .....................................................................................................8 3.3 Intercept Correction ........................................................................................................................8 3.4 Evaluation methods ........................................................................................................................9 3.4.1 Diebold-Mariano test ...........................................................................................................10 4 Estimation and results.......................................................................................................................12 4.1 Estimation.....................................................................................................................................12 4.1.1 The VAR model ..................................................................................................................12 4.1.2 The VEC model ...................................................................................................................12 4.1.3 The ARIMA model .............................................................................................................13 4.2 Results ..........................................................................................................................................14 4.2.1 One step-ahead forecast performance..................................................................................14 4.2.2 Five step-ahead forecast performance .................................................................................15 4.2.3 Overall forecast performance ..............................................................................................16 5 Conclusions ........................................................................................................................................19 References .............................................................................................................................................20 Appendix A - Figures ...........................................................................................................................22 Appendix B Eviews syntax................................................................................................................34

1 Introduction
Forecasts of macroeconomic variables are of great importance to numerous economic agents within a countrys economy. One of the most employed macroeconomic variables is Gross Domestic Product (GDP). GDP is the total market value of all final goods and services produced in a country in a given time period, and is the most common indicator of a countrys financial health and development (Statistics Sweden 2012a). Industrial decision and economic policy making is to a large extent based upon forecasts of economic variables. Due to the primary role of GDP as an aggregated economic measure, it heavily influences most of these decisions. Hence, it is imperative that the forecasts of GDP are as reliable and accurate as possible. Inaccurate forecasts may result in poor economic decisions with a destabilizing effect on the business cycle. The Swedish Ministry of Finance (SMF) provides one of the most influential GDP forecasts for the Swedish economy. In a recent publication, the SMF have employed a modification of a common Vector Autoregressive (VAR) model, which is often used as a reference point for GDP models. The modification was made so that the model better accommodates the Swedish economy (Bjellerup & Shahnazarian 2012). VAR models have proven to offer a number of advantages for forecasting economic time series. The estimation procedure is simple and knowledge of underlying theoretical concepts is not required. The forecasts generated by VAR models are also in many cases better than those from simpler models and large-scale structural models (Brooks 2002). However, one severe disadvantage of the VAR model is that it requires stationary time series. In most cases the stationarity requirement leads to differencing and thereby information on any long-run relationship between the variables will be lost. Granger (1981) presented a solution to this problem by introducing the relationship between cointegration and Error Correction models, which was further extended by Engle and Granger (1987). They showed that although individual time series are non-stationary a linear combination of those series can be stationary without differencing. Such relationships are referred to as co-integration, which means that there exists a long run equilibrium relationship between the variables. Error correction models draw upon the co-integrating relationship by allowing long-run components of variables to abide equilibrium constraints while short-run components have a flexible dynamic specification (Engle & Granger 1987). According to Engle and Granger, a pure VAR is misspecified if there exists a co-integrating relationship between the variables. In presence of such relationships they advocate a restricted VAR model, known as the Vector Error Correction (VEC) model. However, forecasts are rarely based on the estimated models alone, adjustments are often made. In recent literature, dominated by David Hendry and Michael Clements, the importance of such adjustments in VAR model forecasting using non-stationary time series is emphasized. Clements and Hendry (1996) state that models, that assume a constant, time-invariant data generating process (DGP), implicitly rule out structural change or regime shifts in the economy. They imply that such models ignore important aspects of the real world. A solution to robustify forecasts towards structural change is advocated. The idea is to correct the intercept at each forecast origin to realign the forecasts after a deviation has occurred. These adjustments are often referred to as intercept correction (IC) and have long been known to improve forecast performance in practice.

The purpose of this paper is to examine whether it is possible to improve the forecast accuracy of an unrestricted VAR model, by imposing an Error Correction constraint to account for a possible co-integrating relationship and further apply IC to the forecasts. The outline of this paper will proceed as follows. This section will be concluded with a short presentation of previous research. Section 2 describes the method and data. The approach and selection of data are presented and discussed. Section 3 provides an elementary description of the theoretical framework applied in this paper. Section 4 contains a presentation and analysis of the estimations and forecasts. In section 5 the conclusions are presented, followed by the references and finally the appendixes.

1.1 Previous research

A possible improvement in the long-run forecast accuracy by imposing co-integrating constraints is examined by Lin and Tsay (1996). In their paper, they produced multistepahead post-sample forecasts from both simulated and real data to examine this question. They found that for simulated data, imposing co-integration correctly does improve forecast ability. However, for the real dataset the result was ambiguous. By imposing co-integrating constraints, the forecast accuracy improved in some cases but deteriorated in others. Engle and Yoo (1987) ran a simulation study, considering a bivariate system, and found that the forecast accuracy for a model that incorporates co-integrating relations was improved in long-run forecasting but not in short-run. This is in line with Clements and Hendry (1995), who also had findings that imposing long-run constraints yield a more accurate forecast, especially for small estimation sample sizes. However, their overall result indicated little gain from imposing co-integrated restrictions. They also had findings, in empirical practice, which showed that imposing too few co-integrating vectors, instead of allowing non-stationary level terms, may have a worsened effect on forecast accuracy. Hoffman and Rasche (1996) examined the forecast performance of a co-integrated system (VEC model) relative to the forecast performance of a comparable VAR model. Their findings suggested that advantages of co-integration appear at longer forecast horizons. The VAR model performed best for the first four years and the VEC model for four to eight years. Their study also indicates that relative gains in forecasts depend upon the chosen data transformation. In a later paper Anderson et al. (2002) used a VEC model to forecast the US economy. The result showed that imposing a long-run equilibrium relationship constraint may pay dividends. They conclude that VEC models offer just the right balance as an econometric model for economic forecasting. Wallis and Whitley (1991) analyzed published forecasts based on four macroeconomic models and examined the fundamentals of IC by generating two variants of forecasts. Firstly they produced pure model-based forecasts, secondly they used a mechanical adjustment rule to determine the residuals according to previous deviations. When they later compared their two variants, they concluded that improved forecast accuracy was more persuasive from the forecast method that used mechanical adjustments. 2

A simulation study by Clements and Hendry (1996) showed that VAR models in differences may be more robust to structural breaks then models in levels. Further they advocate the use of IC with VEC models, they argue that it has less merit in differenced VAR models. In empirical illustration, based on modelling and forecasting wages, prices and unemployment, they find a significant reduction in forecast bias when incorporating IC on a VEC model compared to on a differenced VAR model.

2 Methodology and Data

2.1 Data
As our aim is to replicate the VAR model presented by SMF, we have used the same data and performed equivalent transformations. The data was collected from the database Reuters Ecowin, except the unemployment rate that was taken from Statistics Sweden (2012b). Further, we have included a dummy variable, similar to the SMF, for the purpose to control for the financial crisis in Sweden during the 1990s. A complete presentation of the variables is presented in Table 1. The data ranges from 1989:4 up and until 2012:2, which gives us a total of 91 observations. To achieve results with as high degree of comparison as possible we chose the same period for our in-sample as the Ministry of Finance. Hence, the in-sample ranges from 1989:4 to 2005:4 for a total of 65 observations. The remaining part, called a holdout sample, will be used to produce out-of-sample forecasts. That gives us 26 observations over the time period 2006:1 to 2012:2 in the holdout sample. In Figures 1 and 2 the complete sample period for GDP is presented, for level and differenced data. The out-ofsample period is highlighted in red and the effect of the contemporary financial crisis is evident. This structural break may yield large variation in forecast accuracy. However, it will be of interest to see how the different models and forecast techniques will cope with this break.

Table 1 Variable description Variable label LnGDPsa* LnKIX Description Logarithmic seasonally adjusted quarterly data of Swedens real GDP. Logarithmic quarterly data of Swedens competitor weighted effective exchange rate index. LnCPIX LnTCW** Logarithmic quarterly data of Swedens underlying inflation index. Logarithmic quarterly data weighted between the US GDP and the euro zones GDP. SSVX UnEMP Dummy Quarterly data of the closing yield for a Swedish 3-months treasury bill. Seasonally adjusted quarterly data Swedens relative unemployment. Dummy variable that takes on value one from 1991:4 to and including 1992:3.
Average has been taken for each 3 month period for the variables LnKIX, SSVX and UnEMP to obtain quarterly data. All variables are in first difference. LnTCW is exogenous while the other variables are endogenous. *LnGDPsa will be named by GDP in the body text, for readability. **LnTCW is a weighted average of GDP in the US(0,25) and the euro zone(0,75).

Figure 1: LnGDPsa in level

Figure 2: LnGDPsa in first difference

The blue line represents the in-sample period and the red line represents the out-of-sample period.

The blue line represents the in-sample period and the red line represents the out-of-sample period.

2.2 Methodology
The foundation of this thesis is based upon the VAR model 1 presented by the SMF in a recent publication. We aim to replicate one of their models by including the same variables during the corresponding time period to their study. We propose two modifications to the unrestricted VAR model, which according to theoretical and empirical research could enhance the forecast accuracy. A co-integration test for the endogenous variables will be performed to establish the existence of a long-run relationship. If such a relationship exists, we will proceed by estimating a VEC model along with the VAR model, to examine a possible improvement in forecast accuracy. Additionally, we will produce forecasts for both models with IC2 for further comparisons. A common approach when evaluating complex models is to compare it against more basic models. The comparison will assure the use of a more complex model is justified based on improved forecasts. In our study we will make use of an ARIMA and a Random Walk (RW) model for benchmarking. This implies that we will estimate a VAR, VEC and an ARIMA model, and generate forecasts for six separate models; VAR, VAR IC, VEC, VEC IC, RW and an ARIMA. Evaluation between the models will be done by producing recursive forecasts over the time period corresponding to our holdout sample. Initially the models will be estimated for the time period 1989:4 to 2005:4, thereafter forecasts will be generated for 1, 2,3, 4 and 5 quarters ahead. In the next step the models will be re-estimated where 2006:1 will be included in the estimation sample. Forecasts will then again be generated for 1 to 5 quarters. This procedure will be repeated for the entire holdout sample which implies that each model will produce 26 one-quarter forecasts, 25 two-quarter forecasts and so on. These forecast series will be the foundation for the evaluation, regarding the models relative forecast accuracy compared to one another. Different evaluation measures can yield conflicting results when applied to identical data. From that reasoning and the different properties possessed by evaluation measures, we have chosen four that will provide variant information regarding the differences in forecasts made by the miscellaneous models. The forecasts measures are; Mean Absolute Percentage Error
1 2

The Makro-mod model. IC will only be applied to the lnGDPsa equation.

(MAPE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and Bias. To further examine the relative performance between the models, we will apply a small sample variant of the Diebold-Mariano test3. We will compare the forecast accuracy between the VAR model relative to the VAR IC, VEC and VEC IC models to test for any significant improvements in forecast ability.

The Diebold-Mariano test will be performed with a one sided null hypothesis. That will be rejected when the VAR model has significantly larger forecast errors, relative to the compared model.

3 Theoretical framework
3.1 Vector Autoregressive (VAR) models
In this section we will present the fundamentals and discuss the pros and cons of the standard VAR model. Our presentation is elementary and heuristic. For a more thorough discussion the reader is advised to consult the references4. The VAR model was introduced by Sims (1980) as a model which disregards the theoretical restrictions of simultaneous, or structural, equation models. The model is formed by using characteristics of our data; therefore there are no restrictions that are based on economic theory. However, economic theory still has an importance for VAR modelling when it comes to the selection of variables. According to Sims there should not be any distinction between endogenous and exogenous variables when there is true simultaneity among a set of variables. The VAR model can be seen as a generalization of the univariate autoregressive model and is used to capture the linear interdependencies in multiple time series. Its purpose is to describe the evolution of a set of k endogenous variables based on their own lags and the lags of the other variables in the model. Following Enders (2004), consider a simple bivariate first order standard VAR model (1)

Equation (1) is known as the standard form of the VAR model. Where it is assumed that and are white noise disturbances with standard deviations and , respectively. Notice that it is possible to use OLS separately on each equation since there are no contemporaneous terms in the equations and white noise disturbances. Equation (1) could be rewritten in matrix form as (2) or more compactly as (3) where vector denotes a vector of constants and a matrix of autoregressive coefficients. The is a vector generalization of white noise.

Regarding the assumptions of the VAR model, there are not many that need to be considered. This is because the VAR model lets the data determine the model and uses no or little theoretical information about the relationships between the variables. Except for the assumption of white noise disturbance terms, it is beneficial to assume that all the variables in the VAR model are stationary, to avoid spurious relationships and other undesirable effects.

See Enders (2004).

3.2 Vector Error Correction (VEC) models

A VEC model is a restricted VAR model. The VEC specification restricts the long run behavior of the endogenous variables to converge to their long run equilibrium relationships, while allowing for short run dynamics. This is done by including an Error Correction Mechanism (ECM) in the model, which has proven to be very useful when it comes to modelling non stationary time series. Before explaining VEC models it is necessary to give a fundamental description of co-integration and the ECM. A process that is stationary after differencing d times is said to be integrated of order d, I(d). When a linear combination of two or more such processes are I(d-b), where b>0, the processes are said to be co-integrated. Hence, the processes have an equilibrium relationship to which they will return to in the long-run. However, in the short-run there can occur deviations from the equilibrium (Engle & Granger 1987). When subtracting one process from the other, the trend in the processes will be eliminated. For that to be true, the processes must share a common stochastic drift (Stock & Watson 2007). The idea with the ECM is that a proportion of the disequilibrium will be corrected from one period to the next (Engle & Granger 1987). When incorporating an ECM to the unrestricted VAR in differences, it converges to a VEC model. The specification of the VEC model restricts the long run behavior of the endogenous variables, so that they converge to their long run equilibrium relationships, while allowing for short run dynamics. Consider the VEC formulation of the VAR representation (2) (4) As can be seen, the VEC model is represented as the VAR model in differences, with an added ECM, given by . Provided that and are co-integrated with as

the co-integrating coefficient, will be stationary. The long-run relationship between and are defined by . and are called the error correction coefficients and measures the proportion of last periods disequilibrium that will be corrected in the next period. A more general explanation would be that they measure the speed of adjustment to equilibrium. (Brooks 2002)

3.3 Intercept Correction (IC)

In this paper a simple form of IC is applied, therefore we only provide an elementary presentation. For a more thorough review of the method the reader is advised to consult the references5. Clements and Hendry (1994, 1996) have strived to establish a theory of economic forecasting that captures three aspects of the real world which purely model based forecasts exclude. Firstly; that the DGP is non-stationary due to unit roots, secondly; the occurrence of structural breaks, and thirdly; that the model differs from the underlying DGP. These features provide a rationale for the theory of IC and emphasize the importance of this method. IC refers to the practice of specifying nonzero values for a models error term, over the forecast period. This is done by adding the most recent residual, or a weighted combination of
5

See Clements and Hendry (1998, 1999).

the preceding periods residuals, before further predictions. Consider the following simplified example of IC for one step-ahead forecasts applied in this paper ) (5)

where is the value of the previous forecast error, which represents the adjustment for the misspecification in the original model, that we assume to continue in to the forecast period. According to Clements and Hendry (1998), IC is generally used with two different approaches. It can be used either to represent the influences of anticipated future events that are not explicitly incorporated in the model; or to represent model misspecification or nonconstancy, of an unknown source which is expected to persist. Regardless of the approach, IC can enhance the forecast performance, albeit the improvement in forecast accuracy may only be achieved at the cost of inflated forecast error variances6.

3.4 Evaluation methods

The conclusion reached when evaluating forecasts can vary for identical data when applying different measures of evaluation. Therefore, it is of interest to select several complementary measures that can expose the differences in the forecasts. We have chosen four measures, presented at the end of this sub section, which will gauge the forecast variation. MAPE presents the forecast error in terms of percentage, hence, it is scale invariant and unit free. Both RMSE and MAE measure the distance of the forecast error with no distinction between positive and negative forecast errors. For RMSE that is due to squaring the forecast error, in contrast to MAE which uses the absolute value. Large deviations from the true value have a larger impact upon RMSE, which implies the greater RMSE relative to MAE, the greater the variation for the forecasts error. That property makes it valuable to include both measures although they have certain similarities. Evaluation criterion for RMSE is the smaller value obtained the better predicting ability of the model. In contrast to the other measures, Bias makes a difference between positive and negative forecast errors. The measure will provide information on whether the models have a bias towards over or underestimating the dependent variable.

In this paper we only examine the forecast bias, for information regarding variances and further properties see Clements and Hendry (1998).

where

represents the number of forecasts,

and

denotes the predicted and true value.

3.4.1 Diebold-Mariano test

The Diebold-Mariano test, named after its instigators, was developed as a method to measure and assess the significance of divergences between models and data. Diebold and Mariano (1995) were of the opinion that evaluation was done by point estimates of forecast accuracy, for most of the available tests, without any attempt to assess their sampling uncertainty. Instead, they suggested a test directly based on predictive performance, which also accommodates a variety of accuracy measures and could be applied to multi-period forecasts as well. Additionally, the test is applicable to data with non-Gaussian, non-zero-mean, serially correlated and contemporaneously uncorrelated error terms. The Diebold-Mariano test has the null hypothesis of equal forecast accuracy between two models, i.e. it tests whether two sets of forecast errors, say and have equal mean value. To assess the forecast error a loss function is determined, mainly squared forecast error and absolute forecast error is used. Allowing for an arbitrary loss function, we will apply squared forecast errors, the null hypothesis can be translated to (6) where g denotes the loss function and H the number of forecasts. Suppose H, j step-ahead forecasts have been generated, Diebold and Mariano suggest that approximate asymptotic variance of (7) where as: is the variance of and is the i-th autocovariance of , which can be estimated has an

(8) Therefore, the corresponding statistic for testing the equal forecast accuracy hypothesis is , which has asymptotic standard normal distribution. For our purpose we will consider a small sample modification of the Diebold-Mariano statistic, proposed by Harvey, Leybourne and Newbold (1998). The modified test statistic, that will follow the t-dsitribution with H-1 degrees of freedom, will be represented as following

(9)

10

As recommended by Diebold and Mariano (1995), we will only include autocovariances up to j-1 for j step-ahead forecasts. We make the assumption that all autocovariances with lag length larger than j-1 are equal to zero, which implies that j step-ahead forecast errors are at most (j1)-dependent. To determine whether the forecast errors are dependent over time and which autocovariances terms that should be included, a Ljung-Box7 test will be performed to examine for significant dependence, at the 5 percent level, between the error terms. To insure that the sum of covariances and variances equals a nonnegative value we will apply Newey-West estimator weights to the autocovarianances, as discussed by Diebold-Mariano. The procedure, proposed by Newey and West (1987), makes sure that the when time between error terms increases, the correlation between error terms decreases. The estimator is represented as (10)

Where i represents the lag length of the autocovariance term and q represents the total number of significant autocovariances terms included.

For information regarding the Ljung-Box test see Ljung and Box (1978).

11

4 Estimation and results

4.1 Estimation
All estimations, tests and forecasts are done in the statistical program package Eviews 7. Syntax for all procedures are presented in Appendix B.

4.1.1 The VAR model

As mentioned earlier, our VAR model is a replication. Therefore we have applied the same differences and lag levels as the SMF. They have solved the stationarity requirement by taking the first difference of all included variables. We have performed Augmented Dicky-Fuller unit-root tests8, the results are presented in Table 2. The test result shows that all variables in first difference, except UnEMP, can reject the null of a unit-root on the five percent level. However, UnEMP has a p-value just slightly larger than the significance level, which we find acceptable. Further, the test indicates that LnCPIX and UnEMP are stationary in levels, which seems unreasonable after examining the data in level. Therefore we find differencing appropriate for all variables. The lag lengths are set from one to four, according to the VAR model presented by the SMF. Model diagnostics was performed by the SMF, which showed no problems with heteroscedasticity and autocorrelation, but the residuals were not normally distributed9.

Table 2 Unit root test results Augmented Dickey-Fuller test Variables LnGDPsa LnKIX LnCPIX LnTCW SSVX UnEMP
The values represent the p-value.

Levels 0.999 0.239 0.001 0.865 0.674 0.070

First differences 0.000 0.000 0.000 0.000 0.000 0.052

4.1.2 The VEC model

We performed a Johansen co-integration test10 on the endogenous variables, to examine if there exists a co-integrating relationship. The result is presented in Table 3. Both the trace and the max-eigen statistic indicate two co-integrating relationships, on the five percent level. It is clear that some of the non-normalized coefficients of the co-integrating equations differ from zero. This result indicates that these endogenous variables are not stationary in level, since a combination of them is. This further strengthens our notion that these endogenous
8 9

For information regarding the Augmented Dickey-Fuller test see Dickey and Fuller (1979). This does not affect the study, since we are only examining the forecast bias. 10 For information regarding Johansen co.integration test see Johansen (1991).

12

variables are non-stationary in level and that differencing should be applied. The small coefficients for UnEMP suggest that it have a small impact in this co-integrating relationship. However, we choose to include all endogenous variables in the co-integrating relationship, since we have no theoretical explanation to exclude UnEMP. Hence, we estimate a VEC model with four lags and two co-integrating equations.

Table 3 Co-integration test results Hypothesized No. Of co-int eq.

* *

Johansen test Trace Crittrace Probtrace

Maxeigen 49,83 36,74 12,47

Critmax Probmax

106,77 56,94 20,20

69,82 47,86 29,80

0,00 0,01 0,41

33,88 27,58 21,13

0,00 0,00 0,50

*Denotes rejection of the hypothesis at the five percent level. Prob denotes MacKinnon-Haug-Michelin (1999) p-values.

Table 4 The co-integrating equations LnKIX

1

SSVX
0

LnGDPsa UnEMP
2,0842 0,0224

LnCPIX
-4,0037

0 1 11,4002 0,0143 13,1201 The equations are normalized by the coefficients of LnKIX and SSVX.

4.1.3 The ARIMA model

We estimated an ARIMA model integrated of order one, since GDP requires first difference for stationarity, see Table 2. Thereafter we considered all ARIMA models from (1,1,1) to (5,1,5) and evaluated which model to estimate by the adjusted R-square, Akaikes information criteria and Schwartzs Bayesian information criteria. The results where unequivocal, an ARIMA(3,1,5) got the best scoring according to all evaluation criteria. No further model diagnostics was performed, due to its role as a simple benchmark model. Hence, we estimate an ARIMA(3.1.5).

13

4.2 Results
In this section the results of the models forecasts are presented, along with an evaluation of their performance. First the results from the one and five step-ahead forecasts are presented, followed by a presentation of all forecast horizons and a more general evaluation regarding the results.

4.2.1 One step-ahead forecast performance

Figure 3 One step-ahead forecast VAR Figure 4 One step-ahead forecast VAR IC

Figure 5 One step-ahead forecast VEC

Figure 6 One step-ahead forecast VEC IC

Table 5 Evaluation measures one step-ahead forecasts Measure RMSE MAE MAPE BIAS VAR 0,01362438 0,01079241 0,03959949 -0,003567 VAR IC 0,012801591 0,010697422 0,039242893 -0,00041619 VEC 0,01347648 0,01041553 0,03820144 0,00056343 VEC IC 0,0117488 0,00944068 0,03462725 0,00080843 RW 0,01406179 0,01066842 0,03913136 0,00094804 ARIMA 0,01197433 0,00979861 0,0359454 0,00256804

Bold values denote the best performing model according to each evaluation measure.

14

By visual examination of the figures over the one step-ahead forecasts, it is hard to see any clear distinctions of the relative forecast performance. However, the IC models seem to handle the large variation in GDP more adeptly. They adjust faster, but overcompensate for small variations. This is expected from the properties of IC, and the more volatile oscillation leads to shorter periods of under and over estimations. The evaluation measures provide a consistent and clear result; RMSE, MAE and MAPE indicate that the VEC IC model were most successful in providing accurate forecasts for this period. IC has improved the forecast performance of the VAR model as well, although not to the same extent. The BIAS shows that the unrestricted VAR models in general have underestimated, while the remaining models have overestimated the true GDP. It is also noteworthy that the simpler ARIMA model has a better forecast performance than both VAR models and the original VEC model, according to all evaluating measures.

4.2.2

Five step-ahead forecast performance

Figure 8 Five step-ahead forecast VAR IC

Figure 7 Five step-ahead forecast VAR

Figure 9 Five step-ahead forecast VEC

Figure 10 Five step-ahead forecast VEC IC

15

Table 6 Evaluation measures five step-ahead forecasts Measures RMSE MAE MAPE BIAS VAR 0,04538599 0,03812888 0,13631272 -0,0144432 VAR IC 0,062684932 0,047886826 0,175751876 -0,001017791 VEC 0,03899388 0,03014077 0,11050791 0,00414448 VEC IC 0,04067357 0,03073266 0,11269019 0,00349675 RW ARIMA

0,05095319 0,06455778 0,03929314 0,04980381 0,14404316 0,1811213 0,00617948 0,01167994

Bold values denote the best performing model according to each evaluation measure.

For this period the figures display visible differences between the models forecasts. The IC models show a relative volatile prognostication, whereas the VAR and VEC model exhibits a slower adjustment pattern. The regular VAR model follows GDP well, but with a large delay, while the VAR IC model shows volatile inaccurate fluctuations. It appears to overcompensate for earlier shifts in GDP, in an unsatisfactory way. As in line with theory, the VEC model seems to have captured the long-run behavior of GDP, but adjusts poorly to the crisis. The VEC IC model has followed GDP relatively well, but with large fluctuations. According to the evaluation measures the VEC model has the best forecast performance; however, the figure displays some major drawbacks with adaption to the structural break. From the Bias measure it can be seen that the VAR models have continued to underestimate, in contrast to the remaining models.

4.2.3 Overall forecast performance

Table 7 RMSE for all forecast periods Period VAR 1 2 3 4 5 0,013624 0,022136 0,031158 0,039955 0,045386 VAR IC VEC 0,012802 0,023396 0,034751 0,049253 0,062685 0,013476 0,020313 0,02775 0,034546 0,038994 VEC IC RW 0,011749 0,01857 0,024132 0,032423 0,040674 0,014062 0,024489 0,034492 0,043718 0,050953 ARIMA 0,011974 0,022742 0,037443 0,051792 0,064558

Bold values denote the best performing model according to each period.

Table 8 MAE for all forecast periods Period VAR 1 2 3 4 5 0,010792 0,017411 0,024775 0,033334 0,038129 VAR IC VEC 0,010697 0,020082 0,028293 0,037276 0,047887 0,010416 0,01599 0,021138 0,026835 0,030141 VEC IC RW 0,009441 0,015187 0,018733 0,024426 0,030733 0,010668 0,017839 0,024564 0,033048 0,039293 ARIMA 0,009799 0,016412 0,02604 0,037456 0,049804

Bold values denote the best performing model according to each period.

16

Table 9 MAPE for all forecast periods Period VAR 1 2 3 4 5 0,039599 0,06389 0,090912 0,122322 0,136313 VAR IC VEC 0,039243 0,073691 0,103827 0,1368 0,175752 0,038201 0,058643 0,077513 0,0984 0,110508 VEC IC RW 0,034627 0,055702 0,068692 0,089569 0,11269 0,039131 0,065428 0,090078 0,12117 0,144043 ARIMA 0,035945 0,059253 0,095514 0,13737 0,181121

Bold values denote the best performing model according to each period.

Table 10 BIAS for all forecast periods Period 1 2 3 4 5 VAR -0,00357 -0,00726 -0,01081 -0,01337 -0,01444 VAR IC VEC 0,00042 -0,00131 -0,00127 -0,00179 -0,00102 0,000563 1,01E-05 0,000872 0,001543 0,004144 VEC IC RW 0,000808 0,000604 0,001966 0,001966 0,003497 0,000948 0,002003 0,003269 0,004467 0,006179 ARIMA 0,002568 0,003951 0,007462 0,009873 0,01168

Bold values denote the best performing model according to each period.

Table 11 Equal accuracy test for all forecasts periods Diebold-Mariano test Period 1 2 3 4 5 VAR IC -0,516 0,406 0,604 0,854 1,081 VEC -0,117 -0,645 -0,766 -0,971 -0,947 VEC IC -0,118 -1,320* -1,614* -1,502* -0,778

The values represent t-statistics. * Denotes rejection of the hypothesis at the ten percent level.

The overall forecast results, when including all forecast horizons, are consistent. The VEC IC model provides the most accurate forecasts for GDP, except for the five quarter period where the VEC model performed best. However, from the figure one can argue how well the VEC model captures the short-run fluctuations. According to theory it is not unexpected that the VEC models perform well for longer forecast horizons relative to the VAR models. It is however noteworthy that the VAR models did not perform better for the shorter forecast periods, as Engle and Yoo (1987) and Clements and Hendry (1995) demonstrated. We can also conclude that IC has not had an improving effect for the VAR model forecast 17

performance, which is in line with the arguments presented by Clement and Hendry (1996). From the Diebold-Mariano test results it is also indicated that IC has a worsened effect on the VAR model. The test further shows that the forecast accuracy of the VEC IC model is significantly better, on the 10 percent level, relative to the VAR model, for the forecast periods two to four. This further strengthens the indication that VEC IC is the best performing model in this study. The VAR models repeatedly underestimated GDP, while the VEC models had a consistent positive BIAS. A further interesting result is how well the more simplistic ARIMA model performed, especially for shorter forecast periods, relative to the VAR models.

18

5 Conclusion
In this paper we have performed an empirical study to examine the usefulness of cointegration and IC in forecasting, relative to the VAR model presented by the Swedish Ministry of Finance. Co-integration, which implies a long-term equilibrium between variables, can theoretically be exploited to improve forecast accuracy. This is confirmed by our results where the VEC model consistently outperformed the VAR models, based on our evaluation methods. According to theory, the VEC model should improve the forecast accuracy for longer time periods. However, our study indicates an improvement in short-term forecasting as well. In this case, there is no trade-off between short-term and long-term forecast performance. Our results also show a consistent improvement when applying IC to the VEC models forecasts. However, IC shows no sign of improving the VAR models forecast accuracy. Despite our unequivocal result, the limitation of this study requires more extensive research before drawing any stronger conclusions. This study is performed over a specific time period characterized by a structural break, therefore it would be of interest to examine if these results hold for a different forecast period. To give these findings additional validity, a study with more data, including longer forecast horizons could be applied. The simplistic form of IC in this paper could also be elaborated on to achieve further improvements in forecast accuracy.

19

References
Articles
Anderson, R.G, Hoffman, D.L & Rasche. R.H. 2002. A vector error-correction forecasting model of the US economy, Journal of Macroeconomics, Vol. 24, 569-598. Clements, M.P. & D.F. Hendry. 1995. Forecasting in Co-Integrated Systems, Journal of Applied Econometrics, Vol. 10, 127-146. Clements, M.P. & D.F. Hendry. 1996. Intercepts Corrections and Structural Change, Journal of Applied Econometrics, Vol. 11, 475-494. Diebold, F.X. & Mariano, R.S. 1995. Comparing Predictive Accuracy, Journal of Business and Economic Statistics, Vol. 13, 253-263. Dickey, D.A & Fuller, W.A. 1979. Distribution of the Estimators for Autoregressive Time Series with a Unit Root, Journal of the American Statistical Association, Vol. 74, 427-431. Engle, R.F. & C.W.J. Granger. 1987. Co-integration and Error Correction: Representation, Estimation, and Testing, Econometrica, Vol. 55, 251-276. Engle, R.F. & S. Yoo. 1987. Forecasting and Testing in Co-integrated Systems, Journal of Econometrics, Vol. 35, 143-159. Granger, C.W.J. 1981. Some Properties of Time Series Data and Their Use in Econometric Model Specification, Journal of Econometrics, Vol. 16, 121-130. Harvey, D.I . Leybourne, S.J & Newbold, P. 1998. Tests for Forecast and Encompassing, Journal of Business and Economic Statistics, Vol. 16, 254-259. Hoffman, D.L. & R.H. Rasche. 1996. Assessing Forecast Performance in a Co-integrated System, Journal of Applied Econometrics, Vol. 11, 495-517. Johansen, S. 1991. Estimation and Hypothesis Testing of Cointegration Vectors in Gaussian Vector Autoregressive Models, Econometrica, Vol. 59, 1551-1580. Ljung, G.M. & Box, G.E.P. 1978. On a Measure of Lac of Fit in Time Series Models, Biometrika, Vol. 65, 297-303. Lin, J.L. & R.S. Tsay. 1996. Co-integration Constraint and Forecasting: An Empirical Examination, Journal of Applied Econometrics, Vol. 11, 519-538. Newey, W.K. & West, K.D. 1987. A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix, Econometrica, Vol. 55, 703-708. Sims, C.A. 1980. Macroeconomics and Reality, Econometrica, Vol. 48, 1-48.

20

Wallis, K.F. & Whitley, J.D. 1991. Sources of error in forecasts and expectations: UK economic models, 1984-1988, Journal of Forecasting, Vol. 10, 231-253.

Literature
Brooks, C. 2002. Introductory Econometrics for Finance, 1st edition. Cambridge: Cambridge University Press. Clements, M.P. & Hendry, D.F. 1998. Forecasting economic time series. Cambridge: Cambridge University Press. Clements, M.P. & Hendry, D.F. 1994. Towards a theory of economic forecasting. In Nonstationary Time-Series Analyses and Cointegration, Oxford: Oxford University Press. Enders, W. 2004. Applied Econometric Time Series. 2nd edition. New York: Wiley. Stock, J.H. & Watson, M.W. 2007. Introduction to econometrics, 2nd edition . Boston, MA: Addison Wesley.

Electronic sources
Bjellerup, M & Shahnazarian, H. 2012. Hur pverkar det finansiella systemet den reala ekonomin?. (23.11.2012.) http://www.regeringen.se/content/1/c6/20/39/51/2d737553.pdf Statistics Sweden, 2012. a. Finding Statistics. (14.11.2012.) http://www.scb.se/Pages/Product____22908.aspx b. Arbetskraftsunderskningar. (14.11.2012.) http://www.scb.se/Pages/ProductTables____23272.aspx

Databases
Reuters EcoWin, (12.11.2012.) Database codes: CPIX: ew:swe11793 GDP: es:q_gdp_km628449188se SSVX: ew:swe14200 KIX: ew:swe19033 TCW: ew:usa01006 & ew:emu01019

21

Appendix A Figures
Figure 11 lnGDPsa in level Figure 12 lnGDPsa in first difference

Figure 13 UnEMP in level

Figure 14 UnEMP in first difference

Figure 15 SSVX in level

Figure 16 SSVX in first difference

22

Figure 17 CPIX in level

Figure 18 CPIX in first difference

Figur 19 LnKIX in level

Figure 20 LnKIX in first difference

Figure 21 LnTCW in level

Figure 22 LnTCW in first difference

23

Figure 23 Two step-ahead forecast VAR

Figure 24 Two step-ahead forecast VAR IC

Figure 25 Two step-ahead forecast VEC

Figure 26 Two step-ahead forecast VEC IC

Figure 27 Three step-ahead forecast VAR

Figure 28 Three step-ahead forecast VAR IC

24

Figure 29 Three step-ahead forecast VEC

Figure 30 Three step-ahead forecast VEC IC

Figure 31 Four step-ahead forecast VAR

Figure 32 Four step-ahead forecast VAR IC

Figure 33 Four step-ahead forecast VEC

Figure 34 Four step-ahead forecast VEC IC

25

Figure 35 Ljung-Box test one-step ahead squared forecast errors Var IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

Figure 36 Ljung-Box test one-step ahead squared forecast errors VEC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

26

Figure 37 Ljung-Box test one-step ahead squared forecast errors VEC IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

Figure 38 Ljung-Box test two-step ahead squared forecast errors VAR IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

27

Figure 39 Ljung-Box test two-step ahead squared forecast errors VEC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

Figure 40 Ljung-Box test two-step ahead squared forecast errors VEC IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

28

Figure 41 Ljung-Box test three-step ahead squared forecast errors VAR IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

Figure 42 Ljung-Box test three-step ahead squared forecast errors VEC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

29

Figure 43 Ljung-Box test three-step ahead squared forecast errors VEC IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

Figure 44 Ljung-Box test four-step ahead squared forecast errors VAR IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

30

Figure 45 Ljung-Box test four-step ahead squared forecast errors VEC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

Figure 46 Ljung-Box test four-step ahead squared forecast errors VEC IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

31

Figure 47 Ljung-Box test five-step ahead squared forecast errors VAR IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

Figure 48 Ljung-Box test five-step ahead squared forecast errors VEC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

32

Figure 49 Ljung-Box test five-step ahead squared forecast errors VEC IC VAR

Q-Stat denotes the Ljung-Box test statistic. Prob denotes the p-value. AC denotes the estimated autocorrelation.

33

Appendix B Eviews syntax

Program for estimation and forecasts for VAR, VAR IC, VEC, VEC IC.
' Open data wfopen FullSample ' Make a new workfile so you do not destroy the old wfcreate(wf=results) q 1989Q4 2012Q2 ' Copy Variables copy FullSample::\* results::\* ' Close orignial data file wfuse FullSample wfclose wfuse results ' Number of forecasts !n = X ' Forecast horizon !f = X for !i = 1 to !n ' Estimation Period smpl @first @last-!n-!f+!i ' Estimate VAR var makro.ls 1 4 d(lncpix) d(unemp) d(lngdp_sa) d(ssvx) d(lnkix) @ dummy d(lntcw(-1)) d(lntcw(-2)) d(lntcw(- 3)) d(lntcw(-4)) makro.makemodel(mod1) ' Retrive residuals makro.makeresids lncpix_residual unemp_residual lngdp_sa_residual ssvx_residual lnkix_residual ' Retrive the last residual for gdp smpl @last-!n-!f+!i @last-!n-!f+!i stom(lngdp_sa_residual,lngdp_sa_addresidual) scalar lngdp_sa_addr=lngdp_sa_addresidual(1,1) ' Forecast period smpl @last-!n-!f+!i+1 @last-!n+!i ' Make add factor for intercept correction series lngdp_sa_a = lngdp_sa_addr ' Forecast mod1.solve(d=d) ' Save forecast at correct time smpl @last-!n+!i @last-!n+!i series lngdp_f_var= lngdp_sa_0 delete lngdp_sa_0

' INTERCEPT CORRECTION VAR ' Forecast period smpl @last-!n-!f+!i+1 @last-!n+!i ' Specify that you want to add factor mod1.addassign(i,c) lngdp_sa ' Forecast mod1.solve(d=d) ' Save forecast at correct time smpl @last-!n+!i @last-!n+!i series lngdp_f_var_ic = lngdp_sa_0 ' Clear things up a bit... delete lngdp_sa_a mod1.addassign(n) lngdp_sa ' Estimation Period smpl @first @last-!n-!f+!i ' Estimate VECM

34

var makroecm.ec(2,c) 1 4 lncpix unemp lngdp_sa ssvx lnkix @ dummy d(lntcw(-1)) d(lntcw(-2)) d(lntcw(-3)) d(lntcw(-4)) makroecm.makemodel(mod2) ' Retrive residuals makroecm.makeresids lncpix_residual unemp_residual lngdp_sa_residual ssvx_residual lnkix_residual ' Retrive the last residual for gdp smpl @last-!n-!f+!i @last-!n-!f+!i stom(lngdp_sa_residual,lngdp_sa_addresidual) scalar lngdp_sa_addr=lngdp_sa_addresidual(1,1) ' Forecast period smpl @last-!n-!f+!i+1 @last-!n+!i ' Make add factor for intercept correction series lngdp_sa_a = lngdp_sa_addr ' Forecast mod2.solve(d=d) ' Save forecast at correct time smpl @last-!n+!i @last-!n+!i series lngdp_f_ecm= lngdp_sa_0 delete lngdp_sa_0

' INTERCEPT CORRECTION VECM ' Forecast period smpl @last-!n-!f+!i+1 @last-!n+!i ' Specify that you want to add factor mod2.addassign(i,c) lngdp_sa ' Forecast mod2.solve(d=d) ' Save forecast at correct time smpl @last-!n+!i @last-!n+!i series lngdp_f_ecm_ic = lngdp_sa_0 ' Clear things up a bit... delete lngdp_sa_a mod2.addassign(n) lngdp_sa next smpl @all group lngdp_forecasts lngdp_sa lngdp_f_var lngdp_f_var_ic lngdp_f_ecm lngdp_f_ecm_ic wfsave results

Program for estimation and forecasts for ARIMA(3,1,5).

' Open data wfopen FullSample ' Make a new workfile so you do not destroy the old wfcreate(wf=results) q 1989Q4 2012Q2 ' Copy Variables copy FullSample::\* results::\* ' Close orignial data file wfuse FullSample wfclose wfuse results ' Number of forecasts !n = X ' Forecast horizon !f = X

35

for !i = 1 to !n ' Estimation Period smpl @first @last-!n-!f+!i ' Estimate VAR equation eql.ls d(lngdp_sa) c AR(1) AR(2) AR(3) MA(1) MA(2) MA(3) MA(4) MA(5) eql.makemodel(mod1) ' Retrive residuals eql.makeresids lngdp_sa_residual ' Retrive the last residual for gdp smpl @last-!n-!f+!i @last-!n-!f+!i stom(lngdp_sa_residual,lngdp_sa_addresidual) scalar lngdp_sa_addr=lngdp_sa_addresidual(1,1) ' Forecast period smpl @last-!n-!f+!i+1 @last-!n+!i ' Make add factor for intercept correction series lngdp_sa_a = lngdp_sa_addr ' Forecast mod1.solve(d=d) ' Save forecast at correct time smpl @last-!n+!i @last-!n+!i series lngdp_f_arima= lngdp_sa_0 delete lngdp_sa_0 next smpl @all wfsave results

Program for estimation and forecasts for Random Walk.

Random Walk ' Open data wfopen FullSample ' Make a new workfile so you do not destroy the old wfcreate(wf=results) q 1989Q4 2012Q2 ' Copy Variables copy FullSample::\* results::\* ' Close orignial data file wfuse FullSample wfclose wfuse results ' Number of forecasts !n = 22 ' Forecast horizon !f = 5 for !i = 1 to !n ' Estimation Period smpl @first @last-!n-!f+!i ' Estimate VAR equation eql.ls lngdp_sa=c(1)+lngdp_sa(-1) eql.makemodel(mod1) ' Retrive residuals eql.makeresids lngdp_sa_residual ' Retrive the last residual for gdp smpl @last-!n-!f+!i @last-!n-!f+!i stom(lngdp_sa_residual,lngdp_sa_addresidual)

36

scalar lngdp_sa_addr=lngdp_sa_addresidual(1,1) ' Forecast period smpl @last-!n-!f+!i+1 @last-!n+!i ' Make add factor for intercept correction series lngdp_sa_a = lngdp_sa_addr ' Forecast mod1.solve(d=d) ' Save forecast at correct time smpl @last-!n+!i @last-!n+!i series lngdp_f_rw= lngdp_sa_0 delete lngdp_sa_0 next smpl @all wfsave results

37