See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/255593311

Theoretical examination of a multi-model composite for seasonal prediction

Article in Geophysical Research Letters · September 2005

DOI: 10.1029/2005GL023513

CITATIONS READS

48 163

2 authors, including:

Jin Ho Yoo
APEC Climate Center
32 PUBLICATIONS 1,383 CITATIONS

SEE PROFILE

All content following this page was uploaded by Jin Ho Yoo on 15 October 2014.

The user has requested enhancement of the downloaded file.

 GEOPHYSICAL RESEARCH LETTERS, VOL. 32, L18707, doi:10.1029/2005GL023513, 2005

Theoretical examination of a multi-model composite for seasonal

prediction
Jin Ho Yoo and In-Sik Kang
School of Earth and Environmental Sciences, Seoul National University, Seoul, Korea
Received 15 May 2005; revised 19 July 2005; accepted 22 August 2005; published 22 September 2005.
[1] The performance of a multi-model composite for terms of a correlation skill and compare the theoretical
seasonal prediction is theoretically examined in terms of a results to those obtained with AGCM seasonal prediction
correlation skill. On the basis of theoretical analysis, we dataset.
discuss the improvement of skill in the multi-model
composite using the APCN multi-model seasonal 2. Data
prediction dataset. Although the skill of multi-model
composite is generally increased by increasing the number [3] The hindcast seasonal prediction data utilized in this
of models, the highest skill can be obtained by selecting study are obtained from the Asia-Pacific Economic Coop-
several skillful models which are less dependent each eration (APEC) Climate Network (APCN). The APCN is
other. Citation: Yoo, J. H., and I.-S. Kang (2005), Theoretical aimed at producing and disseminating a multi-model en-
examination of a multi-model composite for seasonal semble seasonal prediction based on operational prediction
prediction, Geophys. Res. Lett., 32, L18707, doi:10.1029/ products of APEC member countries. Among them, the
2005GL023513. nine prediction systems utilized are listed in Table 1. All
nine models used observed SSTs as prescribed boundary
condition. The variable analyzed is the seasonal mean
1. Introduction precipitation anomaly for the boreal summer of June –
[2] Dynamical seasonal predictability is known to be August. The prediction data of each model consists of six
limited by atmospheric nonlinear processes and large or more ensemble members for the 21 summers of 1979–
biases in the current models. To partly overcome such 1999, and the ensemble mean is used in this study. The
limitations, multi-model ensemble methods has been verification data is obtained from the Climate Prediction
developed and used with some success [Krishnamurti et Center Merged Analysis of Precipitation (CMAP) data set
al., 2000; Palmer et al., 2004]. In particular, attempts [Xie and Arkin, 1997].
have been made to reduce the uncertainty of models by
simply compositing multi-model solutions [Kang et al., 3. Results
2002] and by using the so-called ‘‘superensemble’’ method [4] The characteristics of individual AGCM forecasts are
[Krishnamurti et al., 2000]. The philosophy behind the analyzed before examining the multi-model composite.
multi-model composite is that the composite can cancel Figures 1a – 1d show total variances of summer mean
the errors of various model solutions. Although several precipitation of four models. The magnitudes of the var-
studies have shown that the multi-model ensemble pre- iances are quite different for different models. For instance,
diction outperforms any of a single seasonal prediction the variance of NCEP model is about ten times larger than
[e.g., Kharin and Zwiers, 2002; Krishnamurti et al., that of JMA model in the Indochina region. This large
2000; Palmer et al., 2004], some other studies indicated difference will lead the multi-model composite to be dom-
that for a regional basis, the multi-model composite is not inated by particular models with the large variances. In the
distinctively better than the single best model [Doblas- present study, to avoid such bias in the multi-model com-
Reyes et al., 2000; Peng et al., 2002]. The skill of a posite, the ensemble mean anomaly of each model and
multi-model composite appears to depend on the ensem- observation are normalized by the corresponding standard
ble size of the individual models [Peng et al., 2002], the deviation at each grid point. The correlation skills of each
number of participating models [Kang et al., 2002], model are relatively similar to each other (Figures 1e –1h).
particular spatial patterns [Pavan and Doblas-Reyes, Each model has a good skill in the tropical central and
2000], and the skills of the combined models [Graham eastern Pacific, where the ENSO signal dominates the
et al., 2000; Mo, 2003]. Thus, the performance of multi- interannual variability and is reasonably well simulated by
model composite depends on the characteristics of the the models. On the other hand, correlation skills over the
models utilized and the superiority of a multi-model western North Pacific and western North Atlantic are poor
composite to a single good model appears to be condi- for all models.
tional. Until now, there have been limited studies, which [5] The correlation skill of the composite of nine
provided theoretical consideration on the multi-model AGCM predictions is shown in Figure 2. The multi-
composite. In the present study, we theoretically analyze model composite shows somewhat improved skill com-
the improvement of skill in the multi-model composite in pared to the single models in general. However, it is
worse than single models in some regions, particularly
Copyright 2005 by the American Geophysical Union. where the correlation skill is negative. It is noticed from
0094-8276/05/2005GL023513 the Figures 1 and 2 that the skill improvement by the

L18707 1 of 4
L18707 YOO AND KANG: MULTI-MODEL COMPOSITE L18707

Table 1. List of the Nine Models Used in the Present Study combined (larger value of R) and the composite variance
Institute Resolution Members (V(y)) was smaller.
CWB (Taiwan) T42L18 10
[7] For given models, the improvement by the multi-
IRI (USA) T42L19 24 model composite may be related to the variance of multi-
JMA (Japan) T63L40 6 model composite, because R is just an arithmetic average of
KMA (Korea) T106L21 10 individual skills. The variance of the multi-model composite
METRI (Korea) 5  4L17 10 is examined further by decomposing the forecast anomaly
MGO (Russia) T42L14 6
NASA (USA) 2.5  2L34 9 (y) into the forecast signal and the independent random
NCEP (USA) T62L28 10 noise as was done by previous studies [Kharin and Zwiers,
SNU/KMA (Korea) T63L21 10 2003; Zheng et al., 2004]. The forecast signal can be further
decomposed into the real signal and the model error:
multi-model composite is robust in the regions where yi ¼ ysi þ yni ¼ xs þ ei þ yni ð2Þ
individual models are relatively skillful.
[6] The advantage of the multi-model composite is ana-
where subscripts s and n indicate the signal and the noise,
lyzed by comparing the correlation skills of the multi-model
respectively, and ei is the model error, which is not
composite and the single model at individual grid points.
independent of real signal. Now V(y) can be derived as:
Let x and yi be the observation and the forecast of ith model,
respectively, of a particular seasonal mean anomaly. The
multi-model composite (y) is a simple average of individual 2 X N
V ðyÞ ¼ V ðxs Þ þ hxs ei i
forecasts of N models. The correlation skill of a multi- N i¼1
model composite (RMM) is derived as: 1 X
N X N   1 X
N
þ 2 ei ej þ 2 V ðyni Þ
N i¼1 j¼1 N i¼1
1 X N
hxyi i ¼ V ðxs Þ þ 2hxs ei þ V ðeÞ þ V ðyn Þ  V ðeÞ  V ðyn Þ ð3Þ
hxyi N i¼1
RMM ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
V ð xÞV ðyÞ V ð xÞV ðyÞ 1 X N X N   1 X N
sffiffiffiffiffiffiffiffiffiffiffi! þ e i e j þ V ðyni Þ
N 2 i¼1 j¼1 N 2 i¼1
1 X N
hxyi i V ðyi Þ
¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi N 1 N 1
N i¼1 V ð xÞV ðyi Þ V ðyÞ ¼ V ð yÞ  V ðyn Þ  V ðeÞ  C ðeÞ
! N N
1 X N
Ri R
¼ pffiffiffiffiffiffiffiffiffiffi ¼ pffiffiffiffiffiffiffiffiffiffi ð1Þ
N i¼1 V ðyÞ V ðyÞ where V ð yÞ is the mean of the variances of individual
models, which is unity here, V ðyn Þ the mean noise variance,
where V(x) is the variance of x. Here, V(x) = V(yi) = 1 V ðeP
Þ thePmean error variance, and C ðeÞ (= 1/(N(N 
because of the normalization. R is the mean correlation 1)) Ni¼1 Nj¼1 heieji(1  dij)) the mean covariance between
skill of the individual models. The bracket indicates time the model errors, where dij is 1 for i = j otherwise 0. The
average. The correlation skill of the multi-model second term of right hand side of the last line in (3)
composite is proportional to the mean correlation skill represents the noise reduction by composite, which was
and inversely proportional to the square root of the discussed by Kumar and Hoerling [2000]. The third term
variance of multi-model composite. Thus, the multi-model represents the error reduction. This term always reduces the
composite provides a better skill, if better models were variance of multi-model composite, because the errors of

Figure 1. (a) – (d) Variances of forecasted summer mean precipitation anomalies for the 21 years of 1979 –1999. Each
model is denoted at the upper left corner of each panel. (e) – (h) Temporal correlation skill of each model ensemble mean
forecast for summer mean precipitation anomaly.

2 of 4
L18707 YOO AND KANG: MULTI-MODEL COMPOSITE L18707

pffiffiffiffiffiffiffiffiffiffi
Figure 4. Distribution of (a) 1/ V ðyÞ, (b) R and (c) RMM
for different combinations and different numbers of models.
The values shown are those averaged over the ENSO-
Figure 2. Temporal correlation skill of a multi-model Monsoon region (60E 90W, 20S 20N). Each open
composite (RMM) using nine AGCM forecasts. dot corresponds to the each combination. The thick closed
dot indicates the mean value for each number of models.

two different models are not perfectly correlated and more effective in the extratropics than in the tropics. As
therefore, the error covariance is smaller than the mean shown in (3), the amplification factor depends on the two
error variance of the two models. It is important to note that factors: the noise and error reductions. The second and third
V(y) is smaller as the model errors are larger but less terms of right hand side of (3) were estimated and their
dependent (larger V ðeÞ and smaller C ðeÞ). Furthermore, differences are shown in Figure 3b. It shows that the
it can be derived that V(y) is same as the mean value variance of multi-model composite is affected more by the
P P between individual model forecasts (r =
of correlations error reduction in the tropics and more by the noise
1/N2 Ni¼1 Nj¼1 hyiyji) and thus, V(y) is small when the reduction in the extratropics. Since the noise reduction
models are less dependent each other. Due to the can be effectively achieved by averaging ensemble forecasts
reductions of noise and errors, the variance of multi- of a single model, the benefit of utilizing multi models
model composite is smaller than the variance of a single would be the error reduction, which is relatively large over
model. As a result, the correlation skill of a multi-model the Asian Monsoon region.
composite is higher than the mean skill of the individual [9] Characteristics of the multi-model composite can be
models. However, if the mean skill is negative, the multi- understood through further examination
model composite becomes worse by producing a bigger pffiffiffiffiffiffiffiffiffiffi of (1). Figure 4
shows the distributions of 1/ V ðyÞ, R, and RMM for
negative correlation value. pffiffiffiffiffiffiffiffiffiffi different combinations and for different numbers of models.
[8] Figure 3a shows the amplification factor 1/ V ðyÞ. It The values shown are those averaged over the ENSO-
is a little bigger than one over the tropical ocean but has a Monsoon region (60E 90W, 20S 20N). This area is
relatively large value over land and extratropics. It indicates chosen because it is a region of relatively high potential
that the variance reduction by the multi-model composite is predictability. The thick closed dot indicates the mean value
of different combinations for each number of models (N).
Since the mean value of R is constant for different N, the
change of mean value of RMM is due to the change of the
mean value of amplification factor. As N increases, the mean
amplification factor increases, because of the reductions of
noise and error in the variance of multi-model composite as
indicated in (3). The variance reductions in (3) are as much as
a half of the mean noise variance and a half of the error term
for N = 2, and 2/3 of them for N = 3, and so on. Therefore, as
in Figure 4a the mean amplification factor approaches to a
saturation value as N increases. The variance of multi-model
composite also depends on the selection of the models,
which determines the spread of the amplification factor for
a given N (Figure 4a). And R also depends on the selection of
models (Figure 4b). As a result, the highest skill can be
obtained with the best combination of a small number of
models, although the mean value of RMM is increased by
increasing the number of models.
[10] Then the problem is how to choose the several
models. The best combination can be obtained by choosing
pffiffiffiffiffiffiffiffiffiR
the best models for the highest ffi and/or the least dependent
V ðyÞffi . Figure 5 shows how RMM
models for the largest 1/pffiffiffiffiffiffiffiffiffi
is related to R and to 1/ V ðyÞ for 126 different combina-
pffiffiffiffiffiffiffiffiffiffi tions of four models chosen among the nine models at a
Figure 3. (a) Amplification factor (1/ V ðyÞ) for the particular grid point (120E, Equator). The p figures
ffiffiffiffiffiffiffiffiffiffi show
composite of nine AGCM forecasts. (b) Difference that RMM is mainly determined by R, and 1/ V ðyÞ is not
between V ðeÞ  C ðeÞ and V ðyn Þ in equation (3). Positive well related to RMM. These relationships are similar in the
value indicates V ðeÞ  C ðeÞ is larger than V ðyn Þ. most of the grid points over the globe. The maximum value

3 of 4
L18707 YOO AND KANG: MULTI-MODEL COMPOSITE L18707

[12] Acknowledgment. The present study is supported by Climate

Environment System Research Center sponsored by Korean Science and
Engineering Foundation and Korea Meteorological Administration.

References
Doblas-Reyes, F., M. Déqué, and J.-P. Piedelievre (2000), Multi-model
spread and probabilistic seasonal forecast in PROVOST, Q. J. R. Meteor-
ol. Soc., 126, 2069 – 2087.
Graham, R. J., A. D. L. Evans, K. R. Mylne, M. S. J. Harrison, and K. B.
Robertson (2000), An assessment of seasonal predictability using atmo-
pffiffiffiffiffiffiffiffiffiffi spheric general circulation models, Q. J. R. Meteorol. Soc., 126, 2211 –
Figure 5. (a) Scatterplots of 1/ V ðyÞ and RMM of 2240.
different combinations at a single grid point (120E, Kang, I.-S., et al. (2002), Intercomparison of GCM simulated anomalies
associated with the 1997 – 98 El Niño, J. Clim., 15, 2791 – 2805.
Equator) for the case of N = 4. (b) Same plots for R and Kharin, V. V., and F. W. Zwiers (2002), Climate prediction with multimodel
RMM. Each dot corresponds to the each combination of four ensembles, J. Clim., 15, 793 – 799.
models chosen among nine models. Kharin, V. V., and F. W. Zwiers (2003), Improved seasonal probability
forecasts, J. Clim., 16, 1684 – 1701.
Krishnamurti, T. N., C. M. Kishitawal, Z. Zhang, T. Larow, D. Bachiochi,
and E. Williford (2000), Multimodel ensemble forecasts for weather and
ofpRffiffiffiffiffiffiffiffiffi
is several
ffi times larger than its minimum value, while seasonal climate, J. Clim., 13, 4196 – 4216.
1/ V ðyÞ has a value between 1.2 and 2.0. Since RMM is Kumar, A., and M. Hoerling (2000), Analysis of a conceptual model of
a product of two quantities, the large variation of R would seasonal climate variability and implications for seasonal prediction, Bull.
Am. Meteorol. Soc., 81, 255 – 264.
be crucial in the variation of RMM. Therefore, the skill of Mo, K. (2003), Ensemble canonical correlation prediction of surface tem-
multi-model composite is generally proportional to the perature over the United States, J. Clim., 16, 1665 – 1683.
mean skill of individual models. Palmer, T. N., et al. (2004), Development of a European multimodel en-
semble system for seasonal-to-interannual prediction (DEMETER), Bull.
[11] The present study suggests that a reasonably good Am. Meteorol. Soc., 85, 853 – 872.
multi-model composite prediction can be obtained by Pavan, V., and F. J. Doblas-Reyes (2000), Multi-model seasonal hindcasts
selecting several skillful and independent models. Simply over the Euro-Atlantic: Skill scores and dynamic features, Clim. Dyn., 16,
increasing the number of models may not be an optimal way 611 – 625.
Peng, P., A. Kumar, H. van den Dool, and A. G. Barnston (2002), An
to achieve a good skill, since adding poor models can analysis of multimodel ensemble predictions for seasonal climate
degrade the multi-model composite prediction by reducing anomalies, J. Geophys. Res., 107(D23), 4710, doi:10.1029/
R. In a realistic situation of seasonal prediction, however, 2002JD002712.
Xie, P., and P. A. Arkin (1997), Global precipitation: A 17-year monthly
the optimal selection is difficult due to the sampling analysis based on gauge observation, satellite estimates and numerical
problem of relatively short hindcast data. Except for several model outputs, Bull. Am. Meteorol. Soc., 78, 2539 – 2558.
regions showing significant skill differences among the Zheng, X., M. Sugi, and C. S. Frederiksen (2004), Interannual variability
and predictability in an ensemble of climate simulations with the
models [Mo, 2003], combining many models is a feasible MRI-JMA AGCM, J. Meteorol. Soc. Jpn., 82, 1 – 18.
approach in the operational situation. It is noted that the
present conclusion may not be sensitive to the choice of the
multi-model systems. In fact, similar results are obtained by 
I.-S. Kang and J. H. Yoo, School of Earth and Environmental Sciences,
applying the present theory to the DEMETER system, Seoul National University, Kwanwak-Ku, Seoul 151-742, Korea. (kang@
which can be referred to Palmer et al. [2004]. climate.snu.ac.kr)

4 of 4

View publication stats