LETTER Earth Planets Space, 61, 621–624, 2009

Statistically predicting Dst without satellite data

A. S. Parnowski

Space Research Institute NASU & NSAU, 40 prosp. Akad. Glushkova, Kyiv-187, 03680 MSP, Ukraine

(Received August 14, 2007; Revised September 16, 2007; Accepted September 23, 2007; Online published May 29, 2009)

In this paper we construct a regression relationship for predicting Dst 1 hour ahead. Our model uses only
previous Dst values. This regression is totally unbiased and does not rely on any physical model, except for the
fact that Dst somehow contains the information on the recurrent geomagnetic storms. This regression has the
prediction efficiency of 0.964, linear correlation with official Dst index of 0.982, and RMS of 4.52 nT. These
characteristics are inferior only to our other model, which uses satellite data and provides the prediction efficiency
of 0.975, linear correlation with official Dst index of 0.986, and RMS of 3.76 nT. This makes it quite suitable for
prediction purposes when satellite data are not available.
Key words: Space weather, statistical model, Dst prediction.

1. Introduction al. (1975) published their pioneering work, many authors

Space weather prediction is one of the main tasks of mod- tried to forecast space weather indices. Papers (Kugblenu
ern space research. The necessity of such activities was et al., 1999; Watanabe et al., 2002; Wing et al., 2005;
well understood for a long time (Marubashi, 1989). Space Pallocchia et al., 2006) featured neural network approach;
weather prediction activities divide into two large cate- papers (Zhou and Wei, 1998; Balikhin et al., 2001; Harrison
gories: prediction of space weather directly in space, and and Drezet, 2001) incorporated adaptive filtering; papers
prediction of space weather manifestations on the Earth. (Rangarajan and Barreto, 1999; Oh and Yi, 2004; Wei et al.,
The first category is mostly important for planning of 2004; Johnson and Wing, 2004) applied statistical methods;
space missions, predicting and evading hardware failures papers (Burton et al., 1975; Valdivia et al., 1996; O’Brien
of spacecraft due to arcing in electronic components, and and McPherron, 2000a, b; Temerin and Li, 2002; Ballatore
assuring astronaut safety with respect to radiation hazard. and Gonzales, 2003; Cid et al., 2005; Siscoe et al., 2005;
These tasks mainly require prediction of energetic particle Temerin and Li, 2006) used empirical models; and papers
fluxes. The second category deals with influence of space (Dryer et al., 1984; Raeder et al., 2001) developed global
weather on power grid operation, radio communications, MHD simulations.
and health of people, especially those with cardiovascular The best results for 1-hour prediction were achieved by
diseases. These tasks mainly require prediction of geomag- Temerin and Li (2002, 2006), who used an empirical model.
netic disturbances. This article will focus on space weather They achieved the prediction efficiency of 0.91, linear cor-
prediction on the Earth. relation of 0.95 and RMS of 6.4 nT. Neural network ap-
There are many quantitative indices of geomagnetic ac- proach provides short-term predictions up to 4 hours in the
tivity. The most widely used of them are storm-time distur- paper (Wing et al., 2005). It experiences significant difficul-
bance Dst and planetary geomagnetic activity index Kp. Dst ties predicting geomagnetic storms with Kp > 5, though.
is more convenient for prediction purposes, because it di- Adaptive filtering seems more successful being able to pro-
rectly equals the disturbance of H -component on the Earth vide 8-hour predictions in the paper (Harrison and Drezet,
measured in gammas (1γ = 10−4 Gs = 1 nT). It is aver- 2001). However, in the papers, which incorporate adaptive
aged over several low- and mid-latitude magnetometer sta- filtering, the volume of the dataset usually does not exceed
tions and is usually associated with the westward ring cur- 6 months of data (4380 points), which is not enough to cor-
rent, which appears during the storm at 4–8RE , although rectly describe long-time variations in geomagnetic activity,
this association was strongly criticized by e.g. Campbell caused, e.g., by 11-year solar cycle. Statistical methods give
(1996). At the same time, Kp is an integral and more arti- interesting results, but were rarely used for prediction, and
ficial characteristic of the overall level of geomagnetic dis- much more often for developing and constraining empirical
turbance. In this paper Dst is used for space weather pre- models (Johnson and Wing, 2004). Empirical models were
diction. the most often used, and provided some of the best 1-hour
Space weather prediction is a challenging and nontrivial predictions. Most of them are improvements of the empir-
activity (Joselyn, 1995; Li et al., 2003). Since Burton et ical relationship proposed in a pioneering paper by Burton
et al. (1975), who analysed the ring current injection and
Copyright c The Society of Geomagnetism and Earth, Planetary and Space Sci-
ences (SGEPSS); The Seismological Society of Japan; The Volcanological Society
decay. However, their model suffered from the lack of solar
of Japan; The Geodetic Society of Japan; The Japanese Society for Planetary Sci- wind data and poor physical understanding of solar wind-
ences; TERRAPUB.
magnetosphere interaction at that time. Global MHD simu-

621
622 A. S. PARNOWSKI: STATISTICALLY PREDICTING DST WITHOUT SATELLITE DATA

Fig. 1. Scatter plots of measured Dst versus Dst 1 hour ago (on the left) and versus predicted Dst (on the right).

lations give the longest prediction times but fail to correctly 1963), Ci are the regression coefficients, and xi are the
describe kinetics in boundary layers and ballooning insta- regressors, which are functions of input quantities and their
bilities, which are believed to be responsible for the sub- combinations. Values of Ci are determined by the least
storm onset. square method with equal statistical weights of all points,
Here we use the same method as in our other article and the statistical significance of the regressors–by Fisher
(Parnowski, 2008), which combines statistical and empir- test (Fisher, 1954; Hudson, 1964).
ical approaches. We payed attention mostly to 1-hour pre- The initial number of regressors was deliberately exces-
diction, though we obtained a 9-hour prediction as well. We sive to let Fisher test select the most statistically significant
predicted Dst 1 hour ahead because the temporal resolution of them. This was done in the following way. After pro-
of the dataset was 1 hour, so we just predicted the next value cessing the data with the least square method, Fisher sig-
in the series. Besides, longer prediction times resulted in nificance parameter F was determined for each regressor.
predicted value being shifted in time. For prediction we All F values were compared to the values 2.7055, 3.84,
use only that information, which is available at the moment 5.02, 6.635, 7.879, 10.83 and 12.1, which correspond to
when prediction is made, i.e. 1 hour prior to the predicted statistical significance of 90, 95, 97.5, 99, 99.5, 99.9 and
value. We will reference to this value as “1 hour ago”. We 99.95% respectively. Then, insignificant regressors were
determine the quality of prediction by 3 values: residual rejected and the routine was repeated until all the regres-
mean square (RMS), prediction efficiency (PE), defined as sors were significant. We chose the minimal significance
[1 − (mean squared residual)/(variance of data)] (Temerin level of 90%. In contrast to empirical models we do not
and Li, 2002), and the linear correlation coefficient (LC) add fitting parameters and all the regressors have obvious
between the prediction and Dst. In the article (Parnowski, physical meaning. The described routine was applied to the
2008) we constructed a regression relationship, which pro- complete 43-year dataset sans rejects. More details on the
vides PE = 0.975, LC = 0.986, and RMS = 3.76 nT. routine can be found in the paper (Parnowski, 2008).
However, this relationship requires satellite data to be First, we determined which previous Dst values are sta-
continuous for the previous 20 hours. Thus, when the data tistically significant. For this purpose, we constructed a re-
contain a gap for some reason, we are unable to predict Dst gression
for the next 20 hours. For this reason, we need an ‘emer-
gency’ regression, which would operate without satellite 
N
Dst( j) = C0 + Ci Dst( j − i),
data. Besides, satellite data are often missing during very i=1
strong geomagnetic storms, which are the most interesting
events. Such a regression will be constructed in this article.where N is the oldest Dst value; we reached the value N =
900. We found that there are statistically significant values
2. Data, Routine and Results as far as 801 hours ago (33 days and 9 hours). The statistical
We used the OMNI 2 database, available at NSSDC significance of this oldest value is over 99.9%. However, it
(http://nssdc.gsfc.nasa.gov/omniweb/). It contains IMF, so- is possible that there are even older statistically significant
lar wind and geomagnetic data, averaged over 1-hour inter- values. A similar situation was reported by Johnson and
vals (49 parameters in total, starting from Jan 1, 1963). The Wing (2004) regarding Kp: “the significance is often quite
complete 43-year Dst time series given therein is continu- large for extended periods of time (10–20 days)”. This
ous and features an eye-visible 27-day and 11-year period- might be related in some way to recurrent geomagnetic
icity, which hints for strong dependence on solar activity. storms, but some additional research is required before final
We seek Dst in a regression form explanation could be given to this phenomenon. This will
 be done in a future article.
Dst( j) = Ci xi , After determining which previous Dst values are statisti-
i
cally significant, we added nonlinear terms. We tried differ-
where j is the current step (number of hours since Jan 1, ent powers of the most significant terms and their products.
 A. S. PARNOWSKI: STATISTICALLY PREDICTING DST WITHOUT SATELLITE DATA 623

Fig. 2. Experimental Dst index data and theoretical predictions from several models. Reprinted from (Cid et al., 2005). “Dst from Kyoto” stands for
the official Dst index from WDC-B.

Fig. 3. Our prediction for the same period of time as on Fig. 2. Satellite data is missing in the left part of the plot.

Thus, we constructed a regression, which consisted only of can be switched to high temporal resolution mode, sensitive
Dst terms and a constant regressor. Its characteristics are: equipment can be turned off, etc.
PE = 0.964, LC = 0.982, RMS = 4.53 nT.
To illustrate predictive capabilities of our model we 4. Summary
present several figures: Fig. 1 shows scatter plots of mea- In this article we obtained the following results:
sured Dst versus Dst 1 hour ago (it is the simplest possible
prediction model) on the left and versus prediction on the 1. We derived a regression, which relies only on previous
right; Fig. 2 shows predictions by Cerrato et al. (2004), Dst values.
Fenrich and Luhmann (1998), O’Brien and McPherron 2. It allows predicting Dst 1 hour ahead with PE = 0.964,
(2000b), and Burton et al. (1975) for Jul 15–19, 2000; Fig. 3 LC = 0.982 and RMS = 4.53 nT. Thus, it is very
shows our prediction for the same period of time. convenient for on-line Dst prediction when satellite
More comparison with other models can be found in the data are not available.
paper (Parnowski, 2008). 3. Previous Dst values are statistically significant up to
801 hours ago and possibly more.
3. Conclusion 4. Nonlinear terms appeared to be very significant.
It appeared possible to predict Dst 1 hour ahead using
only its previous values. This hints for recurrent behaviour Acknowledgments. Author is grateful to the National Space Sci-
of geomagnetic activity. In terms of prediction efficiency ence Data Center for the OMNI 2 database.
and linear correlation with the official Dst index this model
is inferior to the model, which uses satellite solar wind data. References
However, satellite data is often missing during strong geo- Balikhin, M. A., O. M. Boaghe, S. A. Billings, and H. St. C. K. Alleyne,
Terrestrial magnetosphere as a nonlinear resonator, Geophys. Res. Lett.,
magnetic storms and this model can be used to fill the gap
28, 1123–1126, 2001.
in predicted Dst time series. Thus, a combination of a re- Ballatore, P. and W. D. Gonzalez, On the estimates of the ring current
gression model using satellite data with the model obtained injection and decay, Earth Planets Space, 55, 427–435, 2003.
in this article can provide accurate on-line operational Dst Burton, R. K., R. L. McPherron, and C. T. Russell, An empirical relation-
ship between interplanetary conditions and Dst, J. Geophys. Res., 80,
forecast. 4204–4214, 1975.
Of course, larger prediction times are desirable, but 1- Campbell, W. H., Geomagnetic storms, the Dst ring-current myth and
hour prediction is still useful. For example, magnetometers lognormal distributions, J. Atm. Terr. Phys., 58, 1171–1187, 1996.
624 A. S. PARNOWSKI: STATISTICALLY PREDICTING DST WITHOUT SATELLITE DATA

Cerrato, Y., E. Saiz, C. Cid, and M. A. Hidalgo, Geomagnetic storms: their Geophys. Res., 105, 7707, 2000b.
sources and a model to forecast Dst index, in Lecture notes and essays Oh, S. Y. and Y. Yi, Relationships of the solar wind parameters with the
in Astrophysics, pp. 131–142, 2004. magnetic storm magnitude and their association with the interplanetary
Cid, C., E. Saiz, and Y. Cerrato, Physical models to forecast the Dst index: shock, J. Korean Astron. Soc., 37, 151–157, 2004.
comparison of results, Proc. Solar Wind 11—SOHO 16 “Connecting Pallocchia, G., E. Amata, G. Consolini, M. F. Marcucci, and I. Bertello,
Sun and Heliosphere”, Whistler, Canada 12–17 June 2005 (ESA SP- ANN prediction of the Dst index, Mem. S.A.It. Suppl., 9, 120–122, 2006.
592, September 2005), 116–119, 2005. Parnowski, A. S., Dst prediction using the linear regression analysis, Kos-
Dryer, M., S. T. Wu, G. Gislason, S. M. Han, Z. K. Smith, J. F. Wang, D. F. michna Nauka i Technologiya, 2008 (accepted, in Russian).
Smart, and M. A. Shea, Magnethydrodynamic modelling of interplan- Raeder, J. et al., Global simulation of the Geospace Environment Modeling
etary disturbances between the Sun and Earth, Astrophys. Space Sci., substorm challenge event, J. Geophys. Res., 106, 381–396, 2001.
105, 187–208, 1984. Rangarajan, G. K. and L. M. Barreto, Use of Kp index of geomagnetic
Fenrich, R. R. and J. G. Luhmann, Geomagnetic response to magnetic activity in the forecast of solar activity, Earth Planets Space, 51, 363–
clouds of different polarity, Geophys. Res. Lett., 25, 2999, 1998. 372, 1999.
Fisher, R. A., Statistical methods for research workers, Twelefth edition, Siscoe, G., R. L. McPherron, M. W. Liemohn, A. J. Ridley, and G. Lu, Rec-
London, Oliver and Boyd, 1954. onciling prediction algorithms for Dst, J. Geophys. Res., 110, A02215,
Harrison, R. F. and P. M. Drezet, The application of an adaptive non- doi:10.1029/2004JA010465, 2005.
linear systems identification technique to the on-line forecast of Dst Temerin, M. and X. Li, A new model for the prediction of Dst on the
index, Proc. Les Woolliscroft memorial Conf. / Sheffield Space Plasma basis of the solar wind, J. Geophys. Res., 107, 1472, doi:10.1029/
Meeting: Multipoint measurements versus theory, Sheffield, UK, Apr 2001JA007532, 2002.
24–26, 2001 (ESA SP-492), 141–146, 2001. Temerin, M. and X. Li, Dst model for 1995–2002, J. Geophys. Res., 111,
Hudson, D. J., Statistics Lectures on Elementary Statistics and Probability, A04221, doi:10.1029/2005JA011257, 2006.
Geneva, CERN, 1964. Valdivia, J. A., A. S. Sharma, and K. Papadopoulos, Prediction of magnetic
Johnson, J. R. and S. Wing, A cumulant-based analysis of nonlinear mag- storms by nonlinear models, Geophys. Res. Lett., 23, 2899–2902, 1996.
netospheric dynamics, Report PPPL-3919rev, http://www.pppl.gov/ Watanabe, S., E. Sagawa, K. Ohtaka, and H. Shimazu, Prediction of the Dst
pub report/2004/PPPL-3919rev.pdf, 2004. index from solar wind parameters by a neural network method, Earth
Joselyn, J. A., Geomagnetic activity forecasting—the state-of-the-art, Rev. Planets Space, 54, 1263–1275, 2002.
Geophys., 33, 383–401, 1995. Wei, H. L., S. A. Billings, and M. A. Balikhin, Analysis of the geomagnetic
Kugblenu, S., S. Taguchi, and T. Okuzawa, Prediction of the geomagnetic activity of the Dst index and self-affine fractals using wavelet trans-
storm associated Dst index using an artificial neural network algorithm, forms, Nonlinear Process. Geophys., 11, 303–312, 2004.
Earth Planets Space, 51, 307–313, 1999. Wing, S., J. R. Johnson, J. Jen, C.-I. Meng, D. G. Sibeck, K. Bechtold,
Li, X., M. Temerin, D. N. Baker, G. D. Reeves, D. Larson, and S. G. J. Freeman, K. Costello, M. Balikhin, and K. Takahashi, Kp forecast
Kanekal, The predictability of the magnetosphere and space weather, models, J. Geophys. Res., 110, A04203, doi:10.1029/2004JA010500,
EOS, 84, 2003. 2005.
Marubashi, K., The space weather forecast program, Space Sci. Rev., 51, Zhou, X.-Y. and F.-S. Wei, Prediction of recurrent geomagnetic distur-
197–214, 1989. bances by using adaptive filtering, Earth Planets Space, 50, 839–845,
O’Brien, T. P. and R. L. McPherron, Forecasting the ring current index Dst 1998.
in real time, J. Atm. Sol.-Terr. Phys., 62, 1295–1299, 2000a.
O’Brien, T. P. and R. L. McPherron, An empirical phase-space analysis
of ring current dynamics: Solar wind control of injection and decay, J. A. S. Parnowski (e-mail: [email protected])