A Statistical Comparison of Vertical Total Electron Content (TEC)

from Three Ionospheric Models

McArthur Mack Jones Jr.
Academic Affiliation, Fall 2008: Senior, Millersville University
SOARS Summer 2008
Science Research Mentor: Mihail Codrescu and Jennifer Gannon
Writing and Communication Mentor: Marijke Unger
ABSTRACT
[1] Total electron content (TEC) exhibits significant variations in both space and time
depending upon latitude, longitude, solar cycle, UTC, and season; these variations can have
potentially negative effects on communication and navigation systems. Recently, three models
have provided accurate results in reconstructing and/or calculating real-time (or near real-time)
vertical TEC values: the Utah State University Global Assimilation of Ionospheric
Measurements (USU GAIM) Gauss-Markov Kalman Filter Model, the United States Total
Electron Content (US-TEC) Model, and the Coupled Thermosphere Ionosphere Plasmasphere
electrodynamics (CTIPe) Model. This research offers a statistical comparison of the vertical
TEC outputs from the previously mentioned models on both a global and local (over the
continental US) scale during the month of July 2008. We present the average difference and root
mean square difference (RMS difference) for three different model comparisons (e.g. US-TEC
vs. GAIM, US-TEC vs. CTIPe, and GAIM vs. CTIPe). We have documented certain model
biases and the differences measured between corresponding data points among the models
relative to each comparison. Two out of the three comparisons showed that the US-TEC models
bias predicted higher values of vertical TEC relative to the other models, while the third
comparison revealed a small bias in the CTIPe model to forecast greater vertical TEC values
when compared to the GAIM model. By computing the RMS difference, we can better examine
the source of these biases relative to the aforementioned model comparisons. This is the first
step in documenting the biases, errors, and uncertainties associated with these three models.

The Significant Opportunities in Atmospheric Research and Science (SOARS) Program is managed by the University
Corporation for Atmospheric Research (UCAR) with support from participating universities. SOARS is funded by the National
Science Foundation, the National Oceanic and Atmospheric Administration (NOAA) Climate Program Office, the NOAA Oceans
and Human Health Initiative, the Center for Multi-Scale Modeling of Atmospheric Processes at Colorado State University, and
the Cooperative Institute for Research in Environmental Sciences. SOARS is a partner project with Research Experience in Solid
Earth Science for Student (RESESS).

1. Introduction
[2] For the last seventy years scientists have studied the Earths ionosphere quite extensively
through the use of multiple techniques (i.e., radiosounders, Faraday radars, top-side soundings
from satellites, GPS signals, etc). Surprisingly, however, most people are unfamiliar with its
existence, despite the fact that the ionosphere plays an integral role in many of their everyday
activities. Extreme variations within the ionosphere induced by storm enhanced density (SED)
events and responses to magnetic storms can adversely affect navigation and communication
systems on Earth [Schunk et al., 2004, Skone and Coster, 2008, Araujo-Pradere and FullerRowell, 2002, Araujo-Pradere, et al., 2002]. According to the American Meteorological Society
[2000] the ionosphere is the upper region of the atmosphere that contains a significant
concentration of charged particles, which affect the propagation of radio waves. When solar
radiation strikes the atoms and molecules of the upper atmosphere, electrons are dislodged
through the process of ionization. Because of the chemical composition of our atmosphere, the
shorter wavelengths of solar radiation (the extreme ultraviolet and X-Rays) are energetic enough
to
ionize
these
atoms
and
molecules
in
the
Earths
atmosphere
[http://www.swpc.noaa.gov/info/Iono.pdf]. Depending on the energy of the incident photon,
photoelectrons (i.e. - electrons emitted from an atom or molecule by an incident photon) are
produced, with great enough energies to ionize other nearby neutrals or moleculars. Particle
precipitation occurs when energetic particles are injected into the upper atmosphere and collide
with neutrals. This process produces increased and variable levels of ionization [Bother and
Daglis, 2007]. In summary, highly variable production, loss, and conditional transport of
ionization are responsible for the extremely variable ionosphere.
[3] Since the ionosphere is the result of a dynamic equilibrium between variable production,
loss and transport processes, its vertical structure can be exceptionally complex and is typically
divided into regions or layers (D, E, F), with each different layer referring to the ionization at
some height. Additionally, a C layer can be present under specific ionospheric conditions. The
amount of layers, their altitude, and their ionization densities vary significantly in time and space
depending on latitude, longitude, universal time, solar cycle, season, and geomagnetic activity
[Davies et al., 1997]. The D-region, which is only present in the daytime and is produced only
by the most energetic solar radiation impinging on Earths upper atmosphere, is the lowest
region in the ionosphere, extending from 50 90 km [Dabas, 2000]. The next layer is the Eregion, which extends from 90 150 km; also located within the E-region are small-scale layers
of variable ionization referred to as sporadic Es. E layers can scatter and or reflect radio waves
to either improve or degrade radio communication (High Frequency communication) [Davies et
al., 1997]. The F-region is divided into two sub-layers (F1 and F2), and collectively these layers
extend from 150 km to 600 km. The D and F regions of the ionosphere are of utmost importance
to radio communication users because the D layer is responsible for the most radio wave
attenuation (frequency range 3 MHz 30 MHz, wavelength range 10 m 100 m) and the F layer
is used for reflecting incident radio waves [Dabas, 2000]. The top of the ionosphere is believed
to be around 1000 km, although there is no definite boundary between the Earths ionosphere,
magnetosphere, and plasmasphere [http://www.swpc.noaa.gov/info/Iono.pdf]. One particular
parameter associated with ionospheric structure is the total electron content (TEC), which is the
integral value of the electron density along a path. Variable electron density levels can affect the

SOARS 2008, McArthur Jones Jr., 2

communication and navigation industries, which can lead to an array of problems [Goodman and
Aarons, 1990].

Figure 1. The different layers of the ionosphere and their predominant ions are displayed at their
particular heights. The electron density profile is also displayed and it varies significantly with height.
The above profile of electron density is shown for average conditions (i.e. daytime, mid-latitude, medium
solar activity. Figure from Space Environment Topics [http://www.swpc.noaa.gov/info/Iono.pdf].

[4] Difficulties encountered by radio communications, GPS, and Differential GPS (DGPS)
users are attributed to variations in electron density and therefore, TEC throughout the
ionosphere.
Radio communication is dependent upon electron density because the
electromagnetic wave may experience a change in propagation direction due to gradients in
electron density [Dabas, 2000]. According to the AMS Policy Statement on Space Weather
[2008], commercial airlines high-frequency (HF) radio signals are often degraded or lost as a
result of space weather conditions. For example, during a strong radio blackout the electron
density in the D-region increases greatly, which can render HF communications almost or
completely unavailable and emergency communications used in disaster recovery on commercial
flights over the polar regions useless [American Meteorological Society, 2008]. At any given
GPS station, ionospheric TEC can change by tens or even hundreds of TEC units (1 x 1016
electrons m-2), bringing about tens of meters of error in the given GPS position [Araujo-Pradere,
2005]. Even today, the ionosphere remains the largest error source of GPS navigation, which in
turn could interfere with the Department of Defenses precision navigation and strike operations
[Coster and Komjathy, 2008, American Meteorological Society, 2008]. DGPS users in North
America experience detrimental effects during SED events [Skone and Coster, 2008].
Furthermore, the U.S. Department of Defense/Department of Transportation [2001] requires a

SOARS 2008, McArthur Jones Jr., 3

95% accuracy reading for marine horizontal positioning, safety, and navigation in inland
waterways, which could be impossible for the Canadian and U.S. Coast Guard DGPS services to
achieve [Skone and Coster, 2008]. Therefore, a technique to better determine electron densities
and thus ionospheric TECs in real time, could lead to extensive improvements in our navigation
and communication industries [Skone and Coster, 2008].
[5] Modeling the TEC in the ionosphere will better allow us to mitigate the aforementioned
effects that this parameter has on our communication and navigation industries. In an effort to
eventually predict the TEC along with ionospheric weather disturbances, both physics-based
models and coupled models have been produced to combine different spatial domains [Scherliess
et al., 2006]. While theoretical/numerical models are fairly accurate at representing the observed
climatological features in the ionosphere, these models generally are unsuccessful in replicating
ionospheric weather [Scherliess et al., 2006]. Presently, the most promising models for
replicating ionospheric weather conditions are physics-based data-driven models that use a
Kalman filter data assimilation techniques [Scherliess et al., 2006].
[6] The Utah State University Global Assimilation of Ionospheric Measurements (USUGAIM) Gauss-Markov Kalman Filter Model is a global data assimilation model that combines
the physics based Ionospheric Forecast Model (IFM) with a suite of ionospheric observations
(for more details see section 2) [Schunk et al., 2004]. The Utah State University Gauss-Markov
Kalman Filter (GMKF) was developed as a part of the Global Assimilation of Ionospheric
Measurements (GAIM) in an effort sponsored by the Department of Defense (DoD) [Scherliess
et al., 2006]. The GMKF filter is used as a foundation for assimilating a distinct set of real-time
(or near-real time) measurements [Scherliess et al., 2006]. Furthermore, this model is currently
used in operations at the Air Force Weather Agency (AFWA). For this study, global TEC as
well as TEC over North America from the USU GAIM Gauss-Markov Kalman Filter Model will
be of particular interest.
[7] The United States Total Electron Content (US-TEC) is a data-driven model that runs in
real time (or near-real time) at the Space Weather Prediction Center (SWPC). US-TEC utilizes a
sophisticated empirical model as background and a Kalman Filter data assimilation method,
which is driven by a ground-based network of real time GPS stations [Fuller-Rowell et al.,
2006]. Recent validation efforts [Araujo-Pradere, et al., 2007, Minter, et al. 2007] have showed
that the accuracy of US-TEC is around 2.7 TECU for slant TEC, or about 2.00 TECU for vertical
TEC. The US-TEC vertical TEC output over the continental United States (CONUS) will prove
to be paramount for this research.
[8] The coupled thermosphere ionosphere plasmasphere electrodynamics (CTIPe) model is a
physically based, non-linear, fully coupled thermosphere-ionosphere-plasmasphere model
[http://helios.swpc.noaa.gov/ctipe/CTIP.html]. CTIPe includes four distinct components, which
run concurrently and will be discussed with much greater depth in section 2. Also contained at
http://helios.swpc.noaa.gov/ctipe/CTIP.html, is the evolution of the CTIPe model. Global TEC
data from the CTIPe machine are used for comparison in this analysis.
[9] Providing adequate framework for this research are verification and validation studies on
the abovementioned models, which include Scherliess et al., [2006], Decker and McNamara,

SOARS 2008, McArthur Jones Jr., 4

2007, Araujo-Pradere et al., 2006, and Araujo-Pradere et al., [2007], Minter et al., [2006],
Fuller-Rowell et al., [2006], and Fuller-Rowell et al., [1995], etc. In particular, this research
focuses on statistical comparisons of global and CONUS (over the continental US) vertical TEC
results from USU GAIM Gauss-Markov Kalman Filter Model, US-TEC, and CTIPe models. By
calculating and later examining certain biases and root mean squared differences between the
models, we will get a better sense of the discrepancies between models. Section 2 offers a more
detailed description of each model used in this study. Section 3 discusses the computation of the
statistical parameters and how the model output was plotted. Section 4 and 5 documents the
biases and differences associated with each model comparison. Section 6 provides a summary
and poses future research questions.
2. Models
2.1 The Utah State University Global Assimilation of Ionospheric Measurements (USU GAIM)
Gauss-Markov Kalman Filter Model
[10] The Utah State University Gauss-Markov Kalman Filter (GMKF) Model was created as
a part of the GAIM program [Schunk et al., 2004]. The GMKF is based on an ionospheric
numerical model and a Kalman filter data assimilation algorithm [Scherliess et al., 2006].
Currently, the GMKF assimilates a number of different measurements including bottomside
electron density (Ne) profiles from a network of 100 Digisondes, nighttime line-of-sight
ultraviolet (UV) radiances measured by satellites, line-of-sight TEC from as many as seventy
GPS ground sites, TECs through occultations between various low-altitude satellites and
between low- and high-altitude satellites, and in situ Ne from four Defense Meteorological
Satellite Program (DMSP) satellites [Schunk et al., 2004 ,Scherliess et al., 2006]. Contained
within the GMKF are plasma densities derived from the IFM, and it is these outputs that provide
a background density field on which the perturbations measured by the GMKF are superimposed
[Scherliess et al., 2006].
[11] The IFM is a global model of the ionosphere based on a numerical solution of the ion
and electron continuity, momentum, and energy equations [Scherliess et al., 2006]. The IFM
simulates these equations for heights from 90 km to 1400 km, which covers the E and F-regions
of the ionosphere. The IFM accounts for all the important chemical and physical processes
including field-aligned diffusion, electron thermal conduction, thermospheric winds,
protonospheric exchange fluxes, energy-dependent chemical reactions, neutral composition
changes, several ion production sources, cross-field electrodynamic drifts, a host of local heating
and cooling processes, and the offset between the geomagnetic and geographic poles [Scherliess
et al., 2006]. In an effort to simulate real-time operations, the model continuously runs and
automatically produces 3-dimensional global electron density structure in fifteen-minute
intervals as well as other auxiliary ionospheric parameters such as NmF2, hmF2, NmE, hmE, slant
and vertical TEC from 90 to 1400 km [Scherliess et al., 2006]. Displayed below (Figure 2a &
2b) are the IDL produced vertical TEC outputs plots from the USU GAIM GMKF Model.

SOARS 2008, McArthur Jones Jr., 5

a.

b.

Figure 2. (a.) Sample map of the vertical global TEC for 20:45 UTC on 22 July 2008. (b.) Example plot
of vertical TEC over CONUS for 20:45 UTC on 22 July 2008. The contour interval for both plots is given
in TEC units.

2.2 The United States Total Electron Content (US-TEC) Model

[12] US-TEC was the first space weather product at the Space Weather Prediction Center
(SWPC) to utilize a data assimilation scheme, and this product was first launched as a test
operational product in November 2004 [Fuller-Rowell et al., 2006], finally transitioning to a full
operational model in June 2007 [Araujo-Pradere and Husler, 2007, Araujo-Pradere et al., 2007,
http://www.swpc.noaa.gov/ustec/docs/USTEC_Doc.html]. As stated in section 1, US-TEC uses a
Kalman filter to constantly update the ionospheric state every fifteen minutes, and as background
the International Reference Ionosphere (IRI) model [Fuller-Rowell et al., 2006]. The Maritime
and Nationwide Differential GPS (M/NDGPS) real time network of stations operated by the US
Coast Guard (USCG) acts as a principal data stream for the model
[http://www.swpc.noaa.gov/ustec/docs/USTEC_Doc.html].
Furthermore, GPS/Met network
(meteorological application of GPS data) and the IGS (International GNSS Service) network
SOARS 2008, McArthur Jones Jr., 6

provide
a
secondary
source
of
data
for
US-TEC
[http://www.swpc.noaa.gov/ustec/docs/USTEC_Doc.html]. Presently, there are about 80 CORS,
30
GPS/MET
and
15
IGS
stations
utilized
in
the
model
[http://www.swpc.noaa.gov/ustec/docs/USTEC_Doc.html]. Figure 3 displays the map of Vertical
TEC over the CONUS up to 1400 km in TEC units for a given 15 minute interval. The US-TEC
product also provides maps of estimated uncertainty, recent trends, and can provide current as
well as past animations. New maps are usually available about thirteen minutes after a given
interval, and are normally updated every fifteen minutes. The ASCII data files from US-TEC are
available at http://www.swpc.noaa.gov/ustec/index.html.

Figure 3. Sample map of the vertical TEC over CONUS for 20:45 UT on 22 July 2008. The symbols
represent the different GPS sites used in the current assimilation cycle. Also the contour interval is given
in TEC units. Data from ninety real-time GPS stations from the CORS, GPS/Met, and IGS networks are
used for these calculations. This figure is from http://www.swpc.noaa.gov/ustec/index.html.

2.3 The Coupled Thermosphere Ionosphere Plasmasphere electrodynamics Model (CTIPe)

[13] The CTIPe Model is the name given to a physically based, global, three-dimensional,
time-dependent, non-linear fully coupled thermosphere-ionosphere-plasmasphere model that
unites
four
separate
components
[http://helios.swpc.noaa.gov/ctipe/CTIP.html,
http://www.issibern.ch/teams/effective-physics/CTIPeModel.pdf]. The first component is a
thermospheric model, which is originally described in Fuller-Rowell and Rees [1980], Rees et al.
[1980], and in the Ph. D. thesis of Fuller-Rowell [1981]. A mid- and high-latitude dynamic
ionospheric convention model described in Quegan et al., [1982] was added as the second
component. These first two components were originally coupled and became known as the
Coupled Thermosphere-Ionosphere Model [Fuller-Rowell et al., 1995]. Additional components
were later added to CTIM; the first of these components was a plasmasphere and low latitude
ionosphere [Millward et al., 1996], which led to the Couple Thermosphere-IonospherePlasmasphere Model. Lastly, the electrodynamics was solved using techniques from neutral
dynamics
and
plasma
components
[http://www.issibern.ch/teams/effectivephysics/CTIPeModel.pdf]. Currently, the CTIPe model is running at the SWPC in real-time
(near real-time) producing plots of global TEC (Figure 4), electron density, NmF2, hmF2, neutral

SOARS 2008, McArthur Jones Jr., 7

temperature, and O/N2 Ratio up to altitudes of 500 to 600 km. All of these aforementioned plots
are displayed at http://helios.swpc.noaa.gov/ctipe, along with some of the model inputs as well.
Figure 4 presents an example of a vertical TEC output map from the CTIPe Model.

Figure 4. A sample plot of global TEC from the CTIPe model on 22 July 2008. The contours are plotted
in TEC units. Figure from http://helios.swpc.noaa.gov/ctipe/TEC.html.

3. Model Outputs and Methodology

[14] The current study focuses on the vertical TEC output from the USU GAIM GaussMarkov Kalman Filter, the USTEC, and CTIPe models. Since these models run in real-time (or
near real-time), output files were readily available every fifteen minutes and for this research we
focused on one or two weeks of daily fifteen minute data files in the month of July in 2008.
During the data collection period we were experiencing a relatively quiescent sun, and therefore
no extreme space weather events were observed. The Kp Index, which quantifies the intensity of
the planetary geomagnetic activity, is displayed in the Appendix (Figure 5) over the entire
period. With Kp Indices less that the threshold value (Kp = 5) for the periods of interest, we can
conclude that the analysis of these outputs was done under quiet conditions. For more
information
on
Kp
Indices
see
Bother
and
Daglis
[2007]
and
http://www.swpc.noaa.gov/info/Kindex.html.
[15] In general, vertical TEC can be calculated by taking the line integral of the threedimensional electron density (ne) profile as a function of height (h) over the signal path from the
GPS ground station to the satellite [Bother and Daglis, 2007],
(1)

TECv ne h dh,

SOARS 2008, McArthur Jones Jr., 8

whereneistheelectrondensity electrons/cm3 .Although, the above equation is solved in

each model, the USU GAIM Gauss Markov Kalman Filter model, the US-TEC model, and the
CTIPe model reconstruct and or calculate the ionospheric vertical TEC differently. Since the
CTIPe model is purely physics based model it solves the continuity and momentum equations for
the plasma, and from there it represents the ionospheric state. The US-TEC model uses a
background model completely driven by data to represent the state and then takes measurements
of TEC. Subsequently, these measurements are then fit in through a data assimilation technique
to give the best estimate of the ionospheric state. USU GAIM Gauss Markov Kalman Filter
model uses as background a physics based model, which solves the background governing
equations for the ionosphere. Then a set of observations is assimilated into the model and the
state is modified to give the best representation of the current ionospheric state.

3.1 Methodology
[16] Creating global and CONUS vertical TEC maps from the USU GAIM Gauss-Markov
Kalman Filter Model over the period of interest, allowed us to statistically and visually compare
its output with the US-TEC and CTIPe models. The US-TEC and CTIPe plots were provided by
Eduardo Araujo and Mihail Codrescu respectively, while the GAIM maps (both global and
CONUS) were constructed using a modified IDL program, for which Michael Carpenter
provided the base code. All the data collected throughout the study were plotted, however not
every map is displayed. To facilitate the data collection process, three separate UNIX scripts
were written (for the three separate models) to continuously provide real-time (or near real-time)
data from each model, so that new maps could be plotted and current statistical parameters could
be computed. For the statistical comparison of the vertical TEC output data was interpolated,
since the resolution of each model was different. Given that we know the accuracy of the USTEC vertical TEC outputs over CONUS, this model served as the basis for this statistical
comparison. The following statistical comparisons are examined:

USU GAIM Gauss-Markov Kalman Filter Model over CONUS vs. US-TEC Model
CTIPe Model vs. US-TEC Model over CONUS
Globally, the USU GAIM Gauss-Markov Kalman Filter vs. CTIPe Model

The US-TEC vs. GAIM comparison started with data from 2 July 2008 to 15 July 2008. The
US-TEC vs. CTIPe and GAIM vs. CTIPe comparisons began with data from 16 July 2008 to 22
July 2008.
[17] The first statistical parameter computed for the above comparison scenarios was the
average difference. The average vertical TEC was solved for every fifteen minute interpolated
output file and to obtain the average difference we subtracted the average vertical TEC between
SOARS 2008, McArthur Jones Jr., 9

the two output files from the models of interest at the same instance in time for each one of our
above comparison scenarios. These are plotted as a function of time over the period of interest
and give us a sense of the consistent differences between the models. The average difference
provides us with a method of monitoring model biases.
[18] The root square difference presents a sufficient statistical technique for measuring
vertical TEC differences between corresponding output points in each one of our interpolated
fifteen-minute data files. The root mean square difference always returns positive values of
vertical TEC, which will tell us absolutely how different the output points are. The root mean
square difference can be represented in general as:

and

where
and
are two data arrays and the first index represents the model while the second
shows the grid point.
The formula then becomes:
(2)
[19] This is done as function of time for the three comparison scenarios. Small values of
RMS difference occur when the predicted values of vertical TEC from each model are relatively
close. Large RMS difference values arise when the forecasted values of vertical TEC from the
two models are comparatively different.
4. Average Difference Validation
4.1 USU-GAIM Gauss-Markov Kalman Filter Model vs. US-TEC Model
[20] The average difference is shown for the GAIM vs. US-TEC comparison in Figure 5.
Both weeks exhibit a robust diurnal variation; furthermore, this diurnal variation is consistent
everyday and is strikingly similar from week-to-week. During week 1 (Figure 5a) the mean
average difference was 5.00 TEC units (TECU), with the maximum average difference (7.96
TECU) occurring on the 5 July 2008 at 2:00 UTC and the minimum average difference (1.63
TECU) observed on the 6 July 2008 at 12:00 UTC. During week 2 (Figure 5b) we recorded a
mean average difference of 4.65 TECU, with maximum average difference of 8.23 TECU on 9
July 2008 at 1:45 UTC, while on 12 July 2008 at 12:00 UTC the minimum average difference
was measured to be 1.38 TECU. The average difference was computed by subtracting the
average GAIM vertical TEC value from the average US-TEC vertical TEC output, meaning there
is a US-TEC model bias to predict higher vertical TEC values than the GAIM model. The mean
bias between the two weeks is 4.825 TECU; also the relative error between the two models was
81.8%.

SOARS 2008, McArthur Jones Jr., 10

a.

b.

Figure 5. Average difference plots for GAIM vs. US-TEC (a) 2 July 2008 at 0:00 UTC to 8 July 2008 at
23:45 UTC and for (b) 9 July 2008 at 0:00 UTC and ends on 15 July 2008 at 23:45 UTC.

[21] For the first two comparisons we are taking the US-TEC model to be the best
representation of the ionospheric state because we know the errors and uncertainties associated
with this model [Araujo-Pradere and Husler, 2007, Minter, et al. 2007]. The majority of the
US-TEC validation has been done exclusively over the CONUS (where the model has a data
input source); however this validation has not been extended to include the edges of the output
plots. Below we have plotted the GAIM (Figure 6a) and US-TEC (Figure 6b) vertical TEC
output maps at the time when we measured the greatest average difference (8.23 TECU) between
the models during our entire two-week period. As can be seen from Figure 6, the US-TEC
model is displaying values of about 30 TECU in the southwest corner of the plot, while the
GAIM plot shows a maximum in that same region only its maximum vertical TEC was about 20
TECU. The southwest corners of both plots display the largest values, therefore dominating the
average difference comparison. Due to the lack of data input in the US-TEC model below 25
N, we limited our comparison region to the regions covered by data (e.g. - 25 N to 60 N
instead of 12 N to 60 N).

SOARS 2008, McArthur Jones Jr., 11

a.

b.

Figure 6. (a) Vertical TEC from the GAIM model over CONUS for 1:45 UTC on 9 July 2008. The
contour interval for both plots is given in TEC units. (b) Vertical TEC output map from the US-TEC
model over CONUS for 1:45 UT on 9 July 2008.

[22] The limited region average difference for the two weeks is displayed in Figure 7. These
average difference plots display the same robust diurnal variation feature as those for the nonlimited case. However, the y-axis scale on these limited coverage plots only extend to 6 TECU
whereas the non-limited coverage y-axis scale reaches to 8 TECU for the first week and 10
TECU for the second week. During week 1 the mean average difference was 3.21 TECU, with a
maximum average difference of 5.65 TECU, which took place at 2:00 UTC on 8 July 2008, and
a minimum average difference of 0.77 TECU occurring at 12:00 UTC on 6 July 2008. In week 2
the minimum average difference was calculated to be 0.53 TECU, which occurred at 12:00 UTC
on 12 July 2008 and a maximum average difference of 5.68 taking place on 9 July 2008 at 2:00
UTC. The mean average difference for week 2 was calculated to be 2.88 TECU. Consequently,
the bias for the US-TEC model to predict higher average values of vertical TEC when compared
to the GAIM model has shrunk to 3.045 TECU. Additionally, our relative error between the two
was reduced to 57.1%; thus, by limiting our coverage area the models start to converge towards
predicting similar vertical TEC values.

SOARS 2008, McArthur Jones Jr., 12

a.

b.
Figure 7. Average difference plots starting at 25 N for GAIM vs. US-TEC (a) 2 July 2008 at 0:00 UTC
to 8 July 2008 at 23:45 UTC and for (b) 9 July 2008 at 0:00 UTC and ends on 15 July 2008 at 23:45
UTC.

4.2 CTIPe Model vs. US-TEC Model

[23] The CTIPe vs. US-TEC average difference (Figure 8) was computed the same way, as
the previous comparison except the period of interest is different. Once again the diurnal
variation is easily observed with a higher average difference experienced in the
afternoon/evening hours and a lower average difference seen in the morning hours. This diurnal
variation is consistent during the entire week. The mean average difference over this week long
period was 5.32 TECU, with a maximum average difference calculated on 22 July 2008 at 21:15
UTC of 10.54 TECU and a minimum average difference of 1.51 TECU observed on 21 July
2008 at 11:15 UTC. Additionally, very slight increase in the local maximum average difference
throughout the week, with the exception of the 19 July 2008 maximum average difference. The
positive average difference calculations reveals a US-TEC bias to predict higher average values
of vertical TEC in comparison to the CTIPe model. The bias in this comparison is slightly
higher than that of the model bias in the previous comparison. Furthermore, the relative error
between the models was calculated to be 79.9%.

SOARS 2008, McArthur Jones Jr., 13

Figure 8. Average difference plot for CTIPe vs. US-TEC starting on 16 July 2008 at 0:00 UTC and
ending on 22 July 2008 at 23:45 UTC.

[24] Given that CTIPe is a global model we will extend our limited coverage case for this
particular comparison as well. As was previously stated we limit our region to start at 25 N and
extend to 60 N instead of 12 N to 60 N. The mean average difference over our week period
has now dropped to 3.96 TECU showing the US-TEC model still tends to predict higher average
vertical TEC values when compared to the CTIPe model. Also the relative error between the
two models has slightly decreased to 75.2%. A maximum average difference of 7.34 TECU
occurred at 1:00 UTC on 22 July 2008 and a minimum average difference of 1.32 TECU was
calculated on 21 July 2008 at 11:15 UTC. The consistent diurnal variation in the average
difference is still the dominant characteristic. Additionally, the y-axis scale decreased from 12
TECU to 8 TECU when we limited our comparison region.

Figure 9. Average difference plot starting at 25 N for CTIPe vs. US-TEC on 16 July 2008 at 0:00 UTC
to 22 July 2008 at 23:45 UTC.

4.3 -GAIM Gauss-Markov Kalman Filter vs. CTIPe Model

[25] The final average difference comparison between GAIM vs. CTIPe is displayed in
Figure 10. The average difference between the predicted global vertical TEC values between

SOARS 2008, McArthur Jones Jr., 14

these two models is much smaller than that of the previous two comparisons discussed in
Sections 4.1-4.2. Once more diurnal variability is observed, however this diurnal dependence is
more variable for this particular situation. A mean average difference of 1.64 TECU, with a
maximum average difference of 2.61 TECU calculated on 21 July 2008 at 17:30 UTC, and a
minimum average difference of 0.57 TECU of was experienced on 16 July 2008 at 0:30 UTC.
With positive average difference calculations, globally, the average vertical TEC predicted by
the CTIPe model is slightly higher than that of the GAIM model. However, this bias is not as
great as the aforementioned comparisons over the CONUS and the relative error is noticeably
smaller (22.4%). Figure 9 also shows that this scenario is unique in the sense that the smallest
average differences are observed in the afternoon/evening and the largest average differences
occur during the morning hours.

Figure 10. Global average difference plot for GAIM vs. CTIPe from 16 July 2008 at 0:00 UTC to 22
July 2008 at 23:45 UTC.

5. RMS Difference Validation

[26] Subsequent to quantifying the differences between the models, some further research
was required to examine where the greatest differences were being experienced. Figure 11
shows the RMS difference over our two-week period (2 July 2008 through 15 July 2008) for the
GAIM vs. US-TEC comparison. In general there are considerably higher RMS differences
between corresponding data points in the southern half of the plot as compared to the northern
half. The largest RMS differences are seen in the southwest part of the CONUS. The region of
largest RMS Difference corresponds to poor or no data coverage from the US-TEC model. This
indicates perhaps that the IRI model used as a background in the data assimilation scheme is
having difficulty reconstructing the vertical TEC in the regions with minimal or no data
coverage. Additionally, in the northeast portion of the map the models agree and there is a small
RMS difference calculated here over our two-week period.

SOARS 2008, McArthur Jones Jr., 15

Figure 11. RMS difference map for GAIM vs. US-TEC from 2 July 2008 to 15 July 2008. The contours
are plotted in TECU.

[27] Below, Figure 12 is strikingly similar to Figure 11 however; Figure 12 displays the
output map from the CTIPe vs. US-TEC model comparison. Another characteristic that is
analogous to both figures in the structure of the RMS difference. Yet again we observed smaller
RMS difference values located is the northeast portion of the graph and larger values of RMS
difference in the southwest region of the map. Also, with the largest RMS difference observed to
be in the southern parts of the CONUS, this reinforces that the maximum differences between the
models occur where there is a lack of real-time vertical TEC data being assimilated into the USTEC model. Moreover, Figure 12 strengthens our above statement that in the northeastern areas
of the CONUS the models are in reasonable accord.

Figure 12. RMS difference plot for CTIPe vs. US-TEC from 16 July 2008 to 22 July 2008. The contours
are plotted in TECU.

[28] Figure 13 shows the global GAIM vs. CTIPe RMS difference. Notice that the scale has
shifted from a maximum of 15 TECU in the preceding maps to 10 TECU in this figure. The
SOARS 2008, McArthur Jones Jr., 16

largest discrepancies between the models are observed in the tropical regions of the globe. The
smallest values of RMS difference are calculated in the mid-latitudes, where the physics of the
ionosphere are better known. Additionally, the ionospheric dynamics exhibited at mid-latitudes
in the ionosphere are not nearly as intense when compared to the equatorial ionosphere. Also,
notice that the polar regions in the northern hemisphere have RMS difference values that are
about a factor of two smaller than the RMS difference at high latitudes in the southern
hemisphere. We believe this is due to the lack of data coverage at high-latitudes in the southern
hemisphere relative to the northern hemisphere in the GAIM model.

Figure 13. Global RMS difference map for GAIM vs. CTIPe from 16 July 2008 to 22 July 2008. The
contours are plotted in TECU.

6. Summary and Conclusions

[29] Vertical TEC, especially in periods of high solar and geomagnetic activity can have
detrimental effects on both navigation and communication systems. Recent studies have shown
that by accurately modeling vertical TEC we can reduce some the difficulties experienced by
radio communications, GPS, and DGPS users. By comparing three different ionospheric models
that are running in real-time (or near real-time) and documenting certain biases and uncertainties
associated with each one of the models, we hope to further alleviate some of these adverse
effects.
[30] Computing the average differences allowed us to observe consistent differences between
the models. The GAIM vs. US-TEC comparison provided evidence that the US-TEC model
generally predicts higher average values of vertical TEC. Once the area was restricted to exclude
the part of the US-TEC model that had poor or no data coverage the GAIM and US-TEC models
were in better agreement. Users of these models should exercise caution when they are looking
at regions where little to no data is being assimilated into the models. The CTIPe vs. US-TEC
comparison further reinforced the above claim showing that when the observed regions were

SOARS 2008, McArthur Jones Jr., 17

limited the models were in better agreement. On a global scale, the GAIM vs. CTIPe are in
reasonable agreement showing a mean average difference of 1.64 TECU over our data collection
period.
[31] By calculating the RMS differences between the models, we were able to determine
geographically where the models predicted vertical TEC values diverged. Both the GAIM vs.
US-TEC and CTIPe vs. US-TEC comparisons showed that the largest differences between the
two models were calculated in the southwest part of the CONUS. Additionally, both these
comparisons showed that the vertical TEC values converged as we approached the northeastern
region of the CONUS. The above RMS difference results provided further evidence that the
forecasted vertical TEC values diverged in the areas where the US-TEC model has poor data
coverage. Globally, the GAIM and CTIPe models are in reasonable agreement, with the
exception of a small area in the tropics and in the high-latitudes of the southern hemisphere.
[32] This research has presented the initial comparison between the GAIM, CTIPe, and USTEC models. Nevertheless, there are additional areas that warrant a greater in-depth analysis so
that we can better quantify the difference between the models and discover the underlying causes
of these discrepancies. Our analysis of reducing the coverage area to show that the agreement
between the models improves can be extended to decrease the coverage region even further to
see if the models continue to converge. Furthermore, determining how these statistics would
change during a period of higher solar activity and therefore higher geomagnetic activity would
provide us with a more holistic synopsis of these model differences. Collecting data over a
longer time scale would prove essential in refining the above findings. In summary, the
culminating goal of this research is to better equip forecasters with a sense of where and when
the different models are proficient at predicting ionospheric weather.
7. Acknowledgements
[33] Thank you to all of the SOARS Staff and Protgs, the RESESS Staff and Protgs,
and each scientist from the National Oceanic and Atmospheric Administration/Earth System
Research Laboratory/Space Weather Prediction Center that contributed to this research. A
special thanks to my scientific mentors Mihail Codrescu and Jennifer Gannon and my writing
and communications mentor Marijke Unger for all of your hard work and devotion in making
this summer a successful one. Additionally, special thanks goes out to Michael Carpenter, Bill
Murtaugh, Rob Craver, Chris Balch, Eduardo Araujo-Pradere, Tim Fuller-Rowell, the Air Force
Weather Agency, Russ Henson, Kelvin Fedrick, Michael Husler, Robert Masten, and Paul
Johnston for their assistance on this project.

References:

SOARS 2008, McArthur Jones Jr., 18

American Meteorological Society (2008), SPACE WEATHER A Policy Statement of the

American Meteorological Society, Bulletin of the American Meteorological Society, 89,
(6), 16-18.
American Meteorological Society (2000), Glossary of Meteorology, 2, 855 pp., American
Meteorological Society, Boston, MA.
Araujo-Pradere, E. A., T. J. Fuller-Rowell, and M. V. Codrescu, STORM: An empirical stormtime ionospheric correction model, 1, Model description, Radio Sci., 37(5), 1070,
doi:10.1029/2001RS002467, 2002.
Araujo-Pradere, E. A., and T. J. Fuller-Rowell, STORM: An empirical storm-time ionospheric
correction model, 2, Validation, Radio Sci., 37(5), 1071, doi:10.1029/2002RS002620,
2002.
Araujo-Pradere, E. A. (2005), GPS-derived total electron content response for the Bastille Day
magnetic storm of 2000 at a low mid-latitude station, Geofisica Internacional, 5, 211218.
Araujo-Pradere, E.A., M. Husler. US-Total Electron Content, Integrated Documentation.
Technical Report, Space Environment Center. National Oceanic and Atmospheric
Administration. USA. 2007.
Araujo-Pradere, E. A., T. J. Fuller-Rowell, P. S. J. Spencer, and C. F. Minter (2007), Differential
validation of the US-TEC model, Radio Sci., 42, RS3016, doi:10.1029/2006RS003459.
Bothmer, V. and I. A. Daglis (2007), Space Weather: Physics and Effects, 213, 315-318, 438 pp.,
Praxis Publishing Ltd., Chichester, UK.
Coster, A. and A. Komjathy (2008), Space Weather and the Global Positioning System, Space
Weather, 6, S06D04, doi: 10.1029/2008SW000400.f.
Dabas, S. R. (2000), Ionosphere and its Influence on Radio Communication, Journal of Science
Education, 5, 28 43.
Davies, N. C., M. J. Maundrell, P. C. Arthur, P. S. Cannon, R. C. Bagwell, J. Cox (1997),
Modern aircraft HF communications into the 21st Century, IEE Colloquium on Air-ToGround Communications (1997/397), 2/1 2/9.
Decker, D. T., and L. F. McNamara (2007), Validation of ionospheric weather predicted by
Global Assimilation of Ionospheric Measurements (GAIM) models, Radio Sci., 42,
RS4017, doi:10.1029/2007RS003632.
Fuller-Rowell, T. J., Three-dimensional, time-dependent model of the thermosphere, Ph.D.
thesis, Univ. College London, London, England, 1981.

SOARS 2008, McArthur Jones Jr., 19

Fuller-Rowell, T.J., and D. Rees, A three-dimensional time dependent global model of the
thermosphere, J. Atmos. Sci., 37, 2545, 1980.
Fuller-Rowell, T., D. Rees, S. Quegan, R.J. Moffett, M.V. Codrescu, and G.H. Millward (1995),
A Coupled thermosphere-ionosphere model (CTIM), STEP Handbook.
Fuller-Rowell, T., E. Araujo-Pradere, C. Minter, M. Codrescu, P. Spencer, D. Robertson, and A.
R. Jacobson (2006),US-TEC: A new data assimilation product from the Space
Environment Center characterizing the ionospheric total electron content using real-time
GPS data, Radio Sci., 41, RS6003, doi:10.1029/2005RS003393.
Goodman, J., M. and J. Aarons (1990), Iosnospheric Effects on Modern Electronic Systems,
Proceedings of the IEEE, 78, 512 528.
Millward, G. H., H. Rishbeth, T. J. Fuller-Rowell, A. D. Aylward, S. Quegan, and R. J. Moffett
(1996), Ionospheric F 2 layer seasonal and semiannual variations, J. Geophys. Res.,
101(A3), 51495156.
Minter, C. F., D. S. Robertson, P. S. J. Spencer, A. R. Jacobson, T. J. Fuller-Rowell, E. A.
Araujo-Pradere, and R. W. Moses (2007), A comparison of Magic and FORTE
ionosphere measurements, Radio Sci., 42, RS3026, doi:10.1029/2006RS003460.
Quegan, S., G. J. Bailey, R. J. Moffett, R. A. Heelis, T. J. Fuller Rowell, D. Rees, and R. W.
Spiro, A theoretical study of the distribution of ionization in the high-latitude ionosphere
and the plasmasphere: first results on the mid-latitude trough and the light ion trough, J.
Atmos. Terr. Phys., 44, 619, 1982.
Rees, D., T. J. Fuller-Rowell, and R. W. Smith, Measurements of mid latitude thermospheric
winds by rocket and ground-based techniques and their interpretation using a threedimensional, time-dependent dynamic model, Planet. Space Sci., 28, 919, 1980.
Scherliess, L., R. W. Schunk, J. J. Sojka, D. C. Thompson, and L. Zhu (2006), Utah State
University Global Assimilation of Ionospheric Measurements Gauss-Markov Kalman
filter model of the ionosphere: Model description and validation, J. Geophys. Res., 111,
A11315, doi: 10.1029/2006JA011712.
Schunk, R. W., et al. (2004), Global Assimilation of Ionospheric Measurements (GAIM), Radio
Sci., 39, RS1S02, doi:10.1029/2002RS002794.
Skone, S., and A. Coster (2008), Potential for issuing ionospheric warnings to Canadian users of
marine DGPS, Space Weather, 6, S04D03, doi:10.1029/2007SW000336.

SOARS 2008, McArthur Jones Jr., 20

8. Appendix

SOARS 2008, McArthur Jones Jr., 21

SOARS 2008, McArthur Jones Jr., 22

Figure 5. 3-day Satellite Environment Plots of estimated planetary Kp indices from 2 July 2008 to 22
July 2008. Plots can be accessed at http://www.swpc.noaa.gov/ftpmenu/plots/satenv.html.

SOARS 2008, McArthur Jones Jr., 23