remote sensing

Article
Evaluation of Aqua MODIS Collection 6 AOD
Parameters for Air Quality Research over the
Continental United States
J. H. Belle and Yang Liu *
Department of Environmental Health, Rollins School of Public Health, Emory University, Atlanta,
GA 30322, USA; [email protected]
* Correspondence: [email protected]; Tel.: +1-404-727-8744

Academic Editors: Jun Wang, Omar Torres, Alexander A. Kokhanovsky, Richard Müller and Prasad S. Thenkabail
Received: 18 July 2016; Accepted: 26 September 2016; Published: 1 October 2016

Abstract: Satellite-retrieved aerosol optical depth (AOD) has become an important predictor of
ground-level particulate matter (PM) and greatly empowered air pollution research worldwide.
We evaluated the AOD parameters included in the Collection 6 aerosol product of the Moderate
Resolution Imaging Spectroradiometer (MODIS) for two key factors affecting their applications in air
quality research—coverage and accuracy—over the continental US. For the high confidence retrievals
(QAC 3), the 10 km DB-DT combined AOD has the best coverage nationwide (29.7% of the days
in a year in any given 12 km grid cell). While the Eastern US generally had more successful AOD
retrievals, the highest spatial coverage of AOD parameters were found in California (>55%) and other
vegetated parts of the Western US. If lower QAC retrievals were included, the coverage of the 10 km
DB AOD was dramatically increased to 49.6%. In the Eastern US, the QAC 3 retrievals of all four AOD
parameters are highly correlated with AERONET observations (correlation coefficients between 0.80
and 0.92). In the Western US, positive retrieval errors existed in all MODIS AOD parameters, resulting
in lower correlations with AERONET. AOD retrieval errors showed significant dependence on flight
geometry, land cover type, and weather conditions. To ensure appropriate use of these AOD values,
air quality researchers should carefully balance the needs for coverage and accuracy, and develop
additional data screening criteria based on their study design.

Keywords: MODIS; AOD; remote sensing; United States; retrieval accuracy; satellite coverage

1. Introduction
Aerosol optical depth (AOD) is ‘the single most comprehensive variable to remotely assess
the aerosol burden in the atmosphere’ [1]. It is used to characterize ambient aerosols, either for
land-based remote sensing applications where it is used to remove atmospheric influences, or directly,
to assess atmospheric pollution, primarily fine particulate matter, and its impacts on the climate,
ecosystems, and human populations. Exposure to fine particulate matter (PM2.5 , airborne particles
with an aerodynamic diameter of 2.5 micrometers or less) was identified as a leading risk factor
for global disease burden with an estimated 2.9 million attributable deaths in the year 2013 [2].
Historically, the estimation of population exposure to PM2.5 depends on filter-based ground monitors.
However, because of its high operation and maintenance costs, these ground-based monitoring
networks do not achieve comprehensive spatial coverage. With its comprehensive spatial coverage,
spatial models driven by MODIS AOD are able to estimate the PM2.5 exposure levels in many parts of
the world where ground observations are sparse or nonexistent [3]. The MODerate Resolution Imaging
Spectroradiometer (MODIS) instruments on board the Aqua and Terra satellite platforms have been
providing daily, near-global satellite coverage since 2000 and 2002, respectively [4]. MODIS-retrieved

Remote Sens. 2016, 8, 815; doi:10.3390/rs8100815 www.mdpi.com/journal/remotesensing

Remote Sens. 2016, 8, 815 2 of 14

aerosol optical depth (AOD) has been used extensively in estimating ground-level fine particulate
matter (PM2.5 ) concentrations [5]. Over the past decade, various MODIS-driven PM2.5 exposure
models have been developed, from relatively simple linear regressions [6] to complex multi-level
spatial models [7] and Bayesian hierarchical models [8]. Because PM2.5 is linked to adverse health
outcomes even at the low concentrations commonly observed in the cities of North America [9], PM2.5
models based on MODIS retrievals have been used to extend ground air quality monitoring networks
to cover the suburban and rural populations in the U.S. [10] and Canada [11].
Accuracy and coverage are the most important factors affecting the application of satellite AOD
in air quality research. The retrieval error in AOD has a major influence in the PM2.5 prediction error,
as AOD is often used as the primary predictor in various PM2.5 exposure models. If the AOD retrieval
error varies by season or with land use types, the PM2.5 prediction error will also display spatiotemporal
patterns. This is especially true at the low AOD levels, typically below 0.2, commonly observed in
developed countries [12]. On the one hand, availability of AOD data coverage determines whether
satellite-driven models are feasible for a given study region. On the other hand, it plays an important
role in determining the design of PM2.5 health effect studies [13]. For example, the health effect of
short-term PM2.5 exposure such as asthma exacerbation is often evaluated in a time series model
where temporal missingness of exposure estimates can substantially limit model performance [10].
Cohort studies designed for associating long-term PM2.5 exposure with cardiovascular morbidity and
mortality would benefit from complete spatial coverage [14].
The most recent MODIS collection 6 (C6) aerosol products include enhanced Dark-Target
(10 km DT) and Deep-Blue (10 km DB) AOD present in collection 5 (C5), a ‘merged’ DB-DT parameter
(10 km DB-DT) and a 3 km AOD based off of the 10 km DT retrieval algorithm (3 km DT) [15,16].
The MODIS science team has conducted a few global validation studies to document the collective
impact of these changes and differences between the various parameters [12,16–18]. These studies
mainly focused on estimating the AOD retrieval errors by comparing with collocated measurements
from the Aerosol Robotic Network (AERONET) at the global scale. Because of the large spatial
differences in aerosol loading, global performance metrics such as regression slopes and correlation
coefficients are often driven by regions of high AOD values. To date, only a handful of evaluation
studies were reported in North America, none of which had both accuracy and coverage as their
primary research objectives [19,20]. Therefore, there remains a need for detailed validation studies
in dominantly low-AOD regions to investigate issues related to surface reflectance treatment and
extreme events [12]. In addition, the accuracy and potential usability of lower quality retrievals needs
to be better characterized, and could have important implications on the coverage issue in air quality
applications of MODIS data.
In the current analysis, we focused on characterizing the accuracy and coverage of various MODIS
AOD parameters in the continental US, a dominantly low-AOD area. We focused on examining the
degree to which changes in surface properties and retrieval conditions, such as viewing angle and land
use, affect AOD retrieval error. In addition to the highest quality AOD data, we evaluate the impact
of including lower quality AOD values on the spatial and temporal coverage statistics. Additionally,
we use a case study to demonstrate the practical implications of including lower-quality retrievals
and accounting for major sources of bias on the ability of each AOD parameter to accurately estimate
ground-level PM2.5 concentrations. Finally, we summarize the strengths and weaknesses of these AOD
parameters in the context of air quality research.

2. Materials and Methods

2.1. Satellite and Ground Datasets

We collected Aqua MODIS level 2 AOD data [21] between 1 January 2004 and 31 December 2013 in
the Continental US. Level 2 cloud-screened and quality assured AOD retrievals from 120 permanent
AERONET stations and 73 temporary stations from the Distributed Regional Aerosol Gridded
Remote Sens. 2016, 8, 815 3 of 14

Observation
Remote Sens. 2016,Networks
8, 815 (DRAGON) were collected to validate MODIS retrievals. Out of3 ofthe 14
120 permanent AERONET stations, 48% had been in operation for less than one year (Figure 1).
Total
columncolumn precipitable
precipitable water water estimates
estimates werecollected
were also also collected fromstations
from these these stations to evaluate
to evaluate their
their impact
impact
on MODIS on MODIS AOD retrieval
AOD retrieval error. AERONET
error. Since Since AERONET doesdirectly
does not not directly measure
measure AODAOD at theat 0.55
the 0.55
µm
wavelength reported by MODIS, values were interpolated to this wavelength with a quadratic fit fit
µm wavelength reported by MODIS, values were interpolated to this wavelength with a quadratic in
in log-log
log-log space
space based
based onon valid
valid AODvalues
AOD valuesatata aminimum
minimumofof44ofofany
anyofofthe
the15
15wavelengths
wavelengths potentially
potentially
reported by AERONET [22]. Ancillary datasets were collected for identifying surface properties and
retrieval conditions that could have affected MODIS retrieval accuracy. accuracy. The MODIS 16-day gridded
NDVI parameter at 1 km spatial resolution [23] was used to to calculate
calculate NDVI values at individual
individual
MODIS
MODIS level
level 2 AOD pixels. The National Land Cover Database (NLCD) with a 30 m spatial spatial resolution
was used for for land
land cover
cover type
type calculation
calculation atat individual
individual MODIS
MODIS level level 22 AOD
AOD pixels
pixels [24].
[24]. The 2006
NLCD was used for collocations occurringoccurring prior
prior to 2009 and the 2011 NLCD was used for collocations
occurring after 2009. Information on scattering, viewing, and solar angles for each AOD retrieval was
obtained for each MODIS pixel from from the
the MODIS
MODIS AQUAAQUA level
level 22 Aerosol
Aerosol product
product[21].
[21].

Figure 1. Spatial
Spatial distribution
distribution of AERONET
AERONET (hexagons) and DRAGON (diamonds) sites over the study
December 2013. The color of the symbols represents the number of
period from 1 January 2004 to 31 December
collocations at each site.

2.2. Coverage
2.2. Coverage
Since MODIS
Since MODISpixelspixelsareare created
created relative
relative to each
to each satellite
satellite view andviewtheand the MODIS
MODIS instrument instrument
exhibits
a fish-eye effect, the size and location of individual pixels is not constant from one day to thetonext.
exhibits a fish-eye effect, the size and location of individual pixels is not constant from one day the
next. To compensate for this, a 12 km grid commonly used in the Community
To compensate for this, a 12 km grid commonly used in the Community Multi-scale Air Quality Multi-scale Air Quality
(CMAQ) modeling
(CMAQ) modeling system
system was was created
created for
for our
our coverage
coverage calculation
calculation (a(a total
total of
of 55,031
55,031 cells).
cells). The
The grid
grid
size roughly corresponds to the nadir resolution of the MODIS 10 km
size roughly corresponds to the nadir resolution of the MODIS 10 km AOD parameters and represents AOD parameters and
represents
an important anapplication
important of application
the MODIS ofdata,
the MODIS
where AOD data,observations
where AODare observations
assimilatedare intoassimilated
air quality
into air to
models quality models
improve model to performance
improve model [25].performance
MODIS pixels [25]. MODIS
were pixels were
determined to be determined
within a gridtocellbe
within a grid cell if, for the 10 km DT, 10 km DB, and 10 km DB-DT AOD,
if, for the 10 km DT, 10 km DB, and 10 km DB-DT AOD, the polygon representing the pixel area, the polygon representing
the pixel area,from
reconstructed reconstructed from the using
the pixel centroids pixel centroids
a Voronoiusing a Voronoi
tessellation tessellation
algorithm algorithm
[26], lay [26], lay
at least partially
within the grid cell. Pixels from the 3 km DT parameter were determined to be within a grid cellwithin
at least partially within the grid cell. Pixels from the 3 km DT parameter were determined to be if the
a grid cellofifthe
centroid the3centroid
km pixeloffell thewithin
3 km pixel fell cell,
the grid within the grid
allowing cell,
the allowing
increased the increased
resolution of theresolution
3 km DT
of the 3 kmrelative
parameter DT parameter
to the grid relative to the gridfor
to compensate to the
compensate for the lack
lack of smoothing of smoothing
between cells thatbetween cells
the Voronoi
tessellation would have provided. The percentage of days with a valid, QAC 3 retrieval were calculated3
that the Voronoi tessellation would have provided. The percentage of days with a valid, QAC
retrieval were calculated for each grid cell and parameter. Results were interpreted relative to
averages over the continental U.S. (CONUS) for each parameter. Coverage statistics were also
calculated for QAC 1, 2, and 3 retrievals together to provide an accounting of the gains in coverage
by including lower confidence retrievals in an analysis.
Remote Sens. 2016, 8, 815 4 of 14

for each grid cell and parameter. Results were interpreted relative to averages over the continental U.S.
(CONUS) for each parameter. Coverage statistics were also calculated for QAC 1, 2, and 3 retrievals
together to provide an accounting of the gains in coverage by including lower confidence retrievals in
an analysis.

2.3. Accuracy
AERONET observations were collocated with each MODIS AOD parameter respectively, so that a
temporal average of AERONET observations within ±30 min of the MODIS pass was compared to
the spatial average of MODIS pixels within a ~25 km radius for the 10 km DT, DB and DB-DT AOD,
and a ~7.5 km radius for the 3 km DT AOD [17]. Following previous work, a collocation was only
considered valid if a minimum of three MODIS pixels, two AERONET measurements, and at least 20%
of the total number of MODIS pixels included in the 25/7.5 km radius had valid values with a QAC
code of 3 assigned to the pixel [27]. AERONET stations were categorized as either being in the East or
the West, using the 100◦ W longitude line [16]. The east/west division was necessary because previous
work had found large differences in MODIS performance between the two regions [25]. Retrieval
error, or the difference between MODIS and AERONET AOD at each collocation (τM − τA ) and the
percentage of MODIS observations within the 10 km DT expected error envelope (EEDT )—defined as
±(0.05 + 0.15)τ—[16] were calculated and linear regression models were used to quantify retrieval
errors. In order to evaluate the QAC code assignments as indicators of retrieval errors, independent
collocations were created for QAC 1 and QAC 2 retrievals with AERONET, using the same criteria as
for the QAC 3 collocations. Finally, for each AOD parameter, retrieval error in MODIS AOD relative to
AERONET was examined within quintiles of the surface and retrieval parameters. These parameters
include median NDVI, total column precipitable water from AERONET, land-cover type mode, mean
solar zenith, sensor zenith, and scattering angles. Linear regression models were used to identify any
significant linear trends in retrieval error for each surface and retrieval parameter.

3. Results
During our study period, 193 ground stations reported a total of 286,055 observations that could
be interpolated to AOD at 550 nm. Of these, 262,491 originated from a permanent AERONET station
and 23,564 were recorded during a DRAGON campaign. In the Eastern US, the number of valid
collocations at the 127 stations with high confidence MODIS retrievals ranges from 5616 for 3 km DT
to 6617 for 10 km DB observations. AERONET AOD ranged from 0.0005 to 1.26, with mean values of
0.12, 0.12, 0.12, and 0.10 for collocations with the high confidence 3 km DT, 10 km DT, 10 km DB-DT
and 10 km DB parameters, respectively. MODIS AOD ranged from −0.05 to 2.77, with mean values of
0.13, 0.13, 0.13, and 0.11 for these four AOD parameters, respectively. In the Western US, the number
of valid collocations at 66 AERONET stations with high confidence MODIS retrievals ranges from
6251 for 3 km DT to 11,590 for 10 km DB-DT AOD. AERONET AOD values ranged from 0.0003
to 1.43, with mean values of 0.09, 0.09, 0.08, and 0.08 for collocations with the high confidence 3 km
DT, 10 km DT, 10 km DB-DT, and 10 km DB parameters, respectively. MODIS AOD values ranged
from −0.05 to 2.35, with mean values of 0.09, 0.12, 0.10, and 0.08 for these parameters, respectively.
In both regions, and for all four products, the majority of collocations occurred in the fall and summer,
while the fewest occurred in winter months.

3.1. Coverage of High-Confidence Retrievals

Table 1 shows that on average, a valid AOD retrieval was available on 25%–30% of days in
any given grid cell. However, there is considerable spatial heterogeneity in coverage rates for each
parameter (Figure 1). The highest rates of coverage are found on the western coast, near the large
cities of Los Angeles and San Francisco, and in California’s central valley. In these areas, all four
AOD parameters achieve coverage rates of over 55%, and in the area right around Los Angeles,
coverage rates are above 70%. Similarly high coverage rates, between 50% and 60%, are also observed
Remote Sens. 2016, 8, 815 5 of 14

over the national forests north of Phoenix in Arizona and rates of 40%–50% are observed over the
south-central plains covering the areas of central Texas, Oklahoma, and Kansas. A north-to-south and
elevation gradient in coverage rates can also be observed in Figure 2. The lowest coverage rates were
Remote Sens. 2016, 8, 815 5 of 14
observed over the Great Salt Lake desert, where a few locations had no valid retrieval. Outside of
the Rockies, average coverage rates in the northern parts of the CONUS—an area that includes the
cities of Chicago and New York—were typically only 10%–20%. Coverage rates further south were
large cities of Chicago and New York—were typically only 10%–20%. Coverage rates further south
30%–40%, slightly higher than the CONUS-wide average. This north-south and elevation-based
were 30%–40%, slightly higher than the CONUS-wide average. This north-south and elevation-based
gradient in coverage rates can be linked to seasonal snow-cover occurring primarily at higher
gradient in coverage rates can be linked to seasonal snow-cover occurring primarily at higher latitudes
latitudes and elevations.
and elevations.
Table 1. Coverage
Table 1. Coverage statistics
statistics for
for both
both QAC
QAC 33 retrievals
retrievals only, and for
only, and for all
all AOD
AOD retrievals.
retrievals. Coverage
Coverage is
is
calculated
calculated as
as the
the percentage
percentage ofof days
days with
with aa valid
valid Aqua
Aqua retrieval
retrieval for
for each
each AOD
AOD parameter.
parameter.

AOD Parameter Coverage % (QAC 3 Only) Coverage % (QAC 1, 2, 3)

AOD Parameter Coverage % (QAC 3 Only) Coverage % (QAC 1, 2, 3)
3 km DT 28.2 28.9
10 km 3DT
km DT 28.2
24.3 28.9 32.8
10 km DT 24.3 32.8
10 km 10
DB-DT
km DB-DT
29.7
29.7 31.1
31.1
10 km10DBkm DB 28.9
28.9 49.6 49.6

Figure 2. Mean coverage

coverage statistics
statisticsforforhigh
highconfidence
confidenceAOD
AODretrievals
retrievals
in in
thethe CONUS
CONUS forfor 10 km
10 km DT DT
(a);
(a);
3 km 3 km
DT DT
(b);(b); 10 km
10 km DB-DT
DB-DT (c);(c);
andand1010
kmkm DB(d).
DB (d).Coverage
Coverageisiscalculated
calculatedas asthe
the percentage
percentage of days
between 1 January 2004 and 31 December 2013 with a valid MODIS retrieval from from Aqua.
Aqua.

The
The 33 km
kmDT DTAOD,
AOD,with
witha CONUS-wide
a CONUS-wide coverage
coverage raterate of 28.2%,
of 28.2%, is comparable
is comparable to theto10the
km10AOD
km
AOD parameters in terms of coverage. In contrast to the 10 km products, the 3 km DT
parameters in terms of coverage. In contrast to the 10 km products, the 3 km DT AOD excels over areas AOD excels
over
whereareas where the
the surface surface
is more is more
complex butcomplex
not arid,but
suchnot
asarid, such as
the Pacific the Pacificand
Northwest, Northwest, and over
over the Carolinas.
the Carolinas.
It achieves It achieves
slightly slightlyrates
higher coverage higher coverage
on the eastern rates on the
coast than the eastern coast thanand
10 km parameters, theretrieves
10 km
parameters, and retrieves at higher rates at high to moderate latitudes and elevations
at higher rates at high to moderate latitudes and elevations than the 10 km AOD parameters (Figure than the 10 km
2a).
AOD parameters
The most (Figure 2a).forThe
likely explanation most
these likely
higher explanation
coverage ratesfor these higher
at higher coverage
elevations rates at higher
and latitudes would
elevations and latitudes
be an increased would
ability, on be an
the part of increased
the higherability, on the
resolution part of the
parameter, to higher
retrieveresolution parameter,
aerosols over patchy
to
snow-cover. However, while it has been previously noted that the higher resolution parameterthe
retrieve aerosols over patchy snow-cover. However, while it has been previously noted that is
higher resolution parameter is able to retrieve aerosol information at higher rates over complex
landscapes, coastlines, and between clouds, the extension of this ability to complex snow-cover has
not been investigated [17,19]. The 10 km merged AOD has the highest overall coverage, averaging
29.7% for the CONUS. This parameter aims to maximize the number of high-confidence AOD
retrievals by using AOD values from the 10 km DT algorithm over locations where the NDVI is higher
Remote Sens. 2016, 8, 815 6 of 14

able to retrieve aerosol information at higher rates over complex landscapes, coastlines, and between
clouds, the extension of this ability to complex snow-cover has not been investigated [17,19]. The 10 km
merged AOD has the highest overall coverage, averaging 29.7% for the CONUS. This parameter aims
to maximize the number of high-confidence AOD retrievals by using AOD values from the 10 km DT
algorithm over locations where the NDVI is higher and to use 10 km DB AOD values over locations
where NDVI is lower and the 10 km DT algorithm is less likely to accurately retrieve AOD. The result is
that the spatial patterns of coverage for the merged AOD are similar to 10 km DT AOD, but without the
gaps in coverage over the arid southwest. The coverage of 10 km DB AOD along the east coast is lower
than the other three AOD parameters especially in the summer months (see Supplementary Figure S1)
and over Florida, and balances out the additional coverage gained in the west and south-central plains
(Figure 2). The reasons for this are currently unknown, but slight differences exist between the DT and
DB retrieval processes in the tests used to identify cloud-cover and distinguish it from aerosols and this
could explain summertime differences in coverage over the highly vegetated Eastern CONUS [15,16].

3.2. Coverage Gains with Lower-Quality Retrievals

When lower-quality retrievals were included, coverage rates increased for all AOD parameters.
However, there were large differences in relative increases between products, reflective of differences
in QAC code assignments among AOD parameters. On the one hand, coverage rates for the 3 km DT
AOD increased only slightly from 28.2% to 28.9%, similar to that observed for the 10 km DB-DT AOD.
Coverage rates for the 10 km DT parameter increased substantially from 24.3% to 32.8%. On the
other hand, coverage of the 10 km DB AOD increased dramatically from 28.9% to 49.6% when
lower-confidence retrievals were included. Of the four AOD parameters, the 10 km merged AOD
provides the highest overall coverage over the CONUS if only high-confidence retrievals are considered.
Coverage patterns are similar to those observed for high confidence observations (Supplementary
Figure S2). Previous studies have examined how AOD missingness impacts the representativeness of
the sample within the CONUS, with mixed results [28,29]. For some applications, the typical coverage
rates from high quality retrievals of ~30% may be too low to preserve study power, and investigators
may seek to boost coverage rates through the use of noisier, lower-confidence observations. In this
instance, the 10 km DB AOD offers the greatest potential gains in coverage.

3.3. Accuracy of High-Confidence Retrievals

Error statistics for all four AOD parameters, broken down by region, are presented in Table 2.
In the eastern region, 76% of QAC 3 3 km DT AOD retrievals were within pre-launch expectations
with a mean retrieval error of 0.01, and the correlation coefficient between AERONET and MODIS
values was 0.89. However, the slope of a regression line fit between the two was 1.24, indicating
over-prediction. In the western region, 49% of 3 km DT observations were within the EE and the
slope of a line relating AERONET AOD values to MODIS observations was 1.41. The QAC 3 10 km
DT AOD retrievals in the East had a slightly positive, but near 0 retrieval error, and its correlation
coefficient with AERONET observations (0.92) was the highest of the four parameters. Over-prediction
was a problem in the western region, but was more problematic at low AOD levels. The performance
of the 10 km DB-DT AOD across accuracy metrics was nearly identical to the 10 km DT AOD in
the Eastern US, but it had an additional 341 QAC 3 collocations. In the western region, it was the
most highly correlated with AERONET (0.73). The QAC 3 10 km DB AOD had somewhat uneven
performance across accuracy metrics relative to findings from global validations [12]. It had the highest
percentage of observations within EEDT (87% in the East, 83% in the West). Correlation coefficients,
however, were lower than the other three parameters (0.80 in the East, 0.63 in the West) and global
estimates [12]. Median retrieval error estimates in both regions were low (0.01 in the East, −0.00 in
the West), but intercepts from regression modeling were higher (0.03 in the East, 0.02 in the West) and
slopes were below 1 (0.79 in the East, 0.75 in the West). When assessed together, these metrics indicated
over-prediction at lower AOD values and under-prediction at higher AOD values. Previous global
Remote Sens. 2016, 8, 815 7 of 14

validations have found similar patterns for 10 km DB retrievals, but over-prediction was more severe
in the Western US than was noted in the global studies.

Table 2. Performance statistics for each AOD parameter.

Region QAC Parameter N Error Intercept Slope R % Above EE % Below EE % Inside EE

3 km DT 546 0.02 0.02 1.03 0.86 9.0 0.9 90.1
10 km DT 1487 0.04 0.05 1.46 0.58 43.4 1.6 55.0
1
10 km DB-DT 482 0.01 0.01 1.06 0.85 7.3 1.5 91.3
10 km DB 10603 0.04 0.07 1.01 0.52 42.0 2.0 55.9
10 km DT 2577 0.08 0.08 1.30 0.58 58.1 2.4 39.6
West 10 km DB-DT 1321 0.01 0.03 0.95 0.47 27.0 3.3 69.6
2
10 km DB 2316 0.01 0.04 0.90 0.62 27.8 3.4 68.9
3 km DT 6251 0.06 0.06 1.41 0.64 48.9 2.5 48.6
10 km DT 7814 0.02 0.03 0.98 0.71 25.3 5.6 69.1
3
10 km DB-DT 11590 0.01 0.01 1.01 0.73 17.3 5.1 77.6
10 km DB 10662 −0.00 0.02 0.75 0.63 11.2 5.9 82.9
3 km DT 883 0.02 0.04 0.80 0.79 12.6 4.6 82.8
10 km DT 1322 0.02 0.02 1.19 0.82 26.6 5.6 67.8
1
10 km DB-DT 440 0.00 0.04 0.50 0.71 3.9 8.4 87.7
10 km DB 7019 0.05 0.08 0.84 0.76 36.4 1.0 62.7
10 km DT 1538 0.04 0.02 1.26 0.88 37.1 6.1 56.8
East 10 km DB-DT 16 0.13 0.17 0.42 0.21 62.5 0.0 37.5
2
10 km DB 368 0.05 0.09 0.70 0.73 39.1 1.4 59.5
3 km DT 5616 0.01 −0.01 1.24 0.89 16.0 7.6 76.4
10 km DT 6409 0.00 −0.01 1.18 0.92 10.8 8.4 80.9
3
10 km DB-DT 6750 0.00 −0.01 1.17 0.91 11.7 7.5 80.8
10 km DB 6617 0.01 0.03 0.79 0.80 10.5 2.6 86.9
N: number of collocations. Error: τM − τA The intercept, slope, and correlation coefficient (r) are calculated
using a linear regression model relating MODIS to AERONET AOD values. The error envelope is defined
as ±(0.05 + 0.15)τA .

3.4. Accuracy Assessment of Lower-Confidence Retrievals

The more mature 10 km DB and 10 km DT AOD had 20,306 and 6924 valid lower-confidence
collocations, respectively. In contrast, the less mature 3 km DT and 10 km DB-DT products had
only 1429 and 2259 valid lower-confidence collocations, respectively. As expected, lower confidence
collocations for the 10 km DB and 10 km DT AOD were noisier and had larger retrieval errors when
compared to QAC 3 observations (Table 2). For both parameters, a lower proportion of low-confidence
observations were within EEDT , ranging from 40% to 68% of retrievals, median retrieval error estimates
were higher, ranging from 0.02 to 0.09, and correlation coefficients were lower, ranging from 0.52 to
0.88. There were a substantial number of valid QAC 1 10 km DB collocations with a positive retrieval
error at lower AOD values. This is typically attributable to cloud contamination, and so it may be
possible to use some of these observations in an analysis with caution and additional cloud screening
procedures [25]. Accuracy statistics for lower-confidence 3 km DT and 10 km merged AOD, on the
other hand, were comparable with high confidence retrievals (Table 2). The 3 km DT AOD had a high
percentage of observations within EEDT , 83 and 90% for the eastern and western regions, respectively,
and strong correlations (0.79 in the East, 0.85 in the West). All low-confidence 10 km DB-DT AOD
except QAC 2 retrievals in the eastern region (only 16 valid collocations) met pre-launch expectations,
having between 70% and 90% of retrievals within EEDT . The QAC code assignments for the two new
AOD parameters do not seem to accurately reflect retrieval errors in the same way as for the more
mature AOD parameters.

3.5. Dependence of Retrieval Errors on Flight Geometry and Land Cover Type
Figure 3 illustrates the dependence of AOD errors on the scattering, solar zenith, and viewing
angles for QAC 3 retrievals. Scattering angle was associated with a statistically significant, positive
trend in retrieval error in all four parameters in both regions. This trend is most pronounced for 3 km
DT AOD in the western region with a median retrieval error of 0.12 for the highest quintile, and 0.04
Remote Sens. 2016, 8, 815 8 of 14

for the lowest quintile of scattering angle. The median retrieval errors of 10 km DT, DB, and DB-DT
AOD in both regions, and 3 km DT observations in the eastern region increase slightly with scattering
angle (but remain below 0.04). This type of dependence could be related to issues with accounting
for anisotropy in the surface reflectance over the CONUS [30]. Our findings in the CONUS disagree
with those presented in global evaluations, which found tendencies of median retrieval error with
scattering angle to be small and negative [12]. Our findings on the association between retrieval errors
and solar zenith angles are only partially consistent with Sayer et al. [12] which found solar zenith
Remote20
angles below Sens. 2016, 8, 815to have positive retrieval errors for the 10 km DT parameter8 and
degrees of 14 negative

retrieval errors forangle

scattering the to 10bekm DB
small and parameter,
negative [12].but
Ourretrievals
findings on theat angles
associationgreater
between than 20 to
retrieval be relatively
errors
unbiased. and
In the
solar zenith angles are only partially consistent with Sayer et al. [12] which found solar zenith et al. [12]
Eastern US, our results show a similar pattern to those present in Sayer
angles below 20 degrees
for 10 km DB retrievals relative to have
to 10positive
km DT retrieval errors for
retrievals, butthe 10 km
with DT parameter
a slight positive andretrieval
negative error for
retrieval errors for the 10 km DB parameter, but retrievals at angles greater than 20 to be relatively
10 km DB. Additionally, we observed fairly substantial retrieval errors at solar zenith angles greater
unbiased. In the Eastern US, our results show a similar pattern to those present in Sayer et al. [12] for
than 20 degrees.
10 km DBInretrievals
the WesternrelativeUS,to 10we
km DTobserved negative
retrievals, but with aretrieval errors
slight positive in 10error
retrieval kmforDB 10 observations
km
spanning solar zenith angles
DB. Additionally, from 25
we observed to 43
fairly degrees,
substantial whileerrors
retrieval the first quintile,
at solar containing
zenith angles observations
greater than
with solar20zenith
degrees. In the less
angles Western
thanUS,25we observedwas
degrees, negative retrievalunbiased.
relatively errors in 10 Thiskm DB observations
finding runs contrary
spanning solar zenith angles from 25 to 43 degrees, while the first quintile, containing observations
to previous observations which have suggested that it is primarily low solar zenith angles that are
with solar zenith angles less than 25 degrees, was relatively unbiased. This finding runs contrary to
problematic [12,25].
previous AOD retrieval
observations which have error showsthat
suggested a small negative
it is primarily lowtrend
solar with
zenithsensor zenith
angles that are angle for
the 10 kmproblematic
DB AOD [12,25]. in bothAOD regions
retrieval error shows a small negative trend with sensor zenith angle for trend for
and for the 3 km DT in the West, and a small positive
the 10 km DB
the 10 km DT, 10 km DB-DT, andAOD in both regions
3 kmand DTforAOD
the 3 km DT in
in the the West,
East. The and a small
largest positivewas
change trendfor
for10 km DB
the 10 km DT, 10 km DB-DT, and 3 km DT AOD in the East. The largest change was for 10 km DB
observations in the East, where the median retrieval error estimate in AOD within the first quintile of
observations in the East, where the median retrieval error estimate in AOD within the first quintile
sensor zenith angle,
of sensor zenith near thenear
angle, nadir, was was
the nadir, 0.024 and
0.024 andthe median
the median retrieval
retrieval errorerror in the quintile
in the highest highest quintile
was −0.005.was −0.005.

Figure 3. The dependence of AOD retrieval error and distributions of values for scattering angle (a);
Figure 3. The dependence of AOD retrieval error and distributions of values for scattering angle (a);
solar zenith angle (b); and sensor zenith angle (c). Median error (points) and the IQR (vertical line
solar zenith angle
ranges from (b);
25thand sensor
to 75th zenith
percentile) anglewithin
is shown (c). Median error
quintiles. The (points)ofand
distribution theis IQR
values shown(vertical
in line
ranges from
the25th to 75thinpercentile)
background is shown
gray and represents within quintiles.
proportional The distribution
frequency, where of values
0.25 on the y-axis is shown in
represents
the most
the background infrequent
gray andvalue in the category.
represents proportional frequency, where 0.25 on the y-axis represents the
most frequent value in the category.
We assessed AOD retrieval errors by six land cover types, i.e., developed, forest, shrub, grass,
cultivated, and wetland (see supplementary Figure S3). All AOD parameters showed positive
retrieval errors
We assessed AODover developed
retrieval areas,by
errors particularly
six landincover
the Western USi.e.,
types, (0.03developed,
for 10 km DB, forest,
and 0.21shrub,
for grass,
3 km DT AOD). Small but consistent positive errors were also observed over wetlands in the Eastern
cultivated, and wetland (see Supplementary Figure S3). All AOD parameters showed positive retrieval
US (mean retrieval errors of 0.04 for 3 km DT, 0.03 for 10 km DT, 0.03 for 10 km DB-DT, and 0.02 for
errors over developed areas, particularly in the Western US (0.03 for 10 km DB, and 0.21 for 3 km
Remote Sens. 2016, 8, 815 9 of 14

DT AOD). Small but consistent positive errors were also observed over wetlands in the Eastern US
(mean retrieval errors of 0.04 for 3 km DT, 0.03 for 10 km DT, 0.03 for 10 km DB-DT, and 0.02 for
10 km DB). The 3 km DT and, to a lesser degree, 10 km DT AOD also showed significant positive
errors over shrub lands in the Western US (mean retrieval errors were 0.10 and 0.06 for 3 km DT
and 10 km DT, respectively). The best agreement between MODIS and AERONET was over forests,
grasslands, and cultivated lands. Overall, the 10 DB AOD had the least retrieval errors across all land
cover types (<0.03), followed by the 10 km DB-DT AOD (greatest mean retrieval error of 0.10 over
developed areas in the Western US). Previous studies have identified high retrieval error in AOD
retrievals over developed areas, and the retrieval error in DT products over poorly vegetated surfaces
to which 10 km DB retrievals are more robust [16,20,31].

3.6. Dependence of Retrieval Errors on Season and Weather Conditions

Figure 4 summarizes monthly retrieval errors from each AOD parameter. Median retrieval
errors in the 10 km DT, 10 km DB-DT, and 3 km DT parameters were the highest in the summer
months. The 3 km DT AOD had the widest fluctuation of retrieval errors over the course of the year
(0.009 in December to 0.056 in May). The 10 km DB product had a more even distribution over time,
from −0.01 in August to 0.01 in February. The reasons for the increased positive retrieval error in the
DT-based AOD parameters in the summer months is unclear, and has not been well-characterized in
previous work on this collection. However, despite the fact that collocations in summer months are
associated with increased mean NDVI values, which typically result in better accuracy statistics for DT
products, collocations in these months also have higher scattering angles, lower solar zenith angles,
and higher values of total column precipitable water, all factors that result in positive retrieval errors
over the CONUS.

(a) 3 km DT (b) 10 km DT

0.2

0.1

0.0

−0.1
Retrieval error ( τM−τA)

−0.2

(c) 10 km DB−DT (d) 10 km DB

0.2

0.1

0.0

−0.1

−0.2
Mar Jun Sep Mar Jun Sep
Month of Year

Figure 4. Boxplot showing the distribution of retrieval errors in MODIS AOD relative to AERONET,
for each month of the year, for 3 km DT (a); 10 km DT (b); 10 km DB-DT (c); and 10 km DB (d).
For each box, the midline represents the median, upper, and lower hinges represent the 25th and 75th
percentiles, whiskers extend out to 5th and 95th percentiles.
Remote Sens. 2016, 8, 815 10 of 14

As mentioned above, lower NDVI has been associated with increased retrieval error and noise
in MODIS
Remote Sens.AOD
2016, 8,retrievals,
815 particularly for DT-based products, in previous works [16].10In of the
14 West,

this is clearly shown in the 3 km DT AOD, and to a lesser degree in 10 km DT AOD (Figure 5).
this is clearly shown in the 3 km DT AOD, and to a lesser degree in 10 km DT AOD (Figure 5). The
The 10 km DB AOD in the Western US was unbiased in the lowest three quintiles of NDVI, but was
10 km DB AOD in the Western US was unbiased in the lowest three quintiles of NDVI, but was
negatively biased in the upper two quintiles (up to −0.03 in the highest quintile). This pattern was
negatively biased in the upper two quintiles (up to −0.03 in the highest quintile). This pattern was
observed
observed ininthe
theglobal
globalvalidations
validations asaswell
well[12]
[12]
andand it likely
it likely points
points to antooverestimation
an overestimation of the surface
of the surface
reflectance over vegetated areas in the eastern US. In the East, AOD retrieval errors
reflectance over vegetated areas in the eastern US. In the East, AOD retrieval errors are less dependent are less dependent
on on
NDVI,
NDVI, and
andthethenegative retrievalerror
negative retrieval errorobserved
observed forfor
DB DB at higher
at higher NDVI NDVI
valuesvalues
was not was not observed.
observed.
Both
Bothhumidity
humidityand potentialcloud
and potential cloud contamination
contamination have have been shown
been shown to biasobservations,
to bias MODIS MODIS observations,
and
andtotal column
total column precipitable water
precipitable (TCPW)
water (TCPW)can becana be
marker for both
a marker for factors [32]. Figure
both factors 5 shows5 ashows a
[32]. Figure
complex
complex relationship between
relationship between AODAODretrieval
retrieval errors
errorsandandTCPW. In the
TCPW. InWestern US, TCPW
the Western has little
US, TCPW has little
impact on the retrieval errors of 10 km DB and DB-DT AOD, but both very high or very low TCPW
impact on the retrieval errors of 10 km DB and DB-DT AOD, but both very high or very low TCPW
values are associated with positive retrieval errors in the 3 km and 10 km DT AOD. In the Eastern
values are associated with positive retrieval errors in the 3 km and 10 km DT AOD. In the Eastern US,
US, the 10 km DB AOD is negatively associated with TCPW. However, the impact of TCPW is
thegenerally
10 km DB AOD is negatively associated with TCPW. However, the impact of TCPW is generally
small for all AOD parameters, except at very high levels where both the 3 km and 10 km
small
DT AOD showed parameters,
for all AOD except
a small positive at very
retrieval error. high levels
At higher where
TCPW boththis
values, thebias
3 km and indicative
is likely 10 km DT AOD
showed
of clouda small positive retrieval
contamination, error.
and the lack At higher
of retrieval errorTCPWin 10values, this biasunder
km DB product is likely
theseindicative
conditionsof cloud
contamination, and the results,
fits with our coverage lack of which
retrieval error more
suggests in 10 conservative
km DB product cloudunder theseprocedures
screening conditionsforfits
thiswith our
product.results, which suggests more conservative cloud screening procedures for this product.
coverage

Figure 5. Dependence of AOD retrieval errors and distributions of values in QAC 3 MODIS AOD for
Figure 5. Dependence of AOD retrieval errors and distributions of values in QAC 3 MODIS AOD
NDVI (a) and TCPW (b). Median AOD retrieval error (dots) and the IQR (vertical line ranges from
for NDVI (a) and TCPW (b). Median AOD retrieval error (dots) and the IQR (vertical line ranges
25th percentile to 75th) is shown within quintiles of NDVI, and total column precipitable water. The
from 25th percentile to 75th) is shown within quintiles of NDVI, and total column precipitable water.
density distribution of values is shown in the background in gray and represents proportional
The density distribution
frequency, where 0.25 on of thevalues is shown the
y-axis represents in the background
most in in
frequent value gray
the and represents proportional
category.
frequency, where 0.25 on the y-axis represents the most frequent value in the category.
4. Case Study
4. Case Study
We conducted a case study over the Atlanta Metropolitan Area from 1 January 2004 to 31
December 2013. The
We conducted study
a case area
study stretched
over fromMetropolitan
the Atlanta 32°N to 36°NArea
latitude
fromand from 83°W
1 January to 31
2004 to 86°W
December
longitude, and included 23 ground-level PM
◦ 2.5 monitors
◦ in 19 distinct grid cells
◦
2013. The study area stretched from 32 N to 36 N latitude and from 83 W to 86 W longitude,from the
◦ same
and~12 km × 12 km grid used in the coverage analysis. This case study compared the ability of each of
included 23 ground-level PM2.5 monitors in 19 distinct grid cells from the same ~12 km × 12 km
the four MODIS AOD products to predict ground-level PM2.5 in a widely used linear mixed effect
grid used in the coverage analysis. This case study compared the ability of each of the four MODIS
(LME) model framework [33]. Three AOD datasets were generated for each of the four MODIS AOD
AOD products to predict ground-level PM2.5 in a widely used linear mixed effect (LME) model
Remote Sens. 2016, 8, 815 11 of 14

framework [33]. Three AOD datasets were generated for each of the four MODIS AOD products:
(1) AOD values with only QAC = 3; (2) AOD values with the highest available QAC (1, 2 or 3);
and (3) filtered and corrected AOD values using the relationships examined in Sections 3.5 and 3.6 prior
to inclusion in the model. To produce the filtered and corrected dataset, AOD values from dataset #2
with scattering angles over 165◦ or solar zenith angles less than 15◦ were first eliminated. This removed
~9% of observations while some were additionally lost in the matching process. This filtered dataset
was then corrected, using a linear regression model fit to the dataset of matched AERONET and
MODIS AOD observations in the Eastern CONUS, used in the accuracy analysis, for QAC value, land
use type, sensor zenith angle, total column precipitable water, and NDVI. All 12 combinations of AOD
were fit using the LME model, of the form: PM2.5,s,t = (b0 + b0,t ) + (b1 + b1,t )AODs,t , where b0 is the
fixed intercept, b0,t the random intercept for each day, b1 the fixed slope, and b1,t the random slope for
each day [33]. These results are presented in Table 3.

Table 3. Performance statistics for each AOD parameter in the Atlanta case study.

QAC 3 Only Best of QAC 1, 2, and 3 Filtered & Corrected AOD

Parameter
N R2 Fixed Slope N R2 Fixed Slope N R2 Fixed Slope
3 km DT 10,438 0.76 14.4 10,438 0.76 14.4 9680 0.75 20.2
10 km DT 8994 0.72 20.4 11,511 0.77 16.1 10,593 0.75 23.4
10 km DB-DT 8994 0.72 20.4 8994 0.72 20.4 8457 0.71 25.7
10 km DB 8560 0.80 31.2 14,706 0.83 11.7 13,448 0.83 16.7

For the parameters where the number of observations increased with the addition of lower QAC
valued observations, 10 km DB and 10 km DT, R2 values for a model relating ground-level PM2.5
concentrations to AOD actually increased slightly. Increasing from 0.80 to 0.83 for 10 km DB and
from 0.72 to 0.75 for 10 km DT. When AOD values were filtered and corrected to remove potentially
biased observations, model fits for the 10 km DT product decreased slightly, from 0.77 to 0.75 and
remained the same for the 10 km DB product. For the parameters with relatively few lower quality
observations, 10 km DB-DT and 3 km DT, neither the number of observations included in the model
nor the resulting R2 values changed when lower confidence observations were included in the model.
When filtering and correction was applied, model fits, as measured by the R2 values, actually decreased
by 0.01 relative to the ‘best of’ models. These results run counter to what would be expected: that R2
values for all four parameters would decrease slightly with the inclusion of lower-confidence retrievals,
given the fact that the lower confidence observations for the 10 km DB and 10 km DT parameters are
noisier. However, in this case study, the additional number of observations appears to have offset
the additional noise introduced via these observations in the model and resulted in better prediction
of ground-level PM2.5 via this simple model. Despite the smaller sample sizes, the models using the
corrected and filtered AOD values achieved similar R2 values as the uncorrected AOD models.
These results illustrate some of the key points made in this paper, namely that coverage is an
often-overlooked but important factor, when considering AOD accuracy statistics, and that, because
of the role played by coverage, the inclusion of lower-quality AOD observations in a model can
provide some benefit. These results additionally highlight our observation that the lower confidence
designations for the newer products, 10 km DB-DT and 3 km DT, are very few in number. The utility
of correcting for major sources of bias or error in the AOD values was demonstrated in this limited
example by greater fixed effect regression slopes, indicating greater sensitivity of the corrected and
filtered AOD values to PM2.5 concentrations.

5. Conclusions
We conducted a detailed analysis on the coverage and accuracy of Collection 6 MODIS AOD
parameters in the CONUS. With their applications in air quality research in mind, we examined the
benefits and risks of including lower QAC retrievals in order to improve data coverage, as well as how
Remote Sens. 2016, 8, 815 12 of 14

AOD retrieval errors depend on various factors. Our recommendation is that, for inexperienced users
who are beginning to explore MODIS AOD data for air quality research, the QAC 3 10 km DB-DT AOD
is their best choice. For more experienced users, the ideal AOD parameter could depend on the purpose
as well as domain of their study. The coverage of QAC 3 retrievals is comparable among all four AOD
parameters, ranging from 25% to 30%. The Eastern US in general had higher and more consistent data
coverage. However, much higher coverage rates were found in highly developed Southern California
and over the south-central plains with limited ground-level air pollution monitoring, a surprising fact
since these areas are traditionally regarded as having too high of surface brightness for DT to retrieve
reliably. These findings are promising to researchers interested in conducting regional air quality
assessments in these regions. Including lower QAC retrievals marginally improved coverage for the
3 km DT and 10 km DB-DT AOD. However, since these QAC assignments do not appear to reflect
retrieval accuracy and are few in number, including lower QAC retrievals of these two parameters
is probably beneficial, and unlikely to be harmful. On the other hand, lower QAC retrievals could
increase the coverage of 10 km DT AOD by ~20% and that of 10 km DB AOD by ~70% on average.
Caution must be given when including them to enhance coverage as these retrievals are often noisier,
as shown in Table 2. However, as demonstrated in the case study, sufficient daily sample sizes can
sometimes be more important than retaining only the high quality AOD values for the purposes of
improving prediction errors with ground-level PM2.5 . To take advantage of the dramatic coverage
gain offered by these lower QAC retrievals, retrieval error correction steps using local AERONET
observations could be valuable [7].
In terms of data accuracy, the 10 km DB-DT AOD had the best performance in terms of correlation
and linear model fit statistics, although QAC 3 retrievals for all but the 3 km DT AOD over the Western
US met pre-launch expectations for the percentage of collocations within EEDT . However, the 10 km
DB product performs well in the context of a prediction model and may be an understudied AOD
parameter in the US, where the 10 km DT product is currently used more frequently. The robustness of
this product to major sources of bias additionally makes it an attractive option in the Western United
States. The noisier 3-km DT AOD, however, can be valuable over dark targets in the Eastern US,
particularly over areas where it tends to retrieve at higher rates than the lower-resolution products,
such as in the Northeast and Northern Midwest, over the South Central plains in Texas, and over
Southern Florida. The errors in MODIS AOD parameters vary in time and space, and are dependent
on various retrieval conditions. Additional data screening and retrieval error correction steps should
be considered other than simply relying on the QAC values, particularly in the Western United
states, where these biases tend to have a larger impact. For example, AOD retrievals associated with
high scattering angles and lower solar zenith angles may be excluded to avoid data contamination.
Such parameters can be found in the operational MODIS aerosol product. In addition, categorical
variables of land cover types as well as time trends can be introduced in PM2.5 exposure models
to control for the systematic retrieval errors in DT-based AOD retrievals. NDVI and TCPW had
statistically significant, distinct impacts on all AOD parameters in the Western US, and therefore are
probably worth considering when analyzing AOD data. Since they must be extracted from separate
MODIS data products, users would need to consider the nontrivial time and computational demands
associated with dealing with these large datasets.

Supplementary Materials: The following are available online at www.mdpi.com/2072-4292/8/10/815/s1,

Figure S1: Seasonal coverage for high confidence retrievals, Figure S2: Seasonal coverage for all-confidence
retrievals, Figure S3: AOD retrieval errors by land cover type.
Acknowledgments: This work was partially supported by the NASA Applied Sciences Program (grant No.
NNX11AI53G, PI: Liu). In addition, this publication was made possible by the AERONET principal investigators
and their staff for establishing and maintaining the 193 sites used in this investigation. The original MODIS
AOD, AERONET AOD, land use, and MODIS NDVI data used in this paper are available free through the links
referenced in Section 2 of the paper. We have additionally made our processing and analysis code, gridded
coverage estimates, and MODIS and AERONET collocations produced for this analysis available on GitHub
(https://github.com/jhbelle/MODIS_C6_Evaluation_over_the_CONUS).
Remote Sens. 2016, 8, 815 13 of 14

Author Contributions: J.H.B. and Y.L. conceived and designed the experiments; J.H.B. performed the experiments;
J.H.B. analyzed the data; J.H.B. and Y.L. contributed reagents/materials/analysis tools; J.H.B. and Y.L. wrote
the paper.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Holben, B.; Tanre, D.; Smirnov, A.; Eck, T.; Slutsker, I.; Abuhassan, N.; Newcomb, W.; Schafer, J.; Chatenet, B.;
Lavenu, F. An emerging ground-based aerosol climatology: Aerosol optical depth from aeronet. J. Geophys.
Res. Atmos. 2001, 106, 12067–12097. [CrossRef]
2. Brauer, M.; Freedman, G.; Frostad, J.; Van Donkelaar, A.; Martin, R.V.; Dentener, F.; Dingenen, R.V.; Estep, K.;
Amini, H.; Apte, J.S. Ambient air pollution exposure estimation for the global burden of disease 2013.
Environ. Sci. Technol. 2015, 50, 79–88. [CrossRef] [PubMed]
3. Van Donkelaar, A.; Martin, R.V.; Brauer, M.; Boys, B.L. Use of satellite observations for long-term exposure
assessment of global concentrations of fine particulate matter. Environ. Health Perspect. 2015, 123, 135–143.
[CrossRef] [PubMed]
4. Remer, L.A.; Kaufman, Y.; Tanré, D.; Mattoo, S.; Chu, D.; Martins, J.V.; Li, R.-R.; Ichoku, C.; Levy, R.;
Kleidman, R. The MODIS aerosol algorithm, products, and validation. J. Atmos. Sci. 2005, 62, 947–973.
[CrossRef]
5. Hoff, R.M.; Christopher, S.A. Remote sensing of particulate pollution from space: Have we reached the
promised land? J. Air Waste Manag. Assoc. 2009, 59, 645–675. [PubMed]
6. Gupta, P.; Christopher, S.A.; Wang, J.; Gehrig, R.; Lee, Y.; Kumar, N. Satellite remote sensing of particulate
matter and air quality assessment over global cities. Atmos. Environ. 2006, 40, 5880–5892. [CrossRef]
7. Ma, Z.; Hu, X.; Sayer, A.M.; Levy, R.; Zhang, Q.; Xue, Y.; Tong, S.; Bi, J.; Huang, L.; Liu, Y. Satellite-based
spatiotemporal trends in PM2.5 concentrations: China, 2004–2013. Environ. Health Perspect. 2015, 124, 184–192.
[CrossRef] [PubMed]
8. Chang, H.H.; Hu, X.; Liu, Y. Calibrating MODIS aerosol optical depth for predicting daily PM2.5
concentrations via statistical downscaling. J. Expo. Sci. Environ. Epidemiol. 2014, 24, 398–404. [CrossRef]
[PubMed]
9. Smith, K.R.; Jantunen, M. Why particles? Chemosphere 2002, 49, 867–871. [CrossRef]
10. Strickland, M.; Hao, H.; Hu, X.; Chang, H.; Darrow, L.; Liu, Y. Pediatric emergency visits and short-term
changes in PM2.5 concentrations in the US State of Georgia. Environ. Health Perspect. 2015, 124, 690–696.
[CrossRef] [PubMed]
11. Crouse, D.L.; Peters, P.A.; van Donkelaar, A.; Goldberg, M.S.; Villeneuve, P.J.; Brion, O.; Khan, S.;
Atari, D.O.; Jerrett, M.; Pope, C.A., III. Risk of nonaccidental and cardiovascular mortality in relation to
long-term exposure to low concentrations of fine particulate matter: A Canadian national-level cohort study.
Environ. Health Perspect. 2012, 120, 708–714. [CrossRef] [PubMed]
12. Sayer, A.; Munchak, L.; Hsu, N.; Levy, R.; Bettenhausen, C.; Jeong, M.J. MODIS Collection 6 aerosol products:
Comparison between Aqua’s e-deep blue, dark target, and “merged” data sets, and usage recommendations.
J. Geophys. Res. Atmos. 2014, 119. [CrossRef]
13. Liu, Y. New directions: Satellite driven PM2.5 exposure models to support targeted particle pollution health
effects research. Atmos. Environ. 2013, 68, 52–53. [CrossRef]
14. Kloog, I.; Ridgway, B.; Koutrakis, P.; Coull, B.A.; Schwartz, J.D. Long-and short-term exposure to PM2.5 and
mortality: Using novel exposure models. Epidemiology 2013, 24, 555–561. [CrossRef] [PubMed]
15. Hsu, N.; Jeong, M.J.; Bettenhausen, C.; Sayer, A.; Hansell, R.; Seftor, C.; Huang, J.; Tsay, S.C. Enhanced
deep blue aerosol retrieval algorithm: The second generation. J. Geophys. Res. Atmos. 2013, 118, 9296–9315.
[CrossRef]
16. Levy, R.; Mattoo, S.; Munchak, L.; Remer, L.; Sayer, A.; Hsu, N. The collection 6 MODIS aerosol products
over land and ocean. Atmos. Meas. Tech. 2013, 6, 2989–3034. [CrossRef]
17. Remer, L.; Mattoo, S.; Levy, R.; Munchak, L. MODIS 3 km aerosol product: Algorithm and global perspective.
Atmos. Meas. Tech. Discuss. 2013, 6, 69–112. [CrossRef]
18. Sayer, A.; Hsu, N.; Bettenhausen, C.; Jeong, M.J. Validation and uncertainty estimates for MODIS collection 6
“deep blue” aerosol data. J. Geophys. Res. Atmos. 2013, 118, 7864–7872. [CrossRef]
Remote Sens. 2016, 8, 815 14 of 14

19. Livingston, J.; Redemann, J.; Shinozuka, Y.; Johnson, R.; Russell, P.; Zhang, Q.; Mattoo, S.; Remer, L.; Levy, R.;
Munchak, L. Comparison of MODIS 3 km and 10 km resolution aerosol optical depth retrievals over land
with airborne sunphotometer measurements during arctas summer 2008. Atmos. Chem. Phys. 2014, 14,
2015–2038. [CrossRef]
20. Munchak, L.; Levy, R.; Mattoo, S.; Remer, L.; Holben, B.; Schafer, J.; Hostetler, C.; Ferrare, R. MODIS 3 km
aerosol product: Applications over land in an urban/suburban region. Atmos. Meas. Tech. 2013, 6, 1747–1759.
[CrossRef]
21. Levy, R. Collection 006 (C6) MODIS Atmosphere l2 Aerosol Product, 6th ed.LAADS Web, 2014. Available
online: https://ladsweb.nascom.nasa.gov/data/search.html (accessed on 15 July 2014).
22. Slutsker, I.; Kinne, S. Wavelength dependence of the optical depth of biomass burning, urban, and desert
dust aerosols. J. Geophys. Res. 1999, 104, D24.
23. Carroll, M.; DiMiceli, R.; Sohlberg, R.; Townshend, J. 1 km MODIS Normalized Difference Vegetation Index,
5th ed.; University of Maryland, LAADS Web: College Park, MD, USA, 2015.
24. Fry, J.A.; Xian, G.; Jin, S.; Dewitz, J.A.; Homer, C.G.; Limin, Y.; Barnes, C.A.; Herold, N.D.; Wickham, J.D.
Completion of the 2006 national land cover database for the conterminous United States. Photogramm. Eng.
Remote Sens. 2011, 77, 858–864.
25. Hyer, E.; Reid, J.; Zhang, J. An over-land aerosol optical depth data set for data assimilation by filtering,
correction, and aggregation of MODIS collection 5 optical depth retrievals. Atmos. Meas. Tech. 2011,
4, 379–408. [CrossRef]
26. Turner, R. Deldir: Delaunay Triangulation and Dirichlet (Voronoi) Tessellation. 2009. R Package Version
0.0-8. Available online: http://cran.r-project.org/web/packages/deldir (accessed on 31 July 2014).
27. Petrenko, M.; Ichoku, C.; Leptoukh, G. Multi-sensor aerosol products sampling system (MAPSS).
Atmos. Meas. Tech. 2012, 5, 913–926. [CrossRef]
28. Yu, C.; Di Girolamo, L.; Chen, L.; Zhang, X.; Liu, Y. Statistical evaluation of the feasibility of satellite-retrieved
cloud parameters as indicators of PM2.5 levels. J. Exposure Sci. Environ. Epidemiol. 2015, 25, 457–466.
[CrossRef] [PubMed]
29. Christopher, S.A.; Gupta, P. Satellite remote sensing of particulate matter air quality: The cloud-cover
problem. J. Air Waste Manag. Assoc. 2010, 60, 596–602. [CrossRef] [PubMed]
30. Levy, R.C.; Remer, L.A.; Mattoo, S.; Vermote, E.F.; Kaufman, Y.J. Second-generation operational algorithm:
Retrieval of aerosol properties over land from inversion of moderate resolution imaging spectroradiometer
spectral reflectance. J. Geophys. Res. Atmos. 2007, 112, D13. [CrossRef]
31. Tao, M.; Chen, L.; Wang, Z.; Tao, J.; Che, H.; Wang, X.; Wang, Y. Comparison and evaluation of the MODIS
collection 6 aerosol data in China. J. Geophys. Res. Atmos. 2015, 120, 6992–7005. [CrossRef]
32. Ford, B.; Heald, C.L. Aerosol loading in the southeastern united states: Reconciling surface and satellite
observations. Atmos. Chem. Phys. 2013, 13, 9269–9283. [CrossRef]
33. Lee, H.; Liu, Y.; Coull, B.; Schwartz, J.; Koutrakis, P. A novel calibration approach of MODIS aod data to
predict PM2.5 concentrations. Atmos. Chem. Phys. 2011, 11, 7991–8002. [CrossRef]

© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC-BY) license (http://creativecommons.org/licenses/by/4.0/).