Algorithm Development of Temperature and Humidity Profile Retrievals for Long-Term HIRS Observations

Abstract

A project for deriving temperature and humidity profiles from High-resolution Infrared Radiation Sounder (HIRS) observations is underway to build a long-term dataset for climate applications. The retrieval algorithm development of the project includes a neural network retrieval scheme, a two-tiered cloud screening method, and a calibration using radiosonde and Global Positioning System Radio Occultation (GPS RO) measurements. As atmospheric profiles over high surface elevations can differ significantly from those over low elevations, different neural networks are developed for three classifications of surface elevations. The significant impact from the increase of carbon dioxide in the last several decades on HIRS temperature sounding channel measurements is accounted for in the retrieval scheme. The cloud screening method added one more step from the HIRS-only approach by incorporating the Advanced Very High Resolution Radiometer (AVHRR) observations to assess the likelihood of cloudiness in HIRS pixels. Calibrating the retrievals with radiosonde and GPS RO reduces biases in retrieved temperature and humidity. Except for the lowest pressure level which exhibits larger variability, the mean biases are within ±0.3 °C for temperature and within ±0.2 g/kg for specific humidity at standard pressure levels, globally. Overall, the HIRS temperature and specific humidity retrievals closely align with radiosonde and GPS RO observations in providing measurements of the global atmosphere to support other relevant climate dataset development.

1. Introduction

Satellite soundings have been providing global measurements of the atmospheric temperature and humidity for several decades with operational measurements dating back to 1972. These measurements form the foundation for long-term monitoring of the atmosphere. Onboard the operational National Oceanic and Atmospheric Administration (NOAA) polar orbiting satellites NOAA-2 through NOAA-5, the first operational sounder, Vertical Temperature Profile Radiometer (VTPR), provided infrared observations from eight channels from 1972 to 1979 [1]. After several years of VTPR operation, its successor High-resolution Infrared Radiation Sounder (HIRS) started making observations, together with the Microwave Sounding Unit (MSU) and the Stratospheric Sounding Unit (SSU), with the launch of the Television Infrared Observation Satellite (TIROS-N) in 1978. The HIRS instruments have been obtaining atmospheric data since then onboard the subsequent NOAA series of satellites and on the meteorological operational satellite program (Metop) series operated by the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT). Routine microwave soundings of the atmosphere began in 1998 with the Advanced Microwave Sounding Unit A (AMSU-A) and Unit B (AMSU-B), and more recently with the microwave humidity sounder (MHS) and the Advanced Technology Microwave Sounder (ATMS). Hyperspectral sounders, such as the Atmospheric Infrared Sounder (AIRS), Infrared Atmospheric Sounding Interferometer (IASI), and Cross-track Infrared Sounder (CrIS), on recent satellites marked the new era of satellite infrared sounders.

Among these satellite soundings, HIRS observations span the longest time period (1978 to present). The HIRS instrument has twenty channels, including twelve channels in the longwave regime, seven channels in the shorter wave regime, and one shortwave channel. The HIRS footprint is approximately 20 km and 10 km at nadir for the HIRS/2 and HIRS/3 instruments, respectively. Among the longwave channels, channels 1 to 7 are in the carbon dioxide (CO₂) absorption band to measure atmospheric temperatures from near-surface to stratosphere, channel 8 is a window channel for surface temperature observation, channel 9 is an ozone channel, and channels 10–12 are for water vapor signals at the near-surface, mid-troposphere, and upper troposphere, respectively. In the present study, temperature and humidity profiles are derived from these HIRS longwave channel observations for long-term studies.

HIRS observations have been used to derive temperature and humidity profiles since its initial operation. For example, in the early years of HIRS observations, a physically-based satellite temperature sounding retrieval system was developed at Goddard Laboratory for Atmospheric Sciences to determine atmospheric temperature profiles along with several surface variables [2]. Later, a physical-statistical algorithm, named the Improved Initialization Inversion (3I) [3], for retrieving meteorological parameters from TIROS-N satellite data at a spatial resolution of 100 km was built. Updated from an earlier version, the International Advanced Television and Infrared Observation Satellite Operational Vertical Sounder (ATOVS) Processing Package (IAPP) was developed for retrieving atmospheric temperature and moisture profiles, total ozone, and other parameters in real-time [4]. NOAA has been maintaining an operational HIRS sounding product system [5,6]. These studies advanced our knowledge on the advantages and limitations of the HIRS observations. However, as many of the past studies were geared toward operational or near-real-time weather applications, the produced datasets may not be suitable for climate research.

To build a long-term dataset for climate applications, development of a Climate Data Record (CDR) for temperature and humidity profiles from inter-satellite calibrated HIRS data is underway. The development is different from weather applications in that the long-term consistency of the algorithm and data is a key component. The project consists of several aspects of development, including inter-satellite calibration, retrieval algorithm development, and evaluation of the consistency of the retrievals with independent observational sources. The focus of the present study is on the retrieval algorithm development. One of the major drivers of the development is to build a temperature and humidity dataset that can be used in the construction of relevant CDR products, such as cloud products in the International Satellite Cloud Climatology Project (ISCCP) [7,8]. This requires the temperature and humidity dataset to have a long-term consistency for different climate regimes.

During the past three decades the atmospheric CO₂ concentration increased substantially. The increase of CO₂ has a significant impact on HIRS channel radiances [9,10]. The development of the retrieval scheme aims to account for the effect of CO₂ on the long-term HIRS observation and to obtain consistency between the upper air retrievals and observations from conventional sources (i.e., homogeneous radiosonde observations in the troposphere and Global Positioning System Radio Occultation (GPS RO) temperatures in the stratosphere). In the following sections the retrieval scheme is described. The retrievals at standard pressure levels are compared with observations not used during algorithm training, and the results are discussed.

2. Algorithm Development

There are three major components in the retrieval algorithm development in this study, including the retrieval scheme design, improvement of cloud screening, and bias-calibration scheme development. The retrieval is based on inter-satellite calibrated HIRS longwave channel data. Details on the data used in the study and each of the development components are described below.

2.1. Data

The temperature profiles are derived from HIRS longwave channel measurements. Channel brightness temperatures are limb-corrected using a linear multivariate regression algorithm based on multiple HIRS channels [11]. Due to the difference in individual HIRS instrumentss channel spectral response functions, along with other factors, there are differences in observations from different satellites. Inter-satellite calibrated HIRS measurements provide an essential dataset for climate studies. Simultaneous nadir overpass (SNO) observations are used to obtain inter-satellite differences between overlapping satellite pairs [12,13,14]. Measurement over the equatorial Western Pacific region is used to assess inter-satellite differences in the warm range of observations. For the majority of the HIRS longwave channels, the inter-satellite differences vary with channel radiances [14]. Inter-satellite differences of these channels are derived from overlapping satellites as a function of brightness temperatures. These values are then applied to individual satellite’s HIRS channel measurements to inter-calibrate data to the reference satellite, for which Metop-A is designated.

For channel 4, the values of inter-satellite differences do not correlate with the values of scene brightness temperatures. When there is a difference in channel weighting functions of different satellites, the channels essentially sense temperatures in different heights. For most of the channels, the scene temperatures are correlated with the lapse rate and therefore the inter-satellite differences are also correlated with scene temperatures. However, channel 4 senses temperatures in the upper troposphere near the tropopause. The inter-satellite difference is largely dependent on the heights of the weighting function peaks in relation to the tropopause. It is found that instead of being correlated to the scene temperature, the inter-satellite difference of channel 4 is highly correlated with the temperature lapse rate represented by the differences of vertically adjacent HIRS sounding channels [15]. Therefore, the inter-satellite calibration of channel 4 uses a method different from other longwave channels. Using SNO observations, calibrations based on linear regression are developed between channel 4 inter-satellite differences and the lapse rate factors below and above channel 4 (represented by the sounding channel brightness temperature difference of channels 5 and 4, and between the difference of channels 4 and 3, respectively). Using the regressions, measurements of channel 4 are calibrated to the same reference satellite, Metop-A.

GPS RO measurements from Constellation Observing System for Meteorology Ionosphere and Climate (COSMIC) and radiosonde observations are incorporated as part of the retrieval process. The re-processed version of GPS RO derived profiles, COSMIC2013, is obtained from University Corporation for Atmospheric Research COSMIC Data Analysis and Archive Center (CDAAC) (http://cosmicio.cosmic.ucar.edu/cdaac/index.html) to use in the project. Various studies have shown that GPS RO missions provide a unique opportunity to measure stratospheric temperatures in high accuracy [16,17,18]. The GPS RO data does not contain mission-dependent biases. This makes them a good candidate as a climate benchmark [19]. The GPS RO dataset is incorporated to calibrate the HIRS stratospheric temperature retrievals.

However, the GPS RO derivation of temperature and humidity for the troposphere relies on the reanalysis data as input and, therefore, the derived profiles are not considered direct observations. One alternative is to incorporate radiosonde observations that are homogeneous both spatially and temporally. Though using radiosonde observations may bring some biases toward profiles over land surfaces, options are very limited for other choices. None of other data sources has a more extensive global dataset of direct observations of temperature and humidity in the troposphere. There are hundreds of radiosonde stations globally with different types of radiosonde systems. The accuracy of radiosonde temperature and humidity measurements can vary significantly for different sensor types [20]. To serve as a calibration database for remotely-sensed data, it is necessary that the selected radiosonde measurements have self-consistency with good quality. Among radiosonde types, measurements from RS92 radiosondes have been shown to provide homogeneous observations in global locations [21] and, thus, they are used in this study for the bias calibration process of the global tropospheric temperature and humidity retrievals.

2.2. Carbon Dioxide Effect

Observations at Mauna Loa show significant rise of CO₂ in the past several decades [22]. Since HIRS observation began in 1978, the atmospheric CO₂ concentration has increased from below 335 ppmv to over 400 ppmv. To examine the CO₂ effect, a radiative transfer model, RTTOV [23], is used to simulate HIRS brightness temperatures in an increasing CO₂ environment. The use of a model facilitates the separation of CO₂ effect from any other changes. A diverse sample of global profiles analyzed by the European Center for Medium-range Weather Forecasts (ECMWF) system [24] is used to represent the atmospheric conditions. For the model simulations, the input profiles are kept unchanged except for the CO₂ concentrations, and RTTOV is run for the CO₂ concentrations of 330 and 410 ppmv, respectively. The averaged brightness temperature of the global results for each HIRS channel is calculated. The differences of channel brightness temperatures between the two CO₂ concentration values are displayed in Figure 1.

The weighting functions of HIRS CO₂ absorption channels vary with the amount of CO₂. With the increase in CO₂, the atmosphere becomes more opaque to the HIRS CO₂ sounding channels, therefore the peaks of these channels’ weighting functions go up. As the mean temperature lapse rate in the troposphere decreases with height and, vice versa, in the stratosphere, there is a negative and positive impact in the troposphere and stratosphere, respectively. For easy reference, the absorbents of HIRS longwave channels and the approximate broad layers where the channels sense are listed in

Table 1. Absorbents of HIRS longwave channels and the approximate broad layers where the channels sense.

Channel	Absorption	Altitude (hPa)
1	CO2	5–100
2	CO2	20–150
3	CO2	30–300
4	CO2	80–500
5	CO2	300–800
6	CO2	400–1000
7	CO2	600–1000
8	(window)	Surface
9	O3	10–100
10	Water vapor	700–1000
11	Water vapor	400–800
12	Water vapor	250–500

. The simulation shows that if CO₂ concentration were the only change in the atmosphere, when CO₂ increases from 330 ppmv to 410 ppmv, the measurements from HIRS would appear to decrease in channels 4–7. The largest impact occurs in channels 5 and 6, with a value as large as −2.0 K. The decreases of brightness temperatures for channels 4, and 7 are −1.6 and −1.0 K, respectively. For channels sensing the stratosphere (channels 1 and 2), the measurements from HIRS observation would appear larger in an increased CO₂ environment. The increases in these two channels are 0.8 and 0.6 K, respectively. Channel 3 senses temperatures in the lower stratosphere near the tropopause where the lapse rate is small and, thus, there is not much effect from the CO₂ change. There are no impacts on channels 8–12 because these are window channel, ozone channel, and water vapor channels. Figure 1 shows that there are large effects from CO₂ increase on several temperature sounding channels. If these effects are not considered in the temperature profile retrieval, it can lead to an underestimation of tropospheric temperatures and an overestimation of stratospheric temperatures in the HIRS measurements in an increasing CO₂ environment.

2.3. Retrieval

The analysis above shows that it is a requirement for the retrieval algorithm to be able to account for CO₂ effect on HIRS observations. This can be accomplished by the use of a radiative transfer model. As the RTTOV model is developed to simulate satellite sounder measurements with HIRS as one of the main sounders in the design, the model is chosen in the present study as a tool to build a retrieval training dataset. Selections of the ECMWF sampled profiles [24] provide the input database to RTTOV. The profile dataset was formed by carefully analyzing reanalysis data to extract a subset that has a global representation. Millions of global profiles from two years of the ECMWF analysis fields were divided into seven groups according to total precipitable water vapor contents at the interval of 0.5 kg·m⁻². About the same number of samples from each group was extracted, except for the group with the smallest total precipitable water vapor content, where twice as many profiles were extracted. Only the clear-sky profiles from the ECMWF sample profiles are used for the present study, which are comprised of 6891 profiles covering all latitudes and longitudes. The HIRS channel brightness temperatures for the reference satellite of the inter-calibrated dataset, Metop-A, are simulated by the radiative transfer model RTTOV.

Tests are carried out with the use of RTTOV on the sensitivity of surface emissivity errors on retrievals. Results show that surface emissivity has a significant impact on the surface and near-surface temperatures. A variation of 0.2 in emissivity can have an impact of 1.1 °C on the surface skin temperature and 0.7 °C on surface air temperature. Therefore, surface emissivity is included as an input for the retrieval of surface and near-surface variables. The impact is reduced to less than 0.3 °C at levels 850 hPa and up. The impact is small on specific humidity. With an emissivity variation of 0.2, the impact is 0.17 g/kg near the surface and smaller toward upper levels.

Among the HIRS channels, the input to the upper air temperature retrievals consists of channels 2–12, and for the upper air specific humidity retrievals the input channels include the tropospheric channels 4–8 and 10–12. For the surface retrievals, only channels with sensitivity to the surface are used, which include channels 7, 8, and 10. Monthly mean values of CO₂ observation from Mauna Loa are included in the input. The surface emissivity values are taken from the ISCCP dataset [7]. Outputs from the three neural networks are temperature profiles, including surface skin temperature, air temperature at the reference height of 2 m, and temperature at standard pressure levels from 1000 or the lowest pressure level above the surface to 50 hPa, and humidity profiles including specific humidity at 2 m and at standard pressures from 1000 hPa or the lowest pressure level above the surface to 300 hPa.

As atmospheric profiles over high surface elevations can differ significantly from those over low elevations, the clear-sky training dataset is divided into three groups according to the surface elevation in terms of surface pressure (Ps) calculated using the hydrostatic equation. The three groups are defined as Ps < 700 hPa, 700 hPa ≤ Ps < 850 hPa, and Ps ≥ 850 hPa. Figure 2 shows a global map of where the groups are located. The vast majority of areas are in the group with Ps > 850 hPa. From each surface elevation group, collocated data are randomly divided into three sub-groups at 60%, 20% and 20%. The 60% sub-group is assigned to be the training dataset. A testing dataset is formed by one 20% sub-group to use during iterations of neural network development. The remaining 20% is set aside for assessing the performance of the retrieval.

A neural network approach is applied to connect temperature and humidity profiles with the HIRS longwave channel measurements. The use of the neural network technique enables the establishment of the non-linear relationship between the retrieved variables and channel radiance observations. Running retrievals with a neural network scheme is fast. Processing 35 years of HIRS retrievals using a Linux computer platform that has a moderate specification takes about a week to complete, and processing in parallel reduces this even further.

Different neural networks are built for the retrievals of the upper air temperature, upper air specific humidity, and surface temperature and humidity. Neural networks have been used in past studies to retrieve atmospheric temperature and humidity along with other variables. For example, a three-layer backpropagation neural network was applied to derive atmospheric temperature profiles from AMSU-A measurement [25]. The retrieved variables included surface air temperature, temperatures at 26 pressure levels from 1000 to 10 hPa, and the tropopause temperature, height, and pressure over both land and ocean surfaces. Multi-layer backpropagation approaches were used to derive water vapor, cloud liquid water path, surface temperature, and emissivity over land from satellite microwave observations [26] and derive temperature, water vapor, and ozone profiles from IASI observations [27]. Three-layer feed-forward neural networks were also used to retrieve near-surface atmospheric variables over the ocean surfaces including SST, Ta, Qa, and wind speed [28]. In the present study, three-layer backpropagation networks, with one input layer, one hidden layer, and one output layer, are constructed for the retrievals of temperature and humidity.

As described in the previous AMSU-A retrieval study [25], in a backpropagation network, each layer is fully connected to the layers below and above. When the network is given an input, the updating of activation values propagates forward from the input layer through the internal layer to the output layer. Each neuron in the output layer produces an output, which is compared to the target output defined in the training dataset. An error value is calculated for each neuron in the output layer. The network corrects its parameters to lessen the errors. The correction mechanism starts with the output neurons and propagates backward through the internal layer to the input layer. The iteration continues until pre-set convergence criteria are met.

The converged neural network parameters are applied to the data that were set aside and were not used in the construction of the neural network to evaluate the network performance. The RMSEs of temperature and specific humidity are calculated assuming the outputs in the validation database as truth data. For temperature, the RMSEs are approximately 2.7 °C for the temperature at the lowest standard pressure levels, 1.3–1.5 °C in the mid-troposphere, and 1.5–2.6 °C around the tropopause and in the lower stratosphere. For the specific humidity, the RMSE is 2.0 g/kg at 1000 hPa, and it steadily decreases to 1.1 g/kg at 700 hPa, and less than 0.4 g/kg above 500 hPa. The RMSEs for temperature and humidity profiles at standard pressure levels calculated from data not used during network calibration are plotted in Figure 3. These error estimates represent the uncertainty error structures from the model simulation and the neural network scheme.

One of the error sources may come from channel observation noises. The NOAA Polar Orbiter Data User’s Guide [29] and NOAA KLM User’s Guide [30] show that values of the noise equivalent differential radiance (NEΔN) for HIRS longwave channels are in the range of 0.10–0.65 mW/(m²·sr·cm⁻¹) for channels 2–12, and 3.00 mW/(m²·sr·cm⁻¹) for channel 1. Channel 1 is not used in the retrieval scheme because of the concern on the high NEΔN. Using the developed neural networks, a sensitivity test is performed by adding noise to each of channels 2–12 in the training dataset, one channel at a time, to examine how much difference it produces in the retrieval results. The result for the pressure level that corresponds to the mean weighting function peak of an individual channel from an added noise value of 0.5 K is displayed in

Table 2. Sensitivity test of temperature sounding channel noises on the retrievals. The impact is based on adding 0.5 K noise to individual channel brightness temperatures. The levels shown are the approximate peaks of individual channels’ weighting functions. The NEΔN values (mW/(m²·sr·cm⁻¹)) are taken from NOAA KLM User’s Guide.

Channel	Weighting Function Peak (hPa)	Impact on Retrieval (K)	NEΔN
2	100	−1.646	0.67
3	200	−2.039	0.50
4	300	2.669	0.31
5	500	2.258	0.21
6	700	1.589	0.24
7	850	1.235	0.20
8	surface	0.827	0.10
9	50	−0.212	0.15

and

Table 3. Similar to Table 2 for sensitivity tests of water vapor channel noise on the retrievals.

Channel	Weighting Function Peak (hPa)	Impact on Retrieval (g/kg)	NEΔN
10	850	−0.364	0.15
11	600	−0.376	0.20
12	400	−0.004	0.20

. The channel NEΔN values are included in the tables to provide typical values of channel noises. For the temperature sounding channels, NEΔN values of 0.10–0.65 mW/(m²·sr·cm⁻¹) approximately correspond to brightness temperature noises of 0.1–0.5 K. Different noise values are tested. A value of −0.5 K produces about the same impact as that from a value of 0.5 K, but with the opposite sign. Halving the noise value approximately halves the impact.

2.4. Cloud Screening

Retrieving atmospheric temperature and humidity profiles from an infrared instrument requires clear-sky conditions. When clouds are present, an infrared sounder, such as HIRS, can only sense to the top of clouds. The development of temperature and humidity profile retrievals is, therefore, based on clear-sky HIRS pixels. The clearing of cloudy pixels employs a two-tiered approach. The first cloud filtering is based on a simplified cloud detection procedure [31] as in ISCCP [32]. Cloudy pixels are first identified by comparisons of brightness temperature differences both spatially and temporally, among neighboring pixels in days before and after. Though the majority of clouds are cleared by this approach, a small portion of cloudy pixels having small spatial and temporal variations can be misclassified as clear pixels in the process. Often these are semi-permanent stratiform clouds. A second approach using cloud products derived from the Advanced Very High Resolution Radiometer (AVHRR) on board the same satellites, is added to further screen the clear-sky pixels identified in the first approach.

The cloud products are part of the AVHRR Pathfinder Atmospheres-Extended (PATMOS-x) CDR dataset acquired from NOAA’s National Centers for Environmental Information [33]. PATMOS-x generates mapped products with a spatial resolution of 0.1 degrees on a global latitude-longitude grid. Two products, the cloud fraction and cloud probability, from the PATMOS-x dataset are chosen to screen the likelihood of cloud contamination in the HIRS pixels identified in the first step.

An optimization scheme is used to find the optimal thresholds for cloud fraction and cloud probability to identify HIRS pixels that have high likelihood of being cloudy and, therefore, should not be used to derive clear-sky profiles. The HIRS temperature retrievals are compared to co-located RS92 observations in the lower atmosphere at 850 hPa. The optimization scheme finds the thresholds for cloud fraction and cloud probability that result in maximum correlation between HIRS retrievals and RS92 observations and minimum standard deviation of their differences, with the maximum amount of HIRS data retained. The optimal thresholds are found to be 0.5812 for cloud fraction and 0.9638 for cloud probability. The HIRS pixels having either higher values of cloud fraction or cloud probability as determined from PATMOS-x products are considered likely cloud-contaminated and should not be used, as is indicated with associated quality flags. Associated with each observation in the final dataset, a quality flag value is set indicating: (0) clear, (1) possibility of partially cloudiness, (2) likely cloudy, and (3) no cloud fraction/probability information available.

2.5. Retrieval Calibration

The aim of the retrieval algorithm development is to have outputs that are consistent with conventional global observations in terms of minimized systematic differences. Multiple factors may contribute to the systematic differences. The use of model simulations may have model-related biases, and a retrieval scheme may carry retrieval biases. In this study the upper air outputs from model-simulation-based neural networks are calibrated to radiosonde observations from RS92 and GPS RO profiles to achieve a systematically consistent dataset with the conventional measurements. For the upper air temperature and humidity retrievals in the troposphere, the calibration database is comprised of RS92 radiosonde observations. For the temperature outputs in the stratosphere, the retrievals are calibrated to GPS RO profiles.

Pixels with cloud quality flags 0 and 1 (as defined above) were used to co-locate with RS92 and COSMIC2013 data for the calibration scheme development. The co-location criteria are within 0.1 latitude/longitude degree and 1 h at each pressure level. For each pressure level, and the northern and southern hemispheres separately, a multiple linear regression was performed. Additionally, for temperature, individual regressions were done for each 10 degree temperature bin; four bins of specific humidity values, each having approximately the same number of matchups, were created for each pressure level from the calibration dataset. Then, setting:

$$\mathsf{\epsilon }=T_{HIRS}- T_{ind}$$

(1)

where T_HIRS is HIRS retrieval at a standard pressure level, and T_ind is COSMIC2013 or RS92 we regress to find a, b, and c such that:

$$\mathsf{\epsilon }=a+b*T_{HIRS}+c*L_{HIRS}$$

(2)

where L_HIRS is the latitude of the T_HIRS observation. From here, the data was corrected as:

$$T_{HIRS,new}=\left(1-b\right)*T_{HIRS}-c*L_{HIRS}-a$$

(3)

An analogous method was used for specific humidity.

To avoid artifacts along the equator which may arise from the hemisphere-specific regressions, smoothing was applied to all data located between −10 and 10 degrees latitude. The associated latitude value was used as the linear interpolant for the smoothing. After bias calibrations were applied to the specific humidity data, a final quality control check was done to ensure that all specific humidity values were nonnegative and monotonically decreasing with increasing altitude. If a specific humidity value is found not monotonically decreasing with altitude, the value above is adjusted to the value below. No vertical adjustment is applied to temperature profiles. In many places, persistent temperature inversions are well captured in the retrievals. Temperature and specific humidity retrievals over southern high latitudes are not calibrated due to the very limited radiosonde measurements in the region for which no meaningful statistics can be generated. Over the Antarctic, more surface in situ long-term observations are required to establish a climate calibration benchmark.

The left panels in Figure 4 and Figure 5 show the remaining errors in terms of RMSEs of the retrievals globally and for different latitude bands after the calibration is applied. The standard deviation (STD) values for both HIRS retrievals (solid lines) and radiosonde or GPS RO observations (dashed lines) are included in the right panels to provide a context of variability at the standard pressure levels. There are about 3500 HIRS and radiosonde match-ups and about 4000 HIRS and GPS RO match-ups for pressure levels 850 hPa and above. The number of match-ups is less for 1000 hPa (about 1600), as the elevations of some areas are higher and thus there are no measurements available. The STD values of radiosondes and GPS RO profiles are consistently larger than those of HIRS retrievals. This indicates the difference between the footprint area measurement of HIRS and the point measurement from radiosondes or GPS RO, among other factors contributing to the differences. Globally, the RMSEs for the temperatures are in the range of 2–3 °C at the lower tropospheric levels, 1.4–1.6 °C at the mid- and upper-tropospheric levels, and 2.1–2.3 °C at the stratospheric levels. The temperature RMSEs in the low latitudes are, generally, smaller than the global RMSEs. The temperature RMSEs in the troposphere increase toward higher latitude bands as there is larger variability of temperatures in mid- to high-latitudes. For specific humidity, the global RMSEs are 2.4 g/kg at 1000 hPa, and steadily decrease to 1.6 g/kg at 700 hPa, 0.5 g/kg at 500 hPa, and 0.05 g/kg at 300 hPa. The largest RMSEs are from the low latitude band proportional to the high humidity values in the tropics.

The mean bias errors (MBEs) after the calibration procedure between the retrievals and 2011–2012 radiosonde and GPS RO profiles are shown in Figure 6. The biases are defined as HIRS values minus radiosonde or GPS RO values. Globally, the MBEs at all levels are zero for both temperature and specific humidity. This shows that the calibration has accomplished the goal of minimizing biases in a global mean. Very small biases remain in differentiated latitude bands. For temperature, the biases are mostly less than ±0.1 °C for the mid-tropospheric to stratospheric levels. For the rest of the pressure levels the biases are less than ±0.6 °C. For the specific humidity, the tropical band has the smallest biases with values less than ±0.04 g/kg, and over the mid-latitude bands, the biases are less than ±0.15 g/kg. Over high latitudes, the co-located data are noisier and have a smaller number of matches; thus, the resulting biases are larger.

The calibration of the surface air temperature was performed separately using U.S. Climate Reference Network (USCRN) observations. The USCRN network consists of more than 200 stations in the contiguous U.S. and more than twenty in Alaska and Hawaii. One important goal of USCRN is to provide data for satellite calibration and validation [34]. Each USCRN site includes triple redundancy for the primary air temperature and precipitation variables and for soil moisture/temperature. Instrumentation is regularly calibrated to National Institute for Standards and Technology standards. HIRS-derived surface temperatures were co-located with a full year of USCRN observations in the development of a linear regression equation for calibrating the retrievals to USCRN observations [35].

3. Evaluation and Discussion

The results shown in Figure 4, Figure 5 and Figure 6 are based on 2011–2012 data that were used in the development of the calibration scheme. To examine whether the scheme can be applied to other years to yield similar results, comparisons of HIRS retrievals with RS92 and GPS RO profiles for two other years (2013–2014) are carried out, and Figure 7 and Figure 8 show comparison results of temperatures and specific humidity at standard pressure levels, respectively. Globally, the temperature RMSEs range 2.2–3.8 °C for the low-tropospheric levels (700–1000 hPa), 1.6–1.9 °C for mid-tropospheric levels (600–400 hPa), and 2.2–2.5 °C for upper tropospheric and stratospheric levels (300–50 hPa). The RMSEs are mostly smaller for tropics, ranging 2.0–2.9 °C for low-tropospheric levels, 1.4–2.0 for mid-tropospheric levels, and 1.8–2.5 °C for upper tropospheric and stratospheric levels. For specific humidity, the global RMSE is 2.4 g/kg at 1000 hPa, and decreases steadily to 1.6 g/kg at 700 hPa, 0.5 g/kg at 500 hPa, and 0.06 g/kg at 300 hPa as the humidity decreases with height.

The right panels of Figure 7 and Figure 8 show the biases of temperature and specific humidity profiles compared to 2013–2014 radiosonde and GPS RO measurements. For temperature, except at 1000 hPa that has a bias of −1 °C, the global mean biases are all within ±0.3 °C for other levels. For specific humidity, the global mean bias is −0.6 g/kg at 1000 hPa, −0.2 g/kg at 850 hPa, 0.04 g/kg at 700 hPa, and within 0.01 g/kg for the levels above.

The RMSEs and MBEs shown above are based on retrievals from HIRS observation only. Operationally, temperature and humidity profiles are often retrieved from multiple instruments in the TIROS Operational Vertical Sounder (TOVS) or ATOVS suite. The inclusion of multiple sounders in the retrievals is expected to reduce RMSEs. Using the TOVS instruments, which include HIRS, MSU, and SSU, the 3I method had temperature retrieval RMSEs of 1.5–2.25 K for the clear-sky case [3]. A study on assessment of the 3I method in the European Arctic showed that the temperature retrievals had mean biases within ±1.5 K [36]. For the IAPP method, the temperature retrieval RMSEs based on HIRS and AMSU-A for atmospheric temperature profiles at 1-km vertical resolution are 2.5 K for 1000–850 hPa, and about 1.3–2.0 K for layers in 500–70 hPa [4]. For the NOAA operational sounding products which uses HIRS, AMSU-A, and AMSU-B, the temperature mean biases are mostly within ±0.5 K for the region 60° N to 60° S [6]. The retrievals from recent hyperspectral sounders oftentimes show reduced retrieval errors when compared to high-quality sondes [27,37]. Though the inclusion of additional sounding instruments in the retrievals may have smaller retrieval errors, none of the sounding instruments except HIRS has more than 37 years of observation time. The use of different sounding instruments in a long-term time series may introduce significant discontinuity and thus render the retrievals unsuitable for climate applications.

When a HIRS pixel retrieval is compared to a radiosonde observation, there are inherent random differences that are unrelated to retrieval errors. For example, there are differences between the areal measurement of a HIRS footprint and the point measurement of a radiosonde. There are sampling differences when the observations are not made at exact same times. There is inhomogeneity among individual radiosonde measurements. The radiosondes used for comparison in this study are operational measurements that may contain launch-dependent biases. All of these factors can contribute to the differences in the comparison analysis. To show the bias patterns between HIRS retrievals and radiosonde observations, Figure 9 displays an example of the bias histograms taken at 850 hPa for temperature (left panel) and specific humidity (right panel). There is a peak near the zero bias line, and the majority of observations is distributed around the zero bias line. The patterns for other levels are similar with a peak near the zero bias and thin tails at both sides. For the HIRS retrievals in this study, the intended use is to derive monthly means for climate monitoring. With the bias patterns similar to those in Figure 9, the tails of the biases are expected to be smoothed out in monthly means that can lead to significantly-reduced RMSEs in monthly means.

HIRS derived surface air temperatures were evaluated using USCRN observations [35]. For the years that were not used in developing the calibration scheme, the mean biases of HIRS retrievals for each year are mostly in the range of 0.5–1.0 °C, and the RMSEs are ~1.6 °C. Results for individual stations have consistent biases within ±1.5 °C and RMSEs are less than 2 °C for most locations. HIRS surface air temperatures were also compared to the Surface Heat Budget of the Arctic Ocean (SHEBA) observations. The SHEBA project was an interdisciplinary effort with a year-long field experiment in the Beaufort and Chukchi Seas. It produced a collection of atmospheric, oceanographic, and cryospheric measurements taken on a Canadian icebreaker frozen in the arctic ice pack [38]. For measurements co-located within 30 km and 30 min, the RMSE of clear-sky HIRS surface air temperature retrievals over the Arctic Sea was found in the order of 1 °C [39].

The retrievals from this study apply to individual HIRS pixels. The retrievals at pixel observation times should not be directly used in a time series to infer climate variabilities without addressing the diurnal effect. HIRS data from one or two satellites may not have a sufficient diurnal sampling. However, there was an overlapping of four satellites (from NOAA-14, -15, -16, and -17) that lasted several years. The twice-a-day observations from each satellite and the drifting of individual satellites’ orbits during that period provided a good diurnal sampling. A diurnal variation model can be built based on these several years of data, such as the one developed in the ISCCP [7] and the analysis discussed in a study of HIRS channel diurnal cycles [40]. With a diurnal model in place, the HIRS retrievals can be further processed to daily and monthly datasets and used for its intended applications, i.e., to provide a long-term, large-scale atmospheric background for relevant climate product generation and for climate monitoring on a monthly basis.

In summary, Figure 10 provides an overview of the development work presented in Section 2 and Section 3 of this study. In the figure new developments in this study are in the blue font, and related developments outside of the study are in the green font. The algorithm development starts with neural network designs together with a test of CO₂ effect. Cloud clearing is improved by additional information from AVHRR cloud products. Conventional observations are used for bias calibration. The retrieved profiles are evaluated by observations not used in the retrieval scheme development.

4. Conclusions

This article describes the retrieval algorithm development aspect of the project, towards developing a CDR of temperature and humidity profiles from HIRS clear-sky measurements. The development combines a neural network technique with the modeling of long-term CO₂ effect and a calibration scheme based on conventional observations.

During the last several decades of HIRS measurements, the atmospheric CO₂ increased from 335 ppm to over 400 ppm. The increased amount of CO₂ resulted mean negative impacts of −1 K to −2 K for the tropospheric channels and positive impacts of above 0.5 K for stratospheric channels 1 and 2. Such a long-term effect is accounted for in the retrieval algorithm for processing more than 36 years of HIRS observations.

Using data from HIRS longwave channels as well as input of CO₂ and emissivity values with the aid of a radiative transfer model, neural networks are built to derive temperature and specific humidity at standard pressure levels and near the surface. A two-tiered approach is applied to screen HIRS pixels to identify clear-sky pixels to reduce cloud contamination. Radiosonde observations and GPS RO profiles are incorporated to calibrate the upper air retrievals to achieve consistency of retrievals with these observations in the troposphere and in the stratosphere. The development of the calibration scheme accomplishes the goal of unbiased results, globally.

To evaluate the retrieval algorithm, the procedure is applied to two years of HIRS observations that are not used in the algorithm development, and the retrievals are compared with radiosonde observations and GPS RO derived profiles. Except for the lowest level which exhibits larger variability, the global mean bias errors are small and within ±0.3 °C for temperature and within ±0.2 g/kg for specific humidity.

The focus of this article is on the algorithm development of the upper air temperature and specific humidity profile retrievals from HIRS as one part of a CDR dataset development project. Other parts of the project include inter-satellite calibration, validation of retrievals from all satellites using independent high-quality global coverage datasets, and the evaluation of surface temperature and humidity retrievals. The integration of all these parts constitutes an effort to build a homogenized long-term atmospheric temperature and humidity dataset based on HIRS observations since 1978 for climate studies.