Automated Segmentation of Tropical Cyclone Clouds in Geostationary Infrared Images
Automated Segmentation of Tropical Cyclone Clouds in Geostationary Infrared Images
Automated Segmentation of Tropical Cyclone Clouds in Geostationary Infrared Images
This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/LGRS.2024.3358733
Abstract—We demonstrate that a convolutional neural network was used to identify tropical cyclone clouds from geostationary
(CNN) based on the U-Net architecture can be used to create infrared brightness temperature images[2], [3]. Motivated by
a cloud mask data set that accurately identifies the clouds earlier work on more general mesoscale convective systems
associated with tropical cyclones (TCs). The CNN can be trained
using a single year of cloud masks produced by an earlier first- [4], we developed a spatiotemporal algorithm that tracked the
principles algorithm, and the results are insensitive to the specific cloud pixels that could be attributed to a particular storm. The
year of training data used. These masks were originally created in net upwelling radiation attributable to TC clouds was then
order to compute the upwelling radiation due to TC clouds, and computed from upwelling radiation data. We first examined
we show that the predicted masks result in both pixel areas and all TCs globally in 2016 by comparing with the clear sky
radiation calculations that are nearly identical to those computed
using the earlier masks. condition[3], and we later extended that work by updating
the labeling algorithm and examining all TCs globally from
Index Terms—Satellite Remote Sensing, Supervised Segmenta- 2000 – 2020 [5]. Our findings were that TC clouds have a net
tion, Tropical Cyclones
cooling effect on average, but that this cooling effect depended
on both location and season of any particular storm.
I. I NTRODUCTION Our previous work has two shortcomings . First, the
cloud pixel mask only considers specific pixels identified
T ROPICAL cyclones (TCs) are mesoscale convective sys-
tems that form between late Spring and early Autumn in
six primary basins around the globe (North Atlantic, Eastern
as cold TC clouds and excludes clear and very low-level
cloud pixels within the tropical cyclone circulation. In addition
to influencing the area covered by TC clouds, a tropical
and Western North Pacific, South Pacific, North Indian, and
cyclone also influences regional weather and even the global
South Indian Ocean basins). Recent research has seen signif-
weather pattern [6], [7]. Thus, the cloud pixel mask likely
icant interest in the relationship between TCs and climate,
misses significant areas whose cloud cover (or lack thereof)
and that research has largely focused on the future frequency
is impacted by TCs. We are developing mask identification
and intensity of TCs in a warming climate. However, there is
methods that will allow better labelling of cyclone vs non-
relatively little research that explores the reverse, namely: How
cyclone areas regardless of cloud temperature. Identifying
does TC activity affect the prevailing climatic conditions?
this region currently requires an expert observer to create a
One mechanism by which TCs impact the climate system is
mask for each image of each storm (comprising hundreds
through their contribution to the Earth’s energy balance (EEB).
of thousands of images in our 20-year data set). The second
The main driving source of energy for the climate system
problem is that our existing algorithm is extremely slow and
is solar radiation. The net radiation flux at the top of the
requires supervision and manual intervention. It is therefore
atmosphere, which is the difference between the downwelling
a daunting task to try to manually create a full 20-year
flux incident from the sun and the upwelling flux reflected
record of TC masks to improve upon our current masks that
from the Earth in the short wave (¡ 3 µm) and emitted in the
only highlight clouds. The goal of the present paper is to
long wave (¿3 µm), is currently negative at about 0.3 PW. This
demonstrate that a stable CNN-based system can be trained
means that the system is trapping energy and hence the current
using a relatively small set of expert-created training data to
warming conditions [1]. If the change in the clouds caused by
reproduce the original 20-year dataset of TC cloud masks
TCs is significant enough to impact the average Earth albedo,
in [5]. We demonstrate that a method based on the U-net
then it can potentially act as a feedback mechanism (positive
convolutional neural network (CNN) architecture[8] can be
or negative) on climate.
trained to create such a dataset with only a single year of
Accurate identification of the regions covered by TCs at
manually labeled training data. While the study presented here
any given time is necessary to study the contribution of TCs
specifically reproduces the 20-year cloud mask dataset we have
to the Earth’s radiation budget. Previously our group presented
reported previously [5], we demonstrate that the masks created
a supervised, first-principles cloud segmentation algorithm that
by the CNN system produce essentially equivalent results for
J. May, M Harandi, L. Hu and J. S. Tyo are with the Electrical and the radiation calculation. This gives us confidence that in the
Computer Systems Engineering Department, and E. Ritchie is with the future we can create TC area masks that include both clear
Civil Engineering and the Earth, Atmospheric, and Environmental Sciences and cloudy pixels using no more than one year’s worth of
Departments, Monash University, Clayton, Australia. This work was supported
by the Australian Research Council under award DP210103203. The authors manually labeled data and then use that training set to train a
acknowledge Himashi Peiris for her assistance in tuning the CNN. CNN system to produce a full 20-year dataset.
Authorized licensed use limited to: Nanjing University of Information Science and Technology. Downloaded on April 01,2024 at 01:29:14 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
This article has been accepted for publication in IEEE Geoscience and Remote Sensing Letters. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/LGRS.2024.3358733
Authorized licensed use limited to: Nanjing University of Information Science and Technology. Downloaded on April 01,2024 at 01:29:14 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
This article has been accepted for publication in IEEE Geoscience and Remote Sensing Letters. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/LGRS.2024.3358733
The model was trained using all TC data globally for each
year between 2001 – 2005 one year at a time. The models V. C OMPUTATION OF U PWELLING R ADIATION
were tested on all five years. Figure 4 shows the training and The ultimate goal is to assess whether cloud masks created
testing loss using the 2005 data as the testing data as function using the CNN are useful in computing the net upwelling
of epoch on single-year training data as well as using all years radiation from TCs [5]. The upwelling radiation is com-
2001 – 2004 to train. With the exception of 2001, the year-to- puted by matching the cloud mask pixels obtained from
year performance is nearly identical. Both training and testing GPM MERGIR data to upwelling radiation in the CERES
loss decrease rapidly through the fourth epoch, and then the dataset and comparing it with background climatology. The
testing loss continues to decrease but more slowly than the details of this computation are presented in our earlier work
Authorized licensed use limited to: Nanjing University of Information Science and Technology. Downloaded on April 01,2024 at 01:29:14 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
This article has been accepted for publication in IEEE Geoscience and Remote Sensing Letters. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/LGRS.2024.3358733
15th
50th
85th
Fig. 8. TC pixel number and net radiation for 44 TCs and their average
diurnal cycle in 2005 computed using the cloud masks predicted by each of
the models in Fig. 6.
predicts cloud size and net radiation for all 44 TCs in 2005
regardless of training set, and the diurnal cycle of net radiation
is almost identical to that computed using the ground truth
cloud masks. Even though at some times there are differences
of as much as 10% in the pixel area of the predicted masks,
Fig. 6. TC labels and predictions for Dennis at 2130UTC on 6th July 2005.
“True” is the mask computed using our earlier algorithm [3]. Images labeled
the impact on the radiation calculation is smaller and does not
2001 – 2004 are masks computed using each single year as the training set change the conclusions of our earlier study that TC clouds
in the U-Net model. “All” is the mask computed using all four years 2001 – are net cooling. The integrated results for the 2005 season are
2004 as training data.
given in Table I compared with the earlier computed values.
The new model has the best result when the entire 2001 - 2004
data set is used to train, but the results are not significantly
[3], [5]. Fig. 6 compares the earlier ground truth cloud different from the models trained on individual years. This
masks for TC Dennis, a storm in the North Atlantic basin gives us high confidence that cloud masks can be computed
at 2130 UTC on July 6, 2005, at the resolution of the CERES from any single year of training data.
dataset. The number of pixels and the net upwelling radiation
computed for each of these masks are plotted in Fig. 7.
Compared with results using our earlier algorithm[3], [5], VI. C ONCLUSIONS
Figs. 6 and 7 indicate that CNN correctly captures the cloud We demonstrate that a U-Net CNN architecture trained
structure, location, net radiation, and time variance of TC only on a single year of data can predict TC cloud masks
Dennis throughout its lifetime. Fig. 8 shows the aggregated that are nearly identical to our earlier supervised method [3].
results for the entire 2005 TC data set using the various Subjective examination indicates that the new masks may
training sets indicated in Fig. 6. The U-Net model accurately better capture the TC clouds than the old ones. These masks
Authorized licensed use limited to: Nanjing University of Information Science and Technology. Downloaded on April 01,2024 at 01:29:14 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
This article has been accepted for publication in IEEE Geoscience and Remote Sensing Letters. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/LGRS.2024.3358733
Authorized licensed use limited to: Nanjing University of Information Science and Technology. Downloaded on April 01,2024 at 01:29:14 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.