Cyc 4
Cyc 4
ABSTRACT
This study first utilizes four well-performing pre-trained convolutional neural networks (CNNs) to gauge the in-
tensity of tropical cyclones (TCs) using geostationary satellite infrared (IR) imagery. The models are trained and
tested on TC cases spanning from 2004 to 2022 over the western North Pacific Ocean. To enhance the models per-
formance, various techniques are employed, including fine-tuning the original CNN models, introducing rotation aug-
mentation to the initial dataset, temporal enhancement via sequential imagery, integrating auxiliary physical informa-
tion, and adjusting hyperparameters. An optimized CNN model, i.e., visual geometry group network (VGGNet), for
TC intensity estimation is ultimately obtained. When applied to the test data, the model achieves a relatively low
mean absolute error (MAE) of 4.05 m s−1. To improve the interpretability of the model, the SmoothGrad combined
with the Integrated Gradients approach is employed. The analyses reveal that the VGGNet model places significant
emphasis on the distinct inner core region of a TC when estimating its intensity. Additionally, it partly takes into ac-
count the configuration of cloud systems as input features for the model, aligning well with meteorological principles.
The several improvements made to this model’s performance offer valuable insights for enhancing TC intensity fore-
casts through deep learning.
Key words: tropical cyclone, intensity, deep learning, satellite imagery
Citation: Yang, W., J. F. Fei, X. G. Huang, et al., 2024: Enhancing tropical cyclone intensity estimation from satel-
lite imagery through deep learning techniques. J. Meteor. Res., 38(4), 652–663, doi: 10.1007/s13351-024-
3186-y.
due to reasons such as the less definitive nature of TC from constrained training datasets, and to reinforce the
genesis and intensity compared to its location, and the in- physical plausibility of models trained via DL, the integ-
tricate physical processes, rendering them challenging to ration of auxiliary physical knowledge becomes crucial.
depict accurately through statistical models or dynamic In light of the potential constraints that these challenges
equations. impose on the effectiveness of DL models, this paper is
In recent years, with the extensive application of artifi- dedicated to addressing these issues to ensure optimized
cial intelligence, there has been growing attention to- precision in TC intensity prediction.
wards how to apply deep learning (DL) techniques to TC In this investigation, we design a CNN model to es-
intensity research due to its advantages in image recogni- timate TC intensity over the western North Pacific using
tion (Li et al., 2020; Zhuo and Tan, 2021). A multi- satellite imageries based on the above-mentioned im-
layered deep convolutional neural network (CNN) is ini- provements. The remainder of the paper is organized as
tially utilized for classifying TCs by intensity. Using only follows. Section 2 introduces the dataset details and pre-
remote sensing cloud images, the model achieves a su- processing methodologies. Section 3 illustrates the archi-
perior result with a lower root-mean-square error tecture of the CNN model, its subsequent enhancements,
(RMSE) value (Pradhan et al., 2018). TC intensity is then and insights into its interpretability. Finally, our conclu-
estimated by using long short-term memory (LSTM) and sions are consolidated in Section 4.
deep neural networks (DNN), enhancing the strength es-
timation for catastrophic weather events (Zahera et al., 2. Data and methods
2019). A multi-model fusion method based on CNN re-
gression is introduced, showcasing the potential of DL in 2.1 Geostationary satellite imagery of TCs
meteorological research (Chen et al., 2018). The ability The observational data obtained from satellite im-
of DL to discern patterns from data makes it especially agery is sourced from the earth’s geostationary meteoro-
suitable for addressing problems with unclear physical logical Himawari series of satellites. Launched in 1977
mechanisms, such as TC changes. Indeed, the explosive by the Japan Meteorological Agency (JMA), the Hi-
growth of satellite, observational, and reanalysis data of- mawari series represents the first geostationary meteoro-
fers ample opportunities for DL in TC forecasting. logical satellites. Specifically, the data for 2004 are col-
However, leveraging satellite imagery and DL meth- lected from Geostationary Operational Environmental
ods to estimate TC intensity presents distinct challenges. Satellite 9 (GOES-9), and for 2005–2015 are derived
(1) As a type of extreme weather, TCs still present obser- from two Multi-functional Transport SATellites (MT-
vational challenges that result in limited datasets despite SATs), MTSAT-1R and MTSAT-2. Beginning in 2016,
the explosive growth of data in recent years. Due to the the data source switches to the Himawari-8 satellite. This
limitations imposed by the constrained datasets, it is ad- new generation satellite in the Himawari series is
visable to utilize transfer learning in combination with launched in October 2014 and becomes operational in July
fine-tuning methodology (see Section 2.4 for details). 2015 (Bessho et al., 2016; Takeuchi, 2018).
This approach contributes to a more effective training These aforementioned satellites primarily provide IR
process, avoiding the need to initiate a new one using cloud imagery data. A key advantage of this form of data
these constrained datasets (Ramdan et al., 2020). (2) Fur- is its round-the-clock availability, regardless of day or
thermore, TC data significantly deviate from conventional night. This continual data stream facilitates a compre-
image recognition (Pradhan et al., 2018). Due to temporal hensive analysis of the entire lifecycle of TCs. Visually,
and spatial shifts, both the configuration and structural the structure of TCs, accurately captured by these data-
dynamics of a TC undergo transformations. This ever- sets, is highly correlated with the intensity of the con-
changing information is not readily discernible from a vective systems within the TCs. Consequently, utilizing a
single IR image, adding complexity to intensity estima- deep learning-based model for computer vision tasks to
tions based on satellite imagery (Chen et al., 2020; Wang assist in convection intensity estimation appears both
et al., 2022). Utilizing temporal sequential images could practical and beneficial.
potentially offer enriched contextual insights, amplifying
the capacity to grasp the intricate spatial and temporal 2.2 TC data
convection patterns. (3) Additionally, the DL techniques Compared to other best track records, the Joint
resemble standard image recognition algorithms from Typhoon Warning Center (JTWC) dataset shows a higher
computer vision. To provide DL models with meteorolo- consistency with numerical results and effectively replic-
gical insights that might be missing or difficult to obtain ates the trends of TC intensity enhancement (Webster et
654 Journal of Meteorological Research Volume 38
al., 2005; Wu and Zhao, 2012). Thus, this paper uses the numerous atmospheric science issues, including the de-
best track dataset from 2004 to 2022 over the western tection of various weather phenomena in observations
North Pacific basin obtained from the JTWC. The data and climate models (Liu et al., 2016; Lagerquist et al.,
are available each 6-h period, including detailed TC tem- 2019; McGovern et al., 2019), the representation of sub-
poral and geographical information, such as time, longit- grid-scale processes (Bolton and Zanna, 2019; Han et al.,
ude, latitude, minimum sea level pressure, and maximum 2020), and the enhancement of weather predictions and
wind speed. climate projections (Ham et al., 2019; Sønderby et al.,
The analysis solely considers named TCs with life- 2020; Weyn et al., 2020). In light of the efficiencies and
time peak intensities reaching at least 17.85 m s−1. benefits offered by CNNs, we elect to utilize them for in-
Moreover, extratropical cyclones and TCs after landfall dicating the TC intensity from IR satellite cloud imagery.
are excluded from the study. To align with the time span A CNN comprises three key layers: convolutional,
of the previously mentioned IR cloud images, a collec- pooling, and dense layers. Convolutional layers contain
tion of 3335 TC records is extracted. multiple filters, which enable the extraction of various
features from the original input pattern. Pooling layers,
2.3 Atmospheric and oceanic data sources typically inserted between successive convolutional lay-
The fifth-generation ECMWF Re-Analysis (ERA5) ers, help preserve key features while concurrently de-
data are utilized in this study. The data are employed to creasing the dimensionality of the data. They serve to
compute the vertical wind shear (VWS) and relative hu- create invariance to minor shifts and distortions, and in-
midity (RH) for the western North Pacific. The dataset crease model generalization while reducing over-fitting
has a spatial resolution of 0.25° × 0.25° and includes 37 (LeCun et al., 2015). Strategically designed convolutional
vertical layers spanning from 1000 to 1 hPa. It features a and pooling layers are capable of extracting well-struc-
temporal resolution of 6 h. For ocean heat content tured features from the raw input data. Dense layers in-
(OHC), we access the Global Ocean Reanalysis dataset tegrate the outcomes of convolution and pooling. CNN is
with a 1/12° horizontal resolution (GLORYS12V1) dis- highly suitable for tasks involving image processing and
tributed across 50 vertical layers. This dataset furnishes a pattern recognition, particularly when dealing with im-
comprehensive three-dimensional portrayal of oceanic ages that possess translational, rotational, and scale in-
parameters. Produced by Mercator Ocean, it is available variance. Given that TC images are produced under 5 ro-
via the Copernicus Marine Environment Monitoring Ser- tational angles that are centered on the storm center, and
vice (CMEMS). Both the atmospheric and oceanic data- typically present well-organized TC structures with dif-
sets extend consistently from January 2004 to December ferent shapes and sizes, it is feasible to assume the afore-
2022. The details for calculating these atmospheric and mentioned three invariances.
oceanic variables are listed in Table 1, and the justifica- In meteorological forecasting, it is imperative to note
tion is extensively discussed in Kaplan et al. (2010). The that one cannot blindly apply a specific model, inclusive
calculation of OHC is detailed in the Appendix. of its architecture and hyperparameters, to varied applica-
tion scenarios. Experimentation is pivotal to ascertain the
2.4 CNN configuration best fit for specific contexts. Given the limitation of data,
In the 1960s, Hubel and Wiesel (1962) discovered which is not extensive enough to train a sophisticated
unique neuronal connectivity patterns in a cat’s visual model from scratch, a viable solution is to leverage mod-
cortex, leading to the conception of CNN. Since their els pre-trained on larger datasets, and subsequently fine-
victory in the 2012 ImageNet Large Scale Visual Recog- tune them to cater to our tasks (Pan and Yang, 2010).
nition Challenge (ILSVRC; Krizhevsky et al., 2017) have This transfer learning combined with fine-tuning meth-
established CNN as the principal algorithm in image pro- odology not only abbreviates the training period but, as
cessing. Due to its remarkable feature handling and ana- some studies suggest, can be more effective than starting
lytic capabilities, CNN has been successfully applied to a new training process (Ramdan et al., 2020).
In our approach to fine-tuning, the models undergo
Table 1. Details for calculating the variables as the auxiliary physical updates during training. These pre-trained models serve
information as the foundation, negating the need for initializing with
Predictor Definition random weights. Fine-tuning can be executed across all
RH (%) 850–700-hPa relative humidity within a 200–800-km radius
VWS 850–200-hPa vertical wind shear within a 500-km radius after layers or merely the upper ones. The latter is more
(m s−1) vortex removal favored (Chollet, 2018) since the primary layers in a con-
OHC (J) Ocean heat content volutional network encode versatile, reusable features,
AUGUST 2024 Yang, W., J. F. Fei, X. G. Huang, et al. 655
whereas the terminal layers capture task-specific fea- The data are further divided into training, validation,
tures. Adjusting these task-specific features proves more and test data. The training and validation data span from
efficient. Conversely, adjusting every layer may lead to 2004 to 2019, while the test dataset is taken from 2020 to
overfitting due to the overwhelming number of paramet- 2022. To enhance model accuracy, we employ the k-fold
ers involved (Chollet, 2018). In our endeavor, the fine- cross-validation method proposed by Geisser (1975) to
tuning is carried out on different layers to evaluate and divide the training and validation sets. We split the data-
ascertain the most promising outcomes. set into 5 equal parts, with each part used once as the val-
To determine the ideal CNN architecture for our idation set and the remaining parts used as the training
project from the myriad of available configurations, we set. This process is repeated 5 times, providing a more
assessed four outstanding models suitable for a transfer accurate estimation of the model performance. The
learning approach. These are VGGNet (Simonyan and sample numbers for the training, validation, and test
Zisserman, 2015), GoogLeNet (Szegedy et al., 2016), datasets prior to augmentation are shown in Table 2.
ResNet (He et al., 2016), and Xception (Chollet, 2017).
All these CNN architectures have been trained on extens- 3. Results
ive image datasets and have achieved first place in the
ImageNet image recognition competition, even exceed- 3.1 Selection of the model architecture
ing human-level image recognition. Early stopping is We first determine an optimal CNN architecture men-
utilized to mitigate potential overfitting. A smaller learn- tioned before as the base model (Simonyan and Zisser-
ing rate is deemed necessary to preserve previously man, 2015; He et al., 2016; Szegedy et al., 2016; Chollet,
learned features because transfer learning methods are 2017; Krizhevsky et al., 2017). Each model is adjusted
used. In addition, the learning rate is reduced by 50% if and trained to minimize the MAE between the network’s
no improvement is observed over five consecutive predictions and the TC intensity derived from JTWC.
epochs. As a result, a relatively small learning rate of The outcomes are presented in Table 3.
0.001 is chosen to preserve pre-trained knowledge, en- It can be observed that the configuration yielding the
sure stable convergence, and reduce the risk of overfit- best performance is the VGGNet model, and hence, it is
ting. To best serve our specific requirements, we further chosen as the backbone for our application. This adjus-
fine-tune the models deemed most appropriate. The pro- ted pre-trained VGGNet model starts with five stacks of
cess of tweaking hyperparameters is pivotal in refining convolutional layers, each utilizing a 3 × 3 receptive
model performance. Ultimately, we will analyze the ef- field, with max pooling layers strategically interspersed
fects of the hyperparameters. between these stacks to retain paramount meteorological
To quantify the discrepancy between predicted and ac- features while simultaneously reducing computational
tual values of our CNN model, we adopted mean abso- demands. Following the last pooling layer, the model in-
lute error (MAE) as loss functions. MAE is the average cludes three fully connected (FC) layers. The original
of the absolute differences between predicted and actual softmax output layer is replaced to suit regression tasks.
values. Its magnitude reflects the degree to which the The Rectified Linear Unit (ReLU) activation function is
predicted values deviate from the actual ones. It has a integrated within the model, accelerating convergence
smaller influence from outliers. Lower MAE indicates a and enhancing its ability to decipher complex atmospheric
better model prediction performance.
Table 2. Sample numbers of the training, validation, and test data.
2.5 Data processing for CNN models Due to the k-fold cross-validation method for splitting the training and
We first normalize the data, which is beneficial for en- validation datasets, they are considered as a whole
Training and validation data Test data
hancing the model’s convergence rate and performance. Sample number 3619 536
The IR cloud images are consistently cropped down to a
subimage of 256 × 256 pixels, with the side lengths of
Table 3. Performance of different CNN frameworks. The results are
1255 km × 1255 km centered on the eye of the TC. The derived from experiments on the test dataset. The model performance
total sample records of TC from 19 yr (2004–2022) are is gauged by MAE, with a lower MAE signifying superior perform-
4155. Training a deep learning model typically demands ance. The minimum error is highlighted in bold typeface
a substantial volume of data. To enlarge our initial data- CNN framework MAE (m s−1)
VGGNet 6.34
set, we implement data augmentation techniques, spe- GoogLeNet 10.34
cifically employing rotations of ±45 degrees to enrich the ResNet 8.58
diversity of our image samples. Xception 7.09
656 Journal of Meteorological Research Volume 38
patterns. Batch normalization has been introduced on top learning its features. But as the number of unfrozen lay-
of the pre-trained model to optimize the training process. ers increases, there is a slight rise in the MAE. There is a
Furthermore, a flatten layer is introduced, preserving all possibility that with too few layers frozen, most of the
the intricate information in the feature maps while offer- model parameters get updated, leading to overfitting des-
ing greater parameter capacity. Subsequently, the model pite employing early stopping and dropout layers. This is
employs multiple fully connected dense layers and dro- especially possible when the training dataset is small,
pout layers, specifically designed for regression-based resulting in poorer prediction performance on test data.
predictions to reduce overfitting and improve the model’s Unfreezing and training only a few layers can reduce this
generalization ability on training data. Considering the risk, allowing the model to be finely tuned for the speci-
dynamic nature of meteorological data, early stopping, fic task.
L2 regularization, and dropout layers are implemented to In conclusion, through numerous experiments unfreez-
avoid overfitting. By leveraging modern CNN strategies ing different layers, a relatively suitable fine-tuning
within a crafted architecture, our model is proficient in scheme has been identified, allowing the model to both
analyzing meteorological data, identifying vital atmo- leverage the pre-trained weights and adapt to our task.
spheric characteristics, and ensuring higher predictive ac- Note that when unfreezing 7 layers, our model more ef-
curacy in regression tasks. From Table 3, the VGGNet fectively extracts and reorders potential features of TC
architecture demonstrates the highest performance in TC intensity, obtaining an MAE as low as 5.98 m s−1, which
intensity estimations among the four models examined. is considered to be a satisfactory and comparable result.
However, it is worth noting that this model may not be 3.2.2 Temporal enhancement by sequential images
universally applicable. Fine-tuning and hyperparameter Based on the above results, it is evident that the
adjustments may be imperative. VGGNet architecture performs well in capturing the
complex relationship between the IR cloud imagery and
3.2 Development of the model TC intensity data. However, the current algorithm is
3.2.1 Fine-tuned VGGNet architecture mostly similar to image recognition methods used in
Based on the optimal CNN architecture, VGGNet, we computer science and does not include specific weather-
utilize the fine-tuning method, which freezes the deeper related knowledge. A series of IR cloud images taken
convolutional layers and adapts the head of the network over time can provide crucial dynamic information on
to the particular regression problem. The experimental TC evolution, such as the formation, growth, relocation,
results are presented in Table 4. The first column in and decay of convective cloud clusters, which are intric-
Table 4 indicates the number of unfrozen layers in the ately linked to TC intensity. Such dynamic information is
VGGNet architecture. It can be illustrated that when only not discernible from a standalone IR image, making
a few layers are unfrozen, the model performance on the time-sequential data more illuminating for understand-
test dataset for estimating TC intensity is inferior. As the ing TC intensity characteristics. Thus, temporal enhance-
number of unfrozen layers increases, the MAE generally ment is introduced into our VGGNet model to provide
decreases. Unfreezing and training the model’s bottom additional contextual information and enhance its ability
layers allows the model to adjust those complex features to capture the complex spatial and temporal distributions
that are closer to the specific task while keeping the gen- of convection, which is beneficial for indicating TC in-
eric features unchanged. Training only a limited number tensity. Moreover, employing multiple time-sequential
of layers might potentially constrain the model’s adapt- images mitigates the risk of missing satellite observa-
ability, preventing it from effectively capturing and tions at any specific time. Given computational con-
straints and the model’s potential utility for practical ap-
Table 4. Performance of the fine-tuned VGGNet architecture with plications, coupled with the aforementioned benefits, we
different bottom layers left unfrozen. The minimum error is high- opt to use a sequence of satellite images spanning 12 h
lighted in bold typeface prior to the current time at 6-h intervals. The results are
Unfrozen layer MAE (m s−1) detailed in Table 5.
1 6.45
2 6.52 As evidenced by Table 5, when utilizing a continuous
3 6.46 time series from −12 h to the current, our model success-
4 6.48 fully captures the feature of TC intensity with an MAE of
5 6.12
7 5.98 5.48 m s−1. The test set error shows an 8% improvement
10 6.43 relative to the experiments without sequential images,
15 6.58 thereby proving the effectiveness of using temporal
AUGUST 2024 Yang, W., J. F. Fei, X. G. Huang, et al. 657
Table 5. Performance of the VGGNet architecture with successive IR translation speed of the TC. As the speed decreases, the
image sequences of different time spans. The minimum error is high- interaction begins to become more important, having a
lighted in bold typeface
significant impact on the intensity of the TC (Price,
Time span MAE (m s−1)
−6 h to current 5.77 1981; Lloyd and Vecchi, 2011). Besides, as pointed out
−12 h to current 5.48 by Zhao and Chan (2017), a TC’s potential for strength-
ening escalates with its speed, since rapid movement can
factors for elevating the model’s performance. Owing to counterbalance the negative effects of ocean cooling.
recent advancements in satellite technology, the Hi- This suggests that there might be a subtle yet crucial in-
mawari-8 satellite is now equipped to provide data with teraction between OHC and the TC translation speed. It
enhanced spatiotemporal resolution. This technological is not merely the individual effects of OHC and TC
progress paves the way for the future integration of higher- speed that matter; their combined interplay also plays a
resolution continuous cloud imagery, offering the poten- significant role in influencing TC intensity. Additionally,
tial to further improve the performance of the model. the TC duration is also a key factor.
3.2.3 Incorporating physical information Considering the aforementioned background and in-
Based on the aforementioned experimental results, we sights, we further explore the effect of integrating more
have discerned that after a series of optimization pro- related physical information and their different combina-
cesses, given a set of satellite IR cloud images, we can tions into the VGGNet model. The following parameters
employ the VGGNet model to estimate TC intensity with are particularly focused on: VWS, RH, OHC, TC transla-
considerable accuracy. Currently, in the field of TC re- tion speed, and duration. The main results are shown in
search, a significant amount of physical information re- Table 6.
garding TC intensity has been explicitly collected and From the results, it can be observed that upon the ad-
validated. However, this has not been fully integrated into dition of physical information, all parameters, except for
deep learning models. Given that the physical know- RH, reduce the MAE value. This may be attributed to the
ledge of TCs could play a critical role in deep learning fact that the TCs in this study primarily occur over the
algorithms for intensity estimation, it can furnish the open ocean, where RH is generally high. Therefore, in-
model with information that may be non-existent or chal- troducing this particular parameter alone does not
lenging to extract from the restricted training data. This markedly enhance the model’s predictive accuracy.
not only facilitates improved training efficiency and en- However, the inclusion of other physical information
hanced model generalization but also fortifies the physical aids in bolstering the model performance, albeit to vary-
plausibility of the model. While the incorporation of se- ing extents. To specify, VWS shows a modest improve-
quential images has already integrated certain weather- ment to the model, with only about a 4% enhancement
related knowledge, to elevate the accuracy in estimating compared to the VGGNet model without any additional
TC intensity, there is a prevailing necessity for the cur- physical information (control model). This may be attrib-
rent model to assimilate additional physical information. uted to the inherent sophistication of the VGGNet model,
Such an integration will be the key point of our sub-
which is adept at discerning intricate features correlated
sequent research.
with VWS from satellite IR imagery, particularly from
The evolution of TC intensity is a complex process,
the distribution of convective cloud clusters. In terms of
primarily driven by three key factors: environmental con-
ditions, internal structure and dynamics of TC, and inter- Table 6. Performance of the VGGNet architecture with different
action between TC and ocean (Marks et al., 1998; combinations of the physical information. The minimum error is high-
Emanuel, 1999). Among these factors, specific paramet- lighted in bold typeface
ers like VWS, HR, and OHC stand out not only for their Physical information MAE (m s−1)
RH 5.56
strong association with the intensity of the TC but also OHC 5.40
for their capability to enhance or supplement the features Translation speed 5.28
that the VGGNet model extracts from satellite cloud im- VWS 5.26
OHC + translation speed + RH 5.14
ages (Kaplan and DeMaria, 2003; Kaplan et al., 2010, OHC + translation speed 5.10
2015; Shu et al., 2012). The intensification of a TC pre- Duration 5.03
dominantly hinges on the energy that it derives from All combinations without VWS 4.87
All combinations 4.80
warm sea waters. Thus, a higher OHC tends to favor TC OHC + translation speed + VWS 4.77
strengthening (Riehl, 1950). Nevertheless, the interplay All combinations without RH 4.73
between TC and ocean is additionally influenced by the OHC + translation speed + duration 4.65
658 Journal of Meteorological Research Volume 38
put. The combination of SmoothGrad and Integrated meteorological principles. In cloud imagery, the arrange-
Gradients takes advantage of the noise-reduction capabil- ment of cloud systems can reflect the energy and mois-
ities of the former and the path-based attribution of the ture influx from the ocean, and it can also potentially
latter, yielding saliency maps that are clearer, more con- provide insights into VWS. Moreover, the decentration
sistent, and more interpretable than using either method between convections and the TC center can indicate
alone. The saliency map illustrates the influence of each VWS. Thus, the configuration and strength of the cloud
pixel in the image on the final intensity estimation, based systems serve as pivotal indicators of modulating TC in-
on their relative gradients to the output. Its value can be tensity. Disorganized, fragmented, or dispersed cloud
positive or negative, indicating a positive or negative in- patterns often hint at a weakening TC, while cohesive
fluence. The larger the absolute value in the map, the and well-structured systems suggest strengthening condi-
more significant the contribution of the corresponding tions. The saliency maps further enhance our understand-
pixel in the cloud image to the final VGGNet model’s ing. The results suggest that while our VGGNet model’s
TC intensity estimate. prediction of TC intensity from IR cloud images is data-
Figure 2a showcases the formation phase of TC Lion- driven, the deep learning model indeed captures physic-
rock. Figure 2b marks the point where Lionrock nears the ally meaningful configurations of the TC cloud systems.
end of its intensification, during which it maintains a ro- Thus, employing deep learning methods might be benefi-
bust speed of around 43.35 m s−1 for an extended dura- cial for probing more intricate TC mechanisms, and fur-
tion. By the time in Fig. 2c, the TC begins its weakening ther research along with more advanced and appropriate
phase, and in Fig. 2d, Lionrock is significantly weakened interpretation techniques is required.
and on the brink of dissipating. Notably, the intensity
predicted by the VGGNet model closely matches the data 4. Conclusions
from the best track. It can be illustrated that the saliency
maps reveal the comprehensive structure of a TC, high- TCs are one of the most severe disasters. Estimating
lighting a prominent inner core at the image’s center. their intensity is crucial for forecasters and institutions to
This model skillfully captures the cloud system distribu- issue disaster warnings. This paper develops a unique
tions from the saliency values shown in Fig. 2. It under- CNN architecture to extract TC intensity values from the
scores that the TC intensity estimation is primarily driven geostationary satellite IR imagery over the western North
by the TC inner core and further shaped by the combina- Pacific Ocean basin during 2004–2022. The methodo-
tion of outer spiral rainbands and convective systems. logy offers a dependable foundation for TC intensity pre-
The interpretation of the saliency maps is grounded in diction systems. We incorporate the pre-trained weights
Original image (28.05) Saliency map (Pred: 30.98) Original image (43.35) Saliency map (Pred: 42.07)
(a) (b)
1.00
0.75
0.50
Saliency value
0.25
Original image (43.35) Saliency map (Pred: 45.45) Original image (33.15) Saliency map (Pred: 28.97) 0
(c) (d) −0.25
−0.50
−0.75
−1.00
Fig. 2. IR satellite cloud images (the first and third columns) and their corresponding saliency maps (the second and fourth columns) for TC Li-
onrock (1610) at (a) 0600 UTC 23 August 2016, (b) 0000 UTC 25 August 2016, (c) 0000 UTC 29 August 2016, and (d) 0600 UTC 30 August
2016. The TC intensity (m s−1) indicated in the parentheses following the original images is from the best track, while the predicted TC intensity
(m s−1) is displayed subsequent to the saliency maps.
AUGUST 2024 Yang, W., J. F. Fei, X. G. Huang, et al. 661
of four prominent CNN architectures and subsequently tab=overview. The GLORYS12V1 is acquired from http://
fine-tune them to meet our specific task. Techniques such marine.copernicus.eu/.
as the rotation augmentation of the initial datasets, tem-
poral enhancement via sequential imagery, incorporating Appendix
auxiliary physical information, and hyperparameter tun-
ing are employed, resulting in an optimized VGGNet Leipper and Volgenau (1972) introduced a new
model. The findings demonstrate a superior representa- concept, termed OHC. They point out that using SST to
describe the latent heat flux is not accurate. OHC is a
tion of TC intensity, as inferred from IR imagery, obtain-
more accurate and superior factor. The formula for calcu-
ing a minimum MAE of 4.05 m s−1.
lating OHC is:
To further enhance the interpretability of the model, w D26
we combine the SmoothGrad and Integrated Gradients OHC = c p ρ [T (z) − 26] dz, (A1)
methods to obtain a clearer, more coherent, and easily in- 0
terpretable saliency map. These maps reveal the input where c p is the specific heat at constant pressure, and ρ
features considered by the model when predicting TC in- is the seawater density. Its value is usually set as a con-
tensity. The saliency maps further unveil that the overall stant, but it is also related to the salinity within the ocean.
structure of the TC, especially its distinct inner core re- D26 represents the thickness of the 26°C isotherm layer,
gion, plays a pivotal role in intensity prediction. The con- and T is the seawater temperature, which changes with
figuration of the cloud system also provides the model the water depth (Shay and Brewster, 2010). Theoretic-
with key insights into the current state and potential de- ally, the larger the OHC value, the thicker the warm wa-
velopment of the TC. In summary, the saliency maps of- ter layer above 26°C, which is more conducive to the
fer us a deep understanding of how the VGGNet model formation or intensification of TCs.
assesses TC intensity, showcasing the contributions of
different cloud system features to the prediction. REFERENCES
The marked enhancement in prediction underscores Asif, A., M. Dawood, B. Jan, et al., 2020: PHURIE: Hurricane in-
the efficacy of our chosen VGGNet model. The numer- tensity estimation from infrared satellite imagery using ma-
ous enhancements implemented in our model’s perform- chine learning. Neural Comput. Appl., 32, 4821–4834, doi:
ance provide valuable perspectives for forecasting TC in- 10.1007/s00521-018-3874-6.
tensity and by extension, other TC metrics. This bears Bessho, K., K. Date, M. Hayashi, et al., 2016: An introduction to
Himawari-8/9—Japan’s new-generation geostationary met-
significant implications for future practical applications,
eorological satellites. J. Meteor. Soc. Japan, 94, 151–183,
potentially mitigating the detrimental impacts of increas-
doi: 10.2151/jmsj.2016-009.
ingly severe TC disasters. However, our models are Bolton, T., and L. Zanna, 2019: Applications of deep learning to
trained and evaluated by using data available up to 2022. ocean data inference and subgrid parameterization. J. Adv.
The performance of the model with the addition of new Model. Earth Syst., 11, 376–399, doi: 10.1029/2018MS00
data remains unpredictable. To ensure sustained accur- 1472.
acy, it is advisable to update the parameters and weights Chen, B., B.-F. Chen, and H.-T. Lin, 2018: Rotation-blended
of the various algorithms before deploying the model in CNNs on a new open dataset for tropical cyclone image-to-in-
practical applications. While our outcomes are prom- tensity regression. Proceedings of the 24th ACM SIGKDD
International Conference on Knowledge Discovery & Data
ising, it is important to acknowledge the intricate factors
Mining, ACM, London, UK, 90–99, doi: 10.1145/3219819.
governing convective intensities and distributions. Be- 3219926.
sides, considering that our VGGNet model estimates TC Chen, R., W. M. Zhang, and X. Wang, 2020: Machine learning in
intensity using only IR satellite cloud image data, further tropical cyclone forecast modeling: A review. Atmosphere,
enhancing accuracy with multi-source data might also be 11, 676, doi: 10.3390/atmos11070676.
beneficial. Chollet, F., 2017: Xception: Deep learning with depthwise separ-
Acknowledgments. The Geostationary Satellite Im- able convolutions. Proceedings of 2017 IEEE Conference on
agery of TCs is sourced at http://agora.ex.nii.ac.jp/digit- Computer Vision and Pattern Recognition, IEEE, Honolulu,
HI, USA, 1251–1258, doi: 10.1109/CVPR.2017.195.
al-typhoon/year/wnp. The TC best track data are ob-
Chollet, F., 2018: Deep Learning with Python. Manning Publica-
tained from https://www.metoc.navy.mil/jtwc/jtwc.html?
tions Co., Shelter Island, 306 pp.
best-tracks. The ECMWF data are from https://cds.cli- Dawood, M., A. Asif, and F. U. A. A. Minhas, 2020: Deep-
mate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5- PHURIE: Deep learning based hurricane intensity estimation
pressure-levels?tab=form and https://cds.climate.coperni- from infrared satellite imagery. Neural Comput. Appl., 32,
cus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels? 9009–9017, doi: 10.1007/s00521-019-04410-7.
662 Journal of Meteorological Research Volume 38
Emanuel, K. A., 1999: Thermodynamic control of hurricane in- Krizhevsky, A., I. Sutskever, and G. E. Hinton, 2017: ImageNet
tensity. Nature, 401, 665–669, doi: 10.1038/44326. classification with deep convolutional neural networks. Com-
Fetanat, G., A. Homaifar, and K. R. Knapp, 2013: Objective trop- mun. ACM, 60, 84–90, doi: 10.1145/3065386.
ical cyclone intensity estimation using analogs of spatial fea- Lagerquist, R., A. McGovern, and D. J. Gagne II, 2019: Deep
tures in satellite data. Wea. Forecasting, 28, 1446–1459, doi: learning for spatially explicit prediction of synoptic-scale
10.1175/WAF-D-13-00006.1. fronts. Wea. Forecasting, 34, 1137–1160, doi: 10.1175/WAF-
Geisser, S., 1975: The predictive sample reuse method with applic- D-18-0183.1.
ations. J. Amer. Stat. Assoc., 70, 320–328, doi: 10.1080/0162 LeCun, Y., Y. Bengio, and G. Hinton, 2015: Deep learning. Nature,
1459.1975.10479865. 521, 436–444, doi: 10.1038/nature14539.
Ham, Y. G., J. H. Kim, and J. J. Luo, 2019: Deep learning for Leipper, D. F., and D. Volgenau, 1972: Hurricane heat potential of
multi-year ENSO forecasts. Nature, 573, 568–572, doi: 10. the Gulf of Mexico. J. Phys. Oceanogr., 2, 218–224, doi: 10.
1038/s41586-019-1559-7. 1175/1520-0485(1972)002<0218:HHPOTG>2.0.CO;2.
Han, Y. L., G. J. Zhang, X. M. Huang, et al., 2020: A moist phys- Li, X. F., B. Liu, G. Zheng, et al., 2020: Deep-learning-based in-
ics parameterization based on deep learning. J. Adv. Model. formation mining from ocean remote-sensing imagery. Natl.
Earth Syst., 12, e2020MS002076, doi: 10.1029/2020MS00 Sci. Rev., 7, 1584–1605, doi: 10.1093/nsr/nwaa047.
2076. Liu, J., X. F. Xu, and X. Y. Luo, 2019: Estimation of tropical cyc-
He, K. M., X. Y. Zhang, S. Q. Ren, et al., 2016: Deep residual lone intensity using infrared data from a geostationary satel-
learning for image recognition. Proceedings of 2016 IEEE lite. SOLA, 15, 189–192, doi: 10.2151/sola.2019-034.
Conference on Computer Vision and Pattern Recognition, Liu, Y. J., E. Racah, Prabhat, et al., 2016: Application of deep con-
IEEE, Las Vegas, NV, USA, 770–778, doi: 10.1109/CVPR. volutional neural networks for detecting extreme weather in
2016.90. climate datasets. Proceedings of 2016 International Confer-
Hubel, D. H., and T. N. Wiesel, 1962: Receptive fields, binocular ence on Advances in Big Data Analytics, Computer Vision
interaction and functional architecture in the cat’s visual cor- and Pattern Recognition, Boston, MA, USA, 81–88.
tex. J. Physiol., 160, 106–154, doi: 10.1113/jphysiol.1962. Lloyd, I. D., and G. A. Vecchi, 2011: Observational evidence for
sp006837. oceanic controls on hurricane intensity. J. Climate, 24, 1138–
Jin, S. H., S. Wang, and X. F. Li, 2014: Typhoon eye extraction 1153, doi: 10.1175/2010JCLI3763.1.
with an automatic SAR image segmentation method. Int. J. Marks, F. D., L. K. Shay, G. Barnes, et al., 1998: Landfalling trop-
Remote Sens., 35, 3978–3993, doi: 10.1080/01431161.2014. ical cyclones: Forecast problems and associated research op-
916447. portunities. Bull. Amer. Meteor. Soc., 79, 305–323, doi: 10.
Jin, S. H., S. Wang, X. F. Li, et al., 2017: A salient region detec- 1175/1520-0477(1998)079<0305:LTCFPA>2.0.CO;2.
tion and pattern matching-based algorithm for center detec- McGovern, A., R. Lagerquist, D. J. Gagne II, et al., 2019: Making
tion of a partially covered tropical cyclone in a SAR image. the black box more transparent: Understanding the physical
IEEE Trans. Geosci. Remote Sens., 55, 280–291, doi: 10. implications of machine learning. Bull. Amer. Meteor. Soc.,
1109/TGRS.2016.2605766. 100, 2175–2199, doi: 10.1175/BAMS-D-18-0195.1.
Jin, S. H., X. F. Li, X. F. Yang, et al., 2019: Identification of trop- Olander, T. L., and C. S. Velden, 2007: The advanced Dvorak
ical cyclone centers in SAR imagery based on template technique: Continued development of an objective scheme to
matching and particle swarm optimization algorithms. IEEE estimate tropical cyclone intensity using geostationary in-
Trans. Geosci. Remote Sens., 57, 598–608, doi: 10.1109/TGRS. frared satellite imagery. Wea. Forecasting, 22, 287–298, doi:
2018.2863259. 10.1175/WAF975.1.
Judt, F., and S. S. Chen, 2016: Predictability and dynamics of trop- Olander, T. L., and C. S. Velden, 2019: The advanced Dvorak
ical cyclone rapid intensification deduced from high-resolu- technique (ADT) for estimating tropical cyclone intensity:
tion stochastic ensembles. Mon. Wea. Rev., 144, 4395–4420, Update and new capabilities. Wea. Forecasting, 34, 905–922,
doi: 10.1175/MWR-D-15-0413.1. doi: 10.1175/WAF-D-19-0007.1.
Kaplan, J., and M. DeMaria, 2003: Large-scale characteristics of Pan, S. J., and Q. Yang, 2010: A survey on transfer learning. IEEE
rapidly intensifying tropical cyclones in the North Atlantic Trans. Knowl. Data Eng., 22, 1345–1359, doi: 10.1109/TK-
basin. Wea. Forecasting, 18, 1093–1108, doi: 10.1175/1520- DE.2009.191.
0434(2003)018<1093:LCORIT>2.0.CO;2. Piñeros, M. F., E. A. Ritchie, and J. S. Tyo, 2011: Estimating trop-
Kaplan, J., M. DeMaria, and J. A. Knaff, 2010: A revised tropical ical cyclone intensity from infrared image data. Wea. Fore-
cyclone rapid intensification index for the Atlantic and east- casting, 26, 690–698, doi: 10.1175/WAF-D-10-05062.1.
ern North Pacific basins. Wea. Forecasting, 25, 220–241, doi: Pradhan, R., R. S. Aygun, M. Maskey, et al., 2018: Tropical cyc-
10.1175/2009WAF2222280.1. lone intensity estimation using a deep convolutional neural
Kaplan, J., C. M. Rozoff, M. DeMaria, et al., 2015: Evaluating en- network. IEEE Trans. Image Process., 27, 692–702, doi: 10.
vironmental impacts on tropical cyclone rapid intensification 1109/TIP.2017.2766358.
predictability utilizing statistical models. Wea. Forecasting, Price, J. F., 1981: Upper ocean response to a hurricane. J. Phys.
30, 1374–1396, doi: 10.1175/WAF-D-15-0032.1. Oceanogr., 11, 153–175, doi: 10.1175/1520-0485(1981)011
Kossin, J. P., J. A. Knaff, H. I. Berger, et al., 2007: Estimating hur- <0153:UORTAH>2.0.CO;2.
ricane wind structure in the absence of aircraft reconnais- Ramdan, A., A. Heryana, A. Arisal, et al., 2020: Transfer learning
sance. Wea. Forecasting, 22, 89–101, doi: 10.1175/WAF985.1. and fine-tuning for deep learning-based tea diseases detection
AUGUST 2024 Yang, W., J. F. Fei, X. G. Huang, et al. 663
on small datasets. Proceedings of 2020 International Confer- tropical cyclone number, duration, and intensity in a warm-
ence on Radar, Antenna, Microwave, Electronics, and Tele- ing environment. Science, 309, 1844–1846, doi: 10.1126/sci-
communications, IEEE, Tangerang, Indonesia, 206–211, doi: ence.1116448.
10.1109/ICRAMET51080.2020.9298575. Weyn, J. A., D. R. Durran, and R. Caruana, 2020: Improving data-
Riehl, H., 1950: A model of hurricane formation. J. Appl. Phys., driven global weather prediction using deep convolutional
21, 917–925, doi: 10.1063/1.1699784. neural networks on a cubed sphere. J. Adv. Model. Earth
Ritchie, E. A., G. Valliere-Kelley, M. F. Piñeros, et al., 2012: Syst., 12, e2020MS002109, doi: 10.1029/2020MS002109.
Tropical cyclone intensity estimation in the North Atlantic Wu, L. G., and H. K. Zhao, 2012: Dynamically derived tropical
basin using an improved deviation angle variance technique. cyclone intensity changes over the western North Pacific. J.
Wea. Forecasting, 27, 1264–1277, doi: 10.1175/WAF-D-11- Climate, 25, 89–98, doi: 10.1175/2011JCLI4139.1.
00156.1. Xu, Q., G. S. Zhang, X. F. Li, et al., 2016: An automatic method
Shay, L. K., and J. K. Brewster, 2010: Oceanic heat content vari- for tropical cyclone center determination from SAR. Proceed-
ability in the eastern Pacific Ocean for hurricane intensity ings of 2016 IEEE International Geoscience and Remote
forecasting. Mon. Wea. Rev., 138, 2110–2131, doi: 10.1175/ Sensing Symposium (IGARSS), IEEE, Beijing, China,
2010MWR3189.1. 2250–2252, doi: 10.1109/IGARSS.2016.7729581.
Shu, S. J., J. Ming, and P. Chi, 2012: Large-scale characteristics Yang, X. F., and K. S. Xiang, 2019: Synergistic use of satellite act-
and probability of rapidly intensifying tropical cyclones in the ive and passive microwave observations to estimate typhoon
western North Pacific basin. Wea. Forecasting, 27, 411–423, intensity. Proceedings of 2019 Photonics & Electromagnetics
doi: 10.1175/WAF-D-11-00042.1. Research Symposium-Spring (PIERS-Spring), IEEE, Rome,
Simonyan, K., and A. Zisserman, 2015: Very deep convolutional
Italy, 1612–1617, doi: 10.1109/PIERS-Spring46901.2019.
networks for large-scale image recognition. Proceedings of
9017799.
the 3rd International Conference on Learning Representa-
Zahera, H. M., M. A. Sherif, and A. C. N. Ngomo, 2019: Jointly
tions, IEEE, San Diego, USA, 313–318.
learning from social media and environmental data for
Smilkov, D., N. Thorat, B. Kim, et al., 2017: SmoothGrad: Re-
typhoon intensity prediction. Proceedings of the 10th Interna-
moving noise by adding noise. Available online at doi:
tional Conference on Knowledge Capture, ACM, Marina Del
10.48550/arXiv.1706.03825. Accessed on 5 March 2024.
Rey, CA, USA, 231–234, doi: 10.1145/3360901.3364413.
Sønderby, C. K., L. Espeholt, J. Heek, et al., 2020: MetNet: A
Zhang, C.-J., Q. Luo, L.-J. Dai, et al., 2019: Intensity estimation of
neural weather model for precipitation forecasting. Available
tropical cyclones using the relevance vector machine from in-
online at doi: 10.48550/arXiv.2003.12140. Accessed on 5
frared satellite image data. IEEE J. Sel. Top. Appl. Earth Obs.
March 2024.
Remote Sens., 12, 763–773, doi: 10.1109/JSTARS.2019.289
Szegedy, C., V. Vanhoucke, S. Ioffe, et al., 2016: Rethinking the
4654.
Inception architecture for computer vision. Proceedings of
2016 IEEE Conference on Computer Vision and Pattern Re- Zhao, X. H., and J. C. L. Chan, 2017: Changes in tropical cyclone
cognition, IEEE, Las Vegas, NV, USA, 2818–2826, doi: intensity with translation speed and mixed-layer depth: Ideal-
10.1109/CVPR.2016.308. ized WRF-ROMS coupled model simulations. Quart. J. Roy.
Takeuchi, Y., 2018: An introduction of advanced technology for Meteor. Soc., 143, 152–163, doi: 10.1002/qj.2905.
tropical cyclone observation, analysis and forecast in JMA. Zhao, Y., C. F. Zhao, R. Y. Sun, et al., 2016: A multiple linear re-
Trop. Cyclone Res. Rev., 7, 153–163, doi: 10.6057/2018TCRR gression model for tropical cyclone intensity estimation from
03.01. satellite infrared images. Atmosphere, 7, 40, doi: 10.3390/at-
Velden, C., B. Harper, F. Wells, et al., 2006: The Dvorak tropical mos7030040.
cyclone intensity estimation technique: A satellite-based Zheng, G., J. S. Yang, A. K. Liu, et al., 2016: Comparison of
method that has endured for over 30 years. Bull. Amer. Met- typhoon centers from SAR and IR images and those from best
eor. Soc., 87, 1195–1210, doi: 10.1175/BAMS-87-9-1195. track data sets. IEEE Trans. Geosci. Remote Sens., 54,
Velden, C., T. Olander, D. Herndon, et al., 2017: Reprocessing the 1000–1012, doi: 10.1109/TGRS.2015.2472282.
most intense historical tropical cyclones in the satellite era us- Zhuge, X.-Y., J. Guan, F. Yu, et al., 2015: A new satellite-based
ing the advanced Dvorak technique. Mon. Wea. Rev., 145, indicator for estimation of the western North Pacific tropical
971–983, doi: 10.1175/MWR-D-16-0312.1. cyclone current intensity. IEEE Trans. Geosci. Remote Sens.,
Wang, C., G. Zheng, X. F. Li, et al., 2022: Tropical cyclone intens- 53, 5661–5676, doi: 10.1109/TGRS.2015.2427035.
ity estimation from geostationary satellite imagery using deep Zhuo, J.-Y., and Z.-M. Tan, 2021: Physics-augmented deep learn-
convolutional neural networks. IEEE Trans. Geosci. Remote ing to improve tropical cyclone intensity and size estimation
Sens., 60, 1–16, doi: 10.1109/TGRS.2021.3066299. from satellite imagery. Mon. Wea. Rev., 149, 2097–2113, doi:
Webster, P. J., G. J. Holland, J. A. Curry, et al., 2005: Changes in 10.1175/MWR-D-20-0333.1.