1

Tropical Cyclone Forecast Using Multitask Deep

Learning Framework
Yuqiao Wu, Xiaoyi Geng, Zili Liu and Zhenwei Shi, Member, IEEE

Abstract—A tropical cyclone is a robust weather system that and intensity of tropical cyclones, so it is very difficult to
affects human daily life. Accurate and rapid tropical cyclone accurately describe it with existing models [1]. Furthermore,
forecast can guide human disaster prevention and mitigation forecasting the intensity of tropical cyclones is more difficult
work against tropical cyclones. The mainstream tropical cyclone
forecasting method is numerical forecasting, which requires than forecasting the path. Researchers believe that this is
abundant prior knowledge and luxurious calculation. Nowadays, because the physical process that causes the intensity change-
machine learning methods have received increasing attention ment of tropical cyclones is so complicated that we process
for they can overcome these disadvantages. However, existing little knowledge of it [2]. These two factors affect each other
machine learning methods usually ignored some potential factors during tropical cyclone development, so it’s challenging but
due to they mainly concentrated on one aspect of the tropical cy-
clone forecast. This letter proposes a multitask machine learning significant work to combine these two tasks together.
framework to forecast tropical cyclone path and intensity, which In this letter, we proposed a framework that can give
possesses two modules: one is the prediction module, the other is a quick and reliable forecast of tropical cyclone path and
the estimate module. We use an improved generative adversarial intensity based on the infrared image. The framework has a
network as the prediction module to predict the tropical cyclone prediction module to predict the future spatial data of tropical
spatial data at a certain moment in the future. Then, we use
two different deep neural networks as the estimation module to cyclones and an estimation module to determine the value of
extract the position and intensity from the generated prediction the indicator from the predicted result. We set a retrospective
data. The method we propose is a general and relatively accurate CycleGAN [3] using Wasserstein loss [4] in the prediction
tropical cyclone forecast method. We reach a 24h path forecast module. Then in the estimation module, we build a new model
error of 116km and a 24h intensity forecast error of 13.06kt. called TIENet to predict the intensity and use TCLNet to
Index Terms—Tropical Cyclone Forecast, Generative Adver- predict position. Our work achieves an average 6-hour path
sarial Network, Wasserstein Distance. forecast error of 61km and an average 24-hour path forecast
error of 116km, while our 6-hour intensity error and 24-hour
I. INTRODUCTION intensity forecast error respectively reach 14.20kt and 13.06kt.
These results are produced with the last 24 hours data within

T ROPICAL cyclones are cyclonic circulations that occur

over the sea in tropical and subtropical regions. It is a
complicated and severe weather system. The power release of
less than 10 seconds. In addition, the path forecast error is 10%
better than Rüttgers’ work [5], which has the best precision
in existing models. It is also a flexible framework. On one
a mature tropical cyclone can reach the level of one hundred hand, the input can be not only the infrared images but also
terawatts, which will bring a series of meteorological disasters satellite images, meteorological reanalysis data. On the other
such as gale, storm surge, and heavy rain. Therefore, accurate hand, the output can be diverse with different tropical cyclone
and rapid forecasting of tropical cyclone indicators can help data analyse models. The overview of the forecast framework
guide human disaster prevention and mitigation work against is shown in Fig 1. Briefly, our work has several distinct
tropical cyclones. It also has important implications in the contributions as follows:
scientific use of tropical cyclones. Among all the indicators, • First, the framework consisting of three networks provides
position and intensity are the most crucial indicators. a one-step solution which can forecast the path and
The forecast of tropical cyclone path movement is usually intensity of tropical cyclones at the same time.
based on a general and accurate understanding of tropical • Second, we combine retrospective CycleGAN [3] with
cyclone motion. This law is affected by many complex factors, Wasserstein loss [4], which strengthens the adversariness
such as large-scale weather patterns, sea level, and atmospheric between the generator and discriminator. This helps ob-
temperature, land topographic characteristics, the structure tain high-similar results to the ground truth. This new
network can also be used in the area of predicting video
The work was supported in part by the National Key Research and
Development Program of China under the Grant 2019YFC1510905, in part by frames.
the National Natural Science Foundation of China under the Grant 62125102, • Third, the TIENet can be a new tool for tropical cyclone
and in part by the Beijing Natural Science Foundation under the Grant image interpretation.
4192034.(Corresponding author: Zhenwei Shi)
Yuqiao Wu(e-mail: wu [email protected]), Xiaoyi Geng(e-mail:
[email protected]), Zili Liu(e-mail: [email protected]) and Zhenwei II. RELATED WORK
Shi (Corresponding Author, e-mail: [email protected]) are with We survey the development of neural network methods
Image Processing Center, School of Astronautics, Beihang University,
Beijing 100191, China. used in the tropical cyclone forecast and generative adversarial
networks. Here are some details.
 2

Fig. 1. An overview of the forecast framework. WCycleGAN generates the predicted data from historical data, next TCLNet and TIENet extract the position
and intensity information from it. The green point shows the forecast location, besides the blue point shows the ground truth location.

The research of neural network methods used in the tropical to accomplish the path and intensity forecast although this
cyclone forecast started at the end of the last century. Until framework can complete more tasks. In this section, we will
the 2010s, MLP and BP network were the mainstream neural talk about the details of the three networks in the two modules
network methods for forecasting the intensity and path of that compose our framework.
tropical cyclones [6] [7]. Since the mid-2010s, due to the
development of deep learning, more new methods have been
A. Prediction module
introduced into the forecast of tropical cyclones. Recurrent
neural network(RNN) is a class of neural networks that exhibit This module consists of one network which we call WCy-
temporal dynamic behavior, Moradi Kordmahalleh et al. [8] cleGAN. The network is inspired by Kwon et al. [3] in predict-
and Alemany et al. [9] used this method to forecast the path ing future frames. We use the same retrospective method as
of tropical cyclones in 2016 and 2019, while Pan et al. [10] they did, and we adopt their network architecture and change
used this method to forecast the intensity of tropical cyclones the input and output layers to adapt the grayscale image. This
in 2019. Rüttgers et al. [5] introduced GAN into the tropical GAN has two discriminators, one is the frame discriminator
cyclone path forecast field in 2018. The LSTM network also likes others own, the other one is the sequence discriminator
plays an important role in tropical cyclone forecast, Kim et that we use to enhance the relationship between the inputs
al. [11] used convLSTM to forecast the tropical cyclone path. and outputs. We improved the loss function with the idea of
Meanwhile, Chen et al. [12] combined LSTM with CNN Wasserstein distance [17], for the original discriminator loss so
to forecast the intensity. These methods mainly concentrated rapidly converges to zero that it can’t provide instructions for
on one aspect of the tropical cyclone forecast, so they were the generator. We apply Wasserstein distance by using gradient
inclined to use specialized data. Furthermore, they ignored penalty [4]. It successfully solves this problem and provides
some potential factors due to this operation. Meanwhile, it a better result. We also believe the Wasserstein loss can help
means we have to call several different models to obtain our find out the connection between long interval sequence data
expected results. such as the image of tropical cyclones. The generator loss can
Since the generative adversarial network was proposed in be formulated as follows:
2014, it has become a research hotspot these years. Re- f rame seq
LG = Limage + λ1 LLoG + λ2 LGadv + λ3 LGadv (1)
searchers have made a lot of effort to improve its performance.
On one hand, researchers changed the network’s structure, where λ1 , λ2 , λ3 are parameters. We call the first two items
for example, DCGAN [13], LAPGAN [14], and CycleGAN in the formula as reconstruction losses and the last two
[15], to help the network fit more tasks. On the other hand, adversarial losses. The reconstruction losses can be formulated
researchers adjusted the loss function of the network, for as follows:
example, LSGAN [16] and WGAN [4], to help strengthen 1 X
Limage = l1 (p, q) (2)
the adversariness between generator and discriminator, which 6 pair
(p,q)∈S m,n
can lead into a higher-quality result. These high-performance
models are of great help to meteorological research. Among 1 X
all these works, the CycleGAN is widely applied in computer LLoG = l1 (LoG (p) , LoG (q)) (3)
6 pair
vision as a result of its ability to build connections among (p,q)∈S m,n

unpaired pictures, while others mainly handle paired data. where

However, Zhu et al. used the LSGAN’s loss function; hence,  0
  00
  0 00 
xm , xm , xm , xm , xm , xm ,
 
it may be difficult to get a perfect result in some situations.
 
pair
=
 
S m,n (4)
 
0 00 0 00
     
 xn+1 , x

, xn+1 , x , x x



III. METHODOLOGY n+1 n+1 n+1 n+1

This method can be regarded as a simple application of where xm is the first picture in a sample, xn+1 is the last
0 00
the framework that we proposed. We only use two networks picture in a sample, x and x respectively represent the two
 3

generated results in a loop. The adversarial losses can be

formulated as follows:
f rame 1 X h i
LGadv = E D f rame (e
x) (5)
4 ex∈P
m,n

seq 1 X h  i
LGadv = E D seq X
e (6)
4e
X∈Mm,n

where n 0 00 0 00
o
Pm,n = xm , xm , xn+1 , xn+1 (7) Fig. 2. The structure of TIENet. It has 5 convolution layers and 2 liner layers.
 0 00
   
 xm , · · · , xn+1 , xm , · · · , xn+1 ,

=
 
Mm,n (8)

 x0 , · · · , xn+1 , x0 , · · · , xn+1 
    
 IV. EXPERIMENTS AND RESULT
m m
A. Datasets
Here are two kinds of discriminator losses, one is the
1) Tropical cyclone infrared time series dataset: This
frame loss, the other one is the sequence loss. The frame
dataset comes from the infrared window channel in the US
discriminator loss can be formulated as follows:
 X  h i h i grid satellite dataset (GridSat) [19]. We intercept a tropical
 −E D f rame (x) + E D f rame (e x)  cyclone at a certain time t according to BST dataset [20]
1 
LDf rame = ex∈Pm,n ,xQm,n 

 with a resolution of 256 x 256 pixels (20 latitudes multiply
4  2  
20 longitudes) and 6h, whose center is ensured to locate in
−λ4 E k∇ex∈Pm,n D f rame (e x) k2 − 1 
the center of the picture. Afterward, keep the position of this
(9)
f rame window in the GridSat global image unchanged, and intercept
where Pm,n is same with Pm,n in L , λ4 is a parameter,
 Gadv 2  the images at time t − 24, t − 18, t − 12, t − 6, t, t + 6, t + 12,
Qm,n = {xm , xn+1 }, E k∇ex∈Pm,n D f rame (e x) k2 − 1 is the gradi- t + 18, t + 24 under this window in turn. All these images
ent penalty, which is used to ensure the Lipschitz continuity. compose a sequence consisting of 9 pictures, and one example
The sequence discriminator loss can be formulated as follows: is shown in Fig 3. Next, we extract 5 continuous images from
 X  h i h i each tropical cyclone sequence as training samples, and each
 −E D seq (x) + E D seq (e x) 
1  sequence provides 5 training samples.
LDseq = ex∈Mm,n ,xNm,n

 (10)
4  2   2) Estimation dataset: This dataset uses the results ob-
−λ5 E k∇ex∈Pm,n D seq (ex) k2 − 1 tained by the WCycleGAN as the input image. The 6h
seq
prediction results are the direct outputs of the network while
where Mm,n is same with  Mm,n in LGadv , λ5 is2a parameter, the 24h prediction results are the 4-step outputs. We get the
Nm,n = (xm , · · · , xn+1 ), E k∇ex∈Pm,n D seq (e
x) k2 − 1 is also the coordinate(u,v) of the tropical cyclone center in the image
gradient penalty. according to the BST dataset, and the intensity label comes
from the maximum wind speed near the tropical cyclone center
extracted from the BST dataset, which is divided into 13
B. Estimation module
categories according to Beaufort scale, corresponding to 7-
This module consists of two networks, one is the TIENet, 17+ levels. The heatmap can be produced as the formula:
the other is the TCLNet. We propose a novel network termed
(x − u)2 + (y − v)2
!
as TIENet to determine the intensity, whose output is a H(x, y) = exp (11)
predicted Beaufort scale when input is the predicted image −2σ2
out from the WCycleGAN. Unlike other methods predicting where H (x, y) means the pixel value at (x,y) in the heatmap,
the specific wind speed of the tropical cyclone, this trick σ is a parameter, which takes 15 here. An example of a set
can reduce the calculation amount while evaluating the effect of images in location and intensity determination dataset is
well. We stress the importance of the detailed structure of shown in Fig 4.
tropical cyclones, so we choose 5 convolutional layers with
convolution kernels doubling and a stride of 1. This network
has far fewer parameters than the ResNet50 but provides B. Training Details
similar performance. We choose the cross-entropy among the To train our networks, we use 4,930 training samples from
predicted Beaufort scales and the real intensity labels as its 986 tropical cyclone sequences. In WCycleGAN, we set λ1 =
loss function. The structure of TIENet is illustrated in Fig 2. 0.005, λ2 = λ3 = 0.003, λ4 = λ5 = 10 according to Kwon et
The TCLNet comes from Tan C.’s work [18] Its output al.’s work [3] and Gulrajani et al.’s work [4]. We use adam
is a heatmap when input is the predicted image out from optimizers [21] with β1 = 0.5, β2 = 0.999 and the learning rate
the WCycleGAN. It uses an improved MSE loss to describe is set as follows: for WCycleGAN, first, we use the learning
the difference between the generated heatmap and the real rate of 0.0001 to train for 20 epochs, then reduce it to 0.00005
heatmap and is trained to narrow this difference. The co- for 30 epochs, finally to 0.00001 for 40 epochs; for TCLNet,
ordinate which has the highest pixel value of the generated after 4 epochs of training with a learning rate of 0.001, it
heatmap represents the location of the tropical cyclone center. is reduced to 0.00005 for 1 epoch; for TIENet, we train 10
 4

Fig. 3. An example of the sequence in the tropical cyclone infrared time series dataset.

TABLE I
Overall result. 50 6h-test samples. 10 24h-test samples. PSNR, SSIM are
the larger the better, while MSE, LD and S E are the smaller the better.

Metrics 6h 24h
Average PSNR 22.46 19.77

Fig. 4. An example of a set of images in location and intensity determination Average SSIM 0.65 0.64
dataset. Average MSE 5.89 11.10
Average LD 61 km 116 km
Max LD 122 km 192 km
epochs with a learning rate of 0.1, then reduce it to 0.01 for Average S E 14.20 kt 13.06 kt
20 epochs, finally to 0.001 for 30 epochs. Max S E 38.49 kt 38.49 kt

C. Metrics
We use PSNR, SSIM, and MSE (we multiplied the original
MSE by a factor of 100 to show the difference) to measure
the quality of the result of WCycleGAN. We use path forecast
error(LD ) to measure the quality of the result of TLCNet. Here
is its calculating formula:

LD = PD ÷ 256 × 20 × 1.852 × 60 × 0.866 (12)

the PD in LD is the pixel distance between the ground truth

and predicted position. We calculate 0.866 based on there are Fig. 5. The output of WCycleGAN of Typhoon Trami.
0.866 nautical miles for each change in latitude and longitude
in the mid-latitude area. We use intensity forecast error(S E )
of intensity forecast to measure the quality of the result of 2) Sample result: Here are the forecast results of Typhoon
TIENet. Here is its calculating formula: Trami, the No. 24 in 2018. The output of WCycleGAN is
1 shown in Fig 5, the result of TCLNet is shown in Fig 6, the
SE = × |WS p − WS gt | (13) 6h forecast result of the TIENet is shown in Table II and the
N
24h forecast result of the TIENet only exists at 2018.09.26
the WS here means wind speed. We use the middle wind 00:00, when Beaufort scale is 14 and tropical cyclone scale is
speed of each Beaufort scale as the predicted wind speed. STY. It can be seen from the above chart that concerning path
forecasting, whether it is a 6h forecast or a 24h forecast, it can
have a good forecast effect; concerning intensity forecast, the
D. Results accuracy rate needs to be further improved, but for tropical
1) Overall result: We test our method on 10 tropical cyclone level judgment is basically accurate.
cyclones in the western Pacific. Our forecast method has 3) Horizontal comparison: We also comprise our work
an average path forecast error of 116km and an intensity with other methods, the result is shown in Table III. It can be
forecast error of 13.06kt under the forecast time limit of 24 seen that the path prediction results of this method have the
hours. In terms of path prediction, we set expected errors of
60km and 110km for 6h and 24h path forecast. Accordingly,
the eligibility rates are 60% and 40%, which need to be
further improved. As to intensity prediction, we consider the
predicted tropical cyclones scale matching the ground-truth as
a successful forecast. Based on this, the accuracy rates of 6h
and 24h intensity forecasts are 36% and 40%. The intensity
forecast results ignore the changes during a short time interval.
This may be solved by using a more efficient intensity estimate
model. More details are shown in Table I. Fig. 6. The output of TCLNet of Typhoon Trami.
 5

TABLE II time with good scalability. Furthermore, the WCycleGAN is

The 6h forecast results from the output of the TIENet of Typhoon Trami in a more effective new model that handles unpaired images,
2018.
and the TIENet is a specialized model for predicted tropical
cyclone image interpretation. In the situation of lacking com-
Time 6h forecast intensity True forecast intensity
puting resources or professional meteorological knowledge, it
09.25 06:00 Beaufort scale: 14 Tropical cyclone scale: STY Beaufort scale: 17 tropical cyclone scale: SuperTY
is of great significance.
09.25 12:00 Beaufort scale: 14 Tropical cyclone scale: STY Beaufort scale: 17 Tropical cyclone scale: SuperTY
09.25 18:00 Beaufort scale: 14 Tropical cyclone scale: STY Beaufort scale: 16 Tropical cyclone scale: SuperTY
References
09.26 00:00 Beaufort scale: 14 Tropical cyclone scale: STY Beaufort scale: 15 Tropical cyclone scale: STY
[1] K. Emanuel, “100 years of progress in tropical cyclone research,”
09.26 06:00 Beaufort scale: 14 Tropical cyclone scale: STY Beaufort scale: 15 Tropical cyclone scale: STY Meteorological Monographs, vol. 59, pp. 15–1, 2018.
[2] R. Chen, W. Zhang, and X. Wang, “Machine learning in tropical cyclone
forecast modeling: A review,” Atmosphere, vol. 11, no. 7, p. 676, 2020.
TABLE III [3] Y.-H. Kwon and M.-G. Park, “Predicting future frames using retro-
Horizontal comparison. The last two methods are the numerical models, the spective cycle gan,” in Proceedings of the IEEE/CVF Conference on
others are machine learning models. The LD is a path forecast index, which Computer Vision and Pattern Recognition, 2019, pp. 1811–1820.
is the smaller the better. The S E is an intensity forecast index, which is the [4] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville,
larger the better. “Improved training of wasserstein gans,” in Proceedings of the 31st
International Conference on Neural Information Processing Systems,
Methods 6h Average LD 24h Average LD 6h Average S E 24h Average S E 2017, pp. 5769–5779.
OURS 61 km 116 km 14.20 kt 13.06 kt
[5] M. Rüttgers, S. Lee, S. Jeon, and D. You, “Prediction of a typhoon track
using a generative adversarial network and satellite images,” Scientific
RNN [8] 72 km - - -
reports, vol. 9, no. 1, pp. 1–15, 2019.
ConvLSTM [11] 141 km - - - [6] J.-J. Baik and H.-S. Hwang, “Tropical cyclone intensity prediction using
CNN-LSTM [12] - - - 7.4 kt regression method and neural network,” Journal of the Meteorological
GAN [5] 67.2 km - - - Society of Japan. ser. ii, vol. 76, no. 5, pp. 711–717, 1998.
ECMWF-IFS - 62 km - 14.3 kt [7] M. Ali, C. Kishtawal, and S. Jain, “Predicting cyclone tracks in the
NCEP-GFS - - - 12.9 kt north indian ocean: An artificial neural network approach,” Geophysical
research letters, vol. 34, no. 4, 2007.
[8] M. Moradi Kordmahalleh, M. Gorji Sefidmazgi, and A. Homaifar, “A
sparse recurrent neural network for trajectory prediction of atlantic hur-
smallest average path forecast error among the neural network ricanes,” in Proceedings of the Genetic and Evolutionary Computation
Conference 2016, 2016, pp. 957–964.
methods listed in the table, but it is near twice the error of the [9] S. Alemany, J. Beltran, A. Perez, and S. Ganzfried, “Predicting hurricane
best numerical method. Besides, the intensity forecast level is trajectories using a recurrent neural network,” in Proceedings of the
close to the numerical method; the gap between CNN-LSTM AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp.
468–475.
and ours may lie in: we choose to give a range of the predicted [10] B. Pan, X. Xu, and Z. Shi, “Tropical cyclone intensity prediction based
wind speed while they give the specific numerical value. on recurrent neural networks,” Electronics Letters, vol. 55, no. 7, pp.
413–415, 2019.
[11] S. Kim, H. Kim, J. Lee, S. Yoon, S. E. Kahou, K. Kashinath, and
E. Ablation study M. Prabhat, “Deep-hurricane-tracker: Tracking and forecasting extreme
climate events,” in 2019 IEEE Winter Conference on Applications of
We also study the impact of Wasserstein loss and LoG loss Computer Vision (WACV). IEEE, 2019, pp. 1761–1769.
on the forecast, the results are shown in Table IV. It can be [12] R. Chen, X. Wang, W. Zhang, X. Zhu, A. Li, and C. Yang, “A hybrid
cnn-lstm model for typhoon formation forecasting,” GeoInformatica,
seen that Wasserstein loss and LoG loss significantly improve vol. 23, no. 3, pp. 375–396, 2019.
the quality of predicted spatial data, significantly reduce the [13] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation
error of the path forecast results, and remarkably improve the learning with deep convolutional generative adversarial networks,” arXiv
preprint arXiv:1511.06434, 2015.
accuracy of the intensity forecast results. [14] E. Denton, S. Chintala, A. Szlam, and R. Fergus, “Deep generative
image models using a laplacian pyramid of adversarial networks,” in
TABLE IV Proceedings of the 28th International Conference on Neural Information
Ablation study. PSNR, SSIM are the larger the better, while MSE, LD , S E Processing Systems-Volume 1, 2015, pp. 1486–1494.
are the smaller the better. [15] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image
translation using cycle-consistent adversarial networks,” in Proceedings
of the IEEE international conference on computer vision, 2017, pp.
OURS Without Wasserstein loss Without LoG loss Without both 2223–2232.
Metrics
6h 24h 6h 24h 6h 24h 6h 24h [16] X. Mao, Q. Li, H. Xie, R. Y. Lau, Z. Wang, and S. Paul Smolley, “Least
Average PSNR 22.46 19.77 22.39 18.99 22.41 18.53 22.20 17.95 squares generative adversarial networks,” in Proceedings of the IEEE
Average SSIM 0.65 0.64 0.64 0.63 0.64 0.62 0.64 0.62 international conference on computer vision, 2017, pp. 2794–2802.
[17] L. Vasershtein, “Markov processes on a countable product of spaces
Average MSE 5.89 11.10 5.94 12.48 5.95 13.27 6.22 15.43
describing large automated systems,” Probl. Inform. Trans, vol. 14, pp.
Average LD 61 km 116 km 68 km 135 km 73 km 147 km 77 km 161 km 64–73, 1969.
Average S E 14.20 kt 13.06 kt 18.37 kt 18.96 kt 22.50 kt 23.71 kt 34.80 kt 38.22 kt [18] C. Tan, “Tclnet: Learning to locate typhoon center using deep neural
network,” arXiv preprint arXiv:2010.01282, 2020.
[19] K. Knapp, “Noaa climate data record (cdr) of gridded satellite data from
isccp b1 (gridsat-b1) infrared channel brightness temperature, version 2,”
V. CONCLUSION NOAA’s CDR Program, 2014.
[20] X. Lu, H. Yu, M. Ying, B. Zhao, S. Zhang, L. Lin, L. Bai, and
In this letter, we propose a flexible and reliable tropical R. Wan, “Western north pacific tropical cyclone database created by
the china meteorological administration,” Advances in Atmospheric
cyclone forecasting framework, which can simultaneously Sciences, vol. 38, no. 4, pp. 690–699, 2021.
forecast the path and intensity of tropical cyclones through a [21] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
set of data. This method can handle multiple tasks at the same arXiv preprint arXiv:1412.6980, 2014.