Earth Science Informatics (2024) 17:1837–1850

https://doi.org/10.1007/s12145-024-01281-y

REVIEW

Cyclone detection with end-to-end super resolution and faster R-CNN

Marwa S. Moustafa1 · Mohamed R. Metwalli1 · Roy Samshitha2 · Sayed A. Mohamed1 · Barma Shovan2

Received: 31 December 2023 / Accepted: 11 March 2024 / Published online: 26 March 2024
© The Author(s) 2024

Abstract
The accurate identification of extreme weather events (EWEs), particularly cyclones, has become increasingly crucial due
to the intensifying impacts of climate change. In the Indian subcontinent, the frequency and severity of cyclones have
demonstrably risen, highlighting the need for reliable detection methods to minimize casualties and economic losses.
However, the inherent limitations of low-resolution data pose significant challenges to traditional detection methods. Deep
learning models offer a promising solution, enabling the precise identification of cyclone boundaries crucial for assessing
regional impacts using global climate models data. By leveraging the power of deep learning, we can significantly enhance
our capabilities for cyclone detection and contribute to improved risk mitigation strategies in the vulnerable Indian sub-
continent. Therefore, this paper introduces an edge-enhanced super-resolution GAN (EESRGAN) leveraging an end-to-
end detector network. The proposed approach comprised of a generator network equipped by residual-in-residual dense
block (RRDB) and discriminator containing Faster RCNN detector. The precise patterns of cyclone had been effectively
extracted to help boundary detection. Extensive experiments have been conducted on Community Atmospheric Model
(CAM5.1) data taken into account only seven variables. Four matrices including precision, recall, intersection over union,
and mean average precision have been considered to assess the proposed approach. The results have been found very
effective while achieving accuracy up to 86.3% and average precision (AP) of 88.63%. Moreover, the proposed method
demonstrates its superiority while compared with benchmarks object detectors methods. Thus, the proposed method can
be employed in the area of extreme climate detection and could enrich the climate research domain.

Keywords Extreme cyclone detection · Deep learning · GAN · Faster R-CNN · Community atmospheric model
(CAM5.1)

Introduction

The intensity of severe climate events, such as heatwaves,

torrential rainfall, prolonged droughts, and violent storms,
Communicated by H. Babaie. has emerged as a worldwide issue in recent times. This con-
cerning pattern greatly enhances the susceptibility of com-
Marwa S. Moustafa munities, especially in urban and rural regions, presenting
[email protected]
a huge hazard and requiring thorough planning. Numeri-
Mohamed R. Metwalli cal climate models provide valuable predictions of shifting
[email protected]
weather patterns, but precisely identifying and forecasting
Sayed A. Mohamed extreme occurrences remains a serious challenge (Flaounas
[email protected]
et al. 2022; Mezősi 2022; Olaoluwa et al. 2022).
Barma Shovan Statistical conventional approaches remain center of
[email protected]
analyzing extreme climatic conditions, offering vital under-
1
Data Reception, Analysis and Receiving Station Division, standing of the features and patterns of these occurrences.
National Authority for Remote Sensing and Space Science, The methods encompass Extreme Value Theory (EVT),
Cairo, Egypt Block Maxima (BM), and Peaks-Over-Threshold (POT)
2
Indian Institute of Information Technology, Guwahati, approaches, Generalized Extreme Value (GEV) distribution,
Assam, India

13
1838 Earth Science Informatics (2024) 17:1837–1850

Index Threshold Method (ITM), and Partial Duration Series Shot MultiBox Detector (SSD) was used to pinpoint Extra-
(PDS) approach (Hulme 2014). Despite their robust theo- tropical Cyclones (ETCs) in the northern hemisphere (Shi
retical basis, simple implementation and understandable et al. 2022). A refined Deep Convolutional Neural Network
outcomes, their limitations include susceptibility to data (DCNN) (Tong et al. 2022) was introduced to accurately
quality issues, reliance on the assumption of stationarity, detect tropical cyclone fingerprints in the northern Pacific
subjectivity in the choosing of thresholds, and restrictions basin. These approaches showed similar levels of perfor-
associated with parametric models. As climate change con- mance in identifying cyclones. In (Pang et al. 2021), the
tinues to intensify, it is becoming increasingly important to GAN was combined by transfer learning to detect tropi-
develop more robust and sophisticated methods for extreme cal cyclones from meteorological images. A novel transfer
climate patterns. Hybrid approaches, such as TECA (Rübel learning model (Wang and Li 2023) was proposed to detect
et al. 2012), introduced as a possible solution for overcom- center of TC by harnessing knowledge from a vast image
ing these constraints by merging the advantages of conven- dataset and fine-tuning it for TC-specific features, the model
tional with more sophisticated approaches. achieves a remarkable 14.1% boost compared to traditional
Deep learning (DL), which draws inspiration from the methods. Another innovative CNN model was introduced
brain architectural, has catalyzed a revolution in artificial to pinpoints the centers of low-intensity tropical cyclones
intelligence. Diverging from conventional approaches, (Wang et al. 2024) by incorporating physical and historical
DL leverages extensive datasets to learn intricate patterns, data alongside satellite imagery, the model captures crucial
thereby facilitating breakthroughs in domains such as com- evolutionary trends in storm structure, achieving excep-
puter vision and natural language processing (Kaur and tional localization accuracy. The Thermal InfraRed (TIR)
Singh 2022; Zaidi et al. 2022). Numerous studies have laid Atmospheric Sounding Interferometer (IASI) on the Metop
the groundwork for exploring the profound influence of satellite was used to detect TCs in the North Atlantic Basin
deep learning in various domains. For instance, the poten- using YOLOv3 (Lam et al. 2023). The model was evaluated
tial of deep learning in reservoir characterization (Zhang et at 0.1 and 0.5 intersection over union (IoU) using the Aver-
al. 2020) was demonstrated to integrate seismic and elec- age Precision (AP) measure. Though promising with an AP
tromagnetic data for improved mapping. Extending beyond of 78.31% at the lower level, precision dropped to 31.05%
the realm of image analysis, (Afzal et al. 2023) delves into at the higher barrier.
the extensive landscape of visualization and visual analyt- Nevertheless, the limited resolution of climate data is
ics techniques empowered by deep learning. Additionally, inadequate for detecting variations in small climatic zones,
deep learning was extended in environmental monitor- such as India, which may experience cyclones of varying
ing (Hittawe et al. 2019) specifically focusing on anomaly magnitudes (Dabhade et al. 2021). Single Image Super-Res-
detection in sea surface temperatures. The remarkable per- olution (SISR) may be used to generate artificially enhanced
formance of hash deep learning model in multi-label remote High Resolution (HR) images, which can subsequently be
sensing image retrieval had been investigated (Moustafa employed to improve the accuracy of object detection sys-
et al. 2020). The convergence between DL and statisti- tems (Park et al. 2003; Anwar et al. 2020; Liu et al. 2021).
cal methods in optimizing traffic management solutions Dong et al. introduced the pioneer deep learning Convolu-
was explored (Harrou et al. 2021). By drawing inspiration tional Neural Network-based Super-Resolution (SRCNN)
and insights from these prior works, the present research method. More complex CNN architectures was introduced,
endeavors to contribute to the ever-evolving landscape of including VDSR (Kim, Kwon Lee et al. 2016) and LapSRN
DL applications. (Lai et al. 2018) which resulted in the production of SR
This paradigm shift has the potential to reshape the images with high Peak Signal-to-Noise Ratio (PNSR) val-
extreme weather analysis (Chen, Zhang et al. 2020). An ues. On other hand, generative adversarial networks (GANs)
ensemble of deep learning methods were utilized to detect has displayed enhancing the perceptual quality and mini-
cyclones (Kumler-Bonfanti et al. 2020) using a twenty-year mize smoothing of reconstructed HR images (Lei et al. 2019;
dataset of simulated data. Another convolutional neural Moustafa and Sayed 2021). Single Image Super-Resolution
network (CNN) architecture (Kim et al. 2017) was devel- Generative Adversarial Networks (SRGANs) leverage the
oped to accurately pinpoint severe occurrences, achieving collaborative power of two subnetworks: a generator and
a remarkable accuracy rate of 99.98%. ClimateNet (Kashi- a discriminator (Ledig et al. 2017). The generator network
nath et al. 2021) was created as a baseline dataset for anno- aimed to reconstruct HR images from their Low Resolution
tating the Community Atmospheric Model (CAM5.1). A (LR) input counterparts. On other hand, the discriminator
deep convolutional neural network (CNN) was specifically network anticipates whether the obtained image is ground
designed to categorize the intensity of Tropical Cyclones truth HR or not. After enough training, the generator creates
(TCs) using infrared geostationary satellite data. The Single HR images that mimic ground truth.

13
Earth Science Informatics (2024) 17:1837–1850 1839

Recently, attention-based models or transformers (Lu Extensive experiments were conducted to assess the effec-
et al. 2022) could be better feature extraction in local cli- tiveness and efficiency of the framework using four metrics:
mate zones, these techniques have shown great potential in precision, recall, intersection over union, and average preci-
various computer vision tasks, including super-resolution. sion. The key contributions of this work are:
Attention mechanisms enable the model to focus on rele-
vant image regions and capture long-range dependencies, ● The proposed end-to-end framework comprised of a
which can be beneficial in extracting meaningful features generator network equipped by residual-in-residual
from local climate zones. By attending to relevant spatial dense block (RRDB) and discriminator containing Fast-
or temporal regions, attention-based models can effectively er RCNN detector.
model the complex relationships and patterns within local ● The generator network employs residual-in-residual
climate zones, leading to improved performance. Trans- dense blocks (RRDB) which provides several advan-
formers, in particular, have gained significant attention in tages compared to traditional convolutional blocks al-
recent years due to their success in natural language pro- lowing extraction of discriminative features. In addition,
cessing and image recognition tasks (Moustafa and Sayed the skip connections of RRDB enhances gradient flow
2021). Despite their booming performance there are some during training.
challenges to be considered when utilizing attentions or ● The discriminator network contains Faster RCNN ob-
transformers in case of very large volumes of data: (1) ject detection where the gradient of the detection loss
Computational Cost: Transformers heavily rely on attention function is propagated back to update the parameters of
mechanisms, which involve comparing every element in the the generator network.
input sequence to each other. This leads to quadratic com- ● The proposed EESRGAN with can efficiently detects
plexity, meaning their computational cost grows with the the tropical cyclone (TC) event which has been verified
square of the data size. While techniques like sparse atten- for India.
tion and efficient implementations can alleviate this issue, it ● Seven critically important variables for cyclone
still remains a hurdle for extremely large datasets. (2) Mem- event analysis from Community Atmospheric Model
ory Bottlenecks: Processing entire large datasets at once (CAM5.1) image data have been taken into account for
might not be possible due to memory limitations. Trans- systematically assessment of the proposed network.
formers usually need the entire input sequence in memory
for attention calculations, making handling massive datasets The remainder of this paper is structured as follows: Sect. 2
in a single batch challenging. (3) Training Stability: Train- introduces the proposed architecture for the Indian cyclone
ing transformers effectively requires careful hyperparam- detection. Experimental setting, and results discussion are
eter tuning, especially with large datasets. Learning rate presented in Sect. 3. Section 4 concluded the findings.
schedules, batch sizes, and optimization algorithms need
to be adjusted to ensure convergence and avoid divergence
(Khan et al. 2022). Methodology
Traditional weather models struggle to accurately identify
cyclones due to two key hurdles: (1) their limited resolution, Figure 1 depicts the overall structure of the proposed frame-
meaning they cannot capture the fine details of cyclones, work. The proposed framework is composed of two main
and (2) the natural variation in cyclone size and structure. subnetworks: generator (G), extended discriminator net-
These limitations can lead to missed identifications, par- work with object detector network. During training, the
ticularly for smaller or weaker cyclones, impacting weather gradient of the detection loss function is propagated back
forecasting and early warning systems. This study tackles to update the parameters of the generator network (G). This
these challenges to improve cyclone detection for better backpropagation process guides the generator to refine its
weather forecasting and early warning systems. To address image reconstruction, enhancing realism and sharpness in
this challenge, we propose a novel end-to-end approach that the output images, ultimately improving the performance of
combines edge-enhanced super-resolution (EESRGAN) the overall framework. On the other hand, the discrimina-
with a Faster RCNN detector. The proposed framework tor network (D) aimed to distinguish between ground truth
comprises three subnetworks: a generator, a discriminator, images and estimated SR images whereas, the detector
and a Faster RCNN detector. We utilize residual-in-residual network Leverages the enhanced quality of the SR images
dense blocks (RRDB) to extract discriminative features for created by the generator (G) to perform accurate object
accurate cyclone detection. We systematically evaluated detection.
the proposed approach on Community Atmospheric Model
(CAM5.1) image data, considering seven distinct variables.

13
1840 Earth Science Informatics (2024) 17:1837–1850

Fig. 1 The overall structure of the proposed end-to-end cyclone detection network

Generator RRDB skip connections improve gradient flow during train-

ing. The RRDB combats the vanishing gradient problem
Building upon the EESRGAN architecture (Jiang et al. by shortening gradient propagation through the network,
2019), we utilized the generator structure outlined in enabling faster and more stable convergence during train-
Fig. 2(a). The key innovation lies in replacing the standard ing. To mitigate computational complexity, curtail unde-
convolution blocks with Residual in Residual Dense Blocks sirable artifacts, and bolster generalization capabilities in
(RRDBs) (Song et al. 2018), as detailed in Fig. 2(b, c), to scenarios where training and testing data exhibit substantial
enhance the generator performance. The inclusion of RRDB statistical disparities, batch normalization layers were judi-
in the network offers several advantages over traditional ciously excluded from the architecture (Karras et al. 2019).
convolutional blocks; (1) Improved feature representation: We stacked 16 RRDB block with dense connections to
RRDB architecture enables complicated and discriminative increase network capacity. To enhance parameter learning, a
feature extraction and representation. The residual con- Parametric Rectified Linear Unit (PReLU) (El Jaafari et al.
nections in the RRDB let the network capture and convey 2021) was implemented in conjunction with residual scaling
low-level and high-level information, improving feature promoting training stability. The PReLU activation func-
learning. (2) Deeper network capacity: RRDB allows for tion is an extension of the traditional rectified unit, offer-
deeper network building without many additional param- ing improved model fitting without significant additional
eters. Densely linking each layer to all subsequent layers computational cost or overfitting concerns. By dynamically
in the block achieves this. Thus, the RRDB may take use of learning the rectifier parameters, PReLU enhances accuracy
deep architectures improved representational capacity and without imposing a noticeable burden on computational
ability to learn abstract features. (3) Efficient gradient flow: resources (He et al. 2015). The initial super-resolution (SR)

13
Earth Science Informatics (2024) 17:1837–1850 1841

Fig. 2 (a) The generator network architecture with RRDBs and EESN network. (b) Residual in Residual Dense Block (RRDB) where ß is residual
scaling parameter. (c) The architecture of the dense block

rW rH
image generated by the network exhibits undesirable arti- 1 
LMSE = (IHR,(w,h) − G (ILR)w,h)2 (2)
facts manifested as noisy edges. The Edge Enhancement 2
r WH w=1
h=1
Sub-Network (EESN) mitigates these artifacts by replacing
the noisy edges with “EESN-purified” edges, yielding the
final refined SR image. During training, the generator (G) Where r represents the upsampling factor, W and H
aims to map the input LR image onto the HR image space, denoted HR image width and height, respectively. IHR ,
replicating the characteristics of the ground truth HR image. G (ILR) stands for the ground truth HR image and SR image.
While the intermediate generator output possesses sharp yet The LV GG loss, defined in Eq. (1), was originally intro-
jagged edges, the final SR image retains crisply defined con- duced by (Ledig et al. 2017) to create visually appealing
tours devoid of spurious artifacts. and detailed images. However, their VGG-19 network
The EESN network aims to remove the noise from the (Simonyan and Zisserman 2014)was trained on the Ima-
initial obtained SR images and sharpen the edges. Laplacian geNet dataset, which differs significantly from the domain of
operator is used to extract edges in the image then this edge satellite images used in this work. To address this mismatch,
information is transferred via convolutional, RRDB, and up we fine-tuned the pre-trained VGG-19 network following
sampling blocks. Following the architectures in (Jiang et the procedure in (Jiang et al. 2019), as shown in Eq. (3).
al. 2019), the mask branch equipped by sigmoid activation This allows us to calculate the Euclidean distance between
aimed to eliminate edge noise. Finally, the refined edges the feature maps extracted from the high-resolution (HR)
are added to the input image. It worth noting that all dense image ( IHR ) and the super-resolution (SR) image ( G (ILR) )
block in EESN were replaced by RRD blocks to improve using the fine-tuned network.
the performance. The generator network (G) consisted of
Wi,j Hi,j
16 RRDB while the EEN (Enhanced Encoder Network) 1 
Lvgg = (∅i,j(IHR)w,h − ∅i,j(G (ILR))w,h)2  (3)
employed five blocks. The overall generator (G) cost func- Wi,jHi,j w=1
h=1
tion ( LG ) is defined as in Eq. (1).
where Wi,jand Hi,j indicate the width and height of the cor-
LG = λ1LM SE + λ2LV GG + λ3LAdversarial + λ4LEESN (1) responding feature map respectively.
The discriminator network loss function can be formu-
where we prioritized content accuracy (λ1 = 1), downplayed lated as in Eq. (4):
perceptual details (λ2 = 0.001), used moderate adversarial
loss (λ3 = 0.01), and emphasized edge preservation (λ4 = 5). LAdversarial = −log (D( G (ILR )) (4)
The mean square loss LM SE defined in Eq. (2), is the
popular in SISR as it is known to increase the PSNR value. Finally, the EESN network loss function is formulated as
defined in Eq. (5):

13
1842 Earth Science Informatics (2024) 17:1837–1850

LEESN = EISR [P (IHR − ISR )] + EIedge_HR [P (Iedge_HR − Iedge_SR)] (5) LD_f = LD + LOD  (6)

where, the first term measures the pixel-wise difference The Adversarial decimator network (D) loss function is
between the generated SR image ( ISR ) and the ground truth defined in Eq. (7)
HR image ( IHR ). P represents the Charbonnier penalty func-
tion. The second term focuses on the preservation of edges LD = log(D (IHR ) − log(1 − D(G (ILR ))). (7)
in the super-resolved image Iedge_HR and Iedge_SR denotes
the edge maps of the HR and SR images, respectively. where I HR denotes the reference High resolution image,
I LR denotes the Low-resolution image
Discriminator The object detection network Faster RCNN loss function
is defined as in Eq. (8).
Building on the success of (Jiang et al. 2019), we designed
1  1 
a robust discriminator network crucial for achieving high- LOD ({pi } , {ti }) = Lcls (pi , ṗi) + λ ṗi Lreg (ti , )  (8)
Ncls i Nreg i
quality super-resolution. This network consists of eight con-
volutional layers with 3 × 3 filters, progressively increasing
in number from 64 to 512, inspired by VGGs architecture. Where pi is the predicted probability of anchor, ṗi is the
To further enhance discrimination, we incorporate VGG- ground-truth label (1: anchor is positive, 0: anchor is nega-
19 features and leverage Faster R-CNN (Girshick 2015) tive), λ is balancing parameter, ti is the predicted box, ṫi
for object detection within the discriminator, enabling it to is the ground-truth box.
effectively differentiate between super-resolved and high-
resolution images. Training strategy
Faster R-CNN (Girshick 2015), developed by Micro-
soft as a two-stage object detector, has gained significant To better suit climate data characteristics through model
popularity for its effectiveness in analyzing satellite images. training, we depended on data normalization and scaling
The model comprises of two interconnected subnetworks, as an important preprocessing step to ensure that the input
namely the region proposal network (RPN) and the detec- seven variable are on a similar scale, which can improve
tor. The primary task of the RPN is to identify and extract the training process and model performance. We applied
region-specific characteristics associated with objects Min-Max Scaling to physical climate parameter data which
of interest. Subsequently, these identified regions, along rescaled each variable to a fixed range, typically between 0
with their corresponding feature maps, are utilized by the and 1. This is achieved by subtracting the minimum value
detector’s classifier and bounding box regressor. To obtain of the variable and dividing by the difference between the
a fixed-size feature map encoding spatial relationships maximum and minimum values, as defined in Eq. (9):
between features, a fully convolutional network known as
the backbone is employed. The RPN can accommodate fea- xnorm = (x?xmin)/(xmax ?xmin ) (9)
ture maps of any size, leading to the generation of numer-
ous rectangular object proposals. For each sliding position To mitigate computational demands associated with train-
within the feature map, the RPN generates K predictions ing the proposed model on the entire dataset, we employed
encompassing diverse sizes and aspect ratios. The regres- a random sampling technique. This resulted in the creation
sion and classification layers produce four location coordi- of a smaller, representative subset of data that maintained
nates and corresponding scores. Consequently, the resulting balanced representation across all four class types, thereby
feature map of size n × n × k represents the regions of inter- ensuring training efficiency and generalizability.
est (ROIs). Through the process of minimizing and refining Instead of training the model from scratch, we benefited
regional proposals, the RPN contributes to improvements from Transfer Learning and adopted the weights from (Jiang
in both speed and accuracy. Several studies (Magdy et al. et al. 2019) as the initial weights then completed training
2022; Wang and Leelapatra 2022) have demonstrated the on climate dataset. This approach leverages the knowledge
superiority of ResNet-50-FPN as the backbone network for learned from the pre-training phase and reduces the amount
this task. This choice stems from its demonstrably higher of training required on the target dataset.
precision compared to VGG-19 and the baseline ResNet-50
architecture without FPN.
The overall discriminative network (D) which minimizes
the cost function is defined in Eq. (6)

13
Earth Science Informatics (2024) 17:1837–1850 1843

Table 1 Class labels for the type of Extreme climate (Cyclone) events Table 2 Dataset-Splitting for training, validation, and testing
Label Type of Extreme Climate Event Training set Validation set Testing set
1 Tropical depression Years 1979–2000 2000–2002 2002–2005
2 Tropical Cyclone #samples 20,616 11,907 9999
3 Extratropical Cyclone
4 Atmospheric River
critically important variables. A sample of the climate data-
set is illustrated in Fig. 3. To narrow down the data to a
Dataset specific region, the dataset was clipped to the extent of the
Indian subcontinent. To avoid overfitting and ensure gener-
The detection task utilized a large-scale Extreme Climate alizability, we split the data into three different sets: train-
Event dataset (Kashinath et al. 2021) specifically designed ing (50%), validation (27%), and testing (23%) as shown
for climate analysis. This dataset contains ground truth in Table 2. Finally, the generalizability of the model was
information for four types of extreme climate events and evaluated on a completely unseen test set (23% of the data),
was generated using the Parallel Toolkit for Extreme Cli- which was never exposed to the model during training or
mate Analysis (TECA), which leverages prior knowledge validation.
of climate analysis to create accurate labels. The dataset is
extensive and stored in a yearly HDF5 file format with a
size of 62GB. Each file consists of two variables: “images” Experiments setting
and “boxes.” The “images” variable has a shape of (1460,
16, 768, 1152), representing 1460 images with 16 channels, The computational environment for all experiments con-
a length of 768, and a width of 1152. On the other hand, sisted of an Intel Core i7 processor equipped with an
the “boxes” variable has a shape of (1460, 15, 5), signify- NVIDIA Quadro RTX 6000 graphics card (NVIDIA, 2023)
ing 1460 images with 15 ground-truth boxes per image. The and 192 GB of RAM. PyTorch (Paszke et al., 2019) served
5 coordinates in each box correspond to x_min, x_max, as the deep learning framework under Windows 10, with
y_min, y_max, and the associated class label. Table 1 pro- CUDA 11.0 and CUDNN 5.1 providing GPU acceleration.
vides a detailed mapping of the class labels for four cyclone Stochastic gradient descent (SGD) with momentum (Ruder,
classes. For Cyclone detection, the study focused on seven 2017) was employed as the optimizer, utilizing momentum

Fig. 3 Worldwide climate parameters generated using CAM5 at face (T500), (d) Zonal wind at 850 mbar pressure surface (U850), (e)
1/6/2001; (a) Sea level pressure (PSL), (b) Temperature at 200 mbar Meridional wind at 850 mbar pressure surface (V850), (f) Z100, and
pressure surface (T200), (c) Temperature at 500 mbar pressure sur- (g) Geopotential Z at 200 mbar pressure surface (Z200)

13
1844 Earth Science Informatics (2024) 17:1837–1850

values of 0.9 and 0.999. The learning rate was set to 1 × 10–4. Where, TP represents true positives, FP represents false
A batch size of 16 was chosen for training efficiency. The positives, and FN represents false negatives. True positives
training took 96 hours for 200 epochs. Faster R-CNN infers (TP) occur when the predicted cyclone type matches the
four images/second. Figure 4 shows the proposed network ground-truth, true negatives occur when the predicted and
training and validation loss curves. ground-truth are both negative, false positives occur when
Low-resolution (LR) training images were obtained by the predicted is positive, but the ground-truth is negative,
downsampling ground-truth images using bicubic interpola- and false negatives occur when the ground-truth is positive,
tion to a size of 128 × 128 pixels. Notably, the experiments but the predicted ground-truth is negative.
were conducted with a 4x scaling factor between the SR out-
puts and the ground-truth images. During training, both the
high-resolution (HR) and low-resolution (LR) images were Results
rescaled to the value ranges of [-1, 1] and [0, 1], respec-
tively. The VGG-19 network (Simonyan and Zisserman First, we evaluated the SSD and Faster RCNN detectors on
2014) was adapted to accept seven input channels instead both LR and HR images. VGG16 backbone was employed
of the original three by prepending additional zero channels. for SSD network while ResNet-50-FPN was employed
To assess the performance of our proposed architecture, for Faster R-CNN (FRCNN) detector. For each detector,
we utilized commonly used metrics for object detection the training and the testing was conducted on LR and HR
tasks, namely precision, recall, and IoU (Intersection over images. Table 3 summarizes the obtained detectors results
Union). These metrics are defined as follows: training/testing. Faster R-CNN achieved 79.7% AP when
adopting only LR images in training and testing. For both
P recision = T P/ (T P + F P ) (10) detectors, the obtained accuracy declined when trained on
HR images and tested with their LR counterparts.
Recall = T P/ (T P + F N ) (11) One can observe that the accuracy of both object detec-
tors excelled in scenario of utilizing HR images in train-
IoU = T P/ (T P + F N + F P ) (12) ing and testing. The accuracy achieved 74.1% and 81.9% in
terms of AP for SSD and Faster RCNN, respectively. This

Fig. 4 The loss curve per epoch

for Weather dataset. (a) Genera-
tor network, (b) discriminator
network

13
Earth Science Informatics (2024) 17:1837–1850 1845

Table 3 The obtained detection results in terms of AP (average preci- Table 5 The detection results in terms of AP using end-to-end training
sion) on LR and HR images for both detectors
Model Image Resolution (Train- Cli- Model Image Resolution Cli-
ing -Testing) mate (Training -Testing) mate
data- data-
set set
SSD LR-LR 61.8% Proposed framework + Faster RCNN SR-SR 86.3%
HR-LR 58% Proposed framework + SSD SR-SR 84.2%
HR-HR 74.1%
Faster-RCNN LR-LR 79.7%
backpropagated into the SR network in order to enhance
HR-LR 72.5%
the network learning during training. The LR-HR images
HR-HR 81.9%
pairs were utilized to train the proposed framework, and the
obtained SR images were used to train Faster RCNN detec-
Table 4 The detection results in terms of AP on the obtained SR
images by the proposed approach, CNNSR, SRGAN networks and tor. In testing, only LR images were feed to the generator
bicubic upsampling. Both Detectors are trained separately with both to create SR image to be feed to detector network. Table 5
SR and HR images indicates that the proposed approach improved outcomes
Model Image Resolution Climate compared with training the detector network with SR from
(Training -Testing) dataset
other SR approaches.
Bicubic + SSD SR-SR 58.9%
Figure 5 shows the precision-recall curves of the pro-
HR-SR 61.3%
posed approach, with and without end-to-end training, in
Bicubic + FRCNN SR-SR 62.6%
HR-SR 66.8% comparison to stand-alone Faster-RCNN using LR training/
SRGAN + SSD SR-SR 80.95% testing images. Precision and recall were determined using
HR-SR 83.23% IoU = 0.5. One can observe that the proposed framework
SRGAN + Faster RCNN SR-SR 81.68% has superior values in precision and recall than standalone
HR-SR 77.92% R-CNN models. End-to-end training improved the proposed
CNNSR + SSD SR-SR 72.83% method performance.
HR-SR 73.78% For better comparison and visualization, we plot
CNNSR + Faster RCNN SR-SR 78.84% (1-recall) for X-axes and (1- precision) for Y-axes, as shown
HR-SR 84.57% in Fig. 6. One can observe that, all detector techniques
Proposed SR-SR 84.2%
achieved superb performance for the four categories despite
framework + SSD HR-SR 86.08%
the size of the cyclone. Overall, the proposed method is very
Proposed frame- SR-SR 86.1%
work + Faster RCNN effective for detecting extreme climate event in the climate
HR-SR 88.2%
dataset.
The proposed approach yields an SR image with
illustrates how image resolution affects object identification improved visual clarity and detail, thanks to adversarial
quality. learning’s ability to simultaneously sharpen images and
Next, we compared the proposed EESRGAN architec- increase detection precision as shown in Fig. 7. In brief,
ture, CNNSR, SRGAN, and 4× HR estimate from LR image the effectiveness of joint training (detector network and
using bicubic upsampling. We trained each network sepa- discriminator), improves the obtained SR image images
rately. For the assessment, we compared detectors trained both visually and in detection measures. Also, the proposed
on SR images obtained from these approaches versus detec- approach achieved a considerable improvement compared
tors trained directly on HR images. Table 4 demonstrated with other approaches by about 1.5% in terms of average
that the proposed framework showed the highest results, precision (AP).
approaching close to HR-only detection rates. After training,
the proposed framework may be immediately used to LR
images without HR data and get excellent results. CNNSR Discussion
and SRGAN have better AP compared with traditional bicu-
bic in prepare LR images. Overall, the proposed framework The proposed approach, when tested with SR images gen-
outperformed the other approaches in climate dataset. erated by itself, improved the detection outcomes com-
Next, we trained the proposed approach using end-to-end pared to training the detector network with SR images from
fashion. The discriminator network and Faster RCNN detec- other approaches. It was evaluated using SSD and Faster
tor were served as the discriminator for the proposed archi- R-CNN as the detector networks. SSD utilized Vgg16
tecture. As a result, the Faster RCNN detector loss being backbone, while Faster R-CNN employed ResNet-50-FPN.

13
1846 Earth Science Informatics (2024) 17:1837–1850

Fig. 5 The precision-recall curves for the proposed technique, with and without end-to-end training, in comparison to stand-alone Faster-RCNN.

The accuracy of both detectors decreased when tested on ability to extract relevant features especially in local climate
LR images. However, the proposed approach utilizing zone. The transformer and attention-based model could help
Faster R-CNN and SSD achieved 81.9% and 74.1% AP. in capturing the discriminative features of cyclone events
A comparison was conducted between the EESRGAN efficiently. (3) Unlike data-driven deep learning-based, the
architecture, CNNSR, SRGAN, and bicubic upsampling traditional detection techniques employ physics parameters
for training detectors. The proposed approach showed the which deep learning-based algorithms disregard. Many
highest results, approaching the performance of HR-only studies strive to combine physics into the deep learning for
detection. CNNSR and SRGAN outperformed traditional climate forecasting to preserve the benefits of numerical and
bicubic upsampling in preparing LR images. Overall, the deep learning-based approaches to enhance deep learning-
proposed framework surpassed other approaches in the cli- based TC track detection.
mate dataset.
Therefore, there are still an open door to integrate recent
deep learning-based revelational models to boost the preci- Conclusion
sion of detection in the future. Technically, three main issues
had to be addressed in the future. (1) the deep learning-based Deep learning can unlock the power of climate data by ana-
detection methods mainly used pre-trained, but the nature of lyzing its low-resolution obtained from numerical models
climate data is different. Although the large volume of used instead of regional high-resolution counterparts. The pro-
data in training limited computation affects the model abil- posed approach tackles the challenges of the computational
ity to learn from data. (2) The obtained results in Table 5 burden and information overload to obtained high-reso-
demonstrate the rather poor performance of the detection lution regional data from weather numerical models. The
utilizing the super resolution images, especially using SSD integration between deep learning and numerical data can
detectors. The reason for this may be due to limited SSD offer faster analysis, targeted feature extraction, uncovering

13
Earth Science Informatics (2024) 17:1837–1850 1847

Fig. 6 Precision vs. Recall curves on climate dataset for Tropical Depression (TD), Tropical Cyclone (TC), Extratropical Cyclone (EC), and
Atmospheric River (AR), respectively

hidden patterns, broader applicability, and real-time insights. generative adversarial network (EESRGAN) coupled with
While acknowledging potential information loss and train- an end-to-end detector network. The proposed approach
ing data challenges, this approach empowers professionals comprises a generator, discriminator, and Faster RCNN
with efficient, scalable, and insightful climate analysis for detector network augmented with residual-in-residual dense
informed decision-making. blocks (RRDB). This architecture effectively extracts pre-
The intensifying impacts of climate change necessitate cise cyclone patterns, facilitating accurate boundary detec-
enhanced detection of extreme weather events (EWEs), par- tion. Extensive experiments were conducted on Community
ticularly cyclones. In the Indian subcontinent, the demon- Atmospheric Model (CAM5.1) data using only seven vari-
strably heightened frequency and severity of cyclones ables and employed four evaluation metrics: precision,
necessitate reliable detection methods for mitigating casu- recall, intersection over union, and mean average precision
alties and economic losses. However, traditional detection to assess the proposed approach. The results demonstrated
approaches face significant challenges due to the inherent remarkable effectiveness, achieving an accuracy of 86.3%
limitations of low-resolution data. Deep learning models and an average precision (AP) of 88.63%. Furthermore, the
present a promising solution by enabling precise identifi- proposed framework outperformed baseline object detector
cation of cyclone boundaries crucial for regional impact methods.
assessment using global climate model data. By leveraging
the power of deep learning, we can significantly improve
cyclone detection capabilities and contribute to refined risk
mitigation strategies in the vulnerable Indian subcontinent.
This paper introduces an edge-enhanced super-resolution

13
1848 Earth Science Informatics (2024) 17:1837–1850

Fig. 7 Examples of the obtained SR images generated from LR images in (a,b). Results of improved edge detection in (c, d)

13
Earth Science Informatics (2024) 17:1837–1850 1849

Author contributions Conceptualization: MR and RS; Methodology: Hittawe MM, Afzal S, Jamil T, Snoussi H, Hoteit I, Knio O (2019)
MSM and SAM. Investigation: MR and RS; Writing - Original Draft: Abnormal events detection using deep neural networks: applica-
MSM, MR and RS; Writing - Review & Editing: All authors; Supervi- tion to extreme sea surface temperature detection in the Red Sea.
sion: MSM and BS. J Electron Imaging 28(2):021012–021012
Hulme M (2014) Attributing weather extremes to ‘climate change’ a
Funding The authors declare no funding sources for this research. review. Prog Phys Geogr 38(4):499–511
Open access funding provided by The Science, Technology & Inno- Jiang K, Wang ZY, Yi P, Wang GC, Lu T, Jiang JJ (2019) Edge-
vation Funding Authority (STDF) in cooperation with The Egyptian enhanced GAN for remote sensing image Superresolution. IEEE
Knowledge Bank (EKB). Trans Geosci Remote Sens 57(8):5799–5812
Jiang K, Wang Z, Yi P, Wang G, Lu T, Jiang J (2019a) Edge-enhanced
GAN for remote sensing image superresolution. IEEE Trans
Data availability No datasets were generated or analysed during the
Geosci Remote Sens 57(8):5799–5812
current study.
Karras T, Laine S, Aila T (2019) A style-based generator architecture
for generative adversarial networks. Proceedings of the IEEE/
Declarations CVF conference on computer vision and pattern recognition
Kashinath K, Mudigonda M, Kim S, Kapp-Schwoerer L, Graubner
Competing interests The authors declare no competing interests. A, Karaismailoglu E, Von Kleist L, Kurth T, Greiner A, Mahesh
A (2021) ClimateNet: an expert-labeled open dataset and deep
Open Access This article is licensed under a Creative Commons learning architecture for enabling high-precision analyses of
Attribution 4.0 International License, which permits use, sharing, extreme weather. Geosci Model Dev 14(1):107–124
adaptation, distribution and reproduction in any medium or format, Kaur R, Singh S (2022) A comprehensive review of object detection
as long as you give appropriate credit to the original author(s) and the with deep learning. Digit Signal Proc : 103812
source, provide a link to the Creative Commons licence, and indicate Khan S, Naseer M, Hayat M, Zamir SW, Khan FS, Shah M (2022)
if changes were made. The images or other third party material in this Transformers in vision: a survey. ACM Comput Surv (CSUR)
article are included in the article’s Creative Commons licence, unless 54(10s):1–41
indicated otherwise in a credit line to the material. If material is not Kim J, Lee JK, Mu Lee K (2016) Accurate image super-resolution
included in the article’s Creative Commons licence and your intended using very deep convolutional networks. Proceedings of the IEEE
use is not permitted by statutory regulation or exceeds the permitted conference on computer vision and pattern recognition
use, you will need to obtain permission directly from the copyright Kim SK, Ames S, Lee J, Zhang C, Wilson AC, Williams D (2017)
holder. To view a copy of this licence, visit http://creativecommons. Massive scale deep learning for detecting extreme climate events.
org/licenses/by/4.0/. Climate Informatics
Kumler-Bonfanti C, Stewart J, Hall D, Govett M (2020) Tropical and
extratropical cyclone detection using deep learning. J Appl Mete-
orol Climatology 59(12):1971–1985
References Lai W-S, Huang J-B, Ahuja N, Yang M-H (2018) Fast and accurate
image super-resolution with deep laplacian pyramid networks.
Afzal S, Ghani S, Hittawe MM, Rashid SF, Knio OM, Hadwiger M, IEEE Trans Pattern Anal Mach Intell 41(11):2599–2613
Hoteit I (2023) Visualization and visual analytics approaches for Lam L, George M, Gardoll S, Safieddine S, Whitburn S, Clerbaux C
image and video datasets: a Survey. ACM Trans Interact Intell (2023) Tropical Cyclone detection from the Thermal Infrared
Syst 13(1):1–41 Sensor IASI Data using the Deep Learning Model YOLOv3.
Anwar S, Khan S, Barnes N (2020) A deep journey into super-resolu- Atmosphere 14(2):215
tion: a survey. ACM Comput Surv (CSUR) 53(3):1–34 Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A,
Chen R, Zhang W, Wang X (2020) Machine learning in tropical Aitken A, Tejani A, Totz J, Wang Z (2017) Photo-realistic single
cyclone forecast modeling: a review. Atmosphere 11(7):676 image super-resolution using a generative adversarial network.
Dabhade A, Roy S, Moustafa MS, Mohamed SA, Gendy RE, Barma Proceedings of the IEEE conference on computer vision and pat-
S (2021) Extreme Weather Event (Cyclone) Detection in India tern recognition
Using Advanced Deep Learning Techniques. 2021 9th Interna- Lei S, Shi Z, Zou Z (2019) Coupled adversarial training for remote
tional Conference on Orange Technology (ICOT), IEEE sensing image super-resolution. IEEE Trans Geosci Remote Sens
El Jaafari I, Ellahyani A, Charfi S (2021) Parametric rectified non- 58(5):3633–3643
linear unit (PRenu) for convolution neural networks. SIViP Liu Z-S, Siu W-C, Chan Y-L (2021) Features guided face super-resolu-
15(2):241–246 tion via hybrid model of deep learning and random forests. IEEE
Flaounas E, Davolio S, Raveh-Rubin S, Pantillon F, Miglietta MM, Trans Image Process 30:4157–4170
Gaertner MA, Hatzaki M, Homar V, Khodayar S, Korres G (2022) Lu Z, Li J, Liu H, Huang C, Zhang L, Zeng T (2022) Transformer
Mediterranean cyclones: current knowledge and open questions for single image super-resolution. Proceedings of the IEEE/CVF
on dynamics, prediction, climatology and impacts. Weather Clim conference on computer vision and pattern recognition
Dynamics 3(1):173–208 Magdy A, Moustafa MS, Ebied HM, Tolba MF (2022) Backbones-
Girshick R (2015) Fast r-cnn. Proceedings of the IEEE international Review: Satellite Object Detection Using Faster-RCNN. Inter-
conference on computer vision national Conference of Remote Sensing and Space Sciences
Harrou F, Zeroual A, Hittawe MM, Sun Y (2021) Road Traffic mod- Applications, Springer
eling and management. Using Statistical Monitoring and Deep Mezősi G (2022) Meteorological Hazards. Natural Hazards and the
Learning, Elsevier Mitigation of their Impact, Springer: 97–136
He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Moustafa MS, Sayed SA (2021) Satellite Imagery Super-resolution
Surpassing human-level performance on imagenet classification. using squeeze-and-excitation-based GAN. Int J Aeronaut Space
Proceedings of the IEEE international conference on computer Sci 22(6):1481–1492
vision

13
1850 Earth Science Informatics (2024) 17:1837–1850

Moustafa MS, Ahmed S, Hamed AA (2020) Learning to hash with Wang M, Leelapatra W (2022) A review of object detection based on
Convolutional Network for Multi-label Remote sensing image convolutional neural networks and deep learning. Int Sci J Eng
Retrieval. Int J Intell Eng Syst 13(5) Technol (ISJET) 6(1):1–7
Olaoluwa EE, Durowoju OS, Orimoloye IR, Daramola MT, Ayobami Wang C, Li X (2023) Developing a data-driven transfer learning model
AA, Olorunsaye O (2022) Understanding weather and climate to locate Tropical Cyclone centers on Satellite Infrared Imagery. J
extremes. Climate Impacts on Extreme Weather, Elsevier: 1–17 Atmos Ocean Technol 40(12):1605–1618
Pang S, Xie P, Xu D, Meng F, Tao X, Li B, Li Y, Song T (2021) Wang H, Xu Q, Yin X, Cheng Y (2024) Determination of low-intensity
NDFTC: a new detection framework of tropical cyclones from tropical cyclone centers in geostationary satellite images using
meteorological satellite images with deep transfer learning. a physics-enhanced deep-learning model. IEEE Transactions on
Remote Sens 13(9):1860 Geoscience and Remote Sensing
Park SC, Park MK, Kang MG (2003) Super-resolution image recon- Zaidi SSA, Ansari MS, Aslam A, Kanwal N, Asghar M, Lee B (2022)
struction: a technical overview. IEEE Signal Process Mag A survey of modern deep learning based object detection models.
20(3):21–36 Digit Signal Proc : 103514
Rübel O, Byna S, Wu K, Li F, Wehner M, Bethel W (2012) TECA: Zhang Y, Mazen Hittawe M, Katterbauer K, Marsala AF, Knio OM,
a parallel toolkit for extreme climate analysis. Procedia Comput Hoteit I (2020) Joint seismic and electromagnetic inversion for
Sci 9:866–876 reservoir mapping using a deep learning aided feature-oriented
Shi M, He P, Shi Y (2022) Detecting extratropical cyclones of the approach. SEG Technical Program Expanded Abstracts 2020,
northern hemisphere with single shot detector. Remote Sens Society of Exploration Geophysicists: 2186–2190
14(2):254
Simonyan K, Zisserman A (2014) Very deep convolutional networks Publisher’s Note Springer Nature remains neutral with regard to juris-
for large-scale image recognition. arXiv preprint arXiv:1409.1556 dictional claims in published maps and institutional affiliations.
Song T, Song Y, Wang Y, Huang X (2018) Residual network with
dense block. J Electron Imaging 27(5):053036–053036
Tong B, Sun X, Fu J, He Y, Chan P (2022) Identification of tropical
cyclones via deep convolutional neural network based on satellite
cloud images. Atmos Meas Tech 15(6):1829–1848

13