How to Cite:

Chakranarayan, A., & Damdoo, R. (2022). Overview on remote sensing image scene classification
using deep learning. International Journal of Health Sciences, 6(S1), 10388–10398.
https://doi.org/10.53730/ijhs.v6nS1.7492

Overview on remote sensing image scene

classification using deep learning

Anjali Chakranarayan
Department of Computer Science & Engineering, Shri Ramdeobaba College of
Engineering & Management, Nagpur, Maharashtra, India
Corresponding author email: [email protected]

Rina Damdoo
Department of Computer Science & Engineering, Shri Ramdeobaba College of
Engineering & Management, Nagpur, Maharashtra, India
Email: [email protected]

Abstract---Now a days, deep learning is becoming very famous dueto its power
fulability of learning. Deep learning has been applied for remote sensing image
analysis tasks including scene classification. Remote sensing has seen an
enormous increase in the enhancement of digital images captured from satellites
that cover almost all angle of the surface of the earth. This growth in data has
forced the community of the remote sensing to apply deep learning algorithms to
solve different remote sensing tasks. Convolutional neural networks have
obtained huge success in the field of scene classification. The current best scene
classification approaches are still far from achieving the human’s ability to
classify scene types with a few labelled samples. This study provides a
systematic survey of deep learning methods for remote sensing image scene
classification by covering mover than 9papers.Inthispaper, we provide a brief
discussion on Convolutional neural networks. Here we also discussed few-shot
learning algorithm to overcome the gap between the current understanding level
of machine and human-level performance.

Keywords---Deeplearning, Convolutional Neural Network, Few-shotlearning,

Training sets.

Introduction

Deepleaningisabranch ofmachinelearningwhich inturn a branch of an artificial

intelligent. It is implemented
withthehelpofNeuralNetworks,andtheideabehindthemotivation of Neural Network
is the biologicalneurons,
whichisnothingbutabraincell.RemoteSensingisatechniqueusedtocollectdataabout

International Journal of Health Sciences ISSN 2550-6978 E-ISSN 2550-696X © 2022.

Manuscript submitted: 27 March 2022, Manuscript revised: 18 April 2022, Accepted for publication: 9 May 2022
10389
10390

theearthwithouttakingphysicalsampleof the earth’s surface. This technology is

used
innumerousfieldslikegeography,hydrology,ecology,oceanography,glaciology,geolo
gy, etc.
Inthissection,weprovidedabriefdiscussiononConvolutionalneuralnetworks.Theres
tofthepaperisorganizedasfollows.Literature review
section,discussesthesystematicliteraturesurveyondeeplearningmethodsforremote
sensing
imagesceneclassificationbycovering9researchpapers.Abriefintroductionandworki
ngoffew-shotlearningwithexample is provided in Proposed word. Finally,
weconclude thisarticleinConclusion.

Deeplearningmodels:

1. Deepbeliefnetworks(DBNs)
2. Auto-encoders
3. Stackedauto-encoders
4. Convolutionalneuralnetworks(CNNs)
5. Generativeadversarialnetworks(GNNs)
6. Recurrentneuralnetworks(RNNs)

ConvolutionalNeuralNetworks:

Convolutionalneuralnetworks,alsoknownasCNNs,area special form of neural

network designed for processing datathat has a known grid-like representation,
for example imagedata, which can be considered a two-dimension (2D) grid
ofpixels. CNNs are a kind of multilayer network with learningability that consists
of convolutional layers, pooling layers, andfullyconnectedlayers.

Convolutionallayers:
Theselayersareusedforfeatureextraction fromimages.

Pooling layers: These layers are to execute a max or averageoperation over a

small area of each input feature map,
poolinglayersareappliedbetweentwosuccessiveconvolutionallayers.

Fullyconnectedlayers: These layers

areusuallyappearedinthetoplayerofCNNs,whichcansummarizethefeaturesextracte
dfromthebottomlayers.

Softmax layer: The SoftMax classifier is used to normalize thefully connected

layer output between 0 and1. It is
basicallyusedtoidentifytheoutputwithhighestprobability.
 10391

Fig 1: Convolutional Neural Network

Literature review

LazharKheliftetal.intheyear2020introducedthefundamentals of deep learning

methods which are frequentlyadopted for change detection. This paper
concentrates on
deeplearningchangedetectionapproachesforremotesensingimagesbyprovidingagen
eraloverviewoftheexistingmethods[1].

Intheyearof2020,GongChenget
al.discussedthechallenges,Methods,Benchmarks,andOpportunitiesforRemoteSen
singImageSceneClassification.Inthispaper,authorsweresurveyedthreekindsofdeep
learning-basedmethodsindetailandintroducedthemainstreamsceneclassification
benchmarks[2].

Hamid Ghanbariet al.in 2021, they discussedameta-

analysisof416peerreviewedjournalarticles,summarizesConvolutional Neural
Networks advancements, and its currentstatus
underremotesensingapplications[4].

Lei Ma et al.in 2019,authorscovernearly every

applicationandtechnologyinthefieldofremotesensingusingdeeplearning.Themajor
Deeplearningconceptspertinenttoremote-sensingareintroduced[5].

DanfengHonget al.in2020,theyshowdifferentfusionstrategies as well as how to

train deep networks and build thenetwork architecture. Specifically, five fusion
architectures areintroduced and developed, further being unified in our
MDLframework [3]

Gong Cheng et al.in 2016,introduced adeep

convolutionalneuralnetwork(CNN)basedsceneclassificationmethod,whichadopted
theexistinghigh-capacityCNNmodelstoextractmorericherhigh-
levelimagerepresentationfeatures[7].
10392

Gong Cheng et al. in 2019, they have tackled the problem

ofHRSSsceneclassificationusingasmallpoolofunlabelledimages and only a few
labelled images per category, namely,few-
shotsceneclassification(FSSC),whichismorechallengingthancommonsceneclassifi
cationtask[6].

Fengpenget al.in 2020,Akey regionorlocationcapturingmethodcalledkey

filterbank(KFB),ThismethodcancombinewithdifferentCNNmodels to improve the
performance of HRRS imagery sceneclassification[8].

DalaAlajaji,et al.in 2019, explained the deep few-shot learning methodfor

classification of Remote Sensing image under the N-wayfew-
shotlearningscenario.Theproposedmethodusedonprototypical deep neural
networks which was combined withpre-trainedconvolutionalneuralnetwork[9].

M.E. Paoletti, et al. in the year of 2017, the authors present

anewconvolutionalneuralnetworksarchitecturefortheclassification of
hyperspectral images. The proposed CNN is a3-Dnetwork that
usesbothspectralandspatialinformation.Theresultsindicatethattheproposednetwo
rkperforms
accuratelyandefficientlywhencomparedwithothertraditionalArtificialneuralnetwor
ks[16].
In the year 2017, Xingrui Yu et al., introduced a basic
dataaugmentationoperationtoaddressthefundamentaldatalimitationprobleminap
plyingdeeplearningforremotesensingimageprocessing.Toimprovetheremotesensin
gsceneclassificationperformanceofConvolutionalneuralnetworks,theauthorshasd
escribedhowtousedataaugmentation[14].

QingWangetal.intheyearof2018,introducedafasterregion-
basedconvolutionalneuralnetworks(FasterR-CNN)forthedetectionofhigh-
resolutionremotesensingimagechange. Faster R-CNN method Achieve higher
accuracy andkappacoefficient[17].
 10393

Table1
Literature Review
10394

Proposed Work

This paper proposed to use few-shot learning method forscene classification.

Few-shot learning is a meta-learning.
Thismodelhaslearnedthesimilarityanddifferencebetweenobjects. The idea is to
train a function that predicts similarity.The major goal is not to let the model
recognize the images inthe training set and then generalize to the test set
instead, thegoal is to learn. The proposed model shown infigure2,
whichisfurtherexplainin theexample.

Fig2: WorkFlowofthemodel

Forexample,thetrainingset(Fig3.)doesnothaveafootballfieldclass,sothemodeldoesn
otknowthatthequery sample(Fig4.).Weneedtoprovidethemodel
withmoreinformation.Wecanshowthecardstothemodel.Everycardhasanimageanda
name.Thesetofcardsisknownassupportset(Fig5).
 10395

Fig3: Trainingsetsample

By comparing the query with the cards, the model finds

thequerymostsimilartothefootballfield.So,themodelpredictsthatthequeryisafootbal
lfield.

Fig4: Querysample

Fig5: Supportset
10396

Terminologies of few-Shot learning

Wayandshotareterminologiesoffew-shotlearning are:
a) K-way:thesupportsethaskclasses.
b) N-shot:everyclasshasnsamples.
c) Infig.6,Kis6,Nis3.Therefore6-way3-shot.

Fig6: Supportset

Similarity function
a) Thebasicideaoffew-shotlearningistotrainafunctionthatpredictssimilarity.
b) Similarityfunctioncanbedenotedbysim(x,x’)(figure7).
c) sim(x1,x2)=1,sim(x2,x3)=0,sim(x1,x3)=0(figure7).
d) First,learnasimilarityfunctionfromalarge-scaletraining dataset.
e) Similarityfunctioncanbeusedaftertrainingformaking
predictionforunseenqueries.

Fig7: Example of similarity function

 10397

Conclusions

Afterdoingaliteraturesurvey,weconcludethatthere isaneedofexploringfew-
shot/zero-shotlearningapproachforscene classification. In this paper, we present
the classificationof remote sensingimage under the N-way few-shot
learningscenario.Convolutionalneuralnetworks(CNNs)haveobtained huge success
in the field of scene classification. Thefutureworkincludesthesolutionforfew-
shotlearningscenariosassupposedtoone-shotonly.

Acknowledgements

We are grateful to all the authors in the reference section, who gave us a way to
move in that direction. They all had used enormous methods for remote sensing
image scene classification.

References

1. LazharKhelifi,andMaxMignotte,“DeepLearningforChangeDetectioninRemoteS
ensingImages:ComprehensiveReviewandMeta-
Analysis”,DepartmentofComputerScienceandOperations Research, Montreal
University, Montreal, QC H3C 3J7, Canada, 2020.
2. Gong Cheng, et
al.“RemoteSensingImageSceneClassificationMeetsDeepLearning:Challenges,
Methods,Benchmarks,andOpportunities”,IEEEtransactionsongeoscienceand
remotesensing,vol.13, 2020.
3. DanfengHong,et
al.“MoreDiverseMeansBetter:MultimodalDeepLearningMeetsRemote-
SensingImageryClassification”, IEEE transactions on geoscience
andremotesensing,vol.13,2020.
4. HamidGhanbari,et al. “AMeta-Analysis of Convolutional Neural Networks
forRemote Sensing Applications”, IEEE transactions
ongeoscienceandremotesensing,vol.14,2021.
5. LeiMa,et al. “Deep learning inremotesensingapplications:Ameta-
analysisandreview”, Vision Laboratory, Department of
ComputerScienceandOperationsResearch,MontrealUniversity,Montreal,QCH
3C3J7,Canada,2020.
6. Gong Cheng, et al. “SceneClassificationofHigh-
ResolutionRemoteSensingImagesviaSelf-PlacedDeepLearning”,IEEEjournalof
selectedtopics inapplied earth observation and remote sensing, vol.13,2019.
7. Gong Cheng, et al. “SceneClassificationofHigh-
ResolutionRemoteSensingImageUsingConvolutionalNeuralNetworks”,IEEEjo
urnalofselectedtopicsinappliedearthobservationandremotesensing,vol.13,201
6.
8. Fengpeng Li, et al. “High-
ResolutionRemoteSensingImageSceneClassificationviaKeyFilterBankBasedo
nConvolutionalNeuralNetwork”,IEEEtransactionson geoscienceand
remotesensing,2020.
9. DalaAlajaji,et al.“Few-
ShotlearningforRemoteSensingSceneClassification.”,IEEEXplore,2020.
10398

10. FatihOzyurt,“Efficientdeepfeatureselectionforremotesensingimagerecognition
withfuseddeeplearningarchitectures”,SpringerScience+BusinessMedia,LLC,p
artofSpringerNature2019.
11. QimingQinDapingLiuHaitaoLiu,“ImagesFusinginRemoteSensingMapping”,IE
EE,2001.
12. J.Pyoetal.“Aconvolutionalneuralnetworkregressionforquantifyingcyanobacter
iausinghyperspectral imagery,” Remote Sens. Environ.,
vol.233,2019,Art.no.111350.
13. L. Zhang, et al. “Deep learning forremote sensing data: A technical tutorial
on the state-of-the-art,” IEEE Geosci. Remote Sens. Mag., vol. 4,no.2,pp.22–
40,Jun.2016.
14. Xingrui Yu, et al. “Deeplearning in remote sensing scene
classification:adataaugmentationenhancedconvolutionalneuralnetwork
framework”, GIScience & Remote Sensing,2017Vol.54,No.5No.5,741–
758,2017.
15. ZhipengDeng,et al. “Multi-scale object detection in remotesensing imagery
with convolutional neural
networks”,ISPRSJournalofPhotogrammetryandRemoteSensingVolume145,Pa
rtA,November2018.
16. M.E. Paoletti, et al. “A
newdeepconvolutionalneuralnetworkforfasthyperspectralimageclassification”
,InternationalSocietyforPhotogrammetryandRemoteSensing,Inc.(ISPRS),2017
.
17. Qing Wang, et al. “ChangedetectionbasedonFasterR-CNNforhigh-
resolutionremotesensingimages”,SimpleourEnvironment&Agriculture
journals,2018.
18. D.Duarte,et al,“Multi-resolutionfeaturefusionforimageclassi_cation of
building damages with convolutionalneural networks,'' Remote Sens., vol.
10, no. 10, p.1636, 2018.
19. M.Qin,et al.
“Dehazingformultispectralremotesensingimagesbasedonaconvolutionalneura
lnetworkwiththeresidualarchitecture,”IEEEJ.Sel.TopicsAppl.Earth Observ.
Remote Sens., vol. 11, no. 5, pp. 1645–1655, Mar.2018.