The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018

ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

SOIL MOISTURE RETRIEVAL USING CONVOLUTIONAL NEURAL NETWORKS:

APPLICATION TO PASSIVE MICROWAVE REMOTE SENSING

Zhongzheng Hu1, Linlin Xu1*, Bowen Yu1,

1CUGB, School Of land Science And Technology , Beijing, China

[email protected], [email protected], [email protected]

Commission III, WG III/2

KEY WORDS: Soil Moisture Retrieval, Microwave Remote Sensing, Deep Learning, Convolutional Neural networks, GPU

ABSTRACT:

A empirical model is established to analyse the daily retrieval of soil moisture from passive microwave remote sensing using
convolutional neural networks (CNN). Soil moisture plays an important role in the water cycle. However, with the rapidly increasing
of the acquiring technology for remotely sensed data, it's a hard task for remote sensing practitioners to find a fast and convenient
model to deal with the massive data. In this paper, the AMSR-E brightness temperatures are used to train CNN for the prediction of
the European centre for medium-range weather forecasts (ECMWF) model. Compared with the classical inversion methods, the deep
learning-based method is more suitable for global soil moisture retrieval. It is very well supported by graphics processing unit (GPU)
acceleration, which can meet the demand of massive data inversion. Once the model trained, a global soil moisture map can be
predicted in less than 10 seconds. What's more, the method of soil moisture retrieval based on deep learning can learn the complex
texture features from the big remote sensing data. In this experiment, the results demonstrates that the CNN deployed to retrieve
global soil moisture can achieve a better performance than the support vector regression (SVR) for soil moisture retrieval.

1. INTRODUCTION Corresponding to this, a lot of traditional soil moisture retrieval

algorithms were proposed, such as SMOS Level 2 and Level 3
A methodology to retrieve soil moisture from passive operational algorithms, SMAP Level 2 operational algorithm,
microwave remote sensing with convolutional neural networks the dual channel algorithm (DCA), land parameter retrieval
(CNN) is presented. Soil moisture is a very important variable model (LPRM), multi-orbit retrievals of soil moisture and
of water cycle, which can be used to predict the drought or crop optical depth (MT-DCA), and various other retrieval algorithms
yield in agriculture. Simultaneously, in some ongoing based on neural networks or local regressions (Wigneron et al.,
engineering construction (like housing, railway, road and bridge 2017). There are many methods to retrieve soil moisture, but
etc.), it also plays an important role. However, with the rapidly they cannot meet the demands of remote sensing big data. So,
increasing of the acquiring technology for remotely sensed data, it's a hard task for remote sensing practitioners to find a fast and
there are lots of different sensors for soil moisture retrieval, and convenient model to retrieve soil moisture with the massive data.
the remotely sensed soil moisture products include: In this paper, a deep learning based empirical model is well
(a) The national aeronautics and space administration (NASA) used for soil moisture retrieval.
soil moisture product (Tuttle et al., 2014);
(b) The land parameter retrieval model (LPRM) soil moisture Presently, a suitable method to inverse soil moisture by passive
product (Dall'Amico et al., 2009); microwave remote sensing is deep learning in the context of
(c) The Japan aerospace exploration agency (JAXA) soil remote sensing big data. Deep learning is a class of machine
moisture product (Cui et al., 2016); learning algorithm. It is capable of “learning features” from the
(d) The JAXA AMSR2 soil moisture product (Lu et al., 2017); data at each level and processing the complex input data and
(e) The soil moisture and ocean salinity soil moisture (SMOS) learning tasks. With the rapid increasing of remote sensing data,
product (Kerr et al., 2012); traditional retrieval methods, either empirical or physical
(f) The advanced scatterometer (ASCAT) soil moisture product models, cannot satisfy the requirement of the high complexity
(Wagner et al., 2013); of the data and the non-linearity of retrieval problems (Ali et al.,
(g) The essential climate variable (ECV) soil moisture product 2015). Therefore, it is vital to develop the advanced method for
(Wang et al., 2016); inversion of soil moisture based on the deep learning technique.
(h) The European centre for medium-range weather forecasts Compared with the classical inversion methods, the deep
(ECMWF) soil moisture product (Buizza et al., 1999). learning-based method is more suitable for soil moisture
The first three products (a-c) are derived from the advanced retrieval.
microwave scanning radiometer — earth observing system
(AMSR-E) (Zeng et al., 2015), and other five products (d-h) are
Support vector regression (SVR) is a popular approach in the
retrieved from other different sensors.
field of geo-/bio-physical parameter retrieval, which however
only has the good intrinsic generalization ability and the
* Corresponding author robustness to noise in the case of limited availability of the
reference samples (Durbha et al., 2007). In addition, traditional

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-3-583-2018 | © Authors 2018. CC BY 4.0 License. 583
 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018
ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

models is not flexible enough to learn more about feature The data used in this paper are as above (Table 1), all the
information. Therefore, it is of great advantage to investigate samples are patches of size 9×9. The images from July 1st to
deep learning based soil moisture retrieval approach in 30th are used as training and validation sets, and there are 35000
comparison with classical algorithms. image patches every day. 30000 pairs image patches are used to
train CNN model, and the rest are validation sets. For
What’s more, the method of soil moisture retrieval based on evaluating the quality of the model, the 50000 image patches
deep learning is very well supported by graphics processing unit from July 31th are used to predict the global soil moisture map.
(GPU) acceleration, which can meet the demand of massive
data inversion. However, traditional retrieval algorithms don’t
support GPU parallel acceleration, which will waste more 2.1 Advanced Microwave Scanning Radiometer-Earth Obs-
resources in the inversion of soil moisture. Once trained, the erving System (AMSR-E)
deep learning model is a very efficient method to retrieve soil
The AMSR-E was a passive microwave sensor which observes
moisture (Rodríguez-Fernández> et al ., 2015). As a result, it is
brightness temperatures at 6.9, 10.7, 18.7, 23, 37 and 89 GHz at
of great value to study using GPU-accelerated soil moisture
vertical and horizontal polarizations (Kolassa et al., 2017). The
inversion based on the deep learning method.
sensor was flown aboard the aqua satellite between 2002 to
2011 with a revisit time of 1-2 days. In this paper, the 6.9 GHz
For the first time, this paper uses deep learning in the inversion and 10.7 GHz channels were used, because they have the
of soil moisture content, which can learn the complex features strongest sensitivity to soil moisture. In order to use the AMSR-
from the big remote sensing data better and retrieve soil E data as input to the CNNs, a pre-processing step is necessary
moisture in real time compared with classical algorithms. The to eliminate outliers which will have a bad influence on the
CNN algorithm used in this research is composed of three pairs precision of training.
of convolution layers and pooling layers with one fully
connected layer on top, whose activation function of the top 2.2 ERA-interim
layer is changed from softmax loss layer to Euclidean loss layer.
The AMSR-E brightness temperature images are used as input, Modelled soil moisture fields from ERA-interim have been used
and the soil moisture value gained from ECMWF model which to calibrate the retrieval and to assess the retrieval performance.
is considered the most accurate value of soil moisture content is The soil moisture value gained from ECMWF model which is
used as ground truth. In this experiment, one month’s global considered the most accurate value of soil moisture content is
data which include 30 pairs of images is used to train the deep used as ground truth. The ECMWF products used in this work
learning model, and then it is used to predict the next month’s are operational integrated forecasting system (IFS) models with
data for soil moisture maps. By comparing the root-mean- the “hydrology-improved tiled ECMWF scheme for surface
square error (RMSE) and the R-square (R^2) with SVR, the exchanges over land” (H-TESSEL) (Slingo, 1987). In this paper,
experiment demonstrates the deep learning method for soil the data is provided on a regular grid of 0.25°spatial resolution,
moisture retrieval can achieve better learn the complex and there are 12 time steps per day and location. The average of
relationship between the observations and the ground truth and these 12 time steps has been used here to get a daily modelled
achieve better generalization performance compared with soil moisture estimate. In this study, only the upper layer soil
traditional retrieval algorithms. moisture, representing a depth of 7 cm, has been used, which is
a few centi-meters deeper than the typical penetration depth of
This paper is organized as follows. Section 2 provides the the microwave satellite data.
datasets used in this study. Section 3 describes the methodology
employed to establish CNN and train regression model for soil 3. METHODOLOGY
moisture retrieval. Section 4 discusses the experiment and the
results of soil moisture retrieval using CNN model. Section 5 In this paper, a supervised CNN model is used as a feature
summarizes the conclusions of the study. extractor and a regression model to retrieve the daily global soil
moisture from the brightness temperature. The processing
2. DATA scheme is shown in Figure 1. It is composed of three major
steps: preprocessing of the images, training of the CNN model,
The soil moisture retrieval is performed using data from a and prediction in new brightness temperature using the trained
passive microwave sensor, AMSR-E, as well as modelled soil CNN model. Only the first steps will be described, because the
moisture reanalysis from ERA-interim in the ECMWF model prediction follows the same as the testing.
which is a global atmospheric reanalysis from 1979,
continuously updated in real time. The study focuses on one 3.1 Preprocessing of the Images
month's global data in July, 2011. All data are projected onto an
equal area grid with a 0.25°resolution at the equator. All the images employed to train CNN model need to be
reprocessed, and the major steps includes image registration,
Method Data Training Validation Testing spatial interpolation, and normalization. Because the one day’s
data cannot overlap the whole world, it was a vital step to
AMSR-E 30×30000 30×5000 1×50000 registrate 1-2 day’s AMSR-E data for obtaining the average
CNN images. In addition, the brightness temperature data and soil
ECMWF 30×30000 30×5000 1×50000 moisture truth value from ECMWF had different resolution. In
this paper, the soil moisture truth value images grid with a 0.5°
AMSR-E 30×30000 30×5000 1×50000 resolution have been projected onto an equal area grid with a
SVR 0.25°resolution the same as brightness temperature data. Then
ECMWF 30×30000 30×5000 1×50000
the inputs and outputs are normalized to [0,1]. After
Table 1. The samples used in this experiment.

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-3-583-2018 | © Authors 2018. CC BY 4.0 License. 584
 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018
ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

reprocessing, many patch images of size 9×9 were generated, (7-2+1,7-2+1,100). The following pooling layer is a array
the number of which is given in Table 1. which size is 2×2 and stride 2. Its feature map is (3,3,100). The
third convolution layer is composed of 200 filters that size is
2×2 and stride 1. Its outputs feature map is (2,2,200). In this
paper, the hidden layer (the first fully connected layer shown as
Fig. 2) of dimension 500×1 follows the third convolution
instead of the max polling layer. The second fully connected
layer takes outputs only one single value, which is the truth
value of soil moisture. By modifying the activation function of
the top layer , the classification CNN model is transferred to
CNN regression model. In this study, the top layer is changed
from softmax loss layer to Euclidean loss layer.

4. DISCUSSION

In this paper, the follow images show the result of the soil
moisture retrieval using CNN model and using SVR.

Figure 1. The flowchart of the major steps of the algorithm.

Figure 3. The soil moisture prediction using the CNN model.

3.2 Structure of CNN

Multilayer CNNs are a very efficient method to image classi-

fication. In this paper, the CNNs is employed to find a function
linking a set of input data to SM. It can satisfy the requirement
of the high complexity of the data and the non-linearity of
retrieval problems.

Figure 4. The soil moisture prediction using the SVR model.

Figure 2. The structure of the proposed CNN model.

The CNNs used in this study has three pairs of convolution

layers and pooling layers with one fully connected layer on top,
whose activation function of the top layer is changed from
softmax loss layer to Euclidean loss layer, as shown in the
picture (Figure. 2). The first convolutional layer has 20 filters of
width and height set to two. Inputs to the CNNs are image
patches of size 9×9×4; thus, the size of the filter is (2,2,4). The
output of the first convolution is the array of size (9-2+1, 9-2+1, Figure 5. The soil moisture ground truth from AMSR-E.
20), that is (8,8,20). The first convolution layer is followed by a
max pooling layer of pooling size 2×2 and stride 1. After the
transfer of max pooling layer, the size of the image feature maps As shown in Figure 3-5, the prediction map using the CNN
changed to (7,7,20). The second convolution layer contains model is closer to the truth label of global soil moisture (Figure.
100 filters of size (2,2,20). It outputs convolved image of size 5). In some parts of southern and northern Africa, as well as

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-3-583-2018 | © Authors 2018. CC BY 4.0 License. 585
 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018
ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China

Australia, the CNN model performs better against the SVR Cui, H., Jiang, L., Du, J., Wang, G., & Lu, Z. (2016).
model. The results suggest that the CNN (Figure 3) deployed to Assessment of QP model based two channel algorithm with
retrieve global soil moisture can achieve a better performance JAXA, LPRM soil moisture products over Genhe area in China.
than the SVR (Figure 4) for soil moisture retrieval. Geoscience and Remote Sensing Symposium (pp.1663-1666).
IEEE.
R^2 RMSE
Lu, Z., Chai, L., Zhang, T., Cui, H., & Li, W. (2017).
CNN 0.6512 0.1272 Evaluation of amsr2 retrievals using observation of soil
SVR 0.5598 0.1817 moisture network on the upper and middle reaches of heihe
river basin. Remote Sensing Technology & Application.
Table 2. The RMSE and R^2 of CNN and SVR.
Kerr, Y. H., Waldteufel, P., Richaume, P., Wigneron, J. P.,
In this study, as the above Table 2 shows, by comparing the Ferrazzoli, P., & Mahmoodi, A., et al. (2012). The smos soil
root-mean-square error (RMSE) and the R-square (R^2), the moisture retrieval algorithm. IEEE Transactions on Geoscience
CNN model have a R^2 (0.6512) higher than the SVR (0.5598), & Remote Sensing, 50(5), 1384-1403.
and the RMSE (0.1272) is less than the latter (0.1817). What’s
more, the prediction using the CNN model takes less time than Wagner, W., Hahn, S., Kidd, R., Melzer, T., Bartalis, Z., &
the SVR model in global soil moisture retrieval. Hasenauer, S., et al. (2013). The ascat soil moisture product: a
review of its specifications, validation results, and emerging
5.CONCLUSION applications. Meteorologische Zeitschrift, 22(1), 5-33.

In this paper, a CNN has been employed to AMSR-E brightness Wang, S., Mo, X., Liu, S., Lin, Z., & Hu, S. (2016). Validation
temperatures images to retrieve the global soil moisture daily. and trend analysis of ecv soil moisture data on cropland in north
The CNN used took the image patches of brightness china plain during 1981–2010. International Journal of Applied
temperature data as input and output soil moisture value directly. Earth Observation & Geoinformation, 48(48), 110-121.
When compared to classical SVR approach, the CNN method
achieved soil moisture results that are closer to the ground truth Buizza, R., Milleer, M., & Palmer, T. N. (1999). Stochastic
map. representation of model uncertainties in the ecmwf ensemble
prediction system. Quarterly Journal of the Royal
The training on 31 images took about 2 hours, the prediction of Meteorological Society, 125(560), 2887–2908.
soil moisture on one image took less than 10 seconds using an
Nvidia GTX 1080Ti graphics card. With more powerful or more Wigneron, J. P., Jackson, T. J., O'Neill, P., Lannoy, G. D.,
quantity graphics cards, the time spent on training and Rosnay, P. D., & Walker, J. P., et al. (2017). Modelling the
predicting will be reduced largely. Once the model between the passive microwave signature from land surfaces: a review of
brightness temperature and soil moisture trained, the soil recent results and application to the l-band smos & smap soil
moisture retrieval can run in parallel and multiple GPUs easily. moisture retrieval algorithms. Remote Sensing of Environment,
However, the prediction of soil moisture on one image using 192, 238-262.
SVR took more than two minutes. Therefore, comparing with
traditional regression approaches, CNN had a great advantage Ali, I., Greifeneder, F., Stamenkovic, J., Neumann, M., &
on both prediction accuracy and computational cost for the Notarnicola, C. (2015). Review of machine learning approaches
retrieval of soil moisture from big remote sensing data. for biomass and soil moisture retrievals from remote sensing
data. Remote Sensing, 7(12), 221-236.
ACKNOWLEDGEMENTS
Durbha, S. S., King, R. L., & Younan, N. H. (2007). Support
This work was supported by the National Natural Science vector machines regression for retrieval of leaf area index from
Foundation of China Grant 41501410. multiangle imaging spectroradiometer. Remote Sensing of
Environment, 107(1), 348-361.
REFERENCES
Rodríguez-Fernández>, N. J., Aires, F., Richaume, P., Kerr, Y.
Zeng, J., Li, Z., Chen, Q., Bi, H., Qiu, J., & Zou, P. (2015).
H., Prigent, C., & Kolassa, J., et al. (2015). Soil moisture
Evaluation of remotely sensed and reanalysis soil moisture
retrieval using neural networks: application to smos. IEEE
products over the tibetan plateau using in-situ observations.
Transactions on Geoscience & Remote Sensing, 53(11), 5991-
Remote Sensing of Environment, 163, 91-110.
6007.
Tuttle, S. E., & Salvucci, G. D. (2014). A new approach for
Kolassa, J., Gentine, P., Prigent, C., Aires, F., & Alemohammad,
validating satellite estimates of soil moisture using large-scale
S. H. (2017). Soil moisture retrieval from amsr-e and ascat
precipitation: comparing amsr-e products. Remote Sensing of
microwave observation synergy. part 2: product evaluation.
Environment, 142(3), 207-222.
Remote Sensing of Environment, 195, 202-217.
Dall'Amico, J. T., Loew, A., Schlenz, F., & Mauser, W. (2009).
Slingo, J. M. (1987). The development and verification of a
SMOS rehearsal campaign 2008: radiometer data analysis and
cloud prediction scheme in the ecmwf model.
soil moisture retrieval using the LPRM. Earth Observation and
Quart.j.roy.meteor.soc, 113.
Water Cycle conferenceEarth Observation and Water Cycle
conference.

This contribution has been peer-reviewed.

https://doi.org/10.5194/isprs-archives-XLII-3-583-2018 | © Authors 2018. CC BY 4.0 License. 586