Accuracy Assessment of Supervised Classification M
Accuracy Assessment of Supervised Classification M
Abstract. Remotely sensed imagery identifying as the best type of data has information
throughout the world. The imagery has a importance information, since it can show up-date-to
day information, and provide a truly information. Different kinds of classifiers have been used
to perform that. However, there is no once test for Land cover and Land use in Hilla city. The
study aims to create land use classification by making a comparison between different
algorithms in Hilla city, Babylon, Iraq. The WorldView-2 imagery is used to perform the per-
processing, analysing of our comparison. Under the steps of pre-processing, the several
corrections were made and performed on the imagery. For processing stages, two approaches
were used; (1) Support Vector Machine and (2) Maximum Likelihood. The result reveals, that
the Support Vector Machine method has the most significant of overall accuracy equal to
94.48% with kappa coefficient equal to 0.90, and these values much better and higher than
those of Maximum Likelihood algorithm in estimating and extracting of Land cover/Land use.
Therefore, this algorithm has been suggested to be applied as an optimal classifier for
extraction of land use maps due to its higher accuracy and better consistency within the study
area.
1. Introduction
Remote sensing and Geographic information system have capability of detecting and/ or monitoring
the features of earth’s surface using satellite images have different radiometric, spectral, spatial, and
temporal resolution. These technologies resolutions have several advantages in order to minimize time
and cost to extract Land cover and Land use (LULC) [1, 2]. In addition, the remote sensing provides
important and different kinds of remotely data source to extract LULC information. Remote sensing
data are widely used and applied to perform classification of LULC throughout the world [2]. Remote
sensing data has the ability of updating information about all the features that locate on earth’s surface
[3]. There are different satellite systems provides different imagery has different spatial resolutions
applied for LULC detection such as WORLDVIEW-2, Quick Bird, Landsat, Spot and so on [1-3]. The
classification of remote sensing imagery is an important method in order to determine the LULC
information [2]. Classification approaches divided into two categories: pixel base and /or objected
oriented base classifiers [3]. The Maximum Likelihood approach (MLC) widely applied for LULC
extraction, by creating decisions of surface regarding of the covariance and mean of each single class
[4]. However, the non-parametric technique Support Vector Machine (SVM) classifier, perform the
classification by no assumptions and separate the features on each class [5, 6]. SVM is a non-
parametric algorithm, which contains a series of learning algorithms are conducted for classes
classification and regression [5-7]. The SVM classification was proposed by Vapnik and
Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution
of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
Published under licence by IOP Publishing Ltd 1
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
Chervonenkis [8] and after that discussed in more detail by Vapnik [9]. Add to that, the SVM ws
widely applied in different remote sensing projects and applications [10, 11]. Huang et al. [12] applied
the SVM classifier for a LULC classification by using Landsat thematic mapper (TM) imagery. The
SVM classification has a good accuracy and superior to the accuracy of using different methods such
as (1) maximum likelihood and (2) a decision tree [1-3]. Yousefi et al. [13] has tested a nine different
supervised classification approaches and they are (neural network, Spectral angle mapper,
maximum likelihood, SVM, mahalanobis distance, binary code, minimum distance, spectral
information divergence and parallelepiped) for LULC mapping in province of Mazandaran, Iran
and they used the imagery of Landsat ETM+. The results showed that the SVM classifier has the best
accuracy than the other classifiers. The main objective of this study is to perform LULC classification
using MLC and SVM approaches in Hilla city, Babylon, Iraq, and make comparison to find out the
perfect algorithm to classify the study area.
2. Materials and methods
The detection technique to extract the thematic map of LULC was tested and referenced by previous
work [12]. In this research, WorldView-2 satellite system imagery of 2017 was used to perform the
pre-processing and processing steps. In this research, geometric, radiometric, and atmospheric
corrections were applied to remove image noise. Then, layer stacking and image sub-setting were
conducted.
The next step was to applied different kinds of supervised classification algorithms (MLC and
SVM) that were applied to the collected imagery to perform imagery classification to map the LULC
theme of Al-Hilla city, Babylon, Iraq. The accuracy assessment and statistical analysis of the expected
results were conducted to get an accurate and suitable approach to classify the study area of this
research. Figure 1 reveals the flowchart of our methodology.
2
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
capital of Babil province, and it is located in central Iraq. Its boundaries area surrounding with other
governorates, they are Baghdad, Kerbala, Anbar, Najaf, Wassit, and Qadissiya. It has Euphrates River,
it is one of biggest rivers in Iraq, and it intersects its governorate and divided into the AL- Hindiyah
and Al- Hilla branches south of the Musayib town, to create a network of canals runs through the lands
of governorate to use as an irrigation system and supply the regions, fields and farms with water. Hilla
city has a population of around 970, 000 people regarding registration of 2018. Figure 2 shows
Location of the Hilla city.
3
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
4
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
3.2. Fieldwork
The fieldwork conducted by using Handheld GPS type Garmin 78s, it is one of the most popular
devices have been used to collect the locations of points locate in the study area with different kind of
remote sensing applications. The accuracy of this device is for horizontal < 10m, and for vertical =
0.05 m/s steady-state. Figure 5 shows the used Garmin GPS.
5
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
In addition, the authors use the Nearest Neighbour approach is used for resampling uncorrected
pixels. Finally, a root mean square error (RMSE) of images is used to obtain to check out their
accuracy and we got less than 0.4 pixels, and this value is acceptable [14]. Figure 7 shows the imagery
after perform all corrections and it became a noise-free image.
6
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
processing. The study area was classified into four major classes to perform different classification
techniques based on the most located features into the study area to obtain the thematic maps of land
cover and land cover. Four classes are (Water bodies, Soil, Urbanization area and Vegetation area)
collected training sites were conducted by using Envi software be selected polygons of the region of
interest (ROIs) for each class, then used these ROIs into classification to generate the thematic map of
Land use and land cover for the study area. For this stage was about collecting the testing sites to study
area, these sites very important to determine the accuracy assessment for each classification algorithms
and to check the validation, producer and user accuracy of classifications. Usually testing sites are
ground truth samples, and the better collecting from the field of the study area to get pure ground truth
samples. Collecting the testing sites for this study was conducted by using office work from the
imagery. Figure 8 (a &b) shows the number of pixels selected to be as training and testing sites for
each class of the study area.
Figure 8. (a &b). Show training and testing sites for each class of our study area
3.3.2. Maximum Likelihood Approach
A maximum-likelihood algorithm is one of the wide classifiers has been used for supervised image
classification in different remote sensing application [14]. Erdas [15] mentioned that this approach
works by computing weighted distances and/or likelihood D relate to an unknown vector X belong to
known classes. However, Mc is regarding Bayesian equation [15]:
Where, C is a particular class, ac refers to the percent probability for any candidate pixel of
class C. when Covc refers to pixels covariance matrix in class c. However, ǀCovcǀ is a Covc
determinant, (Covc-1) inverse of Covc, the T refers to transposition function [15]. After that the
results is revealed and the validation is done by using the confusion matrix. The results of applied
MLC and the producer and user accuracies are presented in Table 3 below for each class. Figure 9
WorldView-2 satellite image after perform MLC classification.
Table 3. Producer and user accuracies of MLC classification
Class Prod. Acc. (%) User Acc. (%)
Water Bodies 99.49 79.81
Vegetation 82.70 99.57
Urbanization 89.32 50.57
Soil Area 74.89 95.10
7
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
8
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
9
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
According to the above figures, it clearly seems that the SVM classifier has the highest results of
overall accuracy and the kappa coefficients than those of the MLC approach for our study area in Hilla
city, Babylon, Iraq. Moreover, both Huang et al. [12] and Otukei & Blaschke [22] are evaluated
various methods for classifications in LULC mapping, and they were revealed that the results of
SVM approach compared to the results of ML algorithm and also to decision trees method has the
most highest accuracy assessment for mapping the LULC. The author Deilmai et al. [23-27] in his
research, compared between 2 classification algorithms SVM and MLC in order to extract LULC
thematic map in Johor city, Malaysia., his results revealed the results of SVM classifier regarding to
kappa coefficient was equal to 0.86 and it was most an accurate algorithm.
10
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
References
[1] H Dibs and S AL-Hedny. 2019. Detection Wetland Dehydration Extent With Multi-Temporal
Remotely Sensed Data Using Remote Sensing Data Using Remote Sensing Analysis and GIS.
IJCIET, 2019. 10 (1)143–154.
[2] H Dibs. 2018a. Comparison of Derived Indices and Unsupervised Classification For Al-Razaza
Lake Dehydration extent Using Multi-Temporal Satellite Data and Remote Sensing Analysis ARPN J
Eng Appl Sci 13 (24).
[3] H Dibs, S AL-Hedny, H S Karkoosh. 2018b. Extraction Detailed Buildings 3D Model With
Using High Resolution Satellite Imagery By Remote Sensing and GIS Techniques AL-Qasim Green
University: A Case Study IJCIET 9 (7) 1097–1108.
[4] J Richards & X Jia. Remote sensing digital image analysis. 4th edition. Heidelberg, springer,
Germany. 2006. 359-88.
[5] M Bray & D Han. Identification of Support Vector Machines for Runoff Modelling. J.
Hydroinform. 2004, 6, 265-80.
11
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
12
The Fourth Postgraduate Engineering Conference IOP Publishing
IOP Conf. Series: Materials Science and Engineering 745 (2020) 012166 doi:10.1088/1757-899X/745/1/012166
[26] H Dibs, M O Idrees, V Saeidi and S Mansor. 2016. "Automatic Keypoints Extractionfrom UAV
Image with Refine and Improved Scale Invariant Features Transform (RI-SIFT)." International Journal
of Geoinformatics 12 (3): 51-58.
[27] H Dibs, S Mansor, N Ahmed and B Pradhan 2015 Band-to-band registration model for near-
equatorial Earth observation satellite images with the use of automatic control point extraction Int J
Remote Sens 36 (8) 2184–2200.
13