SHIP DETECTION FOR KOMPSAT-3A OPTICAL IMAGES

USING BINARY FEATURES AND ADABOOST CLASSIFICATION

Jae Young Chang* , Han Oh, Seung-Jae Lee, and Kwang Jae Lee

National Satellite Operation and Application Center, Korea Aerospace Research Institute (KARI)
IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium | 978-1-7281-6374-1/20/$31.00 ©2020 IEEE | DOI: 10.1109/IGARSS39084.2020.9323847

*
E-mail: [email protected]

ABSTRACT end-to-end method, but difficult to understand the result-

ing mechanism and improve it. On the other hand, traditional
KOMPSAT (KOrea Multi-Purpose SATellite)-3A is a high-
methods that combine human-designed features with machine
resolution earth observation satellite equipped with an electro-
classification can achieve comparable performance while pro-
optic camera and infrared sensors developed by KARI. In this
viding a more detailed understanding of the mechanism [3].
paper, we introduce a method of detecting ships and estimate
Since the radiometric characteristic of the satellite im-
the performance with pan-sharpened optical images of 0.55m
agery varies according to environmental changes, it is impor-
resolution from KOMPSAT-3A. Machine learning based de-
tant to select features that can cope with this problem for ro-
tection technology is essential to detect ship objects with
bust detection. Binary features combine comparison informa-
inconsistent shapes and backgrounds, particularly in satellite
tion between relative pixel intensities, resulting in consistent
images where the radiometric characteristic varies severely
information regardless of brightness and contrast changes [4].
due to environmental factors. To cope with these inconsis-
In this paper, we apply some effective binary features and Ad-
tencies, we introduce a ship detection algorithm using binary
aBoost classification, which rarely suffers from overfitting, to
features and AdaBoost classification. The experimental result
detect ships in high resolution satellite images.
shows that the algorithm effectively captures the character-
istics of the appearance and shows competitive detection
performance. 2. PROPOSED METHOD
Index Terms— Ship detection, machine learning, binary The four binary features used in this study are summarized
feature, AdaBoost, MCT, LAB, random fern in the following subsections. They are all defined as a single
feature value of 9 bits in length, forming a weak classifier in
1. INTRODUCTION the same way for AdaBoost training.

Supervised machine learning is the process of selecting fea- 2.1. Binary features
tures considered to be important to distinguish the objects of
interest from the background and determining their weights 2.1.1. MCT (Modified Census Transform) feature
through trial and error. This requires a dataset that represents
A MCT feature is defined as an ordered set of comparisons of
the correct answer.
pixel intensities in a local neighborhood representing which
Some datasets have been published for object detection in
pixels have lesser intensity than the intensity mean [5]. Let
satellite images, and they differ in the type of sensor used for
N (x) define a local spatial neighborhood of the pixel at x.
image acquisition, the number of object types, and the label ¯
The intensity mean on this neighborhood
is denoted by I(x).
data format. xView dataset consists of horizontal rectangle ¯ ¯
Let a comparison function ζ I(x), I(y) be 1 if I(x) < I(y)
labels for 60 kinds of objects for WorldView-3 optical images N
and let denote the concatenation operation, then a MCT
[1]. In the case of ship objects, such horizontal rectangular
feature is defined as .
label format is not suitable in consideration of long shapes
and clustered situations. DOTA dataset, on the other hand, O
¯


Γ(x) = ζ I(x), I(y) (1)
consists of rotated rectangle labels for 16 kinds of objects for
y∈N (x)
various GSD images, which are suitable for representing ship
position [2]. In this paper, we adopt the rotated rectangle for- An important property of the feature is its robustness to
mat. monotonic gray-scale changes caused, for example, by illu-
In the field of satellite image analysis, deep-learning- mination variations. Another important property is its com-
based detection technology, which is the mainstream in com- putational simplicity, which makes it the feature of the first
puter vision, is being introduced. It is easy to apply as an cascade classifier for early rejection of candidates.

978-1-7281-6374-1/20/$31.00 ©2020 IEEE 968 IGARSS 2020

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on August 28,2024 at 08:33:23 UTC from IEEE Xplore. Restrictions apply.
 x
(x2, y2) 2.2.1. Preparation of training samples
(x1, y1)
The size of the detection template is defined as 50×18 pixels,
h2 θ y and it includes a margin of 5.5 pixels around. So the minimum
detectable breadth of the ship is 7 pixels and is actually 3.85
h1
w2 m. Although the input image is color, only brightness is used
w1 to classify the ship regardless of the color change according
(a) (b) to the atmospheric state. Some examples of training samples
are shown in Fig. 2.
Fig. 1. (a) Two LAB features and (b) an RE feature.

2.1.2. LAB (Locally Assembled Binary) feature

The LAB feature is calculated similarly to the MCT feature, positive samples augmented samples negative samples
except that pixel intensities are compared by area rather than
by pixel. Therefore, this feature should be specified with the Fig. 2. Examples of training samples. Positive samples are
width w and height h of the comparison block as well as the extracted from the dataset and then augmented for translation,
offset [6]. Fig. 1(a) gives two examples of LAB feature. scaling, rotation, and vertical symmetry transform. Negative
Compared with MCT, it can describe overall appearance samples are generated by continuously collecting false posi-
with a single feature value and is robust against noise. In tive samples.
addition, the integral image can be used for fast calculation of
the sum of the brightness values in a block.
2.2.2. Training process
2.1.3. RE (Random Ellipsoidal) feature
The AdaBoost algorithm repeatedly selects effective weak
The RE feature is another extension of the MCT feature. As classifiers from four binary feature types and finally gener-
shown in Fig. 1(b), we define an ellipse with random size ates a strong classifier. Weak classifiers are generated from
(a, b), position (x, y), and rotation angle θ within an input the distribution of features in the same way regardless of the
image, and construct a feature using brightness values of the feature type, because all binary features have the same format
center and 8 points, each being 45◦ apart, on the ellipse. of 9-bit values [3]. Table 1 gives the pseudo code for the
This feature can not only express the overall appearance AdaBoost algorithm using multiple binary features.
as LAB, but can also reflect the comparison of brightness val-
ues for arbitrary angles. 2.2.3. Removal of confusing samples
Object detection in satellite imagery is difficult because it is
2.1.4. RF (Random Fern) feature almost impossible to obtain a perfect ground truth. Miss-
The final feature, RF, has no limit on the pixel location to ing labels by human error not only adversely affects perfor-
be compared. A feature value is calculated by concatenating mance validation, but also causes overfitting in the learning
the result of comparing the brightness values in m random process. Also, there are ships that do not look like ships
pairs in the input image [7]. Let dj,1 and dj,2 are two or backgrounds that look like ships, which also causes class

ran- overlap problems.
¯ j,1 ), I(dj,2 ) be 1 if
dom locations, and let a function ζ I(d
I(dj,1 ) < I(dj,2 ), then a RF feature is defined as We have adopted an iterative method proposed by Vezh-
nevets and Barinova to filter out confusing samples automat-
m
O ically [8]. Training samples are divided into three parts in
¯ j,1 ), I(dj,2 )


Γ(x) = ζ I(d (2) each iteration. The classifier trained with the first part is cal-
j=1 ibrated with the second part and the posterior probability is
The parameter m is fixed to 9 for the consistency with measured for the third part. If the result of classification based
other features. on the average of posterior probabilities is inconsistent with
ground truth, the sample is considered as confusing and ex-
cluded from the actual learning process.
2.2. Training by AdaBoost
In this study, the aspect ratio of ships is limited to a minimum 3. RESULTS AND DISCUSSION
of 4.0 to a maximum of 5.0, because the classification is ex-
ecuted by sliding window method that requires fixed aspect A total of 332 images taken by KOMPSAT-3A were used for
ratio. the experiment, and the pixel resolution of each image was

969

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on August 28,2024 at 08:33:23 UTC from IEEE Xplore. Restrictions apply.
 (a) MCT (b) LAB (c) RE (d) RF
Table 1. Boosting process with binary features
• Given (F1 , c1 ), ..., (F2 , cm ) where ci = 0 for ships
and ci = 1 for backgrounds. Fi is a feature vector of
i-th sample.
i
• A feature value of i-th sample is specified as Fpq ,

stage
where p is a feature type and q is a feature index.
1 1
• Initialize D1 (i) = 2l , 2n where l and n are the number
of ships and backgrounds.
• For t = 1, ..., T :
- Select a feature type pt randomly from four feature
types and prepare a corresponding feature pool.
- Generate weak classifier wt with a feature Fpit qt , error
Fig. 3. Visualization of selected weak classifiers. (a) Posi-
t and distribution Dt . 
1 1−t
tions of MCT features are depicted as red dots. (b) Areas of
- Choose αt = 2 ln t LAB features are superimposed by weights. (c) (d) Positions
of referenced pixels by each feature, RE and RF, are superim-
- Update the distribution
( posed by weights.
Dt (i) e−αt if wt (Fpit qt ) = ci
Dt+1 = Zt × αt
e if wt (Fpit qt ) 6= ci
where Zt is chosen so that Dt+1 is a distribution. classifier mainly observes the lateral edges of the ship. The
- Save the selected feature index qt to a set Qpt RE-based classifier mainly observes the ship’s center and
head, while the RF-based classifier observes the entire inter-
• The resulting elementary classifier of a single feature nal area. Table 2 shows the distribution of weights by the
can be obtained by a combination of the corresponding feature type. The highest proportion of RF can be interpreted
weak classifiers. as difficult to find effective weak classifiers at consistent
positions. The lowest proportion of LAB, which captures
T
X the dominant edges of the ship, shows that the boundaries
hpq (Fpq ) = αt wt (Fpq )I(pt = p)I(qt = q) between ships and backgrounds are very subtle.
t=1

where I(·) is the indicator function that takes 1 if the Table 2. Distribution of weights by feature type.
argument is true and 0 otherwise. Feature type MCT LAB RE RF
• The final strong classifier is the sum of all selected el- Sum of α 501.5 324.0 580.6 715.1
ementary classifiers. Ratio(%) 23.6 15.3 27.4 33.7
X4 X
H(F) = hpq (Fpq )
p=1 q∈Qp

3.2. Performance evaluation

3072 × 3072. We have a total of 2,925 ship labels, and sep-
arated the images in the ratio of 7:3 by the number of labels A ship object is considered to be successfully detected if IOU
for training and testing respectively. with a ground truth label is greater than 0.5. Performance
measures are recall, precision and F2 score. One thing to
note is that recall is measured only for ships with aspect ra-
3.1. Analysis of the training results
tios within a given range, from 4.0 to 5.0, while precision is
In this analysis, the maximum number of elementary classi- measured for ship labels with all aspect ratios.
fiers for each feature type is set to 768, and weak classifiers The detection performance was measured by changing the
are collected at 10 times the elementary classifier. Therefore, number of elementary classifiers for each binary feature, and
the iteration number T is set to 30,720 (4 × 768 × 10). In or- is shown in Table 3 and Fig. 4. As the number of elemen-
der to improve the processing speed, the classifier is divided tary classifiers increases, the precision increases but the recall
to several stages through soft cascading. decreases. In addition, it can be seen that the overall perfor-
Fig. 3 visualizes the elementary classifiers selected by mance is improved by removing the confusing samples. Some
AdaBoost. The MCT-based classifier captures the main fea- detection results on KOMPSAT-3A images with the parame-
tures at the entire boundary sparsely, and the LAB-based ter 768 are shown in Fig. 5.

970

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on August 28,2024 at 08:33:23 UTC from IEEE Xplore. Restrictions apply.
 ages acquired from high resolution optical satellites such as
Table 3. Detection performance on KOMPSAT-3A test set
KOMPSAT-3A. In addition, we introduced the method to ex-
Elementary
Recall Precision F2 score clude confusing samples to reduce the class overlap problem.
classifiers
To improve the performance of the proposed algorithm,
512 78.7 78.2 78.6 we can extend classifiers with other aspect ratios and add fea-
640 72.9 85.8 75.1 tures such as HOG (Histogram of Oriented Gradients), which
768 70.5 88.2 73.4 more efficiently represents subtle changes in ship’s appear-
896 67.7 90.9 71.3 ance. This is the subject of ongoing research.

90.0 5. REFERENCES

85.0 [1] D. Lam, R. Kuzma, K. McGee, S. Dooley, M. Laielli,

Precision

M. Klaric, Y. Bulatov, and B. McCord, “xView:

80.0 confusing Objects in context in overhead imagery,” 2018,
samples removed
75.0 arXiv:1802.07856.
with original
70.0 samples [2] G. Xia, X. Bai, J. Ding, Z. Zhu, S. Belongie, J. Luo,
M. Datcu, M. Pelillo, and L. Zhang, “DOTA: A large-
65.0
scale dataset for object detection in aerial images,” in
67.5 70.0 72.5 75.0 77.5 80.0 82.5
Proc. of CVPR, 2018, pp. 3974–3983.
Recall
[3] P. Viola and M. Jones, “Rapid object detection using a
Fig. 4. Precision-recall curve boosted cascade of simple features,” in Proc. of CVPR,
2001.
[4] D. Huang, C. Shan, M. Ardabilian, Y. Wang, and L. Chen,
“Local binary patterns and its application to facial image
analysis: A survey,” IEEE Trans. on Systems, Man, and
Cybernetics, vol. 41, no. 6, pp. 765–781, Nov. 2011.

(a) [5] B. Froba and A. Ernst, “Face detection with the modified
census transform,” in Proc. of Sixth IEEE International
Conference on Automatic Face and Gesture Recognition,
May 2004, pp. 91–96.
[6] S. Yan, S. Shan, X. Chen, and W. Gao, “Locally assem-
bled binary (lab) feature with feature-centric cascade for
fast and accurate face detection,” in Proc. of CVPR, June
(b) 2008, pp. 1–7.
[7] M. Ozuysal, M. Calonder, V. Lepetit, and P. Fua, “Fast
keypoint recognition using random ferns,” IEEE Trans.
on Pattern Analysis and Machine Intelligence, vol. 32,
no. 3, pp. 448–461, March 2010.
[8] A. Vezhnevets and O. Barinova, “Avoiding boosting
overfitting by removing confusing samples,” in Proc. of
(c) ECML, 2007, pp. 430–441.

Fig. 5. Some examples of detection result on KOMPSAT-3A

images. (a) Ships of various sizes in a port (b) Clustered ships
(c) Noisy situation due to haze

4. CONCLUSION

This study shows that the combination of binary features and

AdaBoost classification is effective for detecting ships in im-

971

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on August 28,2024 at 08:33:23 UTC from IEEE Xplore. Restrictions apply.