This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

Cascade Learning Embedded Vision Inspection of Rail

Fastener by Using a Fault Detection IoT Vehicle
Jianwei Liu, Hongli Liu, Chinmay Chakraborty, Keping Yu, Xun Shao, Ziji Ma

 detection of fasteners, vision technology combined with AI has

Abstract—Fastener needs to be monitored and inspected peri- become the main mean due to it can achieve reliable detection
odically to ensure the rail’s safety due to its easily damaged ac- performance and is with low application cost.
cessory for railway infrastructure. Recently, Industrial Internet of
Things (IIoT) and artificial intelligence (AI) based visual inspec-
tion techniques have been exploited to realize online inspection of
fastener’s fault by using a fault detection IoT vehicle that is
mounted with multi-type sensors and cameras according to the
design of our research team. However, instead of traditional arti-
ficial inspection, AI-based automatic fastener inspection approach
is still faced with some challenges, for example, collection of
enough samples of faulted fastener. In this paper, we propose a
cascade learning embedded vision inspection method of rail fas-
tener based on deep convolutional neural network (DCNN). The Fig. 1 Defective fasteners. (a) Damaged fasteners. (b) Missing fasteners.
proposed method has two steps: region position and fault detec- A. Related Methods and Challenges
tion. Firstly, a modified Single Shot multibox Detector (SSD)
model is adopted to locate the fastener regions from the captured Generally, from the application perspective of IIoT, a vi-
railway images. Then, a key component detection method based sion-based system realizes automatic detection of fasteners by
on improved Faster Region Convolutional Neural Network two steps: One is to locate fastener regions from the captured
(RCNN) is proposed to realize detection of faulted fastener. Ex- railway images. The other is to recognize the fastener state and
tensive experiments are conducted to demonstrate the perfor- realize fastener inspection.
mance of the proposed method. The experiment results show that Fastener region positioning: Li et al [6] first detected the
the proposed method achieves an average precision of 95.38% and
tie plate based on geometric features and then located the fas-
an average recall of 98.62% on fastener detection, which is much
better than the manual operation. tener region according to the position of tie plate. Feng et al [7]
Index Terms—Fastener inspection, IIoT, vision technique, cas- located fastener regions based on the positions of tracks and
cade learning, fault detection IoT vehicle. sleepers. Prasongpongchai et al [8] used the masks of the tracks
and tie regions to locate fastener region, and these masks were
I. INTRODUCTION generated by calculating their edge density. In literature [9],
Fan et al first obtained the candidate fastener regions by de-

F ASTENER play an important role in maintaining railway

safety. However, with the influence of wheel-rail vibra-
tion, fasteners will appear damaged or even missing. The
defective fasteners have a great impact on railway safety and
they need to be inspected periodically. However, the traditional
tecting tracks, and then fastener region was located by template
matching. Similarly, Xia et al [10] realized fastener region
positioning by detecting the position of sleeper. Liu et al [11]
extracted HOG features and realized fastener region position-
ing by template matching and prior information.
manual inspection under the IIoT environments is usually in- Although aforementioned methods can locate fastener region,
efficient, not real-time and even dangerous. It is imperative to they are not suitable for fastener region positioning in practical
develop an automatic fastener detection device to replace the application due to they rely too much on prior information and
manual operation. cannot locate fastener region in turnout section.
In recent years, vision technology and artificial intelligence Fastener state recognition: Before deep learning, Mazzeo
(AI) have been widely used in IIoT [1-2] and industry 4.0 to et al [12] adopted wavelet transforms to extract the fastener
complete various inspection tasks [3-5]. For the automatic features and then took advantage of multi-layer BP and RBF
networks to classify fastener features and realize fastener in-
This work was supported in part by the NNSFC (No. 61771191 and No. spection. Yang et al [13] realized fastener inspection by tem-
61971182), in part by the NSF of Hunan Province (No. 2020JJ4213 and No. plate matching using direction field features. Feng et al [7]
2021JJ30145) and in part by the Japan Society for the Promotion of Science
(JSPS) Grants-in-Aid for Scientific Research (KAKENHI) under Grant
proposed a probabilistic topic model (STM) for the detection of
JP18K18044 and JP21K17736. (Corresponding author: Ziji Ma ,Chinmay defective fasteners such as damaged and missing. Ou et al [14]
Chakraborty and Keping Yu) first adopted conditional random field to segment fastener and
Jianwei Liu, Hongli Liu and Ziji Ma are with the College of Electrical and then build the fastener feature words by Bayesian hierarchical
Information Engineering, Hunan University, Changsha, 410082, China (email:
[email protected], [email protected], [email protected])
model. Finally, the fastener inspection is achieved by classi-
Chinmay Chakraborty is with Birla Institute of Technology, Mesra, India fying the feature words using SVM. Fan et al [9] detected de-
(email: [email protected]) fective fasteners using the proposed LLBP method. The similar
Keping Yu is with Global Information and Telecommunication Institute, work was reported in literature [10].
Waseda University, Japan (email: [email protected])
Xun Shao is with Regional Innovation and Social Design Engineering,
The aforementioned traditional methods can achieve fastener
Kitami Institute of Technology, Kitami, Japan inspection. However, when these methods are applied to real
(email: [email protected])

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

railway, the inspection performance is unsatisfactory due to the of this method is to transform fastener inspection from the
extracted features are susceptible to environmental influence classification task that requires normal and defective fasteners
and have poor stability. to the detection task that only requires normal ones, which is
In recent years, deep learning and IoT technologies achieved especially useful for the detection of imbalanced fasteners.
great success for railway detection [15-17]. For the fastener The structure of this paper is as follows. Section II presents
inspection, Gibert et al [18] recognized defective fasteners the framework of cascade learning approach and the designed
using a multi-task learning architecture. Qi et al [19] proposed a fault detection IoT vehicle. Section III presents the fastener
MYOLO v3-Tiny model to detect whether the fastener is de- region positioning method. The KCD method is introduced in
fective. Wei et al detected fasteners through Faster RCNN [20] Section IV. Section V lists the experimental results and dis-
and the DC-TLMDDNet model [21], respectively. However, it cussion. And the conclusion is given in Section VI.
is difficult to adopt the AI and IoT technologies to achieve the
good inspection performance since the distribution of defective II. DETECTION VEHICLE AND APPROACH
and normal fasteners is heavily biased. Siamese network [22], We briefly introduce the fault detection IoT vehicle and the
matching network [23] and prototypical network [24] can re- proposed cascade learning approach in this section.
alize the detection of imbalanced samples. However, it is un-
clear how these networks perform when applied to the imbal- A. Fault Detection IoT Vehicle
anced fastener inspection. In literature [25], Yao et al first Fig. 2 shows the designed fault detection IoT vehicle. It is
generated defective fastener images by GAN and then used made up of two sub-systems. Camera, light and encoder con-
these images to improve the detection performance. However, stitute image acquisition subsystem (IAS) and the workflow is
due to the lack of defective fasteners, the quality of defective as follows: when the detection vehicle is running along rail, the
fastener images generated by GAN is poor and the detection encoder fixed in the wheel emits pulse signals to trigger camera
performance cannot be effectively improved. Similarity, Liu et to capture railway images. Computer constitutes image pro-
al [11] improve the inspection performance of fasteners by cessing subsystem (IPS) and the workflow is as follows: the
building the sample pairs using Siamese network. However, the captured railway images are transmitted to IPS by network line
improvement effect of this method is also not obvious since the and then fastener inspection is achieved by processing railway
lack of defective fasteners. images using the proposed methods.
After reviewing the above literatures, the challenges of re-
alizing automatic detection of fasteners are as follows: B. Cascade Learning Approach
1) The accuracy of fastener region positioning is insufficient The proposed cascade learning approach achieves fastener
and the fastener regions in turnout section cannot be located. inspection through two stages. The first stage is to locate fas-
2) There is a serious imbalance between the normal and de- tener regions and the second stage is to recognize fastener state.
fective fasteners, and which has a serious impact on the in- The framework of cascade learning method is shown in Fig. 3.
spection performance of fasteners. However, most of existing 1) Fastener region positioning. Considering the complexity
methods cannot solve this problem. of the railway environment and deep convolutional neural
3) The existing detection methods for imbalanced fasteners network (DCNN) performs well on object detection, DCNN is
can only alleviate the influence of imbalanced fastener samples adopted to locate fastener region. In detail, considering the
on the inspection performance and cannot fundamentally solve positioning accuracy and speed, we propose a modified SSD
this problem. model to locate fastener region. In addition, the input railway
images are scaled from 2048×5000 to 512×625 to reduce the
B. Contributions
memory consumption.
To overcome the problems of realizing automatic detection 2) Fastener state recognition. In order to accurately recognize
of imbalanced fasteners, we propose a cascade learning method the fastener state in the case of imbalanced samples, we propose
based on DCNN. The main contributions are as follows: a KCD method based on the improved Faster RCNN. Specially,
1) A fault detection IoT vehicle is designed to realize online the KCD method realizes fastener inspection by detecting the
inspection of fastener. It should be noted that the railway im- number and position of key components. In this way, we can
ages captured by camera are transmitted to the database by IoT only use normal fastener images to realize the detection of
technologies to process. imbalanced fasteners.
2) A cascade learning method which adopts DCNN is pro-
posed for fastener inspection. Firstly, the modified SSD model
is adopted to locate fastener regions. Then, fastener inspection
is achieved according to the detection results of key compo-
nents of fastener and the key component detection is realized by
the proposed improved Faster R-CNN. Specially, the proposed
cascade learning method can be applied to the fault detection
IoT vehicle to realize automatic detection of fasteners.
3) To effectively improve the inspection performance of the
imbalanced fasteners, a key component detection (KCD) based
on improved Faster RCNN method is proposed. The proposed
method judges the fastener state according to the quantity and
position of the detected key components. The main advantage Fig. 2 The fault detection IoT vehicle.

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

Fig. 3 The framework of proposed cascade learning approach.

III. FASTENER REGION POSITIONING The confidence loss is defined as formula (2).

   
N
Lconf  x,c     
As mentioned previously, the fastener region positioning
xijp log cˆ ip  log cˆ i0 (2)
methods should be robust to the complex railway environment.
iPos iNeg
Therefore, we adopt DCNN method to locate fastener region.
Considering the positioning accuracy and speed, SSD model Here, ĉip represents the confidence of the ith default box in
[26] is adopted for fastener region positioning. category p . It can be obtained by formula (3).
1) Network architecture. The core idea of SSD is to generate
a series of default boxes and then predict the categories and
ĉip 
 
exp cip
(3)
probabilities of those boxes. The default boxes are generated in  exp c 
i
p
the multi-scale feature maps and these feature maps are from p
the different convolutional layers. In the SSD model, VGG16 The localization loss is defined as formula (4) and (5).
network [27] is the backbone network. The convolutional lay-
 
N
ers 4_3, 7, 8_2, 9_2, 10_2 and 11_2 act as the output layers and Lloc  x,l,g     xijk smoothL1 lim  gˆ mj (4)
the corresponding feature map sizes are 38×38, 19×19, 10× iPos mcx,cy,w,h
10, 5×5, 3×3 and 1×1, respectively.
However, in the captured railway images, the proportion of  j j i
 ĝ cx  g cx  d cx / d w
i 
fastener region is small and the background is complicated.
Meanwhile, SSD model is not effective in small object detec-
 cy cy
 ĝ j  g j  di / di


cy h

tion, especially the small object in complex background. Thus,  g wj 
 (5)
in order to improve the positioning accuracy of fastener regions  ĝ w
j  log  dw 
in complicated railway environment, we modify SSD model as   i 
follows. Based on the theory that large scale feature map is   gh 
suitable for detecting small object, the conv3_3 is added to the  ĝ hj  log  hj 
  di 
output layer and the conv11_2 is removed from output layer.
Thus, modified SSD model includes the feature maps with sizes Where, cx and cy represent the coordinates of the prior box.
of 156×128, 78×64, 39×32, 20×16, 10×8 and 5×4, and w and h represents the size of the prior box. d wj and dih rep-
these feature maps are from output layers conv3_3, conv4_3,
conv7, conv8_2, conv9_2 and conv10_2. Compared with SSD, resent the scale factor. dicx and d cy
j represent the offsets.
the feature map sizes of modified SSD are increased. In this
way, the more detailed information of fastener region can be IV. FASTENER STATE RECOGNITION
extracted, which is beneficial for the accurate location of fas-
tener regions and the positioning accuracy is improved. The In this section, we introduce the key component detection
network framework of modified SSD model is shown in Fig. 4. (KCD) method. Firstly, an improved Faster RCNN is proposed
2) Loss function. The loss function of modified SSD includes to detect the key components of fastener. Then, fastener state is
two parts: localization loss and confidence loss [26]. And it is judged by analyzing the detection results of key components
the weighted sum of the two losses. according to the corresponding decision rules.
L  x,c,l,g   1
N  Lconf  x,c    Lloc  x,l,g  (1) A. Key Component Detection
Where, N is the number of matched default boxes. l represents The core idea of the proposed KCD method is to detect the
key components of fastener and then obtain the quantity and
the prior boxes and g represents the ground truth boxes. c is
serial number of these key components. Here, considering the
the confidence of the softmax function for each category.  is detection accuracy and speed, we modify Faster RCNN net-
used to control the ratio between the localization loss and con- work to detect the key components of fastener.
fidence loss.

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

Fig. 4 The network architecture of modified SSD.

Fig. 5 The network architecture of improved Faster RCNN.

1) Network architecture. Fig. 5 shows the network archi- Algorithm 1 Automatically determine anchors
tecture of improved Faster RCNN. Specially, we make some Input:
appropriate improvements to the Faster RCNN [28] to build the
Xc: Ground truth boxes
improved Faster RCNN.
In the Faster RCNN, anchors are selected by hand and the K: The number of initial anchors
quantity is 9 (three anchor scales: 8, 16, 32; three anchor ratios: N: The number of iterations
1:1, 1:2, 2:1). The sizes of these generated anchors are far from for number=1 to N do
the labeled bounding boxes of key components (As shown in Randomly select K ground truth boxes as initial anchors
Fig. 6). It will affect the detection accuracy and speed of key Using the IOU metric and assigning each ground truth
components. In addition, according to the idea of KCD method, box in Xc to the nearest anchor based on the formula
the number of detected key components is more important than
whether bounding box and ground truth box of key component d (box, anchor )  1  IOU (box, anchor )
is close or not. Thus, it is not necessary to get a very accurate Calculate the average width and height of all ground
bounding box for key component. Based on the above analysis, truth boxes in each cluster and update anchors
Faster RCNN is modified by the following two aspects. end for
a) Automatically determine the number and size of anchors:
Output: Anchor boxes
We run K-means clustering algorithm to automatically deter-
mine anchors. Algorithm 1shows the pseudo code. b) Simplified the detection network: Since it is not necessary
to get a very accurate bounding box for key component, we
remove the bounding box regression module from the detection
network in the Faster RCNN.
2) Loss function. In improved Faster RCNN, the regression
module of detection network is removed. Therefore, there are
two loss functions: loss function LRPN of RPN and loss function
LDN of detection network. They are defined as follows:

LRPN 
1

 Lcls pi , pi*
N cls i

(6)
Fig. 6 The deviation between labeled bounding box and generated anchors. 
1
 pi* Lreg ti , ti*
N reg i
 

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

Where, i is the index of anchor, pi is the probability of pre- TABLE I

THE DECISION RULES OF FASTENER STATE
dicting a anchor as foreground, pi* is truth label and its value is
Number of key
0 or 1, ti and ti* are the coordinates of predicted bounding box Serial number of key component Fastener state
components
and ground-truth, respectively.  is balance parameter. 3 - Normal
2 1 and 3 Normal
LDN 
1
 Lcls pi , pi*
Ncls i
 (7)  2
2
1 and 2
2 and 3
Damaged
Damaged
1 - Damaged
Where, i is the index of proposal region, pi represents the
0 - Missing
probability of predicting a proposal region being key compo-
nent, pi* is the truth label and its value is 0 or 1.
Specially, Lcls adopts the softmax function. Lreg is defined
as follows:

Lreg ti ,t*i    smooth L1 ti ,t*i   (8)
i x,y,w,h


0.5 x 2 if x  1
smooth L1  x    (9)
 x  0.5 otherwise


B. Fastener State Recognition

Fig.7 The key components of fastener.
After key component detection is completed, fastener state is
judged by analyzing the detection results according to the de-
V. EXPERIMENTAL AND DISCUSSION
cision rules in Table I. For the special case that the number of
detected key components is 2, fastener state is judged with the In this section, a railway image dataset is constructed firstly.
help of serial numbers of key components. If the serial numbers Then, extensive experiments are conducted to demonstrate the
are 1 and 3, the fastener state is normal. For the other serial effectiveness and superiority of our proposed cascade learning
numbers, the fastener state is damaged. approach.
It should be noted that the serial number of key component is In this paper, we conduct all the experiments based on the
determined according to its position in the fastener image. In Tensorflow1.14.0, Python 3.7 and Ubuntu 16.04 system. The
detail, the serial number is 1 if key component on the left side. configuration of computing server is as follows: Intel i7-7820X
The upper and right side correspond to the serial number 2 and CPU and a NVIDIA GPU (RTX 2080Ti).
3, respectively. As shown in Fig. 7. A. Dataset
Based on the previous mentioned, the pseudo code of fas-
tener inspection is shown in Algorithm 2. In this paper, we used the railway images captured from real
track lines (Shimen and Changsha) to construct an experi-
Algorithm 2 pseudo code of fastener inspection mental dataset as listed in Table II and which contains 5792
Input: railway images and 17714 fastener regions.
Rd : training set, the labeled fastener images TABLE II
THE CONSTRUCTED RAILWAY IMAGE DATASET
Kd : number of iterations
for k = 1 to Kd do Railway image No. Fastener region No.
Sample m labeled fastener images from Rd 5792 17714
Generate anchor boxes based on Algorithm 1 Normal railway Turnout railway Normal Damaged Missing
Update the RPN by minimizing LRPN: 4229 1563 16702 396 616

 Lcls  pi , pi*    N  pi*Lreg ti , ti* 

1 1
LRPN  B. Fastener Region Positioning
Ncls i reg i
Fastener region positioning is the key step to realize fastener
Generate the region proposals inspection. Thus, we first evaluate the positioning performance
Update the Fast R-CNN by minimizing LDN: of the modified SSD model. Here, we set Intersection over

 Lcls  pi , pi* 
1 Union (IoU) to 0.8 and the detection rate (DR) [11] acts as the
LDN  evaluating indicator of fastener region positioning.
Ncls i
1) Training Process: We labeled 3700 railway images as the
end for experimental dataset. The training set includes 2700 railway
Output: key component detection model images and validation set includes 1000 railway images. We
Complete key component detection by the detection model trained modified SSD by the way of end-to-end. The momen-
Realize fastener inspection by analyzing the detection results tum is 0.9 and weight decay is 5 104 . The learning rate is set
of key components according to the decision rules in Table I to 0.01 and the batch size is 8.

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

Fig. 8 The positioning results of different methods. From left to right are modified SSD, SSD, DTL [11], FTL [29] and Cross [7].

2) Experiment Results: The modified SSD model is com-

pared with SSD model and the previous works [7] [11] [29] to C. Fastener Inspection
demonstrate its superiority for fastener region positioning. This In this sub-section, we perform the verification experiments
experiment is carried out on the test set and which contains followed by comparative experiments to evaluate effectiveness
2092 railway images. Meanwhile, the size of all railway images of KCD method for fastener inspection. All experimental re-
is same and it is 512×625. sults are evaluated by Precision (P), Recall (R) and F1-score
The comparative results are listed in Table III. The results (F1).
show that the modified SSD model has better positioning per- TP
formance for the fastener regions in normal and turnout railway. Precision  (10)
TP  FP
Meanwhile, compared with the previous works and SSD model,
TP
the detection rate has been significantly improved. In detail, for Recall  (11)
normal railway, all the methods can locate fastener regions and TP  FN
DCNN models (SSD and modified SSD) are superior to the 2  Precision  Recall
previous works. For turnout railway, the previous works cannot F1  score  (12)
Precision  Recall
locate fastener regions, but the DCNN models are still valid.
Here, TP, FP and FN are the true positive sample, false positive
Moreover, the modified SSD model achieves the best perfor-
sample and false negative sample, respectively.
mance. These advantages can be attributed to two reasons, one
is the previous works rely too much on prior information. 1) Training Process: We labeled 6000 normal fastener im-
However, this information is unavailable in turnout railway. ages with size of 80×120 as the experimental dataset. In detail,
Thus, these positioning methods cannot locate fastener region the number of training set is 4500 and the number of validation
in turnout railway and the performance is poor. The other is set is 1500. We train the improved Faster RCNN with the
modified SSD model adds large scale feature map (low-level momentum of 0.9. The batch size is 256. The optimizer is SGD
conv3_3) to the output layer, which contains more local de- and number of iterations is 30000. The initial learning rate is set
tailed information of fastener region. Therefore, it is beneficial to 0.001, and then tuned to one tenth of the original per 10000
for fastener region positioning the performance is best. iterations. It should be noted that the training way is alternating
Fig. 8 shows some visualized results of fastener region po- optimization.
sitioning. These results more intuitively show that our modified 2) Parameter Setting: In the Faster RCNN, anchors are de-
SSD model has better positioning performance. Especially for termined by hand. It has subjective and will affect the detection
the fastener regions in turnout section, the modified SSD model accuracy and speed of key component. Therefore, in the im-
still achieves good positioning effect. Meanwhile, it is superior proved Faster RCNN, instead of choosing anchors by hand, we
to the SSD model. run the K-means clustering algorithm on the training set ground
TABLE III truth boxes to automatically determine the anchors. However,
COMPARATIVE RESULTS OF FASTENER REGION POSITIONING
when using the K-means clustering algorithm, how to choose
Method Normal railway (%) Turnout railway (%) an appropriate K value is an important problem. Considering
Cross [7] 81.01 - the computational complexity, we perform multiple clustering
FTL [29] 84.36 - when K changes in the interval of [1, 9], and then calculate the
DTL [11] 87.23 - average IoU and frames per second (FPS), respectively. Finally,
SSD 92.61 87.12
we can obtain an appropriate K value based on the results of
Modified SSD 98.63 94.36
average IoU and FPS.

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

[31] and YOLO v3-tiny) for key components of fastener. Here,

the detection performance of all networks is evaluated based on
the results of fastener inspection and used the same experi-
mental data (training set, validation set and test set).
Fig.11 shows the comparative results. We can see that im-
proved Faster RCNN achieves the best performance. In detail,
the improved Faster R-CNN is significantly superior to SSD,
YOLO v2 and YOLO v3-tiny. Meanwhile, it outperforms the
Faster RCNN and YOLO v3. These advantages can be at-
tributed to two reasons, one is the improved Faster RCNN
automatically determine the anchors by K-means clustering
algorithm. The size of determined anchors is close to the la-
beled bounding boxes of key components and the number is
decreased. The other is the bounding box regression module is
removed from the detection network. Thus, the detection net-
work is simplified. In summary, the improved Faster RCNN
has better detection performance for key components of fas-
tener, which guarantees KCD method can accurately recognize
the detective fasteners.
To further highlight the performance of the KCD method for
fastener inspection, especially the advantage of detecting the
imbalanced fastener samples, our KCD method is compared
with other methods in literatures [7] [11] [19] [21]. These
comparative methods include the traditional method (STM [7])
and the deep learning-based methods (similarity-based [11],
MYOLOv3-Tiny [19] and DC-TLMDDNet [21]). In this ex-
periment, the quantity of damaged fasteners increased to 792 by
sample augmentation. The training set and test set used for all
methods is shown in Table IV. Specially, we calculate the
average precision ( P ), recall ( R ) and F1 ( F 1 ) score to
Fig. 9 The experimental results using different K values. (a) Average IoU. (b)
evaluate the performance of different methods.
Frames per second (FPS)

Fig. 9 (a) and (b) show the average IoU and FPS using the
different K values. We can see that the large K value is bene-
ficial for obtaining the bigger average IoU. Furthermore, it is
beneficial for the accurate detection of fastener key compo-
nents. However, with the increase of K value, the FPS of the
improved Faster RCNN will be decreased. Thus, in order to
trade-off the detection performance and efficiency, the K value
is set to 5 as the number of anchor. Then, the size of anchors can
be obtained according to the clustering results.
3) Experiment Results: We used the test set (10702 normal,
396 damaged and 616 missing fasteners) to evaluate the in-
spection performance on imbalanced fasteners. It should be
noted that the test set is also used for comparative experiments.
Fig.10 shows the experimental results. The results show that
the KCD method achieves the precision of 89.30% and recall of
96.97% for damaged fasteners. For the missing fasteners, these
Fig. 10 The results of fastener inspection by KCD method.
two indicators are 96.83% and 99.35%, respectively. These
results indicate that KCD method performs well in the detection TABLE IV
of imbalanced fasteners. Thus, the KCD method can be applied TRAINING SET AND TEST SET OF ALL METHODS
in the railway inspection system to detect defective fasteners. Deep learning-based
Fastener Traditional method
4) Comparison with Other Related Works: In addition to the method
verification experiment, we conduct the comparative experi- Normal 100 4500
ments to further highlight the superiority of our proposed KCD Training set Damaged 100 594
Missing 100 462
method. Since the accurate detection of key components is the
Normal 10702
key to realize fastener inspection, we first compare the detec-
Test set Damaged 198
tion performance of our proposed improved Faster RCNN and
Missing 154
other networks (Faster RCNN, SSD, YOLO v2 [30], YOLO v3

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

fective fasteners in the training set is extremely small. Although

the distribution of fastener samples is equalized by building the
sample pairs, the features of anomaly fasteners learned by
similarity-based method are still limited. Thus, the inspection
performance of this method is poor for imbalanced fasteners.
TABLE V
COMPARATIVE RESULTS OF DIFFERENT INSPECTION METHODS
Method P/% R/% F1
MYOLOv3-Tiny [19] 84.08 89.71 0.8680
DC-TLMDDNet [21] 89.79 90.25 0.9002
STM [7] 90.67 91.23 0.9095
Similarity-based [11] 94.23 93.67 0.9394
KCD 95.38 98.62 0.9697

D. Further Analysis
In this sub-section, we further conduct experiments to verify
the robustness of the inspection method (KCD) for the complex
railway environment and the effectiveness of cascade learning
architecture which combines modified SSD and KCD method
for fastener region positioning and state recognition.
1) Robustness verification of KCD method. We design three
experimental schemes to conduct comparative experiment to
verify the robustness of KCD method. The three experimental
schemes are designed based on the related methods in litera-
tures [9] [11] [29], respectively. The detailed configuration of
the three schemes is shown in Table VI. It should be noted that
the railway image dataset used in this experiment is not same as
the dataset in Table II. It is collected from the different railway
line (railway line near Qiao Touyi Station). A total of 238
railway images were captured and which includes 860 normal,
36 damaged and 56 missing fasteners. Moreover, these railway
images were captured in different weather conditions (sunny
with strong illumination, cloudy with weak illumination).

Fig. 12 The collection site of railway images.

Fig. 11 Comparative results of different detection models. (a) Precision.
(b) Recall. (c) F1-score TABLE VII
COMPARATIVE RESULTS OF DIFFERENT METHOD
Table V lists the comparative results. The results show that
Method Class P/ % R/ % F1
KCD method is superior to other state-of-the-art methods. In
Normal 96.88 93.73 0.9527
detail, STM, similarity-based and our KCD methods outper-
First scheme [9] Damaged 32.78 55.56 0.4123
form the MYOLOv3-Tiny and DC-TLMDDNet. Furthermore,
Missing 63.49 71.43 0.6723
similarity-based and KCD methods outperform STM method. Normal 97.56 97.67 0.9761
These advantages can be attributed to three reasons. The first is Second scheme [29] Damaged 70.59 66.67 0.6857
that MYOLOv3-Tiny and DC-TLMDDNet cannot solve the Missing 75.44 76.78 0.7610
problem of imbalanced samples. For the detection of imbal- Normal 98.14 98.60 0.9836
anced fastener samples, the two methods are inefficiency. The Third scheme [11] Damaged 84.85 77.78 0.8116
second is that the deep learning-based methods have stronger Missing 83.64 82.14 0.8288
feature representation ability and they can better recognize the Normal 99.18 99.41 0.9929
Our proposed method
fastener states. Thus, similarity-based and KCD methods are Damaged 94.28 91.67 0.9295
(KCD)
superior to STM method. The third is that the number of de- Missing 92.73 91.07 0.9189

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

Fig. 13 Some fastener inspection reports of online test.

TABLE VI
THE DETAILED CONFIGURATION OF THREE EXPERIMENTAL SCHEMES
Feature extraction method Classification method
Experimental scheme Online learning strategy
LLBP HOG Template matching K-NN
First scheme [9] √ √
Second scheme [29] √ √ √
Third scheme [11] √ √ √ √

Table VII shows the comparative results. We can find that over 90%. These results strongly demonstrate the effectiveness
the KCD method still achieves the good detection performance of cascade learning approach.
for fasteners in complex railway environment. Meanwhile, its TABLE VIII
inspection performance is obviously better than the other three PERFORMANCE OF CASCADE LEARNING APPROACH
methods [9] [11] [29]. Thus, the KCD method is with good
DR / % Fastener P/% R/% F1
robustness and has the potential to be applied to the railway
Normal 99.1 99.4 0.9924
inspection system. In addition, Fig. 13 shows some visualized Fastener
98.2 Damaged 94.2 91.7 0.9293
inspection reports, which can more intuitively highlight the region
Missing 92.7 91.6 0.9214
robustness of our proposed method.
2) Effectiveness verification of cascade learning approach.
We choose 2092 railway images as test set and which contains
7350 normal, 396 damaged and 616 missing fasteners. Spe- VI. CONCLUSION
cially, for this experiment, the input image is railway image and This paper presented a cascade learning method to detect the
fastener region is firstly located. Then, the output of fastener fault fasteners. Compared with state-of-the-arts, the better
region positioning (fastener region images) is used as the input inspection performance for imbalanced fasteners is achieved by
of fastener inspection to recognize the fastener state. our proposed cascade learning method. In detail, the average
Table VIII shows the results of fastener region positioning precision and recall increased by 6.23% and 9.27%, respec-
and fastener state recognition. The proposed cascade learning tively. Meanwhile, the F1-score increased by 7.72%.
approach achieves good performance. In detail, for fastener In the experiment, we test the online inspection performance
region positioning, the detection rate (DR) reaches 98.2%. For of the designed fault detection IoT vehicle. The experiment
the fastener state recognition, the precision and recall can reach results show the fault detection IoT vehicle achieves an average

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

precision of 95.39% and an average recall of 94.05% on fas- study,” Engineering Applications of Artificial Intelligence, vol. 80, pp.
66–81, Apr. 2019.
tener inspection. These results indicate that the detection IoT
[21] X. Wei, D. Wei, D. Suo, L. Jia, and Y. Li, “Multi-Target defect identi-
vehicle can be used for fastener inspection in real railway. fication for railway track line based on image processing and improved
Next step, we are mainly concerned with further improving YOLOv3 model,” IEEE Access, vol. 8, pp. 61973-61988, 2020.
the performance of designed fault detection IoT vehicle and the [22] S. Zagoruyko and N. Komodakis, “Learning to compare image patches
via convolutional neural networks,” in Proc. IEEE Conf. Comput. Vis.
proposed methods in complex railway environment.
Pattern Recognit., pp. 4353–4361, Jun. 2015.
[23] O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra,
REFERENCES “Matching networks for one shot learning,” in Proc. Adv. Neural Inf.
[1] H. Li, K. Ota, and M. Dong, “Learning IoT in Edge: Deep Learning for Process. Syst., pp. 3630–3638, 2016.
the Internet-of-Things with Edge Computing,” IEEE Network Magazine, [24] J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot
vol.32, no.1, pp. 96-101, Jan. 2018. learning,” in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 4077–4087.
[2] H. Li, K. Ota, and M. Dong, “LS-SDV: Virtual Network Management in [25] D. Yao, Q. Sun, J. Yang, H. Liu and J. Zhang, “Railway Fastener Fault
Large-Scale Software Defined IoT,” IEEE Journal on Selected Areas in Diagnosis Based on Generative Adversarial Network and Residual
Communications, vol. 37, no. 8, pp. 1783-1793, Aug. 2019. Network Model,” Shock and Vibration, pp. 1-15, Nov. 2020.
[3] H. Liu, Y. Li, Z. Ma and C. Wang, “Recognition and Calibration of Rail [26] W. Liu, D. Anguelov, D. Erhan, et al., "SSD: Single shot multibox
Profile under Affine-Distortion-Based Point Set Mapping,” IEEE Trans. detector," arXiv preprint arXiv: 1512.02325, 2015.
Instrum. Meas., vol. 66, no. 1, pp. 131-140, Jan 2017. [27] K. Simonyan and A. Zisserman. (Sep. 2014). “Very deep convolutional
[4] K. Yu, L. Tan, L. Lin, X. Cheng, Z. Yi and T. Sato, networks for large-scale image recognition.” [Online]. Available:
“Deep-learning-Empowered Breast Cancer Auxiliary Diagmosis for https://arxiv.org/abs/1409.1556.
5GB Remote E-Health,” IEEE Wirel. Commun., vol. 28, no. 3, pp. 54-61, [28] S. Ren, K. He, R. Girshick, and S. Sun, “Faster R-CNN: Towards re-
Jun. 2021. al-time object detection with region proposal networks,” IEEE Trans.
[5] K. Yu, Z. Guo, Y. Shen, W. Wang, J. C. Lin, T. Sato, “Secure Artificial Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, Jun. 2016.
Intelligence of Things for Implicit Group Recommendations,” IEEE [29] Y. Dou, Y. Huang, Q. Li, and S. Luo, “A fast template matching-based
Internet Things J., 2021, doi: 10.1109/JIOT.2021.3079574 algorithm for railway bolts detection,” International Journal of Machine
[6] Y. Li, H. Trinh, N. Haas, C. Otto and S. Pankanti, “Rail Component Learning and Cybernetics, vol. 6, no. 5, pp. 835-844, 2014.
Detection, Optimization, and Assessment for Automatic Rail Track In- [30] J. Redmon, and A. Farhadi, (Dec. 2016). “YOLO9000: Better, faster,
spection,” IEEE Trans. Intell. Transp. Syst, vol. 15, no. 2, pp.760-770, stronger.” [Online]. Avaliable: https://arvix.org/abs/1612.08242
Apr. 2014. [31] J. Redmon and A. Farhadi, “YOLOv3: An incremental improvement,”
[7] H. Feng, Z. Jiang, F. Xie, P. Yang, and J. Shi, “Automatic Fastener 2018, arXiv:1804.02767. Available: http://arxiv.org/abs/1804.02767
Classification and Defect Detection in Vision-Based Railway Inspection
Systems,” IEEE Trans. Instrum. Meas., vol. 63, no. 4, pp. 877-888, Apr.
2014. Jianwei Liu is with the College of Electrical and
[8] T. Prasongpongchai, T. H. Chalidabhongse, and S. Leelhapantu, “A Information Engineering, Hunan University,
Vision-Based Method for the Detection of Missing Rail Fasteners,” in Changsha, China. He is a Ph.D. student with Hunan
IEEE International Conference on Signal and Image Processing Ap- University.
plications, 2017, pp. 419-424. His research interests include machine vision,
[9] H. Fan, P. C. Cosman, Y. Hou, and B. Li, "High Speed Railway Fastener railway fastener inspection and signal processing.
Detection Based on Line Local Binary Pattern." IEEE Signal Process.
Lett.., vol. 25, no.6, pp. 788-792, Jun. 2018.
[10] Y. Xia, F. Xie, and Z. Jiang, “Broken Railway Fastener Detection Based
on Adaboost Algorithm,” in Proc. Int. Conf. Optoelectron. Image Pro-
cess. Nov. 2010, vol. 1, pp. 313–316.
[11] J. Liu, Y. Huang, et al. “Learning Visual Similarity for Inspecting Hongli Liu received the B.Sc. degree in electrical
Defective Railway Fasteners.” IEEE Sensors J., vol. 19, no. 16, pp. engineering and the Ph.D. degree in control theory
6844-6857, Apr, 2019. and engineering from Hunan University, Changsha,
[12] F. Marino, A. Distante, P. L. Mazzeo, and E. Stella, “A Real-Time China, in 1985 and 2000, respectively. He is cur-
Visual Inspection System for Railway Maintenance: Automatic Hex- rently a Professor with the College of Electrical and
agonal Headed Bolts Detection,” IEEE Trans. Syst., Man, Cybern. C, Information Engineering, Hunan University.
Appl. Rev., vol. 37, no. 3, pp. 418–428, May 2007. His current research interests include intelligent
[13] J. Yang, W. Tao, M. Liu, Y. Zhang, H. Zhang, and H. Zhao, “An effi- information processing and transmission technol-
cient direction field-based method for the detection of fasteners on ogy.
high-speed railways,” Sensors, vol. 11, no. 8, pp. 7364–7381, Jul. 2011.
[14] Y. Ou, J. Luo, B. Li, and B. He, “A classification model of railway
fasteners based on computer vision,” Neural Comput. & Applic., vol. 31,
pp. 9307-9319, Dec. 2019. Dr. Chinmay Chakraborty, MIEEE is an Assistant
[15] Y. Zhang, M. Liu, Y. Chen, H. Zhang and Y. Guo, “Real-Time Vi- Professor in the Electronics and Communication
sion-Based System of Fault Detection for Freight Trains,” IEEE Trans. Engineering, Birla Institute of Technology, Mesra,
Instrum. Meas., vol. 69, no. 7, pp. 5274-5284, Jul. 2020. India and and Post-doctoral fellow of Federal
[16] G. Kang, S.Guo, L. Yu and D. Zhang, “Deep Architecture for University of Piauí, Brazil. He worked at the Fac-
High-Speed Railway Insulator Surface Defect Detection: Denoising ulty of Science and Technology, ICFAI University,
Autoencoder with Multitask Learning,” IEEE Trans. Instrum. Meas., vol. Agartala, Tripura, India as a Sr. Lecturer. He
68, no. 8, pp. 2679-2690, Aug. 2019. worked as a Research Consultant in the Coal India
[17] F. Wu, Q. Li, S. Li and T. Wu, “Train rail defect classification detection project at Industrial Engineering & Management,
and its parameters learning method,” Meas., vol. 151, Feb. 2020. IIT Kharagpur. He worked as a Project Coordinator
[18] Gibert, Xavier, V. M. Patel, and R. Chellappa. “Deep Multitask Learning of the Telecommunication Convergence Switch
for Railway Track Inspection.” IEEE Trans. Intell. Transp. Syst. vol. 18, project under the Indo-US joint initiative. He also worked as a Network Engi-
no. 1, pp. 153-164, Jan. 2017. neer in System Administration at MISPL, India. His main research interests
[19] H. Qi, T. Xu, G. Wang, Y. Cheng and C. Chen, “MYOLOv3-Tiny: A include the Internet of Medical Things, Wireless Body Sensor Networks,
new convolutional neural network architecture for real-time detection of Wireless Networks, Telemedicine, m-Health/e-health, and Medical Imaging.
track fasteners,” Computers in Industry, vol. 123, Dec. 2020. Dr. Chakraborty has published more than 140 papers at reputed international
[20] X. Wei, Z. Yang, Y. Liu et al,“Railway track fastener defect detection journals, conferences, book chapters, more than 20 books and more than 16
based on image processing and deep learning techniques: A comparative special issues. He is an Editorial Board Member in the different Journals and
Conferences. He is serving as a Guest Editor of MDPI-Future Internet Journal,

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2021.3126875, IEEE Internet of
Things Journal

Wiley-Internet Technology Letters, Springer-Annals of Telecommunications, Ziji Ma received the B.Sc. degree in electronic
Springer - Int. J. of System Assurance Engineering and Management, Spring- information engineering from Hunan University,
er-Environment, Development, and Sustainability, Wiley - Business and Soci- Changsha, China, in 2001, and the Ph.D. degree in
ety Review, and Lead Guest Editors of IEEE-JBHI, Hindawi- J. of Health information science from the Nara Institute of
Engineering, Mary Ann Liebert - Big Data J., IGI-Int. J. of E-Health and Science and Technology, Nara, Japan, in 2012. He
Medical Communications, Springer – Multimedia Tools and Applications, is currently an Associate Professor with the Col-
TechScience CMC, Springer - Interdisciplinary Sciences: Computational Life lege of Electrical and Information Engineering,
Sciences, Inderscience- International Journal of Nanotechnology, Bentham- Hunan University.
Science -Current Medical Imaging, Journal of Medical Imaging and Health His research interests include machine vision,
Informatics, Lead Series Editor of CRC- Advances in Smart Healthcare signal processing, and V2V communication. He is
Technologies, and also Associate Editor of International Journal of End-User a member of IEICE.
Computing and Development, Journal of Science & Engineering, Int. Journal
of Strategic Engineering, and has conducted a session of SoCTA-19, ICICC –
2019, Springer CIS 2020, SoCTA-20, SoCPaR 2020, and also a reviewer for
international journals including IEEE Access, IEEE Sensors, IEEE Internet of
Things, Elsevier, Springer, Taylor & Francis, IGI, IET, TELKOMNIKA Tel-
ecommunication Computing Electronics and Control, and Wiley.
Keping Yu received the M.E. and Ph.D. degrees
from the Graduate School of Global Information
and Telecommunication Studies, Waseda Univer-
sity, Tokyo, Japan, in 2012 and 2016, respectively.
He was a Research Associate and a Junior Re-
searcher with the Global Information and Tele-
communication Institute, Waseda University, from
2015 to 2019 and 2019 to 2020, respectively, where
he is currently a Researcher.
Dr. Yu has hosted and participated in more than ten
projects, is involved in many standardization ac-
tivities organized by ITU-T and ICNRG of IRTF,
and has contributed to ITU-T Standards Y.3071
and Supplement 35. He received the Best Paper Award from ITU Kaleidoscope
2020, the Student Presentation Award from JSST 2014. He has authored 100+
publications including papers in prestigious journal/conferences such as the
IEEE Wireless Communications, ComMag, NetMag, IoTJ, TFS, TII, T-ITS,
TVT, TNSE, TGCN, CEMag, IoTMag, ICC, GLOBECOM etc. He is an
Associate Editor of IEEE Open Journal of Vehicular Technology, Journal of
Intelligent Manufacturing, Journal of Circuits, Systems and Computers. He has
been a Lead Guest Editor for Sensors, Peer-to-Peer Networking and Applica-
tions, Energies, Journal of Internet Technology, Journal of Database Man-
agement, Cluster Computing, Journal of Electronic Imaging, Control Engi-
neering Practice, Sustainable Energy Technologies and Assessments and Guest
Editor for IEICE Transactions on Information and Systems, Computer Com-
munications, IET Intelligent Transport Systems, Wireless Communications and
Mobile Computing, Soft Computing, IET Systems Biology. He served as
general co-chair and publicity co-chair of the IEEE VTC2020-Spring 1st
EBTSRA workshop, general co-chair of IEEE ICCC2020 2nd EBTSRA
workshop, general co-chair of IEEE TrustCom2021 3nd EBTSRA workshop,
session chair of IEEE ICCC2020, TPC co-chair of SCML2020, local chair of
MONAMI 2020, Session Co-chair of CcS2020, and session chair of ITU
Kaleidoscope 2016. His research interests include smart grids, infor-
mation-centric networking, the Internet of Things, artificial intelligence,
blockchain, and information security.
Xun Shao received the Ph.D. degree from Osaka
University, Japan, in 2013. He was a Researcher
with the National Institute of Information and
Communications Technology, Japan, from 2013 to
2017. He is currently an Assistant Professor with
the School of Regional Innovation and Social
Design Engineering, Kitami Institute of Technol-
ogy, Japan. His research interests include com-
puter networks and distributed systems. He is a
member of IEICE.

2327-4662 (c) 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on April 05,2022 at 13:45:03 UTC from IEEE Xplore. Restrictions apply.