ACHI 2018 : The Eleventh International Conference on Advances in Computer-Human Interactions

Detection and Classification of RBCs and WBCs in Urine Analysis

with Deep Network

Xingguo Zhang, Guoyue Chen, Kazuki Saruta and Yuki Terata

Faculty of Systems Science and Technology
Akita Prefectural University
Akita, Japan
e-mail: {xingguozhang, chen, saruta, terata}@akita-pu.ac.jp

Abstract—Urinary sediment examination is used to evaluate

the possible urinary tract diseases of patients. Currently,
numerous approaches are applied to automatically detect Red
Blood Cells (RBCs) and White Blood Cells (WBCs) from
urinary sediment images. However, it is still a challenging task
due to the cellular heterogeneity. Deep learning approaches
have been shown to produce encouraging results on image
detection in various tasks. In this paper, we investigate issues
involving Faster Regions with Convolutional Neural Network
(Faster R-CNN) for the construction of an end-to-end urine
analysis system. We propose an effective baseline for RBCs
and WBCs detection on urinary sediment images by using a
pre-train Faster R-CNN model. We evaluate our urine analysis
system on a large dataset of urinary sediment images which
consist of more than 6,000 annotated RBCs and WBCs images.
Our results show competitive accuracy and acceptable run
Figure 1. Isomorphic red blood cells (RBCs), dysmorphic RBCs
time. Prospectively, the proposed methods could provide
and write blood cells (WBCs).
support to pathology practice in terms of quantitative analysis
of tissue constituents in whole-slide images, and it could medical technologists and nephrologists were <25% and 69–
potentially lead to a better understanding of urinary tract
92%, respectively [5]. These differences could be caused by
diseases. Code and dataset will be made publicly available.
the number of dysmorphic red blood cells identified, which
Keywords- Urinary Sediment; RBC; WBC; Faster R-CNN; tend to be underestimated by image processing technologists.
Applications in Medicine. In addition, dysmorphic RBCs were often misclassified as
WBCs, besides, the reports issued by medical laboratory
I. INTRODUCTION technologists overlooked the description of dysmorphic
Currently, urinary sediment microscopy plays an RBCs.
important role in the clinical diagnosis of urinary tract One way to explore these cell types is to use
diseases [1][2]. It is possible to diagnose a patient's disease morphological clues in local neighborhoods to develop
by identifying the type and amount of sediment in the urine automated cellular recognition via image analysis, which can
specimen which is effective on early detection and disease then be deployed for sophisticated tissue morphometry.
control. The development of the Automatic Urinalysis There are several factors that impede automatic detection
System has been attractive to scholars over the past few and classification of RBCs and WBCs. On one hand, the
years. inferior image quality may arise due to autofocus failure
The higher clinical value of sediment in the urine mainly during the digitization of slide. On the other hand, complex
includes RBCs and WBCs. The qualitative and quantitative tissue architecture, clutter of nuclei, and diversity of nuclear
analysis of RBCs and WBCs in urine sediment can not only morphology pose a challenging problem.
be helpful in detecting the disease but also help to explore Particularly, in case of the large numbers of dysmorphic
various options for urethral diseases treatment. RBCs appearing at urinary sediment, the detection accuracy
Several solutions for urinary sediment microscopy using is significantly declined, due to the dysmorphic RBCs is very
computer vision methodology have been proposed by similar to the WBCs in terms of cellular morphology (Figure
different studies [3][4]. However, it is still a challenging task 1). Dysmorphic RBCs are usually transformed by the red
due to the cellular heterogeneity. blood cells being squeezed through the walls of the blood
In 2005, nephrologists began to report concerns about the vessels, causing the damage of cell wall. Furthermore, such
frequent discrepancies in some reported results of urine cases are not rare. Therefore, we need an effective approach
sediment microscopy [5][6]. For instance, the diagnostic to distinguish the variety of objects from urine sediment,
accuracy rates of acute kidney injury based on reports by especially isomorphic RBCs, dysmorphic RBCs and WBCs.

Copyright (c) IARIA, 2018. ISBN: 978-1-61208-616-3 194

ACHI 2018 : The Eleventh International Conference on Advances in Computer-Human Interactions

Figure 2. Our urinary sediment detection pipeline.

Faster R-CNN [14] is a particularly successful method generates candidate boxes, convolutional feature maps, and a
for general object detection. It consists of two components: Boosted Forest that classifies these proposals using these
region proposal and estimate object candidates, which is convolutional features. Korsuk et al. [12] proposed a SC-
generally considered to be more effective for smaller objects CNN classifier to detect colon cancer cells.
detection. However, there are few studies on the use of
Faster R-CNN for urinary sediment detection. III. CONVOLUTIONAL NEURAL NETWORK
In this paper, we would investigate issues involved Faster CNN is one of the basic theories for deep learning,
R-CNN for construction of end-to-end urine analysis system. therefore we briefly recap such network.
In addition, we would propose an effective baseline for A CNN is a composition of a sequence of L functions
RBCs and WBCs detection on urinary sediment images by or layers ( , … , ) that maps an input vector x to an output
using a pre-train Faster R-CNN model. In here, we expect vector y, i.e.,
both isomorphic RBC and dysmorphic RBC to be detected
as RBC.  = ( ; , … , ) 
The rest of the paper is organized as follows: Section 2 = (∙ ; )∘ (∙ ; ) ∘ …
introduces the related works. Section 3 presents the pipeline ∘ (∙ ; ) ∘ (∙ ; )
that we used for detecting RBCs and WBCs. Section 4 is
devoted to experimental evaluation, whereas conclusions are where is the weight and bias vector for the l the layer fl.
drawn in Section 5. Conventionally, is defined to perform one of the
II. RELATED WORK following operations: a) convolution with a bank of filters;
b) spatial pooling; and c) non-linear activation. Given a set
Cell and nucleus classification have been applied to of N training data {( , )} , we can estimate the vectors
diverse histopathology related applications. Most existing ,…, by solving the optimization problem
methods for cells detection share similar computation
pipelines: thresholding followed by morphological
operations, region growing, level sets, k-means, and graph-  argmin ,…, ∑ ( ( ; ,…, ), ) 
cuts.
Cosatto et al. [7] proposed the detection of cell nuclei where l is an appropriately defined loss function. Numerical
using the difference of Gaussian (DoG) followed by Hough optimization of (2) is often performed via backpropagation
transform to find radially symmetrical shapes. Vink et al. [8] and stochastic gradient descent methods.
employed AdaBoost classifier to train two detectors, one
used pixel-based features and the other merged the results of IV. RBC AND WBC DETECTION BASED ON FASTER R-CNN
two detectors to detect the nuclei in immunohistochemistry In this section, we describe our RBCs and WBCs
stained breast tissue images on the base of Haar-like features. detection pipeline for urinary sediment images by Faster R-
Dalle et al. [9] and Cosatto et al. [7] used shape, texture and CNN [14].
size of nuclei for nuclear pleomorphism grading in breast Faster R-CNN consists of two components, as shown in
cancer images. Figure 2: an RPN that generates candidate boxes as well as
Recently, the prevalent success of deep learning convolutional feature maps, and a Fast R-CNN used for
approach in computer vision, such as Regions with object detection. RPN is used to compute candidate
Convolutional Neural Network (R-CNN) [10], Region bounding boxes, scores, and convolutional feature maps.
Proposal Network & Binary Forest (RPN_BF) [11] and The candidate boxes are fed into Fast R-CNN for further
Spatially Constrained Convolutional Neural Network (SC- classification, using the features pooled from the
CNN) [12] have shown good performance on a large number convolutional feature maps computed by RPN. Finally, non-
of histopathological image datasets. The R-CNN method maximum suppression (NMS) is used to merge the similar
[10] trains CNNs end-to-end to classify the proposal regions results and get the output. In here, we use a urinary sediment
into object categories or background. Fast R-CNN [13] image dataset to train the RPN and Fast R-CNN network.
enables end-to-end detector training on shared convolutional
features and shows compelling accuracy and speed. In [11],
an RPN_BF approach has been proposed an RPN that

Copyright (c) IARIA, 2018. ISBN: 978-1-61208-616-3 195

ACHI 2018 : The Eleventh International Conference on Advances in Computer-Human Interactions

A. RPN network V. EXPERIMENT AND RESULTS

The RPN network shares full-image convolutional In this section, we describe our experimental setup details
features with the detection network, thus enabling nearly and the results.
cost-free region proposals. An RPN is a fully convolutional
network that simultaneously predicts object bounds and A. The dataset
objectness scores at each position. The RPN is trained end- This study involves electron microscope image. We
to-end to generate high-quality region proposals, which are collected urine samples from 100 patients, and through urine
used by Fast R-CNN for detection. centrifuge to obtain urine sediment. It was then magnified
We fixed the aspect ratio of anchors (Region Proposal by an electron microscope 400 times and photographed with
Boxes) [14] as 1:1 (width / height). This is because that we a digital camera. All images have a common size of 1280 ×
need a square box to mark the positions of the cells. This is 1024 pixels.
unlike the original RPN [14] for detect general object that Manual annotations of nuclei were conducted mostly by
has anchors of multiple aspect ratios. In order to detect an experienced pathologist and partly by a graduate student
multi-scale RBCs and WBCs, we use anchors of 3 different under supervision of and validation by the same pathologist.
scales, starting from 20 pixels length of square box side with A total number of 5,215 RBCs and 4,828 WBCs were
a scaling stride = 1.2.
Following [14], we adopt the VGG-16 net [15] pre-
trained on the ImageNet dataset [16] to initialize the network
parameters. The RPN is built on top of the Conv5_3 layer,
which is followed by an intermediate 3×3 convolutional
layer and two sibling 1×1 convolutional layers for
classification and bounding box regression (more details in
[14]). The output layer of the RPN net provides confidence
scores and regression coordinate of the predicted boxes,
which can be used as the input for Fast R-CNN network. Figure 3. Example images of urinary sediment images dataset used in this
experiment.
B. Fast R-CNN network
For the detection process, we adopt Fast R-CNN marked. Figure 3 shows some examples of the urinary
network as mentioned at [14]. To speed up the process, [14] sediment images in the dataset. We selected 3,000 random
developed a technique that allows for sharing convolutional images as the training samples. The test samples also
layers between the two networks, rather than learning two included 2,000 images selected from the rest of the dataset.
separate networks. For the convenience of the reader, we B. Evaluation metrics
briefly recap such approach.
A 4-step training algorithm has been adopted to learn The objective of this experiment is to detect all RBCs
shared features via alternating optimization. In the 1st step, and WBCs in an image by locating their positions, and
the RPN network has been train. And this network is obtain their class labels. In particular, the performance of an
initialized with an ImageNet pre-trained model and fine- algorithm is evaluated in terms of the tradeoff between
tuned end-to-end for the region proposal task. In the 2nd precision, Recall, and F1 score.
step, they train a separate detection network by Fast R-CNN First, a detected bounding box and a ground truth
using the proposals generated by the 1st step. In the 3rd bounding box (GT) are considered a true positive (TP) if the
step, they use the detector network to initialize RPN area covered by their intersection ≥ 70%. A GT that does
training, but fix the shared convolutional layers and only not have a match is considered a False Negative (FN), or a
fine-tune the layers unique to RPN. Finally, keeping the Miss. A detected bounding box that does not have a
shared convolutional layers fixed, and fine-tune the unique matching GT is considered as a False Positive (FP). The F1
layers of Fast R-CNN. As such, both networks share the score (also F-score or F-measure) is a measure of a test's
same convolutional layers and form the Faster R-CNN accuracy. It considers both the precision p and the recall r of
network. In this solution, the RPN and Fast R-CNN the test to compute the score: p is the number of correct
networks are merged into one network during training. positive results divided by the number of all positive results,
and r is the number of correct positive results divided by the
C. Implementation Details number of positive results that should have been returned.
For RPN and Fast R-CNN training, an anchor is Here, we use F1 score to quantitatively assess the detection
considered as a positive example if it has an Intersection- performance. The F1 score is computed by the following
over-Union (IoU) ratio greater than 0.7 with one ground truth equation:
box, and otherwise consider as negative. For Fast R-CNN
training, we construct the training set by selecting the top- ∙
 = 2∙  
ranked 100 proposals of each image by RPN network. At test
process, we only use the top 300 proposals in an image,
which are classified by the Fast R-CNN. We adopt NMS to 
output the detect results.

Copyright (c) IARIA, 2018. ISBN: 978-1-61208-616-3 196

ACHI 2018 : The Eleventh International Conference on Advances in Computer-Human Interactions


(a) (b) (c)
Figure 4. Qualitative results for RBCs and WBCs detection in urine images. (a) Ground truth image. (b) Detection results of our approach. (c) Detection
results of RPN_BF.

C. Comparative Results
In this section, our final detectors were evaluated with VI. CONCLUSION
other state-of-the-art methods using our urinary sediment
dataset. In this paper, we investigate issues involving Faster R-
Figure 4 and Table 1 report the performance on CNN for construction of end-to-end urine analysis system.
We proposed an effective baseline for RBCs and WBCs
detection and classification for our dataset. Figure 4 are
detection on urinary sediment images, using a pre-train
qualitative results for RBCs and WBCs detection in urine
Faster R-CNN model. Isomorphic, dysmorphic RBCs and
images. (a) Ground truth image. RBCs are shown as red
WBCs were successfully identified. We comprehensively
dashed Box, WBCs are shown as green dashed box. (b)
evaluate this method, the experiment results presenting
Detection results of our approach. (c) Detection results of
competitive accuracy and acceptable speed.
RPN_BF. Here, detected as RBCs are shown as red box,
Prospectively, the proposed methods could benefit the
and detected as WBCs are shown as green box. It can be
seen that our detector show better performance. pathology practice in terms of quantitative analysis of tissue
constituents in whole-slide images, and could potentially
TABLE I. COMPARATIVE RESULT FOR RBCS AND WBCS DETECTION lead to a better understanding of urinary tract diseases.
However, this study only focused on type 2 cells in
Method Weighted Average F1 score urinary sediment, our current results will require additional
Our 0.914 studies using a wider spectrum of cells and sediment, for
examples, Epithelial Cells, Bacteria, Yeast and Parasites. In
RPN_BF 0.862 future work, more theoretical and experimental studies will
HOG 0.688 be conducted to analyze the performance.

From the results of the images it can be seen that our REFERENCES
detector is competitive in terms of the detection quality with
[1] M. A. Perazella, “The Urine Sediment as a Biomarker of Kidney
respect to RPN_BF and provides significant improvement Disease,” American Journal of Kidney Diseases, vol. 66, pp. 748–
over HOG+SVM. 755, 2015.
In addition, we have observed the shapes of isomorphic [2] J. A. Simerville, W. C. Maxted, and J. J. Pahira, “Urinalysis: a
RBCs can change in response to the osmolarity of urine. comprehensive review,” American family physician, vol. 71, pp.
The isomorphic RBCs swell to spheres in urine with a low 1153–1162, 2005.
specific gravity, and they shrink to the shape of a spiked [3] M. Yasuda, “Japanese guideline for clinical research of antimicrobial
disk or a spiked sphere in urine with a high specific gravity. agents on urogenital infections,” Journal of Infection and
Chemotherapy, vol. 17, pp. 579–594, 2011.
The presence of dysmorphic RBCs leads to the difficulty
of distinguishing, which may seriously affects the accuracy [4] R. Davis, et al. “Diagnosis, evaluation and follow-up of
asymptomatic microhematuria (AMH) in adults,” ,Journal of
of the urinary sediment microscopy system. Therefore, a Urology, vol, 188, pp. 2473–2481, 2012.
precise count of dysmorphic RBCs is very important for [5] J. J. Tsai, J. Y. Yeun, V. A. Kumar, and B. R. Don, “Comparison and
evaluating glomerular bleeding before renal biopsy. interpretation of urinalysis performed by a nephrologist versus a
We profiled the execution of our system on a desktop hospital-based clinical laboratory,” American journal of kidney
architecture which features a 2.4GHz Intel i7 CPU, a diseases, vol, 46, pp. 820–829, 2005.
NVIDIA GTX1080 GPU and 32GB of RAM. The system [6] M. Kanbay, B. Kasapoglu, and M. A. Perazella, “Acute tubular
requires, on average, 96ms to process a frame at a resolution necrosis and prerenal acute kidney injury: utility of urine microscopy
in their evaluation a systematic review,” Int. Urol. Nephrol., vol. 42,
of 1280×1080 pixels. It can be consider as an acceptable pp. 425–433, 2010.
running time.

Copyright (c) IARIA, 2018. ISBN: 978-1-61208-616-3 197

ACHI 2018 : The Eleventh International Conference on Advances in Computer-Human Interactions

[7] E. Cosatto, M. Miller, H. P. Graf, and J. S. Meyer, “Grading nuclear Histology Images,” IEEE Transactions on Medical Imaging, pp.
pleomorphism on histological micrographs,” Int. Conf. Pattern 1196–1206, 2016.
Recognition, pp. 1–4, 2008. [13] R. Girshick, “Fast R-CNN: Fast Region-based Convolutional
[8] J. P. Vink, M. Van Leeuwen, C. Van Deurzen, and G. De Haan, Networks for object detection,” IEEE International Conference on
“Efficient nucleus detector in histopathology images,” Journal of Computer Vision, pp. 1440–1448, 2016.
microscopy, vol.249, no. 2, pp. 124–135, 2013. [14] S. Ren, K. He, R. Girshick, J. Sun, “Faster R-CNN: Towards Real-
[9] J. R. Dalle, H. Li, C. H. Huang, and W. K. Leow, “Nuclear Time Object Detection with Region Proposal Networks,” Advances
Pleomorphism Scoring by Selective Cell Nuclei Detection,” IEEE in neural information processing systems, pp. 91–99, 2015.
Winter Conf. on Applications of Computer Vision, pp. 1–6, 2009. [15] L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L. Yuille,
[10] R. Girshick, J. Donahue, T. Darrell, U. C. Berkeley, J. Malik, “R- “Semantic Image Segmentation with Deep Convolutional Nets and
CNN: Region-based Convolutional Neural Networks,” Computer Fully Connected CRFs,” International Conference on Learning
Vision and Pattern Recognition, pp. 2–9, 2014. Representations , pp. 1–14, 2016.
[11] L. Zhang, L. Lin, X. Liang, K. He, “Is Faster R-CNN Doing Well for [16] O. Russakovsky, et al. “ImageNet Large Scale Visual Recognition
Pedestrian Detection?” European Conference on Computer Vision, Challenge,” International Journal of Computer Vision, vol. 115, pp.
pp. 1–15, 2016. 211–252, 2015.
[12] K. Sirinukunwattana, S.E.A. Raza, Y.W Tsang, I.A. Cree, D.R.J.
Snead, N.M. Rajpoot,” Locality Sensitive Deep Learning for
Detection and Classification of Nuclei in Routine Colon Cancer

Copyright (c) IARIA, 2018. ISBN: 978-1-61208-616-3 198