1SPDFFEJOHT
PG
UIF

*OUFSOBUJPOBM
$POGFSFODF
PO
.

BDIJOF
-FBSOJOH
BOE
$ZCFSOFUJDT
/JOHCP
$IJOB

+VMZ


COMPUTER-AIDED DIAGNOSIS SYSTEM FOR RHEUMATOID ARTHRITIS

USING MACHINE LEARNING
KENTO MORITA1,2 , ATSUKI TASHITA1 , MANABU NII1 , SYOJI KOBASHI1

1
Graduate School of Engineering, University of Hyogo, Hyogo, Japan
2
Research Fellow of the Japan Society for the Promotion of Science
E-MAIL: [email protected]

Abstract: propriate treatment, and it requires enormous amount of time

There are 700,000 Rheumatoid Arthritis (RA) patients in for assessments. The X-ray image analysis based automatic
Japan, and the number of patients is increased by 30,000 annu- mTS score calculation system is needed.
ally. The early detection and appropriate treatment according to The fully automated mTS score calculation system requires
the progression of RA are effective to improve the patient’s prog- the automatic finger joint detection method. Ref. [2] propos-
nosis. The modified Total Sharp (mTS) score is widely used for es the deep learning based finger joint detection method. Their
the progression evaluation of Rheumatoid Arthritis. The mTS s- focus is not on the RA patients but on the children whose finger
core assessments on hand or foot X-ray image is required several joint is growing. Another finger joint detection method detect-
times a year, and it takes very long time. The automatic mTS s- s the finger joints using the X-ray image intensity difference
core calculation system is required. This paper proposes the fin- in the joint space [3]. The joint space based method does not
ger joint detection method and the mTS score estimation method provide the good detection result because severe RA patient’s
using support vector machine. Experimental results on 45 RA collapsed finger joint has no joint space.
patient’s X-ray images showed that the proposed method detect-
The mTS score evaluates the erosion score and JSN score
s finger joints with accuracy of 81.4 %, and estimated the ero-
for each finger joints. Ref. [3] automatically estimates the JSN
sion and JSN score with accuracy of 50.9, 64.3 %, respectively.
score of the mild RA patient. The method can not evaluate
the JSN score of severe RA patient whose joint does not have
Keywords:
enough joint space.
Rheumatoid arthritis; X-ray image; modified Total Sharp S-
core; Machine learning; Computer-aided diagnosis This paper introduces the fully automated finger joint detec-
tion method and mTS score estimation method for the mild-to-
severe Rheumatoid Arthritis patients using hand X-ray image.
1. Introduction

There are 700,000 Rheumatoid Arthritis (RA) patients in 2. Subjects and materials
Japan, and the number of patients is increased by 30,000 annu-
ally. Early treatment improves patient’s prognosis and Quality
of Life (QoL). The appropriate treatment in accordance with This study uses hand X-ray image of 45 mild-to-severe RA
RA progression is required for the better prognosis. patients. We had obtained informed consent from all subjects.
The hand or foot X-ray images are used for the RA diagno- Figure 1 shows the hand X-ray image which have 2010 × 1572
sis. The modified Total Sharp (mTS) score evaluates the ero- pixels. There are 14 finger joints in one hand, and the center
sion and joint space narrowing (JSN) on 32 hand joints and 12 points of finger joints are manually extracted. The erosion and
foot joints [1]. The 5 grades of erosion score and 4 grades of JSN score is manually determined.
JSN score is manually given for each joint. The sum of cal- The X-ray image is divided to left and right side, and the
culated score corresponds to the RA progression. The medical right side is inverted horizontally in order to increase the num-
doctor calculates the mTS score several times a year for the ap- ber of subjects.


` 
*&&&

 1SPDFFEJOHT
PG
UIF

*OUFSOBUJPOBM
$POGFSFODF
PO
.BDIJOF
-FBSOJOH
BOE
$ZCFSOFUJDT
/JOHCP
$IJOB

+VMZ


(a) Positive class

(a) Mild RA patient

(b) Negative class

(b) Severe RA patient FIGURE 2. Image patches for the training of the finger joint detector.

FIGURE 1. X-ray images of RA patients. The finger joints have been

damaged in severe RA patient.
3.1. Training of the finger joint detector

3. Proposed methods The proposed method detects finger joints from hand X-ray
image using two-class SVM. The SVM is trained using 100 ×
RA patients have risk of osteoporosis for many reason. The 100 pixels of image patches segmented from hand X-ray image.
one reason is the localized and the systemic bone metabolism The positive class has 28 image patches for each subject, and
by the RA itself. Therefore, the X-ray signal intensity differ- their center point is manually extracted center points of finger
ence of bone region and the other body parts is reduced accord- joints. In order to increase the robustness against the finger in-
ing to the RA progression. The bone contour or the joint space clination, the positive class image patch is rotated from -30 to
distance based method can not detect with enough accuracy be- 30 degree at and interval of 15 degree. The proposed method
cause the finger joints are collapsed in severe cases. segments 140 image patches from X-ray images for the nega-
This study expresses the rough shape of finger joint using tive class. The center point of the negative class image patche is
histogram of oriented gradients (HOG). The support vector ma- randomly determined in the human body region extracted using
chine (SVM) using HOG detects finger joints on the X-ray im- adaptive thresholding.
age, and the modified Total Sharp (mTS) score is estimated by The proposed method calculates the HOG of 140 and 140
the support vector regression (SVR). The details of the pro- image patches in the positive and negative class. The two-class
posed method are described in the following. SVM is trained using HOG of positive and negative classes.


 1SPDFFEJOHT
PG
UIF

*OUFSOBUJPOBM
$POGFSFODF
PO
.BDIJOF
-FBSOJOH
BOE
$ZCFSOFUJDT
/JOHCP
$IJOB

+VMZ


3.3. mTS score estimation

RA reduces the joint space width and deforms bone contour.

The proposed method expects the intensity gradients of the RA
patient’s finger joint is changed according to the RA progres-
sion. The proposed method estimates the mTS score by the
SVR using HOG feature. The positive class image patches used
in the finger joint detection are used for the training data. We
manually determined the erosion and JSN score of each finger
joint for the teacher data.

4. Experimental results

(a) Mild RA patient The proposed method was applied to 45 RA patients’ hand
X-ray images. Each positive and negative dataset have 6300
image patches. The parameters used in SVC and SVR for the
joint detection and mTS score estimation is shown in Table. 1,
2, respectively. The proposed method was implemented using
scikit-learn [4].
TABLE 1. Training parameters for the finger joint detection.

parameter value
kernel linear
C 0.5

TABLE 2. Training parameters for the mTS score estimation.

parameter value
(b) Severe RA patient kernel rbf
gamma 0.1
FIGURE 3. Detected finger joints. The blue rectangles show the ground C 0.5
truth, and the yellow rectangles show the result of the proposed method.

TABLE 3. mTS score estimation result (erosion).

success rate 50.9 (%)

3.2. Finger joint detection absolute error 0.59 ± 0.24

4.1. Finger joint detection

The proposed method evaluates 100 × 100 pixels of image Figure 2(a) and 2(b) show the positive and negative class im-
patches at all points in the subject’s hand X-ray image. SVM age patches, respectively. We performed the leave-one-subject-
outputs the value from -1 to 1, and 1 corresponds to the fin- out cross validation test in the following experiments. The eval-
ger joint. Finger joints are detected by clustering the evaluat- uating subject’s image patches were excluded from the training
ed patches. The patches are sorted in descending order by the dataset.
SVM output, and 28 patch clusters are extracted in descending Figure 3 shows the finger joint detection results. The blue
order. Note that a patch is merged to the patch cluster when 100 × 100 rectangle show the ground truth whose center is
25 % of the patch region is overlapped to a patch in the patch manually extracted joint. The yellow rectangle show the de-
cluster. tected finger joint, and it covers all image patches in the cluster.


 1SPDFFEJOHT
PG
UIF

*OUFSOBUJPOBM
$POGFSFODF
PO
.BDIJOF
-FBSOJOH
BOE
$ZCFSOFUJDT
/JOHCP
$IJOB

+VMZ


TABLE 4. mTS score estimation result (JSN). proposed method detects finger joints with accuracy of 81.4 %.
success rate 64.3 (%)
The erosion and JSN score were estimated with accuracy of
absolute error 0.43 ± 0.12 50.9 % and 64.3 %, respectively. The estimation error of ero-
sion and JSN score were 0.59 ± 0.24 and 0.43 ± 0.12, respec-
tively. We consider the high estimation error variance harms
These results showed that the proposed method detects finger the estimation accuracy. These results suggested that the image
joints with high accuracy on the mild RA patient. The proposed patch based proposed method improves the estimation accuracy
method failed to detect 6 finger joints on severe RA patient. by increasing the number of subjects.
We calculated the success rate of the finger joint detection for
the numerical evaluation. We defined the successfully detected References
finger joints as its yellow rectangle region covers 75 % of the
blue rectangle region. The average of success rates was 81.4
[1] DM. Van Der Heijde, PL. Van Riel, FW. Gribnau, IH.
%, and 22 finger joints over 28 finger joints were successfully
Nuver-Zwart, and LB. Van De Putte, “Effects of hydrox-
detected by the proposed method.
ychloroquine and sulphasalazine on progression of joint
damage in rheumatoid arthritis”, The Lancet, Vol 333, Is-
4.2. mTS score estimation sue. 8646, pp. 1036-1038, May, 1989.

The leave-one-subject-out cross validation test evaluated the [2] S. Lee, M. Choi, H. Choi, M.S. Park, and S. Yoon, “Fin-
proposed method in which the manually measured erosion and gerNet: Deep learning-based robust finger joint detec-
JSN scores were used as the ground truth value. Table 3 and tion from radiographs”, Biomedical Circuits and Systems
4 showed the mTS score estimation results. We defined the Conference (BioCAS), pp. 619-622, 2015.
success as the rounded SVR output has the same value with
the manually measured ground truth value. The absolute er- [3] Y. Huo, K.L. Vincken, D. van der Heijde, M. J. H. De
ror showed the average and variance of the difference of SVR Hair, F. P. Lafeber, and M. A. Viergever, “Automatic
output and ground truth value. These results showed that the quantification of radiographic finger joint space width of
estimation accuracy was insufficient but the average of the es- patients with early rheumatoid arthritis”, IEEE Transac-
timation error of both features were not high. These results tions on Biomedical Engineering, Vol 63, No. 10, pp.
suggested that the high variance of the absolute error harms the 2177-2186, 2016.
estimation accuracy. [4] F. Pedregosa, G. Varoquaux, A. Gramfort, V.i. Michel, B.
Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss,
5. Conclusions V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M.
Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Ma-
This paper introduced the finger joint detection method and chine Learning in Python”, Journal of Machine Learning
mTS score estimation method for mild-to-severe RA patients. Research, Vol 12, pp. 2825-2830, 2011.
The experimental results on 45 RA patients showed that the