2 Peimankar2019
2 Peimankar2019
Ensemble Learning
Abdolrahman Peimankar* Mona Jafar Jajroodi* Sadasivan Puthusserypady
Department of Health Technology Department of Health Technology Department of Health Technology
Technical University of Denmark Technical University of Denmark Technical University of Denmark
Kongens Lyngby, Denmark Kongens Lyngby, Denmark Kongens Lyngby, Denmark
[email protected] [email protected] [email protected]
Abstract—Objectives: Electrocardiogram (ECG) is an impor- automatic analysis and classification of cardiac rhythms. This
tant clinical tool for diagnosis of cardiac abnormalities. Physi- can significantly simplify the diagnosis of cardiac arrhythmia,
cians make diagnoses by visual examination of ECGs. Analysing which is a great assistance to the clinicians [1], [4]–[6].
huge amounts of ECGs however, can be very time consuming
and cumbersome. Hence, developing analytic software is of great In the past decades, several techniques have been pro-
importance to automatically analyse these ECG signals to detect posed for automatic ECG-based heartbeat classification and
efficiently the common cardiac arrhythmias. Methods: Proposed arrhythmia detection. Faezipour et.al proposed a wavelet based
an ensemble learning approach for automatic processing of ECG technique to classify ECG beats [4]. A linear discriminant
signals and classification of arrhythmias. Twenty six features analysis based algorithm was introduced in [1]. In [5], the
(based on wavelets, heartbeat intervals, and RR-intervals) are
extracted and three algorithms, namely, Random Forest (RF), authors presented a neural network (NN) based method for
Adaptive Boosting (AdaBoost) and Artificial Neural Network automatic classification of five different ECG classes (Table
(ANN) are utilized for classification. Results: The proposed I). Support vector machine (SVM) based methods are also
method is evaluated on ECG signals from 44 recordings of applied to classify ECG signals [3] [7] [8]. Melgani and
the MIT-BIH arrhythmia database. The overall classification Bazi proposed a SVM based method to automatically classify
accuracy of the RF, AdaBoost, and ANN are 96.16%, 96.16%
and 94.49%, respectively. Additionally, the overall classification ECG beats and compared it with two other classifiers (k-
accuracy of the ensemble model is improved to 96.18%. Con- Nearest Neighbors (kNN) and Radial Basis Function (RBF)
clusion: Experimental results show that the performance of the NN classifier) [7]. Mondéjar-Guerraa proposed a new method
ensemble model for ECG heartbeat classification improves the for ECG classification based on an ensemble of SVMs [3].
overall accuracy. Significance: This paper proposes an accurate Ensemble learning is one of the most popular machine
and easy to use approach to classify heartbeats into one of the
five classes recommended by ANSI/AAMI standard, which can learning techniques, which can be utilized in different types
be used in real-time within a tele-health monitoring framework. of problems such as classification and regression [9]. It has
been used in electrical engineering as well as biomedical en-
Index Terms—electrocardiogram (ECG), heartbeat classifica- gineering [12] [10]. Every ensemble learning models contains
tion, ensemble learning, feature extraction. three main parts: (1) Creating a training set by sampling from
the dataset, (2) Training a group of different classification
I. I NTRODUCTION
algorithms, and (3) Combining the predictions of classification
Cardiac arrhythmia is an important issue due to its high algorithms. One of the most desirable advantages of using
prevalence, associated high mortality and high cost of treat- ensemble learning is having a more accurate classification
ment [1]–[3]. According to the World Health Organization model by bypassing the selection of a single weak classifier
(WHO), one of the major causes of death in the world is [9].
cardiovascular diseases [3]. Twenty-six features (Table II) are extracted and used as
In order to diagnose cardiac arrhythmia and provide im- inputs to the three classifiers in order to distinguish between
portant information about heart’s conditions, the most basic normal heartbeats and four different arrhythmia classes. The
and available method is the electrocardiography (ECG) [2], three classification algorithms used in this study are, Random
[4], [5]. However, it can be cumbersome for a physician to Forests (RF) [13], Adaptive Boosting (AdaBoost) [14], and
visually analyse these long ECGs in a short time. For instance, Artificial Neural Networks (ANN) [15]. Each of these three
diagnosis of some arrhythmia takes up to several hours to single classification algorithms is trained by utilizing 5-fold
even days by visual analysis [1], [4]. Therefore, to facilitate cross validation technique. Later, the outputs of these classifi-
the analysis of long ECG recordings, there is a need to use cation algorithms are added by using Dempster-Shafer theory
computational techniques and develop effective algorithms for (DST) [16] to better the performance.
The rest of the paper contains five sections. Section II briefly
*Both authors contributed equally to this work.
This research work is supported by the Innovation fund, Denmark, REAFEL presents the methods used in this study. Ensemble heartbeat
(IFD Project No: 6153-00009B). classification algorithm is explained in Section III. Later,
978-1-7281-1895-6/19/$31.00 2019
c IEEE 383
Section IV shows experimental results as well as comparison Next, various statistical features are extracted from the wavelet
with other algorithms, and lastly, Section V provides the coefficients, such as the minimum, maximum, average and
conclusion of this paper. coefficient variance. These twenty features are extracted from
each subband.
II. M ATERIALS AND M ETHODS
2) RR-interval features: RR-intervals is the distance be-
A. Database tween the heartbeats R peaks. Three RR-intervals based fea-
ECG data for this work are collected from MIT-BIH ar- tures were extracted, which are the pre-RR interval, post-RR
rhythmia database [17], [18]. It contains 48 data records each interval and local average RR-interval. The pre- and post-
about 30 min long (sampled at 360Hz). As per AAMI’s RRintervals are the RR-interval between a given heartbeat with
recommendation, four recordings (#s102, 104, 107, and 217) the previous and following heartbeat, respectively. Lastly, the
which contain paced beats, were removed from the analysis local average RR-interval is defined as the average of the RR-
since these beats do not preserve appropriate signal quality. intervals of ten RR-intervals neighboring one heartbeat i.e. five
In addition, the AAMI convention that is used to combine the RR-intervals to the left and five RR-intervals to the right of
beats into five classes as shown in Table I [19]. the given heartbeat.
3) Heart-beat morphology: Three morphology features,
TABLE I: Heartbeats categories in MIT-BIH database which are QRS duration (time interval between the beginning
and end of the QRS segment (from QRS onset to QRS offset),
Group Class
Normal (N) beat T-wave duration (time interval between the QRS offset and the
Left (L) and right (R) bundle branch block beats T-wave offset) and P-wave absence/present, are extracted from
N Atrial escape (e) beat each ECG heartbeat intervals. The P wave flag is defined as
Nodal (junctional, j) escape beat
Atrial premature (A) beat a Boolean variable indicating the presence or absence of a P-
Aberrated atrial premature (a) beat wave in each heartbeat segmentation. These twenty six features
S Nodal (junctional, j) premature beat are summarised in Table II.
Supraventricular premature (S) beat
V Premature ventricular (V) contraction
Ventricular escape (E) beat TABLE II: Features Description
F Fusion of ventricular and normal (F) beat
Fusion of paced and normal (f) beat Features Feature Description
Q Unclassifiable (U) beat F1-F5 Minimum of wavelet coefficients
F6-F10 Maximum of wavelet coefficients
F11-F15 Mean of wavelet coefficients
F16-F20 Variance of wavelet coefficients
B. Preprocessing F21 Post-RR interval
F22 Pre-RR interval
1) Signal filtering: The ECG signals are filtered by using
F23 Local average RR-interval
a band-pass filter with a minimum-order FIR filter (due to the F24 QRS duration
input signal is long enough) [20], and in range of 0.5-40 Hz, F25 T-wave duration
in order to remove wander and high frequency noise. F26 P-wave flag
2) QRS complex detection and heartbeat segmentation:
One cycle of an ECG signal consists of three component
waveforms: QRS comples, P-wave and T-Wave. The accurate
locations of the signal peaks, beginning and end points of these D. Beat Classification
component waveforms are used to segment hearbeats [8]. In
order to detect the QRS complexes and locating the beginning, 1) Random Forest (RF): RF is one of the commonly used
peak as well as the end of the P, QRS, and T waves, the WFDB ensemble machine learning approaches [22]. This algorithm
Toolbox in Physionet is used [17], [21]. Next, the signals are consists of various decision trees in the learning phase and
segmented into beats, which means that it starts from P-wave makes a decision based on the majority voting [23]. In order
onset and ends at T-wave offset. If P-wave is not present, then to achieve optimal values for trees, different values of trees are
each segment start from QRS onset and end at T-wave offset. implemented and their classification accuracy are compared.
Accordingly, in this study, 20 trees are chosen in RF as it gave
C. Feature Extraction the optimal accuracy results.
1) Wavelet transform: Wavelet transform provides a charac- 2) Adaptive Boosting (AdaBoost): This algorithm was in-
terization in frequency and temporal domains [8]. In this paper, troduced in 1997 [14]. AdaBoost is more sensitive to noisy
the Daubechies wavelet function (db2) is utilized, because it is data and outliers but less susceptible to the over-fitting com-
more similar to the ECG signal morphology compared to other pared to RF. In this method, the decision trees are used
wavelet functions. Additionally, the number of decomposition as base classifiers and assign higher weights to the mis-
levels is four. This means that ECG signals are decomposed classification cases in the next iteration. In this way, the chance
into the details D1 − D4 , as well as one approximation of incorrectly classified samples to be chosen for training the
coefficient A4 . These five decomposition are called subbands. next decision trees are higher.
Validation set
[24]:
Training set
m : 2X → 0, 1 , (1) Train all classifiers 7
m(∅) = 0, (2)
m(A) = 1, (3) Evaluate trained classifiers 8
A⊆X
where X and ∅ are defined as the universal set and the empty Combined the outputs of
9
set, respectively. DST combines the two mass probabilities to the classifiers using DST
define more precise evidence, as below:
1
(m1 ⊕ m2 , ) = m1 (B)m2 (C), (4) Fig. 1: Flowchart of the ECG classification technique.
1−P
B∩C=A=∅
where, P = B∩C=∅ m1 (B)m2 (C).
of biased classification [25]. In order to balance the
III. E NSEMBLE H EARTBEAT C LASSIFICATION
dataset and consequently improve the performance of the
Figure 1 illustrates the flowchart of the proposed ensemble classification algorithms, a method, namely ADASYN
model for heartbeat classification. The algorithm is described (adaptive synthetic over-sampling technique) was used.
in following steps: It contains three steps: (i) First, the degree of class
1) Reducing noise: In this step, the artifacts and baseline imbalance is computed in order to determine the number
wanders are filtered and removed from the ECG records. of synthetic samples for the minority class, (ii) In a
2) Detecting QRS complex: The P-waves, QRS complexes, minority class, the Euclidean distance is determined in
and T-waves in each ECG signal are detected using the order to find the K nearest neighbours, and (iii) Finally,
WFDB toolbox in PhysioNet. the synthetic samples are generated for the minority
3) Segmentation: All the signals are segmented into the class by using, di = xi + λ(xki − xi ). Here, xi is a
heart beats, as mentioned in Section II-B. sample from minority class, xki is one of the samples
4) Feature extraction: The aim of extracting features is to from the K nearest neighbors. Additionally, λ ∈ [0, 1]
reduce the pattern vector to a lower dimension, with the is a random number.
most discriminatory information from original vector. 6) Cross validation: The dataset is split into training and
Three different types of features are extracted as men- validation sets, by utilizing the 5-fold cross validation.
tioned in Section II-C, which are based on RR-Intervals, 7) Training the classifiers: The 5-fold training sets are
Heart-beat morphology, and Wavelet transform. utilized to train the classifiers (RF, AdaBoost and ANN).
5) Synthetic dataset generation: The used dataset is imbal- 8) Evaluating the trained classifiers: Then, the performance
anced and the number of normal heartbeats are much of the algorithms are evaluated on the 5-fold validation
more than the number of heartbeats for other classes. sets.
Using the imbalanced dataset may increase the chance 9) Combining of classifier outputs: Lastly, the predictions
of classifiers are combined by utilizing DST technique. TABLE IV: The Accuracy of Individual Classifiers.