TYPE Original Research

PUBLISHED 08 September 2022

DOI 10.3389/fninf.2022.1006494

Interpretable evaluation for the

OPEN ACCESS Brunnstrom recovery stage of
EDITED BY
Dmitrii Kaplun,
Saint Petersburg State Electrotechnical
the lower limb based on
University, Russia

REVIEWED BY
wearable sensors
Aleksandr Sinitca,
Saint Petersburg State Electrotechnical
University, Russia Xiang Chen1† , DongXia Hu1† , RuiQi Zhang2 , ZeWei Pan3 ,
Kandarpa Kumar Sarma, Yan Chen3 , Longhan Xie3 , Jun Luo1* and YiWen Zhu1*
Gauhati University, India
1
*CORRESPONDENCE Department of Rehabilitation Medicine, The Second Affiliated Hospital of Nanchang University,
Jun Luo Nanchang, China, 2 Fuzhou Medical College, Nanchang University, Nanchang, China, 3 Shien-Ming
[email protected] Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, China
YiWen Zhu
[email protected]
† These authors have contributed
equally to this work and share first With the increasing number of stroke patients, there is an urgent need for an
authorship
accessible, scientific, and reliable evaluation method for stroke rehabilitation.
RECEIVED 29July 2022 Although many rehabilitation stage evaluation methods based on the
ACCEPTED 16 August 2022 wearable sensors and machine learning algorithm have been developed, the
PUBLISHED 08 September 2022
interpretable evaluation of the Brunnstrom recovery stage of the lower limb
CITATION
Chen X, Hu D, Zhang R, Pan Z, Chen Y,
(BRS-L) is still lacking. The paper propose an interpretable BRS-L evaluation
Xie L, Luo J and Zhu Y (2022) method based on wearable sensors. We collected lower limb motion data
Interpretable evaluation and plantar pressure data of 20 hemiplegic patients and 10 healthy individuals
for the Brunnstrom recovery stage
of the lower limb based on wearable using seven Inertial Measurement Units (IMUs) and two plantar pressure
sensors. insoles. Then we extracted gait features from the motion data and pressure
Front. Neuroinform. 16:1006494.
doi: 10.3389/fninf.2022.1006494 data. By using feature selection based on feature importance, we improved
COPYRIGHT
the interpretability of the machine learning-based evaluation method. Several
© 2022 Chen, Hu, Zhang, Pan, Chen, machine learning models are evaluated on the dataset, the results show
Xie, Luo and Zhu. This is an
that k-Nearest Neighbor has the best prediction performance and achieves
open-access article distributed under
the terms of the Creative Commons 94.2% accuracy with an input of 18 features. Our method provides a
Attribution License (CC BY). The use, feasible solution for precise rehabilitation and home-based rehabilitation of
distribution or reproduction in other
forums is permitted, provided the hemiplegic patients.
original author(s) and the copyright
owner(s) are credited and that the
KEYWORDS
original publication in this journal is
cited, in accordance with accepted rehabilitation evaluation, Brunnstrom recovery stage, wearable sensor, machine
academic practice. No use, distribution learning, feature importance
or reproduction is permitted which
does not comply with these terms.

Introduction
Stroke is an acute cerebrovascular disease caused by bleeding or blockage of blood
vessels in the brain. Deaths from stroke account for 11% of all deaths in the world and
rank second among the leading causes of death (World Health Organization, 2020). In
some regions, such as Bulgaria, 3 out of 1,000 deaths are due to stroke (Kim et al., 2020),
indicating the high incidence and mortality of stroke. 80% of stroke survivors exhibit
hemiplegia due to loss of central nervous system control of the motor system, resulting in

Frontiers in Neuroinformatics 01 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

abnormal coordination of the patient’s muscle groups and TABLE 1 Description of each Brunnstrom recovery stage
(Naghdi et al., 2010).
abnormal muscle tones (Qiu et al., 2018), such as pain, swelling,
fatigue, and coordination problems (Alonso-Vázquez et al., Stage Description
2009), which severely affects the patient’s rehabilitation exercises
Stage 1 Lack of movement in the extremities.
and quality of life, and the risk of falls is raised. Therefore,
Stage 2 Slight voluntary motor response in the extremities and the onset of
rehabilitation evaluation and treatment of stroke has become
spasticity.
a major issue in public health, and hemiplegic gait analysis has
Stage 3 Patients can control synkinesis autonomously, spasticity is severe.
become an important part of rehabilitation.
Stage 4 Patients have control of detachment movements and spasticity
Studies have shown that timely, active, and accurate begins to diminish.
rehabilitation interventions can restore self-care to most Stage 5 The diminished role of co-movement and enhanced control of
patients with hemiplegia and performing accurate and separate movement.
appropriate rehabilitation is promising for the function of Stage 6 Normalization of movement and disappearance of spasticity.
the lower limb (Nepveu et al., 2017; Li et al., 2018; Hosseini
et al., 2019). Berengueres et al. (2014) found that if muscles
are not actively exercised when the foot is fixed in corrective is a burden for patients with hemiplegia who are already
shape, the muscles are not activated by training, which can having difficulty walking. Fortunately, the research conducted
make protective shock-absorbing function, effective gait and with the above devices provides a solid practical basis for the
other complex functions to be impaired. A proper hemiplegia clinical application of wearable sensors, such as the feasibility of
evaluation is crucial for correct treatment plan for hemiplegia evaluation methods based on joint angle and plantar pressure.
to better restore the patients’ muscle function. However, current In recent years, wearable sensors have highlighted great
clinical evaluation is based on clinician observation and clinical potential for clinical evaluation (Stuart et al., 2022; Zhang et al.,
scale evaluation, and the evaluation of stroke severity is based 2022). Berengueres et al. (2014) formed a smart insole by means
on the patient’s medical history as well as the examination. of pressure-sensitive sensors placed in the mid-lateral aspect of
The Brunnstrom recovery stage (BRS) is one of the most the insole to monitor the pressure in real-time, use pressure
popular motor function evaluation methods, which consists of thresholds to detect excessive internal rotation and pronation
three items for the arm, hand, and lower limb, each with six of the foot and provide feedback to the user to indicate if an
levels of flaccidity, spasticity, co-movement, partial dissociative abnormality is occurring. The Smart Textile Sock integrates five
movement, dissociative movement, and normal (Brunnstrom, pressure sensors and utilizes a pressure vector algorithm for pre-
1966), as shown in Table 1. Due to its high correlation with and post-rotation detection. It is relatively low cost and can
motor recovery in stroke patients, BRS has been extensively be used both indoors and outdoors (Domínguez-Morales et al.,
used in clinical as well as scientific research (Huang et al., 2019). In addition, diagnosis can be assisted by a physician using
2016). However, the results of the observation method rely an Inertial Measurement Unit (IMU) mounted at the heel by
heavily on the observer’s level of observation skills and clinical calculating parameters such as gait speed and gait (Qiu et al.,
experience. Besides, the evaluation process has a great impact on 2018). With the development of machine learning, deep neural
the patient’s comfort level, and the tedious operation also tends network analysis of the data from the IMU installed at the calf
to cause physical and mental fatigue and discomfort, which has been performed to identify drop-foot gait, pirouette gait,
cannot be recorded in real-time (Zhao et al., 2017). Therefore, hip hiking gait, and rear knee stroke gait (Wang et al., 2021).
the observation method is mostly used for the comparison These methods proved to be effective using wearable sensors to
of patients’ stages of rehabilitation, which cannot meet the assess the hemiplegic gait at a relatively low cost. However, this
requirements of clinical applications (Tran et al., 2018). part of the study could only detect the presence or absence of
To address the shortcomings of the observational scale hemiplegic gait or abnormal gait type and could not quantify
method, devices such as visual monitoring systems and plantar the degree, making it difficult to obtain a definite severity of
pressure monitoring systems have been used in more advanced hemiplegia (Mannini et al., 2016; Hsu et al., 2018). On the
rehabilitation units and laboratories for hemiplegic gait analysis. other hand, most studies have focused on the association of
However, the visual system and the dynamometric platform are upper limb behavior with the evaluation of FMA and ADLs, and
complicated to operate, the testing process can only move within few studies have focused on the degree of lower limb function
a certain area, it does not facilitate the timely adjustment of based on the analysis of gait parameters. Only 7 of 34 papers
the treatment method, and it also causes privacy issues under related to wearable sensors and machine learning mentioned
the surveillance of cameras. Therefore, the above devices still the lower limb and 3 mentioned gait (Boukhennoufa et al.,
do not meet the conditions for community-based rehabilitation, 2022). In general, studies have focused on the recognition of
i.e., they do not allow for immediate evaluation and feedback daily movements, motion classification, and clinical evaluation,
of the patient’s gait problems, and the patient still needs to and further research is needed on the use of wearable sensors
travel between home and hospital with high frequency, which for gait analysis for clinical evaluation. In particular, medical

Frontiers in Neuroinformatics 02 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

professionals are interested in the contextual information of Participants

the evaluation results, i.e., the interpretability of the evaluation
algorithms (Capela et al., 2015).
The following criteria were utilized to recruit participants:
According to the surveyed data, few scholars have focused
(1) Unilateral hemiplegia and undergoing rehabilitation in
on feature importance-based lower limb functional evaluation,
hospital; (2) Age between 18 and 80 years; (3) Brunnstrom
and few have seen a combination of wearable sensors based
recovery stages of lower limb (BRS-L) III-V; (4) Normal mental
on seven wearable IMU as well as dynamometric insoles, or
status and consciousness; (5) Subject can walk 10 meters indoors
feature analysis of lower limb functional evaluation. By using
(with or without assistive devices). Because patients in the stages
dual-source data fusion consisting of IMU and plantar pressure,
of BRS-L I-II are unable to walk (independently or assisted), we
the features extracted in the present paper are more accurate and
excluded them. To increase the sample size of the data set, we
also form a variety of multiple pre-selected features, including
also recruited 10 healthy individuals.
kinematic parameters, plantar pressure parameters, and spatial
Before carrying out this experiment, we obtained approval
parameters. Berengueres et al. (2014) indirectly determined
of the Ethics Committee of the Second Hospital of NanChang
inversion and eversion by monitoring plantar pressure with
University. Voluntary subjects signed informed consents before
sensors placed on the lateral side of the midfoot insole, but
the experiment. Finally, 30 individuals participated in the
neither gave an evaluation of the degree of lower limb function.
experiment, including 20 stroke subjects (aged 57.7 ± 8.7 years,
Also, some studies have used deep learning neural networks to
with a height of 164.5 ± 6.9 cm and a weight of 61.5 ± 9.1 kg)
increase the accuracy of the evaluation (Boukhennoufa et al.,
and 10 healthy subjects (aged 34.3 ± 2.5 years, with a height of
2021), but this failed to provide more contextual information to
173.6 ± 4.6 cm and a weight of 63.2 ± 6.6 kg).
the physician for this evaluation method.
The focus of this study is on using comfortable wearable
sensors for intelligent, self-service and reliable evaluation,
to achieve comprehensive, home-based and immediate
Experimental setup
rehabilitation. In this study, we use the joint motion of
the lower limb and the distribution of plantar pressure to Figure 1 shows the experimental setup. Seven IMUs were
assess the movement status of stroke patients. Considering selected for inclusion in the experiment and were strapped to
that stroke patients generally need to undergo lower the patient’s bilateral feet, bilateral calves, bilateral thighs, and
limb rehabilitation, this study objectively assesses the waist. The selected IMU sensor uses a JY901s inertial sensor chip
BRS-L in a graded manner. Due to the convenience of module made by Wit Motion, which includes an accelerometer,
the monitoring method in this study, the therapist can magnetometer, and gyroscope. Supplementary Table 1 displays
dynamically adjust the treatment plan during the prime the chip’s specific parameters.
rehabilitation period of the hemiplegic patient, which In addition, a smart insole was added to the shoe
is beneficial for the patient’s recovery. One benefit that to obtain plantar pressure. We designed and manufactured
distinguishes this study from other methods is that our method the smart insole, which incorporates 8 pressure sensors.
is interpretable, focusing on the selection of gait parameters and The position arrangement of pressure sensors is derived by
characteristics. minimizing the pressure center position measurement error.
The research methodology is first introduced in sections Please see article (Xian et al., 2021) for further details. The
“Experimental protocols” and “Materials and methods,” sampling frequency of IMU and pressure sensor is 200 Hz. The
including the design and conduct of experiments, equipment above devices use Bluetooth wireless transmission. To achieve
introduction, data preprocessing, calculation, analysis and multi-sensor synchronization, the upper computer examines the
selection of parameters and features, and model construction data transmitted by the sensor at a frequency of 200 Hz and
and performance evaluation. Section “Results,” this paper considers the most recent frame of data received by each sensor
presents the results of the experiments, which are finally as the data of the current moment.
discussed in section “Discussion.” After the devices are worn, the IMUs need to be calibrated
to obtain stable posture data. All patients first walked at their
comfortable walking speed in the shoes with built-in sensory
Experimental protocols insoles to perform an adaptation familiarization test. After
finding their comfortable walking speed and reaching a stable
We recruited volunteers to collect their gait data using gait, they begin to walk formally for about 2 min while recording
wearable IMUs and plantar pressure insoles, and the following plantar pressure data. The experiment is conducted in a quiet
is the experiment setup. environment in a hospital.

Frontiers in Neuroinformatics 03 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

FIGURE 1
Schematic diagram of wearable sensing and data analysis device.

FIGURE 2
Flowchart of the data processing.

Frontiers in Neuroinformatics 04 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

Materials and methods TABLE 2 Descriptions of gait features.

Gait Description
The method designed in this paper is shown in Figure 2. feature
The collected IMU data and plantar pressure data are used for
Gait line The trajectory line formed by the position center of pressure
further parameter and feature calculations. First, the data are
Regional Pressure ratios at different locations on the plantar
filtered and divided into steps. Then, the feature data of each pressure
step are calculated and correlation and importance analyses are Gait phase The proportion of each phase of a gait cycle
performed for each feature. Finally, the filtered features are used Acceleration Three-axis acceleration based on sensor coordinate system
for model training and realistic lower limb function evaluation. Step length Length of each step forward
Joint angle The rotation angle of the joint during the movement

Data preprocessing
the human walking process during the gait cycle can be obtained
To obtain a well-organized dataset, the collected raw data
with the gait cycle time length. The special feature of this paper
needs to go through a series of preprocessing.
is that the moment of touching the ground and leaving the
Low-pass filtering ground judged by the plantar pressure can be more accurate
to the integration start time of the stride length so that the
To eliminate the noise data, a 6-order Butterworth low-
stride length data can be obtained more accurately. And the
pass filter with a cut-off frequency of 15 Hz was used to
calculation process of quadratic integration is as follows:
preprocess the data.
At moment t, the transformation matrix of the attitude of
Gait cycle segmentation the foot sensor relative to the Earth coordinate system is ef R ,
t
and the three-axis acceleration of the foot sensor relative to
A gait cycle consists of heel touch, mid-foot touch, heel
the sensor coordinate system output at this time is Accft . Since
off, forefoot touch, forefoot off and swing. Since normal gait
the accelerometer is affected by the acceleration of gravity at
generally begins with heel touch, this study uses the response
this moment, the actual acceleration of motion during the walk
of heel sensors for touch judgment. We consider the sudden
Accfrt needs to be subtracted from the acceleration of gravity G.
increase in the value of the pressure sensor under the heel as
the activation signal, in other words, as the beginning of current
gait cycle and the end of previous gait cycle. Accfrt = Accft − ef RT · G (1)
t

Gait phase segmentation Since the Y-axis direction of the sensor is basically the same
Because hemiplegic patients do not have multiple support as the travel direction when the foot sensor is arranged, the
subphases as normal due to the muscle weakness, we only Y-axis direction of the sensor motion acceleration Accfrt Y can be
divide gait cycle into the double support phase and single obtained after the quadratic integration of the time from the t1
support phase, i.e., the affected side touches the ground as moment after the toe leaves the ground to the t2 moment when
the affected side double support phase, and then enters the the heel on the same side hits the ground, corresponding to the
affected side’s single support phase until the healthy side’s heel step length l:
touches the ground.
s t2
l = t1 Accfrt Y dt (2)
Feature extraction
The features shown in Table 2 can be classified as follows: for
After the preprocessing, gait features are extracted from the gait phase parameters, the percentage of each gait phase can
the gait cycles. As indicated in Table 2, the gait features be reflected as one of the features for gait. For spatial parameters,
include primarily spatial parameters and their characteristics, ankle mobility and mean stride length are also commonly used
temporal parameters and their characteristics, and plantar gait parameters. For plantar pressure, the ratio of pressure in
pressure parameters and their characteristics. each plantar region is a feature of interest. According to plantar
After the above rotation matrix lf R, the quadratic numbers anatomy, the plantar can be divided into three major regions:
can be transformed into Euler angles, which are finally translated forefoot, midfoot, and hindfoot, and the data are shown by
into plantarflexion and dorsiflexion, inversion and valgus, and the sensors located in the corresponding regions in the smart
internal and external rotation angles of the ankle joint. Besides, insole can be summed up to obtain the total pressure of the
this paper uses the quadratic integration of acceleration for the plantar regions and the regional pressure ratio can be derived. In
calculation of gait length during walking. The average velocity of addition, the trajectory of the plantar center of pressure has been

Frontiers in Neuroinformatics 05 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

FIGURE 3
The proportion of each gait phase of subjects at each stage. Different low case letters above columns indicate statistical differences at P < 0.05.

FIGURE 4
(A) Comparison of step length and (B) knee range of motion (ROM) in patients with different degrees of hemiplegia. Different low case letters
above columns indicate statistical differences at P < 0.05.

shown to reflect a variety of gait characteristics. The Equation for Ultimately, a total of six types of gait features were
calculating the center of plantar pressure is shown below: obtained from the data collected by the wearable device system
PT
and the active joint mobility measured clinically, which were
= 1 p(Xk ,Yk )∗Xk subsequently used as input features for the dataset to train
Xcen = kP
1 , (3)
m p(Xk ,Yk ) different regression models. Because of the patient’s walking
PT
ability, the length of collection time and number of collected
= 1 p(Xk ,Yk )∗Yk
Ycen = kP
1 , (4) steps varied. The number of steps collected by each person
m p(Xk ,Yk )
is between 50 and 150. The data uses one gait cycle as
where p(Xk , Y k ) denotes the pressure value of the pressure a sample, with a total of 2,352 samples. There were 760
sensor k of the smart insole, and (Xk , Y k ) denotes the samples in Healthy, 521 in B-Vn 726 in B-IV, and 345
coordinates of the pressure sensor k in the smart insole in B-III. The dataset’s dimension after feature extraction is
coordinate system. 2352 × 130.

Frontiers in Neuroinformatics 06 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

FIGURE 5
Correlation matrix of top 18 most important features.

Feature analysis and selection After removing some redundant features, the importance
of the features also needs to be ranked. Based on the feature
Since the number of features obtained from the current importance, more robust features can be selected, improving the
calculation is large, the number of input features needs to generalization ability of the model. In addition, by analyzing the
be further streamlined using data analysis techniques. In importance of features, it is possible to further understand the
this regard, the relevance and importance of features are importance of relevant features for lower limb evaluation and
further considered in the dimensionality reduction process. improve the interpretability of the algorithm, which is an area
Eliminating redundant features not only speeds up the model that has received little attention in other studies.
training process, but also avoids prediction errors due to In this paper, we utilize Random Forest (RF) for feature
multicollinearity as much as possible. In this paper, the Pearson importance analysis. It is easy to implement, and it has high
correlation matrix between features is constructed by calculating generalization ability, also it is easy to interpret. Random
the Pearson coefficients between each feature, such as the Forest is a machine learning algorithm whose decision process
correlation between feature a and feature b as shown in Equation integrates the classification predictions of multiple decision
(8). trees to produce a final result. Specifically, the algorithm first
performs random sample sampling in the dataset with put-
cov(Sa ,Sb ) backs, and then randomly selects M features as training inputs to
rab = σ(Sa ) × σ(Sb ) (5)
construct decision trees (DT). After the above steps are repeated
where Sa is the sample vector of feature a, σ (Sa ) is the standard K times, the consequent forest formed by K decision trees is
deviation of feature a, and cov(Sa , Sb ) is the sample vector obtained. The feature importance evaluation is to evaluate the
covariance of feature a and feature b. contribution of each feature to the classification performance of

Frontiers in Neuroinformatics 07 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

the constructed K decision trees, using the out-of-bag (OOB) TABLE 3 Definition of features in Figure 5.

error rate as an indicator. Specifically, for the kth tree, the Feature Description
number of classification accuracies acck is obtained for nk
sample numbers. At this time, the mth feature Fm is randomly A_CopLength COP trajectory length in affected side
scrambled and a new feature Fm2 is formed. After replacing the A_AreaComp_F Forefoot pressure to body weight ratio in affected side
old feature Fm with the new feature Fm2 , the kth tree is retrained A_CopLengthS Standard deviation of COP trajectory in affected side
and the number of classification accuracies acckm2 is obtained. UF_y_ACCave Average of Y axis acceleration of foot IMU in in unaffected side
The classification accuracy is also changed after the replacement UF_y_ACCvar Variance of Y axis acceleration of foot IMU in unaffected side
of features. By measuring the change of accuracy, the feature U_AnkleROM Range of motion of ankle joint in unaffected side
importance of the mth feature Fm for the kth tree IMPkm is: US_y_ACCvar Variance of Y axis acceleration of shank IMU in unaffected side
A_AreaComp_L Left plantar pressure to body weight ratio in affected side
acck −acckm2 A_KneeROM Range of motion of knee joint in affected side
IMPkm = nk (6)
U_KneeROM Range of motion of knee joint in unaffected side
In this, if there is no feature Fm in the kth
tree, then IMPkm A_HipROM Range of motion of hip joint in affected side
is defined as 0. For a RF of k trees, the importance of the feature U_CopLengthS Standard deviation of COP trajectory in unaffected side
Fm is: US_x_ACCvar Variance of X axis acceleration of shank IMU in unaffected side
UF_y_ACCrms Root mean square of Y axis acceleration of foot IMU in
PK
IMPkm unaffected side
IMPm = k=1 (7)
K·σ A_AreaComp_S Sum of plantar pressure to body weight ratio in affected side

where σ is the standard deviation of IMPkm of K tree. US_y_ACCrms Root mean square of Y axis acceleration of shank IMU in
unaffected side
A_AreaComp_ Hind plantar pressure to body weight ratio in affected side
FH
Model training and evaluation U_CopLength COP trajectory length in unaffected side

Our goal is to create a classification model that predicts BRS-

L grade from the selected features. In addition to the Random multi-category, and the performance measures of the model
Forest mentioned above, we examined various common generally have the following metrics.
effective classifiers, including Naïve Bayes (NB), Support Vector The detection rate P (Precision) is the proportion of
Machine (SVM), k-Nearest Neighbor (kNN). We use cross- correctly predicted cases to the total sample size. And for the
validation methods to evaluate the prediction performance of multiclassification problem, can be expressed by the Macro-
the classification models. In specific, we apply the leave-one- average method. The recall for a single category is the ratio
subject-out strategy to divide data into training set and test set. of positive cases correctly predicted to all positive cases. The
For each iteration, we choose the data of one subject as the test F1 value is then summed average of the accuracy rate and the
set, and the data of the remaining 29 subjects as the training set. completion rate, which can characterize the importance of the
The results of the test sets are aggregated and compared to the accuracy rate as well as the completion rate (Park et al., 2020).
entire data set to determine accuracy. And finally, we take the In addition, the results can be plotted using the subject
average prediction accuracy as the final result. operating characteristic curve (ROC), which is based on the
In the training process, according to Bayes’ theorem, the true case rate as well as the false positive case rate, where the
probability of occurrence of the corresponding classification more ideal the classification situation is, the fewer samples are
under different data features can be known, and the category expected to be incorrectly predicted, and the closer the ROC
with the highest probability is finally selected as the final curve should be to the upper left corner. The indicator AUC
result. SVM makes classification by finding the vector in the (Area Under Curve) is defined as the area under the ROC curve,
data feature space that maximizes the classification interval. and the closer to 1, the better the model performance.
In this process, the kernel function plays a crucial role in the
performance of the model and needs to be further tuned during
the training process. The kNN classifies the data by classifying Results
them under known different categories. When a data point to be
predicted appears, it is categorized according to the distance of Comparison of feature characteristics
that data point from other data points already classified, i.e., the
closer to which category it is classified. We analyzed the difference of gait phase time ratio, knee
It should be noted that the model utilized in this paper is motion and step length in different hemiplegia grades. The
a multi-category algorithm model, so the performance of the one-way Analysis of Variance (ANOVA) was used, and the
model needs to be evaluated using the evaluation metrics under homogeneity test of variance had a p-value of 0.05, which

Frontiers in Neuroinformatics 08 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

FIGURE 6
Feature importance ranking (top 18).

TABLE 4 Accuracy of classification results. U-SB among patients with different hemiplegia grades. In A-DB,
Model NB kNN SVM RF U-DB, and A-SB, a few groups had no significant differences,
but the group with no significant differences did not overlap
Accuracy 82.43 94.2 75.35 80.07 in the three features. There were significant differences in gait
F1 64.53 93.18 75.56 71.93 proportion among patients with different grades of hemiplegia.
The percentages of healthy individuals in A-DB, A-SB, U-DB,
and U-SB are near 12, 38, 12, and 38%. In addition to this, the
indicated that variance was not uniform. So, Tamhane’s T2 percentage of bilateral gait phases in healthy individuals with
was used for non-parametric test. The results were show the normal function of both lower limbs in one gait cycle is
in Figure 3, and the significant differences of characteristics also symmetrical. In contrast, patients with B-V reached 20%
between different groups were marked. Different low case letters in both dual support phases, i.e., the proportion of time spent
above columns indicate statistical differences at P < 0.05. in the dual support phase was increased and patients needed
The results for the gait phase parameters are shown in to stay on both feet for a longer period. Patients in B-IV
Figure 3 for the proportion of gait phases of four subjects, stayed in the double support phase for a longer time, while the
namely, the affected side double support phase (A-DB), the swing process was shorter on the affected side. For the B-III
affected side single support phase (A-SB), the healthy side patients which have more severe conditions, the double support
double support phase (U-DB), and the healthy side single phase was longer in proportion, and symmetry became worse.
support phase (U-SB). There were significant differences in We can see that the more serious the degree of hemiplegia,

Frontiers in Neuroinformatics 09 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

FIGURE 7
Accuracy of different classification models with the different number of features.

FIGURE 8
Confusion matrix of the kNN model. FIGURE 9
ROC of different classification models.

the lower the proportion of swing time on the healthy side, Figure 4 shows that patients with different grades of
because the hemiplegia weakens the supporting ability of the hemiplegia showed significant differences in step length and
hemiplegia side. knee motion range. Generally, the knee motion range and step

Frontiers in Neuroinformatics 10 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

TABLE 5 Statistical analysis of ankle range of motion and the healthy side in the presence of nerve damage and muscle
trajectory length of CoP.
strength deficit.
Indicators B-III-A B-III-U B-V-A B-V-U

Mean (◦ ) 20.4 28.1 61.4 72.9

Standard deviation 4.1 7.1 5.2 5.7
Characterization and model analysis
Mean (mm) 65.1 68.1 79.2 81.3
Standard deviation 11.1 23.1 11.2 16.3
The correlation matrix and the degree of redundancy
of the data were obtained by correlation analysis of the
gait parameters and their characteristics. According to the
correlation, features with an absolute value of correlation
length both reduced as the severity of hemiplegia increased. greater than 0.85 should be excluded. The process of feature
From B-III to B-IV, although the range of motion of the rejection also requires the selection of features according to their
healthy side of the knee joint was reduced, but the affected importance. The correlation matrix of some features is shown
side was increased, and the step length was also increased, in Figure 5. Table 3 and Figure 6 demonstrate the top 18 most
indicating that the affected side function was initially restored. important features. The Supplementary material provides the
From B-IV to B-V, the knee motion and step length of full list of features.
the affected and healthy sides were significantly improved. In this paper, the most important features under different
Hospitals also consider B-V to be discharge level. Comparing numbers are selected for model training based on the
with healthy subjects, the step length of B-V patients was importance of the features obtained from the RF algorithm,
close to the normal level, and there was also a large and the results obtained are shown in Figure 7. It can
range of motion of the knee joint, but the variance of the be seen that as the number of features increases, the
affected side of the knee joint was large, indicating that the classification accuracy of different classification models shows
joint stability of the affected side was still poor. According a trend of first increasing and then decreasing. Among
to previous studies on hemiplegic gait, the results of the them, the kNN algorithm shows a more stable classification
present study are consistent with the reality that hemiplegic accuracy for different numbers of features. When the number
patients have slowed gait speed and compensated for the of features is 18, the classification accuracy of kNN is

FIGURE 10
Plantar CoP trajectory in B-III vs. B-V patients. (A) Comparison of CoP trajectory on the affected side. (B) Comparison of CoP trajectory on the
unaffected side.

Frontiers in Neuroinformatics 11 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

the highest, reaching 94.2%. If we use all features as the forefoot to the ground. The standard deviation of the
input, i.e., there is no feature extraction, the classification trajectory length of the center of plantar pressure in both
performance of the models will be very poor (accuracy feet indicates that the greater the trajectory dispersion, the
lower than 65%). weaker the patient’s support of the affected side and the less
Since the highest classification accuracy can be obtained by stable the walking. As shown in Table 5, the dispersion on
using the first 18 significant features, the performance of the the affected side was greater in B-III patients than in B-V
model under this condition is further analyzed in this paper. patients. Also, the trajectory length of the center of pressure was
The confusion matrix (Figure 8) shows that the healthy shorter in B-III patients, which corroborates with the results
individuals are more accurately classified with B-IV. The in Figure 10, indicating that B-III patients do not have a
major recognition errors mostly occur in patients with B-III normal ankle motion process and no forefoot stirrups after
incorrectly predicting patients with bit B-IV. heel contact with the ground. For the other important features,
Comparing the various algorithmic models mentioned also illustrate well their importance for the evaluation, but
above (Table 4), it can be seen that the kNN algorithm has too many features can lead to a decrease in the classification
a higher recognition accuracy of 94.2%. The classification performance of the model.
accuracy of the other algorithmic models is the lowest at only Importantly, this paper found that subtle data variation on
69.72%. As can be seen from the Figure 9, the ROC curve of the robust side was more important for evaluation, possibly
the kNN algorithm is in the upper left corner, and the AUC because the abnormalities on the affected side were too extreme
value at this point is 0.98. Therefore, the most effective and and varied across individuals, which was detrimental to the
convincing results were achieved by using the kNN model. The generalization of the model. In contrast, the stability of the
results shows that machine learning-based degree evaluation affected side was better captured by the data variation on
using wearable sensors is feasible and accurate. the healthy side, and therefore the important features chosen
were reasonable.
Since our method is based on gait characteristics, the next
step can be to design online correction tools to help patients
Discussion perform gait correction during the evaluation process (Yang
et al., 2018), thus helping patients to consciously perform
This paper proposes an evaluation method based on feature rehabilitation training, reshape nerves, activate muscles and
selection and machine learning for automatically assessing BRS- promote recovery. In addition, by adding visual feedback for
L grade. We extracted a large number of features from the lower stroke patients, patients can visualize their lower limb function
limb motion and plantar pressure data collected by wearable and improve the quality of rehabilitation.
sensors. Then we build several machine learning models to
classify BRS-L grade using selected features. The kNN achieved
the highest prediction accuracy of 94.2%. Conclusion
We also discovered several interesting findings. One is that
the BRS-L is highly correlated with 18 features (Table 3), this In contrast to previous studies, this paper provides a
indicates that more clinical attention should be paid to these wearable sensors-based, reliable and interpretable method for
features of the patient. evaluating the BRS-L, providing physicians with contextual
The key features found in this paper are also of clinical information for evaluation. Using the dual-source information
relevance. It is worth noting that the standard deviation of the provided by the wearable device as well as feature analysis,
range of motion of the ankle on the healthy side is greater, as this method is a convenient, accurate, and reliable objective
shown in Table 5, the standard deviation of this index reached quantitative evaluation method. The accuracy of the feature
7.1◦ in B-III patients and 5.7◦ in B-V patients, and the greater selection based method in this paper was up to 94.2%.
standard deviation also indicates that the range of motion of the Our method doesn’t require complex setup and thus can
ankle on the healthy side contains more information. In fact, provide a home-based evaluation, which greatly reduces the
the B-III grade complained of weakness on the affected side for burden on the healthcare system as well as the patient,
nearly 50 days and a circle gait during walking. Therefore, when such as by eliminating the need for frequent trips between
the muscle strength of the affected side is lacking, the lack of home and hospital. Besides, our method greatly improves
strong support of the body will lead to interference of the ankle the relevance and real-time nature of rehabilitation treatment,
motion process on the healthy side, creating a more discrete allowing patients to receive more effective treatment during
ankle range of motion. the prime time of hemiplegia rehabilitation. Because of its
Similarly, for the first three important characteristics, simplicity and ease of use, patients can regularly evaluate
forefoot pressure values indicate that the patient has weak their stage of recovery, our method is suitable to the home-
forefoot stirrups on the affected side, or even does not bring based rehabilitation.

Frontiers in Neuroinformatics 12 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

Data availability statement the Less Developed Regions of National Natural Science
Foundation of China (Grant nos. 81760408 and 8156036),
The original contributions presented in this study are the Jiangxi Provincial Science and Technology Department
included in the article/Supplementary material, further Project of China (Grant no. 20171BBG70016), the Natural
inquiries can be directed to the corresponding author/s. Science Foundation of Jiangxi Province of China (Grant no.
20113BCB22005), and the National Natural Science Foundation
of China (Grant no. 52075177).
Ethics statement
The studies involving human participants were reviewed Conflict of interest
and approved by the Ethics Committee of the Second Hospital of
NanChang University. The patients/participants provided their The authors declare that the research was conducted in the
written informed consent to participate in this study. absence of any commercial or financial relationships that could
be construed as a potential conflict of interest.

Author contributions
XC and DH designed the research and participated in the
Publisher’s note
entire research including data collection, data processing, model
All claims expressed in this article are solely those of the
construction, result interpretation, manuscript drafting, and
authors and do not necessarily represent those of their affiliated
revisions. YZ and JL designed the research and participated
organizations, or those of the publisher, the editors and the
in the data collection and revisions of the manuscripts. LX
reviewers. Any product that may be evaluated in this article, or
designed the research, manuscript drafting, and revisions. RZ
claim that may be made by its manufacturer, is not guaranteed
participated in the data collection. ZP and YC participated in
or endorsed by the publisher.
the analysis of the results. All authors contributed to the article
and approved the submitted version.

Supplementary material
Funding
The Supplementary Material for this article can be
This study was supported by the National Key Research and found online at: https://www.frontiersin.org/articles/10.3389/
Development Project of China (Grant no. 2020YFC2005800), fninf.2022.1006494/full#supplementary-material

References
Alonso-Vázquez, A., Villarroya, M. A., Franco, M. A., Asín, J., and Calvo, elderly, and stroke patients. PLoS One 10:e124414. doi: 10.1371/journal.pone.01
B. (2009). Kinematic evaluation of paediatric forefoot varus. Gait Posture 29, 24414
214–219. doi: 10.1016/j.gaitpost.2008.08.009
Domínguez-Morales, M. J., Luna-Perejón, F., Miró-Amarante, L., Hernández-
Berengueres, J., Fritschi, M., and McClanahan, R. (2014). A smart pressure- Velázquez, M., and Sevillano-Ramos, J. L. (2019). Smart footwear insole for
sensitive insole that reminds you to walk correctly: An orthotic-less treatment recognition of foot pronation and supination using neural networks. Appl. Sci.
for over pronation. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2014, 2488–2491. 9:3970. doi: 10.3390/app9193970
doi: 10.1109/EMBC.2014.6944127
Hosseini, Z., Peyrovi, H., and Gohari, M. (2019). The effect of early passive
Boukhennoufa, I., Zhai, X., Utti, V., Jackson, J., and McDonald-Maier, range of motion exercise on motor function of people with stroke: A randomized
K. D. (2021). “A comprehensive evaluation of state-of-the-art time-series deep controlled trial. J. Caring Sci. 8, 39–44. doi: 10.15171/jcs.2019.006
learning models for activity-recognition in post-stroke rehabilitation evaluation,”
in Proceedings of the annual international conference of the IEEE engineering Hsu, W., Sugiarto, T., Lin, Y., Yang, F., Lin, Z., Sun, C., et al. (2018). Multiple-
in medicine and biology society. IEEE engineering in medicine and biology wearable-sensor-based gait classification and analysis in patients with neurological
society. annual international conference, Vol. 2021, (Mexico), 2242. doi: 10.1109/ disorders. Sensors Basel. 18:3397. doi: 10.3390/s18103397
EMBC46164.2021.9630462 Huang, C., Lin, G., Huang, Y., Song, C., Lee, Y., How, M., et al.
Boukhennoufa, I., Zhai, X., Utti, V., Jackson, J., and McDonald-Maier, K. D. (2016). Improving the utility of the Brunnstrom recovery stages in
(2022). Wearable sensors and machine learning in post-stroke rehabilitation patients with stroke. Medicine 95:e4508. doi: 10.1097/MD.000000000000
evaluation: A systematic review. Biomed. Signal Proces. 71:103197. doi: 10.1016/ 4508
j.bspc.2021.103197
Kim, J., Thayabaranathan, T., Donnan, G. A., Howard, G., Howard, V. J.,
Brunnstrom, S. (1966). Motor testing procedures in hemiplegia: Based on Rothwell, P. M., et al. (2020). Global Stroke Statistics 2019. Int. J. Stroke 15,
sequential recovery stages. Phys. Ther. 46, 357–375. doi: 10.1093/ptj/46.4.357 819–838. doi: 10.1177/1747493020909545
Capela, N. A., Lemaire, E. D., and Baddour, N. (2015). Feature selection Li, Y., Wei, Q., Gou, W., and He, C. (2018). Effects of mirror therapy on walking
for wearable Smartphone-Based human activity recognition with able bodied, ability, balance and lower limb motor recovery after stroke: A systematic review

Frontiers in Neuroinformatics 13 frontiersin.org

Chen et al. 10.3389/fninf.2022.1006494

and meta-analysis of randomized controlled trials. Clin. Rehabil. 32, 1007–1021. Tran, T., Chang, L., Almubark, I., Bochniewicz, E. M., Shu, L., Lum, P. S., et al.
doi: 10.1177/0269215518766642 (2018). Robust classification of functional and nonfunctional arm movement after
stroke using a single Wrist-Worn sensor device. Paper presented at the 2018 IEEE
Mannini, A., Trojaniello, D., Cereatti, A., and Sabatini, A. (2016). A machine
international conference on big data, Seattle, WA, 5457–5459.
learning framework for gait classification using inertial sensors: Application to
elderly, Post-Stroke and huntington’s disease patients. Sensors Basel 16:134. doi: Wang, F., Chen, S., Lin, C., Shih, C., Lin, A., Yuan, W., et al. (2021). Detection
10.3390/s16010134 and classification of stroke gaits by deep neural networks employing inertial
measurement units. Sensors Basel 21:1864. doi: 10.3390/s21051864
Naghdi, S., Ansari, N. N., Mansouri, K., and Hasson, S. (2010). A
neurophysiological and clinical study of Brunnstrom recovery stages in the upper World Health Organization (2020). The top 10 causes of death. Available
limb following stroke. Brain Injury 24, 1372–1378. doi: 10.3109/02699052.2010. online at: https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-
506860 of-death (accessed May 1, 2022)
Nepveu, J., Thiel, A., Tang, A., Fung, J., Lundbye-Jensen, J., Boyd, L. A., et al. Xian, X., Zhou, Z., Huang, G., Nong, J., Liu, B., and Xie, L. (2021). Optimal
(2017). A single bout of High-Intensity interval training improves motor skill sensor placement for estimation of center of plantar pressure based on the
retention in individuals with stroke. Neurorehab. Neural Re. 31, 726–735. doi: improved genetic algorithms. IEEE Sens. J. 21, 28077–28086. doi: 10.1109/JSEN.
10.1177/1545968317718269 2021.3125021
Park, E., Lee, K., Han, T., and Nam, H. S. (2020). Automatic grading of stroke Yang, G., Deng, J., Pang, G., Zhang, H., Li, J., Deng, B., et al. (2018). An
symptoms for rapid evaluation using optimized machine learning and 4-Limb IoT-Enabled stroke rehabilitation system based on smart wearable armband and
kinematics: Clinical validation study. J. Med. Internet Res. 22:e20641. doi: 10.2196/ machine learning. IEEE J. Trans. Eng. Health Med. 6, 1–10. doi: 10.1109/JTEHM.
20641 2018.2822681
Qiu, S., Wang, Z., Zhao, H., Liu, L., and Jiang, Y. (2018). Using Body-Worn Zhang, Q., Jin, T., Cai, J., Xu, L., He, T., Wang, T., et al. (2022). Wearable
sensors for preliminary rehabilitation evaluation in stroke victims with gait triboelectric sensors enabled gait analysis and waist motion capture for IoT-based
impairment. IEEE Access 6, 31249–31258. doi: 10.1109/ACCESS.2018.2816816 smart healthcare applications. Adv. Sci. 9:2103694. doi: 10.1002/advs.202103694
Stuart, T., Hanna, J., and Gutruf, P. (2022). Wearable devices for continuous Zhao, W., Reinthal, M. A., Espy, D. D., and Luo, X. (2017). Rule-Based human
monitoring of biosignals: Challenges and opportunities. APL Bioeng. 6:21502. motion tracking for rehabilitation exercises: Realtime evaluation, feedback, and
doi: 10.1063/5.0086935 guidance. IEEE Access 5, 21382–21394. doi: 10.1109/ACCESS.2017.2759801

Frontiers in Neuroinformatics 14 frontiersin.org