Hindawi

Journal of Healthcare Engineering

Volume 2020, Article ID 8887340, 15 pages
https://doi.org/10.1155/2020/8887340

Research Article
GBDT-Based Fall Detection with Comprehensive Data from
Posture Sensor and Human Skeleton Extraction

Wen-Yu Cai ,1,2 Jia-Hao Guo,1 Mei-Yan Zhang ,3 Zhi-Xiang Ruan,1 Xue-Chen Zheng,1
and Shuai-Shuai Lv 1
1
College of Electronics and Information, Hangzhou Dianzi University, Hangzhou 310018, China
2
Zhejiang Provincial Key Lab of Equipment Electronics, Hangzhou Dianzi University, Hangzhou 310018, China
3
School of Electrical Engineering, Zhejiang University of Water Resources and Electric Power, Hangzhou 310018, China

Correspondence should be addressed to Mei-Yan Zhang; [email protected]

Received 11 March 2020; Revised 26 May 2020; Accepted 2 June 2020; Published 25 June 2020

Academic Editor: Fabrizio Taffoni

Copyright © 2020 Wen-Yu Cai et al. This is an open access article distributed under the Creative Commons Attribution License,
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Since fall is happening with increasing frequency, it has been a major public health problem in an aging society. There are
considerable demands to distinguish fall down events of seniors with the characteristics of accurate detection and real-time alarm.
However, some daily activities are erroneously signaled as falls and there are too many false alarms in actual application. In order
to resolve this problem, this paper designs and implements a comprehensive fall detection framework on the basis of inertial
posture sensors and surveillance cameras. In the proposed system framework, data sources representing behavior characteristics
to indicate potential fall are derived from wearable triaxial accelerometers and monitoring videos of surveillance cameras.
Moreover, the NB-IoT based communication mode is adopted to transmit wearable sensory data to the Internet for subsequent
analysis. Furthermore, a Gradient Boosting Decision Tree (GBDT) classifier-based fall detection algorithm (GBDT-FD in short)
with comprehensive data fusion of posture sensor and human video skeleton is proposed to improve detection accuracy.
Experimental results verify the good performance of the proposed GBDT-FD algorithm compared to six kinds of existing fall
detection algorithms, including SVM-based fall detection, NN-based fall detection, etc. Finally, we implement the proposed
integrated systems including wearable posture sensors and monitoring software on the Cloud Server.

1. Introduction systems on activity monitoring of humans based on wearable

sensors and issues to be addressed to tackle the challenges. As
An increased aging population in the world is forcing rapid far as we know, there are three main categories of fall detection
rises in healthcare requirements [1]. Everyone knows that older technologies: fall detection using wearable sensors [4, 5], fall
people have poor balance ability and slow response ability. Falls detection using environmental sensors [6, 7], and video-based
are a major cause of injury for the elderly and a huge obstacle in fall detection [8, 9]. Although there are some other methods
the independent living of the seniors. Once the elderly falls such as radar-based fall detection [10], they are more com-
down alone at home without help, the injured elderly may be plicated compared to the above three methods.
lying on the ground for several hours or even days. More Wearable sensor based fall detection methods mainly
seriously, it is very likely to extended injury and be life- depend on sensory data gathered from wearable acceler-
threatening if he did not get treatment timely. Therefore, timely ometer and gyroscope. It is generally agreed that the use of
fall incident detection and medical assistance for the elderly are wearable sensors has played a quite important role in
intuitively important. However, due to different application monitoring the physiological parameters of a person to
scenarios and various body activities, satisfactory and reliable minimize any malfunctioning happening in the body. In
fall detection results are too hard to guarantee [2]. recent years, the advancement of sensing technologies,
Some related fall detection algorithms have been proposed embedded systems, wireless communication technologies,
in the literature. Broadley et al. [3] review the latest reported nanotechnologies, and miniaturization makes it possible to
2 Journal of Healthcare Engineering

develop smart wearable sensors to monitor activities of Kinect camera and a device consisting of an accelerometer
human beings continuously. Nag et al. [11] provide a review and a gyroscope, and then a fuzzy inference system is used to
on some of the significant research work done on wearable separate fall from daily activities. Hondori et al. [20] present
flexible sensors. Chen et al. [12] propose a novel intelligent a detection system that helps monitor various dining ac-
fall detection method, named as ESAEs-OCCCH, which uses tivities of poststroke patients using a Kinect camera and an
acceleration data from a wrist-worn smart watch. ESAEs- accelerometer. Nizam et al. [21] propose a novel approach
OCCCH is first adopted for unsupervised feature extraction that uses a depth sensor and employs a unique procedure
to overcome the disadvantages of artificial feature extraction. that identifies the fall risk levels to adapt the algorithm for
Yacchirema et al. [13] propose an innovative IoT (Internet of different people with their physical strength to withstand
Thing) based online system for detecting falls of the aged. falls. Bogdan et al. [22] present a low-cost system for reliable
Sensory readings are processed and analyzed using a deci- fall detection with a very low false alarm ratio on the basis of
sion tree based Big Data model running on a Smart IoT accelerometric data and depth maps. The single drawback of
Gateway [14]. Although these wearable sensors have high the above methods is that Kinect camera is not cheap since a
sensitivity and good real-time characteristics, higher de- considerable computational power is needed to execute
tection accuracy cannot typically be achieved due to the image processing algorithms. He et al. [23] propose a
interference from diverse activities of hand or wrist. Hence, method to integrate the information of video images, sound,
it is easy to cause misjudgment and missed detection of fall infrared, pulse, and other information into the elderly care
actions relying only on wearable sensory data. system. However, it is not very realistic to detect fall acci-
Secondly, a few scholars apply environmental sensors to dents with so many sensors.
detect falls. Li et al. [15] propose a phase transform (SRP- To overcome these shortcomings, we use an ordinary
PHAT) method which can locate the original source of a camera and accelerometer as a data source in this paper so as
certain voice. In terms of sound classification phase, they to improve the practicability of the detection system. Fur-
apply the Mel-Frequency Cepstral Coefficients (MFCC) thermore, a novel data fusion based fall detection online
features with a Nearest Neighbor (NN) approach to improve system and one GBDT based detection algorithm is provided
fall detection performance. However, expensive acoustic in detail. Data fusion of human activity features obtained by
devices have high requirements on the environment, and it is posture sensor and surveillance cameras plays a significant
not feasible to promote accurate detection with certain role in the recognition of abnormal activities. In addition,
ambient noise. Adnan et al. [16] adopt acoustic Local Ter- the proposed platform uses NB-IoT communication and
nary Patterns (acoustic-LTPs) to detect fall events by ana- Ethernet to transmit manifold data to the Cloud Server for
lyzing environmental noise. Acoustic features are extracted further analysis. The platform is able to effectively monitor
from the separated source components using the proposed the daily life of the elderly. When an unexpected fall incident
acoustic-LTPs scheme. Subsequently, fall events would be occurs, the proposed system will send an alarm signal to
identified with SVM based classifier. However, it will cause inform the family relatives or other related guardians. In
noise during the audio signal acquisition, which could lead conclusion, the proposed system would meet the require-
to low accuracy and frequent false alarms. ments of high sensitivity and precision. As a result, necessary
Thirdly, vision-based fall detection typically uses image assistance could be provided in times with high coverage
processing techniques to construct a human body model to communication technology, so it is suitable for application
detect fall. In general, video-based fall detection systems in the elderly care system.
have shown some potential and reliability in detecting falls The rest of the paper is organized as follows. Section 2
in public places. Due to the popularity of video surveillance, describes the whole online fall detection platform framework
vision-based fall detection methods have already become and comprehensive data source. In Section 3, GBDT based
one research hotspot. The boundary extraction method is fall detection algorithm using comprehensive data from an
used to obtain the aspect ratio of the human body and then accelerometer based posture sensor and human skeleton
to judge falls. Sase and Bhandari [17] used contour-based extraction is presented in detail. Comparable experimental
template matching to distinguish human and nonhuman results and actual operating interface are described in
and then judged human fall according to the distance be- Section 4. Finally, a conclusion is drawn in Section 5.
tween the top and center of the external rectangle of the
human body. Shen et al. [18] propose a fall detection method 2. System Framework and Comprehensive
using the Deeper Cut model to exact human key points, and Data Source
it is implemented using Raspberry Pi platform. Vision-based
fall detection can use relatively cheap cameras to quantify Our complete framework of the fall detection system for
and judge various activities; nevertheless, it requires com- seniors is illustrated in Figure 1. Each user is equipped with a
plex handling methods to construct a human body model kind of self-made MEMS (Micro Electro Mechanical Sys-
and it is unsuitable for real-time detection mode. tems) based wearable sensor with hardware block diagram in
In addition, some results have suggested that a single Figure 2, which uses triaxial acceleration and angular ve-
detection model from individual DataSet could easily lead to locity sensor to capture the body posture. Besides, NB-IoT
false detection. Recently, there are already a few methods on (Narrow Band Internet of Things) [24] communication
the basis of wearable sensors and surveillance cameras to mode is used to transmit sensory data to the Cloud Server.
classify body activities. Kepski and Kwolek [19] apply a With the development of Internet of Things technology, the
Journal of Healthcare Engineering 3

User_1
Camera
4G LTE
Guardian
_1

User_2
Internet
Cloud server Guardian
_M

User_3 Camera

Camera
MEMS sensor User_N

Figure 1: Framework of the proposed platform.

one ensemble learning method named Gradient Boost

Alarm button
Decision Tree (GBDT) [27] is applied for self-learning with
MDS, so as to classify fall and other normal activities in a
3-axis Low-power NB-IoT robust way.
MEMS MCU communication Since human activity frequency generally does not ex-
sensor processor module ceed 20 Hz [28], the acceleration acquisition frequency is set
to 30 Hz so as to process data more accurately, and video
acquisition frequency is adjusted to 30 fps (frames per
2 AAA battery second) after software processing. The entire self-made
DataSet comprises six kinds of human activities involving
Figure 2: The block diagram of self-made posture sensor. fall, walk, sit, squat, lie down, and jump.
Due to the diversity and complexity of fall accidents, it
is hard to identify the way and direction of fall events. To
health care field is also affected deeply. As we know, NB-IoT
overcome this shortcoming, we use the sum vector of
is an emerging technology with many good features such as
triaxial acceleration value to measure the human activity.
wide coverage, multiple connections, low speed, low power
Let ax , ay , az denote acceleration value in three dimen-
consumption, etc. In our IoT based health monitoring
sions, respectively; A3-axis represent the actual value of
system, various detection devices are connected together for
triaxial acceleration, which can be calculated with the
data exchange, so as to deliver warnings to medical staff or
following equation:
guardians in time when the elderly fall. Moreover, it is also 􏽱����������
supposed that each user is covered by at least one surveil- A3−axis � a2x + a2y + a2z . (1)
lance camera and so we can monitor and record each user’s
activity. Today, almost all surveillance cameras have the The measured sensory data are illustrated in Figure 4 to
ability to transmit video sequences to the Internet through compare the acceleration variation curve of falls and that of
Ethernet or wireless networks. Therefore, the Cloud Server other normal activities, including squat, lie down, jump,
could obtain both attitude data and video data and store it at walk, and sit down.
the local database for further analysis. The fall detection The other fall characteristics are key skeleton coordinates
algorithm is running on the Cloud Server with high per- of the human body as shown in Figure 5. The rectangular
formance. Once fall events are detected, the Cloud Server coordinate system is established in Figure 5, and the hori-
will send an alarm signal to the specific guardian through 4G zontal and vertical coordinates of N � 18 key points would be
LTE (Long Term Evolution) communication technology. As obtained, respectively. (Xsk−i , Ysk−i ) (i � 1, 2, . . ., N) denotes
a result, each user can get instant help and timely treatment the respective coordinates of each key point. Hence, the
in case of any abnormality with our proposed framework. collection of key skeleton coordinates is as follows:
The principle of the fall detection process is demon-
strated in Figure 3. In this system framework, real-time Xsk , Ysk 􏼁 � 􏼈 Xsk−1 , Ysk−1 􏼁, Xsk−2 , Ysk−2 􏼁, . . . , Xsk−18 , Ysk−18 􏼁􏼉.
acceleration data from posture sensors are transmitted by (2)
NB-IoT communication mode to the Cloud Server, and then
form a data collection named as Acceleration DataSet However, the total 18 key points extracted using
(ADS). Surveillance cameras are used to collect human OpenPose software are not all effective and necessary for
activities’ video, and key point coordinates of human body identifying fall events. Redundant data will only increase
are obtained through OpenPose software [25] processing to computation complexity and even introduce unnecessary
form Video DataSet (VDS). Video DataSet and Acceleration noises. Therefore, the maximum and minimum x-coordi-
DataSet together make up the so-called Merged DataSet nate and y-coordinate of each key skeleton point Xsk−max ,
(MDS). After processing by sliding window strategy [26], Xsk−min , Ysk−max and Ysk−min are picked out, respectively, and
4 Journal of Healthcare Engineering

NB- Acceleration
IoT x
DataSet Input features

Posture sensor Tree 1 Tree 2

NB-IoT Sliding window
Window 1 Window 3 Tree splits

Merged
Transformed Output
DataSet
features
Window 2
w0 w1 w w w
Window 4 2 3 4

∑
Time Linear classifier
Window size Window slide

Video
DataSet Data processing layer

Web camera Ensemble learning layer

OpenPose

Data acquisition layer

Figure 3: The principle of GBDT-FD algorithm.

Fall down Squat Lie down

25 25 25

20 20 20
A3-axis (g)

A3-axis (g)

A3-axis (g)
15 15 15

10 10 10

5 5 5

0 0 0
1 2 3 4 1 2 3 4 1 2 3 4
t (s) t (s) t (s)
(a) (b) (c)
Jump Walk Sit down
25 25 25

20 20 20
A3-axis (g)

A3-axis (g)

A3-axis (g)

15 15 15

10 10 10

5 5 5

0 0 0
1 2 3 4 1 2 3 4 1 2 3 4
t (s) t (s) t (s)
(d) (e) (f )

Figure 4: A3-axis variation with different human activities.

then a body circumscribed rectangle is introduced to rep- Xsk−max − Xsk−min

resent the contour of the human body. The behavior change R� . (3)
Ysk−max − Ysk−min
of the human body can be identified only by paying attention
to the changes of length and width of body rectangle, so as to Due to complex surroundings in different monitoring
reduce calculation complexity significantly compared to that scenarios, sometimes there is no guarantee that OpenPose
using all the key skeleton points. software can obtain 18 key points completely. In this case,
Figure 6 illustrates three different scenarios of standing, the median of the previous occurrence of this point and the
losing balance, and falling down completely. The colored next occurrence of this point will be used instead. Figure 7
rectangle denotes the outline of the human body, and it is compares the aspect ratio R between falls and other normal
very clear that the length and width of the body rectangle activities.
have changed a lot. We regard the aspect ratio R between Since the fall process is a continuous dynamic event in
body width and length as a feature from Video DataSet, the time dimension, it could not be identified by acceleration
which is calculated as follows: data or video data in a moment. In order to search and detect
Journal of Healthcare Engineering 5

Ysk

14 15
16 17
0

2 5
1

3 6

4 8 11 7

9 12

10 13
Xsk

Figure 5: Skeleton scattergram of human body.

(a) (b) (c)

Figure 6: Three different scenarios during a fall process. (a) Standing. (b) Losing balance. (c) Falling down.

Fall down Squat Lie down

6 6 6

4 4 4
R R R
2 2 2

0 0 0
1 2 3 4 1 2 3 4 1 2 3 4
t (s) t (s) t (s)
(a) (b) (c)
Figure 7: Continued.
6 Journal of Healthcare Engineering

Jump Walk Sit down

6 6 6

4 4 4
R R R
2 2 2

0 0 0
1 2 3 4 1 2 3 4 1 2 3 4
t (s) t (s) t (s)
(d) (e) (f )

Figure 7: Aspect ratio R variation with different human activities.

the complete process of the falling event, we use a sliding change rapidly. In conclusion, we should only pay attention
window method to intercept the derived data flow as Fig- to the overall size and change rate of A3-axis and R within
ure 8, where A and R denote the sum vector of triaxial each sliding window. For both Acceleration DataSet and
acceleration value and the aspect ratio of the human body, Video DataSet, 7 characteristic values including mean,
respectively. Following the study in Ref. [29], the complete standard deviation (std), maximum value (max), minimum
fall process ranges from 0.3 s to 0.4 s, so we set width value value (min), average change rate (d), number of mean
and step value to 0.5 s and 0.1 s, respectively. In our GBDT- crossings (MCV), and root mean square (RMS) of each
FD algorithm, the data acquisition frequency is 30 Hz, so sliding window are calculated, respectively. MCV means the
there are W � 15 pieces of data in a 0.5 s sliding window, and number of samples above the mean value in a set of data, so
the step value between each two data sequences is S � 3 too large or too small MCV means this dataset change too
pieces of data. dramatically. As a result, the above 14-dimensional statis-
After obtaining A3-axis and R, it is necessary to extract the tical characteristics are listed in Table 1.
statistical characteristic so as to carry out the classification We extracted 14-dimensional statistical characteristics
process. When fall incident occurs, the acceleration and from each sliding window to constitute the Merged DataSet,
posture of the human body will exceed normal range and which is stored as in the following matrix:

Amean1 , Astd1 , Amax 1 , Amin 1 , Ad1 , AMCV1 , ARMS1 , Rmean1 , Rstd1 , Rmax 1 , Rmin 1 , Rd1 , RMCV1 , RRMS1
⎢
⎡
⎢
⎢ ⎤⎥⎥⎥
⎢
⎢ ⎥
⎢ Amean2 , Astd2 , Amax 2 , Amin 2 , Ad2 , AMCV2 , ARMS2 , Rmean2 , Rstd2 , Rmax 2 , Rmin 2 , Rd2 , RMCV2 , RRMS2 ⎥⎥⎥⎥
⎢
⎢
⎢
⎢ ⎥⎥
⎢
⎢
⎢
⎢
⎢ Amean3 , Astd3 , Amax 3 , Amin 3 , Ad3 , AMCV3 , ARMS3 , Rmean3 , Rstd3 , Rmax 3 , Rmin 3 , Rd3 , RMCV3 , RRMS3 ⎥⎥⎥⎥⎥, (4)
⎢
⎢
⎢
⎢ ⎥⎥⎥⎥
⎢
⎢
⎢ ... ⎥⎥⎥
⎣ ⎦
Ameani , Astdi , Amax i , Amin i , Adi , AMCVi , ARMSi , Rmeani , Rstdi , Rmax i , Rmin i , Rdi , RMCVi , RRMSi

where subscript i denotes that this characteristic sequence is enables it to reduce the interference of noisy data effectively.
extracted from the i-th sliding window. Moreover, normalization is not required in the data pre-
processing stage. Gradient boosting is a machine learning
technique for regression problems, which produces a pre-
3. GBDT-Based Fall Detection Algorithm diction model in the form of an ensemble of weak prediction
3.1. Ensemble Learning-Based Fall Detection. Ensemble models. GBDT (Gradient Boosting Decision Tree) is a
learning [30] is a machine learning method that combines boosting method using Classification and Regression Tree
multiple weak learners into a strong one. Several individual (CART) [31] as its base learner which is illustrated in Fig-
learners with complementary results are trained concur- ure 9. GBDT uses decision tree as the weak prediction model
rently, and the results from each individual learner are in gradient boosting so it has high accuracy.
merged into the final prediction result by a certain com- The described residual is the difference between the
bination strategy, so as to achieve the effect of extensive correct result and the actual result. In GBDT-FD algorithm,
learning and effective classification. If each individual we use the logarithmic loss function to define residuals as
learner is the same type in ensemble learning, these indi- follows:
vidual learners are called base learners. The advantage of
ensemble learning lies in its strong robustness, which L(y, f(X)) � log(1 + exp(−yf(X))). (5)
Journal of Healthcare Engineering 7

A1, A2, A3, A4, A5, A6, A7, ... ... , A15, A16, A17, A18, A19, A20, A21, ... ...
R1, R2, R3, R4, R5, R6, R7, ... ... , R15, R16, R17, R18, R19, R20, R21, ... ...

A1, A2, A3, A4, A5, A6, A7, ... ... , A15
R1, R2, R3, R4, R5, R6, R7, ... ... , R15

A4, A5, A6, A7, ... ... , A15, A16, A17, A18
R4, R5, R6, R7, ... ... , R15, R16, R17, R18
S=3
A7, ... ... , A15, A16, A17, A18, A19, A20, A21
R7, ... ... , R15, R16, R17, R18, R19, R20, R21 ……

W = 15

0 0.1 0.2 0.7

Time (s)
Figure 8: Sliding window.

Table 1: Statistical characteristics.

X � Amean , Astd , Amax , Amin , Ad , AN , ARMS , Rmean , Rstd ,
Acceleration data Skeleton data
Rmax , Rmin , Rd , RN , RRMS 􏼁.
Features Symbol Features Symbol
Mean value Amean Mean value Rmean (7)
Standard deviation Standard deviation The optimal parameters of the GBDT model-based fall
Astd Rstd
value value
detection algorithm are determined by the evaluation
Maximum value Amax Maximum value Rmax
Minimum value Amin Minimum value Rmin method with k-fold cross-validation, which means that the
Average change value Ad Average change value Rd whole DataSet is divided into k mutually exclusive subsets
Mean crossings value AMCV Mean crossings value RMCV with approximately equal size. One subset is taken as the test
Root mean square Root mean square set and the rest as the training set while training GBDT
ARMS RRMS
value value model. In this way, the model can be trained and tested for k
times to evaluate the performance of the model more
objectively.
DataSet GBDT classifier is regarded as an additional model
composed of CART [31] illustrated in equation (8). Clas-
sification is realized by continuously decreasing the residuals
Base Base Base generated in the training process as follows:
classifer classifer classifer
N
f(X) � 􏽘 h X; θn 􏼁, (8)
Learning Learning Learning n�1

where f(X) denotes the overall model, h(X; θn ) denotes the

Weight Weight Weight
n-th decision tree, θn denote parameters of the n-th decision
tree, N denotes the number of decision trees.
The purpose of GBDT is to make loss function reduce as
fast as possible and preferably fall along its gradient di-
Ensemble model
rection. At each round, the negative gradient of the log-
likelihood loss function is used to fit the new CART, which
Output could accelerate the reduction and convergence of loss
function as soon as possible and finally speed up the training
Figure 9: Ensemble learning. process. GBDT-FD will train CART for fall detection
through several iterations.
The following describes the main steps of the GBDT-FD
The 14-dimensional Merged DataSet MDS defined in algorithm.
Section 2 is as follows:
MDS � 􏼈 X1 , y1 􏼁, X2 , y2 􏼁, . . . , Xm , ym 􏼁􏼉, (6)
3.1.1. Initialization. Firstly, GBDT-FD algorithm chooses
where X denotes 14-dimensional data, which is shown as one feature from X as the CART node and then selects an
follows: y denotes labels, y � 1 means fall, and y � −1 means appropriate eigenvalue as the segmentation point. For ex-
normal activity: ample, Amean is taken as the root node. In general, Amean
8 Journal of Healthcare Engineering

value of a fall is usually much larger than that of normal ΔL(y, f(X)) < ε, (15)
activity. Finally, the Merged DataSet is divided into two
categories with the eigenvalue boundary of preset seg- where ΔL(y, f(X)) denotes the decrease of the loss func-
mentation. It can be roughly considered that these two tion, and ε is the training termination threshold set by the
categories represent falls and normal activities, respectively, GBDT-FD algorithm.
but the loss function will be very large. GBDT-FD will iterate
through all combinations of features and segmentation
points to minimize the loss function of CART. Therefore, 3.1.3. Classification. GBDT-FD classification process is as
initial CART has only one root node as in the following follows:
equation and M denotes the number of samples: (1) Input unclassified test dataset
M
(2) The trained CART will start from the root node and
f0 (X) � arg min 􏽘 L ym , c􏼁. (9) compare with the input data based on features and
m�1
segmentation points
(3) After the comparison process is completed, CART
will output the category of each input data
3.1.2. Iterations. Suppose the number of iterations is T, and
(4) Test datasets are classified into two categories: fall
etm denotes the negative gradient error of m-th sample in the
and nonfall
t-round iteration; hence,
zL ym , f Xm 􏼁􏼁 ym
etm � −􏼢 􏼣 � . 3.2. Performance Metrics. As we know, for the dichotomous
zf xm 􏼁 f(X)�ft−1 (X) 1 + exp y m f xm 􏼁 􏼁
problem, the final classifying results can be divided into four
(10) categories: True Positive samples (TP), False Negative samples
(FN), False Positive samples (FP), and True Negative samples
GBDT-FD will train the CARTof this iteration according
(TN) on the basis of different combinations of real category
to (Xm , etm )(i � 1, 2, . . . , M). Leaf node region
and predicted category, which are summarized in Table 2.
Retj (j � 1, 2, . . . , J) will be obtained by J denoting the
Since fall detection belongs to a typical dichotomous
number of regression leaf nodes. Using linear search, esti-
problem (fall and nonfall), five integrated metrics are used to
mate the value of the leaf node region and minimize the loss
verify performance usually [2].
function:
(1) Precision: the ratio between real falling samples and
ctj � arg min 􏽘 log 1 + exp −ym ft−1 Xm 􏼁 + c􏼁􏼁􏼁.
Xm ∈Retj
predicted falling samples:

(11) TP
Ipre � . (16)
TP + FP
Since equation (11) is difficult to optimize, we use ap-
proximation as follows:
(2) Sensitivity: the ratio between samples correctly
􏽐Xm ∈Retj etm identified as falls and real falling samples:
ctj � 􏼌􏼌 􏼌􏼌 􏼌􏼌 􏼌􏼌 . (12)
􏽐X ∈Re 􏼌􏼌etm 􏼌􏼌􏼐1 − 􏼌􏼌etm 􏼌􏼌􏼑
m tj TP
Isens � . (17)
Then a new learner generated by this iteration process TP + FN
can be expressed as follows:
J
(3) Specificity: the ratio between samples correctly
ft (X) � ft−1 (X) + 􏽘 ctj I, Xm ∈ Retj . identified as nonfalls and real nonfalling samples:
(13)
j�1
TN
Ispec � . (18)
However, in order to avoid overfitting, learning rate v TN + FP
ranging in (0, 1] should be considered when updating
learners, so the evolution rule of a new learner is as follows: (4) Accuracy: the ratio between samples correctly
J identified and the total sample set:
ft (X) � ft−1 (X) + v · 􏽘 ctj I, Xm ∈ Retj . (14)
TP + TN
j�1 Iacc � . (19)
TP + FN + TN + FP
In general, much more iterations mean smaller loss
function and better performance. However, the loss function (5) F-score: weighted harmonic mean of precision and
will decrease less obviously as the number of iterations sensitivity:
increases. In order to save training time and computational
resources, GBDT-FD sets training termination conditions: 􏼐β2 + 1􏼑 · Ipre · Isens
The training process will stop if, Iβ−score � , (20)
β2 · Ipre + Isens
Journal of Healthcare Engineering 9

Table 2: Different combinations of real category and predicted

category.
Predicted category
Real category
Fall Non-fall
Fall TP FN
Non-fall FP TN

where β is a nonnegative parameter, and different β indicates

different weights between precision and sensitivity. Ap-
parently, there are three main cases:
(1) β � 1, denoting the weight of Ipre and Isens is equal
(2) β > 1, denoting the weight of Isens is greater than that Figure 10: Experimental scene.
of Ipre
(3) β < 1, denoting the weight of Ipre is greater than that significantly. But we can find from Figure 11 that the time
of Isens required for the training model is still approximately linear,
so the optimal value of iteration times is set to 110 in this
According to the difference of β, I1-score, I0.5-score, and paper.
I2-score are commonly used in statistical occasions. In terms Some main parameters of GBDT-FD are shown in
of fall detection, we should pay attention to the sensitivity of Table 3.
the detection model so as to reduce health hazards effec-
tively, and moreover, ensure the wounded could get im-
mediate medical assistance when fall occurs. Meanwhile, 4.2. Experimental Results. The original sensory data of body
wrong fall alarm may lead to a waste of communication activities in a typical fall case are illustrated in Table 4 and
resource, so it is necessary to pay attention to the precision of Figure 12. Table 4 displays a complete fall process with 6 key
the model, but it is not as important as the sensitivity metric. frames. Figure 12 demonstrates the detailed change law of
the A3-axis during this process. It is clear that both types of
4. Experimental Results acceleration sensory data and human posture perception
data can reflect the fall process on many occasions.
4.1. Experimental Setup. The experimental scene at our Unfortunately, the previous situation is not always
school is illustrated in Figure 10. Ten graduate students (6 guaranteed. In some cases, only a single type of sensory data
male and 4 female) with average age 26 are volunteered for is difficult to classify falls and normal behaviors accurately.
this research testing. Each user’s real-time acceleration data When a user lies down, the aspect ratio R will change a lot as
during daily activities is gathered by our wearable posture given in Figure 13(a), so it will be recognized as fall activity
sensor, and monitoring video is recorded using a surveil- using separated video data. But it is a misjudgment clearly
lance camera. In order to imitate the real behavior of the and we can verify this from A3−axis variation curve. Nev-
aged, we also tie sandbags on our feet, so there is amount of ertheless, the fall classification result of this case should be
falling down slowly situations in our training data set. With right using the accelerometric data, which is displayed in
this scenario, we got more than 150 videos with each lasting Figure 13(b).
for 1–1.5 minutes. Finally, more than 6000 activities records In either case, when a user jumps up, it is possible to
are used to verify our GBDT based algorithm, 20% of them identify this kind of activity as fall because the acceleration
are used for training, and the rest 80% is used for testing. For value changes extremely violently. But it is not a fall through
safety’s sake, the experimental environment is padded video data from Figure 14 because there is no great change of
completely and tight. The acceleration data and video data aspect ratio R. To summarize, these two special examples
are combined to form the merged data to store in Cloud verify that more accurate fall detection results could be
Server and finally input to GBDT based fall down classifier. achieved with data fusion using the Merged DataSet, and not
Firstly, how to determine optimal parameters for GBDT from a single dataset. Hence, this discovery is also the main
based fall detection algorithm is discussed. We treat I2-score purpose and significance of this article.
and accuracy as main evaluation criteria and I1-score and Secondly, we compare the detection accuracy influence
I0.5-score as the reference when training GBDT based de- of each feature. GBDT can output the relative importance of
tection dichotomy model. each feature to model training so as to help understand the
In the whole GBDT model, the iteration times have an influence of each feature on fall down detection. As shown in
important influence on the quality of the model. When the Figure 15, the sum of importance ratio of features in Video
iteration times increase from 20 to 300, the I2-score and DataSet (VDS) is 50.4%, and that in Acceleration DataSet
accuracy value of the GBDT model will increase. However, it (ADS) is 49.6%. This indicates that both datasets play an
is clear from Figure 11 when the iteration times exceed 110, important role in classification, so we choose all these
I2-score and accuracy of GDBT model no longer increase characteristics as our source data in GBDT-FD.
10 Journal of Healthcare Engineering

The performance in different iterations The performance in different iterations

1 1.5

Iteration = 110
0.95

Accuracy = 93.7%
0.9 1

t (s)
0.85 Iteration = 110

0.8 I2-score = 0.826 0.5

t = 0.48 s

0.75

0.7 0
50 100 150 200 250 300 50 100 150 200 250 300
I2-score Iterations
Accuracy
Figure 11: The influence of iteration times of GBDT-FD.

Table 3: GBDT-FD parameters.

Parameter Symbol Optimal value
Learning rate v 0.35
Iteration rounds T 110
Cross validation k 5
Total sample number M 5160
Terminal condition ε 10− 4

Table 4: Fall process.

Table 4: Continued.
Frame no. Scene
Frame no. Scene

1
65

20
85

30

Furthermore, we compare the fall judgment results of

support vector machine (SVM) based Fall Detection (SVM-
FD) [32], Naive Bayes (NB) based Fall Detection (NB-FD)
[33], Decision Tree (DT) based Fall Detection (DT-FD) [34],
49 K-Nearest Neighbor (KNN) based Fall Detection (KNN-FD)
[35], Neural Network (NN) based Fall Detection (NN-FD)
[36], Random Forest (RF) based Fall Detection (RF-FD)
[37], and GBDT based Fall Detection (GDBT-FD) with the
Merged DataSet, acceleration DataSet, and skeleton DataSet,
respectively. The results in Tables 5–7 explain the
Journal of Healthcare Engineering 11

Fall down
30
Frame 49
25 Frame 65

A3-axis (g) 20

15
Frame 85
Frame 1
10 Frame 20

5 Frame 30

0
10 20 30 40 50 60 70 80 90 100
Frame
Figure 12: A3−axis variation when falling down.

(a)
Lie down
30

25

20
A3-axis (g)

15

10

0
20 40 60 80 100 120
Frame
(b)

Figure 13: Sensory data when lying down. (a) Video skeleton data when lying down. (b) A3-axis variation when lying down.

(a)
Figure 14: Continued.
12 Journal of Healthcare Engineering

Jump
30

25

A3-axis (g) 20

15

10

0
20 40 60 80 100 120
Frame
(b)

Figure 14: Sensory data when jumping up. (a) Video skeleton data when jumping up. (b) A3-axis variation when jumping up.

Rstd
Astd
ARMS
Rmin
RRMS
Amin
Amax
Aavg
Ad
Rmax
Rd
Ravg
AMCV
RMCV
0 5 10 15 20 25 30
P (%)
Figure 15: Feature importance in GBDT-FD algorithm.

comparable detection results of these algorithms with three performance is still not good as that with Merged DataSet
kinds of DataSet, respectively. since the Merged DataSet expands the data dimensions, and
It is clear from Table 5 that I2-score and accuracy of more features are used to make the model much easier to
GBDT-FD are, respectively, 0.878 and 95%. The results of train. The results verify that using a posture sensor and video
GBDT based fall detection algorithm outperform that of skeleton fusion will be more accurate than the traditional
SVM-FD, NB-FD, DT-FD, KNN-FD, NN-FD, and RF-FD, individual detection method. However, the other methods
which indicates that GBDT-FD can identify fall events ac- have poor generalization ability due to the defects of al-
curately. NN-FD and RF-FD are slightly worse than GBDT- gorithms. Therefore, the detection results are not
FD. Due to the simplicity of the Naive Bayes model, the satisfactory.
performance of the NB-FD algorithm is not ideal too. The Finally, the most important index I2-score of each algo-
Accuracy of SVM-FD and KNN-FD is not much worse than rithm using three different DataSet is compared in Figure 16.
that of GBDT-FD, but there is a big gap between them in I2- The performance advantages of the proposed GBDT-FD al-
score. gorithm are obvious. From the aspect of system imple-
From detection results in Tables 6 and 7 with inde- mentation, the Web interface is shown in Figure 17. There are
pendent DataSet, it is clear that GBDT-FD has good gen- a few users who have already connected and transmitted
eralization ability and can handle various types of DataSet. sensory data to the Cloud Server. GDBT-FD algorithm is
Moreover, the recognition results with Video DataSet are executed on the Server. At present, the well-designed fall
slightly better than that of Acceleration DataSet. This is detection online platform is already in trial operation in
because the characteristics of video skeleton data are more nursing homes, and so it is with a good result in application
directly perceived than that of acceleration. However, the and worth popularization.
Journal of Healthcare Engineering 13

Table 5: Detection results of each algorithm with merged DataSet.

Merged DataSet
Model I0.5-score I1-score I2-score Accuracy (%)
SVM-FD 0.842 0.793 0.749 91.6
NB-FD 0.630 0.579 0.535 83.5
DT-FD 0.806 0.805 0.789 90.9
KNN-FD 0.785 0.751 0.72 89.7
NN-FD 0.846 0.835 0.825 92.8
RF-FD 0.846 0.831 0.818 92.7
GBDT-FD 0.878 0.886 0.878 95

Table 6: Detection results of each algorithm with Acceleration DataSet.

Acceleration DataSet
Model I0.5-score I1-score I2-score Accuracy (%)
SVM-FD 0.707 0.600 0.516 85.8
NB-FD 0.619 0.557 0.506 83.1
DT-FD 0.659 0.613 0.602 83.8
KNN-FD 0.707 0.656 0.613 86.3
NN-FD 0.685 0.670 0.655 85.8
RF-FD 0.741 0.656 0.588 87.1
GBDT-FD 0.777 0.737 0.696 89.2

Table 7: Detection results of each algorithm with Video DataSet.

Video DataSet
Model I0.5-score I1-score I2-score Accuracy (%)
SVM-FD 0.728 0.639 0.574 86.6
NB-FD 0.608 0.587 0.567 82.6
DT-FD 0.678 0.677 0.631 84.6
KNN-FD 0.692 0.658 0.628 85.9
NN-FD 0.707 0.692 0.677 86.7
RF-FD 0.739 0.695 0.656 87.6
GBDT-FD 0.754 0.728 0.706 88.6

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
SVM-FD NB-FD DT-FD KNN-FD NN-FD GBDT-FD RF-FD

Merged DataSet
Acceleration DataSet
Video DataSet
Figure 16: I2-score comparison of each algorithm with different DataSets.
14 Journal of Healthcare Engineering

Figure 17: Web interface on the cloud server.

5. Conclusions Zhejiang Provincial Key Lab of Equipment Electronics, and

National Innovation and Entrepreneurship Training Pro-
In this paper, we propose one kind of comprehensive gram for College Students.
framework of the fall detection system using inertial triaxial
acceleration sensors and monitoring cameras to detect fall
accidents. The wearable triaxial accelerometer is used to References
detect the body’s posture, and a monitoring camera is used [1] S. C. Mukhopadhyay and S. Chandra, “Wearable sensors for
to extract key points of human skeletons information. The human activity monitoring: a review,” IEEE Sensors Journal,
fall detection is operated on the basis of fusion based data vol. 15, no. 3, pp. 1321–1330, 2015.
including accelerometric data and human skeleton key [2] L. S. Hu, S. Z. Wang, Y. Q. Chen et al., “Fall detection al-
points. In order to reduce false positives of falling incidents, gorithms based on wearable device: a review,” Journal of
GBDT classifier based fall detection algorithm is investigated Zhejiang University (Engineering Science), vol. 52, no. 9,
in depth. The good performance of the proposed GBDT-FD pp. 92–103, 2018.
algorithm is compared with SVM-FD, NB-FD, DT-FD, [3] R. Broadley, J. Klenk, S. Thies, L. Kenney, and M. Granat,
KNN-FD, NN-FD, and RF-FD in terms of I0.5-score, I1-score, I2- “Methods for the real-world evaluation of fall detection
score, and accuracy, so as to verify the performance im-
technology: a scoping review,” Sensors, vol. 18, no. 7, p. 2060,
provements of GBDT-FD. In our future work, multiview 2018.
human skeleton extraction will be adopted, and detection [4] E. Boutellaa, O. Kerdjidj, and K. Ghanem, “Covariance matrix
speed will be improved so as to enhance the real-time based fall detection from multiple wearable sensors,” Journal
performance of our system. of Biomedical Informatics, vol. 94, pp. 103–189, 2019.
[5] J. K. Lee and E. J. Park, “3D spinal motion analysis during
staircase walking using an ambulatory inertial and magnetic
Data Availability sensing system,” Medical & Biological Engineering & Com-
puting, vol. 49, no. 7, pp. 755–764, 2011.
The data supporting the results of this study are available [6] D. Litvak, Y. Zigel, and I. Gannot, “Fall detection of elderly
from the corresponding author upon request. through floor vibrations and sound,” in Proceedings of the
2008 Annual International Conference of the IEEE Engineering
Conflicts of Interest in Medicine and Biology Society, IEEE, Vancouver, Canada,
pp. 4632–4635, August 2008.
The authors declare that they have no conflicts of interest. [7] Y. Zigel, D. Litvak, and I. Gannot, “A method for automatic
fall detection of elderly people using floor vibrations and
Acknowledgments sound-proof of concept on human mimicking doll falls,” IEEE
Transactions on Biomedical Engineering, vol. 56, no. 12,
This research was partially supported by the National pp. 2858–2867, 2009.
Natural Science Foundation of China (Grant Nos. 61801431 [8] A. Edgcomb and F. Vahid, “Automated fall detection on
and 61871163), Fundamental Research Funds for the Pro- privacy-enhanced video,” in Proceedings of the 2012 Annual
vincial Universities of Zhejiang (GK199900299012), International Conference of the IEEE Engineering in Medicine
Journal of Healthcare Engineering 15

and Biology Society, IEEE, San Diego, CA, USA, pp. 252–255, [24] Y. P. E. Wang, X. Lin, A. Adhikary et al., “A primer on 3GPP
August 2012. narrowband internet of things (NB-IoT),” IEEE Communi-
[9] B. Z. Dai, D. Q. Yang, L. G. Ai et al., “A novel video-sur- cations Magazine, vol. 55, no. 3, 2016.
veillance-based algorithm of fall detection,” in Proceedings of [25] Z. Cao, G. Hidalgo, T. Simon et al., “OpenPose: realtime
the 2018 International Congress on Image and Signal Pro- multi-person 2D pose estimation using part affinity fields,”
cessing, BioMedical Engineering and Informatics (CISP- 2019, https://arxiv.org/abs/1812.08008.
BMEI), pp. 1–6, Beijing, China, January 2018. [26] M. Datar, A. Gionis, P. Indyk et al., “Maintaining stream
[10] L. Liu, M. Popescu, M. Skubic et al., “Automatic fall detection statistics over sliding windows (extended abstract),” in Pro-
based on Doppler radar motion signature,” in Proceedings of ceedings of the 13th ACM-SIAM Symposium on Discrete
the 5th International Conference on Pervasive Computing Algorithms, San Francisco, CA, USA, 2002.
[27] J. H. Friedman, “Greedy function approximation: a gradient
Technologies for Healthcare (PervasiveHealth) and Workshops,
boosting machine,” The Annals of Statistics, vol. 29, no. 5,
pp. 222–225, Dublin, Ireland, 2011.
pp. 1189–1232, 2001.
[11] A. Nag, S. C. Mukhopadhyay, and J. Kosel, “Wearable flexible
[28] S. S. Khan and J. Hoey, “Review of fall detection techniques: a
sensors: a review,” IEEE Sensors Journal, vol. 17, no. 13,
data availability perspective,” Medical Engineering & Physics,
pp. 3949–3960, 2017. vol. 39, pp. 12–22, 2017.
[12] L. Chen, R. Li, H. Zhang, L. Tian, and N. Chen, “Intelligent fall [29] L. Liu, T. C. Lu, Y. Y. Lv et al., “MEMS tri-axial accelerometer
detection method based on accelerometer data from a wrist- based fall detection,” Chinese Journal of Sensors and Actuators,
worn smart watch,” Measurement, vol. 140, pp. 215–226, 2019. vol. 4, pp. 570–574, 2014.
[13] S. A. Waheed and P. S. A. Khader, “A novel approach for [30] R. Polikar, “Ensemble learning,” Scholarpedia, vol. 4, no. 1,
smart and cost effective IoT based elderly fall detection system pp. 1–34, 2009.
using Pi camera,” in Proceedings of the 2017 IEEE Interna- [31] B. S. Everitt, Classification and Regression Trees—Encyclopedia
tional Conference on Computational Intelligence and Com- of Statistics in Behavioral Science, John Wiley & Sons,
puting Research (ICCIC), pp. 1–4, Beijing, China, September Hoboken, NJ, USA, 2005.
2017. [32] M. A. Hearst, S. T. Dumais, E. Osuna, J. Platt, and
[14] D. Yacchirema, J. S. de Puga, and C. M. Palau, “Fall detection B. Scholkopf, “Support vector machines,” IEEE Intelligent
system for elderly people using IoT and big data,” Procedia Systems and Their Applications, vol. 13, no. 4, pp. 18–28, 1998.
Computer Science, vol. 130, pp. 603–610, 2018. [33] M.-L. Zhang, J. M. Peña, and V. Robles, “Feature selection for
[15] Y. Li, K. C. Ho, and M. Popescu, “A microphone array system multi-label naive Bayes classification,” Information Sciences,
for automatic fall detection. IEEE Transactions on Bio- vol. 179, no. 19, pp. 3218–3229, 2009.
Medical Engineering,” vol. 59, no. 5, pp. 1291–1301, 2012. [34] J. R. Quinlan, “Induction of decision trees,” Machine
[16] S. M. Adnan, A. Irtaza, S. Aziz, M. O. Ullah, A. Javed, and Learning, vol. 1, no. 1, pp. 81–106, 1986.
M. T. Mahmood, “Fall detection through acoustic local ter- [35] K. Q. Weinberger and L. K. Saul, “Distance metric learning for
nary patterns,” Applied Acoustics, vol. 140, pp. 296–300, 2018. large margin nearest neighbor classification,” Journal of
[17] P. S. Sase and S. H. Bhandari, “Human fall detection using Machine Learning Research, vol. 10, no. 1, pp. 207–244, 2009.
depth videos,” in Proceedings of the 2018 5th International [36] L. K. Hansen and P. Salamon, “Neural network ensembles,”
IEEE Transactions on Pattern Analysis & Machine Intelligence,
Conference on Signal Processing and Integrated Networks
vol. 12, no. 10, pp. 993–1001, 2002.
(SPIN), pp. 546–549, New Delhi, India, February 2018.
[37] V. Svetnik, “Random forest: a classification and regression
[18] L. X. Shen, Q. Y. Zhang, X. C. Guo et al., “Fall detection system
tool for compound classification and QSAR modeling,”
based on deep learning and image processing in cloud en- Journal of Chemical Information & Computer Sciences, vol. 43,
vironment,” in Proceedings of the 2018 Conference on Com- no. 6, pp. 1947–1958, 2002.
plex, Intelligent, and Software Intensive Systems, Matsue,
Japan, July 2018.
[19] M. Kepski and B. Kwolek, “Fall detection on embedded
platform using Kinect and wireless accelerometer,” Com-
puters Helping People with Special Needs, pp. 407–414,
Springer, Berlin, Germany, 2012.
[20] H. M. Hondori, M. Khademi, and C. V. Lopes, “Monitoring
intake gestures using sensor fusion (microsoft Kinect and
inertial sensors) for smart home tele-rehab setting,” in Pro-
ceedings of the 2012 IEEE Conference on Healthcare
Innovation, Houston, TX, USA, 2012.
[21] Y. Nizam, M. N. H. Mohd, and M. M. A. Jamil, “Development
of a user-adaptable human fall detection based on fall risk
levels using depth sensor,” Sensors, vol. 18, no. 2260, pp. 1–14,
2018.
[22] B. Kwolek and M. Kepski, “Human fall detection on em-
bedded platform using depth maps and wireless accelerom-
eter,” Computer Methods and Programs in Biomedicine,
vol. 117, no. 3, pp. 489–501, 2014.
[23] Y. He, F. Yang, Y. Yu, and C. Grebogi, “An elderly care system
based on multiple information fusion,” Journal of Healthcare
Engineering, vol. 2018, Article ID 8102597, 13 pages, 2018.