308 IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, VOL. 66, NO.

4, NOVEMBER 2020

A WiFi-Based Smart Home Fall Detection System

Using Recurrent Neural Network
Jianyang Ding , Graduate Student Member, IEEE, and Yong Wang , Member, IEEE

Abstract—Falls among the elderly living on their own have on the response and rescue time [2]. The delay of medical
been regarded as a major public health worry that can even treatment after a fall can increase the mortality risk in clin-
lead to death. Fall detection system (FDS) that alerts caregivers ical conditions, half of those who experienced an extended
or family members can potentially save lives of the elderly.
However, conventional FDS involves wearable sensors and spe- period of lying on the floor died within six months after an
cialized hardware installations. This article presents a passive incident [3]. In addition to physical injuries and high medi-
device-free FDS based on commodity WiFi framework for smart cal cost, falls can cause psychological damage to the elderly
home, which is mainly composed of two modules in terms of as well, which is termed as the fear of falling cycle by the
hardware platform and client application. Concretely, commer- fall researchers [3]. The fear cycle refers to the fact that after
cial WiFi devices collect disturbance signal induced by human
motions from smart home and transmit the data to a data analy- a fall, even without injury, the elderly become so afraid of
sis platform for further processing. Based on this basis, a discrete falling again that they would reduce physical activities [3].
wavelet transform (DWT) method is used to eliminate the influ- This in turn decreases their fitness, mobility and balance, and
ence of random noise presented in the collected data. Next, leads to decreased social interactions, reduced life satisfaction,
a recurrent neural network (RNN) model is utilized to clas- and increased depression. This fear cycle further increases the
sify human motions and identify the fall status automatically.
By leveraging Web Application Programming Interface (API), risk of another fall. Especially for the elderly who live alone
the analyzed data is able to be uploaded to the proxy server and independently, about 50% of the falls occur within their
from which the client application then obtains the corresponding own home, so timely and automatic detection of falls has the
fall information. Moreover, the system has been implemented as potential to save the lives of the elderly [4].
a consumer mobile App that can help the elderly saving their lives Up to now, many health-monitoring approaches are already
in smart home, and detection performance of the proposed FDS
has been evaluated by conducting comprehensive experiments on common in people’s daily life, and also various applications
real-world dataset. The results confirm that the proposed FDS is are widely developed [5]–[10]. These approaches utilizing
able to achieve a satisfactory performance compared with some wearable sensors can efficiently monitor the elderly’s phys-
state-of-the-art algorithms. ical conditions at home, and their performance is satisfactory.
Index Terms—Artificial intelligence, channel state information, However, all of these approaches require users to wear dedi-
data processing, mobile applications, smart homes, wireless cated devices or sensors on their bodies all the time, which has
communication, Web services. somewhat limited a large-scale deployment of such systems.
Recently, with the rapid development and ubiquity of com-
mercial WiFi devices, there are increasingly applications using
I. I NTRODUCTION WiFi signal [11]–[13]. Specifically, most of the modern off-
ALLS are the leading cause of fatal and nonfatal injuries the-shelf WiFi devices operate on both the 2.4 GHz and
F to the elderly in the modern society. According to the
center for Disease Control and Prevention, one out of three
5 GHz frequency bands [14]. Owing to the advantages of
wide bandwidth and supporting multiple input multiple out-
adults aged 65 and over fall each year at home [1], [2]. Falls put (MIMO) techniques, current commercial WiFi devices start
not only bring a main threat to the elderly’s health, but also to track fine-grained channel measurements using orthogo-
account for a large part of medical cost. Most of the elderly are nal frequency division modulation (OFDM) technology at the
unable to get up by themselves after a fall, and studies have physical layer [14]. In particular, the OFDM channel is a rel-
shown that the medical outcome of a fall is largely dependent atively wide channel divided into multiple subcarriers where
each subcarrier has a different signal amplitude and phase
Manuscript received November 29, 2019; revised March 13, 2020, June 12, in terms of each transmitted signal [15]. For instance, the
2020, and August 19, 2020; accepted August 31, 2020. Date of publication mainstream WiFi systems, such as 802.11 a/g/n, are based
September 3, 2020; date of current version November 25, 2020. This work
was supported in part by the Natural Science Foundation of China under on OFDM where a wideband 20 MHz channel is partitioned
Grant 61671346, and in part by the Natural Science Basic Research Plan in into 52 subcarriers [15]. Owing to the frequency diversity of
Shaanxi Province of China under Grant 2020JM-180. (Corresponding author: these subcarriers, both the shadow fading and multipath effect
Jianyang Ding.)
Jianyang Ding is with the School of Telecommunications Engineering, at different subcarriers can lead to different amplitude and
Xidian University, Xi’an 710071, China (e-mail: [email protected]). phase. This means that a small body movement in indoor
Yong Wang is with the State Key Laboratory of Integrated environments would result in the change of channel state
Services Networks, Xidian University, Xi’an 710071, China (e-mail:
[email protected]). information (CSI) at all the subcarriers [16]. Based on this
Digital Object Identifier 10.1109/TCE.2020.3021398 view, various human motions occurring between a pair of WiFi
1558-4127 
c 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.
DING AND WANG: WiFi-BASED SMART HOME FALL DETECTION SYSTEM USING RECURRENT NEURAL NETWORK 309

Fig. 1. Fall detection system.

transmitter and receiver influence the indoor electromagnetic discrete wavelet transform (DWT) [24], [25] and recurrent
environments in a unique manner which would be changed neural network (RNN) [26]–[28] model plays an important
into the impact on the characteristics of WiFi signal. Then, role in fall status decisions for the elderly. Concretely, in
the significant impact is in turn manifested as distinct per- order to preserve data details while filtering out the noise
turbation in WiFi CSI [17]. Besides, different motions yield presented in the collected data, the DWT method is lever-
different impacts on the characteristics of WiFi signal, and thus aged to eliminate the influence of the random noise by signal
fall detection is able to be feasible by examining the represen- decomposition. Next, the RNN model is used to classify var-
tative patterns of the amplitude information of WiFi CSI. In ious human motions by utilizing the denoised data above,
addition, due to these advantages of low cost, no invasion, and and then identify fall status automatically. As the term recur-
wide deployment of WiFi devices in indoor environments (e.g., rent implies, the RNN model takes not only the current input
home and office), fall detection can become reality. Previous data but also several previous input data. That is to say, it has
studies [18]–[24] have confirmed that WiFi infrastructures can a memory that can obtain the variation in input data. Thus, the
support a high-accuracy-low-cost fall detection for the elderly selected strategy has the capacity for capturing the complicated
with simple deployment, engaging user experience, and limited non-linear relationship between input and output data during
personal privacy. More importantly, the passive device-free training efficiently. To conduct remote monitoring better, the
fall detection using WiFi CSI does not require the users to real-time data analyzed is uploaded to a configured proxy
take any dedicated device or sensor, which means that the server. In addition, the client application interacts with the
nature of this approach is “set it and forget it” requiring no server and obtains the motion information in the form of texts
interaction on the elderly’s part after initial set up. As a result, and graphs from the server in real time. Specially, both of them
this detection approach could help to reveal a wealth of impor- take use of a lightweight architectural style Web Application
tant health information, particularly for the elderly. A change Programming Interface (API) [9] to facilitate communication
in the CSI amplitude, for example, could mean that the per- utilizing Web services. Then, the consumer or the commu-
son has suffered an injury or is at an increased risk of falling. nity owner can be aware of the fall behavior via a mobile
By doing this, it can be applied to some specific scenar- phone or a management platform, and takes emergency mea-
ios on remote heath monitoring, such as hospitals, nursing sures to save the lives of the elderly in time in smart home.
homes, and the elderly living on their own. Due to the advan- Fig. 1 displays the concrete process of fall detection show-
tage of passive device-free detection, the elderly may feel that ing the data flow from home WiFi devices to end consumer
they are not offended. In the present study, our proposed fall application.
detection system (FDS) thus takes advantage of the amplitude The rest of this article is organized as follows. We first
information of fine-grained WiFi CSI to capture the minute review the related work in Section II. Then we present
movement for fall detection in smart home. the detailed system design in Section III. Followed by the
In order to achieve automatic passive fall detection in smart motions dataset collection, evaluation, and comparison results
home, an effective system using commercial WiFi devices in Section IV. In Section V, we present the mobile App and
and the emerging technologies is presented in this article. its analysis. Finally, we conclude the work in Section VI.
At first, we build an environment change indicator exploit-
ing motion-induced signal fluctuation to infer potential motion
interference with a pair of commercial WiFi devices installed II. R ELATED W ORK
in residential units. In the case of a human motion occur- This section reviews related work of fall detection in indoor
ring in the target area, the detection data is collected and environments. Moreover, existing systems can be classified
it is constantly transmitted to the data analysis platform for into two categories: Sensors-based fall detection and WiFi
further processing. Data analysis on the collected data using CSI-based fall detection.

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.
310 IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, VOL. 66, NO. 4, NOVEMBER 2020

A. Sensors-Based Fall Detection array that captured the sounds in a room. Concretely, when
Due to the importance of fall detection, several fall detec- a sound was detected, acoustic-FADE located the source,
tion approaches based on sensors have been proposed, which enhanced the signal, and classified it as “fall” or “non-fall”.
can be broadly classified as videos, wearable sensors, and Rimminen et al. [34] used a near-field imaging (NFT) to
ambient sensors. Khan and Sohn [5] proposed a video sensor- conduct detection utilizing floor sensor and pattern recogni-
based human activity recognition system for the elderly care tion. The floor sensor detected the locations and patterns of
by recognizing six abnormal activities, including forward fall, the elderly by measuring impedances with a matrix of thin
backward fall, chest pain, faint, vomit, and headache, selected electrodes under the floor. These fall detection approaches
from the daily life activities. Besides, the proposed system based on ambient sensors can guarantee certain accuracy as
took use of the kernel discriminant analysis (KDA) to increase presented, but other sources of pressure, vibration, or sound
discrimination among different classes of activities by utilizing in a room could cause false alarms. Besides, in order to
non-linear technique. Then, Hidden Markov Model (HMM) ensure high detection performance, a high-density deployment
was employed for training and recognition of human activities. in indoor environments is required.
Yang and Tian [29] presented a novel framework for recog-
nizing various human activities from video sequences captured
by depth cameras, and extended the surface normal to poly- B. WiFi CSI-Based Fall Detection
normal by assembling local neighboring hypersurface normals Since WiFi signal has been verified to be reliable indica-
from a depth sequence to jointly characterize local motion and tor for device-free passive fall detection, more researchers
shape information. Usually, these video-based approaches are seek for the opportunity to identify fall motion in indoor
highly accurate. However, these approaches are affected heav- environments. Wang et al. [3] presented the design and imple-
ily by some issues, such as high installation overhead, needing mentation of RT-Fall, a real-time, contactless, and low-cost
light, out of sight range, personal privacy problem, etc. accurate indoor fall detection system using the commodity
Wang et al. [6] proposed an enhanced fall FDS for the WiFi devices. Wang et al. [18] proposed a monitoring frame-
elderly monitoring which was based on smart sensors worn work that can run on a single WiFi AP with its connected
on the body and operating through consumer home networks. devices and used the associated profile matching algorithms to
With treble thresholds, accidental fallings can be detected in compare amplitude profiles against those from known activi-
the home healthcare environment. By leveraging information ties. Wang et al. [19] presented a human activity recognition
gathered from an accelerometer, a cardiotachometer, and smart and monitoring system based on CSI. Besides, the system con-
sensors, the impacts of falls can be logged and distinguished sidered two theoretical underpinnings: a CSI-speed model and
from normal daily activities. Yang et al. [10] presented a CSI-activity model. By using these two models, the cor-
a wavelet-neural network recognition system for personal relation between CSI value dynamics and a specific activity
fitness assistance and the elderly daily activity monitoring could be built quantitatively. Abdelnasser et al. [20] proposed
applications by utilizing Internet of Thing (IoT) technology. a novel system that leveraged changes in WiFi signal strength
This discrete wavelet transform radial basis neural network to sense in-air hand gestures around the user’s mobile device.
analyzed vibration induced by human motions, and identified Besides, the system utilized standard WiFi equipment, with
motion status automatically. Lara and Labrador [30] sur- no modifications, and required no training for gesture recog-
veyed the state of the art in human activity recognition based nition. Chen et al. [21] presented a new deep learning-based
on wearable sensors, and proposed a two-level taxonomy in approach, i.e., attention based bi-directional long short-term
accordance to the learning approach and the response time. memory, for passive human activity recognition using WiFi
Gunadi et al. [31] used Kinect device that can be utilized to signal. Additionally, the BLSTM was employed to learn rep-
detect the elderly. Kinect can detect a person in front of it and resentative features in two directions from raw WiFi CSI
process it to create a skeleton of the person. Although these measurements. Zhang et al. [22] designed and implemented
wearable sensors-based detection approaches are very effec- a real-time, non-intrusive, and low-cost indoor fall detector
tive, especially in indoor environments, they may not always by leveraging the fine-grained CSI and multi-antenna setting
be accepted by the elderly. In addition, some people are unwill- in commercial WiFi devices. Wang et al. [23] firstly looked
ing to wear the dedicated sensors on their bodies all the time. for the correlations between different radio signal variations
As a result, this disadvantage somehow restricts large-scale and activities by analyzing radio propagation model. Based
deployment of such systems in typical indoor environments. on this observation, they proposed a truly unobtrusive FDS
Additionally, Nadee and Chamnongthai [32] presented an that can detect the fall of the human without hardware mod-
elderly-falling detection system leveraging ultrasonic sensors. ification, extra environmental setup, or any wearable device.
The ultrasonic technology-based multi sensors were couple of Palipana et al. [24] considered an emerging non-wearable fall
receiver and transmitter together, which could be connected detection approach based on WiFi CSI, and took an altogether
to Arduino microcontroller in order to send the elderly per- different direction, time-frequency analysis as used in radar fall
son’s fall related signal using WiFi to the processing unit. detection. Although these approaches have these advantages
Li et al. [33] developed an acoustic FDS (acoustic-FADE) that of low cost, easy deployment, and passive device-free detec-
automatically detected a fall and reported it promptly to the tion, the abilities of motions monitoring and remote health
caregiver. Acoustic-FADE consisted of a circular microphone monitoring are limited.

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.
DING AND WANG: WiFi-BASED SMART HOME FALL DETECTION SYSTEM USING RECURRENT NEURAL NETWORK 311

Fig. 2. Technical diagram of fall detection.

III. S YSTEM D ESIGN respectively. Specifically, the threshold is able to be deter-

The basic idea of our system is to extract the detection mined by some preliminary measurements. Different from
data from the receiver in smart home and conduct fall recog- relying on scenario-tailored calibration, we do not need to cal-
nition on the data analysis platform. Meanwhile, the detection ibrate for each different case because the threshold can apply
result is sent to the client application. Fig. 2 shows the techni- to various indoor environments gracefully. More importantly,
cal diagram of fall detection. Note that, one important question the design of the environment change indicator is lightweight
for the correct operation of the proposed FDS is how to since the calculation of mean value is effective and fast.
know that one person is performing a motion in smart home 2) Data Collection: As discussed above, when receiving
or when the data is collected. This can help avoiding false a positive response “yes” from the designed environment
motion generated by other noise in indoor environments from change indicator, the system starts collecting the detection
nearby users and also leads to energy-efficiency. Our solution data. Concretely, when the WiFi signal f (tx ) satisfies (1), it
to that is to build the environment change indicator inferring can be thought that there is a motion occurring in the area of
potential motion interference. Based on its decision, if there interests. At the same time, the detection data will also be
is a response “yes”, the proposed FDS would start performing collected from this moment. Moreover, we can assume that
data collection and subsequent processes, otherwise continue an entity performs a motion indoors at time t1 . In view of
conducting motions monitoring. In view of this, the proposed this, with commercial WiFi Network Interface Cards (NICs)
system can be divided into two main modules, including and slight firmware modification, the detection data containing
hardware platform and client application. human motion information is collected with a predefined time
window where the window size is set as n, and we have
 
A. Hardware Platform Module f (t) = f (t1 ), f (t2 ), . . . f (tn ) , (2)
As shown in Fig. 2, the hardware platform module of the where f (t) has a dimension of 1 × n. Then, the collected
system is comprised of the following building blocks. data will be used for next data processing.
1) Environment Change Indicator: It is widely recognized 3) Data Processing: Since the collected data f (t) is
that the amplitude of WiFi signal would fluctuate remarkably obtained from smart home, it inevitably contains the random
when an entity moves within the area of interests, and remain noise from various sources, such as nearby electronic devices
relatively stable in the case of no motion interfering. Based interfering. This random noise is able to add false edges, and
on this basis, the environment change indicator is introduced then influence the accuracy of fall detection and the robustness
to exploit such motion-induced signal fluctuation for inferring in smart home. In order to improve system quality, we need
potential motion interference. It can help determining if the to remove the influence of the random noise presented in the
proposed FDS starts the data collection in the next step by the raw data first.
system’s feedback. Owing to the advantages of being computationally efficient
Concretely, the transmitter is set to transmit continuously (linear time complexity) to fit on mobile devices and making
WiFi signal to the receiver through multiple paths. At the same no particular consumptions with respect to the nature of data,
time, the WiFi CSI accessed from the receiver is timely trans- a wavelet based denoising strategy is introduced to eliminate
mitted to the data analysis platform, and then it is analyzed the random noise presented in the collected data. We divide
if there is an environment change caused by human motions. the denoising into three parts, including signal decomposition,
Based on this view above, the environment change indicator detail coefficients processing, and signal reconstruction. DWT
is built using the amplitude variation of CSI, it is given by can decompose the data into two terms: the approximation
coefficients and detail coefficients. Of the two coefficients,
|f (tx ) − μ| ≥ α, the former can describe the data shape and the latter can cap-
 
μ = m−1 · fp (t1 ) + fp (t2 ) + · · · + fp (tm ) , (1) ture both the random noise and fine data details. In addition,
this splitting is applied recursively a number of steps (i.e.,
where f (tx ) represents the amplitude value of WiFi CSI at levels), J, to the approximation coefficients only to obtain
a certain time and fp (t1 ), fp (t2 ), . . . fp (tm ) is the pre-collected finer details from the data. Finally, DWT produces a coarse
data in indoor environments, and α represents the correspond- approximation projection (scaling) coefficients α (J) , together
ing threshold to trigger the data collection for various motions, with a sequence of finer detail projection (wavelet) coefficients

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.
312 IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, VOL. 66, NO. 4, NOVEMBER 2020

Fig. 3. The raw data amplitude of six motions. (a) Jumping. (b) Walking. Fig. 4. The data amplitude of six motion after denoising. (a) Jumping.
(c) Bending. (d) Falling. (e) Standing up. (f) Lying. (b) Walking. (c) Bending. (d) Falling. (e) Standing up. (f) Lying.

β (1) , β (2) , . . . β (J) . The DWT coefficients in each level can be

calculated as
  
(J) (J) (J)
αk = f (tn ), gn−2J k = f (tn )gn−2J k , J ∈ Z,
n
n∈Z
  
(j) (j) (j)
βk = f (tn ), hn−2j k = f (tn )hn−2j k , j ∈ {1, 2, . . . J},
n
n∈Z
(3)
where f (tn ) is the n-th data of f (t), · represents the dot prod-
uct operation, g’s and h’s, also known as the wavelet basis, are
two sets of discrete orthogonal functions. The inverse DWT
is given by
 (J) (J)

J 
(j) (j)
f (tn ) = αk gn−2J k + βk hn−2j k . (4)
k∈Z j=1 k∈Z

To remove the random noise component while retaining suf-

ficient details for FDS, a soft threshold method is applied to the
detail coefficients. Concretely, we set the coefficients with very Fig. 5. The RNN model with the LSTM blocks. (a) Overall structure.
small absolute value to zero, and the coefficients with large (b) Detailed structure of the LSTM blocks at the p-th time step.
absolute value are decreased. Then, the detail coefficients after
estimation are utilized to reconstruct the detection data directly
as follows: model with the long short-term memory (LSTM) blocks where
x(p) and h(p) represent input vector and hidden layer at the p-th
 (J) (J)

J 
(l) (l) time step, respectively. Besides, the dimension of the hidden
f̂ (tn ) = αk gn−2J k + β̂k hn−2l k , (5)
layer is assumed to be Dh . In addition to the hidden layer, the
k∈Z l=1 k∈Z
LSTM blocks also have a unit, called cell state c(p) , which
(l)
where β̂k denotes estimated detail coefficients. Then, we can can control the information flow through using three gates,
obtain the denoised data f̂ (t). Besides, the raw data amplitude including forget, input, and output gates.
of six human motions in daily life is shown in Fig. 3 and With regard to the input vector, each gate vector can be
Fig. 4 shows the results after denoising. calculated as follows:
4) Motions Classification Based on RNN: In the present
f(p) = σg Wf x(p) + Uf h(p−1) + bf ,
study, the RNN model is introduced to classify human motions
and then identify the fall status. The first step is to create an i(p) = σg Wi x(p) + Ui h(p−1) + bi ,
appropriate input vector from the denoised data. Thus, the
input vector can be expressed by o(p) = σg Wo x(p) + Uo h(p−1) + bo , (7)
T T
x = f̂ (t1 ), f̂ (t2 ), . . . f̂ (tn ) = f̂ (t) , (6) where Wf , Wi , and Wo are the input weights and have
a dimension of Dh × Dx , Uf , Ui , and Uo are the cyclic weights
where the input vector has a dimension of Dx . and have a dimension of Dh × Dh . bf , bi , and bo are the bias
Then, we consider an RNN model taking P sets of the col- and have a dimension of Dh × 1. In addition, σg (·) denotes an
lected data to make a decision. Fig. 5 illustrates the RNN element-wise activation function for all gates, which is given

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.
DING AND WANG: WiFi-BASED SMART HOME FALL DETECTION SYSTEM USING RECURRENT NEURAL NETWORK 313

by using a sigmoid function, i.e., σg (z) = 1/(1 + e−z ). Both

the previous cell state and hidden layer of the LSTM blocks
at the first time step are initialized as zero vectors. Owing to
these vectors, the cell state and hidden layer in LSTM can be
updated in real time as follows:
Fig. 6. Mobile App architecture.
c(p) = f(p) ◦ c(p−1) + i(p) ◦ σc Wc x(p) + Uc h(p−1) + bc ,
(8) known that these optimizers can change adaptively the learning
rate to achieve the minimum cost precisely.
and we can have
Owing to containing many significant parameters, the
h(p) = o(p) ◦ σh c(p) , (9) proposed RNN model can capture non-linearity relationship
between the input and output layers efficiently, but it has a high
where ◦ represents the Hadamard product operator, σc (·) and risk of over-fitting. As a result, an early stopping is employed
σh (·) represent the element-wise activation functions for the in this study to avoid this key issue, and here we adopt
cell state and hidden layer, respectively. In our study, the Adam optimizer. In addition, the dataset of input and out-
hyperbolic tangent is leveraged for these activation functions. put pairs, ς , are separated into three non-overlapping datasets,
Finally, the RNN model output can be obtained directly from including training dataset ςtr , validation dataset ςv , and testing
the hidden layer at the last time step, and it is given by dataset ςte . Then, we train the proposed RNN model to deter-
mine all the model parameters based on the training dataset
h x(1) , . . . , x(P) = σ Vh(P) + b , (10) with true labels. At each iteration, these values of the cost
where V is the Dh dimension row vector, and b denotes function (11) with regard to the validation dataset are evalu-
a bias constant, and the set  contains every parameter of the ated and tracked, and we can select optimal parameters when
proposed RNN model, for instance, elements in the matrices the cost function is minimized.
U and W, vectors b and V, and the constant b.
Every parameter in the RNN model is able to be adjusted B. Client Application Module
appropriately by training motions dataset. The sequence of A cross-platform Integrated Development Environment
P input data vectors is denoted as X = (x(1) , . . . x(P) ) and (IDE) [35] is utilized to develop the front-end mobile user
the corresponding labels are denoted as y, where y can be interface. The advantage of using such an environment is that
viewed as walking, falling, etc. From the denoised data and it can leverage standard Web development languages. Besides,
corresponding motion labels, a set of input and output pairs it also ensures the cross-platform feature for application which
(X, y) are produced to train and verify the proposed RNN means that only one application is developed for different
model. Besides, ς is introduced as such a dataset, and every mobile phone platforms without the need for reimplementa-
parameter in the model can be adjusted in the direction of tion. In addition, the application takes use of two types of
minimizing the cost function, which is expressed by identity authentication. Specifically, the first one is that the
regular username-password combination is used to initiate con-
1 
J() = − C(g), (11) nection. Once the user is authenticated, a random generated
|ς | g∈ς string called API key is utilized to authenticate operations.
where | · | denotes the number of elements in a set, C(g) is This key is able to be changed anytime, and API key changes
the cost of the g-th input and output pair and it measures how per session. For example, when the user logs out, the API key
accuracy the proposed model generates the output compared is changed. The second one is that an additional parameter
to the ground truth data. called secret key is leveraged to make granting privilege, such
In order to evaluate the cost performance, cross-entropy is as making a user a community owner. This key changes daily,
introduced as weekly or monthly and is only to be known by the top-level
users. Fig. 6 displays the mobile App architecture showing
C(g) = y(g) log h X(g) + 1 − y(g) log 1 − h X(g) , the information flow from data analysis platform to end user
(12) application.

where the superscript indicates the index of the input and out- IV. E VALUATION
put pairs. Then, every parameter in the RNN model can be The proposed FDS has the capacity for making the fall dis-
updated in an iterative manner by taking use of the gradient tinguishable from the other motions, and telling whether the
descent method, which is given by elderly are having a fall or not in smart home efficiently. To
n+1 = n − η∇ J(), (13) validate the architecture of the proposed FDS, a prototype is
designed, built, and tested. Then, we conduct some real-world
where ∇ represents the gradient operation with regard to , experiments to evaluate the performance of the proposed FDS
and η is the learning rate which is set to be 5 × 10−4 . Many in three typical indoor environments, including laboratory,
variations of the gradient descent methods have been studied office room, and dormitory, and further show the generality of
widely in the previous works like Adam optimizer. It is well our system. Moreover, in order to verify the high performance,

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.
314 IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, VOL. 66, NO. 4, NOVEMBER 2020

the four state-of-the-art algorithms are evaluated and com-

pared, i.e., Hidden Markov Model (HMM) [19], Long Short-
Term Memory (LSTM) [21], Random Forest (RF) [36], [37],
and Support Vector Machine (SVM) [36], [38]. Besides,
we extend performance comparison with the results of
former research, including E-eyes [18], CARM [19], and
ABLSTM [21].

A. Dataset Collection
In order to collect motions dataset, a pair of commercial Fig. 7. The recognition accuracies for different human motions in three
WiFi devices running on 5 GHz with bandwidth channel of different indoor environments.
20 MHz are utilized, and the WiFi devices equipped with three
antennas act as a transmitter and a receiver, respectively. The
antennas in the WiFi devices are Omni-directional and run on better recognition performance. Among them, the motion of
2.4 GHz and 5 GHz frequency bands, and the gain is 8 dBi. “falling” yields the highest recognition accuracy. The reason
During the whole testing, the shape of the antennas remains is that this motion would generate a significant influence on
unchanged. Besides, in order to collect raw CSI data to obtain the characteristics of WiFi signal with distinct patterns com-
precise information of human motions, the data sample rate pared with the others. Another observation is that the motion
is set as 1 KHz. Based on this, we can access up to nine of “standing” has the lowest recognition accuracy. The possi-
transmission links where each of them has 30 subcarriers. ble reason is that this motion shares similar impact on WiFi
With WiFi NICs and slight firmware modification, the detec- signal with the motion of “bending”. Among the six motions,
tion data containing human motions information is able to be the motion of “falling” is of great importance, especially for
accessed at the receiver by leveraging CSI tool [39] installed the elderly living on their own. The proposed FDS is able
in the data analysis platform. In the present study, to make the to achieve recognition accuracies of 90%, 91%, and 93% for
dataset diverse, we collect human’s motions dataset in three the motion of “falling” in three typical indoor environments,
different indoor environments with regard to laboratory, office respectively.
room, and dormitory. Specifically, these testing scenarios con-
tain various furniture and electronic devices, which are no C. Comparative Evaluation
different from real residential units. During dataset collection,
the WiFi devices are installed on the metallic support in indoor The recognition accuracies for the existing state-of-the-art
environments, and the height of the support as well as the algorithms and the proposed FDS are shown in Fig. 8. As
distance between the supports can be adjusted appropriately it is indicated, the RF performs the worst and the SVM
according to the needs of these experiments. slightly outperforms the RF. In addition, the HMM has a supe-
Totally, ten people are involved for dataset collection rior performance compared with the RF and SVM. Since
with six common daily motions with regard to “jumping”, the LSTM network also considers the temporal dependencies
“walking”, “bending”, “falling”, “standing up”, and “lying”. in sequential data for feature learning, it achieves a better
Specially, each of all the testing objects is asked to perform performance than the HMM. Owing to the proposed effec-
each motion for a period of 6 seconds. Besides, each of them tive system architecture, the proposed FDS can achieve the
is also asked to perform each motion 30 times in the same test- best performance for the recognition of all the six motions.
ing area. In particular, all the testing objects need to remain Similarly, the recognition accuracies for different motions dif-
stationary at the beginning and the end of all the motions for fer from each other. Among all the motions, the motion of
reducing the random noise. “falling” can yield the highest recognition accuracy for all the
algorithms, which will benefit many fall detection applications.
B. Fall Detection
The recognition accuracies for all the six motions in three D. Results With Different Approaches
different indoor environments are shown in Fig. 7. It can be To validate its high effectiveness, we compare the proposed
found that the proposed FDS can achieve average accuracies FDS with E-eyes, CARM, and ABLSTM fairly leveraging the
of 82%, 85%, and 90% for all the motions, respectively. The data collected from the laboratory. Specially, our proposed
recognition accuracies in the laboratory perform the worst. In FDS takes use of the denoised data after DWT processing, E-
contrast, the recognition accuracies in the dormitory have the eyes uses the data after low-pass filtering, CARM utilizes the
best performance. The possible reason is that the laboratory speeds and spectrograms of different motions extracted from
area is much larger than that of the office room and dormitory, raw CSI data, and ABLSTM directly uses the raw CSI data as
which means that the laboratory can yield a more complicated input. The recognition accuracies for different approaches are
multiple environment. shown in Fig. 9. It can be observed that E-eyes performs the
The recognition accuracies for different motions have large worst and CARM slightly outperforms the E-eyes approach. In
difference. Especially, the human motions with large body addition, the deep learning based approach of ABLSTM con-
movement speed, i.e., “walking”, “falling”, and “lying”, have siders the temporal dependencies in sequential data for feature

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.
DING AND WANG: WiFi-BASED SMART HOME FALL DETECTION SYSTEM USING RECURRENT NEURAL NETWORK 315

Fig. 8. The recognition accuracies for different human motions with different
algorithms. (a) In the laboratory. (b) In the office room. (c) In the dormitory.

Fig. 10. Mobile App user interfaces. (a) Identity authentication. (b) Selected
functions. (c) Falling warning page. (d) No falling page.

Fig. 9. The recognition accuracies for different human motions with different V. M OBILE A PP I MPLEMENTATION
approaches in the laboratory.
The proposed FDS has been implemented as a mobile
App using open source technologies. Besides, the architec-
learning and assigns different weights for all the learned fea- ture of the mobile App is presented in Fig. 6. Specially, it
tures, so it can achieve a better performance compared with is divided into three parts, namely mobile App, proxy server,
E-eyes and CARM. The proposed FDS approach is able to and data analysis platform. The API is leveraged as a com-
achieve the best performance for all the motions. The rea- munication link to synchronize data among three parts. The
son is that the proposed FDS uses the deep learning to learn data analysis platform is the main processing unit where
the denoised data, instead of the raw CSI data. As a result, algorithms are implemented. It is also responsible for synchro-
it has strong robustness to the random noise derived from nization of the detection data between local and proxy server.
indoor environments. Similarly, the motion of “falling” yields In addition, the basic function of the proxy server is connec-
the highest recognition accuracy. In contrast, the recognition tion and data storage, which is the data transit station and
accuracies of “bending” and “standing” are still unsatisfied. responsible for forwarding legal network information. Finally,

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.
316 IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, VOL. 66, NO. 4, NOVEMBER 2020

by using the mobile App, the consumer can get important [3] H. Wang, D. Zhang, Y. Wang, J. Ma, Y. Wang, and S. Li, “RT-Fall:
information from the proxy server in a timely manner. The A real-time and contactless fall detection system with commodity WiFi
devices,” IEEE Trans. Mobile Comput., vol. 16, no. 2, pp. 511–526,
working of the mobile application is presented in Fig. 10. Feb. 2017.
Fig. 10(a) shows the identity authentication details page. After [4] C. Han, K. Wu, Y. Wang, and L. M. Ni, “WiFall: Device-free fall detec-
entering ID and password, the consumer can enter the mobile tion by wireless networks,” in Proc. IEEE INFOCOM Conf. Comput.
Commun., Toronto, ON, Canada, 2014, pp. 271–279.
application by clicking the button “Log in”. Once the con- [5] Z. A. Khan and W. Sohn, “Abnormal human activity recognition
sumer logs in, a service will run to get the consumer privileges system based on R-transform and kernel discriminant technique for
and the user interface components that he/she will be able to elderly home care,” IEEE Trans. Consum. Electron., vol. 57, no. 4,
pp. 1843–1850, Nov. 2011.
see consequently. [6] J. Wang, Z. Zhang, B. Li, S. Lee, and R. S. Sherratt, “An enhanced fall
Fig. 10(b) shows the corresponding functions list. For the detection system for elderly person monitoring using consumer home
consumer, there are two services available where the first one networks,” IEEE Trans. Consum. Electron., vol. 60, no. 1, pp. 23–29,
Feb. 2014.
is the motion type and the second one is the falling warning. [7] N. Dey, A. S. Ashour, F. Shi, S. J. Fong, and R. S. Sherratt, “Developing
By clicking on the first button, the motion type in terms of six residential wireless sensor networks for ECG healthcare monitoring,”
daily motions can be determined in smart home. Additionally, IEEE Trans. Consum. Electron., vol. 63, no. 4, pp. 442–449, Nov. 2017.
by clicking on the second button, the application is redi- [8] H.-T. Wu and C.-W. Tsai, “Toward blockchains for health-care systems:
Applying the bilinear pairing technology to ensure privacy protection
rected to detection decision pages as shown in Fig. 10(c) and and accuracy in data sharing,” IEEE Consum. Electron. Mag., vol. 7,
Fig. 10(d). Concretely, after completing the data analysis, the no. 4, pp. 65–71, Jul. 2018.
mobile App obtains the fall information and quickly displays [9] P. Sundaravadivel, E. Kougianos, S. P. Mohanty, and
M. K. Ganapathiraju, “Everything you wanted to know about
the results. Fig. 10(c) shows the warning prompt when a fall smart health care: Evaluating the different technologies and components
occurs in smart home. At the middle of this page, there will of the Internet of Things for better health,” IEEE Consum. Electron.
be a round graph of warning sign, which means that a fall is Mag., vol. 7, no. 1, pp. 18–28, Jan. 2018.
[10] W.-R. Yang, C.-S. Wang, and C.-P. Chen, “Motion-pattern recognition
occurring. Besides, there is a text prompt at the top left of this system using a wavelet-neural network,” IEEE Trans. Consum. Electron.,
page as well. As a result, the consumer needs to take emer- vol. 65, no. 2, pp. 170–178, May 2019.
gency measures to save the lives of the elderly. In contrast, [11] C. Wu, Z. Yang, Z. Zhou, X. Liu, Y. Liu, and J. Cao, “Non-invasive
detection of moving and stationary human with WiFi,” IEEE J. Sel.
when no fall occurs, there is a text prompt in terms of no Areas Commun., vol. 33, no. 11, pp. 2329–2342, Nov. 2015.
falling at the top left of this page as shown in Fig. 10(d). [12] T. Xin, B. Guo, Z. Wang, M. Li, Z. Yu, and X. Zhou, “FreeSense:
Indoor human identification with Wi-Fi signals,” in Proc. IEEE Global
Commun. Conf. (GLOBECOM), Washington, DC, USA, 2016, pp. 1–7.
VI. C ONCLUSION [13] X. Li, S. Li, D. Zhang, J. Xiong, Y. Wang, and H. Mei, “Dynamic-
MUSIC: Accurate device-free indoor localization,” in Proc. ACM Int.
This article presents a smart home FDS based on WiFi CSI Joint Conf. Pervasive Ubiquitous Comput. (UbiComp), NewYork, NY,
framework using RNN, and the system mainly consists of two USA, 2016, pp. 196–207.
modules in terms of hardware platform and client application. [14] Part 11: Wireless LAN Medium Access Control (MAC) and Physical
Layer (PHY) Specifications, IEEE Standard 802.11, Mar. 2012.
And these technologies, including commercial WiFi devices, [15] D. Halperin, W. Hu, A. Sheth, and D. Wetherall, “Predictable 802.11
data analysis platform, API, and consumer mobile App, are packet delivery from wireless channel measurements,” in Proc. ACM
combined together to implement the proposed FDS. SIGCOMM Conf., 2010, pp. 159–170.
[16] J. Zhang, B. Wei, W. Hu, and S. S. Kanhere, “WiFi-ID: Human iden-
Comprehensive experiments have been conducted on real- tification using WiFi signal,” in Proc. IEEE Int. Conf. Distrib. Comput.
world dataset taken from multiple typical indoor environments Sens. Syst. (DCOSS), Washington, DC, USA, 2016, pp. 75–82.
to evaluate the proposed FDS. Specially, the proposed FDS is [17] Y. Zeng, P. H. Pathak, and P. Mohapatra, “WiWho: WiFi-based person
identification in smart spaces,” in Proc. 15th Int. Conf. Inf. Process.
able to achieve recognition accuracies of 90%, 91%, and 93% Sens. Netw., Vienna, Austria, 2016, pp. 1–12.
for the human motion of “falling”, respectively. Furthermore, [18] Y. Wang, J. Liu, Y. Chen, M. Gruteser, J. Yang, and H. Liu, “E-eyes:
comparative evaluations we conduct show that the proposed Device-free location-oriented activity identification using fine-grained
FDS outperforms the existing detection algorithms. WiFi signatures,” in Proc. ACM 20th Annu. Int. Conf. Mobile Comput.
Netw. (Mobicom), Maui, Hawaii, USA, 2014, pp. 617–628.
Finally, the proposed FDS has been implemented as [19] W. Wang, A. X. Liu, M. Shahzad, K. Ling, and S. Lu, “Understanding
a mobile App for the consumers. The mobile App is able and modeling of WiFi signal based human activity recognition,” in Proc.
to provide a user-friendly way to monitor the fall status of ACM 21st Annu. Int. Conf. Mobile Comput. Netw. (Mobicom), Pairs,
France, 2015, pp. 65–76.
the elderly from a distance. Using their smartphones, they are [20] H. Abdelnasser, M. Youssef, and K. A. Harras, “WiGest: A ubiquitous
aware of the fall behavior and take emergency measures to WiFi-based gesture recognition system,” in Proc. IEEE Conf. Comput.
save the lives of the elderly in smart home. Future research will Commun., Kowloon, Hong Kong, 2015, pp. 1472–1480.
[21] Z. Chen, L. Zhang, C. Jiang, Z. Cao, and W. Cui, “WiFi CSI based
focus on multiple people, more motions, recognition accura- passive human activity recognition using attention based BLSTM,” IEEE
cies, and speeding up the total response time to further improve Trans. Mobile Comput., vol. 18, no. 11, pp. 2714–2724, Nov. 2019.
the user’s experience. [22] D. Zhang, H. Wang, Y. Wang, and J. Ma, “Anti-fall: A non-intrusive and
real-time fall detector leveraging CSI from commodity WiFi devices,” in
Proc. Int. Conf. Smart Homes Health Telematics, Geneva, Switzerland,
R EFERENCES 2015, pp. 181–193.
[23] Y. Wang, K. Wu, and L. M. Ni, “WiFall: Device-free fall detection
[1] Falls Among Older Adults: An Overview, Older Adult Falls, by wireless networks,” IEEE Trans. Mobile Comput., vol. 16, no. 2,
Atlanta, GA, USA, 2013. [Online]. Available: http://www.cdc.gov/ pp. 581–594, Feb. 2017.
HomeandRecreationalSafety/Falls/adultfalls.html/ [24] S. Palipana, D. Rojas, P. Agrawal, and D. Pesch, “FallDeFi: Ubiquitous
[2] E. Burns and R. Kakara, “Deaths from falls among persons aged ≥65 fall detection using commodity Wi-Fi devices,” Proc. ACM Interact.
years—United States, 2007-2016,” Morbidity Mortality Weekly Rep., Mobile Wearable Ubiquitous Technol., vol. 1, no. 4, pp. 155–280,
vol. 67, no. 18, pp. 509–514, May 2018. Dec. 2017.

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.
DING AND WANG: WiFi-BASED SMART HOME FALL DETECTION SYSTEM USING RECURRENT NEURAL NETWORK 317

[25] S. Sardy, P. Tseng, and A. Bruce, “Robust wavelet denoising,” IEEE [37] S. Yousefi, H. Narui, S. Dayal, S. Ermon, and S. Valaee, “A survey
Trans. Signal Process., vol. 49, no. 6, pp. 1146–1152, Jul. 2001. on behavior recognition using WiFi channel state information,” IEEE
[26] A. Murad and J.-Y. Pyun, “Deep recurrent neural networks for human Commun. Mag., vol. 55, no. 10, pp. 98–104, Oct. 2017.
activity recognition,” Sensors, vol. 17, no. 11, p. 2556, Nov. 2017. [38] Y. Kim and H. Ling, “Human activity classification based on micro-
[27] M. Inoue, S. Inoue, and T. Nishida, “Deep recurrent neural network doppler signatures using a support vector machine,” IEEE Trans. Geosci.
for mobile human activity recognition with high throughput,” Artif. Life Remote Sens., vol. 47, no. 5, pp. 1328–1337, May 2009.
Robot., vol. 23, no. 2, pp. 173–185, Jun. 2018. [39] D. Halperin, W. Hu, A. Sheth, and D. Wetherall, “Tool release: Gathering
[28] U. Md Zia, W. Khaksar, and J. Torresen, “Activity recognition using deep 802.11n traces with channel state information,” ACM SIGCOMM
recurrent neural network on translation and scale-invariant features,” Comput. Commun. Rev., vol. 41, no. 1, p. 53, Jan. 2011.
in Proc. IEEE Int. Conf. Image Process., Athens, Greece, 2018,
pp. 475–479.
[29] X. Yang and Y. Tian, “Super normal vector for human activity recog-
nition with depth cameras,” IEEE Trans. Pattern Anal. Mach. Intell., Jianyang Ding (Graduate Student Member,
vol. 39, no. 5, pp. 1028–1039, May 2017. IEEE) was born in Henan, China. He received
[30] O. D. Lara and M. A. Labrador, “A survey on human activity recognition the B.E. degree from Xidian University, Xi’an,
using wearable sensors,” IEEE Commun. Surveys Tuts., vol. 15, no. 3, China, in 2018, where he is currently pursuing the
pp. 1192–1209, 3rd Quart., 2013. Ph.D. degree in indoor positioning with the School
[31] K. Gunadi, L. Liliana, and J. Tjitrokusmo, “Fall detection application of Telecommunications Engineering. His current
using Kinect,” in Proc. Int. Conf. Soft Comput. Intell. Syst. Inform. research interests include wireless communication
Technol., Kuta, Indonesia, 2017, pp. 279–282. and signal processing.
[32] C. Nadee and K. Chamnongthai, “Multi sensor system for automatic fall
detection,” in Proc. Asia–Pac. Signal Inf. Process. Assoc. Annu. Summit
Conf. (APSIPA), Hong Kong, China, 2015, pp. 930–933.
[33] Y. Li, K. C. Ho, and M. Popescu, “A microphone array system for
automatic fall detection,” IEEE Trans. Biomed. Eng., vol. 59, no. 5,
pp. 1291–1301, May 2012.
[34] H. Rimminen, J. Lindstrom, M. Linnavuo, and R. Sepponen, “Detection
Yong Wang (Member, IEEE) was born in Shaanxi,
of falls among the elderly by a floor sensor using the electric near
China. He received the B.S., M.E., and Ph.D.
field,” IEEE Trans. Inf. Technol. Biomed., vol. 14, no. 6, pp. 1475–1476,
degrees from Xidian University, Xi’an, China, in
Nov. 2010.
1997, 2002, and 2005, respectively, where he is cur-
[35] A. R. Al-Ali, I. A. Zualkernan, M. Rashid, R. Gupta, and M. Alikarar, “A
rently a Professor with the State Key Laboratory of
smart home energy management system using IoT and big data analytics
Integrated Services Networks. His research interests
approach,” IEEE Trans. Consum. Electron., vol. 63, no. 4, pp. 426–434,
include full-duplex wireless communications and
Nov. 2017.
nonlinear predistorter technique.
[36] M. Afzaal, M. Usman, and A. Fong, “Tourism mobile app with aspect-
based sentiment classification framework for tourist reviews,” IEEE
Trans. Consum. Electron., vol. 65, no. 2, pp. 233–242, May 2019.

Authorized licensed use limited to: Carleton University. Downloaded on November 29,2020 at 16:16:50 UTC from IEEE Xplore. Restrictions apply.