ll

OPEN ACCESS

People of Data
Toward ubiquitous sensing: Researchers turn
WiFi signals into human activity patterns
Jianfei Yang1,* and Xinyan Chen1,*
1Nanyang Technological University, Singapore 639798, Singapore

*Correspondence: [email protected] (J.Y.), [email protected] (X.C.)

https://doi.org/10.1016/j.patter.2023.100707

Jianfei Yang, a principal investigator and postdoc at Nanyang Technological University (NTU), and his stu-
dent Xinyan Chen have developed a comprehensive benchmark and library for WiFi sensing. Their Patterns
paper highlights the advantages of deep learning for WiFi sensing and provides constructive suggestions on
model selection, learning scheme, and training strategy for developers and data scientists in this field. They
talk about their view of data science, their experience with interdisciplinary WiFi sensing research, and the
future of WiFi sensing applications.

What would you like to share about with mmWave radar. I also did a 6-month deep-learning and data science tech-
your background (personal and/or internship at Huawei Singapore research niques on WiFi CSI data to solve complex
professional)? center, where I applied deep-learning tasks in the real world. The key motivation
Jianfei Yang: Let me introduce myself techniques to identify non-line-of-sight for me to work in this field is that deep-
first. I was born in Binzhou, a quiet city ultra-wideband (UWB) data. I found my learning and data science techniques
in Northern China. Due to a large number great interest in research on the internet empower us with the ability to use radio-
of peer students, it is quite challenging to of things (IoT) and wireless sensing and frequency signals to tackle complex
get into a good university via a college decided to pursue further research in real-life challenges and provide a non-
entrance examination and find a path to this field. vision-based approach to solving tradi-
research. No doubt, I studied very hard tional computer-vision problems.
from 6 a.m. to 11 p.m. in high school What motivated you to become a Dr. Yang plays a significant role in guid-
and spent a wonderful time at Sun (data) researcher? Is there anyone/ ing me on my research path. He brought
Yat-Sen University for my undergraduate anything that helped guide you on me into this field and taught me various
study. My fate with research and your path? deep-learning and data science tech-
Singapore began in 2016 when I saw a JY: My interest in data science stemmed niques. He encouraged me to learn tech-
PhD opportunity from my PhD advisor, from my second year of undergraduate niques in different fields of deep learning
Prof. Xie Lihua. My advisor is an expert study when I met my first advisor, Prof. and gave me insights on how to apply
in control and robotics, and this position Chang-Dong Wang at Sun Yat-Sen Uni- these techniques to WiFi CSI data. His
aims at artificial intelligence (AI) of things versity. He is a data-mining scientist and optimistic attitude and passion for this
(AIoT), which is interdisciplinary. I feel has published many high-quality papers field always inspire me to explore new
lucky to meet my supervisor and have a on clustering algorithms. Though I was possibilities in our field.
very interesting research topic. After my quite young, he recruited me to join his
PhD study, I continue to work in interdisci- team. He discussed many interesting
plinary AI research, an AI-aided civil engi- topics in the data-mining field, which What is the definition of data
neering project at BEARS, UC Berkeley. motivated me to explore scientific science in your opinion? What is a
Currently, I am an independent principal research. The first topic we discussed data scientist? Do you self-identify
investigator (PI) at Nanyang Technolog- was how to build a data model for infor- as one?
ical University, conducting my own mation diffusion in social networks. He JY: In my opinion, data science is a new
research on multimodal learning for smart told me that information diffusion is like subject to study data properties and find
sensing. crowd flow in CCTV cameras that can specific ways to model the data. It can
Xinyan Chen: I am currently a final-year be captured and can be used to indicate be regarded as a new powerful tool for
undergraduate student at NTU pursuing a many characteristics of a social network, other disciplines, such as biology and
bachelor of engineering in infocommuni- such as activity level. Looking back on chemistry. It is seen that many papers in
cation engineering. I started the research this experience, I found great interest in bioinformatics (also published in Patterns)
work 3 years ago guided by Dr. Yang. Dur- data research and decided to continue adopt cutting-edge data science algo-
ing these 3 years, I applied deep-learning my PhD study after the undergraduate rithms and derive interesting findings.
approaches to realize large-scale WiFi period. Therefore, data scientists should be a
sensing, optimize supervised WiFi chan- XC: I was first introduced to this field group with wide range, coming from mul-
nel state information (CSI) classification when I was a sophomore. I attended an tiple disciplines. In this sense, I am also
performance, and estimate human pose undergraduate research program to apply regarded as a data scientist and propose

Patterns 4, March 10, 2023 ª 2023 The Authors. 1

This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
 ll
OPEN ACCESS People of Data

algorithms for WiFi sensing by exploring Why did you decide to publish in over the next 2–3 years. Data scientists
data patterns in WiFi data. patterns? What are your standards will gradually turn their tool from statistical
XC: In my view, data science explores for choosing journals? learning to a reasonable integration of
the inner logic within the data, which in- JY: Patterns is one of the most high-qual- both statistical and deep learning.
cludes but is not limited to the patterns, ity journals in the data science field. Nor- XC: With the development of IoT, the
the characteristics, and the information. mally, I publish my paper in specific- usage of smart devices has grown rapidly.
A data scientist works on the data to domain journals, such as IEEE and ACM, The growth of wireless communication
figure out how the inner logic of data and I never think about publishing it in a data has become exponential. Other
could be interpreted and applied to give scientific journal that includes a broader than the basic information communica-
innovative insights into real-life problems range of readers. Nevertheless, when I tion function, wireless communication
across various fields. To become a data read some papers in Patterns, I find that data also have some other abilities, such
scientist, people need to have an open they attract me a lot, despite different dis- as sensing the environment. Data science
mind and meticulous observation ability ciplines. Reading these papers benefits plays an important role in digging poten-
and to think in innovative ways. I regard me not only at the method level but also tial usage of wireless communication
myself as a beginner in data science in an intuitive way. I chose Patterns as it data such as WiFi CSI. I expect data sci-
and there is a long way to go and much has a broader impact, multidisciplinary ence could help to broaden the usage of
effort to be paid to become a data scope, and great editors who make the various kinds of wireless communication
scientist. review process smooth. I really enjoy data and help us use wireless communi-
this journey with Patterns. cation data to tackle more complex
sensing problems.
Aside from supervising research,
Did you encounter any particular
what do you expect from your
difficulties, or were there any A lot of data scientists continue
supervisors to help you develop into
specific challenges about data, data their career outside of academia;
a (data) scientist or researcher?
management, or FAIR data sharing what is your view on that?
XC: Aside from academic guidance, I
that you dealt with? How did you XC: Everyone has their reasons for their
would expect my supervisors to help me
overcome them? choices. In my view, data scientists who
develop the ability to think and solve
XC: In our work,1 we make use of four continue their careers in R&D depart-
problems independently. In the process
different open source datasets for com- ments in the industry will focus more on
of scientific research, it is unavoidable to
parison. However, different datasets the implementation of a project and
face challenges and predicaments. The
have their own organizing format, saving how a project could create benefits for
keys to overcoming these barriers are to
format, and reading format. Some of the company. While data scientists in
calmly analyze the problem and look at
them only provide raw data. For better academia will pay more attention to the
the problem with a positive attitude. I
use of these datasets, we reorganized innovation of the idea and the correctness
would also expect my supervisors to
the format of every dataset and per- of the theory. The purpose of data
help me develop teamwork abilities.
formed preprocessing on raw data. We science research is ultimately to benefit
Complicated work is hard for a person to
also wrote data reading scripts for every all mankind. Thus, our society not only
accomplish alone; it requires all team
processed dataset to make the them needs some data scientists to do
members put in their efforts and work
easy to read and easy to implement. ground-breaking work in academia but
together. Also, just as Shakespeare said,
also needs some data scientist to contin-
‘‘there are a thousand Hamlets in a thou-
What is the role of data science in uously improve existing works and imple-
sand people’s eyes’’—different people
your domain/field? What ment them in the real world as well.
have different views on the same prob-
advancements do you expect in
lem. It will be helpful to listen to others’
data science in this field over the What’s next for the project? What’s
suggestions in the research.
next 2–3 years? next for you?
JY: Data science serves as an important JY: My next project aims at fine-grained
Looking back, what advice would tool in WiFi sensing. The development of WiFi-based pose estimation for Meta-
you have given yourself at the start data science promotes WiFi sensing verse applications. Metaverse technology
of your career? Is there anything research significantly. From statistical maps human beings from the physical
you would have done differently? learning to deep learning, stronger and world to a virtual world for the purposes
JY: I am still an early-career investigator, larger models come into existence and of entertainment, healthcare, and work.
working as a PI for 2 years. I would advise enable better capacity to extract features To feed this requirement, cameras are
myself to imagine a bigger picture 2 years for higher-level tasks. For example, in my usually needed to sense human poses,
ago. The reason is that the research vision field, statistical features with traditional but due to privacy concerns, cameras
is the upper bound of a research team. classifiers (e.g., SVM) only support simple are not preferable in households and hos-
When I started my new role, I made my activity recognition, e.g., falling recogni- pitals. We are studying whether WiFi can
plan mainly including papers and fund- tion, while deep learning enables more be leveraged for human-pose estimation.
ings. However, these are only some complicated human activities. I think If so, in the future, the Metaverse world
result-level plans that should be guided there will emerge more tools for deep could be simulated by WiFi sensing.
and motivated by a bigger picture. learning in multidisciplinary data science With this project, I will focus on my

2 Patterns 4, March 10, 2023

 ll
People of Data OPEN ACCESS

toward civilian popularization and WiFi

sensing products will become a light-
weight, low-cost, more broadly appli-
cable, and more private solution for the
smart home.

How do you see the privacy issues

of WiFi sensing?
JY: This is a very interesting question. In
fact, compared to the most common
sensing tech in our daily life—cam-
era—WiFi sensing has been much
more privacy preserving, as we cannot
see a meaningful visible image in WiFi
data. However, based on our technol-
ogy, WiFi-based human-activity moni-
toring also arouses some privacy con-
cerns. People may worry their WiFi
data could be leaked, which means
that their daily routine can be obtained
by hackers. To this end, we are also
studying how to prevent such cases. I
have several solutions regarding this
Jianfei (right) and Xinyan (left) at their NTU laboratory, with wireless sensing devices on concern. Firstly, we can decrease the
their table. chances of data leakage by not upload-
ing these data to a cloud server. All
the WiFi sensing and analytics are
research and then try to transform it into seen by intruders even with specific tools,
performed in the edge devices. Sec-
commercial usage for future applications. and thus WiFi-based security applications
ondly, we can encrypt the sensing data
XC: In the next stage of our project, we provide a better tool for museums or pri-
and perform the data transmission.
will explore applying some more deep- vate households. Secondly, as WiFi-
Thirdly, we should establish rules or
learning techniques to WiFi sensing and based respiration detection is becoming
instructions to protect user privacy.
try to figure out how could deep-learning more and more accurate, home-based
Different choices of data protocols
models help WiFi sensing play to its health monitoring will be a very good
could be selected by users. Some pro-
strengths under poor environmental con- product. As far as I know, hundreds of
tocols strictly forbid developers and
ditions (e.g., non-line of sight, poor illumi- people die during sleep, and there is no
researchers to use their data, while
nance, and occlusion). I will continue my effective way to detect the sudden stop
some may support future research by
research with Dr. Yang in this field and of respiration. Wearable devices such as
providing their data.
pursue a PhD at NTU in the meantime. smart watches also help, but many
XC: The principle of WiFi sensing is that
elderlies do not like to wear these
CSI could describe the environmental in-
Your recent published papers in watches. WiFi infrastructures have been
fluence during the propagation of the
Patterns is about a WiFi sensing ubiquitous, and I think they will be the
wireless signal. And WiFi signals are in
framework. What are your thoughts cost-efficient yet effective choices for
the electromagnetic spectrum’s non-
on using WiFi sensing products in home-based respiration detection.
visible band. Therefore, WiFi CSI data
your daily life? XC: With the development of the
do not describe the details of private
JY: WiFi sensing products will come into internet era, more wireless devices need
surrounding information and personal in-
existence very soon. IEEE will release information exchange, which means that
formation in essence. Compared with
the new IEEE 802.11bf standardization more WiFi routers will be deployed in our
vision-based sensing approaches, WiFi
that officially includes WiFi sensing into daily life. Nowadays, scientists have
sensing approaches could better protect
the WiFi standard. In 5–10 years, it is ex- shown that WiFi sensing can be used for
users’ privacy and would be easier to
pected that off-the-shelf commodity occupancy detection, human activity
accept because WiFi routers have been
WiFi infrastructures will adopt the stan- recognition, person identification, hu-
widely accepted in home scenarios.
dard, and thus massive WiFi sensing man-pose estimation, and many other
data can be acquired. When it comes to tasks. In the future, the research work on
the fields of these products, I believe WiFi sensing will further explore its poten- REFERENCE
that security and healthcare are the two tial and make it more stable. Moreover, 1. Yang, J., Chen, X., Lu, C.X., Wang, D., Sun, S.,
most important markets. Firstly, WiFi can the WLAN sensing standard 802.11bf is and Xie, L. (2023). SenseFi: a library and bench-
mark on deep-learning-empowered WiFi human
help spot the intruder. Compared to the expected in 2024. By that time, WiFi
sensing. Patterns 4, 100703. https://doi.org/10.
infrared sensor, WiFi signals cannot be sensing technology will begin to develop 1016/j.patter.2023.100703.

Patterns 4, March 10, 2023 3

 ll
OPEN ACCESS People of Data
About the authors BEARS, University of California, Berkeley. His tronic Engineering, Nanyang Technological
Dr. Jianfei Yang is currently an independent prin- research interests include deep learning, wireless University, Singapore. His Undergraduate Re-
cipal investigator and presidential postdoctoral sensing, and artificial intelligence of things. He search Experience on Campus (URECA) Pro-
research fellow at Nanyang Technological University received the best PhD thesis award from NTU and gram and final year project (FYP) is under the
(NTU), Singapore. He received his bachelor’s de- has won many international AI challenges in com- supervision of Prof. Lihua Xie and Dr. Jianfei
gree in engineering from the School of Data and puter vision and interdisciplinary research fields. Yang. He is a scholar of the Ministry of Educa-
Computer Science, Sun Yat-Sen University, in tion (Singapore) under SM2 scholarship. His
2016, and his PhD degree from NTU in 2021. He Xinyan Chen is currently an undergraduate research interests include deep learning and
used to work as a senior research engineer at student at the School of Electrical and Elec- wireless sensing.

4 Patterns 4, March 10, 2023