CHAPTER 1

INTRODUCTION

1.1 OVERVIEW

The real-time detection of falling humans in natural environments using a deep neural network
(DNN) algorithm is a cutting-edge application addressing a critical aspect of public health and
safety. Falls, particularly among the elderly, can lead to severe consequences, necessitating
swift intervention for minimizing injuries. This research introduces an innovative system that
harnesses the capabilities of deep learning to accurately identify instances of falling in real-
time from video streams captured in diverse natural settings. The proposed solution employs a
custom-designed DNN architecture trained on a comprehensive dataset, ensuring robust
performance across various scenarios. Leveraging transfer learning techniques, the model
demonstrates high accuracy and generalization capabilities. The focus on efficient inference
facilitates deployment on edge devices, making it suitable for a wide range of applications,
from home environments to healthcare facilities and public spaces. The system's real-time
capabilities hold significant promise for enhancing proactive intervention and support,
ultimately contributing to the mitigation of fall-related risks and their associated impacts.

1.2 MACHINE LEARNING

Machine Learning is a system of computer algorithms that can learn from example through
self-improvement without being explicitly coded by a programmer. Machine learning is a part
of artificial Intelligence which combines data with statistical tools to predict an output which
can be used to make actionable insights. The breakthrough comes with the idea that a machine
can singularly learn from the data to produce accurate results. Machine learning is closely
related to data mining and Bayesian predictive modeling. The machine receives data as input
and uses an algorithm to formulate answers. A typical machine learning tasks are to provide a
recommendation. For those who have a Netflix account, all recommendations of movies or
series are based on the user's historical data. Tech companies are using unsupervised learning
to improve the user experience with personalizing recommendation. Machine learning is also
used for a variety of tasks like fraud detection, predictive maintenance, portfolio optimization,
automatize task and so on.

1
The way the machine learns is similar to the human being. Humans learn from experience. The
more we know, the more easily we can predict. By analogy, when we face an unknown
situation, the likelihood of success is lower than the known situation. Machines are trained the
same. To make an accurate prediction, the machine sees an example. When we give the
machine a similar example, it can figure out the outcome. However, like a human, if its feed a
previously unseen example, the machine has difficulties to predict. The core objective of
machine learning is the learning and inference. First of all, the machine learns through the
discovery of patterns. This discovery is made thanks to the data. One crucial part of the data
scientist is to choose carefully which data to provide to the machine. The list of attributes used
to solve a problem is called a feature vector. You can think of a feature vector as a subset of
data that is used to tackle a problem.

Fig. 1.1 Learning Phase in Machine Learning

The machine uses some fancy algorithms to simplify the reality and transform this discovery
into a model. Therefore, the learning stage is used to describe the data and summarize it into a
model.

1.2.1 MACHINE LEARNING VS TRADITIONAL PROGRAMMING

Traditional programming differs significantly from machine learning. In traditional

programming, a programmer code all the rules in consultation with an expert in the industry
for which software is being developed. Each rule is based on a logical foundation; the machine
will execute an output following the logical statement. When the system grows complex, more
rules need to be written. It can quickly become unsustainable to maintain.

2
 Fig 1.2 Flow of Traditional Programming

Machine learning is supposed to overcome this issue. The machine learns how the input and
output data are correlated and it writes a rule. The programmers do not need to write new rules
each time there is new data. The algorithms adapt in response to new data and experiences to
improve efficacy over time.

Fig 1.3 Flow of Machine Learning

1.2.2 DEEP LEARNING

Machine learning tools have been a core component of spatial analysis in GIS for decades. You
have been able to use machine learning in ArcGIS to perform image classification, enrich data
with clustering, or model spatial relationships. Machine learning is a branch of artificial
intelligence in which structured data is processed with an algorithm to solve a problem.
Traditional structured data requires a person to label the data, such as a picture of cats and dogs,
so that specific features for each animal type can be understood within the algorithm and used
to identify these animals in other pictures.

3
Deep learning is a subset of machine learning that uses several layers of algorithms in the form
of neural networks. Input data is analyzed through different layers of the network, with each
layer defining specific features and patterns in the data. For example, if you want to identify
features such as buildings and roads, the deep learning model can be trained with images of
different buildings and roads, processing the images through layers within the neural network,
and then finding the identifiers required to classify a building or road.

Fig 1.4 Subsets of Artificial Intelligence

Esri has developed tools and workflows to utilize the latest innovations in deep learning to
answer some of the challenging questions in GIS and remote sensing applications. Computer
vision, or the ability of computers to gain understanding from digital images or videos, is an
area that has been shifting from the traditional machine learning algorithms to deep learning
methods. Before applying deep learning to imagery in ArcGIS AllSource, it is important to
understand the different applications of deep learning for computer vision.

There are many computer vision tasks that can be accomplished with deep learning neural
networks. Esri has developed tools that allow you to perform image classification, object
detection, semantic segmentation, and instance segmentation. All of these computer vision

4
tasks are described below, each with a remote sensing example and a more general computer
vision example.

1.3 PROJECT OBJECTIVE

Our objective is to distinguish normal and abnormal human activity of a person falling in a
scene. The primary objective of the project is to develop an advanced and reliable system for
the automated, real-time detection of falls in diverse natural settings. A falling event is known
to consist of three sequential temporal phases: standing, falling and fallen. We add a fourth
phase, which we refer to as not moving. This is required because often in a real environment
there is no person within view of the camera or a person lies on the ground for a long time. We
do not know beforehand the time of the event occurrence. Neither do we know exactly how
long the event will take. However, it has been estimated that, in general, at least 150 frames
are required to ensure that all four phases have been observed. In light of this, we seek a method
for detecting falls by observing the four action phases in sequence.

1.4 MOTIVATION

The project motivation is to improve the public health and safety, particularly in the context of
an aging population. The real-time aspect of the proposed system is driven by the understanding
that the timely detection of falls is paramount for effective intervention to provide immediate
support, especially in situations where individuals may be alone or lack immediate help.
Traditional fall detection methods, including wearable devices and rule-based algorithms, often
have limitations in terms of accuracy, adaptability, and real-time processing. To explore a
DNN-based approach arises from the desire to overcome these limitations and introduce a more
sophisticated and effective solution that can operate seamlessly in diverse natural
environments. The motivation extends beyond reactive measures to a more proactive approach
to healthcare. By detecting falls in real-time, the system aims to prevent or minimize the
consequences of falls, contributing to a shift from treatment-oriented healthcare to preventative
and personalized care. This aligns with the broader goal of improving overall well-being and
quality of life.

5
1.5 ORGANIZATION OF THE REPORT

Introduction:

Background and Context: Introduce the significance of fall detection, especially in natural
environments, and highlight the increasing importance of leveraging deep neural networks for
real-time detection.

Motivation: Provide an overview of the motivations driving the development of the proposed
fall detection system.

Objectives: Clearly state the project objectives, outlining the specific goals and outcomes
expected.

Existing Work:

Review of Traditional Methods: Discuss the limitations of traditional fall detection methods,
such as wearable devices, rule-based algorithms, and depth sensors.

Advances in Deep Learning: Highlight the role of deep learning, particularly deep neural
networks, in addressing the shortcomings of traditional methods.

Comparative Analysis: Compare the proposed DNN-based approach with existing

technologies, emphasizing the advantages it offers in terms of accuracy, adaptability, and real-
time processing.

Proposed Work:

Custom DNN Architecture: Detail the design and architecture of the deep neural network
specifically developed for fall detection in natural environments.

Dataset Curation and Annotation: Explain the process of curating a diverse dataset and
annotating it with falling and non-falling instances to facilitate model training.

6
Transfer Learning Techniques: Describe the application of transfer learning to pre-train the
model on a general image recognition dataset and fine-tune it for fall detection.

Real-Time Optimization: Discuss the optimization strategies employed to ensure real-time

inference on edge devices, including quantization, model pruning, and hardware acceleration.

Environmental Adaptability: Present mechanisms implemented to address environmental

variability and enhance the system's resilience to changes in natural settings.

Performance Analysis of Proposed Algorithm:

Evaluation Metrics: Define and explain the metrics used for the performance evaluation,
including accuracy, precision, recall, and F1 score.

Experimental Setup: Outline the settings in which the fall detection system was tested,
including controlled laboratory environments and real-world natural scenarios.

Results and Findings: Present the quantitative and qualitative results of the system's
performance, comparing them against benchmarks and discussing any notable observations.

Validation: Discuss the validation process, emphasizing the robustness and generalization
capabilities of the proposed algorithm in real-world conditions.

Conclusion:

Summary of Achievements: Recap the key achievements and contributions of the project,
emphasizing how the proposed algorithm addresses the challenges of real-time fall detection
in natural environments.

Limitations: Acknowledge any limitations or areas for improvement identified during the
project.

7
Future Work: Propose potential avenues for future research and enhancements to the fall
detection system.

Overall Impact: Conclude by discussing the potential impact of the proposed work on public
health, safety, and the broader field of fall detection technology.

By following this organizational structure, the report provides a comprehensive and coherent
narrative, guiding the reader through the introduction, background, methodology, results, and
implications of the real-time fall detection system using a deep neural network algorithm in
natural environments.

8
 CHAPTER 2

LITERATURE SURVEY

2.1 DEEP LEARNING FOR VISION-BASED FALL DETECTION

SYSTEM: ENHANCED OPTICAL DYNAMIC FLOW

Accurate fall detection for the assistance of older people is crucial to reduce incidents of deaths
or injuries due to falls. Meanwhile, a vision-based fall detection system has shown some
significant results to detect falls. Still, numerous challenges need to be resolved. The impact of
deep learning has changed the landscape of the vision-based system, such as action recognition.
The deep learning technique has not been successfully implemented in vision-based fall
detection systems due to the requirement of a large amount of computation power and the
requirement of a large amount of sample training data. This research aims to propose a vision-
based fall detection system that improves the accuracy of fall detection in some complex
environments such as the change of light condition in the room. Also, this research aims to
increase the performance of the pre-processing of video images. The proposed system consists
of the Enhanced Dynamic Optical Flow technique that encodes the temporal data of optical
flow videos by the method of rank pooling, which thereby improves the processing time of fall
detection and improves the classification accuracy in dynamic lighting conditions. The
experimental results showed that the classification accuracy of the fall detection improved by
around 3% and the processing time by 40 to 50ms. The proposed system concentrates on
decreasing the processing time of fall detection and improving classification accuracy.
Meanwhile, it provides a mechanism for summarizing a video into a single image by using a
dynamic optical flow technique, which helps to increase the performance of image pre-
processing.

TECHNIQUES USED

 Optical Flow Analysis

 Temporal Analysis
 Sensor Fusion

9
LIMITATIONS

The accuracy of fall detection becomes critical in some complex environments such as the
change of light condition in the room. It takes more time for processing and require high
processing power.

2.2 VISION-BASED FALL DETECTION WITH CONVOLUTIONAL

NEURAL NETWORKS

One of the biggest challenges in modern societies is the improvement of healthy aging and the
support to older persons in their daily activities. In particular, given its social and economic
impact, the automatic detection of falls has attracted considerable attention in the computer
vision and pattern recognition communities. Although the approaches based on wearable
sensors have provided high detection rates, some of the potential users are reluctant to wear
them and thus their use is not yet normalized. As a consequence, alternative approaches such
as vision-based methods have emerged. We firmly believe that the irruption of the Smart
Environments and the Internet of Things paradigms, together with the increasing number of
cameras in our daily environment, forms an optimal context for vision-based systems.
Consequently, here we propose a vision-based solution using Convolutional Neural Networks
to decide if a sequence of frames contains a person falling. To model the video motion and
make the system scenario independent, we use optical flow images as input to the networks
followed by a novel three-step training phase. Furthermore, our method is evaluated in three
public datasets achieving the state-of-the-art results in all three of them.

TECHNIQUES USED

 Transfer Learning
 Temporal Analysis
 Feature Extraction
 Frame Extraction

LIMITATIONS

Using optical flow images provides a great representational power for motion, but also involves
the heavy computational burden of preprocessing consecutive frames and drawbacks

10
concerning lighting changes. Following the philosophy of end-to-end learning, we would like
to avoid any image preprocessing step and work only on raw images in the future. Therefore,
more complex network architectures will have to be designed to learn complete and
hierarchical motion representations from raw images. As the public datasets have only one
actor per video we believe that the next step in the field of fall detection would be the multi
person fall detection

2.3 FALL DETECTION USING STANDOFF RADAR-BASED SENSING

AND DEEP CONVOLUTIONAL NEURAL NETWORK

Automatic fall detection using radar aids in better assisted living and smarter health care. In
this brief, a novel time series-based method for detecting fall incidents in human daily activities
is proposed. A time series in the slow-time is obtained by summing all the range bins
corresponding to fast-time of the ultra wideband radar return signals. This time series is used
as input to the proposed deep convolutional neural network for automatic feature extraction. In
contrast to other existing methods, the proposed fall detection method relies on multi-level
feature learning directly from the radar time series signals. In particular, the proposed method
utilizes a deep convolutional neural network for automating feature extraction as well as global
maximum pooling technique for enhancing model discriminability. The performance of the
proposed method is compared with that of the state-of-the-art, such as recurrent neural network,
multi-layer perceptron, and dynamic time warping techniques. The results demonstrate that the
proposed fall detection method outperforms the other methods in terms of higher accuracy,
precision, sensitivity, and specificity values. Human fall is one of the very critical health issues,
especially for elders and disabled people living alone. The number of elder populations is
increasing steadily worldwide. Therefore, human fall detection is becoming an effective
technique for assistive living for those people. For assistive living, deep learning and computer
vision have been used largely. In this review article, we discuss deep learning (DL)-based state-
of-the-arts non-intrusive (vision-based) fall detection techniques. We also present a survey on
fall detection benchmark datasets. For a clear understanding, we briefly discuss direct metrics
which are used to evaluate the performance of the fall detection systems. This article also gives
a future direction on vision-based human fall detection techniques.

11
TECHNIQUES USED

 Radar Based Sensing

 Multi-Layer Perceptron
 Dynamic Time Warping

LIMITATIONS

Monitoring indiviudals using radar technology may raise privacy concerns, especially if the
system is deployed in private spaces. It is difficult to create a universal model that covers all
possible scenarios in this approach and the cost of deployment of radar based sensing system
is high.

2.4 VISION-BASED HUMAN ACTION CLASSIFICATION USING

ADAPTIVE BOOSTING ALGORITHM

Precise recognition of human action is a key enabler for the development of many applications,
including autonomous robots for medical diagnosis and surveillance of elderly people in home
environment. This paper addresses the human action recognition based on variation in body
shape. Specifically, we divide the human body into five partitions that correspond to five partial
occupancy areas. For each frame, we calculated area ratios and used them as input data for
recognition stage. Here, we consider six classes of activities namely: walking, standing,
bending, lying, squatting, and sitting. In this paper, we proposed an efficient human action
recognition scheme, which takes advantages of the superior discrimination capacity of adaptive
boosting algorithm. We validated the effectiveness of this approach by using experimental data
from two publicly available databases fall detection databases from the University of
Rzeszow's and the Universidad de Málaga fall detection data sets. We provided comparisons
of the proposed approach with the state-of-the-art classifiers based on the neural network, K-
nearest neighbour, support vector machine, and naïve Bayes and showed that we achieve better
results in discriminating human gestures.

TECHNIQUES USED

 Adaptive Boosting
 Multiple Kernel Learning

12
  E Gaussian Background Subtraction
 Spatial-Temporal Features

LIMITATIONS

The system may struggle to accurately classify actions when there are occlusions or partial
visibility of the human body. Poor image quality, low resolution, or variations in lighting
conditions can affect the algorithm’s performance and it becomes computationally expensive,
particularly if a large number of weak classifiers are used.

2.5 ELDERLY FALL DETECTION USING WEARABLE SENSORS: A

LOW COST HIGHLY ACCURATE ALGORITHM

Every year, more than 37 million falls that require medical attention occur. The elderly suffers
the greatest number of fatal falls. Therefore, automatic fall detection for the elderly is one of
the most important health-care applications as it enables timely medical intervention. The fall
detection problem has extensively been studied over the last decade. However, since the
hardware resources of wearable devices are limited, designing highly accurate embeddable
algorithms with feasible computational cost is still an open research challenge. In this project,
a low-cost highly accurate machine learning-based fall detection algorithm is proposed.
Particularly, a novel online feature extraction method that efficiently employs the time
characteristics of falls is proposed. In addition, a novel design of a machine learning-based
system is proposed to achieve the best accuracy/numerical complexity tradeoff. The low
computational cost of the proposed algorithm not only enables to embed it in a wearable sensor
but also makes the power requirements quite low and hence enhances the autonomy of the
wearable device, where the need for battery recharge/replace is minimized. Experimental
results on a large open dataset show that the accuracy of the proposed algorithm exceeds 99.9%
with a computational cost of less than 500 floating point operations per second.

TECHNIQUES USED

 Accelerometer and gyroscope data fusion

 Threshold-based algorithm
 Feature extraction

13
LIMITATIONS

The most widely used solution is to design simple threshold based fall detection algorithms.
These algorithms compare the acceleration signals on x, y and z axes (or some quantity derived
from these signals) with predefined threshold. Some of them also use time thresholds to take
advantage of the time characteristics of falls. However, the low computational cost of
threshold-based algorithms is at the expense of the accuracy , i.e. huge number of false alarms
are generated by these algorithms if the threshold is too low and, on the other hand, a
considerable number of falls will not be detected if the threshold is too high.

2.6 DETECTION OF THE HUMAN FALLS ON FURNITURE USING

SCENE ANALYSIS BASED ON DEEP LEARNING AND ACTIVITY
CHARACTERISTICS

Automatic human fall detection is one important research topic in caring for vulnerable people,
such as elders at home and patients in medical places. Over the past decade, numerous methods
aiming at solving the problem were proposed. However, the existing methods only focus on
detecting human themselves and cannot work effectively in complicated environments,
especially for the falls on furniture. To alleviate this problem, a new method for human fall
detection on furniture using scene analysis based on deep learning and activity characteristics
is presented in this paper. The proposed method first performs scene analysis using a deep
learning method faster R-CNN to detect human and furniture. Meanwhile, the space relation
between human and furniture is detected. The activity characteristics of the detected people,
such as human shape aspect ratio, centroid, motion speed are detected and tracked. Through
measuring the changes of these characteristics and judging the relations between the people
and furniture nearby, the falls on furniture can be effectively detected. Experiment results
demonstrated that our approach not only accurately and effectively detected falls on furniture,
such as sofa and chairs but also distinguished them from other fall-like activities, such as sitting
or lying down, while the existing methods have difficulties to handle these. In our experiments,
our algorithm achieved 94.44% precision, 94.95% recall, and 95.50% accuracy. The proposed
method can be potentially used and integrated as a medical assistance in health care and
medical places and appliances.

14
TECHNIQUES USED

 3D Convolutional Networks
 Multi-Modal Sensor Fusion
 Contextual Analysis
 Recurrent Neural Networks

LIMITATIONS

The methods using pressure, vibration and other ambient signals are very sensitive to external
factors and hence have poor anti-noise capability. These approaches are confined to where the
sensors are installed. Besides, wearing these auxiliary equipment may be inconvenient and
uncomfortable for people. The accuracy of the fall detection is highly dependent on the quality
and effectiveness of scene analysis, including the extraction of relevant features and context.

2.7 FALL DETECTION IN VIDEOS WITH TRAJECTORY-WEIGHTED

DEEP-CONVOLUTIONAL RANK-POOLING

Automatic fall detection in videos could enable timely delivery of medical service to the injured
elders who have fallen and live alone. Deep ConvNets have been used to detect fall actions.
However, there still remain problems in deep video representations for fall detection. First,
video frames are directly inputted to deep ConvNets. The visual features of human actions may
be interfered with surrounding environments. Second, redundant frames increase the difficulty
of time encoding for human actions. To address these problems, this paper presents trajectory-
weighted deep-convolutional rank-pooling descriptor (TDRD) for fall detection, which is
robust to surrounding environments and can describe the dynamics of human actions in long
time videos effectively. First, CNN feature map of each frame is extracted through a deep
ConvNet. Then, we present a new kind of trajectory attention map which is built with improved
dense trajectories to optimally localize the subject area. Next, the CNN feature map of each
frame is weighted with its corresponding trajectory attention map to get trajectory-weighted
convolutional visual feature of human region. Further, we propose a cluster pooling method to
reduce the redundancy of the trajectory-weighted convolutional features of a video in the time
sequence. Finally, rank pooling method is used to encode the dynamic of the cluster-pooled

15
sequence to get our TDRD. With TDRD, we get superior result on SDU Fall dataset and get
comparable performances on UR dataset and Multiple cameras dataset with SVM classifiers.

TECHNIQUES USED

 Trajectory-Weighted Deep Convolutional Networks

 Rank-Pooling
 Temporal Modelling
 Trajectory Attention Map

LIMITATIONS

Trajectory-weighted deep convolutional rank-pooling may be computationally intensive,

potentially limited real-time deployment or use on resource constrained devices. The model’s
effectiveness may be limited when deployed in enviroments significantly different from the
ones present in the training dataset. Achieving high accuracy may require a large and diverse
dataset of fall events, which may be challenging to obtain and the inaccuracies in trajectory
estimation can lead to false positives or negatives.

16
 CHAPTER 3

EXISTING SYSTEM

3.1 OVERVIEW

Accidental falls are a major source of loss of autonomy, deaths, and injuries among the elderly.
Accidental falls also have a remarkable impact on the costs of national health systems. Thus,
extensive research and development of fall detection and rescue systems are a necessity.
Technologies related to fall detection should be reliable and effective to ensure a proper
response. This article provides a comprehensive review on state-of-the-art fall detection
technologies considering the most powerful deep learning methodologies. We reviewed the
most recent and effective deep learning methods for fall detection and categorized them into
three categories: Convolutional Neural Network (CNN) based systems, Long Short-Term
Memory (LSTM) based systems, and Auto-encoder based systems. Among the reviewed
systems, three dimensional (3D) CNN, CNN with 10-fold cross-validation, LSTM with CNN
based systems performed the best in terms of accuracy, sensitivity, specificity, etc. The
reviewed systems were compared based on their working principles, used deep learning
methods, used datasets, performance metrics, etc. This review is aimed at presenting a
summary and comparison of existing state-of-the-art deep learning based fall detection systems
to facilitate future development in this eld.

3.2 EXISTING WORK

A person falling is potentially dangerous no matter what the circumstances. This situation may
be caused by a sudden heart attack, a violent event, or perhaps group panic bought about by a
terrorist attack. Amongst all kinds of such circumstances, an accidental fall of an elderly person
is taken more seriously because it may cause dangerous injuries or even death. According to
some statistics from the US, 2.5 million elderly people are treated annually for falling injuries
in hospital emergency departments and approximately one-sixth of these die from these injuries
every year. In light of this, there is a great need for an inexpensive real-time system to
automatically detect a falling person and then raise an alarm.

Existing approaches for detecting a falling person that can be found in the literature may
broadly be divided into two groups those that use a variety of non-visual sensors (the most

17
common are accelerometers and gyroscopes) and those exclusively vision-based. Generally
speaking, the former require subjects to actively cooperate by wearing the sensors, which can
be problematic and possibly uncomfortable. On the other hand, vision-based methods are less
intrusive, as all information is collected remotely using video cameras. Fall detection systems
use a combination of non-visual sensors, including accelerometers and gyroscopes, to monitor
and analyze movements in order to detect and respond to potential falls. These sensors play a
crucial role in understanding the dynamics of human motion and can provide valuable data for
fall detection algorithms. Let's delve into the details of how accelerometers and gyroscopes
contribute to fall detection.

3.3 NON-VISUAL SENSORS

The fall detection problem has extensively been studied over the last decade. However, since
the hardware resources of wearable devices are limited, designing highly accurate embeddable
algorithms with feasible computational cost is still an open research challenge. In this paper, a
low-cost highly accurate machine learning-based fall detection algorithm is proposed.
Particularly, a novel online feature extraction method that efficiently employs the time
characteristics of falls is proposed. In addition, a novel design of a machine learning-based
system is proposed to achieve the best accuracy/numerical complexity tradeoff. The low
computational cost of the proposed algorithm not only enables to embed it in a wearable sensor
but also makes the power requirements quite low and hence enhances the autonomy of the
wearable device, where the need for battery recharge/replace is minimized.

3.3.1 ACCELEROMETERS

Accelerometers play a pivotal role in fall detection systems, serving as key sensors that measure
changes in acceleration to identify sudden movements indicative of a fall. In the context of fall
detection, a commonly used type of accelerometer is the MEMS (Micro-Electro-Mechanical
Systems) accelerometer. MEMS accelerometers are small, lightweight, and capable of
accurately measuring accelerations along multiple axes. They are integrated into wearable
devices, smart home systems, and other monitoring solutions. The purpose of using
accelerometers in fall detection lies in their ability to capture variations in motion that occur
during a fall event. Typically, during routine activities, a person experiences a constant
gravitational force acting on the accelerometer, allowing it to measure a standard acceleration
value. In the event of a fall, the sudden deviation from this baseline, both in terms of intensity
18
and direction, triggers the accelerometer to detect the abnormal motion patterns associated with
falling. This data is then processed by fall detection algorithms, often in conjunction with other
sensors like gyroscopes, to accurately discern between normal activities and potential falls,
enabling timely and effective response mechanisms. When a person is stationary or moving at
a constant velocity, the accelerometer records the acceleration due to gravity (1g) in the
direction opposite to the Earth's gravitational pull. This is typically along the vertical axis.
During a fall or sudden movement, the accelerometer detects deviations from the normal
gravitational acceleration. Rapid changes in acceleration, especially along the vertical axis, can
be indicative of a fall. Algorithms analyze the acceleration data in real-time to identify patterns
associated with falls, considering factors such as the intensity, duration, and direction of the
acceleration.

Fig. 3.1 Accelerometer Data indicating the fall

3.3.2 GYROSCOPES

Gyroscopes are integral components in fall detection systems, contributing valuable

Information about angular motion and changes in orientation. In fall detection applications,
Micro-Electro-Mechanical Systems (MEMS) gyroscopes are commonly employed due to their
compact size, low power consumption, and sensitivity to angular velocity: These gyroscopes
measure the rate of rotation around specific axes, providing critical data to complement the

19
information gathered from accelerometers. The primary purpose of incorporating gyroscopes
in fall detection systems is to distinguish between intentional movements and accidental falls.
While accelerometers capture linear motion and can identify changes in velocity, gyroscopes
focus on rotational aspects of movement. During a fall, there is a rapid change in orientation,
and the gyroscope detects these angular changes, enhancing the system's ability to discern
between activities like sitting down or bending over and an actual fall event. By combining
accelerometer and gyroscope data, fall detection algorithms can create a more comprehensive
and accurate analysis of motion patterns, improving the system's overall reliability in
identifying potential falls and reducing false positives. This Integration enhances the
effectiveness of fall detection solutions, especially in wearable devices and smart environments
where real-time monitoring is crucial.

Fig. 3.2 Gyroscopes Data indicating the fall

3.4 VISUAL SENSORS

Vision-based approaches in fall detection leverage visual sensors such as cameras or depth
sensors to monitor and interpret human movements for the identification of potential falls.
These systems analyze video or depth data to extract relevant features related to body posture,
motion patterns, and spatial relationships within the environment. Computer vision algorithms
are then applied to recognize distinctive characteristics associated with a fall, such as sudden

20
changes in posture, the orientation of body parts, or the lack of expected movements. Depth
sensors provide additional depth information, enabling better understanding of the three-
dimensional aspects of the scene. Machine learning techniques, including neural networks and
classifiers, are often employed to train the system on a diverse dataset, enabling it to generalize
and recognize patterns indicative of falls. Vision-based approaches offer the advantage of non-
intrusive monitoring, eliminating the need for wearable sensors. However, challenges include
issues related to privacy, lighting conditions, and occlusions that may obstruct the view of the
camera. Continuous advancements in computer vision, machine learning, and sensor
technologies contribute to improving the accuracy and reliability of vision-based fall detection
systems, making them viable solutions in various environments such as homes, healthcare
facilities, or assisted living spaces.

Fig. 3.3 Vision based Approach

3.4.1 DEPTH SENSOR

Depth sensors, commonly employed in fall detection systems, are devices that provide
information about the distance of objects from the sensor, enabling the creation of a three-
dimensional representation of the environment. One type of depth sensor often used is the time-
of-flight (ToF) camera. ToF cameras emit infrared light pulses and measure the time it takes
for the light to travel to objects and back, allowing for precise depth calculations. Another type
is the structured light camera, which projects a pattern of light onto the scene and analyzes the
deformations in the pattern caused by objects at different distances. The purpose of
incorporating depth sensors in fall detection lies in their ability to enhance the spatial
understanding of the monitored area.

21
By accurately measuring distances, depth sensors contribute to the creation of a detailed 3D
map, aiding in the identification of human body positions, postures, and movements. In fall
detection scenarios, depth sensors can detect the height from which a person falls, the impact
with the floor, and subsequent movements, providing valuable contextual information to the
fall detection algorithms. This depth information complements data from other sensors like
accelerometers and gyroscopes, enabling a more comprehensive and accurate assessment of
potential fall events. The use of depth sensors in fall detection systems is particularly beneficial
in scenarios where traditional cameras alone might struggle, such as low-light conditions or
situations with complex backgrounds, as depth sensors operate independently of ambient
lighting and can provide reliable depth information even in challenging environments.

3.4.2 PRESSURE SENSOR

Pressure sensors are devices that measure the force applied to a surface, and they find
application in fall detection systems to capture information about sudden impacts or changes
in pressure associated with a fall. One type of pressure sensor often used in this context is the
piezoelectric sensor. Piezoelectric sensors generate an electrical charge in response to
mechanical stress, making them sensitive to pressure changes. The purpose of incorporating
pressure sensors in fall detection is to detect the impact force when a person falls, particularly
when they make contact with the ground. The sensor can be strategically placed on the floor or
on furniture surfaces, allowing it to register the rapid increase in pressure during a fall event.
This information provides an additional layer of data that can be valuable for fall detection
algorithms, enhancing their ability to distinguish between normal activities and potentially
harmful incidents.

Pressure sensors contribute to a more comprehensive understanding of the fall dynamics by

capturing the physical interaction between the person and the environment. Integrating pressure
sensor data with other sensor inputs, such as accelerometers and gyroscopes, facilitates a more
accurate and reliable assessment of falls, especially in scenarios where impact forces play a
significant role. The use of pressure sensors in fall detection systems is particularly relevant in
environments where individuals may not be wearing wearable devices, and floor-mounted
sensors offer an unobtrusive means of monitoring and ensuring the safety of individuals in
various settings, including homes and healthcare facilities.

22
3.5 DEMERITS OF EXISTING WORK

The existing work on real-time detection of falling humans in natural environments using deep
neural network algorithms has made significant strides; however, it is essential to recognize its
demerits and areas for improvement. Some of the notable demerits include:

Limited Adaptability to Natural Environments:

Existing methods may struggle to adapt to the diverse and dynamic conditions present in
natural environments. Challenges such as uneven terrain, changing lighting, and complex
backgrounds can impact the accuracy of fall detection.

Dependency on Wearable Devices:

Some traditional fall detection methods rely on wearable devices, which may not be well-
received by users due to discomfort, non-compliance, or limitations in coverage. This
dependency hinders widespread adoption, especially among populations resistant to using such
devices.

Rule-Based Approaches:

Rule-based fall detection approaches often rely on predefined thresholds or criteria, making
them less adaptive to the variability in human movements. These methods may result in high
false positives or negatives, reducing overall accuracy.

Limited Real-Time Processing:

Many existing systems may face challenges in achieving real-time processing, particularly
when deployed on resource-constrained edge devices. Delays in fall detection may lead to
slower response times and reduced effectiveness in preventing fall-related injuries.

Privacy Concerns with Camera-Based Systems:

Camera-based fall detection systems, while effective, may raise privacy concerns, especially
when deployed in private spaces like homes. Balancing the need for surveillance with
individual privacy expectations remains a challenge.

23
Interference from Non-Fall Activities:

Certain existing systems may be susceptible to interference from non-fall activities that share
similar motion characteristics, leading to false alarms. Discriminating between genuine falls
and activities with similar motion patterns remains a challenging aspect.

3.6 SUMMARY

In summary, the project on "Real-Time Detection of Falling Humans in Natural Environments

Using Deep Neural Network Algorithm" aims to address the pressing public health concern of
falls, particularly among the elderly, by leveraging advanced technology. The proposed system
introduces a custom-designed deep neural network (DNN) algorithm specifically tailored for
real-time fall detection in diverse and dynamic natural settings. The project begins with a
thorough examination of the demerits of existing work, highlighting limitations in adaptability,
dependency on wearable devices, reliance on rule-based approaches, challenges in achieving
real-time processing, and privacy concerns. These shortcomings underscore the need for a more
sophisticated and adaptable solution. To overcome these limitations, the project outlines a
comprehensive plan, starting with the development of a custom DNN architecture. This
architecture is designed to capture complex patterns associated with falling human activities,
providing a more accurate and adaptable solution compared to traditional methods. The dataset
used for training is carefully curated to reflect the diversity of natural environments, ensuring
the model's robustness in real-world scenarios. Transfer learning techniques are employed to
pre-train the DNN on a general image recognition dataset, followed by fine-tuning on a fall
detection dataset. This approach enhances the model's ability to recognize intricate patterns
indicative of falls while leveraging knowledge gained from broader image recognition tasks.

24
 CHAPTER 4

PROPOSED SYSTEM

4.1 OVERVIEW

The proposed work, "FALLNET: A Deep Neural Network for Real-Time Fall Detection in
Natural Environments," presents a groundbreaking approach to the critical challenge of
detecting falling humans in uncontrolled and dynamic settings. By leveraging the capabilities
of deep neural networks, the system autonomously learns intricate patterns associated with
falling activities, ensuring high accuracy and adaptability. The custom-designed architecture
integrates convolutional, recurrent, and fully connected layers to capture both spatial and
temporal features essential for precise fall detection. A diverse dataset covering various
scenarios within natural environments is curated, and transfer learning techniques, involving
pre-training on a large-scale image recognition dataset followed by fine-tuning on the fall
detection dataset, enhance the model's task-specific understanding. Real-time processing is a
priority, with optimization techniques like quantization and model pruning, along with
hardware acceleration, facilitating swift inference on edge devices with limited computational
resources. Advanced computer vision techniques ensure adaptability to environmental
variations, including changes in lighting conditions. FALLNET features a robust fall detection
decision mechanism, real-time alerting, and a user-friendly interface for monitoring and
configuration. It significantly contribute to public health and safety, particularly in
environments characterized by natural variability and unpredictability.

Fig. 4.1 Working Flow of Proposed System

25
4.2 TECHNOLOGY USED IN THIS PROJECT

The real-time detection of falling humans in natural environments using a deep neural network
(DNN) algorithm incorporates a sophisticated blend of technologies to achieve robust and
efficient performance. At the core of this project is the utilization of deep learning, a subfield
of artificial intelligence, which has shown remarkable success in complex pattern recognition
tasks. The DNN algorithm, the key technological driver, relies on a neural network with
multiple layers, enabling it to automatically learn hierarchical representations from the input
data, in this case, video frames capturing human activities.

One pivotal aspect of the technology employed in this project is the custom-designed deep
neural network architecture. Tailored to the specific requirements of fall detection in natural
environments, the architecture is crafted to effectively capture and analyze intricate patterns
associated with falling humans. The architecture encompasses convolutional layers for spatial
feature extraction, recurrent layers for temporal dependencies, and fully connected layers for
high-level abstraction. This combination ensures that the model can discern subtle nuances in
body movements indicative of a fall, even in complex and dynamic natural settings.

The training process of the deep neural network involves exposing the model to a diverse and
well-annotated dataset. This dataset includes a wide array of scenarios, backgrounds, and
lighting conditions commonly encountered in natural environments. The diversity in the dataset
is essential for the model to generalize effectively, allowing it to perform reliably across a
spectrum of real-world conditions. Annotated videos depicting instances of falling and non-
falling activities serve as the foundation for the model to learn and differentiate between these
states.

Transfer learning, another critical technological strategy, is employed to enhance the model's
generalization capabilities. Pre-training the neural network on a large dataset related to general
image recognition tasks enables the model to capture generic features and patterns.
Subsequently, fine-tuning the model on the specific fall detection dataset refines its
understanding of human movements, optimizing its performance for the targeted application.
This approach is particularly valuable in scenarios where collecting a massive amount of
labeled data for a specific task might be challenging.

26
The hardware on which the algorithm runs is a crucial component of the technology stack. The
project is designed to be hardware-agnostic, capable of running on a variety of edge devices,
including but not limited to cameras, sensors, and embedded systems. This flexibility in
hardware compatibility increases the adaptability of the fall detection system, allowing it to be
integrated into existing infrastructure without requiring significant hardware upgrades.

The real-time nature of the fall detection system is achieved through parallel processing and
optimization techniques. Parallel processing involves breaking down the computational tasks
into smaller subtasks that can be executed simultaneously, enhancing the overall processing
speed. Additionally, the algorithm leverages parallelism in hardware, such as multi-core
processors and graphical processing units (GPUs), to further accelerate the inference speed.
These parallelization strategies contribute to the system's ability to analyze video streams in
real-time, enabling swift and timely detection of falling humans.

Furthermore, the deployment of the fall detection system in natural environments necessitates
robust handling of environmental challenges, such as varying lighting conditions, weather
effects, and occlusions. Advanced computer vision techniques, including image pre-
processing, adaptive filtering, and background subtraction, are integrated into the system to
address these challenges. These techniques enhance the model's resilience to environmental
variations, ensuring consistent and reliable performance in real-world scenarios. We propose a
two-stage approach for fall detection and locating the temporal extent of the fall. In the
detection stage, the untrimmed video is automatically decomposed into a sequence of video
clips and converted to multiple dynamic images.

Using a deep ConvNet, the dynamic images are scored and classified as falling or not by a so-
called “standing watch” for a situation consisting of the four phases (standing, falling, fallen
and not moving) in sequence. Then, in order to determine the temporal extent of the fall, we
introduce a difference scoring method (DSM) of adjacent dynamic images. We evaluate the
effectiveness of our solution on several of the most widely used public fall detection datasets:
the multiple cameras fall dataset , the high quality fall simulation dataset and the Le2i fall
detection dataset. However, realizing that the existing public fall detection datasets were
recorded in unrealistically controlled environments and the class of non-fall activities is very
limited, we created a new fall dataset for testing, called the YouTube Fall Dataset (YTFD).
This dataset was collected on YouTube, and consists of 430 falling incidents and 176 normal
activities.
27
4.3 ARCHITECTURE

Fig. 4.2 Architecture of Real-Time Fall Detection

28
The useful features are highlighted inside the broken blue line. The limitations in the system
are highlighted inside the broken red border. The system was proposed by Núñez-Marcos et
al.. They used deep CNN to decide if a video contains a person falling or not. This approach
uses optical flow images as an input to the deep network. However, the optical flow images
ignore any appearance-related features such as color, contrast, and brightness. The proposed
approach minimizes the hand-crafted image processing steps by using CNN. CNN can learn a
set of features and improved the performance when enough examples are provided during the
training phase. However, the proposed system has been made more generic .Núñez-Marcos et
al. presented a vision-based fall detection system using a CNN, which applies transfer learning
from the action recognition domain to fall detection. Three different public datasets were used
to evaluate the proposed approach. This model consists of two main stages, as shown in Fig. 1:
Pre-processing stage, and feature extraction, and classification stage.

4.3.1 ADVANTAGES

The real-time detection of falling humans in natural environments using a deep neural network
(DNN) algorithm offers several distinct advantages that contribute to its significance and
potential impact on public health and safety. Firstly, the utilization of deep neural networks
brings about a notable improvement in accuracy compared to traditional fall detection methods.
The inherent ability of DNNs to automatically learn complex patterns and hierarchical features
from diverse datasets enhances the system's capacity to discern nuanced movements associated
with falls, thereby minimizing false positives and negatives. This heightened accuracy is
crucial for ensuring the reliability of the fall detection system in real-world scenarios.

Secondly, the real-time nature of the proposed system is a significant advantage in addressing
the time-sensitive nature of fall-related incidents. The swift identification of a fall enables
immediate response mechanisms, whether it be alerting caregivers, initiating automated
emergency services, or providing timely assistance. In situations where individuals may be
alone or lack immediate help, the real-time capability becomes a critical factor in reducing the
consequences of falls, such as injuries and prolonged immobility.

Another advantage lies in the adaptability of the system to diverse natural environments. The
custom-designed deep neural network architecture, trained on a comprehensive dataset that
reflects various scenarios and backgrounds, ensures that the model can generalize effectively.
This adaptability is essential for addressing the challenges posed by uneven terrains, changing

29
lighting conditions, and unpredictable factors encountered in everyday life. The system's ability
to operate seamlessly in natural settings distinguishes it from traditional methods that may
struggle with the complexity of dynamic environments.

Additionally, the incorporation of transfer learning enhances the generalization capabilities of

the system. By pre-training the neural network on a large dataset related to general image
recognition tasks and subsequently fine-tuning it on the specific fall detection dataset, the
model captures both generic and task-specific features. This approach is particularly
advantageous in scenarios where collecting a massive amount of labeled data for a specific task
might be challenging. Transfer learning not only accelerates the training process but also
contributes to the robustness and efficiency of the fall detection system.

The deployment of the algorithm on edge devices represents another notable advantage. Edge
computing enables local processing on devices, reducing dependence on centralized servers
and mitigating latency issues. This approach not only enhances the speed of fall detection but
also makes the system more practical and scalable for widespread deployment. Edge computing
is especially relevant in situations where real-time processing is crucial, and access to
centralized computing resources may be limited or impractical.

Furthermore, the proposed system contributes to reducing the economic burden associated with
fall-related injuries. By providing timely detection and intervention, the system has the
potential to minimize healthcare costs, including hospitalizations, rehabilitation, and long-term
care. The economic advantages of preventing or mitigating the impact of falls extend beyond
individual well-being to societal benefits, alleviating strain on healthcare systems and
contributing to overall economic resilience.

4.3.2 COMPARISON WITH OTHER TECHNOLOGIES

The real-time detection of falling humans in natural environments using a deep neural network
(DNN) algorithm represents a significant advancement in comparison to other existing
technologies for fall detection. Traditional methods, including wearable devices, depth sensors,
and rule-based algorithms, exhibit limitations in terms of accuracy, adaptability, and real-time
processing. A comparative analysis highlights the distinctive advantages that the proposed
DNN-based approach brings to the forefront.

30
Firstly, compared to wearable devices, which are commonly used for fall detection, the DNN-
based system offers a non-intrusive and comprehensive solution. Wearable devices, such as
accelerometers or smartwatches, rely on motion sensors attached to the body, which may lead
to discomfort, non-compliance, or limitations in coverage. In contrast, the DNN-based
approach utilizes computer vision to analyze video streams, eliminating the need for
individuals to wear specific devices. This not only enhances user comfort but also ensures that
falls are detected regardless of the individual's adherence to wearing a device.

Depth sensors, another prevalent technology for fall detection, often struggle in natural
environments due to their sensitivity to lighting conditions and occlusions. The DNN-based
system, by customizing its architecture to handle diverse scenarios and leveraging transfer
learning, demonstrates superior adaptability to varying lighting conditions and environmental
complexities. The ability to discern falls accurately in dynamic and uncontrolled settings
distinguishes the DNN-based approach from depth sensors, which may face challenges in
natural, everyday environments.

Rule-based algorithms, commonly employed in traditional fall detection systems, lack the
sophistication and adaptability inherent in deep neural networks. Rule-based approaches
typically rely on predefined criteria or thresholds to identify falls, making them less capable of
handling the inherent variability in human movements. The DNN-based system, with its ability
to learn hierarchical features automatically, excels in capturing complex patterns associated
with falls, leading to heightened accuracy and reduced false positives or negatives compared
to rule-based methods.

Moreover, the real-time processing capabilities of the DNN-based system contribute to its
superiority over many existing technologies. In comparison to batch processing approaches or
systems heavily reliant on centralized servers, the proposed system's edge computing
capabilities enable local and immediate analysis of video streams. This real-time aspect is
crucial in situations where prompt intervention is essential, providing a notable advantage over
technologies that may introduce delays due to data transfer or processing bottlenecks.

Additionally, the DNN-based approach outperforms traditional methods in terms of scalability

and deployment flexibility. The ability to optimize the algorithm for edge devices ensures that
the fall detection system can be easily integrated into existing infrastructure without

31
necessitating significant hardware upgrades. This stands in contrast to some conventional
methods that may require specialized, resource-intensive equipment.

4.4 SUMMARY

The proposed FALLNET system represents a groundbreaking innovation in real-time detection

of falling humans in natural environments. Its custom-designed deep neural network (DNN)
integrates convolutional, recurrent, and fully connected layers, providing a robust foundation
for capturing both spatial and temporal features crucial for precise fall detection. The DNN
undergoes transfer learning for adaptability, pre- training on a vast image recognition dataset
and fine-tuning on a curated fall detection dataset. To optimize real-time processing,
FALLNET incorporates quantization, model pruning, and hardware acceleration while
addressing environmental variations through advanced computer vision techniques. A robust
decision mechanism with predefined thresholds minimizes false positives and negatives. In
case of a detected fall, FALLNET triggers a swift real-time alerting mechanism. The user
interface is designed for intuitive monitoring and configuration. Privacy is a priority, with
privacy-preserving measures implemented for ethical deployment. An optional continuous
learning mechanism allows the system to evolve and improve over time, adapting to changing
scenarios. With the implementation of privacy-preserving measures ensuring ethical
deployment, especially in private spaces. The optional continuous learning mechanism allows
the system to evolve and improve its performance over time, adapting to changing scenarios.
In essence, FALLNET stands as a comprehensive and adaptive solution, poised to redefine the
landscape of fall detection technology in natural environments. By seamlessly integrating
cutting-edge deep learning techniques, optimization strategies, and user-friendly interfaces,
FALLNET holds the potential to significantly impact public health and safety, particularly in
the context of an aging population and dynamic natural settings.

32
 CHAPTER 5

SYSTEM SPECIFICATION

5.1 PYTHON

Python is a high-level, general-purpose programming language that has gained immense

popularity due to its simplicity, readability, and versatility. Created by Guido van Rossum and
first released in 1991, Python emphasizes code readability and focuses on providing a clear and
concise syntax, making it easier for developers to write and understand code.

One of Python's key strengths is its vast ecosystem of packages and libraries Python offers a
rich collection of standard libraries for tasks ranging from file manipulation and networking to
web development and data analysis. Moreover, the Python Package Index (PyPI) hosts
thousands of open-source third-party packages. allowing developers to leverage pre-built
functionalities and accelerate their development process. These packages cover various
domains such as scientific computing (NumPy. SciPy), data analysis (Pandas, Matplotlib),
machine learning (Scikit-learn, TensorFlow, PyTorch), web development (Django, Flask),
natural language processing (NLTK. SpaCy), and more.

Python's versatility is evident in its applicability across different domains. It serves as an

excellent choice for web development, with frameworks like Django and Flask offering robust
solutions for building scalable and efficient web applications. For scientific computing and
data analysis, packages such as NumPy and Pandas provide efficient data structures and
powerful tools for handling and analyzing large datasets. In the field of artificial intelligence,
Python's extensive machine learning libraries enable developers to create sophisticated models
and algorithms.

In summary, Python's combination of simplicity, readability, versatility, and a vast ecosystem

of packages has made it a popular choice among developers for a wide range of applications.
Whether it's web development, data analysis, machine learning, or scientific computing.
Python provides the necessary tools and resources to streamline development and deliver
efficient and scalable solutions. Its vibrant community. extensive documentation, and ease of
learning make it an excellent language for beginners and experienced programmers alike.

33
5.2 PYCHARM

PyCharm is an integrated development environment (IDE) used for programming in Python. It

provides code analysis, a graphical debugger, an integrated unit tester, integration with version
control systems, and supports web development with Django. PyCharm is developed by the
Czech company JetBrains. Coding assistance and analysis, with code completion, syntax and
error highlighting, linter integration, and quick fixes

 Project and code navigation: specialized project views, file structure views and quick
jumping between files, classes, methods and usages
 Python code refactoring: including rename, extract method, introduce variable,
introduce constant, pull up, push down and others
 Support for web frameworks: Django, web2py and Flask
 Integrated Python debugger
 Integrated unit testing, with line-by-line coverage
 Google App Engine Python development
 Version control integration: unified user interface for Mercurial, Git, Subversion,
Perforce and CVS with change lists and merge

PyCharm is a dedicated Python Integrated Development Environment (IDE) providing a wide

range of essential tools for Python developers, tightly integrated to create a convenient
environment for productive Python, web, and data science development. PyCharm is a most
excellent tool for increasing the productivity of Python developers. Is is an effectively
integrated development environment. The autocomplete syntax feature helps a lot and saves a
lot of time, reducing turnaround time.

The Python IDE for professional developers

PyCharm is a Python IDE with complete set of tools for Python development. In addition. the
IDE provides capabilities for professional Web development using the Django framework.
Code faster and with more easily in a smart and configurable editor with code completion,
snippets, code folding and split windows support.

34
PyCharm Features

Intelligent Coding Assistance: PyCharm provides smart code completion, code inspections,
on-the-fly error highlighting and quick-fixes, along with automated code refactorings and rich
navigation capabilities.

Intelligent Code Editor: PyCharm's smart code editor provides first-class support for Python,
JavaScript, CoffeeScript, TypeScript, CSS, popular template languages and more. Take
advantage of language-aware code completion, error detection, and on-the-fly code fixes!

Smart Code Navigation: Use smart search to jump to any class, file or symbol, or even any
IDE action or tool window. It only takes one click to switch to the declaration, super method,
test, usages, implementation, and more.

Fast and Safe Refactorings: Refactor your code the intelligent way, with safe Rename and
Delete, Extract Method, Introduce Variable, Inline Variable orMethod, and other refactorings.
Language and framework-specific refactorings help you perform project-wide changes.

Built-in Developer Tools: PyCharm's huge collection of tools out of the box includes an
integrated debugger and test runner, Python profiler, a built-in terminal; integration with major
VCS and built-in database tools; remote development capabilities with remote interpreters; an
integrated ssh terminal: and integration with Docker and Vagrant.

VCS, Deployment and Remote Development: Save time with a unified UI for working with
Git, SVN, Mercurial or other version control systems. Run and debug your application on
remote machines. Easily configure automatic deployment to a remote host or VM and manage
your infrastructure with Vagrant and Docker.

Web Development: In addition to Python, PyCharm provides first-class support for various
Python web development frameworks, specific template languages, JavaScript, CoffeeScript,
TypeScript, HTML/CSS, AngularJS, Node.js, and more.

35
5.3 PACKAGES

Earlier, Machine Learning tasks by were manually performed by coding all the algorithms and
mathematical and statistical formulas. This made the processing time-consuming, tedious, and
inefficient. With Python, it is easier and more efficient compared with various libraries,
frameworks, and modules available. Real-time fall detection involves using various tools and
technologies to identify and respond to instances where individual experience a fall. The
packages mentioned, namely NumPy, pandas, TensorFlow, Pytorch and Matplotlib, play
different role in the development and implementation of fall detection system. The Python
libraries that are used in this project are described below

5.3.1 NUMPY

NumPy, or Numerical Python, is an indispensable library in real-time fall detection, offering

efficient tools for the processing and manipulation of numerical data, typically obtained from
sensors like accelerometers and gyroscopes. Its N-dimensional array ('ndarray') serves as a
versatile data structure for the representation of time-series data, capturing the dynamic
movements recorded by sensors. NumPy's rich set of array manipulation functions is essential
for reshaping, slicing, and indexing sensor data, supporting the intricate operations required by
fall detection algorithms. The library's support for vectorized operations enables the application
of functions across entire arrays, enhancing the speed and efficiency of computations, a crucial
aspect in real-time scenarios.

Fall detection often involves mathematical operations on sensor data, and NumPy provides an
extensive collection of mathematical functions optimized for array-wise calculations. Signal
processing, a fundamental step in fall detection, benefits from NumPy's integration with SciPy,
offering an array of functions for filtering, convolution, and Fourier transforms. The
broadcasting feature simplifies operations on arrays of different shapes, streamlining the
alignment of heterogeneous sensor data. NumPy seamlessly integrates with machine learning
libraries like scikit- learn, TensorFlow, and PyTorch, facilitating the training and deployment
of models for pattern recognition on sensor data. In essence, NumPy's efficiency, flexibility,
and integration capabilities make it a cornerstone in the development of accurate and
responsive real-time fall detection systems. NumPy is a fundamental building block in real-
time fall detection systems, providing efficient and flexible tools for data representation,
manipulation, and mathematical operations on sensor data. Its performance benefits, combined
36
with its integration with other scientific computing and machine learning libraries, make
NumPy an essential component in developing accurate and responsive fall detection
algorithms.

5.3.2 PANDAS

Pandas is the robust data manipulation and analysis library in Python, is instrumental in real-
time fall detection for its diverse functionalities in handling and preprocessing sensor data. Its
center piece, the DataFrame, serves as a versatile two-dimensional labeled data structure,
enabling the organized representation of time-series data collected from various sensors.
Crucially, Pandas facilitates data cleaning processes, allowing for the removal of duplicates,
handling missing values, and addressing outliers that might compromise the accuracy of fall
detection algorithms. With powerful indexing and slicing operations, Pandas supports the
extraction of specific segments or features within the sensor data, pivotal for the nuanced
analysis required in fall detection systems. The library's robust support for time-series data
empowers developers to perform temporal analysis, resampling, and rolling window
operations, uncovering relevant patterns indicative of falls. Additionally, Pandas aids in
efficient data filtering, enabling the isolation of specific events or patterns critical for fall
detection.

In real-time fall detection, Pandas plays a key role in preprocessing sensor data. It simplifies
tasks such as data cleaning, indexing, and filtering, allowing for a more structured and
organized representation of the input data. This makes it easier to apply machine learning
algorithms to detect fall patterns in a systematic and efficient manner. Feature engineering, an
integral aspect of fall detection algorithm development, is made accessible through Pandas,
allowing for the creation and transformation of variables. Seamless integration with other
Python libraries commonly used in fall detection, such as NumPy and scikit-learn, streamlines
the overall preprocessing pipeline. In summary, Pandas serves as a fundamental tool for
organizing, cleaning, and preprocessing sensor data in real-time fall detection systems. Its
capabilities in data manipulation, time-series analysis, and integration with other libraries
contribute to the development of accurate and reliable fall detection algorithms

37
5.3.3 MATPLOTLIB

Matplotlib is the comprehensive 2D plotting library in Python, plays a pivotal role in real- time
fall detection through its diverse applications in data visualization and analysis. Firstly,
Matplotlib allows for the creation of static and dynamic visualizations, enabling developers to
gain insights into patterns and anomalies within sensor data collected from accelerometers or
gyroscopes. Beyond data exploration, it serves as a crucial tool for visualizing the output of
machine learning models integrated into fall detection systems, offering plots of predicted
probabilities, decision boundaries, or confusion matrices. Its flexibility and extensive plotting
options contribute to the development, monitoring, and optimization of accurate and reliable
fall detection systems. Additionally, Matplotlib facilitates the visualization of evaluation
metrics, such as accuracy and precision, providing a comprehensive understanding of the
system's performance. The library's ability to generate real-time plots aids in continuous
monitoring during system development and testing, allowing developers to observe responses
to different scenarios. Matplotlib serves as a versatile tool in real-time fall detection, providing
capabilities for visualizing sensor data, model outputs, and evaluation metrics.

In real-time fall detection, Matplotlib can also be employed for generating alarms upon fall
detection, enhancing the user interface of the system. Furthermore, its utility extends to the
debugging and optimization phases, where developers can use visualizations to identify areas
for refinement within fall detection algorithms. The support for interactive visualizations,
including zooming and panning enhances the detailed examination of specific data segments
or model outputs. Lastly Matplotlib's capability to create professional-looking figures makes it
a valuable tool for presenting and reporting the performance and findings of real-time fall
detection systems. Overall, Matplotlib's flexibility and extensive plotting features contribute
significantly to the development, monitoring, and optimization of accurate and reliable fall
detection systems

5.3.4 PYTORCH

PyTorch is a dynamic deep learning framework for Python, plays a pivotal role in real- time
fall detection by offering a comprehensive set of tools for model development, training, and
deployment. Developers utilize PyTorch's flexibility to design intricate neural network
architectures tailored for fall detection tasks, allowing the models to capture nuanced patterns
in sensor data. The framework excels in training deep learning models on sizable datasets, a
38
critical aspect for improving the model's ability to discern patterns associated with falls.
PyTorch's support for recurrent neural networks (RNNs) is particularly advantageous for
processing time-series data, enabling the model to comprehend temporal dependencies in
sensor readings. Additionally, PyTorch facilitates the implementation of convolutional neural
networks (CNNs), beneficial for spatial feature extraction when dealing with visual data from
cameras or depth sensors. The framework's seamless integration with GPUs accelerates model
training and inference, enhancing the computational speed for real- time predictions in fall
detection. Furthermore, PyTorch supports transfer learning, enabling the utilization of pre-
trained models and easing the training process, especially when labeled fall detection data is
limited. PyTorch empowers real-time fall detection systems through its capabilities in neural
network development, training, and deployment. Its support for RNNs, CNNs, transfer
learning, GPU acceleration, and model interpretability makes PyTorch a versatile and effective
framework for building accurate and responsive fall detection models

In real-time fall detection, Pytorch is used to implement and train neural networks that learn
patterns indicative of falls from sensor data. The framework's compatibility with edge devices
allows for the deployment of trained models in wearable devices or smart environments,
ensuring immediate response and integration into real-world scenarios. PyTorch's commitment
to model interpretability provides developers with tools to understand and interpret neural
network decisions, enhancing transparency in real-time fall detection systems.

5.3.5 OPENPOSE

OpenPose is a cutting-edge library for real-time multi-person keypoint detection and pose
estimation. While the primary implementation is in C++, there are Python bindings and
wrappers available to facilitate the usage of OpenPose in Python. It provides a comprehensive
set of tools for detection and analyzing human body poses in images and videos. It proves
instrumental in real-time fall detection systems through various key features. Primarily,
OpenPose excels in accurately estimating the human body's pose by detecting key body parts,
including the head, shoulders, elbows, hips, knees, and ankles. This capability extends to
tracking keypoints across frames, enabling the understanding of dynamic human movement
over time, a crucial aspect for identifying unusual or abrupt movements indicative of a fall.

OpenPose contributes to fall detection algorithms by detecting and recognizing fall-related

movements or changes in posture. The Integration of OpenPose with deep learning models

39
enhances fall detection capabilities, as the keypoint information serves as valuable input
features for learning complex patterns associated with falls. The real-time processing capability
of OpenPose ensures quick and accurate analysis, providing timely detection and Intervention.
Moreover, OpenPose's ability to handle multi-person detection makes it suitable for
environments with multiple individuals, addressing scenarios where Interactions or events
involving more than one person need consideration. Finally, OpenPose's visualization tools aid
in rendering keypoint information, facilitating debugging and validation of fall detection
algorithms based on pose analysis. In essence, OpenPose emerges as a powerful tool,
contributing significantly to the development of accurate and responsive real-time fall detection
systems through its advanced pose estimation and keypoint tracking capabilities

40
 CHAPTER 6

MODULES DESCRIPTION

6.1 LIST OF MODULES

 Data Preprocessing
 Feature Extraction
 Model Training

6.1.1 DATA PREPROCESSING

The data preprocessing module in a real-time fall detection system plays a crucial role in
preparing and refining the input data before it is fed into the fall detection algorithm. The
objective of this module is to enhance the quality of the data, reduce noise, and extract relevant
features that are essential for accurate and reliable fall detection. Here are some key
components and steps typically involved in the data preprocessing module for real-time fall
detection.

Data Acquistion:

In the realm of real-time fall detection, the initial step involves data acquisition, where sensor
data is collected in real-time from diverse sources. Wearable devices equipped with
accelerometers and gyroscopes, smartphones, or other attached sensors serve as the primary
sources of raw data. This continuous stream of information captures the movement patterns
and dynamics of individuals, forming the basis for subsequent fall detection analysis. The
accuracy and timeliness of fall detection hinge on the quality and richness of the data acquired
during this phase.

Data Cleaning:

Data cleaning is a pivotal stage in the real-time fall detection process, aiming to enhance the
quality and reliability of the acquired sensor data. This step involves the identification and
removal of artifacts, outliers, and the careful handling of missing or corrupted data points. The
objective is to ensure that the data is free from noise and inconsistencies that could adversely

41
affect the performance of the fall detection algorithm. By refining the raw data through cleaning
procedures, the subsequent analysis becomes more robust and capable of accurately identifying
fall events.

Data Transformation:

Following data cleaning, the transformed or preprocessed data undergoes a series of steps
collectively referred to as data transformation. This phase encompasses normalization,
segmentation, and feature extraction. Normalization ensures that sensor readings are on a
consistent scale, allowing for fair comparisons across different devices. Segmentation
involves breaking the continuous data stream into smaller time windows, facilitating the
analysis of short-time patterns and changes. Feature extraction identifies and extracts
relevant features, such as statistical measures and frequency domain features, essential for
training and validating fall detection models. This transformation readies the data for the
subsequent stages of model development.

Data Splitting:

Data splitting is a crucial step in evaluating the performance of the real-time fall detection
model. The preprocessed data is partitioned into training, validation, and testing sets. The
training set is utilized to train the fall detection model, exposing it to a variety of patterns and
scenarios. The validation set is employed for fine-tuning model parameters, optimizing its
performance. The testing set is reserved for assessing the model's ability to generalize to new,
unseen instances, simulating real-world scenarios. This systematic splitting ensures that the fall
detection model is robust and effective in detecting falls in diverse situations, providing
confidence in its real-time applicability.

6.1.2 FEATURE EXTRACTION

The feature extraction module in a real-time fall detection system focuses on identifying and
extracting relevant information from preprocessed sensor data. The goal is to transform the raw
data into a set of meaningful features that capture essential characteristics associated with falls.
Feature extraction plays a pivotal role in training machine learning models and facilitating the
differentiation between normal activities and fall events. Here are the key aspects of the feature
extraction module:

42
Orientation And Posture Information:

Orientation and posture information in the context of real-time fall detection involves capturing
data related to the positioning and rotational movements of an individual's body. This set of
features typically includes angles and angular velocities obtained from sensors such as
accelerometers and gyroscopes. The angles provide insights into the orientation of body
segments, while angular velocities describe the rotational movements. Monitoring changes in
orientation and posture is crucial for identifying abnormal body positions that may indicate a
fall. For example, sudden or unexpected deviations from typical postures can be indicative of
a fall event, making these features valuable in distinguishing between normal activities and
potential falls.

Spatial Features:

Spatial features refer to the relationships between sensor readings, especially when multiple
sensors are employed across different body locations. Cross-correlation or covariance matrices
are common spatial features used in fall detection systems. These features capture the
interdependence and interactions between sensors, offering a more holistic understanding of
the body's movements. By considering spatial relationships, the system gains insights into the
coordinated motion of different body segments during activities. Analyzing spatial features is
particularly useful in detecting falls as abnormal spatial configurations or disruptions in the
usual coordination may signify a fall event.

Temporal Context Integration:

Temporal context integration involves incorporating information from previous and

subsequent time frames to enhance the understanding of the sequential nature of activities. This
feature allows the fall detection system to consider the context leading up to a specific moment
and the aftermath. By examining trends, patterns, or changes in the temporal sequence of sensor
readings, the system can better differentiate between normal movements and those indicative
of a fall. Temporal context integration is vital for capturing the dynamic nature of activities,
ensuring that the fall detection algorithm adapts to the evolving context over time.

43
6.1.3 MODEL TRAINING

Training a real-time fall detection model involves creating a system that can quickly analyze
incoming sensor data and make predictions in real-time. The below are the model training
aspect for real-time fall detection:

Labelling:

In real-time applications, labeling may be challenging as it requires immediate and accurate

annotation. If possible, leverage historical data with known labels for model development.
Alternatively, consider using semi-supervised or unsupervised learning approaches.

Learning:

Implement techniques for learning to adapt the model continuously. Learning allows the model
to update its parameters as new data becomes available, enabling it to adjust to changing
patterns or conditions. The model can learn through the labelled data and identify the action by
matching the activity patterns.

Model Architecture:

Choose a model architecture suitable for real-time processing. Lightweight architectures, such
as shallow neural network is optimized because it ensure low latency. Select an optimization
algorithm that converges quickly and is suitable for real-time applications. Adaptive algorithms
can be effective in balancing fast convergence with stability.

Real-Time Inference:

During training, simulate real-time inference conditions to assess the model’s performance in
a production like environment. Evaluate the latency and accuracy of the model on a validation
set that closely resembles the real world scenario.

Deployment:

Integrate the trained model into the real time fall detection system. This involves implementing
the model in away that allows it to process incoming data and make predictions in real-time.
Optimize the deployment environment for low-latency processing.

44
Monitoring and Adaptation:

Implement a monitoring system to continuously assess the model’s performance in real-time.

If performance degrades or if the system encounters new patterns, trigger adaptations such as
model retraining or updates to ensure ongoing effectiveness.

6.2 EVALUATION METRICS

Evaluation metrics in machine learning projects are metrics that are used to evaluate the
accuracy and effectiveness of a machine learning model. There are several different
performance measures that are commonly used in machine learning, depending on the specific
task, algorithm, and evaluation criteria. Some common performance measures include:

 Accuracy: This measures the proportion of correct predictions made by the model, and
is a common performance metric for classification problems.
 Confusion Matrix: A table that summarizes the true positive, false positive, true
negative, and false negative predictions made by a model. It provides information on
the precision, recall, and FI-score of the model.
 Precision: This measures the proportion of positive predictions that are actually
correct. It is the ratio of true positive predictions to the sum of true positive and false
positive predictions.
 Recall: This measures the proportion of actual positive cases that are correctly
identified by the model. It is the ratio of true positive predictions to the sum of true
positive and false negative predictions.
 F1-Score: This is the harmonic mean of precision and recall, and provides a single
metric that balances both precision and recall.

45
 CHAPTER 7

CONCLUSION

7.1 CONCLUSION

In conclusion, the real-time detection of falling humans in natural environments through the
application of deep neural network algorithms represents a significant advancement in the field
of safety and surveillance. By harnessing the power of advanced machine learning techniques,
this innovative approach enables swift and accurate identification of instances where
individuals experience falls. The utilization of deep neural networks enhances the system's
ability to discern human movements amidst complex and dynamic surroundings, offering a
proactive means to address potential risks and emergencies. This technology holds great
promise in various contexts, such as elderly care, industrial settings, and public spaces, where
prompt response to falling incidents is crucial. The successful implementation of real-time fall
detection not only showcases the potential of artificial intelligence in enhancing safety
measures but also underscores the importance of continuously pushing the boundaries of
technology to address real-world challenges effectively.

7.2 SCOPE OF THE FUTURE WORK

The future of real-time fall detection with deep learning offers exciting possibilities. Refining
algorithms for accuracy and diverse environments is key, while combining multi-sensor data
like cameras and accelerometers promises enhanced understanding. Implementing edge
computing will enable low-latency detection on resource-constrained devices, crucial for
remote contexts. Expanding and diversifying training datasets will boost real-world
adaptability. Beyond just detecting falls, future work may involve analyzing human behavior
for proactive safety monitoring. Privacy and ethical considerations remain paramount,
demanding user consent, data anonymization, and adherence to strict guidelines. Integration
with emergency response systems will streamline the intervention process, while user-friendly
interfaces will ensure adoption and effective management. By exploring these avenues, we can
contribute to a future where real-time fall detection protects vulnerable populations, promoting
safety and independence in everyday environments.

46
REFERENCES
[1] Fan, Y., Levine, M., Wen, G., & Qiu, S. (2017). A deep neural network for real-time
detection of falling humans in naturally occurring scenes. Neurocomputing, 260, 43-58

[2] Wang, S., Chen, L., Zhou, Z., Sun, ..X., & Dong, J. (2016). Human fall detection in
surveillance video based on PCANet. Multimedia Tools And Applications, 75(19),
11603-11613

[3] Núñez-Marcos, A., Azkune, G., & Arganda-Carreras, I. (2017). Vision-Based Fall
Detection with Convolutional Neural Networks. Wireless Communications And Mobile
Computing, 2017, 1-16.

[4] H. Sadreazami, M. Bolic and S. Rajan, "Fall Detection Using Standoff Radar-Based
Sensing and Deep Convolutional Neural Network," in IEEE Transactions on Circuits
and Systems II: Express Briefs, vol. 67, no. 1, pp. 197-201, Jan. 2020.

[5] Zerrouki, N., Harrou, F., Sun, Y., & Houacine, A. (2018). Vision-Based Human Action
Classification Using Adaptive Boosting Algorithm. IEEE Sensors Journal, 18(12),
5115-5121.

[6] M. Saleh and R. L. B. Jeannès, "Elderly Fall Detection Using Wearable Sensors: A Low
Cost Highly Accurate Algorithm," in IEEE Sensors Journal, vol. 19, no. 8, pp. 3156-
3164, 15 April15, 2019.

[7] Ali, S., Khan, R., Mahmood, A., Hassan, M., & Jeon, a. (2018). Using Temporal
Covariance of Motion and Geometric Features via Boosting for Human Fall Detection.
Sensors, 18(6), 1918.

[8] Xiong, X., Min, W., Zheng, W. et al. S3D-CNN: skeleton-based 3D consecutive-low-
pooling neural network for fall detection. Appl Intell (2020). https://doi-
org.ezproxy.uws.edu.au/10.1007/s10489-020-01751-y

47
 [9] Min, W., Cui, H., Rao, H., Li, Z., & Yao, L. (2018). Detection of the human Falls on
Furniture Using Scene Analysis Based on Deep Learning and Activity Characteristics.
IEEE Access, 6, 9324-9335.

[10] Zhang ZM, Ma X, Wu HB, Li YB (2019) Fall detection in videos with trajectory-
weighted deep-convolutional rank-pooling descriptor. IEEE Access 7:4135–4144.

[11] Zerrouki, N., & Houacine, A. (2017). Combined curvelets and hidden Markov models
for human fall detection. Multimedia Tools And Applications, 77(5), 6405-6424.

[12] L. Wang, M. Peng and Q. Zhou, "Pre-Impact Fall Detection Based on Multisource CNN
Ensemble," in IEEE Sensors Journal, vol. 20, no. 10, pp. 5442-5451, 15 May15, 2020.

[13] W.-Cheng and D.-M. Jhan, "Triaxial accelerometer-based fall detection method using
a self-constructing cascade-AdaBoost-SVM classifier", IEEE J. Biomed. Health
Inform., vol. 17, no. 2, pp. 411-419, Mar. 2013.

[14] N. Hnoohom, A. Jitpattanakul, P. Inluergsri, P. Wongbudsri and W. Ployput, "Multi-

sensor-based fall detection and activity daily living classification by using ensemble
learning", Proc. Int. ECTI Northern Sect. Conf. Electr. Electron. Comput. Telecommun.
Eng. (ECTI-NCON), pp. 111-115, Feb. 2018.

[15] Sánchez Pérez, Javier, Meinhardt-Llopis, Enric, & Facciolo, Gabriele. (2013). TV-L1
Optical Flow Estimation. Image Processing on Line, 3, 137-150.

[16] Wedel, Andreas & Pock, Thomas & Zach, Christopher & Bischof, Horst & Cremers,
Daniel. (2009). An Improved Algorithm for TV-L1 Optical Flow. 10.1007/978-3-642-
03061-1_2.

[17] Pingault, M & Pellerin, Denis. (2002). Optical flow constraint equation extended to
transparency.

48
[18] Hamprecht, F. A., Schnörr, C. Jähne, B (2007). A duality based approach for realtime
TV-L 1 optical flow. Pattern Recognition. pp. 214–223.

[19] Wedel A., Pock T., Zach C., Bischof H., Cremers D. (2009). An Improved Algorithm
for TV-L1 Optical Flow. Statistical and Geometrical Approaches to Visual Motion
Analysis. Springer

[20] Khan, S., Rahmani, H., Shah, S. A. A., & Bennamoun, M. (2018). A Guide to
Convolutional Neural Networks for Computer Vision. (Synthesis Lectures on
Computer Vision; Vol. 8, No. 1). Morgan & Claypool Publishers.

[21] Bilen, H., Fernando, B., Gavves, E., & Vedaldi, A. (2017). Action Recognition with
Dynamic Image Networks

49