INGENIERÍA Y REGIÓN

Generic Engineering

Implementation of a real-time human fall detection system

using artificial vision

Implementación de un sistema de detección de caídas de personas en

tiempo real mediante visión artificial.

Juan Diego Vinasco1 and Robinson Zambrano Ortiz2

ABSTRACT
The project is carried out to reduce the fatality with which some falls end in elderly people not attended in time, the proposed solu-
tion arises from the exponential growth of the use of artificial intelligence in broad fields of daily life, thus assisting in tasks that involve
constant human attention in addition to a high risk of failure. A training model is used that, according to what is considered to pro-
vide better results in terms of classification and identification in frames, this is Yolo v5, which also allows real-time predictions, which is
what is required by the project. For the training of the model, images of real people are combined with synthetic images (obtained
from a web platform that is responsible for modeling in 3D character animations) to obtain a conveniently better result than ex-
pected.

As for the web application that allows accessibility to the user, a frontend with React and a REST API is used. To notify nearby people
and employees of the detected crash, a direct and instant message via WhatsApp and an alarm that is automatically activated
upon detection of the crash are used. As for the results obtained after testing the model, they were satisfactory with a 94% success
rate.

Keywords: Algorithms, artificial vision, dataset, label, script.

RESUMEN
El proyecto se realiza para disminuir la fatalidad con la que acaban algunas caídas en personas de la tercera edad no atendidas a
tiempo, la solución planteada surge a partir del crecimiento exponencial del uso de la inteligencia artificial en amplios campos de
la vida cotidiana, de esta forma asiste en las tareas que implican una constante atención humana además de un alto riesgo de
fallos. Se utiliza un modelo de entrenamiento que, según lo considerado brinda mejores resultados en cuanto a la clasificación e
identificación en fotogramas, este es Yolo v5 que, además permite realizar predicciones en tiempo real, que es lo requerido por el
proyecto. Para el entrenamiento del modelo, se combinan imágenes de personas reales con imágenes sintéticas (que se obtienen
de una plataforma web que se encarga de modelar en animaciones de personajes en 3D) para obtener un resultado convenien-
temente mejor al esperado.

En cuanto al aplicativo web que permite accesibilidad al usuario se utiliza un frontend con React y un API REST. Con la finalidad de
notificar a las personas cercanas y empleados de la caída detectada se utiliza un mensaje directo e instantáneo vía WhatsApp y
una alarma que se activa de forma automática al momento de la detección de la caída. En cuanto a los resultados obtenidos
luego de realizar las pruebas al modelo, fueron satisfactorios con un 94% de acierto.

Palabras clave: Algoritmos, conjunto de datos, etiqueta, script, visión artificial.

Received: January __th 20xx

Accepted: January __th 20xx

1
Student Software Engineering, Universidad Surcolombiana, Colombia. Email: [email protected]

2
Student Software Engineering, Universidad Surcolombiana, Colombia. Email: [email protected]

Attribution 4.0 International (CC BY 4.0) Share Adapt

1
Introduction The main idea of the project is to provide help to the victims of
falls as soon as possible and thus avoid the fatality in these, for
According to a study presented by the World Health Organiza- this purpose a notification system is established directly to the
tion (WHO), there are 37.3 million falls annually that require employees of the place, in addition to an alarm that fulfills the
immediate medical attention. Of these cases, approximately same function.
684,000 results in death, (World Health Organization, 2021).
Methodology
According to the study, age is the main risk factor for this type of
accidents. People over 60 years of age are at the greatest risk of The system whose modeling is shown in Figure 1 establishes that
death or serious injury. By the year 2019, in 12 Latin American everything starts from a camera that captures video in real time
countries, about 166,000 elderly people live in adult residences and sends the information obtained to the computer, which is
that require constant supervision 24 hours a day, (Sanders, responsible for processing the information through the fall detec-
2019). tion model, finally the information obtained is displayed to the
end user through a web platform.
Factors such as impaired vision, walking difficulties or slippery
floors are the main causes of these accidents. If a person does In case the system detects a fall, a notification is sent via
not have the necessary supervision and suffers a fall, it cannot be WhatsApp to all employees registered in the system, and an
attended immediately, which would result in fatality. alarm is activated to alert nearby people.
One of the solutions that have been proposed to this problem
are devices that detect this type of accident and automatically call
for help. This is a great option, but conditions the user to carry
the device 24 hours a day, adding that they can be too expensive,
even more so when you want to use it in multiple people at the
same time as it could be in an assisted living center.
Fall Detection Using Omnidirectional Camera proposes a fall
detection method using an omnidirectional camera (camera with
360° focal length allowing omnidirectional imaging in a horizontal
angle or field of view). Experiments with the BOMNI omnidirec-
tional video dataset show that the highest accuracy is 97.07%
when using K-Nearest Neighbor (KNN) as the background
subtraction method and classifier, (Nguyen, Tran, Nguyen, &
Nguyen, 2022). Figure 1. Fall detection system diagram.
Source: Authors
On the other hand, Vision-Based Human Fall Detection in Smart
Homes presents a new human fall detection system for smart Materials and resources
homes, the proposed approach uses people as the sensor itself. The real-time fall detection system is integrated with a security
It is a 2-stage detection process, in the first stage fall-like events camera and a mid-range computer.
are detected and in the second stage the data is hosted in the
cloud to confirm the fall. The approach is evaluated on publicly Security camera V380
available data and real time videos, also the performance of the
model is compared with existing machine vision-based fall detec- This camera is chosen because of its low cost and easy accessibil-
tion systems, (Chaudhari, 2022). ity, it has Wi-Fi ip which allows connection from any source
wirelessly, in addition to having an HD image quality and remote
In turn, Human Body Parts Estimation and Detection for Physical control from the mobile device linked with unlimited distance.
Sports Movements, this project proposes segmented detection,
which is explained as the detection of body parts, but separately Mid-range computer
to determine an approximation of their pose in real time and
A computer with low-cost features is chosen, such as: 8gb of
thus aid human body recognition and classification systems.
RAM memory, Intel Core i5 tenth generation processor or
Experimental results on two difficult video datasets, such as the
Ryzen 5 3400 and storage capacity of 524gb. This to run the
UCF soccer and UCF sports action datasets, showed significant
artificial vision model that is responsible for detecting falls in real
accuracies of 90.01% and 86.67, (Jalal, Nadeem, & Bobasu, 2019).
time.
Likewise, Real-Time Human Body Detection Based on YOLOv2
Network, explains that there is a fundamental problem for the
Fall detection system software
efficient detection of the human body in the field of machine To meet the objectives of the project, a fall detection model is
vision. It is a 22-layer network capable of processing the data needed, for this Yolo v5 is used, which allows identification and
stream at 93.5 frames per second, thus fully meeting real-time recognition in frames in real time, with greater efficiency than in
requirements. At the same time, it achieves an average accuracy other similar models.
of 80.27% in the natural scene, (Liu, Liu, Wang, & Li, 2019).
In addition to the artificial vision model, a web platform is need-
For this project it is proposed the use of Yolo v5 as a model for ed through which the processed information is transmitted to
the training of the neural network, due to its ability to detect and the end user, this has a REST API structure, which allows the
identify in real time. client to make requests via HTTP verbs and the backend to send
responses in JSON format.

INGENIERÍA Y REGIÓN 2
Real-time fall recognition model
To train the artificial vision model it is necessary to have a set of
images, for the creation of which images are searched through
the Internet.

• Data set construction

For the construction of an initial dataset, images are collected
through internet research and own sources.
The initial Dataset is complemented with images obtained with
the Kinetix tool that allows 3D character modeling.
This tool allows the generation of fall simulations in video for- Figure 4. Image labeling in LabelImg.
mat, Figure 2. Source: Authors

For the construction of the dataset, two classes are defined,

fallen persons (fall) and standing persons (standing), so that the
model is able to identify and differentiate between the two clas-
ses.
For each class there are 3000 images of which 2500 are real
models and 500 are 3D models.
The percentage of the images used for training and validation of
the model is 80% while the remaining 20% is focused on testing.
The size of each image is 640 pixels wide by 640 pixels high.
Figure 2. Creating a simulation in Kinetix. The above is shown in Table 1. Data set structure.
Source: Authors
Table 1. Data set structure.
Using a Python script, it is possible to capture each frame of the Class Real M. 3 D M. Total Training Test
created animations, Figure 3. This script uses the OpenCV library
fall 2500 500 3000 2400 600
that allows to take each video and split it into multiple images
depending on the number of frames. standing 2500 500 3000 2400 600
Total 5000 1000 6000 4800 1200
Source: Authors

• Fall detection model training

To detect falls in real time, it is necessary to train a model that
achieves the proposed objectives.
To train artificial vision models there are different neural net-
works or previously created algorithms; Mobilnet, VGG16 and
You Only Look Once (Yolo) are some examples of these.
For the project Yolov5 is used in the model training process, for
this it is necessary to have a Dataset with images for both train-
ing and validation, in this case it is done with a ratio of 70% - 30%
respectively.
Figure 3. Python script to extract frames. According to the above, there are 1680 images for the training
Source: Authors
set and 720 for validation. Figure 5 shows the training set for
The LabelImg tool of Label Studio is used to generate and save each class.
image labels in XML, Pascal VOC, Yolo, and other formats, in
this case the Yolo format is used, Figure 4.

INGENIERÍA Y REGIÓN 3
Figure 5. Training set for each class.
Source: Authors

Once the training phase starts, the batches of labels created for Figure 7. 3D model training batch.
each class can be observed, as shown in Figure 6 and Figure 7. Source: Authors.

The model is trained to 120 epochs with a batch size of 32.

• Fall detection model testing

Before implementing the trained model, it is necessary to per-
form tests to know the percentage of accuracy achieved during
the training phase.
Considering the Table 1, during the construction of the data set,
20% of the total images are destined for the testing phase, this
percentage corresponds to 600 images for each class, being 1200
in total.
To perform the corresponding tests, a Python script is used as
shown in Figure 8. PyTorch is used in this script, which is a ma-
chine learning framework and among its functionalities is
torch.hub.load() that allows loading a previously trained model to
make predictions and see the results.
Once the model is loaded, the images corresponding to the test
set are prepared to subsequently make the predictions.
A counter is then created for each of the possible outcomes; fall
Figure 6. Real model training batch. counter, foot counter and no class detected counter. For each
Source: Authors prediction the corresponding counter is incremented, so that at
the end the total results are obtained.
The prediction process is performed once for each class in the
model.
The following mathematical model represents the mathematical
logic used to find the exact accuracy of the trained model.
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑓𝑢𝑙 𝑡𝑒𝑠𝑡𝑠
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = × 100 (1)
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑒𝑠𝑡𝑠 𝑐𝑎𝑟𝑟𝑖𝑒𝑑 𝑜𝑢𝑡

Variable specification

n = total number of successful tests

N = total number of tests carried out
A = Accuracy
𝑛
𝐴= × 100 (2)
𝑁

INGENIERÍA Y REGIÓN 4
 If after 60 seconds, a fallen person is still detected, the alarm is
automatically activated, and the employees are also alerted again
by the same means.
Any user logged into the web application can access a section to
observe the fall detection system in real time, for this purpose an
Endpoint is assigned in the backend that is responsible for trans-
mitting the output of the predictions to the model.
If the fall detection system does not work properly, the user
administrator can manually activate the alarm and alert nearby
people so that the victim can get attention as quickly as possible.
To meet the requirements of managing both employees and
family members, Endpoints are provided with corresponding
HTPP verbs that allow listing, adding, modifying, and deleting
each record.
For the functionality of deleting employees and family members,
the records are not deleted from the database, but their status is
changed from active to inactive to audit the information.

• Front-end
For the construction of the frontend, it is used JavaScript with
React as framework, in addition to tailwind CSS that plays the
structural role of the styles of the page. Thanks to these two
technologies the requirements are fulfilled and connected with
the backend to complete the full functionality resulting in an
intuitive page for the user.
Figure 8. Python script to make predictions.
Source: Authors Results
Fall detection model testing
Web platform structure
As shown in Table 2, for the tests performed on the "fall" class,
Once the model has been trained, it is necessary to implement it Figure 9, 547 hits were obtained, 40 results for the incorrect
in web technologies to show the result to the end user. class and on 13 occasions none of the classes were detected.
• API REST Therefore, the fall class has a 91.17% accuracy rate.

For the construction of the backend, Python is used as the pro-

gramming language and flask as a framework to receive and 547
respond to client-side requests. 91,17% = × 100 (3)
600

This is done using object-oriented development, to facilitate

database queries through Object Relational Mapping (ORM). Now the same mathematical model is used to find the accuracy
of standing persons.
MySQL, a relational database, is used for storage. SQL alchemy is
an ORM available when using flask as a framework, it allows When testing for the "standing" class, 581 hits are obtained, 9 of
queries to be made using an object-oriented language and auto- the results correspond to the incorrect class and on 10 occa-
matically converts them to SQL queries. sions none of the classes is detected, thus obtaining 96.83%
accuracy.
To load and execute the model in the backend server PyTorch is
used, as in the testing phase, with the difference that this time 96,83% =
581
× 100 (4)
predictions are made on the frames obtained in real time by a 600

video camera.
To obtain these frames, the VideoCapture() function of the Finally, the same operation is applied taking the total values, it is
OpenCv library is used, which requires as parameters the infor- obtained that, of the 1200 images destined for testing, in 1128 of
mation corresponding to the camera to be used and provides as these the expected result is obtained, while in 49 the opposite
a result each frame of the captured video. class to the expected one is obtained and in 23 occasions no
class is detected in the image. Therefore, the fall detection model
Each frame obtained is predicted with the fall detection model to
obtains 94% accuracy.
analyze whether an accident has occurred.
If a fall is detected, an alarm is automatically triggered with a
duration of 60 seconds that serves to alert nearby people, and 94,00% =
1128
× 100 (5)
employees previously registered in the system are notified via 1200
WhatsApp.

INGENIERÍA Y REGIÓN 5
This provides the exact accuracy with which the trained model
detects fallen persons and differentiates them from those who
are standing.
Table 2. Test results of the fall detection model.
Clas s Right Inc orre c t Not de te c t Total Hit pe rc e nt
fall 547 40 13 600 91,17%
standing 581 9 10 600 96,83%
To tal 1128 49 23 1200 94,00%

Source: Authors

According to the research carried out in the state of the art

referenced in the introduction of this document, it can be con-
cluded that the accuracy obtained through the mathematical
model is accepted within the standards shared by the environ-
ment of projects carried out with similar modeling, In the project
Figure 10. Login interface.
5Vision-Based Fall Detection and Alarm System for Older Adults Source: Authors
in the Family Environment it is explained that an average accura-
cy of the trained model of 92.7% is obtained (Liu, Zhou, Zhang, For a user with administrator role, the main page has the options
& Cao, 2019), on the other hand in the project Fall Detection of "employee management", "family management" and "activate
Using Omnidirectional Camera an accuracy of 97.07% (Nguyen, alarm", as well as a space where the fall detection system is
Tran, Nguyen, & Nguyen, 2022) is obtained and the project shown in real time, Figure 11.
Human Body Parts Estimation and Detection for Physical Sports
Movements which presents an accuracy of the trained model of
90.01% (Jalal, Nadeem, & Bobasu, 2019).

Figure 11. Administrator user home page.

Source: Authors

Figure 9. Fall detection test class "fall". A user with the role of family member has access to a single
Source: Authors interface through which he/she can visualize the fall detection
system in real time, Figure 12.
Web platform
A web platform is implemented consisting of several interfaces
that allow the end user to access the functionalities that corre-
spond to him/her depending on his/her role.
Users with the role of administrator or family member can log in
to the platform through an interface designed for this purpose,
Figure 10.

Figure 12. Home page for user with family role.

Source: Authors

INGENIERÍA Y REGIÓN 6
 ministrator can manually activate the alarm so that the victim can
be attended to as soon as possible.
• User management
The user with the role of administrator has some interfaces Fall detection system
through which he/she can manage the users registered in the The main function of the fall detection system is to alert the
system, these can be employee management and family members corresponding people to provide help to the victim as soon as
management, Figure 13. possible, for this purpose two methods of alert are created:
notification by WhatsApp to employees and activation of the
alarm to warn people nearby.

• Controlled environment testing

When testing the fall detection system in a controlled environ-
ment, it is observed that people standing in the range captured
by the camera are correctly detected, regardless of the lighting
conditions in which they are, Figure 15.

Figure 13. Family management interface.

Source: Authors

Among the functions for managing users, there is the possibility

of adding both employees and family members. In the case of
adding family members, certain information is requested, such as
name(s), last name(s), email, telephone, and a password with
which the user can access the web platform, Figure 14.

Figure 15. Controlled environment test N°01.

Source: Authors

The detection system manages to properly differentiate between

a standing person and a fallen person, no matter how many
people are in the captured range, in Figure 16 the expected
results for this test can be observed.

Figure 16. Controlled environment test N°02

Figure 14. Family member registration form. Source: Authors
Source: Authors
Although the mentioned results are correct, the system presents
In the case of adding employees, the same data is required ex- failures when detecting fallen persons in low light conditions, as
cept for the password, since this user cannot access the web can be seen in Figure 17, the person in the room with low light is
platform. During the registration of this user, the telephone field not detected by the system even when he is fallen.
is important since it is used to inform him/her in case of detect-
ing an accident.

• Manual alarm activation

If the fall detection system does not identify such an accident and
therefore does not alert the appropriate persons, the user ad-

INGENIERÍA Y REGIÓN 7
 Conclusions and future work

The fall detection system developed with the Yolov5 model

offers the end user an accuracy of 94% in controlled environ-
ments, thus fulfilling the acceptance criterion that is vital in the
field of artificial vision to avoid unreliable systems.
The fact of using synthetic images as part of the training of the
fall detection model was not 100% satisfactory, because such
images contrasted with the way in which the Yolov5 model
processed the information, so the decision was taken to reduce
Figure 17. Controlled environment test N°03. the percentage (in terms of quantity) of synthetic images.
Source: Authors
One of the main reasons why it was considered to train the
This is because the fall detection model is mostly trained on model to also detect people standing is precisely because in this
images with good illumination, so the neural network cannot way it is much easier for the model to differentiate one from the
correctly identify fallen persons in other conditions. other without the need to fail to recognize those positions of the
human body for which it was not trained. This facilitates cleaner
• WhatsApp notification and more efficient training.
A notification system is developed through the WhatsApp social With the implementation of the remote control of the camera, it
network that is responsible for notifying employees registered in is possible to have a much more complete vision than that of-
the system thanks to the phone number stored in it. It can be fered by a static and/or conventional camera of the current
seen in Figure 18. market.
It is recommended to process a greater number of images, of
both categories, standing and falling, so that in this way the per-
centage of hits increases in proportion to the reliability of the
trained model.
It is recommended that a socioeconomic study be carried out to
know the market, the competition, and the future of the applica-
tion of this system in a real environment.
It is recommended to consider the luminosity of the images
when processing a larger number of images, because in the con-
trolled test environment it is concluded that in a place that does
not have considerable luminosity to the human eye, the fall de-
tection system reduces its accuracy significantly.

Acknowledgements
This project was carried out under the supervision of Dr. Ferley
Rojas Medina, with the help of Julian Marino Trujillo Obregon, to
whom we would like to express our deep gratitude in the im-
plementation of the system, to our families for their patience, to
God for providing us with enough tools to carry out the project
and to ourselves for not faltering and completing 100% of the
project.

References
Chaudhari, S. (2022). Vision-Based Human Fall Detection in Smart
Homes. Missouri: State University.
Jalal, A., Nadeem, A., & Bobasu, S. (2019). Human Body Parts
Estimation and Detection for Physical Sports Movements. Inter-
national Conference on Communication, Computing and Digi-
tal Systems (C-CODE). Pakistan.
Figure 18. Screenshot of whatsapp notification. Liu, F., Zhou, F., Zhang, F., & Cao, W. (2019). Vision-Based Fall
Source: Authors
Detection and Alarm System for Older Adults in the Family Envi-
ronment. Intelligent Robotics and Applications. Shanghai.
• Alarm notification Liu, X., Liu, Y., Wang, H., & Li, J. (2019). Real-Time Human Body
Detection Based on YOLOv2 Network. Proceedings of the 11th
When a fall is detected, the system automatically triggers an International Conference on Modelling, Identification and Con-
alarm to alert more people so that the victim can receive assis- trol (ICMIC2019) . China.
tance sooner.

INGENIERÍA Y REGIÓN 8
Nguyen, V. D., Tran, T. M., Nguyen, X. B., & Nguyen, M. Q. (2022). Retrieved from https://blogs.iadb.org/salud/es/residencias-
Fall Detection Using Omnidirectional Camera. Intelligent Tech- para-adultos-mayores/
nologies and Robotics. Singapore. World Health Organization. (2021, April 26). Falls. WHO. Retrieved
Sanders, D. (2019, February 15). ¿Cómo es la vida en las residen- September 19, 2022, from https://www.who.int/news-room/fact-
cias para adultos mayores? Inter-American Development Bank. sheets/detail/falls

INGENIERÍA Y REGIÓN 9