Machine Learning and Python for Human Behavior, Emotion,

and Health Status Analysis, 1st Edition

Visit the link below to download the full version of this book:

https://medipdf.com/product/machine-learning-and-python-for-human-behavior-emoti
on-and-health-status-analysis-1st-edition/

Click Download Now

 Machine Learning and
Python for Human Behavior,
Emotion, and Health Status
Analysis

Written by
Md Zia Uddin
Designed cover image: Freepik
First edition published 2025
by CRC Press
2385 NW Executive Center Drive, Suite 320, Boca Raton FL 33431
and by CRC Press
4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
CRC Press is an imprint of Taylor & Francis Group, LLC
© 2025 Md Zia Uddin
Reasonable efforts have been made to publish reliable data and information, but the author and publisher
cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication and
apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright
material has not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.com or
contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-
8400. For works that are not available on CCC please contact [email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used
only for identification and explanation without intent to infringe.
ISBN: 978-1-032-54478-6 (hbk)
ISBN: 978-1-032-54635-3 (pbk)
ISBN: 978-1-003-42590-8 (ebk)
DOI: 10.1201/9781003425908
Typeset in Minion
by SPi Technologies India Pvt Ltd (Straive)
Contents

Preface , ix
Acknowledgments, xii
About the Author, xiii

Chapter 1   ◾   Smart Assisted Homes, Sensors, and Machine Learning 1

1.1 SMART HOMES 1
1.1.1 Technologies 2
1.1.2 Benefits 3
1.1.3 Challenges 3
1.2 EXAMPLE SMART ASSISTED HOMES 4
1.3 EVENTS IN SMART ASSISTED HOMES 5
1.4 SENSORS IN SMART HOME 7
1.4.1 Wearable Sensors 7
1.4.2 Ambient Sensors 9
1.5 MACHINE LEARNING 10
1.5.1 Supervised Learning 12
1.5.2 Unsupervised Learning 12
1.5.3 Semi-Supervised Learning 13
1.6 DEEP MACHINE LEARNING 13
1.6.1 Transfer Learning 15
1.7 LIMITATIONS OF MACHINE LEARNING 16
1.7.1 Underfitting and Overfitting 16
1.7.2 Other Limitations 18
1.8 CONCLUSION 19
REFERENCES 20

v
vi   ◾    Contents

Chapter 2   ◾    Python and Its Libraries 27

2.1 PYTHON’S KEY FEATURES 27
2.2 PYTHON IN PRACTICE 28
2.3 PYTHON LIBRARIES AND FRAMEWORKS 29
2.4 PYTHON’S COMMUNITY TO LEARN 29
2.5 PYTHON’S IMPACT ON EDUCATION 30
2.6 PYTHON 2 VERSUS PYTHON 3 30
2.7 PYTHON’S ROLE IN DATA SCIENCE AND MACHINE LEARNING 31
2.8 CHALLENGES AND CONSIDERATIONS 32
2.9 PYTHON BASICS 32
2.10 BUILT-IN PYTHON LIBRARIES 33
2.11 DATA MANIPULATION LIBRARIES 34
2.11.1 NumPy 34
2.11.2 Statistical Analysis 39
2.12 PANDAS 40
2.12.1 Pandas Data Structures 41
2.12.2 Basic Operations 41
2.13 DATA VISUALIZATION 43
2.13.1 Matplotlib 43
2.13.2 Seaborn 45
2.14 CONCLUSION 48
REFERENCES 48

Chapter 3   ◾   Feature Analysis Using Python 49

3.1 FEATURE EXTRACTION 49
3.2 PRINCIPAL COMPONENT ANALYSIS (PCA) 50
3.3 KERNEL PRINCIPAL COMPONENT ANALYSIS (KPCA) 51
3.4 FEATURE EXTRACTION USING ICA 55
3.5 LINEAR DISCRIMINANT ANALYSIS (LDA) 57
3.6 CONCLUSION 60
REFERENCES 60

Chapter 4   ◾   Deep Learning and XAI with Python 64

4.1 INTRODUCTION 64
4.2 NON-DEEP MACHINE LEARNING 66
4.2.1 Support Vector Machines 67
 Contents   ◾    vii

4.2.2 Random Forests 70

4.2.3 AdaBoost and Gradient Boosting 71
4.2.4 Nearest Neighbors 73
4.2.5 More Examples 74
4.3 DEEP MACHINE LEARNING 77
4.3.1 Convolutional Neural Network 78
4.3.2 Pre-trained CNN Models 81
4.3.3 Long Short-Term Memory (LSTM) 90
4.3.4 Neural Structured Learning 93
4.4 EXPLAINABLE AI (XAI) 95
4.4.1 Local Explanations 96
4.4.2 Visual Explanations 101
4.4.3 Feature Relevance Explanations 108
4.5 CONCLUSION 109
REFERENCES 110

Chapter 5   ◾   Behavior and Health Status Recognition 119

5.1 WEARABLE SENSOR-BASED BEHAVIOUR RECOGNITION 119
5.1.1 Mobile Health Dataset and Application 119
5.1.2 PUC-Rio Dataset 130
5.1.3 ARem Dataset 137
5.1.4 WISDM Dataset 138
5.1.5 Real-Time HAR Using Wearable Sensor 147
5.2 VIDEO CAMERA-BASED BEHAVIOR RECOGNITION 158
5.3 AMBIENT SENSOR-BASED BEHAVIOR RECOGNITION 160
5.3.1 Real-Time Home Monitoring Using Ambient Sensors 164
5.3.2 Occupancy Prediction Dataset 167
5.4 HEALTH STATUS MONITORING 173
5.4.1 LSTM for Prediction of Health Status 174
5.4.2 ARIMA for Prediction of Health Status 176
5.4.3 Case Study of Oxygen Saturation, Pulse, and Respiration Prediction 178
5.4.4 Sleep Quality Analysis 183
5.4.5 Case Study of Sleep Quality Analysis 185
5.5 SYNTHETIC DATA GENERATION 195
5.6 CONCLUSION 198
ACKNOWLEDGEMENT 199
REFERENCES 199
viii   ◾    Contents

Chapter 6   ◾   Emotion Recognition 204

6.1 IMAGE-BASED EMOTION RECOGNITION 204
6.2 CASE-STUDIES FOR IMAGE-BASED EMOTION RECOGNITION 208
6.2.1 Local Directional Strength Pattern (LDSP) 208
6.2.2 Principal Component Analysis on LDSP 211
6.2.3 Linear Discriminant Analysis on PCA 213
6.2.4 Facial Expression Modeling 213
6.2.5 Experiments on Depth Dataset 215
6.2.6 Experiments on RGB-based Public Database 216
6.2.7 Experiments on Depth-based Public Database 218
6.3 SAMPLE CODE FOR IMAGE-BASED EMOTION RECOGNITION 218
6.4 REAL-TIME IMAGE-BASED EMOTION RECOGNITION 221
6.5 VOICE-BASED EMOTION RECOGNITION 222
6.6 CASE-STUDIES ON VOICE-BASED EMOTION RECOGNITION 224
6.6.1 Signal Pre-processing 224
6.6.2 Feature Extraction with MFCC 226
6.6.3 Emotion Modelling 227
6.6.4 Experiments and Results 228
6.7 SAMPLE CODE FOR VOICE-BASED EMOTION RECOGNITION 230
6.8 CONCLUSION 234
REFERENCES 234

INDEX240
Preface

This book is a practical guide to smart home applications using Python, divided into six
chapters with hands-­on codes. It starts with an introduction and covers Python basics in
the second chapter. Chapter 3 explores pulling important info from data through feature
extraction techniques, and then Chapter 4 dives into machine learning and Explainable AI
(XAI). The exciting part of the book is to be found in Chapters 5 and 6. Chapter 5 presents
real-­life examples of behavior and health analysis using deep learning and XAI. Then, the
final chapter explores recognizing emotions through cameras and audio using deep learn-
ing and XAI. The book strikes a balance between theory and practical coding, making it
accessible for both smart home enthusiasts and researchers. A little glimpse of each chapter
follows:
The opening chapter introduces the topic of smart homes and machine learning, shed-
ding light on diverse projects. It explores various smart home initiatives, showcasing the
seamless integration of technology into daily life. A central focus is the pivotal role of
machine learning in data extraction and analysis. Machine learning tools emerge as trans-
formative, aiding caregivers and clinical experts in diagnosis and decision-­making. The
chapter unravels insights into the impact of these projects on healthcare and caregiving
landscapes, laying the groundwork for a deeper understanding of the symbiotic relation-
ship between smart homes and machine learning. Readers embark on a journey where
innovative technologies not only enhance our homes but also revolutionize the delivery of
healthcare. This chapter sets the stage for a comprehensive exploration, promising a deeper
dive into the fascinating fusion of smart living and machine intelligence.
Chapter 2 focuses on Python and its libraries, with a number of real-­world examples. It
starts with the basics of Python, in order to ensure that everyone feels comfy and ready for
the journey. We learn about things like variables, data types, and making choices in our
code. It’s like building the ABCs of Python! This part is great for beginners because lots of
hands-­on activities are there to really understand how Python works. It then jumps into
some cool libraries, which are like special tools that make Python even more awesome. It
then checks out NumPy for doing math stuff, Pandas for playing with data, Matplotlib for
creating cool charts, and Seaborn for even fancier visualizations. Each library is explained
with real examples, so it’s not just theory—you get to see how these tools can solve real
problems. By the end of this chapter, you’ll know the ABCs of Python and have some nifty
tools in your coding backpack. It’s like starting with the basics to build a strong foundation
for all the cool things discussed in the next chapters.

ix
x   ◾    Preface

Chapter 3 dives into the world of feature extraction techniques using Python—PCA,
Kernel PCA (KPCA), Independent Component Analysis (ICA), and Linear Discriminant
Analysis (LDA). It’s like uncovering secrets to refine data in machine learning! With
Python, PCA steals the spotlight, making data simpler by capturing the important stuff
and reducing complexity. KPCA takes it a step further, smoothly handling tricky nonlin-
ear data structures. Then, it explores ICA, pulling out hidden factors in the data by focus-
ing on independent components. LDA steps up with Python demos, showing off its skill in
boosting supervised learning by making classes more distinct. These techniques are not
just theoretical—they get down to real-­world applications like image processing, signal
analysis, and text mining. The chapter proves how adaptable and useful these techniques
are across different fields. By using them, machine learning models become champs at
spotting important patterns.
Chapter 4 unfolds the world of deep learning and explainable AI. Recently, researchers
have been drawn to deep learning techniques for modeling patterns in input data, with
Convolutional Neural Networks (CNN) gaining popularity for its superior discriminative
power compared to previous approaches. CNN, a type of deep learning, involves feature
extractions and convolutional stacks to build a hierarchy of abstract features. For time-­
sequential event analysis, Recurrent Neural Networks like long short-­ term memory
(LSTM) shine, offering strong discriminative power. Enter Neural Structured Learning
(NSL), an advanced open-­source framework within deep learning algorithms designed to
grasp events in data. NSL leverages structured signals tied to feature inputs, employing
neural graph learning to train networks based on graphs and structured data. It extends
basic adversarial learning by utilizing structured data with valuable relational information
among samples. Acknowledging the remarkable strides in Artificial Intelligence (AI), the
chapter addresses the challenge of explainability (XAI) arising amid AI success. It explores
various machine learning techniques—both shallow and deep—alongside XAI algorithms,
navigating the dynamic landscape of AI advancements and challenges.
Chapter 5 is packed with cool stories showing how wearable sensors, machine learning,
and explainable AI (XAI) can be combined in real-­life situations. First up is behavior
recognition—it’s like teaching computers to understand how people act using wearable
sensors and smart learning. They look at public datasets to figure out and explain what
our actions mean. Next, it jumps into real-­time activity recognition. Imagine your fitness
tracker not just counting steps but instantly knowing if you’re running, walking, or doing
yoga. That’s the magic of wearable sensors and machine learning working together, mak-
ing our gadgets super-­smart. The tech adventure continues with real-­time body skeleton
tracking using cameras that can see both color and heat. It’s like having a virtual map of
how people move in real time, opening doors for lots of cool applications. Heading home,
we explore real-­time home monitoring and behavior prediction. Ambient sensors keep an
eye on what’s happening, and machine learning predicts what might happen next. It’s like
having a super-­smart home that understands your routines. Health takes the spotlight next
with predictions about oxygen levels and pulse using wearable sensors. It’s not just about
tracking—it’s also predicting, and thereby making health monitoring more personalized
and efficient. We wrap up the tech journey with real-­time respiration prediction, where
 Preface   ◾    xi

ultra-­wideband sensors and machine learning team up to give instant and accurate predic-
tions, taking healthcare to a whole new level. In a nutshell, Chapter 5 is all about real stories
where technology, smart sensors, and intelligent machine learning work together to under-
stand our behavior, predict activities, monitor our homes, and even take health monitoring
to new heights.
Chapter 6 takes us into the world of recognizing emotions, first through cameras and
then using audio. With cameras, it explores different features like Principal Component
Analysis (PCA), Independent Component Analysis (ICA), and Local Directional Patterns
(LDP), teaming up with machine learning to understand facial expressions and emotions
in pictures. The chapter doesn’t stop there—it goes into real-­time emotion recognition,
making things even more exciting. To help us understand how the computer makes these
predictions, it introduces Explainable AI (XAI). Moving on to audio-­based emotion recog-
nition, the chapter introduces Mel-­Frequency Cepstral Coefficients (MFCC) as special fea-
tures to capture emotions in sound. Different machine learning algorithms join the party
to decode the emotional vibes hidden in spoken words or audio signals. In a nutshell, the
chapter unfolds a journey where technology learns to recognize emotions, in terms of both
pictures and also what we say. It’s not just about understanding technical details; it’s about
seeing how these smart systems can grasp and respond to human emotions. Whether it’s
decoding smiles on camera or sensing emotions in our voice, this chapter shows us the
exciting possibilities of machines understanding how we feel.
Acknowledgments

To my beloved wife (Syeda Farzana Zerin) and children (Zayan Adeeb and Fayzan Aabid),
your relentless support, love, and encouragement have been the guiding forces on my jour-
ney. You are my inspiration, my motivation, and my greatest treasures.
To my dear parents (Ahmed Sharif and Monoara Begum), your love, sacrifices, and
commitments have shaped me into the person I am today.
This book is dedicated to you all, as a token of my deepest gratitude and appreciation.

xii
Author

Md Zia Uddin was born in 1981 at Haji Alimuddin’s House, 18

No Ward, East Bakalia, Chittagong, Bangladesh. He is the young-
est child of Ahmed Sharif (Late) and Monoara Begum. Dr. Zia
completed his PhD in Biomedical Engineering in 2011. He is
currently working as a Senior Research Scientist in Sustainable
Communication Technologies department of SINTEF Digital,
Oslo, Norway. SINTEF is the largest research institute in
Scandinavia and one of the largest research institutes in Europe.
He has been leading/working on work packages of various
national and international research projects. His research fields are mainly focused on data
and feature analysis from various sources for physical/mental healthcare using machine
learning/artificial intelligence/XAI. Dr. Zia also has a good teaching experience with more
than 20 computer science-­related courses from bachelor’s degree to PhD.

Dr. Zia Uddin has got more than 150 peer-­reviewed research publications including pres-
tigious international journals, conferences, and book chapters. Half of the publications are
led or mostly done by him. His google scholar citations are around 4500. He got Gold Medal
Award (2008) for academic excellence in undergraduate study. He was also Awarded Korean
Government IT Scholarship (March 2007 to February 2011) and Kyung Hee University
President Scholarship (March 2007 to February 2011). His research works received best/
outstanding paper awards in several peer reviewed international conferences. He acted as
a reviewer in many prestigious journals including IEEE Transactions on Pattern Analysis
and Machine Intelligence (TPAMI), Information Fusion, IEEE Transactions on Industrial
Informatics, and IEEE Transactions on Biomedical Engineering, etc.

Dr. Zia Uddin has been leading/working in different work packages of national and inter-
national research projects. He has been enlisted in the World’s Top 2% Scientists since 2019,
conducted by Stanford University of USA and Elsevier BV. For more information about his
work and background: https://sites.google.com/site/webpagezia/home

xiii
 CHAPTER 1

Smart Assisted Homes,

Sensors, and Machine
Learning

1.1 SMART HOMES
A smart home, or a connected or automated home, is a dwelling equipped with the lat-
est technology and built with systems and devices that can be interconnected to make
your life easier and more comfortable [1–3]. Examples are the Internet of Things (IoT),
artificial intelligence (AI), and various communication technologies that have combined
to pave the way for smart homes. So, an integrated and connected smart home is an inte-
grated and controlled environment that allows the control, monitoring, and automation
of various devices and systems. It is often possible to manage and optimize living spaces
through the Internet or a local network, a capability once considered inconceivable
for homeowners. The concept of smart homes has traversed a remarkable evolutionary
journey. It commenced with rudimentary automation features such as programmable
thermostats and remote-­controlled garage doors. Today, smart homes have become
sophisticated ecosystems due to the rapid advancement of technology, particularly in
the Internet of Things (IoT) and AI. The significance of smart homes extends beyond
mere convenience. They are designed to address the practical needs and challenges of
contemporary living. These connected spaces aim to enhance comfort, energy efficiency,
security, and the overall quality of life. Moreover, smart homes contribute to broader
societal goals, such as sustainable living and improved healthcare. The foundation of
smart homes lies in a synergy of cutting-­edge technologies that enable seamless integra-
tion and intelligent control. Understanding these technologies is crucial to grasping the
essence of smart homes.

DOI: 10.1201/9781003425908-1 1
2   ◾    ML and Python for Behavior, Emotion, and Health Status

1.1.1 Technologies
At the heart of smart homes is the Internet of Things (IoT), a vast network of interconnected
physical objects equipped with sensors, software, and coy features. IoT devices collect and
communicate data over the Internet or other networks, forming the basis for smart home
functionality. AI, specifically machine learning, is the brainpower behind smart homes.
Machine learning algorithms empower devices to learn from user behavior, adapt to pref-
erences, and make intelligent decisions. Voice assistants like Amazon’s Alexa and Google
Assistant are prime examples of AI in action. Effective communication between devices
is essential in a smart home ecosystem. Various communication protocols, such as Wi-­Fi,
Zigbee, and Z-­Wave, facilitate seamless data exchange, ensuring devices can work together
harmoniously. Sensors, ranging from motion detectors to temperature sensors, gather
environmental data. Actuators, like motors or switches, execute actions based on this data.
For instance, smart thermostats use temperature sensors to regulate heating and cooling,
enhancing comfort and energy efficiency. Voice assistants have become the central inter-
face in many smart homes. They utilize natural language processing to understand and
respond to voice commands, serving as the command centers for controlling a wide array
of smart devices. A smart home’s ecosystem comprises various components, each con-
tributing to the overall functionality and experience. Let’s explore the key elements that
define a modern smart home. Smart appliances encompass a range of household machines,
including refrigerators, ovens, washing machines, and dishwashers. What sets them apart
is their connectivity and ability to be remotely controlled and monitored. These appliances
often feature energy-­saving modes and can even provide notifications when maintenance
is required. Intelligent lighting systems offer homeowners complete control over their
lighting environments. Users can adjust brightness, color, and scheduling to create custom
lighting scenarios. Automation capabilities enable lights to adapt based on time of day or
occupancy, enhancing energy efficiency and security.
Smart thermostats have redefined heating and cooling management in homes. They
learn user preferences, adapt to daily routines, and optimize temperature settings for com-
fort and energy savings. They can be controlled remotely via smartphone apps, ensuring a
comfortable environment upon arrival. Security is a top priority for homeowners, and
intelligent security systems provide advanced solutions. These systems include intelligent
cameras, smart doorbells, motion sensors, and locks. With remote monitoring and real-­
time alerts, homeowners can enhance security and peace of mind. Voice assistants, such as
Amazon’s Alexa and Google Assistant, have become integral to many smart homes. They
serve as the central control hubs for various devices, responding to voice commands to
control lighting, music, thermostats, and more. Smart TVs and speakers offer access to
extensive entertainment options, from streaming services to music and games. Voice con-
trol simplifies content access and navigation, providing a seamless and immersive enter-
tainment experience. Smart homes increasingly incorporate health and wellness devices,
catering to residents’ well-­being. These devices include fitness trackers, smart scales, air
quality monitors, and connected medical devices. They empower individuals to monitor
and manage their health from the comfort of their homes. Energy management systems tie
together various devices and sensors to optimize energy usage. Homeowners can track
 Smart Assisted Homes, Sensors, and Machine Learning   ◾    3

energy consumption, receive insights, and make informed decisions to reduce their carbon
footprint and utility bills. Home automation hubs serve as centralized controllers for
diverse smart devices. They enable automation, scheduling, and remote control, offering a
single interface for managing the entire smart home ecosystem.

1.1.2 Benefits
Smart homes bring numerous advantages to homeowners, enhancing their living experi-
ences. Let’s explore the key benefits that have made smart homes increasingly popular. The
most apparent benefit of smart homes is their unparalleled convenience. Homeowners can
effortlessly automate daily tasks, adjust settings, and control devices with a simple tap on
a smartphone or a voice command. From adjusting lighting to regulating temperature,
smart homes prioritize user comfort. Smart homes are designed to be energy-­efficient.
Smart thermostats and lighting systems optimize energy usage, lowering utility bills.
Moreover, they contribute to reducing the overall carbon footprint, aligning with sustain-
ability goals. Security is a paramount concern for homeowners, and smart homes address
this with advanced solutions. Surveillance cameras, motion sensors, and smart locks pro-
vide real-­time monitoring and immediate alerts, bolstering home security. Innovative
entertainment systems offer access to vast content and entertainment options. With voice-­
controlled interfaces, users can navigate their favorite shows, movies, and music, creating
an immersive and enjoyable entertainment experience.
Health and wellness devices within smart homes enable individuals to take charge of
their well-­being. Fitness trackers, for instance, track physical activity, while air quality
monitors ensure a healthy living environment. The ability to remotely monitor and control
smart homes is a game-­changer. Whether adjusting the thermostat while at work or check-
ing security cameras while on vacation, homeowners can stay connected to their resi-
dences, enhancing security and convenience. While the initial investment in smart home
technology can be substantial, long-­term savings from reduced energy consumption,
improved maintenance, and enhanced security often outweigh the upfront costs.

1.1.3 Challenges
Despite their numerous advantages, smart homes also present a set of challenges and con-
cerns that homeowners and manufacturers must address to ensure a secure and seamless
experience. The interconnected nature of smart homes raises concerns about privacy and
data. Unauthorized access to devices or data breaches can compromise sensitive infor-
mation. Implementing robust security measures, such as encryption and regular software
updates, is essential to mitigate these risks. Not all smart devices are compatible, which can
lead to interoperability issues. This can frustrate homeowners who desire a seamless expe-
rience across their devices and ecosystems. The upfront costs of purchasing and install-
ing smart devices can hinder adoption for some homeowners. However, it’s essential to
consider the long-­term savings and benefits when evaluating these costs. Setting up and
configuring a smart home ecosystem can be complex, often requiring technical expertise.
Manufacturers are working to simplify installation, but it remains challenging for some
users. Smart devices rely on stable internet connections, and if the network goes down,