Multi-Class Classifiers for Human

Activity Monitoring

Project Report Submitted in Partial

Fulfillment of Assessment

for

Machine Learning

Prepared By
BSEF19M003 - Laiba Kamal
BSEF19M007 - Anns Shahbaz
BSEF19M012 - Iqra Sarwar
BSEF19M027 - Mahnoor

Lecturer
Adeel Nisar

February 5, 2023
 Table of Contents
1. Introduction 3

2. Pre-processing of Dataset 4

3. Segmentation 5

4. Feature Extraction 5

5. Support Vector Machine 7

6. Deep Neural Network 9

7. Random Forest 14
1. Introduction
Activity Monitoring is a popular field of Ubiquitous Computing that aims to detect specific
activities through sensors worn by a subject, providing real-time data. This can be used for
various applications such as surveillance, gaming, remote control, assistive living, etc. In this
report, Cognitive Village (CogAge) dataset is used for training different classifiers to classify
human activities monitoring data. This section presents a brief introduction to the dataset
and this report.

Cognitive Village (CogAge) dataset

The project will use the Cognitive Village (CogAge) dataset which consists of 61 activities
divided into two categories: state and behavioral. It was collected from 4 subjects using 3
wearable devices, and includes data from accelerometer, gyroscope, and magnetometer
sensors. Data is collected from three wearable devices (smartphone, smartwatch, and smart
glasses) with different sampling rates. The data is stored as Numpy arrays in .npy files, with
one file for each sensor. The data files are 3D arrays with dimensions (N, T, S), where N is the
total number of executions, T is the length of 4 seconds of data, and S is the number of
sensor channels. The label file is a 1D Numpy array of size N and contains integers between
0 and 54 that represent the behavioral activity.

Project Objectives
The objectives of the project are to classify behavioral activities of the CogAge dataset with
both hand executions, implement the PRC to obtain a trained classifier, and evaluate the
performance of the classifier. The challenge is to obtain a well-trained classifier through a
series of steps in the Pattern Recognition Chain (PRC), including choice of sensors,
experimental protocols, subject recruitment, data cleaning, normalization, downsampling,
and dimensionality reduction, as well as computation of values relevant to the recognition
problem, training of a classifier, and classifier evaluation.

We have trained a number of machine learning classifiers for the purpose of classifying
behavioral activities from sensor data. These algorithms include Support Vector Machines
(SVM), Neural Networks (NN), and Random Forest. Section 2 presents the common
preprocessing techniques we have used for each classifier. Section 3 discusses the
segmentation technique. Section 4 presents the feature extraction that we have done for the
Support vector machine. Section 5 discusses the Support vector machine classifier in detail
and its performance measures. Section 6does the same for Dense Neural Networks, and
section 7 for the Random Forest Algorithm.
2.Pre-processing of Dataset
2.1. Loading the dataset
The load_dataset function loads the training and testing data from a specific file path. It
loads ten different arrays of data, including labels and sensor data, and stores them in two
separate tuples, train_data and test_data. These tuples are then returned as the output of the
function.

2.2. Upsampling
Upsampling is the process of increasing the number of samples in a signal. The upsampling
step is needed in this case to make all sensors have the same sampling rate, which is
required for further processing such as feature extraction and modeling.In upsample
function, upsampling is performed on the training and testing data lists in order to make all
of them have the same number of samples (target_shape). The target_shape is determined to
be the maximum number of samples among all sensors in the train_data list (excluding the
first element, which is the label).

The upsample function takes a list of data (data_list) and the target shape as input, and
returns a new list of data with the number of samples increased. For each data in the input
list, the number of rows, original shape, and number of columns are extracted. The ratio of
the target shape to the original shape is calculated, and the scipy.signal.resample function is
used to increase the number of samples to the target shape by resampling along the second
axis (axis=1). The resulting data is then added to the new_data_list. Finally, the train_data
and test_data are updated by concatenating the labels with the new_data_lists obtained from
the upsample function.

2.3. Normalization
Normalization is a preprocessing step to scale the data so that all variables have similar
ranges and distributions. This is important in machine learning because it can help to
reduce the impact of large values, and prevent certain algorithms from being dominated by
one or two large variables. The normalization is done by first defining the normalize_
function, which calculates the mean and standard deviation of the data and scales the data
so that it has zero mean and unit variance. The normalize_data function then applies this
normalization to both the training and testing data.

2.4. Denoising
Denoising is the process of removing noise from a sensor data. It is important because the
presence of noise can negatively impact the quality of the sample and cause problems in
further processing steps such as activity recognition. Denoising is often necessary to obtain
a clean and clear representation of the underlying data, making it easier to extract
meaningful information and insights from the data. Additionally, denoising can improve the
robustness and stability of subsequent analysis, making it more reliable and accurate.
The code is for denoising the data in a train and test dataset. The denoising process is
performed using the wavelet transform. The function takes a sample as input and applies
the wavelet transform to it, then removes small coefficients (noise) using a hard threshold.
The function denoise_data takes in train and test datasets, and applies the denoise function
to each sample in the data. The denoised train and test datasets are returned.
Denoising is important because it helps remove unwanted noise from signals that can affect
the accuracy of subsequent analysis or modeling. The wavelet transform is a useful tool for
denoising as it can effectively separate samples into different frequency components,
allowing for selective removal of noise in the components that contain it. The parameters for
the denoising function, such as wavelet and level, can be changed to meet the specific needs
of the data being processed. The choice of wavelet can affect the type of noise that is
removed and the level determines the number of times the denoising process is performed.
The threshold value sets the minimum level of intensity that the wavelet coefficients must
have to be preserved. By adjusting these parameters, the denoising function is tailored to
achieve the desired results for different classifiers.

3. Segmentation
Window segmentation is a technique used in image (an array) processing and computer
vision. It involves dividing an image (an array) into smaller regions or windows, and
processing each window individually. The concept of a sliding window involves moving a
fixed-size window over an image (an array) in steps. The sliding window acts as a filter and
generates a feature representation for each region. This process is repeated until the entire
image (an array) has been covered.

4. Feature Extraction
The purpose of extracting these features is to provide SVM with a summarized
representation of the input data that captures its essential characteristics, so that SVM can
learn to distinguish between different classes based on these characteristics. The features
calculated in this function include measures of central tendency (mean, median), dispersion
(standard deviation, variance), extreme values (maximum, minimum), quartile values, mean
deviation, kurtosis, skewness, median deviation, range, interquartile range, moments, root
mean square, energy, and sum.

By including these different statistics in the feature set, the SVM is able to learn a more
comprehensive representation of the input data and improve its accuracy in classifying new
data.The compute_feature function is used to extract features from an input array. It
concatenates the outputs of several functions that calculate various statistics about the
input array. These features are then used as inputs for training a Support Vector Machine
(SVM) classifier.

1. The mean_s function calculates the mean of the input array along axis 1 (row-wise
mean). The mean is an important statistic that gives the average value of a set of
numbers and it is significant in understanding the central tendency of the data.

2. The median_s function calculates the median of the input array along axis 1
(row-wise median). The median is a robust statistic that gives the value that
separates the higher half from the lower half of the data. It is significant in
understanding the central tendency of the data, especially for skewed data sets.

3. The std_s function calculates the standard deviation of the input array along axis 1
(row-wise standard deviation). Standard deviation is a measure of spread of the
data, it gives an idea about how much the data is spread around the mean. A small
standard deviation indicates that the data points are close to the mean, while a large
standard deviation indicates that the data points are spread out.

4. The variance_s function calculates the variance of the input array along axis 1
(row-wise variance). Variance is the square of the standard deviation and it
measures the spread of the data.

5. The maxa_s function calculates the maximum value of the input array along axis 1
(row-wise maximum). The maximum value is an important statistic that gives the
highest value in the data set.

6. The mina_s function calculates the minimum value of the input array along axis 1
(row-wise minimum). The minimum value is an important statistic that gives the
lowest value in the data set.

7. The per_s function calculates the pth percentile of the input array along axis 1
(row-wise pth percentile). Percentile is a value below which a certain percentage of
the data falls. The pth percentile is significant in understanding the distribution of
the data and finding values that separate the data into parts.

8. The mean_deviation_s function calculates the mean absolute deviation of the input
array along axis 1 (row-wise mean absolute deviation). The mean absolute deviation
gives an idea about how much the data is spread around the mean. It is significant in
understanding the spread of the data, especially for non-normal distributions.

9. The kurt_s function calculates the kurtosis of the input array along axis 1 (row-wise
kurtosis). Kurtosis is a measure of the peakedness of a distribution. A positive
kurtosis indicates that the data is more peaked than a normal distribution, while a
negative kurtosis indicates that the data is flatter than a normal distribution.

10. The skew_s function calculates the skewness of the input array along axis 1
(row-wise skewness). Skewness is a measure of the asymmetry of a distribution. A
 positive skewness indicates that the data is skewed to the right, while a negative
skewness indicates that the data is skewed to the left.

11. The median_deviation function calculates the median absolute deviation of the
input array along axis 1 (row-wise median absolute deviation). The median absolute
deviation gives an idea about how much the data is spread around the median. It is a
robust statistic and is significant in understanding the spread of the data, especially
for non-normal distributions.

12. The range_s function calculates the range of the input array along axis 1 (row-wise
range). The range is the difference between the maximum and minimum values in
the data set and it gives an idea about the spread of the data.

13. The iqr_s function calculates the interquartile range (IQR) of an input arr along the
first axis. IQR is a measure of dispersion, it is the difference between the third
quartile (75th percentile) and the first quartile (25th percentile) of the data. This
feature is important because it gives information about how spread out the middle
50% of the data is.

14. The moment_s function calculates the nth moment of the input arr along the first
axis. The nth moment of a distribution is a measure of the "spread" of the data. The
nth moment of a set of values can be used to calculate the mean, variance, skewness
and kurtosis. This feature is important as it provides information about the shape of
the data distribution.

15. The rms_s function calculates the root mean square (RMS) of an input arr along the
first axis. The RMS is a measure of the magnitude of a set of values. It is a useful
feature as it provides information about the magnitude of the data, which is
important in many applications.

16. The energy_s function calculates the energy of an input arr along the first axis.
Energy is the sum of squares of the values of the data. This feature is important
because it provides information about the overall magnitude of the data.

17. The sum_s function calculates the sum of the values of an input arr along the first
axis. The sum of the values is a basic statistical feature that provides information
about the overall magnitude of the data.
5. Support Vector Machine
Support Vector Machine (SVM) is a type of supervised machine learning algorithm used for
classification and regression analysis. It is capable of performing linear or non-linear classification,
depending on the choice of kernel function. SVM works by finding the optimal boundary that
separates the data into classes, called a decision boundary. The boundary is chosen such that it
maximizes the margin between the classes, which is the distance between the closest data points
from each class. These closest data points are called support vectors and they have the most impact
on the decision boundary. SVM has many applications in various fields.

5.1. Model Training

The steps to train the model used in our code are as follows:

1. Prerequisite: First connect to google drive, second import libraries and thirdly load the
dataset from the provided path using the load function of numpy.

2. Preprocessing: The data is preprocessed using functions upsample, normalize_data, and

denoise_data. The upsample function makes all the data have the same length,
normalize_data scales the data, and denoise_data removes any noise from the data.

3. Feature Extraction: Given the preprocessed data, 17 statistical features for each segmented
window are computed. And then the features of all the sensors are concatenated or
combined using the np.hstack() function of the numpy, which results in the shape (2284,
31212).

4. Initialize the classifier: The classifier or the model is initialized using svm.SVC(), the name of
the type of kernel is passed as parameter.

5. Train the classifier: The model is trained using the svm.fit() function. In our code, we used 4
different kernels (linear, polynomial, rbf and sigmoid) to train 4 classifiers.

5.2. Model Testing

The testing code tests the trained model on the test data, the test data has already been passed
through the same steps (from step 1 to step 3) as the train data. Now we only evaluate our classifier
using clf.predict() method. We use 4 evaluating measures, accuracy, precision,recall and F1 score.

Here are the results that we obtained using SVM:

Kernel Type Accuracy Precision Recall F1 Score

Linear 57.4% 58.3% 57.5% 56.7%

 Polynomial 13.8% 33.9% 13.5% 15%

RBF 41.6% 51.3% 41% 40%

Sigmoid 41.6% 51.3% 41% 40.6%

5.3. Instructions to use SVM Notebook

Follow these steps to execute the SVM Notebook.

1. Mount Google Drive: Connects to the Google Drive where the required data is stored, by
running the first cell. The Google Drive is mounted to the virtual machine in Colaboratory.

2. Import libraries: Import the required libraries, by running the second cell.

3. Load training and testing data: Loads the training and testing data from Google Drive. The
data is then stored in numpy arrays. Make sure to change the path_train and path_test
variables to match the location of your data. The code assumes that you have access to
Google Drive and the data is stored in the specified path.

4. Pre-processing: Preprocess the training and testing data by running the preprocessing cells.

5. Feature-Extraction: 17 features are provided, run the cells to extract features. After this
section the combined array of features is given as output.

6. Train the models: Provide the training data and training labels to fit the models, code has 4
models you can run model of your choice.

7. Test the models: Evaluate the models using the evaluating metrics.
6. Deep Neural Network
Deep Neural Network is a type of artificial neural network with multiple hidden layers between
input and output layers. These hidden layers process complex non-linear relationships between
input and output and make DNNs capable of learning from vast amounts of data and producing
highly accurate predictions. DNNs are widely used in various applications such as computer vision,
natural language processing, speech recognition and others. We have trained a DNN for our
classification.

6.1. Model Training

The steps to train the model used in our code are as follows:

6. Load the dataset: The function load_dataset is used to load the training and testing data.

7. Preprocessing: The data is preprocessed using functions upsample, normalize_data, and

denoise_data. The upsample function makes all the data have the same length,
normalize_data scales the data, and denoise_data removes any noise from the data.

8. Reshape the data: The training data is reshaped to fit the input format of the neural
network. It is reshaped to have the shape (data.shape[0], data.shape[1] * data.shape[2])

9. Initialize the model: A sequential model is initialized using Sequential() from the keras
library.

10. Add dense layers: The first dense layer is added with 128 units and 'relu' activation function.
Dropout is also added to prevent overfitting. A second dense layer is added with 64 units
and 'relu' activation function. Again, a dropout is added to prevent overfitting. At last, the
output layer is added with 'softmax' activation function for multi-class classification.

11. Compile the model: The model is compiled using categorical cross entropy loss and 'adam'
optimizer.

12. One-hot encodes the labels: The labels are one-hot encoded using keras.utils.to_categorical.

13. Train the model: The model is trained on the training data using model.fit. The epochs
parameter is set to 150 and the batch_size parameter is set to 512. The trained model is
returned.

6.1.1. Analyzing the Training results

● Loss
The loss starts at 7.2413 and gradually decreases over the course of 150 epochs, reaching a
minimum value of approximately 0.2995. The loss seems to be decreasing smoothly and
consistently, indicating that the model is learning effectively.
 ● Accuracy
The accuracy starts at 0.0503 and gradually increases over the course of 150 epochs,
reaching a maximum value of approximately 0.9081. The accuracy seems to be increasing
smoothly and consistently, indicating that the model is improving its prediction accuracy.

● Time
The time taken per epoch varies between 1 and 2 seconds, with an average of approximately
1.5 seconds per epoch. This is a relatively fast training time, especially given that only 5
steps are taken per epoch.

Overall, the results suggest that the model is training effectively and improving in accuracy. The
decrease in loss and increase in accuracy indicate that the model is learning and generalizing well,
while the relatively fast training time suggests that the model is efficient in its use of computational
resources.

6.2. Model Testing

The function test_model tests a trained deep learning model on a set of test data and corresponding
labels. Following steps are carried out:

1. Reshape the test data: The function first reshapes the test data into a 2D array to prepare it
for evaluation.

2. Convert test labels to categorical: The test labels are then converted to a categorical
representation using keras.utils.to_categorical(). This is because most machine learning
algorithms expect the target variable to be in a categorical format.

3. Model evaluation: The model is evaluated using the reshaped test data and the categorical
test labels. The model.evaluate() function returns the test loss and accuracy of the model.

4. Prediction and classification: The function then uses the model to make predictions on the
test data and converts the predictions into class labels using np.argmax(y_pred, axis=1).

5. Evaluation Metrics: The function calculates several performance metrics of the model,
including precision, recall, and F1 score. Precision and recall are two widely used measures
of accuracy in binary and multiclass classification problems. The F1 score is the harmonic
mean of precision and recall.

6. Confusion Matrix: The function then calculates the confusion matrix of the model's
predictions. A confusion matrix is a table that summarizes the performance of a
classification algorithm. It provides a detailed view of the model's performance, including
true positive, false positive, true negative, and false negative predictions.

7. Plotting: Finally, the function plots the confusion matrix using the plot_confusion_matrix
function.

6.2.1. Analyzing the Testing results

 ● Test Accuracy

50.13% indicates the proportion of correctly predicted test instances out of the total test
instances.it shows that the model is not generalizing well to new data. This suggests that the
model might need further tuning or improvement, such as adding more layers, changing the
activation functions, or increasing the number of epochs.

● Precision

51.82% represents the accuracy of the positive predictions. It shows the percentage of
positive instances that the model correctly identified as positive. High precision means that
the model makes fewer false positive predictions, but it doesn't take into account false
negative predictions.

● Recall

50.46% is the ability of the model to find all positive instances in the data. It shows the
percentage of positive instances that the model correctly identified. High recall means that
the model doesn't miss many positive instances, but it may make more false positive
predictions.

● F1 Score

49.85% is a balance between precision and recall, representing the overall accuracy of the
model. It is the harmonic mean of precision and recall, where the best value is 1 and the
worst is 0. The F1 Score gives an idea of how well the model performs overall, taking into
account both false positive and false negative predictions.

Table 6.1. Evaluation Measures of Deep Neural Network

Test Accuracy 50.13%

Precision 51.82%

Recall 50.46%

F1 Score 49.85%

● Confusion Matrix

It is a table that shows the number of true positive, false positive, true negative, and false
negative predictions made by the model. The diagonal elements of the matrix represent the
number of correct predictions made by the model, while the off-diagonal elements
represent the incorrect predictions. This matrix is useful to visualize the performance of the
model, and to identify areas where the model needs improvement. The confusion matrix of
this model is shown below.
6.3. Instructions to Use the Deep Neural Network NoteBook
The notebook provides a step-by-step process for creating and training a deep neural network using
Keras for classifying sensor data. The steps are as follows:
● Import the Google Colab library and mount the Google Drive.
● Import all the necessary libraries including Numpy, Scipy, Pywt, Keras, Matplotlib and
Scikit-learn.
● Run the load_dataset function which loads the training and testing datasets. The datasets
are stored in Google Drive and the path is specified in the code.
● Run the upsample function which upsamples the data to a target shape along the second
axis. Run the calculate_coeffs function which performs discrete wavelet transformation on
the data. Run the preprocess_dataset function which prepares the data for the neural
network model.
● Run the create_model function which creates the neural network model. The model is
created using the Keras Sequential API. The model includes fully connected layers, activation
functions and dropout regularization.
● Run the train_model function which trains the model on the training dataset
● Run the evaluate_model function which evaluates the performance of the model on the
testing dataset. The evaluation is done using accuracy, confusion matrix.
● Run the plot_confusion_matrix function which plots the confusion matrixOnce you have
followed these steps, you will have a working deep neural network model that you can use
to classify the sensor data.
7. Random Forest
Random Forest is a machine learning algorithm that is used for both classification and regression
problems. It is an ensemble method that creates multiple decision trees during training and then
combines the predictions made by each tree to make a final prediction. The trees are constructed
using random samples of the training data and a random subset of features, which helps to reduce
overfitting and improve the overall accuracy of the model. The final prediction is usually made by
combining the predictions of all trees either through majority voting in classification or averaging in
regression. Random Forest is widely used in applications such as image classification.

7.1. Model Training

The steps to train random forest model used in our code are as follows:

1. Load the dataset: The function load_dataset is used to load the training and testing data.
2. Preprocessing: The data is preprocessed using functions upsample, normalize_data, and
denoise_data. The upsample function makes all the data have the same length,
normalize_data scales the data, and denoise_data removes any noise from the data.
3. Reshape the data: The training data is reshaped to fit the input format of the neural
network. It is reshaped to have the shape (data.shape[0], data.shape[1] * data.shape[2])
4. Initialize the model: A sequential model is initialized using RandomForestClassifier() from
the sklearn library.
5. The model is trained on the training data using model.fit(). Preprocessing parameters are
updated to tackle the overfitting of model on training data. Model parameters are also
changed in each advancing version to improve the accuracy of the model. All of these
training iterations and accuracy measure of each is shown below:

We have analyzed the model performance on different values of the below parameters passed to
wavedec() and threshold() function: wavelet type to use for the decomposition(wavelet) ,level of
decomposition to perform (level), threshold value to use in the thresholding operation(threshold),
thresholding mode to use(mode). These parameters are changed to improve the model accuracy.

The different versions of the model are created and trained in order to avoid overfitting. So,
different parameters are passed to RandomForestClassifier and are named as version 0,1,2 on the
basis of these parameters passed.

7.1.1 VERSION 0:

No parameters are passed to the model of RandomForestClassifier

Tackling Overfitting:
7.1.2 VERSION 1:

Initially (version 0) has no parameters passed to model, which are now are updated as:
N_estimators i.e the number of trees in the forest are set to 50,
min_samples_split: minimum number of samples required to split an internal node are set to 20,
min_samples_leaf: The minimum number of samples required to be at a leaf node are set to 10,
max_depth: The maximum depth of the tree is set to 20.

7.1.3 VERSION 2:

Parameters passed to model are now are updated as:

N_estimators i.e the number of trees in the forest are set to 50,
min_samples_split: minimum number of samples required to split an internal node are set to 25,
min_samples_leaf: The minimum number of samples required to be at a leaf node are set to 15,
max_depth: The maximum depth of the tree are set to 60

7.2. Model Testing

The function build_and_evaluate_random_forest_model tests a trained random forest model on a set
of test data and corresponding labels. Following steps are carried out:

1. Reshape the test data: The function first reshapes the test data array using
usingreshape(test.shape[0], test.shape[1] * test.shape[2])
2. Model evaluation: The model is evaluated using the reshaped test data and the test labels.
The model.evaluation_measures() function analyze the model performance by printing
several evaluation metrics.
3. Evaluation Metrics: The function calculates several performance metrics of the model,
including accuracy, precision, recall, and F1 score. Precision and recall are two widely used
measures of accuracy in binary and multiclass classification problems. The F1 score is the
harmonic mean of precision and recall.
4. Confusion Matrix: The function then calculates the confusion matrix of the model's
predictions. A confusion matrix is a table that summarizes the performance of a
classification algorithm. It provides a detailed view of the model's performance, including
true positive, false positive, true negative, and false negative predictions.

7.1.1 VERSION 0:

No parameters are passed to the model. The changed parameters for preprocessing and values of
evaluation measures on those values are shown in the table:
Tackling Overfitting:
7.1.2 VERSION 1:

Initially (version 0) has no parameters passed to model, which are now are updated as:
N_estimators i.e the number of trees in the forest are set to 50,
min_samples_split: minimum number of samples required to split an internal node are set to 20,
min_samples_leaf: The minimum number of samples required to be at a leaf node are set to 10,
max_depth: The maximum depth of the tree is set to 20.

7.1.3 VERSION 2:

Parameters passed to model are now are updated as:

N_estimators i.e the number of trees in the forest are set to 50,
min_samples_split: minimum number of samples required to split an internal node are set to 25,
min_samples_leaf: The minimum number of samples required to be at a leaf node are set to 15,
max_depth: The maximum depth of the tree are set to 60