electronics

Article
Comparative Analysis of Machine Learning Techniques
for Non-Intrusive Load Monitoring
Noman Shabbir 1,2 , Kristina Vassiljeva 3 , Hossein Nourollahi Hokmabad 2,4 , Oleksandr Husev 2, *,
Eduard Petlenkov 3 and Juri Belikov 4

1 FinEst Centre for Smart Cities, Tallinn University of Technology, 19086 Tallinn, Estonia;
[email protected]
2 Department of Electrical Power Engineering & Mechatronics, Tallinn University of Technology,
19086 Tallinn, Estonia; [email protected]
3 Department of Computer Systems, Tallinn University of Technology, 11712 Tallinn, Estonia;
[email protected] (K.V.); [email protected] (E.P.)
4 Department of Software Science, Tallinn University of Technology, 11712 Tallinn, Estonia;
[email protected]
* Correspondence: [email protected]

Abstract: Non-intrusive load monitoring (NILM) has emerged as a pivotal technology in energy man-
agement applications by enabling precise monitoring of individual appliance energy consumption
without the requirements of intrusive sensors or smart meters. In this technique, the load disaggre-
gation for the individual device is accrued by the recognition of their current signals by employing
machine learning (ML) methods. This research paper conducts a comprehensive comparative analysis
of various ML techniques applied to NILM, aiming to identify the most effective methodologies
for accurate load disaggregation. The study employs a diverse dataset comprising high-resolution
electricity consumption data collected from an Estonian household. The ML algorithms, including
deep neural networks based on long short-term memory networks (LSTM), extreme gradient boost
(XgBoost), logistic regression (LR), and dynamic time warping with K-nearest neighbor (DTW-KNN)
are implemented and evaluated for their performance in load disaggregation. Key evaluation met-
Citation: Shabbir, N.; Vassiljeva, K.; rics such as accuracy, precision, recall, and F1 score are utilized to assess the effectiveness of each
Nourollahi Hokmabad, H.; Husev, O.;
technique in capturing the nuanced energy consumption patterns of diverse appliances. Results
Petlenkov, E.; Belikov, J. Comparative
indicate that the XgBoost-based model demonstrates superior performance in accurately identifying
Analysis of Machine Learning
and disaggregating individual loads from aggregated energy consumption data. Insights derived
Techniques for Non-Intrusive Load
from this research contribute to the optimization of NILM techniques for real-world applications,
Monitoring. Electronics 2024, 13, 1420.
https://doi.org/10.3390/ facilitating enhanced energy efficiency and informed decision-making in smart grid environments.
electronics13081420
Keywords: non-intrusive load monitoring; load disaggregation; pattern recognition; machine learning;
Academic Editors: Di Wu, Jing Li,
deep learning
Xianmin Wang and Mingliang Zhou

Received: 15 March 2024

Revised: 4 April 2024
Accepted: 7 April 2024 1. Introduction
Published: 9 April 2024
The rising demand for an increased proportion of renewable energy resources (RES)
in the coming decades, driven by the cost-effectiveness and environmental benefits of
cleaner energy production, is expected to follow an upward trajectory [1]. Although RES
contributes positively to sustainability and environmental concerns, their intermittent
Copyright: © 2024 by the authors.
Licensee MDPI, Basel, Switzerland.
nature poses challenges in the residential energy sector [2,3]. Therefore, it is crucial to strike
This article is an open access article
a balance between demand and supply to effectively manage these energy resources. The
distributed under the terms and inclusion of shiftable and non-essential loads in the residential sector, such as electric vehi-
conditions of the Creative Commons cles (EV) and battery energy storage systems (BESS), can play a pivotal role in optimizing
Attribution (CC BY) license (https:// energy management and enhancing system flexibility. The strategy involves scheduling
creativecommons.org/licenses/by/ these loads to coincide with the availability of RES-like photovoltaic (PV) energy. This
4.0/). approach not only reduces energy consumption costs and promotes sustainability and

Electronics 2024, 13, 1420. https://doi.org/10.3390/electronics13081420 https://www.mdpi.com/journal/electronics

Electronics 2024, 13, 1420 2 of 21

self-reliance but also augments the penetration of renewable energy [4]. Referred to as
energy flexibility (EF), this adaptability is essential for transitioning towards eco-friendly
and efficient energy grids. A noteworthy development in this context is the emergence of
demand-side energy aggregators, which contribute to balancing demand and supply by
minimizing peak loads during periods of high demand, thereby ensuring stability in power
systems, and facilitating EF [5].
The EF is generally referred to as the customer’s capacity to adjust or modify behavior
based on energy demand, production variations, weather conditions, and user or grid
requirements [6,7]. Several devices in the household are known as shiftable loads such
as EVs, washing machines, dishwashers, etc. These devices are not essential and could
be used at a later time, therefore, referred to as shiftable/movable devices. Another
prevalent definition focuses on the earliest start time and ending time of shiftable devices.
Traditional EF characterization involved installing smart meters on residential devices and
continuously monitoring data, which, although straightforward, could be costly and slow
it also raised concerns about data privacy [8].
A new approach, non-intrusive load monitoring (NILM), has been proposed as an
alternative. NILM observes the usage patterns of devices based on their current signals,
eliminating the necessity for smart meters [9]. The total energy consumption of the user
is given to the NILM model as an input and then the device usage times are extracted,
this method is known as load disaggregation. This makes NILM an essential tool for
demand-side management (DSM) and EF applications. Although the most precise method
to measure device usage is through energy meters on individual devices, this approach
is not the most practical [10]. The integration of data-driven technologies, such as ma-
chine learning (ML), into NILM has enhanced its efficiency. A detailed review of the
NILM method is given in [11]. NILM solutions can be categorized into supervised and
unsupervised learning [12]. In supervised learning, the model is trained using a dataset,
followed by testing and verification. On the other hand, unsupervised learning involves
the model extracting information from data and forming clusters without prior training
sets [13]. While unsupervised learning is faster and more convenient, it lacks the accuracy
of supervised learning [14]. Several ML-based methods, including K-nearest neighbor
(kNN), neural networks, support vector machine (SVM), deep learning (DL), and event
matching classification, have been proposed in supervised NILM. There are many studies
that have also incorporated statistical methods such as particle swarm and Markov chain
models. In Table 1, a comparison of previous studies with this study has been presented.

Table 1. Comparison of the current study with existing literature.

Study Year Place Method Used for NILM Dataset Avg. Efficiency (%)
[15] 2019 China Particle swarm 1 year 94.2
[16] 2023 Indonesia Random Forests 1 year 99
[17] 2020 India Markov Chain 31 days 94
[5] 2021 Estonia Extreme Gradient Boost (XgBoost) 3 years 97.2
[18] 2022 Malaysia K-NN, SVM, Ensemble 30 days 98.8
[19] 2020 Iran SVM 1 week 98.2
[20] 2021 Indonesia Convolutional Neural Networks (CNN) 1 month 98
[21] 2023 Italy Random Forests 27 months 96.3
[22] 2024 Spain Long Short-Term Memory Networks (LSTM) 7 months 98
[23] 2023 Greece Recurrent Neural network (RNN) 10 days 97
[24] 2023 Canada LSTM 2 days 98
Current Waveform Features with Rule-Based
[25] 2023 UAE 1 month 96
Set Theory (CRuST)
DTW-KNN, Logistic Regression (LR), XgBoost,
This Study 2024 Estonia 1 year 98
RNN-LSTM

The NILM method has been used for anomaly detection at the appliance level by
incorporating machine learning [26]. In another study [8], NILM is utilized for the event
Electronics 2024, 13, 1420 3 of 21

matching of devices. This study was based on the Pecan Street dataset, and it used a deep
learning algorithm for this event matching. In [27], the NILM technique is used to identify
the load patterns, and then later, these patterns are used to improve the accuracy of the
load forecasting. A NILM-based solution has been proposed for energy management in
microgrids [28]. Furthermore, the solution also provides input for the electricity market
based on the load characterization by NILM. The results indicated that using this technique
the energy costs and load curtailment can be reduced. In [29], a NILM-based algorithm
has been proposed for the monitoring of loads in the power distribution network. This
technique consists of a neural network and improves accuracy by 5%.
In most of the literature presented above, there are several methods used in NILM
modeling. However, the accuracy of these NILM models is challenging as there are many
device variations, different manufacturers with different power ratings and device operat-
ing modes. The inclusion of ML and DL methods improves this performance significantly
but still, these methods require larger datasets of reference device signals which is prob-
lematic. Therefore, there is a gap in the research studies about the comparison of different
ML and DL algorithms on the accuracy of the NILM technique. Moreover, the impact of
the size of the dataset on the performance of NILM is also of interest. This paper tries to
fill this gap by evaluating the performance of several ML and DL algorithms employed
in NILM. These models are designed based on a real-life dataset measured in an Estonian
household for the whole year. These are the main contributions of this work:
• Thorough comparative analysis of ML Techniques for NILM, revealing optimal method-
ologies for load disaggregation.
• Utilization of diverse dataset from an Estonian household for comprehensive evalua-
tion of ML algorithms.
• Implementation and Evaluation of LSTM, XgBoost, LR, and DTW-KNN models, high-
lighting XgBoost’s superior performance.
• Insightful evaluation metrics application includes accuracy, precision, recall, and F1
score for nuanced assessment.
• Identification of XgBoost as the most effective model for load disaggregation, offering
practical implications for enhancing energy efficiency.
The rest of the article is structured as follows: Section 2 provides detailed background
information about NILM and the ML and DL methods used in this research. The case study
of the Estonian household and the development of these NILM models are presented in
Section 3. The results and discussion are given in Section 4. Finally, the conclusion and
future works are summarized in Section 5.

2. Non-Intrusive Load Monitoring (NILM)

Non-Intrusive Load Monitoring (NILM) is a progressive approach for estimating
individual appliance operating states and their energy consumption based on household
total electrical load measured at a single point. It involves acquiring and disaggregating
the overall electricity usage, offering a simple and cost-effective means of monitoring
appliances’ operation and energy consumption formulated as:
K
Ptotal (t) = ∑ k =1 p k ( t ) + e ( t ) (1)

where
Ptotal (t) is the power consumed by all appliances,
pk (t)—power consumed by the kth appliance,
e(t)—error or difference between aggregate meter reading and the sum of actual
power consumption.
The examination relies on the measurement of voltage and current waveforms taken
at the electrical service entrance (ESE). These data serve as the basis for deducing the
operational conditions and power consumption of each individual load. Load signatures,
also known as load features, are derived from these waveforms, providing measurable
 𝑝𝑝𝑘𝑘 (t)—power consumed by the kth appliance,
𝑒𝑒(𝑡𝑡) —error or difference between aggregate meter reading and the sum of actual
power consumption.
The examination relies on the measurement of voltage and current waveforms taken
Electronics 2024, 13, 1420 at the electrical service entrance (ESE). These data serve as the basis for deducing the op-
4 of 21
erational conditions and power consumption of each individual load. Load signatures,
also known as load features, are derived from these waveforms, providing measurable
parameters that
parameters that reveal
reveal information
information about
about the
the nature
nature and
and operating
operating status
status of
of individual
individual
appliances. Appliances can be categorized into distinct types based on their load signa-
appliances. Appliances can be categorized into distinct types based on their load signatures,
tures, shaping the approach to disaggregation:
shaping the approach to disaggregation:
•• Type-I appliances,
Type-I appliances, such
such as as toasters
toasters and
andboilers,
boilers,exhibit
exhibitaastraightforward
straightforwardON/OFF
ON/OFF
state.
state.
•• Type-IIdevices
Type-II deviceslike
likewashing
washingmachines
machinesand andovens
ovens operate
operate with
with multiple
multiple (finite)
(finite) num-
number
beroperating
of of operating states
states withwith recognizable
recognizable patterns.
patterns.
•• Type-III devices,
Type-III devices, presented
presented by by dimmer
dimmer lights,
lights, belong
belong to
to continuously
continuously variable
variable de-
de-
vices (CVD), presenting a challenge in disaggregation due to their constantly
vices (CVD), presenting a challenge in disaggregation due to their constantly varying varying
consumption.
consumption.
•• Type-IV,devices
Type-IV, devicesthat
that are
are constantly
constantly inin operation
operation and
and have
have different
different energy
energy consump-
consump-
tion modes
tion modes like
like smoke
smoke detectors
detectors and
and refrigerators.
refrigerators.
Given the
Given the diversity
diversity outlined
outlined above,
above, developing
developing anan accurate
accurate yet
yet broadly
broadly applicable
applicable
NILM system
NILM system isis aa challenging
challenging task.
task. Consequently, many algorithms
Consequently, many algorithms are
are designed
designed to to focus
focus
on identifying
on identifying only
only the
the most
most significant appliances. This
significant appliances. This strategic
strategic approach
approach acknowledges
acknowledges
the complexity
the complexity of of capturing
capturing thethe varied
varied operational
operational signatures
signatures across
across different
different appliance
appliance
types while aiming to provide targeted and effective load disaggregation. The
types while aiming to provide targeted and effective load disaggregation. The goal
goal is
is to
to
strike a balance
balancebetween
betweenaccuracy
accuracyandandgeneralization, ensuring
generalization, ensuringthat thethe
that NILM
NILM system can
system
reliably
can identify
reliably and and
identify monitor key appliances
monitor without
key appliances becoming
without overly
becoming intricate
overly and chal-
intricate and
lenging to implement
challenging to implement [30,31]. A general
[30,31]. NILM
A general NILM process
processcan
canbebepresented
presentedin in four phases
four phases
and observed
and observed inin Figure
Figure 1.
1.

Figure 1.
Figure 1. The
The general
general flow
flow of
of the
the NILM
NILM technique.
technique.
2.1. Data Collection
2.1. Data Collection
The initial step in any NILM algorithm involves data acquisition, typically obtained
The initial step in any NILM algorithm involves data acquisition, typically obtained
from smart meters. The crucial question in load disaggregation is determining the opti-
from smart meters. The crucial question in load disaggregation is determining the optimal
mal data collection frequency for smart meters to ensure accurate appliance identification
datapower
and collection frequency
estimation. fortrade-off
The smart meters
betweento ensure accurate
high and appliance
low data identification
frequencies and
significantly
power estimation.
impacts The trade-off
NILM algorithm betweenHigh-resolution
effectiveness. high and low data frequencies significantly
measurements, often exceeding im-
pacts NILM algorithm effectiveness. High-resolution measurements, often exceeding
1 Hz, can extract transient features crucial for identifying appliances with similar power con- 1
Hz, can extract transient features crucial for identifying appliances with similar
sumptions, particularly during state transitions. On the other hand, excessively small data power
consumptions,
frequencies limitparticularly duringto
feature extraction state transitions.
steady-state On the otherproving
characteristics, hand, excessively small
it insufficient for
data frequencies limit feature extraction to steady-state
differentiating appliances with comparable power usage. characteristics, proving it insuffi-
cientThe
for differentiating appliances
sampling frequency, with comparable
an essential power
factor in data usage.
pre-processing, varies based on
the appliance signature of interest, with researchers recognizing the utility of both low-
frequency and high-frequency signatures. High-frequency data, however, require high-end
hardware, additional data storage, and have transmission problems, and thereby increasing
costs. Recent NILM solutions strategically balance algorithmic efficiency and performance
across a diverse range of appliances, often favoring low-frequency signals to achieve
satisfactory results [32].
The algorithms differ significantly in their approach to handling data at the collection
stage. The below-mentioned algorithms have the following differences: DTW-KNN excels
Electronics 2024, 13, 1420 5 of 21

in time series classification, accommodating speed variations but lacks explicit handling
of missing data or noise. XgBoost robustly handles tabular data, automatically adapting
to missing data and outliers, despite needing careful tuning and pre-processing. Logistic
Regression, suitable for binary classification, demands meticulous pre-processing, espe-
cially for categorical data, and lacks inherent handling of missing values and noise. LSTM
networks, an expert at processing sequential data, are robust to noise but may struggle with
lengthy sequences, necessitating truncation or summarization and requiring numerical
input. Based on the above mentioned each algorithm offers unique strengths, and optimal
performance relies on meticulous data pre-processing and tuning tailored to the task and
data characteristics.

2.2. Event Detection

Within the domain of NILM, event detection serves as a crucial task, focusing on detect-
ing state transition actions generated by appliances. The event detection module identifies
instances of state transitions in the aggregated power signal, characterizing actions like
ON/OFF switches, changes in appliance speed and mode alternations. Challenges in event
detection arise from high fluctuations, long transitions and near-simultaneity, and misiden-
tification of events can lead to decreased accuracy and increased computational complexity
in NILM methods. The event detection module employs various models, including ex-
pert heuristics, probabilistic models, and matched filter models, to identify events in the
aggregate signal, with a subsequent focus on exploring different signatures for effective
NILM research, including steady-state features extracted from low-frequency sampled
data around the event detection window. Despite their ease of extraction, steady-state
features face challenges of feature overlapping and susceptibility to power disturbances,
highlighting the ongoing efforts to enhance NILM methodologies [30].
In the event detection phase, the process begins with threshold estimation, where
a specific value is set or calculated dynamically to identify when an event, such as an
appliance turning on or off, has occurred. This threshold is typically based on changes in
power consumption and aims to minimize both false positives and negatives. Following
the detection of events, they are classified into different categories, often corresponding to
individual appliances. The performance of this event classification is then evaluated using
various metrics such as precision, recall, and F1-score. These metrics assess the accuracy of
the classification in terms of the proportion of correctly identified events, the proportion of
actual events that were missed, and the balance between precision and recall, respectively.

2.3. Feature Extraction

Effective NILM methods necessitate distinctive features or signatures that capture the
unique behaviors of appliances, facilitating the differentiation of various types of appliances.
These features are derived from the distinctive power consumption patterns exhibited by
individual appliances and are utilized to identify or recognize corresponding appliances
from aggregated signals. Two main categories of features employed in NILM are transient
features and steady-state features. Transient features, extracted from the transition process
between two steady states, require high-frequency data acquisition by smart meters, typi-
cally exceeding 1 Hz. Event detection methods separate the transition process from overall
measurements, posing a challenge to accurately capture the start and end of transitions.
The steady-state features, on the other hand, encompass variables such as active power,
reactive power, current, and voltage waveform, and can be extracted from conventional
smart meter data without the need for high-frequency sampling. Although steady-state
features are commonly used, determining the number of states remains challenging [30,33].
Feature extraction is a crucial step that involves processing the collected data to
extract meaningful information. Features mentioned above (see Figure 1) can be used
individually or in combination to improve the performance of the systems. For example,
“aggregation” refers to the total energy consumption data collected from the main power
line. It serves as the primary input for NILM systems. Date/time features can capture
Electronics 2024, 13, 1420 6 of 21

daily/weekly/monthly/annual patterns in energy usage. Time intervals can refer to the

duration for which an appliance is used, as different appliances tend to be used for different
lengths of time [31,32]. These features can be used individually or in combination to
improve the performance of NILM systems. The choice of features often depends on
the specific characteristics of the problem at hand, such as the number of appliances, the
sampling rate of the data, and the availability of training data [33,34].

2.4. Load Disaggregation

The final stage in the process is load disaggregation, where the identified features and
patterns are used to determine the individual energy consumption and operational states
of specific appliances within a building. During the load disaggregation phase, machine
learning or pattern recognition algorithms, previously trained on labeled datasets in the
earlier stages, are applied to the real-time or historical aggregated energy data [35]. These
algorithms use the learned patterns and features to attribute portions of the total energy
consumption to specific appliances. The complexity of load disaggregation lies in the fact
that multiple appliances may be operating simultaneously, and their energy signatures
may overlap. Advanced machine learning models are often employed to handle these
challenges and improve the accuracy of disaggregation. The choice of algorithm often
depends on specific characteristics of the data and the complexity of the task, including the
number and types of appliances, the intricacy of energy usage patterns, and the availability
of labeled training data [36].
Recognizing the diverse nature of energy consumption patterns, a strategy involving
multiple algorithms has been chosen. NILM studies have explored both supervised and
unsupervised approaches. Supervised methods, utilized in our approach, require a labeled
dataset with sub-metered appliances. However, this kind of dataset may not always
be available. On the other hand, unsupervised methods can be applied without prior
knowledge of the environment. Nevertheless, users are required to validate identified
appliance patterns. As our data are labeled, we primarily use supervised methods in
our approach. Dynamic Time Warping (DTW) is employed for its ability to measure
similarity between sequences, providing flexibility in capturing dynamic variations in
energy consumption. The K-NN algorithm leverages the proximity of data points to classify
patterns, contributing a robust method for identifying similarities in energy signatures.
XgBoost, a powerful ensemble learning technique, excels in handling complex relationships
and boosting predictive performance [37]. Lastly, LSTM is chosen for its effectiveness in
discerning patterns in high-dimensional spaces [38]. By integrating these algorithms, we
aim to enhance the accuracy and versatility of our load disaggregation process, addressing
the complexities inherent in energy consumption data.

3. Machine Learning Techniques

ML revolutionized NILM by providing transparency and precision in energy con-
sumption analysis. ML algorithms excel at analyzing vast datasets and uncovering hidden
patterns within the energy signal. They can learn from historical data to identify unique
signatures of individual appliances, even when operating simultaneously. ML’s dynamic
nature allows it to adapt continuously to evolving usage patterns and seasonal variations,
ensuring sustained accuracy over time. Ultimately, ML transforms raw energy data into
actionable insights, empowering users to optimize energy management.

3.1. Dynamic Time Warping with K-Nearest Neighbor (DTW-KNN)

DTW is a crucial algorithm for time series classification, where the objective is to
train a model capable of accurately predicting the class of a time sequence within the
labeled dataset [39]. The K-NN algorithm is commonly employed for this task, with
a modification using the DWT metric instead of the classic Euclidian distance. DTW
accommodates variations in length and speed between the compared time series, making it
particularly effective for capturing patterns in energy consumption over time [40]. Despite
Electronics 2024, 13, 1420 7 of 21

its efficiency, the challenge lies in the time complexity of DTW, especially for large datasets
with lengthy sequences [41]. However, understanding the nuances of DTW allows for
necessary adjustments to enhance the algorithm’s speed, ensuring practical and efficient
time series classification in the context of NILM.
For one-dimensional time series denoted as f (x(i)) and f (y(j)), where i and j represent
time points in series, and x and y are vectors, characterized by their Euclidian distances [42].
The DTW algorithm involves creating a local cost matrix D storing pairwise distances
between x and y. The algorithm seeks an optimal warping path under certain constraints
using dynamic programming, determining the DTW distance as the minimum accumulated
distance normalized by the length of the optimal warping path. This alignment process
minimizes the “distance” between the two-time series is presented in Equation (2):


 Di−1,j−1 (match)
Di,j = d xi , y j + min Di−1,j (insertion), (2)
Di,j−1 (deletion)


where d xi , y j = xi − y j .
The k-nearest neighbors (K-NN) nonparametric statistical algorithm relies on k training
samples in proximity to the feature space as input. The classification of an object is based
on the most frequently occurring class among the identified k nearest points. The parameter
k denotes the number of nearest neighbors influencing the classification process, and the
selection of an appropriate k is a nuanced yet crucial step for optimizing the model’s
performance [43].
The integration of DTW and K-NN in a combined approach is motivated by the
distinctive strengths of each method. This integration yields a more robust and accurate
predictive model, specially tailored for applications in time series analysis. Essentially, the
synergy between DTW and K-NN capitalizes on DTW’s efficacy in capturing temporal
nuances and K-NN’s proficiency in pattern classification based on similarity. This combined
approach facilitates a more comprehensive analysis of time series data, proving particularly
beneficial when dealing with complex and dynamic patterns [43].

3.2. Extreme Gradient Boosting

XgBoost is a highly efficient machine learning algorithm known for its effectiveness
in predictive modeling tasks. As a gradient-boosting algorithm in the ensemble learning
family, XgBoost excels in capturing intricate patterns and relationships within energy con-
sumption data [44]. Its strength lies in accurately identifying and distinguishing between
energy signatures of diverse appliances, making it invaluable in scenarios with complex
and evolving consumption behaviors [44,45]. Operating as a tree ensemble model with k
trees XgBoost predicts outcomes for data samples ( xi , yi ) through a defined expression [46]:

ŷi = Fk ( xi ) = Fk−1 ( xi ) + f k ( xi ), (3)

where
Fk−1 ( xi ) is the prediction result of previous k − 1 trees,
f k ( xi )—k-th decision tree.
The algorithm’s objective function involves a cost function, assessing the error between
predicted and actual values.
n k
∑i=1 L(yi , ŷi ) + ∑ j=1 Ω


Fobj = fj (4)

The regularization term incorporates the L1-norm, preventing overfitting by penalizing

the number of leaf nodes, and the L2-norm, penalizing leaf node weights. Each iteration
introduces a new tree, and the objective function is approximated using first and second-
order gradients.
Electronics 2024, 13, 1420 8 of 21

3.3. Logistic Regression

Logistic regression plays a pivotal role in load disaggregation within NILM systems
for binary classification tasks, determining the ON/OFF states [47]. In this context, logistic
regression models are trained using labeled data where the state of each appliance is
known. Features extracted from the aggregated power signal, such as voltage, current, and
frequency, serve as input variables for the logistic regression model. The model learns the
relationship between these features and the probability of an appliance being in the on or
off state [48].
During inference, the trained logistic regression model is applied to real-time aggre-
gated power data to predict the probability of each appliance being ON or OFF. By setting
a threshold probability, appliances are classified as either ON or OFF, providing valuable
insights into individual appliance usage patterns. The performance of the regression-based
load disaggregation model is then evaluated using metrics such as accuracy and precision
and recall, with iterative optimization techniques like feature selection and hyperparameter
tuning applied to enhance model efficacy.

3.4. Long Short-Term Memory Networks (LSTM)

The LSTM algorithm has become one of the essential tools in NILM due to its ability
to overcome the limitations of traditional Recurrent Neural Networks (RNNs), especially
in handling long-term dependencies and gradient vanishing issues [49]. LSTMs are par-
ticularly favored for NILM tasks because they excel at capturing the inherent long-term
dependencies present in time series data. Equipped with forget, input, and output gates,
the LSTM architecture provides precise control over information flow within each memory
block, allowing for the retention of relevant information while discarding extraneous data.
The hidden layer of an LSTM network is a crucial component comprising gated units
or cells, which work in tandem to generate both the cell’s output and internal state (see
Electronics 2024, 13, x FOR PEER REVIEW 9 of 21
Figure 2). Consisting of four interconnected elements, including three logistic sigmoid
gates and one hyperbolic tangent (tanh) layer, LSTMs exhibit a sophisticated mechanism
for controlling information flow within the cell [50]. The forget gate, employing a sigmoid
patterns. They
activation enable
function, neural networks,
determines especially
the relevance LSTM models,
of information fromtothebetter capture
previous cellrecur-
state,
ring patterns
aiding in tasks like
in the removal time series
of obsolete forecasting
data. and energy
Subsequently, consumption
the input modeling
gate combines [52].
current
Whenwith
input using circular
the previoustimestamps withfiltering
hidden state, LSTMs, pertinent
it is crucial to encode timestamps
information as angles
and generating new
on a unit values
candidate circle and design
for the networks
cell state throughto ahandle circular
tanh layer. sequences
Finally, effectively.
the output This ap-
gate normalizes
proach enhances LSTM models’ ability to accurately learn cyclic patterns across diverse
cell state values and produces the final output, emphasizing LSTMs’ capability to retain
domains, offering a compact yet powerful representation of time-related data [53].
long-term dependencies and regulate information flow effectively [51].

Figure2.2. Structure
Figure Structureof
ofthe
theLSTM
LSTMNetwork.
Network.

3.5. Performance Indicators

It is essential not to overlook the need for additional signal processing when inte-
grating neural networks
Evaluating (NNs)requires
NILM systems into applications. Circular timestamps
careful consideration provide
since a single metrica cannot
cyclic
representation of time, which is beneficial for handling periodic data such as daily
capture all its nuances. Although metrics like mean squared error (MSE) and false posi- or
tive/negative rates offer insights into overall accuracy, the evaluation should extend to
specific appliance identification metrics such as precision, recall, and F1-score [22]. These
metrics provide a granular understanding of how well the system distinguishes between
individual appliances, which is essential for practical implementation in real-world sce-
narios [10,43].
Electronics 2024, 13, 1420 9 of 21

seasonal patterns. They enable neural networks, especially LSTM models, to better capture
recurring patterns in tasks like time series forecasting and energy consumption model-
ing [52]. When using circular timestamps with LSTMs, it is crucial to encode timestamps as
angles on a unit circle and design networks to handle circular sequences effectively. This
approach enhances LSTM models’ ability to accurately learn cyclic patterns across diverse
domains, offering a compact yet powerful representation of time-related data [53].

3.5. Performance Indicators

Evaluating NILM systems requires careful consideration since a single metric cannot
capture all its nuances. Although metrics like mean squared error (MSE) and false posi-
tive/negative rates offer insights into overall accuracy, the evaluation should extend to
specific appliance identification metrics such as precision, recall, and F1-score [22]. These
metrics provide a granular understanding of how well the system distinguishes between
individual appliances, which is essential for practical implementation in real-world scenar-
ios [10,43].
TP
Precision = , (5)
TP + FP
TP
Recall = , (6)
TP + FN
Precision × Recall
F1 = 2 . (7)
Precision + Recall
A confusion matrix is a fundamental technique in machine learning that serves as a
concise summary of a classification algorithm’s performance. It provides a tabular layout
of the correct and incorrect predictions made by the classifier, mapping these predictions
to the original classes of the data. This matrix offers crucial insights that go beyond
simple accuracy metrics, which is especially valuable when dealing with imbalanced
datasets or multiple classes. In the matrix, columns denote predicted values, while rows
represent actual values. This arrangement offers a clear visualization of the model’s
accuracy and the patterns of its errors across all classes simultaneously. This structured grid
aids in understanding the classifier’s performance by comparing the correct and incorrect
predictions for each class.
The evaluation, however, does not end at core performance metrics. Computational
efficiency, adaptivity, and data handling diversity must also be considered. Metrics such
as execution time and memory usage shed light on the system’s computational demands,
crucial for real-time applications and resource-constrained environments. Flexibility metrics
gauge the system’s ability to adapt to new appliances and environmental changes, ensuring
its relevance and applicability over time. Finally, scalability and robustness metrics assess
how well the system performs across diverse datasets and under varying conditions,
offering a comprehensive picture of its reliability and generalizability.

4. Case Study of an Estonian Household

4.1. Exploratory Data Analysis
In this study, forecasting algorithms were developed using load data from a household
in Estonia. The specific residence is situated in Tallinn city, comprising two levels, four
rooms, and holding a “C” energy rating. With a total area of around 100 square meters, it
was designed for two adults and one child. The data was measured using the Emporia
Gen 2 3-PHASE device with 16 Sensors. Data collection spanned from August 2021 to
August 2022, achieving an accuracy rate with an error margin below 5%. Measurements
were taken at 15-min intervals. The analysis included various household appliances such
as a dishwasher, vacuum cleaner, television, stereo, sauna, ventilation system, refrigerator,
lighting fixtures, electric stove, and washing machine. Additionally, the house featured a
heating system, water heater, and electric heating floor. The key DC power consumers in
the residence comprised interior and exterior lights, multiple phone and laptop chargers, a
TV and sound system, and a floor heater.
 to August 2022, achieving an accuracy rate with an error margin below 5%. Measurements
were taken at 15-min intervals. The analysis included various household appliances such
were taken at 15-min intervals. The analysis included various household appliances such
as a dishwasher, vacuum cleaner, television, stereo, sauna, ventilation system, refrigera-
as a dishwasher, vacuum cleaner, television, stereo, sauna, ventilation system, refrigera-
tor, lighting fixtures, electric stove, and washing machine. Additionally, the house fea-
tor, lighting fixtures, electric stove, and washing machine. Additionally, the house fea-
tured a heating system, water heater, and electric heating floor. The key DC power con-
Electronics 2024, 13, 1420 tured a heating system, water heater, and electric heating floor. The key DC power10con-
sumers in the residence comprised interior and exterior lights, multiple phone and laptop of 21
sumers in the residence comprised interior and exterior lights, multiple phone and laptop
chargers, a TV and sound system, and a floor heater.
chargers, a TV and sound system, and a floor heater.
The data for the entire year is illustrated in Figure 3, showcasing separately measured
The data for
for the entire year is
is illustrated in Figure
Figure 3,3,showcasing separately measured
AC The
and data
DC loads the entirethe
within year illustrated
household. Theincombined showcasing
average separately
value measured
loads hours around
AC
AC and
and DC
DC loads
loads within
within the
the household.
household. The
The combined
combined average
average value
value loads
loads hours
hours around
around
3 kW. The peak recorded load, reaching approximately 19 kW, occurred in February dur-
33 kW. The peak
kW.the
The peak recordedload,load,reaching
reachingapproximately
approximately 19kW, kW,occurred
occurredinin February dur-
ing winter recorded
months when heating demands were at19their February
peak. In Estonia, during
winter spans
ing
the the winter
winter months
months whenwhen heating
heating demandsdemands
werewere at their
at higher
their peak. peak. In Estonia,
In Estonia, winter
winter spansspans
from
from November to March, typically witnessing energy consumption. Conversely,
from November
November to March,
to March, typically
typically witnessingwitnessing
higherhigher
energyenergy consumption.
consumption. Conversely,
Conversely, during
during the summer months between May and August, energy consumption drops signif-
during the summer
the summer monthsmonths
between between
May and May and August,
August, energyenergy consumption
consumption drops drops signif-
significantly
icantly as heating demands diminish. The monthly energy consumption throughout the
icantly
as heating demands diminish. The monthly energy consumption throughout the yearthe
as heating demands diminish. The monthly energy consumption throughout is
year is shown in Figure 4.
year
shownis shown in Figure
in Figure 4. 4.

Figure3.3.The
Figure Theresidential
residentialload
loadthroughout
throughoutthe
theyear.
year.
Figure 3. The residential load throughout the year.

Figure 4. The monthly energy consumption.

Figure
Figure 4.
4. The
The monthly
monthly energy
energy consumption.
consumption.

The hourly energy consumption data is presented in Figure 5 indicating the highest
energy utilization during the evening hours around 7 and 8 p.m., coinciding with most
occupants being at home. While there isn’t a specific hour of lowest energy consumption
evident in the analysis, energy usage tends to be lower in the early morning hours between
2 to 6 a.m. In Figure 6, the individual load patterns of devices like stoves, rainwater
drainers, sauna, sockets, water pumps, washing machines, lights and heating are shown
during a single day. The sauna, washing machine, electric stove, heater, and water pump
are the most energy-consuming loads when they are being utilized.
 occupants
occupants being
being at
at home.
home. While
While there
there isn’t
isn’t aa specific
specific hour
hour of
of lowest
lowest energy
energy consumption
consumption
evident
evident in the analysis, energy usage tends to be lower in the early morning hours
in the analysis, energy usage tends to be lower in the early morning hours between
between
22 to
to 66 AM.
AM. In
In Figure
Figure 6,
6, the
the individual
individual load
load patterns
patterns of of devices
devices like
like stoves,
stoves, rainwater
rainwater drain-
drain-
ers,
ers, sauna,
sauna, sockets,
sockets, water
water pumps,
pumps, washing
washing machines,
machines, lights
lights and
and heating
heating are
are shown
shown during
during
Electronics 2024, 13, 1420 aa single
single day.
day. The
The sauna,
sauna, washing
washing machine,
machine, electric
electric stove,
stove, heater,
heater, and
and water
water pump
pump are
are the
the
11 of 21
most energy-consuming loads when they are being
most energy-consuming loads when they are being utilized. utilized.

Figure 5. The hourly energy consumption throughout the year.

Figure 5. The
Figure 5. The hourly
hourly energy consumption throughout
energy consumption throughout the
the year.
year.

Figure 6. The individual device load pattern in a day.

Figure 6. The individual device load pattern in a day.
Figure 6. The individual device load pattern in a day.
4.2. Development of NILM based Models
4.2.
4.2. Development
Development of
of NILM
In this research,NILM based
based Models
Models
all compilation has been done utilizing Python 3.10, TensorFlow 2.10.1,
In
In this
this research,
research, all
and Scikit-Learn1.4.1,
all compilation
compilation has
has been
running on a desktop
been done
done utilizing
utilizing Python
with Intel(R)
Python 3.10,
3.10, TensorFlow
Core (TM) i7-7700K 4.20 GHz
TensorFlow
2.10.1,
2.10.1, and Scikit-Learn1.4.1, running on a desktop with Intel(R) Core (TM) i7-7700K 4.20
and
CPU, NVIDIA Scikit-Learn1.4.1,
GeForce GTXrunning1080 GPUon a desktop
and 32 GB with
DDR4 Intel(R)
RAM. Core (TM)
The XgBoost i7-7700K
and LSTM
4.20
GHz
GHz CPU,
models haveNVIDIA
CPU, NVIDIA GeForce
been trained on aGTX
GeForce GTX 1080 1080 GPU
GPU and
GPU, utilizing
and 32
32 GB
the CUDA
GB DDR4
DDR4 RAM.RAM. The
toolkit version 12.4. XgBoost
The XgBoost and
However, the
and
LSTM
Logisticmodels have
Regression been trained
and DTW_KNN on a GPU,
modelsutilizing the
have been CUDA
trained toolkit
on version
a CPU.
LSTM models have been trained on a GPU, utilizing the CUDA toolkit version 12.4. How- 12.4. How-
ever,
ever, the
DataLogistic
the Logistic Regression
Regression and
preprocessing lays the
and DTW_KNN
DTW_KNN models
groundwork for
models have
have been
been trained
effective model training. on
trained on aa CPU.
In the initial data
CPU.
preparation phase, handling missing values is crucial. Mean imputation (replacing missing
values with the mean of available data) or forward fill (propagating the last observed
value) can be utilized as more general and potentially effective solutions to address missing
values. These methods offer more flexibility in handling different types of missing data
while preserving the integrity of the dataset. The train-test split, typically at 80/20 ratio
ensures unbiased model assessment, additionally shuffling the data during splitting ensures
randomness and prevents any inherited order from affecting model performance. It is
worth noticing that shuffling the data is not performed when working with LSTM due
to the sequential nature of the data. This process is omitted to maintain the integrity of
the temporal relationships within the dataset, ensuring optimal performance of the LSTM
model. The model specifications for different algorithms are given in Table 2.
Electronics 2024, 13, 1420 12 of 21

Table 2. Model specifications for different algorithms.

Logistic Regression Solver: “lbfgs”, Penalty: “L2”, Class Weighted: “balanced”, Max_Iteration: 150, Data Shuffle: Yes.
DTW_KNN Number of Neighbors: 5, Window Size: 150 min (10 samples), Sample Signal Length: 360, Data Shuffle: No.
Booster: “dart”, Device: “GPU”, eta: 0.5, Max Depth: 4, Min_Child_Depth: 1, Max_Delta_Step: 1, Sub
XgBoost Sample: 0.9, Sampling Method: “Subsample”, Objective: “Binary Logistic”, Evaluation Metrics: “log loss”,
Early Stopping: 10, Data Shuffle: Yes.
Layer 1: Units: 50, Number of Features: 11
Layer 2: Dense:1, Activation: “Sigmoid”
LSTM
Optimizer: “Adam”, Learning Rate: 0.0005, Epochs: 60, Batch size: 32, Early Stopping: 10, metrics:
“Accuracy”, Loss: “Binary Cross Entropy”, Validation Split: 0.1, Class Weight: “Balanced”, Data Shuffle = No

Time-related features play a significant role in modeling. Extracting the hour of the
day, time interval number (with 15 min resolution), day of the week, and month provides
valuable context. Generating cosine and sine values for these above-mentioned features
encodes cyclic behavior and enhances models’ ability to learn from data. Additionally labeling
weekends and holidays provides further insights for predictive modeling. Manual labeling
on/off state of appliances based on specific thresholds ensures that the model can learn the
underlying patterns as these labels serve as our target variable for supervised learning.
Handling power consumption patterns, particularly for devices with consistent steady
consumption, requires a method that effectively identifies meaningful deviations in power
usage, filtering out noise and focusing on relevant changes. The approach involves estab-
lishing a baseline consumption level for devices such as sockets and lights, representing the
minimal power draw when inactive. Significant increases in power consumption beyond
this baseline are then interpreted as the device being turned on. Additionally, recognizing
that certain devices may exhibit consistent consumption patterns, such as modems, allows
for their exclusion to prevent false positives. Overall, this approach balances sensitivity in
detecting genuine “on” states with specificity in avoiding false positives, offering a practical
means to enhance energy consumption prediction models.
To avoid overfitting in our LSTM and XgBoost networks, we employ early stopping.
This method halts the training when the model fails to improve on the validation data after
several attempts. It monitors metrics such as loss or accuracy and terminates the training
prematurely. This ensures that the model performs well with new data by stopping at the
optimal moment. However, we acknowledge that including the training behavior of the
models under investigation can offer additional insights into the design process. To this end,
we have provided some examples related to the training procedures of LSTM and XgBoost
networks below. Note that some of the training sessions ended before reaching the maximum
epoch number due to the early stopping callback. Figure 7 shows the logarithmic loss curves
related to XgBoost. In Figure 8, the LSTM training and validation losses are depicted.
 LSTM and XgBoost networks below. Note that some of the training sessions ended before
reaching the maximum epoch number due to the early stopping callback. Figure 7 shows
Electronics 2024, 13, 1420
the logarithmic loss curves related to XgBoost. In Figure 8, the LSTM training and valida-
13 of 21
tion losses are depicted.

Figure
Figure 7. 7. Logarithmic
Logarithmic loss
loss curves
curves related
related to to XgBoost
XgBoost training
training procedure.
procedure.
 Electronics 2024, 13, x FOR PEER REVIEW 14 of 21
Electronics 2024, 13, 1420 14 of 21

Figure
Figure 8. LSTM
8. LSTM networks
networks training
training and
and validation
validation loss
loss errors
errors forfor some
some of the
of the appliances.
appliances.

5. 5. Results
Results andand Discussion
Discussion
AsAs previously
previously mentioned,
mentioned, our
our data
data comprise
comprise a 1-year
a 1-year aggregated
aggregated record
record of of electricity
electricity
demand,
demand, including
including both
both thethe overall
overall demand
demand andandthethe demands
demands and and consumption
consumption patterns
patterns
of of each
each appliance,
appliance, with
with a resolution
a resolution of of15 15 min.
min. Consequently,
Consequently, thethe dataset
dataset encompasses
encompasses
approximately
approximately 35,000
35,000 data
data measurements
measurements forfor
eacheach sample.
sample. WeWehave have allocated
allocated 80% 80%of of
thethe
data for training and validation, and 20% for testing purposes. It is critical to highlight
data for training and validation, and 20% for testing purposes. It is critical to highlight that
wethat
selected a 1-yearaperiod
we selected 1-yeartoperiod
capturetoall fluctuations
capture related to seasonality.
all fluctuations For example,
related to seasonality. theex-
For
sauna is mostly used during the colder seasons, while during summer,
ample, the sauna is mostly used during the colder seasons, while during summer, the the dataset records
very few instances
dataset of sauna
records very usage. This
few instances ofpattern holds true
sauna usage. Thisfor heating
pattern systems
holds as well.
true for heating sys-
As you
tems as well. truly mentioned a training time and resource usage investigation is also very
important As you truly mentioned a training time and resource usage investigation istraining
to make a fair comparison among proposed methods. To this end, the also very
time and
important RAM to resource
make a fairusage for all theamong
comparison modelsproposed
have been provided
methods. Tointhis
Table
end,3. the training
The XgBoost algorithm stands out with exceptional performance across most cases,
time and RAM resource usage for all the models have been provided in Table 3.
except for the “other” labeled group, which likely encompasses aggregated power con-
sumption or unknown
Table 3. Comparison loads. Given
of computation costitstraining
consistent
times.performance, XgBoost emerges as a
robust choice for the given task. The logistic regression demonstrates varying success rates,
Appliance
achieving Name
optimal Method
results in detecting sauna status RAMand Usage
rainwaterTraining
drainersTime Duration
but faltering in
other cases. LSTM 652 MB 104.62 s
Stove
Dynamic time warping XgBoostwith K-nearest neighbors 13.27 MB segments consumption 2.58 s curves
into 400-length samples, Logistic Regression
employing a warping 17 MB
window size of 10 0.156 s to deter-
samples
mine appliance classesLSTM based on the five nearest neighbors.
456 MB While generally
228.74 seffective,
misidentifications between appliances like washing machines and stoves indicate room
Sockets_2nd XgBoost 4.6 MB 8.07 s
for improvement—perhaps through refined feature engineering. Additionally, mislabeling
Logistic Regression 0.87 MB 0.027 s
lights_1st and lights_2nd as “others” due to their similar patterns underscores the method’s
susceptibility to mixingLSTM closely related patterns. 381 MB 141.05 s
Heating_2nd XgBoost 5.65 MB 3.97 s
Logistic Regression 1.12 MB 0.053 s
LSTM 506 MB 288.38 s
Washing Machine
XgBoost 43 MB 7.84 s
Electronics 2024, 13, 1420 15 of 21

Table 3. Comparison of computation cost training times.

Appliance Name Method RAM Usage Training Time Duration

LSTM 652 MB 104.62 s
Stove XgBoost 13.27 MB 2.58 s
Logistic Regression 17 MB 0.156 s
LSTM 456 MB 228.74 s
Sockets_2nd XgBoost 4.6 MB 8.07 s
Logistic Regression 0.87 MB 0.027 s
LSTM 381 MB 141.05 s
Heating_2nd XgBoost 5.65 MB 3.97 s
Logistic Regression 1.12 MB 0.053 s
LSTM 506 MB 288.38 s
Washing Machine XgBoost 43 MB 7.84 s
Logistic Regression 1.6 MB 0.16 s
LSTM 602 MB 289.56 s
Rainwater Drainer XgBoost 13.12 MB 7.31 s
Logistic Regression 1.15 MB 0.047 s
LSTM 925 MB 169.72 s
Sockets_1st XgBoost 60 MB 6.86 s
Logistic Regression 450 KB 0.05 s
LSTM 319 MB 160.30 s
Lights_1st XgBoost 73 MB 7.24 s
Logistic Regression 626 KB 0.054 s
LSTM 113 MB 131.14 s
Sauna XgBoost 27 MB 1.11 s
Logistic Regression 11 MB 0.027 s
LSTM 168 MB 87.52 s
Lights_2nd XgBoost 65 MB 7.37 s
Logistic Regression 328 KB 0.036 s
LSTM 226 MB 165.17 s
Hot Water pump XgBoost 32 MB 2.26 s
Logistic Regression 10.6 MB 0.16 s

By exploiting the LSTM architecture to analyze 480 samples, representing a window

spanning 120 h, the model effectively captures temporal dependencies and inherent patterns
within the dataset. The strategic handling of imbalanced labels through the implementation
of the class weight method reflects a judicious approach to mitigate bias during model
training. The adjustment of class weights ensures equitable consideration of both “on” and
“off” states, thereby averting the model’s inclination towards the predominant class and
fostering a more balanced learning process.
In the domain of NILM, the classification threshold plays a pivotal role in optimizing
the balance between precision and recall. Precision, prioritizing correctness, and risks
overlook certain energy consumption patterns, while recall, emphasizing completeness
may falsely identify non-existent appliance activations. The choice to adjust the threshold
depends on the consequences within the context of energy management. For instance, in
residential energy monitoring, minimizing false negatives is crucial to accurately detect
Electronics 2024, 13, 1420 16 of 21

appliance usage, ensuring efficient energy usage and potentially identifying malfunctioning
devices. Conversely, in commercial settings like smart buildings, reducing false positives
is essential to avoid unnecessary interventions and maintain occupants’ comfort16while
Electronics 2024, 13, x FOR PEER REVIEW of 21
optimizing energy consumption.
In the provided Figure 9 the confusion matrices indicate that the LSTM method
performed less effectively compared to other techniques. One possible reason for this
discrepancy
discrepancy could
could be
be the
the suboptimal
suboptimal sampling
sampling rate
rate of 15 min.
of 15 min. Previous
Previous studies
studies have
have
shown
shown that
that increasing
increasing the
the measurement
measurement frequency
frequency can
can significantly
significantly enhance
enhance prediction
prediction
accuracy
accuracy in
inNILM
NILMapplications.
applications.Additionally, fine-tuning
Additionally, fine-tuningthethe
LSTM method
LSTM methodthrough ar-
through
chitectural adjustments or hyperparameter tuning may further improve its performance.
architectural adjustments or hyperparameter tuning may further improve its performance.
Therefore,
Therefore, future
future research
research should
should explore
explore optimizing
optimizing the
the sampling
sampling frequency
frequency alongside
alongside
other
other methodological enhancements to maximize the effectiveness of LSTM-based
methodological enhancements to maximize the effectiveness of LSTM-based NILMNILM
approaches.
approaches.

XGBoost LSTM
Sockets_1st heating_2nd Sockets_1st heating_2nd

2657 142 5468 173 2072 727 4802 839

off

off

off

off
Actual Status

Actual Status

148 4071 477 900 857 3362 508 869

on

on

on

on
off on off on off on off on
Predicted Status Predicted Status Predicted Status Predicted Status

Logistic Regression DTW-Knn

Sockets_1st heating_2nd
Sockets_1st heating_2nd

2595 204 5504 137

off

off

1846 953 5433 208

off

off

Actual Status
Actual Status

651 3568 271 1106

on

on

892 3327 963 414

on

on

off on off on
off on off on Predicted Status Predicted Status
Predicted Status Predicted Status

Figure
Figure 9.
9. Confusion
Confusion matrices
matrices obtained
obtained for
for specific
specific threshold
threshold value.
value.

However,
However, comparative
comparativeanalyses
analyses(see
(seeTable
Table 4)
4) reveal
reveal that
that alternative
alternative models
models such
such as
as
XgBoost and DTW with KNN outperform LSTM in the specific scenario under investiga-
XgBoost and DTW with KNN outperform LSTM in the specific scenario under investigation,
tion, emphasizing
emphasizing the importance
the importance of exploring
of exploring diverse
diverse modelmodel architectures
architectures and method-
and methodologies.
ologies. Considerations
Considerations of interpretability,
of interpretability, computational
computational efficiency,
efficiency, and easeand ease of imple-
of implementation
mentation will be
will be pivotal pivotal in inappropriate
in inappropriate modelfor
model selection selection
a givenfor a given task.
task.

Table 4. Performance analysis of different ML algorithms.

Appliance Stove Washing Machine Sauna

Model Metrics Precision Recall F1-Scroe Precision Recall F1-Scroe Precision Recall F1-Scroe
DTW_KNN 0.91 0.92 0.95 0.86 0.48 0.62 0.92 0.79 0.85
XgBoost 0.87 0.73 0.78 0.87 0.74 0.79 0.94 0.93 0.93
LR 0.71 0.56 0.59 0.72 0.56 0.59 0.95 0.92 0.93
LSTM 0.63 0.95 0.69 0.58 0.92 0.63 0.76 1.00 0.84
Appliance Heating_2nd Sockets_2nd Rainwater drainer
 Electronics 2024, 13, x FOR PEER REVIEW 17 of
Electronics 2024, 13, 1420 17 of 21

DTW_KNN 0.91 analysis

Table 4. Performance 1.00of different
0.95 ML algorithms.
0.94 0.83 0.88 1.00 0.95 0.97
XgBoost 0.90 0.91 0.90 0.87 0.75 0.79 0.99 0.99 0.99
Appliance LR Stove 0.81 0.83 Washing
0.81 Machine0.81 0.88 0.82 Sauna 0.86 0.86 0.85
Model Metrics LSTMRecall
Precision 0.90
F1-Scroe 0.89
Precision 0.89
Recall 0.96F1-Scroe 0.92Precision
0.94 Recall 0.93 F1-Scroe
0.96 0.95
Appliance Sockets_1st Lights_1st Hot water pump
DTW_KNN 0.91 0.92 0.95 0.86 0.48 0.62 0.92 0.79 0.85
XgBoost Model Metrics
0.87 0.73 Precision
0.78 Recall
0.87 F1-scroe0.74 Precision
0.79 Recall 0.94
F1-scroe 0.93
Precision 0.93
Recall F1-scro
LR DTW_KNN
0.71 0.56 0.92
0.59 0.79
0.72 0.850.56 0.94 0.59 0.60 0.95 0.73 0.920.80 0.92
0.93 0.86
LSTM 0.63XgBoost0.95 0.96
0.69 0.96
0.58 0.960.92 0.99 0.63 0.98 0.76 0.99 1.000.85 0.88
0.84 0.89
Appliance LR
Heating_2nd 0.74 0.74 0.74
Sockets_2nd 0.77 0.79 0.76
Rainwater 0.70
drainer 0.71 0.70
Model Metrics LSTM
Precision Recall 0.83
F1-scroe 0.81
Precision 0.82
Recall 0.82
F1-scroe 0.73 0.76
Precision 0.60
Recall 1.00
F1-scroe 0.66
Appliance Lights_2nd Others Total Performance
DTW_KNN 0.91 1.00 0.95 0.94 0.83 0.88 1.00 0.95 0.97
XgBoost
Model Metrics
0.90 0.91
Precision
0.90
Recall
0.87
F1-scroe
0.75
Precision
0.79
Recall 0.99
F1-scroe 0.99
Precision 0.99
Recall F1-scro
LR DTW_KNN
0.81 0.83 0.90
0.81 0.82
0.81 0.820.88 0.52 0.82 1.00 0.86 0.68 0.860.93 0.85
0.85 0.86
LSTM 0.90XgBoost0.89 0.92
0.89 0.90
0.96 0.910.92 0.800.94 0.67 0.93 0.71 0.960.91 0.86
0.95 0.88
Appliance LRSockets_1st 0.86 0.88 0.87
Lights_1st 0.95 0.97 0.96 0.81
Hot water pump 0.79 0.78
LSTM 0.96 0.86 0.90 0.70 0.81 0.75 0.79 0.90 0.80
Model Metrics Precision Recall F1-scroe Precision Recall F1-scroe Precision Recall F1-scroe
DTW_KNN 0.92 0.79 0.85 Figure0.94
10 depicts the
0.60comparative
0.73 analysis0.80
of all the ML
0.92algorithms
0.86based on the a
XgBoost 0.96 0.96 0.96 0.99 0.98 0.99 0.85 0.88 0.89
curacy of identification of the appliance at the individual level. The accuracy of LSTM an
LR 0.74 0.74 0.74 0.77 0.79 0.76 0.70 0.71 0.70
XgBoost is comparable for most of the devices, however, the accuracy of LSTM is e
LSTM 0.83 0.81 0.82 0.82 0.73 0.76 0.60 1.00 0.66
tremely low for the lighting loads. On the other side, the LR algorithm has low accura
Appliance Lights_2nd results for lights, heating,
Others and rainwater drainer. The DTW-KNN
Total Performance algorithm shows com
Model Metrics Precision Recall paratively
F1-scroe better results
Precision than theF1-scroe
Recall LR algorithm, but it also
Precision has variations
Recall in accuracy r
F1-scroe
DTW_KNN 0.90 0.82
sults.
0.82
Overall,
0.52
the most consistent
1.00
results
0.68
are from
0.93
the XgBoost
0.85
algorithm.
0.86
XgBoost 0.92 0.90 0.91 There 0.80could be several
0.67 reasons
0.71 why XgBoost
0.91 outperforms
0.86 other
0.88methods. Fir
LR 0.86 0.88 XgBoost effectively
0.87 0.95 handles
0.97 various data types0.81
0.96 and missing values, which
0.79 0.78 can be bene
LSTM 0.96 0.86 cial when dealing
0.90 0.70 with0.81
different sampling
0.75 rates. In our case,
0.79 0.90the sampling
0.80 rate was qui
low. On the other hand, LSTM can be significantly influenced by the sampling rate. Add
Figure 10 tionally,
depicts the
theamount of training
comparative data has
analysis a substantial
of all impact on the
the ML algorithms performance
based on the of neur
networks due to their long-term memory capability. Sampling
accuracy of identification of the appliance at the individual level. The accuracy of LSTM andrates matter in Logistic R
XgBoost is comparable for most of the devices, however, the accuracy of LSTM is extremely boostin
gression as well. Secondly, XgBoost, as a representative of the family of gradient
algorithms,
low for the lighting loads.isOnmore
therobust to overfitting
other side, compared has
the LR algorithm to deep
low learning
accuratemodels
resultslike LSTM
This could be particularly beneficial if the data set is not very
for lights, heating, and rainwater drainer. The DTW-KNN algorithm shows comparatively large, as in our case. XgBoo
has the
better results than manyLRparameters,
algorithm, butwhich gives
it also the
has designer in
variations theaccuracy
opportunity to tune
results. the model an
Overall,
prevent
the most consistent overfitting.
results are from the XgBoost algorithm.

Figure 10. Accuracy analysis of different ML algorithms.

Figure 10. Accuracy analysis of different ML algorithms.
There could be several reasons why XgBoost outperforms other methods. First, Xg-
Boost effectively handles various data types and missing values, which can be beneficial
when dealing with different sampling rates. In our case, the sampling rate was quite low.
Electronics 2024, 13, 1420 18 of 21

On the other hand, LSTM can be significantly influenced by the sampling rate. Additionally,
the amount of training data has a substantial impact on the performance of neural networks
due to their long-term memory capability. Sampling rates matter in Logistic Regression as
well. Secondly, XgBoost, as a representative of the family of gradient boosting algorithms,
is more robust to overfitting compared to deep learning models like LSTM. This could
be particularly beneficial if the data set is not very large, as in our case. XgBoost has
many parameters, which gives the designer the opportunity to tune the model and prevent
overfitting.
In general, all algorithms have their strengths and weaknesses. For instance, LSTM has
the ability to capture long-term dependencies, but it may require a large amount of data and
computational resources. Logistic Regression is a simple and fast algorithm, but it may not
capture complex patterns in the data. DTW-KNN is good at capturing temporal patterns,
but it may be sensitive to noise and outliers. All of the above leads to the conclusion that it
is reasonable to focus on developing hybrid models that combine the strengths of different
algorithms. Moreover, improving the robustness and efficiency of existing algorithms is
valuable as well. This approach not only enhances the performance of the model but also
makes it more adaptable to various types of data and tasks.

6. Conclusions
As energy consumption monitoring becomes increasingly vital in the transition to-
wards sustainable practices, this research provides valuable guidance for the selection and
deployment of ML techniques in Non-Intrusive Load Monitoring systems. This paper
presents a thorough analysis of machine learning techniques employed by NILM through a
meticulous examination and comparison, we have elucidated the efficacy and adaptability
of various algorithms in disaggregating energy consumption data accurately. Our research
underscores the necessity of tailored approaches, emphasizing the significance of selecting
suitable models aligned with the specific characteristics and objectives of the data at hand.
By providing a nuanced understanding of the strengths and limitations inherent in different
methodologies, our study offers valuable insights that can inform the development and
implementation of more efficient NILM systems. Furthermore, our findings highlight
the multifaceted nature of NILM challenges and the complexity involved in accurately
discerning individual appliance signatures from aggregate energy data. The results of this
study indicate that the LSTM and XgBoost algorithms give the most accurate identification
results, however, XgBoost has the best results on average.
Looking ahead, as the field of NILM continues to evolve, further research and innova-
tion are warranted to address persistent challenges and capitalize on emerging opportuni-
ties. By fostering interdisciplinary collaborations and leveraging advances in data science,
artificial intelligence, and energy engineering, we can unlock new avenues for improving
the accuracy, efficiency, and scalability of NILM solutions. Ultimately, our collective efforts
aim to empower consumers with actionable insights, facilitate informed decision-making,
and promote sustainable energy consumption practices in support of a more resilient and
environmentally conscious future.

Author Contributions: Conceptualization, N.S., K.V. and O.H.; methodology, N.S., K.V. and H.N.H.;
software, K.V. and H.N.H.; validation, O.H. and J.B.; formal analysis, N.S., K.V. and O.H.; investiga-
tion, N.S. and K.V.; resources, N.S., J.B. and E.P.; data curation, K.V. and H.N.; writing—original draft
preparation, N.S., K.V. and H.N.H.; writing—review and editing, O.H., J.B. and E.P.; visualization,
K.V. and H.N.H.; supervision, O.H., J.B. and E.P.; project administration, O.H. and E.P.; funding
acquisition, O.H. All authors have read and agreed to the published version of the manuscript.
Funding: This work was supported by the Estonian Research Council grant PRG 675 and the Euro-
pean Commission through DUT Horizon Europe Partnership project FLEDGE grant No. MOB3PRT1.
Data Availability Statement: Data are contained within the article.
Conflicts of Interest: The authors declare no conflicts of interest.
Electronics 2024, 13, 1420 19 of 21

References
1. Ji, X.; Huang, H.; Chen, D.; Yin, K.; Zuo, Y.; Chen, Z.; Bai, R. A Hybrid Residential Short-Term Load Forecasting Method Using
Attention Mechanism and Deep Learning. Buildings 2023, 13, 72. [CrossRef]
2. Jawad, M.; Asghar, H.; Arshad, J.; Javed, A.; Qureshi, M.B.; Ali, S.M.; Shabbir, N.; Rassõlkin, A. A Novel Renewable Powered
Stand-Alone Electric Vehicle Parking-Lot Model. Sustain. Energy Grids Netw. 2023, 33, 100992. [CrossRef]
3. Mirza, Z.T.; Anderson, T.; Seadon, J.; Brent, A. A thematic analysis of the factors that influence the development of a renewable
energy policy. Renew. Energy Focus 2024, 49, 100562. [CrossRef]
4. Abumohsen, M.; Owda, A.Y.; Owda, M. Electrical Load Forecasting Using LSTM, GRU, and RNN Algorithms. Energies 2023, 16,
2283. [CrossRef]
5. Azizi, E.; Ahmadiahangar, R.; Rosin, A.; Bolouki, S. Characterizing Energy Flexibility of Buildings with Electric Vehicles and
Shiftable Appliances on Single Building Level and Aggregated Level. Sustain. Cities Soc. 2022, 84, 103999. [CrossRef]
6. Azizi, E.; Shotorbani, A.M.; Hamidi-Beheshti, M.T.; Mohammadi-Ivatloo, B.; Bolouki, S. Residential Household Non-Intrusive
Load Monitoring via Smart Event-Based Optimization. IEEE Trans. Consum. Electron. 2020, 66, 233–241. [CrossRef]
7. Opoku, R.; Obeng, G.Y.; Adjei, E.A.; Davis, F.; Akuffo, F.O. Integrated System Efficiency in Reducing Redundancy and Promoting
Residential Renewable Energy in Countries without Net-Metering: A Case Study of a SHS in Ghana. Renew. Energy 2020, 155,
65–78. [CrossRef]
8. Azizi, E.; Beheshti, M.T.H.; Bolouki, S. Event Matching Classification Method for Non-Intrusive Load Monitoring. Sustainability
2021, 13, 693. [CrossRef]
9. Lizana, J.; Friedrich, D.; Renaldi, R.; Chacartegui, R. Energy Flexible Building through Smart Demand-Side Management and
Latent Heat Storage. Appl. Energy 2018, 230, 471–485. [CrossRef]
10. Kaselimi, M.; Protopapadakis, E.; Voulodimos, A.; Doulamis, N.; Doulamis, A. Multi-Channel Recurrent Convolutional Neural
Networks for Energy Disaggregation. IEEE Access 2019, 7, 81047–81056. [CrossRef]
11. Schirmer, P.A.; Mporas, I. Non-Intrusive Load Monitoring: A Review. IEEE Trans. Smart Grid 2023, 14, 769–784. [CrossRef]
12. Angelis, G.F.; Timplalexis, C.; Salamanis, A.I.; Krinidis, S.; Ioannidis, D.; Kehagias, D.; Tzovaras, D. Energformer: A New
Transformer Model for Energy Disaggregation. IEEE Trans. Consum. Electron. 2023, 69, 308–320. [CrossRef]
13. Azizi, E.; Beheshti, M.T.H.; Bolouki, S. Quantification of Disaggregation Difficulty with Respect to the Number of Smart Meters.
IEEE Trans. Smart Grid 2022, 13, 516–525. [CrossRef]
14. Shabbir, N.; Kutt, L.; Jawad, M.; Iqbal, M.N.; Ghahfarokhi, P.S. Forecasting of Energy Consumption and Production Using
Recurrent Neural Networks. Adv. Electr. Electron. Eng. 2020, 18, 190–197. [CrossRef]
15. Ding, G.; Wu, C.; Wang, Y.; Liang, Y.; Jiang, X.; Li, X. A Novel Non-Intrusive Load Monitoring Method Based on Quantum
Particle Swarm Optimization Algorithm. In Proceedings of the 2019 11th International Conference on Measuring Technology and
Mechatronics Automation (ICMTMA), Qiqihar, China, 28–29 April 2019; pp. 230–234. [CrossRef]
16. Nuran, A.S.; Murti, M.A.; Suratman, F.Y. Non-Intrusive Load Monitoring Method for Appliance Identification Using Random
Forest Algorithm. In Proceedings of the 2023 IEEE 13th Annual Computing and Communication Workshop and Conference
(CCWC), Las Vegas, NV, USA, 8–11 March 2023; pp. 754–758. [CrossRef]
17. Raiker, G.A.; Reddy, B.S.; Umanand, L.; Agrawal, S.; Thakur, A.S.; Ashwin, K.; Barton, J.P.; Thomson, M. Internet of Things Based
Demand Side Energy Management System Using Non-Intrusive Load Monitoring. In Proceedings of the 2020 IEEE International
Conference on Power Electronics, Smart Grid and Renewable Energy (PESGRE2020), Cochin, India, 2–4 January 2020; pp. 5–9.
[CrossRef]
18. Al-Khadher, O.; Mukhtaruddin, A.; Hashim, F.R.; Azizan, M.M.; Mamat, H.; Mani, M. Comparison of Non-Intrusive Load
Monitoring Supervised Methods Using Harmonics as Feature. In Proceedings of the 2022 International Conference on Electrical,
Computer and Energy Technologies (ICECET), Prague, Czech Republic, 20–22 July 2022; pp. 1–6. [CrossRef]
19. Moradzadeh, A.; Zeinal-Kheiri, S.; Mohammadi-Ivatloo, B.; Abapour, M.; Anvari-Moghaddam, A. Support Vector Machine-
Assisted Improvement Residential Load Disaggregation. In Proceedings of the 2020 28th Iranian conference on electrical
engineering (ICEE), Tabriz, Iran, 4–6 August 2020; pp. 1–6. [CrossRef]
20. Shiddieqy, H.A.; Hariadi, F.I.; Adijarto, W. Plug-Load Classification Based on CNN from V-I Trajectory Image Using STM32. In
Proceedings of the 2021 International Symposium on Electronics and Smart Devices (ISESD), Bandung, Indonesia, 29–30 June 2021.
[CrossRef]
21. Giannuzzo, L.; Minuto, F.D.; Schiera, D.S.; Lanzini, A. Reconstructing Hourly Residential Electrical Load Profiles for Renewable
Energy Communities Using Non-Intrusive Machine Learning Techniques. Energy AI 2024, 15, 100329. [CrossRef]
22. Hernández, Á.; Nieto, R.; de Diego-Otón, L.; Pérez-Rubio, M.C.; Villadangos-Carrizo, J.M.; Pizarro, D.; Ureña, J. Detection of
Anomalies in Daily Activities Using Data from Smart Meters. Sensors 2024, 24, 515. [CrossRef] [PubMed]
23. Koasidis, K.; Marinakis, V.; Doukas, H.; Karamaneas, A.; Nikas, A.; Doumouras, N. Equipment- and Time-Constrained Data
Acquisition Protocol for Non-Intrusive Appliance Load Monitoring. Energies 2023, 16, 7315. [CrossRef]
24. Hosseini, S.S.; Delcroix, B.; Henao, N.; Agbossou, K.; Kelouwani, S. Towards Feasible Solutions for Load Monitoring in Quebec
Residences. Sensors 2023, 23, 7288. [CrossRef] [PubMed]
25. Shareef, H.; Asna, M.; Errouissi, R.; Prasanthi, A. Rule-Based Non-Intrusive Load Monitoring Using Steady-State Current
Waveform Features. Sensors 2023, 23, 6926. [CrossRef]
Electronics 2024, 13, 1420 20 of 21

26. Azizi, E.; Beheshti, M.T.H.; Bolouki, S. Appliance-Level Anomaly Detection in Nonintrusive Load Monitoring via Power
Consumption-Based Feature Analysis. IEEE Trans. Consum. Electron. 2021, 67, 363–371. [CrossRef]
27. Welikala, S.; Dinesh, C.; Ekanayake, M.P.B.; Godaliyadda, R.I.; Ekanayake, J. Incorporating Appliance Usage Patterns for
Non-Intrusive Load Monitoring and Load Forecasting. IEEE Trans. Smart Grid 2019, 10, 448–461. [CrossRef]
28. Tao, Y.; Qiu, J.; Lai, S.; Wang, Y.; Sun, X. Reserve Evaluation and Energy Management of Micro-Grids in Joint Electricity Markets
Based on Non-Intrusive Load Monitoring. IEEE Trans. Ind. Appl. 2023, 59, 207–219. [CrossRef]
29. Ma, C.; Yin, L. Deep Flexible Transmitter Networks for Non-Intrusive Load Monitoring of Power Distribution Networks. IEEE
Access 2021, 9, 107424–107436. [CrossRef]
30. Garcia, F.D.; Souza, W.A.; Diniz, I.S.; Marafão, F.P. NILM-Based Approach for Energy Efficiency Assessment of Household
Appliances. Energy Inform. 2020, 3, 10. [CrossRef]
31. Liu, H.; Wu, H.; Yu, C. A Hybrid Model for Appliance Classification Based on Time Series Features. Energy Build. 2019, 196,
112–123. [CrossRef]
32. Gopinath, R.; Kumar, M.; Prakash Chandra Joshua, C.; Srinivas, K. Energy Management Using Non-Intrusive Load Monitoring
Techniques—State-of-the-Art and Future Research Directions. Sustain. Cities Soc. 2020, 62, 102411. [CrossRef]
33. Du, Z.; Yin, B.; Zhu, Y.; Huang, X.; Xu, J. A NILM Load Identification Method Based on Structured V-I Mapping. Sci. Rep. 2023,
13, 21276. [CrossRef]
34. Reddy, R.; Garg, V.; Pudi, V. A Feature Fusion Technique for Improved Non-Intrusive Load Monitoring. Energy Inform. 2020, 3, 9.
[CrossRef]
35. Ruano, A.; Hernandez, A.; Ureña, J.; Ruano, M.; Garcia, J. NILM Techniques for Intelligent Home Energy Management and
Ambient Assisted Living: A Review. Energies 2019, 12, 2203. [CrossRef]
36. Chen, T.; Guestrin, C. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd Acm Sigkdd International Conference
on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 785–794. [CrossRef]
37. He, K.; Stankovic, V.; Stankovic, L. Building a Graph Signal Processing Model Using Dynamic Time Warping for Load Disaggre-
gation. Sensors 2020, 20, 6628. [CrossRef]
38. Shabbir, N.; Ahmadiahangar, R.; Rosin, A.; Astapov, V.; Kilter, J. Short-Term PV Energy Generation Forecasting Using Deep
Learning. Telematique 2022, 21, 5844–5850.
39. Wang, R.; Tao, D. DTW-KNN Implementation for Touch-Based Authentication System. In Proceedings of the 2019 5th International
Conference on Big Data Computing and Communications (BIGCOM), QingDao, China, 9–11 August 2019; pp. 318–322. [CrossRef]
40. Feng, Z.; Wang, M.; He, J.; Xiao, W. Real-Time Equipment State Monitoring and Cycle Time Calculation Method Based on
DTW-KNN. In Proceedings of the 2023 8th International Conference on Intelligent Computing and Signal Processing (ICSP),
Xi’an, China, 21–23 April 2023; pp. 1350–1353. [CrossRef]
41. Angelis, G.F.; Timplalexis, C.; Krinidis, S.; Ioannidis, D.; Tzovaras, D. NILM Applications: Literature Review of Learning
Approaches, Recent Developments and Challenges. Energy Build. 2022, 261, 111951. [CrossRef]
42. Imtiaz, M.A.; Raja, G. Isolated Word Automatic Speech Recognition (ASR) System Using MFCC, DTW & KNN. In Proceedings
of the 2016 Asia Pacific Conference on Multimedia and Broadcasting (APMediaCast), Bali, Indonesia, 17–19 November 2017;
pp. 106–110. [CrossRef]
43. Yang, N.C.; Sung, K.L. Non-Intrusive Load Classification and Recognition Using Soft-Voting Ensemble Learning Algorithm with
Decision Tree, K-Nearest Neighbor Algorithm and Multilayer Perceptron. IEEE Access 2023, 11, 94506–94520. [CrossRef]
44. Shabbir, N.; Ahmadiahangar, R.; Rosin, A.; Jawad, M.; Kilter, J.; Martins, J. XgBoost Based Short-Term Electrical Load Forecasting
Considering Trends & Periodicity in Historical Data. In Proceedings of the 2023 IEEE International Conference on Energy
Technologies for Future Grids (ETFG), Wollongong, Australia, 3–6 December 2023; pp. 1–6. [CrossRef]
45. Chen, Z.; Chen, J.; Xu, X.; Peng, S.; Xiao, J.; Qiao, H. Non-Intrusive Load Monitoring Based on Feature Extraction of Change-Point
and Xgboost Classifier. In Proceedings of the 2020 IEEE 4th Conference on Energy Internet and Energy System Integration (EI2),
Wuhan, China, 30 October–November 2020; pp. 2652–2656. [CrossRef]
46. Semmelmann, L.; Henni, S.; Weinhardt, C. Load Forecasting for Energy Communities: A Novel LSTM-XGBoost Hybrid Model
Based on Smart Meter Data. Energy Inform. 2022, 5, 24. [CrossRef]
47. Mo, H.; Sun, H.; Liu, J.; Wei, S. Developing Window Behavior Models for Residential Buildings Using XGBoost Algorithm. Energy
Build. 2019, 205, 109564. [CrossRef]
48. Bipin Nair, B.J.; Yadhukrishnan, S.; Manish, A. A Comparative Study on Document Images Classification Using Logistic
Regression and Multiple Linear Regressions. In Proceedings of the 2023 Second International Conference on Augmented
Intelligence and Sustainable Systems (ICAISS), Trichy, India, 23–25 August 2023; pp. 1096–1104. [CrossRef]
49. Shabbir, N.; Kütt, L.; Jawad, M.; Husev, O.; Rehman, A.U.; Gardezi, A.A.; Shafiq, M.; Choi, J.G. Short-Term Wind Energy
Forecasting Using Deep Learning-Based Predictive Analytics. Comput. Mater. Contin. 2022, 72, 1017–1033. [CrossRef]
50. Shabbir, N.; Kutt, L.; Raja, H.A.; Ahmadiahangar, R.; Rosin, A.; Husev, O. Machine Learning and Deep Learning Techniques for
Residential Load Forecasting: A Comparative Analysis. In Proceedings of the IEEE 62nd International Scientific Conference on
Power and Electrical Engineering of Riga Technical University, Riga, Latvia, 15–17 November 2021; pp. 1–5.
51. Mellit, A.; Pavan, A.M.; Lughi, V. Deep Learning Neural Networks for Short-Term Photovoltaic Power Forecasting. Renew. Energy
2021, 172, 276–288. [CrossRef]
Electronics 2024, 13, 1420 21 of 21

52. Hossen, T.; Nair, A.S.; Chinnathambi, R.A.; Ranganathan, P. Residential Load Forecasting Using Deep Neural Networks (DNN).
In Proceedings of the 2018 North American Power Symposium (NAPS), Fargo, ND, USA, 9–11 September 2019. [CrossRef]
53. Shabbir, N.; Ahmadiahangar, R.; Rosin, A.; Husev, O.; Jalakas, T.; Martins, J. Residential DC Load Forecasting Using Long
Short-Term Memory Network (LSTM). In Proceedings of the 2023 IEEE 11th International Conference on Smart Energy Grid
Engineering (SEGE), Oshawa, ON, Canada, 13–15 August 2023; pp. 131–136. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.