See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/337608853

Transformation of Smart Grid using Machine Learning

Conference Paper · November 2019

DOI: 10.1109/AUPEC48547.2019.211809

CITATIONS READS

3 1,417

3 authors:

Salahuddin A. Azad Fariza Sabrina

Central Queensland University Central Queensland University
19 PUBLICATIONS   114 CITATIONS    37 PUBLICATIONS   117 CITATIONS   

SEE PROFILE SEE PROFILE

Saleh A. Wasimi
Central Queensland University
22 PUBLICATIONS   702 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Smart Grid View project

All content following this page was uploaded by Salahuddin A. Azad on 07 May 2020.

The user has requested enhancement of the downloaded file.

 Transformation of Smart Grid using Machine
Learning
Salahuddin Azad Fariza Sabrina Saleh Wasimi
School of Engineering and School of Engineering and School of Engineering and
Technology Technology Technology
Central Queensland University Central Queensland University Central Queensland University
Melbourne, Australia Sydney, Australia Melbourne, Australia
[email protected] [email protected] [email protected]

Abstract— With the advent of distributed and renewable  Demand management: Smart grid allows customers and
energy sources, maintaining the stability of power grid is network operators to manage electricity consumption to
becoming increasingly difficult. Traditional power grid can be reduce peak demand and avoid network overloading.
transformed into a smart grid by augmenting it with
information and communication technologies, and machine  Network stability: Smart grid maintains the balance of
intelligence. Machine learning and artificial intelligence can supply and demand and regulates the voltage by
enable smart grid to make intelligent decisions and respond to varying generation and managing demand.
sudden changes in customer demands, power outages, sudden  Network reliability: Smart grid allows network
drops and rises in renewable energy output or any other operators to anticipate and locate faults and outages
catastrophic events. Machine learning can also help capture remotely that could lead to quicker restoration of
customer consumption patterns, forecast energy demand and power.
power generation of intermittent sources, and predict
equipment failures. Reinforced learning can aid in making  Empowering consumers: Smart grid makes real-time
energy dispatch decisions and activate demand management consumption of data available so that customer can
signals in order to maintain balance of power supply and control their consumption to take advantage of the off-
demand. The usage of wireless technologies in smart grid peak price and reduce power bill.
renders it vulnerable to cyber security threats. With the  Integration of renewable energy sources: Smart grid
increase in data volume, it is now possible to employ machine facilitates integration of intermittent renewable energy
learning for the detection and prevention of anomalous sources by making energy dispatch decisions and
behaviour, intrusion, cyber-attacks, and malicious activities as
controlling loads in real-time.
well as data authentication. This paper reviews the application
of different machine learning approaches that aims at Machine learning is a branch of artificial intelligence
enhancing the stability, reliability, security, efficiency and where machines learn automatically from data without the
responsiveness of smart grid. This paper also discusses some of direct involvement of humans [2]. Machine learning
the challenges in implementing machine learning solutions for algorithms embody the techniques that look for patterns and
smart grid. interrelations in the data and build a model that represent
those patterns and interrelations. The learning experience
Keywords—smart grid, machine learning, transformer loss of
enables the machine to predict future events based on past
life, power quality, energy dispatch decision, electricity market
operation, smart grid security
examples. Machine learning can be supervised or
unsupervised [3]. In the supervised machine learning, the
I. INTRODUCTION algorithm is provided with training data that has
outcomes/labels. The algorithm builds a model/function
With the introduction of distributed and renewable from the pattern buried in the data. When presented with
energy sources, it is becoming increasingly difficult to new data, the algorithm infers outcomes based on the
maintain the balance between demand and supply of power model/function already built. In unsupervised learning, there
as well as the quality of power in the electricity network. As is no outcome/label provided with the data. The algorithm
the traditional power grid is not designed to handle finds the unknown structure or pattern in the data without
bidirectional power flow, electricity networks are struggling any information on how to name or label it.
to handle backflow of power from distributed generation
sources such wind and solar. The intermittent nature of the The supervised learning algorithms generally fall into
renewable energy sources is also making it difficult to two categories – classification and regression. The
maintain stable power flow in the electricity network. The classification problem separates data into predetermined
traditional power grid can be transformed into an intelligent, labels of outcome. In classification problem, the outcome
automated and responsive power grid by augmenting it with variable has a finite number of categories (such as
information and communication technologies, and machine male/female, low/moderate/high). The regression problem
intelligence. This type of grid is often termed as smart grid. deals with predicting the real value of an outcome (such as
The motivation behind developing smart grid is to ensure age, price). Unlike classification problem, the outcome
stable, reliable, efficient, economical and sustainable variable in regression problem has continuous values. The
generation, distribution and usage of conventional and most common supervised learning algorithms are artificial
renewable power [1]. Smart grid can offer the following neural network (ANN), support vector machine (SVM),
benefits to the generators, transmission and distribution decision tree, k-nearest neighbours (KNN) etc.
operators, and consumers.

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

 The unsupervised learning algorithms also generally fall the features, only drops the less important features. The
into two categories – clustering and association. In features extraction by contrast, similar to principal
clustering problem, the algorithm endeavours to discover the component analysis, transforms the features into a smaller
intrinsic grouping of data based on the similarity of data set of features that have fewer features than the original set
points (such as grouping of patients based on symptoms). of features.
Popular clustering algorithms are k-means clustering,
hierarchical clustering, DBSCAN, OPTICS etc. In Machine learning and artificial intelligence can enable
association problem, the co-occurrence of events with high smart grid to make intelligent decisions and respond to
frequency in large dataset is discovered and expressed as sudden changes in customer demands, power outages,
association rules (such as customers who buy bread also buy sudden drops and rises in renewable energy output or any
egg). The most frequently used association rule algorithms other catastrophic events. The smart meters installed in the
are AIS, SETM, Apriori etc. customer premises, and other measurement devices in the
power system, generate a huge amount of data that could be
There is another kind of machine learning algorithm that fed into the machine learning algorithms to capture
fall between supervised and unsupervised machine learning customer consumption patterns. The use of machine
algorithms, and thus called semi-supervised learning learning to forecast energy demand and power generation of
algorithms [3]. This sort of learning algorithms is used when intermittent sources are among the most common
vast majority of data are not labelled, only a small portion of applications of machine learning in the power grid. Machine
the data is labelled. This situation arises when labelling of learning can be used to predict any equipment failure by
data requires high expertise and/or the labelling is resource- analysing industry-wide equipment failure data, thus helping
intensive. in asset maintenance and portfolio management. Besides
supervised machine learning, unsupervised machine
Reinforcement learning is about learning to how to take learning can have applications in the smart grid such as
the best action in a certain environment so that the creating energy demand profiles through clustering.
cumulative reward is maximised [4]. In this type of learning, Reinforced learning algorithms can be used to make energy
an agent learns through trial and error based on the reward it dispatch decisions and activate demand management signals
receives as a result of its actions. The difference between in order to maintain balance of power supply and demand.
supervised learning and reinforcement learning is that the
labels in supervised learning represent the answer, while in This paper endeavours to review the application of
reinforcement learning, the agent has to decide the best different machine learning approaches that aims at
action by correlating the delayed return with the action. enhancing the stability, reliability, security, efficiency and
Reinforced learning can be formulated as a Markov responsiveness of smart grid. The rest of the paper is
Decision Process (MDP) [5] with a finite set of states, a organised as follows: Section 2 reviews the use of machine
fixed set of actions and rewards. An agent in MDP selects learning to estimate transformer loss of life. Section 3
an action to make transition from current state to another reviews the machine learning methods to detect power
state and earn rewards. MDP enables agents to choose the quality events. Section 4 reviews the prospect of machine
best action in each state so that it can earn the maximum learning to make optimal energy dispatch decisions aiming
reward. The collection of optimal actions by an agent at balancing power supply and demand. Section 5 reviews
constitutes a policy. Q-learning is a reinforcement learning the efficacy of using machine learning in improving
that learns an action-value function which yields the electricity market operations. Section 6 reviews the machine
expected utility of a specified action in a specified state [6]. learning techniques to detect vulnerabilities and threats in
smart gird. Section 7 discusses some of the challenges in
Unlike statistical analysis methods, machine learning implementing machine learning solutions for smart grid.
requires vast amount of data to learn the interrelations and Finally, Section 8 concludes the paper.
patters in the data. While most statistical methods require
certain assumptions to hold about the distribution of data, II. ESTIMATING TRANSFORMER LOSS OF LIFE
machine learning algorithms can be applied to any data.
However, data quality is very crucial for machine learning Proactive maintenance of power system equipment
to build a useful model from the data. In many situations, involves prediction of equipment failure and replace or
the available data cannot be directly fed into the machine repair those equipment before the failure occurs, thus
learning algorithms. The data has to go through a helping to minimise the unplanned downtime and
preprocessing stage where it is cleaned, transformed, and unexpected disruption of power supply. However,
enriched before being fed into the machine learning prioritising the equipment for replacement or repair so that
algorithm. The number of attributes or features in the data limited resources are spent towards the equipment that
plays an important role in the design of a machine learning requires the most urgent attention is a challenging task. The
pipeline. When the number of features is too high, the machine learning models can predict the mean time between
number of data points required in the training phase would failures (MTBF) and rank feeders accordingly so that the
be exceedingly high and the model could be exceptionally feeder which is most likely to fail gets replaced or repaired
complex. This is often referred to as the curse of first.
dimensionality [7] for machine learning. To decrease the Transformers are vital part of the electricity network and
complexity of the model and the volume of data required, failure of transformers can jeopardise the reliability of the
the dimensionality needs to be reduced. There are two well- network. The repair and maintenance of transformers are
known approaches to reduce the dimensionality of data: costly as well as time consuming, and therefore, it pivotal to
feature selection and feature extraction [8]. Feature estimate the remaining life and replace the ageing
selection selects a subset of features that are most relevant transformer before the failure happens. The estimation of
for model construction. Feature selection doesn’t transform transformer loss of life is an important part of transformer
asset management. The insulation of a transformer is the used to extract features of current and voltage signals [16].
most vulnerable component, and hence, the life of a Other machine learning algorithms that are used for fault
transformer is dictated by the condition of its insulation. The detection are decision tree, k-nearest neighbours, ELM,
load profile and ambient temperature have an effect on the bagged tree ensemble, fuzzy logic etc. [17]. Machine
aging of transformer insulation. The prediction of load learning techniques can also be used to determine fault
profile and temperature using historical data can help locations. Support vector regressions (SVR), ANN, back
estimate the time to insulation failure for the transformer propagation ANN, k-nearest neighbours are commonly used
[9]. machine learning algorithms for this purpose. The features
are extracted using fast Fourier transform (FFT), wavelet
Majzoobi at al. [10] developed a static model for hourly packet transform (WPT).
estimation of transformer loss of life based on adaptive
network-based fuzzy inference system (ANFIS) which is a Complex event processing (CEP) is a Big Data analytics
combination of ANN based machine learning and fuzzy technique which analyses trends, patterns and correlation in
inference system. Mahoor and Khodaei [11] further the data to identify complex situations. CEP can be quite
integrated ANFIS and radial basis function (RBF) to useful for smart grid as it would allow smart grid to analyse
improve the accuracy of estimation even more. The data complex situations and respond in real-time. An
fusion techniques that were used to integrate the outputs of architectural framework based on Lambda architecture for
ANFIS and RBF are ordered weighted averaging (OWA) complex event processing in smart grid was presented in
and sequential Kalman filter. [18].
III. POWER QUALITY EVENT DETECTION IV. MAKING ENERGY DISPATCH DECISIONS
Machine learning can be used to predict outage or power The power grid operators rely heavily on forecasting of
quality events (PQE) in the power grid by using validated the electricity demand to adjust power generation to meet
models [12]. Self-healing grids can be developed by the demand and to avoid power system overload. The
automatically detecting and mitigating fault events with the introduction of renewable and distributed energy sources is
aid of machine learning. Knowledge of fault signatures and making it increasingly difficult to match the power
their development combined with machine learning can generation with demand. Nowadays, it is essential to
facilitate the detection of fault events in the power grid [13]. forecast both electricity demand and distributed generation
Traditionally power system states are estimated using to know how much power needs to be produced by the
models consisting of high dimensional nonlinear equations; conventional generators to meet the net demand.
however, the computational complexity involved makes it
very time consuming [14]. Consequently, these models are Battery storage plays an important role in smoothing out
unable to predict the system state (such as frequency) soon the fluctuations in energy demand and distributed
enough after a disturbance has happened. Simplified models generations. Batteries can store power during periods of
have been developed that work much faster than the original excess energy generation, and release it later to fill the gap
models, but these ignore some of the factors affecting the during the period of generation deficit. A consumer can
system states, and hence, produce unreliable results in some choose to consume power from any of the three sources –
instances. Machine learning appears to be a viable solution power grid, battery or PV, depending on the grid electricity
for system state prediction as it is not as unreliable as the price and availability of power from the battery or PV. The
simplified models, and on the other hand, it doesn’t need to consumer can sell excess energy from PV or battery when
perform complex calculations every time it needs to the feed-in tariff is higher and store excess energy from PV
estimate the system state. The machine learning models in the battery or consume it when the feed-in tariff is lower.
need to be trained with historical time series data in order to Similarly, the consumer can consume electricity from the
enable it to distinguish fault events from normal situations. power grid during off-peak period when the price is low and
The training process may be time consuming, but once use up stored energy from battery during peak period when
trained, the machine learning models can reliably predict the the price is high.
system state within a short period of time. A huge number of heuristic algorithms (such as particle
Ucar at al. [15] proposed a method to detect power swarm optimisation, genetic algorithm) and game theory
quality events using extreme learning model (ELM). ELM is based methods have been developed to make optimal energy
a single feedforward neural network which consists of a dispatch decisions in smart grid [19],[20],[21].
single layer of hidden nodes. The weights between the input Reinforcement learning based energy management
and hidden layers are randomly assigned and remain fixed, algorithms can also make optimal energy dispatch decisions
while weights between the hidden and output layer is to reduce the cost of energy for the consumers. A Q-learning
based energy management algorithm was developed in [22],
updated during the training phase. Due to having a single
layer, the complexity of ELM is very low. Therefore, it can which learns to make better energy dispatch decisions
be trained extremely fast, while exhibiting acceptable through experiences without having any prior knowledge.
performance. Simulation results using real-life data suggest that the
proposed algorithm can significantly reduce the total energy
Fault detection is a research area where machine learning cost based only on current information, while heuristic and
has gained prominence. SVM and ANN have been game theory based methods rely on future energy demand
extensively used to classify the faults. The features from forecast to make optimal energy dispatch decisions.
fault-induced current and voltage signals are generally
extracted using discrete wavelet transform (DWT) which V. ELECTRICITY MARKET OPERATIONS
can extract the characteristics of a signal in a time window. Maintaining balance between supply and demand is one
Fast discrete orthonormal S-transform (FDOST) can also be of the key goals of smart grid. As mentioned earlier,
increasing penetration of renewable and distributed sources vulnerable to cyber security threats. For reliable operation of
has made it harder to find the balance. Various demand smart grid, security and privacy need to be ensured both at
management and dynamic pricing schemes have physical and cyber level [29]. The security vulnerabilities of
complicated the prediction of energy usage patterns [23]. smart grid could be categorised as process control security,
Electricity market operators match the demand for smart meter security, smart grid communication security,
electricity with supply from generators in five minutes, 30 cyber security etc [26]. The potential security threats to
minutes or one hour period. A more efficient and flexible smart grid include hacking, meter data tampering or meter
approach is to introduce automated electricity brokers. fraud, injection of false information/data/commands, data
These brokers can operate in a localised market to eliminate stealing, privacy breaches, denial-of-service (DDoS) attacks,
the inefficiencies arising from wide-area transmissions. man-in-the-middle attacks, meter viruses and compromise
Automated brokers can effectively trade in two different but of physical security [27],[30]. To develop a secure and
interconnected markets to buy electricity from producers reliable smart grid, it is essential to have a security system
and sell electricity to retailers. The challenge faced by a that can prevent different types of cyber-attacks. With the
broker is to balance the demand and supply of power and increase in data volume, machine learning could be
make profit while competing with other brokers. efficiently used for the detection and presentation of
anomalous behaviour, intrusion, cyber-attacks, and
Q-learning was successfully used in [24] to make an malicious activities (including virus) as well as data
automated broker learn an effective strategy to choose an authentication.
optimal action in each state in a tariff market. As the price
of electricity in real market is continuous, there is unlimited Parvez et al. [27] proposed a two-level security scheme
possibility of prices. To limit the state space, the price range for smart meters where a localization-based key
was divided into some discrete values. Also, the energy management system was used for data encryption and KNN
portfolio was kept limited to a small set of states to control algorithm was used for meter authentication. In this system,
the state space. a maximum likelihood estimator (MLE) based on received
signal strength (RSS) of radio signal was used for the
Recurrent deep Q-learning (DQN) can address localization of a meter. This system creates a local
continuous state space and discrete action space. A DQN positioning map based on the signal strength, where each
based multiagent autonomous broker was employed in [25] meter has its own coordinate. Each meter has a secret key
to trade power in a local tariff market. As the tariff market associated with its coordinates and this key is used along
prices are inherently continuous and temporal, a long short- with a random key index for data encryption to prevent
term memory (LTSM) based DQN was employed to enable potential data hacking.
a multiagent broker to learn pricing strategies. LTSM has
excellent capability in modelling sequential data. The paper Kurt et al. [31] proposed an online attack/anomaly
clustered customers according to usage patterns to limit the detection framework using a model-free reinforcement
state space. Due to multiagent nature of the broker, a learning approach for partially observable MDP. Using this
negative reward of a suboptimal action by an individual approach, an unknown attack type can be detected without
agent may be concealed by the global reward. The authors requiring the knowledge of any previous attack model. The
proposed a reward reshaping method to identify the simulation results demonstrate that the proposed solution
implication of actions taken by each agent so that each agent can efficiently detect cyber-attacks, but some further
is aware of the negative consequence of its suboptimal improvement could be done by developing sophisticated
action. memory management techniques, and incorporating
linear/non-linear function approximation techniques and
VI. SECURING SMART GRID deep reinforcement learning to enhance performance.
Smart grid has two aspects of infrastructure - cyber and Zhang et al. [32] investigated different machine learning
physical infrastructure and consists of several complex algorithms that could be used for efficient detection and
components which need to be interconnected for smooth defense of DDoS attacks. Features that are typically used to
operation [26]. One of the essential components of smart detect anomalies in the traffic are number of packets,
grid is smart meter which collects energy consumption data average packet size, time interval variance, packet size
and sends to the service provider. Smart meters are designed variance, packet rate and bit rate. The authors suggest that
to communicate with the service provider using an advanced machine learning algorithms such as random forest (RF) and
metering infrastructure (AMI) network. AMI provides naive Bayes (NB) could be used for achieving superior
facilities to automate metering, monitoring, and controlling performance in classifying malicious and normal traffic.
of power distribution/outage management through a wireless
network [27]. In the AMI network, millions of smart meters Karimipour et al. [33] proposed a real-time anomaly
communicate with the local utility service provider/control detection method for false data injection (FDI) attacks. In
centre using a bidirectional network which can be a mesh, their work, the authors took an unsupervised approach based
hierarchical, or hybrid topology. Internet of Things (IoT) on statistical correlation between measurements to detect
could also be integrated in all major components of smart anomalies in smart grid. Symbolic dynamic filtering (SDF)
grid such as power generation, transmission, distribution, was used for feature extraction to discover usual interactions
and utilisation. among different smart grid subsystems. A Dynamic
Bayesian network (DBN) based learning model was
Seamless communication through the usage of wireless employed to detect unobservable cyber-attacks using free
technology is a core feature of today’s smart grid. This energy as an anomaly index.
communication involves device to device, device to cloud,
device to gateway communication to name a few [28]. The Ozay et al. [34] employed supervised and semi-supervised
usage of wireless technologies in smart grid makes it more machine learning algorithms for FDI attack detection. Four
different types of learning algorithms such as perceptron, variable, and (4) complex nonlinear dependency among
KNN, SVM, and sparse logistic regression were used as variables.
supervised learning methods for attack detection. Semi-
supervised SVM was used as a semi-supervised learning Efficient classification/characterisation of threat and
method for attack detection. The authors used decision and source of threat is very important for accurate detection of
feature level fusion algorithms as well as both batch and security attacks in smart grid [40]. With new types of
online learning methods for classification of measurements. attacks evolving over time, the classification might need to
The experimental results suggest that the state-of-the-art be regularly updated. Hence, the challenge is to develop a
supervised machine learning algorithms can detect attacks robust classification mechanism that can detect different
more efficiently than state vector estimation (SVE) based types of evolving attacks reliably. As mentioned earlier,
algorithms for detection of different types of attacks. unavailability of past labelled data for using supervised
Perceptron is less sensitive and KNN is more sensitive to the learning-based approaches is a challenge, which makes
system size than other algorithms. Moreover, KNN anomaly detection in smart grid a challenging task. In such
performs better in small scale systems whereas SVM cases, careful consideration should be given to contextual
performs better in large scale systems than other algorithms. information and other events (rather than depending only on
one type of event) for successful characterisation of
However, SVM has challenges as it is sensitive to kernel
type and sparsity of the system. The authors also found that anomalous behaviour [41]. With the huge amount of data
semi-supervised learning algorithms are more robust to the being produced by smart grids, the real-time detection of
sparsity of data than the supervised learning algorithms. The attacks has become an overwhelming task. Efficient feature
fusion algorithms - Adaboost and multiple kernel learning selection techniques should be used to reduce the time delay
(MKL) make the learning models more robust in terms of and computational complexity of real-time attack detection
change in system size and data sparsity. methods [34]. Another challenge for security solutions is
that the smart meters in smart grid have low memory and
Zhang et al. [35] addressed a new security issue in the computational capability, so the security solutions need to
wireless communication channels by proposing a distributed be light weight but robust [27].
intrusion detection system for smart grids (SGDIDS). The
proposed system embeds a number of analysing modules VIII. CONCLUSION
(AM) in each layer of the smart grid. Each AM deploys This paper has reviewed the application of different
SVM and artificial immune system (AIS) for efficient machine learning techniques that could be used to enhance
detection and classification of malicious data and potential the stability, reliability, security, efficiency and
cyber-attacks. responsiveness of smart grid. The findings of the paper
Ahmed et al. [36] proposed a supervised machine show that machine learning algorithms could be efficiently
learning based scheme to detect a new type of cyber-attack used for estimating transformer loss of life, detecting power
named covert cyber deception assault (CCD) where the quality events and faults, making optimal energy dispatch
attacks (data tampering) are initiated by hackers with a good decisions to reduce cost of energy, efficient electricity
knowledge of the system and hence, it is difficult for bad market operations, and securing data and preventing attacks
data detectors in traditional state estimators to detect such on smart grid. This paper has also discussed some of the
attacks. This work used a genetic algorithm (GA) based challenges in implementing machine learning solutions for
feature selection technique for selecting the optimal smart grid. These include limitation of finding past labelled
features, and then used SVM for classifying the CCD data, evolution of new types of attacks, rapid changes in
attacks. The feature selection technique can identify the failure patterns, issues with generating high resolution
most discriminative features and hence, reduces the synthetic data for training, finding efficient feature selection
computational complexity of the attack detection techniques, low memory and computational capability of
mechanism. Moreover, it reduces the delay, and increases
smart meters. With the integration of new energy sources
the accuracy of attack detection.
and technologies, smart grid is becoming increasingly
VII. CHALLENGES complex and vulnerable. Although many machine learning
based smart grid solutions have been proposed in the
The supervised learning algorithms require fully labelled literature, there are still a lot of opportunities for
data for training purposes; however, fully labelled data is
improvement. Deep Learning and Big Data can play a vital
either limited or difficult to obtain [37]. In case of predicting
role in solving problems of smart grid in future.
power outage events, the scarcity and imbalance of event
data poses a significant problem as power grid failures are
REFERENCES
quite rare. For example, distribution feeders have different
kinds of failures and there are few training examples [1] X. Fang, S. Misra, G. Xue, and D. Yang, “Smart grid—The
new and improved power grid: A survey,” IEEE Commun.
available for each kind [38]. The failure pattern may change Surveys Tuts., vol. 14, no. 4, pp. 944–980, 2011.
rapidly and the model for prediction may be obsolete after [2] Machine Learning: The Power and Promise of Computers
some time. If deep learning algorithms are used for That Learn by Example, The Royal Society, 2017.
modelling problems in smart grid, the efficacy of modelling [3] I. Salian, SuperVize Me: What’s the Difference between
will depend on the availability of vast amount of quality Supervised, Unsupervised, Semi-Supervised and
data. It is possible to generate high resolution synthetic data Reinforcement Learning, 2018. Accessed on: 5 August 2019.
for training by modelling and sampling, but this is not [Online]. Available:
without challenges. The challenges mentioned in [39] are: https://blogs.nvidia.com/blog/2018/08/02/supervised-
unsupervised-learning/
(1) high dimensionality, (2) large number of observations,
[4] R. S. Sutton and A. G. Barto, Reinforcement Learning: An
(3) non-Gaussian marginal probability of individual Introduction, 2nd ed. Massachusetts: The MIT Press.
[5] M. van Otterlo and M. Wiering, “Reinforcement learning and [23] M. Peters, W. Ketter, M. Saar-Tsechansky, and J. Collins, “A
Markov decision processes,” in Reinforcement Learning. reinforcement learning approach to autonomous decision-
Adaptation, Learning, and Optimization, vol. 12. M. Wiering making in smart electricity markets,” Machine Learning, vol.
and M.van Otterlo, Eds. Berlin: Springer, 2012. 92, no. 1, 2013.
[6] S. Paul, An introduction to Q-Learning: Reinforcement [24] P. Reddy and M. Veloso, “Strategy learning for autonomous
Learning, 2019. Accessed on: 5 August 2019. [Online]. agents in smart grid markets,” in Proc. of the Twenty-Second
Available: https://blog.floydhub.com/an-introduction-to-q- International Joint Conference on Artificial Intelligence,
learning-reinforcement-learning/ 2011.
[7] V. Spruyt, The Curse of Dimensionality in Classification, [25] Y. Yang, J. Hao, M. Sun, Z. Wang, C. Fan, and G. Strbac,
2014. Accessed on: 5 August 2019. [Online]. Available: “Recurrent deep multiagent Q-learning for autonomous
https://www.visiondummy.com/2014/04/curse- brokers in smart grid,” in Proc. of Joint Conference on
dimensionality-affect-classification/ Artificial Intelligence (IJCAI), 2018.
[8] A. Desarda, Getting Data ready for modelling: Feature [26] S. Ahmad, “Anomaly modeling and detection for smart grid
engineering, Feature Selection, Dimension Reduction (Part communication infrastructures,” PQDT – Global, 2013.
two), 2018. Accessed on: 5 August 2019. [Online]. Available: [27] I. Parvez, A. I. Sarwat, L. Wei, and A. Sundararajan,
https://towardsdatascience.com/getting-data-ready-for- “Securing metering infrastructure of smart grid: A machine
modelling-feature-engineering-feature-selection-dimension- learning and localization based key management approach,”
reduction-39dfa267b95a Energies, vol. 9, no. 9, 2016.
[9] K.T. Muthanna, A. Sarkar, K. Das, and K. Waldner, [28] E. Hossain, I. Khan, F. Un-Noor, S. S. Sikander, and M. S. H.
“Transformer insulation life assessment,” IEEE Transactions Sunny, “Application of big data and machine learning in
on Power Delivery, vol. 21, no. 1, 2006. smart grid, and associated security concerns: A Review,”
[10] A. Majzoobi, M. Mahoor, and A. Khodaei, “Machine learning in IEEE Access, vol. 7, pp. 13960-13988, 2019. doi:
applications in estimating transformer loss of life,” IEEE 10.1109/ACCESS.2019.2894819
Power and Energy Society General Meeting, pp.1-5, 2017. [29] S. Sridhar, A. Hahn, and M. Govindarasu, “Cyber–physical
[11] M. Mahoor and A. Khodaei, “Data fusion and machine system security for the electric power grid,” in Proceedings of
learning integration for transformer loss of life estimation,” the IEEE, vol. 100, no. 1, pp. 210-224, 2012. doi:
IEEE/PES Transmission and Distribution Conference and 10.1109/JPROC.2011.2165269
Exposition (T&D), 2018. [30] S. Bisoi and A. Dash, “The role of utilities in securing a smart
[12] J. McClelland, Connected Assets: How Machine Learning grid,” Electric Light and Power, vol. 89, no. 6, pp. 70-72,
Will Transform the Utilities Industry, 2018. Accessed on: 5 2011.
August 2019. [Online]. Available: [31] M. N. Kurt, O. Ogundijo, C. Li, and Z. Wang, “Online cyber-
https://www.digitalistmag.com/iot/2018/02/26/connected- attack detection in smart grid: A reinforcement learning
assets-how-machine-learning-will-transform-the-utilities- approach,” IEEE Transactions on Smart Grid, 2018. doi:
industry-05921360 10.1109/TSG.2018.2878570
[13] B. Torsæter, Preliminary Results Show that Machine [32] B. Zhang, T. Zhang, and Z. Yu, “DDoS detection and
Learning Can Predict Faults and Disturbances in the Power prevention based on artificial intelligence techniques,” 3rd
System, 2019. Accessed on: 5 August 2019. [online]. IEEE International Conference on Computer and
Available: https://blog.sintef.com/sintefenergy/machine- Communications (ICCC), pp.1276–1280, 2017.
learning-can-predict-faults-and-disturbances-in-the-power-
system/ [33] H. Karimipour, A. Dehghantanha, R. Parizi, K. Choo, and H.
Leung, “A deep and scalable unsupervised machine learning
[14] Y. Tang, H. Cui, and Q. Wang, “Prediction model of the system for cyber-attack detection in large-scale smart grids,”
power system frequency using a cross-entropy ensemble IEEE Access, vol. 7, no. 99, 2019.
algorithm,” Entropy, vol. 19, no. 10, 2017.
[34] M. Ozay. I. Esnaola, F. T. Vural, S. J. Kulkarni, and H. V.
[15] F. Ucar , O. Alcin, B. Dandil, and F. Ata, “Power quality Poor, “Machine learning methods for attack detection in the
event detection using a fast extreme learning machine,” smart grid,” IEEE Transactions on Neural Networks and
Energies, vol. 11, no.1, 2018. Learning Systems, vol. 27, no. 8, pp. 1773–1786, 2016.
[16] P. K. Mishra, A. Yadav, and M. Pazoki, “A novel fault [35] Y. Zhang, L. Wang, W. Sun, R. C. Green II, and M. Alam.
classification scheme for series capacitor compensated “Distributed intrusion detection system in a multi-layer
transmission line based on bagged tree ensemble classifier,” network architecture of smart grids,” IEEE Transactions on
IEEE Access, vol. 6, pp. 27373–27382, 2018. Smart Grid, vol. 2, no. 4, pp. 796–808, 2011.
[17] V. Venkatesh, “Fault classification and location identification [36] S. Ahmed, Y. Lee, S Hyun, and I. Koo, “Feature selection–
on electrical transmission network based on machine learning based detection of covert cyber deception assaults in smart
methods,” Master of Science Thesis, Virginia Commonwealth grid communications networks using machine
University, 2018. learning,” IEEE Access, vol. 6, pp. 27518–27529, 2018.
[18] G. Liu, W. Zhu, C. Saunders, F. Gao, and Y. Yu, “Real-time [37] Y. Zhou, R. Arghandeh, and C. J. Spanos, “Partial knowledge
complex event processing and analytics for smart grid,” data-driven event detection for power distribution networks,”
Procedia Computer Science, vol. 61, 2015. IEEE Transactions on Smart Grid, vol. 9, no. 5, 2018.
[19] S. A. Azad, A. M. T. Oo, and M. F. Islam, “A low complexity [38] C. Rudin et al. “Machine learning for the New York city
residential demand response strategy using binary particle power grid,” IEEE Transactions on Pattern Analysis and
swarm optimization,” in Proceedings of Australasian Machine Intelligence, vol. 34, no. 2, 2012.
Universities Power Engineering Conference, 2012.
[39] M. Sun, I. Konstantelos, and G. Strbac, “A deep learning-
[20] M. Awais, N. Javaid, N. Shaheen, Q. Z. Iqbal, G. Rehman, K. based feature extraction framework for system security
Muhammad, and I. Ahmad, “An efficient genetic algorithm assessment,” IEEE Transactions on Smart Grid, 2018.
based demand side management scheme for smart grid,” in doi:10.1109/TSG.2018.2873001
Proc. of 18th IEEE International Conference on Network-
Based Information Systems (NBiS), 2015. doi: [40] A. O. Otuoze, M. W. Mustafa, and R. M. Larik, “Smart grids
10.1109/NBiS.2015.54 security challenges: Classification by sources of threats,”
Journal of Electrical Systems and Information Technology,
[21] N. Hajj and M. Awad, “A game theory approach to demand vol. 5, no. 3, pp. 468–483, 2018.
side management in smart grids,” in Intelligent Systems'2014.
Advances in Intelligent Systems and Computing, vol 323. D. [41] B. Rossi, S. Chren, B. Buhnova, and T. Pitner , “Anomaly
Filev et al. Eds. Cham: Springer, 2014. detection in smart grid data: An experience report,” IEEE
International Conference on Systems, Man, and Cybernetics
[22] S. Kim and H. Lim, “Reinforcement learning based energy (SMC), pp. 002313–002318, 2016.
management algorithm for smart energy buildings,” Energies,
vol. 11, no. 8, 2018.

View publication stats