This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

Smart Electricity Meter Data Intelligence for

Future Energy Systems: A Survey
Damminda Alahakoon, Member, IEEE and Xinghuo Yu, Fellow, IEEE
bulk generation, transmission, distribution, customers, service
Abstract— Smart meters have been deployed in many providers, operations and markets.
countries across the world since early 2000s. The smart Key technological challenges facing SGs include intermittency
meter as a key element for the smart grid is expected to of RE generation that affects electricity quality; large scale
provide economic, social and environmental benefits for networks of small distributed generation mechanisms, for
multiple stakeholders. There has been much debate over example photovoltaic (PV) panels, batteries, wind and solar,
the real values of smart meters. One of the key factors that plug-in hybrid electric vehicles (PHEVs), that result in high
will determine the success of smart meters is smart meter complexities. Another significant issue is how to use
data analytics which deal with data acquisition, Information and Communication Technologies (ICTs),
transmission, processing, and interpretation that bring advanced electronic and analytic technologies to enhance
benefits to all stakeholders. This paper presents a efficiency and cost-effectiveness of energy use.
comprehensive survey of smart electricity meters and their Managing SGs to deliver Smart Energy require advanced
utilization focusing on key aspects of metering process, the data analytics for acquiring accurate information and
different stakeholder interests and technologies used to automated decision support and handling events in a timely
satisfy stakeholder interests. Furthermore the paper fashion. Significant progresses have been made for using field
highlights the challenges as well as opportunities arising data obtained from intelligent devices installed in substations,
due to the advent of big data and the increasing popularity feeders and various databases and models across the utility
of the cloud environments. enterprises. Some of the examples can be found in [2] and
references therein. Typical information sources include market
Index Terms— Smart meters, Smart Grids, Data analytics, data, lighting data, power system data, geographical data,
Cloud computing, Artificial Intelligence, Machine learning, Big weather data which can be processed and converted into
Data, Automated Meter Infrastructure, Privacy, Internet of
Things.
information and knowledge that can be used for state
estimation, situational awareness, fault detection and
forewarning, stability assessment, wind or solar forecasting.
I. INTRODUCTION Information acquisition is a key for timely data sensing,
processing and knowledge extraction. So far, the mostly talked
Smart Energy has been an important conceptual paradigm for about information about power network operations are from
future energy use. Because of limited non-renewable energy data collected from intelligent electronic devices installed in
resources available on Earth and also high costs of acquiring substations and various parts of the transmission and
renewable energies (REs), how to make energy use more distribution networks.
efficient and effective is critical for future social and economic In recent years, smart meters are being installed in homes
developments [1]. and other premises in many regions of the world [3]. USA and
Smart Grids (SGs) have been a key enabler for Smart Europe have been deploying smart meters for many years
Energy, which refers to power networks that can intelligently while other regions in the world such as Australia and Canada
integrate the behaviors and actions of all stakeholders have also started deployment in the last few years. According
connected to it, for example, generators, customers and those to a recent report [4], due to deployments in 35 emerging
that do both – in order to efficiently deliver sustainable, countries from Central/Eastern Europe, Eurasia, Latin
economic and secure electricity supplies. While there are many America, Middle East/North Africa, South Africa and
definitions for SGs, one commonly used conceptual Southeast Asia, the smart meter numbers have more than
framework is that of the National Institute of Standards and doubled in 2013 compared to 2012. A report by Pike Research
Technology (NIST) which defines seven important domains: estimates the global smart meter installations to triple from
10.3 million in 2011 to 29.9 million units by 2017 [5]. Full
deployment of smart meters has already been completed in
Manuscript received May 5, 2014. Accepted for publication Feb 17, 2015. Italy and Sweden and mass rollout is ongoing in Finland and
Copyright 2011 IEEE. Personal use of this material is permitted. However, Spain [6]. This infrastructure, if used properly, can provide
permission to use this material for any other purposes must be obtained from more than just recording consumption of electricity or a
the IEEE by sending a request to [email protected].
decision support tool to support energy usage by users. For
Damminda Alahakoon is with the La Trobe University, Victoria, Australia.
Xinghuo Yu is with the Platform Technologies Research Institute at RMIT example, advantages include easier processing of billing,
University, Victoria, Australia. automated meter reading and data processing, detection of
energy losses (possible fraud) and early warning of blackouts,
fast detection of disturbances in energy supply, possible real-

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

time pricing schemes, and demand-response for energy saving desirable that multiple systems and components are
and efficient use of energy generated. interoperable under the three aspects of organizational,
The research and development in smart meters and their informational and technical. It has been proposed that the
applications have been progressed rapidly in recent years and Smart Grid Conceptual Model introduced by NIST be
many methods and techniques have been developed. The extended to cater for the above requirements as well as the
technological scope relating to smart meters covers a diverse distributed energy requirements (DER) especially in the
range of ICT technologies, such as electronics and European Union. The Smart Grid Architectural Model
communication, and there have already been several surveys (SGAM) Framework has been introduced to address these
done, e.g. [7] looks into the different applications of smart requirements [13]. With interoperability and the flexibility
grids focusing on the communication needs and also on with technology independent systems and components the
communication requirements of smart meters for integration environment is set for capturing data in near real time from
into power grids. [8] is a comprehensive survey of multiple and diverse sources for generating data intelligence.
communication technologies for smart meters which could be
used to satisfy the identified needs. [9] introduces a novel
smart meter communication technology, [10] examines the
web and data service aspect of smart meter networks, and [11]
proposes a framework for smart meter privacy. In this paper,
we examine these developments from a holistic data analytic
viewpoint. We will also outline potential future applications
and challenges that lie ahead. As a foundation for our holistic
approach the key components of electricity meter intelligence Figure 2: The environment for smart meter data intelligence
is shown as in Figure 1. The three key components capture the A. A Framework for Smart Metering
aspects of data, technology and stakeholders. These aspects
The key components in Figure 1 are further expanded as a
and the role they play in smart meter intelligence is discussed
framework in Figure 3. A high level view of the framework is
in section 2.
presented as Figure 3 where the relationships to the
environment and the key components are highlighted. This
framework is then further discussed in detail including the
impact of these different components in Section 3 and
illustrated as Figure 5.
Figure 1: Key components of electricity meter data intelligence

Section 2 describes the environment in which smart meter

intelligence can occur. The key technology features in smart
meters and their capabilities are also described which are then
used as a stepping stone in to presenting a smart metering
framework. Sections 3, 4 and 5 present each of the three key
components of the framework where the data, technology and
stakeholder aspects are discussed in more detail. Section 6
describes the key challenges in this area and suggests potential
solutions. Section 7 concludes the paper. Figure 3: Smart meter data intelligence framework

II. THE SMART METERING ENVIRONMENT Types of data has been broken down in Figure 3 into
To describe the components of smart meter data intelligence, it consumption or measurement, power generation, power quality
is necessary to understand the environment they exist. Figure 2 and events such as power failures, meter status etc. These data
highlights the main environmental factors being the smart grid types could be used as aggregates or combined with external
which provides the infrastructure and the stakeholders who data such as temperature to derive information for analysis.
generate the need for smart metering. Key elements which Capturing accurate and relevant data in a timely manner is
make up the environment are described below. The essential for smart metering, which includes the collection,
environmental factors presented as Figure 2 provides the transfer and storage (accumulation). Smart meters have
‘bigger picture’ for metering intelligence and positions the resulted in a huge increase in the volume as well as types of
components presented in Figure 1 within the smart metering data generated and collected, leading to many potential
environment. The energy usage cycle and stakeholder opportunities for generating value from such data. As
information are based on National Institute of Standards and mentioned under the metering process, there are several types
Technology (NIST) classification [12]. A further important of data generated measurement or consumption data,
factor is the recent push towards the integration and coupling generation information, power quality and events data.
of multiple systems and components within the smart grid and Consumption data are the more predictable and regular
the understanding of the value of interoperability of such consumption data. With smart meters this could be time-
systems and components [13]. As highlighted in [13], it is interval consumption data as well as aggregated values for

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

billing purposes. Time interval data provides more granular • Ability to capture events such as device status (device
data opening up possibilities of for trend and cycle analysis measured by smart meter), power quality including
and different time of day consumption analysis. Time interval voltage
based consumption also enables to profile consumer behavior • Be interoperable within a smart grid environment (for
and relate consumption to temperature changes. This requires example as specified by NIST and SGAM
the integration of smart meter data with external data such as Framework).
weather, geography and consumer information. Events and
alerts are unscheduled and occur randomly due to unexpected The smart meter is the measurement and information
situations. capture device and in many instances connected to a
It could also be seen that the environment is dynamic and communication device called smart meter gateway to establish
evolving. Technological capabilities change (improve), a secure energy information network. The gateway could
different types of data become available at faster rates and receive and communicate real time information from supplier,
higher volumes and granularity. Stakeholders also become be a point of control for appliances, start and stop energy
more demanding due to life styles, regulations, competition supply etc. It could also have a user interface called the ‘in-
etc. thus creating new applications and changing existing ones. home-display’ (IHD) which displays energy consumption,
It becomes imperative that data analytics technologies have to cost, tariffs with real time updates etc. The smart meter can be
keep pace with this changing environment. Considering the connected to the smart meter gateway which in turn
above, the smart metering framework is proposed as depicted communicates with different appliances (washing machine,
as Figure 3 based on a foundation described in Figure 2. Each refrigerator etc), local generation as well as heating,
of the three components shown in Figure 4 contributes ventilation and air conditioning (HVAC). The measurements
significantly to smart meter data intelligence and as such each and information captured by the smart meter is displayed via
component is explored in more depth in the following sections. the IHD. The smart meter could directly communicate the
consumption information with the utility but the gateway
B. Smart Meters communicates with the next level gateway in the smart grid
infrastructure to pass information for aggregation, demand
The term Smart Meter initially referred to the functionality of response activities as well as to utilities. As such the system
measuring the electricity used and/or generated and the ability consisting of the smart meter, the gateway and the IHD
to remotely control the supply and cut off when necessary. It communicating in real time with appliances, HVAC
was called Automated Meter Reading (AMR) which used one- subsystems and local generation become a key part of a smart
way communication and capable of automated monthly reads, home infrastructure. Smart meters need an environment where
one way outage (or last gasp) and tamper detection and simple they are connected in appropriate structure, and also have the
load profiling. Over time the AMR capability was extended capacity to communicate and transfer the information captured
into short term interval (hourly or less) data capture, on to collection points. The architecture of the network, the
demand reads and linking into and reading other commodities. capacity and speed of data transfer and communication
A major upgrade of functionality occurred after integration of technology will determine whether smart meters generate the
the meters with two-way communication technology which has anticipated value. The smart grid provides the necessary
been called Advanced Metering Integrated (AMI). The environment and infrastructure for the smart meters to function
upgrade included the incorporation of service switching, time and smart meters have been described as the key building
based rates, remote programming, power quality measure and block of the smart grid [15].
a dashboard type user interface for real time usage monitoring One of the key aspects of the smart grid architecture is to
in to the AMR. Although the term smart meter started to be enable real-time decision making, which is possible only if
used only after the smart grid initiatives, it can be seen that the data can be harnessed without latency as it is generated and
features and functionality of the meters evolved from the applied towards a specific objective. This real-time or “active”
manually read meters of the past to the AMIs’ with dashboard data can be harnessed to make just-in-time decisions, such as
interfaces and two way communication capability. Therefore automated outage detection through the last-gasp meter data
in the current metering environment a meter is expected to for proactive customer service and proactive self-healing of
have the following capabilities to be categorized as a smart the grid; detection of current load and critical peak conditions
meter [14]: to initiate automated load curtailment programs to curtail
• Real time or near real time capture of electricity usage power at participating customer premises, or to perform air
and possibly distributed generation. conditioning load curtailment at participating retail
• Providing the possibility of remote and local reading of households. Smart meters can add continuous communications
the meter if needed so that monitoring can be done in real time, and can
• Remote controllability of the meter enabling control be used as a gateway to demand response-aware devices and
and even cut off of supply "smart sockets" in the home. Within the grid the smart meters
• Possibility of linking to other commodity supply (gas, may also simply replace the devices required at sensing points
water) whose large number is not possible for cost and logistic
reasons, by reporting e.g. voltage and current measurements

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

directly to a data acquisition system such as a smart metering In terms of bulk generation, transmission and distribution,
platform [16]. which involve meter company and grid company, smart meters
Smart electricity meter data analytics can assist in the will complement well with existing infrastructure to provide a
distribution network operators in assessment and network more accurate and timely view of the energy consumption by
management. For example, shorter time interval reading (e.g. 5 regions. Events such as suspicious usage areas and potential
minutes) can help derive information of LV network topology faults will be noticed more easily and on-time actions taken
reconstruction, which in turn help identify loading and voltage subsequently when necessary. It may also enable more
profiles, connectivity issues, and distributed energy resources accurate prediction of electricity flows enabling better network
(DER) impact measurement, which will help HV network and maintenance planning.
assessment operational monitoring. It can also help forecast Smart meters can enable consumers to directly review their
future needs and trends of demand management, network tariff electricity usage, even down to the level of separate appliances
optimisation and asset maintenance requirements. [19], and thus adjust their behaviors to reduce energy cost.
Customized rate plans are another key benefit to consumers.
C. The Smart Metering Process
Although not a common practice at present, smart meters
Although there are smart meters of varying technology and enable demand response for consumers where limiting or even
design, there is a common overall process for data collection, cutting off the supply depending on market situations is
communication, analysis leading to decision support. The possible. When all consumers being aware of both
smart metering process in fact could be thought of as part of consumption and production of energy, adapt their energy
the activation and functioning of the smart grid. usage during a period of high demand, high pricing or lower
supply, more reliable and stable supply, better energy
awareness, savings and efficiency will be achieved. In
combination, these activities have been called Demand Side
Management (DSM) which is essential to really benefit
consumers [20]. Consumer awareness of the benefits from
smart meters as well as their functionality will be a key factor
in the successful adoption of this technology [21]. As
discussed in [22] another key factor could be the use of
Figure 4: The Smart Metering Process [16]
disaggregation techniques to extract underlying end use and
appliance level information from an aggregated energy signal.
There are some variations according to deployment in different
For retailers the availability of vast volumes of data which
countries and regions, but the general smart metering process
could be used to profile and understand customers, their needs
is shown in Figure 4 [17]. The smart meter gathers data locally
and behaviors enable better service provision and building
and transfers via a Local Area Network (LAN) to a data
stronger loyalty. Better consumer awareness is expected to
collection point. There are two key categories of data collected
result in reduced energy consumption thus reducing the need
as mentioned before. Usage or consumption data refers to the
for additional power plants which generate greenhouse gases.
actual electricity usage measured in kilowatt Hours (kWh) and
Restricting and reducing electricity usage during peak periods
this is read and transmitted in regular intervals. Depending on
can result in cutting down on the need of using peeker plants
the particular deployment and region the frequency of data
[23] which generally make higher carbon emissions. Load
collection can vary from 1 hour to every 15 minutes. The
control feature in smart meters enables switching individual
issues faced with data collection and the implications and
appliances on and off as required. Retailers could offer this
techniques for making use of this data are further discussed in
feature to customers when the cost of power is very high, while
detail in the next sections of this paper. In terms of the
distributors could use it when a section of the network is close
processing of data, some data processing could be carried out
to capacity.
at the local collection points, but in most cases the data is
Table 1
transferred to the utilities’ central collection center via a Wide
Stakeholder Type of Benefit
Area Network (WAN). The data collected at the utility is used
for a number of business purposes such as billing, network and Distributor/meter Accurate, timely view of consumption
service monitoring, profiling, prediction and planning. company Capture suspicious usage, faults
Better network maintenance
D. The Stakeholders Consumer Directly view electricity usage
A simple classification is consumers, electricity companies Monitor appliance use and consumption
(utilities) and environment [18]. In some literature a further leading to awareness and better
level of granularity has been added to the electricity company electricity usage planning
class by expanding to metering company (distributor), utility Customized rate plans
company and supplier (retailor) [14]. However, a more Retailer Understand and profile customers for
comprehensive classification would be those by NIST where targeted service for better loyalty
the functionality of the whole energy usage cycle is defined to Cut down peak usage
include bulk generation, transmission, distribution, customers, Load control feature offer
service providers, operations and markets.

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

For example load control could be used switch an air alternative generation and profile households and suburbs and
conditioner on and off, which could reduce load on the regions. The integration of consumption, event as well as other
network on a very hot day. The stakeholder involvement and data categories could be useful to understand how the grid
benefits are summarized in Table 1 under each main infrastructure stands up to usage. This can also provide
stakeholder group. insights into capacity planning and budgeting. The fusion of
For operations and markets, the information acquired from external data such as weather, geography with consumption
smart meters will be vital for planning operations, responding can provide useful information for predicting power usage
to market demands, and anticipate changes and disruptions to [26]. Data supporting such activities have been called derived
reduce risks to secure energy supply. data in Figure 3.
Data analytics is the process of examining large amounts of
III. DATA FOR METERING INTELLIGENCE data of a variety of types to uncover hidden patterns, unknown
A smart metering framework has been proposed in the past but correlations and other useful information. Data analytics is
it only looked at consumer characterization and not holistic used to obtain value from such data captured and information
view of the complete smart metering process and environment derived such that stakeholder applications could be satisfied.
[24]. The proposed framework is expanded and enhanced in The stakeholder applications in smart metering are grouped
this section and presented as Figure 5 consisting of two key under several key categories in Figure 3. With AMI and smart
components. The top depicts the current smart metering grid infrastructure for capturing and transferring data,
scenario; the data aspects, stakeholder needs based opportunities for smart meter analytics have moved to a new
applications and technology tools and algorithms which dimension. But to make use of such opportunities analytics
attempt to support the application needs from the available or technologies have to successfully face and resolve new data
derived data. Current technologies and algorithms are shown related issues as shown in Figure 5. The volume of data
as core analytics building blocks which are realized using collected has increased massively due to collection of data in
different tools. The bottom part depicts new requirements that shorter intervals and the ability to store large volumes. The
have arisen due to reasons such as technological data capture at frequent intervals by smart meters and the
advancements, change in human behavior and expectations, infrastructure to transfer such data at high speeds results in
competition, better informed consumers etc. The new streams of data. Due to the new technological capabilities of
requirements and limitations of existing tools lead to new the smart meters as well as the increased demand from
research and development needs. stakeholders including competition among utilities, different
AMI capability provides the base for collecting, transferring types of data are being collected to provide more value for
and accumulating data and information. As shown in Figure 3 stakeholders, which has been called variability issue in
before types of data could be broken down into (power) analytics. It is very difficult to measure the effectiveness of
consumption and generation data, power quality measurements energy efficiency programs. Many factors such as weather,
and event data. Out of these the most widely used consumer profiles, seasons, geographic regions, infrastructure,
measurement data in smart metering activities is the detailed type of homes and fittings all contribute to the complexity. The
consumption data consisting of time based (15 minutes to one combination of volume, velocity and variability as well as
hour), reading of electricity consumption. The other types of different granularities, issues in integrating various types of
consumption data are [25]: data results in much more complexity needing to be addressed
• billing interval data: readings at the beginning and end by analytics techniques.
of billing intervals to enable variable pricing The new technological capabilities have also resulted in
increasing expectation from the stakeholders and the different
• aggregate statistical data: monthly consumption,
applications are highlighted in the right side in Figure 5. Some
comparison with neighbours, usage history etc
of the key issues that have arisen due to such different
• broadcast data: price change information, critical peak
applications are listed below [25]:
time rebates, reliability status etc communicated to
• Latency and bandwidth – infrequent and low volume
users
information (eg: broadcasts) will require low
Event data refers to information that is generated at the
bandwidth and probably low latency as well.
meters’ end points and includes real time device status, power
Consumption data on the other hand will require high
quality information and meter status information. These can be
bandwidth, and could tolerate higher latency
made up of attributes such as: source and proxy, severity level
and category. Source originates the event and the proxy • Batch or real time processing – many event data and
captures and communicates. Main event categories include information for in-home displays will require real
meter status, power quality events (voltage sag), meter time processing while batch processing will suitable
tampering, meter hardware events (low battery) are the. Power for consumption data
quality data is generally used in fault analysis to help improve • Life spans – how long data need to be kept will depend
reliability. Both pre-fault and post-fault analyses are on the usage. For eg: billing and statistical data might
considered as effective techniques of using power quality to have regulatory requirements to be kept for certain
improve the reliability of a distribution network. Generation periods while detailed consumption data need only be
data provides information such as solar usage which enables to kept for short periods
identify patterns of electricity usage, effectiveness of

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

• Communication media – in-home display for real time The technologies and algorithms used for smart metering is
information such as prices and statistical data and presented under the sections core analytics building blocks and
aggregates could be displayed on a portal tools below.
• Privacy and security – Prices are public information A. The Core Analytics Building Blocks
but consumption is confidential and sensitive
Many statistical, machine learning, data mining and
mathematical techniques have been used, separately as well as
in different combinations for smart metering. Since different
techniques could be used to achieve the same purpose in terms
of analytics outcomes, we have presented these as analytics
building blocks in Figure 5. The building blocks of
aggregations, correlations, trending, exception analysis, and
forecasting are the key foundations of analytics in smart
metering.
The meters connected to individual transformers can be
aggregated together to identify transformer loading patterns.
Combining homes or businesses into demand response pools
to deliver sizable demand reductions (or ‘negawatts’) is
another aggregation supported by smart meters.
Heat waves drive spikes in power consumption and
statistical correlation using time-interval consumption data
makes it possible to build algorithms that predict the size of
demand spikes using forecast temperature. Cloud cover,
humidity and time of day can be added to further refine peak
predictions. Correlations are generated by aligning data
temporally, spatially, or across other attributes. This type of
Figure 5: Smart metering framework and new impacts data is used to build analytical models to measure the energy
efficiency of individual commercial properties.
IV. TECHNOLOGY AND ALGORITHMS A web site that shows a simple consumption data trend line
In smart metering activities, the use of technologies and tools can help customers relate power consumption to household
has been triggered from two main directions. The metering activity. The ability to overlay multiple trend lines together is
intelligence activities initiated based on the AMI capability also valuable for purposes such as comparing consumption
could be called data driven exploratory activities. Such across similar seasons and times of day.
activities are triggered mainly due to the availability of new A missing meter read is an exception event. Analyzing
AMI capability; for example, new and more data being exceptions over time may identify problems in
captured, better grid facilities enabling faster and more reliable communications and measurement infrastructure, as well as in
transfer and storage of data and appropriate skills and other the distribution grid. Component degradation or operational
resources being available in an organization. In such cases the breakdowns can be captured by analyzing trends in exception
question could be asked: how can this data and infrastructure events.
be used to gain some benefit for various stakeholders? Forecasts are predictions of future events or values using
Although the stakeholder needs are considered, they are not historical data. A forecast of power consumption for a new
the drivers of these activities. In traditional data mining tasks residential subdivision can be created using historical data
this is called exploratory analytics and tools such as clustering, from similar homes.
visualization and unsupervised machine learning techniques
are utilized [27].
B. Tools for Smart Metering
The second types are the application driven directed
activities. In contrast to the exploratory approach, this There are many mathematical and statistical techniques,
approach is triggered directly based on stakeholder needs. machine learning, data mining tools that can be used for smart
Current knowledge of stakeholder needs, business needs, metering as shown in Figure 5. Widely used techniques
government policies, social and environmental trends (privacy, include Self Organizing Maps (SOMs), Support Vector
green house effects etc) will be the drivers for this approach. Machines (SVMs), Principle Component Analysis (PCA), and
Compared to the exploratory approach, the activities initiated Fuzzy Logic (FL) [29].
in this approach will have a known objective. In data mining The SOM [30] is an unsupervised learning algorithm which
and machine learning terminology, such activities are called is widely used to project high dimensional data vectors on to a
directed, and use supervised learning tools such as summarised two or three dimensional space. Its key desirable
classification, decision trees, and artificial neural networks features in exploratory data analysis are its ability to
[28]. summarise the input space and visualise results for
interpretation. This enables the visual inspection of the
possible patterns and the structure of data which can then be

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

used for clustering and other techniques to elicit useful A. Consumer Profiling, Segmentation and Cluster
information. In smart metering SOM has been mostly used for Analysis
exception capture and profiling [68, 83]. Cluster analysis of smart meter data has been reported from a
SVMs [31] are based on supervised learning with associated number of regions around the world. Cluster analysis aims at
learning algorithms that analyze data and recognize patterns, discovering structures in large data sets. The k means algorithm
used for classification and regression analysis. An SVM model and a combination of k-means and artificial neural networks such
is a representation of the examples as points in space, mapped as SOMs are popular approaches for the clustering and have been
so that the examples of the separate categories are divided by a used in load profiling [48]. A study of German electricity
clear gap that is as wide as possible. New examples are then consumers by Flath et al. [55] highlights the advantages of
mapped into that same space and predicted to belong to a cluster analysis in identifying consumer groups for targeted
category based on which side of the gap they fall on. SVM service innovation by utilities and retailers. They have also
based analytics have been reported for appliance type suggested the value of incorporating cluster analysis as part of
recognition from AMI data [32,33] and electricity theft the utilities business intelligence systems such that process
detection [34,35]. innovation and customer portfolio management could be
PCA is a mathematical technique which generates guided by the results. Cluster analysis has been conducted on
an orthogonal linear transformation that converts the data to a data separated by season and by weekdays and weekends due
new coordinate system such that the greatest variance by any to differing usage patterns. A study reported in [56] identifies
projection of the data becomes the first coordinate (the first customer signatures based on usage readings from 50 meters.
principal component), the second greatest variance the second Daily energy consumption plots yield information such as
coordinate etc. [36]. PCA has been used for power usage data minimum, average, and maximum daily energy consumption,
aggregation [37], a data reduction technique in consumption as well as changes in daily energy use from which one can
analysis [38] and the detection of anomalies due to malicious derive information such as household occupancy and occupant
modification of network data [39]. activities. This highlights privacy as a key concern with
FL is a form of reasoning that is approximate rather than profiling usage patterns. The key privacy concerns are
fixed and exact which may have a truth value that ranges in occupancy detection and possibility of inferring appliance
degree between 0 and 1. In [29] FL has been used to create an usage. It has been suggested that statistical representations of
automated decision-making platform in a smart grid, and in the data be used rather than actual consumer information. Also
[36,40] it has been used to improve the reliability of clustering aggregation, noise addition and consumer signature flattening
of smart meters, which is essential due to scalability problems. has been proposed as ways for privacy protection. [11]. A
Integration of technologies and intelligence into the smart grid SOM [30] based clustering of 12,000 Finnish electricity
also makes it more open to cyberattacks. In [41], a FL based consumers was reported in [57]. The study carried out using
technique for efficiently detecting cyberattacks was described. Viscovery SOMine tool suggested that understanding customer
The above are only some of the examples of how the existing usage patterns can help design better demand response tariffs
techniques can be used. Bayesian and Hidden Markov Model schemes. The SOM has been used in several other studies as a
techniques are being used in a variety of smart metering tool for clustering electricity consumption data [58,59,60]. A
applications such as load disaggregation [42], appliance study by Abreu et al uses pattern recognition techniques to
identification [43] and supply demand analysis [44]. Future capture habitual behaviour of consumers using smart meter
applications will result in a broader range of needs which will readings [61]. The work based on a new data mining algorithm
see more and more methods applied and tailored for smart for finer profiling [62], proposed that such fine grained
metering to bring out greater benefits. profiling can be used to provide tailor made forecasts for
households. Zhang et.al [63] used clustering techniques to
V. STAKEHOLDER APPLICATIONS identify load profiles for large electricity uses in a Chinese
The tools described above have been used separately and in province. This work differs from the previously stated since
combination to achieve metering intelligence. Majority of the they compare three well known clustering techniques, namely,
metering intelligence related work reported uses time varying k-means, Fuzzy cMeans and SOM.
power consumption data to generate consumption (or load) B. Load Forecasting
patterns showing the consumers usage behavior. Clustering
One of the most valuable analytics applications for the smart
usage patterns makes it possible to identify typical behaviors
grid and the availability of time interval data has made it
called typical load profiles (TLPs) [45,46]. TLPs could then
possible to forecast in the short term and with high accuracy.
be used for load forecasting [47,48], load estimation [49], load
Accurate forecasts are important for deciding short-term
control [50], abnormal electricity consumption detection [51],
operations as well as mid-term scheduling, but also decision
designing electricity tariff offers [52], developing market
makers need to have an understanding about the customers
strategies [53] or demand side response policy [54]. Some of
they have to supply for long-term planning. Many applications
the most widely used metering intelligence activities are
of load forecasting have been described in literature where
discussed below.
several statistical and machine learning technologies have been
utilized. For short and medium term forecasting, time series
analysis and neural networks have been used [64,65,66]. A

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

problem with short term forecasting models has been the loss customers’ load profiles could also be used to identify, detect,
of understanding about the bigger picture, which requires and and predict behavior irregularities or abnormalities that
understanding about the different categories of consumers. In ultimately may be due to faulty metering or to human
[67], a PCA based technique was used to identify the type of intervention and fraud [51,71]. Potential theft or technical
demand faced by such consumer categories. In [68], a hybrid losses can also be identified by comparing smart meter data
system of SOMs and SVM was used to forecast mid-term with measurements from sensors attached to transformers or
electricity load. The SOM was used to separate electricity feeders [14].
consumption data into two groups which are then fed into a
SVM in a supervised manner for load prediction. In [48], E. Metering Intelligence to Support Real-Time Operations
Espinoza et al. report on short term forecasting with hourly
load data from a Belgian grid substation highlighting that The rolling out of AMI makes it possible to acquire near real
forecasting and customer profiling are inter-related and time information of energy use, connect renewable energy to
proposed a unified framework which incorporates both. The grids, manage power outages and faster restoration, fault
initial modelling is based on seasonal time series analysis, detection and early warning.
using the Periodic Auto-regression (PAR) model [69], which There are several levels of real-time responses. One level is
was used in modelling of electricity prices [70]. The stationary at the actual control of machines and equipment which
properties obtained from these models are run through a k- requires milliseconds response time. At this level, unless the
means clustering process to capture different customer infrastructure of smart meters and communication networks
profiles. can be significantly upgraded to realise reliability and
millisecond responsiveness required for real time control (e.g.
C. Pricing Intelligence controlling transient responses of rotating machines in the
Smart meters will enable to set up dynamic tariff structures to gird), smart meters cannot necessarily provide adequate
improve efficiency in electricity markets by better representing benefits. For those near real-time critical applications (five
the costs of producing and delivering electricity at different minutes, quarterly hours, half hours, hourly intervals), such as
times. Consumers could benefit from these if they choose more fault identification and localization on MV and LV networks
flexible tariff arrangements that better represent their to ensure faster intervention and reduced outage duration,
electricity cost. To determine the price for electricity as well as monitoring power quality, acting remotely managing peak-
the tariffs, retailers consider payments to distributors for saving, forecasting network conditions, facilitation of
services, wholesale electricity cost, retail services cost, as well integration of renewable energy and PHEVs into the grid,
as costs of any regulatory requirements. Smart meter data smart meters can help [6]. Examples include the measurement
enables deeper analysis for understanding the dynamics of of voltage distortion (harmonic voltages and voltage
supply and demand resulting in better forecasting the needs unbalance) [72] using smart meter data to derive a dynamic
enabling pricing intelligence. Smart meter data can be used to model for improving volt-var control [73], as well as
analyze and plan different rate structures, without controlling congestion and stability in a power market [74].
discriminating by demographics of low income etc. This has Metering data can be also used to derive the knowledge of the
been called ‘time of use’ or dynamic pricing. Real time stream power flows at and near the low voltage end of the distribution
analysis of consumption is required to make such a scheme networks so that the loading and losses of the network can be
practical. The benefits of dynamic pricing have been identified known more accurately. This can help to prevent overloading
as demand reduction, cost reduction and economic efficiency components (transformers and lines) and to avoid power
gain [52]. Prepay is another possible pricing intelligence quality deviations [75]. Another example is the state
technique with the advantages of being familiar to consumers, estimation using smart meter data which may provide an
and also helping utilities limit credit risk. Utilities can take a alternative economical way of estimating states of MV and LV
more opt-in approach where customers elect to select in a new networks [76,77]. Smart meter data can also be used to
rate plan, such as dynamic pricing. Customer demography enhance overcurrent protection [78] and load disaggregation
segmentation related to consumption based profiling has to be [79]. The NOBEL project [80] has developed a smart metering
carried out in these approaches. [57] describes work carried platform [81] that specializes in near real-time acquisition of
out using SOM tool Viscovery for clustering and visualization metering information from the grid as it is reported directly by
of consumer information from a utility in Finland. In addition the smart meters. Data stream analysis [82] for capturing time
to several consumption based attributes, the type of residence based usage patterns has also been reported in [83].
was used for the segmentation.

D. Capturing Irregularities VI. KEY CHALLENGES AND FUTURE OF SMART

Many load profile studies have used data-mining techniques, METERING
pattern recognition, and statistical techniques to obtain
knowledge from customer load records. Knowledge of A. Issues in Smart Meter Data Analytics
consumption behaviors is very important as it is very useful for To achieve metering intelligence as described in the previous
formulating tariffs and developing marketing strategies, as well sections a number of technical issues need to be successfully
as allowing customized billing. Knowledge gathered from addressed. The ability to work with very large volumes of data

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

will be a key requirement. It is also essential that technologies meters the other aspects of velocity and variance are equally
be able to work with a variety of data such as weather important. Velocity refers to the need for collecting,
information and consumer information, geographic data thus processing and using the data speedily and in time. Although
requiring techniques for efficient data fusion and integration. analytical algorithms which can process huge quantities of data
Such a big data integration and analytics engine is being are available many of these are not able to complete such
developed by a C3 Energy, where tasks such as voltage activities in a sufficiently short time period to be of practical
optimization, asset management, outage management and fault value. For example overnight is not good enough for real-time
detection to customer focused services such as demand tasks such as reliability monitoring of equipment, preventing
response, load forecasting and customer segmenting and outage or security monitoring. Although several research
targeting are to be integrated. Real time monitoring and techniques have been reported on analyzing streaming data,
diagnostics focused analytics would be an important much work still needs to be done in making these
requirement in such systems [84,85]. commercially viable. Variety signifies the increasing array of
To achieve real benefits of analytics outcomes it is essential data types, which are collected not only from traditional
to gain consumer’s acceptance and support for smart meters. A sources like industrial control systems but also from security
key requirement for such acceptance is transparency of the cameras, weather forecasting systems, maps, drawings and
process which is currently being addressed by government pictures, and the web. The variety of data is likely to become
regulators as well as utilities [86]. The availability of easy to increasingly important to utilities as they begin to analyze
understand and visual displays of information is also an social media and call center dialogues and to integrate such
important need. Making smart meter data and analytics information in to smart meter and grid generated data as part
outcomes available on the web and on mobile devises will of their decision-making and planning processes [3].
make such information more readily available and also updates
communicated to consumers in near real time [87]. Smart
C. Smart Meters and Cloud Computing
meter analytics will thus continue to evolve, making demands
on the current knowledge and technology available. Although smart grid enables distributed and renewable energy
The available big smart meter data will also present privacy generation, controlling energy generation according to demand
and security concerns that are likely to become more prevalent is still a problem. A new idea is aligning energy supply and
as government-backed initiatives expand deployment of the demand using infrastructure such as a broadband network
meters to millions of homes across the country [88,89]. These access, performance, scalability, and flexibility, which can be
will have to be addressed within the regulatory regime of the provided by a cloud platform [90]. Cloud computing is a
particular country deploying smart meters. For example, paradigm in which services including computation, storage and
obtaining permission from the customers to manage, use and network are packaged and provided as metered utility services
create value from smart meter data is seen as a standard public sold on demand both in terms of duration of use and utilization
service responsibility that already exists in all European [91]. Benefits include on-demand self-service, resource
countries [6]. pooling and use of a cloud service on pay-per-use or charge-
The future of smart metering will also depend on several per-use basis. But this raises security and privacy issues with
key technology revolutions currently in progress, which are protecting the smart meter data from unauthorized usage [92].
discussed below. Using smart meters to develop a dynamic pay-per-use pricing
model for regulation and improvement of overall utilization of
B. Smart Meters and Big Data the cloud infrastructure is described in [91].
Big data is a term that is currently widely being used with data Denmark now has its first cloud-based smart metering
analytics. Big data has many interpretations but there are three solution, targeted at small utilities and communities that
key features that are highlighted; volume, velocity and previously could not afford such technologies. IBM and
variance. Data captured by smart meters clearly relate to all Cable&Wireless Worldwide are working to develop a new
these features thus satisfying the definition for big data. For intelligent communications solution called UK Smart Energy
example, moving from one meter reading per month to smart Cloud to support the UK's smart meter implementation
meter readings every 30 minutes results in a massive volume program with expected roll out of more than 50 million meters.
of data to manage. The data is collected in frequent time The solution is expected to provide for more accurate billing,
periods and if technology is available for near real time greater smart grid functionality and other benefits.
analysis, many advantages could be achieved. The analytics Use of cloud platforms and technologies for smart metering
technologies will have to deal with not only consumption data, and grid has resulted in the need of extended capabilities from
consumer information, weather and many grid behavior based analytics tools. A number of vendors have come up with tools
readings. To manage and use this information to gain insight, such as IBM Coremetrics and Google BigQuery and the cloud
utility companies must be capable of managing high-volume based software platform for data driven analytics described in
data and using advanced analytics to transform data into [93] as part of the Los Angeles Smart Grid Project. The
actionable insights [3]. Utilities that build up this capability described system has attempted to incorporate several of the
can gain insight into their operations and assets and can new advanced analytics requirements highlighted earlier such
become proactive in taking action based on analytics. as real-time data analytics, scalable machine learning
Although the increase in volume, especially in consumption techniques and data integration demonstrating how the new
data capture is the most prominent big data aspect with smart

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

advanced analytics techniques provide the base for future [2] M. Kezunovic, “Data analytics: creating information and knowledge,”
IEEE Power & Energy Magazine, vol. 10, no. 5, pp. 14-23, 2012.
smart metering and smart grid systems. [3] D. Alahakoon and X. Yu, “Advanced analytics for harnessing the power
of smart meter big data,” Proceedings of 2013 IEEE International
D. Smart Meters and Internet of Things Workshop on Intelligent Energy Systems (IWIES), pp. 40-45, November
The Smart Grid and the environment it creates have been 2013.
[4] Available:http://www.prnewswire.com/news-releases/emerging-markets-
called the Internet of Energy [94]. If there is an adaptation on to-more-than-double-smart-meter-growth-in-2013-56bn-market-by-
the behavior of the prosumer devices based on the information 2022-182794261.html [Accessed 25th April 2014]
that they receive such as electricity price, the energy [5] Available:http://www.pikeresearch.com/newsroom/smart-gas-meter-
consuming/producing devices will not be black-boxes but be penetration-to-reach-11-by-2016. [Accessed 25th April 2014]
[6] P. Mallet, P.-O. Grahstrom, P. Hallberg, G. Lorez and P. Mandatova,
able to adapt accordingly. Such an environment is called the “Power to the people,” IEEE Power & Energy Magazine, vol. 12, no. 2,
Internet of Things (IoTs) from a smart grid perspective pp. 51-64, 2014.
[94,95]. IoTs are expected to grow to 50 billion connected [7] V. C. Gungor, D. Sahin, T. Kocak, S. Ergut, C. Buccella,C. Cecati and
G. P. Hancke A Survey on Smart Grid Potential Applications and
devices by 2020 [95]. The connected smart grid provides a Communication Requirements, IEEE Transactions on Industrial
communication network that will connect all the different Informatics, Vol 9, No 1, Feb 2013
energy-related equipment of the future. From the transmission [8] V. C. Gungor, D. Sahin, T. Kocak, S. Ergut, C. Buccella,C. Cecati and
and distribution power infrastructure, electrical, water, gas, G. P. Hancke, Smart Grid Technologies: Communication Technologies
and Standards, IEEE Transactions on Industrial Informatics, Vol 7, No
and heat meters, to home and building automation. The first 4, Nov 2011
key step towards a smart grid that makes the IoT a reality will [9] J. Zhang, D. W. Gao and L. Lin, “Smart meters in smart grid: An
be the mass deployment of smart meters. overview”, Proceedings of 2013 IEEE Green Technologies Conference,
pp. 57-64, April 2013.
The connectivity and accessibility that the IoT brings further [10] S. Prasad and S. B. Avinash, “Smart meter data analytics using
improvement of customer experience and efficiencies thus OPENTSDB and HADOOP,” Proceedings of 2013 IEEE Conference on
greater interaction and control. Additionally the IoT enables Innovative Smart Grid Technologies Asia (ISGT Asia), pp. 1-6, 2013
manufacturers and utility providers to cut costs through [11] L. Sankar, S.R. Rajagopalan, S. Mohajer and H. V. Poor, “Smart meter
privacy: A theoretical framework,” IEEE Transactions on Smart Grid,
diagnostics and neighbourhood based meter reading. As such vol. 4, no. 2, pp. 837-846, May 2013.
IoT will result in building a more connected, cost-effective and [12] National Institute of Standards and Technology, U.S. Department of
smarter smart grid. Commerce, 2014, NIST Framework and Roadmap for Smart Grid
Interoperability Standards, Release 3.0 (Government Report) Available:
http://www.nist.gov/smartgrid/upload/NIST-SP-1108r3.pdf [Accessed
VII. CONCLUSION 28th Dec 2014].
The paper has presented a comprehensive survey of smart [13] CEN-CENELEC-ETSI Smart Grid Coordination Group, European
Commission, 2012, Smart Grid Reference Architecture, (European
metering and electricity smart meter data analytics. Although Commission white paper) Available:
there has been much opposition to smart meters due to privacy http://ec.europa.eu/energy/gas_electricity/smartgrids/doc/xpert_group1_
and health concerns, it is obvious that smart meters are here to reference_architecture.pdf, [Accessed 28th Dec 2014].
stay and that the smart grid and smart metering will be a ‘way [14] R. Gerwen, S. Jaarsma and R. Wilhite, “Smart Metering”, Leonardo
Energy (White Paper), June 2006 Available: http://www.leonardo-
of life’ in the future. A number of different dimensions to energy.org/sites/leonardo-energy/files/root/pdf/2006/SmartMetering.pdf
smart meters have been highlighted including the smart meter [Accessed 28th April 2014]
technology and the process, the various stakeholders, existing [15] G. McKinstry, S. Galloway, and I. Kockar, “An initial assessment of the
potential impact of smart metering on a decentralised energy network,”
analytics technologies and tools, the current technological Proceedings of 2010 45th International Conference on Universities
revolutions such as big data, cloud computing and the internet Power Engineering Conference (UPEC), pp. 1-4, 2010.
of things. The paper has also presented the current smart [16] D. Ilic, S. Karnouskos, and P. G. Da Silva “Sensing in power
distribution networks via large numbers of smart meters”, Proceedings
metering space as the smart metering landscape and then a of 2012 3rd IEEE PES International Conference and Exhibition on
framework has been established to relate smart meter data to Innovative Smart Grid Technologies (ISGT), pp 1-6, 2012.
stakeholders and applications created by their needs and the [17] Edison Electric Institute, 2011, Smart Meters and Smart Meter Systems:
analytics tools and techniques required to achieve the A Metering Industry Perspective (White paper). Available:
http://www.eei.org/issuesandpolicy/grid-
stakeholder needs. The framework would help identify the enhancements/Documents/smartmeters.pdf [Accessed 25th April 2014]
current limitations in smart metering. Another contribution of [18] US Department of Energy, 2009, “Perspectives for Utilities and Others
this paper is the identification of smart meter analytics Implementing Smart Grids” (Government Report) Available:
http://www.epa.gov/cleanenergy/documents/suca/stakeholder_roundtabl
building blocks which enable to link the wide range of tools e_sept09.pdf, [Accessed 26th April 2014]
used for smart metering and identify the main analytics [19] X. Hao, Y. Wang, C. Wu, A. Y. Wang, L. Song, C. Hu and L. Yu,
activities. The smart grid and smart meters will be part of a “Smart meter deployment optimization for efficient electrical appliance
state monitoring,” Proceedings of 2012 IEEE 3rd International
much wider internet of things in the future integrating multiple Conference on Grid Communications (SmartGridComm), pp. 25-30,
aspects of human needs and services to satisfy such needs, and 2012.
the analytics requirements discussed such as big data, real time [20] P. Palensky and D. Dietrich, Demand Side Management: Demand
Response, Intelligent Energy Systems, and Smart Loads, IEEE
analytics, stream analytics will need to be built in to the Transactions on Industrial Informatics, Vol 7, No 3, Aug 2011
processes and workflows for diagnostics in real time. [21] UK Department of energy and Climate Change, Quantitative Research
into Public Awareness, Attitudes, and Experience of Smart Meters,
REFERENCES (Government Report), 2013
[1] X. Yu, C. Cecati, T. Dillon, M. G. Simoes, “New frontier of Smart [22] K. C. Armel, A. Gupta, G. Shrimali, and A. Albert, Is Disaggregation
Grids”, IEEE Industrial Electronics Magazine, vol 5, no. 3, pp 49-63, the Holy Grail of Energy Efficiency ? The Case of Electricity, Energy
2011 Policy, Elsevier, Vol 52, pp 213-234, 2013

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

[23] GTM Research, 2012, “Understanding the Potential of Smart Grid Data [45] S.-J. Huang and K.-R. Shih, “Short term load forecasting via
Analytics” (White Paper). Available: ARMAmodel identification including non-gaussian process
http://www.emeter.com/documents/anylst-papers/Understanding-the- considerations,” IEEE Transactions on Power Systems., vol. 18, no. 2,
Potential-of-Smart-Grid-Data-Analytics.pdf [Accessed 26th April 2014] pp. 673–679, 2003
[24] G. Chicco, R. Napoli, P. Postolache, M. Scutariu, and C. Toader, [46] T Zhang, G Zhang, J Lu, X. Feng, and W. Yang, “A new index and
“Customer characterization options for improving the tariff offer,” IEEE classification approach for load pattern analysis of large electricity
Transactions on Power Systems vol. 18, no. 1, pp. 381–387, Feb. 2003. customers”, IEEE Transactions on Power Systems, vol. 27, no. 1, pp
[25] C. Polyzois, “Slicing and Dicing Smart Grid Data”, IEEE Smart Grid 153-160, February 2012.
News letter. Available: http://smartgrid.ieee.org/february-2011/62- [47] H. L.Willis, A. E. Schauer, J. E.D. Northcote-Green, and T. D.Vismor,
slicing-and-dicing-smart-grid-data, [Accessed 26th April 2014] “Forecasting distribution system loads using curve shape clustering,”
[26] K. Samarakoon, J. Ekanayake and N. Jenkins, “Residential microgrid IEEE Transactions on Power Apparatus and Systems, vol. 102, no. 4,
scheduling based on smart meters data and temperature dependent pp. 893–901, 1983.
thermal load modelling,” IEEE Transactions on Smart Grid, vol. 3, no. [48] M. Espinoza, C. Joye, R. Belmans, and B. DeMoor, “Short-term load
1, pp. 282-292, 2012. forecasting, profile identification, and customer segmentation: A
[27] U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, R. Uthurusamy, 1996, methodology based on periodic time series,” IEEE Transactions on
Advances in knowledge discovery and data mining. AAAI Press/MIT Power Systems, vol. 20, no. 3, pp. 1622–1630, Aug. 2005
Press, Menlo Park/Cambridge [49] J. Nazarko, A. Jurczuk, and W. Zalewski, “ARIMA models in
[28] J. Han, M. Kamber, Data Mining: Concepts and Techniques. Morgan loadmodeling with clustering approach,” in Proceedings of IEEE Russia
Kaufmann, San Francisco, 2006. Power Technology Conference, St. Petersburg, Russia, pp. 1–6, 2005.
[29] N. M. Baqui., Fuzzy Decision Model for the Smart Grid, Masters thesis, [50] H. T. Yang, S. C. Chen, and P. C. Peng, “Genetic k-means-algorithm
North Dakota State University, 2012 based classification of direct load-control curves,” Proceedings of the .
[30] T. Kohonen, "Self-organized formation of topologically correct feature Institute of Electtrical .Engineers, Gen., Transm., Distrib., vol. 152, no.
maps", Biological Cybernetics, vol. 43, no. 1, pp. 59–69, 1982. 4, pp. 489–495, 2005.
[31] C. Cortes and V. Vapnik, "Support-vector networks", Machine [51] A. H. Nizar, Z. Y. Dong, and Y.Wang, “Power utility nontechnical loss
Learning, vol 20, no 3, pp 273, 1995. analysis with extreme learning machine method,” IEEE Transactions on
[32] M. Mittelsdorf, A. Huwel, T. Klingenberg and M. Sonnenschein, Power Systems, vol. 23, no. 3, pp. 946–955, Aug. 2008.
“Submeter based training of multi-class support vector machines for [52] A. Farouqui, “Ethics of Dynamic Pricing” (White Paper), March 2010,
appliance recognition in home electricity consumption data”, Available: http://ideas.repec.org/a/eee/jelect/v23y2010i6p13-27.html
SmartGreens, pp 151-158, SciTePress, 2013. [Accessed 26th April 2014]
[33] J. Z. Moro, L. F. C. Duarte, E. C. Ferreira, J. A. S. Dias "A Home [53] R. F. Chang and C. N. Lu, “Load profiling and its applications in power
Appliance Recognition System Using the Approach of Measuring Power market,” Proceedings of IEEE PES General Meeting, vol. 2, Jul. 2003.
Consumption and Power Factor on the Electrical Panel, Based on [54] S. Valero, M. Ortiz, C. Senabre, C. Alvarez, F. J. G. Franco, and A.
Energy Meter ICs" Circuits and Systems, vol 4, pp.245-251, 2013 Gabaldon, “Methods for customer and demand response policies
[34] S. McLaughlin S, B. Holbert, S. Zonouz and R. Berthier, “AMIDS: A selection in new electricity markets,” IET Generation, Transmission and
multi-sensor energy theft detection framework for advanced metering Distribution, vol. 1, no. 1, pp. 104–110, 2007.
infrastructures” Proceedings of the 2012 3rd IEEE International [55] C. Flath, D. Nicolay, T. Conte, C. van Dinther and L. Filipova-
Conference on Smart Grid Communications (SmartGridComm), pp Neumann, "Cluster analysis of smart metering data - An
354-359, 2012. implementation in practice," Business & Information Systems
[35] M. Anas, N. Javaid, A. Mahmood, S. M. Raza, U. Qasim and Z. Khan, Engineering, vol. 4, no. 1, pp.31-39, 2012.
“Minimizing electricity theft using smart meters in AMI”, Proceedings [56] N. Lu, P. Du, X. Guo, and F. L. Greitzer, “Smart meter data analysis”,
of the 7th International Conference on P2P, Parallel, Grid, Cloud and Proceedimgs of the Transmission and Distribution Conference and
Internet Computing (3PGCIC), Victoria, Canada, 2012 Exposition (T&D), pp 1-6, 2012.
[36] H. Abdi, and L.J. Williams, "Principal component analysis," Wiley [57] H. Liu, Z. Yao, T. Eclund and B. Back, From Smart Meter Data to
Interdisciplinary Reviews: Computational Statistics, vol. 2, pp.433–459, Pricing Intelligence: Real time BI for pricing intelligence, Turku centre
2010. for Computer Science, Finland,TUCS Technical Report, No 1035, 2012
[37] D. Li, Z. Aung, J. Williams, and A. Sanchez, “Efficient authentication [58] I. S´anchez, I. Espin´os, L. Moreno Sarrion, A. L´opez, and I. Burgos, “
scheme for data aggregation in smart grid with fault tolerance and fault Clients segmentation according to their domestic energy consumption
diagnosis”, Proceedings of 2012 3rd IEEE PES International Conference by the use of self-organizing maps”, Proceedings of the 6th
on Innovative Smart Grid Technologies (ISGT), pp 1-8, 2012. International Conference on the European Energy Market (EEM 2009),
[38] J. Abreu, I Azevedo, and F. Pereira, “A contribution to better understand Leuven, Belgium, pages 1–6, May 2009.
the demand for electricity in the residential sector”, Proceedings of the [59] T. Rasanen, J. Ruuskanen, and M. Kolehmainen, “Reducing energy
2011 European Council for Energy Efficient Economies (ECEEE) consumption by using self-organizing maps to create more personalized
Summer Study, pp.1739-1750, 2011. electricity use information”, Applied Energy, vol. 85, no. 9, pp.830–
[39] A. Anwar and A. N. Mahmood, Cyber Security of Smart Grid 840, 2008.
Infrastructure, The State of the Art in Intrusion Prevention and [60] F. McLoughlin, A. Duffy, and M. Conlon, “Analysing domestic
Detection, CRC Press, Taylor & Francis Group, USA, January 2014 electricity smart metering data using self-organising maps”, Proceedings
[40] S. A. Mirmojarabian, “Reliabilty computation of clustered smart meters of the Integration of the Renewables into the Transmission Grid,
using Fuzzy Logic”, Proceedings of the 2013 Iranian Conference on Proceedings of CIRED Workshop, pp 1-4, Lisbon, May 2012.
Fuzzy Systems, pp 1-6, 2013. [61] J. M. Abreu, F. P. Camara, and P. Ferrao, “ Using pattern recognition to
[41] S. Ahmad and Z. Baig, “Fuzzy-based optimization for effective identify habitual behavior in residential electricity consumption”,
detection of smart grid cyber-attacks”, International Journal of Smart Energy and Buildings, vol. 49, pp. 479–487, 2012.
Grid and Clean Energy, vol. 1, no. 1, pp 15-21, September 2012 [62] N. Eagle and A. S. Pentland, “Reality mining: sensing complex social
[42] K. Chahine, K. E. K. Drissi, C. Pasquier, K. Kerroum, C. Faure, T. systems, Personal and Ubiquitous Computing, vol. 10, no. 4, pp. 255–
Jouannet, Michel Michou, Electric Load Disaggregation in Smart 268, 2005.
Metering Using a Novel Feature Extraction Method and Supervised [63] T Zhang, G Zhang, J Lu, X. Feng, and W. Yang, “A new index and
Classification, Energy Procedia, Vol 6, pp 627–632, 2011 classification approach for load pattern analysis of large electricity
[43] R. Lukaszewski, K. Liszewski, W. Winiecki, Methods of Electrical customers”, IEEE Transactions on Power Systems, vol. 27, no. 1, pp
Appliances Identification in Systems Monitoring Electrical Energy 153-160, February 2012
Consumption, Proceedings of the 7th IEEE International Conference on [64] K.-H. Kim, H.-S. Youn, and Y.-C. Kang, “Short-term load forecasting
Intelligent Data Acquisition and Advanced Computing Systems: for special days in anomalous load conditions using neural networks and
Technology and Applications, pp1--14, 2013 fuzzy inference method,” IEEE Transactions on Power Systems., vol.
15, no. 2, pp. 559–565, 2000.
[44] M. Kamgarpoury and H. Tembine, A Bayesian Mean Field Game [65] S. V. Verdu , M. O. Garcia , C. Senabre , A. G. Marin and F. J. G.
Approach to Supply Demand Analysis of the Smart Grid, Proceedings Franco "Classification, filtering, and identification of electrical
of the First IEEE International Black Sea Conference on customer load patterns through the use of self-organizing maps", IEEE
Communications and Networking, pp196-200, 2013 Transactions on Power Systems, vol. 21, no. 4, pp.1672 -1682, 2006.

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TII.2015.2414355, IEEE Transactions on Industrial Informatics

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

[66] H. Pao, "Forecasting electricity market pricing using artificial neural [87] M. Weiss, F. Mattern, T. Graml, T. Staake and E. Fleish “Handy,
networks", Energy Conversion and Management, vol. 48, no. 3, pp. Feedback: Connecting smart meters with mobile phones”, Proceedings
907–912, 2007. of the 8th International Conference on Mobile and Ubiquitous
[67] H. Liao and D. Niebur, “Load profile estimation in electric transmission Multimedia (MUM), pp1-4, 2009
networks using independent component analysis,” IEEE Transactions [88] M. Cheminod, L. Durante and A. Valenzano, “Review of security issues
on Power Systems., vol. 18, no. 2, pp. 707–715, 2003. in industrial networks,” IEEE Transactions on Industrial Informatics,
[68] J. Nagi, K. S. Yap, S. K. Tiong and S. K. Ahmed, “Electrical power load vol. 9, no. 1, pp. 277-293, Jan 2013.
forecasting using hybrid self-organizing maps and support vector [89] B. J. Murrill, E. C.Liu. and R. M. Thompson, Smart Meter Data:
machines”, Proceedings of The 2ndInternational Power Engineering and Privacy and Cybersecurity, CRS Report for Congress, 2012 Available:
Optimization Conference (PEOCO), pp 51-56, Malaysia, June 2008. http://www.fas.org/sgp/crs/misc/R42338.pdf [Accessed 26th April 2014]
[69] D. Osborn and J. Smith, “The performance of periodic autoregressive [90] A. Fromm, F. Kelbert and A. Pretschner, “Data protection in a cloud-
models in forecasting seasonal U.K. consumption,” Journal of Business enabled smart grid”, Smart Grid Security - Lecture Notes in Computer
& Economic Statistics, vol. 7, pp. 117–127, 1989 Science, vol. 7823, pp 96-107, 2013.
[70] G. Guthrie and S. Videbeck, “High frequency electricity spot price [91] D. Bruckner, T. Dillon, S. Hu, P. Palensky, T. Wei, "Guest Editorial
dynamics: An intra-day markets approach,” New Zealand Institute for Special Section on Building Automation, Smart Homes, and
the Study of Competition and Regulation, Tech. Rep., 2002 Communities”, IEEE Transactions on Industrial Informatics, vol. 10, no.
[71] A. Nizar, Advanced Data Mining Methods for Electricity Customer 1, pp. 676-679, 2014
Behaviour Analysis in Power Utility Companies, PhD Thesis, School of [92] C. Gong, J. Liu, Q. Zhang, H. Chen and Z. Gong, “The characteristics of
Information Technol and Elec Engineering, The University of Cloud Computing” Proceedings of the 39th Intl. Conf. on Parallel
Queensland, 2008 Processing Workshops, pp 275-279, 2010
[72] A. Lipsky and N. Mitera, “Some problems of smart meter algorithms for [93] Y. Simmhan, S. Aman, A. Kumbhare, R. Liu, S. Stevens, Q. Zhou, and
electric power quality measurements,” Proceedings of IEEE V. Prasanna, “Cloud-based software platform for data-driven smart grid
International Conference on Smart Measurements for Future Grid management,” Computing in Science and Engineering (CiSE), vol 15,
(SMFG), pp 10-15, 2011. no 4, pp 38-47, 2013.
[73] S. T. Mak, “Dynamic modeling of the distribution feeder using smart [94] S. Karnouskos, “The cooperative Internet of Things enabled smart grid”,
meters data to support feeder feeder volt-var control,” Proceedings of Proceedings of the 2010 14th International Symposium on Consumer
IEEE PES Transmission and Distribution Conference and Exposition Electronics (ISCE), pp 1-6, 2010
(TDCE), pp 1-3, 2012 [95] L. Xu, W. He and S. Li, “Internet of Things in industries: A survey,”
[74] A. Kiani and A. Annaswamy, “The effect of a smart meter on IEEE Transactions on Industrial Informatics, to appear, 2013.
congestion and stability in a power market,” Proceedings of the 49th
IEEE Conference on Decisions and Control (CDC), pp.194-199, 2010 Damminda Alahakoon graduated from the
[75] P. Koponen, L. D. Saco, N. Orchard, T. Vorisek, J. Parsons, C. Rochas, University of Colombo, Sri Lanka with a first class
A. Z. Morch, V. Lopes and M. Togeby, “Definition of Smart Metering honours degree in Computer Science. In 2002 he
and Applications and Identification of Benefits”, European Smart received his PhD degree from Monash University,
Metering Alliance Project Report, 2008. Australia and was also awarded the Monash
[76] A. Abdel-Majeed and M. Braun, “Low voltage system state estimation Artificial Intelligence Prize. He is currently with La
using smart meters,” Proceedings of 2012 47th International Trobe University, Australia, as a Professor in
Universities Power Engineering Conference (UPEC), pp 1-6, London, Business Analytics. His work has been published in
Sept 2012 the areas of data mining, clustering, neural
[77] J. B. Leite and J. R. S. Mantovani, “State estimation of distribution networks, machine learning, information fusion,
networks through the real-time measurements of the smart meters,” incremental learning, selforganization and
Proceedings of 2013 IEEE PowerTech (POWERTECH), Grenoble, pp cognitive systems. He is closely involved with the Australian artificial
1-6, 2013 intelligence community and chaired the 22nd Australasian Artificial
[78] R. H. Douglin, F. A. Ituzaro, and K. L. Butler-Purry, "Using smart meter Intelligence Conference, December 2009, in Melbourne.
data to improve the performance of overcurrent protective devices in
distribution systems with DG,” Proceedings of 2012 IEEE Power and
Energy Society General Meeting, San Diego, pp 1-8, July 2012 Xinghuo Yu (M'92-SM'98-F'08) received BEng and
[79] S. Mahonin, F. Popowich and B. Gill, “The cognitive power meter: MEng degrees from the University of Science and
Looking beyond the smart meter,” Proceedings of 2013 26th IEEE Technology of China, Hefei, China, in 1982 and 1984,
Canadian Conference on Electrical and Computer Engineering and PhD degree from Southeast University, Nanjing,
(CCECE), pp. 1-5, May 2013. China in 1988, respectively. He is currently with
[80] Available: http://www.ict-nobel.eu/ [Accessed 26th April 2014] RMIT University (Royal Melbourne Institute of
[81] S. Karnouskos, P. Goncalves Da Silva, and D. Ilic, “Energy services for Technology), Melbourne, Australia, where he is the
the smart grid city,” Proceedings of 6th IEEE International Conference Founding Director of RMIT Platform Technologies
on Digital Ecosystem Technologies – Complex Environment Research Institute. Professor Yu's research interests
Engineering (DEST-CEE), pp 1-6, Campione d’Italia, Italy, Jun. 2012. include variable structure and nonlinear control,
[82] M. M. Gaber, “Advances in data stream mining”, Wiley complex and intelligent systems and applications. He is an IEEE Distinguished
Interdisciplinary Reviews Data Mining and Knowledge Discovery vol. Lecturer and Vice-President (Publications) of IEEE Industrial Electronics
2, no. 1, pp. 79-85, 2012. Society. Professor Yu received a number of awards and honors for his
[83] D. De Silva, X. Yu, D. Alahakoon and G. Holmes, “A data mining contributions, including 2013 Dr.-Ing. Eugene Mittlemann Achievement
framework for electricity consumption analysis From meter data”, IEEE Award of IEEE Industrial Electronics Society and 2012 IEEE Industrial
Transactions on Industrial Informatics, vol. 7, no. 3, pp. 399-407, 2011. Electronics Magazine Best Paper Award. He also holds Fellowship of the
[84] M. Valls, I. López, and L. V. Iland, “An enhanced middleware for real- Institution of Engineering and Technology (UK), International Energy
time reconfiguration of service oriented distributed real-time systems”,
Foundation, Engineers Australia, Australian Computer Society, and Australian
IEEE Transactions on Industrial Informatics, vol 9, no. 1, pp. 228-236,
Institute of Company Directors.
Feb 2013.
[85] B. Zhang, R. Simon, and H. Aydin. “Harvesting-Aware energy
management for time-critical wireless sensor networks with joint
voltage and modulation scaling”, IEEE Transactions on Industrial
Informatics vol 9, no. 1, pp 514-526, Feb 2013
[86] Essential Services Commission 2011, Smart meters regulatory review –
Capacity control and meter reads final decision (Government Report),
Available: http://www.esc.vic.gov.au/getattachment/5c4c3b6b-94d3-
4c19-b015-8631420397ed/Final-Decision-Supply-Capacity-Control.pdf
[Accessed 26th April 2014]

1551-3203 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.