Machine Learning For NY Power Grid

328 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 34, NO.
2, FEBRUARY 2012
Machine Learning for the

New York City Power Grid
Cynthia Rudin, David Waltz, Senior Member, IEEE, Roger N. Anderson, Member, IEEE,
Albert Boulanger, Member, IEEE, Ansaf Salleb-Aouissi, Maggie Chow, Haimonti Dutta,
Philip N. Gross, Bert Huang, Steve Ierome, Delfina F. Isaac, Arthur Kressner,
Rebecca J. Passonneau, Axinia Radeva, and Leon Wu, Member, IEEE
Abstract—Power companies can benefit from the use of knowledge discovery methods and statistical machine learning for preventive
maintenance. We introduce a general process for transforming historical electrical grid data into models that aim to predict the risk of
failures for components and systems. These models can be used directly by power companies to assist with prioritization of
maintenance and repair work. Specialized versions of this process are used to produce 1) feeder failure rankings, 2) cable, joint,
terminator, and transformer rankings, 3) feeder Mean Time Between Failure (MTBF) estimates, and 4) manhole events vulnerability
rankings. The process in its most general form can handle diverse, noisy, sources that are historical (static), semi-real-time, or real-
time, incorporates state-of-the-art machine learning algorithms for prioritization (supervised ranking or MTBF), and includes an
evaluation of results via cross-validation and blind test. Above and beyond the ranked lists and MTBF estimates are business
management interfaces that allow the prediction capability to be integrated directly into corporate planning and decision support; such
interfaces rely on several important properties of our general modeling approach: that machine learning features are meaningful to
domain experts, that the processing of data is transparent, and that prediction results are accurate enough to support sound decision
making. We discuss the challenges in working with historical electrical grid data that were not designed for predictive purposes. The
“rawness” of these data contrasts with the accuracy of the statistical models that can be obtained from the process; these models are
sufficiently accurate to assist in maintaining New York City’s electrical grid.
Index Terms—Applications of machine learning, electrical grid, smart grid, knowledge discovery, supervised ranking, computational
sustainability, reliability.
1 INTRODUCTION
O NE of the major findings of the US Department of

Energy’s “Grid 2030” strategy document [1] is that
“America’s electric system, ‘the supreme engineering
transform throughout the next several decades, and grid
maintenance will become even more critical than it is
currently. A 2007 survey by the NERC [2] states that “aging
achievement of the 20th century’ is aging, inefficient, infrastructure and limited new construction” is the largest
congested, incapable of meeting the future energy needs challenge to electrical grid reliability out of all challenges
[...].” Reliability will be a key issue as electrical grids considered by the survey (also see [3]). The smart grid will
bring operations and maintenance more online—moving
the industry from reactive to proactive operations. Power
. C. Rudin is with the MIT Sloan School of Management, Massachusetts
companies keep historical data records regarding equip-
Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139
and also with the Center for Computational Learning Systems, Columbia ment and past failures, but those records are generally
University, 475 Riverside Drive, MC 7717 (850 Interchurch Center), New not being used to their full extent for predicting grid
York, NY 10115. E-mail: [email protected]. reliability and assisting with maintenance. This is starting
. D. Waltz, R.N. Anderson, A. Boulanger, A. Salleb-Aouissi, H. Dutta,
B. Huang, R.J. Passonneau, A. Radeva, and L. Wu are with the Center to change. This paper presents steps toward proactive
for Computational Learning Systems, Columbia University, 475 River- maintenance programs for electrical grid reliability based
side Drive, MC 7717 (850 Interchurch Center), New York, NY 10115. on the application of knowledge discovery and machine
E-mail: {waltz, ansaf, haimonti, axinia, leon}@ccls.columbia.edu, learning methods.
{anderson, aboulanger}@ldeo.columbia.edu, {bert, becky}@cs.columbia.edu.
. M. Chow, S. Ierome, and D.F. Isaac are with Consolidated Edison Most power grids in US cities have been built gradually
Company of New York, 4 Irving Place, New York, NY 10003. over the last 120 years. This means that the electrical
E-mail: {chowm, ieromes, isaacd}@coned.com. equipment (transformers, cables, joints, terminators, and
. P.N. Gross is with Google, Inc., 76 Ninth Avenue, New York, NY 10011.
E-mail: [email protected]. associated switches, network protectors, relays, etc.) vary
. A. Kressner is with Grid Connections, LLC, Consolidated Edison Company in age; for instance, at least 5 percent of the low voltage
of New York, 4 Irving Place, New York, NY 10003. cables in Manhattan were installed before 1930, and a few
E-mail: [email protected].
of the original high voltage distribution feeder sections
Manuscript received 1 Oct. 2010; revised 3 Mar. 2011; accepted 15 Apr. 2011;
published online 13 May 2011. installed during the Thomas Edison era are still in active
Recommended for acceptance by D. Schuurmans. use in New York City. In NYC there are over 94,000 miles
For information on obtaining reprints of this article, please send e-mail to: of high voltage underground distribution cable, enough to
[email protected], and reference IEEECS Log Number
TPAMI-2010-10-0753. wrap around the earth three and a half times. Florida
Digital Object Identifier no. 10.1109/TPAMI.2011.108. Power and Light Company (FPL) has 24,000 miles of
0162-8828/12/$31.00 ß 2012 IEEE Published by the IEEE Computer Society
RUDIN ET AL.: MACHINE LEARNING FOR THE NEW YORK CITY POWER GRID 329
underground cable1 and many other utilities manage aggregation, formation of features and labels, ranking
similarly large underground electric systems. Maintaining methods), and evaluation (blind tests, visualization). Specia-
a large grid that is a mix of new and old components is lized versions of the process have been developed for:
more difficult than managing a new grid (for instance, as is
being laid in some parts of China). The US grid is generally 1. feeder failure ranking for distribution feeders,
older than many European grids that were replaced after 2. cable section, joint, terminator, and transformer
WWII, and older than grids in places where infrastructure ranking for distribution feeders,
must be continually replenished due to natural disasters 3. feeder Mean Time Between Failure (MTBF) esti-
(for instance, Japan has earthquakes that force power mates for distribution feeders, and
systems to be replenished). 4. manhole vulnerability ranking.
The smart grid will not be implemented overnight; to Each specialized process was designed to handle data with
create the smart grid of the future, we must work with the particular characteristics. In its most general form, the
electrical grid that is there now. For instance, according to process can handle diverse, noisy, sources that are historical
the Brattle Group [4], the cost of updating the grid by 2030 (static), semi-real-time, or real time; the process incorpo-
could be as much as $1.5 trillion. The major components of rates state-of-the-art machine learning algorithms for
the smart grid will (for an extended period) be the same as prioritization (supervised ranking or MTBF), and includes
the major components of the current grid, and new an evaluation of results via cross-validation on past data
intelligent meters must work with the existing equipment. and by blind evaluation. The blind evaluation is performed
Converting to a smart grid can be compared to replacing on data generated as events unfold, giving a true barrier to
worn parts of an airplane while it is in the air. As grid parts information in the future. The data used by the machine
are replaced gradually and as smart components are added, learning algorithms include past events (failures, replace-
the old components, including cables, switches, sensors, ments, repairs, tests, loading, power quality events, etc.)
etc., will still need to be maintained. Further, the state of the and asset features (type of equipment, environmental
old components should inform the priorities for the conditions, manufacturer, specifications, components con-
addition of new smart switches and sensors. nected to it, borough and network where it is installed, date
The key to making smart grid components effective is to of installation, etc.).
analyze where upgrades would be most useful, given the Beyond the ranked lists and MTBF estimates, we have
current system. Consider the analogy to human patients in designed graphical user interfaces that can be used by
the medical profession, a discipline for which many of the
managers and engineers for planning and decision support.
machine learning algorithms and techniques used here for
Successful NYC grid decision support applications based
the smart grid were originally developed and tested. While
on our models are used to assist with prioritizing repairs,
each patient (a feeder, transformer, manhole, or joint) is
prioritizing inspections, correcting of overtreatment, gen-
made up of the same kinds of components, they wear and
age differently, with variable historic stresses and heredi- erating plans for equipment replacement, and prioritizing
tary factors (analogous to different vintages, loads, manu- protective actions for the electrical distribution system.
facturers) so that each patient must be treated as a unique How useful these interfaces are depends on how accurate
individual. Nonetheless, individuals group into families, the underlying predictive models are, and also on the
neighborhoods, and populations (analogous to networks, interpretation of model results. It is an important property
boroughs) with relatively similar properties. The smart grid of our general approach that machine learning features are
must be built upon a foundation of helping the equipment meaningful to domain experts in that the data processing
(patients) improve their health so that the networks and the way causal factors are designed is transparent. The
(neighborhoods) improve their life expectancy and the transparent use of data serves several purposes: It allows
population (boroughs) lives more sustainably. domain experts to troubleshoot the model or suggest
In the late 1990s, NYC’s power company, Con Edison, extensions, it allows users to find the factors underlying
hypothesized that historical power grid data records could the root causes of failures, and it allows managers to
be used to predict, and thus prevent, grid failures and understand, and thus trust, the (non-black-box) model in
possible associated blackouts, fires, and explosions. A order to make decisions.
collaboration was formed with Columbia University, begin- We implicitly assume that data for the modeling tasks
ning in 2004, in order to extensively test this hypothesis. will have similar characteristics when collected by any
This paper discusses the tools being developed through this power company. This assumption is broadly sound but
collaboration for predicting different types of electrical grid there can be exceptions; for instance, feeders will have
failures. The tools were created for the NYC electrical grid; similar patterns of failure across cities, and data are probably
however, the technology is general and is transferable to collected in a similar way across many cities. However, the
electrical grids across the world. levels of noise within the data and the particular conditions
In this work, we present new methodologies for main- of the city (maintenance history, maintenance policies,
taining the smart grid, in the form of a general process for network topologies, weather, etc.) are specific to the city
failure prediction that can be specialized for individual and to the methods by which data are collected and stored
applications. Important steps in the process include data by the power company.
processing (cleaning, pattern matching, statistics, integra- Our goals for this paper are to demonstrate that data
tion), formation of a database, machine learning (time collected by electrical utilities can be used to create
statistical models for proactive maintenance programs, to
1. http://www.fpl.com/faqs/underground.shtml. show how this can be accomplished through knowledge
330 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 34, NO. 2, FEBRUARY 2012
weaknesses in any of these components: a feeder may go out

of service, the cables, joints, and terminators can fail,
transformers can fail, and insulation breakdown of cables
in the secondary electrical grid can cause failures. In what
follows, we discuss how data-driven preemptive mainte-
nance policies can assist with preventing these failures.
2.1 Feeder Rankings

Primary distribution feeder cables are large cables; in NYC
they operate at 13 kV or 27 kV. They generally lie along
main streets or avenues and distribute power from
substations to the secondary grid.
Fig. 1. Typical electrical infrastructure in cities. Source: Con Edison. A feeder may experience an outage due to a fault
somewhere along the feeder or due to deliberate de-
discovery and machine learning, and to encourage compa- energizing (so maintenance can be performed). If one
nies across the world to reconsider the way data are being component, such as a feeder, fails or is taken out of service,
collected and stored in order to be most effective for this failure is called a “first contingency,” and if two
prediction and decision-support applications. components in the same network fail, it is called a “second
In Section 2, we discuss the electrical grid maintenance contingency,” and so forth. Loss of a small number of
tasks. Section 3 contains the general process by which data feeders generally does not result in any interruption in
can be used to accomplish these tasks. In Section 4, we customers’ electricity service due to extensive redundancy
discuss the specific machine learning methods used for the in the system. (For instance, Con Edison’s underground
system is designed to operate under second contingency.)
knowledge discovery process. Section 5 presents the specia-
However, once one or more feeders in a network are out of
lized versions of the general process for the four prediction
service, the remaining feeders and their associated trans-
tasks. In Section 6, we give sample results for the NYC power
formers have to pick up the load of the feeders that were
grid. Section 7 discusses the prototype tools for management
disconnected. This added load elevates the risk of failure for
we have developed in order to make the results useable and
the remaining feeders and transformers and, past a certain
to assist in knowledge discovery. Section 8 presents related
point, the network will experience a cascading failure,
work. Section 9 presents lessons learned from the imple-
where the remaining components are unable to carry the
mentation of these systems on the NYC grid.
network’s load and the entire network must be shut down
until the system can be repaired.
2 PROACTIVE MAINTENANCE TASKS Each feeder consists of many cable sections (called
“sections” in what follows); for instance, the average number
Power companies are beginning to switch from reactive
of sections per feeder in NYC is approximately 150. Each
maintenance plans (fix when something goes wrong) to
section runs between two manholes and has “joints” at each
proactive maintenance plans (fix potential problems
end. Sections are often made up of three bundled cables,
before they happen). There are advantages to this:
one for each voltage phase. Joints can attach two single
Reactive plans, which allow failures to happen, can lead
cable sections or can branch two or more ways. Ultimately,
to dangerous situations, for instance fires and cascading
feeder sections end at transformers that step down the
failures, and costly emergency repairs. However, it is not
voltage to 120 or 240 Volts needed for the secondary system.
a simple task to determine where limited resources should Feeder sections are connected to transformers by “hammer-
be allocated in order to most effectively repair potentially heads,” which are terminators that are named for their
vulnerable components. distinctive shape. Feeder failures generally occur at the
In large power systems, electricity flows from source to joints or within a cable section. In this section, we discuss
consumer through transmission lines to substations, then to the problem of predicting whether a given feeder will have
primary feeder cables (“feeders”), and associated cable a failure (including failures on any of its subcomponents),
sections, joints, and terminators, through transformers, and and in the following section, we discuss the prediction of
to the secondary (low-voltage) electrical distribution grid failures on individual feeder components, specifically on
(see Fig. 1). There are two types of feeders, “distribution the individual cable sections, joints, and hammerheads. We
feeders” and “transmission feeders.” Our work has mainly use the results from the individual component failure
focused on distribution feeders (the term “feeder” will predictions as input to the feeder failure prediction model.
indicate distribution feeders), which are large medium-to- One kind of joint, the “stop joint,” is the source of a
high-voltage cables that form a tree-like structure, with disproportionate number of failures. Stop joints connect old
transformers at the leaves. In some cities, these transformers “PILC” to modern cables with solid dielectrics. PILC stands
serve buildings or a few customers, and a feeder failure leads for Paper-Insulated Lead-sheathed Cable, an older technol-
to service interruptions for all of these downstream ogy used in most urban centers from 1906 through the
customers. In other cities, including NYC, the secondary 1960s. PILC sections are filled with oil, so stop joints must
cables form a mesh or grid-like structure that is fed by not only have good electrical connections and insulation
redundant high-voltage feeders, with the goal of continuing (like all joints) but must also cap off the oil to prevent it
service, even if one or more feeders fail. There can be possible from leaking. Even though all utilities are aggressively
Fig. 3. Excerpt from sample smoking manhole (SMH) trouble ticket.
important: Scheduled replacement of each section costs

about $18,000. Feeder failures require even more expensive
emergency replacements and also carry a risk of cascading
failures.
Fig. 2. Number of feeder outages in NYC per day during 2006-2007, 2.2 Cable Sections, Joints, Terminators and
lower curve with axis at left, and system-wide peak system load, upper Transformers Ranking
curve with axis at right.
In Section 2.1, we discussed the task of predicting whether a
failure would happen to any component of a (multi-
removing lead cable from their systems, it is going to be a
component) feeder. We now discuss the task of modeling
long time before the work is complete.2 For instance, in
failures on individual feeder components; modeling how
NYC, the Public Service Commission has mandated that all individual components fail brings an extra level to the
30;000 remaining PILC sections be replaced by 2020. Note, understanding of feeder failure. Features of the components
however, that some PILC sections have been in operation can be more directly related to localized failures and kept in
for a very long time without problems, and it is important a nonaggregated form; for instance, a feature for the
to make the best use of the limited maintenance budget by component modeling task might encode that a PILC section
replacing the most unreliable sections first. was made by Okonite in 1950, whereas a feature for the
As can be seen in Fig. 2, a small number of feeder failures feeder modeling task might instead be a count of PILC
occur daily in NYC throughout the year. The rate of failures sections greater than 40 years old for the feeder. The
noticeably increases during warm weather; air conditioning component rankings can also be used to support decisions
causes electricity usage to increase by roughly 50 percent about which components to prioritize after a potentially
during the summer. It is during these times when the susceptible feeder is chosen (guided by the results of the
system is most at risk. feeder ranking task). In that way, if budget constraints
The feeder failure ranking application, described in
prohibit replacement of all the bad components of a feeder,
Section 5.1, orders feeders from most at-risk to least at-risk.
the components that are most likely to fail can be replaced.
Data for this task include: physical characteristics of the
For Con Edison, the data used for ranking sections,
feeder, including characteristics of the underlying compo-
joints, and hammerheads were diverse and fairly noisy,
nents that compose the feeder (e.g., percent of PILC
though in much better shape than the data used for the
sections); date put into service; records of previous “open
autos” (feeder failures), previous power quality events manhole events prediction project we describe next.
(disturbances), scheduled work, and testing; electrical 2.3 Manhole Ranking
characteristics, obtained from electric load flow simulations
A small number of serious “manhole events” occur each year
(e.g., how much current a feeder is expected to carry under
in many cities, including fires and explosions. These events
various network conditions); and dynamic data, from real-
are usually caused by insulation breakdown of the low-
time telemetry attached to the feeder. Approximately 300
voltage cable in the secondary network. Since the insulation
summary features are computed from the raw data, for
can break down over a long period of time, it is reasonable
example, the total number of open autos per feeder over the
to try to predict future serious events from the character-
period of data collection. For Con Edison, these features are
istics of past serious and nonserious events. We consider
reasonably complete and not too noisy. The feeder failure
events within two somewhat simplified categories: serious
rank lists are used to provide guidance for Con Edison’s
events (fires, explosions, serious smoking manholes) and
contingency analysis and winter/spring replacement pro- potential precursor events (burnouts, flickering lights, etc.).
grams. In the early spring of each year, a number of feeders Potential precursor events can be indicators of an area-wide
are improved by removing PILC sections, changing the network problem, or they can indicate that there is a local
topology of the feeders to better balance loading, or to problem affecting only 1-2 manholes.
support changing power requirements for new buildings. Many power companies keep records of all past events in
Loading is light in spring, so feeders can be taken out of
the form of trouble tickets, which are the shorthand notes
service for upgrading with low risk. Prioritizing feeders is
taken by dispatchers. An example ticket for an NYC smoking
2. For more details, see the article about replacement of PILC in NYC manhole event appears in Fig. 3. Any prediction algorithm
http://www.epa.gov/waste/partnerships/npep/success/coned.htm. must consider how to effectively process these tickets.
Fig. 4. Bathtub curve. Source Wikipedia: http://en.wikipedia.org/wiki/

Bathtub_curve.
2.4 MTBF Modeling

A common and historical metric for reliability performance
is mean time between failures for components or systems
that can be repaired, and mean time to failure (MTTF) for
components that cannot.3 Once MTBF or MTTF is esti-
mated, a cost versus benefit analysis can be performed, and
replacement policies, inspection policies, and reliability
improvement programs can be planned. Feeders are made
up of multiple components that can fail, and these
components can be replaced separately, so MTBF (rather
than MTTF) is applicable for feeder failures. When an Fig. 5. Process diagram.
individual joint (or other component of a feeder) fails, it is
then replaced with a new one, so MTTF is applicable or more machine learning problems are formulated over an
instead for individual component failures. appropriate timescale. Ideally, the features used in the
In general, the failure rate of a component or a composite machine learning models are meaningful to the domain
system like a feeder will have a varying MTBF over its experts. The parameters in the machine learning algorithm
lifetime. A system that is new or has just had maintenance are tuned or tested by cross-validation, and evaluated for
may have early failures, known as “infant mortality.” Then, prediction accuracy by blind prediction tests on data that
systems settle down into their midlife with a lower failure are not in the database. Domain experts also evaluate the
rate, and finally, the failure rate increases at the end of their model using business management tools and suggest
lifetimes. (See Fig. 4.) PILC can have very long lifetimes and improvements (usually in the initial handling and cleaning
it is hard to determine an end of life signature for them. of data).
Transformers do show aging with an increase in failure rate. The data processing/cleaning is the key piece that
ensures the integrity of the resulting model. This view
3 A PROCESS FOR FAILURE PREDICTION IN POWER agrees with that of Hsu et al. [9], who state that “... the often
neglected preprocessing and postprocessing steps in knowl-
GRIDS
edge discovery are the most critical elements in determining
Our general goal is “knowledge discovery,” that is, finding the success of a real-life data mining application.” Data
information in data that is implicit, novel, and potentially cleaning issues have been extensively discussed in the
extremely useful [5]. Harding et al. [6] provide an overview literature, for instance, in e-commerce [10]. Often, the
of knowledge discovery in manufacturing. The general application of machine learning techniques directly (with-
CRISP-DM framework [7] captures the data processing for out the data cleaning step) does not lead to useful or
(potentially) extremely raw data; however, the traditional meaningful models. In electrical utility applications, these
knowledge discovery in databases (KDD) outline [8] does data can be extremely raw: Data can come from diverse
not encompass this. The general process presented here can sources throughout the company, with different schemes
be considered a special case of CRISP-DM, but it is outside for recording times for events or identities of components,
the realm of KDD. they may be incomplete or extremely noisy, or they may
The general knowledge discovery process for power grid
contain large numbers of free-text documents (for example,
data is shown in Fig. 5. The data can be structured text or
trouble tickets). The data processing step fully defines the
categorical data, numerical data, or unstructured text
interpretation of the data that will be used by the machine
documents. The data are first cleaned and integrated into
learning model. This processing turns historical data from
a single database that can be accurately queried. Then, one
diverse sources into useable features and labels for learning.
3. See Wikipedia’s MTBF page: http://en.wikipedia.org/wiki/Mean_ Data cleaning can include many steps, such as pattern
time_between_failures. matching (for instance, finding regular expressions in
structured or unstructured data), information extraction, . AUC or weighted AUC: Area under the ROC curve
text normalization, using overlapping data to find incon- [11], or Wilcoxon Mann Whitney U statistic, as
sistencies, and inferring related or duplicated records. formulated in Section 4 below. The AUC is related to
Statistics can be used to assess whether data are missing, the number of times a failure is ranked below a
and for sanity checks on inferential joins. nonfailure in the list. Weighted AUC metrics (for
An inferential join is the process by which multiple raw instance, as used the P-Norm Push algorithm [12]
data tables are united into one database. Inferential joins are derived in Section 4) are more useful when the top of
a key piece of data processing. An example to illustrate the the list is the most important.
logic behind using basic pattern matching and statistics for For MTBF/MTTF estimation, the sum of squared differ-
inferential joining is the uniting of the main cable records to ences between estimated MTBF/MTTF and true MTBF/
the raw manhole location data for the manhole event MTTF is the evaluation measure.
process in NYC, to determine which cables enter into which The evaluation stage often produces changes to the initial
manholes. Main cables connect two manholes (as opposed processing. These corrections are especially important for
to service or streetlight cables that enter only one manhole). ranking problems. In ranking problems where the top of the
The cable data come from Con Edison’s accounting list is often the most relevant, there is a possibility that top
department, which is different from the source of the of the list will be populated completely by outliers that are
manhole location data. A raw join of these two tables, based caused by incorrect or incomplete data processing, and thus
on a unique manhole identifier that is the union of three the list is essentially useless. This happens particularly
fields—manhole type, number, and local 3-block code— when the inferential joins are noisy; if a feeder is incorrectly
provided a match to only about half of the cable records. linked to a few extra failure events, it will seem as if this
We then made a first round of corrections to the data, where feeder is particularly vulnerable. It is possible to trouble-
we unified the spelling of the manhole identifiers within shoot this kind of outlier by performing case studies of the
both tables, and found matches to neighboring 3-block components on the top of the ranked lists.
codes (the neighboring 3-block code is often mistakenly
entered for manholes on a border of the 3 blocks). The next 4 MACHINE LEARNING METHODS: RANKING FOR
round of corrections used the fact that main cables have
RARE EVENT PREDICTION
limited length: If only one of the two ends of the cable was
uniquely matched to a manhole, with several possible The subfield of ranking in machine learning has expanded
manholes for the other end, then the closest of these rapidly over the past few years as the information retrieval
manholes was selected (the shortest possible cable length). community has started developing and using these meth-
This processing gave a match to about three-quarters of the ods extensively (see the LETOR website4 and references
cable records. A histogram of the cable length then therein). Ranking algorithms can be used for applications
indicated that about 5 percent of these joined records beyond information retrieval; our interest is in developing
represented cables that were too long to be real. Those and applying ranking algorithms to rank electrical grid
cables were used to troubleshoot the join again. Statistics components according to the probability of failure. In IR,
can generally assist in finding pockets of data that are not the goal is to rank a set of documents in order of relevance
joined properly to other relevant data. to a given query. For both electrical component ranking and
Data can be either: static (representing the topology of IR, the top of the list is considered to be the most important.
the network, such as number of cables, connectivity), The ranking problems considered here fall under the
semidynamic (e.g., only changes when a section is removed general category of supervised learning problems and,
or replaced, or when a feeder is split into two), or dynamic specifically, supervised bipartite ranking. In supervised
(real time, with timestamps). The dynamic data can be bipartite ranking tasks, the goal is to rank a set of randomly
measured electronically (e.g., feeder loading measure- drawn examples (the “test set”) according to the probability
ments), or it can be measured as failures occur (e.g., trouble of possessing a particular attribute. To do this, we are given
tickets). For the semidynamic and dynamic data, a timescale a “training set” that consists of examples with labels:
of aggregation needs to be chosen for the features and labels fðxi ; yi Þgm
i¼1 ; xi 2 X ; yi 2 f1; þ1g:
for machine learning.
For all four applications, machine learning models are In this case, the examples are electrical components, and the
formed, trained, and cross-validated on past data, and label we want to predict is whether a failure will occur within
evaluated via “blind test” on more recent data, discussed a given time interval. It is assumed that the training and test
further in Section 4. examples are both drawn randomly from the same unknown
For ranking algorithms, the evaluation measure is distribution. The examples are characterized by features:
usually a statistic of a ranked list (a rank statistic), and
fhj gnj¼1 ; hj : X ! R:
ranked lists are visualized as Receiver Operator Character-
istic (ROC) curves. Evaluation measures include: The features should encode all information that is relevant
for predicting the vulnerability of the components, for
. Percent of successes in the top k percent: The percent
instance, characteristics of past performance, equipment
of components that failed within the top k percent of
the ranked list (similar to “precision” in information 4. http://research.microsoft.com/en-us/um/beijing/projects/letor/
retrieval (IR)). paper.aspx.
A common quality measure in supervised ranking is the

probability that a new pair of randomly chosen examples is
misranked (see [14]), which should be minimized:
PD fmisrankðf Þg
ð1Þ
:¼ PD ff ðxþ Þ f ðx Þ j yþ ¼ 1; y ¼ 1g:
The notation PD indicates the probability with respect to a
random draw of ðxþ ; yþ Þ and ðx ; y Þ from distribution D
Fig. 6. Sample timeline for rare event prediction. on X f1; þ1g. The empirical risk corresponding to (1) is
the number of misranked pairs in the training set:
manufacturer, and type of equipment. To demonstrate, we X X
can have: h1 ðxÞ ¼ the age of component x, h2 ðxÞ ¼ the Rðf Þ : ¼ 1½f ðxi Þf ðxk Þ
number of past failures involving component x, and h3 ðxÞ ¼
fk:yk ¼1g fi:yi ¼1g ð2Þ
1 if x was made by a particular manufacturer. These features ¼ #misranksðf Þ:
can be either correlated or uncorrelated with failure predic- The pairwise misranking error is directly related to the
tion; the machine learning algorithm will use the training set (negative of the) area under the ROC curve; the only
to choose which features to use and determine the impor-
difference is that ties are counted as misranks in (2). Thus, a
tance of each feature for predicting future failures.
natural ranking algorithm is to choose a minimizer of Rðf Þ
Failure prediction is performed in a rare event prediction
with respect to :
framework, meaning the goal is to predict events within a
given “prediction interval” using data prior to that interval. 2 argmin Rðf Þ;
There are separate prediction intervals for training and 2Rn
testing. The choice of prediction intervals determines the and to rank the components in the test set in descending
P
labels y for the machine learning problem and the features hj . order of f ðxÞ :¼ j j hj ðxÞ.
Specifically, for training, yi is þ1 if component i failed There are three shortcomings to this algorithm: First, it is
during the training prediction interval and 1 otherwise. computationally hard to minimize Rðf Þ directly. Second,
The features are derived from the time period prior to the the misranking error Rðf Þ considers all misranks equally,
prediction interval. For instance, as shown in Fig. 6, if the in the sense that misranks at the top of the list are counted
goal is to rank components for vulnerability with respect to equally with misranks toward the bottom, even though, in
2010, the model is trained on features derived from prior to failure prediction problems, it is clear that misranks at the
2009 and labels derived from 2009. The features for testing top of the list should be considered more important. A third
are derived from pre-2010 data. The choice of the prediction shortcoming is the lack of regularization usually imposed to
interval’s length is application dependent; if the interval is enable generalization (prediction ability) in high dimen-
too small, there may be no way to accurately characterize sions. A remedy for all of these problems is to use special
failures. If the length is too large, the predictions may be too cases of the following ranking objective that do not fall into
coarse to be useful. For manhole event prediction in NYC, any of the traps listed above:
this time period was chosen to be one year, and time 0 1
aggregation was performed using the method of Fig. 6 for X X
manhole event prediction. A more elaborate time aggrega- R‘g ðf Þ :¼ g@ ‘ðf ðxi Þ f ðxk ÞÞA þ Ck
k2 ;
tion scheme is discussed in Section 5.1 for feeder failure fk:yk ¼1g fi:yi ¼1g
ranking, where “time shifted” features were used. ð3Þ

The ranking algorithm uses the training set to
construct a scoring function, which is a linear combina- where g is called the price function and ‘ is called the loss
tion of the features: function. Rðf Þ given in (2) is a special case of R‘g ðf Þ with
‘ðzÞ ¼ 1z0 and gðzÞ ¼ z. The objective is convex in when
X
n
the exponential loss is used, ‘ðzÞ ¼ ez [14], or the SVM
f ðxÞ ¼ j hj ðxÞ; (support vector machine) hinge loss, ‘ðzÞ ¼ ð1 zÞþ [15];
j¼1
several other convex loss functions are also commonly used.
and the examples are rank-ordered by their scores. The The norm used in the regularization term is generally either
ranking algorithm constructs f by minimizing, with respect a norm in a Reproducing Kernel Hilbert space (for PSVMs),
2 2
to the vector of coefficients :¼ ½1 ; . . . ; n , a quality which, in the simplest
P case, is the ‘ 2 norm k
k2 ¼ j j , or
measure (a statistic) of the ranked list, denoted Rðf Þ. The an ‘1 norm k k1 ¼ j jj j. The constant C can be set by
procedure for optimizing Rðf Þ is “empirical risk mini- cross-validation.
mization” where the statistic is optimized on the training set Special cases of the objective (3) are: SVM Rank [15],
and the hope is that the solution generalizes to the full which uses the hinge loss, with gðzÞ ¼ z as the price
unknown distribution. Particularly, it is hoped that the function, and Reproducing Kernel Hilbert space regulariza-
scoring function will rank the test examples accurately so tion; RankBoost [14], which uses the exponential loss and no
that the positive examples are on the top of the list. regularization; and the P-Norm Push [12]. The P-Norm
Probabilistic generalization bounds are used to theoretically Push uses price function gðzÞ ¼ zp , which forces the value of
justify this type of approach (e.g., [13], [14]). the objective to be determined mainly by the highest ranked
negative examples when p is large; the power p acts as a soft in Section 5.1. We discuss the data processing challenges for
max. Since most of the value of the objective is determined cables, joints, terminators, and transformers in Section 5.2.
by the top portion of the list, the algorithm concentrates The manhole event prediction process is discussed in
more on the top. The full P-Norm Push algorithm is Section 5.3, and the MTBF estimation process is discussed
in Section 5.4.
2 arg inf Rp ð
Þ where
0 1p 5.1 Feeder Ranking in NYC
X X
Rp ð
Þ :¼ @ expð½f ðxi Þ f ðxk ÞÞA : Con Edison data regarding the physical composition of
fk:yk ¼1g fi:yi ¼1g feeders are challenging to work with; variations in the

database entry and rewiring of components from one feeder
Vector is not difficult to compute, for instance by to another make it difficult to get a perfect snapshot of the
gradient descent. The P-Norm Push is used currently in the current state of the system. It is even more difficult to get
manhole event prediction tool. An SVM algorithm with snapshots of past states of the system; the past state needs
‘2 regularization is currently used in the feeder failure tool. to be known at the time of each past failure because it is
Algorithms designed via empirical risk minimization are used in training the machine learning algorithm. A typical
not designed to be able to produce density estimates, that is, feeder is composed of over a hundred cable sections,
estimates of P ðy ¼ 1jxÞ, though in some cases it is possible, connected by a similar number of joints, and terminating in
particularly when the loss function is smooth. These algo- a few tens of transformers. For a single feeder, these
rithms are instead designed specifically to produce an subcomponents are a hodgepodge of types and ages; for
accurate ranking of examples according to these probabilities. example, a brand-new cable section may be connected to
It is important to note that the specific choice of machine
one that is many decades old; this makes it challenging to
learning algorithm is not the major component of success in
“roll-up” the feeder into a set of features for learning. The
this domain; rather, the key to success is the data cleaning
features we currently use are statistics of the ages, numbers,
and processing as discussed in Section 3. If the machine
and types of components within the feeder; for instance, we
learning features and labels are well constructed, any
have considered maxima, averages, and 90th percentiles
reasonable algorithm will perform well; the inverse holds
(robust versions of the maxima).
too, in that badly constructed features and labels will not
Dynamic data presents a similar problem to physical
yield a useful model regardless of the choice of algorithm.
data, but here the challenge is aggregation in time instead
For our MTBF application, MTBF is estimated indirectly
of space. Telemetry data are collected at rates varying from
through failure rates; the predicted failure rate is converted
hundreds of times per second (for power quality data) to
to MTBF by taking the reciprocal of the rate. The failure rate
only a few measurements per day (weather data). These
is estimated rather than MTBF for numerical reasons: Good
can be aggregated over time, again using functions such as
feeders with no failures have an infinite MTBF. The failure
max or average, using different time windows (as we
rate is estimated by regression algorithms, for instance,
describe shortly). Some of the time windows are relatively
support vector machine regression (SVM-R) [16], Classifica-
simple (e.g., aggregating over 15 or 45 days), while others
tion and Regression Trees (CART) [17], ensemble-based
take advantage of the system’s periodicity, and aggregate
techniques such as Random Forests [18], and statistical
methods, e.g., Cox Proportional Hazards [19]. over the most recent data plus data from the same time of
year in previous years.
One of the challenges of the feeder ranking application is
5 SPECIFIC PROCESSES AND CHALLENGES that of imbalanced data, or scarcity of data characterizing
In this section, we discuss how the general process needs to the failure class, which causes problems with general-
be adapted in order to handle data processing and machine ization. Specifically, primary distribution feeders are sus-
learning challenges specific to each of our electrical ceptible to different kinds of failures, and we have very few
reliability tasks in NYC. Con Edison currently operates training examples for each kind, making it difficult to
the world’s largest underground electric system, which reliably extract statistical regularities or determine the
delivers up to a current peak record of about 14,000 MW of features that affect reliability. For instance, failure can be
electricity to over 3 million customers. A customer can be an due to: concurrent or prior outages that stress the feeder
entire office building or apartment complex in NYC so that and other feeders in the network; aging; power quality
up to 15 million people are served with electricity. Con events (e.g., voltage spikes); overloads (that have seasonal
Edison is unusual among utilities in that it started keeping variation, like summer heat waves); known weak compo-
data records on the manufacturer, age, and maintenance nents (e.g., joints connecting PILC to other sections); at-risk
history of components over a century ago, with an topologies (where cascading failures could occur); the stress
increased level of Supervisory Control and Data Acquisition of “HiPot” (high potential) testing; and deenergizing/
(SCADA) added over the last 15 years. While real-time data reenergizing of feeders that can result in multiple failures
are collected from all transformers for loading and power within a short time span due to “infant mortality.” Other
quality information, that is much less than will be needed data scarcity problems are caused by the range in MTBF of
for a truly smart grid. the feeders; while some feeders are relatively new and last
We first discuss the challenges of feeder ranking and for a long time between failures (for example, more than
specifics of the feeder failure ranking process developed for five years), others can have failures within a few tens of
Con Edison (also called “Outage Derived Data Sets—ODDS”) days of each other. In addition, rare seasonal effects (such as
Fig. 7. Example illustrating the training and test time windows in

ODDS. The current time is 8/13/2008, and failure data for training
was derived from the prediction period of 7/30/2007—8/27/2007 and
7/30/2008—8/13/2008.
particularly high-summer temperatures) can affect failure

rates of feeders.
We have focused on the most serious failure type for
distribution feeders where the entire feeder is automatically
taken offline by emergency substation relays due to some
type of fault being detected by sensors. Our current system
for generating data sets attempts to address the challenge of
learning with rare positive examples (feeder failures). An
actual feeder failure incident is instantaneous, so a snapshot
of the system at that moment will have only one failure
example. To better balance the number of positive and
negative examples in the data, we tried the rare event Fig. 8. Process diagram for feeder ranking, using ODDS.
prediction setup shown in Fig. 6, labeling any example that
had experienced a failure over some time window as address this challenge, ODDS creates a new model every
positive. However, the dynamic features for these examples 4 hours on the current data set. (See also [20], [21], [22].)
are constructed from the time frame before the prediction An outline of the overall process is shown in Fig. 8. A
period, and thus do not represent the precise conditions at business management application called the Contingency
the time of failure. This was problematic, as the domain Analysis Program (CAP), discussed in Section 7, uses the
experts believed that some of the dynamic data might only machine learning results to highlight areas of risk through
have predictive value in the period right before the failure. graphical displays and map overlays.
To solve this problem, we decided to switch to “time- As in many real-life applications, our application suffers
shifted” positive examples, where the positive examples are from the problem of missing data. Techniques such as mean
still created from the past outages within the prediction imputation are used to fill in missing values.
period, but the dynamic features are derived only from the 5.2 Cables, Joints, Terminators, and Transformers
time period shortly before the failure happened. This allows Ranking in NYC
our model to capture short-term precursors to failures. The The main challenges to constructing rankings of feeder
features of nonfailures (negative examples) are character- components overlap somewhat with those faced in con-
istics of the current snapshot of all feeders in the system. Not structing rankings for feeders: the use of historical data and
only does this approach, which we call “ODDS” for Outage the data imbalance problem.
Derived Data Sets, capture the dynamic data from right Ideally, we should be able to construct a consistent and
before the failure, but it also helps to reduce the imbalance complete set of features for each component and also its
between positive and negative examples. Fig. 7 shows an connectivity, environmental, and operational contexts at the
example of the periods used to train and test the model. time of failure. At Con Edison, the cable data used for cable,
Another challenge raised by our feeder failure ranking joint, and terminator rankings resides in the “Vision
application is pervasive “concept drift,” meaning that Mapping” system and are designed to only represent the
patterns of failure change fairly rapidly over time, so that a current layout of cables in the system, and not to provide
machine learning model generated on data from the past the layout at particular times in the past. We began to
may not be completely representative of future failure archive cable data starting in 2005 and also relied on other
patterns. Features can become inactive or change in quality. snapshots of cable data that Con Edison made, for example,
Causes of this include: repairs being made on components, cable data captured for Con Edison’s “Network Reliability
causing the nature of future failures to change; new Indicator” program that allowed us to go back as far as 2002
equipment having different failure properties than current configurations.
equipment; and seasonal variation in failure modes (e.g., a Generating training data for joints is especially challen-
greater likelihood of feeder failure in the summer). To ging. Joints are the weakest link in feeders with certain
heat-sensitive joint types having accelerated failure rates

during heat waves. Con Edison keeps a database of feeder
component failures called CAJAC. It captures failure data
of joints in detail. Con Edison autopsies failed components
and the failure reasons they discover are captured in this
database. Though the joint failure data are recorded in
detail, it is challenging to construct a complete list of the set
of installed joints within the grid; the set of installed joints
is imputed from the features of the cables being connected.
In addition, short lengths of cable, called “inserts,” that are
sometimes used to make connections in manholes, are not
yet captured in the Vision Mapping system, so the number
of joints in any manhole can only be estimated in general.
Also, the nature of the joint (type of joint, manufacturer,
etc.) has had to be inferred from the date of installation. We
do this by assuming that the policy in force at the
installation date was used for that joint, which allows us
to infer the manufacturers and techniques used.
To create the transformer database, several data sources
were merged using inferential joins, including data from
Con Edison’s accounting department, the inspection record
database, and the dissolved gas database. Transformer
ranking has several challenges. We are working with a
transformer population that is actively monitored and
aggressively replaced by Con Edison at any sign of
impending trouble, meaning that vulnerable transformers
that had not failed have been replaced, leading to right
censoring (meaning missing information after a certain time Fig. 9. Process diagram for manhole event ranking.
in the life of the transformer). Further, for a transformer that
was replaced, it is always a challenge to determine whether table of manhole locations and types, and a snapshot of
a failure would have occurred if the transformer had not recent cable data from Con Edison’s accounting department
been replaced, causing label bias for the machine learning. (type of cable, manholes at either end of cable, installation
As demonstrated for several of the projects discussed dates); five years of inspection reports filled out by
here, components that have multiple roles or that act as inspectors; and most importantly, event data. The event
interfaces between multiple types of components present data came from several different sources including:
the challenge of bringing together multiple databases to Emergency Control Systems (ECS) trouble tickets which
capture the full context for the component. In order to rank included both structured fields and unstructured text, a
hammerheads, we built a database that joined splice ticket table of structured data containing additional details about
data, cable data, and transformer data, where transformer manhole events (called ELIN—ELectrical INcidents), and a
data itself came from an earlier join of large databases table regarding electrical shock and energized equipment
described above. events (called ESR_ENE). These data were the input for the
While working with various data sets involving date-time manhole event prediction process outlined in Fig. 9.
information, we had to be careful about the meaning of the The trouble tickets are unstructured text documents, so
date and time. In some cases, the date entry represents a date a representation of the ticket had to be defined for the
when work was done or an event occurred; in other cases, learning problem. This representation encodes information
the date is when data were entered into the database. In about the time, location, and nature (degree of seriousness)
some instances, there was confusion as to whether time was of the event. The timestamps on the ticket are directly used,
provided in GMT, EST, or EDT, leading to some cases where but the location and seriousness must be inferred (and/or
our machine learning systems made perfect predictions but learned). The locations of events were inferred using
for the wrong reasons: They learned to detect inevitable several sources of location information present in the
outcomes of failures, but where these outcomes apparently trouble tickets, including a street address (possibly mis-
predated the outages because of data timing skew. spelled or abbreviated, e.g., 325 GREENWHICH ST),
structure names typed within the text of the ticket (S/B
5.3 Manhole Ranking in NYC 153267), and structure names sometimes included in the
One major challenge for manhole event prediction was to structured fields of three tables (ECS, ELIN, or ESR_ENE).
determine which of many data sources, and which fields All location information was typed by hand, and these data
within these sources, to trust; it only made sense to put a lot are very noisy—for instance, the term “service box” was
of effort into cleaning data that had a higher chance of written in at least 38 different ways—and no one source of
assisting with prediction. The data used for the manhole information is complete. The redundancy in the data was
event prediction process is described in detail in [23] and used in order to obtain reliable location information:
includes: information about the infrastructure, namely, a Structure numbers were extracted from the ticket text
Fig. 11. Predictions from a support vector censored regression algorithm

on PILC sections of 33 feeders in Queens.
originally trying to predict manhole events on a short

Fig. 10. Ticket processing. timescale (on the order of 60 days) based on the domain
experts’ intuition that such a timescale would yield useful
using information extraction techniques (see Fig. 10), then predictions. However, it became clear that manhole events
tickets were geocoded to determine the approximate could not be easily predicted over such a short time; for
location of the event. If the geocoded address was not instance, if it is known that a manhole event will occur
within a short distance (200 m) of the structure named within 60 days after a prior event, it is almost impossible to
within the ticket, the information was discarded. The predict when within those 60 days it will happen. In fact,
remaining (twice verified) matches were used so that insulation breakdown, which causes manhole events, can be
the ticket was identified correctly with the manholes that a slow process, taking place over months or years. A
were involved in the event. prediction period of one year was chosen for the machine
It was also necessary to determine the seriousness of learning ranking task, as illustrated in Fig. 6.
events; however, ECS trouble tickets were not designed to The cable data, which are a snapshot at one (recent) point
contain a description of the event itself, and there is no in time, was unified with the other data to construct “static”
structured field to encode the seriousness directly. On the features and labels for the ranking task. This implicitly
other hand, the tickets do have a “trouble type” field, which assumes that the snapshot approximately represents the
is designed to encode the nature of the event (e.g., an number and type of cables over the time period of
underground AC event is “UAC,” flickering lights is “FLT”). prediction; this assumption is necessary since the exact
state of cables in the manhole at a given time in the past may
Originally, we used the trouble type to characterize the
not be available. However, this assumption is not uni-
seriousness of the event: the codes manhole explosion
versally true; for instance, it is not true for neutral
(“MHX”), manhole fire (“MHF”), and smoking manhole
(noncurrent carrying, ground) cables at Con Edison, and
(“SMH”) were used to identify serious events. However, we
neutral cable data thus cannot be used for failure prediction,
later performed a study [24] that showed that the trouble
as discussed in [23]. Often, manholes that have had serious
type did not agree with experts’ labeling of tickets, and is not
events also have had cables replaced and more neutrals put
a good measure of seriousness. In order to better estimate
in; a higher percentage of neutral cables indicates an event
the seriousness of events, we created a representation of
in the past, not necessarily an event in the future.
each ticket based on information extracted from the ticket
The P-Norm Push (see Section 4) was used as the main
text, including the length of the ticket, the presence of
ranking algorithm for manhole ranking.
serious metadata (for instance, the term “SMOKING
LIGHTLY”), and whether cable sizes appeared in the text 5.4 MTBF Modeling in NYC
(indicating the replacement of a cable). This information It became apparent that to really make our feeder prediction
extraction was performed semi-automatically using text- models valuable for proactive maintenance, we had to also
processing tools, including the Generalized Architecture for produce estimates that allow for an absolute measure of
Text Engineering “GATE” [25]. vulnerability, rather than a relative (ranking) measure;
The ticket representation was used to classify the tickets many asset replacement decisions are made by assessing
into the categories: serious events, possible precursor how much reliability in days is gained if a particular choice
events, and nonevents. This classification was performed is made (for instance, to replace a PILC section versus
with either a manual, rule-based method, or general another replacement at the same cost).
machine learning clustering methods (k-means clustering). Machine learning techniques can be used to estimate
So there are two machine learning steps in the manhole MTBF. Fig. 11 shows the application of one of these
event ranking process: a ticket classification step and a techniques [26] to predicting survival times of PILC
manhole ranking step. sections in Queens. This technique can accommodate
One challenge faced early on was in choosing the time censored data through inequality constraints in SVM
frames for the rare event prediction framework. We started regression. Each row of the table represents one feeder,
Fig. 13. ROC-like curve for blind test of Crown Heights feeders.
We track the performance of our machine learning

models by checking the rank of the failed feeder and the
Fig. 12. ROC-like curves from tests of the machine learning ranking of
specific components. ranks of its components whenever a failure happens. We
also compile ROC-like curves showing the components that
and each column indicates a time interval (in years). The failed and the feeder that automatically opened its circuit
color in a particular bin gives the count of cable sections breaker when the failure occurred. These blind tests
within the feeder that are predicted to survive that time provide validation that the algorithms are working suffi-
interval. That is, each row is a histogram of the predicted ciently to assist with operations decisions for Con Edison’s
MTBF for the feeder’s cable sections. The histogram for one maintenance programs.
feeder (one row) is not necessarily smooth in time. This is Fig. 13 shows the results of a blind test for predicting
because the different cable sections within the feeder were feeder failures in Crown Heights, Brooklyn, with prediction
installed at different times (installation not being a smooth period from May 2008 to January 2009. Fig. 12 shows results
function of time), and these installation dates influence the of various tests on the individual components. At each
predicted survival interval. point ðx; yÞ on the plot, x gives a position on the ranked list
and y is the percent of failures that are ranked at or above x
in the list.
6 EVALUATION IN NYC We use rank statistics for each network to continually
We describe the results of our specific processes as applied measure performance of the ODDS system. For instance,
to the NYC power grid through the Columbia/Con Edison AUC is reported in Figs. 12 and 13. The machine learning
collaboration. We have generated machine learning models system has improved to the point where 60 percent of
for ranking the reliability of all 1;000þ high voltage failures occur in the 15 percent of feeders that are ranked as
(13-27 KV) feeders that form the backbone of the NYC’s most susceptible to failure. As importantly, fewer than
power distribution system; for each of the 150;000 cable 1 percent of failures occur on feeders in the best 25 percent
sections and 150;000 joints that connect them; for the of ODDS feeder susceptibility rankings (Fig. 14).
50;000 transformers and 50;000 terminators that join the
transformers to the feeders; and for 50;000 secondary
structures (manholes and service boxes) through which low
voltage (120-240 V) power from the transformers is
distributed to buildings in NYC.
6.1 Feeder and Component Ranking Evaluation

Our machine learning system for computing feeder
susceptibility based on the ODDS system has been online
since July 2008, generating a new model every 4 hours.
ODDS is driven by the feeds from three dynamic real-time
systems: load pocket weight,5 power quality, and outage
history. We found that separate training in Brooklyn and
Queens, with their 27 KV networks, and Manhattan and
Bronx, with their 13 KV networks, produced better results.
Fig. 14. Percent of feeder outages in which the feeder that failed was
5. Load Pocket Weight (LPW) is a expert-derived measure of trouble in within the worst 15 percent (left) of the ranked list or best 25 percent
delivering power to the secondary network in localized areas. It is a (right), where the predictions being evaluated are those just before the
weighted score of the number of open (not in service) network protector time of failure. The system improved from less than 30 percent of the
switches, open secondary mains, open fuses, and nonreporting transfor- failures in the worst 15 percent in 2005 to greater than 60 percent in
mers, and other signs of service outage. 2008, for example.
Fig. 16. Linear regression used to determine the mean time between
failures for 1 January 2002 (purple), and 31 December 2008 (yellow) in
each underground network in the Con Edison system. Networks are
arranged along the horizontal axis from worst (left) to best (right),
according to Con Edison’s “Network Reliability Index.”
Fig. 15. Influence of different categories of features under different 6.2 MTBF Modeling Evaluation
weather conditions. Red: hot weather of August 2010; Blue: snowy We have tracked the improvement in MTBF for each
January 2011; Yellow: rainy February 2011; Turquoise: typical fall
network as preventive maintenance work has been done by
weather in October 2010.
Con Edison to improve performance since 2002. To test
whether this improvement is significant, we use a non-
To determine what the most important features are, we
parametric statistical test, called the logrank test, that
create “tornado” diagrams like Fig. 15. This figure
compares the survival distributions of two samples. In this
illustrates the influence of different categories of features
case, we wished to determine if the 2009 summer MTBF
under different weather conditions. For each weather
values are statistically larger than the 2002 summer MTBF
condition, the influence of each category (the sum of
values. The performance of the system showed significant
coefficients j Pfor that category divided by the total sum
improvement, in that there is a less than one in a billion
of coefficients j j ) is displayed as a horizontal bar. Only
chance that the treatment population in 2009 did not
the top few categories are shown. For both snowy and hot
improve over the control population from 2002. In 2009,
weather, features describing power quality events have
for example, there were 1,468 out of 4,590 network-days that
been the most influential predictors of failure according to
were failure free, or one out of every three summer days,
our model.
but in the 2002 control group, there were only 908 network-
The categories of features in Fig. 15 are: power quality,
days that were failure free, or one out of five summer days,
which are features that count power quality events
that were failure free. The larger the percentage of network-
(disturbances) preceding the outage over various time
days that were failure free, the lower the likelihood of
windows; system load in megawatts; outage history, which
multiple outages happening at the same time.
includes features that count and characterize the prior
Fig. 16 shows MTBF predicted by our model for each
outage history (failure outages, scheduled outages, test
underground network in the Con Edison system on both
failures, immediate failures after reenergization, and urgent
1 January 2002 (purple) and 31 December 2008 (yellow). The
planned outage); load pocket weight, which measures the
yellow bars are generally larger than the purple bars,
difficulty in delivering power to the end user; transformers,
indicating an increase in MTBF.
particularly features encoding the types and ages of
We have performed various studies to predict MTBF of
transformers (e.g., percent of transformers made by a
feeders. Fig. 17 shows the accuracy of our outage rate
particular manufacturer); stop joints and paper joints, which
predictions for all classes of unplanned outages over a
include features that count joints types, configurations, and
age, where these features are associated with joining PILC
to other PILC and more modern cable; cable rank, which
encodes the results of the cable section ranking model; the
count of a specific type of cable (XP and EPR) in various age
categories; HiPot index features, which are derived by Con
Edison to estimate how vulnerable the feeders are to heat
sensitive component failures; number of shunts on the
feeder, where these shunts equalize the capacitance and also
condition the feeder to power quality events; an indicator
for nonnetwork customers, where a nonnetwork customer
is a customer that gets electricity from a radial overhead
connection to the grid; count of PILC sections along the
feeder; percent of joints that are solid joints, which takes into
account the fact that joining modern cable is simpler and
less failure-prone than joining PILC; shifted load features
that characterize how well a feeder transfers load to other Fig. 17. Scatter plot of SVM predicted outage rate versus actual rate for all
feeders if it were to go out of service. classes of unplanned outages. The diagonal line depicts a perfect model.
Fig. 18. ROC-like curve for 2009 Bronx blind test of the machine Fig. 19. Screen capture of the Contingency Analysis Program tool during
learning ranking for vulnerability of manholes to serious events (fires a fourth contingency event in the summer of 2008, with the feeders at
and explosions). most risk of failing next highlighted in red. The feeder ranking at the time
of failure is shown in a blow-up ROC-like plot in the center.
three-year period using a support vector machine regres-

7.1 Contingency Analysis Program
sion model that predicts feeder MTBF. While the results
CAP is a tool designed by Con Edison and used at their
are quite strong, there are two sources of inaccuracy in
main control centers. It brings together information relevant
this study. First, the study did not model “infant
to the outage of a primary feeder cable. When a contingency
mortality,” the increased likelihood of failure after a
occurs, Con Edison already has applications in use
repaired system is returned to service. This led to an
(integrated into the CAP tool) that preemptively model
underestimation of failures for the more at-risk feeders the network for the possibility of additional feeders failing.
(visible particularly in the upper right of the graph). These applications determine the failures that could have
Empirically, we observed an increased likelihood of infant the worst consequences for the system. Columbia’s key
mortality for about six weeks following an outage. Second, contribution to the CAP tool is a feeder susceptibility
the study has difficulties handling censored data. If events indicator (described in Section 5.1) that gives the operators a
are very infrequent, it is not possible for the algorithm to new important piece of information: an indicator of which
accurately predict their frequency. This right-censoring feeders are most likely to fail next. Operators can use this
effect for the low outage rate feeders, due to lack of information to help determine the allocation of effort and
failures in the three-year observation window, is visible in resources toward preventing a cascade. The “worst con-
the lower left of the plot. sequences” feeder may not be the same as the “most likely
to fail” feeder, so the operator can choose to allocate
6.3 Manhole Ranking Evaluation resources to feeders that are both likely to fail and for which
The most recent evaluation of the manhole rankings was a a failure could lead to more serious consequences. Fig. 19
blind test for predicting 2009 events in the Bronx. The shows a snapshot of the CAP tool interface.
Columbia database has data through 2007, incomplete 2008
data, and no data from 2009 or after. There are 27,212 7.2 CAPT—Capital Asset Prioritization Tool
manholes in the Bronx. The blind test showed: CAPT is a prototype application designed by Columbia
and Con Edison that offers an advanced mechanism for
. the most at-risk 10 percent (2,721/27,212) of the helping engineers and managers plan upgrades to the
ranked list contained 44 percent (8/18) of the man- feeder systems of NYC. Using a graphic interface, shown in
holes that experienced a serious event, Fig. 20, users first enter constraints on work they would
. the most at-risk 20 percent (5,442/27,212) of the hypothetically like to do. For instance, users can specify a
ranked list contained 55 percent (10/18) of the borough or network, one or more specific feeder sections or
trouble holes for serious events. type of feeder section, dollar amount to be allocated, etc.
Fig. 18 contains the ROC-like curve for the full ranked list. CAPT then produces benefit versus cost curves of various
Before the start of the project, it was not clear whether replacement strategies with the objective of optimizing
manhole events could be predicted at all from the “bang for the buck”—the greatest increase in system MTBF
secondary data. These results show that indeed manhole for the dollars spent. Such a tool, if proven robust in
events are worthwhile to model for prediction. production tests, could become a valuable contributor to
capital asset allocations in the future. Typical maintenance
plans might attempt to target replacement of at-risk
7 MANAGEMENT SOFTWARE sections, joints, or secondary components. The key compo-
Prototype interfaces were developed jointly with Con nents of CAPT include 1) the model (currently an ODDS
Edison in order to make the results useful and to assist in model along with a regression between SVM scores and
knowledge discovery. observed MTBF) used to estimate MTBF for feeders both
Fig. 20. A screen capture of the Con Edison CAPT evaluation, showing
an improvement in MTBF from 140 to 192 days if 34 of the most at-risk
PILC sections were to be replaced on a feeder in Brooklyn at an
estimated cost of $650,000.
before and after any hypothetical changes; 2) the ranked

lists for cable sections and joints, based on component
rankings, allowing CAPT to recommend good candidates Fig. 22. Images from the manhole events visualization tool, where labels
for replacement; and 3) a system that displays, in chart were enlarged for clarity. Top: Geocoded ticket addresses, colored by
form for the user, tradeoff (Pareto) curves of benefit versus trouble type. Yellow indicates a serious event type, purple indicates a
potential precursor. If the user clicks on a ticket, the full ticket text is
cost for various replacement strategies (Fig. 21). displayed. Bottom: Manholes and main cables within the same location,
where manholes are colored by predicted vulnerability. Note that a ticket
7.3 Manhole Event Structure Profiling Tool and within the top figure does not necessarily correspond to the nearest
Visualization Tool manhole on the bottom figure.
We developed several tools that allow a qualitative
evaluation of results and methods by secondary system vulnerability of a manhole to be roughly estimated at a
engineers. The most useful of these tools is the “structure glance by domain experts. We also developed a visualiza-
profiling tool,” (also called the “report card” tool at Con tion tool (discussed in [28]) that uses Google Earth6 as a
backdrop to display the locations of events, manholes, and
Edison) that produces a full report of raw and processed
cables. Fig. 22 displays two screen shots from the
data concerning any given individual manhole [27]. Before
visualization tool.
this tool was implemented, an individual case study of a
manhole took days and resulted in an incomplete study.
This tool gives the reasons why a particular manhole was 8 RELATED WORK
assigned a particular rank by the model, and allows the Machine learning has been used for applications in power
engineering since the early days of artificial intelligence, with a
surge of recent interest in the last decade. Venues for these
works include the 1999 ACAI workshop on Machine
Learning Applications to Power Systems (summarized by
Hatziargyriou [29]), the proceedings of the yearly Interna-
tional Conference on Intelligent System Applications to
Power Systems,7 and the 2009 Special Session on Machine
Learning in Energy Applications at the International Con-
ference on Machine Learning and Applications (ICMLA ’09).
There are also several books summarizing work on machine
learning in power engineering (e.g., [30], [31]). Applications
include the prediction of power security breaches, forecast-
ing, power system operation and control, and classification
of power system disturbances. The power engineering work
bears little similarity to the current work for two reasons.
First, much of the power engineering work focuses on
specific machine learning techniques, yet for our applica-
tion, the specific machine learning techniques are not the
Fig. 21. Example of cost benefit analysis of possible replacement primary reason for success, as discussed earlier. In our
strategies for specific at-risk components analyzed by the machine
learning system. The solid line approximates the “efficient frontier” in 6. earth.google.com.
portfolio management theory. 7. http://www.isap-power.org/.
applications, the predictive accuracy gained by using a event project, many Con Edison engineers did not view
different technique is often small compared to the accuracy manhole event prediction as a realistic goal. The Con Edison
gained through other steps in the discovery process or by trouble ticket data could easily have become what Fayyad
formulating the problem differently. The data in power et al. [8] consider a “data tomb.” In this case, the remedy
engineering problems are generally assumed to be amen- created by Columbia and Con Edison involved a careful
able to learning in their raw form, in contrast with our problem formulation, the use of sophisticated text processing
treatment of the data. The second reason our work is tools, and state-of-the-art machine learning techniques.
distinct from the power engineering literature is that the
machine learning techniques that have been developed by 9.2 Data Are the Key
the power engineering community are often “black-box” Power companies already collect a great deal of data;
methods such as neural networks and genetic algorithms however, if these data are going to be used for prediction of
(e.g., [32], [33]). Neural networks and genetic algorithms failures, they should ideally have certain properties: First,
can be viewed as heuristic, nonconvex optimization the data should be as clean as possible, meaning, for
procedures for objectives that have multiple local minima; instance, that unique identifiers should be used for each
the algorithms’ output can be extremely sensitive to the component. Second, if a component is replaced, it is
initial conditions. Our work uses mainly convex optimiza- important to record the properties of the old component
tion procedures to avoid this problem. Further, “black-box” (and its surrounding context if it is used to derive features)
algorithms do not generally produce interpretable/mean- before the replacement; otherwise, it cannot be determined
ingful solutions (for instance, the input-output relationship what properties are common to those being replaced.
of a multilayer neural network is not generally interpre- For trouble tickets, unstructured text fields should not be
table), whereas we use mainly simple linear combinations eliminated. It is true that structured data are easier to
of features. analyze; on the other hand, free-text can be much more
We are not aware of any other work that addresses the reliable. This was also discussed by Dalal et al. [38] in
challenges in mining historical power grid data of the same dealing with trouble tickets from web transaction data; in
level of complexity as those discussed here. Our work their case, a 40 character free-text field contained more
contrasts with a subset of work in power engineering where information than any other field in the database. In the case
data come entirely from Monte Carlo (MC) simulations [34], of Con Edison trouble tickets, our representation based on
[35], and the MC simulated failures are predicted using the free-text can much more reliably determine the
machine learning algorithms. In a sense, our work is closer seriousness of events than the (structured) trouble type
to data mining challenges in other fields such as code. Further, the type of information that is generally
e-commerce [10], criminal investigation [36], or medical recorded in trouble tickets cannot easily fit into a limited
patient processing [9] that encompass the full discovery number of categories and asking operators to choose the
process. For instance, it is interesting to contrast our work category under time pressure is not practical. We have
on manhole events with the study of Cornélusse et al. [37], demonstrated that analysis of unstructured text is possible
who used domain experts to label “frequency incidents” at and even practical.
generators, and constructed a machine learning model from
the frequency signals and labels. The manhole event 9.3 Machine Learning Ranking Methods Are Useful
prediction task discussed here also used domain experts for Prioritization
to label trouble tickets as to whether they represent serious Machine learning methods for ranking are relatively new,
events; however, the level of processing required to clean and currently they are not used in many application
and represent the tickets, along with the geocoding and domains besides information retrieval. So far, we have
information extraction required to pinpoint event locations, found that in the domain of electrical grid maintenance, the
coupled with the integration of the ticket labeling machine key to success is in the interpretation and processing of
learning task with the machine learning ranking task makes data, rather than in the exact machine learning method
the latter task a much more substantial undertaking. used; however, these new ranking methods are designed
exactly for prioritization problems, and it is possible that
these methods can offer an edge over older methods in
9 LESSONS LEARNED
many applications. Furthermore, as data collection becomes
There are several “take-away” messages from the develop- more automated, it is possible that the dependence on
ment of our knowledge discovery processes on the NYC grid. processing will lessen, and there will be a substantial
advantage in using algorithms designed precisely for the
9.1 Prediction Is Possible
task of prioritization.
We have shown successes in predicting failures of electrical
components based on data collected by a major power utility 9.4 Reactive Maintenance Can Lead to
company. It was not clear at the outset that knowledge Overtreatment
discovery and data mining approaches would be able to We have demonstrated with a statistical method called
predict electrical component failures, let alone assist domain “propensity” [39] that the High Potential (HiPot) testing
engineers with proactive maintenance programs. We are program at Con Edison was overtreating the “patient,” i.e.,
now involved in a Smart Grid Demonstration Project to the feeders. HiPot is, by definition, preventive maintenance
verify that these techniques can be scaled to robust system in that incipient faults are driven to failure by intentionally
use. For example, prior to our successes on the manhole stressing the feeder. We found, however, that the direct
REFERENCES
[1] “Office of Electric Transmission United States Department of
Energy and Distribution,” “Grid 2030” a Nat’l Vision for Electricity’s
Second 100 Years, July 2003.
[2] “North American Electric Reliability Corporation (NERC),”
Results of the 2007 Survey of Reliability Issues, Revision 1, Oct. 2007.
[3] S.M. Amin, “U.S. Electrical Grid Gets Less Reliable,” IEEE
Spectrum, Jan. 2011.
[4] M. Chupka, R. Earle, P. Fox-Penner, and R. Hledik, “Transforming
America’s Power Industry: The Investment Challenge 2010-2030,”
technical report, The Brattle Group, prepared for The Edison
Foundation, Washington, D.C., 2008.
[5] W.J. Frawley, G. Piatetsky-Shapiro, and C.J. Matheus, “Knowl-
edge Discovery in Databases: An Overview,” AI Magazine, vol. 13,
no. 3, pp. 57-70, 1992.
[6] J.A. Harding, M. Shahbaz, S. Srinivas, and A. Kusiak, “Data
Mining in Manufacturing: A Review,” J. Manufacturing Science and
Eng., vol. 128, no. 4, pp. 969-976, 2006.
[7] A. Azevedo and M.F. Santos, “KDD, SEMMA and CRISP-DM: A
Parallel Overview,” Proc. Int’l Assoc. Development of the Information
Soc. European Conf. Data Mining, pp. 182-185, 2008.
Fig. 23. Overtreatment in the High Potential Preventive Maintenance [8] U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth, “From Data
program was identified by comparing to control group performance. Mining to Knowledge Discovery in Databases,” AI Magazine,
Modified and A/C Hipot tests are now used by Con Edison instead of DC vol. 17, pp. 37-54, 1996.
Hipot tests. [9] W. Hsu, M.L. Lee, B. Liu, and T.W. Ling, “Exploration Mining in
Diabetic Patients Databases: Findings and Conclusions,” Proc.
Sixth ACM SIGKDD Int’l Conf. Knowledge Discovery and Data
current (DC) HiPot testing, in particular, was not out- Mining, pp. 430-436, 2000.
performing a “placebo” control group which was scored by [10] R. Kohavi, L. Mason, R. Parekh, and Z. Zheng, “Lessons and
Con Edison to be equally “sick” but on which no work was Challenges from Mining Retail E-Commerce Data,” Machine
Learning, special issue on data mining lessons learned, vol. 57,
done (Fig. 23). When a new alternating current (AC) test pp. 83-113, 2004.
was added by Con Edison to avoid some of the over- [11] A.P. Bradley, “The Use of the Area under the ROC Curve in the
treatment, we were able to demonstrate that as the test was Evaluation of Machine Learning Algorithms,” Pattern Recognition,
being perfected on the system, the performance level vol. 30, no. 7, pp. 1145-1159, July 1997.
[12] C. Rudin, “The P-Norm Push: A Simple Convex Ranking
increased and has now surpassed that of the control group. Algorithm that Concentrates at the Top of the List,” J. Machine
Indeed, operations and distribution engineering at Con Learning Research, vol. 10, pp. 2233-2271, Oct. 2009.
Edison has since added a modified AC test that improved [13] C. Rudin and R.E. Schapire, “Margin-Based Ranking and an
on the performance of the control group also. This Equivalence between AdaBoost and RankBoost,” J. Machine
Learning Research, vol. 10, pp. 2193-2232, Oct. 2009.
interaction among machine learning, statistics, preventive [14] Y. Freund, R. Iyer, R.E. Schapire, and Y. Singer, “An Efficient
maintenance programs, and domain experts will likely Boosting Algorithm for Combining Preferences,” J. Machine
identify overtreatment in most utilities that are predomi- Learning Research, vol. 4, pp. 933-969, 2003.
nantly reactive to failures now. That has been the [15] T. Joachims, “A Support Vector Method for Multivariate Perfor-
mance Measures,” Proc. Int’l Conf. Machine Learning, 2005.
experience in other industries, including those for which
[16] H. Drucker, C.J.C. Burges, L. Kaufman, A. Smola, and V. Vapnik,
these techniques have been developed, such as the “Support Vector Regression Machines,” Proc. Advances in Neural
automotive and aerospace industries, the military, and Information Processing Systems, vol. 9, pp. 155-161, 1996.
healthcare. [17] L. Breiman, J. Friedman, C.J. Stone, and R.A. Olshen, CART:
Classification and Regression Trees. Wadsworth Press, 1983.
[18] L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1,
10 CONCLUSIONS pp. 5-32, Oct. 2001.
[19] D.R. Cox, “Regression Models and Life-Tables,” J. the Royal
Over the next several decades we will depend more on an Statistical Soc., Series B (Methodological), vol. 34, no. 2, pp. 187-220,
aging and overtaxed electrical infrastructure. The reliability 1972.
[20] P. Gross, A. Salleb-Aouissi, H. Dutta, and A. Boulanger, “Ranking
of the future smart grid will depend heavily on the new Electrical Feeders of the New York Power Grid,” Proc. Int’l Conf.
preemptive maintenance policies that are currently being Machine Learning and Applications, pp. 725-730, 2009.
implemented around the world. Our work provides a [21] P. Gross, A. Boulanger, M. Arias, D.L. Waltz, P.M. Long, C.
Lawson, R. Anderson, M. Koenig, M. Mastrocinque, W. Fairechio,
fundamental means for constructing intelligent automated J.A. Johnson, S. Lee, F. Doherty, and A. Kressner, “Predicting
policies: machine learning and knowledge discovery for Electricity Distribution Feeder Failures Using Machine Learning
prediction of vulnerable components. Our main scientific Susceptibility Analysis,” Proc. 18th Conf. Innovative Applications of
contribution is a general process that can be used by power Artificial Intelligence, 2006.
[22] H. Becker and M. Arias, “Real-Time Ranking with Concept Drift
utilities for failure prediction and preemptive maintenance. Using Expert Advice,” Proc. 13th ACM SIGKDD Int’l Conf.
We showed specialized versions of this process to feeder Knowledge Discovery and Data Mining, pp. 86-94, 2007.
ranking, feeder component ranking (cables, joints, ham- [23] C. Rudin, R. Passonneau, A. Radeva, H. Dutta, S. Ierome, and D.
Isaac, “A Process for Predicting Manhole Events in Manhattan,”
merheads, and transformers), MTBF/MTTF estimation, Machine Learning, vol. 80, pp. 1-31, 2010.
and manhole vulnerability ranking. We have demon- [24] R. Passonneau, C. Rudin, A. Radeva, and Z.A. Liu, “Reducing
strated, through direct application to the New York City Noise in Labels and Features for a Real World Dataset:
Application of NLP Corpus Annotation Methods,” Proc. 10th Int’l
power grid, that data already collected by power compa-
Conf. Computational Linguistics and Intelligent Text Processing, 2009.
nies can be harnessed to predict, and to thus assist in
preventing, grid failures.
[25] H. Cunningham, D. Maynard, K. Bontcheva, and V. Tablan, Albert Boulanger received the BS degree in physics from the
“GATE: A Framework and Graphical Development Environment University of Florida, Gainesville, in 1979 and the MS degree in
for Robust NLP Tools and Applications,” Proc. 40th Anniversary computer science from the University of Illinois, Urbana-Champaign, in
Meeting Assoc. for Computational Linguistics, July 2002. 1984. He is a senior staff associate at Columbia University’s Center for
[26] P. Shivaswamy, W. Chu, and M. Jansche, “A Support Vector Computational Learning Systems. He is a member of the IEEE.
Approach to Censored Targets,” Proc. Int’l Conf. Data Mining,
2007. Ansaf Salleb-Aouissi received the MS and PhD degrees from the
[27] A. Radeva, C. Rudin, R. Passonneau, and D. Isaac, “Report Cards University of Orleans, France,) and an engineer degree in computer
for Manholes: Eliciting Expert Feedback for a Machine Learning science from the University of Science and Technology Houari
Task,” Proc. Int’l Conf. Machine Learning and Applications, 2009. Boumediene (USTHB), Algeria. She joined Columbia University’s
[28] H. Dutta, C. Rudin, R. Passonneau, F. Seibel, N. Bhardwaj, A. Center for Computational Learning Systems as an associate research
Radeva, Z.A. Liu, S. Ierome, and D. Isaac, “Visualization of scientist after a postdoctoral fellowship at INRIA Rennes, France.
Manhole and Precursor-Type Events for the Manhattan Electrical
Distribution System,” Proc. Workshop Geo-Visualization of Dy- Maggie Chow received the BE degree from City College of New York
namics, Movement and Change, 11th AGILE Int’l Conf. Geographic and the master’s degree from NYU-Poly. She is a section manager at
Information Science, May 2008. Consolidated Edison of New York. Her responsibilities focus on lean
[29] N.D. Hatziargyriou, “Machine Learning Applications to Power management and system reliability.
Systems,” Machine Learning and Its Applications, pp. 308-317,
Springer-Verlag 2001. Haimonti Dutta received the PhD degree in computer science and
[30] A. Ukil, Intelligent Systems and Signal Processing in Power electrical engineering (CSEE) from the University of Maryland. She is an
Engineering. Springer, 2007. associate research scientist at the Center for Computational Learning
[31] L.A. Wehenkel, Automatic Learning Techniques in Power Systems. Systems, Columbia University.
Springer, 1998.
[32] A. Saramourtsis, J. Damousis, A. Bakirtzis, and P. Dokopoulos,
Philip N. Gross received the BS degree from Columbia University in
“Genetic Algorithm Solution to the Economic Dispatch Problem— 1999 and the MS degree from Columbia University in 2001. He is a
Application to the Electrical Power Grid of Crete Island,” Proc. software engineer at Google.
Workshop Machine Learning Applications to Power Systems (ACAI),
pp. 308-317, 2001.
[33] Y.A. Katsigiannis, A.G. Tsikalakis, P.S. Georgilakis, and N.D. Bert Huang received the BS and BA degrees from Brandeis University
Hatziargyriou, “Improved Wind Power Forecasting Using a and the MS and MPhil degrees from Columbia University. He is working
Combined Neuro-Fuzzy and Artificial Neural Network Model,” toward the PhD degree in the Department of Computer Science,
Columbia University.
Proc. Fourth Helenic Conf. Artificial Intelligence, pp. 105-115, 2006.
[34] P. Geurts and L. Wehenkel, “Early Prediction of Electric Power
System Blackouts by Temporal Machine Learning,” Proc. ICML Steve Ierome received the BS degree in electrical engineering from the
’98/AAAI ’98 Workshop Predicting the Future: AI Approaches to Time City College of New York in 1975. He has 40 years of experience in
Series Analysis, pp. 21-28, 1998. distribution engineering design and planning at Con Edison, and three
[35] L. Wehenkel, M. Glavic, P. Geurts, and D. Ernst, “Automatic years of experience in power quality and testing of overhead radial
Learning for Advanced Sensing Monitoring and Control of equipment.
Electric Power Systems,” Proc. Second Carnegie Mellon Conf. Electric
Power Systems, 2006. Delfina F. Isaac received both the BS degree in applied mathematics
[36] H. Chen, W. Chung, J.J. Xu, G. Wang, Y. Qin, and M. Chau, and statistics in 1998 and the MS degree in statistics in 2000 from the
“Crime Data Mining: A General Framework and Some Examples,” State University of New York at Stony Brook. She is a quality assurance
Computer, vol. 37, no. 4, pp. 50-56, Apr. 2004. manager and was previously a senior statistical analyst in the
[37] B. Cornélusse, C. Wera, and L. Wehenkel, “Automatic Learning Engineering and Planning organization at Con Edison.
for the Classification of Primary Frequency Control Behaviour,”
Proc. IEEE Lausanne Power Tech Conf., 2007. Arthur Kressner is the president of Grid Connections, LLC. He recently
[38] S.R. Dalal, D. Egan, M. Rosenstein, and Y. Ho, “The Promise and retired from Consolidated Edison Company in New York City with more
Challenge of Mining Web Transaction Data,” Statistics in Industry than 40 years experience, most recently as the director of research and
(Handbook of Statistics), R. Khatree and C.R. Rao, eds., vol. 22, development.
Elsevier, 2003.
[39] P.R. Rosenbaum and D.B. Rubin, “The Central Role of the Rebecca J. Passonneau received the doctorate degree from the
Propensity Score in Observational Studies for Causal Effects,” University of Chicago Department of Linguistics. She is a senior
Biometrika, vol. 70, no. 1, pp. 45-55, 1983. research scientist at the Center for Computational Learning Systems,
Columbia University, where she works on knowledge extraction from
Cynthia Rudin received the BS and BA degrees from the University at noisy textual data, spoken dialogue systems, and other applications of
Buffalo and the PhD degree in applied and computational mathematics computational linguistics.
from Princeton University. She is an assistant professor in the
Operations Research and Statistics Group at the MIT Sloan School of Axinia Radeva received the MS degree in electrical engineering from
Management, and an adjunct research scientist at the Center for the Technical University in Sofia, Bulgaria, and a second MS degree in
Computational Learning Systems, Columbia University. computer science from Eastern Michigan University. She is a staff
associate at Columbia University’s Center for Computational Learning
David Waltz received all the degrees from the Massachusetts Institute Systems.
of Technology. He is the director of the Center for Computational
Learning Systems at Columbia University, with prior positions as Leon Wu received the BSc degree in physics from Sun Yat-sen
president of the NEC Research Institute, director of Advanced University and the MS and MPhil degrees in computer science from
Information Systems at Thinking Machines Corp., and faculty positions Columbia University. He is working toward the PhD degree in the
at Brandeis University and the University of Illinois at Urbana- Department of Computer Science and is a senior research associate at
Champaign. He is a fellow and past president of Association for the the Center for Computational Learning Systems, Columbia University.
Advancement of AI (AAAI), and fellow of the ACM. He is a senior He is a member of the IEEE.
member of the IEEE.
Roger N. Anderson received the PhD degree from the Scripps . For more information on this or any other computing topic,
Institution of Oceanography, the University of California at San Diego. please visit our Digital Library at www.computer.org/publications/dlib.
He is a senior scholar at the Center for Computational Learning
Systems, Columbia University. He is a member of the IEEE.

Machine Learning For NY Power Grid

Uploaded by

Copyright:

Available Formats

Machine Learning For NY Power Grid

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Machine Learning For NY Power Grid

Uploaded by

Copyright:

Available Formats

328 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 34, NO.

Machine Learning for the

O NE of the major findings of the US Department of

weaknesses in any of these components: a feeder may go out

2.1 Feeder Rankings

Fig. 3. Excerpt from sample smoking manhole (SMH) trouble ticket.

important: Scheduled replacement of each section costs

Fig. 4. Bathtub curve. Source Wikipedia: http://en.wikipedia.org/wiki/

2.4 MTBF Modeling

A common quality measure in supervised ranking is the

ranking, where “time shifted” features were used. ð3Þ

Fig. 7. Example illustrating the training and test time windows in

particularly high-summer temperatures) can affect failure

heat-sensitive joint types having accelerated failure rates

Fig. 11. Predictions from a support vector censored regression algorithm

originally trying to predict manhole events on a short

We track the performance of our machine learning

6.1 Feeder and Component Ranking Evaluation

three-year period using a support vector machine regres-

before and after any hypothetical changes; 2) the ranked

You might also like