COMMUNICATIONS

ACM
cACM.acm.org OF THE 07/2012 VOL.55 NO.7

Computational
Folkloristics

Behavioral
Programming
Controlling
Queue Delay
Predatory Scholarly
Publishing
Google’s Hybrid
Approach to Research
HTML5 Leading
a Web Revolution

Association for
Computing Machinery
 “ It was great “
having so many different talks that are applicable
to what my team deals with on a daily basis.

“ Getting first-hand accounts of lessons learned in the real world.

“
Lots of great theoretics and how they play out in a production system.
Rubber, meet road.

Baltimore, MD Sept. 27th - 28th

Surge brings together experts from around the world to share their stories
and demonstrate proven concepts in Web architectures and scalable design.
Hear from the most established practitioners in our field, as they describe
operational bottlenecks and how they overcome them with proven technologies,
open standards and rational thinking.

Special Rate for ACM Members! Save $100

http://goo.gl/9PufB
h t tp: // w w w. ac m.or g /dl
communications of the acm

Departments News Viewpoints

5 Editor’s Letter 22 Technology Strategy and Management

Predatory Scholarly Publishing Business Models for
By Moshe Y. Vardi Strategy and Innovation
While often ambiguously
7 Letters To The Editor defined, business models
An Integral Number are central to innovation.
and Its Consequences By Mari Sako

10 BLOG@CACM 25 Legally Speaking

CS and Popular Culture; Can Online Piracy
Learning From Console Games Be Stopped by Laws?
Mark Guzdial writes about why Considering the legal responsibilities
computer science should of Internet intermediaries in
permeate popular culture. the aftermath of the Stop Online
Judy Robertson discusses 21 Privacy Act controversy.
the educational benefits of using By Pamela Samuelson
console games in the classroom. 13 Degrees of Separation
Researchers now have the capability 28 Computing Ethics
27 Calendar to look at the small-world An Information Strategy for
problem from both the traditional Environmental Sustainability
124 Careers algorithmic approach and Seeking solutions to
the new topological approach. a problem of change.
By Gregory Goth By R.T. Watson, J. Corbett,
Last Byte M.C. Boudreau, and J. Webster
16 HTML5 Leads a Web Revolution
128 Future Tense Propelled by a proliferation of 31 Historical Reflections
They Just Click mobile devices and social networks, Alan Turing’s Other Universal Machine
When glasses track glances, will eyes an enhanced family of Web Reflections on the Turing ACE
still meet across a crowded room? specifications is bringing new computer and its influence.
By Ken MacLeod power to developers and By Martin Campbell-Kelly
new capabilities to users.
By Gary Anthes 34 Viewpoint
Google’s Hybrid Approach to Research
18 Patently Inadequate By closely connecting research
The biggest change to U.S. and development Google is able
patent law in nearly 60 years to conduct experiments on an
brings many changes, but fails unprecedented scale, often resulting
to solve the software industry’s in new capabilities for the company.
most vexing problems. By Alfred Spector, Peter Norvig,
By Marina Krakovsky and Slav Petrov

21 Lost and Found 38 Viewpoint

Photogra ph by A P- Photo/ Karsten T hielker

Researchers discover computer The Challenges of Privacy by Design

pioneer Konrad Zuse’s Heralded by regulators, Privacy by
long-forgotten Z9, the world’s Design holds the promise to solve
first program-controlled the digital world’s privacy problems.
binary relay calculator using But there are immense challenges,
floating-point arithmetic. including management commitment
By Paul Hyman and step-by-step methods to
Association for Computing Machinery
Advancing Computing as a Science & Profession integrate privacy into systems.
By Sarah Spiekermann

2 comm unicatio ns o f the ac m | j u ly 201 2 | vo l. 5 5 | no. 7

 07/2012 vol. 55 no. 07

Practice Contributed Articles Review Articles

54 60 90

42 Controlling Queue Delay 60 Computational Folkloristics 90 Behavioral Programming

A modern AQM is just one piece A searchable meta-graph can A novel paradigm for programming
of the solution to bufferbloat. connect even troublesome house reactive systems centered on
By Kathleen Nichols and Van Jacobson elves and other supernatural beings naturally specified modular behavior.
to scholarly folk categories. By David Harel, Assaf Marron,
51 My Compiler Does Not By James Abello, Peter Broadwell, and Gera Weiss
Understand Me and Timothy R. Tangherlini
Until our programming languages
catch up, code will be full of horrors. 71 Large-Scale Complex IT Systems Research Highlights
By Poul-Henning Kamp The reductionism behind
today’s software-engineering 104 Technical Perspective
54 Getting What You Measure methods breaks down in For Better or Worse,
Four common pitfalls in the face of systems complexity. Benchmarks Shape a Field
using software metrics for By Ian Sommerville, Dave Cliff, By David Patterson
project management. Radu Calinescu, Justin Keen,
By Eric Bouwers, Joost Visser, Tim Kelly, Marta Kwiatkowska, 105 Looking Back and Looking Forward:
and Arie van Deursen John McDermid, and Richard Paige Power, Performance, and Upheaval
By Hadi Esmaeilzadeh, Ting Cao,
Articles’ development led by 78 Why On-Chip Cache Xi Yang, Stephen M. Blackburn,
queue.acm.org Coherence Is Here to Stay and Kathryn S. McKinley
On-chip hardware coherence
can scale gracefully as the number
Illustrations by Gary Neill, J.F. Podevin, Lea nd ro Cast el ao

of cores increases. 115 Technical Perspective

By Milo M.K. Martin, Mark D. Hill, Why Study the Price of Anarchy?
and Daniel J. Sorin By Amos Fiat
COMMUNICATIONS About the Cover:
ACM
cAcM.AcM.org OF THE 07/2012 VoL.55 No.7

Computational Do Small IT Firms Benefit from 116 Intrinsic Robustness

Computational
Folkloristics folkloristics open vast Higher Process Capability? of the Price of Anarchy
collections of classic
tales, beloved characters, Evidence suggests small firms can By Tim Roughgarden
and inspiring fantasies
for wondrous exploration
reap rewards from developing a high
Behavioral
Programming
as never before. This level of formal process capability.
month’s cover story
Controlling
Queue Delay
Predatory Scholarly (p. 60) imagines a world
By Matthew Swinarski,
Diane H. Parente, and Rajiv Kishore
Publishing
Google’s Hybrid
Approach to Research
of literary fancy at our
HTML5 Leading
a Web Revolution
fingertips. Illustration by
Association for
Jean-Francois Podevin;
http://www.podevin.com/
Computing Machinery

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n ic at ion s o f the acm 3

 communications of the acm
Trusted insights for computing’s leading professionals.

Communications of the ACM is the leading monthly print and online magazine for the computing and information technology fields.
Communications is recognized as the most trusted and knowledgeable source of industry information for today’s computing professional.
Communications brings its readership in-depth coverage of emerging areas of computer science, new trends in information technology,
and practical applications. Industry leaders use Communications as a platform to present and debate various technology implications,
public policies, engineering challenges, and market trends. The prestige and unmatched reputation that Communications of the ACM
enjoys today is built upon a 50-year commitment to high-quality editorial content and a steadfast dedication to advancing the arts,
sciences, and applications of information technology.

ACM, the world’s largest educational STA F F edi torial Board

and scientific computing society, delivers  
resources that advance computing as a Director of G roup Publis h i ng E ditor -i n-c hief
science and profession. ACM provides the Scott E. Delman Moshe Y. Vardi ACM Copyright Notice
computing field’s premier Digital Library [email protected] [email protected] Copyright © 2012 by Association for
and serves its members and the computing Executive Editor Computing Machinery, Inc. (ACM).
profession with leading-edge publications, News Permission to make digital or hard copies
Diane Crawford Co-Chairs
conferences, and career resources. Managing Editor of part or all of this work for personal
Marc Najork and Prabhakar Raghavan or classroom use is granted without
Thomas E. Lambert Board Members
Executive Director and CEO Senior Editor fee provided that copies are not made
John White Hsiao-Wuen Hon; Mei Kobayashi; or distributed for profit or commercial
Andrew Rosenbloom William Pulleyblank; Rajeev Rastogi;
Deputy Executive Director and COO Senior Editor/News advantage and that copies bear this
Patricia Ryan Jeannette Wing notice and full citation on the first
Jack Rosenberger
Director, Office of Information Systems Web Editor page. Copyright for components of this
Wayne Graves Viewpoi nts work owned by others than ACM must
David Roman
Director, Office of Financial Services Co-Chairs be honored. Abstracting with credit is
Editorial Assistant
Russell Harris Susanne E. Hambrusch; John Leslie King; permitted. To copy otherwise, to republish,
Zarina Strakhan
Director, Office of SIG Services J Strother Moore to post on servers, or to redistribute to
Rights and Permissions
Donna Cappo Board Members lists, requires prior specific permission
Deborah Cotton
Director, Office of Publications P. Anandan; William Aspray; and/or fee. Request permission to publish
Bernard Rous Art Director Stefan Bechtold; Judith Bishop; from [email protected] or fax
Director, Office of Group Publishing Andrij Borys Stuart I. Feldman; Peter Freeman; (212) 869-0481.
Scott E. Delman Associate Art Director Seymour Goodman; Shane Greenstein;
Alicia Kubista Mark Guzdial; Richard Heeks; For other copying of articles that carry a
Assistant Art Directors Rachelle Hollander; code at the bottom of the first or last page
ACM Co u n c i l Richard Ladner; Susan Landau;
Mia Angelica Balaquiot or screen display, copying is permitted
President Carlos Jose Pereira de Lucena;
Brian Greenberg provided that the per-copy fee indicated
Vinton G. Cerf Beng Chin Ooi; Loren Terveen
Production Manager in the code is paid through the Copyright
Vice-President
Lynn D’Addesio Clearance Center; www.copyright.com.
Alexander L. Wolf P ractice
Director of Media Sales
Secretary/Treasurer
Jennifer Ruzicka Chair Subscriptions
Vicki L. Hanson
Public Relations Coordinator Stephen Bourne An annual subscription cost is included
Past President
Virgina Gold Board Members in ACM member dues of $99 ($40 of
Alain Chesnais
Publications Assistant Eric Allman; Charles Beeler; Bryan Cantrill; which is allocated to a subscription to
Chair, SGB Board
Emily Williams Terry Coatta; Stuart Feldman; Benjamin Fried; Communications); for students, cost
Erik Altman
Pat Hanrahan; Marshall Kirk McKusick; is included in $42 dues ($20 of which
Co-Chairs, Publications Board Columnists Erik Meijer; George Neville-Neil; is allocated to a Communications
Ronald Boisvert and Jack Davidson Alok Aggarwal; Phillip G. Armour; Theo Schlossnagle; Jim Waldo subscription). A nonmember annual
Members-at-Large Martin Campbell-Kelly;
The Practice section of the CACM subscription is $100.
Eric Allman; Ricardo Baeza-Yates; Michael Cusumano; Peter J. Denning;
Radia Perlman; Mary Lou Soffa; Shane Greenstein; Mark Guzdial; Editorial Board also serves as
the Editorial Board of . ACM Media Advertising Policy
Eugene Spafford Peter Harsha; Leah Hoffmann; Communications of the ACM and other
SGB Council Representatives Mari Sako; Pamela Samuelson;
C o ntributed A rticles ACM Media publications accept advertising
Brent Hailpern; Joseph Konstan; Gene Spafford; Cameron Wilson
Co-Chairs in both print and electronic formats. All
Andrew Sears
Al Aho and Georg Gottlob advertising in ACM Media publications is
Co n tact P o in ts at the discretion of ACM and is intended
Copyright permission Board Members
Board Chai rs to provide financial support for the various
[email protected] Robert Austin; Yannis Bakos; Elisa Bertino;
Education Board Gilles Brassard; Kim Bruce; Alan Bundy; activities and services for ACM members.
Andrew McGettrick Calendar items Current Advertising Rates can be found
[email protected] Peter Buneman; Erran Carmel;
Practitioners Board Andrew Chien; Peter Druschel; Blake Ives; by visiting http://www.acm-media.org or
Stephen Bourne Change of address by contacting ACM Media Sales at
[email protected] James Larus; Igor Markov; Gail C. Murphy;
Shree Nayar; Bernhard Nebel; Lionel M. Ni; (212) 626-0686.
Regional C o u nc i l C hairs Letters to the Editor
[email protected] Sriram Rajamani; Marie-Christine Rousset;
ACM Europe Council Avi Rubin; Krishan Sabnani; Single Copies
Fabrizio Gagliardi Fred B. Schneider; Abigail Sellen; Single copies of Communications of the
ACM India Council W e b S I TE ACM are available for purchase. Please
http://cacm.acm.org Ron Shamir; Yoav Shoham; Marc Snir;
Anand S. Deshpande, PJ Narayanan Larry Snyder; Manuela Veloso; contact [email protected].
ACM China Council Michael Vitale; Wolfgang Wahlster;
Jiaguang Sun Au tho r G u id elin es Comm un ication s of the ACM
http://cacm.acm.org/guidelines Hannes Werthner; Andy Chi-Chih Yao
(ISSN 0001-0782) is published monthly
Research Hig hlig h ts by ACM Media, 2 Penn Plaza, Suite 701,
Pu bl icati o n s Board
ACM Adve rt i sin g Departme n t New York, NY 10121-0701. Periodicals
Co-Chairs Co-Chairs
2 Penn Plaza, Suite 701, New York, NY postage paid at New York, NY 10001,
Ronald F. Boisvert; Jack Davidson Stuart J. Russell and Gregory Morrisett
10121-0701 and other mailing offices.
Board Members Board Members
T (212) 626-0686
Marie-Paule Cani; Nikil Dutt; Carol Hutchins; Martin Abadi; Stuart K. Card; Jon Crowcroft;
F (212) 869-0481 PO STMAST E R
Joseph A. Konstan; Ee-Peng Lim; Alon Halevy; Monika Henzinger;
Maurice Herlihy; Norm Jouppi; Please send address changes to
Catherine McGeoch; M. Tamer Ozsu; Director of Media Sales
Andrew B. Kahng; Mendel Rosenblum; Communications of the ACM
Vincent Shen; Mary Lou Soffa Jennifer Ruzicka
Ronitt Rubinfeld; David Salesin; 2 Penn Plaza, Suite 701
[email protected] New York, NY 10121-0701 USA
ACM U.S. Public Policy Office Guy Steele, Jr.; David Wagner;
Cameron Wilson, Director Media Kit [email protected] Alexander L. Wolf; Margaret H. Wright
1828 L Street, N.W., Suite 800
Washington, DC 20036 USA Web
T (202) 659-9711; F (202) 667-1066 Association for Computing Machinery Chair
(ACM) James Landay
Computer Science Teachers Association 2 Penn Plaza, Suite 701 Board Members A
SE
REC
Y

Chris Stephenson, New York, NY 10121-0701 USA Gene Golovchinsky; Marti Hearst;
E

CL
PL

Executive Director T (212) 869-7440; F (212) 869-0481 Jason I. Hong; Jeff Johnson; Wendy E. MacKay Printed in the U.S.A.
NE
TH

S
I

Z
I

M AGA

4 c ommunic atio ns o f the acm | j u ly 201 2 | vo l. 5 5 | no. 7

 editor’s letter

DOI:10.1145/2209249.2209250 Moshe Y. Vardi

Predatory Scholarly Publishing

Scholarly publishing is a very unique business.
In a typical business, you have two parties: sellers
and buyers. In scholarly publishing you also have
sellers and buyers, these are the publishers and
the research libraries. However, you seem like a scam, but a quick Web an unbalanced system: both publish-
have two additional parties. On one search finds many bibliographies that ers and authors want to publish, while
side, you have authors, who freely and include publications in the GESTS In- libraries have little say. Editors and re-
eagerly provide content (“publish or ternational Transactions. Apparently, viewers may try to uphold quality, but
perish”). On the other side, there are the publish-or-perish pressure creates their power is diminished when pub-
editors and reviewers, who act as gate- a market, and enterprising publishers lishers and authors are so eager to pub-
keepers. They do so for a variety of rea- are keen to meet the demand. By shift- lish. (I resigned a few years ago from the
sons: sometimes for financial remu- ing the costs to authors, GESTS is freed editorial board of a prominent proceed-
neration, but mostly out of civic duty from dealing with librarians. ings series, foregoing an annual hono-
and to gain scholarly prestige. Here is another tantalizing offer: rarium of €6,000, when I realized quality
For scholarly publishing to be suc- “Dear Professor Vardi, we would like was declining as publishing decisions
cessful as a business, publishers must to invite you as keynote speaker for one were made by the publisher rather
convince libraries to subscribe to of the next WSEAS Conferences.” This than by the editorial board.) This new
their publications. Because budgets may seem as a genuine invitation to a imbalance in scholarly publishing has
have become tighter over the last few bona fide scientific meeting, until one given rise to “predatory” publishing,
years, librarians are quite resistant to encounters a sentence such as “So, our whose main goal is to generate profits
increase their subscription inventory. plenary speakers can publish a mini- rather than promote scholarship. An
The trend, in fact, is to prune, prune, mum of one paper, maximum of three informal directory of predatory pub-
and prune. Librarians, therefore, must papers, without registration fees” and lishers and journals has over 50 entries
be convinced of a journal’s high quality “New prospective plenary speakers (see http://scholarlyoa.com/).
before adding it to their subscription must send their CV to ...” You can find As I have written previously, I be-
inventory. This resistance by libraries a vigorous online debate on the precise lieve the partnership that once existed
has been an important force for main- nature of the WSEAS conferences, with between the scholarly community and
taining quality in scholarly publishing. allegations that the organizer is an en- commercial publishers is fundamen-
Recent trends have upset this deli- terprising academic, providing a forum tally broken. Frankly, I do not under-
cate balance between publishers, li- where needy scholars can publish with- stand why Elsevier is practically the sole
braries, authors, and editors, by freeing out battling hypercritical reviewers. target for the recent wrath directed at
publishers from the need to get librar- So what has upset the traditional scholarly publisher. Elsevier is no worse
ies to subscribe. Here is an example: scholarly publishing marketplace? than most other commercial publish-
I regularly receive email solicita- On one hand, digital publishing has ers, just bigger, I believe. While not all
tions such as: “Dear Author, as a gen- gained legitimacy. In fact, there are commercial publishers are predatory
eral chair of GESTS, I am happy to many high-quality publications that publishers, they are all primarily driven
invite you for the acceptance of your are published purely digitally. On the by profits, which creates a conflict of
paper [sic] to be published in the other hand, the growing popularity of interest between publishers and au-
GESTS International Transactions.” open-access publishing popularized thors. The future of scholarly publish-
(GESTS stands for Global Engineering, the author-pays model, in which pub- ing belongs to association publishing,
Science, and Technology Society.) The lishers obtain revenues from author where the members are the publishers,
offer even provides volume and issue fees rather than from subscription fees. authors, editors, and reviewers, sharing
numbers. One has to read further down While there is nothing inherently wrong commitment to scholarship.
the invitation to find the coy reference with the author-pays model, the remov-
to “registration fees.” Again, this may al of libraries from the equation created Moshe Y. Vardi, editor-in-chief

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | com m u n i c at ion s of the acm 5

 membership application &
Advancing Computing as a Science & Profession
digital library order form
Priority Code: AD10

You can join ACM in several easy ways:

Online Phone Fax
http://www.acm.org/join +1-800-342-6626 (US & Canada) +1-212-944-1318
+1-212-626-0500 (Global)
Or, complete this application and return with payment via postal mail

Special rates for residents of developing countries: Special rates for members of sister societies:
http://www.acm.org/membership/L2-3/ http://www.acm.org/membership/dues.html
Please print clearly
Purposes of ACM
ACM is dedicated to:
Name
1) advancing the art, science, engineering,
and application of information technology
2) fostering the open interchange of
Address information to serve both professionals and
the public
3) promoting the highest professional and
City State/Province Postal code/Zip ethics standards
I agree with the Purposes of ACM:
Country E-mail address

Signature

Area code & Daytime phone Fax Member number, if applicable ACM Code of Ethics:
http://www.acm.org/serving/ethics.html

choose one membership option:

PROFESSIONAL MEMBERSHIP: STUDENT MEMBERSHIP:
o ACM Professional Membership: $99 USD o ACM Student Membership: $19 USD

o ACM Professional Membership plus the ACM Digital Library: o ACM Student Membership plus the ACM Digital Library: $42 USD
$198 USD ($99 dues + $99 DL) o ACM Student Membership PLUS Print CACM Magazine: $42 USD
o ACM Digital Library: $99 USD (must be an ACM member) o ACM Student Membership w/Digital Library PLUS Print
CACM Magazine: $62 USD

All new ACM members will receive an payment:

ACM membership card. Payment must accompany application. If paying by check or
For more information, please visit us at www.acm.org money order, make payable to ACM, Inc. in US dollars or foreign
currency at current exchange rate.
Professional membership dues include $40 toward a subscription
to Communications of the ACM. Student membership dues include o Visa/MasterCard o American Express o Check/money order
$15 toward a subscription to XRDS. Member dues, subscriptions,
and optional contributions are tax-deductible under certain
o Professional Member Dues ($99 or $198) $ ______________________
circumstances. Please consult with your tax advisor.
o ACM Digital Library ($99) $ ______________________
RETURN COMPLETED APPLICATION TO:
o Student Member Dues ($19, $42, or $62) $ ______________________
Association for Computing Machinery, Inc.
General Post Office Total Amount Due $ ______________________
P.O. Box 30777
New York, NY 10087-0777

Questions? E-mail us at [email protected] Card # Expiration date

Or call +1-800-342-6626 to speak to a live representative

Satisfaction Guaranteed! Signature

 letters to the editor

DOI:10.1145/2209249.2209251

An Integral Number and Its Consequences

P
o n d e r i n g M o s h e Y. VA RDI ’ S (Sept. 2011) apparently fell victim to a
Editor’s Letter “What Is an Author’s Response: C fallacy when citing the “celebrated C
Algorithm?” (Mar. 2012), A mathematical model tries to capture one-liner”
one should consider the fact (abstractly) the essence of the phenomenon
that in its most abstract and being modeled. From this perspective, a while (*s++ = *t++);
tangible form an algorithm is simply person saying, “What is an algorithm?”
an integral number and that an inter- would not be satisfied with the answer “An explaining its effect as “copies the
pretation of an algorithm as an ab- algorithm is an integral number.” string at s to the string at t.” But in C it
stract state machine reflects this truth. Moshe Y. Vardi, Editor-in-Chief is the other way round, because *s++ =
A researcher or engineer can then hy- *t++ is an assignment expression that
pothesize other representations of the works from right to left.1 This means
algorithm’s number that might en- Give the Past Its Due the assignment is a noncommutative
hance computational efficiency. David Anderson’s Historical Reflec- operation that in C would be represent-
One truly elegant notion supporting tions column “The Future of the Past” ed by the symmetric symbol =, not by
computational theory is that each of us (May 2012) made a persuasive case for an asymmetric symbol (such as := or ←).
can interpret a number’s symbolic rep- preserving computer hardware and With such a symbol this fallacy would
resentation differently. Depending on software artifacts. I would add that disappear like this
which lens one uses, the view might re- computer science has not preserved its
veal an integer, a floating-point value, a paper well, either; for example, when while (*s++ := *t++); or while
character in a collating sequence, or a The American Federation of Informa- (*s++ ← *t++);
computational behavior (such as func- tion Processing Societies, the parent
tion, procedure, or subroutine). of both ACM and IEEE, folded in 1990, These one-liners also observe the de-
Likewise, one could take the view its records, periodicals collection, and sign rule that “form follows function.”
that an algorithm in its most abstract books went to the dump, as did the Using this notation, the statement
state is a complete collection of all the records of the Microelectronics and (with the effect mentioned by Abraha-
intangible thoughts and ideas that Computer Technology Corporation, ms) should instead be
generate its design, whether based on or MCC, a major consortium for which
lists of imperatives, nesting functions, I worked in the 1980s. I welcome ef- while (*s++ =: *t++); or while
states and transitions between them, forts such as those led by Bruce Damer, (*s++ → *t++);
transfer of tokens between containers whose Digibarn Museum (http://www.
in a Petri net, or any equivalent rep- digibarn.com/) houses paper materi-  ürgen F.H. Winkler,
J
resentation of a Turing machine. As als, as well as working versions of vin- Feldkirchen-Westerham, Germany
such, an algorithm represents these tage PC hardware and GUI software.
ephemeral thoughts and ideas com- Computer science can do even Reference
bined, ultimately resisting definitive more to preserve its legacy, encour- 1. Ritchie, D.M. C Reference Manual. Bell Telephone
Laboratories, Murray Hill, NJ, 1975; http://cm.bell-
and universal quantification and mea- aging and assisting ethnographers labs.com/cm/cs/who/dmr/cman.pdf
surement. to look into technology development
To gain deeper insight into the and use. Otherwise, future historians, Concerning the letter to the editor by
abstractions surrounding a given al- as well as computer professionals, will Paul W. Abrahams (Apr. 2012), saying
gorithm, one would do well to also be left to infer or guess how preserved “C was designed with the PDP-11 in-
revisit earlier works on the topic. Pon- artifacts were designed, used, and ulti- struction set very much in mind” con-
dering the original intent of the early mately abandoned, much as we specu- flicts with the fact that the C language
visionaries of computing by repub- late today about the use of unearthed was based on the B language, which
lishing their work with critiques from artifacts of ancient civilizations. was written for the PDP-7; the PDP-11
today’s leaders would be enlightening Jonathan Grudin, Redmond, WA did not yet exist.
and enjoyable. As with many advanc- Concerning ++ and – modes, C’s
es in science, their efforts were often creator Dennis M. Ritchie wrote, “Peo-
met with stark criticism. However, Assignment Is Asymmetric ple often guess that they were created
reassessing their intent today could When discussing a mistake in the C to use the auto-increment and auto-
spark further refinement of modern language, Paul W. Abrahams in his decrement address modes provided
concepts derived from computing’s letter to the editor “Still Paying for a C by the DEC PDP-11 on which C and
inherently iterative and optimizing Mistake” (Apr. 2012) concerning Poul- Unix first became popular. This is his-
research life cycle. Henning Kamp’s practice article “The torically impossible, since there was no
Jody Sharpe, Omaha, NE Most Expensive One-Byte Mistake” PDP-11 when B was developed… Even

ju ly 2 0 1 2 | vol . 55 | n o. 7 | c om m u n ic at ion s o f the acm 7

letters to the editor

the alleged similarity between PDP-11 taken in by the folk myth, which seems crucial for composability, was lacking.
assembly and C is apparently ‘a folk to have had wide currency. Neither of For example, for the SelectMany op-
myth,’ pure and simple, which contin- us was aware of Dennis M. Ritchie’s erator, I had to infer from nearby text
ues propagating because it’s a handy enlightening paper from the 1993 History what SelectMany is supposed to do,
sound bite, not because there is any of Programming Languages Conference, but the article’s other “examples” (one
truth to it whatsoever”; see The Develop- as Caffin usefully cited. In it, Ritchie sentence and Figure 7) contributed
ment of the C Language at http://cm.bell- definitively clarified what he and Ken nothing to my understanding of what
labs.com/cm/cs/who/dmr/chist.html. Thompson were thinking: “This change the category is supposed to do. Why
Moreover, while the C statement was made partially to avoid the limitation would “all orders from all customers”
on the length of a string caused by holding require flattening/SelectMany
while (*s++ = *t++) ; the count in an 8- or 9-bit slot, and partly when only a single collection—or-
because maintaining the count seemed, in ders—is involved?
can be compiled into two lines of PDP- our experience, less convenient than using Finally, I disagree with the claim
11 code, the PDP-11 code given by Abra- a terminator.” that orders of arguments can ruin the
hams was not correct. The mov instruc- None of this negates Kamp’s main point, property of operator composability.
tion moves a whole 16b word, or 2B at on which I totally agree, that the decision to Unless Beckman expects developers to
a time, and is likewise not correct. The use a terminating null rather than a length (manually?) encode queries in URIs by
auto-increment operator was +, not ++. count in representing strings was a mistake pasting in text fragments, a machine
A programmer would use (R0) to point that ultimately proved very costly. might reconstruct an AST from a repre-
to the character string, not (@R0). The Paul W. Abrahams, Deerfield, MA sentation with the “wrong” order of ar-
programmer can sometimes replace guments and use it as a starting point
(R0) with @R0, but this code should for constructing the AST of another
not be concatenated in the manner Still Wondering Why LINQ Matters query. Not clear is how this might af-
cited by Abrahams. It is also possible to The sole purpose of the article “Why fect embedding in URIs, since the AST
write @(R0)+, but its meaning is differ- LINQ Matters: Cloud Composability must still be reconstructed, modified,
ent from what is needed here. The code Guaranteed” (Apr. 2012) by Brian Beck- and reencoded.
should be man seemed to be to try to show that Zeljko Vrba, Oslo, Norway
LINQ is buzzword-compatible, as in
A: movb (R0)+, (R1)+ “cloud” and “RESTful Web services.”
bne A ;Branch if Not Equal (to zero) While I agree with Beckman’s main Author’s Response:
point—that composability and design- LINQ’s foundation is the monad type class
However, this does not mean the PDP- ing for it are important and useful—I borrowed from the Haskell programming
11 could not have handled a length vari- object to the article’s exposition for the language. A monad is an interface for
able just as efficiently. Using R0 and R1 following reasons: managing monoidal container structures,
to hold the source and destination ad- First, it claimed LINQ is applicable including lists, sets, bags, and permutations.
dresses for the string cited by Abraha- to graphs (among other things). If a LINQ does not presuppose or guarantee
ms, it would suffice to move the length graph is to be treated as a collection, order. No matter how one might linearize a
of the string into another register R2. it must first be linearized (such as graph into, say, an ordered list or unordered
The programmer would then have through a topological sort), but what bag, LINQ will not change the given order.
about cycles? Moreover, not at all obvi- As for convolutions, continuations,
A: movb (R0)+, (R1)+ ous was that the result of a query is in- and exceptions, I need only show that a
sob R2,A dependent of the chosen linearization. collection is monadic to show it is also
Without concrete examples, the article LINQable. For convolutions, under the
The sob instruction, which was mis- was rather unconvincing when stating Fourier representation, LINQ operates
understood by some programmers, that, say, convolutions, continuations, on vector spaces, which are monads. For
means “Subtract One and Branch (if and exceptions can be viewed as se- the continuation and exception monads
equal to zero)” and was clearly quite quences usable for LINQ operators. (called the Maybe monad), see Haskell
powerful. Second, the article was largely de- documentation http://www.haskell.org/
Roger Caffin, Sydney, Australia voted to convincing the reader that a haskellwiki/Haskell.
LINQ query can be represented as an Vrba and I will respectfully disagree
abstract syntax tree (AST) traversable on the order of arguments. I grant that
Author’s Response: in postorder and encodable in a uni- machine reordering is a functional fix, but it
Winkler is correct; I inadvertently reversed form resource identifier (URI)—which also introduces an extra step at the call site
s and t, with s the target and t the source. is kind of a boring, well-known fact. As I would rather avoid as a point of style.
The code might have been more transparent an engineer-practitioner, I object to Brian Beckman, Redmond, WA
if I had written it as while (*t++ = *s++); the idea of encoding queries in URIs
since t would indeed then stand for “target” though accept it for the sake of the in- Communications welcomes your opinion. To submit a
and s for “source.” tellectual exercise. Letter to the Editor, please limit yourself to 500 words or
less, and send to [email protected].
As for Caffin, I suppose both Poul- Third, its description of the EX-
Henning Kamp (Sept. 2011) and I were PAND category of operators, claimed as © 2012 ACM 0001-0782/12/07 $15.00

8 communicatio ns o f the ac m | j u ly 201 2 | vol . 5 5 | no. 7

Computing Reviews is on the move

Our new URL is

ComputingReviews.com

A daily snapshot of what is new and hot in computing

 The Communications Web site, http://cacm.acm.org,
features more than a dozen bloggers in the BLOG@CACM
community. In each issue of Communications, we’ll publish
selected posts or excerpts.

Follow us on Twitter at http://twitter.com/blogCACM

doi:10.1145/2209249.2209253 http://cacm.acm.org/blogs/blog-cacm

CS and Popular “computer science.”) I have heard

there is an effort to create a television

Culture; Learning From

show that features a computer scien-
tist as its hero. Television is incredibly
powerful in popular culture, and such

Console Games a show would only help to explain who

we are and what we do. I wonder if we
should also be thinking about slower,
Mark Guzdial writes about why computer science should permeate more pervasive ways of influencing
popular culture. Judy Robertson discusses the educational benefits of popular culture.
We also need the equivalent of pop
using console games in the classroom. culture paperbacks about computer
science. When I was a student in high
school and an undergraduate in col-
Mark Guzdial Computer science is working hard lege, many of my classes also required
“The Long Road to to be considered an equal player us to read some mass-culture paper-
a Seat at the Table” among the STEM (science, technology, back that connected to the class. I re-
http://cacm.acm.org/ engineering, and mathematics) disci- member reading Future Shock for a high
blogs/blog-cacm/107743 plines. We try to get computer science school class and Cat’s Cradle in an un-
April 23, 2011 into the core curriculum alongside dis- dergraduate engineering class (to lead
I recently gave a talk at the American ciplines that are hundreds (in some into a discussion about the unexpect-
Educational Research Association’s cases, more than a thousand) years ed effects of technological advances).
annual meeting, my first time at that older than our own. We worry about My daughter just read The Dragons of
venue in more than 10 years. I spoke how people outside of our community Eden for her high school science class.
for my student, Lijun Ni, about her understand computer science. These Many (maybe even most or all?) ar-
work studying how U.S. high school are well-founded worries, and I strong- eas of science have books written for
teachers come to see themselves as ly support these efforts. But I think a the educated-but-not-specialist reader
computer science teachers (which really deep cultural understanding of about topics in that area. These books
is different than other STEM fields who we are will take awhile. are not textbooks, and they are not
since few states have CS teacher certi- It takes time to permeate popular surveys of the whole field. They are a
fication). In response to one of the ob- culture the way that other disciplines slice, written in approachable though
servations about these teachers, one have. Sure, people’s understanding not necessarily simple prose. They can
of the other speakers, an education of disciplines like engineering is not be useful to assign in a class to get stu-
post-doc at a top U.S. university, said, always accurate, but it is a lot closer dents to think about a perspective on
“Maybe that’s because they’re comput- than ours. Many layperson’s defini- the course that might not otherwise
er science teachers. Computer science, tion of “computer scientists” is that come up, and to feed into discussions.
as a discipline, is not interested in re- we are expert at using applications Where are the popular culture pa-
search and is only interested in imme- like Photoshop and Excel (unfortu- perback books on computer science?
diate answers.” That is what scholars nately, Lijun found that definition There are a few. Danny Hillis’ The Pat-
in other disciplines think about com- even among some of the high school terns on the Stone meets the definition.
puter scientists? teachers who claim to be teaching James Gleick’s book The Information

10 com municatio ns o f the ac m | j u ly 201 2 | vo l. 5 5 | no. 7

 blog@cacm

may serve that role. Few books like opinion, but both are completely appropriate the curriculum, or they were pan-
these actually contain code or describe and useful to share in the “popular icked about not being able to use the
algorithms, the stuff that computer sci- paperback” kinds of representations that technology themselves, or they were
entists talk and think about. How many might be used to reach the wider public. worried about parents’ perceptions
of us CS educators actually assign these —Mark Guzdial of it. However, the experience of see-
books in class and then discuss them? ing their students become enthused
We need books like these—and Judy Robertson about and engaged with their learning
maybe not just books but also bits of “The Impact of convinced many of the teachers that
software, simulations, videos, elec- Console Games the risks were worth it. From the per-
tronic books, and active essays. We in the Classroom” spective of keeping teachers invested
need media that are aimed at the ed- http://cacm.acm.org/ in their work and extending their pro-
ucated-but-not-specialist reader with blogs/blog-cacm/108486 fessional repertoire, managing a chal-
approachable prose (maybe with other May 19, 2011 lenging games-based learning project
modalities), that are not textbooks, Futurelab recently published a report is likely to be beneficial.
that do not aim to cover the whole on the impact of console games in Scot- A striking aspect of the classroom
field, that describe a particular slice tish classrooms. It contains case study projects is that the pupils typically
or perspective on computer science, evidence from 19 schools in which pu- spend very little time actually playing
and that could be assigned in a CS pils are involved in game-based learn- a game at school—perhaps five min-
class for breadth and to spur discus- ing projects, and considers the impact utes, three times a week. A lot of the
sion. We need a lot of media like this, from the point of view of school lead- classroom activities are based on a
as much as has been written like this ers, teachers, and pupils themselves. game but do not directly involve play-
about mathematics, biology, chemis- I find these types of studies very inter- ing it, e.g., balancing a budget for your
try, physics, and other disciplines. esting because they are about the way rock band’s forthcoming tour or writ-
If we want to take our place in popu- technology gets incorporated into edu- ing about how to best look after your
lar culture, we have to make the same cational practice in the everyday messy Nintendog pet. It seems like a small
contributions of ideas to the broad world of schools. Most of the console amount of game playing translates to a
public and provide accessible media. games used in the classes—Guitar lot of motivational “buzz.”
It is the long, slow road into permeat- Hero, Nintendogs, Endless Ocean—were Critics of game-based learning in
ing our culture the way that other dis- never intended to be educational, and school sometimes argue that playing
ciplines do. certainly not in school settings. They games in school tarnishes children’s
have been enthusiastically adopted by enjoyment of games at home, that it
Reader’s Comments teachers who happen to notice their takes the fun out of it. The children
I see a serious problem. Computing is both educational potential while playing interviewed in these case studies com-
an academic discipline and a profession them at home. Contrast this to the mented that they liked to receive sup-
for millions of people. But the two are academic development of interactive port and advice from their classmates
not the same. It is not dissimilar to the learning environments that may be about solving puzzles in the games,
difference between physics and electrical carefully designed to solve a specific and they liked the increased competi-
engineering. Computer science is most educational problem but are often suf- tion. They also liked being challenged
often presented in a fashion that parallels ficiently obscure that they never see the in school. Some children noticed the
the presentation of physics. Typically, the light of day in a classroom. The adop- titles they played at home were not as
academic discipline of computer science tion of common off-the-shelf games in good for learning as those at school—
pays but little attention to the concerns of classrooms is characterized by evan- perhaps the project taught them to ex-
the practicing IT professional. gelism. Once converted, practitioners pect more from games?
We confuse our audience if we do not tend to get quite overexcited by it all I will end with one of my favor-
clearly recognize the difference between and joyously sing the praises of such ite quotes from an interview with an
the academic discipline of computer science projects. This might go some way to eight-year-old who took part in a proj-
and the practice of IT professionals. Until explaining Futurelab’s bias in choos- ect based on Cooking Mama. “I burnt
we are clear about the difference, the great ing to identify the educational benefits a cake at home and I learned that it’s
unwashed masses can hardly be expected of console game-based learning rather just life—you have to cope with it not
to be clear about what it means to be either than the drawbacks or challenges. going right… and the topic helped us
a computer scientist or an IT professional. That said, there are some inter- with that and keeps us from getting
—Bob Fabian esting findings in the report that go bad tempers.” Coping gracefully with
beyond the excitement about moti- failure seems like a great skill to learn
It is a great point, Bob. I think we need to vating learners. One is that teachers at school whether learned through real
convey both to the general public. Computer became motivated by the projects be- or virtual baked goods.
science is a fascinating, rigorous academic cause they could see what an impact
discipline that is critical to innovation in our it had on their classes. Initially some Mark Guzdial is a professor at the Georgia Institute of
Technology. Judy Robertson is a lecturer at Heriot-Watt
world. Being an IT professional has aspects teachers were worried about adopt- University.
of both engineering and craft. The former ing games in the classroom because
is more critical for the K–12 core, in my they did not see how it would fit with © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 11
 ACM
ACM’s Member
Career & Job Center News
Vinton G. Cerf Elected
ACM President
Looking for your next IT job? Vinton G. Cerf,
vice president
Need Career Advice?

and chief Internet
evangelist for
Google, was
elected president
Visit ACM’s Career & Job Center at: of ACM in the
May 2012 general election.
An ACM member since 1967,

http://jobs.acm.org
Cerf says his “primary function
will be to convey to Council and
ACM leadership the policy views
of the general membership.
To this end, I will invite open
Offering a host of career-enhancing benefits: dialogue with any and all
members of ACM so as to be
informed of their views.”
➜ A highly targeted focus on job opportunities in
The election results include:
PRESIDENT
the computing industry Vinton G. Cerf, Google
(term: July 1, 2012–June 30, 2014)
➜ Access to hundreds of corporate job postings VICE PRESIDENT
Alexander L. Wolf,
Imperial College London
➜ Resume posting keeping you connected to the (July 1, 2012–June 30, 2014)
employment market while letting you maintain SECRETARY/TREASURER
Vicki L. Hanson,
full control over your confidential information University of Dundee
(July 1, 2012–June 30, 2014)
➜ An advanced Job Alert system notifies you of MEMBERS AT LARGE
Eric Allman, Sendmail, Inc.
new opportunities matching your criteria (July 1, 2012–June 30, 2016)
Ricardo Baeza-Yates,
➜ Career coaching and guidance from trained
Yahoo! Research Barcelona
(July 1, 2012–June 30, 2016)
experts dedicated to your success Radia Perlman, Intel
(July 1, 2012–June 30, 2016)
Mary Lou Soffa,
➜ A content library of the best career articles University of Virginia
(July 1, 2012–June 30, 2016)
compiled from hundreds of sources, and much Eugene H. Spafford,
Perdue University
more! (July 1, 2012–June 30, 2016)
The number of votes polled
by each candidate:

The ACM Career & Job Center is the perfect place to President
Vinton G. Cerf 3,843
begin searching for your next employment opportunity! Barbara G. Ryder 3,727
Vice President

http://jobs.acm.org
Alexander L. Wolf 4,992
Mathai Joseph 2,361
Secretary/Treasurer
Vicki L. Hanson 4,747
George V. Neville-Neil 2,584
Members at Large
Mary Lou Soffa 5,454
Radia Perlman 5,111
Eric Allman 4,840
Eugene H. Spafford 4,739
Ricardo Baeza-Yates 4,546
Feng Zhao 3,658
P J Narayanan 3,342

12 c omm unicatio ns o f the ac m | j u ly 201 2 | vo l. 5 5 | no. 7

CareerCenter_TwoThird_Ad.indd 1 4/3/12 1:38 PM
N
news

Science | doi:10.1145/2209249.2209255 Gregory Goth

Degrees of Separation
Researchers now have the capability to look at the
small-world problem from both the traditional algorithmic
approach and the new topological approach.

T
h e i d ea of six degrees of
separation—that is, that ev-
ery person in the world is no
more than six people away
from every other person on
earth—has fascinated social scientists
and laymen alike ever since Hungarian
writer Frigyes Karinthy introduced the
concept in 1929.
For the greater public, the cultural
touchstone of the theory was the 1990
play entitled Six Degrees of Separation
by John Guare. Although the drama
was not an exploration of the phe-
nomenon by any means, it spawned
countless versions of parlor games.
For scientists, however, the wellspring
of the six degrees phenomenon, also
called the small-world problem, was
a 1967 study undertaken by social psy-
chologist Stanley Milgram, in which
a selected group of volunteers in the
Midwestern U.S. were instructed to
forward messages to a target person A study of 721 million Facebook users showed an average of 3.74 intermediaries between a
in Boston. Milgram’s results, pub- source and target user, as opposed to social psychologist Stanley Milgram’s mean of five.
lished in Psychology Today in 1967,
Image by wikipedia user Da nnie-wa lker

were that the messages were delivered bilistic algorithms for best use of net- capability to look at the small-world
by “chains” that comprised anywhere work resources to epidemiologists problem from both the traditional al-
between two and 10 intermediaries, exploring the interplay of infectious gorithmic approach, which explores
with the mean being five. diseases and network theory. the probabilities of how each person
In the ensuing years, the problem Most recently, the vast architectur- (or network node) in a chain seeks out
has become a perennial favorite among al resources of Facebook and Twitter the next messenger using only the lim-
researchers of many disciplines, from have supplied researchers with some- ited local knowledge they possess, and
computer scientists exploring proba- thing they never possessed before—the the new topological approach, which

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | com m u n i cat ion s o f t he acm 13

news

can examine the entire structure of a ing a message via terrestrial delivery all chains can be completed in six or
network as it also observes the progres- routes—in order to fully appreciate the seven steps, “thus supporting the ‘six
sion of the algorithmic chains. similarity of results across the board. degrees of separation’ assertion,” they
“It’s amazing how far we’ve come,” While the Facebook experiment yield- wrote, “but on the other hand, esti-
says Duncan Watts, a founding part- ed approximately four degrees of sepa- mates of the mean are much longer,
ner at Microsoft Research New York ration, work by distinguished scientist suggesting that for at least some of the
City, who was until recently a senior Eric Horvitz of Microsoft Research and population, the world is not ‘small’ in
researcher at Yahoo! Watts is one of Stanford University assistant profes- the algorithmic sense.”
the world’s leading authorities on the sor Jure Leskovec, on connections be- Discovering the reason why “the
small-world problem, dating to the tween users of the Microsoft Instant world is not ‘small’ in the algorithmic
publication of “Collective Dynam- Messaging network, yielded an average sense” presents a wide swath of fertile
ics of ‘Small-world’ Networks,” co- 6.6 degrees of separation between any ground for those researchers, includ-
authored with Steven Strogatz, in Na- two users. In their 2009 paper “Social ing Watts and Leskovec, who are still
ture in 1998. At that time, Watts says, Search in ‘Small-world’ Experiments” plumbing the many vectors of net-
the largest available network, actors examining the algorithmic approach, work navigation.
listed in the Internet Movie Database, Watts, Sharad Goel, and Roby Muha- One ironic, or counterintuitive,
contained about 225,000 edge nodes mad discovered that roughly half of factor in examining the small-world
(individual actors). A recent study by problem as online communities grow
researchers from Facebook and the ever larger is that the experiments’
University of Milan, however, looked One ironic factor attrition rates are also vastly greater
at 721 million Facebook users, who than in the past. For instance, Watts
had 69 billion unique friendships in examining says only 12% of those who signed up
among them, and revealed an aver- the small-world for a joint small-world experiment at
age of 3.74 intermediaries between a Yahoo! and Facebook completed their
source and target user, suggesting an problem as online chains, compared with 75% of those
even smaller world than Milgram’s communities grow who participated in Milgram’s ex-
original study showed. periment and the 35% who completed
“In fact, the whole motivation of the ever larger is that chains in a 2001–2002 experiment
thing I did with Strogatz was precisely the experiments’ run by Watts.
that you couldn’t do the exercise Face- However, Watts says the data they
book just did,” Watts says. “Now the attrition rates are have should allow them to still answer
empirical exercise is possible. That’s a also vastly greater the questions they care about most,
remarkable change.” which is exploring the efficiency of in-
than in the past. termediary connections selected.
A Similarity of Results “We know how far you are from
One must consider the large variety of the target, Facebook knows how far
online communities and compare the your friends are from the target, and
small-world experiments performed we know who you picked, so we can
on them to Milgram’s method—send- establish whether you made the right

Science

Quantum Mechanics Increases Randomness

A partially random string of perfect random numbers?” asks measurement of the other, even measurements and if the
digits can be “amplified” into Roger Colbeck of the Institute if they are widely separated. distribution of outcomes strays
total randomness, according to of Theoretical Physics at ETH Using the partially random too far from the distribution
a pair of theoretical physicists in Zurich. As Colbeck and fellow numbers to decide which predicted by quantum
Switzerland. ETH Zurich physicist Renato property to measure—the angle mechanics, they know the
Commercial random Renner explain in an online of polarization of a photon, system is unreliable.
number generators such as publication that appeared say—and then assigning a 0 or Colbeck would like to
a beam splitter can send a last May in the journal Nature 1 based on the outcome of that find a way to generate perfect
photon through one of two slits, Physics, “Provided the adversary measurement, produces a truly randomness starting with
generating either a 0 or a 1. doesn’t know too much, the random sequence. even tiny amounts of initial
But if someone has tampered answer is yes.” If a high enough proportion randomness. That would
with the generator so its output The solution relies on of the initial sequence is not probably require performing
is not perfectly random, the entanglement, the fact that random, that affects the final another, more complicated
individual could theoretically two particles can be tied sequence in a detectable way. procedure, but just what
deduce patterns in the output together in such a way that The ETH Zurich researchers that might be, the researchers
and thus break the code. “Can measuring a physical property can run a statistical analysis do not yet know.
we turn this into a source of of one immediately provides a of the outcomes of their —Neil Savage

14 comm unicatio ns o f th e acm | j u ly 201 2 | vol . 5 5 | no. 7

 news

choice,” Watts says. “So we can get the Leskovec’s most recent work based
most science out of it, it’s just a little on small-world algorithms explores
bummer that the attrition was so bad.” “What is the right the paths that humans take in connect-
The logic behind finding the most definition of distance ing concepts that, on the surface, seem
efficient paths may produce payoffs rather disparate, such as Wolfgang
unforeseen for both theoretical mod- when you’re looking Amadeus Mozart and the Termina-
eling and production networks such at social networks?” tor character from the science-fiction
as search engine optimization. Find- films starring Arnold Schwarzenegger.
ing the best ways to determine those asks Jon Kleinberg. “As a human, I sort of know how
paths, though, will necessitate a leap “It’s not just the knowledge fits together,” Lesk-
from the known models of small-world ovec says. “If I want to go from Mozart
networks to a better understanding of how many steps to Terminator and I know Mozart was
the intermediary steps between any I have to go.” from Austria and Schwarzenegger was
two endpoints of a chain. from Austria, maybe I can go through
Leskovec says, given constants from the Austrian connection. A computer
graph theory, the diameter of any given that is truly decentralized has no clue,
network will grow logarithmically with it has no conception that getting to
its size; that is, the difference between Schwarzenegger is good enough.”
five and six degrees of separation man- Interestingly enough, Leskovec
dates a graph an order of magnitude tion in everyday life and when you’re says, computers fared better than
larger or denser. Jon Kleinberg, Tisch designing some online system.” humans on average on solving such
University professor in the department search chains, but humans also were
of computer science at Cornell Univer- Mozart Meets The Terminator less likely to get totally lost and were ca-
sity, whose “The Small-World Phenom- Recent research is beginning to use the pable of forming backup plans, which
enon: An Algorithmic Perspective” is re- short-path principles of social search the Web-crawling agents could not do.
garded as one of the problem’s seminal in the online systems discussed by Effectively, he says, the payoff of such
modeling documents, says this basic Kleinberg. In “Degrees of Separation research is “understanding how hu-
property is precisely what makes the in Social Networks,” presented at the mans do this, what kind of cues are we
small-world theory so appealing while Fourth International Symposium on using, and how to make the cues more
also presenting the research communi- Combinatorial Search 2011, research- efficient or help us recognize them, to
ty the greatest challenge inherent in it. ers from Shiraz University, Carnegie help us understand where we are, right
“It’s something that still feels coun- Mellon University, and the University now, in this global network.”
terintuitive when you first encounter of Alberta designed a search algorithm,
it,” Kleinberg says. “It makes sense in tested on Twitter, intended for uses be-
Further Reading
the end: I know 1,000 people and my yond social search.
friend knows 1,000 people—and you For example, they reported in Voice Backstrom, L., Boldi, P., Rosa, M.,
Ugander, J., and Vigna, S.
don’t have to multiply 1,000 by itself over Internet Protocol (VoIP) networks,
Four degrees of separation, http://arxiv.org/
too many times for it to make sense.” when a user calls another user in the abs/1111.4570, Jan. 6, 2012.
However, this logarithmic progres- network, he or she is first connected
Bakhshandeh, R., Samadi, M.,
sion also precludes the ability to ex- to a VoIP carrier, a main node in the Azimifar, Z., and Schaeffer, J.
amine or design intermediate levels network. The VoIP carrier connects the Degrees of separation in social networks,
of scale, Kleinberg says. “We thought call to the destination either directly Proceedings of the Fourth International
the right definition of distance was or, more commonly, through another Symposium on Combinatorial Search,
Barcelona, Spain, July 15–16, 2011.
going to be ‘Here I am, and how many VoIP carrier.
steps do I have to go to get to you?’ “The length of the path from the Goel, S., Muhamad, R., and Watts, D.
but that turns out not to be. We need caller to the receiver is important since Social search in “small-world”
experiments, 18th International World
some other measure and I think that it affects both the quality and price of Wide Web Conference, Madrid, Spain,
remains an interesting open question, the call,” the researchers noted. “The April 20–24, 2009.
that people are actively looking at: Is algorithms that are developed in this
Kleinberg, J.
there some kind of smoother scale paper can be used to find a short path The small-world phenomenon: an
here? Who are the 10,000 people clos- (fewest carriers) between the initial algorithmic perspective, 32nd ACM
est to me? The 100,000? (sender) and the goal (receiver) nodes Symposium on Theory of Computing,
“We need a much more subtle way in the network.” Portland, OR, May 21–23, 2000.
to do that and it is going to require These algorithms, such as greedy West, R., and Leskovec, J.
some sophisticated mathematical algorithms enhanced by geographic Human wayfinding in information networks,
22nd International World Wide Web
ideas and sophisticated combination- heuristics, or probabalistic bidirection-
Conference, Lyon, France, April 16–20, 2012.
al ideas—what is the right definition of al methods, have the potential to cut
distance when you’re looking at social some of the overhead, and cost, of net- Gregory Goth is an Oakville, CT-based writer who
networks? It’s not just how many steps work search sessions such as the sam- specializes in science and technology.

I have to go. That’s an important ques- ple VoIP session, the authors believe. © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 15
news

Technology | doi:10.1145/2209249.2209256 Gary Anthes

HTML5 Leads
a Web Revolution
Propelled by a proliferation of mobile devices and social networks,
an enhanced family of Web specifications is bringing new power
to developers and new capabilities to users.

D
evelopers of softwarefor the for more than two decades remained Indeed, the popular definition of
World Wide Web say the new at the core of Web software, HTML5 HTML has expanded and matured as
HTML5 standard is revolu- is most often thought of broadly to the Internet has grown more powerful
tionizing the way the Web include new versions of the markup and its reach has increased, says Ian Ja-
evolves, works, and is used. language itself and its associated cobs, editor of the W3C’s HTML4 recom-
It is simplifying the work of program- standard for accessing and manipu- mendations and other standards. “The
mers, harmonizing access to diverse lating HTML documents, the Docu- Web over 20 years has developed from
devices and applications, and giving ment Object Model; Cascading Style a Web of more-or-less static documents
users amazing new capabilities, they Sheets (CSS), a language to define the to, now, a platform for applications.”
say. Yet, HTML, the HyperText Markup presentation and appearance of an There are two driving forces behind
Language, is just a way to tag parts of HTML document; and the JavaScript this evolution, Jacobs says. First is the
a document so that Web browsers can scripting language. The term is often proliferation of diverse devices that,
deal with them intelligently. used even more broadly to include coupled with the variety of browsers,
How could a humble mechanism specific application programming greatly complicate life for developers,
for tagging Web pages have such a interfaces (APIs), such as those that who want to “write once and deploy ev-
big impact? Is the hype surrounding enable new browser-based graphics, erywhere.” Second, he says, “the Web
the fifth version of HTML just a lot of geolocation, local storage, and video has now embraced the social network-
geeky noise? And why should comput- capabilities. ing model, and when you can tap into
er professionals care about it, anyway? And HTML5 is at the heart of the that, you can reach many more custom-
The hype is justified; the difficulty World Wide Web Consortium’s (W3C’s) ers.” In some cases, hundreds of mil-
lies in the definition because HTML5 Open Web Platform, an umbrella term lions of more customers.
is both a single specification and a that changes over time and that refers
whole set of technologies. to the markup language and various A Developer-Friendly Standard
While the markup language has technologies that pertain to it. Ocupop, a graphics and Web design
and marketing firm, has begun us-
ing HTML5 for all its Web work. Not
all of the highly touted features in the
umbrella standard are strictly speak-
ing “new,” says Matthew McVickar, a
Web developer and usability designer
at Ocupop. “They are codifications of
techniques used in the past. It takes
stuff that developers were trying to do,
or wanted to do, and made them into
stuff that’s natively supported in the
browser.” For example, he says, HTML5
has a standard JavaScript interface for
Screensh ots courtesy of mobile- pat t erns.co m

geolocation, so that a Web browser on

a mobile device can access GPS data
without invoking a custom-written API
to a GPS device or application.
That may not be apparent to the end
user, but it is a big deal for software
developers. Developers can, by writing
to HTML5, enable transparent access
Thanks to HTML5, developers can now enable transparent access to geolocation information to geolocation information without
without having to write separate code for each browser and device. having to write separate code for each

16 comm unicatio ns o f the ac m | j u ly 201 2 | vo l. 5 5 | no. 7

 news

browser and device. It is exactly what a developing products to improve the vi-
standard is intended to do. sual quality of Internet video. He says
Similarly, CSS3, the newly updated The newly updated video is the most complex of Internet-
presentation language, allows a devel- presentation borne information and that vendors’
oper to produce certain looks natively technologies for codecs, streaming
in the browser, without constructing language CSS3 protocols, and DRM will be “the slow-
them externally—say, in Photoshop— allows a developer est to converge” to standards.
and then importing them. For exam- Officially, HTML5, when narrowly
ple, a heading can be given a text shad- to produce certain defined as the hypertext markup lan-
ow quickly and easily in the browser, looks natively in guage specification, is on track to
and it can be changed “on the fly,” says become a full specification and an of-
McVickar. “It’s a huge timesaver.” the browser without ficial recommendation by the W3C in
McVickar notes the new standards constructing them 2014. “But the individual specifica-
are providing a useful catalyst for ven- tions are at different maturity levels
dors eager to take advantage of the externally and then and will become standards at different
Web’s latest capabilities. “The browser importing them. times,” the W3C’s Jacobs says.
vendors are falling all over themselves Philippe Le Hégaret, interaction do-
to develop their browsers as quickly and main lead at the W3C, says, “It is not a
as cutting edge as they can,” he says. product where we say, ‘OK, now we are
In a feature referred to as “local stor- done.’ The scope keeps increasing.”
age,” HTML5 allows persistent storage work, but it’s critically important stuff.” It includes about 60 APIs now, and re-
of structured session data on the Web Still, although it is being deployed quests for additions keep coming in,
client. Unlike cookies, which can only now, HTML5 is not a finished standard, he says. For example, the W3C recently
efficiently store data in small amounts and its adoption varies by company and received a request to include support
and have other technical limitations, industry. For example, it does not spec- for conversion between speech and
this capability allows the storage of large ify a single standard for video compres- text inside the Web browser, a request
amounts of data for access later when a sion (codec), streaming protocol, or that is under consideration.
connection to a Web server may not be digital rights management (DRM). The “There’s a near-infinite volume of
possible or desirable. And HTML5 sup- industry was well on the way toward things that the Web doesn’t support
ports richer graphics, such as native adopting Adobe Flash as a de facto yet,” says Google’s Hickson. Asked
support for embedded Scalable Vector standard for video until Apple in 2010 about HTML6, he says, “HTML is just
Graphics, and the raster-based Canvas declared its iPhone and iPad would not HTML, we [WHATWG] dropped the
that enables users to draw 2D and 3D support Flash but would employ Ap- number early last year. It’s just con-
graphics in a Web page using JavaScript. ple’s own collection of Web technolo- tinuously developed, like the browsers.
gies, including HTML5. Meanwhile, Development of HTML will just pro-
Tighter Specs Microsoft and Google have their own ceed until HTML is dead.”
The new specifications are a big help not entirely compatible approaches to
to the vendors, agrees Ian Hickson, a video, and different browsers support
Further Reading
software engineer at Google and the different audio and video codecs. So,
company’s liaison with W3C and the software developers still must accom- Bidelman, E., et al.
HTML5 Rocks, http://www.html5rocks.com.
Web Hypertext Application Technology modate multiple methods if they want
Working Group (WHATWG). WHATWG to have comprehensive coverage. Khan, S.
is a complementary standards body Hui Zhang, an Internet specialist HTML5 unleashed: tips, tricks and
techniques, http://www.w3avenue.com,
founded in 2004 by Hickson, then at Ap- and computer science professor at May 7, 2010.
ple, and others from Apple, the Mozilla Carnegie Mellon University, explains
Lawson, B. and Sharp, R.
Foundation, and Opera Software. “One that “HTML5 has tremendous momen- Introducing HTML5, 2nd Edition,
of the most important strides we’ve tum, but it’s not 100% supported across New Riders Press, Berkeley, CA, 2011.
made with the HTML effort has been all browsers because it’s not a standard
Web Hypertext Application
the significant increase in the quality yet. Many vendors are contributing Technology Working Group
of our specifications,” says Hickson. “It to the process.” The challenge is not HTML–Living Standard,
used to be that the specs were pretty unique to video, he notes. Vendors see http://www.whatwg.org/specs/web-apps/
vague; two browser vendors could im- the advantages of standards for them- current-work/multipage.
plement the same specs and honestly selves and their customers, but at the World Wide Web Consortium
claim compliance without those brows- same time they strive to differentiate HTML5—A vocabulary and associated APIs
for HTML and XHTML, Editor’s Draft, May
ers truly being compatible. Now, the their products for competitive advan-
8, 2012, http://dev.w3.org/html5/spec/
specs are so detailed that if you imple- tage. “You want to get a bigger share of Overview.html.
ment the spec as written you really will the pie,” he warns. “But if you are too
be compatible with all the other brows- greedy, the pie is destroyed.” Gary Anthes is a technology writer and editor based in
ers and with all the content that already Zhang is keenly interested in video Arlington, VA.

exists. This has been a huge amount of as a cofounder of Conviva Inc., a firm © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l. 55 | n o. 7 | c om m u n i c at ion s o f t he acm 17
news

Society | doi:10.1145/2209249.2209257 Marina Krakovsky

Patently Inadequate
The biggest change to U.S. patent law in nearly 60 years
brings many changes, but fails to solve the software industry’s
most vexing problems.

W
ell before President
Obama signed the Lea-
hy-Smith America In-
vents Act (AIA) into law
last September, the bill
was already being hailed as the biggest
overhaul to U.S. patent law since 1952.
Promising to spur innovation, shorten
application backlogs, and curtail legal
costs, the bipartisan bill easily passed
both houses of Congress. But despite
grand aims, lawmakers did not rebuild
the patent law from the ground up. In-
stead, they assembled a hodgepodge
of compromises, particularly between
the interests of large software compa-
nies, for whom patents have largely
been a net drain, and those of biotech-
nology firms, which favored strong
patent protection. To the bill’s mild-
est critics, AIA did not go far enough President Obama signed the America Invents Act into law on September 16, 2011, at Thomas
in meeting the needs of the software Jefferson High School for Science and Technology in Alexandria, VA.
industry. To bigger detractors, the
new law is even worse than the old Although the full implications are thorn in its side. Consider, for exam-
system—it is the legislative equivalent not known, most experts agree on ple, the Texas case earlier this year in
of spaghetti code, a jumble of rules which handful of changes will have which World Wide Web inventor Sir
whose meaning and implications will the greatest impact on the software in- Tim Berners-Lee testified for the de-
take judges and intellectual property dustry, for which the threat of patent- fense against a group of plaintiffs who
(IP) lawyers years to untangle. infringement claims has long been a claimed that anyone using interactive
“This law is what the British call a Web features was trampling on their in-
‘dog’s breakfast’—a little bit of every- tellectual property. Though interactive
thing,” says University of California– One of the biggest features seem obvious to Internet pro-
Irvine law professor Dan Burk, who grammers today and have been a main-
testified during Congressional delib- and most positive stay of Web sites for years, the ease of
erations last March. Unfortunately, changes with the both patenting and suing made it easy
Photogra ph by Rex Features (144352 8G) Unit ed States VIA Ap i mages

Burk says, the lawmakers’ attempts for holders of two older patents to try
to reconcile competing aims led to America Invents Act wringing money out of anyone using in-
provisions that have not been tested is the establishment teractive elements. (The plaintiffs lost.)
in other countries. “None of this new The overwhelming majority of patent
stuff has ever been seen anywhere of prior-user disputes never actually go to trial, but
on the planet Earth before,” he says. rights as a defense even settling lawsuits is expensive. Un-
“This is brand-new stuff that they der the old law, Burk says, “a patent
made up in the halls of Congress, so against patent holder could sue, offer to settle for $1.5
nobody really knows what it means infringement suits. million, and walk away with a nice pile
or how a lot of it is going to turn out. of cash without putting the patent sig-
As a consequence, I’m guessing we nificantly at risk. It was cheaper for the
have 20 years of litigation ahead of us defendant to pony up the $1.5 million.”
before we know what the rules of the This problem will not go away un-
game are.” der the new law, but some of the law’s

18 com municatio ns o f th e ac m | j u ly 201 2 | vol . 5 5 | no. 7

 news

changes will help at least somewhat. are now is ‘first-inventor-to-disclose.’” Legal

That is because the law gives inventors
Jim Gray
One of the biggest and most positive
changes is the establishment of prior- up to a year to file a patent after pub-
user rights as a defense against patent licly disclosing the invention. “That
infringement suits. “There are thou-
sands and thousands of examples of
then blocks everybody out for a year.”
An unintended consequence of this,
Declared
software used by companies for making
things,” explains law professor John Al-
Burk believes, will be an impetus to
game the system, especially around the Dead
lison of the University of Texas–Austin. March 16, 2013 changeover to the new
In many other countries, patent laws en- system, with software developers and On a clear and sunny day
in January 2007, Jim Gray
able companies to keep such internally their lawyers strategizing about when disappeared with his 40-foot
used techniques a trade secret without to disclose an invention and when to yacht Tenacious off the coast
worrying that they would be on the hook file a patent. The law is not retroactive: of San Francisco. Despite an
exhaustive search, neither Gray
for patent infringement if someone else Patents filed before the March change-
nor the vessel was ever found.
decided to patent the same technique. over will be subject to the old system, Under California law, a person
In the U.S., on the other hand, “under so anyone with a patentable idea today cannot be legally declared
the old law you could lose your ability to faces the tough decision of whether to dead until five years following a
disappearance.
use technology that you invented,” says file now or to delay for the sake of fall- A California court granted
John Duffy, a law professor at the Uni- ing under the new system. Burk says Gray’s death certificate on
versity of Virginia School of Law. he is already seeing this kind of head- May 16. He was 63 when he
The AIA’s introduction of the prior- scratching among some companies’ vanished while heading out to
the Farallon Islands to spread
user defense changes all of that. Es- in-house counsel. The rules of the his mother’s ashes at sea.
pecially given the costs of patenting, game are even more muddled by the “I don’t focus on the mystery—
firms will likely have less incentive to fact that the law does not actually de- it’s a problem that could not
be solved and that’s extremely
disclose their inventions under this fine the term “inventor,” he says. frustrating for those in the
new provision, so the upshot could Despite the one-year grace period, computing sciences arena,”
well be fewer patent filings and more most experts predict the change from says Donna Carnes, his widow.
trade secrecy. Even so, only time will a first-to-invent system will spur a rush “I have had to learn to accept
the situation. But this does
tell whether companies will increase to the patent office, which could mean a bring closure to a long and
investment in technologies that they pile of hasty, slapdash applications and, difficult event.”
will not be patenting, says Allison, an most likely, an even bigger edge than Gray was instrumental in
developing several important
empirical legal scholar. ever for corporate giants with the means
database and transactional
to hire teams of pricey IP lawyers. systems, including IBM’s
First to Disclose Some say that to compete with over- System R, which served as a
Another major change, which like the seas patent applicants the larger com- building block for SQL. Gray
also helped develop granular
prior-user defense tries to harmonize panies were already thinking in terms database locking and widely
America’s patent process with that of first-to-file even before the new law. used OLAP cubes. He played a
of the rest of the world, is the much- As Gruner puts it, “Any company that key role in building Microsoft’s
discussed move from a first-to-invent filed [later than others] in Europe or Virtual Earth platform as
well. Gray received the ACM
system to a first-inventor-to-file sys- Japan was already out of luck in those A.M. Turing Award in 1998
tem. The old system was a recipe for systems.” But the AIA’s expansion of “for seminal contributions
priority disputes (called interferences) what counts as “prior art” that can be to database and transaction
processing research and
because if two people working inde- used to invalidate a patent creates new technical leadership in system
pendently both filed for a patent, there incentives to file early and often. This implementation.”
was a question of who was the first to expansion, of course, also means it The news that Gray’s yacht
invent. “Those proceedings were very will be harder to obtain a patent, since was missing sent shock waves
through the computer science
messy and hard to resolve because more prior art will mean that fewer community. Many of Gray’s
people didn’t document very well what new patent applications will pass the colleagues and friends—from
they were thinking on a given day,” novelty and non-obviousness tests. Amazon, Google, IBM, NASA,
Oracle, and Microsoft, where he
says John Marshall Law School profes- worked as a Technical Fellow
sor Richard Gruner, who directs the Weeding Out the Worst when he disappeared—poured
school’s Center for Intellectual Prop- Just about everyone agrees the AIA will their knowledge and expertise
erty Law. The new law aims to ward off do away with the very worst patents. “A into the search.
“At this point I am simply
such contests by creating a bright-line lot of people in the software industry grateful for the time I had
rule—priority would simply go to the complain about obvious patents being with Jim,” says Carnes. “He
first person to file the patent. issued,” says the University of Virginia’s made enormous contributions
But in practice things will not be Duffy, adding that obvious or non-novel to the world and those around
him. I am hoping that this
so clear-cut, says Burk. With the new patents increase costs without improv- event brings a bit of peace
law, he says, “we’re neither first-to- ing business conditions. “If there are to everyone.”
invent nor first-to-file. What we really a lot of bad patents, that’s like stupid —Samuel Greengard

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he ac m 19
news

regulation—the kind of thing trade or- The new joinder provisions change
ganizations lobby to prevent.” There- the NPEs’ cost-benefit analysis, since
fore, even though Duffy is wary of big The America Invents plaintiffs will no longer be able to inex-
government (and concerned the AIA Act’s expansion of pensively target additional defendants.
may have given the U.S. patent office The marginal cost of filing separate
the fee-setting power to bloat itself), what counts as lawsuits is not huge, either, but the
he is glad to see new ways for anyone to “prior art” that can new math means the NPEs will not be
challenge bad patents directly through suing willy-nilly.
the patent office rather than the courts. be used to invalidate It is possible the law will have posi-
That opens up the possibility of com- a patent will create tive effects on the larger economy, but
munal policing—a kind of crowdsourc- probably not in ways the bill’s support-
ing of the work of weeding out bad pat- new incentives to ers would want. When reporters called
ents, which could drive down the costs file early and often. Irvine’s Burk for a comment on Senator
of maintaining high patent quality. But Patrick Leahy’s claim that the AIA would
like other legal scholars, Duffy is hesi- create 200,000 new jobs, Burk was quick
tant to make predictions until the evi- to offer a quip. “I said, ‘Yeah, for patent
dence comes in. “It’s not at all clear that lawyers.’ If I were running I business, I
it’ll reduce costly litigation,” he says. wouldn’t be so excited about it.”
“That’s the hope, not proven reality.” What about patent trolls, the so-
It is also not clear that getting rid of called “non-practicing entities” (NPEs)
only the worst patents will do much to that profit from settlements in nui- Further Reading
slow large firms’ current patent arms sance suits against software companies Crouch, D. and Rantanen, J.
race that, most recently, spurred Apple, that actually develop and sell products? Patently-O blog, www.patentlyo.com.
Microsoft, and Google to pay billions of Such legalized shakedowns are another Coyne, P.J.
dollars to acquire arsenals of patents bane to the industry, particularly to The America Invents Act: How does it
from Nortel and Motorola. This Cold large corporations, so the patent-troll strengthen the patent system? Bloomberg
BNA’s Patent, Trademark & Copyright
War-like strategy of mutually assured problem was a major impetus for pat- Journal, Oct. 28, 2011.
destruction may be an effective de- ent reform. Yet only the new joinder
Durham, A.
fense against lawsuits and injunctions provisions, meant to limit plaintiffs’ Patent Law Essentials: A Concise Guide,
to stop selling products, but every dol- ability to target multiple defendants in Praeger, Westport, CT, 2009.
lar spent stockpiling patents is a dollar a single lawsuit, are likely to make any
Hawley, J.J.
not spent on innovation. Improving kind of dent in that problem. “Suppose Patent Law Forum: America Invents Act,
patent quality should help relieve this you have a piece-of-junk patent,” ex- http://youtu.be/P383R9MFtiE, March 12, 2012.
problem, and the law aimed to make plains Sichelman, “and the odds of you Pressman, D.
it easier to challenge bad patents, says winning a [patent infringement] law- Patent It Yourself, Nolo, Berkeley, CA, 2011.
Ted Sichelman, a former software en- suit are pretty low; but you know it will
trepreneur who is now a professor at cost everyone involved $50,000, and in Based in San Francisco, Marina Krakovsky is the
the University of San Diego School of patent cases it’s hard to get attorneys’ co-author of Secrets of the Moneylab: How Behavioral
Economics Can Improve Your Business.
Law. “But those provisions were wa- fees back. So the old model was to sue a
tered down in the final bill.” bunch of companies at once.” © 2012 ACM 0001-0782/12/07 $15.00

Milestones

Computer Science Awards

The Technology Academy of was recognized for his work to both automated reasoning is Barbara Liskov, Institute
Finland, Royal Society, National on pluripotent stem cells.) and the automated formation Professor, Massachusetts
Academy of Sciences, and Torvalds received the award in and evolution of representations Institute of Technology.
Cambridge Center for Behavioral recognition “of his widely used of knowledge.”
Studies recently honored leading Linux kernel and the creation Loebner Prize
computer scientists. of a new open source operating National Academy The Cambridge Center for
system for computers.” Members Behavioral Studies awarded the
Millennium Technology The National Academy of 2012 Loebner Prize to Mohan
Prize Royal Society Fellows Sciences elected 84 new Embar, a software consultant
The Technology Academy of The Royal Society named members and 21 foreign based in Milwaukee, for the
Finland named Linus Torvalds Alan Bundy, professor of associates from 15 countries performance of his chatbot,
as one of the two laureates for automated reasoning, School in recognition of their Chip Vivant, which was voted
its 2012 Millennium Technology of Informatics, University of distinguished and continuing to be the most humanlike
Prize. (The other laureate is Edinburgh, as a 2012 Fellow for achievements in original conversationalist.
Dr. Shinya Yamanaka who his “world-leading contributions research. Among those elected —Jack Rosenberger

20 comm unic atio ns o f the acm | j u ly 201 2 | vo l. 5 5 | no. 7

 news

History | doi:10.1145/2209249.2209258 Paul Hyman

Lost and Found

Researchers discover computer pioneer Konrad Zuse’s
long-forgotten Z9, the world’s first program-controlled binary relay
calculator using floating-point arithmetic.

W
hen researchers stum- his time programming and maintain-
bled across the Zuse Z9 ing the Z9. Another, electrician Max
in the storage room of Forrer, was the technician in charge of
a Swiss museum two the Z9 at a Swiss spinning and weaving
years ago, they initially company from 1956 to 1968. The ma-
did not recognize the significance of chine was mostly used to keep track of
what they had found. In fact, the Z9 the salaries of the company’s 700 em-
was unknown even to many historians ployees, print invoices, manage inven-
of computing. tory, and create sales reports.
But they soon came to realize what “The Z9 was very expensive to main-
they had uncovered: The world’s first tain,” Forrer recalls, “and had many
workable program-controlled binary problems necessitating having a tech-
relay calculator using floating-point nician in the company.” He remembers
arithmetic, which was the creation of being sent to Zurich for six months
computer pioneer Konrad Zuse, whose where he met Konrad Zuse, who over-
100th birthday they were celebrating. saw his training.
A German civil engineer, Zuse had During his search for eyewitnesses,
created the world’s first functional, Bruderer talked to many Remington
program-controlled Turing-complete Rand customers and to museums in
computer, the Z3, in 1941. That same Switzerland and abroad to find surviv-
year he founded one of the earliest Konrad Zuse tinkers with a reproduction of
ing machines.
commercial computer companies, an early mechanical computer in 1980. But, to Bruderer’s surprise, the
which produced the Z4, the world’s curator of the Museum für Kommu-
first commercial computer. And in to Herbert Bruderer, a lecturer in the nikation in Berne contacted him to
1946, he designed the first high-level department of computer science at say that, in 2010, it had acquired a Z9
programming language, Plankalkül. ETH Zurich, who has written a book, from the Technorama, which had re-
He founded another company, Zuse Konrad Zuse And Switzerland: Who In- cently divested itself of its computer
KG, in 1949 and developed the Z9 for vented The Computer?, about Konrad collection. The Z9 now resides in the
Remington Rand Zurich and other Zuse and other early computer pion- museum’s main repository.
large companies. The Z9 consisted of neers and their inventions. Zuse KG built a total of 251 com-
a processor, an input and output unit, On June 22, 2010, Konrad Zuse’s puters by 1967, but suffered financial
and a rectifier, and was embedded in 100th birthday, the Zurich newspaper problems. It was eventually purchased
a group of punch card machines. The Tages-Anzeiger published an article by by Siemens and no longer exists.
Z9 was Zuse KG’s first relay calculator Bruderer about Zuse, who had died in One final note adding to the mys-
manufactured in series, with delivery 1995, and the Z4. “Two days later, I got tery: The Z9 is often referred to as
starting in 1953. Some 20 to 30 Z9s a phone call from Josef Steinmann, the M9 as a result of legal wrangling
were manufactured. who lives near Lucerne, [Switzerland] that occurred when Konrad Zuse was
The confusion surrounding the Z9 who told me about the existence of forced to change its name to conclude
stems in part from the fact that Kon- the Z9 and his experience with the ma- his contract with Remington Rand. Be-
rad Zuse describes the machine in chine,” Bruderer recalls. cause his patents had been temporar-
Photogra ph by A P- Photo/ Karsten T hielker

his memoirs but never refers to it by Steinmann is one of the few living ily transferred elsewhere, the company
name. According to his writings, the Z9 users, having worked on the tech- decided to rename the device the M9,
Z9 was able to perform all four basic nical staff of Remington Rand Zurich with M standing for “Mithra,” report-
arithmetic operations, among oth- from 1956 to 1960. “Together we start- edly “a mysterious subsidiary of Rem-
ers. The calculating punch was used ed looking for other eyewitnesses,” ington Rand Zurich,” says Bruderer.
mostly for office work but also for en- says Bruderer. “We found five.”
gineering tasks. One, electronic engineer Ernst In- Paul Hyman is a science and technology writer based in
Great Neck, NY.
However, all but one of the Z9s has auen, began working at Remington
apparently been destroyed, according Rand Zurich in 1958, spending most of © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i cat ion s o f t he ac m 21
V
viewpoints

doi:10.1145/2209249.2209259 Mari Sako

Technology Strategy
and Management
Business Models for
Strategy and Innovation
While often ambiguously defined, business models are central to innovation.

W
e frequently hear the vations that precede the U.S.-centric in the face of users’ presumption that
term business model roll management education in the 20th information should be made available
off the tongue of not century. However, the proximate ori- free on the Web.
only technology entre- gin of the use of the term (rather than
preneurs and venture the concept) of business models may Business Models Triggered
capitalists, but also corporate manag- be traced to specialists in computing by Digital Technology
ers, public officials, and professionals. and systems modeling. They used the Business models are essential for con-
When the President of the Law Society term to refer to computer simulations verting new technologies into com-
of England and Wales asks what the of business processes. It was argued mercial value. Every company has an
business model for lawyers should be, that as the business environment be- implicit business model in the sense
or when a commodore states that his came more complex, the potential val- of linking ideas and technologies to
naval acumen has been put to test by ue of business models would inevitably commercial outcomes. However, it is
the business model of Somali pirates, increase, and that as managers gain the startups that had to articulate their
it is time to become more circumspect. more knowledge in modeling tech- business model, not least to obtain ini-
What is so enticing but also elusive niques, computerized models should tial funding. A fantastic piece of new
about business models and business become an indispensable aid in many technology remains unexploited if it
model innovation? This column consid- business functions. lacks the “go to market” mechanisms
ers when and how the term came into However, business model did not for creating value for end users.
use, and then addresses three intrigu- acquire prominence in the lexicon of Starting with e-commerce and more
ing questions: Why does it continue to the start-up community until the mid- recently with interactive Web appli-
be an important concept in practice? 1990s.2,6 With the exponential growth cations and cloud computing, digital
Who should be in charge of innovating in the usage of the term during the technology has created opportunities
in business models? How transferrable dot-com boom, the early literal mean- for new business models. Technology
is a business model across companies? ing of business model (and modeling) impacts two sides of business mod-
The notion of business models is morphed into a new meaning, refer- els: cost and revenue. For example,
as old as the hills, and human history ring to the creation of value in digital the disintermediation of retailers ad-
is strewn with business model inno- business. This value focus came about opted by Amazon and Dell touches on

22 c ommunicatio ns o f th e ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 viewpoints

both sides, as does the “pay per click” When we see a well-functioning capture? Some argue that a business
advertising-based search by Google. By business model in action, it is not dif- model is about co-defining total value
contrast, a pricing model (such as the ficult to see there are several compo- creation—the overall size of the pie—
“freemium” model adopted by Adobe, nents that fit into a coherent whole. for all parties involved. How much of
Skype, Myspace, and open source soft- In particular, a business model articu- the total pie the focal firm actually cap-
ware companies to give basic service lates the customer value proposition; tures depends on its revenue model,
away for free and charge premium pric- it identifies a market segment; it de- which should remain separate from its
es for value-added services) affects the fines the structure of the value chain; it business model.5 By contrast, others
revenue side. specifies the revenue generation mech- argue that a business model defines
The so-called Web 2.0 and cloud anisms; it describes the positioning both the “go to market” and “value
computing lie on this continuum in within the value network or ecosystem; capturing” strategies.4 In the latter, de-
the potential development of new busi- and it also elaborates on competitive signing and implementing a business
ness models, addressing both sides. strategy by which the firm gains and model is the essence of strategy. In
On the cost side, cloud computing re- holds advantage over rivals.1 the former, a business model must be
moves the need to invest up front in For example, Apple, with its iPods, combined with other things (a revenue
expensive servers, thus turning capital iTunes, iPhones, and iPads, provides model, resources and capabilities, and
expenditure into operational expenses a unique experience to consumers, by so forth) to constitute a good strategy.
and lowering barriers to entry. How- targeting the market segment that cares A second point on which strategists
ever, to turn various service models about style as well as performance. disagree lies in the balance between
such as “infrastructure as a service” or Apple has an outsourced positioning planning and experimentation to ar-
“software as a service” into a business in the supply chain, has specified its rive at an appropriate business model.
model, providers must work out the revenue generation mechanisms that The planning perspective regards a
revenue side with the choice of a specif- bundle products and services, and has business model as a matter of design-
ic pricing policy and targeted customer created an ecosystem that includes ing an activity system.5 By contrast, the
segments. Without working out these telecom providers, music content pro- experimentation perspective empha-
dimensions, the money will not follow viders, and other suppliers. Apple’s sizes a discovery process of trial and
no matter how cool the technology. business model behaves like a platform error, just as Google experimented in
as it attracts external companies to in- the process of a transition from pay-
Business Models in Strategy vest in activities that enhance its value. per-transaction to advertising-based
Business strategists started to pay Thus, business models are firm-cen- Internet search. Any novel idea goes
greater attention to the business model tric, but with a lot of thought that goes through a process of trial and error be-
once its central focus on value creation into their boundary-spanning func- fore it becomes viable as a prototype,
became evident. In contrast to busi- tion. The latter ensures the focal firm then a model to be adopted widely.
ness process design at the operational can rely on the resources and capabili- In this sense, followers may attempt
level, a business model defines the ties of third parties, and harness ex- to imitate a well-articulated business
overall business logic of a company at ternal technologies and ideas through model. But inventing a novel business
Illustration by Mich ael Aust in

the strategic level. However, there are “open business models.”1 model is a matter for innovators taking
two ambiguities that, if resolved, would However, the first unhelpful dis- risks in the process.
make the business model a more use- agreement among strategists is on
ful tool in strategy: first, what elements how to bound the business model Business Model Innovation
constitute a business model; and sec- concept. In particular, should the busi- Thus, business model can be a subject
ond, the extent to which business mod- ness model focus on value creation or of innovation in itself, pursuing novel
els can be planned in advance. also incorporate the notion of value forms of value creation and captur-

ju ly 2 0 1 2 | vo l. 55 | n o. 7 | c om m u n i c at ion s of t he acm 23
viewpoints

ing mechanisms. Even without novel for coining the term “creative destruc-
technology, new business models can tion” in economics.
be central in their own right to ensure Even without novel
business success. For example, Dell technology, new Conclusion
did not focus on improving the person- Business model is a term much used
al computer, but innovated in an as- business models can but seldom defined explicitly. The
pect of its business model, namely sup- be central in their next time someone asks you “what
ply chain and distribution systems, to is the business model?” you know,
deliver compelling value to end users. own right to ensure at a minimum, to refer to the way
A usage-based auto insurance business success. your enterprise creates and delivers
scheme, called Pay As You Drive value to customers. Some strategists
(PAYD) developed by Progressive would also want to know the man-
Corporation, is a good example of a ner in which the enterprise captures
business model that has undergone a value and converts it into profit. If
process of innovation and adaptation this notion is adopted, designing and
over a decade.a Based on a family of intellectual property management, implementing business models is the
business method patents, this insur- marketing, procurement, and finance. essence of strategy, to ensure sustain-
ance method is part of a new business able competitive advantage.
model that empowers drivers to con- What Causes Disruption Despite some definitional ambigu-
trol their consumption (customer val- in Business Models? ity, business model remains an impor-
ue proposition), serves latent markets Business model innovation can be tant notion precisely because of its in-
for consumers previously reluctant to subjected to continuous and incre- tegrative nature. Unlike technological
insure because premiums were too mental changes, as previously men- innovation led by the R&D department,
high (market segment identification), tioned, but it may also have the poten- business model innovation requires
and fundamentally restructures the tial capacity for disruption. So, what cross-functional mechanisms for creat-
revenue model through lower premi- are the sources of disruption in busi- ing (and capturing) new value for users.
um levels but also lower claims. Back ness model innovation? Moreover, business models are at
in 1999, PAYD used GPS satellites Disruptions occur, of course, as a re- the heart of innovation in distinctive
to determine when, where, and how sult of technological progress such as ways. First, new technologies create
much an insured vehicle was driven in information and communications opportunities for new business mod-
in order to determine the premium. technology. They also occur as a re- els. Second, appropriate business
High technology costs and policy- sult of identifying new latent markets. models are necessary to translate tech-
holders’ perception of privacy inva- Within the developed world, compa- nical success into commercial success.
sion posed challenges to implement- nies in industries as diverse as airlines, Third, business models themselves
ing the business model. Progressive automobiles, banking, and media have are subject to innovation involving dis-
worked in partnership with insurance seen their markets invaded by new and continuous changes in the paradigm
telematics suppliers to lower the cost disruptive business models. New en- used by firms to go to market. In this
of monitoring devices. Progressive trants, such as Southwest Airlines and sense, the ability to sense deep truths
also took a long-term view in educat- EasyJet in air travel, have captured mar- about what consumers really want, to
ing drivers via the PAYD portal, in or- ket share by targeting distinctive mar- satisfy consumers’ unmet needs, is
der to achieve a right balance between ket segments. More recently, low-cost perhaps the most important driver of
the perceived cost of privacy invasion disrupters hail from emerging markets, business model innovation.
and the benefits of greater control such as Tata Motors from India with the
over driving behavior and hence in- Nano in automobiles and Galanz from References
1. Chesbrough, H. Open Business Models: How to Thrive
surance cost. Thus, the ecosystem China with microwave ovens. in the New Innovation Landscape. Harvard Business
for a business model takes time to What is common across these new School Press, Boston, MA, 2006.
2. Ghaziani, A. and Ventresca, M.J. Keywords and
develop, not only in terms of the de- players is their ability to produce value- cultural change: Frame analysis of business model
velopment of affordable supporting for-money products or services for low- public talk, 1975–2000. Sociological Forum 20, 4
(2005), 523–559.
technologies, but also in terms of the end markets that have been hitherto 3. Hart, S.L. and Christensen, C. The great leap: Driving
emergence of educated consumers underserved and latent.3 Ultimately, innovation from the base of the pyramid. MIT Sloan
Management Review (Fall 2002), 51–56.
and regulatory bodies. what turns such low-cost innovation 4. Teece, D. Business models, business strategy, and
As firms experiment with novel into a disruptive business model is innovation. Long Range Planning 43 (2010), 172–194.
5. Zott, C. and Amit, R. Business model design: An
mechanisms for value creation and the new players’ ability to create new activity based perspective. Long Range Planning 43
(2010), 216–226.
capture, they must bring to bear cross- markets with new value propositions— 6. Zott, C. et al. The business model: Recent
functional capabilities in technology, offering high tech and niche market developments and future research. Journal of
Management 37, 4 (2011), 1019–1042.
products at low cost—for customers.
Thus, in identifying new markets, we
a This case is based on Panos Dessyllas and Mari Mari Sako ([email protected]) is Professor of
Sako, “Profiting from Business Model Innova-
are back to basics, to one of the fun- Management Studies at Saïd Business School, University
damental principles of innovation ac- of Oxford, U.K.
tion: Evidence from Pay-As-You-Drive Auto In-
surance,” Research Policy, 2012, forthcoming. cording to Joseph Schumpeter, known Copyright held by author.

24 c ommunic atio ns o f th e ac m | j u ly 201 2 | vol . 5 5 | no. 7

 V
viewpoints

doi:10.1145/2209249.2209260 Pamela Samuelson

Legally Speaking
Can Online Piracy
Be Stopped by Laws?
Considering the legal responsibilities of Internet intermediaries
in the aftermath of the Stop Online Privacy Act controversy.

W
hile on a scuba diving
trip in the Seychelles
Islands earlier this
year, I found myself
worrying about pirates.
Real pirates, as in people who attack
boats, take hostages, and sometimes
kill their prey. This kind of piracy has
become unfortunately common in
that part of the world.
On board our ship were four for-
mer British special forces soldiers who
served as security guards. They were
armed with semiautomatic weapons
and on patrol, 24/7, for the entire trip.
The danger was not just hypothetical.
The frigate berthed next to us as we
boarded had 25 pirates in its brig.
Waking up every morning to the
prospect of encountering real pirates
added brio to our excursion. It also in-
duced reflections on use of the word
“piracy” to describe copyright infringe-
ments. Downloading music is really
not in the same league as armed at-
tacks on ships. A Stop Online Piracy Act protest rally in New York City in January, 2012.
As we were cruising from Mahe
to Aldabra, I expected to be far away industry thought were either indiffer- This column will explain the key fea-
from it all. But the ship got a daily fax ent or acquiescent to storage of in- tures of SOPA, why the entertainment
of the main stories being published in fringing materials. industry believed SOPA was necessary
the New York Times. Among them were For a time, it seemed virtually inevi- to combat online piracy, and why SOPA
stories about the controversy over the table that SOPA would become law. Yet came to be perceived as so flawed that
Photogra ph by A l a in-Ch rist ia n

proposed legislation known as the because strong opposition emerged numerous sponsors withdrew their
Stop Online Piracy Act (SOPA). SOPA from technology companies, computer support from the bill.
would have given the entertainment security experts, civil liberties groups
industry new legal tools to impede ac- and members of the general public, Blocking Access to
cess to foreign “rogue” Web sites that SOPA has been put on hold. It is un- “Foreign Rogue Web Sites”
host infringing content and to chal- likely to be enacted in anything like its As introduced, SOPA would have em-
lenge U.S.-directed Web sites that the original form. powered the Attorney General (AG) of

ju ly 2 0 1 2 | vo l. 55 | n o. 7 | c om m u n i cat ion s o f t he ac m 25
viewpoints

the U.S. to seek court orders requir- a plug-in or directory that mapped U.S. property” shortly after receiving
ing foreign Web sites to cease provid- blocked URLs with numerical DNS such letters.
ing access to infringing copies of U.S. representations). Payment providers and Internet ad-
works. Because “rogue” Web sites Frustrated by the weak enforcement vertising services were then tasked with
seemed unlikely to obey a U.S. court of intellectual property rights (IPRs) notifying the challenged sites about
order, SOPA further empowered the abroad, the U.S. entertainment indus- the “dedicated-to-theft” allegations
AG to serve these orders on U.S. Inter- try urged Congress to adopt SOPA as against them. Challenged sites could
net intermediaries who would then the best way to impede online infringe- contest these allegations by sending
have been required to take “technical- ments. Foreign rogue Web sites might counter-notices to the payment provid-
ly feasible and reasonable measures” still be out there, but if U.S.-based In- ers and Internet advertising services.
to block their users from accessing ternet intermediaries blocked access But without a counter-notice, payment
the foreign Web sites. This included to the sites, users would not be able to providers and Internet advertising ser-
“measures designed to prevent the do- access infringing materials through vices had to cease further dealings with
main name of the foreign infringing U.S. intermediaries. the challenged Web sites.
site…from resolving to that domain Because ISPs in the U.S. and abroad Content owners could also sue ded-
name’s Internet protocol address.” have no duty to monitor what users icated-to-theft sites directly to enjoin
These measures needed to be under- do on their sites, it is easy for sites to them from undertaking further ac-
taken “as expeditiously as possible,” become hosts of large volumes of in- tions that evidenced their dedication
but no later than five days after receipt fringing materials. Some operators to theft. SOPA also authorized content
of the orders. seemingly turn a blind eye to infringe- owners to sue payment providers or ad-
Upon receiving a copy of a rogue- ment, some encourage posting of vertising services who failed to comply
Web site order, search engines would infringing content, while other sites with demands that they cease dealing
have been required to block access to may just be misused by infringers. By with challenged Web sites.
the sites even if users were searching cutting off sources of transactional
for items that would otherwise have and advertising revenues, the hope SOPA’s Flaws
brought the sites to their attention. In- was to discourage these sites from The main problems with SOPA insofar
ternet service providers would have had continuing to operate. as it would have employed DNS block-
to ensure that users who typed certain ing to impede access to foreign rogue
URLs (for example, http://thepiratebay. Challenging U.S.-Directed Web Sites Web sites were, first, that it would un-
se) into their browsers could not reach SOPA would also have given holders of dermine the security and stability of
those sites. Payment providers (such as U.S. intellectual property rights (IPRs) the Internet as a platform for commu-
Visa or Mastercard) would have had to power to challenge “U.S.-directed sites nication and commerce and second,
suspend services for completing trans- dedicated to the theft of U.S. property.” that it would be ineffective.
actions. Internet advertising services At first blush, it might seem that rea- SOPA is fundamentally inconsis-
would have had to discontinue serving sonable persons should support a law tent with DNSSEC (DNS Security Ex-
ads and providing or receiving funds crafted to target such sites. But “dedi- tensions), a protocol developed to
for advertising at these sites. cated to the theft of U.S. property” was avoid abusive redirections of Internet
Those who failed to comply with defined in a troublingly ambiguous traffic, whether by criminals, autocrat-
the DNS blocking obligations could and overbroad way. It included opera- ic governments, or other wrongdoers.
expect the AG to sue them. The AG tors of sites that were taking “deliber- Computer security experts spent more
was also empowered to sue those who ate actions to avoid confirming a high than a decade developing DNSSEC,
provided a service designed to circum- probability of the use of [that] site to which is now being implemented all
vent this DNS blocking (for example, carry out acts” in violation of copyright over the world, including by U.S. gov-
or anti-circumvention rules. Also in- ernment agencies.
cluded was any site that was “primarily As the USACM Public Policy Com-
The safe harbors designed or operated for the purpose mittee observed in a letter sent to mem-
of, ha[d] only a limited use other than, bers of Congress, DNSSEC Web site
have been an or [wa]s marketed by its operator or operators cannot reliably block offend-
important factor in another acting in concert with that op- ing sites “and so may be faced with the
erator in, offering goods or services in choice of abandoning DNSSEC or being
the extraordinary a manner that engages in, enables, or in violation of issued court orders.”
growth of the facilitates” violations of copyright or This letter explained why DNS
anti-circumvention laws. blocking would be ineffective. “[I]t is
Internet economy. SOPA would have enabled firms effectively impossible to bar access to
who believed themselves to be harmed alternate DNS servers around the globe
by one of these sites to send letters to because there are millions of them on
payment providers and/or to Internet the Internet.” Use of those servers “al-
advertising services to demand that lows for bypassing of DNS blocking.”
they cease providing services to sites Circumvention of DNS blocking is,
alleged to be “dedicated to the theft of moreover, “technically simple and uni-

26 communicatio ns o f the ac m | j u ly 201 2 | vol . 5 5 | no. 7

 viewpoints

versally available.” Browser add-ons to

avoid DNS blocking have already been
Because SOPA
Calendar
developed and would be available on
servers outside the U.S., even if illegal
in the U.S.
was a flawed piece of Events
The main problems with the ded- of legislation, July 15–16
icated-to-theft provisions of SOPA the collapse was Extreme Scaling Workshop,
Chicago, IL,
were, first, that it was too imprecise
and second, that it represented a dra- a good thing. Contact: Bill Kramer,
Email: [email protected]
matic change in the rules of the road
affecting Internet intermediaries. July 15–19
26th ACM/IEEE/SCS Workshop
What does it mean, for instance,
on Principles of Advanced and
for an Internet intermediary to take Distributed Simulation,
“deliberate actions to avoid confirm- Zhangjiajie, China,
ing a high probability” of infringement Sponsored: SIGSIM,
Contact: Jason Liu,
on the site? If Viacom tells YouTube it Internet service providers (including Email: [email protected]
has found infringing clips of “South Wikipedia, which went “dark” one day
Park” shows on its site, does YouTube to protest SOPA), computer security ex- July 15–20
become a site dedicated to the theft of perts, civil society groups, and millions International Symposium
on Software Testing and
Viacom’s property if it does not inves- of Internet users who contacted their Analysis,
tigate these claims? If Universal Music Congressional representatives to voice Minneapolis, MN,
Group objects to the resale of MP3 files opposition to the bill. Sponsored: SIGSOFT,
of its music on eBay, does eBay become Because SOPA was a flawed piece Contact: Mats Heimdahl,
Email: [email protected]
a site dedicated to theft of Universal’s of legislation, the collapse was a good
property because one or more of its us- thing. It would, however, be a mistake July 16–18
ers offer the MP3 files for sale there? to think the battle over Internet inter- ACM Symposium on Principles
Many Internet companies consid- mediary liability for infringing acts of of Distributed Computing,
Funchal, Portugal,
ered the dedicated-to-theft defini- users has been won for good. Sponsored: SIGOPS,
tion to be fundamentally inconsistent The entertainment industry is al- Contact: Dariusz Kowalski,
with the safe harbors established by most certainly going to make further Email: [email protected]
the Digital Millennium Copyright Act efforts to place greater legal respon-
July 16–20
(DMCA). Under the DMCA, Internet in- sibilities on Internet intermediaries. the 6th ACM International
termediaries are obliged to take down This industry believes intermediar- Conference on Distributed
infringing materials after they are no- ies are the only actors in the Internet Event-based System,
Berlin, Germany,
tified about specific infringements at ecosystem who can actually affect the Sponsored: SIGSOFT,
specific parts of their Web sites. They level of online infringements that Contact: Francois Bry,
have no obligation to monitor their contributes to entertainment indus- Email: [email protected]
sites for infringement. The safe har- try panics.
July 16–20
bors have been an important factor in An odd thing about the entertain- User Modeling, Adaptation
the extraordinary growth of the Inter- ment industry is its deeply skewed and Personalization,
net economy. views about piracy. In movies such as Montreal, Canada,
It may be apt to characterize sites Pirates of the Caribbean, the industry Contact: Nkambou Roger,
Email: [email protected]
such as Napster, Aimster, and Grokster glamorizes brigands who attack ships
as having been dedicated to the theft of by depicting them as romantic he- July 22–25
U.S. intellectual property, but existing roes who have great adventures and International Symposium
copyright law supplied copyright own- engage in swashbuckling fun. Yet, it on Symbolic and Algebraic
Computation,
ers with ample tools with which to shut demonizes fans who download music Grenoble, France,
down those sites. and movies as pernicious evildoers Contact: Joris van der Hoeven,
Had the entertainment industry who are, in its view, destroying this Email: vdhoeven@lix.
polytechnique.fr
sought more narrowly targeted rules vital part of the U.S. economy. Some-
aimed at inducing payment providers thing is amiss here, and it is contrib- July 24–27
and Internet advertising services to uting to a profound disconnect in per- International Conference
stop the flow of funds to sites that were spectives about how much the law can on e-Business,
Rome, Italy,
really dedicated to infringement, such do to bring about changes in norms
Contact: Prof. Mohammad S.
a law might have passed. But that was about copyright. Obaidat,
not SOPA. Email: [email protected]
Pamela Samuelson ([email protected]) is the
Conclusion Richard M. Sherman Distinguished Professor of Law and
Information at the University of California, Berkeley.
The collapse of support for SOPA was
principally due to concerted efforts by Copyright held by author.

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he ac m 27
V
viewpoints

doi:10.1145/2209249.2209261 R.T. Watson, J. Corbett, M.C. Boudreau, and J. Webster

Computing Ethics
An Information Strategy for
Environmental Sustainability
Seeking solutions to a problem of change.

T
h ro ug h t h re e ce ntu ri e s of
industrialization, humans
have changed the chemical
composition of the air, land,
rivers, and oceans. The es-
sentials of life for the majority of the
Earth’s flora and fauna have experi-
enced profound alterations, and these
species-threatening transformations
show signs of accelerating unless we
take action to change direction. Many
of the proposed solutions, such as car-
bon sequestering, have an engineering
orientation and are based on current
technologies.7 Often the suggested
remedies do not adequately recognize
the potential of information systems
to greatly increase energy efficiency12
or to influence individual and orga-
nizational behaviors. This is a critical
oversight. Computer-based informa-
tion systems have been the driving
force for productivity improvements10
in the last five decades. We contend
that, in a similar way, information sys-
tems can be a driving force for sustain-
ability improvements. Our quest for
environmental sustainability needs
an information strategy to parallel and
complement engineering solutions:
ACM members could and should play
a critical role in creating and imple-
menting an information strategy.
Computing professionals and aca-
Illustration by Ya rek waszu l

demics have many of the key skills

needed to help solve the complex tensive knowledge of the design and through action and research focusing
problems surrounding sustainabil- implementation of systems. In this on several imperatives at the heart of
ity. This community is highly innova- column, we want to stimulate the an information strategy. These imper-
tive, focused on efficiency gains and computing profession to address en- atives arise from two views about hu-
the customer experience, and has ex- vironmental sustainability collectively man behavior—rational and social—

28 comm unicatio ns o f th e ac m | j u ly 201 2 | vo l. 5 5 | no. 7

 viewpoints

that influence how we make decisions vironmental consequences of those

about sustainability and many other decisions. Information, which helps
aspects of life. The rational view, as Many people to form perceptions, is critical to the
typically espoused in economics, takes want to maintain functioning of our social as well as ra-
two perspectives: humans are entirely tional side. Three streams of informa-
motivated by narrow self-interest and a sustainable tion can influence perceptions about
the pursuit of utility maximization6; lifestyle, but sustainability: organizational sustain-
and firms should focus exclusively ability reporting, product informa-
on maximizing shareholder wealth.1 they do not tion, and feedback on individual envi-
Individual rationality, however, does know how. ronmental impact.
not always lead to collective rational- If consumers and enterprises want
ity. The tragedy of the commons4 and to favor environmentally responsible
the paradox of thrift8 illustrate how organizations, where can they find
individual self-interest can lead to the objective and reliable information to
depletion of a community resource identify such organizations? Inves-
and how individual austerity in times tors can turn to audited financial re-
of recession exacerbates the problem, tion systems that economists and leg- ports, but what if people and organi-
respectively. Social forces, such as the islators could use for setting fees and zations want to invest in the future of
desire to build long-term relation- establishing regulations and social the Earth by favoring environmentally
ships, dampen self-interest.5 Social interventions to ensure that prices conscious suppliers and firms, and
constraints and interventions, such as reflect externalities. One solution is ecologically sound products? Regula-
culture, laws, and ethical codes, can extensive sensor networks for record- tions for sustainability reporting are
rein in naked self-interest to create ing the location and characteristics of emergent. At the global level, there
outcomes that are more attuned to the pollution. Then we would have the raw are now about a dozen standards,
collective interest. material for accurate pricing. Thus, a most of which are voluntary and not
first imperative for the profession is necessarily audited, with the Global
The Role of Information in to: Create information systems and net- Reporting Initiative (GRI)2 the most
Rational and Social Behavior works that provide the capacity to incor- widely used. Voluntary reporting can
The economic organization of current porate significant environmental costs work against the attainment of sus-
society predominantly reflects the op- into prices. tainability by hampering the imple-
eration of self-interest. By the late 20th This pricing perspective favors one mentation of other mechanisms
century, most countries had decided view of how to transition to an environ- that could be more effective, such as
that the allocation of many scarce re- mentally sustainable society. Treating stricter regulations. Consequently, in
sources, a fundamental economic people and organizations as autono- 2010, GRI issued a declaration calling
problem, should be assigned to prices, mous maximizing utilitarians, how- on governments to require companies
markets, and free enterprise. The glob- ever, fails to recognize that our innate to report on environmental and social
al economy largely operates through social nature and non-economic pri- factors. Compulsory reporting will set
prices, particularly for commodities orities greatly influence our decisions, the stage for our profession to enter
(such as coal and iron), which are ma- needs, and behaviors, and that we are the fray because we excel at the explo-
jor factors in the price of nearly all con- strongly influenced by others. Behav- ration, collection, presentation, and
sumer products and services. From a ioral economics addresses some of dissemination of the information con-
sustainability and societal perspective, the shortcomings of traditional indi- tained in organizational sustainability
prices are not always an effective signal vidual self-interest oriented economic reports that can influence behavior.
because of the presence of externali- analysis. Solutions to sustainability, The profession has already contrib-
ties, which represent costs absorbed particularly information-driven strat- uted extensively to the development of
by society rather than the producer. For egies, need to also consider our social organizational financial reporting sys-
example, the costs of CO2 emissions side. Our environment is the result of tems to make markets more efficient.
of coal-fired power stations are borne what we consume now and in the fu- Another imperative before us is to: De-
by everyone, irrespective of how much ture. Hence, societal transition ulti- sign corporate sustainability reporting
electricity they consume. When soci- mately hinges on consumers—as indi- systems that achieve the goal of a more
ety allows this externalization of such viduals and organizations—changing sustainable society.
costs, markets and self-interest work their consumption patterns for the If consumers want to purchase
against sustainability,3 which is why greater good of the environment and products that advance sustainability,
some advocate internalization of exter- not just because of a product’s price. where can they find reliable informa-
nalities.9 Thus, the economic system Many organizations and individuals tion? Governments require labeling
would better serve the greater social have a positive attitude toward the en- information to provide data on the nu-
good if prices were aligned with sus- vironment and seek to make sustain- tritional value of food and the chemi-
tainability goals. able choices, yet it is often difficult for cals in drugs. This practice could be
At present, we do not have in place them to follow these beliefs because extended to sustainability. Informa-
data streams and associated informa- they lack information about the en- tion has far more value when it can be

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 29
viewpoints

digitally processed, and what is need- for Information Systems special inter-
ed is a product database containing est group on green information sys-
the environmental calculations for the What matters now tems. It matters that we take action,
vast majority of commonly consumed is that we stimulate and as we learn, we will refine our
items so consumers can readily under- notions of an effective information
take green comparisons of products. debate within the strategy and sound tactical solutions.
Computing professionals can play a profession about John Holdren, President of the Ameri-
key role in defining the data standards can Association for the Advancement
for consumer environmental calcula- its role in creating of Science and advisor to U.S. Presi-
tions and then designing systems for a sustainable society. dent Barack Obama for Science and
making this data publicly and conve- Technology, called on his colleagues
niently accessible. We can help con- to tithe 10% of their time to working
sumers choose greener products and on the globe’s significant problems;
understand the environmental conse- it might well be appropriate for ACM
quences of their purchasing decisions. to make a similar appeal to its mem-
Consequently, another imperative for bership. In summary, ACM members,
the profession is to: Implement efficient both collectively and individually, must
approaches for collecting and persuasive Thus, in designing and developing an apply their computing knowledge to con-
means of presenting product sustain- information strategy for sustainabil- tribute to the creation and implementa-
ability information to promote green ity, we must take into account the full tion of an information strategy for a sus-
purchasing decisions. life cycle impact of these solutions and tainable society.
Many people want to maintain a their own intrinsic demands on the
sustainable lifestyle, but they do not environment for materials and energy References
1. Friedman, M. The Social Responsibility of Business is to
know how. For example, the house- consumption. Critically, we need to: Increase its Profits. Springer Berlin Heidelberg, 2007.
holder who wants to do the laundry Develop professional standards for data 2. Global Reporting Initiative. Sustainability Reporting
Guidelines: Version 3.0. 2006.
with electricity primarily from renew- processing and storage that minimize 3. Grunert, K.G. and Thøgersen, J. Consumers, Policy and
able sources typically lacks relevant their environmental consequences, while the Environment: A Tribute to Folke Ölander. Springer,
2005.
information. Furthermore, even when simultaneously helping to create a sus- 4. Hardin, G. The tragedy of the commons. Science 162,
some information is available, it is of- tainable society. 3859 (1968), 1243–1248.
5. Lawrence, P.R. Being human: A Darwinian theory
ten aggregated so that individual ac- of human behavior. Unpublished draft, 2007;
tion is hidden in collective behavior. A Call for Action http://prlawrence.com/beinghumandownload.html%3E.
6. Milgrom, P.R. and Roberts, J. Economics, Organization,
For most, the monthly electricity bill Large-scale endeavors, such as re- and Management. Prentice-Hall, Englewood Cliffs,
omits the necessary details to promote ducing the effects of global climate N.J., 1992.
7. Pacala, S. and Socolow, R. Stabilization wedges:
change. Consumers would be better change, require the incremental and Solving the climate problem for the next 50 years with
informed if each action were accompa- cumulative action of many working to- current technologies. Science 305 (2004), 968–972.
8. Samuelson, P.A. and Nordhaus, W.D. Economics.
nied by information about its environ- ward a common outcome. Like those McGraw-Hill Irwin, Boston, 2010.
9. Stern, N. The Economics of Climate Change: The Stern
mental effects. Therefore, as a profes- in other fields,11 we believe it is our Review. Cambridge University Press, 2007.
sion, we need to: Develop information ethical imperative to address environ- 10. Stiroh, K.J. Information technology and the U.S.
productivity revival: What do the industry data say?
systems that provide individuals with mental sustainability issues. Collec- American Economic Review 92, 5 (2002), 1559–1576.
accurate, meaningful, and actionable tively, we can start on this path by first 11. Swim, J. K. et al. Psychology’s contributions to
understanding and addressing global climate change.
information about the environmental tackling current tractable issues, such American Psychologist 66, 4 (2011), 241.
impact of personal decisions. as designing and building local sensor 12. Watson, R.T., Boudreau, M.-C., and Chen, A.J.W.
Information systems and environmentally sustainable
In summary, accurate pricing and networks, and then scaling up as we development: Energy informatics and new directions
well-informed perceptions among all learn how to create a global network of for the IS community. MIS Quarterly 34, 1 (2010),
23–38.
members of society—individual, or- linked sensor networks, using the In-
ganizational, governmental—are the ternet as both a platform and a model.
Richard T. Watson ([email protected]) is the J. Rex
foundations of an information strat- Similarly, the behavioral scientists in Fuqua Distinguished Chair for Internet Strategy in the
egy for environmental sustainability. our community can begin exploring Department of MIS at the Terry College of Business at
the University of Georgia, Athens, GA.
Such an information strategy requires the relationship between information,
processing and storing more informa- perceptions, and environmental con- Jacqueline Corbett ([email protected]) is
an assistant professor in the Department of Management
tion (along with studying its reception sequences, and scale as they learn. Information Systems in the School of Business
by consumers). Sensor networks and What matters now is that we stim- Administration at Laval University in Quebec City, Canada.

product sustainability information will ulate debate within the profession Marie-Claude Boudreau ([email protected]) is
add new streams of data that we need about its role in creating a sustainable an associate professor in the Department of MIS at the
Terry College of Business at the University of Georgia,
to manage securely and sustainably in society. Despite differences between Athens, GA.
a world already experiencing a data del- ACM members, there is ongoing work
Jane Webster ([email protected]) is
uge. We certainly do not want a situa- on which we can build—see the work the E. Marie Shantz Professor of Management
tion where the energy required for stor- on computational sustainability of the Information Systems at Queen’s University in Kingston,
Ontario, Canada.
ing and processing this data results in Computing Community Consortium
a net increase in harmful emissions. as well as the work of the Association Copyright held by author.

30 communic atio ns o f the ac m | j u ly 201 2 | vol . 5 5 | no. 7

V
viewpoints

doi:10.1145/2209249.2209277 Martin Campbell-Kelly

Historical Reflections
Alan Turing’s Other
Universal Machine
Reflections on the Turing ACE computer and its influence.

A
ll c o mput e r s c i e n t i s t s
know about the Univer-
sal Turing Machine, the
theoretical construct the
British genius Alan Turing
described in his famous 1936 paper on
the Entscheidungsproblem (the halting
problem). The Turing Machine is one
of the foundation stones of theoreti-
cal computer science. Much less well
known is the practical stored program
computer he proposed after the war
in February 1946. The computer was
called the ACE—Automatic Comput-
ing Engine—a name intended to evoke
the spirit of Charles Babbage, the pio-
neer of computing machines in the
previous century.
Almost all post-war electronic com-
puters were, and still are, based on the
famous EDVAC Report written by John
von Neumann in June 1945 on behalf The Pilot ACE, May 1950. Jim Wilkinson (center right) and Donald Davies (right).
of the computer group at the Moore
School of Electrical Engineering at the roles. Computers were in the air and of one, and over the next few months
University of Pennsylvania. Von Neu- universities at Manchester, Cam- he evolved the design of the ACE. His
mann was very familiar with Turing’s bridge, and elsewhere established report was formally presented to the
1936 Entscheidungsproblem paper. In electronic computer projects. Out- NPL’s executive committee in Febru-
1937, Turing was a research assistant side the academic sphere, in Lon- ary 1946.
at the Institute for Advanced Study at don, the National Physical Laboratory Although the ACE drew heavily on
Princeton University, where von Neu- (NPL—Britain’s equivalent of the Na- the EDVAC Report, it had many novel
mann was a professor of mathematics. tional Bureau of Standards) started features and was a design of great
Beyond these known facts, however, it a mathematics division to provide a originality. When he heard about the
is not possible to say how much influ- computing service for industry. With EDSAC that was being planned at
ence the Turing Machine had on the a staff of 40 and equipped with desk Cambridge University by Maurice Wil-
design of the EDVAC. But there is no calculating machines, punched card kes, which was based very directly on
doubt the ACE was heavily influenced equipment, and a differential analyz- the EDVAC, Turing was unimpressed.
by the EDVAC Report. er, it was an impressive organization. He wrote in a private note: “The ‘code’
The war in Europe ended in May In October 1945, Turing was brought which [Wilkes] suggests is however
1945 and the institutions of Britain in to establish an electronic com- very contrary to the line of develop-
began to get back to their peacetime puter project. He was a department ment here, and much more in the

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | com m u n i cat ion s o f t he acm 31

viewpoints

ACE Design
The ACE was designed to fully exploit
Turing’s brilliant the potential of a mercury delay-line
innovation in the memory. The delay line was a spin-off
from echo-cancellation devices in ra-
ACE was to eliminate dar, and in 1946 it was the only proven
delay-line waiting memory technology. When used in the
main memory of a computer, the delay
time by what was line consisted of a five-foot mercury-
later called optimum filled steel tube. Digital data was con-
verted to acoustic energy at one end of
programming. the tube and converted back to electri-
cal signals at the other end. By feeding
the output back into the input, digi-
tal data could be stored indefinitely.
A five-foot tube had an acoustic de-
lay of about one millisecond and the
typical pulse duration of a computer
American tradition of solving one’s was one microsecond; hence the de-
problems by means of much equip- lay line could store about 1,000 bits
ment rather than by thought. I should of information (usually 1,024 bits, of
imagine that to put his code (which is course). This was enough to store sev-
advertised as ‘reduced to the simplest eral instructions or numbers. Apart
possible form’) into effect would re- from its high cost and unreliability,
quire a very much more complex con- the mercury delay line memory also
trol circuit than is proposed in our full- had the problem of latency. That is,
ACM’s size machine.” when an instruction or number was
Wilkes took the view that build- needed, the machine had to wait un-
interactions
ing the EDSAC to a conventional de- til it emerged from the delay line—an
magazine explores sign would be enough of a challenge. average time of half a millisecond.
critical relationships Moreover, the EDSAC would be several For this reason most delay-line based
between experiences, people, thousand times faster than existing computers spent more time waiting
techniques and users would be best for instructions and numbers than
and technology, showcasing served by having a conventional ma- processing them.
emerging innovations and industry chine sooner rather than a novel de- Turing’s brilliant innovation in
leaders from around the world sign later. There was merit in both of the ACE was to eliminate this waiting
their positions. time by what was later called optimum
across important applications of
design thinking and the broadening
field of the interaction design.
Our readers represent a growing
community of practice that
is of increasing and vital National A rch ive for th e H istory of Comp ut ing, M anch ester Universit y

global importance.
e
ib
cr
s
ub
/s
rg
.o
cm
a
w.
w
w
://
tp
ht

Control desk of the English Electric DEUCE, 1955.

32 c omm unicatio ns o f th e ac m | j u ly 201 2 | vo l. 5 5 | no. 7

 viewpoints

programming. On a conventional eral months. Then in 1947 an Ameri-

machine instructions were stored in can, Harry Husky, who had been a
consecutive locations, so that latency member of the Moore School com-
effects dramatically slowed up the puter group, began a sabbatical year
machine. In the ACE design, however, at the NPL. He proposed building a
each instruction specified the location small prototype of the ACE, which he
of its successor. In this way instruc- called the Test Assembly. Still, little
tions (and numbers) could be placed progress was made and Turing left
optimally in the memory so that they the NPL in September 1947 for a sab-
emerged from a delay line just as they batical year at King’s College, Cam-
were needed. The optimum coding bridge University. He never returned
idea showed a rather direct connec- to the NPL. At last in early 1949, work
tion to the “instruction tables” of the really got going on another small ver-
Turing Machine, in which each or- sion known as the Pilot ACE and the
der specified the next to be obeyed. development logjam was broken by
This simple idea alone would make Donald Davies, a new recruit to the
the ACE about three times faster than NPL. Davies was an outstanding sci-
other delay-line based computers. entist and administrator. He later
Optimum coding, it should be noted, became head of computer science at A brochure for the Bendix G-15 computer,
was much more difficult than conven- the NPL and was one of the inventors introduced in 1956.
tional programming, but for a decade of packet switching technology.
it was the way to get maximum perfor- The Pilot ACE sprang into life in tives. The most important was the
mance from delay-line or drum-based May 1950. It had just 10 delay lines, but Bendix G-15 introduced in 1956. The
machines. when augmented with a drum store it machine was designed for the West
But there was more. In the ACE de- was a highly capable computing ma- Coast’s Bendix Aviation Corporation
sign Turing avoided having a single chine. Benchmarks showed that it was by Harry Husky, who had returned to
central register (known as the accu- five to 10 times faster than contem- the U.S. and joined the faculty of the
mulator in EDVAC-type machines) in porary machines. Moreover, the Pilot University of California, Berkeley. Op-
which all computation took place. In- ACE had just 800 tubes compared with timum coding gave the machine an
stead, he had separate registers for ad- 3,000 in the Cambridge EDSAC and edge over rival machines, and the G-15
dition and subtraction, multiplication, 3,500 in the Manchester Mark I. became a workhorse for the American
logical operations, shifting, and so engineering industry. About 400 were
on. To a degree Turing was anticipat- ACE Derivatives eventually sold.
ing what would later be called the von The English Electric Company, a
Neumann bottleneck. manufacturer of airplanes and elec- Conclusion
trical equipment and a contractor to As to the ACE itself, the full-scale,
ACE Construction the NPL, decided make a copy of the 200-delay-line machine was finally
Turing proposed that the ACE would Pilot ACE; unsurprisingly, the ma- completed in 1958. Sadly, by the time
contain 200 delay lines in the main chine was named the DEUCE. This it was built the heyday of delay-line
memory and estimated the con- set the ball rolling—English Electric’s storage was over, and new comput-
structional cost at £11,200 (about competitors decided they would like a ers were using random access core
$45,000). Turing was a better math- machine too and the company found memory. Even the NPL admitted the
ematician than an engineer and his itself in the computer business. Sold machine was a technological dino-
estimates were wildly unrealistic. from 1955, the DEUCE was especially saur. Turing died in 1954, so he never
Building the ACE got off to a shaky popular with engineering companies lived to see either the triumph of the
Bend ix g-15 broch ure image cou rtesy of th e Co mput er Histo ry M useum

start and not much happened for sev- that had heavy computational needs. DEUCE or the whimper of the full-
Sales to aerospace companies were scale ACE. English Electric went on
given a boost following the Comet air to become a cornerstone of the early
Besides the DEUCE, disaster of 1954, which had result- British computer industry. After the
ed in the statutory requirement for DEUCE went out of production in
the Pilot ACE airplane manufacturers to conduct 1960 it was succeeded by the KDF9,
spun off several other so-called flutter calculations. The an innovative machine with a stack-
DEUCE came with a superb library based architecture—it showed the
worthy derivatives. of matrix software developed by Jim company had lost none of the design
Wilkinson, originally Turing’s assis- verve it had inherited from Turing.
tant but later a distinguished numeri-
cal analyst and winner of the 1970 Martin Campbell-Kelly (M.Campbell-Kelly@warwick.
ac.uk) is a professor in the Department of Computer
ACM A.M. Turing Award. Science at the University of Warwick, where he specializes
Besides the DEUCE, the Pilot ACE in the history of computing.

spun off several other worthy deriva- Copyright held by author.

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i cat ion s o f t he acm 33

V
viewpoints

doi:10.1145/2209249.2209262 Alfred Spector, Peter Norvig, and Slav Petrov

Viewpoint
Google’s Hybrid
Approach to Research
By closely connecting research and development Google is able
to conduct experiments on an unprecedented scale, often resulting
in new capabilities for the company.

I
n th i s Vi e w p oin t, we describe
how we organize computer sci-
ence research at Google. We
focus on how we integrate re-
search and development and
discuss the benefits and risks of our
approach. The challenge in organiz-
ing R&D is great because CS is an in-
creasingly broad and diverse field. It
combines aspects of mathematical
reasoning, engineering methodology,
and the empirical approaches of the
scientific method. The empirical com-
ponents are clearly on the upswing, in
part because the computer systems we
construct have become so large that
analytic techniques cannot properly
describe their properties, because the
systems now dynamically adjust to the
difficult-to-predict needs of a diverse
user community, and because the sys-
tems can learn from vast datasets and Google Fellow Jeffrey Dean discusses MapReduce, Google File System, and BigTable during
large numbers of interactive sessions a keynote session.
that provide continuous feedback.
We have also noted that CS is an ex- mercial systems that set new standards product needs. Recent articles, such as
panding sphere, where the core of the upon which others then build are in- those by Leifer et al.8 and Enkel et al.,6
field (theory, operating systems, and so creasingly important. illustrate related issues on how firms
forth) continues to grow in depth, while To compare our approach to re- do research and catalyze innovation.
the field keeps expanding into neigh- search with that of other companies The goal of research at Google is to
boring application areas. Research re- is beyond the scope of this Viewpoint. bring significant, practical benefits to
sults come not only from universities, But, for reference, we note that in the our users, and to do so rapidly, within
but also from companies, both large terminology of Pasteur’s Quadrant,11 a few years at most. Research happens
Photogra ph by Nial l Kennedy

and small. The way research results are we do “use-inspired basic” and “pure ap- throughout Google, exploring techni-
disseminated is also evolving and the plied” (CS) research. Buderi2 and Dodg- cal innovations whose implementation
peer-reviewed paper is under threat as son et al.5 discuss information technol- is risky, and may well fail. Sometimes,
the dominant dissemination method. ogy research generally, pointing out research at Google operates in entire-
Open source releases, standards speci- the movement in industrial labs to- ly new spaces, but most frequently,
fications, data releases, and novel com- ward research that strongly considers the goals are major advances in areas

34 comm unicatio ns o f th e acm | j u ly 201 2 | vol . 5 5 | no. 7

 viewpoints

where the bar is already high, but there the potential to impact the world both
is still potential for new methods. In through Google’s products and servic-
these cases, simply establishing the Google’s approach to es, and through the academic research
feasibility of a research idea may be a research is iterative community. We recognize that the
substantial task, but even greater effort wide dissemination of fundamental
is required to create a true success or and usually involves results often benefits us by garnering
useful negative result. writing production, or valuable feedback, educating future
Because of the time frame and ef- hires, providing collaborations, and
fort involved, Google’s approach to re- near-production, code seeding additional work.
search is iterative and usually involves from day one. In no way do we feel our model
writing production, or near-produc- precludes long-term research: we just
tion, code from day one. Elaborate re- try to “factorize” it into shorter-term,
search prototypes are rarely created, measurable components. This pro-
since their development delays the vides benefits to us in terms of team
launch of improved end-user services. motivation (based upon evidence of
Typically, a single team iteratively ex- ˲˲ The services-based delivery model concrete progress in reasonable time
plores fundamental research ideas, de- brings significant benefits to research periods) and the potential for commer-
velops and maintains the software, and and development. Even a small team cial benefit (in advance of the complete
helps operate the resulting Google ser- has at its disposal the power of many fulfillment of all objectives). Even if we
vices—all driven by real-world experi- internal services, allowing the team to cannot fully factorize work, we have
ence and concrete data. This long-term quickly create complex and powerful sometimes undertaken longer-term
engagement serves to eliminate most products and services. Design, testing, efforts. For example, we have started
risk to technology transfer from re- production, and maintenance pro- multiyear, large systems efforts (in-
search to engineering. This approach cesses are simplified. Additionally, the cluding Google Translate, Chrome,
also helps ensure the research efforts services model, particularly one where Google Health) that have important
produce results that benefit Google’s there is significant consumer engage- research components. These projects
users, by allowing research ideas and ment, facilitates empirical research. were characterized by the need for
implementations to be honed on em- ˲˲ Google has been able to hire a tal- complex systems and research (such
pirical data and real-world constraints, ented team across the entire engineer- as Web-scale identification of paral-
and by utilizing even failed efforts to ing operation. This gives us the op- lel corpora for Translate12 and various
gather valuable data and statistics for portunity to innovate everywhere, and complex security features in Chrome9
further attempts. for people to move between projects, and Health). At the same time, we have
whether they be primarily research or recently shown that even in longer-
Implications of Google’s primarily engineering. term, publicly launched efforts, we are
Mission and Capabilities unafraid to refocus our work (for exam-
Google’s mission “To organize the Hybrid Research at Google ple, Google Health), if it seems we are
world’s information and make it uni- Google’s focus on innovation, its ser- not achieving success.
versally accessible and useful,” both vices model, its large user community, Clearly, this approach benefits from
supports and requires innovation in its talented team, and the evolutionary the mainly evolutionary nature of CS
almost all CS disciplines. For example, nature of CS research has led Google research, where great results are usu-
we aim to “understand” user intent to a “Hybrid Research Model.” In this ally the composition of many discrete
and the meaning of documents, to model, we blur the line between re- steps. If the discrete steps required
translate between languages with ever- search and engineering activities and large leaps in vastly different direc-
higher fidelity, and to be able to trans- encourage teams to pursue the right tions, we admit that our primarily hill-
form content in one modality (say, im- balance of each, knowing that this bal- climbing-based approach might fail.
age) into relevant content in all others ance varies greatly. We also maintain Thus, we have structured the Google
(say, text). Google’s entire organization considerable fluidity in terms of mov- environment as one where new ideas
is focused on rapid innovation, and ing both people and projects as needs can be rapidly verified by small teams
three aspects of Google’s technology change. As such, even in areas where through large-scale experiments on
and business model support this: there is a much higher proportion of real data, rather than just debated. The
˲˲ Organizing all of the world’s in- research to engineering, the “Research small-team approach benefits from the
formation requires large amounts of Team” we have established is not as services model, which enables a few
resources. By providing a rich set of formally separate from engineering ac- engineers to create new systems and
computing abstractions and power- tivities as those in other organizations, put them in front of users. This in turn
ful processors, storage, and network- and for example runs large production enables us to conduct experiments at a
ing capabilities in our data centers, systems, too. Overall, we undertake scale that is generally unprecedented
Google has been able to gain econo- research work when we feel its sub- for research and development proj-
mies of scale and to sidestep some of stantially higher risk is warranted by ects. One consequence is that many
the complexity of heterogeneous com- a chance of more significant potential projects can directly affect billions of
puting environments. impact. Additionally, research also has users. This naturally influences how re-

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 35
viewpoints

searchers choose to spend their time, product team to achieve its goals. The
balancing the opportunity to have im- most successful high-profile examples
pact through Google’s services with the of this pattern are systems infrastruc-
opportunity to have impact in the aca- ture projects such as MapReduce,4
demic community. Google encourages Google File System,7 and BigTable.3
both kinds of impact, and some of the 2. A project in the research group that
most successful projects achieve both. results in new products or services. The
We thus define our hybrid research second pattern is research followed by
model as one that aims to generate the operation of the production service
scientific and engineering advances in based on that research. Both Google
fields of import to Google; that does so Translate and Voice Search10 are ex-
in a way that tends to factorize longer amples of this pattern, where the cloud
projects (perhaps with very challeng- computing infrastructure enabled
ing goals) into discrete, achievable small research teams to build systems
steps (each of which may be of com- that could be deployed. This pattern
mercial value); where we maximally applies best when continuing research
leverage our cloud computing mod- can further improve and extend the re-
els and large user base to support in sulting products.
vivo research; where we allow for the 3. A project in the research group
maximal amount of organizational that creates new concepts and technolo-
flexibility so we can support both proj- gies, which are then applied to existing
ects that require some room to grow products or services. The third pattern
unfettered by current constraints and is a traditional research and develop-
projects that require close integration ment model. Google’s success with
with existing products; and where we this model of research benefits from
emphasize knowledge dissemination the services model and from the em-
using a flexible collection of different phasis on data-driven evaluation. For
approaches. instance, some new audio and video
fingerprinting techniques,1 which re-
Example Research Patterns searchers were able to demonstrate
1. An advanced project in a product-fo- not only on small test cases, but on real
cused team that, by virtue of its creativity data at production scale, were then
and newness, changes the state of the art productized by YouTube engineers.
and thereby produces new research re- 4. A joint research project between an
sults. The first and most prevalent pat- engineering team and the research group
tern exemplifies how blurry the line be- that is then used by that engineering
tween research and development work team. The fourth pattern is a collabora-
can be. Operating at large scale, engi- tive integration of research and devel-
neering teams are often faced with nov- opment teams. Many of our products
el challenges which, when overcome, require novel algorithmic solutions to
constitute research results. Organiza- support high performance, thus pos-
tionally, research is done in situ by the ing a blend of research and engineer-
ing challenges. An example for this
pattern is the work done by our Market
Our hybrid approach Algorithms group in collaboration with
teams working on our advertisement
to research enables systems. Together, they design, modi-
us to conduct fy, and analyze the core algorithms and
economic mechanisms used for ad se-
experiments at a lection and optimization.
scale that is generally 5. A research project in an engineering
team that is transitioned to the research
unprecedented for group (and eventually becomes (2.), (3.),
research projects. or (4.) here). The fifth pattern, transi-
tioning a project from an engineering
team to the research team is an impor-
tant mechanism for giving a project
more time or resources, when the work
is important more broadly than for a
specific engineering team. An exam-

36 c ommunic atio ns o f th e ac m | j u ly 201 2 | vol . 5 5 | no. 7

 viewpoints

ple of this pattern is work on YouTube reducing the risk of getting blindsided nical leaders, and our user community.
recommendations, which started in by new technology developments. We have chosen to organize computer
various engineering groups, but then Research at Google is built on the science research differently at Google
moved to a research team, where the premise that connecting research by maximally connecting research
work continued using a different, and with development provides teams with and development. This yields not only
perhaps deeper, algorithmic basis. powerful, production-quality infra- innovative research results and new
structure and a large user base, result- technologies, but also valuable new ca-
Successes ing not only in innovative research, but pabilities for the company. Our hybrid
In the same way that it is difficult to also in valuable new commercial capa- approach to research enables us to
define what exactly constitutes “re- bilities. By coupling research and de- conduct experiments at a scale that is
search,” it can be difficult to measure velopment, our goal is to minimize or generally unprecedented for research
its “success.” In our opinion, a re- even eliminate the traditional technol- projects, generating stronger research
search project is successful if it has ogy transfer process, which has proven results that can have a wider academic
academic or commercial impact, or challenging at other companies. Most and commercial impact. We also pro-
ideally, both. Commercial impact at of our projects involve people work- vide flexible opportunities across the
Google is perhaps easier to measure, ing with a given technology from the R&D spectrum for our team members.
and the company has benefitted from research stage through to the product While our hybrid research model ex-
numerous advances in systems, speech stage. This close collaboration and ploits a number of things particular to
recognition, language translation, ma- integration furthermore ensures the Google, we hypothesize that it may also
chine learning, market algorithms, reality of the problems being investi- serve as an interesting model for other
computer vision, and more. gated: research is conducted on real technology companies.
By academic impact we refer to systems and with real users. Our flex-
impact on the academic community, ible organization also provides diverse References
1. Baluja, S. and Covell, M. Waveprint: Efficient wavelet-
other companies or industries, and the opportunities for our employees and based audio fingerprinting. In Pattern Recognition, 2008.
field of computer science in general. has positive implications for our inno- 2. Buderi, R. Engines of Tomorrow: How The World’s
Best Companies Are Using Their Research Labs To Win
Of course, this type of impact has most vation culture and hiring ability. The Future. Simon & Schuster, 2000.
traditionally come from publications, Of course, this close integration 3. Chang, F. et al. Bigtable: A distributed storage system
for structured data. In Proceedings of OSDI 2006.
and Google continues to publish re- also brings some risks with it. Being so 4. Dean, J. and Ghemawat, S. MapReduce: Simplified
search results at increasing rates (from close to the users and to the day-to-day data processing on large clusters. In Proceedings of
OSDI 2004.
13 papers published in 2003, to 130 in activities of product teams, it is easy 5. Dodgson, M., Gann, D. and Salter, A. The Management
of Technological Innovation: Strategy and Practice.
2006, to 279 in 2011). Some of our pa- to get drawn in and miss new develop- Oxford University Press, 2008.
pers are highly regarded and have been ments. To mitigate this risk, we engage 6. Enkel, E., Gassmann, O. and Chesbrough, H. Open R&D
and open innovation: Exploring the phenomenon. In
extensively cited.3,4,7 But we feel that with the academic community through R&D Management, 2009.
publications are by no means the only various initiatives such as our visiting 7. Ghemawat, S., Gobioff, H., and Leung, S.T. Google file
system. In Proceedings of ACM SIGOPS 2003.
mechanism for knowledge dissemina- faculty program, our intern program 8. Leifer, R., O’Connor, G. and Rice, M. Implementing
tion: Googlers have led the creation or our faculty research awards pro- radical innovation in mature firms: The role of hubs.
In The Human Side of Managing Technological
of over 1,000 open source projects, gram. We also encourage publication Innovation. R. Katz, Ed., Oxford University Press, 2004.
contributed to various standards (for of research results, though we some- 9. Reis, C., Barth, A., and Pizano, C. Browser security:
Lessons from Google Chrome. ACM Queue 7, 5
example, as editor of HTML5), and pro- times get criticized for not publishing (May 2009).
duced hundreds of public APIs for ac- enough. One reason for this is that re- 10. Schalkwyk, J. Google Search by Voice: A case
study. In Advances in Speech Recognition: Mobile
cessing our services. In some cases, we searchers at Google have multiple av- Environments, Call Centers, and Clinics. A. Neustein
have used these different channels in enues for having impact, publishing Ed., Springer, 2010.
11. Stokes, D.E. Pasteur’s Quadrant—Basic Science
symbiotic ways, following up an initial papers not being the only method. As and Technological Innovation. Brookings Institution
publication describing the high-level a result, Googlers publish fewer pa- Press, 1997.
12. Uszkoreit, J., Ponte, J., Popat, A., and Dubiner, M.
ideas (MapReduce, GFS, BigTable) pers, but the ones they publish can be Large scale parallel document mining for machine
translation. In Proceedings of COLING 2010.
with open source implementations of more impactful, because they describe
particular aspects (Protocol Buffers). experience with well-tested and imple- Additional references can be found at http://research.
In other cases, projects have started as mented systems, not just proposed google.com/pubs/papers.html.
open source initiatives from day one: ideas. Another potential pitfall of the
Android and Chromium are probably hybrid research model is that it is prob- Alfred Spector ([email protected]) is Vice President
of Research and Special Initiatives at Google, Inc.
the two most well-known examples of ably more conducive to incremental
Peter Norvig ([email protected]) is Director of
open source projects and demonstrate research. We therefore do support par- Research at Google, Inc.
the effectiveness of this approach. adigmatic changes as well, as exempli- Slav Petrov ([email protected]) is Senior Research
fied by our autonomous vehicles proj- Scientist at Google, Inc.

Discussion ect, Google Chauffeur, among others.

We acknowledge many discussions on this topic with Dan
Technology companies invest in re- Huttenlocher, who spent a summer at Google in 2008,
search for a number of reasons, in- Conclusion and contributions and reviews from Bill Coughran, Úlfar
Erlingsson, Fernando Pereira, Matt Welsh, and John
cluding: importance to the company’s Many of the world’s computer science Wilkes. We also thank the anonymous reviewers for their
products and services, prestige and research questions are of great rel- valuable feedback.

contributions to the public good, and evance to Google’s business, our tech- Copyright held by author.

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 37
V
viewpoints

doi:10.1145/2209249.2209263 Sarah Spiekermann

Viewpoint
The Challenges
of Privacy by Design
Heralded by regulators, Privacy by Design holds the promise to solve the digital
world’s privacy problems. But there are immense challenges, including management
commitment and step-by-step methods to integrate privacy into systems.

P
r i vacy ma i n t e n a n c e and
control is a social value
deeply embedded in our
societies. A global survey
found that 88% of people are
worried about who has access to their
data; over 80% expect governments to
regulate privacy and impose penalties
on companies that do not use data re-
sponsibly. But privacy regulation is not
easy. The Internet’s current economics
as well as national security manage-
ment benefit from the collection and
use of rich user profiles. Technology
constantly changes. And data is like
water: it flows and ripples in ways that
are difficult to predict. As a result, even
a well-conceived, general, and sustain-
able privacy regulation, such as the
European Data Protection Directive
95/46/EC, struggles to ensure its ef-
fectiveness. Companies regularly test
legal boundaries and many risk sanc-
tions for privacy breaches to avoid con-
straining their business.
Against this background, the Eu-
ropean Commission and other regu-
latory bodies are looking for a more Technologies (PETs) and add a good as privacy default settings or end-to-
effective, system- and context-specific dose of security, thereby creating a end security of personal data) and the
Photogra ph by Refat/ sh utt erstock. co m

balance between citizens’ privacy fault-proof systems’ landscape for recognition of privacy in a company’s
rights and the data needs of compa- the future. But the reality is much risk management processes.1 PbD can
nies and governments. The apparent more challenging. According to Ann thus be defined as “an engineering
solution proposed by regulators now, Cavoukian, the Ontario information and strategic management approach
but barely specified, is Privacy by De- and privacy commissioner who first that commits to selectively and sus-
sign (PbD). At first sight, the power- coined the term, PbD stands for a pro- tainably minimize information sys-
ful term seems to suggest we simply active integration of technical privacy tems’ privacy risks through technical
need to take a few Privacy-Enhancing principles in a system’s design (such and governance controls.”

38 communicatio ns o f the ac m | j u ly 201 2 | vo l. 5 5 | no. 7

 CACM_JOCCH_one-third_page_vertical:Layout viewpoints

However, a core challenge for PbD

is to get organizations’ management
involved in the privacy strategy. Man- Managing personal
agement’s active involvement in the data means
corporate privacy strategy is key be-
cause personal data is the asset at the optimizing its
heart of many companies’ business
models today. High privacy standards
strategic use, quality, ACM
and long-term
can restrict the collection and use of
data for further analysis, limit strate- availability. Journal on
Computing and
gic options, and impact a firm’s bot-
tom line. Consider advertising rev-
enues boosted by behavioral targeting
practices and peoples’ presence on
social networking sites: without per-
Cultural
sonal data, such services are unthink-
able. PbD proponents hardly embrace
start of a new IT project. It needs to en-
ter the system development life cycle at
Heritage
these economic facts in their reason- such an early point that architectural
ing. In contrast, they take a threat per- decisions around data processing,
spective arguing that low privacy stan- transfer, and storage can still be made.
dards can provoke media backlash Managers and engineers (as well as
and lead to costly legal trials around other potential stakeholders) need to
privacy breaches. And indeed, distrust assess the privacy risks they are willing
caused by privacy breaches is probably to take and jointly decide on techni-
the only real blemish on the image of cal and governance controls for those
technology companies such as Google risks they are not willing to bear.
or Facebook. Brands are a precious
company asset, the most difficult to Privacy by Design Challenges
build and the most costly to maintain. Even when both managers and engi-
Hence, brand managers should be neers are committed to PbD, more
keen to avoid privacy risks. Equally, re- challenges must be overcome:
cent data breach scandals have forced ˲˲ Privacy is a fuzzy concept and is ◆ ◆ ◆ ◆ ◆
CEOs to quit. thus difficult to protect. We need to
Despite these developments, many come to terms on what it is we want JOCCH publishes papers of
managers still do not understand that to protect. Moreover, conceptually significant and lasting value in
a sustainable strategy for one of their and methodologically, privacy is often all areas relating to the use of ICT
company’s core assets—personal confounded with security. We need to
data—requires them to actively man- start distinguishing security from pri- in support of Cultural Heritage,
age this asset. Managing personal data vacy to know what to address with what seeking to combine the best of
means optimizing its strategic use, means. computing science with real
quality, and long-term availability. Un- ˲˲ No agreed-upon methodology sup-
attention to any aspect of the
fortunately, few of today’s managers ports the systematic engineering of pri-
cultural heritage sector.
want to take on this new challenge. In- vacy into systems. System development
stead, they derive what they can from life cycles rarely leave room for privacy
the information bits they get and leave considerations. ◆ ◆ ◆ ◆ ◆
the privacy issue as a nuisance that is ˲˲ Little knowledge exists about the
better left to be fixed by their lawyers. tangible and intangible benefits and
But even if managers took up the risks associated with companies’ pri-
privacy challenge and incorporated the vacy practices. www.acm.org/jocch
How can these challenges be over-
active governance of personal data into
their companies’ strategic asset man- come? A Privacy Impact Assessment
www.acm.org/subscribe
agement, they would not be able to de- (PIA) Framework recently created for
termine the right strategy without their RFID technology (see http://ec.europa.
IT departments: PbD requires the guts eu/information_society/policy/rfid/pia/
and ingenuity of engineers. As the term index_en.htm) has been called a “land-
implies, the design of systems needs to mark for PbD” because it offers some
be altered or focused to technically em- answers: The PIA Framework suggests
brace the protection of peoples’ data. concrete privacy goals and describes a
Consequently, privacy must be on en- method to reach them. Pragmatically,
gineers’ requirements radar from the it recommends that organizations use

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 39
viewpoints

the specific legislative privacy prin- chitectures.2 Privacy scholars still willing to pay for advertisement-free,
ciples of their region or sector or the put too much focus on information privacy-friendly services? Will they in-
OECD Privacy guidelines as a starting practices only (such as Web site pri- cur switching costs and move to com-
point to determine privacy protection vacy policies). Instead, they should petitive services that are more privacy
goals. In Europe, for example, the Eu- further investigate how to build sys- friendly? Would the 83% of U.S. con-
ropean Data Protection Directive 95/46/ tems in client-centric ways that maxi- sumers who claim that they would stop
EC or its successor should be taken. It mize user control and minimize net- doing business with a company that
includes the following privacy goals: work or service provider involvement. breaches their privacy really do so? We
˲˲ Safeguarding personal data qual- Where such privacy-friendly architec- need to better understand these dy-
ity through data avoidance, purpose- tures are not feasible (often for busi- namics as well as the current changes
specific processing, and transparency ness reasons), designers can support in the social perception of what we re-
vis-à-vis data subjects. PbD by using technically enforceable gard as private.
˲˲ Ensuring the legitimacy of person- default policies (“opt-out” settings) But research on the behavioral eco-
al and sensitive data processing. or data scarcity policies (erasure or nomics of privacy has clearly demon-
˲˲ Complying with data subjects’ granularity policies), data portabil- strated that regardless of what people
right to be informed, to object to the ity, and user access and delete rights. say, they make irrational privacy deci-
processing of their data, and to access, Where such technical defaults are not sions and systematically underesti-
correct, and erase personal data. feasible, concise, accurate, and easy- mate long-term privacy risks. And this
˲˲ Ensuring confidentiality and secu- to-understand notices of data-han- is not only the case for privacy-seeking
rity of personal data. dling practices and contact points for individuals, but also for managers
Security and privacy in this view are user control and redress should come who are making PbD decisions for
clearly distinguished. Security means into play. their companies.
the confidentiality, integrity, and avail- A challenge, however, is that system Therefore, I appreciate that PIAs
ability of personal data are ensured. development life cycles and organiza- are suggested to become mandatory
From a data protection perspective se- tional engineering processes do not in the new European data protection
curity is one of several means to ensure consider such practices. So far, privacy legislation. However, they must be ac-
privacy. A good PbD is unthinkable is simply not a primary consideration companied by a clear set of criteria for
without a good Security by Design plan. for engineers when designing systems. judging their quality as well as sanc-
The two approaches are in a “positive This gap raises many questions: When tions for noncompliance.
sum” relationship. That said, privacy should privacy requirements first en- Most important, as this Viewpoint
is about the scarcity of personal data ter the system development life cycle? makes clear: PIAs need to be made
creation and the maximization of in- Who should be responsible? Given that mandatory for the designers of new
dividuals’ control over their personal privacy controls impact business goals, technologies—the IBMs and SAPs of
data. As a result, some worry that PbD who can actually decide on appropriate the world—and not just data control-
could undermine law enforcement measures? Must there be ongoing pri- lers or processors who often get system
techniques that use criminals’ data vacy management and practices moni- designs off the shelf without a say.
traces to find and convict them. More toring? If organizations purchase stan- Making PIAs mandatory for system
research and international agreement dard software solutions or outsource designers could be a great step toward
in areas such as anonymity revocation operations, pass data to third parties or PbD and support compliance with the
are certainly needed to demonstrate franchise their brands, who is respon- policies defined in Europe, in U.S. Pri-
this need not be the case even if we sible for customer privacy? vacy sectors laws, as well as the Safe-
have privacy-friendly systems. Harbor Framework.
After privacy goals are clearly de- Conclusion Only if we force those companies
fined, we must identify how to reach For privacy to be embedded in the sys- that design systems, their manage-
them. The PIA Framework mentioned tem development life cycle and hence ment and their engineers, to embrace
earlier is built on the assumption that in organizational processes, compa- such process-driven, bottom-up ways
a PbD methodology could largely re- nies must be ready to embrace the do- to embed laws and ethics into code
semble security risk assessment pro- main. Unfortunately, we still have too can we really protect the core values
cesses such as NIST or ISO/IEC 27005. little knowledge about the real dam- of our Western liberal democracies
These risk assessment processes iden- age that is being done to brands and and constitutions.
tify potential threats to each protec- a company’s reputation when privacy
tion goal. These threats and their prob- breaches occur. The stock market sees References
1. Cavoukian, A. Privacy by Design Curriculum 2.0, 2011;
abilities constitute a respective privacy some negligible short-term dips, but http://privacybydesign.ca/publications/.
risk. All threats are then systematically people flock to data-intensive services 2. Spiekermann, S. and Cranor, L.F. Engineering privacy.
IEEE Transactions on Software Engineering 35, 1
mitigated by technical or governance (such as social networks); so far, they (Jan./Feb. 2009), 67–82.
controls. Where this cannot be done, do not sanction companies for pri-
remaining risks are documented to be vacy breaches. So why invest in PbD Sarah Spiekermann ([email protected]) is the head of
the Institute for Management Information Systems at
addressed later. measures? Will there be any tangible the Vienna University of Economics and Business, Vienna,
As in security engineering, PbD benefits from PbD that justifies the in- Austria.

controls heavily rely on systems’ ar- vestment? Would people perhaps be Copyright held by author.

40 c om municatio ns o f th e ac m | j u ly 201 2 | vol . 5 5 | no. 7

Previous
A.M. Turing Award
Recipients

1966 A.J. Perlis

1967 Maurice Wilkes
1968 R.W. Hamming
1969 Marvin Minsky
1970 J.H. Wilkinson
1971 John McCarthy
1972 E.W. Dijkstra
1973 Charles Bachman
1974 Donald Knuth
1975 Allen Newell ACM A.M. TURING AWARD
NOMINATIONS SOLICITED
1975 Herbert Simon
1976 Michael Rabin
1976 Dana Scott
1977 John Backus
1978 Robert Floyd
1979 Kenneth Iverson Nominations are invited for the 2012 ACM A.M. Turing Award. ACM’s oldest
1980 C.A.R Hoare
1981 Edgar Codd and most prestigious award is presented for contributions of a technical nature to
1982 Stephen Cook the computing community. Although the long-term influences of the nominee’s
1983 Ken Thompson
1983 Dennis Ritchie work are taken into consideration, there should be a particular outstanding
1984 Niklaus Wirth and trendsetting technical achievement that constitutes the principal claim to
1985 Richard Karp
1986 John Hopcroft
the award. The award carries a prize of $250,000 and the recipient is expected
1986 Robert Tarjan to present an address that will be published in an ACM journal. Financial support
1987 John Cocke
of the Turing Award is provided by the Intel Corporation and Google Inc.
1988 Ivan Sutherland
1989 William Kahan
1990 Fernando Corbató Nominations should include:
1991 Robin Milner
1992 Butler Lampson 1) A curriculum vitae, listing publications, patents, honors, other awards, etc.
1993 Juris Hartmanis
1993 Richard Stearns 2) A letter from the principal nominator, which describes the work of
1994 Edward Feigenbaum the nominee, and draws particular attention to the contribution which
1994 Raj Reddy
1995 Manuel Blum is seen as meriting the award.
1996 Amir Pnueli
1997 Douglas Engelbart 3) Supporting letters from at least three endorsers. The letters should not
1998 James Gray all be from colleagues or co-workers who are closely associated with
1999 Frederick Brooks
2000 Andrew Yao the nominee, and preferably should come from individuals at more than
2001 Ole-Johan Dahl one organization. Successful Turing Award nominations usually include
2001 Kristen Nygaard
2002 Leonard Adleman substantive letters of support from a group of prominent individuals
2002 Ronald Rivest broadly representative of the candidate’s field.
2002 Adi Shamir
2003 Alan Kay For additional information on ACM’s award program
2004 Vinton Cerf
2004 Robert Kahn please visit: www.acm.org/awards/
2005 Peter Naur
2006 Frances E. Allen Additional information on the past recipients
2007 Edmund Clarke of the A.M. Turing Award is available on:
2007 E. Allen Emerson http://amturing.acm.org/byyear.cfm .
2007 Joseph Sifakis
2008 Barbara Liskov
2009 Charles P. Thacker Nominations should be sent electronically
2010 Leslie G. Valiant by November 30, 2012 to:
2011 Judea Pearl Ravi Sethi, Avaya Labs c/o [email protected]
practice
d oi:10.1145/ 2209249.2209264
usefulness and hamper the growth of
Article development led by
queue.acm.org
new applications.
This article aims to provide part of
the bufferbloat solution, proposing
A modern AQM is just one piece an innovative approach to AQM suit-
of the solution to bufferbloat. able for today’s Internet called CoDel
(for Controlled Delay, pronounced like
By Kathleen Nichols and Van Jacobson “coddle”). This is a “no-knobs” AQM
that adapts to changing link rates and

Controlling
is suitable for deployment and experi-
mentation in Linux-based routers (as
well as silicon).
Packet networks require buffers to

Queue Delay
absorb short-term arrival rate fluctua-
tions. Although essential to the opera-
tion of packet networks, buffers tend
to fill up and remain full at congested
links, contributing to excessive traffic
delay and losing the ability to perform
their intended function of absorbing
bursts. The “full buffer problem” was
recognized in the early days of the In-
ternet, and mitigations were explored
then.9,15,17
In 1998, the Internet Research Task
N early th ree d eca d e safter it was first diagnosed, Force urged the deployment of active
queue management in the Internet,1
the “persistently full buffer problem,” recently specifically recommending Random
exposed as part of bufferbloat,6,7 is still with us and Early Detection (RED).5 Although RED
made increasingly critical by two trends. First, was simple and can be effective at re-
ducing persistent queues, little guid-
cheap memory and a “more is better” mentality ance was available to set its configu-
have led to the inflation and proliferation of buffers. ration parameters and it functioned
poorly in a number of cases. This led to
Second, dynamically varying path characteristics are a general reluctance to use it. As RED’s
much more common today and are the norm at the problems became apparent, research
consumer Internet edge. Reasonably sized buffers documented those issues and pro-
posed new AQMs, adding more config-
become extremely oversized when link rates and path uration and complexity to the original
delays fall below nominal values. RED. Though Feng et al.22 pointed out
in 2002 that queue length is not a good
The solution for persistently full buffers, active predictor of congestion, it continued
queue management (AQM), has been known for to be used. Although research contin-
two decades but has not been widely deployed ued, deployment did not.
The Internet has been saved from
because of implementation difficulties and general disaster by a constant increase in link
misunderstanding about Internet packet loss and rates and by usage patterns. Over the
Illustration by A nd reas Ko ll er

past decade, evidence has accumu-

queue dynamics. Unmanaged buffers are more critical lated that this whistling in the dark
today since buffer sizes are larger, delay-sensitive cannot continue without severely im-
applications are more prevalent, and large (streaming) pacting Internet usage: independent
measurement studies4,13 have found
downloads common. The continued existence of edge queuing delays from hundreds of
extreme delays at the Internet’s edge can impact its milliseconds up to multiple seconds.

42 c ommunic atio ns o f th e ac m | j u ly 201 2 | vol . 5 5 | no. 7

1 2 3 4 5 6 7 8

4 5 6 7 8

12.45459430,34.43962930,0.39884989
9.29475655,39.19462381,22.30878452
1 2 3 4 5 6 7 8

40.20392125,8.29991467,52.31173403 12.45459430,34.43962930,0.39884989
9.29475655,39.19462381,22.30878452

3.98672430,82.32754750,93.26614247
40.20392125,8.29991467,52.31173403
21.29056765,49.29917436,20.94763227 21.29056765,49.29917436,20.94763227
3.98672430,82.32754750,93.26614247

1 2 3 4 5

1 2 3 4 5 6 7 8

ju ly 2 0 1 2 | vol . 55 | n o. 7 | c om m u n i cat ion s o f t he acm 43

practice

Figure 1.TCP connection startup. packet to depart. The third packet

would have to wait for an additional
18ms for both one and two to depart,
and so on. This bottleneck-induced
waiting is what creates the queues that
form in the packet buffers at the link.
Figure 2 shows the connection one
Sender Receiver RTT (round-trip time) later. The bot-
tleneck spaces out the packets, and
they retain that spacing after leaving.
The receiver just turns a data packet
into an ack (acknowledgment packet),
so the ack stream retains the spacing
on the return path. The sender turns
Figure 2. TCP connection after one RTT. each ack into a data packet, so, after
just one RTT, the packet-arrival rate
at the bottleneck exactly equals the
departure rate, and the queue won’t
grow. This is clearly visible in Figure
3, which shows the bottleneck queue
size vs. time. There is an initial peak
Sender Receiver corresponding to the initial window’s
worth of back-to-back packets, but it
dissipates after one RTT, leaving just
a ±1 packet variation resulting from
small phase differences in the arrival
and departure processes.
Note, however, that the steady-state
These measurement studies and the contention, transport conversation queue in Figure 3 is not zero. Figures
bufferbloat project2 document the startup transients, and/or changes in 1 and 2 are sized so that the distance
harmful effects of large, unmanaged the number of conversations sharing in time between the sender and re-
buffers at the network edge. a link. Unfortunately, other network ceiver is 10 bottleneck packet times
Correct buffer sizing is not an easy behavior can cause buffers to fill, with (for example, 100ms if the bottleneck
problem. Undersizing—making buf- effects that are not nearly as benign. packet spacing is 10ms); thus, it takes
fers smaller than the traditional BDP With the wrong conceptual model for 20 packets in flight to “fill the pipe”
(bandwidth-delay product)—is fraught queues, AQMs have limited operation- and use 100% of the bottleneck capac-
with problems,8 nicely illustrated by al range, require a lot of configuration ity. The window in this example is 25
Guillaume Vu-Brugier et al.20 Today’s tweaking, and frequently impair rather packets—five more than will fit in the
links vary in bandwidth, and individual than improve performance. pipe (the BDP, measured in packets).
connections vary in round-trip times Figure 1 shows a TCP connection These create a standing queue that can-
(the “delay” used in BDP). This makes shortly after startup (see “Congestion not dissipate. This is clearly visible in
it impossible to properly pick a static Avoidance and Control”8 for more Figure 2: as the 26th packet is about
size for most edge links.13,14 A simple, discussion). The sender launches its to arrive at the bottleneck, five pack-
robust algorithm that can manage buf- window of 25 packets back-to-back, ets are still in its queue; but, from this
fer delay regardless of buffer size and and they flow through the network time on, whenever one packet leaves,
link bandwidth without a negative im- until they hit a bottleneck (bandwidth another will arrive. Thus, there will al-
pact on utilization can make oversized reduction). There, as each packet is ways be five packets in the queue (±1),
buffers irrelevant. squeezed down in bandwidth, it must even though the sender’s rate is con-
stretch out in time since its size stays stant. The standing queue has nothing
Understanding Queues constant. The vertical direction is to do with the sender’s rate but rather
Developing effective active queue bandwidth (bits/sec), and the horizon- with how much the sender’s window
management has been hampered by tal direction is time (sec), so the area exceeds the pipe size.
misconceptions about the cause and of each rectangle is the packet size This standing queue, resulting
meaning of queues. Network buffers (bits/sec × sec = bits). For example, from a mismatch between the win-
exist to absorb the packet bursts that if the packets were 1ms long and the dow and pipe size, is the essence of
occur naturally in statistically multi- bandwidth reduction was 100Mbps bufferbloat. It creates large delays
plexed networks. Queues occur in the to 10Mbps, then the second packet but no improvement in throughput.
buffers as a result of short-term mis- would arrive 1ms after the first but It is not a phenomenon treated by
matches in traffic arrival and departure would have to wait for an additional queuing or traffic theory, which, un-
rates that arise from upstream resource 9ms, since it takes 10ms for the first fortunately, results in it being almost

44 com munic atio ns o f th e ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 practice

universally misclassified as conges- sient repeats every RTT and the buffer ing time window that is longer than
tion (a completely different and much is always occupied. The window ex- the nominal RTT.
rarer pathology). These theories usu- actly fills the pipe, however, and there
ally assume Poisson arrival processes, is no excess queue, so any attempt to Controlled Delay Management
which are, by definition, uncorrelated. reduce this queue will result in poor In early 1998, we set out to under-
The arrivals of a closed-loop, reliable utilization of the bottleneck. Thus, oc- stand why the original RED was dif-
transport process such as TCP are cupancy time contains no information ficult to configure and came up with
completely correlated, resulting in about excess queue. a new RED algorithm (described in
an arrival and departure rate equality Figure 4 and the leading part of a talk and an unpublished paper10,12)
that theorists have dismissed as un- Figure 3 show queues doing their that needed only one parameter: the
natural and wildly improbable. Since job—acting as shock absorbers to queue’s output bandwidth or aver-
normal cures for congestion such as convert bursty arrivals into smooth, age departure rate. Despite improve-
usage limits or usage-based billing steady departures. This is good queue. ments in performance, issues re-
have no effect on bufferbloat but an- The tail end of Figure 3 shows a queue mained—almost anything worked for
noy customers and discourage net- doing nothing but creating excess long-lived TCP transfers and almost
work use, addressing the real problem delay. This is bad queue. The core of nothing worked for bursty traffic. In
would be prudent. the bufferbloat-detection problem is the decade since, many researchers
The bufferbloat problem, making separating good from bad. Figure 3 have made strides in AQM, but no one
the window match the pipe size, is hints at the successful strategy: good has produced an AQM that has the fol-
hard to address. Window sizes are cho- queue is occupancy that goes away in lowing characteristics:
sen by senders while queues manifest about one RTT; bad queue persists for ˲˲ It is parameterless—it has no
at bottleneck gateways. It is difficult several RTTs. An easy, robust way to knobs for operators, users, or imple-
for senders to compute a window size, separate the two is to take the mini- menters to adjust.
since both terms—bottleneck band- mum of the queue length over a slid- ˲˲ It treats good queue and bad queue
width and RTT—change constantly
as connections come and go, paths Figure 3. Queue size vs. time.
change because of rerouting, layers 1
and 2 protocols adapt bandwidth for
changing physical conditions, and
so on. Since queues can be directly
measured at the bottleneck, the most
promising approach is to detect the
problem there, then signal senders to
reduce their windows (via TCP conges-
Queue length

tion-control mechanisms).
Reliable detection is difficult, how-
ever. The big queue at the start of
Figure 3 is necessary for connections
to get started, so queue magnitude
contains no information about excess Time
queue. The fact that the problem lies
in the flat part of Figure 3 has caused
some researchers to look at the time
the buffer is occupied as an indicator. Figure 4. Queue size vs. time for ack-per-window receiver and 20-packet window.
Figure 4 shows the queue vs. time
for a TCP receiver that sends one
ack per window rather than one per
packet. This is a legal, though brittle,
operating mode used by some com-
mercial operating systems with very
Queue length

high per-packet overhead to reduce

the number of acks generated so as to
perform credibly in benchmark tests.
This kind of queue length variation is
also seen in Web traffic—a superposi-
tion of mostly small transfers that are
essentially all startup transient. Since
this ack policy means a full window of
Time
packets is always delivered as a back-
to-back burst, the initial turn-on tran-

ju ly 2 0 1 2 | vol . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 45

practice

differently—that is, it keeps the delays that distinguish it from prior AQMs. time through the queue. Use of the ac-
low while permitting bursts of traffic. First, CoDel’s algorithm is not based tual delay experienced by each packet
˲˲ It controls delay, while insensitive on queue size, queue-size averages, is independent of link rate, gives supe-
to round-trip delays, link rates, and queue-size thresholds, rate measure- rior performance to use of buffer size,
traffic loads. ments, link utilization, drop rate or and is directly related to the user-visi-
˲˲ It adapts to dynamically changing queue occupancy time. Starting from ble performance.
link rates with no negative impact on Van Jacobson’s 2006 insight,11 we used Using the minimum value has some
utilization. the local minimum queue as a more important implications. The mini-
˲˲ It is simple and efficient—it can accurate and robust measure of stand- mum packet sojourn can be decreased
easily span the spectrum from low- ing queue. Then we observed that it is only when a packet is dequeued, which
end, Linux-based access points and sufficient to keep a single-state vari- means all the work of CoDel can take
home routers up to high-end commer- able of how long the minimum has place when packets are dequeued for
cial router silicon. been above or below the target value transmission and that no locks are
for standing queue delay rather than needed in the implementation. The
One Code Module, No keeping a window of values to com- minimum is the only statistic with this
Knobs, Any Link Rate pute the minimum. Finally, rather property. The only addition to packet
CoDel (Controlled Delay Manage- than measuring queue size in bytes or arrival is that a timestamp of packet ar-
ment) has three major innovations packets, we used the packet-sojourn rival time is created. If the buffer is full
when a packet arrives, then the packet
Figure 5. CoDel and RED performance variation with link bandwidth. can be dropped as usual.
CoDel assumes a standing queue of
CoDel Median Pkt Delay (sec) RED Median Pkt Delay (sec)
target is acceptable and that it is un-
by Bandwidth (Mbps) by Bandwidth (Mbps) acceptable to drop packets when there
0.020 0.020 are fewer than one MTU’s (maximum
transmission unit’s) worth of bytes in
0.015 0.015
the buffer. CoDel identifies the per-
0.010 0.010 sistent delay by tracking the (local)
0.005 0.005 minimum queue delay packets expe-
rience. To ensure the minimum value
0 0
3 10 45 100 3 10 45 100
does not become stale, it has to have
been experienced within the most re-
(a) (c)
cent interval. When the queue de-
CoDel Link Utilization by REDLink Utilization by lay has exceeded target for at least
Link Bandwidth (Mbps) Link Bandwidth (Mbps)
interval, a packet is dropped and
1.0 1.0
a control law sets the next drop time.
0.9 0.9
The next drop time is decreased in in-
0.8 0.8
0.7
verse proportion to the square root of
0.7
0.6
the number of drops since the drop-
0.6
0.5 0.5
ping state was entered, using the well-
0.4 0.4 known relationship of drop rate to
3 10 45 100 3 10 45 100 throughput to get a linear change in
(b) (d) throughput.12,16 When the queue de-
lay goes below target, the controller
stops dropping. No drops are carried
out if the buffer contains fewer than
Figure 6. CoDel performance variation with link bandwidth for low bandwidths. an MTU’s worth of bytes. Additional
logic prevents reentering the drop-
ping state too soon after exiting it and
Median Packet Delays Link Utilization by
by Bandwidth Link Bandwidth (in Mbps)
resumes the dropping state at a recent
0.16 1.0 control level, if one exists.
— 500B MTU
0.14
— 1500B MTU 0.9
Target and interval are con-
0.12
0.10 1.0
Web-only Traffic stants with straightforward interpreta-
0.08 0.8
tions: acceptable standing queue delay
0.06 and a time on the order of a worst-case
0.04 0.6
0.02 RTT of connections through the bottle-
0.4
0 neck. We experimented to determine
0.128 0.256 0.512 1.5 0.128 0.256 0.512 1.5 values for target and interval
(a) (b) that give a consistently high utiliza-
tion with a controlled delay across a
range of bandwidths, RTTs, and traffic

46 comm unicatio ns o f the acm | j u ly 201 2 | vo l. 5 5 | no. 7

 practice

loads. Below a target of 5ms, utiliza- slightly lower utilizations, as expect- zation values from a number of traffic
tion suffers for some conditions and ed. To emulate Web-browsing loads, loads (FTPs with and without added
traffic loads; above 5ms there is very we used PackMime,21 which runs with Web-browsing and constant-bit-rate
little or no improvement in utilization. the ns-2 Full-TCP protocol and uses applications) and RTTs from 10ms–
Interval is loosely related to RTT the New Reno congestion-control al- 500ms, sorted them by link band-
since it is chosen to give endpoints gorithm. A constant bit-rate applica- widths, and box plotted the results.
time to react without being so long tion used UDP (User Datagram Pro- These are displayed separately for
that response times suffer. A setting tocol) and was configured at 64Kbps larger bandwidths (3Mbps, 10Mbps,
of 100ms works well across a range of in 100-byte packets. All TCPs were 45Mbps, and 100Mbps in Figure 5)
RTTs from 10ms to 1 second (excellent configured with modern parameters, and smaller bandwidths (128Kbps,
performance is achieved in the range including use of SACK. In most cases, 256Kbps, 512Kbps, and 1.5Mbps in
from 10ms to 300ms). (Pseudocode CoDel’s managed buffer was set very Figure 6). The results for RED are also
for CoDel is included in the appendix, large (~8xBDP packets) to verify that shown for the larger bandwidths, but
available at http://queue.acm.org/ap- the actual size of the buffer does not median delays for the smaller set were
pendices/codel.html.) matter. For comparison, we also ran excessive (100ms–200ms).
CoDel’s efficient implementation many of our test scenarios with the CoDel drops only when its mini-
and lack of configuration are unique ns-2 RED AQM module substituted mum delay statistic exceeds 5ms and
features that make it suitable for man- for CoDel. We used the most recent the buffer contains at least a link
aging modern packet buffers. The settings and code for ns-2 RED, which MTU’s worth of bytes. For links with
three innovations—using minimum reads the initial link bandwidth and MTUs transmitting in less than 5ms
rather than average as the queue mea- delay to adjust its settings, rather (the larger bandwidth group) and traf-
sure, simplified single-state variable than the original RED of Floyd and Ja- fic loads consisting of all long-lived
tracking of minimum, and use of cobson.5 We also attempted to run the FTPs, the median delay should ap-
queue-sojourn time—lead directly to scenarios with BLUE22 but its default proach this target. CoDel is designed
these unique features. settings performed badly and limited to permit short-term bursts; thus, me-
experimentation did not result in set- dian values for bursty traffic patterns
Simulation Experiments tings that worked well so we did not should be higher than the 5ms target.
No network simulation provides a pursue it. For larger bandwidths, delay resulting
high-fidelity representation of real- Metrics. The metrics of interest for from bursts of packets is small com-
ity; the real test of CoDel will come AQMs—queue delay and size, link utili- pared with the target (for example, a
through deployment on networks. zation, and “fairness” of drops between 1,500-byte packet is transmitted in
Though CoDel’s functionality and an flows—can be quite sensitive to the 0.1ms at 100 Mbps), so load variations
evaluation of its merits are our main types of traffic mixes and link band- have less effect on the median. At
concerns, some AQM comparisons widths,10,12,20 so we tested a range of traf- smaller bandwidths the delay caused
can serve as a sanity check. We did fic loads and conditions. The per-packet by an additional packet is significant
several thousand simulation runs, queue delay for successfully transmit- compared with the target (or example,
varying bottleneck bandwidth from ted (not dropped) packets is particu- 4ms for a 3Mbps link), and median
64kbps to 100mbps, end-to-end delay larly useful. This can be viewed against delays will be noticeably larger. This is
from 5ms to 1sec., simultaneous bulk simulation time for a single run or the exactly the desired behavior: the lon-
data transfers from 0–50, PackMime statistics over the run (for example, me- ger delays are not excessive and per-
Web traffic intensity from 0–80, with dian and 95th percentile) can be used for mit link utilizations to be maintained
and without CBR traffic, with both comparisons and trends. Monitoring at high levels. Figure 5 shows CoDel
Reno and CUBIC TCP congestion link utilization ensures queue manage- delays are as expected and the link
control, and all with CoDel using the ment does not have a negative impact utilizations are good. RED’s delays
same, constant configuration values. on throughput. Though not our primary and utilizations are similar for 3Mbps
As the next section will show, CoDel concern, we used the Jain fairness in- links, but delays and utilizations are
performed very well. Most impor- dex to see if drop share per source was smaller for 10Mbps and above. The
tantly, when link rates vary dynami- somewhat in proportion to the number low utilizations show that RED is like-
cally—even as much as two orders of of packets transmitted, computed for n ly overcontrolling.
magnitude—CoDel adapts. Results samples as: The lower bandwidth values de-
are compelling enough to move on to picted in Figure 6 use a smaller range
the next step of extensive real-world (Σixi)2 of RTTs (30ms–100ms). Such link rates
testing in Linux-based routers. n ⋅ (Σixi2) can be expected at the consumer In-
Simulator Configuration. For expe- ternet-access edge (handheld, home)
diency, we used the freely available Performance Across a Range or on degraded Wi-Fi links. Static low-
ns-2 simulator.18 Our file-transfer ap- of Static Link Rates bandwidth links usually use a smaller
plication used the Linux TCP suite, CoDel is the same code with the same link MTU, so we collected data where
primarily with CUBIC congestion settings, regardless of egress link rate. the link MTU was set to 500 bytes, as
control. Results (available online) To see how well it works, we collected well as the 1,500-byte MTU used for all
were similar with New Reno but with the median packet delay and link utili- other runs. As expected, the larger MTU

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n ic at ion s o f t he acm 47
practice

Figure 7. Wireless example.

Per-Packet Queue Delay Detail of Per-Packet Queue Delay Cumulative Kbytes transferred
for Dynamic Bandwidth vs. Simulation Time (sec) vs. simulation time (sec)

— CoDel RED — Tail drop — CoDel RED — Tail drop

10 1.0
1 × 105

ps
sp

s
Mb
bp
Mb

bp
ps

ps
1M

50

1M
10
s

s
10

b
b
bp

bp

0M
0M
ps

s
0M

0M
bp

s
0.5
s

10
Mb

bp

10
bp

M
10

10
1M
50
10

1M

2 5 10 pckt buffers

0 0 0
0 50 100 150 200 250 300 0 50 100 150 200 250 300 0 50 100 150 200 250 300
(a) (b) (c)

increases delays, but less significantly per second and changed link rates a buffer full of packets at the current
as bandwidth increases. Utilizations at 50-second intervals (over the 300 link rate. That delay can be as much
for low-bandwidth links are gener- simulated seconds), first dropping to as 10 seconds. RED keeps the queue
ally good since they are easy to fill with 10Mbps, then to 1Mbps, then jump- delay smaller than Tail Drop but does
mixes of FTPs. A few runs were done ing to 50Mbps, dropping to 1Mbps, not respond to changes as quickly as
at each bandwidth with only the Pack- and finally jumping back to 100Mbps. CoDel. CoDel permits an initial spike
Mime Web browsing, which makes it Buffer capacity is a single BDP (830 as the FTPs get started and a dropping
difficult to get high utilizations; we had packets) for the nominal rate. This rate is learned for the current condi-
to accept a drop rate of more than 10% scenario was repeated for CoDel, Tail tions. The delays spike when the link
to get over 60% utilization. Drop, and RED. rates drop (at 50, 100, and 250 sec-
Figure 7 shows the per-packet onds), as the queue size is now too
Performance on Dynamic Links queue delay and the cumulative long for the new rate. CoDel computes
Some dynamic link testing is possible number of kilobytes transferred as a new control point within 100 ms,
in simulation. To roughly emulate a simulation time progresses. As ex- the maximum interval a local mini-
(nominal) 100Mbps Wi-Fi link sub- pected, Tail Drop generally keeps its mum is valid. It is an open question
ject to degradation, we used a load of buffer full, delaying packets by the whether anything should be done to
four FTPs and five Web connections amount of time it takes to transmit speed this; preliminary studies show
it is better not to attempt to “clear
Figure 8. CoDel and RED performance by RTT. out” the backlog. If rate changes of an
order of magnitude or more several
times a minute are common, then the
CoDel 95th Percentile Delay (sec) RED 95th Percentile Delay (sec)
0.06 issue may require further study.
0.06
0.04 Comparing the kilobytes trans-
0.04
0.02 ferred with the queue delay, it is easy
0 0.02
to see that long delays are not required
0
CoDel Median Delays (sec) RED Median Delays (sec) for high utilization. CoDel transfers
0.02 0.02
almost the same total kilobytes as Tail
0.01 0.01 Drop, with the differences coming at
the rate jumps (150 and 250 seconds),
0 0
10 30 50 100 150 200 500
where CoDel’s FTPs have to ramp
10 30 50 100 150 200 500
(a) (c) up to fill the new pipe size while Tail
Drop’s queues send their backlog. Un-
CoDel Median Link Utilizations 1.0 RED Median Link Utilizations
1.0 dersized buffers are not the answer
0.9 to bufferbloat. Figure 7 shows the
0.9
same scenario using 10 packet buf-
0.8 0.8
fers, a size suitable for the 1Mbps rate.
0.7 0.7 Throughput is about 75% less for all
three schemes (Tail Drop and CoDel
0.6 0.6
are identical). Tail Drop tends to keep
0.5 0.5 its 10-packet buffer full, which would
result in a worst-case delay of 120ms
10 30 50 100 150 200 500 10 30 50 100 150 200 500
(b) (d) but with 25% of the throughput that
can be achieved with a large CoDel-
managed buffer—which gets a medi-

48 comm unicatio ns o f the acm | j u ly 201 2 | vol . 5 5 | no. 7

 practice

an delay of 2.7ms, 75th percentile de- make drops less randomly distributed the delay up and the utilization down.
lay of 5ms, and is less than 90ms 95% while CoDel gets randomness from the There are known mitigations to im-
of the total simulation time. independence of drop intervals and prove the mixing of acks and data pack-
packet arrivals. ets that will be performed in the home-
Dropping the Right Packets router implementation. Even the
Although most network analysis today Consumer Edge unmitigated simulation experiments
assumes connections with an unload- This scenario roughly emulates a con- showed acceptable performance.
ed 100-ms RTT, in practice RTTs vary. sumer edge for two (symmetric) band- AQM is not a substitute for differen-
Unless a path includes a satellite link, widths: 512KB and 1.5MB. The load tiated queuing to provide priority for
RTTs in the one-second range are usu- includes a two-way 64Kbps CBR (VoIP- packets that need low latency and jit-
ally caused by bufferbloat, not the in- like), an infinite FTP as a download, ter. We have had a lot to say in the past
trinsic path characteristics. At the con- Web browsing at a rate of two connec- about solutions for that type of traffic;
sumer edge, few connections will have tions per second, and uploads of small AQM should be employed on the pack-
less than a 30ms RTT. Since CoDel’s in- FTPs—1MB with short idle periods (5 et queues handling common Internet
terval is weakly related to RTT, we test- to 15 seconds, uniformly distributed) traffic and a Delay Bound per-hop be-
ed the effectiveness of a 100-ms setting between. The table lists results, where havior used for latency sensitive traffic.
over a wide range of likely RTTs and “C” is CoDel and “T” is Tail Drop and CoDel lends itself to efficient imple-
report on the range from 10ms–500ms. each link direction is shown separately. mentation in Linux-based routers, cru-
Figure 8 shows results for a variety Although CoDel never drops packets at cial to deployment at the edge. Other
of traffic loads, sorted by RTT. Both a higher rate than Tail Drop, it keeps a AQM implementations require locks
CoDel and RED keep the median delay much smaller queue and transfers sim- on the queue that are not necessary for
low, but CoDel has higher link utiliza- ilar amounts of data, offering encour- CoDel. There are also a small number
tions and better drop-share fairness, agement for taming bufferbloat. of state variables. We believe CoDel’s
showing that CoDel’s design is more The experiments presented here algorithm can be efficiently imple-
effective at dropping the right pack- mainly consist of “forward” traffic mented in silicon.
ets. CoDel’s utilization is very close to where all the data traffic is going in the
that of Tail Drop (except for an RTT analyzed direction. Reverse traffic has Manage the Right Queue
of 500ms) but with much less delay. well-known issues of ack compression At this point, a savvy user could be
CoDel’s performance metrics are not or data pendulum, which tend to push tempted to deploy CoDel through
significantly different between 30ms
and 200ms RTTs. Utilizations are Figure 9. Jain fairness for drop share, CoDel (shown in black) and RED (shown in red).
slightly less and have a larger range
as the RTT increases, because some
Fairness Index of Source Drop Share
traffic loads have difficulty keeping 1.0
the larger pipe full. The 500ms RTT
utilization shows more variation, the 0.8

low end corresponding to single FTPs, 0.6

which have difficulty keeping a very
large pipe full. 0.4
Figure 9 compares the Jain fairness
index for the source-drop shares of 0.2

CoDel and RED for these runs. CoDel 0

consistently outperforms RED for this 10 30 50 100 150 200 500
metric. This seems to be, in part, be-
cause the changes to the original RED

Consumer edge example.

512Kbps Links 1.5Mbps Links

download upload download upload
metric C T C/T C T C/T C T C/T C T C/T
pkt drop % 8 8 100% 1.5 5.8 25% 3.5 4.7 75% 1.4 2.5 56%
median delay 18 73 25% 9 37 25% 8 49 17% 0 0 100%
(ms)
total Mbytes 17 18 95% 13 14 92% 37 40 92% 22 21 103%
fairness 0.87 0.9 98% 0.9 0.9 104% 0.8 1 84% 0.6 0.7 85%
of drops

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 49
practice

Figure 10. Rate mismatch in a cable modem. References

1. Braden, R. et al. Recommendations on queue
management and congestion avoidance in the
Internet. RFC 2309 (1998).
CPE Router 2. Bufferbloat Project; http://www.bufferbloat.net.
3. CeroWrt Project; http://www.bufferbloat.net/projects/
cerowrt.
Cable Modem 4. Dischinger, M. et. al. Characterizing residential
Ethernet upstream broadband networks. In Proceedings of the Internet
user Scheduler CM buffer Measurement Conference, San Diego, CA, 2007.
(100 Mbps–1Gbps) (2Mbps-?)
pkts 5. Floyd, S. and Jacobson, V. Random early detection
gateways for congestion avoidance. IEEE/ACM
Transactions on Networking, (1993).
Rate mismatch will 6. Gettys, J. Bufferbloat: Dark buffers in the Internet.
cause CM buffer to fill Backspace Column. IEEE Internet Computing 15, 3
When uplink is heavily utilized (bufferbloat) and may (2011), 95-96.
the router buffers will fill cause priority inversion 7. Gettys, J. and Nichols, K. 2011. Bufferbloat: Dark
buffers in the Internet. Commun. ACM 9, 11 (Sept.
2011) 57–65.
8. Jacobson, V. Congestion avoidance and control.
Proceedings of SIGCOMM ’88, (Stanford, CA, 1998).
9. Jacobson, V. Reported in Minutes of the Performance
Working Group. Proceedings of the Cocoa Beach
a CeroWrt-enabled edge router to vendors to make changes but uses Internet Engineering Task Force. Corporation for
make bufferbloat disappear. Unfor- Ethernet flow control to permit only National Research Initiatives, Reston, VA, 1989.
10. Jacobson, V. Notes on using RED for queue
tunately, large buffers are not always the number of packets in the modem management and congestion avoidance. Talk
located where they can be managed buffer needed for good transmission presented at North American Network Operators’
Group (1998); ftp://ftp.ee.lbl.gov/talks/vj-nanog-red.pdf.
but can be ubiquitous and hidden.6,7 utilization while pushing the queue 11. Jacobson, V. A rant on queues. A talk presented at
Examples include consumer-edge into the router where it can be man- MIT Lincoln Labs, Lexington, MA, 2006; http://www.
pollere.net/Pdfdocs/QrantJul06.pdf.
routers connected to cable modems aged and where new algorithms can 12. Jacobson, V., Nichols, K. and Poduri, K. RED in a
and wireless access points with ring be more readily deployed. Options 2 different light, 1999; http://www.cnaf.infn.it/~ferrari/
papers/ispn/red_light_9_30.pdf.
buffers. Many users access the In- or 3 are preferable but require a cable- 13. Kreibich, C. et. al. Netalyzr: Illuminating the edge
ternet through a cable modem with modem vendor and/or a cable data network. In Proceedings of the Internet Measurement
Conference, (Melbourne, Australia, 2010).
varying upstream link speeds: 2 Mbps network service provider to make this 14. Li, T. and Leith, D. Adaptive buffer sizing for TCP
is a typical value. The home network part of the modem requirements. flows in 802.11e WLANs. In Proceedings of the 2008
Communications and Networking in China.
or home computer connects to the A modern AQM is just one piece of 15. Mankin, A. Random drop congestion control. In
Proceedings of SIGCOMM ’90.
cable modem with an Ethernet cable the solution to bufferbloat. Concat- 16. Mathis, M., Semke, J. and Mahdavi, J. The macroscopic
in the range of 100Mbps–1Gbps (Fig- enated queues are common in packet behavior of the TCP congestion avoidance algorithm.
ACM SIGCOMM Computer Communication Review 27,
ure 10). The modem’s buffers are at communications with the bottleneck 3 (1997).
the fast-to-slow transition, and that is queue often invisible to users and many 17. Nagle, J. Congestion control in IP/TCP internetworks.
RFC 896 (1984); http://www.ietf.org/rfc/rfc896.txt,.
where queues will build up: inside a network engineers. A full solution has 18. Network Simulator - ns-2; http://nsnam.isi.edu/nsnam/
sealed device outside of user control. to include raising awareness so that the index.php/User_Information.
19. http://www.pollere.net/CoDel.html.
Any differentiated services (DiffServ) relevant vendors are both empowered 20. Vu-Brugier, G., et. al. 2007. A critique of recently
queuing and buffer management at and given incentive to market devices proposed buffer-sizing strategies. ACM SIGCOMM
Computer Communication Review 37(1).
the router can be defeated by a full with buffer management. 21. Weigle, M. C. 2002. Web traffic generation in ns-2 with
queue in the cable modem. Three ap- PackMime-HTTP; http://www.cs.odu.edu/~mweigle/
research/packmime.
proaches have been suggested: limit Next Steps 22. Feng, W. et. al. 2002. The BLUE Active Queue
the Ethernet link to the upstream The open source project CeroWrt3 is Management Algorithm. IEEE/ACM Transactions on
Networking 10, 4 (2002), 513–528.
rate; put buffer management and Dif- using OpenWrt to explore solutions to
fServ queues in the cable modem with bufferbloat. A CoDel implementation Kathleen Nichols is the founder and CTO of Pollere Inc.,
a configuration interface for the Diff- is in the works, after which real-world a consulting company working in both government and
commercial networking. She has 30 years of experience
Serv queues; and implement Ethernet data can be studied. We plan to make in networking, including a number of Silicon Valley
flow control between the modem and our ns-2 simulation code available, as companies and as a cofounder of Packet Design.
the upstream router. well as some further results.19 Van Jacobson is a Research Fellow at PARC where he
Option 1 has the advantage that it leads its content-centric networking research program.
He has worked at Lawrence Berkeley National Laboratory,
can be implemented by a user with- Cisco Systems, and was a cofounder of Packet Design.
out cable-modem changes and the Related articles
disadvantage that it must rate limit to on queue.acm.org
the expected rate of the upstream. If
Bufferbloat: Dark Buffers in the Internet
the rate drops, then a queue will still Jim Gettys, Kathleen Nichols
build up in the cable modem, and any http://queue.acm.org/detail.cfm?id=2071893
additional short-term bandwidth can-
Revisiting Network I/O APIs:
not be utilized. Option 2 puts buffer The netmap Framework
management right at the bottleneck Luigi Rizzo
link but requires the vendor to make http://queue.acm.org/detail.cfm?id=2103536
(possibly significant) changes to the The Robustness Principle Reconsidered
modem architecture and permit con- Eric Allman
figuration. Option 3 also requires http://queue.acm.org/detail.cfm?id=1999945 © 2012 ACM 0001-0782/12/07 $15.00

50 c om municatio ns o f th e acm | j u ly 201 2 | vol . 5 5 | no. 7

 d oi:10.1145/ 2209249 . 2 2 0 9 2 6 5

Article development led by

queue.acm.org

Until our programming languages

catch up, code will be full of horrors.
By Poul-Henning Kamp

My Compiler
Does Not
Understand Me
Only lately— and after a long wait—have a lot of smart
people found audiences for making sound points
about what and how we code. Various colleagues have
been beating drums and heads together for ages trying
to make certain that wise insights about programming
stick to neurons. Articles on coding style in this and

other publications have provided fur- turned them all up to 11 in the Varnish
ther examples of such advocacy. source code. One of these tricks has
As with many other educational ef- been called the red-haired stepchild
forts, examples that are used to make of good software engineering and is
certain points are, for the most part, widely shunned by most program-
good examples: clear, illustrative, and mers for entirely wrong and outdated
easy to understand. Unfortunately, reasons. So let me try to legitimize it
the flame kindled by an article read with an example.
over the weekend often lasts only Here is a surprisingly difficult pro-
until Monday morning rolls around gramming problem: What do you do
when real-world code appears on the when close(2) fails?
screen with a bug report that just does Yes, close(2) does in fact return
not make sense—as in, “This can’t an error code, and most programmers
even happen.” ignore it, figuring that either: it can-
When I began writing the Varnish not fail; or if it does, you are in trouble
HTTP accelerator, one of my design anyway, because obviously the ker-
decisions—and I think one of my best nel must be buggy. I do not think it is
decisions—was to upgrade my OCD to OK just to ignore it, since a program
CDO, the more severe variant, where should always do something sensible
you insist letters be sorted alphabeti- with reported errors. Ignoring errors
cally. As an experiment, I pulled to- means you have to deduce what went
gether a number of tricks and practic- wrong based on the debris it causes
es I had picked up over the years and down the road, or worse, that some

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 51
practice

criminal will exploit your code later In Varnish the resulting compro- happens too much, we should handle
on. The one true ideal might appear mise typically looks like this: it better.”
to be, “Keep consistent and carry on,”
but in the real world of connected and AN(vd); retval = strdup(of);
interacting programs, you must make AZ(close(vd->vsm _ fd)); XXXAN(retval);
a careful determination as to whether return (retval);
it is better to abort the program right AN is a macro that means Assert Non-
away or to soldier on through adversi- zero and AZ means Assert Zero, and This distinction is also made in the
ty, only to meet certain ruin later. if the condition does not hold, the dump message, which for AZ() is “As-
Realizing that “I have only a very program core-dumps right then and sert error” vs. XXXAZ()’s “Missing er-
small head and must live with it,”1 sen- there. ror-handling code.”
sible compromises must be made— Yes, the red-haired stepchild I want Where I want to ignore a return val-
for example, a trade-off between the to sell you is the good old assert, which ue explicitly, I explicitly do so:
probability of the failure and the effort I feel should be used a lot more in to-
of writing code to deal with it. There day’s complicated programs. Where I (void)close(fd);
is also a real and valid concern about judge the probability of failure is rel-
code readability—handling unlikely evant, I use two other variants of those Of course, I also use “naked” asserts
exceptions should not dominate the macros, XXXAN and XXXAZ, to signal, to make sure there are no buffer over-
source code. “This can actually happen, and if it runs:

Figure 1. Mini objects. assert(size < sma->sz);

struct lru {
or to document important assump-
unsigned magic; tions in the code:
#define LRU_MAGIC 0x3fec7bb0
... assert(sizeof (unsigned short)
};
... == 2);
struct lru *l;
ALLOC_OBJ(l, LRU_MAGIC); But we are not done yet. One very typ-
XXXAN(l);
...
ical issue in C programs is messed-up
FREE_OBJ(l); lifetime control of allocated memory,
typically accessing a struct after it has
The ALLOC _ OBJ and FREE _ OBJ macros ensure that the MAGIC field is set to the randomly been freed back to the memory pool.
chosen nonce when that piece of memory contains a struct lru and is set to zero when it does not.
In code that gets called with an lru pointer, another macro checks asserts the pointer points Passing objects through void*
to what we think it does: pointers, as one is forced to do when
simulating object-oriented program-
int
foo(struct lru *l)
ming in C, opens another can of
{ worms. Figure 1 illustrates my brute-
CHECK_OBJ_NOTNULL(l, LRU_MAGIC); force approach to these problems.
... In terms of numbers, 10% of the
If the pointer comes in as a void *, then a macro casts it to the desired type and asserts its validity:
non-comment source lines in Varnish
static void * are protected with one of the asserts
vwp_main(void *priv) just shown, and that is not counting
{
what gets instantiated via macros and
struct vwp *vwp;
CAST_OBJ_NOTNULL(vwp, priv, VWP_MAGIC); inline functions.
...
A Method to the Madness
All this checking is theoretically re-
dundant, particularly the cases where
Figure 2. Compile time asserts. function A will check a pointer before
calling function B with it, only to have
function B check it again.
#define CTASSERT(x,z) _CTASSERT(x, __LINE__, z) Though it may look like madness,
#define _CTASSERT(x, y, z) __CTASSERT(x, y, z)
there is reason for it: these asserts
#define __CTASSERT(x, y, z) \
typedef char __ct_assert ## y ## __ ## z [(x) ? 1 : -1] also document the assumptions of the
... code. Traditionally, that documenta-
CTASSERT(sizeof(struct wfrtc_proto) == 32, \ tion appears in comments: “Must be
Struct_wfrtc_proto_has_wrong_size);
called with a valid pointer to a foobar
larger than 16 frobozz” and so on.
The problem with comments is the

52 c omm unic atio ns o f the ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 practice

compiler ignores them and does not guages, I still see plenty of time before it.h> with UCHAR _ MAX, UINT _
complain when they disagree with the progress goes too far and we are no MAX, ULONG _ MAX, but, inexplicably,
code; therefore, experienced program- longer stuck on compilers, but rather USHRT _ MAX, which confused even
mers do not trust them either. Docu- on languages. the person who wrote od(1) for The
menting assumptions so the compiler Compilers today know things about Open Group.
pays attention to them is a much better your code that you probably never This approach has so many things
strategy. All this “pointless checking” realize, because they apply a chess- wrong with it that I barely know where
grinds a certain kind of performance grandmaster-like analysis to it. Pro- to start. If you feel like exploring it, try
aficionado up the wall, and more than gramming languages, however, do not to find out how to portably sprintf(3)
one has tried stripping Varnish of all become better vehicles for expressing a pid_t right-aligned into an eight-
this “fat.” intent; quite the contrary, in fact. character string.
If you try that using the standard- It used to be that you picked a The last time I looked, we had not
ized -DNDEBUG mechanism, Varnish width for you integer variable from even found a way to specify the exact
does not work at all. If you do it a little whatever register sizes your computer layout of a protocol packet and the
bit smarter, then you will find no rel- had: char, short, int, or long. But byte-endianess of its fields. But, hey,
evant difference and often not even a how could you choose between a short it is not like CPUs have instructions
statistically significant difference in and a long if you did not know their ac- for byte swapping or that we ever use
performance. tual sizes? packed protocol fields anyway, is it?
Asserts are much cheaper than they The answer is that you couldn’t, so Until programming languages catch
used to be for three reasons: everybody made assumptions about up, you will find me putting horrors as
˲˲ Compilers have become a lot the sizes, picked variable types, and those shown in Figure 2 in my source
smarter, and their static analysis and hoped for the best. I do not know how code, to try to make my compiler un-
optimization code will happily remove this particular mistake happened. derstand me.
a very large fraction of my asserts, hav- We would have been in much better
ing concluded that they can never trig- shape if the fundamental types had
Related articles
ger. That is good, as it means I know been int8, int16, int32, and int64
on queue.acm.org
how my code works. from the start, because then pro-
˲˲ The next reason is the same, only grammers could state their inten- Reveling in Constraints
Bruce Johnson
the other way around: the asserts put tions and leave the optimization to http://queue.acm.org/detail.cfm?id=1572457
constraints on the code, which the the compiler, rather than try to out-
Sir, Please Step Away from the ASR-33!
static analysis and optimizer can ex- guess the compiler.
Poul-Henning Kamp
ploit to produce better code. That is Some languages—Ada, for exam- http://queue.acm.org/detail.cfm?id=1871406
particularly good, because that means ple—have done it differently, by allow-
Coding Smart: People vs. Tools
my asserts actively help the compiler ing range constraints as part of vari- Donn M. Seeley
produce better code. able declarations: http://queue.acm.org/detail.cfm?id=945135
˲˲ Finally, the sad fact is that today’s
CPUs spend an awful lot of time wait- Month : Integer range 1..12; References
ing for stuff to come in from memo- 1. Dijkstra, E.W. Programming considered as a human
activity (1965); http://www.cs.utexas.edu/~EWD/
ry—and performing a check on data This could be a pretty smooth and transcriptions/EWD01xx/EWD117.html.
already in the cache in the meantime easy upgrade to languages such as C
is free. I do not claim that asserts are and C++ and would provide much- Poul-Henning Kamp ([email protected]) has
totally free—if nothing else, they do needed constraints to modern compil- programmed computers for 26 years and is the inspiration
behind bikeshed.org. His software has been widely
waste a few nanojoules of electricity— er analysis. One particularly strong as- adopted as “under-the-hood” building blocks in both open
but they are not nearly as expensive as pect of this format is that you can save source and commercial products. His most recent project
is the Varnish HTTP accelerator, which is used to speed up
most people assume, and they offer space and speed without losing clarity: large Web sites such as Facebook.
a very good bang-for-the-buck in pro-
gram quality. Door _ Height: Integer range
150..400;
Intentional Programming
In the long term, you should not need This fits comfortably in eight bits,
to use asserts, at least not as much as and the compiler can apply the required
I do in Varnish, because at the end of offset where needed, without the pro-
the day, they are just hacks used to pa- grammer even knowing about it.
per over deficiencies in programming Instead of such increased granu-
languages. The holy grail of program- larity of intention, however, 22-plus
ming is “intentional programming,” years of international standard-
where the programmer expresses his ization have yielded <stdint.h>
or her exact and complete intention, with its uint_least16_t, to which
and the compiler understands it. <inttypes.h> contributes PRIu-
Looking at today’s programming lan- LEAST16, and on the other side <lim- © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 53
practice
doi:10.1145/ 2209249.2209266
˲˲ Metric in a bubble;
Article development led by ˲˲ Treating the metric;
queue.acm.org
˲˲ One-track metric; and
˲˲ Metrics galore.
Four common pitfalls in using software
metrics for project management. Knowing about these pitfalls will
help you recognize them and, hopeful-
By Eric Bouwers, Joost Visser, and Arie van Deursen ly, avoid them, which ultimately leads
to making your project successful. As

Getting What
a software engineer, your knowledge
of these pitfalls helps you understand
why project managers want to use soft-
ware metrics and helps you assist the

You Measure
managers when they are applying met-
rics in an inefficient manner. As an
outside consultant, you need to take
the pitfalls into account when pre-
senting advice and proposing actions.
Finally, if you are doing research in
the area of software metrics, knowing
these pitfalls will help place your new
metric in the right context when pre-
senting it to practitioners. Before div-
ing into the pitfalls, let’s look at why
software metrics can be considered a
Are software metrics helpful tools or a waste of time? useful tool.
For every developer who treasures these Software Metrics Steer People
mathematical abstractions of software systems “You get what you measure.” This
there is a developer who thinks software metrics are phrase definitely applies to software
project teams. No matter what you de-
invented just to keep project managers busy. Software fine as a metric, as soon as it is used to
metrics can be very powerful tools that help achieve evaluate a team, the value of the metric
moves toward the desired value. Thus,
your goals but it is important to use them correctly, as to reach a particular goal, you can con-
they also have the power to demotivate project teams tinuously measure properties of the
and steer development in the wrong direction. desired goal and plot these measure-
ments in a place visible to the team.
For the past 11 years, the Software Improvement Ideally, the desired goal is plotted
Group has advised hundreds of organizations alongside the current measurement to
indicate the distance to the goal.
concerning software development and risk Imagine a project in which the run-
management on the basis of software metrics. time performance of a particular use
We have used software metrics in more than 200 case is of critical importance. In this
case it helps to create a test in which
investigations in which we examined a single snapshot the execution time of the use case is
of a system. Additionally, we use software metrics to measured daily. By plotting this daily
data point against the desired value,
track the ongoing development effort of more than
Illustration by Gary Neill

and making sure the team sees this

400 systems. While executing these projects, we have measurement, it becomes clear to ev-
learned some pitfalls to avoid when using software ery one whether the desired target is
being met or whether the development
metrics in a project management setting. This actions of yesterday are leading the
article addresses the four most important of these: team away from the goal.

54 comm unicatio ns o f th e ac m | j u ly 201 2 | vol . 5 5 | no. 7

cred it t k

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i cat ion s o f t he ac m 55
practice

Figure 1. The lines of code of a software system from January 2010 to July 2011. Even though it might seem simple,
this technique can be applied incor-
rectly in a number of subtle ways. For
Lines of code
350,000 example, imagine a situation in which
325,000 customers are unhappy because they
300,000 report problems in a product that are
275,000 not solved in a timely manner. To im-
250,000 prove customer satisfaction, the proj-
225,000
ect team tracks the average resolution
200,000
175,000
time for issues in a release, following
150,000 the reasoning that a lower average res-
125,000 olution time results in higher custom-
100,000 er satisfaction.
75,000 Unfortunately, reality is not so
50,000 simple. To start, solving issues faster
25,000
might lead to unwanted side effects—
0
Jan Mar May Jul Sep Nov Jan Mar May Jul
for example, a quick fix now could re-
2010 2010 2010 2010 2010 2010 2011 2011 2011 2011 sult in longer fix times later because of
incurred technical debt. Second, solv-
ing an issue within days does not help
the customer if these fixes are released
Figure 2. Measuring lines of code in two different ways. only once a year. Finally, customers
are undoubtedly more satisfied when
Lines of code Lines no fix is required at all—that is, issues
400,000
375,000 do not end up in the product in the
350,000 first place.
325,000 Thus, using a metric allows you
300,000
275,000
to steer toward a goal, which can be
250,000 either a high-level business proposi-
225,000 tion (“the costs of maintaining this
200,000 system should not exceed $100,000
175,000
150,000
per year”) or more technically ori-
125,000 ented (“all pages should load within
100,000 10 seconds”). Unfortunately, using
75,000
metrics can also prevent you from
50,000
25,000 reaching the desired goal, depend-
0 ing on the pitfalls encountered. In the
Jan Mar May Jul Sep Nov Jan Mar May Jul remainder of this article, we discuss
2010 2010 2010 2010 2010 2010 2011 2011 2011 2011
some of the pitfalls we frequently en-
countered and explain how they can
be recognized and avoided.
Figure 3. Measuring number of files used.
What Does the Metric Mean?
Nr. of files Software metrics can be measured on
5,000 different views of a software system.
4,750
4,500 This article focuses on metrics calcu-
4,250
4,000 lated on a particular version of the code
3,750 base of a system, but the pitfalls also ap-
3,500
3,250 ply to metrics calculated on other views.
3,000 Assuming the code base contains
2,750
2,500 only the code of the current project,
2,250
2,000 software product metrics establish
1,750 a ground truth. Calculating only the
1,500
1,250 metrics is not enough, however. Two
1,000 more actions are needed to interpret
750
500 the value of the metric: adding context;
250
0 and establishing the relationship with
Jan Mar May Jul Sep Nov Jan Mar May Jul the goal.
2010 2010 2010 2010 2010 2010 2011 2011 2011 2011 To illustrate these points, we use
the LOC (lines of code) metric to pro-

56 comm unicatio ns o f the ac m | j u ly 201 2 | vo l. 5 5 | no. 7

 practice

vide details about the current size of “How is this metric measured?” in-
a project. Even though there are mul- stead of “What do these data points
tiple definitions of what constitutes tell me?” In most cases the exact con-
a line of code, such a metric can be struction of a metric is not important
used to reason about whether the ex-
amined code base is complete or con- To be useful, for the conclusion drawn from the data.
For example, consider the three plots
tains extraneous code such as copied-
in libraries. To do this, however, the
the value of shown in figures 2 and 3 represent-

the metric should

ing different ways of computing the
metric should be placed in context, volume of a system. Figure 2 shows
bringing us to our first pitfall.
Metric in a bubble. Using a metric
be compared the lines of code counted as every
line containing at least one character
without proper interpretation. Recog- against data that is not a comment or white space
nized by not being able to explain what
a given value of a metric means. Can be
points taken from (blue) and lines of code counted as all
new line characters (orange). Figure 3
solved by placing the metric inside a con- the history shows the number of files used.
text with respect to a goal.
The usefulness of a single data of the project or The trend lines indicate that, even
though the scale differs, these vol-
point of a metric is limited. Knowing from a benchmark ume metrics all show the same events.
that a system is 100,000 LOC is mean-
ingless by itself, since the number of other projects. This means that each of these met-
rics is a good candidate to compare
alone does not explain if the system is the volume of a system against other
large or small. To be useful, the value systems. As long as the volume of the
of the metric should, for example, be other systems is measured in the same
compared against data points taken manner, the conclusions drawn from
from the history of the project or from the data will be very similar.
a benchmark of other projects. In the The different trend lines bring up
first scenario, you can discover trends a second question: “Why does the vol-
that should be explained by external ume decrease after a period in which
events. For example, the graph in Fig- the volume increased?” The answer
ure 1 shows the LOC of a software sys- can be found in the normal way in
tem from January 2010 to July 2011. which alterations are made to this
The first question that comes to particular system. When the volume
mind here is: “Why did the size of the of the system increases, an action is
system drop so much in July 2010?” scheduled to determine whether new
If the answer to this question is, “We abstractions are possible, which is
removed a lot of open source code usually the case. This type of refac-
we copied in earlier,” then there is toring can significantly decrease the
no problem (other than the inclusion size of the code base, which results in
of this code in the first place). If the lower maintenance effort and easier
answer is, “We accidentally deleted ways to add functionality to the sys-
part of our code base,” then it might tem. Thus, the goal here is to reduce
be wise to introduce a different pro- maintenance effort by (among others)
cess of source-code version manage- keeping the size of the code base rela-
ment. In this case the answer is that tively small.
an action was scheduled to drastically In the ideal situation a direct rela-
reduce the amount of configuration tionship exists between a desired goal
needed; given the amount of code that (such as, reduced maintenance effort)
was removed, this action was appar- and a metric (such as, a small code
ently successful. base). In some cases this relationship
Note that one of the benefits of plac- is based on informal reasoning (for ex-
ing metrics in context is that it allows ample, when the code base of a system
you to focus on the important part of is small it is easier to analyze what the
the graph. Questions regarding what system does); in other cases scientific
happened at a certain point in time research has shown that the relation-
or why the value significantly deviates ship exists. What is important here is
from other systems become more im- that you determine both the nature of
portant than the specific details about the relationship between the metric
how the metric is measured. Often and the goal (direct/indirect) and the
people, either on purpose or by acci- strength of this relationship (informal
dent, try to steer a discussion toward reasoning/empirically validated).

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | com m u n i cat ion s o f t he acm 57

practice

Thus, a metric in isolation will not Sometimes, however, parameters

help you reach your goal. On the other are, for example, moved to the fields
hand, assigning too much meaning to of the surrounding class or replaced by
a metric leads to a different pitfall. a map in which a (String,Object)
Treating the metric. Making altera-
tions just to improve the value of a met- The most common pair represents the different param-
eters. Although both strategies reduce
ric. Recognized when changes made to
the software are purely cosmetic. Can be
pitfall is making the number of parameters inside

changes to
methods, it is clear that if the goal is to
solved by determining the root cause of improve readability and reduce future
the value of a metric.
The most common pitfall is mak-
a system just maintenance effort, then these solu-
tions are not helping. It could be that
ing changes to a system just to improve to improve this type of refactoring is done because
the value of a metric, instead of trying
to reach a particular goal. At this point,
the value the developers simply do not under-
stand the goal and thus are treating the
the value of the metric has become of a metric, symptoms. There are also situations,
a goal in itself, instead of a means
of reaching a larger goal. This situa- instead of trying however, in which these non-goal-ori-
ented refactorings are done to game
tion leads to refactorings that simply to reach the system. In both situations it is im-
“please the metric,” which is a waste
of precious resources. You know this a particular goal. portant to make the developers aware
of the underlying goals to ensure that
has happened when, for example, one effort is spent wisely.
developer explains to another develop- Thus a metric should never be used
er that a refactoring needs to be done as-is, but it should be placed inside
because “the duplication percentage a context that enables a meaningful
is too high,” instead of explaining that comparison. Additionally, the rela-
multiple copies of a piece of code can tionship between the metric and de-
cause problems for maintaining the sired property of your goal should be
code later on. It is never a problem that clear; this enables you to use the met-
the value of a metric is too high or too ric to schedule specific actions that
low: the fact this value is not in line will help reach your goal. Make sure
with your goal should be the reason to the scheduled actions are targeted
perform a refactoring. toward reaching the underlying goal
Consider a project in which the instead of only improving the value of
number of parameters for methods the metric.
is high compared with a benchmark.
When a method has a relatively large How Many Metrics Do You Need?
number of parameters (for example, Each metric provides a specific view-
more than seven) it can indicate that point of your system. Therefore, com-
this method is implementing dif- bining multiple metrics leads to a bal-
ferent functionalities. Splitting the anced overview of the current state of
method into smaller methods would your system. The number of metrics to
make it easier to understand each be used leads to two pitfalls, we start
function separately. with using only a single metric.
A second problem that could be sur- One-track metric. Focusing on only a
facing through this metric is the lack of single metric. Recognized by seeing only
a grouping of related data objects. For one (or just a few) metrics on display.
example, consider a method that takes Can be solved by adding metrics relevant
as parameters a Date object called to the goal.
startDate and another called end- Using only a single software metric
Date. The names suggest that these to measure whether you are on track
two parameters together form a Date- toward your goal reduces that goal to
Period object in which startDate a single dimension (that is, the metric
will need to be before endDate. When that is currently being measured). A
multiple methods take these two pa- goal is never one dimensional, how-
rameters as input, introducing such a ever. Software projects experience
DatePeriod object to make this ex- constant trade-offs between delivering
plicit in the model could be beneficial, desired functionality and nonfunc-
reducing both future maintenance ef- tional requirements such as security,
fort, as well as the number of param- performance, scalability, and main-
eters being passed to methods. tainability. Therefore, multiple met-

58 com municatio ns o f the ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 practice

rics are necessary to ensure that your ining the default configuration. When If you are already using metrics in
goal, including specified trade-offs, the tool in question contains, for ex- your daily work, try to link them to
is reached. For example, a small code ample, a check that flags the use of a specific goals. If you are not using any
base might be easier to analyze, but if tab character instead of spaces, the metrics at this time but would like to
this code base is made of highly com- first run of the tool can report an enor- see their effects, we suggest you start
plex code, then it can still be difficult mous number of violations for each small: define a small goal (methods
to make changes. check (running into the hundreds of should be simple to understand for
In addition to providing a more bal- thousands). Without proper inter- new personnel); define a small set of
anced view of your goal, using multiple pretation of this number, it is easy to metrics (for example, length and com-
metrics also assists you in finding the conclude that reaching zero violations plexity of methods); define a target
root cause of a problem. A single met- cannot be done within any reasonable measurement (at least 90% of the code
ric usually shows only a single symp- amount of time (even though some should be simple); and install a tool
tom, while a combination of metrics problems can easily be solved by a that can measure the metric. Commu-
can help diagnose the actual disease simple formatting action). Such an in- nicate both the goal and the trend of
within a project. correct assessment sometimes results the metric to your colleagues and ex-
For example, in one project the in the tool being considered useless perience the influence of metrics.
equals and hashCode methods by the team, which then decides to ig-
(those used to implement equality nore the tool.
Related articles
for objects in Java) were among the Fortunately, in other cases the
on queue.acm.org
longest and most complex methods team adapts the configuration to suit
within the system. Additionally, a rela- the specific situation by limiting the Making a Case for Efficient Supercomputing
Wu-chun Feng
tively large percentage of duplication number of checks (for example, by http://queue.acm.org/detail.cfm?id=957772
occurred in these methods. Since they removing checks that measure highly
Power-Efficient Software
use all the fields of a class, the metrics related properties, can be solved auto-
Eric Saxe
indicate that multiple classes have a matically, or are not related to the cur- http://queue.acm.org/detail.cfm?id=1698225
relatively large number of fields that rent goals) and instantiating proper
Sifting Through the Software
are also duplicated. Based on this ob- default values. By using such a specific Sandbox: SCM Meets QA
servation, we reasoned the duplicated configuration, the tool reports a lower William W. White
fields form an object that was miss- number of violations that can be fixed http://queue.acm.org/detail.cfm?id=1046945
ing from the model. In this case we in a reasonable amount of time.
advised looking into the model of the To ensure all violations are fixed Eric Bouwers (at [email protected] ) is a software
engineer and technical consultant at the Software
system to determine whether extend- eventually, the configuration can Improvement Group in Amsterdam, The Netherlands.
ing the model with a new object would be extended to include other types He is a part-time Ph.D. student at Delft University of
Technology. He is interested in how software metrics
be beneficial. of checks or more strict versions of can assist in quantifying the architectural aspects of
In this example, examining the met- checks. This will increase the to- software quality.
rics in isolation would not have led to tal number of violations found, but Joost Visser ([email protected] ) is head of research
at the Software Improvement Group in Amsterdam,
this conclusion, but by combining sev- when done correctly the number of The Netherlands, where he is responsible for innovation
eral unit-level metrics, we were able to reported violations does not demo- of tools and services, academic relations, and general
research. He also holds a part-time position as professor
detect a design flaw. tivate the developers too much. This of large-scale software systems at the Radboud
Metrics galore. Focusing on too many process can be repeated to extend the University Nijmegen,The Netherlands.
metrics. Recognized when the team ig- set of checks slowly toward all desired Arie van Deursen ([email protected]) is a
full professor in software engineering at Delft University
nores all metrics. Can be solved by reduc- checks without overwhelming the de- of Technology, The Netherlands, where he leads the
ing the number of metrics used. velopers with a large number of viola- Software Engineering Research Group. His research
topics include software testing, software architecture,
Although using a single metric over- tions at once. and collaborative software development.
simplifies the goal, using too many
metrics makes it difficult (or even Conclusion
impossible) to reach your goal. Apart Software metrics are useful tools for
from making it hard to find the right project managers and developers
balance among a large set of metrics, alike. To benefit from the full potential
it is not motivating for a team to see of metrics, keep the following recom-
that every change they make results in mendations in mind:
the decline of at least one metric. Ad- ˲˲ Attach meaning to each metric by
ditionally, when the value of a metric placing it in context and defining the
is far off the desired goal, then a team relationship between the metric and
can start to think, “We will never get your goal, while at the same time avoid
there, anyway,” and simply ignore the making the metric a goal in itself.
metrics altogether. ˲˲ Use multiple metrics to track dif-
For example, there have been mul- ferent dimensions of your goal, but
tiple projects that deployed a static- avoid demotivating a team by using too
analysis tool without critically exam- many metrics. © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l. 55 | n o. 7 | c om m u n ic at ion s o f t he acm 59
contributed articles
d oi:10.1145/ 2209249.2209267
Japan). By exploring the traditional
A searchable meta-graph can connect even expressions of group members across
time and space, scholars can develop
troublesome house elves and other a sophisticated understanding of the
supernatural beings to scholarly folk categories. complex processes of culture develop-
ment and change.
by James Abello, Peter Broadwell, Since inception of the field in the
and Timothy R. Tangherlini 19th century, folkloristics has been
based on a three-part process:

Computational
˲˲ Fieldwork consisting of the identi-
fication of a folk group and the collec-
tion of its traditions;
˲˲ Archiving, editing, and publishing

Folkloristics
these collections; and
˲˲ Analyzing the collected folklore
based on a single or combination of
multiple theoretical perspectives.
For the most part, folklorists limit
their studies to small, well-defined
collections or subsets of much larger
collections. A study corpus is often
selected based on criteria like genre
or topic. The “variants” in the result-
ing study corpus are then subjected to
“close readings”; from an ethnograph-
The study of folklore, or folkloristics, is predicated on ic perspective, close readings focus on
analysis of the symbolic aspects of the
two premises: traditional expressive culture circulates expression as an important meaning-
across and within social networks, and discrete making process for storytellers and
“performances” of traditional expressive forms their audiences. Conclusions drawn
from close readings are subsequently
serve an important role as the locus for the ongoing abstracted to make more general com-
negotiation of cultural ideology (norms, beliefs, and ments about the changing contours of
the cultural ideology of the group in
values). The underlying assumption is that folklore question. This approach has worked
emerges from the dialectic tension between individual well for generations of folklorists, par-
members of a cultural group on the one hand and the ticularly in the context of the relatively
small size of accessible collections
“traditions” of that group on the other. This ongoing and the general alignment of research
tug-of-war ensures that traditional expressive culture
is constantly changing, adjusting to the needs of the key insights
individuals within a group.  T he field of computational folkloristics
weds algorithmic approaches to classic
The goal of any folkloristic investigation is to interpretive problems from the field of
folklore.
understand how traditional expressions create
 A multimodal network representation of
meaning for the people who create and receive them; a folklore corpus (hypergraphs) liberates
folklore exploration from the limitations
studies range from a consideration of fairy-tale of existing classification schemes.
telling by 19th-century peasants (such as in the classic  I magine a system in which the
works of the Grimm brothers) to analysis of rumor complexities of a folklore corpus can be
explored at different levels of resolution,
propagation in the aftermath of disasters (such as from the broad perspective of “distant
reading” down to the narrow perspective
Hurricane Katrina in 2005 and the 2011 tsunami in of traditional “close reading.”

60 com munic atio ns o f th e acm | j u ly 201 2 | vol . 5 5 | no. 7

 questions with existing archival-find-
ing aids.
As folklore collections are increas-
ingly digitized, and as Web-based so-
cial media sites become established
loci for the circulation of traditional
expressive forms, earlier methods of
folklore scholarship have begun to
falter for several reasons: First, it is
increasingly difficult to determine the
boundaries of a study corpus, not only
because existing “finding aids” that
rely on hand-indexing have not kept up
with the growth of digital resources but
also because modern research ques-
tions no longer align with the origi-
nal ideas underpinning the early aids.
Second, the size and scope of folklore
corpora have increased so dramatically
it is impossible to apply close-reading
methods to even a fraction of the items
in a study corpus.
Due to this dramatic change in the
scholarly landscape, where issues of
access no longer pose absolute limits
on the scope of a study, scholarship
in folklore, as in the humanities as a
whole, must adapt to a new environ-
ment where many thousands, if not
millions, of texts are readily available.
At the same time, canonical ap-
proaches to corpus selection that guar-
anteed high precision but also resulted
in low recall are more difficult to de-
fend intellectually. Whereas a scholar
could, until recently, reasonably pro-
pose a study focused exclusively on,
say, the several hundred folksongs
indexed in the Child collection of the
Anglo-American ballad,8 the availabil-
ity of many more related texts from the
same time period or the same cultural
area necessitates that the scholar ac-
knowledge this greatly expanded do-
main. This fundamental change in the
accessibility and size of target domains
is a welcome development but also re-
quires that folklorists augment exist-
Illustrations by J.F. Podevin

ing methodologies to take advantage

of the change in scope.
Along with increased accessibility of
machine-readable primary-source ma-
terials in the form of digitized archives
(such as The Danish Folklore Archive),

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he ac m 61
contributed articles

proaches (such as network-based role torical and geographic environment in

discovery).30 which they lived and worked, and the
These methods augment, rather folklore they created, and also allow for
than supplant, earlier close-reading incorporation of pertinent scholarly
methods prevalent in folklore, allow- feedback annotations. A central com-
ing for a more consistent approach to ponent of such a system can be based
the selection of study materials for a on “hierarchical graph maps,”21 an al-
given research question. Ideally, one gorithmic counterpart to distant read-
should be able to move seamlessly be- ing. Hierarchical graph maps fuse two
tween a bird’s-eye view of the overall of the most common abstractions of
study corpus and the complex inter- visual navigation—graphs and maps.
connections among people, places, They also align with Moretti’s concep-
and artifacts that underpin the corpus tualization of methods suited for the
at one end of the spectrum and the implementation of distant reading in
born-digital archives (such as The Sho- close reading that has characterized a the humanities.23
ah Foundation Visual History Archive), great deal of prior folklore scholarship
and enormous informal repositories of at the other. Limitation of Classification Schemes
personal interaction (such as Facebook Classification is a vexing first-order
and Twitter), there has been an equally Distant Reading problem in folklore. Since the field’s
rapid increase in methodologies for In 2000, Franco Moretti25 proposed the inception in the 19th century, folklor-
search and discovery in large, poorly theoretically rich concept of “distant ists have been concerned with the
labeled corpora (such as Google). How- reading,” anticipating the profound classification of texts, devising nu-
ever, many of these new algorithmic changes in humanities scholarship merous classificatory systems. They
approaches for search, discovery, and presaged by developments in informa- have evolved into a series of overlap-
analysis developed for the Web do not tion technology. In Moretti’s analysis ping and relatively shallow ontologies.
readily translate to the disconnected “Distance… is a condition of knowl- Genre classifications play an impor-
realm of most collections of literature edge: It allows you to focus on units tant role in the organization of most
or folklore. On the other hand, many of that are much smaller or much larger folklore collections, with, say, narra-
the research questions scholars want than the text: devices, themes, tropes— tive folklore being sorted into genres
to address—generally inconceivable or genres and systems.”25 Distant read- (such as myth, folktale, legend, rumor,
prior to wide-scale availability of large ing is thus a complementary approach fairy tale, and ballad).3 Such classifica-
digital corpora—demand more target- to the close-reading approaches that tions are problematic as a model of
ed approaches than those developed have characterized humanities re- folklore production, given the vast ar-
for the biological and physical scienc- search for centuries. It is a particularly ray of native genre distinctions from
es, scientific co-citation networks, and apt approach for folklore, since folklor- target cultures that do not align with
e-commerce. Availability of substan- ists are not only interested in the par- scholarly categories.12 Other efforts
tial digital folklore corpora ultimately ticular features of a discrete text but have concentrated on organizing folk-
speaks of the need for computational also the much broader picture of how lore by topics within a single genre
folkloristics. that discrete text (or performance) fits or group of closely associated genres
As a sub-discipline of folklore, com- into the wide range of traditional ex- (such as folk tale and legend). Perhaps
putational folkloristics encompasses: pression through time and space. best known is “The Types of Interna-
˲˲ Digitization of existing resources We endorse this view of the need to tional Folktales” commonly referred
or collection of new resources in ma- fuse close reading and distant read- to as the ATU index.37 The underlying
chine-actionable form; ing. The backbone of computational indexing philosophy of folklore type
˲˲ Development of extensible data folkloristics must leverage current indices, which exist for specific na-
structures for description and storage technological developments that allow tional traditions9 or specific genres,10
of these resources; us to semantically interconnect large are best summed up as “one story/
˲˲ Novel methods for classifying folk- amounts of data and explore them one classification.” A slightly more
lore data based not only on existing relatively easily in integrated analyti- fine-grain index that allows for greater
classification schemes and metadata cal and visual environments inexpen- overlap at the unit of “tale” focuses
but also on surface-level phenomena sively. Here, we outline some of the on “motif,” or constitutive narrative
(such as the linguistic features of texts); components of an ideal computational element of a tale. In it, a tale consists
˲˲ Domain-sensitive methods for folkloristics system and the main ele- of a series of motifs, and it is the mo-
search and discovery; and ments that provide quick classification tifs, rather than entire tales, that are
˲˲ Algorithmic methods for cor- of a large, poorly labeled corpus cou- indexed, then cross-referenced to
pus study, including visually rich pled with intuitive visual navigation. published collections.13,36 In terms of
approaches that fuse statistical rep- This system would allow a researcher search and retrieval, these traditional
resentations of the data with appro- to explore the complex relationships classifiers provide a high degree of
priate historical maps,35 as well as among tradition participants (the precision but, due to that precision,
combinatorial graph analytical ap- people in the tradition group), the his- sacrifice a great deal of recall. Recall

62 communicatio ns o f the ac m | j u ly 201 2 | vol . 5 5 | no. 7

 contributed articles

is so low that searches based on the collected folklore from approximately in the published collection.
indices miss many potentially inter- 4,000 named individuals whose ages Since 1999, we have been digitiz-
esting variants. Although these clas- and life histories he meticulously re- ing and transcribing not only the pub-
sificatory schemes have served folk- corded. His collection spans more lished collection but also an unpub-
lorists for so long, the systems fail than 24,000 manuscript pages, includ- lished collection consisting largely of
dramatically when suddenly address- ing nearly a quarter-million stories, field-collection manuscript pages. We
ing tens of thousands or even hun- songs, proverbs, riddles, rhymes, and have extracted from Tang Kristensen’s
dreds of thousands of texts from pub- descriptions of everyday life. Approxi- memoirs, letters, and personal pa-
lished and unpublished collections. mately one-third of this collection has pers the names of his informants and
High-precision indices (such as been published in a series of printed the places these people lived. These
standard folklore type and motif) have volumes, organized first by genre and places, along with places mentioned
a second problem—cost of implemen- subsequently indexed according to in their stories, have been geo-coded
tation. When target collections are rel- Tang Kristensen’s own idiosyncratic using historical gazetteers. We also
atively small—dozens of examples to topic categories; these categories are extracted Tang Kristensen’s collecting
perhaps hundreds—classification sys- not consistent across the 80 volumes routes through Denmark indexed to
tems can be applied through manual
methods. But when target collections
expand to 10,000 or more examples,
they become much too costly to imple-
ment. This cost is amplified by the
need to revise and update existing indi-
ces and re-index existing collections, a
problem that is particularly acute when
the archives are open-ended and con-
tinuously growing (such as blog posts
about uprisings in the Middle East).
As a simple, historical example from
the Danish materials, no one has yet
classified (according to the ATU index)
the several thousand fairy tales in the
collections of the Danish Folklore Ar-
chive (http://www.dafos.dk), nor does it
seem anyone ever will. We may well be
nearing the end of the useful lifespan Figure 1. A simple GUI for exploring a subset of a folklore archive, here the Evald Tang
of single-classifier schemes, given their Kristensen collection, includes historical maps and the ability to drill down to images of the
collector’s field notes.
implementation costs and inability to
describe the complexity of tradition. So
how does a scholar perform search and
discovery to answer research questions
in folklore? What is needed is a system
for the automatic indexing of a collec-
tion that is flexible enough to allow for
shifting priorities of folklore research-
ers and that can adapt to a sudden
increase in the scale of the target do-
main. In the following sections, we de-
scribe one such approach to problems
in search and discovery based on a net-
work model of a folklore collection. As
our test corpus, we use a subset of the
Tang Kristensen collection of Danish
folklore housed at the Danish Folklore
Archives in Copenhagen.

Ghost Stories?
Evald Tang Kristensen, a schoolteach-
er in Jutland, Denmark, is today widely
regarded as the most prolific folklore
Figure 2. The faceted browsing system ETKspace (based on the mSpace faceted browser)
collector ever.35 Over the course of his allows users to quickly select a subset of the overall corpus based on multiple criteria (such
active collecting career, 1865–1923, he as storyteller, places mentioned, places of story collection, and date of collection).

ju ly 2 0 1 2 | vol . 55 | n o. 7 | c om m u n i c at ion s of t he acm 63

contributed articles

Table 1. Attributes for each story text.

Attributes Term frequency Evald Tang

Notes of keywords Shallow-ontology Kristensen index Place Names Personal Names
Description A term frequency A shallow, hierarchical ontological A topic index to each A geo-referenced An index of the people
count of keywords representation of each text of Tang Kristensen’s representation of mentioned in a story
selected from in the corpus. The nine top published collections. places mentioned and the storytellers.
the vocabulary categories—Actions or Events; in a story and the
of the entire corpus; Animals; People; Places; place where a story
keywords can Resolution; Stylistics; Super- was collected.
be bigrams. natural Beings; Time, Season,
Weather; Tools, Items, and
Conveyances—along with 125
second-level categories, offer
a hierarchical overview of the
content elements of the story.
Theory The underlying theory The underlying theory of this Nearly all published Folklore theory Recent folklore theory
of this approach is approach is predicated on folklore collections include has always been emphasizes the role of
known as “bag of words,” Vladimir Propp’s Morphology of their own topic indices concerned with storytellers in shaping tradition
where the vocabulary of the Folktale28 and, later, Alan based on the collector’s the relationship as part of their own creative
a given text is recognized Dundes’s refinement of that or editor’s evaluation of between traditional and ideological expression.31
as meaning bearing. work.12 Dundes proposed that “what the story is about.” expression and People mentioned in stories
individual elements of a story In Tang Kristensen’s the physical help situate the stories in
(allomotifs) often fulfill a struc- publications, he separated environment.22 the local social environment.
tural role in the narrative the collections Inclusion of
(motifemes). The shallow first according to geo-referenced place
ontology first developed in genre, then according names here allows
Tangherlini34 catalogs the stories’ to “top level” topics one to explore the
content on multiple levels, allow- (such as “Witches geographic location
ing comparison across stories and their Games”), of stories in regard
even when the vocabularies of then into more specific to attributes related
the stories are divergent. categories (such as to content.
“Driving with three
wheels”).
Computed? Yes. Keywords were Not yet. No. Partially. The identifi- Partially. Named-entity
identified using AutoMap. cation and alignment detection implemented in
The vocabulary of each of place names with Mallet was used to generate
individual story was historical place name a list of potential personal
subjected to a stop word gazetteers is a semi- names from the corpus.24
filter (articles, pronouns, supervised operation; This list was aligned with
prepositions, conjunctions) the de-duplication the partial list of personal
and rudimentary stem- of place names in names in one of Tang
ming (Snowball stemmer). the corpus was Kristensen’s indices.
A term-frequency count assisted by DDupe.4
for each individual story
was computed based on
this limited vocabulary.
Benefits Can be computed quickly. Allows for comparison of These are preexisting Places mentioned The emphasis in folklore
stories that share ontologically indices that represent the reveal association theory on the role of the
similar content, even when collector’s or editor’s his- between topics and individual in the creation of
the vocabulary is divergent. torical view of the collec- places in the real tradition and the highly localized
tion; folklorists are familiar world made by and historicized nature of much
with these indices. storytellers. folkloric expression can be
captured by this index.
Challenges Although Danish is not For this work, we applied The indices fall prey Many of the places Danish names are complex.
and highly inflected, there is the shallow ontology by hand to the “one-text-one- mentioned in stories A century of at times contradic-
drawbacks a need to at the very and are working on automating classification” pitfall are difficult to resolve tory laws have led to a situation
least stem the the process using DanNet, of early folklore to existing place- where many people have a last
vocabulary. Umlaut the Danish version of WordNet. classification systems. name gazetteers. name based on a patronymic
and limited cases of Some place names ending in -sen, with a limited
syncope introduce have changed over number (approximately 116)
inaccuracies into the the years, some have possible first components (such
stemming. Without ceased to exist, and as Pedersen from Peder’s son)
stemming, inflected others are only known and a second last name derived
forms of words are in local culture. entirely from place names.
counted as discrete Small orthographic The former phenomenon makes
lemmata. Bag-of-words variations in it difficult to align people
approaches discard all place-name spelling mentioned with external archival
important grammatical also contribute to resources (such as a census),
information that contrib- some uncertainty. while the latter makes it difficult
utes to meaning making. to easily discern between place
names and personal names.

64 comm unicatio ns o f the acm | j u ly 201 2 | vo l. 5 5 | no. 7

 contributed articles

places, people, and time. Moreover, we folklore collectors, and storytellers, as

have searched census data and church their country struggled toward greater
records to situate his informants in democratic freedom and free-market
both space and time. As part of this ef- organization. It also makes an excel-
fort, we developed two basic modules lent testbed for developing and testing
to address corpus navigation: The first tools for computational folkloristics.
is a relatively simple study environ- One of Tang Kristensen’s published
ment appropriate for exploration of a collections of legends includes the
subset of a larger corpus incorporat- following story: “It was the old coun-
ing rich biographical and interpretive selor from Skårupgård who came rid-
data, along with historically accurate ing with four headless horses to Tod-
maps (see Figure 1). The second is a bjærg church. He always drove out of
faceted browsing application intended the northern gate, and by the gate was
to help select corpus resources that can a stall. They could never keep the stall
be ported to more sophisticated study door closed. They had a farmhand cation schemes for the vast majority of
environments (see Figure 2). who closed it once after it had sprung folklore collections.
Although the work is far from com- open. But one night, after he’d gone to
plete, this growing window into the bed, something came after him and it Folk Story Hypergrahs
daily life of the rural population of late- lifted his bed straight up to the rafters Since folklore can be classified accord-
19th-century Denmark offers an intrigu- and crushed him quite hard. Then the ing to numerous criteria, connecting
ing glimpse into the conflicting agen- farmhand shouted and asked them to items together based on them allows
das—cultural, economic, intellectual, stop lifting him up there. ‘No, you’ve information to flow throughout the
and political—of scholars, amateur tormented us, so now you’ll die...’ I corpus. That is, building a multi-
heard that’s how two farmhands were modal network representation of the
Table 2. Attributes associated with DS IV crushed to death. He wanted to close corpus offers a stark contrast to the
650, the misclassified ghost story.
the door and then they never tried to compartmentalization of information
close it again.”32 represented by a singly classified print-
The story concerns a vengeful ed text model. As a bonus, a network
ETK Index Manor lords, ladies
and mistresses
haunt, yet Tang Kristensen classified it model of the corpus opens up the ap-
Places Mentioned Skårupgård, Todbjærg
as being about unnamed manor lords. plicability of multi-resolution network
Personal Names [none]
Consequently, a researcher interested analysis. The intention is to represent
in the relatively simple topic of ghost folklore as a dynamic hypergraph of
keyword frequency stories in late-19th-century Denmark people connected in time and space to
bed 2 would be unable to discover the story places and their stories. This represen-
church 1 through existing finding aids. Even in tation is extensible as more resources
counselor 1 the digital realm, the discovery of the become available. One can thus imag-
death 3 story as a ghost story is a challenge. ine a system that stores the social net-
door 4 Since it makes no overt mention of works of tradition participants, the
farm_hand 3 ghosts, simple keyword searching academic networks of scholars and
headless_horse 1 on Danish terms for ghosts (spøgelse, collectors, communication and trans-
night 1 genganger) would fail. Experiments portation networks linking geographi-
north 1 with supervised text classifiers, in this cal places, repertoire networks linking
old 1 case a naïve Bayes learner trained on storytellers to their stories, and finally
rafters 1 200 stories labeled “ghost stories” by linguistic and semantic networks that
riding 1 Tang Kristensen, also fails to classify link stories. The underlying theory is
stall 3 the story as a ghost story. Indeed, only that it is difficult to find something if
torment 1 a relatively high degree of domain ex- it is not connected to something else.
pertise would allow a researcher to find By connecting the three main actors
Shallow ontology Entry
the story, recognizing that headless of the folklore equation—people (sto-
Resolution Negative
animals are often related to hauntings. rytellers and scholars), places (where
Animals Farm animal
Unfortunately, headless animals are stories were collected and mentioned
Animals Horse
also related to portents and the devil in in stories), and stories (or folkloric ex-
Places Barn
Danish tradition; thus the recall of sto- pression in general)—a researcher can
Places Church
ries for a keyword search on the Danish discover not only specific texts but pat-
Actions or Death
term for “headless” (hovedløs) would terns in the network data not readily
Events
trigger the opposite problem with far apparent if the collection is presented
People Farmhand/Shepherd
greater recall than is probably help- in disconnected fashion. The result-
Supernatural Ghost/Revenant
Beings ful. The story is not an anomaly in the ing model of a folklore collection rests
collection but rather illustrative of the on a series of graphs, each describing
underlying problem of existing classifi- a mode of the complex folklore hy-

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n ic at ion s o f t he acm 65
contributed articles

pergraph or meta-network. This hy- systems of classification (such as the our initial limited experimental dataset
pergraph model of folklore offers the tale type, motif indices, and collection- of stories; Table 1 lists the attributes
potential for productively spanning specific indices) but incorporate the for stories in the corpus, and Table 2
the distant-reading and close-reading information from them into the story lists the attributes of a single story.
spectrum explored earlier. representation. More generally, the The dataset includes 342 storytell-
chosen story representation should of- ers and 942 stories, along with Tang
Connections fer mechanisms for incorporating ac- Kristensen’s topic index for two col-
Connecting texts to one another is cumulated scholarly knowledge. Con- lections—Danske sagn33 and Danske
one of the most complex tasks of the sequently, our approach is to model sagn, ny række32—as the basis for the
hypergraph-generation process. Net- each story as an attribute-valued vec- network. We supplemented this infor-
works describing the connection of tor, where the values of some attributes mation with two additional weighted
stories to their tellers and stories to are known a priori and others are com- graphs: The first was based on a simple
the places they mention are relatively puted. Some attributes may take as val- “shared keyword” weight computed
easy to generate since they are mainly ues either simple scalars or “links” to for each pair of stories. Using AutoMap
an alternative representation of the more complex structures or processes. we generated a set of 1,201 keywords
corresponding databases. More com- Other attributes may have associated shared among all stories across the
plex is generating <story to story> with them time-varying functions re- corpus, after eliminating pronouns,
networks based on multi-objective cri- cording a sequence of attribute values prepositions, articles, and conjunc-
teria. As a case in point, one criterion over time, or a time series. We did not tions.7 This bottom-up approach gave
of our work is to not abandon earlier incorporate time-varying attributes in us an important view of the corpus
based on shared vocabulary across
texts. The second was a top-down ap-
proach that categorized stories accord-
ing to a shallow ontology for the corpus
developed specifically for the realm of
Danish folklore.35 Such shallow on-
tologies depend on domain expertise,
providing a view of the corpus attuned
to the “tradition dominants” of a par-
ticular tradition group.15,21,26 The use
of natural language processing (NLP)
tools in concert with DanNet, the Dan-
ish version of WordNet, may allow us
to develop further the first-generation
shallow ontology for the Danish folk-
lore corpus.27,29 Topic-modeling meth-
Figure 3. Overview of the hypergraph browser for computational folkloristics. The target ods (such as Latent Dirichlet Alloca-
ghost story (story id 340) is upper left, identified with a landmark. The story of the house elf tion and Latent Semantic Analysis)
(nisse) is right of center, also identified with a landmark (story id 348). The upper navigation
screen shows a tree view of the organized hypergraph; the left panes provide navigation and could be used as additional methods
selection; the lower right includes a graph-navigation toggle. This hypergraph browser is for generating a <story to story>
based on Abello et al.2
graph.6,11,14 The advantage of the hyper-
Figure 4. Overall story space for the test corpus, with a cluster selected for further inquiry. graph model, or multimodal network,
is that new network representations
can be added to the overarching mod-
el as their applicability to the study of
folklore is further understood.
Our hope is that the end result is a
system with applicability for research-
ers working with a range of folklore
and ethnographic collections and ul-
timately for any collection or series
of collections of culturally expressive
forms. In our own earlier work, we ex-
plored development of a shallow ontol-
ogy for the Danish folklore collection
as a part of a method for revealing nar-
rative tendencies in the corpus based
on four attributes of the storytellers:
gender, occupation/class, age when
the stories were told, and education.35

66 c om municatio ns o f th e acm | j u ly 201 2 | vol . 5 5 | no. 7

 contributed articles

In 2000, when digitizing large collec- collection of key words of interest, or Figure 4). Based on this information,
tions became practical, we revisited known scholarly categorization. The a researcher might decide to place a
the coding of the stories from the 1994 system then brings up both a visual landmark tag in one of the story’s ele-
study35 and began refining the story- network representation of a “cluster” ments, get a projection of the meta-
attribute set into a consistent shallow of stories related to the target story, to- network on a particular subset of at-
ontology. We further refined the shal- gether with a Web link, to a color-cod- tributes, and/or provide feedback on
low ontology in 2008–2009 to develop ed textual representation of the related the perceived quality of the system’s
a geo-referenced topic browser for a stories and other pertinent scholarly suggested answer (see Figure 5). At
collection called the “Danish Folklore information (such as authors, places this point, the researcher can visually
Nexus” (http://projects.cdh.ucla.edu/ mentioned, and historical context or navigate to clusters of stories adjacent
danishfolklore) (see Figure 1) and a period where the story is placed) (see to the system-provided cluster, select
faceted browsing system for Danish
folklore still under development (http:// Figure 5. Close-up view of a selected story cluster focusing on stories concerning
economic threat.
etkspace.ucla.edu) (see Figure 2). Over
2008, the initial test corpus was coded
Note the clear semantic divide between terms in the upper half of the cluster graph
with attributes from the shallow ontol-
and terms in the lower half, with those in the upper half focusing on historical figures
ogy in a semi-supervised system. Tang- and those in the lower half focusing on supernatural figures.
herlini, an expert in Danish folklore,
checked automatic assignments made
by AutoMap’s thesaurus function for
accuracy.7 This iterative process of
expert checking of automatic assign-
ments also yielded further refinement
of the ontology.

Typical Folklore Tasks

The resulting network for our test cor-
pus includes 2,973 nodes with 52,663
edges, and the system’s design aims to
address two questions emblematic of
regular research problems in folklore:
˲˲ Given a poorly labeled target story
(such as the ghost story mentioned
earlier), how can the system place the
target story in a neighborhood of “simi-
lar” stories?; and Figure 6. Three related story clusters, expanded for discovery of candidate stories for
further exploration.
˲˲ Can the system suggest candidate
stories that would likely be of interest
to a folklore researcher?
The two related clusters are related more closely with supernatural threats. The cluster on the left
We addressed them through the fol- is bifurcated, with stories at the top related to robbers, theft, and murder, and those at the bottom
lowing basic computational modules related to Satan. In the cluster on the right, the stories at the top involve witches and animals related
of a system: to Satan, with both categories representing significant economic threat. The stories at the bottom
involve interaction with mound folk, a supernatural group closely related to economic issues.
Hypergraph Constructor → Hyper-
graph Projector → Hierarchical Graph
Map Constructor1 → Graph Naviga-
tor2 → Visual and Textual Selectors →
Landmark Annotators → Web Browser
Finder → Answer Verifier → Report
Generator → Hypergraph Curator
The system interface lets users in-
teract with each computational mod-
ule through textual input/output, tex-
tual search, drop-down menus and
tabs, mouse selectors, visual sliders,
zooming, a navigation wheel, a land-
mark annotator, and statistical reports
(see Figure 3).
Initial user interaction amounts
to specifying a target story descrip-
tor—choosing from a story identifier,

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | com m u n i c at ion s o f t he acm 67

contributed articles

another story, and iterate the process tree. These cluster labels can be used
(see Figure 6). The system keeps the to help provide and receive feedback
user-selected landmarks as aggregated from scholars studying the corpus, as
counts that can be used by authorized well as judge the effectiveness of the
story “curators” to update the existing
hypergraph for future analysis. This in- The trouble suggested classification.

corporation of scholarly feedback is es-

sential for the system’s performance,
with house elves Performance on the
Two Folklore Tasks
as it allows for the aggregation of ac- begins with The first task we included in the sys-
cumulated expert knowledge, which
in turn helps guide researchers as they
their unpredictable tem was to place the target story in a
neighborhood of closely related ghost
navigate the hypergraph. nature, which, stories. The system succeeded admira-
The graph-theoretic techniques we
applied to generate the final hierarchi-
for folklorists, bly, moving the story from a group of
stories about manor lords in the origi-
cal graph map involve four major tasks: makes them nal classification scheme to a neigh-
˲˲ Define similarity measures in
stories; difficult to track borhood of stories about ghosts and
other supernatural beings that haunt
˲˲ Construct a weighted graph among through the workers in farm buildings and barns—
stories, where the weight of the connec-
tion between two stories is obtained landscape of a new implicit category that did not
exist in Tang Kristensen’s original in-
through order-preserving transforma-
tions of their similarity measures;
the story corpus. dices. Although we left success criteria
for this work fuzzy, we placed the target
˲˲ Decompose the graph(s); and story in the same part of the organized
˲˲ Compute a “hierarchy tree” for the meta-graph as several ghost stories
obtained graph(s). exhibiting certain topical similari-
The similarities measures we used ties; for example, another story where
are based on several algorithms, in- a haunt makes it impossible to use
cluding: Hellinger distance,23 weighted part of the farm appears nearby in the
Jaccard coefficient,19 cosine similar- hypergraph: “A woman died in a farm
ity,18 and scholar-defined weights. The in Dokkedal… and she showed herself
measures are transformed through every noon at a specific place in one of
Gaussian kernels and other standard the rooms and always as a black shape.
kernels20,30 that assign to every pair of Because of that, the room stood empty
stories that share at least a minimum for a long time, since nobody dared go
set of attributes a corresponding simi- in there...”33
larity weight. In general, the graphs ob- Perhaps more interesting is the sys-
tained through these transformations tem’s ability to meet the second task—
vary depending on the type of simi- suggest candidate stories, given the
larity measure used. Consequently, researcher’s interest in a target story.
we obtain a final weight between two In this case, the visual representation
stories by computing a norm of their of the meta-network reveals a close af-
similarity-weights vector. Because the filiation of the target story with a story
obtained graph has different density not classified by Tang Kristensen as a
“regions,” we follow the approach de- story about ghosts but as a story about
scribed in earlier work by Abello et al.,2 house elves (nisse), a category of super-
decomposing the collection of stories natural beings not usually considered
into maximal sub-graphs according to thematically related to ghosts: “When
their inherent k-connectivity. Each ob- they got home, the farmhand was
tained sub-graph is then clustered us- happy because now he’d gotten some-
ing an adapted version of Markov Clus- thing to use for feed, and afterward
tering.2 The entire process is encoded nis [a house elf] could go and feed the
in a data structure called a “hierarchy animals just as much as he wanted to.
tree” visually represented in the user Then they got another farmhand, and
interface as a hierarchical graph map he didn’t want to let him go on like
(see Figure 3),2 constituting one of the that. But he got lifted up in his bed
central modes of interaction between and all the way up to the rafters, so he
user and story corpus. Worth noting is lay there dead when people got up the
that each cluster includes a unique as- next morning.”33
sociated label string that encodes its The intersection between the target
“semantic” placement in the hierarchy story and this latter story, likewise not

68 com municatio ns o f th e ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 contributed articles

classified as a ghost story and com- ed clusters, a feature we have not yet
pletely lost to the researcher using ear- implemented. The system could then
lier finding aids to find ghost stories in incorporate this researcher-generated
the corpus, suggests the broad applica- feedback in its labeling algorithms.
bility of the method. A researcher with The more expert feedback the system
a specific story in hand can find stories incorporates, the more definitive and
that relate to that story on a thematic useful the cluster labels it could gener-
level, while another researcher, with- ate. Researchers’ inability to provide
out a particular target story in hand, a short textual description of a given
can discover neighborhoods of “like- cluster could suggest the need for revi-
minded” stories. By adding or sub- sion of similarity measures and cluster
tracting network modes or by testing methods. Such interaction between a
different weights for edges in different folklore system and researchers should
projections of the overall meta-net- be central to improving the quality of
work, a researcher can quickly reorder this type of classification system. inconsistent orthographies, at times
the folklore meta-graph for ongoing varying from collector to collector.
discovery of information links not eas- Extending the System Digitization will require close collabo-
ily discovered through existing finding The goal of a scalable multilingual ration among archivists, librarians,
aids. This information can be added folklore system necessarily relies on computer scientists, and domain spe-
back into the now dynamic representa- NLP. Folklore has been collected in cialists. Indeed, one reason we chose
tion of the folklore corpus, allowing for many countries and languages and the Danish folklore archive as our ini-
more sophisticated analysis of stories from many eras. Each language poses tial target corpus was our team’s rep-
and story patterns. special challenges, from transcrip- resentation in all these fields. Finally,
To the best of our knowledge no tion and representation in machine- many folklore collections exist almost
well-established criteria exist for judg- readable form to morphological and entirely as audio and visual recordings,
ing the performance of such a compu- syntactic complexity.a Higher-level posing significant challenges for digi-
tational system. Evaluation requires problems (such as named-entity detec- tal archiving and requiring significant
specification of a series of well-defined tion and disambiguation, extracting advances in NLP technology before
tasks that engage potential users in a structural elements, and sentiment de- they are accessible to a computational
“natural manner.” The two tasks we tection) complicate the representation folklore system.
defined earlier are typical of folklore of texts significantly. Our system does The system’s generalizability de-
researchers, and the system performs not yet include narrative structures as pends on these advances and develop-
well in answering the folklorist’s ques- attributes of a story, as explored by El- ers’ ability to automatically recognize
tions. We also imagine other natural son and McKeown14 and Finlayson,16,17 and code narrative features and nar-
tasks that might be used to evaluate Our own future work will include cod- rative structures. As part of the effort
and refine the system. Two that seem ing of narrative structure into the at- to extend our work to other corpora,
fundamental are “multiple key attri- tribute set of a given story, as it should domain experts could develop shal-
bute searching” and “cluster labeling.” add additional insight into the target low ontologies for other tradition areas
Because folklore research is predicated folklore corpora. Finally, our system and other types of expressive forms.
on understanding the “context” of a does not include sentiment detection, Shallow ontologies for, say, Korean or
cultural expressive form, the visual rep- though it would be a useful extension Mexican folklore would have to include
resentation of the search results must to be able to label texts or parts of texts types of supernatural beings beyond
be a projection onto the overall hierar- as expressing doubt or uncertainty or those currently described in the sys-
chical-graph-map view of the corpus. characterizing the emotional aspects tem; for example, the Korean tokkaebi,
Answers must be provided at human- of characters in stories, but it lies be- a blue gremlin with a particularly nasty
interactive rates, a performance crite- yond our current capabilities. disposition, and the legendary Mexi-
rion our current system meets after the One remaining yet potentially re- can chupacabra, feared for its habit
initial hierarchical graph map for the warding challenge is the large-scale of attacking and sucking the blood of
corpus has been computed. digitization of folklore archives around goats, would have to be incorporated
Cluster labeling is an important the world. Many folklore collections are into ontologies describing these tradi-
task for researchers in light of this new based on recordings of spoken dialects tions. The advantage of shallow ontol-
interactive environment for the study or languages with poorly described or ogy is that it is extensible at the level of
of a folklore corpus. Hierarchical graph nonexistent written grammars; not specific manifestations (such as nisse,
maps are presented to the researcher surprisingly, many of them also have tokkaebi, duende, and chupacabra)
visually and include the string-labeled while also gathering these supernatu-
representations of the clusters in the a Our group is developing a system for automat- ral beings into a single higher-level cat-
hierarchy tree; the system computes ed lemmatization and morpho-syntactic tag- egory as well. Researchers can further
ging for Old Icelandic (NSF # BCS-0921123);
the cluster labels. Researchers should modern Icelandic inflection is fully described,
extend the ontological categories to de-
also be able to provide textual feed- allowing for more rapid lemmatization of scribe other cultural expressive forms,
back about the labels for their select- modern Icelandic texts.5 from literary works (such as the novel)

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 69
contributed articles

to narrative expression (storytelling) in Many of the ideas were developed at the Conference on the World Wide Web (Chiba, Japan, May
10–14). ACM Press, New York, 2005, 641–650.
social media. National Endowment for the Humani- 19. Gasperin, C., Gamallo, P., Agustini, A., Lopes, G., and
ties Institute for Advanced Topics in Lima, V.D. Using syntactic contexts for measuring
word similarity. In Proceedings of the ESSLLI
Conclusion Digital Humanities through “Networks Workshop on Semantic Knowledge Acquisition and
In rural Denmark in the 19th century, and Network Analysis for the Humani- Categorisation (Helsinki, Aug. 13–17). ESSLLI,
Helsinki, 2001, 18–23.
stories would reflect the trouble farm- ties” NEH grant HT5001609. The work 20. Hofmann, T., Schölkopf, B., and Smola, A.J. Kernel
ers had with everything from ghosts of James Abello is partially supported methods in machine learning. Annals of Statistics 36,
3 (June 2008), 1171–1220.
to house elves. The trouble with house by the Center for Discrete Mathemat- 21. Jiang, X. and Tan, A.-H. Mining ontological knowledge
elves begins with their unpredictable ics and Theoretical Computer Science from domain-specific text documents. In Proceedings
of the Fifth IEEE International Conference on Data
nature, which, for folklorists, makes (DIMACS), Rutgers University, Piscat- Mining (Houston, Nov. 27–30). IEEE Computer Society,
Los Alamitos, CA, 2005, 665–668.
them difficult to track through the land- away, NJ, “Special Focus on Algorith- 22. Krohn, K. Die Folkloristische Arbeitsmethode. H.
scape of the story corpus. This explora- mic Foundations of the Internet” NSF Aschehoug & Co., Oslo, 1926.
23. Le Cam, L.M. and Yang, G.L. Asymptotics in Statistics:
tion of the contours of computational grant #CNS-0721113, and mgvis.com Some Basic Concepts. Springer-Verlag, New York,
folkloristics and our description of pre- (http://mgvis.com), New Jersey. 2000.
24. McCallum, A.K. MALLET: A Machine Learning for
liminary experiments in multimodal Language Toolkit. University of Massachusetts,
network classification for folklore cor- References
Amherst, 2002; http://mallet.cs.umass.edu
25. Moretti, F. Conjectures on world literature. New Left
pora offer not only the possibility of be- 1. Abello, J. Hierarchical graph maps. Computers & Review 1 (Jan.–Feb. 2000), 54–66.
Graphics 28, 3 (June 2004), 345–359.
ing able to track house elves but promis- 2. Abello, J., van Ham, F. and Krishnan, N. ASK-
26. Paneva, D., Rangochev, K., and Luchev, D. Ontological
model of the Knowledge in Folklore Digital Library. In
ing directions for future work. GraphView: A large-scale graph visualization system. Proceedings of the Fifth HUBUSKA Open Workshop
IEEE Transactions on Visualization and Computer
One troubling aspect of working Graphics 12, 5 (Sept./Oct. 2006), 669–676.
on Knowledge Technologies and Applications, T.
Urbanova, I. Simonics, and R. Pavlov, Eds. (Kosice,
with a large amount of humanities 3. Bascom, W.R. The Forms of Folklore: Prose Narratives. Slovakia, May 31-June 1). HUBUSKA, Kosice, Slovakia,
University of California, Berkeley, 1965. 2007, 47–55.
data is that scholars often cannot see 4. Bilgic, M., Licamele, L., Getoor, L., and Shneiderman, 27. Pedersen, B.S., Nimb, S., and Trap-Jensen, L. DanNet:
the forest for the trees, to borrow a B. D-Dupe: An interactive tool for entity resolution Udvikling og anvendelse af det Danske WordNet.
in social networks. In Proceedings of the IEEE Nordiske Studier i leksikografi 9 (2008), 353–370.
folk expression. Moretti spoke con- Symposium on Visual Analytics Science and 28. Propp, V.I.A. Morfologija Skazki. Academia, Leningrad,
vincingly of distant reading, a correc- Technology (Baltimore, Oct. 31–Nov. 2). IEEE, 1928.
Piscatway, NJ, 2006, 43–50. 29. Sanfilippo, A., Tratz, S., Gregory, M., Chappell, A.,
tive to the long-standing tradition in 5. Bjarnadóttir, K. Um Beygingarlýsingu íslensks Whitney, P., Posse, C., Paulson, P., Baddeley, B.,
the humanities of very close reading.25 nútímamáls, 2009; http://www.lexis.hi.is/kristinb/ Hohimer, R., and White, A. Ontological annotation with
umBIN.html WordNet. In SemAnnot 2005: Proceedings of the Fifth
Distant reading allows scholars to “see 6. Blei, D.M., Ng, A.Y., and Jordan, M.I. Latent dirichlet International Workshop on Knowledge Markup and
allocation. Journal of Machine Learning Research 3
the forest,” discovering patterns that (Jan. 2003), 993–1022.
Semantic Annotation, S. Handschuh, T. Declerck, and
M.-R. Koivun, Eds. (Galway, Ireland, Nov. 7). Ceur-WS,
might otherwise be obscured by too 7. Carley, K., Columbus, D., Bigrigg, M., and Kunkel, Aachen, Germany, 2005, 27–36.
F. AutoMap User’s Guide 2010. Technical Report.
close attention to the detail of a text or Institute for Software Research, School of Computer
30. Schölkopf, B. and Smola, A.J. Learning with Kernels:
Support Vector Machines, Regularization, Optimization,
performance; the same can be said of Science, Carnegie Mellon University, Pittsburgh, PA, and Beyond. MIT Press, Cambridge, MA, 2002.
2010.
the methods outlined here. 8. Child, F.J. and Kittredge, G.L. The English and Scottish
31. Siikala, A.-L. Interpreting Oral Narrative. Suomalainen
Tiedeakatemia, Helsinki, 1990.
Fortunately, with these computa- Popular Ballads. Houghton Mifflin Company, Boston 32. Tang Kristensen, E. Danske sagn, som de har lydt
and New York, 1882.
tional methods, researchers are able 9. Choe, I.-h. A type index of Korean folktales. Myong Ji
i folkemunde, udelukkende efter utrykte kilder. Ny
Række. Woels Forlag, Copenhagen, 1928–1939.
to combine distant reading with close University Publishing, Seoul, 1979. 33. Tang Kristensen, E. Danske sagn, som de har lydt i
10. Christiansen, R.T. The Migratory Legends. folkemunde, udelukkende efter utrykte kilder. Aarhus
reading. In so doing, they can inter- Suomalainen Tiedeakatemia, Helsinki, 1992. Folkeblads Bogtrykkeri, Aarhus, 1892–1901.
rogate the relationship between folk 11. Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, 34. Tangherlini, T.R. Legendary performances: Folklore,
T.K., and Harshman, R. Indexing by latent semantic repertoire and mapping. Ethnologia Europaea 40, 2
expressive culture and the individuals analysis. Journal of the American Society for (2010), 103–115.
who created and perpetuated these ex- Information Science 41, 6 (Sept. 1990), 391–407. 35. Tangherlini, T.R. Interpreting Legend: Danish
12. Dundes, A. From etic to emic units in the structural Storytellers and Their Repertoires. Garland Publishing,
pressions in time and place. Computa- study of folktales. Journal of American Folklore 75, New York, 1994.
tional folkloristics offers an opportu- 296 (Apr.–June 1962), 95–105. 36. Thompson, S. Motif-Index of Folk-literature. A
13. El-Shamy, H.M. Folk Traditions of the Arab World: A Classification of Narrative Elements in Folktales,
nity to read and interpret culture in a Guide to Motif Classification. Indiana University Press, Ballads, Myths, Fables, Mediaeval Romances, Exempla,
Bloomington, IN, 1995.
more holistic fashion than ever before. 14. Elson, D.K. and McKeown, K.R. A tool for deep
Fabliaux, Jest-Books, and Local Legends. Indiana
University Press, Bloomington, 1955–1958.
semantic encoding of narrative texts. In Proceedings 37. Uther, H.-J. The Types of International Folktales: A
of the ACL-IJCNLP 2009 Software Demonstrations
Acknowledgments Joint Conference of the 47th Annual Meeting of the
Classification and Bibliography Based on the System
of Antti Aarne and Stith Thompson. Suomalainen
We wish to thank Nischal Devanur for Association for Computational Linguistics and Fourth
Tiedeakatemia Helsinki, 2004.
International Joint Conference on Natural Language
help in processing the data. We also Processing of the AFNLP (Suntec, Singapore, Aug.
thank colleagues at the Institute for 3). Association for Computational Linguistics,
James Abello ([email protected]) is research
Stroudsburg, PA, 2009, 9–12.
Pure and Applied Mathematics (UCLA), 15. Eskeröd, A. Årets äring. Etnologiska studier i skördens
professor in the Center for Discrete Mathematics and
Theoretical Computer Science of Rutgers University,
participants in Rice University’s “Tech- och julens tro och sed. Nordiska Museets handlingar
Piscataway, NJ.
26. Håkan Ohlssons boktryckeri, Lund, Sweden, 1947.
nology, Cognition, and Culture” lec- 16. Finlayson, M.A. Deriving narrative morphologies via Peter M. Broadwell ([email protected])
ture series, and Indiana University’s analogical story merging. In New Frontiers in Analogy is a Council on Library and Information Resources
Research: Proceedings of the Second International postdoctoral fellow in the Digital Initiatives Department
“Networks and Complex Systems” Conference on Analogy, D. Gentner, K. Holyoak, and of the Charles E. Young Research Library at the University
symposium for comments and sugges- B. Kokinov, Eds. (Sofia, Bulgaria, July 24–27). New of California, Los Angeles.
Bulgarian University Press, Sofia, 2009, 127–136.
tions on earlier versions of this work. 17. Finlayson, M.A. Collecting semantics in the wild: Timothy R. Tangherlini ([email protected]) is
The Story Workbench. In Proceedings of the AAAI a professor of folklore in the Scandinavian Section and
The research was funded by an Ameri- Fall Symposium on Naturally Inspired Artificial the Department of Asian Languages and Cultures at the
can Council of Learned Societies Digi- Intelligence (Arlington, VA, Nov. 7–9). AAAI Press, University of California, Los Angeles and a fellow of the
Menlo Park, CA, 2008, 46–53. American Folklore Society.
tal Innovation Fellowship and National 18. Fogaras, D. and Rácz, B. Scaling link-based similarity
Science Foundation grant IIS-0970179. search. In Proceedings of the 14th International © 2012 ACM 0001-0782/12/07 $15.00

70 c om municatio ns o f the acm | j u ly 201 2 | vol . 5 5 | no. 7

 doi:10.1145/ 2209249 . 2 2 0 9 2 6 8

The reductionism behind today’s software-

engineering methods breaks down in the face
of systems complexity.
by Ian Sommerville, Dave Cliff, Radu Calinescu,
Justin Keen, Tim Kelly, Marta Kwiatkowska,
John McDermid, and Richard Paige

Large-Scale
Complex
IT Systems
On the afternoon of May 6, 2010, the U.S. equity
markets experienced an extraordinary upheaval. Over
approximately 10 minutes, the Dow Jones Industrial
Average dropped more than 600 points, representing
the disappearance of approximately $800 billion of
market value. The share price of several blue-chip

multinational companies fluctuated curred, it reversed, so over the next few

dramatically; shares that had been at minutes most of the loss was recovered
tens of dollars plummeted to a penny and share prices returned to levels
in some cases and rocketed to values close to what they had been before the
over $100,000 per share in others. As crash.
suddenly as this market downturn oc- This event came to be known as the
“Flash Crash,” and, in the inquiry re-
key insights port published six months later,7 the
trigger event was identified as a single
 C oalitions of systems, in which the system block sale of $4.1 billion of futures con-
elements are managed and owned
independently, pose challenging new tracts executed with uncommon ur-
problems for systems engineering. gency on behalf of a fund-management
W
 hen the fundamental basis of company. That sale began a complex
engineering—reductionism—breaks pattern of interactions between the
down, incremental improvements to
current engineering techniques are
high-frequency algorithmic trading
unable to address the challenges of systems (algos) that buy and sell blocks
developing, integrating, and deploying of financial instruments on incredibly
large-scale complex IT systems. short timescales.
 D eveloping complex systems requires A software bug did not cause the
a socio-technical perspective involving
human, organizational, social, and
Flash Crash; rather, the interactions of
political factors, as well as technical independently managed software sys-
factors. tems created conditions unforeseen

ju ly 2 0 1 2 | vo l. 55 | n o. 7 | c om m u n i c at ion s o f t he acm 71
contributed articles

(probably unforeseeable) by the own- and managerially independent. Char-

ers and developers of the trading sys- acterizing SoS, he covered a range of
tems. Within seconds, the result was a systems, from directed (developed for
failure in the broader socio-technical a particular purpose) to virtual (lack-
markets that increasingly rely on the
algos (see the sidebar “Socio-Techni- Developers cannot ing a central management authority or
centrally agreed purpose). LSCITS is a
cal Systems”).
Society depends on complex IT sys-
analyze inherent type of SoS in which the elements are
owned and managed by different orga-
tems created by integrating and or- complexity nizations. In this classification, the col-
chestrating independently managed
systems. The incredible increase in
during system lection of systems that led to the Flash
Crash (an LSCITS) would be called a
scale and complexity in them over the development, “virtual system of systems.” However,
past decade means new software-engi-
neering techniques are needed to help
as it depends since Maier’s article was published in
1998, the word “virtual” has generally
us cope with their inherent complexity. on the system’s taken on a different meaning—virtual
Here, we explain the principal reasons
today’s software-engineering meth- dynamic operating machines; consequently, we propose
an alternative term that we find more
ods and tools do not scale, proposing environment. descriptive—“coalition of systems.”
a research and education agenda to The systems in a coalition of systems
help address the inherent problems work together, sometimes reluctantly,
of large-scale complex IT systems, or as doing so is in their mutual interest.
LSCITS, engineering. Coalitions of systems are not explicitly
designed but come into existence as
Coalitions of Systems different member systems interact ac-
The key characteristic of these systems cording to agreed-upon protocols. Like
is that they are assembled from other political coalitions, there might even
systems that are independently con- be hostility between various members,
trolled and managed. While there is and members enter and leave accord-
increasing awareness in the software- ing to their interpretation of their own
engineering community of related is- best interests.
sues,10 the most relevant background The interacting algos that led to the
work comes from systems engineering. Flash Crash represent an example of a
Systems engineering focuses on devel- coalition of systems, serving the pur-
oping systems as a whole, as defined by poses of their owners and cooperating
the International Council for Systems only because they have to. The owners
Engineering (http://www.incose.org/): of the individual systems were compet-
“Systems engineering integrates all the ing finance companies that were often
disciplines and specialty groups into mutually hostile. Each system jealous-
a team effort forming a structured de- ly guarded its own information and
velopment process that proceeds from could change without consulting any
concept to production to operation. other system.
Systems engineering considers both Dynamic coalitions of software-
the business and the technical needs intensive systems are a challenge for
of all customers with the goal of pro- software engineering. Designing de-
viding a quality product that meets the pendability into the coalition is not
user needs.” possible, as there is no overall design
Systems engineering emerged to authority, nor is it possible to central-
take a systemwide perspective on com- ly control the behavior of individual
plex engineered systems involving systems. The systems in the coalition
structures and electrical and mechani- can change unpredictably or be com-
cal systems. Almost all systems today pletely replaced, and the organiza-
are software-intensive, and systems tions running them might themselves
engineers address the challenge of cease to exist. Coalition “design” in-
constructing ultra-large-scale software volves the protocols for communica-
systems.17 The most relevant aspects of tions, and each organization using
systems engineering is work on “sys- the coalition orchestrates the constit-
tem of systems,” or SoS,12 about which uent systems its own way. However,
Maier said the distinction between the designers and managers of each
SoS and complex monolithic systems individual system must consider how
is that SoS elements are operationally to make it robust enough to ensure

72 c omm unic atio ns o f the ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 contributed articles

their organizations are not threat-

ened by failure or undesirable behav-
ior elsewhere in the coalition. Socio-Technical Systems
System Complexity Engineers are concerned primarily with building technical systems from hardware
and software components and assume system requirements reflect the organizational
Complexity stems from the number
needs for integration with other systems, compliance, and business processes.
and type of relationships between the Yet systems in use are not simply technical systems but “socio-technical systems.”
system’s components and between the To reflect the fact they involve evolving and interacting communities that include
system and its environment. If a rela- technical, human, and organizational elements, they are sometimes also called “socio-
technical ecosystems,” though the term socio-technical systems is more common.
tively small number of relationships Socio-technical systems include people and processes, as well as technological
exist between system components and systems. The process definitions outline how the system designers intend the system
they change relatively slowly over time, should be used, but, in practice, users interpret and adapt them in unpredictable ways,
then engineers can develop determin- depending on their education, experience, and culture. Individual and group behavior
also depend on organizational rules and regulations, as well as organizational culture,
istic models of the system and make or “the way things are done around here.”
predictions concerning its properties. Defining technical software-intensive systems intended to support an
However, when the elements in a organization’s work in isolation is an oversimplification that hinders software
engineering. So-called system requirements represent the interface between the
system involve many dynamic relation- technical system and the wider socio-technical system, yet requirements are inevitably
ships, complexity is inevitable. Com- incomplete, incorrect, and/or out of date. Coalitions of systems cannot operate on this
plex systems are nondeterministic, basis. Rather, engineers must recognize that by taking advantage of the abilities and
and system characteristics cannot be inventiveness of people, the systems will be more effective and resilient.
predicted by analyzing the system’s
constituents. Such characteristics
emerge when the whole system is put Even when the relationships be- and 1980s, modern software is much
to use and changes over time, depend- tween system elements are simpler, larger, more complex, more reliable,
ing how it is used and on the state of its relatively static, and, in principle, un- and often developed more quickly.
external environment. derstandable, there may be so many Software products deliver astonishing
Dynamic relationships include elements and relationships that un- functionality at relatively low cost.
those between system elements and derstanding them is practically impos- Software engineering has focused
the system’s environment that change. sible. Such complexity is called “epis- on reducing and managing epistemic
For example, a trust relationship is a temic complexity” due to our lack of complexity, so, where inherent com-
dynamic relationship; initially, com- knowledge of the system rather than plexity is relatively limited and a single
ponent A might not trust component some inherent system characteris- organization controls all system ele-
B, so, following some interchange, tics.16 For example, it may be possible ments, software engineering is highly
A checks that B has performed as ex- in principle to deduce the traceability effective. However, for coalitions of
pected. Over time, these checks may relationships between requirements systems with a high degree of inherent
be reduced in scope as A’s trust in B and design, but, if appropriate tools complexity, today’s software engineer-
increases. However, some failure in B are not available, doing so may be prac- ing techniques are inadequate.
may profoundly influence that trust, tically impossible. This is reflected in the failure that is
and, after the failure, even more strin- If you do not know enough about all too common in large government-
gent checks might be introduced. a system’s components and their re- funded projects. The software may be
Complexity stemming from the lationships, you cannot make predic- delivered late, be more expensive to
dynamic relationships between ele- tions about overall behavior, even if the develop than anticipated, and inad-
ments in a system depends on the ex- system lacks dynamic relationships equate for the needs of its users. An
istence and nature of these relation- between its elements. Epistemic com- example of such a project was the at-
ships. Engineers cannot analyze this plexity increases with system size; as tempt, from 2000 to 2010, to automate
inherent complexity during system ever-larger systems are built, they are U.K. health records; the project was ul-
development, as it depends on the inevitably more difficult to understand timately abandoned at a cost estimated
system’s dynamic operating environ- and their behavior and properties at $5 billion–$10 billion.19
ment. Coalitions of systems in which more difficult to predict. This distinc- The fundamental reason today’s
elements are large software systems tion between inherent and epistemic software engineering cannot effec-
are always inherently complex. The complexity is important. As discussed tively manage inherent complexity is
relationships between the elements of in the following section, it is the prima- that its basis is in developing individ-
the coalition change because they are ry reason new approaches to software ual programs rather than in interact-
not independent of how the systems engineering are needed. ing systems. The consequence is that
are used or of the nature of their op- software-engineering methods are
erating environments. Consequently, Reductionism and unsuitable for building LSCITS. To ap-
the nonfunctional (often even the Software Engineering preciate why, we need to examine the
functional) behavior of coalitions of In some respects, software engineering essential divide-and-conquer reduc-
systems is emergent and impossible to has been incredibly successful. Com- tionist assumption that is the basis of
predict completely. pared to the systems built in the 1970s all modern engineering.

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 73
contributed articles

Reductionism is a philosophical nical criteria. Decision making in orga- stitute at Carnegie Mellon University
position that a complex system is no nizations is profoundly influenced by (http://www.sei.cmu.edu/) on ultra-
more than the sum of its parts, and political considerations, with actors large-scale systems (ULSS)13 triggered
that an account of the overall system striving to maintain or improve their research leading to creation of the
can be reduced to accounts of individ- current positions to avoid losing face. Center for Ultra-Large Scale Software-
ual constituents. From an engineering Technical considerations are rarely the Intensive Systems, or ULSSIS (http://
perspective, this means systems engi- most significant factor in large-system ulssis.cs.virginia.edu/ULSSIS), a re-
neers must be able to design a system decision making; and search consortium involving the Uni-
so it is composed of discrete smaller The problem is definable, and sys- versity of Virginia, Michigan State
parts and interfaces allowing the parts tem boundaries are clear. The nature University, Vanderbilt University,
to work together. A systems engineer of “wicked problems”15 is that the and the University of California, San
then builds the system elements and “problem” is constantly changing, de- Diego. In the U.K., the comparable
integrates them to create the desired pending on the perceptions and status LSCITS Initiative addresses problems
overall system. of stakeholders. As stakeholder posi- of inherent and epistemic complexity
Researchers generally adopt this tions change, the boundaries are like- in LSCITS, while Hillary Sillitto, a se-
reductionist assumption, and their wise redefined. nior systems architect at Thales Land
work concerns finding better ways to However, for coalitions of systems, & Joint Systems U.K., has proposed
decompose problems or systems (such these assumptions never hold true, ULSS design principles.17
as software architecture), better ways and many software project “failures,” Northrop et al.13 made the point
to create the parts of the system (such where software is delivered late and/ that developing ultra-large-scale sys-
as object-oriented techniques), or bet- or over budget, are a consequence of tems needs to go beyond incremental
ter ways to do system integration (such adherence to the reductionist view. To improvements to current methods,
as test-first development). Underlying help address inherent complexity, soft- identifying seven important research
all software-engineering methods and ware engineering must look toward areas: human interaction, computa-
techniques (see Figure 1) are three re- the systems, people, and organizations tional emergence, design, computa-
ductionist assumptions: that make up a software system’s envi- tional engineering, adaptive system
System owners control system devel- ronment. We need to represent, ana- infrastructure, adaptable and predict-
opment. A reductionist perspective lyze, model, and simulate potential op- able system quality and policy, and ac-
takes the view that an ultimate control- erational environments for coalitions quisition and management. The SEI
ler has the authority to take decisions of systems to help us understand and ULSS report suggested it is essential to
about a system and is therefore able manage, so far as possible, the com- deploy expertise from a range of disci-
to enforce decisions on, say, how com- plex relationships in the coalition. plines to address these challenges.
ponents interact. However, when sys- We agree the research required is
tems consist of independently owned Challenges interdisciplinary and that incremental
and managed elements, there is no Since 2006, initiatives in the U.S. improvement in existing techniques is
such owner or controller and no cen- and in Europe have sought to ad- unable to address the long-term soft-
tral authority to take or enforce design dress engineering large coalitions of ware-engineering challenges of ultra-
decisions; systems. In the U.S., a report by the large-scale systems engineering. How-
Decisions are rational, driven by tech- influential Software Engineering In- ever, a weakness of the SEI report was
its failure to set out a roadmap outlin-
Figure 1. Reductionist assumptions vs. LSCITS reality. ing how large-scale systems engineer-
ing can get from where it is today to the
research it proposed.
Reductionist assumptions Software engineers worldwide cre-
ating large complex software systems
require more immediate, perhaps
Owners of Decisions made Definable problem
a system control rationally, driven by and clear system
more incremental, research, driven by
its development technical criteria boundaries the practical problems of complex IT
systems engineering. The pragmatic
Control Rationality Problem definition
proposals we outline here begin to ad-
dress some of them, aiming for medi-
Wicked problem and um-term, as well as a longer-term, im-
No single owner Decisions driven by constantly renegotiated
or controller political motives system boundaries
pact on LSCITS engineering.
The research topics we propose
here might be viewed as part of the
roadmap that could lead us from cur-
Large-scale complex IT systems reality rent practice to LSCITS engineering.
We see them as a bridge between the
short- and medium-term imperative to
improve our ability to create coalitions

74 comm unicatio ns o f the ac m | j u ly 201 2 | vol . 5 5 | no. 7

 contributed articles

of systems and the longer-term vision a failure for some users may have no ef-
set out in the SEI ULSS report. fect on others. Because some failures
Developing coalitions of systems in- are ambiguous, automated systems
volves engineering individual systems cannot cope on their own. Human op-
to work in the orchestration, as well as
configuration, of a coalition to meet The nonfunctional erators must use information from the
system, intervening to enable it to re-
organizational needs. Based on the
ideas in the SEI ULSS report and on our
(and, often, the cover from failure. This means under-

functional) behavior
standing the socio-technical processes
own experience in the U.K. LSCITS Ini- of failure recovery, the support the
tiative, we have identified 10 questions
that can help define a research agenda
of coalitions operators need, and how to design co-
alition members to be “good citizens”
for future LSCITS software engineering: of systems is able to support failure recovery.
How can interactions between inde-
pendent systems be modeled and simu-
emergent and How can socio-technical factors be
integrated into systems and software-
lated? To help understand and manage impossible to engineering methods? Software- and
coalitions of systems LSCITS engineers
need dynamic models that are updated predict completely. systems-engineering methods support
development of technical systems and
in real time with information from generally consider human, social, and
the system itself. These models are organizational issues to be outside
needed to help make what-if assess- the system’s boundary. However, such
ments of the consequences of system- nontechnical factors significantly af-
change options. This requires new fect development, integration, and
performance- and failure-modeling operation of coalitions of systems.
techniques where the models adapt au- Though a considerable body of work
tomatically due to system-monitoring covers socio-technical systems, it has
data. We do not suggest simulations not been industrialized or made acces-
can be complete or predict all possible sible to practitioners. Baxter and Som-
problems. However, other engineering merville2 surveyed this work and pro-
disciplines (such as civil and aeronau- posed a route to industrial-scale use
tical engineering) have benefited enor- of socio-technical methods. However,
mously from simulation, and compa- much more research and experience is
rable benefits could be achieved for required before socio-technical analy-
software engineering. ses are used routinely for complex sys-
How can coalitions of systems be mon- tems engineering.
itored? And what are the warning signs To what extent can coalitions of sys-
problems produce? In the run-up to the tems be self-managing? Needed is re-
Flash Crash, no warning signs indicat- search into self-management so sys-
ed the market was tending toward an tems are able to detect changes in both
unstable state. To help avoid transition their operation and operational envi-
to an unstable system state, systems ronment and dynamically reconfigure
engineers need to know the indicators themselves to cope with the changes.
that provide information about the The danger is that reconfiguration will
state of the coalition of systems, how create further complexity, so a key re-
they may be used to provide both early quirement is for the techniques to op-
warnings of system problems, and, if erate in a safe, predictable, auditable
necessary, switch to safe-mode operat- way ensuring self-management does
ing conditions that prevent the possi- not conflict with “design for recovery.”
bility of damage. How can organizations manage com-
How can systems be designed to re- plex, dynamically changing system con-
cover from failure? A fundamental prin- figurations? Coalitions of systems will
ciple of software engineering is that be constructed through orchestration
systems should be built so they do not and configuration, and desired system
fail, leading to development of meth- configurations will change dynami-
ods and tools based on fault avoidance, cally in response to load, indicators
fault detection, and fault tolerance. of system health, unavailability of
However, as coalitions of systems are components, and system-health warn-
constructed with independently man- ings. Ways are needed to support con-
aged elements and negotiated require- struction by configuration, managing
ments, avoiding failure is increasingly configuration changes and recording
impractical. Indeed, what seems to be changes, including automated chang-

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | com m u n i cat ion s o f t he acm 75

contributed articles

es from the self-management system, sive. For some safety-critical systems, key problem will not be compatibility
in real time, so an audit trail includes the cost of certification can exceed the but understanding what the informa-
the configuration of the coalition at cost of development, and certification tion exchange really means. This is ad-
any point in time. costs will increase as systems become dressed today on a system-by-system
How should the agile engineering of larger and more complex. Though cer- basis through negotiation between
coalitions of systems be supported? The tification as practiced today is almost system owners to clarify the meaning
business environment changes quickly certainly impossible for coalitions of of shared information. However, if
in response to economic circumstanc- systems, research is urgently needed dynamic coalitions are allowed, with
es, competition, and business reorga- into incremental and evolutionary cer- systems entering and leaving the coali-
nization. Likewise, coalitions of sys- tification so our ability to deploy criti- tion, negotiation is not practical. The
tems must be able to change quickly to cal complex systems is not curtailed by key is developing a means of sharing
reflect current business needs. A model certification requirements. This issue the meaning of information, perhaps
of system change that relies on lengthy is social, as well as technical, as societ- through ontologies like those pro-
processes of requirements analysis and ies decide what level of certification is posed by Antoniou and van Harmelen1
approval does not work. Agile methods socially and legally acceptable. involving the semantic Web.
of programming have been success- How can systems undergo “probabilis- A major problem researchers must
ful for small- to medium-size systems tic verification”? Today’s techniques of address is lack of knowledge of what
where the dominant activity is systems system testing and more formal analy- happens in real systems. High-profile
development. For large complex sys- sis are based on the assumption that a failures (such as the Flash Crash) lead
tems, development processes are often system involves a definitive specifica- to inquiries, but more is needed about
dominated by coordination activities tion and that behavior deviating from the practical difficulties faced by de-
involving multiple stakeholders and it is recognized. Coalitions of systems velopers and operators of coalitions
engineers who are not colocated. How have no such specification nor is sys- of systems and how to address them
can agile approaches be effective for tem behavior guaranteed to be deter- as they arise. New ideas, tools, and
“systems development in the large” to ministic. The key verification issue will methods must be supported by long-
support multi-organization global sys- not be whether the system is correct term empirical studies of the systems
tems development? but the probability that it satisfies es- and their development processes to
How should coalitions of systems sential properties (such as safety) that provide a solid information base for re-
be regulated and certified? Many such take into account its probabilistic, real- search and innovation.
coalitions represent critical systems, time, nondeterministic behavior.8,11 The U.K. LSCITS Initiative5 address-
failure of which could threaten individ- How should shared knowledge in a es some of them, working with partners
uals, organizations, and national econ- coalition of systems be represented? We from the computer, financial services,
omies. They may have to be certified by assume the systems in a coalition in- and health-care industries to develop
regulators checking that, as far as pos- teract through service interfaces so an understanding of the fundamental
sible, they do not pose a threat to their the system has no overarching con- systems engineering problems they
operators or to the wider systems en- troller. Information is encoded in a face. Key to this work is a long-term en-
vironment. But certification is expen- standards-based representation. The gagement with the National Health In-
formation Center to create coalitions
Figure 2. Outline structure for master’s course in LSCITS. of systems to provide external access to
and analysis of vast amounts of health
and patient data.
The project is developing practical
Systems Engineering Business
techniques of socio-technical systems
Ultra-large-scale Systems Organizational engineering2 and exploring design for
systems engineering change failure.18 It has so far developed prac-
Complexity Systems Legal and tical, predictable techniques for au-
procurement regulatory issues tonomic system management3,4 and
Mathematical
modeling Systems Technology is investigating the scaling up of agile
integration innovation methods14 and exploring incremental
Socio-technical
systems Systems Program system certification9 and development
resilience management of techniques for system simulation
and modeling.
Education. To address the practical
Options from computer science, engineering, psychology, business
issues of creating, managing, and op-
erating LSCITS, engineers need knowl-
Industrial project
edge and understanding of the systems
and with techniques outside a “nor-
mal” software- or systems-engineering
education. In the U.K., the LSCITS Ini-
tiative provides a new kind of doctoral

76 comm unicatio ns o f th e ac m | j u ly 201 2 | vo l. 5 5 | no. 7

 contributed articles

degree, comparable in standard to a equate, saying: “For 40 years, we have Regarding the Market Events of May 6th, 2010. Report
of the CFTC and SEC to the Joint Advisory Committee
Ph.D. in computer science or engi- embraced the traditional engineering on Emerging Regulatory Issues, 2010; http://www.
neering. Students get an engineering perspective. The basic premise under- sec.gov/news/studies/2010/marketevents-report.pdf
8. Ge, X., Paige, R.F., and McDermid, J.A. Analyzing
doctorate, or EngD, in LSCITS,20 with lying the research agenda presented in system failure behaviors with PRISM. In Proceedings
the following key differences between this document is that beyond certain of the Fourth IEEE International Conference
on Secure Software Integration and Reliability
EngD and Ph.D.: complexity thresholds, a traditional Improvement Companion (Singapore, June). IEEE
Industrial problems. Students must centralized engineering perspective is Computer Society Press, Los Alamitos, CA, 2010,
130–136.
work on and spend significant time no longer adequate nor can it be the 9. Ge, X., Paige, R.F., and McDermid, J.A. An iterative
on an industrial problem. Universities primary means by which ultra-complex approach for development of safety-critical software
and safety arguments. In Proceedings of Agile 2010
cannot simply replicate the complex- systems are made real.” A key contri- (Orlando, FL, Aug.). IEEE Computer Society Press, Los
Alamitos, CA, 2010, 35–43.
ity of modern software-intensive sys- bution of our work in LSCITS is articu- 10. Goth, G. Ultralarge systems: Redefining software
tems, with few faculty members hav- lating the fundamental reasons this engineering. IEEE Software 25, 3 (May 2008), 91–94.
11. Kwiatkowska, M., Norman, G., and Parker D. PRISM:
ing experience and understanding of assertion is true. By examining how en- Probabilistic model checking for performance and
the systems; gineering has a basis in the philosophi- reliability analysis. ACM SIGMETRICS Performance
Evaluation Review 36, 4 (Oct. 2009), 40–45.
Range of courses. Students must take cal notion of reductionism and how 12. Maier, M.W. Architecting principles for system of
a range of courses focusing on com- reductionism breaks down in the face systems. Systems Engineering 1, 4 (Oct. 1998),
267–284.
plexity and systems engineering (such of complexity, it is inevitable that tradi- 13. Northrop, L. et al. Ultra-Large-Scale Systems: The
as for LSCITS, socio-technical systems, tional software-engineering methods Software Challenge of the Future. Technical Report.
Carnegie Mellon University Software Engineering
high-integrity systems engineering, will fail when used to develop LSCITS. Institute, Pittsburgh, PA, 2006; http://www.sei.cmu.
empirical methods, and technology in- Current software engineering is simply edu/library/abstracts/books/0978695607.cfm
14. Paige, R.F., Charalambous, R., Ge, X., and Brooke, P.J.
novation); and not good enough. We need to think dif- Towards agile development of high-integrity systems.
Portfolio of work. Students do not ferently to address the urgent need for In Proceedings of the 27th International Conference on
Computer Safety, Reliability, and Security (Newcastle,
have to deliver a conventional thesis, a new engineering approaches to help U.K., Sept.) Springer-Verlag, Heidelberg, 2008, 30–43.
book on a single topic, but can deliver construct large-scale complex coali- 15. Rittel, H. and Webber, M. Dilemmas in a general theory
of planning. Policy Sciences 4 (Oct. 1973), 155–73.
a portfolio of work around their select- tions of systems we can trust. 16. Rushby, J. Software verification and system
assurance. In Proceedings of the Seventh IEEE
ed area; it is a better reflection of work International Conference on Software Engineering and
in industry and makes it easier for the Acknowledgments Formal Methods (Hanoi, Nov.). IEEE Computer Society
Press, Los Alamitos, CA, 2009, 1–9.
topic to evolve as systems change and We would like to thank our colleagues 17. Sillitto, H.T. In Proceedings of the 20th International
new research emerges. Gordon Baxter and John Rooksby of St. Council for Systems Engineering International
Symposium (Chicago, July). Curran & Associates, Inc.,
However, graduating a few advanced Andrews University in Scotland and Red Hook, NY, 2010.
doctoral students is not enough. Uni- Hillary Sillitto of Thales Land & Joint 18. Sommerville, I. Designing for Recovery: New
Challenges for Large-scale Complex IT Systems.
versities and industry must also create Systems U.K. for their constructive Keynote address, Eighth IEEE Conference on
master’s courses that educate com- comments on drafts of this article. The Composition-Based Software Systems (Madrid, Feb.
2008); http://sites.google.com/site/iansommerville/
plex-systems engineers for the coming work report here was partially funded keynote-talks/DesigningForRecovery.pdf
decades; our thoughts on what might by the U.K. Engineering and Physical 19. U.K. Cabinet Office. Programme Assessment Review
of the National Programme for IT. Major Projects
be covered are outlined in Figure 2. Science Research Council (www.epsrc. Authority, London, 2011; http://www.cabinetoffice.
The courses must be multidisciplinary, ac.uk) grant EP/F001096/1. gov.uk/resource-library/review-department-health-
national-programme-it
combining engineering and business 20. University of York. The LSCITS Engineering Doctorate
disciplines. It is not only the knowl- Centre, York, England, 2009; http://www.cs.york.ac.uk/
References EngD/
edge the disciplines bring that is im- 1. Antoniou, G. and van Harmelen, F. A Semantic Web
Primer, Second Edition. MIT Press, Cambridge, MA,
portant but also that students be sen- 2008. Ian Sommerville ([email protected])
sitized to the perspectives of a variety 2. Baxter, G. and Sommerville, I. Socio-technical systems: is a professor in the School of Computer Science, St.
From design methods to systems engineering. Andrews University, Scotland.
of disciplines and so move beyond the Interacting with Computers 23, 1 (Jan. 2011), 4–17.
3. Calinescu, R., Grunske, L., Kwiatkowska, M., Mirandola, Dave Cliff ([email protected]) is a professor in the
silo of single-discipline thinking. R., and Tamburrelli, G. Dynamic QoS management Department of Computer Science, Bristol University,
and optimisation in service-based systems. IEEE England.
Transactions on Software Engineering 37, 3 (Mar.
Conclusion 2011), 387–409. Radu Calinescu ([email protected]) is a lecturer in
the Department of Computer Science, Aston University,
Since the emergence of widespread 4. Calinescu, R. and Kwiatkowska, M. Using quantitative
England.
analysis to implement autonomic IT systems. In
networking in the 1990s, all societies Proceedings of the 31st International Conference Justin Keen ([email protected]) is a professor in the
have grown increasingly dependent on on Software Engineering (Vancouver, May). IEEE School of Health Informatics, Leeds University, England.
Computer Society Press, Los Alamitos, CA, 2009,
complex software-intensive systems, 100–110. Tim Kelly ([email protected]) is a senior lecturer
with failure having profound social 5. Cliff, D., Calinescu, R., Keen, J., Kelly, T., Kwiatkowska, in the Department of Computer Science, York University,
M., McDermid, J., Paige, R., and Sommerville, I. The England.
and economic consequences. Indus- U.K. Large-Scale Complex IT Systems Initiative 2010;
http://lscits.cs.bris.ac.uk/docs/lscits_overview_2010. Marta Kwiatkowska ([email protected].
trial organizations and government uk) is a professor in the Department of Computer Science,
pdf
agencies alike build these systems 6. Cliff, D. and Northrop, L. The Global Financial Markets: Oxford University England.
without understanding how to analyze An Ultra-Large-Scale Systems Perspective. Briefing John McDermid ([email protected]) is a professor in
paper for the U.K. Government Office for Science the Department of Computer Science, York University,
their behavior and without appropriate Foresight Project on the Future of Computer Trading England.
engineering principles to support their in the Financial Markets, 2011; http://www.bis.gov.
uk/assets/bispartners/foresight/docs/computer- Richard Paige ([email protected]) is a professor in
construction. trading/11-1223-dr4-global-financial-markets- the Department of Computer Science, York University,
systems-perspective.pdf England.
The SEI ULSS report14 argued that 7. Commodity Futures Trading Commission and
current engineering methods are inad- Securities and Exchange Commission (U.S.). Findings © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 77
contributed articles
doi:10.1145/ 2209249.2209269
Cache coherence has come to domi-
On-chip hardware coherence can scale nate the market for technical, as well as
for legacy, reasons. Technically, hard-
gracefully as the number of cores increases. ware cache coherence provides per-
formance generally superior to what is
By Milo M.K. Martin, Mark D. Hill, and Daniel J. Sorin achievable with software-implemented
coherence. Cache coherence’s legacy

Why On-Chip
advantage is that it provides backward
compatibility for a long history of soft-
ware, including operating systems,
written for cache-coherent shared-

Cache
memory systems.
Although coherence delivers value
in today’s multicore systems, the con-
ventional wisdom is that on-chip cache

Coherence Is
coherence will not scale to the large
number of cores expected to be found
on future processor chips.5,10,13 Coher-
ence’s alleged lack of scalability aris-

Here to Stay
es from claims of unscalable storage
and interconnection network traffic
and concerns over latency and energy.
Such claims lead to the conclusion that
cores in future multicore chips will not
employ coherence but instead commu-
nicate with software-managed coher-
ence, explicitly managed scratchpad
memories, and/or message passing
(without shared memory).
S ha red memor y i s the dominant low-level Here, we seek to refute this con-
ventional wisdom by presenting one
communication paradigm in today’s mainstream way to scale on-chip cache coherence
multicore processors. In a shared-memory system, in which coherence overheads—traf-
fic, storage, latency, and energy—grow
the (processor) cores communicate via loads and slowly with core count and are similar
stores to a shared address space. The cores use caches to the overheads deemed acceptable in
to reduce the average memory latency and memory today’s systems. To do this, we syner-
gistically combine known techniques,
traffic. Caches are thus beneficial, but private caches including shared caches augmented
lead to the possibility of cache incoherence. The
mainstream solution is to provide shared memory key insights
and prevent incoherence through a hardware cache  T he approach taken here scales on-chip
hardware cache coherence to many
coherence protocol, making caches functionally cores with bounded traffic, storage,
latency, and energy overheads.
invisible to software. The incoherence problem and
 F or the same reason system designers
basic hardware coherence solution are outlined in will not abandon compatibility for
the sidebar, “The Problem of Incoherence,” page 86. the sake of eliminating minor costs,
they likewise will not abandon cache
Cache-coherent shared memory is provided by coherence.

mainstream servers, desktops, laptops, and mobile  C ontinued coherence support lets
programmers concentrate on what
devices and is available from all major vendors, matters for parallel speedups: finding
work to do in parallel with no undo
including AMD, ARM, IBM, Intel, and Oracle (Sun). communication and synchronization.

78 com municatio ns o f the ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 to track cached copies, explicit cache overheads of accessing shared data do ence because many applications do
eviction notifications, and hierarchi- not increase appreciably with increas- not scale well with coherence. True,
cal design. Using amortized analysis, ing core count. Consequently, on-chip but is the root cause the algorithm,
we show that interconnection network coherence is here to stay. Computer program implementation, or coher-
traffic per miss need not grow with systems tend not to abandon compat- ence? If, for example, an algorithm
core count and that coherence uses at ibility to eliminate small costs (such as requires frequent updates to be com-
most 20% more traffic per miss than a those found for scaling coherence). In municated to many readers, coher-
system with caches but not coherence. particular, system designers will not ence and most alternatives do poorly.
Using hierarchical design, we show likely replace conventional operat- At least with hardware coherence,
that storage overhead can be made ing systems now that they have been programmers can concentrate on cho-
Illustration by Dave Bollinger

to grow as the root of core count and shown to scale using cache coher- reographing independent work rather
stay small (such as 2% of total cache ence.2,3 Boyd-Wickizer et al.2 concur, than on low-level mechanics.
size for even 512 cores). We find neg- writing “There is no scalability reason Our claim for the continued viabil-
ligible energy and latency overheads to give up on traditional operating sys- ity of on-chip cache coherence does
for cache misses to data that is not tem organizations just yet.” not imply other communication para-
actively shared; our analysis suggests Some architects and software devel- digms will disappear. There will still be
the relative miss penalty and energy opers might object to retaining coher- applications for which message pass-

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | com m u n i c at ion s o f t he ac m 79
contributed articles

ing is appropriate (such as to scale-out clusion requires that block A (cached

high-performance computing) or for by core 0) and block B (cached by cores
which incoherent scratchpad memo- 1 and 2) must be present in the shared
ries are appropriate (such as real-time cache with appropriate tracking bits
systems), and so those communication
paradigms will persist. However, they We anticipate {1000} and {0110}, respectively. If the
block size of the shared cache is larger
are likely to continue to coexist with
on-chip cache-coherent shared memo-
alternatives to than the private cache’s block size,
each entry in the shared cache main-
ry for the foreseeable future. cache-coherent tains coherence state and tracking

Cache Coherence Today

shared memory will bits at the granularity of the private
cache block size.
Before investigating the issues in- continue to exist When a core issues a load or store
volved in coherence’s future, we first
describe today’s cache coherence
and thrive in certain that misses in its private cache, it is-
sues a coherence request message
protocols. Rather than survey coher- domains but that to the shared cache. Based on the
ence protocol design, we focus on
one concrete coherence protocol on-chip coherence block’s coherence state and per-core
tracking bits, the shared cache either
loosely based on the on-chip cache will continue responds directly or forwards the re-
coherence protocol used by Intel’s
Core i7,17 which represents the state to dominate in quest to the one or more cores that
need to respond to the request. For ex-
of the art and can scale to a moder-
ate number of cores (such as 16). In
mainstream ample, if the request is for read/write
access, and one or more cores is pri-
such a system, the cores on the chip multicore chips. vately caching the block in a read-only
communicate via loads and stores to state, then the shared cache forwards
the shared memory. Each core has its the request to all private caches in the
own private cache hierarchy (referred tracking list, and these private caches
to hereafter as “private cache”). There invalidate their copies of the block. If
is a single shared last-level cache (re- no cores are caching the block, then
ferred to hereafter as “shared cache”). a request has the negligible overhead
Such shared caches typically employ of looking up only the coherence state
address-interleaved banking with one bits in the shared cache. This protocol
bank per core, thus proportionally is essentially a directory-based cache
scaling the bandwidth of the shared coherence protocol in which the di-
cache as the number of cores increas- rectory entries are co-located with the
es; this system model is outlined in tags of the shared cache. Inclusion
the sidebar figure. ensures that each private block has a
To make our analysis simpler and corresponding shared block to hold its
more concrete, we assume for now that coherence tracking bits.
the shared cache is inclusive with re- To maintain inclusion, when the
spect to all the private caches. Inclusion shared cache evicts a block for which
means, at all times, the shared cache some per-core tracking bits are set,
contains a superset of the blocks in the the shared cache first issues a recall
private caches. Intel’s Core i7 is an ex- request (also known as a back-invalida-
ample of a chip with inclusive caches. tion or notification) to any core current-
Because inclusion is not a universally ly caching that block as determined
adopted design decision, we discuss by the per-core tracking state. Upon
extending our results to non-inclusive receipt of a recall message, the private
shared caches later. cache is forced to evict the block.
With inclusion, cache coherence This approach to coherence has
can be maintained with a coherence many attractive features, helping ex-
protocol that tracks copies of blocks in plain why current Intel systems re-
private caches using state embedded semble it. This protocol avoids the
in the shared cache; that is, each block need for a snooping bus and avoids
in the shared cache is augmented with broadcasting; communication in-
a few bits of coherence state (such as volves only point-to-point messages.
to denote if the block is writable by Because the protocol embeds the
any core) and per-core tracking bits per-core tracking bits in the shared
that denote which cores are privately cache, it avoids adding additional
caching the block (one bit per core). structures dedicated solely to coher-
As outlined in the sidebar figure, in- ence. For small-scale systems (such

80 com municatio ns o f the ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 contributed articles

as four cores to 16 cores), the stor- some caveats and potential criticisms cache to send an acknowledgment
age cost is negligible; a 16-core sys- of this work. message in response to each eviction
tem adds just 16b for each 64B cache notification. Fortunately, clean evic-
block in the shared cache, or approxi- Concern 1: Traffic tion messages are small (enough to,
mately 3% more bits. For a miss to a Here we tackle the concerns regarding say, hold a 8B address) and can oc-
block not cached by other private the scalability of coherence traffic on cur only subsequent to cache misses,
caches, the miss latency and energy the on-chip interconnection network. transferring, say, a 64B cache block.
consumed incur the negligible over- To perform a traffic analysis, we con- Coherence’s additional traffic per
head of checking a couple of state bits sider for each cache miss how many miss is thus modest and, most im-
in the shared cache rather than just a bytes must be transferred to obtain and portant, independent of the number
single valid bit. As we show later, even relinquish the given block. We divide of cores. Based on 64B cache blocks,
when blocks are shared, the traffic per the analysis into two parts: in the ab- the table here shows that coherence’s
miss is limited and independent of sence of sharing and with sharing. This traffic is 96B/miss for clean blocks
the number of cores. Overall, this ap- analysis shows that when sharers are and 160B/miss for dirty blocks.
proach is reasonably low cost in terms tracked precisely, the traffic per miss With sharing. In a coherent sys-
of traffic, storage, latency, and energy, is independent of the number of cores. tem, when a core reads a block that is
and its design complexity is tractable. Thus, if coherence’s traffic is accept- shared, the coherence protocol might
Nevertheless, the question for archi- able for today’s systems with relatively need to forward the request but to at
tects is: Does this system model scale few cores, it will continue to be accept- most only one core; thus the traffic
to future manycore chips? able as the number of cores scales up. for each read miss is independent of
We conclude with a discussion of how the number of cores. However, when
Scalability coherence’s per-miss traffic compares a core incurs a write miss to a block
Some prognosticators forecast that to that of a system without coherence. that is cached by one or more other
the era of cache coherence is nearing Without sharing. We first analyze cores, the coherence protocol gener-
its end5,10,13 due primarily to an alleged the worst-case traffic in the absence ates extra messages to invalidate the
lack of scalability. However, when we of sharing. Each miss in a private block from the other cores. These in-
examined state-of-the-art coherence cache requires at least two messages: validation messages are often used to
mechanisms, we found them to be a request from the private cache to the argue for the nonscalability of cache
more scalable than we expected. shared cache and a response from the coherence, because when all cores
We view a coherent system as “scal- shared cache to provide the data to the are sharing a block, a coherent system
able” when the cost of providing co- requestor. If the block is written during must send an invalidation message to
herence grows (at most) slowly as core the time it is in the cache, the block is all other cores. However, our analysis
count increases. We focus exclusively “dirty” and must be written explicitly shows that when sharers are tracked
on the cache-coherence aspects of back to the shared cache upon eviction. precisely, the overall traffic per miss
multicore scaling, whereas a fully scal- Even without sharing, the traffic de- of cache coherence is independent of
able system (coherent or otherwise) pends on the specific coherence proto- the number of cores; the storage cost
also requires scalability from other col implementation. In particular, we of precise tracking is addressed later,
hardware (such as memory and on- consider protocols that require a pri- in the section on storage.
chip interconnection network) and vate cache to send an explicit eviction Consider an access pattern in which
software (operating system and appli- notification message to the shared a block is read by all cores and then
cations) components. cache whenever it evicts a block, even written by one core. The writer core is
Here, we examine five potential con- when evicting a clean block. (This de- indeed forced to send an invalidation
cerns when scaling on-chip coherence: cision to require explicit eviction noti- message to all cores, and each core
˲˲ Traffic on the on-chip interconnec- fications benefits implementation of will respond with an acknowledgment
tion network; inclusive caching, as discussed later message, or a cost of 2N messages for
˲˲ Storage cost for tracking sharers; in the section on maintaining inclu- N sharers. However, such an expen-
˲˲ Inefficiencies caused by maintain- sion.) We also conservatively assume sive write operation can occur only
ing inclusion (as inclusion is assumed that coherence requires the shared after a read miss by each of the cores.
by our base system);
˲˲ Latency of cache misses; and Traffic cost of cache misses.
˲˲ Energy overheads.
To calculate traffic, we must assume values for the size of addresses and cache blocks (such as 8B
The following five sections ad- physical addresses and 64B cache blocks). Request and acknowledgment messages are typically
dress these concerns in sequence short (such as 8B) because they contain mainly a block address and a message type field. A data
message is significantly larger because it contains both an entire data block plus a block address
and present our analysis, indicating (such as 64B + 8B = 72B).
that existing design approaches can
Clean block Dirty block
be employed such that none of these
Without coherence (Req+Data) + 0 = 80B/miss (Req+Data) + Data = 152B/miss
concerns would present a fundamen-
With coherence (Req+Data) + (Evict+Ack) = 96B/miss (Req+Data) + (Data+Ack)= 160B/miss
tal barrier to scaling coherence. We
Per-miss traffic overhead 20% 5%
then discuss extending the analysis
to noninclusive caches and address

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 81
contributed articles

More generally, for every write that sis, the overall average traffic per miss counts for an access pattern param-
invalidates N caches, the write must is constant; a write miss that causes N eterized by the number of read misses
have been preceded by N read misses. messages can occur at most only once to a block between each write miss to
The traffic cost of a read miss is inde- every Nth miss. the block. A workload consisting of
pendent of the number of cores; a read In support of this general analysis, all write misses (zero read misses per
miss is forwarded to a single core, at Figure 1a shows the traffic (in average write miss; far left of Figure 1a) has
most. Thus, through amortized analy- bytes per miss) over a range of core the highest traffic per miss because
all blocks are dirty. Traffic per miss is
Figure 1. Communication traffic for shared blocks. independent of the number of cores
because the shared cache forwards the
write misses to at most one core, the
most recent writer. With an increasing
number of read misses per write miss
700
(moving to the right in Figure 1a), the
average traffic per miss actually de-
600
creases slightly because fewer writes
500 lead to fewer dirty blocks. More impor-
Bytes per miss

tant, the traffic is independent of the

400 number of cores in the system, because
each write miss that causes N messages
300
is offset by N previous read misses.
200
Traffic overhead of coherence. We
have shown that coherence’s per-miss
100 traffic scales because it is indepen-
dent of the number of cores. We now
0 1,024
256
consider coherence’s traffic overhead
0
1

per miss with respect to a hypothetical

2
3

64
4
5

design with caches but no hardware co-

6

Rea 16
7

dm
8

isse Cores herence (such as when software knows

16

s pe 4
32

r wr
64

precisely when cached data is stale

128

ite m
256

iss 1
512

without extra traffic). We continue to

1,024

measure traffic in terms of bytes of traf-

fic on the interconnection network per
(a) With exact tracking of sharers
cache miss, thus assuming that coher-
ence does not change the number of
cache misses. However, this assump-
tion is potentially compromised by
false sharing and inefficient synchro-
700
nization, which can cause nonscalable
600
increases in the number of cache miss-
es. Both of these phenomena are well-
500 known challenges with well-known
Bytes per miss

techniques for their mitigation; we

400 cannot completely eliminate their im-
pact nor cleanly incorporate them into
300
our intentionally simplistic models.
200 The table lists the traffic per miss
for this system without coherence. We
100 now compare this traffic to the system
with coherence. For a clean block, the
0 1,024
256
system without coherence eliminates
0
1

the need for the eviction notification

2
3

64
4
5

and the acknowledgment of this noti-

6

Rea 16
7

dm
8

isse Cores fication. For a dirty block, the system

16

s pe 4
32

r wr
64

without coherence avoids the acknowl-

128

ite m
256

iss 1
512

edgment of the dirty eviction message.

1,024

The key is that none of these three extra

(b) With inexact tracking of sharers messages contain the data block, and
such “control” messages are signifi-
cantly smaller than “data” messages.

82 comm unicatio ns o f th e ac m | j u ly 201 2 | vo l. 5 5 | no. 7

 contributed articles

Coherence’s overhead is small, bound- Figure 2. Hierarchical system model; additions for coherence are shaded.
ed, and—most important—indepen-
dent of the number of cores. Based on
64B cache blocks, the table shows that Cluster 1 Cluster K
coherence adds a 20% traffic overhead Cluster of K cores Cluster of K cores
for clean blocks and a 5% overhead for
dirty blocks. core core core core core core
Conclusion. Coherence’s intercon-
nection network traffic per miss scales private private private private private private
cache cache cache cache cache cache
when precisely tracking sharers.
Intra-cluster Intra-cluster
Concern 2: Storage interconnection network interconnection network
The scalable per-miss traffic result as-
sumed a precise tracking of sharing Cluster Cache Cluster Cache
state in the shared cache, requiring N tracking tracking
state bits tag block data state bits tag block data
bits of state for a system with N cores.
This assumption leads to the reason-
able concern that such an increase in
tracking state for systems with more
cores could pose a fundamental bar-
rier to scalability. Here, we show that Inter-cluster interconnection network
the storage cost scales gracefully by
quantifying the storage cost and de-
scribing two approaches for bounding tracking
this cost: the first is the traditional use state bits tag block data
Shared
of inexact encoding of sharers,1,8 which last-level
we discard in favor of often-overlooked cache
use of on-chip hierarchy to efficiently
maintain an exact encoding of sharers.
The storage cost at the private caches
is negligible; supporting coherence in gin would use the bits as an inexact, fic of the exact encoding cases. Though
the private caches adds just a few state coarse-vector encoding, in which each conventional wisdom might have pre-
bits for each cache block, which is less bit represents multiple cores. Though dicted an even larger traffic spike for
than 1% storage overhead and inde- the storage can be bounded, the traf- 1,024 cores, we next describe an alter-
pendent of the number of cores, so our fic of such schemes could suffer due to native design that eliminates any such
analysis focuses on additional storage unnecessary invalidations. spike in traffic.
in the shared cache. To quantify the traffic impact of such Less conventional approach: On-
Conventional approach: Inexact inexact encodings, Figure 1b shows the chip hierarchy for exact tracking. To
encoding of sharers. The conventional result of applying the analysis from the avoid a spike in traffic for some sharing
approach to limiting storage—inexact previous section on traffic when using patterns, an alternative is to overcome
encoding of sharers—can work well the Origin’s inexact encoding scheme this scalability problem through an
but has poor worst-case behavior. It to bound the storage at 32b per block on-chip hierarchy of inclusive caches.
represents a conservative superset in the shared cache (approximately 6% Hierarchy is a natural design meth-
of sharers using fewer bits than one overhead for 64B blocks). When the odology for scalable systems. With
bit per potential sharer and was well- 32b is enough for exact tracking (up many cores, the size of private caches
studied in the early 1990s.1,8 As a con- to 32 cores) or when the number of is limited, and the miss latency from
crete example, the SGI Origin 200014 sharers is smaller than the number of a private cache to the chipwide shared
used a fixed number of bits per block, limited pointers (far left of Figure 1b), cache is likely large. As such, many-
regardless of the number of cores. For the sharers are encoded exactly, result- core systems,4,16 GPUs,15 and proposed
small systems, the Origin used these ing in the same traffic-per-miss as the manycore architectures12 cluster some
bits as a bit-vector that tracks shar- exact encoding. When the number of number of cores/threads to share an
ers exactly. For larger systems, the sharers is large (far right of Figure 1b), intermediate level of cache. For exam-
Origin alternated between two uses the write invalidations must be sent to ple, Sun/Oracle’s T2 systems16 share a
of these tracking bits. If there were all cores (independent of encoding), so small L1 cache between two pipelines,
only a few sharers, the bits would be the inexact encoding incurs no traffic each with four threads. NVIDIA’s Fermi
used as a limited number of pointers penalty. However, when the number of GPU15 clusters 32 execution pipelines
(each of which requires log2N bits to cores grows and the number of sharers into a “shared multiprocessor.” In
encode) that can exactly track sharers. is in the middle of the range, the traffic AMD’s Bulldozer architecture,4 each
If the number of sharers exceeds this overheads spike. With 1,024 cores, the pair of cores has per-core private L0
limited number of pointers, the Ori- spike reaches almost six times the traf- caches and shares an L1 cache. Such

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 83
contributed articles

systems fill the gap between a private dation acknowledgment independent 4,096-core 3-level system would have 16
cache and a large, distributed shared of the number of cores in the cluster clusters, each with 16 subclusters of 16
cache, allowing the cluster cache to de- that were caching the block. Compared cores, with storage overhead of only 3%.
liver faster access to data shared within to a flat protocol, which must send ac- Conclusion. Hierarchy combined
the cluster. An additional benefit is knowledgments to every requestor, with inclusion enables efficient scaling
that coherence requests may be satis- the total cross-chip traffic is reduced, of the storage cost for exact encoding
fied entirely within the cluster (such and the protocol avoids the bottleneck of sharers.
as by a sibling node caching the block) of sequentially injecting hundreds or
that can be significant if the software is thousands of invalidation messages Concern 3: Maintaining Inclusion
aware of the hierarchy. and later sequentially processing the In the system model covered here,
The same techniques described same number of acknowledgments. we initially choose to require that the
earlier—inclusion, integrating track- The second benefit is that a hierar- shared cache maintain inclusion with
ing state with caches, recall messages, chical system that enforces inclusion respect to the private caches. Main-
and explicit eviction notifications—are at each level reduces the storage cost taining an inclusive shared cache
straightforward to apply recursively to of coherence. Recall from the previous allows efficient tracking of blocks
provide coherence across a hierarchi- section on traffic that using an exact in private caches by embedding the
cal system. Rather than just embed encoding of sharers allows for scalable tracking information in the tags of the
tracking state at a single shared cache, communication for coherence but shared cache, and is why we use this
each intermediate shared cache also that we deferred the seeming problem design point. Inclusion also simpli-
tracks sharers—but just for the caches of the storage cost of exact encoding. fied our earlier analysis of communi-
included by it in the hierarchy. Con- Now we show that by using hierarchy cation and storage.
sider a chip (see Figure 2) in which we can also make the storage cost scale Inclusion requires that if a block is
each core has its own private cache, gracefully. Consider first a two-level cached in any private cache, it must also
each cluster of cores has a cluster system (three levels of cache) consist- be cached in the shared cache. When
cache, and the chip has a single shared ing of K clusters of K cores each (K2 = C the shared cache evicts a block with
last-level cache. Each cluster cache is total cores). Each cluster cache is in- nonempty tracking bits, it is required
shared among the cores in the cluster clusive with respect to all private cach- to send a recall message to each private
and serves the same role for coherence es within the cluster, and the shared cache that is caching the block, adding
as the shared cache in nonhierarchi- last-level cache is inclusive with re- to system traffic. More insidiously, such
cal systems; that is, the cluster cache spect to all cluster caches. Each cluster recalls can increase the cache miss rate
tracks which private caches within cache block uses one bit for each of the by forcing cores to evict hot blocks they
the cluster have the block. The shared K private caches it includes, plus a few are actively using.11 To ensure scalabil-
last-level cache tracks which cluster bits of state. Likewise, each shared ity, we seek a system that makes recall
caches are caching the block but not last-level cache block consumes a bit messages vanishingly rare.
which specific private cache(s) within for each of the K cluster caches it in- Recalls occur when the shared cache
the cluster are caching it. For exam- cludes, plus a few bits of state. Impor- is forced to evict a block with one or
ple, a balanced 256-core system might tantly, these storage costs grow as a more sharers. To reduce the number
consist of 16 clusters of 16 cores each linear function of K and thus propor- of recalls, the shared cache always
with a 16KB first-level cache, a 512KB tional to √—C. Even if C increases great- chooses to evict nonshared blocks over
second-level shared cluster cache, and ly, √—
C grows more slowly. shared blocks. Because the capacity
a 16MB third-level (last-level) cache This storage cost at the cluster of an inclusive shared cache often ex-
shared among all clusters. caches and last-level cache could be re- ceeds the aggregate capacity of the pri-
Such a hierarchical organization duced even further by extending the hi- vate caches (for example, the ratio is 8
has some disadvantages—extra com- erarchy by one level. Consider a system for the four-core Intel Core i7 with 8MB
plexity and layers of cache lookups— with K level-2 clusters, each consisting shared cache and four 256KB second-
but also two key benefits for coherence: of K level-1 clusters, with each level-1 level private caches), it is highly likely
First, the hierarchy naturally provides cluster consisting of K cores. This sys- that a nonshared block will be available
a simple form of fan-out invalidation tem has C = K3 cores and a storage cost to evict whenever an eviction occurs.
and acknowledgment combining. For proportional to cube root of C. Unfortunately, the shared cache
example, consider a block cached by all In Figure 3, we plot coherence’s stor- sometimes lacks sufficient informa-
cores; when a core issues a write miss age overhead (coherence’s storage as tion to differentiate between a block
to this block, the cluster cache lacks a fraction of the total cache storage) possibly being cached and certainly
write permission for the block, so it for- in terms of the bits needed to provide being cached by a core. That is, the
wards it to the shared last-level cache. precise tracking of sharers, for conven- tracking bits in the shared cache are
The shared last-level cache then sends tional flat (nonhierarchical) systems, updated when a block is requested, but
an invalidation message to each cluster 2-level systems, and 3-level systems. As the shared cache in some systems does
(not to each core), triggering the clus- a very large example, a 1,024-core 2-lev- not always know when a private cache
ter cache to perform an analogous in- el system might have 32 clusters of 32 has evicted the block. In such systems,
validation operation within the cluster. cores, thus 32b per 64B cache block at clean blocks (those not written during
The cluster then sends a single invali- each level, which is just 6%. An extreme their lifetime in the cache) are evicted

84 comm unicatio ns o f the acm | j u ly 201 2 | vol . 5 5 | no. 7

 contributed articles

silently from the private caches, intro- tivity on recall rate, we performed a Figure 4 shows the recall rate, or per-
ducing ambiguity at the shared cache simulation modeling recalls due to en- centage of misses that cause a recall,
as to what is still being cached and what forcing inclusion in such a system. We for shared caches of various sizes (as
has already been evicted. This lack of pessimistically configured the private a ratio of aggregate per-core capacity)
information manifests as poor replace- caches to be fully associative. To factor for several shared cache associativi-
ment decisions at the shared cache. out the effect of any particular bench- ties. When the capacity of the shared
To remedy this lack of information, mark, we generated a miss-address cache is less than the aggregate per-
a system can instead require the pri- stream to random sets of the shared core capacity (ratio < 1.0), almost every
vate caches to send explicit notification cache that prior work found accurately request causes a recall, because the
messages whenever a block is evicted, approximates conflict rates.9 We also private caches are constantly contend-
even when evicting clean blocks. For pessimistically assumed no data shar- ing for an unrealistically underprovi-
example, AMD’s HT-Assist protocol ing among the cores that would reduce sioned shared cache. As the size of the
uses explicit eviction notifications on the inclusive capacity pressure on the shared cache increases, the recall rate
clean-exclusive block replacements to shared cache. drops quickly. When the capacity ratio
improve sharer state encoding.6 If such Fortunately, recalls can be made reaches four times, even an eight-way
eviction notifications occur on every rare in the expected design space. set-associative shared cache keeps
cache eviction, the protocol enables
the shared cache to maintain precise Figure 3. Storage overhead in shared caches.
up-to-date tracking of private caches
that hold each block, transforming the Single-Level Two-Level Three-Level
tracking information from conserva- 20%
tive to exact. When an eviction deci- 18%
sion does occur, the shared cache thus
Storage Overhead (percent)

16%
knows which blocks are no longer be-
14%
ing cached and likely have a choice to
evict a nonshared block to avoid a re- 12%
call. However, this precision comes at a 10%
cost in the form of increased traffic for 8%
evictions of clean blocks, the overhead 6%
of which was already included in the
4%
traffic analysis.
Explicit eviction notifications can 2%

potentially eliminate all recalls, but 0%

only if the associativity, or number of 1 2 4 8 16 32 64 128 256 512 1024 2048 4096
places in which a specific block may Cores
be cached, of the shared cache ex-
ceeds the aggregate associativity of
the private caches. With sufficient as-
sociativity, whenever the shared cache Figure 4. Likelihood a shared cache miss triggers a recall.
looks for a nonshared block to evict, if
it has exact sharing information, it is Associativity of Shared Cache
guaranteed to find a nonshared block 1-way 2-way 4-way 8-way
and thus avoid a recall. Without this 100

worst-case associativity, a pathological 90

Percentage of misses causing recalls

cluster of misses could lead to a situa-

80
tion in which all blocks in a set of the
shared cache are truly shared. Unfortu- 70

nately, even with a modest number of 60

cores, the required associativity is pro-
50
hibitive, as reported by Ferdman et al.7
For example, eight cores with eight-way 40
set-associative private caches require a 30
64-way set-associative shared cache,
20
and the required associativity doubles
for each doubling of the number of 10
Expected
cores. 0 Design
Space
Rather than eliminate all recalls, 0 1 2 3 4 5 6 7 8
we focus on a system in which recalls Ratio of aggregate private cache capacity to shared cache capacity
are possible but rare. To estimate the
effect of limited shared cache associa-

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 85
contributed articles

The Problem of Incoherence

Incoherence. To illustrate the problem incoherence is a hardware cache- A write to block B by core 1 (which
of incoherence, consider the multiple coherence protocol. Though there in our example led to incoherence)
cores and corresponding private are many possible coherence is not allowed to update its read-
caches in the upper-right of the figure protocols, all maintain coherence by only copy of the block. Instead,
here. If core 1 writes the block labeled ensuring the single-writer, multiple- core 1 must first obtain write
B by updating its private cache only, reader invariant; that is, for a given permission to the block. Obtaining
subsequent reads by core 2 would block at any given moment in time, write permission without violating
see the old value indefinitely. This there is either: the single-writer, multiple-reader
incoherence can lead to incorrect ˲˲ Only a single core with write (and invariant requires invalidating any
behavior; for example, if the block read) permission to the block (in copies of the block in other caches
holds a synchronization variable state M for modified); or (core 2 in this case, as encoded
for implementing mutual exclusion ˲˲ Zero or more cores with read by the tracking bits in the shared
using a lock, such incoherent permission to the block (in state S for cache). Such actions are handled in
behavior could allow multiple cores shared). hardware by cache-coherence logic
into a critical section or prevent cores Enforcing coherence. The figure integrated into the cache controllers.
waiting for the release of the lock here outlines core 0 caching block A The section on cache coherence
from making forward progress. with read/write permission (state M) today presents a protocol (and
The coherence invariant. The and cores 1 and 2 caching block B describes the rest of the diagram); for
mainstream solution to preventing with read-only permission (state S). more, see Sorin et al.18

System model; additions for coherence are shaded.

Core Core Core Core

0 1 2 3
Block in private cache
private cache private cache private cache private cache
state tag block data
A: M, …
B: S, … B: S, … B: I
~2 bits ~64 bits ~512 bits

Interconnection network

Block in shared cache

tracking
bits state tag block data Bank O Bank 1 Bank 2 Bank 3
A: {1000} M …
B: {0110} S …
~1 bit ~2 bits ~64 bits ~512 bits
per core

Shared cache
(banked by block address)
State — Meaning
M (Modified) — Read/write permission
S (Shared) — Read-only permission
I (Invalid) — No permissions

the recall rate below 0.1%. For com- sign a system with an inclusive shared caches are typically interleaved by ad-
parison, the Intel Core i7 has a 16-way cache with negligible recall rate, and dresses with banks physically distrib-
set-associative cache with eight times thus can efficiently embed the tracking uted across the chip (see the sidebar
capacity ratio. Based on this analysis, state in the shared cache. figure), so the expected best-case la-
we conclude that the traffic overhead tency of a miss that hits in the shared
of enforcing inclusion is negligible for Concern 4: Latency cache is the access latency of the cache
systems with explicit eviction notifi- In a non-coherent system, a miss in a bank plus the round-trip traversal of
cations and reasonable shared cache private cache sends a request to the the on-chip interconnect to reach the
sizes and associativities. shared cache. As discussed earlier, to appropriate bank of the shared cache.
Conclusion. Chip architects can de- provide sufficient bandwidth, shared Requests that miss in the shared cache

86 com municatio ns o f th e ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 contributed articles

are in turn routed to the next level of in a non-coherent system) roughly pro- at most a fixed per-core, thus scalable,
the memory hierarchy (such as off- portionally, keeping the ratio largely leakage energy overhead.
chip memory). unchanged. Moreover, if latency is still Furthermore, many energy-inten-
In a coherent system with private deemed too great, for either coherent sive parts of the system—the cores
caches and a shared cache, four cases or non-coherent systems, these sys- themselves, the cache data arrays, off-
are worth consideration with regard to tems can use prefetching to hide the chip DRAM, and storage—are largely
miss latency: a hit in the private cache; latency of anticipated accesses. unaffected by coherence, so energy
a direct miss in which the shared Similar reasoning can be applied re- overheads incurred by coherence are
cache can fully satisfy the request, that cursively to calculate the latency ratio relatively smaller when weighed against
is, to a block not cached in other pri- for a system with more layers of hier- the context of the overall system.
vate caches; an indirect miss, in which archy. Though the effect of hierarchy Conclusion. Based on these traffic
the shared cache must contact one or may hurt absolute latency (such as due and storage scalability analyses, we
more other caches; and a miss in the to additional layers of lookup), we see find no reason the energy overheads
shared cache, incurring a long-latency no reason why hierarchy should signifi- of coherence must increase with the
access to off-chip memory. Coherence cantly affect the ratio of the latencies of number of cores.
adds no latency to perhaps the two direct to indirect misses. Furthermore,
most performance-critical cases: pri- the cluster caches introduced by hier- Non-Inclusive Shared Caches
vate cache hits (the first case) and off- archy may help mitigate the growing So far we have assumed an inclusive
chip misses (the fourth case). Coher- cross-chip latency by providing a closer shared cache, like that of Intel’s Core
ence also adds no appreciable latency mid-size cache that allows faster shar- i7, but this choice is not universal.
to direct misses because the coher- ing within a cluster and reducing the Rather than require a private cache
ence state bits in the tags of the shared number of longer-latency accesses to block to be present in the shared cache
cache can be extended to unambigu- the chipwide distributed shared last- (inclusion), a system can forbid it from
ously distinguish between direct and level cache. Modeling the full effect of being present (exclusion) or allow but
indirect misses. hierarchy on latency (and traffic) is be- not require it to be present (neither in-
However, indirect misses do incur yond the reach of the simple models we clusion nor exclusion). Not enforcing
the extra latency of sending a mes- use here. inclusion reduces redundant caching
sage on the on-chip interconnection Conclusion. Though misses to ac- (less important for the Core i7 whose
network to the specified private cores. tively shared blocks have greater la- shared cache size is eight times the
Such messages are sent in parallel, and tency than other misses, the latency ra- sum of its private cache sizes) but has
responses are typically sent directly to tio is tolerated, and the ratio need not implications for coherence.
the original requester, resulting in a grow as the number of cores increases. A non-inclusive system can retain
“three-hop protocol.” Thus, the criti- the coherence benefits of an inclusive
cal path latency of direct and indirect Concern 5: Energy shared cache by morphing it into two
misses can be approximated by the fol- Though a detailed energy analysis is structures: a new noninclusive shared
lowing formulas: perhaps not as straightforward as the cache that holds tags and data, but not
analyses we have reported here, we can tracking state, and is free to be of any
Non-coherent use these analyses to support the con- size and associativity; and a “directory”
tnoncoherent = tinterconnect + tcache + tinterconnect clusion that the energy cost of coher- that holds tags and per-core tracking
Coherent ence is also not a barrier to scalabil- state, but not data blocks, and uses
tdirect = tinterconnect + tcache + tinterconnect ity. Energy overheads generally come inclusion to operate like a dataless ver-
tindirect = tinterconnect + tcache + tinterconnect + tcache from both doing more work (dynamic/ sion of the previous inclusive shared
+ tinterconnect switching energy) and from additional cache; this design roughly resembles
transistors (static/leakage energy). some systems from AMD.6
The indirect miss latency for coher- For dynamic energy, the primary To the first order, the communi-
ence is from 1.5 to two times larger concerns are extra messages and ad- cation between the directory and its
than the latency of a non-coherent ditional cache lookups. However, we private caches is the same as with the
miss; the exact ratio depends on the have shown that interconnect traffic original inclusive shared cache, provid-
relative latencies of cache lookup (tcache) and message count per-miss do not ed the directory continues to be large
and interconnect traversal (tinterconnect). increase with the number of cores, in- enough to keep recalls rare. Moreover,
This ratio is considered acceptable in dicating the protocol state transitions designers now have more freedom in
today’s multicore systems, in part be- and number of extra cache lookups are setting the new non-inclusive shared
cause indirect misses are generally in likewise bounded and scalable. cache configuration to trade off cost
the minority for well-tuned software. For static energy, the primary con- and memory traffic. Though the direc-
The ratio also indicates scalability, as cerns are the extra tracking state we tory-tracking state is the same as with
the ratio is independent of the num- have also shown scales gracefully and an inclusive shared cache (total direc-
ber of cores. Even if the absolute inter- leakage due to any extra logic for pro- tory size is proportional to the sum of
connect latency increases with more tocol processing. Protocol process- private cache sizes), the storage effect
cores, such increases will generally ing logic is added per core and/or per is more significant because the directo-
increase the latency of all misses (even cache bank and thus should also add ry must also include tags (there for free

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 87
contributed articles

in the original inclusive shared cache), ligible latency to cache hits, off-chip
and the relative overhead becomes accesses, and misses to blocks not ac-
larger if the hardware designer opts for tively shared; miss latency for actively
a smaller shared cache. shared blocks is higher, but the ratio
To be concrete, let S1 be the sum
of private cache sizes, S2 the shared Forcing software of the latencies for these misses is tol-
erable today and independent of the
cache size, D the directory entry size
relative to the size of a private cache
to use software- number of cores. Energy overheads of

managed coherence
coherence are correlated with traffic
block and tag, and R, the ratio of the and storage, so we find no reason for
number of directory entries to the to-
tal number of private cache blocks. R
or explicit message energy overheads to limit the scalabil-
ity of coherence. Extensions to a non-
should be greater than 1 to keep recalls passing does not inclusive shared cache show larger but
rare, as discussed earlier in the section
on maintaining inclusion. Directory
remove complexity manageable storage costs when shared
cache size is larger than the sum of pri-
storage adds R×S1×D to cache storage but rather shifts vate cache size. With coherence’s costs
S1+S2 for a relative overhead of (R×D)/
(1+S2/S1). Assume that R=2 and D=64b/ complexity from shown to scale, we expect on-chip co-
herence is here to stay due to the pro-
(48b+512b). If S2/S1 is 8, as in Core i7, hardware to grammability and compatibility ben-
then directory storage overhead is only
2.5%. Shrinking S2/S1 to 4, 2, and 1 in- software. efits it delivers.
Nevertheless, this work has limita-
creases relative overhead to 4.6%, 7.6%, tions and potential criticisms. First, we
and 11%, respectively. did not include detailed architectural
The use of hierarchy adds another simulations with specific benchmarks
level of directory and an L3 cache. With- or consider difficult-to-model queuing
out inclusion, the new directory level effects due to cache and interconnect
must point to an L2 bank if a block is contention. Instead, we showed that
either in the L2 bank or in its co-locat- coherence’s per-miss traffic is inde-
ed directory. For cache size ratio Z = S3/ pendent of the miss pattern and num-
S2 = S2/S1 = 8, the storage overhead for ber of cores. Though less precise than
reaching 256 cores is 3.1%. Shrinking Z detailed simulation, our results are
to 4, 2, or 1 at most doubles the relative more robust, as they are not limited to
overhead to 6.5%, 13%, or 23%, respec- the specific benchmarks studied. Fur-
tively. Furthermore, such storage over- thermore, we described our protocol
heads translate into relatively lower as an existence proof of a scalable co-
overheads in terms of overall chip area, herence protocol but do not claim it is
as caches are only part of the chip area. the best. To this more modest end, less
Overall, we find that directory storage precision is required.
is still reasonable when the cache size Second, we did not compare our
ratio Z > 1. protocol against multicore chips with-
out caches or without a shared address
Caveats and Criticisms space. Though these approaches have
We have described a coherence proto- been successful in high-performance
col based on known ideas to show the computing, they are not common in
costs of on-chip coherence grow slowly mainstream multicore systems. Given
with core count. Our design uses a hier- that coherence’s costs can be kept low
archy of inclusive caches with embed- and that some operating systems use
ded coherence state whose tracking in- hardware coherence to scale to many
formation is kept precise with explicit cores,2,3 we find no compelling reason
cache-replacement messages. Using to abandon coherence. We thus an-
amortized analysis, we have shown that ticipate alternatives to cache-coherent
for every cache miss request and data shared memory will continue to ex-
response, the interconnection network ist and thrive in certain domains but
traffic per miss is independent of the that on-chip coherence will continue
number of cores and thus scales. Em- to dominate in mainstream multicore
bedding coherence state in an inclu- chips. Furthermore, coherent systems
sive cache hierarchy keeps coherence’s can support legacy algorithms from
storage costs small; for example, 512 these other domains, as any program
cores can be supported with 5% extra that works for scratchpad systems
cache area with two cache levels or 2% (such as the Cell processor) or message
with three levels. Coherence adds neg- passing systems (such as an MPI clus-

88 communicatio ns o f th e acm | j u ly 201 2 | vo l. 5 5 | no. 7

 contributed articles

ter) maps easily to a shared memory limitations, scalability of the on-chip performance. IEEE Micro 31, 2 (Mar./Apr. 2011), 6–15.
5. Choi, B., Komuravelli, R., Sung, H., Smolinski, R.,
system with caches. interconnect, and critical problems Honarmand, N., Adve, S.V., Adve, V.S., Carter, N.P., and
Third, we are aware of the complexi- of software non-scalability. Despite Chou, C.-T. DeNovo: Rethinking the memory hierarchy
for disciplined parallelism. In Proceedings of the 20th
ty challenge posed by coherence and do advances in scaling operating systems International Conference on Parallel Architectures
not underestimate the importance of and applications, many applications and Compilation Techniques (Galveston Island, TX,
Oct. 10–14). IEEE Computer Society, Washington,
managing complexity but also that the do not (yet) effectively scale to many D.C., 2011, 155–166.
chip-design industry has a long history cores. This article does not improve 6. Conway, P., Kalyanasundharam, N., Donley, G., Lepak,
K., and Hughes, B. Cache hierarchy and memory
of managing complexity. Many com- that situation. Nevertheless, we have subsystem of the AMD Opteron processor. IEEE Micro
panies have sold many systems with shown that on-chip hardware coher- 30, 2 (Mar./Apr. 2010), 16–29.
7. Ferdman, M., Lotfi-Kamran, P., Balet, K., and Falsafi,
hardware cache coherence. Designing ence can be made to scale gracefully, B. Cuckoo directory: Efficient and scalable CMP
coherence. In Proceedings of the 17th Symposium
and validating the coherence protocols freeing application and system soft- on High-Performance Computer Architecture (San
in them is not easy, but industry con- ware developers from having to re- Antonio, TX, Feb. 12–16). IEEE Computer Society,
Washington, D.C., 2011, 169–180.
tinues to overcome these challenges. implement coherence (such as know- 8. Hill, M.D., Larus, J.R., Reinhardt, S.K., and Wood, D.A.
Moreover, the complexity of coherence ing when to flush and refetch data) or Cooperative shared memory: Software and hardware
for scalable multiprocessors. ACM Transactions on
protocols does not necessarily scale up orchestrating explicit communication Computer Systems 11, 4 (Nov. 1993), 300–318.
with increasing numbers of cores. Add- via message passing. 9. Hill, M.D. and Smith, A.J. Evaluating associativity in
CPU caches. IEEE Transactions on Computers 38, 12
ing more cores to an existing multicore Conclusion. On-chip coherence can (Dec. 1989), 1612–1630.
design has little effect on the conceptu- be made to scale gracefully, enabling 10. Howard, J. et al. A 48-core IA-32 message-passing
processor with DVFS in 45nm CMOS. In Proceedings
al complexity of a coherence protocol, programmers to concentrate on what of the International Solid-State Circuits Conference
though it may increase the amount of matters for parallel speedups—find- (San Francisco, Feb. 7–11, 2010), 108–109.
11. Jaleel, A., Borch, E., Bhandaru, M., Steely Jr., S.C., and
time necessary to validate the protocol. ing work to do in parallel without undo Emer, J. Achieving noninclusive cache performance
However, even the validation effort communication and synchronization. with inclusive caches: Temporal locality-aware
cache management policies. In Proceedings of the
may not pose a scalability problem; 43rd Annual IEEE/ACM International Symposium on
research shows it is possible to de- Acknowledgments Microarchitecture (Atlanta, Dec. 4–8). IEEE Computer
Society, Washington, D.C., 2010, 151–162.
sign hierarchical coherence protocols We thank James Balfour, Colin 12. Kelm, J.H., Johnson, D.R., Johnson, M.R., Crago, N.C.,
Tuohy, W., Mahesri, A., Lumetta, S.S., Frank, M.I.,
that can be formally verified with an Blundell, Derek Hower, Steve Keckler, and Patel, S.J. Rigel: An architecture and scalable
amount of effort that is independent Alvy Lebeck, Steve Lumetta, Steve Re- programming interface for a 1,000-core accelerator.
In Proceedings of the 36th Annual International
of the number of cores.19 Furthermore, inhardt, Mike Swift, and David Wood. Symposium on Computer Architecture (Austin, TX,
the complexity of the alternative to This material is based on work sup- June 20–24). ACM Press, New York, 2009, 140–151.
13. Kelm, J.H., Johnson, D.R., Tuohy, W., Lumetta, S.S.,
hardware coherence—software imple- ported by the National Science Foun- and Patel, S.J. Cohesion: An adaptive hybrid memory
mented coherence—is non-zero. As dation (CNS-0720565, CNS-0916725, model for accelerators. IEEE Micro 31, 1 (Jan./Feb.
2011), 42–55.
when assessing hardware coherence’s CNS-1117280, CCF-0644197, CCF- 14. Laudon, J. and Lenoski, D. The SGI Origin: A ccNUMA
overheads—storage, traffic, latency, 0905464, CCF-0811290, and CCF- highly scalable server. In Proceedings of the 24th
Annual International Symposium on Computer
and energy—chip architects must be 1017650); Sandia/Department of Ener- Architecture (Denver, June 2–4). ACM Press, New
careful not to implicitly assume the gy (MSN123960/DOE890426); and the York, 1997, 241–251.
15. Nickolls, J. and Dally, W.J. The GPU computing era.
alternative to coherence is free. Forc- Semiconductor Research Corporation IEEE Micro 30, 2 (Mar./Apr. 2010), 56–69.
ing software to use software-managed (2009-HJ-1881). Any opinions, find- 16. Shah, M., Barren, J., Brooks, J., Golla, R., Grohoski,
G., Gura, N., Hetherington, R., Jordan, P., Luttrell, M.,
coherence or explicit message passing ings, and conclusions or recommen- Olson, C., Sana, B., Sheahan, D., Spracklen, L., and
does not remove the complexity but dations expressed here are those of Wynn, W. UltraSPARC T2: A highly treaded, power-
efficient SPARC SOC. In Proceedings of the IEEE
rather shifts the complexity from hard- the authors and do not necessarily re- Asian Solid-State Circuits Conference (Jeju, Korea,
Nov. 12–14, 2007), 22–25.
ware to software. flect the views of the National Science 17. Singhal, R. Inside Intel next-generation Nehalem
Fourth, we assumed a single-chip Foundation, Sandia/DOE, or SRC. The microarchitecture. Hot Chips 20 (Stanford, CA, Aug.
24–26, 2008).
(socket) system and did not explicitly authors have also received research 18. Sorin, D.J., Hill, M.D., and Wood, D.A. A Primer on
address chip-to-chip coherence in to- funding from AMD, Intel, and NVIDIA. Memory Consistency and Cache Coherence. Morgan &
Claypool Publishers, 2011.
day’s multisocket servers. The same Hill has a significant financial interest 19. Zhang, M., Lebeck, A.R., and Sorin, D.J. Fractal
sort of tagged tracking structures can in AMD. coherence: Scalably verifiable cache coherence.
In Proceedings of the 43rd Annual IEEE/ACM
be applied to small-scale multisocket International Symposium on Microarchitecture
systems,6 essentially adding one more References
(Atlanta, Dec. 4–8). IEEE Computer Society,
Washington, D.C., 2010, 471–482.
level to the coherence hierarchy. More- 1. Agarwal, A., Simoni, R., Horowitz, M., and Hennessy,
J. An evaluation of directory schemes for cache
over, providing coherence across mul- coherence. In Proceedings of the 15th Annual
Milo M.K. Martin ([email protected]) is an
tisocket systems may become less im- International Symposium on Computer Architecture
associate professor in the Computer and Information
(Honolulu, May). IEEE Computer Society Press, Los
portant, because single-chip solutions Alamitos, CA, 1988, 280–298. Science Department of the University of Pennsylvania,
2. Boyd-Wickizer, S. Clements, A.T., Mao, Y., Pesterev, Philadelphia, PA.
solve more needs, and “scale out” so-
A., Kaashoek, M.F., Morris, R., and Zeldovich, N. Mark D. Hill ([email protected]) is a professor in both
lutions are required in any case (such An analysis of Linux scalability to many cores. In the Computer Sciences Department and the Electrical and
as for data centers), but that is an argu- Proceedings of the Ninth USENIX Symposium on Computer Engineering Department of the University of
Operating Systems Design and Implementation Wisconsin-Madison.
ment for another article. (Vancouver, Oct. 4–6). USENIX Association, Berkeley,
CA, 2010, 1–8. Daniel J. Sorin ([email protected]) is an associate
Finally, even if coherence itself 3. Bryant, R. Scaling Linux to the extreme. In professor in the Electrical and Computer Engineering
scales, we did not address other is- Proceedings of the Linux Symposium (Boston, June and Computer Science Departments of Duke University,
27–July 2, 2004), 133–148. Durham, NC.
sues that might prevent practical 4. Butler, M., Barnes, L., Sarma, D.D., and Gelinas, B.
multicore scaling, such as die-area Bulldozer: An approach to multithreaded compute © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 89
review articles
doi:10.1145/ 2209249.2209270
To illustrate the naturalness of con-
A novel paradigm for programming reactive structing systems by composing be-
haviors, consider how children may be
systems centered on naturally specified taught, step-by-step, to play strategy
modular behavior. games (See Gordon et al.14). For exam-
ple, in teaching the game of Tic-Tac-
By David Harel, Assaf Marron, and Gera Weiss Toe, we first describe rules of the game,
such as:

Behavioral
EnforceTurns: To play, one player
marks a square in a 3 by 3 grid with X,
then the other player marks a square
with O, then X plays again, and so on;
SquareTaken: Once a square is

Programming
marked, it cannot be marked again;
DetectXWin/DetectOWin: When
a player places three of his or her marks
in a horizontal, vertical, or diagonal
line, the player wins;
Spelling out the requirements for a software system Now we may already start playing.
Later, the child may infer, or the teach-
under development is not an easy task, and translating er may suggest, some tactics:
captured requirements into correct operational software AddThirdO: After placing two Os in
can be even harder. Many technologies (languages, a line, the O player should try to mark
the third square (to win the game);
modeling tools, programming paradigms) and PreventThirdX: After the X player
methodologies (agile, test-driven, model-driven) were marks two squares in a line, the O play-
er should try to mark the third square
designed, among other things, to help address these (to foil the attack); and
challenges. One widely accepted practice is to formalize DefaultOMoves: When other tac-
requirements in the form of use cases and scenarios. tics are not applicable, player O should
prefer the center square, then the cor-
Our work extends this approach into using scenarios
for actual programming. Specifically, we propose key insights
scenario-coding techniques and design approaches for 
Behavioral programming is a novel,
constructing reactive systems28 incrementally from their language-independent paradigm for
programming reactive systems, centered
expected behaviors. on natural and incremental specification
of behavior, and implemented in the visual
The work on behavioral programming began formalism of live sequence charts (LSC),
with scenario-based programming, a way to create and in the BPJ Java package.

executable specifications of reactive systems, 

The approach allows coding
applications as multi-modal scenarios,
introduced through the language of live sequence each corresponding to an individual
requirement, specifying what can, must,
charts (LSC) and its Play-Engine implementation.11,21 or may not happen following certain
sequences of events.
The initial purpose was to enable testing and refining

To facilitate full behavioral modularity
specifications and prototypes, and it was later via the independent coding of separate
facets of behavior, all scenarios run
extended toward building actual systems. To this end, simultaneously, and all are consulted at
the underlying behavioral principles have also been every decision point during execution.

implemented in Java via the BPJ package25 and in 

The paradigm is supported by tools
for debugging, execution planning,
additional environments,26,34,42,43 adding a programming learning-based adaptivity, and
model-checking for early detection of
point of view to that of requirement specification. conflicting and incomplete requirements.

90 comm unicatio ns o f the acm | j u ly 201 2 | vo l. 5 5 | no. 7

 ners, and mark an edge square only system is achieved that plays legally and product is modular, in that tactics
when there is no other choice. assures the computer never loses. and rules can be flexibly added and re-
Such required behaviors can be cod- This example already suggests the moved to create versions with different
ed in executable software modules using following advantages of behavioral functionalities, for example, to play at
behavioral programming idioms and programming. First, we were able to different expertise levels.
infrastructure, as we will detail. Full be- code the application incrementally in Naturally, composing behaviors
Illustration by Leandro C astel ao

havioral implementations of the game modules that are aligned with the re- that were programmed without di-
exist in Java25 and Erlang.43 In Harel et quirements (game rules and tactics), rect consideration of mutual de-
al.20 we show how model-checking tech- as perceived by users and program- pendencies raises questions about
nologies allow discovery of unhandled mers. Second, we added new tactics conflicting requirements, under-
scenarios, enabling the user to incre- and rules (and still more can be add- specification, and synchronization.
mentally develop behaviors for new tac- ed) without changing, or even looking We deal with these issues by using
tics (and forgotten rules) until a software at, existing code. Third, the resulting composition operators that allow

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he ac m 91
review articles

both adding and forbidding behav- In this article, we present the prin- behavioral programming with the LSC
iors, analysis tools such as model ciples of behavioral programming and language and elaborate on how one
checkers, and architectures for large- illustrate how to program behavioral deals with conflicting behaviors, un-
scale applications. applications in Java. We detail visual derspecification, and a large number
of simultaneous behaviors. We con-
Figure 1. A schematic view of the execution cycle of behavior threads using an enhanced clude with a comparison to other devel-
publish/subscribe protocol.
opment approaches, applications, and
future research.

behavior
Requested Events Basic Behavioral Idioms
thread We propose the term behavioral ap-
plication for software consisting of
behavior Blocking
thread independent components (called
b-threads) that generate a flow of
behavior events via an enhanced publish/sub-
thread
scribe protocol, as follows (see Fig-
behavior ure 1). Each b-thread is a procedure
thread Selected Event that runs in parallel to the other b-
threads. When a b-thread reaches a
point that requires synchronization,
it waits until all other b-threads reach
1. All behavior threads synchronize and place their “bids”:
synchronization points in their own
˲˲ Requesting an event: proposing that the event be considered for triggering, and asking to be
flow. At synchronization points, each
notified when it is triggered;
b-thread specifies three sets of events:
˲˲ Waiting for an event: without proposing its triggering, asking to be notified when the event is
requested events: the thread proposes
triggered;
that these be considered for trigger-
˲˲ Blocking an event: forbidding the triggering of the event, vetoing requests of other behavior
ing, and asks to be notified when any
threads. of them occurs; waited-for events: the
2. An event that is requested and not blocked is selected; thread does not request these, but
asks to be notified when any of them
3. The behavior threads that requested or wait for the selected event are notified;
is triggered; and blocked events: the
4. The notified behavior vthreads progress to their next states, where they place new bids.
thread currently forbids triggering
any of these events.
When all b-threads are at a synchro-
nization point, an event is chosen, that
Figure 2. B-threads for increasing water flow. The first two b-threads request the events is requested by at least one b-thread
addHot and addCold three times, respectively. The third b-thread, Interleave, repeatedly
waits for addHot while blocking addCold and vice versa, forcing alternation of these
and is not blocked by any b-thread.
Figure 2. B-threads for increasing water flow. The selected event is then triggered by
events. Without Interleave, the run would be three addHot followed by three addCold,
due to b-thread priorities. resuming all the b-threads that either
The first two b-threads
requested it or are waiting for it. Each
class AddHotThreeTimes extends BThread { request addHot and of these resumed b-threads then pro-
public void runBThread() { addCold three times, ceeds with its execution, all the way to
for (int i = 1; i <= 3; i++) { respectively. The third
bp.bSync( addHot, none, none ); its next synchronization point, where
b-thread, Interleave,
} repeatedly waits for it again presents new sets of request-
} addHot while blocking ed, waited-for and blocked events. The
} addCold and vice versa, other b-threads remain at their last
forcing alternation of
class AddColdThreeTimes extends BThread { these events. Without
synchronization points, oblivious to
public void runBThread() { Interleave, the run would the triggered event, until an event is
for (int i = 1; i <= 3; i++) { Event log of
be three addHot followed
the coordinated run selected that they have requested or
bp.bSync( addCold, none, none ); by three addCold, due
} are waiting for. When all b-threads are
to b-thread
addHotpriorities.
} addCold again at a synchronization point, the
} addHot event selection process repeats. For a
addCold formal definition of this process see
class Interleave extends BThread addHot
public void runBThread() { addCold
Harel et al.25,26
while (true) { When more than one event is re-
bp.bSync( none, addHot, addCold ); quested and not blocked, the seman-
bp.bSync( none, addCold, addHot );
}
tics of event selection may vary. For ex-
} ample, the selection may be arbitrary
} or random, as in the default (a.k.a. naïve)
semantics of the LSC Play-Engine;21

92 com municatio ns o f the ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 review articles

choices may depend on some priority to specify its requested, waited-for and that controls hot and cold water taps,
order, as in standard BPJ execution; the blocked events as follows: whose output flows are mixed.
mechanism may use look-ahead sub- As shown in Figure 2, let AddHot-
ject to desired properties of the result- bSync(requestedEvents, ThreeTimes be a b-thread that re-
ing event sequence, as in smart play- waitedForEvents, quests three times the event of opening
out19,29 in LSC; it may vary over time, blockedEvents); the hot water-tap some small amount
based on learning;13 or, as in Kugler et (addHot), and then stops. The b-thread
al.,34 the entire execution may diverge By calling bSync the b-thread sus- AddColdThreeTimes performs a sim-
into multiple concurrent paths. pends itself until all other b-threads ilar action on the cold water tap (with
The programming idioms of re- are at a synchronization point and the event addCold). To increase water
quest, wait for, block thus express multi- is resumed when an event that it re- flow in both taps more or less at the
modality. Reminiscent of modal verbs quested or waited for is selected, as same time, as may be desired for keep-
in a natural language (such as shall, described below. ing the temperature stable, we activate
can or mustn’t), they state not only what To enforce predictable and repeat- the above b-threads alongside a third
must be done (and how) as in standard able execution, we require that the one, Interleave, which forces the al-
programming, but also what may be event selected at each synchronization ternation of their events. Interleave
done, and, more uniquely to behavior- point be uniquely defined. To this end, repeatedly waits for addHot while
al programming, what is forbidden and the programmer assigns a unique pri- blocking addCold, followed by waiting
therefore must not be done. ority to each b-thread, and places the for addCold while blocking addHot.
Behavioral programming prin- requested events of each b-thread in an Later, we illustrate a similar program
ciples can be readily implemented ordered set. The event selection mech- written in the visual LSC language.
as part of different languages and anism in BPJ then uses this ordering to Behavioral execution can be further
programming approaches, with pos- choose the first event that is requested analyzed with table-like visuals, as in
sible variations of idioms. In addition and not blocked. Figure 3, which was generated by the
to Java with the BPJ package25 (dis- The source code package of BPJ is TraceVis trace-comprehension and
cussed later in more detail) we have available online at http://www.b-prog. debugging tool.12 Briefly, b-threads
implemented them in the functional org with examples and video demon- are depicted in columns ordered by
language Erlang26,43 and Shimony et strations. priority, and successive synchroniza-
al.42 applied them in the PicOS envi- Example: Water flow control. To il- tion points and associated triggered
ronment using C. Implementations lustrate how these constructs can be events appear in rows intersecting
in visual contexts beyond the original used to allow new behaviors to non- the b-thread columns. Each table cell
Play-Engine include PlayGo23 and SBT intrusively affect existing ones, con- describes a b-thread’s state at a given
by Kugler et al.34 sider scenarios that are part of a system synchronization point. The sets of
In behavioral programming, all
one must do in order to start develop- Figure 3. Visualizing an execution of the water-tap application with TraceVis. Selected
events are marked with a green star; blocked events are marked with a red square; cells
ing and experimenting with scenarios marked R/W/B show requested, waited for, and blocked events.
that will later constitute the final sys-
tem, is to determine the common set
of events that are relevant to these sce- AddHotThreeTimes AddColdThreeTimes Interleave
Reset Leader Leader Active
narios. While this still requires con-
templation, it is often easier to answer R AddHot R AddHot R
the question “what are the events?” W W W AddHot
1 AddHot
than “which are the objects/functions,
B B B AddCold
etc.?” By default, events are opaque en-
tities carrying nothing but their name,
but they may be extended with rich R AddHot R AddCold R

data and functionality. 2 AddCold W W W AddCold

B B B AddHot
Programming Behaviors In Java
Our implementation of behavioral pro- R AddHot R AddHot R
gramming in Java uses the BPJ pack- 3 AddHot W W W AddHot
age.25 With BPJ, each behavior thread
B B B AddCold
is an instance of the class BThread.
Events are instances of the class
R AddHot R AddCold R
Event or classes that extend it (mainly
for adding data to events). The logic 4 AddCold W W W AddCold

of each behavior is coded as a method B B B AddHot

supplied by the programmer, which
in turn invokes the method bSync to
synchronize with other behaviors, and

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t he acm 93
review articles

requested, waited-for, and blocked We describe the incremental de- moves and declares the end of the game
events are shown in sub-cells marked velopment of all the b-thread classes with no winner, and PreventThirdX
R, W, and B respectively. In each row, needed for the rules and tactics in ignores O moves before requesting its
all appearances of the selected event Harel et al.25 Here, we describe the flow own desired move.
are marked with a green star, and re- of some of the b-threads to illustrate Focusing on a narrow facet of a be-
quested events that are blocked are how the natural language descriptions havior can simplify the b-thread and
marked by red squares, providing in- in the beginning of the article, can be can be accomplished by instantiat-
sight into the rationale of event selec- translated to code which includes calls ing copies of it with different param-
tion and b-thread progression. The cell to bSync. The b-thread for the game- eters. For example, we implemented
containing the request that drove the rule SquareTaken, for example, first SquareTaken with an instance for
event triggering is emphasized with calls bSync to wait for any X or O event each square, and DetectXWin with an
a bold border, and cells of b-threads and then calls bSync again to block instance for each permutation of three
that did not advance are marked by a all events in the newly marked square. X events in each line.
dashed border. As another example, the b-thread De- The autonomy afforded by a narrow
Example: Strategies for Tic-Tac-Toe. faultOMoves uses a Java loop to re- world view is facilitated also by the fact
Behavioral programming supports in- peatedly request (by calling bSync) that all b-threads that request a given
cremental development, where new the set of all nine possible O moves event at a particular synchronization
behaviors may be added non-intru- ordered with center first, then corners, point are notified when it occurs, and
sively, that is, with little or no change and then edge squares. An example of are unaware of whether the selected re-
to existing code. We demonstrate this a longer scenario is AddThirdO, which quest was theirs or came from another
trait with an application for playing waits for an O event, then waits for an- b-thread. For example, a single mark-
the game of Tic-Tac-Toe, described in other O event in the same line, and ing of an O in a particular square could
detail in Harel et al.25 As outlined in then requests an O event marking the result from simultaneous requests by
the introduction, players X (a human) third square in the line. the AddThirdO, PreventThirdX,
and O (the computer) alternately mark To demonstrate incremental devel- and DefaultOMoves b-threads. Us-
squares on a grid of 3 rows by 3 col- opment, consider how when we learn ing blocking and priorities, autono-
umns, each attempting to place three that our defense behaviors are insuffi- mous b-threads can “carve out” un-
of her marks in a full horizontal, verti- cient against a corner-center-corner at- desired behaviors of other b-threads,
cal or diagonal line. Each marking of tack (for example, Xá0,0ñ, Oá1,1ñ, Xá2,2ñ) as, say, with coding DefaultOmoves
a square labeled árow, colñ is repre- for which the only defense is a coun- to repeatedly ask for the same set of
sented by a move event, Xárow,colñ or terattack, we can add a b-thread as fol- events without checking which one
Oárow,colñ. The events XWin, OWin lows. To foil X’s plan, the new b-thread was triggered, and then adding the b-
and Draw mark possible conclusions waits for this sequence of events (and thread SquareTaken.
of the game. equivalent ones), and attacks back Example: Real-time aircraft stabili-
A play of the game can be described by requesting the move Oá0,1ñ. Later, zation. Given the principles described
as a sequence of events. For example, we discuss how this development ap- so far, one may ask how behavioral pro-
the sequence Xá0,0ñ, Oá1,1ñ, Xá2,1ñ, proach can be enhanced using a mod- grams deal with external events, such
Oá0,2ñ, Xá2,0ñ, Oá1,0ñ, Xá2,2ñ, XWin, el-checking tool. as physical ones originating in the en-
describes a play in which X wins, and B-threads may autonomously watch vironment, or user actions. This sec-
whose final configuration is depicted out for very specific sequences of events tion briefly introduces elements that
in Figure 4. embedded in larger traces, with expres- can serve in a layer above the behav-
siveness that goes beyond responding ioral programming infrastructure for
Figure 4. The Tic-Tac-Toe gameboard to a single event or to a combination development of real-time systems. See
configuration following the move events
Xá0,0ñ, Oá1,1ñ, Xá2,1ñ, Oá0,2ñ,
of conditions, as is common in basic Harel et al.26 for more details.
Xá2,0ñ, Oá1,0ñ, Xá2,2ñ. rule engines. Moreover, in our experi- Behavioral applications can de-
ence, a given “world configuration” or tect external events at any time, using
a complete event sequence may be as- all the features available in the host
signed different meanings by different language, and can introduce them as
behaviors as they individually work to- behavioral events in the next synchro-
ward different goals. For example, De- nization point. For the integration of
tectXWin and PreventThirdX can behavioral and non-behavioral parts of
independently observe the same two X an application, we adopt the following
moves in the same line, but while the scheme, based on the concept of super-
former then waits for another X move steps, which is similar to the timing se-
toward announcing a win, the latter mantics of Statecharts.27
proceeds to make an O move in the third The first super-step begins when
square to prevent a loss. In fact, most of the system starts. Then, internal b-
our Tic-Tac-Toe b-threads do not check thread-driven events are triggered
the game configuration; for example, a until there are no more such events
b-thread DetectDraw counts any nine to trigger. At this point the behavioral

94 com municatio ns o f th e acm | j uly 201 2 | vol . 5 5 | no. 7

 review articles

system halts, all b-threads are inside a of different sizes, many small RPM- picts behavior using vertical lifelines
bSync method call, and the system is change events occur before new input to represent objects and horizontal
waiting for an external event. When an of desired forces is obtained in the next arrows for messages passed between
external event occurs and introduced super-step. The actual b-threads are them, with time flowing from top to
as a behavioral one, it marks the begin- more involved than those shown here, bottom. This yields a partial order for
ning of a new super-step, which then but they maintain their naturalness occurrences of the events in a chart.
continues until there are no events to and independence. For a fuzzy-logic However, the expressive power of MSC
trigger, and so on. We propose a con- based approach to implementing con- is limited,11 as these charts describe
vention, whereby external events are trol with behavioral programming, see possible scenarios and cannot specify,
not introduced as behavioral events as Harel et al.24 for example, what is mandated or what
long as there are other internal events is not allowed. In fact, given a set of
to trigger. In LSC, this is enforced by Live Sequence Charts objects and events, a system that gen-
the Play-Engine and PlayGo tools. In The visual language of live sequence erates all possible sequences of events
BPJ, the programmer can assign to charts (LSC) introduced scenario- would satisfy any MSC.
a b-thread that introduces external based programming, and implicitly To address this, in a live sequence
events a priority lower than that of any also the basic concepts of behavioral chart one can distinguish what must
b-thread that may request other (inter- programming; see Damm and Harel.11 happen (called hot in LSC terminol-
nal) events at the same time. One may One continuation of that work was the ogy, and colored red) from what may
view the super-step as an ordered se- invention of the play-in and play-out happen (termed cold, and colored
quence of events, which ideally takes techniques for constructing and exe- blue), and can also express what is not
“zero time,” as in Berry’s synchrony cuting LSCs, which were implemented allowed to happen (forbidden). More-
hypothesis4 and in Statecharts,27 and in the Play-Engine tool.21 A more re- over, event specifications that are to be
similar to hybrid time sets and logical cent tool, PlayGo, has been developed, executed in a proactive manner can be
execution time (LET) design.31 and is currently being extended and distinguished from ones that specify
We now outline parts of the soft- strengthened.23 The LSC approach monitoring; that is, merely tracking
ware for controlling a quadrotor, an also inspired the SBT tool.34 While the the event. LSC also distinguishes be-
aircraft lifted and propelled by four current status of these tools does not tween universal charts, which depict
fixed rotors, as detailed in Harel et al.26 yet enable broad usage in real-world executions that are to apply to all runs,
One of the challenges in stabilizing a applications, the versatility of the LSC and existential charts—which serve as
quadrotor is using a fixed set of con- language has been demonstrated in “examples” and are required to apply
trols, namely the rotors’ speed (RPM), various application domains, includ- only to at least one run. A universal
to balance competing goals like de- ing hardware, telecommunication, LSC consists of a prechart and a main
sired forces and moments along differ- production control, tactical simula- chart, as in Figure 5. The semantics
ent axes: flight direction, roll (side-to- tors, and biological modeling.2,10,41 is that if and when the behavior de-
side), pitch (raising and lowering the LSC adds liveness and execution scribed by the prechart occurs, the
front), and yaw (rotation of the entire semantics to behaviors described us- behavior described by the main chart
quadrotor). These goals compete with ing message sequence charts (MSC) by must occur too.
each other as changes in any rotor extending MSC with modalities, sym- Using a designated chart area, one
speed may affect multiple forces. For bolic instances, and more. An MSC de- can forbid occurrence of events at
example, changing the back rotor’s
RPM affects the thrust, the pitch and Figure 5. A universal LSC. Whenever a telephone user presses the sequence of a star,
a digit, and send (see hexagonal prechart), the chip must retrieve the corresponding
the yaw. Behavioral programming al- number from memory and call it by sending a message to the environment. If a busy
lows decomposing the application into signal is returned, the call must be tried up to three times. The events in the main chart
b-threads, each of which takes care of may occur only in the order specified.
only one force. For example, “when
thrust is too low, request the increase
of at least one of the rotors’ RPM and
block the decrease of all rotors’ RPM” * Keyboard Send key Chip Memory ENV

or “when pitch angle is too high, re- Click

quest the increase of the back rotor’s Click (digit)
RPM or the decrease of the front rotor’s Click

RPM while blocking the increase of the Retrieve (digit)

back rotor’s RPM and the decrease of number

the front rotor’s RPM.” Note that the 3 Call(number)

event selection mechanism will weave signal
these two behavior threads, in such signal busy
a way that when the thrust is too low
and the pitch is too high, only the back
rotor’s RPM will increase, address-
ing both deviations. To fix deviations

ju ly 2 0 1 2 | vo l. 55 | n o. 7 | c om m u n ic at ion s o f t he acm 95
review articles

certain times. There are other ways to event is to be executed or is only moni- This process is often referred to
forbid things from occurring; one of tored, while the red and blue color as naïve play-out. In a more advanced
which is done by indicating that events retain their respective modalities of mechanism, called smart play-out19,29
in the main chart must occur only in must and may. The PlayGo tool18,23 is the Play-Engine uses either model-
the specified (partial) order. For ex- currently based on this version of the checking or AI planning algorithms
ample, one can specify that when the language. Figure 6 depicts a PlayGo to look ahead, in an attempt to select
main chart is active, events that appear example similar to the water-tap appli- events in ways that do not eventually
in the chart but are not presently en- cation discussed earlier, with the addi- lead to violation of the specification
abled cannot be triggered at that point tion of the user pressing a start button or deadlock.
in time by other charts. to activate all scenarios. In addition to the interpreter-like
A modest view of LSC considers it Internally, the LSC play-out mecha- approach of play-out, a compiler for
to be a requirements and specification nism uses the request / wait / block LSC has been developed, which pro-
language for making assertions about idioms for collective execution, as fol- duces executable code by compiling
sequences of events. In this view, a sys- lows. Initially, the next enabled event the specification into Java and weaving
tem satisfies an LSC specification if all for each lifeline in each chart is the top- the results with AspectJ.38
its runs satisfy all the universal charts most event in the lifeline. All enabled One notable difference between LSC
in the specification, and for each exis- events on all lifelines are considered and the BPJ package is that BPJ benefits
tential chart, there is at least one run waited-for. All enabled events that are from the power of the Java host lan-
that satisfies it. also to be executed (and not just moni- guage. By contrast, the LSC language
However, the play-out technique fa- tored) are considered also as request- provides its own constructs for objects
cilitates the execution of an LSC speci- ed. All events that are forbidden, either and properties, flow of control, excep-
fication, that is, a collection of charts, explicitly or implicitly, are considered tions, variables, symbolic objects and
just like any computer program. Play- blocked. An event that is requested and messages, a notion of time, sub-chart
out does this by tracking events that not blocked is then triggered. When scope, access to functions in other lan-
may be selected next in all lifelines no event can be triggered, the system guages, and external communication.21
in all charts, selecting and triggering waits for an event from the user or the Play-in. An essential element of
events subject to the must/may/forbid- environment. When an event is trig- programming is the process by which
den modalities, and advancing affect- gered, a rich unification algorithm de- programmers perform actual coding.
ed charts accordingly.21 As described termines which event specifications in In behavioral programming, it seems
in more detail below, play-out may different charts refer to that event, and only natural to allow this activity to
be viewed as interpreting charts with all lifelines in which it is enabled are include walking through a scenario,
modal events as threads of behavior advanced to their next state. Whenever generating events and sequences
with their requested, waited-for, and this advancing causes a prechart to be thereof, and using them in specifying
blocked events. completed, the main chart portion of what we want done or forbidden. To-
A dialect of LSC has been designed the chart is activated. When a forbid- ward that purpose, the LSC language
to be compliant with UML 2.0,22 and den event nevertheless occurs, for ex- allows a new way of coding, called
can be defined as a profile therein. ample, as driven by the environment play-in,15,21 which captures scenarios
Instead of precharts it uses solid and or the user, a violation occurs and the as follows: whenever possible, the de-
dashed arrows to indicate whether an execution terminates. veloper actually performs the event—

Figure 6. UML-compliant LSCs. Each chart begins with the user pressing the start button. Two charts request tap-turning events, and the
third causes their interleaving by alternately waiting for these events. Events can occur only when enabled for triggering in all charts in
which they appear. The SYNC construct forces order between events in different lifelines.

LSC AddHotThreeTimes LSCAddColdThreeTimes LSC Interleave

User startButton hotWater User startButton hotWater User startButton hotWater coldWater

start()
start() start()
SYNC
SYNC SYNC loop
[–1,–1]
addHot() addCold() addHot()

addHot() addCold() SYNC

addHot() addCold() addCold()

96 comm unicatio ns o f th e ac m | j u ly 201 2 | vol . 5 5 | no. 7

 review articles

for example, by pressing “send” on a (as described earlier) that the victory
telephone—and the tool captures the of X could have been avoided had the
event and includes it as part of the application played Oá1,0ñ in its last
gradually generated LSC. The reader turn, preventing the completion of
is referred to Harel and Marelly21 and
the Web site http://www.wisdom.weiz- Play-in is similar three X marks in a line, instead of its
default preference to mark corners.
mann.ac.il/~playbook for more details.
Play-in is similar to programming
to programming Note that in coding refinements and
corrections, counterexamples pro-
by example37 in that both try to make by example in that vided by the tool can be used directly
programming easier for humans us-
ing visualization and physical actions,
both try to make since they are sequences of events.
See Harel et al.17 for an application of
and the approaches can certainly gain programming easier this property to the automatic local
from one another. The main differ-
ence is that programming by example
for humans using repair of behavioral programs.
From the model-checking perspec-
is a way to avoid writing code in small visualization and tive, the BPmc tool (which currently
programs, for educational purposes,
where play-in is intended for use as physical actions, applies to our Java implementation of
behavioral programming) reduces the
part of programming modal scenar- and the approaches size of the state-space of a Java pro-
ios to be executed collectively as a
complex system. can certainly gain gram using an abstraction that focuses
on the behaviorally interesting states

Can It Work In The Real World?

from one another. and treats transitions between them
as atomic. To the existing standard ex-
In way of trying to tackle such ques- ecution control, which consists of de-
tions as “can the approach deal with terministic progression along a single
conflicts and underspecification?” or path in the behavioral program state
“can one coordinate thousands of si- graph, we add two model-checking ex-
multaneous behaviors?” we outline ecution modes: safety and liveness. The
some relevant research results. safety mode explores the different paths
One concern associated with align- in the graph in search of a state that vio-
ing application scenarios with require- lates the given safety property, while
ments is that individually valid require- the liveness mode seeks cycles that
ments may conflict. Thus, coding them violate the given liveness property. The
independently of each other and com- graph traversal in BPmc is carried
posing them without consideration out with established model check-
may yield undesired joint behavior. ing algorithms and uses the Apache
We first observe that, as described, javaflow package to save and restore
our approach suggests resolving con- continuations—objects that hold the
flicts using new b-threads and priori- states of participating threads—for the
ties. For example, in Tic-Tac-Toe, the required backtracking.
conflict (which may emerge very early Synthesis techniques have also been
in development) concerning both applied to LSC, in order to check for
players requesting a move at the same conflicts and, when possible, to gen-
time, is resolved by a b-thread that erate a program that correctly imple-
enforces turn alternation. Similarly, a ments a system complying with the
conflict between a defensive move and specification.30,35
a move that yields an immediate win is Model checking and planning algo-
resolved by prioritizing the latter. rithms are used when running LSCs to
Further, we present a methodol- help avoid conflicts when these can be
ogy and a supporting model-check- resolved via “smart” event selection us-
ing tool (called BPmc) for verifying ing look-ahead within a super-step.19,29
behavioral programs without having Future research directions include
to first translate them into a spe- applying BPmc to achieve look-ahead
cific input language for the model in Java execution too, as well as going
checker.20 Our method facilitates beyond a single super-step in the smart
early discovery of conflicting or un- play-out method in LSC.
derspecified scenarios. For example, As for underspecification and
when model-checking a behavioral adaptability, it is well accepted in soft-
Tic-Tac-Toe application, the coun- ware engineering that a requirements
terexample Xá0,0ñ, Oá1,1ñ, Xá0,1ñ, document is never really complete,16
Oá0,2ñ, Xá2,0ñ, Oá2,2ñ, Xá1,0ñ suggests and that new requirements keep

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | com m u n i c at ion s o f t he ac m 97
review articles

emerging as developers and users which is fully synchronized internally, different modularization may cause
learn about and experiment with the and where the communication be- scenario encodings to be more subtle;
developed system. tween nodes is carried out by external rendering them visible only at run-
Similarly to the case of conflicts, events. The resulting system will still time. One of the main contributions
new requirements in behavioral pro- be incremental, in that new functional- of behavioral programming is the abil-
gramming can often be coded as new ity can be implemented by adding sce- ity to program multimodal scenarios
behaviors. For example, while develop- narios to different behavior nodes to incrementally using modules that are
ing the quadrotor application we real- generate and interpret (new) external aligned with requirements.16
ized that rotor speed (RPM) cannot be events, with little or no modification to Relative to object-oriented program-
negative. We solved this by adding a existing ones. ming, behavioral modules and events
b-thread that blocks speed reduction Consider a manager-employee rela- may involve objects, but they are not
events when the speed is too low. tion in a corporation. Each of the two necessarily anchored to a single one.
Obviously, both model checking and is constantly driven by a multitude of When programming behaviorally, one
the look-ahead mentioned here may (personal) behaviors, but without par- focuses on system behavior, and less
help detect and deal with such under- ticipation in each other‘s decisions. on identifying actors. Often, behavior
specification. The problem can also be The overhead of a communication pro- threads represent inter-object scenari-
handled by making the program learn tocol, the delays in reacting to messag- os that are not directly visible when the
and adapt as part of its development. es while continuing autonomous work, software is implemented as methods
For example, extending the semantics and indeed, the occasional correctable of individual objects. The states of such
of behavioral programming with rein- misunderstanding, are tolerable, and scenarios often conveniently replace or
forcements allows applications that are balanced with the efficiency and ef- complement data in standard objects.
also specify, in addition to what should ficacy afforded by autonomy. Ideas for using behaviors that are
be done or not done at every step, broad- The assignment of b-threads to specified as refinements and con-
er goals.13 Reinforcements are captured nodes should allow for discovering and straints over other modules are dis-
by b-threads, each one contributing a dealing with synchronization issues cussed in the context of superimposi-
narrow assessment of the current situ- in a local manner, using both model tions.7 Behavioral programming offers
ation relative to a longer-term goal. Le- checking and standard development practical programming mechanisms
veraging the unique structure of behav- and testing techniques. The division for implementing implicit, indirect
ioral programs, an application-agnostic into behavior nodes also simplifies pri- control of one behavior over all other
learning mechanism translates the re- ority assignment, in that one needs to relevant behaviors, without explicit
inforcements into event-selection deci- consider priorities only within a node. references from a controlling or con-
sions that improve over time. This abil- Different behavior nodes may be as- straining module to the controlled,
ity to learn and adapt allows removal of sociated with different time-scales, base module. Additionally, in behavior-
the need for a total order on b-threads reducing synchronization delays.26 al programming all system behaviors
and event requests, thus simplifying For example, a behavior node for han- are treated equally, without the distinc-
development. For example, a salad- dling a multistop travel itinerary of a tion between base and refinements.
making robot is specified in Eitan and quadrotor may synchronize at a much Aspect-oriented programming (AOP)33
Harel,13 with scenarios for picking up slower pace than the one responsible focuses on implementing cross-cut-
vegetables, and washing, cutting, and for stabilizing the aircraft at all times. ting concerns as separate modules
serving them in designated locations. This division may help also in the run- that can modify the behavior of base
With the help of reinforcements, the time detection of runaway b-threads, application modules. AOP’s relation
robot learns to perform these tasks in by using node-specific timers. to superimposition was pointed out
the correct order, while overcoming ob- The concept of behavior nodes in Katz and Gil.32 We believe behav-
stacles in the kitchen and dealing with that communicate only via events also ioral programming can contribute
refueling tasks. makes it easier to avoid race condi- toward implementing symmetric as-
Another concern around behavioral tions. In this context it should be noted pects, complementing the currently
programming execution is that if one di- that race conditions are completely prevalent asymmetric approach that
vergent b-thread (a runaway) fails to syn- avoided in behavioral programming if distinguishes base code from aspects.
chronize, the entire application stops. behaviors communicate only through In addition, while behavioral program-
The problem is of course aggravated events, and do not use host language ming allows the triggering of behaviors
when many behaviors are involved. facilities to share data.26 by sequences of events, in present AOP
As described in Harel et al.26 and implementations, join-points com-
following the work in Barak et al.,3 we Related Work and Future Directions monly represent individual events, and
expect that in large behavioral applica- In some languages (for example, work- triggering behaviors following rich se-
tions not all behaviors will be required flow engines or simulation specifica- quences of events requires non-trivial
to synchronize with each other. In- tions) scenarios and behaviors may be state management in the aspect code.
stead, we anticipate that synchroniza- encoded quite directly and visibly. In In robotics and hybrid-control there
tion requirements will be reduced by others (for example, procedural and ob- are behavior-based architectures, in-
dividing large numbers of naturally ject oriented programming, functional cluding Brooks’s subsumption archi-
specified behaviors into nodes; each of programming and logic programming) tecture,9 Branicky’s behavioral program-

98 comm unic atio ns o f the acm | j u ly 201 2 | vo l. 5 5 | no. 7

 review articles

ming,8 and leJOS,37 which construct sulting performance gains. In fact,

systems from behaviors (see the review some implementations of the behav-
in Arkin1). Our behavioral programming ioral execution mechanism are sin-
approach may serve as a formalism, gle-threaded. It would be interesting
implementation, or possible extension,
of some coordination and arbitration We believe to explore the synergy of behavioral
programming with such languages,
components in these architectures.
The test-driven or behavior-driven
that behavioral which could be done, for example, by
introducing blocking and synchro-
development methodologies (for ex- programming can nization idioms into non-behavioral
ample, JBehave, see http://jbehave.
org) emphasize the importance of ar-
contribute toward platforms and using established plat-
forms to connect behaviorally pro-
ticulating scenarios of expected over- implementing grammed nodes.
all system behavior early in develop-
ment. As the formal description of
symmetric aspects, The execution semantics of behav-
ioral programming has similarities
scenarios is shown to be valuable, we complementing the to the event-based scheduling of Sys-
propose that with behavioral program-
ming it may be possible to actually use currently prevalent temC,39 which performs co-routine
scheduling in three phases, evaluation,
such specifications as part of the de- asymmetric update, and notification, as follows: all
veloped system.
We feel that a key contribution of approach that runnable processes are run, one at a
time, up to a synchronization point;
behavioral programming to estab-
lished programming paradigms seems
distinguishes base queued updates are recorded; and,
processes affected by these updates are
to be the addition of a concise and au- code from aspects. then made runnable.
tonomous way for a process to block, or The BIP language (behavior, inter-
veto, events that other processes may action, priority) and the concept of glue
attempt to trigger. In common pub- for assembling components proposed
lish/subscribe mechanisms, for exam- by Sifakis and his colleagues (see, for
ple, such blocking would require ad- example, Bliudze and Sifakis5) pursue
ditional inter-process communication. goals similar to ours. Though some of
In research to be published separately, the terminology is similar, the specifics
we prove that the explicit blocking idi- are different. BIP focuses on creating a
om can make behavioral programs ex- system that is correct-by-construction
ponentially more succinct (in the num- with regard to safety properties like
ber of states) than traditional publish/ freedom from deadlock, while behav-
subscribe idioms. ioral programming concentrates on
Clearly, behavioral programming programming in a natural way, and
principles can be implemented in other turns to other techniques, including
languages and environments. We view model checking, to discover and re-
the approach as an enrichment of, not solve potential conflicts. A possible
an alternative to, current programming research direction involves adding syn-
approaches. In particular, constructs chronization and blocking as composi-
like semaphores/rendezvous, chan- tion idioms of BIP.
nels/message queues, and threads/con- Finally, behavioral programming
tinuations, can be used to implement may be suitable for software projects
and to complement the synchroniza- that call for feature-oriented develop-
tion and blocking of behavioral pro- ment40 and product-line packaging.
gramming. More specifically, in rich For example, an expert version of a
decentralized applications, behavioral game-playing program could differ
programming can coexist with actor- from the novice version by simply in-
oriented, agent-oriented, and other cluding behavior threads for addition-
concepts that enable coordination of al strategies. In Harel et al.25 we discuss
concurrent processes (see, for example, behavioral programming in relation
the survey in Bordini et al.6). to additional programming languages
Illustration by Leandro C astel ao

In this context, the main point and models.

about behavioral programming is its As to application domains, we note
focus on interweaving independent how features of behavioral program-
behaviors to yield a desired run (a se- ming contribute to making it useful for
quence of events), and the lesser focus particular domains, as follows. Coding
on issues related to parallel execution inter-object scenarios can be useful,
of independent behaviors and the re- for example, for orchestrating valves,

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i cat ion s o f t he acm 99

review articles

pumps, and the like in automation and Atir, Michael Bar-Sinai, Daniel Barkan, Computer 41, 1 (2008).
17. Harel, D., Katz, G., Marron, A. and Weiss, G. Non-
process control. The ability to pack dis- Dan Brownstein, Nir Eitan, Michal intrusive repair of reactive programs. ICECCS. To
tinct, seemingly unrelated behaviors Gordon, Amir Kantor, Hillel Kugler, appear July 2012.
18. Harel, D., Kleinbort, A. and Maoz, S. S2A: A compiler for
into a single operating entity seems Robby Lampert, Shahar Maoz, Yaarit multi-modal UML sequence diagrams. Fundamental
promising in areas like robotics, self- Natan, Amir Nissim, Yaniv Saar, Avital Approaches to Software Engineering, 2007.
19. Harel, D., Kugler, H., Marelly, R. and Pnueli, A. Smart
guided vehicles, and the modeling of Sadot, Itai Segall, Nir Svirsky, Smadar play-out of behavioral requirements. FMCAD, 2002.
biological systems. The combination Szekely, Moshe Vardi, Moshe Wein- 20. Harel, D., Lampert, R., Marron, A. and Weiss, G. Model-
checking behavioral programs. In EMSOFT, 2011.
of reactivity and rich scripts can be ap- stock, Guy Wiener, and Guy Weiss for 21. Harel, D. and Marelly, R. Come, Let’s Play: Scenario-
plied to information-system manage- their valuable comments and contri- Based Programming Using LSCs and the Play-Engine.
Springer, 2003.
ment including workflow control, event butions throughout the development 22. Harel, D. and Maoz, S. Assert and negate revisited:
Modal semantics for UML sequence diagrams.
processing, root-cause analysis, and of this article and the ideas behind it. Software and System Modeling 7, 2 (2008), 237–252.
automated configuration, among oth- Thanks to Shmuel Katz for insights on 23. Harel, D., Maoz, A., Szekely, S. and Barkan, D. PlayGo:
Towards a comprehensive tool for scenario based
ers. Finally, the ability to trace events positioning behavioral programming programming. ASE, 2010.
in the context of their respective sce- relative to aspect-orientation. We also 24. Harel, D., Marron, A., Nissim, A. and Weiss, G.
Combining behavioral programming and fuzziness
narios, may allow decision-making ap- thank the anonymous reviewers for for hybrid control systems. In Proc. 2012 IEEE
plications to explain their behavior and their valuable suggestions. International Conference on Fuzzy Systems. To appear
June 2012.
facilitate ongoing human validation. Harel’s and Marron’s research was 25. Harel, D., Marron, A. and Weiss, G. Programming
Behavioral programming may also supported in part by the John von Neu- coordinated scenarios in Java. ECOOP, 2010.
26. Harel, D., Marron, A., Weiss, G. and Wiener, G.
accommodate customization as part mann Minerva Center for the Develop- Behavioral programming, decentralized control, and
of the development cycle, where end- ment of Reactive Systems at the Weiz- multiple time scales. AGERE!, 2011.
27. Harel, D. and Naamad, A. The STATEMATE semantics
users can enhance, change, or remove mann Institute of Science, and by an of statecharts. TOSEM 5, 4 (1996).
functionality of delivered systems (for Advanced Research Grant to DH from 28. Harel, D. and Pnueli, A. On the Development of
Reactive Systems, in Logics and Models of Concurrent
example, smartphones), by coding or the European Research Council (ERC) Systems. NATO ASI Series, Vol. F-13. 1985.
downloading new behaviors without under the European Community’s 29. Harel, D. and Segall, I. Planned and traversable play-
out: A flexible method for executing scenario-based
accessing the core product code. FP7 Programme. Weiss’s research was programs. Tools and Algorithms for the Constr. and
Anal. of Systems, 2007.
As a general paradigm, behavioral supported by the Lynn and William 30. Harel, D. and Segall, I. Synthesis from live sequence
programming is still in its infancy. It Frankel Center for Computer Science chart specifications. Computer System Sciences, 78:3
(2012), 970-980.
has been applied to a relatively small at Ben-Gurion University and by a rein- 31. Henzinger, T.A., Kirsch, C.M., Sanvido, M.A.A. and
number of projects, and the existing tegration (IRG) grant under the Euro- Pree, W. From control models to real-time code using
Giotto. IEEE Control Systems Magazine 23, 1 (2003).
tools are not yet of commercial power. pean Community’s FP7 Programme. 32. Katz, S. and Gil, J.Y. Aspects and superimpositions.
More research and development is AOP Workshop at ECOOP, 1999.
33. Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C.,
needed in expanding implementations References Lopes, C., Loingtier, J.M. and Irwin, J. Aspect-oriented
1. Arkin, R.C. Behavior-Based Robotics. MIT Press, 1998.
in a variety of programming contexts 2. Atir, Y. and Harel, D. Using LSCs for scenario authoring
programming. ECOOP, 1997.
34. Kugler, H., Plock, C. and Roberts, A. Synthesizing
and for larger real-world applications. in tactical simulators. In Summer Computer biological theories. In CAV, 2011.
Simulation Conference. Soc. for Comp. Simulation Int.,
We should also experiment with the 2007.
35. Kugler, H. and Segall, I. Compositional synthesis
of reactive systems from live sequence chart
collaboration of multiple development 3. Barak, D., Harel, D. and Marelly, R. Interplay: specifications. Tools and Alg. for the Constr. and Anal.
Horizontal scale-up and transition to design of Systems, 2009.
groups, and expand the work on formal in scenario-based programming. Lectures on 36. LEJOS. Java for LEGO Mindstorms; http://lejos.
verification, synthesis, performance Concurrency and Petri Nets, (2004), 66–86. sourceforge.net/.
4. Berry, G. and Cosserat, L. The Esterel synchronous 37. Lieberman, H. Your Wish is My Command: Programming
and scalability, automated learning programming language and its mathematical by Example. Morgan Kaufmann, 2001.
and adaptability, the use of natural lan- semantics. In Seminar on Concurrency, Springer, 38. Maoz, S. and Harel, D. From multi-modal scenarios to
1985, 389–448. code: Compiling LSCs into AspectJ. In FSE, 2006.
guage, and enhanced play-in. 5. Bliudze, S. and Sifakis, J. A notion of glue expressiveness 39. OSCI. Open SystemC Initiative. IEEE 1666 Language
We feel that the natural incremental for component-based systems. CONCUR, 2008. Reference Manual; http://www.systemc.org.
6. Bordini, R.H., Dastani, M. Dix, J. and Seghrouchni, 40. Prehofer, C. Feature-oriented programming: A fresh
development afforded by behavioral A.E.F. Multi-Agent Programming: Languages, Tools look at objects. ECOOP, 1997.
and Applications. Springer, 2009.
programming could become valuable 7. Bouge, L. and Francez, N. A compositional approach to
41. Sadot, A., Fisher, J., Barak, D. Admanit, Y. Stern, M.J.,
Hubbard, E.J.A and Harel, D. Toward verified biological
for novices and seasoned programmers superimposition. In POPL, 1988. models. IEEE/ACM Trans. Comput. Biology Bioinform
8. Branicky, M.S. Behavioral programming. In Working
alike. We hope the paradigm, with its Notes AAAI Spring Symp. on Hybrid Sys. and AI, 1999.
5, 2 (2008).
42. Shimony, B., Nikolaidis, I., Gburzynski, P. and Stroulia,
current implementations in LSC, Java, 9. Brooks, R. A robust layered control system for a mobile E. On coordination tools in the PicOS tuples system.
robot. IEEE J. of Robotics and Automation 2, 1 (1986).
and other platforms, contributes to the 10. Bunker, A., Gopalakrishnan, G. and Slind, K. Live
SESENA, 2011.
43. Wiener, G., Weiss, G. and Marron, A. Coordinating and
vision of liberating programming,16 and sequence charts applied to hardware requirements visualizing independent behaviors in Erlang. In 9th ACM
specification and verification. Int. J. on Software Tools
that this article will encourage debate for Technology Transfer 7, 4 (2005).
SIGPLAN Erlang Workshop, 2010.

about the ideas, as well as further re- 11. Damm, W. and Harel, D. LSCs: Breathing Life into
Message Sequence Charts. J. on Formal Methods in David Harel ([email protected]) is The William
search and development. System Design 19, 1 (2001). Sussman Professorial Chair in the Department of
12. Eitan, N., Gordon, M., Harel, D., Marron, A. and Weiss, Computer Science and Applied Mathematics at
G. On visualization and comprehension of scenario-
Acknowledgments based programs. ICPC, 2011.
The Weizmann Institute of Science, Rehovot, Israel.

David Harel thanks Werner Damm and 13. Eitan, N. and Harel, D. Adaptive behavioral programming. Assaf Marron ([email protected]) is a
IEEE Int. Conf. on Tools with Artificial Intelligence, 2011. researcher at The Weizman Institute of Science, Rehovot,
Rami Marelly for the wonderful col- 14. Gordon, M., Marron, A., and Meerbaum-Salant, O. Israel.
laborations of 1998–1999 and 2000– Spaghetti for the main course? Observations on
naturalness of scenario-based programming. ITICSE. Gera Weiss ([email protected]) is an assistant professor
2003, respectively, without which the To appear July 2012. in the Department of Computer Science at Ben Gurion
15. Harel, D. From play-in scenarios to code: An achievable University of the Negev, Be’er Sheva, Israel.
ideas described here would not exist. dream. IEEE Computer 34, 1 (2001).
We are grateful to Shai Arogeti, Yoram 16. Harel, D. Can programming be liberated, period? IEEE © 2012 ACM 0001-0782/12/07 $15.00

100 co mm unicatio ns o f t h e ac m | j u ly 201 2 | vo l . 5 5 | no. 7

THIRD ANNUAL ACM
CONFERENCE ON

SYSTEMS,
PROGRAMMING,
LANGUAGES, TUCSON, ARIZONA
APPLICATIONS:
SOFTWARE FOR
OCTOBER 19-26, 2012
HUMANITY

LOCATION
Loews Ventana Canyon Resort
Tucson, Arizona, USA

EVENTS
• OOPSLA
• Onward!
• Dynamic Languages Symposium,
• Wavefront
• Workshops
• ACM Student Research Competition
• Pattern Languages of Programming
• ...and more!

GENERAL CHAIR
Gary T. Leavens
University of Central Florida
[email protected]

OOPSLA PROGRAM CHAIR

Matthew B. Dwyer
University of Nebraska
[email protected]

ONWARD! PROGRAM CHAIRS

Jonathan Edwards/Julia Steele
MIT/O’Reilly
[email protected] ONW
ARD!
[email protected] OO P S L A

DLS PROGRAM CHAIR

Alessandro Warth
Google
[email protected]

FOR MORE INFORMATION

[email protected]
http://splashcon.org/

SPLASH is sponsored by
ACM SIGPLAN
ACM TechNews Goes Mobile
iPhone & iPad Apps Now Available in the iTunes Store
ACM TechNews—ACM’s popular thrice-weekly news briefing service—is now
available as an easy to use mobile apps downloadable from the Apple iTunes Store.
These new apps allow nearly 100,000 ACM members to keep current with
news, trends, and timely information impacting the global IT and Computing
communities each day.

TechNews mobile app users will enjoy:

• Latest News: Concise summaries of the most
relevant news impacting the computing world
• Original Sources: Links to the full-length
articles published in over 3,000 news sources
• Archive access: Access to the complete
archive of TechNews issues dating back to
the first issue published in December 1999
• Article Sharing: The ability to share news
with friends and colleagues via email, text
messaging, and popular social networking sites
• Touch Screen Navigation: Find news
articles quickly and easily with a
streamlined, fingertip scroll bar
• Search: Simple search the entire TechNews
archive by keyword, author, or title
• Save: One-click saving of latest news or archived
summaries in a personal binder for easy access
• Automatic Updates: By entering and saving
your ACM Web Account login information,
the apps will automatically update with
the latest issues of TechNews published
every Monday, Wednesday, and Friday

The Apps are freely available to download from the Apple iTunes Store, but users must be registered
individual members of ACM with valid Web Accounts to receive regularly updated content.
http://www.apple.com/iphone/apps-for-iphone/  http://www.apple.com/ipad/apps-for-ipad/

ACM TechNews
 research highlights
p. 104 p. 105
Technical
Perspective Looking Back and Looking
For Better or Worse, Forward: Power, Performance,
Benchmarks
Shape a Field and Upheaval
By David Patterson By Hadi Esmaeilzadeh, Ting Cao, Xi Yang,
Stephen M. Blackburn, and Kathryn S. McKinley

p. 115 p. 116
Technical
Perspective Intrinsic Robustness
Why Study of the Price of Anarchy
the Price By Tim Roughgarden
of Anarchy?
By Amos Fiat

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | com m u n i c at ion s o f t h e acm 103

research highlights
doi:10.1145/ 2209249.2 2 0 9 2 71

Technical Perspective
For Better or Worse,
Benchmarks Shape a Field
By David Patterson

fields, computer archi-

L i k e oth er I T a victim of its own success. The SPEC Given this measurement frame-
tects initially reported incomparable organization has been selecting old work, the authors then measured eight
results. We quickly saw the folly of programs written in old languages that very different Intel microprocessors
this approach. We then went through reflect the state of programming in the built over a seven-year period. The au-
a sequence of performance metrics, 1980s. Given the 1,000,000X improve- thors evaluate these eight micropro-
with each being an improvement on ment in cost-performance since C++ was cessors using 61 programs, which each
its predecessor: average instruction unveiled in 1979, most programmers fit into one of the four quadrants in the
time, millions of instructions per sec- have moved on to more productive lan- matrix here.
ond (MIPS), millions of floating point guages. Indeed, a recent survey supports This treasure chest of data—record-
operations per second (MEGAFLOPS), that claim: only 25% of programs are be- ed in large tables in the ACM Digital Li-
synthetic program performance ing written in languages like C and C++.c brary in addition to this paper—allows
(DHRYSTONES), and ultimately aver- Hence, the authors supplement SPEC’s the authors (and the rest of us) to ask
age performance improvement rela- C and C++ programs, which manage and answer many questions based on
tive to a reference computer based on a storage manually, with Java programs real hardware. This opportunity is a re-
suite of real programs (SPEC CPU). that manage storage automatically. They freshing change from research results
When a field has good benchmarks, called the former programs native lan- based on simulation, which has domi-
we settle debates and the field makes guages and the latter managed. nated the literature for the last decade.
rapid progress. Indeed, the accelera- Here are four examples of questions
Sequential Parallel
tion in computer performance from (Non-Scalable) (Scalable) we can now address:
25% to 50% per year starting in the mid- ˲˲ Do performance, power, or energy
C and C++
1980s is due in part to our ability to fair- (Native) of the red quadrant programs predict
ly compare competing designs as well Java the results of any of the programs from
(Managed)
as to Moore’s Law. Similarly, computer the other quadrants? (If not, the archi-
vision made dramatic advances in the The paper reflects a second impor- tects must extend their conventional
last decade after it embraced bench- tant trend. The power limit of what a evaluation methods.)
marks to evaluate innovations in vision chip could dissipate forced micropro- ˲˲ Do the native parallel (yellow) pro-
algorithms.a cessor manufacturer to switch from grams predict the managed parallel
Sadly, when a field has bad bench- a single high-clock rate processor per (green) programs? (If not, given the
marks, progress can be problematic. chip to multiple processors or cores per shift in popularity of programming
For example, despite being discredited chips. Thus, the authors include both languages, then many evaluations of
in textbooks since 1990,b embedded sequential and parallel benchmarks; multiprocessors should be redone.)
computing still reports DHRYSTONES they call the former non-scalable and the ˲˲ Is multicore more energy efficient
when making performance claims. latter scalable. Moreover, the authors than multithreading? (If not, then mul-
How do we know whether a new em- report on power and energy in addition ticore designs that do not offer multi-
bedded processor is a genuine break- to performance. In this PostPC Era, bat- threading may be suspect.)
through, or simply the result of cynical tery life can trump performance in the ˲˲ Surely recent in-order execution
benchmarketering, in that it runs the client, and the architects of warehouse- processors are more energy efficient
benchmark quickly but real programs scale computers try to optimize the than modern four-instruction-issue,
slowly? The answer is we cannot know costs of powering and cooling 100,000 out-of-order execution processors? (If
from DHRYSTONE reports. servers as well as improving cost-perfor- not, conventional wisdom on energy
In the following paper, the authors mance. Just as we learned that measur- efficient processor design is incorrect.)
point out that while computer architec- ing time in seconds is a safer measure Hint: The answers are: No, No, No,
ture has a glorious past, it has become of program performance than a rate and No.
a D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A
like MIPS, we are learning that Joules is Readers interested in trying to find
Database of Human Segmented Natural Imag- a better measure than a rate like Watts, the answer to these and computing’s
es and its Application to Evaluating Segmen- which is just Joules/second. The au- other persistent questions just need to
tation Algorithms and Measuring Ecological thors report both Watts and Joules in read the following paper.
Statistics.” Proc. 8th Int’l Conf. Computer Vision, addition to relative performance.
(July 2001), 416–423.
David Patterson is the Pardee Professor of Computer
b J. Hennessy and D. Patterson, Computer Ar- c R.S. King, “The Top 10 Programming Languages.” Science at the University of California at Berkeley.
chitecture: A Quantitative Approach, 1st Edition, (Oct. 2011); http://spectrum.ieee.org/at-work/tech-
Morgan Kauffman, 1990. careers/the-top-10-programming-languages/ © 2012 ACM 0001-0782/12/07 $15.00

104 com municatio ns o f th e ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 doi:10.1145/ 2209249 . 2 2 0 9 2 72

Looking Back and Looking

Forward: Power, Performance,
and Upheaval
By Hadi Esmaeilzadeh, Ting Cao, Xi Yang, Stephen M. Blackburn, and Kathryn S. McKinley

Abstract Society has benefited enormously from exponential

The past 10 years have delivered two significant revolu- ­ ardware performance improvements. Moore observed that
h
tions. (1) Microprocessor design has been transformed by transistors will be smaller and more numerous in each new
the limits of chip power, wire latency, and Dennard scal- generation.15 For a long time, this simple rule of integrated
ing—leading to multicore processors and heterogeneity. (2) circuit fabrication came with an exponential and transpar-
Managed languages and an entirely new software landscape ent performance dividend. Shrinking a transistor lowers
emerged—revolutionizing how software is deployed, is sold, its gate delay, which raises the processor’s theoretical clock
and interacts with hardware. Researchers most often exam- speed (Dennard scaling3). Until recently, shrinking transis-
ine these changes in isolation. Architects mostly grapple tors ­delivered corresponding clock speed increases and more
with microarchitecture design through the narrow software ­transistors in the same chip area. Architects used the transis-
context of native sequential SPEC CPU benchmarks, while tor bounty to add memory, prefetching, branch prediction,
language researchers mostly consider microarchitecture in multiple instruction issue, and deeper pipelines. The result
terms of performance alone. This work explores the clash was exponential single-threaded performance improvements.
of these two revolutions over the past decade by measur- Unfortunately, physical power and wire-delay limits
ing power, performance, energy, and scaling, and considers will derail the clock speed bounty of Moore’s law in cur-
what the results may mean for the future. Our diverse find- rent and future technologies. Power is now a first-order
ings include the following: (a) native sequential workloads hardware design constraint in all market segments.
do not approximate managed workloads or even native Power constraints now severely limit clock scaling and
­parallel workloads; (b) diverse application power profiles prevent using all transistors simultaneously.6, 8, 16 In addi-
suggest that future applications and system software will tion, the physical limitations of wires prevent single cycle
need to participate in power optimization and management; access to a growing number of the transistors on a chip.9
and (c) software and hardware researchers need access to To effectively use more transistors at smaller technologies,
real measurements to optimize for power and energy. these limits forced manufacturers to turn to chip multi-
processors (CMPs) and recently to heterogeneous parallel
systems that seek power efficiency through specializa-
1. INTRODUCTION tion. Parallel heterogeneous hardware requires parallel
Quantitative performance analysis is the foundation for software and exposes software developers to ongoing
computer system design and innovation. In their classic hardware upheaval. Unfortunately, most software today
paper, Emer and Clark noted that “A lack of detailed tim- is not parallel, nor is it designed to modularly decompose
ing information impairs efforts to improve performance.”5 onto a heterogeneous substrate.
They pioneered the quantitative approach by characterizing Moore’s transistor bounty also drove orthogonal and
instruction mix and cycles per instruction on time-sharing disruptive changes in how software is deployed, is sold, and
workloads. They surprised expert reviewers by demonstrating a interacts with hardware over this same decade. Demands for
gap between the theoretical 1 MIPS peak of the VAX-11/780 correctness, complexity management, programmer pro-
and the 0.5 MIPS it delivered on real workloads. Industry and ductivity, time-to-market, reliability, security, and portability
academic researchers in software and hardware all use and pushed developers away from low-level compiled ahead-of-time
extend this principled performance analysis methodology. (native) programming languages. Developers increasingly
Our research applies this quantitative approach to measured
power. This work is timely because the past decade heralded Original version: “Looking Back on the Language and Hardware
the era of power- and energy-constrained hardware design.a Revolutions: Measured Power, Performance, and Scaling,” ACM
Furthermore, demand for energy efficiency has intensified Conference on Architecture Support for Programming
in large-scale systems, in which energy began to dominate Languages and Operating Systems, pp. 319–332, Newport
costs, and in mobile systems, which are limited by battery Beach, CA, 2011. Also published as “What Is Happening to
life. A lack of detailed energy measurements is impairing Power, Performance and Software?,” IEEE Micro Top Picks
efforts to reduce energy consumption on modern workloads. from the Computer Architecture Conferences of 2011,
May/June 2012, IEEE Computer Society.
a
  Energy = power × execution time.

ju ly 2 0 1 2 | vo l . 5 5 | n o. 7 | com m u n i c at ion s o f t h e ac m 105

research highlights

choose high-level managed programming languages with Figure 1. Eight findings from an analysis of measured chip power,
a selection of safe pointer disciplines, garbage collection performance, and energy on 61 workloads and eight processors.
(automatic memory management), extensive standard librar- The ASPLOS paper includes more findings and analysis.
ies, and dynamic just-in-time compilation for hardware porta-
Findings
bility. For example, modern Web services combine managed
languages, such as PHP on the server side and JavaScript on Power consumption is highly application dependent and is poorly correlated
the client side. In markets as diverse as financial software to TDP.
and cell phone applications, Java and .NET are the domi- Power per transistor is relatively consistent within microarchitecture family,
nant choices. The exponential performance improvements independent of process technology.
provided by hardware hid many of the costs of high-level Energy-efficient architecture design is very sensitive to workload.
languages and helped create a virtuous cycle with ever more Configurations in the native non-scalable Pareto Frontier substantially differ
capable and high-level software. This ecosystem is result- from all the other workloads.
ing in an explosion of developers, software, and devices that
Comparing one core to two, enabling a core is not consistently energy efficient.
continue to change how we live and learn.
Unfortunately, a lack of power measurements is impair- The Java Virtual Machine induces parallelism into the execution of single-
ing efforts to reduce energy consumption on traditional and threaded Java benchmarks.
modern software. Simultaneous multithreading delivers substantial energy savings for recent
hardware and for in-order processors.
2. OVERVIEW Two recent die shrinks deliver similar and surprising reductions in energy,
Our work quantitatively examines power, performance, even when controlling for clock frequency.
and scaling during this period of disruptive software and
Controlling for technology, hardware parallelism, and clock speed, the out-of-
hardware changes (2003–2011). Voluminous research
order architectures have similar energy efficiency as the in-order ones.
explores performance analysis and a growing body of work
explores power (see Section 6), but our work is the first to
systematically measure the power, performance, and energy
characteristics of software and hardware across a range of ­ arallel benchmarks and (b) managed runtimes exploit par-
p
processors, technologies, and workloads. allelism even when executing single-threaded applications.
We execute 61 diverse sequential and parallel bench- The results recommend that systems researchers include
marks written in three native languages and one managed managed and native, sequential and parallel workloads
language, all widely used: C, C++, Fortran, and Java. We when designing and evaluating energy-efficient systems.
choose Java because it has mature virtual machine technol- Architecture. Hardware features such as clock scaling,
ogy and substantial open source benchmarks. We choose gross microarchitecture, simultaneous multithreading, and
eight representative Intel IA32 processors from five tech- chip multiprocessors each elicit a huge variety of power,
nology generations (130 nm to 32 nm). Each processor has performance, and energy responses. This variety and the
an isolated processor power supply with stable voltage on difficulty of obtaining power measurements recommend
the motherboard, to which we attach a Hall effect sensor exposing on-chip power meters and, when possible, power
that measures power supply current, and hence processor meters for individual structures, such as cores and caches.
power. We calibrate and validate our sensor data. We find Modern processors include power management techniques
that power consumption varies widely among benchmarks. that monitor power sensors to minimize power usage and
Furthermore, relative performance, power, and energy are boost performance. However, only in 2011 (after our origi-
not well predicted by core count, clock speed, or reported nal paper) did Intel first expose energy counters, in their
Thermal Design Power (TDP). TDP is the nominal amount production Sandy Bridge processors. Just as hardware event
of power the chip is designed to dissipate (i.e., without counters provide a quantitative grounding for performance
exceeding the maximum transistor junction temperature). innovations, future architectures should include power and/or
Using controlled hardware configurations, we explore the energy meters to drive innovation in the power-constrained
energy impact of hardware features and workload. We per- computer systems era.
form historical and Pareto analyses that identify the most Measurement is key to understanding and optimization.
power- and performance-efficient designs in our architecture
configuration space. We make all of our data publicly avail- 3. METHODOLOGY
able in the ACM Digital Library as a companion to our origi- This section presents an overview of essential elements of
nal ASPLOS 2011 paper. Our data quantifies a large number our methodology. We refer the reader to the original paper
of workload and hardware trends with precision and depth, for a more detailed treatment.
some known and many previously unreported. This paper
highlights eight findings, which we list in Figure 1. Two 3.1. Software
themes emerge from our analysis: workload and architecture. We systematically explore workload selection and show that it
Workload. The power, performance, and energy trends of is a critical component for analyzing power and performance.
native workloads substantially differ from managed and par- Native and managed applications embody different trade-offs
allel native workloads. For example, (a) the SPEC CPU2006 between performance, reliability, portability, and deployment.
native benchmarks draw significantly less power than It is impossible to meaningfully separate language from

106 comm unicatio ns o f t h e ac m | j u ly 201 2 | vol. 5 5 | no. 7

workload and we offer no commentary on the virtue of lan- To explore the influence of architectural features, we selec-
guage choice. We create four workloads from 61 benchmarks. tively down-clock the processors, disable cores on these chip
multiprocessors (CMP), disable simultaneous multithread-
Native non-scalable: C, C++, and Fortran single-threaded
ing (SMT), and disable Turbo Boost using BIOS configuration.
compute-intensive benchmarks from SPEC CPU2006.
Native scalable: Multithreaded C and C++ benchmarks from 3.3. Power, performance, and energy measurement
PARSEC. We isolate the direct current (DC) power supply to the pro-
Java non-scalable: Single and multithreaded benchmarks cessor on the motherboard and measure its current with
that do not scale well from SPECjvm, DaCapo 06-10-MR2, Pololu’s ACS714 current sensor board. The supply voltage
DaCapo 9.12, and pjbb2005. is very stable, varying by less than 1%, which enables us to
Java scalable: Multithreaded Java benchmarks from DaCapo correctly calculate power. Prior work used a clamp amme-
9.12 that scale in performance similarly to native scalable ter, which can only measure alternating current (AC), and is
on the i7 (45). therefore limited to measuring the whole system.10, 12 After
publishing the original paper, Intel made chip-level and
We execute the Java benchmarks on the Oracle HotSpot 1.6.0 core-level energy measurements available on Sandy Bridge
virtual machine because it is a mature high-performance processors.4 Our methodology should slightly overstate chip
­virtual machine. The virtual machine dynamically optimizes power because it includes losses due to the motherboard’s
each benchmark on each architecture. We use best practices for voltage regulator. Validating against the Sandy Bridge energy
virtual machine measurement of steady state performance.2 counter shows that our power measurements consistently
We compile the native non-scalable workload with icc at –o3. measure about 5% more current.
We use gcc at –o3 for the native scalable workload because We execute each benchmark multiple times on every
icc did not correctly compile all benchmarks. The icc compiler architecture, log its power values, and then compute average
generates better performing code than gcc. We execute the power consumption. The aggregate 95% confidence inter-
same native binaries on all machines. All the parallel native vals of execution time and power range from 0.7% to 4%. The
benchmarks scale up to eight hardware ­contexts. The Java scal- measurement error in time and power for all processors and
able workload is the subset of Java benchmarks that scale well. benchmarks is low. We compute arithmetic means over the
four workloads, weighting each workload equally. To avoid
3.2. Hardware biasing performance measurements to any one architec-
Table 1 lists our eight Intel IA32 processors which cover four ture, we compute a reference performance for each bench-
process technologies (130 nm, 65 nm, 45 nm, and 32 nm) mark by averaging the execution time on four architectures:
and four microarchitectures (NetBurst, Core, Bonnell, and Pentium 4 (130), Core 2D (65), Atom (45), and i5 (32). These
Nehalem). The release price and date give context regarding choices capture four microarchitectures and four technol-
Intel’s market placement. The Atoms and the Core 2Q (65) ogy generations. We also normalize energy to a reference,
Kentsfield are extreme market points. These processors are since energy = power × time. The reference energy is the aver-
only examples of many processors in each family. For example, age benchmark power on the four processors multiplied by
Intel sells over 60 Nehalems at 45 nm, ranging in price from their average execution time.
around $190 to over $3700. We believe that these samples are We measure the 45 processor configurations (8 stock
representative because they were sold at similar price points. and 37 BIOS configurations) and produce power and

Table 1. Specifications for the eight processors used in the experiments.

VID
Release Price CMP Clock Trans Die Range TDP FSB B/W DRAM
Processor mArch Processor sSpec date (USD) SMT LLC (B) (GHz) nm M (mm2) (V) (W) (MHz) (GB/s) Model
Pentium 4 NetBurst Northwood SL6WF May ‘03 – 1C2T 512K 2.4 130 55 131 – 66 800 – DDR-
400
Core 2 Duo Core Conroe SL9S8 Jul ‘06 316 2C1T 4M 2.4 65 291 143 0.85– 65 1066 – DDR2-
E6600 1.50 800
Core 2 Quad Core Kentsfield SL9UM Jan ‘07 851 4C1T 8M 2.4 65 582 286 0.85– 105 1066 – DDR2-
Q6600 1.50 800
Core i7 Nehalem Bloomfield SLBCH Nov ‘08 284 4C2T 8M 2.7 45 731 263 0.80– 130 – 25.6 DDR3-
920 1.38 1066
Atom Bonnell Diamondville SLB6Z Jun ‘08 29 1C2T 512K 1.7 45 47 26 0.90– 4 533 – DDR2-
230 1.16 800
Core 2 Duo Core Wolfdale SLGTD May ‘09 133 2C1T 3M 3.1 45 228 82 0.85– 65 1066 – DDR2-
E7600 1.36 800
Atom Bonnell Pineview SLBLA Dec ‘09 63 2C2T 1M 1.7 45 176 87 0.80– 13 665 – DDR2-
D510 1.17 800
Core i5 Nehalem Clarkdale SLBLT Jan ‘10 284 2C2T 4M 3.4 32 382 81 0.65– 73 – 21.0 DDR3-
670 1.40 1333

ju ly 2 0 1 2 | vo l . 5 5 | n o. 7 | com m u n i c at ion s o f t h e ac m 107

research highlights

performance data for each benchmark and processor. variation between maximum and minimum is on the Atom
Figure 2 shows an example of this data, plotting the power (45) at 30%. This trend is not new. All the processors exhibit
versus performance characteristics for one of the 45 proces- a range of benchmark-specific power variation. TDP loosely
sor configurations, the stock i7 (45). correlates with power consumption, but it does not provide a
good estimate for (1) maximum power consumption of indi-
4. PERSPECTIVE vidual processors, (2) comparing among processors, or (3)
We organize our analysis into eight findings, as summarized approximating benchmark-specific power consumption.
in Figure 1. The original paper contains additional analyses
and findings. We begin with broad trends. We show that Finding: Power consumption is highly application dependent
applications exhibit a large range of power and performance and is poorly correlated to TDP.
characteristics that are not well summarized by a single
number. This section conducts a Pareto energy efficiency Figure 2 plots power versus relative performance for each
analysis for all of the 45 nm processor configurations. Even benchmark on the i7 (45), which has eight hardware contexts
with this modest exploration of architectural features, the and is the most recent of the 45 nm processors. Native (red) and
results indicate that each workload prefers a different hard- managed (green) are differentiated by color, whereas scalable
ware configuration for energy efficiency. (triangle) and non-scalable (­ circle) are differentiated by shape.
Unsurprisingly, the scalable benchmarks (triangles) tend to
4.1. Power is application dependent perform the best and consume the most power. More unex-
The nominal thermal design power (TDP) for a processor is pected is the range of power and performance characteristics of
the amount of power the chip may dissipate without exceed- the non-scalable benchmarks. Power is not strongly correlated
ing the maximum transistor junction temperature. Table 1 with performance across workload or benchmarks. The points
lists TDP for each processor. Because measuring real proces- would form a straight line if the correlation were strong. For
sor power is difficult and TDP is readily available, TDP is often example, the point on the bottom right of the figure achieves
substituted for real measured power. Figure 3 shows that this almost the best relative performance and lowest power.
substitution is problematic. It plots measured power on a log-
arithmic scale for each benchmark on each stock processor 4.2. Historical overview
as a function of TDP and indicates TDP with an “✗.” TDP is Figure 4(a) plots the average power and performance for
strictly higher than actual power. The gap between peak mea- each processor in their stock configuration relative to the
sured power and TDP varies from processor to processor and reference ­performance, using a log/log scale. For example,
TDP is up to a factor of four higher than measured power. The the i7 (45) points are the average of the workloads derived
variation among benchmarks is highest on the i7 (45) and i5 from the points in Figure 2. Both graphs use the same color
(32), likely reflecting their advanced power management. For for all of the experimental processors in the same family.
example, on the i7 (45), measured power varies between 23 W The shapes encode release age: a square is the oldest, the
for 471.omnetpp and 89 W for fluidanimate! The smallest diamond is next, and the triangle is the youngest, smallest
technology in the family.
Figure 2. Power/performance distribution on the i7 (45). Each point
represents one of the 61 benchmarks. Power consumption is highly Figure 3. Measured power for each processor running 61 benchmarks.
variable among the benchmarks, spanning from 23 W to 89 W. The Each point represents measured power for one benchmark. The
wide spectrum of power responses from different applications “✗”s are the reported TDP for each processor. Power is application
points to power saving opportunities in software. dependent and does not strongly correlate with TDP.

100
90 100
Measured power (W) (log)

80
70
Power (W)

60
50 10

40
30
20
2.00 3.00 4.00 5.00 6.00 7.00 8.00 1
1 10 100
Performance/Reference TDP (W) (log)
Native non-scale Native scale P4 (130) C2D (65) C2Q (65) i7 (45)
Java non-scale Java scale Atom (45) C2D (45) AtomD (45) i5 (32)

108 co m municatio ns of t h e ac m | j u ly 201 2 | vol . 5 5 | no. 7

Figure 4. Power/performance trade-off by processor. Each point is
find that the i5 (32) delivers 50% better performance, while
an average of the four workloads. (a) Power/performance trade-offs consuming around 25% more power than the Core 2D (45).
have changed from Pentium 4 (130) to i5 (32). (b) Power and Contemporaneous comparisons also reveal the ten-
performance per million transistors. Power per million transistors sion between power and performance. For example, the
is consistent across different microarchitectures regardless of the contrast between the Core 2D (45) and i7 (45) shows that
technology node. On average, Intel processors burn around 1 W for
every 20 million transistors.
the i7 (45) delivers 75% more performance than the Core
2D (45), but this performance is very costly in power, with
Pentium4 (130) an increase of nearly 100%. These processors thus span a
C2D (65) wide range of energy trade-offs within and across the gen-
Power (W) (log)

C2Q (65) erations. Overall, these results indicate that optimizing for
20.0
i7 (45) both power and performance is proving a lot more chal-
Atom (45) lenging than optimizing for performance alone.
C2D (45) Figure 4(b) explores the effect of transistors on power and
performance by dividing them by the number of transistors
AtomD (45)
in the package for each processor. We include all transistors
2.0 i5 (32)
because our power measurements occur at the level of the
0.30 3.00 package, not the die. This measure is rough and will downplay
Performance/Reference performance (log) results for the i5 (32) and Atom D (45), each of which have a
(a) Graphics Processing Unit (GPU) in their package. Even though
the benchmarks do not exercise the GPUs, we cannot discount
Power (W)/106 transistors (log)

Pentium4 (130) them because the GPU transistor counts on the Atom D (45)
C2D (65) are undocumented. Note the similarity between the Atom (45),
C2Q (65) AtomD (45), Core 2D (45), and i5 (32), which at the bottom right
0.22 of the graph are the most efficient processors by the transistor
i7 (45)
Atom (45) metric. Even though the i5 (32) and Core 2D (45) have five to
eight times more transistors than the Atom (45), they all eke
C2D (45)
out very similar performance and power per transistor. There
AtomD (45)
are likely bigger differences to be found in power efficiency per
i5 (32)
0.02 transistor between chips from different manufacturers.
0.004 0.008 0.012 0.016
6
Performance/10 transistors (log) Finding: Power per transistor is relatively consistent within micro-
architecture family, independent of process technology.
(b)

The leftmost processors in the graph yield the smallest

While historically mobile devices have been extensively amount of performance per transistor. Among these proces-
optimized for power, general-purpose processor design sors, the Core 2Q (65) and i7 (45) yield the least performance
has not. Several results stand out that illustrate that power per transistor and use the largest caches among our set. The
is now a first-order design goal and trumps performance in large 8MB caches are not effective. The Pentium 4 (130) is per-
some cases. (1) The Atom (45) and Atom D (45) are designed haps most remarkable—it yields the most performance per
as low-power processors for a different market; however, transistor and consumes the most power per transistor by a
they successfully execute all these benchmarks and are the considerable margin. In summary, performance per transistor
most power-efficient processors. Compared to the Pentium 4 is inconsistent across microarchitectures, but power per tran-
(130), they degrade performance modestly and reduce power sistor is more consistent. Power per transistor correlates well
enormously, consuming as little as one-twentieth the power. with microarchitecture, regardless of technology generation.
Device scaling from 130 nm to 45 nm contributes ­significantly
to the power reduction from Pentium to Atom. (2) A compari- 4.3. Pareto analysis at 45 nm
son between 65 nm and 45 nm generations using the Core 2D The Pareto optimal frontier defines a set of choices that are
(65) and Core 2D (45) shows only a 25% increase in perfor- most efficient in a trade-off space. Prior research uses the
mance, but a 35% drop in power. (3) A comparison of the two Pareto frontier to explore power versus performance using
most recent 45 nm and 32 nm generations using the i7 (45) and models to derive potential architectural designs on the fron-
i5 (32) shows that the i5 (32) delivers about 15% less perfor- tier.1 We present a Pareto frontier derived from measured
mance, while consuming about 40% less power. This result performance and power. We hold the process technology con-
has three root causes: (a) the i7 (45) has four cores instead stant by using the four 45 nm processors: Atom (45), Atom
of two on the i5 (32); (b) since half the benchmarks are scal- D (45), Core 2D (45), and i7 (45). We expand the number of
able multithreaded benchmarks, the software parallelism processor configurations from 4 to 29 by configuring the
benefits more from the additional two cores, increasing the number of hardware contexts (SMT and CMP), by clock scal-
advantage to the i7 (45); and (c) the i7 (45) has significantly ing, and disabling/enabling Turbo Boost. The 25 non-stock
better memory performance. Comparing the Core 2D (45) to configurations represent alternative design points. For each
the i5 (32) where the number of processors are matched, we configuration, we compute the averages for each workload

ju ly 2 0 1 2 | vo l . 5 5 | n o. 7 | c om m u n i c at ion s of t h e acm 109

research highlights

and their average to produce an energy/performance scat- points, the points fall in different places on the curves
ter plot (not shown here). We next pick off the frontier—the because each workload exhibits a different energy/perfor-
points that are not dominated in performance or energy effi- mance trade-off. Compare the scalable and non-scalable
ciency by any other point—and fit them with a polynomial benchmarks at 0.40 normalized energy on the y-axis. It is
curve. Figure 5 plots these polynomial curves for each work- impressive how well these architectures effectively exploit
load and the average. The rightmost curve delivers the best software parallelism, pushing the curves to the right and
performance for the least energy. increasing performance from about 3 to 7 while holding
Each row of Figure 6 corresponds to one of the five curves energy constant. This measured behavior confirms prior
in Figure 5. The check marks identify the Pareto-efficient model-based observations about the role of software parallel-
configurations that define the bounding curve and include ism in extending the energy/performance curve to the right.1
15 of 29 configurations. Somewhat surprising is that none of
the Atom D (45) configurations are Pareto efficient. Notice Finding: Energy-efficient architecture design is very sensitive to
the following: (1) Native non-scalable shares only one choice workload. Configurations in the native non-scalable
with any other workload. (2) Java scalable and the average Pareto frontier differ substantially from all other
share all the same choices. (3) Only two of eleven choices workloads.
for Java non-scalable and Java scalable are common to both.
(4) Native non-scalable does not include the Atom (45) in In summary, architects should use a variety of workloads,
its frontier. This last finding contradicts prior simulation and in particular, should avoid only using native non-scalable
work, which concluded that dual-issue in-order cores and workloads.
dual-issue out-of-order cores are Pareto optimal for native
non-scalable.1 Instead, we find that all of the Pareto-efficient 5. FEATURE ANALYSIS
points for native non-scalable in this design space are Our original paper evaluates the energy effect of a range of
quad-issue out-of-order i7 (45) configurations. hardware features: clock frequency, die shrink, memory
Figure 5 starkly shows that each workload deviates sub- hierarchy, hardware parallelism, and gross microarchitec-
stantially from the average. Even when the workloads share ture. This analysis resulted in a large number of findings
and insights. Reader and reviewer feedback yielded a diver-
Figure 5. Energy/performance Pareto frontiers (45 nm). The energy/ sity of opinions as to which findings were most surprising
performance optimal designs are application dependent and and interesting. This section presents results exploring chip
significantly deviate from the average case.
multiprocessing (CMP), simultaneous multithreading (SMT),
technology scaling with a die shrink, and gross microarchi-
0.60
tecture, to give a flavor of our analysis.
Normalized workload energy

0.55
0.50
0.45
5.1. Chip multiprocessors
0.40
Average Figure 7 shows the average power, performance, and energy
0.35 Native non-scale effects of chip multiprocessors (CMPs) by comparing one
0.30 Native scale core to two cores for the two most recent processors in our
0.25 Java non-scale study. We disable Turbo Boost in these analyses because
0.20 Java scale it adjusts power dynamically based on the number of idle
0.15 cores. We disable Simultaneous Multithreading (SMT)
0.00 2.00 4.00 6.00
to maximally expose thread-level parallelism to the CMP
Workload performance/Workload reference performance
hardware feature. Figure 7(a) compares relative power, per-
formance, and energy as a weighted average of the work-
loads. Figure 7(b) shows a break down of the energy as a
Figure 6. Pareto-efficient processor configurations for each function of workload. While average energy is reduced by
workload. Stock configurations are bold. Each “✔” indicates that the 9% when adding a core to the i5 (32), it is increased by 12%
configuration is on the energy/performance Pareto-optimal curve.
Native non-scalable has almost no overlap with any other workload.
when adding a core to the i7 (45). Figure 7(a) shows that the
source of this difference is that the i7 (45) experiences twice
the power overhead for enabling a core as the i5 (32), while
i7( )1C2 @2. Hz N GHz

Hz B
i7( 4C @1. Hz oTB
i7( )1C2 @1. Hz TB
z

.7G NoT
i7( )1C1 5)2C @1.6 z
T@ 1T@ GH
H

producing roughly the same ­performance improvement.

o
T 7G 3.1

i7( )4C2 @2. Hz N

i7( 2D( )2C1 1.7G

i7( )4C @2. Hz

@2 Hz
i7( )2C2 @1. Hz

2T 2.7G z
i7( 4C1 @2. Hz
i7( )2C1 @2. Hz

i7( )4C @1. Hz

H
45 2T 6G
T 4G

45 2T@ 1G
T 7G
T 7G
45 1T 6G
T 6G

T 6G
@

G
T

Finding: Comparing one core to two, enabling a core is not

Co D(4 2T

i7( 1C1 2.
re2 )1C

consistently energy efficient.

T
5
45 4
Co (45

)4C
)

)
m

re

45
45
45
45
45
45

45
45
Ato

Figure 7(b) shows that native non-scalable and Java non-

Native non-scalable     scalable suffer the most energy overhead with the addition
Native scalable      
Java non-scalable        of another core on the i7 (45). As expected, performance
Java scalable      for native non-scalable is unaffected. However, turning on
Average      an additional core for native non-scalable leads to a power
increase of 4% and 14%, respectively, for the i5 (32) and

110 co m munic atio ns of t h e ac m | j u ly 201 2 | vol . 5 5 | no. 7

Figure 7. CMP: Comparing two cores to one core. (a) Impact of Figure 8. Scalability of single-threaded Java benchmarks.
doubling the number of cores on performance, power, and energy, Counterintuitively, some single-threaded Java benchmarks scale
averaged over all four workloads. (b) Energy impact of doubling well. This is because the underlying JVM exploits parallelism for
the number of cores for each workload. Doubling the cores is not compilation, profiling, and garbage collection.
consistently energy efficient among processors or workloads.
1.60
1.60
1.50 1.50
1.40 1.40

2 Cores/1 Core
2 Cores/1 Core

1.30
1.20 1.30
1.10 1.20
1.00
0.90 1.10
0.80 1.00
0.70
0.60 0.90
Performance Power Energy

tlr

ck

db

ss

io

c
s
oa
de

va
fo

es

ud
je
ja
an

bl

ja
in

pr

ga
i7 (45) i5 (32)

lu

pe
co

m
(a)
1.20
1.10 Figure 9. SMT: one core with and without SMT. (a) Impact of enabling
2 Cores/1 Core

1.00 two-way SMT on a single-core with respect to performance, power, and

energy, averaged over all four workloads. (b) Energy impact of enabling
0.90
two-way SMT on a single core for each workload. Enabling SMT delivers
0.80 significant energy savings on the recent i5 (32) and the in-order Atom (45).
0.70 1.60
0.60 1.50
1.40
2 Threads/1 Thread

Native Native Java Java

non-scale scale non-scale scale 1.30
1.20
i7 (45) i5 (32)
1.10
(b) 1.00
0.90
0.80
i7 (45), translating to energy overheads. 0.70
0.60
More interesting is that Java non-scalable does not incur Performance Power Energy
energy overhead when enabling another core on the i5 (32).
Pentium 4 (130) i7 (45) Atom (45) i5 (32)
In fact, we were surprised to find that the reason for this is
that the single-threaded Java non-scalable workload runs (a)
faster with two processors! Figure 8 shows the scalability 1.20
of the single-threaded subset of Java non-scalable on the 1.10
2 Threads/1 Thread

i7 (45), with SMT disabled, comparing one and two cores.

1.00
Although these Java benchmarks are single-threaded, the
JVMs on which they execute are not. 0.90
0.80
Finding: The JVM induces parallelism into the execution of sin- 0.70
gle-threaded Java benchmarks. 0.60
Native Native Java Java
Since virtual machine runtime services for managed lan- non-scale scale non-scale scale
guages, such as just-in-time (JIT) compilation, profiling, and Pentium 4 (130) i7 (45) Atom (45) i5 (32)
garbage collection, are often concurrent and parallel, they (b)
provide substantial scope for parallelization, even within
ostensibly sequential applications. We instrumented the
HotSpot JVM and found that its JIT compilation and gar- and i7 (45). Each processor supports two-way SMT. SMT pro-
bage collection are parallel. Detailed performance counter vides fine-grain parallelism to distinct threads in the proces-
measurements revealed that the garbage collector induced sors’ issue logic and in modern implementations; threads
memory system improvements with more cores by reducing share all processor components (e.g., execution units and
the collector’s d
­ isplacement effect on the application thread. caches). Singhal states that the small amount of logic exclu-
sive to SMT consumes very little power.18 Nonetheless, this
5.2. Simultaneous multithreading logic is integrated, so SMT contributes a small amount to
Figure 9 shows the effect of disabling simultaneous multi- total power even when disabled. Our results therefore slightly
threading (SMT)19 on the Pentium 4 (130), Atom (45), i5 (32), underestimate the power cost of SMT. We use only one core,

ju ly 2 0 1 2 | vo l . 5 5 | n o. 7 | com m u n i c at ion s o f t h e acm 111

research highlights

ensuring that SMT is the sole opportunity for thread-level Figure 10. Die shrink: microarchitectures compared across technology
parallelism. Figure 9(a) shows that the performance advan- nodes. “Core” shows Core 2D (65)/Core 2D (45) while “Nehalem”
tage of SMT is significant. Notably, on the i5 (32) and Atom shows i7 (45)/i5 (32) when two cores are enabled. (a) Each processor
(45), SMT improves average performance significantly with- uses its native clock speed. (b) Clock speeds are matched in each
comparison. (c) Energy impact with matched clocks, as a function of
out much cost in power, leading to net energy savings.
workload. Both die shrinks deliver substantial energy reductions.

Finding: SMT delivers substantial energy savings for recent 1.20

Core Nehalem 2C2T
hardware and for in-order processors. 1.00
0.80

New/Old
Given that SMT was and continues to be motivated by the
0.60
challenge of filling issue slots and hiding latency in wide
issue superscalars, it may appear counterintuitive that 0.40
performance on the dual-issue in-order Atom (45) should 0.20
benefit so much more from SMT than the quad-issue i7
0.00
(45) and i5 (32) benefit. One explanation is that the in-order Performance Power Energy
pipelined Atom (45) is more restricted in its capacity to fill
(a)
issue slots. Compared to other processors in this study, the
Atom (45) has much smaller caches. These features accen- 1.20
tuate the need to hide latency, and therefore the value of Core 2.4 GHz Nehalem 2C2T 2.6 GHz
1.00
SMT. The performance improvements on the Pentium 4
(130) due to SMT are half to one-third that of more recent New/Old 0.80
processors, and consequently, there is no net energy advan- 0.60
tage. This result is not so surprising given that the Pentium
0.40
4 (130) is the first commercial implementation of SMT.
Figure 9(b) shows that, as expected, the native non-scal- 0.20
able workload experiences very little energy overhead due 0.00
to enabling SMT, whereas Figure 7(b) shows that enabling a Performance Power Energy
core incurs a significant power and thus energy penalty. The (b)
scalable workloads unsurprisingly benefit most from SMT.
The excellent energy efficiency of SMT is impressive on 0.65
Core 2.4 GHz Nehalem 2C2T 2.6 GHz
recent processors as compared to CMP, particularly given 0.60
its very low die footprint. Compare Figures 7 and 9. SMT 0.55
New/Old

provides less performance improvement than CMP—SMT 0.50

adds about half as much performance as CMP on average 0.45
but incurs much less power cost. The results on the modern 0.40
processors show that SMT in a much more favorable light 0.35
than in Sasanka et al.’s model-based comparative study of 0.30
the energy efficiency of SMT and CMP.17 Native Native Java Java
non-scale scale non-scale scale
5.3. Die shrink (c)
We use processor pairs from the Core (Core 2D (65)/Core
2D (45) ) and Nehalem (i7 (45)/i5 (32) ) microarchitectures to
explore die shrink effects. These hardware comparisons are Figure 10(a) shows the power and performance effects of
imperfect because they are not straightforward die shrinks. the die shrinks with the stock clock speeds for all the pro-
To limit the differences, we control for hardware parallelism cessors. Figure 10(b) shows the same comparison with
by limiting the i7 (45) to two cores. The tools and processors matched clock speeds, and Figure 10(c) breaks down the
at our disposal do not let us control the cache size, nor do workloads for the matched clock speeds. The newer proces-
they let us control for other microarchitecture changes that sors are significantly faster at their higher stock clock speeds
accompany a die shrink. We compare at stock clock speeds and significantly more power efficient. Figure 10(b) shows
and control for clock speed by running both Cores at 2.4 GHz the same experiment, but down-clocking the newer pro-
and both Nehalems at 2.66 GHz. We do not directly control cessors to match the frequency of their older peers. Down-
for core voltage, which differs across technology nodes clocking the new processors improves their relative power
for the same frequency. Although imperfect, these are the and energy advantage even further. Note that as expected,
first published comparisons of measured energy efficiency the die-shrunk processors offer no performance advantage
across technology nodes. once the clocks are matched; indeed, the i5 (32) performs
10% slower than the i7 (45). However, power consumption
Finding: Two recent die shrinks deliver similar and surprising is reduced by 47%. This result is consistent with expecta-
reductions in energy, even when controlling for clock tions, given the lower voltage and reduced capacitance at the
frequency. smaller feature size.

112 c o mm unicatio ns o f th e acm | j u ly 201 2 | vol . 5 5 | no. 7

 Figures 10(a) and (b) reveal a striking similarity in power yields a small 14% performance improvement. This finding is
and energy savings between the Core (65 nm/45 nm) and not inconsistent with Nehalem’s stated primary design goals,
Nehalem (45 nm/32 nm) die shrinks. This data suggests that that is, delivering scalability and memory performance.
Intel maintained the same rate of energy reduction across the
two most recent generations. As a point of comparison, the Finding: Controlling for technology, hardware parallelism,
models used by the International Technology Roadmap for and clock speed, the out-of-order architectures have
Semiconductors (ITRS) predicted a 9% increase in frequency similar energy efficiency as the in-order ones.
and a 34% reduction in power from 45 nm to 32 nm.11 Figure
10(a) is both more and less encouraging. Clock speed increased The comparisons between the i7 (45) and Atom D (45) and
by 26% in the stock configurations of the i7 (45) to the i5 (32) Core 2D (45) hold process technology constant at 45 nm. All
with an accompanying 14% increase in performance, but power three processors are remarkably similar in energy consump-
reduced by 23%, less than the 34% predicted. To more deeply tion. This outcome is all the more interesting because the i7
understand die shrink efficiency on modern processors, one (45) is disadvantaged since it uses fewer hardware contexts
requires measuring more processors in each technology node. here than in its stock configuration. Furthermore, the i7 (45)
integrates more services on-die, such as the memory control-
5.4. Gross microarchitecture change ler, that are off-die on the other processors, and thus outside
This section explores the power and performance effect of the scope of the power meters. The i7 (45) improves upon
gross microarchitectural change by comparing microarchitec- the Core 2D (45) and Atom D (45) with a more scalable, much
tures while matching features such as processor clock, degree higher bandwidth on-chip interconnect, which is not exer-
of hardware parallelism, process technology, and cache size. cised heavily by our workloads. It is impressive that, despite
Figure 11 compares the Nehalem i7 (45) with the NetBurst all of these factors, the i7 (45) delivers similar energy effi-
Pentium 4 (130), Bonnell Atom D (45), and Core 2D (45) micro- ciency to its two 45 nm peers, particularly when compared to
architectures, and it compares the Nehalem i5 (32) with the the low-power in-order Atom D (45). It is unsurprising that
Core 2D (65). Each comparison configures the Nehalems to the i7 (45) performs 2.6× faster than the Pentium 4 (130),
match the clock speed, number of cores, and hardware threads while consuming one-third the power, when controlling for
of the other architecture. Both the i7 (45) and i5 (32) compari- clock speed and hardware parallelism (but not for factors
sons to the Core show that the move from Core to Nehalem such as memory speed). Much of the 50% power improve-
ment is attributable to process technology advances. This
Figure 11. Gross microarchitecture: a comparison of Nehalem with speedup of 2.6 over 7 years is however substantially less than
four other microarchitectures. In each comparison, the Nehalem the historical factor of 8 improvement experienced in every
is configured to match the other processor as closely as possible.
prior 7-year time interval between 1970 through the early
(a) Impact of microarchitecture change with respect to performance,
power, and energy, averaged over all four workloads. (b) Energy 2000s. This difference in improvements marks the begin-
impact of microarchitecture for each workload. The most recent ning of the power-constrained architecture design era.
microarchitecture, Nehalem, is more energy efficient than the
others, including the low-power Bonnell (Atom). 6. RELATED WORK
The processor design literature is full of performance mea-
Bonnell: i7 (45)/AtomD (45) NetBurst: i7 (45)/Pentium4 (130) surement and analysis. Despite power’s growing impor-
Core: i7 (45)/C2D (45) Core: i5 (32)/C2D (65) tance, power measurements are still relatively rare.7,10,12 Here,
3.00 we summarize related power measurement and s­ imulation
2.50 work. Our original paper contains a fuller treatment.
Nehalem/Other

2.00
Power measurement. Isci and Martonosi combine a clamp
ammeter with performance counters for per unit power esti-
1.50
mation of the Intel Pentium 4 on SPEC CPU2000.10 Fan et al.
1.00 estimate whole system power for large-scale data centers.7
0.50 They find that even the most power-consuming workloads
0.00 draw less than 60% of peak possible power consumption.
Performance Power Energy We measure chip power and support their results by show-
(a) ing that TDP does not predict measured chip power. Our
1.20 work is the first to compare microarchitectures, technology
1.00
generations, individual benchmarks, and workloads in the
Nehalem/Other

context of power and performance.

0.80
Power modeling. Power modeling is necessary to thor-
0.60
oughly explore architecture design.1, 13, 14 Measurement
0.40
complements simulation by providing validation. For exam-
0.20
ple, some prior simulators used TDP, but our measure-
0.00
Native Native Java Java
ments show that it is not accurate. As we look to the future,
non-scale scale non-scale scale we believe that programmers will need to tune their appli-
(b) cations for power and energy, not only performance. Just
as hardware performance counters provide insight into

ju ly 2 0 1 2 | vo l . 5 5 | n o. 7 | c om m u n ic at ion s o f t h e ac m 113
research highlights

applications, so will power and energy measurements. 2. Blackburn, S.M. et al. Wake up Methodology and empirical data. In
Methodology. Although the results show conclusively that and smell the coffee: Evaluation MICRO (2003).
methodologies for the 21st century. 11. ITRS Working Group. International tech­
managed and native workloads have different responses CACM 51, 8 (2008), 83–89. nology roadmap for semiconductors, 2011.
to architectural variations, perhaps this result should not 3. Bohr, M. A 30 year retrospective on 12. Le Sueur, E., Heiser, G. Dynamic voltage
Dennard’s MOSFET scaling paper. IEEE and frequency scaling: the laws of
be surprising. Unfortunately, few architecture or operating SSCS Newsletter 12, 1 (2007), 11–13 diminishing returns. In HotPower (2010).
(http://dx.doi.org/10.1109/N-SSC.2007. 13. Li, S., Ahn, J.H., Strong, R.D., Brockman,
system publications with processor measurements or sim- 4785534). J.B., Tullsen, D.M., Jouppi, N.P. McPAT:
ulated designs use Java or any other managed workloads, 4. David, H., Gorbatov, E., Hanebutte, U.R., an integrated power, area, and timing
Khanna, R., Le, C. RAPL: memory power modeling framework for multicore and
even though the evaluation methodologies we use here for estimation and capping. In ISLPED manycore architectures. In MICRO (2009).
real processors and those for simulators are well developed.2 (2010). 14. Li, Y., Lee, B., Brooks, D., Hu, Z., Skadron, K.
5. Emer, J.S., Clark, D.W. A characterization CMP design space exploration subject to
of processor performance in the VAX- physical contraints. In HPCA (2006).
7. CONCLUSION 11/780. In ISCA (1984). 15.  Moore, G.E. Cramming more components
6. Esmaeilzadeh, H., Blem, E., St. Amant, onto integrated circuits. Electronics 38, 8
These extensive experiments and analyses yield a wide R., Sankaralingam, K., Burger, D. Dark (19 Apr 1965), 114–117.
range of findings. On this basis, we offer the following rec- silicon and the end of multicore scaling. 16.  Mudge, T. Power: a first-class
In ISCA (2011). architectural design constraint.
ommendations in this critical time period of hardware and 7.  Fan, X., Weber, W.D., Barroso, L.A. Power Computer 34, 4 (Apr. 2001), 52–58.
software upheaval. Manufacturers should expose on-chip provisioning for a warehouse-sized 17. Sasanka, R., Adve, S.V., Chen, Y.K., Debes,
computer. In ISCA (2007). E. The energy efficiency of CMP vs.
power meters to the community. System software and appli- 8. Hardavellas, N., Ferdman, M., Falsafi, SMT for multimedia workloads. In ICS
cation developers should understand and optimize power. B., Ailamaki, A. Toward dark silicon in (2004).
servers. IEEE Micro 31, 4 (2011), 6–15. 18. Singhal, R. Inside Intel next generation
Researchers should use both managed and native workloads 9. Hrishikesh, M.S., Burger, D., Jouppi, N.P., Nehalem microarchitecture. Intel
to quantitatively examine their innovations. Researchers Keckler, S.W., Farkas, K.I., Shivakumar, Developer Forum (IDF) presentation
P. The optimal logic depth per pipeline (August 2008), 2011.
should measure power and performance to understand and stage is 6 to 8 FO4 inverter delays. In 19. Tullsen, D.M., Eggers, S.J., Levy,
International Symposium on Computer H.M. Simultaneous multithreading:
optimize power, performance, and energy. Architecture (2002). maximizing on-chip parallelism. In
10. Isci, C., Martonosi, M. Runtime power ISCA (1995).
monitoring in high-end processors:
Acknowledgments
We thank Bob Edwards, Daniel Frampton, Katherine Coons, Hadi Esmaeilzadeh (hadianeh@ Kathryn S McKinley (mckinley@
Pradeep Dubey, Jungwoo Ha, Laurence Hellyer, Daniel cs.washington.edu), Department of microsoft.com), Microsoft Research,
Computer Science and Engineering, Redmond, WA; The University of Texas at
Jiménez, Bert Maher, Norm Jouppi, David Patterson, and University of Washington. Austin, Department of Computer Science,
John Hennessy. This work is supported by ARC DP0666059 Austin, TX.
Cao Ting, Stephen M Blackburn, Xi Yang
and NSF CSR-0917191. ({Cao.Ting, Steve.Blackburn, Xi.Yang}@
anu.edu.au), Research School of Computer
References tradeoffs in processor architecture and Science, Australian National University.
1. Azizi, O., Mahesri, A., Lee, B.C., Patel, circuit design: a marginal cost analysis.
S.J., Horowitz, M. Energy-performance In ISCA (2010). © 2012 ACM 0001-0782/12/07 $15.00

You’ve come a long way.

Share what you’ve learned.

ACM has partnered with MentorNet, the award-winning nonprofit e-mentoring network in engineering,
science and mathematics. MentorNet’s award-winning One-on-One Mentoring Programs pair ACM
student members with mentors from industry, government, higher education, and other sectors.
• Communicate by email about career goals, course work, and many other topics.
• Spend just 20 minutes a week - and make a huge difference in a student’s life.
• Take part in a lively online community of professionals and students all over the world.

Make a difference to a student in your field.

Sign up today at: www.mentornet.net
Find out more at: www.acm.org/mentornet
MentorNet’s sponsors include 3M Foundation, ACM, Alcoa Foundation, Agilent Technologies, Amylin Pharmaceuticals, Bechtel Group Foundation, Cisco
Systems, Hewlett-Packard Company, IBM Corporation, Intel Foundation, Lockheed Martin Space Systems, National Science Foundation, Naval Research
Laboratory, NVIDIA, Sandia National Laboratories, Schlumberger, S.D. Bechtel, Jr. Foundation, Texas Instruments, and The Henry Luce Foundation.

114 co mm unic atio ns o f th e ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 doi:10.1145/ 2209249 . 2 2 0 9 2 73

Technical Perspective
Why Study the Price
of Anarchy?
By Amos Fiat

O n M ay 1 6 , 2 0 1 2 , ACM ’ s Special Inter- mization dynamics, which means that

est Group on Algorithms and Com- they hold for so-called coarse correlat-
putation Theory) (SIGACT) and the Roughgarden uses ed equilibria, which implies that they
European Association for Theoreti- a technique hold for correlated equilibria, from
cal Computer Science (EATCS) an- which it follows that they hold for
nounced the recipients of the 2012 of “smooth games” mixed Nash equilibria, and thus for
Göedel prize to be awarded for the to give bounds, pure Nash equilibria, when they exist.
papers: Another important contribution
˲˲ “Worst-case Equilibira,” by Elias not only on the price of Roughgarden’s in the following pa-
Koutsoupias and Christos H. Papad- of anarchy with per is so show there exist interesting
imitriou, families of games for which smooth
˲˲ “How Bad Is Selfish Routing?” by respect to pure analysis gives the best possible price of
Tim Roughgarden and Éva Tardos, and
˲˲ “Algorithmic Mechanism Design,”
Nash equilibria, anarchy bounds (with respect to pure
Nash equilibria, and thus with respect
by Noam Nisan and Amir Ronen. but also for much to all classes that include pure Nash
This happy event will take place more general equilibria: mixed, correlated, coarse
this month at the International Collo- correlated, and no-regret dynamics).
quium on Automata, Languages and (and convincing) For such classes of games, smoothed
Programming in Warwick, England. So settings. analysis gives the total answer, no oth-
it is most fitting that this new paper by er technique is needed.
Tim Roughgarden appears in this issue A caveat to the complete generality
of Communications. of such smoothness techniques are
In 1999, Elias Koutsoupias and other natural dynamics such as best
Christos Papadimitriou initiated the response dynamics and so-called sink
study of “How much worse off are we equilibria for which smooth analysis
due to selfishness?” They compared may be insufficient to determine the
the worst case pure Nash equilibria ized) Nash equilibria are guaranteed price of anarchy.
to the optimal solution (in terms of to exist. Christodoulou and Koutsoupias
some target function, usually, social ˲˲ Computing the Nash equilibria is, used similar smoothness techniques
welfare). This ratio was later called the in general, difficult. To quote Papad- to study the (special case) of conges-
price of anarchy. Study of the price of imitriou quoting Kamal Jain: “If your tion games, in the context of mixed
anarchy gave birth to an entirely new laptop can’t find it then neither can and correlated equilibria, for both
field of study. the market.” So, why study the price of the price of anarchy (STOC 2005) and
One of the areas where negative ef- anarchy of a concept that may not exist for the price of stability (ESA 2005).
fects of selfish behavior were observed in practice? Smoothness techniques were implic-
was in the context of Wardrop equilib- In the following paper, Roughgar- itly used in many earlier papers to
ria in the analysis of traffic networks den studies “smooth games” to give price of anarchy results. Many of the
(essentially, Nash equilibria where bounds, not only on the price of anar- previous price of anarchy results can
there are infinitely many infinitesi- chy with respect to pure Nash equilib- be more easily understood by reading
mally small agents). Roughgarden and ria, but also for much more general the following paper than by reading
Tardos showed tight bounds on the (and convincing) settings. He extends the original.
price of anarchy for non-atomic rout- the class of games for which this tech-
ing with linear latency functions. The nique was known to work, and extends Amos Fiat is a professor of computer science at Tel Aviv
University, Israel.
worst case example was known as the the class of equilibria for which this
Braess paradox (D. Braess, 1968), but technique gives good bounds.
Roughgarden and Tardos showed that In particular, this technique is
this was tight. shown to work for so-called “regret-
There are two troubling issues re- minimization” dynamics, which are
garding the price of anarchy: quite natural and believable as rep-
˲˲ The pure Nash equilibria does not resenting what really happens. The
always exist. Mixed strategy (random- resulting bounds hold for regret-mini- © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l. 55 | n o. 7 | c om m u n ic at ion s o f t h e acm 115

research highlights
d oi:10.1145/ 2209249.2 2 0 9 2 74

Intrinsic Robustness
of the Price of Anarchy
By Tim Roughgarden

Abstract a network. The cost Ci(s) incurred by a player i in a game is a

The price of anarchy, defined as the ratio of the worst-case function of the entire vector s of players’ chosen strategies,
objective function value of a Nash equilibrium of a game which is called a strategy profile or an outcome. By definition,
and that of an optimal outcome, quantifies the inefficiency a strategy profile s of a game is a pure Nash equilibrium if no
of selfish behavior. Remarkably good bounds on this mea- player can decrease its cost via a unilateral deviation:
sure are known for a wide range of application domains.
However, such bounds are meaningful only if a game’s Ci(s) ≤ Ci (s′i, s−i )(1)
participants successfully reach a Nash equilibrium. This
drawback motivates inefficiency bounds that apply more for every i and si′ ∈ Si, where s−i denotes the strategies cho-
generally to weaker notions of equilibria, such as mixed sen by the players other than i in s. These concepts can be
Nash equilibria and correlated equilibria, or to sequences of defined equally well via payoff-maximization rather than
outcomes generated by natural experimentation strategies, cost-minimization; see also Example 2.5.
such as simultaneous regret-minimization. The price of anarchy (POA) measures the suboptimality
We prove a general and fundamental connection between caused by self-interested behavior. Given a game, a notion
the price of anarchy and its seemingly more general rela- of an “equilibrium” (such as pure Nash equilibria), and an
tives. First, we identify a “canonical sufficient condition” for objective function (such as the sum of players’ costs), the
an upper bound on the price of anarchy of pure Nash equilib- POA of the game is defined as the ratio between the largest
ria, which we call a smoothness argument. Second, we prove cost of an equilibrium and the cost of an optimal outcome.
an “extension theorem”: every bound on the price of anar- An upper bound on the POA has an attractive worst-case fla-
chy that is derived via a smoothness argument extends auto- vor: it applies to every possible equilibrium and obviates the
matically, with no quantitative degradation in the bound, to need to predict a single outcome of selfish behavior. Many
mixed Nash equilibria, correlated equilibria, and the aver- researchers have proved remarkably good bounds on the
age objective function value of every no-regret sequence of POA in a wide range of models; see Chapters 17–21 of Nisan
joint repeated play. Third, we prove that in routing games, et al.17 and the references therein.
smoothness arguments are “complete” in a proof-theoretic
sense: despite their automatic generality, they are guaran- 1.1. The need for more robust bounds
teed to produce an optimal worst-case upper bound on the A good bound on the price of anarchy of a game is not
price of anarchy. enough to conclude that self-interested behavior is rela-
tively benign. Such a bound is meaningful only if a game’s
participants successfully reach an equilibrium. For pure
1. INTRODUCTION Nash equilibria, however, there are a number of reasons
Every student of game theory learns early and often that why this might not occur: perhaps the players fail to coor-
equilibria are inefficient—self-interested behavior by auton- dinate on one of multiple equilibria, or they are playing
omous decision makers generally leads to an outcome a game in which computing a pure Nash equilibrium is a
inferior to the one that a hypothetical benevolent dictator computationally intractable problem9 or, even more fun-
would choose. Such inefficiency is ubiquitous in real-world damentally, a game in which pure Nash equilibria do not
situations and arises for many different reasons: congestion exist. These critiques motivate worst-case performance
externalities, network effects, mis-coordination, and so on. bounds that apply to as wide a range of outcomes as pos-
It can also be costly or infeasible to eliminate in many situa- sible, and under minimal assumptions about how players
tions, with large networks being one obvious example. The play and coordinate in a game.
past ten years have provided an encouraging counterpoint This article presents a general theory of “robust” bounds
to this widespread equilibrium inefficiency: in a number of on the price of anarchy. We focus on the hierarchy of
interesting application domains, decentralized optimiza- ­fundamental equilibrium concepts shown in Figure 1; the
tion by competing individuals provably approximates the full version22 discusses additional generalizations of pure
optimal outcome. Nash equilibria, including approximate equilibria and
A rigorous guarantee of this type requires a formal behav-
ioral model, in order to define “the outcome of self-inter-
The original version of this article was published in the
ested behavior.” The majority of previous research studies
Proceedings of the 41st Annual ACM Symposium on Theory
pure-strategy Nash equilibria, which are defined as follows.
of Computing, May 2009.
Each player i selects a strategy si from a set Si, like a path in

116 c omm unic atio ns of t h e ac m | JUly 201 2 | vol . 5 5 | no. 7

Figure 1. Generalizations of pure Nash equilibria. “PNE” stands for
(B) We prove an “extension theorem”: every bound on
pure Nash equilibria, “MNE” for mixed Nash equilibria, “CorEq” for the price of anarchy that is derived via a smooth-
correlated equilibria, and “No Regret (CCE)” for coarse correlated ness argument extends automatically, with no
equilibria, which are the empirical distributions corresponding to quantitative degradation in the bound, to all of the
repeated joint play in which every player has no (external) regret. more general equilibrium concepts pictured in
Figure 1.
Easy to (C) We prove that routing games, with cost functions
No Regret (CCE) compute/ restricted to some arbitrary set, are “tight” in the fol-
learn lowing sense: smoothness arguments, despite their
CorEq automatic generality, are guaranteed to produce opti-
mal worst-case upper bounds on the POA, even for
Always the set of pure Nash equilibria. Thus, in these classes
MNE exists, hard of games, the worst-case POA is the same for each of
to compute the equilibrium concepts of Figure 1.

PNE Need not 2. SMOOTH GAMES

exist, hard
to compute 2.1. Definitions
By a cost-minimization game, we mean a game—players,
strategies, and cost functions—together with the joint cost
objective function . Essentially, a “smooth
game” is a cost-minimization game that admits a POA bound
outcome sequences generated by best-response dynamics. of a canonical type (a “smoothness argument”). We give the
We formally define the equilibrium concepts of Figure 1— formal definition and then explain how to interpret it.
mixed Nash equilibria, correlated equilibria, and coarse cor-
related equilibria—in Section 3.1, but mention next some of Definition 2.1 (Smooth Games): A cost-minimization
their important properties. game is (λ, m)-smooth if for every two outcomes s and s*,
Enlarging the set of equilibria weakens the behavioral
and technical assumptions necessary to justify equilib- (2)
rium analysis. First, while there are games with no pure
Nash ­ equilibria—“Matching Pennies” being a simple Roughly, smoothness controls the cost of a set of “one-
example—every (finite) game has at least one mixed Nash dimensional perturbations” of an outcome, as a function of
equilibrium.16 As a result, the “nonexistence critique” for both the initial outcome s and the perturbations s*.
pure Nash equilibria does not apply to any of the more gen- We claim that if a game is (λ, m)-smooth, with λ > 0 and
eral concepts in Figure 1. Second, while computing a mixed m < 1, then each of its pure Nash equilibria s has cost at most
Nash equilibrium is in general a computationally intrac- λ/(1 − m) times that of an optimal solution s*. In proof, we
table problem,5, 8 computing a correlated equilibrium is derive
not (see, e.g., Chapter 2 of Nisan et al.17). Thus, the “intrac-
tability critique” for pure and mixed Nash equilibria does (3)
not apply to the two largest sets of Figure 1. More impor-
tantly, these two sets are ­“easily learnable”: when a game
is played repeatedly over time, there are natural classes of (4)
learning dynamics—processes by which a player chooses
its strategy for the next time step, as a function only of its
own payoffs and the history of play—that are guaranteed to (5)
converge quickly to these sets of equilibria (see Chapter 4
of Nisan et al.17). where (3) follows from the definition of the objective func-
tion; inequality (4) follows from the Nash equilibrium condi-
1.2. Overview tion (1), applied once to each player i with the hypothetical
Our contributions can be divided into three parts. deviation s*i ; and inequality (5) follows from the defining
condition (2) of a smooth game. Rearranging terms yields
(A) We identify a sufficient condition for an upper bound the claimed bound.
on the POA of pure Nash equilibria of a game, which Definition 2.1 is sufficient for the last line of this three-
encodes a canonical proof template for deriving such line proof (3)–(5), but it insists on more than what is needed:
bounds. We call such proofs “smoothness argu- it demands that the inequality (2) hold for every outcome s,
ments.” Many of the POA upper bounds in the litera- and not only for Nash equilibria. This is the basic reason why
ture can be recast as instantiations of this canonical smoothness arguments imply worst-case bounds beyond
method. the set of pure Nash equilibria.

JUly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t h e acm 117

research highlights

We define the robust POA as the best upper bound on the illustrates how smoothness arguments can be defined and
POA that is provable via a smoothness argument. used in payoff-maximization games, and also with a “one-
sided” variant of sum objective functions (cf., Remark 2.3).
Definition 2.2 (Robust POA): The robust price of anarchy of
a cost-minimization game is Example 2.4 (Atomic Congestion Games): A congestion
game is a cost-minimization game defined by a ground set E
or resources, a set of k players with strategy sets S1, . . . , Sk ⊆ 2E,
and a cost function ce: Z+ → R for each resource e  E.20 In
with m always constrained to be less than 1. this article, we always assume that cost functions are non-
negative and nondecreasing. A canonical example is routing
Remark 2.3 (Variations on Smoothness): Examining the games, where E is the edge set of a network, and the strategies
three-line proof (3)–(5) reveals that the assumptions can be of a player correspond to paths between its source and sink
weakened in two ways. First, the assumption that the objec- vertices. Given a strategy profile s = (s1, . . . , sk), with si ∈ Si for
tive function satisfies can be replaced by the each i, we say that xe = |{i : e ∈ si}| is the load induced on e by
assumption ; we exploit this in Example 2.5 s, defined as the number of players that use it in s. The cost
below. Second, in Definition 2.1, the inequality (2) only to player i is defined as , where x is the vector
needs to hold for all outcomes s and some optimal solution of loads induced by s. For this example, we assume that every
s*, rather than for all pairs s, s* of outcomes. This relaxation cost function is affine, meaning that ce(x) = aex + be with ae, be ≥
is useful in some applications.4, 23 0 for every resource e ∈ E.
Finally, there is an analogous definition of smooth games We claim that every congestion game with affine cost
for maximization objectives; see Example 2.5. functions is (5/3, 1/3)-smooth. The basic reason for this was
identified by Lemma 1 of Christodoulou and Koutsoupias,7
2.2. Intuition who noted that
Smoothness arguments should be interpreted as a class of
upper bound proofs for the POA of pure Nash equilibria that
are confined to use the equilibrium hypothesis in a minimal
way. To explain, recall the canonical three-line proof (3)–(5). for all nonnegative integers y, z. Thus, for all a, b ≥ 0 and
The first inequality (4) uses the Nash equilibrium hypothesis, nonnegative integers y, z,
but only to justify why each player i selects its equilibrium
strategy si rather than its strategy s*i  in the optimal outcome. If (6)
we care only about the POA of pure Nash equilibria, then we
are free to invoke the Nash equilibrium hypothesis again to To establish smoothness, consider a pair s, s* of outcomes of
prove the second inequality (5) or, more generally, to establish a congestion game with affine cost functions, with induced
an upper bound using any argument that we please. Using a loads x, x*. Since the number of players using resource e in
smoothness argument—that is, proving inequality (5) for all the outcome (s*i , s–i ) is at most one more than that in s, and
outcomes s—is tantamount to discarding the Nash equilib- this resource contributes to precisely x*e terms of the form
rium hypothesis after it is used to justify the first inequality (4). Ci (s*i , s−i ), we have

2.3. Two examples

Concern about the range of applicability of a definition
grows as its interesting consequences accumulate. Given

that smoothness arguments enable the extension theorem
discussed in Section 1.2, how many games can be (λ,  m)-
smooth with interesting values of λ, m? To alleviate such (7)
fears and add some concreteness to the discussion, we next
single out two well-known POA analyses that can be recast as where (7) follows from (6), with x*e and xe playing the roles of y
smoothness arguments. More generally, many but not all of and z, respectively. The canonical three-line argument (3)–(5)
the known price of anarchy bounds follow from smoothness then implies an upper bound of 5/2 on the POA of pure Nash
proofsa; see the full version22 for a detailed discussion. equilibria in every congestion game with affine cost func-
The first example is a special class of congestion games; tions. This fact was first proved independently in Awerbuch
Section 4 studies the general case in detail. The second exam- et al.2 and Christodoulou and Koutsoupias,7 where matching
ple concerns Vetta’s well-studied utility games,25 and also lower bounds were also supplied. Our extension theorem
(Theorem 3.1) implies that the bound of 5/2 extends to the
other three sets of outcomes shown in Figure 1. These exten-
a
  The most common reason that a price of anarchy bound fails to qualify as sions were originally established in two different papers3, 6
a smoothness proof is that the Nash equilibrium hypothesis is invoked for a subsequent to the original POA bound.2, 7
hypothetical deviation s*i that is a function of the other players’ equilibrium
actions s–i. In most of these cases, it is also known that the worst-case POA of
mixed Nash equilibria is strictly worse than that of pure Nash equilibria, and Example 2.5 (Valid Utility Games): Our final example
hence no lossless extension theorem exists. concerns a class of games called valid utility games.25 These

118 c om munic atio ns of t h e ac m | JUly 201 2 | vo l . 5 5 | no. 7

games are naturally phrased as payoff-maximization games, automatically to the more general equilibrium concepts in
where each player has a payoff function Pi(s) that it strives to Figure 1 and to the corresponding outcome sequences in
maximize. We use P to denote the objective function of a pay- games played over time. Several less direct consequences
off-maximization game. We call such a game (λ, m)-smooth if of smoothness arguments are discussed in the full ver-
sion.22 We work with cost-minimization games, though
similar results hold for smooth payoff-maximization games
(cf., Example 2.5).
for every pair s, s* of outcomes. A derivation similar to (3)–(5)
shows that, in a (λ, m)-smooth payoff-maximization game, 3.1. Static equilibrium concepts
the objective function value of every pure Nash equilibrium We begin with implications of Definition 2.1 for random-
is at least a λ/(1 + m) fraction of the maximum possible. We ized equilibrium concepts in one-shot games; the next sec-
define the robust POA of a payoff-maximization game as tion treats outcome sequences generated by repeated play.
the supremum of λ/(1 + m) over all legitimate smoothness A set (s1, . . . , sk) of independent probability distributions
parameters (λ, m). over strategy sets—one per player of a cost-minimization
A valid utility game is defined by a ground set E, a nonneg- game—is a mixed Nash equilibrium of the game if no player
ative submodular function V defined on subsets of E, and a can decrease its expected cost under the product distribu-
strategy set Si ⊆ 2E and a payoff function Pi for each player tion s = s1 × . . . × sk via a unilateral deviation:
i  =  1, 2, . . . , k.b For example, the set E could denote a set of
locations where facilities can be built, and a strategy si ⊆ E Ess   [Ci(s)] ≤ Es s [Ci(s′i , s−i)]
−i −i

could denote the locations at which player i chooses to build

facilities. For an outcome s, let U(s) ⊆ E denote the union for every i and s′i, ∈Si, where s−i is the product distribution
of players’ strategies in s. The objective function value of all sj’s other than si. (By linearity, it suffices to consider
of an outcome s is defined as P(s) = V(U(s)). Furthermore, only pure-strategy unilateral deviations.) Obviously, every
the definition requires that two conditions hold: (i) for each pure Nash equilibrium is a mixed Nash equilibrium and not
player i, Pi(s) ≥ V(U(s)) − V(U(0/, s−i )) for every outcome s; and conversely; indeed, many games have no pure Nash equilib-
(ii) for every outcome s. One concrete exam- rium, but every finite game has a mixed Nash equilibrium.16
ple of such a game is competitive facility location with price- A correlated equilibrium of a cost-minimization game G
taking markets and profit-maximizing firms.25 is a (joint) probability distribution s over the outcomes of G
We claim that every valid utility game with a nondecreas- with the property that
ing objective function V is (1, 1)-smooth. The proof is essen-
tially a few key inequalities from Theorem 3.2 of Vetta25, as Ess [Ci(s)½si] ≤ Ess [Ci(s′i , s−i)½si](8)
follows. Let s, s* denote arbitrary outcomes of a valid utility
game with a nondecreasing objective function. Let Ui  ⊆ E for every i and si, s′i ∈Si. A classical interpretation of a corre-
denote the union of all of the players’ strategies in s, together lated equilibrium is in terms of a mediator, who draws an
with the strategies employed by players 1, 2, . . . , i in s*. outcome s from the publicly known distribution s and pri-
Applying condition (i), the submodularity of V, and the non- vately “recommends” strategy si to each player i. The equi-
decreasing property of V yields librium condition requires that following a recommended
strategy always minimizes the expected cost of a player, con-
ditioned on the recommendation. Mixed Nash equilibria
are precisely the correlated equilibria that are also product
distributions. Correlated equilibria have been widely stud-
ied as strategies for a benevolent mediator, and also because
of their relative tractability. Because the set of correlated
equilibria is explicitly described by a small set of linear
as desired. This smoothness argument implies a lower inequalities, computing (and even optimizing over) corre-
bound of 1/2 on the POA of pure Nash equilibria in every lated equilibria can be done in time polynomial in the size of
valid utility game with a nondecreasing objective function— the game (see, e.g., Chapter 2 of Nisan et al.17). They are also
a result first proved in Vetta,25 along with a matching upper relatively “easy to learn,” as discussed in the next section.
bound. Our extension theorem shows that this lower bound Finally, a coarse correlated equilibrium of a cost-minimi-
applies more generally to all of the equilibria depicted in zation game is a probability distribution s over outcomes
Figure 1, a fact first established in Blum et al.3 that satisfies

3. AN EXTENSION THEOREM Esσ[Ci(s)] ≤ Esσ [Ci(s′i , s−i)](9)

This section states and proves the extension theorem dis-
cussed in Section 1.2: every POA bound on pure Nash for every i and s′i, ∈si. While a correlated equilibrium (8) pro-
equilibria derived from a smoothness argument extends tects against deviations by players aware of their recom-
mended strategy, a coarse correlated equilibrium (9) is only
b
  A set function V : 2E → R is submodular if V(X ∩ Y) + V(X ∪ Y) ≤ V(X) + V(Y ) constrained by player deviations that are independent of the
for every X, Y ⊆ E. sampled outcome. Since every correlated equilibrium is also

JUly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t h e acm 119

research highlights

a coarse correlated equilibrium, coarse correlated equilibria di(st) can be positive or negative. We can mimic the deriva-
are “even easier” to compute and learn, and are thus a still tion in (3)–(5) to obtain
more plausible prediction for the realized play of a game.
We now give our extension theorem for equilibrium (17)
concepts in one-shot games: every POA bound proved via a
smoothness argument extends automatically to the set of for each t.
coarse correlated equilibria. With the “correct” definitions Suppose that every player i experiences vanishing average
in hand, the proof writes itself. (external) regret, meaning that its cost over time is competitive
with that of every time-invariant strategy:
Theorem 3.1 (Extension Theorem—Static Version)
For every cost-minimization game G with robust POA r(G), every
coarse correlated equilibrium s of G, and every outcome s* of G, (18)

Esσ[C(s)] ≤ r(G) · C(s*).

Repeating the same pure Nash equilibrium over and over
Proof: Let G be a (λ, m)-smooth cost-minimization game, s again yields a degenerate example, but in general such
a coarse correlated equilibrium, and s* an outcome of G. We sequences can exhibit highly oscillatory behavior over arbi-
can write trarily large time horizons (see, e.g., Blum et al.3 and Kleinberg
et al.13).
Averaging (17) over the T time steps and reversing the
(10) order of the resulting double summation yields

(11) (19)

(12) Recalling from (16) that di(st) is the additional cost incurred
by player i at time t due to playing strategy sti instead of the
(time-invariant) strategy s*i , the no-regret guarantee (18)
implies that is bounded above by a term
(13)
that goes to 0 with T. Since this holds for every player i,
inequality (19) implies that the average cost of outcomes
(14) in the sequence is no more than the robust POA times the
minimum-possible cost, plus an error term that approaches
zero as T → ∞.
(15)
Theorem 3.2 (Extension Theorem—Repeated Version)
where equality (10) follows from the definition of the For every cost-minimization game G with robust POA r(G), every
­objective function, equalities (11), (13), and (15) follow from outcome sequence s 1, . . . , s T that satisfies (18) for every player,
linearity of expectation, inequality (12) follows from the and every outcome s*of G,
­definition (9) of a coarse correlated equilibrium (applied
once per player, with the hypothetical deviation s*i ), and
inequality (14) follows from the assumption that the game is
(λ, m)-smooth. Rearranging terms completes the proof.   as T → ∞.
Blum et al.3 were the first to consider bounds of this type,
3.2. Repeated play and no-regret sequences calling them “the price of total anarchy.”
The extension theorem (Theorem 3.1) applies equally well We reiterate that the type of bound in Theorem 3.2 is sig-
to certain outcome sequences generated by repeated play, nificantly more compelling, and assumes much less from
because of a well-known correspondence between such both the game and its participants, than the one that applies
sequences and static equilibrium concepts. To illustrate only to Nash equilibria. While Nash equilibria can be intrac-
this point, consider a sequence s1, s2, . . . , sT of outcomes of a table or impossible to find, there are several computation-
(λ, m)-smooth game and a minimum-cost outcome s* of the ally efficient “off-the-shelf” learning algorithms with good
game. For each i and t, define convergence rates that are guaranteed, in any game, to gen-
erate outcome sequences with vanishing average regret (see,
(16) e.g., Chapter 4 of Nisan et al.17). Of course, the guarantee in
Theorem 3.2 makes no reference to which learning algo-
as the hypothetical improvement in player i’s cost at time t rithms (if any) the players’ use to play the game—the bound
had it used the strategy s*i in place of sti. When st is a Nash equi- applies whenever repeated joint play has low regret, whatever
librium, di(st) cannot be positive; for an arbitrary outcome st, the reason.

120 c omm unicatio ns o f th e ac m | JUly 201 2 | vo l. 5 5 | no. 7

Remark 3.3 (Theorems 3.1 and 3.2): Theorems 3.1 and 3.2 Recall from Example 2.4 the definition of and notation for
are essentially equivalent, in that either one can be derived congestion games. Here, we consider arbitrary nonnegative
from the other. The reason is that the set of coarse correlated and nondecreasing cost functions ce. The worst-case POA in
equilibria of a game is precisely the closure of the empirical congestion games depends on the “degree of nonlinearity”
distributions of (arbitrarily long) sequences in which every of the allowable cost functions. For example, for polynomial
player has nonpositive average regret. cost functions with nonnegative coefficients and degree at
most d, the worst-case POA in congestion games is finite but
Remark 3.4 (Correlated Equilibria and Swap Regret) exponential in d.1, 2, 7, 18
There is a more stringent notion of regret—swap regret— Example 2.4 shows that, if G is the set of congestion
under which there is an analogous correspondence between games with affine cost functions, then the right-hand side
the correlated equilibria of a game and the outcome of (20) is at most 5/2. Constructions in Awerbuch et al.2 and
sequences in which every player has nonpositive (swap) Christodoulou and Koutsoupias7 show that the left-hand
regret. There are also computationally efficient “off the side of (20) is at least 5/2 for this class of games. Thus, con-
shelf” learning algorithms that guarantee each player van- gestion games with affine cost functions form a tight class.
ishing average swap regret in an arbitrary game.10, 12 Our final result shows that this fact is no fluke.

4. CONGESTION GAMES ARE TIGHT theorem 4.2: For every non-empty set C of nondecreasing,
The worst-case POA for a set of allowable outcomes can only ­ ositive cost functions, the set of congestion games with cost
p
increase as the set grows bigger. This section proves that, in functions in C is tight.
congestion games with restricted cost functions, the worst-
case POA is exactly the same for each of the equilibrium In addition to showing that smoothness arguments always
concepts of Figure 1. We prove this by showing that smooth- give optimal POA bounds in congestion games, this result
ness arguments, despite their automatic generality, provide and its proof imply the first POA bounds of any sort for
a tight bound on the POA, even for pure Nash equilibria. congestion games with nonpolynomial cost functions, and
More precisely, let G denote a set of cost-minimization the first structural characterization of universal worst-case
games, and assume that a nonnegative objective function examples for the POA in congestion games.
has been defined on the outcomes of these games. Let A(G) The proof of Theorem 4.2 is technical and we provide only
denote the parameter values (λ, m) such that every game of a high-level outline; the complete proof can be found in the
G is (λ, m)-smooth. Let  ⊆ G denote the games with at least full version.22 For the following discussion, fix a set C of cost
one pure Nash equilibrium, and rpure(G) the POA of pure functions. The first step is to use the fact that, in a conges-
Nash equilibria in a game G ∈ . The canonical three-line tion game, the objective function and players’ cost functions
proof (3)–(5) shows that for every (λ, m) ∈A(G) and every G ∈ are additive over the resources E. This reduces the search for
, rpure(G) ≤ λ/(1 − m). We call a set of games tight if equality parameters (λ, m) that satisfy condition (2) of Definition 2.1—
holds for suitable choices of (λ, m) ∈A(G) and G ∈ . which imposes one constraint for every congestion game
with cost functions in C, and every pair s, s* of outcomes in
Definition 4.1 (Tight Class of Games): A set G of games that game—to the search for parameters (λ, m) that satisfy
is tight if
(21)
(20)
for every cost function c ∈ C, nonnegative integer x, and posi-
tive integer x*. This condition is the same as (6) in Example 2.4
The right-hand side of (20) is the best worst-case upper bound for the case where C is the set of affine cost functions.
provable via a smoothness argument, and it applies to all of The second step of the proof is to understand the
the sets shown in Figure 1. The left-hand side of (20) is the ­optimization problem of minimizing the objective function
actual worst-case POA of pure Nash equilibria in G—corre- λ/(1 − m) over the “feasible region” A(C), where A(C) denotes
sponding to the smallest set in Figure 1—among games with the set of values (λ, m) that meet the condition (21) above.
at least one pure Nash equilibrium. That the left-hand side is This optimization problem is almost the same as the right-
trivially upper bounded by the right-hand side is reminiscent hand side of (20), and it has several nice properties. First,
of “weak duality.” Tight classes of games are characterized by there are only two decision variables—λ and m—so A(C) is
the min-max condition (20), which can be loosely interpreted contained in the plane. Second, while there are an infinite
as a “strong duality-type” result.c In a tight class of games, number of constraints (21), each is linear in λ and m. Thus,
every valid upper bound on the worst-case POA of pure Nash A(C) is the intersection of halfplanes. Third, the objective
equilibria is superseded by a suitable smoothness argument. function λ/(1 − m) is decreasing is both decision variables.
Thus, every such bound—whether or not it is proved using Thus, ignoring some edge cases that can be handled sepa-
a smoothness argument—is “intrinsically robust,” in that it rately, the choice of (λ, m) that minimizes the objective func-
applies to all of the sets of outcomes in Figure 1. tion lies on the “southwestern boundary” of A(C), and can
be characterized as the unique point of A(C) that satisfies
c
  See Nadav and Roughgarden15 for a formal treatment of the duality between with equality a particular pair of constraints of the form (21).
equilibrium concepts and POA bounds. The third and most technical part of the proof is to show a

JUly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i c at ion s o f t h e acm 121

research highlights

matching lower bound on the left-hand side of (20). The intui­ given independently in Aland et al.1 and Olver.18
tion behind the construction is to arrange a congestion game Many previous works recognized the possibility of and
in which each player has two strategies, one that uses a small motivation for more general POA bounds. The underlying
number of resources, and the other a disjoint strategy that bound on the POA of pure Nash equilibria can be formulated
uses a large number of resources. In the optimal outcome, as a smoothness argument in almost all of these cases, so our
all players use their small strategies and incur low cost. (This extension theorem immediately implies, and often strength-
outcome is also a pure Nash equilibrium.) In the suboptimal ens, these previously proved robust bounds. Specifically, Aland
pure Nash equilibrium, all players use their large strategies, et al.,1 Awerbuch et al.,2 Christodoulou and Koutsoupias,7 and
thereby “flooding” all resources and incurring a large cost. Vetta25 each observe that their upper bounds on the worst-case
How can this suboptimal outcome persist as a Nash equilib- POA of pure Nash equilibria carry over easily to mixed Nash
rium? If one player deviates unilaterally, it enjoys the benefit of equilibria. In Christodoulou and Koutsoupias,6 the worst-case
fewer resources in its strategy, but each of these new resources POA of correlated equilibria is shown to be the same as that
now has load one more than that of each of the resources it for pure Nash equilibria in unweighted and weighted conges-
was using previously. Implemented optimally, this construc- tion games with affine cost functions. Blum et al.3 rework and
tion produces a congestion game and a pure Nash equilibrium generalize several bounds on the worst-case POA of pure Nash
of it with cost a λ/(1 − m) factor larger than that of the optimal equilibria to show that the same bounds hold for the average
outcome, where (λ, m) are the optimal smoothness parameters objective function value of no-regret sequences. Their applica-
identified in the second step of the proof. tions include valid utility games25 and the (suboptimal) bounds
of Awerbuch et al.2 and Christodoulou and Koutsoupias7 for
Remark 4.3 (poa bounds for all cost functions) unweighted congestion games with polynomial cost func-
Theorem 4.2 gives the first solution to the worst-case POA in tions, and also a constant-sum location game and a fairness
congestion games with cost functions in an arbitrary set C. Of objective, which falls outside of our framework.
course, precisely computing the exact value of the worst-case Versions of our two-parameter smoothness definition
POA is not trivial, even for simple sets C. Arguments in Aland are implicit in a few previous papers, in each case for a spe-
et al.1 and Olver18 imply a (complex) closed-form expression for cific model and without any general applications to robust
the worst-case POA when C is a set of polynomials with non- POA guarantees: Perakis19 for a nonatomic routing model
negative coefficients. Similar computations should be possi- with nonseparable cost functions, Christodoulou and
ble for some other simple sets C. More broadly, the second and Koutsoupias6 for congestion games with affine cost func-
third steps of the proof of Theorem 4.2 indicate how to numer- tions, and Harks11 for splittable congestion games.
ically produce good upper and lower bounds, respectively, on
the worst-case POA when there is a particular set C of interest. 6. CONCLUSION
Pure-strategy Nash equilibria—where each player deter-
Remark 4.4 (worst-Case Congestion Games): The ministically picks a single strategy—are often easier to rea-
details of the construction in the third step of the proof of son about than their more general cousins like mixed Nash
Theorem 4.2 show that routing games on a bidirected cycle equilibria, correlated equilibria, and coarse correlated equi-
are universal worst-case examples for the POA, no matter what libria. On the other hand, inefficiency guarantees for more
the allowable set of cost functions. This corollary is an analog general classes of equilibria are crucial for several reasons:
of a simpler such sufficient condition for nonatomic conges- pure Nash equilibria do not always exist; they can be intrac-
tion games—where there is a continuum of players, each of table to compute, even when they are guaranteed to exist;
negligible size—in which, under modest assumptions on C, and even when efficiently computable by a centralized algo-
the worst-case POA is always achieved in two-node two-link rithm, they can elude natural learning dynamics.
networks.21 This article presented an extension theorem, which auto-
matically extends, in “black-box” fashion, price of anarchy
5. FURTHER RELATED WORK bounds for pure Nash equilibria to the more general equilib-
The price of anarchy was first studied in Koutsoupias and rium concepts listed above. Such an extension theorem can
Papadimitriou14 for makespan minimization in scheduling only exist under some conditions, and the key idea is to restrict
games. This is not a sum objective function, and the worst- the method of proof used to bound the price of anarchy of pure
case POA in this model was immediately recognized to be Nash equilibria. We defined smooth games to formalize a
different for different equilibrium concepts.3, 14 See Chapter canonical method of proof, in which the Nash equilibrium
20 of Nisan et al.17 for a survey of the literature on this model. hypothesis is used in only a minimal way, and proved an exten-
The POA with a sum objective was first studied in Rough­ sion theorem for smooth games. Many of the games in which
garden and Tardos24 for nonatomic selfish routing games. the price of anarchy has been studied are smooth games in
The first general results on the POA of pure Nash equilibria our sense.d For the fundamental model of congestion games
for (atomic) congestion games and their weighted variants with arbitrarily restricted cost functions, we proved that this
are in Awerbuch et al.2 and Christodoulou and Koutsoupias,7
which gave tight bounds for games with affine cost functions d
  Since the conference version of this article, the definition of smooth games
and reasonably close upper and lower bounds for games with has been refined and extended in several ways, and new smoothness argu-
polynomial cost functions with nonnegative coefficients; ments have been discovered for a number of interesting models. See the full
matching upper and lower bounds for the latter class were later version22 for details and references.

122 co m munic atio ns o f t h e acm | JUly 201 2 | vo l . 5 5 | no. 7

canonical proof method is guaranteed to produce an opti- and collusion in network games Cambridge University Press, 2007.
with splittable flow. Proceedings of 18. Olver, N. The price of anarchy and a
mal upper bound on the worst-case POA. In this sense, POA the 6th International Workshop on priority-based model of routing.
bounds for congestion games are “intrinsically robust.” Approximation and Online Algorithms M.S. thesis, McGill University (2006).
(WAOA’08), E. Bampis and 19. Perakis, G. The “price of anarchy”
M. Skutella, eds. Volume 5426 of under nonlinear and asymmetric costs.
Acknowledgments LNCS (2008), 133–146. Math. Oper. Res. 32, 3 (2007), 614–628.
12. Hart, S., Mas-Colell, A. A simple 20. Rosenthal, R.W. A class of games
This research was supported in part by NSF CAREER Award adaptive procedure leading to possessing pure-strategy Nash
CCF-0448664, an ONR Young Investigator Award, an AFOSR correlated equilibria. Econometrica equilibria. Int. J. Game Theor. 2, 1
68, 5 (2000), 1127–1150. (1973), 65–67.
MURI grant, and an Alfred P. Sloan Fellowship. 13. Kleinberg, R.D., Piliouras, G., Tardos, É. 21. Roughgarden, T. The price of anarchy
Multiplicative updates outperform is independent of the network
generic no-regret learning in congestion topology. J. Comput. Syst. Sci. 67, 2
References 6. Christodoulou, G., Koutsoupias, E. games. In Proceedings of the 41st (2003), 341–364.
1. Aland, S., Dumrauf, D., Gairing, M., On the price of anarchy and stability Annual ACM Symposium on Theory of 22. Roughgarden, T. Intrinsic robustness
Monien, B., Schoppmann, F. Exact of correlated equilibria of linear Computing (STOC) (2009), 533–542. of the price of anarchy. In 41st ACM
price of anarchy for polynomial congestion games. In Proceedings 14. Koutsoupias, E., Papadimitriou, C.H. Symposium on Theory of Computing
congestion games. SIAM J. Comput. of the 13th Annual European Worst-case equilibria. In Proceedings (STOC) (2009), 513–522.
40, 5 (2011), 1211–1233. Symposium on Algorithms (ESA), of the 16th Annual Symposium on 23. Roughgarden, T. The price of anarchy
2. Awerbuch, B., Azar, Y., Epstein, A. The volume 3669 of Lecture Notes in Theoretical Aspects of Computer in games of incomplete information.
price of routing unsplittable flow. In Computer Science (2005), 59–70. Science (STACS), volume 1563 of In 13th ACM Conference on
Proceedings of the 37th Annual ACM 7. Christodoulou, G., Koutsoupias, E. The Lecture Notes in Computer Science Electronic Commerce (EC)
Symposium on Theory of Computing price of anarchy of finite congestion (1999), 404–413. (2012).
(STOC) (2005), 57–66. games. In Proceedings of the 37th 15. Nadav, U., Roughgarden, T. The 24. Roughgarden, T., Tardos, É. How
3. Blum, A., Hajiaghayi, M., Ligett, K., Annual ACM Symposium on Theory of limits of smoothness: A primal-dual bad is selfish routing? J. ACM 49, 2
Roth, A. Regret minimization and the Computing (STOC) (2005), 67–73. framework for price of anarchy (2002), 236–259.
price of total anarchy. In Proceedings 8. Daskalakis, C., Goldberg, P.W., bounds. In The 6th International 25. Vetta, A. Nash equilibria in
of the 40th Annual ACM Symposium Papadimitriou, C.H. The complexity of Workshop on Internet and Network competitive societies, with
on Theory of Computing (STOC) computing a Nash equilibrium. SIAM Economies (WINE) (2010), 319–326. applications to facility location, traffic
(2008), 373–382. J. Comput. 39, 1 (2009), 195–259. 16. Nash, J.F. Equilibrium points in routing and auctions. In Proceedings
4. Caragiannis, I., Kaklamanis, C., 9. Fabrikant, A., Papadimitriou, C.H., N-person games. Proc. Natl. Acad. Sci. of the 43rd Annual Symposium on
Kanellopoulos, P., Kyropoulou, M., Talwar, K. The complexity of pure Nash 36, 1 (1950), 48–49. Foundations of Computer Science
Lucier, B., Leme, R.P., Tardos, É. On the equilibria. In Proceedings of the 36th 17. Nisan, N., Roughgarden, T.,Tardos, É., (FOCS) (2002), 416–425.
efficiency of equilibria in generalized Annual ACM Symposium on Theory of Vazirani, V., eds. Algorithmic Game Theory.
second price auctions. Submitted Computing (STOC) (2004), 604–612.
2012. Earlier versions appeared in 10. Foster, D., Vohra, R. Calibrated
FOCS, ’10 and EC, ’11. learning and correlated equilibrium. Tim Roughgarden ([email protected].
5. Chen, X., Deng, X., Teng, S.H. Settling Game Econ. Behav. 21(1–2) (1997), edu), Stanford University, Stanford, CA.
the complexity of two-player Nash 40–55.
equilibria. J. ACM 56, 3 (2009). 11. Harks, T. Stackelberg strategies © 2012 ACM 0001-0782/12/07 $15.00

World-Renowned Journals from ACM

ACM publishes over 50 magazines and journals that cover an array of established as well as emerging areas of the computing field.
IT professionals worldwide depend on ACM's publications to keep them abreast of the latest technological developments and industry
news in a timely, comprehensive manner of the highest quality and integrity. For a complete listing of ACM's leading magazines & journals,
including our renowned Transaction Series, please visit the ACM publications homepage: www.acm.org/pubs.

ACM Transactions ACM Transactions

on Interactive on Computation
Intelligent Systems PLEASE CONTACT ACM MEMBER
Theory SERVICES TO PLACE AN ORDER
Phone: 1.800.342.6626 (U.S. and Canada)
+1.212.626.0500 (Global)
Fax: +1.212.944.1318
(Hours: 8:30am–4:30pm, Eastern Time)
Email: [email protected]
Mail: ACM Member Services
General Post Office
PO Box 30777
New York, NY 10087-0777 USA

ACM Transactions on Interactive ACM Transactions on Computation

Intelligent Systems (TIIS). This Theory (ToCT). This quarterly peer-
quarterly journal publishes papers reviewed journal has an emphasis
on research encompassing the on computational complexity, foun-
design, realization, or evaluation of dations of cryptography and other
interactive systems incorporating computation-based topics in theo-
some form of machine intelligence. retical computer science. www.acm.org/pubs

PUBS_halfpage_Ad.indd 1 6/7/12 11:38 AM

JUly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i cat ion s o f t h e ac m 123
careers

University of South Carolina Upstate

Assistant Professor of Computer Science –
Requisition #004744

The Division of Mathematics and Computer Sci- Advertising in Career Opportunities

ence has a tenure-track Assistant Professor of
Computer Science position beginning 1/1/2013 How to Submit a Classified Line Ad: Send an e-mail to
to teach undergraduate courses in computer sci-
[email protected]. Please include text, and indicate the
issue/or issues where the ad will appear, and a contact name and number.
ence, develop curriculum, engage in scholarly ac-
tivities, advise students. Estimates: An insertion order will then be e-mailed back to you. The ad will
PhD in Computer Science or closely related by typeset according to CACM guidelines. NO PROOFS can be sent. Classified
field and excellent communication skills re- line ads are NOT commissionable.
quired. Prefer research interest in one of the fol- Rates: $325.00 for six lines of text, 40 characters per line. $32.50 for each
lowing areas: Networking and Computer Secu- additional line after the first six. The MINIMUM is six lines.
rity, Software Engineering, Database, Robotics.
Deadlines: 20th of the month/2 months prior to issue date. For latest deadline
Review of applications begins immediately
info, please contact: [email protected]
and continues until position is filled. For com-
plete information, requirements and online ap- Career Opportunities Online: Classified and recruitment display ads receive a
plication submission process, go to www.uscup- free duplicate listing on our website at: http://jobs.acm.org
state.edu/jobs. Contact: Dr. Jerome Lewis, Chair,
Ads are listed for a period of 30 days. For More Information Contact:
[email protected].
USC Upstate is an equal ACM Media Sales at 212-626-0686 or [email protected]
opportunity institution.

ISTFELLOW
IST AuSTrIA IS lookIng for PoSTDoCTorAl fElloWS
IST Austria (Institute of Science and Technology Austria) invites applications for
postdoctoral fellows in all fields of the natural and mathematical sciences and related
disciplines.
The Institute, which is on the outskirts of Vienna, was established by the Austrian government with
a focus on basic research. IST Austria has English as its working language, and its Graduate School
awards PhD degrees. It has an international mix of scientists chosen solely on the basis of their
individual excellence and potential contribution to research.
The Institute has set up a program for exceptional postdoctoral researchers partially funded by the
European Union. Appointments will be for 2–4 years. Applications will be accepted at any time, but
fellows will be selected twice a year in October and April. The deadlines for each selection are
the 15th of September and March, respectively. Applicants must have the support of one or more
Professors or Assistant Professors of IST Austria who will host them in their research group.
The Institute offers an internationally competitive salary, full social security coverage, and additional
benefits.

For further information about the program and the online application
process, please refer to the ISTFELLOW website: http://ist.ac.at/istfellow
IST Austria values diversity and is committed to equality. Female researchers are encouraged to apply.

124 c o m municatio ns o f th e ac m | j u ly 201 2 | vo l . 5 5 | no. 7

 THE UNIVERSITY OF TENNESSEE, KNOXVILLE
HEAD, DEPARTMENT OF ELECTRICAL ENGINEERING & COMPUTER SCIENCE
Applications and nominations are invited for the position of Professor and Head of the Department of Electrical Engineering and Computer
Science (EECS) at the University of Tennessee, Knoxville (UTK). EECS is housed in a new $37.5 million teaching and research facility completed
in 2012. The department currently has an enrollment of over 500 undergraduate and 220 graduate students, with a faculty of 40, and research
expenditures that exceed $12 million per year. The department was awarded an NSF Engineering Research Center in 2011 focusing on Wide
Area Control of Power Transmission and also houses the Center for Information Technology Research which is one of nine UTK Centers of
Excellence. The College of Engineering is a leading research institution with strong research partnerships with organizations such as the nearby
Oak Ridge National Laboratory.
Primary responsibilities of the department head are to provide visionary leadership; to encourage excellence and innovation in research,
teaching, and service; to advance professional development of faculty, staff and students; to promote productive relationships with all
constituents including students, parents, alumni, industry and government agencies; and to foster productive interdisciplinary relationships
with a variety of entities across the University community. The next department head will have the opportunity to critically shape the future of
the department through strategic hires and Governor’s Chairs in key areas.
Applicants must hold a doctorate degree in electrical engineering, computer engineering, computer science or other closely related field and be
eligible for appointment at the rank of full Professor. Commitment to and knowledge of affirmative action, equal employment opportunity and
diversity are required. The successful candidate must also have a balanced perspective on research and teaching, as well as the vision and
ability to lead a faculty representing a diversified range of interests. Demonstrated excellence in research, technical leadership, management,
and graduate and undergraduate teaching is preferred.
The Knoxville campus of the University of Tennessee is seeking candidates who have the ability to contribute in meaningful ways to the
diversity and intercultural goals of the University. The University welcomes and honors people of all races, genders, creeds, cultures, and
sexual orientations, and values intellectual curiosity, pursuit of knowledge, and academic freedom and integrity. Applications should be
submitted electronically following instructions found at http://jobs.eecs.utk.edu/DeptHead12. Applications should include 1) a curriculum vitae,
2) a statement that addresses both the applicant’s leadership experience/philosophy, and the applicant’s vision for the future of research and
education in the EECS Department, and 3) the full names, postal and email addresses, and telephone numbers of five references. Review of
applications will begin on July 1, 2012, and continue until the position is filled.
The University of Tennessee is an EEO/AA/Title VI/Title IX/Section 504/ADA/ADEA institution in the provision of its education and employment
programs and services. All qualified applicants will receive equal consideration for employment without regard to race, color, national origin,
religion, sex, pregnancy, marital status, sexual orientation, gender identity, age, physical or mental disability, or covered veteran status.
1981 CR Arrow Circle Ad2x4-F-crops.pdf 5/29/2012 10:08:38 AM

devastating capability, revolutionary advantage

Computing Reviews
C

M
is now at
Y
ComputingReviews.com
CM
We’re a global leader in cyber technologies seeking
professionals to join our team …
MY

CY
• kernel/linux developers
CMY • reverse engineers
K • pen testers

Our salaries and benefits are among the best in the

industry, including flexible hours, generous vacation,
and 100% paid premium health plans.

www.ainfosec.com/careers

A daily snapshot of what is hot and new.

eeo m/f/d/v

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i cat ion s o f t h e ac m 125

 tHE ACM A. M. turing AwArd
by the community ◆ from the community ◆ for the community

ACM, Intel, and Google congratulate

JUDEA PEARL
for fundamental contributions to artificial intelligence
through the development of a calculus for probabilistic
and causal reasoning.

“Dr. Pearl’s work provided the original paradigm “Judea Pearl is the most prominent advocate for proba-
case for how to do statistical AI. By placing bilistic models in artificial intelligence. He developed
structured knowledge representations at the heart mathematical tools to tackle complex problems that
of his work, and emphasizing how these represen- handle uncertainty. Before Pearl, AI systems had more
tations enabled efficient inference and learning, success in black and white domains like chess. But robot-
he showed the field of AI how to build statistical ics, self-driving cars, and speech recognition deal with
reasoning systems that were actually telling us uncertainty. Pearl enabled these applications to flourish,
something about intelligence, not just statistics.” and convinced the field to adopt these techniques.”

Limor Fix Alfred Spector

Director, University Collaborative Research Group Vice President, Research and Special Initiatives
Intel Labs Google Inc.

For more information see www.intel.com/research. For more information, see http://www.google.com/corporate/
index.html and http://research.google.com/.

Financial support for the ACM A. M. Turing Award is provided by Intel Corporation and Google Inc.
 CACM_TACCESS_one-third_page_vertical:Layo last byte

[c on t i n u e d fro m p. 128] Again the ping algorithm with this dating algo-
man walked past the department- rithm…”
store window. Slowed down, the gaze- “‘Mated’ is right!,” I muttered.
tracker showed his eye alight on a “C’mon c’mon, guys, iced coffee…”
hem here, the hip-tilt on a mannequin A minute or so apart, B and A en-
there, a neckline farther along. tered the same café. The screen fol-
“This is creepy,” said Mallory. lowed A in.
I jumped the view forward to where “This is just so unethical,” Mal- ACM
the guy was now. lory wittered on. “There’ll be privacy
“No,” I said. “It’s how I first spot-
ted this pattern, in the shopping-glass
issues, it’ll get hacked and become a
stalker’s paradise, it’ll freak people
Transactions on
logs. Men notice women’s clothes, just
not the way women do. Same with faces
out, especially women, you wait and
see, and what if people find out things
Accessible
and bodies; look…”
Subject A had passed the block of
about themselves, suppose they’re gay
and don’t know it, or don’t want to Computing
shops and was paying more attention know, and…”
to the people in the busy street. Again “We can stick a counseling mod-
I rolled back and forwarded slowly, this ule on the back,” I said, abstractedly,
time filtering out glances at inanimate following Subject A’s idle gaze as he
stuff—sidewalk, cars, street furni- queued. “Details, details… Don’t you
ture—leaving the looks at people. see? This is going to shape human
Subject A was straight. He reg- evolution! Just think, people’s real un-
istered other men with a sublimi- conscious attractions are often geneti-
nal check that they weren’t a threat. cally the most suitable for healthy off-
Women he noticed quite differently: spring, but they get overridden by all
He didn’t stare or lech, but he defi- sorts of extraneous…”
nitely checked them over. Now and Ah, yes. Subject B. There she was, at
again he’d give a woman a second, the till, paying. Subject A had noticed
conscious glance. What caught his her, all right. My next step would be a
eye could be the obvious—the sway discreet message on their glasses: “A
of a hip, say—but far more often it man/woman who might like you and
wasn’t. Rather, an ear, a drift of hair, a who you might like is nearby. Would
snazzy blouse, eyes, a cuff that set off you like to meet?” The usual Yes/No
a wrist… blink option. The Yes had to be mu-
“This guy is weird,” Mallory said. tual—ethics, see?
“He’s typical,” I said. “I’ve seen this My finger hovered over Send. ◆ ◆ ◆ ◆ ◆
scores of times. Men don’t even notice “Excuse me,” said a voice, above the This quarterly publication is a
what they notice about women. It’s clatter of cutlery. quarterly journal that publishes
more than they’ll say it is, that’s for A turned. The young woman beside
refereed articles addressing issues
sure.” I brushed my palms together. him in the queue looked quite unlike
“Now for Subject B.” subject B, and she didn’t have glasses. of computing as it impacts the
I clicked a search for a woman with- “What?” lives of people with disabilities.
in a 500-meter radius whose profile “It’s your turn.” Irritated… The journal will be of particular
matched A’s data-mined tastes and— “Oh, sorry.” I heard his smile. He interest to SIGACCESS members
crucially—whose own taste in men called for a tall iced coffee, just as I’d
nominally matched A. Several pos- expected. Then he looked back at the and delegrates to its affiliated
sibles popped up, with the street heav- woman. conference (i.e., ASSETS), as well
ing with shoppers, mostly young. Like “And for you… ?” as other international accessibility
A, all had clicked on the agreement to “Oh, I’ll…” She looked flustered, conferences.
be included in experiments with the then smiled. “A skim mocha, thanks.”
glasses, almost certainly unaware that Together, chatting, they headed for ◆ ◆ ◆ ◆ ◆
that was what they were doing. a table. www.acm.org/taccess
I picked the nearest: about 100 “Oh bugger.”
meters from A, walking toward him. I
www.acm.org/subscribe
flashed to A’s and B’s glasses an ad for Ken MacLeod ([email protected]) is the author
of 13 novels, from The Star Fraction (Orbit Books, London,
cool drinks, flagging-up the salience of 1995) to Intrusion (Orbit Books, London, 2012) and blogs
a Caffè Nero. at The Early Days of a Better Nation (http://kenmacleod.
blogspot.com).
Mallory watched, horrified.
“This is sick,” he said. “I see what
you’ve done; you’ve mated the shop- © 2012 ACM 0001-0782/12/07 $15.00

ju ly 2 0 1 2 | vo l . 55 | n o. 7 | c om m u n i cat ion s o f t h e acm 127

last byte

Future Tense, one of the revolving features on this page, presents stories and
essays from the intersection of computational science and technological speculation,
their boundaries limited only by our ability to imagine what could be.

DOI:10.1145/2209249.2209275 Ken MacLeod

Future Tense
They Just Click
When glasses track glances, will eyes
still meet across a crowded room?

“ P e o ple d o n ’ t re a d end-user license

agreements,” said Mallory, unhappily.
“They just click.”
“I know.” I fired up the demo for its
first walk-through. “Shrink-wrap’s wa-
tertight.”
The screen lit up with the view from
Subject A’s glasses: a typical English
Saturday suburban High Street.
“See?,” I said. “Anonymized. Ethical.”
I knew Subject A was male. I didn’t
know his name, age, location, or ap-
pearance. The system did.
On the screen the gaze-tracker
flicked about like a jittery cursor. Every
millisecond moment when something
snagged the gaze was logged, too fast
to read except when he deliberately
looked. Right now: running shoes in a
sports-shop window.
The view swung as he turned and
walked on. He passed a department-
store display of the latest summer
dresses without a glance. The next
shop was stationery. Even in real time,
you could see his gaze linger on a nifty
gift-pen cluster.
“So far, so shopping-glass,” said
Mallory, sounding skeptical rather
than dubious.
“Roll back,” I told the system. “Now
forward, slow.” [ c o ntin u e d on p. 127]
Photogra ph by NemesisINC/ sh utt ersto ck. com

Mallory watched,
horrified.

128 c o mm unicatio ns o f t h e ac m | j u ly 201 2 | vol . 5 5 | no. 7