CACM.ACM.ORG 12/2020 VOL.63 NO.

12

Silicon Politics
Measuring Internet Speed
XNOR-Networks Association for
Computing Machinery
The Dark Triad and Insider Threats in Cyber Security
Dr. Jerald has recognized a
great need in our community
and filled it. The VR Book is a
scholarly and comprehensive
treatment of the user interface
dynamics surrounding the
development and application
of virtual reality. I have
made it required reading for
my students and research
colleagues. Well done!”

- Prof. Tom Furness, University

of Washington, VR Pioneer
 / AARHUS 2021 / JULY 12-16
THE 35TH EDITION OF ECOOP https://2021.ecoop.org

ECOOP is Europe’s longest-standing annual Programming GENERAL CHAIR

Languages conference, bringing together researchers, practitioners, Anders Møller, Aarhus University Julia Lawall
and students to share their ideas and experiences in all topics Mira Mezini
related to programming languages, software development, object- PROGRAM CHAIR Todd Mytkowicz
oriented technologies, systems and applications. It welcomes high Manu Sridharan, UC Riverside Robert O’Callahan
quality research papers relating to these fields in a broad sense; Hakjoo Oh
solicits both innovative and creative solutions to real problems as PROGRAM COMMITTEE Uday P. Khedker
well as evaluations of existing solutions—evaluations that provide Jonathan Aldrich Hila Peleg
new insights; encourages the submission of reproduction studies Walter Binder Fernando M. Q. Pereira
Eric Bodden Michael Pradel
AREAS OF INTEREST Camil Demetrescu Sukyoung Ryu
Design, implementation, optimization, analysis, and theory Werner Dietl Alexandra Silva
of programs, programming languages, and programming Jens Dietrich Alexander J. Summers
environments. George Fourtounis Sam Tobin-Hochstadt
Colin Gordon Omer Tripp
PUBLICATION Michael Greenberg Eelco Visser
Affordable cc-by open access in Dagstuhl LIPIcs David Grove John Wickerson
or as journal first in ACM TOPLAS or Elsevier SCP. Murali Krishna Ramanathan Lingming Zhang
Burcu Kulahcioglu Ozkan Lu Zhang
Viktor Kunčak Elena Zucca

In-Cooperation
COMMUNICATIONS OF THE ACM

Departments News Viewpoints

5 Cerf’s Up 18 Computing Ethics

Repairability Redux Operationalizing AI Ethics Principles
By Vinton G. Cerf A better ethics analysis guide
for developers.
6 BLOG@CACM By Cansu Canca
Pitting Computers
Against Each Other … in Chess 22 Broadening Participation
Guest blogger Monroe Newborn U.S. States Must Broaden
on the 50th anniversary of Participation While
ACM computer chess tournaments. Expanding Access to
Computer Science Education
100 Careers Incorporating equity and
inclusion in the effort
toward access for everyone.
12
Last Byte By W. Richards (Rick) Adrion,
Sarah Dunton, Barbara Ericson,
104 Future Tense 9 Tracking COVID, Discreetly Renee Fall, Carol Fletcher,
Walden Three Tracing the contacts of those and Mark Guzdial
Can humanity take its next step who come into contact with the
forward by taking a step back? coronavirus is not that simple. 26 The Profession of IT
By William Sims Bainbridge By Neil Savage Navigating in Real-Time
Environments
12 Softening Up Robots An interview with Jim Selman.
Giving robots soft, artificial skin By Peter J. Denning
would enable them to work more
closely with people. 29 Kode Vicious
By Esther Shein Removing Kode
Dead functions and dead features.
15 Technologies for By George V. Neville-Neil
the Visually Impaired
The last decade has seen major 30 Viewpoint
advancements in technology Silicon Politics
for the blind and visually impaired, Tracing the widening path between
but problems remain. Silicon Valley and Washington, D.C.
By Logan Kugler By Margaret O’Mara

33 Viewpoint
Federated Learning
for Privacy-Preserving AI
Engineering and algorithmic
IMAGE COURTESY OF UNIVERSIT Y OF T EXAS AT ARLING TON

framework to ensure
data privacy and user confidentiality.
By Yong Cheng, Yang Liu,
Tianjian Chen, and Qiang Yang

Association for Computing Machinery

Advancing Computing as a Science & Profession

2 COMMUNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 12/2020 VOL. 63 NO. 12

Practice Contributed Articles Review Articles

72 Measuring Internet Speed—

Current Challenges
and Future Recommendations
Speed testing methods have
flourished over the last decade,
but none without at least
some limitations.
By Nick Feamster and Jason Livingood

Watch the authors discuss

this work in the exclusive
Communications video.
https://cacm.acm.org/
videos/measuring-internet-
speed
38 54

Research Highlights
38 The Life of a Data Byte 54 Green AI
Be kind and rewind. Creating efficiency in AI research 82 Technical Perspective
By Jessie Frazelle will decrease its carbon footprint XNOR-Networks—
and increase its inclusivity as deep Powerful but Tricky
46 Security Analysis of SMS as learning study should not require By David Alexander Forsyth
a Second Factor of Authentication the deepest pockets.
The challenges of multifactor By Roy Schwartz, Jesse Dodge, 83 Enabling AI at the Edge
authentication based on SMS, Noah A. Smith, and Oren Etzioni with XNOR-Networks
including cellular security By Mohammad Rastegari,
deficiencies, SS7 exploits, Vicente Ordonez, Joseph Redmon,
and SIM swapping. Watch the authors discuss
and Ali Farhadi
By Roger Piqueras Jover this work in the exclusive
Communications video.
https://cacm.acm.org/ 91 Technical Perspective
Articles’ development led by videos/green-ai The Future of Large-Scale
queue.acm.org
Embedded Sensing
64 The Dark Triad and Insider Threats By Joseph A. Paradiso
in Cyber Security
Tracing the relationship between 92 SATURN: An Introduction
pathological personality traits to the Internet of Materials
and insider cyber sabotage. By Nivedita Arora, Thad Starner,
By Michele Maasberg, Craig Van Slyke, and Gregory D. Abowd
LEF T IMAGE BY ALESA NKO RODRIGUEZ; RIGH T IMAGE BY LISA SH EEH AN

Selwyn Ellis, and Nicole Beebe

About the Cover:

Computational power for
deep learning research
takes deep pockets
and leaves a big carbon
footprint. This month’s
cover story (p. 54)
advocates a practical
solution for cutting both.
Cover illustration by
Lisa Sheehan.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF THE ACM 3

 COMMUNICATIONS OF THE ACM
Trusted insights for computing’s leading professionals.

Communications of the ACM is the leading monthly print and online magazine for the computing and information technology fields.
Communications is recognized as the most trusted and knowledgeable source of industry information for today’s computing professional.
Communications brings its readership in-depth coverage of emerging areas of computer science, new trends in information technology,
and practical applications. Industry leaders use Communications as a platform to present and debate various technology implications,
public policies, engineering challenges, and market trends. The prestige and unmatched reputation that Communications of the ACM
enjoys today is built upon a 50-year commitment to high-quality editorial content and a steadfast dedication to advancing the arts,
sciences, and applications of information technology.

ACM, the world’s largest educational STA F F EDITORIAL BOARD ACM Copyright Notice
and scientific computing society, delivers DIRECTOR OF PU BL ICATIONS E DITOR- IN- C HIE F Copyright © 2020 by Association for
resources that advance computing as a Scott E. Delman Andrew A. Chien Computing Machinery, Inc. (ACM).
science and profession. ACM provides the [email protected] [email protected] Permission to make digital or hard copies
computing field’s premier Digital Library Deputy to the Editor-in-Chief of part or all of this work for personal
and serves its members and the computing Executive Editor Morgan Denlow or classroom use is granted without
profession with leading-edge publications, Diane Crawford [email protected] fee provided that copies are not made
conferences, and career resources. Managing Editor S E NIOR E DITOR or distributed for profit or commercial
Thomas E. Lambert Moshe Y. Vardi advantage and that copies bear this
Executive Director and CEO Senior Editor notice and full citation on the first
Vicki L. Hanson Andrew Rosenbloom NE W S page. Copyright for components of this
Deputy Executive Director and COO Senior Editor/News Co-Chairs work owned by others than ACM must
Patricia Ryan Lawrence M. Fisher Marc Snir and Alain Chesnais be honored. Abstracting with credit is
Director, Office of Information Systems Web Editor Board Members permitted. To copy otherwise, to republish,
Wayne Graves David Roman Tom Conte; Monica Divitini; Mei Kobayashi; to post on servers, or to redistribute to
Director, Office of Financial Services Editorial Assistant Rajeev Rastogi; François Sillion lists, requires prior specific permission
Darren Ramdin Danbi Yu and/or fee. Request permission to publish
Director, Office of SIG Services VIE W P OINTS from [email protected] or fax
Donna Cappo Art Director Co-Chairs (212) 869-0481.
Director, Office of Publications Andrij Borys Tim Finin; Susanne E. Hambrusch;
Scott E. Delman Associate Art Director John Leslie King For other copying of articles that carry a
Margaret Gray Board Members code at the bottom of the first or last page
Assistant Art Director Virgilio Almeida; Terry Benzel; Michael L. Best; or screen display, copying is permitted
ACM CO U N C I L
Mia Angelica Balaquiot Judith Bishop; Lorrie Cranor; Boi Falting; provided that the per-copy fee indicated
President
Production Manager James Grimmelmann; Mark Guzdial; in the code is paid through the Copyright
Gabriele Kotsis
Bernadette Shade Haym B. Hirsch; Anupam Joshi; Richard Ladner; Clearance Center; www.copyright.com.
Vice-President
Intellectual Property Rights Coordinator Carl Landwehr; Beng Chin Ooi; Francesca Rossi;
Joan Feigenbaum
Barbara Ryan Len Shustek; Loren Terveen; Marshall Van Subscriptions
Secretary/Treasurer
Advertising Sales Account Manager Alstyne; Jeannette Wing; Susan J. Winter An annual subscription cost is included
Elisa Bertino
Ilia Rodriguez in ACM member dues of $99 ($40 of
Past President
Cherri M. Pancake P R AC TIC E which is allocated to a subscription to
Chair, SGB Board Columnists Co-Chairs Communications); for students, cost
Jeff Jortner David Anderson; Michael Cusumano; Stephen Bourne and Theo Schlossnagle is included in $42 dues ($20 of which
Co-Chairs, Publications Board Peter J. Denning; Mark Guzdial; Board Members is allocated to a Communications
Jack Davidson and Joseph Konstan Thomas Haigh; Leah Hoffmann; Mari Sako; Eric Allman; Samy Bahra; Peter Bailis; subscription). A nonmember annual
Members-at-Large Pamela Samuelson; Marshall Van Alstyne Betsy Beyer; Terry Coatta; Stuart Feldman; subscription is $269.
Nancy M. Amato; Tom Crick; Nicole Forsgren; Camille Fournier;
Susan Dumais; Mehran Sahami; C O N TAC T P O IN TS Jessie Frazelle; Benjamin Fried; Tom Killalea; ACM Media Advertising Policy
Alejandro Saucedo Copyright permission Tom Limoncelli; Kate Matsudaira; Communications of the ACM and other
SGB Council Representatives [email protected] Marshall Kirk McKusick; Erik Meijer; ACM Media publications accept advertising
Sarita Adve and Jeanna Neefe Matthews Calendar items George Neville-Neil; Jim Waldo; in both print and electronic formats. All
[email protected] Meredith Whittaker advertising in ACM Media publications is
BOARD C HA I R S Change of address at the discretion of ACM and is intended
[email protected] C ONTR IB U TE D A RTIC LES to provide financial support for the various
Education Board Co-Chairs
Letters to the Editor activities and services for ACM members.
Mehran Sahami and Jane Chu Prey James Larus and Gail Murphy
[email protected] Current advertising rates can be found
Practitioners Board Board Members by visiting http://www.acm-media.org or
Terry Coatta Robert Austin; Kim Bruce; Alan Bundy;
W E B S IT E by contacting ACM Media Sales at
http://cacm.acm.org Peter Buneman; Premkumar T. Devanbu; (212) 626-0686.
REGIONA L C O U N C I L C HA I R S Jane Cleland-Huang; Yannis Ioannidis;
ACM Europe Council WEB BOARD Trent Jaeger; Somesh Jha; Gal A. Kaminka; Single Copies
Chris Hankin Chair Ben C. Lee; Igor Markov; m.c. schraefel; Single copies of Communications of the
ACM India Council James Landay Hannes Werthner; Reinhard Wilhelm; ACM are available for purchase. Please
Abhiram Ranade Board Members Rich Wolski contact [email protected].
ACM China Council Marti Hearst; Jason I. Hong;
Wenguang Chen Jeff Johnson; Wendy E. MacKay RES E A R C H HIGHLIGHTS COMMUN ICATION S OF THE ACM
Co-Chairs (ISSN 0001-0782) is published monthly
PUB LICATI O N S BOA R D AU T H O R G U ID E L IN ES Shriram Krishnamurthi by ACM Media, 1601 Broadway, 10th Floor
Co-Chairs http://cacm.acm.org/about- and Orna Kupferman New York, NY 10019-7434 USA. Periodicals
Jack Davidson and Joseph Konstan communications/author-center Board Members postage paid at New York, NY 10001,
Board Members Martin Abadi; Amr El Abbadi; and other mailing offices.
Jonathan Aldrich; Phoebe Ayers; Animashree Anandkumar; Sanjeev Arora;
ACM ADVERTISIN G DEPARTM E NT
Chris Hankin; Mike Heroux; James Larus; Michael Backes; Maria-Florina Balcan; POSTMASTER
1601 Broadway, 10th Floor
Tulika Mitra; Marc Najork; Azer Bestavros; David Brooks; Stuart K. Card; Please send address changes to
New York, NY 10019-7434 USA
Michael L. Nelson; Theo Schlossnagle; Jon Crowcroft; Lieven Eeckhout; Communications of the ACM
T (212) 626-0686
Eugene H. Spafford; Divesh Srivastava; Alexei Efros; Bryan Ford; Alon Halevy; 1601 Broadway, 10th Floor
F (212) 869-0481
Bhavani Thuraisin; Robert Walker; Gernot Heiser; Takeo Igarashi; New York, NY 10019-7434 USA
Julie R. Williamson Advertising Sales Account Manager Srinivasan Keshav; Sven Koenig;
Ilia Rodriguez Ran Libeskind-Hadas; Karen Liu;
ACM U.S. Technology Policy Office [email protected] Tim Roughgarden; Guy Steele, Jr.; Printed in the USA.
Adam Eisgrau Robert Williamson; Margaret H. Wright;
Director of Global Policy and Public Affairs Media Kit [email protected] Nicholai Zeldovich; Andreas Zeller
1701 Pennsylvania Ave NW, Suite 200,
Washington, DC 20006 USA S P EC IA L S EC TIONS
T (202) 580-6555; [email protected] Association for Computing Machinery Co-Chairs
(ACM) Sriram Rajamani, Jakob Rehof, and Haibo Chen
Computer Science Teachers Association 1601 Broadway, 10th Floor Board Members A
SE
REC
Y

Jake Baskin New York, NY 10019-7434 USA Sue Moon; PJ Narayama; Tao Xie;
E

CL
PL

Executive Director T (212) 869-7440; F (212) 869-0481 Kenjiro Taura; David Padua
NE
TH

S
I

Z
I

M AGA

4 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 cerf’s up

DOI:10.1145/3429267 Vinton G. Cerf

Repairability Redux
I wrote about repairability in the February
2020 issue of Communications (p. 7) and
here I am at year’s end harping on the same
topic. My excuse is COVID-19. I have been
at home for the past six months while didn’t they provide diameter specifi- drill bit—#108—set for 6000 RPM.
my normal schedule would have had cations? Well, the original parts are Holey Moley! The drill bit got yanked
me on the road three weeks out of 22 years old and I guess they changed out of the chuck! My engineer friend
four. Of course, like many of you, I sizes somewhere in between. Maybe says, “Brass grabs like that.” OK, new
have been all over the world—virtu- I can sand down the .29-inch tube … plan. Let’s get a much smaller drill
ally—since mid-March. There are days Nope, can’t do that without making bit and program the machine to ream
when I can visit Australia and Austria the plastic tube too brittle. Time to out a larger hole by making a circular
and be home in time for dinner. So, call my “go-to engineer” friend who traverse multiple times, drilling deep-
what does that have to do with stuff has a $30,000 Computer Numerical er on each traverse. Sonfagun, that
that breaks? Mostly, I am actually here Control (CNC) machine that can pret- works! Habemus soap dispenser!
when it does. Under more normal ty much mill anything. In the last several weeks I have en-
conditions, my wife would have to call This CNC thing has a bazillion in- countered several similar problems
a repair person to fix or replace a bro- terchangeable drills and other gad- with “replacement parts” that don’t
ken item. Now that I am home, I am gets for making measurements accu- quite fit the <ancient> piece of equip-
sometimes the one who discovers the rate to a ten thousandth of an inch. ment I am trying to repair. I am begin-
problem, or I am told about it before The machine is programmable to ning to appreciate Cuban ingenuity.
a repair service gets the call. I am an carry out complex milling operations Have you seen all those cars from the
engineer of sorts, so broken things on a variety of materials including my 1950s in Havana? I think manufactur-
attract my attention. Engineers love chrome-plated brass soap dispenser ers today should take lessons from
problems to solve. “Fix me! Fix me! fixture. Measuring carefully, we are the LEGO company. They have pro-
You can do it!” Of course, if you are going to drill out the .25-inch hole un- duced interchangeable LEGO parts
like me, you go to the hardware store til it is .29 inches in diameter so the since 1932. If you are going to make
three times: first to get the stuff you new siphon tube will fit snugly into products that are intended to last for
need, second to get the stuff you for- the enlarged hole. Here’s the right decades, you should maintain spare-
got, and third to get the stuff you need part compatibility for the lifetime of
to fix what you broke. My basic rant is the product. That’s what standards
that manufactured goods today do not My basic rant are for. Maybe 3D printing is a par-
seem to take into account the possibil- tial solution for some products since
ity of repair. is that manufactured printing the part on demand might
Case in point: a broken soap dis- goods today be less expensive than maintaining
penser. Expensive, shiny chrome inventory. I think my basic complaint
built-in kitchen soap dispenser whose do not seem is that if something is advertised as a
<insert adjectives> spring gave out. to take into account replacement part, it should really be a
OK, no problem. Look up replacement replacement part that fits. Well, there
parts on the Internet. Found it! Only the possibility is always eBay, I suppose. Thanks for
$4.79 too. I ordered the part and it ar- of repair. listening.
rived in the mail a few days later. OK,
just pull out the old one and slip in Vinton G. Cerf is vice president and Chief Internet Evangelist
at Google. He served as ACM president from 2012–2014.
the new one … Hmm. The new siphon
has a diameter of .29 inches. The hole
is .25 inches. <Many bad words>. Why Copyright held by author.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF THE ACM 5

 The Communications Web site, http://cacm.acm.org,
features more than a dozen bloggers in the BLOG@CACM
community. In each issue of Communications, we’ll publish
selected posts or excerpts.

Follow us on Twitter at http://twitter.com/blogCACM

DOI:10.1145/3427786 http://cacm.acm.org/blogs/blog-cacm

Pitting Computers person, the former will be higher-rated

and can be considered more intelligent

Against Each Other …

than the latter at playing chess. That, in
part, was the motivation for using
chess as a barometer of intelligence.

in Chess There is also no luck involved, as with

many other games.
This brings us back to the game of
Guest blogger Monroe Newborn on the 50th anniversary chess and, in particular, ACM’s role in it.
of ACM computer chess tournaments. In 1970, ACM’s Annual Conference took
place at the New York Hilton. Ken King,
head of Columbia University’s comput-
Monroe Newborn those stars may now and/or in the past er center, served as co-chairman with
The Laughing is Over have supported some sort of intelligent me of the conference’s Special Events
September 18, 2020 life. I suspect there might be creatures program. About the same time, Tony
https://bit.ly/35Nc1SS far more advanced than Earthlings. Marsland, a Bell Telephone Laborato-
For those of us involved One of the great disappointments of my ries researcher, approached me suggest-
in programming com- life is that we have not made greater ef- ing a demonstration of his chess-playing
puters to play chess, it has been a great forts to explore the universe, especially computer at the conference as a special
adventure. ACM annual tournaments our Moon and Mars. event. We met, discussed this, and con-
began in 1970 (50 years ago!) and were Historically, there have been argu- cluded we could put together something
hosted year after year for a quarter- ments supporting the belief the ani- more exciting: the first chess tourna-
century by the organization. They were mals of this planet do not possess the ment exclusively for computers. It was
terrific catalysts for progress in the mental capabilities of humans. They named the U.S. Computer Chess Cham-
field, and deserve major credit for the were not thought to be able to use tools pionship. In addition, we included a
eventual 1997 defeat of then-World to solve problems, and they were not computer music festival and a computer
Champion Garry Kasparov. thought to be able to recognize them- art festival in the program. These consti-
I feel human intelligence has been selves in a mirror. More and more, we tuted our special events. I have forgotten
vastly overrated. We humans haven’t find animals are more intelligent than how the music and art festivals turned
learned how not to fight wars over vari- they have been given credit for. out. I do recall Charles Dodge’s music
ous explanations of how the universe Researchers in the field decided in involved transforming the intensity of
or man came into being. The telescope the 1940s that chess was the task over energy from the sun into musical notes,
that allows us to peer into the universe which human intelligence might be and the resulting music’s randomness
at 1,000,000,000,000,000,000,000 stars studied. The definition of intelligence, was surprisingly delightful.
is only 400 years old. Women were of course, is problematic. How do we Marsland and I rounded up six con-
burned alive as witches only 300 years say someone is more intelligent than testants for the chess tournament.
ago. With the universe in existence for someone else? Chess provides an envi- Hans Berliner, who had been the
14.7 billion years, the Earth in existence ronment to study this. Ratings are as- world correspondence chess champion,
for 4.6 billion years, and humans in ex- signed to chess players based on their was a doctoral student at Carnegie Mel-
istence for several million years, there performance against opposition. If one lon University at that time; he entered
is an excellent chance at least one of person consistently defeats another his program J. Biit. Ken King arranged

6 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 blog@cacm

for it to run on Columbia University’s That was the situation when the first theory; most prominently, one in the
powerful IBM 360/91 during the compe- ACM U.S. Computer Chess Champion- January 1970 issue of IEEE Transactions
tition. Berliner’s program entered the ship was held. on Computers entitled Iteratively Real-
tournament as the favorite. The format of the competition was ized Sequential Circuits.1)
Northwestern University students a three-round Swiss-style tourna- A half-century later, chess programs
David Slate, Larry Atkin, and Keith Gor- ment beginning Aug. 31, 1970, and are so much better than top humans
len entered their program CHESS 3.0. It ending Sept. 2, 1970. Entries had two that there is no contest. The top players
ran on their university’s CDC 6400, a hours to make their first 40 moves, use computers to help them learn to play
powerful machine, but not in the same then 30 minutes to make each suc- better! The laughing has ended.
class as Columbia’s IBM 360/91. cessive 10 moves. Bugs cropped up Computers are, in round numbers,
Marsland entered his program, The as the competition went on, with the 1,000 times faster than in 1970, and
Marsland CP; it ran on a Burrough’s most dramatic seen in the early their memories are clearly more than
B5500 located in Burrough’s New York moves of the round 1 game between 1,000 times larger. Imagine driving a
City sales office. The Marsland CP (White) and J. Biit car that goes 1,000 times faster than
Running on IBM 360/65s were two (Black). Marsland’s program made the one you currently drive; a 10-mile
other entries: COKO III, developed by the worst possible 8th and 9th moves, drive to work at 60 mph would take 10
Dennis Cooper and Ed Kozdrowicki at leading to a quick victory for J. Biit and minutes, while at 60,000 mph it would
Bell Telephone Laboratories’ Whippany laughter from the audience of com- take less than a second. Chess pro-
facilities, and SCHACH, developed at puter and chess experts. Attendees grams require, for each additional level
Texas A&M by Franklin Ceruti and Rolf Grandmaster Pal Benko and Interna- of search, approximately four times the
Smith, U.S. Air Force captains at the time. tional Master Al Horowitz may have amount of time; a speedup of 1,000 al-
Lastly, Chris Daly, working with Ken been among those laughing. lows computers to search about five
King (not Columbia’s Ken King), brought In the next round, J. Biit was defeat- levels deeper (4x4x4x4x4 = 1024). On
their computer, an IDIOM system based ed by the Slate/Atkin/Gorlen program, top of faster computers and much larg-
on a Varian 620/i processor, to the site. with the audience cheering CHESS er memories, there have been many
Three entrants used terminals con- 3.0’s 47th move, a short-term sacrifice software improvements and even dif-
nected to their remote computers, two leading to an easier, shorter path to vic- ferent approaches all together, such as
others spoke by telephone to a human tory. Quite unlike a human tourna- the use of Monte Carlo search.
operator at their computers’ sites, and ment, the audience was very vocal as Over the years, ACM headquarters
one was at the site. the games progressed, cheering and supported the yearly tournaments. I
Jacques Dutka, a mathematician laughing. Also unlike human tourna- would like to single out two individuals
known for calculating the square root of ments, programmers would get togeth- in particular: Jim Adams and Joe DiBla-
2 to a million decimal digits, served as er for coffee after the games ended, to si. In addition, Drexel University profes-
tournament director. discuss the day. A close community of sor Frank Friedman provided support,
Missing from the competition was programmers developed over the years. as did Ben Mittman, head of Northwest-
Mac Hack, developed at MIT by Richard CHESS 3.0 went on to win the tour- ern University’s Vogelback Computer
Greenblatt, established leading up to nament, winning all three games. It Center. Lastly, British International
the tournament as the strongest chess- dominated the field for a decade, until Chess Master David Levy helped, espe-
playing program. It had competed in a Ken Thompson’s BELLE arrived in the cially in setting up the 1996 Kasparov
number of human tournaments and late 1970s with special chess hard- versus DEEP BLUE match.
was rated around 1600, the level of a ware. BELLE stayed on top until Bob
good high school player. Hyatt’s CRAY BLITZ and Hans Berlin- Reference
1. Arnold, T.F., Tan, C.J., and Newborn, M. Iteratively
So began a very significant, long- er’s HITECH caught up in the middle realized sequential circuits. IEEE Trans on Computers
lasting experiment. Could a computer 1980s. In the late 1980s and early (Jan. 1970), 54–66.

be designed to exhibit the intelligence 1990s, IBM’s Deep Blue, developed by

My books, one with David Levy, chronicle the great
of a chess expert, master, grandmas- Feng-Hsiung Hsu with major help progress computers have made at chess and ACM’s
ter, or world champion? Could one be from Murray Campbell, Joe Hoane, involvement in it:
designed to match the chess intelli- and Jerry Brody, with Chung-Jen Tan Computer Chess, Academic Press, New York, 1975.
gence of the top human mind? What serving as boss, rose to the top of the How Computers Play Chess, with David Levy, W. H.
would it take? How long would it take? pack. Deep Blue, playing grandmas- Freeman, 1991.

In 1970 there were no cellphones, no ter-level chess, went on to defeat Garry Kasparov versus DEEP BLUE: Computer Chess Comes of
Age, Springer-Verlag, 1997.
email, no drones, no self-driving cars, Kasparov in their classic 1997 match,
Deep Blue: An Artificial Intelligence Milestone, Springer, 2002.
no Siri. Yet, the computer revolution a landmark in the world of artificial in-
Beyond Deep Blue: Chess in the Stratosphere, Springer, 2011.
was heating up! telligence.
Grandmasters were generally in de- (Tan and I go back to the late 1960s,
Monroe “Monty” Newborn, formerly chairman of ACM’s
nial in 1970. Some contended good when he was a doctoral student at Co- computer chess committee, was a professor of electrical
chess players used intuition when play- lumbia University and I was a young pro- engineering in Columbia University, and later a professor
of computer science in McGill University, where he is
ing chess, and intuition could not be fessor there. We lived in the same apart- currently a Professor Emeritus.
programmed. The programs were the ment building. We published several
laughingstocks of the top chess players. papers together in the field of automata © 2020 ACM 0001-0782/20/12 $15.00

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF THE ACM 7

ACM Transactions on
Computing for Healthcare (HEALTH)
A multidisciplinary journal for
Open for high-quality original work on how
Submissions computing is improving healthcare

Computing for Healthcare has emerged as an important and

growing research area. By using smart devices, the Internet
of Things for health, mobile computing, machine learning,
cloud computing and other computing based technologies,
computing for healthcare can improve the effectiveness,
efficiency, privacy, safety, and security of healthcare (e.g.,
personalized healthcare, preventive healthcare, ICU without
walls, and home hospitals).

ACM Transactions on Computing for Healthcare (HEALTH)

is the premier journal for the publication of high-quality
original research papers, survey papers, and challenge
papers that have scientific and technological results
pertaining to how computing is improving healthcare. This
journal is multidisciplinary, intersecting CS, ECE, mechanical
engineering, bio-medical engineering, behavioral and
social science, psychology, and the health field, in general.
All submissions must show evidence of their contributions
For more to the computing field as informed by healthcare. We
do not publish papers on large pilot studies, diseases, or
information other medical assessments/results that do not have novel
and to submit computing research results. Datasets and other artifacts
needed to support reproducibility of results are highly
your work, encouraged. Proposals for special issues are encouraged.
please visit:

health.acm.org
N
news

Science | DOI:10.1145/3427933 Neil Savage

Tracking COVID,
Discreetly
Tracing the contacts of those who come into
contact with the coronavirus is not that simple.

A
S T H E WORLD continues to
grapple with the coronavirus
pandemic, health officials
are relying on a tried-and-
true method of limiting the
spread of the potentially deadly disease:
contact tracing. Figuring out who has
been close enough to an infected person
long enough to catch the disease, then
taking steps to prevent those people
from passing it to others, is a method
that dates back to the 1920s, when health
authorities used it to rein in the spread
of syphilis. In the era of smartphones, it
seems only natural to add a technologi-
cal dimension to contact tracing.
Using smartphone apps for con-
tact tracing raises questions, though.
For one thing, it is not entirely clear
how effective that is; the answer de-
pends on both how well a smartphone
can measure contacts and on how
many people actually decide to use the
IMAGE BY RICH A RD M LEE/ SH UTT ERSTO CK .COM

apps. Perhaps the chief concern,

though, is privacy. How do you design privacy. A report in June from Amnesty lance tools which go far beyond what
a system that identifies who has been International warned governments’ is justified in efforts to tackle COV-
in contact with whom without giving collection and storage of too much in- ID-19,” said Claudio Guarnieri, head
all sorts of personal information to formation about individuals posed a of Amnesty International’s Security
governments or data thieves that significant threat, especially in some Lab, in a statement that accompanied
might abuse it? countries. “Bahrain, Kuwait, and Nor- those findings. The Norwegian gov-
Civil libertarians have raised way have run roughshod over people’s ernment, also facing pressure from
alarms about potential invasions of privacy, with highly invasive surveil- the European Data Protection Author-

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF THE ACM 9

news

ity, suspended use of its app, called by not collecting such data in the first
Smittestopp. place. “A solution should not be about
Problems with these apps included An app avoids the risk confidentiality or anonymity. It should
identifiers that could be tied to people. of someone, whether be about privacy, in which even the
Smittestopp made users register with company doesn’t know who you are,”
their telephone number, for instance, the government says Ramesh Raskar, professor in the
while Bahrain, Kuwait, and Qatar re- or a hacker, getting Massachusetts Institute of Technolo-
quired use of a national identification gy’s Media Lab who launched the
number. Early apps often had location- their hands on PathCheck Foundation. The non-prof-
based tracking, which can make it pos- personal data by not it has developed an app based on the
sible to re-identify anonymized users Apple-Google framework to track CO-
and keep tabs on potentially sensitive collecting such data VID-19 cases using “privacy by compu-
information such as who someone was in the first place. tation.” Data can only be seen by
visiting or whether they were taking someone with physical access to the
part in protests. Several countries, in- phone handset, and even then the
cluding France, Iceland, and Singa- amount of useful information that
pore, had centralized storage of the could be obtained would be limited,
data, where access to it would be out of Raskar says.
an individual’s control. stored on the user’s phone and auto-
matically deleted after a specified pe- Location, Location, Location
The Tech Giants riod of time. Last March, Robert Kleinman, a psy-
Over the summer, however, govern- If someone using the app tests posi- chiatrist then at Stanford University
ments adopting apps moved to a dif- tive for COVID-19, they enter that into and now at Massachusetts General
ferent model, and many now rely on the app, which then uploads all stored Hospital, decided to combine his inter-
the Exposure Notification System contacts to a master list on cloud stor- ests in geospatial data and access to
specification jointly developed by Ap- age platforms. Another user’s app peri- healthcare. He and software engineer
ple and Google. In that system, indi- odically checks that master list, and if Colin Merkel designed a prototype
vidual phones generate random num- it finds its own key, it notifies the user tracking app that used GPS location
bers, which change every few minutes, that she should get tested or self-quar- data to identify where exposures took
and share them with nearby phones antine. Once the incubation period has place. They eventually scrapped it, as
over Bluetooth. The Bluetooth signal passed, keys are deleted from the mas- did most other developers who focused
can be used to estimate the distance ter list. The system specifically bars on GPS early on.
between phones and the length of the apps from collecting GPS data. The problem, aside from possible
contact—being within six feet of An app avoids the risk of someone, public discomfort with having one’s lo-
someone for 15 minutes is generally whether the government or a hacker, cation tracked, is that GPS location is
defined as a contact. The data is getting their hands on personal data not precise enough for contact tracing.

Milestones
ACM Releases Study of U.S. Bachelor’s Programs in Computing
ACM recently released its eighth of the state of enrollment such as computer engineering ˲ Of 40 non-doctoral-granting
annual Study of Non-Doctoral and graduation in bachelor’s (12.2% of degree earners in the departments reporting over 56
Granting Departments in programs. 2018/2019 academic year). faculty departures, only 10.7% of
Computing (NDC), with the aim The study also includes ˲ Bachelor’s programs had a faculty departed for non-academ-
of providing a comprehensive data from private, for-profit stronger representation of Afri- ic positions. Most departed due
look at computing education. institutions. can American and Hispanic stu- to retirement (46.4%) or other ac-
This year’s ACM NDC Important findings of the dents than Ph.D. programs, as ademic positions (26.9%).
study includes enrollment study include: recorded by the Computer Re- The study was authored
and degree completion data ˲ Between the 2017/2018 and search Association’s (CRA) Taul- by Stuart Zweben, professor
from the National Student the 2018/2019 academic years, bee Survey. emeritus, Ohio State University;
Clearinghouse Research Center there was a 4.7% increase in degree ˲ In some disciplines of com- Jodi Tims, a professor at
(NSC). In previous years, ACM production across all computing puting, African Americans and Northeastern University; and
directly surveyed computer disciplines, with the greatest in- Hispanics are actually overrepre- ACM education and professional
science departments, and creases in software engineering sented, based on their percentage development manager Yan
would work with a sample of (9%) and computer science (7.5%). of the U.S. population. Timanovsky. By employing
approximately 18,000 students. ˲ The representation of wom- ˲ Based on aggregate salary NSC data in future studies,
By accessing the NSC’s data, en in information systems (24.5% data from 89 non-doctoral-grant- the co-authors said they were
the ACM NDC study now of degree earners in the 2018/2019 ing computer science depart- confident an even fuller picture
includes information on academic year) and information ments (including public and pri- will emerge regarding student
approximately 300,000 students technology (21.5% of degree earn- vate institutions), the average retention with respect to
across the U.S., allowing for a ers in the 2018/2019 academic median salary for a full professor computing disciplines, gender,
more reliable understanding year) is much higher than in areas was $109,424. and ethnicity.

10 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 news

“It’s quite remarkable for a consumer use. Models made in April by the Big
technology to get within three meters Data Group at Oxford University in
or five meters, but there is a lot of vari- One outstanding the U.K. suggested if 60% of the popu-
ability in that, and it’s different in in- question is how many lation would use them, it could stop
door settings and outdoor locations,” the disease in its tracks. More recent
Kleinman says. “It would just have a lot people must adopt pilot studies Oxford ran on England’s
of limitations for identifying contacts contract tracing Isle of Wight indicated the spread
in a reliable way, and you would end up could be slowed if just 15% to 20% of
getting a lot of false positive and false apps for them to be people used the app as recommend-
negative identifications.” effective in slowing ed. That includes scanning QR codes
Either is a problem. False negatives to check into stores and restaurants,
could mean missing actual cases of dis- the pandemic. so health authorities can keep tabs on
ease transmission, but false positives those businesses.
would result in people being told to iso- Tessaro says a lot is still unknown
late themselves unnecessarily, which about what impact contact tracing
can be disruptive to work, personal life, apps could have, but they are unlikely
and mental health. to provide a quick fix. He thinks they
There are also problems with Blue- may work best as a supplement to hu-
tooth, which is not designed to mea- While Tessaro understands why man-run contact tracing, with the apps
sure distance. The space between people might be uncomfortable hav- filling in information that would be dif-
phones can be inferred by the strength ing their location tracked, he also rec- ficult for people to find, such as who
of the signal, but the orientation of the ognizes public health experts would shared a subway car with an infected
phone, a wall, or even the user’s own love to have GPS data help them trace person. “The metric that people apply
body can alter apparent signal strength. the spread of the disease and identify to these tools is that of a silver bullet,
hotspots. He and his colleagues have that there’s one thing that is going to
Keeping Score proposed what they call narrowcast- fix everything,” he says. “But it’s really
One way to deal with that weakness in ing, in which a user’s phone collects not true.”
Bluetooth is to use scoring algorithms its own location data but does not
to help decide whether a phone contact send it to anyone. Then, if a health de-
Further Reading
is enough of an in-person contact to partment finds an infected person was
trigger an alert, says Stefano Tessaro, a in a particular park or grocery store at Kleinman, R. and Merkel, C.
Digital Contact Tracing for COVID-19,
cryptographer and computer security a given time, it could broadcast that
CMAJ, 192 (24) 2020. www.cmaj.ca/
expert at the University of Washington information through an app, and if content/192/24/E653
(UW). Tessaro and a loose coalition of the health department information
Chan, J., Foster, D., Gollakota, S., Horvitz, E.,
researchers from UW, Microsoft Re- matches data stored on the phone, the Jaeger, J. Kakade, S., Kohno, T., Langford, J.,
search, the University of Pennsylvania, user gets an alert. Larson, J., Sharma, P., Singanamalla, S.,
and the Boston Public Health Commis- Of course, old-fashioned manual Sunshine, J., and Tessaro, S.
sion developed what they dubbed PACT, contact tracing does not strictly pro- PACT: Privacy-Sensitive Protocols and
Mechanisms for Mobile Contact Tracing,
privacy-sensitive protocols and mecha- tect people’s privacy, either. Health
ArXiv, 2020. arxiv.org/abs/2004.03544
nisms for mobile contact tracing. workers talk with infected people and
It would be useful to come up with ask where they have been and who has Singh, P., Singh, A., Cojocaru, G.,
Vepakomma, P., and Raskar, R.
formulas that use factors such as sig- been near them. “It’s considered a PPContactTracing: A Privacy-Preserving
nal strength and length of contact to fairly essential public health approach Contact Tracing Protocol for COVID-19
score whether something counts as ac- to addressing infectious disease. The Pandemic, ArXiv, 2020. arxiv.org/
tual exposure, rather than triggering question of where the appropriate line abs/2008.06648
an alert for, say, every student who is, that’s really a question for society,” Ferretti, L., Wymant, C., Kendall, M., Zhao, L.,
walks by a professor’s window and lat- Kleinman says. One major difference Nurtay, A., Abeler-Dörner, L., Parker, M.,
er tests positive. The difficulty, Tessaro is that traditional contact tracing Bonsall, D., and Fraser, C.
Quantifying SARS-CoV-2 transmission
says, is that despite a large number of starts with a known infected person suggests epidemic control with digital
cases, the disease is still rare enough and builds outward, ignoring those contact tracing, Science, Vol. 368, Issue
that real-world data is lacking. “There’s who have not been in contact with a 6491, 2020. science.sciencemag.org/
not enough positive cases, fortunately, patient, whereas apps collect some content/368/6491/eabb6936
to be in a situation where you can really amount of information from everyone Exposure Notifications System: Helping
see a lot of such false positives,” Tessa- who uses them, he says. Health Authorities fight COVID-19
ro says. Additionally, the same restric- One outstanding question is how https://www.youtube.com/
watch?v=1Cz2Xzm6knM&feature=emb_logo
tions that protect users’ privacy also many people must adopt the contact
make it more difficult for researchers tracing apps for them to be effective
Neil Savage is a science and technology writer based in
to collect data that can tell them how in slowing the pandemic, and uptake Lowell, MA, USA.
good a job an app is doing at correctly depends in part on how comfortable
identifying contacts. people are that the apps are safe to © 2020 ACM 0001-0782/20/12 $15.00

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 11

news

Technology | DOI:10.1145/3427942 Esther Shein

Softening Up Robots
Giving robots soft, artificial skin would
enable them to work more closely with people.

W
H E N YO U P I C T U R E a
robot, you likely envi-
sion one large and rig-
id, with limited move-
ment and an outer
shell that is hard to the touch. Several
projects currently underway seek to
change that, with the use of soft, more
human-like artificial skin.
Artificial skins include any surface-
based device or distributed network of
sensors that enable an agent to perceive
mechanical deformations, touch, tem-
perature, vibration, and/or pain, ac-
cording to Ryan Truby, a post-doctoral
fellow in the Massachusetts Institute of
Technology (MIT) Computer Science &
Artificial Intelligence Lab (CSAIL). Engi-
neers are working to create skins that University of Texas at Arlington’s patented smart skin technology, which the university says
include as many of these sensations as will give robots more sensitive tactile feeling than humans.
possible, while also possessing high
sensitivity and spatial resolution in comes into contact with something says Majidi, who is spearheading an ef-
sensing, he adds. can be done better.” fort to develop new classes of soft ma-
“Artificial skin will liberate today’s Robots today are not able to help in terials in the fields of soft matter engi-
robots and bring them to a higher level dire situations like natural disasters. neering and soft robotics.
of consciousness,” notes Yiannis Alo- “There is a conspicuous absence of ro- A significant step in making ma-
imonos, a professor at the University of bots whenever we are in trouble,’’ in- chines safe to engage in physical con-
Maryland who leads the school’s Com- cluding in the midst of the coronavirus tact with humans is to build them from
puter Vision Laboratory (CVL). “They pandemic, says Aloimonos. materials that share lot of the same
will become much safer, especially That is not to say the work robots properties as us: water, fluids, soft ma-
when operating near people.” do has not been important; it is just terials, and nervous tissues that are soft
Robotic skin cells, in one sense, can limited. Up until now, industrial and and stretchy, Majidi says.
do more than animal skin cells because commercial applications of robotics “Tactile sensing can be thought of as
they can sense proximity to a surface for have been large in scale and focused on an ‘artificial skin’ technology that en-
anticipating and avoiding accidents, Al- factories, “and they are incredibly well- ables robots to have a sense of touch,’’
oimonos says. “Having this additional engineered for what they need to do, but he says. “They are used in a wide range of
sense, the robots will be able to perceive they are not designed for use with hu- robotics applications, from humanoid
their environment in greater detail.” mans,” says Carmel Majidi, director of robots to wearable technologies. They
The fusion of vision and tactility will the Soft Machines Lab and an associate can also be used in soft robotics, but this
IMAGE COURTESY OF UNIVERSIT Y OF T EXAS AT ARLING TON

make possible many new tasks that professor of mechanical engineering at is not their primary goal.”
were not possible before, Aloimonos Carnegie Mellon University (CMU). Autonomy in robotics can be ad-
says. One problem being studied in his One of the main ideas for develop- vanced with improved perception, Tru-
lab is how to teach robots to use tools. ing soft robotics is to create machines by says. “The abilities to sense touch,
When humans use tools, we remember that are safe for physical interaction stretch, temperature, vibration, and
the feeling of a tool in our hands, and with humans and won’t injure us in a many other forms of tactile and pro-
use this information to learn how to collision, Majidi says. prioceptive feedback are critical for the
control its movement. “Recognition Often, industrial robots must be autonomy of any robot tasked with op-
from vision and touch also becomes a caged behind metal fences to prevent erating in the real world.”
new powerful technique,’’ he says. “In them from harming humans who un- Artificial robot skins are a straight-
general, any tasks where the robot wittingly enter their areas of operation, forward way to begin introducing these

12 COM MUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 news

perceptual capabilities in robots, says artificial skins of large area,’’ he says.

Truby. “Innovations in robotic skins “Thus, when designing artificial skins,
will likely be key to advancing the au- ”We are working to there is a clear trade-off between perfor-
tonomy of robots designed for real- improve the materials mance, size or area, and cost.”
world deployment.” Yet more expensive artificial skins
Additionally, he says, softer skins that go into these can provide “a very high resolution of
will improve the safety of human-robot skins to enable tactile and touch feedback,’’ he notes.
interactions and also open up new They provide a lot more functionality
ways for us to interact with machines. tactile sensing, and are commonly used in prosthetics,
These advances would take humans as we would expect Truby says. “But these available meth-
out of harm’s way, he says. The field of ods are expensive to implement, do not
soft robotics has generally shown that from a sophisticated scale well for large soft robots like ours,
soft materials like the rubbers and hy- soft robot skin.” and can be difficult to interface with soft
drogels soft robots are being construct- robots,’’ he says. “Our approach repre-
ed from “do not just make robots safer sents a new strategy for rapidly, inexpen-
for human-robot interactions; they sively, and easily giving soft robots im-
also offer important advantages over portant perception capabilities, which
materials like metals and thermoplas- remains a challenge in soft robotics.”
tics that are used in ‘rigid’ robots to- Right now, MIT’s soft robotic skins
day,” Truby says. and is now focused on improving the are not appropriate for tactile or touch
These advantages can help soft ro- accuracy of the configuration estima- sensing, Truby says. “We are working to
bots become more resilient in harsh tions through new sensor skin designs improve the materials that go into these
environments such as in a forest fire, and deep learning methods. “We are skins to enable tactile sensing, as we
for example, compared to their rigid also working to use this feedback to dy- would expect from a sophisticated soft
counterparts, he says. “Thermoplastics namically control the arm for a variety robot skin. We’re also working on algo-
would melt or burn from the heat of of applications.” rithms that complement these simple
fires. “Excellent thermal conductors There are many types of artificial soft sensor skins and enable new con-
like metals can reach high tempera- skins, but many are expensive to make, tact sensing motifs from external cam-
tures in a fire, potentially damaging not easy to interface with soft robots, eras and actuation inputs.”
other onboard components due to and are not scalable, Truby says. “We’re Majidi’s group at CMU is research-
heating. In contrast, soft robotic mate- fighting against cost and scalability,’’ ing materials and systems that com-
rials avoid these issues because they are and accessibility. “The advantage of bine mechanical, electrical, and ther-
often good thermal insulators and can soft robots is they are cheap.” mal properties and can function as
demonstrate higher flame resistance.” The MIT researchers worked with artificial skin, nervous tissue, and mus-
Truby and his colleagues at MIT de- “commodity conductive silicone rub- cle for soft robotics and wearables.
veloped a system of soft sensors that bers available right off the shelf,” he His lab created a self-healing mate-
cover a robot’s body to enable its aware- says. Comparable skins comprised of rial that repairs itself under extreme
ness of motion and position. “If we other types of conductive rubbers, or mechanical damage, according to
want to have soft-skin robots in the real materials like liquid metals or ionically CMU. The soft-matter composite mate-
world, we need to bring skins and sens- conductive gels, have been made rial is composed of liquid metal drop-
ing capabilities to these systems that through relatively simple and inexpen- lets suspended in a soft elastomer;
can provide the feedback needed to au- sive techniques such as 3D printing when damaged, the droplets rupture to
tonomously control the robot’s mo- and molding, he says. form new connections with neighbor-
tion,’’ Truby says. “We are going to Ordinary rubbers is a good insulator ing droplets and reroute electrical sig-
have to solve the material engineering but not a good conductor, Truby says. nals without interruption. Circuits pro-
challenge of bringing those capabili- The materials used by the researchers duced with conductive traces of this
ties into soft robot bodies.” “have conductive particles in them, so material remain fully and continuously
The research team tested its system we can conduct electricity and can cre- operational even when severed, punc-
on a soft robotic arm equipped with very ate sensory skins from them.” tured, or with material removed. The
simple sensors to predict its own posi- More complicated skins that can per- goal is for electrical connections to
tion as it autonomously swings. The test ceive multiple sensations and/or pos- spontaneously restore themselves
proved eye-opening: “The very simple, sess a higher density of “receptors” for when damaged, Majidi says.
inexpensive sensors we gave this soft ro- these stimuli are often made with slow- Applications for the soft material
bot arm enabled a deep neural network er, more expensive microfabrication include bio-inspired robotics, human-
to provide a satisfactory estimate of the techniques, Truby says. “While micro- machine interaction, and wearable
arm’s 3D configuration as it moved fabrication methods open up opportu- computing. Because the material also
about,’’ Truby says. “To our knowledge, nities for more readily building with exhibits high electrical conductivity
this is a first for the field of soft robotics.” materials like thin metal films and con- that does not change when stretched,
He says the team is always working ductive polymers, it’s often very expen- it could be used in power and data
to improve its soft robots’ capabilities sive to scale these techniques to produce transmission, according to CMU.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 13

news

The greatest challenge is delivering increase in the rate of evaporation and

adequate power to electronics within cools the muscle’s body, similar to
these skins, Majidi says. Skin that is “We’ve been creating perspiration in hot-blooded animals,
soft and stretchable cannot be charged soft thermally according to Anand Kumar Mishra, a
with a “big hulking battery … so there’s post-doctoral student at Cornell work-
the challenge of creating batteries that conductive rubber ing on the project. The job of the fluids is
are soft and flexible,’’ which is one that can help to actuate the muscle, says Mishra, “But
project his lab is working on. when the robotic body temperature el-
The amount of energy contained in manage or dissipate evates, then the actuating fluid can
a small lithium battery is limited, he the heat that’s also be used for perspiration to regu-
says, and if you want to incorporate late the body temperature.”
sensors and electronics in artificial generated by This will enable a robot to operate
skin, today they are not capable of de- these machines for long periods without overheating,
tecting their environment or physiolo- Shepherd said.
gy. Current lithium battery cells have and electronics.” The idea of an artificial skin for ro-
enough power to operate the sensors bots has been researched for decades,
that detect the environment or physiol- and there are numerous applications
ogy, he says. “However, they don’t have being developed with human-robot
enough power to also operate the mi- collaboration (HRC), says Werner
croprocessor used to process the sig- Kraus, head of the Robot and Assis-
nals and the radio transceiver used to tive Systems department at the
transmit the signals wirelessly. For ing soft thermally conductive rubber Fraunhofer Institute for Manufactur-
there to be enough power for long-term that can help manage or dissipate the ing Engineering and Automation IPA,
sensing, signal processing, and wire- heat that’s generated by these ma- in Stuttgart, Germany. “Artificial skin
less data transmission, the current chines and electronics.” could enable this type of [HRC] inter-
miniature battery would have to be re- Sweating and synthetic skin are also action, since its sensors can detect a
placed with a much larger battery.” being studied at Cornell University, contact between human and robot
A larger battery would add to the ri- where scientists have created a soft and stop the robot’s movement ac-
gidity of the soft artificial skin technolo- robotic muscle that stays cool and cording to safety requirements.”
gy, Majidi says. This introduces a chal- mimics how a hand moves. Artificial skin offers more sensor data
lenge with supplying power to an The ability for a muscle to stretch is to perceive the environment, Kraus says.
artificial skin. helpful in adapting to unpredictable “In times of powerful machine learning
There are two potential solutions. circumstances, like walking through a algorithms, this data can be the enabler
The first is to use a battery that is soft, disaster zone with rubble everywhere, for implementing software on robots
flexible, and stretchable, he says. says Cornell associate professor Rob that makes them more autonomous and
This type of battery could be larger in Shepherd. “It’s hard to predict what more efficient.”
size than the existing miniature lithi- the next meter of terrain will look like.”
um batteries and therefore supply Shepherd and other Cornell engi-
Further Reading
more power, Majidi says. Moreover, neering professors use polymers—in
because it is soft and compliant, it this case, rubber—for the skin, since it Liu, D., Su, L., Liao, J., Reeja-Jayan, B.,
and Majidi, C.
will not add to the rigidity of the soft stretches and is similar to human skin,
“Rechargeable Soft-Matter EGaln-Mn
artificial skin. he says. Like Truby, he adds, “It is also O2 Battery for Stretchable Electronics,”
Another solution is to use energy- pretty cheap, and you can make them Advanced Energy Materials, Wiley Online
harvesting to convert energy sources in self-heal.’’ Library, https://onlinelibrary.wiley.com/doi/
the environment into electricity. “My Such benefits come at many costs, abs/10.1002/aenm.201902798
lab is currently working on harvesting Shepherd notes, one being that poly- Carmel Majidi: Self-Healing Electrical
energy from body heat using soft and mers and rubber don’t conduct heat Material, https://www.youtube.com/
watch?v=N_ijvkl51LM
stretchable thermoelectric generators, very well, unlike metal. “Cooling ro-
as well as from knee and elbow motions bots hasn’t been as much of a prob- inSPIREd – Dr. Ryan Truby - Can soft
using a soft conductive material for tri- lem as it will be in future for these materials revolutionize the meaning of
robotics?, https://www.youtube.com/
boelectric energy harvesting.” polymer ones,’’ he says. “If we expect watch?v=eRTyWgTHQZI
Eventually, if enough energy can be them to do a lot of work quickly, they’ll
Chortos, A., Liu, J., and Bao, Z.
harvested, “It could power motors to generate heat and that will damage “Pursuing prosthetic electronic skin,”
get robots to move and actuate,’’ he the robot, so we have to dissipate the Nature Materials, https://www.nature.com/
says. “That’s the really critical step.” heat somehow.” articles/nmat4671
The inability of artificial skin to The Cornell researchers developed a
sweat is another challenge. Soft robots system that automatically dilates pores Esther Shein is a freelance technology and business
writer based in the Boston area.
produce heat, and it is important to in an actuator when the temperature
have materials that can manage that reaches 86 degrees Fahrenheit. The idea
heat, Majidi says. “So we’ve been creat- is when the pores dilate, it allows for an © 2020 ACM 0001-0782/20/12 $15.00

14 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 news

Society | DOI:10.1145/3427936 Logan Kugler

Technologies for
the Visually Impaired
The last decade has seen major advancements in technology
for the blind and visually impaired, but problems remain.

T
H A N KS TO RE CE N T advances
in technology, the blind and
visually impaired are now
able to lead more indepen-
dent lives than ever.
The WeWALK Smart Cane is a great
example of what is now possible. The
WeWALK looks similar to the cane that
some blind and visually impaired peo-
ple have used for decades to avoid ob-
stacles while walking, but it incorpo-
rates a few modern twists.
With a standard cane, you can still
run into obstacles that are not imme-
diately underfoot, like poles, tree
branches, and barriers. The WeWALK,
however, detects objects above chest
level and audibly alerts you if you’re
getting too close, which can save you
from a painful fall.
Also, when using a standard cane,
you have to hold a smartphone in one The WeWALK Smart Cane detects objects in the vicinity and provides audible alerts to
hand to listen to directions, making it prevent collisions.
even more difficult and dangerous to
navigate your environment. The We- still inaccessible to the visually im- Aaron Steinfeld, associate research pro-
WALK integrates with a smartphone’s paired, making it difficult or impos- fessor in The Robotics Institute at Carn-
map app to read directions out loud, al- sible to use key online services, tools, egie Mellon University.
lowing you to keep one hand free. and experiences that the sighted may Most of today’s innovations in tech-
Finally, the WeWALK costs less than take for granted. nology for the blind and visually im-
$500, making it affordable for many of The result is an imbalance in assis- paired are delivered via smartphone.
the estimated 10 million visually im- tive technology progress. On the one These include sophisticated applica-
paired people in the U.S., and 250 mil- hand, it’s never been easier or more af- tions that empower the visually im-
lion worldwide. fordable to acquire new, powerful ac- paired to navigate, count money, and
The WeWALK is just one example of cessibility technology; on the other, obtain assistance with basic tasks. They
a larger trend. Technology to help the there are still serious accessibility is- also include tools such as artificial in-
blind and visually impaired has become sues with the technology that powers telligence and machine learning to rec-
dramatically more powerful and signifi- much of modern life. ognize facial expressions and describe
cantly cheaper in the last 10 years. In the the external world in real time.
process, it has revolutionized how blind A Decade of Progress The sheer ubiquity of smartphones
and visually impaired individuals navi- Technological progress during the last has made them the perfect enabling
IMAGE COURTESY OF W EWA LK/W EWA LK .IO

gate the world. decade has created major benefits for device for the blind and visually im-
However, despite the progress, there society as a whole, as well as for the blind paired. The percentage of U.S. adults
are still major unresolved problems. and visually impaired in particular. who own smartphones has risen
Technologies like sophisticated “Computer vision, fast mobile pro- from 35% in 2011 to 81% in 2019, ac-
smartphone apps and artificial in- cessors, high-speed wireless Internet, cording to Pew Research, and Cisco
telligence have empowered mil- and cloud computing have enabled estimates more than 70% of the glob-
lions of blind and visually impaired some truly fascinating innovations for al population will have mobile con-
individuals, but many websites are resolving barriers in everyday life,” says nectivity by 2023.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 15

news

Mellon University, who works on acces-

sibility issues. “When I did my Ph.D. in
the 00s, a screen reader (a software pro-
gram that reads aloud text displayed on
a computer screen, or displays it in
braille) cost more than $1,000; now,
the VoiceOver screen reader is avail-
able on every iOS device for free, even
the Apple watch,” he says.
Affordable smartphone apps have
empowered the blind and visually im-
paired. Now, artificial intelligence
(AI) is taking those apps’ capabilities
to the next level.
AI and machine learning technolo-
gies, specifically computer vision, have
grown sufficiently robust to improve the
lives of the blind and visually impaired.
Computer vision is a machine’s
ability to recognize objects in an envi-
ronment. Microsoft is leading the way
The LookTel Money Reader app captures an image of paper money on a smartphone, and with its Seeing AI smartphone app,
speaks aloud the denomination of the bill. which uses computer vision to process
the immediate environment, other
Navigation is a huge part of the value same phone, however; there is no da- peoples’ facial expressions, and text,
smartphones provide for the blind and tabase of locations labeled by others and then tells what is happening
visually impaired. for users to access.) around them. Users simply hold up
GPS-enabled devices mean you ˲ The Aira app connects the blind their smartphones, and the intelligent
now have access to accurate, audible and visually impaired to remote camera app “sees” for them.
real-time navigation pretty much agents who assist with navigation, Another example of AI at work is the
wherever you go. At least one popular reading printed media, and other captioning built into tools like Skype
navigation app—Google Maps—of- everyday tasks. The app is free to (owned by Microsoft) and Google Slides.
fers detailed voice guidance for the download and use. The technology uses another applica-
blind and visually impaired. In all cases, the value of these apps tion of AI and machine learning, name-
Smartphone navigation integra- is that they’re affordable, a big change ly natural language processing, to un-
tions also matter. The WeWALK Smart from a decade ago, says Jeffrey Bigham, derstand the text in a chat or on a slide,
Cane’s app integrates with the actual an associate professor in the Human- then read it aloud. This happens in real
cane hardware, so you receive turn-by- Computer Interaction and Language time, allowing the blind and visually im-
turn instructions and clockwise navi- Technologies Institutes at Carnegie paired to navigate conversations and
gation directions right from the cane presentations with less friction.
itself, without having to remove your While AI has unlocked promising
smartphone from your pocket. Smartphone apps applications, it is still early days, says
Navigation isn’t limited to walking Bigham, who points out that AI tech-
down the street, either; smartphone make it easier nologies and tools still make plenty of
applications now make it easier for the for the blind and errors, and few people would want to
blind and visually impaired to navigate rely on them exclusively to navigate
other difficult situations. For example, visually impaired the world. “We are at the ‘better than
˲ LookTel Money Reader helps the
to navigate other nothing’ stage of AI in accessibility,”
visually impaired count money. Usesr he says. “That’s honestly a lot better
capture an image a picture of a bill on difficult situations, than the nothing many people had in
their smartphones, and the app speaks like counting paper some situations not too long ago.”
the denomination of the bill.
IMAGE COURTESY OF LOOK TEL/YO UTUBE.COM

˲ TalkingTag LV is an iPhone app money and labeling New Tech, Same Problems
that allows users to label their sur- their surroundings. While smartphone and AI technology
roundings with special coded stick- have advanced in the last decade, less
ers. When scanned by the app, the progress has been made on the accessi-
stickers play an audio message re- bility of our current digital infrastructure.
corded by the user, enabling them to According to the World Wide Web
custom-label their environment. (The Consortium (W3C), many sites and
stickers only work when using the tools online today have design barriers

16 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 news

that make them impossible for the

blind and visually impaired to use, be-
Artificial intelligence
ACM
cause they don’t follow accessibility
best practices. In the U.S., it’s a legal
requirement under the Americans with
is taking the ability Member
Disabilities Act (ADA) to make sites ac- of affordable News
cessible, but accessibility rules are un- smartphone apps to
evenly enforced.
“Digital accessibility is still in- empower the blind CONSIDERING THE
SOCIETAL IMPACT OF
credibly limited,” says Carnegie Mel- and visually impaired COMPUTING
“In college, at
lon’s Steinfeld. “It is critical that
Web and software companies imple- to the next level. my school,
students had
ment and support established acces- the choice to
sibility features to ensure full access take either the
biology track or
to their products.” the computer
W3C offers a host of guidelines on science track,” recalls Kavita
making websites accessible for the Bala, dean of the Faculty for
blind and visually impaired. These in- through voice commands, if needed, Computing and Information
Science at Cornell University.
clude notions like providing text alter- using Voice Command capabilities. “If you took the biology track
natives to non-text content so content Bigham notes the accessibility groups you had to dissect a frog, so I
can be converted into large print or at these three major tech companies picked computer science.”
After that, “I was hooked,”
braille. It also includes making all have “varying levels of influence,” and
she adds, “there was no
functionality on a website accessible companies like Amazon, Facebook, turning back.”
using only a keyboard. and Twitter are lagging behind when it She earned her Bachelor of
The W3C organization even has a comes to accessibility. Technology degree in computer
science at the Indian Institute
set of Web Content Accessibility Holding technology companies of Technology in Bombay, India,
Guidelines (WCAG), a series of docu- and developers accountable is im- and her master’s and Ph.D.
ments that “explain how to make web portant work. It also perfectly high- degrees, both in computer
content more accessible to people lights how frustrating it can be to science, at the Massachusetts
Institute of Technology. After
with disabilities,” according to the or- balance getting existing technology obtaining her Ph.D., she joined
ganization’s website. providers to follow the rules, while the faculty at Cornell, and has
As one example of what this looks also inventing better technologies remained there since.
Bala’s research interest
like in practice, many sites are not to increasingly enable the blind and is in the area of computer
marked up correctly with HTML, so text- visually impaired. graphics and computer vision,
to-speech software cannot accurately “I would personally prefer to be work- with a focus on research in
translate words into audio for the blind ing on exciting new accessibility applica- visual recognition, physically
based rendering, and material
and visually impaired. In most cases, tions, but I spend a lot of my time argu- modeling and perception. Her
these are easy technology fixes, but they ing with organizations to make their research on scalable rendering
often are overlooked or neglected. websites accessible,” says Bigham. technology has been adopted in
industrial products for virtual
“We have known how to make com- design and prototyping.
puting technologies accessible to peo- In 2015, Bala co-founded
Further Reading
ple who are blind for decades,” says the start-up GrokStyle,
Bigham. “Yet the biggest problem re- Charters, L. which created an app that
Apps now online to aid patients with visual automatically identifies
maining is to convince people to code furniture and home decor from
impairment, Ophthalmology Times,
their websites and other applications Jun. 5, 2020, https://bit.ly/2GPWxTC just about any picture or angle.
correctly so they are accessible.” Ikea became a client of the
Henry, S. company, which was acquired
Bigham does note that companies Accessibility Fundamentals Overview, by Facebook in 2019.
such as Apple, Google, and Microsoft World Wide Web Consortium, Feb. 19, 2020, As dean, Bala feels it is
are getting much better at handling is- https://www.w3.org/WAI/fundamentals necessary to consider the
sues of accessibility. All three now societal impact of computing.
Mobile Fact Sheet, Pew Research Center,
“We will have to grapple with
have large groups dedicated to acces- Jun. 12, 2019, https://www.pewresearch. data science, and information
sibility issues relating to their prod- org/internet/fact-sheet/mobile/ and computer technology,
ucts and digital experiences. Cisco Annual Internet Report (2018-2023) within a societal context,
For instance, Apple uses its White Paper, Cisco, Mar. 9, 2020, whether it is fairness or access,”
https://bit.ly/3djrgVH she said. “These things are
VoiceOver capabilities to describe what going to be more important
is happening on your Apple device. within the next decade.”
Logan Kugler is a freelance technology writer based —John Delaney
The company also offers visual filters in Tampa, FL, USA. He has written for over 60 major
to help the colorblind and magnifiers publications.
to help the visually impaired. Apple
devices can also be navigated entirely © 2020 ACM 0001-0782/20/12 $15.00

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 17

V
viewpoints

DOI:10.1145/3430368 Cansu Canca

• Susan J. Winter, Column Editor

Computing Ethics
Operationalizing
AI Ethics Principles
A better ethics analysis guide for developers.

A
RTIFICIAL INTELLIGENCE (AI) not complete systems for ethical deci- an interactive toolbox with features to
has become a part of our sion-making and not suitable for solv- (1) sort, locate, and visualize sets of AI
everyday lives from health- ing complex ethical problems. But once principles demonstrating their chrono-
care to law enforcement. operationalized, they provide a valuable logical, regional, and organizational
AI-related ethical chal- tool for detecting, conceptualizing, and development; (2) compare key points
lenges have grown apace ranging from devising solutions for ethical issues. of different sets of principles; (3) show
algorithmic bias and data privacy to With the aim of operationalizing AI distribution of core principles; and (4)
transparency and accountability. As a principles and guiding ethical practice, systematize the relation between prin-
direct reaction to these growing ethical in February 2020, at the AI Ethics Lab we ciples.b By collecting, sorting, and com-
concerns, organizations have been pub- created the Dynamics of AI Principles,a paring different sets of AI principles,
lishing their AI principles for ethical we discovered a barrier for operation-
practice (over 100 sets and increasing). a AI Ethics Lab, Dynamics of AI Principles, pub- alization: many of the sets of AI princi-
However, the multiplication of these lished in February 2020, https://bit.ly/3k7VgpN ples mix together core and instrumental
mostly vaguely formulated principles principles without regard for how they
has not proven to be helpful in guid- relate to each other.
ing practice. Only by operationalizing These philosophical b Similar efforts have been done before and we
AI principles for ethical practice can
we help computer scientists, develop- theories answer the have cross-checked our list with these other
works. See Fjeld et al., “Principled Artificial
ers, and designers to spot and think central question of Intelligence: Mapping Consensus in Ethical
through ethical issues and recognize and Rights-Based Approaches to Principles
when a complex ethical issue requires applied ethics: What for AI,” Berkman Klein Center Research Publi-
in-depth expert analysis. These op- is the right/good cation 1 (2020), https://bit.ly/2T2sivG; Jobin,
Ienca, and Vayena, “The Global Landscape of
erationalized AI principles for ethical
practice will also help organizations action or policy AI Ethics Guidelines,” Nature Machine Intel-
ligence 1 (2019), 389–399, https://go.nature.
confront unavoidable value trade-offs com/3jcvtMc; and Zeng, Lu, Huangfu, “Link-
and consciously set their priorities. At ing Artificial Intelligence Principles,” AAAI
Workshop on Artificial Intelligence Safety (2019),
the outset, it should be recognized that https://bit.ly/3j6fjUj. These works look at the
by their nature, AI ethics principles—as frequency of principles rather than categoriz-
any principle-based framework—are ing them conceptually.

18 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 viewpoints

In any given set of AI principles, one Core versus Instrumental Principles certain values: respect autonomy, do
finds a wide range of concepts like priva- The most widely utilized set of core good, ensure justice. They are core
cy, transparency, fairness, and autono- principles in applied ethics is: respect principles because they invoke those
my. Such a list mixes core principles that for autonomy, beneficence (avoiding values that theories in moral and polit-
have intrinsic values with instrumental harm and doing good), and justice.d ical philosophy argue to be intrinsical-
principles whose function is to protect These philosopical principles pre- ly valuable, meaning their value is not
these intrinsic values.c Human auton- scribe an appropriate attitude toward derived from something else. These
omy, for example, is an intrinsic value; theories answer the central question
it is valuable for its own sake. Consent, of applied ethics: “What is the right/
d In 1978, U.S. National Commission for the Pro-
privacy, and transparency, on the other tection of Human Subjects of Biomedical and good action or policy to choose?” By
hand, are instrumental: we value them Behavioral Research published the Belmont encapsulating these theories’ intrinsic
to the extent they protect autonomy and Report; https://bit.ly/3dwyOV5. The Report laid values in an attitude-setting format,
other intrinsic values. Understanding out three core ethical principles for human core principles help us immediately
subject research: respect for persons, benefi-
these categories and their relation to recognize if we are facing an ethical
cence (which divides further into two general
each other is the key to operationalizing rules of “do not harm” and “maximize possible challenge: Any action that disrespects
AI principles that can inform both devel- benefits and minimize possible harms”), and autonomy, inflicts harm, or discrimi-
opers and organizations. justice. In 1979, philosophers Tom Beauchamp nates is ethically problematic, if not
(who was at the Commission co-authoring the straightforward unethical.
c The only document that conceptually cat- Report) and James Childress published the ca-
egorizes principles is the paper by AI4People, nonical book Principles of Biomedical Ethics,
When we categorized all of the
IMAGERY F ROM SHUT TERSTOCK .COM

which recites the four “core” principles (be- where they identified four prima facie ethical published AI principles into these
neficence, non-maleficence, autonomy, and principles: respect for autonomy, beneficence, three core principles, we found a
justice) and adds the principle of explicability non-maleficence, and justice (see Beauchamp surprisingly balanced and consis-
as an “enabling” principle. Floridi et al., “AI- and Childress, Principles of Biomedical Eth- tent picture. Across industries and
4People—An Ethical Framework for a Good ics, 8th edition, Oxford University Press, 2019).
AI Society: Opportunities, Risks, Principles, Principles in the book and in the report over-
regions, similar weight is given to
and Recommendations, Minds & Machines 28, lap in their content and form what is now often each of these principles and neither
(2018), 689–707; https://bit.ly/343poNt called the “traditional bioethics principles.” one really outweighs the others: As of

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 19

viewpoints

The Box.

October 2020, approximately 25%–30% Operationalizing the AI Principles outcome- and safety-related informa-
of all principles are autonomy-focused, to Guide AI Practice tion to exercise their autonomy, and de-
about another 32%–36% are focused To guide AI practice, it is important velopers need rigorous testing for safety
on avoidance of harm and maximizing to distinguish core and instrumental and accuracy to minimize harm. Ex-
benefits, and approximately 36%–42% principles because instrumental prin- plainability in this case should not be
of principles are justice-focused.e ciples are interchangeable. Individuals prioritized especially if it would compro-
In contrast to core principles encap- and organizations can weigh instru- mise accuracy and safety.
sulating intrinsic values, instrumental mental principles to determine which The general point is there is no deep
principles build on concepts whose val- to prioritize and how to use them to ethical dilemma when an instrumen-
ues are derived from their instrumen- best achieve the core principles. tal principle is not suitable for a given
tal effect in protecting and promoting For example, the instrumental prin- case. Once we correctly categorize
intrinsic values. Take transparency for ciple of explainability may be required principles as core and instrumental,
example. We do not think that some- or optional for ethical AI depending on we can turn many vague AI principles
thing is valuable in and of itself solely the situation. Explainability is about into an operational checklist to guide
because it is transparent. Rather trans- understanding a system’s technical practice. We created a simplified in-
parency is valuable because it allows process and how it reaches its out- teractive checklist called The Boxf for
us to understand, engage with, and au- come. It is an instrumental principle computer scientists, developers, and
dit the systems that affect us. In other that can uphold human autonomy by designers to use for basic ethics analy-
words, transparency is instrumental to allowing individuals to interact mean- ses (see the figure here). The Box helps
uphold intrinsic values of human au- ingfully with the system. It can also them determine the relevant ethical
tonomy and justice. Similarly, account- help minimize harm and safeguard concerns and weigh applicable in-
ability in itself is not an end but rather justice by making it easier to detect er- strumental principles to determine
it is a means to safeguard justice by rors and biases. Explainability is cru- how to best satisfy core principles by
assigning responsibility and to avoid cial for a risk analysis AI system that substituting or supporting one instru-
harm through deterrence. helps judges set bail and parole in the mental principle with another.
criminal justice system. Here, explain- The Box also serves as a tool for com-
e The principle of justice holds the largest pie (ap- ability enables judges and defen- puter scientists, developers, and design-
proximately 40%). This is not surprising because dants to engage autonomously with ers to recognize when an ethical conflict
the justice principle is much more general than the system and better monitor its
the other two principles referring as it does to
various theories of justice and their main fair-
fairness. In contrast, explainability is f AI Ethics Lab. “Tool: The Box.” Toolbox: Dy-
ness claims such as equal treatment, equal op- not necessary for an AI system that op- namics of AI Principles (June 2020); https://
portunity, and protection of the worst off. timizes energy use in cars. Drivers need bit.ly/2HKKqZ5

20 COM MUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 viewpoints

is between core principles. Since each core

principle is intrinsically valuable, we can-
not simply ignore one for the sake of anoth- Principle-based
er. When core principles point in opposite frameworks only
directions, we face a real ethical dilemma.
In these cases, an ethical issue is com- provide a list of
plex and cannot be easily resolved solely considerations rather
based on guidance from principles; eth-
ics expert should be brought in to apply than a complete
ethical theories. An example could be and coherent
developing an AI system for diagnosing
a dangerous and highly contagious dis- decision-making tool.
ease. To minimize harm and suffering,
scientists need to quickly create a highly
accurate system. To train the algorithm
they need large amounts of personal data
that cannot be completely anonymized.
Asking for proper consent would delay AI principles, we need to utilize their
the project causing more infections and strengths and recognize their limita-
more suffering. However, circumventing tions, acknowledging that AI prin-
consent and disregarding privacy would ciples are only a first step for develop-
be a violation of individual autonomy. An ment and deployment of ethical AI. >> Opened in 2005, the newest
AI ethics framework that relies solely on University of California campus
principles is unable to solve this ethical di- Setting an Organization’s
lemma suggesting that this complex prob- AI Principles
lem should involve ethics experts who can Lastly, let us go back to the proliferation >> #3 Young University in U.S.,
2020 Times Higher Education
conduct an in-depth analysis and apply of AI principles. What is the point of com- Young University Rankings
ethical theories. Principle-based frame- pany-specific sets of principles if the con-
works only provide a list of considerations tent is largely the same, as we have seen? If
rather than a complete and coherent de- well done, the point is that organizations >> #40 Public University in U.S.,
cision-making tool.g Ethical theories, on with their own customized set of AI princi- U.S. News & World Report
the other hand, provide comprehensive ples can determine how to weigh compet- Best Colleges Rankings
guidance for decision making and ethics ing principles and which intrinsic value to
experts can utilize these detailed theories prioritize when core principles (and theo-
>> #63 Computer Science program
to analyze complex issues in-depth. ries) conflict. When no single argument in U.S., CSRankings: Computer
When properly operationalized, AI emerges as the strongest even after a full- Science Rankings
principles provide a helpful start for scale ethics analysis, developers and oth-
an ethics analysis and they can guide er organizational decision-makers must
developers and organizations through choose between equally ethically permis- >> PhD, MS and undergraduate
many ethical questions, even though sible options. This is when the organiza- programs in Computer Science
they are not sufficient for complex ethi- tion’s stance about core principles shows
cal problems. We need to understand through. When push comes to shove, does
>> Positioned near Silicon Valley
AI principles for what they are: A list this organization prioritize individual au-
of fundamentally and instrumentally tonomy or minimization of harm? An or-
important ethical considerations but ganization’s AI principles can guide this
not a complete system for complex decision and be useful both for computer
ethical decision making. Categorizing scientists and ethics experts when they eecs.ucmerced.edu
and using AI principles ensures we do are deciding a hard case. To help them engineering.ucmerced.edu
not overlook a crucial ethical concern. clarify their own values, we invite orga- graduatedivision.ucmerced.edu
By revealing when a case presents a nizations to use our toolbox to compare
conflict between core principles rather their efforts to others, systematize their AI
than instrumental ones, principles can principles, and engage in in-depth ethics
also help us recognize when we face analysis with experts to determine their
a complex case and need a full-scale own priorities for coherent and consistent
ethics analysis. As we operationalize ethical decision making.

Cansu Canca ([email protected]) is a philosopher

g An excellent paper on this is Clouser and Gert, and the Founder and Director of the AI Ethics Lab in
“A Critique of Principlism,” Journal of Medicine Boston, MA, USA.
and Philosophy 15, 2 (1990), 219–236; https://
bit.ly/342fITB Copyright held by author.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 21

V
viewpoints

DOI:10.1145/3430375 W.R. Adrion, S. Dunton, B. Ericson, R. Fall, C. Fletcher, and M. Guzdial

• Richard Ladner, Column Editor

Broadening Participation
U.S. States Must Broaden
Participation While
Expanding Access to
Computer Science Education
Incorporating equity and inclusion in the effort toward access for everyone.

M
AKING SWEEPING CHANGES
to education in the U.S.
is difficult because of
its highly decentralized
primary and secondary
school system. Each of the U.S. states
and territories makes its own deci-
sions about how education is struc-
tured. Some of those states push the
decision-making to districts and even
individual schools. Reforms, such as
providing high-quality computer sci-
ence (CS) education to all students,
require states to engage every school
district, if not every school. In order to
broaden participation in computing,
rather than exacerbate existing inequi-
ties as we expand K–12 CS education
to more U.S. schools, explicit attention
needs to be placed on how equity is ad-
dressed and measured in policy, practice, Participation in Computing Alliancesb (BPC) is a priority.1 Supported by addi-
and professional development. funded by the National Science Foun- tional NSF funding in 2018, the ECEP
Many states are making progress dation. Begun in 2012 (and described 2.0 leadership team continues to build
on CS education. As of October 2020, a previous Communications Education on the early work of ECEP while scal-
18 states have started or completed column, “Broadening Access to Com- ing the model of state-level BPC work
statewide plans; 37 states have defined puting Education State by State,” in beyond the ECEP states.
PHOTO BY NEONBRAND ON UNSPL ASH

CS standards; 40 states plus the Dis- February 2016), ECEP focuses on state- ECEP member states are making
trict of Columbia have teacher certi- level educational systems and now great strides toward increasing the
fication for CS.2 The Expanding Com- works with 22 states and the territory number and diversity of students
puting Education Pathways (ECEP) of Puerto Rico to ensure the goal of who have access to high-quality com-
Alliancea is one of eight Broadening broadening participation in computing puting education. But often, there is
more pressure to increase the number
a See https://bit.ly/31APN3V b See https://bit.ly/37h074r of students who have access than to

22 COMM UNICATIO NS O F THE AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 viewpoints

ensure the diversity of those students. also has magnified the inequities faced

Coming Next Month in COMMUNICATIONS

It is easier to increase access to com- by families with limited technology
puting education in well resourced access or literacy and limited skills or
schools, which tend to be more ho- time to support their students’ online
mogenous. Increasing access, partic- learning. In many communities, par-
ipation, and experience to students ents or guardians do not see technol-
who have been historically under- ogy careers as achievable pathways for
represented in computingc requires their children.
broad-based teams who are commit- Computer science may be an aspi-
ted to making change through data- ration, but it is not a priority for many
driven strategic efforts.5 school administrators and princi-
Why is it difficult to develop and pals.4 School leaders find themselves
maintain goals for BPC in state edu- with a limited understanding of what
cation systems? First, such goals are CS is and why all students need to
hard to define. How can progress be learn it. Pressures to address tradi-
measured? What data is informing tional, tested curricula, and a lack of
the development of goals? How do we resources and teachers to meet the Does Facebook Use
know that changes, be they state edu- demand for a new discipline further Sensitive Data for
cation policies or classroom practices, compound the challenges. The pres-
Advertising Purposes?
result in engaging a more diverse set sures placed on schools, school lead-
of students in CS? Also some goals, ers, teachers and students by stan-
while they seem ambitious, are not dardized testing of reading, writing, Digital Instruments
enough to broaden participation. En- mathematics, and science, particular- as Invention Machines
suring there is a CS teacher in every ly in schools where large numbers of
school is not enough. Many schools are students struggle with such tests, can How to Transition
highly diverse, but the computer sci- result in lower priority for computer
ence classes tend to be mostly White or science curricula. The “time in the to Microservice
Asian and male. If the CS teacher offers day” for traditional subjects can push Architecture
one small elective class, most students CS off as an elective. The pandemic
still do not have access. Making com- has increased online learning, limit- Security Multiparty
puter science classes a requirement is ing hours of interactive learning and Computation
not enough. Many states are enacting exacerbating the availability of time
a requirement that CS classes be made CS curricula. CS teachers are hard to
available in every school, but are not find and retain. Too few university pro- The Ethics of
providing funding for teacher profes- grams are training potential computer Zero-Day Exploits
sional development nor are intention- science teachers. Professional devel-
ally defining what content makes a opment for in-service teachers is lim- The Die is Cast:
course a CS course. Schools can often ited. Teachers are often shifted from
Hardware Security
meet this requirement by simply pro- teaching CS to perceived higher prior-
viding access to online classes. Howev- ity subjects such as math or science. Is Not Assured
er, historically marginalized students The outcome is too often restricted op-
face numerous barriers to success in portunities for students who have had The Identity in
online courses, so the result is still too historically limited access to learning Everyone’s Pocket
few students getting access.3 the computing skills they need to suc-
The historic inequities embedded in ceed in any career, much less having
our social systems make the challenges been encouraged into computer sci- Constant Overhead
in reaching “CS for All” enormous, but ence pathways. Computer science edu- Fault-Tolerance
not insurmountable. COVID-19 and cation must not be seen as yet another
the onslaught of virtual educational, burden on schools and teachers. It is SkyCore: Moving Core
professional, and community spaces beyond time that we flip the educational
to the Edge
have brought new awareness to our script and elevate “CS for All” as a prior-
dependence on technologies and the ity and see its diversity, equity, and in-
computer science that is inherent in clusion goals as an opportunity to equip
every aspect of our lives. The pandemic all graduates with the skills they need to
succeed in the 21st century. Plus, the latest news about
c See “Women, minorities (African Americans/ Several ECEP states are rising to physics and AI, how big data
Blacks, Hispanic Americans, American Indi- is used to understand
this challenge by putting broadening
ans, Alaska Natives, Native Hawaiians, Native epidemics, and the pros
Pacific Islanders, and persons from economi-
participation in computing goals at and cons of general vs.
cally disadvantaged backgrounds), and persons the center of their advocacy and policy specialized supercomputers.
with disabilities” from https://bit.ly/2Hl0uzZ efforts. They are creating plans for

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 23

viewpoints

growing CS education and broadening relies on four key actions: identifying

participation at the same time. A pri- a member of the Core Team to serve
mary focus for ECEP 2.0 is to support as the diversity coordinator; requir-
states to develop ways to disaggregate ing that all state-approved CS content
and track data that measure BPC providers meet a rubric for culturally
goals. Because computing is a new responsive and accessible curricula
2018 JOURNAL IMPACT
subject for many states, it has been and professional development; of-
FACTOR: 6.131 difficult to define computing educa- fering an online diversity course for
tion and even more difficult to mea- all RI CS teachers; and partnering

ACM Computing
sure the demographics of students with research projects to identify the
engaged in computing. Due to con- level of engagement of students with
cerns about confidentiality and mask- disabilities. CS4RI also supports dis-
Surveys (CSUR) ing identifiable information when stu- trictwide planning workshops for CS
dent numbers are small, it has been educators and administrators that in-
challenging to disaggregate and report clude inclusion and diversity training.
ACM Computing Surveys on ethnicity, gender, and disability. As Rhode Island’s efforts are showing
(CSUR) publishes a collective impact network, ECEP is results. By the end of 2017, the state
helping state leaders develop consis- had reached its goal of having CS of-
comprehensive, tent, replicable strategies for analyz- fered in every public elementary, mid-
readable tutorials and ing state data through the lens of BPC. dle, and high school in at least one of
Two examples, from Rhode Island and three modes: standalone courses; CS
survey papers that give
Nevada, where state goals and plans lessons integrated into existing cours-
guided tours through explicitly aim to increase the number es; or access to CS courses during the
the literature and and diversity of students getting ac- school day from another Rhode Island
cess to high-quality CS education, are school. By the end of 2018, 46% of high
explain topics to those described here. schools were offering AP computer
who seek to learn the science, a 26% increase from 2015. Fe-
Rhode Island male students in AP courses rose from
basics of areas outside
In 2016, Rhode Island Governor Rai- 5% in 2015 to 30% in 2018. And the
their specialties. These mondo launched Computer Science number of female CS major college
carefully planned and for Rhode Island (CS4RI)d with the aim graduates in the state rose from 17% to
of having computer science taught in 25% over the same period.
presented introductions every public school. CS4RI brought
are also an excellent together state government and its Nevada
education department, K–12 public Nevada’s Board of Education vice
way for professionals to
education, universities, industry and president Mark Newburn and the
develop perspectives on, nonprofits. The state provided more STEM Coalition K–12 CS Task Force
and identify trends in, than $200,000 per year for the past leveraged membership in ECEP to cre-
three years to support staff, events, ate a CS coalition that built a cohesive
complex technologies. and teacher professional develop- statewide CS education expansion ef-
ment. CS4RI’s team selected provid- fort. Over two years, NV leaders con-
ers of CS educational content, cur- tributed to the national K–12 CS
riculum, and teacher professional Framework and also developed a pro-
development that would support RI’s cess for writing CS standards for NV.
initiative to bring computer science to The broad-based team framed a strat-
all students. One of the guiding prin- egy to broaden participation and built
ciples when the state developed CS statewide support for CS education.
standards stated: “All students regard- Concurrently, the Nevada legislature
less of age, race, ethnicity, gender, identified gaps in K–12 CS education
socioeconomic status, special needs, instruction and drafted Senate Bill
English proficiency, or any other de- 200. Passed in June 2017, the bill re-
mographic will have the opportunity quires all public high schools, char-
to participate in computer science. ter schools, and university schools
For further information The content and practices of the stan-
dards will be accessible to all.”e
for gifted students to offer a state-ap-
proved CS course by 2022. Notably,
and to submit your Rhode Island’s strategy for broad- the legislationf states: “These schools
ening participation in computing must also make efforts to increase
manuscript, enrollment of girls, students with
visit csur.acm.org d See https://bit.ly/2T5IYT3
e See https://bit.ly/3m3yjof f https://bit.ly/3oQcUkR

24 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 viewpoints

disabilities, and underrepresented participation and enable all stu-

minorities in the field of computer sci- dents to succeed, no matter the tools
ence as identified by the state board.” In the movement or delivery mode.
Senate Bill 200 also mandates that Ne- toward “CS for All,” As members of ECEP, Rhode Island
vada’s half-credit Computer Literacy and Nevada have developed and shared
graduation requirement must now in- state K–12 education their approaches to expanding CS edu-
clude at least 50% of computer science systems often focus cation in ways that focus on broaden-
content, thus guaranteeing all stu- ing participation among students un-
dents receive some exposure to com- on access, but derrepresented in computing fields.
puter science concepts. too often equity The 22 ECEP states and the territory of
After the bill was passed, NV ECEP Puerto Rico are working hard to ensure
leaders and state department of edu- and inclusion broadening participation in computing
cation staff organized a statewide CS are afterthoughts remains a top priority and that the poli-
education summit at which stake- cies and resources target that goal.
holders from K–12 and postsecondary in policy and
education, government and industry strategy efforts. References
1. Adrion, R. et al. Broadening access to computing
learned strategies to support equi- education state by state. Commun. ACM 59, 2 (Feb.
table CS education and wrote strate- 2016), 32–34.
2. Code.org. State Tracking 9 Policies (Public), (Oct. 2020);
gic plans that focused on broadening https://bit.ly/3dDUq1H
participation, which were collected 3. Dynarski, S. Online courses are harming the students
who need the most help. New York Times. (Jan. 19,
and shared. The state education de- documents. Nevada’s CS team’s work 2019); https://nyti.ms/3dFnmpX.
4. Herold, B., and Schwartz, S. Principals warm up to
partment also followed up with an has never been about the number of computer science, despite obstacles. Education Week
online seminar series and training for students reached, it has been inten- 37, 27 (Apr. 18, 2018), 24–25; https://bit.ly/31gakKz
5. Ladner, R. and Israel, M. Broadening participation “for
school counselors. tionally and systematically focused on all” in “computer science for all.” Commun. ACM 59, 9
Nevada allocated $1.4 million in offering high-quality computing edu- (Sept. 2016), 26–28.
6. Nevada Department of Education. Status Report
state funding to districts and charter cation to all students and reaching Senate Bill 200 Computer Science Education. (2019);
schools for teacher and school leader those underrepresented in computing. https://bit.ly/2T7ZIJy
professional development, curricu-
lum, and equipment. Nevada’s goal to Conclusion: Getting W. Richards (Rick) Adrion ([email protected])
is professor emeritus at the College of Computer and
broaden participation was integrated Access to Everyone Information Sciences at the University of Massachusetts
into how it distributed state funds. In the movement toward “CS for All,” Amherst, USA, and was UMass ECEP principal
investigator 2012–2019.
Each applicant had to show how funds state K–12 education systems often
Sarah T. Dunton ([email protected]) is Director of
would be used to increase the enroll- focus on access, but too often equity the Expanding Computing Education Pathways (ECEP)
ment of females, students with dis- and inclusion are afterthoughts in Alliance at the Massachusetts Green High Performance
Computing Center, Holyoke, USA.
abilities, and underrepresented mi- policy and strategy efforts. Even man-
norities. Of the 17 districts in NV, 14 dates to offer CS in every school may Barbara Ericson ([email protected]) is assistant
professor of information, School of Information at the
applied for funding, and nine charter not mean that students with disabili- University of Michigan, USA, and former co-principal
schools were funded as well.6 ties are fully included, or that every investigator of Georgia Tech ECEP.

Nevada’s commitment is showing girl participates, or that students in Renee Fall ([email protected]) is senior research scholar at
the National Center for Computer Science Education at
dramatic results. From 2016 to 2019, schools that offer online CS courses the College of St. Scholastica in Duluth, MN, USA, and
the number of students taking com- succeed in those courses. As states former co-principal investigator of UMass ECEP.
puter science doubled and the num- enact new policies, we recommend Carol Fletcher ([email protected]) is director of
EPIC at The University of Texas at Austin, USA. She is the
ber of high schools offering computer the following actions: current principal investigator of the ECEP Alliance.
science more than tripled. From 2017 ˲ Include people with expertise in
Mark Guzdial ([email protected]) is professor of
to 2019, Nevada students taking the diversity and inclusion on the team electrical engineering and computer science in the College
Advanced Placement Computer Sci- developing policies and plans. Broad- of Engineering at the University of Michigan, USA, and was
Georgia Tech ECEP principal investigator 2012–2018.
ence Principles exam increased by based teams bring a wide range of per-
fivefold. Hispanic students’ repre- spectives that are vital to serving a wide Thanks to ECEP members from Rhode Island (Carol M.
sented 40% of all students in 2019 (a range of students. Giuriceo, Rhode Island College and Victor Fay-Wolfe,
˲ Tie resources to goals to broaden University of Rhode Island) and Nevada (Cindi L. Chang,
more than eight times increase over Nevada Department of Education, and Mark Newburn,
2017), and female students represent- participation. Provide funding to all Nevada Board of Education) for their contributions to this
column.
ed 38% of Nevada’s CSP exam-takers, districts, not just the well-off schools
up from 30% in 2017.g that are poised to take advantage of This material is based upon work supported by the
National Science Foundation (NSF) under Grant Nos.
Nevada continues to focus on competitive grant funds. Ask districts NSF-CNS-1822011, NSF-CNS-1228355, and
broadening participation as it cre- to show how new resources will in- NSF-CNS-1228352. Any opinions, findings, and conclusions
or recommendations expressed in this material are those of
ates a state strategic plan, pre-service crease participation by historically un- the author(s) and do not necessarily reflect the views of the
National Science Foundation. Expanding Computing Education
and in-service professional develop- derrepresented students. Pathways (ECEP) is part of the National Science Foundation’s
ment for K–12 teachers, and guidance ˲ Ensure state-approved CS curricu- Broadening Participation in Computing Alliances (NSF BPC-A).
lum and professional development
g See https://bit.ly/2T5IKeF incorporates strategies to broaden Copyright held by authors.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 25

V
viewpoints

DOI:10.1145/3430370 Peter J. Denning

The Profession of IT
Navigating in Real-
Time Environments
An interview with Jim Selman.

J
I M S E L M A N H A S been a profes-
sional leadership coach for
over 30 years. He frequently
encounters executives and
team leaders who are dumb-
founded because the world is chang-
ing so rapidly and sometimes chaoti-
cally that their best laid plans are
useless and ineffective. Many comput-
ing professionals have a similar experi-
ence today after the COVID-19 ava-
lanche swept through. In his recent
book, Living in a Real-Time World, he
summarized his conclusions about
what professional leaders should learn in
order to be effective in this environment.
I explored this issue with him.
—Peter J. Denning

DENNING: Your book Living in a Real-

Time World, 6 Capabilities to Prepare
Us for an Unimaginable Future seems
particularly relevant given what’s hap-
pening around the world. What do you
mean by ‘real-time world’?
SELMAN: I started my career in the
1960s in IT as a programmer and sys-
tems analyst. At the beginning, com-
puters were 100% information process- sponds rapidly and effectively to inputs many people. Today, in my opinion,
ing machines. As processor speeds as they appear, without knowing when we need to be more like the real-time
got faster and faster, the gap between or if they will occur. computers, responding to what actu-
inputs and outputs got shorter and I’m using the term in the same ally appears rather than what our best
shorter to the point of being impercep- way. The technology-charged world is laid plans expect to appear. We need to
tible—computers transformed from changing so rapidly that many of our transform our worldview, our practic-
informing machines to performing plans and expectations are dashed by es, and our skills to successfully navi-
machines. The idea of “real-time” com- surprises. The future we imagined gate a reality that is increasingly unpre-
puting and capabilities like process- and planned for never appears. This dictable and beyond our control.
control came into existence. Real-time is why I call the future unimagina- What do you think is driving all this
computing means computing that re- ble. This is immensely frustrating to chaos and unpredictability?

26 COM MUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 viewpoints

I don’t know why the world is the way are playing. Surrender is a choice. It
it is or what causes anything really given isn’t succumbing or being defeated.
the overwhelming complexity we’re ex- “Computer ˲ Being or what I call the Art of Con-
periencing. I suspect that computers, professionals text. Computer professionals know
networks, and the whole of technological better than most that computers op-
progress is a big part of what has accel- in the future erating without context are limited to
erated the pace of change beyond our need to be purely mechanical operations. In the
comprehension. The massive increas- human domain, context is all-impor-
es in efficiency and knowledge over the philosophers, tant. It gives us meaning and purpose
past few decades have been amazing leaders, and and is the key to have freedom and
and beneficial. Yet, as you pointed out power. In a real-time world we need to
in your last piece on the current “ava- navigators be continuously aware of the fact that
lanche” of change, the world economy in addition to being we have a choice about our way of be-
is subject to “avalanches”—disruptive ing—the embodied way we relate to
changes that sweep away jobs, iden- technical experts.” our context. When you shift your way
tities, wealth, and opportunities. An of being, you shift the context in which
avalanche leaves people feeling lost, you observe and that will always open
confused, left behind, or afraid. While new choices and possibilities that you
many avalanches are small, affecting had previously been unable to perceive.
only a limited segment of the econo- ˲ Listening or what I call the Art of
my, COVID-19 is an avalanche that no Mastering Moods. I am not talking
one could escape. It triggered other av- thing else. We learn from the past and we about hearing sounds but something
alanches such as the collapse of some gather techniques and recipes. We apply much deeper—our background of
industry sectors (for example, air trans- all that to the current situation in the in- awareness and interpretation of what
port), oil prices, international trade, terest of controlling future outcomes or other people are saying. We call this
and some higher education systems. solving specific problems. This mindset background our mood. Our mood ori-
Like it or not, this avalanche calls us all doesn’t work in a real-time world. When ents us to interpret the world in par-
to navigate unimagined environments. you make decisions and commitments ticular ways. Our mood includes our
The ‘Four Horsemen” of Silicon Val- and allocate resources based on an as- stereotypes, prejudices, and “mental
ley (Amazon, Google, Facebook, and sumption that the future will be much models.” Our moods, our thinking,
Apple) are leading the charge into a fu- like the past, you only guarantee dis- and ultimately our behaviors are in-
ture that is unimaginable. But the cur- satisfaction and frustration when that separable from our listening. If we
rent pandemic, threats from climate future does not appear. This simply want to be successful navigators, we
change, and unprecedented levels of creates a counterproductive vicious cy- must become aware of our moods
unemployment make us realize this cle that worsens the situation and is of- and shift them to ones that accept the
is the unimaginable future—a perfect ten self-destructive. world as it is and give us choice about
storm of disruptive change. Why we got What do we need to learn in order to what we do next.
here isn’t very relevant. The only ques- regain our effectiveness in this kind of ˲ Communicating or what I call the
tion is what to do now—how do we nav- environment? Art of Relating. Everyone knows the
igate, make choices, plan and invest in In my view, human beings already importance of communication and re-
the future when we don’t trust our pre- are fully equipped and have all the ca- lationship. Not everyone appreciates
dictions and we have little control over pabilities to become successful naviga- that communication and relationship
changes that we cannot comprehend? tors to succeed in a world of permanent are two sides of the same coin and are
I am fond of Star Trek as a metaphor uncertainty. For example, everyone has learnable skills. In a real-time world
for this situation …. we’re “going where an ability to read even if they are ini- mastering how we relate to others,
no one has gone before” and we have no tially illiterate. What are the capabili- how we relate to our circumstances,
certainty of our destination or maps to ties inherent in all of us that allow us and even how we relate to ourselves
guide us. I suggest that our traditional to be effective in a real-time world? In becomes essential if we’re to stay cen-
notions of leadership are of necessity my book, I mention six of these inher- tered in the flow of life and be free to
transforming from “Leader” to “Naviga- ent capabilities: continually choose our actions.
tor.” Navigators don’t know any more ˲ Accepting or what I call the Art of ˲ Appropriating or what I call the Art
than anyone else, but they keep us cen- Surrender. This means accepting the of Situational Learning. Most things
tered in where we are and where we’ve world as it shows up. Acceptance cul- are changing faster than we can com-
come from. With that we can move and tivates our capacity to choose. When prehend and what we’re learning is
shape the new world. you are resisting or reacting you are often obsolete before we learn it fully.
What are the traditional ways we not choosing. When you have discov- This is especially obvious in the techni-
have coped with disruptive change? ered the impossibility of winning the cal fields. In the distant past, human
Why aren’t they working anymore? game you are playing you can choose beings viewed learning as a team ef-
Historically, we’ve dealt with change to surrender. You can’t play a new fort and the idea of an individual being
pretty much the way we deal with every- game until you give up the game you the container of knowledge made no

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 27

viewpoints

sense. Today there is so much informa- possibility there might be a better way
tion available to us that it still makes no of dealing with things. I don’t ask peo-
sense that a single person can contain ple to believe anything, or agree with
all knowledge. Today we can rebuild anything or even understand every-
practices for spreading the learning thing. I ask only they consider having
across multiple people and then bring- a serious conversation about what they
ing it all together through collabora- are already dealing with and be open to
Digital Threats: tion and focus on an objective. testing or trying some of the practices
˲ Caring or what I call the Art of Love. I suggest. It doesn’t take long for them
Research and Practice One aspect of the emerging reality is to engage and realize they are learning
the recovery of our appreciation for a kind of new language—a language
each other and the planet. Every age is of leadership. It is as if they learned a
characterized by core values and ide- higher level programming language,
Digital Threats: Research als. For most of the last few centuries which opens choices and possibilities
the focus has been on production and not previously available.
and Practice (DTRAP) is a control. In our real-time world, there How would this help computing
peer-reviewed journal that is a shift from these to what I believe is professionals?
targets the prevention, more fundamental and relevant—care. It’s pretty obvious that computers
Nothing matters without care. are expanding exponentially. Look at all
identification, mitigation, These responses may strike some the speculation around AI, robots, and
and elimination of digital people as “stuff we all learned in kin- virtual realities. Computers may be the
dergarten.” Or perhaps as “soft” and most critical element driving us into a
threats. DTRAP aims to lacking in rigor. You are saying we real-time world. Historically, informa-
bridge the gap between really didn’t learn this in kindergar- tion systems and processes are mate-
academic research ten or any other part of our school- rial processes that move signals repre-
ing. Why do you say this? Why is your senting symbols in time and space, in a
and industry practice. framework rigorous? cause-effect world. The emerging world
Accordingly, the journal At the end of the day our “reality” is is different. The big question is where
a function of action—our future will human beings fit into the technologi-
welcomes manuscripts
be shaped by our actions. One of the cal picture. I believe the answer is in
that address extant premises in my work is if something the synthesis between the physical and
digital threats, rather isn’t observable, it isn’t actionable. social sciences. Computer profession-
This is why I prefer a more phenom- als in the future, I believe, need to be
than laboratory models enological approach in which we see philosophers, leaders, and navigators
of potential threats, and we swim in an ocean of language, not in addition to being technical experts.
presents reproducible a pool of words. We inhabit networks They cannot be relegated into narrow
of conversations over which we have technical niches. To me, this is the fu-
results pertaining to little visibility or control. The skill I ture essence of professionalism.
real-world threats. want to cultivate looks closely at lan-
guage to see where action is created Suggested Readings
1. Flores, F. and Winograd, T. Understanding Computers
in conversations. Action is created and Cognition. Addison-Wesley, 1987.
by commitments. We can be rigor- 2. Selman, J. Living in a Real-Time World. Independently
published, 2019; available at amazon.com.
ous observers of the commitments 3. Selman, J. Leadership. CreateSpace independent
publishing platform, 2016; available at amazon.com.
we make in our conversations, and
whether our actions line up with our
Jim Selman ([email protected]) is a seminal
commitments. leader in the theory and practice of business coaching.
What is your experience? Are proj- He contributed several new concepts and techniques
to the field of management, notably organizational
ect leaders receptive to the idea of real- transformation, coaching, the Merlin method for
time world? designing the future, breakthroughs, and breakdowns.
He developed new approaches for leaders to producing
If you believe a new post-avalanche broad “paradigm shifts.” He is a former partner in the
firm of Touche Ross (Deloitte Touche), co-founder
reality is emerging, you must confront and CEO of a consulting network, Transformational
many assumptions you’ve taken for Technologies, and founder and CEO of Paracomm
Partners, Petaluma, CA, USA.
granted but no longer relevant or work-
For further information able. Even our common sense breaks Peter J. Denning ([email protected]) is Distinguished
Professor of Computer Science and Director of the
down. Are people receptive to this in-
and to submit your terpretation? Often not initially. But
Cebrowski Institute for information innovation at the
Naval Postgraduate School in Monterey, CA, is Editor

manuscript, most people know our current way of ACM Ubiquity, and is a past president of ACM. The
author’s views expressed here are not necessarily those
of approaching things isn’t working of his employer or the U.S. federal government.
visit dtrap.acm.org so well and something is profoundly
missing. They are actually open to the Copyright held by author.

28 COMMUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

V
viewpoints

DOI:10.1145/3430378 George V. Neville-Neil

Article development led by
queue.acm.org
your system that is probably preeminent
in your mind, or at least it should be.

Kode Vicious
Sometimes a dead feature will have a
noisy minority of users or an internal
champion who has tended that feature

Removing Kode
for many years. At this point, the removal
becomes a political (that is, human)
problem. There are several ways to solve
Dead functions and dead features. political problems at work, but some of
those will get you 10 to 20 years in jail if
Dear KV, not simply conditionally compiled-out you carry them out. If the noisy minority
Your columns often discuss issues test code (see my February 2017 col- is very small, it is possible that feature
that come up when adding new code, umn, “The Unholy Trinity of Software could be broken out into its own separate
but when is the right time to remove Development”) then that code should program so that the code is no longer
a piece of code? I have been working be removed immediately. A good com- part of the larger whole. If the problem
on an enormous legacy code base and piler or other tool will tell you when is developers, well, it is time to give
most of the questions that come up in you have dead code, which should them a new challenge, far away from
our stand-up meetings concern which make the job fairly straightforward. their pet features. Once you have devel-
modules we think we can remove from Complications arise when you have opers with pet features, you have a very
the system. code that is infrequently used but has different sort of management prob-
Ready for Removal been in the system for a long time. At this lem, one I will not address here.
point, you have to do a bit more thinking You will notice I did not give you a
about your code and whether or not the definite timeline in answer to your ques-
Dear Ready, code you are looking at is—for want of a tion, and that is because there really
I am surprised and amused at the nar- better term—nearly dead. Any system that isn’t one other than removing truly
rowness of your question. Most software survives for a number of years will tend to dead code as soon as it is dead. For fea-
developers, when confronted with a leg- grow functionality that is used—some- ture removal, that really depends on
acy code base, simply want to throw it times briefly—and then discarded or ig- how you deal with your users and devel-
out and start again. I have to say this hap- nored. The logic for not removing dead opers. A good rule of thumb, though, is
pens to me at least once a week, just features is usually stated as, “If it ain’t to remove features only at a major re-
looking around at the astounding moun- broke ... ” which is actually quite foolish. If lease so as to limit the level of surprise.
tains of legacy that pass as software. your software is so fragile that removing a KV
Removing dead code from systems feature breaks the whole system, you
is one of KV’s favorite coding pastimes have much bigger problems than need-
Related articles
because there is nothing quite like that ing a bit of code cleanup, and that means on queue.acm.org
feeling you get when you get rid of the system as a whole probably needs
Code Hoarding
something you know was not being rototilling if not an outright rewrite.
Kode Vicious
used. Code removal is like cleaning A dead feature is different from a https://queue.acm.org/detail.cfm?id=2897034
house, only sometimes you clean house dead function. Dead features are either
Surviving Software Dependencies
with a flamethrower, which, honestly, is completely unused or used by only a Russ Cox
very satisfying. Since you are using a minority of users. The risk with dead https://queue.acm.org/detail.cfm?id=3344149
version-control system (you had better or nearly dead features is that they Writing a Test Plan
be using a VCS!), it is very easy to re- leave a larger attack surface in your Kode Vicious
move code without worry. If you ever code. Once upon a time, we also cared https://queue.acm.org/detail.cfm?id=3294770
need the code you removed, you can re- that dead features left code in the exe-
trieve it from the VCS at will. cutable that bloated the system, but George V. Neville-Neil ([email protected]) is the proprietor of
Neville-Neil Consulting and co-chair of the ACM Queue
As to when you should be removing only those of us working in the embed- editorial board. He works on networking and operating
systems code for fun and profit, teaches courses on
code, there are several answers. The ded-systems area seem still to care various programming-related subjects, and encourages
first, of course, is that if you have truly about that. For modern server-based your comments, quips, and code snips pertaining to his
Communications column.
dead code that can no longer be reached systems, it is the risk of the dead feature
from anywhere in the system and that is giving an attacker a place to infiltrate Copyright held by author.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 29

V
viewpoints

DOI:10.1145/3431480 Margaret O’Mara

Viewpoint
Silicon Politics
Tracing the widening path between
Silicon Valley and Washington, D.C.

T
ECH AND POLITICS don’t mix.
That has been the story
Silicon Valley leaders have
broadcast to the world
since the region first sprang
into the forefront of public conscious-
ness as the land of silicon chips, per-
sonal computers, and video games. It
is an attitude in keeping with the cel-
ebration of rugged individualism and
disdain for centralized political power
that has been part of American politi-
cal culture since the nation’s found-
ing, ideas that gained additional al-
lure amid the stagflating malaise of
the post-Vietnam, post-Watergate
1970s. In the Reagan Revolution year
of 1980, the sole election-year com-
mentary in the microelectronics-
industry newsletter InfoWorld was a
cartoon tucked into a bottom corner U.S. President Donald J. Trump meets with Facebook CEO Mark Zuckerberg at the White
of the editorial page. “I was going to House in Washington. D.C., in September 2019.
keep track of all the candidates’ sig-
nificant statements,” one man sighed Six years ago, as I embarked on the re- Democrats like Bernie Sanders and Eliz-
to another as they stood in front of a search process for my recently published abeth Warren while members of tech’s
computer terminal, “but there’s no history of Silicon Valley, The Code, I told a C-suites threw high-dollar fundraisers
way to process an empty disk.” longtime tech-industry veteran that part for Democratic centrists and the Re-
Four years later, Steve Jobs declared, of my goal was to show how tightly the publican incumbent alike. The size and
without embarrassment, that he had industry’s rise and evolution intersected scale of tech’s biggest platforms and
never voted in his life. 1990s-era mo- with modern American political history. products triggered sharp public criti-
guls including Bill Gates and Jeff Bezos “Can I tell you something?” my inter- cism and fresh lawmaker scrutiny. Tech
OFF ICIAL WH IT E H OUSE PHOTO BY J OIYCE N. BO GHO SIA N

ducked questions about their voting viewee interjected, not unkindly. “If you regulation was the rare issue on which
preferences for years. In the early 2000s write about tech and politics, that book the two parties seemed to agree.
the loudest and most unapologetically is going to end up on the remainder ta- This is not as much of a turnabout as it
political voices coming out of Silicon ble. Because there’s no story there.” seems. Silicon Valley’s mythology of in-
Valley were libertarians such as PayPal My proposition is not such a tough dependence to the contrary, politics and
cofounder and Facebook board mem- sell in the America of 2020. Before the government are absolutely central to its
ber Peter Thiel. Politicians of both par- COVID-19 pandemic ground normal life story. This was the case in other industri-
ties long courted Silicon Valley’s affec- to a halt, tech CEOs ricocheted between alized nations as well, but only in the
tions, but for many in tech, politics was White House photo-ops and scorching U.S. have tech entrepreneurs and many
something to be publicly ignored, if presidential tweets. Many in the tech of their allies embraced such a thor-
not actively disdained. rank-and-file donated to progressive oughly market-driven understanding

30 COMM UNICATIO NS O F THE AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 viewpoints

of their accomplishments. American

technologists’ collective amnesia has INTER ACTIONS
served a political purpose of its own, in- Silicon Valley’s
stilling a sense of entrepreneurial agen- mythology
cy amid intensive government mobili-
zation and market intervention, and of independence to
also building a political case for, among the contrary, politics
other things, self-regulation of online
platforms. To understand how and why and government
to grapple with today’s tech policy land- are absolutely
scape, we must reckon with this history
and the institutional reality that the U.S. central to its story.
government built the tech industry, but
it did so in a way that helped the indus-
try’s leaders believe they did so all on ACM’s Interactions magazine
their own. explores critical relationships
One point of connection comes with Sending satellites into orbit, or as- between people and
an institutional symbiosis familiar to tronauts to the Moon, required light, technology, showcasing
any technologist who has worked in powerful electronics like the semi- emerging innovations and
an NSF-funded laboratory or entered conductors and integrated circuits industry leaders from around
a DARPA robotics competition: the developed by Fairchild. The technol- the world across important
academic-military-industrial complex ogy was so new that only Fairchild and applications of design thinking
established in the early Cold War. As other specialist electronics firms like it and the broadening field of
historians of science remind us, this could supply what the space program interaction design.
purportedly apolitical Big Science was needed. The spike in demand coin- Our readers represent a growing
in fact deeply political. Competing cided with budget-minded Defense community of practice that is
with the Soviets provided the rationale Department reforms to the prime con- of increasing and vital global
for unprecedented public investment tracting system that dislodged larger importance.
in everything from university science and more well-established firms. The
and engineering programs to the rock- result was that in Fairchild’s early
ets of the Apollo program. years, this granddaddy of all Silicon
Yet as significant as the money itself Valley startups had more than 80% of
was the way in which it flowed: indirect- its book of business come from gov-
ly, via grants and contracts to private ernment contracts. Large purchase
defense contractors, to research univer- orders from the Apollo program for in-
sities, to industrial research labs. This tegrated circuits drove down the price
obscured the extent to which govern- and allowed chipmakers like Fairchild
ment spending was undergirding the to scale up production, enabling a
enterprise, yet another example of the commercial market to emerge by the
indirect and privatized process of state- end of the 1960s.
building that has been a hallmark of A similar combination of entrepre-
American federalism since the 19th cen- neurial energy and government sub-
tury. In the early Cold War period, this sidy and encouragement lies behind
structure spurred innovation and mar- the origins and evolution of the Sili-
ket competition in advanced electronics con Valley-based personal computer
industries at a time when commercial industry. Unlike the Cold Warriors
demand was nearly nonexistent, creat- who inhabited the early electron- To learn more about us,
ing a foundation for later growth of con- ics industry, the personal-computer visit our award-winning website
sumer and business markets. makers hailed largely from a Viet- http://interactions.acm.org
Take for example the case of the sili- nam generation who wanted noth-
con semiconductor industry that gave ing to do with the military-industry Follow us on
Facebook and Twitter
the Valley its name. Fairchild Semicon- complex and instead saw computers
ductor, the pioneering startup whose as tools of liberation from the estab- To subscribe:
many spinoffs include Intel and the lishment’s rules and gatekeepers. http://www.acm.org/subscribe
legendary venture capital firm Kleiner Instead, they crafted an anti-politics
Perkins, had the lucky break of incor- that was in itself a political position
porating in September 1957, only a of its own. Alternatively called “new Association for
few weeks before the Soviets launched communalism,” “cyberlibertarian- Computing Machinery
Sputnik into orbit and sent the Ameri- ism,” or “the Californian ideology,”
can space program into overdrive. this techno-utopian variant of 1960s

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 31

viewpoints

countercultural thought was both the nation’s most populous state.

deeply cynical—governments and The well-trod path between Silicon
traditional authorities were not to be Valley and Washington became even
trusted—and earnestly techno-op- wider in the 1990s, as the commercial-
timistic.a Place a computer on every ization of the Internet spurred a new
desk, and connect their users in non- set of technologists to collaborate with
hierarchical networks of communica- lawmakers in an effort to influence the
Digital Government: tion and creation, and you would fix rules and regulations that would guide
what ails an ailing world. the new platform. Silicon Valley dot-
Research and Practice But the proclamation and dissemi- com companies and their allies then
nation of this ideology obscured the were the David to the Goliath of the tele-
fact that, particularly for those running com industry, and a main goal of their
high-tech enterprises, tight connec- engagement in the legislative debates
Digital Government: tions to traditional political institutions around telecommunication reform
and political processes remained an and Internet regulation was to ensure
Research and Practice essential part of doing business. Val- that online information flowed as freely
(DGOV) is an Open ley leaders were active and consistent as possible—without the Comcasts and
Access journal on lobbyists for help from Washington as AT&Ts of the world choking it off.
the industry weathered the economic In a divided Washington, the infor-
the potential and storms of the 1970s and launched a mation-should-be-free message was
impact of technology new set of consumer-facing companies one with appeal to both the Demo-
on governance in the 1980s. Unlike today’s armies of crats in the Clinton-Gore White House
hired tech lobbyists who line K Street and the Republicans in the Gingrich-
innovations and its and advocate on companies’ behalf, led Congress. The result was that the
transformation of early lobbying efforts involved the CEOs Internet industry was allowed to regu-
themselves, bringing a little California late itself—precipitating an explosion
public institutions. It casual to Capitol Hill’s marble hallways. in online content, and enabling the
promotes applied and Semiconductor executives de- rise of new platforms and products
empirical research scended on Washington to plead for that now form the core of tech’s most
protection from the heavily subsidized significant companies.
from academics, Japanese competitors who were flood- So when it comes to tech and politics,
practitioners, designers, ing, and dominating, the chip market. there is a story there—a history urgently
and technologists, Venture capitalists and electronics relevant to the technology landscape of
executives successfully petitioned for 2020. The long history of defense work,
using political, policy, reductions in the capital gains taxes and of companies that were both feder-
social, computer, that, in their telling, scared investors al contractors and merchant producers,
and data sciences away from making “risk capital” bets helps make legible the remarkable role
on new tech companies. that tech giants now play in national se-
methodologies. Even the proud nonvoter Steve Jobs curity and defense contracting. The con-
spent two weeks on the Hill in 1982 sistent lobbying for tax and regulatory
attempting to persuade lawmakers to relief belies the Silicon Valley mythos
pass a tax break that would encour- that its innovative story was something
age widespread use of computers in that happened on its own, without gov-
schools—an education market that, ernment help, and it extends the politi-
conveniently, Apple already dominat- cal history of tech beyond only the mili-
ed. The legislation failed in Washing- tary-industrial complex. The regulatory
ton, but then-Governor Jerry Brown of environment that governs the Internet
California swooped in to sign a similar today was the right response to the Sili-
state-level measure, thereby helping con Valley of the early dot-com era, but
ensure an Apple would be the first com- it now is as outdated as pop-up ads and
puter for most of the schoolchildren in Windows 95.
Linking the two histories together,
a Fred Turner, From Counterculture to Cyberculture:
and appreciating their relevance, is es-
For further information Stewart Brand, The Whole Earth Catalog, and the sential grounding for where tech, and we
who make and use it, might go next.
and to submit your Rise of Digital Utopianism. University of Chi-
cago Press, Chicago, 2006; Langdon Winner,

manuscript, “Cyberlibertarian Myths and the Prospects

for Community,” Computers and Society 27,
Margaret O’Mara ([email protected]) is Howard and
Frances Keller Endowed Professor in the Department of

visit dgov.acm.org 3 (Sept. 1997), 14–19; Richard Barbrook and History, University of Washington, Seattle, WA, USA.
Andy Cameron, “The Californian Ideology,”
Mute 1, 3 (Sept. 1, 1995); https://bit.ly/37p8Wt5 Copyright held by author.

32 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 V
viewpoints

DOI:10.1145/3387107 Yong Cheng, Yang Liu, Tianjian Chen, and Qiang Yang

Viewpoint
Federated Learning for
Privacy-Preserving AI
Engineering and algorithmic framework to ensure
data privacy and user confidentiality.

T
H ERE H A S BE E N remarkable
success of machine learning
(ML) technologies in empow-
ering practical artificial in-
telligence (AI) applications,
such as automatic speech recognition
and computer vision. However, we are
facing two major challenges in adopt-
ing AI today. One is that data in most
industries exist in the form of isolated
islands. The other is the ever-increas-
ing demand for privacy-preserving AI.
Conventional AI approaches based
on centralized data collection cannot
meet these challenges. How to solve
the problem of data fragmentation and
isolation while complying with the pri-
vacy-protection laws and regulations
is a major challenge for AI researchers
and practitioners.
On the legal front, lawmakers and
regulatory bodies are coming up with
new laws ruling how data shall be man-
aged and used.3 One prominent exam- medical records) prohibits free data models that do not rely on collecting
ple is the adoption of the General Data circulation and forces the data to exist data to a centralized storage where
Protection Regulation (GDPR) by the in data silos. Due to competition, user model training takes place. One at-
European Union in 2018. In the United privacy, data security, and complicated tractive idea is to train a sub-model at
IMAGE BY AND RIJ BORYS ASSOCIAT ES, USING SH UTT ERSTOC K

States, the California Consumer Priva- administrative procedures, even data each location with only local data, and
cy Act will be enacted in 2020. China’s integration among different depart- then let the parties at different sites
Cyber Security Law, came into effect in ments of the same company faces heavy communicate their respective sub-
2017, also imposed strict controls on resistance. As the old privacy-intrusive models in order to reach a consensus
data collection and transactions. way of collecting and sharing data are for a global model. In order to ensure
Under this new legislative land- no longer allowed, data consolidation user privacy and data confidentiality,
scape, collecting and sharing data involving different data owners is be- the communication process is care-
among different organizations are be- coming extremely challenging.10 fully engineered so that no site can
coming increasingly difficult, if not Data silos and privacy concerns are reverse-engineer the private data of
outright impossible. In addition, the two of the most challenging impedi- any other sites. In the meanwhile, the
sensitivity nature of certain data (for ments to the AI progresses. It is thus model is built as if the data sources
example, financial transactions and natural to seek solutions to build ML were combined. This is the idea be-

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 33

viewpoints

Figure 1. Categorization of FL.10 way,2,10 so that parties cannot access

the content of others’ data. FL is an al-
gorithmic framework for building ML
Data from A
models that can be characterized by
the following features.
˲ There are two or more parties in-
Labels
terested in jointly building a model.
˲ Each party holds some data that it
Samples

Horizontal can use for model training.

Federated Learning ˲ In the model-training process, the
Labels data held by each party does not leave
that party.
˲ The model can be transferred in
Data from B
part from one party to another under
Features
an encryption scheme, such that any
(a) party cannot reverse-engineer the data
of other parties.
˲ The performance of the federated
Data from A
model is a good approximation of an
ideal model built with centralized data.
Techniques for privacy-preserving
ML have been extensively studied,1 such
Samples

as employing differential privacy (DP)4

Vertical Federated Learning
Labels and secure multi-party computation.10
DP involves in adding noise to the train-
ing data, or using generalization methods
to obscure certain sensitive features until
Data from B the third party cannot distinguish the in-
dividual, thereby making the data impos-
Features sible to be restored to protect user privacy.
(b) However, DP still requires that the data
is transmitted elsewhere and involves a
trade-off between accuracy and privacy.
Federated Transfer Learning
Data from A
Categorization
FL can be classiffied into horizontal
FL (HFL),7 vertical FL (VFL),9 and feder-
Samples

ated transfer learning (FTL),6 accord-

ing to how data is distributed among
Data from B Labels
the participating parties in the feature
and sample spaces.10 Figures 1a–1c il-
lustrate the three FL categories respec-
tively for a two-party scenario.
HFL refers to the scenarios where
Features
(c) the parties share overlapping data fea-
tures, but differ in data samples. Dif-
ferent from HFL, VFL applies to the
hind “federated machine learning,” or been extended to support “cross-silos” scenarios where the parties share over-
“federated learning (FL)” for short.7,9 scenarios and B2B (business-to-busi- lapping data samples, but differ in data
ness) applications by the AI researchers features. FTL is applicable for the sce-
Definition in WeBank,a where each party has dif- narios when there is little overlapping
FL was practiced by Google for next- ferent sets of data features.5,6,9 in data samples and in features. We
word prediction on mobile devices.2,7 In a nutshell, a fundamental change also refer to HFL as sample-partitioned
Google’s FL system serves as an ex- in algorithmic design with FL is, in- FL,10 or example-partitioned FL,5 as in a
ample of a secure distributed learning stead of transferring raw data from matrix form, samples correspond to the
environment for B2C (business to con- sites to sites or to a server, we trans- rows and features correspond to the col-
sumer) applications where all parties fer ML model parameters in a secure umns (see Figure 1a). HFL is carried out
share the same data features and collab- across different horizontal rows, that
oratively train an ML model. Besides the a WeBank—China’s first Internet-only bank, is, data is partitioned by samples. We
B2C paradigm, the FL framework has see https://bit.ly/3o508ym also refer to VFL as feature-partitioned

34 COMMUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 viewpoints

FL,5,10 as VFL is carried out across dif- and inference procedures, such as There is an invoice agency A, which
ferent vertical columns, that is data is the convergence speeds of the exiting has invoice related data features, such
partitioned by features (see Figure 1b). training algorithms, in the existing as for the kth SME. There is
For example, when two organiza- works5,10 and references therein. bank B, which has credit-related data
tions providing different services (for features, such as and label
example, a bank and an e-commerce Application Examples Y (k) for the kth SME, with N > M. The
company), but having a large intersec- FL enables us to build cross-enterprise, agency A and bank B collaboratively
tion of common customers (that is, cross-data and cross-domain AI ap- build a risk control model for SME
aligned data samples), they may col- plications while complying with data loans using VFL.10
laborate on the different data features protection laws and regulations. It has Before model training, we need to
they respectively own to achieve better potential applications in finance, insur- find the common SMEs served by A and
ML models using VFL.5,9 ance, healthcare, education, smart city, B to align the training data samples,
and edge computing, and so forth.10 which is called private set intersection
Architecture We present here two FL applications or secure entity alignment.10 After de-
An FL system architecture can employ selected from the use casesb that have termining the aligned data samples be-
the client-server model, as shown in been deployed in practice by WeBank. tween A and B, we can then follow the
Figure 2a. The coordinator C can be steps shown in Figure 2 for training a
played by an authority such as a gov- Use Case 1: FedRiskCtrl risk control model for SME loans.
ernment department or replaced by The first use case is an FL application FedRiskCtrl is implemented with
a secure computing node.10 The com- in finance. It is an example of federated the FATE (Federated AI Technology
munication between the coordinator C risk control (FedRiskCtrl) for small and Enabler) platform.d With VFL, the
and the data owners A and B (a.k.a. par- micro enterprise (SME) loans deployed agency A and the bank B do not need
ties) may be encrypted (for example, by WeBank.c to expose their private data to each
using homomorphic encryption2,10) to other, and the model built with FL is
further defend against any informa- expected to perform as well as the
b For more use cases deployed by WeBank, see
tion leakage. Further, the coordinator https://bit.ly/37lgjlq
C may also be a logical entity and be d For more information on FATE, see https://bit.
c For more details on FedRiskCtrl, see https://
ly/37hBC7r
located in either A or B. An FL system bit.ly/3o8ECbY
architecture can also employ the peer-
to-peer model, without a coordinator, Figure 2. Examples of VFL Architecture.10
as illustrated in Figure 2b. The data
owners A and B communicate directly Federated model
without the help of a third party. While Collaborator
there are only two data owners in Fig- x C x
ure 2, an FL system may generally in- w w
Model A Model B u u
clude two or more data owners.7,10
Data
v Data
Taking the client-server model shown Encrypted model training
from A from B
in Figure 2a as an example, we summa- v
rize the encrypted and secure model Encrypted entity alignment u Sending public keys
training with VFL into the following four v Exchanging intermediate results
steps, after aligning the data samples
w Computing gradients and loss
between the two data owners.10
Corp. A Corp. B
˲ Step 1: C creates encryption key No data exchange x Updating models
pairs, and sends the public key to A
and B. (a)

˲ Step 2: A and B encrypt and ex-

Federated model
change intermediate computation re-
sults for gradient and loss calculations. w w
˲ Step 3: A and B compute encrypted
Model B x v v x
gradients and add an additional mask, Model A

respectively. B also computes the en- Encrypted model training

v
Data Data
crypted loss. A and B send encrypted from A u from B

results to C. Encrypted entity alignment u Sending public keys

˲ Step 4: C decrypts gradients and
v Exchanging intermediate results
loss and sends the corresponding re-
sults back to A and B. A and B unmask w Computing gradients and loss
Corp. A Corp. B
the gradients, and update their model No data exchange x Updating models
parameters accordingly.
Readers can find more more infor- (b)
mation about the FL model training

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 35

viewpoints

Outlook
FL can overcome the challenges of data
FL can overcome silos, small data, privacy issues, and
the challenges of lead us toward privacy-preserving AI. It
Distinguished data silos, small
will become the foundation of next-gen-
eration ML that caters to technological

Speakers data, privacy issues,

and lead us toward
and societal needs for responsible AI
development and applications.10

Program
While FL has great potential, it also
privacy-preserving AI. faces several practical challenges.5,10
The communication links between
the local data owners and the coordi-
A great speaker nator may be slow and unstable. There
may be a very large number of local
can make the model built with centralized dataset
. The model built
data owners (for example, mobile de-
vices) to manage. Data from different
difference between with FL performs significantly better
than the model built only with the bank
data owner in an FL system may fol-
low non-identical distributions, and
a good event and B’s data. different data owners may have un-
balanced numbers of data samples,
a WOW event! Use Case 2: FedVision which may result in a biased model
The second use case is an FL applica- or even failure of model training.5
tion in edge computing. It is an exam- Incentivizing mobile device owners
ple of federated computer vision (Fed- or organizations to participate in FL
Students and faculty Vision) for object detection deployed also needs further studies. Incentive
can take advantage of by WeBank.e mechanism design for FL should be
Due to privacy concerns and high done in such a way to make the federa-
ACM’s Distinguished cost of transmitting video data, it is dif- tion fair and sustainable.8
Speakers Program ficult to centrally collect surveillance
video data for model training in prac- References

to invite renowned tice. With FedVision, surveillance video

1. Al-Rubaie, M., and Chang, J.M. Privacy-preserving
machine learning: Threats and solutions. IEEE
data collected and stored in the edge Security and Privacy (Apr. 2019).
thought leaders in cloud of each surveillance company are
2. Bonawitz, K. et al. Practical secure aggregation for
privacy-preserving machine learning. In Proceedings
academia, industry no longer required to be uploaded to a of ACM SIGSAC CCS’17 (Nov. 2017).
3. DLA Piper. Data protection laws of the world: Full
central cloud for centralized model
and government to
handbook (Jan. 2020); https://bit.ly/354nDiC
training.10 In FedVision, an initial object 4. Dwork, C. Differential privacy: A survey of results. In
Proceedings of TAMC’08 (Apr. 2008).
deliver compelling and detection model is sent from the FL
server to each surveillance company
5. Kairouz, P. et al. Advances and open problems
in federated learning. (Dec. 2019); arXiv preprint

insightful talks on the (that is, to each edge cloud), which then
arXiv:1912.04977
6. Liu, Y., Chen, T., and Yang, Q. Secure federated transfer
learning. In Proceedings of IJCAI’19 (Aug. 2019).
most important topics uses the locally stored data to train the
object detection model. After a few local
7. McMahan, H.B., Moore, E., Ramage, D., and y Arcas,
B.A. Communication-efficient learning of deep

in computing and IT training epochs, the model parameters networks from decentralized data. In Proceedings of
AISTATS’17 (Apr. 2017).
from each surveillance company are en- 8. Richardson, A., Filos-Ratsikas, A., and Faltings, B.
today. ACM covers the crypted and sent to the FL server. The
Rewarding high-quality data via influence functions.
(Aug. 2019); arXiv preprint arXiv:1908.11598
cost of transportation local model parameters are aggregated 9. Yang, Q. et al. Federated machine learning: Concept
and applications. ACM Trans. Intell. Syst. Technol.
into a global federated model by the FL
for the speaker to
(TIST) (Feb. 2019).
server and sent back to each surveil- 10. Yang, Q. et al. Federated Learning. Morgan & Claypool,
Dec. 2019.
travel to your event. lance company. This process iterates
until the stopping criterion is met. Yong Cheng ([email protected]) is a Senior
The model training process in Fed- Researcher at WeBank, China.
Vision is very similar to the federated
speakers.acm.org
Yang Liu ([email protected]) is a Senior Researcher
averaging procedure for HFL model at WeBank, China.

training.2,7 The final global federated Tianjian Chen ([email protected]) is a Deputy

General Manager of AI at WeBank, China.
model will be distributed to the partici-
pating surveillance companies in the Qiang Yang ([email protected]) is Chief AI Officer at
WeBank, China, and a Chair Professor at The Hong Kong
federation, to be used for object detec- University of Science and Technology, China.
tion, such as fire detection.

e For more details on FedVision, see https://bit.

ly/3o6runW Copyright held by authors.

36 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 SHAPE THE FUTURE OF COMPUTING.
JOIN ACM TODAY. www.acm.org/join/CAPP

SELECT ONE MEMBERSHIP OPTION

ACM PROFESSIONAL MEMBERSHIP: ACM STUDENT MEMBERSHIP:
q Professional Membership: $99 USD q Student Membership: $19 USD
q Professional Membership plus q Student Membership plus ACM Digital Library: $42 USD
ACM Digital Library: $198 USD q Student Membership plus Print CACM Magazine: $42 USD
($99 dues + $99 DL) q Student Membership with ACM Digital Library plus
Print CACM Magazine: $62 USD

q Join ACM-W: ACM-W supports, celebrates, and advocates internationally for the full engagement of women
in computing. Membership in ACM-W is open to all ACM members and is free of charge.

PAYMENT INFORMATION

Name
Purposes of ACM
ACM is dedicated to:
Mailing Address 1) Advancing the art, science, engineering, and application
of information technology
2) Fostering the open interchange of information to serve
both professionals and the public
City/State/Province
3) Promoting the highest professional and ethics standards
ZIP/Postal Code/Country
By joining ACM, I agree to abide by ACM’s Code of Ethics
q Please do not release my postal address to third parties (www.acm.org/code-of-ethics) and ACM’s Policy Against
Harassment (www.acm.org/about-acm/policy-against-
harassment).
Email Address
I acknowledge ACM’s Policy Against Harassment and agree
q Yes, please send me ACM Announcements via email that behavior such as the following will constitute
q No, please do not send me ACM Announcements via email grounds for actions against me:

q AMEX q VISA/MasterCard q Check/money order • Abusive action directed at an individual, such as

threats, intimidation, or bullying
• Racism, homophobia, or other behavior that
Credit Card #
discriminates against a group or class of people
Exp. Date • Sexual harassment of any kind, such as unwelcome
sexual advances or words/actions of a sexual nature
Signature

BE CREATIVE. STAY CONNECTED. KEEP INVENTING.

ACM General Post Office 1-800-342-6626 (US & Canada) Fax: 212-944-1318
P.O. Box 30777 1-212-626-0500 (Global) [email protected]
New York, NY 10087-0777 Hours: 8:30AM - 4:30PM (US EST) acm.org/join/CAPP
practice
DOI:10.1145/ 3410625
devices, such as tapes and disks, a bit is

Article development led by
queue.acm.org
represented by the polarity of a certain
area of the magnetic film. In modern
DRAM (dynamic random-access mem-
Be kind and rewind. ory), a bit is often represented as two
levels of electrical charge stored in a
BY JESSIE FRAZELLE capacitor, a device that stores electrical
energy in an electric field. (In the early

The Life
1960s, the paper cards used to input
programs for IBM mainframes were
known as Hollerith cards, named after
their inventor Herman Hollerith from
the Tabulating Machines Company—
which through numerous mergers is
what is now known as IBM.)

of a Data
In June 1956, Werner Buchholz
coined the word byte to refer to a group
of bits used to encode a single character
of text. Let’s address character encod-
ing, starting with ASCII (American Stan-
dard Code for Information Interchange).
ASCII was based on the English alpha-

Byte
bet; therefore, every letter, digit, and
symbol (a-z, A-Z, 0–9, +, -, /, “, !, among
others) were represented as a seven-bit
integer between 32 and 127. This wasn’t
very friendly to other languages. To sup-
port other languages, Unicode extended
ASCII so that each character is repre-
sented as a code-point, or character; for
example, a lowercase j is U+006A, where
U stands for Unicode followed by a hexa-
decimal number.
UTF-8 is the standard for represent-
ing characters as eight bits, allowing
• more online data has been stored in a number
A B YTE O F every code-point from 0 to 127 to be
of different ways through the years as
A version of
this article
stored in a single byte. This is fine for
English characters, but other languag-
newer, better, and faster storage media
with embedded
informational es often have characters that are ex-
links is
are introduced. A byte is a unit of digital
available at pressed as two or more bytes. UTF-16
https://queue. is the standard for representing char-
information that most commonly refers to
acm.org/detail.
acters as 16 bits, and UTF-32 is the
cfm?id=3419941
eight bits. A bit is a unit of information that standard for 32 bits. In ASCII every
can be expressed as 0 or 1, representing a logical state. character is a byte, but in Unicode,
that’s often not true—a character can
Let’s take a brief walk down memory lane to learn be one, two, three, or more bytes.
about the origins of bits and bytes. Groups of characters might also be re-
ferred to as words, as in this linked Uni-
Going back in time to Babbage’s Analytical Engine,
IMAGE BY ALESA NKO RODRIGUEZ

vac ad calling out “1 kiloword or 12,000

you can see that a bit was stored as the position of a characters.” This article refers through-
mechanical gear or lever. In the case of paper cards, a out to different sized groupings of
bits—the number of bits in a byte vary-
bit was stored as the presence or absence of a hole in ing according to the design of the stor-
the card at a specific place. For magnetic storage age medium in the past.

38 COM MUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 39
practice

This article also travels in time have typically broken. A removable

through various storage media, diving plastic ring on the back of the tape reel
into how data has been stored through- provided write protection. About 1.1MB
out history. By no means does this in- could be stored on one reel of tape.
clude every single storage medium ever
manufactured, sold, or distributed. What was Think back to VHS tapes: What was
required before returning a movie to
This article is meant to be fun and in-
formative but not encyclopedic. It
required before Blockbuster? Rewinding the tape! The
same could be said for the tape used
wraps up with a look at the current and returning a movie for computers. Programs could not
future technologies for storage.
To get started, let’s assume we have a
to Blockbuster? hop around a tape, or randomly access
data—they had to read and write in se-
byte of data to be stored: the letter j, or as Rewinding the tape! quential order.
an encoded byte 6a, or in binary
01001010. As we travel through time, this
The same could be 1956
data byte will come into play in some of said for the tape The era of magnetic-disk storage began
the storage technologies covered here.
used for computers. in 1956 with IBM’s completion of the
305 RAMAC computer delivered to
1951 Programs could not Zellerbach Paper in San Francisco. This
The story begins in 1951 with the Unis-
ervo tape drive for the Univac 1 com- hop around a tape, computer was the first to use a moving-
head HDD (hard-disk drive). The
puter, the first tape drive made for a
commercial computer. The tape was
or randomly access RAMAC disk drive consisted of 50 mag-
netically coated 24-inch-diameter met-
three pounds of a thin strip (half-inch) data—they had to al platters capable of storing about five
of nickel-plated phosphor bronze,
called vicalloy, which was 1,200 feet
read and write in million characters of data, seven bits
per character, and spinning at 1,200
long. Our data byte could be stored at a sequential order. revolutions per minute. The storage ca-
rate of 7,200 characters per second on pacity was about 3.75MB.
tape moving at 100 inches per second. RAMAC allowed real-time random
At this point in history, you could mea- access memory to large amounts of
sure the speed of a storage algorithm data, unlike magnetic tape or punch
by the distance the tape traveled. cards. IBM advertised the RAMAC as be-
ing able to store the equivalent of 64,000
1952 punched cards. Previously, transactions
Let’s fast forward a year to May 21, 1952, were held until a group of data was ac-
when IBM announced its first magnetic cumulated and batch processed. The
tape unit, the IBM 726. Our data byte RAMAC introduced the concept of con-
could now be moved off Uniservo metal tinuously processing transactions as
tape onto IBM’s magnetic tape. This new they occurred so data could be retrieved
home would be super cozy for our very immediately when it was fresh. Our data
small data byte since the tape could store byte could now be accessed in the
up to two million digits. This magnetic RAMAC at 100,000 bits per second. Pri-
seven-track tape moved at 75 inches per or to this, with tapes, people had to write
second with a transfer rate of 12,500 dig- and read sequential data and could not
its or 7,500 characters—called copy randomly jump to various parts of the
groups at the time—per second. For ref- tape. Real-time random access of data
erence, this article has 35,123 characters. was truly revolutionary at this time.
Seven-track tapes had six tracks for
data and one to maintain parity by en- 1963
suring that the total number of 1-bits in DECtape was introduced in 1963. Its
the string was even or odd. Data was re- namesake was the Digital Equipment
corded at 100 bits per linear inch. This Corporation, known as DEC for short.
system used a vacuum channel method DECtape was inexpensive and reliable
of keeping a loop of tape circulating be- so it was used in many generations of
tween two points. This allowed the tape DEC computers. This ¾-inch tape was
drive to start and stop the tape in a split laminated and sandwiched between
second. This was done by placing long two layers of Mylar on a four-inch reel.
vacuum columns between the tape DECtape could be carried by hand,
reels and the read/write heads to ab- as opposed to its large weighty prede-
sorb sudden increases in tension in the cessors, making it great for personal
tape, without which the tape would computers. In contrast to seven-track

40 COM MUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 practice

tape, DECtape had six data tracks, two was released in 1977. Built in to the PET Packard was the first adopter of the
mark tracks, and two clock tracks. Data was a Commodore 1530 Datasette (a technology in 1982 with its HP-150.
was recorded at 350 bits per inch. Our portmanteau of data plus cassette). The This put the 3½-inch floppy disk on the
data byte, which is eight bits but could PET converted data into analog sound map and gave it wide distribution in
be expanded to 12, could be trans- signals that were then stored on cas- the industry. The disks were single sid-
ferred to DECtape at 8,325 12-bit words settes. This made for a cost-effective and ed with a formatted capacity of 161.2KB
per second with a tape speed of 93 +/-12 reliable storage solution, albeit very and an unformatted capacity of
inches per second. This is 8% more dig- slow. Our small data byte could be trans- 218.8KB. In 1982 the double-sided ver-
its per second than the Uniservo metal ferred at a rate of around 60–70 bytes per sion was made available, and the Mi-
tape from 1952. second. The cassettes could hold about crofloppy Industry Committee, a con-
100KB per 30-minute side, with two sortium of 23 media companies, based
1967 sides per tape. For example, you could fit a spec for a 3½-inch floppy on Sony’s
Four years later in 1967 a small team at about two (“rickroll” warning) 55KB im- original designs, cementing the format
IBM started working on the IBM flop- ages on one side of the cassette. The Da- into history. Our data byte could now
py-disk drive, code-named Minnow. tasette also appeared in the Commo- be stored on the early version of one of
At the time, the team was tasked with dore VIC-20 and Commodore 64. the most widely distributed storage
developing a reliable and inexpensive media: the 3½-inch floppy disk.
way to load microcode into IBM Sys- 1978
tem/370 mainframes. The project then Let’s jump ahead a year to 1978 when 1984
got reassigned and repurposed to load the LaserDisc was introduced as Disco- In 1984, the CD-ROM (compact disc
microcode into the controller for the vision by MCA and Philips. Jaws was the read-only memory), holding 550MB of
IBM 3330 Direct Access Storage Facili- first film sold on a LaserDisc in North prerecorded data, was announced by
ty, code-named Merlin. America. The audio and video quality on Sony and Philips. This format grew out
Our data byte could now be stored on a LaserDisc was far better than the com- of CD-DA (compact disc digital audio),
read-only eight-inch flexible Mylar disks petitors’, but too expensive for most developed by the two companies in
coated with magnetic material, known as consumers. As opposed to VHS tape, 1982. The CD-DA, which was used for
floppy disks. At the time of release, the re- which consumers could use to record distributing music, had a capacity of 74
sult of the project was named the IBM TV programs, the LaserDisc could not minutes. When Sony and Philips were
23FD Floppy Disk Drive System. The be written to. LaserDiscs used analog negotiating the standard for a CD-DA,
disks could hold 80KB of data. Unlike video with analog FM stereo sound and legend has it that one of the four people
hard drives, a user could easily transfer a PCM (pulse-code modulation) digital involved insisted it be able to hold all of
floppy in its protective jacket from one audio. The disks were 12 inches in di- Beethoven’s Ninth Symphony. The first
drive to another. Later, in 1973, IBM re- ameter and composed of two single-sid- product released on CD-ROM was Gro-
leased a read/write floppy-disk drive, ed aluminum disks layered in plastic. lier’s Electronic Encyclopedia, which
which then became an industry standard. The LaserDisc is remembered today as came out in 1985. The encyclopedia con-
being the foundation CDs that DVDs tained nine million words, occupying
1969 were built upon. only 12% of the available space, which
In 1969, the AGC (Apollo Guidance was 553 mebibytes. There would be
Computer) read-only rope memory 1979 more than enough room for the encyclo-
was launched into space aboard Apol- A year later in 1979 Alan Shugart and Fi- pedia and our data byte. Shortly thereaf-
lo 11, which carried American astro- nis Conner founded Seagate Technolo- ter in 1985, computer and electronics
nauts to the moon and back. This rope gy with the idea of scaling down a hard-
memory was made by hand and could disk drive to be the same size as a Rope memory.
hold 72KB of data. Manufacturing rope 5¼-inch floppy disk, which at the time
memory was laborious, slow, and re- was the standard. Their first product, in
quired skills analogous to textile work; 1980, was the Seagate ST506, the first
it could take months to weave a pro- HDD for microcomputers. The 5¼-inch
gram into the rope memory, illustrated disk held 5MB of data, which at the time
in the accompanying figure. But it was was five times more than the standard
FIGURE BY M A RK RICH A RDS/COM PUTER H ISTO RY M USEUM

the right tool for the job at the time to floppy disk. It was a rigid, metallic plat-
resist the harsh rigors of space. When a ter coated on both sides with a thin layer
wire went through one of the circular of magnetic material to store data. Our
cores, it represented a 1. Wires that data byte could be transferred at a speed
went around a core represented a 0. Our of 625KB/s onto the disk. That’s about
data byte would take a human a few one (second and final “rickroll” warn-
minutes to weave into the rope. ing) 625KB animated GIF per second.

1977 1981
The Commodore PET, the first (success- A couple of years later Sony introduced
ful) mass-market personal computer, the first 3½-inch floppy drive. Hewlett-

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 41

practice

companies worked together to create a While on the topic of flash memory, them compelling. Our data byte could
standard for the disks so any computer let’s look at the difference between be written onto a Zip disk at 1.4MB/s. At
would be able to access the information. NOR and NAND flash. Flash stores the time, a 1.44MB 3½-inch floppy
Also in 1984, Fujio Masuoka pub- information in memory cells made would write at about 16kB/s. In a Zip
lished his work on a new type of float- up of floating-gate transistors. The drive, heads are noncontact read/write
ing-gate memory, called flash memory, names of the technologies are tied and fly above the surface, which is simi-
which was capable of being erased and directly to the way the memory cells lar to a hard drive but unlike other flop-
reprogrammed multiple times. are organized. pies. Because of reliability problems
Let’s first review how floating-gate In NOR flash, individual memory and the affordability of CDs, Zip disks
memory works. It uses transistors, which cells are connected in parallel, allow- eventually became obsolete.
are electrical gates that can be switched ing the random access. This architec- Also in 1994, SanDisk introduced
on and off individually. Since each tran- ture enables the short read times re- CompactFlash, which was widely ad-
sistor can be in two distinct states (on or quired for the random access of opted into consumer devices such as
off), it can store two different numbers: 0 microprocessor instructions. NOR digital and video cameras. Like CD-
and 1. Floating gate refers to the second flash is ideal for lower-density applica- ROMs, CompactFlash speed is based
gate added to the middle transistor. This tions that are mostly read only. This is on x-ratings (8x, 20x, 133x, and so on).
second gate is insulated by a thin oxide why most CPUs typically load their The maximum transfer rate is calculated
layer. These transistors use a small volt- firmware from NOR flash. Masuoka based on the original audio CD trans-
age applied to the gate of the transistor to and colleagues presented the inven- fer rate of 150kB/s. This winds up look-
denote whether it is on or off, which in tion of NOR flash in 1984 and NAND ing like R = K × 150kB/s, where R is the
turn translates to a 0 or 1. flash in 1987. transfer rate and K is the speed rating.
With a floating gate, when a suitable In contrast, NAND flash designers For 133x CompactFlash, our data byte
voltage is applied across the oxide lay- gave up the ability for random access in would be written at 133 × 150kB/s or
er, the electrons tunnel through it and a tradeoff to gain a smaller memory around 19,950kB/s or 19.95MB/s. The
get stuck on the floating gate. There- cell size. This also has the benefits of a CompactFlash Association was found-
fore, even if the power is disconnected, smaller chip size and lower cost-per- ed in 1995 to create an industry stan-
the electrons remain present on the bit. NAND flash’s architecture consists dard for flash-based memory cards.
floating gate. When no electrons are on of an array of eight memory transistors
the floating gate, it represents a 1; and connected in a series. This leads to 1997
when electrons are trapped on the high-storage density, smaller memory- A few years later in 1997, the CD-RW
floating gate, it represents a 0. Revers- cell size, and faster write and erase (compact disc rewritable) was intro-
ing this process and applying a suitable since it can program blocks of data at a duced. This optical disc was used for
voltage across the oxide layer in the op- time. This comes at the cost of having data storage, as well as backing up and
posite direction causes the electrons to to overwrite data when it is not sequen- transferring files to various devices.
tunnel off the floating gate and restore tially written and data already exists in CD-RWs can be rewritten only about
the transistor to its original state. a block. 1,000 times, which at the time was not
Therefore, the cells are made program- a limiting factor since users rarely over-
mable and nonvolatile. Our data byte 1991 wrote data on one disk.
could be programmed into the transis- Let’s jump ahead to 1991 when a pro- CD-RWs are based on phase-change
tors as 01001010, with electrons totype SSD (solid-state disk) module technology. During a phase change of a
trapped in the floating gates to repre- was made for evaluation by IBM from given medium, certain properties of the
sent the zeros. SanDisk, at the time known as Sun- medium change. In the case of CD-
Masuoka’s design was a bit more af- Disk. This design combined a flash RWs, phase shifts in a special com-
fordable but less flexible than EEPROM storage array and nonvolatile memory pound, composed of silver, tellurium,
(electrically erasable programmable chips with an intelligent controller to and indium, cause reflecting lands and
read-only memory), since it required detect and correct defective cells automat- non-reflecting bumps, each representing
multiple groups of cells to be erased to- ically. The disk was 20MB in a 2½-inch a 0 or 1. When the compound is in a
gether, but this also accounted for its form factor and sold for approximately crystalline state, it is translucent, which
speed. At the time, Masuoka was work- $1,000. IBM wound up using it in the indicates a 1. When the compound is
ing for Toshiba but quit the company ThinkPad pen computer. melted into an amorphous state, it be-
shortly after to become a professor at comes opaque and nonreflective, which
Tohoku University. He was displeased 1994 indicates a 0. We could write our data
with Toshiba for not rewarding him for In 1994, Iomega released the Zip disk, a byte 01001010 as non-reflecting bumps
his work and sued the company, de- 100MB cartridge in a 3½-inch form fac- and reflecting lands this way.
manding compensation for his work. tor, a bit thicker than a standard 3½- CD-RWs eventually lost much of
The case settled in 2006 with a one- inch disk. Later versions of the Zip disk their market share to DVDs.
time payment of ¥87m, equivalent to could store up to 2GB. These disks had
$758,000. This still seems low, given the convenience of being as small as a 1999
how impactful flash memory has been floppy disk but with the ability to hold a In 1999, IBM introduced the smallest
on the industry. larger amount of data, which made hard drives in the world at the time: the

42 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 practice

IBM microdrive in 170MB and 340MB PMR can deliver more than three times
capacities. These were small hard the storage density of LMR. The key dif-
disks, one inch in size, designed to fit ference between the two is that the
into CompactFlash Type II slots. The grain structure and the magnetic ori-
intent was to create a device to be used
like CompactFlash but with more stor- Storage class entation of the stored data of PMR me-
dia is columnar instead of longitudi-
age capacity. These were soon replaced
by USB flash drives, however, and larg-
memory nal. PMR has better thermal stability

is persistent
and improved SNR (signal-to-noise ra-
er CompactFlash cards once they be- tio) as a result of better grain separa-
came available. Like other hard drives,
microdrives were mechanical and con-
but goes further tion and uniformity. It also benefits
from better writability because of
tained small, spinning disk platters. by also providing stronger head fields and better mag-

2000
performance better netic alignment of the media. Like
LMR, PMR’s fundamental limitations
USB flash drives were introduced in than or comparable are based on the thermal stability of
2000. These consisted of flash memory
encased in a small form factor with a to primary memory, magnetically written bits of data and
the need to have sufficient SNR to read
USB interface. Depending on the ver- as well as byte back written information.
sion of the USB interface used, the
speed varies: USB 1.1 is limited to addressability. 2007
1.5Mbps, whereas USB 2.0 can handle Hitachi Global Storage Technologies
35Mbps, and USB 3.0 can handle announced the first 1TB HDD in 2007.
625Mbps. The first USB 3.1 type C The Hitachi Deskstar 7K1000 used five
drives were announced in March 2015 3.5- inch 200GB platters and rotated at
and have read/write speeds of 530Mbps. 7,200 RPM. This is in stark contrast to
Unlike floppy and optical disks, USB de- the world’s first HDD, the IBM RAMAC
vices are harder to scratch but still de- 350, which had a storage capacity of ap-
liver the same use cases of data storage proximately 3.75MB. How far we have
and transferring and backing up files. come in 51 years! But wait, there’s more.
Because of this, drives for floppy and
optical disks have since faded in popu- 2009
larity, replaced by USB ports. In 2009, technical work was beginning
on NVMe (nonvolatile memory ex-
2005 press). NVM is a type of memory that
HDD manufacturers started shipping has persistence, in contrast to volatile
products using PMR (perpendicular memory, which needs constant power
magnetic recording) in 2005. Interest- to retain data. NVMe filled a need for a
ingly, this happened at the same time scalable host controller interface for
that Apple announced the iPod Nano, PCIe (peripheral component intercon-
which used flash as opposed to the nect express)-based SSDs. More than
one-inch hard drives in the iPod Mini, 90 companies were part of the working
causing a bit of an industry hoohaw. group that developed the design. This
A typical hard drive contains one or was all based on prior work to define
more rigid disks coated with a magneti- the NVMHCIS (nonvolatile memory
cally sensitive film consisting of tiny host controller interface specification).
magnetic grains. Data is recorded when Opening up a modern server would
a magnetic write-head flies just above likely result in finding some NVMe
the spinning disk, much like a record drives. The best NVMe drives today can
player and a record, except a needle is do about a 3,500MB/s read and
in physical contact with the record. As 3,300MB/s write. For the data byte we
the platters spin, the air in contact with started with, the character j, that is ex-
them creates a slight breeze. Just as air tremely fast compared with a couple of
on an airplane wing generates lift, the minutes to handweave rope memory
air generates lift on the head’s airfoil. for the Apollo Guidance Computer.
The write-head rapidly flips the magne-
tization of one magnetic region of Today and the Future
grains so that its magnetic pole points Now that we have traveled through time
up or down to denote a 1 or a 0. a bit, let’s take a look at the state of the
The predecessor to PMR was LMR art for SCM (storage class memory). Like
(longitudinal magnetic recording). NVM, SCM is persistent but goes further

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 43

practice

by also providing performance better on differences in magnetic directions.

than or comparable to primary memory, One magnetic layer, called the reference
as well as byte addressability. SCM aims layer, has a fixed magnetic direction,
to address some of the problems faced while the other magnetic layer, called
by caches today such as the low density
of SRAM (static random access memo- Shingled magnetic the free layer, has a magnetic direc-
tion that is controlled by passing cur-
ry). DRAM provides better density, but recording results rent. For a 1, the magnetization direc-
tion of the two layers are aligned. For
in a much more
this comes at a cost of slower access
times. DRAM also suffers from requir- a 0, the two layers have opposing
ing constant power to refresh memory.
Let’s break this down a bit. Power is
complex writing magnetic directions.
Resistive random access memory. A
required since the electric charge on process, since ReRAM cell consists of two metal elec-
the capacitors leaks off little by little;
this means that without intervention,
writing to one trodes separated by a metal-oxide layer.
This is similar to Masuoka’s original
the data on the chip would soon be track winds up flash-memory design, where electrons
lost. To prevent this leakage, DRAM re-
quires an external memory-refresh cir- overwriting an would tunnel through the oxide layer
and get stuck in the floating gate or vice
cuit that periodically rewrites the data adjacent track. versa. With ReRAM, however, the state
in the capacitors, restoring them to of the cell is determined by the concen-
their original charge. tration of oxygen vacancy in the metal-
To solve the problems with density oxide layer.
and power leakage, a few SCM technol- SCM downsides and upsides. While
ogies are in development: PCM (phase- these SCM technologies are promising,
change memory), STT-RAM (spin-trans- they still have downsides. PCM and STT-
fer torque random access memory), and RAM have high write latencies. PCM’s
ReRAM (resistive random access mem- latencies are 10 times that of DRAM,
ory). One nice aspect of all these tech- while STT-RAM has 10 times the laten-
nologies is their ability to function as cies of SRAM. PCM and ReRAM have a
MLCs (multilevel cells). This means limit on write endurance before a hard
they can store more than one bit of in- error occurs, meaning a memory ele-
formation, compared with SLCs (single- ment gets stuck at a particular value.
level cells), which can store only one bit In August 2015, Intel announced
per memory cell, or element. Typically, a Optane, a product built on 3DXPoint,
memory cell consists of one MOSFET pronounced 3D cross-point. Optane
(metal-oxide-semiconductor field-effect claims performance 1,000 times faster
transistor). MLCs reduce the number of than NAND SSDs with 1,000 times the
MOSFETs required to store the same performance, while being four to five
amount of data as SLCs, making them times the price of flash memory.
denser or smaller to deliver the same Optane is proof that SCM is not just ex-
amount of storage as technologies using perimental. It will be interesting to
SLCs. Let’s go over how each of these watch how these technologies evolve.
SCM technologies work. Helium hard-disk drive. An HHDD
Phase-change memory. PCM is is a high-capacity HDD filled with he-
similar to phase change for CD-RWs, lium and hermetically sealed during
described earlier. Its phase-change manufacturing. Like other hard disks,
material is typically GST, or GeSbTe (ger- covered earlier, it is much like a re-
manium-antimony-tellurium), which cord player with a rotating magnetic-
can exist in two different states: amor- coated platter. Typical HDDs would
phous and crystalline. The amorphous just have air inside the cavity, but that
state has a higher resistance, denoting a air causes an amount of drag on the
0, than the crystalline state, denoting a spin of the platters.
1. By assigning data values to intermedi- Helium balloons float, so we know
ate resistances, PCM can be used to helium is lighter than air. Helium is, in
store multiple states as an MLC. fact, one-seventh the density of air,
Spin-transfer torque random access therefore reducing the amount of drag
memory. STT-RAM consists of two ferro- on the spin of the platters, causing a re-
magnetic, permanent magnetic layers duction in the amount of energy re-
separated by a dielectric, an insulator quired for the disks to spin. This is actu-
that can transmit electric force without ally a secondary feature; the primary
conduction. It stores bits of data based feature of helium is to allow for packing

44 COMM UNICATIO NS O F THE AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 practice

seven platters in the same form factor operating system to know how to han- 2013. It started shipping • more online
that would typically hold only five. At- dle the complexity of the drive. the first drives in 2018. A version of
this article
tempting this with air-filled drives Seagate started shipping SMR drives with embedded
would cause turbulence. If you recall in 2013, claiming a 25% greater density End Of Tape, Rewind informational
links is
the airplane-wing analogy from earlier, than PMR. This article started off available at
this ties in perfectly. Helium reduces Microwave-assisted magnetic re- with the state of the art in https://queue.
drag, thus eliminating turbulence. storage media in 1951 acm.org/detail.
cording. MAMR is an energy-assisted cfm?id=3419941
As everyone knows, however, helium- magnetic storage technology—like and concludes after look-
filled balloons start to sink after a few HAMR, which is covered next—that ing at the future of storage technology.
days because the gas is escaping the bal- uses 20- to 40-GHz frequencies to bom- Storage has changed a lot over time—
loons. The same could be said for these bard the disk platter with a circular mi- from paper tape to metal tape, magnetic
drives. It took years before manufactur- crowave field. This lowers its coercivity, tape, rope memory, spinning disks, op-
ers had created a container that pre- meaning that the magnetic material of tical disks, flash, and others. Progress
vented the helium from escaping the the platter has a lower resistance to has led to faster, smaller, and more per-
form factor for the life of the drive. Back- changes in magnetization. As already formant devices for storing data.
blaze found that HHDDs had a lower an- discussed, changes in magnetization Comparing NVMe to the 1951 Unis-
nualized error rate of 1.03%, while stan- of a region of the platter are used to de- ervo metal tape shows that NVMe can
dard hard drives resulted in 1.06%. Of note a 0 or a 1, so since the platter has a read 486,111% more digits per second.
course, that is so small a difference, it is lower resistance to changes in magne- Comparing NVMe to the Zip disks of
hard to conclude much from it. tization, the data can be written more 1994, you see that NVMe can read
A helium-filled form factor can have densely. The core of this new technolo- 213,623% more digits per second.
an HDD encapsulated that uses PMR, or gy is the spin torque oscillator used to One thing that remains true is the
it could contain an MAMR (microwave- generate the microwave field without storing of 0s and 1s. The means by
assisted magnetic recording) or HAMR sacrificing reliability. which that is done vary greatly. I hope
(heat-assisted magnetic recording) Western Digital, also known as WD, the next time you burn a CD-RW with a
drive. Any magnetic storage technology unveiled this technology in 2017. Toshi- mix of songs for a friend, or store home
can be paired with helium instead of air. ba followed shortly after in 2018. While videos in an optical disc archive (yup,
In 2014, HGST (Hitachi Global Storage WD and Toshiba are busy pursuing you heard that right), you think about
Technologies) combined two cutting- MAMR, Seagate is betting on HAMR. how the nonreflective bumps translate
edge technologies into its 10TB HHDD Heat-assisted magnetic recording. to a 0 and the reflective lands of the disk
that used host-managed SMR (shingled HAMR is an energy-assisted magnetic translate to a 1. If you are creating a mix-
magnetic recording). storage technology for greatly increas- tape on a cassette, remember that those
Shingled magnetic recording. As ing the amount of data that can be are closely related to the Datasette used
noted earlier, PMR was SMR’s prede- stored on a magnetic device, such as in the Commodore PET. Lastly, remem-
cessor. In contrast to PMR, SMR writes an HDD, by using heat delivered by a ber to be kind and rewind (this is a trib-
new tracks that overlap part of the pre- laser to help write data onto the sur- ute to Blockbuster, but there are still
viously written magnetic track, which face of a platter. The heat causes the open formats for using tape today).
in turn makes the previous track nar- data bits to be much closer together on Acknowledgments. Thank you to
rower, allowing for higher track densi- the platter, which allows greater data Robert Mustacchi, Trammel Hudson,
ty. The technology’s name stems from density and capacity. and Rick Altherr for tidbits (I can’t help
the fact that the overlapping tracks are This technology is quite difficult to myself) throughout this article.
much like that of roof shingles. achieve. A 200-milliwatt laser heats a
SMR results in a much more com- teensy area of the region to 750 degrees F
Related articles
plex writing process, since writing to (400 degrees C) quickly before writing on queue.acm.org
one track winds up overwriting an adja- the data, while not interfering with or
The Most Expensive One-byte Mistake
cent track. This doesn’t come into play corrupting the rest of the data on the
Poul-Henning Kamp
when a disk platter is empty and data is disk. The process of heating, writing the https://queue.acm.org/detail.cfm?id=2010365
sequential. Once you are writing to a data, and cooling must be completed in
Should You Upload or
series of tracks that already contain less than a nanosecond. These challeng- Ship Big Data to the Cloud?
data, however, this process is destruc- es required the development of nano- Sachin Date
tive to existing adjacent data. If an ad- scale surface plasmons, also known as a https://queue.acm.org/detail.cfm?id=2933408
jacent track contains valid data, then it surface-guided laser, instead of direct Injecting Errors for Fun and Profit
must be rewritten. This is quite similar laser-based heating, as well as new types Steve Chessin
to NAND flash, as covered earlier. of glass platters and heat-control coat- https://queue.acm.org/detail.cfm?id=1839574
Device-managed SMR devices hide ings to tolerate rapid spot-heating with-
this complexity by having the device Jessie Frazelle is the cofounder and chief product officer
out damaging the recording head or any of the Oxide Computer Company. Before that, she worked
firmware manage it, resulting in an in- nearby data, and various other technical on various parts of Linux, including containers as well as
the Go programming language.
terface like any other hard disk you challenges that needed to be overcome.
might encounter. Host-managed SMR Seagate first demonstrated this Copyright held by author/owner.
devices, on the other hand, rely on the technology, despite many skeptics, in Publication rights licensed to ACM.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 45

practice
DOI:10.1145/ 3424260
loose. The cybersecurity landscape was

Article development led by
queue.acm.org
nowhere near as challenging as it is
now, particularly when it comes to the
security of consumer accounts. There
The challenges of multifactor authentication were exceptions for certain industries,
based on SMS, including cellular security such as banking, where password re-
quirements were slightly more strin-
deficiencies, SS7 exploits, and SIM swapping. gent and a hidden form of two-factor
authentication, based mainly on IP
BY ROGER PIQUERAS JOVER geolocation, was used before it became
an option for other sites. Neverthe-

Security Analysis
less, the average consumer generally
needed just a simple password to ac-
cess even highly critical repositories of

of SMS as
data, and the same password was often
reused for multiple accounts.
Today Internet security requires

a Second Factor
much more attention. A good example
of what can go wrong is the hacking or-
deal detailed by a technology reporter
for Wired who was not using two-factor

of Authentication authentication for his email account.8

Email accounts have become, over the
years, not only large repositories of
highly sensitive and private data, but
also single points of failure for digital
footprints on the Internet. For exam-
ple, the majority of online services al-
low for the resetting of a password by
T H E INTER NET START E Dbecoming the main reason sending an email to the user’s main
email account. As a result, if an email
that household PSTN (public switched telephone account gets compromised, many oth-
network) lines stayed busy sometime in the mid to er accounts can also be compromised
in short order.
late 1990s. Over the next decade, a number of online As the security-threat landscape has
services completely changed the interaction of changed, so too have the way pass-
society with technology. From email to the dawn of words are used and their complexity
requirements. Although many online
e-commerce, these services increasingly tied people to services did not really adhere to best
technology and the Internet. practices, it became widely acknowl-
edged that passwords should be highly
Although the concept of a password was prevalent in complex in order to maximize their en-
many technology disciplines and domains, the general tropy and, thus, substantially increase
public had little knowledge of it, with the exception the amount of time it would take to
crack them.
of PINs for their debit cards. It was the Internet that Eventually, though, some scientific
introduced the concept of password security to them. studies12,29 and a viral online cartoon30
argued that increasing password com-
From that point on, people realized they had to plexity was not the right solution. Pass-
remember passwords to access their email accounts, phrases are proven to have much higher
favorite e-commerce sites, and so on. entropy and are much easier to remem-
ber. On the other hand, forcing pass-
At that time, a password was all it took to unlock an word rotation, in combination with a
account, and password requirements were very strict password complexity policy, has

46 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 been shown to result in much weaker mandate, the use of two-factor authen- thentication. For example, while SMS
passwords. Moreover, as a rule of tication. The second factor needed to (short message service) as a second
thumb, it is now acknowledged that a be something the user has—the obvi- form of authentication is a good idea
password might not necessarily have to ous and simple choice was clear from for certain types of online accounts, it
be rotated if it is not present in any of the beginning: the user’s smartphone. is not the best option for those who
the public repositories of leaked cre- Enabling two-factor authentication own a large amount of cryptocurrency
dentials in the wild.10 for online accounts is critical to their in an online exchange.3
Along with the ongoing challenge of security. Everyone should enable this SMS-based authentication tokens
making passwords secure but still us- feature in (at the very least) their email are popular options for securing online
able and easy to remember, the secu- accounts, as well as other accounts accounts, and they are certainly more
rity industry recognized the security that store critical and sensitive data secure than using a password alone.
of an online account should not be such as credit card numbers. Crypto- The history of cellular network security,
protected only by something you currency exchange accounts, which are however, indicates that SMS is not a se-
IMAGERY F ROM SHUT TERSTOCK .COM

know (your password). Somewhat commonly the target of cybercrimi- cure method of communication. From
similar to the banking approach, nals, should also be secured by multi- rogue base stations and stingrays to
which requires something the user ple forms of authentication. The po- more sophisticated attacks, there are a
has (for example, a debit card) and tentially high monetary value of what number of known methods to eaves-
something the user knows (for exam- these accounts protect makes them an drop on and brute-force text messages,
ple, the card’s PIN), online accounts interesting case study of what might be both locally and remotely. As such, this
started to support, and in some cases the best choice for a second form of au- method is not the most reliable for ac-

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 47

practice

Figure 1. Pros and cons of different types of token. and disadvantages in the context of
convenience and usability—important
factors to consider when designing a
Authentication
Type Advantages Disadvantages secure system. The pros and cons of
Application • Convenient. • Critical to generate and keep backup both types of authentication are sum-
• No network connectivity required. codes. marized in Figure 1.
• The same application can be used to • Phone loss or theft. App-generated token. As noted
generate tokens for multiple accounts. • Cryptographic keys not always
included in the backup of the phone.
throughout this article, a one-time to-
SMS • Not tied to cryptographic keys • Requires network connectivity.
ken generated by an application on the
on the device. • Generally insecure. user’s device is the most secure way of
• Easier to recover from device loss implementing two-factor authentica-
or theft. tion for online accounts without requir-
ing nonstandard hardware to be used
by the consumer (for example, an RSA
Figure 2. Three methods of SMS interpretation. token, a YubiKey, and so on, which are
more common in an enterprise con-
text). Aside from that, there are a num-
Method of SMS
Interception Advantages Disadvantages Cost
ber of advantages and disadvantages.
One of the main considerations for
Over the air • Fast. • Attacker must be in the • Low
• Does not require any vicinity of the victim. either option is network connectivity.
time-consuming prepa- • Technical knowledge The convenience of an app-generated
ration steps. of GSM, SW-radios and token, which requires no network con-
SW-based network
stack.
nectivity, contrasts with the strict re-
SS7 • Can be done remotely. • Requires access to SS7 • High.
quirement of connectivity to receive a
• No need to interact with nodes and knowledge of • For-sale access to SS7 token via SMS. Although network con-
the operator or leave SS7 protocols. nodes on the dark web. nectivity is considered a ubiquitous
any trail.
commodity, there are a number of sce-
• Fast.
• Does not require any narios in which a user could require ac-
time-consuming prepa- cess to an account while out of range.
ration steps Another advantage of app-based to-
SIM swap • Can be done remotely. • Requires attacker to • Low ken delivery is that these apps can gen-
• Low cost. interact with network
• Low-hanging fruit. operator’s customer
erally be registered and used with mul-
support, which can be a tiple online accounts. The main
time-consuming prepa- usability challenge with generating to-
ration stage.
kens with a smartphone app, however,
• Leaves a trace.
• Some operators in is that administering such an app—
African nations have and the cryptographic material it lever-
mitigated this method. ages—requires some extra effort. In
general, backing up a smartphone to
the cloud, the most common method,
counts that store assets with a high fi- (Full disclosure: the author uses does not save such cryptographic mate-
nancial value, such as cryptocurrencies. SMS to secure some rather vanilla on- rial as part of the backup data. Nor
This article provides some insight line accounts, mainly those that do not does this material get saved on an un-
into the security challenges of SMS- require storing a credit card number or encrypted local backup on a computer.
based multifactor authentication: main- other sensitive financial information.) Even when locking a local backup with
ly cellular security deficiencies, exploits a password, not all seeds are stored
in the SS7 (Signaling System No. 7) pro- SMS vs. One-Time Token App with it. This can lead to users getting
tocol, and the dangerously simple yet For standard consumer online ac- locked out of their accounts if their
highly efficient fraud method known as counts, the two main options for pro- smartphones are lost or stolen, for ex-
SIM (subscriber identity module) swap- viding a second factor of authentica- ample, or even if they get a new phone.
ping. Based on these insights, readers tion are generally via SMS or leveraging In these scenarios, the so-called “back-
can gauge whether SMS tokens should a one-time token generated by an app up codes” are important. As a rule of
be used for their online accounts. This on the user’s smartphone. The latter thumb, users should never wipe their
article is not an actual analysis of multi- is more secure and should be used for old smartphones until the new ones
factor authentication methods and what highly secure and sensitive accounts, have been fully set up and two-factor
can be considered a second (or third, but the former is the most widely used authentication apps have been reset.
fourth, and so on) factor of authentica- option and could be a valid choice in SMS token. Two-factor authentica-
tion; for such a discussion, the author certain circumstances. Aside from tion tokens received via text message
recommends reading security expert their security, however, these two op- tend to work well for standard consum-
Troy Hunt’s report on the topic.9 tions have very different advantages er use because they’re easy for the user.

48 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 practice

There is no requirement to install an The main advantages and challenges intended for another user.6 In order to
application on the user’s device, and it of three different methods of SMS inter- geolocate a given target victim to inter-
doesn’t require any management of ception are summarized in Figure 2. cept a token over SMS, privacy and loca-
backup codes or a backup plan to deal Cellular network security. The first tion leaks have also been investigated in
with a lost or stolen device. When a generation of mobile networks (1G) the context of legacy GSM networks.17
user gets a new device, there is no need lacked support for encryption. Legacy Specific efforts were made to en-
to reset the two-factor authentication 2G GSM (Global System for Mobile Com- hance confidentiality and authentica-
system, as text messaging is tied to the munications) networks lack mutual au- tion in 3G and LTE (Long-term Evolu-
phone number, which generally re- thentication and implement an outdat- tion) mobile networks, with stronger
mains unchanged on a new device. ed encryption algorithm. Combined cryptographic algorithms and mutual
On the downside, SMS-based au- with the wide availability of open source authentication implemented in both
thentication requires an active connec- implementations of the GSM protocol standards. Because of this, LTE has
tion to the cellular network. Even stack, this resulted in the discovery of generally been considered secure, giv-
though the majority of text message- many possible exploits on the GSM ra- en its mutual authentication and strong
based communication occurs over IP dio link over the past decade23 (illustrat- encryption scheme. As such, confiden-
(for example, iMessage and WhatsApp), ed in Figure 3). Specifically, both the tiality and authentication were wrongly
SMS second-factor authentication to- techniques and the tools necessary to assumed to be sufficiently guaranteed.
kens are generally delivered over cellu- deploy a malicious GSM base station Researchers demonstrated a few years
lar networks’ standard SMS. Therefore, and implement a full MITM (man-in- ago, however, that LTE mobile net-
Wi-Fi connectivity alone is not suffi- the-middle) attack against a GSM con- works are still vulnerable to protocol
cient; an active cellular connection is nection are commodities, although they exploits, location leaks, and rogue
necessary. This can be challenging in require the adversary to be in physical base stations.14,27
certain situations where cell service is proximity to a given target. Low-cost Despite the strong cryptographic
spotty or nonexistent or connectivity is software radios and open source imple- protection of user traffic and mutual
constrained to 802.11 networks. mentations of the GSM protocol stack authentication, a large number of con-
Despite their security challenges, can be used to intercept mobile traffic, trol-plane (signaling) messages are
SMS-based authentication tokens are a including SMS messages.22,24 regularly exchanged over an LTE radio
widely utilized option, which is cur- Over the past few years, researchers link in the clear. Before the authentica-
rently receiving active support from de- have also demonstrated how to inter- tion and encryption steps of a connec-
vice manufacturers. As an example, cept SMS traffic with less strict proximi- tion are executed, a mobile device en-
Apple recently announced a new fea- ty constraints by triggering a race condi- gages in a substantial conversation
ture in iOS 14 to harden SMS codes tion when replying to paging messages with any LTE base station (real or rogue)
against applications attempting to
trick the user into inputting the code in Figure 3. GSM traffic eavesdropping and interception.
a malicious app (https://developer.ap-
Trivial to geo-locate users
ple.com/news/?id=z0i801mg). by intercepting unencrypted
paging messsages.
Security Challenges cell
of SMS-Based Authentication phone
Despite its convenience and use by a
large number of online services, two-
factor authentication via text message legitimate GSM
base station
has significant security challenges.
This section presents an overview of
the main security challenges of using
SMS for two-factor authentication to-
ken delivery. These range from some- Brute force A5 and decode/decrypt
phone calls, messages (SMS) and
what sophisticated threats to cellular IP traffic.
network protocols, which require an
adversary to be in the vicinity of the eavesdropper
target victim, to low-hanging fruit tech-
niques that, despite being much less
technically complex, have no range
constraints and can be implemented
cell Fully MitM traffic, connection
at near-zero cost. For example, one of phone hijack, DNS hijack, etc.
the biggest security threats in mobile
communication systems is SIM swap-
ping, a systemic problem related to rogue GSM
how mobile operators authenticate us- base station
ers in their customer care platforms.18

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 49

practice

that advertises itself with the correct signaling support for a number of func-
broadcast information. This results in tions in the PSTN, namely call establish-
a critical threat caused by the implicit ment, billing, routing, and information
trust placed—from the mobile de- exchange.25 From its inception in 1988,
vice’s point of view—in the messages
coming from the base station. Many In the age of large- when mobile operators started leverag-
ing it for out-of-band signaling, this pro-
operations with critical security impli-
cations are executed when triggered by
scale cyberattacks, tocol’s security mostly relied on the im-
plicit trust among operators. It was
some of these implicitly trusted mes- one of the regarded as a closed trusted network
sages, which are neither authenticated
nor validated.
largest civilian and had limited to no authentication
built in. As a result, the security features
In the age of large-scale cyberattacks, communication of this network and protocol were mini-
one of the largest civilian communica-
tion systems must rely on privacy proto-
systems must mal and depended on a small number of
operators globally that were either state-
cols far more sophisticated than just rely on privacy controlled or large corporations. This is
basic implicit trust anchored in the
base station looking like a legitimate protocols far more not the case anymore, as the number of
operators is much larger as a result of
station. Note that the same applies in sophisticated than the steep increase in mobile usage, as
reverse, with the base station implicitly
trusting all preauthentication messages just basic implicit well as the growth in the number of MV-
NOs (mobile virtual network operators)
coming from mobile devices.
Although a malicious LTE base sta-
trust anchored in around the globe over the past decade.
3GPP (3rd Generation Partnership
tion is incapable of launching a full the base station Project) added two new protocols to SS7
MITM attack, several studies have dem-
onstrated and prototyped techniques to
looking like a in the 1990s and 2000s: MAP (Mobile
Application Part) and CAMEL (Custom-
silently downgrade a modern smart- legitimate station. ized Applications for Mobile Networks
phone to a vulnerable GSM connec- Enhanced Logic). These were aimed at
tion.13,26,27 What all these techniques supporting some of the new services
have in common is they leverage and that mobile networks provide, as well
abuse such preauthentication messages. as new features for mobile operators.19
As the industry prepares to embrace Among other functionality, CAMEL al-
the advent of 5G, the security architec- lows the implementation of carrier-
ture for such next-generation mobile grade value-added services such as
networks is being put under scrutiny. fraud control and prepaid services. In
Several studies over the past year have parallel, MAP provides services to geo-
highlighted the fact that most preau- locate devices globally, such as the any-
thentication message-based protocol TimeInterrogation service and LCS (Lo-
exploits in LTE still apply to 5G net- cation Service). Discouragingly, none of
works.11,15,16,26 As a result, silently down- these new SS7 subprotocols added au-
grading the connection of a smartphone thentication or security features.
to a GSM link is still possible, given the Researchers have identified a num-
current specifications for such mobile ber of critical security vulnerabilities in
communication systems.28 By abusing SS7 that could be exploited to geolocate
these vulnerabilities, an adversary could users and intercept their traffic from
successfully intercept a two-factor au- nearly anywhere.5 In some cases, the
thentication token delivered over SMS. only requirement is to have access to the
It is important to note, however, that SS7 network, which, despite being more
intercepting tokens from SMS messages restricted now than in the past, can still
by intercepting GSM traffic is the most easily be purchased on the dark web. Re-
technologically complex option. Even searchers have also gained access to the
though such attacks can be carried out SS7 network via hacked femtocells and,
with low-cost software radios and mi- in some scarce cases, actually purchas-
nor modifications to open source tools, ing access from mobile operators.
the vast majority of fraud conducted by To make matters worse, researchers
intercepting authentication tokens de- were also able to demonstrate tech-
livered via SMS leverages vulnerabili- niques to intercept phone calls and
ties in either SS7 or SIM swapping. text messages remotely by means of ex-
SS7 security. SS7 is a legacy architec- ploiting flaws in the CAMEL protocol.
ture and protocol developed more than Figure 4 illustrates this process. Such
30 years ago. It performs out-of-band security threats in mobile communica-

50 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 practice

tion networks came to wide public at- SIM, the rest of the attack is fairly sim- cost and low effort a SIM swap attack
tention when a German researcher ple. From that moment until the victim requires, fraud and scamming rings
demonstrated these attacks in a news notices the loss of coverage and calls are devising more sophisticated meth-
report on primetime TV.1 customer service, the attacker will be the ods to scale up the number of accounts
Once the attacker has access to an destination of any call and text message they can take over. For example, it was
entry point to the SS7 network, all it routed to the victim’s MSISDN (mobile recently discovered that SIM swappers
takes is one message to modify the reg- station international subscriber direc- were bribing customer service employ-
istration for a given target in the MSC tory number)—that is, the victim’s ees to perform the swaps for them and
(mobile switching center), in the case of 10-digit phone number. Therefore, any even leveraging malware that targets
GSM. From that moment on, the MSC two-factor authentication token re- the remote desktop technology used in
will reach out to the attacker instead. quested will be received by the attacker. call centers.4
Modern LTE networks largely mi- This type of attack is simple to im- Interestingly, fraudulent online ac-
grated most SS7-based services over to plement and accounts for a majority of count takeovers based on a SIM swap at-
the Diameter protocol. This new pro- breaches that require intercepting au- tack are not too complex to mitigate. De-
tocol, despite providing some improve- thentication tokens. Given the low spite being a widely acknowledged
ments, still suffers from a number of
vulnerabilities, most of which are flaws Figure 4. Exploiting flaws in the Camel SS7 protocol.
inherited from SS7.2 As a result, similar
remote interception of calls and text
messages could be possible in LTE. Re-
gardless, as discussed earlier, silently
downgrading a smartphone to a much
SMS base
less secure GSM connection is simple, gateway station
bank MSC/VLR fraudster user
given enough proximity to the target.
Exploiting security flaws in SS7 net-
works and their protocols is a fairly ef- SMS message flow
ficient way to intercept two-factor au-
thentication tokens delivered over
SMS. In general, this is an attack vec- insertSubscriberData Request
(address of fraudster as gsmSCF)
tor that is known for being used by
hacker groups and has become so rel-
connection setup
evant that it is even considered within
the MITRE ATT&CK framework, which
initialDP
has been widely adopted by many
technology companies as part of their
security postures.20 hijacked SMS message flow
SIM swapping. Despite the effective-
ness of the SMS interception techniques
that exploit flaws in cellular network
protocols and in legacy SS7 networks,
SIM swapping is arguably the number- Figure 5. SIM swap attack.
one security threat against SMS commu-
nications being used to deliver one-time
tokens for multifactor authentication.
As illustrated in Figure 5, a SIM
swap attack consists of fooling a mo- SMS mobile base customer
bile operator, usually over a phone call bank gateway network core station user fraudster service
with customer service, that the legiti-
mate owner of a cellular subscription SMS message flow
needs the account to be ported to a new
SIM card. The caller may claim, for ex- impersonate victim and
port service to a SIM card
ample, that the phone was lost over- in possession of the
seas and access needs to be recovered fraudster
as soon as possible on a newly acquired
customer’s account updated with
phone and a new SIM card. The credi- fraudster’s SIM card information
bility of such a story is actually not that
important; SIM porting attacks are
frustratingly easy to accomplish.18 hijacked SMS message flow

Once an attacker successfully man-

ages to get a victim’s account ported to a

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 51

practice

security risk in America and Europe, it is ent method for the account of a celeb- 12. Inglesant, P.G., Sasse, M.A. The true cost of unusable
password policies: password use in the wild. In
a much mitigated threat in African na- rity or politician. Proceedings of the SIGCHI Conf. Human Factors in
tions.7 For example, cellular operators in The current security landscape is Computing Systems, 2010; 383–392; https://dl.acm.
org/doi/10.1145/1753326.1753384.
Mozambique provide a means for banks very different from that of two decades 13. Jover, R.P. LTE security and protocol exploits.
to check their records for recent SIM ago. Regardless of the critical nature of ShmooCon 2016 Proceedings; https://shmoo.gitbook.
io/2016-shmoocon-proceedings/bring_it_on/05_lte_
swaps for a given account. If a SIM swap an online account or the individual security_and_protocol_exploits.
has recently occurred, the bank will deny who owns it, using a second form of au- 14. Jover, R.P. LTE security, protocol exploits and location
tracking experimentation with low-cost software
a transaction or prevent a security token thentication should always be the de- radio. CoRR, 2016, abs/1607.05171; https://arxiv.org/
from being sent via text message. fault option, regardless of the method abs/1607.05171.
15. Jover, R.P. 5G protocol vulnerabilities and exploits.
Such a simple solution is reported chosen. In the wake of a large number ShmooCon 2020; http://rogerpiquerasjover.net/5G_
ShmooCon_FINAL.pdf.
to have reduced SIM swap-based bank- of leaks and other intrusions, there are 16. Jover, R.P., Marojevic, V. Security and protocol exploit
ing fraud to nearly zero overnight. SIM many username and password combi- analysis of the 5G specifications. IEEE Access,
2019; https://ieeexplore.ieee.org/stamp/stamp.
swapping, however, is still arguably nations out there in the wrong hands jsp?arnumber=8641117.
one of the biggest security risks for the that make password spraying attacks 17. Kune, D.F., Koelndorfer, J., Hopper, N., Kim, Y. Location
leaks on the GSM air interface. In Proceedings of
average consumer of online banking cheap and easy to accomplish. the 19th Annual Network and Distributed System
and financial services. For example, Security Symp., 2012; https://www-users.cs.umn.
edu/~hoppernj/celluloc.pdf.
the prevalence of SIM swap-driven 18. Lee, K., Kaiser, B., Mayer, J., Narayanan, A. An
Related articles
fraud in the U.S. resulted in an official empirical study of wireless carrier authentication for
on queue.acm.org SIM swaps. In Proceedings of the 16th Symp. Usable
warning by New York State’s Division Privacy and Security; https://www.ieee-security.org/
of Consumer Protection.21 VoIP: What is it good for? TC/SPW2020/ConPro/papers/lee-conpro20.pdf.
Sudhir R. Ahuja and J. Robert Ensor 19. Liu, C.-H., Chang, Y.-C., Huang, N.-F., Ling, Y.-L., Jan,
H.-J. CAMEL evolution and PPS evaluation. IEEE
https://queue.acm.org/detail.cfm?id=1028897
Final Thoughts Intelligent Network 2001 Workshop, 9–13. IEEE;
Communications Surveillance: https://ieeexplore.ieee.org/document/915288.
Despite their popularity and ease of use, 20. Mitre Corporation. Exploit SS7 to redirect phone calls/
Privacy and Security at Risk SMS. MITRE ATT&CK Framework; https://attack.
SMS-based authentication tokens are Whitfield Diffie and Susan Landau mitre.org/techniques/T1449/.
arguably one of the least secure forms https://queue.acm.org/detail.cfm?id=1613130 21. New York State Department of Consumer Protection.
ATT SIM-card switch scam; https://www.dos.ny.gov/
of two-factor authentication. This does
ACM CTO Roundtable on consumerprotection/scams/att-sim.html.
not imply, however, that it is an invalid Mobile Devices in the Enterprise 22. Nohl, K. Breaking GSM phone privacy. Black
Hat USA; https://srlabs.de/wp-content/
method for securing an online account. Andrew Toy, André Charland, uploads/2010/07/100729.Breaking.GSM_.Privacy.
True, there are a number of services George Neville-Neil, Carol Realini, BlackHat1-1.pdf.
that should not be used with tokens de- Steve Bourne, Mache Creeger 23. Nohl, K., Munaut, S. Wideband GSM sniffing. In
https://queue.acm.org/detail.cfm?id=2016038 Proceedings of the 27th Chaos Communication
livered via SMS—for example, banking Congress; https://fahrplan.events.ccc.de/
congress/2010/Fahrplan/events/4208.en.html.
and financial services, cryptocurrency 24. Perez, D., Pico, J. A practical attack against GPRS/
References
services, and anything containing sensi- 1. Alfonsi, S. Hacking your phone. CBS News, 2016;
EDGE/UMTS/HSPA mobile data communications.
Black Hat DC, 2011; https://media.blackhat.com/
tive financial information, and credit https://www.cbsnews.com/video/hacking-your-phone/. bh-dc-11/Perez-Pico/BlackHat_DC_2011_Perez-
2. Cimpanu, C. Newer Diameter telephony protocol just
card numbers. Personal email addresses as vulnerable as SS7. Bleeping Computer, 2018; https://
Pico_Mobile_Attacks-wp.pdf.
25. Russell, T. 2002. Signaling System# 7, 2 (2002).
also fall into this category. An email ac- www.bleepingcomputer.com/news/security/newer- McGraw-Hill, New York, NY.
diameter-telephony-protocol-just-as-vulnerable-as-ss7/. 26. Shaik, A., Borgaonkar, R. New vulnerabilities in 5G
count takeover can have devastating con- 3. Coonce, S. The most expensive lesson of my life: networks. Black Hat 2019; https://i.blackhat.com/
sequences if that account is the corner- details of SIM port hack. Medium, 2019; https:// USA-19/Wednesday/us-19-Shaik-New-Vulnerabilities-
medium.com/coinmonks/the-most-expensive-lesson- In-5G-Networks-wp.pdf.
stone to the user’s online digital identity. of-my-life-details-of-sim-port-hack-35de11517124. 27. Shaik, A., Borgaonkar, R., Asokan, N., Niemi, V., Seifert,
On the other hand, there are many 4. Cox, J. Hackers are breaking directly into telecom J.-P. Practical attacks against privacy and availability
companies to take over customer phone numbers. in 4G/LTE mobile communication systems. In
online services for which SMS-based Motherboard Tech by Vice, 2020; https://www.vice. Proceedings of the 23rd Annual Network and
tokens do suffice for the average com/en_us/article/5dmbjx/how-hackers-are-breaking- Distributed System Security Symp; https://www.ndss-
into-att-tmobile-sprint-to-sim-swap-yeh. symposium.org/wp-content/uploads/2017/09/06_5-
consumer—for example, any vanilla 5. Engel, T. SS7: Locate. track. manipulate. 31st Chaos ndss2016-slides_0.pdf.
Communication Congress, 2014.
accounts that store no sensitive or fi- 6. Golde, N., Redon, K., Seifert, J.-P. 2013. Let me answer
28. Third Generation Partnership Project (3GPP)
Technical Specification Group Services and System
nancial information, which attackers that for you: exploiting broadcast information in Aspects. Security architecture and procedures for
cellular networks. In Proceedings of the 22nd Usenix
could not easily monetize, thereby dis- Security Symp. 33–48; https://www.usenix.org/
5G system. 3GPP TS 33.501, V1.0.0, 2018; https://
portal.3gpp.org/desktopmodules/Specifications/
couraging them from trying to take system/files/conference/usenixsecurity13/sec13- SpecificationDetails.aspx?specificationId=3169.
paper_golde.pdf.
over the account in the first place. 7. Greenberg, A. The SIM swap fix that the U.S. isn’t
29. Weber, J.E., Guster, D., Safonov, P., Schmidt, M.B.
Weak password security: An empirical study.
Other variables should be factored using. Wired, 2019; https://www.wired.com/story/sim- Information Security J.: A Global Perspective
swap-fix-carriers-banks/.
into the equation when deciding which 8. Honan, M. How Apple and Amazon security flaws led
17, 1 (2008), 45–54; https://dl.acm.org/
doi/10.1080/10658980701824432.
multifactor authentication method is to my epic hacking. Wired, 2012; https://www.wired. 30. XKCD. Password strength; https://xkcd.com/936/.
com/2012/08/apple-amazon-mat-honan-hacking/.
most appropriate. The security implica- 9. Hunt, T. Beyond passwords: 2FA, U2F and Google
tions for a social media account for a Advanced Protection, 2018; https://www.troyhunt. Roger Piqueras Jover is a senior security architect with
com/beyond-passwords-2fa-u2f-and-google- the CTO Security Architecture Team at Bloomberg, where
well-known individual with millions of advanced-protection/. he is a technical leader in mobile security architecture and
followers are very different from those 10. Hunt, T. Have i been pwned, 2020; https:// strategy, corporate network security architecture, wireless
haveibeenpwned.com/. security analysis and design, and data science applied to
for an account with just a handful of fol- 11. Hussain, S. R., Echeverria, M., Karim, I., Chowdhury, network anomaly detection. He is a technology adviser on
lowers. Therefore, while using SMS as a O., Bertino, E. 5GReasoner: a property-directed the security of LTE/5G mobile networks and wireless short-
security and privacy analysis framework for 5G range networks for academia, industry, and government.
second factor of authentication for cellular network protocol. In Proceedings of the
ACM SIGSAC Conf. Computer and Communications
some social media accounts is perfectly Security, 2019; 669–684; https://dl.acm.org/doi/ Copyright held by author/owner.
valid, it would be wise to opt for a differ- abs/10.1145/3319535.3354263. Publication rights licensed to ACM.

52 COMM UNICATIO NS O F THE AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 Call for Papers (1st Cycle: End of November 2020)
The annual ACM/IFIP Middleware conference is a major forum for the discussion of innovations and recent scientific
advances of middleware systems with a focus on the design, implementation, deployment, and evaluation of
distributed systems, platforms and architectures for computing, storage, and communication. Highlights of the
conference will include a high quality single-track technical program, invited speakers, an industrial track, panel
discussions involving academic and industry leaders, poster and demonstration presentations, a doctoral
symposium, tutorials and workshops.
Topics of Interest: Original submissions of research papers on a diverse range of topics are sought, particularly
those identifying new research directions. The topics of the interest for the conference include, but are not limited
to:
• Cloud and data centers • Networking, network function virtualization, software-
• Virtualization, auto-scaling, provisioning, and defined networking
scheduling • Middleware for multimedia Systems
• Data-intensive computing (big data) and • Fault tolerance and Consistency
data analytics • Blockchains
• Stream Processing • Middleware support for security and privacy
• Middleware Systems for Machine learning • Monitoring, resource management and analysis
• Mobile and pervasive systems and services • Middleware Design principles
• Middleware techniques for IoT, smart cities • Programming abstractions and paradigms for
• Fog, Edge computing middleware
• Middleware for cyber-physical and Real-time • Reconfigurable, adaptable, and reflective approaches
systems • Reviews of middleware paradigms, e.g., object models,
• Energy and power-aware techniques aspect orientation, etc.
• Event-based, publish/subscribe, and peer- • Methodologies and tools for middleware systems design,
to-peer solutions implementation, verification, and evaluation

Important Dates
First Cycle Second Cycle
Abstract Submission: November 20th, 2020 Abstract Submission: May 15th, 2021
Full Paper Submission: December 1st, 2020 Full Paper Submission: May 22nd, 2021
Author Notification: February 28th, 2021 Author Notification: August 15th, 2021
Revised Submissions: April 2nd, 2021 Revised Submissions: September 15th, 2021
Notifications of Revised Papers: April 22nd, 2021 Notifications of Revised Papers: September 30th, 2021

General Co-Chairs PC Co-Chairs

• Kaiwen Zhang, ÉTS Montréal • Nalini Venkatasubramanian, University of California,
• Abdelouahed Gherbi, ÉTS Montréal Irvine
• Luís Veiga, Instituto Superior Técnico (U.Lisboa) &
INESC-ID
For more details, please visit the Middleware 2021 website: http://2021.middleware-conf.org/
contributed articles
DOI:10.1145/ 3381831
This trend is driven by the strong fo-
Creating efficiency in AI research will cus of the AI community on obtaining
“state-of-the-art” results,b as exemplified
decrease its carbon footprint and increase by the popularity of leaderboards,53,54
its inclusivity as deep learning study should which typically report accuracy (or other
not require the deepest pockets. similar measures) but omit any men-
tion of cost or efficiency (see, for ex-
ample, leaderboards.allenai.org).c De-
BY ROY SCHWARTZ, JESSE DODGE,
spite the clear benefits of improving
NOAH A. SMITH, AND OREN ETZIONI model accuracy, the focus on this sin-
gle metric ignores the economic, envi-

Green AI
ronmental, and social cost of reaching
the reported results.
We advocate increasing research
activity in Green AI—AI research that
is more environmentally friendly and
inclusive. We emphasize that Red AI
research has been yielding valuable
scientific contributions to the field,
but it has been overly dominant. We
want to shift the balance toward the
Green AI option—to ensure any in-
spired undergraduate with a laptop
has the opportunity to write high-
quality papers that could be accepted
at premier research conferences. Spe-
SINCE 2012, THE field of artificial intelligence (AI) has cifically, we propose making efficien-
reported remarkable progress on a broad range of cy a more common evaluation criteri-
capabilities including object recognition, game playing, on for AI papers alongside accuracy
and related measures.
speech recognition, and machine translation.43 Much of
this progress has been achieved by increasingly large b Meaning, in practice, that a system’s accuracy
on some benchmark is greater than any previ-
and computationally intensive deep learning models.a ously reported system’s accuracy.
c Some leaderboards do focus on efficiency
Figure 1, reproduced from Amodei et al.,2 plots training (https://dawn.cs.stanford.edu/benchmark/).

cost increase over time for state-of-the-art deep learning

models starting with AlexNet in 201224 to AlphaZero in key insights
2017.45 The chart shows an overall increase of 300,000x, ˽ The computational costs of state-of-the-
art AI research has increased 300,000x in
with training cost doubling every few months. An even recent years. This trend, denoted Red AI,
stems from the AI community’s focus on
sharper trend can be observed in NLP word-embedding accuracy while paying attention to efficiency.
approaches by looking at ELMo34 followed by BERT,8 ˽ Red AI leads to a surprisingly large
openGPT-2,35 XLNet,56 Megatron-LM,42 T5,36 and GPT-3.4 carbon footprint, and makes it difficult for
academics, students, and researchers to
An important paper47 has estimated the carbon engage in deep learning research.

footprint of several NLP models and argued this trend is ˽ An alternative is Green AI, which treats
efficiency as a primary evaluation
ILLUSTRATION BY LISA SH EEH A N

both environmentally unfriendly and prohibitively criterion alonside accuracy. To measure

efficiency, we suggest reporting the
expensive, raising barriers to participation in NLP number of floating-point operations
required to generate a result.
research. We refer to such work as Red AI.
˽ Green AI research will decrease AI’s
a For brevity, we refer to AI throughout this article, but our focus is on AI research that relies on deep environmental footprint and increase
learning methods. its inclusivity.

54 COM MUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 55
contributed articles

AI research can be computationally Figure 1 illustrates, the computational long history of investigating sustainable
expensive in a number of ways, but cost of high-budget research is in- and energy-efficient computing (for ex-
each provides opportunities for effi- creasing exponentially, at a pace that ample, see the Journal Sustainable Com-
cient improvements; for example, pa- far exceeds Moore’s Law.33 Red AI is on puting: Informatics and Systems).
pers can plot performance as a function the rise despite the well-known dimin- In this article, we analyze practices
of training set size, enabling future ishing returns of increased cost (for that move deep-learning research into
work to compare performance even example, Figure 3). the realm of Red AI. We then discuss
with small training budgets. Reporting This article identifies key factors that our proposals for Green AI and con-
the computational price tag of develop- contribute to Red AI and advocates the sider related work, and directions for
ing, training, and running models is a introduction of a simple, easy-to-com- future research.
key Green AI practice (see Equation 1). pute efficiency metric that could help
In addition to providing transparency, make some AI research greener, more Red AI
price tags are baselines that other re- inclusive, and perhaps more cognitively Red AI refers to AI research that seeks
searchers could improve on. plausible. Green AI is part of a broader, to improve accuracy (or related mea-
Our empirical analysis in Figure 2 long-standing interest in environmen- sures) through the use of massive
suggests the AI research community tally friendly scientific research (for ex- computational power while disregard-
has paid relatively little attention to ample, see the Journal Green Chemistry). ing the cost—essentially “buying”
computational efficiency. In fact, as Computer science, in particular, has a stronger results. Yet the relationship
between model performance and
Figure 1. The amount of compute used to train deep learning models has increased model complexity (measured as num-
300,000x in six years. Figure taken from Amodei et al.2
ber of parameters or inference time)
has long been understood to be at best
AlexNet to AlphaGo Zero: A 300,000x Increase in Compute
logarithmic; for a linear gain in per-
10,000
formance, an exponentially larger
AlphaGo Zero • model is required.20 Similar trends ex-
1,000
AlphaZero • ist with increasing the quantity of
100 Neural Machine Translation • training data14,48 and the number of
Neural Architecture Search • experiments.9,10 In each of these cases,
Petaflops/s-day (Training)

10
• Xception• TI7 Dota 1v1 diminishing returns come at in-
1
creased computational cost.
VGG This section analyzes the factors
• DeepSpeech2
.1 •• Seq2Seq • ResNets contributing to Red AI and shows how
• GoogleNet
it is resulting in diminishing returns
.01 over time (see Figure 3). We note that
• AlexNet • Visualizing and Understanding Conv Nets
• Dropout Red AI work is valuable, and in fact,
.001
much of it contributes to what we
.0001 know by pushing the boundaries of AI.
Our exposition here is meant to high-
.00001 light areas where computational ex-
2013 2014 2015 2016 2017 2018 2019 pense is high, and to present each as
Year an opportunity for developing more
efficient techniques.
To demonstrate the prevalence of
Figure 2. AI papers tend to target accuracy rather than efficiency. The figure shows the Red AI, we randomly sampled 60 pa-
proportion of papers that target accuracy, efficiency, both or other from a random sample
of 60 papers from top AI conferences.
pers from top AI conferences (ACL,
NeurIPS, and CVPR).d For each paper
we noted whether the authors claim
Accuracy Both
their main contribution to be (a) an
Efficiency Other
improvement to accuracy or some re-
16
lated measure, (b) an improvement to
14
efficiency, (c) both, or (d) other. As
12
Number of Papers

shown in Figure 2, in all conferences

10
we considered, a large majority of the
8
papers target accuracy (90% of ACL
6
papers, 80% of NeurIPS papers and
4
75% of CVPR papers). Moreover, for
2
both empirical AI conferences (ACL
0
ACL 2018 CVPR 2019 NeurIPS 2018
d https://acl2018.org; https://nips.cc/Conferences/
2018; and http://cvpr2019.thecvf.com.

56 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 contributed articles

Figure 3. Diminishing returns of training on more data: object detection accuracy increases linearly as the number of training examples
increases exponentially.30

Instagram (1.5k tags)/32×4d Instagram (17k tags)/32×4d

Instagram (1.5k tags)/32×8d Instagram (17k tags)/32×8d
Instagram (1.5k tags)/32×16d Instagram (17k tags)/32×16d

Target task: ImageNet–1k Target task: ImageNet–5k

90 60
ImageNet top-1 accuracy (in %)

ImageNet top-1 accuracy (in %)

85
55
80
50
75

70 45

65
40
60
35
55

50 30
107 108 109 107 108 109
Number of training images in source task (Instagram) Number of training images in source task (Instagram)

Target task: ImageNet–9k Target task: CUB2011

50 90
ImageNet top-1 accuracy (in %)

45
80
Accuracy (in %)

40
70
35
60
30

50
25

20 40
107 108 109 107 108 109
Number of training images in source task (Instagram) Number of training images in source task (Instagram)

and CVPR) only a small portion (10% training dataset and evaluated on a
and 20% respectively) argue for a new test dataset, and the process of devel- Cost (R) ∝ E ∙ D ∙ H
efficiency result.e This highlights the oping that model often involves mul-
Equation 1. The equation of Red AI:
focus of the AI community on mea- tiple experiments to tune its hyperpa- The cost of an AI (R)esult grows linearly with
sures of performance such as accura- rameters. We thus consider three the cost of processing a single (E)xample,
cy, at the expense of measures of effi- dimensions that capture much of the the size of the training (D)ataset and the
number of (H)yperparameter experiments.
ciency such as speed or model size. In computational cost of obtaining such
this article, we argue that a larger a result: the cost of executing the
weight should be given to the latter. model on a single (E)xample (either Equation 1 is a simplification (for
To better understand the different during training or at inference time); example, different hyperparameter as-
ways in which AI research can be red, the size of the training (D)ataset, signments can lead to different costs
consider an AI result reported in a sci- which controls the number of times for processing a single example). It
entific paper. This result typically the model is executed during train- also ignores other factors such as the
characterizes a model trained on a ing, and the number of (H)yperparam- number of training epochs or data
eter experiments, which controls how augmentation. Nonetheless, it illus-
e Interestingly, many NeurIPS papers included many times the model is trained dur- trates three quantities that are each an
convergence rates or regret bounds that de- ing model development. The total important factor in the total cost of
scribe performance as a function of exam- cost of producing a (R)esult in ma- generating a result. Next, we consider
ples or iterations, thus targeting efficiency
(55%). This indicates an increased awareness
chine learning increases linearly with each quantity separately.
of the importance of this concept, at least in each of these quantities. This cost can Expensive processing of one example.
theoretical analyses. be estimated as follows: Our focus is on neural models, where it

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 57

contributed articles

is common for each training step to re- can have stronger performance, which uncompressed data,l so even storing the
quire inference, so we discuss training is a valuable scientific contribution. data is expensive. Finally, as in the case of
and inference cost together as “pro- However, this implies the financial and model size, relying on more data to im-
cessing” an example (though see dis- environmental cost of increasingly prove performance is notoriously ex-
cussion below). Some works have used large AI models will not decrease soon, pensive because of the diminishing re-
increasingly large models in terms of, as the pace of model growth far exceeds turns of adding more data.48 For instance,
for example, model parameters, and as the resulting increase in model perfor- Figure 3, taken from Mahajan et al.,30
a result, in these models, performing mance.18 As a result, more and more shows a logarithmic relation between
inference can require a lot of computa- resources are going to be required to the object recognition top-1 accuracy
tion, and training even more so. For in- keep improving AI models by simply and the number of training examples.
stance, Google’s BERT-large8 contains making them larger. Massive number of experiments.
roughly 350 million parameters. Ope- Finally, we note that in some cases Some projects have poured large
nAI’s openGPT2-XL model35 contains the price of processing one example amounts of computation into tuning
1.5 billion parameters. AI2, our home might be different at training and test hyperparameters or searching over neu-
organization, released Grover,57 also time. For instance, some methods tar- ral architectures, well beyond the reach
containing 1.5 billion parameters. get efficient inference by learning a of most researchers. For instance, re-
NVIDIA released Megatron-LM,42 con- smaller model based on the large searchers from Google59 trained over
taining over 8 billion parameters. trained model. These models often do 12,800 neural networks in their neural
Google’s T5-11B36 contains 11 billion not lead to more efficient training, as architecture search to improve perfor-
parameters. Most recently, openAI re- the cost of E is only reduced at infer- mance on object detection and lan-
leased openGPT-3,4 containing 175 bil- ence time. Models used in production guage modeling. With a fixed architec-
lion parameters. In the computer vi- typically have computational costs ture, researchers from DeepMind31
sion community, a similar trend is dominated by inference rather than evaluated 1,500 hyperparameter assign-
observed (Figure 1). training, but in research training is typ- ments to demonstrate that an LSTM
Such large models have high costs ically much more frequent, so we advo- language model17 can reach state-of-
for processing each example, which cate studying methods for efficient the-art perplexity results. Despite the
leads to large training costs. BERT- processing of one example in both value of this result in showing that the
large was trained on 64 TPU chips for training and inference. performance of an LSTM does not pla-
four days at an estimated cost of Processing many examples. Increased teau after only a few hyperparameter
$7,000. Grover was trained on 256 TPU amounts of training data have also trials, fully exploring the potential of
chips for two weeks, at an estimated contributed to progress in state-of-the- other competitive models for a fair
cost of $25,000. XLNet had a similar ar- art performance in AI. BERT-large had comparison is prohibitively expensive.
chitecture to BERT-large, but used a top performance in 2018 across many The value of massively increasing
more expensive objective function (in NLP tasks after training on three bil- the number of experiments is not as
addition to an order of magnitude more lion word-pieces. XLNet outperformed well studied as the first two discussed
data), and was trained on 512 TPU chips BERT after training on 32 billion word- previously. In fact, the number of ex-
for 2.5 days, costing more than $60,000.f pieces, including part of Common periments performed during model
It is impossible to reproduce the best Crawl; openGPT-2-XL trained on 40 bil- construction is often underreported.
BERT-large results or XLNet results us- lion words; FAIR’s RoBERTa28 was Nonetheless, evidence for a logarith-
ing a single GPU,g and models such as trained on 160GB of text, roughly 40 mic relation exists here as well.9,10
openGPT2 are too large to be used in billion word-pieces, requiring around Discussion. The increasing costs of
production.h Specialized models can 25,000 GPU hours to train. T5-11B36 AI experiments offer a natural econom-
have even more extreme costs, such as was trained on 1 trillion tokens, 300 ic motivation for developing more effi-
AlphaGo, the best version of which re- times more than BERT-large. In com- cient AI methods. It might be the case
quired 1,920 CPUs and 280 GPUs to puter vision, researchers from Face- that at a certain point prices will be too
play a single game of Go,44 with an esti- book30 pretrained an image classifica- high, forcing even researchers with
mated cost to reproduce this experi- tion model on 3.5 billion images from large budgets to develop more efficient
ment of $35,000,000.i,j Instagram, three orders of magnitude methods. Our analysis in Figure 2
When examining variants of a single larger than existing labeled image da- shows that currently most effort is still
model (for example, BERT-small and tasets such as Open Images.k being dedicated to accuracy rather
BERT-large) we see that larger models The use of massive data creates barri- than efficiency. At the same time, AI
ers for many researchers to reproducing technology is already very expensive to
f https://syncedreview.com/2019/06/27/the-
staggering-cost-of-training-sota-aimodels/
the results of these models, and to train- train or execute, which limits the abili-
g See https://github.com/google-research/bert ing their own models on the same setup ty of many researchers to study it, and
and https://github.com/zihangdai/xlnet. (especially as training for multiple ep- of practitioners to adopt it. Combined
h https://towardsdatascience.com/too-big-to- ochs is standard). For example, the July with environmental pricetag of AI,47 we
deploy-how-gpt-2-is-breakingproduction- 2019 Common Crawl contains 242TB of believe more effort should be devoted
63ab29f0897c
i https://www.yuzeh.com/data/agz-cost.html
toward efficient AI solutions.
j Recent versions of AlphaGo are far more effi- k https://opensource.google.com/projects/
cient.46 open-images-dataset l http://commoncrawl.org/2019/07/

58 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 contributed articles

We want to reiterate that Red AI we argue should be reported when AI

work is extremely valuable, and in fact, research findings are published.
much of it contributes to what we know Measures of efficiency. To measure
about pushing the boundaries of AI. In- efficiency, we suggest reporting the
deed, there is value in pushing the lim-
its of model size, dataset size, and the Some projects amount of work required to generate a
result. Specifically, the amount of work
hyperparameter search budget.
In addition, Red AI can provide op-
have poured required to train a model, and if appli-
cable, the aggregated amount of work
portunities for future work to promote large amounts required for all hyperparameter tuning
efficiency; for example, evaluating a
model on varying amounts of training
of computation experiments. As the cost of an experi-
ment decomposes into the cost of a
data will provide an opportunity for fu- into tuning processing a single example, the size of
ture researchers to build on the work
without needing a budget large enough
hyperparameters the dataset, and the number of experi-
ments (Equation 1), reducing the
to train on a massive dataset. Current- or searching amount of work in each of these steps
ly, despite the massive amount of re-
sources put into recent AI models, over neural will result in AI that is more green.
We do encourage AI practitioners to
such investment still pays off in terms architectures, well use efficient hardware to reduce energy
of downstream performance (albeit at
an increasingly lower rate). Finding the beyond the reach of costs, but the dramatic increase in
computational cost observed over re-
point of saturation (if such exists) is an
important question for the future of AI.
most researchers. cent years is primarily from modeling
and algorithmic choices; our focus is
Moreover, Red AI costs can even some- on how to incorporate efficiency there.
times be amortized, because a Red AI When reporting the amount of work
trained module may be reused by many done by a model, we want to measure a
research projects as a built-in compo- quantity that allows for a fair compari-
nent, which doesn’t require retraining. son between different models. As a re-
The goal of this article is twofold: sult, this measure should ideally be
first, we want to raise awareness to the stable across different labs, at different
cost of Red AI and encourage researchers times, and using different hardware.
that use such methods to take steps to al- Carbon emission. Carbon emission
low for more equitable comparisons, is appealing as it is a quantity we want
such as reporting training curves. Sec- to directly minimize. Nonetheless it is
ond, we want to encourage the AI com- difficult to measure the exact amount
munity to recognize the value of work by of carbon released by training or exe-
researchers that take a different path, cuting a model, and accordingly—gen-
optimizing efficiency rather than accu- erating an AI result, as this amount de-
racy. Next, we turn to discuss concrete pends highly on the local electricity
measures for making AI more green. infrastructure (though see initial ef-
forts by Henderson et al.16 and Lacoste
GREEN AI et al.25). As a result, it is not comparable
The term Green AI refers to AI research between researchers in different loca-
that yields novel results while taking into tions or even the same location at dif-
account the computational cost, encour- ferent times.16
aging a reduction in resources spent. Electricity usage. Electricity usage is
Whereas Red AI has resulted in rapidly correlated with carbon emission while
escalating computational (and thus car- being time- and location-agnostic.
bon) costs, Green AI promotes approach- Moreover, GPUs often report the
es that have favorable performance/effi- amount of electricity each of their
ciency trade-offs. If measures of cores consume at each time point,
efficiency are widely accepted as im- which facilitates the estimation of the
portant evaluation metrics for research total amount of electricity consumed
alongside accuracy, then researchers by generating an AI result. Nonethe-
will have the option of focusing on the less, this measure is hardware depen-
efficiency of their models with positive dent, and as a result does not allow for a
impact on both inclusiveness and the fair comparison between different mod-
environment. Here, we review several els developed on different machines.
measures of efficiency that could be re- Elapsed real time. The total running
ported and optimized, and advocate time for generating an AI result is a nat-
one particular measure—FPO—which ural measure for efficiency, as all other

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 59

contributed articles

things being equal, a faster model is do- between different approaches, unlike
ing less computational work. Nonethe- the measures described above. Third,
less, this measure is highly influenced FPO is often correlated with the run-
by factors such as the underlying hard- ning time of the model5 (though see
ware, other jobs running on the same
machine, and the number of cores The term Green AI discussion below). Unlike asymptotic
runtime, FPO also considers the
used. These factors hinder the compar-
ison between different models, as well
refers to AI amount of work done at each time step.
Several packages exist for comput-
as the decoupling of modeling contri- research that yields ing FPO in various neural network
butions from hardware improvements.
Number of parameters. Another
novel results while libraries,n though none of them con-
tains all the building blocks required
common measure of efficiency is the taking into account to construct all modern AI models. We
number of parameters (learnable or
total) used by the model. As with run-
the computational encourage the builders of neural net-
work libraries to implement such func-
time, this measure is correlated with cost, encouraging tionality directly.
the amount of work. Unlike the other
measures described previously, it a reduction in Discussion. Efficient machine learn-
ing approaches have received attention
does not depend on the underlying resources spent. in the research community but are gen-
hardware. Moreover, this measure erally not motivated by being green. For
also highly correlates with the example, a significant amount of work
amount of memory consumed by the in the computer vision community has
model. Nonetheless, different algo- addressed efficient inference,13,38,58
rithms make different use of their pa- which is necessary for real-time process-
rameters, for instance by making the ing of images for applications like self-
model deeper vs. wider. As a result, driving cars,27,29,37 or for placing models
different models with a similar num- on devices such as mobile phones.18,40
ber of parameters often perform dif- Most of these approaches only minimize
ferent amounts of work. the cost of processing a single example,
FPO. As a concrete measure, we while ignoring the other two red practic-
suggest reporting the total number of es discussed perviously.o Other meth-
floating-point operations (FPO) re- ods to improve efficiency aim to devel-
quired to generate a result.m FPO pro- op more efficient architectures, starting
vides an estimate of the amount of from the adoption of graphical process-
work performed by a computational ing units (GPU) to AI algorithms, which
process. It is computed analytically by was the driving force behind the deep
defining a cost to two base operations, learning revolution, up to more recent
ADD and MUL. Based on these opera- development of hardware such as ten-
tions, the FPO cost of any machine sor processing units (TPUs22).
learning abstract operation (for exam- The examples here indicate the path
ple, a tanh operation, a matrix multipli- to making AI green depends on how it
cation, a convolution operation, or the is used. When developing a new model,
BERT model) can be computed as a re- much of the research process involves
cursive function of these two opera- training many model variants on a
tions. FPO has been used in the past to training set and performing inference
quantify the energy footprint of a on a small development set. In such a
model,13,32,50,51 but is not widely adopt- setting, more efficient training proce-
ed in AI. FPO has several appealing dures can lead to greater savings, while
properties. First, it directly computes in a production setting more efficient
the amount of work done by the run- inference can be more important. We
ning machine when executing a spe- advocate for a holistic view of computa-
cific instance of a model and is thus tional savings which doesn’t sacrifice in
tied to the amount of energy con- some areas to make advances in others.
sumed. Second, FPO is agnostic to FPO has some limitations. Most im-
the hardware on which the model is portantly, the energy consumption of a
run. This facilitates fair comparisons
n For example, https://github.com/Swall0w/
torchstat; https://github.com/Lyken17/
m Floating point operations are often referred to pytorch-OpCounter
as FLOP(s), though this term is not uniquely o In fact, creating smaller models often results
defined.13 To avoid confusion, we use the term in longer running time, so mitigating the dif-
FPO. ferent trends might be at odds.52

60 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 contributed articles

Figure 4. Increase in FPO leads to diminishing return for object detection top-1 accuracy. Plots (bottom to top): model parameters (in million),
FPO (in billions), top-1 accuracy on ImageNet. 4(a). Leading object recognition models: AlexNet,24 ResNet,15 ResNext,55 DPN107,6 SENet154.19
4(b): Comparison of different sizes (measured by the number of layers) of the ResNet model.15

87 81.3
78.4 79.0 79.7
acc.

70
56.4
55
26
20.8
18.4
15.5
FBO (B)

13 11.6

0.7
0
130
115.1
params (M)

90 83.5
61.1 60.2
50
AlexNet ResNet152 ResNext DPN107 SENet154
2012 2015 2017 2017 2018
Model/Year

82
77.4 78.4
76.0
acc.

75 73.6
70.1
68
15
11.6
FBO (B)

10 7.8
5 3.7 4.1
1.8
0
75
params (M)

50

50

0
18 34 50 101 152
Number of Layers

model is not only influenced by the work done by a model largely depends lead to efficient models should be cred-
amount of work, but also from other on the model implementation, as two ited by the AI community.
factors such as the communication be- different implementations of the same FPO cost of existing models. To
tween the different components, which model could result in very different demonstrate the importance of report-
is not captured by FPO. As a result, FPO amounts of processing work. Due to the ing the amount of work, we present
doesn’t always correlate with other mea- focus on the modeling contribution, the FPO costs for several existing models.q
sures such as runtime21 and energy con- AI community has traditionally ignored Figure 4(a) shows the number of pa-
sumption.16 Second, FPO targets the the quality or efficiency of models’ im- rameters and FPO of several leading
number of operations performed by a plementation.p We argue the time to re- object recognition models, as well as
model, while ignoring other potential verse this norm has come, and that ex- their performance on the ImageNet
limiting factors for researchers such as ceptionally good implementations that
the memory used by the model, which q These numbers represent FPO per inference,
can often lead to additional energy and p We consider this exclusive focus on the final that is, the work required to process a single
monetary costs.29 Finally, the amount of prediction another symptom of Red AI. example.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 61

contributed articles

dataset.7,r A few trends are observable. to recognize and value contributions climate change. For example, ma-
First, as discussed earlier, models get that do not strictly improve state of the chine learning has been used for re-
more expensive with time, but the in- art but have other benefits such as effi- ducing emissions of cement plants1
crease in FPO does not lead to similar ciency. Finally, we note that the trend of and tracking animal conservation
performance gains. For instance, an releasing pretrained models publicly is outcomes,12 and is predicted to be
increase of almost 35% in FPO between a green success, and we would like to useful for forest fire management.39
ResNet and ResNext (second and third encourage organizations to continue Undoubtedly these are important ap-
points in graph) resulted in a 0.5% top-1 to release their models in order to save plications of machine learning; we
accuracy improvement. Similar pat- others the costs of retraining them. recognize they are orthogonal to the
terns are observed when considering content of this article.
the effect of other increases in model Related Work
work. Second, the number of model pa- Recent work has analyzed the carbon Conclusion
rameters does not tell the whole story: emissions of training deep NLP mod- The vision of Green AI raises many ex-
AlexNet (first point in the graph) actu- els47 and concluded that computation- citing research directions that help to
ally has more parameters than ResNet ally expensive experiments can have a overcome the challenges of Red AI.
(second point), but dramatically less large environmental and economic Progress will find more efficient ways
FPO, and also much lower accuracy. impact. With modern experiments us- to allocate a given budget to improve
Figure 4(b) shows the same analysis ing such large budgets, many re- performance, or to reduce the compu-
for a single object recognition model, searchers (especially those in aca- tational expense with a minimal re-
ResNet,15 while comparing different ver- demia) lack the resources to work in duction in performance. Also, it would
sions of the model with different num- many high-profile areas; increased seem that Green AI could be moving us
bers of layers. This creates a controlled value placed on computationally effi- in a more cognitively plausible direc-
comparison between the different mod- cient approaches will allow research tion as the brain is highly efficient.
els, as they are identical in architecture, contributions from more diverse It is important to reiterate that we see
except for their size (and accordingly, groups. We emphasize that the con- Green AI as a valuable option, not an ex-
their FPO cost). Once again, we notice clusions of Stubell et al.47 are the re- clusive mandate—of course, both Green
the same trend: the large increase in sult of long-term trends, and are not AI and Red AI have contributions to
FPO cost does not translate to a large in- isolated within NLP, but hold true make. Our goals are to augment Red AI
crease in performance. across machine learning. with green ideas, like using more effi-
Additional ways to promote Green While some companies offset elec- cient training methods, and reporting
AI. There are many ways to encourage tricity usage by purchasing carbon training curves; and to increase the prev-
research that is more green. In addi- credits, it is not clear that buying cred- alence of Green AI by highlighting its
tion to reporting the FPO cost for each its is as effective as using less energy. benefits, advocating a standard measure
term in Equation 1, we encourage re- In addition, purchasing carbon cred- of efficiency. Here, we point to a few im-
searchers to report budget/perfor- its is voluntary; Google clouds and Mi- portant green research directions, and
mance curves where possible. For ex- crosoft Azuret purchase carbon credits highlight a few open questions.
ample, training curves provide to offset their spent energy, but Ama- Research on building space- or time-
opportunities for future researchers to zon’s AWSu (the largest cloud comput- efficient models is often motivated by
compare at a range of different bud- ing platformv) only covered 50% of its fitting a model on a small device (such
gets and running experiments with dif- power usage with renewable energy. as a phone) or fast enough to process ex-
ferent model sizes provides valuable The push to improve state-of-the-art amples in real time, such as image cap-
insight into how model size impacts performance has focused the research tioning for the blind (as discussed previ-
performance. In a recent paper,9 we ob- community’s attention on reporting the ously). Here, we argue for a far broader
served that the claim as to which model single best result after running many ex- approach that promotes efficiency for
performs best depends on the compu- periments for model development and all parts of the AI development cycle.
tational budget available during model hyperparameter tuning. Failure to fully Data efficiency has received signifi-
development. We introduced a method report these experiments prevents fu- cant attention over the years.23,41,49
for computing the expected best vali- ture researchers from understanding Modern research in vision and NLP of-
dation performance of a model as a how much effort is required to repro- ten involves first pretraining a model
function of the given budget. We argue duce a result or extend it.9 on large “raw” (unannotated) data
that reporting this curve will allow us- Our focus is on improving efficien- then finetuning it to a task of interest
ers to make wiser decisions about their cy in the machine learning communi- through supervised learning. A strong
selection of models and highlight the ty, but machine learning can also be result in this area often involves
stability of different approaches. used as a tool for work in areas like achieving similar performance to a
We further advocate for making ef- baseline with fewer training examples
ficiency an official contribution in ma- s https://cloud.google.com/sustainability/ or fewer gradient steps. Most recent
jor AI conferences by advising reviewers t https://www.microsoft.com/en-us/environ- work has addressed fine-tuning data,34
ment/carbon
u https://aws.amazon.com/about-aws/sustain-
but pretraining efficiency is also im-
r Numbers taken from https://github.com/ ability/ portant. In either case, one simple
sovrasov/flops-counter.pytorch. v https://tinyurl.com/y2kob969 technique to improve in this area is to

62 COM MUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 contributed articles

simply report performance with dif- and restoration efforts. In Proceedings of ICML 43. Shoham, Y. et al. The AI index 2018 annual report.
Workshop on Climate Change, 2019. AI Index Steering Committee, Human-Centered AI
ferent amounts of training data. For 13. Gordon, A., Eban, E., Nachum, O., Chen, B., Wu, H., Initiative, Stanford University; http:// cdn.aiindex.org/
example, reporting performance of Yang, T-J, and Choi, E. MorphNet: Fast & simple 2018/AI%20Index%202018%20Annual%20Report.pdf.
resource-constrained structure learning of deep 44. Silver, D. et al. Mastering the game of Go with deep
contextual embedding models trained networks. In Proceedings of CVPR, 2018. neural networks and tree search. Nature 529, 7587
on 10 million, 100 million, 1 billion, 14. Halevy, A., Norvig, P. and Pereira, F. The unreasonable (2016) 484.
effectiveness of data. IEEE Intelligent Systems 24 45. Silver, D. et al. Mastering chess and shogi by self-play
and 10 billion tokens would facilitate (2009), 8–12. with a general reinforcement learning algorithm, 2017;
faster development of new models, as 15. He, K., Zhang, X., Ren, S. and Sun, J. Deep residual arXiv:1712.01815.
learning for image recognition. In Proceedings of 46. Silver, D. et al. Mastering the game of Go without
they can first be compared at the CVPR, 2016. human knowledge. Nature 550, 7676 (2017), 354.
smallest data sizes. 16. Henderson, P., Hu, J., Romoff, J., Brunskill, E., 47. Strubell, E., Ganesh, A. and McCallum, A. Energy and
Jurafsky, D. and Pineau, J. Towards the systematic policy considerations for deep learning in NLP. In
Research here is of value not just to reporting of the energy and carbon footprints of Proceedings of ACL, 2019.
machine learning, 2020; arXiv:2002.05651. 48. Sun, C., Shrivastava, A., Singh, S. and Gupta, A.
make training less expensive, but be- Revisiting unreasonable effectiveness of data in deep
17. Hochreiter, S. and Schmidhuber, J. Long short-term
cause in areas such as low resource lan- memory. Neural Computation 9, 8 (1997), 1735–1780. learning era. In Proceedings of ICCV, 2017.
18. Howard, A.G. et al. MobileNets: Efficient convolutional 49. Tsang, I., Kwok, J.T. and Cheung, P.M. Core vector
guages or historical domains it is ex- neural networks for mobile vision applications, 2017; machines: Fast SVM training on very large data sets.
tremely difficult to generate more data, arXiv:1704.04861. JMLR 6 (Apr. 2005), 363–392.
19. Hu, J., Shen, L. and Sun, G. Squeeze-and-excitation 50. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones,
so to progress we must make more ef- networks. In Proceedings of CVPR, 2018. L., Gomez, A.N., Kaiser, L. and Polosukhin, I. Attention is
ficient use of what is available. 20. Huang, J. et al. Speed/accuracy trade-offs for modern all you need. In Proceedings of NeurIPS, 2017.
convolutional object detectors. In Proceedings of 51. Veniat, T. and Denoyer, L. Learning time/memory-
Finally, the total number of experi- CVPR, 2017. efficient deep architectures with budgeted super
ments run to get a final result is often 21. Jeon, Y. and Kim, J. Constructing fast network through networks. In Proceedings of CVPR, 2018.
deconstruction of convolution. In Proceedings of 52. Walsman, A., Bisk, Y., Gabriel, S., Misra, D., Artzi, Y.,
underreported and underdiscussed.9 NeurIPS, 2018. Choi, Y. and Fox, D. Early fusion for goal directed
The few instances researchers have of 22. Jouppi, N.P. et al. In-datacenter performance analysis robotic vision. In Proceedings of IROS, 2019.
of a tensor processing unit. In Proceedings of ISCA 1, 53. Wang, A. Pruksachatkun, Y., Nangia, N., Singh,
full reporting of the hyperparameter 1 (2017), Publ. date: June 2020. A., Michael, J., Hill, F., Levy, O. and Bowman, S.R.
search, architecture evaluations, and 23. Kamthe, S. and Deisenroth, M.P. Data-efficient SuperGLUE: A stickier benchmark for general-
reinforcement learning with probabilistic model purpose language understanding systems, 2019;
ablations that went into a reported ex- predictive control. In Proceedings of AISTATS, 2018. arXiv:1905.00537.
perimental result has surprised the 24. Krizhevsky, A., Sutskever, I. and Hinton, G.E. Imagenet 54. Wang, A., Singh, A., Michael, J., Hill, F., Levy, O. and
classification with deep convolutional neural Bowman, S.R. GLUE: A multi-task benchmark and
community.47 While many hyperpa- networks. In Proceedings of NeurIPS, 2012. analysis platform for natural language understanding.
In Proceedings of ICLR, 2019.
rameter optimization algorithms ex- 25. Lacoste, A., Luccioni, A., Schmidt, V. and Dandres, T.
55. Xie, S., Girshick, R., Dollar, P., Tu, Z. and He, K.
Quantifying the carbon emissions of machine learning. In
ist, which can reduce the computa- Proceedings of the Climate Change AI Workshop, 2019. Aggregated residual transformations for deep neural
networks. In Proceedings of CVPR, 2017.
tional expense required to reach a 26. Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A. and
56. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov,
Talwalkar, A. Hyperband: Bandit-based configuration
given level of performance,3,11 simple evaluation for hyperparameter optimization. In R. and Le, Q.V. XLNet: Generalized autoregressive
Proceedings of ICLR, 2017. pretraining for language understanding, 2019;
improvements here can have a large arXiv:1906.08237.
27. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed,
impact. For example, stopping train- S. Fu, C-Y and Berg, A.C. SSD: Single shot multibox 57. Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi,
detector. In Proceedings of ECCV, 2016. A., Roesner, F. and Choi, Y. Defending against neural
ing early for models that are clearly fake news, 2019; arXiv:1905.12616.
28. Liu, Y. et al. RoBERTa: A robustly optimized BERT
underperforming can lead to great pretraining approach, 2019; arXiv:1907.11692. 58. Zhang, X., Zhou, X., Lin, M. and Sun, J. ShuffleNet: An
29. Ma, N., Zhang, X., Zheng, H.T and Sun, J. ShuffleNet extremely efficient convolutional neural network for
savings.26 V2: Practical guidelines for efficient cnn architecture mobile devices. In Proceedings of CVPR, 2018.
Acknowledgment. This research was design. In Proceedings of ECCV, 2018. 59. Zoph, B. and Le, Q.V. Neural architecture search with
30. Mahajan, D. et al. Exploring the limits of weakly reinforcement learning. In Proceedings of ICLR, 2017.
conducted at the Allen Institute for AI. supervised pretraining, 2018; arXiv:1805.00932.
31. Melis, G., Dyer, C. and Blunsom, P. On the state of Roy Schwartz ([email protected]) is Senior Lecturer at
References the art of evaluation in neural language models. In the Hebrew University of Jerusalem, Israel.
1. Acharyya, P., Rosario, S.D., Flor, F., Joshi, R., Li, D., Linares, Proceedings of EMNLP, 2018.
R, and Zhang, H. Autopilot of cement plants for reduction 32. Molchanov, P., Tyree, S., Karras, T., Aila, T. and Kautz, Jesse Dodge ([email protected]), Language
of fuel consumption and emissions. In Proceedings of J. Pruning convolutional neural networks for resource Technologies Institute, Carnegie Mellon University,
ICML Workshop on Climate Change, 2019. efficient inference. In Proceedings of ICLR, 2017. Pittsburgh, PA, USA.
2. Amodei, D. and Hernandez, D. AI and compute, 2018. 33. Moore, G.E. Cramming more components onto
Blog post. integrated circuits, 1965. Noah A. Smith ([email protected]) is a professor of
3. Bergstra, J.S., Bardenet, R., Bengio, Y. and Kégl, B. 34. Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, computer science and engineering at the University
Algorithms for hyper-parameter optimization. In C., Lee, K. and Zettlemoyer, L. Deep contextualized of Washington and senior research manager for the
Proceedings of NeurIPS, 2011. word representations. In Proceedings of NAACL, 2018. AllenNLP team at Allen Institute for AI and, Seattle,
4. Brown, T.B. et al. Language models are few-shot 35. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D. and WA, USA.
learners, 2020; arXiv:2005.14165. Sutskever, I. Language m odels are unsupervised
5. Canziani, A., Paszke, A. and Culurciello, E. An multitask learners.. OpenAI Blog, 2019. Oren Etzioni ([email protected]) is Chief Executive
analysis of deep neural network models for practical 36. Raffel, C. et al. Exploring the limits of transfer Officer of the Allen Institute for AI, and a professor
applications. In Proceedings of ISCAS, 2017. learning with a unified text-to-text transformer, 2019; of computer science at the University of Washington,
6. Chen, Y., Li, J., Xiao, H., Jin, X., Yan, S. and Feng, J. Dual arXiv:1910.10683. Seattle, WA, USA.
path networks. In Proceedings of NeurIPS, 2017. 37. Rastegari, M., Ordonez, V., Redmon, J. and Farhadi,
7. Deng, J., Dong, W., Socher, R., Li, L-J, Li, K. and Fei- A. Xnornet: Imagenet classification using binary
Fei, L. ImageNet: A large-scale hierarchical image convolutional neural networks. In Proceedings of Copyright held by authors/owners.
database. In Proceedings of CVPR, 2009. ECCV, 2016. This work is licensed under a Creative Commons
8. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. BERT: 38. Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. You Attribution International 4.0 License.
Pretraining of deep bidirectional transformers for only look once: Unified, real-time object detection. In
language understanding. In Proceedings of NAACL, 2019. Proceedings of CVPR, 2016.
9. Dodge, J., Gururangan, S., Card, D., Schwartz, R. and 39. Rolnick, D. et al. Tackling climate change with machine
Smith, N.A. Show your work: Improved reporting of learning, 2019; arXiv:1905.12616.
experimental results. In Proceedings of EMNLP, 2019. 40. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A. and
10. Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Chen, L.C. MobileNetV2: Inverted residuals and linear
Hajishirzi, H. and Smith, N.A. Fine-tuning pretrained bottlenecks. In Proceedings of CVPR, 2018.
language models: Weight initializations, data orders, 41. Schwartz, R., Thomson, S. and Smith, N.A. SoPa:
and early stopping, 2020; arXiv:2002.06305. Bridging CNNs, RNNs, and weighted finite-state
11. Dodge, J., Jamieson, K. and Smith, N.A. Open loop machines. In Proceedings of ACL, 2018. Watch the authors discuss
hyperparameter optimization and determinantal point 42. Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, this work in the exclusive
processes. In Proceedings of AutoML, 2017. J., Catanzaro, B. Megatron-LM: Training multi-billion Communications video.
12. Duhart, C., Dublon, G., Mayton, B., Davenport, G. and parameter language models using GPU model https://cacm.acm.org/videos/
Paradiso, J.A. Deep learning for wildlife conservation parallelism, 2019; arXiv:1909.08053. green-ai

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 63

contributed articles
DOI:10.1145/ 3408864
Understanding the individual, so-
Tracing the relationship between pathological cial, and organizational influences on
insider threats is important to the de-
personality traits and insider cyber sabotage. velopment of security-related poli-
cies and controls. Cyber sabotage as
BY MICHELE MAASBERG, CRAIG VAN SLYKE, part of a broader insider threat issue
SELWYN ELLIS, AND NICOLE BEEBE is addressed in the context of an or-
ganizational security risk manage-

The Dark Triad

ment plan. Such plans should in-
clude security controls intended to
mitigate the risk of a human threat
from the inside. In the U.S., in some

and Insider
cases in which classified material is
involved, formal insider threat cyber
security programs are mandated by
Presidential Executive Order.

Threats in
The security controls prescribed by
insider threat programs often in-
clude automated employee monitor-
ing systems for detection, education

Cyber Security
and training programs for aware-
ness.9 These controls often include
technical and behavioral indicators
derived from the observed psycholog-
ical traits and specific behaviors of
high-risk insiders. These indicators
should be based on empirical evi-
dence in order to avoid false accusa-
tions that harm employees7 and nega-
tive ethical and legal consequences
associated with biased systems.12
Insiders possess unique personal
“I WAS D ISM AYED to learn this weekend about a predispositions, stressors, and concern-
Tesla employee who had conducted quite extensive ing behaviors that have been identified
and damaging sabotage to our operations. This as risk factors; these have been included
in models of insider threat behaviors.8,18
included making direct code changes to the Tesla Past research suggests that robust cyber-
Manufacturing Operating System under false security systems include psychological
or personality factors in their design.9
usernames and exporting large amounts of highly Several insider threat frameworks in-
sensitive Tesla data to unknown third parties.” clude personal predispositions (includ-
—Tesla CEO Elon Musk in ing personality traits) as the origin point
of threat behaviors.8,17,18 This suggests it
an email to Tesla employees.6 is important to recognize personal fac-
IMAGE BY ALICIA KUBISTA /A ND RIJ BORYS ASSOCIAT ES

tors, especially personality traits, before

Insider cyber sabotage4 such as that mentioned by
Mr. Musk is one of the reasons cyber security remains key insights
a top managerial concern. Insider threats, such as ˽ Malicious insider threats often exhibit
the Tesla sabotage, are among the greatest of these the malevolent personality traits
subclinical narcissism, psychopathy, and
security concerns.24 A major reason for this is that Machiavellianism known as the dark triad.

insider security breaches are seen as more costly than ˽ The cost of hiring a toxic worker typically
exceeds any benefit that they might bring
those from outsiders.16,20 to an organization.

64 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 65
contributed articles

they lead to malicious behaviors. Such nizational resources, it may put the
recognition can be the earliest point of survival of the organization at risk.
threat agent identification. When discussing insider threats, it is
Much of the existing research into useful to distinguish between malicious
personality traits and cybersecurity is
based on case studies, anecdotal evi- Machiavellians and unintentional threats. Not surpris-
ingly, the key difference is intent. Unin-
dence, or conceptual reasoning. There
is a lack of quantitative empirical evi-
engage in bad tentional threats come from actions (or
inactions) undertaken without any ma-
dence to guide our understanding of behaviors for some licious intent. Using an easy-to-guess
the relationship between personality
traits and insider threats.13 Under-
gain, narcissists password or responding to a phishing
email are examples of unintentional
standing the role of traits related to an- engage in bad threats. In contrast, malicious threats
tisocial behavior in malicious insider
threats is especially important due to
behaviors because come from intentional acts. The CERT
National Insider Threat Center (NITC)
the link between these traits and ma- they are only defines a malicious insider threat as “a
levolent behavior. The findings of our
research may help enhance and extend concerned with current or former employee, contractor,
or business partner who has or had au-
existing models and frameworks in- themselves, and thorized access to an organization’s net-
cluding advanced technical systems.
In this article, we focus on a set of psychopaths work, system, or data and intentionally
exceeded or misused that access in a
pathological personality traits known as
the dark triad. Evidence from recent in-
behave badly for manner that negatively affected the
confidentiality, integrity, or availability
sider threat cases leads us to believe these the thrill, of the organization’s information or in-
traits may correlate with intentions to en-
gage in malicious behavior.23 After dis-
regardless formation systems.”22 The research pre-
sented here pertains to malicious, rath-
cussing insider threats and the dark triad of the risk er than unintentional, insider threats.
traits, we present results from an empiri-
cal study that illustrate the relationship to themselves Malicious insider threats are often
described by the nature of the crime or
between the dark triad traits and mali- or an organization. abuse.7 For example, a common catego-
cious intent. We then discuss the impor- rization of malicious insider threats in-
tance of these results and make recom- cludes espionage, cyber sabotage, fraud,
mendations for security managers and and theft of intellectual property.1,20
practitioners based on our findings. De- Cyber sabotage, or the infliction of
spite the inclusion of personality traits in harm on some area of an organization
insider threat frameworks, to our knowl- using technology,4,20 can result in par-
edge no known studies have empirically ticularly significant and diversified
investigated the relationships between damage to that organization.5
the dark triad traits (individually or col- There are numerous methods for
lectively) and insider cyber sabotage. The dealing with the threat of insider sabo-
findings of our research may help en- tage. These include technical and ad-
hance and extend existing models and ministrative preventive and deterrent
frameworks of insider threat behavior. mitigation techniques. Technical ap-
Additionally, the findings may contribute proaches include user and enterprise
to empirically validating rulesets in tech- level systems for detection focusing on
nical systems and traits used in insider monitoring of cyber data.9 These sys-
threat training and awareness programs. tems include capabilities for collection,
storage, analysis, and reporting based
Background on activities and actions of individuals.
Insider threats. Insiders represent Administrative controls include train-
greater threats to organizations than ing and awareness programs, security
outsiders due to their access to organi- policies, and processes that include se-
zational information and information curing system access paths upon pre-
systems, especially when coupled with cipitating events, such as demotion or
their advanced organizational knowl- termination.20 Our research can be used
edge and the trust that is often afford- to enhance both technical and adminis-
ed to them. Insider threats exist when trative approaches, as discussed later.
trusted current or former organization- Personality factors, particularly
al members act in ways that expose the pathological personality traits, have
organization to risk.9 Inappropriate in- been cited as one of three essential fac-
sider behavior not only threatens orga- tors predisposing individuals to mali-

66 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 contributed articles

cious insider threat behavior (along The dark triad consists of three so- Table 1 summarizes key characteristics
with opportunity and states of crisis).23 cially averse personality traits—Machi- the dark triad personalities.
Past insider threat cases have noted key avellianism, narcissism, and psychopa- While all three dark triad personali-
personal predispositions as precursors thy.15 All three of these exhibit a socially ties are socially malevolent, they differ
to espionage and cyber sabotage. These malevolent character: self-promotion, in how social interactions and others
predispositions include an unusual emotional coldness, duplicity, and ag- are viewed. All three are unconcerned
need for attention, sense of entitle- gressiveness. However, the three traits with any potential negative impacts
ment, arrogance, impulsivity, lack of exhibit these tendencies to different their behaviors may have on others. Ma-
conscience, and lack of empathy.1 degrees. Further, all three are to some chiavellians seem to be concerned
These key characteristics reported of extent manipulative, Machiavellianism about how they can manipulate and use
convicted insiders are also elements of most markedly so. Machiavellianism is others to achieve personal goals, with-
the dark triad of personality. a manipulative personality character- out consideration of how others might
The underlying psychology of indi- ized by interpersonal relationship strat- be negatively affected. Machiavellians
vidual threat agents lies at the heart of egies oriented toward manipulation, will manipulate and exploit others, but
the insider threat problem. Although self-interest, and deception. Those with with some goal in mind. If the manipu-
technical controls are helpful in miti- Machiavellian personalities are primar- lation and exploitation is unlikely to ad-
gating harmful behaviors, they are in- ily concerned with self-interest and are vance the Machiavellian’s goal, the Ma-
sufficient. As experience repeatedly dem- typically unconcerned about others be- chiavellian is unlikely to bother.
onstrates, insider threat behaviors occur yond how they can serve that self-inter- Narcissists are highly self-focused.
in spite of sophisticated technical secu- est. Not surprisingly, Machiavellian- They will engage in malevolent behav-
rity mechanisms. One reason for this is ism is negatively correlated with ior, but that behavior may be due to the
that these mechanisms typically detect empathy. Narcissism is characterized narcissist’s sense of grandiosity, enti-
threat activity only after the activity oc- by a general sense of superiority, gran- tlement, and superiority rather than an
curs. In addition, clever bad actors are diosity, entitlement, and dominance. explicit desire to do harm or negatively
often able to circumvent technical secu- Narcissists focus on themselves and impact others. Narcissists do not care
rity controls. On the other hand, com- have an inflated self-view. As is the case about impacts on others because they
mon administrative controls such as se- with Machiavellianism, narcissism is view others as unimportant. Narcissists
curity policies and associated sanctions negatively correlated with empathy. are less likely than Machiavellians or
may not account for the underling psy- Psychopathy is characterized by an ar- psychopaths to consciously engage in
chology of malevolent individuals. As we rogant, deceitful approach to relation- behavior that harms others. Narcissists
explain later, such individuals may ig- ships, along with high impulsivity and engage in such behaviors because they
nore policies and be unmoved by the thrill-seeking, and irresponsible be- do not think of others.
risks associated with potential sanc- havior. In addition, psychopathic per- A more complex set of factors lead to
tions. Because of this, it is important to sonalities are deficient in affect, and malevolent behaviors among those
understand the traits of atypical, malev- exhibit low empathy and low anxiety. with psychopathic personalities. These
olent insiders. Thus, neither technical
nor administrative controls alone can Table 1. Dark Triad traits.1,11,15,22
address the malicious insider threat
Key Characteristics Machiavellianism Narcissism Psychopathy
problem. There needs to be an ad-
vanced holistic approach that considers Duplicity × × ×
insiders’ psychological factors. Fair and Self-promotion × × ×
trustworthy algorithms to support the Aggressiveness × × ×
advanced systems depend on empirical Interpersonal coldness × × ×
evidence derived from rigorous studies. Tendency to manipulate and exploit others × × ×
Dark triad. There are numerous mod-
Sense of superiority × × ×
els of personality. Perhaps the most com-
monly known is the five-factor model, Low empathy × × ×
which consists of five constructs of per- Callousness/lack of conscience × × ×
sonality that are robust across cultures.2 Attention to reputation × ×
However, according to some, the five- Cynical world view ×
factor model fails to fully account for Strategic calculation ×
individual differences in personality-re- Sense of entitlement ×
lated behaviors, particularly when relat-
Sense of grandiosity ×
ed to antisocial behaviors.22 Recent re-
Ego-reinforcement
search addresses this weakness by all-consuming motive
×
adding traits that represent socially ma-
Thrill-seeking ×
levolent behavior. The dark triad of per-
sonality represents a set of personality Low anxiety ×
characteristics that are only partially ac- Lack of impulse control ×
counted for by the five-factor model.22

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 67

contributed articles

individuals are not focused on the end a raise, accessed a restricted database three on narcissism had a dark triad
goal that drives the Machiavellian. They that had been left open inadvertently and score of four. This measure gives a more
engage in malevolent behavior for the discovered that several peers had 20% holistic view of the relationship be-
thrill. Further, due to their low levels of higher salaries than the employee. The tween the dark triad concept and ma-
anxiety they are not concerned about employee copied the database and post- levolent intent. After we computed the
“getting caught.” These factors, when ed it to the company’s website. Subjects dark triad index score, we prepared a
combined with deficient affect and low were asked to put themselves in the place scatter plot with intent on the x-axis and
empathy, make for a dangerous combi- of the employee when answering ques- the dark triad index score on the y-axis.
nation. To summarize, Machiavellians tions regarding the sabotage performed Then we used color to represent the psy-
engage in bad behaviors for some gain, by the employee. A total of 768 usable ob- chopathy score, and symbol size to indi-
narcissists engage in bad behaviors be- servations were obtained after removing cate the Machiavellianism score. Narcis-
cause they are only concerned with cases with missing data. The participants sism is shown by the three symbols; with
themselves, and psychopaths behave ranged in age from 18 to 73 years, with high, moderate, and low narcissism
badly for the thrill, regardless of the mean age of 34.7 years. The sample was scores shown by the circle, plus sign, and
risk to themselves or an organization. 42% female and 58% male. All partici- square respectively. We defined the three
Dark triad personality traits can pre- pants were employed, with a mean ten- different levels according to the scores
dict and explain workplace behavior.10 ure in their current position of 6.08 years. distance from the mean, with plus or mi-
This thinking has been applied after the We used previously validated scales nus one standard deviation from the
fact to explain insider threat incidents. to measure the dark triad traits11 and a mean being high and low respectively.
Numerous insiders involved in well- previously validated scale for revenge, The right-hand side of the plot may be
known threat cases displayed personal- which was adapted to the vignette, to thought of as the danger zone, as these in-
ity traits similar to those included in the measure intentions.3 All scale items dividuals are above the intention mid-
dark triad. While not labeled as such, were measured on a seven-point scale. point. There is an interesting contrast
current insider threat research identi- The relationships were tested using between the lower- and upper-right
fies several dark triad traits as being re- Mplus Version 7 software to run covari- quadrants. The lower-right quadrant is
lated to insider threats, including a ance based latent variable modeling the least populated quadrant. One inter-
sense of entitlement and lack of empa- using weighted least squares means pretation of this is that, generally speak-
thy.19 However, to date no known study and adjusted variances (WLSMV) esti- ing, those who have low dark triad scores
has used the dark triad as a framework mation for categorical data using a are unlikely to have malevolent inten-
for examining insider cyber sabotage. polychoric correlation matrix. All fit in- tions. Further, no individual with a dark
Despite the emergence of dark triad dices met the minimum requirements triad score of less than approximately
personality traits in insider threat cas- for interpretations of results. Validity 1.75 is on the higher end of the intent
es, there is a lack of systematic empiri- and reliability were satisfactory. scale. In contrast, the upper-right quad-
cal research into the relationship be- The results of model testing showed rant has numerous circles. Circles in this
tween personality traits and insider that the relationships between each of quadrant are relatively large, indicating
threat behavior. Our study addresses the dark triad traits and intentions to en- high Machiavellianism scores, and tend
this issue by empirically demonstrat- gage in insider threat behavior were pos- to be red, rather than blue, indicating
ing the link between dark triad traits itive and significant, as shown in Table relatively high psychopathy scores. Tak-
and intentions to engage in an insider 2. The strength of the relationships var- en together, these results seem to indi-
threat behavior (cyber sabotage). ied across the traits. Psychopathy had cate meaningful relationships between
the strongest relationship (β = 0.559, dark triad scores and intention to com-
Methods and Results p < 0.001), followed by Machiavellianism mit insider cyber sabotage.
The study was conducted using the Ama- (β = 0.379, p < 0.001) and narcissism (β = We must caution that this visual anal-
zon Mechanical Turk (MTurk) market- 0.286, p < 0.001). ysis is exploratory. To our knowledge,
place to deploy an experimental vignette To further examine the relationship there is no established method for creat-
to a sample of working professionals between the dark triad and intent, we ing a formal dark triad score. Further,
from a variety of technology, healthcare, created an index of the dark triad by there is no theoretically or empirically
manufacturing, finance, academic, and computing a mean of an individual’s established rationale for causal relation-
service-oriented organizations. The vi- score on all three traits. For example, an ships among the dark triad traits. For
gnette described an event in which an individual who scored five on Machia- these reasons, we are reluctant to per-
employee, who had recently been denied vellianism, a four on psychopathy, and a form statistical tests on the relationship
between dark triad scores and inten-
Table 2. Results. tions; such tests may imply a higher level
of precision that we can claim.
Dark Triad Trait Beta P-value
Machiavellianism 0.379 < 0.001
Discussion
Narcissism 0.286 < 0.001
Our results clearly show the link between
Psychopathy 0.559 < 0.001
the dark triad and intentions to engage
in malicious insider threat behavior.
While there are strong, positive relation-

68 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 contributed articles

ships between each of the three dark tri- Figure 1. Dark triad index and insider cyber sabotage intention scatterplot.
ad traits and intentions, the strength of
the relationships varied across the traits.
Psychopathy had the strongest relation- Psychopathy
ship, followed by Machiavellianism and 7.0 1.00 5.67

narcissism. In this section, we discuss 6.5

these results, including implications,
recommendations for practice, and ave- 6.0 Machiavellianism
nues for future research. 5.5
1.00
2.00
Before discussing our results, it is im- 4.00

Dark Triad Index

portant to understand that the presence 5.0 6.00
7.00
of dark triad traits is only one potential 4.5
precursor to insider threat behavior. A
sound cybersecurity risk assessment 4.0

system should include assessing the Narcissism

3.5
High
dark triad traits along with other indi-
Low
vidual and organizational factors.9 The 3.0
Medium
presence of dark triad traits, when found 2.5
with these other factors, increases risk.
Our research answers the call for em- 2.0

pirical validation of precursors of insider 1.5

threats9 by providing evidence of the rela-
tionship between specific personality 1.0

traits and cyber sabotage. Numerous 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0
Intention
frameworks, guidelines and awareness
campaigns include personal characteris-
tics, which demonstrates the impor-
tance of personality in cyber security risk viduals may bring is important. Employ- thorough screening through multiple
management. However, there has been a ing such strategies is important due to interviews and pre-employment test-
lack of empirical evidence that can be the grave damage these individuals can ing14 may help avoid hiring high-risk
used to guide the understanding of spe- do through their access to information individuals. The costs of bad hires sig-
cific concerning personality types. The assets and systems. Our data show that nificantly exceed the potential benefits
empirical evidence provided in this study most employees are not inclined toward they may bring an organization, even if
should be useful for refining existing cy- malevolent personality traits or mali- these individuals are “superstars.”21
bersecurity resources related to insider cious behaviors. However, as Figure 1 il- Hiring managers should follow the
threats and refine indicator develop- lustrates, malevolent people do exist, and medical community’s credo, primum
ment in technical monitoring systems. those individuals are significantly more non nocere, first do no harm,21 and ac-
There is considerable good in orga- likely to intend to use information tech- tively avoid hiring high-risk insiders.
nizations; most individuals are hard- nology systems to do harm. These indi- Even the most stringent hiring prac-
working and ethical. However, our re- viduals are dangerous—managers tices may not prevent hiring malicious
search shows that individuals who should be aware of the dangers they pose. individuals. Because of this, a well-de-
tend toward the dark triad traits are Both administrative and technical signed risk management plan should as-
more likely to intend to engage in techniques exist for mitigating the po- sume that high-risk insiders exist in the
harmful behaviors. Hence, managers tential harm from insiders. Although organization and should seek to identify
should be aware of the traits and the as- the following recommendations focus such individuals. However, it is important
sociated tendencies toward harmful on administrative controls, technical that such practices not be illegally dis-
behaviors. Further, managers should controls are equally important. Our re- criminatory. Balancing the line between
have some understanding of how to search pertains to both administrative prudence and practices that may be seen
deal with the threats brought about by and technical controls. The results can as discriminatory requires careful
insiders who exhibit dark triad traits. provide guidance for developing and thought and planning, along with the in-
refining policy, training and awareness volvement of human resource and legal
Recommendations programs, and technical monitoring specialists.4 This can be accomplished
Overall, managers can be thankful for the systems. All of these are important ele- by implementing issue-specific adminis-
good in their organizations and for their ments of a well-designed cybersecurity trative-preventative security policies as
hard working, principled employees. risk management system. part of an overall risk mitigation plan.
However, some individuals will tend to- Educating managers on the dark tri- Many organizations have formal pro-
ward malevolent personality traits. Be- ad traits may help them identify high- cesses in place for insider threat cases,
cause of this, developing and employing risk individuals, which is an important and if staff is well trained, behavioral
risk management strategies directed at administrative mitigation technique. patterns associated with threats result-
mitigating the potential harm these indi- Strong hiring practices that include ing from the personality traits can be

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 69

contributed articles

recognized sooner, with formal risk- paths—the risk associated with the de- 7. Greitzer, F.L., Frincke, D.A. and Zabriskie, M. Social/
ethical issues in predictive insider threat monitoring.
management processes triggered.4 terrent may actually be a motivator. It is Information Assurance and Security Ethics in Complex
Sound leadership practices may likely that what management perceives Systems: Interdisciplinary Perspectives. Information
Science Reference, 2010, 132–161.
help mitigate the risk from malevolent as deterrents may not be viewed as such 8. Greitzer, F.L., Purl, J., Becker, D.E. (Sunny), Stitcha, P.J.
insiders. For example, psychopathic or by malevolent individuals. Another im- and Leong, Y.M. Modeling expert judgments of insider
threat using ontology structure: Effects of individual
Machiavellian individuals who per- portant goal for future research is to indicator threat value and class membership. In
ceive that some people (such as man- gain a better understanding of what mo- Proceedings of the 52nd Hawaii Intern. Conf. System
Sciences (Maui, HI, USA, 2019), 3202–3211.
agers) are allowed to break rules that tivates individuals high on the dark tri- 9. Greitzer, F.L., Purl, J., Leong, Y.M. and Sticha, P.J.
others must follow may seek revenge ad traits to engage in harmful behav- Positioning your organization to respond to insider
threats. IEEE Engineering Management Review 47, 2
through the misuse of information as- iors. Are there particular events that (Jun. 2019), 75–83.
10. Harrison, A., Summers, J. and Mennecke, B. The
sets. Broken promises (real or per- lead dark triad individuals to engage in effects of the dark triad on unethical behavior. J.
ceived) may also trigger adverse behav- malevolent behaviors? A related ques- Business Ethics 153, 1 (Nov. 2018), 53–77.
11. Jones, D.N. and Paulhus, D.L. Introducing the short
ior in those high in dark triad traits. tion concerns how the dark triad traits dark triad (SD3): A brief measure of dark personality
For example, such individuals may be influence cybersecurity decision mak- traits. Assessment 21, 1 (2014), 28–41.
12. Kiser, A.I.T., Porter, T. and Vequist, D. Employee monitoring
motivated to do harm when a prom- ing. Finally, researchers should seek to and ethics: Can they co-exist? Intern. J. Digital Literacy
ised bonus, raise, or promotion does understand the extent to which domi- and Digital Competence 1, 4 (Oct. 2010), 30–45.
13. Liang, N., Biros, D.P. and Luse, A. An Empirical Validation
not come to pass. The motivation may nant behavioral security theories apply of Malicious Insider Characteristics. J. Management
be even stronger when the promised to those exhibiting the dark triad traits, Information Systems 33, 2 (Apr. 2016), 361–392.
14. Montealegre, R. and Cascio, W.F. Technology-driven
benefit is given to someone else. particularly in the context of malicious changes in work and employment. Commun. ACM 60,
We would be remiss if we did not insider cyber security threats. 12 (Nov. 2017), 60–67.
15. Paulhus, D.L. and Williams, K.M. The dark triad of
mention the clear ethical dilemma personality: Narcissism, Machiavellianism, and
posed by our recommendations. Man- Conclusion psychopathy. J. Research in personality 36, 6 (2002),
556– 563.
agers face a trade-off between their re- This study establishes the relationships 16. Sanders, G.L., Upadhyaya, S. and Wang, X. Inside the
sponsibility to protect the organization between dark triad personality traits Insider. IEEE Engineering Management Review. 47, 2
(Jun. 2019), 84–91.
and the rights of current and potential and intentions to commit insider cyber 17. Schultz, E.E. A Framework for understanding and
predicting insider attacks. Computers & Security 21, 6
employees. This trade-off is similar to sabotage. We found strong, significant (2002), 526–531.
that presented in employee monitoring, relationships between the dark triad 18. Shaw, E. and Sellers, L. Application of the critical-path
method to evaluate insider risks. Internal Security and
which requires balancing organization- traits and such intentions. Dark triad Counterintelligence 59, 2 (2015), 1–8.
al risk and employee privacy.12 Hence, individuals have a propensity toward 19. Shaw, E.D., Post, J.M. and Ruby, K.G. Inside the mind of the
insider. Security Management 43, 12 (Dec. 1999), 34–44.
we consider the use of personality traits malicious insider threat acts. Although 20. Software Engineering Institute. The CERT Insider
as elements of screening or monitoring it is ideal to avoid hiring individuals ex- Threat Center. Common Sense Guide to Mitigating
Insider Threats, Fifth Edition. Technical Report #CMU/
efforts because assessment devices hibiting high risk traits, the difficulty in SEI-2015-TR-010. SEI, Carnegie Mellon University.
have been designed for use in normal identifying such individuals during the 21. Torres, N. It’s better to avoid a toxic employee than
hire a superstar. Harvard Business Review, 2016.
populations and there is no reason to hiring process means that they likely 22. Veselka, L., Schermer, J.A. and Vernon, P.A. The dark
believe the individuals are clinically im- exist in most organizations. When it triad and an expanded framework of personality.
Personality and Individual Differences 53, 4 (Sep.
paired regarding critical functions of comes to malicious insider threat risk 2012), 417– 425.
the job for which they were hired.25 mitigation, a false positive is much bet- 23. Wilder, D.U.M. The psychology of espionage and leaking
in the digital age. Studies in Intelligence 61, 2 (2017), 1–36.
Limitations and directions for future ter than a false negative. 24. Willison, R. and Warkentin, M. 2013. Beyond
research. This study has limitations deterrence: An expanded view of employee computer
abuse. MIS Q. 37, 1 (2013), 1–20.
that provide future research opportu- Acknowledgments
25. Wu, J. and Lebreton, J.M. Reconsidering the
Financial support was provided by the Dr. Jan Clark dispositional basis of counterproductive work behavior:
nities. First, the study confirmed a Memorial Research Fund and the Herbert McElveen The role of aberrant personality. Personnel Psychology
relationship but was limited in the Professorship made available through the state of Louisiana 64, 3 (Sep. 2011), 593–626.
Board of Regents Support Funds and Mary Nell Condren.
ability to establish full causality. Due
to limitations with the temporal pre- References
Michele Maasberg ([email protected]) is a
Cyber Security Scientist at the Johns Hopkins University
cedence criterion, only association 1. Band, S.R., Cappelli, D.M., Fischer, L.F., Moore, A.P., Applied Physics Laboratory, Laurel, MD, USA; https://orcid.
Shaw, E.D. and Trzeciak, R.F. Comparing Insider IT
can be concluded. Future research Sabotage and Espionage: A Model-Based Analysis.
org/0000-0003-4306-0559.

using laboratory experimental or lon- Technical Report #CMU/SEI-2006-TR-026. Carnegie Craig Van Slyke ([email protected]) is the Mike
Mellon University Software Engineering Institute McCallister Eminent Scholar Chair in Information
gitudinal methods are called for. Sec- Pittsburgh, PA. Systems at Louisiana Tech University, Ruston, LA, USA;
ond, the generalizability of our find- 2. Barrick, M.R. and Mount, M.K. The big five personality https://orcid.org/0000-0003-3924-1859.
dimensions and job performance: A meta-analysis.
ings is limited by our use of a specific Personnel Psychology 44, 1 (1991), 1–26. Selwyn Ellis ([email protected]) is Balsley-Whitmore
scenario that included a particular 3. Bradfield, M. and Aquino, K. 1999. The effects of blame Endowed Professor, an associate professor, department
attributions and offender likableness on forgiveness head, and Interim Associate Dean of Graduate Programs
trigger and threat. Future research and revenge in the workplace. J. Management 25, 5 at Louisiana Tech University, Ruston, LA, USA;
should consider a range of scenarios (1999), 607–631. https://orcid.org/0000-0002-2816-8441.
4. Cappelli, D. An unaddressed threat to critical
that include additional triggering infrastructure and national security: Insider cyber Nicole Beebe ([email protected]) is Department
sabotage. 2018; https://bit.ly/2CpdphW. Chair of Information Systems and Cyber Security
events and threat responses. and Melvin Lachman Distinguished Professor in
5. Clark, J.W. Threat from within: Case studies of
One particularly important open insiders who committed information technology Entrepreneurship Director of the Cyber Center for Security
sabotage. In Proceedings of the 11th Intern. Conf. and Analytics at the University of Texas at San Antonio,
question concerns the effectiveness of Availability, Reliability and Security (Salzburg, TX, USA; https://orcid.org/0000-0002-0151-1617.
commonly used deterrents. Some com- Austria, Aug. 2016), 414–422.
6. CNBC. Elon Musk emails employees about “extensive
mon deterrents such as sanctions may and damaging sabotage” by employee. 2018; https://
not only be ineffective for psycho- cnb.cx/2YnYgGr. © 2020 ACM 0001-0782/20/12

70 COMMUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

Publish Your Work Open Access With ACM!

ACM offers a variety of Open Access publishing options

to ensure that your work is disseminated to the widest possible
readership of computer scientists around the world.

Please visit ACM’s website to learn more about

ACM’s innovative approach to Open Access at:
https://www.acm.org/openaccess
review articles
DOI:10.1145/ 3372135

Speed testing methods have flourished over

the last decade, but none without at least
some limitations.
BY NICK FEAMSTER AND JASON LIVINGOOD

Measuring
Internet Speed
Current Challenges
and Future
Recommendations

VARIOU S GOVER NME NTAL O RG ANI Z AT IO N Shave begun

to rely on so-called Internet speed tests to measure
broadband Internet speed. Examples of these
programs include the Federal Communications
Commission’s “Measuring Broadband America”
program,7 California’s CALSPEED program,4 the U.K.’s
Home Broadband Performance Program,23 and various
on outmoded technology, making
other initiatives in states including Minnesota,18 the resulting data unreliable or
New York,19–21 and Pennsylvania.27 These programs misleading. This article describes
have various goals, ranging from assessing whether the current state of speed testing
tools, outlines their limitations, and
ISPs are delivering on advertised speeds to assessing explores paths forward to better in-
potentially underserved rural areas that could benefit form the various technical and policy
ambitions and outcomes.
from broadband infrastructure investments. Some current speed test tools were
The accuracy of measurement is critical to these well-suited to measuring access link
assessments, as measurements can inform everything capacity a decade ago but are no lon-
ger useful because they made a design
from investment decisions to policy actions and even assumption that the Internet Service
litigation. Unfortunately, these efforts sometimes rely Provider (ISP) last mile access net-

72 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 work was the most constrained (bot- home Wi-Fi network, network inter- Web browser opens multiple connec-
tleneck) link. This is no longer a good connections, speed testing infra- tions in parallel between an end user
assumption, due to the significant in- structure, and other areas. and the server to increasingly local-
creases in Internet access speeds due A wide range of factors can influ- ized content delivery networks
to new technologies. Ten years ago, a ence the results of an Internet speed (CDNs), reflecting an evolution of ap-
typical ISP in the United States may test, including: user-related consid- plications that ultimately effects the
have delivered tens of megabits per erations, such as the age of the de- user experience.
second (Mbps). Today, it is common vice; wide-area network consider- These developments suggest the
IMAGE BY YUGANOV KONSTA NT IN

to have ten times faster (hundreds of ations, such as interconnect capacity; need to evolve our understanding of the
megabits per second), and gigabit test-infrastructure considerations, utility of existing Internet speed test
speeds are available to tens of mil- such as test server capacity; and test tools and consider how these tools may
lions of homes. The performance bot- design, such as whether the test runs need to be redesigned to present a more
tleneck has often shifted from the ISP while the user’s access link is other- representative measure of a user’s Inter-
access network to a user’s device, wise in use. Additionally, the typical net experience.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 73

review articles

Background stant; it changes from minute to min- packet loss as the feedback signal to
In this section, we discuss and define ute based on many factors, including determine the best transmission rate.
key network performance metrics, in- what other users are doing on the In- Many applications such as video
troduce the general principles of Inter- ternet. Many network-performance streaming are designed to adapt well to
net “speed tests” and explore the basic tests, such as the FCC test7 and Ook- packet loss without noticeably affect-
challenges facing any speed test. la’s speed test, include additional ing the end user experience, so there is
Performance metrics. When people metrics that reflect the user’s quality no single level of packet loss that auto-
talk about Internet “speed,” they are of experience. matically translates to poor application
generally talking about throughput. Latency is the time it takes for a single performance. Additionally, certain net-
End-to-end Internet performance is data packet to travel to a destination. work design choices, such as increas-
typically measured with a collection of Typically, latency is measured in terms ing buffer sizes, can reduce packet loss,
metrics—specifically throughput (that of roundtrip latency, since measuring but at the expense of latency, leading to
is, “speed”), latency, and packet loss. one-way latency would require tight a condition known as “buffer bloat.”3,12
Figure 1 shows an example speed test time synchronization and the ability to Speed test principles and best
from a mobile phone on a home Wi-Fi instrument both sides of the Internet practices. Active measurement. Today’s
network. It shows the results of a “na- path. Latency generally increases with speed tests are generally referred to as ac-
tive” speed test from the Ookla An- distance, due to factors such as the tive measurement tests, meaning that
droid speed test application24 run in speed of light for optical network seg- they attempt to measure network perfor-
New Jersey, a canonical Internet speed ments; other factors can influence la- mance by introducing new traffic into the
test. This native application reports tency, including the amount of queue- network (so-called “probe traffic”). This is
the user’s ISP, the location of the test ing or buffering along an end-to-end in contrast to passive tests, which ob-
server destination, and the following path, as well as the actual network path serve traffic passing over a network inter-
performance metrics: that traffic takes from one endpoint to face to infer performance metrics. For
Throughput is the amount of data another. TCP throughput is inversely speed testing, active measurement is the
that can be transferred between two proportional to end-to-end latency;31 all recognized best practice, but passive
network endpoints over a given time things being equal, then, a client will measurement can be used to gauge oth-
interval. For example, throughput can see a higher throughput to a nearby er performance factors, such as latency,
be measured between two points in a server than it will to a distant one. packet loss, video quality, and so on.
given ISP’s network, or it can be mea- Jitter is the variation between two la- Measuring the bottleneck link. A typi-
sured for an end-to-end path, such as tency measurements. Large jitter mea- cal speed test sends traffic that tra-
between a client device and a server at surements are problematic. verses many network links, including
some other place on the Internet. Typ- Packet loss rate is typically computed the Wi-Fi link inside the user’s home
ically, a speed test measures both as the number of lost packets divided network, the link from the ISP device
downstream (download), from server by the number of packets transmitted. in the home to the ISP network, and
to client, and upstream (upload), from Although high packet loss rates gener- the many network level hops between
client to server (Bauer et al.2) offer an ally correspond to worse performance, the ISP and the speed test server,
in-depth discussion of throughput some amount of packet loss is normal which is often hosted on a network
metrics). Throughput is not a con- because a TCP sender typically uses other than the access ISP. The
throughput measurement that results
Figure 1. Example metrics from an Ookla Speedtest, a canonical Internet speed test. from such a test in fact reflects the ca-
pacity of the most constrained link,
sometimes referred to as the “bottle-
Upload Throughput neck” link—the link along the end-to-
Download Throughput
end path that is the limiting factor in
Round-Trip Latency Packet Loss end-to-end throughput. If a user has a
1Gbps connection to the Internet but
Jitter
their home Wi-Fi network is limited to
200Mbps, then any speed test from a
device on the Wi-Fi network to the In-
ternet will not exceed 200Mbps. Bot-
tlenecks can exist in an ISP access net-
work, in a transit network between a
client and server, in the server or serv-
Server Endpoint
er data-center network, or other plac-
es. In many cases, the bottleneck is
User’s Internet
Service Provider
located somewhere along the end-to-
end path that is not under the ISP’s or
user’s direct control.
Use of transmission control protocol.
Speed tests typically use the Transmis-

74 COM MUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 review articles

Figure 2. TCP Dynamics. Figure 3. Successive runs of different throughput tests.

Additive Increase
Multiplicative Decrease
(AIMD)
(Window Size)
Sending Rate

Slow Start

Time

sion Control Protocol (TCP) to measure

throughput. In keeping with the nature (a) (b)
of most Internet application transfers Five successive runs Internet Health Test runs in succession to six different
of Ookla Speedtest yield servers. The test measures consistently lower throughput
today—including, most notably, Web variable results on and also shows variability, both to different servers
browsers—most speed tests use multi- downstream throughput. and across successive test runs.
ple parallel TCP connections. Under-
standing TCP’s operation is critical to
the design of an accurate speed test. der-reports throughput, especially at devices that operate on the same spec-
Any TCP-based speed test should be: higher speeds. trum (for example, microwave ovens,
long enough to measure steady-state baby monitors, security cameras).
transfer; recognize that TCP transmis- Limitations of Existing Speed Tests Many past experiments demon-
sion rates naturally vary over time; and, Existing speed tests have a number of strate that the user’s Wi-Fi—not the
use multiple TCP connections. Figure limitations that have become more ISP—is often the network perfor-
2 shows TCP’s dynamics, including the acute in recent years, largely as a result mance bottleneck. Sundaresan et al.
initial slow start phase. During TCP of faster ISP access links and the pro- found that whenever downstream
slow start, the transmission rate is far liferation of home wireless networks. throughput exceeded 25Mbps, the us-
lower than the network capacity. In- The most profound change is that as er’s home wireless network was al-
cluding this period as part of a through- network access links have become faster, most always the bottleneck.30 Al-
put calculation will result in a through- the network bottleneck has moved from though the study is from 2013, and
put measurement that is less than the the ISP access link to elsewhere on the both access link speeds and wireless
actual available network capacity. If network. A decade ago, the network bot- network speeds have since increased,
test duration is too short, the test will tleneck was commonly the access ISP the general trend of home wireless
tend to underestimate throughput. As link; with faster ISP access links, the bottlenecks is still prevalent.
a result, accurate speed test tools must network bottleneck may have moved Client hardware and software. Client
account for TCP slow start. Additional- any number of places, from the home types range from dedicated hardware,
ly, instantaneous TCP throughput con- wireless network to the user’s device it- to software embedded in a device on the
tinually varies because the sender tries self. Other design factors may also play user’s network, to native software made
to increase its transfer rate in an at- a role, including how measurement for a particular user operating system,
tempt to find and use any spare capac- samples are taken and the provision- and Web browsers. Client type has an
ity (a process known as “additive in- ing of the test infrastructure itself. important influence on the test results,
crease multiplicative decrease” or User-related consideration. The home because some may be inherently limit-
AIMD). wireless network. Speed tests that are ed or confounded by user factors. Dedi-
Inherent variability. A speed test run over a home wireless connection cated hardware examples include the
measurement can produce highly vari- often reflect a measurement of the us- SamKnows whitebox and RIPE Atlas
able results. Figure 3 shows an illustra- er’s home wireless connection, not that probe. Embedded software refers to ex-
tive example of typical variability that a of the access ISP, because the Wi-Fi amples where the software is integrated
speed test might yield, both for Inter- network itself is usually the lowest ca- into an existing network device such as
net Health Test (IHT) and Ookla Speed- pacity link between the user and test cable modem, home gateway device, or
test. These measurements were per- server.1,5,16,26,28,30 Many factors affect the Wi-Fi access point. A native application
formed successively on the same performance of the user’s home wire- is software made specifically to run on a
Comcast connection provisioned for less network, including: distance to the given operating system such as Android,
200Mbps downstream and 10Mbps up- Wi-Fi Access Point (AP) and Wi-Fi sig- iOS, Windows, and Mac OS. Finally,
stream throughput. The tests were per- nal strength, technical limitation of a Web-based tests simply run from a Web
formed in succession. Notably, succes- wireless device and/or AP, other users browser. In general, dedicated hard-
sive tests yield different measurements. and devices operating on the same net- ware and embedded software approach-
IHT, a Web front-end to a tool called work, interference from nearby APs us- es tend to be able to minimize the effect
the Network Diagnostic Test (NDT), ing the same spectrum, and interfer- of user-related factors and are more ac-
also consistently and significantly un- ence from non-Wi-Fi household curate as a result.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 75

review articles

Many users continue to use older connection, their speed tests will never 2012–2015. This shows that any user
wireless devices in their homes (for ex- exceed 100Mbps and that test result with an iPhone 5s or older is unlikely to
ample, old iPads and home routers) cannot be said to represent a capacity is- reach 100Mbps, likely due to the lack of
that do not support higher speeds. Fac- sue in the ISP network; it is a device lim- a newer 802.11ac wireless interface.
tors such as memory, CPU, operating itation. As a result, many ISPs document Router-based testing vs. device-based
system, and network interface card recommended hardware and software testing. Figure 5 shows an example of
(NIC) can significantly affect through- standards,33 especially for 1Gbps con- two successive speed tests. Figure 5a
put measurements. For example, if a nections. The limitations of client hard- uses software embedded in the user’s
user has a 100Mbps Ethernet card in ware can be more subtle. Figure 4 shows router, so that no other effects of the lo-
their PC connected to a 1Gbps Internet an example using iPhone released in cal network could interfere. Figure 5b
shows the same speed test (such as,
Figure 4. Distribution of download speeds across different device types. Older devices do Ookla Speedtest), on the same net-
not support 802.11ac, so fail to consistently hit 100Mbps.
work, performed immediately follow-
ing the router-based test using native
Download Speed by Device (Aug. 1, 2015–Jan. 31, 2016) software on a mobile device over Wi-Fi.
The throughput reported from the us-
340
er’s mobile device on the home net-
320
work is almost half of the throughput
300
280
that is reported when the speed test is
260 taken directly from the router.
Download Speed (Mbps)

240 Competing “cross traffic.” At any

220 given time, a single network link is si-
200 multaneously carrying traffic from
180 many senders and receivers. Thus, any
160 single network transfer must share the
140
available capacity with the competing
120
traffic from other senders—so-called
100
80
cross traffic. Although sharing capacity
60 is natural for normal application traf-
40 fic, a speed test that shares the avail-
20 able capacity with competing cross
0 traffic will naturally underestimate the
-5 -5
s -6 -6
s 5 5s - 6 6s -5 5s -6 6s -5 5s - 6 6s
50 50 0- 0 - 100 0 - 200 0 - 200 0 - 300 0 - 300 0- total available network capacity. Cli-
50 50 10 10 10 20 20 30 30

Device (Tier – Model) ent-based speed tests cannot account

for cross traffic; because the client can-
not see the volume of other traffic on
the same network, whereas a test that
Figure 5. Two forms of Ookla Speedtest: A router-based test and native desktop test from runs on the user’s home router can ac-
the same home network.
count for cross traffic when conduct-
ing throughput measurements.
Wide-area network considerations.
Impaired ISP access network links. An
ISP’s “last mile” access network links
can become impaired. For example,
the quality of a DOCSIS connection to a
home can become impaired by factors
such as a squirrel chewing through a
line or a bad ground wire. Similarly,
fixed wireless connections can be im-
paired by weather or leaves blocking
the antenna. To mitigate the potential
for an individual impairment unduly
influencing ISP-wide results, tests
should be conducted with a large num-
ber of users.
Access ISP capacity. Capacity con-
straints within an ISP’s network can ex-
(a) (b) ist, whether in the access network, re-
Ookla router-based test. Ookla native desktop test.
gional network (metropolitan area), or
backbone network. Regional and back-

76 COMM UNICATIO NS O F THE AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 review articles

bone networks usually have excess ca- research has focused on HTTP-based
pacity so the only periods when they video streaming de-prioritization and
may be constrained would be the re- rate-limiting. Such rate limiting could
sult of a disaster (for example, hurri- exist at any point on the network path,
cane damage) or temporary condi-
tions such fiber cuts or BGP hijacking. Many past though most commonly it may be ex-
pected in an access network or on the
Usually ISP capacity constraints arise
in the last-mile access networks,
experiments destination server network. In the lat-
ter case, virtual servers or other hosted
which are by nature shared in the first demonstrate services may be priced by peak bitrate
mile or first network element, (for ex-
ample, passive optical networking
that the user’s and therefore a hard-set limit on total
peak bitrate or per-user-flow bitrate
(PON), DOCSIS, DSL, 4G/5G, Wi-Fi, Wi-Fi—not the ISP— may exist. Web software such as Nginx
point-to-point wireless).
Transit and interconnect capacity.
is often has features for configuring rate limit-
ing,22 as cloud-based services may
Another significant consideration is the network charge by total network usage or peak
the connection to “transit” and “mid-
dle mile” networks. The interconnects performance usage; for example, Oracle charges for
total bandwidth usage,25 and FTP ser-
between independently operated net- bottleneck. vices often enforce per-user and per-
works may also introduce throughput flow rate limits.11
bottlenecks. As user speeds reach Rate-boosting. Rate-boosting is the
1Gbps, ensuring that there are no ca- opposite of rate limiting; it can enable
pacity constraints on the path between a user to temporarily exceed their nor-
the user and test server— especially mal provisioned rate for a limited peri-
across transit networks—is a major od. For example, a user may have a
consideration. In one incident in 2013, 100Mbps plan but may be allowed to
a bottleneck in the Cogent transit net- burst to 250Mbps for limited periods if
work reduced NDT throughput mea- spare capacity exists. This effect was
surements by as much as 90%. Test re- noted in the FCC’s first MBA report in
sults improved when Cogent began 2011 and led to use of a longer duration
prioritizing NDT test traffic over other test to measure “sustained” speeds.8
traffic. Transit-related issues have often Such rate-boosting techniques appear
affected speed tests. In the case of the to have fallen out of favor, perhaps
FCC’s MBA platform, this prompted partly due greater access speeds or the
them to add servers on the Level 3 net- introduction of new technologies such
work to isolate the issues experienced as DOCSIS channel bonding.
with M-Lab’s infrastructure and the Co- Test infrastructure considerations.
gent network, and M-Labs has also add- Because speed tests based on active
ed additional transit networks to re- measurements rely on performing
duce their reliance on one network. measurements to some Internet end-
Middleboxes. End-to-end paths of- point (that is, a measurement server),
ten have devices along the path, called another possible source of a perfor-
“middleboxes,” which can affect per- mance bottleneck is the server infra-
formance. For example, a middlebox structure itself.
may perform load balancing or security Test infrastructure provisioning. The
functions (for example, malware detec- test server infrastructure must be ade-
tion, firewalls). As access speeds in- quately provisioned so that it does not
crease, the capacity of middleboxes become the bottleneck for the speed
may increasingly be a constraint, which tests. In the past, test servers have been
will mean test results will reflect the ca- overloaded, misconfigured, or other-
pacity of those middleboxes rather wise not performing as necessary, as
than the access link or other measure- has been the case periodically with M-
ment target. Lab servers used for both FCC MBA
Rate-limiting. Application-layer or testing and NDT measurements. Simi-
destination-based rate limiting, often larly, the data center switches or other
referred to as throttling, can also network equipment to which the serv-
cause the performance that users ex- ers connect may be experiencing tech-
perience to diverge from conventional nical problems or be subject to other
speed tests. Choffnes et al. have devel- performance limitations. In the case of
oped Wehe, which detects applica- the FCC MBA reports, at one point this
tion-layer rate limiting;32 thus far, the resulted in discarding of data collected

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 77

review articles

Figure 6. IHT and Ookla geolocation. work paths between two endpoints can
be circuitous, and other factors such as
network congestion on a path can af-
fect latency. Some speed tests mitigate
these effects with additional tech-
niques. For example, Ookla’s Speed-
test uses IP geolocation to select an
initial set of servers that are likely to be
close, and then the client selects from
that list the one with the lowest RTT
(other factors may also play into selec-
tion, such as server network capacity).
Unfortunately, Internet Health Test
(which uses NDT) and others rely
strictly on IP geolocation.
(a) (b)
Internet Health Test mistakenly Ookla Speedtest directing a client Figure 6 shows stark differences in
locating a client in Princeton, N.J., in Princeton, N.J., to an on-net Speedtest server server selection between two tests: In-
to Philadelphia, P.A. (50+ miles away), in Plainfield, N.J. Ookla also allows a user ternet Health Test (which relies on IP
and performing a speed test to a server to select another nearby server.
in New York City.
geolocation and has a smaller selec-
tion of servers); and Ookla Speedtest
(which uses a combination of IP geo-
location, GPS-based location from
from M-Lab servers due to severe im- hand, uses a fixed, dedicated set of mobile devices, and RTT-based server
pairments.6,9 The connection between servers as part of a closed system and selection to a much larger selection of
a given datacenter and the Internet infrastructure. For many years, these servers). Notably, the Internet Health
may also be constrained, congested, or servers have been: constrained by a Test not only mis-locates the client
otherwise technically impaired, as was 1Gbps uplink; shared with other mea- (determining that a client in Prince-
the case when some M-Lab servers surement experiments (recently, ton, New Jersey is in Philadelphia),
were single-homed to a congested Co- Measurement Lab has begun to up- but it also selects a server that is in
gent network. Finally, the servers grade to 10Gbps uplinks). Both of New York City, which is more than 50
themselves may be limited in their ca- these factors can and did contribute miles from Princeton. In contrast, the
pacity: if, for example, a server has a to the platform introducing its own Ookla test, which selects an on-net-
1Gbps Ethernet connection (with real- set of performance bottlenecks. work Comcast server in Plainfield, NJ,
world throughput below 1Gbps) then Server placement and selection. A which is merely 21 miles away, and
the server cannot be expected to mea- speed test estimates the available ca- also gives the user the option of using
sure several simultaneous 1Gnps or pacity of the network between the cli- closer servers through the “Change
2Gbps tests. Many other infrastruc- ent and the server. Therefore, the Server” option.
ture-related factors can affect a speed throughput of the test will naturally de- Test design cConsiderations. Num-
test, including server storage input and pend on the distance between these ber of parallel connections. A signifi-
output limits, available memory and endpoints as measured by a packet’s cant consideration in the design of a
CPU, and so on. Designing and operat- round trip time (RTT). This is extreme- speed test is the number of parallel
ing a high scale, reliable, high-perfor- ly important, because TCP throughput TCP connections that the test uses to
mance measurement platform is a dif- is inversely proportional to the RTT be- transfer data between the client and
ficult task, and as more consumers tween the two endpoints. For this rea- server, since the goal of a speed test is
adopt 1Gbps services this may become son, speed test clients commonly at- to send as much data as possible and
even more challenging.17 tempt to find the “closest” throughput this is usually only possible with mul-
Different speed test infrastruc- measurement server to provide the tiple TCP connections. Using multiple
tures have different means for incor- most accurate test result and why many connections in parallel allows a TCP
porating measurement servers into speed tests such as Ookla’s, use thou- sender to more quickly and more reli-
their infrastructure. Ookla allows vol- sands of servers distributed around the ably achieve the available link capaci-
unteers to run servers on their own world. to select the closest server, some ty. In addition to achieving a higher
and contribute these servers to the tests use a process called “IP geoloca- share of the available capacity (be-
list of possible servers that users can tion,” whereby a client location is de- cause the throughput test is effectively
perform tests against. Ookla uses em- termined from its IP address. Unfortu- sharing the link with itself), a transfer
pirical measurements over time to nately, IP geolocation databases are using multiple connections is more
track the performance of individual notoriously inaccurate, and client loca- resistant to network disruptions that
servers. Those that perform poorly tion can often be off by thousands of may result in the sender re-entering
over time are removed from the set of miles. Additionally, latency resulting TCP slow start after a timeout due to
candidate servers that a client can from network distance typically ex- lost packets.
use. Measurement Lab, on the other ceeds geographic distance, since net- A single TCP connection cannot

78 COMM UNICATIO NS O F THE AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 review articles

typically achieve a throughput ap- sults in a TCP timeout, and subsequent Infrequent testing. If tests are too
proaching full link capacity, for two re-entry into slow start infrequent or are only taken at cer-
reasons: a single connection takes lon- Estimating the throughput of the tain times of day, the resulting mea-
ger to send at higher rates because link is not as simple as dividing the surements may not accurately reflect
TCP slow start takes longer to reach amount of data transferred by the total a user’s Internet capacity. An analo-
link capacity, and a single connection time elapsed over the course of the gy would be looking out a window
is more susceptible to temporarily transfer. A more accurate estimate of once per day in the evening, seeing it
slowing down transmission rates the transfer rate would instead mea- was dark outside, and concluding
when it experiences packet loss (a sure the transfer during steady-state that it must be dark 24 hours a day.
common occurrence on an Internet AIMD, excluding the initial slow start Additionally, if the user only con-
path). Past research concluded that a period. Many standard throughput ducts a test when there is a transient
speed test should have at least four tests, including the FCC/SamKnows problem, the resulting measure-
parallel connections to accurately test, omit the initial slow start period. ment may not be representative of
measure throughput.29 For the same The Ookla test implicitly omits this pe- the performance that a user typically
reason, modern Web browsers typical- riod by discarding low-throughput experiences. Automatic tests run
ly open as many as six parallel connec- samples from its average measure- multiple times per day at randomly
tions to a single server in order to max- ment. Tests that include this period selected times during peak and off-
imize use of available network capacity will result in a lower value of average peak times can account for some of
between the client and Web server. throughput than the link capacity can these factors.
Test duration. The length of a test support in steady state.
and the amount of data transferred Self-selection bias. Speed tests that The Future of Speed Testing
also significantly affect test results. are initiated by a user suffer from Speed testing tools will need to evolve
As described previously, a TCP send- self-selection bias:14 many users initi- as end user connections approach and
er does not immediately begin send- ate such tests only when they are ex- exceed 1Gbps, especially given that so
ing traffic at full capacity but instead periencing a technical problem or are many policy, regulatory, and invest-
begins in TCP slow start until the reconfiguring their network. For ex- ment decisions are based on speed
sending rate reaches a pre-config- ample, when configuring a home measurements. As access network
ured threshold value, at which point wireless network, a user may run a speeds increase and the performance
it begins AIMD congestion avoid- test over Wi-Fi, then reposition their bottlenecks move elsewhere on the
ance. As a result, if a transfer is too Wi-Fi AP and run the test again. These path, speed test design must evolve
short, a TCP sender will spend a sig- measurements may help the user op- to keep pace with both faster network
nificant fraction of the total transfer timize the placement of the wireless technology and evolving user expecta-
in TCP slow start, ensuring the trans- access point but, by design, they re- tions. We recommend the following:
fer rate will fall far short of available flect the performance of the user’s Retire outmoded tools such as NDT.
capacity. As access speeds increase, home wireless network, not that of NDT, also known as the Internet Health
most test tools have also needed to the ISP. Tests that are user-initiated Test,15 may appear at first glance to be
increase test duration. (“crowdsourced”) are more likely to suitable for speed tests. This is not the
Throughput calculation. The method suffer from self-selection bias. It can case, though it continues to be used for
that tests use to calculate results ap- be difficult to use these results to speed measurement despite its unsuit-
pears to vary widely; often this method draw conclusions about an ISP, geo- ability and proven inaccuracy.10 Its in-
is not disclosed. Tests may discard graphic region, and so forth. adequacy for measuring access link
some high and/or low results, may use
the median or the mean, may take only Figure 7. Throughput vs. number of TCP threads.
the highest result and discard the rest,
and so on. This makes different tests
Single-threaded TCP never achieves
difficult to compare. Finally, some more than 80% capacity,
tests may include all of the many phas- even for speeds of 10 Mbps.

es of a TCP transfer, even though some 1.0

of those phases are necessarily at rates Single-threaded HTTP UDP Capacity
below the capacity of a link: 0.8 Passive Throughput Multi-threaded HTTP
˲ the slow start phase at the begin-
0.6
ning of a transfer (which occurs in ev-
CDF

ery TCP connection); 0.4

˲ the initial “additive increase”
0.2
phase of the TCP transfer when the
sender is actively increasing its send- 0.0
0.0 0.2 0.4 0.6 0.8 1.0
ing rate but before it experiences the
Normalized Throughput
first packet loss that results in multipli-
cative decrease;
˲ any packet loss episode which re-

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 79

review articles

speeds has been well-documented.2 tion (for example, streaming video) 12. Gettys, J. Bufferbloat: Dark Buffers in the Internet.
In IEEE Internet Computing, 2011.
One significant problem is that NDT under the available network condi- 13. Google Group. MLab speed test is incorrect? 2018;
still uses a single TCP connection, tions. As previously mentioned, even https://groups.google.com/a/measurementlab.net/
forum/#!topic/discuss/vOTs3rcbp38.
nearly two decades after this was the most demanding streaming video 14. Heckman, J. J. Selection bias and self-selection. In
shown to be inadequate for measuring applications require only tens of Econometrics, pages 201–224. 1990
15. Internet Health Test, 2019. http://internethealthtest.
link capacity. NDT is also incapable of megabits per second, yet user experi- org/.
reliably measuring access link through- ence can still suffer as a result of ap- 16. Lifewire. Change the Wi-Fi channel number to
avoid interference, 2018; https://www.lifewire.
put for speeds of 100Mbps or more, as plication performance glitches, such com/Wi-Fi-channel-numberchange-to-avoid-
we enter an era of gigabit speeds. The as changes in resolution or rebuffer- interference-818208.
17. Livingood, J. Measurement Challenges in the Gigabit
test also includes the initial TCP slow ing. As access network speeds in- Era, June 2018. https://blog.apnic.net/2018/06/21/
measurementchallenges-in-the-gigabit-era/.
start period in the result, leading to a crease, it will be important to monitor 18. Minnesota.gov. CheckspeedMN; https://mn.gov/deed/
lower value of average throughput than not just “speed testing” but also to de- programsservices/broadband/checkspeedmn.
19. New York Attorney General’s Office. A.G.
the link capacity can support in TCP velop new methods that can monitor Schneiderman Encourages New Yorkers To Test
steady state. It also faces all of the user- and infer quality metrics for a variety Internet Speeds And Submit Results As Part Of
Ongoing Investigation Of Broadband Providers, 2017;
related considerations that we dis- of applications. https://ag.ny.gov/press-release/agschneiderman-
cussed previously. It is time to retire Adopt standard, open methods to facili- encourages-new-yorkers-testinternet-speeds-and-
submit-results-part.
the use of NDT for speed testing and tate better comparisons. It is currently 20. New York Attorney General’s Office. Are You Getting
look ahead to better methods. very difficult to directly compare the the Internet Speeds You Are Paying For? https: //
ag.ny.gov/SpeedTest.
Use native, embedded, and dedicated results of different speed tests, because 21. New York State Broadband Program Office-Speed
measurement techniques and devices. the underlying methods and platforms Test; https: //nysbroadband.ny.gov/speed-test.
22. Nginx Rate Limiting, 2019; https://www.nginx.com/
Web-based tests (many of which rely on are so different. Tools that select the blog/ rate-limiting-nginx/.
Javascript) cannot transfer data at rates highest result of several sequential 23. Ofcom. Broadband speeds: Research on fixed
line home broadband speeds, mobile broadband
that exceed several hundred megabits tests, or the average of several, or the performance, and related research; https: //www.
per second. As network speeds in- average of several tests after the high- ofcom.org.uk/research-and-data/telecomsresearch/
broadband-research/broadband-speeds.
crease, speed tests must be “native” ap- est and lowest have been discarded. As 24. Ookla Speedtest, 2019; https://speedtest.net/.
25. Oracle IaaS Pricing, 2019; https://cloud.oracle.com/
plications or run on embedded devices the FCC has stated:13 “A well-docu- en_US/iaas/pricing.
(for example, home router, Roku, Eero, mented, public methodology for tests 26. PC World. Six things that block your Wi-Fi, and
how to fix them, 2011; https://www.pcworld.com/
and AppleTV) or otherwise dedicated is critical to understanding measure- article/227973/six_ things_that_block_your_Wi-
devices (for example, Odroid, Raspber- ment results.” Furthermore, tests and Fi_and_how_to_fix_ them.html.
27. Penn State University. A Broadband Challenge:
ry Pi, SamKnows “white box,” and RIPE networks should disclose any circum- Reliable broadband internet access remains elusive
Atlas probes). stances that result in the prioritization across Pennsylvania, and a Penn State faculty
member is studying the issue and its impact, 2018;
Control for factors along the end-to- of speed test traffic. https://news.psu.edu/story/525994/2018/06/28/
end path when analyzing results. As we Beyond being well-documented and research/broadband-challenge.
28. Revolution Wi-Fi. The 2.4 GHz Spectrum congestion
outlined earlier, many factors can af- public, the community should also problem and AP form factors, 2015; http://www.
fect the results of a speed test other come to agreement on a set of stan- revolutionWi-Fi.net/ revolutionWi-Fi/2015/4/the-
dual-radio-apform-factor-is-to-blame-for-24-ghz-
than the capacity of the ISP link—rang- dards for measuring access link perfor- spectrumcongestion.
ing from cross-traffic in the home to mance and adopt those standards 29. Sundaresan, S., De Donato, W., Feamster, N.,
Teixeira, R., Crawford, S., and Pescape, A. broadband
server location and provisioning. As ac- across test implementations. Internet performance: A view from the gateway.
cess ISP speeds increase, these limit- In Proceedings of ACM SIGCOMM (Aug. 2011),
134–145.
ing factors become increasingly impor- References
30. Sundaresan, S., Feamster, N., and Teixeira, R.
1. Apple: Resolve Wi-Fi and Bluetooth Issues Caused
Home network or access link? Locating last-mile
tant, as bottlenecks elsewhere along by Wireless Interference, 2019. https://support.
downstream throughput bottlenecks. In Proceedings
the end-to-end path become increas- apple.com/en-us/ HT201542.
of the Intern. Conf. on Passive and Active Network
2. Bauer, S., Clark, D.D., and Lehr, W. Understanding
Measurement (2016), 111–123.
ingly prevalent. Broadband Speed Measurements. In Technology
31. Tanenbaum, A.S. et al. Computer Networks. Prentice
Policy Research Conference (TPRC), 2010.
Measure to multiple destinations. As 3. Bufferbloat. https://www.bufferbloat.net.
Hall, 1996.
32. Wehe, 2019; https://dd.meddle.mobi.
access network speeds begin to ap- 4. CALSPEED Program, 2019; http://cpuc.ca.gov/
33. Xfinity Internet Minimum System Recommendations;
General. aspx?id=1778.
proach and exceed 1Gbps, it can be dif- 5. Electronic Engineering Times. Avoiding iInterference
https://www.xfinity.com/support/articles/
requirementsto-run-xfinity-internet-service.
ficult to identify a single destination in the 2.4-GHz ISM band, 2006; https: //www.
eetimes.com/document.asp?doc_id=1273359.
and end-to-end path that can support 6. FCC. Measuring Broadband America Program (Fixed), Nick Feamster ([email protected]) is the Neubauer
the capacity of the access link. Looking GN Docket No. 12264, Aug. 2013; https://ecfsapi.fcc. Professor in the Computer Science Department and
gov/file/ 7520939594.pdf. Director of the Center for Data and Computing at the
ahead, it may make sense to perform 7. FCC: Measuring Broadband America Program; University of Chicago, IL, USA.
active speed test measurements to https://www.fcc. gov/general/measuring-broadband-
america. Jason Livingood ([email protected]) is
multiple destinations simultaneously, 8. FCC. Measuring Broadband America. Technical Vice President at Comcast, Philadelphia, PA, USA.
to mitigate the possibility that any sin- report, 2011; https://transition.fcc.gov/ cgb/
measuringbroadbandreport/Measuring_U.S._- Copyright held by authors/owners.
gle destination or end-to-end network _Main_Report_Full.pdf.
path becomes the network bottleneck. 9. FCC. MBA Report 2014, 2014; https://www.
fcc.gov/ reports-research/reports/measuring-
Augment active testing with applica- broadbandamerica/measuring-broadband-
tion quality metrics. In many cases, a america-2014.
10. FCC. Letter to FCC on Docket No. 17-108, 2014; Watch the authors discuss
user’s experience is not limited by the https://ecfsapi.fcc.gov/file/1083088362452/fcc-17- this work in the exclusive
108-reply-aug2017.pdf. Communications video.
access network speed, but rather the 11. FTP Rate Limiting, 2019; https://forum. https://cacm.acm.org/videos/
performance of a particular applica- filezillaproject.org/viewtopic.php?t=25895. measuring-internet-speed

80 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 research highlights
P. 82 P. 83
Technical
Perspective Enabling AI at the Edge
XNOR-Networks— with XNOR-Networks
Powerful but Tricky By Mohammad Rastegari, Vicente Ordonez,
By David Alexander Forsyth Joseph Redmon, and Ali Farhadi

P. 91 P. 92
Technical
Perspective SATURN: An Introduction
The Future to the Internet of Materials
of Large-Scale By Nivedita Arora, Thad Starner, and Gregory D. Abowd
Embedded Sensing
By Joseph A. Paradiso

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 81

research highlights
DOI:10.1145/ 3 42 9 9 5 0

Technical Perspective
To view the accompanying paper,
visit doi.acm.org/10.1145/3429945 rh

XNOR-Networks—
Powerful but Tricky
By David Alexander Forsyth

YOU C A N N OW run computations on deep network detects patterns of pat- used for training with a measure of
your phone that would have been un- terns of patterns. image similarity; these refinements
thinkable a few years ago. But as small All this means that CNNs tend to are adjusted dynamically throughout
devices get smarter, we discover new have a very large number of floating- the training process.
uses for them that overwhelm their point parameters, meaning that These tricks result in a compres-
resources. If you want your phone to running a CNN has traditionally re- sion procedure that can be applied
recognize a picture of your face (im- quired a GPU (or patience!). Building to any network architecture, with
age classification) or to find faces in networks with few parameters tends weights learned on any dataset. But
pictures (object detection), you want to result in classifiers that aren’t ac- compression produces a loss of ac-
it to run a convolutional neural net curate. But a CNN’s parameters are curacy. The ideal way to evaluate this
(CNN). redundant. For example, once a CNN procedure is to find others that pro-
Modern computer vision applica- has been trained, some procedures duce networks of the same size and
tions are mostly built using CNNs. for compressing its parameters don’t speed on the same dataset. Then the
This is because vision applications significantly affect its accuracy. CNNs compressor that produces the small-
tend to have a classifier at their can respond badly to apparently mi- est loss in accuracy wins. It’s hard to
heart—so, for example, one builds nor changes. For example, changing match size and speed, but a compres-
an object detector by building one from single precision to double pre- sor that produces the smallest loss
classifier that tells whether locations cision arithmetic can significantly af- of accuracy with acceptable size and
in an image could contain an object, fect accuracy. speed is the standard to beat.
then another that determines what How, then, to produce a CNN that The procedures described result
the object is. CNNs consist of a se- is small enough to run on a mobile de- in accuracies much higher than is
quence of layers. Each takes a block vice, and accurate enough to be worth achievable with comparable meth-
of data which has two spatial dimen- using? The strategies in the following ods. This work has roots in a paper
sions (like an image) and one feature paper are the best known to date. One that appeared in the Proceeding of the
dimension (for an image, red, green, builds a CNN where every parameter 2016 European Conference on Comput-
and blue), and then makes another is represented by a single bit, yield- er Vision. Since then, xnor.ai, a com-
such block, which it passes to the ing a very large reduction in size and pany built around some of the tech-
next layer. Most layers apply a convo- a speedup (a binary weight network nologies in this paper, has flourished.
lution and a nonlinearity to their in- or BWN). In a BWN, layers apply bi- The technologies described mean
put, typically increasing the feature nary weights to real valued inputs to you can run accurate modern com-
dimension. Some layers reduce the get real valued outputs. Even greater puter vision methods on apparently
spatial dimension by pooling spatial improvements in size and speed can quite unpromising devices (for exam-
windows in the input block. So, one be obtained by insisting that layers ple, a pi0). There is an SDK and a set of
might pool by replacing 2x2 non- accept and produce single bit data tutorials for this technology at https://
overlapping windows with the largest blocks (an XNOR-network). Multiply- ai2go.xnor.ai/getting-started/python.
value in that window. The final layer ing data by weights in an XNOR-net- Savings in space and computation
is usually classification by logistic re- work is particularly efficient, so very turn into savings in energy, too. An
gression. significant speedups are available. extreme example—a device that can
CNNs yield excellent classifiers, Producing a useful XNOR-network run accurate detectors and classifiers
because the training process chooses requires a variety of tricks. Pooling using only solar power—was just an-
image features that are useful for the binary values loses more informa- nounced (https://www.xnor.ai/blog/
particular classification task in hand. tion than pooling real values, so pool- ai-powered-by-solar).
For this to work, some layers must ing layers must be adjusted. Batch
have quite large feature dimensions, normalization layers must be moved David Alexander Forsyth is a professor of computer
science at the University of Illinois, Urbana-Champaign,
and the network needs to have many around. Training a conventional CNN IL. USA.
layers (yielding a “deep” network). then quantizing the weights produc-
Deep networks produce image fea- es a relatively poor classifier. Better
tures that have very wide spatial sup- is to train the CNN so it “knows” the
port and are complicated composites. weights will be quantized, using a se-
One should think of a single convo- ries of clever tricks described in the
lutional layer as a pattern detector; a paper. It helps to adjust the labels Copyright held by author.

82 COMMUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 DOI:10.1145/ 3 42 9 9 45

Enabling AI at the Edge

with XNOR-Networks
By Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi

Abstract on expensive, GPU-based machines, they are often unsuitable

In recent years we have seen a growing number of edge devices for smaller devices like cell phones and embedded electron-
adopted by consumers, in their homes (e.g., smart cameras ics. For example, AlexNet,11 one of the most well-known DNN
and doorbells), in their cars (e.g., driver assisted systems), and architecture for image classification, has 61M parameters
even on their persons (e.g., smart watches and rings). Similar (249MB of memory) and performs 1.5B high precision opera-
growth is reported in industries including aerospace, agricul- tions to classify one image. These numbers are even higher
ture, healthcare, transport, and manufacturing. At the same for deeper CNNs for example, VGG17 (see Section 3.1). These
time that devices are getting smaller, Deep Neural Networks models quickly overtax the limited storage, battery power,
(DNN) that power most forms of artificial intelligence are and compute capabilities of smaller devices like cell phones.
getting larger, requiring more compute power, memory, In this paper, we introduce simple, efficient, and accu-
and bandwidth. This creates a growing disconnect between rate approximations to CNNs by binarizing the weights and
advances in artificial intelligence and the ability to develop even the intermediate representations in convolutional neu-
smart devices at the edge. In this paper, we present a novel ral networks. Our binarization method aims at finding the
approach to running state-of-the-art AI algorithms at the best approximations of the convolutions using binary opera-
edge. We propose two efficient approximations to standard tions. We demonstrate that our way of binarizing neural net-
convolutional neural networks: Binary-Weight-Networks works results in ImageNet classification accuracy numbers
(BWN) and XNOR-Networks. In BWN, the filters are approxi- that are comparable to standard full precision networks
mated with binary values resulting in 32× memory saving. In while requiring a significantly less memory and fewer float-
XNOR-Networks, both the filters and the input to convolu- ing point operations.
tional layers are binary. XNOR-Networks approximate convo- We study two approximations: Neural networks with
lutions using primarily binary operations. This results in 58× binary weights and XNOR-Networks. In BWN all the weight
faster convolutional operations (in terms of number of the values are approximated with binary values. A convolutional
high precision operations) and 32× memory savings. XNOR- neural network with binary weights is significantly smaller
Nets offer the possibility of running state-of-the-art networks (∼32×) than an equivalent network with single-precision
on CPUs (rather than GPUs) in real-time. Our binary networks weight values. In addition, when weight values are binary,
are simple, accurate, efficient, and work on challenging visual convolutions can be estimated by only addition and subtrac-
tasks. We evaluate our approach on the ImageNet classifica- tion (without multiplication), resulting in ∼2× speed up.
tion task. The classification accuracy with a BWN version of Binary-weight approximations of large CNNs can fit into the
AlexNet is the same as the full-precision AlexNet. Our code is memory of even small, portable devices while maintaining
available at: urlhttp://allenai.org/plato/xnornet. the same level of accuracy (see Sections 3.1 and 3.2).
To take this idea further, we introduce XNOR-Networks
where both the weights and the inputs to the convolutional
1. INTRODUCTION and fully connected layers are approximated with binary val-
In recent years, the approach of using Deep Neural Networks ues. Binary weights and binary inputs allow an efficient way of
(DNN) to create artificial intelligence has been highly success- implementing convolutional operations. If all of the operands
ful in teaching computers to recognize8, 11, 17, 18 and detect4, 5, 16 of the convolutions are binary, then the convolutions can be
objects, read text, and understand speech.7 Such capabilities estimated by XNOR and bit-counting operations.2 XNOR-Nets
could have significant impacts on industries such a health- result in accurate approximation of CNNs while offering ∼58×
care, agriculture, aerospace, transport, and manufacturing, speed up in CPUs (in terms of number of the high precision
yet to date there are limited real world applications of DNN operations). This means that XNOR-Nets can enable real-time
and Convolutional Deep Neural Networks (CDNN). While inference in devices with small memory and no GPUs (infer-
there has been some progress made with virtual reality (VR ence in XNOR-Nets can be done very efficiently on).a
by Oculus),13 augmented reality (AR by HoloLens),6 and smart
wearable devices, the majority of applications rely on edge a
  Fully connected layers can be implemented by convolution, therefore, in
devices that have limited or no bandwidth, are low powered, the rest of the paper, we refer to them also as convolutional layers.12
and require the data to be stored locally for privacy and security
reasons. These constraints are at odds with the current state-of-
The original version of this paper is entitled “XNOR-Net:
the-art CNNs and DCNNs that require large amounts of com-
ImageNet Classification Using Binary Convolutional Neural
pute power and are therefore currently limited to the cloud.
Networks” and was published in (European Conference
CNN-based recognition systems need large amounts of
on Computer Vision (ECCV) 2016.
memory and computational power. While they perform well

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 83

research highlights

Figure 1. We propose two efficient variations of convolutional neural networks. Binary-Weight-Networks, when the weight filters contains
binary values. XNOR-Networks, when both weigh and input have binary values. These networks are very efficient in terms of memory and
computation, while being very accurate in natural image classification. This offers the possibility of using accurate vision techniques in
portable devices with limited resources.

Accuracy
* Operations Memory Computation Res-Net-50 (top-1)

Full precision R * R + – x 1x 1x 75.7%

Binary weight R * B + – ˜32x ˜2x 75.1%

XNOR-Networks B * B XNOR ˜32x ˜58x 70.3%

Bit-count

To the best of our knowledge this paper is the first attempt it is possible to reduce the precision of the parameters and
to present an evaluation of binary neural networks on large- the activation values for the neurons from 32 bits all the way
scale datasets like ImageNet. Our experimental results show down to a single bit. By reducing the precision we can save
that our proposed method for binarizing convolutional neu- in memory and computation. Single bit precision enables
ral networks outperforms the state-of-the-art network bina- using logical operations instead of floating point opera-
rization method of 2 by a large margin (16:3%) on top-1 image tions. Mathematically we present binary values in {−1, +1},
classification in the ImageNet challenge ILSVRC2012. Our therefore the arithmetic operations translates to logical
contribution is two-fold: First, we introduce a new way of operations in {0, 1}. As it is shown in Figure 2 the multipli-
binarizing the weight values in convolutional neural net- cation translates to XNOR operation and addition and sub-
works and show the advantage of our solution compared traction translate to popcount operations. These operations
to state-of-the-art solutions. Second, we introduce XNOR- are natively available in the most of the commodity CPUs in
Nets, a DNN model with binary weights and binary inputs edge devices and can be parallelized inside the CPU. Hence,
and show that XNOR-Nets can obtain similar classification it eliminated the need of GPU for fast computation.
accuracies compared to standard networks while being sig-
nificantly more efficient. Our code is available at: urlhttp:// 2.1. Binary-Weight-Networks
allenai.org/plato/xnornet. In order to constrain a convolutional neural network to
have binary weights, we estimate the real-value weight filter
2. BINARY CONVOLUTIONAL NEURAL NETWORK W ∈ Rc×w×h using a binary filter B ∈ {+1, −1}c×w×h. The best
To process an image for a variety of computer vision tasks, approximation is easy to find; the sign values of the ele-
we need to pass the image through a multi-layer convolu- ments in W. However, this approximation enforces a large
tional neural network. The major computational bottleneck amount of quantization error. To compensate this quanti-
is in the convolutional operations, which are combinations zation error, we introduce a scaling factor α ∈ R+ such that
of simple floating point arithmetic operations. In the state- W ≈ αB. A convolutional operation can be approximated by:
of-the-art CNN models the floating point operations are in
the order of billions. This is the main reason that processing I * W ≈ (I ⊕ B) α(1)
images with the state-of-the-art CNN models require GPU
servers. GPUs can parallelize these huge amount of float- where, ⊕ indicates a convolution without any multiplica-
ing point operations. But GPUs are expensive and consume tion. Since the weight values are binary, we can implement
extensive power to run. In this paper, we are questioning the convolution with additions and subtractions. The binary
floating point arithmetic operations in CNNs. We show that weight filters reduce memory usage by a factor of ∼32×

84 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

compared to single-precision filters. In Ref.,14 we found a Figure 2. Mathematically we present binary values in {−1, +1},
closed form optimal estimation for W ≈ αB by solving a con- therefore the arithmetic operations translates to logical operations
strained optimization problem. The optimal estimation of a in {0, 1}. The multiplication translates to XNOR operation and
binary weight filter can be simply achieved by taking the sign addition and subtraction translate to popcount operations.
of weight values. The optimal scaling factor is the average of
absolute weight values. {–1,+1} {0,1}
Training Binary-Weights-Networks. A naive solution for MUL XNOR
training the BWN could be first training a full precision model
and then simply quantizing the weight values as describes ADD, SUB Bit-Count (popcount)
above. This approach does not work. Figure 3 shows an
experiment on image classification task in ImageNet data-
set using a ResNet models with 50 layers. The top-1 accuracy Figure 3. This bar chart compares the top-1 accuracies in image
in the second bar from the left shows the accuracy of this classification on ImageNet challenge ILSVRC2012 using Residual
approach in compare with the full precision (first bar from Network model with 50 layers. From the left side the first bar shows
the accuracy of the full precision model, the second bar shows the
the left). The naive quantization destroys all the information accuracy when the model parameters are binarized with a naive
in the parameters of the network. The main challenge here approach as discussed in Section 2.1.1, the third bar shows the
is to find a set of real value weight filters that if we quantize accuracy when only the weights are binarized, the forth bar shows
them, we can reliably classify the categories of objects in an the accuracy when both weights and inputs are binarized, and the
image. To find this set of weight filters, we adopt a modified last bar shows the accuracy when the XNOR-Net model is trained
with Label Refinery (see Section 2.3).
version of gradient backpropagation algorithm.
ResNet-50 Top-1 (%) ILSVRC2012
Algorithm 1: Training a CNN with binary weights: 80

Input: A set of training images X 70

Output: Model parameters W 60

1:  Randomly initialize W 50
2:  for iter = 1 to N do
40
3: Load a random image X from the train set
4: Quantize the model parameters W as described 30
above 20
5: Forward pass the image X using the quantized
10
parameters 0.2
6: Compute the loss function (cross-entropy for 0
on

ive

ht

et

ne t
classification)

efi Ne
-N
eig

ry
isi

Na

l R R-
OR
ec

7: Backward pass to compute gradients with respect of be NO

Pr

XN
ry

La X
na
ll

the quantized parameters

Fu

Bi

8: Update the real-value weights W using the gradients

with a proper learning rate

Each iteration of training a CNN involves three steps; for- 2.2. XNOR-Networks
ward pass, backward pass and parameters update. To train So far, we could find binary weight filters for a CNN model.
a CNN with binary weights (in convolutional layers), we only The inputs to the convolutional layers are still real-value
quantize the weights during the forward pass and backward tensors. Now, we explain how to quantize both weight filters
propagation. To compute the gradient for the sign function, and input tensors, so convolutions can be implemented effi-
we follow the same approach as.2 For updating the parame- ciently using XNOR and bitcounting operations. This is the
ters, we use the high precision (real-value) weights. Because, key element of our XNOR-Networks. In order to constrain
in gradient descend the parameter changes are tiny, quanti- a convolutional neural network to have binary weights and
zation after updating the parameters ignores these changes binary inputs, we need to enforce binary operands at each
and the training objective cannot be improved. References2, step of the convolutional operation. A convolution consist of
3
also employed this strategy to train a binary network. repeating a shift operation and a dot product. Shift opera-
Algorithm 1 demonstrates a high-level schema of our proce- tion moves the weight filter over the input and the dot prod-
dure for training a CNN with binary weights. Once the train- uct performs element-wise multiplications between the
ing finished, there is no need to keep the real-value weights. values of the weight filter and the corresponding part of
Because, at inference we only perform forward propagation the input. If we express the dot product in terms of binary
with the binarized weights. In Figure 3 the third bar from operations, convolution can be approximated using binary
the left shows the accuracy of the binary weight network operations. Dot product between two binary vectors can be
trained with the proposed algorithm. As it can be seen, the implemented by XNOR–bitcounting operations.2 In Ref.,14
top-1 accuracy is as high as the full precision model while we explain how to approximate the dot product between two
the model size is about 32× smaller. vectors in Rn by a dot product between two vectors in {+1,

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 85

research highlights

−1}n. Similar to the binary weight approximation, we intro- of the values are +1, which means again we loose the infor-
duced scaling factor for the quantized input tensor and we mation for the next layer. However, this configuration does
found the optimal solution by solving a constrained optimi- not have the penalization problem in the backward pass.
zation that has a closed form solution for the weight filters Because the pooling here usually has one maximum per
and the input tensors. The optimal estimation of a binary each window. The XNOR-Net block configuration shown in
weight filter and an input tensor can be simply achieved by Figure 4(right), start with BatchNormalization and activa-
taking the sign of their values. The optimal scaling factors tion then a convolution and at the en the pooling. This con-
are the average of absolute values. figuration passes a binary input to the convolution, which
Next, we demonstrate how to use this approximation for generates a real-value tensor followed by a pooling which
estimating a convolutional operation between two tensors. produces a tensor with mostly positive values. This tensor
Now, using this approximation we can perform convo- goes to the batch normalization in the next layer and the
lution between input I and weight filter W mainly using mean centering in the batch normalization generates neg-
binary operations: ative values that when it passed to the activation, a proper
binary tensor can be generated that we pass it to the next
I * W ≈ (sign(I)  sign(W))  Kα(2) convolution.
Therefore, we put the pooling layer after the convolu-
where  indicates a convolutional operation using XNOR tion. To further decrease the information loss due to bina-
and bitcount operations and α is the scaling factor for the rization, we normalize the input before binarization. This
weight filter and K is a matrix of scaling factors for all of the ensures the data to hold zero mean, therefore, thresholding
spatial sections of the input tensor in the convolution. Note at zero leads to less quantization error. The order of layers in
that the number of non-binary operations is very small com- a block of binary CNN is shown in Figure 4(right).
pared to binary operations. Once we have the binary CNN structure, the training algo-
Training XNOR-Networks:  A typical block in CNN con- rithm would be the same as Algorithm 1.
tains several different layers. Figure 4(left) illustrates a Binary Gradient: The computational bottleneck in the
typical block in a CNN. This block has four layers in the backward pass at each layer is computing a convolution
following order: 1-Convolutional, 2-Batch Normalization, between weight filters and the gradients with respect of the
3-Activation, and 4-Pooling. Batch Normalization layer9 inputs. Similar to binarization in the forward pass, we can
normalizes the input batch by its mean and variance. The binarize the gradients in the backward pass. This leads to
activation is an element-wise non-linear function (e.g., a very efficient training procedure using binary operations.
Sigmoid, ReLU). The pooling layer applies any type of pool- Note that if we use the same mechanism to compute the
ing (e.g., max,min or average) on the input batch. For a scaling factor for quantized gradient, the direction of maxi-
binarized convolution the activation layer is the sign func- mum change for SGD would be diminished. To preserve the
tion. With the typical CNN block structure, binarization maximum change in all dimensions, we use the maximum
does not work. Applying pooling on binary input results in of the absolute values in the gradients as the scaling factor.
significant loss of information. For example, max-pooling k-bit Quantization: So far, we showed 1-bit Quantization
on binary input returns a tensor that most of its elements of weights and inputs using sign(x) function. One can
are equal to +1. Moreover, in the backward pass we often easily extend the quantization level to k-bits by using
observe more than one maximum that leads to uncer- qk(x) = 2( ) instead of the sign function. Where [.]
tainty in the penalization. One may assume that switch- indicates rounding operation and x ∈ [−1, 1].
ing between activation and the pooling layer will solve this
issues. In this case, the input to the activation layer is real 2.3. Improving accuracy using Label Refinery
value. Max pooling will often gives us a positive tensor. To further improve the accuracy of the XNOR-Networks,
Then the activation turns it into a unity matrix where most we introduced an iterative training methods in1 to update

Figure 4. This figure contrasts the block structure in our XNOR-Network (right) with a typical CNN (left).

A typical block in CNN A block in XNOR-Net

BNorm

BNorm
BNorm
Conv

Activ
Activ

Pool
Pool

Activ

Conv
W

Max-Pooling

Information No information Loss

Loss Single Maximums
Multiple Maximums

86 COMMUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

ground truth labels using a visual model trained on the convolution has cNWNI binary operations and NI non-binary
entire dataset. The Label Refinery produces soft, multi-cat- operations. With the current generation of CPUs, we can per-
egory, dynamically-generated labels consistent with the form 64 binary operations in one clock of CPU, therefore the
visual signal. The training images are labelled with the speedup can be computed by .
single category. After a few iterations of label refining, the The speedup depends on the channel size and filter size
labels from which the final model is trained are informa- but not the input size. In Figure 5(b) and (c), we illus-
tive, unambiguous, and smooth. This results in major trate the speedup achieved by changing the number of
improvements in the model accuracy during successive channels and filter size. While changing one param-
stages of refinement as well as improved model generaliza- eter, we fix other parameters as follows: c = 256, nI = 142
tion. The last column from left in Figure 3 shows the top-1 and nW = 32 (majority of convolutions in ResNet8 archi-
accuracy improvement for the XNOR-Network, which is tecture have this structure). Using our approximation
trained by Label Refinery. of convolution we gain 62.27× theoretical speed up, but
in our CPU implementation with all of the overheads,
3. EXPERIMENTS we achieve 58× speed up in one convolution (Excluding
We evaluate our method by analyzing its efficiency and the process for memory allocation and memory access).
accuracy. We measure the efficiency by computing the With the small channel size (c = 3) and filter size
computational speedup (in terms of number of high pre- (NW = 1 × 1) the speedup is not considerably high. This
cision operation) achieved by our binary convolution vs. motivates us to avoid binarization at the first and last layer
standard convolution. To measure accuracy, we perform of a CNN. In the first layer the channel size is three and in
image classification on the large-scale ImageNet dataset. the last layer the filter size is 1 × 1. A similar strategy was
This paper is the first work that evaluates binary neural used in.2 Figure 5(a) shows the required memory for three
networks on the ImageNet dataset. Our binarization tech- different CNN architectures(AlexNet, VGG-19, ResNet-18)
nique is general, we can use any CNN architecture. We with binary and double precision weights. BWN are so
evaluate AlexNet11 and two deeper architectures in our small that can be easily fitted into portable devices. BNN2
experiments. We compare our method with two recent is in the same order of memory and computation effi-
works on binarizing neural networks; BinaryConnect (BC)3 ciency as our method. In Figure 5, we show an analysis of
and BinaryNet (BNN)2. The classification accuracy of our computation and memory cost for a binary convolution.
BWN version of AlexNet is as accurate as the full precision The same analysis is valid for BNN and BC. The key differ-
version of AlexNet. This classification accuracy outper- ence of our method is using a scaling-factor, which does
forms competitors on binary neural networks by a large not change the order of efficiency while providing a sig-
margin. We also present an ablation study, where we evalu- nificant improvement in accuracy.
ate the key elements of our proposed method; computing
scaling factors and our block structure for binary CNN. We 3.2. Image classification
shows that our method of computing the scaling factors is We evaluate the performance of our proposed approach
important to reach high accuracy. on the task of natural image classification. So far, in the
literature, binary neural network methods have presented
3.1. Efficiency analysis their evaluations on either limited domain or simplified
In a standard convolution, the total number of operations datasets for example, CIFAR-10, MNIST, SVHN. To com-
is cNWNI, where c is the number of channels, NW = wh and pare with state-of-the-art vision, we evaluate our method
NI = winhin. Note that some modern CPUs can fuse the multiplica- on ImageNet (ILSVRC2012). ImageNet has ∼1.2M train
tion and addition as a single cycle operation. On those CPUs, images from 1K categories and 50K validation images.
BWN does not deliver speed up. Our binary approximation of The images in this dataset are natural images with

Figure 5. This figure shows the efficiency of binary convolutions in terms of memory (a) and computation (b and c). (a) Figure label a is
contrasting the required memory for binary and double precision weights in three different architectures(AlexNet, ResNet-18, and VGG-19).
(b, c) Figure labels b and c show speedup gained by binary convolution under (b) different number of channels and (c) different filter size.

Speedup by varying channel size Speedup by varying filter size

1200
1GB Double Precision 80x 65x
1000 Binary Precision
60x
800 60x
600 475MB
40x
400 55x
20x
200 100MB
16MB 1.5MB 7.4MB
0
0x 50x
1 32 1024 0x0 10x10 20x20
VGG-19 ResNet-18 AlexNet
number of channels filter size
(a) (b) (c)

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 87

research highlights

reasonably high resolution compared to the CIFAR and 0.9 for updating parameters in BWN and BC. For XNOR-Net
MNIST dataset, which have relatively small images. We and BNN we used ADAM.10 ADAM converges faster and usu-
report our classification performance using top-1 and ally achieves better accuracy for binary inputs.2 The learn-
top-5 accuracies. We adopt three different CNN architec- ing rate starts at 0.1 and we apply a learning-rate-decay =
tures as our base architectures for binarization: AlexNet,11 0.01 every four epochs.
Residual Networks (known as ResNet),8 and a variant Test: At inference time, we use the 224 × 224 center crop
of GoogLenet.18. We compare our BWN with BC3 and for forward propagation.
our XNOR-Networks (XNOR-Net) with BinaryNeuralNet Figure 6 demonstrates the classification accuracy for
(BNN).2 BC is a method for training a DNN with binary training and inference along the training epochs for top-1
weights during forward and backward propagations. and top-5 scores. The dashed lines represent training
Similar to our approach, they keep the real-value weights accuracy and solid lines shows the validation accuracy. In
during the updating parameters step. Our binarization is all of the epochs our method outperforms BC and BNN by
different from BC. The binarization in BC can be either large margin (∼17%). Table 1 compares our final accuracy
deterministic or stochastic. We use the deterministic with BC and BNN. We found that the scaling factors for
binarization for BC in our comparisons because the sto- the weights (α) are much more effective than the scaling
chastic binarization is not efficient. The same evalua- factors for the inputs (β). Removing β reduces the accu-
tion settings have been used and discussed in.2 BNN2 is a racy by a small margin (less than 1% top-1 AlexNet).
neural network with binary weights and activations dur- Binary Gradient: Using XNOR-Net with binary gradient
ing inference and gradient computation in training. In the accuracy of top-1 will drop only by 1.4%.
concept, this is a similar approach to our XNOR-Network Residual Net: We use the ResNet-18 proposed in8 with
but the binarization method and the network structure short-cut type B.
in BNN is different from ours. Their training algorithm is Train: In each training iteration, images are resized ran-
similar to BC and they used deterministic binarization in domly between 256 and 480 pixel on the smaller dimension
their evaluations. and then a random crop of 224 × 224 is selected for training.
CIFAR-10: BC and BNN showed near state-of-the-art per- We run the training algorithm for 58 epochs with batch size
formance on CIFAR-10, MNIST, and SVHN dataset. BWN equal to 256 images. The learning rate starts at 0.1 and we
and XNOR-Net on CIFAR-10 using the same network archi- use the learning-rate-decay equal to 0.01 at epochs number
tecture as BC and BNN achieve the error rate of 9.88% and 30 and 40.
10.17% respectively. In this paper, we explore the possibility Test: At inference time, we use the 224 × 224 center crop
of obtaining near state-of-the-art results on a much larger for forward propagation.
and more challenging dataset (ImageNet). Figure 7 demonstrates the classification accuracy (top-1
AlexNet: Reference11 is a CNN architecture with five con- and top-5) along the epochs for training and inference. The
volutional layers and two fully-connected layers. This archi- dashed lines represent training and the solid lines repre-
tecture was the first CNN architecture that showed to be sent inference. Table 2 shows our final accuracy by BWN and
successful on ImageNet classification task. This network XNOR-Net.b
has 61M parameters. We use AlexNet coupled with batch GoogLenet variant: We experiment with a variant of
normalization layers.9 GoogLenet18 that uses a similar number of parameters and
Train: In each iteration of training, images are resized to connections but only straightforward convolutions, no
have 256 pixel at their smaller dimension and then a ran- branching. It has 21 convolutional layers with filter sizes
dom crop of 224 × 224 is selected for training. We run the alternating between 1 × 1 and 3 × 3.
training algorithm for 16 epochs with batch size equal to Train: Images are resized randomly between 256 and
512. We use negative-log-likelihood over the soft-max of the 320 pixel on the smaller dimension and then a random
outputs as our classification loss function. In our imple-
mentation of AlexNet we do not use the Local-Response- b
 Our implementation is followed by https://gist.github.com/szagoruyko/
Normalization (LRN) layer. We use SGD with momentum = dd032c529048492630fc.

Figure 6. This figure compares the imagenet classification accuracy on top-1 and top-5 across training epochs. Our approaches BWN and
XNOR-Net outperform BinaryConnect (BC) and BinaryNet (BNN) in all the epochs by large margin (∼17%).

Top-1, Binary-Weight Top-1, Binary-Weight-Input Top-5, Binary-Weight Top-5, Binary-Weight-Input

60 60 80 80
Accuracy(%)
Accuracy(%)

Accuracy(%)

Accuracy(%)

60 60
40 40
40 40
BWN-train XNOR-Net-train BWN-train XNOR-Net-train
BWN-val XNOR-Net-val BWN-val XNOR-Net-val
20 BC-train 20 BNN-train
20 BC-train
20 BNN-train
BC-val BNN-val BC-val BNN-val

0 10 20 0 10 20 0 10 20 0 10 20
Number of epochs Number of epochs Number of epochs Number of epochs

88 COM MUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

Table 1. This table compares the final accuracies (top-1–top-5) of the full precision network with our binary precision networks;
Binary-Weight-Networks (BWN) and XNOR-Networks (XNOR-Net) and the competitor methods; BinaryConnect (BC) and BinaryNet (BNN).

Classification accuracy (%)

Binary-Weight Binary-Input-Binary-Weight Full-precision
BWN BC2 XNOR-Net BNN2 AlexNet11
Top-1 Top-5 Top-1 Top-5 Top-1 Top-5 Top-1 Top-5 Top-1 Top-5
56.8 79.4 35.4 61.0 44.2 69.2 27.9 50.42 56.6 80.2

Figure 7. This figure shows the classification accuracy; (a) top-1 and (b) top-5 measures across the training epochs on ImageNet dataset by
Binary-Weight-Network and XNOR-Network using ResNet-18.

ResNet, Top-1 ResNet, Top-5

80 80
Accuracy(%)

Accuracy(%)
60 60

40 40
BWN-train BWN-train
BWN-val BWN-val
20 XNOR-Net-train
XNOR-Net-val
20 XNOR-Net-train
XNOR-Net-val

0 20 40 60 0 20 40 60
Number of epochs Number of epochs
(a) (b)

Table 2. This table compares the final classification accuracy Table 3. In this table, we evaluate two key elements of our ap-
achieved by our binary precision networks with the full precision proach; computing the optimal scaling factors and specifying
network in ResNet-18 and GoogLenet architectures. the right order for layers in a block of CNN with binary input. (a)
Demonstrates the importance of the scaling factor in training
Binary-Weight-Networks and (b) shows that our way of ordering
ResNet-18 GoogLenet the layers in a block of CNN is crucial for training XNOR-Networks.
Network variations Top-1 Top-5 Top-1 Top-5 C, B, A, P stands for Convolutional, BatchNormalization, Active
function (here binary activation), and Pooling respectively.
Binary-Weight-Network 60.8 83.0 65.5 86.1
XNOR-Network 51.2 73.2 N/A N/A
Full-precision-network 69.3 89.2 71.3 90.0 Binary-Weight-Network
Strategy for computing α Top-1 Top-5
Using the scaling factor 56.8 79.4
crop of 224 × 224 is selected for training. We run the Using a separate layer 46.2 69.5
training algorithm for 80 epochs with batch size of 128. (a)
The learning rate starts at 0.1 and we use polynomial rate XNOR-Network
decay, β = 4.
Block structure Top-1 Top-5
Test: At inference time, we use a center crop of 224 × 224.
C-B-A-P 30.3 57.5
B-A-C-P 44.2 69.2
3.3. Ablation studies (b)
There are two key differences between our method and the
previous network binarization methods; the binarization
technique and the block structure in our binary CNN. For
binarization, we find the optimal scaling factors at each for each filter. This is similar to computing the affine
iteration of training. For the block structure, we order parameters in batch normalization. Table 3(a) compares
the layers in a block in a way that decreases the quan- the performance of a binary network with two ways of com-
tization loss for training XNOR-Net. Here, we evaluate puting the scaling factors. As we mentioned in Section
the effect of each of these elements in the performance 2.2.1 the typical block structure in CNN is not suitable
of the binary networks. Instead of computing the scal- for binarization. Table 3(b) compares the standard block
ing factor α, one can consider α as a network parameter. structure C-B-A-P (Convolution, Batch Normalization,
In other words, a layer after binary convolution multi- Activation, and Pooling) with our structure B-A-C-P. (A, is
plies the output of convolution by an scalar parameter binary activation).

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 89

research highlights

4. CONCLUSIONC 8. He, K., Zhang, X., Ren, S., Sun, J. Deep 14. Rastegari, M., Ordonez, V., Redmon, J.,
residual learning for image recognition. Farhadi, A. XNOR-Net: Imagenet
We introduce simple, efficient, and accurate binary approx- In Proceedings of the IEEE classification using binary
imations for neural networks. We train a neural network Conference on Computer Vision and convolutional neural networks. In
Pattern Recognition (2016), 770–778. European Conference on Computer
that learns to find binary values for weights, which reduces 9. Ioffe, S., Szegedy, C. Batch Vision (2016), Springer.
the size of network by ∼32× and provide the possibility of normalization: Accelerating deep 15. Redmon, J. Darknet: Open source
network training by reducing internal neural networks in C, 2013–2016.
loading very DNN into portable devices with limited mem- covariate shift. arXiv preprint http://pjreddie.com/darknet/.
ory. We also propose an architecture, XNOR-Net, that uses arXiv:1502.03167 (2015). 16. Ren, S., He, K., Girshick, R., Sun, J.
10. Kingma, D., Ba, J. Adam: A method for Faster R-CNN: Towards real-time
mostly bitwise operations to approximate convolutions. stochastic optimization. arXiv preprint object detection with region proposal
arXiv:1412.6980 (2014). networks. In Advances in Neural
This provides ∼58× speed up and enables the possibility 11. Krizhevsky, A., Sutskever, I., Hinton, G.E. Information Processing Systems
of running the inference of state of the art DNN on CPU Imagenet classification with deep (2015), 91–99.
convolutional neural networks. In 17. Simonyan, K., Zisserman, A. Very deep
(rather than GPU) in real time. Advances in Neural Information convolutional networks for large-
Processing Systems (2012), 097–1105. scale image recognition. arXiv preprint
c
 We used the Darknet15 implementation: http://pjreddie.com/darknet/ 12. Long, J., Shelhamer, E., Darrell, arXiv:1409.1556 (2014).
T. Fully convolutional networks 18. Szegedy, C., Liu, W., Jia, Y., Sermanet, P.,
imagenet/#extraction. for semantic segmentation. In Reed, S., Anguelov, D., Erhan, D.,
Proceedings of the IEEE Conference Vanhoucke, V., Rabinovich, A. Going
on Computer Vision and Pattern deeper with convolutions. In
Recognition (2015), 3431–3440. Proceedings of the IEEE Conference
References 4. Girshick, R. Fast R-CNN. In 13. Oculus, V. Oculus rift-virtual reality on Computer Vision and Pattern
1. Bagherinezhad, H., Horton, M., Ras- Proceedings of the IEEE International headset for 3D gaming, 2012. URL: Recognition (2015), 1–9.
tegari, M., Farhadi, A. Label refinery: Conference on Computer Vision http://www.oculusvr.com.
Improving imagenet classification (2015), 1440–1448.
through label progression. arXiv 5. Girshick, R., Donahue, J., Darrell, T.,
preprint arXiv:1805.02641 (2018). Malik, J. Rich feature hierarchies
2. Courbariaux, M., Hubara, I., Soudry, for accurate object detection
Mohammad Rastegari, Joseph Redmon, Vicente Ordonez (vicente@virginia.
D., El-Yaniv, R., Bengio, Y. and semantic segmentation. In
and Ali Farhadi ({mohammad,joe,ali}@ edu), Department of Computer Science,
Binarized neural networks: Training Proceedings of the IEEE Conference
xnor.ai), XNOR.AI, Seattle, WA, USA. University of Virginia, Charlottesville,
deep neural networks with weights on Computer Vision and Pattern
VA, USA.
and activations constrained Recognition (2014), 580–587.
to +1 or –1. arXiv preprint 6. Gottmer, M. Merging Reality and
arXiv:1602.02830 (2016). Virtuality with Microsoft Hololens, 2015.
3. Courbariaux, M., Bengio, Y., David, J.-P. 7. Hannun, A., Case, C., Casper, J.,
Binaryconnect: Training deep neural Catanzaro, B., Diamos, G., Elsen, E.,
networks with binary weights during Prenger, R., Satheesh, S., Sengupta, S.,
propagations. In Advances in Neural Coates, A., et al. Deep speech: Scaling
Information Processing Systems up end-to-end speech recognition.
(2015), 3105–3113. arXiv preprint arXiv:1412.5567 (2014). © 2020 ACM 0001-0782/20/12 $15.00

90 COMMUNICATIO NS O F TH E ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

 DOI:10.1145 / 3 42 9 9 5 2

Technical Perspective
To view the accompanying paper,
visit doi.acm.org/10.1145/3429948 rh

The Future of Large-Scale

Embedded Sensing
By Joseph A. Paradiso

T HE D REA M O F computational material nant structures all over the U.S. Embas- bedded sensors and provide a more eco-
has been in the air for decades, dating sy in Moscow from over a half-century nomical and practical energy solution.
at least to the Smart Matter program at ago—microwave backscatter from cavi- On the other hand, if we have a multi-
Xerox PARC in the late 1990s. Inspired ties with a flexible surface could pick tude of devices in our environments that
by the complexity of biological skin, my up audio across the complex (see Eric must last decades, energy harvesting
own team (see Sensate Media, Communi- Haseltine’s The Spy in Moscow Station, for may be mandated, and here we will need
cations, Mar. 2005, p. 70) and others have example, Leon Theremin, famous for his area for photovoltaics, thermoelectrics,
looked to integrate distributed sensing free-gesture electronic musical instru- piezoelectrics, or even, as in this case,
into large flexible membranes, a trend ment from circa 1920, is purported to be perhaps triboelectrics or maybe RF or
that continues in research today (see the the inventor of these devices). But as this inductive energy receivers. Large flat
IEEE 2019 Proceedings on Flexible Elec- paper attests, it’s back in vogue again— sheets provide such area, and research-
tronic Skin). Most of these devices, how- some recent incarnations of passive ers have built systems using all of these
ever, are actively powered. The SATURN acoustic sensor backscatter can also be approaches (see ‘The Superpowers of
system described in the following paper found in the recent work of Josh Smith’s Super-Thin Materials,’ NYT Jan. 7, 2020),
works passively, energized essentially team at UW (for example, his battery- even building sensors and electronics
by static electricity generated as layers less cellphone) and my colleague Fadel into fibers and fabrics, but it’s not yet
move relative to each other during vibra- Afib’s self-powered underwater sonar clear what the driving applications are.
tion, hearkening perhaps to, at a smaller backscatter sensor, which is acoustically We will see flexible display ‘wallpaper’ in
scale, electret microphones, which ex- interrogated instead of using radio. the not too distant future, but this will
ploit charge trapped on their foil mem- The authors espouse the vision of definitely be a powered system (this
brane to produce a vibration-dependent large surface-area passive sensors that world will witness an interesting tension
voltage. Using only two components— can be cheaply manufactured, perhaps between photons beamed to our retinas
an FET and matching inductor—the au- by a roll-roll process, and laminated via ubiquitous AR glasses vs light from
thors are able to modulate the resonance onto the walls and surfaces in our envi- everywhere displays). Perhaps its first
of a RF antenna that can be embedded ronment. There is potential competition market will be in building materials (for
in the material and read out via passive here, however, from the opposite tack— example, passively detecting dampness,
backscatter from an external transmit- making the sensors small and compact strain, or temperature, after they are in-
ter, allowing a material to work as an au- using MEMS and standard IC technol- stalled, as envisioned in my team’s origi-
dio pickup without a power source. ogy and embedding them into the smart nal ‘Sensor Tape’ project from 2012).
Traditional work in this area has surface like raisins in pudding. Looking Passive sensate structures, as es-
tended to exploit piezoelectric polymers at the application proposed here, for poused in this paper, will enable sens-
like PVDF, which generate voltage under example, Jon Bernstein and his team ing everywhere. We are already living
strain. Triboelectrics present a different at Draper Lab have recently built a pas- in a world where networked sensing
approach, although provide probably sive MEMS acoustic switch that closes risks privacy behind every door—once
an even higher source impedance that at a particular sonic amplitude—this our commonplace materials beam new
would challenge power conversion even could easily toggle a backscatter anten- streams of ubiquitous sensor data, this
more. SATURN sidesteps this entirely, na to enable remote readout. The world reaches another level, as even coarse but
using the generated voltage directly at also begins to see implementations of plentiful data can leverage potentially
the gate of the resonance-modulating ‘Smart Dust’ as envisioned by Kris Pister invasive contextual determination. The
FET (ironic, in that we usually work to in 2001—for example, compact stacks paper describes some simple ideas of
avoid destructive static charge there— of bare IC die, sparsely powered by pho- physically ‘opting in’ with these mate-
but the potentials are much lower here). todetectors on the top layer and talking rials, but I think the details of how pri-
Hence, the key contribution of this pa- via backscatter, such as prototyped by a vacy will be managed will be much more
per is a means of transmitting audio fea- University of Michigan consortium. complex when life is enveloped with so
tures from a passive triboelectric-gener- How we will power these sensors is an many digital peepholes looking at us
ating material. area of similar technical tension. When from everything.
Remotely monitoring audio from the power requirement is sufficiently
backscatter in passive structures has a low to warrant energy scavenging, a Joseph A. Paradiso is the Alexander W. Dreyfoos (1954)
Professor and Associate Academic Head of the Program in
long and notorious history in electronic small, embedded battery will generally Media Arts and Sciences at the Massachusetts Institute
espionage—classic stories abound of of Technology, Cambridge, MA, USA.
survive close its shelf life, which can ap-
ingenious Russian bugs built into reso- proach the product life cycle of the em- Copyright held by author.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 91

research highlights
DOI:10.1145/ 3 42 9 9 48

SATURN: An Introduction
to the Internet of Materials
By Nivedita Arora, Thad Starner, and Gregory D. Abowd

Sound impacts the SATURN vibration sensor, which is formed in the shape of the elephant’s ear. SATURN’s triboelectric components vibrate
(inset), generating an electrical signal.

Abstract research. Weiser correctly predicted an era of proliferation

We envision a new generation of computation devices, com- of many differently sized devices aimed at augmenting our
putational materials, which are self-sustainable, cheaply human experience with technology. Today’s realization of
manufactured at scale and exhibit form factors that are easily ubiquitous computing devices—smartphones, tablets, elec-
incorporated into everyday environments. These materials tronic whiteboards, and wearables—is still fairly easy to dis-
can enable ordinary objects such as walls, carpet, furniture, tinguish from everyday objects in the physical world. Sizes
jewelry, and cups to do computational things without look- may vary, but there are still very distinctive characteristics
ing like today’s computational devices. Self-powered Audio of something that is computational. The Internet of Things
Triboelectric Ultra-thin Rollable Nanogenerator (SATURN) (IoT) has tried to hide computing into more and more every-
is an early example of a computational material that can day objects, such as light bulbs, television sets, and speakers,
sense vibration, such as sound. SATURN can be manufac- but we are still far from a complete blurring of the physical
tured from inexpensive components, is flexible so that it can and digital worlds. To make something computational still
be integrated into many different surfaces, and powers itself requires “smarts” composed of off-the-shelf integrated cir-
through the sound or vibration it is sensing. Using radio cuits housed in rigid modules that are packaged with exist-
backscatter, we demonstrate that SATURN’s sensed data is ing objects. Computing is too separate from the materials of
passively transmitted to remote computers, alleviating the everyday objects. We propose a different direction based on
need for batteries or any wired power for the material itself. Weiser’s vision, that is, starting with the materials of every-
The proliferation of these types of computational materials day life and creating computation from there. In doing so,
ushers an era of Internet of Materials, further blurring the we propose an Internet of Materials (IoM), where the very
distinction between the physical and digital worlds. materials of objects and surfaces are augmented or manu-
factured to have computational capabilities. This recasting
1. INTRODUCTION
The most profound technologies are those that disappear.
They weave themselves into the fabric of everyday life until The original version of this paper is entitled “SATURN:
they are indistinguishable from it.18 A thin and ­flexible self-powered microphone designed
on the principle of ­triboelectric nanogenerator” and was
published in Proceedings of the ACM on ­Interactive, Mobile,
This poetic and mostly metaphorical vision from Mark
Wearable and Ubiquitous Technologies 2.2 (60), 2018, ACM
Weiser inspired nearly 30 years of ubiquitous computing

92 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

of ubiquitous computing as “computational materials” SATURN,3 a flexible, self-powered sound sensor and sound
presents three major challenges: energy harvester that is constructed with thin and inex-
pensive materials. The novelty of the work lies in creating a
• Power: Computational materials should be self-­ device that considers power, cost, and form factor as central
sustainable in terms of power consumption. They design parameters without sacrificing signal quality (Table 1).
should not require a wired, constant power source or
battery replacements. Instead, they should be able to 2.1. Triboelectric nanogenerator (TENG)
harvest power from the environment for operation. We When two different materials come into contact and sepa-
would never want to plug in an everyday object, such as rate, or rub alongside each other, they tend to either gain or
a cup, nor do would we ever want to worry about replac- lose electrons, based on their position relative to each other
ing a battery or recharging it. in the triboelectric series.21 This common phenomenon of
• Cost: Computational materials should be cheap to exchange of electrons is called triboelectrification. The redis-
manufacture at large scale. Despite progress in manu- tribution of charge creates an electric potential between the
facturing increasingly complex and powerful integrated layers. If there is a conductive path between the two layers,
circuits, as dictated by Moore’s law, we cannot afford to the charge difference will balance due to electrostatic induc-
make computational materials the same way. It would tion. Repeated contact and separation, therefore, produces
be too expensive and would not scale to cover entire an alternating current.15 This multilayer structure, consist-
buildings, outdoor surfaces, or clothing. ing of different materials that are both conductive on one
•  Form Factor: Computational materials should be flexi- side, is called a triboelectric nanogenerator (TENG).
ble or dispersible such that they can be easily integrated
into everyday objects and surfaces. The key to blurring 2.2. Device design and fabrication
the physical and digital worlds is to make the digital SATURN is an example of TENG and consists of two layers
objects look and feel more like physical objects. (Figure 1). The first is the copper that acts as a triboelectri-
cally positive material and is coated onto paper for mechani-
For several years, we have been driving our research cal support. Paper is used because of its low cost, although
with these three challenges. We present here one of our first its flexibility, light weight, and ease of perforation all sup-
successful examples of a computational material, SATURN port its vibration in the presence of sound. The second layer
(Self-powered Audio Triboelectric Ultra-thin Rollable is a dielectric plastic, PTFE (polytetrafluoroethylene), which
Nanogenerator). SATURN is a demonstration of a thin and acts as a triboelectrically negative material and is coated on
flexible multilayer material for detecting mechanical vibra- one side with copper using physical vapor deposition. The
tions, which are ubiquitous in our everyday environment. first and second layers are placed on top of each other, with
Inspired by recent results in materials science, SATURN the copper side of the paper touching the noncopper-coated
produces energy from mechanical vibrations using a truly side of the PTFE. The layers are anchored to each other
ubiquitous phenomenon that occurs between any two dif- using glue with a specific dot pattern. A potential difference
ferent materials rubbing against each other. We show how is caused by vibration and is measured between the two
this fundamental property of materials can be turned into copper-­coated surfaces (see Figure 2).
a self-sustaining sensor and then demonstrate how other SATURN’s structural design is tuned to increase its elec-
simple materials can be used to help turn the sensor into a trical response across a wide frequency range. Structural
wireless sensor.
We outline the paper in six main sections. Section 2 intro- Table 1. Power generated by SATURN at 100 dB and 250 Hz sound
duces the design and working principles of SATURN. Section 3 frequency.
characterizes SATURN as a self-sustainable microphone and
loud sound energy harvester. It is followed by Section 4 which Size of patch 4 × 4 cm2
Resonant frequency 255 Hz
demonstrates different ways of building a self-­sustainable
Load impedance 0.9 MΩ
computational system leveraging SATURN. Next we dis-
Max. Vpp 2.5 V
cuss applications in Section 5, some of which are imple- Max. power 6944 nW
mented and some are exploratory in nature. In Section 6,
we discuss how this work impacts our thinking in develop-
ing computational materials.
Figure 1. Structural design of a SATURN microphone consisting of
2. SATURN: WORKING PRINCIPLES copper-coated paper and PTFE.
Recent advances in materials science demonstrate the pos-
Paper PTFE SATURN
sibility of self-powered, easy-to-manufacture sensors that
Front
take advantage of the triboelectric nanogenerator (TENG) Front
Back Back
effect which converts mechanical vibrations into electri-
cal energy.14, 16, 17 When made in the right form factor, these + =
mechanical energy generators could be manufactured as
self-sustainable sound and vibration sensors. We use these
principles for the design, fabrication, and evaluation of

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 93

research highlights

design parameters (Figure 3) such as hole size and spacing, in the paper moving toward the PTFE again, resulting in a
the geometry of the patch, and the glue points attaching reversed direction of current flow (Figure 4d), completing
the two layers are varied to understand the effects on signal the cycle of electricity generation.
quality using a combination of evaluation techniques. These
parameters are optimized empirically based on measured 3. PERFORMANCE CHARACTERIZATION
voltage generated by the microphone at a standardized deci-
bel level and by using a mechanical model simulation that 3.1. Self-sustainable microphone
compares the separation distance between the two layers of Even though SATURN is self-sustainable, flexible, and thin,
SATURN when vibrating. its quality as a sensor is comparable to an active micro-
phone that consumes power. After structural optimization
2.3. Operation of SATURN, the best acoustic sensitivity of −25.63 dB (rela-
Change in air pressure due to sound vibrations causes con- tive mV/Pa) is achieved at 1000 Hz. The resulting SATURN
stant contact and separation in the multilayer structure of patch has a circular shape with a 16 cm2 area, a grid pat-
SATURN. When the two layers are in contact with each other, tern of holes 0.4 mm in diameter with 0.2 mm spacing, and
charges are induced in the copper and the PTFE due to tri- glue attachments of the two layers at the center and at eight
boelectrification (Figure 4a). PTFE, which has a greater elec- equally distant points around the edges of the PTFE (Figure
tron affinity, is able to gain electrons from the copper and 5). In this configuration, the SATURN microphone compares
becomes negatively charged, whereas the copper layer on the favorably to an active microphone (MEMS ADMP-401 and
paper becomes positively charged. Subsequent separation iPhone INMP441) for frequencies as high as 5000 Hz (Figure
of the paper and the PTFE (Figure 4b) induces a potential dif-
ference across the two copper electrodes, causing current to Figure 4. Cycle of electricity generation process under external
flow from the paper toward the PTFE when the device is con- acoustic excitation.
nected to an external load. This flow of current reverses the
polarity (Figure 4c) of charges on the two copper electrodes Paper b
(i.e., now the copper on PTFE has more positive charge than Copper
PTFE
the copper layer on the paper). The next compression results c
a
Figure 2. Fabrication process: (1) preparation of micro-hole paper;
(2) deposition of copper layer; (3) attaching copper tape as Maximum Contact Minimum Contact
electrodes; (4) stacking paper and PTFE; and (5) gluing paper and d
PTFE. All dimensions are in mm.
Step 4
44 40 44 6
copper
5

10

Figure 5. Sensitivity across acoustic bands (20 Hz to 20 kHz).

47

40

10 10

Paper Laser Cutting Copper Deposition Copper Tape Attachment

–20
10

Step 1 Step 2 Step 3 Copper Tape Attachments

5 5
–30 Acoustic Sensitivity : –26
64
copper dB , 1000 Hz @94 dB
35

35

–40
Power (dB)

Step 5
–50
67

Glue
–60

PTFE Copper Deposition Copper Tape Attachment –70

–80
Paper Copper PTFE Copper Tape Glue
Gluing –90
0 2 4 6 8 10 12 14 16 18 20
Frequency (KHz)

Figure 3. SATURN’s structural design parameters.

Hole size and spacing Hole pattern Patch Size Figure 6. Acoustic sensitivity of SATURN sensor compared to an
2cm active microphone.
4cm

8cm
Ground Truth SATURN mic
10 –40 10 –40
9 9
8
–50
8 Georgia Institute of Technology –50

Glue attachment points Geometry 7 7

Back support and material
Frequency (kHz)

Frequency (kHz)

–60 –60
6 6
Full back Support Unsupported 5 –70 5 –70

4 4
–80 –80
3 3

2 2
–90 –90
1 1

0 –100 0 –100
3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 5 5.2 3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6
Time (secs) Time (secs)

94 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

6). Because approximately 90% of the acoustic information detection systems for loud sounds. These self-sustainable
for human speech lies within this range,4 SATURN can be wireless sensing and communication systems maintain the
used a good quality microphone for a variety of applications. form factor of the computational material although still
Bending of SATURN reduces the acoustic sensitivity as being potentially cheap to manufacture.
the bend angle increases due to the increase in the stiff-
ness of the structure, which results in lesser vibration of the 4.1. Analog backscatter-based communication for
layers. At a bending angle of 45°, SATURN is still a usable sound and vibration
microphone and is comparable to an active microphone up We combine the self-sustainable mechanical vibration
to 3000 kHz, allowing capture of more than 60% of the sound sensing property of SATURN with a self-sustainable com-
information of speech.4 See Figure 7. munication technique called analog backscatter, to build
a thin flexible material tag that can self-sustainably both
3.2. Loud sound energy harvester sense and communicate audio. The word backscatter in
Next we look at SATURN as an energy harvester for loud acous- analog backscatter means reflection. We can explain ana-
tic events. We performed an experiment to determine the log backscatter by using a simple analogy of reflection of
peak voltage and peak power of the SATURN microphone at light rays (Figure 9). When light hits a mirror, it bounces off
its resonant frequency as functions of an external load resis- the surface at an angle equal to the angle of the incoming
tance. We analyzed the 4 × 4 cm2 SATURN microphone patch light wave. However, when light hits an irregular surface it
as a power harvester when exposed to loud sounds (100 dB). gets diffused, resulting in modulation in both the intensity
The voltage is approximately 0.5 Vpp at 150 Hz and rises to a and angle of reflection of the light waves. Analysis of the
maximum at 2.5 Vpp at 250 Hz and then falls below 1.0 Vpp at received diffused light can help deconstruct the shape of
350 Hz. The same behavior is shown in the power curve, with the irregular surface from which it is reflected. Similar to the
a maximum power of 6499 nW (Figure 8 and Table 1). light waves, when radio frequency (RF) waves are incident on
a specially designed tag, they get modulated in amplitude
4. SELF-SUSTAINABLE COMPUTATIONAL and phase. This reflected RF from the tag can be processed
SYSTEM DESIGN at the receiver to extract information of the phenomenon
To use SATURN in a practical application, it needs to be which caused the modulation. In our tag design, the modu-
embedded in a self-sustaining computational system. In lation is due to voltage generated in SATURN resulting from
this section, we look at three different ways of building such sound or mechanical vibration (Figure 10).
system using SATURN. The first system leverages SATURN as Our prototype tag consists of SATURN, a junction gate
a self-sustainable microphone to transfer a wide-­spectrum field-effect transistor ( JFET) that acts as a voltage-controlled
audio sounds (e.g., speech or taps). The second and the impedance device, and a printed antenna (Figure 11).1
third system demonstrate how we can leverage SATURN as Sounds and vibrations in the environment result in genera-
a sound energy harvester to build a self-sustainable event tion of voltage in SATURN which in turn changes the imped-
ance in the circuit via the JFET. This change in impedance
Figure 7. Experimental results for the effect of flexibility: change in changes the radar cross section of the antenna resulting in
acoustic sensitivity (1000 Hz at 94 dB obtained for different radii of amplitude modulation of the incoming RF waves. Our tag
curvature).
is an example of a computational material for audio and
Acoustic Sensitivity (dB re mV/Pa)

–25
∞ Radius of curvature (cm) Figure 9. RF analog backscatter can be explained using an analogy
–30 to the modulation of intensity and angle of reflection of light from a
45.8 rough surface.
–35
15.2 7.6 3.05
–40
5
3.8 2.2
–45

–50
0 10 20 30 40 50 60 70 80 90 100
Central angle theta (Degrees) for arc length 4cm

Regular light reflection lrregular light reflection

Figure 8. Voltage and power generated at different working from a smooth surface from a rough surface
frequencies.

2.5 7000
Figure 10.  Self-sustainable sound sensing and communication
6000
architecture using SATURN and analog backscatter technique.
Voltage peak-peak (Volt)

2
Power (nano Watt)

5000
1.5 4000

1 3000
CARRIER WAVE
2000
0.5
1000

0 0
100 150 200 250 300 350 400 450 500 550 600
Frequency (Hz) with 0.9 Mohm load SOUND SOURCE BACKSCATTERED SIGNAL TRANSCEIVER
TAG WITH AUDIO

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 95

research highlights

vibration sensing and communication. The simple circuit monitoring can be performed without the cost and envi-
design of our tag maintains the thin, flexible form factor of ronmental difficulties of batteries.
SATURN and can easily be embedded in everyday objects
and surfaces. 5. APPLICATION SCENARIOS
Amplitude modulation-based analog backscatter does In this section, we explore how self-sustainable compu-
not allow unique IDs for different SATURN patches, but other tational systems based on SATURN can be used in every-
methods such as frequency shift keying (FSK)11 can allow this day scenarios.
capability by the addition of a subhundred microwatt power
energy harvesting method based on easily available sources 5.1. Ubiquitous microphone for interaction and control
such as wireless energy or light sources in the room. The thin and flexible form factor of SATURN allows it to
be placed on many different surfaces (Figure 12). SATURN
4.2. Flip-bit data storage and RF interrogation system patches might be placed on walls or lamp shades in the
The 6.9 µW of power generated by SATURN due to a loud home to act as a baby monitor or extend the range of audio
sound can be used to flip a bit in nonvolatile memory to input for home assistants (e.g., Amazon Echo or Google
record the occurrence. Considering the maximum power Home). In addition to audio sensing, SATURN can be used
transfer theorem (Jacobi’s law), the usable power we can as a vibration sensor to detect simple input tap touch to con-
obtain is approximately 50%. Thus, we might harvest up to trol objects. For example, imagine a SATURN patch in the
3.4 µJ/s. The energy required to program a “1” into NAND form factor of a Post-it Note which could be placed on walls
flash memory is 2 µJ.8 Given that the sounds we wish to mon- or tables as a wireless light switch.
itor will probably last for several seconds, there is more than
enough power to record the acoustic event. Going further, 5.2. Audio sensing Post-it Notes for infrastructure
SRAM bits can be flipped at approximately 10–100 pW of mapping and authentication
power,5, 10 suggesting that rudimentary computation might Imagine placing a SATURN Post-it Note at a conference
be performed to determine if the flash memory bit should room door entrance which only authorized users can open
be written. Recorded bits might be read later using a pas- using a password consisting of unique combinations of
sive RFID interrogation system, which can both read the blow, whistle, or tap (Figure 13). In a public restroom sce-
recorded state and reset it. nario, SATURN Post-it Notes can be placed on restroom
doors; a special sequence of tapping can start a mainte-
4.3. Ultralow power radio communication system nance request. Multiple SATURN patches with pre-mapped
SATURN could power longer range radio transmitters allow- IDs can be placed at multiple places in a nursing home and
ing real-time alerts to sounds that exceed a loudness thresh- could be used for easy access to an emergency help button
old. Talla et al.13 have recently demonstrated a 915 MHz by tapping or speaking to the patch. The main advantage of
analog LoRa backscatter communications device that can such audible Post-it Notes is that they are cheap and dispos-
communicate at greater than 11 bits/s although hundreds able, reducing concerns of them being lost or stolen.
of meters away from its RF source and receiving antenna.
Although their system currently uses a battery, their theo- 5.3. Context sensing and localization
retical IC design consumes only 9.25 µW of power. With Using multiple SATURN patches can allow beam-form-
sound events lasting on the order of seconds, one can ing such that individual voices may be better isolated
imagine a SATURN-based system storing power until it
has enough to enable a 915 MHz backscatter transmis- Figure 12. SATURN is flexible and can be made in different shapes
sion to the receiving antenna, announcing the event. As and sizes, allowing instrumentation of everyday objects such as a
soda bottle, shirt, and paper crafts.
long as the event continues to occur, the SATURN system
can transmit alerts every few seconds to a remote moni-
toring station. In this manner, acoustic environmental

Figure 11. Tag consisting of simple circuitry—SATURN sensor

connected to printed antenna using a JFET and an impedance
matching circuit.

96 COMM UNICATIO NS O F THE ACM | D EC EM BER 2020 | VO L . 63 | NO. 1 2

acoustically. For example, placing several SATURN patches of large rooms. Thus, a SATURN array can be used as a remote
on a conference table can help localize a speaker to improve glass-break detector for security systems in office buildings.
speech quality (Figure 14). Outdoor arrays of SATURN
patches might sense loud sounds, such as a gunshot, and 5.5. Ambient monitoring of noise pollution
transmit the sound to indoor transceivers for real-time tri- Imagine an airport located in the center of a city, such as San
angulation (Figure 15). Jose International in California, which would like to moni-
tor its acoustic environment so as to not exceed safe noise
5.4. Industrial and home monitoring levels for its employees and to keep overly loud aircraft noise
In more industrial environments, SATURN can be used for footprints within airport boundaries (Figure 16). A SATURN-
acoustic failure monitoring and diagnostics in places where based system can be tiled on various buildings and at various
it is difficult or dangerous for humans to access. For example, distances on the runway. As planes take off, they generate
SATURN patches might be placed on turbines in a nuclear loud sounds due to engines, fans, and air turbulence. The
power plant to monitor them for vibrations that indicate dam- peak in the sound spectrum generated by aircraft is near the
age. Arranging SATURN patches in an array2 increases the 200–300 Hz band (Figure 45 in Khorrami et al.6) with deci-
range of audio sensing to almost 10 m allowing monitoring bel levels reaching 105 dBSPL at 5 m. These values are consis-
tent with the resonant frequency of the SATURN patch and
Figure 13. Examples of sounds sensed by SATURN. would result in generation of power >6.9 µW accumulated
10
–40 10 –40
over different frequency bands. This power could be used to
9

8
–50
8
–50 communicate the event by either of the two self-sustainable
7 –60 –60 systems described in Sections 4.2 and 4.3.
Frequency (kHz)

Frequency (kHz)

6 6

5 –70 –70 Other work zones that might monitor for sound thresh-
4
olds exceeding human hearing tolerance include construc-
4
–80 –80
3

tion zones, mines, music venues, power stations, airports,

2
–90 2
–90
1

0
1.8 2 2.2 2.4
Time (secs)
2.6 2.8 3
–100
0
11 12 13 14
–100
space-ports, and military environments. Similarly, SATURN-
Time (secs)

Whistle Music instrument notes based sensors might be used for monitoring catastrophic
10

9
–40 10

9
–40
events such as landslides, avalanches, polar ice calving,
8
–50
8
–50
mine caveins, and mine gas explosions.
7 7
–60 –60
Frequency (kHz)

Frequency (kHz)

6 6

5.6. Localization of tanks in war zones

5 –70 5 –70

4 4

In a more futuristic application, the United Nations might

–80 –80
3 3

2 2

drop SATURN-based sensors from an airplane into a con-

–90 –90
1 1

–100 –100
flict zone. The sensors would monitor the acoustic and
0 0
0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 2 2.5 3
Time (secs) Time (secs)

Taps Blows ground vibration environment for the movement of tanks,

heavy chemical transports, mortars, or exploding ordnance.
As tanks go by, the sound flips a bit in the sensor. Later, an
Figure 14. Multiple SATURN patches placed on a table can localize official with a reading device might sweep the field to inter-
speakers. rogate the sensors (Figure 17). In a more extreme scenario,
Voltage response SATURN microphone 2 Voltage response SATURN microphone 1
0.04 0.05

Figure 16. Recording a loud acoustic event using power generated

Speaker A Speaker B 0.04 Speaker A Speaker B
0.03
0.03
0.02
from a SATURN microphone.
0.02
Voltage(V)

Voltage(V)

0.01 0.01
0
0
–0.01
–0.01 speaker B speaker A
–0.02
–0.02 –0.03
SATURN SATURN
–0.04
–0.03
0 0.2 0.4 0.6 0.8 1 1.2 1.4
microphone 2 microphone 1 –0.05 WRITE READ AND RESET
Time(s) 0 0.2 0.4 0.6 0.8 1 1.2 1.4
Time(s)
non-volatile
Rectifier memory (NAND interrogate with
DC flash, SRAM) RF
Voltage
50% loss
6.9 micro Watt ∼ 3.4 micro Watt

Figure 15. Outdoor SATURN patches can sense and communicate

gunshot sounds to indoor transceivers for real-time triangulation.
Figure 17. Dropping remotely interrogable SATURN sensors to
monitor tanks and ordnance in a war zone.

Area and direction of events

0
0
0 0 0

0
1
0
1
0
1
1
1
0

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 97

research highlights

a low flying drone aircraft might sweep a strong RF signal Post-it Note-sized SATURN microphone is less than a cent,
over the region and record which sensors report hearing an but its manufacturing cost is still high due to the way we
event. The pattern of reporting sensors can reveal the direc- are depositing copper on paper and PTFE. When SATURN
tion of travel of a vehicle and point to possible hiding areas is placed in a self-sustainable computational scenario, the
for that equipment, providing proof for treaty violations. cost is driven higher depending on the active transistor com-
ponent being used. This expense suggests reexamining the
6. EMERGING RESEARCH THEMES manufacturing process such that it can be more efficiently
Work on SATURN suggests several research directions for scaled. How should the traditional semiconductor industry
computational materials in the future. We describe some of best support large-scale ubiquitous sensing? How can we
those themes here. change traditional manufacturing techniques to be able to
support applications where objects and surfaces have com-
6.1. Self-sustaining sensing opportunities putation embedded in it.
There are many other opportunities for designing self-­
sustaining sensors based on different energy harvesting 6.4. Designing the user experience
phenomena and for placing them in the context of self-­ Computational materials promise to further blur the divide
sustainable computational systems. SATURN is just one between the physical and digital worlds, opening a very
example; we can design many other TENG-based self-­ interesting set of research challenges for HCI and design
powered sensors in different form factors for varied appli- practitioners. How do we design user experiences around
cations. A photodiode or solar cell can be re-cast as a this new technology? What tools and design frameworks can
self-sustainable motion sensor whose rate of wireless be adapted for an Internet of Materials? When the form fac-
transmission is directly coupled to the amount of light tor of computing is more like the objects we use in crafting,
to which the sensor is exposed. Another example is to how does that change the way we think about designing user
re-think batteries, which generate power due to an elec- experiences? Indeed, as the playful art object in Figure 12
trochemical reaction, as sensors. Design changes to the suggests, computation that looks more like paper or other
cathode, anode, and electrolyte can make the cell rela- material inspires more creative uses by those familiar with
tively inert until exposed to a catalyst such as air, water, those materials. And computation that looks and feels more
or other chemicals, thus making the cell a self-powered like everyday objects will change the way we as humans
sensor for detecting (and communicating) the presence experience, understand, and build relationships with that
of the chemical catalyst. A third opportunity regards ther- technology. It also creates opportunities and challenges for
moelectric harvesters, which generate energy due to heat infusing computation with values, such as sustainability,
flow and temperature gradients. Imagine a steam pipe through the deliberate choice of materials used to create a
constructed out of the dissimilar metals typically used for computational effect.
thermoelectric generators. When steam flows through the
pipe, the temperature difference would cause energy to be 6.5. In-material privacy frameworks
generated which would power the wireless transmission of One of the biggest challenges and opportunities with ubiq-
packets. The higher the temperature difference, the more uitous computing systems is designing for privacy.7, 19
the power and the higher the rate of packet transmissions. Privacy-aware designs often focus on both technological
and social approaches.7, 12 What new approaches will the
6.2. Transitioning from devices to materials Internet of Materials era inspire? What privacy models will
Although such self-sustainable sensors can be made as need changing? One opportunity comes in the form of
individual macro units, current technology trends support the privacy principles of choice and consent and proxim-
a lower level integration of the sensing into the material ity and locality.7 Computational materials focus on local
itself. Advancements in printed and flexible electronics sensing, which may provide users a natural mental model
are enabling the production of self-sustainable sensors of their range of operation. For example, SATURN micro-
in thin and flexible form factors that can be conveniently phone patches could be constructed at a physical level
added to current materials. New research in flexible to require tapping with a finger to prime the backscat-
antennas, transistors, and integrated circuits9 demon- ter circuit before audio could be transmitted (e.g., when
strates how simple computation and communication can a speaker taps a microphone to ask “is this mic on?”).
be added although maintaining flexibility and low cost. Furthermore, the patch can be constructed to have a lim-
Finally, with radio backscatter technology and applica- ited sensing range for human voice.
tions improving rapidly,11, 20 a surprisingly low amount of Another way to design SATURN for privacy is to tune the
energy needs to be generated locally for communication resonant frequencies of the patch to focus on only certain
to the external infrastructure. types of sound or vibration. A SATURN patch for monitor-
ing an industrial machine might only listen to certain low-
6.3. Rethinking traditional semiconductor pitched hums, outside normal speech ranges, which indicate
manufacturing techniques upcoming failures. Similarly, a glass-break detector might be
In an ideal situation, we would like these paper microphones tuned for high pitches outside of human hearing ranges. It
to be cheap and disposable so we would not worry if they is our hope that by focusing on embedding “privacy in the
are lost or stolen. Currently, the bill of materials for a single material structure” as a first-class research priority, we will

98 COM MUNICATIO NS O F TH E AC M | D EC EM BER 2020 | VO L . 63 | NO. 1 2

also find ways of improving the function of the device along acoustic conditions. Appl. Acoust. 13. Talla, V., Hessar, M., Kellogg, B.,
114 (2016), 79–91. Najafi, A., Smith, J.R., Gollakota, S.
the often-related dimensions of power usage, networking, 5. Gilbert, N., Zhang, Y., Dinh, J., Calhoun, B., LoRa backscatter: Enabling the
and user interface. Hollmer, S. A 0.6 v 8 pJ/write vision of ubiquitous connectivity.
non-volatile CBRAM macro embedded In Proceedings of the ACM on
in a body sensor node for ultra low Interactive, Mobile, Wearable and
6.6. Building a community of multidisciplinary energy applications. In Proceedings of Ubiquitous Technologies (volume 1(3) )
the 2013 Symposium on VLSI Circuits (2017), 105.
researchers (VLSIC, 2013), IEEE, 204–205. 14. Wang, Z.L. Triboelectric
Building SATURN involves solving technical, system level, 6. Khorrami, M.R., Fares, E., Casalino, D. nanogenerators as new energy
Towards full aircraft airframe technology and self-powered
and design challenges that span many fields. Materials sci- noise prediction: Lattice Boltzmann sensors—Principles, problems and
ence is required to design SATURN patches; mechanical simulations. In 20th AIAA/CEAS perspectives. Faraday Discuss. 176
Aeroacoustics Conference (2014), 2481. (2015), 447–458.
engineering helps characterize the effect of vibration on a 7. Langheinrich, M. Privacy by design— 15. Wang, Z.L. On Maxwell’s displacement
patch; wireless, low-power electronics is necessary for build- Principles of privacy-aware ubiquitous current for energy and sensors: The
systems. In International Conference origin of nanogenerators. Mater.
ing self-sustainable communication; flexible electronics on Ubiquitous Computing (2001), Today 2, 20 (2017), 74–82.
Springer, 273–291. 16. Wang, Z.L., Chen, J., Lin, L. Progress in
are required for manufacturing the prototype; and design 8. Mohan, V., Bunker, T., Grupp, L., triboelectric nanogenerators as a new
and HCI knowledge help us explore applications in every- Gurumurthi, S., Stan, M.R., Swanson, S. energy technology and self-powered
Modeling power consumption sensors. Energy Environ. Sci 8, 8
day settings. Creating devices for an Internet of Materials of NAND flash memories using (2015), 2250–2282.
needs researchers who can adopt an aggressively multidis- flashpower. IEEE Trans. Comp. Aid. 17. Wang, Z.L., Wang, A.C. Triboelectric
Des. Integ. Circuits Sys. 7, 32 (2013), nanogenerator for self-powered
ciplinary mindset to collaborate and learn the language of 1031–1044. flexible electronics and internet of
many different fields. 9. Myny, K. The development of flexible things. In Meeting Abstracts (26)
integrated circuits based on thin-film (2018), The Electrochemical Society,
transistors. Nat. Electr. 1, 1 (2018), 1533–1533.
7. CONCLUSION 30–39. 18. Weiser, M. The computer for the 21st
10. Raikwal, P., Neema, V., Verma, A. century. Scientific American 3, 265
SATURN is an initial example of a computational material. High speed 8t SRAM cell design with (1991), 94–105.
It senses vibration and can convey that information wire- improved read stability at 180 nm 19. Weiser, M. Some computer science
technology. In 2017 International issues in ubiquitous computing.
lessly without batteries or wired power sources. Its simple Conference of Electronics, Commun. ACM 7, 36 (1993), 75–84.
multilayer construction resembles an everyday material; Communication and Aerospace 20. Zhang, Y., Iravantchi, Y., Jin, H.,
Technology (ICECA, volume 2) (2017), Kumar, S., Harrison, C. Sozu: Self-
it looks and feels like paper, yet it behaves like a wireless IEEE, 563–568. powered radio tags for building-scale
microphone. The multidisciplinary, out-of-the-box think- 11. Ranganathan, V., Gupta, S., Lester, J., activity sensing. In Proceedings of
Smith, J.R., Tan, D. RF Bandaid: A fully- the 32nd Annual ACM Symposium
ing that resulted in SATURN’s creation and application analog and passive wireless interface on User Interface Software
for wearable sensors. In Proceedings and Technology (2019), ACM,
is meant to inspire a new direction for computing. Using of the ACM on Interactive, Mobile, 973–985.
power (requiring no external power source), cost (large- Wearable and Ubiquitous Technologies 21. Zou, H., Zhang, Y., Guo, L., Wang, P.,
(volume 2(2) ) (2018), 79. He, X., Dai, G., Zheng, H., Chen, C.,
scale manufacturing with simple materials), and form 12. Richards, N.M. The dangers of Wang, A.C., Xu, C., et al. Quantifying
factor (looking more like an everyday object) as driving surveillance. Harvard Law Rev. 7, 126 the triboelectric series. Nat.
(2013), 1934–1965. Commun. 1, 10 (2019), 1427.
factors leads to a rethinking of ubiquitous computing that
can drive the community forward for the coming decades
in the same way that Weiser’s vision propelled us for the Nivedita Arora, Thad Starner, and Gregory D. Abowd ({nivedita.arora,thad,arbowd}@
gatech.edu), School of Interactive Computing, Georgia Institute of Technology, USA.
past few decades.

Acknowledgments
We would like to thank our collaborators at Georgia Tech’s
Nanotechnology Lab—Prof. Zhong Lin Wang, Yi-Cheng
Wang, Steven L. Zhang, and Zhengjun Wang—for their work
and guidance on the original version of the SATURN paper.3
We thank Diego Osorio and Fereshteh Shahmiri for their
help in making diagrams for the original SATURN paper,
some of which have been adopted in this review paper. We
thank Mohit Gupta for the device vibration simulations, and
we are grateful to Jin Yu and Hyunjoo Oh for their help in
making the teaser image for this paper.

References Conference on Mobile Systems,

1. Arora, N., Abowd, G.D. ZEUSSS: Zero Applications, and Services (2019),
energy ubiquitous sound sensing ACM, 543–544.
surface leveraging triboelectric 3. Arora, N., Zhang, S.L., Shahmiri, F.,
nanogenerator and analog backscatter Osorio, D., Wang, Y.-C., Gupta, M.,
communication. In The 31st Annual Wang, Z., Starner, T., Wang, Z.L.,
ACM Symposium on User Interface Abowd, G.D. SATURN: A thin
Software and Technology Adjunct and flexible self-powered
Proceedings (2018), ACM, 81–83. microphone leveraging triboelectric
2. Arora, N., Xue, Q., Bansal, D., nanogenerator. In Proceedings of the
McAughan, P., Bahr, R., Osorio, D., ACM on Interactive, Mobile, Wearable
Ma, X., Sample, A.P., Starner, T.E., and Ubiquitous Technologies 2, 2
Abowd, G.D. Surface++: A scalable (2018), 60.
and self-sustainable wireless sound 4. Galbrun, L., Kitapci, K. Speech
sensing surface. In Proceedings of intelligibility of English, Polish, Arabic
the 17th Annual International and Mandarin under different room © 2020 ACM 0001-0782/20/12 $15.00

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T HE ACM 99

CAREERS

Boston College detailed CV, and teaching and research state- Harvard John A. Paulson School of
Tenure Track Assistant Professor of Computer ments. Arrange for three confidential letters of Engineering and Applied Sciences
Science recommendation to be uploaded directly to In- Tenure-Track Faculty Positions in Computer
terfolio. To apply go to: https://apply.interfolio. Science and Theory of Quantum Information
The Computer Science Department of Boston com/79609. and Computation
College seeks a tenure-track Assistant Professor Boston College conducts background checks
beginning in the 2021-2022 academic year. Suc- as part of the hiring process. Information about The Harvard John A. Paulson School of Engineer-
cessful candidates for the position will be expect- the University and our department is available at ing and Applied Sciences (SEAS) seeks applicants
ed to develop strong research programs that can bc.edu and cs.bc.edu. for a tenure-track position in Computer Science
attract external funding in an environment that Boston College is a Jesuit, Catholic university at the assistant or associate professor level. Addi-
also values high-quality undergraduate teaching. that strives to integrate research excellence with tionally, the SEAS area of Computer Science and
Outstanding candidates in all areas of Computer a foundational commitment to formative liberal the Harvard Quantum Initiative seeks applicants
Science will be considered, with a preference for arts education. We encourage applications from for a tenure-track position in the area of the the-
those who demonstrate a potential to contribute candidates who are committed to fostering a di- ory of quantum information and/or computation
to cross-disciplinary teaching and research in con- verse and inclusive academic community. Boston at the assistant or associate professor level. Both
junction with the planned Schiller Institute for College is an Affirmative Action/Equal Opportu- have an expected start date of July 1, 2021.
Integrated Science and Society at Boston College. nity Employer and does not discriminate on the For the position in Computer Science, we in-
A Ph.D in Computer Science or a closely re- basis of any legally protected category including vite applications in all areas of Computer Science.
lated discipline is required. See cs.bc.edu and disability and protected veteran status. To learn Areas of special interest include (but are not lim-
https://www.bc.edu/bc-web/centers/schiller-in- more about how BC supports diversity and inclu- ited to) both machine learning and algorithms
stitute.html for more information. Application sion throughout the university, please visit the (broadly construed). For the position in the area
review is ongoing. Office for Institutional Diversity at http://www. of the theory of quantum information and/or
Applicants should submit a cover letter, a bc.edu/offices/diversity. computation, areas of interest may include but
are not limited to quantum algorithms, commu-
nication, complexity, control, cryptography, and
information processing.
Computer Science at Harvard benefits from
outstanding undergraduate and graduate stu-
dents, world-leading faculty, significant indus-
trial collaboration, and substantial support from
SEAS. Starting in Spring 2021, Computer Science
Department of Electrical and Computer Engineering
will be among the founding occupants of Har-
Graduate School of Engineering and Management vard’s new Science and Engineering Complex,
Air Force Institute of Technology (AFIT) with ample space for growth and surrounded by
state-of-the-art facilities. Information about Har-
Dayton, Ohio
vard’s current faculty, research, and educational
Faculty Position programs in computer science is available at
http://www.seas.harvard.edu/computer-science.
The Department of Electrical and Computer Engineering at the Air Force
The associated Institute for Applied Compu-
Institute of Technology is seeking applications for a tenured or tenure-
track faculty position. All academic ranks will be considered. Applicants tational Science (http://iacs.seas.harvard.edu),
must have an earned doctorate in Electrical Engineering, Computer Data Science Initiative (https://datascience.har-
Engineering, Computer Science, or a closely affiliated discipline by the vard.edu/), and Center for Research on Compu-
time of their appointment (anticipated 1 September 2021). tation and Society (http://crcs.seas.harvard.edu/)
We are particularly interested in applicants specializing in one or more foster connections among computer science, ap-
of the following areas: autonomy, artificial intelligence / machine learning, plied math, data science, and various domain sci-
navigation with or without GPS, cyber security, and VLSI. Candidates in ences at Harvard through its graduate programs
other areas of specialization are also encouraged to apply. This position
and events.
requires teaching at the graduate level as well as establishing and
sustaining a strong DoD relevant externally funded research program with We seek candidates who have a strong re-
a sustainable record of related peer-reviewed publications. search record and a commitment to undergradu-
The Air Force Institute of Technology (AFIT) is the premier Department of ate and graduate teaching and training. We par-
Defense (DoD) institution for graduate education in science, technology, ticularly encourage applications from historically
engineering, and management, and has a Carnegie Classification as a underrepresented groups, including women and
High Research Activity Doctoral University. The Department of Electrical minorities. A doctorate or terminal degree in a re-
and Computer Engineering offers accredited M.S. and Ph.D. degree lated field is required by the expected start date
programs in Electrical Engineering, Computer Engineering, and Computer
for each position.
Science as well as an MS degree program in Cyber Operations.
Required application documents include a
Applicants must be U.S. citizens. Full details on the position, the cover letter, CV, a statement of research interests,
department, applicant qualifications, and application procedures can be
found at http://www.afit.edu/ENG/. Review of applications will begin on a teaching statement, and up to three represen-
4 January 2021. The United States Air Force is an equal opportunity, tative papers. In addition, we ask for a statement
affirmative action employer. describing efforts to encourage diversity, inclu-
sion, and belonging, including past, current, and

100 CO MM UNICATIO NS O F T H E AC M | D EC EM BER 2020 | VO L . 63 | N O. 1 2

anticipated future contributions in these areas. working on computational complexity theory Southern University of Science and
Candidates are also required to submit the names but outstanding applicants in all areas of TCS Technology (SUSTech)
and contact information for at least three and up will be considered. Consistent with the aims Faculty Positions in Computer Science and
to five references, and the application is complete of the Simons Junior Faculty Fellows program, Engineering
only when three letters have been submitted. At which provides partial funding, the department
least one letter must come from someone who also welcomes applicants who are most affected The Department of Computer Science and Engi-
has not served as the candidate’s undergraduate, by the COVID-19 pandemic: postdocs and new neering (CSE, http://cse.sustc.edu.cn/en/), South-
graduate, or postdoctoral advisor. We encourage PhDs. ern University of Science and Technology (SUS-
candidates to apply by December 15, 2020, but will The appointment will start September 1, Tech) has multiple Tenure-track faculty openings
continue to review applications until the position 2021. Responsibilities include research in the at all ranks. We are looking for outstanding candi-
is filled. Applicants for the tenure-track position area of Theoretical Computer Science, supervi- dates with demonstrated research achievements
in Computer Science can apply online at http:// sion of PhD students, and teaching undergradu- and keen interest in teaching, in the following
academicpositions.harvard.edu/postings/9834. ate and graduate level courses in Computer Sci- areas (but are not limited to):
Applicants for the tenure-track position in the ence. Pursuit of external research funding is ˲ Data Science
area of the theory of quantum information and/or expected. ˲ Artificial Intelligence
computation can apply online at http://academic- Qualifications: Successful completion of a ˲ Computer Systems
positions.harvard.edu/postings/9835. PhD or equivalent in Computer Science or a close- ˲ Software Engineering (senior positions only)
We are an equal opportunity employer, and ly related field is required by the start date. ˲ Cognitive Robotics and Autonomous Systems
all qualified applicants will receive consideration Application Instructions: Applicants should ˲ Programming Languages and Compilers
for employment without regard to race, color, submit their CV, a research statement addressing Applicants should have an earned Ph.D. de-
religion, sex, national origin, disability status, both past and future work, a teaching statement, gree and demonstrated achievements in both
protected veteran status, gender identity, sexual and contact information for three references at research and teaching. The teaching language at
orientation, pregnancy and pregnancy-related http://jobs.rutgers.edu/postings/120527. Appli- SUSTech is bilingual, either English or Putong-
conditions or any other characteristic protected cations received by January 15, 2021 will be given hua. It is perfectly acceptable to use English in all
by law. priority. lectures, assignments, exams. In fact, our exist-
For questions, contact: martin@farach- ing faculty members include several non-Chinese
colton.com. speaking professors.
Rutgers University The CS Department is strongly committed The Department of Computer Science and
Tenure Track Assistant Professor in Theoretical to increasing the diversity of our faculty and wel- Engineering at SUSTech was founded in 2016.
Computer Science comes applications from women, dual-career It has 27 tenured or tenure-track professors, all
couples, historically underrepresented popula- of whom hold doctoral degrees or have years of
The Computer Science Department at Rutgers tions and candidates with disabilities. Offer is experience in overseas universities. 24 of the 27
University invites applications for a Tenure- contingent upon successful completion of all pre- were recruited from outside the mainland China.
Track Assistant Professor position in Theoreti- employment screenings. Rutgers University is an Among them, three are IEEE fellows, two were
cal Computer Science. We welcome candidates affirmative action/equal opportunity employer. editors-in-chief of IEEE journals. The department

Faculty Position in Computer and Communication Sciences

at the Ecole polytechnique fédérale de Lausanne (EPFL)

The School of Computer and Communication Sciences (IC) at EPFL You will be required to submit in PDF form a cover letter, a curricu-
invites applications for tenure-track faculty positions in all areas of lum vitae including a publication list, brief statements of research
computer and communication sciences. Some areas of particular and teaching interests, and contact information (name, postal
interest this year include unconventional computing (e.g., applied address, and email) of 3 references for junior positions and at least
quantum computing, DNA computing), programming languages 5 for senior positions
and verification, and intelligent systems. Screening will start on December 1, 2020. Further questions can
Senior faculty appointments may be possible. be addressed to:
We seek candidates with an outstanding academic record and a Prof. George Candea
strong commitment to teaching and mentoring students. Chair of the Faculty Recruiting Committee
EPFL offers its faculty excellent students from all over the world, School of Computer and Communication Sciences
competitive salaries, generous research support, and outstanding [email protected]
research infrastructure. Switzerland has an exceptionally high human For additional information on EPFL and IC, please visit: www.epfl.ch
development index and is consistently ranked top in economic com- or https://ic.epfl.ch
petitiveness and innovation.
EPFL is an equal opportunity employer and family friendly uni-
To apply, follow the application procedure at versity. It is committed to increasing the diversity of its faculty. It
https://facultyrecruiting.epfl.ch/position/23691281 strongly encourages women to apply.

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T H E ACM 101

CAREERS

is expected to grow to 50 tenured or tenure-track University of Michigan College of world’s leading research universities, consisting
faculty members eventually, in addition to teach- Engineering of highly ranked departments and colleges across
ing-only professors and research-only professors. Multiple Tenure-Track and Teaching Faculty engineering, sciences, medicine, law, business,
In 2019, the department has secured external (Lecturer) Positions and the arts, with a commitment to interdisci-
grants of RMB123.5 million (approx USD18 mil- plinary collaboration. CSE is a vibrant and inno-
lion). Computer Science and Engineering (CSE) at the vative community, with over 70 world-class fac-
SUSTech is a pioneer in higher education University of Michigan College of Engineering ulty members, over 300 graduate students, and
reform in China. The mission of the University invites applications for multiple tenure-track and a large and illustrious network of alumni. Ann
is to become a globally recognized research uni- teaching faculty (lecturer) positions, as part of its Arbor is known as one of the best small cities in
versity which emphasizes academic excellence aggressive long-term growth plan. We seek excep- the nation, and the University has a strong dual-
and promotes innovation, creativity and entre- tional candidates in all areas across computer sci- career assistance program.
preneurship. Set on five hundred acres of wooded ence and computer engineering, with special em- Michigan Engineering’s vision is to be the
landscape in the picturesque Nanshan (South phasis on candidates at the early stages of their world’s preeminent college of engineering serv-
Mountain) area, the campus offers an ideal envi- careers; we also have a targeted search for an en- ing the common good. This global outlook, lead-
ronment for learning and research. dowed professorship in theoretical computer sci- ership focus, and service commitment permeate
SUSTech is committed to increase the diversi- ence (the Fischer Chair). our culture. Our vision is supported by our mis-
ty of its faculty, and has a range of family-friendly Qualifications include an outstanding aca- sion and values that, together, provide the frame-
policies in place. The university offers competi- demic record; an awarded or expected doctorate work for all that we do. Information about our vi-
tive salaries and fringe benefits including medi- (or equivalent) in computer science, computer sion, mission and values can be found at http://
cal insurance, retirement and housing subsidies, engineering, or a related area; and a strong com- strategicvision.engin.umich.edu/.
which are among the best in China. Salary and mitment to teaching and research. We seek facul- The University of Michigan has a storied
rank will commensurate with qualifications and ty members who commit to excellence in gradu- legacy of commitment to Diversity, Equity and
experience. More information can be found at ate and undergraduate education, will develop Inclusion (DEI). The Michigan Engineering com-
http://talent.sustech.edu.cn/. impactful, productive and novel research pro- ponent of the University’s comprehensive, five-
We provide some of the best start-up packag- grams, and will contribute to the department’s year, DEI strategic plan—with updates on our
es in the sector to our faculty members, including goal of eliminating systemic racism and sexism programs and resources dedicated to ensuring a
one PhD studentship per year, in addition to a sig- by embracing our culture of Diversity, Equity and welcoming, fair, and inclusive environment—can
nificant amount of start-up grant (which can be Inclusion. be found at: http://www.engin.umich.edu/col-
used to fund additional PhD students and post- We encourage candidates to apply as soon as lege/about/diversity.
docs, research travels, and research equipment). possible. Positions remain open until filled and The University of Michigan is an equal oppor-
To apply, please provide a cover letter identify- applications can be submitted throughout the tunity/affirmative action employer and is respon-
ing the primary areas of your research and listing year. For more details on these positions and to sive to the needs of dual-career families.
your five best publications, curriculum vitae, and apply, please visit https://cse.engin.umich.edu/
research and teaching statements, and forward about/faculty-hiring/.
them to [email protected]. The University of Michigan is one of the

MULTIPLE FACULTY POSITIONS

Department of Electrical and
Systems Engineering
The School of Engineering and Applied Science at the University of Pennsylvania is growing its ADVERTISING
faculty by 33% over a five-year period. As part of this initiative, the Department of Electrical and
Systems Engineering is engaged in an aggressive, multi-year hiring effort for multiple tenure-track
IN CAREER
positions at all levels. Candidates must hold a Ph.D. in Electrical Engineering, Computer Engineering,
Systems Engineering, or related area. The department seeks individuals with exceptional promise
OPPORTUNITIES
for, or proven record of, research achievement, who will take a position of international leadership in
defining their field of study and who will excel in undergraduate and graduate education. Leadership How to Submit a Classified Line Ad: Send an e-mail to
in cross-disciplinary and multi-disciplinary collaborations is of particular interest. We are interested [email protected]. Please include text, and indicate
in candidates in all areas that enhance our research strengths in: the issue/or issues where the ad will appear, and a contact
1. Nanodevices and nanosystems (nanoelectronics, MEMS/NEMS, power electronics, name and number.
nanophotonics, nanomagnetics, quantum devices, integrated devices and systems at
nanoscale); Estimates: An insertion order will then be e-mailed back to
2. Circuits and computer engineering (analog, RF, mm-wave, digital circuits, emerging you. The ad will by typeset according to CACM guidelines.
circuit design, computer engineering, IoT, beyond 5G, and cyber-physical systems);
NO PROOFS can be sent. Classified line ads are NOT
3. Information and decision systems (control, optimization, robotics, data science,
machine learning, communications, networking, information theory, signal processing).
commissionable.
Prospective candidates in all areas are strongly encouraged to address large-scale societal Deadlines: 20th of the month/2 months prior to issue date.
problems in energy, transportation, health, agriculture, food and water, economic and financial For latest deadline info, please contact:
networks, social networks, critical infrastructure, and national security. We are especially
interested in candidates whose interests are aligned with the school’s strategic plan, [email protected]
https://www.seas.upenn.edu/about/penn-engineering-2020/ Career Opportunities Online: Classified and recruitment
Diversity candidates are strongly encouraged to apply. Interested persons should submit an online
display ads receive a free duplicate listing on our website at:
application at https://www.ese.upenn.edu/faculty-staff/ and include curriculum vitae, research,
teaching, and diversity statements, and at least three references. Review of applications will http://jobs.acm.org
begin on January 4, 2021.
The University of Pennsylvania values diversity and seeks talented students, faculty and staff from diverse Ads are listed for a period of 30 days.
backgrounds. The University of Pennsylvania is an equal opportunity and affirmative action employer. Candidates For More Information Contact:
are considered for employment without regard to race, color, sex, sexual orientation, gender identity, religion,
creed, national or ethnic origin, citizenship status, age, disability, veteran status or any other legally protected ACM Media Sales
class. Questions or concerns about this should be directed to the Executive Director of the Office of Affirmative at 212-626-0686 or
Action and Equal Opportunity Programs, University of Pennsylvania, 421 Franklin Building, 3451 Walnut Street,
Philadelphia, PA 19104-6205; or (215) 898-6993 (Voice) or (215) 898-7803 (TDD).
[email protected]

102 COMM UNICATIO NS O F T H E ACM | D EC EM BER 2020 | VO L . 63 | N O. 1 2

 last byte

[ C ONTI N U E D FRO M P. 104] extra minus own personal relationships decided by

sign in the program, purely by acci- his crazy brother, or even a democratic
dent, so we adopted counterclockwise Walden Three had vote of all members of this group of los-
as our norm. We made a few thousand an internal virtual ers. However, he shuddered in realiza-
bolts with the same program, then re- tion that he also was a loser, because
alized they would not work in ordinary currency the the Mars colonization effort had failed.
nuts, so we updated the nut-milling Goldmark, named for He thought about the other 22 survi-
program. Soon we built our clocks with vors, a number that could be prime fac-
number circle and hand movements Peter Goldmark, who tored into 2 times 11, or 11 married cou-
going the other way, and began using predicted in 1972 that ples. One of them he personally cared
our left arms when shaking human very much about, Catniss, might be
hands. As the British can testify, driv- something like the his partner if she joined Walden Three
ing cars on the proverbially wrong side Internet would render with him. Rather than strengthening
of the road is exactly as good, and dis- their relationship, living together in
tinguishes their culture from the rest cities irrelevent. the tiny spaceship crew compartment
of the world.” for months had driven them apart. Sur-
Frederic went on to explain all very viving on the hostile Martian surface
technical skills were programmed into had required all 41 colonists to func-
the machinery, so human workers could tion like gears in a mechanical system,
shift from job to job, gaining new expe- rather than as human beings sharing
riences and avoiding drudgery. For ex- cause the outside governments want to personal relationships. They returned
ample, each month, one member of the impose real estate, income, and sales to Earth exhausted in every way.
community would be selected to make taxes on us, as they greedily are chang- Rather than reunite with Catniss, Bur-
the regular deliveries of milk and ice, ing the laws concerning volunteering rhus wondered if he should recruit some
and would temporarily take the name and barter organizations. I don’t know of the Mars survivors to help Walden
Hickman, derived poetically from Eu- how we can survive. The great com- Three, whether or not they joined it.
gene O’Neill’s play, The Iceman Cometh, munes of the 19th century all produced Perhaps starting with one would be a
about the human need for extreme salable agriculture. Oneida made ani- proper experiment, and he immediately
hope. Burrhus asked why ice needed to mal traps, silk garments, and eventu- thought of three from which to select the
be delivered, and Frederic explained, ally became a silverware company. The initial research subject:
“We have iceboxes in our homes, rather Shakers made furniture and beautiful 1. Catniss Rockefeller had been the
than electric refrigerators. Every winter, wooden boxes. The Amana communi- chief of Mars logistics and was trained
our robots cut the ice off the top of our ties began manufacturing refrigera- in economics; she could find a way to
lake, filling the ice barn and covering tors, of all things! But we have nothing turn big profits trading the Goldmark
with sawdust to prevent the huge pile to sell.” Burrhus expressed sympathy, currency to pay the taxes, or marketing
from melting during the warm months. but tended to think the governments unconventional picnic tables.
The Hickman also delivers milk and oth- were right, given they protected the 2. The French international coor-
er liquids in big reusable bottles, as was commune and provided a range of im- dinator for the expedition, Paris Ma-
common a century ago.” plicit services and social insurance. son, had a law degree, so potentially
They then talked about the economy, At suppertime they sat at a picnic he could delay the taxes indefinitely
and Burrhus learned Walden Three had table, surrounded by all members through constant litigation in court.
an internal virtual currency called Gold- of Walden Three, as they sang, “‘Tis 3. Communication programmer
marks. The name came from the in- the gift to be simple, ‘tis the gift to Thomas Sanderson could put his hack-
ventor of the long-playing phonograph be free... in the valley of love and de- er skills to a new use: threatening the
record, Peter Goldmark, who in a 1972 light.” Frederic explained that was the external economy by erasing bank ac-
government report and Scientific Ameri- Shaker theme song, especially relevant counts to make governments relent.
can article had predicted something because his community was strug- Which one should he recruit to this
like the Internet would render cities ir- gling to decide its policy about fam- challenging cause, if any?
relevant, and allow people to live hap- ily relationships. The Shakers never Then Burrhus noted that the num-
pily again in small towns. Goldmarks divided into couples and produced no bers 1, 2, 3 were all primes, and there
essentially were the same as hours in children. Then a second song began: was another prime in their sequence
time banking, paying people equally, “We will build us a dome on our beau- that was more realistic: zero.
and totally separated from the dollars of tiful plantation, and we’ll all have one
the surrounding economy. Because resi- home, and one family relation.” That William Sims Bainbridge ([email protected])
is a sociologist who taught classes on crime and deviant
dents of Walden Three earned no dol- was the theme song of Oneida, where behavior at respectable universities before morphing
all members were married to all oth- into a computer scientist, editing an encyclopedia of
lars, they paid no income tax. human-computer interaction, writing many books on
Frederic had been smiling and lec- ers, and many children were born ac- things computational, from neural nets to virtual worlds
to personality capture, then repenting and writing
turing his brother excitedly, but then cording to spiritual eugenic rules im- harmless fiction.
he became gloomy. “Burr, our commu- posed by its supreme leader. Burrhus
nity is really in trouble right now, be- decided he definitely did not want his © 2020 ACM 0001-0782/20/12 $15.00

DEC E MB E R 2 0 2 0 | VO L. 6 3 | N O. 1 2 | C OM M U N IC AT ION S OF T H E ACM 103

last byte

From the intersection of computational science and technological speculation,

with boundaries limited only by our ability to imagine what could be.

DOI:10.1145/3427948 William Sims Bainbridge

Future Tense
Walden Three
Can humanity take its next step forward by taking a step back?

Bur-
W H E N N AV I G AT I O N P RO G R A M M E R
rhus Skinner returned from NASA’s
aborted three-year attempt to colonize
the planet Mars, he faced a difficult
transition. The plan had been to send
a fleet of spaceships carrying supplies,
equipment, and 41 people to estab-
lish the beginning of a socially and
biologically perfect society. Everyone
agreed to the same set of social norms,
and humans would be the only living
species on the red planet. Of course
there could be no birds, because the
atmosphere was too thin for flying,
but also no mammals, reptiles, bac-
teria or viruses, so infectious diseases
would be impossible. But equipment
kept breaking down, in the worst case decide upon a strict set of norms, then to hold the parts of a table together were
depriving an outpost with 13 unlucky use behavioral psychology to ensure automatically milled in the workshop to
people of electric power; five did not that all members of the group followed the West. Come, I was about to assemble
survive the crash of a transport, and them. In 1974, Kathleen Kinkade pub- one, and you can do it to see how much
the prime number who returned to lished A Walden Two Experiment about you would enjoy joining us!”
Earth was 23. For several weeks they a very real community she helped found The many wooden boards for a
waited in quarantine, as germs were in 1967 that continues to thrive. picnic table were already shaped and
gently returned to their digestive sys- On the basis of its website, given Bur- drilled for assembly with screws, but
tems so they could survive the biologi- rhus had not talked with Frederic for Burrhus was amazed to see that each
cal complexity of their home world. many years, he gathered the principle screw was as fat as his thumb. Fred-
This gave Burrhus time to ponder added by Walden Three was distributed eric explained that the computer mill-
what he would do next. He had not manufacturing, in which AI-controlled ing machine in the opposite log cabin
abandoned his utopian ambitions, so he 3D printing and milling machines could could not handle iron or steel, so they
naturally thought of the Walden Three help workshops in the community pro- made all metal components by re-
IMAGE BY AND RIJ BORYS ASSOCIAT ES, USING SH UTT ERSTOC K

experimental community set up by his duce most of the products it needed. cycling copper pipes and aluminum
crazy brother, Frederic Skinner. In prep- When Burrhus arrived at Walden trash, which required screws to be big
aration for visiting, he read the 1948 nov- Three, he found a few decrepit old hous- to be strong enough. Burrhus com-
el Walden Two written by their psycholo- es and a set of four new log cabins ar- plained, “Oh, look Fred, these screws
gist ancestor, B. F. Skinner. Then he read ranged in the shape of a plus sign around your robot miller made don’t work!
the 1854 memoire, Walden, by Henry some picnic tables where workers could Using my screwdriver, I turn and turn,
David Thoreau, that was the origin of socialize over communal meals. In his and they don’t go in!”
the community’s name. Walden had em- habitually over-emotional voice, Fred- “Oh, Burr, sorry about that mis-
phasized the human experience of unity eric explained, “We cut the wood parts communication,” Frederic said. “The
with nature during social isolation. of these tables and all other furniture in screws work fine if you turn them coun-
Walden Two had argued that a group of the pine log workshop to the East, and terclockwise rather than clockwise.
like-minded people could voluntarily mill metal components like the screws There was an [C O NTINUED O N P. 103]

104 COMM UNICATIO NS O F T H E AC M | D EC EM BER 2020 | VO L . 63 | N O. 1 2

 ipdps.org

ANNOUNCING 19 IPDPS 2021 WORKSHOPS

IPDPS Workshops are the “bookends” to the three-day conference GENERAL CO-CHAIRS
technical program of contributed papers, invited speakers, student David Bader (New Jersey Institute of Technology, USA)
programs, and industry participation. They provide the IPDPS community Aparna Chandramowlishwaran (University of California, Irvine, USA)
an opportunity to explore special topics and present work that is more
preliminary or cutting-edge than the more mature research presented in PROGRAM CHAIR
the main symposium. Each workshop has its own website and submission
Karen Karavanic (Portland State University, USA)
requirements, and the submission deadline for most workshops is after
the main conference author notification dates.
WORKSHOPS CHAIR and VICE-CHAIR
IPDPS WORKSHOPS MONDAY 17 May 2021 Erik Saule (University of North Carolina Charlotte, USA)
HCW Heterogeneity in Computing Workshop Jaroslaw Zola (The State University of New York at Buffalo, USA)
RAW Reconfigurable Architectures Workshop
HiCOMB High Performance Computational Biology STUDENT PROGRAM
GrAPL Graphs, Architectures, Programming, and Learning The conference plans to hold the traditional PhD forum of poster
EduPar NSF/TCPP Workshop on Parallel and Distributed Computing presentations, to provide mentoring in scientific writing and
Education presentation skills, and to create opportunities for students to hear
HIPS High-level Parallel Programming Models and Supportive from and interact with senior researchers attending the conference.
Environments
AsHES Accelerators and Hybrid Emerging Systems INDUSTRY PARTICIPATION
PDCO Parallel / Distributed Combinatorics and Optimization There are several ways for industry to partner with IPDPS and share
APDCM Advances in Parallel and Distributed Computational Models the benefits of associating with our international community of top
researchers and practitioners in fields related to parallel processing
IPDPS WORKSHOPS FRIDAY 21 MAY 2021 and distributed computing. IPDPS is a “walk-up-and-talk” venue
JSSPP Job Scheduling Strategies for Parallel Processing that encourages industry partners to use the conference as a way to
PDSEC Parallel and Distributed Scientific and Engineering Computing introduce their technology in an informal setting.
iWAPT Automatic Performance Tuning
MPP Parallel Programming Models - Emerging Technologies on IMPORTANT DATES
Machine Learning Acceleration
SNACS Scalable Networks for Advanced Computing Systems Conference Preliminary Author Notification – Dec. 8, 2020
PAISE Parallel AI and Systems for the Edge Conference Final Author Notification – Jan. 19, 2021
RADR Resource Arbitration for Dynamic Runtimes
Workshops’ Call for Papers Deadlines – Most Fall in
ScaDL Scalable Deep Learning over Parallel And Distributed
Late Dec. and Jan.
Infrastructures
HPS High-Performance Storage
ParSocial Parallel and Distributed Processing for Computational IPDPS 2021 VENUE & PROGRAM
Social Systems Portland, Oregon sits on the Columbia and Willamette rivers in the
shadow of snow-capped Mount Hood and is known for its parks, roses and
rhododendrons, bridges and bicycle paths, as well as for its eco-friendliness.
Planning for the 35th edition of IPDPS will be informed by experience
in 2020 and collaboration with the IPDPS community to continue the
programming that promotes international cooperation in seeking to
apply computer science technology to the betterment of our global
village. Whether presented in person or virtually, every paper accepted
for IPDPS 2021 will be published in the proceedings, and valuable
interaction whatever the platform will be assured.
In-depth.
Innovative.
Insightful.
Inspired by the need for high-quality
computer science publishing at the
graduate, faculty, and professional
levels, ACM Books are affordable,
current, and comprehensive in scope.

Full Collection I Title List

Now Available
For more information, please visit
http://books.acm.org

M
&C
Association for Computing Machinery
1601 Broadway, 10th Floor, New York, NY 10019-7434, USA
Phone: +1-212-626-0658 Email: [email protected]