Handbook of Collective Intelligence

Handbook of Collective Intelligence

Handbook of Collective Intelligence

Thomas W. Malone and Michael S. Bernstein, editors

The MIT Press

Cambridge, Massachusetts

London, England

© 2015 Massachusetts Institute of Technology

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

Cataloging-in-Publication information is available from the Library of Congress.

ISBN: 978-0-262-02981-0

Epub 3.0

For Virginia. —T.M.

For Marvin, Ethel, Leonard, and Sylvia. —M.B.

Contents

Acknowledgments

Introduction

Thomas W. Malone and Michael S. Bernstein

Economics

Editors’ Introduction

The Wisdom of Crowds vs. the Madness of Mobs

Andrew W. Lo

Biology

Editors’ Introduction

Collective Behavior in Animals: An Ecological Perspective

Deborah M. Gordon

Human–Computer Interaction

Editors’ Introduction

Human–Computer Interaction and Collective Intelligence

Jeffrey P. Bigham, Michael S. Bernstein, and Eytan Adar

Artificial Intelligence

Editors’ Introduction

Artificial Intelligence and Collective Intelligence

Daniel S. Weld, Mausam, Christopher H. Lin, and Jonathan Bragg

Cognitive Psychology

Editors’ Introduction

Cognition and Collective Intelligence

Mark Steyvers and Brent Miller

Organizational Behavior

Editors’ Introduction

Collective Intelligence in Teams and Organizations

Anita Williams Woolley, Ishani Aggarwal, and Thomas W. Malone

Law, Communications, Sociology, Political Science, and Anthropology

Editors’ Introduction

Peer Production: A Form of Collective Intelligence

Yochai Benkler, Aaron Shaw, and Benjamin Mako Hill

Conclusion

Thomas W. Malone

List of Contributors

Index

d_r0

Acknowledgments

We are grateful to the MIT Center for Collective Intelligence, the MIT Sloan School of Management, and the Stanford Computer Science Department for support during the preparation of this book, and to the National Science Foundation for support of the first two Collective Intelligence Conferences, held in 2012 and 2014, which helped make the field this book describes a reality (NSF grant number IIS-1047567). In addition, Malone’s work on this volume was supported in part by grants from the National Science Foundation (grant numbers IIS-0963285, ACI-1322254, and IIS-0963451), and the U.S. Army Research Office (grant numbers 56692-MA and 64079-NS). Bernstein’s work on this volume was supported in part by a grant from the National Science Foundation (IIS-1351131).

We are also grateful to Richard Hill for excellent administrative assistance in many stages of this work, to Lisa Jing for bibliographic research used in the introduction, and to the members of the online crowd who commented on earlier versions of the chapters.

Introduction

Thomas W. Malone and Michael S. Bernstein

In nine hours, a team successfully scoured the entire United States to find a set of red balloons that was worth $40,000 (Pickard et al. 2011). In three weeks, citizen scientists playing a game uncovered the structure of an enzyme that had eluded scientists for more than fifteen years (Khatib et al. 2011). In ten years, millions of people authored the most expansive encyclopedia in human history. If interconnected people and computers can accomplish these goals in hours, days, and years, what might be possible in the next few years, or the next ten?

This book takes the perspective that intelligence is not just something that arises inside individual brains—it also arises in groups of individuals. We call this collective intelligence: groups of individuals acting collectively in ways that seem intelligent (Malone, Laubacher, and Dellarocas 2009). By this definition, collective intelligence has existed for a v(ry long time. Families, armies, countries, and companies have all—at least sometimes—acted collectively in ways that seem intelligent. And researchers in many fields—from economics to political science to psychology—have studied these different forms of collective intelligence.

But in the last two decades a new kind of collective intelligence has emerged: interconnected groups of people and computers, collectively doing intelligent things. For example, Google harvests knowledge generated by millions of people creating and linking Web pages and then uses that knowledge to answer queries in ways that often seem amazingly intelligent. In Wikipedia, thousands of people around the world have collectively created a large intellectual product of high quality with almost no centralized control and with mostly volunteer participants. And in more and more domains, surprisingly large groups of people and computers are writing software (Lakhani, Garvin, and Lonstein 2010; Benkler 2002), solving engineering problems (Lakhani and Lonstein 2011), composing and editing documents (Kittur, Smus, Khamkar, and Kraut 2011; Bernstein et al. 2010), and predicting presidential elections (Berg, Forsythe, Nelson, and Rietz 2008).

These early examples, we believe, are not the end of the story but just the beginning. And in order to understand the possibilities and constraints of these new kinds of intelligence, we need a new interdisciplinary field. Such a field can help in the exploitation of the often-unrecognized synergies among disciplines that have studied various forms of collective intelligence without realizing their commonalities, and can develop new knowledge that is specifically focused on understanding and creating these new kinds of intelligence. Helping to form such a field is the primary goal of this volume.

Defining Collective Intelligence

As with many important but evocative terms, there have been almost as many definitions of collective intelligence as there have been writers who have described it (see the appendix to this introduction for a representative list). For instance, Hiltz and Turoff (1978) define collective intelligence as a collective decision capability [that is] at least as good as or better than any single member of the group. Smith (1994) defines it as a group of human beings [carrying] out a task as if the group, itself, were a coherent, intelligent organism working with one mind, rather than a collection of independent agents. And Levy (1994) defines it as "a form of universally distributed intelligence, constantly enhanced, coordinated in real time, and resulting in the effective mobilization of skills."

Of course, intelligence itself can be defined in many different ways. Sometimes it is defined in terms of specific processes—for example, intelligence is a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience (Gottfredson 1997). Another common way of defining intelligence is in terms of goals and the environment—for example, in 2006 the Encyclopaedia Britannica defined it as the ability to adapt effectively to the environment, and in 1983 Howard Gardner defined it as the ability to solve problems, or to create products, that are valued within one or more cultural settings. The most common operational definition of intelligence in psychology is as a statistical factor that measures a person’s ability to perform well on a wide range of very different cognitive tasks (Spearman 1904). This factor (often called general intelligence or g) is essentially what is measured by IQ tests. There is even controversy about whether it would be legitimate to call behavior intelligent, no matter how intelligent it seemed, if it were produced by a computer rather than a person (see, e.g., Searle 1999).

In view of all this complexity, our definition of collective intelligence, as given above, is a simple one: groups of individuals acting collectively in ways that seem intelligent. Several aspects of this definition are noteworthy:

• The definition does not try to define intelligence. There are so many ways to define it, and we do not want to prematurely constrain what we believe to be an emerging area of study. Our definition is, therefore, compatible with all of the above definitions of intelligence.

• By using the word acting, the definition requires intelligence to be manifested in some kind of behavior. By this definition, for instance, the knowledge represented in a collection such as Wikipedia would not, in itself, be considered intelligent, but the group of people who created the collection could be.

• The definition requires that, in order to analyze something as collective intelligence, one be able to identify some group of individuals who are involved. In some cases this may be straightforward, such as noting the individual humans in an organization; in other cases, it may be useful to draw these boundaries in unusual ways. For instance, if one regards the whole brain as a group of individual neurons or brain regions, one may analyze the operation of a single human brain as collective intelligence. Or one may analyze the collective intelligence of a whole economy by noting that the economy is a collection of many different organizations and people.

• The definition requires that the individuals act collectively—that is, that there be some relationships among their activities. We certainly do not intend this to mean that they must all share the same goals or that they must always cooperate. We merely mean that their activities are not completely independent—that there are some interdependencies among them (see, e.g., Malone and Crowston 1994). For instance, different actors in a market may buy and sell things to one another even though they may each have very different individual goals. And different problem solvers in an open-innovation community such as InnoCentive compete to develop the best solutions to a problem.

• By using the word seem, the definition makes clear that what is considered intelligent depends on the perspective of the observer. For instance, to evaluate whether an entity is acting intelligently an observer may have to make assumptions about what the entity’s goals are. IQ tests, for example, do not measure intelligence well if the person taking the test is only trying to annoy the person giving the test. Or an observer may choose to analyze how intelligently a group of Twitter users filters information even if none of the individual users have that goal.

How Does Collective Intelligence Relate to Other Fields?

In establishing an interdisciplinary field such as collective intelligence, it is useful to indicate how the new field overlaps with and differs from existing fields. In that spirit, we suggest the following loose guidelines for thinking about how collective intelligence relates to several existing fields.

Computer Science

Collective intelligence overlaps with the subset of computer science that involves intelligent behavior by groups of people, computers, or both. For instance, groups of one person and one computer (human–computer interaction) can be viewed as a kind of collective intelligence, and studying larger groups of people and computers (e.g., human computation, crowdsourcing, social software, computer-supported cooperative work, groupware, collaboration technology) is at the heart of the field. Similarly, studies of how groups of artificially intelligent agents can exhibit intelligent behavior together can be very relevant for collective intelligence, but studies that don’t focus on how different processing units work together probably are not.

Cognitive Science

Cognitive science focuses on understanding the nature of the human mind, including many aspects of mental functioning that may be regarded as components of intelligent behavior (such as perception, language, memory, and reasoning). Collective intelligence overlaps with cognitive science only when there is an explicit focus on how intelligent behavior arises from groups of individuals. Most obviously, this occurs with groups of people (as in group memory, group problem solving, and organizational learning), but, as was noted above, studies of how different parts of a single brain interact to produce intelligent behavior can also be part of collective intelligence.

Sociology, Political Science, Economics, Social Psychology, Anthropology, Organization Theory, Law

These fields all study the behavior of groups. They overlap with collective intelligence only when there is a focus on overall collective behavior that can be regarded as more or less intelligent. For instance, analyzing how individual people’s attitudes are determined or how they make economic choices would not be central to collective intelligence, but analyzing how different regulatory mechanisms in markets lead to more or less intelligent behavior by the markets as a whole would be central to collective intelligence. Similarly, analyzing how different organizational designs in a company lead to better or worse performance by the company as a whole would also be central to collective intelligence. And so would analyzing how well democratic governments make decisions and solve problems.

Biology

Collective intelligence overlaps with the parts of biology that focus explicitly on group behavior that can be regarded as intelligent. For instance, studies of beehives and ant colonies sometimes focus on how the individual insects interact to produce overall behavior that is adaptive for the group.

Network Science

Collective intelligence focuses on the subset of network science that involves intelligent collective behavior. For instance, simply analyzing how rapidly news diffuses in networks with different topologies would not be central to collective intelligence, because there is no overall intelligent behavior being explicitly analyzed. But if such a study also analyzed how effectively the network as a whole filtered different kinds of news or how the speed of information diffusion affected the speed of problem solving, it would be central to collective intelligence.

The History of Collective Intelligence as a Topic of Study

The phrase collective intelligence has been used descriptively since the 1800s if not longer. For instance, the physician Robert Graves (1842, pp. 21–22) used it to describe the accelerating progress of medical knowledge, the political philosopher John Pumroy (1846, p. 25) used it to describe the people’s sovereignty in government, and C. W. Shields (1889, pp. 6–7) used it to describe science as a collective endeavor. In 1906, the sociologist Lester Frank Ward used the term in something like its modern sense (Ward 1906, p. 39): The extent to which [society will evolve] will depend upon the collective intelligence. This is to society what brain power is to the individual.

The earliest scholarly article we have found with collective intelligence in the title was by David Wechsler, the psychologist who developed some of the most widely used IQ tests (Wechsler 1971). In it Wechsler argues that collective intelligence is more than just collective behavior; it also involves cross-fertilization resulting in something that could not have been produced by individuals. Around the same time, the computer scientist Doug Engelbart was doing his pioneering work on augmenting human intellect with computers, including computational support for team cooperation (Engelbart 1962, p. 105; Engelbart and English 1968). Later Engelbart used the term collective IQ to describe such work and its broader implications (e.g., Engelbart 1995).

In 1978, Roxanne Hiltz and Murray Turoff used the term collective intelligence to describe the goal of the computerized conferencing systems they pioneered (Hiltz and Turoff 1978). In the 1980s and the 1990s, collective intelligence was used more and more to describe the behavior of insects (Franks 1989), of groups of mobile robots (Mataric 1993), of human groups (Pór 1995; Atlee 1999; Isaacs 1999), and of electronically mediated human collaboration (Smith 1994; Levy 1994; Heylighen 1999). As best we can tell, the first two books with collective intelligence in their titles also appeared during that period. Smith’s (1994) focused on computer-supported work groups, Levy’s (1994) on the worldwide exchange of ideas in cyberspace.

In the 2000s, the term collective intelligence was used even more widely—in some of the publications mentioned later in this volume and in many other publications in computer science, spirituality, and business (e.g., Szuba 2001; Hamilton 2004; O’Reilly 2005; Segaran 2007; Jenkins 2008; Howe 2009). Of particular importance to the spread of the concept were The Wisdom of Crowds (Surowiecki 2004) and other books for a general audience featuring the concept of collective intelligence—e.g., Wikinomics (Tapscott and Williams 2006) and The Rational Optimist (Ridley 2010).

The 2000s also saw the first academic conferences on collective intelligence (Kowalczyk 2009; Bastiaens, Baumol, and Kramer 2010; Malone and von Ahn 2012) and the first academic research centers focusing specifically on this topic (the Canada Research Chair in Collective Intelligence, started in 2002 at the University of Ottawa; and the Center for Collective Intelligence, started in 2006 at the Massachusetts Institute of Technology).

Related Concepts

In addition to those who have used the specific term collective intelligence, writers in many fields have talked about closely related concepts. For example, psychologists have talked about similar concepts since the 1800s, including crowd psychology (Tarde 1890), crowd mind (Le Bon 1895; Freud 1922), and the collective unconscious (Jung 1934). Emile Durkheim (1893) used the term collective consciousness for the shared beliefs and values that lead to group solidarity. Adam Smith (1795) talked about an invisible hand controlling allocation of resources in a market. And several writers talked about forms of collective intelligence on a global scale, using terms such as world brain (Wells 1938), planetary mind (Teilhard de Chardin 1955), and global brain (Russell 1983; Bloom 2000).

More recently, social scientists have discussed numerous phenomena that can be regarded as examples of collective intelligence. For instance, theories of transactive memory analyze how groups (such as married couples and business organizations) divide and coordinate the work of remembering things (Wegner, Giuliano, and Hertel 1985; Wegner 1986). Studies of distributed cognition examine how real-world cognitive processes (such as navigating a naval vessel) do not occur solely within a single brain but instead include other people and objects as critical components (Hutchins 1995a,b). Studies of trust in testimony illuminate how much of what we think we know about science, religion, and other topics is based, not on our own firsthand experience, but on what others we trust have told us (Harris and Koenig 2006). And philosophers have argued that it is arbitrary to say that the mind is contained only within the skull. Instead, many cognitive processes are actively coupled with external objects, such as the pen and paper used to do long multiplication and the rearrangement of letter tiles to prompt word recall in Scrabble, and thus these external objects should be considered part of the mind that carries out the cognitive processes (Clark and Chalmers 1998). Some of these concepts, and many others closely related to collective intelligence, will be discussed in later chapters.

Why Is Collective Intelligence a Timely Topic Now?

The past few years have seen a significant increase in the popularity and maturity of research in collective intelligence. As was noted above, new forms of collective intelligence made possible by information technology are affecting the daily lives of great numbers of people all over the world. And many disciplines, including neuroscience, economics, and biology, are making fundamental breakthroughs in understanding how groups of individuals can collectively do intelligent things.

But if we do not make an effort to synthesize these insights across fields, we will end up with silos of knowledge, redundant efforts in different academic communities, and lost opportunities for interdisciplinary synergies. We believe this is a critical time for these different fields to come together and begin sharing insights.

This urgency is balanced by pragmatic considerations: the scope of the challenge is large, so we must draw on all the resources at our disposal to tackle them. Several of the authors in this volume have already crossed disciplinary lines in their research. Our goal is to help others to do the same. We hope to provide readers with the tools they need to know when each disciplinary perspective might be useful and with leads to follow when they want to find out more.

An Overview of the Book

This book aims to help coalesce the field of collective intelligence by laying out a shared set of research challenges and methodological perspectives from a number of different disciplines. In the chapters that follow, authors from each discipline introduce the discipline’s foundational work in collective intelligence, including its methods (such as system engineering, controlled experiment, naturalistic observation, mathematical proof, and simulation), its important research questions, and its main results. The chapters and their introductions include references to classic works and to recent research results, providing both places to start to learn more and the beginnings of a shared set of references for this field.

Since our goal in organizing the chapters is to stimulate useful connections, we do not believe there is any one best way of grouping different disciplinary topics. Accordingly, the grouping of disciplines in the chapters that follow is somewhat arbitrary, depending not only on intellectual considerations but also on the availability of potential chapter authors.

We begin with what are, perhaps, the most surprising kinds of collective intelligence. First, Andrew Lo summarizes what the field of economics can teach us about how collective intelligence happens (and sometimes doesn’t happen) in markets. Then Deborah Gordon introduces a biological-sciences perspective on collective intelligence.

Next we move on to the newest kinds of collective intelligence: those enabled by computers. In two separate chapters, one on human–­computer interaction and one on artificial intelligence, Jeff Bigham, Michael Bernstein, Eytan Adar, Daniel Weld, Mausam, Christopher Lin, and Jonathan Bragg lay out computer science’s interest in building platforms for crowdsourcing, guiding complex crowd tasks, automating workflows, quality control, and creating hybrid artificial intelligence / crowd systems.

The last three chapters focus on what the social sciences (other than economics) can teach us about collective intelligence involving people. First, Mark Steyvers and Brent Miller introduce ideas from cognitive psychology that are relevant to one important kind of collective intelligence: the wisdom of crowds effect. Then, Anita Williams Woolley, Ishani Aggarwal, and Thomas Malone describe perspectives from social psychology and organizational theory, introducing ideas about how human groups work and how they can be collectively intelligent. Finally, Yochai Benkler, Aaron Shaw, and Benjamin Mako Hill focus on peer production as a form of collective intelligence. Though this chapter focuses on one specific theme, the chapter and the editors’ introduction represent a much wider range of work that is relevant to collective intelligence in law, communications, sociology, political science, and anthropology

At first glance, these disciplines and their writings may seem disconnected, a series of ivory towers with no bridges to connect them. However, in these chapters, the disciplines often focus on the same basic issues using different methodologies and perspectives. For example, many of the chapters grapple with questions such as these:

• What basic processes are involved in intelligent behavior of any kind, collective or not?

• How can these processes be performed by groups of individuals? For instance, how can the processes be divided into separate activities done by different individuals? And what additional activities are needed to coordinate the separate activities?

• What kinds of incentives and other design elements can lead to coherent behavior from the overall group?

Pairs of chapters also complement one another. For example, the chapter on biology and the chapter on artificial intelligence both explore mechanisms for deciding how many members of a collective to place on a task. Similarly, the chapter on human–computer interaction views Wikipedia as a set of design decisions that can inspire future systems while the chapter on peer production studies the design of Wikipedia at a micro level to understand its success.

Together, we hope, these foundations will help researchers from many disciplines to join together in tackling what we believe are some of the most exciting research challenges of our age: How can groups of individuals—collectively—be more intelligent than any of their members? How can new combinations of people and computers be more intelligent than any person, group, or computer has ever been before?

Appendix: Representative Definitions of Collective Intelligence

a collective decision capability [that is] at