The Ascent of Market Efficiency: Finance That Cannot Be Proven

THE ASCENT OF MARKET EFFICIENCY

Finance That Cannot Be Proven

Simone Polillo

CORNELL UNIVERSITY PRESS ITHACA AND LONDON

Contents

1. Introduction

2. The Rhetoric of Market Efficiency

3. Collaborations and Market Efficiency

4. Winners and Losers in Financial Economics, or Fama versus Black

5. How Financial Economics Got Its Science

6. Conclusion

Acknowledgments

Appendix A

Appendix B

Notes

References

Index

1

INTRODUCTION

The Data-Method-Theory Triad, or Why Finance Cannot Be Proven

Over the past half century, in the United States and many other countries, finance has become deeply enmeshed in, and consequential for, everyday life. Virtually ubiquitous and all-powerful, financial markets now play a fundamental role in determining the life chances of individuals, serving as indispensable sources of funding both for everyday necessities, as testified by the expansion of credit card–fueled expenses, and for lifelong savings strategies—witness the proliferation of long-term, heftier investments, from retirement portfolios to car loans and home mortgages.

There are many complex reasons behind the rise of financial markets, ranging from the emergence of institutional investors (i.e., nonfinancial institutions whose budgets rely on substantial trades in financial securities) to the political support national governments and international financial institutions have lent to the financial sector over this multidecade period; from the move of mass production away from developed countries, and its replacement by financial activities, to the rise of globalization. Each of these dimensions has drawn substantial scholarly interest. We know less about how financial markets have gained intellectual credibility: how financial economists successfully defend financial markets and make a case for their fundamental and positive role in the economy when they have failed to provide an analytically sound and theoretically robust framework to explain how markets behave. This puzzle is the focus of this book.

In the 1950s, financial markets became the object of analysis of a new discipline—financial economics—and have shaped its evolution ever since. It would not be an exaggeration to say that the new discipline of financial economics was made possible by a certain way of theorizing, collecting data about, and analyzing markets. Central to this new approach to modeling market behavior is what came to be known as the efficient-market hypothesis. The hypothesis entered the canon of financial economics and went on to shape its trajectory for years to come. In the process, it changed the professional status of financial analysts: self-professed new finance men, armed with this new analytical toolkit, with its new methods, mathematical models, and data, could now claim that their work was scientific rather than merely technical, systematic rather than impressionistic, and based on rigorous empirical evidence rather than on anecdote.

How did the efficient-market hypothesis allow financial economists to recast their work in scientific terms? According to the hypothesis, markets are efficient when individuals can make financial decisions on the basis of all relevant information. No theory can fully or even satisfactorily explain why they make these decisions, and a theory will certainly not predict what specific decisions individuals will make. Some investors may want to hold an asset as a form of long-term saving; others may want to hold an asset on a bet that its price will go up in the very short term. Some may feel intimidated by the risky nature of an investment; others may be drawn to it precisely because of the risk.

None of these contingencies matters, though, because when financial markets operate free from outside interference, they price assets such that all relevant information is incorporated within them. Therefore, it becomes impossible to predict how prices will change over time. The moment that new information becomes available, prices change to reflect it. The theory of efficient markets is therefore a theory of unpredictable markets.

This foundation of unpredictability is surprising for several reasons, both political and intellectual. Why should unpredictable markets be entrusted with the central role they now play in modern economies? In the period leading up to the Great Depression, and then again after the end of World War II, there was renewed faith in attempts to theorize what scholars thought were the underlying driving forces operating behind markets; and inchoate theories about the possibility of predicting the behavior of financial assets, and of the markets in which they were traded, started gaining traction. Financial analysts understood that past economic behavior influenced the future; so, in the hope of learning useful lessons they could apply to their own economic situation, they began to look back at economic history, considering it a source of knowledge about markets.

But the perspective of unpredictable markets, based on the idea that the market makes such exercises in prediction futile, and the analyses of individual strategies they encourage a foolish waste of time, vanquished this conflicting view once and seemingly for all. By the 1960s, it was becoming common sense that, through the workings of the market itself, the moment new information about assets traded in the market became available, the prices were updated accordingly. No other arrangement could accomplish such a feat: as long as the market set prices in such a way that they would reflect all available information, rational investors were better off trusting the market. Those financial analysts who claimed to be able to beat the market were thus seen as charlatans and hypocrites; rational investors should ignore them. Because market prices were the best evaluations of the value of an asset, they held no secrets. Rational economic decisions could, and should, be made on the basis of market prices alone.

The idea that markets are unpredictable and therefore better at allocating resources than the recommendations and plans devised by specialists was partly an attack on the investment management profession. It also became the core idea of a broader movement—some call it neoliberalism (see, for instance, Slobodian 2018), others market fundamentalism (Block and Somers 2016)—whose central ideological message was that markets were superior to any other sort of economic intervention. According to this view, financial markets, and by extension all markets, can allocate claims to ownership over valued resources efficiently when they are freed from the interference of noneconomic actors, like governments in their capacity as regulators. It is, in fact, precisely when markets are infused with personal contacts, collusions, and counterproductive outside regulation that they stop working efficiently. Impersonality and unpredictability, by contrast, go hand in hand with efficiency. Let the market serve as the ultimate arbiter of value, and the economy will flourish.

Even when we take into account the broader political context, the intellectual puzzle of how financial economists came to associate efficiency with unpredictability remains unsolved. Early scholarly investigations into market behavior may have looked like predictive failure, or at least a potential deficiency in explanation. Some of these early studies of the behavior of financial securities did indeed uncover anomalous patterns, and not always small ones, which could have made market efficiency seem like nothing more than an inability to understand the nature and causes of these patterns in complex data. More important is that the efficient-market hypothesis (and this is perhaps the reason why financial scholars keep referring to it as a hypothesis rather than a theory), because it theorizes on the basis of absence of evidence, cannot be proved. Testing the theory empirically requires making additional assumptions about the behavior of markets: only if those assumptions are correct can the failure to detect patterns be construed as the absence of patterns. Financial economists recognized almost immediately the severity of the problem posed by the joint hypothesis: understanding how markets behave and whether markets behave efficiently are two sides of the same coin. Yet financial economists invested their energies on efficiency, bracketing off anomalies or treating them as additional factors that, once they were accounted for, helped preserved the notion of efficiency. Financial economists then used the efficient-market hypothesis to back up a broader claim that their research was based on science: that they were not engaging in the impressionistic and therefore inexact analysis of financial markets. The joint hypothesis was treated, de facto, as a unitary hypothesis, shifting with the context of the specific study at hand, with efficiency serving as the foundational framework that gave the hypothesis meaning, and with empirical tests proving not efficiency per se but the myriad (and often changing) assumptions necessary for efficiency to be operationalized.

By the early 1980s, as a result, market efficiency was widely recognized as financial economists’ main explanatory achievement, indeed as a resounding success. Alternative perspectives lost out and were forgotten. By accepting that financial markets priced assets rationally, in the sense that investors and traders always used all available and relevant information as they assessed the value of financial assets, such that no systematic patterns could be uncovered after their intervention had been properly taken into account, financial economists moved away from the kinds of questions that were initially quite central to their emerging field. These questions would return with a vengeance later in the wake of the financial crisis, especially after 2007: Why do markets expand and then crash? How can markets be structured to encourage certain kinds of behaviors and discourage others? To be sure, as a result, the efficient-market hypothesis is no longer as dominant today as it was in the 1980s, as investigations into the irrational behavior of markets have begun to make an impact. But proponents of efficient markets continue to stand their ground.

Understanding the rise and persistence of market efficiency as a theory of market behavior, I argue, requires a focus on processes that are, ironically, quite personal, filled with trust and emotion, contingent but not unpredictable, or at least amenable to historical reconstruction. These are the processes that gave rise to scholarly networks, specifically to the intellectual relationships that financial economists established with one another. Accordingly, this book focuses on the people who gave us the fullest elaboration of this perspective: those US-based scholars of finance, such as Eugene Fama and Richard Roll, who played a crucial role in granting coherence and credibility to the idea that markets are unpredictable. These financial economists argued, with increasing success over time, that it was not possible to anticipate the future movement of stock prices, because any relevant information was already incorporated into present prices. Financial economists argued that unpredictability is a signal of markets processing information efficiently rather than a symptom of underlying problems or of explanatory deficiencies in the model itself.

Zeroing in on how financial economists collaborated, provided mentorship to new scholars, and developed distinctive ideas about the kind of work they considered to fall under the purview of financial economics will reveal the social processes and social practices that allowed market efficiency to flourish as a theory of market behavior. Financial economists, I argue, built their discipline through a new interplay of theory, methods, and data: they recombined theory, method, and data so that data analysis would drive the development of the discipline. In general, theory, data, and methods form a triad in social research: they can each provide the focus of attention of distinctive networks, and disciplines crystallize through the combination of any of these networks in different ways. In the case of financial economics, networks of data analysts provided the original, creative impetus; theory and methods developed around data analysis. This particular combination gave financial economics a distinctive affect: as a social structure of collaboration and coauthorship, financial economics came to be held together by a common appreciation of distinctive scholarly virtues, through which financial scholars increased trust in one another, made their work credible, and built a self-sustaining network pursuing research on efficient markets.

Centered on Eugene Fama, financial economists developed collective understandings of what counted as knowledge and what made a scholar trustworthy. In both respects, data analysis became the primary driver of research that financial economists came to consider creative. Cutting-edge financial research analyzed data; as data analysis climbed to the apex of the theory-data-method triad, theory descended to the bottom. Over time, consensus grew about the virtues that should characterize sound scholarship in finance, and this allowed the structure of the network to expand and reproduce itself. Understanding how this network emerged, why it took the specific shape it did, and why it proved to be durable and resistant to critique for so long is the first set of goals of this book. Using the theory-method-data triad as a lens, and showing how it can yield a sharper focus on the dynamics of social knowledge, is the second and more general goal.

Is Efficiency an Objective Property of Markets?

Finding evidence for the configuration of data, theory, and methods that steered the network of financial economics toward market efficiency, and understanding its contours and significance, will take us on a journey through many different sociological theories. But before we embark on this journey, some potential objections should be addressed straightaway. A financial economist might suggest an easier and more straightforward answer to the puzzle of unpredictable markets: that the theory of market efficiency prevailed because it worked, and that it worked because it captured fundamental aspects of market behavior that other models, theories, or hypotheses simply did not. A sociologist of professions might say that the research program underlying market efficiency was made possible by the particular configuration of institutional resources and professional opportunities open to financial economists in the post–World War II period, and that understanding group dynamics comes with the risk of missing the forest for the proverbial trees. What if market efficiency is simply based on comprehensive data on financial markets, making it not a subjective theory based on experience but a systematic, scientific investigation of market processes? Let us begin with our hypothetical economist.

This economist would have a strong ally on her side: the committee that, in 2013, awarded Eugene Fama the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel. This is the brief justification motivating the award: In the 1960s, Eugene Fama demonstrated that stock price movements are impossible to predict in the short-term and that new information affects prices almost immediately, which means that the market is efficient (NobelPrize.org 2013). That it took the committee more than fifty years to recognize the importance of Eugene Fama—one of the founding fathers of financial economics, and its most steadfast proponent of the efficient-market hypothesis—can be construed as a testament to the rigor of the scientific process behind his work. It was only after sufficient evidence in support of the hypothesis had accumulated that Fama’s hypothesis would receive such an important recognition. But there are at least two reasons to be doubtful about this account. One has to do with the kind of theory the efficient-market hypothesis is; the second, with the limits data play in proving, or disproving, such a theory.

First, as efficient-market theorists would themselves recognize, efficiency is, and has always been, less a factual representation of markets than an idealized model that nevertheless shapes the behavior of market agents (and often market regulators and policymakers as well). Economic theory is an intervention into the economic world that constitutes the practices through which that world is then constructed. Over the past two decades, scholars who have paid attention to this relationship between economic models and their object of analysis have described it as a process of mutual influence: put in the stronger language of MacKenzie (2006), one of the most sophisticated proponents of this approach, economic theory performs the economy. The approach has given rise to a school of thought labeled social studies of finance, whose proponents are concerned with the consequential linkages between the economy and economic theory. They say, for instance, that market efficiency becomes a powerful theory because it has practical consequences for the financial world, such as the creation of index funds, financial portfolios whose composition replicates the risk profile of the market as a whole, thereby directly incorporating fundamental insights from the efficient-market hypothesis. Scholars in this tradition would suggest that as index funds become more popular and widespread, the market starts behaving in the ways hypothesized by the theory behind these financial instruments.

The social studies of finance’s claim that economic models are not neutral representations of some objective economic reality but rather constitute and perform that reality is crucial to our understanding of the power of economic theory. However, I argue that there is still theoretical work to be done to explain the successful rise of a theory of efficient markets. For one, as also argued by MacKenzie (2006), the simple adoption of a model by authoritative sources, either scholarly communities or broader groups of practitioners, and its subsequent dissemination do not imply that the model will affect the economy. Rather, the model has to produce some relatively stable outcome as well, which means that, for instance, it cannot be based on mathematical mistakes or produce results that others armed with better models may find easy to exploit to their own advantage. The properties of the model itself increase the likelihood of the model affecting the economy.

Does this mean that the model has to work before it can be put to work? The problem, I believe, is a larger one. Market efficiency is an abstract and general concept; it is only weakly related to the many and more specific models that scholars have shown to be amenable to practical use by market operators. It may well be that the success of these more specific models reinforces the theory. Yet, if we examine the historical trajectory of the theory and the details in the research process that led to its articulation, we will immediately see that the path to market efficiency was riddled with stumbling blocks and multiple forks in the road, and that choosing a route other than the one actually taken would have created different kinds of hypotheses about the nature of markets. As a distinct though related tradition in the sociology of knowledge and culture broadly termed the strong program informs us, a sociologist must treat any of the alternative paths not taken in the same way she or he would treat research deemed successful. Truthfulness is accomplished through social processes (Barnes 1986; Bloor 1991). It is therefore important to find out what theories, models, and hypotheses emerged as potential contenders to the efficient-market hypothesis, and understand their demise without resorting to the power of hindsight. I pursue this line of inquiry throughout the book.

More important, it also follows from this argument that there is no single test that can prove or disprove a general theory like market efficiency. This is the second reason we should be skeptical of the efficient-market hypothesis: even more complex tests are ultimately vulnerable to critiques of the adequacy of their own assumptions and the quality of the data on which they are based. Underdetermination is the core point of the Duhem-Quine thesis in the sociology of science:¹ theories are tested with data that may be flawed or incomplete; working assumptions, necessary to operationalize key constructs, make it impossible to directly test the central predictions of a theory (MacKenzie 2006, 23). For this reason, theories often survive a long time after they have allegedly been disproved. A field like financial economics, where data represent human activities and therefore cannot be controlled, manipulated, and experimented with in the ways data about natural objects can, is no exception (see, e.g., Hacking 1983).

One might even say that a theory like efficient markets is always, to some degree, underdetermined because its empirical content can be evaluated only by making additional hypotheses, which are themselves not tested. Eugene Fama (1970, 413–14) recognized as much: The theory only has empirical content … within the context of a more specific model of market equilibrium, that is, a model that specifies the nature of the market ‘equilibrium’ when prices ‘fully reflect available information.’ But the theory did not specify what this model of market equilibrium should look like; one had to make an assumption in this regard. As Richard Roll (1977, 145), a student of Fama, would later add: There lies the trouble with joint hypotheses. One never knows what to conclude. Indeed, it would be possible to construct a joint hypothesis to reconcile any individual hypothesis to any empirical observation.

As a consequence, the success of market efficiency cannot be attributed exclusively either to some objective quality of the market itself or to its success in generating models that shape market behavior, because there is no direct, unmediated connection between the theory and the market, and the theory and the specific models employed in the market. Tests of market efficiency are always, also, tests of additional hypotheses about markets. While this flexibility has been generative of a vibrant research tradition, it forces us to rethink the possibility of a direct relationship between the efficient-market hypothesis and specific financial practices. By the same token, market behavior is shaped by academic theory, to be sure; but it responds to other forces as well. And so does financial economics as a discipline. It might be informed and shaped by developments in real financial markets, but this is not the whole story.

One might then argue that understanding market efficiency is not so much a matter of assessing the accuracy of the model but more a matter of understanding how the discipline that produced the model became more powerful. And, to be sure, between 1950 and 1980, financial economics underwent a process of mathematization, formalism, and abstraction, mimicking the path followed by financial economists’ more powerful disciplinary counterparts, the economists, in their quest for scientific authority. As shown by scholars who emphasize the cultural and political foundations of expertise, working under the umbrella of the sociology of professions and expertise, US economists tend to understand themselves as autonomous professionals for hire as private consultants (merchant professionals, as Marion Fourcade calls them): formalism allows them to claim that their expertise is based on technical skills, the development of which takes years of specialized training (Fourcade 2009; Steinmetz 2005; Whitley 1986a).

Perhaps, then, it is complex mathematics, embedded in a disciplinary framework capable of accumulating increasing amounts of resources for research, that gave us the theory of efficient markets? Pragmatically, recasting financial economics in a mathematical language allowed financial economists to distance themselves from financial analysts, the kinds of practitioners they accused of pseudoscientism (in an early period in the discipline, for instance, financial economists would openly deride chartalists, the practitioners who claimed to be able to predict the future value of stock prices based on their analysis of past stock price behavior). Mathematization also served to make financial economics look more legitimate in the eyes of economists, opening up funding opportunities as well as institutional venues within which financial economics could thrive (especially the business school).

Yet, the rise of mathematics in financial economics is not quite the same as the rise of the theory of efficient markets. Mathematics is a language, and this term implies that more than one theory of market behavior can be articulated in it; indeed, more than one theory was. In fact, as we shall see, market efficiency was less the result of investigations formalizing market