Economists’ interest in collective decision after World War II: a history

Abstract

This paper tracks economists’ rising, yet elusive and unstable interest in collective decision mechanism after World War II. We replace their examination of voting procedures and social welfare functions in the 1940s and 1950s in the context of their growing involvement with policy-making. Confronted with natural scientists’ and McCathythes’ accusations of ideological bias, positive studies emphasizing that collective decisions mechanisms were unstable and inefficient, and normative impossibilities, economists largely relied on the idea the policy ends they worked with reflected a “social consensus.” As the latter crumbled in the 1960s, growing disagreement erupted on how to identify and aggregate those individual values which economists believed should guide applied work, in particular in cost-benefit analysis. The 1970s and 1980s brought new approaches to collective decision: Arrow’s impossibility was solved by expanding the informational basis, it was showed that true preferences could be revealed by making decision costly, and experimentalists and market designers enabled these mechanisms to be tested in the lab before being sold to those public bodies looking for decision procedures that emulated markets. In this new regime, the focus paradoxically shifted to coordination, revelation and efficiency, and those economists studying collective decision processes were marginalized.

Quadratic Voting is an idea meant to change the theory and practice of collective decision-making. As such, that it was initially conceived by an economist, Glen Weyl, rather comes as a surprise, for economists’ recent and substantial concern with the topic tends to mask its historically elusive and unstable status in the field. True, JEL codes, the intellectual roadmap of the discipline, contain “Analysis of Collective-Decision Making” as a subcategory (D7) in which many papers published in Public Choice or Social Choice and Welfare belong. But the story behind code D7’s content and name testifies to the paradoxical status of collective decision-making research in economics.

When John Pencavel, then editor of the Journal of Economic Literature, embarked on a thorough revision of the classification system for economic literature in 1988, he proposed to create a “D6. Economic Welfare and Public Choice” section within the Microeconomics category. It was to include cost-benefit analysis, externalities and property rights, normative criteria and their measurement, as well as three entries on normative theory, positive theory and empirical studies in public choice.¹ Gordon Tullock soon complained that the tentative scheme was “downgrading” his field, which previously was an independent subcategory. The JEL’s editors thus contemplated the creation of a new section within H. Public Economics, but they felt that topics such as constitutions, free riding, Clark taxes, and Groves mechanisms did not belong there. It was Roger Noll who eventually suggested the addition of a whole new section within the Microeconomics category. It was, he explained, the only way to do justice to the five percent of the profession who shared “a theme and method: analyzing public sector outcomes on the basis of the economic model of individual behavior, …mak[ing] explicit connection between policy and political behavior, … [and] aid[ing] in the advancement of work on policy making.” The new section was to cover social choice theory, the theory of teams, economic models of political processes, bureaucracy, and policy analysis of the “Tabellini-Alesina” type. Finding a name proved tricky, however. Noll wanted to avoid calling the new section Public Choice (“to avoid association with the political views of Gordon Tullock and Jim Buchanan”), Political Economy (too Marxist), or Political Economics (a term he believed had wide currency only at Stanford and Caltech and might therefore sound parochial). He thus settled on Collective Decision-Making, a wording that he thought was in limited currency, yet “neutral” and consistent with a section that sought to “list everything published that looks at political aspects of policy or at collective choice processes.”

The D7 JEL category illustrates how fuzzy the notion of collective decisions is for economists, as it bundles together disparate contributions in which connections to these decision processes are not always readily apparent. The reasons for this are manifold. First, the topic traditionally was considered the province of philosophers, political scientists and social psychologists (Musgrave 1941; Samuelson 1954; Boulding 1970). Second, what the term refers to and covers exactly is not made explicit, even by the growing number of economists who did research relevant to the field after World War II (see, for instance, Sen 1970). Social choice theorists routinely define it as the study of voting mechanisms. Public choice economists, for their part, suggest that “collective decision” encompasses all “non-market decision making,” the expression they had originally chosen to name their community.² Yet, strong disagreement exists among them as to whether state intervention should be construed as deriving from collective decision, or as an alternative to it. Collective decision is sometimes conceived as encompassing the market, and sometimes conceived in opposition to it. Kenneth Arrow (1951, p. 2) viewed voting and market mechanism as alternative methods for “amalgamating the tastes of many individuals in the making of social choices,” but he makes little distinction between the two in his monograph. In the late 1950s, Paul Samuelson wrote to Richard Musgrave that he “considered the market mechanism as one special kind of voting system.”³

The terminology deployed by contributors to this literature likewise is vague. Economists have thus used a number of close yet not fully synonymous expressions in their work, including “social choice”, “public choice”, “team”, “committee”, “club” and “group” decisions. That “social choice” and “public choice” characterize research topics as well as communities whose interests overlap but are not restricted to studying collective decisions adds further to the confusion. Also problematic is the practice of defining collective decision making as a mechanism whereby a group of individuals with different preferences and valuations agree on a candidate, a public policy, a tax rate and so forth. As Hausman (2011) points out, notions of preferences, values and choice are often difficult to disentangle.⁴ Finally, those theorists, experimentalists, and mechanism designers who have studied the public sector, voting, bureaucracy, constitutions, taxation, public goods, to name only a few topics, often have focused on information, coordination, allocation and preference-revelation, rather than on collective decisions per se.

The purpose of this paper is to track and explain economists’ increasing, yet elusive and diverse interests in collective decision-making processes after World War II. Several historians have associated the emergence of social scientific studies of collective decisions with the Cold War. Amadae (2003), for instance, claims that grounding democratic processes and public policy-making into rational choice theory was a deliberate move by a small set of economists and political scientists, mostly associated with RAND, to rescue the foundations of American democracy from the Soviet threat. Kenneth Boulding (1970), on the other hand, saw in economists’ growing interest in collective decision making a sign of their “imperialistic” ambitions. Although they were undoubtedly interested in the philosophical and theoretical aspects of collective choice, we believe that economists also were practitioners progressively more involved in the daily business of policy design and evaluation, so that even their most theoretical works were informed by concerns with practical policy-making. Our claim, therefore, is that economists’ interest in collective decisions is rooted in the transformation of their relationship to policy-making. During the late 1940s and 1950s, economists’ growing involvement in policy-making implied that they had to adopt normative perspectives. Yet, at the same time, they were pressured strongly to develop “value-free” analyses. Though the study of collective decision-making is usually traced back to Pliny the Younger, or at least to Borda and Condorcet, its taking-off in economics can be seen as a response to this postwar dilemma.⁵ This concern with collective decision within public bodies, groups, firms, political parties, and so on flourished in the 1960s as economists qua objective scientists faced the challenge of dissecting, but also legitimizing or de-legitimizing government intervention in an era riven with civil unrest and international conflict. During the 1970s, the development of new approaches—positive, normative, axiomatic, institutional, experimental—led to the fragmentation of the study of collective decision into distinct fields. As the different modeling strategies (i.e., social welfare functions, game-theoretic models) stabilized in the 1980s, the research agenda shifted toward a focus on information, coordination, preference revelation and strategic behavior.

1 Wars, value dilemmas in policy-making and the emergence of a collective decision research program

1.1 Economists and values: from benign neglect to dilemmas

Before the postwar period, there was little economic scholarship directly addressing collective decision mechanisms. Economists studied wealth, production, labor, capital and whether producers and consumers’ plans should be coordinated through the adjustment of prices (markets) or through the decisions of a central planner.⁶ Whether economists were legitimate in discussing the values underpinning public policies and market outcomes was also a matter of debate. George Stigler (1943, p. 358) claimed that “at the level of economic policy, then, it is totally misleading to talk of ends as individual and random; they are fundamentally collective and organized […] the economist may […] cultivate a second discipline, the determination of the ends of his society particularly relevant to economic policy.”⁷ Abram Bergson (1938, p. 323) also regarded “the determination of prevailing values for a given community [as]… a proper and necessary task for the economist.” On the other hand, new welfare economists agreed with Lionel Robbins’s (1932) admonition that they should shy away from working with values. They attempted to build a value-free analysis devoid of any interpersonal utility comparisons (see Baujard 2016 for a detailed survey).

Scholars specializing in public finance addressed topics that were the closest to the question of collective decision-making and its relationship to individual preferences. Indeed, the works of European economists, such as Antonio De Viti de Marco, Knut Wicksell, or Erik Lindahl—which formed the so-called “continental tradition” in public finance—had developed a “benefit approach” linking public decisions regarding taxation and expenditures directly to the individuals’ preferences and willingness to pay for public goods (Musgrave and Peacock 1958; Musgrave 1959). As early as the 1880s, those questions had led some of them to inquire into political problems and institutions related to democratic systems, such as the voting process, albeit not in a formalized way (ibid.). In the context of 1940s’ American economics, however, this tradition largely was neglected. As Musgrave and Peacock (1958) noted, the works of Edward Allen and Oswald Brownlee, Hugh Dalton, and Henry Simons created in the United States a different tradition, which focused mostly on taxation. Public expenditures, meanwhile, were separated out of the analysis, considered as a political decision outside the realm of economics. The task of public finance was to analyze the mechanisms that could be utilized to raise the revenue necessary to achieve given expenditure ends. In the late 1930s, American scholarly debates focused on the “ability to pay” approach. Mostly grounded on cardinal utility, this approach, dealt with questions of welfare economics (following Alfred Marshall, A.C. Pigou, or, more recently, Abba Lerner), when, for instance, addressing the corrective effects of taxation. It also dealt with questions of distributive justice, when addressing the problem of how (and on what criteria) to allocate and redistribute the tax burden. Yet, because it considered that individuals should pay taxes on the basis of their means and not of their benefits from the public sector, the links between the individual and the state were not conceptualized as a “voluntary exchange” (or quid pro quo) relationship, contrary to the continental tradition.⁸ Overall, only a few attempts to link the tax and expenditures sides of the public budget (in Pigou, Dalton and a few others) along the lines of this ability to pay approach, and none yet addressed the crucial question of the origins and construction of social utility.

It was this continental tradition that Richard Musgrave, a German émigré thoroughly trained in European public finance theory, hoped to revive when settling at Harvard in the late 1930s. Yet, his attempt at grounding public intervention directly on individual preferences was discarded rapidly as falling outside the boundaries of economics or too abstract and impractical as a guide for policy-making. Musgrave (1939) initially envisioned an integrated theory of taxes and public expenditures around a voluntary exchange relationship between the individual and the state, but was left to conclude that the idea bore “little practical significance,” because individuals were in practice constrained to pay taxes, and because of the underlying competitive price assumption, deemed too unrealistic. In the same vein, Howard Bowen (1943) understood that the optimal provision of public goods required the revelation and aggregation of individual preferences and thought the voting process was “the closest substitute for consumer choice.” But in the end he concluded that though the decision of the “modal voter” sometimes approximated the optimum allocation point, overall majority rule was unreliable, sometimes “hopeless” and “virtually useless.” Majority rule “can seldom be regarded as an unequivocal indication of public desires,” he warned, and he recommended the use of polls, questionnaires and interviews instead. In the context of World War II, Bowen’s difficulties with the apparent dead-end of democracy as a preference-revealing mechanism fell short of spurring the large literature that, a few years later, attempted to tackle Kenneth Arrow’s impossibility theorem.⁹ Indeed, in a context of all-out war, it was relatively easy to discard the problem of constructing a social value scale since government’s goals were not really the subject of debate.

Before the late 1940s, therefore, the values guiding government action were not conceived as derived from individual choices. The economist’s goal was not to question the legitimacy of the (usually assumed to be benevolent) social planner’s objective function, but to evaluate the effect of such and such policies on citizens’ welfare, so as to study the comparative merits of public policies and market outcomes. This was Bergson’s motivation for constructing his social welfare function, and at the forefront of Musgrave’s (1941, p. 323) mind when he developed a “planning approach” to public finance that separated the “determination of group preferences” from their attainment through planning. The determination of the social value scale was a task for psychologists, Musgrave warned, meaning that economists should take collective wants as given.

The outbreak of the War opened an era when the legitimacy of government planning commanded broad support. In the United States, public expenditures rose from $9.5 billion to $93 billion in 1945, from 12% of the GDP in 1942 to 42.1% in 1945 (CEA 2008). Overnight, the government found itself in charge of devising new weapons (aircraft, radar, nuclear bombs) and ways to use them against the enemy, of allocating national resources to organize production on an unprecedented scale, of funding the war effort by levying taxes and borrowing, and of preventing prices from soaring. Whether in charge of identifying the “right” amount of taxes at the Treasury (Milton Friedman) or at the Department of Commerce (Howard Bowen), of containing inflation at the Federal Reserve Board (Richard Musgrave), or of improving resource allocation at the National Resources Planning Board (Paul Samuelson), virtually every economist in the United States was suddenly thrown into the daily business of policy-making, compelled to identify the proper means to fulfill the government’s ends. The immediate postwar context consolidated the government’s unprecedented involvement in the society and the economy, notably by the Employment Act of 1946, which also institutionalized the economist’s role as a forecaster and policy advisor by establishing the Council of Economic Advisors and the Joint Economic Committee (Bernstein 2001; Backhouse 2010). And, as is now well known, the Cold War additionally entrusted economists with macroeconomic stabilization.

Carrying out those tasks required that economists work with policy ends, values and goals in mind—that is, with an explicit or implicit mathematical function representing the government’s objectives. But doing so exposed them to others’ doubts at a time when their scientific credentials were not firmly settled. The social sciences, including economics, often were considered ideological. It was the explanation that MIT physicists, such as Vannevar Bush and Karl Compton, gave for refusing to open a social science division within the National Science Foundation in 1950 (Solovey 2013). Not only was the very idea of government planning associated with Soviet-like modes of organization, but economists’ previous participation in the New Deal made their work especially vulnerable to accusations of value-ladenness. Keynesian scholars, such as Bowen and Lawrence Klein, increasingly were suspected (Solberg and Tomlinson 1997). Textbooks spreading Keynesian ideas, such as Lorie Tarshis’s, but also Samuelson’s Economics, were attacked (Giraud 2014). Moreover, financial support for economic research largely came from military agencies and major philanthropic foundations (especially the Ford Foundation), whose funding efforts emphasized problems-oriented research, quantification, theoretical and mathematical methodologies, and, what is most important, promoted the notion of scientific neutrality. Overall, this postwar context provided a strong impetus, in economics as in sociology and political science, for the development of “value-free” analyses.

The problem was that carrying out policy-relevant economic analysis without making value judgments seemed like squaring the circle. Most economists rejected sociologist Gunnar Myrdal’s (1944) proposal that economists work with explicit value premises, but they also concurred that the so-called New Welfare Economics, allegedly value-free, “cannot be used as a guide to social policy” (Arrow 1951).¹⁰ If working with some values or ends was inescapable, then the use of the social welfare function devised by Bergson appeared as a more promising venue, perhaps because Samuelson (1947, p. 221) was careful in how he characterized it in his Foundations. “Without inquiring into its origins, we take as a starting point for our discussion a function… which is supposed to characterize some ethical belief—that of a benevolent despot, or a complete egoist, or ‘all me of good will’, a misanthrope, the state, race, or group mind, God, etc.”, he wrote (emphasis added). But the most widespread solution endorsed by economists was to point to some form of consensus found in the American society. Musgrave (1948, p. 388) retained as a policy objective “the requirement of a tax structure which is ‘good’ in the sense of contributing most to the maintenance of high employment and price-level stability”, one he believed consensual and minimal. Friedman (1948, p. 246), remarked similarly that, “the basic long-run objectives, shared I am sure by most economists, are political freedom, economic efficiency, and substantial equality of economic power”. And while Samuelson (1947, p. 224–227) acknowledged that getting rid of values was a “delusion,” he contended that the values usually imposed on the government’s social welfare function were “more or less tacitly acknowledged by extremely divergent schools of thoughts,” and “characteristic of much modern thought of the last century… typical of the beliefs of the classical and neo-classical economists.”

1.2 Policy decisions as the aggregation of individual choices: an impossible solution?

A few economists, however, began to develop another conception of the State that neither forced an idiosyncratic notion of “public interest” onto citizens nor assumed a consensus. The viewpoint underlying this work was that the US government and its economists were implementing citizens’ collective choices, that is, choices derived directly from their own preferences for particular policies. “State decisions are, in the final analysis, the collective decisions of individuals,” James Buchanan (1949, p. 498) wrote, adding that the state had “no ends other than those of its individual members.” In their 1953 book, Politics, Economics, and Welfare, political scientist Robert Dahl and economist Charles Lindblom (1953, p. xxi) asked, “what are the conditions under which numerous individuals can maximize the attainment of their goals through the use of social mechanisms?” This vision was also echoed by Kenneth Arrow, who rejected the “mysticism” of “the organism approach to social problems”, arguing instead that collective choice ought to be built solely on the basis of individual preferences (Arrow 1950, p. 133). This same desire to dismiss the existence of “mystical collective mind” underlies Samuelson’s (1954, p. 387) theory of public expenditures.

If policy ends were to reflect individual preferences, then one had to inquire into the appropriate mechanism for aggregating those preferences into a common (“well-behaved”) social function. Because they were drawing on political science as well as economic insights, Dahl and Lindblom (1953, p. 54) were able to consider a large range of collective decision and coordination processes “through which the […] values […] can be maximized whenever scarce resources are significantly involved,” including the market, polyarchy (democracy), hierarchy and bargaining. The Scottish economist Duncan Black likewise proposed the blending of insights from economics and political science into a theory of committee decisions in order to “contribute to the development of the theory of trade-unions, the firm, and the cartel; and to provide the basis for a theory of the equilibrium distribution of taxation or public expenditures” (Black 1948, p. 23). Against the earlier pessimism of Bowen (1943), Black (1948) demonstrated that simple majority voting could yield interesting results for those seeking optimal decisions. Specifically, if individual ordinal preferences are single-peaked, the collective choice is one-dimensional and everyone votes, committee decisions following simple majority rule result in a political equilibrium corresponding to the preferences of the median voter. Here, the criterion to assess the rationality of collective choices was shifted away from Pareto and traditional welfare economics. A rational collective choice, necessarily derived from individual preference rankings, would elect the candidate that “stands highest on the average of the electors’ schedules of preferences” (Black 1949, p. 159).

Black’s encouraging analysis notwithstanding, economists found their early efforts to study the properties of collective decision rules, and to use them as a basis for policy making, utterly discouraging. The multiple approaches they developed were all riven with difficulty. Arrow’s (1951) Social Choice and Individual Values immediately garnered attention because of his (im)possibility theorem stating that no collective choice could satisfy rationality conditions and another set of five conditions—for instance that the social welfare function should responsive to changes in individual preference ordering and not be imposed on individuals. Another impossibility, this time dealing with policy implementation, came from Samuelson’s (1954) seminal paper on the provision of public goods. Playing with the reader’s nerves, he proposed a new definition of public goods, derived the associated optimum conditions, only to smash hopes for implementation on the next page. Because economic agents have an incentive to lie to the planner about their demands for a public good, the planner is unable to calculate the optimal amount to produce.¹¹

Adding to the internal drawbacks of economists’ attempts to revamp government intervention as the outcome of a collective decision process was James Buchanan’s (1954) critique of Arrow and Samuelson’s whole modeling strategies. Arrow’s framework was predicated on the assumption that both the market process and majority voting could be represented by social welfare functions, Buchanan explained, but in doing this Arrow committed an “organismic” fallacy in conceiving a group with the properties of an individual. Though Buchanan also intended to ground public decisions in individual preferences, his own approach to collective decision-making implied a different way to position the economist vis à vis the citizen. Samuelson’s approach to the provision of public goods, Buchanan (1959) pointed out, assumed that the economist is omniscient, in that he is able to identify Pareto-improving policies. “A presumption of ignorance” (p. 126) was a better starting point, he argued. All that the “positive political economist” could do was to expose citizens to various policies. Then, only the citizens’ unanimous consent would effectively reveal the Pareto improving policy to be implemented (an idea he took from Wicksell).¹² Few economists, however, were prepared to agree with Buchanan’s (1951, p. 178) contention that this Wicksellian unanimity “scarcely seems more impractical and unrealistic than many of the more sophisticated modern proposals for the application of the marginal cost pricing rule.”

Although they diverged in their approaches, the few scholars interested in how collective decision evolved from individual preferences handled the difficulties of aggregation and concrete implementation by resorting to some sort of consensus. In an ironic twist, Buchanan’s reliance upon “consensus” is also found in Arrow’s work, though Buchanan refused to assume that such consensus could be identified readily by the policy-maker. Indeed, Arrow (1950, p. 339) conceded that the five conditions he had applied to the domain of social choice were value judgments, but stressed that they were “apparently reasonable” expressions of “the doctrines of citizens’ sovereignty and rationality in a very general form” (Arrow 1950, p. 339).¹³ And even Dahl and Lindblom (1953) found themselves emphasizing the similarity of individuals’ values and the resulting stability of the American society. Because of some kind of “social indoctrination”, they claimed, citizens shared a common set of values which encompassed freedom, rationality, democracy, subjective equality, security and progress.¹⁴ As domestic and international conflict mounted in the next decade, however, escaping the difficulty of grounding public decisions in collective choice processes by appealing to social consensus would become untenable.

2 Government intervention and collective decisions in an age of conflict

2.1 A theoretical emphasis on the costs and limits of collective decision making

The path-breaking work of Arrow and Black stimulated a small but steady stream of papers and books. Arrow’s framework, as well as his axiomatization of individual preferences as complete and transitive orderings, largely shaped the emerging voting literature during the 1950s. Public finance, meanwhile, absorbed the insights of Arrow and Black, a move perhaps best illustrated by Musgrave’s change of mind regarding the determination of collective ends. In his 1959 Theory of Public Finance, arguably the most significant synthesis of the field for the decade to come, Musgrave confessed that: “I have reversed my original view […] that the theory of the public household need not concern itself with how social preferences scales are determined. As I see it now […] the theory of the revenue-expenditure process remains trivial unless these [the social preferences] scales are determined” (Musgrave 1959, p. 74). The determination of the optimal amount of public goods, for instance, was to be achieved through democratic voting, and the determination of government’s budget was seen as “a special application of the general problem of social choice” (p. 116).¹⁵ Interestingly, although he conceded that simple majority rule had its limits, he maintained that it was the best collective decision process known. Musgrave also claimed that democracies were, in practice, stable systems, as “evidence on measurable characteristics of people… lends credence to the assumption that there is a fair degree of similarity among individuals living in a given society” (pp. 108–109), an argument again reminiscent of the “consensus” advocated throughout the 1950s.

Musgrave’s optimistic take on the benefits of majority voting, though, increasingly was challenged by the development of many positive studies of various collective decision processes, the results of which often pointed to inefficiencies. True, Anthony Downs (1957), a student of Arrow, showed by using spatial competition à la Hotelling (1929) that the result of political competition would lead a two-party system to lean towards the center of the distribution of voter’s preferences—a result that echoed Black’s median voter theorem. But he also pointed out that in some cases where preferences showed strong heterogeneity, or when there were more than two parties, cyclical majorities and the danger of political instability might arise. The articulation of a consistent “public choice” outlook was made possible by the efforts of Buchanan and Tullock to organize a community of scholars, initially centered at the University of Virginia.¹⁶ Their most visible and lasting results were the establishment of regular “Conferences on Non-Market Decision Making” and of an associated committee and journal in 1963 (Medema 2000, 2011). The name of the journal (and of the community) was changed to Public Choice in 1967, and in 1970, the new approach was recognized with a separate JEL code (317), a sign of institutional maturity.

Buchanan and Tullock’s (1962) The Calculus of Consent is a good illustration of this community’s investigative strategy. First, it offered a theory of how individuals negotiate to reach a consensus on constitutional rules, and, consequently, how the latter contribute to collective outcomes. When choosing a constitution, individuals had to balance the costs, benefits and possible failures of market interaction with those of governmental intervention. An inability to reach a collective decision consistent with the preferences of all individuals was undeniably part of the costs of government intervention. On the one hand, a less-than-unanimity rule imposed costs on the defeated minority. On the other, unanimous consent often would require a long and costly negotiation. Overall, the Calculus blended together a positive, normative and contractarian bents, reflective of Buchanan’s research, with the positive analysis of the costs and benefits of actual collective choice procedures (majority votes, qualified majority, and so on) and institutions (bicameralism versus unicameral legislatures), characteristic of Tullock’s interests. From the late 1950s onward, Tullock’s positive analyses were supplemented by a flow of studies examining, e.g., winning coalitions and cyclical majorities, competition between political parties, the so-called paradox of voting, bureaucratic behavior, budgeting processes, and other specific decision-making rules (see Mueller 2003 for a thorough survey of this literature). Buchanan and Tullock’s approach also typified a new way of criticizing government intervention. In a context of rising government spending and intervention into economic and social life, the early public choice literature established that economists could not identify market failures and automatically consider the State as a possible solution to them (or even as having an important role in stabilizing the macroeconomic situation) without considering collective decision costs or the possibility of government failure leading to even worse outcomes. One obvious point developed by Buchanan and Tullock lay in the tyrannical aspect of majority rule without logrolling: government decisions did not reflect the will of the citizens in that, for instance, they did not allow a minority to express the intensity of their preferences. Additional inefficiencies would result from rent-seeking and bureaucratic behavior, majority cycles, and rational abstention.

The inefficiencies pointed to by these early studies in public choice only compounded the problems for collective choice flowing from Arrow’s impossibility result. This pessimistic conclusion of two decades of research was well-summarized by Mueller (1976, p. 424) in his landmark survey of the Public Choice literature: “the positive literature [was] riddled with demonstrations of the instability, inefficiency, or irrationality of various voting outcomes; the normative literature by impossibility proofs.” Moreover, by systematically applying the assumption of rational self-interested agents to the study of how individuals interacted within groups, these economists opened many black boxes—e.g., governmental bodies, parties, and clubs, but also firms, trade unions and other small and large groups. The heterogeneity of individual values revealed through this work strengthened the position of those critical of the view that the group as a whole could be represented by a single objective function to be maximized.¹⁷ These economic analyses were draining the relevance of resorting to some set of commonly shared values, of notions such as the “public interest”. In contrast to economists’ tendency to stress the similarities between individuals’ values in the previous decade, this new generation of studies emphasized conflict as a pervasive characteristic of societies. This shift reflected the mounting social unrest and discontent that would mark the late-1960s.

2.2 Government choices reflecting citizens’ choices? The difficult task of applied economists

As the 1960s unfolded, conflict became a pervasive feature of the American institutional and social life. The escalating US intervention in Vietnam divided opinions sharply. The country also realized progressively that in the midst of plenty and affluence, a significant fraction of the population had been left behind. Racial unrest was not far away: although desegregation had started in 1955, sometimes violent civil rights-related protests continued to mount, and the 1964 Civil Rights Act divided the country. Urban riots soared and civil disorder made the headlines. From the mid-1960s on, large numbers of college students joined the “movement”, which provided academics with a first-hand view of society’s tensions, as unrest disrupted the peaceful order within university campuses (Anderson 1995). Finally, the consensus on policy ends that characterized the debates among economists in the 1950s began to unravel as they argued about the proper goals for macroeconomic and microeconomic policies more openly than a decade before. Debates between Friedman on the one hand, and James Tobin, Robert Solow, and various Keynesians on the other, highlighted the disagreement over the importance of pursuing long-term efficiency as against short-term stabilization, and whether taming inflation was a realistic goal for the Federal Reserve.

In this time of growing discontent, Lyndon Johnson’s Great Society program, the cornerstone of which was the War on Poverty, involved rethinking public policy. Conflict and unrest created tensions at the theoretical as well as applied levels: “all kinds of social scientists, practicing social workers and the like did not seem to agree on the diagnosis; they certainly didn’t agree on the cures. It was quite clear that poverty had… many faces, that to talk of some cutoff below which everyone was a glob had no programmatic meaning at all because you were talking about widely disparate groups, and that a single magic answer was not to be found,” reflected CEA member William Capron (quoted in Jardini 2013, p. 154). This context proved favorable to the development of important policy innovations designed to follow as closely as possible the preferences of individuals.

One innovation came out of the Community Action Program (CAP), whereby local communities would be empowered to design their own experimental attacks on poverty, which subsequently provided bases for legislative action. This innovative bottom-up process, though, soon was perceived by Johnson as conflicting with the centralized guidance of economists at the Bureau of Budget and Department of Defense. To many, the empowering actions of the CAP encouraged protests and, overall, had destabilizing effects. Thus, Johnson approved the development of yet another important innovation for public decision making, the Program Planning and Budgeting System (PPBS; see Fleury 2010). Drawing on systems analysis developed by the economists Charles Hitch and Roland McKean, PPBS promised that a value-free conception of policies was possible. It notably reorganized departmental budgeting in terms of goals instead of inputs, goals that were given to executives (heads of federal agencies), allowing for a thorough cost-benefit evaluation of various government programs. Johnson’s War on Poverty, then, “virtually created a new and well-funded discipline: policy analysis” (Jardini 2013, p. 302). The many economists summoned to identify solutions for poverty or the urban and environmental crises, effectively were forced to grapple with questions of how to legitimize the normative criteria underpinning their policy choices at a time when relying on some notion of “public interest” or “value consensus” seemed less and less possible. The ongoing efforts to disprove Arrow’s impossibility theorem and shortcomings suggested by public choice analysis of virtually all theoretical and existing collective decision-making procedures did nothing to lift the mounting burden weighing down on those applied economists.

They did not surrender, though. A number of them looked into possible bridges between theoretical economics and applied work. A characteristic example is the 1966 conference on the “analysis of the public sector” organized by Julius Margolis and Henri Guitton in Biarritz. It combined theoretical contributions by Samuelson, Musgrave, Marglin, Amartya Sen, Edmond Malivaud and Robert Dorfman with presentations of policy implementations by Lionel Stoleru, Hollis Chenery and V.P. Gloushkov. Margolis’s introduction to the resulting 1969 volume, Public Economics, made it very clear that discussion largely had centered on the choice of an objective function to guide policy makers’ decisions, and the process whereby those decisions were implemented. He contrasted two conceptions of the government’s objectives: one he associated with political scientists, sociologists and applied economists, in which the government was considered as an independent social body endowed with its own view of the public interest, and another, upheld by welfare economists, in which the government’s objective was represented as a social ordering deriving from the aggregation of individual preferences. He added immediately that the distinctions between the two approaches can be blurred “if one believes that the political process is a mechanism by which individual preferences are aggregated,” and becomes stronger “if one views the political process as a distinctly different form of resource allocation where the motive force is a view of public interest.” Not everyone agreed: some economists, Guitton reflected, thought that it was their task to formulate citizens’ values by the “assignment of value weights which would have been revealed by the market behavior” in the absence of market failure.

Those debates also permeated the research on cost-benefit analysis (CBA). As the Water Resources Council attempted to redefine its procedures for the evaluation of water and land resources projects in the late 1960s, two visions of CBA were pitched against one another. Robert Haveman and A. Myrick-Freeman from Resources for the Future believed the multiple objectives in water resource management could be collapsed into a single social welfare function grounded in individuals’ preferences. Also, they doubted that the political process provided a reliable way to define these objectives, as it was subjected to “pork-barrel, logrolling, and empire building.” Yet Dorfman, Otto Eckstein and other economists associated with Harvard’s Water Program supported a multiple objective approach to CBA, for those objectives (food production, regional development, flood protection, health) essentially were irreconcilable. It was, ultimately, the political process that should decide on the weighting: “in no event should the technician arrogate the weighting of objectives to himself”, Eckstein (1961, p. 449) argued. The former group thus emphasized consumer sovereignty, while the latter promoted a vision of political sovereignty in which elected officials represented the collective choice of citizens (Banzhaf 2009). Similar debates took place in urban economics, which explained why MIT welfare economist Jerome Rothenberg (1967, pp. 21–22) tried to revamp the CBA techniques used to evaluate urban renewal programs as the reflection of a democratic decision process. He argued that each homogenous cluster of American citizens should be weighted in the social welfare function in proportion to its number of representatives elected in the last congressional election, so as to relate “the social expression of value judgments about distribution” to “central decision making process […] in a representative democracy.”

During the 1960s, in theoretical developments as in practice, a number of economists wondered whether taking into account citizens’ preferences in the choice of a policy was enough, or whether they needed genuinely to derive public decisions from individual preferences. Whether government decisions could or did reflect the outcome of a collective decision process became a pervasive question, but no definitive answers were given. Normative and positive analyses identified many flaws related to collective choices: public decisions usually were influenced by vested interests rather than citizens’ will, policy objectives were irreconcilable, and it was not clear that the resulting costs of public intervention were less than those associated with market failures.

3 Becoming mature, becoming estranged: the fragmented institutionalization of collective decisions

In the 1970s, economists continued to investigate problems involving voting cycles, strategic voting, free-riding, rent-seeking, and other related shortcomings of political processes. Buchanan’s contractarian approach increasingly was infused with pessimism, as his own experience of the turmoil of the 1960s had led him to reject his previous belief that positive-sum institutional processes were the norm. “Zero-sum and negative sum analogues yield better explanation”, he now observed (Buchanan 1975). And the domain of pessimism was only expanding, as economists introduced public choice intuitions into areas such as industrial organization (see for instance Stigler’s work on regulation) and macroeconomics (Nordhaus 1975 on political business cycles). Those works nurtured doubts that microeconomic and macroeconomic policies could be considered as the indirect result of citizens’ collective decisions, and gave credence (in the minds of some) to proposals favoring market solutions. At the same time, however, economists began to make headway in resolving many of the supposed impossibilities identified in the previous decades, and in developing empirical techniques to implement collective decision mechanisms.

3.1 Theory: a way out of collective decision shortcomings?

By the mid-1970s, several independent research programs were converging to provide a solution to Samuelson’s “revelation” impossibility as well as Arrow’s “aggregation” problem. Leonid Hurwicz’s (1972) statement that decentralized mechanisms failed to provide efficient revelation mechanisms even in the case of private goods spurred research on “incentive compatibility.” That also was a problem explored by Vickrey (1961) in proposing his single-item, second-price bid auction, as well as by Becker et al. (1964) mechanism for making decisions by teams. Theodore Groves (1970) completed a dissertation on the design of efficient incentives in Marschak and Radner’s team setting and worked on the game-theoretic representation of a class of mechanisms in which participants were incentivized to truthfully reveal their preferences when forced to bear the marginal costs of their action through transfer payments. At Northwestern, Groves collaborated with his student Martin Loeb and with John Ledyard to extend these revelation mechanisms to the identification of the optimal taxation scheme for the provision of public goods—a topic addressed independently by Edward Clarke (1971) at Chicago (see Plassmann and Tideman, this volume, for extensive discussion of and references to this literature). Such was the scattered and fluid nature of this strand of research that only gradually did these researchers become aware that they were working on the same set of mechanisms, and that they could be framed as a “new and superior process for making social choices,” as the title of a 1976 survey by Tideman and Tullock put it.

Other important breakthroughs came from Caltech, as John Ferejohn and Morris Fiorina addressed the paradox of voting by showing that agents needed little incentive to vote if their maximization-of-expected-utility decision rule was changed to a minimax regret decision. Around the same time, Sen (1970) argued that Arrow’s impossibility theorem could be avoided if the informational basis on which social choice relied was enlarged beyond Arrow’s, to include non-utilitarian information such as the intensity of preferences and distribution of utility, and ordinal as well as cardinal interpersonal comparisons of utility. He showed formally that the Pareto criterion cannot be upheld if individuals are endowed with minimum liberty, opening the door to the study of individual rights and justice issues in collective choice. The conjunction of these new theoretical insights offered by Sen, Clark, Groves, and others, launched a flourishing literature spanning issues of domain restriction and manipulation.

3.2 New economic expertise meets new public and private demands

The 1970s’ intellectual milieu that had fostered the study of new collective decision mechanisms also provided new insights into how they could be tested and implemented using experiments. This move from theory to experimental testing was facilitated by institutional connections: Vernon Smith, Charles Plott, Hugo Sonnenschein, Morton Kamien, and Thomas Muench all were associated with Purdue University in the 1960s. A number of them moved subsequently to Northwestern University, a hothouse for the development of mechanism design in the 1970s that hosted Ledyard, Groves, Satterthwaite, Ehud Kalai, Roger Myerson, Bengt Holmstrom and Paul Milgrom. Plott moved to Caltech in 1971 and made the department the central spot of experimental economics, notably by recruiting John Ferejohn, Morris Fiorina, Peter Ordeshook and many others.¹⁸

A notable breakthrough came from the series of experiments on alternative mechanisms to provide public goods that Smith and Plott ran at Caltech around 1973, and also from Smith’s subsequent test of the Ledyard-Groves mechanism in a classroom experiment at Northwestern. The experiment suggested initially that agents do not behave according to the theory, but Smith was able to match theoretical and observed behavior by altering the information structure of the experiment. The realization that slight changes in procedures and institutional constraints imposed on individual strategies induced major changes in outcomes prompted experimentalists to develop extensive contributions to “institutional design.” Soon, the combination of experiments and theoretical work was advertised as having the potential to yield huge improvements in the efficiency of those collective decisions grounding public policies, as seen in the 1985 report of the Market and Organization working group (Lee 2016). In response to the Reagan administration’s plan to slash the National Science Foundation’s budget for its Division of Social and Economic Science by 75%, economists, led by Stanley Reiter, established a priority list of topics that emphasized the promises of interactions between mechanism designers and public choice-oriented experimental economists, illustrated in particular by the Walker mechanism. The trend was strengthened further by the computerization of experimental laboratories, coupled with the contiguous move away from axiomatics and towards Nash game theoretic and Bayesian game theoretic frameworks (see the opening statement of Roth 2002).

Since the bulk of this blend of theoretical and experimental analyses of collective decision making explicitly was aimed at tackling real-world mechanisms and policymaking (Guala 2001), its effective implementation was predicated upon the demands of clients—firms, unions, regulatory agencies and policy-makers. And the policy regime framing those clients’ demands was itself shifting—specifically, towards mechanisms that emphasized individual incentives and market-based solutions. Although Richard Nixon had eliminated PPBS management in 1970, economists were still asked to work on new policy tools that would supply a better balance between market and government allocation, and a better estimation of the costs and benefits of regulations, especially during the Ford administration. Likewise, Carter’s CEA chair, Charles Schultze, campaigned for a government intervention favoring market-like incentives over “command-and-control” techniques. As explained by Berman (2016), the Office of Information and Regulatory Affairs (OIRA), created to promote Carter’s approach to regulation and market failures, soon turned into the armed wing of Ronald Reagan’s “regulatory relief” policy.

These changes in policy regimes generated a demand for microeconomic tools. In 1981, Reagan signed an Executive Order providing that “regulatory action shall not be undertaken unless the potential benefits to society from the regulation outweigh the potential costs to society.” Federal agencies thus were required to provide cost-benefit analysis of regulations, which the OIRA would review. Economists’ mandate had thus widened gradually from macroeconomic stabilization to the delivery of public goods, the choice of regulations, and the associated implementation of efficient ways of making collective decisions. One such example was the work that David Grether, R. Mark Isaac and Plott conducted on the allocation of airport slots in the wake of the Airline Deregulation Act of 1978 (see Svorencik 2017 for a full account). Instead of allocating slots based on unanimous consent, the recommended replacement mechanism was based on a sealed-bid, one-price auction with a continuous aftermarket of “block transactions.” Ferejohn, Roger Forsythe and Roger Noll likewise devised a mechanism to induce payment for public television programs, and experiments also were used to propose new mechanisms for regulating the pricing of natural gas transmission, the allocation of resources on a space station, and inland waterways transportation. The 1994 Federal Communications Commission’s radio spectrum auction, which became the poster child for the success of mechanism design, also underwent an experimental test stage before being implemented (Roth 2002; Guala 2001). Conducted by the Caltech team, it led to the choice of the ‘simultaneous ascending-bid auction’ design favored by Paul Milgrom, Robert Wilson and Preston McAfee, over the combinatorial auction plus sealed-bid package auction proposed initially by the FCC.

3.3 A fragmentation of collective decision research in economics?

While the previous section suggests that a combination of theoretical and experimental breakthroughs, as well as technical advances and political transformations, definitely furthered the study of collective decisions, other forces acted towards fragmentation and marginalization. First, the development of the new policy regime exerted a contradictory influence. That economists acted as sellers of expertise to many institutions (Breslau 1997) reinforced the tendency to consider them as engineers whose role was to provide the means of fulfilling given aims rather than thinking more broadly about how these ends were constructed. Though Maskin (2008) described market design as a case of reverse engineering in which the expert begins with “identifying our desired outcome or social goal” and then proposes a mechanism to attain that goal, the reality is that goals were defined for the economist, outside of the model. This, then, obviated the need to be concerned with the collective decision process. As a consequence, with the exception of those aforementioned cases when the mechanism to be designed was related to voting schemes or public goods allocation, the notion of collective decision moved to the background of economists’ minds.

Even the study of voting and public goods allocation increasingly was fraught with difficulty. Theoretical dead-ends in the development of incentive-compatible mechanisms were pointed to already in the 1970s. Hurwicz had explained that, under standard conditions, those mechanisms failed to satisfy Pareto-optimality conditions. Allan Gibbard and Mark Satterthwaite independently showed that when at least three alternatives exist, the only unanimous, strategy-proof social choice function is a dictatorship. Maskin had demonstrated that no single-valued social choice rule was Maskin-monotonic, so that voters’ strategic behavior systematically would produce suboptimal Nash equilibria. And Jerry Green and Jean-Jacques Laffont proved that Vickrey-Clarke-Groves (VGC) mechanisms often were not budget-balanced. The shift to Bayesian game theoretic models as well as the relaxation of dominant strategy and the single-value requirement made it extremely difficult simultaneously to satisfy the requirements of Pareto optimality, voluntary participation and strategy-proofness, and often created indeterminacy. The development of experiments added another layer of negative results. The computation of some equilibria was found Non deterministic polynomial-time complete, outcomes were not budget-balanced and could be manipulated by coalitions, bid preparation costs precluded the existence of dominant strategy equilibria, and so forth.¹⁹ Overall, the decisions of governmental agencies did not need to be interpreted as a result of consistent citizens’ choices, while debates on free-riding, revelation and strategy-proof mechanisms often focused on coordination, information and allocation issues, not collective decision-making per se, becoming more technical in the process.

The transformation of public economics in the 1970s likewise contributed to the generalization of economic models in which the process whereby agents agree on social goals was considered exogenous. Following Diamond and Mirrlees’s (1971a, b) seminal contribution, economists’ attention shifted to optimal taxation, which led to the rediscovery of Frank Ramsey’s (1928) approach, the development of models with a representative agent or resorting to an exogenous social welfare function in order to represent the government’s objectives. This transformation is best illustrated by the most cited work in public economics in the 1980s, Anthony Atkinson and Joseph Stiglitz’s (1980) graduate textbook entitled Lectures in Public Economics (Claveau and Gingras 2016). The book aimed at analyzing “in a systematic manner the principal consequences of… economic activities by the government and their relation to social objectives” (Atkinson and Stiglitz 1980, p. 3), and it included two chapters devoted to the discussion of governmental objectives. One of these chapters closed the first section of the book, dealing with the positive analysis of government policies, and was explicitly aimed at making “the state’s decisions endogenous rather than exogenous” by examining models in which public decisions are influenced by voters, political parties, legislators, and administrators” (p. 11). The other chapter opened the book’s second part dealing with the normative analysis of state decision. It surveyed “ways in which the objectives of the government have been formulated and the resulting criteria for decision making” and “investigate[d] the sensitivity of the policies chosen to the formulation of objectives” (p. 12). Although these chapters summarized the literature on positive and normative collective decisions previously mentioned in this paper, they largely were disconnected from the public economics scholarship presented in the book and later spread in the profession. Positive public economics was tackled within a framework in which government decisions were represented by a social welfare function of the Bergson-Samuelson type. How the function was determined was, like in mechanism design, pushed outside the scope of public economists’ analysis. Moreover, as Samuelson had attempted to rescue his work from Arrow’s attacks during the 1970s, he emphasized repeatedly that, while Arrow’s function genuinely reflected a collective ethical choice, Bergson’s reflected how a single individual took into account citizen’s preferences (Igersheim 2016).²⁰ Using a social welfare function to model the government’s objective was thus consistent with Bergson and Samuelson’s prescription, but it also shows that economists had ceased to consider public goals as reflecting a collective decision process.

The spread of the Bergson-Samuelson social welfare function did nothing to curb the marginalization of the community that had remained closely focused on the Arrovian program, e.g., the normative study of collective decision mechanisms. By the 1980s, this program had morphed into the identification of maximal strategy-proof and non-dictatorial domains with mathematicians, the analysis of justice and fairness issues through collaboration with philosophers, and the axiomatic analysis of real-life decision processes and measurement values (e.g., matching algorithms, fair division procedures). The field increasingly was considered too abstract and remote from economists’ practical concerns by journal editors, who became more reluctant to accept social choice submissions (Salles 2005). Maurice Salles and Prasanta Pattanaik responded by creating a new journal, Social Choice and Welfare to provide an outlet for this work. Meanwhile, the boundaries of public choice had extended well beyond collective decision concerns. Though constitutional political economy remained central to Buchanan’s research, it was perceived as outdated by those public choice economists who were building a strong empirical program to study the working of actual institutions (see Boettke and Marciano 2015).

4 Conclusion

The study of collective decisions may well have stabilized in the 1980s, but the topic remained quite elusive and never really became central to economic theory. The only field explicitly concerned with the positive and normative analysis of collective decisions, namely social choice, was marginalized. Public choice economists expanded their interests in a large number of directions, but only a few of them, most notably Mueller (2003) and Tideman (2006), explicitly framed their field and its core concepts—free-riding, public goods, constitutions, rent-seeking—as dealing with the collective decisions made necessary by living in society and by so-called market failures. While acknowledging the development of social and public choice, public economists largely endorsed models in which the objectives embodied in the government’s social welfare function were considered exogenous, while aggregation problems were solved with the use of representative agent models, thereby suppressing the preferences of individuals other than the hypothetical representative agent. What happened in the 1980s was not so much that the new theories and tools developed by economists ceased to be relevant to the study of collective decision making. It was, rather, that theoretical and applied investigations of revelation, coordination and allocation mechanisms ceased to be framed as studies of collective decisions because economists were not asked, by their clients, to answer questions such as “whose preferences and values do this policy reflect, or should reflect?” Those questions were taken up (or back) by scholars from political science, social psychology, law, philosophy and the new policy science that emerged in the 1960s. As Richard Tuck (2007) remarked, the publication of Rawls’s Theory of Justice in 1971 put an end to a century without grand political philosophy synthesis. And the language it was couched in, that of analytical philosophy, as well as philosophers’ development of axiomatization and the dissemination of game theoretic tools among political scientists, enabled them all to converse and compete with economists, and reclaim collective decision expertise.²¹ Nowadays, courses on collective decision are offered in these departments while the topic seems to have deserted economics curricula, a somewhat ironic twist when considering that some of the canonical contributions were the works of economists such as Duncan Black, Kenneth Arrow, Anthony Downs, and Mancur Olson.²²

This is not to say that economists’ research on collective decision has ground to a halt, as exemplified by Casella’s (2012) work on storable votes, Saez and Stancheva’s (2016) refinement of social welfare functions, or Feddersen and Pesendorfer’s (1996) investigation of strategic voting. Quadratic voting is one major instance of innovative attempts to relate theoretical innovation, mathematical formalism, concrete applications and implementation. From an historical perspective, it belongs to the Ledyard-Groves-Hylland-Zeckauser theoretical tradition of devising strategy-proof mechanisms in which voters have to pay for the costs of their choices (Plassmann and Tideman, this volume). It also incorporated at a very early stage the experimental designs aimed at testing how the voting mechanism performed against alternatives (Lalley and Weyl 2016). Finally, its development makes it the latest instantiation of our claim that economists’ thinking about collective decision is rooted in concerns with applications, both public and private. Quadratic voting originally was conceived in reaction to the shortcomings of eminent domain in allocating land, then framed as a solution to the more general issue of the tyranny of the majority in Posner and Weyl (2014), and several contributions to this special issue examine how it could be implemented in local elections or court decisions on patents.