1 Introduction

The thesis of this paper is that the scholarly field of industrial organization is shaped and directed by the debate concerning some of the most pressing policy issues at any historical time period about the link between the organization of industries and economic performance. By industrial organization, we mean the scholarly field of research as defined by Scherer (1970, 1980) and Scherer and Ross (1990). By the organization of industries, we mean the actual way in which economic activity is organized within the unit of observation of an industry. As Scherer (1970) explained in his path-breaking book, Industrial Market Structure and Economic Performance, the first stirrings of industrial organization as a field came as a response to the emergence of the trusts of the late 1800s and their perceived adverse impact on performance criteria such as prices and profits.

Not only were the trusts alleged to have demolished family businesses, farms in the Midwest and entire communities, but also the public policy debate at the time accused them of threatening the underpinnings of democracy in the United States. In arguing for the passage of the 1890 Act, Senator Sherman argued, “If we will not endure a King as a political power we should not endure a King over the production, transportation, and sale of the necessaries of life. If we would not submit to an emperor we should not submit to an autocrat of trade with power to prevent competition and to fix the price of any commodity”.Footnote 1

As Scherer (1970) also made abundantly clear, the field of industrial organization solidified in the 1930s when there was concern that prices were not downwardly flexible. Scholars in the field (Scherer 1970, chapter 12, pp. 284–303) developed theories of administered pricing to explain the power of large corporations to maintain price levels even in a regime of falling demand. As Scherer also explained, a series of studies in the field of industrial organization were undertaken attempting to link price flexibility to the degree of concentration, which suggested that the Great Depression may have been prolonged by large corporations exerting market power to main price levels.

A central focus of Industrial Structure and Economic Performance (Scherer 1970) was on the relationship between firm size and productive efficiency. One of the central policy concerns throughout the post-World War II era was the perceived military and economic competition with the Soviet Union. Nikita Khrushchev had squarely provoked the United States with the provocative challenge that the “growth of industrial and agricultural production is the battering ram with which we shall smash the capitalist system.”Footnote 2 The Soviet Union, thanks to its system of centralized planning and production, could enjoy the efficiency gains accruing from large-scale production without worrying about any deleterious effects due to the shortcomings of market competition.

This left public policy in the United States as throughout the west with a dilemma. The commitment to market competition might place limits on permissible firm size and market concentration, which could compromise technological efficiency and productivity in this perceived national competition versus the Soviet Union. At the same time, maximizing efficiency and productivity to ward off this “red scare” might result in levels of market concentration not commensurate with competition, low prices and allocative efficiency.

Perhaps it was the ability and relevance in addressing one of the most pressing public policy issues of that era that led to the ascendance of industrial organization as one of the most important and recognized fields of economics during the Zeitalter in which Scherer published the three editions of his book. In organizing and assessing a massive literature examining those very issues, Industrial Market Structure and Economic Performance provided a framework for identifying and analyzing this tradeoff and for framing the public policy response in terms of antirust or competition policy, regulation, and public ownership to deal with this tradeoff (Williamson 1968). In particular, Industrial Market Structure and Economic Performance spans and synthesized a broad spectrum of painstaking and meticulous empirical studies to identify how much market concentration existed and how it was evolving, along with its impact on economic performance.

Long before the cold war was resolved, a different policy challenge emerged. The OPEC price shock for crude oil in October 1973 triggered unprecedented waves of simultaneous increases in unemployment and inflation, or what become known as “stagflation” throughout the remainder of the decade. The field of industrial organization was where thought leaders in business and policy again turned for solutions. In rising to the call, scholars in industrial organization responded by resurrecting the Administered Pricing thesis and undertook a wave of studies to determine the impact of market power on price increases.

The stagflation of the 1970s gave way to a new pressing policy issue in the 1980s: the loss of competitiveness in the traditional manufacturing industries to countries such as Germany and Japan (Derouzos et al. 1989). As American corporations responded with massive waves of downsizing, and employment levels fell in the traditionally strong industries such as autos and steel and tires, the public policy debate shifted from constraining large corporations towards enabling more success in terms of creating sustainable, high-paying jobs (Thurow 2002).

In this paper we examine how and why the field of industrial organization generated so much interest and excitement during the era corresponding to publication of the three editions of Scherer’s seminal work, Industrial Structure and Economic Performance. The paper concludes that it was the ability to address the compelling issues of its era that confronted public policy that made the academic field of industrial organization not just interesting but also highly relevant and timely. In particular, by analyzing how and why the organization of industries matters for economic performance, along with the various policy approaches available to public policy, under the stewardship of F.M. Scherer the scholarly field of industrial organization prospered and flourished.

2 Industry Structure and Economic Performance

Prior to the Civil War in the United States, production—whether manufacturing or agricultural—was typically at a small scale, which reflected the limited markets inherent in geographically isolated communities (Kolko 1965). As Chandler (1977) documents, the minimum efficient scale (MES) level of production was typically at low levels of output. The effort to win the Civil War changed all that. New transportation networks—Principally, the railroads—created regional and even national markets. New technologies ushered in large-scale production using assembly lines.

The technological revolution facilitating the emergence of large-scale production also required new managerial techniques, most predominantly “command and control of effort.” As Frederick Winslow Taylor (1911) put it in The Principles of Scientific Management, workers could be transformed into a commodity analogous to other inputs of production, enabling a new and unprecedented division of labor, “The science of handling pig iron is so great and amounts to so much that it is impossible for the man who is best suited to this type of work to understand the principles of the science, or even to work in accordance with the principles without the aid of a man better educated than he is”.Footnote 3

The evolution and development of this triad following the Civil War—the advent of regional and national geographic markets rather than isolated local markets; new technologies enabling large scale production; and new management techniques to facilitate an unprecedented extent of the division of labor—increased the level of the minimum efficient scale of production. Large-scale production emerged with a marked competitive advantage vis-à-vis the small-scale, family owned businesses, which were traditionally rooted in their communities. The result was not just the rise of the “trusts” in the second half of the nineteenth century but also the demise of small, main street business.

Upton Sinclair’s (1905) sensational book, The Jungle, depicted the erosion in the competitiveness of traditional small-scale business and the emergence of large-scale, and considerably more efficient—if not ruthless—“trusts” in the meat-packing industry.

Confronted by an existential threat, the formerly prosperous small-scale farmers, businessmen, and merchants in Midwestern towns, villages, and rural regions turned in desperation to politics. First the Granger and subsequently the Populist movements emerged with a broad and widely supported mandate from the electorate to restrict the power of big business in general. Widely enjoyed freedoms by private firms to contract were no longer working in a positive way for much of the country. It was this demand to restrict the freedom of firms to contract, as articulated by the Granger and Populist movements, which voiced the disgruntled concerns of affected small businesses and consumers, that ultimately turned to government with a mandate to constrain the power of big business.

As a result of the populist mandate for the government to intervene to constrain the perceived unmitigated power of big business, new and unprecedented types of regulations were imposed on business. When the State of Illinois enacted a new law to regulate the rates that were charged by grain elevators and warehouses, one of the affected grain elevator owners, Munn, petitioned the Supreme Court. Munn maintained that the statute violated the Fourteenth Amendment to the U.S. Constitution because it deprived him of potential profits, which constituted private property. In Munn v. Illinois,Footnote 4 however, the Supreme Court ruled in 1877 that such regulation of private property did not violate the Fourteenth Amendment, because the product was affected with the public interest (Scherer 1970, p. 519).

The large corporations, dominant in their industries, or the trusts, as they were referred to during that era, may have been able to manage production at a scale that had previously been unthinkable but were less able to shape the external environment in which they did business. The lessons and insights of Taylor’s (1911) scientific management had little to say beyond the boundaries of the corporation. In an economy that was subject to considerable macroeconomic shocks and demand volatility, even the trusts seemed at the mercy of forces beyond their control (Scherer 1996, p. 153, for steel; 1970, p. 196, for railroads). As Scherer (1970, pp. 192–198) pointed out, the very factor that facilitated large-scale production—physical capital—also rendered the large corporation vulnerable to market volatility. The high level of fixed costs that emanated from the requisite costly investments also required commensurate high levels of capacity utilization to cover those costs. In addition, investments in physical capital were vulnerable to technological obsolescence through innovation. New technologies would benefit the early adaptors; but a company with sunk costs in an obsolete technology was burdened by a severe competitive disadvantage.

In an effort to salvage at least some of the variable costs and ward off disastrously low levels of capacity utilization, large-scale manufacturing companies in some cases resorted to cut-throat pricing. As Scherer (1970, p. 449) made abundantly clear in his quotation of the frontispiece of Eddy’s (1912) The New Competition: “Competition is War and War is Hell”. Why this pricing strategy earned the graphic designation as being predatory is explained by Kolko (1963, pp. 30–31), who quotes upper management at the American Tobacco Company: “Unrestricted competition had been tried out to a conclusion, with the result that the industrial fabric of the nation was confronted with an almost tragic condition of impeding bankruptcy. Unrestricted competition had proven a deceptive mirage, and its victims were struggling on every hand to find some means of escape from the perils of their environment. In this trying situation, it was perfectly natural that the idea of rational cooperation in lieu of cut-throat competition should suggest itself.”

Large corporations did not simply succumb to externality volatility. Rather, in an effort to stabilize prices, they turned towards industry-wide agreements, or in some cases outright collusion to either restrict output or maintain price levels. Industry trade associations typically provided a viable platform that facilitated such agreements among otherwise competing companies. One poignant example involved the Bessemer Pig Iron and the Bessemer Steel Associations: Dating to the 1880s, the Associations consisted of some 700 blast furnace, steel work, and rolling mill companies. Still, it took more than a trade association to stabilize prices when confronted by market volatility, as was evidenced by substantial price declines for most steel products during the downturn of 1894–1895.

A different strategy to stabilize market volatility involved consolidating companies throughout the industry. Nelson (1959) documented the extent of massive waves of consolidation in the steel industry towards the end of the nineteenth century and early years of the twentieth century. Most notably, U.S. Steel was the result of massive acquisitions and consolidations, emerging as one of the largest and most powerful corporations of its era. However, Kolko (1963, p. 27) concludes that even industry consolidation ended with frustration at an inability to stabilize external market volatility adequately, “The new mergers, with their size, efficiency, and capitalization were unable to stem the tide of competitive growth. Quite the contrary. They were more unlikely than not unable to compete successfully or hold on to their share of the market.” Kolko’s observation that consolidations and acquisitions fueled the creation of a large, dominant company apparently did not, however, equip that firm with the ability to mitigate or compensate for market volatility. The United States Supreme Court seemingly reached the same determination in observing that “Size alone is not an offense, “in U.S. v. U.S. Steel Corp.Footnote 5

Thus, in an era that was characterized by decreasing costs—triggered by the emergence of the trusts—corporations had an unprecedented scale of production and (the Supreme Court’s perspective of U.S. Steel notwithstanding) market power. Perhaps for the first time, attaining maximum efficiency seemed to be squarely at odds with notions of the Jeffersonian vision of democracy and capitalism. Instead, economic policy centered on what (no doubt) was perceived to be a dismal tradeoff. On the one hand, small-scale production and ownership could be sacrificed to attain efficiency; on the other hand, traditional decentralized and small-scale production could be maintained but only by sacrificing efficiency and lower costs of production.

Marx (1912) clearly gleaned that this tradeoff resulted in an inherent incompatibility of capitalism with democracy. Efficiencies and lower costs that accrued from large-scale production would inevitably drive less efficient smaller companies out of the market in a process of centralization and concentration of economic power, “The battle of competition is fought by the cheapening of commodities. The cheapness of commodities depends, ceteris paribus, on the productiveness of labour, and this again on the scale of production. Therefore, the large capitals beat the smaller.”Footnote 6

Schumpeter (1942) found a similar inherent tension in capitalism. Scherer (1992, p. 1416) sifted through Schumpeter’s seminal work and pointed out that “Capitalism, Socialism and Democracy had three main themes: the spectacular success of capitalism in generating economic progress, an analysis of how that success came about, and the perceived ‘march’ of capitalist economies into socialism.”

The prevalent economic doctrine during the Great Depression of the 1930s looked to price declines to reinstate purchasing power and ultimately find the way back to economic prosperity. It did not happen. While real output in the United States declined by one-quarter, prices did not fall to any significant degree. In an effort to explain the prolonged duration of the Great Depression, Keynes (1936) developed a new theory of why prices are downward rigid. Other scholars, such as Joan Robinson (1933), looked to the organization of industry as the source of rigid prices. One theory posited by the industrial organization scholars—the administered price thesis—explained the downward rigidity of prices as a result of oligopolistic pricing markets that were highly concentrated. As Scherer (1970) pointed out wholesale prices did fall about 30% between 1928 and 1933. Also, the prevalent political doctrine as of 1933 was that falling prices were part of the problem of the Great Depression and that higher prices could be the cure. Hence, the NRA specifically allowed producer cartels to form (Scherer 1970).

The point here is not to support or refute the validity of this thesis, but to emphasize that the field of industrial organization was once again doing what it did best—addressing one of the most pressing issues confronting public policy of that Zeitalter.

More than a few scholars and thought leaders in business and policy were pleasantly surprised as the Great Depression faded following the Second World War. However, a new problem emerged from a very different direction: the cold war. The perceived threat to the U.S. and her western allies was not just in the military sphere but also in terms of the economy. Again it was the scholars of industrial organization who led the way in economic thinking about how best to understand, analyze and address this new challenge.

It was during this post-war era when Solow (1956, 1957) published his path-breaking work that identified the key to economic prosperity and growth: the technological change and improvements in the quality of the labor force that accompanied the investments in physical capital. At the industry and firm levels, scholars such as Chandler (1977, 1990) and Bain (1956) showed that it was not just the amount of capital in an economy but the manner in which industrial production was organized that influenced economic performance.

The Soviet Union, with its centralized economic planning and industrial combines, where production was purposefully concentrated into just one organization, seemed to have an advantage in both of these two key dimensions: the ability to invest in physical capital and to organize that capital to maximize efficiency and productivity. Rosenberg (1992, p. 197) explains how the “gigantism embedded in Soviet doctrine” was thought to generate a superior and more competitive economic performance.

The challenge inherent in the organization of industries was not lost upon the great scholars of the day. For example, Schumpeter (1942, p. 134) echoed Marx in predicting that the inherent competitive advantage that accrued from large-scale production and scale economies would lead to an inevitable march towards increased concentration in the industrial structure and organization of most industries, “Since capitalist enterprise, by its very achievements, tends to automatize progress, we conclude that it tends to make itself superfluous—to break to pieces under the pressure of its own success. The perfectly bureaucratic giant industrial unit not only ousts the small- or medium-sized firm and ‘expropriates’ its owners, but in the end it also ousts the entrepreneur and expropriates the bourgeoisie as a class which in the process stands to lose not only its income but also, what is infinitely more important, its function.”

In Industrial Structure and Economic Performance (1970), Scherer meticulously synthesized and documented a plethora of studies that found that the levels of concentration had systematically increased in both individual markets as well as for the overall economy, or what was termed to constitute aggregate concentration. While inferring long-term historical trends based on data that was incomplete and not without flaws and was tenuous at best, after reviewing the empirical evidence Scherer (1970, p. 44) concluded that, “Despite [the statistical] uncertainties, one thing is clear. The increasing domestic dominance of the 100 largest manufacturing firms since 1947 is no statistical illusion.”Footnote 7

Not only was the share of economic activity accounted for by the largest corporations, or the extent of aggregate concentration as well as concentration in individual markets, increasing over time, but analogous empirical evidence identified a marked decline in the role that was played by small- and medium-sized enterprises. Scholars in the increasingly important academic field of industrial organization found similar trends in the underlying organization and structure of industries that drove economic performance in the U.S. as for the Soviet Union. As Scherer (1970, 1980) explains, the major industries that served as the engine of American economic success—such as automobiles, steel, tires, chemicals, aluminum, and later computers—were all characterized by an oligopolistic market structure that consisted of just a handful of dominant firms, which resulted in high and increasing rates of concentration.

Since large corporations operating in highly concentrated markets seemed to be not just the key to efficiency and productivity but also to matching the perceived economic threat from the Soviet Union, the social, political, and institutional environments adapted to support them by providing complementary and ancillary inputs and services. At the same time, the concern with and suspicion towards unmitigated and unchecked economic power, which dated to the founding of the country, remained vigilant. A series of Congressional Hearings—along with bold decisions that were reached by the U.S. Supreme Court—confirmed the country’s commitment to economic decentralized decision making as a foundation for decentralized political decision making.

In the 1950 Study of Monopoly Power, the Committee on the Judiciary of the U.S. House of Representatives expressed considerable concern about the trends towards increased economic concentration. A little over a decade later, the U.S. Senate held hearings that addressed the negative impacts of economic concentration, which resulted in the publication of Economic Concentration in 1964. After reviewing the Congressional testimony that debated the efficacy of enacting the Celler–Kefauver Amendment to the Clayton Act, Markham (1965, p. 166) concluded that, “Whatever else Congress may have had in mind when it amended that statute, it is clear from the Senate and House reports on the bill that one of its purposes was to check the rise of market concentration.”

A series of decisions that were handed down by the U.S. Supreme Court also reflected a concern about economic concentration: For example, in US v. Aluminum Co. of America, the opinion, on behalf of and with the authority of the Supreme Court, ruled that “Congress…did not condone good trusts and condemn bad ones; it forbade all.”Footnote 8 Thus, the inference of the Supreme Court seemed to be that monopoly power—and therefore a violation of Section 2 of the Sherman Act—would be inferred, unless the defendant could persuade the Court that the market power in question was “thrust upon it” due to “superior skill, foresight and industry.”

The Supreme Court took an equally harsh stance against mergers that might contribute to increases in market concentration, as evidenced by the decision handed down in 1962 by the Supreme Court.Footnote 9 The Brown Shoe v. U.S. ruling rendered large mergers by competitors within the same industry—horizontal mergers—to be virtually illegal per se. Until the Supreme Court back peddled on its strict interpretation of the amended Clayton Act in 1974,Footnote 10 acquisitions and consolidations among competitors in the same industry with any significant market share were unlikely to make it through the enforcement process. As the Supreme Court explained in handing down its decision, “We cannot avoid the mandate of Congress that tendencies toward concentration in industry are to be curbed in their incipiency.”Footnote 11

The same stringent standard was imposed by the Supreme Court in its rulings with respect to acquisitions that involved product extensions and geographic extensions.Footnote 12 An analogous stringent standard was attempted, at the U.S. Federal Trade Commission (FTC) in cases that involved tacit collusion, or shared monopoly.Footnote 13 Scherer (1979) analyzed the novel theory of the case: In particular, Scherer’s (1979) article explained how the behavior of the defendants, in filling up all the niches in the product space, created barriers to entry and sustained the market power of the incumbents.

The Bureau of Competition at the FTC brought the case, attempting to push the frontier with respect to tacit collusion beyond the established treatment under Section 1 of the Sherman Act that required conspiracy among the defendants for a violation. The FTC case here was an attempt to establish that a conspiracy was not needed for a group of firms together to violate Section 5 of the Federal Trade Commission Act.; however, the Commission’s own Administrative Law Judge (ALJ) dismissed the case with a long opinion that ended with the observation that despite all of the evidence presented, nowhere in the complaint had a conspiracy been alleged. The ALJ would have none of the Commission’s new view of Section 5, and ruled—after going through all of the evidence and testimony amassed in the trial—that it was obvious that if the complaint had intended to specify conspiracy, it could have easily done so; it did not, so the case was dismissed. When it came to vertical restrictions on retailers as part of the contract with manufacturing companies, the Supreme Court took an equally dim view.Footnote 14

The scholarly field of industrial organization provided the framework and thought leadership for government policy to constrain market power and concentration using the instrument of antitrust during the post-World War II era. The vigor of antitrust enforcement during this period reflected a broad consensus that economic concentration—at both the market and aggregate levels—needed to be curbed in order to preserve the decentralized economic decision-making that is the cornerstone of democracy.

At the same time, there was the growing Angst about the ability of the country to economically compete against the perceived economic threat from the Soviet Union. Industrial organization emerged as one of the most vital and prominent fields in economics during this era because it was addressing some of the pressing issues of the day. Industrial organization mattered because the scholars, led by Scherer (1970), made it clear that the organization of industries matters. It was the scholars of industrial organization who were able to shed light on the choices that confronted public policy with respect to the trade-off between democracy and economic decentralization on the one hand and efficiency and productivity on the other.

The choices that were available to policy makers were perhaps most succinctly articulated by Williamson (1968). His model carefully depicted that productive efficiency might be enhanced by allowing mergers between competitors to facilitate productivity gains through attaining scale economies. However, he also analyzed the increased concentration and loss of competition to achieve those gains. Due to the static nature of his model, it was possible to present and articulate the choice that confronted public policy but not to find a way to attain both. Still, the scholars of industrial organization were able to frame, measure, and analyze this policy trade-off concerning some of the most compelling issues of that era in a way that caught the attention of not just scholars throughout economics and the social sciences but the public policy world as well.

That stagflation emerged as one of the most perplexing economic challenges of the 1970s is not hyperbole. As had been the case in the earlier periods of the trust movement of the late 1800s, the Great Depression, and the post-war period, the field of industrial organization once again rose to the challenge by attempting to find the links between the organization of industries and the prevailing economic problem: in this case, stagflation. For example, Robert Heilbroner, a leading scholar of his generation, observed with alarm that, “To a very great degree, the big companies hold the market at bay, raising or lowering prices when they want to, not when an oceanic flood of competition forces them to.”Footnote 15

The responding debate in the field of industrial organization turned to a familiar theory—administered pricing—but with a new twist. The original impetus for the administered price theory was to explain price rigidities during the Great Depression. By contrast, when applied to the stagflation of the 1970s, the impetus was exactly the opposite: Prices were rising due to market power rather than exhibiting a tendency towards rigidity (Demsetz 1973). While much was made in the literature at that time about this theoretical discrepancy, in retrospect both historical episodes reflect an intellectual academic field applying intellectual tools to grapple with some of the most compelling problems of its Zeitalter.

This debate concerning the posited link between market concentration and price increases—the 1970s version of the administered price thesis—was subjected to considerable empirical scrutiny. For example, Blair (1959, 1972) provided empirical evidence that suggested that higher prices resulted from a higher degree of market concentration. Qualls (1975), Weiss (1966) and Dalton (1973) similarly found that prices tend to be higher in more highly concentrated markets. Some empirical evidence seemed to support the link between market concentration and price increases. As Ackley (1959, p. 1) found, “The inflationary process is essentially an administrative one. It arises from largely autonomous upward pressure on wage rates relative to the cost of living, interacting with administered-price markups applied to rising wage costs, compounded again through agricultural prices, raw materials, the cost of living, wage rates and industrial prices in an endless chain.” Similar results were published by Adelman (1961, p. 18) who found that, “business concerns have so much discretion or power to raise prices and wages that they can choose to inflate or not to inflate.”

Stigler (1962), however, focused on key measurement issues by adjusting for discrepancies between actual and listed prices. While market concentration was related to higher listed prices, the effect disappeared after correcting for this measurement discrepancy and using the prices that were actually charged by sellers.

In fact, the empirical evidence with respect to the administered pricing thesis remained mixed and ambiguous at best. The point to be emphasized in this paper is not the validity of a theory posited nearly a half century ago. Rather, the point is that by rising to the challenge posed by one of the most perplexing challenges of its Zeitalter—stagflation—industrial organization continued to ascend to ranking among the most important and valued fields within economics, and certainly one which had the attention of thought leaders in policy and business.

3 Innovation and a Shifting Comparative Advantage

The policy focus on large companies that enjoyed considerable market power in concentrated markets gave way to a new and almost orthogonal concern by the end of the 1970s. Rather than posing a threat to consumers and competitors, those stalwart companies in highly capital-intensive industries—such as U.S. Steel and General Motors—were themselves under an existential threat from international competition, especially from Japan and Germany (Derouzos et al. 1989; Thurow 1984). The loss of profitability and market shares by American companies in leading capital-intensive industries reflected a new phenomenon shaping the policy focus: a shift of comparative advantage away from physical capital (Maskus et al. 1989). What at that time was referred to as the internationalization of U.S. markets exposed companies that previously were feared to be too powerful as instead being too vulnerable (Thurow 1984; Aw 1983; Maskus et al. 1989; Bowen et al. 1987; Bowen and Sveikauskas 1989).

However, not all companies and industries suffered from the internationalization of markets, or what ultimately would be referred to as globalization by the late 1990s (Stiglitz 2004; Leamer 2007 Thurow 2002; Spence 2011). Systematic empirical evidence found that the comparative advantage in the U.S. was shifting towards innovative activity. Firms in industries where factors of production such as human capital, research and development, and patents play an important role tended to perform well, while those in highly capital intensive industries did not (Bowen et al. 1987; Thurow 2002; Bowen and Sveikauskas 1989).

Again, the field of industrial organization provided the intellectual framework for analyzing and understanding why some firms and industries exhibited more innovative activity. In Industrial Structure and Economic Performance, Scherer (1970) compiled both the theory and empirical evidence that linked innovation to market structure. As Scherer (1970, 1980) and Scherer and Ross (1990) made clear, thinking about innovation in the field of industrial organization could be traced back to Schumpeter.

In Theorie der wirtschaftlichen Entwicklung, Schumpeter (1911) posited that it is new firms in an industry that embodied the entrepreneurial spirit that triggered the process of creative destruction that catalyzes innovative activity. As Scherer (1992, p. 1417) explained, “Schumpeter insisted that innovations typically originated in new, characteristically small, firms commencing operation outside the ‘circular flow’ of existing production activities. To be sure, the small innovating firms that succeeded would grow large, and their leaders would amass great fortunes. They started, however, as outsiders.”

However, Schumpeter’s thinking evolved over time. By the time he wrote Capitalism, Socialism and Democracy (1942, p. 106), he reversed his earlier views, and instead argued that it was the large corporation that would provide an engine of innovation and technological change, “What we have got to accept is that (the large-scale establishment or unit of control) has come to be the most powerful engine of…progress and in particular of the long-run expansion of output not only in spite of, but to a considerable extent though, this strategy which looks so restrictive.”

As Scherer (1970, pp. 20–21) made sense of the reversal in Schumpeter’s thinking, “Previously it was suggested that monopolists, sheltered from the stiff gale of competition, might be sluggish about developing and introducing technological innovations, which increase productivity (reducing costs) or enhance product quality. Yet, some economists, led by the late Professor Joseph A. Schumpeter, have argued exactly the opposite; firms need protection from competition before they will bear the risks and costs of invention and innovation, and that a monopoly affords an ideal platform for shooting at the rapidly and jerkily moving targets of new technology. If this is true, then progress will be more rapid under monopoly than under competition.”

Even before the advent of Silicon Valley entrepreneurship, the field of industrial organization was able carefully both to generate and to weigh a body of empirical evidence, which suggested that the innovative returns to firm size were not so overwhelming. In reflecting upon and summarizing both the theory and the systematic body of empirical evidence, Scherer (1992, p. 1425) reached the conclusion that, “Theory and empirical evidence suggest that Capitalism, Socialism and Democracy provided faulty guidance concerning the industrial structures most conducive to technological innovation….Half a century after the publication of Capitalism, Socialism, and Democracy, Schumpeter’s vision of the industrial structure most conducive to technological progress and hence to economic growth remains both relevant and controversial. The book’s publication stimulated a growing stream of theoretical and empirical research. Most of that research supports a conclusion that Schumpeter overstated the advantages of large, monopolistic corporations as engines of technological change” (Scherer 1992, p. 1430).

Scherer (1970) was prescient in his focus on innovation and the roles of small as well as large firms, even during an era when the policy focus was on attaining lower costs through large-scale production. That the field of industrial organization would subsequently spin off two distinct, robust and dynamic fields of scholarship—innovation and entrepreneurship—is testimony to the acumen contained in Industrial Structure and Economic Performance and its author.

4 Policy Implications

During the post-World War II era when the comparative advantage of economic activity in the United States emanated largely from the factor of physical capital (Aw 1983; Bowen et al. 1987; Bowen and Sveikauskas 1989), the field of industrial organization focused on policy approaches that essentially constrained the freedom of firms to contract: antitrust, regulation, and public ownership (Scherer 1970, 1980).

The first, antitrust, was clearly rejected by some of the leading scholars of that era, such as John Kenneth Galbraith (1956). He was so persuaded by the importance of large-scale production that he felt that the antitrust approach would inevitably doom the U.S. to less efficient and less productive companies—hardly an effective way to win the economic race vis-à-vis the Soviet Union. According to Galbraith (1956), the only way to at least maintain pace with rapidly rising Soviet economic performance was through convergence. This convergence was to be accomplished through adapting what he termed managed capitalism. As Scherer (1970, p. 400) explained, “Professor Galbraith has likened the American economy to the bumblebee. According to aerodynamic theory (as interpreted by Galbraith), the bumblebee cannot fly. Yet it does. Similarly, even though the American economy is shot through with monopolistic and oligopolistic elements which might lead one to predict the direst consequences, performance is in fact rather good.”

Managed capitalism centered around what Galbraith (1956) referred to as countervailing power, where the economic power that emanated from concentrated markets and large-scale production was offset by the countervailing forces of unionized labor that represented the concerns of workers and influential government that represented the needs of society.

Thus, both the communist countries of the Soviet Union and her eastern European satellites and the U.S. and her western allies were converging on a remarkably similar industrial structure and organization, where large-scale production resulted in a handful of large companies in any particular industry. The managed capitalism advocated by Galbraith (1956) was achieved by a mix of the three main policy approaches: public ownership, regulation, and antitrust. Some countries, such as the U.S., opted more for antitrust and less for public ownership. Other countries, such as Sweden and France, relied more on public ownership and did not rely greatly on antitrust (or competition policy, as it is generally referred to in Europe).Footnote 16 Still other countries, such as Germany and the Netherlands, focused more on regulation.Footnote 17 During that era, much was made about the great distinctions among the Swedish Model, the French Model, the German Model, etc. In fact, each country was trying to reap the positive gains that accrue from large-scale production in concentrated industries while minimizing the losses that would occur in terms of decentralized decision-making and democracy. Still, most of the western countries deployed its own mix of these three policy instruments, all with a singular goal—to enjoy the benefits of large-scale production without suffering the negative consequences.

However, as first internationalization (Maskus et al. 1989; Bowen et al. 1987; Bowen and Sveikauskas 1989), which subsequently evolved into contemporary globalization (Leamer 2007), shifted the comparative advantage to economic activity based on knowledge, ideas and human capital (Bowen et al. 1987; Bowen and Sveikauskas 1989; Magee 1989), there was a noticeable “retreat of the state” in terms of constraining the freedom of firms to contract, and certainly in term of the triad of policy options emanating from the industrial organization literature—regulation, antitrust and public ownership (Strange 1996)—for four main reasons.

The first was that rather than possess an excess of market power, the one-time dominant stalwarts in capital-intensive industries in the United States were exposed to foreign competition and were rapidly losing market shares (Derouzos et al. 1989; Thurow 1984, 2002). The second was that as production shifted to other countries as a result of both outsourcing and offshoring by U.S. firms but also the advent of foreign competition, the challenges that were posed by large-scale production for industrial organization shifted beyond the legal jurisdiction of the U.S. (Jorde and Teece 1991).

The third may have been the most important and compelling. As the comparative advantage for the United States shifted towards the factors of knowledge and human capital (Maskus et al. 1989; Bowen et al. 1987; Bowen and Sveikauskas 1989), there was a corresponding shift in the focus of policy, from constraining firms to enabling them to create, access and commercialize economic knowledge (Martin and Scott 2000; Link and Scott 2005b). This policy shift was reflected in the academic literatures of innovation and entrepreneurship, where the enabling policy instruments such as the small business innovation research (SBIR) program (Link and Scott 2010, 2012), the Bayh–Dole Act, science or research parks (Link and Scott 2003a, b, 2006, 2007), universities (Siegel et al. 2003; Hall et al. 2003a, b; Link and Scott 2005a, b, c), incubators, and local development policies such as the Research Triangle Park (Link and Scott 2003a, b; Link 1995) became the focus.

The fourth reason is that along with the shift in comparative advantage towards knowledge came the emergence of new industries and firms (Stiglitz 2004). Klepper (1996) has provided a theoretical framework supported by compelling empirical evidence that industries evolve over time, which he characterizes as a “life cycle”. The early stages of an industry are characterized by high rates of entry, particularly among new-firm startups (Gort and Klepper 1982). By contrast, in his model, which is generally confirmed by the empirical analyses, the mature stages of an industry are characterized by high levels of concentration and the emergence of a handful of large, dominant companies. Thus, during 1990s and early part of this century, many of those new knowledge-based industries were driven by entrepreneurial startups (Thurow 2002).

However, more than a few of these knowledge-based industries—which conformed to the early stage of the industry life cycle—have subsequently evolved to a more mature stage of the life cycle. Just as Klepper’s (1996) industry life-cycle model would predict, such industries are becoming more concentrated as the emergence of large, dominant companies—such as Microsoft, Apple, Amazon, Facebook, and Google—suggest. There is increased alarm expressed in the media about this new generation of monopoly power in the contemporary high-tech industries.Footnote 18 Perhaps the framework that links economic performance to the underlying industrial structure and organization—the analysis of which Scherer catalyzed half a century ago—will prove useful in confronting what may well be on the horizon for the next challenge for public policy.

5 Conclusions

The main thesis of this paper is that the scholarly field of industrial organization has been shaped and directed by the most pressing policy issues of the day and how they can be linked to the actual organization of industries. It is not our desire or our task to list and discuss all of the examples of industrial organization scholars who have addressed the big policy issues of the day; we have focused on a selected set of the possible examples. Nor is it our desire to take sides either in the policy debates or the theories and evidence developed by the industrial organization scholars to shed light on those debates. Just as these policy debates have typically been fueled by passions and deep-rooted convictions, the scholarly response from the field of industrial organization has rarely been without substantial ambiguities.

What does emerge is that industrial organization evolved and grew in stature in economics during the three decades that corresponded to the three editions of Scherer’s Industrial Economics and Economic Performance by responding to and addressing the most pressing policy issues of the day. The Voraussetzung for focusing on the organization of industries—as an important and valuable unit of observation worthy of the economist’s tool kit—is its connection to some compelling real world problem of considerable policy concern.

Perhaps one reason why a New Industrial Organization keeps emerging with remarkable temporal regularity is because the policy issues of the day continue to evolve over time. That the field of industrial organization from just a few years earlier typically seems antiquated to the next generation of scholars, may less reflect the repudiation of incorrect knowledge and methods by correct ones and more reflect a discipline whose inherent value is based on the evolution of public policy issues.