During the June 2015 European Workshop on Efficiency and Productivity Analysis in Helsinki, we sat together and tried to map a broad view of performance analysis. The prevailing approach is frontier analysis. The production frontier of a “decision making unit” (such as a firm, an industry, an economy, or conglomerates of the aforementioned) maps the maximum amount of output given the available input, where input and output are multidimensional objects, comprising different types of labor, capital, intermediate inputs, goods, services, and other products. If the “distance” between the actual input-output combination of a decision making unit and the frontier is small, then the unit is efficient. If the frontier is far out, then efficient units are productive. This framework is amenable to precise measurements of efficiency and productivity, but numerous issues surround it and cast shadows on numerical results. This compendium collects a set of works that explore these issues.

First, the framework suggests there are given, fixed lists of input components and output components, but what about new inputs, outputs, and intermediate products? Innovation ought to play a role in performance analysis. Second, the production frontiers reflect the best practices, or benchmarks, of producing outputs given inputs. But are these benchmarks relevant to competing decision making units? Observed or unobserved conditions may be favorable to the benchmarks. Third, in which direction do we measure the distance between an actual input-output combination and the frontier? The literature analyzes alternatives and shows when results differ and when they don’t. But the bulk of it is mechanical and we believe this issue should be related to the objective of an economy. Fourth, what is the appropriate or most informative level of analysis, individuals, firms, industries, or economies? The majority of frontier analyses are industry studies, such as banking, and the same techniques, particularly data envelopment analysis and stochastic frontier analysis, are applied without much further ado at either a more microlevel, such as productivity rankings of workers, or a more macrolevel, such as the performance of national economies. However, the more aggregated the level of analysis, the more scope there is for eliminating sources of inefficiencies, because of the assumptions that inputs are considered given water down, or at least are averaged away. Fifth, by directly working with inputs and outputs, frontier analysis can examine the role that prices pay. Technical frontier analysis does not need price information. Yet prices play a shadow role. In evaluating the frontier output given the available inputs, one solves a mathematical program, and the Lagrange multipliers of the constraints are shadow prices which measure the productivities of the inputs. This raises the issue which prices are relevant: observed market prices or shadow prices?

These issues are the focus of this Handbook. To address them, we will review new developments in frontier analysis. We will extend the analysis by including contributions to the performance literature which we think throw fresh light on the issues. A topic not included in the Handbook was engineering performance, such as reliability analysis. The title of the volume is Handbook of Economic Performance Analysis. Economic theory is also used to organize the related issues of distance directions, objective functions, and performance measures.

Although the focus of the performance literature is on production, we believe that the issues can also be addressed by bearing in mind that the ultimate aim of an economy is to serve the well being of consumers. Consumers maximize utility U(x) subject to the budget constraint px ≤ pω, where x is an n-dimensional commodity bundle, p is the n-dimensional market price (row) vector, and ω is the n-dimensional endowment of the consumer. The first-order condition is that the marginal utilities are proportional to the prices, ∂U/∂x = λp, where λ ≥ 0 is the marginal utility of income. Hence, the direction of steepest utility increase is the market price, irrespective of the specifics of utility, i.e., for all consumers. So market price is a natural direction to measure efficiency. Productivity, however, measures the level of utility attainable given the resources. Utility differences do not matter, so the principle is easiest explained by assuming all consumers have the same utility function U. The frontier is determined by maximizing U(x) subject to the material balance constraint x ≤ y + ω where y is an n-dimensional member of the production possibility set Y and ω is the n-dimensional endowment of all consumers. This maximization problem does feature prices. Assuming constant returns to scale in production (at the aggregate level), the first-order condition is that the marginal utilities are proportional to the prices, ∂U/∂x = p*, where p* ≥ 0 is now the shadow price vector of the balance constraint. So shadow price is a natural direction to measure productivity.

Performance is related to both efficiency and productivity. Roughly speaking, performance can be improved by raising efficiency or by pushing out the frontier. This Handbook presents useful formulas that illuminate the connections in a variety of settings. Clearly, an important, complicating issue is the choice of price. The Handbook discusses more such issues and we think that this broad approach will prove fruitful in advancing performance analysis. For example, the markup between a market price and the shadow price measures market power and this insight may be used to interconnect the efficiency and productivity components of performance in an industrial organization framework.

We attempt to bridge the gap between the two main methodologies of performance analysis, data envelopment analysis (nonparametric) and stochastic frontier analysis (parametric). Often it is not clear what variables are included, if they can be controlled, what their nature is, from a mathematical or statistical point of view. For example, do some firms perform better because their technology is superior or because they are more innovative? Standard practice is to divide variables between inputs, outputs, and environmental variables, to use the inputs and outputs for performance measurement using either methodology, and finally to analyze the relationship between performance results and the environment. This practice, however, raises theoretical and statistical issues. We approach the methodological issues by reviewing variants of stochastic frontier analysis, e.g., with discrete variables and alternative distributions, and by exploring statistical analyses of nonparametric performance measurement, by Bayesian analysis. We will conclude by reviewing commonalities between nonparametric and parametric analyses.

One of the novelties of this Handbook is the coverage and interrelation of microeconomic, mesoeconomic, and macroeconomic performance analysis, in this order, intertwined by methodological contributions.

The first topic is “Micro Foundations of Earnings Differences,” in next chapter, by Tirthatanmoy Das and Solomon W. Polachek. The authors review the distribution of earning differences. The paper finds one’s ability to learn and one’s ability to retain knowledge are most influential in explaining earnings variations. The chapter is a detailed overview of the human capital view of earnings differentials.

In chapter “Performance: The Output/Input Ratio,” Thijs ten Raa critically reviews and interrelates the fundamental concepts of performance analysis, including total factor productivity, the Solow residual, the Farrell efficiency measure, the Debreu-Diewert coefficient of resource utilization, and the Malmquist, Törnqvist, and Fisher indices, for alternative numbers of outputs and inputs.

The main drivers of performance analysis are reviewed in chapter “R&D, Innovation and Productivity,” by Pierre Mohnen. He analyzes the indicators used to perform firm R&D, innovation, and performance analyses and explains the theoretical link between innovation and productivity growth. He then considers the estimated magnitudes in that relationship using the different innovation indicators.

In chapter “The Choice of Comparable DMUs and Environmental Variables,” John Ruggiero addresses the important issues of choosing comparable decision making units and environmental variables in efficiency analysis. In standard nonparametric analysis, decision making units produce common outputs from common inputs, under common conditions. There are methodological trade-offs. On the one hand, the commonality assumptions are better fulfilled by limiting the analysis to smaller numbers of more comparable decision making units. As a consequence, however, the reduction in the number of potentially competing benchmarks increases the efficiency estimates, often toward 100% when the number of production units goes down to the number of outputs. And if the number of environmental values goes up to the number of production units, all inefficiencies may be explained away as well.

Scale efficiency is an important form of efficiency. Shubash Ray analyzes this case in chapter “Data Envelopment Analysis with Alternative Returns to Scale.” Production units are more efficient when they operate at a scale with lower average costs and data envelopment analysis is a convenient methodology in which to model this outcome.

In chapter “Ranking Methods Within Data Envelopment Analysis,” Nicole Adler and Nicola Volta present the issue of rankings. They address the lack of discrimination in DEA applications, in particular when the number of inputs and outputs is relatively high in comparison with the number of production units, borrowing techniques from the social sciences, including multiple-criteria decision making.

The Handbook proceeds in chapter “Distributional Forms in Stochastic Frontier Analysis,” to stochastic frontier analysis. In chapter “Distributional Forms in Stochastic Frontier Analysis,” Alexander Stead, Phill Wheat, and William Greene survey the developments in stochastic frontier modeling. The basic function in this literature is models of errors of measurement based on the normal distribution and (in)efficiency, a signed concept, by the half-normal distribution. The former distribution is well vested in theory, building around the central limit theorem. But the latter is ad hoc and various alternatives have emerged. These are reviewed from a practitioner’s point of view.

In chapter “Stochastic Frontier Models for Discrete Output Variables,” Eduardo Fé addresses another important issue, encountered in labor, industrial, and health economics, where outputs are nontangible and nonpecuniary outcomes and often measured through indicators of achievement (employment status, academic certification, success in a labor market scheme), ordered categories (like scales describing job satisfaction, health status, personality traits), or counts (numbers of patents or infant deaths). This chapter generalizes the standard stochastic frontier model to encompass such situations.

The gap between the two main frontier methodologies has attracted developers of intermediate approaches, which are reviewed in the next three chapters. Chapter “Nonparametric Statistical Analysis of Production,” by Camilla Mastromarco, Leopold Simar, and Paul Wilson, analyzes the data envelopment analysis of chapters “The Choice of Comparable DMUs and Environmental Variables”, “Data Envelopment Analysis with Alternative Returns to Scale” and “Ranking Methods Within Data Envelopment Analysis,” but attaches that approach to a statistical analysis. Their approach requires large samples of, e.g., production units (with their input-output combinations), but then combines the strengths of nonparametric analysis and stochastic frontier analysis to make statements about expected performance measures and their confidence intervals.

Ultimately distributional assumptions differentiate the different approaches presented thus far. A natural variant is reviewed in chapter “Bayesian Performance Evaluation,” by Mike Tsionas. The Bayesian approach provides a formal and straightforward way to facilitate statistical inference, which is always a problem in data envelopment analysis, despite the recent advances exposited in the previous chapters.

Chapter “Common Methodological Choices in Parametric and Nonparametric Analyses of Firms’ Performance,” by Luis Orea and José Zofío finds common ground for the two frontier analyses, using distance functions as the organizing principle, which is standard in data envelopment analysis, but novel in stochastic frontier analysis. Key issues related to alternative technological assumptions and alternative economic objectives of the firm are thus reviewed. The issues of the number of production units relative to the numbers of outputs and inputs and the number of environmental variables are addressed. The choice of direction of the distance function is discussed.

The next four chapters take us from microproduction analysis to meso- and macroeconomics. The crucial choice of prices is related to the objective of an economy in chapter “Pricing Inputs and Outputs: Market Prices Versus Shadow Prices, Market Power and Welfare Analysis,” by Aditi Bhattacharyya, Levent Kutlu, and Robin C. Sickles. Market prices cannot be simply taken to signal the social value of production. There are two ways forward: (i) correct the observed market prices to derive the shadow price and (ii) try to derive the shadow price directly without using market prices (e.g., when market prices do not exist). The chapter surveys the methods used in the literature to derive social valuations when market prices are not accurate or not available due to the following considerations: imperfect competition, effects on income distribution and growth that are not factored into prices and external effects, like on the environment.

Chapter “Aggregation of Individual Efficiency Measures and Productivity Indices,” by Andreas Mayer and Valentin Zelenyuk, reviews the key existing results on aggregate efficiency measures and aggregate productivity indices and outlines new results for the aggregation of the Hicks-Moorsteen productivity index, and outlines some insights into ongoing and future directions of research in this area.

Chapter “Intermediate Inputs and Industry Studies: Input-Output Analysis,” by Victoria Shestalova, reviews performance measurement for industries and the whole economy, taking into account the interindustry deliveries, using the workhorse of applied general equilibrium analysis, the modern input-output model with possibly different numbers of inputs, outputs, and industries. The methodology, a synthesis of frontier and input-output analyses, is suitable for performance measurements within both national and international industrial studies, environmental analysis, and other policy-relevant analyses. Data requirements and international databases are reviewed, as are applications, including the assessment of emission caps and pricing policies supporting the adjustments in industrial outputs.

In chapter “Modeling Environmental Adjustments of Production Technologies: A Literature Review,” Hervé Dakpo and Frederic Angwe present a theoretical discussion of negative externalities, including the production of “bads,” without an explicit allusion to performance. They summarize the lessons from the different models in the literature and the challenges that need to be dealt with in modeling environmentally adjusted production technologies.

The last two chapters are macroeconomic. In chapter “An Overview of Issues in Measuring the Performance of National Economies,” Anthony Glass, Karligash Kenjegalieva, Robin Sickles, and Thomas Weyman-Jones measure the aggregate economic performance of national economies, considering a wide range of different measures including the value-added definition of GDP and economic welfare. They show how stochastic frontier analysis and data envelopment analysis modeling has been able through the idea of total factor productivity (TFP) decomposition and the measurement of inefficiency to tell us much more about TFP than the more conventional approaches. They review the issue of whether the performance of national economies converges over time, or whether, as suggested by endogenous growth models, the individual performance of different countries is endogenous to the country itself. Technological spillovers among neighboring countries at the level of the aggregate production function are analyzed.

In chapter “Productivity Indexes and National Statistics: Theories, Methods and Challenges,” Erwin Diewert and Kevin Fox provide the theoretical justifications for the index number formulae for productivity growth measurement that are commonly used by national statistical offices. They then turn to a discussion of data used in index number construction in practice and highlight the measurement challenges. The choice of index number formula is examined based on an “axiomatic” approach and from the perspective of economic theory, recognizing that the resulting indexes are measuring economic concepts. The results provide the justification for the index number choices made by national statistical offices in constructing productivity growth estimates. Data needs for constructing the productivity indexes are discussed and the concepts, sources, and methods that are used for the output, labor, and capital components are reviewed. National statistical offices face several difficult measurement problems and this chapter suggests ways forward.

We are grateful that our proposal to edit a Handbook which highlights issues of mainstream efficiency and productivity analysis and offers a broad perspective on economic performance analysis, was well received and accepted by the chapter contributors we had in mind and thank them for their work. We are also grateful to the referees who wrote anonymous reports, but whom we now reveal. Thank you Antonio Amores, Bert Balk, Jan Boone, Walter Briec, Maria da Conceição Andrade e Silva, Rolf Färe, Shawna Grosskopf, Reza Hajargasht, Joop Hartog, Jens Krüger, Chris O’Donnell, Raquel Ortega-Argilés, Inmaculada Sirvent Quilez, Mark Steel, and Emmanuel Thanassoulis.