Productivity Indexes and National Statistics: Theory, Methods and Challenges

Abstract

This chapter reviews the theory underlying the index number approaches used by National Statistical Offices in the construction of productivity indexes. It covers approaches for measuring output, labour and capital and highlights persistent and emerging measurement problems.

… it is not reasonable for us to expect the government to produce statistics in areas where concepts are mushy and where there is little professional agreement on what is to be measured and how. (Griliches 1994 Presidential Address to the American Economic Association, p. 14)

1 Introduction

Productivity is a major driver of long-term economic growth and welfare improvements. Productivity indexes are used in a wide variety of policy contexts, such as for government budget forecasting, designing innovation policy and assessing the relative effectiveness of government policies.

Productivity growth slowdowns cause much policy debate and concern. The slowdown from the early 1970s to the mid-1990s in many industrialized countries was much debated, especially given that this was the period during which personal computers diffused rapidly into workplaces. This resulted in much attention to the measurement of productivity—if the slowdown was simply a case of measurement lagging behind developments in an increasingly complex economy, then the solution is to modernize the collection and construction of economic statistics (see Diewert and Fox [1999] and references therein for more on this slow growth episode and potential explanations).

From around the mid-1990s, a measured increase in productivity allayed concerns and was viewed in most countries as the benefits of computerization finally being realized. However, a subsequent decline in productivity growth since 2004 across all industrialized countries has again heightened concerns. It raises the possibility that the earlier productivity recovery was an unusual episode and that lower growth is the new norm. In the latter case, we should expect lower increases in living standards in the future. This is the view of, for example, Gordon (2016) and Cowen (2011).¹ Alternatively, perhaps measurement problems associated with the digital economy and rapid quality change in products have come to the fore. In his Independent Review of UK Economic Statistics, interim report, Bean (2016, p. 7) noted that “Statistics have failed to keep pace with the impact of digital technology”. This concern has yielded a growing literature on measurement problems for National Statistical Offices (NSOs), their potential to explain away the productivity slowdown and alternative approaches to measuring economic activity in a modern economy.²

In this chapter, to provide a means to better understand such debates, we begin by examining the theoretical basics of productivity growth measurement as employed by NSOs. In particular, we provide the theoretical justifications for the index number formulae that are commonly used. We then turn to a discussion of data used in index number construction in practice and highlight the measurement challenges.

The productivity of a production unit is defined as the output produced by the unit divided by the input used over the same time period.³ If the input measure is comprehensive, then the productivity concept is called Total Factor Productivity (TFP) or Multifactor Productivity (MFP).⁴ If the input measure is labour hours, then the productivity concept is called Labour Productivity.

The Bureau of Labor Statistics in the USA was the first NSO to introduce an official program to measure MFP in 1983 (see Dean and Harper 2001). Other countries with MFP programs now include Canada, Australia, the UK and New Zealand. The OECD also publishes MFP and Labour Productivity statistics for member countries (see OECD 2018).

We will focus on MFP and how to measure it rather than Labour Productivity. The Labour Productivity concept has its uses but the problem with this concept is that it could be very high in one country compared to another country with the difference being entirely due to a larger amount of non-labour input in the first country. On the other hand, if MFP is much higher in country A compared to country B, then country A will be genuinely more efficient than country B and it will be useful to study the organization of production in country A in order to see if the techniques used there could be exported to less efficient countries.

A problem with the MFP concept is that it depends on the units of measurement for outputs and inputs. Hence, MFP can only be compared across production units if the production units are basically in the same line of business so that they are producing the same (or closely similar) outputs and using the same inputs. However, in the time series context, Multifactor Productivity growth rates can be compared over dissimilar production units, and hence, we will focus most of our attention on measuring Multifactor Productivity Growth (MFPG).

We begin by providing an introduction to the issues involved in measuring MFPG by considering the special case where the production unit produces only a single output and uses only a single input. It turns out in this case that there are four equivalent ways for measuring MFPG. Section 3 generalizes this framework to the multiple input and output case, as faced by NSOs. This requires the choice of index number formula. Section 4 examines this problem from the test (or “axiomatic”) approach. Essentially, this involves comparing the mathematical properties of the formula against a battery of tests which are deemed as desirable. Section 5 examines this formula choice problem from the perspective of economic theory, recognizing that the resulting indexes are measuring economic concepts. Thus, these sections provide the justification for the index number choices made by NSOs in constructing productivity growth estimates.

Section 6 discusses data needs for constructing the productivity indexes and reviews the concepts, sources and methods that are used for the output, labour and capital components. Section 7 highlights several difficult measurement problems faced by NSOs and suggests some ways forward. Section 8 concludes.

2 Productivity Measurement in the Case of One Input and One Output

We consider in this section the problem of measuring the Multifactor Productivity (MFP) (and the growth of Multifactor Productivity, MFPG) of a one output, one input firm.⁵ To do this, we require data on the amounts of output produced, y⁰ and y¹, during two time periods, 0 and 1, and on the amounts of input utilized, x⁰ and x¹, during those same two time periods. It is also convenient to define the firm’s revenues R^t and total costs C^t for period t where t = 0, 1. The average selling price of a unit of output in period t is assumed to be p^t and the average cost of a unit of input in period t is w^t for t = 0, 1. Thus, we have:

$$R^{t} = p^{t} y^{t} \quad {\text{for}}\;t = 0,1$$

(1)

and

$$C^{t} = w^{t} x^{t} \quad {\text{for}}\,t = 0,1.$$

(2)

Our first definition of the MFPG of the firm going from period 0 to period 1 (or more briefly, of the productivity of the firm) is:

$$MFPG(1) = \left( {y^{1} /y^{0} } \right)/\left( {x^{1} /x^{0} } \right).$$

(3)

Note that y¹/y⁰ is (one plus) the firm’s output growth rate going from period 0 to period 1 while x¹/x⁰ is the corresponding input growth rate going from period 0 to period 1.⁶ If MFPG(1) > 1, then the output growth rate was greater than the input growth rate and we say that the firm has experienced a productivity improvement going from period 0 to period 1. If MFPG(1) < 1, then we say that the firm has experienced a productivity decline.

The output growth rate, y¹/y⁰, can also be interpreted as a quantity index of outputs. Indeed, in the following section where we consider the case of multiple outputs, we will replace y¹/y⁰ by a quantity index for outputs. However, if there is only one output, it can be verified that the output quantity indexes defined there all reduce to the output growth rate, y¹/y⁰. Similarly, the input growth rate, x¹/x⁰, can be interpreted as a quantity index of inputs. Hence, our first definition of productivity growth, MFPG(1) defined by (3), can be interpreted as an output quantity index divided by an input quantity index.

An alternative method for measuring productivity in a one output, one input firm is the change in technical coefficients method. Define the input-output coefficient of the firm in period t as:

$$a^{t} \equiv y^{t} /x^{t} ,\quad t = 0,1.$$

(4)

Thus, a^t is the total amount of output y^t produced by the firm in period t divided by the total amount of input utilized by the firm in period t, x^t. It can be interpreted as a coefficient which summarizes the engineering and economic characteristics of the firm’s technology in period t: a^t describes the rate at which inputs are transformed into outputs during period t.

Our second definition of total factor productivity can be expressed in terms of the output-input coefficients, a⁰ and a¹, as follows:

$$MFPG(2) = a^{1} /a^{0} .$$

(5)

Thus, if a¹ is greater than a⁰, so that the firm is producing more output per unit input in period 1 compared to period 0, then MFPG(2) and the firm has experienced an increase in productivity going from period 0 to period 1.

It should be noted that the two productivity growth concepts that we have defined thus far, MFPG(1) and MFPG(2), are both relative concepts. This is a general feature of economic definitions of productivity: the performance of the firm in a current period 1 is always compared to its performance in a base period 0. In contrast, an engineering concept of productivity or efficiency is usually an absolute one, concerned with obtaining the maximum amount of output in period one, y¹, given an available amount of input in period one, x¹, consistent with the laws of physics.⁷

Using (3), (4) and (5), it is easy to show that MFPG(2) coincides with an earlier MFPG(1) concept in this simple one output, one input model of production; i.e., we have:

$$\begin{aligned} MFPG(2) & = a^{1} /a^{0} = \left( {y^{1} /x^{1} } \right)/\left( {y^{0} /x^{0} } \right) = \left( {y^{1} /y^{0} } \right)/\left( {x^{1} /x^{0} } \right) = MFPG(1). \\ \end{aligned}$$

(6)

We turn now to a third possible method for defining productivity:

$$MFPG(3) \equiv \left[ {\left( {R^{1} /R^{0} } \right)/\left( {p^{1} /p^{0} } \right)} \right]/\left[ {\left( {C^{1} /C^{0} } \right)/\left( {w^{1} /w^{0} } \right)} \right].$$

(7)

Thus, MFPG(3) is equal to the firm’s revenue ratio R¹/R⁰ deflated by the output price index p¹/p⁰ divided by the cost ratio between the two periods C¹/C⁰ deflated by the input price index w¹/w⁰.

Using (1), we have

$$\left( {R^{1} /R^{0} } \right)/\left( {p^{1} /p^{0} } \right) = \left( {p^{1} y^{1} /p^{0} y^{0} } \right)/\left( {p^{1} /p^{0} } \right) = y^{1} /y^{0}$$

(8)

and using (2), we have

$$\left( {C^{1} /C^{0} } \right)/\left( {w^{1} /w^{0} } \right) = \left( {w^{1} x^{1} /w^{0} x^{0} } \right)/\left( {w^{1} /w^{0} } \right) = x^{1} /x^{0} .$$

(9)

Thus, in this simple one input, one output model, (8) says that the deflated revenue ratio is equal to the output growth rate and (9) says that the deflated cost ratio is equal to the input growth rate. Hence, (7) equals (3) and we have, using (6):

$$MFPG(1) = MFPG(2) = MFPG(3).$$

(10)

There is a fourth way for measuring productivity change that is a generalization of a method originally suggested by Jorgenson and Griliches (1967). In order to explain this fourth method, we need to introduce the concept of the firm’s period t margin, m^t; i.e., define

$$1 + m^{t} \equiv R^{t} /C^{t} ;\quad t = 0,1.$$

(11)

Thus, 1 + m^t is the ratio of the firm’s period t revenues R^t to its period t costs C^t. If m^t is zero, then the firm’s revenues equal its costs in period t and the economic profit of the firm is zero. If m^t is positive, then the bigger m^t is, the bigger are the firm’s profits.

We can now define our fourth way for measuring productivity change in a one output, one input firm:

$$MFPG(4) \equiv \left[ {\left( {1 + m^{1} } \right)/\left( {1 + m^{0} } \right)} \right]\left( {w^{1} /w^{0} } \right)/\left( {p^{1} /p^{0} } \right).$$

(12)

Thus, MFPG(4) is equal to the margin growth rate (1 + m¹)/(1 + m⁰) times the rate of increase in input prices w¹/w⁰ divided by the rate of increase in output prices p¹/p⁰.

If we use Eqs. (11) to eliminate (1 + m¹)/(1 + m⁰) in (12), we find that

$$MFPG(4) = MFPG(3)$$

(13)

and thus, by (10), MFPG(1) = MFPG(2) = MFPG(3) = MFPG(4). Thus, in a one output, one input firm, we have four conceptually distinct methods for measuring productivity change that turn out to be equivalent. Unfortunately, this equivalence does not generally extend to the multiple output, multiple input case.

Definition (12) of productivity can be used to show the importance of achieving a productivity gain: a productivity improvement is the source for increases in margins or increases in input prices or decreases in output prices. Equation (12) also indicates the relationship between total factor productivity and increased profitability. Rearranging (12), we have:

$$\left( {1 + m^{1} } \right)/\left( {1 + m^{0} } \right) = \left[ {MFPG(4)} \right]\left( {p^{1} /p^{0} } \right)/\left( {w^{1} /w^{0} } \right).$$

(14)

Thus, the rate of growth in margins is equal to MFPG times the output price growth rate divided by the input price growth rate.

If there are constant returns to scale in production or margins m^t are zero for whatever reason in periods 0 and 1, then MFPG(4) reduces to (w¹/w⁰)/(p¹/p⁰), which is the input price index divided by the output price index, a formula due to Jorgenson and Griliches (1967, p. 252).

We conclude this section with a rather lengthy discussion of the problem of distinguishing MFPG from the concept of technical change or technical progress, TP. In order to distinguish MFPG from TP, it is necessary to introduce the concept of the firm’s period t production function f ^t; i.e., in period t, y = f ^t(x) denotes the maximum amount of output y that can be produced by x units of the input. We assume that in periods 0 and 1, the observed amounts of output, y⁰ and y¹, are produced by the observed amounts of input, x⁰ and x¹, according to the following production function relationships:

$$y^{0} = f^{0} \left( {x^{0} } \right);$$

(15)

$$y^{1} = f^{1} \left( {x^{1} } \right).$$

(16)

Note that we are now explicitly assuming that production is technically efficient during the two periods under consideration.⁸

We define technical progress TP as a measure of the shift in the production function going from period 0 to period 1. There are an infinite number of possible shift measures but it turns out that four measures of technical progress (involving the observed data y⁰, y¹, x⁰ and x¹ in some way) are the most useful. First, define:

$$y^{0*} = f^{1} \left( {x^{0} } \right)\,{\text{and}}\,y^{1*} = f^{0} \left( {x^{1} } \right).$$

(17)

Thus, y^0* is the output that could be produced by the period 0 input x⁰ if the period 1 production function f ¹ were available and y^1* is the output which could be produced by the period 1 input x¹ but using the period 0 technology which is summarized by the period 0 production function f ⁰. Note that in order to define these hypothetical outputs y^0* and y^1*, a knowledge of the period 0 and 1 production functions f ⁰ and f ¹ is required. This knowledge is not easy to acquire but it could be obtained by engineering studies or by nonparametric or econometric methods for obtaining a suitable reference technology.

With y^0* and y^1* defined, we can define the following two output-based indexes of technical progress TP(1) and TP(2)⁹:

$$TP(1) = y^{0*} /y^{0} = f^{1} \left( {x^{0} } \right)/f^{0} \left( {x^{0} } \right);$$

(18)

$$TP(2) = y^{1} /y^{1*} = f^{1} \left( {x^{1} } \right)/f^{0} \left( {x^{1} } \right).$$

(19)

Thus, TP(1) is one plus the percentage increase in output due to technical and managerial improvements (going from period 0 to period 1) evaluated at the period 0 input level x⁰ and TP(2) is one plus the percentage increase in output due to the new technology evaluated at the period 1 input level x¹.

It is also possible to define input-based measures of technical progress TP(3) and TP(4). First, define x^0* and x^1* as follows:

$$y^{0} = f^{1} \left( {x^{0*} } \right)\,{\text{and}}\,y^{1} = f^{0} \left( {x^{1*} } \right).$$

(20)

Thus, x^0* is the input required to produce the period 0 output y⁰ but by using the period 1 technology, and so x^0* will generally be less than x⁰ (which is the amount of input required to produce the period 0 output using the period 0 technology). Similarly, x^1* is the amount of input required to produce the period 1 output y¹ but by using the period 0 technology, and x^1* will generally be larger than x¹ (because the period 0 technology will generally be less efficient than the period 1 technology). Now define the following two input-based measures of technical progress, TP(3) and TP(4)¹⁰:

$$TP(3)= x^{0} /x^{0*} ;$$

(21)

$$TP(4) = x^{1*} /x^{1} .$$

(22)

The above four measures of TP can be illustrated with the aid of Fig. 1. The diagram shows that each of the TP measures can be different.

Fig. 1

Production-based measures of technical progress

The lower curved line is the graph of the period 0 production function; that is, it is the set of points (x, y) such that $x \ge 0$ and y = f ⁰(x). The higher curved line is the graph of the period 1 production function; that is, it is the set of points (x, y) such that $x \ge 0$ and y = f ¹(x). The observed data points are A, which has coordinates (x⁰, y⁰) and B, which has coordinates (x¹, y¹). Note that the absolute amounts of production function shift in the direction of the y-axis are y^0* − y⁰(at point A) and y¹ − y^1* (at point B). The absolute amounts of production function shift in the direction of the x-axis are x⁰ − x^0*(at point A) and x^1*− x¹(at point B). We have chosen to measure TP in terms of the relative shifts, y^0*/y⁰, y¹/y^1*, x⁰/x^0* and x^1*/x¹ rather than the absolute shifts, y^0* − y⁰, y¹ − y^1*, x⁰ − x^0* and x^1* − x¹ in order to obtain measures of shift that are invariant to changes in the units of measurement. Note that MFPG = MFPG(2) = (y¹/x¹)/(y⁰/x⁰) is equal to the slope of the straight line OB divided by the slope of the straight line OA.

It turns out that there is a relationship between each of our technical progress measures, TP(1), TP(2), TP(3), TP(4), and MFPG. We have:

$$MFPG = TP(i)RS(i);\quad i = 1,2,3,4$$

(23)

where the four returns to scale measures RS(i) are defined as follows:

$$RS(1) \equiv \left( {y^{1} /x^{1} } \right)/\left( {y^{0*} /x^{0} } \right);$$

(24)

$$RS(2) \equiv \left( {y^{1*} /x^{1} } \right)/\left( {y^{0} /x^{0} } \right);$$

(25)

$$RS(3) \equiv \left( {y^{1} /x^{1} } \right)/\left( {y^{0} /x^{0*} } \right);$$

(26)

$$RS(4) \equiv \left( {y^{1} /x^{1*} } \right)/\left( {y^{0} /x^{0} } \right).$$

(27)

The returns to scale measures RS(1) and RS(3) pertain to the period 1 production function f ¹ while the measures RS(2) and RS(4) pertain to the period 0 production function f ⁰. To interpret each of these returns to scale measures geometrically, see Fig. 1. Each of these returns to scale measures is the ratio of two input-output coefficients, say y^j/x^j divided by $y^{k} /x^{k}$, the two points on the same production function, with $x^{j} > x^{k}$. Thus, if the returns to scale measure is greater than 1, then $y^{j} /x^{j} > y^{k} /x^{k}$ and we say that the production function exhibits increasing returns to scale between the two points. If RS(i) = 1, then the production function exhibits constant returns to scale between the two points and finally if RS(i) < 1, then the production function exhibits decreasing returns to scale between the two points.

The decompositions given by Eq. (23) tell us that MFPG is equal to the product of a technical progress term TP(i) (this corresponds to a shift in the production function going from period 0 to period 1) and a returns to scale term RS(i) (this corresponds to a movement along one of the production functions). In Fig. 1, definitions (18)–(22) and definitions (24)–(27) can be used to verify that each of the four decompositions of MFPG given by (23) corresponds to a different combination of shifts and movements along a production function that take us from point A to point B.

For firms in a regulated industry, returns to scale will generally be greater than one, since increasing returns to scale in production is often the reason for regulation in the first place. Thus, MFPG will exceed TP for growing firms in a regulated industry (provided that there are increasing returns to scale for that firm).

We note that the technical progress and returns to scale measures defined above cannot in general be calculated without a knowledge of the production functions that describe the technology for the two periods under consideration. However, in a one input, one output firm, the MFPG measures defined above can be calculated unambiguously provided that we know inputs used and outputs produced during the two periods.

Next, we shall generalize the above production function-based definitions of productivity and technical progress to cover the case of many outputs and many inputs.

3 Productivity Measurement in the Case of Many Outputs and Inputs

The approach taken in this section will be to replace the single output growth factor, y¹/y⁰, by an output quantity index and to replace the single input growth factor, x¹/x⁰, by an input quantity index. The approach outlined in this section is a practical one that is implemented by statistical agencies to calculate industry estimates of MFP growth.

Recall our first definition of productivity growth in the one output, one input case, $MFPG(1) \equiv \left( {y^{1} /y^{0} } \right)/\left( {x^{1} /x^{0} } \right)$, which was the output ratio divided by the input ratio between periods 0 and 1. In order to find a counterpart to this definition in the multiple output, multiple input case, we need only replace the output ratio by an output quantity index, $Q\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)$, and replace the input ratio by an input quantity index, $Q^{ * } \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)$, where $p^{t} \equiv \left( {p_{1}^{t} , \ldots ,p_{M}^{t} } \right)$ and $w^{t} \equiv \left( {w_{1}^{t} , \ldots ,w_{N}^{t} } \right)$ are the period t output and input price vectors and $y^{t} \equiv \left( {y_{1}^{t} , \ldots ,y_{M}^{t} } \right)$ and $x^{t} \equiv \left( {x_{1}^{t} , \ldots ,x_{M}^{t} } \right)$ are the period t output and input quantity vectors for t = 0, 1. Thus, an output quantity index, $Q\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)$, is defined to be a function of the output prices and quantities for the two periods under consideration. Similarly, an input quantity index, between periods 0 and 1, $Q^{ * } \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)$, is simply a function of 4N variables, the input prices and quantities pertaining to the two periods under consideration.

Two of the most frequently used functional forms for quantity indexes are the Laspeyres (1871) and Paasche (1874) quantity indexes.¹¹ The Laspeyres output quantity index between periods 0 and 1 is defined as:

$$\begin{aligned} Q_{L} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) & \equiv \sum\limits_{m = 1}^{M} {p_{m}^{0} y_{m}^{1} } /\sum\limits_{m = 1}^{M} {p_{m}^{0} y_{m}^{0} } \\ & = \sum\limits_{m = 1}^{M} {\left( {y_{m}^{1} /y_{m}^{0} } \right)p_{m}^{0} y_{m}^{0} } /\sum\limits_{m = 1}^{M} {p_{m}^{0} y_{m}^{0} } \\ & = \sum\limits_{m = 1}^{M} {\left( {y_{m}^{1} /y_{m}^{0} } \right)s_{m}^{0} } \\ \end{aligned}$$

(28)

where the period t revenue share for output m is defined as

$$s_{m}^{t} \equiv p_{m}^{t} y_{m}^{t} /\sum\nolimits_{k = 1}^{M} {p_{k}^{t} y_{k}^{t} } ;\quad m = 1, \ldots ,M;\;t = 0,1.$$

(29)

Thus, the Laspeyres output quantity index is a base period revenue share weighted sum of the M individual quantity ratios, $y_{m}^{1} /y_{m}^{0}$.

The Paasche output quantity index between periods 0 and 1 is defined as:

$$\begin{aligned} Q_{P} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) & \equiv \sum\limits_{m = 1}^{M} {p_{m}^{1} y_{m}^{1} } /\sum\limits_{m = 1}^{M} {p_{m}^{1} y_{m}^{0} } \\ & = \left( {\sum\limits_{m = 1}^{M} {p_{m}^{1} y_{m}^{0} } /\sum\limits_{m = 1}^{M} {p_{m}^{1} y_{m}^{1} } } \right)^{ - 1} \\ & = \left[ {\sum\limits_{m = 1}^{M} {\left( {y_{m}^{1} /y_{m}^{0} } \right)^{ - 1} p_{m}^{1} y_{m}^{1} } /\sum\limits_{m = 1}^{M} {p_{m}^{1} y_{m}^{1} } } \right]^{ - 1} \\ & = \left[ {\sum\limits_{m = 1}^{M} {\left( {y_{m}^{1} /y_{m}^{0} } \right)^{ - 1} s_{m}^{1} } } \right]^{ - 1} . \\ \end{aligned}$$

(30)

Thus, the Paasche output quantity index is a current period revenue share weighted harmonic mean of the M individual quantity ratios, $y_{m}^{1} /y_{m}^{0}$.

In what follows, we shall concentrate on the problems involved in choosing a functional form for the output index Q; an analogous discussion applies to the choice of a functional form for the input index Q^*.

Another commonly used functional form for a quantity index is the Fisher (1922, p. 234) ideal quantity index Q_F which is equal to the square root of the product of the Laspeyres and Paasche quantity index defined by (28) and (30), i.e.:

$$Q_{F} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) \equiv \left[ {Q_{L} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)Q_{P} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)} \right]^{1/2} .$$

(31)

Another commonly used functional form for a quantity index is the Törnqvist (1936) quantity index Q_T. The natural logarithm of Q_T is defined to be the right-hand side of (32) below:

$${ \ln }\,Q_{T} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) \equiv 1/2\sum\limits_{m = 1}^{M} {\left( {s_{m}^{0} + s_{m}^{1} } \right)\;{ \ln }\left( {y_{m}^{1}/y_{m}^{0} } \right)}$$

(32)

where the revenue shares $s_{m}^{t}$ are defined by (29) above. Note that the quantities $y_{m}^{t}$ must all be positive in order for Q_T to be well defined.

The quantity index Q_T is also known as the translog quantity index (e.g. see Jorgenson and Nishimizu [1978] who introduced this terminology) because Diewert (1976, p. 120) related Q_T to a translog production function. This index is also known as the Divisia index since Jorgenson and Griliches (1967, 1972) used Q_T to provide a discrete time approximation to the continuous time Divisia index.¹²

The four quantity indexes Q_L, Q_P, Q_F and Q_T, defined by (28), (30), (31) and (32) respectively, all have a common property: if the number of outputs M equals one, then each of these quantity indexes reduces to the output ratio, $y_{1}^{1} /y_{1}^{0}$. Thus, it can be seen that the use of quantity indexes for outputs and inputs can be used to generalize our one output, one input measure of productivity change, MFPG(1), discussed in the previous section. More formally, let us define the direct quantity index measure of productivity growth MFPG(5) in the general multiple output, multiple input case as follows:

$$MFPG(5) \equiv Q\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)/Q^{*} \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)$$

(33)

where Q is the output quantity index and Q^* is the input quantity index. If the number of outputs equals one and the number of inputs equals one, if Q equals one of Q_L, Q_P, Q_F or Q_T, and if $Q^{ * }$ equals one of $Q_{L}^{ * }$, $Q_{P}^{ * }$, $Q_{F}^{ * }$ or $Q_{T}^{ * } ,$ then MFPG(5) = MFPG(1). Thus, the approach to productivity measurement outlined in this section reduces to the approach outlined in the previous section if there is only one input and only one output.

In the general multiple output, multiple input case, we still have to address a problem: Which functional forms for the output index Q and the input index $Q^{ * }$ should we choose? We shall return to this functional form problem shortly.

We turn now to an index number measure of productivity that generalizes the deflated revenues divided by deflated costs productivity measure MFPG(3) that was defined earlier by (7) in the previous section.

Denote period t revenue by R^t and period t cost by C^t. We have:

$$R^{t} \equiv \sum\limits_{m = 1}^{M} {p_{m}^{t} y_{m}^{t} } ;\;C^{t} \equiv \sum\limits_{n = 1}^{N} {w_{n}^{t} x_{n}^{t} } ;\quad t = 0,1.$$

(34)

The multiple output analogue to the output price ratio which occurred in formula (34) in the previous section is the output price index, $P\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)$, which is a function of 4M variables, the output prices and quantities that pertain to the two periods under consideration. The multiple input analogue to the input price ratio which occurred in the previous section is the input price index, $P^{ * } \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)$, which is a function of 4N variables, the input prices and quantities that pertain to the two periods under consideration.

Using the output price index P as a deflator for the revenue ratio R¹/R⁰ between periods 0 and 1 and using the input price index P^*as a deflator for the cost ratio C¹/C⁰ between the two periods leads to the following definition of the productivity growth of the production unit going from period 0 to 1:

$$MFPG(6) \equiv \left[ {\left( {R^{1} /R^{0} } \right)/P\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)} \right]/\left[ {\left( {C^{1} /C^{0} } \right)/P^{*} \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)} \right].$$

(35)

Note that (35) is a generalization to multiple inputs and outputs of our earlier productivity change measure MFPG(3) defined in the previous section.

Suppose that the output quantity index Q(p⁰, p¹, y⁰, y¹) which appeared in definition (33) matches up with the output price index P(p⁰, p¹, y⁰, y¹) which appears in definition (35) in the sense that the product of the price and quantity index equals the revenue ratio for the two periods under consideration so that we have:

$$R^{1} /R^{0} = P\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)Q\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right).$$

(36)

Suppose further that the input quantity index Q^*(w⁰,w¹,x⁰,x¹) which appeared in definition (33) matches up with the input price index P^*(w⁰, w¹, x⁰, x¹) which appears in definition (35) in the sense that the product of the price and quantity index equals the cost ratio for the two periods under consideration so that we have:

$$C^{1} /C^{0} = P^{*} \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)Q^{*} \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right).$$

(37)

Now substitute (36) and (37) into (35) and we find that:

$$MFPG(5) = MFPG(6).$$

(38)

Thus if the two pairs of price and quantity indexes satisfy the relations (36) and (37), we find that both of the productivity measures introduced in this section, MFPG(5) defined by (33) and MFPG(6) defined by (35) are equal to each other.

Recall that in the previous section, we defined the period t markup, m^t, for the production unit by 1 + m^t = R^t/C^t for t = 0,1. Using these definitions of the markup in each period again, it can be seen that we can rewrite MFPG(6) as follows:

$$\begin{aligned} MFPG(6) & = \left[ {\left( {R^{1} /R^{0} } \right)/P\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)} \right]/\left[ {\left( {C^{1} /C^{0} } \right)/P^{*} \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)} \right] \\ & = \left[ {\left( {R^{1} /R^{0} } \right)/\left( {C^{1} /C^{0} } \right)} \right]\left[ {P^{*} \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)/P\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)} \right] \\ & = \left[ {\left( {1 + m^{1} } \right)/\left( {1 + m^{0} } \right)} \right]\left[ {P^{*} \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)/P\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)} \right] \\ & = MFPG(7). \\ \end{aligned}$$

(39)

The above definition says that MFPG(7) is equal to the margin growth rate times the input price index divided by the output price index. Defining profitability as R^t/C^t for t = 0,1, we can see from the second line of (39) that we have productivity growth equal to the growth in profitability times the relative growth of input prices to output prices. Reorganizing, we get that profitability growth equals productivity growth times the relative growth of output prices to input prices. This highlights the role of productivity as a key determinant of profitability (see, e.g., Balk [2003, p. 2] for more on this).

Note that MFPG(7) is an exact analogue to our earlier one output, one input MFP growth measure MFPG(4) defined by (12) in Sect. 1. Equations (38) and (39) show that this “new” measure of MFP growth is equal to the previous measure MFPG(5), which was the ratio of the output quantity index to the input quantity index, and to MFPG(6), which was equal to the revenue growth rate deflated by the output price index divided by the cost growth rate deflated by the input price index.¹³ Thus, we have obtained multiple output, multiple input counterparts to the following Sect. 1 equalities:

$$MFPG(1) = MFPG(3) = MFPG(4).$$

(40)

There remains the problem of choosing a functional form for the output price index P and the input price index P^*. The same four index number formulae that were used for quantity indexes, (28), (30), (31) and (32), can also be used for price indexes, except that the role of prices and quantities are interchanged. Thus, define the Laspeyres price index P_L, the Paasche price index P_P, the Fisher price index P_F and the translog price index P_T by (41), (42), (43) and (44), respectively:

$$P_{L} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) \equiv Q_{L} \left( {y^{0} ,y^{1} ,p^{0} ,p^{1} } \right);$$

(41)

$$P_{P} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) \equiv Q_{P} \left( {y^{0} ,y^{1} ,p^{0} ,p^{1} } \right);$$

(42)

$$P_{F} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) \equiv Q_{F} \left( {y^{0} ,y^{1} ,p^{0} ,p^{1} } \right);$$

(43)

$$P_{T} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) \equiv Q_{T} \left( {y^{0} ,y^{1} ,p^{0} ,p^{1} } \right).$$

(44)

The Laspeyres, Paasche, Fisher and Translog input price indexes, $P_{L}^{ * } \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)$, $P_{P}^{ * } \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)$, $P_{F}^{ * } \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)$, and $P_{T}^{ * } \left( {w^{0} ,w^{1} ,x^{0} ,x^{1} } \right)$, respectively, may be defined in an analogous manner.

If M = 1, so that there is only one output, then it can be verified that the output price indexes defined by (41)–(44) all collapse down to the output price ratio, $p_{1}^{1} /p_{1}^{0}$. Similarly, if N = 1, so that there is only one input, then $P_{L}^{ * }$, $P_{P}^{ * }$, $P_{F}^{ * }$ and $P_{T}^{ * }$ all collapse down to the input price ratio, $w_{1}^{1} /w_{1}^{0}$. Thus, the use of the Laspeyres, Paasche, Fisher or translog price indexes in (35) or (39) leads to the following equalities in the M = 1, N = 1:

$$MFPG(6) = MFPG(7) = MFPG(1).$$

(45)

Thus, our new definitions of productivity change defined by (33), (35) or (39) are generalizations to the case of many outputs and inputs of our earlier one output, one input measure of productivity change defined by (3) in the previous section.

Returning to the general case of many outputs and many inputs, it can be seen that different choices of the output price index P and the input price index P^* will generate different productivity change measures MFPG(6) defined by (35). Similarly, different choices of the output quantity index Q and the input quantity index Q^* will generate different productivity change measures MFPG(5) defined by (33).

However, the degree of arbitrariness in the formulae (33) and (35) is not quite as large as it might seem at first glance. It turns out that the two families of productivity measures are related, because the deflated revenue ratio which occurs in the numerator of the right-hand side of (35), $\left( {R^{1} /R^{0} } \right)P\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)$, can be interpreted as an implicit quantity index of outputs, and the denominator in (35), $\left( {C^{1} /C^{0} } \right)P^{ * } \left( {w^{0} ,w^{1} ,w^{0} ,w^{1} } \right)$, can be interpreted as an implicit quantity index of inputs.

From Sect. 2, it was evident that the total factor productivity growth measures that were defined there measure the combined effects of technological progress, movements towards the production frontier and increasing (or decreasing) returns to scale. The MFP growth measures defined in this section also measure the combined effects of these three factors. When we allow for the possibility of increasing returns to scale in production, it turns out to be very difficult to estimate separately the effects of increasing returns to scale from technical progress. In general, in order to perform this separation, it is necessary to have panel data or to perform an econometric study on time series data.¹⁴ Econometric approaches are, in general, not practical for a statistical agency. And usually, statistical agencies do not have usable panel data on hand in order to undertake nonparametric studies of relative efficiency. Hence, we will not cover these econometric approaches and applications of nonparametric methods utilizing cross-sectional data in this brief survey of how to measure MFP growth.¹⁵

In the next two sections, we turn to an assessment of the alternative index number formulae introduced in this section. This will help explain the properties of index number formulae used by NSOs, and why some formulae are favoured over others.

4 The Test Approach to Index Number Theory

First, we introduce another index number formula to be assessed. It can be shown that (R¹/R⁰)/P_T(p⁰,p¹,y⁰,y¹) is not equal to the Törnqvist quantity index, Q_T. Hence, we simply define the implicit Törnqvist quantity index, Q_IT, as follows:

$$Q_{IT} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) \equiv \left( {R^{1} /R^{0} } \right)/P_{T} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right).$$

(46)

The five quantity indexes, Q_L, Q_P, Q_F, Q_T and Q_IT, are the five functional forms for quantity indexes that are used most frequently in applied economics. The question now arises: Which of these five formulae should we use in the multiple output, multiple input definition of MFP growth, MFPG(5) defined by (35)?

Using the results from Diewert (1976), it can be shown that from the perspective of the economic approach to index number theory, Q_F, Q_T and Q_IT are clearly preferred to the Paasche and Laspeyres quantity indexes, Q_P and Q_L. Again, from the perspective of the economic approach to index number theory, P_F, P_T and P_IT are clearly preferred to the Paasche and Laspeyres price indexes, P_P and P_L. The economic approach provides equal justifications for Q_F, Q_T and Q_IT or for P_F, P_T and P_IT. Hence, any of these indexes would be equally good from the economic perspective.¹⁶ We will pursue the economic approach in more detail in the following section.

Another major approach to index number theory is the test or axiomatic approach to index number theory. This approach to the determination of the functional form for P and Q works as follows: researchers suggest various mathematical properties that P or Q should satisfy based on a priori reasoning—these properties are called “tests” or “axioms”—and then mathematical reasoning is applied to determine: (i) whether the a priori tests are mutually consistent and (ii) whether the a priori tests uniquely determine the functional form for P or Q. The main contributors to the test or axiomatic approach were Walsh (1901, 1921a, b), Fisher (1911, 1922), Frisch (1936), Eichhorn (1978), Eichhorn and Voeller (1976) and Funke and Voeller (1978, 1979).¹⁷

We will not cover the test approach in great detail in this chapter but we will present some material on this important approach to index number theory.

One fundamental test that the price and quantity index should jointly satisfy is the test (36) above; that is, the product of the output price and quantity indexes between periods 0 and 1 should equal the revenue or value ratio between the two periods, $R_{1}/R_{0} = \sum\nolimits_{m = 1}^{M} p_{m}^{1} y_{m}^{1} /\sum\nolimits_{m = 1}^{M} {p_{m}^{0} y_{m}^{0} }.$ This test was called the product test by Frisch (1930, p. 399), but it was first formulated by Irving Fisher (1911, p. 388).

If we accept the validity of the product test (and virtually all researchers do accept its validity), then P and Q cannot be determined independently. For example, if the functional form for the price index P is given, then (36) determines the functional form for the quantity index Q.

Thus, in what follows, we focus on the determination of the functional form for the price index P. Once P has been determined, Q will be determined residually by (36).

We list a few examples of tests that have been proposed for price indexes.

The Identity or Constant Prices Test, originally proposed by Laspeyres (1871, p. 308) and also by Walsh (1901, p. 308), and Eichhorn and Voeller (1976, p. 24) is the following test:

$$P\left( {p,p,y^{0} ,y^{1} } \right) = 1;$$

(47)

i.e., if $p^{0} = p^{1} \equiv p$, so that for each commodity, prices are equal in the two periods being compared, then the price index is equal to 1 no matter what the quantities are in period 0 and 1, y⁰ and y¹ respectively.

The Constant Basket Test or the Constant Quantities Test, proposed by many researchers including Walsh (1901, p. 540), is the following test:

$$P\left( {p^{0} ,p^{1} ,y,y} \right) = \sum\limits_{m = 1}^{M} {p_{m}^{1} y_{m} } /\sum\limits_{m = 1}^{M} {p_{m}^{0} y_{m} } ;$$

(48)

i.e., if quantities are constant over the two periods 0 and 1 so that $y^{0} = y^{1} \equiv y$, then the level of prices in period 1 compared to period 0 is the value of the constant basket of quantities evaluated at the period 1 prices, $\sum\nolimits_{m = 1}^{M} {p_{m}^{1} y_{m} }$, divided by the value of the basket evaluated at the period 0 prices, $\sum\nolimits_{m = 1}^{M} {p_{m}^{0} y_{m} }$.

The Proportionality in Period t Prices Test, proposed by Walsh (1901, p. 385) and Eichhorn and Voeller (1976, p. 24), is the following test:

$$P(p^{0} ,\lambda p^{1} ,y^{0} ,y^{1} ) = \lambda \,P\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)\quad {\text{for}}\,{\text{all}}\,\lambda > 0;$$

(49)

i.e., if each price in period 1 is multiplied by the positive constant $\lambda$, then the level of prices in period 1 relative to the level of prices in period 0 increases by the same positive constant $\lambda$.

Our final example of a price index test is the Time Reversal Test, which was first informally proposed by Pierson (1896, p. 128) and more formally by Walsh (1901, p. 368; 1921b, p. 541) and Fisher (1922, p. 64):

$$P\left( {p^{1} ,p^{0} ,y^{1} ,y^{0} } \right) = 1/P\left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right);$$

(50)

i.e., if the prices and quantities for periods 0 and 1 are interchanged, then the resulting price index is the reciprocal of the original price index.

The four tests (47)–(50) will suffice to give a flavour of the test approach to index number theory. For a much more extensive list of twenty or so tests (see Diewert (1992)).

There are five leading functional forms for the output price index P that are most frequently used in empirical work: (i) the Laspeyres price index P_L, (ii) the Paasche price index P_P, (iii) the Fisher price index P_F, (iv) the Törnqvist price index P_T defined by (44) and (v) the implicit Törnqvist price index P_IT defined by:

$$P_{IT} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right) \equiv \left[ {\sum\limits_{m = 1}^{M} {p_{m}^{1} y_{m}^{1} } /\sum\limits_{m = 1}^{M} {p_{m}^{0} y_{m}^{0} } } \right]/Q_{T} \left( {p^{0} ,p^{1} ,y^{0} ,y^{1} } \right)$$

(51)

where the Törnqvist quantity index Q_T is defined by (32). The Fisher index satisfies the four tests (47)–(50), but P_L fails (50), P_P fails (50), P_T fails (48) and P_IT fails (47).

When more extensive lists of tests are compiled, the Fisher ideal price index P_F continues to satisfy more tests than other leading candidates (see Diewert 1976, p. 131; 1992). In fact, the Fisher price index satisfies all twenty tests utilized by Diewert (1992). Moreover, satisfactory axiomatic characterizations of P_F have been obtained (see Funke and Voeller 1978, p. 180; 1979; Diewert 1992). Thus, from the viewpoint of the test approach to index number theory, the Fisher quantity index Q_F defined by (31) and the corresponding Fisher price index P_F defined by (43) seem to be the best choices. It should also be noted that P_F and Q_F satisfy the product test in (36). Hence, if the Fisher indexes are used in the productivity measures defined by (33) or (35), then both of these productivity measures will coincide; that is, if we use Fisher price and quantity indexes for P and Q and P^* and Q^* wherever they occur in (33), (35) or (39), we obtain the following equality:

$$MFPG_{F} (5) = MFPG_{F} (6) = MFPG_{F} (7)$$

(52)

where we have added a subscript F to the three productivity measures to indicate that Fisher indexes are being used. Thus, an added benefit of using Fisher price and quantity indexes is that three conceptually distinct (but equally attractive) productivity change measures become identical.

While the Törnqvist index fails nine of twenty tests of Diewert (1992), it passes the time reversal test, which is regarded as an important property. Also, it usually approximates the Fisher index closely in empirical applications, so we can regard it as satisfying all twenty tests to a high degree of approximation. The Laspeyres and Paasche indexes fail only three (“reversal”) tests, but the failure to satisfy the time reversal test is regarded as serious. Hence, from the test approach to index numbers, the Fisher and Törnqvist indexes are preferred.

In the next section, we look at an index number method for estimating MFP growth in the time series context that draws on the economic approach to the measurement of MFP growth and the theory of exact index numbers.

5 The Exact Index Number Approach to Productivity Measurement

The test approach to index number choice discussed in Sect. 4 related to the mathematical properties of the index formulae. There was no direct connection with economic theory. However, such a connection can be made, as will be shown in this section. This “economic” or “exact” approach to index number choice has been influential in guiding index number choice by NSOs and is a reason why the USA switched to using a Fisher index formula for calculating gross domestic product (GDP) in the mid-1990s. It is also a reason why it is common practice to use Törnqvist in constructing industry-level MFP estimates.¹⁸

Konüs (1939) introduced the idea of a true cost of living index, which is a ratio of cost functions where utility is held constant. The corresponding concept in the production context is that the true price index is the ratio of revenue functions R(p^t,y)/R(p^t−1,y), where y is a reference output level. For a choice of functional form for the revenue functions, this unobserved theoretical true price index can be exactly calculated. In this case, we say that there is an “exact” relationship between the functional form and an index number formula. For example, it can be shown that for a linearly homogeneous quadratic unit revenue function, assuming optimizing behaviour (so that Hotelling’s Lemma can be used), the true price index exactly equals the Fisher price index.¹⁹

The justification for the Törnqvist index can be argued to be stronger than for the Fisher index from this approach, as the assumption of linear homogeneity is not required to establish its exact relationship with the translog functional form. Both translog and quadratic functional forms have the property of “flexibility”; Diewert (1974) defined a flexible functional form as one that provides a second-order approximation to a twice continuously differentiable function at a point. Many popular functional forms in economics (e.g. Cobb-Douglas and CES) do not have this rather minimal property. An index number which is exact for a flexible functional form was defined by Diewert (1976) as being superlative. Thus, Fisher and Törnqvist indexes are superlative indexes.

Laspeyres and Paasche indexes are not superlative. They are exact for a linear unit cost function (Konüs and Byushgens 1926), which is dual to a (zero substitution) Leontief production function. Thus, these indexes are regarded as quite restrictive from the economic approach to index numbers.

In this section, we appeal to the exact index number approach to develop our approach to measuring MFP growth when there are many outputs and many inputs. We describe the exact index number approach to the measurement of technical change and productivity growth that was initially developed by Diewert and Morrison (1986) and Kohli (1990). This theory is adapted into a method for measuring the growth in the real income generated by a production unit with a decomposition of this growth in real income into components that reflect:

• technical progress;

• changes in the prices of outputs; and

• growth of primary inputs.

This methodology can provide measures of how changes in the prices of imports and exports can affect real income growth.

We assume that there is a period t market sector technology set S^t that exhibits constant returns to scale. The components of net output are the usual components of GDP, namely C + G + I + X − M (household and government consumption, investment, exports minus imports). Later we will also subtract depreciation and revaluation terms from GDP in order to obtain net domestic product, which is closer to an income concept. For now, we interpret the net output vector for period t, y^t, as the net output components of market sector of the economy. The corresponding market sector primary input vector for period t is denoted by x^t. The components of x^t consist of different types of labour services supplied to the market sector by households and the various types of capital services used by the market sector. The corresponding vectors of period t net output prices is denoted by P^t and the corresponding vector of period t primary input prices is denoted by W^t. In period t, we assume that there is a feasible set of output vectors y that can be produced by the market sector if the vector of primary inputs x is utilized by the market sector of the economy; denote this period t production possibilities set by S^t. We assume that S^t is a closed convex cone that exhibits a free disposal property.²⁰

Given a vector of output prices P and a vector of available primary inputs x, we define the period t market sector GDP function, g^t(P,x), as follows²¹:

$$g^{t} \left( {P,x} \right) \equiv { \hbox{max} }_{\text{y}} \left\{ {Py:\left( {y,x} \right)\;{\text{belongs}}\,{\text{to}}\,S^{t} } \right\};\quad t = 1,2, \ldots .$$

(53)

Thus, market sector GDP depends on t (which represents the period t technology set S^t), on the vector of output prices P that the market sector faces and on x, the vector of primary inputs that is available to the market sector.

If P^t is the period t output price vector and x^t is the vector of inputs used by the market sector during period t and assuming that actual outputs equal the theoretical market sector outputs given by the solution to Eq. (53), then the period t vector of market sector outputs y^t will be equal to the vector of first-order partial derivatives of g^t(P^t,x^t) with respect to the components of P; that is, we will have the following equations for each period t²²:

$$y^{t} = \partial_{P} g^{t} \left( {P^{t} ,x^{t} } \right);\quad t = 1,2, \ldots .$$

(54)

Thus, assuming profit maximization, the period t market sector (net) supply vector y^t can be obtained by differentiating the period t market sector GDP function with respect to the components of the period t output price vector P^t.

Assuming that actual primary inputs equal the theoretical market sector inputs that minimize the cost of producing a given amount of GDP, then the period t vector of input prices W^t will be equal to the vector of first-order partial derivatives of g^t(P^t,x^t) with respect to the components of x; that is, we will have the following equations for each period t²³:

$$W^{t} = \partial_{x} g^{t} \left( {P^{t} ,x^{t} } \right);\quad t = 1,2, \ldots .$$

(55)

Thus, assuming cost minimization, the period t market sector input prices W^t paid to primary inputs can be obtained by differentiating the period t market sector GDP function with respect to the components of the period t input quantity vector x^t.

The assumptions of price-taking behaviour in relating quantities to prices, i.e., the assumption of pure competition, will be maintained in the remainder of this chapter. The fascinating violations of this assumption are analysed in Chapters 13 and 15 of this Handbook.

The constant returns to scale assumption on the technology sets S^t implies that the value of outputs will equal the value of inputs in period t; that is, we have the following relationships:

$$g^{t} \left( {P^{t} ,x^{t} } \right) = P^{t} \cdot y^{t} = W^{t} \cdot x^{t} ;\quad t = 1,2, \ldots .$$

(56)

This says that nominal GDP constructed using the production approach (value of outputs) should equal GDP constructed using the income approach (payments to the factors of production). NSOs typically aim to ensure that this is the case. Whether or not the assumption of constant returns to scale is desirable could be questioned, as it forces the value of output to equal the value of input, but here we simply note that it is standard NSO practice to do so.²⁴

Our focus is on the income generated by the market sector or more precisely, on the real income generated by the market sector. However, since market sector net output is distributed to the factors of production used by the market sector, nominal market sector GDP will be equal to nominal market sector income, as in (56). As an approximate welfare measure that can be associated with market sector production,²⁵ we will choose to measure the real income generated by the market sector in period t, $\rho^{t}$, in terms of the number of consumption bundles that the nominal income could purchase in period t; that is, define $\rho^{t}$ as follows:

$$\rho^{t} \equiv W^{t} \cdot x^{t} /P_{C}^{t} = w^{t} \cdot x = p^{t} \cdot y^{t} = g^{t} \left( {p^{t} ,x^{t} } \right);\quad t = 0,1,2, \ldots$$

(57)

where $P_{C}^{t} > 0$ is the period t consumption expenditures deflator and the market sector period t real output price p^t and real input price w^t vectors are defined as the corresponding nominal price vectors deflated by the consumption expenditures price index; that is, we have the following definitions²⁶:

$$p^{t} \equiv P^{t} /P_{C}^{t} ;w^{t} \equiv W^{t} /P_{C}^{t} ;\quad t = 0,1,2, \ldots .$$

(58)

The first and last equality in (57) imply that period t real income, $\rho^{t}$, is equal to the period t GDP function, evaluated at the period t real output price vector p^t and the period t input vector x^t, g^t(p^t,x^t). Thus, the growth in real income over time can be explained by three main factors: Technical Progress or Total Factor Productivity growth,²⁷ growth in real output prices and the growth of primary inputs. We will shortly give formal definitions for these three growth factors.

Using the linear homogeneity properties of the GDP functions g^t(P,x) in P and x separately, we can show that the following counterparts to the relations (54) and (55) hold using the deflated prices p and w²⁸:

$$y^{t} = \nabla_{p} g^{t} \left( {p^{t} ,x^{t} } \right);\quad t = 0,1,2, \ldots$$

(59)

$$w^{t} = \nabla_{x} g^{t} \left( {p^{t} ,x^{t} } \right);\quad t = 0,1,2, \ldots .$$

(60)

Now we are ready to define a family of period t productivity growth factors or technical progress shift factors $\tau \left( {p,x,t} \right)$²⁹:

$$\tau (p,x,t) \equiv g^{t} (p,x)/g^{t - 1} (p,x);\quad t = 1,2, \ldots .$$

(61)

Thus, $\tau (p,x,t)$ measures the proportional change in the real income produced by the market sector at the reference real output prices p and reference input quantities used by the market sector x where the numerator in (61) uses the period t technology and the denominator in (61) uses the period t−1 technology. Thus, each choice of reference vectors p and x will generate a possibly different measure of the shift in technology going from period t−1 to period t. Note that we are using the chain system to measure the shift in technology.

It is natural to choose special reference vectors for the measure of technical progress defined by (61): a Laspeyres type measure $\tau_{L}^{t}$ that chooses the period t−1 reference vectors p^t−1 and x^t−1 and a Paasche type measure $\tau_{P}^{t}$ that chooses the period t reference vectors p^t and x^t:

$$\begin{aligned} \tau_{L}^{t} & \equiv \tau \left( {p^{t - 1} ,x^{t - 1} ,t} \right) \\ & = g^{t} \left( {p^{t - 1} ,x^{t - 1} } \right)/g^{t - 1} \left( {p^{t - 1} ,x^{t - 1} } \right);\quad t = 1,2, \ldots ; \\ \end{aligned}$$

(62)

$$\tau_{P}^{t} \equiv \tau \left( {p^{t} ,x^{t} ,t} \right) = g^{t} \left( {p^{t} ,x^{t} } \right)/g^{t - 1} \left( {p^{t} ,x^{t} } \right);\quad t = 1,2, \ldots .$$

(63)

Since both measures of technical progress are equally valid, it is natural to average them to obtain an overall measure of technical change. If we want to treat the two measures in a symmetric manner and we want the measure to satisfy the time reversal property from the index number theory in Sect. 2, then the geometric mean will be the best simple average to take.³⁰ Thus, we define the geometric mean of (62) and (63) as follows³¹:

$$\tau^{t} \equiv \left( {\tau_{L}^{t} \tau_{P}^{t} } \right)^{1/2} ;\quad t = 1,2, \ldots .$$

(64)

At this point, it is not clear how we will obtain empirical estimates for the theoretical productivity growth factors defined by (62)–(64). One obvious way would be to assume a functional form for the GDP function g^t(p,x), collect data on output and input prices and quantities for the market sector for a number of years (and for the consumption expenditures deflator), add error terms to Eqs. (59) and (60) and use econometric techniques to estimate the unknown parameters in the assumed functional form. However, econometric techniques are generally not completely straightforward: different econometricians will make different stochastic specifications and will choose different functional forms.³² Moreover, as the number of outputs and inputs grows, it will be impossible to estimate a flexible functional form. Thus, we will suggest methods for estimating measures like (64) that are based on exact index number techniques.

We turn now to the problem of defining theoretical indexes for the effects on real income due to changes in real output prices. Define a family of period t real output price growth factors $\alpha \left( {p^{t - 1} ,p^{t} ,x,s} \right)$³³:

$$\alpha \left( {p^{t - 1} ,p^{t} ,x,s} \right) \equiv g^{s} \left( {p^{t} ,x} \right)/g^{s} \left( {p^{t - 1} ,x} \right);\quad s = 1,2, \ldots .$$

(65)

Thus, $\alpha \left( {p^{t - 1} ,p^{t} ,x,s} \right)$ measures the proportional change in the real income produced by the market sector that is induced by the change in real output prices going from period t−1 to t, using the technology that is available during period s and using the reference input quantities x. Thus, each choice of the reference technology s and the reference input vector x will generate a possibly different measure of the effect on real income of a change in real output prices going from period t−1 to period t.

Again, it is natural to choose special reference vectors for the measures defined by (65): a Laspeyres type measure $\alpha_{L}^{t}$ that chooses the period t−1 reference technology and reference input vector x^t−1 and a Paasche type measure $\alpha_{P}^{t}$ that chooses the period t reference technology and reference input vector x^t:

$$\begin{aligned} \alpha_{L}^{t} & = \alpha \left( {p^{t - 1} ,p^{t} ,x^{t - 1} ,t - 1} \right) \\ & = g^{t - 1} \left( {p^{t} ,x^{t - 1} } \right)/g^{t - 1} \left( {p^{t - 1} ,x^{t - 1} } \right);\quad t = 1,2, \ldots ; \\ \end{aligned}$$

(66)

$$\alpha_{P}^{t} = \alpha \left( {p^{t - 1} ,p^{t} ,x^{t} ,t} \right) = g^{t} \left( {p^{t} ,x^{t} } \right)/g^{t} \left( {p^{t - 1} ,x^{t} } \right);\quad t = 1,2, \ldots .$$

(67)

Since both measures of real output price change are equally valid, it is natural to average them to obtain an overall measure of the effects on real income of the change in real output prices³⁴:

$$\alpha^{t} = \left( {\alpha_{L}^{t} \alpha_{p}^{t} } \right)^{1/2} ;\quad t = 1,2, \ldots .$$

(68)

Finally, we look at the problem of defining theoretical indexes for the effects on real income due to changes in real output prices. Define a family of period t real input quantity growth factors $\beta \left( {x^{t - 1} ,x^{t} ,p,s} \right)$³⁵:

$$\beta \left( {x^{t - 1} ,x^{t} ,p,s} \right) \equiv g^{s} \left( {p,x^{t} } \right)/g^{s} \left( {p,x^{t - 1} } \right);\quad s = 1,2, \ldots .$$

(69)

Thus, $\beta \left( {x^{t - 1} ,x^{t} ,p,s} \right)$ measures the proportional change in the real income produced by the market sector that is induced by the change in input quantities used by the market sector going from period t−1 to t, using the technology that is available during period s and using the reference real output prices p. Thus, each choice of the reference technology s and the reference real output price vector p will generate a possibly different measure of the effect on real income of a change in input quantities going from period t–1 to period t.

Again, it is natural to choose special reference vectors for the measures defined by (69): a Laspeyres type measure $\beta_{L}^{t}$ that chooses the period t−1 reference technology and reference real output price vector p^t−1 and a Paasche type measure $\beta_{P}^{t}$ that chooses the period t reference technology and reference real output price vector p^t:

$$\begin{aligned} \beta_{L}^{t} & \equiv \beta \left( {x^{t - 1} ,x^{t} ,p^{t - 1} ,t - 1} \right) \\ & = g^{t - 1} \left( {p^{t - 1} ,x^{t} } \right)/g^{t - 1} \left( {p^{t - 1} ,x^{t - 1} } \right);\quad t = 1,2, \ldots ; \\ \end{aligned}$$

(70)

$$\beta_{P}^{t} \equiv \beta \left( {x^{t - 1} ,x^{t} ,p^{t} ,t} \right) = g^{t} \left( {p^{t} ,x^{t} } \right)/g^{t} \left( {p^{t} ,x^{t - 1} } \right);\quad t = 1,2, \ldots .$$

(71)

Since both measures of real input growth are equally valid, it is natural to average them to obtain an overall measure of the effects of input growth on real income³⁶:

$$\beta^{t} \equiv \left( {\beta_{L}^{t} \beta_{P}^{t} } \right)^{1/2} ;\quad t = 1,2, \ldots .$$

(72)

Recall that market sector real income for period t was defined by (57) as $\rho^{t}$ equal to nominal period t factor payments $W^{t} \cdot x^{t}$ deflated by the household consumption price deflator $P_{C}^{t}$. It is convenient to define $\gamma^{t}$ as the period t chain rate of growth factor for real income:

$$\gamma^{t} \equiv \rho^{t} /\rho^{t - 1} ;\quad t = 1,2, \ldots .$$

(73)

It turns out that the definitions for $\gamma^{t}$ and the technology, output price and input quantity growth factors $\tau (p,x,t)$, $\alpha \left( {p^{t - 1} ,p^{t} ,x,s} \right)$, $\beta \left( {x^{t - 1} ,x^{t} ,p,s} \right)$ defined by (61), (65) and (69), respectively, satisfy some interesting identities, which we will now develop. We have:

$$\begin{aligned} \gamma^{t} & \equiv \rho^{t} /\rho^{t - 1} ;\quad t = 1,2, \ldots . \\ & = g^{t} \left( {p^{t} ,x^{t} } \right)/g^{t - 1} \left( {p^{t - 1} ,x^{t - 1} } \right)\quad {\text{using}}\,{\text{definitions}}\; ( 5 7 )\\ & = \left[ {g^{t} \left( {p^{t} ,x^{t} } \right)/g^{t - 1} \left( {p^{t} ,x^{t} } \right)} \right]\left[ {g^{t - 1} \left( {p^{t} ,x^{t} } \right)/g^{t - 1} \left( {p^{t - 1} ,x^{t} } \right)} \right]\left[ {g^{t - 1} \left( {p^{t - 1} ,x^{t} } \right)/g^{t - 1} \left( {p^{t - 1} ,x^{t - 1} } \right)} \right] \\ & = \tau_{P}^{t} \alpha \left( {p^{t - 1} ,p^{t} ,x^{t} ,t - 1} \right)\beta_{L}^{t} \quad {\text{using}}\,{\text{definitions}}\;(63),\;(65)\,{\text{and}}\,(70). \\ \end{aligned}$$

(74)

In a similar fashion, we can establish the following companion identity:

$$\gamma^{t} \equiv \tau_{L}^{t} \alpha \left( {p^{t - 1} ,p^{t} ,x^{t - 1} ,t} \right)\beta_{P}^{t} \quad {\text{using}}\,{\text{definitions}}\,(62),\,(65)\,{\text{and}}\,(71).$$

(75)

Thus multiplying (74) and (75) together and taking positive square roots of both sides of the resulting identity and using definitions (64) and (72), we obtain the following identity:

$$\gamma^{t} \equiv \tau^{t} \left[ {\alpha \left( {p^{t - 1} ,p^{t} ,x^{t} ,t - 1} \right)\alpha \left( {p^{t - 1} ,p^{t} ,x^{t - 1} ,t} \right)} \right]^{1/2} \beta^{t} ;\quad t = 1,2, \ldots .$$

(76)

In a similar fashion, we can derive the following alternative decomposition for $\gamma^{t}$ into growth factors:

$$\gamma^{t} \equiv t^{t} \alpha^{t} \left[ {\beta \left( {x^{t - 1} ,x^{t} ,p^{t} ,t - 1} \right)\beta \left( {x^{t - 1} ,x^{t} ,p^{t - 1} ,t} \right)} \right]^{1/2} ;\quad t = 1,2, \ldots .$$

(77)

It is quite likely that the real output price growth factor $\left[ {\alpha \left( {p^{t - 1} ,p^{t} ,x^{t} ,t - 1} \right)\alpha \left( {p^{t - 1} ,p^{t} ,x^{t - 1} ,t} \right)} \right]^{1/2}$ is fairly close to $\alpha^{t}$ defined by (68), and it is quite likely that the input growth factor $\left[ {\beta \left( {x^{t - 1} ,x^{t} ,p^{t} ,t - 1} \right)\beta \left( {x^{t - 1} ,x^{t} ,p^{t - 1} ,t} \right)} \right]^{1/2}$ is quite close to $\beta^{t}$ defined by (72); that is, we have the following approximate equalities:

$$\left[ {\alpha \left( {p^{t - 1} ,p^{t} ,x^{t} ,t - 1} \right)\alpha \left( {p^{t - 1} ,p^{t} ,x^{t - 1} ,t} \right)} \right]^{1/2} \approx \alpha^{t} ;\quad t = 1,2, \ldots ;$$

(78)

$$\left[ {\beta \left( {x^{t - 1} ,x^{t} ,p^{t} ,t - 1} \right)\beta \left( {x^{t - 1} ,x^{t} ,p^{t - 1} ,t} \right)} \right]^{1/2} \approx \beta^{t} ;\quad t = 1,2, \ldots .$$

(79)

Substituting (78) and (79) into (76) and (77), respectively, leads to the following approximate decompositions for the growth of real income into explanatory factors:

$$\gamma^{t} \approx \tau^{t} \alpha^{t} \beta^{t} ;\quad t = 1,2, \ldots$$

(80)

where $\tau^{t}$ is a technology growth factor, $\alpha^{t}$ is a growth in real output prices factor and $\beta^{t}$ is a growth in primary inputs factor.

Rather than look at explanatory factors for the growth in real market sector income, it is sometimes convenient to express the level of real income in period t in terms of an index of the technology level or of Total Factor Productivity in period t, T^t, of the level of real output prices in period t, A^t, and of the level of primary input quantities in period t, B^t.³⁷ Thus, we use the growth factors $\tau^{t}$, $\alpha^{t}$ and $\beta^{t}$ as follows to define the levels T^t, A^t and B^t:

$$T^{0} = 1;\;T^{t} = T^{t - 1} \tau^{t} ;\quad t = 1,2, \ldots ;$$

(81)

$$A^{0} = 1;\;A^{t} = A^{t - 1} \alpha^{t} ;\quad t = 1,2, \ldots ;$$

(82)

$$B^{0} = 1;\;B^{t} = B^{t - 1} \beta^{t} ;\quad t = 1,2, \ldots .$$

(83)

Using the approximate equalities (80) for the chain links that appear in (81)–(83), we can establish the following approximate relationship for the level of real income in period t, $\rho^{t}$, and the period t levels for technology, real output prices and input quantities:

$$\rho^{t} /\rho^{0} \approx T^{t} A^{t} B^{t} ;\quad t = 1,2, \ldots .$$

(84)

We now consider a set of assumptions on the technology sets that will ensure that the approximate real income growth decompositions (80) and (84) hold as exact equalities.

Specifically, we follow the example of Diewert and Morrison (1986, p. 663) and assume that the log of the period t (deflated) GDP function, g^t(p,x), has the following translog functional form³⁸:

$$\begin{aligned} { \ln }g^{t} \left( {p,x} \right) & \equiv a_{0}^{t} + \sum\limits_{m = 1}^{M} {a_{m}^{t} { \ln }p_{m} } + 1/2\sum\limits_{m = 1}^{M} {\sum\limits_{k = 1}^{M} {a_{mk} { \ln }p_{m} { \ln }p_{k} } } \\ & \quad + \sum\limits_{n = 1}^{M} {b_{n}^{t} { \ln }x_{n} } + 1/2\sum\limits_{n = 1}^{M} {\sum\limits_{j = 1}^{M} {b_{nj} { \ln }x_{n} { \ln }x_{j} } } \\ & \quad + \sum\limits_{m = 1}^{M} {\sum\limits_{n = 1}^{M} {c_{mn} { \ln }p_{m} { \ln }x_{n} ;\quad t = \, 0,1,2, \ldots .} } \\ \end{aligned}$$

(85)

Note that the coefficients for the quadratic terms are assumed to be constant over time. The coefficients must satisfy the following restrictions in order for g^t to satisfy the linear homogeneity properties that we have assumed in Sect. 4 above³⁹:

$$\sum\limits_{m = 1}^{M} {a_{m}^{t} } = 1\quad {\text{for}}\,t = 0,1,2, \ldots ;$$

(86)

$$\sum\limits_{n = 1}^{N} {b_{n}^{t} } = 1\quad {\text{for}}\,t = 0,1,2, \ldots ;$$

(87)

$$a_{mk} = a_{km} \quad {\text{for}}\,{\text{all}}\,k,m;$$

(88)

$$b_{nj} = b_{jn} \quad {\text{for}}\,{\text{all}}\,n,j;$$

(89)

$$\sum\limits_{k = 1}^{M} {a_{mk} } = 0\quad {\text{for}}\,m = 1, \ldots ,M;$$

(90)

$$\sum\limits_{j = 1}^{N} {b_{nj} } = 0\quad {\text{for}}\,n = 1, \ldots ,N;$$

(91)

$$\sum\limits_{n = 1}^{N} {c_{mn} } = 0\quad {\text{for}}\,m = 1, \ldots ,M;$$

(92)

$$\sum\limits_{m = 1}^{M} {c_{mn} } = 0\quad {\text{for}}\,n = 1, \ldots ,N.$$

(93)

Recall the approximate decomposition of real income growth going from period t−1 to t given by (80) above, $\gamma^{t} \approx \tau^{t} \alpha^{t} \beta^{t}$. Diewert and Morrison (1986, p. 663) showed that if g^t−1 and g^t are defined by (85)–(93) above, and there is competitive profit-maximizing behaviour on the part of market sector producers for all periods t, then (80) holds as an exact equality⁴⁰; that is, we have

$$\gamma^{t} = \tau^{t} \alpha^{t} \beta^{t} ;\quad t = 1,2, \ldots .$$

(94)

In addition, Diewert and Morrison (1986, pp. 663–665) showed that $\tau^{t}$, $\alpha^{t}$ and $\beta^{t}$ could be calculated using empirically observable price and quantity data for periods t−1 and t as follows:

$$\begin{aligned} { \ln }\alpha^{t} & = \sum\limits_{m = 1}^{M} {1/2\left( {p_{m}^{t - 1} y_{m}^{t - 1} /p^{t - 1} y^{t - 1} + p_{m}^{t} y_{m}^{t} /p^{t} y^{t} } \right)\ln \left( {p_{m}^{t} /p_{m}^{t - 1} } \right)} \\ & = { \ln }P_{T} \left( {p^{t - 1} ,p^{t} ,y^{t - 1} ,y^{t} } \right); \\ \end{aligned}$$

(95)

$$\begin{aligned} { \ln }\beta^{t} & = \sum\limits_{n = 1}^{N} {1/2\left( {w_{n}^{t - 1} x_{n}^{t - 1} /w^{t - 1} x^{t - 1} + w_{n}^{t} x_{n}^{t} /w^{t} x^{t} } \right)\ln \left( {x_{n}^{t} /x_{n}^{t - 1} } \right)} \\ & = \ln Q_{T} \left( {w^{t - 1} ,w^{t} ,x^{t - 1} ,x^{t} } \right); \\ \end{aligned}$$

(96)

$$\tau^{t} = \gamma^{t} /\alpha^{t} \beta^{t}$$

(97)

where $P_{T} \left( {p^{t - 1} ,p^{t} ,y^{t - 1} ,y^{t} } \right)$ is the Törnqvist output price index and $Q_{T} \left( {w^{t - 1} ,w^{t} ,x^{t - 1} ,x^{t} } \right)$ is the Törnqvist input quantity index.⁴¹

Since (80) now hold as exact identities under our present assumptions, Eq. (84), the cumulated counterparts to Eq. (80), will also hold as exact decompositions; that is, under our present assumptions, we have

$$\rho^{t} /\rho^{0} = T^{t} A^{t} B^{t} .\quad t = 1,2, \ldots .$$

(98)

Thus, it is very easy to implement the above decompositions of real income growth into explanatory growth factors, including the observable measure of technical progress $\tau^{t}$ defined by the right-hand side of (97), which corresponds to MFP growth due to the assumption of constant returns to scale. This result illustrates the exact index number method for estimating productivity growth.⁴²

6 Measurement of Output, Labour, Capital and Productivity Indexes in Practice

This section discusses the data needed to implement calculate the index numbers used by NSOs. The key components are an output index and indexes of the primary inputs of labour and capital.

At the aggregate national level, GDP from the national accounts is usually taken as the output measure. In most countries, GDP is calculated using a Laspeyres quantity index. However, due to it being a superlative index, the USA uses a Fisher quantity index. For labour input, hours worked from household labour force surveys are typically used. A simple and popular measure of productivity growth is then GDP growth divided by the growth in aggregate hours worked. This is often used as an indication of the wage growth that can be expected given it represents the growth in real value added produced by workers that exceeds the growth in hours worked. However, some of this growth can be caused by an increase in capital inputs, or capital deepening, hence the interest in MFP. With more than one input, an index number formula is used to construct an aggregate input.

Before going into more specifics, it is important to note that many countries, such as the USA, Australia, Canada and the UK, have productivity programs which produce industry-level productivity statistics. The case of Australia is given as an example. Figure 2 plots the cumulated MFP indexes for the Australian Market Sector,⁴³ so that the lines represent relative productivity levels compared to the base year, which is the fiscal year 1989–1990.⁴⁴ What is immediately striking is the diversity of the productivity growth experience between 1989–1990 and 2017–2018. This illustrates the benefit of industry-level analysis rather than simply a national aggregate approach.⁴⁵

Fig. 2

(Source ABS [2018a]. Note that the indicated years are fiscal years, which run from July 1 to June 30. The plotted series are cumulated indexes, indicating the level of productivity relative to the base year of 1989–1990)

Multifactor productivity levels, Australian market sector industries

For each industry, value-added growth is taken to be the output series. This is taken from the national accounts and is calculated using a Laspeyres formula. The labour series used can either be a raw hours worked series or a quality-adjusted labour input series. This adjusts hours worked to take into account the changing composition of the labour force. The idea is to adjust for improvements in education and for changes in the age and sex distribution of the workforce, reflecting the assumption that differences in wages among types of workers are determined by differences in their productivity. Wage equations are estimated and the predicted wages used in constructing the weights for aggregating over the hours growth of different types of workers. In doing this, the Australian Bureau of Statistics (ABS) follows the approach of the US Bureau of Labor Statistics (see BLS 1993, 2016; Reilly et al. 2005).

The ABS provides productivity estimates using both quality-adjusted and quality-unadjusted labour inputs. An argument can be made of the use of either. Raw hours worked represent society’s time resources dedicated to production. Being able to get more output growth relative to hours worked growth can be interpreted as an enhancement in the use of these resources, which can be interpreted as productivity growth. Alternatively, the fact that society invested in the improvement of labour quality, through, e.g., education, means that more of society’s resources are tied up in the hours worked, and hence ignoring that can give a distorted view to the meaning of productivity growth, or improvements in the ability to turn inputs into outputs. Quality change effectively means that the inputs are not the same between periods, and it can be argued that this change needs to be accounted for in productivity measure; this leads to the use of quality-adjusted labour inputs.

For capital, it is too complex to go into detail in this chapter, except to note the following. Estimates of productive capital stocks, based on data on past investment along with estimates of how an asset’s services deteriorate over its service life are calculated. Then, it is standard to use a user cost approach for the rental prices of capital (see OECD 2009; Diewert and Fox 2018b). Combined with information from the capital accounts in the national accounts, these rental prices can be used to calculate the cost share of each type of capital considered. The ABS then uses a Törnqvist index to aggregate over the different types of productive capital to create a capital services index. The (quality-adjusted) labour input index and the capital services index are then aggregate, again using a Törnqvist index. Productivity is then defined as the ratio of the value-added index calculated using a Laspeyres index divided by an input index constructed using a Törnqvist index (see ABS 2015a, Chapter 19 for further details).

This mismatch of index number formula between the output and input indexes is not commonly seen in the academic literature, where it is more common to consistently use one index formula for both outputs and inputs, as in the Diewert and Morrison (1986) approach in Sect. 5. The reason why this mismatch occurs in NSO practice is that value added by industry is readily available from the national accounts, where the Laspeyres index tends to be favoured due to being less demanding in terms of data needs (only base period shares are needed for its calculation, whereas the Törnqvist index requires shares from both periods being compared). In addition, the Laspeyres index has an additivity property which is valued in the national accounts community due to it providing a simple additive way of seeing how each component of value added contributes to aggregate growth. From a national accounts point of view, having different outputs indexes for the same industry (depending on the purpose of the index) would be somewhat inconsistent.

For the USA, the numerator of the BLS MFP calculation for major sectors is an index of real value added excluding the government, non-profit or household sectors. This is a Fisher index (see BLS 2007, pp. 7–8). The denominator is a Törnqvist quantity index of quality-adjusted labour (BLS 1993) and capital services (BLS 2006).⁴⁶ For MFP of individual industries, the numerator used by BLS is total gross output for the sector, and the input index is comprised of capital, labour, energy, non-energy materials and purchased business services inputs, where both aggregate output and input indexes have the Törnqvist. That is, intermediate inputs which are subtracted from gross output to produce value added are treated the same as the primary inputs of labour and capital in this case (see Moulton 2018, p. 12). The reason for this approach given by the OECD (2001) is as follows: “At the aggregate level of the economy, gross-output and value-added based measures converge when the gross-output measures are defined as sectoral output. Sectoral output is a measure of production corrected for deliveries within a given sector”.⁴⁷

As for Australia, the BLS calculates capital services (BLS 2006), and the measure of labour input is adjusted for changes in labour composition in addition to changes in hours worked and uses a Törnqvist index to aggregate over inputs.

Using gross output in the numerator in calculating productivity is often referred to as a KLEMS approach, as the input index in the denominator is then comprised of capital (K), labour (L), energy (E), materials (M) and services (S). This approach is synonymous with Dale Jorgenson and his collaborators (see Jorgenson and Timmer 2016).⁴⁸

7 Measurement Challenges

Why don’t we know more after all these years? Our data have always been less than perfect. What is it about the recent situation that has made matters worse? The brief answer is that the economy has changed and that our data-collection efforts have not kept pace with it. “Real” national income accounts were designed in an earlier era, when the economy was simpler…. (Griliches 1994, p. 10)

While productivity slowdowns intensify interest in measurement issues, there remain persistent measurement challenges. New (administrative) data sources and (digital) collection methods can help address these challenges,⁴⁹ but the changing nature of the economy presents new challenges or intensifies old ones. Here, we briefly acknowledge some selected challenges and provide references for those interested in potentially contributing to solutions.

Net Output: Depreciation is not a productive asset, yet is part of GDP. It can be argued that depreciation should be subtracted from value added before constructing productivity indexes. This requires removing the depreciation charge in user cost from factor income and subtracting it like an additional intermediate input from the product side (see, e.g., Diewert and Fox 2016). Even if it is agreed that net output is the appropriate output measure, a conceptual issue arises for which there is perhaps no clear resolution yet; Pigou (1941) argued that the key issue was the maintenance of physical capital, so only wear-and-tear depreciation should be subtracted from output (or income). This contrasted with the view of Hayek (1941) who argued for the concept of real financial maintenance of capital. This means that asset revaluations represented a decline in income and therefore should also be subtracted along with physical depreciation.

Missing Inputs: Land is often omitted from the list of capital inputs in productivity databases. This is true for the EUKLEMS and World KLEMS databases, which also omit inventories (see Jorgenson and Timmer 2016). When land is included, often extreme assumptions are made (e.g. no change in quantity or quality of the land over time), and the estimates can exhibit concerning patterns (see Alston 2018; Diewert and Fox 2018b).⁵⁰

More broadly, environmental and ecosystem services are typically omitted, even water input for agricultural. This is due to the measurement difficulties of accounting for these inputs. However, some progress continues to be made on improving measurement of these key inputs (see UN 2014a, b). In the productivity context, Brandt et al. (2016) looked at the impact of explicitly accounting for non-renewable resources in productivity measurement, which is a start for a broader economic accounting of natural capital and ecosystem services in productivity measurement. However, several important issues remain unresolved regarding the inclusion of natural resources and ecosystems in productivity measurement, such as the appropriate method for valuing the services.⁵¹

There are many more assets that could potentially be considered. For example, Diewert and Fox (2019) advocate having cash balances as part of the asset base.

New Goods and Quality Change: Hulten (2001, p. 29) quoted Adam Smith as follows: “Quality … is so very disputable a matter, that I look upon all information of this kind as somewhat uncertain”. However, quality change is an important feature of any modern economy that should not be ignored. NSOs typically try to adjust for quality change using a variety of methods. For example, Landefeld and Grimm (2000) reported that 18% of US final GDP expenditures were deflated using indexes that are calculated with hedonic methods. However, rapid entry of new goods increases the challenge for NSOs. Finding appropriate prices and quantities for goods like cloud computing can be challenging, and appropriately calculating the price declines for the goods early in their lives can be problematic, leading to nominal output deflators being too high and hence downwardly biasing real quantity growth. Much recent measurement effort has gone into dealing with such measurement challenges.⁵²

Financial and Insurance Services: This is a very difficult and developing area of measurement. This is problematic for producing aggregate and industry-level productivity growth estimates. Output from Financial and Insurance Services is included in GDP and sectoral productivity estimates tend to be high (see Fig. 2 for Australia). However, debate continues about what exactly this sector does and how its output should be measured (for more on this, see, e.g., Diewert et al. 2016).

Intangibles: A major change in the UN System of National Accounts 2008 was there cognition of expenditure on Research and Development (R&D) as capital formation. Many countries have now implemented this recommendation, along with capitalizing other intangibles such as artistic originals, mineral exploration and computer software. However, there remain other classes of intangibles which could also be capitalized, as investments in these also create assets which lasts more than a period. Due to the pioneering contributions for Corrado et al. (2005, 2006), there are now many studies which calculate estimates of broader classes of intangibles. They classified intangibles into three groups: Computerized Information, Innovative Property and Economic Competencies.⁵³ See Haskel and Westlake (2017) for an excellent description of the nature, measurement and increasing importance of intangible capital in modern economies.⁵⁴

Digital Economy: As noted in the introduction, the presence of new and free digital goods and services provides significant challenges for measurement. There are broader reasons than the productivity slowdown to suggest that economic statistics are not keeping up with developments in the economy. With the advent of digital cameras, we are taking more photographs than ever; worldwide an estimated 80 billion photos in 2000 and 1.6 trillion in 2015. The price per photo gone from 50 cents to 0 cents, meaning that we are consuming more yet the activity is vanishing from GDP (Varian 2016). Such examples, and the massive increase in consumption of free digital entertainment and communication, raise concerns about the measurement of economic activity and the welfare benefits accruing from the digital economy. Traditional NSO economic statistics are increasingly seen as not reflecting the experiences of businesses and consumers, leading for calls to completely jettison standard frameworks (e.g. Coyle and Mitra-Kahn 2017). Others have advocated extending traditional statistics (Jones and Klenow 2016; Corrado et al. 2017; Heys 2018; Sichel 2019) and improving measurement methodologies to incorporate new data sources (Bean 2016). If measurement is lacking, through methodological challenges, statistical agency budgets or data availability, then we are severely hampered in our ability to understand the impact of new technologies and goods on the economy, and consequently the prospects for future productivity, economic growth and welfare change.

Recent work by Brynjolfsson, Eggers and Gannamaneni (BEG) (2019), Brynjolfsson, Collis, Diewert, Eggers and Fox (BCDEF) (2019) and Diewert et al. (2018, 2019) has opened up new avenues for exploring the impact of the digital economy on core economic statistics. BEG (2019) demonstrated that massive online choice experiments can be used to elicit valuations of free digital goods. Specifically, consumers’ willingness to accept compensation for losing access to various digital goods can be elicited, providing a valuation. They demonstrated this approach using non-incentive compatible and incentive compatible experiments online, along with laboratory experiments. The incentive compatible choice experiments required participants to potentially give up Facebook for a certain period in exchange for compensation. Their results indicated that digital goods have created large gains in well-being that are missed by conventional measures of GDP and productivity. They concluded that by “periodically querying a large, representative sample of goods and services, including those which are not priced in existing markets, changes in consumer surplus and other new measures of well-being derived from these online choice experiments have the potential for providing cost-effective supplements to existing national income and product”.

BCDEF (2019) extended this work to the measurement of welfare change and derived an extended concept of GDP (“GDP-B”) which is consistent with this welfare change. This framework provides a means by which to understand the potential mismeasurement that arises from not fully accounting for the digital goods. From running incentive compatible laboratory experiments on the willingness to accept (WTA) to forego consumption for eight popular digital goods, they found that valuations vary dramatically across goods, from a median monthly WTA of over €500 for WhatsApp to €0 for Twitter. Yet the measured prices by NSOs is the same: zero. BCDEF (2019) suggest that a new measure of productivity, Productivity-B, could be calculated, using their extended definition of output, GDP-B.

This literature is still in its infancy. Yet it provides an example of how new data collection approaches, utilizing the reach of the digital economy through online experiments, can be used to enhance our traditional measures of welfare and growth.

8 Conclusion

This chapter has considered productivity theory, measurement and challenges with particular reference to productivity statistics produced by NSOs. As should be clear, the challenges facing NSOs in constructing productivity estimates are not insignificant. This has led to questions about the adequacy of methods and data for appropriately measuring economic activity and productivity in modern economies.

There are too many complexities (in each of output, capital and labour measurement) to provide a thorough step-by-step guide to NSO practice in a single book chapter, especially given that agencies can follow different approaches. However, the references to the NSO documentation and international manuals, such as those of the OECD (2001, 2009), provide sources for further reading on the range of decisions, methods and data required for producing aggregate and industry productivity statistics.

While the attention here has been on NSO methods, much of material is of course relevant to productivity measurement at the firm level. With the advent of more firm-level databases, constructed using administrative data and using data linking techniques, much more can be learned about macroeconomic performance from examining performance from the firm level upwards (see, e.g., Syverson 2011; Australian Treasury 2018). Combined with new data collection and emerging measurement techniques which take into account the unique features of a digital economy, it can be concluded that this is a very exciting era for productivity measurement.