Keywords

1 Introduction

Detailed descriptions of the method and its multiple extensions have been published in a variety of methodological references; a thorough breakdown is available, e.g., in Charnes et al. [6, 7] and Cooper et al. [10]. In brief, classical approach to DEA can be described as follows [7, 10, 14, 15]. There are K objects, or the decision-making units, each of which has multiple indices: a vector m of expended resources, or inputs, Ikm, and the vector of n results or outputs, Okn. Each resource has a weight of xm, and each result—a weight of yn in the formula for DEA efficiency calculation. It is important to note that the weight of each input or output is not known a priori. Using linear programming techniques, DEA algorithm solves a system of equations that optimizes a combination of weights, defining the object as either efficient or inefficient relative to other objects in the evaluation process. The efficiency of each object, formally, is the ratio of sum of all obtained results to the sum of all expended resources:

$$\frac{{\sum\nolimits_{n = 1}^{N} {O_{kn} } y_{n} }}{{\sum\nolimits_{m = 1}^{M} {O_{km} } x_{m} }}$$
(1)

Certain limits are imposed on the optimization algorithm. In basic DEA models, the value of the results cannot exceed the value of the expended resources. However, this assumption has been relaxed in more advanced models, called super-efficiency models (see, e.g., [16, 21]). A unit is 100% efficient if none of its inputs or outputs can be further improved without worsening other inputs or outputs. Adding new units into the mix may change the outcome of the estimation for any of the remaining units.

The resulting formulation of the optimization problem for the k-th object appears as follows. The first equation is the objective function, maximization of the efficiency of each evaluated unit, subject to imposed constraints (equations that follow).

$$\hbox{max} z = \frac{{\sum\nolimits_{n = 1}^{N} {o_{kn} y_{n} } }}{{\sum\nolimits_{m = 1}^{M} {I_{km} x_{m} } }}$$
(2)

Subject to:

$$\sum\limits_{n = 1}^{N} {O_{kn} y_{n} } - \sum\limits_{m = 1}^{M} {I_{km} x_{m} } \le 0\,\quad k = 1,2, \ldots ,K$$
(3)
$$x_{m} , y_{n} \ge 0 \;for\; all\; m, n$$
(4)

Such objective functions are linear fractional, and they are linearized by making the denominator of the objective function equal to 1. The resulting optimization system looks as follows:

$$\hbox{max} z = \sum\limits_{n = 1}^{N} {O_{kn} y_{n} }$$
(5)

Subject to:

$$\sum\limits_{m = 1}^{M} {I_{km} } x_{m} = 1$$
(6)
$$\sum\limits_{n = 1}^{N} {O_{kn} y_{n} } - \sum\limits_{m = 1}^{M} {I_{km} x_{m} } \le 0\;\quad k = 1,2, \ldots K$$
(7)
$$x_{m} , y_{n} \ge 0 \;for \;all \;m, n$$
(8)

Models can be output-oriented (as the system above, aimed at maximizing the output), or input-oriented (aimed at minimizing inputs). Input minimization is used when the purpose of research is to examine the extent to which resources can be reduced while maintaining the existing output levels. Output maximization looks at the levels to which the results can be raised given the current level of inputs. For the above system, the dual linear programming problem, minimizing inputs, is

$$\beta * = \hbox{min} \beta_{k}$$
(9)
$$\sum\limits_{k = 1}^{K} {O_{kn} } z_{k} \ge O_{kn} \quad \,n = 1,2, \ldots ,N$$
(10)
$$\sum\limits_{k = 1}^{K} {I_{km} } z_{k} - I_{km} \beta_{k} \le 0\quad \;m = 1,2, \ldots ,M$$
(11)
$$z_{k} \ge 0\,k = 1,2, \ldots ,K$$
(12)

The optimal value of βk becomes the efficiency of the DMU k. By virtue of dual theorem of linear programming (LP), z = β*. This is derived from the foundational property of linear programming, where a dual of a LP is created from the original program. It is done in the following way: each variable in the primal LP is a constraint in a dual LP; each constraint in the primal LP is a variable in the dual LP; the objective direction is reversed—maximum in the primal is the minimum in the dual and vice versa [17]. Therefore, in DEA, either of the problems can be used, depending on the need, to evaluate efficiency as the ability to maximize outcomes or minimize inputs. As a result of the optimization procedure, we have a matrix of the optimum values of {xkm; ykn}, and due to the imposed constraints, the resulting efficiency of each DMU cannot be greater than 1.

2 DEA Research Design

Variable selection and choice of return to scale are crucial for DEA research design. Both should be either theoretically driven or depend on an applied request from the client.

2.1 Variable Selection

In standard DEA, it is assumed that a variable constitutes either an input or an output. However, just as with regression methods, where selection of a dependent variable depends on a research context, in DEA some measures can play either input or output roles. When theoretical roles of variables are not clear, a mathematical approach to variable selection may be necessary. Zhu and Cook [22] developed models that can help classify variables as either input or output; Cook et al. [8] provide the methodology for evaluating performance of the units where some factors can simultaneously play both the input and the output roles.

In addition, in real life, variables used for analysis are rarely homogeneous. For example, in the context of marketing research, such variables as internal company resources (choice of marketing mix, tangible and intangible resources) are discretionary for each company (meaning, the company has the ability to control their production and use), whereas the global competitive environment in which the company operates is non-discretionary. DEA allows for such heterogeneity in the data, providing an option for the researcher to select the non-discretionary indices that the DMU cannot control and appropriately exclude them from the calculation of an individual unit’s efficiency, while allowing them to remain in the model for calculating the efficient frontier. Detailed description of models with non-discretionary inputs is provided, e.g., in Muñiz [18], Bessent et al. [4], and Banker and Morey [3].

2.2 Choice of Return to Scale

The economic concept of returns to scale (RTS) has further extended the applicability of DEA models [9]. Original extensions were suggested by Banker [1] and Banker et al. [2]. Nicholson [19] provides an intuitive explanation of constant returns to scale (CRS) as an “increase in outputs proportionate to the increase in inputs;” less than proportionate increase means diminishing returns, and more than proportional—increasing returns to scale (IRS). Most DEA software tools now provide options that incorporate all types of returns to scale for estimating the efficiency of the units; VRS models allow for units’ returns to vary between the DMUs. Non-increasing returns to scale (NIRS) put an additional limitation on the VRS, where returns to scale for individual units are limited to constant or decreasing, forming a concave production function. While it is possible to determine the return to scale mathematically, there should be a theoretical foundation behind the model that determines that type of returns that one expects the DMUs to follow.

3 DEA-Generated Results

Depending on the program used, in addition to evaluation of efficiency, the output includes weights of each index in the efficiency calculations, slacks, improvements, graphical outputs, a subset of peer DMUs for each inefficient unit, and some others. Theoretical description of the most common outputs follows.

3.1 Efficiency

Efficiency is the relative performance indicator of a DMU in a group that it is being compared against. It is measured on a scale from 0 to 1, or as a percentage, depending on the software. Efficiency of one (100%) means that the unit is relatively efficient; anything less indicates that it is inefficient, with the actual value indicating the level of efficiency. Inefficiency means that either the inputs are not fully utilized, given the outcome, and could potentially generate a greater outcome (input-based models), or the outcome needs to be augmented (output-based models).

3.2 Weights

In the original model, Charnes et al. [7] accounts for the fact that DMUs may value inputs and outputs differently, so each unit is given the “freedom,” mathematically, to vary the set of weights that allow it to achieve the maximum possible efficiency position relative to other units. The weights, therefore, are very flexible and unit-free, and are unrelated to any other input or output. If a unit is inefficient even with the most favorable choice of weights, relative to other units, then it is a strong statement to the unit’s true lack of efficiency.

3.3 Slacks

Reducing inputs or augmenting outcomes alone may not always be sufficient. This situation is defined as a “slack,” meaning excessive inputs or missing outputs. This result shows which resources are utilized completely to achieve a certain outcome, and which have an excess and could be reduced to improve performance.

3.4 Improvements (Targets)

One of the main benefits of DEA is estimating the magnitude of changes in the resource that is required for the unit to achieve the 100% efficiency. Improvements are provided in the form of changes to variables on the original scale. Some software packages may offer an option of a percentage change.

3.5 Peer Group (Peers, Reference Sets)

For every unit that is inefficient, DEA finds at least one unit that is efficient with the same basic pattern of inputs and outputs. Direct comparison of the peers is not always possible without additional data scaling, which can be done on either inputs or outputs. However, the peer group allows evaluating each inefficient unit’s unique path to efficiency.

3.6 Cross-Efficiency

In addition to building frontiers to identify the most efficient cases, DEA constructed the cross-efficiency matrix—a table where the number of rows (i) and columns (j) equals the number of units in the analysis. For each cell (ij), the efficiency of unit ij is computed with weights that are optimal to unit j. The higher the values in a given column j, the more likely it is that the unit ij is an example of more efficiently operating practices. Doing so DEA allows us to match all units depending on the similarity of their efficiency function.

4 Evaluating Efficiency Over Time

DEA works with panel data. Because quite often scientists and practitioners are interested in the development of a phenomenon over time [13], this option can become invaluable for longitudinal efficiency comparison.

In classical productivity literature, changes in efficiency over time can be broadly divided into five categories [12]: (1) producing the same outputs with fewer resources, (2) producing more outputs without changing the resources used, (3) producing more outputs with fewer resources, (4) provide a larger increase in the outputs for an increase in inputs, and (5) provide a smaller reduction in outputs for the expected decrease in resources. The first three components are referred to as “technical efficiency,” and the last two—as “scale efficiency.” The total productivity growth achieved by the unit is estimated through the Malmquist index, first introduced by Caves et al. [5] and later extended by Fare et al. [11]. It is defined using the distance functions and does not require any assumptions regarding efficiency; therefore, it can distinguish between the factors causing chances in productivity. In a DEA setting, Malmquist index measures the change between data points by providing the ratio of distances from each data point to a common technology. As a result, it is decomposed into the technological change (TC) and efficiency change (EC). The latter is in turn is divided into the PEC (pure efficiency change) and SEC (scale efficiency change) indices for variable and constant returns to scale [20], respectively. Technical efficiency is the overall change in production technology, which reflects the movement in the production frontier. Pure efficiency change component is measured relative to the true VRS frontier; where SEC is the “magnitude” component [11], which quantifies the productivity loss or gain of the unit itself. As a result, the Malmquist index is the product of change in relative efficiency between two different time periods, and the value of index equal to 1 implies no change in total factor productivity, less than 1—deterioration, more than 1—growth. Same interpretation applies to each individual component, and the different DMUs can be compared to each other directly in their changes of the Total Factor Productivity Change (TFPC).

5 An Application: Effectiveness of the Marketing Strategies

To illustrate the DEA capacities to evaluate efficiency of marketing strategy we choose an automotive company in Russia. The effectiveness of the company’s marketing strategy is measured relative to other competing brands. In Table 1 we can see the efficiency scores, where effective brands get hundred, and non-efficient strategies have scores below hundred percent. We can also observe the change of efficiency in time. For example, marketing strategy of the car 9 was inefficient in 2017, and become efficient in 2018; and vice versa, brands of the cars 3 and 12 was inefficient in 2017 and become efficient in 2018. Finally, six brands have inefficient marketing strategies in both periods (Fig. 1).

Table 1 Marketing efficiency scores, %
Full size table
Fig. 1
figure 1

Marketing peers, 2018

Full size image

Tables 2 and 3 shows efficiency breakdown for the specific models of the analyzed brand with respect to spending via certain advertising channel. In Fig. 2, a one (1) against a channel shows that advertising through it was contributing to increased sales of an indicated model. To increase sales of the brands, they use advertising channels differently. For example, car 1 use regional TV channel only, and car 2 also use satellite TV. Regional TV is the most popular advertising channel to contribute in the cars sales. It is used by sixteen brands out of seventeen under study. Next popular advertising channels are outdoor marketing (10 cars use it) and satellite TV with 9 cars which use it.

Table 2 Marketing weights, 2018
Full size table
Table 3 Marketing slacks, 2018
Full size table
Fig. 2
figure 2

Marketing cross efficiency, 2018

Full size image

In Table 3, the one against a channel for a specific model indicates that the company spent too much in resources for the level it achieved for a certain brand. When taken together, this information can help the company evaluate its marketing strategy. For example, spending on advertising in national TV was efficient for Cars 1, 9, 10, 12, 13–15; for all the other ten models, the company can cut national TV advertising without hurting the sales. DEA provides the exact “slack” numbers that could be cut without losing efficiency.

Table 4 presents marketing improvements or targets. For inefficient brands it shows the percentage of how much advertising spending should be reduced to increase sales. Also, it shows how sales will increase, in percentage, if the spending will be decreased respectively, as recommended. For example, car 3 have to cut digital advertising costs by approximately 19%, press advertising costs—by 72%, radio—by 19%, satellite TV—by 99%, national TV—by 40%, and regional TV—by 18%; which will lead to increase in sales by 20%.

Table 4 Marketing improvements (targets), 2018, %
Full size table

Table 5 presents information for inefficient units about their peers. If brand is inefficient it means that it is under efficient frontier and the target position of this inefficient brand is a linear combination of the efficient brands. Therefore, each brand has two or more values (more than zero and less than one) related to its peers, which is called lambdas. Efficient brands have lambdas equal to one. For example, for car 3 we have car 1 and car 15 as it peers, which mean that the target position of the car 3 on the frontier is the linear combination of the spending and sales of car 1 and 15. If we will dichotomize lambdas in Table 5 in a way that zeroes remain zeroes and values more than zero will be ones, than we can get adjacency matrix for the network of marketing peers, Fig. 1.

Table 5 Marketing peers (lambdas), 2018
Full size table

Figure 1 shows a directed network, where the inefficient models are connected to the efficient—those which, with similar costs, achieved a greater sales result. Several things can be learned from this graph. First, connected car models are competitors to the efficient models; in other words, they use the same marketing strategy, but less efficiently. Second, because they follow the same marketing strategy, they are also competitors to each other—or, to similar inefficient models. Finally, this network can also be interpreted and analyzed as pseudo-bimodal, with one mode of “efficient” units, and another—of inefficient. Standard bimodal network analysis can then apply.

Table 6 present marketing cross efficiency values for each brand. We can interpret the values in the matrix in a way that each brand get efficiency scores calculated with the resources of related brand (e.g. car 3 with resources of car 1 will get 99.9% efficiency score; car 3 with car 2 resources – 20.4 efficiency score, etc.). If car has efficient advertisement spending than its diagonal element is equal to 100. If not—below 100. The idea is that we get numbers that show similarities between brands not only for peers but for all brands. Let’s take an example of comparison car 10 with car 1, here the score is more than hundred percent (100.1). So, the interpretation is that if car 10 will use resources of car 1 it will be more efficient compare to itself in 46.8%.

Table 6 Marketing cross efficiency, 2018
Full size table

Table 6 is also adjacency matrix, so it can be presented as the weighted complete network, where we have weighed links from each car to each car, weights are the similarity values from Table 6. This network (Fig. 2) can be used to study further similarities between brands utilizing clustering and classification methods.

These indices show the increase/decrease of the efficiency of each unit between time periods. The change in efficiency of a brand other time can happened due to the change of efficiency of other brands (move of efficiency frontier); or due to the change of efficiency of a brand itself. TFPG index summarized both kinds of change. Therefore, efficient brands (have 100% efficient score stable through time) get only ones in all indices or get more than ones, mining that they even improve their position over time. Those brands where we observe decrease in efficiency (car 3, 6, 7, 8, 10, 12) have PEC index below one, and TC index more than one, mining that the decrease of their efficiency happened due to the worse performance relating to the brand itself and not to the frontier to which actually they become closer. Cars 9, 16, and 17 increase their efficiency both based better performance related to itself and others (frontier) (Table 7).

Table 7 Evaluating marketing efficiency over time (Ray and Desli Malmquist Index), 2017–2018
Full size table

6 Conclusion

Using the example of spending on marketing, we demonstrated the methodological capabilities of Data Envelopment Analysis, a non-parametric and mathematically rigorous method of evaluating effectiveness of different companies relative to each other, with a variety of inputs and outputs. DEA is rarely used in applied research compared to other popular methods such as linear regression. Yet, it has a number of advantages. First, as a non-parametric method, it is free from the limiting assumptions of data independence and identical distribution. Second, it can be used for data of all types, not just scaled on an interval or ratio levels, and can be applied to any combination of such data. Third, it can take a large number of both inputs and outputs. Because in applied research a large number of studies involve multiple independent and dependent variables, often measured on different scales and with different strengths of association, the usability of DEA should not be underestimated. This method overcomes many limitations that are inherent to data in the computer science field, and we hope that researchers will take advantage of this tool to answer important questions not previously examined because of data issues.