Introduction

In any production unit, the characteristics of the external environment generally influence the ability of management to transform inputs into outputs, i.e. to achieve high levels of technical efficiency. Examples of such external variables include type of ownership, location characteristics, labor relations, government regulations and other factors not controllable by management. In order to identify true managerial inefficiency, which is important for designing policies to improve resource allocation, it is necessary to account for the effects of these factors when comparing organizations in terms of efficiency.

Dialysis, due to the large and rapidly increasing number of patients, in conjunction to the large treatment costs, has been among the first medical interventions to be assessed with regard to its efficient use of resources. Investigation of optimal resource use has been the objective of previous efficiency [1–3] and productivity studies [4, 5] in various countries. Interestingly, all showed room for considerable efficiency improvement, particularly in public facilities, but also suffered from common shortcomings. The most profound limitation was that measured inefficiency could be attributed only to management, since particularities of the external operating environment were not accounted for.

Almost 10,000 Greek patients are on some form of renal replacement therapy (RRT) and approximately 75% of them are on in-center dialysis provided in public and private facilities. The estimate of €240 for the average cost of a single dialysis session, [6] implies that the aggregate annual economic impact of dialysis exceeds 240 million Euros in Greece alone. A recent study showed that the treatment was used for 0.05% of the population and accounted for 2% of the national health expenditure in the year 2000. [7] A previous efficiency study showed that the private sector was more technically efficient than the public one. Regarding location, i.e. Athens-based vs. regional, private facilities appeared to perform equally well. [3] However, the private sector has not really invested in dialysis in regions outside of Athens, perhaps because they are regarded as unfavorable operating environments.

The purpose of this study was to measure inefficiency, attributable only to management, in dialysis facilities by implementing a previously outlined technique which eliminates inefficiency components due to the external environment. [8] Data Envelopment Analysis (DEA) was used to yield a radial measure of technical efficiency [9, 10] and additional non-radial input savings, known as slack. Slack represents the under-production of output or the over-use of input. It represents the improvements needed to make an inefficient unit become efficient. These improvements are in the form of an increase/decrease in inputs or outputs. In most efficiency studies, slacks are usually neglected. In the proposed approach, they are used to re-calculate a measure of technical inefficiency that adjusts for differences in the operating environment across production units.

Technical efficiency and the external environment

Technical inefficiency of a production unit is measured by the proportionally equal reduction in inputs required to produce predetermined levels of outputs. [11] The idea is to obtain an efficient counterpart, for each unit being evaluated, and this is empirically achieved in a non-parametric, mathematical programming framework with DEA, or in a parametric statistical framework with Stochastic Frontier Analysis (SFA). Both methods require the construction of a production frontier to indicate maximum production under the current technology, and evaluate individual production with respect to the frontier. [12] DEA, which dominates the efficiency literature, incorporates inputs which are controllable by management. However, socio-economic, environmental and other exogenous factors, known as non-discretionary inputs, may be important in determining efficiency variations across facilities, particularly in public sector production applications. [13]

The effect of such external factors is usually explored with “second stage DEA”, for which various approaches have been described. [14] Some studies use ordinary least squares (OLS), however, the most often encountered approach is the two-limit Tobit technique, which has been adopted as the natural choice for modeling DEA scores in second stage evaluations. This method is suitable when the dependent variables are censored or corner solution outcomes, [15] as in the case of DEA scores which are continuous on the 0-1 interval and take the value 1 with positive probability, while the probability of obtaining the limiting-value 0 is zero. Tobit regression has been used in various 2-stage efficiency studies in health care, such as in hospitals, [16–18] nursing homes, [19, 20] primary care centers, [21, 22] oral health provision, [23] and for estimation of physician efficiency. [24]

An advantage of the two-stage approach is that the influence of the external variables on the production process can be tested in terms of both sign and significance. However a disadvantage is that the second stage regression ignores the information contained in slacks and surpluses (depending on the orientation of the DEA model). This may bias the parameter estimates and give misleading information regarding the impact of each external variable on efficiency, by not providing a separate measure of managerial efficiency. This can be achieved using the “four-stage” procedure which was introduced almost 10 years ago by Fried et al., [8] and tested in a sample of hospital-affiliated nursing homes in the US. Since then the procedure has not been used, to our knowledge, in any healthcare application.

The four-stage technique

The four-stage DEA technique is briefly outlined here and for more detail the reader can resort to Fried et al. [8] In the first stage, inputs and outputs are specified and DEA is used to compute Farrell technical efficiency (TE), which is a measure of efficiency under the restriction that a linear combination of efficient units produces the same or more of all outputs and that the reduction in inputs is equally proportional. The first condition establishes a best-practice frontier, whereas the second is the result of the input-oriented radial efficiency measure, which is only a partial measure of efficiency since it neglects output surpluses and input slacks.

The concept of radial and non-radial input slack is illustrated in Fig. 1. Four units: A, B, C and D use two inputs: x 1 and x 2 to produce the same quantity of output y. Units A and B are technically efficient, whereas C and D are not. Radial technical efficiencies for C and D are TE C  = OC*/OC and TE D  = OD*/OD respectively, and both are less than unity, implying e.g. that unit C could use the fraction TE C of its current levels of inputs to produce output y, were it to operate efficiently. The amount (1-TE C )x c is radial input slack, which is the same proportion for all units by definition. Unit D on the other hand, can further reduce input x 1 from x 1 D to x 1 B after proportionally reducing its current inputs by (1-TE D ) to D * to become radially efficient. The potential additional reduction in input x 1 equal to x 1 B D* is referred to as non-radial slack in input x 1 , and varies across inputs.

Fig. 1
figure 1

Illustration of radial and non-radial input slack

Full size image

The radial measure computed here ignores possible additional inefficiencies implied by non-radial slacks. Furthermore, since the external environment is not yet accounted for, radial efficiency scores generated by the initial DEA model may under— or overestimate the efficiency of units operating in unfavorable or favorable conditions respectively. Unfavorable external conditions mean that additional inputs are required to produce the same level of output in order to overcome the external disadvantage. In the second stage, N input equations are estimated using an appropriate econometric technique. Dependent variables are radial plus non-radial input slack which are regressed against exogenous variables applicable to the particular input. The N (number of inputs) equations are specified as:

$$TS_j^k = f_j \left( {Q_j^k ,\beta _j ,u_j^k } \right),\,j = 1, \ldots ,N,k = 1, \ldots ,K$$
(1)

\(TS_j^k \) is unit k’s total radial and non-radial slack for input j based on the first stage DEA results, \(Q_j^k \) is a vector of variables characterizing the operating environment for unit k, that may affect the utilization of input j, β j is a vector of coefficients and \(u_j^k \) is a disturbances term. It should be noted that the specification is analogous for an output-oriented DEA model, with input slacks replaced by output surpluses. The explanatory variables need neither to be the same across equations, nor to have a linear relationship with the dependent variables, and can be a mixture of continuous and categorical variables. [8] Since the total slack for each input is mostly censored at zero, a system of Tobit regressions is appropriate.

In the third stage, estimated coefficients from the regression are used to predict total input slack for each input and for each unit based on its external variables. These predictions are used to adjust the primary input data for each unit according to the difference between maximum predicted slack and predicted slack. This creates a new pseudo data set where the inputs are adjusted for the influence of external conditions:

$$x_j^{k_{adj} } = x_j^k + [Max^k \{ TS_j^k \} - TS_j^k ]\;j = 1, \ldots ,N,k = 1, \ldots ,K$$
(2)

The fourth and final stage is to use the adjusted data set to run the DEA model again under the initial input-output specification and generate new measures of radial inefficiency. These radial scores measure the inefficiency that is attributable to management.

Materials and methods

Data and sample

The present study is based a 2004 dataset extracted from the monthly archives of the Hellenic Board of Registry Coordination and Control of RRT. These data represent the most updated, detailed and complete information on dialysis facilities’ workload, organization and processes in Greece. Overall 126 facilities were in operation at that time, and only two were excluded because of missing or difficult to interpret data, resulting in a final sample of 124 facilities. Approximately two-thirds were under public ownership, specifically 85 dialysis facilities (68.5%), whereas the remaining 39 (31.5%) were privately owned. Forty-eight (38.7%) were located in the greater Athens area, and this figure confirms the centralized nature of Greece, as approximately 40% of the country’s population resides in this area, according to the last national census, which took place in 2001. Finally, 69 (55.6%) dialysis facilities (mostly public) have been functioning for over 12 years (i.e. before 1992, the year for which complete and updated records exist), and this could imply more experience in processes and procedures on one hand, but less innovation on the other.

Model specification

In the absence of diagnostic checks for DEA model misspecification input–output variable selection is usually guided by expert opinion, past experience and economic theory. [25] In the context of dialysis, an ideal output variable would require patient-level data on clinical indicators, long-term medical outcomes, age and co-morbidities, however such data were practically unavailable. Alternatively, the annual number of dialysis sessions administered in each facility was used to capture the notion of maintaining renal functioning for patients to survive and carry out daily activities with an acceptable quality of life. Inputs reflecting essential health resources needed to provide dialysis were nursing staff and dialysis machines. Nurses, expressed as full time equivalents (FTE), are required for attaching patients to the machines and monitoring all processes. The machines reflect the capacity for dialysis at the required level of frequency.

Physicians were not included in the model because their time is usually split up between the nephrology clinic (where one exists), the outpatient departments and the dialysis unit itself, making it difficult to calculate accurate FTE figures in the present study. Medical nephrology staff is undoubtedly responsible for monitoring the patients’ medical condition and for making all the important clinical decisions related to treatment. However, it is well-acknowledged that a dialysis session can be provided without the presence of a physician [26], implying that physicians are not part of the immediate production process and can be omitted from the analysis, without biasing the results. It is worth noting that, based on this reasoning, many dialysis efficiency studies having used DEA, did not include physicians as one of the input variables. [2, 3, 5]

Analysis

Technical efficiency scores and input slacks for each unit were calculated with the Efficiency Measurement System (EMS) software version 1.3, using an input-oriented, variable returns to scale (VRS) model. The sum of radial and non-radial input slacks was regressed against a set of environmental characteristics, reflecting differences in facility organization, economic incentives, financial constraints, location, production scale, innovation and experience. Four explanatory variables, beyond the influence of managerial control, were chosen to be included in two Tobit regressions: i) ownership, i.e. public or private to account for structural and organizational differences, ii) location, i.e. Athens area or elsewhere to account for accessibility and centralization, iii) years in operation, i.e. <12 or ≥12 years to account for experience and innovation and iv) size, defined by the annual number of dialysis sessions, i.e. <6,000 (small), 6,000–12,000 (medium) or >12,000 (large), to account for scale effects in the external environment. It should be noted the number of sessions reflects upon the number of patients served in each facility, since the correlation between the two was very strong (Pearson’s r = 0.972, P < 0.001). Tobit regressions were performed with STATA ver. 8.0.

Results

The initial DEA results (Table 1) showed a large variation in efficiency scores across dialysis facilities. The average efficiency score for the entire sample was 0.682 and 9 best practice units (benchmarks) were identified. Thus, theoretically, if production in the 115 inefficient facilities followed the practice of the 9 benchmarks, all else being equal, current levels of mean outputs could have been achieved with an average 32% reduction of resources utilized. Potential savings were even higher, by approximately 4.5% (nurses) and 3.8% (dialysis machines), if non-radial slacks were to be included. However, part of these potential input savings may not be achievable since some facilities operated under unfavourable conditions that prevented them from fully utilizing their available resources. Furthermore, statistically significant efficiency score differences were observed according to environmental factors, with the private (P < 0.001), the Athens-based (P = 0.059), the newer (P < 0.001) and the smaller (P = 0.024) dialysis facilities performing better.

Table 1 Stage one efficiency scores overall and by environmental factors
Full size table

The dependent variables in the two second-stage regression equations (one for each input) were the total radial plus non-radial slacks, and independent variables were dummies for ownership-location-years-size combinations. Public ownership, Athens-based, <12 years operation and small size were taken as the reference cases. Single equation Tobits were estimated since the independent variables were the same across the two input slack equations. Parameter estimates are summarized in Table 2. A (positive) negative coefficient on a variable suggests that the external environment is (un) favorable compared to the dummy taken as the reference case, since it is associated with (greater) less excess use of input.

Table 2 Tobit regression analyses (N = 124)
Full size table

Private ownership had a significant negative coefficient in one input equation (nurses) and an insignificant coefficient on the other (dialysis machines), suggesting that it is a favorable operating environment probably due to the cost-discipline imposed by its for-profit status. Non-Athens location had a positive coefficient in both input equations, though statistically insignificant. This implies an unfavorable operating environment compared to facilities based in the greater Athens area. More years in operation also suggests an unfavorable operating environment compared to newer facilities since it had positive coefficients in both inputs and significant in the case of the nurses input. This could be the result of older and less innovative procedures applied in these facilities.

Size had the most profound impact as the coefficients for medium— and large-sized facilities were positive and statistically significant in both input equations. This implies that decreasing returns to scale, under which the larger facilities typically operated, constitute an unfavorable working environment compared to increasing returns, which were predominant in smaller facilities. Obviously both models, as a whole, were statistically significant (Chi-square P < 0.001). Pseudo R2 values were 0.102 and 0.063 respectively. However, this may not be the best measure of fit and, hence, was improved by calculating R2 between predicted and observed values. The R2 values were now 0.452 and 0.265 respectively, much closer to what an OLS regression would have given.

The parameter estimates presented in Table 2 were used in stage three to adjust the initial input data according to Eq. (2), and Table 3 shows the predicted slacks and the maximum predicted slacks for both inputs by ownership-location-years and size combinations. The adjusted data controls for the influence of the external operating environment, as far as the above variables are concerned. The small number of observations in some size categories was not a problem since the coefficients were estimated based upon the total number of dialysis facilities in the specific size category, regardless of their external circumstances. For example, although there were only two public-Athens-newer facilities in the small size category, the coefficient was based upon 37 observations. The maximum predicted slack (least favorable external environment) was observed for older, public, non-Athens units in both slack equations, and this applied for all size categories. On the other hand, the lowest predicted slack, implying the most favorable operating environment appeared in newer, private, Athens-based units, again for both input slack equations and facility size categories.

Table 3 Predicted slacks and maximum predicted slacks by ownership-location-age-size (N = 124)
Full size table

The fourth stage of the procedure was to re-run the initial DEA model using adjusted input data, according to the parameters presented in Table 2 and Eq. (2). This produced a composite radial score, which incorporated technical inefficiency and the effects of the external environment. As a result for controlling for the external environment, the average efficiency score increased by approximately 10% (68.2% to 78.1%). This suggests that initially, the penalty to dialysis facilities operating under unfavorable circumstances was greater than the benefit to dialysis facilities operating under favorable circumstances. The decrease in standard deviation may reflect the fact that without controlling for the external environment, efficiency scores of facilities that operated in favorable circumstances were biased up, and the efficiency scores of facilities that operated in unfavorable circumstances were biased down. By adjusting the data, the spread was narrowed.

The correlation coefficient between stage one and stage four efficiency scores was 0.67 (Pearson’s r, P < 0.001). Adjusting for the external environment apparently made a difference in terms of efficiency scores and this is illustrated in Table 4. Newer, private, Athens-based units operated in the most favorable external environment and the average scores for these units were lower in stage four (after adjustment) compared to stage one in all size categories. On the other hand, older, public, non-Athens units operated in the least favorable external environments and the average scores for these units were higher in stage four.

Table 4 Mean efficiency scores by ownership-locattion-age-size for stages 1 and 4 (N = 124)
Full size table

Discussion

DEA, as a deterministic method, suffers from measurement errors in the included variables and from the omission of various unobserved and potentially relevant variables. The impacts of such variables could be captured by a disturbance term in a stochastic model. [27] Another problem is that among the omitted variables are those referred to as environmental variables, which typically capture features of the operating environment thought to have an impact on efficiency, but are usually omitted due to the lack of prior knowledge of the direction of their impacts. These drawbacks result in typical DEA models not providing a good measure of managerial performance as they may penalize efficient units operating in unfavorable external environments and reward less efficient units operating in favorable environments.

In the present study, information contained in input slacks was used to adjust the variables in the DEA data set and to generate new measures of radial efficiency, which represent inefficiency attributable to management. The multi-stage technique used has been previously described and tested in a health care application, [8] but has not been used to inform decisions in the health care sector. The field of study was the Greek dialysis sector, which has been previously shown to suffer from both technical and scale inefficiencies. [3, 5] However, these inefficiencies were implicitly assumed to be the result of poor managerial decisions concerning organization and operation of the dialysis facilities.

The suggested contribution of the present study is the isolation of the managerial component of inefficiency for individual dialysis facilities. This was achieved by adjusting primary input data by the difference between maximum predicted slack and predicted slack in order to make efficiency comparisons on a level “playing field”. The rationale of maximum predicted slack was to establish a base equal to the least favorable set of external conditions, which in this case corresponded to older, public-sector, non-Athens dialysis facilities. All other facilities were operating under more favorable external conditions and were penalized, via input adjustment, for the fewer inputs required to operate in these conditions. This made it possible to isolate managerial inefficiency by re-running the DEA model on an adjusted data set. The variables chosen to represent the external environment, i.e. ownership, location, age and size, were tested with univariate analyses (t-Test and ANOVA) and were shown to generate statistically significantly different (P < 0.001) technical efficiency scores. This justified their inclusion in multivariate Tobit regressions.

Efficiency scores should always be interpreted with caution, [28] and this applies even more so in the case of slack-adjusted measures which represent theoretically feasible input reductions if the unit operated in the worst environment and performed up to best practice. Facilities operating in more favorable environments should be able to decrease inputs further. In this study, mean technical efficiency increased from 68.2% to 78.1% after adjustment. This was a result of public dialysis facilities, which dominated the sample, increasing their mean technical efficiency from 63.5% to 78.3%, while private facilities witnessed a slight decrease from 78.5% to 77.6%. These figures suggest that the statistically significant differences observed in first stage efficiency scores between the public and private dialysis sectors (not shown) were the result of the public facilities being highly penalized for their unfavorable environment and not due to lapses in managerial judgment. Indeed after adjustment, stage four efficiency scores for the two sectors were 78.3% and 77.6% respectively (P = 0.783), implying that on a level playing field, the public sector was just as efficient (or inefficient) as the private one.

The results from this study could have important policy implications for the Greek dialysis sector. It has been suggested that ownership type is one of the main determinants of efficiency in health care units [29] and this has been also confirmed in the context of dialysis. [1–3] Private dialysis facilities operating under market competition conditions are forced to be efficient and profit motivation drives them to suppress the resources utilized and increase their outputs as much as possible. However, this study demonstrated the importance of taking into account the external environment in which the facilities operate in. A previous Greek study suggested that the higher efficiency scores observed in the private sector could be attributed to better working practices, [3] whereas the current results imply that this assertion should be revisited. The same applies for the other three external factors, namely location, age and size which also produced significantly different technical efficiency scores initially (Table 1), but after slack adjustment score differences were statistically insignificant, supporting the fact that environmental factors which are beyond managerial control must not remain unaccounted for.

Efficiency is usually the primary criterion in resource allocation, however in the healthcare arena it is not the only one desired by society and its agents, i.e. politicians and/or policy-makers. Equity or “fairness” is equally important and conscientious efforts are required to achieve an acceptable balance between the two, as they are usually mutually exclusive. [30, 31] In the present study, assuming that patient numbers (and hence dialysis sessions) are practically uncontrollable, efficiency improvement in public-sector facilities implies reducing resources and, from an extreme point of view, even shutting down facilities which appear to be technically inefficient. From a social perspective, this is undesirable for populations with limited options of health care access. For example, it would be contradicting the criterion of equity and social cohesion to shut down a dialysis facility on a small island, despite the fact that an efficiency analysis may be pointing in that direction. Future efficiency studies could perhaps consider the generated “social good” as a weighted parameter in a production model.

This study has some limitations, which should be taken into account. The production model for the dialysis facilities units incorporated two input variables and one output, and it is possible that it is too simplistic. Dialysis patients are not a homogenous group and there is variation, between facilities, in patient factors affecting the process (e.g. age, co-morbidity). Furthermore, the data did not allow for a separation of the possible skill mix of nursing staff and hence the study was based on the assumption that a FTE nurse was equally weighted across all units. The other input measure, i.e. the dialysis machines, refers to the number of machines functioning (i.e. providing dialysis slots) and not to the total number of machines owned by each unit. Although this study has achieved its primary goal, which was to disentangle management efficiency from environmental factors, it did not deal with another phenomenon known to influence producer performance, i.e. statistical noise. [27] The impact of good and bad luck, omitted variables and other related phenomena would be collected in a random error term in a regression-based stochastic performance evaluation and this could be a future goal for this line of research.

Conclusions

This study used a previously introduced four-stage DEA technique to account for the influence of external environmental factors on the technical efficiency of dialysis facilities in Greece. These factors included ownership, location, years in operation and size (proxied by the annual number of dialysis sessions provided), all of which are thought to be outside the control of management. The results clearly suggest that not controlling for external influences may lead to erroneous DEA efficiency measurements, which in turn may provoke uninformed policymaking decisions with unknown but, most likely, significant consequences for patients, service providers and the health system overall.