Introduction

Over the last decades, Chile has made a significant effort to guarantee its population equitable access to good quality healthcare. A positive health index is a good indicator of development for countries. Among others, the infant mortality rate has been reduced by an order of magnitude in the last 40 years. Thus, while in 1960 the number of infant deaths age one or younger per 1,000 live births was 120.3, in 2005 this rate had been reduced to 7.9. The projected rate for year 2010 is 5.5 [1]. Another important index is life expectancy at birth, which has increased over the same period by 29%. In 1960 [2] the value of this index equaled 58.05 years, while in 2007 this value reached 78.45 years, a rate similar to those of developed countries [3]. These qualitative and quantitative improvements required devoting more financial resources to the public health budget, mainly for improving infrastructure and technological upgrades, socio-demographic changes, increased demand, and additional health benefits provided to the population.

Rodríguez and Tokman [4] estimated that Chilean´s healthcare budget increased by 169% in real terms between 1990 and 1999. They also acknowledge an annual reduction in the efficiency of public healthcare spending of 5.2%, accumulating up to 45% from 1992 to 1999. Furthermore, they mentioned that some problems persist in some health services due to operational complexities. These factors apparently reduce the levels of performance and cause disappointment among the population.

Currently, the achievement of the Primary Healthcare Center (PHC) main goals is measured by several indicators, such as the Activity Index of Primary Healthcare (AIPH) and the Explicit Healthcare Guarantees (EHC). The main purpose of the AIPH, which is controlled biannually, is to determine the extent of the reduction of the municipalities’ budget for next year, if the PHCs they manage fails to meet its healthcare performance targets. The main purpose of the EHC index, which is computed monthly, is to provide follow up information for each specific healthcare guarantee, mainly in terms of waiting lists and goals achieved. The EHC, as part of the healthcare reform, provides a health policy to guarantee access, opportunity, financial protection, and quality of healthcare nationwide.

From a managerial point of view, understanding how scarce resources are being used in the public system is essential for decision making and resource allocation policies. The public primary healthcare system provides services for up to 50% of the Chilean population. Worldwide, several studies have used parametric and nonparametric methods to assess the technical efficiency of hospitals and healthcare centers [5–7]. However, efficiency applications in primary healthcare are scarce, mainly in Chile.

Despite the country’s concern for the efficient use of the resources and the encouragement to use continuous improvement programs in the public sector, this research is the first empirical study supported by the Department of Primary Healthcare at the Chilean Ministry of Health, which uses DEA to measure technical efficiency. This research comprehensively evaluates the technical efficiency of primary healthcare, and looks into the sources of inefficiency from a managerial perspective. To the best of our knowledge, in Chile there is only one independent and privately performed study that attempted to cover this topic [8]. However, such study was limited in scope, and did not determine the sources of operational inefficiencies from a managerial standpoint.

Most municipalities in Chile have either a Health Office Director or a Corporation to run their PHCs. A few PHCs are managed by the Ministry of Health through regional offices. PHC are mainly financed by the government through the municipalities’ annual budget. This budget is determined as follows: a per capita amount provided by the Ministry of Health (adjusted by various factors), which represents up to 95% of the total budget for some municipalities; local government funds, which varies according to their annual income; special health government programs; and infrastructure investments (centralized by region or nationwide). The per capita assignment is generally not enough to cover the requirements of the public demand. Furthermore, wealthier municipalities are able to make higher contributions to the health expenses of the PHCs under their administration.

The main objective of this research is to empirically determine which factors affect more the technical efficiency of the PHCs managed by each municipality using a multiple step analysis. First, a cluster analysis is performed to homogenize the population under study. Second, the relationship among the variables is estimated using a Data Envelopment Analysis (DEA) model. From several input specifications, four models are used to calculate the technical efficiency of each municipality. Third, confidence intervals and bias-corrected efficiency values are obtained by applying a bootstrap algorithm. Fourth, a Biplot method is employed to reduce the number of managerial external variables that could explain the inefficiency of the municipalities. Finally, the variables selected in the previous step are used in a Tobit Regression Model. This last stage helps determine the effect those variables have on the inefficiency scores.

Problem definition

The productivity of the Chilean Public Health System has not improved as expected [9]. The Municipalities National System of Information gathers data regarding, among others, the annual per capita spending in primary healthcare. This amount varies considerably between different municipalities in the country. While performing straightforward comparisons between municipalities, some relevant discrepancies among productivity rates have been identified. This could be an indication of inefficiency.Footnote 1 For example, whereas the municipality of Temuco has an annual per capita expenditure of U$ 51.8 in year 2004, and a medical visits rate per thousand inhabitants of 1,278, the municipality of Laja has a medical visits rate of 8,109, 6.35 more times than Temuco’s for the same per capita level of spending.

There also exist significant differences in health equity. As an example, the infant mortality rate in the municipality of La Reina (Metropolitan Region) was 2.6 in year 2004, while the municipality of Nacimiento (Bio Bio Region) had a rate of 27.2, that is 3.23 times higher than the national average and 9.5 times higher than La Reina, which is a much wealthier municipality.

Several factors can affect the efficiency of primary care. Sáez [10] mentions several constraints on the use of primary care services, including: health status of the individual, gender and age, socioeconomic variables, family and cultural variables, and variables related to the healthcare professionals, e.g. staff mix. According to Poblete et al. [11], primary healthcare in Chile has a wide variability in its capacity to diagnose, define strategies, and evaluate the ongoing processes. This implies that it could be possible to achieve different levels of performance and productivity at the level of the healthcare centers.

Material and methods

Data envelopment analysis

Jacobs et al. [12] highlight the importance of considering how external conditions can affect the ability of a system to achieve its objectives. However, given the lack of a competitive market in public healthcare, all systems should be regulated in certain ways. This requires the development of comparative measurements of performance, a task that is addressed effectively by means of efficiency models. Traditional performance management indicators measure how the activities transform inputs into outputs, either goods or services. The measurement of efficiency allows determining how well resources are used to produce the desired outputs. Models for assessing healthcare efficiency usually involve multiple inputs and outputs generated by the unit of analysis usually known as Decision Making Unit (DMU), and whose level of efficiency could be influenced by external factors. Several methods for assessing the performance of health systems are designed to measure their technical efficiency. Two main approaches are widely used: Stochastic Frontier Analysis (SFA) and Data Envelopment Analysis (DEA). The concept of a “frontier” assumes we can identify benchmark units, which are a combination of efficient organizations that can virtually exist. As a result, inefficient units could try to emulate the practices of the efficient ones in order to improve their performance.

The DEA technique does not need to define a priori a functional form for the frontier, which is one of its main advantages. It assumes that an organization that uses fewer inputs than another to produce the same amount of outputs can be considered more efficient. The frontier is constructed with the empirical units that achieve the highest output/input rate, thus forming a series of linear segments connecting the “best practices”. This gives an approximation of the true frontier of efficiency. The units outside the “ideal frontier” are considered to be inefficient. The level of (in) efficiency is then measured by the distance between its location in the plane and the estimated frontier. Its deterministic condition is a drawback of this method. Furthermore, the results are highly sensitive to outliers and insensitive to statistical noise.

Additionally, the measurement of efficiency depends on the assumption that efficient organizations are in fact efficient [13]. Among DEA’s most important benefits we can mention the ability of using multiple inputs and outputs with different weights assigned to each variable; the potential to identify sources and amounts of inefficiency for each input and output of those entities under study; and its capability to identify “benchmark” units within those classified as efficient [14]. Although there is no consensus on which method of evaluation is better, DEA seems to have a wider acceptance and application to assess efficiency, being the dominant method in the health sector and other sectors of the economy. For readers interested in the SFA technique we refer them to Kumbhakar and Lovell [15].

Rosko [16] summarizes one of the first revisions of DEA models applied to healthcare´s efficiency estimation, where most of the studies have an input orientation (cost reduction) and are related to hospital settings. Hollingsworth et al. [5] review 91 publications related to DEA in Health Systems. Puig-Junoy [7] evaluates 12 articles related to primary care in Spain, 11 of which used DEA, and one used SFA as the efficiency evaluation method. Tavares [17] makes a review of DEA applications from 1978 to 2001. The author summarizes 3,202 publications among articles, events, research, books, and dissertations. The most recent literature review of Hollingsworth [6] includes 188 publications using frontier methods. More recently, Emrouznejad et al. [18] published a review of DEA applications in several areas totaling more than 4,000 research articles published in scientific journals or book chapters. Worthington [19] mentions the increasing interest of incorporating factors which could help explaining inefficiency where the use of regression models prevails. Puig-Junoy and Dalmau [20] review Spaniard Publications, which includes 81 documents and 46 studies between 1980 and 2000. Of these, only 16 were related to primary healthcare.

Methods

Sample and data collection

The dataset used in our research was retrieved from national databases belonging to the Ministry of Health or from public databases available in Chilean Web pages.Footnote 2 All databases were accessed between July and October 2008. This dataset includes a total of 345 municipalities nationwide (year 2006). As mentioned previously, the main source of financial resources is provided by the central government. However, when a municipality has less than 3,500 enrolled inhabitants as users of its PHCs, the funding formula changes from per capita to a fixed budget. This is the case of 52 municipalities nationwide. Therefore, for the purpose of consistency, we are excluding those municipalities from this study. Additionally, we leave out those municipalities with missing or incomplete data. This reduces our population for the DEA study to a total of 259. For confidentiality reasons, we cannot disclose the name of the municipalities so a random number is used to identify them.

Descriptive statistics of input/output variables are summarized in Table 1. Since DEA models are sensitive to the type and amount of variables included, we test the robustness of the results by using models with different combinations of input variables while keeping the output fixed.

Table 1 Descriptive statistics—DEA analysis
Full size table

Variable selection

Measuring the results of the healthcare system activity is a complex task. Due to the heterogeneity of the population that uses this type of service, people’s health might be affected by different factors, such as biological, lifestyle, genetics, environmental, or by difficulties in measuring the added value of healthcare. It is not possible to observe health status of inhabitants prior to serve them, or to know the effects of not performing some type of medical treatment. Thus, long-term medical outcomes and patient-level data indicators are not available. The literature uses intermediate outcomes, such as the rate of healthcare visits, compliance with technical or healthcare standards, and activity rates. In this study, the variable selection for inputs and outputs in DEA is guided by expert opinion, economic theory, and literature research of the primary care sector. From the data available, we selected those variables thought to better represent the production system.

Inputs

Inputs should include the resources needed to provide care: staff, facilities, managerial expenses, and salaries. Municipalities, which run PHCs, also provide patient care through public health programs and ambulatory services. Most studies evaluating efficiency in hospitals, or bigger PHCs facilities, do include “number of beds” as indicator of capital. In Chile, patients requiring inpatient care in PHCs are sent to a proper facility (e.g., a hospital) since their infrastructure does not include beds. An appropriate input variable for infrastructure within the Chilean context could be the number of boxes, or cubicles for attending patients, a PHC has. However, such information is not available on the dataset. Besides, since most PHCs work with similar basic equipment, they could be considered as fairly homogeneous. Therefore, no variable related to infrastructure is included in this analysis.

Regarding medical staff, a new model for primary healthcare practice based on family and community care has been implemented since 2004 emphasizing preventive care. This model suggests having an “ideal” medical team in charge of providing primary healthcare to the population under their medical responsibility. It also suggests the number of enrolled inhabitants each team should take care of. An ideal team is composed of: a physician, a nurse, a paramedic, a midwife, and a social worker. Thus, we assume that the structure of the medical staff (ideal teams) of the municipalities is quite similar considering their medical qualifications. Besides the ideal team, a PHC has support teams composed of: a dentist, a psychologist, a kinesiologist, and a pharmacist, which interact with medical teams. Therefore, we are not interested in a particular specialty (e.g., physicians), which could be the case of other healthcare settings and studies worldwide. Within this context, the variable for medical staff could be expressed in terms of “working hours” or “total expenses” per ideal medical and support team. Unfortunately, there is no aggregated data available for such variables.

Therefore, we use as input variables: (i) the annual medical staff cost, which includes the total cost spent during the year on all medical staff that works for the municipality at a PHC (this variable has also been used as input in primary healthcare settings by other authors [21–27]); (ii) the annual general service cost, which includes all expenses related to the management and maintenance of all PHCs run by a municipality (this variable has been used in primary healthcare by Bryce et al. [28] and Rosenman et al. [26]); and (iii) the annual pharmacy cost, which is also part of the budget assigned to each municipality. Its expenditure is influenced by the demographics characteristics of each municipality. This variable has also been used by several authors in this setting [29–33].

Outputs

Most of the literature uses the annual medical visits rate as proxy of outcomes, being in some articles, the only “output” included in the DEA analysis [25, 29, 30, 34–40]. Hollingsworth and Smith [41] suggest that using rates would be appropriate if a Variable Returns to Scale (VRS) approach is used, such as in our case. However, we decide to avoid using rates while using DEA, even in a VRS context, as suggested by Emrouznejad and Amin [42]. Thus, we use as outputs: (i) the annual number of medical visits, and (ii) the annual number of medical check-up visits. The medical visits are related to sickness and the check-up visits are related to preventive care. We make this distinction because the healthcare care reforms states the relevance of preventive care.

Managerial and other control variables

Following Chen et al. [43] we use variables related to management, demographics, and organization structure in a Tobit Regression analysis. We collect data for 20 variables that might have some influence on (in) efficiency rates. In order to characterize the organization’s structure, the variable adm is used to represent the type of administration (Office or Corporation). For management practice characterization we use: gest (annual management rate of meetings and/or activities); prom (annual health rate of activities to encourage healthcare among the community); gcapac (the proportion of financial resources invested on incentives and training programs for human resources); inv (as a proxy for commitment, the proportion of primary healthcare financial resource spend on infrastructure); pcs (as a proxy for access/connectivity to information technology, measured by the proportion of PCs interconnected); contab (as proxy of the use, or not, of a health accounting information system); sfarm (as a proxy of the use, or not, of a pharmacy information system); rur, urb, otrocg (the proportion of rural, urban, and other PHCs each municipality runs respectively);Footnote 3 lab (describes the quantity of laboratories run by the municipality). The ability of the system to solve primary care pathologies is characterized by: inter (annual referrals rate to specialists); domi (annual home visits rate); dis (measured as the distance to the closest reference hospital); prev (the proportion of preventive exams performed annually); iapps (AIPH: composed of four main performance targets, as proxy of the quality of the service provided). Finally, regarding demography, we include the following: seis (the proportion of enrolled inhabitants under age six); secinco (the proportion of enrolled inhabitants over age 65); ipobre (poverty index) and denpob Footnote 4 (population density).

Model specification

Cluster analysis

Before estimating the level of efficiency, a cluster analysis is performed in order to homogenize the DMUs under study. The main reason for using this approach is Chile´s wide range of demographic and economic conditions. It is well known that environmental variables might affect the efficiency scores derived using DEA. In view of that, and as suggested by Jacobs et al. [12], pp.34 “The first approach to accommodating environmental influences is to cluster organizations into similar families, using techniques as cluster analysis”. Thus, in order to homogenize our sample on the basis of external variables within the Chilean primary healthcare context, we follow several authors when using a cluster analysis before using DEA [44–47]. Also other authors have used a cluster analysis while applying DEA with other objectives [48–54].

In this stage, four variables are used: (1) localización, which is related to the physical establishment of the municipality classified as rural or urban; (2) ipobreza, which represents the poverty index of the municipality; (3) ganual, which includes the annual per capita primary healthcare cost; and (4) poblac, which is the amount of population enrolled in the PHC that a municipality runs. All data belong to year 2006. The variables are standardized before being used, and outliers are checked to avoid forming one-entity clusters. Preliminary results show the existence of statistically significant differences for the variable localización (urban and rural) among the municipalities. Therefore, we split the sample into two main groups: rural an urban, and analyze them separately. Later, the second analysis includes the remaining variables: poblac, ipobreza, and ganual. We applied the Wilcoxon-Mann-Whitney U Test and conclude that the group average for the three variables is statistically different (p < 0.01). Therefore, we reject the hypothesis that the average among the subgroups formed by the cluster is the same, and conclude that the subgroups (three for the urban group and two for the rural group) defined by those three variables belong to different samples (Table 2).

Table 2 Descriptive statistics—cluster data
Full size table

DEA analysis

The DEA technique was initially proposed by Charnes, Cooper, and Rhodes [55] as an extension of the work of Farrell and has been known as the Constant Returns to Scale (CRS) model. A more flexible model was later developed by Banker et al. [56] to allow for Variable Returns to Scale (VRS).

Most studies in healthcare use an input-oriented model, and primary healthcare is not an exception [21, 24, 40, 46, 57–63]. However, several other authors have used an output-oriented model in primary healthcare [21–23, 26, 27, 29, 32, 33, 64–71]. As pointed out by Coelli et al. [72], pp.180 “In some industries, the firms may be given a fixed amount of resources and asked to produce as much output as possible. In this case, an output orientation would be more appropriate”.

Furthermore, Coelli and Perelman [73] demonstrate the existence of a strong degree of correlation between the input- and output-orientated results for many instances. When resources, e.g., budget, are a constraint, decisions regarding how to best use the inputs are noteworthy. This implies that technical efficiency is measured considering how to obtain the best possible outcome without altering the amount of inputs, suggesting an output orientation model. We believe that primary healthcare in Chile should aim at delivering the maximum number of services (in this study measured as check-up and medical visits) with the available resources. Our decision was based on the following: (i) municipalities are responsible for the administration and provision of care through their PHCs, but they have limited control over inputs, particularly the deployment of human resources; (ii) decisions regarding the main source of funding are taken at the upper levels (e.g., Ministry of Health), including minimum staff needed, staff’s salaries, investment on infrastructure, and so on; (iii) municipalities could use local income to finance additional staff, but the resources are so scarce, that in most cases this income is negligible compared to government funding. A severe budget constraint makes it difficult to increase inputs, even assuming that municipalities have control over them (which they do not have). Therefore, if inputs (e.g., medical staff) might be underutilized, it is not within their power to dispose of excess inputs; (iv) The medical staff has to induce demand for preventive health services as mandated by the latest health care reform, which includes family planning services, prenatal care, immunizations, women programs, children and elderly well-being, among others.

Both CRS and VRS models are analyzed in order to measure the impact of scale inefficiency. This is measured as the ratio of the CRS and VRS efficiency scores. To estimate the VRS efficiency using an output orientation, we use the following model:Footnote 5

$$ \begin{array}{*{20}{c}} {{{\hat{\theta }}_j}} \hfill & { = Max\;\theta } \hfill \\{s.t.} \hfill & {} \hfill \\\end{array} $$
(1.0)
$$ \begin{array}{*{20}{c}} {\sum\limits_k {{\lambda_k}{y_{rk}} \geqslant \theta {y_{rj}}} } \hfill & {\forall \quad r = 1, \ldots, R,} \hfill \\\end{array} $$
(1.1)
$$ \begin{array}{*{20}{c}} {\sum\limits_k {{\lambda_k}{x_{ik}} \leqslant {x_{rj}}} } \hfill & {\forall \quad i = 1, \ldots, I,} \hfill \\\end{array} $$
(1.2)
$$ \sum\limits_k {{\lambda_k} = 1} $$
(1.3)
$$ \theta, {\lambda_k} \geqslant 0. $$
(1.4)

where

θ:

efficiency

λ:

weight

x ik :

inputs

y rk :

outputs

j :

DMU under evaluation

This linear programming problem must be solved for each of the entities under study, comparing each unit with the “best” ones. Four model specifications are considered. By using different specifications we are able to perform a sensitivity analysis, to verify if the efficiency scores and ranks remain stable when using less or more variables. We built four specifications of the output oriented models for each group (urban and rural): model I includes the input variable gper;Footnote 6 model II uses gper and gfunc; model III uses the variables gper and gfarm; and model IV uses gper, gfunc, and gfarm as input variables. Both output variables cons Footnote 7 and cont remain fixed for all four models. Besides these four specifications, we use two approaches. First, we use a VRS model [Eq. 1.0] for all rural municipalities (177 DMUs) and one VRS model for all urban municipalities (82 DMUs). In the second approach, we use the results from the cluster analysis and run a VRS categorical model as suggested by Cooper et al. [14] when having subgroups within a group. In this case, we include the subgroups identified within the rural (2) and the urban (3) groups. Since the results from both approaches are very similar, and no statistical difference is found among the average efficiency scores estimated for each subgroup within the main two groups, rural and urban, we decided to keep the efficiency scores estimated for municipalities using the VRS formulation of Eq. 1.0 (non categorical).

The overall “inefficiency” score for each municipality j, is defined as INEF = (1/θ j )-1 as suggested by Chilingerian [74] in order to have a convenient normalization in Tobit studies and so, to assume a censoring point at zero.

This empirical analysis attempts to address some of the issues associated with modeling the efficiency of municipalities which manage and provide primary healthcare. Before applying the Tobit Model, two steps are taken. First, a bootstrap algorithm developed by Simar and Wilson [75] is applied to generate biased corrected efficiency scores and confidence intervals. This is also used as sensitivity analysis. Second, we identify which variables should be included in the Tobit Model by applying a Multivariate Analysis known as Biplot [76]. To the best of our knowledge the Biplot method has not been previously used with DEA.

Biplot analysis

Including irrelevant variables or over-specifying the model in multiple regression analysis is not harmless [77]. It can have undesirable effects on the variances of the OLS estimators. For our study we have identified 20 external variables that could influence the (in) efficiency of municipalities—the dependent variable of the analysis. Wooldridge [77] suggest including a minimum of 10 “observations” (municipalities) per independent variable included in the regression analysis. In the two groups under analysis, one of them has 82 observations (urban) and the other 177 (rural). This means that we could use up to 8 and 17 independent variables respectively for each group. The choice of whether to include a particular independent variable in the Tobit Model is assessed in this study by means of a Biplot analysis.

The Biplot was originally proposed by Gabriel [76] as a graphical tool of multivariate data. In the same way a scatter plot displays a rank-2 matrix, a Biplot can be used to effectively investigate a rank-3 or higher matrix. It is mostly revealing at displaying variances and correlations of the variables. As pointed out by Blaisus et al. [78] “...the multivariate distribution of a set of variables can be approximated in a low dimensional space (usually two dimensions) giving a useful visualization of the structure of the sample relative to the variables”. The variables are visualized as vectors pointing out to the best direction, either positive or negative, that each individual variable could move on. The GH-Biplot or CMP-Biplot (Column Metric Preserving) allows variables to have the same graphical quality. The length and magnitude of the angle formed by the vectors characterizing each variable are taken in terms of variability and covariability respectively [76]. The cosine of the angle formed by two vectors pointing out the same direction corresponds to the correlation among the variables. When two vectors form an almost right angle, the cosine of the angle is close to zero, meaning that the variables are independent. On the other hand, when the angle is close to zero, those two variables have a high positive correlation. If the angle is straight, the correlation is high and negative. For more detail on the Biplots, we suggest to the readers Gower and Hand [79]. Considering the characteristics of this method, we use a Biplot method to reduce the number of explanatory variables to be included in the Tobit Model.

Tobit analysis

The selected variables with the Biplot are used to better explain the inefficiency score defined by Eq. 1.0 using a Tobit Regression Model characterized by:

$$ INEF = \left\{ \begin{gathered} x\beta + u\quad if efficiency score > 0, \hfill \\0, otherwise \hfill \\\end{gathered} \right. $$

where

INEF :

estimated inefficiency using DEA

u :

error term

β:

parameter to be estimated.

Results

Efficiency scores are estimated using both a CRS and VRS output-oriented model. The results obtained using four specifications are shown in Table 3. The CRS model identifies more efficient municipalities for both, rural and urban groups. The efficiency scores for the urban group show a mean overall efficiency of 24%, 24%, 43%, and 44% for models I, II, III, and IV respectively. The proportion of efficient municipalities increases from 1% in model I up to 3% in model IV. Regarding the rural group, it achieves a mean overall efficiency of 9%, 29%, 16%, and 31% for models I, II, III, and IV respectively. The best performing municipalities identified are those who maximize technical efficiency.

Table 3 Technical efficiency
Full size table

On the other hand, the VRS model estimates for the urban group a mean overall efficiency of 61%, 64%, 68%, and 71% for models I, II, III, and IV respectively. The proportion of efficient municipalities increases from 8% in model I up to 18% in model IV. Regarding the rural group, it achieves a mean overall efficiency of 51%, 54%, 53%, and 56% for models I, II, III, and IV respectively. The Scale Efficiency (SE) is lower than one (SE <1) for all four models in both groups, showing a tendency for decreasing returns to scale. These suggest that the proportional increment of the output is less than the proportional increment of the inputs. These results are shown on Table 4.

Table 4 Scale efficiency
Full size table

The correlation among efficiency scores while using different specifications of the VRS model varies from 0.93 to 0.98 for the rural group, and from 0.88 to 0.98 for the urban group. Regarding the ranking of the municipalities, the correlation varies from 0.94 to 0.98 for the rural group and from 0.87 to 0.98 for the urban group.

Since using DEA is very sensitive to the input-output variables included in the model, it is better to approximate confidence intervals. By recognizing that the data obtained could be biased, and that the results estimated are subject to random noise, we are able to correct the efficiency scores obtained. As an example, the results for selected urban municipalities are shown in Table 5 (we do not include all municipalities for space reasons). The first column lists the municipalities; the second column has the estimated values for the efficiency using a VRS formulation; column three shows the corrected efficiency scores; columns four and five present the estimated bias and the variance, respectively; columns six and seven illustrate the values for the lower and the upper limit; finally, column eight presents the range. In order to decide if the corrected efficiency scores by the bootstrap should be used in the Tobit Model, we estimate for each municipality:

$$ \frac{{\left| {{{\widehat{{BIAS}}}_B}\left( {{{\hat{\theta }}_{VRS}}\left( {x,y} \right)} \right)} \right|}}{{\hat{\sigma }}} > \frac{1}{{\sqrt {3} }} $$

If the above is true, we use the corrected scores. We obtain corrected estimated values close to the original efficiency scores. Besides, most intervals have a moderate range. Given that for the majority of municipalities, the bias is small compared to the variance, we rather use the original DEA estimates of efficiency than the corrected ones for our next step.

Table 5 Selected data of corrected efficiency scores—urban group
Full size table

Before applying the Tobit Model, we perform a multivariate analysis applying a Biplot method [76]. The dependent variable used in the Biplot analysis is the (in) efficiency score. Twenty explanatory variables are graphically displayed in a scatter plot—not included for space reasons. From this analysis, we select those explanatory variables that: (i) form the lowest angle with the dependent variable and (ii) have the highest variance. The results of the Biplot analysis identified eight explanatory variables for the urban group and seven ones for the rural group. The variables identified for the urban group are: adm, gest, cont, sfarm, inter, secinco, seis, and ipobre. For the rural group, the Biplot analysis suggested using: inter, domi, secinco, seis, cgr, denpob, and ipobre.

Before using the Tobit Model, we test for multiple colinearity, computing a Variance Inflation Factor <10 for both groups, thus no multicolinearity is present. Testing for auto correlation, the Durbin-Watson Test provide a value close to 2 for both groups, suggesting no auto correlation. We also test for homoscedasticity by applying the Glesjer test. A summary of the results is presented in Tables 6 and 7. One variable needs to be transformed due to heteroscedasticy in the urban group—denpob—and two in the rural group—denpob and inter. We transform the variables by applying a log function as suggested by Wooldridge [77]. We observe that the constant value and the coefficients of the variables pcs, prev, seis, cgu, logdenpo, and loginter are statistically significant (p < 0.05) in the rural group. Only two coefficients inter and cgu are statistically significant (p < 0.05) for the urban group.

Table 6 Tobit results
Full size table
Table 7 Tobit results
Full size table

Discussion

In this study, we first present the measurement of the technical efficiency for 259 PHCs run by Chilean municipalities. A cluster analysis allows us to homogenize the municipalities of this study. The variables included in the DEA model are selected following the literature, and by consensus with the Ministry of Health Primary Care Department experts. The efficiency scores estimated for urban and rural groups vary depending upon the model used. The evidence from this study shows that urban municipalities are more efficient than rural ones, as suspected. Therefore, the inefficient municipalities identified need to enhance their technical efficiency in order to improve their performance. The inclusion of outputs related to the achievement of the healthcare reform offers promise as a benchmarking tool for municipalities seeking to maximize performance. With the purpose of testing the sensitivity of the VRS model to adding input variables, we test four different specifications, resulting in a high correlation among the efficiency scores and the ranking position for both groups. The correlation is even higher among specifications III and IV for both groups. Although some of the municipalities’ ranking could vary from one position to another among specifications this variation is irrelevant for most municipalities. The input variable gper seem to be the one that influences the most when estimating the efficiency scores and ranking municipalities. Therefore, we need to decide which specification should be used in the next steps.

For the urban group we notice that specification III—two inputs/two outputs—and specification IV—three inputs/two outputs—are the most similar ones regarding the average efficiency score, the standard deviation, and the number of efficient municipalities. This means that the inclusion of additional input variables did not seem to significantly affect the main results. Thus, we chose to use the efficiency scores estimated with the inputs included on specification III for the urban group. Regarding the rural group, despite the higher correlation achieved in specifications III and IV, specifications II and III look more alike in terms of the average efficiency score, the standard deviation, and the number of efficient units. Thus, since it seems that there are no major differences between the two, we chose specification II for the rural group.

Using a VRS categorical model with the subgroups identified by means of the cluster analysis did not make a significant difference among the average efficiency scores estimated with the regular VRS formulation of Eq. 1.0. Thus, it is not necessary to keep the municipalities divided into subgroups within the main groups for further analysis.

Not including inputs variables such as the personnel mix while estimating DEA efficiency scores could prevent us from drawing some meaningful conclusions on which specific staff “team” may require remedial actions. In our particular context, this data is not available, thus the study is based on the assumption that staff teams are similar across the units. However, efforts should be made to disaggregate the input variable annual medical cost. Since policy orientation suggests having “ideal” medical teams, we propose each municipality collects information regarding the annual medical cost per number of ideal teams, and per number of support teams.

A bootstrap analysis was performed to correct efficiency scores and to obtain a 95% confidence interval, which gives an estimated range of values likely to include the estimated values. Most intervals have a moderated range. All corrected efficiency scores fall inside the interval, which is not the case for some original values, in particular at the upper level of the efficient units. However, since the corrected efficiency scores are close enough to the original DEA estimates, we keep the original scores for further analysis.

The overall efficiency score reflects how operationally efficient a municipality is. However, in order to determine the sources of its inefficiency we need to identify specific factors affecting it. A traditional DEA model focused only on estimating efficiency scores is not enough for this purpose. In order to gain more insight and provide some recommendations for policy making, we use a multiple stage approach to search for factors that could improve municipalities’ management. If the sources of inefficiency are reduced, better results from the services provided should be expected. Therefore, identifying factors that influence the relative performance of the municipalities could help develop a plan, and a strategy for better provision of services. The results obtained from this study could be a first step in this direction since we are defining a frontier of best performing units that could assist policy makers. Using the inefficiency score estimated for each municipality, we use a Tobit Regression Model to examine the effect of those factors previously selected as relevant with a Biplot multivariate analysis. Mainly, those demographic variables were found to be statistically significant for both groups.

For the rural group we expected to find that both variables seis and secinco would have the same sign (positive) and that both of them would positively influence the inefficiency score. Seis was found to be statistically significant with p < 0.05 and secinco with p < 0.10. This suggest that a rural municipality with a higher number of registered population within this age group (lower than 6 and older than 65 years old) would most probably be classified as inefficient under similar conditions than the ones with a lower proportion of population this age. Thus, if seis increases by 1%, the inefficiency would increase by 12.3% (considering a marginal effect of 0.8658). Using the same criteria, if prev rises by 1%, the inefficiency score would increase by 13.3%. Regarding the cgu variable, a reduction of 1% in the proportion of urban PHCs, increases the efficiency levels by 3%. Similarly, a reduction of 1% in the proportion of personal computers (PCs) online, increases the efficiency level by 0.71%. The variable loginter has a negative coefficient but its value is so small that its influence on the inefficiency score is marginal, although it is statistically significant. The same is true for logdenpob, also on a marginal basis. For the urban group we have two variables which are statistically significant with p < 0.05: inter and cgu. Both of them increase the inefficiency score of 0.00002 and 0.722% respectively, which are marginal.

We expected to find a higher significant impact on the overall inefficiency score of the municipalities than the one observed. As mentioned previously, the main source of financial resources for most municipalities is the central government. Local governments do not have a constant source of income, since the amount collected through local taxes varies every year. Rural municipalities are worse off since they tend to have a lower income from local taxes than urban ones. Corporations are supposed to be more flexible, in terms of uses they give to financial resources, than Municipal Offices. We suspected that this variable could limit municipalities’ ability to make decisions thus reducing their opportunities for performance improvement. However, with the results obtained while applying the Tobit Model we are not able to verify the influence of this variable in the efficiency scores.

The implications of these results could mean that other management related variables need to be developed in consensus with healthcare authorities, specialists, and other interested parties, e.g., local government authorities. These new variables could be collected nationwide in form of indicators so they are included in future analysis. We also need to point out that the quality of the information used may be questionable since some datasets are made of self-reported data by the municipalities. Thus, we have to rely on their accuracy. Also, there are currently no quality survey results that will allow us to include patient perceptions as a variable for the analysis. Therefore, the quality of the healthcare provided is measured only by the self-reported information each municipality provides on a monthly basis in the form of AIPH and EHC indexes. Clearly more efforts should be devoted to identify relevant variables. This could be done as a consensus activity coordinated by the Ministry of Health on a nationwide basis that includes relevant stakeholders (e.g., physicians, nurses, managers, and so on). The inclusion of standardized quality measures as outcomes represents a challenge for future research.

Conclusions

This study makes a significant contribution to the measurement of technical efficiency using DEA for the Chilean context. Given the growing interest in measuring the efficiency of health management and the poor documentation related to primary care analysis, particularly in Chile, it is important to improve the quality of the datasets in order to use them to evaluate efficiency. This requires the definition of which variables are important to include in this type of study, and the development of a method for collecting this data, making sure that this new data is unbiased as possible.

The present study applies a nonparametric technique for evaluating the performance of PHCs run by municipalities through a multiple stage analysis, which includes DEA. The objective is to identify those units that operate more efficiently, and to recognize the best practices that could eventually be implemented by those classified as less efficient. As a result of the applications of DEA, we classify and rank the municipalities according to their technical efficiency score. Four specifications using a VRS formulation are used. The results show that urban municipalities are, in average, more efficient (61–71%) than rural ones (51–56%). Furthermore, while 8–18% of urban municipalities are classified as being “technically efficient”, only 6–15% of rural ones reach the maximum score value. We also determine the degree of improvement required by each of the entities under study to become “efficient”.

Once the degree of (in) efficiency is estimated, it becomes necessary to investigate why this happens, and to identify whether those scores are due exclusively to the management practices of the PHCs or to some external factors. We use a Tobit Regression Model to investigate the effect of introducing explanatory variables of control. Six variables are found to be statistically significant for rural municipalities, while two variables are found significant for urban municipalities. Although most municipalities do achieve an acceptable level of performance for the targets established, the use of the resources used to produce them could be improved. Municipalities with lower scores should try to emulate best practices used by their peers in order to become more efficient. If the Ministry of Health could motivate each municipality director to improve their practices on the areas detected as insufficient, e.g., through some kind of incentive, substantial improvements could be achieved in the variables that are the focus of this research.

Observed patterns of resource usage for different groups of municipalities could offer opportunities to develop health policies for better budget allocation, especially for those located in rural areas have lower local income. Another implication for policy making is that the findings indicate that a managerial focus on improving technical efficiency needs to rely on other variables not included in this study due to lack of information. Also, the quality of perceived care needs to be measured as a performance indicator in order to properly evaluate the quality of the service offered, and to be included as an output variable in future DEA studies. Considerations for future work will require us to refine the results obtained in this study.

Most DEA studies in healthcare studies have used ratios as inputs and/or outputs instead of absolute values. Under these circumstances, Hollingsworth and Smith [52] suggest using the VRS formulation of Banker, Charnes and Cooper [56]. However, the most recent paper of Emrouznejad and Amin [53] discusses the convexity problem this may cause and propose alternative DEA models when output and/or input ratios are the only data available. Among other options available, we are considering to apply DEA plus ordinary least squares (OLS) regression analysis and/or DEA plus maximum likelihood estimation (MLE) as recently suggested by Banker y Natarajan [80]. This approach might provide insightful results.

Limitations of the study

This study has some limitations. First, missing data in the databases and that some municipalities do not have a per capita funding precluded their use in the DEA model. Thus, the study had to exclude 23% of the municipalities. Consequently, the conclusions from this study cannot be used for those units. Second, only aggregated personnel (medical staff) cost data was used since disaggregated information in the form of medical teams was not available. Thus, the definition of inputs and the degree of aggregation should be reevaluated in future studies to include, for example, “number of ideal medical teams”. The inclusion of different inputs could alter the conclusions from this study and their implications. Third, missing data on important management and control variables precluded their use in the Tobit analysis. Fourth, since the databases used are made of self-reported data from municipalities, this raises some questions on the quality and veracity of the data. Finally, the lack of information regarding capital variables, such as number of boxes and the use of medical laboratories in each PHC precluded using them in this study. If these issues can be resolved, a more parsimonious characterization of the Chilean primary healthcare system could be achieved to represent its productive activities.