Introduction

Quality never ceases to be a key issue in the context of higher education. Substantial reforms have taken place in recent years in order to make “higher education not just bigger but also better” (Giannakou 2006, p. 12). The move towards higher standards has also been pushed because in many countries a substantial part of the funding received by universities is public. As an analysis carried out by the Organization for Economic Co-operation and Development (OECD) remarked, “formulas to allocate public funds to higher education institutions are now related to performance indicators such as graduation or completion rates.” Moreover, “research funding has also increasingly been allocated to specific projects through competitive processes rather than block grants” and has been linked “to assessments of research quality” (OECD 2008, p. 49). In other words, universities are financed according to their virtuosity level in order to achieve higher research performances and to promote academic excellence. Following this direction, the Italian higher education system was reformed.Footnote 1 Both quantitative and qualitative indicators were developed to accurately evaluate the management of public universities, their productivity in research and teaching, and the overall success of their administration. Borrowing an expression made in an OECD Report (2008, p. 50), which also describes the situation the Italian universities started to deal with, “higher education institutions have become increasingly accountable for their use of public funds and are required to demonstrate value for money.” Clearly efficiency cannot be the only goal of higher education. Equity considerations play an important role as well. The extent to which the expansion of post-compulsory education has enhanced equality of access and the distribution of costs and benefits of public spending on post-compulsory education are important issues to take into account. Indeed, “over the past 30 years participation rates in post-compulsory education have increased rapidly. This is reflected in the higher attainment rates of people. A question arises over whether this overall expansion in educational opportunity has been equitably shared” (Blöndal et al. 2002, p. 38). In fact, some efficiency loss may be traded off against equity gain, depending on political preferences. Indeed, given the social outcomes of higher education and its contribution to social mobility and equity “efforts to improve student completion and institutional productivity must be carefully undertaken so that they do not further inhibit access and success for sub-populations already underrepresented in higher education” (Tremblay et al. 2012, p. 31).

A growing number of researchers analysed the efficiency of higher education institutions (HEIs) through non-parametric methods.Footnote 2 Using these approaches, the vast majority of the literature has traditionally focused on two types of efficiency evaluation. One type uses data at the institutional level, comparing the efficiency of different universities (see Agasisti and Dal Bianco 2009; Bonaccorsi et al. 2006; Agasisti and Johnes 2010 for an analysis of Italian universities). The other type focuses on measuring efficiency at the department level, both across different HEIs (Madden et al. 1997; Thursby 2000; Johnes and Johnes 1993, 1995; Leitner et al. 2007) and, more significantly for the context of our paper, within the same university (Halkos et al. 2012; Buzzigoli et al. 2010; Tauer et al. 2007; Kao and Hung 2008; Tyagi et al. 2009; Koksal and Nalcaci 2006).Footnote 3

In this paper data envelopment analysis (DEA) is applied to evaluate departments and faculties at the University of Salerno (a big public university in the South of Italy), and to assess, respectively, their research and teaching activities. Clarification on why teaching and research performances are separately assessed is needed. Indeed, whether they are joint or separate products is not new in the literature (Chizmar and Zak 1983, 1984). In general, that the university typically engages in both teaching and research activities suggests that there might be economies from joint production (i.e., economies of scope). However, in Italy, until a reform was made,Footnote 4 faculties were taking care of teaching activities and departments were taking care of research. In other words, the university undertakes both activities in the same institution, but reaches their institutional teaching and research goals through different structures, namely faculties and departments (see the Research design section for more details on their characteristics and structures).

Beyond the many DEAs of universities already carried out, our contribution brings new evidence to the importance of evaluating the efficiency of different units operating within a tertiary education institution. Such evaluation processes could be helpful for university managers to shed some light on the effectiveness of various entities within the university as well as to better allocate both human and financial resources. In addition to the standard inputs and outputs already used, we rely on three alternative indices as a measure of research quality and on a student questionnaire-based evaluation as a measure of teaching quality. Specifically, in order to increase the homogeneity of the decision making unitsFootnote 5 (DMUs), departments and faculties have been divided according to their characteristics into two large groups, namely the Science and Technology (ST) sector and the Humanity and Social Science (HSS) sector. Moreover, we contribute to the literature using a bootstrap technique in order to provide more accurate estimates and confidence intervals. Finally, in order to measure changes in productivity across time, a Malmquist index approach has been used. The study shows that the ST sector is more efficient in terms of quality of research than the HSS sector, achieving higher efficiency in teaching activities. Several output measures are used to highlight the sensitivity of efficiency analysis showing that the results strongly depend on the output specification. Bootstrapped bias-corrected estimates are obtained, pointing out the sensibility of efficiency scores relative to sampling variations of the estimated frontier. Finally, the Malmquist index is calculated, revealing that the change in productivity is due to a mixed pattern of technological change (i.e.,an outwardly shifting production frontier) and changes in technical efficiency.

Methodology

In the literature, the main methods used to calculate efficiency are non-parametric and parametric. The non-parametric methods, such as DEA and FDH (Free Disposable Hull), proposed by Charnes et al. (1978) and due to the original contribution of Farrel (1957), are based on deterministic frontier models (also Cazals et al. 2002). The DEA model, extended by Banker et al. (1984), is especially adequate to evaluate the efficiency of non-profit entities that operate outside the market, since for them performance indicators, such as income and profitability, do not work satisfactorily (for more theoretical details on DEA see Coelli et al. 1998; Cooper et al. 2004; Thanassoulis 2001). Instead, parametric approaches, such as the Stochastic Frontier Approach (SFA), Distribution-Free Approach (DFA) and Thick Frontier Approach (TFA) are based on stochastic frontier models (Aigner et al. 1977). This study focuses on a non-parametric method such as DEA because it does not require building a theoretical production frontier, but does require the imposition of certain a priori hypotheses about the technology (free-disposability, convexity, constant or variable returns to scale).Footnote 6 Moreover, the multiple input–output nature of production in HEI makes DEA, rather than SFA, a more reliable technique in this context. Indeed, the DEA approach allows us to overcome some well-known problems concerning computation of technical efficiency in a parametric multi input–output set up (Greene 1980). A disadvantage of this technique, however, is that it is very sensitive to the presence of outliers. The likelihood of confusion regarding deviations from the efficient frontier generated by extreme values versus deviations caused by inefficiencies in the production process is very high.Footnote 7

In this paper, we focus on technical efficiencyFootnote 8 using an output-oriented DEA method, with variable returns to scale (VRS). The DEA-VRS is probably the most reliable in our case as suggested by Agasisti (2011, p. 205) who argued that the assumption of constant returns to scale (CRS) is restrictive because it is reasonable “that the dimension (number of students, amount of resources, etc.) plays a major role in affecting the efficiency” especially if we consider, the DMUs achieving pre-determined outputs, given certain inputs. Efficiency estimates have been obtained through an output oriented model, following Agasisti and Dal Bianco (2009, p. 487) who claimed that “as Italian universities are increasingly concerned with reducing the length of studies, and improving the number of graduates, in order to compete for public resources, the output-oriented model appears the most suitable to analyse higher education teaching efficiency.” Moreover, output-oriented models seem to be particularly appropriate in the context of tertiary education because the resources used can be considered fixed and universities cannot influence, at least in the short run, available human, financial and physical capital (Bonaccorsi et al. 2006). Consequently, we present an output-oriented version of the model.

Research Design

Selection of Decision Making Units (DMUs)

This study focuses on measuring both research and teaching performances of departments and faculties at the University of Salerno. With respect to research performances, one type of DMU has been considered, the departments. According to university regulations, departments have the role of promoting and developing research. They also gather professors and researchers according to the scientific research activities they are in charge of in the higher education institutions.Footnote 9 It is debated in the literature whether departments within the same university are treated as homogenous. We base ours on the Tyagi et al. (2009) argument according to which departments inside a university may be considered as homogenous because they conduct similar activities and are willing to achieve similar goals. In respect to teaching performance, one type of DMU has been taken into account, namely the faculties. They group different departments according to similarities. They are organized into different subject areas, each offering a number of degree courses, aiming to coordinate teaching activities.Footnote 10 We again treat faculties within the same university as homogenous because they conduct similar activities, using both academic and non-academic staff for teaching purposes (Tyagi et al. 2009).

Moreover, in order to consider more carefully the homogeneity issue and to explore whether the subject mix might adequately affect and explain the efficiency differentials between DMUs, we divide both departments and faculties, according to their characteristics, into two large groups, namely the ST sector and the HSS sector.Footnote 11

Inputs

The first input (used for measuring the efficiency of both departments and faculties) is what we call the equivalent personnel (EP), namely the total number of academic staff and non-academic staff Footnote 12 human resources available to departments and faculties, respectively, for research and teaching activities. The academic staff has been categorized as professors, associate professors, researchers and assistant professors. We assign weights to each category according to their salary and the number of institutional, educational, and research duties the academic staff has to deal with (Madden et al. 1997) and assuming that a professor is expected to produce more research and teaching work than an associate professor and so on (Carrington et al. 2005). Similarly to Halkos et al. (2012)Footnote 13 we use the following aggregate measure of human capital input:Footnote 14

$$ \begin{array}{l}EP=1* professors+0.8* associate\ professors+0.6* researchers+\hfill \\ {}+0.4* assistant\ professors+0.2*non- academic\ staff\hfill \end{array} $$
(1)

The second input (measuring the efficiency of departments) is the total amount of financial resources the department spends on research activities (ER). The third input (measuring the efficiency of faculties) is the total amount of financial resources the faculty allocates for teaching activities (ET). The fourth input (measuring the efficiency of faculties) is the total number of students enrolled (STU)Footnote 15 as a measure of faculty teaching burden.

Outputs

The first output (measuring the efficiency of departments) is the number of publications (NP). We follow Harris (1988) and Halkos et al. (2012), including all articles in refereed journals,Footnote 16 in order to deal with an important issue such as how many and which journals to include. Number of publications is categorized as articles published in international journals, national journals, international books and national books. We assign weightsFootnote 17 to each category according to the importance of the publication. We apply a procedure similar to the one proposed by Tyagi et al. (2009) using the following aggregate index of publications:Footnote 18

$$ \begin{array}{l}NP=1* articles\ in\ in ternational\ journals+0.75* articles\ in\ national\ journals+\hfill \\ {}+0.5* articles\ in\ in ternational\ books+0.25* articles\ in\ national\ books\hfill \end{array} $$
(2)

The second output (measuring the efficiency of departments) is the total external research funding obtained by the university (FR).Footnote 19 It is a bit controversial though whether it should be used as an input or as an output.Footnote 20 In agreement with the main part of the literature (among others Bonaccorsi et al. 2006; Agasisti et al. 2011; Tomkins and Green 1988), and following Buzzigoli et al. (2010),Footnote 21 we consider the amount of money received for financing research as a good proxy for the value of the research and therefore as an output.

The third output (measuring the efficiency of departments) is an alternative way of measuring scientific production. There are three different indices (defined in publications of “Nucleo di Valutazione” of the University of Salerno) which can be used, namely research productivity index (RPI), capacity of attracting resources index (CARI) and research productivity per cost of the academic staff indexFootnote 22 (RPCASI).

The fourth output (measuring the efficiency of faculties) is the number of graduates weighted by their degree classification (NG), which captures the quantity and the quality of teaching.Footnote 23 According to Catalano et al. (1993) the task assigned to universities is to produce graduates with the utilization and the combination of different resources. Madden et al. (1997) used the number of graduates under the hypothesis that the higher the number of graduates the higher the quality of teaching.Footnote 24

The fifth output (measuring the efficiency of faculties) is represented by two different indices which have been calculated evaluating a questionnaire given respectively to regular students, the student satisfaction index (SSI), and to those students who are specifically preparing for the degree, the undergraduate satisfaction index (USI). The aim of the surveys was to collect student’s opinions on the organization of faculties, facilities used such as libraries, classrooms, computer rooms, and on classes attended (such as appeal of the topic studied and teacher quality).Footnote 25

Specification of the Models

The use of different proxies for both research and teaching performances allows us to explore whether the results obtained are sensitive to the specification of the outputs used. For this reason, we implement different models (Appendix Table 1).

When we analyse the departments’ performance in the benchmark model (Model 1a, Appendix Table 1), EP and the expenses for research (ER)Footnote 26 are used as inputs, while NP is used as output. Keeping constant the input side, we explore first whether FR and then the three research indices might represent an alternative way of measuring department performance in terms of research activities (Models 2a, 3a, 4a, 5a and 6a, Appendix Table 1). On the other hand, in order to analyse faculty performance, in the benchmark model (Model 1b, Appendix Table 1) the EP, ETFootnote 27 and STU are used as inputs, while NG is used as output. We then use the quality teaching indices as outputs to informally test for the reliability of the benchmark teaching output (Models 2b and 3b, Appendix Table 1).

The input and output choice, from both the quantity (number of inputs and outputs) and quality (type of inputs and outputs) has many implications in the analysis. Unfortunately, as pointed out by Johnes and Johnes (1995, p. 307), “while statistical inference can be used as a means of judging whether or not a variable should be included as a regressor in a statistical analysis, DEA is not a statistical technique, and no such guide is available.” However, the number of DMUs relative to the number of input and output performance measures must be large enough to obtain meaningful efficiency estimates. Dyson et al. (2001) claimed that the number of DMUs must be at least 2*m*s where m is the number of inputs and s the number of outputs. Following Halkos et al. (2012), we use this approach. In our study we have, at most, 2*2*3 = 12 when departments are taken into account and 2*3*1 = 6 when faculties are analyzed, indicating an appropriate number of inputs/outputs used.Footnote 28

Data

The dataset used in this paper was constructed using annual publications of “Nucleo di Valutazione” of the University of Salerno and data which are publicly available on the National Committee for the Evaluation of the University Sector (CNVSU) website (http://www.cnvsu.it). Moreover, for evaluation of teaching activity we use two CNVSU questionnaires which anonymously collect information and opinions about teaching activity from regular students and from students who are preparing for degrees. For more details about the scheme of the questionnaires and the way indices have been constructed, see the CNVSU website. The data refer to the period from 2005 to 2009. All financial data have been deflated to 2007 values using the Retail Price Index (RPI) data from the National Institute of Statistics (http://www.istat.it). For more details, see descriptive statistics in Tables 2 and 3 in the Appendix.Footnote 29 In estimating our DEA models, Malmquist index and for bootstrapping, we rely on two packages based on the freeware R (FEAR 1.13, Benchmarking 0.18).

The Empirical Evidence

DEA Efficiency Scores

The DEA method has been applied to estimate technical efficiency of departments (Models 1a, 2a, 3a, 4a, 5a and 6a) and faculties (Models 1b, 2b and 3b) within the University of Salerno over the period 2005–2009, taking into account that technology might change over time (i.e., estimates are carried out year by year). The efficiency estimates are presented in the Appendix (Tables 4 and 5) for both departments and faculties belonging to the HSS sector and ST sector and across time.Footnote 30

The analysis of departments aims to capture the quality of research activity. Starting from the baseline model (Model 1a, Appendix Table 4), where NP is used as output, it is clearly evident that the HSS sector is less efficient than the ST sector. We then use funds obtained for research and scientific productivity indices in order to capture the quality of output in an alternative way (see Models 2a, 3a, 4a, 5a and 6a, Appendix Table 4). The empirical evidence of Model 1a is confirmed by the results of Model 2a (where FR is used as output), although we note an increase in efficiency estimates both for the ST and HSS sectors. Interestingly, the empirical evidence of Models 1a and 2a is only partly confirmed by the results of Models 3a, 4a, 5a (where RPI, CARI and RPCASI are used, respectively, as outputs), where the differentials between the ST and HSS sectors are reduced, as also showed by the results of Model 6a, where the three scientific productivity indices are used, at the same time, as outputs. This is particularly evident in Model 3a where the weighted sum of the publications (RPI) is measured taking into account that departments belonging either to the ST or HSS sector (i.e., departments belonging to the ST sector have a higher number of publication in international journals).Footnote 31 To be more precise, results obtained in Model 5a look as good as those in Model 1a and 2a, giving credit to the research productivity per cost of the academic staff index (RPCASI) used as output. Given the political nature of university funding, this result suggests that the RPCASI output would seem to be as important as any of the other main output measures.

The analysis of faculties aims to capture the quality of teaching. Starting from the baseline model (Model 1b, Appendix Table 5), where the output is represented by NG, we find that the HSS sector is more efficient than the ST sector.Footnote 32 This evidence is partially confirmed using the satisfaction indices as output. Nevertheless, considering the SSI (Model 2b), we find that the HSS sector efficiency estimates (Appendix Table 5) are drastically lowered, even if the HSS sector maintains higher scores than the ST sector. Using the USI as output (Model 3b), instead, reduces the differentials between the ST and HSS sector, compared to the baseline model (Model 1b).Footnote 33

Summing up, our evidence suggests that the efficiency estimates strongly depend on the output specification and the use of quality proxies reduces the performance and its differentials for both research and teaching activities. Moreover, it seems that the scientific sector has an impact in explaining efficiency differentials, confirming that “scientific areas tend to differ regarding their teaching and especially their research productivity” (Sarrico 2009, p. 290).

Efficiency Changes Over 2005–2009 Period: A Malmquist Analysis

We perform a productivity analysis using the Malmquist index (see Caves et al. 1982 for more technical details) to disentangle the changes in efficiency due both to pure efficiency improvements (or worsening) and technological improvements (or worsening),Footnote 34 focusing on Models 1a, 2a, 6a and Models 1b, 2b and 3b described in Appendix Table 1, over the 2005–2009 period.Footnote 35 We follow a generalized approach suggested by Fare et al. (1994).

Several issues could be addressed in the computation of the Malmquist indices of productivity growth over the period analyzed. The first one is the measurement of productivity change over the 2005/2006-2008/2009 period (TFPC). Indeed, if TFPC is >1 then productivity gains occur, but if TFPC <1, productivity losses occur. The second one is to decompose changes in productivity into technical efficiency change (E) and technological change (TC) in order to analyse whether the productivity change is due, respectively, to changes in technical efficiency or to an outward shift in the production frontier.Footnote 36 The third issue that could also be addressed is that (E) could be further decomposed to identify the main source of improvement, in pure technical efficiency change (PEFC) or changes in scale efficiency (SC).Footnote 37

Starting from the analysis of departments, and considering the TFPC the empirical evidence shows that for both ST and HS sectors overall there is an improvement in productivity (TFPC > 1, Appendix Table 6). Given that the productivity change can be decomposed in E and TC we can further underline whether most of the increase was due to E (movement towards the frontier) or to TC (outward shift in the efficiency frontier). Take for instance model 1a, Appendix Table 6, and taking into account both sectors (ALL), there is a decrease in TFPC between 2005–2006 and 2008–2009 of around 50 % (from 1.7954 to 1.2850).Footnote 38 It seems that in this specific case the fall in productivity was brought about predominately by a decrease in E rather an outward shift in the TC. Indeed, the empirical evidence shows that TC worsened by around 4 % (from 1.0975 to 1.0580) while E decreased by almost 40 % (from 1.6334 to 1.2146). This result is mainly driven by the HHS sector. Indeed, when we analyse separately the two sectors we still find a rise in productivity by 30 % (from 1.0653 to 1.3694) for the ST sector while there is a slight fall in productivity by 0.7 % (from 1.9857 to 1.9783), for the HHS sector. For the ST sector the increase in productivity seems to be due more to improvements in efficiency more than from an expansion in the frontier-related inputs to outputs (E increases by around 35 % from 0.9320 to 1.2860 and TC decreases by 8 % from 1.1416 to 1.0649). For the HSS sector the sustained improvement in productivity is a consequence of technology change rather than technical efficiency change (E decreases by around 15 % from 1.7664 to 1.6136 and TC increases by 13 % from 1.1242 to 1.2561). Decomposing E in pure technical (PEFC) and scale (SC) efficiencies, overall and for both sectors (Appendix Table 6), the results indicate that PECF is higher than SC suggesting that the major source of efficiency change is due to an improvement in pure technical efficiency rather than an improvement in scale efficiency.

Regarding the faculties (Appendix Table 7), results still show an improvement in technology (TFPC > 1) for Models 2b and 3b. Considering the ST sector, except for Model 1b, where the change in technology seems to drive the improvement in productivity,Footnote 39 in Models 2b and 3b the sustained improvement in productivity seems to be more the result of changes in technical efficiency.Footnote 40 This mixed evidence is also observed in the HSS sector. Indeed, it is evident that, considering Model 2b, the main source of productivity is represented by the improvement in technology,Footnote 41 while in Models 1b and 3b the change in technical efficiency explains the rise in productivity.Footnote 42 When we decompose E, the empirical evidence shows that both PEFC and SC efficiencies contribute equally to the technical efficiency change

Overall, the estimates show a mixed pattern of positive (negative) technology change versus negative (positive) efficiency change. While technology change was found to be the main source of technological improvement (Flegg et al. 2004; Worthington and Lee, 2007; Johnes 2008), in some cases the improvement in efficiency largely drives the positive productivity change rather than an outward shifting of the frontier. It has to be mentioned, however, that the empirical estimation of this decomposition of the Malmquist productivity change index (Johnes 2008) should be treated with caution, since it mixes VRS and CRS efficiencies in the estimation of its component (Ray and Desli 1997). A possible interpretation of these results is that the university, in order to improve its research (i.e., departments) and teaching (faculties) performances, relies on technological changes but without doing so at the price of technical efficiency. Among the most important sources of change in production activity involving universities are information technology and e-learning. As Johnes (2008) underlined, the increased use and application of technology might have positive effects on many aspects of university activities. For instance, information is more accessible to users (i.e., students), causing changes in teaching, and increasing administrator efficiency. Thus, these policies do not have to emphasize financial and individual outcomes more than non-financial and social outcomes (i.e., equality of access is not always achieved). Given these social costs and consequences, efforts to improve student completion and institutional productivity must be carefully undertaken so that they do not further inhibit access and success for sub-populations already underrepresented in higher education. The evidence of changes in technical efficiency indicates that most departments and faculties operate near the best-practice frontier suggesting the use of management, teaching and research practices with the aim of improving outputs.

Efficiency Bias Correction and Confidence Intervals Construction

The bootstrapping technique, introduced by Efron (1979) and Efron and Tibshirani (1993), was attractive in analysing the sensitivity of efficiency and productivity measures to sampling variation. Many researchers advocate for this technique (Atkinson and Wilson 1995; Ferrier and Hirschberg 1997; Simar and Wilson 1998). Basically, bootstrapping is particularly useful when little or nothing is known about the underlying data generating process (DGP) for a sample of observations. In the higher education sector this method, which ascertains estimates precision, is still very rare. Following Simar and Wilson (1998, 1999), bootstrapping has been used to calculate confidence intervals for efficiency scores.Footnote 43 Our evidence suggests the importance of using a bootstrapping DEA approach (confirming what has been found by Halkos et al. 2012). Indeed, the main results are confirmed, but a strong bias is found in our estimation (Appendix Tables 4 and 5), meaning that the efficiency scores calculated without bootstrapping might be over-estimated. Specifically, considering the analysis of departments and when bias-corrected efficiency estimates are taken into account, it is evident that the HSS sector is less efficient than the ST sector when the scientific productivity indices are used as alternative outputs (i.e., the reduction in the differentials underlined before is less accentuated).

Conclusion

This paper analyses the performances (i.e., efficiency) of departments and faculties at the University of Salerno over the 2005–2009 period. We apply a DEA approach in order to estimate efficiency scores using an output-oriented model when variable returns to scale are assumed. First, we take into account that the tertiary education institution analyzed carries out two of its main activities, such as teaching and research, through different structures, namely faculties and departments. Second, differently from the main literature and in order to achieve a higher degree of homogeneity of DMUs, departments and faculties have been divided, according to their characteristics, into the ST sector and the HSS sector. Third, we propose the use of different outputs such as three alternative indices as a measure of research quality as well as two student questionnaire-based evaluation indices as teaching performance measure. The empirical findings suggest that the ST sector is more efficient in terms of quality of research than the HSS sector that instead achieves higher efficiency in teaching activities. This suggests the importance of taking into account differences in subject mix when measuring efficiency in higher education, in order to avoid distorted estimates (Sarrico et al. 2009). Our evidence also suggests that the efficiency estimates strongly depend on output specification because the use of quality proxies reduces performance and its differentials for both research and teaching activities. When research activities are taken into account, the results suggest that the number of publications and the funds for research seem to better capture scientific production within higher education institutions. On the other hand, when teaching activities are considered, the use of different output measures did narrow the distance between the SC and HSS sectors. More specifically, it is interesting to note that the distance between sectors did particularly narrow when the student satisfaction index was used as output. Indeed, student opinion is becoming an important qualitative indicator for higher educational institutes and shows the key factors for meeting users’ needs. Moreover, higher education is a customer-oriented to the students. This evidence could be very useful for university administrators and call into question whether universities should incorporate some measure of satisfaction in their recruitment initiatives. Fourth, we apply a bootstrapping method in order to investigate the efficiency score sensibility relative to sampling variations of the estimated frontier obtaining bias-corrected efficiency estimates, in contrast to a straightforward application of DEA (Halkos et al. 2012). Finally, the Malmquist index has also been calculated in order to disentangle changes in efficiency due both to pure efficiency improvements (or worsening) and technological improvements (or worsening) finding that the change in productivity is due to a mixed pattern of technological change (i.e.,an outwardly shifting production frontier) and changes in technical efficiency (for a comparison on the Malmquist analysis results, see Flegg et al. 2004; Worthington and Lee, 2007; Johnes 2008). To sum up, through this analysis we contribute to the literature an attempt to measure the performances of HEIs. Universities’ regulators might take advantage of these studies and make, through appropriate policy decisions (i.e., focusing on the distribution of available additional resources either among more efficient units, as a reward, or more inefficient units, helping them to improve their efficiency), the tertiary education system more effective. In the immediate future, our task will be to deepen the role of the variables used as quality proxies in the production process to give regulators additional information to make more accurate policy decisions.