Abstract
The introduction of competitive funding mechanisms in higher education is found to generally increase research productivity. However, the diversity within higher education systems may lead universities to behave in substantially different ways in response to the adoption of competitive funding criteria. In particular, we argue that the legitimacy of universities, defined as their level of recognition based on the adherence to socially accepted norms and expectations, is crucial in shaping their reaction. This paper investigates the change in research productivity experienced by Italian universities following the introduction of the first Performance-based Research Funding System (PRFS) in 2003, focusing on the moderating effect of university legitimacy. Using a sample of 75 universities observed during the period 1999–2011, we find that the introduction of PRFS leads to an increase in research productivity, and this increase is significantly more pronounced among more legitimate universities.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
European higher education systems have faced important changes over the past decade, as a result of the convergence of governance mechanisms towards New Public Management practices (see Leisyte and Kizniene 2006; De Boer et al. 2007). The introduction of national research evaluations, implemented in the form of Performance-based Research Funding Systems (PRFS), represents a major change in a weakly competitive environment such as the one in which universities were typically operating. Considerable attention has been devoted by academics in order to assess whether such evaluation mechanisms are effective in improving research productivity and efficiency at the university level (e.g., Orr et al. 2007; Strehl et al. 2007), but consensus is still far from being reached (Geuna and Martin 2003).
This paper proposes the idea that universities may react in sensibly different ways to the implementation of a competitive funding mechanism, based on their level of legitimacy. Since PRFSs trigger a funding competition based on research performance, a university’s level of legitimacy, i.e. its degree of social acceptance deriving from the adherence to normative rules and expectations (Suchman 1995; Deephouse and Carter 2005), may determine the extent to which its incentives need to be realigned once a PRFS is introduced. In particular, highly legitimate universities are known to put considerable effort in developing and preserving their inter-organizational linkages, providing their academics with the opportunity to gather valuable research ideas from the interaction with third parties. Further, more legitimate universities, driven by the stronger need to protect their acquired legitimacy in order to keep benefiting from interaction with external parties, face larger expected costs in the event of a negative evaluation assessment associated with the PRFS. We therefore expect that more legitimate universities increase their productivity to a greater extent as a response to the PRFS. The contribution of this paper lies in showing how legitimacy explains the different reactions of universities to the implementation of PRFS.
The core units of our analysis are universities. While existing studies mainly focus either on a country (Butler 2003; Geuna and Martin 2003; Auranen and Nieminen 2010) or an individual (Moed 2008) level, universities are the ultimate entities to which research performance and ranking assessments, such as PRFSs, are addressed (Hicks 2012). The aim of introducing a PRFS is indeed to move funding towards better performing institutions in terms of scientific production, in order to pursue a more efficient allocation of resources (Herbst 2007). This perspective also allows to overcome some limitations. On one hand, the individual level does not account for the mobility of researchers, which has sharply increased over the last two decades (Hicks 2009). On the other hand, the national level fails to consider institutional diversity within higher education systems. Since universities are required to meet a wide range of needs associated with the development of the knowledge society, they are characterized by highly peculiar roles, competences, and specializations (Reichert 2009).
We measure the research productivity of a sample of 75 Italian universities during the period 1999–2011 by observing the number of scientific publications per faculty member. Our findings can be summarized as follows. First, Italian universities react to the introduction of the PRFS by increasing their overall productivity. Second, starting from the evidence that more legitimate universities are more productive in terms of publications per faculty member, we show that the increase in productivity triggered by the introduction of PRFS sensibly varies according to a university’s level of legitimacy. In particular, this is more pronounced among more legitimate universities. We argue that these universities have a higher legitimacy at stake to preserve, which might be endangered by a possible negative evaluation resulting from the PRFS, thereby pushing them to react more than their less legitimate peers. Furthermore, more legitimate universities need to put relatively lower effort in order to increase their research productivity, given their superior learning and funding opportunities arising from inter-organizational linkages.
The contribution of this paper to the existing literature is twofold. We first analyze the research productivity of universities by adopting an institutional perspective. The university-based approach, which received scarce attention by previous literature, allows to produce evidence that accounts for the diversity within a higher education system. This analysis offers a more complete picture of the phenomenon than country-level studies. Second, to the best of our knowledge, this is the first study that investigates the role played by a university’s level of legitimacy in its reaction to the implementation of a PRFS. The extent to which universities are socially recognized and accepted, and are therefore involved in inter-organizational linkages with both local and national partners, shapes their behavior as a response to their research productivity being evaluated.
The paper is organized as follows. Section 2 is dedicated to the development of the hypotheses. Section 3 presents the Italian setting and the PRFS mechanism investigated in our study. Section 4 describes data and variables. Section 5 presents the results, and Sect. 6 reports some robustness checks. Section 7 concludes.
2 Hypotheses development
2.1 The impact of PRFS on universities’ research productivity
The evidence that the introduction of competing funding schemes has generally increased scientific productivity is widely documented. At a national level, Butler (2003) provides evidence of this phenomenon in the Australian higher education system. Auranen and Nieminen (2010) investigate research productivity in different European countries and find an overall positive impact of the presence of competitive funding systems. At an individual level, Osuna et al. (2011) show that the increase in the number of publications by Spanish researchers is not directly attributable to the introduction of research evaluation systems, but rather to the increase in expenditure levels and number of researchers in the system. Abramo and D’Angelo (2011b) explain that in non-competitive higher education systems, such as Italy, a PRFS may penalize top scientists in terms of funding in low-ranked universities, missing their objective to stimulate excellence in public research organizations. There are, therefore, no conclusive studies able to reconcile the effect of evaluation systems on research performance, neither within a national nor individual framework.
At the same time, the institutional perspective has been largely neglected by former literature in interpreting the effects of PRFS on scientific productivity. Only a few papers explore the impact of the introduction of PRFS at a university-level. Liefner (2003) shows that the quality of academics and the ability of students are more important factors than resource allocation criteria in determining the productivity level of a university. On the contrary, López (2006) highlights that Spanish universities are able to increase their scientific performance when funding models are shaped on the different strategies of each university.
The adoption of research-output incentives is therefore expected to increase national competition among academic institutions for research funding, thereby strengthening the incentive for universities to enhance their scientific performance (Auranen and Nieminen 2010). Based on these arguments, we formulate the following hypothesis:
HP 1
The introduction of a PRFS increases the research productivity of universities
2.2 The moderating effect of university legitimacy
Legitimacy is the extent to which an entity is recognized and accepted within some socially constructed sets of norms, values and beliefs (Suchman 1995). In the case of universities, the recent budgetary pressure and the decreasing government funding transfers have seriously challenged their financial sustainability, forcing them to develop new strategies in order to enhance their attractiveness towards students. Universities, and more in general public research and education, have a strong positive impact on new business formation (Fritsch and Aamoucke 2013), and allow innovative entrepreneurs to signal their quality by means of their education (Backes-Gellner and Werner 2007). In this context, more legitimate universities are able to signal themselves to prospective students, given the nature of higher education as a positional good (Marginson 2006). Highly legitimate universities tend to be characterized by inter-organizational linkages, i.e. voluntary ties with third parties (Barringer and Harrison 2000), such as academic spin-offs and other informal university technology transfer mechanisms (Link et al. 2007). Spin-off companies both receive and give legitimacy to their parent university,Footnote 1 especially if they are as successful as to go public (Bonardo et al. 2010, 2011), to be targeted in M&As (Meoli et al. 2013) even by foreign acquirers (Cattaneo et al. 2014), or to keep innovating independently at later stages of their life (Lejpras 2014).
The link between university legitimacy and research productivity stems from the fact that, once the research evaluation process is completed, national governments publicly disclose a ranking where universities are compared with each other (Hicks 2012). This ranking tends to receive considerable attention by national media and stimulates the debate on the quality of the higher education system. While previous studies document that the overall reaction to such mechanisms is an increase in research productivity, they fail to capture complexity and internal diversity within the university system (Rebora and Turri 2013). We argue that universities react in different ways to the implementation of PRFSs based on their level of legitimacy, and propose two main theoretical arguments explaining this view.
First, universities with higher legitimacy at stake have more to lose in their relationships with other parties in the event of an unsatisfactory outcome of the evaluation process associated with the PRFS. Although inter-organizational linkages with third parties allow to enhance institutional legitimacy per se, external partners are more willing to maintain relationships with organizations that appear in agreement with social norms and expectations of external constituents (Oliver 1990). Therefore, more legitimate universities have a stronger incentive to positively react to the implementation of competitive funding mechanisms, in order to preserve their acquired status. A lack of such recognition would be more detrimental for universities that rely more on their acquired legitimacy.
Second, the effort more legitimate universities need to put in order to increase their research productivity is relatively lower than that required by less legitimate ones. More legitimate universities benefit from a sort of competitive advantage arising from superior industry collaborations, that offer manifold opportunities for academics to acquire new insights for their research production (Cassia et al. 2013). University-industry relationships can be win–win solutions if they allow companies to deal with the technological challenges of the global economy (Audretsch et al. 2014), while simultaneously enhancing the scientific productivity of universities. Also consulting activities, when research- and commercialization-oriented, can be conducted at the same time with positive effects on research productivity. Therefore, third party linkages often produce valuable externalities in terms of privileged access to learning, in the form of new research insights about new industrial applications (Lee 2000), and funding opportunities (Melkers and Xiao 2012).
Based on these arguments, we formulate the following hypothesis:
HP 2
The increase in the research productivity of a university following the introduction of a PRFS is positively moderated by its level of legitimacy
3 The Italian performance-based research funding system
The Italian higher education system is an interesting setting where to empirically test our predictions, given the profound transformation of its funding criteria over the last two decades. Since 1993, the government has been transferring to universities a lump-sum fund, managed by universities themselves (Minelli et al. 2012). In December 2003, the Italian Ministry of Education, Universities and Research (MIUR) conducted the first peer-reviewed evaluation of the scientific output produced by universities during the period 2001–2003 (Valutazione Triennale della Ricerca, VTR), inspired by the British Research Assessment Exercise. This is the PRFS on which our paper is focused. While government funding was traditionally input-oriented, aimed at equally satisfying the needs of all universities as a function of their size and activities, from 2009 onwards the allocation criterion of 7 % of the total funding became competitive, based on the results of the VTR. As pointed out by Abramo et al. (2012), the previous absence of competitive funding criteria, that was limiting the development of top universities, led to a much lower performance variability between universities than within them. In practical terms, universities were asked to submit research products for evaluation from not more than 50 % of the full-time-equivalent researchers. This peculiarity makes the institutional perspective adopted by this paper particularly suitable, as the VTR was aimed at evaluating ex-post the best outputs produced by Italian research institutions rather than by individual researchers (Rebora and Turri 2013).
A second PRFS was launched in Italy in November 2011, pertaining to the outputs produced by universities during the period 2004–2010 (Valutazione della Qualità della Ricerca, VQR). We therefore choose 2011 as the year in which to terminate our sample, due to the presence of this new systemic shift whose effects cannot be fully addressed, given the limited time span since its introduction. This second evaluation round was profoundly different from the previous one, as universities were free to choose the evaluation method between peer-reviews or bibliometric indicators. Additionally, the attention of the VQR towards individual productivity was higher than in the VTR (Rebora and Turri 2013).
Studying the reaction of Italian universities to the introduction of a competitive funding mechanism is of interest also because all faculty members in Italy have to conduct research. The absence of universities exclusively dedicated to teaching results in a homogeneous teaching load, defined by law at a national level (Donina et al. 2014), which increases the degree of comparability of research efforts across universities. Investigating the impact of PRFS on research productivity based on the legitimacy of a university is therefore important to understand to what extent inter-organizational linkages are enabling factors of research productivity, and how they shape each university’s reaction to competitive funding allocations.
4 Research design
4.1 Sample and data sources
The sample is composed of 75 Italian universitiesFootnote 2 observed from 1999 to 2011, leading to 975 university-year observations. Information are gathered from several databases. Data on research productivity of universities are obtained from the Scopus Sciverse database, introduced by Elsevier in 2004. Literature offers numerous justifications for the use of such bibliometric data. The number of publications on bibliometric datasets are a proxy of the overall research output (Moed et al. 2005), and of the amount of knowledge that a university externally conveys. Moreover, papers may signal the presence of other assets, such as tacit knowledge and laboratories.
From a practical point of view, scanning the publications of a university can be misleading due to identification issues, such as university name changes or alternative specifications.Footnote 3 Scopus allows to minimize this concern by making use of the Scopus Affiliation Identifier, which provides all the different names through which a university has been indexed in the affiliation field of each author. We also consider the name reported in the EUMIDA (European University Data collection) classification to increase the robustness of our data. The way our research performance data is collected is in line with previous analyses of productivity of higher education systems (Gómez et al. 2009). Our legitimacy indicator is obtained by manually searching Factiva, which allows to gather media coverage attributes for each university. The characteristics of universities, such as the number of students and faculty members, are made available by the MIUR. We then complement and cross-check the information through the Cineca database.
4.2 Variables definition
The number of articles published in scientific journals per faculty member is our dependent variable (research productivity). Scaling the number of total publications by the number of faculty members of each university tackles well-known issues associated to the effects of size on productivity changes (e.g., Dundar and Lewis 1998). Our analysis is focused on quantitative measures of scientific productivity, without taking into account qualitative parameters. As a matter of fact, the VTR assessment, which is the PRFS on which our study is focused, was performed through a peer-review approach, therefore the judgment in assessing the quality of individual output might be arguable with respect, for instance, to the bibliometric indicators used in the subsequent VQR evaluation implemented in Italy (Abramo and D’Angelo 2011a). Different from the VQR, which was primarily designed to evaluate the qualitative dimension of scientific productivity, the VTR heavily relies on quantitative indicators.
The explanatory variable through which we test our first hypothesis is a dummy equal to 1 for observations starting from 2004 onwards (PRFS), aimed at capturing the effects of the introduction of the PRFS on research productivity of universities. The evaluation program was launched in December 2003, so we consider 2004 as the break-point where universities are expected to react to the implementation of this funding mechanism. Although the evaluation period (2001–2003) is prior to the launching date, this is the first time universities realize that their future share of public funding will be determined competitively, based on research performance. We therefore investigate a change in universities’ research productivity starting from 2004.
Our second explanatory variable, through which our second hypothesis is tested, is legitimacy (Legitimacy). We rely on Factiva news media database to collect the number of articles published in each sample year using university name as search criterion. This number is then scaled by the maximum value (obtained by Sapienza University of Rome in 2008). This provides a measure of public endorsement from press citations in local, national and international newspapers (Desai 2008). Since some print articles may also discredit universities by reporting cases of corruption and scandals, we randomly select 10 % of the university-year articles to have an idea of the frequency with which this occurs. On average, only 1 % of this fraction reports unfavorable news.
We employ well-known determinants of universities’ research productivity as control variables. We include the Age of each university because older institutions are recognized to have greater scientific prestige associated with higher visibility. The size of universities, measured as the total number of students enrolled each year (Size), captures the effects of larger institutions in covering a large diversity of disciplines on their research outputs. Research productivity may also be affected by the extent to which a university is focused on teaching activities, although the relationship is not necessarily negative (Horta et al. 2012). We therefore proxy for the teaching commitment of each faculty member by computing the number of students per professor (Student–Faculty ratio). We also include a variable related to universities’ source of income, i.e. the average annual tuition fee paid by students (Average fee). A variable counting the number of active PhD programs in a university (No. PhD tracks) is also considered to capture the expected higher research productivity due to their presence. As long as research universities compete for international reputation, their research productivity is stimulated by the necessity to well position themselves in the global market and to attract the best students in the world (Marginson 2006). A dummy variable (ARWU ranked) is therefore included to identify universities belonging to the University of Shanghai’s Academic Ranking of World Universities (ARWU). We then control for the presence of a department of medicine by a dummy variable (Medicine Dept.), since research in medical areas is known to be more structured and extensively collaborative than in other departments, thereby boosting overall research productivity. Finally, a set of variables allows to control for the specificities of private universities (Private university dummy) and the geographic location of each academic institution in the Italian setting (North, Centre, South dummies).
4.3 Descriptive statistics
Table 1 describes the features of our sample of Italian universities, reporting both time-varying (Panel A) and fixed (Panel B) characteristics. Panel A shows that Italian universities receive, on average, 323 citations per year by local, national and international press. This is our measure of university legitimacy. They are on average old institutions (228 years of age in 2011), but the sensibly lower median value (49) suggests that the age distribution is right-skewed. There are indeed some very ancient universities, such as the University of Bologna, established in 1088, and University of Modena and Reggio Emilia, established in 1175. On average, universities have enrolled approximately 24 thousands students per year. In terms of teaching resources, academic institutions in our sample are recognized to suffer from a relatively high student–faculty ratio over the last decade (median value of 32) which may be the cause of a potential decrease in research activity by faculty members (Audretsch et al. 2012). This is consistent with the evidence that Italy is one of the countries with the highest number of students per teacher worldwide (OECD 2013). An annual average of 28 % of our sample is included in the University of Shanghai’s Academic Ranking of World Universities (ARWU). Panel B highlights that 23 % of universities have a department of medicine, while 16 % are private. The sample is quite homogeneously distributed from a geographical point of view, with prevalence in Northern Italy: according to the ISTAT (Italian National Institute of Statistics) macro-area classification, 30 universities are established in the North, 19 in the Centre, and 26 in the South of the country.
Figure 1 shows the trend of research productivity exhibited by our sample of universities. The graph plots the average ratio of the number of publications per faculty member for the whole sample, and by distinguishing between high- and low-legitimacy universities. These are defined as universities with an above or below median value of legitimacy, on a yearly basis. The upward trend in research productivity triggered by the introduction of PRFS stands out clearly. Before 2004, productivity looks quite stable at an aggregate level, around approximately 0.45 publications per faculty member. The graph also documents that more legitimate universities show a persistently higher level of productivity than less legitimate universities, with the former presenting an almost constant level of publications per faculty member (around 0.5), while the latter are associated with a slight decline, which reaches a minimum in 2002 (around 0.35).
Research productivity of universities. The graph shows the average ratio of the number of publications over faculty members of the sample of 75 universities. High- (low-) legitimacy universities are those with an above (below) median value of legitimacy, on a yearly basis
The introduction of PRFS in December 2003 causes a substantial change in the trend of research productivity. Universities seem to anticipate such effect already in 2003, where the number of publications per faculty member considerably increases compared to the previous year. From 2004 onwards, productivity keeps growing. The only exception occurs in 2005, just after the introduction of PRFS, where the aggregate level of productivity is lower than 2004. This is, however, the result of two opposite reactions by universities to the PRFS, according to their level of legitimacy. While highly legitimate universities report a remarkable, persistent increase in research productivity, less legitimate universities seem to be less prompt in aligning their incentives with those imposed by the PRFS.
Table 2 reports the univariate analysis of research productivity of universities, by splitting the sample before and after the introduction of the PRFS. Panel A refers to the whole sample of universities, while panels B and C to the subsamples of high- and low-legitimacy universities, respectively. The average number of publications per faculty member in our sample is 0.58 (0.55 in median). Although this figure is not easily comparable with other studies, due to the limited number of papers evaluating research productivity at an institutional level, it seems in line with prior literature. For instance, Abramo et al. (2012, Table 1) report a sensibly higher average number of research outputs per faculty member (0.995) in the Italian higher education system during 2004–2008. However, they include also article reviews and conference proceedings, while we limit our analysis to articles published in scientific journals.
As revealed by the tests for the difference between the pre- and post-PRFS distributions, research productivity is significantly lower prior to the introduction of the PRFS, with an average of 0.44 (0.50 in median) compared to 0.58 (0.55). Both the number of publications and faculty members significantly increase over time, but the increase in the number of publications is more pronounced. While the average number of faculty members per university raises from 752.6 before the PRFS to 789.8 afterwards, the average number of publications jumps from 397.2 to 610.1.
Panels B and C show the same figures for the subsamples of high- and low-legitimacy universities, defined as those with an above- and below-median value of average legitimacy, respectively. As previously suggested by the graph, both the average and median number of publications per faculty member are higher among more legitimate universities. Most importantly, the increase in productivity after the introduction of the PRFS is significant for both categories, but is more pronounced among more legitimate universities (from 0.54 to 0.80, on average) than among less legitimate ones (from 0.35 to 0.50).
5 Empirical tests and results
We now test our hypotheses in a multivariate setting.Footnote 4 We employ Tobit models for panel data given that our dependent variable is characterized by a lower limit (zero). We do not include fixed effects because the maximum likelihood estimator in non-linear panel data models with fixed effects is widely understood to be biased and inconsistent when T, the length of the panel, is small and fixed.
Table 3 reports estimates of the Tobit panel regression. The coefficient of the PRFS dummy variable is positive and significant at the 1 % level, both when tested alone (column 1) and in the full model specification (column 3). This is consistent with our hypothesis 1, stating that the introduction of PRFS increases the research productivity of universities. When universities become aware that research productivity is the criterion by which they will compete for funds, they react accordingly. This evidence is in line with previous literature. The coefficient of the legitimacy variable is also positive and significant. This documents that more legitimate universities exhibit greater research productivity, thanks to inter-organizational linkages that contribute to enhancing scientific outputs by facilitating learning and funding opportunities.
Consistent with our second hypothesis, the coefficient of the interaction term between PRFS and legitimacy is positive and significant at the 1 % level (column 3). The increase in research productivity following the introduction of the PRFS is, therefore, more pronounced among highly legitimate universities. This may be due to their stronger incentive to preserve such status, leading to a greater reaction to the introduction of the PRFS. A potential negative outcome from the ranking would indeed be more detrimental for these universities that already have considerable legitimacy at stake. At the same time, it is easier for these universities to increase research productivity, given the greater benefits in terms of learning and funding opportunities arising from superior third party linkages. Overall, we find empirical evidence in support of both our hypotheses.
Concerning our control variables, those reporting significant coefficients have the predicted sign. Larger universities tend to be more productive, while those with a larger student–faculty ratio show a lower level of research productivity. Predictably, a larger number of students per faculty member leads to more time and effort spent by academics in teaching-oriented activities, which penalizes their focus on research. Universities with larger fees per student tend to have higher availability of financial resources that can be effectively used to stimulate research productivity. Finally, results show that private universities are less research-oriented (similar evidence is provided in the Spanish setting (Gómez et al. 2009), while those located in northern Italy tend to be more productive.
Since the presence of biomedical and engineering departments are associated to higher interactions with organizations outside the academia (Lach and Schankerman 2008; Bonaccorsi et al. 2013), we investigate the same relationship by distinguishing universities with at least one Engineering or Medicine (EM) department from those with no EM departments. We expect the effect of university legitimacy to be stronger among universities with EM departments, due to the higher marketability of the inventions developed inside their laboratories (Huyghe and Knockaert 2014). We therefore test our hypotheses by splitting the sample in 59 universities with EM departments versus 16 universities with no EM departments. Results are reported in Table 4. Consistent with our expectation, the aggregate effect of university legitimacy on research productivity is mainly driven by the subsample of universities with EM departments, as pointed out by the positive and significant coefficient in this subsample. On the other hand, the effect of university legitimacy is not significant for universities with no EM departments.
6 Robustness checks
6.1 Endogeneity between research productivity and legitimacy of universities
There is a potential endogeneity issue between research productivity and legitimacy of a university. In particular, reverse causality may arise from the fact that the level of legitimacy is clearly affected by prior research performance. More productive universities benefit from higher legitimacy in the eyes of stakeholders. Furthermore, one could reasonably argue that also other university characteristics included in our analysis, such as size, student–faculty ratio, average fees, and number of PhD tracks, might be endogenously determined.
In order to fix potential endogeneity problems, two closely related dynamic panel data models were presented, among others, in former literature. The first is called the Arellano-Bond “first-differences GMM” model, while the second is an Arellano-Bover augmented version of the Arellano-Bond estimator, known as the “system GMM” model. Both estimators are designed for dynamic “small time-series and large cross-section” panels that may contain fixed effects and idiosyncratic errors. However, a problem with the original Arellano-Bond estimator is that lagged levels are poor instruments for first-differences if the variables are close to a random walk. The Arellano-Bover model benefits from the possibility to use additional instruments to increase efficiency. Therefore, we employ the system GMM model instead of the first-differences GMM model in our estimation. The two-step Arellano–Bond estimator we employ is designed for dynamic small T, large N panels, and can also account for the endogeneity of the vector of university characteristics. Legitimacy and other university characteristics are instrumented by including their lags of order 1 and 2.Footnote 5 Table 5 presents the results.
Consistent with our first hypothesis, the coefficients of the PRFS dummy are still significant, documenting a positive effect of the introduction of competitive funding mechanisms on research productivity of universities (columns 1 and 3). Our second hypothesis also finds support, as the interaction term with the legitimacy variable still has a positive and significant coefficient, documenting that more legitimate universities are those who experience the largest increase in productivity following the introduction of the PRFS. Overall, after controlling for potential endogeneity between research productivity and university characteristics, including legitimacy, our hypotheses still find empirical support.
6.2 Legitimacy versus reputation
The literature on higher education systems suggests that legitimacy and reputation are similar and overlapping concepts. As explained by Deephouse and Carter (2005), they result from similar social construction processes by which stakeholders evaluate an organization (e.g., Ashforth and Gibbs 1990), and are determined by similar factors, such as organizational size, strategic alliances, regulatory compliance (e.g., Galaskiewicz 1985). They also both allow to benefit from an increased ability in resource acquisition (e.g., Suchman 1995). Therefore, since we employ both legitimacy and reputation as independent variables in our multivariate analysis, there might be potential overlapping in the effects captured by these two parameters.
We address this issue by purging the legitimacy variable from its fraction of variance that is explained by reputation, as follows. First, we regress legitimacy against two measures of reputation, namely a dummy variable equal to 1 for universities belonging to the University of Shanghai’s Academic Ranking of World University (previously defined as ARWU ranked), and an additional variable referring to the position in the ranking. This is calculated as the reciprocal of the ranking position, which is the mid-point of the extremes of the ranking interval in which the university is included (e.g., 125.5 for universities included in the 101–150 ranking class). Both variables are determinants of legitimacy at the 1 % significance level. Second, we take the residuals from this first step regression and include them as the new legitimacy variable (Legitimacy_Res) to explain research productivity. Table 6 shows the results of the second step regressions.
PRFS is still found to increase research productivity, as expected. The interaction term with legitimacy, which is now computed as the residuals of the first step regression, is also positive and significant, which confirms that more legitimate universities react more to the introduction of PRFS in terms of increased research productivity. Overall, after correcting for potential overlapping effects of legitimacy and reputation, the empirical evidence still provides support to the hypotheses of our paper.
7 Summary and conclusions
This study adopts a relatively unexplored perspective in investigating the impact of the introduction of Performance-based Research Funding Systems (PRFS) on research productivity of universities. Specifically, we evaluate how universities react to the implementation of competitive funding mechanisms in terms of research productivity, based on their level of legitimacy. We analyze the research productivity of a sample of Italian universities over the last decade in a multivariate panel setting. Consistent with prior literature, we find that the introduction of the PRFS positively stimulates research productivity of universities. Furthermore, after showing that more legitimate universities tend to be more productive, we document that they exhibit the largest increase in productivity once the competitive funding mechanism is introduced. While more legitimate organizations have a greater incentive to preserve such status, and therefore react more to the introduction of the PRFS than their less legitimate peers, they also have to put a relatively lower effort to increase their research productivity.
Our results shed further light on how the diversity within a higher education system shapes the impact that a regulatory change exerts on the organizations belonging to this system. While previous studies conduct either country- or individual-level analyses, this paper is one of the first to investigate the introduction of competitive funding mechanisms at an institutional level. By doing so, we are able to account for the role played by the characteristics of each organization in the process of realignment of incentives. Macro-level analyses, such as cross-country studies, fail to control for these specificities. The assessment exercise of complex activities such as scientific research needs not to ignore the peculiarities of each organization, as these determine the complexity and internal diversity of the whole higher education system. In particular, this paper contributes to unveil the importance of university legitimacy. While higher legitimacy provides university with a sort of competitive advantage in terms of research productivity, it also shapes the university’s reaction to competitive funding systems.
Notes
For a detailed review, see Djokovic and Souitaris (2008).
In order to guarantee homogeneity we exclude 6 doctoral universities, 11 long distance learning institutions, and 3 universities for foreigners from the population of 96 Italian universities.
For instance, the University of Milan is reported in three different ways: Università degli Studi di Milano, University of Milan, Università di Milano.
The correlation matrix of the variables employed in our analysis is reported in Table 7 in “Appendix”.
The test for autocorrelation fails to reject the null hypothesis that average auto-covariance in residuals of order 2 is zero (z = 1.48; p = 0.139 for the baseline specification), thus suggesting that it is safe to restrict the instrument set to up to 2 lags. The Sargan test for over identifying restrictions is valid all over our specifications.
References
Abramo, G., Cicero, T., & D’Angelo, C. (2012). Revisiting size effects in higher education research productivity. Higher Education, 63(6), 701–717.
Abramo, G., & D’Angelo, C. (2011a). Evaluating research: From informed peer review to bibliometrics. Scientometrics, 87(3), 499–514.
Abramo, G., & D’Angelo, C. A. (2011b). National-scale research performance assessment at the individual level. Scientometrics, 86(2), 347–364.
Ashforth, B. E., & Gibbs, B. W. (1990). The double-edge of organizational legitimation. Organization Science, 1(2), 177–194.
Audretsch, D. B., Hülsbeck, M., & Lehmann, E. E. (2012). Regional competitiveness, university spillovers, and entrepreneurial activity. Small Business Economics, 39(3), 587–601.
Audretsch, D., Lehmann, E., & Wright, M. (2014). Technology transfer in a global economy. Journal of Technology Transfer, 39(3), 301–312.
Auranen, O., & Nieminen, M. (2010). University research funding and publication performance: An international comparison. Research Policy, 39(6), 822–834.
Backes-Gellner, U., & Werner, A. (2007). Entrepreneurial signaling via education: A success factor in innovative start-ups. Small Business Economics, 29(1–2), 173–190.
Barringer, B. R., & Harrison, J. S. (2000). Walking a tightrope: Creating value through interorganizational relationships. Journal of Management, 26(3), 367–403.
Bonaccorsi, A., Colombo, M. G., Guerini, M., & Rossi-Lamastra, C. (2013). University specialization and new firm creation across industries. Small Business Economics, 41(4), 837–863.
Bonardo, D., Paleari, S., & Vismara, S. (2010). The M&A dynamics of European science-based entrepreneurial firms. Journal of Technology Transfer, 35(1), 141–180.
Bonardo, D., Paleari, S., & Vismara, S. (2011). Valuing university-based firms: The effects of academic affiliation on IPO performance. Entrepreneurship Theory and Practice, 35(4), 755–776.
Butler, L. (2003). Explaining Australia’s increased share of ISI publications: The effects of a funding formula based on publication counts. Research Policy, 32(1), 143–155.
Cassia, L., De Massis, A., Meoli, M., & Minola, T. (2013). Entrepreneurship research centers around the world: Research orientation, knowledge transfer and performance. Journal of Technology Transfer, 2013, 1–17.
Cattaneo, M., Meoli, M., & Vismara, S. (2014). Cross-border M&As of biotech firms affiliated with internationalized universities. Journal of Technology Transfer, 1–25. doi:10.1007/s10961-014-9349-8.
De Boer, H., Enders, J., & Schimank, U. (2007). On the way towards new public management? The governance of university systems in England, the Netherlands, Austria, and Germany. In D. Jansen (Ed.), New forms of Governance in Research Organizations (pp. 137–152). New York: Springer.
Deephouse, D. L., & Carter, S. M. (2005). An examination of differences between organizational legitimacy and organizational reputation. Journal of Management Studies, 42(2), 329–360.
Desai, V. M. (2008). Constrained growth: How experience, legitimacy, and age influence risk taking in organizations. Organization Science, 19(4), 594–608.
Djokovic, D., & Souitaris, V. (2008). Spinouts from academic institutions: A literature review with suggestions for further research. Journal of Technology Transfer, 33(3), 225–247.
Donina, D., Meoli, M., & Paleari, S. (2014). Higher education reform in Italy: Tightening regulation instead of steering at a distance. Higher Education Policy. doi:10.1057/hep.2014.6.
Dundar, H., & Lewis, D. (1998). Determinants of research productivity in higher education. Research in Higher Education, 39(6), 607–631.
Fritsch, M., & Aamoucke, R. (2013). Regional public research, higher education, and innovative start-ups: An empirical investigation. Small Business Economics, 41(4), 865–885.
Galaskiewicz, J. (1985). Interorganizational relations. Annual Review of Sociology, 11(1), 281–304.
Geuna, A., & Martin, B. R. (2003). University research evaluation and funding: An international comparison. Minerva, 41(4), 277–304.
Gómez, I., Bordons, M., Fernández, M., & Morillo, F. (2009). Structure and research performance of Spanish universities. Scientometrics, 79(1), 131–146.
Herbst, M. (2007). Financing public universities. Higher Education Dynamics (Vol. 18). New York: Springer.
Hicks, D. (2009). Evolving regimes of multi-university research evaluation. Higher Education, 57(4), 393–404.
Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251–261.
Horta, H., Dautel, V., & Veloso, F. M. (2012). An output perspective on the teaching-research nexus: An analysis focusing on the United States higher education system. Studies in Higher Education, 37(2), 171–187.
Huyghe, A., & Knockaert, M. (2014). The influence of organizational culture and climate on entrepreneurial intentions among research scientists. Journal of Technology Transfer. doi:10.1007/s10961-014-9333-3.
Lach, S., & Schankerman, M. (2008). Incentives and invention in universities. The Rand Journal of Economics, 39(2), 403–433.
Lee, Y. S. (2000). The sustainability of university-industry research collaboration: An empirical assessment. Journal of Technology Transfer, 25(2), 111–133.
Leisyte, L., & Kizniene, D. (2006). New public management in Lithuania’s higher education. Higher Education Policy, 19(3), 377–396.
Lejpras, A. (2014). How innovative are spin-offs at later stages of development? Comparing innovativeness of established research spin-offs and otherwise created firms. Small Business Economics, 43(2), 327–351.
Liefner, I. (2003). Funding, resource allocation, and performance in higher education systems. Higher Education, 46(4), 469–489.
Link, A. N., Siegel, D. S., & Bozeman, B. (2007). An empirical analysis of the propensity of academics to engage in informal university technology transfer. Industrial and Corporate Change, 16(4), 641–655.
López, M. J. G. (2006). Towards decentralized and goal-oriented models of institutional resource allocation: The Spanish case. Higher Education, 51(4), 589–617.
Marginson, S. (2006). Dynamics of national and global competition in higher education. Higher Education, 52(1), 1–39.
Melkers, J., & Xiao, F. (2012). Boundary-spanning in emerging technology research: Determinants of funding success for academic scientists. Journal of Technology Transfer, 37(3), 251–270.
Meoli, M., Paleari, S., & Vismara, S. (2013). Completing the technology transfer process: M&As of science-based IPOs. Small Business Economics, 40(2), 227–248.
Minelli, E., Rebora, G., & Turri, M. (2012). Waiting for the market: Where is the Italian university system heading? Higher Education Policy, 25(1), 131–145.
Moed, H. F. (2008). UK research assessment exercises: Informed judgments on research quality or quantity? Scientometrics, 74(1), 153–161.
Moed, H. F., Glänzel, W., & Schmoch, U. (2005). Handbook of quantitative science and technology research. Netherlands: Springer.
OECD. (2013). Education at a Glance 2013: OECD indicators. OECD Publishing.
Oliver, C. (1990). Determinants of interorganizational relationships: Integration and future directions. Academy of Management Review, 15(2), 241–265.
Orr, D., Jaeger, M., & Schwarzenberger, A. (2007). Performance-based funding as an instrument of competition in German higher education. Journal of Higher Education Policy and Management, 29(1), 3–23.
Osuna, C., Cruz-Castro, L., & Sanz-Menéndez, L. (2011). Overturning some assumptions about the effects of evaluation systems on publication performance. Scientometrics, 86(3), 575–592.
Rebora, G., & Turri, M. (2013). The UK and Italian research assessment exercises face to face. Research Policy, 42(9), 1657–1666.
Reichert, S. (2009). Institutional diversity in European higher education. Brussels: EUA.
Suchman, M. C. (1995). Managing legitimacy: Strategic and institutional approaches. Academy of Management Review, 20(3), 571–610.
Strehl, F., Reisinger, S., & Kalatschan, M. (2007). Funding systems and their effects on higher education systems. OECD Education Working Papers. doi:10.1787/220244801417.
Acknowledgments
This work was supported by the Fondazione Cariplo under Grant 2010-1106. We thank Davide Donina and Stefano Paleari for useful discussions and helpful comments as well as two referees for their stimulating suggestions.
Author information
Authors and Affiliations
Corresponding author
Appendix
Rights and permissions
About this article
Cite this article
Cattaneo, M., Meoli, M. & Signori, A. Performance-based funding and university research productivity: the moderating effect of university legitimacy. J Technol Transf 41, 85–104 (2016). https://doi.org/10.1007/s10961-014-9379-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10961-014-9379-2