Introduction

The very conditions responsible for scientific and economic backwardness may operate through the poor management of R&D activities (Wang and Huang 2007). So, there is a potentially important gap or omission in an urgent investigation into the production quality of macro-R&D activities themselves over time. The existing relevant literatures of measuring R&D activities themselves neglect the operational process of R&D activities, and, therefore, can’t grasp the weaknesses and bottlenecks resulting in the production inefficiency of R&D activities. A process-oriented productive efficiency measure may be a more effective approach to investigate and benchmark macro-R&D performance based on the management or economic perspective (Guan and Wang 2004; Sharma and Thomas 2008).

The first part of our methodological framework employs an enhanced non-radial data envelopment analysis (DEA) model (Zhu 2003) to investigate the productive efficiency of macro-R&D activities. The efficiency model employed here can conquer the limitation of the existing R&D efficiency measures by means of a radial DEA model (e.g., Rousseau and Rousseau 1997; Guan and Wang 2004; Abramo et al. 2008; Sharma and Thomas 2008) or stochastic frontier analysis (SFA) (e.g., Fritsch and Slavtchev 2007; Wang 2007).

Besides, the measurement of R&D productive efficiency change across periods (i.e., the chronological transition of productivity over time) would be more important for benchmarking the improvement or progress on new management methods or policy instruments. The continuously improved productivity of R&D activities needs not have been from the efficiency improvement alone, but may have been due to the R&D technological progress or the exploitation of R&D scale economies or from some combination of these three factors. However, the relevant study of dynamic performances has so far not been found in the existing literature yet. For obtaining an all-sided and accurate measure, an improved nonparametric Malmquist index (Ray and Desli 1997) has been introduced to measure the dynamic total factor productivity change of macro-R&D activities themselves in the second part of our methodological framework.

In the next section, the analytical framework of measuring R&D PFP is constructed. Then, the empirical study based on a panel dataset on Chinese province-level R&D activities is implemented. Finally, concluding remarks are drawn based on the preceding discussion.

Analytical framework of R&D PFP

Inputs and outputs of R&D activities

This study designs a multi-input and multi-output indictor system for accurately measuring the R&D PFP. Following Guan and Wang (2004) and Meng et al. (2006), the inputs mainly include R&D expenditure (RDE) and R&D personnel (RDP). The RDP is measured by the full-time equivalent scientists and engineers in all sectors. R&D activities consist of basic research (BR), applied research (AR) and experimental development (ED). Published academic papers are the main products of BR. The products of AR mainly involve academic papers, inventive patents or theoretical models used to produce new products. The main outcome forms of ED are new products models, proprietary technology and patents. So, the outputs of R&D activities can be measured by patents, academic papers and new products.

Non-radial DEA for R&D productive efficiency within a period

In this study, we choose DEA rather than SFA since DEA can accommodate multiple outputs production activities. In addition, DEA doesn’t need to impose an explicit functional form for the underlying technology and an explicit distributional assumption for the inefficiency term in contrast to SFA (Sharma et al. 1997). However, the traditional radial DEA models represented by CCR (Charnes et al. 1978) and BCC (Banker et al. 1984) optimize all outputs or inputs of a decision making unit (DMU) at a certain radial proportion, which can’t account for non-radial slacks (Pastor et al. 1999). Moreover, the simultaneous radial change of all inputs or all outputs in the radial DEA models deviates from the production reality. So, this paper chooses an input-oriented and non-radial DEA model (Zhu 2003) for R&D efficiency measures.

If look on each macro-scale R&D production system as a DMU and let \( x_{ij} (i = 1,2, \ldots ,m) \) and \( y_{rj} (r = 1,2, \ldots ,s) \) be inputs and outputs of the \( {\text{DMU}}_{j} (j = 1,2, \ldots ,n) \) respectively, the linear program (LP) estimating the R&D technical efficiency under variable return to scale (Hereinafter abbreviated as RDVE) is as follows:

$$ \begin{gathered} \min\quad{\frac{1}{m}}\sum\limits_{i = 1}^{m}{\theta_{i} } \hfill \\ {\text{s}} . {\text{t}} .\quad y_{ro} \le\sum\limits_{j = 1}^{n} {\lambda_{j} y_{rj} } ,\quad r = 1,2,\ldots ,s, \hfill \\ \theta_{i} x_{io} = \sum\limits_{j = 1}^{n}{\lambda_{j} x_{ij} } ,\quad i = 1,2, \ldots ,m, \hfill \\\sum\limits_{j = 1}^{n} {\lambda_{j} = 1} , \hfill\\\quad\lambda_{j} \ge 0,1 \ge \theta_{i} \ge 0,\quad j = 1,2,\ldots ,n;i = 1,2, \ldots ,m. \hfill \\ \end{gathered} $$
(1)

Here, \( \lambda_{j} (j = 1,2, \ldots ,n) \) are structure variables. The LP (1) without the constraint \( \sum\nolimits_{j = 1}^{n} {\lambda_{j} = 1} \) is used to measure the R&D technical efficiency under constant return to scale (Hereinafter abbreviated as RDCE). Besides, following Byrnes et al. (1984), we employ the ratio value of RDCE to RDVE to measure the scale efficiency of R&D activities (Hereinafter abbreviated as RDSE).

Non-radial Malmquist index for R&D productivity change over adjacent periods

In this study, we employ the RD-Malmquist index (Ray and Desli 1997) under variable return to scale to measure the R&D productivity change instead of the FGNZ-Malmquist index (Färe et al. 1994) under constant return to scale. The RD-Malmquist index can not only provide more accurate measures and sufficient decompositions of productivity change (Ggifell-Tatjé and Lovell 1999), but also measure the autonomous shift in frontier and conquer the inherent contradiction in the FGNZ decomposition (Ray and Desli 1997).

Let \( D^{a} (x_{o}^{b} ,y_{o}^{b} ) \) denote the relative efficiency of the object R&D DMUo in period b against the performance of all R&D DMUs in period a. When (a, b) takes (t, t), (t, t + 1), (t + 1, t) and (t + 1, t + 1), there are four efficiencies, \( D^{t} (x_{o}^{t} ,y_{o}^{t} ) \), \( D^{t} (x_{o}^{t + 1} ,y_{o}^{t + 1} ) \), \( D^{t + 1} (x_{o}^{t} ,y_{o}^{t} ) \) and \( D^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} ) \). Obviously, the two within-period efficiencies, \( D^{t} (x_{o}^{t} ,y_{o}^{t} ) \) and \( D^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} ) \), can be obtained by the LP (1). The two adjacent-period efficiencies, \( D^{t} (x_{o}^{t + 1} ,y_{o}^{t + 1} ) \) and \( D^{t + 1} (x_{o}^{t} ,y_{o}^{t} ) \), can be obtained by the LP (2) as follows.

$$ \begin{gathered} D^{a} (x_{o}^{b} ,y_{o}^{b} ) = \min {\frac{1}{m}}\sum\limits_{i = 1}^{m} {\theta_{i} } \hfill \\ {\text{s}}. {\text{t}}. \quad y_{ro}^{b} \le \sum\limits_{j = 1}^{n} {\lambda_{j} y_{rj}^{a} + } \lambda_{n + 1} y_{ro}^{b} ,\quad r = 1,2, \ldots ,s, \hfill \\ \theta_{i} x_{io}^{b} = \sum\limits_{j = 1}^{n} {\lambda_{j} x_{ij}^{a} } + \lambda_{n + 1} x_{io}^{b} ,\quad i = 1,2, \ldots ,m, \hfill \\ \sum\limits_{j = 1}^{n + 1} {\lambda_{j} = 1,} \hfill \\ \lambda_{j} \ge 0,1 \ge \theta_{i} \ge 0 , j = 1,2, \ldots ,n + 1;i = 1,2, \ldots ,m. \hfill \\ \end{gathered} $$
(2)

Like the LP (1), the LP (2) without \( \sum\nolimits_{j = 1}^{n + 1} {\lambda_{j} = 1} \) is used to measure the two adjacent-period efficiencies under constant return to scale.

R&D efficiency change (RDECH)

The gains on R&D productive capacity usually are attributable to the R&D efficiency improvement associated with such soft factors as production management, institutional conditions and policy instruments. Of course, the R&D efficiency retrogression can weaken R&D productive capacity. It’s possible that the R&D efficiency change (RDECH) may outweigh the effect from the R&D technological change on R&D productive capacity. The RDECH, embodying “catch-up” of R&D efficiency, of the object DMUo is estimated by

$$ {\text{RDECH}} = {{D_{v}^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} )} \mathord{\left/ {\vphantom {{D_{v}^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} )} {D_{v}^{t} (x_{o}^{t} ,y_{o}^{t} )}}} \right. \kern-\nulldelimiterspace} {D_{v}^{t} (x_{o}^{t} ,y_{o}^{t} )}} $$

The RDECH is a pure technical efficiency change of R&D activities since the relative efficiencies \( D_{v}^{t} (x_{o}^{t} ,y_{o}^{t} ) \) and \( D_{v}^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} ) \)are obtained under variable return to scale (subscripted with a “v”). The change results can be judged as R&D efficiency growth, stagnation or decline according to the calculation value of RDECH above unity, equivalent to unity or below unity (Similar judgment can be applied for other change indexes hereinafter).

R&D technological change (RDTCH)

R&D technological change (RDTCH) is the result of the improvement in R&D technology/skill, such as introducing new R&D experiment equipments or new R&D process and system. It results in the movement of R&D best practice production frontier. It is important to know how far one is off the R&D technological frontier at any point in time, and how quickly one can reach the frontier in terms of the renewal and modernization of R&D equipments. The RDTCH, meaning “frontier shift” of R&D technology, of the object DMUo is estimated by

$$ {\text{RDTCH}} = \left[ {{\frac{{D_{v}^{t} (x_{o}^{t + 1} ,y_{o}^{t + 1} )}}{{D_{v}^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} )}}} \cdot {\frac{{D_{v}^{t} (x_{o}^{t} ,y_{o}^{t} )}}{{D_{v}^{t + 1} (x_{o}^{t} ,y_{o}^{t} )}}}} \right]^{1/2} $$

R&D scale efficiency change (RDSECH)

Macro-scale R&D activities implemented by regional or national innovation systems, unlike simple micro-R&D activities taking place in certain R&D labs or R&D institutes, exhibit a large-scale characteristic, which provides a prerequisite for scale economics effect. In a well-functioning innovation system implementing macro-scale R&D activities, effective R&D collaboration networks can make the R&D cost reduced depending on technological knowledge and equipments effectively shared, i.e., economies of scale can be obtained. So, the “exploitation” of R&D scale economics effect is pursued by macro-scale R&D activities, whose quality is measured by the R&D scale efficiency change (RDSECH) index.

Since \( {{D_{c}^{a} (x_{o}^{b} ,y_{o}^{b} )} \mathord{\left/ {\vphantom {{D_{c}^{a} (x_{o}^{b} ,y_{o}^{b} )} {D_{v}^{a} (x_{o}^{b} ,y_{o}^{b} )}}} \right. \kern-\nulldelimiterspace} {D_{v}^{a} (x_{o}^{b} ,y_{o}^{b} )}} \) (The subscript “c” indicates constant return to scale.) can be used to estimate the scale efficiency score, the relative R&D scale efficiency change (RDSECH) of the object DMUo in the period t and period t + 1 can be gauged by \( {\text{RDSECH}}^{t} = {\frac{{{{D_{c}^{t} (x_{o}^{t + 1} ,y_{o}^{t + 1} )} \mathord{\left/ {\vphantom {{D_{c}^{t} (x_{o}^{t + 1} ,y_{o}^{t + 1} )} {D_{v}^{t} (x_{o}^{t + 1} ,y_{o}^{t + 1} )}}} \right. \kern-\nulldelimiterspace} {D_{v}^{t} (x_{o}^{t + 1} ,y_{o}^{t + 1} )}}}}{{{{D_{c}^{t} (x_{o}^{t} ,y_{o}^{t} )} \mathord{\left/ {\vphantom {{D_{c}^{t} (x_{o}^{t} ,y_{o}^{t} )} {D_{v}^{t} (x_{o}^{t} ,y_{o}^{t} )}}} \right. \kern-\nulldelimiterspace} {D_{v}^{t} (x_{o}^{t} ,y_{o}^{t} )}}}}} \) and \( {\text{RDSECH}}^{t + 1} = {\frac{{{{D_{c}^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} )} \mathord{\left/ {\vphantom {{D_{c}^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} )} {D_{v}^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} )}}} \right. \kern-\nulldelimiterspace} {D_{v}^{t + 1} (x_{o}^{t + 1} ,y_{o}^{t + 1} )}}}}{{{{D_{c}^{t + 1} (x_{o}^{t} ,y_{o}^{t} )} \mathord{\left/ {\vphantom {{D_{c}^{t + 1} (x_{o}^{t} ,y_{o}^{t} )} {D_{v}^{t + 1} (x_{o}^{t} ,y_{o}^{t} )}}} \right. \kern-\nulldelimiterspace} {D_{v}^{t + 1} (x_{o}^{t} ,y_{o}^{t} )}}}}} \) respectively. For balancedly and comprehensively aggregating the equilibrium impacts of \( {\text{RDSECH}}^{t} \) and \( {\text{RDSECH}}^{t + 1} \), we use the geometric mean of two components (Ray and Desli 1997) to estimate the RDSECH of the object DMUo, i.e.

$$ {\text{RDSECH}} = \left[ {{\text{RDSECH}}^{t} \cdot {\text{RDSECH}}^{t + 1} } \right]^{1/2} $$

R&D productivity change (RDPCH)

Following the RD-Malmquist index (Ray and Desli 1997), the total factor R&D productivity change (RDPCH) of the object DMUo can be gauged by

$$ {\text{RDPCH = RDECH}} \cdot {\text{RDTCH}} \cdot {\text{RDSECH}} $$

.

An application

The Chinese innovation system is too large and complex to be summarized with a single model, and the provincial (e.g., regional versus national) dimension should not be overlooked (OECD 2008). We employ the proposed methodological framework to analyze the PFP of Chinese R&D production at the provincial level.

Sample and data

A total of 30Footnote 1 selected Chinese provinces form the sample used in this study. The granted domestic inventive patent counts are used as the measure of patent outputs (e.g., Guan and Liu 2005). The academic papers include SCI, EI and ISTP papers which are recognized internationally (e.g., Huang et al. 2006). The sale revenue of new products (SRNP) is used as the measure of new products (e.g., Faber and Hesen 2004). The SRNP dataset is collected from Chinese large and medium-sized industrial enterprises (LMEs).

Because R&D production needs a productive transformation process, there is a time lag from R&D inputs to R&D outputs. However, there has been no generally accepted length of the time lag on R&D outputs until now (Wang and Huang 2007). In terms of the difference in the time lag span on different R&D outputs and the characteristics of Chinese R&D activities, we have set a time lag of 2 years on both the granted process of patents (e.g., Johnson 2002) and the published process of academic papers but a time lag of 1 year on the new products. So, in terms of data sources, the input dataset on RDE and RDP is collected from 2000 to 2003 and the output dataset on patents and papers is collected from 2002 to 2005, while the SRNP dataset is collected from 2001 to 2004. All data are based on official statistics: China Statistical Yearbook on Science and Technology (2001–2007) (Beijing: China Statistics Press). Besides, the observations of both RDE and SRNP are deflated to the 2000 value.

Results of PFP

Table 1 shows the respective results of seven indexes related to PFP of 30 selected Chinese provinces from 2000 to 2003. We can compare 30 selected Chinese provinces on these production frontier indexes. Generally speaking, there are big differences whether across 30 selected Chinese provinces in terms of individual indexes or among seven indexes in terms of individual provinces.

Table 1 Average efficiency and productivity change of R&D activities over 2000–2003 period at the Chinese provincial level
Full size table

Specifically, seven provinces (23%), Beijing, Tianjin, Jilin, Heilongjiang, Shanghai, Hainan and Gansu, were always efficient from 2000 to 2003 across three R&D efficiency indexes. However, these provinces have relatively poorer performance in terms of the ranking result of RDPCH in the last column of Table 1. With respect to the time-series disaggregated scores of seven indexes from 2000 to 2003, an ANOVA reveals that F = 25.231 (n = 120) for the three sets of R&D efficiency scores, which is significant at the 1% level (the critical value is 3.02) and means there is remarkable difference across three efficiency sets. Another ANOVA reveals that F = 17.624 (n = 90) for the four sets of R&D productivity change scores, which is also significant at the 1% level (the critical value is 2.63) and means there is also a considerable difference among four R&D efficiency change scores.

We turn to examine the variation on provincial average and interprovincial dispersion of the annual disaggregated scores of each PFP index from 2000 to 2003. Table 2 and 3 show that the provincial average of each PFP index is in the unbalanced and unstable state. Moreover, Table 2 shows that the interprovincial dispersion on any one of three R&D efficiency indexes is widening. In contrast, the respective interprovincial dispersions on RDTCH, RDSECH and RDPCH are in unstable state except that the interprovincial dispersion on RDECH becomes wider and wider.

Table 2 Provincial average and interprovincial dispersion of annual disaggregated scores of R&D efficiency indexes from 2000 to 2003
Full size table
Table 3 Provincial average and interprovincial dispersion of annual disaggregated scores of R&D productivity change indexes from 2000 to 2003
Full size table

Concluding remarks

The objective of this paper is to provide a methodological framework for measuring the production frontier performance of macro-R&D activities at the regional and national level. It’s increasingly important that enough attention should be paid to the production frontier performance of macro-R&D activities underpinning the development of regional or national innovative capacity with increasing R&D investments. It’s true that if R&D resources are not used effectively, additional investment may be of little help in stimulating scientific and technological progress (Wang and Huang 2007). This results generated by the framework can be employed as an equal and comparative standard for screening and financing macro-R&D projects at the regional or national level.

Of course, the assessment of scientific research is an extremely delicate and sophisticated venture (Braun et al. 1985). With respect to the future work, it would be interesting and valuable to incorporate statistical noise and environmental factors into our framework, which can more effectively reflect the real production process of macro-R&D activities at the regional or national level.