1 Introduction

Technology Transfer Organisations (TTO) are important at the intersection between academia and industry, but there are, to date, no comprehensive approaches to their performance management. Consequently, this paper presents the development and initial use of a holistic, practice-based TTO model to improve the understanding of TTO performance. Despite recent findings that the institutional context and TTO resources (Cartaxo and Godinho 2017) and their mission statements (Fitzgerald and Cunningham 2016) influence performance, internal practices and external contextual factors have been largely neglected. The few exceptions are studies that consider external factors, such as the academic field (Rogers et al. 2000) or stakeholders (Siegel et al. 2003a) when determining TTO effectiveness or efficiency (Anderson et al. 2007).

It is clear that the nature of TTO practices needs to be better documented and understood (Siegel et al. 2003b) in view of the rising number of TTO offices as a result from legislation such as the Bayh-Dole Act (Link and van Hasselt 2019). Commercialisation practices are the “set of activities performed by TTO staff in order to fulfil the TTO’s role” (Weckowska 2015, p. 63). They differ as a function of the activities that the TTO is involved in, ranging from intellectual property creation in the form of patents and licenses, to academic entrepreneurship, and the creation of start-ups and spin-offs (Benneworth and Jongbloed 2010; Perkmann et al. 2013). Although TTO managerial practices have been studied in the context of specific activities, such as collaborative university–industry research projects (Morandi 2013) or the distribution of licensing royalties to academics (Link and Siegel 2005), to the best of our knowledge, there have been no attempts to carry out a holistic analysis; in the innovation context, the TTO remains a black box (Bercovitz and Feldman 2006).

Here, we address this issue by focusing on the role of practices in TTO performance. We seek to answer the following research question: “What is the impact of TTO practices on other dimensions of TTO performance? The objective is to find out whether there is a link between the maturity of managerial practices and resources and competences as well as organisational context, on the one hand, and outputs and outcomes, on the other hand. This study builds on previous work which hypothesised that TTO managerial practices may be an important determinant of performance (Link and Siegel 2005; Siegel et al. 2003b) and responds to the need expressed by Alexander and Martin (2013, p.46) “to develop a better understanding of the inter-dependencies within the knowledge transfer activities and how these interrelationships relate to economic prosperity.” Consequently, the aim of the study is to improve our understanding of TTO performance in three ways: by taking stock of current TTO practices; by identifying TTO outcomes for impact metrics; and by enabling TTOs to holistically assess their performance with respect to internal and external parameters.

Our conceptual framework draws upon the TTO literature, as well as program management, intellectual capital and organisational capabilities. It consists of TTO performance dimensions that are derived from the IC-dVal (Intellectual Capital dynamic Value) framework (Bounfour 2003), while the dynamic capabilities (Teece 2007; Teece et al. 1997) and the Logic Model (McCawley 2001; McLaughlin and Jordan 1999) are also integrated. The IC-dVal framework is the foundation for an empirical model that was used in a survey of European TTOs. TTO managers were asked to determine the maturity of their practices, document their organisation’s current situation and identify areas for improvement. The survey is intended to be used as a strategic management tool, and responds to the longstanding need to support TTO managers in optimising their efforts (Souder et al. 1990) and documenting organisational practices (Siegel et al. 2003b).

The tool was developed and deployed in several steps according to which this paper is structured. First, a conceptual framework is derived from the review of literature (Sect. 2). The method consisted of model development (Sect. 3.1), measurement of practice maturity (Sect. 3.2), expert validation (Sect. 3.3) and characterisation of the TTO sample (Sect. 3.4). The findings from initial data collection are presented (Sect. 4) and discussed (Sect. 5). The paper concludes with implications for research and practice, limitations and opportunities for future research (Sect. 6).

2 Literature review

The literature review is structured in two parts. First, TTO performance studies are reviewed to understand current approaches to TTO performance measurement (Sect. 2.1). Second, a conceptual framework for TTO performance management is created (Sect. 2.2), which forms the basis for the development and initial use of a holistic, practice-based TTO maturity model.

2.1 Five characteristics of TTO performance studies

The conception and holistic measurement of TTO performance is in its infancy, despite increasing interest from policymakers in evaluation and TTO managers in strategic decision-making tools. Current studies share five characteristics.

First, TTO commercialisation activities have been used as performance parameters, mostly in isolation. See, for example, TTO studies on royalty sharing (Arqué-Castells et al. 2016), patenting (Baldini et al. 2006), spin-off creation (Nosella and Grimaldi 2009), marketing procedures (Thursby et al. 2001), IP management (Siegel et al. 2004), etc. Despite the merits of these studies, focussing on an overly-narrow range of activities and impacts might not only limit the ability to draw representative conclusions, but also compromise the comparability of TTO performance across organisations, even leading to undesirable incentives (Rossi and Rosli 2015).

Second, the economic value dimensions has dominated TTO performance metrics, despite the importance of social, non-monetary objectives (Bianchi et al. 2013). In fact, recent work highlights that open innovation intermediaries generate both external and internal value, during their involvement in collaborative innovation (De Silva et al. 2018). This underlines the importance of a more comprehensive approach.

Third, studies have been conducted within the confines of national boundaries. Examples include Portugal (Cartaxo and Godinho 2017), Spain (Caldera and Debande 2010), the United States (Baglieri et al. 2018; Rogers et al. 2000), the United Kingdom (Chapple et al. 2005), the Netherlands (Bekkers and Bodas-Freitas 2010; Vinig and Lips 2015), Italy (Bigliardi et al. 2015), Ireland (Fitzgerald and Cunningham 2016), Taiwan (Hsu et al. 2015) and South Africa (Alessandrini et al. 2013). Although these studies clearly make valuable contributions to policy and studying TTO performance in their respective countries, their findings are shaped by the idiosyncrasies of the national context. This precludes wider comparisons and transferability beyond the confines of national borders.

Fourth, studies of productivity, economic efficiency and effectiveness have dominated TTO performance research since the 1990s (Anderson et al. 2007; Besson et al. 2012; Chapple et al. 2005; Kim et al. 2008; Rogers et al. 2000; Secundo et al. 2016).

Fifth, existing TTO maturity models focus on organisational processes. Examples include innoSPICE (Besson et al. 2012) and the Toolbox for ICT Technology Transfer Professionals (Aguirre and Bobelyn 2011).

2.2 A framework for holistic TTO performance management

The proposed conceptual framework consists of five TTO performance dimensions that originate largely from extant literature on intellectual capital (IC) management, organisational capabilities and program management. IC management has been studied with dedicated maturity models applied to university TTOs, and the findings suggest that IC can be utilised to manage and improve their efficiency (Secundo et al. 2017). Consequently, we develop the dimensions ‘resources and competences’ (Sect. 2.2.1), ‘generic practices’ (Sect. 2.2.3) and ‘outputs’ (Sect. 2.2.4) following the IC-dVal framework (Intellectual Capital dynamic Value), notably its definition of inputs, intangible assets and outputs (Bounfour 2003).

The Logic Model is a tool that is used in program management to evaluate performance (McCawley 2001; McLaughlin and Jordan 1999) as well as in an innovation study, when conceptualising a university–business ecosystem (Galan-Muros and Davey 2017). However, it has also been adopted in an innovation study, when conceptualising a university–business ecosystem (Galan-Muros and Davey 2017). The dimension ‘organisational context’ (Sect. 2.2.2) draws upon the environmental and organisational factors found in this model. Moreover, it consists of activities which are reflected in ‘generic practices’ (Sect. 2.2.3) as well as outputs and outcomes dimensions (2.2.4) in the framework.

‘Organisational capabilities’ is an umbrella concept that refers to the knowledge, skills and experience of an organisation (Richardson 1972). It is important to consider this dimension because this study focuses on the role of TTO practices. Consequently, the ‘generic practices’ dimension (Sect. 2.2.3) is based on this theoretical pillar. In particular, dynamic capabilities can be linked with organisational performance (Teece 2007) and may improve, decay or stay the same over time (Easterby-Smith et al. 2009). Moreover, it has been found that they can have significant commonalities across firms, often referred to as ‘best practices’ (Eisenhardt and Martin 2000).

2.2.1 Resources and competences

Resources and competences are key inputs that determine TTO performance. They are important to consider as it has been shown, for example, that the ability to identify licensees is positively related to innovation speed, i.e. commercialisation time (Markman et al. 2005). Resources have been more broadly defined as “tangible and intangible assets a firm uses to choose and implement its strategies” (Barney 2001, p. 54). Applied to a TTO, it is fundamentally important to know how it functions (academic entrepreneurship, business development, IP creation, collaborative projects), its resources (labour force/full-time equivalents) and the prior experience of staff (Perkmann et al. 2013). Khadhraoui et al. (2017) highlight the importance of marketing and negotiation skills. The latter finding, in turn, underlines the point made by Siegel (2018), namely that the change in emphasis from IP protection to technology commercialisation and entrepreneurship has resulted in a change in the expertise required of TTO staff, which has broadened from a focus on legal skills to commercialisation skills. Competences, knowledge and skills are elements that comprise human capital—a key component of an organisation’s IC (Bounfour 2009, 2011). In a recent study of TTOs in Argentina, Becerra et al. (2019) analysed four TTO core competences in terms of governance type and proposed transfer channel. Finally, Katzy et al. (2013) refer to TTO competences as ‘strategic innovation capabilities’, and identify ‘matchmaking and innovation process design’, ‘management of collaborative projects’ and ‘project valuation and portfolio management’ as examples.

2.2.2 Organisational context

The organisational context is key to the effectiveness and success of TTOs. For example, Debackere (2018) found that proximity to the science base was an important success factor, which explains why most TTOs are embedded within universities. Contextual elements are fixed factors that cannot be changed by managers (Galan-Muros and Davey 2017). They are external and internal to the organisation and, clearly, both matter for effective technology transfer operations (Debackere 2018). The former include the regional and institutional environment (Munir 2002), such as industrial and academic characteristics. Other factors include the scientific domain and type of public research organisation, along with key regional economic sectors and industries. According to Schoen et al. (2014), internal factors are TTO age, its size, control structures, legal form, type of institutions served, governance structures and areas of activity.

2.2.3 Generic practices

The generic practices (GP) dimension is the most detailed, given this study’s focus on the impact of TTO practices on other dimensions of performance, and the lack of empirical and conceptual knowledge. As Siegel et al. (2003b, p. 36) point out: “there are no existing data on UITT [University-Industry Technology Transfer] organisational practices, nor is it even precisely clear what needs to be measured.” Therefore, there is a need to develop novel associations between commercialisation practices, i.e. activities by TTO staff to fulfil its role (Weckowska 2015), and TTO capabilities. This observation led to the identification of six, specific practices each corresponding to a distinct TTO capability. Table 1 provides an overview, and each generic practice is presented in detail in the sub-sections that follow.

Table 1 Overview of TTO capabilities and generic practice areas
Full size table

GP3.1 Sensing and seizing opportunities

Sensing and seizing opportunities draws upon the dynamic capability framework’s notions of ‘sensing and shaping’ and ‘seizing’ opportunities (Teece 2007). Dynamic capabilities originate in the business and their performance extends over the long term, as they are essential in facilitating the creation, deployment and protection of intangible assets (Teece et al. 1997). The ability to identify knowledge transfer opportunities in both academia and industry, on the one hand, and act upon them, on the other hand, is what we refer to as TTO intelligence capabilities. They include the capacity to produce inventions with commercial potential (Weckowska 2015), in the form of invention disclosures, and the ability to identify the needs of industry. This idea is encapsulated in the role of the ‘disseminator’, a person who ensures user awareness of technologies and counsels users regarding their needs (Souder et al. 1990).

GP3.2 Boundary spanning

Boundary spanning in the context of university–industry technology transfer refers to “actions (…) to serve as a bridge between ‘customers’ (entrepreneurs/firms) and ‘suppliers’ (scientists), who operate in distinctly different environments” (Siegel et al. 2003b, p. 45). The aim is to ensure that needs, capabilities and interests are communicated between parties, and tailored to the needs of respective stakeholders, creating alliances between previously-disconnected parties. This requires what we term cross-fertilizing capabilities to facilitate innovation projects between disparate actors and span boundaries (Cross and Prusak 2002). In the context of TTOs, direct and indirect boundary work (O’Kane 2018), ‘boundary spanning’ as a TTO function (Comacchio et al. 2012), organisational role (Awazu 2004; Cross and Prusak 2002; Santoro et al. 2006) and core competence (Alexander and Martin 2013) have been discussed.

GP3.3 Translation and combination

Mediation between academic, industry and administrative actors is a prerequisite for the deployment of cross-fertilizing capabilities when acting as a boundary spanner (Weckowska 2015). Innovation intermediaries connect previously-unconnected actors across a structural hole (Burt 2009) or bridge the ‘managerial’ (Bessant and Rush 1995) or ‘cultural’ gap (Siegel et al. 2003a). This requires matching capabilities that seek to ensure mutual understanding and compatible capabilities and assets (Teece 2007). Here, TTOs act as a central connector and gatekeeper (Awazu 2004), or a broker when activating “direct contacts among unrelated organisations” (Comacchio et al. 2012, p. 947).

GP3.4 Co-creation and development

Active involvement in knowledge and technology transfer innovation processes requires platformic bundling capabilities designed to shape innovation and knowledge outcomes (De Silva et al. 2018); examples include narrating, storytelling and agenda setting (Yström and Aspenberg 2017). This capability consists in reconfiguring assets (Teece 2007), excellent knowledge exploitation capabilities, greater integration flexibility (Awazu 2004), and acting as an information broker (Cross and Prusak 2002) or consultant (Bessant and Rush 1995). Katzy et al. (2013) stress the importance of acting as process managers in co-creation and economic exchange, while Trencher et al. (2013) point out the academic function of ‘co-creation for sustainability’, which can be applied to classical, university TTOs.

GP3.5 Cultural change management

TTOs enable the formation of projects and collaborations between academics and practitioners from different organisational cultures. Getting these two groups to work together is challenging and requires wearing two hats: scientific and business (O’Kane et al. 2015). Moreover, TTOs need to be able to change mindsets, which means, on the one hand, commercialising university research and, on the other hand, the absorption of external knowledge by industry. It requires constructing a sense of identity (Ollila and Yström 2015) and the provision of “knowledge sharing and support services to enterprises” (Alexander and Martin 2013). Ollila and Yström (2015) point out that this relates to encouraging the use of dialogical practices to encourage others to talk or act in different ways.

GP3.6 Knowledge management

The knowledge based-view of the firm emphasises the management of the organisation’s own knowledge base (Santoro et al. 2006). This means leveraging and combining existing knowledge, acquiring and absorbing new knowledge, and accessing unrelated knowledge for the creation of internal value. De Silva et al. (2018) studied the influence of knowledge-based practices on internal value creation in innovation intermediaries. They found that such practices capitalise on internal and external human resources, such as employees or collaborators. In doing so, they generate both financial and non-financial value, in the form of knowledge, market and network-based benefits.

2.2.4 Impact

The Cambridge Dictionary defines the term ‘impact’ as “to have an influence on something”. In the context of the organisation, this implies that activities result in a tangible or intangible change. The term tends to be used as an umbrella concept as the next step is to determine who or what is influenced and when. In terms of the latter, impacts can be divided into short and long term. The former refers to outputs, which are measurable results of the organisation’s activities in the short term (McCawley 2001). They include tangible outputs such as spin-off companies and licenses Established metrics include tangible outputs such as invention disclosures received, number of industry agreements and patents, IP agreements in force, and the number of start-ups or spin-offs created (Rasmussen et al. 2006). They have been used in both econometric TTO performance studies (Anderson et al. 2007; Besson et al. 2012; Chapple et al. 2005; Kim et al. 2008; Rogers et al. 2000; Secundo et al. 2016, Siegel et al. 2003a), and national and transnational TTO performance surveys. Examples of TTO impact metrics include those developed and deployed by the professional associations of science and technology transfer professionals in Europe (ASTP) and in the United States (AUTM).

Long-term impacts are referred to as outcomes, which also play an important role in TTO value creation and can be, unlike measurable short-term outputs, either tangible or intangible (Nogeste and Walker 2005). Here, we differentiate between two forms: stakeholder impact (Lerro 2011) and value creation with societal impact (Benneworth and Jongbloed 2010), also referred to as the “triple bottom line” (Elkington 1998). Stakeholders can be divided into academia, industry, policymakers and the broader public, while TTOs create economic, social and environmental value.

A summary of key references of the conceptual framework can be found in Table 11 in the “Appendix”.

3 Methodology

For the model to be deployed in TTOs, it was operationalised by a survey. In line with the research focus, i.e. the role of practices on TTO performance dimensions. The study consisted of exploratory and explanatory approaches. For the constructions of the conceptual framework, an exploratory approach was adopted. In an inductive process, differing views, drawn from current theories were synthesised into concepts which are then joined together in a conceptual framework (Imenda 2014). On the other hand, the survey adopted an explanatory approach, as it enabled the collection and analysis of data, with the aim of identifying causal relationships between variables (Saunders et al. 2009).

The following sections provide detail about the creation of the practice-based TTO maturity model. Particularly two phases are presented in greater detail which were crucial to our model and diverged from the typical development phases, namely: the definition of maturity levels in practice areas (Sect. 3.2); and the validation by TTO experts (Sect. 3.3). The latter, in particular, led to the revision of the initial conceptual framework (Miles and Huberman 1994). These two phases are presented in depth as the study’s two objectives are to take stock of the state of TTO practices and enable TTOs to holistically assess their performance.

3.1 Model development

The model was developed in six steps, shown in Table 2. Steps one to four are briefly described in the paragraph that follows, they refer to the development of the framework presented in Sect. 2.2. Steps five and six are described in detail in Sects. 3.2 and 3.3 respectively.

Table 2 Development process of the practice-based TTO maturity model
Full size table

Steps one and two (the identification of suitable texts and data sources, reading and categorisation) corresponds to the performed literature review, which resulted in the framework presented in Sect. 2.2. Ideally, we would have followed the steps outlined in Jabareen (2009), namely conducting initial interviews with practitioners and scholars. This, however, was not possible owing to time constraints and the lack of a conceptual framework at the time. The identification of concepts and their categorisation was the subject of phases three and four. In line with Jabareen (2009), most concepts emerged from the literature. This was particularly the case for generic practices (dimension 3), which resulted in the creation of six specific TTO practice areas (see Sect. 2.2.3).

3.2 Measuring the maturity of practices

The organisational maturity of TTO practices is assessed in the generic practice dimension. Organisational maturity is, according to Bititci et al. (2015, p. 3025), “a matrix of practices that define, for each organisational area the level of formality, sophistication and embeddedness of practices from ad hoc to optimising”. Measuring practice maturity is inherently difficult, a key challenge is that “practices are a complex bundle involving social, material, and embodied ways of doing that are interrelated and not always articulated or conscious to the actor involved in doing” (Jarzabkowski and Paul Spee 2009, p. 82). Consequently, for practice maturity assessment, ways had to be found that TTO managers could indicate the maturity that corresponds with each practice.

The maturity levels were adapted from Bounfour (2011), as the latter were deemed to be more suitable in this context than those proposed by Bititci et al. (2015). However, the descriptions in Bounfour (2011) were defined for the assessment of organisational capital, and thus had to be adapted to suit the TTO context. Furthermore, the definition of each maturity level had to be worded in a way that could be understood and operationalised by TTO managers, the ultimate users of the survey. Furthermore, each maturity level had to be translated into a score that could, at the data analysis stage, be transformed into a numerical value, i.e. score. The Table 3 shows the five maturity levels, their respective descriptions and survey wording, together with their respective score.

Table 3 Maturity levels for the assessment of generic practices (
Full size table

In a second step, each generic practice was associated with statements that either originated directly from the literature or were adapted based on practitioner feedback during validation (see next section). Each practice statement was allocated a unique identifier that consisted of a three-digit number beginning with three (corresponding to the third dimension of the model).

As Table 4 shows, the number of statements per practice area varied. In particular ‘co-creation & development’ was only associated with three statements, compared to 15 for ‘knowledge management’. The other practice areas are close to the average of 7.3 practice statements per practice area. It is worth noting that although we initially attempted to arrive at a fairly even distribution of statements, validation by TTO practitioners revealed this did not accurately reflect reality. We therefore decided to retain only those statements that made sense to practitioners. This iterative process led to the creation of the final version of the statements included in the survey which are shown in Table 4.

Table 4 Detailed generic practice statements (model dimension 3)
Full size table

3.3 Expert validation and creation of an online survey

The conceptual framework was presented at an annual event of European technology transfer practitioners in autumn 2017. It became clear that the framework not only needed refinement, but also to be turned into a format that could be used by practitioners. Both aspects were achieved with the creation of an online survey which was validated using practitioner feedback from individual interviews with TTO managers.

The final version of the survey consisted of 26 questions spanning the model’s five dimensions. Open, closed and scored questions were included, although the latter was solely used for the generic practice dimension (dimension 3). Each generic practice area (e.g. ‘sensing & seizing opportunities’) was counted as one question, although three to fifteen individual practice statements were scored, as described in the previous section. Closed questions were used for ‘resources and competences’ (dimension 1), ‘organisational context’ (dimension 2) and ‘outputs’ (dimension 4). Finally, open questions were used to measure dimension 5 (outcomes). These questions evaluated the impact on different types of stakeholders, namely academia, industry, policymakers, the public sector and the general public (question 25), as well as economic, social and environmental value creation (question 26). This approach was taken because, unlike TTO output, there are no established metrics to measure TTO impact.

The formulation of questions and practice statements was validated with practitioners in four rounds of validation conducted between January and March 2018. TTO managers were interviewed and asked to go through the latest version of the survey—some even started to fill it into see if the questions made sense to them. Through this process of participant observation and feedback, both the concepts and their phrasing were jointly improved with practitioners.

Figure 1 shows the different practitioner groups that were involved in the process.

Fig. 1
figure 1

Survey validation, the iterative finalisation process

Full size image

The initial survey was validated with a Hungarian TTO team. This led to a revised version, which a Portuguese TTO manager validated thereby creating a revised version of the survey that a Spanish TTO director revised in the third validation round. The fourth round took place with a Norwegian TTO manager who wanted to start using it. Some small updates were made, resulting in the final version of the survey. The order in which practitioners participated was based on their availability, and they all completed the final survey.

Practitioner feedback addressed various levels—not only the formulation of questions and terminology but also the model itself. The most fundamental change to the conceptual framework concerned the deletion of the former practice dimension ‘managerial leadership’, which was replaced by ‘knowledge management’ practices (GP3.6). This change was motivated by concerns about possible bias regarding the assessment of managerial leadership that emerged from the first two rounds of validation, while knowledge management was found to be lacking. Please see Table 10 in the “Appendix” for details of the changes made to the survey after each round of validation. Even though the number of remarks and types of feedback varied between the four practitioner groups, in general, the number and extent of changes diminished with each round. Table 5 shows the final list of survey questions.

Table 5 List of the 26 final survey questions
Full size table

The statements making up questions 15–20 in dimension 3 (generic practices) are shown in Table 4.

3.4 Characteristics of the TTO sample

The final online survey was shared with European TTO managers from the authors’ personal network. Seventeen TTOs from 11 countries responded, namely: Norway (2), Ireland (1), Spain (1), the Netherlands (2), Portugal (1), Hungary (1), Lithuania (1), Belgium (3), Germany (1), the Czech Republic (1) and France (3). At the request of the majority of participants, they remained anonymous and were instead assigned a letter (between A to Q). The organisational context (dimension 2) concerns the internal and external structural characteristics of TTOs and highlighted the diversity of the sample, as shown in Table 6.

Table 6 Overview of the TTO sample
Full size table

Participants were free to select the reporting year, depending on the most recent available data. In practice, the majority (12) had access to data from 2017, while five reported on 2016 data.

Three TTOs can be considered ‘old’ as they were established before 2000, nine are ‘young’ (founded after 2009), while five came into existence between 2000–2009. Five had existed in the form of another organisation before their creation, four of these were created after 2009. Average size, measured as full-time equivalent employees (FTEs) was 14.8, ranging from 32 to one.

The ‘classical’ TTO type dominates the dataset. Eight TTOs exclusively serve one institution and are integrated into its administrative structure. Five are ‘discipline-integrated alliances’, indicating that they serve several institutions working in various disciplines. Four are autonomous or a discipline-specialised alliance (two for each type). In terms of innovation activities, the majority (eight) are active at the local, national and international levels. Two act solely at the local level, while there is one each at national and international level.

The most popular activity is ‘IP management’, which all 17 TTOs are involved in, followed by ‘business development’ (14 TTOs). Eight are involved in all four TTO activities namely: Academic entrepreneurship (spin-off/start-up creation/training, etc.); business development; IP management (licenses or patents); and joint/collaborative projects. Additional activities are: consultancy, contract review, IP teaching, knowledge transfer training, technological maturation, management of shared facilities, campus development and running a science shop.

4 Results

4.1 The holistic, practice-based TTO maturity model

The holistic, practice-based TTO maturity model consists of performance dimensions, which are drawn from the conceptual framework that is, itself, based on the literature on the management of programs, IC and organisational capabilities (see Sect. 2.2). Figure 2 shows an overview of the model.

Fig. 2
figure 2

The holistic, practice-based TTO maturity model

Full size image

The proposed model is descriptive and consists of three types of variables: dependent variables representing TTO impact that consist of outputs and outcomes; independent variables that represent resources and the organisational context; and generic practices as mediating variables. They reflect the study’s objective, which is to identify if there is a link between the maturity of TTO managerial practices and the organisation’s resources and context, on the one hand, and outputs and outcomes, on the other hand.

4.2 Findings from initial data collection

With the data from the TTO sample (see Sect. 3.4.), three analyses were run to determine: the state of TTO practices (Sect. 4.1); identify the relation between practices and performance (Sect. 4.2); and analyse outcomes (Sect. 4.3).

4.2.1 The state of TTO practices

To assess the state of TTO practices, dimension 3 (generic practices) was designed to evaluate their maturity with respect to six capabilities. Fifteen of the 17 respondents in our sample scored all 44 practice statements with regards to their respective organisation. Maturity scores for each generic practice are shown in Table 7, in which the 15 TTOs are sorted in descending order based on their total maturity score (last column) across all six areas.

Table 7 Practice-area maturity scores per TTO—grouped for analysis
Full size table

The practice area maturity scores for each TTO are presented as a percentage of the maximum number of points that could be awarded in each area (this differed with the number of questions per practice area), and the maturity score that the TTO allocated to each practice. As shown in Table 3 (Sect. 3.2), scores could range from zero (non-existent) to four (fully deployed).

For example, the figure of 75% for TTO-H in the practice area ‘boundary spanning’ indicates that it scored 18 points (75% of 24 possible points) for the statements in this practice area. The actual score is not as useful as the relative score, since the latter allows comparisons across practice areas. On average, relative scores range between 53 and 70%. The heatmap shows trends across the full dataset. Across the 15 TTOs, maturity scores for ‘co-creation & development’, ‘cultural change management’ and ‘knowledge management’ were, on average, 53%, 56% and 54% respectively, and lower than ‘sensing & seizing opportunities’, ‘boundary spanning’ and ‘translation & combination’, which scored 70%, 61% and 66% respectively.

If we focus on relative scores for each practice area, a diverse picture emerges, which is shown in Fig. 3.

Fig. 3
figure 3

Range of maturity scores in TTO practice areas

Full size image

Figure 3 gives an indication of practice maturity in our sample of TTOs. It shows, on the one hand, that the range of scores for ‘boundary spanning’ and ‘knowledge management’ were similar, 42% and 47% respectively, indicating that scores for ‘cross-fertilizing’ and ‘managing the knowledge base’ capabilities were also similar. On the other hand, scores for ‘co-creation & development’ were more varied: three TTOs scored below 20% (the lowest overall score), while four scored above 80% (one reaching 100%, the highest score in the entire dataset).

4.2.2 The relation between practices and performance

To investigate the influence of TTO practices on organisational performance, the 15 TTOs that scored all practice statements were categorised as a function of their total maturity score. This identified three groups and two outliers: TTO-H had the highest relative maturity score (81%) across all dimensions, while TTO-F had the lowest (37%). The remaining TTOs were allocated to three groups: group 1 (five TTOs) had relative overall maturity scores of 69–73%; group 2 (three TTOs) scored between 57 and 65%; and group 3 (five TTOs) scored between 45 and 52%. Respective absolute and relative scores for each participating TTO in Table 7.

This analysis is based on the model’s other performance dimensions. Each of the three groups and two outliers is reported on separately to investigate whether the maturity of their practices influences these performance dimensions. Particular attention is paid to TTO-H and TTO-F.

In terms of resources and competences (dimension 1), it appears that staff background and practice maturity are linked: TTOs with more staff with a private sector background have higher maturity scores. This is notably the case for TTO-H, where 13 of its 15 FTEs have a private sector background, and group 1, where 60% of staff fall into this category. Furthermore, it is worth noting that half of the 15 FTEs in TTO-H work on ‘academic entrepreneurship’, while the work of other groups is more balanced and most staff work on ‘IP management’. The results for ‘innovation capabilities’ are inconclusive.

Results for the external organisational context for the three groups and two outliers were diverse. Overall, the life sciences play an important role for most TTOs (except those in group 2). In terms of industry sector, the results show that the secondary sector (manufacturing/construction of finished products) is popular among TTO-H, and groups 1 and 2. In addition, the quaternary sector (i.e. intellectual services: education, training, the development of technology, and research and development), is important for most TTOs in the sample. However, this observation applies primarily to those with a lower maturity score, such as TTO-F and group 3.

The analysis of data regarding the internal organisational context reveals a similarity between high and low outliers. Both TTO-H and TTO-F are solely active at the local level, while TTOs in the three groups are at least as active in two of the three levels (international, national and local). Although most TTOs (11) had not emerged from another organisation, the two with the highest maturity scores had (TTO-H and TTO-O). While both of these TTOs can be considered ‘young’ and engage in ‘discipline-integrated alliances’, overall, ‘TTO age’ and ‘TTO type’ are diverse and no conclusive relation was found with maturity.

Regarding outputs, TTO-F and TTO-H (organisations with the highest and lowest maturity scores) had both created 10 licenses, options or assignments in the reporting period. This is below the group 1 average of 15. Overall, no relation was found between maturity and this dimension of performance.

4.2.3 TTO outcomes

TTO outcomes (dimension 5) are reported in terms of stakeholder impact (Lerro 2011) and value creation (Benneworth and Jongbloed 2010), both of which are related to the impact of the TTO in the long term. As qualitative data was collected in open questions (see Table 5, Sect. 3.3), the comparison of the three groups and two outliers (as in the previous section) was not deemed appropriate. Instead, responses were structured into meta-categories derived from the Sustainable Development Goals (SDGs) adopted in 2015 by all United Nations Member States in the context of the 2030 Agenda for Sustainable Development.

In terms of stakeholder impact, meta-categories were broadly similar for academic and industry stakeholders, and policy, institutional and general societal stakeholders. This is shown in Table 8, which summarises responses from participating TTOs.

Table 8 Overview of outcomes regarding stakeholder impact
Full size table

Unlike most TTOs which responded separately to each stakeholder group, two TTOs provided answers based on concrete examples of spin-out companies. This was a particularly effective way to highlight impacts on different stakeholder groups. For example, TTO-B reported on a newly-created company in the area of bees and hive health. Academic stakeholders profited from improved knowledge on bee diseases for the development of new treatments. Industry was able to access the technology via license agreements, and develop a product pipeline. For policymakers, these products, which ensure bee health, contribute directly and indirectly to the economy—the former in the form of production (wax, honey, etc.) and the latter in the form of ecosystem services due to pollination. Society benefits from the availability of these direct and indirect products, and the sustainability of natural ecosystems.

The analysis found that the long-term impact of TTOs on academic and industry stakeholders is similar for competitive advantage, finance, marketing, productivity and quality. The details vary for each group; for example, for the category ‘competitive advantage’ academics benefit from better access to consultancy projects (TTO-E) and opportunities to take on scientific advisory roles (TTO-H), or promote the value of IP internally (TTO-J). Industrial stakeholders benefit from access to new knowledge (TTO-A and TTO-P) or specialised equipment (TTO-J). License deals can provide them with a first mover advantage in new product development (TTO-D). By using these opportunities to their advantage, industrial stakeholders can position themselves for their future survival in a changing world (TTO-A).

The long-term impacts off TTOs for policy and institutional stakeholders, and society at large are, as for academic and industrial stakeholders, in similar areas. In terms of ‘health and well-being’ (SDG 3), TTO-D gives examples of the development of new cancer treatments that are being adopted across Europe. Also, TTO project outcomes can result in access to new opportunities for humankind (TTO-H), which policymakers and society at large value. The health of the general public can be impacted positively by improved knowledge of diseases (TTO-H), the availability of safer and more effective cancer treatments (TTO-D), more convenient treatment regimes for patients (TTO-K), the development of new drugs, and the creation of social policies that focus on the needs of the population (TTO-J).

In terms of value creation from their activities, responding TTOs provided a wide range of examples, which are summarised in Table 9.

Table 9 Overview of outcomes regarding value creation
Full size table

In terms of economic value, TTOs I and M provided examples of projects that resulted in the creation of employment, either through new companies or the hiring of new talent in existing firms. Other benefits included a return-to-work program for disadvantaged young people (TTO-M) and greater interest in returning to obtain an education (TTO-A). TTO-J reported improved efficiency of public institutions, while TTO-A noted the increased attractiveness of education as a long-term outcome of their work. TTO-A observed improved productivity (in the oil sector). Other outcomes included more secure and fairer transactions through new blockchain technology, and better security in chatrooms to avoid pestering and harassment. Several TTOs highlighted the improved competitiveness of industry partners (TTO-E, H, P) or entire sectors (TTO-J) as a result of technology transfer projects.

Social value was created by projects on language preservation, the empowerment of women and training minorities in the use of information and communications technology (TTO-A), along with social integration of migrant populations and encouraging young people to become politically aware (TTO-J). Value, in terms of improved well-being, was generated in the form of global patient benefits (TTO-K), and the use of new ‘internet of things’ technologies in cities (TTO-N). Given that TTOs often occupy the role of ‘middlemen’, a key social value created by their work lies in the connection of people. This manifests directly in the creation of a community of researchers and industry employees (TTO-H), and indirectly in the provision of internet access to rural areas (TTO-I). An overall outcome is to enable collaborations between policymakers, citizens and scientists (TTO-J).

Regarding environmental value, TTOs reported on projects to encourage urban farming and improve air quality (TTO-J), CO2 recovery (TTO-M) and the disinfection of water and air (TTO-A). Project outcomes have resulted in energy savings (TTO-H), alternative energy production solutions (TTO-M) and the enforcement of a local, circular economy (TTO-J). In terms of conservation and biodiversity, the spin-off company at TTO-B aims to ensure pollination as a vital ecosystem service for food security, which is similar to the project launched by TTO-J to preserve bee diversity. Finally, TTO-M reports on a chemistry project to improve production yields while reducing reaction temperatures required for production.

5 Discussion

5.1 Model novelty—TTO performance dimensions and maturity of practices

The proposed holistic approach to measuring TTO performance has several novel elements, which extend and complement extant studies in terms of TTO performance dimensions and practices. First, it provides a more detailed and comprehensive evaluation of resources and competences, the organisational context, and impacts than earlier TTO performance studies that only consider a narrow range of TTO activities and impacts (Arqué-Castells et al. 2016; Baldini et al. 2006; Nosella and Grimaldi 2009; Siegel et al. 2004; Thursby et al. 2001).

Second, and consistent with Bianchi et al. (2013), the proposed holistic approach highlights the importance of social or non-monetary value dimensions that seeks to balance the prevailing focus on economic value creation. The inclusion of knowledge management, in the form of concrete, ‘knowledge-based practices’ responds to the need identified by De Silva et al. (2018).

Third, the proposed model and the online survey were developed in collaboration with TTOs from across Europe. This means that their use is not confined to a national border unlike most, previous TTO performance studies (e.g. Bigliardi et al. 2015; Cartaxo and Godinho 2017; Chapple et al. 2005).

Fourth, ‘generic practices’ is the dimension at the heart of the proposed model, and give it its name: the ‘practice-based maturity model’. The measurement of the maturity of practices is operationalised in concrete statements (Table 4). Unlike previous research, we did not study individual TTO managerial practices with respect to specific activities (Link and Siegel 2005; Morandi 2013), but instead developed a comprehensive list of practices across six TTO capabilities (Table 1). This responded to the observation noted by Siegel et al. (2003b, p. 45) that “there is a need to simply document the nature of these practices”.

Fifth, consistent with Eisenhardt and Martin (2000), the proposed framework emphasises that TTO practices and capabilities are linked. This is, to the best of our knowledge, a new development as extant TTO maturity models tend to focus on processes (Aguirre and Bobelyn 2011; Besson et al. 2012) rather than practices.

5.2 Insights from initial model use

The conceptual framework (Sect. 2.2) was turned into an empirical model that was implemented in an online survey with which data was collected from 17 European TTOs. The analysis of the current state of TTO practices found that maturity was lower for ‘co-creation & development’, ‘cultural change management’ and ‘knowledge management’ than ‘sensing & seizing opportunities’, ‘boundary spanning’ and ‘translation & combination’. This suggests that the latter three practice areas are more established than the former. In terms of TTO capabilities, these results indicate that ‘TTO intelligence’, ‘cross-fertilizing’ and ‘matching’ capabilities are more established than ‘managing the knowledge base’, ‘platformic bundling’ and ‘changing the mindset’. The rather low maturity scores for ‘platformic bundling’ capabilities could be be explained by the relatively new notion that TTOs are actively involved in the transfer process, by acting as co-creation managers (Katzy et al. 2013), shaping innovation and knowledge outcomes (De Silva et al. 2018) and being involved in activities such as narrating, storytelling and agenda setting (Yström and Aspenberg 2017). Agogué et al. (2017) refer to this role as the “architect of the ecosystem”, a position that is intrinsically challenging, to manage given the high ambiguity and degree of unknown in which they operate; as well as the need to adapt practice to the specific context (Yström and Aspenberg 2017).

The findings regarding long-term outcomes show that the impact on academic and industry stakeholders is in the areas of competitive advantage, finance, marketing, productivity and quality. Long-term impacts on the general public, policy and public institutions are found in meta-areas that are in line with the following United Nation’s SDGs: health and well-being; quality education; work and economic growth; industry, innovation and infrastructure; and sustainable cities and communities. The use of the SDG framework is consistent with Schot and Steinmueller’s (2018) call for transformative change in socio-technical innovation systems to address the challenges laid out in the SDGs.

Two TTOs referred to impact studies to illustrate their influence on different stakeholder groups. This practice resembles the approach taken in the 2014 Research Excellence Framework of the Higher Education Funding Council for England, which collected 6975 case studies on the socioeconomic impact of university research (https://impact.ref.ac.uk/casestudies/).

Concerning economic value, TTOs report job creation, better job security and improved efficiency and productivity. Social value was created in the areas of culture and well-being, and the ability to connect people. Environmental value related to improved water and air quality, the conservation of energy, waste reduction, and the protection of biodiversity. These observations are in line with Benneworth and Jongbloed (2010, p. 569), who argue that “real impacts evolve in response to simultaneous changes across different scales”.

While most TTOs reported long-term outputs, some found this ‘hard to make concrete’, ‘not relevant’ or ‘not applicable’ (TTO-D, TTO-I). In particular, TTO-D stated, “we have to admit that this value creation is not that important so far to be detailed. Furthermore, our transfer results do not allow us to assess the different stakeholders impacts.” This comment sums up the challenges that TTOs are currently facing in terms of their visibility and ability to determine the long-term impact of their work. It can be linked to the lack of TTO outcome metrics (unlike output metrics), as the latter are readily available from national and transnational data collection initiatives, and are thus more often used in academic studies.

The wide range of answers relating to outcomes highlights that evidence is, so far, rather anecdotal. Furthermore, the relationship between practices and performance dimensions varied across dimensions, and it proved difficult to establish a robust relation between measured maturity and most performance dimensions. This leaves ample room for future research, which is addressed in the concluding section.

6 Conclusion

This study presents the development and initial use of a holistic, practice-based TTO maturity model that not only considers input and output metrics, but enables contextualising them. The model improves our understanding of TTO performance and facilitates its holistic measurement. Furthermore, it addresses the lack of conceptual and empirical approaches to TTO performance management, such as the ability to optimise transfer efforts (Souder et al. 1990), documentation of organisational practices (Siegel et al. 2003b) and the development of a better understanding of the inter-dependencies within the knowledge transfer activities (Alexander and Martin 2013).

Our model draws its theoretical foundations from intellectual capital and organisational capabilities, and the Logic Model. Its novelty arises from the positioning of TTO practices at the heart of TTO performance, linking it with internal and external dimensions—namely resources and competences, organisational context as well as outputs and outcomes. It therefore differs from existing TTO maturity models, which focus on organisational processes (Aguirre and Bobelyn 2011; Besson et al. 2012) or the evaluation of efficiency based on non-monetary indicators (Secundo et al. 2015, 2016). To the best of our knowledge, the formulation of TTO practice statements and the assessment of their maturity has not been done before. It enables managers to strategically manage their TTOs by stating the expected performance and defining how this is to be achieved. The model can not only assess the maturity of TTO practices, but also considers performance holistically by linking the state of TTO managerial practices with the organisation’s resources, competences and context, along with its outputs and outcomes.

In addition to advancing the assessment of the maturity of TTO practices, this work also marks a step forward in the development of TTO-specific outcome metrics. Overall, the model represents a comprehensive approach to TTO performance management—its development, initial use and continued adoption has implications for research and practice and points to areas for improvements, as well as opportunities for future research.

6.1 Implications for research and practice

The development and use of the proposed, holistic practice-based TTO maturity model can benefit both TTO practitioners and innovation researchers. It provides a concrete measure of the maturity of TTO practices and offers insights into their relationship with organisational and ecosystem parameters.

Our study contributes to the strategic management of TTOs, particularly the body of knowledge on TTO practices and the study of capabilities. First, it addresses the need for an improved understanding of TTO practices and their documentation (Siegel et al. 2003b), as it proposes specific statements to assess their maturity in six generic practice areas. These correspond to specific capabilities, which are adapted from dynamic capabilities (Teece 2007) and represent ‘best practices’ that are common across organisations (Eisenhardt and Martin 2000).

Capabilities are not only developed on the conceptual level, but also operationalised in the form of a survey. The survey is a structured way for TTO managers to assess and decide on areas for improvement, with a particular emphasis on the state of practices. As such, our model could serve as a managerial tool as it not only helps in assessing current practice maturity, but also supports setting targets for individual projects and processes. This strategic decision-making tool could improve organisational performance and become a building block for the strategic management of TTOs, as it enables TTOs to go about developing their practices strategically by means of roadmaps, etc.

The survey was used by a sample of 17 European TTOs in the study. To increase its outreach, its translation into a more widely-available online tool would be beneficial. The higher the adoption rate, the more researchers will have access to data to improve their understanding of the maturity of practices on TTO performance.

In the next development stage, each participating TTO is to receive a report of the information submitted. The aim is to help them to reflect on the status quo and inform their decision-making with respect to the development of selected practice areas.

6.2 Limitations and future research

Our study shows that TTO performance needs to be considered holistically. In terms of impacts, it goes beyond short-term outputs to include outcomes that have a more long-term horizon and are less established in standard TTO metrics today. Outputs and outcomes were consciously differentiated by separating them into two dimensions. Our analysis highlighted that more and better TTO impact metrics are needed. An important point to note is that findings in terms of TTO outcomes were qualitative and anecdotal, and their analysis in relation to the maturity of TTO practices was not feasible. To address this limitation, we believe that metrics should be jointly developed with TTO practitioners to ensure they can realistically be assessed. More work on this needs to be done on both the conceptual and empirical levels.

The involvement of TTO managers in the validation of the conceptual framework and the survey was a crucial step in ensuring that both concepts and the terminology were understood by practitioners. It proved to be a major challenge in this research, and hence we recommend that any future research that seeks to develop an empirical tool from a conceptual framework should involve practitioners at an early stage.

The model’s use is currently limited, given that the survey is only accessible via a proprietary link. However, it could well be distributed on a public, online platform. Hence, future research could focus on its roll-out to encourage broader adoption of the tool. For example, it could be included in professional qualifications, like the Registered Technology Transfer Professionals (RTTP) or other forms of technology transfer manager training. Furthermore, the promotion of the proposed model by national or transnational technology transfer associations would aid greatly in its establishment and adoption in the TTO community.

Insights from the initial use of the model cannot be considered sufficiently conclusive to unequivocally answer our research question on the impact of TTO practices on TTO performance. The proposed model remains descriptive and should be considered as a first step. With the data collected, initial insights suggest that there is a link between staff background and practice maturity which requires further research. Moreover, research could formulate hypotheses based on the model, in order to develop a theoretical foundation. This would add a prescriptive perspective to the maturity of TTO practices and help in understanding their influence on TTO performance. In sum, our model provides a basis for future research that can collect data to more comprehensively comprehend TTO performance.