Introduction

Research impact typically takes a long time to materialise. Often evaluation (especially when evaluating impacts) comes towards the end. Even where evaluators are accompanying the implementation of a research programme, societal outcomes—and the pathways to them—can be elusive. In this paper, we focus on opportunities for assessing the potential for impact at earlier stages in the research and innovation (R&I) process. We argue that such a focus can help both research programme managers and evaluators learn about the pre-conditions for impact and adjust accordingly.

Drawing on the evaluation of the Global Challenges Research Fund (GCRF), as an example of a large-scale Research for Development (R4D) evaluation, we identify and explore some of the building blocks that need to be in place to increase the likelihood of impact.Footnote 1 GCRF is a large-scale, nine-year fund, representing an unprecedented investment of £1.5bn of Official Development Assistance (ODA) into research and innovation for development, from 2016 to 2025 (BEIS 2017). The GCRF evaluation follows the implementation of the fund from 2020 to 2025, through a number of stages and evaluation modules (described further below).

Exploring the insights from an evaluation of a large-scale R4D fund such as GCRF offers R4D programme managers and evaluators lessons into how R4D initiatives can work to promote impact, and how to evaluate them in practice. GCRF as an R4D is fund is highly diverse and interdisciplinary, supporting arts and humanities research to space technology innovations for development. As such, GCRF’s theory of change (ToC) spans multiple pathways to impact and provides a fund-level opportunity to look across numerous types of projects and programmes to understand how R4D evaluators might assess the potential for impact at earlier stages in the R&I process. Given the scale of both GCRF and the evaluation, the GCRF evaluation is presented as a single case in this paper. The evaluation has now been running for three years and completed a number of modules, increasing the evidence and insights into GCRF’s portfolio of approximately 3,000 grants and 140 programmes.

In this paper, we draw on emerging evaluation evidence that highlights the importance of ways of working that support (i) alignment with development issues on the ground; (ii) fairness and mutual capacity building in partnerships between UK and low and middle income country (LMIC) organisations, including non-academic partners; (iii) a focus on gender, social inclusion and poverty reduction; and (iv) mobilising stakeholders networks for uptake. We conclude by drawing out a unifying construct around the standards of ‘development excellence’ to sit alongside notions of scientific excellence for research. We propose that by setting standards around the building blocks for development excellence, this can help programme managers, researchers and evaluators learn and adapt to increase the likelihood of impact.

Framing research impact in R4D

For a number of decades, there has been an increased focus on research impact, as governments and other funders of research seek evidence of the benefits to society. There is a lack of consensus in how to define and measure research impact (Reed et al. 2021; Alla et al. 2017), and while there is an extensive social science literature on research-policy relations, often a more narrow interpretation makes its way into the guidance and practice (Smit and Hessels, 2021; Guthrie et al. 2018; Boswell and Smith 2017; Cairney 2016; Boaz et al. 2009; Blume, 1977). The UK’s Research Excellence Framework (REF) 2014 and 2021 exemplifies the drive towards evaluating societal impact. REF provides a system for assessing the quality of research, which includes impact case studies comprising of 20% and 25% of total scores in 2014 and 2021, respectively—with funding linked to the REF outcome. While different models are adopted by funding councils, they tend towards an expectation that the researchers’ own efforts will achieve impact, and that this can be measured and documented. These models draw on a narrow interpretation that impact is achieved through policy makers adjusting their beliefs in response to research findings (Smit and Hessesl 2021; Boswell and Smith 2017; Cairney 2016). Such instrumental models of knowledge utilisation—where knowledge ‘drives’ policy or policy problems stimulate research to find solutions—rarely acknowledge the subjectivity of who benefits from research and how, as well as the messy complexity of the relationships between knowledge flows and policy networks (Smit and Hessels, 2021; Pinnington and Barnett, 2020; Georgalakis and Rose 2019; Cairney 2016).

GCRF is funded through the UK’s Official Development Assistance (ODA) commitment—the overseas aid budget—and this places additional expectations around development impact, as well as extra layers of scrutiny.Footnote 2 ODA funding must be used to deliver the UK’s aid strategyFootnote 3 through: (a) strengthening global peace, security and governance; (b) strengthening resilience and response to crises; (c) promoting global prosperity; and (d) tackling extreme poverty and helping the world’s most vulnerable. As such, ODA-funded research projects must comply with Organisation for Economic Cooperation and Development (OECD) definitions and official reporting requirements with the primary purpose of ODA being to bring benefit to a country (or countries) on the OECD Development Assistance Committee (DAC) list.Footnote 4 This places additional expectations around assessing and achieving research impact.

Indeed, GCRF follows the long tradition of Research for Development (R4D), i.e. research that addresses critical global challenges (such as funded by IDRC, the Swiss R4D programme, the Ecosystem Services for Poverty Alleviation). Such research is typically use-orientated, multi-disciplinary and people-centred (McLean et al. 2022, p. 4) and requires a broader assessment of impact beyond the typical measures of reach and instrumental knowledge utilisation. For example, IDRC’s RQ + framework highlights research legitimacy (including the engagement process with local knowledge) and positioning for use (Ofir et al 2016), whereas others highlight capacity strengthening, networks and connectivity (Georgalakis and Rose 2019). More recent critiques, such as around the decolonisation of aid, also highlight the need to go further than R4D’s use orientation to encompass local stakeholders’ perspectives on societal value of research—especially given unequal power dynamics and the positionality of researchers (Taylor and Tremblay 2022; Lawrence and Hirsch 2020; Fransman 2018; Langdon 2013). It is in this context, that the evaluation of GCRF provides valuable lessons on the pre-requisites for impact: what evaluators and programme managers might consider and support to have a better chance of achieving transformative change.

Global challenges research fund: a case of R4D evaluation

GCRF was launched in 2016 by the UK Government to support pioneering research and innovation (R&I) that addresses sustainable development challenges faced by developing countries (BEIS 2017). Aiming to spur progress towards the achievement of the SDGs, as discussed above, GCRF forms part of the UK’s ODA commitment. The fund is managed by the UK’s Department for Business, Energy and Industrial Strategy (BEIS). GCRF will run until March 2025, making it an unprecedented scale of investment and an unusually long-term commitment of ODA funds for research and innovation.

GCRF supports challenge-led, interdisciplinary work which mobilises multi-stakeholder partnerships across the Global North and South, and across sectoral boundaries. Its goal is to promote innovative solutions to complex global development challenges and build lasting R&I capabilities and infrastructures in LMICs (BEIS 2017). The pathway to impact set out in the fund’s Theory of Change (ToC) intends that widespread adoption of GCRF’s research-based solutions and technological innovations in LIMCs contributes to achieving the SDGs. This impact is expected to be sustained through equitable R&I partnerships between UK and LMICs, and the improved capabilities for challenge-oriented R&I developed over the life of the fund (Barr et al. 2018).

GCRF is implemented through a devolved model involving the UK’s ecosystem of research councils and academies, as well the devolved higher education funding councils.Footnote 5 These partner organisations (POs) have developed a wide-ranging set of GCRF-funded grant programmes and calls through their existing systems to commission a large-scale and highly diverse portfolio of research and innovation projects. These are being implemented through partnerships between UK and LMIC institutions in many countries on the global South. Over 3,000 awards were made between 2016 and 2022, creating a highly diverse portfolio covering a wide range of development challenges, disciplines, modalities, partnerships and geographies.

GCRF’s emphasis on challenge-led, interdisciplinary work and multi-stakeholder partnerships to support use aligns GCRF with other UK government, ODA-funded research for development funds and programmes. GCRF shares aims with programmes such as the Ecosystems Services for Poverty Alleviation programme (ESPA), which ran from 2009 to 2018, funded jointly by then-DFID and NERC (UKRI), and the Joint Fund for Poverty Alleviation, that ran from 2005 to 2021, funded jointly by FCDO and ESRC (UKRI).

GCRF, however, differs in terms of its scale, being a large-scale fund rather than a programme, more aligned to the Newton Fund (2014–2021). It also has a broad focus—on all the SDGS, all LMICs. This breadth, as well as the sheer scale of its funding, has given GCRF a comprehensive reach across the whole UK R&I community, disciplines and sectors.

GCRF evaluation: strategy and methods

The GCRF evaluation takes a theory-based design, tracking the GCRF ToC over five of the nine years of the fund (2020–2025). The evaluation was launched in 2020, when GCRF was starting its fourth year of implementation. As a large-scale R4D evaluation of a complex fund, the evaluation strategy opted for a multi-module, multi-method approach, implemented in three stages that sequentially assess different aspects of the GCRF ToC through a range of evaluation modules and methods (Barr et al 2018).

Stage 1a (2020–2021) took a formative lens to assess the pre-conditions and assumptions in the activity level of the ToC, establishing the extent to which the drivers of development impact (mentioned in the introduction), such as fair partnerships and gender and inclusion, have been integrated into the commissioning of awards (Vogel et al. 2022). A review of GCRF’s fund management processes was also conducted as part of this stage (Guthrie et al. 2018). The 186 sampled awards covered at this stage represented around £189m of the budgeted expenditure.

Stage 1b (2021–2023) took a process lens to understand how GCRF’s processes are working and the extent to which they are starting to deliver results. Stage 1b has involved a number of evaluation modules to look both in breadth across the whole fund (survey and data science), in depth at the large-scale ‘signature’ programmes (process evaluations), and a R4D quality assessment (RQ++ assessment). The modules conducted in Stage 1b included:

  1. a.

    Six process evaluations to examine six of GCRF’s large-scale ‘signature’ programmes—those that are most aligned to its strategy of interdisciplinary, partnership-led, impact-oriented R&I—to establish how these are working and the extent to which they are positioned for promoting results and outcomes as proposed in the ToC. These programmes represent about £860m of GCRF’s budgeted activities (Vogel et al. 2022).

  2. b.

    a fund-wide survey of award holders and LMIC partners, sent to 12,000 individuals and obtaining a final dataset of 3200 responses (approximately 44% are from Principal Investigators in the UK and 55% are from Co-Investigators in LIMCs) (Vogel et al. 2022).

  3. c.

    an assessment of research quality, positioning for use and early results, through an application of a modified version of IDRC’s Research Quality Plus instrument, covering 150 awards representing around £127 m of GCRF’s budgeted expenditure (Carden et al. forthcoming)

  4. d.

    data science analysis of the outputs from the whole fund, identifying around 12,571 GCRF- associated publications.

Stage 2 (2023–2025) will assess the extent to which GCRF’s programmes and awards are translating into development outcomes in LMIC settings and local systems, through a series of country case studies and ongoing data science of the whole fund. Stage 2 will draw on mid-level theories about how R&I promotes development outcomes in LMIC settings to understand GCRF’s contributions.

This paper draws primarily on the evidence and insights gained through Stage 1a of the evaluation (Vogel et al. 2022; Brocklesby et al. 2022; Izzi et al. 2022; Punton and Lohani 2022). We highlight examples of mainly strong performance to illustrate the discussion.

Tensions between scientific excellence  and managing for development impact

GCRF delivers funding for research through the existing ecosystem of research councils, academies, higher education funding councils and universities. Its highly devolved structure of delivery is designed to provide a level of independence for the UK research community—in accordance with established principlesFootnote 6 that govern UK public funding for research. 

While these funders have well-established commissioning systems that are geared towards promoting scientific excellence in R&I, scientific excellence is merely a necessary but not sufficient condition for impact. This tension has been highlighted previously in the context of R4D, where narrow conceptualisations of research, scientific merit and impact have led to calls for improvement in evaluation methods for research (e.g. Hicks et al. 2015) and in the development of broader instruments more suited for R4D, such as IDRC’s Research Excellence Framework (McLean et al. 2022). In GCRF’s case, the ICAI Review (ICAI 2017) first pointed to a potential tension in GCRF’s priority focus on research excellence (i.e. which ‘may continue to advantage developing countries that already have credible research institutions’), and its aim of capacity building, which would instead ‘[direct] investments towards poorer countries where capacity building may be most needed’. Our evaluation further untangles this tension: that while GCRF’s funders  have robust systems for supporting excellent research in open competition through peer review, an integrated focus on aspects such as gender, social inclusion, poverty, fairness and coherence is less consistently embedded in structures, capacities and processes. This is despite these being key building blocks of achieving transformative, challenge-led research with development impact, as set out in GCRF’s strategy and theory of change.

Unlike other UK Government R&I funding, as mentioned above, Official Development Assistance (ODA) is conditional on being used ‘to promote and specifically target the economic development and welfare of developing countries’. What the evaluation found was that, in practice, in GCRF this condition within ODA funding had led to a focus that was more about legal compliance than effectiveness. For instance, a narrow interpretation of OECD’s ODA definition – and one that is easier to check – is an adherence to the DAC List of ODA recipient countries. But this is a minimum threshold to be reached rather than the more ambitious pursuit of excellence through the consequential impacts of research on welfare and socio-economic development. Thus, the evaluation of GCRF echoed the ICAI findings by finding a mismatch  in how research excellence is combined with development outcomes and broader considerations of equity (Vogel et al. 2022, pp. 49–52). While there are pockets of better practice, research excellence is typically prioritised in the commissioning process, and yet this gives way to a delegation to the research community for ‘managing development impact’. In GCRF, this delegation resulted in inconsistent approaches to some key aspects of GCRF projects that we consider to be the drivers of development impact—capacity building and partnerships, as well as how fairness, gender, inclusion and poverty are addressed in awards. To achieve its ambitions for development impact, the evaluation recommended that GCRF go beyond considering research excellence alone to considering how it can support excellent R&I with development impact. But how should this be implemented in practice? To facilitate improvement in this area, the evaluation coined the term ‘ODA research and innovation excellence’Footnote 7 (Vogel et al. 2022, pp. 14–16). This concept incorporates a wider understanding of what GCRF should strive towards - a focus on the building blocks of impact, together with a proactive management approach to optimise these from fund level through to project implementation. We now discuss four of the key building blocks in turn. 

Building blocks of impact

Taking a theory-based approach means that the GCRF evaluation has been able to work backwards from the intended outcomes and pathways to impact, to test if the foundations for these have been established in the early stages of GCRF’s implementation, before outcomes have started to emerge. For example, a key outcome area in the ToC is that ‘sustainable, equitable, global research and innovation partnerships are established across geographies and disciplines.’ (Barr et al. 2018). The evaluation has been able to explore whether the foundations for this outcome have been set up in the early stages of the fund by examining how GCRF programmes and awards have prioritised equitable partnerships in their commissioning and how LMIC partners have experienced collaboration through the early stages of implementation. In this way, guided by the ToC, the GCRF evaluation identified four building blocks of impact – elements and processes that projects need to build into their research to position it for impact. These include:

  • scoping of issues on the ground to enhance relevance

  • integrating a focus on gender, inclusion and poverty

  • establishing fairness and mutual capacity building in partnerships

  • developing stakeholder networks right from the outset

We will discuss each of these in turn, drawing on evidence from the first phase of the evaluation (Vogel et al. 2022).  

Scoping of development issues with stakeholders on the ground for relevance

The first building block for impact is relevance, i.e. alignment with the challenges and needs of the benefitting community, institutions and/or country. A R4D fund like GCRF needs to fund relevant research in order to effectively position research for impact in the contexts in which it is intended to contribute solutions. Experience in the R4D field highlights the importance of ‘early and ongoing consideration of the wider context for research application’ as depicted in the GCRF ToC (Barr et al 2018, p. B2). We have framed this as ‘relevance’, following the definitions on the OECD DAC criteria.Footnote 8

In GCRF, the evaluation found that award teams generally had considered relevance in detail, mainly in response to  application requirements, and most awards aligned with country or regional priorities in a broad sense. Relevance in the awards depended, to a large extent, on how well networked the researchers were with the communities of focus prior to the grant, as relevance was achieved more through a reliance on existing personal and professional experience, knowledge and pre-existing partnerships in focal countries, rather than formal needs assessments or scoping activities (Punton and Lohani 2022; Vogel et al. 2022).

Some awards, notably the larger-scale ones, took a more systematic approach to relevance as part of the research design and implementation process. In these cases, inception and partner mobilisation periods were funded, which enabled structured scoping to take place and subsequent refocusing and refinement of R&I designs (Punton and Lohani 2022).

Whereas most awards aligned with development priorities through the written application process and informal relations with Southern partners, it is these more deliberate engagements to understand the issues and research problems from the local perspective—even to the point where the research questions and focus may need to change in response—that build the foundations for development impact.

Fair and equitable partnerships between partners in the global North and  South

The second building block for impact relates to the extent to which GCRF works through equitable and fair partnerships with academic institutions, policy and practice stakeholders and communities in the global South. A focus on fairness in partnerships acknowledges that a research project is never only about finding answers to a particular research question. It is also about the process itself, the differential interests of the various actors involved, and the way in which these interests are reflected in the setting of agendas, research questions and methods (Izzi et al. 2022; Fransman et al 2018).

These issues were of paramount importance in GCRF, where the mobilisation of considerable resources through international partnerships and in-country networks could have led to multiple negative impacts. It was also a key focus of the evaluation.

For R4D efforts, the importance of fairness in partnerships is amplified, given that partnerships typically involve entities from the global North and South. Historical inequalities and power dynamics between regions in the global north and south are inherently reflected in research partnerships, creating asymmetries in terms of control of resources and influence on agendas, unless conscious efforts are made to mitigate these.

There is a further practical relevance in R4D, where the engagement of different stakeholders beyond researchers is considered essential to generating impact. Fairness in R4D thus extends beyond the immediate research partnerships to encompass all those involved—partners, participants, users and beneficiaries, as well as considerations of how the research impacts on the broader context, including national research systems and research capacity (see Box 1).

Box 1: Research fairness is defined as a way of designing, conducting and evaluating research that takes into consideration the potential effects (positive and/or negative) of the research on all those involved (as partners, participants, users, and beneficiaries), as well as the broader impact on the context where the research takes place. This encompasses three dimensions:

• fairness of opportunity- who has a say in designing the research and identifying who will participate;

fairness of process—that there are procedures and structures to support transparency and accountability and for everyone to have a voice;

• fairness of benefit sharing—there are transparent processes for agreeing how the benefits of the partnership will be distributed. (The ‘fairness’ definition draws on the Research Fairness Initiative (RFI), developed by the Council on Health Research for Development (COHRED) https://rfi.cohred.org.)

Equitable partnerships can therefore be seen as a necessary but not sufficient condition of fairness in R4D. In principle, it would be conceivable to have a situation where a partnership is equitable (as both partners have equal voice, there are transparent and jointly agreed procedures, and the benefits are distributed in a mutually satisfactory manner) and yet not fair (if, for example, inherent inequalities based on access to resources and knowledge are reproduced and legitimised, or if the partners act as gatekeepers to prevent other institutions and researchers from accessing similar opportunities) (Izzi et al 2022, p. 17).

In GCRF, there was a prioritisation of equitable partnerships as an aspiration across the fund, widely seen by managers, award holders and partners as a flagship characteristic of GCRF. There is widespread recognition of the need for meaningful and fair engagement of Southern partners, although implementing this in practice has represented a learning curve for fund managers and award holders alike. There have been efforts to increase the fairness, equity and representativeness of GCRF partnerships, delivering a number of initiatives that were consistently mentioned as milestones in this process, including a collaborative process amongst researchers based in the Global South to identify equitable partnership principles , a handbook and an online equitable partnership resource hub for R&I teams. However, the awareness of – and approach to – fairness issues across remains uneven across the fund. Notably the relative lack of Southern voices and perspectives within GCRF governance and decision making remains limited, with implications for future impact (Izzi et al. 2022).

Gender, social inclusion and poverty (GESIP) prioritised in policies and implementation

The third building block for impact is a strong focus on gender, social inclusion and poverty reduction (GESIP), integrated into commissioning, research design and project implementation. For R4D funds and projects working in low-income countries, an integral focus on GESIP is key to ensuring that research contributes to improving lives and tackling inequality through its process as well as its results.

This was a key assumption in the GCRF ToC—that addressing gender, social inclusion and poverty reduction systematically, coherently and across all levels of GCRF would enhance the development impact of the fund. This assumption was affirmed by various reviews on gender equality and inclusion that had been carried out prior to the GCRF evaluation, and a new gender equality policy developed by BEIS for its ODA investments in 2021. As noted in the new policy it should not be assumed that the impact of new technologies and knowledge production will have equal benefit or positive effects for everybody, and it is important that the research and innovation sector considers this alongside ensuring that there are equal opportunities to access and participate in the research and innovation process itself. (BEIS 2021, p. 6).

Building on previous gender and inclusion assessments, in 2020–2021, the GCRF evaluation focused on the extent to which GESIP has been integrated at all levels of the fund, from strategy to implementation in awards, and identified opportunities for strengthening this (Brocklesby et al. 2022). The high-level focus placed on equitable partnerships with relatively good performance, provided a benchmark for assessing the extent to which GESIP had been integrated.

The evaluation found a very different but evolving picture. In 2020–2021, GCRF was moving from only scattered pockets where GESIP issues were focused towards establishing more comprehensive policies, expertise and MEL systems for integrating and tracking GESIP concerns. Nevertheless, important gaps remained around building up dedicated senior management capacity and clear accountabilities for implementation of GESIP consistently throughout the fund. A focus on poverty reduction was found to be the least supported by structures and processes. Overall, this suggests that gender and social inclusion can be designed into R&I for inclusive impacts, as set out in the ToC assumptions, but that this is not yet being fully realised across the fund.

By focusing on identifying the policies, structures and processes to integrate a consistent GESIP focus at all levels of the fund, the evaluation was able to identify promising pockets of good practice, and provide an analysis of the gaps and the implications for future impact. Good practices at this early stage were reliant on award holders sharing information and organising learning, driven by energised groups of individuals.

Stakeholder engagement in LMICs to support positioning research for use

The fourth building block is stakeholder engagement to support positioning research for use. Engaging stakeholders early and at appropriate subsequent points of the research process is an established practice within several fields of research intended to have a societal impact, particularly areas of public health and development research. Experience and evidence support the view that stakeholder engagement, both academic and non-academic, is critical to catalysing local impact pathways ( Veras de Sandes-Guimarães, 2022; Boaz et al. 2018; Izzi, 2022). Involvement of stakeholders can help to identify in-country policy needs and priorities so that the research is directly aligned to user needs, strengthen contextual relevance of research questions and framing, inform locally appropriate research designs and develop research findings that are tailored to local needs and owned by local stakeholders, thus be more readily applied and adopted (McLean et al. 2022).

Early-stage evaluation of development research funds can focus on the extent to which research funds and programmes have prioritised the involvement of stakeholders and the effectiveness of structures and processes to enable this.

In the early stages of the GCRF evaluation, the awards reviewed reported extensive engagement with stakeholders, including local and national governments, national and international non- governmental organisations (NGOs), and local communities – and, less frequently, the private sector (Izzi et al. 2022). Large scale awards, such as the Interdisciplinary Hubs, were able to engage non-academic stakeholders early on for scoping with to identify gaps and opportunities, and develop these into stakeholder partnerships for co- designing research (Punton and Lohani 2022). These approaches represent a step forward for GCRF in terms of stakeholder engagement, and future modules will probe more deeply into how these are working and what has been achieved.

How can programmes and projects implement the building blocks?

Development excellence standards

In this paper, we have highlighted four building blocks that emerging  evidence in the GCRF evaluation highlights as having a close association with an increased likelihood of development impact. These, however, are not the only conditions to achieve impact. A rapid review of the literature on challenge funds in international development, as well as mission-orientated R&I, informed the evaluation framework for GCRF. From this literature, we can distil a number of other elements that are necessary to support impactful development research (Murray et al. 2021; O'Riordan et al. 2013); both in terms of the design and delivery of the research itself (such as applying a gender lens), as well as for the management of awards and research programmes (e.g. challenge-led portfolios, flexibility, learning and adaptation). See Fig. 1  below.

figure a
Fig. 1
figure 1

Elements to support impactful development research

Full size image

The GCRF ToC and evaluation identified the four bolded elements as the key foundations for development impact in GCRF. However, all of these dimensions could potentially form the basis of a set of shared standards to strengthen the design and delivery of R4D. In a large, devolved, researcher-led system like GCRF's, high-level strategic leadership – supported by effective coordination and improvement structures – are required to integrate the fundamentals for positioning R&I for development impact. Learning from the successes in GCRF, such as the prioritisation of equitable partnerships, suggests that ODA R&I excellence standards could provide a way of building a culture of learning and improvement, e.g. monitoring, capture and sharing of good practices, supported by resources and training produced by the community, with case studies to showcase best practice and inspire research teams to reach for ODA research excellence. IDRC’s RQ+ is an established standard for R4D that could provide a starting point for the UK R&I community (McLean et al. 2022).

Addressing the implicit tension between prioritising scientific excellence and ensuring a focus on development outcomes, is not a zero-sum game. It is a creative tension, where rather than viewing quality and impact as trade-offs, there are gains to be made if the tension is navigated effectively with high-quality research being designed and carried out with use and impact in mind. For example, a meta-analysis of 170 research studies suggested that contrary to conventional wisdom, there is no clear trade-off between rigour and the utility of research; and indeed, research capacity strengthening was found to be positively correlated with the scientific merit of the research (McLean et al. 2019). There can also be gains in terms of creating new kinds of development expertise and stakeholder relationships between the research systems in the UK and the Global South by mobilising broad-based efforts to achieve excellent ODA R&I.Footnote 9

Conclusions: pre-conditions for success: a basis for evaluation?

Identifying the four building blocks for development impact provides a guide for researchers and research managers as they progress towards impact. In this way, they might be viewed as early indicators of likely success; i.e. if a research programme or award is working in a way that is unfair, lacks inclusion, not challenge-led (relevant) and so on, there is a lower likelihood that it will have a development impact.

In terms of evaluation, assessing these building blocks for development research excellence provide a basis to: (i) test assumptions and pathways in the theory of change; (ii) provide a criteria to assess progress towards (or likelihood of) impact; and, (iii) provide a way to learn and adapt during implementation. The latter is important, given that the research-to-development impact pathway is typically a lengthy process, and evaluative insights perhaps provide most value during implementation.

In large-scale funds and programmes, however, the foundations for development impact are not always straightforward to spot. The insights from the GCRF evaluation suggest that it is not enough for evaluators to look for formal structures and processes that prioritise a focus on development impact. A focus on ODA research and innovation excellence should be stimulating new ways of thinking, partnering and working—a different culture as was seen in the Hubs programme. So, evaluators also need to look for the behaviours that indicate that people involved in the research are coming together to figure out how integrate, sustain and improve a focus on development impact throughout the lifetime of their programmes and projects.

The efforts on equitable partnerships in GCRF have shown how an R&I community can come together around key priorities—and importantly set definitions and standards that influence practice while still adhering to the Haldane principles for publicly funded research. Collective processes to agree, implement and monitor quality standards have helped drive practice in other fields. In the humanitarian field, for instance, the Sphere standards processFootnote 10 has improved the quality and accountability of humanitarian assistance. Standards of research quality relevant to development contexts, such as IDRC’s Research Quality Plus (RQ +) framework, similarly provide a useful starting point for such a process.Footnote 11 The evaluation recommends that a process for agreeing ambitious quality standards for ‘ODA research and innovation excellence’ for the whole fund would similarly help the different stakeholders (BEIS, DPs and award holders) cultivate a culture of improvement—and that this would help realise the ambition of the Fund to deliver transformative research that addresses the SDG challenges (Vogel et al. 2022, p. 78). GCRF offers very rich learning for shaping standards that could help guide the whole field of research for development.