Introduction

In recent years, the concept of Responsible Research and Innovation (RRI) has been addressed frequently in academic literature (e.g. De Saille 2015a; Rip 2014; Shelley-Egan et al. 2018). According to Rip (2014), responsible development of nanotechnology was already discussed in the 2000s but not labelled as RRI yet (Royal Society and Royal Academy of Engineering 2004). RRI has gradually developed out of concerns for emerging technologies, such as nanotechnology, and implies changing roles and responsibilities for actors and stakeholders involved in the science and technology development, and a changing embedding in society.

Meanwhile, De Saille (2015b) described developments in thinking about RRI as a policy concept, and concluded that the translation of academic theory into the daily practice of European policy is difficult due to the acknowledgement that implementing an engagement process in practice may, for example, also mean that developments could be halted. She argues that often moral, ethical and social questions are excluded from such engagement processes and concludes that structures which allow for meaningful exchange between policy and citizens-at-large are not yet in place. In turn, this will risk that RRI remains too vague. Also, Shelley-Egan et al. (2018) made a plea for a more practical perspective. They see RRI as a means of integrating society and research and innovation communities and call for evaluating the embedding of RRI, in other words, they argue to investigate the uptake of RRI in practice. According to them, learning from the way RRI is implemented in, for example, universities, public research organisations and firms is needed.

Hence, despite debate about the concept, much remains unclear when it comes to moving from the academic debate to the day-to-day practices of research and innovation. And, although practices of RRI have been reported and suggested in the literature, yet, there is no overview available of ways to bring RRI into research and innovation practices. Analysing practices and mapping out main approaches as well as the values, dimensions or characteristics of RRI pursued with those practices, can fill the gap between conceptual discussions of RRI and its practices as (c.f. Shelley-Egan et al. 2018). Therefore, this paper presents the results of a systematic literature review of RRI practices which aimed to gather insights to further both the theoretical and practical development of RRI.

The article proceeds as follows. First, a framework for studying practices of RRI is provided. Then, the methodology for the systematic literature review will be detailed. Thereafter, findings are presented upon which these will be discussed.

Values, Dimensions and Characteristics of RRI

To guide the analysis of practices of RRI, first, current definitions of RRI were considered. Several authors have presented definitions of RRI but, according to, for example, Burget et al. (2017), the concept is still in development. There is, therefore, not one generally accepted definition in use (Ruggiu 2015; Wickson and Forsberg 2015). At the moment, several definitions are widely used in the literature and also inform the European Commission’s research and innovation policy. These definitions provide insight in the values, dimensions and characteristics of RRI.

First, Von Schomberg (2013a, 63) offered the most cited definition:

Responsible Research and Innovation is a transparent, interactive process by which societal actors and innovators become mutually responsive to each other with a view to the (ethical) acceptability, sustainability and societal desirability of the innovation process and its marketable products (in order to allow a proper embedding of scientific and technological advances in our society).

Von Schomberg’s definition refers to the research and innovation process as well as the resulting marketable products. According to him, the products of research and innovation should reflect normative anchor points, for example, be ethically acceptable, sustainable, and socially desirable. He argues that the innovation process should become “more responsive, adaptive, and integrated” (Von Schomberg 2013a, 65).

Stilgoe et al. (2013, 1570) provided another definition:

Responsible innovation means taking care of the future through collective stewardship of science and innovation in the present.

RRI, in their view, requires anticipation, reflexivity, inclusion, and responsiveness. This means that the intended and unintended impacts and consequences of research and innovation should be assessed. Also, researchers and innovators should reflect upon their research process, activities, and underlying assumptions. Researchers and innovators are, furthermore, advised to open up to other voices in the discourse on research and innovation, such as stakeholders and the public. Finally, research and innovation should develop “a capacity to change shape or direction in response to stakeholder and public values and changing circumstances” (Stilgoe et al. 2013, 1572). The authors argued that, together, these four dimensions influence research and innovation and thereby contribute to “taking care of the future” (Stilgoe et al. 2013, 1570).

The Directorate-General for Research and Innovation (2013, 55–56) in their Report of the Expert Group on the State of Art of Responsible Research and Innovation, defined RRI as:

Responsible Research and Innovation refers to the comprehensive approach of proceeding in research and innovation in ways that allow all stakeholders that are involved in the processes of research and innovation at an early stage (A) to obtain relevant knowledge on the consequences of the outcomes of their actions and on the range of options open to them and (B) to effectively evaluate both outcomes and options in terms of societal needs and moral values and (C) to use these considerations (under A and B) as functional requirements for design and development of new research, products and services.

There are two “points of reference [that] should be reflected in the design of research and innovation processes and products”, according to the Directorate-General for Research and Innovation (2013, 56). These are ethical acceptability and orientation towards societal needs. Ethical acceptability “includes compliance with both the EU charter on fundamental rights as well as the safety of products”, while the orientation towards societal needs includes contributing to sustainable development and normative objectives, such as realising gender equality. The societal needs can also relate to improving the quality of life.

Finally, a fourth definition is used by the European Commission (2018):

Responsible research and innovation is an approach that anticipates and assesses potential implications and societal expectations with regard to research and innovation with the aim to foster the design of inclusive and sustainable research and innovation. […] Responsible Research and Innovation (RRI) implies that societal actors (researchers, citizens, policy makers, business, third sector organizations, etc.) work together during the whole research and innovation process in order to better align both the process and its outcomes with the values, needs and expectations of society.

In this definition, the European Commission (2018) further stated that there are five characteristics or dimensions of RRI, also labelled keys, which make RRI tangible: public engagement, open access, gender, ethics, and science education. Governance of the whole process is a sixth dimension, or key, which serves to integrate the other five.

The four definitions of RRI showcase that the term responsible in relation to research and innovation is conceptualised in various ways. The Directorate-General for Research and Innovation (2013), for instance, emphasises functional requirements for the design and development of research, products and services. Meanwhile, the other definitions are less specific on how RRI can guide the research and innovation process. Furthermore, the definitions emphasize different characteristics of RRI. For example, Stilgoe et al. (2013) distinguish anticipation, reflexivity, inclusion, and responsiveness as dimensions or characteristics whereas those given by Von Schomberg (2013a) relate to the process and the product. The European Commission (2018) uses three dimensions—gender, open access, and science education—that are not mentioned in the other definitions. All definitions aim to clarify the core of RRI. The authors discuss values, or activities realising those values, that contribute to responsibility. These values include, for example, sustainability (Von Schomberg 2013a) or inclusion (Stilgoe et al. 2013). The activities can be processes, such as anticipating future impacts (European Commission 2018) or developing function requirements (Directorate-General for Research and Innovation 2013). The practices of RRI studied in this paper can therefore reflect a rich variety of characteristics, dimensions and values.

A second consideration for the review was how the notion of practice itself in relation to RRI can best be understood. MacIntyre (2007, 205) argued that a practice is a complex cooperative human activity which tries to realise internal goods which form that activity. Applying MacIntyre’s definition of practice, RRI practices in this study are seen as efforts which try to realise values of RRI, or, in other words, goods internal to the idea of RRI. A practice is therefore defined as one or more actions or events related to the research and innovation process through which one or more actors intend to realise at least one of the values, dimensions, or characteristics of RRI.

Methodology for a Systematic Literature Review

This study followed criteria for a systematic literature review as described by Petticrew and Roberts (2006). Together with an information specialist from the library of the University of Twente, a search string was developed and tested. The search string can be found in text box 1. The search string aimed to maximise the chance of finding relevant articles and, therefore, included the terms ‘responsible research and innovation’, ‘responsible innovation’, and the six keys or dimensions of responsible research and innovation used in the European Union definition, which are public engagement, science literacy, ethics, open access, gender, and governance. The search was run in three databases Scopus, Web of Science, and the Philosopher’s Index, which together were expected to cover most scientific publications on the topic. To include other relevant publications, in addition, a snowball search on the included articles was conducted. However, due to time and resources limitations an active search for reports on practices of RRI was not conducted.

Text box 1. Search string

((TS = ((”responsible research and innovation” OR “science NEAR society” OR “responsible innovation” OR “responsible research”)) AND TS = ((”open access” OR “public engagement” OR engagement OR “gender diversity” OR gender OR “gender equality” OR ethics OR “science education” OR “science popularization” OR “science communication” OR governance)))) OR ((TS = ((”responsible research and innovation” OR “science NEAR society” OR “responsible innovation” OR “responsible research”)) AND TS = ((implementation OR “best practice” OR “good practice” OR obstacle OR barrier OR facilitat* OR regulation OR policy)))) OR (TS = ((”responsible research and innovation” OR “science NEAR society” OR “responsible innovation” OR “responsible research”)))

The search, executed on Tuesday 12 January 2016, generated the following results:

Scopus: 326

Web of science: 258

Philosopher’s Index: 23

Total: 607

Articles were included when they met the following three inclusion criteria:

  • The publication is published in English between 2005 and 2015, as a journal article, a contribution to an edited volume, or conference proceedings; AND

  • The publication is about responsible research and innovation; AND

  • The publication includes examples of, or suggestions for, RRI practices, which are actions or events related to the research and innovation process through which one or more actors intend to realise at least one RRI value, dimension or characteristic as exemplified in the definitions.

Thus, next to being published within the given time frame, to meet the second criterion, articles had to contain a reference to RRI or a related term such as responsible innovation or socially responsible innovation. In addition, since RRI is the subject of a developing discourse instead of an established discourse, practices which met the third criterion could either be descriptions of past events or actions or suggestions for future ones.

The definition of practice allowed to distinguish articles describing practices which intended to bring out values of RRI from articles describing practices that use similar methodologies without the intention to realise these RRI values. A survey amongst citizens, for instance, can be used to involve them in the research and innovation process or as an object of study that has no implications for the research and innovation process. Not all studies thus bring RRI in practice. Such a distinction therefore helped to select the publications containing practices related to the RRI discourse.

Furthermore, whether or not a value or characteristic of RRI is indeed realised in a practice can only be determined by an evaluation of that practice focusing on this question. Such evaluations or assessments were beyond the scope of this study. Successful realisation of values or characteristics was, therefore, not necessary to be included in this study. To qualify as a practice, only the intention to realise at least one RRI value, dimension, or characteristic by means of an action or event related to the research and innovation process was necessary. Therefore, the practice could still be in the theoretical stage or could already have been used in research and innovation processes. Equally, in the analysis there was no distinction between short suggestions or detailed descriptions and both were included when the articles met the inclusion criteria.

The first database search generated 607 articles. This number was brought down to 391 after removal of duplicates. The 391 articles were checked for the inclusion criteria in two rounds. First, based on the titles and abstracts, articles were excluded that did not meet the inclusion criteria. This narrowed down the articles to 132. Thereupon, the full texts of these 132 articles were checked regarding the inclusion criteria, after which 49 studies remained. In the next step, a snowball search was conducted by checking the references from the 49 included articles for additional relevant studies and 3 new articles were included. In total, 52 publications were included in the review (see Table 1 for an overview of the steps).

Table 1 Steps towards inclusion with the number of included publications
Full size table

Next, an iterative and qualitative analysis was conducted on the included articles, focusing on understanding the values, characteristics or characteristics of the practices. To facilitate analysis, an appraisal form for the articles was used which summarized relevant findings. Part of the sample was reviewed by two independent assessors and these results were compared to enhance greater validity.

From the analysis four main overarching themes emerged. Further analysis focused on relating the articles to the themes, refining the understanding of the themes, and understanding the similarities, differences, and meta-characteristics of the articles. The themes reflected the following dimensions of RRI: opening up to stakeholders and the public; stimulating reflection on the research and innovation process amongst researchers and innovators; ways of anticipating ethical, legal and socio-economic aspects of research and innovation, and, finally, institutionalisation of responsibility through governance of research and innovation processes.

Some articles related to more than one theme. However, to facilitate the analysis, it was decided to assign the practices reflected in the article to the theme or group they would fit best, that is according to their perceived main aim. For instance, when the main aim of a practice was stimulating reflection amongst innovators by discussing the potential implications of their research with them, such a practice also involved elements of dialogue. One article, however, discussed two different practices at equal length and, therefore, was assigned to two themes (Fisher and Rip 2013).

Characterizing the Practical Uptake of RRI

First, some general characteristics of the articles will be given. The majority of the included articles was published towards or at the end of the inclusion period. Geographically, first authors of the publications were often affiliated to institutions in north-western Europe which means that practices of RRI are mainly found in this area. Other first authors were affiliated to institutions in other parts of Europe, for example, Italy or Poland, and other parts of the world, such as Indonesia or the US. Of the total sample, 20 articles were written by a female first author (38%). Less than half of the articles were published under open access licenses or in open access journals.

The articles covered a range of research and innovation fields. See also Table 2 for an overview. The studies most frequently covered nanotechnology (n = 10), the field where RRI more or less started, then ICT (n = 9), and synthetic biology (n = 4). Other articles presented practices from a variety of research fields, including engineering (not further specified), neuroimaging, bio-economy, and agriculture, or, contained practices described by philosophers or science and technology studies scholars. The humanities and social sciences were not represented on a regular base. Next, the articles will be discussed in relation to the themes they reflect.

Table 2 Overview of included articles, category of practice, and research field
Full size table

Opening Up Research and Innovation

The largest group of practices (n = 23) aimed at opening up the research and innovation process to stakeholders who are not directly involved in research and innovation or the public. Within this group three sub themes emerged related to stories (n = 5) when fiction, non-fiction or news stories played a central role; involving experts and stakeholders (n = 16) when the practice involved one or more events with a limited number of stakeholders or experts; and societal dialogue (n = 2) when large-scale dialogues aimed to involve society as well as experts were analysed.

Story Telling

In the articles dealing with stories as a way of opening up research and innovation, a new approach to science journalism came from De Jong et al. (2015). They proposed reporting that aims to deal with inaccurate portrayal of neuroimaging in the news: Experts suggest what needs to be reported and come up with outlines of newspaper stories that paint realistic pictures of the technology, its application and wider relevant aspects; journalists should then be invited to base their stories on these outlines. De Jong et al. (2015) argued that such an approach would mitigate hypes surrounding neuroimaging by creating a better balanced, more nuanced image of neuroimaging. In addition to using journalistic stories in RRI practices, using existing narratives and literature was suggested by Grinbaum (2013) and Grinbaum and Groves (2013). For instance, Mary Shelley’s Frankenstein can serve, according to Grinbaum (2013), as a metaphor to encourage ethical thinking on responsibility in relation to research and innovation. Meanwhile, Stahl et al. (2014), encouraged developing science fiction stories as a way of engaging “with the conflicts, emotions and cultural change that a new technology brings. It enables the technology developer to explore the social, ethical and cultural implications of a developing technology” (p. 82). Their article included a science fiction radio play about empathic robots that care for the elderly. In the play, the robot’s empathy, combined with its problem-solving logic, led to disastrous, undesired consequences. Finally, Benford et al. (2015) studied cultural performances developed in collaboration with artists. These narratives intended to explore research findings and innovations on human–computer interaction.

Involving Stakeholders

The majority of the articles in the second sub theme about engaging experts and stakeholders described practices with meetings or procedures which intended to actively involve experts or stakeholders in the research and innovation process. Different formats and guidelines to shape interactions can lead to opening up, including awareness scenario workshops (Gemen et al. 2015), co-evolutionary scenarios (Robinson 2009), and dimensions for good engagement (Malsch 2015).

When participants were described, they were mainly experts and stakeholders, but not all articles described who participated in the workshop. The majority of the articles described a one-off workshop (e.g. De Bakker et al. 2014; Douglas and Stemerding 2013; Robinson 2009), but, for example, Gemen et al. (2015) described a series of engagement activities organised during the INPROFOOD project. In addition, Ravesteijn et al. (2014) described a procedure for stakeholder involvement and management which included a proposal for a so-called multi-stakeholder advisory committee to, amongst other things, facilitate the consultation of stakeholders.

Several articles highlighted points of attention for the practice of RRI. De Bakker et al. (2014) found that all participants in a workshop need to voluntarily engage in a dialogue with other stakeholders, which means they have to be willing to share information with others, otherwise, it is difficult to deal with existing power and information asymmetries which exist between, for instance, industry or academics and other stakeholder groups. Therefore, De Bakker et al. (2014) proposed safe discussion arenas where confidentiality is agreed upon by all participants to enable open communication between stakeholders.

In addition, Gemen et al. (2015) reported learning outcomes for organising successful expert and stakeholder engagement activities such as balancing participant samples, professional facilitation, comprehensive recording, as well as evaluating the activity immediately after it takes place. They used those insights to draft a mobilisation and mutual learning action plan describing actions for policy makers as well as best practices for the food innovation sector with regard to stakeholder engagement in food and health innovation research programming (Gemen et al. 2015).

De Saille (2015a) discussed her experiences from an ethnographic study in which she analysed the politics of technology. She observed members of a social movement organisation and concluded that, first, a “sophisticated discourse around technoscience […] exists within and beyond this [social movement organization]” (De Saille 2015a, 104), and concluded that these members of the public may be interested in deliberations about research and innovation. Her second observation was that the participants in the study did not always feel welcome to participate in public debate and were sceptical about what would happen with their input: “All expressed a deep scepticism about value of such efforts, which also functioned as a passive disinvitation to participate in a process, which, […], was largely seen as going through the motions required for legitimacy, rather than actively taking the public’s concerns into account” (De Saille 2015a, 104).

Societal Dialogue

Societal dialogues were analysed in two studies. Krabbenborg and Mulder (2015) who studied a societal dialogue on nanotechnology in the Netherlands, advocated seeing a societal dialogue as a process, rather than a series of events. To them, carefully preparing the process and the events is necessary to ensure that the dialogue becomes embedded at the institutional level, for instance, by asking officials whether and how public dialogues could fit the aims and strategies of their organisations. Without such a careful design of the dialogue, it is difficult to transcend a risk–benefit discourse and hard to talk about broader societal and ethical aspects of a technology. Simaková and Coenen (2013) analysed two narratives about nanotechnology in Germany and concluded that societal dialogues should include an alternative to the risk–benefit discourse and dominant narratives. They argued that, for a successful societal dialogue, both at a larger or smaller scale, spaces for conversation need to be found and interests of the stakeholders need to be taken into account, such as political or corporate interests.

Stimulating Reflection

The second group of articles contained practices of RRI focusing on stimulating reflections amongst researchers and innovators (n = 12). The practices in the articles aimed to encourage researchers and innovators to reflect, for instance, on the potential impacts and consequences of their work on society, the environment, or other aspects. Three types of practices could be distinguished: practices that took place before the research process began, and practices which stimulated reflection during ongoing research processes. A specific form of the latter are reflection resources that can guide researchers in their reflection.

Before the Research

Owen and Goldberg (2010) stimulated researchers to already reflect on the impacts and consequences of their work during the writing phase of a research proposal. To foster this, the authors added reflection on the implications of projects in a call for funding by the UK Engineering and Physical Sciences Research Council. They asked applicants to complete a “risk register in tabular form, which required [applicants] to reflect on the wider implications of their proposed research, identify potential impacts, and qualitatively assess their associated risks” (Owen and Goldberg 2010, 1701). By requiring a risk register in the funding process, collaboration with the social sciences to promote “continuous reflexivity, participation, and the enhancement of societal learning”, and, feedback processes were enabled (1706).

During the Research

Most other studies described practices focusing on stimulating reflection during the research process. One example was the socio-technical evaluation research methodology (STIR) described by Fisher and Rip (2013). In STIR, or midstream modulation, a so-called embedded humanist talks with researchers about decisions in their research and the potential implications of such decisions. Flipse et al. (2013a) reported an explorative study on midstream modulation among industry biotechnology researchers which showed that midstream modulation can “facilitate and enhance researchers’ active inclusion of social and ethical aspects in their daily laboratory practice” (Flipse et al. 2013a, 1161).

In addition, Betten et al. (2013) described an interactive learning and action approach, which was applied to synthetic biology and other fields. According to these authors, successful reflection using the interactive learning and action approach requires four key factors: articulation of experiential knowledge, knowledge co-creation, embedding, and process facilitation. Field specific procedures for stimulating reflection have been published as well, such as steps for responsible robotics research described by McBride and Stahl (2014).

Reflection Resources

Finally, Stahl et al. (2015) described developing an observatory, an online repository of resources about ethical issues and dilemmas in ICT that researchers can use to reflect on their own research, and Malsch (2013) gave an overview of resources for RRI for nanotechnology of which several were aimed at educating researchers about RRI and what it entails in practice. Examples included an Ethics Toolkit made in the ObservatoryNano project and a set of educational DVDs made by the EthicsSchool.

Anticipating Ethical, Legal and Social Aspects

The third group of articles described practices focusing on assessing the potential ethical, legal, socio-economical and other aspects of research and innovation with the aim to be able to manage them (n = 9). This group is distinct from the practices described in the second group in that the practices focused on assessing the broader issues in a researcher’s own research instead of getting other researchers to reflect on the broader issues. These practices should be implemented at the early stages of the research and innovation process and onwards, to enable that the outcomes can be used to steer the research and innovation process.

Von Schomberg (2013b) proposed a combination of methods to implement RRI, which relate to anticipating ethical, legal, and socio-economic aspects via technology assessment (TA) and foresight; application of the precautionary principle; and incorporating normative principles to the design of technology. In addition, Van der Hoven (2013) proposed value sensitive design, a procedure to incorporate values into the design process of new technologies based on an assessment of the potential implications of the innovations and the values at stake. Timmermans et al. (2011) outlined how such value sensitive design can contribute to the development of nano-pharmaceuticals, while Ikonen et al. (2015) discussed a related approach, that is, ethics by design. In this latter study, design was based on ethical guidelines which made it into a human-driven design process that is holistic, strives for collaboration with stakeholders, and is ethically reflective. “The heart of E[thics by] D[esign] is positive, forward-looking and proactive ethical thinking. Ethical points of view are considered in the early project phases, with the aim of creating a positive, ethical-solution-oriented mind-set among project partners. The ethical approach should not just identify current or future problems, but actively design for and be inspired by achieving ethically sustainable solutions” (Ikonen et al. 2015, 125).

Wodzisz (2015), furthermore, argued that, research and innovation need to comply with relevant regulations such as safety regulations, to incorporate values in the design process and open up to deliberation. He presented conclusions from a case-study on a refrigerant used in the automotive industry. Since the anticipation of—and being able to change—the future is an inherent responsibility of science, the incorporation of values in the research process can be the way to operationalise RRI.

In addition, Fisher and Rip (2013) discussed constructive technology assessment (cTA) as a methodology that holds research and innovation actors “co-responsible for taking societal embedding and potential impacts into account” (2013, 177–178). In cTA, actors in the research and innovation process are brought together with, amongst others, stakeholders, future users, and regulators to bridge gaps between their worlds and explore the societal embedding and potential impacts of the technology.

Other studies described practices which were intended to be used in later stages of the research and innovation process or even after its completion. Two studies related to the ICT field. Wright et al. (2011) proposed using privacy impact assessments, a form of risk assessment, to assess the impacts of new technologies on privacy and consider mitigating measures. Brey (2012) proposed anticipatory technology ethics. This form of ethics can be used for, but is not limited to, information technologies. Three levels of analysis—technology, artefact, and application—were used to forecast ethical issues. To be able to analyse the levels, forecasting methods were collected and a checklist was developed. An overview of identified ethical issues can then be used to provide ethical recommendations about design or governance.

Institutionalisation and Governance

The fourth group of studies distinguished in the corpus of articles, described practices which focused on institutionalising and governance mechanisms (n = 9) by means of institutionalisation, soft governance and mixed governance.

Institutionalisation

Practices favouring institutionalisation were described by Spruit et al. (2015), Wickson and Forsberg (2015) and König et al. (2015). Since responsibility in RRI surpasses the responsibility an individual researcher is able to bear, forming a collective to share responsibility is another way of dealing with its demands according to Spruit et al. (2015). Wickson and Forsberg (2015) argued that implementing international standards opens up so-called interstitial spaces, spaces within which there is room to address societal needs, ethical values and environmental challenges. König et al. (2015) proposed that intellectual property, at least for synthetic biology, can be seen as offering choices with regard to responsibility. Researchers can choose to combine “one’s interests with those of our society and the environment” (König et al. 2015, 1059) by deciding whether to protect research findings or not—and, if so, under which intellectual property scheme (e.g. patent or open license). Forsberg et al. (2015) proposed to rethink the role of science and technology advisory and assessment bodies. In their view, RRI urges scientists as well as policymakers to rethink the relation between science and society. The current governance landscape of new and emerging research and innovation is not yet properly equipped to deal with the new demands of RRI.

Soft Governance

Several articles suggested implementing RRI via so-called soft governance measures. Sometimes these can take place at the small scale, for example, individual research and innovation organisations can adopt codes of conduct as a way to show which values and norms guide their actions, which increases their trustworthiness (Asveld et al. 2015). Articles to enable practices of RRI were not limited to the governance of individual organisations, research fields, or groups of researchers. Meanwhile, Lee (2012) drew attention to the soft law policy framework of RRI. He pointed out that the object, scale and governance of the soft law policy framework of research and innovation needs to carefully consider these aspects of policies to produce desired changes and outcomes.

Mixed Governance

Finally, Ellwood et al. (2013) proposed a combination of hard policy measures and voluntary efforts to realise RRI. The debate on RRI is, in their view, related to the capability life cycle debate. They suggested that innovation governance can consist of prescribed actions, for example by means of regulations regarding the ability to re-use a technology at the end of its life, and voluntary actions such as the desire to be a socially responsible corporation or research institute.

Cross-Cutting Considerations and General Observations

Based on the four themes, several general observations and cross-cutting considerations can be made. The analysis showed that practices targeted different stages in the research and innovation process. Only one article described a practice that started during a call for funding (Owen and Goldberg 2010). Other articles described practices which started early in or in the middle of the research process. Examples are the studies on midstream modulation (e.g. Flipse et al. 2013a) or the studies describing value-sensitive design (Van der Hoven 2013; Timmermans et al. 2011). Other articles described practices that are more suitable to implement later in or even after the research and innovation process, such as reporting on the outcomes of the research (De Jong et al. 2015). Therefore, practices of RRI can be found in all stages of a research process, although it is not common yet to apply RRI values or include RRI characteristics at the proposal stage.

The practices of RRI varied with respect to the number of researchers as well as stakeholder groups involved. In the studies on midstream modulation, the target group was small and only consisted of the researchers the embedded humanist works with (e.g. Schuurbiers 2011). Other studies targeted a larger group, such as a group of invited stakeholders to be present during a meeting. An example is the workshop on synthetic biology (Douglas and Stemerding 2013). Practices of RRI, however, can also target a large group. Societal dialogues are an example of such a practice (Simaková and Coenen 2013).

In the majority of the articles, some level of participation is required of the targeted group. There are, however, differences in levels of engagement. In more outreach-style practices, such as a balanced newspaper reporting (De Jong et al. 2015), readers were invited to read the stories. More engagement and active input from participants is expected in studies describing engagement workshops (Robinson 2009), stimulating reflection (Owen and Goldberg 2010), and anticipation of impacts and consequences (Fisher and Rip 2013). Despite that RRI practices can reach or involve large groups with different levels of engagement, the findings did not make clear what reasons or local circumstances influence the selection or tailoring of the practices.

Authors described RRI practices that influence their own research trajectories and findings (e.g. workshops to discuss research findings). Other practices focused on nudging other researchers to be more sensitive to RRI. When stimulating researchers to reflect on societal views (Flipse et al. 2013a) or to complete a risk register (Owen and Goldberg 2010), such practices influence the processes of other researchers. RRI practices can thus relate both to one’s own and to other researcher’s work and this is probably is a factor in selecting a practice.

Another topic is how elaborate the practice was described and how detailed the suggestions for a practice were. Some articles dealt with loose ideas, for example, Grinbaum (2013) analysed the idea to use literary classics to stimulate discussion and while the meaning of Mary Shelley’s Frankenstein and the Golem narrative was discussed and it became not immediately clear how these stories can be used in current practices of RRI. Other articles included a proposal for bringing RRI into practice, but yet did not consider all details. For instance, De Jong et al. (2015) did not clarify how they intended to deal with freedom of press or journalistic freedom and newspapers preferring not to change how they report on neurotechnologies. Meanwhile, some studies described practices with detailed steps, such as the idea for responsible development of contested infrastructure projects (Ravesteijn et al. 2014). Other articles gave specific guiding questions or checklists. Brey (2012) is an example of the latter.

Related to the description of steps but specifically about the evaluation of the practice itself, an evaluation of the practice was often not included. An exemption is the article by Gemen et al. (2015) who emphasised the value of evaluating practices and included findings of their evaluation. Both a description of relevant details and evaluation findings can add to the growing understanding of practicing RRI.

Conclusion and Discussion

This review has aimed to gain insights from practices of RRI. These practices can inform theoretical as well as practical development of RRI. In total, 52 articles were reviewed (see Table 2) and related to four overarching themes which described a rich variety in values, dimensions or characteristics of RRI.

The first theme categorised practices related to inclusion or opening up research and innovation to stakeholders and the public via either science education and outreach or by means of including stakeholders. For example, stories and narratives can encourage ethical thinking on responsibility (Grinbaum and Groves 2013) while formats for participation as well as criteria provide insight in stimulating active engagement (Gemen et al. 2015) and societal dialogues should be seen as a process rather than an outcome (Krabbenborg and Mulder 2015). The second theme focused on stimulating reflection on broader aspects in the research process. That can happen at the start of the research process when applying for funding (Owen and Goldberg 2010) or during the process, for example, via sharing insights or collecting knowledge (Betten et al. 2013; Stahl et al. 2015). The practices in the third theme all related to managing ethical, legal, and social issues of research within a researcher’s own research and in both an early stage of the research process when outcomes still can steer the process, and, later, for future implementation. The articles in this group describe various methods, such as foresight or TA (in Von Schomberg 2013b) and design processes (Ikonen et al. 2015; Van der Hoven 2013) which will enable such an implementation. In the fourth theme, the practices described a level of institutionalisation of responsibility. This can happen not only at the governance level (Asveld et al. 2015) but also, for example, by applying both soft and hard policy measures (Lee 2012).

The definitions discussed at the start each emphasized different characteristics, dimensions, or values of RRI. The articles included in the corpus reflect many of these characteristics, dimensions, or values and show that RRI is already implemented in the practices in a rich and varied way. For example, Timmermans et al.’ (2011) work on value sensitive design provides functional requirements for the design and development of research (Directorate-General for Research and Innovation 2013) as well as product suggestions (Von Schomberg 2013a), while RRI dimensions (Stilgoe et al. 2013) can be recognised in various studies: anticipation is part of the study conducted by Brey (2012), reflexivity is found in Fisher and Rip (2013), while De Bakker et al. (2014) address inclusion, and Gemen et al. (2015) deal with responsiveness. The process dimension of RRI is shown in the article by Ravesteijn et al. (2014) with their ideas on facilitating and managing the consultation of stakeholders. The RRI characteristics open access and gender were not identified in the sample, however, other European Commission RRI characteristics were. For example, enhancing responsibility in research and innovation through governance was sought by Spruit et al. (2015); while Van der Hoven (2013) addressed ethics. Science education and public engagement were addressed in various articles such as the studies about journalism (De Jong et al. 2015), stakeholder involvement (e.g. Gemen et al. 2015) and societal dialogues (e.g. Krabbenborg and Mulder 2015) It means that the uptake of RRI in practices is well-underway as is shown in the review results.

Yet, while some articles focused on more than the characteristic, dimension, or value they were grouped into, most articles appear to focus on one RRI aspect. This raises the question why this is the case. The categorisation into four groups of practices might have eliminated some of the more nuances of additional RRI characteristics, dimensions, and values in the practices. However, it appears that that is not the only explanation. While it is to be assumed that the definitions of RRI have been developed with not only conceptual validity but also practical usage in mind, it could be hypothesized that the characteristics, dimensions, and values presented in the definitions overburden either the research process, or the (streamlined) presentation in journals thereof. Further research is needed to explain why research practices tend to focus on certain characteristics, dimensions, or values and whether and how the conceptual complexity of the RRI discussion contributes to it. After all, if RRI with all its elements is too complex to be realised in practice, maybe focusing on specific characteristics, dimensions, or values will lead to a more complete uptake in practice.

To conclude, three points can be drawn from this review study which may be given attention in future studies. First, articles dealing with RRI practices can be improved by including the reasons to select the practice, the methodological decisions which shaped the practice, and the results of the evaluation. Often, such information was scarce or even missing. Local circumstances play an important role in selecting and shaping practices, as is stressed in the NUCLEUS Project (Dijkstra et al. 2017; NUCLEUS Annual Conference Report 2017). As Gemen et al. (2015) pointed out, evaluations provide valuable information for both the authors and others. A better understanding of which practices are suited for which circumstances, which methodological decisions tailored a practice, and how successful this practice was, will lead to a knowledge-base that can aid future practitioners.

Second, including more reflection on the theoretical implications of a practice will contribute to further the conceptual RRI discourse. Burget et al. (2017) showed that the discourse is still developing, which aligns with the findings. Not all articles included a reflection on the implications of their RRI practice for the understanding or theoretical underpinning of the concept. Including such implications, when there are relevant ones, can aid the theoretical understanding of RRI.

Third, the practices described in the articles focus rather on opening up research and innovation than on early anticipation of their impacts. A possible explanation is that opening up research is a more important value than anticipating impacts. It may be that—after further experiments and debates—RRI definitions will distinguish between essential and supporting values or characteristics of RRI. In addition to insights from single practices, reviews of particular types of practices might be able to contribute too. For instance, Reijers et al. (2018) published a review of RRI practices focused on ethics. More cross-references between the practice-oriented and the conceptual-oriented contributions to the RRI discourse can foster mutual responsiveness between and progress in RRI theory and practice. It would be interesting to learn from other studies which include early practices and compare their effect on the research process.

Despite the enriched insights about the current state-of-the-art of RRI practices, it has to be stressed that—for sake of time and efforts—the review was based on a selection of databases while grey literature, such as reports about RRI, was not included. Reports, for example, deliverables from EU projects, may contain additional rich insights. In addition, results from the review are only exploratory and indicative in nature. Hence, insights may need to be substantiated by further research.

In all, the RRI discourse is relatively young and the concept and practices will develop further. When more RRI practices are analysed and published, a knowledge base will emerge that both future researchers as well as practitioners can consult. In this paper a first overview of practices is discussed, however, more practices and further research will be valuable for the growing understanding of RRI.