Introduction

In 1979, Bruno Latour and Steve Woolgar introduced the cycle of credit, based on extensive ethnographic observations in a life sciences laboratory (Latour and Woolgar 1986). This model, also known as the credibility cycle (CC), depicts the knowledge production process in a number of repetitive steps, drawing particular attention to the bidirectional relationships between efforts of researchers and (symbolic) rewards they earn: researchers earn credit for their contribution to scientific knowledge production, which they can subsequently invest as resources to generate new research activities. This quasi-economic model of science is generally seen as a significant contribution to the field of science studies, because it has improved the understanding of the activities of researchers in relation to the scientific incentive and reward structure. The fundamental assumption of the model is that, in order to participate successfully in the knowledge production process, researchers need to accumulate credibility.

Over the past few decades, the CC has proven a powerful analytical tool. It has been used both as a generic model of scientific practice and to investigate variation in research practices over time. Scholars have suggested several additions and modifications to the model, for example, the possibility of patenting as a source of credibility (Packer and Webster 1996), the linkage of the CC to a citation cycle that has developed because of the rise of comprehensive citation databases (Wouters 1997), and the possibility of practical applications of research results to serve as a source of credibility (Hessels and van Lente 2011).

In this literature, however, the CC has been treated as a rather generic representation of the knowledge production process, without differentiating across disciplines or research areas. Given the differences in social organization (Whitley 2000), epistemic cultures (Knorr-Cetina 1999) and academic cultures (Becher and Trowler 2001), we can expect significant differences between research areas in the way researchers accumulate credibility. The literature about diversity of research practices suggests differences in the way researchers earn credibility, but these have not been sufficiently articulated. Whitley (2000) has shown how task uncertainty and mutual dependence vary across disciplines, but he does not elaborate on how these variables influence the credibility conversions. Moreover, his framework is mainly theoretical, lacking an empirical foundation. In ethnographic science studies, some authors have made comparative analyses (Knorr-Cetina 1999; Collins 1992) but these mainly focus on natural sciences, and they often neglect the organizational and financial aspects of running a research group. Becher and Trowler (2001) have introduced a useful analytical distinction between urban and rural academic cultures, but their analysis focuses mainly on variation in the writing and publication practices of different research areas, paying little attention to the different practices of knowledge production. There is also a vast literature in scientometrics about different publication practices across scientific fields (Costas et al. 2015; Tsai et al. 2016; Newman 2004), but in this literature credibility is usually reduced to citations, for methodological reasons.

Altogether, an empirically supported systematic understanding of disciplinary differences in the relationship between knowledge production and credibility is still lacking. Against this background, this paper aims to refine the CC based on a set of case studies of four epistemic cultures. In contrast to earlier studies, this paper analyzes four very diverse epistemic cultures in an attempt to acquire a reasonable coverage of the wide variety of academic research practices. Based on this empirical foundation, we propose a number of additions to the CC that can make it more sensitive to the heterogeneity of academic knowledge production.

In the following, we first introduce our theoretical framework, which is rooted in constructivist science studies (Latour and Woolgar 1986; Knorr-Cetina 1999) and valuation studies (Lamont 2012). Next we introduce the data and methods we have used for our analysis. We then discuss our observations regarding the CC in the four research groups we have studied. Subsequently, we make a comparative analysis and attempt to explain the differences we have found. We conclude with discussing our theoretical contribution and implications for further research.

Theoretical Framework

The credibility cycle (CC) provides an abstracted representation of the process of knowledge production, and shows how researchers or research groups accumulate credibility over time in a cyclical process. In this model, the construction of knowledge that comes to count as facts is inherently connected to the accumulation of credibility. Conceived in this way, the research process can be depicted as a repetitive cycle in which conversions take place between money, staff, data, arguments, articles, recognition, and so on (see Fig. 1). According to Latour and Woolgar, credibility is a form of credit that scientists need to be able to continue their work. In contrast to credit as reward, which symbolizes peers’ recognition of a past scientific achievement, credibility is a form of credit that concerns ‘scientists’ abilities actually to do science’ (Latour and Woolgar 1986: 198). The essential feature of the CC is that the acquisition of credibility enables a researcher to reinvest it and gain more credibility. In this sense, credibility can be regarded as capital, coming in different forms. While some may conceive recognition as a synonym for credibility, in the model of Latour and Woolgar it constitutes one of its six manifestations.

Fig. 1
figure 1

The credibility cycle (Latour and Woolgar 1986)

Full size image

This paper proposes a refinement of the CC, in order to make the CC sensitive to disciplinary differences and to address some of the critical reactions that it has received in the science studies literature. First, we will connect the CC to community structures and institutional incentives, in response to Knorr-Cetina, who has warned about the danger of internalism when using quasi-economic models of science such as the credibility cycle (Knorr-Cetina 1982). One should not conceive researchers as if ‘they were isolated in a selfcontained, quasi-independent system’ (Knorr-Cetina 1982: 109). The activities or researchers go beyond the boundaries of their specialist community. They receive recognition from peers, but also from academic colleagues in other disciplines and various actors in society. In many cases, teaching tasks are intertwined with the research and as such enter into the CC as well. What is more, the way research groups acquire credibility and build a reputation depends strongly on their funding sources and their orientation to other external audiences (Joly and Mangematin 1996). In order to overcome this limitation of the original CC, we conceive the CC in the context of an institutional environment. The conversions of resources in the CC are ruled not only by scientific processes, but partly also by external actors, including funding agencies, science policy makers, students and the general public (Hessels et al. 2009).

We believe that this position should also solve a concern uttered by Whitley (1983) that the CC neglects the structure of social communities. In his reading the CC addresses only the motives and behavior of individual researchers, which ‘seem to follow extremely short term goals and be highly flexible in changing them when circumstances change’ (p. 702). According to Whitley, the CC in this way dismisses the influence of collective ordering principles and goals in the development of knowledge. In the current study we take as a starting point that the collective ordering principles of academic research are embedded in the institutions that guide the credibility conversions.

Another criticism from Knorr-Cetina is that quasi-economic models of scientific activity start from unrealistic assumptions about the drivers of human behavior (Knorr-Cetina 1982). She stresses that agents often do not consciously calculate the outcomes of their activities (p. 105). In our understanding, however, the accumulation of credibility is not an end in itself. Neither is it necessary to assume that (all) researchers consciously calculate the benefits of all their activities in terms of a CC. We rather view the CC as an analytical model of the steps that researchers go through in their continued research efforts.

Our refinement of the CC model is also informed by valuation studies. The attribution of credibility can be regarded as instances of valuation: the process in which the value of goods, people, and practices is constructed (Lamont 2012; Stark 2011). In this sense, the credibility cycle is a model of the relationship between knowledge production and valuation processes. It shows how particular knowledge production activities and outcomes become valuable. In a recent paper on epistemic capitalism, Fochler (2016) has shown several forms of value ‘made durable’ as capital in academia and industry. Fochler indicates that scientific practices can be characterized by a particular cultural way of producing, attributing, and accumulating specific forms of worth which can but need not be monetary. In this paper, we will extend this perspective by exploring the differences between epistemic cultures in the way different aspects of knowledge production become valuable. As value is socially and culturally constructed, the manifestation of the different forms of credibility will also vary across research practices. This motivates us to look beyond a generic CC model, in an attempt to develop a model that does justice to the diversity of science. The paper focuses on the mechanisms behind the six different conversions of the CC, and presents an empirical analysis of the social and cultural practices that imbue research data, arguments, people, grants, and publications with value.

In the science studies literature there are several concepts to characterize differences between scientific disciplines such as strategic task uncertainty, search regimes, communication cultures and epistemic cultures. Here we use epistemic cultures (Knorr-Cetina 1999) as a heuristic tool to characterize different research areas,Footnote 1 in order to direct more attention to the epistemic dimension of knowledge production. The notion of epistemic cultures enables us to get beyond the traditional way of capturing differentiation of knowledge in terms of disciplines of scientific specialties. Though the concepts of disciplines and specialties are helpful for grasping the organizational principles and structures of research practices, they cannot capture important strategies and policies of knowing that are not codified in textbooks but do inform research practices. By emphasizing configurations of persons and objects rather than institutions, the notion of epistemic cultures makes research practices and epistemic objects central to the analysis, which are also essential elements of the credibility cycle. Below, we will compare the way research groups accumulate credibility across four different epistemic cultures to explore disciplinary variation in how value is ascribed to research activities, objects, and people in processes of knowledge production.

Although the CC focuses in its original presentation on the activities of individual researchers, in this paper we apply the model to research groups. The cyclical process of accumulating credibility at this level can also be understood in terms of the CC, since it boils down to the same principles as on the individual level (Leišytė 2007; Lepori et al. 2016).

For each epistemic culture, we systematically analyze the accumulation of recognition by juxtaposing the six forms of credibility with different valuation processes generating this recognition, and the underlying types of activities and achievements involved. We pay attention to: manifestations of data (types of resource intensity, e.g., labor/capital; function of data in the epistemic process); the generation of arguments (collective vs. individual; creative vs. formalized), writing of publications (collective vs. individual; creative vs. formalized), sources of recognition (publications vs. other output such as policy advice, contributions to societal debates or prototypes; direct recognition for data or arguments), typical funding sources and associated selection criteria, type of capital required for the generation of data or arguments (relative importance of staff vs. research facilities; types of facilities: standard vs. advanced/expensive).

Methods

Our analysis is based on case studies of four research groups in social sciences, humanities, geosciences and mathematics, respectively. The four groups represent very different epistemic cultures. They use different types of research facilities, ranging from laboratory equipment to digital archives. The groups also vary in terms of the common practices of collaboration and co-authorships. In mathematics and humanities it is more common to publish individually while in the other two research areas co-authored publications are most common.

All groups included in this study are high-performing groups with an excellent reputationFootnote 2. All four groups include at least one member that has won a Spinoza prize from the Dutch research council NWO or an Advanced Grant from the European Research Council (ERC). Both grants amount to about 2.5 Million Euros and serve as indicator of academic excellence (Hessels et al. 2016). This common characteristic increases the comparability of the cases. In order to protect our respondents, this paper will not reveal the particular groups we studied. For the same reason we also conceal the gender of respondents and refer to all group leaders as female.

We carried out interviews with approximately ten researchers for each case. In each group we interviewed the group leader at least twice. In addition we conducted semi-structured interviews with a number of other researchers from the group in various positions. We also interviewed a couple of relevant individuals outside of each group, researchers who do not work in the particular group but who do have a clear view on it, because they work in the same institute, have previously worked in the group or carry managerial responsibility for the group. Our interview questions are derived from the various parts of the credibility cycle and deal with topics such as the group’s research activities, internal and external collaborations, acquiring research funding, HR policies, publication strategies and experiences with acquiring and attributing academic recognition. The interview data were categorized using the interview topics as the initial coding categories (supported by Atlas.ti). Quotes or other data that we could not categorize in the existing categories were coded under a new category.

For each case, we also studied relevant documents, such as self-assessments, research evaluation reports, financial reports, research proposals or mid-term reviews. In addition, in each group we observed a couple of meetings in order to get a better understanding of the interactions among group members. We have written confidential case reports with our detailed analysis of each group, which were validated by the group leaders.

Results

In this section we will present our case studies of four epistemic cultures. In each case we will analyze valuation practices observed at the different positions in the CC. After a brief introduction, the case reports all follow the logical order of the six central resources in the CC: staff and equipment, data, arguments, publications, recognition and money. In some cases we discuss two resources in combination, because we found them to be integrated in these epistemic cultures.

Social Science

The social science group answers fundamental sociological and demographic, and in some cases policy-relevant, research questions mainly by using large international datasets. The group’s core team consists of two PhD-students, one postdoc researcher, one senior researcher and the group leader (full professor). This core team is embedded in a larger team of affiliated researchers working on similar topics.

Staff and Equipment

For the group’s research no physical devices are necessary to generate data, except for a regular computer to do statistical analyses. Therefore, almost all funding can be allocated to human resources. The group leader selects her staff based on scholarly expertise (including statistical skills), communicative capacity and a social match with the group in order to develop a competent, motivated and loyal team of researchers.

Data

The main deviation from the original credibility cycle concerns the resource data. We observed two different practices of data acquisition in this group.

In the first practice the researchers acquire their data from existing large-scale and international data sets. These are often open data sets, or at least easily accessible to them. The data were collected at an earlier stage by (international) networks of researchers solely committed to the data collection, and this group adds value to these data by linking them directly to the production of arguments. These activities concern data analyses that will constitute the main arguments for the publications. The idea of a desirable end-product (publication) in some cases significantly guides the data analyses, as shown by the following quote.

Associate professor: ‘We see a kind of bias that most journals don’t want research with no significant results. Because then there must have been something wrong with the data or how you formulated you research question. So what you do is think: what other factors, that didn’t come up in our theoretical framework, could be of influence? And that is actually a kind of data mining: where can I get something significant? Because you want to get it published.’

The second practice concerns large-scale data collections by group members themselves. In the past, the group leader was one of the initiators of a large-scale data collection that has been important for the theoretical and empirical development of her sub-discipline. The fact that this survey has been repeated several times and served as the blueprint for other international surveys increases the value of the dataset.

Thanks to a large and prestigious grant the group leader obtained, the group had the occasion to start a new large-scale data collection among a specific population. One of the group members has worked for two years on the data collection, and therefore could not find the time to do any data analyses, but she is recognized by the group leader for what is seen as her service to the discipline. After all, other researchers can make use of the group’s data collection as well.

Postdoctoral researcher: ‘I invested two years in the infrastructure, that’s the way to look at it. Because the data will not only be used by me and my two or three colleagues. But in these two years I hardly had any time to publish myself. […] That was the idea of the professor: “you invested your time and I think you should be rewarded for that with a new contract. Then you can show what you are actually capable of”.’

The group does not only need the staff to set up a survey and collect the data, but also a well-functioning data infrastructure. The infrastructure consists of a network of peers contributing to the data collection, the software, digital network and devices to gather and process the data, and in some cases an external partner to carry out all the technicalities. The data infrastructure, in this case, can be seen as an indispensable facility.

Arguments

The production of arguments and doing statistical analyses is a rather individual process, often conducted by PhD-students or post-doc researchers. In this culture the data analysis is typically structured by the format of a scientific paper, in the sense that the analysis is informed by its target journal and a manuscript outline. A project design typically consists of a number of ‘papers,’ indicating that these papers do not only serve to communicate the results but also to organize the work.

Publications

Generating publications again is a collective process. A researcher may write a first draft by herself, but will then discuss the theoretical positioning, the methodological limitations and the implications of the findings with peers and co-authors. The group carefully chooses the target journal for each paper. The group leader is involved in decisions about the choice of journal, the framing of the content, and the final writing itself. The group leader is praised by the other group members for her skills to get papers published.

Recognition

Regarding the accumulation of recognition, the sixth form of credibility, we found four different valuation processes generating this recognition, each based on different types of activities and achievements. Generally speaking, publications in journals with a high impact factor and number of citations are seen as important signs of recognition for the group and its group members. This implies that acceptance of a manuscript by a prestigious journal increases the value of the text: it marks the article with a quality indicator, which increases the degree of recognition that the authors receive for the underlying research work. Second, the group leader gains recognition through her visibility in several arenas. In addition to her strong publication record, she holds multiple managerial functions and intervenes in certain public policy issues. New large-scale data sets constitute a third source of recognition. The group leader's contribution to a large-scale data collection is appreciated by peers as service to this sub-discipline. The group leader gained recognition because of this data set. With support of a large infrastructure grant she has led the design and execution of a large-scale and innovative survey. Over the years, several waves of this survey have been carried out and also social scientists outside her group work with the data from the survey. Furthermore, a large-scale American survey was modeled after her initial questionnaire. In her own words: ‘The [data collection] does not give publications, but it gives prestige.’ This suggests that there is a shortcut in the credibility cycle: it jumps from data directly to recognition. This shortcut does not imply that the social scientists in the group could well do without publishing. It is rather the combination of using existing data and collecting new data that characterizes the social sciences group best.

A fourth valuation process producing recognition is more direct and can be described as a mini-cycle from money to recognition. In certain grant schemes research excellence – of the proposal and of the applicant – is the decisive criterion for awarding the grants. And since the number of applicants is much higher than the number of grants, colleagues and superiors of grantees tend to give recognition for obtaining these grants. The grant does not only bear monetary capital but also symbolic capital, and becomes an indicator of excellence, even when the other elements of the credibility cycle have not (yet) been brought into practice. The group members explain that they prefer excellence grants over other external funding because of the recognition and prestige the grants provide.

Postdoctoral researcher: ‘[Obtaining excellence grants] really makes a difference for your reputation and the doors that open up. Other grants are splendid, because they give the opportunity to work on things you’re really interested in. But when you have excellence grants, then of course they provide more prestige. We all aim for the external [excellence grants], because they give you the most reputation. Once you have [an excellence grant for young researchers], for example, the number of doors that open up is impressive.’

And after the group leader obtained a large excellence grant the university and department administrators used her as a prime example of their research quality.

Money

In the social science group, the acquisition of money depends to a great deal on external funding. Its overarching department has cut back on expenses, so the remaining block-grant funding is fully required for the permanent positions of more experienced staff. Project-based external funding is needed to be able to hire young researchers and to ‘buy’ research time for the (assistant/associate/full) professors. The group members are expected to apply for suitable grants, most often from the national research council (NWO) and the European Research Council. The group leader is closely involved in the application process, to make all group members benefit from her experience both in writing research proposals and in participating in grant committees. In this way the group has managed to acquire an increasing number of large research grants over the past few years. Together these are sufficient to hire a growing number of staff that can continue and expand the research activities of the group.

Geoscience

The geoscience group operates at the interface of three natural sciences. The group exists since the mid-1970s and currently comprises two full professors, three senior researchers, a few technicians, and a larger number of PhD students and post-doc researchers. As the institute does not house any educational programs, all group members are focused on research. The epistemic culture is characterized by a relatively high output (in numbers of peer-reviewed articles) especially compared to the groups in mathematics and humanities. The conversion rate between data, articles and recognition through citations is very high. For funding acquisition, however, the group follows the rhythm of NWO and ERC funding cycles, which is similar to the other groups.

Staff and Equipment

Both the collection and analysis of data are labor-intensive and expensive processes. The group studies soil samples from the bottom of lakes and oceans. After the soil samples have been collected on boat trips the analysis takes place in the lab. Researchers use a range of analytical chemistry techniques that require advanced and expensive equipment. Within the field, this group has one of the best equipped labs of in the world. PhD-students and post-docs spend a large amount of their time in this lab preparing samples and performing analyses. They are supported by permanent technicians and a research staff member who acts as an intermediary between the junior researchers, technicians and senior researchers.

Data

While PhD-students from the lab will usually work with data collected in the group, senior researchers are often approached by outside groups for collaborations because of the lab’s quality and reputation in the field. In these cases an outside group is eager to use the lab’s expertise in certain analytical techniques to analyze particular soil samples. Research in this group, and the field at large, is characterized by collaboration in the set-up, analysis of results and writing phase (while the preparation and analysis of the data is done by the junior member of a collaboration in consultation with technicians and senior researchers). One of the senior researchers explains the supervision process:

Interviewer: ‘How much time of your week would you think you are working either with them face-to-face or on their papers?’

Senior researcher: ‘I would say like maybe 70%. (…) Well I have a lot of PhD-students [she has four]. It is a very direct supervision. Because it is not like, of course [group leader] has many meetings to attend and she is like really on top of everything the same way as the staff and, but for us it is more like the ground-work. We are here for the day-by-day kind of thing. There is always things that need to be fixed or they need to be supervised in a specific technique or something. It gets better. I mean, when they get a little bit further in their thesis they become a little bit more independent. But of course I really like to check on them, if it is possible once a day. To make sure that everything is on track.’

Arguments and Publications

After data have been analyzed, they are made valuable by developing the argument in the writing process. The writing process is itself collaborative. Often a PhD-student and day-to-day supervisor (often a senior researcher) together prepare a first draft; the group leader or second professor in the group typically engages with the text at a later stage. After the manuscript has been written, almost always in the form of a journal article, it is submitted to a journal that fits with the scope of the argument. This can be a general science journal such as Nature, Nature Communications or Science or one of the disciplinary journals.

Recognition

These arguments and publications become valuable mainly through citations in journal articles. However, in interviews the members of the group also highlight the impact of their work in policy, such as the use of their work in documents prepared by the Intergovernmental Panel on Climate Change (IPCC). Citations, more than publication counts, are important measures of recognition in this field. One of the professors explains:

Professor: ‘Citations are, talking about innovation, a reasonable indicator, not the total amount, but for individual papers gives a reasonable impression of the importance of the advance.’

Recognition through citations is also important to succeed in the next valuation process, obtaining funding. The group leader, a frequent member of grant committees of NWO and the ERC, explained that in the ERC consolidator grant committee she would expect a prospective grantee to have obtained at least 500 citations, as an indication of being an established and productive researcher. Citations are seen as markers of both the quality and relevance of research lines.

Money

Obtaining external project funding is crucial in the career of all researchers in this group. Permanent positions are rare in this field. After completing their dissertation, researchers who aim to stay in academia have to obtain a position in a large externally funded project or have to apply for funding for their own post-doc position. For mid-career researchers, obtaining a large grant is crucial to establish themselves with the research group and have resources to collect data and hire PhD-students. External grants carry a symbolic value as markers of quality, too, especially the highly prestigious grants from NWO and the ERC. Early and mid-career researchers can establish their own research groups through these grants. Typically, they would first obtain funding for a limited number of PhD-students (often in larger grants written with the group leader) and start a group within the larger research group. Through subsequent grant funding, if successful, such a group could grow and establish itself as an autonomous organizational entity and start to collect data on their own and be recognized for their specific expertise in particular methods and forms of analysis.

Mathematics

The mathematics group is part of an institute that, in general terms, consists of two parts: one focusing on applied mathematics such as stochastics and one studying fundamental mathematics such as algebra. Our case study deals with the latter part of the institute. The group consists of approximately 50 researchers, including 30 PhD students. The group is loosely organized around the sub-disciplines of four full professors, but the main unit of research is the individual.

Staff and Equipment

Because the group members do not use any equipment other than computers, the group can spend most of its funding on human resources. The many grants and prizes the senior group members have obtained have enabled them to attract a large number of younger group members working on fixed-term contracts. In addition, the group has occasionally attracted high-performing researchers for senior positions that have helped to acquire prestigious grants and awards. The financial viability of the group made it possible to appoint these top researchers independent of the formal group structure.

Data and Arguments

The fundamental mathematicians of this group do not work with data but only with axioms, because of the formal and theoretical nature of their field. The researchers aim to solve theoretical mathematical problems or to prove (or disprove) conjectures. The group members do work with trial and error to test ideas and formulas, but these trials have no connection to empirical data. They are rather mental or digital exercises. In terms of the credibility cycle, this group skips the data step and converts human resources directly into arguments, in the form of solutions and proofs.

Despite a strong feeling of unity within the group, the production of arguments is a solitary endeavor. The researchers do help each other with their problems, puzzles or conjectures, but most of the time they are not sufficiently similarly specialized to work on an argument together. Cooperation, in terms of co-authorship, mostly takes place with colleagues outside their own group and university.

The absence of equipment and data and the individual nature of the research make the conversion from staff to arguments a highly flexible and unpredictable process.

Professor: ‘It’s like cutting through the bush with a machete: you solve one problem after the other and when you continue for a hundred meters, you have a dissertation. But imagine that after ten meters you reach an open pathway and no one has ever been there. Well, then you follow that path! You won’t go and solve the original problem. And in mathematics, you didn’t make any investments, didn’t hire any analysts or specialists to man certain equipment. From one day to another you can change your research. […] That’s the flexibility.’

Some of the group members explain that they have often worked on an idea that ultimately proved fruitless. When they found out, they changed their approach or switched to another problem. Other experiences resemble the open pathway the professor describes in the previous quote. Occasionally one finds a solution for a problem while working on another problem. In other stories one comes up with a breakthrough idea in unexpected moments (under the shower or while brushing one’s teeth). These dynamics imply that the pace of the research process fluctuates.

Publications

For mathematicians, the form in which an argument is presented (and assessed by peers) is straightforward. A solution to a problem is either right or it is wrong. A theorem is either proven or it is not.

Professor: ‘It is of course very different if you study [another discipline]… if you ask a lot of people how they think about something, I mean, that is really different. In mathematics you have to construct the proof, you must have a theorem for which you want to give proof. That is just your work and not the result of a measurement or something, it is just proof. You must construct it and others will check if it is correct.’

Therefore, a solid argument on its own can sometimes be valued more by scientific peers than a publication in a highly ranked journal. Other mathematicians do not strictly need an (anonymous) journal peer review process to assess a contribution and to attribute recognition. In order to accelerate communication with their peers, most group members publish their (draft) papers in an online and open archive, before the paper is published in a peer reviewed journal – which is still common practice as well. A good and solid argument can already gain ample recognition when it is only published on the online archive. A famous example of this valuation process is a series of publications of the Russian mathematician Grigori Perelman in 2002 and 2003. He did not submit these papers to a journal, but only made them available in an online and open archive. Then it was immediately recognized that the papers provided proof for the Poincaré conjecture. Perelman was awarded a million US-dollars from the Clay Mathematics Institute for his achievement, because the institute had selected the Poincaré conjecture as one of the seven most important unsolved mathematical problems. The emphasis on conference papers and invitations as a speaker or guest lecturer is another example of the relative importance of arguments and ideas over the formal publications of these arguments and ideas.

Recognition

In this case of epistemic culture we witness two parallel processes in the valuation of research output. While (1) mathematicians typically gain informal recognition form peers based directly on arguments rather than publications, peer reviewed journal publications (2) are a source of formal recognition, in the sense that they add value to a competitive resume. In the formal procedures of grant allocation, recruitment of staff and tenure track decisions, publications in highly ranked journals function as a valuable resource.

Money

The fundamental mathematicians of this group fund their activities with two main sources of money, which depend directly and indirectly on (formal) recognition. First, external excellence grants constitute a substantial and growing share of the group budget. Every eligible group member is encouraged to apply for these grants, for which individual research excellence is a decisive criterion. Candidates consult other group members with experience in grant applications or grant committees in order to write the best possible grant application. In comparison to similar mathematics groups, in the Netherlands the group has been very successful in obtaining these grants. The second large share of the group budget consists of university funding, the amount of which mainly depends on education parameters (e.g., number of students) and the amount of external funding. This means that obtaining excellence grants creates a multiplier effect. Because the group hardly needs any research equipment, it can use both types of funding mainly to hire the staff necessary for conducting the desired research.

Humanities

The humanities group acts on the interface of comparative literature, European studies and cultural history. The professor of the group aims to develop a database which maps material and immaterial cultural expressions, for a heterogeneous geographical area which consists of many cultural communities over a long period of time. Because of the scale of the project, this is only possible by collaborating with many others, both within and outside the research group. The group leader works on her project mainly with student assistants but also with (part-time) post-docs, a tenure track professor and a number of PhD-students who partake in particular sub-projects. The size and boundaries of the group itself (and the research project of the professor) cannot be precisely defined as these change depending on the task at hand.

Staff and Equipment

The research in this group does not require any advanced facilities, it mainly depends on the efforts of group members or personal connections of the group leader. The professor is knowledgeable of many cultural communities and cultural expressions she is interested in and directs a group of student-assistants towards those expressions she wants to include in the database.

Data

Student-assistants partly process data delivered by the professor and in this way search for data points following the professor’s lead. Moreover, outside experts in particular cultural expressions, such as musicology, or cultural communities from area or regional studies will often point to interesting cultural expressions in their area of expertise or share their own data with the group to be included in the database. In this iterative process new cultural expressions are also ‘discovered.’ An outside expert may suggest that a particular person or type of expression could be of interest or a certain source suddenly becomes digitally available, which enables the group to include data sources not known beforehand. For instance, a PhD-student who is writing her thesis supervised by the professor explains how she is involved as an expert in the research project:

PhD-student: ‘I wrote an article on [topic of dissertation] for the encyclopedia and sometimes I am consulted about what [type of artistic professional] to include, what could be interesting projects to develop in [name of the research program], in your area of expertise. So I have been consulted regularly as an expert, reading a new contribution to the encyclopedia to judge its quality, advise about who to ask for a certain topic in the encyclopedia, so also using my network and knowledge of [academic discipline], that are the most important things.’

The professor has also established an online encyclopedia connected to the database, which describes particular events, persons, types of expressions and cultural communities. While this encyclopedia is curated by the group itself, most of the contributions are written by outside experts who often participate in one of the workshops the group organizes. The success of this encyclopedia is partly based on the position and network of the group leader in the field. Her prominence has the effect that many outside experts are happy to write something for the encyclopedia, because their collaboration with this group leader can also function as a source of recognition for their own expertise.

Arguments

The processes of data analysis and data collection are intertwined. After analysis, new data is collected, and data collection again gives rise to new analysis. In interviews, the group leader highlighted the role of outside experts and stressed the networked nature of her research practice. Moreover, the database as well as the encyclopedia are open initiatives and available online as a scholarly resource for any interested scholars.

The professor has developed an influential theory through an analysis of different aspects of the cultural communities and expressions collected in the database. The work of PhD-students and post-docs also often engages with this framework, but they develop empirical expertise and specific arguments in a more individual way. Group members typically focus their empirical work on a limited number of topics, which could be a certain cultural expression, period or cultural community or combination of these. For instance, a PhD-student whose dissertation revolved around a particular form of cultural expression in a particular cultural community is expected to become an expert on this topic. Scholars can gather an extensive amount of knowledge around such a topic and each scholar in the group is expected to become an expert in at least one topic.

Publications

Connected to the expectation of individual topical expertise, publications are more often single-authored than multi-authored and appear more often in books than in journals. The professor herself manages to work across cultural communities and forms of expressions to develop theoretical insights drawing on the differences and similarities, but this is an exception rather than a rule. Because of the individual nature of the work, the credibility cycle moves more slowly. It takes more time to develop data and argument into output that can be recognized through, for instance, citations.

Workshops and conferences are important venues to present and discuss arguments. An edited volume following from a conference or workshop can in this sense be understood as a collaborative effort of the editors and authors of the chapters. They reflect a communal engagement with a particular theme and/or theoretical framework.

Recognition

Our interviews suggest that scholars in this epistemic culture can gain recognition for a wide range of activities. Recognition does not rest on academic publications alone. The professor, for instance, is recognized for her important role in a range of scholarly debates, but also for editing a yearbook for a number of years, speaking frequently at both academic and public events and for teaching in a large bachelor program in the humanities. Moreover, she played an important role as an expert in scientific committees and the development of new funding programs. The scholars in the group reflect this multidimensional ideal as all post-PhD group members are active both as lecturers and as researchers and find both activities very important. Moreover, all are actively engaging with audiences outside the university and service to the academic community.

Money

The relation between recognition and funding is not clear-cut in this field. Most scholars in and around the group conduct research in the limited research time they have or outside of professional hours. There are not many funding availabilities after the post-doc career phase to increase formal research time. Most early career scholars will apply for a competitive ´VENI´-grant from the national research council NWO, since this can help to obtain one of the few tenure track assistant professor positions in the field. However, given the fierce competition, being recognized as a good or even excellent scholar does not convert easily into research funding or a (permanent) position. The idea of starting their own research group was not mentioned by the early career scholars we interviewed. Their primary concern was obtaining research time for themselves.

In terms of the credibility cycle this case suggests that some groups in the humanities are not sustained by external grants but rather as networks of scholars with very variable (and precarious) academic positions. Moreover, as scholars are expected to develop as individual experts and as all-round academics, a lab-like group with post-docs with full-time research positions does not readily fit with this ideal. This implies that individual early career researchers are less oriented in converting recognition into money and staff, than in converting recognition into a (tenure track) position including research time as part of a broader portfolio of tasks. In other words, the valuation of recognition takes place in the appointment and promotion of academic staff rather than the allocation of research grants.

Comparative Analysis

Before addressing the variation in valuation processes across epistemic cultures, let us note that the cyclical process of valuation and accumulating credibility is clearly recognizable in all four cases. The research groups have earned recognition for various contributions, they benefit from this form of credibility when acquiring new research funding, use this money for staff and/or equipment, generate new data and/or arguments to write new papers, and eventually earn more recognition. In all four cases the credibility cycle has functioned very successfully over the past years, which has resulted in the accumulation of credibility in these groups.

In correspondence with our theoretical expectations, we found a number of differences in the valuation processes taking place in the CC. In this section, we will first discuss variation in the individual conversions that are part of the CC, and then make a comparative observation about the general speed of the CC. Some of our observations have implications for the shape of the CC, which are depicted in Fig. 2.

Fig. 2
figure 2

The implications of the variations found for the shape of the credibility cycle: (1) recognition as a source of data, (2) direct conversion of staff and equipment to arguments, and (3) additional sources of recognition (beside publications)

Full size image

Staff and equipment: While researchers in the geoscience group need to spend the money available to them both on expensive equipment and on people to be able to continue and expand their research lines, the other groups do not need expensive facilities so they invest mainly in people. The mathematics group uses the money as much as possible for hiring staff, the social science and humanities groups also use it for external support in data collection.

Data: as expected, we observed a variation in the way the groups acquire data, a central aspect of their epistemic cultures. In the mathematics case researchers hardly use any data. The researchers of the social science group do use data, but in most projects they don’t need to make the effort of collecting data themselves. These two epistemic cultures involve a shortcut in the CC, in the sense that they can turn staff directly into valuable arguments. This happens either when their work does not involve data at all (mathematics) or when they can conduct the required analyses on data that have been made available in earlier projects by themselves or others (social science).

The geoscience group and the humanities group invest a lot of resources in data collection, because their research approaches are data-intensive. The geoscience group does so by using expensive equipment.

The humanities group uses its network rather than money to acquire new data, indicating the valuation of social relationships. In this epistemic culture we observe a shortcut from recognition via networks to data. The group leader’s reputation helps to mobilize colleagues across the world to contribute to the collection of data, without any substantial payments.

Arguments and publications: across the four epistemic cultures studied, we found two different practices of creating arguments and publishing findings. In the humanities and mathematics case the production of arguments is a distinct step, separate from writing papers or books. In these cases the production of arguments is highly individual. And there are a number of different possible communication formats for making research results valuable, including conference proceedings, edited volumes or monographs. In the other two groups (social science and geoscience), the creation of arguments is more collaborative. And it often is embedded in the process of writing journal publications. These two groups have a strong culture of publishing in journals.

Recognition: as expected, we observed that recognition is not only based on publications. In all cases, publications function as sources of recognition, which implies that peer review is a crucial valuation process. But to a varying extent, other expressions, achievements or contributions are also recognized as valuable. Three of these are already represented in the CC: arguments, data and money. First, we found in the mathematics group that arguments can directly yield recognition, without the publication process required for their valuation and we observed in the humanities group that the group leader acquired recognition more for her arguments than for publications (although the most valuable arguments have been published). The communication practices in these two areas are not dominated by academic journals. Second, we found in the social science group that data can directly yield recognition, in the case that other research groups make them valuable by using them for their own analyses.

Third, we found weaker or stronger instances of a direct connection between money and recognition in all cases, which can be seen as a positive feedback loop in the CC. In this sense, money is not only the outcome but also the subject of a valuation practice. Some of the available grants and prizes, in particular the ones that can be called ‘excellence funding,’ are associated with such a degree of prestige and are so competitive that they are considered in the community as a direct indicator of scientific quality. This implies that beside the monetary value of the grants, they also allocate an amount of symbolic capital (recognition) to recipients, which increases their possibilities for acquiring additional funding. In this way, the grantees receive recognition based on their research potential rather than achievements, and are able to attract a larger amount of funding without going through the complete CC. This feedback loop resonates with the logic of some types of excellence funding (e.g., ERC), which explicitly aim to empower talented researchers and boost their careers. The existence of this feedback loop seems to be facilitated by the current scarcity of research positions and research funding, which makes research managers eager to hire staff based on their (potential) acquisition power rather than their research achievements alone. In this way a reseacher’s organization career and community career may become uncoupled from her cognitive career (Laudel and Gläser 2008).

But there are also other sources of recognition, which are located outside the CC. In all epistemic cultures studied apart from mathematics, we found that recognition can also be earned for work in committees, efforts in education, policy impacts or contributions to non-scientific debates. In other words, we have found that not only publications, but also data, arguments and other contributions are subject to valuation practices.

Money: As a consequence of our sampling strategy, we found relatively little variety in the way the groups acquire funding for their work. NWO and ERC are crucial for all groups, and the procedures of getting funding are universal. Beside these they employ a variety of sources, including Marie Curie, H2020 thematic projects and investment funds from their own university. None of the groups use a lot of funding from industry or other organizations that expect direct benefits from the projects.

Finally, we observed variation in the overall speed of the CC. In the epistemic cultures of social science and geoscience, we observed fast cycles in which data collection and the production of arguments and papers are efficiently organized and typically take about one or two years. We observed slower valuation practices in the humanities where data collection is time-consuming and the production of arguments a more creative process. In mathematics, we observed a combination of different paces. In this epistemic culture, the production of arguments is often a slow process, but an unexpected discovery can make the CC run very fast, instantaneously generating recognition for the solution of a fundamental problem.

Conclusions

In this paper we have compared the ways in which research groups accumulate credibility across four different epistemic cultures. On the basis of this, we propose to refine the credibility cycle (CC). Our analysis confirms the theoretical expectation that valuation practices vary strongly across epistemic cultures. In our empirical case studies many of the differences in the six credibility conversions that we expected were found. The variation we found in these valuation practices relates to a great deal to differences in epistemic cultures.

One variation that we did not expect concerns the connection between arguments and publications. In cultures where research teams collaboratively produce arguments, this work is more strongly embedded in the process of writing publications. The (aspired) journal publication can structure the valuation of research efforts by serving as a platform to specify different roles in the intellectual process and to assign potential credits to these in the form of an expected co-authorship (Larivière et al. 2016). By embedding the production of arguments into the writing of papers, the collaborating researchers create a sense of security that the efforts made by different people will be rewarded in the end (Strange 2008). However, the integration of the production of arguments with the writing of publications may pose a risk to creativity, because the intellectual process is strongly structured by the format of the associated communication medium.

The differences across epistemic cultures in the ways data are used and arguments are produced directly shape the bottom part of the CC, but they also have implications for the attribution of recognition. In an epistemic culture where it is common to work with data collected by peers, the production of a rich and relevant dataset can function as a direct source of recognition. This creates incentives to take good care of data management. And in an epistemic culture where the assessment of arguments does not only depend on the way it is presented in a paper, a peer review process is not required for their valuation, and the arguments can function as a direct source of recognition. This alleviates the publication pressure for researchers in these fields and encourages them to focus their attention on the content of their findings rather than the communication medium.

Strikingly, we also observed a new valuation practice that has received little attention in the literature so far: money as a direct source of recognition, generating a positive feedback loop between money and recognition. In all four groups, we have found that researchers who have received prestigious grants receive a significant amount of recognition for the acquisition of these particular grants. Money is here at the same time the outcome and the subject of valuation practices. This mechanism amplifies the ‘Matthew-effect’ (Merton 1968) which predicts that researchers that have acquired a certain level of recognition, more easily receive more recognition for their achievements than researchers that are relatively unknown in the academic community. In this way, prestigious excellence grants, which are of growing prominence in many science systems (Cremonini et al. 2017), may create a positive feedback loop that increases the inequality of resources in a given science system. Although the purpose of excellence funding is to selectively support top performance, policymakers need to reflect on the degree of inequality that is desirable.

We argue that, as an implication of the variation we found for research councils, and in line with earlier claims (Laudel and Gläser 2014), funding instruments could cater more to the specific needs of individual epistemic cultures. For example, epistemic cultures that depend more on staff than equipment (mathematics, humanities) for the production of data need smaller grants than cultures requiring costly facilities. The advancement of knowledge in these epistemic cultures would probably benefit more from a large amount of smaller grants than a small amount of large grants. Research councils, however, supply a substantial share of their funding using generic instruments.

This paper took a broad perspective on valuation processes by analyzing the whole process of knowledge production rather than focusing on the valuation of one particular object. In the science studies literature, the valuation of publications has received a lot of attention. Further research is needed to acquire a more sophisticated understanding of the mechanisms responsible for ascribing value to other objects, in particular research data, arguments and competitive grants.

In its original presentation (Latour and Woolgar 1986) the CC was introduced to characterize the drivers and patterns in the behavior of individual researchers in a molecular biology laboratory. Although the authors have never claimed universal validity, the science studies literature has adopted the CC as a rather generic analytical tool to describe the behavior of researchers or research groups (Hessels and van Lente 2011; Lepori et al. 2016). Our analysis has shown that more culture-specific models are required to do justice to the epistemic dimensions of the CC. This paper has made a start in exploring the variation in the structure of the CC across different epistemic cultures, and the various manifestations of different resources that are part of the CC. Further research will be needed to get a more complete picture of the variation of valuation practices across epistemic cultures. In particular, it would be interesting to explore analytical relationships between epistemic cultures and the CC. This would require a more systematic conceptualization of epistemic cultures than provided by Knorr-Cetina (1999), in terms of a coherent set of variables. This line of research could build on the work of Gläser et al. (2015), who distinguish between a number of dimensions of epistemic cultures, such as ontology, epistemology and work processes. Further research could investigate the relationships between these dimensions and valuation practices in the different steps in the CC.