Sotudeh et al. (2015) conduct a much needed study to determine if the author-pay model of publishing Open Access that is now used in some scholarly journals leads to a paper being cited more frequently. This is an important issue with implications for research assessment that could affect researchers’ careers. Consequently, Sotudeh et al.’s research deserves our attention and critical scrutiny.

In this short response piece, I argue that there appears to be no benefit for scholars publishing in Author-Pay Open Access journals in the social sciences and humanities. The apparent benefit Sotudeh et al. (2015) report is an artifact of their method of measuring impact. If the single most cited paper is removed from the sample, the reported effect disappears. Thus, social scientists and scholars in the humanities should be reticent to publish Open Access in the author-pay model. There is insufficient evidence that the Author-Pay Model of Open Access is a worthwhile investment for social scientists and scholars in the humanities.

Background

In the Author-Pay Model of Open Access, an author has the opportunity to pay a fee to have their paper published Open Access immediately, even before the paper appears in a print version of the journal. Author-Pay Open Access is often referred to as Gold Open Access. It is contrasted with Green Open Access Published. With Green Open Access Publishing, an author does not pay a fee, but they must wait some period of time stipulated in the Copyright Agreement with the publisher before they make their publication accessible in a personal website or institutional archive. Often the embargo on archiving is 18 months or 2 years. The Author-Pay Model of Open Access Publishing feeds off the hunger of researchers to get cited, both early and often.

As Sotudeh et al. (2015) note, Springer and Elsevier are leaders in implementing this publishing model. But there are now, in addition, many predatory publishers who promise Open Access to authors for a fee, but do not have the reputation that Springer and Elsevier have in academic publishing (see Sotudeh et al. 2015). Importantly, the fees associated with the Author-Pay Open Access model are often quite steep. The Springer journal Synthese, a leading journal in philosophy, for example charges US$3000 or €2200 to publish Open Access. And the Springer journal Social Indicators Research, a social science journal, has the same fees (see http://www.springer.com/gp/open-access/springer-open-choice). In the Humanities and the Social Sciences, researchers often have meager funding to support their research. So such a costly investment would need to yield results if it is to be worthwhile.

Sotudeh et al. (2015) seek to determine if Author-Pay Open Access Publishing is worth the money that authors must pay for it. More precisely, they aim to determine if papers published in this format receive more citations than papers that are not published in this format. Their study focuses on articles published in Springer and Elsevier journals as these publishers publish (1) papers that are Author-Pay Open Access and (2) papers that are not, in the same journal. This provides a control group of sorts for their study.

I do not want to recount the details of their study, but it is worth noting that there were greater benefits in the Health Sciences, the Life Sciences, and the Natural Sciences, than in the Social Sciences and Humanities. In fact, Sotudeh et al. (2015) report that there is only a 3.14 % increase in the citations to papers published Author-Pay Open Access in the social sciences and humanities. This finding is grim news for those authors who have bothered to pay the non-negligible fee in order to have their paper published Open Access. The social sciences and humanities do not compare well with other fields. In the Life Sciences, there is an 8.26 % increase in citations for Author-Pay Open Access Publications. In the Health Sciences the increase in citations is 33.29 % for Author-Pay Open Access. And in the Natural Sciences, the increase is 35.95 % for Author-Pay Open Access.

The problem

My concern is that the news is actually even worse than Sotudeh et al. (2015) imply. They report the mean number of citations to papers published Author-Pay Open Access, as well as the number of citations to those papers. They also report the maximum number of citations to a single paper published Author-Pay Open Access. The maximum number of citations to an Author-Pay Open Access Publication in the social sciences and humanities in the authors’ sample is 681 (see Sotudeh et al. 2015, Table 3). The mean number of citations to an Author-Pay Open Access Published paper in the social sciences and humanities in the authors’ sample is 5.92 (see Sotudeh et al. 2015, Table 3).

The authors also provide comparable data for the papers published in the same journals that were not published Author-Pay Open Access. The maximum number of citations to a paper that was not an Author-Pay Open Access Publication in the social sciences and humanities in the authors’ sample is 432 (see Sotudeh et al. 2015, Table 3). The mean number of citations to a paper that was not published Author-Pay Open Access in the social sciences and humanities in the authors’ sample is 5.74 (see Sotudeh et al. 2015, Table 3).

My concern is as followings. I was struck by the fact that the means were so similar, and yet the maximums were so far from the means for both populations. I suspected that the citation advantage that Sotudeh et al. (2015) report may be an artifact of the most cited paper, and not an indication of a citation bonus to either all or most papers published Author-Pay Open Access. That is, the maximums appear to be extreme outliers. They are data points that the typical researcher deciding whether to pay for Open Access should be reticent to consider relevant to their own situation.Footnote 1

To test this hypothesis, I subtracted the value of the single most cited articles in the two groups of publications, Author-Pay Open Access and Traditional, and then recalculated the mean. My calculations are below. The data are drawn from Table 3 in Sotudeh et al. (2015).

  • Author-Pay Open Access

  • 13069 citations − 681 citations = 12388 citations

  • 2207 papers − 1 paper = 2206 papers

  • Mean citations per paper = 12388 ÷ 2206 = 5.62 citations/paper

  • Traditional

  • 233040 citations − 432 citations = 232608 citations

  • 40617 papers − 1 paper = 40616 papers

  • Mean citations per paper = 232608 ÷ 40616 = 5.73 citations/paper

As the calculations show, once the most highly cited paper is eliminated from the data set, Author-Pay Open Access papers in the social sciences and humanities are cited on average less often than those that are not published Open Access. This suggests that the apparent citation advantage that Sotudeh et al. (2015) report was just an artifact of the methods they employed.

Incidentally, I made similar calculations for the other fields as well, the life sciences, the health sciences, and the natural sciences. In all those cases the result reported by Sotudeh et al. (2015) was robust. That is, removing the most highly cited paper did not reverse the citation advantage, as it did in the case of the social sciences and humanities.

Discussion

In an earlier study of Open Access Publication, Xia (2010) found that scholars had concerns about this model of publishing (2010, 622). Specifically, they were concerned that such papers were not properly or thoroughly refereed, and that their peers would perceive research published Open Access to be of lesser quality than research published in traditional journals (2010, 622). These attitudes do not in fact accurately reflect the reality of the culture of Open Access Publishing. In fact, as noted above, the notion of Open Access Publishing is applied to a variety of different practices, one of them being the Author-Pay model. And despite the negative attitudes reported by researchers in Xia’s study, more and more researchers are publishing Author-Pay Open Access (2010, 622).

As mentioned above, there are alternative models to the so-called Gold Open Access model, that is, the Author-Pay model. There is what is referred to as “Green Open Access.” In the Green Open Access model “articles that were published in top-tier journals … were converted into OA scholarship because they were self-archived on the authors’ personal websites and/or in institutional repositories” (see Atchison and Bull 2015, 129). In this model the author gets the benefits of having their papers widely accessible without the costs associated with the Author-Pay Open Access model. In fact Antelman (2004) found that in philosophy, political science, mathematics, and electrical and electronic engineering “freely available articles do have greater research impact” (2004, 372).

The Scholarly Publishing and Academic Research Coalition of Europe (SPARC Europe) posts a list of studies investigating “The Open Access Citation Advantage” (see http://sparceurope.org/oaca_table/). There are 15 studies that include data on the social sciences and humanities. Only seven studies report unequivocal citation advantages, whereas six studies suggest that there is no such advantage.Footnote 2

It may seem that Green Open Access is the obvious solution to the problem of ensuring that research is widely read and reaches its intended audience, at least from the researchers’ point of view. But, even the Green Open Access model of publishing is not without its problems. Atchison and Bull note “a large percentage (45 %) of the OA articles … have been posted in violation of the publisher’s copyright and self-archiving policies” (2015, 134). Rightly, journal publishers are far less enamored to this model. It is undermining their revenue, and it fails to recognize the value that academic publishers add to research articles.

We have little reason to believe that there is a citation advantage for publishing papers Author-Pay Open Access in the social sciences and humanities. Perhaps this should not surprise us. The research practices, publication practices, and citation practices in the social sciences and humanities are different from those in the natural sciences, life sciences, and health sciences (for the humanities, see, for example, Wray and Bornmann 2015; for the social sciences see Wray 2015 and Cole 1992, Chapter 5). Social scientists and scholars in the humanities should save their money and resist the temptation to buy into the Author-Pay Open Access Publishing model, a model that may serve other scientific fields well. The traditional publishing model seems to serve social scientists and scholars in the humanities well. Though there is some promise in Green Open Access, the legal issues will need to be resolved.