The propagation of error: retracted articles in marketing and their citations

Abstract

A retraction is the removal of an article from the scientific record at any time after its publication. This study investigates the characteristics of retracted articles in marketing. A total of 30 retracted articles published in 18 marketing journals were identified using Google Scholar and then analyzed. The analysis shows that the main reason for retracting marketing articles is duplication, followed by errors in data and data fabrication. On average, it took 2.371 years for each of these articles to be retracted. Using Clarivate Analytics’ Web of Science, it has been found that these retracted articles received 421 citations, 196 of which are post-retraction citations. More specifically, 22 of the 30 retracted marketing articles continue to be cited several years after their retraction. The most cited retracted marketing article gathered 67 citations of which 30 are post-retraction citations with all of them being positive citations referencing it as valid and legitimate work. The citation pollution caused by that retracted article transcends marketing to cover such disciplines as information science, psychology, and health nutrition.

1 Introduction

The retraction of a published article is an exceptional event. According to Brainard and You (2018), in about one in 2500 cases, a published article is retracted. Though retracted articles represent a minuscule party of the published literature, the topic of retractions has been lately receiving an upward interest across disciplines (e.g., Cox et al., 2018; Craig et al., 2020; Greitemeyer, 2014; Halevi, 2020; Hamilton, 2019; Serghiou et al., 2021; Teixeira da Silva & Bornemann-Cimenti, 2017; Walsh et al., 2019).

This increasing interest in retractions could be attributed to:

• The fact that the number of retracted articles is growing at a furious pace. As Serghiou et al., (2021, p. 2) report, the number of retracted articles per annum grew dramatically, in the last decade, from less than 100 before 2000 to about 1000 in 2014 and 1772 in 2019; and

• The important question of what damage is done by retracted articles. Though they are formally no longer part of the body of the scientific record, several studies concur that retracted articles continue to have an impact on scientific research (see e.g., Bar-Ilan & Halevi, 2017, 2018, 2021; Chen et al., 2013; Halevi, 2020; Hamilton, 2019).

While the extant literature on retracted articles remains mainly focused on STEM (i.e., sciences, technology, engineering, and medicine) (see e.g., Davis, 2012; Furman et al., 2012; Rapani et al., 2020; Rubbo et al., 2019), there have been some recent studies that cover arts and humanities (e.g., Halevi, 2020), psychology (e.g., Craig et al., 2020), and economics (e.g., Cox et al., 2018). However, to the best of the author’s knowledge, no published studies have investigated this topic in marketing.

The purpose of this study is twofold: (1) to find out the reasons behind the retractions of marketing articles and (2) to investigate the citation impact of these retracted articles. This study aims to answer the following three research questions (RQs):

• RQ1: Why do articles in marketing journals get retracted?

• RQ2: Do retracted marketing articles continue to be cited even after their retraction?

• RQ3: Does the damage caused by a retracted marketing article remain confined to the marketing discipline or largely transcend it?

2 Literature review

2.1 Retractions and retracted articles

Retracted articles are articles that are “pulled from the literature due to ethical issues and containing erroneous, or even fabricated data, analysis, and findings” (Bar-Ilan & Halevi, 2021, p. 48). Retraction “refers to the formal withdrawal of a publication, most often due to scientific misconduct or an error that invalidates the purported conclusions” (Serghiou et al., 2021). A more official definition of retraction is provided in the Retraction Guidelines by the Committee on Publication Ethics (COPE, 2019). In that document, the COPE defines a retraction as “a mechanism for correcting the literature and alerting readers to articles that contain such seriously flawed or erroneous content or data that their findings and conclusions cannot be relied upon” (COPE, 2019, p. 4).

Typically, a retraction implies issuing a freely available retraction notice (albeit not all journals issue a retraction notice upon retraction) that clearly mentions the reasons for retraction and identifies who is retracting the article (COPE, 2019). The retraction notice is commonly directly linked to the original article which is tagged as “Retracted”. The retracted article’s PDF is usually digitally watermarked as “Retracted” (Bar-Ilan & Halevi, 2017; Chen et al., 2013; Grieneisen & Zhang, 2012; Halevi, 2020). The retracted article is then left online to maintain the scholarly record. Such retractions are often publicized by the journal itself, the publisher of the journal, and/or by some initiatives that keep track of these retractions, such as Retraction Watch (https://retractionwatch.com/) of the Center for Scientific Integrity.

2.2 Reasons for retraction

Retractions occur for a multitude of reasons. According to Bar-Ilan and Halevi (2018), the reasons for retraction could be classified roughly into three main categories: (1) ethical misconduct (e.g. duplicate publication, plagiarism, missing credit, ownership issues, authorship issues, interference in the review process, citation manipulation); (2) scientific distortion (e.g. data manipulation, fraudulent data, unsupported conclusions, questionable data validity, non-replicability, data errors—even if unintended); (3) administrative error (e.g. article published in a wrong issue, not the final version published, publisher errors).

COPE’s (2019, p. 3) Retraction Guidelines list eight such reasons. For the COPE, an article should be considered for retraction if: (1) there is clear evidence that the findings are unreliable, either as a result of a major error (e.g., miscalculation or experimental error), or as a result of fabrication (e.g., of data) or falsification (e.g., image manipulation); (2) it constitutes plagiarism; (3) its findings have previously been published elsewhere without proper attribution to previous sources; (4) It contains material or data without authorization for use; (5) it infringes copyrights; (6) it reports unethical research; (7) It has been published solely on the basis of a compromised or manipulated peer review process; (8) its author(s) failed to disclose a major competing interest.

2.3 Citations of retracted articles

A citation, as Zinkhan (2004, p. 370) described it, “is an indicator that a published article is not being ‘lost’ or ‘overlooked’ and that it is having some impact on the field and the work of future authors”. A citation “represents the continued lifeline of a scientific paper, and in general shows that the work is being used, or appreciated, by those who cite it” (Teixeira da Silva & Bornemann-Cimenti, 2017, p. 366).

When it comes to retracted articles, there seems to be a consensus about the fact that they should not be cited. For instance, Atwater et al., (2014, p. 1179) state that “retracted articles should not be cited because they have been effectively removed from the scientific record”. Similarly, Teixeira da Silva and Bornemann-Cimenti (2017, p. 366) put that “a retracted scientific paper should not be used, or cited”. For Van der Vet and Nijveen (2016, p. 2), retracted articles “pollute their citation environments”.

There are two major types of citations of retracted papers: citations that an article received prior to its retraction and the citations that it received post retraction and despite the retraction notice (Bar-Ilan & Halevi, 2017).

Both types of citations undermine any scientific endeavor via the propagation of erroneous, fraudulent, unethical, or unreliable evidence that has the potential to mislead both current and future research and practice. Although the former type of citation is unavoidable (i.e., pre-retraction citations), the latter (i.e., post-retraction citations) is particularly concerning, especially when erroneous retracted articles are still cited as valid work and presented as central to the argument of the study citing it (Hamilton, 2019).

Prior studies investigating citations of retracted articles in disciplines other than marketing have shown that even after being flagged, retracted articles continue to be cited (see e.g., Hagberg, 2020; Hamilton, 2019; Rubbo et al., 2019; Van der Vet & Nijveen, 2016). The studies by Bar-Ilan and Halevi (2017) and Hamilton (2019) are of particular interest as they have found that the majority of the examined post-retraction citations referenced retracted articles as legitimate work. These are called positive post-retraction citations. Bar-Ilan and Halevi (2017, p. 550) define a positive post-retraction citation as a citation that “indicates that the retracted article was cited as legitimate prior work and its findings used to corroborate the author/s current study”. A negative post-retraction citation “indicates that the authors mentioned the retracted article as such [i.e., retracted article] and its findings inappropriate” (Bar-Ilan & Halevi, 2017, p. 550).

Do retracted articles published in marketing journals continue to be cited even after their retraction? How many of the post-retraction citations are positive citations? These are some of the questions that this study aims to respond to.

3 Methodology

3.1 Finding retracted marketing articles

Retracted marketing articles were searched (during the first half of May 2021) using Google Scholar, the free scholarly search engine by Google. Google Scholar was used as it has been found to outperform Elsevier’s Scopus and Clarivate Analytics’ Web of Science in finding retracted articles (see e.g., Hamilton, 2019, p. 1038).

Retracted articles are commonly identified with the prefixes “Retracted” or “Retracted article”. Retraction notices, and depending on the publisher, are called “Statement of retraction” (e.g., Taylor & Francis) or “Retraction notice” (e.g., Elsevier). Building on this, the search procedure was as follows: Using the advanced search features of Google Scholar, the author inserted “retraction” or “retracted” in the “find articles with the exact phrase” field and where these two words occur in the title of the article. The words “marketing”, “consumer”, “product”, “advertising”, “retail”, “service”, and “brand” were inserted one in a turn in the “Return articles published in” field (i.e., in the name of the journal). These seven keywords were selected as they are commonly used in the titles of marketing journals (see e.g., Moussa, 2019a, pp. 576–577). For instance, the Google Scholar query with “retracted” in the article’s title and “marketing” in the journal’s name returned retracted articles that were published not solely in such mainstream publication outlets as the Journal of the Academy of Marketing Science or Marketing Letters, but also retracted articles that appeared in specialized publication venues like the Journal of Interactive Marketing, Social Marketing Quarterly, and Journal of Strategic Marketing. Using this search procedure, the author identified 30 retracted articles that were published in 18 English-language marketing journals (see Appendix 1).

3.2 Collected data

The following bibliographic data were collected for each of the 30 retracted articles: the Digital Object Identifier (DOI) of the retracted article; journal name; date the article was first published online; DOI of the corresponding retraction notice; date the retraction notice was first issued; and the main reason for retraction as per the retraction notice.

Time to retraction for a retracted article is defined “as the time between its publication and the time of its retraction” (Chen et al., 2013, p. 242). For this study, time to retraction is counted in days. It could be straightforwardly calculated given the availability of the exact date of publication and the exact date of retraction from the journal’s online page.

The author used Clarivate Analytics Web of Science (accessed via an institutional subscription on 17 May 2021) to collect data about citations received by the 30 retracted articles (see also Rubbo et al., 2019). A “Basic Search” or a “Cited Reference Search” was performed conditional on the inclusion of the publishing journal in Clarivate Analytics databases. For instance, the Journal of the Association for Consumer Research (JACR) is not indexed in the Social Sciences Citation Index. Citations to the retracted JACR article were collected using the “Cited Reference Search” option rather than the “Basic Search” option.

Web of Science was retained as the citation source instead of Google Scholar as the latter “indexes all output regardless of whether or not it is peer-reviewed” (Halevi et al., 2017, p. 825). Prior studies indicate that Google Scholar covers citations in master thesis, working papers, preprints, and any other document types visible to Google Scholar, articles in predatory journals included (Moussa, 2019a, 2021a).

Consistent with prior studies, a post-retraction citation was defined as any publication that referenced a retracted article for which the date of publication was after the date the retraction notice was issued (Bar-Ilan & Halevi, 2017).

3.3 Statistical analyses

In addition to descriptive statistics, it is perhaps interesting to investigate the relationship between pre-retraction citations and post-retraction citations. Prior research indicates that articles that are cited strongly before retraction continue to be cited strongly afterward (Hagberg, 2020; Hamilton, 2019).

Another relationship worth studying is the one between the length of the time to retraction and the number of pre-retraction citations. It is expected that the lengthier is the time to retraction, the larger is the number of pre-retraction citations.

Because of the limited sample size (i.e., N = 30) and the non-normal distribution of most bibliometric and informetric variables (Ajiferuke & Famoye, 2015), the author used Kendall’s tau-b (the non-parametric correlation coefficient) to investigate the relationships between pre and post-retraction citations. The same coefficient was also used to estimate the relationship between pre-retraction citation and time to retraction. Kendall’s tau-b has a range between − 1 and 1. A τ_b “of at least 0.7 represents a very strong relationship; 0.4–0.699, a strong relationship; 0.3–0.399, a moderate relationship, 0.2–0.299, a weak relationship; and 0–0.199 implies that the variables are likely to be unrelated” (Ferrucci et al., 2020, p. 3).

It is also expected that the number of citations that an article receives will drop after the publication of the retraction notice (Hamilton, 2019). To statistically test for the difference between the number of pre-retraction citations and the number of post-retraction citations, a paired t-test was performed. All statistical analyzes were run under IBM’s SPSS (version 18).

3.4 Post-retraction citation visualization

Once the most cited retracted marketing article is identified and its number of positive post-retraction citations known, VOSviewer (Van Eck & Waltman, 2010) will be used to visualize the indirect citation impact that retracted article has had. VOSviewer is a freely available software used for the construction and visualization of bibliometric maps. The bibliographic coupling option in VOSviewer will be used. Bibliographic coupling happens when two works reference a common third work in their bibliographies. Two documents are said to be bibliographically coupled if they both cite one or more documents in common (see Glänzel & Czerwon, 1996).

4 Results and discussion

4.1 Retraction notices

The oldest retraction notice dates back to 23 February 2010. The most recent one is dated 14 October 2020. Eight of the collected retraction notices were issued during 2020. Two and four retraction notices were publicized in 2018 and 2019, respectively. As such, nearly half of the retraction notices appeared in the last three years (2018–2020) (see Fig. 1).

Fig. 1

Distribution of the retraction notices by year of issuing

An explicit statement of the reason for retraction was found for 29 of the 30 retracted articles. In one case, the reason for the retraction was not clearly mentioned. In another case, a single retraction notice provided the reason for retraction for two different articles (co)authored by the same researcher and published in the same journal.

4.2 Time to retraction

When analyzed, the collected data shows that it took on average about 866 days (M = 865.833 days or 2 years and 4 months and 14 days) for an article to be retracted. The median time to retraction is Mdn = 626.5 days (Min–Max: 0–2541 days). Chen et al., and and’s (2013, p. 242) large-scale study of retracted articles has shown that the mean and the median time to retraction were M = 2.57 and Mdn = 2 years, respectively.

4.3 Journals

The 30 retracted articles were found in 18 marketing journals (see Fig. 2). As Fig. 2 indicates, 24 of the 30 retracted articles were published in Impact Factor journals. Of interest is the fact that 10 of the 30 retracted articles are Journal of Consumer Research (JCR) articles. This finding is quite surprising since the JCR (an Oxford University Press journal) is commonly known for being one of the three most renowned marketing journals and “the” leading journal in consumer behavior (Moussa, 2019a). The JCR is also a publication venue included in the prestigious Financial Times’ list of Top 50 Business and Management journals (or FT50 list for short) (Moussa, 2021b).

Fig.2

Distribution of the 30 retracted articles by journals (Note: * denotes FT50 journals and ‡denotes Impact Factor journals)

Two other retracted articles appeared in two different FT50 journals, namely the Journal of the Academy of Marketing Science and the Journal of Consumer Psychology.

Four of the remaining 18 retracted articles appeared in such publication outlets as Psychology & Marketing, International Journal of Consumer Studies, Journal of Consumer Protection and Food Safety, and the JACR (with one article each). To put it more plainly, half (i.e. 15/30) of the retracted articles were published in consumer behavior journals.

4.4 Why articles published in marketing journals get retracted?

A response to the question of what marketing researchers are doing wrong to get their articles retracted (i.e., RQ1) could be found in Table 1. Of the 30 analyzed articles, 11 were retracted because they are duplicate publications. Covering arts and humanities journals, the study by Halevi (2020) has shown that the most prevalent reason for retraction was “significant overlap with previously published research” (or duplication for short). Whether it is due to the “Publish or Perish” precept or not, some marketing researchers are submitting their previously published research to new journals after slightly altering their titles and some of their contents.

Table 1 Main reasons for retraction

The second and the third most frequent reasons for retraction are, in order of frequency, “Error in data” and “Fabricated data” with seven and five occurrences, respectively. Data play a significant role in conducting contemporary marketing research (Wedel & Kannan, 2016). Most marketing journals favor, accept and publish research articles which findings are based on empirical data-driven analyzes (Hubbard & Lindsay, 2002). As such, it is possible that some marketing researchers are inadvertently making errors when processing their collected data. Though they represent a minority, there are also those marketing researchers that have no ethical qualms in fabricating the data they analyze.

4.5 Citations of retracted articles: pre and post-retraction citations

Taken together, 26 (i.e., 86.67%) of the 30 retracted articles received a total of 421 citations from publication venues indexed in the Web of Science. Four retracted articles received no citations up to 17 May 2021.

Twenty-six articles received 225 citations before retraction, and 22 articles (73.33%) received 196 afterward. The average number of citations slightly dropped from 7.50 to 6.53 after the publication of the retraction notice. The t-test for mean difference yielded a non-statistically significant value of t = 0.578 (p > 0.05). Stated differently, these articles continued to collect citations at almost the same pace, irrespective of the issuing of the retracted notices. Kendall’s tau-b for the relationship between pre and post-retraction citations got a statistically significant positive value of τ_b = 0.355 (with p < 0.05). This value indicates a positive and moderate relationship between pre and post-retraction citations. So, the answer to RQ2 of “Do retracted marketing articles continue to be cited even after their retraction?” is a yes.

Kendall’s tau-b correlation coefficient for the relationship between time to retraction and pre-retraction citations obtained a statistically significant positive value of τ_b = 0.700 (with p < 0.001). Such a value is indicative of a very strong relationship. It suggests that the less prompt the retraction notice is, the larger the to-be retracted article gathers citations.

4.6 The most cited retracted article published in a marketing journal

The most cited retracted article is a JCR paper that was first published online on 20 October 2009. It is an article that appeared in 2010. To be more precise, it appeared in JCR’s Volume 36 in the April issue. The retraction notice of that article is dated 10 April 2014. The time to retraction for that highly cited retracted article is 1633 days. In the retraction notice, the following reason is forwarded: this article “was found to involve blameworthy inaccuracies in the way the research was carried out”. The data collected and analyzed by the first of the three co-authors of that paper, a Science Magazine article states, “were ‘too good to be true’” (Enserink, 2012).

According to the Web of Science, that retracted article received, as of 17 May 2021, 67 citations. Of these, 30 are post-retraction citations. Three of these citations were made in articles published in 2021. As can be seen from this example, a retracted article continues to be cited years after retraction and despite the retraction notice being posted on the journal’s/publisher’s platform (see Fig. 3). One possible explanation for the long life of that retracted article resides in the fact that its non-watermarked PDF is freely available via the following link: https://repub.eur.nl/pub/18666/9DA0599Ed01.pdf (last accessed 12 December 2021). RePub is Erasmus University Rotterdam’s institutional repository which provides access to the academic output of the university and makes it available to everyone, free of charge.

Fig. 3

Number of post-retraction citations per year for the most cited retracted marketing article

Are all of these 30 post-retraction citations positive citations? Judging only by the titles of the citing 30 papers, nothing seemed to suggest that these 30 citations are negative post-retraction citations. None of the citing 30 papers deals with retracted articles, research misconduct, or scientific integrity. To confirm the positive nature of these citations, the author downloaded each of the 30 citing articles and examined whether the in-text reference cited the retracted paper positively or negatively. The author was able to get access to all the 30 citing papers. After downloading each of these 30 citing papers, the author counted how many times the retracted article was referenced in the text and examined the positivity or negativity of the in-text citation.

The citing papers comprise 27 journal articles and three conference papers. Twenty-eight of the citing papers are in English. Of the two remaining citing papers, one was published in a Chinese-language journal and the other appeared in a Spanish-language journal. The author used Google Translation to figure out whether the citations in these two particular papers were positive or negative. Sentences that explicitly cite the retracted article were extracted from the full-text of the 30 citing papers. They are made available in Appendix 2. Upon examination, all the 30 citations turned out to be positive post-retraction citations.

Some of the citing papers appeared in marketing journals (e.g., European Journal of Marketing; Psychology & Marketing; Marketing Letters; Asia Pacific Journal of Marketing and Logistics), business journals (e.g., Journal of Business Research), fashion and textile journals (e.g., Fashion Theory; Fashion & Textiles; Fashion, Style & Popular Culture), and information systems journals (e.g., Journal of Management Information Systems; Internet Research). Worse, some of these citing articles were published in psychology (e.g., Health Psychology Open) and health nutrition journals (e.g., Journal of Obesity; Eating Behaviors).

The response to RQ3 of “Does the damage caused by a retracted marketing article remain confined to the marketing discipline or largely transcend it?” is evident in Fig. 4. It presents the overlay visualization of bibliographic coupling of the citing 30 papers. As Fig. 4 indicates, the citation pollution this retracted article continues to create after its retraction transcends marketing to cover several other disciplines.

Fig. 4

The overlay visualization of bibliographic coupling of the 30 papers that positively cited the retracted article after its retraction

5 Implications

Retracted articles pose serious threats to any scientific endeavor in any discipline. This study has focused exclusively on marketing, in an attempt to draw attention to retracted articles, the reasons why marketing articles are retracted as well as to explore the extent to which they continue to have an academic impact in the form of citations.

This study has several implications for a variety of parties, including researchers, business schools, academic associations, journal editors, peer reviewers, publishers, and university repositories curators. This study also involves implications for marketing theory and practice.

This study has found that 28 of the 30 identified retracted articles were withdrawn because they involve ethical misconduct (in 13 cases) or scientific distortion (in 15 occurrences). It also shows that 12 of the 30 retracted articles were published in three FT50 journals and that 24 of the 30 retracted articles appeared in Impact Factor-carrying marketing journals. Given these findings, it is perhaps safe to speculate that some marketing researchers are doing whatever it takes to get their papers published in highly ranked, Impact Factor journals. Related to this, the author would like to put forth the following question: are these findings the upshot of the “publish in top-ranked/Impact Factor journals or perish” pressure that pervades some business schools? Business school deans and managers should not prioritize Impact Factors and journal rankings over responsible authorship and ethical publishing. Business schools have to instruct marketing researchers to not engage themselves in suspect, unethical activities.

For researchers, this study indicates that retracted articles occur in prestigious marketing journals. All marketing researchers must be aware of these retracted articles, particularly those with erroneous or fabricated data, as a means of preventing their propagation like it could be seen in Fig. 4. Though it is hard to know precisely why these retracted articles continue to be cited, the researchers that are citing them are likely unaware that they were retracted.

Awareness campaigns are needed, urgently. Academic associations like the Association for Consumer Research (ACR) have played a key role in shaping the marketing discipline as it is known today (Wilkie & Moore, 2003). Given that half of the 30 retracted articles were published in consumer behavior journals, the ACR has to lead such awareness campaigns. For instance, the Center for Global R&D and Innovation provides a list of retracted and withdrawn articles in the field of management, and especially in innovation, technology management, R&D management, incubation, and international business (see http://www.glorad.org/retracted-papers.html). The ACR has to instigate an analogous initiative.

For journal editors, this study has shown that 11 of the 30 retracted articles are duplicate publications. These retracted articles should have been identified very early and before undergoing any peer-reviewing process if they were checked for plagiarism. A journal editor has to perform a plagiarism check for each submitted manuscript before sending it to the peer reviewers.

Discussing the importance of the review process in marketing, Lehmann and Winer (2017, p. 589) stated that a “strong, effective review process helps ensure that papers that are accepted do not have fatal flaws”. Peer reviewers are the gatekeepers of academic journals. Peer-reviewing is a hard yet voluntary duty. As this study seems to indicate, the peer-review process has failed to detect, in the case of 12 of the 30 retracted articles, that the data were erroneous or fabricated. Peer reviewers should remain diligent in ensuring that the manuscripts they accept are not scientifically distorted.

Publishers have to avoid administrative errors that might lead to retractions. As this study has sought to demonstrate, one of the 30 retracted articles was retracted, as its retraction notice points out, because “it was published in error by the publisher whilst still in the peer review process”. Publishers have also to make every effort to issue a prompt, free access, and clear retraction notice. As the COPE advises, a prompt retraction can “minimize the number of researchers who cite the erroneous work, act on its findings, or draw incorrect conclusions” (COPE, 2019, p. 6). This study has shown that the average time to retraction was 2.371 years. This study has also revealed that there is a strong positive relationship between pre-retraction citations and time to retraction. This study has additionally demonstrated that there is a moderately positive relationship between pre-retraction citations and post-retraction citations. To put it differently, the less timely the retraction is, the more the article is cited pre retraction, and the more the article is cited pre retraction the more it is cited post retraction. This chain of effects has to be broken as quickly as possible so to prevent the retracted article from further polluting its citation environment.

Curators of university repositories have to carefully curate their repositories. Retracted articles should be tagged as such. Original non-watermarked PDF versions of retracted articles should be replaced by watermarked ones.

Authors, journal editors, peer reviewers, and the broader marketing research community should remain meticulous in ensuring that citations of retracted articles are identified and removed before, during, and possibly even after publication. Articles that heavily cite retracted articles could cause in their turn indirect citation pollution.

Retracted articles, especially those retracted for reasons of erroneous and falsified data (i.e., 12 of the 30 retracted articles), impose a significant threat to marketing theory. They are sources of unreliable findings that could be used for conducting new studies, formulating new hypotheses and frameworks, and conducting meta-analyzes. Findings in such a kind of retracted articles can damage the validity of both previous and future marketing theories.

Articles published in marketing journals commonly contain managerial and practical implications (e.g., Belvedere et al., 2021; Sardanelli et al., 2021). Marketers, managers, and organizational policymakers make up a (more or less significant) fraction of the readership of marketing journals. Retracted articles and particularly those with findings based on incorrect and fabricated data provide misleading recommendations for managers and practitioners. Marketers and organizational policymakers should not base their decisions on the implications enumerated in such unreliable or fraudulent articles. Such a type of retracted articles can harmfully influence marketing policies and practices.

6 Conclusion, limitations, and further research directions

Several are those studies that have investigated the topic of retracted articles. This study is however the first to focus on that topic within the marketing discipline. It shows that retraction is something new to marketing as the oldest retraction notice dates back to 23 February 2010. In a decade, only 30 marketing articles were retracted. Eight of the articles were retracted in 2020. Is marketing starting to self-correct its record? How many marketing articles will be retracted in the upcoming years? How many highly-cited marketing articles involving fabricated and erroneous data will be discovered? These and other questions should be revisited a few years ahead.

Though this study provides several implications, it has limitations too. However, some of these limitations point to questions that could be the subject of future work. First, this study covers only marketing-focused journals. Retracted articles appearing in marketing-related journals, like the Journal of Business Ethics (see e.g., https://doi.org/10.1007/s10551-008-9733-0) or Journal of Business Venturing (see e.g., https://doi.org/10.1016/j.jbusvent.2007.06.002) were not included in the analysis. Future studies may cover retracted articles appearing in these and other business and management journals. Second, this study is limited to articles that were published and retracted prior to 2021. Future studies should include articles for which the retraction notices were issued as of 2021. Third, this study focused on retracted articles, not on articles for which expressions of concern have been issued. An expression of concern is used to draw attention to a possible problem in a published article. For instance, the Journal of Marketing—or marketing’s foremost journal—has recently (on 20 August 2021) issued an expression of concern about one of its 2019 articles (see https://doi.org/10.1177/00222429211026311). This and other expressions of concern may ultimately result in retractions and hence warrant both inspection and caution. Fourth, this study measured the scientific impact of retracted articles using citations. The scientific impact of retracted articles could also be gauged using the number of downloads, reads, as well as social media mentions (Halevi, 2020; Serghiou et al., 2021). For instance, the recent study by Halevi (2020) uses PlumX to show that retracted articles continue to be read, downloaded, and mentioned on social media channels. A more recent study by Serghiou et al. (2021) uses Altmetric to measure media and social media attention to retracted articles. Future researchers may collaborate with PlumX or Altmetric to measure the impact of retracted marketing articles in terms of downloads, reads, as well as social media mentions. Another promising research avenue is to use social media listening and sentiment analysis to detect very early questionable, unreliable, or unethical publications. Sentiment analysis could be broadly defined as “the computational treatment of opinion, sentiment, and subjectivity in text” (Pang & Lee, 2008, p. 10). In sentiment analysis, social media posts (e.g., Facebook posts or tweets on Twitter) are commonly classified as positive, neutral, or negative (Moussa, 2019b). Articles with a high number of negative sentiments should be further examined to detect whether they involve ethical misconduct (e.g., plagiarism) or scientific distortion (e.g., fabricated data) (see e.g., Haunschild & Bornmann, 2021).

Appendices

Appendix 1: The 30 retracted marketing articles

Article #	DOI of the retracted article	Journal acronym	Article publication date	DOI of the retraction notice	Retraction notice publication date	Total number of citations (WoS)	Pre-retraction citations (WoS)	Post-retraction citations (WoS)	Time to retraction (in days)	Main reason for retraction as per retraction notice
1	https://doi.org/10.1086/648688	JCR	20/10/2009	https://doi.org/10.1086/676823	10/04/2014	67	37	30	1633	Fabricated data
2	https://doi.org/10.1080/02642060802188023	SIJ	27/07/2010	https://doi.org/10.1080/02642069.2016.1258819	09/01/2017	66	31	35	2358	Duplicate publication
3	https://doi.org/10.1002/mar.20375	P&M	04/11/2010	https://doi.org/10.1002/mar.20565	30/07/2012	44	5	39	634	Fabricated data
4	https://doi.org/10.1086/677225	JCR	19/06/2014	https://doi.org/10.1093/jcr/ucz057	22/11/2019	36	28	8	1982	Error in data
5	https://doi.org/10.1080/10548408.2014.884964	JT&TM	15/07/2014	https://doi.org/10.1080/10548408.2018.1429558	26/01/2018	30	16	14	1291	Duplicate publication
6	https://doi.org/10.1016/j.jcps.2015.01.001	JCP	13/01/2015	https://doi.org/10.1002/jcpy.1196	14/10/2020	21	21	0	2101	Unavailable data
7	https://doi.org/10.1016/j.intmar.2017.07.001	JIM	23/09/2017	https://doi.org/10.1016/j.intmar.2018.10.001	01/02/2019	20	8	12	496	Duplicate publication
8	https://doi.org/10.1086/661553	JCR	14/07/2011	https://doi.org/10.1086/676822	01/06/2014	18	10	8	1053	Fabricated data
9	https://doi.org/10.1007/s11002-012-9215-0	ML	11/11/2012	https://doi.org/10.1007/s11002-016-9401-6	05/03/2016	14	8	6	1210	Error in data
10	https://doi.org/10.1093/jcr/ucx036	JCR	23/01/2017	https://doi.org/10.1093/jcr/ucaa039	30/07/2020	12	9	3	1284	Error in data
11	https://doi.org/10.1086/697083	JACR	12/03/2018	https://doi.org/10.1086/710128	09/06/2020	12	10	2	820	Error in data
12	https://doi.org/10.1086/657430	JCR	14/10/2010	https://doi.org/10.1086/667237	12/06/2012	11	3	8	607	Fabricated data
13	https://doi.org/10.1093/jcr/ucx061	JCR	18/04/2017	https://doi.org/10.1093/jcr/ucaa048	08/10/2020	9	9	0	1269	Error in data
14	https://doi.org/10.1111/j.1540-5885.2011.00863.x	JPIM	13/10/2011	https://doi.org/10.1111/jpim.12058	24/05/2013	9	7	2	589	Inconsistencies in results
15	https://doi.org/10.2753/MER1052-8008200202	MER	08/12/2014	https://doi.org/10.1080/10528008.2017.1347451	06/07/2017	8	7	1	941	Duplicate publication
16	https://doi.org/10.1080/0965254X.2018.1430056	JStratM	25/01/2018	https://doi.org/10.1080/0965254X.2018.1430056	25/01/2018	7	0	7	0	Duplicate publication
17	https://doi.org/10.1111/jpim.12409	JPIM	31/07/2017	https://doi.org/10.1111/jpim.12490	09/04/2019	6	2	4	617	Duplicate publication
18	https://doi.org/10.1007/s11747-015-0430-0	JAMS	07/03/2015	https://doi.org/10.1007/s11747-015-0430-0	01/07/2016	5	1	4	482	Error in data
19	https://doi.org/10.1111/j.1470-6431.2011.01063.x	IJCS	27/06/2012	https://doi.org/10.1111/ijcs.12040	05/08/2013	5	2	3	404	Duplicate publication
20	https://doi.org/10.1086/662139	JCR	05/10/2011	https://doi.org/10.1086/662139	11/07/2012	4	1	3	280	Not mentioned
21	https://doi.org/10.1093/jcr/ucy060	JCR	06/07/2018	https://doi.org/10.1093/jcr/ucaa033	03/07/2020	4	3	1	728	Fabricated data
22	https://doi.org/10.1108/08876040810862840	JSM	25/04/2008	https://doi.org/10.1108/jsm.2010.07524aaa.002	23/02/2010	4	0	4	669	Plagiarism
23	https://doi.org/10.1007/s00003-010-0579-x	JCP&FS	10/03/2010	https://doi.org/10.1007/s00003-017-1099-8	22/02/2017	4	4	0	2541	Duplicate publication
24	https://doi.org/10.1093/jcr/ucy053	JCR	09/06/2018	https://doi.org/10.1093/jcr/ucaa009	18/02/2020	3	2	1	619	Error in data
25	https://doi.org/10.1080/10454446.2012.653779	JFPM	24/02/2012	https://doi.org/10.1080/10454446.2013.748343	18/12/2012	1	0	1	298	Duplicate publication
26	https://doi.org/10.1177/1524500419831084	SMQ	17/02/2019	https://doi.org/10.1177/1524500420945546	03/08/2020	1	1	0	533	Plagiarism
27	https://doi.org/10.1093/jcr/ucz056	JCR	08/11/2019	https://doi.org/10.1093/jcr/ucaa002	20/01/2020	0	0	0	73	Duplicate publication
28	https://doi.org/10.1080/10548408.2017.1341361	JT&TM	30/06/2017	https://doi.org/10.1080/10548408.2017.1383066	27/09/2017	0	0	0	89	Duplicate publication
29	https://doi.org/10.1080/1046669X.2019.1647914	JMC	19/08/2019	https://doi.org/10.1080/1046669X.2019.1666639	16/09/2019	0	0	0	28	Publisher error
30	https://doi.org/10.1111/j.1540-5885.2012.00957.x	JPIM	12/06/2012	https://doi.org/10.1111/jpim.12058	24/05/2013	0	0	0	346	Inconsistencies in results

IJCS = International Journal of Consumer Studies; JCP&FS = Journal of Consumer Protection & Food Safety; JCP = Journal of Consumer Psychology; JCR = Journal of Consumer Research; JFPM = Journal of Food Products Marketing; JIM = Journal of Interactive Marketing; JMC = Journal of Marketing Channels; JPIM = Journal of Product Innovation Management; JSM = Journal of Services Marketing; JStratM = Journal of Strategic Marketing; JAMS = Journal of the Academy of Marketing Science; JACR = Journal of the Association for Consumer Research; JT&TM = Journal of Travel & Tourism Marketing; MER = Marketing Education Review; ML = Marketing Letters; P&M = Psychology & Marketing; SIJ = Service Industries Journal; SMQ = Social Marketing Quarterly.

Appendix 2: The 30 papers referencing the most cited retracted marketing article

DOI of the citing Paper	Publication year	Journal/Conference	Times cited within the text	Citation in-context	Positive/negative post-retraction citation
https://doi.org/10.1111/ijcs.12679	2021	International Journal of Consumer Studies	1	“Prior research repeatedly found that exposure to idealized advertising images influences the way consumers feel about themselves” (p. 11)	Positive
https://doi.org/10.1108/JSOCM-09-2019-0137	2021	Journal of Social Marketing	1	“This study adapted and used the Nutritional Consciousness Scale” (p. 101)	Positive
https://doi.org/10.1108/APJML-05-2020-0343	2021	Asia Pacific Journal of Marketing and Logistics	1	“Regarding human models, previous research mostly focused on the aspects of advertising, such as the model’s size” (p. 12)	Positive
https://doi.org/10.1108/EJM-09-2019-0731	2020	European Journal of Marketing	2	“as exposure to both thin and heavy models can lead to lower self-esteem for high-BMI individuals” (p. 958)	Positive
https://doi.org/10.1155/2020/7396948	2020	Journal of Obesity	1	“Female body image has generally been considered more responsive to social contexts and mass media” (p. 2)	Positive
https://doi.org/10.3724/SP.J.1041.2020.00645	2020	Acta Psychologica Sinica	1	“in the past, most scholars only focused on a single social comparison…” (p. 646)	Positive
https://doi.org/10.1108/JFMM-07-2019-0140	2020	Journal of Fashion Marketing & Management	1	“has long been a popular research topic in investigating the impact of ideal beauty in fashion photography on fashion readers” (p. 652)	Positive
https://doi.org/10.1386/fspc_00022_1	2020	Fashion, Style & Popular Culture	1	“normal-weight women reported lower self-esteem after exposure to extremely thin or moderately heavy models” (p. 335)	Positive
https://doi.org/10.1108/INTR-07-2018-0331	2020	Internet Research	2	“It is argued that consumers utilise information from different media” (p. 114)	Positive
https://doi.org/10.1016/j.jretconser.2019.07.002	2019	Journal of Retailing & Consumer Services	2	“Women with low BMIs react positively to thin models” (p. 364)	Positive
https://doi.org/10.1080/07421222.2019.1628894	2019	Journal of Management Information Systems	1	“to rate their appearance self-esteem which captures the self-worth derived from their appearance and weight” (p. 812)	positive
https://doi.org/10.1561/1700000060	2019	Foundations & Trends in Marketing	1	“…showed ads featuring moderately/extremely thin/heavy models to mostly female undergrads…” (p. 275)	Positive
https://doi.org/10.5020/2318-0722.2019.8116	2019	Revista Ciencias Administrativas	1	“also analyze how advertisement containing lean or heavy models…” (p. 6)	Positive
https://doi.org/10.1177/2055102918819	2019	Health Psychology Open	1	“are more responsive to body images generated by mass media” (p. 2)	Positive
https://doi.org/10.1186/s40691-018-0130-8	2018	Fashion & Textiles	1	“including self-image and self-esteem, especially among females” (p. 3)	Positive
https://doi.org/10.1186/s40691-017-0116-y	2018	Fashion & Textiles	1	“women with lower BMIs were more likely to aspire to the thin ideal as compared to their heavier counterparts within the sample because they saw this ideal as within more “ready reach” (i.e., they identifed within this ideal)” (p. 13)	Positive
https://doi.org/10.1109/ACSSC.2018.8645090	2018	52nd Asilomar Conference on Signals, Systems, and Computers	1	“…provide labels for altered images of this kind” (p. 1532)	Positive
https://doi.org/10.1007/978-3-030-02131-3_49	2018	17th IFIP conference on e-Business, e-services, and e-Society	1	“A study on the effect of thin and heavy weight images in Social Media Users” (p. 568)	Positive
https://doi.org/10.1002/mar.21004	2018	Psychology & Marketing	2	“place increased emphasis on the perceived discrepancy between the consumer’s body and that of the model” (p. 539)	Positive
https://doi.org/10.1108/JFMM-08-2016-0076	2017	Journal of Fashion Marketing & Management	1	“the goal of such labeling is to induce greater self-confidence in one’s physical appearance” (p. 441)	Positive
https://doi.org/10.1108/IntR-11-2015-0321	2017	Internet Research	1	“assessing the appearance of their own body relative to that of others, and this assessment affects their responses and subsequent behaviors” (p. 455)	Positive
https://doi.org/10.1080/1362704X.2016.1138658	2017	Fashion Theory	1	“in that some parts of the body always have a potential for looking better” (p. 88)	Positive
https://doi.org/10.1016/j.chb.2016.02.067	2016	Computers in Human Behavior	1	“yet effects of anti-obesity campaigns on underweight or normal-weight recipients are hardly researched” (p. 566)	Positive
https://doi.org/10.1177/1090198116630527	2016	Health Education & Behavior	1	“hence converging their beliefs toward current gender and SES-specific social norms” (p. 60S)	Positive
https://doi.org/10.1016/j.jbusres.2015.07.042	2016	Journal of Business Research	3	“The three components of self-compassion played an important role within social comparisons” (p. 768)	Positive
https://doi.org/10.1108/IJPHM-09-2014-0052	2016	International Journal of Pharmaceutical and Healthcare Marketing	1	“intense need to be thin among young women can be traced largely to the media and marketing communications” (p. 325)	Positive
https://doi.org/10.1016/j.eatbeh.2014.12.006	2015	Eating Behaviors	1	“Two recent studies found evidence that when women view themselves as similar to plus-sized models they are negatively impacted” (p. 35)	Positive
https://doi.org/10.1109/HICSS.2015.387	2015	48th Annual Hawaii International Conference on System Sciences (HICSS)	1	“found out that the social comparison processes as well as the self-evaluative and behavioral results differentiate between people with different BMIs” (p. 3209)	Positive
https://doi.org/10.1002/mar.20765	2015	Psychology & Marketing	2	“such that being thin seems normative for citizens and employees” (p. 95)	Positive
https://doi.org/10.1007/s11002-013-9249-y	2014	Marketing Letters	7	“Body size of advertising models has received notable attention” (p. 167)	Positive