Xu, Yalcinkaya, and Seggie’s (2008) article, “Prolific authors and institutions in leading international business journals” published in the Asia Pacific Journal of Management, contributes to the important debate of how to fairly assess individual and institutional productivity in international business (IB). Rankings of authors and institutions, however, are subject to a range of potentially arbitrary decisions: choice of journals, weighting of data, and aggregation of individuals to an institutional level. This commentary briefly critiques the article of Xu et al. to exemplify some of the problems involved.

Choice of journals

The first choice in most rankings is to select the journals to be used as the basis for the assessment. Most rankings of the IB discipline evaluate individuals and institutions based on publication in a specified set of journals. There seems to be a consensus that Journal of International Business Studies (JIBS), Journal of World Business (JWB) and, more recently, Management International Review (MIR) and International Business Review (IBR) are considered as the core IB journals (see Morisson & Inkpen 1991; Inkpen & Beamish, 1994; Kumar & Kundu, 2004; Chan, Fung & Lai, 2006). Xu et al. deviate from this consensus by including International Marketing Review (IMR) and Journal of International Marketing (JIMar). They substantiate their addition of these two journals by referencing Dubois and Reeb (2000) who listed IMR and JIMar among the top six IB journals. Dubois and Reeb, however, also define what they consider to be “core IB” journals on the basis of editorial policies, indicating that their journal content is not exclusive to any single IB foundation discipline (p. 692). IMR and JIMar are not amongst this core. Therefore, I would argue that the results of Xu et al. are biased towards academics and institutions that work in the area of international marketing, rather than those working in IB more broadly defined. If the field includes International Marketing journals in a ranking of IB scholars, then there would also be an argument for including other single-disciplinary journals, for example, those with a specific focus on international management, such as Journal of International Management (JIMan). Although JIMan is not ISI listed, an alternative citation assessment, based on the analysis provided in Harzing and van der Wal (2008) shows that JIMan closely matches IBR and MIR in terms of citation impact.Footnote 1

As I do not have access to the full data on which Xu et al. based their analyses, I do not know what the rankings would look like if they focused only on Dubois and Reeb’s “core IB journals”. An analysis of publications in IMR and JIMar for the 1996–2006 period, however, shows that the 1st ranked institution in Xu et al., Michigan State University (MSU), takes a clear lead for publication in these two journals with no less than 57 articlesFootnote 2, four times as many as the second (University of TexasFootnote 3) and third (Old Dominion University) ranked universities. MSU is clearly the top university in the world in the discipline of international marketing (at least as measured by publication in these two journals). When JIBS, JWB and two years of IBR Footnote 4 are analyzed, the highest ranked institutions are the Chinese University of Hong Kong (sixth in Xu et al.), the University of TexasFootnote 5 (12th [Dallas] and 16th [El Paso] in Xu et al.) and the University of South Carolina (tied for 17th in Xu et al.). MSU drops from being first ranked to sixth ranked with two other institutions. Although given their position in this and other ranking studies, all four institutions should be considered to have made a significant contribution to the IB discipline, it is clear that their specific position in rankings is very dependent on the choice of journals.

Weighting of the data

The second factor affecting ranking systems is the set of choices concerning how to weight the data. One of the most important decisions is how the assessment system treats the contribution of multi-authored articles. Some evaluation systems focus on the total number of articles an individual (or an institution) has produced, whereas other systems calculate fractional weights for multi-authored articles. For example, an evaluation system might separately give 0.5 points to each author for an article with two authors. Most previous ranking studies have favoured this fractional method (see Morisson & Inkpen, 1991; Inkpen & Beamish, 1994; Kumar & Kundu, 2004; Chan, Fung & Lai, 2006) for ranking both individuals and institutions. Using a seemingly inconsistent approach, Xu et al. have used the fractional method for individual authors, but a non-fractional, total appearance methodology for institutions. In their article, they fail to provide the reason for such an inconsistent choice.

As shown in Figure 1, ranking institutions according to the generally preferred fractional method would not dramatically change the ranking of Xu et al. of the top-ten universities; MSU would still lead the pack. However, using the more widely accepted fractional method, as opposed to the method Xu et al. used, would significantly reduce the lead MSU has on Leeds. The reason quickly becomes evident: MSU has the highest average number of authors per article (2.5) among the top ten universities. Using the more commonly accepted fractional method, The University of Reading, for example, would move up from a tie for fourth to a third place ranking, reflecting its emphasis on single-authored articles (average number of authors per article: 1.4). Even more dramatically, the University of Miami would move up from ninth to fifth place, caused primarily by Miami’s only prolific scholar, Yadong Luo’s large number of single-authored publications. Overall, differences among universities are generally much smaller when the fractional method is used.

Figure 1
figure 1

Top ten institutional ranking based on total versus fractional number of articles. MSU Michigan State University, CUHK Chinese University of Hong Kong, MBS Melbourne Business School, NYU New York University

Full size image

Aggregation to the institutional level

A third important choice in constructing a ranking system is the approach used to aggregate individuals to institutions. The first question is how to assign university affiliation, based on current affiliation or on affiliation at the time a particular paper was published. Xu et al. deviate from other ranking studies by using current affiliation rather than affiliation at the time of publication. Given their stated aim of identifying those institutions likely to have the greatest impact in driving the future research agenda in IB, this might be defendable (assuming one presupposes that these prolific authors have ceased to be nomads and won’t, once again, move on to other universities). The choice of Xu et al., however, has important implications, especially for the majority of institutions with only one or a limited number of prolific researchers (out of the 81 institutions ranked by Xu et al., three-quarters have only one prolific author). The University of Miami, for instance, would not have done as well if its only prolific researcher, Yadong Luo, had had his earlier publications counted towards the University of Hawaii (where he had been a faculty member before moving to Miami). A similar example is that of Klaus Meyer, who moved from The University of Reading (where he was employed for only 2 years) to The University of Bath in the Autumn of 2007. If Meyer had moved a bit earlier, Bath would now be featured in the upper half of the list of institutions hosting the most prolific authors, whereas Reading would have dropped out of the top-ten altogether.

The second important issue concerning aggregation is whether to count all authors in a specific institution or only prolific authors. The method of Xu et al. again deviates from other ranking methodologies by including only prolific authors in their institutional count. There could certainly be an argument for wanting to focus only on researchers who have published a significant number of papers. When assessing institutions, however, their breadth of researchers active in IB and therefore their lack of vulnerability to rankings-loss caused any one particular scholar leaving, should also count. Not counting all authors risks creating idiosyncratic results. When I searched the UTD Top 100 Business School Research Rankings™ database maintained by the University of Texas at Dallas (http://citm.utdallas.edu/rankings)Footnote 6 for publications in JIBS (the only IB journal included in this database) between 1996 and 2006 (the period coved by Xu et al.), I found the University of South Carolina (USC) at the top with 16 articles in JIBS. This institution ranks only 17th in Xu et al. with 12 articles in six journals. So counting only prolific USC researchers results in a total of only 12 articles published in six different journals, whilst including all USC researchers results in a total of 16 articles published in just one of these six journals. Wharton and INSEAD suffer an even more severe fate. Although they are ranked fourth and sixth respectively in terms of the total number of articles published in JIBS between 1996 and 2006 (13 and 11 respectively), they do not even appear among the top-81 institutions when the process of Xu et al. of counting only the publications of prolific authors in six journals is used. Whereas the institutional ranking of Xu et al. is correctly entitled “institutions currently hosting the most prolific authors in the leading international business journals (1996–2006)”, their ranking, unfortunately, is likely to be interpreted very differently by some readers. More concerning, the authors claim that their approach identifies educational institutions poised to lead IB scholarship, yet they never justify the inherent confound. Are they in fact identifying researchers, at least based on their selection of six journals and not modified for their single- or multi-author status, rather than institutions, who are most likely to lead the field in the future? They have certainly given us food for thought and I hope their paper will spark a discussion about this important topic.