Identifying patterns and structural influences in the scientific communication of business knowledge

Abstract

This study uses several quantitative techniques to enable a multidimensional analysis of 47 key business journals by analyzing the scientific communication patterns and structural influences of these journals. Apart from using clustering techniques to establish research clusters in the Business domain, we apply a refined PageRank method by differentiating between the citation types to enable a cross-sectional evaluation of the selected journals. The results indicate that the five most influential journals are from Finance and Economics. The selected Finance journals are knowledge hubs and the selected Economics journals are knowledge sources when ISI’s entire journal database is considered. However, within the Business domain, the selected Finance journals appear to be high impact knowledge hubs while the selected Economics journals appear to be high impact journals despite weak citation activity. All in all, such analyses are beneficial to scholars when selecting publication outlets to showcase their research, and to agencies such Financial Times and Bloomberg when selecting their journals basket for their annual journal evaluation exercises.

Introduction

Evaluations of scholarly business journals¹ are increasingly being documented in the literature in lieu of the wide-ranging intellectual and practical utility they provide for a variety of academic constituents today (Cheang et al. 2014a). Indeed, other then government-linked agencies that use journal evaluations to aid decision-making where the allocation of scholarly research funding is concerned, university administrators use journal evaluations to facilitate decision-making on matters such as faculty hiring, tenure, and promotion, while librarians use information from such evaluations to manage their literary procurement budgets, and scholars use such information to direct their research resources and foci (Nisonger and Davis 2005; Xu et al. 2011).

By that extension, and because of the commercial implications associated with student enlistment, annual rankings of business school programs such as the MBA and EMBA are also increasingly being reported; where such program ranking reports are based on an institution’s research output from a specially selected set of business journals. For instance, Financial Times provides annual rankings of business schools, MBA programs and EMBA programs based on 45 selected journals,² while Bloomberg Business Week annually ranks full-time MBA programs based on 20 selected business journals,³ and the Naveen Jindal School of Management at the University of Texas at Dallas (hereafter known as UTD) annually provides rankings of business schools based on faculty publications from 24 selected business journals.⁴

Undeniably, journal evaluations facilitate a host of purposes. However, the quality or impact of a journal is a multifaceted concept (Rousseau 2002) since the nature and outputs of research are also characteristically multifaceted (Martin 1996). For in reality, research impacts may be scientific (where “scientific” may also be from theoretical, empirical, or methodological points of view), educational (which is concerned with the advancement of knowledge, skills or training), technological (which chiefly pertains to the development of new or improved technological applications), and/or cultural/societal (which typically involves research into understanding or uncovering cultural/social phenomena) (Cheang 2014; Kostoff 1995). Accordingly, Martin (1996) contends that since the nature of research is multifaceted, its outputs would also yield multifaceted impacts; that therefore, “no single indicator of research output or performance will ever reveal more than a small part of the multi-dimensional picture” (p. 346). Forasmuch, leading bibliometricians appear to concur with Martin’s view (i.e. Butler 2008; Kostoff 1995; Moed et al. 2012; van Raan 2005; Weingart 2005). In the same vein, Kostoff (1995) suggests that because there are multiple facets of research impact, “its assessment must use as many methods and as many types of experts as required to address as many of these components as possible” (p. 8). Yet, studies have shown that research evaluations tend to be ranked-based, one-dimensional, and deficient in depth and scope (Cheang 2014; Bornmann et al. 2008; Frey and Osterloh 2006; Martin 1996; Moed 2008; Nederhof 1988; Lawrence 2003; Weingart 2005).

Technically speaking, it defies logic to rank journals because journals are but a mode of scientific communication; that the publication of scientific knowledge is not a form of competition (Cheang 2014). Instead, journal evaluations should serve as a complementary tool for the peer review process (i.e. in identifying the characteristics or impacts of scholarly journals) (van Raan 2005). However, such forms of evaluations are impracticable without the employment of objective yet valid scientific approaches, computing technologies, and usable data sources from which to analyze. Accordingly, van Raan (2005) proposes that it is imperative that multidimensional/qualitative journal evaluations are quantitatively derived; but as a consequence, some forms of evaluations may involve rank-based methods.

Thereupon, the motivation of this study is to furnish the business research community with a multidimensional assessment of 47 key business journals using a combination of citation-based methodologies; that by analyzing their citation behavior, we can identify patterns and structural influences of these key business journals in the scientific communication network (Pollock 2009). In particular, we first apply Pieters and Baumgartner’s (2002) log-multiplicative citation model to establish the cohesiveness of the selected journals. Then, we apply the standard Ward’s clustering method to derive the journal clusters based on their citation patterns. These two steps enable the identification of highly related journal clusters that facilitate a more reflective evaluation (Cheang 2014). Upon clustering the 47 key journals, we apply a refined PageRank method originally proposed by Xu et al. (2011) to enable a cross-sectional evaluation of the selected journals. Additionally, we apply both Lim et al. (2009) and Cheang’s (2014) approaches to identify the types of knowledge roles that the selected journals play.

The remainder of this paper is organized as follows. In Section “Literature review”, we review the relevant literature pertaining to this study. Section “Methodology” expounds on the various methodologies that this study utilizes. Delimitations and parameters used for our study are provided in Section “Delimitations and parameters used for study”. We then present and discuss the results in Section “Results and analysis” and conclude our work in Section “Discussion and conclusion”.

Literature review

Journal clustering

The past several decades have witnessed an immense surge in the production of business knowledge. This is, in part, due to new disciplines and sub-disciplines being formed over time (research diversification), leading to the subsequent development of new or hybridized research problems (research specialization) that have, in turn, sparked the introduction of new scientific communication outlets that reflect the dynamic and organic business research landscape (Martin 1996; Meredith et al. 2011). Indeed, before the turn of the new century, Operations Research/Management Sciences (OR/MS) and Management Information Systems (MIS) joined the ranks of longstanding domains such as Accounting, Economics, Finance, Marketing and Management as key disciplines in most business schools (Cheang et al. 2014a). But in spite of their youth, the sub-domain research scopes of both disciplines have developed and continue to develop rather quickly. For instance, the MIS domain, which properly began in the early 1980s, has developed quickly in terms of research diversification and specialization to include topics such as artificial intelligence in IS, business process re-engineering data management and modeling, decision support systems, electronic commerce, group support systems, human computer interface, Graphical User Interfaces, hypermedia, virtual organizations, inter-organizational systems, IT enabled learning and knowledge management, IS project management, policy, strategy, security and privacy, technology transfer and innovation adoption, and so forth. Resultantly, high-impact journals from these newer domains are being included by agencies such as Financial Times, Bloomberg, and UTD in their annual school or program ranking exercises to reflect their relevance to business research and knowledge.

However, although the aforementioned domains are assumed under the business knowledge umbrella, it is observed that citation tendencies and dynamics can and do differ among and within related domains and sub-domains (Seglen 1997). For instance, while Brown (2003) evaluated 18 accounting and finance journals and observed that “different journals do not equally represent papers published in the financial versus other areas” (p. 305), Rainer and Miller (2005) evaluated 50 of the top MIS journals and identified that 29 of them are “pure” MIS journals, and that out of the top 20 journals derived from their evaluation method, six are considered to be “pure” MIS journals, while two are from management, three are from operations research, and nine are from computer science. Therein, there are domains and sub-domains of business research that may be strongly or weakly associated with one another. Thus, it may be more germane to cluster highly related journals so that the impact or quality of a journal can be directly compared to other highly related journals (Inkpen 2001). In line with this concept, Pieters and Baumgartner (2002) proposed the symmetric log-multiplicative citation model to analyze the cohesiveness among economic journals to demonstrate the efficacy in establishing intra- and inter-disciplinary communications of economic journals. Subsequently, Baumgartner and Pieters (2003) and Xu et al. (2011) applied the log-multiplicative citation model to identify subareas in marketing and OR/MS, respectively. Based on field knowledge, both studies were able to rationalize the identified clusters of both journal sets. Hence, the results of these studies encourage us to employ the model and identify journal clusters in business research.

Underlying method for multidimensional evaluation

Although opinion-based surveys are quite common,⁵ citation-based journal evaluations have become more sought after with its seeming objectivity (compared to opinion-based surveys), the emergence of information and communication technologies (ICT), and the burgeoning accessibility and availability of online databases facilitated by cutting-edge web technologies (Patra et al. 2005; Olson 2005; Vokurka 1996). For the latter approach, bibliometricians rate journals via some specified evaluation criteria based on citation data, such as considering total citations, considering only internal citations,⁶ excluding self-citations,⁷ and so on. Having said that, since its inception in the mid 1950s, numerous studies⁸ have either based their evaluations on or decried the validity and reliability of the Impact Factor (IF), perhaps the most widely adopted approach to derive a journal’s impact score. Indeed, the issues surrounding the IF are numerous and well documented, including measuring only citation frequency within a narrow time window, how the IF distorts results, and the method’s susceptibility to manipulation due to its simplicity (Alberts 2013; Butler 2008; Cameron et al. 2001; Harter and Nisonger 1997; Nederhof 2006; Nederhof et al. 2001; Lewison 2002; Saha et al. 2003; Seglen 1997). Furthermore, because the nature and outputs of research are characteristically multifaceted, it is often argued that the IF hardly captures the quality or impact(s) of a journal (Martin 1996; Olson 2005; Rousseau 2002; van Raan 2005).

Forasmuch, the numerous issues of the IF and the constant need for reliable journals information by the various academic constituents have instigated the impetus for bibliometricians to craft better journal evaluation mechanisms (Leydesdorff 2008). Wherefore, of the newer approaches, more and more independent groups of scholars appear to favor the PageRank method—a web-search technique originally developed by Google’s founders in the late 1990s to rank webpages by their popularity (Page et al. 1999; Cheang 2014). Indeed, Google’s PageRank method is so popular that it has since been modified and used as the underlying method in at least three recent online bibliometric platforms⁹ and even more journal evaluation studies across numerous domains (Cheang 2014). But of the many PageRank-based studies, those by Xu et al. (2011); Cheang et al. (2014a) and Cheang et al. (2014b) produced multifaceted journal evaluations through isolating citations by types (i.e. internal citations, self-citations, external citations, total citations). Apparently, the isolation process effectively enabled them to assess journals from multiple dimensions, such as how influential the selected journals are among all SCI-indexed journals (total citations), how influential the selected journals are within and out of their domains (internal and external citations, respectively), and how specialized the selected journals are in general (self-citations). Therefore, as our aim is to produce multifaceted evaluations of a selected set of business journals, we adopt the same approach as undertaken by Xu et al. (2011), Cheang et al. (2014a) and Cheang et al. (2014b). Additionally, the journals under assessment are further categorized into clusters of highly related journals to enhance the quality of comparisons of journal impact among related journals (Inkpen 2001).

Identifying knowledge roles of journals

In a recent study by Yan et al. (2013), a trading metaphor was used to analyze citation characteristics of academic journals to determine how self-dependent, dynamic and impactful an academic discipline is, as well as, which disciplines serve as knowledge importers/exporters based on whether a given discipline possesses knowledge deficit/surplus when compared to other disciplines. Along a similar vein, Lim et al. (2009) had earlier proposed and applied another citations-based approach to assess the structural influence of MIS journals. The difference, however, is whereas the trading approach that Yan et al. (2013) undertook served to study knowledge exchange between academic disciplines, Lim et al.’s (2009) was to determine a journal’s overall impact in transferring knowledge. According to Lim et al. (2009), a journal may be classified as a knowledge source, hub, or store: whereby (a) knowledge sources are immensely influential journals that receive significantly more citations compared to other journals in a particular research domain. In the context of Yan et al.’s (2013) trading metaphor, knowledge sources tend to have trade surpluses; (b) knowledge hubs are influential journals that not only exchange significant knowledge in a given research domain, they also transfer significant knowledge to journals from other domains. Therefore, when compared to other journals, knowledge hubs are more frequently cited in, as well as, out of their primary research domains. But unlike knowledge sources, knowledge hubs comparatively tend to cite numerous other sources; and (c) knowledge stores are generally not influential journals as they are typically less cited than the rate they cite others. In the context of Yan et al.’s (2013) trading metaphor, knowledge stores typically have trade deficits.

Over and above the proposal of the knowledge roles approach by Lim et al. (2009) is that the results produced are presented in the form of plots on a graph; where Cheang (2014) notes that such a method is especially conducive for studies that evaluate large numbers of journals since the results are easily visualized and referenced. Hence, recent studies such as the one by Cheang et al. (2014a) have also begun implementing this method as a supplementary form of evaluation to identify the structural influence of journals.

Be that as it may, Cheang (2014) recently pointed out that pioneer studies merely evaluated the roles of journals with an overall perspective as they only took total citations into consideration, and subsequently proposed studying internal citations in isolation to identify how influential/impactful a given journal is in a particular domain since some academics may regard internal impact/influence to be more important than overall impact/influence. Thereupon, with all these factors in mind, we apply both Lim et al. (2009) and Cheang’s (2014) approaches to analyze the overall and internal citation relationships, behaviors and therefore, the structural influences of the selected journals under study.

Methodology

Log-multiplicative citation model and clustering analysis

We apply the log-multiplicative citation model (Pieters and Baumgartner 2002) on a 47 × 47 citation matrix to derive the cohesion score for each journal. Here [C _ab] is used to denote this matrix and C _ab denotes the number of times journal a cites journal b. We note the normalized citation matrix as [P _ab ], where $P_{\text{ab}} = \frac{{C_{\text{ab}} }}{{\mathop \sum \nolimits_{i = 1}^{47} C_{\text{ai}} }}$. The log-multiplicative citation model we use for this study is shown below,

$$ \log \hat{P}_{ab} = u + u_{a}^{S} + u_{b}^{S} \;{ + }\;\sigma_{ab} + \mathop \sum \limits_{m = 1}^{M} \xi_{a}^{m} \varPsi^{m} {\in_{b}^{m}} $$

where $\hat{P}_{ab}$ is the estimated value for $P_{\text{ab}}$. The $u$ is the constant log-linear parameter, while the u ^S _a and u ^S _b represent the overall effect of the citing and cited data for all other journals, respectively. The σ _ab denotes the effect of self-citations, while the term $\mathop \sum \limits_{m = 1}^{M} \xi_{a}^{m} \varPsi^{m} {\in_{b}^{m}}$ is a log-multiplicative term that represents the cohesion of journals in the citation graph network.

We employ ℓEM (Vermunt 1996) to derive the parameters in the above model. Based on the goodness of fit, we select M = 3 as the ideal number of characteristics.¹⁰ There upon, for each journal a (1 ≤ a ≤ 47), we focus on three characteristics (namely $\xi_{\text{a}}^{1}$, $\xi_{\text{a}}^{2}$ and $\xi_{\text{a}}^{3}$) of each journal to determine their cohesiveness to other journals. After computing the three characteristics for each journal, we then apply the standard Ward’s method, a classical hierarchical clustering procedure (Punj and Stewart 1983), to identify the inherent clusters for the 47 selected journals.

Transposed PageRank for journal evaluations

Briefly discussed in the literature review, the original PageRank method developed by Page et al. (1999) to rank webpages has since been transposed to analyze citations data for journal evaluation purposes (Cheang et al. 2014b). The transposed PageRank essentially consists of two major components. The first is the development of a citations graph network made up of nodes that serve as proxies for journals and edges that indicate the citation information of journals. The second component involves modeling the problem. According to Lim et al. (2007, 2009); Xu et al. (2011), and Cheang et al. (2014a, b), the problem can be modeled via solving a set of linear equations where the impact score of a given journal is expressed as a variable of positive value. Using the random walk method (Pearson 1905), the equations iteratively compute the transitive relationships among the citations until the values converge/stabilize. As such, the iterative equation can be formulated as follows:

$$ {\text{PRI}}_{\text{i}} = \mathop \sum \limits_{{{\text{j }} \in J}} p_{\text{ji}} {\text{PRI}}_{\text{j}} $$

where J is the set of journals; PRI represents the journal impact based on the PageRank model; p _ji is the proportion of citations from journal j to journal i to citations from journal j to all journals in journal set J. Thereafter, journal i’s impact score is divided by the number of articles the journal publishes every year to establish the average impact/influence of the journal’s articles. The derived value is known as the Article PageRank Impact score (or APRI).

Refined PageRank for multidimensional journal evaluations

In reality, the concept of journal impact is highly subjective and therefore, it has become a contentious topic of discussion (Seglen 1997; Glänzel and Moed 2002; Harzing 2007; Weingart 2005). Indeed, some academic constituents simply regard high total citation count as being of high impact while others may, for instance, regard a journal as being manipulative if it has high self-citation counts (Rousseau 2002; Smith 2006; Weingart 2005). Having said that, the subjective nature of research evaluation has subsequently brought about the hypothesis that some of these viewpoints may be appropriated if techniques can be refined to reflect them (Xu et al. 2011). More specifically, Xu et al. (2011) and later, Cheang et al. (2014a, b) suggest that enabling varying viewpoints may be accomplished by differentiating between the various citation types. According to the authors, there are three types of citations, namely internal citations, external citations and self-citations. Here, we employ the same concept as in these previous studies and use Fig. 1 to illustrate the different citation types. The corresponding citation relationships among these four journals are also provided in Table 1, which we further explain as follows:

Fig. 1

Illustration of three different citation types

Table 1 Examples of quantifying different citation types as shown in Fig. 1

Let us, for example, consider two different areas: Business and Engineering; Let us denote that J1 is the Journal of Finance, J2 be Management Science, J3 be the IEEE Transactions on Pattern Analysis and Machine Intelligence, and J4 be the IEEE Transactions on Computers. Under the “Internal Citations” column, we denote the citation patterns of journals in the same area (i.e. Business Journals). Thus, for example, C _2,1 represents the number of citations that J1 received from J2 and vice versa for C _1,2. Under the “External Citations” column, we denote the citation patterns of journals with journals from an unrelated area (i.e. Engineering). Thus, for example, C _3,1 represents the number of citations J1 received from J3, and C _4,1 represents the number of citations J1 received from J4. Under the “Self-Citation” column, we denote the citation patterns of journals that cite themselves. Thus, for example, C _1,1 represents the number of citations that J1 received from itself.

Identifying overall knowledge roles

Depending on the APRI score and percentage of total citations garnered, a journal is positioned as a plot in a graph. This is represented as RJ _i  = (X _i , Y _i ), where RJ _i refers to the overall role that journal i plays, while X _i denotes the APRI score of journal i, and Y _i denotes the percentage of citations that journal i made to a selected set of journals (denoted as CJ) in relation to the total number of citations that journal i made to the universal set of journals (denoted as UJ) in a given time period. According to Cheang et al. (2014b), we can formulate Y _i as:

$$ Y_{i} = \frac{{\mathop \sum \nolimits_{{j \in {\text{CJ}}}} C_{i,j} }}{{\mathop \sum \nolimits_{{j \in {\text{UJ}}}} C_{i,j} }} \times 100\% $$

where C _i,j represents number of citations that journal i made to journal j in a given time period.

Figure 2 shows four quadrants that journals may be positioned in a graph: where journals plotted anywhere inside the top left quadrant are considered knowledge stores, journals plotted anywhere inside the top right quadrant are referred to as knowledge hubs, journals plotted anywhere inside the bottom left quadrant are considered indistinguishable journals, and journals plotted anywhere inside the bottom right quadrant are said to be knowledge sources (Cheang 2014).

Fig. 2

Illustration of overall knowledge roles

Identifying in-domain knowledge roles

According to Cheang (2014), in order to identify the roles that journals play within a particular domain, only internal citations need be mathematically manipulated. In addition, three data elements are required: incoming citations from the core (or “InValue” for short), outgoing citations to the core (or “OutValue” for short), and the APRI value of the journal. Cheang (2014) also gives the mathematical definitions of the InValue and OutValue of journal j as follows:

$$ {\text{InValue}}_{j} = \frac{{\mathop \sum \nolimits_{i \in CJ} C_{i,j} }}{{N_{j} }},\;{\text{OutValue}}_{j} = \frac{{\mathop \sum \nolimits_{i \in CJ} C_{j,i} }}{{N_{j} }} $$

where N _j is the number of articles that journal j published in a given time period.

However, this means that there are therefore three dimensions to consider; which complicates the presentation and thus visualization of the results using a three-dimensional (3D) space drawing to map so many journals in. To resolve this issue, Cheang (2014) proposed the elimination of the third dimension via the bifurcation of the results into two two-dimensional (2D) graphs. Thereupon, each graph’s x-axis would denote the APRI score of each core journal while each graph’s y-axis would denote a journal’s InValue score. But to incorporate the OutValue dimension of each core journal into the computation, Cheang (2014) suggests the bifurcation of the journals into two categories, namely journals with “Low OutValue” of journals with “High OutValue”; that whether a journal is classified as having low OutValue or high OutValue depends on the cut-off point of the bifurcation, which is essentially the median value of all OutValue scores. Figures 3 and 4 illustrate the general positional outputs of the two graphs as proposed by Cheang (2014).

Fig. 3

Illustration of in-domain journal roles with low OutValues

Fig. 4

Illustration of in-domain journal roles with high OutValues

Delimitations and parameters used for study

For our study, we are particularly concerned with the following components: (1) data sources and datasets, and (2) period and parameter settings. These components are described below:

Data sources and datasets

Although data are available from sources such as Google Scholar, SCOPUS, and the Institute of Scientific Information (ISI), we use citation data from the ISI’s Journal Citation Report (JCR) because their database presents the most efficient means of the three to extract the incoming and outgoing citations.

In terms of the datasets, a universal journals set (or UJ) and a core journals set (or CJ) are required. The UJ consists of all the journals indexed by the ISI’s JCR while the CJ is composed of 47 business journals that are evaluated by the Financial Times, Bloomberg and UTD. The UJ covers all subject categories in the ISI’s JCR including not only Business and Business Finance but also all the remaining subject categories such as Acoustics, Agricultural Economics & Policy, Agricultural Engineering, and so forth. The number of journals in the UJ in our study is 10,144.

Period and parameter settings

Other than establishing the UJ and CJ, we outline the relevant period and parameters for our study. First, to obtain a timely evaluation, we consider the time period that would have relevance for the various constituents. As a result, we elected to focus our study on citations from journal articles published during the time period from 2006 to 2010 since it is a recent point in time.

As for the settings, there are actually four different citation parameters: the first setting is where we only consider internal citations, therefore it is denoted as (S = 0, E = 0) to mean that self and external citations are disregarded; the second setting is denoted as (S = 1, E = 0) to mean that while internal citations are also considered, we now include the self-citation parameter; the third setting is denoted as (S = 0, E = 1) to mean that while internal citations are also considered, we now also factor external citations but disregard self-citations; the fourth setting is denoted as (S = 1, E = 1) to mean that all citations are considered.

Results and analysis

Analysis of clusters

In the process of determining the number of clusters, we took two factors into consideration, namely the values of M (which represents the ideal number of characteristics to consider) and the number of clusters. Based on the goodness of fit, the hierarchical results and the interpretability of the partitions based on discussions with colleagues from business schools, we finally chose the results of a nine-cluster solution for when M = 3¹¹. In comparing the clustering results based on different values of M, we observed the following:

• All in all, Finance, Economics, Marketing, Accounting, OR/MS, Management, Management Information Systems, Applied Psychology and Interdisciplinary Magazine/Journal have been identified as distinct journal clusters; these identified clusters are also consistent with existing clusters under the Business umbrella. Additionally, the clustering results are overall quite stable when different values of M were applied.

• While the journal compositions for the Finance and Marketing clusters did not change with different values of M, the Economics cluster was found to consist of the same set of journals with the exception of when M = 4 (where Journal of the American Statistical Association (JASA) is not considered as being in the Economics cluster). Similarly, journals in the Accounting cluster did not appear to vary with different values of M (with the exception of Accounting Organizations and Society (AOS) being in this cluster when M = 2).

• The OR/MS cluster is observed to have a number of variations when different values of M are applied. For instance, when M = 4, Operations Research (OR) was left out of the cluster while the same occurred for INFORMS Journal of Computing (IJOC) when M = 5. However, we found Journal of Operations Management (JOM) to be within the cluster when M = 4.

• With the exception of two instances, the Management cluster is more or less consistent when different values of M were applied: (1) Human Resource Management (HRM) is out of the cluster while Accounting, Organizations and Society (AOS) is in the cluster when M = 5; and (2) Journal of Business Ethics (JBE) is out of the cluster when M = 4. We also found that when combined (FT, Bloomberg, and UTD), there are disproportionately more Management journals than all other journal clusters.

• We found that while the Applied Psychology cluster is consistently made up by Journal of Applied Psychology (JAP) and Organizational Behavior and Human Decision Processes (OBHDP), Accounting Organizations and Society (AOS) was included in this cluster when M = 3.

• We also found that Information System Research (ISR), MIS Quarterly (MISQ), Harvard Business Review (HBR), MIT Sloan Management Review (SMR) and California Management Review (CMR) formed different clusters when different values of M were applied. For instance, when M = 2, these five journals along with JOM were identified to be in the same cluster. However, when M = 3, only HBR, SMR and CMR appeared to belong together. Thus, we coined this cluster “Interdisciplinary Magazine” (IM) since the readership of these three magazines, in particular, are for both practitioners and scholars.

• Last but not least, we found the IM, Applied Psychology (AP) and Management Information Systems (MIS) clusters to have the least amount of journals (three each) compared to the other clusters.

Multidimensional results in clusters

Table 2 lists the multi-dimensional placements of the 47 selected journals based on the clusters they are computed to belong to. Column 1 shows the cluster name for each journal while Columns 2 and 3 display the acronym and full names of the 47 journals, respectively. Columns 4, 5, 6 and 7 show the placements of journals based on their APRI values as defined in Section “Transposed PageRank for Journal Evaluations” and based on the four parameter settings established in Section “Delimitations and Parameters used for study”. These parameters are (S = 0, E = 0), (S = 1, E = 0), (S = 0, E = 1) and (S = 1, E = 1).

Table 2 Multidimensional results in clusters

Analysis of journal placements by cluster

The following is a cluster-by-cluster analysis of some of the selected journals:

Finance

• It is interesting to note that although there are only four journals selected in this cluster, each one is rated in the top ten with the exception of Journal of Finance and Quantitative Analysis (JFQA) when (S = 1, E = 0), (S = 0, E = 1) and (S = 1, E = 1). Having said that, in spite of JFQA being placed lowest of all of the journals in the cluster, it is still placed in the top 30 % for all but the (S = 1, E = 1) parameter.

• Overall, the relative placements of journals in this cluster make Finance the most influential cluster. For indeed, Finance is highly related to Economics and Accounting, two longstanding business research disciplines. At the same time, Finance-related research is also a hot topic in non-business domains such as statistics and applied mathematics.

Accounting

• It appears that Journal of Accounting and Economics (JAE) and Journal of Accounting Research (JAR) are very similar in their level of influence throughout the four parameters, making them the most influential journals in their cluster. Review of Accounting Studies (RAS) and Accounting Review (AR) are also similar in their level of influence throughout the four parameters, while Contemporary Accounting Research (CAR) fares the worst in all four parameters among the selected Accounting journals. Having said that, in terms of internal influence, JAE and JAR are among the top ten journals, while AR and RAS are in the top 30 % list, and CAR is not too far off the top 30 %.

• As mentioned in the analysis of the Finance cluster, Accounting is highly associated with Finance and Economics. Therefore, their internal influence is generally high. Having said that, their external influence is also relatively high. This citation behavior also indicates an abundance of Accounting research in non-business domains such as statistics and applied mathematics.

OR/MS

• As stated earlier, OR/MS is a relatively new discipline in business research. In spite of that, five OR/MS journals are regarded (by FT, Bloomberg, and UTD) as key journals in the discipline. This is one journal more than Finance, and two journals more than AP, IM and MIS; with the exception of MIS, the other clusters have been key disciplines in business research for much longer. Thus, we believe that FT, Bloomberg, and especially UTD recognize the importance of OR/MS research in the business domain.

• But drilling deeper into their internal influence, the picture is not as clear. With Management Science (MS) leading the way, Manufacturing & Service Operations Management (M&SOM), and Production and Operations Management (POM) are all placed in the 20 s, while Operations Research (OR) is placed 33rd, and INFORMS Journal of Computing (IJOC) is placed 42nd out of 47 journals. From another perspective, as a relatively new discipline, such placements show OR/MS has come a long way.

• In a recent evaluation of 31 OR/MS journals, Cheang et al. (2014b) found M&SOM to be the most influential journal in the field but placed 5th out of 31 OR/MS journals for external influence, while MS was found to be the most influential journal externally, but was placed 4th out of 31 OR/MS journals internally. As a result, in the context of our study, we note that MS is placed higher (20th) than M&SOM (25th) internally since it involves the evaluation of journals in other related disciplines under the business research umbrella.

• Without an intimate knowledge of OR/MS research, the citation behavior among the selected journals indicate that OR and IJOC are less influential journals internally. However, we are of the opinion that the citation behavior does not capture the fact that these two journals predominantly publish research based on quantitative techniques, whereas journals from longstanding disciplines such as Management, AP, and Marketing have traditionally published research based on qualitative methods. In a way, this plausibly explains why MS, M&SOM, and POM generally fare better than OR and IJOC internally; for indeed, MS, M&SOM, and POM do publish research based on qualitative methods. So this plausibly explains why OR (31st) and (especially) IJOC (32nd) fare better in their placements externally. Both of these journals generally attract highly technical research associated with techniques from disciplines such as computer science and applied mathematics; but these journals only publish research in the context of operational research. Resultantly, because they are highly specialized, journals from non-business disciplines such as computer science or applied mathematics only sparingly cite them.

Economics

• In this cluster, with the exception of the Journal of the American Statistical Association (JASA) (45th out of 47), all of the journals are well placed internally, making the Rand Journal of Economics (RJE) placing 12th out of 47 the second lowest placement.

• As stated earlier, Economics is highly related to Finance and Accounting. Therefore, it is unsurprising that Economics journals are also of a high level of internal influence. Having said that, we turn our attention to JASA, which has placed poorly internally. But when looking at its external influence, JASA is placed 7th out of 47. This enormous disparity shows JASA to be far more influential externally; indeed, it is a highly influential journal in domains such as statistics and/or applied mathematics. And perhaps, JASA’s significant external influence may explain why FT included the journal in their journals selection.

AP

• As a long-standing discipline in business research, we had expected that there would be more selected journals than just the three. Perhaps, AP has become a highly specialized area of research.

• What is also interesting is that Accounting Organizations and Society (AOS) has been clustered under AP. AOS is the least influential journal in this cluster (38th out of 47) and while it fares slightly better under the self-citation parameter (33rd out of 47), it fares even worse externally (44th out of 47). These placements suggest that the research published by AOS is more behavioral-based and is likely specialized in scope.

• As well, Journal of Applied Psychology (JAP) and Organizational Behavior and Human Decision Processes (OBHDP) appear to be significantly more influential externally.

Management

• With 13 journals, Management not only has the largest number of selected journals, it has significantly more journals than any other cluster. Moreover, Management is a well-established discipline. Therefore, it is rather surprising that the selected Management journals, in general, are not as internally influential as disciplines such as Finance and Economics (with Academy of Management Review (AMR) being placed highest in this cluster at 14th out of 47, and Journal of Business Ethics (JBE) being placed last for all parameter settings).

• Perhaps the respective agencies should reconsider their journals evaluation basket and replace some of these journals with journals from other business research areas.

Marketing

• The five journals in this cluster have rather consistent placements for all four parameter settings. Of these, Journal of Marketing Research (JMR) (18th out of 47) and Marketing Science (MKS) (19th out of 47) appear to be of similar influence internally, while the Journal of Consumer Psychology (JCP) appears to be the least internally influential journal in the cluster (29th out of 47th). Even so, JCP is more internally influential than eight of the 13 Management journals.

• It is also interesting to note that JCP has been clustered under Marketing rather than AP. This suggests that the research scope of JCP is closely associated with Marketing.

IM

• Traditionally, there has not been a cluster for business magazines. However, magazines such as California Management Review (CMR), Harvard Business Review (HBR), and MIT Sloan Management Review (SMR) have different publishing behaviors and structures from scholarly journals. One of the reasons for these differences is that these magazines not only attract scholars, they predominantly attract practitioners. These magazines tend to provide scientific information without going into technical or methodological specifics, and magazines like HBR do not provide detailed references in their articles. Additionally, the authors that publish in such magazines typically have technical versions of the work published in scholarly journals as well, resulting in the citations being diverted to those scholarly journals rather than the magazines. These explain why their placements are generally poor (although overall, HBR is a stunning 6th pout of 47). This speaks volumes of HBR’s overall influence as a scientific communication outlet in the academic community despite the lack of citation evidence.

MIS

• On the one hand, as a relatively new discipline, it is understandable that only three journals have been selected as key journals for this cluster. On the other hand, we note that OR/MS was established as a key discipline about the same time as MIS and it has five selected journals. This may suggest that MIS research is not as influential as OR/MS. Indeed, we note that compared to placements of OR/MS journals, MIS journals are clearly less internally influential. With somewhat an intimate knowledge of MIS, we believe that MIS became a key discipline because of the prolific development and application of information and communication technologies (ICT) in business networks. The research scope appears to be how information systems are managed, developed, and implemented in organizations; that while the research scopes of internal journals are very different to MIS, MIS journals likely tend to cite themselves, other journals in the MIS discipline, or cite journals in other associated yet external disciplines (i.e. computing or artificial intelligence). As a result, two of the three journals in this cluster (Journal of Operations Management (JOM) and Management Information Systems Quarterly (MISQ) are placed higher externally and overall compared to their internal placements.

• It is also interesting to note that the citation behavior of JOM indicates that it is more closely associated with MIS rather than OR/MS or Management since the research context of JOM is predominantly based on OR/MS research while the research methods used by JOM authors are predominantly qualitative in nature (which is also the predominant approach for Management research). Having said that, we note that MIS research is predominantly based on qualitative approaches as well; this plausibly explains the higher connection of JOM to MIS.

Structural influences of business research domain

In this subsection, we aggregate journals in each cluster into one node. As there are nine clusters, there are then nine distinct nodes. All citations received from journals outside a particular cluster (or node) to all journals inside a given cluster are considered incoming citations of the node; All citations cited by any journal within a given cluster to journals outside the cluster are considered outgoing citations of the node; All citations that are received and/or cited by journals within a given cluster are considered self-citations of the node. With these nodes, we are then able to employ the overall knowledge role analysis and the in-domain knowledge role analysis described in Sections “Identifying overall knowledge roles” and “Identifying in-domain knowledge roles” respectively. Accordingly, we first report and analyze the results for the overall (see Fig. 5) and then the in-domain (see Figs. 6, 7) structural influences of the business research domain below:

Fig. 5

Overall knowledge roles of the 47 selected business research journals

Fig. 6

In-domain roles of clusters with low OutValues

Fig. 7

In-domain roles of clusters with high OutValues

Based on the results in Fig. 5, we note that the selected Finance journals have the highest aggregated APRI and the second highest aggregated citation percentage of the nine business disciplines. Therefore, these journals are overall knowledge hubs because they are not only highly cited, but these journals also cite journals in and out of the business domain. In the bottom right quadrant, Economics is the only cluster situated there. The results show that the selected Economics journals serve as knowledge sources because while these journals have the least citation activity among all nine clusters, their aggregated APRI score is only second to that of the selected Finance journals. Therefore, the selected Economics journals are extremely influential journals.

There are five clusters in the top left quadrant, which is also referred to as the knowledge store quadrant. In general, knowledge stores tend to cite a lot of journals although fewer journals cite them (Cheang et al. 2014a). Of these, the selected Accounting journals not only have the highest citation activity of all nine clusters, these journals also have the highest aggregated APRI score (in the quadrant). This makes the selected Accounting journals the most influential knowledge store among the five clusters in the quadrant. Lastly, there are two clusters in the bottom left quadrant, which is also known as the indistinguishable quadrant. This means that it is hard to ascertain the structural influence of these journals. Having said that, although the selected MIS journals appear to have higher citation activity as compared to the selected AP journals, the latter cluster has a higher aggregated APRI score.

Next, we report and analyze the in-domain structural influences of the various business-research clusters below:

In Fig. 6, we note that the selected AP and OR/MS journals are situated in the top left quadrant, or the low impact source quadrant. What this means is that these two clusters have low APRI scores and make little outgoing citations. Having said that, these clusters generally receive high incoming citations from other business-related clusters. As for the selected IM journals, the cluster is positioned within the bottom left quadrant and is considered a weak cluster because on top of their low APRI score, they register low incoming and outgoing citation activity. The various reasons for IM being in such a position are stated in the cluster-by-cluster analysis section. Finally, the selected Economics journals are in the low activity, yet high impact quadrant. This is because the selected Economics journals have low incoming and outgoing citation activity, yet the cluster has a high aggregated APRI score.

In Fig. 7, we note that the selected Marketing and Management journals are in the (top left) low impact hub quadrant; these two journal clusters are hubs because they have high InValues and OutValues (high incoming and outgoing citation activity), but because they have low aggregated APRI scores, they are of low structural influence. Next, we note that the selected Accounting and Finance journals are in the (top right) high impact hub quadrant. Both clusters have high incoming and outgoing citation activity and both have high APRI scores. Having said that, Finance trumps Accounting on all indicators. Lastly, MIS is situated in the (bottom left) low impact store quadrant: that while the selected journals often cite other journals in the business domain, fewer journals from other clusters cite these MIS journals. Furthermore, these selected MIS journals are of a very low aggregated APRI score. This means that these journals are the least structurally influential in the business research domain.

Structural influence of the 47 selected business journals

In this subsection, we report and analyze the overall (see Fig. 8) and in-domain (see Figs. 9, 10) structural influences of the 47 selected journals in the business research domain. We begin with the report and analysis of the overall structural influence of each journal below:

Fig. 8

Overall structural influence of the 47 selected journals

Fig. 9

In-domain roles of journals with low OutValues

Fig. 10

In-domain roles of journals with high OutValues

Based on Fig. 8, the majority of the selected journals serve as knowledge stores (top left quadrant) overall, with HBR leading the pack in terms of overall influence. Recall that for various reasons stated earlier, HBR is rated poorly internally (39th out of 47) but is highly cited and rated in the top ten overall (6th out of 47). In the knowledge hub (top right) quadrant, we note that Review of Financial Studies (RFS) sits at the APRI borderline but has high citation activity overall. Therefore, it is the least influential knowledge hub of the three journals there, while Journal of Finance (JF) is the most influential knowledge hub. Next, there are three journals in the knowledge source (bottom right) quadrant. While all three journals have similar citation activity levels, Journal of Political Economy (JPE) is the least influential source while Quarterly Journal of Economics (QJE) is the most influential knowledge source. Tying in with Fig. 5, the selected Economics journals cluster is observed to be overall a knowledge source. We can pinpoint why this is so from Fig. 8; that three of the journals (JPE, QJE, and Econometrica) are all in the knowledge source quadrant while the other three journals in the Economics cluster (namely JASA, AER, and RJE) are relatively close to the knowledge source quadrant. Finally, in the indistinguishable (left bottom) quadrant, we note that two of three selected AP journals (namely OBHDP and JAP) are in this quadrant.

Next, we report and analyze the in-domain structural influences of the 47 selected business journals. In Fig. 9, we report the in-domain journals with low OutValues. We note that there are no journals in the bottom right quadrant while there are two journals (namely QJE and JPE) in the high impact knowledge source (top right) quadrant. Both journals are from the Economics cluster. Interestingly, the other four journals from the Economics cluster (JASA, RJE, AER, and Econometrica) are in the bottom left quadrant; which means these journals are not as influential internally and receive little citations from within the business domain. Having said that, the majority of journals with low OutValues are in this cluster and RJE, AER, and Econometrica, all Economics journals, have much higher APRI scores than the other journals in the quadrant. Journals in the top left quadrant are low impact knowledge sources.

In Fig. 10, we note the in-domain knowledge roles of journals with high OutValues. In particular, we note that Journal of Finance (JF) is the only journal in the high impact knowledge hub (top right) quadrant as it has the highest APRI score, has high OutValue and relatively high InValue scores. JFE and RFS, also Finance journals, are the only two journals in the high impact knowledge store (bottom right) quadrant. These outputs of the selected Finance journals serve to demonstrate why the Finance cluster is the most influential cluster in the business-research domain. AMR and AMJ, both Management journals are the only two journals in the low impact knowledge hub (top left) quadrant. What this means is that they have low APRI scores in general but they have comparatively high InValue and OutValue percentages. Finally, the majority of journals in the bottom left quadrant are low impact knowledge stores since they have low APRI scores in general and have comparatively lower InValues and OutValues.

Discussion and conclusion

Instead of the typical one-dimensional, rank-based approach to journal evaluations, this study used several quantitative techniques to identify the scientific communication patterns and structural influences of 47 key business journals. The results show numerous interesting findings. For instance, JOM’s citation activity shows the journal to be more closely associated to the selected MIS journals rather than the selected OR/MS or Management journals. Other analyses also show that the selected Finance journals are knowledge hubs while the selected Economics journals are knowledge sources when ISI’s entire journal database is taken into account. But when only Business research activity were analyzed and compared, the selected Finance and Accounting journals appear to be high impact knowledge hubs while the selected Economics journals appear to be high impact journals despite weak citation activity in the Business domain. Resultantly, we find that the five most influential journals are from Finance and Economics, while the majority of the selected journals from other Business disciplines such as Management, OR/MS, Accounting and Marketing appear to be knowledge stores and are generally of lower impact compared to the selected Finance and Economics journals.

Although a ranked-based methodology was used, the framework proposed for this study was aimed at identifying the source(s) of a journal’s impact from multiple perspectives. This is to showcase that quantitative techniques can be used to derive meaningful (qualitative) evaluations and characterizations of journals as opposed to ranking journals. All in all, the results also suggest that agencies such FT, Bloomberg, and UTD may want to reevaluate their journals basket for future journal evaluation exercises since certain journals from longstanding research clusters such as Management have become less influential or impactful with the emergence of new areas and subareas of Business-related research.

Having said all that, a multidimensional evaluation of all Business journals would have been more ideal; but due to space constraints, this study only focused on a small albeit key subset of Business journals. Additionally, this study lacks a longitudinal analysis of the communication patterns and structural influences of Business journals, which may shed more light on the upward or downward shifts of journals under evaluation.

Appendices

Appendix 1: 45 journals used in FT research rank

The list below details the 45 journals used by the Financial Times in compiling the Business School research rank, included in both the Global MBA and EMBA rankings.

• 1. Academy of Management Journal (Academy of Management)

• 2. Academy of Management Perspectives (AMP)

• 3. Academy of Management Review (Academy of Management)

• 4. Accounting, Organisations and Society (Elsevier)

• 5. Accounting Review (American Accounting Association)

• 6. Administrative Science Quarterly (Cornell University)

• 7. American Economic Review (American Economic Association)

• 8. California Management Review (UC Berkeley)

• 9. Contemporary Accounting Research (Wiley)

• 10. Econometrica (Econometric Society, Wiley)

• 11. Entrepreneurship Theory and Practice (Baylor University, Wiley)

• 12. Harvard Business Review (Harvard Business School Publishing)

• 13. Human Resource Management (Wiley)

• 14. Information Systems Research (Informs)

• 15. Journal of Accounting and Economics (Elsevier)

• 16. Journal of Accounting Research (University of Chicago, Wiley)

• 17. Journal of Applied Psychology (American Psychological Association)

• 18. Journal of Business Ethics (Kluwer Academic)

• 19. Journal of Business Venturing (Elsevier)

• 20. Journal of Consumer Psychology (Elsevier)

• 21. Journal of Consumer Research (University of Chicago)

• 22. Journal of Finance (Wiley)

• 23. Journal of Financial and Quantitative Analysis (Cambridge University Press)

• 24. Journal of Financial Economics (Elsevier)

• 25. Journal of International Business Studies (Academy of International Business)

• 26. Journal of Management Studies (Wiley)

• 27. 27 Journal of Marketing (American Marketing Association)

• 28. Journal of Marketing Research (American Marketing Association)

• 29. Journal of Operations Management (Elsevier)

• 30. Journal of Political Economy (University of Chicago)

• 31. Journal of the American Statistical Association (American Statistical Association)

• 32. Management Science (Informs)

• 33. Marketing Science (Informs)

• 34. MIS Quarterly (Management Information Systems Research Centre, University of Minnesota)

• 35. Operations Research (Informs)

• 36. Organization Science (Informs)

• 37. Organization Studies (SAGE)

• 38. Organizational Behaviour and Human Decision Processes (Academic Press)

• 39. Production and Operations Management (Wiley)

• 40. Quarterly Journal of Economics (MIT)

• 41. Rand Journal of Economics (The Rand Corporation, Wiley)

• 42. Review of Accounting Studies (Springer)

• 43. Review of Financial Studies (Oxford University Press)

• 44. Sloan Management Review (MIT)

• 45. Strategic Management Journal (Wiley)

Source: http://www.ft.com/intl/cms/s/2/3405a512-5cbb-11e1-8f1f-00144feabdc0.html#axzz2lSDcJben

Appendix 2: 20 top academic journals in business week

How is the intellectual capital score determined?

Bloomberg Businessweek scours 20 top academic journals for articles published by each school’s faculty, reviewing all editions published in the previous 4 years. The journals are The Harvard Business Review, Journal of Marketing, Operations Research, Information Systems Research, Journal of Finance, American Economic Review, Journal of Accounting Research, Journal of Financial Economics, Management Science, Academy of Management Review, Journal of Marketing Research, Strategic Management Journal, Accounting Review, Academy of Management Journal, Production & Operations Management, Journal of Business Ethics, Journal of Consumer Research, Review of Financial Studies, Administrative Science Quarterly and Marketing Science. Extended articles receive three points; short articles receive one point.

Source: http://www.businessweek.com/articles/2012-03-19/faq-full-time-mba-rankings#p2

Appendix 3: 24 journals used in UTD ranking

• The Accounting Review

• Journal of Accounting and Economics

• Journal of Accounting Research

• Journal of Finance

• Journal of Financial Economics

• The Review of Financial Studies

• Information Systems Research

• Journal on Computing

• MIS Quarterly

• Journal of Consumer Research

• Journal of Marketing

• Journal of Marketing Research

• Marketing Science

• Management Science

• Operations Research

• Journal of Operations Management

• Manufacturing and Service Operations Management

• Production and Operations Management

• Academy of Management Journal

• Academy of Management Review

• Administrative Science Quarterly

• Organization Science

• Journal of International Business Studies

• Strategic Management Journal

Source: http://jindal.utdallas.edu/the-utd-top-100-business-school-research-rankings//index.php

Appendix 4: Hierarchical results of standard ward’s clustering procedure

In this Appendix, we supply Hierarchical results of Standard Ward clustering analysis for M with values of 2, 3, 4 and 5. We also provide explanations of identified clusters in the following Figures.

M = 2

M = 3

M = 4

M = 5