Introduction

Since their emergence just less than a decade ago,Footnote 1 global university rankings (GURs) are perceived as having had a significant impact on the structure, organisation and practices of higher education internationally. Their path of growth and adoption has been multi-linear, both in terms of the range of audiences they have attracted, and the range of uses they have been put to. An international study by Hazelkorn (2009) concluded that rankings had transformed from a benchmarking tool to a quasi-policy instrument, used to direct how funds are allocated at institutional and national levels.

This paper considers the ways the academic research community has responded to the growing prevalence of GURs and influenced the effects on higher education (HE) policy and practice internationally. It considers the different ways research relating to rankings has been problematized and where it falls short. The analysis shows how the research perspective adopted relates to the nature of policy and practice responses proposed; both in terms of the audience(s) they are addressed to; and the extent to which policy and practice solutions are structural and systemic or practice oriented.

This paper contributes to the field of research on GURs by making explicit some significant dimensions of difference in the research perspectives that have been adopted. It highlights some of the implications of the prevailing research orientation for the trajectory of GUR development. The implications of this analysis are considered and the paper makes recommendations in terms of research practices which can strengthen dialogue between research discourses in the field of GURs.

The global university rankings (GUR) phenomenon

The growing influence of rankings is attributed to the rising rates of participation in higher education across the world (OECD 2010), the increasing importance placed on universities as countries strive towards knowledge economy status, and the way research and innovation are becoming key commodities in the global economic context (Marginson and Van der Wende 2007).

The discourse of rankings has penetrated the international higher education (IHE) community and become embedded in the practices of: agenda setting (at governmental as well as institutional level); advertising and self-promotion; and in the mediation of HE partnerships. Rather than being hapless victims of rankings measurements, research studies are beginning to show the active practices of HEIs and governments in the use of global rankings (Hazelkorn 2011; Wedlin 2011; Locke 2008).

Over the last decade, research on GURs has proliferated and, in the main part, has been critical of rankings for a variety of reasons: such as the inherent problems of trying to suggest there is a single ‘best’ university (Marginson and Van Der Wende 2007); the cultural values embedded in the rankings systems (Ishikawa 2009); the weaknesses of proxy measures used in rankings methodologies (Altbach 2006; Salmi and Saroyan 2007); and the adverse effects that rankings have on individuals, universities (Ackers 2008; Hazelkorn 2007) and national education sectors (Hazelkorn 2009).

Despite the growing research evidence on their shortcomings, the uptake and prominence of GURs has been increasing. Several studies have demonstrated the growing number of universities globally who claim to have aspirations to achieve a top 200 ranking and the extent to which institutional practices are increasingly geared towards rankings optimisation (Hazelkorn 2009; Hazelkorn 2011).

The stakes are high. Rankings are introducing a new way of attributing value which is influencing the relationships within and between HE institutions and their nation states. There is now a concerted set of intergovernmental initiatives responding to rankings which seek to mediate their role in certain waysFootnote 2 and which have the potential to create far reaching effects. It is vital that such developments are informed by a broad spectrum of research which tests the claims inherent in rankings as well as the underlying reasons for their appeal.

Theoretical perspective

This paper uses an analytical framework developed by Deetz (1996) to assist in framing the research perspectives represented by different inquiry communities relating to GURs.

Several research framing perspectives have been applied to characterise and distinguish prevailing orientations in research (e.g. Paulston 1992; Burrell and Morgan 1979). The potential limitation of such classifications is that they reify distinct research orientations and position them in ways that appear mutually exclusive. The ‘metatheory of representational practices’ framework (Deetz 1996) has been selected, therefore, for its explicit aim of exploring the interrelationships between research perspectives. Deetz refers to research ‘discourses’ rather than research ‘paradigms’ to emphasise this fluidity.

The first axis of the framework relates to how research questions are formulated (see Fig. 1). This characterises research projects which bring ‘a priori’ hypotheses to the field of study and are concerned with the testing of theory driven propositions. In such studies, the research ‘object’ is predefined and remains relatively static during the process. By contrast, Deetz draws attention to research studies where the research object is ‘emergent’ and defined during the research process in a way that is informed by, or achieved in collaboration with, research participants.

Fig. 1
figure 1

Contrasting dimensions from the metatheory of representational practices (Deetz 1996)

Full size image

In the second dimension, Deetz distinguishes research which achieves ‘closure’ through differing discursive practices which emphasise unity or separation. Consensus orientated research is characterised by its tendency to focus on the relationship between research and ‘existing social orders’. Those located at the dissensus pole “consider struggle, conflict and tensions to be the natural state” (Deetz 1996, p. 197).

These contrasting research perspectives seek to achieve different outcomes. The dissensus orientation perspectives research value in terms of the ‘generative capacity’ of an observation to maintain and open up further lines of enquiry whereas the consensus orientation is portrayed as one searching for ‘closure’ and restoring of social order.

This framework is not intended to label and fix research studies; it is recognised that researchers are, to an extent, ‘multilingual’ and can bring more than one perspective (fruitfully) and, in other cases, can mix metaphors and perspectives in ways that are not helpful.

This analytical framework is used to review a series of research studies that have occurred during the period co-incident with the development of the major GURs (2003 onwards) as these studies are posed directly in relation to global university rankings systems.

A literature review was conducted on education and social science databases.Footnote 3 A variety of search terms related to ‘global university rankings’ were used: world, global, higher education, university, league tables, rankings. A total of six searches were applied using different permutations of these terms and this yielded between 29 and 129 results per search. In each case, more than half the results did not relate to global rankings directlyFootnote 4 and nearly a quarter of search results related to articles in Higher Education Trade press rather than research journals.Footnote 5 With a few exceptions, these press articles were excluded from the review on the basis that they related to media commentary on rankings performance rather than reporting primary research on GURs. Each literature search identified a number of articles in common with the result that the literature review is based on a sample of 36 papers (see “Appendix” for more detail on the literature search).

Research discourses on global university rankings

The analysis of the literature was conducted in three main stages. Firstly, a simple analysis of each paper was undertaken. This summarised the research question(s) of the study, the methods of data collection and analysis and the study’s conclusions and policy/practice recommendations. Secondly the studies were mapped against the a priori/emergent axis of the framework according to whether the research problem was formulated in relation to existing bodies of knowledge, privileged theories or whether they were formulated into a local and emergent problem or phenomenon. Thirdly, the main conclusions and recommendations of each study were evaluated in relation to the consensus/dissensus axis of the framework. A final area of consideration was on those aspects not easily captured by the Deetz framework.

The mapping of research studies against the research discourses framework suggests that a larger proportion of studies can be located at the ‘a priori’ end of the spectrum (see Fig. 2). Of these studies, there is a dominant tendency to propose policy and practice solutions that are consensus orientated.

Fig. 2
figure 2

Mapping of GUR research literature in relation to the metatheory of representational practices framework

Full size image

A smaller proportion of studies are located at the emergent end of the spectrum in relation to the way research questions are formulated. Of these studies, a stronger tendency is evident towards conclusions and recommendations of a dialogic nature.

As such, the research literature is depicted as being quite polarised and clustered in two, somewhat opposing quadrants of the framework. This mapping process is intended to illustrate tendencies in the literature, not to reify them. Several studies appear to straddle perspectives to a degree and some researchers appear to develop new perspectives in subsequent studies. The ways in which the differing perspectives can be seen in the nature of research questions and study recommendations and the different forms of ‘research multilingualism’ will be considered in the following sections.

Elite/a priori orientations in rankings research

The analysis demonstrates the concentration of studies which can be located in the elite/a priori perspective; in testing a pre-determined hypothesis. Such studies tend to be asking and answering quite tightly defined questions: of what rankings measure (Dill and Soo 2005; Usher and Savino 2006); whether they produce differences that are statistically significant (Locke 2008; Halffman and Leydesdorff 2010); if they measure what they purport to (Locke 2008; Docampo 2008; Salmi and Saroyan 2007; Ioannidis et al. 2007; Taylor and Braddock 2007); if they use the correct unit of analysis (Pascarella 2001; Saisana 2011).

Moloney (2011) and Aguillo et al. (2010) identify the strong concordance in terms of those universities at the top rankings both within and across several of the global rankings year on year. Whilst there is more volatility among mid and lower level positions there is very little overall movement in and out of the top 200 (in the case of THES) and 500 in the case of AWRU. Bowman and Bastedo (2011) attribute this to an ‘anchoring effect’ whereby the institutional ranks corresponded highly with reputational scores derived from peer reputation surveys in the first year of the THES world university ranking.

This notion is further supported by research which uses secondary statistical analysis of global rankings data. Locke (2008) undertakes a statistical analysis of the THES ‘World University Rankings’ and ‘Academic Ranking of World Universities’ (ARWU) and demonstrated that the scores within each variable differed from a normal distribution and also in the standard deviation scores meaning that the contribution of each variable varied in its contribution to the overall score. Analysis of the ARWU showed that only the top ranked institution (Harvard) was clearly separated from the others and the statistical differences between other institutions were much smaller and fairly consistent across the range.

Common to these types of studies is a search for order and pattern and to detect implicit theory governing how rankings are constituted and applied. This extends to macro studies which seek to correlate rankings performance with other forms of country performance (Marginson and Van Der Wende 2007), national HE policy (Yonezawa 2007); and those studies which seek to impose order through improving categorisation on rankings systems themselves (Dill and Soo 2005; Usher and Savino 2006).

Other studies within this perspective frame the question more broadly and evaluate the propositions implied by rankings against other established theories relating to learning quality and educational effectiveness. Pascarella (2001) classifies three types of rankings according to those which measure: (1) resources and reputation; (2) student and alumni outcomes; and (3) effective educational practice and processes. He draws on an established literature on correlates of learning quality and challenges rankings which are based on the first two implicit theories and argues for those which are framed around the third.

From a slightly different stance, two major studies have assessed the composition of ranking systems in terms of how they relate to established models of quality assurance. Dill and Soo (2005) pose the question of ‘whether there is a converging international definition of academic quality’ and, in their analysis of domestic ranking systems from four countries (America, UK, Canada and Australia), suggest there is a degree of convergence. However, in an expanded study of 19 global and regional rankings systems, Usher and Savino (2006) challenge this assertion and identify a considerable degree of heterogeneity of indicators (across the 19 ranking systems, they found no single indicator in common) .

A group of statistical studies ask a more fundamental question relating to whether rankings measure what they purport to measure (Ioannidis et al. 2007; Coates 2007; Pike 2004; Billaut et al. 2010). Coates (2007) applies linear modelling and regression analysis to the Course Experience Questionnaire administered in all Australian universities, and used as a basis for national benchmarking and ranking. He demonstrates three approaches to modelling the data which is suggestive of different ways of ‘reading’ quality. Through this analysis he demonstrates that intra-university variation can be greater that inter-university variation and, therefore, levels a serious challenge to the use of ‘university’ as the unit of analysis in GURs.

Likewise, Pike (2004) demonstrates the effect of remodelling rankings data by using the variable of student entry scores as a control variable rather than an input measure and demonstrates the significant effect this has on reducing variation. He also demonstrates the low correlation between the component measures of rankings with measures taken in the American National Survey of Student Engagement (NSSE).

A further vein of research can be defined in terms of meta-analysis of rankings systems against good practice. Pike (2004) identifies five dimensions by which the value and utility of rankings can be assessed. He draws attention to a key shortfall of rankings—functionality; ie. how do they help an institution identify ways to improve performance. A common criticism levelled at rankings is their a-theoretical nature and lack of transparency in what is really being measured with the result that HEIs are more likely to engage in ‘gaming behaviour’ to optimise rankings positions.

Within the research literature there is a surprisingly small focus on whether rankings produce beneficial outcomes for HEIs. Monks and Ehrenberg (1999) explored the impact of changes in rankings position of US universities (in the US World Report domestic rankings). This study showed an increase in applications, resulting in increased selectivity, resultant in an increased grade point average of entrants and consequent increase in tuition fees. In an expanded study with a larger number of institutions, Meredith (2004) found a similar effect on admissions outcomes and, furthermore, identified that a one rank drop in the US News ‘World Report’ ranking could alter the social and racial demographic of the institutional intake.

A similarly small number of research studies have focused on the extent to which countries outside the ranking leaders (currently US and UK) are ‘gaining ground’. Analyses have shown that universities in the lower reaches of the top 200 are better at translating research income into citation impact than the top ranked universities (Morgan 2010) and, using regression analysis, Li et al. (2011) demonstrates that non-western universities are ‘catching up’ in the rankings.

Whilst these studies share a degree of similarity in respect of a more predetermined problem definition, they can be compared according to the way in which they seek closure and draw conclusions and policy implications from their findings. This will be considered in the following section.

Forms of closure: normative or critical

From the ‘elite/a priori’ dimension, there are vested questions at stake. For national funders and policy makers, do rankings support or conflict with national policy objectives? For the social science community, is there congruence with established educational theory? For HE practitioners, do rankings increase the institution’s access to resources and increase autonomy?

In their definition of the ‘research problem’, these studies have a shared concern to reveal some sort of truth; to objectively and rigorously test a proposition implicit or explicitly contained in rankings or in the wider research literature.

Such studies have drawn attention to the balance of power vested in measures, how rankings, rankings formulae and rankings practices create new ‘winners and losers’. The implications, conclusions and policy recommendations posited from these studies tend towards a normative orientation, that is, of conforming to the discourse of consensus and re-establishing of order. Few of the studies challenge directly the fundamental basis of rankings discourse in terms of the conclusions and recommendations formulated.

Many of these studies are critical but conclusions are largely related to how the problems can be corrected, how gaps can be lessened, of how congruence can be enhanced. Posited solutions include: the use of a different unit of analysis (by discipline rather than institutional level) (Coates 2007); clustering institutions in bands rather than vertical ranks (Salmi and Saroyan 2007; Nolle 2010); disaggregating data (Billaut et al. 2010); using input variables such as entry scores as a control variable (Pike 2004; Pascarella 2001); and reducing reliance on peer reputation surveys (Bowman and Bastedo 2011; Taylor and Braddock 2007).

Recommendations at the macro level tend towards structural solutions: for example, a classification system to create clearer typologies of institution (Marginson and Van Der Wende 2007; Salmi and Saroyan 2007); and developing international datasets to augment rankings (Coates 2007).

A lesser but discernible dissensus discourse can be identified in this literature. Altbach (2006) challenges the practices of media companies largely responsible for producing rankings for ‘engineering volatility’ within rankings to maintain interest and sell newspapers. A similar critique is put forward by Saisana (2011) and recommendations are centred on reducing the influence of rankings by calling for greater proliferation and multidimsionality of measures which are driven more by academic needs and interest.

This form of critique aligns with the ‘a priori/dissensus’ quadrant of the Deetz framework in which power is conceptualised as something that can be quite clearly defined and located (e.g. in media companies and top ranked HEIs). Such analyses can point to the nature and direction of power shifts and for example, a western hegemony in terms of the constructs of excellence inherent in rankings (Altbach 2006; Marginson 2009). However, this perspective can overlook the active practices of the wider HEI community in using rankings for a range of purposes and the active role taken by governments in applying rankings to achieve specific policy ends.

Local/emergent orientation in rankings research

‘Local/emergent’ studies take a different perspective. Such studies can be characterised more by a search for meaning than truth and, as such, the research object can be transformed or reconceptualised within the process.

The ‘local/emergent’ research problem can be found in the rankings research literature from a number of vantage points. The initial issue for research is framed by the researcher in a way that is often stimulated by direct experience: of being a recipient of a request for data from a rankings agency (Ishikawa 2009); of being faced with a growing rankings discourse while an academic leadership role within an HEI (Cuthbert 2011) or in a policy leadership role in an intergovernmental organisation (Salmi and Saroyan 2007); or perceiving rankings related issues in research interviews on researcher mobility (Ackers 2008).

This research perspective is in evidence not so much in the methods (of which there is great plurality) as the research questions which tend to be of a relational nature. Characteristic to these studies is an intention to engage research participants either in the research methods chosen, or the methods of analysis, or in defining alternative policy and practice responses at micro, meso or macro levels.

There is a perceptibly different discourse of research intention from this perspective. Ishikawa’s (2009) starting point is to provide an ethnographic account of being located in a planning unit of a Japanese university approached by a UK based global rankings organisation (THES/QS) to provide global rankings information. The specific intention of the study is to provide a “behind the scenes view” and draw attention to some of the ranking agency’s practices which were experienced as inappropriate and unethical or culturally insensitive.

Acker’s study (2008) provides a perspective from early career researchers who experience the adverse influence of EU policies oriented towards fostering staff mobility (within the THES global rankings, a high percentage of international staff is taken as a marker of internationalisation). She addresses the question of whether mobility and international excellence are mutually constitutive.

Such studies tend to be constructed to focus on the interaction between GURs and the contexts in which they are experienced. Hazelkorn (2009) focuses on the interplay between rankings and their recipients: in the case of HEIs as being both the object of rankings as well as consumers of this information. In an adjacent field of international business school rankings, Wedlin (2011) draws attention to the active role played by European business schools in engaging in rankings development to broaden the field and confer legitimacy on programmes which differ from the US model. Martins (2005) examines the role managerial sensemaking and organizational identity play in determining the extent to which organizations conform to, or resist, the pressure toward homogenization imposed by rankings.

Sauder and Espeland (2009) locate their study in the field of law schools where the uptake of the rankings discourse has been particularly vigorous. Their focus of interest is on the representational value and symbolic effectiveness of rankings within these contexts. As such their methods include ethnographic and open interview formats.

Common to many of these studies is their ‘problematisation’ which in many cases is conceptualised as a matter of discourse: the use of rankings as a short hand for expressing global aspirations, as a conflation of two metrics (Ackers 2008); or a reductive managerial discourse (Cuthbert 2011).

More common to the emergent perspective is a reflexivity on the part of the researcher in maintaining an openness in relation to how the research problem is approached and constructed. Salmi and Saroyan (2007) pursue an appreciative enquiry method in their study of the growing use of rankings by national governments as a means of challenging their own conception and understanding the roles and functions served by rankings.

Such studies place value on understanding reasons for uptake as well taking a broader focus on the developing practices around rankings. In this way, this research orientation de-objectifies rankings; thus opening up avenues of further enquiry. In terms of their contribution towards theory development, these studies display a greater generative capacity (Deetz 1996) than those of an a priori orientation.

Closure: dialogic or interpretive?

Many of these studies display a dissensus orientation but in ways that position the source of power or oppression as something more decentred and constituted by a confluence of contributing factors, not least the behaviour of universities themselves. Cuthbert (2011) perceives a choice for senior managers in responding to rankings as ‘amplify or educate’. Hazelkorn (2011), too, recognises that rankings are both a symptom of, and accelerator for, heightened international competition.

Recent studies have acknowledged the disciplining effect of rankings, whereby universities appear to internalise, or ‘institutionalise’ the values inherent in rankings (Sauder and Espeland 2009; Locke 2011). Sauder and Espeland (2009) liken rankings to a contemporary form of discipline as exemplified by Foucault in medical and penal institutions. Key to the pervasive nature of rankings is their construction as commensurate, relative and broadly circulating measures. To analyse how rankings impose discipline one needs intimate knowledge or organisational practice:

As Deetz perceives it, from the ‘dialogic’ perspective attention is focused on creating a space for a continually changing world, recovering the marginalised. “The problem is not group against group but suppression of parts of one’s individual being. Processes of domination are often located in the micro practices.”(Deetz 1996: p. 197).

This is evident in the forms of policy and practice solutions expressed within these studies which are less ‘structural’ in nature and tend not to be aimed at particular targets of oppression. As a consequence, these studies tend to propose responses to the effects of rankings which are similarly practice-based and call upon the direct action of those who are ranked: in terms of sense-making, re-interpretation, re-framing and creating new discourses. There is a more evident belief in the possibilities of local action rather than the necessity of systemic change. (Cuthbert 2011) highlights the managerial options he perceives as a senior academic manager in a UK university and opts for the role of reframing the debate. Adler and Harzing (2009), in a similar way, point to the possibilities for institutional leaders to reframe priorities and for academic researchers to ask different questions in relation to research impact.

Hazelkorn’s (2009) analysis of effects of rankings highlights a strong interplay between rankings and the behaviour of HEIs. Her policy response is non-prescriptive but ‘calls to action’ in emphasising that policy does matter. Rather than making specific systemic policy recommendations she restates the long term aspirations which she believes are shared by education policy makers and seems to reflect the situational and context-related aspects of policy making, leaving the detail to be guided by local conditions.

Research from this perspective focuses more on following the logic of rankings and identifying possible scenarios and trajectories within a wider frame of reference. Ishikawa (2009) extrapolates far beyond the local site of ethnographic study and speculates on a newly forming hegemony and urges the need to discredit rankings or risk seeing a new social group become dominant.

Unlike those studies from the elite/a priori perspective which have a tendency to address the solutions and policy recommendation to specific communities which look in on the problem (e.g. researchers, policy makers), studies from this perspective more commonly address calls to action which involve the community directly affected by the rankings (the HEI practitioner community).

Considerations of consensus and congruence exist also from the local/emergent research perspective but less in relation to established, privileged bodies of truth. Rather, consideration of congruence is in relation to how values implicit in rankings align with values implicit in HEIs. Malcolm Grant (Grant 2011), Vice Chancellor of University College London, and vociferous critic of rankings, frames his critical analysis in term of a series of ‘fractures’ in which the intent internalised in rankings runs counter to, and in tension with, the values and intent of universities.

In contrast to many of the studies located in the local/emergent perspective, Lo (2011) presents an analysis which is more aligned with the interpretive quadrant of the framework. In the study, he conceptualises rankings as something actively deployed and, to some extent, ‘managed’ by national governments. He proposes that GURs provide a missing link in a continuum of hard and soft power applied by nation states to maintain a level of self-determination in response to forces of globalisation.

Salmi and Saroyan (2007) also express a more interpretive stance and consensus orientation in their proposal to ‘de-objectify’ rankings. In their exploration of the uses and misuses of rankings as a policy instrument, they focus initially on the constructive roles that rankings have played in opening debate, creating greater transparency and availability of HEI performance indicators across national borders. In perceiving the options as: ignore, boycott or engage with rankings, they favour the latter and highlight the importance of seeking to understand the significance of rankings.

Discussion

This paper has demonstrated the dominant pattern of research relating to global university rankings as normative, a priori and orientated towards correcting the flaws identified in rankings systems. A smaller proportion of research studies take a relational perspective and investigate the role and meanings attached to rankings; and the dynamics and trajectory of their developments. These differing orientations can be seen as quite polarised when analysed and mapped against the research practices framework (as shown in Fig. 2).

The analysis presented in this paper has considered the contribution made from studies that can be located in an ‘elite/a priori’ perspective. Taken collectively, these studies have provided a robust critique of claims of measurement inherent in global university rankings systems. Several studies have demonstrated significant weaknesses in using the ‘university’ as the unit of analysis and have highlighted flaws in the modes of presentation and interpretation of rankings information.

Beyond the critique, a range of alternatives are portrayed, largely through a consensus orientation: changing the unit of analysis, changing the status of some input measures to control variables (thereby creating a stronger focus on the value added by universities); and by proposing strategies of holding rankings agencies to account.

The forms of policy response proposed can have the effect of reframing how GURs are read and what rankings are taken to mean. However, there are some limitations imposed by this research orientation. There tends to be an underplaying of the role HEIs have themselves played in accelerating the trajectory and impact of rankings. Locke’s (2008) comprehensive analysis and broad range of methods (institutional survey, case studies, interviews with rankings) was constrained in part by this perspective. In the closing remarks of the research report, Locke concluded that HEIs were, in fact, using the global rankings in quite vigorous ways. Given the pre-formulated nature of the research question and a priori perspective the study could not entirely take account of this dynamic in the way data was gathered.

This type of a priori research makes a significant contribution in holding rankings agencies to account, educating consumers in how to interpret rankings information and identifying ways of enhancing rankings systems to achieve a more functional, enhancement driven outcome. However, this consensus orientation still runs the risk of maintaining and reproducing the status quo within higher education, creating risk-averse behaviours (Hughes 2010) and reasserting traditional hierarchies of knowledge production (Hazelkorn 2011 p. 200). Furthermore, there is a growing perception that global rankings can directly or unwittingly entrench a set of norms of higher education quality and excellence which reflect largely westernised constructs and measures (Ishikawa 2009).

The tendency within research conceptualised in this way is to evaluate rankings in their own terms and less in terms of the meanings attached by a growing range of audiences. The proposed responses to the issue are largely framed in these terms too and are, in the main, concerned to with solutions that ‘close the gaps’ or ‘perfect the tool’. The frame of reference in such studies can be inevitably narrower than those located in the emergent perspective and horizon of possible alternatives is more constrained. There is a stronger possibility that research located in this perspective further elevates the research object and strengthens the rankings discourse; thereby closing down other alternative discourses relating to quality in international higher education.

Those research studies located at the ‘local/emergent’ end of the spectrum can be seen to make a different sort of contribution. With a greater interest and focus on the meaning and symbolic effectiveness of rankings, several studies have provided greater insight on the reasons why rankings have gained such currency, among HEIs in particular (Hazelkorn 2009; Sauder and Espeland 2009; Wedlin 2011; Salmi and Saroyan 2007). These studies have helped to reveal the complex practices surrounding rankings at institutional level that show there is a significant interplay between rankings and rankings recipients.

This research perspective tends to have a broader horizon of interest and long term focus; both in describing the potential outcomes of following the logic of rankings discourse and identifying where suppression occurs as a consequence. These studies have value in aiming to refocus attention on what the broader aims are within the international higher education context; and how to continue to pursue them. More commonly following a dissensus discourse, these studies tend to demonstrate that alternatives exist; that mindsets can be changed; and that local action is possible, with or without systemic change.

The use of an analytical framework such as this one can fall prey to the criticism that is reifies different research perspectives. However, a key part of the research discourse framework’s utility is its claim to restore commensurability and open fruitful discourse and debate across different research communities.

Several studies considered in this analysis demonstrate the ability of researchers to straddle perspectives fruitfully. Salmi and Saroyan (2007) use appreciative enquiry methods in their problem formulation and investigation. Hazelkorn (2009) incorporates a ‘co-researcher/co-constructor’ aspect to the type of questions formulated in a primarily quantitative survey to deconstruct which aspects of rankings are most and least aligned with academic values and practices. Furthermore, some researchers seem to develop new orientations and reconceptualise the object of research in subsequent studies; as is evidenced in Locke’s shift from a critical towards a more dialogic orientation in later work (Locke 2011). What is less common however, is an exchange of questions ‘across the lines’ of different perspectives; consideration of what questions are not answered in the research and opening up further lines of enquiry.

Conclusion

This study has considered the ways in which differing research perspectives influence the nature of public policy recommendations proposed in the emerging field of interest in global university rankings. The analysis has shown that research problems formulated in relation to a priori theoretical positions tend to favour structural and systemic policy responses. Those research questions formulated through more contextualised, local situations seem more connected with practice responses that recognise the possibilities of local action and discursive re-framing of the issues addressed by rankings.

Despite the high concentration of research exposing the methodological weaknesses of GURs and significant questions regarding their validity, interest and engagement with GURs continues to spread across the globe. In its normative orientation, the prevailing research perspective demonstrates a consensus focus in recommendations and contributes, therefore, to the legitimization of rankings (Hazelkorn 2011, p. 11).

Ignoring rankings may not be an option given their ‘potency’ (Marginson 2012). Yet, seeking to see beyond the current appeal of rankings remains crucial. As Hazelkorn asserts, it is the need for national and international comparisons which is here to stay:

Rankings can be seen as the favoured format of the moment because of their ability to provide simple information to a wide-ranging audience but this is also their Achilles heel. (Hazelkorn 2011, p. 81)

This calls for something of a reorientation in research terms. In furthering the research line, this study argues for a stronger dialogical orientation. There is a need for more studies informed by theoretical perspectives which draw attention to the complex dynamics at play, on how the role of rankings are mediated in different contexts and how the role of rankings is evolving and changing in relation to broader systems of activity (O’Connell and Saunders 2012).

The desired result of such a refocusing of research would be a better understanding of the heuristic value of rankings and of where the current tools fall short. Such research may help identify new developmental lines for international transparency instruments that are regarded as being more ‘fit for purpose’.