Mediating the Use of Global University Rankings

Abstract

This study explores responses to rankings from a group of staff working as education partnership facilitators for a professional intermediary organisation, the British Council. The study adopts an activity systems perspective from which to view the contexts in which rankings are encountered and the range of practices used to reduce tensions created by rankings, or reconcile their effects. The article illustrates how rankings have become embedded as a new form of infrastructure in the international education “context of practice”; as a tool (to enable benchmarking); as a form of exchange (to demonstrate credentials to customers and stakeholders); and as a division of labour (in the task of assessing value across national borders). The range of responses to rankings displayed by respondents in this study demonstrates a significant space that has been created for interpretation and reconstruction of meaning. The implications of this analysis for policy makers, practitioners, and researchers are discussed.

Keywords

globalization and international higher education internationalization of higher education national and transnational policies for internationalization in higher education strategic institutional management of internationalization cross-border delivery of education

Introduction

In this article we argue that global university rankings can be examined as a form of “runaway object” within the international higher education community, whose pace and trajectory of development, and range of applications, has gone beyond their initial design intent. In this context, an activity systems analysis (Engestrom, 2005) gives a useful historical, contextual frame to examine the dimensions of development and range responses applied to rankings.

This article explores responses to rankings in the context of a professional intermediary organisation, the British Council (BC), which occupies a distinctive vantage point at the intersection of a broad range of consumers and producers of rankings in a wide range of geographical contexts. As such, this group of respondents are not based in an organisation directly scrutinised by global rankings but experience the effects of rankings in quite direct ways in their interactions with education customers, partners, and clients.

This article examines the ways in which global university rankings represent a new form of infrastructure generated by the international higher education community as it seeks to develop tools of coordination and collaboration. The article explores responses to rankings and how these relate to the range of brokerage roles performed by education facilitators in the international higher education context. The findings within this study give an optimistic view of the ways in which conceptualisations of rankings can be renegotiated and reshaped to serve specific needs more effectively.

The Rise of Rankings Practices

Increasingly, universities express the motivation to be “world class” and are perceived to be developing strategies orientated to “credentialling” and brand building in the global context. Seemingly in parallel, a new industry in global rankings is emerging, which is having significant effects within higher education communities, government agencies and potential customers of Higher Education (HE).

Although some proponents argue that rankings assist in identifying best practice and encourage quality enhancement across HE globally (Wildavsky, 2010), the critics of these rankings systems are numerous and objections range from the philosophical to the pragmatic: The misguidedness of trying to suggest there is a single “best” university (Marginson & Van Der Wende, 2007); the often “westernised” values embedded in the rankings systems (Ishikawa, 2009); the datasets and rankings methodologies being inadequate proxies for what they purport to measure (Altbach, 2006; Salmi & Saroyan, 2007); and the adverse and unanticipated effects that rankings have on individuals (Ackers, 2008), universities (Hazelkorn, 2008) and national education sectors (Hazelkorn, 2009).

Despite these objections, the “appetite” for rankings persists at HE institutional level as well as national governmental level. This can be attributed to the rising rates of participation in HE across the world (Organization for Economic Co-operation and Development [OECD], 2010), the increasing strategic importance of universities as countries move toward knowledge economy status, and the way research and innovation are becoming key commodities in the global economic context (Marginson & Van Der Wende, 2007). In some ways the global rankings phenomenon can be perceived as a manifestation of globalisation—a supra national form of agency, which has introduced a new way of attributing value that is changing the relationship between HE institutions and nation states and, indeed, the status of rankings systems themselves.

Although national governments can choose to maintain varying degrees of control over HE provision, as reflected through the “openness” of their policy environment (Economist Intelligence Unit, 2010), HE institutions are operating in an increasingly globalised environment and global rankings (such as the Times Higher World University Rankings and the Shanghai Jiao Tong University index) are gaining currency as they appear to give an “explanatory framework through which global economy and national (and supra national) positioning can be understood” (Hazelkorn, 2009).

Despite the many criticisms of rankings there appears to be an inexorable rise of rankings practices (activities and behaviours oriented to maximising value derived from rankings or mitigating the adverse effects of a low rankings position), and a proliferation of responses at micro and macro levels. This can be seen, at one level, as inevitable part of a higher education “policy ensemble” (Saunders, Trowler, & Bamber, 2010) demanding of greater levels of accountability and consumer focus. However, rankings can be seen as part of the broader force of globalisation and rankings have been recognised as both symptom of, and accelerator for, cross-border educational competition and collaboration (Hazelkorn, 2011; Lo, 2011). There is a lot at stake. Rankings are changing who defines quality, how it is measured and how it is communicated. The composition of the predominant rankings systems produces a set of norms that are being perceived to have a homogenising (Beer, 2011; Haffman, 2010) and hegemonic effect on global higher education (Altbach, 2006; Hazelkorn, 2008; Marginson, 2009).

Research on Rankings

There is a considerable research literature surrounding global university rankings. A rich vein of this research can be characterised as having a “normative” orientation (Deetz, 1996). These studies have tended to characterise the rankings systems as the primary research object. By definition the research problem is closely tied to the particular characteristics of the rankings artefact. Such studies appear to be asking and answering quite tightly defined questions and there have been in depth analyses of how rankings systems are constituted; the extent to which components measure what they purport to (Salmi & Saroyan, 2007), the relationship of measures to underpinning educational theory (Dill & Soo, 2005; Pascarella, 2001; Usher & Savino, 2006), consideration of implicit theories of value, and cultural bias embedded in measures (Ishikawa, 2009), and correlations between rankings position and organisational and national outcomes (Marginson & Van Der Wende, 2007).

Such research is valuable in challenging and testing the claims inherent in rankings but does not explain and account for the inexorable increase in rankings related practices across the international higher education sector over the past 10 years. In partial answer to this question, there is a body of research that takes a more interpretive stance and seeks to highlight the effects of rankings practices on their recipients. The focus of such research is less on the rankings artefact directly but on the practices surrounding them.

In this vein, there have been a number of powerful and penetrating studies of the effects of rankings on practices at the national and institutional levels. Hazelkorn (2008) has shown the significant affect global rankings are having on strategy and management practice in a study of Higher Education Institutions (HEIs) covering 41 countries and concludes that rankings have transformed from benchmarking tool to a management and policy instrument used in many countries to “direct or inform initiatives or as a quasi-funding mechanism” in a manner that appears to be accelerating.

A study by Ackers (2008) identifies “on the ground” effects on academic staff created by some of the new orthodoxies associated with rankings metrics. She notes the tendency of several rankings systems to conflate mobility with internationalisation and the consequent effect for many of the academics in her study of staff mobility practices that were highly instrumental rather than research-driven.

The research literature demonstrates how a new power dynamic has been introduced to the international higher education environment, one which is significantly affecting practices of different groups. This is vested in rankings measures (research output is judged in terms of publications published in English language medium) and rankings formulae (which favour “full-service” rather than niche institutions); and there is growing evidence that funding follows success in this context (Marshall, 2010). However, rather being than passive recipients of this new form of evaluation, HE institutions are found to be actively engaged in optimising and capitalising on the benefits conferred from a good rankings position (Hazelkorn, 2008); and doing so in quite “vigourous” ways (Locke, 2008).

There appears to be a clear interplay between this new form of infrastructure and its recipients. However, this aspect has tended to emerge as a research finding rather than as a research lens with which to study rankings in context. In this article we propose a theoretical perspective, which reflects this dialectical relationship and provides an appropriate framework to look at this interplay.

Theoretical Perspective

The emergence of the two currently dominant global ranking systems has not been a conventional “top down,” intergovermentally led, process but was, in one case, a domestic benchmarking exercise (in case of ARWU [Academic Ranking of World Universities] Shanghai Jiao Thong Ranking); and in the second a tactical “first mover” action by a media organisation (THES [Time Higher Education Supplement] World University Rankings) to offer a comprehensive consumer guide for students and parents.¹ What is striking is how quickly these rankings have been accepted into discourse and practice within IHE community and how, in their way, they have introduced a certain set of norms and constructions of quality in the International Higher Education (IHE) context. Their path of growth and adoption has been multilinear, both in terms of the range of audiences they have attracted and the range of uses they have been put to.

As such, rankings can be seen as a form of “runaway object,” characterised by Engestrom (2005) as a “rapid proliferation and escalation in range of uses of artefacts that goes beyond the original design intent; and in a trajectory that is not managed and ordered in any “top-down” sense.”

Cultural historical activity systems theory provides a useful perspective through which to view these developments. Its origins trace back through German philosophy (Kant & Hegel) and Russian cultural psychology (Vygotsky & Leont’ev) and have been significantly developed and applied to developmental work research by Engestrom since the 1980s. Challenging cognitive theories of learning, activity systems theory takes a broader unit of analysis and takes as its focus the socially distributed, collective, artefact mediated aspects of human behaviour.

Activity systems theory is predicated on “activity” as the driving force and basis of action. Crucially, activities are conceptualised as social practices oriented at “objects” (Engestrom, 2005, p. 319). The concept of “object-orientedness” is both a central explanation for what makes activity happen and the means of overcoming the traditional gulf, or dualism, between collective and individual units of analysis between macrostructural perspectives and micro, intraindividual perspectives.

Activity systems theory views human and social activity in terms of three central subsystems of social activity: distribution (mediated by division of labour), exchange (mediated by rules), and production (mediated by tools; see Figure 1).

Figure 1.

The human activity system (Engestrom, 1987, p. 78).

In this form of modelling, global university rankings can be seen as functioning in a number of these constituent triangles: as a tool (to enable benchmarking), as a form of exchange (to demonstrate credentials to customers and stakeholders), and as a division of labour (as a form of delegation of the task of assessing value across national borders).

The perspective of activity systems gives a view of global university rankings, which locates their development in a wider historical and social context. This perspective overcomes the dualism of cognitive and sociological perspectives and looks instead at the interplay between different parts of the activity system and indeed between different, overlapping activity systems. This analytical framework draws attention to the interplay between a new form of infrastructure, such as rankings, and the network of actors and artefacts in which it resides. This form of modelling assists, also, in locating sources of tensions and (historically accumulating) contradictions, which make change possible, or perhaps, inevitable. In respect of rankings, the resultant focus is oriented toward the productive roles fulfilled by rankings and the constraints and tensions created by their mediating effects.

The growth in demand and production of global university rankings can be seen in the context of a new international higher education community in its early formation. Rankings can be seen as a new form of infrastructure, which shapes, and is shaped, by the wider activity system:

. . . the notion of infrastructure needs to be located in human practice . . . . We suggest infrastructure is embedded in different ways and serves different purposes in the constituent triangles of activity systems: production, exchange, distribution, consumption. (Engestrom, 2005, p. 369)

In studies of biological research, as examples of infrastructures, Bowker and Star (1999) show how infrastructure shapes practice:

The standardisation of genetic research on a few specially bred organisms (notably drosophila) has constrained the pacing of research and the ways questions may be framed, and it has given biological supply houses an important, invisible role in research horizons.(p. 36)

An activity systems perspective encourages a view of rankings as something relational, mediating between individual subject and object (as depicted in the uppermost triangle in Figure 1) and also between the individual and the wider community (as depicted in the two triangles at the bottom of Figure 1).

Mediation is an important concept drawn on by activity systems theory: Instead of assuming that individuals, acting alone, are the agents of actions, the appropriate designation of agent is “individual operating with mediational means” (Wertsch, 1995, p. 64).

In his contribution to the “third generation” of activity theory, Engestrom (2005) expands the theoretical framework to characterise interacting activity systems as a means of understanding “dialogue, multiple perspectives and networks of interacting activity systems” (2005, p. 63). The concept of “boundary crossing” is elaborated through the concept of a third space, or “contact zone” as a way of conceptualising the interaction between different activity systems to describe the important learning and development that takes place as different worlds meet and interact to form new meanings.

In this context an activity systems perspective generates fruitful research questions relating to the way the mediating artefact of rankings operates across different contexts and how new meanings are negotiated are constructed.

Context and Research Design

This study explores the rankings phenomenon from the perspective of 11 BC education managers representing a broad geographical spread of countries (see Appendix A). Their management remit includes services directed toward prospective students, higher education institutions and government education ministries. As such, these individuals can be seen as being located within multiple activity systems and they can offer a valuable perspective on the ways rankings are embedded in practices of student recruitment, HE partnership, and international co-operation.

The activity systems perspective informed the research focus and the types of questions used within the interviews (see Appendix B). In particular it encouraged focus on the forms of social activity in which rankings seemed to be embedded and the material and symbolic function of rankings. Questions were formulated to elicit the particular contexts in which rankings were encountered, accounts of rankings-related practices in which interviewees engaged and perceived strengths and limitations of rankings as mediating in different clients contexts. These accounts helped to construct, from the interviewees’ perspectives, the object of a particular activity, the particular ways in which rankings mediated activity; and whether these were predominantly in domain of production, exchange or division of labour (see Figure 1).

As a professional intermediary organisation, the British Council is underpinned by values of partnership and mutuality in international collaboration; the organisational remit is focused around responding to, collaborating in, and facilitating developments in international higher education.² One of the authors, a former member of the British Council had become aware, over the past 5 years or so, of the increasing relevance of these rankings for the range of customers and clients interacted with and the ways in which colleagues sought to find a space in which to operate around rankings in which to minimise the tensions created, and optimise the opportunities made available, by rankings. This position, as a former member of the organisation, granted a level of access and familiarity with the organisational context. Ethically, we sought to ensure respondents were clear about the objectives of the research and ensure their informed consent.

The process of identifying the group of interviewees was informed by secondary data analysis to seek to reflect a diverse range of operating environments. Organization for Economic Co-operation and Development (OECD; 2010) data helped to identify a mix of countries with (a) differing methods of financing tertiary education³; (b) differing levels of outward student mobility; (c) countries with increasing/decreasing market share⁴; (d) countries, which are identified as overperforming/underperforming relative to national economic capacity (Marginson & Van Der Wende, 2007); and countries who fare well and less well in global rankings. The British Council “Internationalisation index” provides an overview of dimensions including measures including degree of “openness” of policy environment⁵; student mobility indicators; transnational and research collaboration indicators (Economist Intelligence Unit, 2010).

The semistructured interview schedule focused initially on particular encounters with rankings in different geographic contexts. Specific case examples and critical incidents on where rankings had featured in working interactions were sought (see Appendix B). Questions were formulated to encourage respondents to express a grounded, contextualised account of their interactions with rankings rather than to express something more opinion-based or ideological (Rubin, 2005).

The interviews were conducted via telephone⁶ (these ranged from 30 to 45 min in duration) and recorded and transcribed the interviews vertabim. The analysis of interview transcripts followed three main phases. The first phase involved a close reading of the accounts to appreciate the different contexts in which rankings were encountered; how and through what activities rankings emerged as a focus of discussion and the dynamics of local context. In the second phase of reading the transcripts, attention was more closely related to the actual description of the specific incidents and examples, the ways in which respondents described their own practices and the language and discourse they used to do this. In the third phase of analysis concepts derived from the activity systems literature were used and tested on the data to see how they related to the contexts and practices described.

The focus of this study, and the analysis, is idiographic in its intent to draw out the range of practices from a collective group of organisational colleagues rather than trying to quantify intragroup differences and variation.

Analysis

Local Dynamics

Interviewees’ accounts demonstrated the considerable differences between geographical contexts, which varied from low visibility/relevance to extremely high visibility/relevance of rankings and with different trajectories of rankings dominance in play.

The U.S. context is described by respondents as having historically low levels of interest in global rankings due to its high placing in world rankings and relatively low levels of outward student mobility. Indicators of increasing interest were perceived and attributed to U.S. policies of encouraging outward student mobility and transnational education partnerships. The U.S.-based Chronicle for HE had, for the first time, displayed THES world university rankings alongside the domestic rankings.

In South East Asia, high levels out outward student mobility and increasing hub status for trans-national education (TNE) delivery have been strong drivers for interest in rankings. These have been observed at close range by BC staff⁷ but a more balanced view of rankings is now being perceived (in countries such as Malaysia), which is attributed to other competing domestic agendas and the growth in local and regional rankings instruments.

A rapid intensification of interest in many countries across Central and South Asia (CSA) region is attributed to emerging economies within the region, and consequently strong HE reform plans; and the effect of global university rankings (GUR) in providing a global performance indicator that creates an aspirational target:

. . . it is still very much in the mind of the minister for higher education [country in CSA] because he relates back to the days when universities in [country in CSA] were known throughout the world, not just in the CSA region, as having a very high reputation so it’s very much at the foremost of peoples’ minds even if it is a long way off which is something that is very interesting I think. (interviewee 7)

Some countries in the CSA region appear to be interested in global rankings from the point of view of strategy formulation (e.g., rankings are used for international benchmarking and target setting); and for others, in strategy implementation (e.g., rankings influence choice of partners); and for others, in evaluating the success of education reforms.

In the United Kingdom (all respondents had regular interaction with U.K. HE representatives) an intensification of rankings-oriented practices and strategies was perceived. These ranged from practices observed at overseas education exhibitions (finding niche selling points drawn from rankings; eg., “the highest ranked university in the West of London”); the recruitment of international directors with a strong remit to enhance university ranking; and the pursuit of broader based, high-level strategies focused around rankings:

. . . If you look at [institution x], I mean, that’s what they are aiming for is to be in that global league table. Way up there; where they want to be and they are using [the rankings]. So whether[rankings] are good, bad or indifferent they are driving strategy because they have to be global leaders if they want to get the research money, the recognition from governments that they are working with, from supranational agencies that they are working with. It is a short hand way of doing it by saying “well I’m in this position in the league table” I must be good. (Interviewee 5)

Contexts of Encounter

Respondents encountered rankings in a variety of contexts. Those in more student-facing roles had a longer standing awareness and engagement in rankings of up to 15 years in the case of two of the respondents:

Actually, I think I became aware of rankings of universities as far back as 15 years ago . . . I’m not even sure it was the Times Higher, I think it predates the Times; possibly it was the Guardian; when they started to bring out the Guides. At the time that was a real problem for the British Council in trying to be impartial but it was quite difficult because there were all these people overseas who used these guides and were using them as a proxy for excellence. (interviewee 6)

In the cases of student-orientated activities all respondents working in this sphere gave examples of how their practices relating to rankings had become more “codified.” Explanations of different ranking systems, the ways in which they were constituted, “health warnings” and advice on other sources of information to support decision making were widely published on websites, corporate publications and in standardised briefings.

In university-facing activities (e.g., facilitating transnational education collaboration and international partnerships) and for those who worked with overseas education ministries there had been a more recent intensification of interest and interaction with rankings:

I’ve always been aware of rankings since I started working with universities but I have to say that it’s come up in discussions I’ve had with universities in South Asia and governments in South Asia in just about every meeting I go to now . . . I think that started about two or maybe three years ago. (interviewee 7)

For all respondents, the Times Higher World University Rankings were the most commonly referenced by clients and customers and by staff themselves. The role of regional rankings was identified in certain cases but was perceived to be of a secondary nature.

How Rankings are Used

For the majority of respondents rankings would not constitute a natural starting point in their own professional practice. Yet due to the range of contexts in which clients/customers used rankings they were called on to interact with global rankings in a variety of ways.

In local working contexts rankings, in many cases, represented the means by which an interaction with customers and clients was structured; a common reference point. In a sense, it constituted a new form of discourse in which they had to engage

Well you’d be trying to . . .; you know, you’d be talking about institutions or the potential for partnership with institutions and literally if they weren’t in the list they’ve got on their desk they wouldn’t be interested in talking about it. (interviewee 7)

Many of the accounts point to the symbolic role of rankings in representing something of value and representing a form of “boundary object” between different communities:

We can use it as a prop to any particular argument that needs to be made . . . using it as a piece of leverage to raise expectation for partnership. The world rankings help you show them that outside of the 3 names that have brand recognition in U.S. there is a better partnership fit. We very much feel there is a role here in the British Council in the U.S. to provide advice, where it is appropriate—to ensure that successful partnerships and relationships are achieved. So where the world rankings help us—identify a better fit—then we definitely lean on these at that point. (Interviewee 2)

Other respondents talked of rankings in similar terms: “It becomes a brand” (interviewee 4); a springboard in a conversation (interviewee 2); “it bolsters a message” (interviewee 5). However, it was emphasised that there were other ways recognition could be achieved:

. . . these are the reasons [U.K. university] has a brand name in the U.S.; not ranking and they have very strong partnerships. So a lot of times in the U.S. rankings do not matter as much because the way in which brand recognition is earned is through study abroad partnerships and then you can see some institutions will be very, very strong and you think “how did they earn this.” It’s through hard work, relationship building and providing a quality student experience and not through the rankings by themselves. (interviewee 2)

To a lesser extent, some respondents (interviewees 2, 6, & 8) talked of the intrinsic value of global rankings; in one sense for their utility as a market intelligence tool:

. . . so in terms of market intelligence we have been saying for a long time that institutions want to know about the quality of overseas institutions . . . and the rankings, particularly the Times Higher rankings are very good in that it gives you access to a lot of information very easily; which actually you can break down a little bit. (interviewee 6)

Three respondents (interviewees 4, 6, & 7) perceived rankings as having a transformative role in encouraging countries to establish their own domestic rankings systems and, therefore, to make national HE data more transparent and accessible.

and I think what I find interesting is that you are starting to see to see a lot more governments developing rankings, or even publishers developing rankings of other countries . . . so it allows you to access information about other countries . . . so I think India is a good example and Pakistan where publishers have been ranking but also the government have done their own ranking system as well (interviewee 6)

Through the interview accounts, different conceptualisations of global rankings are articulated, which relate to the functionality and symbolic effectiveness of the artefact and also their role in creating a new division of labour (see Figure 2)

Figure 2.

Global rankings as form of instrastructure.

In one type of account from respondents, rankings are described in terms of their functionality as a tool; as a source of market intelligence and of saying something valid about organisations. Within this account, rankings provide institutions with a better means of benchmarking; and achieving a better partnership fit and, in this sense, can be located in the mediating artefact/tool dimension of the activity system.

In another type of account evident in the interviews, rankings support a discourse of “positioning” and agenda building. The utility of rankings in this construction is its value in gaining a “foothold” in a conversation and, in this way, can be located in the domain of “exchange” and representing symbolic value.

In another construction, there is utility in global rankings as being a source of information residing outside the organisation, which enables the respondent to treat it as any other organisation would and enable a side stepping of the issue when being asked to identify the “best institutions.” Here it operates as a mediating object and is constituted as a new division of labour.

Alternatively, rankings are positioned as an artefact that has transformative effects and a role in encouraging other countries to make domestic data on HEI performance more widely available.

Mediating the Use of Rankings

Only three respondents described using rankings in a proactive way to position United Kingdom effectively (e.g., in speech writing, press releases). The majority of practices were more situated and shaped by context.

Across all contexts (students, governments, and institutions) respondents drew attention, without exception, to their interpretive role in informing customers and clients about how different rankings systems are constituted and of demonstrating alternative information sources and related “health warnings.” Some respondents harnessed the market intelligence potential of the tool and used secondary analysis to provide information to support partnerships mediation and effective market intelligence. These accounts reflected mediating practices of staff where rankings were being constructed through interactions with clients as a form of tool.

Respondents seemed to reflect a range of ways of interacting with rankings. At one end of the spectrum, respondent narratives foreground activities and practices aimed at reducing the symbolic value of rankings (interviewees 5, 8, 9, & 11) through briefings, presentations and publications with the purpose of explaining how rankings are constituted; and advising of alternative information sources.

In other cases, (respondents 2, 3, 4, & 7) describe a more relational approach, which is influenced by context characteristics:

I think the way in which we read our rankings depends on the particular audience. So while the demand is sometimes there in the conversation we can push it to the fore or the back depending on where we want it to remain relevant and when the U.K. has low branding recognition for the U.S. for partnerships or with prospective students. And so for institutions that have a good ranking but low recognition in the U.S., it is a springboard in a conversation. (interviewee 2)

In the U.K. if I am talking to top institutions I am talking about how they can use their rankings. If I am talking to middle rankings institutions I am saying don’t worry about it too much. If I am talking to low ranking ones I am saying “you are going to have to look for a niche” or “this market isn’t really for you.” So the way I might talk about them is going to depend on the actual audience. (interviewee 4)

Most respondents took a contingent approach to drawing on rankings to buttress discussions. This form of practice seemed to relate to contexts where rankings were functioning for education clients as a reputation carrier and determining which universities they might engage with. To be able to operate effectively in such contexts, British Council staff maintained a strong commitment to being well informed, being conversant and capable of conforming to a rankings-based discourse. However, within this activity domain of “exchange,” several practices were described by respondents, which related to a role in mediating what the rankings signify. This included practices such as bringing in an alternative narrative, fostering debate on the fitness for purpose of rankings in different contexts.

Respondents described a set of practices concerned with facilitating debate between different consumer groups on the utility of rankings. For example, the British Council team in Taiwan had convened a panel discussion on the value of rankings in facilitating TNE partnerships at a recent education exhibition. Three other respondents had contributed to the inclusion of an academic session exploring rankings as a western construct at an international education conference organised by the British Council “Going Global.”

In these instances, it appears that respondents are engaged in challenging the status quo and seeking, through the engagement with partners, a reconceptualization of how rankings are used.

To a lesser extent, the role of rankings as constituting a new division of labour is evident in the accounts. Two respondents make reference to the “cooptation” of some of the brokerage roles around facilitating effective partnerships; in terms of managing expectations of institutions with high aspirations.

As such, the most frequently cited practices are clustered around the “tool” and “rule” dimensions of the activity system (as highlighted by “A” and “B” on Figure 2 above). However, practices located in the domain of “production” appear to be oriented to tool amplification; drawing attention to how the rankings are comprised and different ways they can be read. By contrast, those practices located in the domain of “exchange” are more orientated to reducing the symbolic value of rankings.

The activity systems perspective encourages an appreciative form of enquiry, which highlights both the productive roles fulfilled by rankings as we as some of the constraints they impose for staff in these contexts. The explicit modelling of the activity system within the triangular formation provides an “analysable inner structure” to these elements and assists in locating sources of tensions and contradictions and the related practices of staff in seeking to reduce tensions and reconcile contradictions.

Interacting Activity Systems

Many instances were recounted where respondents must use ranking as a mode of interaction to negotiate boundaries, or conform to a required discourse, with a customer or client.

When I am talking to HE departments or government ministries here, rankings do come up as a broad agenda in just about every meeting so I need to be quite well informed about what global rankings there are out there and which ones are going to be relevant to the particular context that I am in (interviewee 7).

Also when you are looking at scholarship programmes in the Middle East; again they won’t entertain institutions who are not in HE league tables; in certain subjects. (interviewee 11)

These accounts portray the different ways in which rankings interact with the professional brokerage performed by this group of staff. In several country contexts, as described by a regional adviser for Central and South Asia, global rankings operate as a benchmarking tool used by countries to gauge the effectiveness of their education reforms. The basis of the discussion with the British Council is to gain professional “on the ground” advice, in helping to identify appropriate strategies for HE capacity building. For the British Council adviser this often entails trying to unpick the particular development objectives that lie behind the rankings aspiration to identify the areas of potential for bilateral collaboration.

In a different way, several respondents (3, 4 and 11) described the role rankings played in helping to manage expectations among universities in different countries in terms of determining their suitability for collaboration and identifying the right sort of institution to partner with. The British Council is often involved in facilitation of these partnerships, often through government funded schemes and has a delicate brokerage role to play in helping manage the expectations and aspirations of prospective participants. Global rankings provide an external source of information that can help to side step some of the judgements sought by the organisation that would jeopardise the organisation’s impartiality.

In summary, the data suggested a contingent approach to the use of rankings in the context of representing U.K. HEIs as a whole. Staff interviewed would seek to draw on other narratives of value (such as other things the institution was famous for, other aspects of its brand, or the types of work it had been doing in a particular country) if the rank of a particular institution was relatively low. At the same time, interviewees felt rankings performed a useful function sometimes in encouraging U.K. HEIs to self-select; in the sense that if BC was advertising an inward visit to a country; or partnership opportunity with a university of certain status, then rankings encouraged U.K. HEIs to be more self-aware about the attractions of meeting or partnering with them.

In these ways, the role and function of rankings can be located and interpreted differently from the vantage points of a higher education institution and education ministries and for BC education staff (as depicted by the separate objects “1” and “2” of their activity systems (see Figures 3a and 3b below). However, through the sorts of interactions being described in the interviews, the role and function of rankings is reconceptualised and re-constructed (as object “3”).

Figure 3a.

Contact zone for British Council and Higher Education Institution.

Figure 3b.

Contact zone for British Council and education ministry.

In the case depicted in Figure 3a, the object of activity is reconstructed as a tool to gauge the level of international commitment and one where the education manager (interviewee 10) has probed further to understand the type of research and research culture of the institution seeking partnership to assist in partnership facilitation in a more nuanced way.

In the case depicted in Figure 3b, the object is reconstructed as a tool to gauge level of international aspiration and which enable agenda matching and targeted areas of bilateral collaboration that serve national interests:

So we are looking long term; so it is really translating what it is they see as important in the world rankings into “on the ground” reform policy that the British Council and the U.K. can contribute to . . . in areas that are important for that country’s economic and development priorities; and for Bangladesh climate change is a very important one that we work with and so sort of really focussing on strategic areas so that when they come to world rankings they will be getting/building a reputation right across the board but to have niches that are really important to their country . . . . (interviewee 7)

Perceived Limitations in Rankings

In the analysis we were interested in the ways this group of respondents talked of the limitations they perceived in global rankings. This served to illuminate the broader set of motives within international higher education community of competition, coordination and collaboration; of which rankings only partially fulfilled.

By expanding the activity systems model to reflect a broader subject/object relationship, Figure 4 depicts a broader activity system in which the IHE community has, collectively, created the demand for rankings and depicts the broader range of needs and motives they fulfil. The projected outcome is “no longer momentary and situational. Rather, the projected outcome consists of societally important new objectified meanings and relatively lasting patterns of interaction” (Engestrom, 2005, p. 30).

Figure 4.

Representation of GURs in an enlarged activity system.

Several tensions and contradictions relating to rankings were perceived by respondents and highlighted the partiality of this artefact in supporting surrounding systems of activity.

For the regional education adviser in South East Asia region, a weakening in power and influence of rankings was perceived and attributed to the recent proliferation in global rankings systems, leading to reduced impact and reduction of their symbolic power. This can be seen as tension in terms of fragmentation in division of labour.

Respondent 5 drew attention to a ranking system originating in France, which incorporated “employability” as a more central measure. This demonstrates the partial nature of the value and role of rankings for their strong emphasis on research performance over other measures.

Although rankings are perceived to play a significant role, currently, in mediating international partnerships, limitations were perceived in terms of their static focus, as being a measure of past performance (rather than indicating current levels of research investment); indicating a further shortcoming in the value of this tool. Furthermore, limitations in effectiveness were perceived in the fact that rankings typically identify only the top 200 or top 500 institutions globally, which means that it is not comprehensive enough to support bilateral and multilateral partnership support:

so if you look at somewhere like India, you’ve got 1000 institutions, you can’t get any info about Indian institutions in the Times rankings info . . .but actually there are some great institutions in India and you need some sort of help to understand the types of institutions and how good they are. (interviewee 6)

Currently, global rankings are perceived as a tool more closely aligned for facilitating institutional to institution partnerships. Due to the institutional unit of analysis, it was regarded as blunt tool for supporting departmental/discipline based partnerships and student decision making.

Conclusions

Global university rankings have emerged as a powerful form of infrastructure in the last decade to meet an apparently strong need for cross border benchmarking and for tools which facilitate competition, co-ordination and collaboration across the international higher education community.

The theoretical perspective of activity systems encourages a focus on the interplay between contexts and use of rankings and in how different parties identify appropriate forms of agency in responding to rankings, in optimising situations and/or reconciling effects. The analysis within this study has helped to illustrate these dynamics more effectively, providing further insight into the range of meanings and functions attached to rankings and the trajectory of their use within interactions.

In this small research sample, it is evident that the infrastructure of rankings is embedded in international partnership activity in complex ways. The practitioners represented in this study showed adept strategies of remediating the way rankings are interpreted and used by education clients and partners.

The ways in which rankings mediate interactions and practices shows a form of infrastructure that is developing and changing and which is not stable; and one, which only partially fits the needs of the surrounding activity system.

This study has shown how readily rankings are used within the discourse of policy makers and in interactions, which underpin many transnational educational partnerships and how they influence national policy development. The analysis demonstrates the ways rankings serve, for some governments, as an aspirational goal, for others a measure of effectiveness of reform or, alternatively, as a statement of credentials and international standing. The analysis points to the likelihood that these “use values” will evolve.

The contexts described in this study show how rankings are serving in many cases as a “shorthand benchmark” or “statement of aspiration” that obscures the more complex underpinning goals and aspirations. The interviews demonstrate also the potential for alternative narratives of value.

A common refrain voiced across the policy, research and media communities is that rankings are “here to stay”; but in what form and serving what purpose and meaning? The fluid range of roles and meanings attached to rankings should be recognised at a time when we are seeing a more concerted range of intergovernmental policy responses to rankings.⁸ It is important to avoid the danger of further elevating and entrenching the role and significance of rankings in a context where other tools may emerge as being more useful.

Respondents’ perceptions of partiality and limitations of rankings give further illustration of how rankings are being conceptualised and help define possible trajectories for development of rankings development and/or alternative instruments.

For practitioners, the findings within this study give an optimistic view of the ways in which conceptualisations of rankings can be renegotiated and reshaped to serve specific needs more effectively; and that space can still be found for an alternative discourse of value.

This brief attempt at modelling activity systems in this IHE context is inevitably partial but hopefully will encourage further iterations and encourage a broader dialogue on how best to facilitate international co-ordination and collaboration.

Footnotes

Appendix A

Appendix B

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

Catherine O’Connell is a current student on the doctoral programme in Educational Research at Lancaster University. Her career background includes roles as institutional-wide curriculum innovation facilitator and manager (at the University of Manchester) and professional development manager for the international higher education team at the British Council.

Murray Saunders holds a chair in the evaluation of education in the Department of Educational Research at Lancaster University, UK. He has directed some 40 evaluations over 30 years. He has been president of the European Evaluation Society and the UK Evaluation Society and was a founder member of the International Organisation for Co-operation in Evaluation. His interests are in the development of theory and practice of evaluation and the evaluative research of policy and change.

References

Ackers

(2008). Internationalisation, mobility and metrics. A new form of indirect discrimination. Minerva, 46, 4411-4435.

Altbach

(2006). The dilemmas of ranking. International Higher Education, 42, 2-3.

Beer

(2011, June 2). Let variety flourish. Times Higher Education Supplement. Retrieved from http://www.timeshighereducation.co.uk/story.asp?storyCode=416378&sectioncode=26

Bowker

G. C.

Star

S. L.

(1999). Sorting things out: Classification and its consequences. Cambridge: Massachusetts Institute of Technology Press.

Deetz

(1996). Describing difference in approaches to organisation science: Rethinking Burrell and Morgan and their Legacy. Organization Science, 7, 191-207.

Dill

Soo

(2005). Academic quality, league tables, and public policy: A cross-national analysis of university rankings. Higher Education, 49, 495-533.

Economist Intelligence Unit. (2010). Measuring and benchmarking the internationalisation of education. Retrieved from http://www.britishcouncil.org/going_global_4_-_shaping_the_next_generation_-_j_ncnamara_a_williamson_-_pp.pdf

Engestrom

(1987). Learning by expanding: An activity theoretical approach to developmental research. Helsinki, Finland: Orienta-Konsultit.

Engestrom

(2005). On the materiality of social capital: An activity theoretical explanation. In Engestrom

(Ed.), Developmental work research. Expanding activity theory in practice. (Vol. 12, pp. 367-380). Berlin, Germany: Lehmanns Media.

10.

Haffman

(2010). Is inequality among universities increasing? Gini coefficients and the elusive rise of the elite universities. Minerva: A Review of Science, Learning and Policy, 48, 55-72.

11.

Hazelkorn

(2008). Learning to live with league tables and ranking: The experience of institutional leaders. Higher Education Policy, 21, 192-215.

12.

Hazelkorn

(2009). Rankings and the battle for world-class excellence: Institutional strategies and policy choices. Higher Education Management and Policy, 21(1), 55-76.

13.

Hazelkorn

(2011). Rankings and the reshaping of higher education. The battle for world class excellence. London, UK: Palgrave MacMillan.

14.

Ishikawa

(2009). University rankings, global models and emerging hegemony. International Journal of Studies in International Education, 13, 159-173.

15.

W. Y.

(2011). Soft power, university rankings and knowledge production: Distinctions between hegemony and self-determination in higher education. Comparative Education, 47, 209-222.

16.

Locke

W. V.

(2008). Counting what is measured or measuring what counts? League tables and their impact on HE institutions in England. Retrieved from http://www.hefce.ac.uk/pubs/hefce/2008/08_14/08_14.pdf

17.

Marginson

(2009). Open source knowledge and university rankings. Thesis Eleven, 96, 9-39.

18.

Marginson

Van der Wende

(2007). To rank or to be ranked: The impact of global rankings in higher education. Journal of Studies in International Education, 11, 306-329.

19.

Marshall

(2010, March). League tables: Flawed rankings or key benchmarks.Retrieved from http://www.britishcouncil.org/goingglobal-gg4-turning-the-tables.htm

20.

Organization for Economic Co-operation and Development. (2010). Education at a Glance 2010.Retrieved from http://www.oecd.org/document/52/0,3746,en_2649_39263238_45897844_1_1_1_1,00.html

21.

Pascarella

(2001). Identifying excellence in undergraduate education: Are we even close? Change, 33(3),18-23.

22.

Rubin

H. R.

(2005). Qualitative interviewing. The art of hearing data. London, UK: Sage.

23.

Salmi

Saroyan

(2007). League tables as policy instruments: Uses and misuses. Higher Education Management and Policy, 19(2), 31-68.

24.

Saunders

M. T.

Trowler

Bamber

(2010). Reconceptualising evaluation in higher education: The practice turn. London, UK: Open University Press.

25.

Usher

Savino

(2006). A world of difference: A global survey of university league tables. (Canadian Education Report Series). Toronto, Ontario, Canada: Education Policy Institute.

26.

Wertsch

(1995). The need for action in sociocultural research. In Wertsch

J. D.

(Ed.), Sociocultural studies of mind (pp. 1-36). Cambridge, MA: Harvard University Press.

27.

Wildavsky

(2010). The Great Brain race: How global universities are reshaping the world. Princeton, NJ: Princeton University Press.