Revisiting Global University Rankings and Their Indicators in the Age of Sustainable Development

Introduction

Despite their recent history, global university rankings (GURs) have gained an important place in the scenery of higher education. ¹ The first global ranking, Academic Ranking of World Universities (ARWU), was published in 2003 by Shanghai Jiao Tong University in China. Soon after, others, including Times Higher Education (THE) and QS World University Rankings, joined in to create a global “reputation race” ² among universities. These two rankings form the basis for some well-known publications and their annual rankings, for example, U.S. News and World Report.

The indicators and methodologies used by these highly-solicited GURs have altered the social construction of universities. In effect, higher education policies, governance strategies, and institutional practices have been reconstructed in an international competition in which aggressive marketing and image-making precede transformative internationalization, effective pedagogy, and social mobility.

The relevance and impact of GURs have received much criticism in the academic world as they systematically fail to reflect quality of education. Yet, these systems have ceased to exist with little or no change in their indicators and methodologies, mainly as facilitators for service providers and consumers in the higher education market.

Following the adoption of the 2030 Agenda, doubts about the impact and relevance of global university rankings is even more pertinent. The question is whether these GURs are compatible with the international aspirations toward sustainable and just societies.

This article attempts to answer this question and discusses the necessity of revamping global university ranking indicators in line with the ambitions of sustainable development. To this end, it first depicts the sustainable development goals (SDGs) and the 2030 Agenda. It then delineates the irreconcilable nature of a sample of indicators used by university ranking systems with the principles of equity, education quality, gender equality, and social justice in the SDGs, and finally suggests some remedial adjustments.

The 2030 Agenda: A Call for Renewed Global Partnerships and Instruments

Dissociated from business-as-usual methods that have proved inefficient in the past, the 2030 Agenda seeks to “transform our world” ³ by taking bold and transformative steps in social, economic, and environmental development to ensure just and sustainable societies. It is a global contract that requires revisiting and transforming our worldviews and discourse, as well as tools and practices in line with the SDGs. After all, it is only logical that our global mechanism, including GURs, would shift to avoid creating parallel or contradictory systems that could undermine efforts toward the implementation of the SDGs. ⁴

Many actors, in both public and private sectors, have embraced the SDGs by developing new instruments or partnerships. Higher education institutions are no exception. More than 300 universities have partnered with United Nations departments and agencies to create a network called the Higher Education Sustainability Initiative (HESI). ⁵ Their aim is to promote sustainable development through teaching, research, green campus initiatives, information sharing, and networking in higher education systems around the world. In a similar vein, the European Network of Higher Education for Sustainable Development and the Copernicus Alliance ⁶ including its three-year professional development project called University Educators for Sustainable Development (UE4SD) ⁷ have aimed to transform learning, teaching, and research, and embed sustainable development in institutional practices and policies in Europe. The 2018 sustainable campus best practices report ⁸ by the International Sustainable Campus Network (ISCN) and Gulf universities is another way to showcase and promote sustainable development in higher education.

Against this background, the relevance ⁹ and compatibility of current GUR indicators with the 2030 Agenda is disputable. The GURs represent the business-as-usual perspectives that the world decided to do away in 2015 due to their failure to ensure equity, equality, and social justice for all.

The following discussion revolves around two sample indicators and a few general concerns regarding the three most influential GURs: THE, QS, and ARWU.

Why Are Global University Ranking Indicators Incompatible with the SDGs?

Although Agenda 2030 was designed to reverse social injustice and inequality, the GURs indicators exacerbate the social stratifications that already exist in our societies. As one example, the indicator on “research production and citation” is a matter of concern. This indicator is used by all three ranking systems (ARWU, THE, and QS), albeit with different weights. This indicator counters SDG 4 (quality education), SDG 5 (gender equality), and SDG 10 (reducing inequalities).

The irrelevance of evaluation systems, including external quality assurance and rankings, to the actual quality of teaching and learning has already been discussed in several studies. ¹⁰ In addition, by instilling a culture of publish or perish, this indicator has a negative impact on the quality of research. According to Richard et al. (2015), ¹¹ this indicator carries a punishment incentive and has already led to increasing occurrences of questionable research practices, including: plagiarism, self-plagiarism, data fabrication, academics' conformity to avoid conflict of interest, unethical authorships, fake reviewers, and other forms of violations that undermine research credibility, integrity, and validity. ¹² In addition, as publish or perish lies at the root of academic advancement, the pressure to publish can “harm the quality of life of researchers, decreasing their satisfaction and undermining creativity.” ¹³ It can be concluded that this indicator conflicts with SDG 4.

In addition, by using quantity of publications to rank universities, GURs have actually moved from simple benchmarking instruments to policy-making ones ¹⁴ by promoting excellence initiatives that funnel national and international funding to top-ranking research-intensive universities. An example is the Russell Group in the UK, which includes 24 elite universities that receive up to half of the funding of the whole higher education sector; their share of funding increased from 44.7 percent in 2001–2002 to 49.1 percent in 2013. ¹⁵ As governments allocate more funding to these universities, those with lower ranks are deprived of much of the necessary resources that can permit widening participation, equity in access, and diversity in higher education. As Altbach and Hazelkorn also affirm, the bibliometric data used by rankings not only devalue arts, social sciences, and humanities, where more female students end up, but also disparage “research with a regional or national orientation—especially research published in languages other than English.” ¹⁰ As such, the publication indicator also negates SDG 5 and SDG 10.

Another issue is related to the international student indicator used by ranking systems. Some of the immediate questions that come to mind include: What the socioeconomic backgrounds of these students are and if gender parity is respected in this student population?

Despite some recent efforts to increase diversity, ¹⁶ in general, students from higher socioeconomic status ¹⁷ can afford to pay expensive university fees and are more likely to pass the highly selective admission processes as they graduate with good marks from elite high schools, something that children with lower socioeconomic status cannot afford. Therefore, “number of international students” as an indicator, is simply irrelevant to social mobility and “inclusive internationalisation.” ¹⁸ In fact, underneath this apparently neutral indicator lies a bias that favors those healthy and wealthy enough to afford international mobility and its costs. A 2016 study by University UK, ¹⁹ for instance, shows that students with upper managerial jobs during their studies were almost five times more likely to take part in mobility than students with long-term unemployment. Similarly, from the 13,335 sample of this study, “5.8% of white students were mobile, compared to 2.9% of black students and 3.3% of Asian students,” (p. 11) and the least mobile were the black males with only 2.2 percent. ¹⁹

The international student indicator is used by both THE and the QS rankings, which validates existing social and gender stratifications; it also hampers the realization of SDG 6 (creating decent employment for all) because it continues promoting a market-based perception of who can access the best universities and consequently better jobs based on their socioeconomic backgrounds and not their personal merits. It also contravenes SDG 16, which pursues peace and justice in democratic societies where everyone can equally, in terms of their knowledge and rights, participate in building transparent institutions.

Another perplexing issue is that GURs indicators fall short of accentuating gender equality, (SDG 5). Gender equality is clearly included in the German national ranking system (CHE), which also includes indicators on quality of academic and pastoral care and quality of students' lives. The CHE is indeed “both an expression of the new thinking as well as a contributor to the change” (p.4). ²⁰ Main global rankings, including THE, QS, and ARWU, on the other hand, completely neglect gender balance in national and international student and staff populations. For instance, they do not measure whether and to what extent women have received higher level degrees in STEM fields; what number of men have received degrees in arts and humanities; or what percentage of women work in professorial and upper management roles in universities. The ARWU indicator for the number of Nobel Prize winners and Fields Medals awarded, for example, clearly negates SDG 5 because it indirectly promotes gender inequality as there are fewer women researchers in STEM fields. ²¹

Furthermore, since social justice is a pillar of the 2030 Agenda, GURs have no indicator to evaluate social inclusion and therefore fail to maintain a real connection with equality and participative equity. As such, they undermine sustainable development, in particular SDG 10 and SDG 5, and also weaken regional policy frameworks, for example the 2015²² Yerevan communiqué to promote gender equality and the 2015 European Ministers' declaration on non-discrimination and fostering educational opportunities for disadvantaged young people, particularly refugees and migrants. ²³

Last, but not least, GURs remain detached from the realities of climate change and the efforts of countries and universities ²⁴ toward environmental sustainability. They have no indicator to evaluate and hence validate university practices in line with SDG 6: avoiding wasting water; SDG 7: using affordable and clean energy; SDG 9: innovative ways to repurpose old materials; SDG 12: recycling plastic, paper, aluminium, and glass; SDG 13: educating students on climate action; SDG 14: avoiding the use of plastic to safeguard life under water; SDG 15: planting trees and creating green spaces; and SDG 17: creating partnerships for sustainable development. They also fall short in drawing on the data obtained by, for example, the U.S.-based organization, the Association for the Advancement of Sustainability in Higher Education (AASHE).

The irrelevance, or better said, the harmful impact of rankings to sustainable development is best manifested in the university car parking spaces rankings that has emerged more recently. It is of course important to facilitate access to campus for academics and students but such a ranking demonstrates no inclination to reducing carbon footprints and fails to promote accessibility for all by public transport.

Toward Equity, Inclusion, and Diversity in Higher Education: University Ranking Systems as Game Changers

The question is whether GURs have the capacity to serve as powerful external push factors and help transform higher education discourses, policies, and practices toward the implementation of Agenda 2030. In their current format it seems unlikely. At present, they remain the legitimate and borderless means of elite reproduction, and therefore stand in opposition to the SDGs. But this can be changed.

However, university rankings possess the potential capacity to reorganize HEIs' policies and practices toward sustainable development. They can revisit their indicators and methodologies to ensur diversity, inclusion, participative equity, equality, and quality of teaching/learning. Unfortunately, we live in a world where inequality gaps are widening in an unprecedented way. ²⁵ The wealth of a mere 42 people is equivalent to the total wealth of 3.7 billion of the poorest people, ²⁶ hence the need for our global, regional, and national mechanisms to address inequalities in all their forms. To this end, the afore-mentioned GURs shall revisit and adapt their indicators to facilitate change in line with the SDGs.

Some of the sample indicators this article proposes include, but are not limited to:

gender parity in research and at higher management levels in universities

Conclusion

In 2018, the Times Higher Education (THE) announced their intention to promote SDGs by creating a new ranking focused on universities' impact on society. ²⁷ Their 2019 impact ranking ¹¹ was released in April and universities from New Zealand, Canada, the United Kingdom, Sweden, Italy, and Hong Kong rank as the top 10. This is, in itself, a positive move and demonstrates that ranking institutions are recognizing that HEIs as well as the broad public want to tackle sustainable development.

However, to truly address the implementation of 2030 Agenda, the SDG-related indicators must be merged with the main annual ranking of universities. Creating a separate ranking for SDGs in higher education—although a genuine initiative—is only another ranking added to the already proliferated and overlapping market of rankings and will therefore fail to effectively promote the SDGs. To genuinely facilitate the implementation of the SDGs, THE and their counterparts shall merge the new social impact ranking indicators with a revised selection of their annual world university ranking indicators. Such a ranking can finally claim to be about education, universities, and its people, and not further metrication, and hence would have the potential to advance the SDGs.

Against such revised rankings, it would be interesting to see whether the top-ranking HEIs can hold on to their almost permanent places, as they usually have in the annual world rankings. For instance, how would MIT's rank change against an indicator that measures the number of female graduates in STEM and engineering fields, or the number of female professors, or maternity leaves taken? In a similar vein, how would Oxbridge, Stanford, and Princeton rank if an indicator evaluated the number of their students from lower socioeconomic backgrounds or from among refugees and migrants; and would Harvard rank high as an inclusive university demonstrating the impact of its inspiring and bold “inclusive excellence” project? ²⁸

In general, in their current shape, global university rankings are foreign to quality of teaching, learning, and research. This is a view shared among many, if not all, involved in teaching or research in higher education systems around the world. These rankings negate the Berlin Principles on Ranking of Higher Education Institutions ²⁹ as they fail to provide a comprehensive and relevant appreciation of HEIs around the world. They also conflict with principles of equity, equality, diversity, and social mobility that are at the heart of Agenda 2030. The time is ripe for GURs to transform and respond to the current context of the world.