Abstract
This article provides a 30-year history of the development, use and impact of performance measures on tertiary education institutions in New Zealand. The study reveals that performance measurement emerged from the new public management initiatives and the multiple logics of government reforms to help address efficiency and accountability concerns of the tertiary education sector. The performance measurement culture became central to shaping and reshaping the character of educational politics, government policies and the management of educational outcomes. Performance measurement also created a web of unintended consequences with its own tensions, cynicism and complications. Enacted by the market logic and complemented by the business logic to maximise profits, performance measures compromised academic quality and caused rivalry with the norms of the academic profession. The study recommends that the distorting effects of performance measurement requires profound rethinking and careful management to ensure that it accomplishes what it is intended to accomplish.
Keywords
Introduction
Since the 1980s, performance measurement has emerged as a major policy innovation within the larger neoliberalist orientation of educational institutions in many Western countries. Not only has its development signalled ‘a fundamental change in the political ideology’ of the management and accountability of educational institutions, performance measurement has come to play an important role in shaping government policies and managing educational outcomes (Peters, 1992: 268). This article adopts a historical perspective to examine the development, use and impact of performance measures in public tertiary education institutions (TEIs) in New Zealand. A historical analysis provides a richer and wider understanding of events and issues, how they evolved over time, their causes and their relative importance (Porra et al., 2014). Within this overall context, the article also explores how performance measurement became associated with the multiple logics of government reforms over a period of nearly three decades to help drive changes within public TEIs.
A number of studies have been critical of new public management (NPM) reforms in higher education (e.g. Olssen and Peters, 2005; Parker, 2013; Parker and Guthrie, 2010; Roberts, 2013). However, the effects of performance measurement as a specific accounting change driven by NPM, through time and longer-term value shifts to transform TEIs, remain largely unexplored (Parker, 2011). This study contributes towards a deeper and richer understanding of performance measurement as a social, political and technical practice influencing changes in educational institutions through different periods of policy reforms. The study provides a well-documented accounting history literature on how successive New Zealand governments have used performance measurement as a political process to reform tertiary education and help improve accountability. The article also contributes to the performance management literature by revealing the use of performance measures as a technical and social process in TEIs. As a technical process, performance measurement uses quantitative measures to improve financial and educational performance. Performance measurement, as a social process, has helped achieve social and equity goals of increased participation in tertiary education.
This study is framed within the neoliberal political and economic context and associated discourses of NPM to help understand the development and use of performance measures within these specific settings. NPM introduced the notion of accountability and accountable management in the public sector (Hood, 1991; Humphrey et al., 1993). In the education sector, the global importance of NPM is widely recognised from the significant impact it has made in transforming educational systems worldwide (Olssen and Peters, 2005). The study also draws literature from the new or neo-institutional theory (NIT) to build an explanation of the influence and impact of performance measurement on public TEIs. NIT posits that accountability is enforced on organisations by external institutional factors, and organisations will succumb to the institutional pressures to attain legitimacy (Scott, 2001). NIT is useful in understanding the relationship between external factors and organisational practices such as performance measurement and accountability systems. Recent developments in NIT suggest that multiple institutional logics exist in organisations that will accommodate or adjust practices and norms associated with different logics (Thornton and Ocasio, 1999). In the accounting literature, studies have shown that a complex interplay of multiple, coexisting and conflicting institutional logics can influence accounting change and practice variation (e.g. (Chenhall et al., 2013; Ezzamel et al., 2012; Rautiainen and Jarvenpaa, 2012).
A historical research method was selected for the study to meet the research aims. In addition to archival documents, interview data were gathered from two public TEIs in New Zealand, the Ministry of Education and the Tertiary Education Commission (TEC). New Zealand was recognised as leading the revolutionary introduction of neoliberalism in 1984 (Kelsey, 1995) and the reform of educational administration was the most radical of any undertaken in an industrialised nation (Fiske and Ladd, 2000). Hence, the neoliberal context and political agendas of successive governments offer unique opportunities for a historical study into the development and use of performance measures in public TEIs.
The remainder of the article is organised as follows. The next section provides a review of the research literature and theoretical perspectives that inform the study. This is followed by an explanation of the research method and a presentation of the historical development of performance measures, its use and impact on TEIs. The final two sections present a discussion of the findings and some concluding comments.
Research literature
The research literature encompasses discourses on NPM and NIT to provide an integrated framework to help examine the development of performance measurement practices. Both the NPM and NIT literature have similarities in advocating the view that performance measurement systems in higher education institutions are shaped by institutional environmental pressures (e.g. Broadbent and Laughlin, 2009; Gleeson and Husbands, 2003; Olssen and Peters, 2005). The education sector is noted for its highly institutionalised character comprising policies, programmes, structures and systems that are enforced by institutional pressures such as public opinion, government legislation and the views of societal stakeholders (Bessant et al., 2015; Olssen and Peters, 2005). NPM, which emerged as a response to institutional pressures for change, led to the transmission of the performance measurement discourse to the education sector (Deem and Brehony, 2005; Gruening, 2001; Hood, 1991). NIT complements NPM by providing an appropriate lens to examine the effect of institutional pressures on the development of performance measurement systems in TEIs. The following sub-sections provide further discussions on NPM and NIT.
NPM
The theoretical foundations of NPM can be traced to neoliberalism that became widely accepted by many Western nations in the 1980s as ‘the defining political-economic paradigm’ of the time to help restructure and reconstitute essential public services including education (McChesney, 1998: 7). The neoliberal ideology is based on the principles of economic liberalisation and decentralisation, thus, economic prosperity is seen to come from free trade, market competition, privatisation, deregulation and a downsizing of the welfare state (Harvey, 2007). Sikka (2000) points out that the neoliberal ideology places emphasis on ‘markets’, ‘individualism’ and the ‘minimalist state’, with free ‘markets as the only magical solution to everything’(2000: 371–372). Neoliberalism gave rise to the devolution of state power to public institutions to be run as independent corporations. Paradoxically, governments that employ neoliberal strategies using marketisation and decentralisation often rely heavily on measures of state control over social and economic matters (Bessant et al., 2015). The centralised steering and state control of the public sector using neoliberal ideology became known as NPM or new managerialism (Bessant et al., 2015). NPM is characterised by ‘the integration of private sector management concepts and market mechanisms into the public sector’ in the quest to modernise and reduce public expenditure (Almquist et al., 2013: 481). NPM uses strategies based on market contestability to improve ‘efficiency, effectiveness and excellence’ (Deem, 2001: 10). The principles of greater accountability, transparency and autonomy are central to NPM (Hood, 1991).
The rise of NPM in the 1980s is closely linked to the evolution of the performance measurement discourse in the public sector (Hood, 1991). Here, performance measurement is broadly defined as a process of developing measurable indicators to manage the effective and efficient allocation of resources and demonstrate the achievement of outcomes at a political, societal and organisational level (Broadbent and Laughlin, 2009). Peters (1992) notes that performance measurement is ‘the major policy innovation within the larger managerialist orientation which, in itself, signals a fundamental change in the political ideology of the management of the public sector’ (1992: 268). In an attempt to improve accountability, transparency, cost-effectiveness and value for money, it became mandatory for public sector organisations to formulate goals, specify targets, measure performance and maintain a focus on outputs (Deem and Brehony, 2005; Gruening, 2001; Hood, 1991). According to Humphrey, et al. (1993: 7), NPM introduced new practices of ‘accountable management’ to secure change in the management and control of public sector organisations. Central to the notions of accountable management were ‘(E)fficiency scrutinies, value-for-money audits, performance indicators, resource management initiatives, computerised financial information systems, cash limits, delegated budgets and internal markets’ (Humphrey, et al., 1993: 7).
Neoliberalism and the associated discourses of NPM made a significant impact in transforming the education sector worldwide (Olssen and Peters, 2005). Many governments in the United Kingdom, Western Europe, North America, Australia and New Zealand have used NPM to restructure and deregulate the education sector (Gleeson and Husbands, 2003). Neumann and Guthrie (2002) highlighted that academic activities such as teaching and research have ‘become calculable, marketable and tradable under the commercialised and managerialist regime in which universities must now operate’ (2002: 724). Alvesson et al. (2017) highlighted that a growing weight of performance expectations is producing more and more meaningless research in educational institutions, while Courpasson (2013) bemoaned that a research performance culture of publishing for its own sake was eroding passionate scholarship. Ryan (2012: 5) claims that performance measurement regimes make academics ‘zombiefied’, that is, they respond rather mindlessly to demands and institutional pressures. In New Zealand, the role of accounting and the use of research performance measurement have been questioned. Narayan and Stittle (2018) highlighted processes of accounting polarising teaching and research. Curtis (2008: 179) revealed ‘how a rhetoric of rewarding excellence in research is subordinated to a new mangerialist thrust for efficiency’ while Narayan (2012) noted that performance based research funding drives the research culture of tertiary institutions. Ashcroft (2005:125) has argued that research performance measures create a culture of ‘winners and losers’, with less attention paid to excellence in teaching and ‘academic collegiality and quality research’ giving way to ‘widespread career anxiety and apprehension’.
With the encroachment of a corporate management culture in academic institutions essentially driven by pressures for greater accountability (Christopher, 2012), performance measurement has appeared high on government and educational institutions policy agendas (Peters, 1992). The drive to enhance performance has accelerated as ‘(E)ducation worldwide is increasingly concerned with performance and the performativity of teachers, students, and managers’ (Gleeson and Husbands, 2003: 500).
Based within this overall NPM context, this article draws on perspectives from NIT to build an explanation of the development and use of performance measures in New Zealand TEIs.
NIT
NIT has become a dominant theory often drawn on by researchers to help inform developments in performance measurement and management in the public sector (Modell, 2005, 2009). NIT offers a range of theoretical perspectives that emphasise the broader social, economic and political aspects of performance measurement, thus broadening the view of performance measurement as a mere technical practice of measuring the achievement of organisational objectives (Modell, 2009). Among the theoretical perspectives are the regulative and normative elements (Scott, 2001) and institutional logics (Thornton and Ocasio, 1999) that influence organisational change.
NIT suggests that public TEIs do not operate in isolation (Meyer and Rowan, 1977). They operate within a broader institutional environment dominated by roles, requirements, norms, values and beliefs about what constitutes acceptable organisational forms and behaviour (DiMaggio and Powell, 1983; Meyer and Rowan, 1977). As such, TEIs will adopt practices shaped by institutional factors that are rooted in the institutional context. NIT suggests that institutional isomorphic pressure forces TEIs to converge to common practices such as using performance measures. DiMaggio and Powell (1983) explain how coercive isomorphism (government regulation, policy, and market pressure), mimic isomorphism (imitating others) and normative isomorphism (dominant professions) influence organisational practices. TEIs will normally conform to the institutional pressures of performance measurement, monitoring and reporting to maintain their legitimacy and secure vital funding and resources.
However, performance measurement does not simply serve an institutionally conditioned, legitimacy-seeking behaviour. It serves a broader purpose of providing information to a range of constituencies. Its use goes beyond symbolic displays from instrumental action to efficiency-centred rational choice decision-making (Meyer and Rowan, 1977). As institutional environments are subjected to market forces, the change dynamics associated with the interplay between institutional and economic mechanisms comes to the fore (Davis and Marquis, 2005). More recent advances in NIT have emphasised how the rationalities guiding instrumental and economic action are situated in institutional logics (Lounsbury, 2008; Schneiberg and Clemens, 2006).
Institutional logics
Institutional logics provide an insightful complement to NPM by enhancing our understanding of factors driving government reforms in educational institutions. Thornton and Ocasio (1999) defined institutional logic as a ‘socially constructed, historical pattern of material practices, assumptions, values, beliefs, and rules by which individuals produce and reproduce their material substance, organise time and space, and provide meaning to their social reality’ (1999: 804). Institutional logics represent frames of reference that condition actors’ cognition and sense-making (Ezzamel et al., 2012). Most organisational actors rely on institutional logics as the common guiding principle to aid in decision-making (see Lounsbury, 2008).
NIT recognises that accounting systems operate under conditions of institutional complexity in organisational settings characterised by the multiplicity of institutional logics that compete, shift and interact (Ezzamel et al., 2012; Friedland and Alford, 1991; Marquis and Lounsbury, 2007; Thornton and Ocasio, 2008). As new logics are introduced, organisations and individuals accommodate or adjust practices and norms with the technologies and practices associated with the new logics (Thornton and Ocasio, 1999). Mobilisation of different institutional logics can bring about resistance to institutional change but organisational actors can collaborate and maintain their original logics (Reay and Hinings, 2009). However, favouring one logic over the other can cause fragmentation in institutional arrangements (Marquis and Lounsbury, 2007). Multiple logics may continue to coexist but when there is a rival logic, it may perpetuate fragmentation and practice variation (Lounsbury, 2007; Reay and Hinings, 2009).
The theoretical concept of institutional logics has received increasing attention in accounting studies (e.g. Ezzamel et al., 2012; Lounsbury, 2008 Modell, 2009; Reay and Hinings, 2009). The extant accounting literature acknowledges the notion of the socially constructed nature of reality and the plurality of institutional constructions. The common themes in this literature imply that the most obvious and rational way of implementing accounting change in one social context may be very different in another social setting. As such, organisations will rely on institutional logics to respond to diverse expectations and demands from the external environment to help resolve organisational complexity and contradictions. However, the interplay of multiple logics may produce accounting practice variations. Rautiainen and Jarvenpaa (2012) found that under competing institutional logics, the responses to performance measurement systems could be more complex, especially when changes are hindered, because trust and collaboration between groups with rival logics are unattainable. Chenhall et al., (2013) identified ‘the factors that promote and/or damage efforts to reach compromise’ between different logics (2013: 269). The authors suggest that performance measurement systems function as ‘compromising accounts’ if they speak of the demands of each group of internal stakeholders, as this provides ‘confirmation and reassurance that a particular mode of evaluation is, indeed, recognised and respected, thus making productive debate more likely’ (Chenhall et al., 2013: 282). Sundin et al. (2010: 219) found that a balanced scorecard had the ability to manage multiple competing logics when managers agreed that ‘the ultimate goal was to achieve a balance between the objectives’, rather than to single out one objective as the ‘most important’ one.
This article uses four key ‘institutional logics’ as appropriate lenses to examine and explain the development and use of performance measurement in TEIs. These are the accountability logic, logic of markets, logic of business and professional logic. The accountability logic includes both public and political accountability and requires TEIs to manage resources in an effective and efficient manner based on objectives, targets and results (Hood, 1991; Humphrey et al., 1993). The logic of markets require TEIs to rely on market forces to pursue growth strategies and compete for students and revenue streams, whereas the business logic constructs TEIs as businesses that seek to generate profits (Deem, 2001; Parker, 2011). The professional logic guides the norms of the academic profession and emphasises academic freedom and collegiality (Townley, 1997).
The next section outlines the research method employed in this study.
Research method
The study used a historical research method to collect, analyse and interpret data. Golder (2000) explains that ‘(t)he overriding characteristic of historical method is that all evidence is approached critically and sceptically’ (2000: 158).
Data were collected from two primary sources: archival documents and interviews. The major sources of archival data were the official government reports, legislation and relevant documents produced mainly by The Treasury, Ministry of Education, TEC and the Office of the Controller and Auditor-General between the period 1987 and 2017. It also included data from the annual reports, strategic plans, charters, profiles, investment plans and performance reports of two public TEIs. Published data were obtained from the websites of the TEIs and government departments. To further understand the use and impact of performance measures, 12 semi-structured interviews and various informal conversations were conducted with key actors in the TEIs, the Ministry of Education and TEC at different periods between December 2007 and November 2017. On average, the interviews lasted 45 minutes to 1 hour and all interviews were tape-recorded and transcribed. Semi-structured interviews allowed different perspectives and a richer data set to be obtained. Informal conversations with academic and management staff at the two TEIs helped keep up to date with developments in the field. To preserve the anonymity of both TEIs and interview participants, the two TEIs are referred to as TEI A and TEI B. These TEIs are typical of many public TEIs in New Zealand with similar strategic planning, performance measurement and reporting requirements.
Data analysis was carried out by organising significant historical developments in performance measurement over the 30-year period. Performance measurement and monitoring reports of government were crosschecked with TEIs’ annual reports and internal documents over different periods to provide information on the actual use of performance measures. Changes in performance measures over different periods were noted. The frequency of significant themes in interview transcripts, official documents and web sites were identified. The final stage involved identifying clusters of meaning from recurring patterns and themes and forming opinions. As Porra et al. (2014) state, the purpose of the historical method is to present the interpretations of past events to help understand what happened and why in the context of historical environmental forces. The following sections present the historical developments in performance measures and its impact on TEIs.
Performance measures in the 1980s and 1990s
The 1980s and 1990s were periods of major NPM reforms that led to the evolution of the performance measurement discourse in the public sector. Hence, this timeline is shown separately to emphasise the considerable importance assigned to the development of performance measures in TEIs during this period.
In New Zealand, the origins of the performance measures in TEIs can be tracked back to the more general reforms of the state sector. In the 1980s and 1990s, successive governments used public policy characterised by neoliberalism and NPM to open markets to competition as a means of regulating the economy. The overall trend of the government’s major macroeconomic policies and the socio-economic reform agenda had economic rationalism, market liberalisation, deregulation, decentralisation and enhanced accountability as its key features. The Treasury (1987) put out its ‘NPM manifesto’ in the document Government Management: Brief to the Incoming Government that set out the guiding principles of effective management as: […] clarity of objectives; freedom to manage; accountability; effective assessment of performance; and adequate information flows. (The Treasury, 1987: 55)
The Treasury was of the view that public sector performance measurement was severely constrained by a number of important factors, some of which are listed below:
Foremost is the fact that managers have typically faced unclear, and at times, conflicting, objectives.
The restrictions on managers’ ability to control the resources at their disposal to achieve those objectives has been severely constrained by input controls;
[…] there has been inadequate information on which to assess the quality of managerial decision making-the level of output achieved and the efficiency of resource use in achieving that output.
[…] the absence of sufficient authority to affect outcomes … [and] … the current lack of incentives … (The Treasury, 1987: 86)
Performance measurement and accountability were key concerns and the government relied on the coercive power of legislation to drive changes. The enactment of the State Sector Act 1988 and the Public Finance Act 1989 cleared the way for state sector reforms to help address performance measurement, management constraints and lack of incentives facing managers. The State Sector Act 1988 established the departmental Chief Executive’s accountability to the appropriate Minister for the conduct, management, duties and functions of the department with minimal input control from central agencies, thus allowing for more explicit accountability for performance. The Public Finance Act 1989 specified requirements for accountability, improved managerial performance and preparation of accrual budgeting and financial reporting by departments including a statement of service performance, which reported on the actual performance against the specified objectives. The Treasury, in its brief to the incoming government was of the view that An effective financial management system should impose powerful incentives on managers to perform (The Treasury, 1987: 82), and [a] move towards accrual accounting and budgeting would improve the information available to assess managerial performance. (The Treasury, 1987: 84)
The state sector reforms set a precedent for the radical reforms of the tertiary education sector. Specifically in the tertiary sector, The Treasury’s concern was that […] the Government has greater difficulty measuring the benefits of education […] As a result inputs tend to be used as measures of success instead, leading to the misapprehension that more is necessarily better, and to problems with determining the goals and parameters for intervention. (The Treasury, 1987: 134)
Throughout the late 1980s, a series of reports recommended market-driven logic to improve the efficiency and accountability of the tertiary education sector. The Picot Report and The Hawke Report (Department of Education, 1988a, 1988b) invoked neoliberal market-efficiency and business-managerial discourses. Most of the reports’ recommendations were incorporated as the government’s intended policies in the Learning for Life documents (Ministry of Education, 1989a, 1989b) that maintained a business-commercial logic complemented by the logic of free market forces to drive efficiency in the tertiary education sector. These documents provided the initial onslaught of neoliberal market policies that substantially changed the tertiary education system in New Zealand. The NPM reforms granted the TEIs institutional autonomy, each with their own governing council operating on a devolved contractual model of accountability through charters, mission statements, corporate plans and performance objectives (Ministry of Education, 1989a). These elements were embodied in The Education Amendment Act, 1990 with section (a) specifically,
[g]iving tertiary institutions as much independence and freedom to make academic, operational, and management decisions as is consistent with the nature of the services they provide, the efficient use of national resources, the national interest and the demands of accountability.
Section 203 (2) (d) of The Education Amendment Act 1990 imposed the requirement on the council of the TEI to submit to the Ministry of Education a statement of objectives and ‘a list of the performance indicators that will enable the preparation of a statement of service performance’ for purposes stated in the Public Finance Act 1989. The legislative framework of The Education Amendment Act 1990, the Public Finance Act 1989 and the neoliberal policy framework of the Learning for Life reforms paved the way for the development and implementation of a performance measurement system in the tertiary education sector.
Recommended performance measures
In 1989, government produced three reports on the development of performance indicators for use by the tertiary education sector. Initially, The Treasury prepared a separate report on financial performance indicators and the Performance Indicators Taskforce [the Taskforce] prepared a Draft Report on Performance Indicators. Both these reports were then incorporated in a single document titled Performance Indicators for Tertiary Institutions (Peters, 1992). As noted by Peters (1992: 272), the terms of reference for the Taskforce were as follows: To develop a set of performance indicators which will assist the Ministry of Education and other agencies to monitor the performance of tertiary institutions to ensure:
That the resources allocated to each institution are efficiently and effectively managed;
That each institution’s obligations, as set out in its charter, are met;
That the goals and objectives as set out in each institution’s approved corporate plan are met.
From the terms of reference, it seems that the prevailing logic of political and public accountability led to the development of performance indicators. To ensure compliance with such broad terms of reference, the taskforce recommended an extensive set of 56 performance indicators categorised into indicators for: efficiency, effectiveness, honouring the Treaty of Waitangi 1 , access to education and training, equal employment opportunities, research and scholarship, and financial performance. These performance measures were shaped by multiple logics. Efficiency indicators were intended to help determine how well institutions used public funds to achieve their objectives, whereas effectiveness indicators determined how successful institutions were in achieving their objectives. Financial performance indicators were intended to help ensure accountability for money spent and provide incentives for good management. Driven by the neoliberal agenda and the logic of the market, the access to education and training indicators was designed to make tertiary education accessible to as wide a range of students, including greater participation from under-represented and disadvantaged groups. The equal employment opportunities indicators were designed to be used to monitor the progress of institutions in implementing equal employment opportunity programmes, especially women, Māori 2 , Pacific Islanders and persons with disabilities. The special status of the Māori in terms of honouring the Treaty of Waitangi required that indicators on equity issues be separated from under-represented groups. The Taskforce recognised the problem of judging research performance, so they simply suggested a list of measures that could be published by a faculty or department periodically and assessed by an independent panel of practising researchers.
Appendix 1 provides a list of performance indicators recommended by the Taskforce. Only quantitative measures were recommended since the Taskforce found the ‘development of qualitative or descriptive measures is fraught with difficulties’ and ‘much of the data would not be hard enough to provide a basis of uniformity of comparison among institutions’ (Performance Indicators Taskforce, 1989: 13).
Use and impact of performance measures
The performance indicators recommended by the taskforce, principally shaped by the institutional logic of public and political accountability, was for use by the Ministry of Education to assist in its monitoring role. However, various interested parties including the TEIs could use these performance indicators to serve multiple purposes, including resource allocation, performance assessment and external monitoring of institutional performance:
[…] the Ministry will require information to enable it to compile national education and training statistics to assist in providing policy advice to the Government.
The Controller and Auditor General will also conduct periodic reviews which will comment on how well institutions and other chartered providers have used public funds to achieve the goals in their charters.
[…] the Education Review Office will report on ‘the extent to which institutions are successful in avoiding the creation of unnecessary barriers to the progress of students; implementing appropriate equal employment opportunities programmes; developing programmes to attract students from under-represented and disadvantaged groups’. (Performance Indicators Taskforce, 1989: 5)
The legislative framework of Education Amendment Act 1990 and the Public Finance Act 1989 made it mandatory for all TEIs to produce charters and corporate plans outlining their educational, financial and social goals with recommended performance indicators, which formed the basis for performance review and funding negotiations with the Ministry of Education. It also was mandatory for all TEIs to include in their annual report a statement of service performance with performance indicators specifying information about educational, financial and equity performance. The regulatory mechanisms (The Education Amendment Act 1990 and the Public Finance Act 1989) were the carriers of institutional logics of market and public accountability. TEIs were granted autonomy to compete in the market for students and funding as well as be accountable to the public for performance. In 1990, the government implemented bulk funding in TEIs designed to give: [… ]a high measure of autonomy and flexibility in their use of resources and to ensure public accountability through clear reporting requirements. (Tertiary Education Advisory Commission (TEAC), 2001c: 6.1.1)
From 1990, coercive isomorphic pressure from government forced TEIs to report on all performance indicators of efficiency, effectiveness and financial performance, and most indicators related to widening access and promoting equity. Since the Ministry of Education principally developed the performance indicators for its use in a monitoring role, it is unsurprising that the Ministry exerted its regulatory and coercive pressure on TEIs to use these specified performance indicators in funding negotiations.
From an analysis of annual reports and archival documents, it became obvious that TEIs struggled in their efforts to measure and report on their performance achievements. Concerns related to the sheer number of performance measures, manual collection and reporting of extensive data elements and an excessive focus on short-term performance outcomes, for example: […] such systems produce other unwanted outcomes: excessive focus on individual rather than collective performance, emphasis on short term versus long-term results, greater concern for extrinsic rather than intrinsic rewards. (Association of University Teachers, 1989: 4)
Even the Ministry of Education had difficulty getting performance information that was ‘either not yet publicly available or not in a form that can be reliably reported’ (Ministry of Education 1999: 8). The Report of The Controller and Auditor-General (2001: 3) on Reporting Public Sector Performance also expressed concern that
Stakeholders are not getting the best information they could on how public sector entities are performing.
There were concerns that quantitative performance measurement data used for future resource allocations collected at a technical level did not have adequate safeguards for interpretation. Despite attempts to relate inputs to outputs, resource decisions favoured considerations of quantity over quality: […] unless such systems are carefully structured, the quantity of activity rather than the quality of outcome will, inevitably, take precedence. (Association of University Teachers, 1989: 4)
The quantity of data taking precedence over quality of performance outcome was very evident at both the institutions. For example, TEI A devoted 44 pages of its 1994 Annual Report to report performance achievements against statement of objectives across seven different faculties and departments.
A review of various Ministry of Education performance monitoring reports demonstrated the widespread use of performance data to compare TEIs performance across different types of institutions, and across different faculties and departments. To meet the demands of public and political accountability, the Ministry of Education derived national performance indicators from TEIs and monitored everything from student enrolments, equivalent full-time students (EFTS), field of study, course and qualification completion rates to individual institution’s financial performance. The Ministry used performance indicators to establish benchmarks for course cost categories for funding and bulk funding entitlements for TEIs. From 1998, the Ministry of Education began publishing on an annual basis a Profile & Trends report that has become a primary information resource on New Zealand’s tertiary education system and performance: It describes participation in tertiary education and training in New Zealand, profiling the students, their courses of study and their achievements and outcomes … along with the … financial performance of public tertiary education institutions … (Ministry of Education 1999: 6)
Performance measures had a significant impact in making TEIs more accountable for the use of government funding. The linking of TEIs’ corporate planning and budgeting processes to performance indicators enabled measurement of progress towards the attainment of objectives. The use of performance indicators, shaped by NPM and the market logic, resulted in a phenomenal growth in student enrolments, courses and types of programmes offered at both the TEIs. For example, TEI A’s Annual Report 1994 revealed that enrolments grew by 55 per cent from 1991 to 1994 with 36 new programme approvals granted in 1994.
Several negative impacts of performance measurement were also revealed. With funding becoming entirely demand driven and guided by the market logic, ‘(q)uality assurance was weak, and information on value-added lacking. … The result was a very large increase in participation in programmes that were of doubtful value’ (Crawford, 2016: v).
Peters (1992: 280) citing The New Zealand Vice-Chancellors’ Committee (1990) noted the imposition of ‘a bureaucratic regime of reporting, monitoring and control far beyond that required for proper accountability’ resulted in the erosion of the principles of institutional autonomy and academic freedom. It conflicted with the logic of the academic profession. The crude quantitative measures used for funding allocation, external review and monitoring ran counter to the notion of TEIs developing a culture and practice of self-evaluation to address genuine accountability concerns related to efficiency, effectiveness and quality of education. TEIs A and B relied on institutionalised performance measures that served an external monitoring role but largely ignored the development of performance measures tailored for internal use, except for student evaluation of teaching, courses and staff performance appraisals. For example, they were not examining ‘with much rigour how well they [were] pursuing equity issues’ (Peters, 1992: 280). Performance measures showed that Māori, women and under-represented groups continued to lag behind. Performance measures guided by the logic of markets increased participation rates but failed to […] deliver better opportunities to access quality tertiary education; to assure quality of all taxpayer-subsidised qualifications, teaching and research; and to provide accountable, well-run public tertiary institutions that … provide high quality education. (Ministry of Education 1999: 9)
While the logics of public accountability and markets coexisted and were complementary in enabling financially oriented performance actions, they were not effective in achieving social performance outcomes. TEI A and B managers said that they used performance measures (e.g. EFTS measures) that were simple and had funding implications: We monitor EFTS very closely because it determines the funding we get …, if we get higher EFTS, it makes all our other performance measures look better. (Faculty Dean, TEI A)
The dysfunctional effect of performance measures was obvious in the TEIs. It encouraged staff to concentrate only on goals that were measured and funded: […] we just enrolled more students to achieve our EFTS targets and get our funding…the reality is that we were not concerned about the quality of students we were enrolling… we had high failure rates. (Head of School, TEI B)
Using performance measures to manage resources influenced by market forces and dictated by a desire to maximise funding introduced a complementary business logic. The business logic gave the TEIs powerful incentives to pursue growth strategies, make profits and become financially less dependent on the government.
Financial performance measures
Since 1990, financial performance has been the key focus of government reforms. The implementation of accrual accounting made it a requirement for TEIs to prepare a statement of objectives and statement of service performance. The financial performance indicators recommended by the taskforce and used by TEIs in meeting their reporting obligations were short-term liquidity, exposure to debt, exposure to debt servicing obligations, change in financial value of the institution and return on assets. The Taskforce report emphasised that these performance indicators reflected ‘[…] normal commercial practice’ and helped ‘ensure accountability for money spent and provide incentives for good management’ and, [i]f sound financial performance (e.g. avoiding insolvency and making good use of assets) can be ensured by a systematic use of indicators involving targets, reporting and monitoring, there will be less temptation to try and ensure it through central controls. (Performance Indicators Taskforce, 1989: 18)
A review of the annual reports and supporting commentary suggested that both TEIs had recognised the need to tightly couple their internal financial performance measures (revenue earned, cost per EFTS and financial surplus) to operations, to maintain financial control and help achieve the overall TEI financial performance targets. The internal financial performance measures were set, down to the faculty and department levels.
In 1990, the government introduced a user pays fee policy and opened up markets for export education. Facilitated by neoliberalism and guided by the logic of the markets, TEIs were incentivised by the business logic to maximise enrolments and subsequently improve profitability and financial performance. The interplay between the logics of the market and business saw the construction of TEIs shifting from a community of scholars producing knowledge as a public good for social consumption to that of a market led business producing knowledge for profit. As competition intensified, TEIs started branding themselves as ‘leading’ or ‘world class’ institutions to attract international fee-paying students. Annual reports for both TEIs revealed that for the 1990–2000 period, international fee income, research, consultancy and entrepreneurial income provided major alternative sources of funding. The total income of all New Zealand TEIs grew by 64 per cent between 1992 and 1999 (Ministry of Education, 2000).
Since the 1990s, with an increasing emphasis on financial performance measures, TEIs effectively transformed themselves into powerful consumer-oriented corporations that led to the adoption of corporate governance structures and the rise in power of senior management. Both TEIs A and B appointed professional business managers to contractual senior management positions with high remuneration packages often contingent on growing student enrolments, substantial budget responsibility and demanding financial performance targets. Senior management were quick to impose demanding financial performance targets on schools and units through restructuring, fixed term contracts for lecturers and greater use of less-costly casual teaching assistants. These were indicative of the business logic firmly embedded within TEI operations. The following comments highlight the impact of the business logic that had crept into academia: […] the power of academics have been redefined … profit measures is leading to increasing trends towards the commodification of teaching and research. (Academic Staff, TEI A) We have been transformed from a teaching institution to a business organisation managed by financial performance targets … of course, this will affect the quality of teaching and research. (Senior Academic, TEI B)
The business logic and the prevailing logic of markets precipitated a dramatic increase in staff workloads at both TEIs that reported average class size increases of up to 50 per cent to cater for high enrolments. Higher enrolments with no corresponding increase in staff costs gave the TEIs higher profits that signalled overall good performance.
Performance measurement beyond 2000
From 2000, there was greater realisation by the government that the competitive model of provision of tertiary education, guided by the logic of the market, had too little coordination. While the competitive market-based model prevailed, the government moved towards a system of central steering of the tertiary sector to ensure better alignment of tertiary education with national priorities (Ministry of Education, 2001). In April 2000, the government established a Teacher Education Advisory Commission (TEAC), which recommended the creation of a long-term Tertiary Education Strategy (TES) and a Statement of Tertiary Education Priorities (STEP) to allocate government funding: In general, the TES focuses on improving the ability of TEOs to manage for improved outcomes, rather than setting specific outcome targets. This is to be achieved through a mix of shifting attitudes and culture and the implementation of new funding and accountability policies. (Ministry of Education, 2004: 8)
The new accountability policies signalled the government’s intention to let the logic of accountability dominate the field with the logic of markets having a minor role. The enactment of the Education (Tertiary Reform) Amendment Act 2002 provided a statutory basis for TES and STEPS. The government also introduced a new integrated funding framework in 2002 to ‘support the alignment of the tertiary education system with New Zealand’s TES, and emphasise the importance of performance and excellence’ (Ministry of Education, 2002: 23).
Based on TEACs recommendations, The TEC, a new government agency, was established in 2003. The TEC created a system of charters and profiles with key performance indicators aligned with TES and STEP to make funding allocations and help influence the direction of tertiary education organisations towards national goals (Tertiary Education Advisory Commission (TEAC), 2000, 2001a, 2001b): The annual profile is to describe in greater detail the organisation’s strategic direction, activities, policies, and performance targets for the next three years … The profile contributes to a more detailed map of the tertiary education sector and establishes more consistent monitoring, reporting and accountability for publicly funded organisations. (Ministry of Education (2002: 22)
The first TES was issued for the 2002–2007 period. It emphasised greater alignment with national goals, increased quality, performance, effectiveness, efficiency and transparency. New performance measures such as course completion, retention, attrition and progression rates were introduced into TEIs to give a greater focus on the outputs of tertiary education (Tertiary Education Advisory Commission (TEAC), 2000, 2001a). The academic profession is based on norms of quality. TEAC recommended more outcome-focused measures to give a more complete picture of education quality since the traditional notions of quality indicators of performance (such as staff:student ratios) did not comprehensively measure the effectiveness of teaching and learning. In addition to the new performance indicators, TEIs were still required to maintain the current high overall rates of participation; increase overall achievement and completion rates; and increase participation, achievement, completion and progression by under-represented groups, especially Māori and Pacific peoples. Interestingly, the government was of the view that the logic of the market should coexist with the accountability logic to maintain high participation rates and strengthen accountability.
An analysis of profiles and strategic plans of TEIs A and B revealed that they exhibited coercive and mimetic isomorphic behaviours by replicating government’s tertiary education goals and priorities in their plans. Between the periods 2000 and 2004, participation rates increased and reached new highs with heightened concerns about quality and relevance of tertiary courses:
This reinforces the need for government’s new emphasis for tertiary education – having made substantial gains in participation, it is time to focus attention on the value of that participation – through a renewed emphasis on the quality and relevance of tertiary education …
3
Following the government’s release of the new TES 2007–2012, the focus returned to quality and relevant education and research that supported the nation’s goal to become an innovative, creative and knowledge-based economy. STEP was removed and the requirement for charters and profiles was abolished and replaced by institutional investment plans, negotiated between providers and the TEC. The investment plan set out performance targets that became the basis of funding allocations. Driven by the accountability logic, TEC removed the demand-driven funding and replaced it with EFTS targets that had a 3 per cent tolerance band, meaning that TEIs were expected to reach at least 97 per cent of the allocated total but no more than 103 per cent.
The TEC continued its focus on improving educational outcomes and in line with its TES 2010–2015, introduced a core set of educational performance indicators (EPIs) that measured TEIs’ educational performance through successful course completion, qualification completion, student progression to higher level study and student retention within a qualification (TEC, 2010). Performance-linked funding was introduced to promote continuous improvement in educational performance, improve outcomes for learners and increase the value of taxpayers’ investment in education. A key feature of the performance-linked funding was the reliance on previous year(s) performance against the EPIs to base the current year’s funding levels. The assemblage of performance measures and increase in TECs monitoring role clearly indicated the dominance of the accountability logic.
With greater emphasis placed on increased quality and course completion rates, TEIs introduced new or substantially revised internal performance measures on a variety of aspects of student satisfaction. These included measures of student satisfaction with the courses (including programme evaluation surveys), the support services, teaching staff quality and standards, learning resources and support, administration, the environment and campus facilities, and their overall educational experience. The TEC prescribed measures of successful course completion, pass rates, retention rates and progression rates were linked to funding. Hence, performance was closely monitored against benchmarks set at the department and faculty levels at TEIs A and B. This had a marked effect on academic staff, most particularly in terms of narrowing the focus on teaching. Academics at both the TEIs became increasingly concerned with covering limited content, and teaching to the test to achieve higher pass rates to meet the specified TEC performance targets on success, retention and course completion rates. The increasing pressure on academics to meet the predetermined pass rate and the exam grades were identified as major factors inhibiting good teaching and effective learning. According to one senior academic at TEI B: We are now passing students who normally will not pass … there is pressure from the top … course completion and retention rates are closely monitored … it’s all driven by the new funding system. (Senior Academic S2-TEI B)
The dominance of the accountability logic permeating into classrooms with the concomitant focus on achieving educational performance targets, such as pass rates, was seen as undermining the professional logic of the academics. Concerns expressed by academics at TEIs A and B were that good pedagogic practice and academics’ professionalism were undermined in the new performance measurement culture. The locus of key pedagogical decision-making had shifted from academics to senior finance and administrative personnel.
Financial monitoring
The business logic that drove TEIs also posed a number of risks, forcing the TEC to further tighten its monitoring role. The TEC established four measures of financial performance to monitor TEIs: operating surplus, net cash flow, liquid funds ratio, and the return on property, plant and equipment (TEC, 2010). The TEC also established a benchmark surplus of three per cent of operating revenue for prudent financial performance, including several performance risk bands for each financial measure. TEIs with a high-risk rating were very closely monitored by the TEC. From 2010, the TEC started a series of annual performance publications, which it said was focused on enhancing the transparency and accountability of the tertiary education sector. The report provided an overview of the financial performance of each TEI. This included performance achievements in the TEC’s four key measures of financial performance, and comparisons were made against the TEC’s minimum guidelines. The report also provided summary financial statements for the past three years. Indeed, the impact of the financial performance measures on TEIs was very significant: In 2010, 28 of the 31 institutions had an operating surplus (before abnormals) above the 3% of operating revenue … compared to 14 of 33 in 2005 … (Ministry of Education, 2012: 16)
However, tighter controls on student enrolments coupled with new funding restrictions eroded the surplus position of most TEIs. In 2014, only 13 of the 29 institutions had an operating surplus of above three per cent of operating revenue – the low-risk performance band score set by the TEC (Ministry of Education, 2016: 7).
Since 2010, an increasing number of cases of manipulation of enrolment provisions and retention of student fees for courses that were never completed came to light. Consequently, some tertiary organisations had to pay millions of dollars back to the government. All such cases were reported on the investigations page of the TEC website (www.tec.govt.nz). The Controller and Auditor-General’s concerns over performance measures were that Development over the last decade has been uneven, with heavy emphasis on financial and output reporting, and too little emphasis on other areas – such as impact evaluation and outcome reporting. (The Controller and Auditor-General, 2001: 3)
From the interviews, it became obvious that too much emphasis on meeting external financial performance measures was inhibiting internal resource allocation decisions at TEIs. However, there was general acceptance within TEIs that maintaining financial surplus and financial viability was important.
Research performance measurement
Research performance measurement was driven by the government’s desire to separate the funding of teaching from research and hold TEIs accountable for research performance. Strongly influenced by the accountability logic, TEAC recommended the introduction of a Performance-Based Research Fund (PBRF) to hold TEIs accountable for research (Tertiary Education Advisory Commission (TEAC), 2000).
TEC, respectful of the professional logic of the academic researchers, reiterated in its PBRF-2008 Annual Report that
The purpose of conducting research in the tertiary education sector is twofold: to advance knowledge and understanding across all fields of human endeavour; and to ensure learning and especially research training at the postgraduate level occurs in an environment characterised by vigorous and high-quality research activity.
The primary goal of the PBRF is to ensure excellent research in the tertiary education sector was encouraged and rewarded through funding on the basis of their performance. (TEC, 2008: 3)
In July 2002, the PBRF Working Group established by the government recommended detailed design and implementation of a performance-based system for funding research in TEIs (Performance-Based Research Fund Working Group (PBRF), 2002). The PBRF included elements similar to overseas research funding schemes in the United Kingdom and Australia. The PBRF funding formula was designed to measure both the quantity and quality of research based on three performance metrics. The first measure, referred to as ‘quality evaluation’, assessed the research quality of staff based largely on peer review of a researcher’s evidence portfolio of research outputs. This accounted for 60 per cent of the PBRF. The second measure, research degree completions, was the number of postgraduate research-based degrees completed in the TEI, accounting for 25 per cent of the PBRF. The third performance measure was the amount of income for research purposes received by the TEI from external sources. This accounted for 15 per cent of the PBRF (TEC, 2008).
The quality evaluation of research outputs, an important aspect of the PBRF, became compulsory for all academic staff in TEIs deemed ‘eligible’ by the TEC (TEC 2007: 17). TEIs have been through three rounds of PBRF assessment carried out in 2003, 2006 and 2012. The results of the 2012 quality evaluation showed ‘an overall increase in research quality … and proportionally more PBRF-eligible staff assigned an “A” or a “B” …’ (TEC, 2012: 5).
Indeed, PBRF created a research performance measurement system that ranked academics, TEIs and journals. Since the introduction of research measures, there has been growing competition among academics, TEIs and journals to achieve high rankings. The quality rankings of academic staff and journals created a global market for academics who had published in high-ranking journals. TEIs have been in competition to recruit ‘A’ ranked researchers whom they may reward with good salaries and incentives. Both TEIs in this research actively promoted their research performance on their websites, often trumpeting their successes based on PBRF results.
Academic staff at TEIs A and B were concerned that research productivity was reduced to simply counting publications in high-ranking journals rather than genuinely fostering relevant knowledge. Curtis (2008) explored issues arising from the first round of the PBRF and concluded that Despite the PBRF creating pressures for universities to manage academics in new ways its imperatives are so diverse and complex as to appear debilitating. (Curtis, 2008:190)
Similarly, Roberts (2013: 27) examined developments in PBRF and argued ‘that the trends evident in changes under the PBRF constitute a form of academic dystopia’.
PBRF measures permeated down to individual school and department level at TEIs A and B and became significant in the performance management of academics and workplace politics. Research measures of publications and external funding caused divisions among academic staff: We have some academic staff who just do research and very little teaching … might I add, some are not very successful in teaching … yet got promoted … research publications alone is not a full measure of an academic’s work performance. (Academic Staff 2-TEI B)
Performance measures for research quality intensified the focus on individual staff to publish as many papers as possible. It became obvious that only a few researchers were driven by a strong intrinsic motivation to pursue the truth, or to make new discoveries from their research. The sentiments of many staff are reflected in the following observations made by Roberts (2013: 35):
What the PBRF demands is not a rich, well rounded, complex portrait of a research life, but simply a list. It is measured performance that matters, not the knowledge or ideas, research cultures or commitments that give meaning and substance to lists of items and outputs.
PBRF created pressure on TEIs to manage academics in new ways. Both the TEIs in the study created teaching and research pathways for academics, implemented workload models to balance teaching, research and service responsibilities and introduced teaching and research excellence awards to recognise good performance. Despite these measures, concerns were raised that when it came to appointments, progression and promotions, it was obvious that research publications were the main criteria. Indeed, as suggested by Ashcroft (2005), PBRF had created a culture of ‘winners and losers’ and became a key criterion for promotions and academic progression. The ‘publish or perish’ culture was evident at both the TEIs.
Discussion
This article provides a 30-year history of the development, use and impact of performance measurement on TEIs. The study reveals how performance measurement emerged and evolved within the larger neoliberal and NPM context of government reforms to help address efficiency and accountability concerns of the tertiary education sector. Since the 1980s, there has been a proliferation of performance measures in TEIs. Successive governments have used performance measures as a set of policy interventions to steer TEIs towards the achievement of tertiary education outcomes. Regulatory interventions caused isomorphic convergence towards a common set of performance measures developed by the government to help monitor TEIs’ performance. A number of performance measures became institutionalised and enforced within TEIs because of multiple coercive pressures from government regulations and funding mechanisms. Governance by objectives, targets and results to meet the demands of accountability and the reinforcement of market mechanisms laid the foundations of a performance measurement culture in TEIs. TEIs became increasingly conditioned towards successive regimes of performance measures imposed on them via the strict monitoring and reporting role of the government. 4
However, the greater role of the government in setting performance measures and the effectiveness of performance measurement as a set of government policy interventions was questionable and caused tensions within TEIs. For example, there were concerns that the government had imposed bureaucratic regimes of performance measurement that went far beyond that required for proper accountability. Furthermore, the government imposed institutionalised performance measures that ran counter to the notion of TEIs developing a culture and practice of self-evaluation using internal performance measures to address genuine accountability concerns related to efficiency, effectiveness and quality of education. Performance measures used by government as a centralised form of control of TEIs also led to an erosion of institutional autonomy. The dysfunctional effect of performance measures in the form of gaming was obvious as staff were encouraged to concentrate only on goals that were measured and funded. Performance measurement in the governance of academic work conflicted with academic freedom, diminished academic collegiality and undermined good pedagogic practice and academics’ professionalism.
The study has revealed how performance measurement linked to institutional logics drove institutional changes within TEIs. The prevalence of four key institutional logics played a critical role in the development and use of performance measures in TEIs. These were the logic of public and political accountability, logic of markets, logic of business and logic of the academic profession. Indeed, institutional logics articulated the link between the broader institutional pressures with notions of choices, interests, identities, values and assumptions of individuals and TEIs (Thornton and Ocasio, 1999). The interplay and interaction between multiple logics facilitated the implementation of performance measures at different stages of institutional change. The logics of market and accountability were central to the introduction of performance measurement in TEIs. The market logic, implicitly echoing the neoliberal ideology, was intended to make TEIs autonomous, independent, effective and efficient. Similarly, the accountability logic was intended to ensure that TEIs managed their resources effectively and efficiently and met their goals and objectives prescribed by the government’s tertiary education strategies. These logics coexisted as ‘compromising accounts’ (Chenhall et al., 2013) and facilitated the implementation of efficiency, effectiveness and financial performance measures, mainly because all these measures were equally important, mandatory and, more importantly, were linked and enforced by regulatory mechanisms which became carriers of both the institutional logics of market and public accountability.
Using performance measures to manage resources influenced by market forces and dictated by a desire to maximise funding also introduced a complementary business logic. Performance measures based on the business logic gave TEIs powerful incentives to pursue growth strategies, make profits and become financially less dependent on the government. The resulting impact was that enrolments and tuition income grew exponentially between the periods 1990 and 2004 and TEIs showed improved profitability and financial performance. Interestingly, while performance measures linked to the logics of market and public accountability drove positive performance-oriented actions in some areas (e.g. increasing participation rates and improving financial performance), they were not effective in facilitating social performance outcomes (e.g. addressing equity issues).
From early 2000, the government started tightening marketisation policies and imposed stringent controls on demand-driven funding. Accountability logic became the prevailing logic and featured more prominently in the design and implementation of the new educational performance measures in 2010.
The use of some performance measures was not always aligned with the logic of the academic profession. The academic profession is based on norms, values and identities associated with the preservation of quality and autonomy. Performance measures, enacted by the market logic and complemented by the business logic to maximise profits, compromised quality and caused rivalry with the professional logic of academics. In fact, performance measures became separated from the lived reality and experience in which teaching and learning was taking place. As such, issues related to the quality and value of tertiary education surfaced and despite the government’s attempts to address quality issues using EPIs, they were not resolved. TEIs came under enormous pressure to meet the financial performance targets and new threats emerged. This manifested itself in forms of creative compliance often resulting in the manipulation and fabrication of results to fit the government’s imposed targets and measures of performance. The increase in reported incidents of ‘manipulation’ of enrolment provisions and retention of student fees for courses that were never completed resulted in further tightening of performance measures and monitoring by TEC.
Research performance measurement, introduced in 2002 in the form of PBRF, was driven by the government’s desire to separate the funding of teaching from research and hold TEIs accountable for research performance. Clearly, these objectives were achieved suggesting that the accountability logic prevailed. PBRF not only became a more comprehensive measure of research performance in TEIs, it arguably provided a more systematic allocation of research funding. It has contributed to the overall increase in research outputs and research quality in TEIs. However, PBRF also created a web of unintended consequences. It created a culture of ‘winners and losers’ that caused divisions among academic staff. With the ‘publish or perish’ culture well established within TEIs, research became ‘not an integral part of one’s being an academic – the manifestation of a desire to know – but a matter of survival’ (Roberts, 2013: 40). These unintended consequences of PBRF are indicative of the accountability logic not being aligned with the professional logic of academic researchers. Yet the coexistence of both logics suggested that PBRF systems functioned as ‘compromising accounts’ by speaking of the performance measurement needs of the government, TEIs and academic researchers.
The study found that performance measures had both positive and negative impacts. Performance measurement provided the government with the means to better allocate resources and made tertiary education accessible to as wide a range of students as possible. Its greatest impact was realised from the phenomenal growth in student enrolments and increased student choice in terms of courses and types of programmes offered at TEIs. Increased participation rates also provided much needed revenue and made TEIs less reliant on government funding. TEIs diversified their sources of income and international students became a major source of revenue for most TEIs. Being in a competitive international environment, TEIs needed a measured performance culture to be able to position themselves for the best students, the best staff and funding opportunities. Within TEIs, individual responsibility and competitiveness became increasingly conditioned via successive regimes of performance measures. Making publicly available information on the educational performance measures, such as course completion and retention rates, strengthened the accountability of TEIs. It also enabled students and employers to be better informed on choices about tertiary education.
While quantitative market-based performance measures had increased participation rates, it failed to ‘deliver quality tertiary education, teaching and research and accountable, well-run public tertiary institutions’ (Ministry of Education 1999: 9). In fact, the performance measurement culture based on quantifiable targets became an end in itself. Meeting such targets seemed to be what mattered the most. The explosion in participation rates until 2005 is the case in point. There was no real improvement in quality, but very large increases in participation rates in programmes, some that were of doubtful value, implying that TEIs simply shifted their priorities to meet the performance targets.
Performance measures permeating into classrooms with the concomitant focus on maximising educational performance targets, such as higher pass rates, potentially undermined the professional logic of the academics. The new performance measurement culture of regulating teaching and learning towards corporate goals through ever-tightening performance measures shifted the locus of key pedagogical decision-making from academics to senior finance and administrative personnel. Academics viewed this as a threat to good pedagogic practice and academics’ professionalism, especially when the ‘de-professionalizing’ tendency of tying performance measurement to government targets had failed to connect with the contextual realities of teaching and learning.
Performance measures shaped by the logics of the market and business saw the construction of TEIs shifting from a community of scholars producing knowledge as a public good for social consumption to that of a market-led business producing knowledge for profit. The increasing trends towards commodification of teaching and research had effectively transformed TEIs into powerful consumer-oriented corporations branding themselves as ‘leading’ or ‘world class’ institutions.
Not only did performance measures precipitate a dramatic increase in academic workloads, it had a distorting effect on the behaviour of academics as they sought to maximise their individual scores. PBRF measures became significant in the performance management of academics and workplace politics. Furthermore, an emphasis on ranking, driven by a desire to identify winners and losers in a game of academic prestige, caused divisions among staff. Research quality measures had intensified the focus on individual academics who were often rewarded for publications in top journals. Quality rankings of academic staff and journals created a global market for academics who published in the few top journals. Performance measures for research were criticised for reducing research productivity to simply counting publications in high-ranking journals rather than genuinely fostering relevant knowledge. The PBRF, in keeping up with NPM, ‘made research more individualistic, more competitive, more ‘outputs’ driven than ever before. In doing so, it has reduced ‘“knowledge” to a shallow imitation of its former self and narrowed conceptions of what counts as worthwhile research’ (Roberts, 2013: 41).
Conclusion
Tertiary education policymaking is complex and remains a deeply contested terrain. Over the past three decades, a succession of government policies based on NPM discourses led to the development and growing role of performance measurement in TEIs. This study makes a number of important contributions to the accounting history literature. It provides a well-documented historical research on how performance measurement, associated with NPM and multiple institutional logics, became central to shaping and reshaping the character of educational politics, government policies and the management of educational outcomes. The study also contributes to a deeper and richer understanding of performance measurement as a social, political and technical practice influencing changes in educational institutions. Using a historical context of government reforms, the study demonstrates how performance measurement became firmly embedded into the core functioning and culture of TEIs and influenced tertiary policymaking.
Indeed, performance measurement is important for the discharge of public accountability, but its socio-political dimension extends well beyond the NPM ideology. This study has revealed the transformative power of performance measurement in systematically redefining the very purpose, structure and internal dynamics of TEIs, including regulating the conduct of academics and research. The proliferation of performance measures has created a web of unintended consequences with its own tensions, cynicism and complications. The study suggests that what is measured is what matters the most in the functioning of TEIs. The enormous pressure to meet the government’s imposed targets and measures of performance can manifest itself into forms of creative compliance, manipulation and fabrication of results.
This study raises important issues for government policymakers and TEIs. The distorting effects of performance measurement require profound rethinking and must be managed at all levels. Both the government and TEIs must regularly review their key performance measures to ensure that they accomplish what they are intended to accomplish. A careful selection and use of performance measures should help minimise any dysfunctional effects. While the government prescribes certain mandatory performance measures for its own use in tertiary policy formulations, TEIs need not just solely rely on these measures. TEIs must develop their own internal measures of performance that are not separated from the lived reality and experience in which teaching, learning and research take place. This will help avoid dysfunctional behaviours resulting in the manipulation or fabrication of results to fit targets and measures demanded for external reporting purposes. Indeed, a good performance measurement system should lead to an effective process of continuous improvements to boost performance. In terms of future research, the unintended consequences of performance measurement in TEIs are worthy of further investigation.
Footnotes
Appendix
| Effectiveness indicators: Enrolled students Retention of students Successful students Progression to further education and training Progression to employment |
| Efficiency indicators: Student-staff ratio Operating cost per EFTS Net teaching area per EFTS |
| Honouring the Treaty of Waitangi: Equal educational opportunities: Ratio of actual Māori students to targeted Retention of Māori students Māori students satisfactorily completing courses Māori students progressing to further education and training Māori students progressing to employment Equal employment opportunities: Composition of teaching staff Ratio of permanent to non-permanent Māori teaching staff Composition of allied staff Ratio of permanent to non-permanent Māori staff Māori staff in senior positions Māori staff in staff development courses Earmarked funding: Ratio of actual Māori students to targeted for courses attracting earmarked funding Retention of Māori students within courses attracting earmarked funding Māori students satisfactorily completing courses attracting earmarked funding Operation costs per EFTS for Māori language and Tikanga Māori courses |
| Facilitating access: Ratio of actual EFTS to target for women, Pacific island students and students with disabilities Composition of full-time enrolments within programmes by gender and for Pacific Island and students with disabilities Composition of part-time enrolments within programmes by gender and for Pacific Island students and students with disabilities Retention of students by gender for Pacific Island students and students with disabilities Satisfactory course completions by gender, for Pacific Island students with disabilities Programme graduates by gender and for Pacific Island students with disabilities Progression to further education and training by gender and for Pacific Island and students with disabilities Progression to employment by gender, for Pacific Island and students with disabilities |
| Equal employment opportunities indicators: Composition of teaching staff Distribution of teaching staff by category of employment Composition of allied staff Distribution of allied staff by category of employment Composition of staff in senior teaching positions Participation in staff development courses |
| Financial performance indicators: Short Term Liquidity Exposure to Debt Exposure to Debt Servicing Obligations Change in Financial Value of the Institution Return on Assets |
| Measures to evaluate research and scholarship: Number of research students by subject Number of research degrees awarded by subject each year First destination of graduates with a research degree The number of individual research grants Their collective dollar-value relative to total income (ratio) Number of books published Number of books edited Number of refereed chapters in books Number of refereed papers and articles Number of published conference papers Other significant outputs, for example, musical compositions, recordings, patents, licence agreements, films, videos and arts exhibitions. The number of invited lectures to learned and professional bodies Contributions and consultancy to bodies outside the institution, including reviewer and editorial participation in publications of a national or international significance Elected membership of national and international learned bodies. |
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.