Balancing Narrow and Broad Public Service Professionalism: Experience With the New Zealand G-REG Qualifications Framework

Abstract

For many years, governments around the globe have been called on to increase the professionalism of their public services. The New Zealand Government Regulatory Practice Initiative (G-REG) is an illustrative example of a network of government agencies responding to this call by providing a program of standardized training for public servants. This article maps, explores, and interrogates this example to obtain a better understanding of whether a standardized program can help to nurture and increase the professionalism of a community of public servants. It finds that the main challenge of such an undertaking is finding a balance between narrow professionalism (technical expertise and knowledge) and broad professionalism (acting proficiently and ethically).

Keywords

regulatory delivery regulatory practice public service training public service skills narrow professionalism broad professionalism

Introduction

Over the last decade and around the globe, we have witnessed ongoing calls from policymakers, public servants, and scholars for the increased professionalism of public service delivery (Chen et al., 2019; Grugulis & Vincent, 2009; Nhede, 2018; Organisation for Economic Co-operation and Development [OECD], 2017; Wang et al., 2019). More and more, those working in public services are expected to have specialized knowledge in public service delivery and to apply this knowledge ethically for the good of the broader society (Bøgh Andersen & Holm Pedersen, 2012; Dyer, 2018). Such calls for increased public service professionalism fit well with the broader calls for innovation, creativity, and continuous improvement in the organization and functioning of government that have been dominating public administration discourses since the 1980s—such as New Public Management, New Public Governance, and Post-Bureaucracy (Budd, 2006; Johnson et al., 2009; Osborne & Gaebler, 1992).

Although these normative calls for increased public service professionalism have been made repeatedly on the pages of public administration books and journals, we lack insight into how governments around the globe are living up to them (OECD, 1997, 2017). This article seeks to address this knowledge gap through an in-depth case study of the G-REG qualifications framework (G-REG is the New Zealand Government Regulatory Practice Initiative, which is a network of government agencies that aims to increase the professionalism of regulatory practice). Introduced in 2016, this framework is a set of formal and standardized training programs for a broad group of public servants engaged in regulatory practice in New Zealand. The framework is illustrative of a response to the long-standing criticism that public servants often have specialist training in a profession but lack professional knowledge of government and public administration (cf. Wilson, 1893); it is also illustrative of a response to the long-standing criticism that professional training is often not an integrated part of the public service process (cf. Caldwell, 1962).

The central question addressed in this article is whether (and, if so, how) a standardized program, such as the G-REG qualifications framework, can help to nurture and increase the professionalism of a community of public servants. In the remainder of this article, first, the link between formal training and professionalism in public service delivery is further explored. After this, the background to the G-REG initiative and its qualifications framework are discussed. A brief discussion of the research methodology is presented, followed by the main insights from the case study. The article concludes with a discussion and an overview of the main lessons learnt from this case study for public administration scholarship and practice. In sum, the G-REG initiative illustrates the challenges of trying simultaneously to improve public service professionalism in a narrow sense (technical expertise and knowledge) and a broad sense (acting proficiently and ethically).

Ongoing Calls for Public Service Training and Professionalism

Professionalism can be conceptualized in a narrow and a broad sense. In a narrow sense, being professional means having completed advanced and specific training and holding expertise based on recognized credentials; in a broad sense, being professional means acting proficiently and ethically, being competent in one’s job, and striving for quality (Ospina, 1996; Saks, 2012). Framed differently, professionalism relates to “being a professional” as well as to “practising professionally” (Dyer, 2018). Nurturing and increasing the professionalism (in both the narrow and the broad sense) of a community of people is anything but easy, as becomes clear when reviewing the history of traditional professions such as accountancy and planning, and newer ones such as project management and data science (Lee, 1991; Peck, 2009; Pollack & Alego, 2015; Prier et al., 2010; Walker, 2015). At the same time, these professions indicate that it is possible to nurture professionalism. An overview of the literature on the history of these professions shows that two elements stand out as necessary (but often not sufficient) for nurturing the professionalism of a community of practitioners: the development of a shared body of specialized knowledge, and the intensive training of people in that knowledge (De Beelde, 2002; Greenwood, 1957; Tapper & Millett, 2015).

Those two necessary elements should come as little surprise to public administration scholars and practitioners (Lauer Schachter, 2007). Calls for (professional) public service training can be traced back to, at least, the writings of Woodrow Wilson (1887, 1893). Wilson observed that both the content of public administration and the general public’s expectation of public service had changed rapidly in the second half of the 19th century. Wilson believed that this called for dedicated and well-trained public servants: people with a deep understanding of government and public affairs. However, Wilson argued, no such training was provided, leading to a “harmful” situation in which “ignorant” specialists (i.e., people trained in the traditional professions) with at best “partial knowledge” of government made up the public service. Wilson’s call for dedicated public service training has often been repeated since. For example, despite their differences about what the study of public administration can and cannot achieve, Dwight Waldo (1948/2007) and Herbert Simon (1945/1997) agreed that the training of public servants is paramount for achieving public policy goals. In short, these godfathers of public administration (Wilson, Waldo, Simon, and their contemporaries) were calling for the increased professionalism of public services in a narrow sense (“being a professional public servant”).

By the end of the 20th century, the calls for public service training had not diminished (e.g., Dwivedi & Engelbert, 1981; Sherwood, 1997; Sims, 1993b), and they were often included in broader calls for a revision of government and public service delivery along the lines of “new” paradigms such as New Public Management, New Public Governance, and, later, Post-Bureaucracy (Raadschelders, 2011). In line with these paradigms, scholars began to envisage training as a means to enhance the professionalism of public service in a broad sense (“practising professionally as a public servant”; M. Clarke, 1998; N. Clarke, 2001). In a nutshell, public servants of the 21st century are expected to behave proficiently and ethically, to understand how their actions affect broader public policy goals, and to reflect on what it means to be delivering public services beyond their day-to-day work and beyond their policy area, and it is expected that they can be trained in these qualities (Bilmes & Gould, 2009; Nhede, 2018). However, despite these ongoing calls for public service training and the high hopes expressed in the academic literature, governments around the world have a poor track record of delivering or making such training available to their staff (OECD, 1997, 2017). It is against this background that the New Zealand case study that follows should be understood.

Toward G-REG and Toward Qualifications for Regulators

Since the early 1990s, the New Zealand government has introduced various regulatory policies and quality control tools as part of its ongoing regulatory reforms (New Zealand Treasury, 2017). The focus of these reforms has shifted over the years, and it is now firmly on the competency of regulators and the quality of regulatory implementation. A central aim is to increase the professionalism and contextual knowledge of those working in regulatory environments (Manch, 2014). In New Zealand, this is referred to as “regulatory practice,” a term that overlaps with “administering regulation” or “regulatory delivery” activities carried out by “front-line” regulators and “street-level bureaucrats” (Lipsky, 1980; May & Winter, 2009; Russell & Hodges, 2019). Regulatory practice includes, but is not limited to, public service delivery tasks such as informing, educating, and advising the public on regulation and compliance, and assessing, processing, and issuing permits and other rights. In New Zealand, more than 12,000 employees are involved in regulatory practice.

The idea of improved regulatory practice (as a form of public service delivery) emerged from the bottom up within several regulatory agencies in New Zealand (Fantham et al., 2020). In 2008, various government agencies approached the public-sector Industry Training Organisation (ITO) with requests for training in regulatory practice. These agencies had identified the risks of the current model, as well as other factors such as a lack of consistency in how different regulatory agencies and regulatory practitioners carried out their work; a lack of collective and individual consciousness of what it means to be a “good” regulator; and explicit calls by regulatory staff for training and professional development. This led to the launch of the Compliance Common Capability Programme (CCCP) by a group of national and local agencies, recognizing the need within the community of regulatory professionals to develop expertise and core competencies in the regulatory sector (Manch, 2014). The CCCP was in operation from 2008 until the advent of G-REG in 2015.

At the same time, in 2013, the New Zealand government tasked the Productivity Commission¹ to carry out an extensive review of New Zealand’s regulatory institutions and practices. In 2014, the Commission issued its report, Regulatory Institutions and Practices (New Zealand Productivity Commission, 2014). Among other things, the report suggested that “New Zealand needs a more professionalised regulatory workforce, with better training and career pathways” (New Zealand Productivity Commission, 2014). In 2015, the New Zealand government formally responded to the Productivity Commission’s report. It acknowledged the need for greater professionalization in the regulatory workforce, and indicated that

chief executives of regulatory agencies have agreed to work together to build regulatory capability [through] a cross-government forum, the Government Regulatory Practice Initiative . . . The initial focus will be on further developing a qualifications framework and best practice compliance strategies. (New Zealand Government, 2015, p. 4)

By that time, the CCCP had morphed into G-REG (Fantham et al., 2020).

One of the flagship initiatives of G-REG is the development and implementation of a suite of New Zealand Certificates in Regulatory Compliance (colloquially referred to as the “G-REG qualifications”) to “formalise training and improve leadership, culture and capability in regulatory practice throughout local and central government” (G-REG, 2019). The G-REG qualifications are believed to be the first public-sector regulatory training initiatives in the world (Skills, 2017b). They aim to do the following: to create a common language among regulators; to ensure that regulatory practitioners have access to and are exposed to a similar knowledge base; and to provide a professional pathway for those who work in or wish to work in a regulatory environment. “The qualifications allow public-sector staff working in regulation to operate at a consistent, translatable level across sectors. This could be across areas as varied as maritime safety inspectors, food safety officers, fisheries officers, parking wardens and investigators” (Skills, 2017a, p. 14). The development of the G-REG qualifications began in 2015 as a collaborative project between 23 government agencies and Skills, a standard setting body within New Zealand.²

The G-REG Qualifications Framework

The full suite of G-REG qualifications consists of six qualifications at different levels, providing a learning and development pathway from very basic regulatory knowledge and competencies all the way to highly specialist knowledge (Levels 3–6 within the 10 levels of the New Zealand Qualifications Framework³). They reflect what can be conceptualized as horizontal and vertical dimensions, or a trajectory, in public service training and development (Corson, 1963). The horizontal dimension ensures a shared level of knowledge and understanding for those who enter the regulatory profession without training in this area (they can take the entry-level qualifications). The vertical dimension provides a pathway for those who want to make a career in this area and want to move up the ranks (they can take the more advanced qualifications). There is no mandatory pathway for completing the suite of G-REG qualifications.

Structure

The base qualification is Core Knowledge (Level 3), which exposes learners to a broad overview of regulatory theory, core legal knowledge, and regulatory activities. It provides them with a common language, gives them an understanding of the New Zealand regulatory system, and touches on professional and ethical conduct and behavior. This qualification is provided through online learning in six modules. Each module has a knowledge transfer and an examination component. In each module, learners are offered three examination opportunities from a randomized question bank of closed questions. If a learner fails an examination twice, their assessment is paused and their local learning supporter (discussed later) is notified and contacts the learner to review the failed assessment and determine whether additional training may be necessary. The typical duration of the training for this certificate is 4 to 7 months, and the target audience includes regulatory practitioners, policy advisors, and support staff.

The qualifications that follow are primarily targeted at regulatory practitioners. The Operational Knowledge and Operational Practice qualifications (both Level 4) aim to increase participants’ knowledge and understanding of regulatory best practices. They help participants develop a stronger understanding of regulatory activities through the application of real-life case studies and by exposing them to the legal framework required for work across a range of agencies. Operational Knowledge is targeted at regulatory professionals who work in routine operational roles. It addresses topics ranging from legal systems (powers and obligations, offenses, and evidence) to the core regulatory activities in which regulatory professionals may engage, the interventions available to them, and proper modes of communication. As with Core Knowledge, the Operational Knowledge qualification is provided and assessed through an online learning system. It consists of three modules that give participants a deeper knowledge of the New Zealand legal system, and it uses both an audit case study and an investigation case study to inform participants about the practicalities of regulation in a real-world context. Thus, Levels 3 and 4 address the different learning styles of participants (Kolb, 2015).

The Operational Practice qualification is targeted at those working in routine frontline roles. It addresses topics ranging through professional ethics, safety and well-being, communication and stakeholder engagement; conducting various regulatory activities; carrying out interventions; and creating and managing records. This qualification is best understood as in-service training provided “in-house” by the various regulatory agencies. For example, WorkSafe New Zealand provides its staff with 27 weeks of training specifically addressing the regulatory activities in which WorkSafe is involved, such as workplace health and safety. The in-house training programs are evaluated by Skills against the New Zealand Qualifications Framework. This ensures that the in-house programs meet Level 4 requirements and, which is equally important, that there is some consistency across the programs. Critically, this provides participants with transferable knowledge and competencies. During their training, participants build up an evidence portfolio to indicate how they have applied their knowledge and competencies in their day-to-day work. This evidence portfolio is assessed against the qualification criteria by an assessor.⁴

The next level is a more specialized set of qualifications (at Level 5) with strands in Audit, Inspection, Investigation, and Operations. These qualifications help graduates to strengthen their knowledge and competencies and to operate independently in a range of situations within their function area. The topics addressed in this set of qualifications are using a risk-based approach to regulatory governance and practice; planning, managing, and conducting audits, inspections, investigations, and operations; and establishing daily practices consistent with ethical, legal, and organizational requirements. The mode of delivery and evaluation for these qualifications is comparable with that for Operational Practice.

Finally, the most advanced qualification is the New Zealand Diploma in Regulatory Compliance Investigations (Level 6). This qualification provides learners with the knowledge and competencies to lead, manage, and carry out complex regulatory compliance investigations that involve high potential risks or liabilities, a high level of public interest, complex or multiple legal or jurisdictional aspects, uncooperative subjects, and so on. This qualification can best be compared with the writing of a research dissertation by a university student: The participant writes an investigation file and puts that up for defense. The file is peer-assessed by three experienced experts in the field of the topic of investigation. The assessment process is monitored by Skills.

Development and Implementation of the Qualifications

Prior to 2014, many regulatory agencies already had in-house training in place for their staff (at the Core Knowledge level), but the CCCP and later G-REG wished to see greater coherence and continuity across the sectors, and the provision of deeper knowledge (particularly for Level 4 and above). In the design and development of the modules for Core Knowledge (Level 3), which were developed first, Skills and G-REG could build on some of the available material. A Skills representative shared the information that, in 2015 and 2016, design, development, and implementation often went hand-in-hand for the first version of Core Knowledge: “We were building it while we were flying it.” A pilot version of Core Knowledge was developed in 2015 and 2016. Based on a graduate profile already developed by the CCCP, a G-REG working group consisting of eight individuals with different experience and expertise in regulatory practice, and supported by Skills, developed the content of the online learning material and the question bank of closed test questions.

The Core Knowledge qualification is aimed at both experienced staff and staff new to regulatory practice. For experienced staff, the qualification is intended to reinforce and standardize their knowledge and, ideally, change some of their habits. For new staff, the qualification is intended to provide them with the basics of regulatory practice. Because the qualification must be scalable (there is a potential audience of 12,000 government staff working in a regulatory environment), it was decided that Core Knowledge would be delivered through an online platform and assessed through closed questions. The Skills representative explained, “A workplace assessor, an evidence-based portfolio, or even open questions would require a massive resource that we knew the sector did not have. The most effective way was an automated system for Core Knowledge.” The first module of Core Knowledge was implemented as a pilot in July 2016. That year, just more than 500 learners from a small number of regulatory agencies were enrolled. Lessons were drawn from their experience, and changes were made to the module. Other pilot modules were developed throughout 2016, and a first full revised version of Core Knowledge was implemented in 2017. In 2019, a second version of Core Knowledge was launched, based on lessons learnt from implementing the first version.

In developing Core Knowledge, Skills was responsible for developing the online delivery of knowledge, and it was decided that it was up to the agencies whether or not they would develop in-house training. In-house training has the advantage that the information presented in the online modules can be contextualized within the work area of an agency. Some of the agencies work with cohorts of learners in a classroom-like setting and go through the learning material in the group supported by a learning supporter. Other agencies leave it to their staff to explore the learning material individually. A similar structure is in use for Operational Knowledge (Level 4). Other qualifications follow an in-service training structure, where participants are supervised by in-house senior staff or peers. The certificates in Operational Practice (Level 4) and the various specializations in Audit, Inspection, Investigation, and Operations (Level 5) were rolled out in 2017, and the Diploma in Regulatory Compliance Investigations will be rolled out throughout 2020. In 2020, G-REG and Skills will also roll out a continuous professional education program that addresses specialist topics in regulatory practice for both graduates and nongraduates of the qualifications. Finally, at present (June 2020), G-REG, Skills, and the Victoria University of Wellington are exploring the possibility of allowing graduates of the G-REG qualifications to enter for a university degree. Depending on the number of credits collected through the qualifications, up to 1 year of a bachelor’s degree or master’s degree training could then be waived if the student entered a targeted university course (e.g., in public policy or public administration).

Data Collection and Analysis

By the end of 2019, the suite of qualifications had a total of 6,804 past and current enrolments, with 4,722 courses having been successfully completed. Of the suite, the base qualification—Operational Knowledge—had seen the highest number of enrolments and successful completions: 4,508 and 3,442, respectively. This indicates that approximately a quarter of the people working in the New Zealand regulatory practice community obtained a Core Knowledge qualification between 2016 and 2019. Upon completion of this qualification, graduates are expected to be able to do the following: to apply their knowledge of operating and regulatory activities in a regulatory context; to apply their knowledge of record and evidence management and information and privacy legislation in a regulatory context; and to interact professionally with others when working in a regulatory context.⁵

In what follows, experiences with Core Knowledge are evaluated to gain an understanding of whether a standardized program can help to lift the performance standards of public service delivery and to nurture a sense of professionalism among public servants. The evaluation of the Core Knowledge certification follows the widely applied “four levels of training evaluation” model by Donald Kirkpatrick (Bates, 2005; J. Kirkpatrick & Kayser, 2016; D. L. Kirkpatrick & Kirkpatrick, 2005). The model includes an assessment of (a) the reaction of participants to their training program (referred to as “reaction” in the evaluation section below); (b) indicators of the learning that has taken place over the course of the training program (“learning”); (c) behavior outcomes that indicate the extent to which knowledge and competencies obtained over the course of the training program are being applied by the learners after their training (“behavior”); and (d) the impact that the training has had on the broader organizational goals, objectives, and operations (“results”). The assessment that follows builds on different data sources. Some of these data were provided by G-REG, local and national regulatory agencies in New Zealand, and Skills; other data were obtained specifically for the assessment that follows. Table 1 provides an overview of the data sources used for each level of the evaluation, and their origins.

Table 1.

Data Sources and Origins.

Kirkpatricklevel	Data source	Origin of data
Reaction	• Learner experience survey(n = 465) • Manager/supervisor survey(n = 43)	• Skills (existing data), 2018–2019 • Skills (existing data), 2018–2019
Learning	• Pretraining test (n = 1,941) and posttraining test (n = 23)• Learner experience survey(n = 465) • Manager/supervisor survey (n = 43) • Interviews with managers/supervisors (n = 11)	• JvdH and Skills (new data), 2019• Skills (existing data), 2018–2019• Skills (existing data), 2018–2019• JvdH (new data), 2019
Behavior	• Learner experience survey(n = 465)• Manager/supervisor survey(n = 43)• Interviews with managers/supervisors (n = 11)• Interviews with G-REG and Skills representatives (n = 2)• Author participation in G-REG meetings (n = 12)	• Skills (existing data), 2018–2019• Skills (existing data), 2018–2019• JvdH (new data), 2019• JvdH (new data), 2019• JvdH (new data), 2019
Results	• Interviews with managers/supervisors (n = 11) • Interviews with G-REG and Skills representatives (n = 2) • Author participation in G-REG meetings (n = 12)	• JvdH (new data), 2019 • JvdH (new data), 2019 • JvdH (new data), 2019

Note. G-REG = Government Regulatory Practice Initiative; JvdH = Jeroen van der Heijden.

Table 1 indicates that six core data sets are used for the evaluation. The first and second sets of data were provided by Skills. Skills periodically carries out consistency reviews of the qualifications for New Zealand Qualifications Authority (NZQA; review reports are available from www.nzqa.govt.nz). These evaluations build on self-evaluations by learners and evaluations by their managers and supervisors. In 2018, Skills sent a survey request to all the 1,832 Core Knowledge graduates at that time, and received 465 responses (response rate: 25.3%). Skills also sent a survey request to 665 managers in regulatory agencies and received 43 responses (response rate: 6.5%). The survey asked the respondents to react to five statements about the Core Knowledge qualification on a 5-point scale, and provided them with an opportunity to add comments.⁶ All the raw survey data were made available by Skills for the evaluation that follows, and the survey and the raw data are referred to as the “Skills” survey in what follows.

The third set of data was collected in collaboration with Skills. In 2019, all new participants (n = 1,941) were given a pretraining test to complete before they could begin their first Core Knowledge training module. The test consisted of six sets of closed questions. Each set of questions was representative of one of the six Core Knowledge modules. In addition, upon completion of the final Core Knowledge module, the learners were asked whether they would agree to fill out a posttraining test. This test had the same questions as the pretraining test. The total number of participants who completed the post-training test was low (n = 23). We were, unfortunately, not able to make the posttraining test mandatory, and could not provide tangible incentives to boost the response rate.

The fourth set of data consists of a series of semi-structured interviews carried out with managers and supervisors (n = 11) at nine regulatory agencies. The interviews followed a set of prepared questions and allowed for free discussion following up from them.⁷ The interviews were carried out at agencies in which a high number of staff had completed the Core Knowledge qualification, and at agencies that had seen only a modest number of their staff going through the qualification process. These interviews lasted approximately 1 hr each. The interviews were recorded and transcribed.

The fifth set of data is a series of open interviews with a G-REG representative and a Skills representative (n = 2). These representatives were interviewed multiple times throughout the evaluation, for a total of approximately 3 hr per interviewee. These interviews were also recorded and transcribed.

The sixth and final set of data consists of extensive notes that I have taken at periodical G-REG meetings (n = 12) that bring together agency chief executives, managers, frontline staff, and administrators. The G-REG qualifications were a recurring theme at these meetings. This form of participatory observation allowed me to gain additional information about how various stakeholders in the regulatory practice community perceive the qualifications, without having to prompt individuals to share their insights and opinions. In addition to these core data sources, I have completed the Core Knowledge qualification (Version 1) as part of the evaluation process. Furthermore, G-REG has given me access to video and written evidence in which learners and their managers and supervisors share their experiences while working toward the qualification. Finally, other existing data, including, but not limited to, blogs, newspaper articles, and promotional videos on the qualifications, were used for the validation of the above-mentioned data.

The data were processed following conventional practice for this type of research (Brady & Collier, 2004; Goertz & Mahony, 2012; Silverman, 2001). The transcripts and notes were coded using a systematic coding scheme. The data were systematically explored, and insights were gained into the “repetitiveness” and “rarity” of the experiences shared by the interviewees as well as the information provided by the additional sources. Out of respect for the interviewees’ requests to stay anonymous in publication, where direct quotations from interview transcripts are included in the remainder of this article, the respondents are identified by Roman numerals. Out of respect for the survey respondents’ anonymity, they are identified by an Arabic number and the designation “G” for graduate or “M” for manager (and typos in their responses have been corrected for ease of reading).

Still, a note of caution is required. Although care has been taken to include as many different viewpoints and datapoints as possible within the time and budget constraints faced, it needs to be acknowledged that the data largely rely on the self-evaluations of Core Knowledge participants and on anecdotal evidence provided by managers and supervisors in a modest number of regulatory agencies. With this type of research and data, there is a risk that positive experiences are overrepresented in the voluntary self-evaluations and that, specifically, those managers and supervisors who were generally positive about the qualifications were more willing to participate in the interviews than those who were generally less positive about them (Roller & Lavrakas, 2015).

Findings

Reaction

Overall, the participants graded the qualification positively in the Skills’ survey—an 4.25 score of 5. Participant satisfaction was shown most strongly by the response to the statement: “This qualification has been relevant to my role,” which received an average score of 4.17 of 5 (with 5 representing strongly agree). The managers at the regulatory agencies, however, were slightly less positive about the qualification overall, and gave it an overall score of 4.23 of 5. However, when presented with the statement, “This qualification has relevance to the graduate in their role,” the managers responded with an average score of 4.40 of 5, indicating a slightly more positive view than the graduates. Of course, there are individual experiences behind these numbers, and these are seen in the comments provided by the graduates.

A first set of experiences about the qualification that was shared in the Skills’ survey highlights that mode of delivery matters. Very few graduates criticized the online mode of delivery, but several questioned whether and how the general knowledge on regulatory practice that was provided in the course could be applied to their specific work environment. One graduate commented, “I found the course too general to be of much use” (G345). Another was a little more detailed:

There was so much information that is not relevant to my role. Many examples and scenarios [come] from the Ministry of Justice, from Fisheries, and so on. I would have preferred it if it had related closer to my specific role in my Ministry. (G45)

Such comments were mainly expressed by graduates from regulatory agencies that do not work with cohorts of learners by taking them through the learning material as a group in a classroom-like setting. Over time, many agencies have developed a mode of delivery that puts the general knowledge of the online modules in the context of their in-house learners. For example, the New Zealand Transport Authority (NZTA) has developed a training program in which cohorts of learners work through the modules together. Senior NZTA staff contribute to this program and give brief lectures in which they present examples from their work within the NZTA as it relates to the Core Knowledge qualification. Graduates who had been exposed to this kind of tailored delivery of the learning material tended to have had a positive learning experience. One of them, from an agency other than the NZTA, stated, “Modules supported with face-to-face presentations worked well” (G332).

Another set of experiences shared in the Skills’ survey related to the examination of the six modules. Although only a few critical comments were made about the format of the assessments (one graduate mentioned, “[it] felt like it was operating at a pre-school level . . . including fill in the blanks!” [G381]), a considerable number of graduates questioned the actual content of the assessments. Most of these graduates were critical about the “not well written questions” (G123) with “ambiguities in wording” (G269) and “answer options [that] are confusing” (G353) such as “double negative language” (G52). This meant that they perceived the assessment as “a test of grammatical understanding rather than regulatory knowledge” (G52). Some even had a feeling that the aim of the assessment was to trick them into giving faulty answers. As one critical graduate stated, “I think a lot of the questions . . . are poorly worded, designed to trip up the candidate” (G359). Such experiences are particularly problematic because they were shared (sometimes a considerable length of time) after the graduates had completed their Core Knowledge qualification. Their comments are an indicator of how they experienced and remember the assessments. There is a risk of an atmosphere of distrust emerging should too many participants receive these experiences and share them with others. It must be noted, however, that Version 2 of Core Knowledge has addressed a range of reported language issues in the assessments. The above comments concern Version 1 of Core Knowledge.

Learning

Throughout 2019, a total of 1,941 people started Core Knowledge. They were all presented with the mandatory pretraining test described above. The average score received for the pretraining test was 70% (i.e., participants on average answered 70% of the questions correctly and 30% incorrectly), indicating that many participants were not able to answer all questions correctly prior to completing the qualification.⁸ There does not appear to be a clear pattern in the knowledge gaps of the participants prior to taking the training. Incorrect answers were given to all questions (ranging from 16% to 70% of incorrect answers per set of questions). Of the 1,941 people tested, 1,789 successfully completed Core Knowledge in 2019. The remaining 152 participants did not complete their Core Knowledge qualification in 2019. Almost half of the cohort was able to complete all assessments on their first attempt (46% of the cohort). A large minority needed two assessment attempts for one or more of the modules (39%). Only a small part of the cohort needed a third attempt before successfully completing all six Core Knowledge modules (11%; the remaining 4% of the cohort completed the modules in other ways). Although the above numbers at first glance paint a picture of an improved understanding of the concepts and principles taught in the modules (knowledge transfer), we must, again, go behind the numbers to get a fuller picture.

A first set of experiences about learning shared by graduates in the Skills’ survey, particularly those new to working in a regulatory environment and those not directly working in regulatory frontline positions, is that they generally experienced a considerable improvement in their understanding of the concepts, principles, and ideas taught in Core Knowledge. One graduate remarked, “I came from a corporate background. If it were not for G-REG it would have taken twice as long to understand government policy” (G211). Another mentioned, “It has been helpful for my interaction with frontline inspectors and to improve my understanding of their work” (G399), and yet another stated, “I work in communications, so I was doing this to better understand the regulators I support” (G141). Likewise, some graduates with a few years of experience in their jobs also experienced an increase in their knowledge, such as the following: “This course has given me a huge leap forward in my understanding of what is required of me in my role” (G194). Or, “I have worked in the public sector for a number of years, but the qualification further expanded my core knowledge and allowed me to make links that I may not have previously been aware of” (G222). Still, not everyone was as positive as these graduates, and a considerable number of negative comments were also expressed. One graduate stated, “I did not learn anything new . . . [I have] over 20 years [experience] and you make me do this” (G61), and another one stated, “But overall the course [material] was of marginal value for a person with a lot of experience” (G350). These comments point out one of the main challenges faced by G-REG and Skills in developing the Core Knowledge qualification: It is aimed at everyone working in the New Zealand regulatory environment. Thus, some who take the qualification have very little or no knowledge of and experience in a regulatory (frontline) environment, while others have many years of experience. The qualification will be highly beneficial for the former group in terms of acquiring new knowledge but will be—at best—a refresher for the latter group. It goes without saying that some graduates understood this challenge: “Relevant, but I have 40 years regulatory experience, so it was more a way of confirming my currency/competencies than learning [anything] new” (G275).

A related set of experiences was shared in the interviews with managers and supervisors. They explained that, overall, those holding tertiary education degrees questioned the value of Core Knowledge. One of the managers explained, “Why we don’t have a lot of uptake is because a lot of people come in already clutching a bachelor’s degree. [They] don’t see [Core Knowledge] as an add-on” (Interviewee I). Another manager at another agency explained, “When you ask [them] to do something like Level 3, then their reaction is, well, that is not really relevant in my field.” He added that those without much formal training or education did, however, especially appreciate the Core Knowledge qualification: “The return on investment [for them] is that they are getting a recognized qualification . . . which in some cases is likely a higher level of qualifications than some of them have ever achieved” (Interviewee VII). In summary, these interviewees explained, there appears to be two broad groups of participants who feel they learned a great deal from undertaking the Core Knowledge qualification: new entrants to the regulatory sector, often without specialist training in regulation, to whom Core Knowledge (and, for some, the resulting certificate) is a real bonus; and seasoned staff for whom Core Knowledge is a refresher that sometimes confirms their knowledge and sometimes puts things in a broader perspective. The final group, those who reported having learned little from Core Knowledge, consists of mid-career staff, who do not necessarily work in frontline functions, and early career staff with university degrees. One of the managers who was interviewed summed it up: “There may be a bit of snobbery around the Level 3” (Interviewee IV).⁹

Behavior

The next element the Kirkpatrick model addresses is whether on-the-job behavior has changed as a result of the training received by the graduates (and, if so, to what extent and how). In contrast to the first two elements of the Kirkpatrick model explored above (“reaction” and “learning”), no quantitative data are available as a starting point for mapping, exploring, and interrogating changes in reported behavior after training. This is a general challenge when applying the Kirkpatrick model: Pretraining data are rarely available in public-sector organizations (Bates, 2005; Reio et al., 2017). Here, we must rely completely on the experiences shared in the interviews with managers and supervisors (who were explicitly asked questions about observed changes in on-the-job behavior), the G-Reg meetings, and the open questions in the Skills’ survey. However, before we zoom in on these data, it is relevant to recall that Core Knowledge is intended to introduce participants to the basic principles and concepts of regulation and of working as a regulatory practitioner (professionalism in a broad sense), rather than providing them with tailored competencies (professionalism in a narrow sense).

Very few graduates or managers reported observed changes in on-the-job behavior. A typical survey comment by one of the managers is, “I’m not sure that completion of this course (though a worthwhile achievement for the whole team), has had a huge influence on the professionalism of their interactions” (M32), and another stated, “My [staff are] experienced investigators. They were able to do this prior to completing Level 3” (M35). Graduates tended to make similar comments. One of them summed up a shared feeling by stating: “[I] was doing this before the certificate” (G397). Another was more specific: “The certificate provides assurance and gives confidence that one can apply the core knowledge, but I don’t believe one would be able to apply the knowledge having just done the certificate alone, without on-the-job experience” (G287). These experiences were also emphasized in the interviews with managers and supervisors. For example, one of the managers interviewed said, “In my team, I haven’t seen an improvement out in the field. But what they do have is a better understanding of [regulation and regulatory compliance]” (Interviewee I). Another shared the following:

We have to remember [Core Knowledge] is very basic. It doesn’t turn a lay person into a regulator. And that is one danger that I see . . . people think they are a regulator if they have passed the test. That is just as dangerous as having unqualified people. (Interviewee IX)

In summary, graduates and managers had, generally, a realistic view of what this specific qualification may achieve in terms of posttraining behavioral change. Still, they expressed concerns about whether, in the long run, the core knowledge taught would stick and thus achieve lasting change in behavior. For example, one of the managers wrote in the Skills’ survey: “I can foresee some of the acquired knowledge ‘slipping’ if it is not utilised regularly” (M12). One of the graduates made a comment in the Skills’ survey along similar lines: “It would be good to have a reference [document] for ongoing application. Over time people’s memory fades as to what they learnt” (G123). Indeed, the proof of the pudding for Core Knowledge will be whether graduates maintain their newly acquired knowledge, or whether it is diluted over time. The outcomes of our (voluntary) posttraining test indicate that their newly acquired knowledge may wane—and may do so rather quickly. The total number of participants who filled out the posttraining test is low (23 participants), and the findings should therefore be treated with some care. On their pretraining test, this cohort of participants answered, on average, 67% of the questions correctly. On their posttraining test, they answered, on average, 86% of the questions correctly (indicating a 21-percentage point test score increase, or a relative improvement of 28%). Although this is a promising outcome, one must recall that to complete the qualification, participants must have a perfect (i.e., 100%) score for each module. Thus, between completing their first module and taking the posttraining test only a few months later (often less than 6 months), this cohort of participants was unable to retain the perfect test scores required for completing the qualification.

Results

The final element that must be evaluated in the Kirkpatrick model is whether changes have occurred at the organizational level (or beyond) as a result of the training received by the graduates (and, if so, to what extent and how). As with the previous element (behavior), it is generally considered challenging to measure this element of the Kirkpatrick model—a direct causal relationship between observed organizational change (if any) and training of staff cannot often be evidenced (Bates, 2005; Reio et al., 2017). The answers to the open questions in the Skills’ survey and the interviewee transcripts do not provide clear examples or a general sense of improvements at the organizational level or beyond as a result of the Core Knowledge qualification (bearing in mind that, at the time of evaluation, approximately a quarter of the staff in regulatory functions had completed the qualification). In part, this may be because G-REG seeks organizational change and change in the New Zealand regulatory community through a range of initiatives, with Core Knowledge being just one of these. In part, also, it may simply take a long time for results to be experienced. As one manager aptly summarized in an interview: “It is too early to really see the benefits yet” (Interviewee I). However, that is not to say that the Core Knowledge qualification (and the suite of qualifications of which it is part) has had no observable impact at organizational levels and at the more systemic level of regulatory governance in New Zealand.

A higher-level change mentioned in the interviews and in G-REG meetings is the sense of pride of some graduates as a result of completing Core Knowledge. One of the interviewed managers commented, “We notice people are proud of it. They mention it in their job applications. They say, oh, I have done Level 3” (Interviewee II). Another manager explained,

We are seeing an increasing number of people [from other agencies] that apply for jobs [in our agency] that have the Core Knowledge qualification. It has some currency . . . if you are competing with somebody relatively equal, and you have the qualification and the other has not. (Interviewee IV)

Managers and supervisors generally appreciated Core Knowledge for creating a base level of regulatory knowledge across government staff. It gave them some measure of the quality of staff knowledge. Managers at different agencies mentioned, “From a management perspective, it is good to know that people know their stuff” (Interviewee II), and “We know, as an employer, what knowledge you have been exposed to” (Interviewee IV). Managers and supervisors indicated that it is not currently common practice to ask for the Core Knowledge qualification when hiring new staff, or to make this part of the performance criteria for existing staff, but they said that such practice may develop over time. It is likely, then, that over time, a virtuous circle will develop, where more staff have completed Core Knowledge, allowing agencies to make Core Knowledge a preference when recruiting new staff, which will also send a message to nongraduates to obtain the qualification. These comments were underlined by other interviewees, who indicated that after a relatively slow initial uptake in their agencies, more staff were becoming interested in completing Core Knowledge. An illustrative example is the following:

I’m starting to find now that those who didn’t take the initial opportunity [see that] their peers are getting qualifications handed over by their supervisors. We get them framed and everything and hand them over. And they are going, oh, I want to be a part of it. (Interviewee VII)

This latter example, however, also underlines a broader shared experience across interviewees and in G-REG meetings. Agencies must actively champion Core Knowledge, and they must be clear about the value of Core Knowledge when promoting it to their staff. As one of the interviewees from an agency with a relatively low number of graduates explained, “It is now just sitting on our list of available training. It is not being actively promoted” (Interviewee I).

A second higher-level change mentioned in surveys and interviews is that Core Knowledge has raised questions in the regulatory agencies (across local and national government) about in-house and on-the-job training. Such training was not a topic of much discussion in government agencies prior to the 2014 Productivity Commission report discussed above. Obviously, Core Knowledge is appreciated by many agencies, as it has eased the burden of them having to develop in-house training. One of the managers explained, “Before G-REG we had to go out and find all the individual bits and pieces” (Interviewee II). Even for those agencies that challenge the value of Core Knowledge for their staff (perhaps because they feel it is too general, or not targeted at the right level), the qualification (and the broader suite of qualifications) has set a benchmark against which they can evaluate their own in-house learning and development pathways.

Discussion

The above analysis set out to map, explore, and interrogate G-REG’s involvement in improving the professionalism of public service delivery across a range of regulatory agencies. The G-REG experience is of particular interest to public administration scholars and practitioners because of its strong focus on both of the elements that the literature identifies as necessary for nurturing professionalism in a community of practitioners: the development of a shared body of specialized knowledge, and intensive training of people in that knowledge. The G-REG experience helps to give a better understanding of whether (and, if so, how) a standardized program, such as the G-REG qualifications framework, can help to improve the professionalism of a community of public servants. In addition, the application of the Kirkpatrick evaluation model helps with the careful unpacking of the (experienced) differences in how the qualifications framework has helped (or failed) to increase narrow public service professionalism and broad public service professionalism.

The key finding from the first level of the analysis (“reaction”) is that the aim of the Core Knowledge qualification is not immediately clear to many participants (or their managers and supervisors). This lack of clarity affects the performance the program. That is, many learners expected that Core Knowledge would help them to obtain specialist or technical knowledge suitable for their day-to-day jobs; in other words, they expected Core Knowledge to raise their professionalism in a narrow sense. G-REG and Skills have, however, developed Core Knowledge with the particular aim of exposing learners to a minimum set of shared knowledge, challenging them to reflect on their day-to-day work, and considering the broader ethical challenges of being a regulatory practitioner—in other words, to raise the professionalism of regulatory practitioners in a broad sense. Insights from the second level of analysis (“learning”) confirm this key finding. Those who had approached Core Knowledge as an opportunity to gain new knowledge that was directly applicable in their day-to-day job (narrow professionalism) were highly critical of the program, and were sometimes even offended by it. However, those who had treated Core Knowledge as a refresher of their broad knowledge, or as an introduction to working in public service as a regulatory practitioner (broad professionalism) were generally highly positive about the program. This finding was validated in interviews with managers and supervisors, and was further refined by the insight that a particular cohort of learners had approached Core Knowledge to improve their professionalism in a narrow sense and were disappointed about the program’s focus on professionalism in a broad sense (recall, this is the cohort of mid-career staff who do not necessarily work in frontline functions, and early career staff with university degrees).

The third level of analysis (“behavior”) reinforces these findings. Learners, managers, and supervisors did not, generally, experience an increased level of narrow professionalism in the posttraining behavior of graduates. They also wondered whether a one-off training program such as Core Knowledge is capable of increasing the level of broad public service professionalism of regulatory practitioners. In the interviews, managers and supervisors acknowledged that within G-REG, there are ongoing possibilities for Core Knowledge graduates (and others) to engage in activities that help to raise their broad public service professionalism. No insights were shared, however, as to whether Core Knowledge graduates are more likely than others to participate in these activities. Thus, up to this point, the analysis raises questions about what a standardized training program can and cannot achieve in terms of nurturing and increasing public service professionalism. G-REG may very well have been too ambitious in seeking to increase both the narrow and the broad professionalism of regulatory practitioners; it may very well have marketed the qualification too strongly as something that would help learners to increase their technical knowledge (narrow professionalism) and too weakly as something that would expose learners to what it means to be a professional regulator (broad professionalism), or it may have done both. That being said, the fourth and final level of analysis (“results”) provides a silver lining. G-REG’s persistence in offering and improving the program appears to have paid off: A cohort of graduates takes pride in having completed Core Knowledge, regulatory agencies see merit in having a standardized program for their staff (or, at least, a standardized induction for their new staff), and managers and supervisors are beginning to choose job applicants who have completed the certificate over those who have not.

Lessons for Scholars of Public Administration

For scholars of public administration, the G-REG experience is of interest, as it is one of the few documented efforts to increase broad public service professionalism. With the shift to paradigms such as New Public Management, New Public Governance, and Post-Bureaucracy, scholars have extended their calls for an increase in the technical skills and knowledge of public servants (the narrow professionalism called for by Wilson, Waldo, Simon, and their contemporaries) with calls for proficient and ethical public servants (the broad professionalism currently demanded). Although this development makes sense historically and has great normative appeal, the G-REG experience indicates that the scholarly community may need to provide more guidance on how public service professionalism (specifically in a broad sense) can be nurtured and increased. This logically results in a renewed call for empirical research (along the lines of the earlier research of Wilson, Waldo, Simon, and their contemporaries) into real-world examples of training and of other approaches to nurturing public service professionalism, to understand which approaches are promising, where, and why. The research presented here has pointed to the relevance of distinguishing between different types of professionalism when studying training and other programs, and the relevance of distinguishing between what professionalism means for those inside the workforce and those outside of it (Evetts, 2013). A related area for future empirical research into public service professionalism could be the forms and types of learning that result from training and other programs. This article has briefly touched on the value of joint-learning processes (learning as a team or cohort rather than learning individually), but more research is necessary to understand how deep learning can be achieved, whether this calls for a staged learning process (as per the G-REG program) or parallel processes, and whether there are optimal meta-learning practices for achieving deep learning (Wolfram et al., 2019).

In a similar vein, the research presented here challenges us to think about the role of professionalism within (some of) the major paradigms in public administration. Arguably, the New Public Management mainly expects public servants in the higher echelons of organizations to become more proficient in managing their teams and organizations—a focus on increased cost-effectiveness and allocative efficiency. This asks for an expansion of their narrow professionalism (i.e., additional training in management and economics) but not necessarily an expansion of their “broad professionalism.” The New Public Governance asks public servants across the public sector to be more than agents of instrumental government efficiency and embrace a much broader set of values—such as equity, equality, and civic capacity. Naturally, this calls for an expansion of their broad professionalism (i.e., training in reflexivity, mediation, capacity building, etc.) but not necessarily an expansion of their narrow professionalism. Finally, Post-Bureaucracy envisions a decentralized and nonhierarchical public service guided by shared values and visions. Arguably, this paradigm asks the most of public servants as professionals. They need to master a broad set of technical skills and expertise (i.e., their own profession complemented with, among others, relevant management and economic knowledge) to be able to operate effectively in a decentralized setting, and they need to understand and adhere to a wide range of public values to be able to deliver public services in a manner that enhances the capacity of service receivers to become self-authoring (Torfing et al., 2020). In sum, in the balancing act of merging narrow and broad professionalism, New Public Management may tip the balance toward narrow professionalism at the expense of broad professionalism; New Public Governance may tip the balance toward broad professionalism at the expense of narrow professionalism; and Post-Bureaucracy may strike the right balance but at the expense of asking more of public servants than what they can reasonably deliver.

Lessons for Practitioners (and Those Advising Them)

For practitioners (those delivering public services, managing public service delivery, or involved in the design of policy for public service delivery), a core lesson is that first impressions and clarity about the aims of a standardized program matter when setting off on the road toward improving the professionalism of public servants. We have seen that the Core Knowledge qualification was, overall, well received, but a substantial cohort of graduates remains critical about the online provision of learning material and, particularly, the online evaluations. These issues have been largely addressed in the revised version of Core Knowledge, but the initial challenges experienced by this cohort of graduates appear to have had a lasting impression (as reflected in their responses to the Skills’ survey). Combined with the critique of another cohort of graduates that Core Knowledge did not provide them with new knowledge or capabilities, there may be a risk that those who have undertaken Core Knowledge and are critical will not champion it within their respective agencies and, instead, may create an atmosphere that questions why public service staff would invest their time in the (at present, voluntary) qualification at all. Thus, it seems essential to be very clear, at the start of and throughout these types of programs, about their aim and the type of knowledge they seek to deliver. In addition, it seems essential to be responsive to criticism that is received and to be transparent in how this criticism is addressed, perhaps even marketing revised versions of learning material more actively to ensure that graduates of earlier versions know that improvements were made.

In terms of seeking improved public service professionalism through in-service training, some additional hands-on lessons stand out. While the Core Knowledge qualification is a laudable initiative, it is fair to question whether a lasting effect can be expected from guiding public service staff through a one-off training program. The classic and contemporary literature on learning and training has often stressed that training to change on-the-job behavior should be an ongoing process (cf. Bates, 2001), and feedback from graduates of Core Knowledge and their managers supports this view. Another insight from the literature that is supported by this study is that learning as an active experience is preferable to learning as a passive experience (cf. Sims, 1993a). Graduates and their managers had a clear preference for the delivery of learning material in tailored in-house programs that created a bridge between the more general Core Knowledge learning material and the local agency context and working environment of learners (cf. Bluestone et al., 2013). Such contextualization of general learning material was experienced as a meaningful way to emphasize the relationship between theory and practice (cf. Dwivedi & Engelbert, 1981). Of course, Core Knowledge is the entry point to a much broader suite of qualifications, and the recently introduced continuous professional education program provides graduates with an opportunity to keep their knowledge current. Still, it seems essential to consider programs of this kind as part of a process of improving the knowledge, attitudes, and values of those who work in or pursue a career in public service, rather than an endpoint in itself (cf. Chen et al., 2019; Rosenbloom, 1984). In sum, nurturing narrow public service professionalism may be possible with these kinds of standardized programs, but to achieve broad public service professionalism, a much larger set of activities is required.

Conclusion

The case presented provides lessons for both scholars and practitioners of public administration. These lessons all come back to the practical challenges of seeking to nurture and increase public service professionalism in a broad sense through a program that, essentially, is a traditional mode of technical knowledge transfer (narrow professionalism). This is the challenge of balancing narrow and broad public professionalism—a challenge that is, perhaps, too easily overlooked in the popular literature on New Public Management, New Public Governance, and Post-Bureaucracy.

Besides the lessons for public administration theory and practice presented before, the case raises a range of normative questions for public administration that I like to conclude this article with. The G-REG experience challenges us to think about what society expects of professionals in public service, and whether these expectations are (justifiably) similar to or different from what society expects of professionals in the private sector. Can society expect a similar level of professionalism, or should it ask for a higher level? If the latter, how do we envisage a professional public service? To what extent do the ethics and norms of the professions of those working in public service (such as city planners, accountants, and lawyers) conflict with the ethics and norms of a professional public service, and how can these conflicts be solved? Does society ultimately need (and do paradigms such as New Public Management, New Public Governance, and Post-Bureaucracy ultimately ask for) a profession of public service or parts thereof (e.g., a profession of public regulation, or a profession of public welfare workers)? If society needs a profession of public service, should this meet all the characteristics of traditional professions (including, but not limited to, a shared body of specialized and applied knowledge; shared ethics and a grounded culture; community sanctions against violations of professional ethics; a representative institution or organization; intensive training and formal qualification for entry into and development within the profession; continuing education requirements; and a career development system for members of the profession)? Answering these questions about what public service professionalism (and perhaps even a profession of public service) can or should look like is as important as understanding how to achieve that desired state of affairs (Noordegraaf, 2013).

To conclude, building on the research presented here, it will come as little surprise that I argue that public service professionalism requires more than staff with expert knowledge (narrow professionalism, as per the New Public Management) and that it requires more than shared expectations about acting proficiently and ethically (broad professionalism, as per the New Public Governance). A high-quality public service requires the nurturing and improvement of professionalism in a narrow and a broad sense (balancing narrow and broad public service professionalism, as per Post-Bureaucracy). The G-REG initiative indicates that it is not easy to address these understandings of professionalism simultaneously, but, at the very least, it also indicates that it is not impossible to pursue this ambition. Still, in pursuing this ambition, we need to be careful to not ask more of public servants than what they can reasonably deliver.

Footnotes

Acknowledgements

The author wishes to thank all interviewees, the New Zealand Government Regulatory Practice Initiative (G-REG), and Skills for their support.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jeroen van der Heijden

Notes

Author Biography

Dr. Jeroen van der Heijden is a Professor of Public Governance and Chair in Regulatory Practice at the Victoria University of Wellington, and an Honorary Professor at the Australian National University. He works at the intersection of public governance and regulation, with a specific interest in regulatory stewardship and dynamic governance regimes. Another area of his research concerns urban climate governance.

References

Bates

R. A.

(2001). Public sector training participation: An empirical investigation. International Journal of Training and Development, 5(2), 136–152.

Bates

R. A.

(2005). Kirkpatrick four-level evaluation model. In Mathison

(Ed.), Encyclopedia of evaluation (pp. 221–222). SAGE.

Bilmes

L. J.

Gould

W. S.

(2009). The people factor. Brookings Institution Press.

Bluestone

Johnson

Fullerton

Carr

Alderman

BonTempo

(2013). Effective in-service training design and delivery: Evidence from an integrative literature review. Human Resources for Health, 11(1), 1–26.

Bøgh Andersen

Holm Pedersen

. (2012). Public service motivation and professionalism. International Journal of Public Administration, 35(1), 46–57.

Brady

H. E.

Collier

(Eds.). (2004). Rethinking social inquiry. Diverse tools, shared standards. Rowman & Littlefield.

Budd

(2006). Post-bureaucracy and reanimating public governance. International Journal of Public Sector Management, 20(6), 531–546.

Caldwell

L. K.

(1962). Improving the public service through training. Agency for International Development.

Chen

C.-A.

Hsieh

C.-W.

Chen

D.-Y.

(2019). Can training enhance public employees’ public service motivation? A pretest–posttest design. Review of Public Personnel Administration. Advance online publication. https://doi.org/10.1177/0734371X19872244

10.

Clarke

(1998). The challenges facing public service training institutions. International Review of Administrative Sciences, 64(3), 399–407.

11.

Clarke

(2001). The impact of in-service training within social services. British Journal of Social Work, 31(5), 757–774.

12.

Corson

J. J.

(1963). Equipping men for career growth in the public service. Public Administration Review, 23(1), 1–9.

13.

De Beelde

. (2002). Creating a profession “out of nothing”? The case of the Belgian auditing profession. Accounting, Organizations and Society, 27(4), 447–470.

14.

Dwivedi

O. P.

Engelbert

E. A.

(1981). Education and training for values and ethics in the public service: An international perspective. Public Personnel Management, 10(1), 140–145.

15.

Dyer

(2018). Being a professional or practising professionally. European Early Childhood Education Research Journal, 26(3), 347–361.

16.

Evetts

(2013). Professionalism: Value and ideology. Current Sociology, 61(5–6), 778–796.

17.

Fantham

Kale

Manch

McGirr

Mumfort

Raj

(2020). Professionalizing regulatory practice within the public service: Lessons from the New Zealand G-REG Initiative. In Sullivan

Dickinson

(Eds.), Handbook of the public servant (pp. 1–19). Palgrave.

18.

Goertz

Mahony

(2012). A tale of two cultures. Princeton University Press.

19.

Government Regulatory Practice Initiative. (2019). Qualifications. https://g-reg.govt.nz/qualifications/

20.

Greenwood

(1957). Attributes of a profession. Social Work, 2(3), 45–55.

21.

Grugulis

Vincent

(2009). Whose skill is it anyway? “Soft” skills and polarization. Work, Employment & Society, 23(4), 597–615.

22.

Johnson

Wood

Brewster

Brookes

(2009). The rise of post-bureaucracy. International Sociology, 24(1), 37–61.

23.

Kirkpatrick

D. L.

Kirkpatrick

(2005). Transferring learning to behavior. Berrett-Koehler.

24.

Kirkpatrick

Kayser

(2016). Kirkpatrick’s four levels of training evaluation. Association for Talent Development Press.

25.

Kolb

(2015). Experiential learning: Experience as the source of learning and development. Pearson Education.

26.

Lauer Schachter

. (2007). When political science championed public service training. The American Review of Public Administration, 37(3), 362–375.

27.

Lee

(1991). Professional foundations and theories of professional behavior. Accounting Historians Journal, 18(2), 193–203.

28.

Lipsky

(1980). Street-level bureaucracy: Dilemmas of the individual in public services. Russell Sage Foundation.

29.

Manch

(2014). Improving the implementation of regulation: Time for a systemic approach. Policy Quarterly, 10(4), 17–22.

30.

May

Winter

(2009). Politicians, managers and street-level bureaucrats: Influences on policy implementation. Journal of Public Administration Research and Theory, 19(3), 453–476.

31.

New Zealand Government. (2015). Government response to the New Zealand Productivity Commission report on regulatory institutions and practices.

32.

New Zealand Productivity Commission. (2014). Regulatory institutions and practices.

33.

New Zealand Treasury. (2017). Regulatory timeline. https://treasury.govt.nz/information-and-services/regulation/intro-regulatory-management/regulatory-timeline

34.

Nhede

(2018). The Fourth Industrial Revolution and novel skills needed for the public sector. In de Vries

M. S.

van Dijk

Chitiga-Mabugu

(Eds.), Public administration: Reflection, relevance and readiness (pp. 201–215). The Network of Institutes and Schools of Public Administration in Central and Eastern Europe.

35.

Noordegraaf

(2013). Reconfiguring professional work: Changing forms of professionalism in public services. Administration & Society, 48(7), 783–810.

36.

Organisation for Economic Co-operation and Development. (1997). Public service training in OECD countries [SIGMA paper].

37.

Organisation for Economic Co-operation and Development. (2017). Skills for a high performing civil service.

38.

Osborne

Gaebler

(1992). Reinventing government: How the entrepreneurial spirit is transforming the public sector. Addison-Wesley.

39.

Ospina

(1996). Illusions of opportunity: Employee expectations and workplace inequality. Cornell University Press.

40.

Peck

(2009). Celebrating the birth of a profession. Journal of Financial Planning, 2009(1), 6–7.

41.

Pollack

J. B.

Alego

(2015). Should project management aspire to be an archetypal profession: Evidence from Australian-based research. International Journal of Project Organisation and Management, 7(4), 352–368.

42.

Prier

McCue

Behara

(2010). The value of certification in public procurement: The birth of a profession? Journal of Public Procurement, 10(4), 512–540.

43.

Raadschelders

(2011). Public administration: The interdisciplinary study of government. Oxford University Press.

44.

Reio

Rocco

Smith

Chang

(2017). A critique of Kirkpatrick’s evaluation model. New Horizons in Adult Education and Human Resource Development, 29(2), 35–53.

45.

Roller

Lavrakas

(2015). Applied qualitative research design: A total quality framework approach. Guilford Press.

46.

Rosenbloom

D. H.

(1984). Public administrative professionalism and public service law. State & Local Government Review, 16(2), 52–57.

47.

Russell

Hodges

(Eds.). (2019). Regulatory delivery. Hart Publishing.

48.

Saks

(2012). Defining a profession: The role of knowledge and expertise. Professions & Professionalism, 2(1), 1–10.

49.

Sherwood

F. P.

(1997). Responding to the decline in public service professionalism. Public Administration Review, 57(3), 211–217.

50.

Silverman

(2001). Interpreting qualitative data (2nd ed.). SAGE.

51.

Simon

H. A.

(1997). Administrative behavior: A study of decision-making processes in administrative organization. Free Press. (Original work published 1945)

52.

Sims

R. R.

(1993a). The enhancement of learning in public sector training programs. Public Personnel Management, 22(2), 243–255.

53.

Sims

R. R.

(1993b). Evaluating public sector training programs. Public Personnel Management, 22(4), 591–616.

54.

Skills. (2017a). Growing momentum: Annual report 2017.

55.

Skills. (2017b, 3 July). New qualifications mark a “sea of change” for regulatory practice. https://skills.org.nz/blog/new-qualifications-mark-a-sea-of-change-for-regulatory-practice/

56.

Tapper

Millett

(2015). Revisiting the concept of a profession. In Beard

Lynch

(Eds.), Conscience, leadership and the problem of “dirty hands” (pp. 1–18). Emerald Group.

57.

Torfing

Bøgh Andersen

Greve

Klausen

(2020). Public governance paradigms: Competing and co-existing. Edward Elgar.

58.

Waldo

(2007). The administrative state: A study of the political theory of American Public Administration. Routledge. (Original work published 1948)

59.

Walker

M. A.

(2015). The professionalisation of data science. International Journal of Data Science, 1(1), 7–16.

60.

Wang

Burgess

Xiao

(2019). Literature analysis of the evaluation of public training programmes in the USA, Europe and China: Implications for the evaluation of farmer training programmes in China. Pacific Economic Review, 24(1), 69–91.

61.

Wilson

(1887). The study of administration. Political Science Quarterly, 2(2), 197–222.

62.

Wilson

(1893, July 25–28). Should an antecedent liberal education be required of students in law, medicine, and theology? [Paper presentation]. The International Congress of Education, Chicago, IL, United States.

63.

Wolfram

Van der Heijden

Juhola

Patterson

(2019). Learning in urban climate governance: Concepts, issues and challenges. Journal of Environmental Policy & Planning, 21(1), 1–15.