Abstract
This paper reports on an evaluation of a national action learning set for health policy managers from three Australian state/territory health authorities, conducted during 2010–2011. We collected and analysed qualitative data about the major problems the participants encountered in their work, their experiences of the learning set and their perceptions of the outcomes. The predominant concerns of participants were ‘wicked problems’ in four areas: managing the environment, managing the policy process, managing the self and managing the policy team. Participants reported that the learning set had assisted them to gain greater awareness of and ability to navigate their environment, developed their judgement, strategic and problem-solving skills, contributed to empowerment and self-efficacy, and assisted them in providing support for their staff and building capacity in their organisation. Aspects of the method that contributed to learning included engaging with problems experienced by other members and discussing one’s own problems and strategies; and gaining new insight into ways of framing, analysing and responding to problems. The findings suggest that learning sets can be a powerful method for building policy capacity amongst middle-senior policy managers.
Introduction
This paper reports the findings of an evaluation of a national action learning set for health policy managers. The learning set was conducted during 2010–2011 with eight policy managers from three Australian state/territory health authorities: Queensland Health, SA Health and ACT Health. The learning set was one of the several strategies trialled as part of a larger project to explore ways of ‘building’ policy capacity in the participating authorities.
Policy capacity
The term ‘policy capacity’ refers to the capacity of governments to make intelligent, collective decisions and to effectively implement them.1–7 Policy capacity includes the competencies of policy workers (knowledge, skills and personal attributes) and organizational policy capacity (the structures, processes and cultures that support ‘good’ policy making).7,8 While policy capacity also has system-wide dimensions, 9 our focus in this paper is on policy capacity at the levels of the policy workers and the organisations in which they work.
Recent empirical research, such as Page and Jenkins’ 10 study of middle-ranking officials in the United Kingdom, has highlighted the role of bureaucratic policy workers in determining policy outcomes, in comparison with common conceptions of bureaucrats as merely implementing the decisions of elected leaders and senior executives. Howard’s study of the former Commonwealth Department of Social Security in Australia found that senior administrators played a strategic role in actively influencing and garnering support for the policy they were working on. 11 An empirical study of Australian policy workers by Gill and Colebatch 12 found that they frequently conceived their roles as working strategically to achieve an agenda and frame issues.
A study of policy capacity in a health authority in the Australian state of Victoria by Gleeson 13 identified the individual competencies that contribute to policy capacity. These included specific knowledge sets (e.g. knowledge of the health system, the departmental environment and the policy process). They also included a range of skill sets, from conceptual and analytic skills, strategic and political skills and judgement, through to more practical skills in communication and project management. Gleeson also identified a number of personal attributes that contributed to policy capacity, such as openness to different ideas and the ability to balance conflicting accountabilities.
Gleeson 13 identified eight ‘domains’ of organisational policy capacity: utilising information and evidence; people management; managing stakeholder relationships; managing intra-portfolio, cross-portfolio and inter-governmental relationships; working between policy development and program management; policy evaluation and monitoring; managing the policy process; and leadership. Gleeson et al. 8 found that policy workers struggle with tensions inherent in competing priorities and rapidly fluctuating policymaking environments, and that developing judgement and leadership at the level where policy work is undertaken is key to successfully negotiating these tensions.
Based on Gleeson’s work in Victoria, a subsequent research project was undertaken involving health authorities in three Australian states and territories. The first stage of this research involved defining the challenges in undertaking policy work and describing, evaluating and comparing health policy capacity in the three jurisdictions. Stage two involved developing, implementing and evaluating a range of interventions to enhance policy capacity.
Findings from the first stage 14 pointed to shared challenges, despite differences in organisational size and structure and the key policy concerns of the day. The policy competencies identified as needing development were largely ‘soft’ skills such as contextual knowledge, communication skills, personal attributes and political astuteness. Senior managers emphasized competencies that would enable policy workers to cope with short time frames, the vagaries of the political environment and less than ideal scenarios for policy making. The organisational barriers identified in this stage largely confirm the findings of previous research.
Action learning sets and their use in the health sector
Action learning is ‘a continuous process of learning and reflection, supported by colleagues, with an intention of getting things done’. 15 An action learning set is a group of people who work together for a period of time, using reflection on action to learn from their experiences. 15
Action learning sets are essentially an individual learning approach, but working with individuals on organisational issues contributes to both individual and organisational development. 15 Learning set members can take more effective action on organisational issues and apply their action learning skills in other aspects of their work, thus contributing to organisational learning.15,16
Action learning is particularly useful for developing leadership skills, including the ‘lateral influencing skills’ necessary for leading at levels other than the top positions in the hierarchy. 16 Marquardt 16 argues that action learning fosters the development of all of the important leadership skills: cognitive, execution, relationship and self-management skills.
A typical learning set has four to eight participants. 16 Sets usually last for a period of between 6 months and 3 years with a cycle usually lasting 6–12 months. 15 Each set member is allocated time in each meeting. 15
The literature suggests that action learning contributes to empowerment, by encouraging learners to take responsibility for their actions and interactions and for finding solutions to their own problems. 15 Findings from two published evaluations of action learning sets for health sector managers suggest that this is a useful approach for supporting more effective management practice in a range of settings.17,18
Leggat et al. 18 evaluated year-long learning sets in which 137 Australian health care managers participated. They measured psychological empowerment, self-efficacy and psychological safety using internationally accepted instruments administered before and after the intervention. Participants reported significantly higher levels of empowerment and self-efficacy after participating in the learning sets. They reported that ‘… their work was more meaningful; they were more confident about their ability to do their job and solve problems; they were more self-assured about their capabilities; and they had greater opportunity to decide how to do their work’.18(p.200)
In this study, the use of action learning sets for health policy managers has been reported for the first time. In this paper, we examine the way in which middle to senior level policy managers discuss and resolve complex and difficult problems in action learning sets. We are particularly interested in the unstructured policy problems that governments find hard to manage and resolve, defined by Hisschemöller and Hoppe 19 as ‘wicked’ problems. We suggest that it is the capacity to analyse and manage the wicked problems that is critical for success in more senior levels of policy work; and that it is these problems that set members typically bring to learning set meetings.
Methods
This study used the analysis of two data sets, the results of discussions in a learning set (to identify the nature and type of wicked problems), and the reflections of the participants after its completion regarding the ways in which the set had assisted them (to assess whether they believed the set had helped their capacity to deal with those wicked problems).
The learning set was conducted during 2010–2011 with eight policy managers from three Australian state/territory health authorities. Participants were middle- to senior-level policy officers or managers with significant policy responsibilities, selected by senior managers in their departments. The learning set met bi-monthly (for a full day) with a facilitator over a 10-month period. Meetings rotated between the participating jurisdictions. Prior to the day of each meeting, participants and the facilitator met informally in the afternoon/evening and shared dinner together. No participants withdrew from the learning set. One participant changed workplaces but stayed in the learning set.
During learning set meetings, participants were asked to present a current real unresolved problem in their policy work. Group members listened to the ‘story’ of the problem, then asked questions for clarification. Supported by the facilitator (JD), they suggested ways of approaching or resolving the problem, and debated the merits of various analyses and strategies. After the discussion, the participant presenting the problem was asked to outline their intended course of action; and to report back at the next meeting on their actions and the results.
The problems were selected by the participants on the basis that they had been unable to solve them in the normal course of their roles, using the skills, knowledge and resources at their disposal – these are the ‘wicked’ problems for this group of policy managers. As a group, the nature of the problems, and the conditions that cause them or undermine their solutions, can provide insight into the difficulties of policy work in practice, and the characteristics of the roles and environmental factors that impede its effectiveness.
Notes of the 37 problem stories, taken by the facilitator during meetings, were analysed. The nature of the problem, and the major factors that contributed to it or undermined its resolution, were articulated. The problem elements articulated in this process were analysed for common themes (by JD), and categorised inductively. The data and the preliminary results were reviewed by the learning set members (who each had the code for their own stories, but not those of others). Their responses verified the accuracy of the analysis.
At the completion of the learning set, participants completed a short qualitative survey, asking them to describe what they had gained from their participation and to comment on the method and processes of the learning set. All participants completed the survey. The survey data were analysed independently (by DG) using the categories derived from the problem story themes. The two data sets were then considered together, and the ‘match’ between the problems and the perceived impact of the set was assessed.
Ethics approval was granted by the La Trobe University Human Ethics Committee (reference no. 07-131). Ethics committees at each of the health authorities and at the other participating universities also approved the project. Informed consent was obtained from participants according to ethics committee requirements.
Results
Overall, there was a strong match between the problems and the learnings. As expected, the problems were those for which technical skills and knowledge were necessary but not sufficient, and where environmental factors (the ‘givens’, outside the control of the workers), and factors in the policy processes (generally, those where the worker had some influence but not command-and-control authority) dominated. The participants reported benefits in the enhancement of capacities like self-efficacy, judgment, self-awareness and the enactment of leadership and relationship skills.
Problem analysis
The eight learning set participants were all managing policy units or teams. Five had responsibilities that included managing major projects or the implementation of health programs. All were women. A total of 37 ‘problem stories’ were presented and discussed out of a potential total of 40 – representing a participation rate of 92.5%.
Problem types and examples.
The problems policy managers brought to the learning set challenged them across the full range of policy and management competencies, although only rarely was the difficulty seen to arise from lack of technical skills or knowledge.
Environmental factors
Many of the problems brought to the learning set concerned difficulties in managing the policy environment – that is, factors that were ‘givens’ in the context, and not amenable to change by the participant or their close superiors. Participants had a strong sense of the structural barriers imposed by their working environments, but were actively looking for strategies around them, for themselves and others. It is to be expected that staff at any level will hold superiors responsible for problems that can only be addressed or managed at more senior levels, and perhaps sometimes overestimate the authority available to senior leaders, so the extent of perceived leadership failure is not surprising.
The policy process
Many problems participants were dealing with concerned management of particular policy development and/or implementation processes. The extent of engagement with program implementation problems is also unsurprising given that participants’ roles included operational responsibilities. Participants often reported being caught between top-level policy agenda drivers (e.g., political commitments) and the reality of program implementation which has its own timelines, imperatives and stakeholders. There was a strong sense that the participants were at an interface between executive/senior management and the exigencies of getting both policy and implementation tasks done, and had deep insight into contradictions they faced.
The extent of concern about the effectiveness of external relationships – horizontally across the health system and vertically to political levels – may be particularly strong for policy managers (as opposed to those in operational roles) because they need to use lateral influencing skills to effect change, rather than relying on positional authority. 16
Managing the self
Discussion about individual career management was a feature of the learning set, and participants valued the opportunities to focus on their own careers in a supportive and well-informed group. Concerns about workload management were largely related to down-sizing or recruitment challenges in policy units. For several participants, the need to develop their confidence to address problems with difficult interpersonal and political aspects was a feature of discussion.
Managing the policy team
The final broad category of problems related to managing the people one supervises. This is usually the most challenging aspect of any management role, and all participants had this responsibility. However, the focus on problems in enabling staff to ‘see the big picture’ and read it correctly is more specific to policy work (cf. [17, p. 21]). The lack of capacity was seen to result in the inability to progress complex work without close supervision. This problem was recognised by all members and energetically discussed, but there was a lack of confidence in their capacity to address it. It seems that there is some tacit knowledge about how to analyse and frame complex policy issues (developed through experience) that is difficult to communicate to one’s staff.
Impact evaluation
At the conclusion of the learning set, participants were asked to identify what they had gained from their participation, and what they had learned about the issues their learning set explored. Their responses are analysed below.
Managing the environment
Many participants described having gained insights into others’ working environments from listening to their stories, and reflecting on their own experience in a new light. Examples include: insights into the politicisation of policymaking and the importance of good leadership; and learning about the impact of organisational re-structures.
It was helpful to know the grass isn’t greener anywhere else and that the environments we work in are by nature complex and difficult. (P1)
Now I have got a greater awareness of the challenges and the incredibly complex environment we work in … (P2)
Several participants reflected on how important judgement, experience and ‘plain gut instinct’ (P2) were in policy work. Sometimes this meant ‘… knowing when to duck your head and when to put it up …’ (P2).
These skills were seen as vital for policy management. As one participant explained, ‘… if you can’t get the work on the agenda and completed within the recommended timeframes and manage the politics in the process, your content skills may account for very little’ (P8). She referred to these skills, which she characterised as ‘leadership, strategic thinking, managing conflict, negotiating difficult politics and managing up’ as ‘enabling skills’.
One policy manager explained that her ‘knowledge of the tactics that can be used in organisations to achieve strategy’ (P5) had improved. She described gaining from the learning set ‘… advice on how to manage challenging staff in a policy context, how to manage up executives in this context, and how to engage with challenging stakeholders in the health sector’. (P5)
Managing the policy process
Some participants reported being better able to exercise leadership for sound policy process. This included clarity regarding goals, processes, networking and relationship building, and a measure of persistence and judgement.
Learning set members pointed out that it was not content or technical skills or ‘the fundamentals of public policy making’ (P4) that they gained. Rather, the learning set was seen to add to ‘an existing armoury of strategic skills’ (P8). A relatively junior member commented that she had gained a greater awareness of thinking at an ‘executive level with regard to policy development and implementation’, which she described as ‘navigating systems, dealing with power brokers and gate keepers, establishing clarity and scope on initiatives as well as clear governance’. (P2)
Several participants reported learning to approach things differently, to identify the real issue, and to apply problem-solving approaches to seemingly intractable policy problems. One learning set member described this as ‘sifting through the pros and cons of various approaches until you arrive at a solution tailored to your/your agency requirements … using a structured approach to hone in on the actual rather than perceived issue or problem’. (P7). She had applied these new skills to problematic policy development projects and to more general issues such as managing a new policy team. She described learning that ‘problems presented in the group that may seem insurmountable do have some sort of solution’, even when the solution might be ‘do nothing’ or ‘walk away’ (P7).
Managing the self
Empowerment was evident for some participants, who described feeling more confident that they were doing a good job, more resilient and more able to persist and believe they could meet their goals.
My understanding that these issues are everywhere, rather than being unique to my organisation or situation has given me the confidence to take action …. (P5)
Personal benefits included the sheer joy of working through policy problems with an experienced, warm and witty set of peers in a supportive environment and building my confidence as a policy officer. (P7)
Managing the policy team
Several participants commented that they better understood how to nurture and mentor their staff, and how to provide ‘structural support’ (P2). One participant noted that she was more reflective, and that she actively encouraged critical reflection and review in her team. Another participant reported being ‘more likely to keep senior staff engaged so that they fulfil their leadership duties fully’. (P8)
Some participants gave examples of steps they had taken to build policy capacity in their team. One participant reported discussing with her director how to increase the capacity of her team, and subsequently putting systems in place ‘to ensure my team understand what is expected, better manage our workflow and ensure I have the right level of oversight’ (P6). Another member had been advocating for independent project management support, having realised that this was a gap not just for her, but also for others in her department.
Process evaluation
Participants identified a number of aspects of the learning set that contributed to their learning. Each learning set member commented on the value of taking time out from the pressures of their work for respite in a supportive and confidential environment; and of talking issues through with people who are their peers but outside of their workplace. A typical comment was ‘… it provided an extremely valuable time of reflection and rest’. (P4)
Most participants indicated they valued the opportunities to reflect with their peers on their own experience. Participants reported that they had learned they were not alone in the difficult issues they faced, and appreciated ‘… validation from the group that the approach you are taking is the correct one …’ (P7).
It has been extremely useful to share issues and experiences with a group of people who understand the context in which I work but who are independent from the actual environment. (P3)
Discussing my own specific practices of policy making provided me with a greater appreciation of how we all struggle with similar challenges. It confirmed that some of my challenges are pretty ‘universal’ […] This allowed me to be more objective about my own work issues and to learn from other member’s experience on how to ‘get stuff done’. (P4)
Five participants commented on the opportunity to develop networks with people working in similar areas in other jurisdictions. One person valued getting to know the other participants from her own jurisdiction and could see potential areas where they might be able to assist each other. Four learning set members found the learning set useful for sharing resources such as articles of interest, documents produced by others in the group, and policy and governance frameworks.
Participants generally agreed that having an independent, experienced facilitator was essential, particularly in the early stages. The public sector experience and theoretical knowledge of the facilitator, as well as her facilitation skills, were seen to be a critical factor in the success of the learning set.
Discussion and conclusions
The problems discussed by participants in learning sets are largely consistent with those observed in sets conducted with general managers in health,17,18 but there are important differences. Three aspects of the results are relevant to the question of policy capacity.
First, problems with relationships were prominent, highlighting the extent to which policy managers depend on influence rather than authority; and on leadership capacity in themselves and their superiors.
Second, the difficulty of enabling junior staff to develop higher level policy skills was seen as a ‘wicked problem’. The importance of tacit knowledge derived from experience and reflection was acknowledged, as was the value of discussing with staff the political and covert aspects of organisational life. The related problem of staff capacity to write complex policy briefs was seen as arising from challenges in comprehending the breadth of the task, and analysing complex situations.
Third, there was little separation between policy and operations in the roles of most of the participants. Their roles included some elements of implementation, including responsibility for managing public health programs, for design and development of organisation-wide systems for service delivery, or for service improvement projects. This may be seen to reflect the (then) centralised structure of the health systems in each of the jurisdictions which tended to either integrate, or blur the boundaries between, regulatory and operational functions and support services.
The benefits of participating in the learning set identified by participants suggest that it was a useful strategy for developing their own capacity to understand and manage their environment and relationships. The impact on empowerment and self-efficacy is consistent with the evidence on learning sets.15,18 The reported benefits for improving strategic skills and the exercise of judgement suggest that sets are useful for meeting the specific challenges of working in a policy environment and for developing the individual policy capacities that have been identified in earlier research. 8
We note that the learning set method is applied at the level of the individual, rather than the organisation, and thus does not directly intervene in the organisational level of policy capacity. However, it is clear from the responses of participants that many felt better equipped to build organisational policy as a result of their participation in the learning set.
The participants’ assessment of the impact and process of the learning sets indicates that the method was well suited to assisting them with the kind of problems they found challenging. The dynamics of learning appear to rely on two aspects of the method. First, engaging with problems experienced by other members enables members to ‘see’ a wider range of ways of framing and responding to problems. Second, discussing one’s own problems and strategies in this setting enables participants to gain new insight into their own ways of framing, analysing and responding to problems.
The small sample size (one learning set with eight participants) is a limitation of the research. However, the findings suggest that learning sets can be a powerful method for building policy capacity among middle-senior policy managers. While health policy competencies are clearly different from health services management, a similar approach to capacity development appears to work well at middle-senior levels in both settings. This approach may also be worthwhile for other areas of public policy. The cross-jurisdictional approach used in this study may also be particularly useful in the context of a federal system.
Footnotes
Acknowledgements
The authors wish to thank the policy managers who participated in the learning set and the industry partners from Queensland Health, ACT Health and SA Health who assisted in the design and conduct of the study. The authors also thank our co-investigators from Monash University, the University of Wollongong, the University of Queensland, Queensland University of Technology and the University of South Australia.
Declaration of conflicting interests
The authors declare that there is no conflict of interest.
Funding
This research was supported under the Australian Research Council's Linkage Projects funding scheme (Project number LP0776269).
Ethical approval
Ethics approval was granted by the La Trobe University Human Ethics Committee (Reference no. 07-131).
Guarantor
Deborah Gleeson
Contributorship
Vivian Lin led the research project. Judith Dwyer facilitated set meetings, took notes and analysed the problem ‘stories’. Deborah Gleeson conducted the evaluation and analysed the results; Judith Dwyer was not involved in any aspect of this process. All authors were involved in the project design and data analysis and contributed to drafting the paper.
