Evaluating Health Futures in Aotearoa *

Introduction

In 1997, the New Zealand health system was coming to the end of a decade of radical change and still faced a challenging future. The centralized Ministry of Health was responsible for developing policy for a system in which policy had been dominated by successive structural changes aimed at reining in health expenditure and redirecting funding toward improved health outcomes. From 1993, wide-ranging changes had been made to organizations and financing arrangements, and to incentivise efficiencies informed by market- and management-orientated theories (Gauld 2009). By 1997, as prospects for efficiency gains faded, new arrangements focused instead on broader population-based perspectives on health. At the time, the scenarios described in this article were being developed, four Regional Health Authorities with responsibility for purchasing health services for regional populations were amalgamating into one central purchasing agency, and twenty three Crown (State) Health Enterprises were being reshaped into Hospital and Health Services. Gauld (2009, 170) characterizes this period of reform as one of moving away from the earlier competitive market model for health services, but still saddled with shifting directions and uncertainties as central agencies and providers struggled to define their roles. At this time, there was a widely held view that health policy decisions were being made to address needs in the short term, but thinking about the long term was conspicuous by its absence. This was a problem, given there were emergent trends that would clearly have significant future impact on the health system (e.g., the aging of the population) and several uncertainties (e.g., developments in medical technology) that New Zealand needed to be prepared for.

To help galvanize some futures thinking, the Institute of Policy Studies at Victoria University of Wellington conducted a series of workshops about potential long-term futures for New Zealand’s health sector. From those workshops was compiled a summary report entitled “Health Futures: 2020 Visions” (Krieble and Middleton 1997) which contained five scenarios for 2020 and proposed itself as a starting point for the development of a vision “of what a diverse society may want (from its health system) and how the future may be influenced.” The titles of the five scenarios, which are somewhat self-explanatory but described more fully in a later section of this article, are as follows:

Given that we are almost in a future whose possibilities were being imagined over two decades ago, considerable insight can be gained from evaluating the extent to which the 1997 scenarios have come to pass and been useful to decision-makers in the health sector. By extension, such an evaluation is a test of the efficacy of scenario development and futures thinking in general.

While the evaluation should be of interest in its own right, it also comes at a time when interest is heightened on the long-term future of the New Zealand health sector. Terms of reference for a Ministerial review of the health and disability sector include consideration of future needs and issues (New Zealand Ministry of Health 2018). With a set of 2020 scenarios for the New Zealand health sector already available, it seemed timely to consider what was learnt from the exercise that could be fed into another round of futures thinking.

The History of Scenarios

The lineage of scenarios has been consistently traced back to its beginnings after the Second World War (e.g., Amer et al. 2013; Millett 2003; Varum and Melo 2010), although there were contemporaneous developments in the broader field of future thinking or foresight, with implications for how scenarios were to be perceived and used. In the late 1950s, Gaston Berger had established the French school of prospective thinking, which emphasized preparation for multiple futures to unfold—leading in turn to an insight that good planning spurs action that changes the present in preparation for the future (Durance 2010; Spaniol and Rowland 2018). Berger’s work in the “French School” of futures thinking was carried on by Michel Godet (1982) among others. The work of Bernard de Jouvenel (1967) was of seminal importance. On the other side of the English Channel/La Manch, the term “foresight” was derived as a counterpoint to the “hindsight” gained from retrospective studies of how technological innovations had come about (Martin 2010). Processes described later in this article involve a fair amount of hindsight.

In his oft-cited papers, Pierre Wack (1985a, 1985b) outlined issues that still resonate today. He saw that the world was changing rapidly and thought that organizations that could not quickly adapt would die. What was required was not so much new ways of planning as new ways of managerial thinking (Wack coined the phrase “the gentle art of reperceiving”).

From the perspective of today, it might be assumed that the best way to ensure connection with and between scenarios and managers, and to achieve “gentle reperception” would be to use highly participatory processes. However, later analysis of Wack’s writings (Chermack and Coons 2015) shows that he thought scenario planning as a group process was a “dangerous trap” which led to “regression to the mean” or conventional, mediocre thinking. He favored instead an approach based on workshops as a form of group interviewing, providing input to expert developers who would follow up with stunning presentations to win decision makers’ support. Wack did see side benefits from group process, such as team building, group dialogue and the sharing of mental models. But to him, the primary purpose of scenarios was to change the way decision-makers saw the world, so that they would act with a wider, more informed point of view.

Schoemaker (1993) defined scenarios as focused descriptions of fundamentally different futures presented in coherent script-like or narrative fashion. If scenarios are presented as possibilities, rather than firm predictions, they become psychologically less threatening to those holding different worldviews. Acceptance of scenarios is influenced by source credibility (i.e., who developed them), content credibility (what they say) and channel credibility (by whom and how they are presented). Schoemaker differed from Wack in arguing that the scenario building process should not be entirely entrusted to an intellectual elite.

Other useful criteria for measuring the quality of scenarios are included in another paper which also provides a practical guide to their development (Schoemaker 1995): scenarios should be relevant, internally consistent and archetypal (describe generically different futures rather than variations on one theme). Fourthly, each scenario ideally should describe an equilibrium or a state in which the system might exist for some length of time, as opposed to being highly transient. Unusually among early writers, Schoemaker addressed the question of “do scenarios work?,” albeit narrowly, by attempting to measure impact on sales.

Obviously, the Health Sector Scenarios developed in 1997 (Krieble and Middleton 1997) drew on knowledge that had been developed up to that point. Since that time, there has been continuous growth in the literature about scenarios, including a number of useful histories (e.g., Amer et al. 2013; Millett 2003; Spaniol and Rowland 2018; Varum and Melo 2010). With some exceptions (e.g., Campbell-Hunt 1998) most of this literature has emanated from the northern hemisphere. Many pre-1997 tenets of scenarios remain in place, but some controversies have continued while others have been resolved (more or less) and new ones have arisen. Charges of “methodological chaos” (Martelli 2001) and the “dismal status” of theory and theory development (Chermack 2002) have been followed by efforts to set things to rights (Chermack 2003, 2004, 2007; Ramirez and Wilkinson 2016) and the production of comprehensive reviews and guides to methodology (Bishop et al. 2007; Cairns et al. 2006; Hines and Bishop 2015).

There is still a degree of chaos, however, or at least diversity of approach. A number of purposes and definitions for scenarios have been offered (Bishop et al. 2007; Cairns et al. 2006; Coates 2000; Durance and Godet 2010; Millett 2003; Varum and Melo 2010; Wright et al. 2013). Unsurprisingly then, there is no “one type of scenario” (Durance and Godet 2010; Millett 2003). A helpful typology (Börjeson et al. 2006) identifies three major categories (predictive, explorative, and normative) and six sub-categories. However, most scenarios in practice are descriptive rather than normative (identify an ideal or desired future). The creation of normative futures is more often addressed through visioning techniques (Bishop et al. 2007; Bodinet 2018).

Although more criteria have been offered by which the quality or effectiveness of scenarios might be measured (Amer et al. 2013; Cairns et al. 2006; Coates 2000) very few attempts have been made to evaluate quality, outcome, or impact (Varum and Melo 2010; Wright et al. 2013). One exception found support for improved financial performance resulting from scenario planning in two industries (Phelps et al. 2001). Another diagnosed receptiveness to scenario thinking and impact on organizational “inertia” in decision making (Wright et al. 2008). A third study, of a small sample of executives/senior managers from seven companies, found that participants generally regarded scenario planning as an effective intervention with a positive contribution to the firms’ performance. The scenario approach was found to be useful in exploring the business environment and future risks, isolating trends, understanding interdependent forces, and considering the implications of strategic decision-making (Visser and Chermack 2009).

Despite the paucity of evaluations, the use of scenarios has continued to increase (Amer et al. 2013) which begs the questions “why is that?” and “how are scenarios being developed and used?” It may be that the evolution of corporate forms and increasing complexity of decision-making creates the need for more, nuanced sources of information and insight (Scharmer 2007) ¹ and scenarios fit the bill.

Mostly, emphasis seems to be placed on participatory processes, team building and organizational learning (Cairns et al. 2006; Coates 2000; Millett 2003; Varum and Melo 2010). However, Durance and Godet (2010) distinguish between:

scenario (processes) which are highly confi-dential and used exclusively by executive managers and those which are used as a tool for group process in order to mobilize the collective intelligence of an organization faced with a rapidly evolving external environment. These latter studies are highly focused on the communication of strategy as a central objective; whereas with the former, foresight is specifically used for developing enterprise strategy.

This duality of purpose and “access” to the fruits of scenarios is reprised to some extent by Varum and Melo (2010) and echoes Wack’s apparent “elitist” view of process (Chermack and Coons 2015).

Scenarios in Health

While some things impacting on the health sector are highly likely to happen—such as the aging of the population—no other sector faces more disruptive uncertainty. Paradoxically, although (one hopes) there will continue to be a focus on human beings who remain biologically the same, there is also the potential for them to evolve rapidly or be augmented by new technologies:

When thinking about the future of healthcare, images of robot surgeons, predictive algorithms, nanoparticles and nanobots in your blood-stream, (genetic technologies), and wearable or ingestible sensors may come to mind. A future of big technological advances that help get rid of once acute conditions (like cancers, infectious diseases), and improve the monitoring and treatment of chronic ones (like heart disease, diabetes and obesity).

Robot surgeons, diagnostic Artificial Intelligence systems, new cancer treatments, and online health … have the potential to displace (or work with) highly trained medical staff, or provide new methods of treatment. (Hickson 2018)

Hickson suggests there is already a reorganization in health from “sickness to wellness,” more of a focus on the “customer” and increasing recognition of the importance of holistic approaches. Advice for those leading and managing health services stresses the resulting complexity of their roles, given the dilemma of how much they should focus on the concerns and priorities of those individual patients who are clearly unwell in comparison to how far they should address the broader preventative health needs of the wider population (Walshe and Smith 2011). One example where this patient/population tension has played since the early 2000s in New Zealand has been the implementation of the New Zealand Primary Health Care Strategy. Here, the reform focused more on the development of population health within a public health view of primary care, with less management and policy attention being paid to better integrated primary care services for individual patients within each primary care practice (Smith 2011).

Given the combination of some solid underlying trends, a fast pace of change and uncertainties about the future that will finally emerge, scenarios are potentially an ideal technique for supporting development of relevant and resilient strategies in health. This potential is reflected in a considerable literature and Vollmar et al. (2015) provide an invaluable map to numerous scenarios relating to particular health conditions or disabilities, or particular aspects of health services (Beeck and Mackenbach 1997; Bierbooms et al. 2011; Bijl and Ketting 1991; Eberl and Schnepp 2006; Gregório et al. 2014; Nielsen 1996; Schaapveld and Cleton 1989; Zentner 1991). A few papers deal with scenarios for whole health systems (Ashton 1992; Ling and Hadridge 2000; Longley and Warner 1995; Neiner et al. 2004; Nguyen et al. 2014; Schreuder 1995; Stephan 1983).

Vollmar et al. (2015) used a rigorous process to select scenario papers for their review, but these tend to be Eurocentric and again, there is not a single example from the Southern Hemisphere. This might be a fair reflection of where the weight of experience and research lies, although as the authors (Vollmar et al. 2015) point out, many scenario projects are not published in academic journals and a substantial number of reports have been published as gray literature by government institutions, non-government organizations or private (commercial/consulting) firms. Much of the scenarios work undertaken in New Zealand would fall into this category (Krieble and Middleton 1997 being a case in point).

Vollmar et al. (2015) have not evaluated any of the papers or projects they reviewed. Although they state that “(N)one of the projects has been designated as unsuccessful by the authors,” no criteria for success are identified. They claim that methods are not clearly described, and there is no clear-cut scenario method. Others would probably dispute that (Bishop et al. 2007; Hines and Bishop 2015), and it would perhaps be more accurate to say that there is no single description or clear-cut method.

In thinking about future scenarios for health, there are several trends and issues that are common across the whole world, but national health systems might be regarded as culturally bound. This is certainly the case in Aotearoa/New Zealand, which has evolved in unique ways, influenced by a history of early settlement by Māori people followed by their colonization and decimation, then a demographic, cultural and economic resurgence (Belich 1996, 2001). Many disparities still exist, however, and Māori remain overrepresented in negative health statistics (Ministry of Health 2018). While highlighting these statistics has been important for advocacy and change, so has challenging the New Zealand health sector to honor earlier expectations of partnership with Māori, ² and to incorporate wider cultural understanding of health and wellbeing. It also has to be acknowledged that New Zealand is becoming much more demographically diverse, with Pasifika and Asian people making up an increasingly larger proportion of the population (Hawke et al. 2014).

New Zealand’s uniqueness does not mean that it cannot learn from other countries, and contemporaneous scenarios for the British Health Service (Longley and Warner 1995) were reviewed as part of the 1997 process. Much of the discussion in that 1995 British paper is still relevant today, although there are elements which have not turned out as foreseen. When we looked back at the paper from the perspective of 2018, we saw the kinds of insights that might also be gained from our evaluation of “Health Futures: 2020 Visions” (Krieble and Middleton 1997).

For example, in 1995, it was assumed there would be continuous, successful growth in immunization programs. Health professionals now know not to take this for granted, due in no small part to unanticipated developments (possibly qualifying as “wildcards”) such as some rogue research, resistance from an “antivaxxer” movement, and complacency due to the virtual disappearance of some diseases. In 1995, the incidence of AIDS was seen as likely to continue to grow, but as things have turned out this disease has been considerably beaten back (for technological and social reasons). Some radical ideas have come to pass, but not in the form that was envisioned: for example, rather than “patient hotels” providing cost-effective, high-quality hotel services on hospital sites for those who do not need twenty-four-hour care, we have Ronald McDonald houses to accommodate the families of young, out-of-town cancer patients receiving treatment. This form of corporate sponsorship of the health sector has been something of a surprise (In New Zealand) but is now mainstream. There is also a burgeoning retirement village sector with several companies being listed on the share market. Some of these companies incorporate hospital care facilities in their business models.

On the other hand, there are issues such as mental health and euthanasia that were missed in 1995 but have emerged as high-priority concerns. Then there are elements of normative futures that twenty years later have evolved into more explicit, visionary thinking about holistic health services:

the (British) National Health Services must help and encourage others—housing bodies, emplo-yers, local planners, national policy makers, and individual people—to act in a way which is conducive to better general health … this—often called the forging of “healthy alliances” has only recently achieved prominence in the NHS, and consequently has fewer manifestations in practice so far. But its time has come, and we can expect this sort of approach to grow in importance in the remaining years of this decade. (Longley and Warner 1995)

The extent to which this vision has been achieved in Britain (or elsewhere) is outside the scope of this article, but in 2018, holistic approaches are still being promoted in discussions about New Zealand’s future health policy. Interestingly, a separate methodology has recently been employed in a joint British/New Zealand process for looking back at the health reforms of the 1980s and 1990s. This “witness seminar,” a form of oral history involving key participants looking back at past events, has strong elements of hindsight (Brown et al. 2018).

The Process of Developing Health Futures: 2020 Visions

The report Health Futures: 2020 Visions (Krieble and Middleton 1997) ³ opens with a preface that states,

Our long-term thinking for the health sector may be less visible than our shorter-term responses to the health needs of New Zealanders, but it is every bit as crucial to our wellbeing as a society. When confronting pictures of what the future could look like, we can choose how we deal with the changes that lie ahead; we can resist change as long as possible, we can accept whatever change happens regardless of who it helps or hurts, or we can develop and re-develop our focus along with the change, encouraging it to come as close as possible to a future desirable to us all.

There are forecasting techniques which extend current lines of sight into the future, calling our attention to points along the line. However, it was decided in the current exercise to go further and adopt a three-dimensional approach, using the idea of health futures. This approach lets us pull in various ideas from many different angles, so that we develop a number of scenarios illustrating what might or might not be possible for the health sector, given whichever values the community may choose to support.

The general definition of scenarios was taken from Schwartz (1991): they were described as stories “about the way the world may turn out tomorrow, stories that can help us recognise and adapt to changing aspects of our present environment.” Scenarios were clearly distinguished from forecasts, and the broader field of Futures was described as one which helps to anticipate the future, set directions and make commitments to create the future we want. ⁴ “By understanding various future scenarios, policy-makers have a basis for strategic thinking to create the desired future and avoid undesirable ones” (Krieble and Middleton 1997). Here, prospective thinking was clearly at play.

The methodology used in 1997 is not the main focus of this article, but many common elements of scenario development were involved. The Institute of Policy Studies at Victoria University of Wellington ⁵ conducted a series of workshops involving twenty-eight people (including the current authors) from government, business, academia, and parts of the health service. Participants were provided with a background document which described a range of drivers of change based on health policy literature, New Zealand health policy documents and recent health futures exercises in Australia, the United Kingdom and United States of America. Uncertainties were also identified and with the assistance of decision support software, four draft scenarios for the state of New Zealand’s health sector in the year 2020 were developed. An iterative process of review and feedback refined the drafts between meetings and a fifth scenario was later added. The final scenarios were “designed to be the basis for future discussion, so no one scenario (was) marked as the preferred one.”

The five scenarios (in summary) were as follows:

The Institute of Policy Study’s stated aim in publishing the scenarios also included “to stimulate and focus thinking about the preferred future of the health sector” because

(T)he long-term nature and complexity of issues regarding possible health and disability support services require constructive thought about what we as New Zealanders want to achieve from our health sector. A desirable future should reflect on what a diverse society may want rather than a predetermined decision on what is needed.

The final document containing the scenarios was seen as providing a “starting point for others in the health sector to develop a vision of what a diverse society may want and how the future can be influenced.” Readers were invited to debate the credibility of the scenarios and test their internal consistency to consider who would be badly affected if a particular scenario came to pass and to use the scenarios to identify what they wanted to do to mold the future to where they wanted to go. More specifically, the following questions were asked:

Interestingly, around the time Health Futures was in circulation, a critique was published which among other things noted that there had been insufficient attention paid to the future role of the health workforce (Bichan 1998). As will be seen below, this is a conclusion that we have also reached in the current evaluation.

It can also be seen in retrospect, particularly with the benefit of twenty years’ further writing and thinking about scenarios, that the production of Health Futures: 2020 Visions drew from a mix of different influences on futures thinking or foresight and sought to achieve a range of goals and objectives which were not always compatible. For example, there was a mix of descriptive and normative (visionary) elements in the design and prospective use of the scenarios. In parts, there was also a blending of futures thinking and strategic planning. However, as discussed below, the conflation and separation of these dimensions are constant sources of challenge to futures thinkers and hardly unique to this project.

Learning from History

Most of the general literature on scenarios is derived from organizations such as Shell rather than whole sectors, which tend to be much more complex. There is also very little written on evaluation of scenarios from the perspectives of those standing in the actual future and looking back. A question arises about the extent to which lessons from the literature on scenarios can be applied to the New Zealand Health System. One possible answer is that existing knowledge can be tapped to inform the process of learning described in this article, as it sets out into largely unexplored territory.

For example, from scenarios’ early beginnings there have been debates about the technique which continue in the present day. The term scenario planning has become less favored (notwithstanding the view of human agency implied by la prospective) because it suggests a mechanistic or deterministic view more associated with forecasting. Integrating scenarios with strategic planning has also remained problematic.

There continues to be different views as to whether scenarios should be descriptive of possible futures or normative, that is, paint pictures of one or more desirable futures (more akin to vision[s] of the future). An associated question relates to the purpose of scenario development. Is it to derive “correct” or “accurate” scenarios, in which case an exhaustive set of steps and much testing may be required, or is the process itself that is most important? Proponents of the latter view hold that in the long-term accuracy is impossible and the process is more important because it builds understanding of futures issues, contributes to shared learning, challenges conventional wisdom and opens up minds to alternative (likely better) strategies that would otherwise be overlooked. In which case, diverse participation becomes more important since it allows a broader range of inputs and the opening up of more minds. This stance in favor of process has been supported by the advent of electronic media which have made participation and the sharing of information much easier. ⁶

It is not the authors’ intention to resolve any or all of these controversies. However, in establishing a logic for evaluating the scenarios described in “Health Futures: 2020 Visions,” and in considering how that evaluation might be useful to the 2018–2020 review of the health and disability sector, it is necessary to articulate a point of view on what scenarios are, and what they are for. To some extent, this view must be retrospective, since the five 1997 scenarios were obviously not able to be designed with the benefit of subsequent knowledge about scenario development. What is more, some of the original design work has not been included in the final report and is now lost in the mists of time.

We acknowledge that the following “point of view” inevitably limits the scope of our article, but also that we are making but a small contribution to a much wider and growing literature on scenarios and futures in general.

For our purposes, scenario development or scenario processes are the terms used. Scenarios are descriptive rather than normative. The purpose of their development is to open up decision-makers’ minds to future possibilities and the extent to which change might occur, so as to inform separate strategic planning processes and make strategy more relevant and resilient in the face of whatever future eventually emerges. “Opening up” of minds can be assisted by hindsight—looking back at the extent of change that has occurred in a similar time frame as scenarios are looking forward. ⁷

In New Zealand with its particular history, it is culturally a given that participatory processes are ideal, if only to ensure acceptability and uptake of scenarios and other outputs. While still contentious, the implications of this stance are that the so-called “accuracy” of scenarios (how well they describe the future that has arrived) is of lesser importance than process criteria and those proposed by Schoemaker (1993, 1995) which have been used as the basis of our evaluation:

That is not to say, however, that an assessment of the extent to which things have come to pass would not be useful—along the lines of the commentary on the 1995 British paper referred to above (Longley and Warner 1995) for example,

Evaluation

Evaluation is about the quality of what is done or delivered. In this section, we describe what was learnt when we undertook a retrospective assessment of the 1997 scenarios. The underlying logic of the 1997 exercise was that good-quality scenarios would open the minds of decision makes to possibilities they would not have otherwise have considered, leading to better decisions, more relevant, resilient strategies, and better health outcomes for New Zealanders. We did not know how much this logic was shared by participants, so our approach was exploratory. We started with criteria informed by the foregoing discussion with respect to Health Futures: 2020 Visions. In an ideal world (and in future), these criteria would have been measured, and where possible benchmarked, from the beginning of the process:

These criteria were addressed through semi-structured interviews with five original participants, three of whom are still involved in health policy at senior levels, and one “modern day equivalent” about their views of developments in the health sector over the last twenty years. The involvement of people with continuous involvement proved useful, because they were well equipped to recall the state of play in 1997 and developments since. They provided us with insight that was not available from other sources into the extent to which the 1997 process “opened minds,” along with respondents’ views on “how things had turned out.”

Futures studies, and by implication scenarios, aim to help individuals better understand the processes of change in order to create wiser preferred futures (Inayatullah 2008). Given this context, those we interviewed were being asked to reflect on how they made sense of changes in the New Zealand health sector. The 1997 scenarios were provided as prompt for this assessment. Our approach to these interviews was exploratory because we were aware that there are diverse perceptions about the primary purpose of evaluation with respect to scenarios. Scenarios are not predictions. A sole focus on the accuracy of what was contained in the 1997 scenarios would not be appropriate, but equally exploring how minds were opened to new possibilities required some form of backcasting—that is, asking a subset of those involved in the 1997 exercise what signals were spotted and what was missed.

The questions traversed in interviews were as follows:

Our analysis of the interviews was based on basic coding and sorting of key themes. In order to orientate our work to the same shared history of change, we supplemented our interviewees’ assessment of the underpinning drivers and values that shaped each scenario with the relevant literature. We sought literature providing an account of where New Zealand’s health policy attention had been directed over the past two decades (Cheyne et al. 2008; Cumming 2017; Gauld 2008; Tenbensel et al. 2012). While interviewees could recall key events since 1997, the literature was able to serve as a cross check on these developments.

Findings

A consistent theme in discussions with our interviewees was the complex and layered nature of health sector change. While health sector strategies for change in the last twenty years have been progressive and good at painting a picture of the direction needed, the realities of implementation have often had less attention. Re-reading the five scenarios reinforced the value of distilling the different design principles that shaped each scenario and surfaced potential conflict between these principles. For the interviewees, these principles and conflicts provided a basis for recognizing the intersecting issues and battles that can (and still do) hinder health sector change and improvement.

The 1997 scenarios were drafted at a time when New Zealand was moving away from a conviction that widespread structural reforms were going to translate seamlessly into improvements. The health futures exercise offered an opportunity to “safely” explore, outside of entrenched ideological positions, what health sector change could look like. Today’s interviewees were struck by the ongoing relevance of the underpinning drivers and values that had shaped each scenario in 1997. Combinations of these drivers and values were explored in each scenario to present distinctive chains of plausible events. Table 1 summarizes each of these drivers and values alongside an assessment of how dominant these continue to be from those interviewed. A brief summary of where New Zealand health policy attention has concentrated over the past two decades is included in the third column.

OPEN IN VIEWER

Table 1.

Looking Back on the Underpining Drivers and Values.

Underpinning values and drivers (1997)	Reflections from those interviewed (2018)	Post 1997 health policy attention
Rationing pressures	Less dominant in wider health policy discourse	Decisions on mix of health services moved from national agencies into twenty locally based District Health Boards (DHBs)
Information technology	Continued attention	Signaling about the opportunities from increased data and smart information systems continues to be a regular feature of New Zealand health strategies (see, for example, Minister of Health 2016)
Research and development	Continued attention	While the disruptive potential of big technological advances in medicine is regularly heralded, policy attention has also centered on research which grows New Zealand’s understanding of how best to enable communities to develop their own approaches to supporting better health (Hickson 2018).
Access/equity	Higher prominence	From 2000, a higher level of attention was given to changes in primary care financing reflecting the significant barrier to access and equity from earlier fee for service financing arrangements for primary care (Robin Gauld 2008). More recently, calls have strengthened for policies to overcome barriers to health access and utilization for Ma¯ori, Pasifika and other high need populations (Waitangi Tribunal 2018).
Quality	Higher prominence	Creation of the Health Quality and Safety Commission has increased policy attention toward quality and safety (Shuker et al. 2015)
Health gain	Continued attention	From the early 2000s a series of population health targets were introduced to incentivise health gain. More recently, these targets have been replaced by networked governance arrangements between DHBs and primary care organizations working collectively on plans to achieve system level measures (Gauld 2017)
Efficiency	Continued attention	From 2000, a return to a less competitive more cooperative model for determining health priorities set an environment where decisions directed toward maximizing health gains (within a budget constraints) were delegated to District Health Boards (Cumming 2017)
Consumer sovereignty	Higher prominence	Initial interest in democratic governance of DHBs through elected Board members has been further supplemented by: (1) calls for consumer engagement to improve quality in the system and (2) the introduction of new forms of care based on Ma¯ori philosophies of holistic health and development.
Personal responsibility	Less dominant in wider health policy discourse	Earlier changes built around center right philosophies preferring consumer choice in private markets have given way to a greater emphasis on private–public partnerships (Cheyne et al. 2008).

The semi-structured interviews roamed widely as interviewees reflected on the signals in the scenarios that did come to pass, and key changes for which they had seen no signal. The summary account below is derived from our discussions.

A prominent driver across all scenarios was Rationing Pressures. In 1997, considerable policy effort was expected to go toward managing public expectations, rising costs, and constrained healthcare budgets. Today, rationing pressures continue but discourses about rationing are not as prominent. A cynical view is that there has been a reframing and devolution of rationing “out of sight” and the country is in denial about this still looming issue (New Zealand Treasury 2016). In the last twenty years, the locus of decision-making moved away from centrally accountable health sector agencies into twenty locally based District Health Boards. Interviewees pointed out that hard prioritization calls are still being made by these Boards, but the debate is now shifting toward how much New Zealand wants to have national consistency in these decisions and how much is it prepared to live with local variation based on local assessment of needs. In part, these changes are linked to the prevailing political climate and it is worth noting that New Zealand has had two different Governments of nine years’ duration—one center-left and one center-right, since the 1997 exercise.

The pressures on an aging population were clearly foreseeable in 1997, prompting concerns about the health sectors’ ability to cope with chronic conditions such as diabetes and aging related neuro-generative diseases. These fears have been realized. Current policy attention is being paid to improve long-term condition management with a strong emphasis on greater responsiveness from the primary care sector in managing these conditions (Ministry of Health 2016). Looking back on the scenarios, little attention was paid to the distinct role of the primary sector in the health system in managing chronic conditions. The collectivisation of primary care through new meso-level organizations representing general practice interests was not foreseen, though the potential for better integration of primary and secondary care services due to the spread of larger primary care provider organizations was recognized.

The scenarios also foresaw the potential impact of Research and Development and Information Technology drivers. The Technocrats Dream scenario highlighted the potential for better information collection and sharing across the sector, including hospital booking systems and unique personal identifiers. What was underplayed was the potential for a digital divide, the social processes needed to support technological change and the move toward myriad, personal information technology (IT) systems. Calls in the recent New Zealand health strategy for “smart systems” (Minister of Health 2016) reflect the long run interest in the gains expected from new digital ways of working, but interviewees pointed out much of the potential is still unrealised. In particular, concerns were raised that current ways of delivering health services are not keeping pace with consumer expectations, nor efficiently leveraging mobile and digital technologies.

Early signals of the importance of patient-centered care were evident in the Power to the People scenario. Important ideas concerning the broader concept of well-being rather than illness, the socio-economic determinants of health and the importance of consumer empowerment were all anticipated. These centered on one scenario only, but were a weak early signal of a set of ideas that have received significant health policy attention since 1997. The introduction of whānau ora ⁸ in New Zealand as a philosophy of holistic health and development operationalised by Māori providers being one obvious example (Boulton and Gifford 2014). Indeed, it may be argued that “Power to the People” has taken on some of the characteristics of an aspirational “vision” for the sector.

One scenario—“Two tiers”—presented a health sector in 2020 where New Zealanders had given away any desire to have a universally accessible public health system. This scenario was designed to direct attention toward a future to prevent rather than aspire to. A future where the state provides an inadequate safety net for the uninsured, public confidence in the public health system fades and policy gridlock prevents progress. The arc of health policy change since 1997 has avoided this scenario, with policies focused on managing a largely state-funded system with an emphasis on quality, efficiency and responsiveness alongside social democratic values (Cheyne et al. 2008). Interviewees suggested the debate about privatization of the health sector and withdrawal of the state encompassed in the two tiers scenario missed the more nuanced ways the private sector has made inroads into the New Zealand system. The examples noted included the quiet influx of corporate players into primary care as a way of managing increased demand through shared services, and the rise of private sector responses such as retirement villages in response to home care demands. Moreover, the ongoing tension between the marketing of some products and health promotion activities—for example, high levels of sugar in processed food and drink conflicting with efforts to reduce sugar intake—are further examples of the type of private–public issue not anticipated in any scenario.

Standing back, a key area that was missed in nearly all the scenarios was consideration of the health sector workforce alongside a sense of how the structural power of the professions may hinder or enhance change. New occupational groups originating from nursing were anticipated to “fulfil the need for hybrid skill sets resulting from consumer demand” (Muddling Through Scenario). However, while the global nature of the workforce was acknowledged in the Positively Private and Global Scenario, missed in all five scenarios were the challenges of an aging general practice workforce, uneven distribution of the workforce between rural and urban areas of New Zealand and the need to increase the number of Māori students entering health science, medicine, and other professional programs. Representatives from the professional colleges and other health force unions were not included in the list of workshop participants, which may explain why workforce issues were underplayed. Interestingly, the area where the most extensive futures work has since been undertaken since the scenario exercise has been the work of Health Workforce New Zealand which sought to build a picture of the health workforce in 2020. This work involved assembling small groups of clinicians to assess the current situation in fifteen specialized areas and provide recommendations for improvements, ⁹ having stronger parallels with strategic planning than with futures methodology.

Using criteria identified by Schoemaker (1993, 1995), the scenarios themselves were relevant, credible, and coherent, but not particularly archetypal (this was intentional—they were designed to overlap each other). The process used was valuable in opening up decision-makers’ minds to possibilities without them needing to feel threatened or defensive, but given the New Zealand context, it could have been more inclusive. That said, the conundrum of a highly consultative process versus a “think tank” approach cannot be resolved by a single evaluation. It may also be that the 1997 scenarios did not have a long enough time frame, since there have been repeated of nine-year swings of the political pendulum in New Zealand, with consequent changes of direction in the health system. A longer timeframe would allow for “political swings and roundabouts” to be treated as a factor to be considered in the development of robust strategy.

In terms of impact on decision-making, all interviewees remembered the 1997 scenarios being talked about, albeit for a relatively short time before being overtaken by other developments. The initiative “slipped away” from decision-makers’ fields of vision for a number of reasons: its discretionary nature, with no explicit follow-up required; It was time bound rather than continuous—once finished it was out of sight and easily forgotten; it was championed by a small group rather than the whole of senior management. It would be fair to say that if minds were opened up to the future, it was only for a short time. One respondent suggested that “scenario thinking is not a natural way of thinking” and would take years to embed properly. Another pointed out that the scenarios were referred to in academic circles, more so than in policy.

Conclusion

This article looks back at a process that set out two decades ago to help achieve better health outcomes for today’s New Zealanders. Many of the changes that emerged over those decades were inevitable, and the Health System has responded as best it could. We have been unable to determine whether the 1997 scenarios helped, or whether “better” scenarios would have made a positive difference. However, hindsight leads us to conclude that any future consideration of health futures (or other scenario development projects) should be designed with evaluation in mind. This implies a need for more clarity of purpose. In New Zealand at least, the purpose of “opening up minds” is more likely to get the engagement of decision makers, and to be successfully realized. In future projects, it will be important to establish criteria and benchmarks against which success can be measured, to get more explicit buy-in of key stakeholders and to draw on a more diverse range of inputs. The criteria included in this article are proposed as set of possible starting points for evaluating scenarios and the processes by which they are developed. We would also suggest that rather than being carried out during a discrete period of time, scenario development should be a continuous and constantly updated process.