Moving policy implementation theory forward: A multiple streams/critical juncture approach

Abstract

Meta-reviews of the implementation literature have constantly bemoaned a lack of theory in this area. This is partially a function of the policy sciences having inherited a tradition of descriptive work in public administration, a historical phenomenon exacerbated by the more recent addition to this corpus of an equally atheoretical set of works in public management. As a result, the study of policy implementation within the policy sciences remains fractured and largely anecdotal, with a set of proto-theories competing for attention – from network management to principal–agent theory, game theory and others – while very loose frameworks like the ‘bottom-up vs. top-down’ debate continue to attract attention, but with little progress to show for more than 30 years of work on this subject. This article argues the way out of this conundrum is to revisit the subject and object of policy implementation through the lens of policy process theory, rather than appropriating somewhat ill-fitting concepts from other disciplines to this area of fields of study. In particular, it looks at the recent synthesis of several competing frameworks in the policy sciences – advocacy coalition, multiple streams and policy cycle models – developed by Howlett, McConnell and Perl and argues this approach, hitherto applied only to the ‘front end’ activities of agenda setting and policy formulation, helps better situate implementation activities in public policy studies, drawing attention to the different streams of actors and events active at this phase of public policy-making and helping to pull implementation studies back into the policy science mainstream.

Keywords

Advocacy coalition framework multiple streams policy cycle policy implementation stages approach

Introduction: The poor state of policy implementation studies within policy process theory

In order for policies to be put into place, among other important tasks funding must be allocated, personnel assigned, and rules of procedure developed in order to make a sometimes very abstract policy ‘work’ on the ground. Implementation research within the policy sciences over the past 30 years has generated insights into many specific activities and practices prevalent in different jurisdictions at this stage of the policy cycle. However, periodic meta-reviews of the subject continually find this area of policy studies to be largely descriptive and poorly integrated into mainstream policy theorizing (Durlak and DuPre, 2008; Hupe and Sætren, 2015; O’Toole, 2004).

As this article argues, this is due in large measure to the fact that existing studies of policy implementation have inherited a tradition and body of largely descriptive work in public administration and law, a corpus of studies whose atheoretical nature has been exacerbated by the more recent addition to it of an equally descriptive set of works in public management. As a result, the current state of studies of policy implementation remains fractured, with a set of proto-theories competing for attention – from network management to principal–agent theory, game theory and others – with no single approach helping to integrate the study of implementation activities with that of other stages of the policy process, from agenda-setting to policy evaluation, which have received detailed treatment and contributed to the advance of models and frameworks of policy processes and outputs (Howlett et al., 2009).

The way out of this conundrum, it is argued below, is to revisit the subject and object of policy implementation through the lens of policy process theory, rather than continuing to appropriate somewhat ill-fitting concepts from other fields of study such as public administration, public management, law and organization, and regulatory studies, among others. Doing so, however, requires some adjustments to be made to existing policy process theories in order to contribute to the fruitful dialectic of policy and administrative studies needed to move policy implementation studies forward.

In particular, as this article argues, it requires a synthesis of several currently competing frameworks in the policy sciences – advocacy coalition, multiple streams and policy cycle models – which can help to better situate implementation activities within the policy sciences, drawing attention to the different streams of actors and events active at this phase of public policy-making and helping to pull implementation studies back into the policy studies mainstream. The elements of this synthesis, the reasons why it is needed, and the resulting model of policy process tasks, including implementation, which emerges from it, are set out below.

The problems with policy implementation theory: Something borrowed and something blue

After a public problem has reached the policy agenda, various options proposed to address it, and a government has decided on a course of action to follow, the decision must be put into practice. The effort, knowledge and resources devoted to translating policy decisions into action comprise a set of activities commonly described as the ‘implementation stage’ of policy-making.

Until the early 1970s, implementation was often regarded as largely unproblematic or quasi-automatic within the policy sciences, with the execution of a policy expected to occur through the invocation of standard operating procedures and practices in well-staffed and expert civil services and other public organizations. Much early policy research was embued with this idea and ignored or downplayed the pitfalls arising at this stage of policy-making, for example, assuming that once a policy decision was made, the administrative arm of government would simply marshal from within its own ranks the resources and knowledge needed to carry it out (Hargrove, 1975; Hupe and Hill, 2016).

For most public policy researchers at that time, the most significant and research-worthy activities in policy-making studies were felt to occur at the ‘front-end’ of the policy-making process: namely agenda-setting and policy formulation and decision-making itself (Jones, 1984). This was where policy problems and solutions were articulated, defined, framed and contested, with implementation and other ‘back-end’ activities such as policy evaluation simply expected to be carried out by neutral, technical, officials in a reasonably effective and efficient manner. In an interesting case of selective borrowing, this was done despite the availability of a large, century-old, literature in public administration, organizational behaviour and management concerned with the difficulties involved in the effective execution of government decisions (Gaus, 1931; Goodnow, 1900; Wilson, 1887).

Some later studies of implementation continued to be influenced by the idea that implementation was largely technical in nature. However, study after study, including notably following the publication of Pressman and Wildavsky’s (1973) work on program implementation difficulties in 1960s era US urban and social welfare policy, showed that many problems were inherently involved in the effective execution of policy aims and that effective implementation should not be taken for granted or viewed simply through a ‘barriers’ or (in)capacity lens. Pressman and Wildavsky’s study of federal programs for unemployed inner-city residents of Oakland, CA showed that job-creation programs were not actually being carried out in the manner anticipated by policy-makers and led to a renewed emphasis on the need for better empirical studies, and better informed and carefully developed theories of policy implementation, if better policies and outcomes were to be consistently achieved (Bardach, 1977; van Meter and van Horn, 1975). Research in other countries arrived at similar conclusions (Hjern, 1982; Mayntz, 1979) about the need to address political, epistemological, behavioural and contextual factors affecting target compliance and agency actions, among others, in order for successful implementation to occur (Spicker, 2005, 2006), and about the difficulties governments and policy-makers encountered in attempting to carry out these tasks.

Within the newly emerging policy sciences, the upshot of these studies in 1970s was the start of a more systematic effort in the 1980s to better understand the factors that influenced public policy implementation (Sabatier and Mazmanian, 1981; Mazmanian and Sabatier, 1983). However, this ‘second generation’ of implementation research in the policy sciences quickly became embroiled in a dispute over the most appropriate focus for describing and analysing its subject matter – the so-called debate between ‘top-down’ versus ‘bottom-up’ approaches to the study of implementation (Barrett, 2004; Sabatier, 1986) which effectively blocked theoretical development in the area for decades. This debate centred around whether empirical studies and theories should be developed from the perspective of government officials and policy-makers at the top of bureaucratic pyramids or from that of officials on the shop floor or ‘street-level bureaucrats’ actually facing customers and programme clients, defining policy implementation through their many day-to-day interactions (Lipsky, 1980).

Although this debate was largely methodological, it effectively froze theorization and model building into two falsely competitive paradigms. Some studies generated analyses and prescriptions that presented policy implementation from a ‘top-down’ perspective concerned with better understanding those mechanisms that ensured that implementing officials could do their job more effectively (defined as keeping to the original intent of the public officials who had ratified the policy). This approach was opposed by those who subscribed to the more ‘bottom-up’ perspective, which carefully examined the actions of those affected by and engaged in the implementation of a policy rather than the desires and plans of higher-level planners and decision-makers (Sabatier, 1986).

Studies conducted in bottom-up fashion showed that the success or failure of many programs often depended on the commitment and skills of the actors directly involved in implementing programs (Lipsky, 1980), and these studies focused attention on the formal and informal relationships among actors involved in both designing and implementing policies. Here, effectiveness was seen to arise from the adaptive behaviour of ‘street-level bureaucrats’ seeking to attain and sustain the means to achieve policy goals on the ground (Lipsky, 1980) rather than from the activities of agents closer to the centre of policy-making processes in designing structures and procedures intended to achieve this.

While both of these approaches generated valuable insights into implementation activities at both levels of government, like many similar dichotomous debates in the field, they tended to ossify into hardened positions that stifled conceptual development and research, leading to renewed calls in the late 1980s and 1990s for new approaches and a ‘third generation’ of implementation research that would yield more ‘scientific’ implementation research methods and results (see deLeon, 1999; Goggin et al., 1990; Lester et al., 1987).

Many scholars answered this call and did move beyond the top-down versus bottom-up debate during the 1990s, yielding a fertile decade of implementation research (Lester and Goggin, 1998; O’Toole, 2000). Much of this effort focused quite properly on better understanding and modelling administrative behaviour. But it remained one which continued to emphasize the unique administrative and managerial aspects of implementation and failed to integrate these aspects into larger, extant, models of policy processes (Saetren, 2014).

In addition to studies using the insights of new approaches and models of administrative behaviour such as game and principal–agent theory (e.g., Hawkins and Thomas, 1989; Scholz, 1984, 1991), a second approach also emerged at this time that concentrated on the nature of the administrative tools or ‘policy instruments’ used by implementers in their work (see Bobrow, 2006; Mayntz, 1983; Salamon, 1981). This latter approach was better situated in the policy literature but benefited little from the behavioural insights generated from game and principle–agent modelling.

The strengths and weaknesses of both of these approaches in moving understandings of policy implementation and policy processes forward are discussed briefly below.

Game theory

Game theory concepts were used in policy implementation studies in 1980s and 1990s in the effort to better model and predict administrative behaviour in dealing with issues of policy compliance. Regulatory theorists such as Hawkins (1984) and Thomas (1989), for example, had already noted how different regulatory styles could be observed in specific sectors and issue areas, not to mention jurisdictions (Kagan, 1994, 1996), as regulators opted to construct oversight systems based on choices they made about the need to employ coercion or persuasive methods to secure ‘target’ compliance.

This insight was used by analysts such as Scholz (1984, 1991) to apply game-theoretic principles to the regulatory situation. Scholz demonstrated that the incentives and payoff for compliance and non-compliance on the part of the regulated could be matched to payoffs and incentives realized by both regulators and regulatees which could inform the choices to use education or enforcement as implementation strategies.

A typical implementation game, for example, could be one in which regulators would initiate implementation with efforts at persuasion, efforts that typically could be met by a failure of the regulated to comply. This would lead regulators to move towards more coercive rules in the next iteration, yielding a worse-off situation for regulators and the regulated, who would face, respectively, high enforcement and high compliance costs. The game would then progress to an intermediate position in which coercion would be scaled back in exchange for compliance by the regulated, although this would be an unstable equilibrium requiring monitoring and temporary increases in coercion on the part of regulators to maintain compliance.

This application of game theory to regulatory implementation generated interesting insights. However, it did not take into account either how existing arrangements arose or the unequal ability of some actors to resist or direct state actions. Nor did it address the divisions within the state itself that affected the ability of implementation on the ground to match the aims and expectations of enacting politicians. This latter concern resulted in the application of a second game theoretic model to the implementation stage, that of principal–agent theory, in a renewed effort to explain and merge bottom-up and top-down findings, perspectives and dynamics.

Principal–agent theory

As even top-down theorists recognized, civil servants can acquire a great deal of discretion in pursuing policy goals under changing environments as they tend to become more expert in specialized administrative areas than the generalists who staff political offices. These civil servants can then decide how and to whom the full force of laws and regulations will be applied (Calvert et al., 1989; McCubbins et al., 1987, 1989), placing politicians and administrators in a particular kind of principal–agent relationship, such as those commonly found in associations between lawyer and client, physician and patient, or buyer–broker–seller, in which the principal is dependent on the goodwill of the agent to further his or her interests when it may not be in the interests of the agent to do so (Banks, 1995; Ellig and Lavoie, 1995; Francis, 1993).

One principal–agent problem that has long been recognized by policy researchers, for example, is the tendency for regulators (the agents in this case), over time, to identify more with the needs of the regulated than with their erstwhile political principals. At the extreme, this tendency is thought to undermine the regulatory impulse and trigger its demise and replacement (Bernstein, 1955). Career patterns where individuals move back and forth between the government bureaucracy and industry employment over time also consciously or unconsciously blur their interests and ambitions (Sabatier, 1975). This theory of regulatory capture is based on flaws in principal–agent relationships that permit, and even encourage, such behaviour.

Principal–agent theory thus pointed to the implications of the design of administrative structures for effective implementation and underlined the importance of including mechanisms in implementation designs that ensure effective oversight of administrative actors by their political ‘masters’ in order to curtail agent discretion (Bozeman, 1993; Hammond et al., 2003; Milward and Provan, 1998).

This focus extended the insight of ‘bottom-up’ implementation studies of the need for structures allowing senior officials to control street-level ones while granting those on the ground enough autonomy to perform their work effectively (McCubbins and Lupia, 1994; McCubbins and Schwartz, 1984). However, it did not discuss how such designs might come to be adopted, or not, in policy decision-making or of what elements and components they were comprised. In other words, while advancing thinking about implementation, per se, like game theory models, in general, it did not integrate these insights into mainstream public policy thinking about policy processes and outputs.

Instrument Choice Theory

An alternate approach to policy implementation which emerged around the same time was the ‘instrument choice’ approach. This approach was more closely linked to studies of policy processes and outputs than either of the other two and did try to incorporate and consider some aspects of policy formulation and decision-making into its approach to programme design. It began from the observation that, to a great extent, policy implementation involves applying one or more of the basic techniques of government – variously known as policy tools, policy instruments or governing instruments – to the resolution of policy problems (see Bressers and Klok, 1988; Elmore, 1978, 1987; McDonnell and Elmore, 1987; Schneider and Ingram, 1990) and recognized that much policy activity, including policy formulation and decision-making, revolved around the selection and implementation of these tools, which served as the basic building blocks and content of policy designs.

This approach thus did attempt to integrate implementation with policy process studies, arguing that the process of giving substance to a government decision always involves choosing among or combining the several tools which are available to make a contribution to advancing policy goals and aims and that it is these choices which ultimately are evaluated in policy appraisals (Hood, 1986; Linder and Peters, 1991). After having developed basic inventories of these tools, the instrument choice perspective addressed the question of why policy-makers do, or should, choose a particular instrument from among those available, focusing on the role played by the nature of the tool itself – for example, its level of precision of targeting and the degree of coercion involved in its use – and the manner in which its use was supported by prevailing norms and political ideologies (Howlett, 1991).

Unlike game-theoretic approaches, this clearly set implementation studies within a larger policy process framework, linking implementation structures and policy mechanisms to earlier stages of policy-making activity around the articulation and design of alternative policy measures such as policy formulation (Linder and Peters, 1987, 1989). Later, these studies turned to larger questions such as whether these choices resulted in distinctive implementation patterns or styles that could be discerned among the policy efforts of different jurisdictions (Enkler et al., 2017; Rothmayr et al., 1997). Answering these questions moved implementation analysis away from its roots in the study of public administration and helped to integrate implementation research with the policy sciences. Specifically, these studies highlighted the close links among policy formulation, decision-making and implementation.

This approach towards implementation, however, largely ignored top-down and bottom-up behavioural issues and other concerns having to do with problems around the design and operation of compliance and service delivery mechanism. The exact mechanisms through which tools are chosen and applied are not clear in this model, which often lapses into an apolitical, technical conception of policy-making in which the ‘best’ choices are always made (Lascomes and LeGales, 2007).

Each of these approaches is thus revealing of some aspects of policy implementation, but also underlined that policy implementation involves more than simply executing previously arrived at decisions or abstractly matching goals with means and must be understood in the context of larger policy-making processes.

Hence none of these approaches presented a fully-fledged model of implementation. But each did implicitly endorse the notion that policy implementation can only be meaningfully understood and evaluated within the context of its position in the entire policy process. And they highlighted certain key variables which implementation theory needs to examine, including the nature of the existing range of policy actors, the kind of resources that these actors have at their disposal, the nature of the problems they are trying to address, and the institutional arrangements in which these efforts take place.

A fourth wave? Better integrating implementation within policy process theory

Given this background, it is not surprising that meta-reviews of the very large implementation literature constantly bemoan the state of theory in the area of policy implementation (O’Toole, 2000) and detail how the study of policy implementation continues to be fractured, largely descriptive and poorly integrated with other works in the policy sciences (Hill and Hupe, 2002; Hupe and Hill, 2016; Hupe and Sætren, 2015).

But each of these earlier eras and models give us some ideas about how to improve implementation studies and where to look for evidence and concepts in so doing. Instrument selection, for example, showed implementation to be a complex activity influenced by factors such as the nature of the subsystem or network involved in policy-making and especially its propensity to allow new actors and new ideas to penetrate into policy deliberations. And behavioural and game-theoretic research showed that whether or not the selected instrument will actually be able to address an issue depends on which options were chosen by governments and upon the implementation context – complete with compliance games and principal–agent problems (Bressers, 1998; Bressers and O’Toole, 1998, 2005).

All of these are features of policy-making that affect the ability of a government to accomplish desired policy goals and require the better integration of implementation studies with those developed to help explain, understand and predict the flow and outputs of policy processes in order to push studies, and understandings, of implementation activities forward.

Combining policy process theories to further policy implementation studies

Applying contemporary policy process models such as the advocacy coalition framework (ACF) (Sabatier, 1987), multiple streams framework (MSF) (Kingdon, 1984) and others (see Sabatier and Weible, 2014) to implementation, however, is no mean feat. Not only do implementation studies rarely deal with policy processes but process theories also (a) rarely deal directly with implementation and (b) disagree among themselves about many fundamental attributes to policy-making such as the nature of the process itself and its key actors, among others (Weible and Carter, 2017). Nevertheless, the effort is worthwhile and is the subject of contemporary ‘fourth generation’ implementation studies.

A first step in this undertaking requires reconciling existing policy process frameworks with respect to what they say about the basic components of such processes, including implementation. Not every model claims to explain all aspects of policy-making but examining several of the major ones helps identify their strengths and weaknesses, their overlaps and omissions and suggests how they may be combined to shed light on how policy implementation relates to other stages and activities of policy-making (Breunig et al., 2016; Howlett et al., 2015, 2016, 2017). Once this has been done, it is possible to get a clearer sense of how the resulting model relates to implementation and also how existing models and studies of implementation fit within these theories and frameworks.

The policy cycle model

The longest-standing conceptual process framework employed in the policy sciences is the notion that the policy process is constituted by a relatively small number of more or less sequential phases or ‘stages’ of governmental problem-solving (Anderson, 1975; Jones, 1984; Lasswell, 1956, 1971). Lasswell’s work was highly influential in creating this vision and formed the basis for many later approaches and numerous permutations of his original stages framework (e.g. Brewer, 1974; Jones, 1981; Lyden et al., 1968; Simmons et al., 1974). The model ultimately evolved into the now ubiquitous ‘cycle’ construct of five main tasks of policy-making: from agenda-setting and policy formulation through decision-making to policy implementation and evaluation (Anderson, 1975; Bridgman and Davis, 2003; Howlett et al., 2009; Jann and Wegrich, 2007; Brewer and deLeon 1983).

The analytical goal of the ‘policy cycle’ framework was to simplify the complexity of public policy-making by identifying its fundamental processual and cyclical nature. It deconstructed the policy-making process into several discrete stages corresponding with the sequence of tasks involved in conceptualizing the creation of and outputs of government (Howlett et al., 2009; Lasswell, 1956).

This approach is arguably the most enduring conceptual construct in the policy sciences (Burton, 2006; deLeon, 1999; Weible et al., 2012). Although many authors, including most notably those involved in the development of the ACF model (Sabatier, 1991) have over the years called for the supersession of the stages approach on the grounds that it is overly linear and rationalistic and technocratic, ignoring important elements of the political horse-trading and legislative, administrative and other battles which frequently characterize day-to-day policy-making, nevertheless, it lives on as the dominant framework applied to public policy-making (Bridgman and Davis, 2003; Howlett et al., 2009; Jann and Wegrich, 2007).

The problem-oriented, multiple task-oriented stages approach to policy processes can be contrasted with the two other current major models of policy-making that emerged in the 1980 and 1990s and became its closest rivals for theoretical hegemony in the contemporary policy sciences. Both these approaches began in a different way and focused on particular dimensions of policy-making processes, somewhat inadvertently elevating one task to a position of supremacy vis-a-vis the others, and in neither case paying very much attention to implementation.

One of these alternatives is the ‘multiple streams’ framework and its ‘garbage can’ perspective on policy-making dynamics found in the work of Kingdon (1984) and later work by Zahariadis and others on this subject (Zahariadis, 1995, 2007; Zohlnhöfer et al., 2015). This model, of course, took shape after the appearance of Kingdon’s path-breaking studies of agenda-setting, a study which in turn was heavily influenced by Cohen et al.’s (1972, 1979) earlier work on administrative practices in complex organizations. While dipping into other tasks like formulation and decision-making, however, Kingdon’s work, unlike its predecessor, had very little to say about implementation or evaluation, which largely fell outside of the ambit of his studies (Saetren, 2016). Nevertheless, as that work showed, many of its precepts could be stretched or adapted to address implementation activities.

The multiple streams framework

As is well known, in the Cohen et al.-inspired MSF approach which Kingdon developed, policy-making is examined not through tasks per se but through the study of the interactive behaviour of several sets of actors pursing particular visions of policy problems and solutions or the politics surrounding them. That is, policy processes are viewed through the lens of what Kingdon (1984) referred to as several semi-independent ‘streams’ of events and actors interacting with each other to define and control the policy agenda.

In Kingdon’s model, three quasi- or semi-independent ‘streams’ of political, problems and policy (solutions) events and activities periodically flow together across realms. Not to be ever mistaken for a more calm and technocratic ‘policy cycle’, the streams model stresses the constant complexity of agenda-setting behaviour, its occasional chaos and sometimes highly contingent nature, facets sometimes lost in the cycle approach (Colebatch, 2006). In this understanding, policy development does not occur automatically or spontaneously in response to a social problem as the cycle model sometimes seems to suggest, but rather emerges from a much more complex and contingent process as the result of the interaction and intersection of the three streams, which lead to certain issues being taken up by governments and not others; defining their agendas and future activities.

Kingdon’s operative idea in his work was that in certain circumstances, sometimes driven by institutional events such as budgetary or legislative deadlines, or by focusing events such as airplane crashes or earthquakes, the three streams would join together to provide a window of opportunity for entrepreneurs to move their preferred issues and solutions onto government agendas (Birkland, 1997, 1998; Mintrom, 1997; Mintrom and Norman, 2009). Although the exact timing of some of these occurrences might be fortuitous, at other times they would be more or less predictable, such as immediately following an election (Howlett, 1998).

Kingdon’s own ideas about policy streams, however, dealt primarily with one specific policy task, agenda-setting. As such, many of the other tasks in policy-making, including implementation, receive almost no treatment in his work. This was especially noticeable in the case of implementation, which is a subject with which MSF theorists rarely if ever deal, but is also true of evaluation and, somewhat less so, of formulation. While often promoted as a rival to the policy cycle framework, then, Kingdon’s work, while pathbreaking and innovative, remains at best a partial theory, with evidence of stream dynamics in two important policy tasks – agenda-setting and decision-making as a whole – but weak or non-existent evidence in other areas. However, this was not always the case with ‘streams’-based studies, as Cohen et al.’s (1972, 1979) earlier work had emphasized its utility in understanding administrative behaviour, albeit by adding a fourth ‘choice opportunity’ stream. Although Kingdon ignored this additional stream, it does help provide the basis for the expansion of his work to address implementation issues, as is discussed further below.

The advocacy coalition framework

Similarly, a second well known alternative model to the stages one, the ACF, was put forward by Sabatier and his colleagues (Sabatier, 1987, 1988; Sabatier and Jenkins-Smith, 1993) in the late 1980s and 1990s. The ACF model centres on policy formulation activities and the roles played by actors sharing common beliefs in articulating and promoting specific definitions of problems and the means to solve them.

Sabatier developed the ACF specifically out of a critique of the cycle model in a landmark series of articles beginning in the mid-1980s (Sabatier, 1987, 1988). The ACF essentially posited that focusing on the beliefs motivating collective action within a subsystem would generate a superior understanding of the conflict inherent within policy-making by comparison to the ‘actorless’ vision of the staged cycle approach (Weible and Nohrstedt, 2011; Weible et al., 2009, 2011).

Like the MSF, the ACF postulated a much messier policy process than typically envisioned by problem-centred cycle theory, one in which duelling coalitions of actors vied to have their policy-related ideas adopted in practice. In this model, the gravitational pull that draws actors into a particular coalition is exerted by core beliefs, grounded in deeply held normative values about the way the world works, or should work (Sabatier, 1988). The political rivalry between these coalitions over time served to establish the contours and content of policies. Policy-making was thus much less about the unfolding of a sequence of problem-solving activities on the part of disinterested actors than about how coalitions formed and engaged their competitors and how that process led one to establish hegemony over problem definitions and policy alternatives (Weible, 2005; Zafonte and Sabatier, 1998).

In an interesting twist, however, the ACF framework said almost nothing about implementation, despite Sabatier’s expertise, experience and many writings in this area. And this is also despite the fact that the principles of the ACF were induced from extensive content analysis of public input records and associated Congressional hearings into federal rulemaking on environmental and natural resource policies, as well as case studies of policy-making in areas such as agriculture and the environment in states such as California (Zafonte and Sabatier, 1998).

Again, instead, as with Kingdon, the central focus of analysis was placed upon the ‘front-end’ of policy-making, namely agenda-setting and formulation. Although it would make sense to include implementers as members of specific coalitions, most ACF studies treated civil servants, in classic pluralist style, as above the fray or as a kind of neutral arbitrator or referee of coalition struggles. Nevertheless, the ACF model did focus attention upon important topics such as the role of ideas, learning and coalition behaviour in policy-making which orthodox models of policy cycles tended to ignore or downplay.

From this brief discussion, it is clear that each of the approaches cited above has some strengths and weaknesses when it comes to explaining some aspects of policy-making activity and how the policy process, including implementation, unfolds. But, more to the point, these strengths and weaknesses are related in the sense that the strengths of one approach help overcome the limits of another.

Thus, for example, the MSF approach helps to explain the timing of movements within the policy cycle, an outstanding question with respect to the latter model. And the ACF model helps clarify the nature of ideas which subsystem actors have and which guide their behaviour, a weakness of MSF theorizing which tends towards endorsing random, chaotic and anarchistic models of policy-making. And, while helpful in specifying who was involved in policy-making and how they interacted, for example, the strength of the ACF model came at the expense of ignoring the decision making and implementation processes, which are well described by policy cycle theory, either assuming these were unproblematic or reverting to a pre-Lasswellian ‘black box’ in which the inputs formulated by a successful coalition somehow were melded together to produce policy outcomes. Policy evaluation was seen to exist as a form of learning (Bennett and Howlett, 1992) in which feedback from outputs affects subsequent inputs, but this vision was not clearly linked to the ACF belief structures set out above.

Thus the policy cycle model sets out a more comprehensive view of the policy process than does either of the other two approaches and prima facie, therefore, it is reasonable to suggest that a merger of elements of these three approaches into a revised cycle model may provide a superior understanding of policy processes, especially agenda-setting and the formulation activities with which the ACF and MSF specifically deal, but also other stages such as decision-making, implementation and evaluation with which cycle theory deals, than any single approach taken on its own terms. Taken together with the insights of the policy cycle framework the MSF and ACF approaches can help provide the basis for an approach to studying and understanding policy implementation that can overcome some of the problems listed above with both process theories and with other approaches to implementation borrowed from organization theory, public administration and public management.

Merging the cycle, multiple streams and advocacy coalition approaches to help better situate and understand policy implementation activity

A reconciliation of streams, advocacy coalition and cycles models becomes possible once it is recognized that rather than being understood as conceptual rivals, each model yields complementary and cumulative insights into how policies are made. This is especially significant for implementation since this merger yields further appreciation of the policy process as a whole, enhancing the utility of the cycle framework by addressing astute and longstanding criticisms of its lack of clear and realistic agency, oversimplified depictions of policy deliberation, and obscure drivers of change (Colebatch, 2005; Sabatier, 1991).

Some scholars, for example, have urged a simple direct extension of the multiple streams approach to cover all policy-making activities (Barzelay, 2006; Guldbrandsson and Fossum, 2009). Although Kingdon defined his task quite narrowly and only sought to explain how issues moved onto government agendas and became targets for action, rather than how, for example, solutions were decided upon and implemented or put into action, with some modifications his analysis of agenda-setting dynamics can be applied to other stages of the policy process (Howlett et al., 2014).

These modifications are required if this merger is to occur, however. Using a combination of elements from each model advances both policy thinking and thinking about implementation better than does the existing penchant towards the continued use of any single model or the continued atheoretical borrowing of alternative models of organizational behaviour from fields such as public administration or public management (Barzelay, 2006; Zahariadis, 2003).

First, using the ACF perspective, we can conceive of a stream as being shaped to some degree by competing coalitions of interests within a policy subsystem. The problem stream, for example, can be seen to be shaped to a substantial degree by the outcome of the interactions of competing coalitions of interests (from scientific experts to lobby groups and public servants) who seek others’ acceptance of their authoritative definition of ‘policy problems’. Similarly, we can see that Kingdon’s core assumption of the existence of only three streams, for example, does not travel well to the other stages of policy-making beyond agenda-setting such as implementation or evaluation when other central actors such as program administrators and policy evaluators are active. We can identify a fourth ‘process stream’ shaped by coalitions who contest the most appropriate process to be followed from consultation with citizens to concerns about the best administrative practices to follow in implementing policies (Mukherjee and Howlett, 2015). This process stream is similar to the ‘choice opportunities’ stream first mooted by Cohen et al. (1972, 1979).

Similarly, policy instrument theory suggests that not only is there a powerful coalition of interests shaping procedures for implementation but that their focus can be seen to be upon specific set of policy instruments adopted to implement a policy. The dynamics of their interactions in this ‘program’ stream can lead to outcomes which may range from classic and idealized top-down implementation where the dominant coalition has provides little wiggle room for ‘street-level bureaucrats’ and others involved in the implementation process, to a much more contested implementation process where competing coalitions vie over procedures and open up room for more street-level discretion (Howlett et al., 2009).

This five-stream framework retains the conceptual architecture and analytical vocabulary developed by Kingdon but builds on cycle and ACF thinking to build a more comprehensive framework, one capable of capturing the full range of policy-making dynamics across each stage of the policy process (Howlett et al., 2015; Mukherjee and Howlett, 2015). Each of these component streams and how they relate to implementation is set out below.

The process stream

The first of these streams is the overall ‘process’ one, which has been discussed above in the context of policy cycle theory. It establishes the basic set of tasks and events which lead to policy outputs, including the various stages noted earlier through which policies are processed, from agenda-setting to evaluation (Howlett et al., 2009).

The problem stream

The second stream is the ‘problem’ stream identified by Kingdon. This stream remains focused on the articulation or framing of policy problems and involves a key set of actors involved in defining policy issues: ‘epistemic communities’. Developed in the international relations literature to describe groups of scientists involved in articulating and delimiting problem spaces in areas such as oceans policy and climate change (Gough and Shackley, 2001; Haas, 1992; Zito, 2001), this term can be used as a descriptor of the principle set of actors involved in problem definition.

These problem-defining actors are involved in earlier stages of policy-making, such as agenda-setting and formulation, but also implementation. They include actors from scientists to political partisans, and others depending on the case, who remain active beyond agenda setting and into policy formulation and are engaged in discourses which lead to the definition of broad or specialized implementation issues or problems (Cross and Davis, 2015; Hajer, 1997, 2005; Howlett et al., 2009; Knaggård, 2015).

The policy stream

As the ACF rightly noted, knowledge regarding a policy problem is the ‘glue’ that unites actors within an epistemic community, differentiating it from those actors involved in political negotiations and practices around policy goals and solutions, discussed below, as well as those actors who specialize in the development, design and articulation of policy tools or solutions (Biddle and Koontz, 2014). This second group of actors involved in implementation is comprised of those members of ‘instrument constituencies’ whose focus is much less upon problems than upon solutions (Voß and Simons, 2014).

That is, unlike epistemic communities that pursue the translation of broad issues into distinct problems that policymakers can act upon, constituency members are more concerned with policy tools and supplying policymakers with the information about the design and mechanics of these tools (Béland et al., 2018). These constituencies are thus ‘networks of heterogeneous actors from academia, policy consulting, public policy and administration, business, and civil society, who become entangled as they engage with the articulation, development, implementation and dissemination of a particular technical model of governance’ (Voß and Simons, 2014). Think-tanks, for example, fall into this category, as they provide policymakers with ‘basic information about the world and societies they govern, how current policies are working, possible alternatives and their likely costs and consequences’ (McGann et al., 2014: 31).

Such constituencies advocate for particular tools or combinations of tools to address a range of problem areas and hence are active in the ‘policy’ stream Kingdon identified, one that heightens in activity as policy alternatives and instruments are formulated and combined to address policy aims as policy implementation proceeds. They form conscious groupings attempting to realize their particular version of the instrument and develop ‘a discourse of how the instrument may best be retained, developed, promoted and expanded’ (Voß and Simons, 2014).

The politics stream

The ‘politics’ stream can be thought of as a third stream of events; being the milieu where ‘advocacy coalitions’ as Sabatier and others identified them (Sabatier and Weible, 2007; Schlager and Blomquist, 1996) are most active. These actors compete to get their choice of problem definitions as well as solutions adopted and implemented and continue to act throughout implementation activities.

Such politically active policy actors are usually more publicly visible than members of epistemic communities or instrument constituencies. More visible actors of the politics stream can include, for example, in the case of the US Congress Kingdon examined, ‘the president and his high-level appointees, prominent members of the congress, the media and such elections-related actors as political parties and campaigns’ (Kingdon, 1984) while less visible actors include lobbyists, political party brokers and fixers, and other behind-the-scenes advisors and participants at work in implementation, for example, through lobbying to reduce regulatory enforcement or enhance subsidies (Weishaar et al., 2015). All of these, of course, play a significant role at many different stages of policy-making, including implementation.

The programme stream

The fifth stream, one which is crucial to implementation, features the introduction of new ‘programme’ actors, mainly administrators but also some members of affected publics and stakeholders involved in delivering, distributing or consuming government supplied or affected goods and services. Of course, policy implementation often relies on civil servants and administrative officials to establish and manage the necessary actions and they are key actors in the programme stream who apply their knowledge, experience, expertise and values to shaping the launch and evolution of programs implementing policy decisions. Indeed, bureaucrats are typically the most significant actors in most policy implementation, bringing the endemic intra- and inter-organizational conflicts of public agencies to the fore of this stage in the policy cycle (Dye, 2001). Different bureaucratic agencies at various levels of government (national, state or provincial and local) are usually involved in implementing policy, each carrying particular interests, ambitions, and traditions that affect the implementation process and shape its outcomes, in a process of ‘multi-level’ government or governance (see Bache and Flinders, 2004; Bardach, 1977; Elmore, 1978). These form the principle base of the ‘programme stream’.

Non-governmental actors who are part of the policy subsystem can also be involved in implementation activities as occurs in instances of co-production and shared or collaborative service delivery. Although usually only a narrow range of subsystem actors are commonly involved in the implementation process, in some countries, like Sweden, there may be a tradition of non-governmental actors directly implementing some important social programs (Ginsburg, 1992; Johansson and Borell, 1999). In other countries, like the US, which have only recently attempted to implement some programs through community and religious (‘faith-based’) groups (Kuo, 2006), non-governmental actors are typically involved in the design and evaluation of policies and less in their actual administration and management.

Epistemic communities, instrument constituencies and members of political coalitions all continue to be actively involved in implementation activities and can be members of this stream. They can continue to influence and constrain programme design and designers as often so do members of political coalitions. This is most apparent in some countries with corrupt or highly politicized or clientelistic administrations which may be tightly wound around the programme streams but is also the case in less venal situations when more diffuse interests may be at stake, such as public perceptions of government activities or more diffuse electoral concerns, among others.

Applying the framework to implementation activities

In this five-stream framework, each confluence point in the policy process can be viewed as bringing something new together, such as new actors, new tactics, new resources who join the flow of policymaking events in the ‘process’ stream which unfolds after an issue enters onto a government agenda (see Figure 1).

Figure 1.

Five thread (stream) model of policy process.

As shown in Figure 1, when each stream, including a ‘programme’ stream of implementation actors, intersects, the merger point represents a ‘window’ in Kingdon’s sense, and yields a different configuration of policy inputs that generate a distinct policy pattern through each particular juncture.

Much as the ‘rounds’ theories of policy-making have suggested (Klijn and Teisman, 1991; Timmermans, 2001), this new process model helps to situate implementation activities firmly within the larger flow of policy-making and helps to understand its distinct modalities and problems, such as the game-like nature of regulatory compliance and the continual principal agent problems encountered in effective implementation.

In this way of thinking, the first confluence point occurs in agenda-setting much as Kingdon suggested, when the three problem, politics and policy streams coalesce temporarily in the typical ‘policy window’ fashion that he described. This intersection creates a new policy process stream which becomes the main or central pathway upon which other streams subsequently converge, creating critical junctures which set up the future impetus for policy deliberations and establish the initial conditions which animate subsequent policy process advances (or retreats); essentially becoming a fourth ‘choice’ stream first mooted by Cohen et al. in their early writing which inspired Kingdon is his later work (1972, 1979).

After this critical agenda-setting process has occurred, the political stream separates from the problem and policy streams as specific sets of subsystem actors such as policy analysts and stakeholders organized in advocacy coalitions contribute to deliberations and propose policy alternatives (Craft and Howlett, 2013). This mobilization of ideas about what to do continues until a second critical point occurs once these actors have blended policy problems and solutions together creating some configuration or policy array comprised of alternative choice possibilities. The contents of this array provide the basis for a new phase when the politics streams return to connect with the process stream creating momentum towards a decision.

Significantly for policy implementation studies, the third critical point occurs if and when a decision is taken, and policy then requires implementation. At this point, the ‘policy’ stream separates from the main flow which is comprised of the process, politics and problem streams and is now joined by a program stream composed of the actors and interests working to calibrate new program instruments (and integrating or alternating them with established ones) to generate new outputs.

Conclusion: Understanding implementation processes as critical junctures and stream crossings

A pivotal feature of policy studies since the mid-1980s has been the development and use of several different analytical frameworks to help capture the main characteristics and dynamics of policy processes (Pump, 2011; Cairney 2012; Cairney and Jones 2015; John 2012, 2013). These frameworks are oriented towards moving beyond the particularities of policy-making events to the description of generic factors and mechanisms which underlie such processes and lead to policy changes and outcomes of different kinds. This has been done in such a way as to guide investigators and help both students and practitioners make sense of the complex set of socio-political activities which constitute policy-making as well as help explain its outputs and outcomes (Althaus et al., 2013; Cairney, 2013; Howlett et al., 2009).

While most policy decisions identify some means to pursue their goals, implementation is required to attain results and many studies from the earliest days of the policy sciences have stressed the significance and importance of effective (and ineffective) implementation in affecting policy outputs. While it is clear that the activities associated with this phase of the policy-making process should be integrated into policy process models, in general, they are not. None of the mainstream policy frameworks cited above has been used to aid or further the study of policy implementation despite its similarities with other stages such as agenda-setting and policy formulation to which they have been deployed. Rather, as stated above, implementation has inherited a tradition of atheoretical work in public administration, exacerbated by the more recent addition of an equally descriptive set of works in public management and, more recently, it has been addressed with a set of proto-theories competing for attention – from network management to principal–agent theory, game theory, to instrument choice approaches and others – none of which has an explicitly policy process focus.

This situation needs to be rectified if implementation is to assume its rightful place near the heart of policy-making and if implementation studies are to contribute wholeheartedly to policy studies. The model developed here combines many elements of existing policy process frameworks and links them to earlier work on implementation adopted by the policy sciences from fields such as public administration, management and law. It provides a model of policy-making in which implementation is fully integrated and where both policy studies benefit from the insights of implementation studies and implementation studies from those of the policy sciences.

Above all, this model captures the fact that implementation by public agencies is often an expensive, multi-year effort, meaning that continued funding for programs and projects is usually neither permanent nor guaranteed but rather requires continual negotiation and discussions within and between the political and administrative arms of the state. This creates opportunities for politicians, agencies and other members of policy subsystems to use the implementation process as another opportunity for continuing the conflicts they may have lost at earlier stages of the policy process. Such processes greatly complicate implementation and move it far away from being simply a ‘technical’ issue of decision-processing (Keyes, 1996; Nicholson-Crotty, 2005; Ziller, 2005) as many students of public policy-making first suggested. Such a complex reality requires a fully-fledged model of policy implementation highlighting the different actors involved and their interests if the nuances and dynamics of this critical stage of policy-making are to be better understood and fully incorporated into the policy sciences.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Althaus

Bridgman

Davis

(2013) The Australian Policy Handbook, Sydney: Allen and Unwin.

Anderson

(1975) Public Policy Making, New York: Praeger.

Bache

Flinders

(2004) Multi-Level Governance, New York: Oxford University Press.

Banks

(1995) The design of institutions. In: Weimer

(ed.) Institutional Design, Boston: Kluwer, pp. 17–36.

Bardach

(1977) The Implementation Game: What Happens after a Bill Becomes a Law, Cambridge, MA: MIT Press.

Barrett

(2004) Implementation studies; time for a revival? Personal reflections on 20 years of implementation studies. Public Administration 82(2): 249–262.

Barzelay

(2006) Introduction: The process dynamics of public management policymaking. International Public Management Journal 6(3): 251–282.

Béland

Howlett

Mukherjee

(2018) Instrument constituencies and public policy-making: An introduction. Policy and Society 37(1): 1–13.

Bennett

Howlett

(1992) The lessons of learning: Reconciling theories of policy learning and policy change. Policy Sciences 25(3): 275–294.

10.

Bernstein

(1955) Regulating Business by Independent Commission, Princeton: Princeton University Press.

11.

Biddle

Koontz

(2014) Goal specificity: A proxy measure for improvements in environmental outcomes in collaborative governance. Journal of Environmental Management 145: 268–276.

12.

Birkland

(1997) After Disaster: Agenda Setting, Public Policy and Focusing Events, Washington, DC: Georgetown University Press, 1997.

13.

Birkland

(1998) Focusing events, mobilization, and agenda setting. Journal of Public Policy 18(1): 53–74.

14.

Bobrow

(2006) Policy Design: Ubiquitous, Necessary and Difficult. In: Peters

Pierre

(eds) Handbook of Public Policy, SAGE, pp. 75–96.

15.

Bozeman

(1993) Public Management: The State of the Art, San Francisco: Jossey-Bass.

16.

Bressers

Klok

(1988) Fundamentals for a theory of policy instruments. International Journal of Social Economics 15(3/4): 22–41.

17.

Bressers

HTA

(1998) The choice of policy instruments in policy networks. In: Peters

Nispen

FKMV

(eds) Public Policy Instruments: Evaluating the Tools of Public Administration, New York: Edward Elgar, pp. 85–105.

18.

Bressers

HTA

O'Toole

(2005) Instrument Selection and Implementation in a Networked Context. In: Eliadis

Hill

Howlett

(eds) Designing Government: From Instruments to Governance, Montreal: McGill-Queen's University Press, pp. 135–153.

19.

Bressers

HTA

O’Toole

(1998) The selection of policy instruments: A network-based perspective. Journal of Public Policy 18(3): 213–239.

20.

Breunig

Koski

Workman

(2016) Knot policy theory. Policy Studies Journal 44(S1): S123–S132.

21.

Brewer

deLeon

(1983) The Foundations of Policy Analysis, Homewood: Dorsey.

22.

Brewer

(1974) The policy sciences emerge: To nurture and structure a discipline. Policy Sciences 5: 239–244.

23.

Bridgman

Davis

(2003) What use is a policy cycle? Plenty, if the aim is clear. Australian Journal of Public Administration 63(3): 98–102.

24.

Burton

(2006) Modernising the policy process: Making policy research more significant? Policy Studies 27(3): 172–192.

25.

Cairney

(2012) Understanding Public Policy, Basingstoke, UK: Palgrave.

26.

Cairney

(2013) Standing on the shoulders of giants: How do we combine the insights of multiple theories in public policy studies? Policy Studies Journal 41(1): 1–21.

27.

Cairney

Jones

(2015) Kingdon’s multiple streams approach: What is the empirical impact of this universal theory? Policy Studies Journal 44(1): 37–58.

28.

Calvert

McCubbins

Weingast

(1989) A theory of political control and agency discretion. American Journal of Political Science 33(3): 588–611.

29.

Cohen

March

Olsen

(1972) A garbage can framework of organizational choice. Administrative Science Quarterly 17(1): 1–25.

30.

Cohen

March

Olsen

(1979) People, problems, solutions and the ambiguity of relevance. In: March

Olsen

(eds) Ambiguity and Choice in Organizations, Bergen: Universitetsforlaget, pp. 24–37.

31.

Colebatch

(2005) Policy analysis, policy practice and political science. Australian Journal of Public Administration 64(3): 14–23.

32.

Colebatch

(2006) Beyond the Policy Cycle: The Policy Process in Australia, Crow’s Nest: Allen and Unwin.

33.

Craft

Howlett

(2013) Policy advisory systems and evidence-based policy: The location and content of evidentiary policy advice. In: Young

(ed.) Evidence-Based Policy-Making in Canada, Toronto: University of Toronto Press, pp. 27–44.

34.

Cross

Davis

(2015) The limits of epistemic communities: EU security agencies. Politics and Governance 3(1): 90.

35.

deLeon

(1999) The stages approach to the policy process: What has it done? Where is it going?”. In: Sabatier

(ed.) Theories of the Policy Process., Boulder: Westview, pp. 19–34.

36.

Durlak

DuPre

(2008) Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology 41(3–4): 327–350.

37.

Dye

(2001) Top Down Policymaking, New York: Chatham House, 2001.

38.

Ellig

Lavoie

(1995) The principal–agent relationship in organizations. In: Foss

(ed.) Economic Approaches to Organizations and Institutions: An Introduction, Aldershot: Dartmouth, pp. 267–295.

39.

Elmore

(1978) Organizational models of social program implementation. Public Policy 26(2): 185–228.

40.

Elmore

(1987) Instruments and strategy in public policy. Policy Studies Review 7(1): 174–186.

41.

Enkler

Schmidt

Eckhard

, et al.(2017) Administrative styles in the OECD: Bureaucratic policy-making beyond formal rules. International Journal of Public Administration 40(8): 637–648.

42.

Francis

(1993) The Politics of Regulation: A Comparative Perspective, Oxford: Blackwell.

43.

Gaus

(1931) Notes on administration. American Political Science Review 25(1): 123–134.

44.

Ginsburg

(1992) Divisions of Welfare: A Critical Introduction to Comparative Social Policy, London: Sage.

45.

Goggin

, et al.(1990) Implementation Theory and Practice: Toward a Third Generation, Glenview, IL: Scott, Foresman/Little, Brown.

46.

Goodnow

(1900) Politics and Administration: A Study in Government, New York: Russell and Russell.

47.

Gough

Shackley

(2001) The respectable politics of climate change: The epistemic communities and NGOs. International Affairs 77: 329–346.

48.

Guldbrandsson

Fossum

(2009) An exploration of the theoretical concepts policy windows and policy entrepreneurs at the Swedish Public Health arena. Health Promotion International 24(4): 434–444.

49.

Haas

(1992) Introduction: Epistemic communities and international policy coordination. International Organization 14(1): 1–36.

50.

Hajer

(1997) The Politics of Environmental Discourse: Ecological Modernization and the Policy Process, Oxford, UK: Oxford University Press.

51.

Hajer

(2005) Setting the stage: A dramaturgy of policy deliberation. Administration & Society 36(6): 624–647.

52.

Hammond

Krause

Meier

(2003) Veto points, policy preferences, and bureaucratic autonomy in democratic systems. In: Politics, Policy and Organizations: Frontiers in the Scientific Study of Bureaucracy, Ann Arbor: University of Michigan Press, pp. 73–103.

53.

Hargrove

(1975) The Missing Link: The Study of the Implementation of Social Policy, Washington, DC: The Urban Institute.

54.

Hawkins

(1984) Environment and Enforcement: Regulation and the Social Definition of Pollution, Oxford: Clarendon Press, 1984.

55.

Hawkins

Thomas

(1989) Making policy in regulatory bureaucracies. In: Hawkins

Thomas

(eds) Making Regulatory Policy, Pittsburgh: University of Pittsburgh Press, pp. 3–30.

56.

Hill

Hupe

(2002) Implementing Public Policy: Governance in Theory and in Practice London: SAGE.

57.

Hjern

(1982) Implementation research – The link gone missing. Journal of Public Policy 2(3): 301–308.

58.

Hood

(1986) The Tools of Government, Chatham: Chatham House Publishers.

59.

Howlett

(1991) Policy instruments, policy styles, and policy implementation: National approaches to theories of instrument choice. Policy Studies Journal 19(2): 1–21.

60.

Howlett

(1998) Predictable and Unpredictable Policy Windows: Institutional and Exogenous Correlates of Canadian Federal Agenda-Setting. Canadian Journal of Political Science/Revue Canadienne de Science Politique 31(3): 495–524.

61.

Howlett

McConnell

Perl

(2015) Streams and stages: Reconciling Kingdon and policy process theory. European Journal of Political Research 54(3): 419–434.

62.

Howlett

McConnell

Perl

(2017) Moving policy theory forward: Connecting multiple stream and advocacy coalition frameworks to policy cycle models of analysis. Australian Journal of Public Administration 76(1): 65–79.

63.

Howlett

McConnell

Perl

(2016) Weaving the fabric of public policies: Comparing and integrating contemporary frameworks for the study of policy processes. Journal of Comparative Policy Analysis: Research and Practice 18(3): 273–289.

64.

Howlett

Ramesh

Perl

(2009) Studying Public Policy: Policy Cycles & Policy Subsystems, 3rd edn. Toronto: Oxford University Press.

65.

Hupe

Sætren

(2015) Comparative implementation research: Directions and dualities. Journal of Comparative Policy Analysis: Research and Practice 17(2): 93–102.

66.

Hupe

Hill

(2016) And the rest is implementation. Comparing approaches to what happens in policy processes beyond great expectations. Public Policy and Administration 31(2): 103–121.

67.

Jann

Wegrich

(2007) Theories of the policy cycle. In: Fischer

Miller

Sidney

(eds) Handbook of Public Policy Analysis: Theory, Politics and Methods, Boca Raton: CRC Press, pp. 43–62.

68.

Johansson

Borell

(1999) Central steering and local networks: Old-age care in Sweden. Public Administration 77(3): 585–598.

69.

John

(2012) Analyzing Public Policy, New York: Routledge.

70.

John P (2013) New directions in public policy: Theories of policy change and variation reconsidered. DOI: 10.2139/ssrn.2286711.

71.

Jones

(1984) An Introduction to the Study of Public Policy, Monterey, California: Brooks/Cole.

72.

Kagan

(1996) The political construction of American adversarial legalism. In: Ranney

(ed.) Courts and the Political Process, Berkeley: Institute of Governmental Studies Press, pp. 19–39.

73.

Kagan

(1994) Regulatory Enforcement. In: Rosenbloom

Schwartz

(eds) Handbook of Regulation and Administrative Law, New York: Marcel Dekker, pp. 383–422.

74.

Keyes

(1996) Power tools: The form and function of legal instruments for government action. Canadian Journal of Administrative Law and Practice 10: 133–174.

75.

Kingdon

(1984) Agendas, Alternatives, and Public Policies, Boston: Little Brown and Company.

76.

Klijn

Teisman

(1991) Effective policymaking in a multi-actor setting: Networks and steering. In: In’T Veld

Schaap

Termeer

CJAM

, et al.(eds) Autopoiesis and Configuration Theory: New Approaches to Societal Steering, Dodrecht: Kluwer, pp. 99–111.

77.

Knaggård

(2015) The multiple streams framework and the problem broker. European Journal of Political Research 54(3): 450–465.

78.

Kuo

(2006) Tempting Faith: An Inside Story of Political Seduction, New York: Free Press.

79.

Lascomes

LeGales

(2007) Introduction: Understanding public policy through its instruments – From the nature of instruments to the sociology of public policy instrumentation. Governance 20(1): 1–21.

80.

Lasswell

(1956) The Decision Process: Seven Categories of Functional Analysis, College Park: University of Maryland.

81.

Lasswell

(1971) A Pre-View of Policy Sciences, New York: American Elsevier.

82.

Lester

Goggin

Bowman

, et al.(1987) Public policy implementation: Evolution of the field and agenda for future research. Policy Studies Review 7: 200–216.

83.

Lester

Goggin

(1998) Back to the future: The rediscovery of implementation studies. Policy Currents 8(3): 1–9.

84.

Linder

Peters

(1987) A design perspective on policy implementation: The fallacies of misplaced prescription. Review of Policy Research 6(3): 459–475.

85.

Linder

Peters

(1989) Instruments of government: Perceptions and contexts. Journal of Public Policy 9(1): 35–58.

86.

Linder

Peters

(1991) The logic of public policy design: Linking policy actors and plausible instruments. Knowledge in Society 4: 125–151.

87.

Lipsky

(1980) Street-Level Bureaucracy: Dilemmas of the Individual in Public Services, New York: Russell Sage Foundation.

88.

Lyden

Shipman

Wilkinson

, et al.(1968) Decision-flow analysis: A methodology for studying the public policy-making process. In: Le Breton

(ed.) Comparative Administrative Theory, Seattle: University of Washington Press, pp. 155–168.

89.

Mayntz

(1979) Public bureaucracies and policy implementation. International Social Science Journal 31(4): 633–645.

90.

Mayntz

(1983) The conditions of effective public policy: A new challenge for policy analysis. Policy & Politics 11: 123–143.

91.

Mazmanian

Sabatier

(1983) Implementation and Public Policy, Glenview: Scott, Foresman.

92.

McCubbins

Lupia

(1994) Learning from oversight: Fire alarms and police patrols reconstructed. Journal of Law, Economics and Organization 10(1): 96–125.

93.

McCubbins

Noll

Weingast

(1987) Administrative procedures as instruments of political control. Journal of Law, Economics, and Organization 3(2): 243–277.

94.

McCubbins

Noll

Weingast

(1989) Structure and process, politics and policy: Administrative arrangements and the political control of agencies. Virginia Law Review 75(2): 431–482.

95.

McCubbins

Schwartz

(1984) Congressional oversight overlooked: Police patrols versus fire alarms. American Journal of Political Science 28(1): 165–179.

96.

McDonnell

Elmore

. Alternative Policy Instruments.. Center for Policy Research in Education; Santa Monica, CA: Publications Department, Rand Corporation, 1987.

97.

McGann

Viden

Rafferty

(2014) How Think Tanks Shape Social Development Policies, Philadelphia, PA: University of Pennsylvania Press.

98.

Milward

Provan

(1998) Principles for controlling agents: The political economy of network structure. Journal of Public Administration Research and Theory 8(2): 203–222.

99.

Mintrom

(1997) Policy entrepreneurs and the diffusion of innovation. American Journal of Political Science 41(3): 738–770.

100.

Mintrom

Norman

(2009) Policy entrepreneurship and policy change. Policy Studies Journal 37: 649–667.

101.

Mukherjee

Howlett

(2015) Who is a stream? Epistemic communities, instrument constituencies and advocacy coalitions in public policy-making. Politics and Governance 3(2): 65–75.

102.

Nicholson-Crotty

(2005) Bureaucratic competition in the policy process. Policy Studies Journal 33(3): 341–361.

103.

O’Toole

(2000) Research on policy implementation: Assessment and prospects. Journal of Public Administration Research and Theory 10(2): 263–288.

104.

O’Toole

(2004) The theory-practice issue in policy implementation research. Public Administration 82(2): 309–329.

105.

Pressman

Wildavsky

(1973) Implementation: How Great Expectations in Washington Are Dashed in Oakland, Berkeley: University of California Press.

106.

Pump

(2011) Beyond metaphors: New research on agendas in the policy process. Policy Studies Journal 39: 1–12.

107.

Rothmayr

Serduelt

Maurer

(1997) Policy instruments: An analytical category revised. In: ECPR Workshop No. 21: Towards a More Integrative Approach in Policy Studies, Switzerland: Bern.

108.

Sabatier

(1987) Knowledge, policy-oriented learning, and policy change. Knowledge: Creation, Diffusion, Utilization 8(4): 649–692.

109.

Sabatier

Jenkins-Smith

(1993) The advocacy coalition framework: Assessment, revisions, and implications for scholars and practitioners. In: Policy Change and Learning: An Advocacy Coalition Approach, Boulder: Westview, 1993, pp. 211–236.

110.

Sabatier

(1975) Social movements and regulatory agencies: Toward a more adequate – and less pessimistic – Theory of ‘clientele capture.’. Policy Sciences 6: 301–342.

111.

Sabatier

(1986) Top-down and bottom-up approaches to implementation research: A critical analysis and suggested synthesis. Journal of Public Policy 6: 21–48.

112.

Sabatier

(1988) An advocacy coalition framework of policy change and the role of policy-oriented learning therein. Policy Sciences 21(2/3): 129–168.

113.

Sabatier

(1991) Toward better theories of the policy process. PS: Political Science and Politics 24(2): 144–156.

114.

Sabatier

Mazmanian

(1981) Effective Policy Implementation, Lexington: Lexington Books.

115.

Sabatier

Weible

(2014) Theories of the Policy Process, 3rd edn. Boulder, CO: Westview Press, 2014.

116.

Sabatier

Weible

(2007) Theories of the Policy Process, Boulder: Westview Press.

117.

Saetren

(2014) Implementing the third generation research paradigm in policy implementation research: An empirical assessment. Public Policy and Administration 29(2): 84–105.

118.

Saetren

(2016) Lost in translation: Re-conceptualising the multiple-streams framework back to its source of inspiration. In: Zoelnhofer

Rüb

(eds) Decision-Making under Ambiguity and Time Constraints: Assessing the Multiple Streams Framework, Essex: ECPR Press, pp. 21–34.

119.

Salamon

(1981) Rethinking Public Management: Third-Party Government and the Changing Forms of Government Action. Public Policy 29(3): 255–275.

120.

Schlager

Blomquist

(1996) Emerging political theories of the policy process: Institutional rational choice, the politics of structural choice, and advocacy coalitions. Political Research Quarterly 49: 651–672.

121.

Schneider

Ingram

(1990) Behavioural assumptions of policy tools. Journal of Politics 52(2): 511–529.

122.

Scholz

(1984) Cooperation, deterrence, and the ecology of regulatory enforcement. Law & Society Review 18(2): 179–224.

123.

Scholz

(1991) Cooperative regulatory enforcement and the politics of administrative effectiveness. American Political Science Review 85(1): 115–136.

124.

Simmons

, et al.(1974) Policy flow analysis: A conceptual framework for comparative public policy research. Western Political Quarterly 27(3): 457–468.

125.

Spicker

(2005) Targeting, residual welfare and related concepts: Modes of operation in public policy. Public Administration 83(2): 345–365.

126.

Spicker

(2006) Policy Analysis for Practice, Bristol: The Policy Press.

127.

Timmermans

(2001) Arenas as institutional sites for policymaking: Patterns and effects in comparative perspective. Journal of Comparative Policy Analysis 3: 311–337.

128.

van Meter

van Horn

(1975) The policy implementation process: A conceptual framework. Administration and Society 6(4): 445–488.

129.

Voß

Simons

(2014) Instrument constituencies and the supply side of policy innovation: The social life of emissions trading. Environmental Politics 23(5): 735–754.

130.

Weible

(2005) Beliefs and perceived influence in a natural resource conflict: An advocacy coalition approach to policy networks. Political Research Quarterly 58: 461–475.

131.

Weible

Carter

(2017) Advancing policy process research at its overlap with public management scholarship and nonprofit and voluntary action studies. Policy Studies Journal 45(1): 22–49.

132.

Weible

Heikkila

deLeon

, et al.(2012) Understanding and influencing the policy process. Policy Sciences 45(1): 1–21.

133.

Weible

Nohrstedt

(2011) The advocacy coalition framework: Coalitions, learning and policy change. In: Araral

Fritzen

Howlett

, et al.(eds) Routledge Handbook of Public Policy, New York: Routledge, pp. 125–137.

134.

Weible

Sabatier

McQueen

(2009) Themes and variations: Taking stock of the advocacy coalition framework. Policy Studies Journal 37(1): 121–140.

135.

Weible

Sabatier

Jenkins‐Smith

, et al.(2011) A quarter century of the advocacy coalition framework: An introduction to the special issue. Policy Studies Journal 39(3): 349–360.

136.

Weishaar

Amos

Collin

(2015) Best of enemies: Using social network analysis to explore a policy network in European smoke-free policy. Social Science & Medicine 133: 85–92.

137.

Wilson

(1887) The study of administration. Political Science Quarterly 2(2): 197–222.

138.

Zafonte

Sabatier

(1998) Shared beliefs and imposed interdependencies as determinants of ally networks in overlapping subsystems. Journal of Theoretical Politics 10(4): 473–505.

139.

Zahariadis

(2003) Ambiguity & Choice in Public Policy: Political Decision-Making in Modern Democracies, Washington, DC: Georgetown University Press, 2003.

140.

Zahariadis

(1995) Markets, States, and Public Policy: Privatization in Britain and France, Ann Arbor: University of Michigan Press.

141.

Zahariadis

(2007) The multiple streams framework: Structure, limitations, prospects. In: Sabatier

(ed.) Theories of the Policy Process, Boulder: Westview, pp. 65–92.

142.

Ziller

(2005) Public law: A tool for modern management, not an impediment to reform. International Review of Administrative Sciences 71(2): 267–277.

143.

Zito

(2001) Epistemic communities, collective entrepreneurship and European integration. Journal of European Public Policy 8(4): 585–603.

144.

Zohlnhöfer

Herweg

Rüb

(2015) Theoretically refining the multiple streams framework: An introduction. European Journal of Political Research 54(3): 412–418.