Abstract
Governance increasingly relies on intergovernmental and intersectoral collaboration in providing public services. This research reports perceptions of state representatives to the multistate collaborative Drug Effectiveness Review Project about the importance of factors influencing successful collaboration. Findings reveal state motivation to collaborate is closely tied to project governance and suggest how future efforts might be structured to achieve success. Among other factors, a disconnect between ends and means allows the process to function objectively; objective production of drug effectiveness information and subjective use of information in state policy decisions are distinct processes, reducing conflict that might otherwise jeopardize project sustainability.
Introduction
At times, governments find common ground where collaborative advantage outweighs the relative costs, and they strike new iron in addressing common problems. The field of evidence-based practice has made a profound mark on policy and decision making at the state level during the past decade. The performance management movement and its concomitant drive for accountability have led decision makers to rely extensively on evidence of program effectiveness in determining agency programming. The movement has not progressed uniformly across policy areas, in part due to difficulty measuring effectiveness, determining causality, or sorting out multiple goals in some policy fields. Those areas that lend themselves easily to scientific study, particularly health policy, have been at the forefront of evidence-based practice.
The Drug Effectiveness Review Project (DERP) is a multistate collaboration that generates evidence regarding pharmaceutical effectiveness by classification. Public health research has adequately described, parsed, and criticized the DERP on various grounds, but has not explored it from the perspective of public sector collaboration. This research note explores DERP as a model of collaborative evidence generation. We use a survey of state DERP representatives to examine partner perceptions regarding the importance of various structural factors for DERP’s success. We briefly explore the literature on public sector collaboration, discuss evidence based practice, and introduce DERP before introducing our hypotheses and findings. We conclude by exploring salient opportunities for expanded research and application.
Cross-State Collaboration
Public services are increasingly provided by consortia of intergovernmental and intersectoral actors. Collaborations pool energy and funds to overcome the fragmentation of services created by a host of current practices and institutional arrangements (Bardach and Lesser 1996). Collaborations assume tasks that are paramount to concurrent actions by independent organizations. They show strong commitment to multiorganizational-level goals where resource sharing is both risky and extensive, in response to long-term problems or the emergent, sequential, redefinition of problems (McGuire 2006). Collaborative decisions are more effective because they take multiple perspectives into account, but they are less efficient because assimilation of such diverse perspectives into conclusions takes time (Kiefer and Montjoy 2006).
Collaborations that cross traditional boundaries are in high demand. As Kettl puts it, “Boundary-based solutions are out of sync with 21st century problems” (2006, 13). Reasons for collaboration include improved effectiveness or efficiency derived from a combination of economy of scale, nonduplication of effort, or collaborative advantage. They can increase the capacity of existing limited institutional arrangements to accomplish policy purposes more effectively or at a lower cost (Bardach and Lesser 1996, 221). Partners pursue collaboration to improve their ability to address important issues, acquire new funds, new competencies, and knowledge to support activities (Lasker, Weiss, and Miller 2001). Bryson, Crosby, and Stone (2006) suggest that public policy makers are more likely to attempt cross-sector collaboration when the independent efforts of different sectors to address a problem have failed, when linking mechanisms are present (such as a strong sponsor or general agreement on the problem), when formal agreements displace informal agreements (because they better support accountability), when legitimacy of the collaboration is established among both internal and external stakeholders, and when cross-sector collaborations are resilient and frequently reassessed.
Benefits notwithstanding, there are real challenges associated with forming and operating effective collaborations. There are collaborative costs, with the most common being the relinquishment of agency authority and resources (Agranoff 2006)—a high price to pay for many agencies steeped in a tradition emphasizing competition for resources across vertical boundaries (Kettl 2006). “Collaboration for collaboration’s sake or to achieve only individual goals is likely to result in failure” (Thomson and Perry 2006, 28). The capacity needed to work in networks is different than that needed to work within a single organization (Agranoff and McGuire 1998).
Evidence-Based Practice
At the state level, evidence-based practice (EBP) has become broadly influential in selected policy fields, especially public health. Evidence-based practice is a challenging endeavor for public agencies that are forced to make decisions under constraints bounded by time and resources. Available research ranges from randomized control trials with high internal validity but questions about generalizability to observational studies that are more open to bias but less costly and often better reflect real-world circumstances. Hall and Jennings (2008) point out that what constitutes evidence in a given setting tends to vary; the challenge for administrators is to evaluate their confidence in the reliability of available evidence and make choices given tradeoffs and constraints. EBP has to account for the reality of the political process and the likelihood of goal multiplicity. Any given program may have an identifiable core goal, but it may also represent a plurality of varied goals that were linked to the policy during the legislative process. Diverse goals complicate program effectiveness measurement, and evidence-based findings with respect to any single goal may offer little guidance on how to use the evidence in a setting of multiple or competing goals.
The Case: The DERP
The Collaborative Process
The DERP came into being in 2003 to prioritize and conduct systematic reviews of drug classes (Fox 2005, 117). It began in Oregon when legislation removed an existing prohibition on Oregon Medicaid’s use of preferred drug schedules. Oregon’s Office of Health Policy initially financed reviews of four drug classes, contracting with the Oregon Evidence-based Practice Center (EPC) to perform the reviews. The results quickly interested Washington and Idaho, and additional states were invited to collaborate in further reviews. The Center for Evidence-based Policy at Oregon Health Science University was established to provide administrative support for the collaboration, contracting with EPC to conduct the reviews. (For a full description of DERP’s developmental history, see Fox 2010, 77–103). The reviews DERP produces provide objective evidence that states then use to facilitate their independent decision-making processes. Collaboration is not static, and so this research note reports a snapshot of DERP as it existed in fall, 2009. At that time, DERP included fourteen states and a Canadian nonprofit foundation (AR, Canadian Agency for Drugs and Technologies in Health, CO, ID, KS, MI, MN, MO, MT, NC, NY, OR, WA, WI, and WY).
Agreement on the use of the evidence resulting from the review is not necessary and is explicitly kept out of DERP’s domain. “Reports are not usage guidelines, nor should they be read as an endorsement of, or recommendation for, any particular drug, use or approach” (DERP “Documents” 2007). According to Neumann (2006), systematic reviews—at best—clarify the evidence base; they do not tell policy makers what to do and they do not ensure good decision making. The collaboration is not replacing the effort of independent agencies; implementation is left out of the partnership’s scope and the process provides little interference with state decision making. In this way, it also avoids antagonizing states over their policy decisions.
DERP does not take cost into account in its reviews; in fact, it shifts the debate from a focus on financing and costs to the benefits provided by each drug within a class (Neumann 2006). Each drug class review is based on careful assessment of evidence from multiple studies. In selecting studies for inclusion in the review, reviewers are attentive to the scientific quality of the study, with an emphasis on internal validity (including study characteristics such as methods used to enroll patients, assign people to treatment groups, assess outcomes, and analyze data).
Oregon Health Science University, the physical home for the project, and participating sponsors are not endorsing particular uses of the evidence reviews by the states. Each partner has its own local process to decide how to use the information provided by the reviews. Despite this, the project collaborators clearly intend for the reviews to guide practice in state-operated health programs like Medicaid and the Child Health Insurance Program.
Theoretical Development: Determinants of Collaboration and its Success
Our review of the public management, collaboration, and EBP literatures suggests fourteen factors that facilitate collaboration; we develop hypotheses based on these factors to examine their impact on collaborative success from the vantage point of DERP participants. We explore each factor and present its associated hypothesis in turn.
Professionalized policy area. Public health rose to professional status during the twentieth century (Adler, Kwon, and Heckscher 2008). Professionalization within a policy area means shared commitment to a set of values that form a sense of community that enforces norms of behavior (Streib 2005; Adler, Kwon, and Heckscher 2008). Professional policy areas are characterized as apolitical, emphasizing scientific or cost–benefit criteria such as efficiency, and thus subject to less conflict (Bourdeaux 2008). The presence of a professional, rather than a political, policy environment is expected to lead to less conflict, facilitating agreement on approach and tactics. Hypothesis 1: Deference to scientists on scientific questions facilitates collaboration.
Economy of scale. The advantage of cooperation with respect to costs has been well documented, especially in regionalism. Fewer units providing services are argued to contribute to scale economies (Olberding 2002; Zeemering 2009). It stands to reason that collaboration to provide shared knowledge—a public good—would improve scale economies that reduce costs relative to independent production. Hypothesis 2: Reduced cost to participants facilitates collaboration.
Absence of an external provider. The traditional argument for government intervention in the event of market failure explains the public sector provision of public goods. The nonrival nature of public goods creates a disincentive for any single actor to provide them on their own. As Bozeman (2002) notes, a scarcity of providers of public value results in market failure and necessitates public action. Public failure, Bozeman (2002) argues, results when neither the market nor the public sector provides necessary goods or services. Hypothesis 3: The federal government’s failure to provide sufficient drug effectiveness information facilitates collaboration.
Significant risks associated with policy decisions. According to Boschken (1998, 606) “collaboration, rather than going it alone, provides better positioning and sharing of risk in a dynamically changing public economy.” Organizations seek partnerships to reduce uncertainty, increase stability, reduce resource dependence, and decrease transaction costs (Bryson, Crosby, and Stone 2006). Hall and Jennings (2008) indicate that risk is an important consideration in agency effort to collect evidence. Hypothesis 4: Great risks associated with public pharmaceutical programs facilitate collaboration.
Shared policy goals among participants. Interlocal cooperation occurs when two or more governments work together to solve common problems (Honadle 1984). The most amenable goal structure for collaborative development is one with a singular purpose, well understood and shared by participants. Koppell (2006) explains the challenge of operating under multiple or ambiguous goals, and Chun and Rainey (2005) reveal a negative association between goal ambiguity and performance. Hypothesis 5: Shared policy goals among participants facilitate collaboration.
Equal participation in governance. According to Lasker, Weiss, and Miller (2001, 191), “Minimizing the drawbacks arising from a partner’s participation in the partnership may be just as effective as providing additional benefits.” One such drawback that concerns partners is insufficient influence in the partnership’s activities. Partner organizations desire to have their opinions and ideas heard and counted. “Governance is key to partnership functioning, and it is likely to have a profound effect on a partnership’s synergy level. Through procedures that determine who is involved in partnership decision making and how partnerships make decisions and do their work, governance influences the extent to which partners’ perspectives, resources, and skills can be combined” (Lasker, Weiss, and Miller 2001, 195). Governance arrangements that provide and encourage open and equal participation in decisions will lead to successful collaboration. Hypothesis 6: Partner equality in decision making facilitates collaboration.
Collaboration independence from participant policy process. Agency suspicion toward the collaborative and the delegate acting on its behalf can be unhealthy for partnership function. Agency representatives “usually need some protection against the demands and suspicions emanating from their own agencies” (Bardach and Lesser 1996, 210). Fear of losing local control over a function (Honadle 1984) or reduced independence in making decisions about their own activities (Lasker, Weiss, and Miller 2001) are impediments to cooperation. State legislators’ and policy makers’ disdain for a collaborative effort can disrupt its efficient function. Participants in the collaborative may be more attentive to such concerns when such suspicion arises from the funding source for the effort. At the same time, acting alongside other states in a collaborative to address similar problems can provide needed political cover to politicians and policy makers. Hypothesis 7: Independence from partner policy processes facilitates collaboration.
Accountability and transparency. Transparency and other accountability mechanisms foster trust in public sector activities, whether independent or collaborative. Given that transparency is a core democratic value (Piotrowski and Rosenbloom 2002) and “trust is important for the legitimacy and stability of the political system” (Tolbert and Mossberger 2006, 355), openness of partnerships to their stakeholders ensures support without cynicism or suspicion. Gaps in expectations between individual entities and the collaborative complicate the accountability dynamics and complicate collaboration (Radin and Romzek 1996). Page (2004) observes that collaboratives can achieve varying degrees of accountability for performance needed to manage stakeholder expectations, but stops short of analyzing accountability platforms’ effect on performance. Hypothesis 8: A transparent and accountable process facilitates collaboration.
Equitable distribution of costs. Efficiency gains and the division of costs among cooperating localities are additional impediments to successful intergovernmental cooperation (Honadle 1984). Partnerships that do not equitably distribute the costs to participating agencies will be less solvent and sustainable than those that do. Equitable distribution will naturally be dependent on the partnership setting and may only be definable through the eyes of participating organizations. Tschirhart, Christensen, and Perry (2005) observe that norms of mutualism and reciprocity play an important role in facilitating collaboration. Trust and cooperation can be increased by sharing costs equally (Ostrom 1998). Hypothesis 9: Equal distribution of costs facilitates collaboration.
History of successful interaction among partners. Ostrom’s (1998) work highlights the value of behavioral norms that develop from past interaction; trust and reciprocity enhance cooperation. Boschken (1998) presents additional characteristics that might stimulate success in collaborative efforts, including that “a history of dyadic working relationships makes intergovernmental exchange familiar and more predictable” (p. 606). Hypothesis 10: A history of successful interaction among partners facilitates collaboration.
Similarity in purpose and values. Lasker, Weiss, and Miller (2001) identify synergy as the productivity-enhancing output of effective partnership functioning. They argue that synergistic partnership actions are strengthened “by bringing together similar partners who share particular views or provide the same type of services” (Lasker, Weiss, and Miller 2001, 185). Partner agencies that are identical in purpose, function, or views, are better able to form, operate, and sustain successful collaborative efforts than those whose purpose, function, or views differ. Hypothesis 11: Similar purpose among partners facilitates collaboration.
Collaborative capacity/independent professional staff. A partnership’s ability to pay full-time administrative staff is also a likely determinant of partnership synergy (Lasker, Weiss, and Miller 2001, 195). Partnerships that do not rely exclusively on donated staff time will have greater capacity to fulfill their intended purpose. Administrative capacity has clear linkages to collaborative performance (Thomson and Perry 2006) and outcomes. Collaborative efforts that have full-time professional and administrative staff should realize greater success and sustainability than those that do not. Hypothesis 12: Dedicated administrative/professional staff facilitates collaboration.
Low transaction costs in enforcing the agreement. Accountability for setting collaborative priorities requires establishing an internal system of self-governance (Bardach and Lesser 1996). Formal procedures sustain partnership governance beyond the tenure of any single participant (Lasker, Weiss, and Miller 2001). “Partnership contracts are most likely to emerge when potential benefits are high and the transaction costs of developing, negotiating, monitoring, and enforcing the political contract are low” (Lubell et al. 2002, 149). Hypothesis 13: Low transactions costs in enforcing the agreement facilitate collaboration.
Objective analytic environment. Jewell and Bero (2008) identify characteristics of the evidence supply, such as quantity and quality of evidence as central hindrances to evidence-informed policy making. In fact, they advocate researchers collaborate to develop networks for generating and sharing policy-relevant evidence. Thus, when quality information is needed, but individual actors lack the resources or time to synthesize it, collaboration offers a mechanism to fill this need. Hypothesis 14: Generating high-quality, objective evidence as a product facilitates collaboration.
Methods and Data
We first solicited assistance from DERP administrators in identifying individual-state DERP representatives; to facilitate participant response, a key DERP administrator forwarded our request along with an electronic link to an online survey instrument. Survey data were collected in September 2009 using the Web-based portal www.SurveyMonkey.com. Of the fifteen participants at the time, ten completed the online survey, for a 67 percent response rate. The survey included questions about each factor’s presence and importance to DERP’s sustained success.
Findings
Table 1 provides results arranged by ranked index scores for each identified factor. The index is based on Likert-type responses to two items: if the characteristic is observed and how important it is to the success of the DERP collaboration. The Likert-type scales ranged from strongly disagree (1) to strongly agree (5) and not important (1) to extremely important (5). The index was created by multiplying the score for the presence of the factor times the score for the importance of the factor. A score of 9 would indicate a neutral value for the combined response to the questions. Exact item wording is provided in Table 1.
Presence and Importance of Factors Influential to Collaborative Success
We can note several findings. First, almost all of the hypotheses are supported by a majority of the responses. All but one hypothesis has a score above the neutral value of 9, but there is considerable variability in the level of support for the hypotheses. There is little support for the idea that the risk associated with the drug effectiveness policy outcomes is important to the success of the collaborative.
Second, several factors stand out as being critical for the success of the DERP collaborative. At the top of the list is the fact that all of the partners participate equally in DERP decision making. This is followed closely by the fact that the states have the freedom to decide how to use the DERP reports. Ranking third in importance is the history of successful interaction among the partners. These rankings suggest that governance processes are critical to the success of the collaboration from the perspective of the participants. Decision-making processes, autonomy, and successful interactions are central to success. Other factors with a score of 17.8 and higher are: the quality of the pharmaceutical reviews produced by DERP; the transparency and accountability of the DERP process; equitable distribution of costs among participants, and DERP independence from state policy decisions. The latter three of these also bear on governance processes.
We have already noted the lack of support for the idea that the risk associated with drug effectiveness policy outcomes is important for the success of the collaborative. Two other factors on which respondents place little importance are legislative deference to professional scientists and the lack of federally provided information on drug effectiveness. Surprising at one level, given the emphasis on goal sharing in the literature, is the relatively low score for shared policy goals among DERP participants. At another level, this is not surprising given the importance attributed to the states’ freedom to decide how to use the reports. A shared interest in quality information need not translate into shared policy goals.
Discussion
Drawing on the literature, we identified fourteen characteristics as relevant to the success of government collaborative efforts. We evaluated the presence of these conditions and their importance to the success of DERP. All but one met with some level of agreement from state DERP participants, and all but one scored above neutral on a scale of importance to sustaining the collaborative. Overall, the three factors DERP participants found to be most important to sustaining the collaborative effort were: (1) state freedom to decide how to use DERP reports, (2) equal participation of each partner state in the decision-making process, and (3) the quality of the product created by the collaborative. The DERP collaborative effort has proven sustainable as a result of shared output, but not shared state policy making. In an environment of goal and program multiplicity, state agencies have been able to improve performance through a partnership established on the expectation that governance will be open and equitable, the product will be of high quality, and participants will have unique uses for the information produced.
Footnotes
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The authors received no financial support for the research, authorship, and/or publication of this article.