Abstract
Existing research on policy diffusion focuses almost exclusively on “successes” where many jurisdictions adopted the policy or policies under examination. Some have speculated that this “pro-innovation bias” compromises scholars’ ability to draw valid inferences about the factors that influence the diffusion process. We argue that the study of interstate compacts in the United States provides an analytic opportunity to assess whether these concerns are warranted because it allows us to examine an entire universe of cases with unusually wide variability in their adoption patterns. Based on a pooled event history analysis of the interstate compacts that are open to all fifty states, we conclude that the tendency to limit diffusion research to widely adopted policies affects the results of previous studies. Specifically, it appears to lead scholars to systematically overestimate the impact of geographic diffusion pressures and policy attributes, and to underestimate the importance of professional associations and the opportunity to learn from previous adoptions. In sum, the longstanding concerns about a pro-innovation bias in diffusion research seem to be warranted.
Keywords
The process through which political phenomena spread from one jurisdiction to another, typically called diffusion, is a central concern in political science. By one count, political science journals have published nearly eight hundred articles on the subject, with over half of them appearing within the last decade (Graham, Shipan, and Volden 2013). Interest in diffusion cuts across virtually every subfield of the discipline. Scholars of comparative politics, international relations, and American politics have examined the diffusion of democracy (Brinks and Coppedge 2006), international norms about human rights (Keck and Sikkink 1998), pension and health care reform (Weyland 2006), political institutions (Bridges 1997), and many other political phenomena. Few topics have simultaneously engaged so many scholars in so many disparate areas of the discipline.
The recent proliferation of diffusion research has produced numerous theoretical, methodological, and empirical advances (Graham, Shipan, and Volden 2013). Despite these impressive gains, however, some have speculated that existing studies suffer from a “pro-innovation bias” because they examine only episodes during which an innovation was adopted by a large number of jurisdictions (Rogers 1995). According to these critics, constructing a compelling account of why diffusion occurs, or fails to occur, requires that scholars investigate both “successes” and episodes of limited adoption or non-adoption. The primary goal of this study is to assess whether their concerns are warranted.
This study estimates the pro-innovation bias by turning to the American states, a setting that has been the subject of an extensive scholarly literature on policy diffusion. Specifically, it examines state participation in interstate compacts, constitutionally sanctioned agreements between two or more states that are used to address joint problems in many policy areas (Zimmerman 2002). Interstate compacts are not perfect analogues for the policies on which diffusion research tends to focus, yet in both cases state officials have the opportunity to adopt a preexisting template. Moreover, previous research suggests that compact membership and policy innovation are driven by similar forces (Bowman and Woods 2007; Nice 1987; Nicholson-Crotty et al. 2014). One advantage of our approach is that we can observe the entire universe of interstate compacts rather than examining a small purposive sample of innovations. Moreover, compacts vary widely in their geographic spread. This combination of comprehensiveness and variation permits greater confidence in the inferences one can draw about the diffusion process.
Our empirical strategy relies on pooled event history analysis (EHA), which can be used to examine multiple policy innovations and ensure that any results are not due to the idiosyncrasies of an individual policy or state (Boehmke 2009; Bouche and Volden 2011; Makse and Volden 2011; Shipan and Volden 2006). Specifically, we estimate a model that interacts many well-known predictors of adoption—neighboring adoptions, legislative professionalism, and so forth—with an indicator coded “1” for those compacts that would be in a “traditional” diffusion sample because they have been widely adopted. In this context, the “traditional” sample includes the nineteen national compacts joined by twenty states or more. The extent to which the interactions attain conventional levels of statistical significance allows us to assess whether the standard focus on widely adopted policies yields results that differ from those for the “full” sample, which includes all forty-three national compacts.
Our analysis suggests that an emphasis on diffusion “successes” may lead scholars to overestimate the impact of geographic diffusion pressures and policy attributes but understate the effect of professional associations and the number of previous adopters. Although these results must be interpreted carefully given the substantive differences between interstate compacts and “conventional” policy innovations, they imply that scholars must be cognizant of the ways in which case selection may affect our understanding of the diffusion process.
The remainder of the article proceeds as follows. The next section examines the potential existence of the pro-innovation bias before explaining why the study of interstate compacts helps assess this possibility. We then describe the predominant explanations of policy adoption. The empirical analysis that follows investigates the applicability of these explanations in a setting in which policy innovations have not been purposively chosen. It illuminates questions that have long preoccupied scholars about the political forces that facilitate and hinder the widespread adoption of political phenomena. The final section of the paper considers the broader implications of our findings.
The Pro-innovation Bias, Policy Diffusion, and Interstate Compacts
In his classic assessment of research on the diffusion of innovations, Everett M. Rogers (1995, 100) defines the pro-innovation bias as “the implication . . . that an innovation should be diffused and adopted by all members of a social system, that it should be diffused more rapidly, and that the innovation should be neither reinvented nor rejected.” Most studies of diffusion among the American states do not necessarily favor the adoption of the policy being studied. However, they typically gravitate toward policies for which there is sufficient variation in the “adopt or not” dependent variable to permit productive empirical analysis. In addition, they have “assumed that all states will eventually adopt the policy in question” (Mooney and Lee 1995, 606). As a result, existing research focuses almost exclusively on policies that were adopted by a relatively large number of jurisdictions.
Consider, for example, the longstanding question of whether some American states are more innovative than others. Many studies address this issue by constructing innovativeness scores based on the speed with which state officials endorsed a specific set of policies (Boehmke and Skinner 2012; Boushey 2010; Canon and Baum 1981; Gray 1973; Lutz 1987; Savage 1978; Walker 1969). All these studies incorporate only innovations that reach a specific adoption threshold, usually twenty states. In fact, one analysis of state health care policy includes eight policies that were adopted by at least thirty-eight states (Carter and LaPlant 1997).
Studies that focus on specific diffusion mechanisms, or on the emergence and spread of specific innovations, also tend to examine widely adopted policies. In their path-breaking article on the impact of federal financial incentives, for example, Welch and Thompson (1980, 721) note that forty-two of the fifty-seven included policies “were chosen as examples of policies that have diffused completely, or nearly so.” Perhaps the strongest evidence of this tendency, however, comes from a recent literature review that cites eighteen scholarly articles that investigated state-level adoption patterns for one to three policy innovations (Graham, Shipan, and Volden 2013). 1 The articles analyze twenty-three distinct policies where the number of adopters ranges from a low of fifteen states (school choice) to a high of fifty states (child abuse reporting laws). The mean number of adopters is 29.3 with a standard deviation of 9.9, and the median is 28.
Focusing on popular policies offers considerable advantages. For instance, it facilitates data collection because an episode of non-diffusion “does not leave visible traces that can be very easily reconstructed” (Rogers 1995, 105). Moreover, it also can help identify the causal mechanisms that transfer programs across jurisdictional boundaries, such as the national government (Karch 2012) and policy entrepreneurs (Mintrom 1997). Thus, it promotes a richer understanding of why specific policies gain widespread adoption and helps distinguish between simple policy convergence and the interactive dynamic that constitutes diffusion (Volden, Ting, and Carpenter 2008). Finally, the widespread use of EHA, the conventional statistical approach to the study of policy diffusion, has allowed scholars to model the internal and external determinants of adoption simultaneously and to identify several factors that affect whether and when state officials will enact public policies (F. S. Berry and Berry 1990). As a result, political scientists have learned a tremendous amount about how popular policies diffuse.
One way to build constructively on the recent proliferation of diffusion research is to ask analogous questions about policies that are not widely adopted. Although existing studies offer critical insights into the mechanisms that facilitate the spread of certain policies, for example, they are less well suited to explain why some policies are more popular than others. Indeed, an emphasis on diffusion successes offers limited leverage on the factors that distinguish them from instances of limited adoption. The specific question that motivates our analysis is whether the insights described in the preceding paragraph are applicable to less popular policies or, to frame the question more provocatively, whether they have come at the cost of introducing a “pro-innovation bias” into diffusion research. This bias need not call into question the inferences drawn in individual studies, so long as authors are careful not to generalize from the specific policies under investigation to broader policy diffusion processes. As these studies accumulate, however, they may provide a cumulative knowledge base that is built on a systematically unrepresentative sample of policies, thus leading scholars to make incorrect or questionable inferences about the general nature of policy diffusion dynamics. These unwarranted inferences could take several forms. Researchers could identify certain factors as influential even though they do not have a significant effect on program adoption when the full range of policies is considered. The reverse could likewise occur. Finally, researchers could identify the appropriate factors as influential but inflate or deflate their substantive impact. If any of these patterns exist, it might imply that political scientists have drawn inferences about the diffusion process that apply only to widely adopted policies, rather than the universe of possible policies.
Interstate Compacts as an Analytic Opportunity
Interstate compacts are negotiated agreements between two or more states that have the force and effect of statutory law. 2 Professional associations like the Council of State Governments (CSG) may also be involved in compact development. Once a state ratifies a compact, its provisions take precedence over conflicting state laws. 3 Interstate compacts’ ramifications for state sovereignty are especially profound when they create an administrative structure to enforce their dictates (Broun et al. 2006). The implications of ratification mean that states do not enter into compacts casually (Hill and Weissert 1995; Schlager and Heikkila 2009). However, interstate compacts offer several benefits. They enable states to address shared problems, promote a common agenda, and produce collective goods (Zimmerman 2002). Moreover, participation can allow states to reduce the costs associated with policy design and experimentation (Bowman and Woods 2007).
Interstate compacts represent a collective policy-making approach that has largely been ignored by diffusion scholars. Yet the dynamics surrounding the “diffusion” of a compact are reasonably similar to those surrounding the spread of conventional policy innovations. In both contexts, a template is developed and state officials have the option of adopting it. Moreover, states may join compacts for multiple reasons, just as the adoption of policy innovations may be driven by diverse forces like competition, emulation, and imitation (Boehmke and Witmer 2004; Shipan and Volden 2008). Prior research also suggests that compact membership is driven by similar political and institutional forces as policy innovation (Nicholson-Crotty et al. 2014). This broad overlap recommends interstate compacts as a rational extension of the diffusion literature.
Moreover, interstate compacts offer a major advantage for the assessment of a possible pro-innovation bias: it is possible to examine the complete universe of compacts. This study focuses on compacts that are national, meaning that participation is open to every state regardless of how involved it was in its development. Like the policies that have been examined in prior research, national compacts respond to emerging societal issues and are legitimate policy-making options for every state. They are not regional policies. 4 By early 2012, thirty-eight national compacts were in force and another five had been endorsed by a single state. 5 The compacts offer a comprehensive and manageable universe of cases, making it possible to draw general conclusions about the factors that facilitate or hinder their adoption.
This advantage is not simply a matter of increasing the number of observations. Indeed, in recent years many diffusion studies have incorporated dozens of programs (Boehmke and Skinner 2012; Boushey 2010; Makse and Volden 2011; Nicholson-Crotty 2009). Despite their impressive scope, however, their findings may be subject to external validity concerns because they rely on purposively chosen samples of policy innovations. There is no way to guarantee that the cases are representative of the virtually limitless number of policies that states have adopted. Jack Walker (1973, 1190) explained this dilemma four decades ago: “Although each of us has constructed criteria for issue selection that seem reasonable, we have no guarantee that the issues we are studying are not highly unusual and somehow unrepresentative of their class.” More recently, Boehmke and Skinner (2012, 310) conceded, “We . . . remain uncertain as to whether our database of 189 policies constitutes a representative sample from the universe of state policies.” Both studies systematically exclude policies adopted by less than twenty states, an especially salient consideration for our purposes.
Examining interstate compacts can help ameliorate potential concerns about external validity while simultaneously offering leverage on the question of pro-innovation bias. Issues of convenience sampling and representativeness are essentially irrelevant when all forty-three national compacts are examined. Table 1 lists the relevant compacts. The wide variation in state participation helps address the pro-innovation bias issue. Although all fifty states entered into the original Interstate Compact for Juveniles and forty-seven states joined the Emergency Management Assistance Compact, the only states to join the Interstate Insurance Receivership Compact are Illinois, Michigan, and Nebraska. Five national compacts technically have not gone into effect because they claim only a single member. Thus, national compacts include examples of widespread adoption, limited adoption, and non-adoption, a rarity in diffusion research. They are also substantively meaningful policy initiatives that address the most salient issues in state policy making—criminal justice, education, health care, and taxation—as well as other topics that generate less publicity.
List of National Compacts.
Source. Council of State Governments (2012).
Parentheses contain the years when the first and most recent states joined the compact. Participation rates are as of March 2012.
In sum, we believe that national compacts offer diffusion scholars a promising opportunity. It is possible to examine the complete universe of potential cases, so there is no reason to worry about sample selection bias and external validity. Moreover, the varied reach of the forty-three national compacts facilitates an analysis of a wide range of adoption patterns. It is therefore possible to assess the dominant explanations of policy adoption and ascertain whether a focus on widely adopted policies has influenced scholars’ estimates of their accuracy.
Of course, there are several differences between compacts and “conventional” policies that keep them from being perfect analogues. For example, compacts offer states the opportunity to share cost and policy expertise and may be relatively more attractive to states with low institutional or fiscal capacity (Bowman and Woods 2007; Nice 1987). In addition, the decision to enter into a compact may be made more cautiously than the decision to adopt a traditional policy because the former may entail a loss of sovereignty that the latter does not. Finally, whereas states can make technical or political adjustments to policy templates prior to adopting them (Mossberger 2000), they lack such flexibility in the context of compact participation. Although we recognize these important differences and the resultant need to be circumspect in interpreting our results, we do not feel that they outweigh the considerable benefits of turning to interstate compacts.
Policy Adoption: Conventional Explanations
Since the seminal research of Jack Walker (1969) and Virginia Gray (1973) was published, scholars have attempted to isolate the state attributes, external pressures, and policy characteristics that influence the adoption of policy innovations in the American states. The standard use of EHA has facilitated comparability across studies and enabled scholars to identify several influential factors. However, some of these conventional explanations may be especially vulnerable to issues of case selection. The smaller literature on state participation in interstate compacts tends to use similar explanatory variables (Bowman and Woods 2007; Nice 1987).
State Attributes
Policy adoption may be more likely in states with certain characteristics. Some of these attributes serve as proxies for resource availability. Rogers (1995) emphasizes the potential impact of slack resources, arguing that states are more likely to innovate if they possess the financial wherewithal to invest in policy design and implementation. Scholars have consistently found that fiscal capacity facilitates the early adoption of innovations (Boehmke and Skinner 2012; Dye 1966; Gray 1973; Kousser 2005; Rogers 1995; Walker 1969). Our analyses therefore include a measure of state per capita personal income and a measure of total state government expenditures per capita. 6
Several demographic traits also have been associated with the availability of slack resources. For example, many studies have found that more populous states and states with higher urbanization levels tend to be more innovative (Boehmke and Skinner 2012; Boushey 2010; Walker 1969). Our analyses therefore include the natural log of the state population and the percentage of the state population residing in urban areas. A third demographic characteristic incorporated into our analyses is the percentage of the state population who are college graduates because there may be a connection between education and innovativeness (Boushey 2010; Kousser 2005; Walker 1969).
Existing research also highlights the potential impact of political and institutional factors. State officials’ reactions to innovative policies can depend on their partisan affiliation (Spill, Licari, and Ray 2001; Roh and Haider-Markel 2003). Our analyses therefore include three measures. The first is a dichotomous variable that indicates whether the governor is a Republican. The second variable measures the percentage of seats in the state legislature that are held by the Democratic Party. The final measure is a dichotomous variable indicating whether all three institutions of state government (the governorship and both chambers of the legislature) are controlled by the same party. One might expect unified control to facilitate governmental action, though empirical support for this hypothesis is mixed (Binder 2003; Bowling and Ferguson 2001; Mayhew 1991). We estimate a separate model that accounts for the impact of ideology on compact adoption decisions. This variable is omitted from the primary analysis because our study includes compacts dating back to the 1950s, and our measure of state ideology is unavailable before 1960 (W. D. Berry et al. 1998).
In terms of governmental institutions, legislative professionalism is associated with higher salaries, longer sessions, and greater staff resources (King 2000; Squire 1992). Legislators in highly professionalized chambers may be better able to identify and evaluate new policy options (Boushey 2010; Shipan and Volden 2006). Our models therefore include a measure of state legislative professionalism (King 2000).
Before moving on, it is important to recognize that some state attributes may operate differently in the context of interstate compacts. For example, the absence of institutional or fiscal capacity might provide an incentive for state officials to adopt a collective policy-making approach. Existing research, which focuses on fewer compacts over a shorter time period, suggests that states with weaker policy-making institutions may be more receptive to compacts (Bowman and Woods 2007; Nice 1987). Similarly, less affluent states might view compact membership as a way to enhance their policy-making capacity (Nice 1987). In short, compact membership can help states adopt a new policy at a lower cost or of higher quality than they could provide on their own. Institutional and fiscal capacity might therefore function differently when states operate collectively than when states act individually.
Of the three categories of explanations considered here, state attributes are the least likely to be affected by the pro-innovation bias. Previous studies have focused on “liberal” innovations like hate crime laws, policies like school choice that appeal to conservatives, valence issues like child abuse reporting requirements, and licensing requirements that lack ideological content. Some of these policies require substantial government spending, whereas others do not. The conventional focus on widely adopted policies may be problematic, but it seems unlikely to lead scholars to overestimate or underestimate the impact of specific state attributes.
External Pressures
Scholars have also long recognized that external pressures influence adoption decisions. In the American states, this implies that the existence of a policy in one state affects the probability that it will be adopted elsewhere. Most diffusion research focuses on the potential impact of geographic proximity, presuming that the existence of a policy in nearby states makes officials more likely to adopt it. 7 A common statistical proxy for geographic proximity is the number or percentage of a state’s neighbors in which a policy has already been enacted (F. S. Berry and Berry 1990; Haider-Markel 2001; Mintrom 1997). The empirical record provides mixed support for the geographic proximity hypothesis (Mooney 2001), but most analyses, including ours, include the standard proxy for the neighboring state effect.
Recent research has also suggested that policy adoption is driven, in part, by the “opportunity to learn” (Makse and Volden 2011; Shipan and Volden 2008). This opportunity is a function of the decisions made by all potential adopters, not only those with which a state shares a border. Officials can gather more relevant information when multiple governments have tried a policy. Ultimately, decision makers may “interpret the broad adoption of a policy without subsequent abandonment over time as evidence of success . . . or at least of maintained political support” (Shipan and Volden 2008, 842). This dynamic may influence the adoption of interstate compacts, so our models include the number of compact members prior to the year of measurement.
The pro-innovation bias seems especially likely to inflate estimates of the impact of geographic proximity and the opportunity to learn. When diffusion “failures” occur, many states will not have enacted their neighbors’ policies, implying that they are not positively influenced by developments in nearby jurisdictions. Yet these episodes are precisely the ones that are excluded systematically from existing research. Instead scholars have tended to focus on cases of widespread adoption, where states are more likely to have adopted their neighbors’ policies. A similar dynamic may inflate estimates of the impact of the opportunity to learn. When an innovation gains enactment in a small number of states, it might mean that state officials do not find the “lessons” offered by early adopters compelling enough to take action. Once again, however, most diffusion research excludes these episodes.
Other external forces, such as organizations that operate in multiple jurisdictions, might also affect the adoption decision. The geographic reach of professional associations and interest groups enables them to transport innovative policy ideas across state lines (Balla 2001; Glick 1992; Haider-Markel 2001). Organizations like the CSG and the National Conference of State Legislatures are active on many issues, and legislative leaders value the information that they disseminate (Clark and Little 2002). If the information and visibility provided by these organizations helps facilitate the widespread adoption of policy innovations, then existing research might understate their impact. By focusing exclusively on instances of widespread adoption, scholars inadvertently might be excluding policies that lack interest group support. As a result, the typical diffusion study may lack sufficient variation on the interest group dimension to assess their significance. In contrast, the universe of national compacts does not suffer from this potential shortcoming. Professional associations and interest groups helped design approximately half of our national compacts, 8 and our analyses include a dichotomous variable indicating that one of these groups was involved in a compact’s initiation.
Policy Characteristics
Specific features of a policy, such as its complexity, can affect the speed and extent of its adoption (Makse and Volden 2011; Nicholson-Crotty 2009). Complexity invokes both the ease of use of a new policy and potential adopters’ ability to predict its relative advantage over the status quo. Existing research suggests that more complex policies are less likely to be adopted due to uncertainty about their potential costs and benefits. By focusing on episodes of widespread adoption, existing research might therefore systematically exclude complex policies, leading to inaccurate inferences about the impact of this policy attribute. Our analyses therefore include a dichotomous variable indicating whether a compact qualifies as complex. Based on previous research, we code policies that deal with health care (finance and licensing), taxation, trade, and fiscal regulation as complex, while treating criminal justice, education, disaster response, consumer protection, and electoral policies as noncomplex (Gormley 1986; Nicholson-Crotty 2009). This coding scheme identifies 19 percent of compacts as complex, which is similar to the proportion coded as such in other studies. 9 This similarity might imply that case selection has not affected previous research.
Analysis and Results
In the remaining sections of the paper, we use pooled EHA to evaluate whether existing research has been affected by a pro-innovation bias. Pooled EHA is an increasingly common approach in diffusion research in which the unit of analysis is the state-policy-year (Boehmke 2009; Bouche and Volden 2011; Makse and Volden 2011; Shipan and Volden 2006). It enables us to assess the impact of state characteristics, external pressures, and policy attributes with greater confidence that our results are not due to an unusual state or policy.
For each of the compacts under examination, states enter the “risk set” in the year the first state adopts. They remain in the dataset until they join the compact, at which point they are removed to avoid “overcounting” adopters. This process is repeated for each compact. For example, the first states joined the Agreement on Qualifications of Educational Personnel in 1962. As New Jersey joined in 1969, it appears in the dataset from 1962 through 1969 but not in subsequent years. Similarly, the first states joined the Adoption Assistance Compact in 1984. As New Jersey never joined, it appears in the dataset for the entire adoption period (1984–1991). Following the precedent set in previous research, we use standard logistic regression methods on our stacked dataset and cluster standard errors at the state-year level to account for cases where states might adopt multiple compacts in a single year (Makse and Volden 2011; Shipan and Volden 2006). 10
Our empirical strategy leverages the combined comprehensiveness and variation offered by the study of national compacts. Specifically, we estimate a model that interacts the predictors of adoption described in the previous section with an indicator coded “1” for those compacts that would be in a “traditional” diffusion sample because they have been widely adopted. In this context, the “traditional” sample includes the nineteen national compacts joined by twenty states or more. The extent to which the interactions attain conventional levels of statistical significance allows us to assess whether the standard focus on widely adopted policies yields results that differ from those for the “full” sample, which includes all forty-three national compacts.
This comparison provides a sense of whether and how a focus on widely adopted policies affects our understanding of the diffusion process. Any discrepancies between their results, in terms of which factors achieve conventional levels of statistical significance, the direction of those relationships, or their substantive impact, is likely due to issues of case selection. The demonstrated existence of such discrepancies would suggest that the accumulated evidence on policy diffusion drawn from prior research may lead scholars to draw unwarranted inferences, possibly by identifying the wrong factors as influential or by misestimating the impact of significant factors. If the interactions do not isolate any major discrepancies, then the concerns about case selection may be potentially overstated.
Results
Table 2 displays the results of our analysis. The first column contains a model of all compacts, whereas the second includes state ideology and can, therefore, only model the adoption of compacts after 1960. The table is formatted so that predictors are presented in pairs with the main effect followed by the interaction between that effect and the indicator of widely diffusing compacts (the “traditional” sample in diffusion studies). Our primary interest is in the differences across the traditional and full samples, and those distinctions are easier to interpret by examining the marginal effects, which are plotted in Figure 1 along with their 95 percent confidence intervals. The variables have been standardized, so each point in the plot represents the effect of a 0 to 1 shift for dichotomous indicators and a one-standard-deviation shift for continuous measures. We will focus on the “differences” where the confidence interval for a variable in one sample does not overlap the coefficient for the same variable in the other sample. The differences represent the potential bias introduced by focusing only on widely adopted policies.
Interactive Model of the Predictors of Compact Adoption.
Standard errors clustered by state-year.
p< .1. *p < .05. **p < .01. ***p < .001.
Marginal effects from the primary model.
In some cases, our results highlight the overestimation of key effects. The impact of neighboring state activity, the most common measure of interdependence, is over twice as large in the traditional sample as it is in the full model. Geographic proximity certainly merits a prominent place in diffusion research because it often makes adoption more likely, but scholars should be careful not to overstate its general impact. It is also essential to distinguish among the many forces—economic competition, cultural similarities, overlapping media markets, and many others—that can produce a geographic pattern. Understanding the conditions under which these mechanisms are most likely to operate will greatly enhance our understanding of the diffusion process.
The effect of policy complexity is also much larger in the traditional sample than in the full model. Indeed the observed impact is more than four times as large. Many recent diffusion studies have emphasized the importance of policy content, but the results in Figure 1 imply that existing studies likely overstate the negative impact of complexity. Identifying the source of this overestimation, which does not seem to result from the systematic exclusion of complex policies based on the universe of national compacts, requires further investigation. In sum, diffusion scholars should continue to assess the influence of geographic proximity and policy attributes while remaining attentive to potential ways in which case selection might affect their estimates.
Underestimating the importance of key factors is equally important. On this dimension, one notable difference concerns the impact of the number of previous adopters, a proxy for policy learning. The impact of this variable on the probability of adoption is positive and significant in both samples, but its effect is almost twice as large in the full sample relative to the traditional sample. This result is somewhat surprising. Our expectation was that the traditional sample would inflate estimates of the impact of the opportunity to learn, due to the fact that adoption by a small number of states implies that state officials do not find the “lessons” offered by early adopters persuasive enough to take action. It is precisely these episodes, though, that are excluded from the traditional sample. Our results, however, suggest that the traditional sample understates the effect of previous adopters.
One possible explanation for this finding lies in the amount of information that may be gained from another state’s adoption. The number of previous adopters is assumed to reflect the opportunity for potential adopters to learn about a policy. When policies diffuse widely, and thus are adopted by many states, the marginal information gain from each additional adoption likely diminishes. However, in instances of limited diffusion, each additional adoption represents a significant increase in information. We think that this process of policy-relevant information acquisition is worthy of further investigation.
The general impact of professional associations and interest groups also appears to be underestimated in diffusion research. In the traditional sample, the involvement of organizations like the CSG in compact development had a null effect on the likelihood of adoption. In the full model, however, their involvement has a significant positive effect. This discrepancy suggests that the traditional sample suffered from a selection bias that did not allow sufficient variance in interest group involvement to detect their true importance. Indeed, professional associations or interest groups helped initiate thirteen of the nineteen compacts (68%) that were joined by twenty or more states but only ten of the twenty-four compacts (42%) that were joined by fewer than twenty states. Thus, existing diffusion research might understate the role of organizations as diffusion mechanisms. Investigating the potential impact of professional associations and interest groups by studying episodes where they are and are not engaged in the policy process represents a promising avenue for future diffusion research.
The second column of Table 2 presents the results from the model that includes state ideology. The main effect of ideology and the interaction appear to be insignificant in the table, but a plot of the marginal effects, presented in Figure 2, suggests that the negative effect of ideology is significantly larger in the traditional versus the full sample, where the variable appears to have no effect on compact adoption decisions. More importantly, the plotted effects in Figure 2 affirm our major findings, which remain unchanged by the inclusion of ideology and the exclusion of the earliest compacts. The traditional sample continues to overestimate the substantive importance of the neighboring state effect and policy complexity, while underestimating the influence of the number of previous adopters and organized interests.
Marginal effects from the model including ideology.
Conclusion
Scholars have long worried that our understanding of the diffusion process may be limited by a tendency to focus on innovations that have gained widespread adoption. In the many years since these concerns were first aired, however, there has not been a systematic assessment of this potential bias. The absence of such a test is due primarily to the difficulty of identifying the universe of policies against which it could be estimated. We address this dilemma by focusing on interstate compacts that are open to every American state. There is an identifiable and discrete universe of national compacts that vary in their geographic reach. These collective policy solutions and conventional public policies differ in key ways, but the decision-making processes that lead states to join a compact or adopt a policy are sufficiently similar to provide suggestive evidence regarding the lessons that can be learned by examining episodes of limited adoption in addition to the “successes” that make up the bulk of the existing scholarly literature. Future research would benefit greatly from the examination of similar “universes” of policies. For instance, professional associations and interest groups often author model legislation. Some of their recommendations are widely adopted, whereas others are not, providing useful variation that can be leveraged to extend the analysis presented here. Similarly, these organizations sometimes publish reviews that track noteworthy developments in a specific policy area during a legislative session. These reviews might also provide a universe of policies that vary in their geographic reach. The broader point, however, is that studying episodes of limited adoption represents a logical and promising extension of existing diffusion research.
Our analysis suggests that focusing exclusively on episodes of widespread adoption may cause scholars to draw unwarranted inferences about the factors that influence policy decisions. Specifically, such analyses appear to overestimate the impact of geographic diffusion pressures and policy characteristics. The substantive effect of neighboring state activity and complexity is consistently much larger in models that include only popular compacts. The conventional focus on geographic proximity as a diffusion facilitator and policy complexity as an impediment is appropriate, but scholars must be more attentive to the specific conditions under which these factors are likely to be relevant.
Our results do not imply that things such as geographic diffusion are unimportant. However, they accord well with recent formal work, which argues that scholars often mistake other decision processes for external influence because of how adoption and diffusion have been modeled (Volden, Ting, and Carpenter 2008). They also resonate with the findings of two earlier studies that examined the adoption of multiple innovations. F. S. Berry and Berry (1992) found that developments in neighboring states affected the adoption of gasoline taxes but not income taxes. Similarly, Grossback, Nicholson-Crotty, and Peterson (2004) found that the neighboring state effect influenced the adoption of lotteries and academic bankruptcy statutes but not sentencing guidelines. Income taxes and sentencing guidelines were the least popular policies in their respective samples. Thus, empirical evidence that scholars may have inflated the significance of external pressures might have been hiding in plain sight for some time.
Our analysis offers other important lessons for diffusion scholars. For example, the importance of the opportunity to learn from previous adopters may have been underestimated in earlier research. The number of previous adopters, which is often treated as a proxy for policy learning, has a much larger impact when the universe of policies is considered. More research is necessary to confirm the reason for this finding, but it might be an artifact of focusing exclusively on widely diffusing policies where the marginal information available from each additional adoption is smaller. Focusing on episodes of widespread adoption also seems to underplay the role of professional associations as diffusion mechanisms that transport policies from one jurisdiction to another (Balla 2001). Similarly, the standard approach to diffusion research seems to inflate the substantive impact of policy complexity. Policy complexity has the expected negative effect on adoption, but its influence may not be as profound as is often assumed.
Our findings have potentially profound implications for future diffusion research. They imply that scholars must engage in more careful theorizing about the conditions under which external forces and policy attributes are likely to influence the adoption decision. Rather than presuming that developments in neighboring states will have an impact, for example, they must articulate specific mechanisms—such as communications networks, program externalities, and demographic similarities—that promote interdependence. “Placebo tests,” in which episodes where interdependence is expected are compared with those where it is not, represent a promising empirical approach that can accompany this theoretical refinement (Gilardi 2015). Finally, our results suggest that additional theoretical development may be necessary to more fully understand what factors differentiate diffusion successes from instances of limited adoption.
Although the interstate compacts examined here are not perfect analogues for the public policies and other political phenomena investigated in existing studies, we nevertheless believe our approach offers several constructive lessons. In addition to being more circumspect about the effect of interdependence and policy attributes and more attentive to the potential impact of organizations as diffusion mechanisms, scholars must recognize that case selection has profound implications for what can be inferred about the diffusion process. There is nothing inherently problematic about studying phenomena that have achieved a particular adoption threshold. By virtue of their widespread acceptance, they merit a prominent place in academic research. Our results suggest, however, that scholars should resist using such studies to make generalizations about the factors affecting all adoption decisions.
Footnotes
Acknowledgements
The authors thank Graeme Boushey and Julie Pacheco for constructive feedback and Jordan Simundson, Brandon Haugrud, and Sarina Rapini for helpful research assistance.
Authors’ Note
A previous version of this paper was presented at the 2013 State Politics and Policy Conference, Iowa City, Iowa, May 24-25.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
