Clarifying the Measurement of Relative Ideology in Policy Diffusion Research

Abstract

Research on policy diffusion has recently paid more attention to ideological patterns of policy adoption. Grossback, Nicholson-Crotty, and Peterson operationalized a measure of ideological diffusion; however, it has not been consistently calculated in subsequent studies. This is mainly due to difficulties in interpreting how to measure ideological distance based solely on the original article. Specifically, there are three factors that prevent common measurement of the concept: starting values, adoption ties, and weighting of recent adoptions. Recommendations are made for each of these. The purpose is to establish a consistent ideological distance measure. To illustrate, a replication of the original lottery diffusion model in the authors’ paper shows how the results change with different measurement choices. Consistently measuring this concept is important as scholars increasingly recognize that states do not always follow their geographic neighbors but increasingly their ideological “neighbors.”

Keywords

policy diffusion ideology public policy policy process state politics

There is an extensive literature on the diffusion of policy among the states. Early studies revealed how some states are more innovative than others; they adopt policies more often and more quickly than other states (Gray 1973; Walker 1969). With the introduction of event history analysis, a statistical method for modeling the timing of adoptions, Berry and Berry (1990) inspired a large body of work that examines both the internal and external determinants of policy adoption. Traditionally, the primary external influence on whether a state adopts a policy was thought to be a policy adopted by a neighboring state. The assumption is that states are learning from the policy lessons of their neighbors. Grossback, Nicholson-Crotty, and Peterson (2004), however, identified political learning as an equally important mechanism in diffusion. These are lessons that lawmakers draw from past adopting states regarding the political success of the policy, particularly from states that have a similar general ideological makeup. Subsequent evidence suggests that learning from ideological “neighbors” is just as important as learning from contiguous ones (Desmarais, Harden, and Boehmke 2015) and has become more important to the states over time (Mallinson 2019). Accurately understanding the role of ideology in shaping state innovation has both theoretical and substantive importance, particularly in an era of national gridlock where so much policy activity is occurring not in the federal government but in the states.

While it is an important concept in the study of policy adoption, Grossback, Nicholson-Crotty, and Peterson’s (2004) of ideological distance is difficult to implement based on the authors’ original paper. There are three specific aspects of the measure that are unclear:

How to handle the first adoption of a policy (when there are no prior adopters),

How to handle prior adoptions that occur in the same year, and

How to handle the time of adoption with more recent adopters having a greater impact on the adoption decision.

This study provides clarity on these issues so that future research will consistently measure ideology. The approach presented in this article clarifies these aspects of the measure and produces different results when replicating Grossback, Nicholson-Crotty, and Peterson’s (2004) state lottery adoption analysis.

Measuring Ideological Diffusion

Before discussing Grossback et al.’s measure, it is first useful to illustrate and contrast the concepts of policy and political learning. In policy learning, lawmakers are looking to other states for lessons in how they implemented a new policy. This is often for the purpose of solving a social problem. They observe what aspects of the policy led to positive, as well a negative, outcomes and will then decide two things. First, whether to adopt the policy. Second, whether to adjust the policy to address the problems experienced in other states and also tailor it to match the specific preferences and demands of citizens in their state (Glick and Hays 1991; Hays 1996). Practical lessons are not the only ones that legislators learn, however. They are also looking to see whether a policy was politically advantageous. This will shape the political strategy that proponents use to pursue adoption of the policy (May 1992). Grossback et al. argue that a prior adopter's ideological makeup is a signal of potential political success for states considering the adoption of a new policy. Meaning, if a policy is being adopted by liberal states, that sends a signal to other liberal states that the policy may be politically advantageous and makes them more likely to adopt it. Conversely, conservative states will observe liberal adoptions and learn that the policy would not be politically advantageous. While the concept of political learning had been discussed in the policy diffusion literature for some time, Grossback, Nicholson-Crotty, and Peterson (2004) were the first to offer a measure of it.

To test for whether states use ideology as a signal of political favorability when considering the adoption of a policy, Grossback et al. proposed using the relative ideological distance between states to capture whether being closer in ideology makes it more likely that states will adopt a policy and vice versa for being further apart in ideology. Their measure of ideological distance is as follows:

Ideological Distance = | \frac{(Last Adopter Ideology + All Other Adopter Ideologies)}{2} - Potential Adopter Ideology |

The intuition behind the measure is that the ideology of each state that has not yet adopted a policy is being compared to the ideologies of the states that have already adopted. Again, if they are close together, then it is more likely that the state will adopt the policy, if there is a greater distance adoption is less likely. Here is the meaning of each of the terms in the equation:

Last adopter ideology: A measure of ideology for the most recent state that adopted the policy. This is assumed to have the most substantive impact on new adoptions.

All other adopter ideologies: An average of the ideologies of all other past adopters, except for the most recent. This provides an aggregate sense of the other states that have adopted.

2: This increases the value of the most recent adopter’s ideology by treating it equally with all the other past adopters, regardless of how many there have been.

Potential adopter ideology: The ideology of a state that has not yet adopted the policy.

|−|: The final step is to take the absolute value of the difference between a potential adopter’s ideology and the combined measure of past adopters’ ideologies. The absolute value is used because the measure is calculating the relative distance, regardless of whether that distance is in a conservative or liberal direction.

An example of how this calculation is conducted is provided in the Online Appendix to this article. The equation is presented here because some of our concerns about how the measure is implemented come directly from the equation. Finally, it is necessary to note that it does not matter which of the multiple measures of ideology available to researchers are used for making this calculation. This study uses citizen ideology (Berry et al. 1998), but other researchers have used different measures.¹ There can be good theoretical reasons for using an alternative ideology measure, but they are interchangeable from the standpoint of calculating the ideological distance measure. There are three core components of the GNCP measure of ideological distance that need to be approached consistently for it to be possible to have consistent and comparable findings from the research on state policy adoption. Each will be discussed in turn.

Measuring the First Year of Adoption

In Grossback et al.’s article, there is no guidance on what value to use for the first observed adoption year in a data set. For example, both their article and this study use the lottery adoption data from Berry and Berry (1990). New Hampshire was the first state to adopt a lottery in 1964. How can the measure of ideological distance be calculated for states in 1964 when there are no past adopters?² The first adoption year should be included in the analysis, and thus, researchers should simply use zero as the measure of ideological distance for all fifty states observed in that year. Leaving out the first observed year will result in biased regression coefficients and larger standard errors (Leung, Elashoff, and Afifi 1997), so it is not ideal to simply leave out the first year.

Grossback et al. graciously provided their data, and they, in fact, included 1964 in the data set with a nonzero value for ideological distance though their article indicates that 1964 was dropped for their analysis. The replication below, however, finds that this was not the case. Alas, the authors are unsure as to how they calculated ideological distance for that first year. Zero is the most logical first value as there is no distance between each state and past adopters. There are no past adopters in this case. Doing so effectively sets the origin of the measure at zero. The Online Appendix addresses another proposed approach (Hannah and Mallinson 2018; Mallinson 2019).

Dealing with Adoption Ties

Given that policy diffusion data are often aggregated to the year, adoption ties are common. A tie occurs when more than one state adopts in a year. In the case of state lotteries, three states adopted in 1972: Massachusetts, Michigan, and Pennsylvania. This presents an issue for calculating the ideological distance measure in 1973, for example. Grossback et al. do not address this issue in their original article, but the equation presented above (equation [1]) assumes that there is only one most recent adopter. The most straightforward approach would be simply to average the ideologies of the tied most recent adopters and use that for the value of last adopter ideology. One possible alternative is addressed in the Online Supplement (Supplemental Text 1).

Weighting of Recent Adoption(s)

Our final point addresses the articles that have implemented the ideological distance measure. As noted in the discussion of the equation, there are important theoretical reasons for increasing the weight of the most recent adopter’s ideology. In doing so, last adopter ideology has increasing weight over time as more and more states are added to calculating the average for all other adopter ideologies (see Cruz-Aceves 2018). Some researchers, however, have not chosen to weight at all though it is often unclear whether this is due to theoretical reasons or practical (Hannah and Mallinson 2018; Karch and Cravens 2014; Seljan and Weller 2011; Sylvester and Haider-Markel 2016). In one case, weighting did not occur because of the use of a different approach for handling tied recent adoptions (Hannah and Mallinson 2018). If, however, researchers feel that most recent adopters do not deserve increasing weight due to the specific context of the policy they are studying, it is vitally important to make the theoretical argument for why they altered the specification of the original measure and to report that decision in their study. The specification of the ideological distance measure described below provides slightly, but also substantively, different results than were found in Grossback et al.’s original paper. A comparison to the Hannah and Mallinson (2018) measure is provided online (Supplemental Text 2).

The Effect of Measurement Differences on Model Results

In order to assess the extent to which the different specifications of ideological distance alter model results, the lottery diffusion analysis from Grossback, Nicholson-Crotty, and Peterson (2004) is replicated and extended. The original authors generously provided these data and comments about their formula and data. The data include observations for forty-eight states from 1964 to 1986. Grossback et al. reported in their paper that the data for lottery adoptions consisted of forty-nine states. However, the data set did not contain observations for Alaska or Hawaii. Nonetheless, the data set with forty-eight states is accurate, for it matches the summary statistics, coefficients, and z scores from the original article. Additionally, Grossback, Nicholson-Crotty, and Peterson (2004) reported not having included the first adopting state (New Hampshire). On the contrary, if the lone observation of New Hampshire in 1964 is omitted, neither the resulting summary statistics, coefficients, z scores nor the number of observations matches the figures presented in their paper. For the lottery data, a total of twenty-seven adoptions occurred; eighteen of which occurred in five different years (1972, 1974, 1982, 1984, and 1986), meaning there are multiple tied events.

Event history models using logistic regression with a linear measure of time to account for duration dependence are estimated (Buckley and Westerland 2004). Doing so maintains the original analytical approach used by Grossback et al. As in the original article, the outcome variable is coded 0 for state years in which no lottery was adopted, 1 for state-years that represent an adoption. States drop from the data set once they experience the event.

The models include two alternative specifications to measure ideological distance. The measures are named after the authors who use the different strategies. CAM refers to the authors of this paper. GNCP refers to the original authors. The two measures differ in (1) their approach to measuring ideological distance in the first adoption year, (2) their treatment of tied adoptions in the most recent year, and (3) the weighting of the most recent adopters. Per the recommendations above, CAM ideology uses zero as the measure of ideological distance for all fifty states in 1964, uses the average of tied recent adopters for last adopter ideology in equation (1), and uses a denominator of 2 to provide up-weighting for last adopter ideology. As noted above, for GNCP ideology, it is unclear how the authors calculated their ideological distance measure for 1964 and how they approached weighting and tied recent adoptions. The two measures are not highly correlated, r(901) = .55, p < .05; thus, the new specification clearly makes a difference in the final calculation of the measure. R and STATA code are available for calculating CAM ideology at https://doi.org/10.7910/DVN/ELI51F.

In addition to the two measures of ideological distance, there are several control variables that are included for a reliable replication of the original lottery analysis from Berry and Berry (1990) and the updated analysis from Grossback, Nicholson-Crotty, and Peterson (2004). These include measures for election years, state fiscal health, per capita income, religious fundamentalism, government liberalism, and adoptions by neighbor states. Given that the ideological distance variables are of interest to this study, the control variables are not discussed but are included in the results table.

Replication Results

Table 1 provides the results of replicating GNCP’s analysis of Berry and Berry’s (1990) lottery data, but with also substituting the CAM ideology measure of ideological distance. The Berry and GNCP models replicate Table 1 of Grossback, Nicholson-Crotty, and Peterson (2004). The results for models 1 and 2 are largely the same as GNCP’s though different z scores are estimated, which result in not all the same variables reaching statistical significance. This is still the case if one-tailed tests are used, as they were in GNCP’s original paper. The core result, however, is replicable. Namely, neighbor adoptions are statistically significant in the original Berry model but not when ideological distance is added as a covariate. Instead, ideological distance has the expected negative effect. Namely, states that are further from past adopters ideologically are less likely to adopt a lottery. Curiously, GNCP reported that neighbor adoptions fell below the .05 threshold of statistical significance when they added their relative ideology measure; however, they also report using p values from one-tailed tests. Their reported z score for neighbor adoptions was 1.68, which is above the 1.645 threshold used for one-tailed tests and a .05 critical value. Thus, it would have been significant. The replication using their data yields a z score of 1.599, which would not be statistically significant using a one-tailed test and a .05 critical value.

Table 1.

Determinants of State Lottery Adoptions (1964–1986) Using GNCP Ideological Distance and Alternative Specification.

Variables	Berry Model	GNCP Model	CAM Ideology
GNCP ideology		−0.110* (4.139)
CAM ideology			−0.015 (0.833)
Gubernatorial election	1.719* (2.181)	1.693* (2.085)	1.704* (2.155)
Year prior to election	1.293 (1.650)	1.364 (1.700)	1.288 (1.640)
Fiscal health	−3.214 (1.247)	−5.206 (1.714)	−4.881 (1.672)
Per capita income	0.044* (3.175)	0.021 (1.052)	0.010 (0.605)
Religious fundamentalism	−0.080* (2.235)	−0.167* (2.789)	−0.163* (3.011)
Government liberalism		−0.002 (0.016)	−0.002 (0.199)
Neighbor adoptions	0.474* (2.581)	0.335 (1.599)	0.215 (1.102)
Year count		0.179* (2.968)	0.180* (3.270)
Constant	−8.791* (5.175)	−6.473* (2.960)	−6.523* (3.364)
N	901	901	901

*p < 0.05.

Moving to the CAM model, the negative effect of ideological distance is not replicable using the CAM ideology specification of ideological distance. This is striking, given that the measure is based on reasonable decisions resulting from the authors’ verbal description of the measure in their original article. The decisions must be different, but there is no clear documentation regarding how their decisions were made and, thus, how they may be affecting the results. Even when one-tailed hypothesis tests are conducted (as they were by Grossback et al.), the alternative ideological distance variable does not reach a traditional level of statistical significance. This contradicts GNCP’s conclusion—at least for state lotteries—that ideological distance inhibits policy adoption. Furthermore, the differences between models 2 and 3 illustrate how not knowing the exact decisions made in the original measurement makes it difficult to accurately replicate the results. The CAM ideology measure is based on reading GNCP’s article, whereas the results in the GNCP model are based on the data sent by the original authors. To be perfectly clear, no form of misconduct is being alleged, as it appears in correspondence with the authors that measurement decisions were simply lost in the fifteen years since the original article’s publication. This does, however, illustrate the importance of documenting such decisions and making that documentation available for future replication.

Concluding Thoughts on Measuring Ideological Distance

In order to properly accumulate evidence for the purpose of understanding the generalizability of ideological diffusion across policies and time, researchers should ensure that their measures are reliable and consistent. The CAM ideology specification makes the most sense for diffusion study applications. Setting the starting value at zero is useful because there are no past adopters to compare to in the first observed adoption year. It allows the use of the first year of diffusion data by using zero for the measure of relative ideology. For tied adoptions in the most recent adopting year, CAM ideology adds the average of all other past adopters and the average of those tied most recent adoptions. Finally, it weights all other past adopters and the most recent adopters equally, effectively upweighting the ideology of recent adopters. Perhaps most importantly, however, researchers must be explicit in describing these choices. Such context is necessary for reliably comparing results across studies and for future replication efforts. Without an explicit understanding of these choices, it is possible that researchers are measuring different things, even though they think they are measuring the same concept.

Supplemental Material

Supplemental Material, SLGR_19-0047R1,_Text_1_Supplement - Clarifying the Measurement of Relative Ideology in Policy Diffusion Research

Supplemental Material, SLGR_19-0047R1,_Text_1_Supplement for Clarifying the Measurement of Relative Ideology in Policy Diffusion Research by Victor D. Cruz-Aceves and Daniel J. Mallinson in State and Local Government Review

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Daniel J. Mallinson

Supplemental Material

The supplemental material for this article is available online.

Notes

References

Baybeck

Brady

Berry

William D.

Siegel

David A.

. 2011. A strategic theory of policy diffusion via intergovernmental competition. Journal of Politics 73:232–47.

Berry

Frances Stokes

Berry

William D.

. 1990. State lottery adoptions as policy innovations: An event history analysis. American Political Science Review 82:395–415.

Berry

William D.

Ringquist

Evan J.

Fording

Richard C.

Hanson

Russell L.

. 1998. Measuring citizen and government ideology in the American states, 1960-93. American Journal of Political Science 42:327–48.

Boushey

Graeme

. 2016. Targeted for diffusion? How the use and acceptance of stereotypes shape the diffusion of criminal justice policy innovations in the American states. American Political Science Review 110:198–214.

Bradford

Ashley C.

Bradford

David W.

. 2017. Factors driving the diffusion of medical marijuana legalisation in the United States. Drugs: Education, Prevention and Policy 24:75–84.

Bromley-Trujillo

Rebecca

Butler

J. S.

Poe

John

Davis

Whitney

. 2016. The spreading of innovation: State adoptions of energy and climate change policy. Review of Policy Research 33:544–65.

Buckley

Jack

Westerland

Chad

. 2004. Duration dependence, functional form, and corrected standard errors: Improving EHA models of state policy diffusion. State Politics & Policy Quarterly 4:94–113.

Caughey

Devin

Warshaw

Christopher

. 2016. The dynamics of state policy liberalism, 1936–2014. American Journal of Political Science 60:899–913.

Chandler

Jess

. 2009. Trendy solutions: Why do states adopt sustainable energy portfolio standards? Energy Policy 37:3274–81.

10.

Cruz-Aceves

Victor

. 2018. The diffusion of morality policies. PhD diss. Kiel, Germany: Christian-Albrechts-Universität.

11.

Desmarais

Bruce A.

Harden

Jeffrey J.

Boehmke

Frederick J.

. 2015. Persistent policy pathways: Inferring diffusion networks in the American states. American Political Science Review 109:392–406.

12.

Enns

Peter K.

Koch

Julianna

. 2013. Public opinion in the U.S. states: 1956 to 2010. State Politics & Policy Quarterly 13:349–72.

13.

Finger

Leslie K.

2018. Vested interests and the diffusion of education reform across the states. Policy Studies Journal 46:378–401.

14.

Glick

Henry R.

Hays

Scott P.

. 1991. Innovation and reinvention in state policymaking: Theory and the evolution of living will laws. Journal of Politics 53:835–50.

15.

Gray

Virginia

. 1973. Innovation in the states: A diffusion study. American Political Science Review 67:1174–85.

16.

Grossback

Lawrence J.

Nicholson-Crotty

Sean

Peterson

David A. M.

. 2004. Ideology and learning in policy diffusion. American Politics Research 32:521–45.

17.

Hannah

A. Lee

Mallinson

Daniel J.

. 2018. Defiant innovation: The adoption of medical marijuana laws in the American states. Policy Studies Journal 46:402–23.

18.

Hays

Scott P.

1996. Influences on reinvention during the diffusion of innovations. Political Research Quarterly 49:631–50.

19.

Karch

Andrew

Cravens

Matthew

. 2014. Rapid diffusion and policy reform: The adoption and modification of three strikes laws. State Politics & Policy Quarterly 14:461–91.

20.

Leiser

Stephanie

. 2017. The diffusion of state film incentives: A mixed-methods case study. Economic Development Quarterly 31:255–67.

21.

Leung

Kwan-Moon

Elashoff

Robert M.

Afifi

Abdelmonem A.

. 1997. Censoring issues in survival analysis. Annual Review of Public Health 18:83–104.

22.

Makse

Todd

Volden

Craig

. 2011. The role of policy attributes in the diffusion of innovations. Journal of Politics 73:108–24.

23.

Mallinson

Daniel J.

2019. Who are your neighbors? The role of ideology and decline of geographic proximity in the diffusion of policy innovations. Policy Studies Journal. https://doi.org/10.1111/psj.12351

24.

May

Peter J.

1992. Policy learning and failure. Journal of Public Policy 12:331–54.

25.

Ritchey

Mark

Nicholson-Crotty

Sean

. 2015. “Blue ribbon” commissions, interest groups, and the formulation of policy in the American states. Policy Studies Journal 43:70–92.

26.

Seljan

Ellen C.

Weller

Nicholas

. 2011. Diffusion in direct democracy: The effect of political information on proposals for tax and expenditure limits in the U.S. states. State Politics & Policy Quarterly 11:348–68.

27.

Sylvester

Steven M.

Haider-Markel

Donald P.

2016. Buzz kill: State adoption of DUI interlock laws, 2005–11. Policy Studies Journal 44:491–509.

28.

Walker

Jack L.

1969. The diffusion of innovations among the American states. American Political Science Review 63:880–99.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.19 MB