The Robust Relationship Between Conspiracism and Denial of (Climate) Science

Data-Analytic Choices

First, Dixon and Jones dismiss our conclusion for the blogs survey on the grounds that the data are strongly skewed. They fail to recognize that for these data—for the very reason they cite—we used an ordinal rank-based analysis. The claims made by Dixon and Jones (e.g., about the effects of removing supposedly outlying observations) are tied to their use of a metric framework that their own critique identifies as inappropriate and that we did not use. They have thus provided no justification to dismiss our results, as our analysis recognized and accounted for the skew.

Second, for both data sets, we modeled the full covariance matrix of the data using structural equation modeling (SEM), thereby eliminating measurement error (Coffman & MacCallum, 2005). Dixon and Jones, by contrast, rely on bivariate linear regression. Linear regression is susceptible to attenuation through measurement error (e.g., Osborne & Waters, 2002), and a bivariate focus prevents identification of potential suppressor variables that may also attenuate the relationship of interest (e.g., Paulhus, Robins, Trznesniewski, & Tracy, 2004). The absolute magnitudes of the coefficients reported by Dixon and Jones are thus more than 3 times smaller (≈ .06) than the error-free estimates (≈ .20) we reported for both surveys.

Third, Dixon and Jones further weakened the focal bivariate effect by removing from the panel-survey data 35 respondents who responded “neutral” to all CY and CLIM items. Post hoc removal of data is one of the degrees of freedom available to researchers that has recently been brought into critical focus (Simmons, Nelson, & Simonsohn, 2011). Researchers have many options to remove subsets of data on the basis of one or another plausible-sounding criterion. The removal has no effect when the data are more appropriately modeled using SEM (see Fig. 1b).

Reversing the Dependent and Independent Variables

Dixon and Jones then reverse the role of the dependent and independent variables in the panel-survey data. They give no justification for this reversal other than claiming that “with nonlinear models, it is important to explore relationships in both directions” (p. 665). However, no nonlinear model has ever been proposed for these data by us or anyone else, so this justification is moot. The only model motivated by theoretical considerations and prior empirical findings is one in which CLIM is predicted by CY. When CLIM is regressed on CY, there is no evidence for a nonlinear relationship, quadratic F(1, 998) = .001, p > .1 (Pedhazur’s method of creating a product term was used for the quadratic effect), which nullifies the purported statistical justification for the reversal. Moreover, it has been well established that “nonlinear models often do not hold up well in new samples . . . , and that nonlinear relations may be approximated by more complex linear models” (Bentler & Chou, 1987, p. 87).

Dixon and Jones furthermore claim it is “inappropriate to use a linear function of CY to predict CLIM, as the underlying relationship in that direction is multivalued” (p. 665), citing Bentler and Chou (1987) in support. The nonlinear function expressing CY as a function of CLIM, when the roles are reversed, is indeed multivalued—but the data are not, as Figure 1 in Dixon and Jones shows: The data are simply spread out more at the lower end of the CY scale, which introduces nothing but heteroscedasticity in the regression of CLIM on CY. (Because of that heteroscedasticity, for the panel-survey data we also computed bootstrapped confidence intervals for the parameter estimates in the SEM, thereby affirming the robustness of the effect.)

The fact that Dixon and Jones’s reversed-variable model obtained results different from those we reported for both data sets is unsurprising: Any correlation matrix can be fit equally well by more than one model. This issue of equivalent models has been discussed repeatedly (e.g., Raykov & Marcoulides, 2001; Tomarken & Waller, 2005). The consensus solution is to limit the models under consideration to those that have a meaningful theoretical interpretation (MacCallum, Wegener, Uchino, & Fabrigar, 1993). Alternative models should reflect alternative theoretically motivated hypotheses, any mention of which is conspicuously lacking in Dixon and Jones’s Commentary. Some statistical models are more theoretically motivated, meaningful, and justifiable than others. In both of our studies, we theorized that individual differences in conspiracist ideation, a cognitive style or personality attribute that is reliably associated with various predictors (e.g., Darwin, Neave, & Holmes, 2011), would predict attitudes toward scientific propositions. By reversing the role of variables, Dixon and Jones tacitly claim that attitudes toward scientific propositions cause one’s personality, although this implication is obscured by the failure to mention any theory. The reader is left in the dark as to what any of this means, which is ironic in light of Dixon and Jones’s admonition against use of SEM as a “black box” (p. 666).

Conclusion

In summary, Dixon and Jones’s analysis has no bearing on the results we reported for either survey because it reaches its main conclusion only by reversing the role of criterion and predictor without any theoretical justification. The only statistical justification offered for that reversal (“with nonlinear models, it is important to explore relationships in both directions”) demonstrably does not apply. Without that reversal, Dixon and Jones’s criticism involving nonlinear relationships is moot because none are present.

Contrary to what Dixon and Jones assert, the association between CLIM and CY we reported is robust and, self-evidently, replicable. Unless interfered with by contrived data-analytic choices, the observed association is comparable in magnitude to, or greater than, well-established effects with notable public-health implications, such as the correlations between combat exposure and risk for posttraumatic stress disorder (r = .11) and between lead exposure and childhood IQ (Meyer et al., 2001). Similarly, Greenwald, Banaji, and Nosek (in press) have shown that even small correlations (.15 < r < .24) between performance on an Implicit Association Test measuring racial attitudes and discriminatory impacts can be societally significant. We conclude that the data of our panel and blogs surveys may be societally relevant. They also mesh well with existing literature, which has repeatedly reported that the rejection of scientific findings is predicted by various forms of motivated cognition, including conspiracist ideation (Bogart & Thorburn, 2005; Kalichman, 2009; Nattrass, 2011; Smith & Leiserowitz, 2012).