Abstract
Many scholars have argued for the need to communicate openly not only scientific successes to the public but also limitations, such as the tentativeness of research findings, in order to enhance public trust and engagement. Yet, it has not been quantitatively assessed how the perception of scientific uncertainties relates to engagement with science on an individual level. In this article, we report the development and testing of a new questionnaire in English and German measuring the perceived uncertainty of scientific evidence. Results indicate that the scale is reliable and valid in both language versions and that its two subscales are differentially related to measures of engagement: Science-friendly attitudes were positively related only to ‘subjectively’ perceived uncertainty, whereas interest in science as well as behavioural engagement actions and intentions were largely uncorrelated. We conclude that perceiving scientific knowledge to be uncertain is only weakly, but positively related to engagement with science.
Keywords
1. Introduction
Gathering empirical evidence is probably the most important element of science, distinguishing it from pseudoscience and irrational beliefs (Franklin, 2009). But scientific evidence is uncertain in nature: First, it is always tentative, being subject to revision on the basis of new findings. Second, it can offer only a ‘degree of confirmation’ (Popper, 1954) for a given idea or hypothesis, or as science writer Ben Goldacre (2005) puts it: ‘science is all about the error bar’. This holds true for critical rationalist or falsificationist approaches in the tradition of Popper (1959), but even more pronounced for Bayesian approaches to assessing the truth or certainty of conflicting hypotheses (e.g. Howson and Urbach, 1989).
This uncertainty inherent to scientific evidence, once a concern of scientists and philosophers of science only, has increasingly gained attention in the general public, especially in terms of the risks associated with societal decisions based on uncertain or controversial evidence (Bammer and Smithson, 2008; Pellizzoni, 2003). With calls for more public engagement with science on the rise (e.g. Pearson, 2001; cf. Bauer and Jensen, 2011), many scholars have argued for a more open way of presenting scientific uncertainty to the public (e.g. Leshner, 2003). For instance, regarding health-risk assessment, Johnson and Slovic (1995) noted that the ‘explicit discussion of uncertainties in risk estimates would […] improve the public’s views of environmental hazard management’ (p. 485). As Jensen (2008) points out, even Karl Popper was convinced (and implicitly suggested in his works) that communicating uncertainty would increase scientists’ trustworthiness.
With regard to societal decisions on risk-related issues, Wynne (1992) describes several kinds of scientific uncertainties: In his terminology, uncertainty simply refers to unknown probabilities due to a lack of research or evidence (‘known unknowns’), while ignorance refers to facts and circumstances decision-makers are totally unaware of (‘unknown unknowns’). Indeterminacy, on the other hand, underlies every other kind of uncertainty and refers to the complexity of causal chains and contingencies in social behaviour that can lead to unforeseeable outcomes.
While the public concern with scientific uncertainty has mostly been the subject of large-scale societal analyses (e.g. Irwin and Wynne, 1996) or case studies (Tytler, 2001), it has rarely been tested how uncertainties inherent to the scientific method affect engagement with science on an individual level. However, the extent to which laypeople recognize these uncertainties may be crucial for their engagement with science: interest in and attitudes towards science, but also actual engagement behaviours, such as attending town hall meetings on scientific topics.
To gain insight into these relationships, we first designed a short rating scale measuring the perceived uncertainty of scientific evidence, which was developed and tested in two language versions (Study 1). Personal background variables such as participants’ level of education and their individual need for cognitive closure were assessed to estimate the construct validity of the scale. Questionnaires on interest in science and attitudes towards science served as proxy measures for engagement.
Another population-based survey was conducted in order to cross-validate the scale and to further assess the association between perceived uncertainty of scientific evidence and different measures of engagement such as interest in science, attitudes and actual engagement behaviour (Study 2). To examine the scale’s construct validity, in this second survey we also assessed its convergent validity by comparing it with measures of epistemological beliefs (a nomologically related construct), and its discriminant validity by comparing it with the Big Five personality dimensions.
Scientific uncertainty and engagement with science
Naturally, researchers are the principal source of information about scientific uncertainty. Adult laypersons receive such information mostly through the mass media and, with new communication channels such as personal websites and blogs on the rise, increasingly directly from scientists (Bonetta, 2007). The extent to which the two sources – scientists themselves and the media – communicate scientific uncertainty is still disputed. Traditionally, scientists seem to have shown a tendency to present their findings as overly certain when communicating with journalists or the general public, perhaps to appear more respectable and authoritative (Pinch, 1981), or for fear that informing laypersons about the tentativeness or inconsistency of scientific findings may strengthen anti-scientific attitudes and lessen support for research funding (Jensen, 2008). Yet, readiness to admit scientific uncertainty seems to be on the rise (Poliakoff and Webb, 2007; Zehr, 1999), which can be seen as an attempt by scientists to appear more honest and trustworthy (Wildson and Willis, 2004).
As for the role of the media, Stocking (1999) reported that most media studies at the time suggested that journalists routinely ‘present science as more solid and certain’ than it is, for instance, by ignoring concerns or contradictions stated in the original sources or by failing to present the findings in context (p. 24). Many recent studies support this notion (Brechman et al., 2009; Lai and Lane, 2009). However, later work by Stocking and Holstein (2009) has shown that sometimes journalists also let science appear more uncertain and fallible than it is, depending on the context and the message they want to convey. This is in line with research on risk perception by Kasperson et al. (1988), who stated that technological risks can either be amplified or attenuated by psychological and cultural processes such as the flow of information through the media.
Given this tradition of overly ‘definitive’ science communication, it could be assumed that on an individual level, scientific uncertainty negatively affects laypersons’ attitudes towards science. Yet, the rare empirical tests of this hypothesis suggest otherwise. In a study by Crismore and Vande Kopple (1997), high school students had more positive attitudes towards a scientific idea when the text describing it used ‘hedges’: words expressing tentativeness or uncertainty. Using a more rigorous design, Jensen (2008) found that including hedges in the media coverage of scientific results increased participants’ view of the scientists as trustworthy, especially when expressed by scientists themselves. Conversely, inducing uncertainty by exposing participants to conflicting news articles about a scientific controversy either increased or decreased scientists’ credibility ratings, depending on the topic of the controversy (Jensen and Hurley, 2012).
Measuring the perceived uncertainty of scientific knowledge
Psychologically, uncertainty can be conceptualized in many ways, but as Powell and colleagues point out, most concepts encompass people’s individual perception of ‘not knowing’ something, the accompanying emotional experiences and an engagement in communication processes concerning the subject of uncertainty (Powell et al., 2007). Here, we focus on laypersons’ individual perceptions of how uncertain scientific knowledge is in general, that is, not related to a particular research question or a specific kind of risk assessment (such as the risk-benefit evaluation of nuclear power).
Based on research on risk perception, Slovic et al. (2004) popularized the idea that people process uncertainties in two different ways: ‘Risk as analysis’ – where the analytical or rational system assesses risk based on normative rules and formal logic – and ‘risk as feelings’, an intuitive way of experiencing risks that is less accessible to conscious reasoning and rooted in everyday experience. This distinction is based on well-established dual-process models of information processing (Epstein, 1994), and thus was also considered in our assessment of perceived scientific uncertainty.
Previous work from two strands of research addresses how laypersons understand the nature of scientific evidence: research on epistemological beliefs and on scientific literacy, more specifically, nature of science. Studies on epistemological beliefs have assessed subjective perspectives on the source and the nature of knowledge (Hofer and Pintrich, 2002). The construct comprises beliefs about ‘the source, certainty, and organization of knowledge, as well as the control and the speed of learning’ (Schommer, 1994: 29). Recent approaches have been more confined to beliefs about the nature of scientific knowledge and often measure domain-specific beliefs (Stahl and Bromme, 2007). While epistemological beliefs have indeed been shown to vary for different scientific domains (Hofer, 2000), a measure of domain-general science beliefs is needed for a comparison with general interest in and attitudes towards science.
Scientific literacy refers to the understanding of science on three dimensions: knowledge of scientific facts, knowledge of scientific methods and an understanding of the societal impacts of science (Miller, 1983, 2004). In practice, research on scientific literacy has focused mostly on factual knowledge, while well-established measures to assess the public’s understanding of the scientific process or the logic of experimentation (hereafter called ‘methodological scientific knowledge’) are still lacking (Pardo and Calvo, 2004). This is the focus of the second strand of research, ‘nature of science’, which examines laypersons’ understanding of science and how scientific results are obtained, with a strong emphasis on formal science education (Lederman, 1999; Ratcliffe, 2004). Quantitative questionnaires measuring nature of science beliefs have been developed but are criticized with regard to test economy and psychometric properties (Lederman et al., 1998).
Taken together, in order to assess the perceived uncertainty of scientific evidence and its relationship to engagement with science on an individual level, no validated, well-established questionnaires exist that focus on domain-general perceptions of the uncertainty of scientific evidence, reflect dual-process theories of perception and are suitable for large-scale assessments of adults. Study 1 describes the development of such an instrument.
2. Study 1: Questionnaire development and validation
In order to measure laypersons’ general perception of scientific evidence as uncertain (versus stable and unchanging), a short and concise rating scale in a German and an English version was developed and tested in population-based surveys in Germany and the United States. Concurrently, we assessed a number of variables which could provide information about the validity of the questionnaire (methodological scientific knowledge, formal education and need for cognitive closure). As proxy measures for engagement with science, we assessed participants’ interest in science and attitudes towards science.
Concerning the construct validity of the instrument, we hypothesized that methodological scientific knowledge and formal education would positively relate to perceived uncertainty of scientific findings. The more people know about science, the more they should acknowledge inconsistent and conflicting results, leading to a greater perceived tentativeness of scientific knowledge. Conversely, we expected higher levels of need for cognitive closure to correlate negatively with the perception of scientific uncertainty, since this construct describes a person’s discomfort with ambiguity and open questions (Kruglanski, 2004). Additionally, a recent study by Feist (2012) has shown a positive relationship between interest in science and need for cognition (closely and inversely related to need for cognitive closure).
No hypotheses could be formulated about how the perception of scientific evidence as uncertain relates to interest in science and attitudes towards science, two major factors that determine personal engagement with science. Since greater perceived uncertainty may signal a higher affinity with science, it could as well indicate more interest in science and, accordingly, more positive attitudes towards science. Conversely, perceived uncertainty could also lead laypersons to view science and scientists as less knowledgeable, trustworthy or relevant (Bak, 2001), which may result in decreased interest and less favourable attitudes.
Method
Participants and procedure
In both countries, the set of questionnaires was administered to participants of an online panel managed by a globally operating market research institute (GfK, Nuremberg). Stratified sampling was used to maximize representativeness of the sample with regard to sex, age groups (18–29 years, 30–39 years, 40–49 years, 50 years and older), region of residence and level of education. A total of 574 (Germany) and 590 (United States) participants completed the survey. Interview data were excluded on the basis of interview duration (less than 7 minutes) and ‘straightlining’ (on two or more of the eight question grids), resulting in a final sample of N = 502 (Germany) and N = 492 (United States) respondents, respectively. In both countries, the final sample was approximately representative of the adult population with regard to sex (52.8% female, Germany; 59.1% female, United States), age groups (age M = 46.3, standard deviation (SD) = 13.3 years (Germany); M = 47.9, SD = 18.6 years (United States)), region of residence and level of education.
Measures
For measuring the perceived uncertainty of scientific evidence, a pool of 30 items was generated in German and in English. Some of these were adapted and, if necessary, translated from two previously published instruments: the Epistemological Questionnaire (Schommer, 1998; Schommer-Aikins, 2004) and the VOSTS (Aikenhead and Ryan, 1992). A distinction was made between ‘objective’ items, which were general statements about scientific evidence (e.g. ‘What has been published in a prestigious scientific journal can be seen as proven’), and ‘subjective’ items, which assess how the perceived uncertainty of scientific evidence is reflected in everyday behaviour (e.g. ‘I constantly change my lifestyle and diet in accordance with the latest scientific findings’). As a response format, a six-point Likert-type scale ranging from ‘strongly disagree’ to ‘strongly agree’ was chosen. For these pilot surveys only, a seventh category labelled ‘don’t know’ was included. Prior to its application, the item pool (in German) was checked for face validity by colleagues from psychology and social sciences, which led to minor modifications in wording.
Methodological scientific knowledge, that is, knowledge about experimental procedures and the logic of scientific inquiry, was assessed using a short test consisting of five vignettes, each followed by a multiple-choice question (see online Appendix at http://pus.sagepub.com). Two vignettes (about using a control group and about the chance of passing on a heritable illness) had been used in previous surveys (National Science Board, 2010); the remaining three address the ideas of randomization, replication and double-blinding of studies (GRADE Working Group, 2004; Kiene, 2001). After each vignette, participants had to choose from two or more alternatives the one that would ensure the most sound or valid scientific methodology; correct answers were summed (Kuder–Richardson coefficient = .43 (German sample); .56 (English sample)).
To assess need for cognitive closure, in the US survey, a seven-item scale was used based on the instrument by Houghton and Grewal (2000; items were selected based on item statistics). For the German survey, the 16-NCCS (Schlink and Walther, 2007) was also shortened to seven items. Since no item statistics were available, we selected those that most closely matched the statements contained in the English questionnaire. As for all remaining scales in this study, answers were scored on a six-point Likert-type scale ranging from ‘strongly disagree’ to ‘strongly agree’, but without a ‘don’t know’ category. Cronbach’s α coefficient was low in the US survey (alpha = .33), but acceptable for the German version (alpha = .69).
Participants’ interest in science was measured using an adaption of the Short Scale of Political Interest (SSPI; Otto and Bacherle, 2011) which is available both in English and in German. In the five items of the SSPI, the word ‘politics’ was replaced with ‘science’; a sixth item was added asking about participants’ career interest in science. Internal consistency was satisfactory in both language versions (alpha = .91, Germany; .86, United States).
For the assessment of attitudes towards science, we used an adapted version of the Test of Science-Related Attitudes (TOSRA; Fraser, 1978), selecting only the subscales ‘normality of scientists’ (sample item: ‘Scientists usually like to go to their laboratories when they have a day off’), ‘adoption of scientific attitudes’ (sample item: ‘I like to listen to people whose opinions are different from mine’) and ‘social implications of science’ (sample item: ‘Scientific discoveries are doing more harm than good’). The subscales were further shortened to five items each, based on content and face validity, and translated into German by the authors. ‘Social implications of science’ had a satisfactory internal consistency (alpha = .86, Germany; .78, United States), while alpha scores for ‘normality of scientists’ (.53, Germany; .38, United States) and ‘adoption of scientific attitudes’ (.63, Germany; .54, United States) were lower.
Statistical analyses
For the construction of the scale and subsequent affirmation of its dimensionality, we randomly split into half the data sets of both language samples. On one half, exploratory factor analyses were performed, while the other half was used for confirmatory testing. Fit indices were classified according to the thresholds by Browne and Cudeck (1992). Simple correlation coefficients were calculated to test for the hypothesized relationships between perceived uncertainty of scientific evidence and the remaining variables.
Results and discussion
Item analysis and scale construction
In both language versions, item characteristics such as mean, skewness and kurtosis, as well as item difficulty, were within normal ranges (Bühner, 2006; Fisseni, 1997). On one half of each data set, a principal axis factor analysis using varimax rotation with Kaiser normalization was conducted. According to the Kaiser criterion, seven factors (US data set) or eight factors (German data set) would have had to be retained. Scree plot (Cattell, 1966) and parallel analysis (Allen and Hubbard, 1986) criteria recommended three different factors, while theoretically we assumed two underlying dimensions. Thus, we ran additional analyses with the number of factors restricted to two and three, respectively. Counterbalancing common factor loadings, meaningfulness of factors and range of content, a two-factor solution was chosen consisting of the same 10 items in English and German, all reverse coded. Table 1 provides the wording, factor loadings, selectivity and item difficulty for each item.
Factor loadings for the ‘perceived uncertainty of scientific evidence’ subscales.
The first subscale, measuring ‘objectively’ perceived uncertainty of scientific evidence, contains five statements of universal validity concerning the nature of scientific knowledge (e.g. ‘If scientists have worked carefully, their results can be seen as certain’). The subscale measuring ‘subjectively’ perceived uncertainty contains three items on everyday cognition and behaviour, and two more universal statements that imply subjectivity in their wording (e.g. ‘as far as I am concerned … ’; see Table 1).
Confirmatory factor analyses for this two-factor solution were performed on the remaining halves of the data sets. For each language sample, a one-factor model, a two-factor model and a bifactor model (Holzinger and Swineford, 1937) were tested. Table 2 reports the fit indices for each of these factor solutions. In both language versions, the two-factor models yielded much better fit indices than the single factor models. Overall, the bifactor model had a somewhat better model fit than the two-factor solution. This was especially true for the English language sample; for the German subset, the two-factor and bifactor models were of comparable quality. These results indicate that the two subscales most likely measure different aspects of the perception of scientific evidence, while providing evidence for an underlying general or group factor influencing all 10 items which was found especially in the English language sample.
Fit indices for different factor solutions (see the text for details), cut-off criteria according to Browne and Cudeck (1992).
RMSEA: Root Mean Square Error of Approximation; CFI: Comparative Fit Index.
Internal consistencies of the newly developed scales were analysed using the full data sets of both language versions. Cronbach’s α for the subjective subscale was .75 (German version) and .76 (English version), respectively, and for the objective subscale .83 (German version) and .87 (English version), which is satisfactory for very short scales with no item intercorrelations greater than r = .50.
Preliminary validation and relationship with engagement measures
Table 3 provides correlations of both subscales with the remaining variables. As for construct validity, results support the hypothesized positive correlation with methodological scientific knowledge, especially in the US sample, and the expected negative correlation with need for cognitive closure. In contrast, level of education was only weakly related to perceived scientific uncertainty. Correlations with interest in science were relatively weak and the pattern inconsistent: A negative relationship emerged between interest and the ‘objective’ subscale in the United States, whereas in the German sample interest and the ‘subjective’ subscale were positively related. Regarding attitudes towards science, in both samples the scores on all three TOSRA subscales were substantially positively related to subjectively perceived uncertainty, but not or only weakly related with objectively perceived uncertainty (Table 3).
Study 1 correlations with perceived uncertainty of scientific evidence.
Taken together, these findings demonstrate satisfactory reliability and model fit for a bifactor solution, as well as first evidence for the scale’s construct validity. The results suggest a differential relationship between the two subtypes of perceived uncertainty and attitudes toward science: Viewing scientific evidence as ‘objectively’ uncertain and tentative was only weakly and inconsistently associated with the TOSRA subscales, while subjectively perceiving scientific findings as tentative was substantially related to science-friendly attitudes. Results on the association between perceived uncertainty and interest in science were inconclusive.
3. Study 2: Cross-validation of the questionnaire and its relationship to engagement with science
The English questionnaire developed and tested in Study 1 was cross-validated in a new population-based sample in the United States. One aim of this study was to further analyse the construct validity of the scale. As in Study 1, we measured participants’ methodological scientific knowledge, level of education and need for cognitive closure. In addition, to further explore the scale’s nomological network, we assessed its convergent validity by comparing it with two measures of epistemological beliefs. We hypothesized that perceived uncertainty of science would relate positively to more sophisticated epistemological beliefs because of the two constructs’ similarities (see the ‘Introduction’ section). Discriminant validity, on the other hand, was assessed by measuring the Big Five personality dimensions, as we expected no substantial correlations with perceived scientific uncertainty.
The major goal of our research was to explore the relationship between perceived scientific uncertainty and measures of engagement with science; thus, interest in science and attitudes towards science were assessed. Since the correlations with the TOSRA subscales in Study 1 proved inconclusive, this construct was operationalized differently (see the ‘Method’ section). Previously, we had only assessed proxy measures of engagement with science. However, a well-documented attitude–behaviour gap exists (Glasman and Albarracín, 2006; Wicker, 1969). Therefore, we additionally assessed actual engagement behaviour and behavioural intentions.
Method
Participants and procedure
Questionnaires were administered to 754 participants of a US online panel by GfK, Nuremberg. There was no overlap of participants between this study and the previous one. After exclusions based on interview duration (less than 12 minutes) and ‘straightlining’ (a quarter or more of the question grids), the final sample consisted of N = 587 participants, representative of the adult US population with regard to sex (53.3% female), age groups (age M = 50.4, SD = 15.0 years), region of residence and level of education.
Measures
Table 4 gives an overview of the constructs assessed in this study. To measure perceived uncertainty of scientific evidence, the questionnaire described in the previous section was used with the same six-point Likert-type scale, but without the ‘don’t know’ category (Cronbach’s α = .88, objective subscale; .61, subjective subscale). Measures for methodological scientific knowledge, interest in science and need for cognitive closure were the same as in the pilot investigation (Study 1), with a Kuder–Richardson coefficient of .55 and Cronbach’s α .82 (interest in science) and .70 (need for cognitive closure).
Study 2 correlations with perceived uncertainty of scientific evidence.
Attitudes towards science were measured with the ‘Thinking about Science’ questionnaire introduced by Cobern and Loving (2002). Due to space constraints, only six of the original nine subscales were used: ‘Science & the Economy’ (7 items, sample item: ‘Scientific knowledge is useful in keeping our national economy competitive in today’s world’, alpha = .89), ‘Science & the Environment’ (3 items, sample item: ‘Science can help us preserve our natural environment and natural resources’, alpha = .58), ‘Public Regulation of Science’ (3 items, sample item: ‘Scientists should not be allowed to research anything they wish’, alpha = .47), ‘Science and Public Health’ (2 items, sample item: ‘Scientific research makes important contributions to medicine and the improvement of public health’, alpha = .52), ‘Science and Religion’ (2 items, sample item: ‘Science is a more important source of knowledge than religion’, alpha = .63) and ‘Science for All’ (7 items, sample item: ‘All students should study science during the secondary school levels’, alpha = .85). To better assess actual behavioural engagement actions and intentions, a self-designed three-item scale was used asking about attendance of town hall meetings on scientific topics, being involved as a ‘citizen scientist’ (or wishing to be), and feeling the need to personally discuss relevant topics with scientists (alpha = .78).
To assess convergent validity with a related construct, we measured epistemological beliefs using two questionnaires: the Epistemic Belief Inventory (EBI; Schraw et al., 2002) with its subscales ‘Simple Knowledge’ (sample item: ‘Most things worth knowing are easy to understand’), Certain Knowledge’ (‘What is true today will be true tomorrow’), ‘Innate Ability’ (‘People’s intellectual potential is fixed at birth’), ‘Omniscient Authority’ (‘People shouldn’t question authority’) and ‘Quick Learning’ (‘If you don’t learn something quickly, you won’t ever learn it’; overall alpha = .79). The second inventory was introduced by Conley et al. (2004) and comprises four dimensions, ‘source’ (sample item: ‘Whatever the teacher says in science class is true’), ‘certainty’ (‘All questions in science have one right answer’), ‘development’ (‘Ideas in science sometimes change’) and ‘justification’ (‘A good way to know if something is true is to do an experiment’). Since this instrument has been developed for surveying fifth-graders, caution is required when interpreting the results. Cronbach’s α for the four subscales ranged from .81 (certainty) to .91 (development). To assess discriminant validity, we also administered the 10-item Big Five questionnaire BFI-10 (Rammstedt and John, 2007). All instruments in this study were scored using a six-point Likert-type scale ranging from ‘strongly disagree’ to ‘strongly agree’.
Statistical analyses
In order to confirm the dimensionality of the scale found in Study 1, confirmatory factor analyses were conducted. Simple correlation coefficients were used to test the scale’s convergent and discriminant validity, as well as its relationships with measures of engagement with science.
Results and discussion
As with the English language sample in the first study, confirmatory factor analysis revealed that the bifactor model describes the data better than a one-factor or two-factor solution. Fit indices for RMSEA and CFI were very similar to those obtained in Study 1, whereas χ2/df statistics indicated a poorer model fit, perhaps in part due to the larger sample size of Study 2. Furthermore, Mardia’s test showed that the items were not multivariate normally distributed, which also leads to a decline in chi-square test statistics.
Convergent validity of the scale
As in the previous study in the United States, perceived uncertainty of scientific evidence was positively related to methodological scientific knowledge, as well as negatively to need for cognitive closure, pointing to the general validity of the instrument (see Table 4). As before, only the subjective subscale was weakly associated with participants’ level of education (rs = .18, p < .001; objective subscale: rs = .02, p = .714). The comparison with the two epistemological beliefs questionnaires indicates that perceiving scientific evidence as uncertain only moderately correlates with epistemological beliefs, depending on the specific beliefs and the instrument used. Correlations with the EBI total score show that, as expected, participants who perceive scientific evidence to be less certain or stable generally hold more ‘sophisticated’ epistemological beliefs (objective subscale: r = .33, p < .001; subjective subscale: r = .51, p < .001). Correlations with the separate subscales are weaker on average (Table 4). The ‘certain knowledge’ subscale yielded the weakest correlations. This is plausible because ‘knowledge’ in the EBI items refers to an unspecified truth (e.g. ‘what is true today will be true tomorrow’, ‘sometimes there are no right answers to life’s big problems’) without specifically addressing scientific knowledge (see the ‘Introduction’ section).
Correlations with the questionnaire by Conley et al. (2004) are mixed: perceived uncertainty most closely correlates with the ‘source’ and ‘certainty’ dimensions, the latter explicitly referring to scientific knowledge. According to Conley and colleagues, higher ‘source’ scores represent a view of knowledge as internal to the self, not originating from authorities; higher scores on the ‘certainty’ dimension reflect the belief that there usually is no single right answer to complex problems. Thus, perception of scientific evidence as being uncertain is accompanied by rejection of easy answers and the idea of an ‘Omniscient Authority’. In contrast, relationships to the other two sub-dimensions were less clear: ‘Justification’ (using empirical data to support arguments) was differentially related to objectively and subjectively perceived uncertainty, while ‘development’ (recognizing scientific ideas and theories as evolving) was only related to subjectively perceived uncertainty.
Discriminant validity
We expected zero correlations of the two uncertainty subscales with all Big Five measures, which was true for 7 of the 10 tests (Table 4). However, three weak but statistically significant relationships emerged. Agreeableness was negatively related to objectively perceived scientific uncertainty (r = −.15, p < .001), thus more agreeable participants tended to rate scientific evidence as more stable. Openness to experience was positively related to subjectively perceived uncertainty (r = .13, p = .002). This association became weaker, but remained significant when controlling for level of education (r = .11, p = .009). Finally, neuroticism correlated negatively with subjectively perceived uncertainty (r = −.10, p = .010). This might indicate a tendency for emotionally less stable people to more readily believe in the certainty of scientific knowledge.
Perceived uncertainty and engagement with science
In line with the results of Study 1, interest in science was negatively correlated with objectively perceived uncertainty of evidence and, in this case, also weakly positively with the subjective subscale (Table 4). The short engagement behaviour scale was completely unrelated to perceived uncertainty of scientific evidence. Instead, it was strongly correlated with interest in science (r = .59, p < .001) and to a lesser extent with level of education (rs = .17, p < .001) and openness (r = .15, p < .001), as well as negatively with need for cognitive closure (r = −.20, p < .001), agreeableness (r = −.15, p < .001), female gender (rs = −.21, p < .001) and age (r = −.14, p = .001). This suggests that ‘real world’ engagement activities are more a function of specific interests, demographics and personality variables than of general perceptions of science. Positive attitudes towards science were largely correlated with subjectively perceived uncertainty of evidence (except for the ‘less public regulation’ dimension), and these relationships remain substantial when controlling for methodological knowledge. Objectively perceived uncertainty, in contrast, was unrelated or moderately negatively related to attitudes towards science: Participants who favoured scientific over religious knowledge (‘Science & Religion’) believed in the economical value of scientific research (‘Science & the Economy’), and thought that science education should be mandatory (‘Science for All’), also perceived scientific evidence as slightly less tentative or uncertain. Here, the underlying factor may be a belief in authoritative or superior scientific knowledge compared to other sources of knowledge.
Pooled analysis
For a more comprehensive analysis, data from the two US population-based surveys in Studies 1 and 2 were pooled, resulting in a total of N = 1079 participants. Identical questionnaires were used in both studies for the assessment of perceived uncertainty of scientific evidence, methodological scientific knowledge, need for cognitive closure and interest in science as well as level of education. To compare the two different attitudes towards science inventories, scores were standardized by dividing the total scores by the number of items (i.e. TOSRA, 15; Thinking about Science, 24). This resulted in a score ranging from 1 to 6 for both instruments, with higher scores indicating more positive attitudes towards science, which could be pooled for analysis.
All intercorrelations between these seven variables are shown in Table A1 (see online Appendix at http://pus.sagepub.com). For reasons of clarity, Figure 1 only shows correlation coefficients greater than .20. Considering merely these relationships, ‘objectively’ perceived uncertainty correlates only with methodological knowledge (apart from the subscale intercorrelation), while ‘subjectively’ perceived uncertainty is positively related to knowledge, level of education and attitudes towards science, and negatively related to need for cognitive closure. Positive attitudes towards science correlate only with ‘subjectively’ perceived uncertainty, knowledge and, most strongly, with interest in science.
Pooled sample intercorrelations with correlation coefficients ≥.20. All correlations shown are significant at p < .001.
4. General discussion
In this article, we report the development and testing of a German and an English rating scale measuring the perceived uncertainty of scientific evidence, as well as its relationships with measures of engagement with science. One German and two US population-based surveys indicated satisfactory reliability and validity for the new instrument. Relationships with measures of engagement with science indicate that the perception of scientific evidence as being uncertain is differently related to attitudes towards science, depending on the facet of perceived uncertainty measured. In contrast, interest in science and actual engagement behaviours (or behavioural intentions), while highly intercorrelated, were largely unrelated to perceived uncertainty of scientific evidence.
Based on exploratory factor analyses and a priori considerations, the scale was divided into an ‘objective’ and a ‘subjective’ subscale. This dimensionality is in line with recent dual-process theories of risk and uncertainty perception (Slovic et al., 2004). Confirmatory factor analyses revealed that a bifactor model describes the dimensionality of the scale partially better than a two-factor solution, especially so for the English language version of the questionnaire. This hints at a common dimension representing scientific laypersons’ tendency to see scientific evidence as uncertain, which influences intuitive uncertainty assessments and day-to-day decision making in the same way as it influences more formal, explicit statements regarding the nature of scientific evidence.
Concerning construct validity, in both language versions we found evidence for the theoretically expected relationships with knowledge about scientific methodology, level of education and need for cognitive closure, but correlations were much more pronounced for the ‘subjective’ subscale. One possible explanation is that scientific uncertainty could be easier to discern in this subscale, especially for participants who are less familiar with academic practice, because the items refer to their everyday experience and behaviour instead of requiring deliberate reasoning or use of formal logic (Epstein, 1994). This would also be in line with S. Miller’s (2001) finding that scientific literacy is higher when the measurement items refer to real-world problems or to scientific facts meaningful for everyday life.
Comparison with two nomologically related instruments on epistemological beliefs yielded mixed results. In general, more sophisticated epistemological beliefs were moderately positively related to perceived uncertainty of scientific evidence, with correlations for the subjective subscale being more consistent and, on average, larger. This finding again suggests that when asking laypeople about scientific uncertainty, it might be more feasible to refer to their everyday experience with inconsistent or conflicting research findings. However, the divergent findings even within the subjective subscale point to the heterogeneity of how epistemological beliefs are operationalized, which makes it difficult to use them as a uniform construct (Bromme et al., 2008).
The major goal of this research was to explore the relationship between perceived scientific uncertainty and engagement with science on an individual level. Participants’ attitudes towards science, used as a proxy measure for engagement with science, were positively correlated to their ‘subjectively’ perceived scientific uncertainty, while results on the relationship with ‘objectively’ perceived uncertainty were inconclusive. We found no compelling evidence for a relationship between perceived uncertainty and interest in science. Consistent with prior research (Jensen, 2008), our cross-sectional data suggest that being aware of the uncertainty inherent to scientific knowledge does not necessarily lead to anti-scientific attitudes.
Again, the difference between subjectively and objectively perceived uncertainty is intriguing. One possible explanation is that participants with negative attitudes towards science might more readily reject the absolute statements of the objective subscale (such as ‘Scientific facts, if carefully examined, are valid for all times’). For instance, the most pronounced negative correlation of the objective subscale was for ‘Science and Religion’. That is, participants who do not see science as a superior source of knowledge compared to religion also perceived scientific evidence as more uncertain, which is in line with data on religiosity and support for science (Miller, 2004; Scheufele et al., 2008). Thus, a higher score on ‘objectively’ perceived uncertainty could indicate either more sophisticated beliefs about scientific evidence or, conversely, negative attitudes toward science. Participants with science-friendly attitudes, however, might agree more readily to absolute statements about the certainty of scientific evidence, while they may still acknowledge the tentative and often conflicting nature of scientific findings in their everyday behaviour (hence, the positive correlations with the ‘subjective’ subscale).
Science-friendly attitudes were positively related to methodological scientific knowledge in this study. As for the much-debated relationship between knowledge and attitudes (Evans and Durant, 1995; Pardo and Calvo, 2002, 2004), on the basis of our specialized knowledge test that does not examine scientific ‘textbook’ facts but rather knowledge of research methodology, our data support a moderate positive correlation.
Interest in science, while unrelated to the perception of scientific uncertainty, was clearly related to actual engagement behaviours and behavioural intentions, such as going to town hall discussions or working as a citizen scientist. This behavioural measure, in turn, was unrelated to perceived uncertainty of scientific evidence and only marginally related to attitudes towards science, thus replicating the well-established attitude–behaviour gap (Glasman and Albarracín, 2006; Wicker, 1969).
As a shortcoming of the research presented here, it is not possible to directly compare the scores on the German and the English language version of the questionnaire, because we cannot ensure cross-cultural equivalence of the construct nor can we exclude the possibility that some items may have a different psychological meaning across the two cultures (Van de Vijver and Leung, 1997). Further studies are needed in order to investigate the precise nomological network of the construct and especially to clarify the psychological meaning participants infer on the two different subscales. Also, while the approach taken in this research was domain-general, perceptions of uncertainty may vary for different scientific or risk-related issues. And finally, the level of uncertainty measured by our instrument is confined to ‘known unknowns’ in Wynne’s (1992) typology, while it would be worthwhile to psychologically assess laypersons’ understandings of other kinds of uncertainty inherent to the scientific method.
On the basis of these cross-sectional findings, our overall conclusion is that just being aware of the tentative nature of scientific evidence does not deter laypersons from engaging with science. Additionally, in an experimental study using the questionnaire described here, the Retzbach et al. (2013) could demonstrate that media reports can affect perceptions of scientific evidence as certain or uncertain, but in line with the findings in this article, participants presented with scientific uncertainty did not show less interest in science. A recommendation for communicators of scientific uncertainty would be to focus on the idea that while science is about testing and revising ideas or hypotheses, evidence can have different degrees of certainty, and therefore, some pieces of scientific knowledge are more conclusive than others.
Footnotes
Funding
This work was supported by the by the German Research Foundation (DFG, Grant number MA 2244/4-1) in the Priority Programme 1409.
Author biographies
) funded by the German Research Foundation (DFG), she’s been involved in the Project ‘Communication of Evidence Concerning Emerging Technologies from the Life Sciences’ since 2009.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
