Impact of the Arizona NExSS Winter School on Astrobiology Knowledge and Attitudes

Abstract

Astrobiology is an inherently interdisciplinary area of study, demanding communication across multiple fields: astronomy, geochemistry, planetary science, and so on. Successful communication requires that researchers be aware of the basic findings, open questions, and tools and techniques of allied fields and possess an appreciation and respect for what these fields consider good science. To facilitate this communication between early-career researchers, the Arizona NExSS Winter School was hosted in February 2016, bringing together graduate students and postdoctoral researchers from backgrounds spanning the field of astrobiology. Students virtually attended a scientific Workshop Without Walls and participated in lectures, discussions, field trips, and hands-on activities, culminating in the writing and review of mock proposals by interdisciplinary teams. We assess the impact of the school on interdisciplinarity using a pre- and posttest survey of 24 students, informed by National Science Foundation impact categories (Friedman et al., 2008) within the Impact Analysis Method (IAM) described by Davis and Scalice (2015). We demonstrate that students gained knowledge, especially in fields outside their home discipline. Furthermore, an underlying disciplinary divide between geochemists and planetary scientists on the role of life in planetary evolution is observed and interpreted. These findings demonstrate that the Arizona NExSS Winter School had measurable impact on interdisciplinarity and that the IAM rubric has utility in measuring impact. We make recommendations for further research to understand the interdisciplinary gaps in astrobiology and how best to bridge them. Key Words: Interdisciplinarity—Attitudes—Knowledge—Scientific dialogue—Training. Astrobiology 18, 365–375.

1. Introduction

Astrobiology is the science of where and how to search for life in the Universe and is an inherently interdisciplinary field. It combines research from astronomy and astrophysics, geology, planetary science, biochemistry, genomics, and other disciplines. For astrobiology research to succeed, scientists across these fields must communicate their findings to each other and understand the importance of findings in other fields (Hays, 2015). Facilitating this communication is a major objective of the NASA Astrobiology Institute (NAI) and is important to NASA's Nexus for Exoplanet System Science (NExSS). Barriers to communication are deserving of serious study so that they may be turned into bridges for fruitful communication.

Even though astrophysics, geology, and biology are all natural sciences, they approach problems unique to their fields, with different tools and techniques, idiosyncratic jargon and different meanings of common terms, and even different philosophies about what constitutes good science. The central questions of astrobiology require interdisciplinary integration: a process of learning the research questions in fields other than your own, their jargon and essential theories, and a respect for the way that science is conducted in these sister fields (Boix Mansilla et al., 2016). These collaborations are slow to occur. For example, a bibliometric analysis of 1210 papers produced by scientists affiliated with the NAI between 2008 and 2012 revealed that 34.5% of papers published were described by two or more Web of Science journal categories; but the two largest category clusters, Astronomy & Astrophysics and Geochemistry & Geophysics, had no links in common (Taşkın and Aydinoglu, 2015).

Rectifying this situation by training early-career scientists in interdisciplinarity was a key objective of the Arizona/NExSS Winter School, held February 21–27, 2016. This school was co-hosted by the “Exoplanetary Ecosystems” NExSS team at Arizona State University (ASU; PI Steve Desch) and the “Earths in Other Solar Systems” NExSS team at the University of Arizona (UA; PI Daniel Apai). The organizing committee consisted of co-chairs Daniel Apai (UA) and Steve Desch (ASU), and Rory Barnes (Univ. Washington), Sara Imari Walker (ASU), Patrick Young (ASU), and Thomas Zega (UA). The Winter School was held in conjunction with the related “Upstairs Downstairs: Consequences of Internal Planet Evolution for the Habitability and Detectability of Life on Extrasolar Planets” Workshop, held February 17–19, 2016, at ASU (Anbar et al., 2016). The primary goals of the Winter School were training early-career scholars in basic knowledge across astrobiology disciplines, building bridges between different disciplinary approaches to astrobiology research questions, and seeding new scholarly collaborations.

The Scientific Organizing Committee for the Workshop consisted of chair Ariel Anbar (ASU), Rory Barnes (Univ. Washington), Steve Desch (ASU), Shawn Domagal-Goldman (NASA Goddard Space Flight Center), Cin-Ty Lee (Rice Univ.), Tim Lyons (Univ. California, Riverside), Victoria Meadows (Univ. Washington), and Christy Till (ASU). Both the School and the Workshop received financial support from NASA's Topical Workshops, Symposia and Conferences (TWSC) program. The format of the School was as follows: A total of 32 graduate students and postdoctoral researchers from 18 institutions participated, drawn mostly from the fields of astronomy and planetary science and geology, with some participation from biology. Upon selection to attend the school, students were divided into eight teams of four: Red1 through Red4 and Blue1 through Blue4. Students were informed that the capstone activity at the school would be for each team to write a 5-page science proposal in the same style as a NASA proposal.

Because the Workshop followed the Workshop Without Walls format (Arslan et al., 2011; Desch et al., 2014), students could virtually attend the workshop, which explored the themes of how further progress in understanding how to detect life on exoplanets will rely on research at the intersections of the traditionally divided disciplines of the geophysics of planet interiors, the geochemistry of planet surfaces, and the astronomy of characterizing planet atmospheres. Students were required to attend the workshop, either virtually or in person, and begin to communicate with their teammates about potential proposal topics. Each team was composed of students from diverse scientific backgrounds, and proposal topics had to be interdisciplinary. Students convened on Sunday, February 21, at the site of the school, the Biosphere 2 Facility in Oracle, Arizona. Students sat for lectures given by instructors Daniel Apai (UA), David Catling (Univ. Washington), Steve Desch (ASU), Shawn Domagal-Goldman (NASA Goddard Space Flight Center), Hiroshi Imanaka (NASA Ames Research Center), Tom Sharp (ASU), Christy Till (ASU), and Maggie Turnbull (Global Science Institute). These lectures covered the same scientific topics covered at the workshop, as well as career advice and lessons on writing proposals. Students learned astronomical software, toured the Biosphere, participated in a geological field trip to the San Carlos Indian Reservation, and learned about mineral identification. By the end of the week, students had written their 5-page science proposals: Red teams wrote proposals designed to be submitted to NASA's Emerging Worlds (EW) program, and Blue teams wrote proposals for NASA's Exoplanets Research (XRP) program. On Friday, February 26, the Red team students convened as a panel to review the XRP proposals, and Blue team students reviewed the EW proposals.

The formats of the workshop and the school were designed to drive interdisciplinarity, and as is true for any federally supported education and public outreach (EPO) or training activities, it is vital to demonstrate the impacts of the activities and assess whether the objectives were met. The nature and measurement of those impacts has not always been clearly articulated, but it is important to do so for accountability and effectiveness. Evaluation of EPO activities closes the learning loop, allowing primary investigators to treat them as integral parts of the research plan rather than as ancillary duties. Precise definitions of impact reduce the uncertainty and fuzziness associated with designing EPO and training activities (Davis and Scalice, 2014). Therefore, a critical Winter School activity was participation by students in a questionnaire (Appendix A) administered before and after the school, designed to assess the impact of the school on interdisciplinarity. As outlined by Davis and Scalice (2015), Friedman and coauthors' (2008) impact categories provide for measurement across multiple dimensions, denoted by the acronym BASIK: Behaviors, Attitudes, Skills, Interest, and Knowledge. Our questionnaire assessed whether students possessed working knowledge and skills of allied fields, their attitudes toward possible research directions in astrobiology, and if these measures of knowledge and attitude changed as a result of the NExSS Winter School.

This article is a report on our findings. We find that students acquired working knowledge about allied disciplines by attending the Winter School. We show that students entered the Winter School with a set of beliefs corresponding to scientific approaches in their own disciplines and less positive attitudes about the approaches taken by other disciplines. After the school, their attitudes converged, representing consensus without uniformity. We therefore conclude that the Winter School had measurable impact in breaking down the walls between disciplines in the participants' minds and fostering greater interdisciplinarity across astrobiology.

2. Disciplinary Divides across Astrobiology

Astrobiology by design demands that researchers trained in one discipline communicate with researchers in other disciplines. Participants in the Winter School self-identified their area of research, and because of the way students were recruited at meetings and through the NExSS network, these fell into a few main disciplines: astronomy/astrophysics, planetary science, and geochemistry. Most astronomy and astrophysics students were involved in observing exoplanets or modeling the formation of planets. Planetary scientists generally were concerned with the structure and evolution of bodies in our solar system such as Mars and icy moons, especially as pertaining to the search for life on them. Geochemists were mostly involved in understanding chemical cycles on Earth and in interpreting the paleorecord. These disciplines represent the breadth of knowledge that contributes to just one astrobiology topic area: categorizing exoplanet biosignatures in the context of planetary evolution.

The communication gaps between researchers in these fields are sometimes stark. Fundamental yet specialized knowledge may not be covered in undergraduate scientific requirements and not learned as students specialize in graduate school. Astronomers may not know that photosynthesis can occur that does not produce O₂ (Kulp et al., 2008). Geophysicists may not be aware that astronomers observe stellar abundance ratios to differ by factors of 2 from the Sun's (Suda et al., 2008). Biologists may not be aware that, even in the best-case scenario, astronomers will only observe exoplanets on a single pixel of a detector, and that disk-integrated atmospheres and surfaces are all that can be observed (Stone et al., 2015).

Gaps in knowledge are inevitable, especially among early-career researchers. As they focus on mastering a coherent body of knowledge, defining a doable research question for their dissertation, becoming part of a scientific community, and completing original research, graduate students are most strongly influenced by their home department and their dissertation committee. Interdisciplinarity is relatively risky, a process that requires additional work on the part of the graduate student to master additional knowledge and flexibility from their program in supporting a project that may not fit into a defined disciplinary bin (Morse et al., 2007). For astrobiology, which lacks any graduate programs of its own, the process of how new researchers come to see themselves as astrobiologists is of paramount importance (Boden et al., 2011). We premise that the making of a new astrobiologist happens outside of formal graduate education, that is, by taking a class on astrobiology as one part of a program, by attending field schools like the NExSS Winter School, or by attending the Astrobiology Science Conference and reading Astrobiology. The basic questions of astrobiology are of deep and abiding interest: Are we alone in the Universe? Where did life come from? How do planets become habitable?

Major projects in astrobiology are collaborations that demand more than individual excellence as a researcher in a single field. Experimental work in the philosophy of science shows that scientific epistemologies, standards for credible evidence, reasonable doubt, and good data—essentially how a scientist knows what they know—vary between researchers (Eigenbrode et al., 2007). Working across disciplines requires an understanding of what the other disciplines regard as good research practice, knowledge of basic facts and approaches in those disciplines, and respect for what fields can and cannot easily answer (Miller et al., 2008). These interdisciplinary skills and attitudes are something that can be deliberately fostered as part of a scholar's professional development (Nash, 2008).

Based on our experiences within the NAI and NExSS, we hypothesize that such an epistemological divide exists between astronomy/astrophysics and fields such as biogeochemistry. Astronomers and astrophysicists tend to be trained in physics, where the emphasis is on reductionist approaches: construction of simple, predictive models from first principles. In other words: assume a spherical planet, write the relevant differential equation, and solve. This approach works very well for understanding the orbits of planets or the evolution of stars, so within the astrobiological field of exoplanets and biosignatures, it is natural for an astronomer to ask questions like the following: “Since this star has twice as much phosphorus as the Sun, how does the planetary-scale O₂ production by microbes scale with phosphorus abundance?”

To their consternation, astronomers will learn that biological systems are not amenable to a reductionist approach. Individual microbes respond to elemental availability in complicated ways, switching genes on and off. Ecosystems cycle elements rapidly between different organisms, making it difficult even to define what “twice the phosphorus” means in an ecosystem. Production rates of O₂ are due to net imbalances of respiration and photosynthesis attributable to subtle effects like carbon burial. Controlled experiments are not possible at the scale of planets or even ecosystems. In the end, there are so many variables to correct for that reductionist models are not feasible, and biogeochemists tend to ask completely different scientific questions.

These differences in what constitutes an answerable research question are not merely hypothetical examples. At the workshop, an astrophysicist proposed that it may be possible to predict the exact chemical composition of a planet, precisely how many Fe, Mg, Si, and O atoms it has, from stellar abundances. It seems natural to then ask a planetary scientist, “What is the state of carbon that is outgassed? Is it CO₂, or is it CH₄?” However, the answer depends on knowing the redox state of the planet's upper mantle, a problem geochemists generally consider too complicated to make predictions about.

These epistemological divides are more pernicious than simple gaps in knowledge because they can be perceived as a lack of scientific competence, and provoke arguments rather than productive discussion. We have observed astrophysicists (not at the Winter School!) dismiss geochemists as not understanding simple questions or, worse, being unable to “do the math.” Physicists (and by extension astronomers and astrophysicists) have a reputation (deserved or not) for underappreciating the complexity of other fields and for valuing simplification at the expense of realism (Gibson, 2003; see also https://xkcd.com/793). Conversely, researchers in other fields may underappreciate the value of reductionist approaches to robustly describe causal factors and make definitive predictions. Planetary models must do more than describe what is happening on Earth; they must be predictive and able to accept the non-terrestrial inputs observed by astronomers to predict and characterize biosignatures on exoplanets.

We believe that individual researchers, and astrobiology as a field, can benefit from more extensive training across relevant disciplines. As students move from a receptive role as consumers of scientific knowledge to active producers of scientific knowledge, it is important that they broaden their scientific horizons and become more respectful and inclusive toward the approaches taken by allied disciplines. Our purpose for conducting the Arizona NExSS Winter School was to create the condition to bridge these divides and show conclusively that we had done so.

The facilitation and study of interdisciplinary and transdisciplinary science are topics of longstanding concern, primarily in the biomedical and ecological sciences. Best practices in the organization and leadership of research institutions have been the subject of a recent National Academy of Sciences report (Cooke and Hilton, 2015). The debate over reductionist and holistic approaches parallels previous conversations in ecology (Holling, 1998), a field that has seen a great deal of dialogue over terminology, concepts, and the application of knowledge (Starzomski et al., 2004). These essential tensions are an innate part of interdisciplinary research, the source of its intellectual creativity, and ultimately something to be managed rather than erased (Turner et al., 2015).

With some exceptions (Hackett and Rhoten, 2009; Parker and Hackett, 2012), the potential of medium-duration experiences like the Winter School to catalyze interdisciplinary scholarship is relatively unexamined. Field schools, with their ability to create interdisciplinary networks and initiate interdisciplinary conversations, are especially important for astrobiology. Because astrobiology is a term that encompasses multiple disciplines, field schools must play a dual role of teaching basic knowledge and skills across disciplines and promoting an interdisciplinary mind-set. Assessing the impact of these schools on interdisciplinarity is essential for developing best practices in these schools.

3. Methods

The Winter School's impact on interdisciplinarity can be measured against several dimensions, as outlined by the BASIK rubric for measuring the impact and effectiveness of EPO activities and ensuring that the objectives of the EPO activities are met. As participants were graduate students and postdoctoral researchers in fields within astrobiology, their interest was assured. The short duration of the Winter School precluded a study of whether scientific behaviors underwent a change or whether significant skills were acquired. Our questionnaire (Appendix A) assessed whether students acquired new knowledge and whether their attitudes changed. This questionnaire was designed by two of the authors (Desch and Apai) in consultation with experts in assessing science EPO activities (Davis and Scalice) to reflect the unique issues facing astrobiology and the NExSS Winter School. As such, it was not validated prior to being administered for this study.

All students were administered identical surveys one week before the Winter School started, which they returned upon arrival. They were also administered the same survey after the Winter School. Students returned these after various intervals, from days to months after the Winter School end. Ultimately, 24 students completed both surveys to a sufficient degree to be included in this study. Of these 24, 14 identified as astronomers or astrophysicists, 5 as geochemists or geologists, and 5 as planetary scientists. For expediency, we will refer to these as “Astro,” “Geo,” and “Planetary.”

The surveys consisted of five sections, the first three concerning academic background, scientific skills, and previous exposure to interdisciplinarity. Section IV consisted of 10 questions asking students to agree or disagree with approaches to open questions in astrobiology on a 5-point Likert scale. Section V consisted of a 52-question quiz testing core knowledge in Astronomy/Exoplanets, Geology/Geochemistry, and Planetary Science/Geophysics. Questions were graded out between zero and two points, with partial credit given for answers that captured the major features of the question while missing some of the details. Grading was carried out according to a rubric developed by the survey team. Respondents were allowed not to answer if they felt they did not know, gaining zero points for that question.

This type of one-group pretest/posttest evaluation is a study design commonly used for assessing educational and training activities. This study design balances the educational goals of the activity, reaching as many people as possible, with achieving valid assessment that the activity increased relevant knowledge (Bernard, 2011). Particularly for EPO activities, measures based solely on the posttest lack validity. Surveys of how well an activity was enjoyed without a fair comparison of alternatives offer little explanatory information. It is impossible to register whether attitudes were shifted if one does not know what attitudes were before the EPO activity. Similarly, self-reporting of knowledge is generally unreliable; a person may claim to understand a great deal while in fact having significant gaps or errors in knowledge (Kruger and Dunning, 1999).

We developed and tested four hypotheses, as follows:

H1: Self-report academic identity would correlate with greater starting knowledge in that field, as measured by the pretest quiz.

H2: Attending the NExSS Winter School would increase knowledge, especially in other fields, as measured by differences between the pre- and posttests.

H3: Self-reported academic discipline would correlate with preferred approaches in astrobiology, as measured by the pretest.

H4: Attending the NExSS Winter School would cause changes in attitudes about scientific approaches, and in particular would cause preferences in approaches to astrobiology to converge across disciplines, as measured by differences in the pre- and posttests.

4. Results

Hypothesis 1, that NExSS Winter School participants would be more knowledgeable in their self-identified discipline, was easily verified. Pre- and posttest scores are displayed in Fig. 1 and Table 1, which show the scores by “Planetary,” “Geo,” and “Astro” students on the astronomy, geology, and planetary science portions in Section V of the questionnaire, both before and after the Winter School. “Astro” students on average scored 14.4 points (equivalent to getting 7 out of 16 questions correct) on the astronomy portion of the quiz, while correctly answering just 2–3 questions in geology or planetary science. “Geo” and “Planetary” students show similar levels of expertise in their fields. “Planetary” and “Geo” students were familiar with each other's disciplines, and Winter School attendees overall demonstrated the highest level of knowledge with astronomy. The lower scores of “Astro” students overall are associated with a greater number of blank answers, and we speculate they may have felt less comfortable guessing outside of their area of training than respondents from the other disciplines. Astro students on average answered just 2 questions on the Geology/Geochemistry section of the quiz. An ANOVA test on the difference in knowledge between groups is statistically significant with a p-value of 0.003.

FIG. 1.

Mean scores by “Astro,” “Geo,” and “Planetary” students on the astronomy, geology, and planetary science portions of the Knowledge section of the questionnaire, before and after the Winter School, with net improvement.

Table 1.

Table of Mean Scores and “No Answers” on Section V: Knowledge by Discipline

	Astronomy/ Astrophysics	Geology/ Geochemistry	Planetary Science	Astronomy/ Astrophysics	Geology/ Geochemistry	Planetary Science
	(1.1) Pre Mean Score			(1.2) Pre Mean “No Answers”
Astro	14.62	3.00	5.85	2.62	14.85	9.77
Geo	8.00	18.67	14.00	5.00	1.83	2.50
Planetary	16.40	14.40	17.60	3.20	5.40	2.20
	(1.3) Post Mean Score			(1.4) Post Mean “No Answers”
Astro	16.62	10.31	10.62	2.08	8.85	6.69
Geo	12.83	19.33	18.33	1.67	1.67	1.17
Planetary	18.60	20.40	21.20	1.00	2.20	0.60
	(1.5) Change in Mean Score			(1.6) Improvement Mean “No Answers”
Astro	2.00	7.31	4.77	0.54	6.00	3.08
Geo	4.83	0.67	4.33	3.33	0.17	1.33
Planetary	2.20	6.00	3.60	2.20	3.20	1.60

Hypothesis 2, that quiz scores would increase, especially in other disciplines, was also easily demonstrated. As shown in Fig. 1C, students in all fields demonstrated increases in knowledge across all areas. “Astro” students improved least in astronomy and the most in geology. “Geo” students improved marginally in geology but significantly in other areas. Planetary students improved least in astronomy but the most in geology. This is congruent with the assumption that graduate students have already mastered a body of knowledge corresponding to their own discipline. Paired t-tests on the difference in pre and post scores demonstrated statistically significant improvements in knowledge with p-values less than 0.001.

Hypothesis 3, that different disciplines would have different attitudes about scientific approaches to understanding exoplanets, was tested by using responses to the 10 questions of Section IV of the questionnaire. These questions probed attitudes about whether planetary outcomes could be predicted from initial conditions, the feasibility of determining exoplanet atmospheric nitrogen levels via a variety of models, and a theorized reductionist versus holistic divide in approaches.

Figure 2 plots the Likert scale responses by discipline to the Section IV: Attitudes questions. Students were asked how much they agree with a variety of approaches using a 5-point scale from strongly disagree to strongly agree. Some indications of difference by discipline can be seen: for example, “Planetaries” strongly agree with the statement that planetary systems are physical processes that can be robustly predicted. As seen in Fig. 2, drawing conclusions from this data is non-obvious.

FIG. 2.

Plot of Likert scales on Section IV “Attitudes” questions by discipline before and after the NExSS Winter School.

We analyzed the responses using a principal component analysis (PCA), which projects the 10-dimensional opinion space spanned by the questions onto orthogonal vectors that represent the greatest degree of difference in responses into a two-dimensional plane, allowing easy visualization and analysis of underlying factors (Husson et al., 2011). In Fig. 3, we plot each student's pretest score along the two most prominent orthogonal vectors, labeled Dimension 1 (x axis) and Dimension 2 (y axis). Dimension 1 explains 25.1% of the variance in answers, a value typical of data sets of this size. It corresponds to the responses to questions 1, 6, 7, and 8, which correlate with each other. Dimension 2 explains 16.0% of the variation and corresponds to responses to questions 2 and 4. It is seen that the different disciplines appear in different regions of the two-dimensional diagram, meaning that responses vary with discipline.

FIG. 3.

PCA of pre–Winter School attitudes. Students are color-coded by discipline. The spatial distribution of responses by each discipline is depicted using ellipses whose areas denote the 2σ variations. These ellipses also indicate 95% confidence intervals for predicted responses by discipline.

Dimension 1 (x axis) corresponds to questions 1, 6, 7, and 8, which all relate to the physical modeling from initial conditions in geophysical and geochemical systems. All three disciplines have similar mean values along this dimension, with “Planetary” students showing greater variability. We interpret Dimension 1 as reflecting a general optimism or pessimism about the feasibility of modeling complex systems related to exoplanets or Earth history. Each discipline has roughly equal proportions of members who are optimistic or pessimistic.

Dimension 2 (y axis) corresponds to questions 2 and 4, which describe life as such a complicating factor on Earth that extrapolating between lifeless worlds and Earth is essentially fruitless. In contrast to Dimension 1, the different disciplines are seen to array differently along Dimension 2. “Planetary” students disagreed with questions 2 and 4, indicating that they tended to believe that Earth's atmosphere could be understood even in the absence of life. “Geo” students tended to agree with questions 2 and 4, implying that they believe that life is so integral to atmospheric processes that it is impossible to know a lifeless Earth. “Astro” students were divided on the question, perhaps reflecting agnosticism on the questions. We interpret these results as demonstrating an epistemology unique to geology training, as discussed in the conclusion.

Hypothesis 3 is therefore verified, although weakly. Few trends with discipline were seen, but for a specific subset of questions, about modeling of an alternative Earth on which life did not arise, “Geo” students were less likely than “Planetary” students to agree such an approach was possible.

To test Hypothesis 4, we carried out the same analysis using the posttest responses. The results are displayed in Fig. 4. The hypothesis that changes in attitudes occurred and that they converged is verified.

FIG. 4.

We observed that some responses to the Attitudes questions varied for 15 of the 24 respondents, but in most cases the changes were slight. ANOVA analyses do not provide enough statistical significance to reject the null hypotheses, namely, that there is no difference in attitudes between NExSS Winter School students by discipline, or that the Winter School made no changes in attitudes.

However, statistical significance is not the same as practical significance. The small sample size, the minor shifts in measured attitudes, the preponderance of astronomers who occupied a middle position along Dimension 2, and the way the questions were phrased all complicate measuring and confirming these shifts in attitude. Though the results do not exceed a p-value of 0.05, which would permit reasonable certainty that these effects are more than random noise, we regard this convergence as indicative of the possibility that attitudes can change. Further work is required to confirm or disprove Hypothesis 4.

5. Discussion

The Arizona NExSS Winter School was designed to promote interdisciplinarity among researchers in various fields across astrobiology. This meant more than teaching basic knowledge and jargon and definitions about geology to astronomers and vice versa. The organizers aimed to promote understanding of the scientific approaches across fields and foster an appreciation for the abilities and limitations of other disciplines. Our starting theory was that astronomers and astrophysicists favored a reductionist approach of constructing mathematical models from first principles, while the fields of geochemistry and planetary science, which rely less heavily on physics and tend to work with more complicated multidimensional systems, may regard such models as full of unrealistic simplifications. The scientific project of characterizing potential exoplanet signatures for habitability requires bridging these two modes of thinking.

We administered a questionnaire before and after the Winter School and received pretest and posttest responses from 24 participants. The questionnaire was designed by using the Impact Analysis Method (IAM; Davis and Scalice, 2014) to assess changes in knowledge and attitudes. Students self-identified into three cohorts: students in astronomy and astrophysics (14 “Astro” students), those in geology and geochemistry (5 “Geo” students), and those in planetary sciences (5 “Planetary” students). The questionnaire asked questions about basic knowledge in astronomy, geology, and planetary science. From students' responses, we confirmed our first two hypotheses (H1 and H2). Students started off with more knowledge in their home disciplines. After attending the Winter School, students gained knowledge in all fields, and the gains were generally slight in their own areas and greater when learning about other areas. We confirm that students learned basic knowledge across disciplines and therefore that the Winter School achieved its primary purpose.

We also assessed our theory that students' attitudes about scientific approaches correlated with their home disciplines (H3). Through a PCA, we found clusters of responses that explained the preponderance of the variance. Dimension 1 (questions 1, 6, 7, and 8 of Section IV of the survey) explains 25% of the variance and corresponds to general optimism or pessimism about modeling planetary systems from initial conditions. Responses from students in all three cohorts showed similar means across this dimension, without any specificity by discipline.

Differences by discipline appeared in the responses along Dimension 2 (questions 2 and 4 of Section IV of the survey), which explains 18% of the variance. These two questions asked about whether the existence of life on Earth so complicates the Earth system that predictions of atmospheric composition or geochemical cycles on an Earth without life are impossible. It is sensible that responses to these two questions should correlate with each other. The responses of “Astro” students tend to be distributed around the mean. “Geo” students are more likely to agree with the statement, while “Planetary” students are less likely to agree. A clear difference of opinion exists between “Geo” and “Planetary” students, despite the tendency for students in both fields to come from a geological background. This suggests an epistemological difference about what constitutes good science.

We interpret the difference as follows: Geologists and geochemists are trained to understand Earth systems as they operate today and to explain the sequence of events that brought Earth to its present state. The data set geologists work with is rich, nuanced, and literally all around us. Life is such an integral part of this history, it is impossible to separate geochemical cycles from the biology. Even such notionally abiotic geochemical cycles as the carbonate-silicate cycle, wherein CO₂ dissolves in the oceans and is converted to CaCO₃, are overwhelmingly biologically mediated (Ridgwell and Zeebe, 2005). It may be the case that geologists and geochemists are simply not used to thinking of Earth without life, and given the deep relationship between their field and life, it is difficult to imagine a comparison between Earth and another planet without also thinking of reasons why that comparison would be invalid. In contrast, planetary scientists studying other planets work with much sparser data sets. The existence of life is neither assured nor ruled out, and indeed, planetary scientists in astrobiology routinely devise tests to detect life by considering planets with and without life. It is a natural exercise to imagine Earth as a planet in the same way. We believe that a week-long workshop discussing precisely these topics may lead geologists and geochemists to view Earth as a planet that is the highest quality data set in a larger collection of data about planets. Conversely, a planetary scientist might take pause as they realize how inextricable life has been to the evolution of just one planet.

We did not find conclusive evidence to support our initial theory, that astronomers and astrophysicists are more amenable to a reductionist approach than researchers in other fields. The divergence of opinion among scientists within any discipline was greater than the differences of opinions between disciplines. It is possible that the perceived degree of epistemological disagreement between scientists may be generally overstated. The Michigan State University Toolbox, a philosophy of science instrument specifically designed to test and bridge these differences, found no significant clusters of attitudes in a much broader survey across the natural and social sciences (Robinson et al., 2016). The consistency of results within Dimension 1 on Figs. 3 and 4 suggests that reductionist attitudes are a distinct component of individual mind-sets but that these individuals are distributed across all surveyed disciplines.

The data from our questionnaire suggest some differences in attitudes about scientific approaches, but only weakly, and not in the way we expected. Larger and better studies are needed to test whether epistemological differences between the disciplines exist, whether they hinder interdisciplinary communication, and whether activities like the Arizona NExSS Winter School are effective in bridging the gaps in attitudes between scientific disciplines. We recommend that future interdisciplinary schools design tests to explore these epistemological differences. We suggest larger sample sizes, reworking the survey instrument, and including participants from the life sciences. Education and public outreach efforts become more relevant when they have clear standards for assessment and produce information that can be used to improve future efforts. Astrobiology is so deeply defined by interdisciplinary questions and approaches that explicitly discussing the underlying epistemologies and conceptions of good science held by the researchers involved in a project may be a critical first step. Questionnaires such as the one described in this paper can be used to track interdisciplinary attitudes and knowledge, and changes in those dimensions due to interventions such as the NExSS Winter School.

Appendix A: Winter School Questionnaire

I. Background

1. What's your name? What stage of your career are you (year in grad school or years since Ph.D.)?

2. What fields are you trained in? [for Qs 2–4, you may list more than one]

3. What fields do you regularly work in?

4. When you finish your formal training, with what discipline will you primarily identify (can be an interdisciplinary designation such as biogeochemist)?

5. What fields was your adviser trained in?

6. Would you say your current institution promotes interdisciplinarity? If so, how? If not, are there barriers you can identify?

7. At your current institution, which of the following regularly expose you to researchers in other fields (you can choose more than one)? a) colloquia; b) informal seminars (brown bag lunches/coffee hours); c) diverse faculty in department; d) required to do project outside your field; e) other (please describe); f) I'm not really exposed to researchers from other fields.

II. Skills

1. Please briefly list skills you have acquired to carry out your science.

2. Which skills were acquired as part of formal training (e.g., classes)?

3. Which skills were acquired as part of your directed research (e.g., thesis project)?

4. Which skills did you acquire through individual, non-directed research?

5. Do you have experience writing computer code or routines? Which languages?

6. Please list any software packages you use for research.

7. Do you have experience using data from telescopes/remote sensing? Describe.

8. Do you have experience gathering data through field work? Describe.

9. Do you have experience conducting laboratory experiments/analyses? Please describe.

10. Do you have experience writing proposals? How many? What programs?

III. Interest/Behaviors

1. On a scale of 1 (not so interested) to 5 (very interested), how interested would you say are you about the findings of other scientific fields?

2. How often do you attend colloquia/seminars by speakers not in your immediate area of study?

3. When attending a talk by someone in your own field, rate on a scale of 1 (very often) to 5 (almost never) how often do you encounter jargon or concepts you don't understand?

4. When attending a talk by someone in a different field, rate on a scale of 1 (very often) to 5 (almost never) how often do you encounter jargon or concepts you don't understand?

5. How often do you read blogs or popular articles about results in fields outside your immediate area of study?

6. List as many scientific journals as you can that you remember reading articles from.

7. If you are in a conversation with a peer (grad student or postdoc) in a different field, and they use jargon you don't understand, how likely are you to interrupt and ask them to explain, on a scale from 1 (not very likely) to 5 (very likely)?

8. If you are in a conversation with a peer (grad student or postdoc) in your own field, and they use jargon you don't understand, how likely are you to interrupt and ask them to explain, on a scale from 1 (not very likely) to 5 (very likely)?

IV. Attitudes

In the questions below, use a scale of 1 (strongly disagree) to 5 (strongly agree).

1. An exoplanet's atmosphere should be predictable if the starting conditions (e.g., chemical composition, stellar radiation history) could be determined. Rate 1–5.

2. Earth has so many ecosystems, each with so many variables, it is pointless to try to predict what composition Earth's atmosphere ought to have. Rate 1–5.

3. Planetary atmospheres, and even life, are physical systems following physical rules, leading to universal, robust outcomes. Rate 1–5.

4. Life has so fundamentally changed Earth's atmosphere that it is impossible to know what Earth's atmosphere would be like without life. Rate 1–5.

5. Say we wanted to calculate the biological photosynthetic production of oxygen on an exoplanet with oceans but no continents. A good approach would be to extrapolate from field studies of terrestrial ecosystems. Rate 1–5 and explain.

6. Say we wanted to calculate the biological photosynthetic production of oxygen on an exoplanet with oceans but no continents. A good approach would be to conduct laboratory experiments on cultured samples. Rate 1–5 and explain.

7. Say we wanted to calculate the biological photosynthetic production of oxygen on an exoplanet with oceans but no continents. A good approach would be to construct a theoretical model using input energy as the driving mechanisms. Rate 1–5, explain.

8. Say we wanted to anticipate what atmospheric nitrogen levels should be on a 2 M_E Earth-like exoplanet. A good approach would be to create a chemical evolution model for the planet, starting with initial conditions 4.5 billion years ago, evolving it to the present day, and calculating the atmospheric composition. Rate 1–5, explain.

9. Say we wanted to anticipate what atmospheric nitrogen levels should be on a 2 M_E Earth-like exoplanet. A good approach would be to probe Earth's geologic record, judging N₂ partial pressure from mineralogy or petrology. Rate 1–5, explain.

10. Say we wanted to anticipate what atmospheric nitrogen levels should be on a 2 M_E Earth-like exoplanet. A good approach would be to use data from Earth, Venus or Mars, and extrapolate to the larger planet. Rate 1–5 and explain.

V. Knowledge

The following is only an assessment of your current inventory of knowledge and field-specific jargon and concepts. Please answer questions off the top of your head without books or the Internet. If you don't know the answer to these technical questions, it's OK to skip or say “outside my field.”

Astronomy/Exoplanets

1. What are the axes on the HR diagram?

2. What is the Sun's spectral type?

3. What's an approximate formula for how a star's luminosity varies with its mass?

4. What defines whether a planet is in its star's habitable zone?

5. The habitable zone around an M star is about how many AU from the star?

5b. What can we learn about an exoplanet based on RV measurements?

6. Why is the RV method less successful on M stars than on FGK stars?

7. Why is the RV method not applicable to most stars in the Kepler sample?

8. What can we learn about an exoplanet based on transit measurements?

9. Why is the transit method simply not applicable to most exoplanet-hosting stars?

10. For those systems where the transit method can be used, what is the most uncertain input parameter?

11. How long did the solar nebula persist/do protoplanetary disks persist?

12. How long did it take the Earth to form from accretion of planetary embryos?

13. What is the typical mass of a planetary embryo?

14. What is the most common planetary mass in the Kepler sample?

15. What wavelengths of light will JWST observe?

16. Name four molecules potentially observable by JWST in a transiting planet's atmosphere.

Geology/Geochemistry

1. What type of volcano exists at plate boundaries?

2. Which type of lava is likely to erupt in arc volcanoes (felsic or mafic)?

3. What type of volcano are many ocean islands?

4. What type of lava is likely to erupt in these volcanoes (felsic or mafic)?

5. What does MORB stand for?

6. Why are MORBs thought to probe the Earth's mantle?

7. In the Urey cycle, name a major cation that oceanic carbonic acid reacts with.

8. In the Urey cycle, how does carbon in subducted carbonates return to the surface?

9. What does “IW” in “IW buffer” stand for?

10. Relative to this buffer, how oxidizing is the Earth's crust, and the Earth's mantle?

11. Name one other commonly used redox buffer.

12. What atmospheric photochemical effect mostly likely increased the oxidation state of planetary surfaces like Venus's and Mars'?

13. About how long ago did Earth's atmosphere increase in oxygen?

14. What does “BIF” stand for? How did they form?

15. What are the oldest radiometrically dated minerals on Earth?

16. How old are the oldest minerals on Earth? What does their presence say about the extent of land in the Hadean?

17. When did the Archean begin and end?

18. There are few komatiites younger than Archean in age. Why is that?

Planetary Science/Geophysics

1. How old is the Solar System?

2. The age of the Solar System has been most accurately dated using what radiometric dating system?

3. What samples have been dated as the oldest objects formed in the Solar System?

4. What are the five most abundant elements in the Earth's crust, in order by mass?

5. What are the five most abundant elements in Earth's mantle, in order by mass?

6. What is the percentage of the mass of the Earth's core that is Fe + Ni?

7. What are the most likely other constituents of the Earth's core?

8. What is the most abundant mineral in the Earth's mantle?

9. What are the different ways silicate tetrahedra are arranged in olivines, pyroxenes and micas?

10. Hf-W dating relies on W being sequestered in a planet's core while Hf remains in its mantle. In the Goldschmidt classification scheme, what is the word to describe the chemical affinity of Hf? And of W?

11. Which radioactive isotopes are most responsible for the heat flux in the Earth?

12. In order for convection to happen, a certain dimensionless number must exceed a critical value. What do we call that number?

13. What input to that dimensionless number is the least well known, because it is so sensitive to conditions like temperature, grain size, etc.?

14. What are the 5 most abundant species in Earth's atmosphere (in order by molar abundance)?

15. The atmospheric greenhouse effect on Earth raises the mean temperature by about how much?

16. What is the greenhouse gas most responsible for the increase in Earth's surface temperature?

17. What defines where a planet's atmosphere's exobase lies?

18. About how far above the Earth's surface is its exobase?

Footnotes

Acknowledgments

We gratefully acknowledge support from the NASA Nexus for Exoplanet System Science (NExSS) and the NASA Topical Workshops, Symposia and Conferences (TWSC) programs.

Abbreviations Used

References

Anbar

A.D.

, Barnes

, Domagal-Goldman

, Lee

C.-T.

, Lyons

, Meadows

, and Till

(2016, February 17–19) Upstairs Downstairs: Consequences of Internal Planet Evolution for the Habitability and Detectability of Life on Extrasolar Planets, NASA Astrobiology Institute. Available online at https://nai.nasa.gov/calendar/www-upstairs-downstairs

Arslan

B.K.

, Boyd

E.S.

, Dolci

W.W.

, Dodson

K.E.

, Boldt

M.S.

, and Pilcher

C.B.

(2011) Workshops Without Walls: broadening access to science around the world. PLoS Biol, 9, doi:10.1371/journal.pbio.1001118.

Bernard

H.R.

(2011) Research Methods in Anthropology: Qualitative and Quantitative Approaches, 5 ^th ed., AltaMira Press, Lanham, MD.

Boden

, Borrego

, and Newswander

L.K.

(2011) Student socialization in interdisciplinary doctoral education. Higher Education, 62:741–755.

Boix Mansilla

, Lamont

, and Sato

(2016) Shared cognitive-emotional-interactional platforms: markers and conditions for successful interdisciplinary collaborations. Sci Technol Human Values, 41:571–612.

Cooke

N.J.

and Hilton

M.L.

, editors. (2015) Enhancing the Effectiveness of Team Science, The National Academies Press, Washington, DC.

Davis

H.B.

and Scalice

(2014) Defining and measuring impact for ourselves. In Ensuring STEM Literacy: A National Conference on STEM Education and Public Outreach, Astronomical Society of the Pacific Conference Series Vol. 483, edited by Manning

J.G.

, Jensen

J.B.

, Hemenway

M.K.

, and Gibbs

M.G.

, Astronomical Society of the Pacific, San Francisco, CA.

Davis

H.B.

and Scalice

(2015) Defining, enhancing, and documenting impact of STEM education and public outreach projects through a quantitative collaborative impact analysis evaluation method. In Celebrating Science: Putting Education Best Practices to Work. Proceedings of a Conference held 2–6 August 2014 at Burlingame, California USA; edited by Schultz

, Buxner

, Shore

, and Barnes

. ASP Conference Series Vol. 500 (San Francisco: Astronomical Society of the Pacific), p. 211.

Desch

S.J.

, Young

P.A.

, Anbar

A.D.

, Hinkel

, Pagano

, Truitt

, and Turnbull

(2014) Report on a NASA Astrobiology Institute–funded Workshop Without Walls: Stellar Stoichiometry. Astrobiology, 14:271–276.

10.

Eigenbrode

S.D.

, O'Rourke

, Wulfhorst

J.D.

, Althoff

D.M.

, Goldberg

C.S.

, Merrill

, Morse

, Nielsen-Pincus

, Stephens

, Winowiecki

, and Bosque-Pérez

N.A.

(2007) Employing philosophical dialogue in collaborative science. BioScience, 57:55–64.

11.

Friedman

A.J.

, Allen

, Campbell

P.B.

, Dierking

L.D.

, Flagg

B.N.

, Garibay

, Korn

, Silverstein

, and Ucko

D.A.

(2008) Framework for Evaluating Impacts of Informal Science Education Projects, National Science Foundation, Alexandria, VA. Available online at http://insci.org/resources/Eval_Framework.pdf

12.

Gibson

J.M.

(2003) Arrogance: a dangerous weapon of the physics trade. Phys Today, 56, doi:10.1063/1.1564350.

13.

Hackett

E.J.

and Rhoten

D.R.

(2009) The Snowbird Charrette: integrative interdisciplinary collaboration in environmental research design. Minerva, 47:407–440.

14.

Hays

L.E.

, editor-in-chief. (2015) NASA Astrobiology Strategy 2015. Available online at http://astrobiology.nasa.gov/uploads/filer_public/01/28/01283266-e401-4dcb-8e05-3918b21edb79/nasa_astrobiology_strategy_2015_151008.pdf

15.

Holling

C.S.

(1998) Two cultures of ecology. Conservation Ecology, 2, article 4.

16.

Husson

, Lê

, and Pagès

(2011). Exploratory Multivariate Analysis by Example Using R, CRC Press, Boca Raton, FL.

17.

Kruger

and Dunning

(1999) Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc Psychol, 77:1121–1134.

18.

Kulp

T.R.

, Hoeft

S.E.

, Asao

, Madigan

M.T.

, Hollibaugh

J.T.

, Fisher

J.C.

, Stolz

J.F.

, Culbertson

C.W.

, Miller

L.G.

, and Oremland

R.S.

(2008) Arsenic(III) fuels anoxygenic photosynthesis in hot spring biofilms from Mono Lake, California. Science, 321:967–970.

19.

Miller

T.R.

, Baird

T.D.

, Littlefield

C.M.

, Kofinas

, Chapin

F.S.

, and Redman

C.L.

(2008) Epistemological pluralism: reorganizing interdisciplinary research. Ecol Soc, 13, article 46.

20.

Morse

W.C.

, Nielsen-Pincus

, Force

J.E.

, and Wulfhorst

J.D.

(2007) Bridges and barriers to developing and conducting interdisciplinary graduate-student team research. Ecol Soc, 12, article 8.

21.

Nash

J.M.

(2008) Transdisciplinary training. Am J Prev Med, 35:S133–S140.

22.

Parker

J.N.

and Hackett

E.J.

(2012) Hot spots and hot moments in scientific collaborations and social movements. American Sociological Review, 77:21–44.

23.

Ridgwell

and Zeebe

(2005) The role of the global carbonate cycle in the regulation and evolution of the Earth system. Earth Planet Sci Lett, 234:299–315.

24.

Robinson

, Vasko

S.E.

, Gonnerman

, Christen

, O'Rourke

, and Steel

(2016). Human values and the value of humanities in interdisciplinary research. Cogent Arts Humanit, 3, doi:10.1080/23311983.2015.1123080.

25.

Starzomski

B.M.

, Cardinale

B.J.

, Dunne

J.A.

, Hillery

M.J.

, Holt

C.A.

, Krawchuk

M.A.

, Lage

, McMahon

, and Melnychuk

M.C.

(2004) Contemporary visions of progress in ecology and thoughts for the future. Ecol Soc, 9, article 14.

26.

Stone

, Alkalai

, and Friedman

(2015) Science and Enabling Technologies for the Exploration of the Intersellar Medium, Keck Institute for Space Studies, Pasadena, CA. Available online at http://www.deepspace.ucsb.edu/wp-content/uploads/2015/04/Final-KISS-ISM-Report.pdf

27.

Suda

, Katsuta

, Yamada

, Suwa

, Ishizuka

, Komiya

, Sorai

, Aikawa

, and Fujimoto

M.Y.

(2008) Stellar Abundances for the Galactic Archeology (SAGA) Database—compilation of the characteristics of known extremely metal-poor stars. Publ Astron Soc Jpn Nihon Tenmon Gakkai, 60:1159–1171.

28.

Taşkın

and Aydinoglu

A.U.

(2015) Collaborative interdisciplinary astrobiology research: a bibliometric study of the NASA Astrobiology Institute. Scientometrics, 103:1003–1022.

29.

Turner

V.K.

, Benessaiah

, Warren

, and Iwaniec

(2015) Essential tensions in interdisciplinary scholarship: navigating challenges in affect, epistemologies, and structure in environment–society research centers. Higher Education, 70:649–665.