Abstract
We report the results of a large-scale study of the state of science content knowledge of volunteers in Galaxy Zoo (www.galaxyzoo.org), an online citizen science project in which public volunteers classify galaxies in an effort to benefit cutting-edge astronomy research. We were interested in whether participating in Galaxy Zoo leads to any increase in participants’ astrophysical content knowledge. To assess volunteer content knowledge, we examined the responses of 1476 Galaxy Zoo volunteers to 32 conceptually challenging multiple-choice questions. We looked for any relationships between participants’ assessment scores and the number of galaxies classified upon answering the first assessment question, the number of galaxies classified between their first response and their final response to the assessment, and the length of time since they first created their Galaxy Zoo account. All relationships were of small effect size. These results suggest that participation in the project’s central galaxy classification task, in and of itself, is not associated with increased astrophysical content knowledge. We strongly recommend that future studies of online citizen science environments examine how volunteers take advantage of opportunities to develop their knowledge and skills outside of the self-contained central task, especially in the context of opportunities for interactions with other volunteers.
Keywords
1. Introduction
Many authors have expressed concern that adults in the United States and worldwide lack science literacy (e.g. Hazen and Trefil, 2009), which is increasingly seen as essential to making informed decisions about science, technology, and public policy issues (Miller, 2012). Science literacy has been defined in many different ways, and has been theorized to include many components, but nearly all definitions include knowledge of scientific concepts and theories (American Association for the Advancement of Science, 1993). Because most adults will never take a science course in the formal education system beyond what they experience in high school or to satisfy a general education requirement in college, the informal science education (ISE) community has identified promoting adult science literacy as a major goal for research and practice (McCallie et al., 2009).
One approach to public informal science education that has shown great promise has been that of citizen science, in which volunteers participate in scientific investigations alongside professional scientists. Researchers estimate that more than a million adults have participated in some form of citizen science (Theobald et al., 2015). The large number of people participating in citizen science demonstrates its potential for informal science education. Because citizen science gives volunteers an opportunity to directly participate in a real and active science investigation, there are good reasons to believe that participation should enhance their scientific literacy. This assertion is testable and should be investigated carefully. Such an investigation is particularly important because impacting science literacy has been identified as an important outcome of citizen science (Bonney et al., 2009).
In this article, we investigate the impact that participating in a citizen science program has on the science content knowledge of the volunteers. We focus on a single citizen science project called Galaxy Zoo (www.galaxyzoo.org). This work builds on prior work to study content knowledge among Galaxy Zoo volunteers (Prather et al., 2013), in which conclusions were limited by a small sample size. This work uses a revised methodology, which resulted in much greater participant response rate and a dataset that allows us to draw meaningful inferences about volunteers’ science content knowledge.
This article is organized as follows. In section 2, we review prior research about volunteers in citizen science programs and their learning of science content, both in Galaxy Zoo and in other citizen science projects. In section 3, we describe our research methods, including our research instrument and how it was implemented through the Galaxy Zoo online user interface. In section 4, we present our results and analysis. In section 5, we discuss what these results tell us about Galaxy Zoo volunteers’ science content knowledge and then outline paths for future work. Note that throughout this article, we will use the term “volunteers” when referring to individuals working within a citizen science program, and “participants” when referring to the individuals who provided responses for this study.
2. Prior work
Background on citizen science and Galaxy Zoo
The term “citizen science” has long been used in discussions of public understanding of and engagement with science (e.g. Irwin, 1995), but we use the term “citizen science” in a narrower sense to describe projects in which a network of distributed volunteers from the general public work with professional scientists to complete research investigations. Some such projects have been ongoing for more than a century (Silvertown, 2009). Many organizations have created citizen science projects, and the results provided by these projects have had a major impact on scientific research (Dickinson et al., 2012).
Over the past decade, a new form of citizen science has arisen: projects in which volunteers analyze real data entirely in an online environment. Such online citizen science projects have been developed in many areas of science (e.g. Mendez, 2008; Price and Lee, 2013; Voss and Cooper, 2010). This article investigates one such online citizen science project called Galaxy Zoo, in which volunteers classify galaxies by shape (e.g. spiral and elliptical) as part of a research program into galaxy structure and cosmology (Lintott et al., 2008).
Since its launch in 2007, Galaxy Zoo has attracted more than 400,000 volunteers, who have performed more than 11 million classification tasks. Their work has led to a robust database of galaxy classifications (Lintott et al., 2011), which has so far led to 57 peer-reviewed publications (an up-to-date list is maintained at www.zooniverse.org/about/publications). Galaxy Zoo has gone through four iterations; in this work, we study volunteers in Galaxy Zoo 4, which launched in 2012 (Simmons et al., 2016). Galaxy Zoo was the first citizen science project in the Zooniverse citizen science platform (www.zooniverse.org), which has so far crossed more than 90 projects in fields ranging from astronomy to ecology to archeology to art history (an up-to-date list of projects is maintained at www.zooniverse.org/projects).
Prior studies of citizen science volunteers
An important emerging area of research is the study of those members of the public who choose to volunteer as citizen scientists. Researchers have explored their demographics, the nature of their participation in citizen science activities, and their motivations for participating (e.g. Nov et al., 2011; Raddick et al., 2013; Rotman et al., 2012), as well as the impact of citizen science participation both on their understanding of the nature of science (e.g. Cronje et al., 2011; Trumbull et al., 2000) and their attitudes toward science (e.g. Bonney et al., 2015; Price and Lee, 2013).
Other work has focused on exploring the impact of citizen science participation on science content knowledge. A study of the NestWatch citizen science project (www.nestwatch.org) from the Cornell Lab of Ornithology showed that participants increased their knowledge of bird biology (Brossard et al., 2005). A study of another citizen science program in which hikers identify invasive plants showed that participants gained knowledge about invasive plants and their management (Jordan et al., 2011). It is worth noting that both of these citizen science projects engaged participants in data gathering and analysis tasks much more closely aligned with the work of an expert in the field than what is experienced by the Galaxy Zoo participants. We believe this alignment can significantly affect the results of investigations into the level of discipline knowledge and skills attained by participants.
With regard to studies specifically of Zooniverse participants, a recent investigation found evidence of volunteer learning in five different citizen science projects, including Galaxy Zoo (Masters et al., 2016). Respondents were given “science quizzes” in which they were shown a series of images and asked to identify the objects in the image. Objects shown were either specific to the project content (e.g. a spiral galaxy, an elliptical galaxy) or were related to general science knowledge (e.g. a molecule, an insect). Participants’ science activity level was measured by the total number of classification tasks and the total length of time volunteers had engaged classifying. The authors found positive correlations between these measures of engagement and project-specific knowledge, but not between engagement and general science knowledge. For our work, we wish to investigate the degree to which participants gain a greater knowledge of specific discipline concepts that are central to an expert’s understanding of the topic being investigated. Instead of providing participants with images similar to those they analyzed in Galaxy Zoo, our assessment asked participants to answer conceptually challenging questions designed to measure their level of discipline knowledge.
The current investigation began with a pilot project to study whether participation in Galaxy Zoo was associated with gains in astronomy content knowledge (Prather et al., 2013); we describe this pilot study in section 3. The pilot study found a greater percentage of correct responses for participants who had completed a greater number of classification tasks; however, the conclusions of our pilot study were deeply limited by the issue that only 160 participants answered all questions on our instrument (out of more than 11,000 total Galaxy Zoo volunteers).
To ensure that the current study includes a large sample of the Zooniverse population, in which participants both answered all questions and represented enough variation in important population characteristics, we significantly redesigned our study implementation methods, as described below.
3. Methods
Many of the research methods employed in formal science education are difficult to implement when studying an online volunteer population. For example, traditional pre-/post-testing is extremely difficult with our audience, since a pretest would require these volunteers to put off participating in any citizen science activities until answering a series of what may seem like “unrelated” questions. Some potential volunteers might leave the project before even starting, which would not only result in a biased study population but could also cause harm to the overall research project by removing potential volunteers before they even begin their work. Similarly, our inability to implement a pre-test makes it nearly impossible to assess volunteers’ prior knowledge before joining Galaxy Zoo.
This limitation means that a traditional pre-/post- study design of learning gains is impractical with our population of citizen scientists. Instead, we study a large sample of volunteers with a wide range of experiences with Galaxy Zoo. By comparing the state of knowledge of new and veteran volunteers, we can gain an insight into whether different levels of experience are associated with differences in understanding of scientific concepts. Our revised implementation strategy resulted in enough volunteer responses, across a wide-enough range of experiences, to allow for meaningful comparisons of volunteer content knowledge.
We will now describe in detail the methods we used to ensure that all participants answered all questions, and that our sample included participants with a wide range of experiences. All Galaxy Zoo 4 volunteers create an account, read a one-page tutorial about galaxy types, and complete one required sample classification before classifying galaxies.
To ensure that we are getting a sample that includes new volunteers, we wanted to begin asking questions as soon as possible after volunteers first encountered Galaxy Zoo 4. But, mindful of the concerns raised above, we wanted to make sure that volunteers would not be bored, intimidated, or distracted from the classification task by immediately answering questions from our instrument. Therefore, we chose to display questions after volunteers had classified five galaxies—likely enough to ensure that volunteers would persist in classifying after a brief interruption, but few enough that volunteers could still be considered new for the purposes of our study.
After classifying their first five galaxies in Galaxy Zoo 4, a volunteer was presented with an invitation to answer questions selected from our instrument. The invitation to participate in this study was presented in a message displayed over the website text, requiring a response to proceed (in the language of web development, a “modal window”). The invitation described the questions as a survey, and began with the message, “Galaxy Zoo needs your help! We are doing research to better understand what Galaxy Zoo volunteers think about the universe.” Volunteers were told that each time they completed a set of questions, they would be entered into a raffle to win a US$20 Amazon gift certificate. We chose this incentive because we reasoned that since Zooniverse is an online project, an incentive involving another website would not lead to a biased sample. Figure 1 shows a screenshot of the invitation as volunteers saw it.
A screenshot of the invitation that was shown to users after classifying five galaxies.
The invitation closed with the question “Would you like to participate?” and three buttons for responses: “Yes!,” “No, thanks,” and “Ask later.” If the user answered yes, they were immediately presented with the first set of questions. If they answered no, we did not ask again. If they answered ask later, they were presented with the same invitation one week later, provided they had classified five or more galaxies during that week.
Questions were presented in sets of five to six questions each; there were a total of five question sets, presented in the same order to each volunteer. Participants were required to complete all questions in a set before they would be allowed to move forward in the study. Once they answered all questions, they had the choice of immediately answering the next set of questions or waiting. If they chose to wait, they were given the next set one week later, provided they had completed five or more classifications during that week.
We began implementing our assessment questions when Galaxy Zoo 4 launched on September 11, 2012. The dataset described in this article includes responses through January 3, 2014. During this period, 39,008 volunteers responded to our prompt. Of those, 19,997 volunteers (51%) eventually said yes (either immediately or after having previously clicked “Ask later”).
For us to make any claims about participants’ understanding of astronomy concepts, it is important that we analyze only responses from participants who had an equivalent opportunity to demonstrate their science content knowledge. We therefore set three requirements for a participant to be included in our study sample:
They must have answered all questions from the instrument (all questions from all question sets);
Have spent more than 10 minutes in total responding to the instrument (i.e. between their first submitted response and their final one);
Have five or more galaxy classifications at the time they answered the first question.
We selected the 10-minute cutoff based on our experience with designing and testing the instrument; even content experts were unable to provide answers to all questions in less than 10 minutes. We selected the five-classification cutoff to ensure that all volunteers had had some exposure to the Galaxy Zoo classification task at the time they answered our questions.
Figure 2 illustrates the steps we took in our sample selection. After completing all the selection steps, our final dataset contained responses to the full instrument from 1476 volunteers. This represents 3.8% of all volunteers who saw the prompt about the study.
A diagram showing our sample selection process. Of the 39,008 volunteers who saw the prompt about participating in the study, 19,997 (51%) clicked yes to agree to participate. We included only responses from volunteers who answered every question on the instrument. We then selected only responses from volunteers who spent an appropriate amount of time answering questions, and those who had an appropriate amount of experience with Galaxy Zoo. Our final study sample included responses from 1476 volunteers (3.8% of the original sampling frame).
Next, we will provide a brief discussion of the assessment instrument used in this study.
Astronomy assessment instrument
Our research objective was to measure the extent to which participants demonstrated knowledge of astronomy concepts, rather than simply measuring their skill at classifying galaxies. One might hope that engagement in a science investigation would lead to increased understanding among participants. Measuring robust understanding of a topic requires moving beyond asking only factual questions, and must involve asking conceptually challenging items that reveal a participant’s deeper understanding of the central principles of the field. In the case of galactic astronomy, these central principles include ideas involving matter–energy interactions, stellar properties and evolution, light and spectroscopy, and gravitational forces.
Before describing our astronomy assessment instrument further (hereafter, the Zooniverse Astronomy Concept Survey, or ZACS), it is important to consider the scope of our research. We are neither measuring participants’ skill in classifying galaxies nor are we measuring understanding of concepts specifically associated with the task of classifying galaxies. Rather, we focus on assessing the relationship between participants’ knowledge related to galactic astronomy and their level of experience with the Galaxy Zoo project. Assessing such a relationship between two variables requires valid and reliable metrics for each variable. Level of experience with Galaxy Zoo (our independent variable) can be easily quantified, and we discuss several possible metrics below in the “Measuring volunteer engagement with Galaxy Zoo” section. The ZACS has been designed to measure the dependent variable in our study: participants’ understanding of topics related to galactic astronomy.
We designed the ZACS to measure the extent to which participants have or gain the type of content understanding that would be expected of a learner encountering galactic astronomy for the first time. The study of citizen science volunteers is so new that we were unable to find research-based concept inventories that had been tested with a citizen science volunteer population. However, there is another population of novice astronomy learners for which there exists a wealth of pre-existing concept inventories: students in undergraduate Astronomy 101 courses. Astronomy 101 students are mostly non-science majors; thus, like citizen science volunteers, they likely have not previously encountered astronomy concepts in their education—but they have likewise self-selected for at least some interest in astronomy. In addition, our prior research shows that more than 95% of Galaxy Zoo volunteers have at least some college coursework (Raddick et al., 2013), suggesting that they should be familiar with the content and style of introductory college course test questions.
We have more than 20 years of combined experience creating research-validated instructional materials and assessment instruments for use with introductory astronomy undergraduate courses, with content coverage over a significant range of foundational topics in astronomy. The items used in the ZACS are adapted from the Center for Astronomy Education’s (CAE) extensive bank of Astro 101 summative and formative evaluation questions, which have been used in several prior research efforts (e.g. Bailey et al., 2012; Bardar et al., 2007; Hudgins et al., 2006; Prather et al., 2004, 2009; Wallace et al., 2011; Williamson et al., 2013).
In our pilot study described above, we drew on these field-tested items to create an assessment instrument called the ZACS to measure the state of conceptual astronomy content knowledge of Galaxy Zoo volunteers (Prather et al., 2013). From the analysis of data from implementing the ZACS in our pilot study (Prather et al., 2013), we made a few changes to the instrument. The revised instrument, known as the ZACS v2, is included with this article as –Supplemental Appendix A; for further information, readers may contact the primary author.
4. Results
Instrument properties
Before using results from the ZACS v2 to evaluate volunteer content knowledge, our first task is to evaluate the extent to which the instrument really is a valid and reliable measure of volunteer knowledge.
To investigate the reliability of the ZACS v2, we first evaluated responses to its individual items. We removed responses to two items from our analysis—one because only 16 percent of participants responded correctly (indicating the item’s difficulty level was beyond the understanding of the majority of the participants), and one because its results anticorrelated with results of the full instrument. These two items are noted on the instrument in Supplemental Appendix A. With those items removed, the ZACS v2 consists of 32 questions and has a Cronbach’s alpha reliability value of α = .820, well within suggested reliability guidelines (Nunnally, 1978).
The face validity of the reduced ZACS v2 was established through testing with content experts before its implementation with the Galaxy Zoo volunteer population. In addition, the validity of its 32 component items has been well-established through extensive testing with general education college students. All these factors taken together help to establish the ZACS v2 as a valid and reliable instrument to measure content knowledge of Galaxy Zoo volunteers.
Volunteer total scores on the 32-item ZACS v2 are approximately normally distributed, as shown by a K–S test of normality (D = 0.50, n = 1476, p = .057). The mean participant score in our dataset is 19.3 out of 32, with a standard error of the mean estimate of 0.15. The 95% confidence limits for the mean are (19.0, 19.6). The normal distribution and wide range of scores suggest that our instrument can indeed distinguish among varying levels of astronomy content knowledge in our population.
Measuring volunteer engagement with Galaxy Zoo
The ZACS v2 provides a measure of our dependent variable—astronomy content knowledge—but we must also define a measure for our independent variable—participant engagement with Galaxy Zoo. Potential measures of engagement might include measures of either number of galaxies classified or time since joining the project; fortunately, the Galaxy Zoo database offers multiple options for measuring either construct.
At the time a participant submits their responses to the first ZACS v2 question set, and at the time they submit their final responses, Galaxy Zoo records the number of galaxies they have classified and the time elapsed since they first created their account. We constructed three independent variables from these data:
The number of galaxies they had classified at the time of their first responses (hereafter “starting classifications”);
The number of galaxies they classified in the time between their first and final responses (hereafter “delta classifications”); and
The time elapsed between when they first created a Galaxy Zoo account and when they responded to the first question set (hereafter “account age”).
We will look for relationships between participants’ total scores on the reduced ZACS v2 and each of these three independent variables. These independent variables are not proposed as a complete model of Galaxy Zoo involvement, but they provide multiple ways for meaningfully measuring involvement.
It is important to note that account age cannot be used as a straightforward proxy for a participant’s length of involvement in Galaxy Zoo. The account age as defined here cannot distinguish between very different patterns of participant behavior. An account age of one year indicates only that the participant created an account one year before answering their first ZACS v2 question set. It is not a direct indication that the participant has logged in recently or performed any classification tasks during the past year.
Defining groups of volunteers for analysis
Our ultimate goal is to explore whether there is a correlation between any of our independent variables and the dependent variable of ZACS v2 score.
A linear correlation measure would be misleading, because the distributions of all independent variables are highly non-normal and coupled to a wide range of values. In terms of starting classifications, most participants were new volunteers who had performed only the five required to receive ZACS v2 questions, but some were long-standing volunteers with more than 5000 classifications. Account ages similarly ranged from only a few minutes to more than four years. Delta classifications ranged from zero (meaning participants answered all 32 ZACS v2 questions in one sitting) to more than 20,000 classifications.
With such a wide range of values, a rank correlation approach (sometimes known as Spearman correlation) is more appropriate. We calculated the rank correlation coefficient between each of our independent variables and total score on the ZACS v2 instrument. None of the correlations is statistically significant: all are small, and some are negative. A similar approach—finding the value–value correlation between the log transforms of these independent variables with the log transform of ZACS score—provided statistically insignificant correlation coefficients of similar magnitude.
Clearly, our correlation-based approach does not indicate any relationship between our measures of Galaxy Zoo engagement and ZACS v2 score. But the distribution of engagement measures has additional complications, such as the fact that so many participants start out with only five classifications. A better approach might be to bin these variables and then analyze how scores vary among the bins.
The ideal binning of any of the independent variables is not immediately obvious. To help us examine how ZACS v2 scores vary among such a wide range of values, we tried to create bins for each independent variable that had roughly equal numbers of participants. An exception to the principle of equal binning was for starting classifications; because so many people created accounts and began answering questions immediately after completing their five required classifications, we therefore had a much larger bin for five classifications.
Tables 1 (a,b,c) identify the independent variable bins, and how many participants are in each bin.
Bins used for independent variables in our subsequent analysis, for the number of galaxies participants had classified at the time of their first responses.
Bins used for independent variables in our subsequent analysis, for the number of galaxies classified in the time between their first and final responses.
Bins used for independent variables in our subsequent analysis, for the time elapsed between when they first created a Galaxy Zoo account and when they responded to the first question set.
Content knowledge by group
Having binned groups of participants based on their level of Galaxy Zoo engagement, we then looked for variations in ZACS v2 scores among the groups. Figure 3 and Table 2 show our results. Error bars in the figure show 95% confidence intervals for the mean. Table 2 also gives the results of an analysis of variance (ANOVA) test for statistical significance, along with the effect size expressed as Cohen’s f (Cohen, 1988) as calculated in SPSS.
The mean score on the ZACS v2 astronomy assessment instrument for volunteers in each bin on (a) the number of galaxies they had classified at the time of their first responses (starting classifications); (b) the number of galaxies they had classified in the time between their first and final responses (delta classifications); and (c) the time elapsed between when they first created a Galaxy Zoo account and when they responded to the first question set (account age).
Data and summary statistics for the relationships shown in Figure 3.
ANOVA: analysis of variance; df: degrees of freedom.
ANOVA results report the test statistic value (F), df, and the associated p-values.
We had hoped that Figure 3 and Table 2 would show that increased engagement with Galaxy Zoo would be associated with increased mean ZACS v2 scores for all three independent variables. However, for starting classifications, it appears that there is no difference in scores among bins. Surprisingly, for delta classifications, it appears that there is a slight decrease in scores with increasing delta classifications. Only for starting account age does it appear there is an increase in mean score.
The results in Table 1 show statistically significant differences in mean ZACS v2 scores for two out of three independent variables: delta classifications and account age at first response, but it is important to note that a result may be statistically significant even though the calculated difference is quite small. Thus it is important to also consider the effect size, which we will discuss next.
In Tables 1 and 2, we report effect sizes with Cohen’s f. Cohen’s f can be interpreted as the average difference in means across bins, divided by the overall standard deviation of the entire sample. The role of Cohen’s f in ANOVA is analogous to the role of r2 in linear regression—a measure of the extent to which differences in the dependent variable can be explained by differences in the independent variable. Cohen (1988) suggested that f values between .10 and .25 should be considered a “small” effect, f values between .25 and .4 should be considered a “medium” effect, and f values greater than .4 should be considered a “large” effect. All three of the relationships we examine are of small effect size.
It may be tempting to conclude from the trend shown in Figure 3, and the statistical significance identified in Table 2, that volunteers with greater starting account age know more astronomy content, but the effect size is still too small to meet our standards for drawing firm conclusions. In conducting similar research into the science content knowledge of college non-science majors before and after instruction, we observed much larger effect sizes (e.g. Hudgins et al., 2006). In other words, if Galaxy Zoo were definitively enabling volunteers to develop more robust understandings of astronomy, we would expect to see much greater effect sizes.
5. Conclusion
This study investigates a question that has been identified as important for informal science education research: To what degree is participation in citizen science activities associated with an increased understanding of science? We examine this question in the context of Galaxy Zoo, a highly successful citizen science project in astronomy.
We conducted a prior study that received too few responses to draw meaningful conclusions. In response, we adapted our implementation strategy, asking Galaxy Zoo volunteers to answer sets of questions until they had responded to every question on our revised assessment instrument (the ZACS v2).
Our revised implementation was successful in obtaining complete responses from 1476 volunteers, and our analysis indicates that the ZACS v2 is valid and reliable for the Galaxy Zoo volunteer population. The resulting dataset allowed us to examine relationships between various measures of participants’ Galaxy Zoo engagement and their understanding of astronomy concepts.
We found no statistically significant relationship between mean reduced ZACS v2 scores and the number of starting classifications. We found a very small but statistically significant decrease in mean scores with increasing numbers of classifications completed during the course of answering items. We found a statistically significant increase in mean reduced ZACS v2 scores with increased starting account age, although with an effect size that we consider too small to suggest a meaningful change in knowledge.
Taken together, these results suggest that participating in the main science task of Galaxy Zoo—classifying galaxies by shape—is not associated with increased astronomy content knowledge. Although the data we collected cannot say why this should be the case, comparison with prior work (e.g. Brossard et al., 2005; Jordan et al., 2011) suggests a possible explanation. Whereas participant tasks in those projects required extended engagement with concepts central to their scientific disciplines, the Galaxy Zoo task could be accomplished quickly by participants without any prior experience in astronomy. Such a task can be advantageous for increasing participation in scientific research but may simply be insufficient to promote learning. It is also possible that while the independent variables used in this study do not seem to be associated with learning, other choices of independent variables might reveal relationships between Galaxy Zoo participation and increased astronomy content knowledge. Such variables to examine might include measures of the time taken to classify each galaxy, degree of agreement with other classifiers (discussed in more detail in Lintott et al., 2011), or motivational factors (Raddick et al., 2013). Qualitative research into learning among volunteers will likely lead to other meaningful insights.
One factor that we expect will be particularly important in promoting volunteer learning is social interaction. We know that learning is, to a great extent, a social phenomenon (Jarvis, 1987). In addition, we have some anecdotal suggestion that learning is taking place within Galaxy Zoo’s social spaces. Discussions with volunteers in the Galaxy Zoo forum (www.galaxyzooforum.org) and talk tool (https://talk.galaxyzoo.org) indicate that some volunteers have become deeply engaged in self-study of science, technology, engineering, and mathematics (STEM) topics.
Thus, variables that measure volunteers’ activities within the social networking spaces of Galaxy Zoo might be associated with increased astronomy content knowledge. Opportunities for volunteer social engagement within Galaxy Zoo and Zooniverse include reading and commenting on blog posts written by Galaxy Zoo project scientists, discussing science with other volunteers in the Galaxy Zoo Forum and Galaxy Zoo Talk tool, and even meeting up in person with other volunteers. All these activities provide opportunities for volunteer interaction. We know these social interactions happen, but we do not know much about their role in learning, if any. We suggest that these interactions should be the focus of the next set of studies into science learning through citizen science.
Our study can provide a few recommendations to other citizen science projects. We expect the best opportunities for learning to come from projects whose participant tasks are complex and central to the project’s science domain—although we do not know this with certainty, and it should be a research question for future studies. Similarly, we expect that providing volunteers with opportunities to engage with each other and with project scientists will provide increased opportunities for learning. Finally, learning opportunities should result from science learning activities being explicitly designed into citizen science activities. Some Zooniverse projects provide learning content and activities available from the main task page, and many other citizen science projects offer learning resources and/or in-depth trainings. The Zooniverse team, along with many other groups, have adapted these resources for use in formal education settings, such as K–12 classrooms. The instrument and methodology described in this article may be useful for researching the effectiveness of Galaxy Zoo and its associated learning activities in formal education settings.
Supplemental Material
PUS-_supplemental_Material – Supplemental material for Galaxy zoo: Science content knowledge of citizen scientists
Supplemental material, PUS-_supplemental_Material for Galaxy zoo: Science content knowledge of citizen scientists by Michael Jordan Raddick, Edward E. Prather and Colin S. Wallace in Public Understanding of Science
Footnotes
Acknowledgements
The data for this article are the result of the efforts of the Galaxy Zoo volunteers, without whom none of this work would be possible. Their efforts are individually acknowledged at 
. SDSS-III is managed by the Astrophysical Research Consortium for the Participating Institutions of the SDSS-III Collaboration including the University of Arizona, the Brazilian Participation Group, Brookhaven National Laboratory, Carnegie Mellon University, University of Florida, the French Participation Group, the German Participation Group, Harvard University, the Instituto de Astrofisica de Canarias, the Michigan State/Notre Dame/JINA Participation Group, Johns Hopkins University, Lawrence Berkeley National Laboratory, Max Planck Institute for Astrophysics, Max Planck Institute for Extraterrestrial Physics, New Mexico State University, New York University, Ohio State University, Pennsylvania State University, University of Portsmouth, Princeton University, the Spanish Participation Group, University of Tokyo, University of Utah, Vanderbilt University, University of Virginia, University of Washington, and Yale University.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work has been supported by the Sloan Digital Sky Survey III (SDSS-III). Funding for SDSS-III has been provided by the Alfred P. Sloan Foundation, the Participating Institutions, the National Science Foundation, and the U.S. Department of Energy Office of Science. The SDSS-III website is . This work was also supported by the National Science Foundation (award number 0802876). The Galaxy Zoo 4 citizen science project and website was supported by the National Science Foundation (award number 0941610).
Supplemental material
Supplemental material for this article is available online.
Author biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
