Impacts and Characteristics of Computer-Based Science Inquiry Learning Environments for Precollege Students

Abstract

The National Science Foundation–sponsored report Fostering Learning in the Networked World called for “a common, open platform to support communities of developers and learners in ways that enable both to take advantage of advances in the learning sciences.” We review research on science inquiry learning environments (ILEs) to characterize current platforms. We searched databases and 11 major science and technology journals and identified 30 distinct ILEs investigated in articles published from 2008 onwards. We use research-based inquiry principles to analyze ILE features that support learners, teachers, developers, and researchers. We identify ILEs that are taking advantage of learning sciences research, building on the accomplishments of others, and creating the platforms envisioned in the report. We hope that this review will help teachers, designers, and researchers identify platforms they can customize and enhance rather than starting anew if unnecessary. Enhancing existing platforms combines the efforts of many individuals and, thus, strengthens the field.

Keywords

inquiry learning environment platform visualization scaffolding autonomy socioscientific context knowledge integration

The National Science Foundation (NSF) sponsored report Fostering Learning in a Networked World (Borgman et al., 2008, p. 23) called for “a common, open platform to support communities of developers and learners in ways that enable both to take advantage of advances in the learning sciences.” The authors of the report argued that “the platform architecture must be designed so that it can evolve, especially over the coming decade as computing shifts come into their own” (Borgman et al., 2008, p. 23). We review articles published in 2008 or later to map the landscape of contemporary inquiry learning environments (ILEs); to analyze their impact on science learning gains; to identify opportunities for students, teachers, researchers, and designers; and to seek evidence that the field is developing a cumulative, generative set of open platforms. To capture progress in achieving the goals of the Fostering Learning in a Networked World report, we address the following research questions: (a) What inquiry features characterize contemporary ILEs; (b) what is the impact of ILEs on student learning and do features aligned with research-based design principles contribute; (c) how are teachers supported in implementing contemporary ILEs; and (d) what are emerging platforms for cumulative improvements of ILEs?

Science ILEs support design and delivery of curriculum materials for middle and high school students. ILEs have been referred to by many names including digital teaching platforms (Dede & Richards, 2012), model-based science learning environments (Sun & Looi, 2013), and computer-based learning environments (Roscoe, Segedy, Sulcer, Jeong, & Biswas, 2013). We define ILEs as curriculum delivery systems that provide instruction for one or more science topics, take advantage of technology to represent complex ideas using visualizations and/or ask students to represent their ideas visually, scaffold inquiry, enable embedded assessments, and support registration and logging of progress to monitor student outcomes. For example, an ILE with a climate change unit would support students by using visualizations to illustrate the effect of carbon dioxide in the atmosphere, allow students to explain their ideas about climate change, and log student responses. Teachers could use the logged responses to give guidance and for assessment.

Value of Inquiry Learning

Despite the widespread research on scientific inquiry over the past two decades, a universal definition for scientific inquiry remains elusive (Anderson, 2007; Cuevas, Lee, Hart, & Deaktor, 2005). However, core features of scientific inquiry are evident in various definitions across the literature, including questioning and generating hypotheses, experimenting, designing, and planning, predicting, modeling/visualizing, observing and data collection, analyzing data, interpreting and explaining, developing/evaluating/arguing, reaching conclusions, and communicating findings (Bell, Urhahne, Schanze, & Ploetzner, 2010; Furtak, Seidel, Iverson, & Briggs, 2012; McNeill, Pimentel, & Strauss, 2011; Minner, Levy, & Century, 2010; National Research Council [NRC], 1996, 2000; Osborne, Erduran, & Simon, 2004; Zion & Mendelovici, 2012). For example, in a review of inquiry instruction, Quintana et al. (2004, p. 341) incorporated some of these features into their definition: [inquiry is] “the process of posing questions and investigating them with empirical data, either through direct manipulation of variables via experiments or by constructing comparisons using existing data sets.”

Building on these definitions, the Next Generation Science Standards (NGSS) characterize practices for engineering and science, such as asking questions and defining problems, constructing explanations and solutions, and engaging in evidence-based arguments (NRC, 2011). The NGSS call for instruction that develops students’ ability to use these practices in the context of inquiry activities (Furtak et al., 2012; Quintana et al., 2004). The NGSS identify the focus of inquiry as core disciplinary ideas and cross-cutting concepts including concepts associated with the nature of science (NRC, 2011).

Teachers and researchers need to help students make judicious connections between practices, core ideas, and cross-cutting concepts. A powerful ILE can help them refine inquiry materials using evidence from trials with students (e.g., Edelson, Gordin, & Pea, 1999). For example, a climate-change unit where students design a house with reduced CO₂ emissions implemented in an ILE could be tested using embedded assessments that measure the use of the NGSS practices. Instruction could be refined to strengthen the likelihood of learning the practices.

Researchers have called for inquiry methods to strengthen understanding of science concepts and to enhance motivation to learn science. Inquiry can support the development of skills students need in the current workplace such as collaboration, communication, digital literacy, citizenship, problem solving, critical thinking, creativity, and productivity (Dede, 2010; Voogt, Knezek, Cox, Knezek, & Ten Brummelhuis, 2013). Reviews document the impact of inquiry on student learning (Furtak et al., 2012; Minner et al., 2010; Quintana et al., 2004). Inquiry methods may succeed by increasing student motivation in science, guiding students to delve into authentic problems, or providing students with insight into what practicing scientists do (Chinn & Hmelo-Silver, 2002; McCrory Wallace, Kupperman, Krajcik, & Soloway, 2000; Schwarz & White, 2005).

Operationalizing Inquiry

Blanchard et al. (2010) offered a way to categorize inquiry instruction that could help evaluate ILEs. They described teacher-led inquiry that involves verification of a scientific concept, principle, or law with all instructions provided by the teacher (Level 0). Structured inquiry (Level 1) provides students with instructions toward finding answers to a particular problem defined by the teacher. In guided inquiry (Level 2), the teacher gives students a question, but the students must design the experimental procedure and develop their own findings. In open-ended inquiry (Level 3), students must decide the question and the means to solve it, but the teacher may also support the development of the question to be answered. Many ILEs can support Level 0 inquiry by recording student work in a gradebook. Most ILEs support teachers to use Level 1 and 2 inquiry activities and some support aspects of Level 3. We use these categories to analyze the potential of ILEs reviewed in this article.

Design Principles for ILEs

Researchers have synthesized valuable features of ILEs in design principles (Barab & Luehmann, 2003; Bell et al., 2010; Edelson, 2001; Hmelo-Silver, 2006; Kali, 2006; Linn & Hsi, 2000; Quintana et al., 2004; Reiser et al., 2001; van Joolingen, de Jong, & Dimitrakopoulou, 2007). Many early uses of the Web neglected these principles. For example, assignments often called for unguided search to locate the “right” answers online (McCrory Wallace et al., 2000) and websites replicated the textbook rather than taking advantage of visualization technologies (Tuvi & Nachmias, 2001). Use of technology can be improved when supported by an ILE that incorporates these principles.

We reviewed the design principles associated with inquiry learning and extracted four cross-cutting design principles that capture the features that influence student learning and the development of inquiry skills. Researchers generally agree that ILEs guide students to develop inquiry skills by (a) exploring meaningful and authentic scientific contexts such as climate change, thermodynamics, healthy eating, and so on (Barab, Makinster, Moore, & Cunningham, 2001; Krajcik & Blumenfeld, 2006; Linn & Hsi, 2000; Schwarz & White, 2005); (b) using powerful visualizations to illustrate phenomena that are too small (chemical reactions), fast (airbags), or vast (the solar system) for classroom experimentation (Höffler & Leutner, 2007; Kozma, 2003); (c) encouraging collaboration with others (Barab & Luehmann, 2003; Bell, Gess-Newsome, & Luft, 2008; Vygotsky, 1978); and (d) developing autonomous, metacognitive learning practices that involve having students set their own learning goals, distinguish ideas, reflect, and link their ideas into complex arguments (NRC, 1996; Quintana et al., 2004; Slotta & Chi, 2006; Williams, DeBarger, Montgomery, Zhou, & Tate, 2012). We use these four successful design principles for inquiry instruction to analyze features of contemporary ILEs.

Classroom Implementation of Inquiry

Even though the potential of inquiry to positively impact learning is evident, enacting inquiry has met considerable difficulties within school contexts (Anderson & Helms, 2001; Roehrig & Luft, 2004). Students and teachers are unsure of the new roles and responsibilities that inquiry requires (van der Valk & de Jong, 2009) and teachers often lack the experience, content knowledge, and pedagogical strategies to succeed (Crawford, 2007; Kim, Hannafin, & Bryan, 2007). In addition, inquiry teaching often takes more time than transmitting information and may seem unaligned with the face validity of high-stakes assessments (Blanchard et al., 2010; Hodson, 1988; Lewis, 2006; NRC, 2011; Penuel, Fishman, Gallagher, Lopez-Prado, & Korbak, 2009; Singer, Marx, Krajcik, & Clay Chambers, 2000; Stage, Asturias, Cheuk, Daro, & Hampton, 2013).

Fortunately, recent research shows that ILEs offer many new tools and resources for teachers to successfully enact inquiry teaching strategies and are becoming more pervasive in middle and high schools (Webb, 2005). ILEs have the advantage of scaffolding learners and ensuring that students work in their zone of proximal development (ZPD), “the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance, or in collaboration with more capable peers” (Vygotsky, 1978, p. 86). ILEs provide ZPDs for students through technological, content specific, and social scaffolds (Dolonen & Ludvigsen, 2012; Pea, 2004). ILEs can help teachers guide students to explore interactive media such as visualizations (Hickey, Ingram-Goble, & Jameson, 2009; McElhaney & Linn, 2011). ILEs can also provide embedded assessments that help teachers monitor progress (Liu, Horton, Olmanson, & Toprac, 2011; McElhaney & Linn, 2011). Advanced ILEs can log student interactions and help teachers manage inquiry instruction. The logs can reveal patterns of interaction that might be difficult to extract by observation.

For example, balancing teacher, moderator, or online control with opportunities for individual exploration is a challenge for the design of inquiry instruction (Lee et al., 2011; Nicolaidou, Kyza, Terzian, Hadjichambis, & Kafouris, 2011; Raes, Schellens, De Wever, & Vanderhoven, 2012; Reiser et al., 2001). ILEs support teachers and researchers to compare alternative teaching strategies and to use logged data to evaluate the advantages of each (Hill & Hannafin, 2001). Using these tools, teachers could explore the balance between learner control and online scaffolds by assigning distinct guidance to groups of students. For instance, in developing explanations about what causes climate change, some students may simply need a prompt to elaborate on their explanations, whereas other students may be better served through a prompt to critique a sample student explanation in light of their original explanation.

To take advantage of ILEs, teachers benefit from professional development. Research shows that initially teachers focus on logistic issues such as accessing the Internet (Gerard, Varma, Corliss, & Linn, 2011). Once logistics are under control, teachers benefit from professional development that helps them ask inquiry questions and scaffold student interpretation of visualizations of complex phenomena (McElhaney & Linn, 2011). Successful adaptation of ILEs into teachers’ practice requires professional development that supports teachers to customize the ILE and experiment with different teaching strategies to enhance their students’ inquiry (Donnelly, O’Reilly, & McGarr, 2013; Hmelo-Silver, Duncan, & Chinn, 2007). For example, teachers appreciate insights from other teachers who have found ways to take advantage of tools that monitor student progress. They also benefit from opportunities to analyze when online guidance can succeed for their students and when they need to intervene.

Method

To conduct a review of the design and impact of ILEs on student learning, we searched databases and recent journals for all studies of inquiry learning. We selected studies that tested the impact of an ILE to teach precollege students about a science curriculum topic such as thermodynamics, plate tectonics, or photosynthesis. We focused on ILEs that took advantage of the technology by (a) using visualizations or asking students to generate visualizations to enhance representations for learners and (b) including scaffolding to structure the learner experience. We limited our review to ILEs that had embedded assessments to track student progress and supported student registration and logging to record student activities. To identify research on contemporary and active ILEs, we focused on articles published in 2008 or later.

Finding Articles

Databases

We initially searched the EFT (Education Full Text) and ERIC databases consistent with recommendations of Lee et al. (2011) concerning the value of such databases. We used the keywords “Learning Environment,” “Science,” “Online,” and “On-line” and identified in excess of 2,000 articles published over 15 years (1998–2012). We applied the criteria, full text, peer-reviewed, middle school, and high school, and achieved a more manageable number of articles: 282 in EFT and 179 in ERIC. Other keywords were tested in the databases—for example, “Computer,” “Web,” “Internet,” and “Technology”—but they yielded similar results. On reviewing the results from the initial search terms, many articles were excluded because they focused solely on teachers and not students, were not focused on science education, lacked registration systems, or involved only content generation in blogs or wikis. Our final set consisted of 18 relevant articles from 2008 onwards. We then extended the search to journals concerned with science or technology education.

Journals

We identified and searched 11 journals with science or technology foci: Cognition and Instruction, Computers and Education, Education Technology Research and Development (ETRD), Interactive Learning Environments, International Journal of Computer-Supported Collaborative Learning (CSCL), International Journal of Science Education (IJSE), Journal of Research in Science Teaching (JRST), Journal of Science Education and Technology (JSET), Journal of the Learning Sciences (JLS), Learning & Instruction, and Science Education. We identified all articles that possibly included an ILE over five and a half years of these journals (2008–June 2013). This search added 96 articles to the 18 articles found from the database search making 114 articles. Eight of the articles identified involved frameworks, reviews, or meta-analysis that in some way referred to ILEs, leaving 106 articles focused on individual ILEs.

We applied five criteria to the 106 articles: teaching science topic to precollege students, using visualization, enabling scaffolding, including embedded assessments, and supporting registration. These criteria resulted in 76 articles (identified in the References) that represented 30 ILEs. See Table S1 (online only) for keywords used for the ILE criteria and Table S2 (online only) for excluded ILEs (all supplementary materials are available online at http://rer.sagepub.com/supplemental).

Coding for Inquiry and Infrastructural and Implementation Features

To identify the value of the ILEs for supporting inquiry, we developed a rubric and coded each one for inquiry features. We analyzed the ILEs for the four successful elements of inquiry skills development noted from the literature: (a) exploring meaningful and authentic scientific contexts, (b) using powerful visualizations, (c) collaborating with others, and (d) developing autonomous, metacognitive learning practices. Analyzing the ILEs for these features provides a broad overview of how contemporary ILEs align with research from the learning sciences on how students develop inquiry skills. Table S3 (online only) includes the keywords used to help identify inquiry principles in the articles. Some of the keywords overlap, but these keywords were interpreted based on the context in which they were used in the articles (e.g., “draw” in the context of drawing a representation or drawing conclusions). We established interrater reliability using a sample of 10 articles representing 8 ILEs. The first author and another rater agreed 86% of the time. Disagreements were distributed across categories and mostly reflected the need to increase the clarity of definitions (e.g., refining the meaning of “selecting resources”). We used these findings to refine the rubric.

We searched and coded each ILE article for infrastructural features and teacher implementation experience. We recorded the frequency of infrastructural features that ILEs have to directly support teachers’ enactment of inquiry: authoring of content, grading of student work, computer-generated automated guidance/feedback, multiple units, student progress monitoring tools, and flagging of student work. Automated guidance/feedback includes the use of multiple choice responses or sophisticated algorithms for short open responses. The Web-based Inquiry Science Environment (WISE), for example, uses natural language processing to detect the presence of scientific concepts in student responses, thus allowing for targeted guidance (Linn et al., 2014). We coded the duration of studies of the ILE (1 year or less, 2 years or less, or longer than 2 years). We also coded for evidence of iterative refinement based on student learning outcomes and searched for the types of professional development provided to teachers. We used this information to illustrate the level of teacher support for ILE implementation and the refinements made based on teacher use of the ILE.

Computing Effect Sizes

We characterized the impact of the ILEs by computing effect sizes for pre/post and comparison-based design studies (Appendices A and B). Not all articles included data that could be used to compute effect sizes and these are not included in the tables. Appendix A includes the effect sizes of studies with pre/posttest designs that measure learning gains. Appendix B categorizes the weighted effect sizes of comparison studies (Hedge’s g).

Selecting Case Studies

To enrich the quantitative findings from reviewing studies of ILEs, we provide a deeper analysis of four specific ILEs selected based on their distinctive features, the number of inquiry features they include, and their availability with an open source license. We specifically selected all the ILEs that include authoring and could qualify as a platform for accumulating inquiry features. We discuss nQuire, Science Created by You (SCY), STOCHASMOS, and the WISE platforms (see Figure 1).

Figure 1.

Case study ILEs: WISE (top left), SCY (top right), nQuire (bottom left), and STOCHASMOS (bottom right).

The purpose of nQuire is to guide students’ overall inquiry process both inside and outside the classroom. nQuire focuses on scripted support for learning (Anastopoulou et al., 2011). SCY is focused on the student development of products (emerging learning objects [ELOs]) that are derived from answering socioscientific questions (e.g., how to build a CO₂ friendly house; de Jong et al., 2010). STOCHASMOS also supports students in answering a socioscientific problem and encourages students to develop evidence-based arguments through processes of planning, monitoring, and evaluating their investigations (Nicolaidou et al., 2011). WISE focuses on the enhancement of student knowledge integration for a broad range of science concepts and inquiry processes. Students engage in a variety of activities (virtual experiments, graphing, modeling) centered on an overarching, contextually relevant inquiry question (e.g., how to make airbags more effectively to reduce casualties; McElhaney & Linn, 2011). The activities are stepwise, but students are free and often encouraged to review or revisit previous steps (Gerard & Linn, 2013). Thus, the four ILE cases have distinctive features that should offer additional insight into inquiry learning.

Limitations of the Study

As noted already, there are many ways to define an ILE. In attempting to categorize ILEs and what they encompass, we may have omitted environments that others would have included. To ensure transparency, we explicitly define the criteria we use in this study so that others may be able to carry out similar studies or focus on particular aspects of ILEs in more detail. We have included a list of the excluded ILEs in the supplementary materials (Table S2; online only). Most ILE studies were excluded for lacking features of registration, taking advantage of visualization by computer or student, or collecting embedded assessments. A number of ILEs supporting argumentation were excluded, as they did not take advantage of visualization. We included pre/post studies to show the relative magnitude of the effect sizes across categories. There were a limited number of comparison studies within the included articles and as a result, evaluating the impact of ILEs on student learning was restricted.

The parameters we have used for inclusion may not be agreed upon by others. There is also a certain level of interpretation in determining whether an ILE has a specific feature based on descriptions in peer-reviewed journals and on the ILE’s website (if it has one) as reflected in the interrater agreement for a sample of items (86%). Although we did not include unpublished studies, based on the pre/post learning gains, we calculated a fail-safe N (Orwin, 1983) of 33.1 (p < .001); thus, we would need to identify 33 additional studies with a mean effect of essentially zero to reduce the combined effect to statistical nonsignificance.

If a feature was not noted in an article it may be left out in our analysis, but the authors may have noted it in another article not within the journals we searched. Nevertheless, given the large investments being made in ILEs and their promising role in science learning, a review of their features and impacts can contribute to the field. We call on researchers to characterize the features of their ILEs so that others can benefit from their work. The characteristics we have defined for analysis are a starting point toward making such features explicit.

Results

Overall, we identified 30 ILEs that met our criteria (Table 1). These ILEs were developed in 8 countries and studied by researchers in 12 countries (other countries not in the table, but where studies were conducted are Belgium, China, Estonia, and France). Seven of the ILEs serve middle school students (Grades 6–8, 23%), 15 serve high school (Grades 9–12, 50%), and 8 serve grades spanning across both middle and high school (27%). Of the 30 ILEs, 13 (noted with an asterisk in Table 1) are open source and can be used in classrooms (see Table S4, online only, for these websites; some may require a download or request for access).

Table 1

Inquiry learning environments within the selection criteria

Inquiry learning environment	Authors	Country	Grade
*Alien Rescue	Bulu & Pedersen, 2010; Hsieh, Cho, Liu, & Schallert, 2008; Liu, Horton, et al., 2011; Pedersen, Arslanyilmaz, & Williams, 2009	USA	6
APoME—Air Pollution Modeling Environment	Wu, 2010	Taiwan	10
Betty’s Brain	Chase, Chin, Oppezzo, & Schwartz, 2009; Roscoe et al., 2013; Segedy, Kinnebrew, & Biswas, 2013	USA	7 and 8
Buoyancy	Künsting, Wirth, & Paas, 2011	Germany	10
CCL—Case-based Computerized Laboratory	Dori & Sasson, 2008	Israel	12
*Co-Lab—Collaborative Laboratories across Europe	Bravo, Van Joolingen, & De Jong, 2009; Manlove, Lazonder, & de Jong, 2009; Mulder, Lazonder, & De Jong, 2010; Mulder, Lazonder, & De Jong, 2011; Mulder, Lazonder, De Jong, Anjewierden, & Bollen, 2012; Sins, Savelsbergh, Van Joolingen, & Van Hout-Wolters, 2011; van Borkulo, van Joolingen, Savelsbergh, & de Jong, 2012	Netherlands	10–12
COTF—Classroom of the Future: Exploring the Environment, Astronomy Village, Investigating the Universe, BioBlast	Hickey, Taasoobshirazi, & Cross, 2012; Cross, Taasoobshirazi, Hendricks, & Hickey, 2008	USA	10 and 11
*GenScope	Hickey & Zuiker, 2012	USA	9–12
IMMEX Project—Interactive Multi-Media Exercises	Thadani, Stevens, & Tao, 2009	USA	6–8
Inquiry Island	Eslinger, White, Frederiksen, & Brobst, 2008	USA	6–12
IQWST—Investigation and Questioning our World through Science and Technology	Berland, 2011; Berland & Lee, 2012; Krajcik, McNeill, & Reiser, 2008	USA	6–8
LensesSim	Chang, Chen, Lin, & Sung, 2008	Taiwan	11
*MAC—Modelling across the Curriculum: Dynamica, Biologica, Connected Chemistry	Gobert et al., 2011; Levy & Wilensky, 2009; Stieff, 2011	USA	9–12
*nQuire	Anastopoulou et al., 2011	England	10
PEC—Principles of Earthquakes Curriculum	Yeh et al., 2012	Taiwan	10
PhysHint	Ding, 2009; Ding, Bosker, & Harskamp, 2011; Pol, Harskamp, Suhre, & Goedhart, 2008; Pol, Harskamp, Suhre, & Goedhart, 2009	Netherlands	10
Polution Solution	Zydney & Grincewicz, 2011	USA	10
*QuestAtlantis	Barab, Pettyjohn, Gresalfi, Volk, & Solomou, 2012; Hickey et al., 2009	USA	6 and 7
RiverCity	Clarke & Dede, 2009; Nelson & Erlandson, 2008; Nelson & Ketelhut, 2008	USA	6–8
Scientific Concept Construction and Reconstruction (SCCR) Project	She & Lee, 2008; She & Liao, 2010; Tseng, Tuan, & Chin, 2010	Taiwan	6–9
*SCY—Science Created by You	de Jong et al., 2010; de Jong et al., 2012; Furberg, Kluge, & Ludvigsen, 2013; Krange & Ludvigsen, 2008	Netherlands	6–12
*SimQuest	Gijlers & de Jong, 2009; Kolloffel, Eysink, & Jong, 2011; Saab, van Joolingen, & van Hout-Wolters, 2009	Netherlands	10
STOCHASMOS—biotechnology Web-based learning environment (CoReReflect)	Hansson, Redfors, & Rosberg, 2011; Nicolaidou et al., 2011; Kyza, Constantinou, & Spanoudis, 2011	Cyprus	10
*SURGE	Clark et al., 2011	USA	7–9
*Uncommon Scents	Klisch, Miller, Wang, & Epstein, 2012	USA	6–8
Virtual Field Trip	Tutwiler, Lin, & Chang, 2012	Taiwan	10
*Viten	Mork, 2011	Norway	8–12
*Whyville	Feldon & Kafai, 2008; Kafai, 2008	USA	6–12
WiMVT	Sun & Looi, 2013	Singapore	6–12
*WISE—Web-based Inquiry Science Environment	Clark & Sampson, 2008; Clark, Touchman, Martinez-Garza, Ramirez-Marin, & Skjerping Drews, 2012; Gerard, Spitulnik, & Linn, 2010; Levy, 2012; Liu, Lee, & Linn, 2010; Liu, Lee, et al., 2011; Makitalo-Siegl, Kohnle, & Fischer, 2011; McElhaney & Linn, 2011; Raes et al., 2012; Svihla & Linn, 2012; Ryoo & Linn, 2012; Shen & Linn, 2011; Varma & Linn, 2012; Williams et al., 2012; Zhang & Linn, 2011	USA	6–12

Indicates open source Inquiry Learning Environment.

Inquiry Features

To answer our first research question, we analyze the four inquiry features in the ILEs.

Exploring Meaningful and Authentic Scientific Contexts

Of 30 ILEs, 19 (63%) featured an authentic context (see Table 2). Even when there are many appropriate, authentic, socioscientific questions or problems (for ecosystems, systems modeling, genetics, inclined planes), they were sometimes omitted or left to the teacher. Research shows that designing effective instruction around an authentic context is difficult because of the complexity of these situations. Most curriculum materials featuring meaningful contexts are designed by partnerships of discipline experts, educational researchers, and teachers, and have undergone several trials and refinements (e.g., Krajcik & Blumenfeld, 2006; Svihla & Linn, 2012). Classroom teachers generally do not have the time and resources to develop meaningful contexts on their own but can personalize a meaningful context for their students.

Table 2

Disciplinary focus of each inquiry learning environment (ILE) with relevant context

ILE	Disciplinary focus	Relevant context
Alien Rescue	Supporting alien life in our solar system	•
APoME	Determining air quality	•
Betty’s Brain	Food webs, Ecosystems
Buoyancy	Buoyancy
CCL	Acids and bases
Co-Lab	Systems modeling, Fluid dynamics, Speed control cameras, Greenhouse effect, Water management	•
COTF	Maintaining the survival of six crew members on a research facility	•
GenScope	Evolution/genetics	•
IMMEX	Cases—Why a crop of local potato plants is dying?	•
Inquiry Island	Genetics—Crosses between plants
IQWST	“How can I make new stuff from old stuff?” Ecosystems, Recycling, Making soap	•
LensesSim	Lenses
MAC	Genetics, Forces, Gas laws
nQuire	Healthy eating	•
PEC	Understanding earthquakes	•
PhysHint	Forces on a steam valve, Forces on an incline plane
Polution Solution	Make recommendations as an intern at an environmental consulting firm	•
QuestAtlantis	Supporting a doctor to find a cure for a plague	•
RiverCity	Help the mayor find out why town residents have become sick	•
SCCR	Combustion, Acids, Bases, Salts
SCY	How to make a carbon dioxide friendly house?	•
SimQuest	Collosions, Probability
STOCHASMOS	Extraterrestrial Life, Ecosystems, Genetic engineering—Should we colonize Mars?	•
SURGE	Help the female alien SURGE save the Fuzzies from the evil Emperor Hooke	•
Uncommon Scents	Understanding the exposure of toxic chemicals on sick scientists	•
Virtual Field Trip	National Park Virtual Environment
Viten	Radioactivity, Radioactive waste	•
Whyville	Understanding the outbreak of the epidemic Whypox	•
WiMVT	Current, Electricity, and Direct current circuit
WISE	Airbags, Cancer, Global climate change	•
Total (out of 30)		19

All four cases featured a socioscientific issue and/or an overarching inquiry question. nQuire tasks students with documenting the food they eat and analyzing the implications for healthy eating (Anastopoulou et al., 2011). SCY challenges students to create designs to solve socioscientific challenges such as making a house with low CO₂ emissions. STOCHASMOS engages students in using scientific evidence in a debate about a scientific dilemma, for example, considering whether genetically modified plants should be grown in their country. WISE design requirements call for including an overarching inquiry question such as “What impacts your genetic inheritance?” (Williams et al., 2012), “How can global warming be reduced?” (Varma & Linn, 2012), or “What are causes and possible cures for cancer?” (Linn & Eylon, 2011).

There are many calls in science education literature to portray relevant, accessible, and accurate science to students to encourage their interest and motivation in science (Saab et al., 2009). ILEs have the capability of supporting investigations of relevant scientific questions, and our analysis highlights that about two thirds of the ILEs provide meaningful and authentic contexts for students. These ILEs respond to numerous calls for personally meaningful student inquiry. These ILEs show how students can be supported to critically engage in science content while investigating novel and exciting scientific contexts. To succeed, the teacher plays an important role by guiding students to reflect on the role of scientific principles in the science contexts incorporated into the ILEs.

Using Powerful Visualizations

Using powerful visualizations refers to both making scientific concepts more visible through ILEs and making students’ thinking visible. All the ILEs used visualizations to take advantage of technology, consistent with the selection criteria (see Table 3). Overall, 13 of 30 ILEs (43%) guide students to use visualization for inquiry. Many of these ILEs, but not all, support Level 2 inquiry (Blanchard et al., 2010) guiding exploration of the visualization (Table 3). Some ILEs support aspects of Level 3 inquiry, where students must decide the question and the means to solve it.

Table 3

Nature of visualization and explicit inquiry process represented in inquiry steps (IS) for the inquiry learning environments (ILEs)

ILE	Nature of visualization(s)	Guided inquiry process/framework
Alien Rescue	Simulation
APoME	Modeling, Simulation, Graph	Model-based learning: Prompts for inquiry
Betty’s Brain	Mapping
Buoyancy	Simulation	Discovery learning: IS
CCL	Graph
Co-Lab	Modeling, Graph	Model-based learning: IS
COTF	Simulation, Video, Animation	Situative learning: Inquiry path provided
GenScope	Simulation
IMMEX	Animation
Inquiry Island	Modeling, Simulation	Inquiry cycle: IS
IQWST	Simulation, Modeling, Graph
LensesSim	Simulation	Simulation-based learning: IS
MAC	Simulation, Animation, Graph, Modeling
nQuire	Graph	Personal-inquiry learning: IS
PEC	Animation
PhysHint	Drawing
Polution Solution	Video	Cognitive flexibility theory: IS
QuestAtlantis	Virtual environment, Video
RiverCity	Virtual environment, Video
SCCR	Video, Animation
SCY	Simulation, Modeling, Mapping	Sociocultural learning: Inquiry map
SimQuest	Simulation
STOCHASMOS	Video, Graph	Evidence-based reasoning: IS
SURGE	Virtual environment, Simulation
Uncommon Scents	Virtual environment, Simulation
Virtual Field Trip	Virtual environment, Video
Viten	Simulation, Video, Modeling	Sociocultural learning: IS
Whyville	Virtual environment, Simulation
WiMVT	Simulation, Video, Graph, Modeling	Model-based collaborative inquiry: Inquiry step types
WISE	Simulation, Video, Graph, Modeling	Knowledge integration: IS
Total (out of 30)		13

The four cases capture the variety of guidance and interactivity available for visualizations. In nQuire, students generate graphs from the data they collect on their nutrient intake and compare outcomes. nQuire guides students in using inquiry processes such as “Decide my inquiry question or hypothesis,” “Plan my methods, equipment, and action,” and “Decide my conclusions” (Anastopoulou et al., 2011). SCY guides students’ visualization in the mission map using tools such as concept mapping, modeling, and simulations. Students can view their “missions” to complete in the mission map (de Jong et al., 2010; de Jong et al., 2012). These tools can be adapted to specific students or to the overall content. In STOCHASMOS, students gather and evaluate the credibility of evidence from sources such as video and graphs. STOCHASMOS provides students with an Inquiry Environment that is organized by tabs to help students choose appropriate inquiry processes (Nicolaidou et al., 2011). WISE uses knowledge integration guidance to engage students in visualizations including simulations, virtual experiments, video, graphing, and modeling for various science topics (McElhaney & Linn, 2011; Ryoo & Linn, 2012). Knowledge integration guidance involves asking students to make predictions; guiding students to use the visualization to add new, normative ideas; engaging students in distinguishing among their predictions and the new ideas in the visualization; and requiring students to reflect on their experiences and construct an argument to explain the topic under study (Liu et al., 2010).

The four cases all make the inquiry process explicit to students. nQuire and STOCHASMOS provide students with specific inquiry steps to help students identify inquiry processes. WISE is similar to nQuire and STOCHASMOS in using inquiry steps, but these steps are determined based on principles of knowledge integration. SCY gives students an inquiry mission map to help them identify the inquiry processes embedded in their designs.

ILEs incorporate varied, powerful visualizations to communicate complex scientific ideas. ILEs typically provide students with visualizations of the scientific concepts, but less than half of the ILEs support students as they select inquiry processes. ILEs use distinct approaches to guide students to consider or select appropriate processes for interacting with the visualizations. For example, when interacting with a climate change visualization, students may investigate the overall process of how global climate change occurs or may specifically analyze what happens to solar radiation when greenhouse gases increase in the earth’s atmosphere. Designers grapple with the tradeoff between providing too much guidance (and making the inquiry seem formulaic) and providing too little guidance (so students either skip the visualization entirely or flounder because they cannot determine how to proceed).

Supporting Collaboration With Others

From the 76 studies in this analysis, 30 involved students working individually (39%), 31 involved students collaborating (41%), and 14 (18%) included aspects of students working individually and collaboratively (See Table S5; online only). Eleven of the 30 ILEs include chat features to support student collaboration. One study compared online chat to face-to-face chat (Sins et al., 2011) and found advantages for engagement in face-to-face chat. Many studies built on prior research to design supports that succeeded in orchestrating student interactions (e.g., Hickey & Zuiker, 2012; Zhang & Linn, 2011).

The cases vary in the forms of collaboration they support. nQuire and SCY have both individual and collaborative elements. In nQuire, students collect their own data, but are also encouraged to share their findings with each other. nQuire notes that there is a fine line between finding personally meaningful inquiry topics for students and the potential for embarrassment, that is, some students felt uncomfortable about discussing their diets with others (Anastopoulou et al., 2011). In SCY, students could do certain tasks both individually or in a group, for example, reflect on their approaches, critique and evaluate each other’s emerging ideas, and discuss other aspects of their designs through the chat features (de Jong et al., 2010; de Jong et al., 2012). Students who revisit each other’s ideas and refine their ideas based on input from others are more likely to gain insights. This pattern, which is found in both science learning and science laboratories, can promote conceptual change (Furberg et al., 2013). For STOCHASMOS and WISE, students work collaboratively. Students working with STOCHASMOS share their ideas in a common workspace and also have to reach agreement with their peers on the credibility of their evidence. Students have difficulty engaging in extended collaboration for long-term investigations (Nicolaidou et al., 2011). In WISE, students work in collaborative groups but the ILE does not specifically guide the collaborative process or allow students to keep track of their own ideas. In some cases, researchers report that students collaborating in WISE spontaneously monitor each other’s progress, thus increasing self-regulation (Raes et al., 2012).

In summary, a key aspect of workplace success, collaboration, is incorporated into more than half of the ILE studies. In many cases, collaboration is dictated by limited computer resources rather than designed to take full advantage of the expertise of the students. Collaboration is most effective when scaffolded. Students benefit from tools and tailored content that facilitates the process of learning from each other. Such guided collaboration has proven valuable and could strengthen most ILEs (Makitalo-Siegl et al., 2011).

Developing Autonomous and Metacognitive Learning Practices

A central goal of inquiry instruction concerns promoting autonomous learning so that students can become lifelong learners. It is useful to distinguish between ILEs that primarily add and distinguish ideas, primarily extend and refine ideas, or do all these things (Bell et al., 2010; Etheredge & Rudnitsky, 2003). We characterize the processes the ILE uses to support student decision making in Table 4. (A description of each inquiry process is found in Table S6, online only.) Scaffolding and assessment tools are defining features of ILEs. ILEs strongly support student autonomy and metacognition for inquiry processes such as making hypotheses/predictions, collecting data, analyzing/interpreting, and drawing conclusions. ILEs provide less support for processes such as making an argument, planning, choosing variables, manipulating variables, presenting, and reflecting. Finally, ILEs give little support for inquiry processes such as selecting resources, drawing, graphing, avatar/role playing, and modeling (Table 4).

Table 4

Inquiry processes coded for the inquiry learning environments

	Add and distinguish ideas							Extend and refine ideas
Inquiry learning environments	Make hypothesis or predictions	Collect data	Manipulate variables	Explain science content	Plan an experiment	Choose variables	Select resources	Analyze or interpret	Draw conclusions	Reflect	Present	Make an argument	Graph	Avatar/role play	Model	Draw or annotate	Total
Alien Rescue	•	•	•	•	•	•	•	•	•	•	•	•		•
APoME	•	•	•	•	•	•		•	•		•		•		•		11
Betty’s Brain					•			•		•	•			•			5
Buoyancy	•	•	•		•	•	•	•	•					•			9
CCL	•	•	•		•	•		•	•	•	•		•				10
Co-Lab	•	•	•		•	•		•	•	•	•		•		•	•	12
COTF	•	•						•	•	•	•	•					7
GenScope			•	•		•		•	•			•					6
IMMEX	•	•	•		•	•		•	•	•	•						9
Inquiry Island	•	•	•	•	•	•		•	•	•	•				•		11
IQWST	•	•	•			•		•	•	•	•	•	•		•		11
LensesSim	•	•	•	•				•	•	•							7
MAC	•	•	•	•				•	•	•	•	•	•		•		11
nQuire	•	•	•	•	•	•		•	•	•	•	•	•				12
PEC								•	•								2
PhysHint								•	•	•		•				•	5
Polution Solution	•			•	•			•	•	•	•			•			8
QuestAtlantis	•	•		•	•			•	•	•	•	•		•			10
RiverCity	•	•	•	•	•	•		•	•	•	•	•		•		•	13
SCCR	•	•	•	•	•			•	•	•		•					9
SCY	•	•	•	•	•	•		•	•	•	•	•			•	•	13
SimQuest	•	•	•		•			•	•			•					7
STOCHASMOS	•			•	•			•	•	•	•	•	•			•	10
SURGE	•		•	•				•	•					•			6
Uncommon Scents		•		•				•	•					•			5
Virtual Field Trip		•		•				•	•					•			5
Viten	•	•	•	•	•	•		•	•	•	•	•	•		•		13
Whyville	•	•	•	•				•	•		•			•			8
WiMVT	•	•	•	•	•			•	•	•			•		•	•	11
WISE	•	•	•	•	•	•		•	•	•	•	•	•		•	•	14
Total	24	23	21	20	19	14	2	30	29	21	19	15	10	10	9	7

A number of the ILEs do not scaffold student autonomy and metacognition of important inquiry processes such as planning, argumentation, graphing, and modeling. Scaffolding these inquiry processes can support students in developing important skills necessary for lifelong science learning (and closely align to those used by practicing scientists). Few ILEs give students autonomy in selecting resources or adding variables to investigate. The scaffolding needed to support students in choosing appropriate variables requires further research (Lee, Plass, & Homer, 2006).

All four platforms offer numerous and diverse supports to help students distinguish and refine their ideas: nQuire (12), SCY (13), STOCHASMOS (10), and WISE (14). Each of the ILEs have some missing features such as avatars and role play, graphing, choosing variables, selecting resources, or collecting data. Overall, all the ILEs have inquiry features to support visualizations and many also support student autonomy and metacognition (see Table S7, online only). Only some of the ILEs address the inquiry features of meaningful and relevant science context and collaboration. Fourteen of the 30 ILEs (47%) include features aligned with all four of the inquiry features, including Alien Rescue, Co-lab, and GenScope (Table S7). All the cases address the four inquiry features (nQuire, SCY, STOCHASMOS, WISE). Eight of the 30 ILEs (27%) have three of the inquiry features including APoME, PhysHint, and WiMVT (Table S7). Most of the ILEs support Level 1 or Level 2 inquiry (Blanchard et al., 2010). Some ILEs (e.g., Alien Rescue, nQuire, Whyville) purposefully primarily support aspects of Level 3 inquiry (Blanchard et al., 2010). For these ILEs, teachers have a major responsibility to promote inquiry practices within their classrooms (Pedersen et al., 2009).

Impacts on Student Learning

We address the second research question by reviewing impacts of using the ILEs on student learning. We analyzed all studies that reported data necessary for computing effect sizes (44 out of 76 articles). We look at both pre/post studies and comparison studies. The effect sizes are mostly positive, consistent with reviews of inquiry instruction (e.g., Furtak et al., 2012; Minner et al., 2010; significant effect sizes are bolded in Appendix A). The studies show learning gains in conceptual understanding resulting from inquiry instruction, with an average weighted effect size of 0.87 for pre/post studies. Studies show value of inquiry for a broad range of science topics: air quality, gas laws, kinetic molecular theory, motion, global climate change, and genetics. Three studies (Gerard et al., 2010; Krajcik et al., 2008; Svihla & Linn, 2012) demonstrate the value of iterative refinements for improving outcomes from inquiry instruction. In these studies, the effect sizes increased in the second and/or third year after designers or teachers used evidence from earlier studies to refine instruction.

The comparison studies show mostly favorable results for the interventions (see effect sizes in Appendix B). Features of successful interventions include the value of a teachable agent (students teach a computer agent) for students over having their own avatar, collaboration over individual student work, and dynamic visualizations over nondynamic visualizations. For example, a comparison study on the topic of photosynthesis showed that dynamic visualizations can illustrate dynamic scientific processes that static images struggle to communicate, such as the transformation of carbon dioxide and water in a plant’s chloroplast to form glucose and oxygen (Ryoo & Linn, 2012). A negative effect size was recorded for online chat versus a face-to-face condition probably because students in the face-to-face condition exchanged more comments (Sins et al., 2011).

Each of the comparison studies have also been coded for the four successful inquiry features. The comparison studies mostly focus on using powerful visualizations or developing autonomous students (Appendix B). Thirteen of the 26 comparison studies focus on some aspect of visualization and 9 of the 26 comparison studies have some focus on student autonomy. The average weighted effect size for visualization is 0.59 and for autonomy it is 0.41 based on these studies. For exploring meaningful and authentic science and collaboration, there are two studies each (Appendix B). The average weighted effect size for meaningful and authentic science is 0.32 and for collaboration it is 0.67. Of the four cases, three reported effect sizes that demonstrate value for inquiry learning: nQuire, STOCHASMOS, and WISE (see Appendix A and Appendix B). WISE research also shows that with iterative refinements, effect sizes increase.

Teacher Infrastructural and Implementation Features

The findings have so far explored the inquiry features embedded within ILEs and illustrated positive learning gains for ILE-based interventions. However, a crucial factor that underpins both short-term and long-term success of precollege ILEs is the contribution of the teacher (Furberg et al., 2013; Gerard et al., 2010; Krajcik & Blumenthal, 2006). Teachers benefit from infrastructural supports to implement inquiry. Half of the 30 ILEs have multiple units and one third have grading (see Table S8, online only). Less common infrastructural features across the ILEs are authoring (8), automated guidance/feedback (10), progress monitoring (2), and flagging of student work (4; Table S8). The four cases vary in how many of these six infrastructural features they include: nQuire (2), SCY (5), STOCHASMOS (4), and WISE (6; Table S8). WISE and SCY include all the teacher infrastructural features, except SCY does not have progress monitoring. STOCHASMOS does not include automated feedback or progress monitoring, and nQuire does not incorporate grading, automated feedback, progress monitoring, and flagging student work. The absence of grading and automated feedback in some of the platforms could deter designers from using them for authoring new units.

We found that most research on ILEs depends on implementations conducted by novice users who were using the ILE for the first time (see Table S9, online only). Twenty of the ILEs (66%) had studies that involved implementation for less than 1 year, six ILEs lasted for 2 years, and four ILEs had studies more than 2 years. Eight ILEs showed some aspects of iterative refinement based on student learning outcomes. We grouped the average effect sizes in the pre/post learning gains (Appendix A) based on studies that lasted 1 year or less and studies that lasted longer than 1 year. Studies that lasted 1 year or less had a weighted average effect size of 0.69 (p < .001, 95% CI [0.614, 0.765, fixed; 0.244, 1.128, random]), and studies that lasted over a year had a weighted average effect size of 0.96 (p < .001, 95% CI [0.904, 1.011, fixed; 0.734, 0.982, random]).

Among the case studies, nQuire, SCY, and STOCHASMOS typically involved teachers with less than 1 year experience of using the platform. The nQuire study involved one teacher and it took place over a semester. nQuire held many informal meetings with the teacher where they discussed inquiry in detail (Anastopoulou et al., 2011). In SCY, the teachers collaborated with the research team on the design of the environment, including the content, pedagogy, and planning for design research in the schools (Dolonen & Ludvigsen, 2012). For one STOCHASMOS study, the teacher in the study had 16 years experience, but was teaching an extended ILE investigation for the first time (Kyza et al., 2011). In another study, the teacher had 17 years experience, but had no experience teaching with inquiry methods (Nicolaidou et al., 2011). The many teachers using WISE participated in longitudinal studies occurring over 2 years or more. Most participated in some form of professional development.

In summary, most of the research on ILEs involves testing innovations with teachers new to inquiry. Some studies include trial and refinement over several years. These studies typically involve researchers guiding teachers in the classroom. Follow-up and scaling studies are needed to determine the most effective ways to sustain and improve the use of inquiry materials.

Discussion

Fostering Learning in the Networked World (Borgman et al., 2008) envisions a common and open platform that can readily evolve to encompass emerging technologies such as automated scoring, educational games, new visualizations, and so on. Such technologies can build on and improve existing ILEs and also support the development of new units within ILE platforms (Krajcik et al., 2008; McElhaney & Linn, 2011; Svihla & Linn, 2012). Evidence from the studies in this review show research-based aspects of inquiry help students learn science in ILEs: meaningful and authentic science, powerful visualizations, collaboration, and autonomy/meta-cognition.

The ILEs use a plethora of ways to structure student inquiry including a variety of cycles, steps, and maps. The structuring approaches are often quite precise, making inquiry seem more systematic than it is in scientific labs (Furtak et al., 2012; Windschitl, 2004). More research is needed to determine the progress students make in understanding inquiry practices. Developing a common, flexible, and abstracted approach for structuring inquiry could reinforce the cumulative impact of courses and topics and strengthen student understanding of the scientific practices called for in the Next Generation Science Standards (NRC, 2011).

More research exploring the tradeoff between too much guidance (thus making science appear formulaic) and too little guidance (thus encouraging superficial or mindless explorations) is needed. Students benefit from flexible, abstract frameworks that can be used for both classroom and personal inquiry activities. WISE, for example, uses the knowledge integration pattern in organizing inquiry both within and across units. Students experience multiple ways to make predictions; gather evidence from visualizations, virtual experiments, or hands on experiments; distinguish between predictions and new evidence; and reflect on their experiences by building an argument (Linn & Eylon, 2011).

Based on our study, researchers investigating ILEs can contribute to understanding by conducting more sustained implementations of ILEs. Only 10 of the 30 ILEs had studies that lasted over a year in teacher implementation. This finding is consistent with a review of professional development literature for inquiry teaching, which indicates that successful professional development programs require at least 2 years but that few investigations involve that level of support (Gerard et al., 2011). Given the complex and broad nature of many units delivered by ILEs, short-term studies can place unrealistic expectations on teachers. As was highlighted in the case studies, teachers may need to learn how to use both the technology and inquiry methods. Thus, studies using the 10 ILEs that lasted over a year and focused on iterative refinement showed increased weighted average effect sizes, suggesting the advantage of sustained professional development. However, further short-term and longitudinal studies are needed. This research also shows that iterative refinement based on student and teacher feedback improves both the curriculum and inquiry teaching strategies. Studies with experienced teachers who are familiar with the ILE make it easier to understand the impact of improvements to the curriculum.

The NSF report (Borgman et al., 2008) encourages researchers to consolidate their efforts and build on the work of others rather than reinventing resources that exist. Our study demonstrates progress on developing common platforms in that 16 of the 30 ILEs have multiple curriculum units. However, less than half of the ILEs have two or more infrastructural features such as authoring, grading, or feedback, and only a small handful have progress monitoring or flagging of student work. The small number of technology features in most ILEs is consistent with the significant investment ILE platforms require to develop. We hope that this review will help teachers and researchers new to ILEs identify platforms that they can build on rather than starting anew if unnecessary. In this way, users can focus on strengthening ILEs.

Developing flexible platforms can achieve the vision of the NSF report and, more important, meet the needs of teachers. Teachers seek flexible yet durable materials that can be readily assimilated into their classroom practice (Cuban, 1986). Teachers can see the benefit of investing in learning how to use an ILE if there are multiple units, if units are customizable, and if developers add features that they desire (such as support for tablets and mobile computing).

Teachers desire platforms that support customization of curricula to their own needs. At the same time, researchers are commonly concerned with maintaining the integrity of ILEs (Barab & Luehmann, 2003; Plesch, Kaendler, Rummel, Wiedmann, & Spada, 2013; Squire, MaKinster, Barnett, Luehmann, & Barab, 2003). We found eight ILEs that support customization or authoring for researchers or teachers. For these ILEs, teachers can use evidence from their students’ work to customize units and align instruction with classroom needs (Svihla & Linn, 2012). Researchers and teachers can collaborate to build on evidence from student work and maintain the integrity of inquiry instruction. Teachers need to be equal players in testing and refining units. Researchers need to welcome teachers as partners and take advantage of significant teacher expertise as they refine ILEs to adjust to changing classroom, student, policy, and institutional demands.

In summary, the pre/post and comparison studies demonstrate the impact of ILEs on student learning. Furthermore, these results show that students using ILEs in typical classrooms can gain deep and coherent understanding of science concepts. At the same time, these studies show that better technology integration and more effective professional development is needed to increase the number of students who benefit from these technologies.

Implications for Future Design and Research

Aligning with calls from the Fostering Learning in the Networked World report, researchers and designers of ILEs should seriously consider the extension of rather than recreation of existing, open source, and evidence-based ILE platforms. Researchers should first consider existing open source and evidence-based ILE platforms (see Table S4, online only), such as those that have large userbases (e.g., Questatlantis, Whyville, and WISE). Students, teachers, and researchers benefit when new tools build on familiar platforms and take advantage of tested and useful existing features. Investment in such an approach increases the sustainability of valuable platforms and allows designers to focus on adding their innovations rather than reinventing things that already work. Of course, creative innovations must not be constrained by existing platforms. Balancing innovation and alignment with existing platforms is necessary in allocating resources to support a vibrant, creative research community and strengthen opportunities for student learning.

WISE, for example, in 2014 supported a community of more than 9,000 registered teachers and 76,000 registered students. Collaborators continuously add new features to WISE, for example, the NetLogo project uses WISE for curriculum (Sengupta & Wilensky, 2011). WISE also has over seven instances of its platform in the United States, Taiwan, Argentina, and other countries. The studies in this article by international researchers not directly affiliated with the WISE group attest to the successful use of existing platforms to support new teachers and researchers.

ILE platforms can enable researchers and designers to investigate powerful questions concerning the impact of instruction on students’ cumulative learning in science and increase the validity of their results. Studies can compare representations of science phenomena, ways to track student progress over time, methods for providing personalized and automated guidance to students, new forms of assessments, and ways to provide guidance to teachers. Currently, the studies are thinly spread across many ILEs with varied technology supports. Converging on a unifying platform could ensure that all studies take advantage of established supports and make syntheses of findings more robust.

Studies are needed to broaden the range of inquiry assessments. Many ILEs support multiple-choice questions, but few take advantage of more sophisticated inquiry assessments such as concept mapping, drawings, or essays. ILEs can feature embedded assessments that also serve as learning opportunities (Hickey et al., 2012). Research and development is needed to show how these items can become valid and reliable options for high-stakes assessment. There is also a need for greater research on matching ILEs to student needs and capabilities (Wu, 2010). Within ILEs, offering alternative media elements (video, animations, drawings, online chat, etc.) may allow for customization to student needs (Zydney & Grincewicz, 2011).

Researchers can increase the impact of ILEs by strengthening teacher professional development (Yeh et al., 2012). Researchers and teachers can collaborate to use evidence from classroom learning to improve inquiry scaffolds. Helping teachers develop criteria based on research for designing and evaluating inquiry teaching and learning could lead to productive customizations of units for specific users. Building and sustaining a research program focused on developing criteria for effective inquiry instruction is especially important given the NGSS focus on sustained investigations and cumulative learning. Future studies can test and refine the design principles and features noted in the ILEs we reviewed and connect them to achievement of the NGSS.

Conclusion

In reviewing the characteristics of contemporary ILEs, we identified many highly creative and effective ILEs. ILEs typically enhance student learning, but greater collaboration, cumulative development, and comparison studies on ILEs are critically needed. There is a paucity of research on how ILEs affect student learning over consecutive years. More studies of sustained use of ILEs by teachers could enhance design of curricula as well as inform professional developers. However, a cumulative understanding of design principles that support inquiry is emerging (Kali, 2006; Quintana et al., 2004). We drew on four components of inquiry that were widely supported in the literature: meaningful and authentic science, powerful visualizations, collaboration, and autonomous, metacognitive learners. These four components were evident across many of the ILEs. They would benefit from more precise characterizations to help researchers make informed design and implementation decisions. For example, specific guidelines for designing instruction that promotes student autonomy are needed. Most ILEs align with the suggestions of Quintana and others to provide guidance (either on demand or in the form of questions) but do not help designers resolve the tradeoffs between too much and too little guidance.

The role of the teacher within ILEs needs more attention. Some ILEs leave much of the guidance to the teacher, giving teachers few guidelines (Furberg et al., 2013; Krange & Ludvigsen, 2008). Most ILEs lack supports for teachers that take full advantage of embedded assessments of student work. Professional development research shows that teachers need time to develop skill in implementing inquiry (Gerard et al., 2011). These findings support the idea that teachers can develop effective monitoring and guiding skills when they implement inquiry activities such as those supported by most of the ILEs and refine their practices based on logged student work. As teachers develop these skills, they become prepared to enhance inquiry opportunities across the curriculum.

Footnotes

Appendix

Appendix B

Comparison studies effect sizes for the ILEs

ILE (author)	Grade	n	Nature of comparison (exp./cont.)	Hedge’s g	Type
Co-Lab—van Borkulo et al., 2012	11	74	Modeling group vs. expository group (n = 38/36)	0.40 (p = .089)	V
MAC—Stieff, 2011	10–12	431	Connected Chemistry Curriculum (visualization tools) vs. Lecture-based methods (n = 217/214)	0.25 (p < .001)	V
PhysHint—Pol et al., 2009	10	59	Immediacy of feedback—During/After (DA) vs. control (n = 18/23)	0.81 (NR)	V
				Immediacy of feedback—After (A) vs. control (n = 18/23)	0.41 (p < .05)
Questatlantis—Hickey et al., 2009	6	116	Virtual environment vs. text-book approach (n = 54/62)—Proximal Test (PT) and Distal Test (DT)	0.72 (PT) (p < .0001)	V
				0.89 (DT) (p = .003)
Questatlantis—Barab et al., 2012	7	65	Game based (GB) vs. story based (SB) (n = 33/32)	0.46 (p < .001)	V
SCCR—She & Lee, 2008	6	61	SCCR intervention in topic of combustion vs. control—Combustion Achievement Test (CAT), Scientific Reasoning Test (SRT), Combustion Dependent Reasoning Test (CDR) (n = 31/30)	1.51 (CAT) (p < .0001)	V
				0.32 (SRT) (p < .01)
				0.93 (CDR); (p < .0001)
STOCHASMOS—Kyza et al., 2011	6	53	Workspace vs. PowerPoint condition (n = 13/13 groups)—Total explanation	0.75 (p = .06)	V
STOCHASMOS—Nicolaidou et al., 2011	11	25	Biotechnology intervention vs. typical curriculum (n = 12/13)	1.07 (p < .01)	V
WISE—Clark et al., 2012	8	50	Spanish-English support vs. English only support (n = 24/26)	0.54 (p = .065)	V
WISE—Ryoo & Linn, 2012	7	167	Dynamic vs. static visualization (n = 81/86)	0.43 (p < .01)	V
WISE—Shen & Linn, 2011	HS	79	Experienced students vs. typical students (n = 38/41)	0.79 (p < .001)	V
WISE—Svihla & Linn, 2012	6	186	Year 1 curriculum (GCC1) vs. Year 2 curriculum (GCC2)—Global climate change (n = 119/67)	0.43 (p < .05)	V
WISE—Zhang & Linn, 2011	8	133	Drawing generation vs. simulation interaction (n = 81/52)	1.57 (p < .0001)	V
Alien Rescue—Bulu & Pedersen, 2010	6	267	Scaffolding conditions—Specific vs. general domain (n = 127/140)	0.62 (p < .001)	Au
				Continuous scaffold vs. Faded scaffold (Domain specific, n = 74/66)	0.85 (p < .001)
				Continuous scaffold vs. Faded scaffold (Domain general, n = 70/57)	0.4 (p > .79)
Betty’s Brain—Chase et al., 2009	8	62	Students with Teachable Agent (TA) vs. Avatar (Av)—TA spend more time learning (n = 31/31)	1.15 (p < .001)	Au
Co-Lab—Sins et al., 2011	11	44	Chat vs. face-to-face communication (n = 11/11) dyads—Number of process episodes	−4.72 (p < .001)	Au
				Surface processes	−1.93 (p < .001)
				Deep processes	0.19 (p < .72)
PhysHint—Pol et al., 2008	11–12	37	Problem solving: PhysHint vs. control (n = 11/26)	0.49 (NR)	Au
SimQuest—Gijlers & de Jong, 2009	10	66	Shared Proposition Table Scaffold vs. Control (n = 22/22)—Proposition test	0.58 (p < .01)	Au
				Shared Scratchpad Scaffold vs. Control (n = 22/22)—Proposition Test	−0.25 (NR)
WISE—Lee et al., 2011	MS and HS	4,745	Inquiry vs. typical instruction (n = 2,685/2,060)	0.34 (p < .001)	Au
WISE—Makitalo-Siegl et al., 2011	HS	108	Help seeking (Dyads)—Low scripted (n = 35) vs. High scripted (n = 19)	0.85 (p < .01)	Au
Co-Lab—Mulder et al., 2012	10	91	Comparison of model progression for different restrictions		Au
			Restricted variables (n = 20) vs. control (n = 30)
			Performance Success—Variables	1.13 (p < .001)
			Performance Success—Relations	0.86 (p < .001)
			Semirestricted variables (n = 19) vs. control (n = 30)
			Performance Success—Variables	0.57 (p < .001)
			Performance Success—Relations	1.07 (p < .001)
			Unrestricted (n = 22) vs. control (n = 30)
			Performance Success—Variables	1.48 (p < .001)
			Performance Success—Relations	1.52 (p < .001)
LensesSim—Chang et al., 2008	11	384	Simulation with experiment prompting vs. control (n = 39/39)	0.74 (p < .05)	Au
				Simulation with hypothesis menu vs. control (n = 40/39)	1.05 (p < .05)
				Simulation with step guidance vs. control (n = 35/39)	0.63 (p <. 05)
				Experiment prompting vs. step guidance (n = 78/74)	0.28 (p < .05)
				Hypothesis menu vs. step guidance (n = 79/74)	0.49 (p < .05)
nQuire—Anastopoulou et al., 2011	10	36	nQuire students vs. control condition (n = 21/15):		Me
				Attitudes to scientific inquiry (ASI)	−0.20 (ASI) (NR)
				Adoption of scientific attitudes (ASA)	0.44 (ASA) (NR)
				Leisure interest in science (LIS)	−0.35 (LIS) (NR)
				Enjoyment of science lessons (ESL)	0.86 (ESL) (p < .02)
PEC—Yeh et al., 2012	10	153	Considering Prior Knowledge vs. not Considering Prior Knowledge (n = 78/75)		Me
				Lower cognitive load in learning process (CLLP)	0.48 (CLLP) (p = .005)
				Curriculum operation (CO)	0.28 (CO) (p = .09)
				Total cognitive load (TCL)	0.44 (TCL) (p = .008)
				Science conception learning outcome (SCL)	0.18 (SCL) (p = .27)
				Open-response question (OQ)	0.36 (OQ) (p = .04)
Co-Lab—Manlove et al., 2009	10–12	42	Pairs (n = 12) versus individuals (n = 18)		C
				Elaborate Laboratory Report Content	0.80 (p < .05)
				Better Models	1.05 (p < .01)
SimQuest—Kolloffel et al., 2011	9	215	Collaborative dyads (n = 60) vs. individual (n = 95) students’ conceptual understanding	0.57 (p < .001)	C

Note. NR = not reported; V = visualization; Au = autonomy; Me = meaningful context; C = collaboration.

Notes

Authors

DERMOT F. DONNELLY is a postdoctoral researcher in the Graduate School of Education at the University of California, Berkeley, 4407 Tolman Hall, Berkeley, CA 94720, USA; e-mail: dfdonnelly@berkeley.edu . His research interests are in the use of technology to support inquiry-based practices, in particular how technology can support valid and reliable assessments that encompass automated scoring. He was a EUOSSIC (European Union-Oceanic Social Sciences Interregional Consortium) fellow in New Zealand in 2011 and was an IRC (Irish Research Council) scholar for his PhD studies. His PhD studied the use of a virtual chemistry laboratory to support inquiry-based practices in Irish secondary schools.

MARCIA C. LINN is a professor of cognition and development specializing in education in mathematics, science, and technology in the Graduate School of Education at the University of California, Berkeley, 4407 Tolman Hall, Berkeley, CA 94720, USA; e-mail: mclinn@berkeley.edu . She is a member of the National Academy of Education and a Fellow of the American Association for the Advancement of Science, the American Psychological Association, and the Association for Psychological Science. She has served on the American Association for the Advancement of Science (AAAS) board, as Chair of the AAAS Education Section, and as President of the International Society of the Learning Sciences.

STEN LUDVIGSEN is a professor at the University of Oslo, Oslo, Norway; e-mail: s.r.ludvigsen@medisin.uio.no . He has performed studies in educational institutions and work settings based on a cultural-historical activity theory and sociocultural perspective. His long-term interest is in the relation between social interaction, cognition, and representations and how people change their understanding of concepts (in specific domains). He has specialized in research on the use of digital knowledge representations and the relationship between co-located and distributed settings. He has been involved in the leadership of Network of Excellence and Integrated Projects and national research projects. He was director and professor at InterMedia, an interdisciplinary research center at the University of Oslo from 2004 to 2009. He is the former scientific leader of NATED, the National Graduate Research School in Educational Science in Norway.

References

*Anastopoulou

Sharples

Ainsworth

Crook

O’Malley

Wright

(2011). Creating personal meaning through technology-supported science inquiry learning across formal and informal settings. International Journal of Science Education, 34, 251–273. doi:10.1080/09500693.2011.569958#.UmR7U3CsiSo

Anderson

R. D.

(2007). Inquiry as an organizing theme for science education. In Abell

S. K.

Lederman

N. G.

(Eds.), Handbook of research on science education (pp. 807–830). Mahwah, NJ: Erlbaum.

Anderson

R. D.

Helms

J. V.

(2001). The ideal of standards and the reality of schools: Needed research. Journal of Research in Science Teaching, 38, 3–16. doi:10.1002/1098-2736(200101)38:1<3::AID-TEA2>3.0.CO;2-V

Barab

S. A.

Luehmann

A. L.

(2003). Building sustainable science curriculum: Acknowledging and accommodating local adaptation. Science Education, 87, 454–467. doi:10.1002/sce.10083

Barab

Makinster

Moore

Cunningham

(2001). Designing and building an on-line community: The struggle to support sociability in the inquiry learning forum. Educational Technology Research and Development, 49(4), 71–96. doi:10.1007/BF02504948

*Barab

Pettyjohn

Gresalfi

Volk

Solomou

(2012). Game-based curriculum and transformational play: Designing to meaningfully positioning person, content, and context. Computers & Education, 58, 518–533. doi:10.1016/j.compedu.2011.08.001

Bell

Gess-Newsome

Luft

(2008). Technology in the secondary science classroom. Arlington, VA: NSTA Press.

Bell

Urhahne

Schanze

Ploetzner

(2010). Collaborative inquiry learning: Models, tools, and challenges. International Journal of Science Education, 32, 349–377. doi:10.1080/09500690802582241

*Berland

L. K.

(2011). Explaining variation in how classroom communities adapt the practice of scientific argumentation. Journal of the Learning Sciences, 20, 625–664. doi:10.1080/10508406.2011.591718

10.

*Berland

L. K.

Lee

V. R.

(2012). In pursuit of consensus: Disagreement and legitimization during small-group argumentation. International Journal of Science Education, 34, 1857–1882. doi:10.1080/09500693.2011.645086

11.

Blanchard

Southerland

Osborne

Sampson

Annetta

Granger

(2010). Is inquiry possible in light of accountability? A quantitative comparison of the relative effectiveness of guided inquiry and verification laboratory instruction. Science Education, 94, 577–616. doi:10.1002/sce.20390

12.

Borgman

C. L.

Abelson

Dirks

Johnson

Koedinger

K. R.

Linn

M. C.

. . . Szalay

(2008). Fostering learning in the networked world: The cyberlearning opportunity and challenge. A 21st century agenda for the National Science Foundation. Arlington, VA: National Science Foundation. Retrieved from http://www.nsf.gov/pubs/2008/nsf08204/nsf08204.pdf

13.

*Bravo

Van Joolingen

W. R.

De Jong

(2009). Using co-lab to build system dynamics models: Students’ actions and on-line tutorial advice. Computers & Education, 53, 243–251. doi:10.1016/j.compedu.2009.02.005

14.

*Bulu

S. T.

Pedersen

(2010). Scaffolding middle school students’ content knowledge and ill-structured problem solving in a problem-based hypermedia learning environment. Educational Technology Research and Development, 58, 507–529. doi:10.1007/s11423-010-9150-9

15.

*Chang

K.-E.

Chen

Y.-L.

Lin

H.-Y.

Sung

Y.-T.

(2008). Effects of learning support in simulation-based physics learning. Computers & Education, 51, 1486–1498. doi:10.1016/j.compedu.2008.01.007

16.

*Chase

Chin

Oppezzo

Schwartz

(2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18, 334–352. doi:10.1007/s10956-009-9180-4

17.

Chinn

C. A.

Hmelo-Silver

C. E.

(2002). Authentic inquiry: Introduction to the special section. Science Education, 86, 171–174. doi:10.1002/sce.10000

18.

*Clark

D. B.

Nelson

B. C.

Chang

H.-Y.

Martinez-Garza

Slack

D’Angelo

C. M.

(2011). Exploring Newtonian mechanics in a conceptually-integrated digital game: Comparison of learning and affective outcomes for students in Taiwan and the United States. Computers & Education, 57, 2178–2195. doi:10.1016/j.compedu.2011.05.007

19.

*Clark

D. B.

Sampson

(2008). Assessing dialogic argumentation in online environments to relate structure, grounds, and conceptual quality. Journal of Research in Science Teaching, 45, 293–321. doi:10.1002/tea.20216

20.

*Clark

D. B.

Touchman

Martinez-Garza

Ramirez-Marin

Skjerping Drews

(2012). Bilingual language supports in online science inquiry environments. Computers & Education, 58, 1207–1224. doi:10.1016/j.compedu.2011.11.019

21.

*Clarke

Dede

(2009). Design for scalability: A case study of the River City curriculum. Journal of Science Education and Technology, 18, 353–365. doi:10.1007/s10956-009-9156-4

22.

Crawford

(2007). Learning to teach science as inquiry in the rough and tumble of practice. Journal of Research in Science Teaching, 44, 613–642. doi:10.1002/tea.20157

23.

*Cross

Taasoobshirazi

Hendricks

Hickey

D. T.

(2008). Argumentation: A strategy for improving achievement and revealing scientific identities. International Journal of Science Education, 30, 837–861. doi:10.1080/09500690701411567

24.

Cuban

(1986). Teachers and machines: The classroom use of technology since 1920. New York, NY: Teachers College Press.

25.

Cuevas

Lee

Hart

Deaktor

(2005). Improving science inquiry with elementary students of diverse backgrounds. Journal of Research in Science Teaching, 42, 337–357. doi:10.1002/tea.20053

26.

Dede

(2010). Technological supports for acquiring 21st century skills. In Baker

McGaw

Peterson

(Eds.), International encyclopedia of education (3rd ed.). New York, NY: Elsevier.

27.

Dede

Richards

(2012). Digital teaching platforms: Customizing classroom learning for each student. New York, NY: Teachers College Press.

28.

*de Jong

Van Joolingen

W. R.

Giemza

Girault

Hoppe

Kindermann

. . . Van Der Zanden

(2010). Learning by creating and exchanging objects: The SCY experience. British Journal of Educational Technology, 41, 909–921. doi:10.1111/j.1467-8535.2010.01121.x

29.

*de Jong

Weinberger

Girault

Kluge

Lazonder

Pedaste

. . . Zacharia

(2012). Using scenarios to design complex technology-enhanced learning environments. Educational Technology Research and Development, 60, 883–901. doi:10.1007/s11423-012-9258-1

30.

*Ding

(2009). Visualizing the sequential process of knowledge elaboration in computer-supported collaborative problem solving. Computers & Education, 52, 509–519. doi:10.1016/j.compedu.2008.10.009

31.

*Ding

Bosker

R. J.

Harskamp

E. G.

(2011). Exploring gender and gender pairing in the knowledge elaboration processes of students using computer-supported collaborative learning. Computers & Education, 56, 325–336. doi:10.1016/j.compedu.2010.06.004

32.

Dolonen

J. A.

Ludvigsen

S. R.

(2012). Analyzing students’ interaction with a 3D geometry learning tool and their teacher. Learning, Culture and Social Interaction, 1, 167–182. doi:10.1016/j.lcsi.2012.08.002

33.

Donnelly

O’Reilly

McGarr

(2013). Enhancing the student experiment experience: Visible scientific inquiry through a virtual chemistry laboratory. Research in Science Education, 43, 1571–1592. doi:10.1007/s11165-012-9322-1

34.

*Dori

Y. J.

Sasson

(2008). Chemical understanding and graphing skills in an honors case-based computerized chemistry laboratory environment: The value of bidirectional visual and textual representations. Journal of Research in Science Teaching, 45(2), 219–250. doi:10.1002/tea.20197

35.

Edelson

D. C.

(2001). Learning-for-use: A framework for integrating content and process learning in the design of inquiry activities. Journal of Research in Science Teaching, 38, 355–385. doi:10.1002/1098-2736(200103)38:3<355::AID-TEA1010>3.0.CO;2-M

36.

Edelson

D. C.

Gordin

D. N.

Pea

R. D.

(1999). Addressing the challenges of inquiry-based learning through technology and curriculum design. Journal of the Learning Sciences, 8, 391–450. doi:10.1080/10508406.1999.9672075

37.

*Eslinger

White

Frederiksen

Brobst

(2008). Supporting inquiry processes with an interactive learning environment: Inquiry Island. Journal of Science Education and Technology, 17, 610–617. doi:10.1007/s10956-008-9130-6

38.

Etheredge

Rudnitsky

(2003). Introducing students to scientific inquiry. San Francisco, CA: Pearson Education.

39.

*Feldon

D. F.

Kafai

Y. B.

(2008). Mixed methods for mixed reality: understanding users’ avatar activities in virtual worlds. Educational Technology Research and Development, 56, 575–593. doi:10.1007/s11423-007-9081-2

40.

*Furberg

Kluge

Ludvigsen

(2013). Student sensemaking with science diagrams in a computer-based setting. International Journal of Computer-Supported Collaborative Learning, 8, 41–64. doi:10.1007/s11412-013-9165-4

41.

Furtak

E. M.

Seidel

Iverson

Briggs

D. C.

(2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research, 82, 300–329. doi:10.3102/0034654312457206

42.

Gerard

L. F.

Linn

M. C.

(2013, April). What kind of feedback improves understanding? A comparison of knowledge integration and generic feedback on students’ scientific explanations. Paper presented at the annual meeting of the American Education Research Association, San Francisco, CA.

43.

*Gerard

L. F.

Spitulnik

Linn

M. C.

(2010). Teacher use of evidence to customize inquiry science instruction. Journal of Research in Science Teaching, 47, 1037–1063. doi:10.1002/tea.20367

44.

Gerard

L. F.

Varma

Corliss

S. B.

Linn

M. C.

(2011). Professional development for technology-enhanced inquiry science. Review of Educational Research, 81, 408–448. doi:10.3102/0034654311415121

45.

*Gijlers

De Jong

(2009). Sharing and confronting propositions in collaborative inquiry learning. Cognition and Instruction, 27, 239–268. doi:10.1080/07370000903014352

46.

*Gobert

J. D.

O’Dwyer

Horwitz

Buckley

B. C.

Levy

S. T.

Wilensky

(2011). Examining the relationship between students’ understanding of the nature of models and conceptual learning in biology, physics, and chemistry. International Journal of Science Education, 33, 653–684. doi:10.1080/09500691003720671

47.

*Hansson

Redfors

Rosberg

(2011). Students’ socio-scientific reasoning in an astrobiological context during work with a digital learning environment. Journal of Science Education and Technology, 20, 388–402. doi:10.1007/s10956-010-9260-5

48.

*Hickey

Ingram-Goble

Jameson

(2009). Designing assessments and assessing designs in virtual educational environments. Journal of Science Education and Technology, 18, 187–208. doi:10.1007/s10956-008-9143-1

49.

*Hickey

D. T.

Taasoobshirazi

Cross

(2012). Assessment as learning: Enhancing discourse, understanding, and achievement in innovative science curricula. Journal of Research in Science Teaching, 49, 1240–1270. doi:10.1002/tea.21056

50.

*Hickey

D. T.

Zuiker

S. J.

(2012). Multilevel assessment for discourse, understanding, and achievement. Journal of the Learning Sciences, 21, 522–582. doi:10.1080/10508406.2011.652320

51.

Hill

Hannafin

(2001). Teaching and learning in digital environments: The resurgence of resource-based learning. Educational Technology Research and Development, 49, 37–52. doi:10.1007/BF02504914

52.

Hmelo-Silver

C. E.

(2006). Design principles for scaffolding technology based inquiry. In O’Donnell

A. M.

Hmelo-Silver

C. E.

Erkens

(Eds.), Collaborative reasoning, learning and technology (pp. 147–170). Mahwah, NJ: Erlbaum.

53.

Hmelo-Silver

Duncan

Chinn

(2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42, 99–107. doi:10.1080/00461520701263368

54.

Hodson

(1988). Toward a philosophically more valid science curriculum. Science Education, 72, 19–40. doi:10.1002/sce.3730720103

55.

Höffler

T. N.

Leutner

(2007). Instructional animation versus static pictures: A meta-analysis. Learning and Instruction, 17, 722–738. doi:10.1016/j.learninstruc.2007.09.013

56.

*Hsieh

Cho

Liu

Schallert

D. L.

(2008). Examining the interplay between middle school students’ achievement goals and self-efficacy in a technology-enhanced learning environment. American Secondary Education, 36(3), 33–50.

57.

*Kafai

(2008). Understanding virtual epidemics: Children’s folk conceptions of a computer virus. Journal of Science Education and Technology, 17, 523–529. doi:10.1007/s10956-008-9102-x

58.

Kali

(2006). Collaborative knowledge-building using the Design Principles Database. International Journal of Computer Support for Collaborative Learning, 1, 187–201. doi:10.1007/s11412-006-8993-x

59.

Kim

M. C.

Hannafin

M. J.

Bryan

L. A.

(2007). Technology-enhanced inquiry tools in science education: An emerging pedagogical framework for classroom practice. Science Education, 91, 1010–1030. doi:10.1002/sce.20219

60.

*Klisch

Miller

Wang

Epstein

(2012). The impact of a science education game on students’ learning and perception of inhalants as body pollutants. Journal of Science Education and Technology, 21, 295–303. doi:10.1007/s10956-011-9319-y

61.

*Kolloffel

Eysink

T. S.

Jong

(2011). Comparing the effects of representational tools in collaborative and individual inquiry learning. International Journal of Computer-Supported Collaborative Learning, 6, 223–251. doi:10.1007/s11412-011-9110-3

62.

Kozma

(2003). The material features of multiple representations and their cognitive and social affordances for science understanding. Learning and Instruction, 13, 205–226. doi:10.1016/S0959-4752(02)00021-X

63.

Krajcik

Blumenfeld

(2006). Project-based learning. In Sawyer

R. K.

(Ed.), The Cambridge handbook of the learning sciences (pp. 317–334). New York, NY: Cambridge University Press.

64.

*Krajcik

McNeill

K. L.

Reiser

B. J.

(2008). Learning-goals-driven design model: Developing curriculum materials that align with national standards and incorporate project-based pedagogy. Science Education, 92, 1–32. doi:10.1002/sce.20240

65.

*Krange

Ludvigsen

(2008). What does it mean? Students’ procedural and conceptual problem solving in a CSCL environment designed within the field of science education. International Journal of Computer-Supported Collaborative Learning, 3, 25–51. doi:10.1007/s11412-007-9030-4

66.

*Künsting

Wirth

Paas

(2011). The goal specificity effect on strategy use and instructional efficiency during computer-based scientific discovery learning. Computers & Education, 56, 668–679. doi:10.1016/j.compedu.2010.10.009

67.

*Kyza

E. A.

Constantinou

C. P.

Spanoudis

(2011). Sixth graders’ co-construction of explanations of a disturbance in an ecosystem: Exploring relationships between grouping, reflective scaffolding, and evidence-based explanations. International Journal of Science Education, 33, 2489–2525. doi:10.1080/09500693.2010.550951

68.

Lee

Plass

J. L.

Homer

B. D.

(2006). Optimizing cognitive load for learning from computer-based science simulations. Journal of Educational Psychology, 98, 902–913. doi:10.1037/0022-0663.98.4.902

69.

Lee

S. W.

Tsai

Liu

Hwang

. . .Chang

(2011). Internet-based science learning: A review of journal publications. International Journal of Science Education, 33, 1893–1925. doi:10.1080/09500693.2010.536998

70.

*Levy

(2012). How dynamic visualization technology can support molecular reasoning. Journal of Science Education and Technology, 22, 702–717. doi:10.1007/s10956-012-9424-6

71.

*Levy

Wilensky

(2009). Students’ learning with the Connected Chemistry (CC1) Curriculum: Navigating the complexities of the particulate world. Journal of Science Education and Technology, 18, 243–254. doi:10.1007/s10956-009-9145-7

72.

Lewis

(2006). Design and inquiry: Bases for an accommodation between science and technology education in the curriculum? Journal of Research in Science Teaching, 43, 255–281. doi:10.1002/tea.20111

73.

Linn

Eylon

B.-S.

(2011). Science learning and instruction: Taking advantage of technology to promote knowledge integration. New York, NY: Routledge.

74.

Linn

M. C.

Gerard

Ryoo

McElhaney

Liu

O. L.

Rafferty

A. N.

(2014). Computer-guided inquiry to improve science learning. Science, 344, 155–156. doi:10.1126/science.1245980

75.

Linn

Hsi

(2000). Computers, teachers, peers. Mahwah, NJ: Erlbaum.

76.

*Liu

Horton

Olmanson

Toprac

(2011). A study of learning and motivation in a new media enriched environment for middle school science. Educational Technology Research & Development, 59, 249–265. doi:10.1007/s11423-011-9192-7

77.

*Liu

Lee

Linn

(2010). An investigation of teacher impact on student inquiry science performance using a hierarchical linear model. Journal of Research in Science Teaching, 47, 807–819. doi:10.1002/tea.20372

78.

*Liu

O. L.

Lee

H.-S.

Linn

M. C.

(2011). Measuring knowledge integration: Validation of four-year assessments. Journal of Research in Science Teaching, 48, 1079–1107. doi:10.1002/tea.20441

79.

*Makitalo-Siegl

Kohnle

Fischer

(2011). Computer-supported collaborative inquiry learning and classroom scripts: Effects on help-seeking processes and learning outcomes. Learning and Instruction, 21, 257–266. doi:10.1016/j.learninstruc.2010.07.001

80.

*Manlove

Lazonder

A. W.

de Jong

(2009). Collaborative versus individual use of regulative software scaffolds during scientific inquiry learning. Interactive Learning Environments, 17, 105–117. doi:10.1080/10494820701706437

81.

McCrory Wallace

Kupperman

Krajcik

Soloway

(2000). Science on the web: Students online in a sixth-grade classroom. Journal of the Learning Sciences, 9, 75–104. doi:10.1207/s15327809jls0901_5

82.

*McElhaney

K. W.

Linn

M. C.

(2011). Investigations of a complex, realistic task: Intentional, unsystematic, and exhaustive experimenters. Journal of Research in Science Teaching, 48, 745–770. doi:10.1002/tea.20423

83.

McNeill

K. L.

Pimentel

D. S.

Strauss

E. G.

(2011). The impact of high school science teachers’ beliefs, curricular enactments and experience on student learning during an inquiry-based urban ecology curriculum. International Journal of Science Education, 35, 2608–2644. doi:10.1080/09500693.2011.618193

84.

Minner

D. D.

Levy

A. J.

Century

(2010). Inquiry-based science instruction—What is it and does it matter? Results from a research synthesis years 1984–2002. Journal of Research in Science Teaching, 47, 474–496. doi:10.1002/tea.20347

85.

*Mork

S. M.

(2011). An interactive learning environment designed to increase the possibilities for learning and communicating about radioactivity. Interactive Learning Environments, 19, 163–177. doi:10.1080/10494820802651060

86.

*Mulder

Y. G.

Lazonder

A. W.

De Jong

(2010). Finding out how they find it out: An empirical analysis of inquiry learners’ need for support. International Journal of Science Education, 32, 2033–2053. doi:10.1080/09500690903289993

87.

*Mulder

Y. G.

Lazonder

A. W.

De Jong

(2011). Comparing two types of model progression in an inquiry learning environment with modelling facilities. Learning and Instruction, 21, 614–624. doi:10.1016/j.learninstruc.2011.01.003

88.

*Mulder

Lazonder

De Jong

Anjewierden

Bollen

(2012). Validating and optimizing the effects of model progression in simulation-based inquiry learning. Journal of Science Education and Technology, 21, 722–729. doi:10.1007/s10956-011-9360-x

89.

National Research Council. (1996). National science education standards. Washington, DC: National Academies Press.

90.

National Research Council. (2000). Inquiry and the national science education standards: A guide for teaching and learning. Washington, DC: National Academies Press.

91.

National Research Council. (2011). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.

92.

*Nelson

Erlandson

(2008). Managing cognitive load in educational multi-user virtual environments: Reflection on design practice. Educational Technology Research and Development, 56, 619–641. doi:10.1007/s11423-007-9082-1

93.

*Nelson

Ketelhut

(2008). Exploring embedded guidance and self-efficacy in educational multi-user virtual environments. International Journal of Computer-Supported Collaborative Learning, 3, 413–427. doi:10.1007/s11412-008-9049-1

94.

*Nicolaidou

Kyza

E. A.

Terzian

Hadjichambis

Kafouris

(2011). A framework for scaffolding students’ assessment of the credibility of evidence. Journal of Research in Science Teaching, 48, 711–744. doi:10.1002/tea.20420

95.

Orwin

R. G.

(1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157–159. doi:10.2307/1164923

96.

Osborne

Erduran

Simon

(2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 41, 994–1020. doi:10.1002/tea.20035

97.

Pea

R. D.

(2004). The social and technological dimensions of scaffolding and related theoretical concepts for learning, education, and human activity. Journal of the Learning Sciences, 13, 423–451. doi:10.1207/s15327809jls1303_6

98.

*Pedersen

Arslanyilmaz

Williams

(2009). Teachers’ assessment-related local adaptations of a problem-based learning module. Educational Technology Research and Development, 57, 229–249. doi:10.1007/s11423-007-9044-7

99.

Penuel

W. R.

Fishman

B. J.

Gallagher

L. P.

Lopez-Prado

Korbak

(2009). Is alignment enough? Investigating the effects of state policies and professional development on science curriculum implementation. Science Education, 93, 656–677. doi:10.1002/sce.20321

100.

Plesch

Kaendler

Rummel

Wiedmann

Spada

(2013). Identifying areas of tension in the field of technology-enhanced learning: Results of an international Delphi study. Computers & Education, 65, 92–105. doi:10.1016/j.compedu.2013.01.018

101.

*Pol

Harskamp

Suhre

C. M.

Goedhart

(2008). The effect of hints and model answers in a student-controlled problem-solving program for secondary physics education. Journal of Science Education and Technology, 17, 410–425. doi:10.1007/s10956-008-9110-x

102.

*Pol

H. J.

Harskamp

E. G.

Suhre

C. J. M.

Goedhart

M. J.

(2009). How indirect supportive digital help during and after solving physics problems can improve problem-solving abilities. Computers & Education, 53, 34–50. doi:10.1016/j.compedu.2008.12.015

103.

Quintana

Reiser

B. J.

Davis

E. A.

Krajcik

Fretz

Duncan

R. G.

. . . Soloway

(2004). A scaffolding design framework for software to support science inquiry. Journal of the Learning Sciences, 13, 337–386. doi:10.1207/s15327809jls1303_4

104.

*Raes

Schellens

De Wever

Vanderhoven

(2012). Scaffolding information problem solving in web-based collaborative inquiry learning. Computers & Education, 59, 82–94. doi:10.1016/j.compedu.2011.11.010

105.

Reiser

B. J.

Tabak

Sandoval

W. A.

Smith

B. K.

Steinmuller

Leone

A. J.

(2001). BGuILE: Strategic and conceptual scaffolds for scientific inquiry in biology classrooms. In Carver

S. M.

Klahr

(Eds.), Cognition and instruction: Twenty five years of progress (pp. 263–305). Mahwah, NJ: Lawrence Erlbaum.

106.

Roehrig

G. H.

Luft

J. A.

(2004). Constraints experienced by beginning secondary science teachers in implementing scientiﬁc inquiry lessons. International Journal of Science Education, 26, 3–24. doi:10.1080/0950069022000070261

107.

*Roscoe

R. D.

Segedy

J. R.

Sulcer

Jeong

Biswas

(2013). Shallow strategy development in a teachable agent environment designed to support self-regulated learning. Computers & Education, 62, 286–297. doi:10.1016/j.compedu.2012.11.008

108.

*Ryoo

Linn

M. C.

(2012). Can dynamic visualizations improve middle school students’ understanding of energy in photosynthesis? Journal of Research in Science Teaching, 49, 218–243. doi:10.1002/tea.21003

109.

*Saab

van Joolingen

W. R.

van Hout-Wolters

B. H. A. M.

(2009). The relation of learners’ motivation with the process of collaborative scientific discovery learning. Educational Studies, 35, 205–222. doi:10.1080/03055690802470357

110.

Schwarz

C. V.

White

B. Y.

(2005). Metamodeling knowledge: Developing students’ understanding of scientific modeling. Cognition and Instruction, 23, 165–205. doi:10.1207/s1532690xci2302_1

111.

*Segedy

J. R.

Kinnebrew

J. S.

Biswas

(2013). The effect of contextualized conversational feedback in a complex open-ended learning environment. Educational Technology Research and Development, 61, 71–89. doi:10.1007/s11423-012-9275-0

112.

Sengupta

Wilensky

(2011). Lowering the learning threshold: Multi-agent-based models and learning electricity. In Khine

M. S.

Saleh

I. M.

(Eds.), Dynamic modeling: Cognitive tool for scientific inquiry (pp. 141–172). New York, NY: Springer.

113.

*She

H.-C.

Lee

C.-Q.

(2008). SCCR digital learning system for scientific conceptual change and scientific reasoning. Computers & Education, 51, 724–742. doi:10.1016/j.compedu.2007.07.009

114.

*She

H.-C.

Liao

Y.-W.

(2010). Bridging scientific reasoning and conceptual change through adaptive web-based learning. Journal of Research in Science Teaching, 47, 91–119. doi:10.1002/tea.20309

115.

*Shen

Linn

M. C.

(2011). A technology-enhanced unit of modeling static electricity: Integrating scientific explanations and everyday observations. International Journal of Science Education, 33, 1597–1623. doi:10.1080/09500693.2010.514012

116.

Slotta

J. D.

Chi

M. T. H.

(2006). Helping students understand challenging topics in science through ontology training. Cognition and Instruction, 24, 261–289. doi:10.1207/s1532690xci2402_3

117.

Singer

Marx

R. W.

Krajcik

Clay Chambers

(2000). Constructing extended inquiry projects: Curriculum materials for science education reform. Educational Psychologist, 35, 165–178. doi:10.1207/S15326985EP3503_3

118.

*Sins

P. H. M.

Savelsbergh

E. R.

Van Joolingen

W. R.

Van Hout-Wolters

B. H. A. M.

(2011). Effects of face-to-face versus chat communication on performance in a collaborative inquiry modeling task. Computers & Education, 56, 379–387. doi:10.1016/j.compedu.2010.08.022

119.

Squire

K. D.

MaKinster

J. G.

Barnett

Luehmann

A. L.

Barab

S. L.

(2003). Designed curriculum and local culture: Acknowledging the primacy of classroom culture. Science Education, 87, 468–489. doi:10.1002/sce.10084

120.

Stage

E. K.

Asturias

Cheuk

Daro

P. A.

Hampton

S. B.

(2013). Opportunities and challenges in Next Generation Standards. Science, 340, 276–277. doi:10.1126/science.1234011

121.

*Stieff

(2011). Improving representational competence using molecular simulations embedded in inquiry activities. Journal of Research in Science Teaching, 48, 1137–1158. doi:10.1002/tea.20438

122.

*Sun

Looi

C.-K.

(2013). Designing a web-based science learning environment for model-based collaborative inquiry. Journal of Science Education and Technology, 22, 73–89. doi:10.1007%/s10956-012-9377-9

123.

*Svihla

Linn

M. C.

(2012). A design-based approach to fostering understanding of global climate change. International Journal of Science Education, 34, 651–676. doi:10.1080/09500693.2011.597453

124.

*Thadani

Stevens

R. H.

Tao

(2009). Measuring complex features of science instruction: Developing tools to investigate the link between teaching and learning. Journal of the Learning Sciences, 18, 285–322. doi:10.1080/10508400902797941

125.

*Tseng

Tuan

Chin

(2010). Investigating the influence of motivational factors on conceptual change in a digital learning context using the dual-situated learning model. International Journal of Science Education, 32, 1853–1875. doi:10.1080/09500690903219156

126.

*Tutwiler

M. S.

Lin

M.-C.

Chang

C.-Y.

(2012). Determining virtual environment “fit”: The relationship between navigation style in a virtual field trip, student self-reported desire to visit the field trip site in the real world, and the purposes of science education. Journal of Science Education and Technology, 22, 351–361. doi:10.1007/s10956-012-9398-4

127.

Tuvi

Nachmias

(2001). Current state of web sites in science education—Focus on atomic structure. Journal of Science Education and Technology, 10, 293–303. doi:10.1023/A:1012239001340

128.

*van Borkulo

van Joolingen

Savelsbergh

de Jong

(2012). What can be learned from computer modeling? Comparing expository and modeling approaches to teaching dynamic systems behavior. Journal of Science Education and Technology, 21, 267–275. doi:10.1007/s10956-011-9314-3

129.

van der Valk

de Jong

(2009). Scaffolding science teachers in open-inquiry teaching. International Journal of Science Education, 31, 829–850. doi:10.1080/09500690802287155

130.

van Joolingen

W. R.

de Jong

Dimitrakopoulou

(2007). Issues in computer supported inquiry learning in science. Journal of Computer Assisted Learning, 23, 111–119. doi:10.1111/j.1365-2729.2006.00216.x

131.

*Varma

Linn

(2012). Using interactive technology to support students’ understanding of the greenhouse effect and global warming. Journal of Science Education and Technology, 21, 453–464. doi:10.1007/s10956-011-9337-9

132.

Voogt

Knezek

Cox

Knezek

Ten Brummelhuis

(2013). Under which conditions does ICT have a positive effect on teaching and learning? A call to action. Journal of Computer Assisted Learning, 29, 4–14. doi:10.1111/j.1365-2729.2012.00487.x

133.

Vygotsky

(1978) Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

134.

Webb

(2005). Affordances of ICT in science learning: Implications for an integrated pedagogy. International Journal of Science Education, 27, 705–735. doi:10.1080/09500690500038520

135.

*Williams

DeBarger

A. H.

Montgomery

B. L.

Zhou

Tate

(2012). Exploring middle school students’ conceptions of the relationship between genetic inheritance and cell division. Science Education, 96, 78–103. doi:10.1002/sce.20465

136.

Windschitl

(2004). Folk theories “inquiry”: How preservice teachers reproduce the discourse and practices of an atheoretical scientific method. Journal of Research in Science Teaching, 41, 481–512. doi:10.1002/tea.20010

137.

*Wu

(2010). Modelling a complex system: Using novice-expert analysis for developing an effective technology-enhanced learning environment. International Journal of Science Education, 32, 195–219. doi:10.1080/09500690802478077

138.

*Yeh

T.-K.

Tseng

K.-Y.

Cho

C.-W.

Barufaldi

J. P.

Lin

M.-S.

Chang

C.-Y.

(2012). Exploring the impact of prior knowledge and appropriate feedback on students’ perceived cognitive load and learning outcomes: Animation-based earthquakes instruction. International Journal of Science Education, 34, 1555–1570. doi: 10.1080/09500693.2011.579640

139.

*Zhang

Z. H.

Linn

M. C.

(2011). Can generating representations enhance learning with dynamic visualizations? Journal of Research in Science Teaching, 48, 1177–1198. doi:10.1002/tea.20443

140.

Zion

Mendelovici

(2012). Moving from structured to open inquiry: Challenges and limits. Science Education International, 23, 383–399.

141.

*Zydney

Grincewicz

(2011). The use of video cases in a multimedia learning environment for facilitating high school students’ inquiry into a problem from varying perspectives. Journal of Science Education and Technology, 20, 715–728. doi:10.1007/s10956-010-9264-1

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.41 MB