Aligning information literacy terminology to STEM disciplinary language used in the scientific method

3.1 Examples used in mapping

The scientific method used in this study is a basic example of the method as a cycle (Fig. 1). The cyclical depiction was selected, as opposed to a linear depiction, because it most closely relates to the research lifecycle commonly used in information literacy instruction (Network of the National Library of Medicine, n.d.). The scientific method is itself a form of the research lifecycle that helps scientists and engineers navigate the process of obtaining new information. Both are iterative processes in which different steps are revisited as needed throughout the duration of the project. This particular diagram of the scientific method is a combination of two examples from the US Department of Health and Human Services and Wikimedia Commons (ArchonMagnus, 2015; Office of Research Integrity, US Department of Health and Human Services, n.d.). The two examples were combined for this project because both had elements that merited inclusion. The example from Health and Human Services included two steps of particular interest to librarians, Share Results and Search Literature. The Wikimedia Commons example contains a feedback loop that helps researchers to reflect after initial data collection. Reflection is a key aspect of academic research and the concept of meta-literacy lies at the core of the ACRL Framework. The combined scientific method used the Wikimedia Commons example with the feedback loop as the base with three additional steps, Search Literature, Share Results and Ask New Questions from the Health and Human Services example.

OPEN IN VIEWER

This paper uses ACRL’s Framework for Information Literacy in Higher Education adopted in 2016 as its example information literacy framework in the mapping exercise (Association of College & Research Libraries, 2016). The ACRL Framework was chosen because it is widely applied in academic libraries in the United States and beyond. It has officially been translated into multiple languages, including Chinese, French, German, and Spanish.

The Framework is designed to be broadly applied by focusing on the learner’s trajectory from novice to expert by drawing on the theories of threshold concepts and meta-literacy. Threshold concepts are “those ideas in any discipline that are passageways or portals to enlarged understanding or ways of thinking and practicing within that discipline” (Association of College & Research Libraries, 2016). Meta-literacy helps the learner reflect on their learning and build connections between concepts. The Framework is comprised of six frames that represent the breadth of information literacy. Each frame contains an introduction, knowledge practices (KP), and dispositions. The KPs, or skills practiced by the learner, are the focus on this study. They are written in the format of learning objectives, beginning with active verbs that can be directly used to design lessons and instructional content. The active voice in the KPs allow them to be mapped to the practical steps of the scientific method. The complete text of KPs by frame can be found in Appendix A.

The mapping process consisted of two steps. Initially, the knowledge practices of the ACRL Framework were mapped to the scientific method diagram independently by each author. This step resulted in three maps. The first step was done individually to capture nuanced understanding of the Framework based on the authors’ STEM and library experience. Each author brings a different academic and professional background to their information literacy instruction. They have varying levels of experience with the scientific method and its application in STEM disciplines. Two authors have an educational background in STEM, while the third is a humanities major trained in STEM librarianship after graduate school. Their experience in STEM librarianship range from one year to 30 + . All three have at least a basic understanding of the ACRL Framework and use it in their information literacy instruction.

The independent maps were completed manually using a print copy of the scientific method diagram. Each author was given stickers with code names corresponding to each of the 45 Framework knowledge practices (KP) and a spreadsheet with the language of the knowledge practices. For example, “transfer knowledge of capabilities and constraints to new types of information products” is the seventh KP under Information Creation as a Process, so it was given the code name ICP 7. As authors completed the independent maps, they noted any questions and reflections that arose. They were not required to use every KP if there were any that did not appear to fit the scientific method. They were also allowed to align a single KP to more than one step of the scientific method if appropriate. Ultimately, all three authors mapped all 45 KPs to at least one step of the scientific method resulting in 135 data points for analysis.

Then, the three individual maps were combined into a single map. Analysis was conducted to determine the level of agreement among the authors for each knowledge practice. Each KP was given an author agreement score depending on the number of authors who agreed on the placement of that KP. As Table 1 depicts, all three authors agreed on the placement of 11 KPs and had no agreement on 13 KPs. Partial agreement between two authors (where one author placed a KP between steps) merited a score of 1.5 and partial agreement between all three authors merited a score of 2.5. One example of a 2.5 score was “determine an appropriate scope of investigation” under Research as Inquiry (RAI 2). One author placed it on Formulate Hypothesis, another on Develop Testable Predictions and the third split it between both steps.

The 13 KPs that received an author agreement score of zero were discussed individually and consensus was reached among the authors as to placement on the scientific method. Discussion included reasons for selecting the chosen method step, the intention of that particular KP and its relation to the scientific research process. Once consensus was reached on the 13 KPs that had an author agreement score of zero, a complete map was created. Of the 45 KPs, 44 were placed on the scientific method and every step of the method had at least one assigned KP. The consensus steps were mapped to the image of the scientific method to create a cohesive map. Significant points of discussion and disagreements between individual maps were also noted for further exploration as a team.

The authors had a high level of agreement on the KPs from three of the six ACRL frames. The three were Information has Value (IHV), Research as Inquiry (RAI) and Scholarship as Conversation (SAC). The frame with the greatest level of author agreement was Information Has Value at 77%. The authors were unanimous on the placement of five of the eight KPs for Information has Value and had only one with no level of author agreement. These frames contain many of the scholarly communications aspects of research, such as citation (IHV 1 and SAC 1), intellectual property (IHV 2) and literature review or source synthesis (RAI 7). They are significant knowledge practices for academic librarians working closely with STEM faculty and graduate students in research intensive institutions like the authors’ home institution. The KPs with the highest level of author agreement can be used to start conversations with research-active faculty about the role that information literacy standards align with the scientific method.

The authors were in less agreement about placement of KPs from the other three frames, Authority is Constructed and Contextual (ACC), Information Creation as a Process (ICP) and Searching as Strategic Exploration (SSE). The frame with the least level of author agreement was Authority is Constructed and Contextual with only 19%. Four KPs had no consensus, one received an author agreement score of 1.5 and one had two authors in agreement. These results raise the question – when should a researcher first consider authority? Further examination of the individual maps found each author favored a different step of the scientific method. One author placed most ACC KPs on Make Observations because authority should be considered at the beginning of the research process. Another author placed most ACC KPs on Search Literature. This could be considered a typical instruction librarian response, considering authority as sources are located and evaluated for use. The final author placed most of the ACC KPs on Gather Data to Test Predictions, emphasizing that authority should be considered when data and sources are being evaluated in relation to the hypothesis. Ultimately, in reaching consensus, four of the six ACC KPs were placed in the first three steps of the method. “Authority” should be revisited at each step in the process, because it is so highly contextual; after it has been established early in the scientific method.

4.2 Author agreement by step of the scientific method

The level of agreement on which KPs to place on each step of the scientific method also varied considerably. As Table 3 depicts, many KPs were placed by one author on a given step but few steps had a high level of author agreement. An author agreement score of 0.5 was used for KPs that authors split between steps. For example, one author split RAI 1, “Formulate questions for research based on information gaps or on reexamination of existing, possibly conflicting, information” between the Make Observations and Think of Interesting Questions steps. Each step received a score of 1 in the 0.5 column. Half of the steps had zero KPs with complete agreement from all three authors and only the Search Literature and Share Results steps had more than one KP with an author agreement score of 3.

Authors had the highest levels of agreement for three steps, Formulate Hypothesis, Develop General Theories, and Share Results; Share Results had the highest percentage of agreement (80%). Formulate Hypothesis and Develop General Theories both had few KPs placed on them, 6 and 9.5 respectively. They were among the steps with the fewest assigned KPs. Share Results, however, received a lot of attention from the authors both individually and collectively. It had 30 KPs assigned individually with 6 in complete agreement. The attention paid to that step aligns with the importance placed on publication at research intensive institutions. That step was ultimately the consensus location for 4 Information has Value KPs and 10 total KPs. This step could be the easiest point to begin discussions with faculty about the intersections of the ACRL Framework and the scientific method.

OPEN IN VIEWER

The step Gather Data to Test Predictions also received a lot of attention from authors, but little consensus on the KPs placed there. In whole or part, the authors placed 15 KPs on this step, yet all but one of those placements was not repeated by another author (Fig. 3). Only one KP, “Use various research methods, based on need, circumstance, and type of inquiry” (RAI 4) had the agreement of two authors. Ultimately, two KPs were assigned to this step, RAI 4 and ICP 6 “Monitor the value that is placed upon different types of information products in varying contexts.” The authors with a STEM educational background placed more KPs on this step, 5 and 7 KPs respectively. The author trained in STEM librarianship, but without a formal STEM background, assigned 2 KPs to this step. This disparity could be due to familiarity with the scientific method. It is a very important step in the process for researchers, and learning about both needs and barriers in that process could help librarians to find common ground with faculty.

5.1 Placement of KPs with no initial author agreement

Seven of the KPs with a score of zero were ultimately aligned to a step identified by one of the authors. Elaboration of that author’s choice was sufficient to sway at least one other author to agreement. Only one KP was tagged to a scientific method step not identified in the initial mapping step. Initially, authors placed “Define different types of authority, such as subject expertise (e.g., scholarship), societal position (e.g., public office or title), or special experience (e.g., participating in a historic event)” (ACC 1) on three very different steps: Make Observations, Search Literature and Gather Data to Test Predictions. After discussion, this KP was aligned to the second step, Think of Interesting Questions. The authors decided that the definition of authority belongs between making initial observations and searching the literature, so it was placed on the step in between.

Only one of the 45 KPs was not placed in the map because authors were unable to reach any consensus on placement. Despite lengthy discussion, “make informed choices regarding their online actions in full awareness of issues related to privacy and the commodification of personal information” (IHV 8) could not be confidently assigned to any specific portion of the scientific method. Depending on perspective, and the type of researcher in question, this KP could be assigned to different steps. One author placed the KP early in the method, Search Literature. Their justification was that this step becomes the home for many library-focused concepts that do not really fit elsewhere; it was the best of several bad locations for that particular KP. Another author viewed the KP through the lens of confirmation bias and placed it on the Gather Data to Test Predictions step. Researchers need to be especially careful to not view their data in ways that confirm pre-established predictions. They need to critically evaluate found information online to avoid the trap of targeted search results and confirmation bias. The final author placed the KP late in the method, on the Share Results step, because they viewed it through the lens of a researcher publishing their own work. The commodification of information and privacy could relate to the protection of intellectual property and avoidance of predatory publishers.

5.2 KPs placed between multiple steps of the scientific method

The first two KPs, ACC 3 and ICP 3, were split between the Search Literature and Share Results steps. They were “understand that many disciplines have acknowledged authorities in the sense of well-known scholars and publications that are widely considered ‘standard,’ and yet, even in those situations, some scholars would challenge the authority of those sources” (ACC 3) and “Articulate the traditional and emerging processes of information creation and dissemination in a particular discipline” (ICP 3). Both KPs received an author agreement score of zero in the initial mapping. Their placement could be debated depending on the objective of the product examined. When examining authorities and creation processes used by other authors in a field, both would be placed in Search Literature as part of the search activity. When examined introspectively as establishment of one’s own authority and or publication choices, both would be placed in Share Results as part of the publication activity.

The placement of KPs like the two above on either Search Literature or Share Results is heavily influenced by the librarian’s goals for their program, and the needs of their institutional community. More specifically, librarians need to consider how that KP is being used, or could be used, in their program. For example, are you helping a novice researcher understand that disciplines have acknowledged authorities? Or, are you helping a more advanced researcher to establish their own authority in their discipline by placing their work in well-respected journals? These goals provide two different starting points for conversation with faculty depending on the needs of the researcher.

Two KPs were placed in the middle of the scientific method’s feedback loop encompassing Develop Testable Predictions, Gather Data to Test Predictions, and Refine, Alter, Expand or Reject Hypothesis. They were “recognize the implications of information formats that contain static or dynamic information” (ICP 5) and “Manage searching processes and results effectively” (SSE 8). Both received an author agreement score of zero in the initial mapping. Both KPs are aligned closely to concepts of research data management, a process that is established and used in all three steps of the feedback loop. These two KPs can be revisited at each step of the loop as researchers develop testing methods, gather data and review that data.

One KP was placed along the continuum between the Formulate Hypothesis and Develop Testable Predictions steps. RAI 2, “determine an appropriate scope of investigation” was used in the methodology section as an example of a 2.5 author agreement score. There was a high level of author agreement that determination of project scope is developed and solidified in both these scientific method steps. It can also be discussed in conjunction with the KPs placed on each of the steps to communicate with disciplinary researchers about the hypothesis and experiment set up stages of the scientific method. The KP on Formulate Hypothesis, “Deal with complex research by breaking complex questions into simple ones, limiting the scope of investigations” (RAI 3) helps researchers to consider the scale of their inquiry. Then, RAI 2 can be used to help determine the scope of the experiment that will be used. Finally, “Determine the initial scope of the task required to meet their information needs” (SSE 1), placed on Develop Testable Predictions, helps researchers to formulate an experiment that will test their refined, specified research hypothesis. Explaining these three KPs in terms of science literacy and experimental design can help to reconcile information literacy terminology with that of disciplinary faculty.