Abstract
Full participation in the “knowledge economy” (i.e., economic activity driven by technical and scientific innovation) requires, in part, a robust skill set for understanding and analyzing content in science, technology, engineering, the arts, and mathematics (STEAM; Sochacka, Guyotte, & Walther, 2016). One of the primary means of managing and displaying data in STEAM content areas is through data visualization using elements such as charts, graphs, and diagrams (Pandora & Fendrick, 2017). Students with visual impairments (i.e., those who are blind or have low vision) require an expanded toolkit of adapted materials and strategies to access these data, including the use of alternate format versions (i.e., braille or tactile; enlarged print) of data visualizations. Although researchers have examined the perspectives of students (e.g., Rosenblum & Hertzberg, 2015; Zebehazy & Wilton, 2014a) and educators (e.g., Zebehazy & Wilton, 2014b) on graphics use, there is little extant research that directly examines the processes through which students with visual impairments engage with graphics. The current research used a think-aloud protocol with students in grades 4–12 to examine the use of strategies by students through a “metacognitive” lens.
Given the unique access considerations for graphics use among learners with visual impairments, researchers have sought to better understand critical supports for students accessing graphics via haptic and visual modalities. Zebehazy and Wilton (2014a) surveyed 97 students with visual impairments (64 using tactile graphics, 33 using print graphics) on their perceptions and preferences related to graphics use. Across tactile and print media, students emphasized the importance of the usability of graphics, motivation to engage with graphics, and skills for access and interpretation of graphical data. From these findings, Zebehazy (2014) sought to account for the factors implicated in interpreting print and tactile graphics by students with visual impairments by proposing the Model of Graphics Interpretation (MoGI). Within the MoGI, the process of reading and analyzing graphical data relies on four key considerations: (1) the quality of the graphic; (2) the student’s familiarity with the graphic type and its content; (3) the student’s level of confidence accessing the graphic type and their motivation to do so; and (4) the student’s toolkit of strategies (technical and metacognitive; see Figure 1). Model of components for graphic interpretation. (Zebehazy, 2014; included with permission)
In further work with student samples, Rosenblum and Hertzberg (2015) sent tactile graphics created using a range of production methods to 14 visually impaired students from across the United States. Participants were contacted by the researchers and asked to respond to interview questions and to respond to comprehension questions related to the content of the graphics. Participants were also asked to describe their strategies for approaching a new graphic and problem solving when encountering challenges. Analysis of qualitative data indicated that participants demonstrated a high level of awareness of their own skill levels with graphics. However, the researchers caveated this finding since strategy use was self-reported; direct examination of student’s strategy use was encouraged in future research.
More recent research has sought to add greater definition to the role of the teacher of students with impairments. Rosenblum, Cheng, and Beal (2018) conducted focus groups with 11 teachers of students with visual impairments in which they asked participants to elaborate on the ways in which they support students in accessing graphics. The researchers noted, with support from educators, that students rely on a variety of strategies that are specific to the type of graphic and medium in which it is presented. Further, there is a need for further research into students’ strategic decision making around graphics use to ensure that instructional approaches are accurate.
Understanding students’ in-vivo strategic action when engaged with graphics requires consideration of metacognitive awareness and self-regulated learning. Metacognition refers to “one’s own knowledge concerning one’s own cognitive processes and products or anything related to them” (Flavell, 1976, p. 232) and comprises “metacognitive” knowledge (e.g., memory), experiences (e.g., self-assessment, judgments), and skills (e.g., strategies; Gascoine, Higgins, & Wall, 2017). Self-regulated learning refers to students’ ability “to inspect and, as they are able, strive to improve how they go about learning” (Winne, 2018, p. 11). Given these constructs are not directly observable, methodological considerations must look beyond evaluations of the products of students’ strategic action to include data on cognitive processes in real time.
The current study used a think-aloud protocol to examine students’ metacognitive strategy use across a series of graphics tasks using the aforementioned MoGI as a coding framework (Zebehazy, 2014). A think-aloud protocol is a “voluntary activity in which learners, having been asked to tackle a relevant task, talk their thoughts out loud while they are engaging in that task” (Cowan, 2017, p. 219). As a methodological approach to examining metacognitive processes, a think-aloud protocol is advantageous in that it does not hinder/alter the processes under study and provides insight not easily gleaned from their products (i.e., scores; Gascoine, et al., 2017; Hu & Gao, 2017). However, the effectiveness of the think-aloud protocol can be limited by its labor-intensiveness and its novelty to the participant (Hu & Gao, 2017)
The following research questions guided the method of qualitative coding used in the analysis: 1. Using the categories of the MoGI, what differences are apparent in the think-aloud transcripts based on participant performance, medium, and level? 2. How does verbosity and metacognitive level relate to performance?
The researchers hypothesized that high-performing participants as well as high school participants would evidence strengths within the areas of the MoGI framework and be more metacognitively connected to their engagement with the graphics to complete the tasks.
Methods
Participants
Forty students with visual impairments participated in the current study. 20 students used tactile graphics and 20 used print graphics (5 used standard print and 15 used large print). Participants were enrolled at either a school for blind students (n = 27) or public school (n = 13) in grades 4 through 12 in Canada or the United States. Seven students of the 40 were identified as having an additional disability. More comprehensive data on participant demographics can be found in Zebehazy and Wilton (2021).
Procedure
Each student met with the first author in a quiet space to complete multiple-choice questions related to five different graphic types (bar graph, Venn diagram, rotation, geometry, and map) at their relative grade level (elementary, middle school, or high school). Test items were selected from Test Ready Plus Mathematics and Test Ready Social Studies Quick Study programs and were produced by the American Printing House for the Blind in both tactile and print formats in adherance to Braille Authority of North America (BANA) guidelines for graphic production (2010). During each task, the students were asked to “think aloud”—verbalize what they were thinking and doing to answer the question—and then answer questions about the difficulty of the tasks after finishing the questions for each graphic type. At the beginning of the session, before starting the five tasks, the researcher (first author) used two practice graphics (finding a point on a grid, analog clock) to allow students to practice how to think aloud. The researcher coached and modeled as necessary. Once the student understood the think-aloud process, he or she began the five graphic tasks. Once the students started the tasks, the researcher prompted only when necessary (e.g., reminding the student to think aloud). After each individual graphic type, before beginning the next type of graphic, the researcher asked the students to rate the difficulty of the task (see Zebehazy & Wilton, 2021) and to reflect on the following: 1. Have you used similar graphics before? 2. What was easy or hard about the graphic? 3. What was easy or hard about the questions? 4. If you could redesign or make this task better, what would you do? Why? 5. What would have helped you to complete the task better than you already did? 6. What else would you like to share about your experience doing this task?
The whole session was recorded on an iPad tablet computer. A third-party professional transcriber created written transcripts of the think-aloud sessions for analysis.
Analysis
The verbosity level of the students was examined to understand how easily students engaged in thinking aloud and to provide context to the qualitative findings based on the think-aloud transcripts. Verbosity referred to how much the student stated what they were doing aloud without prompting (high, moderate, or low). Verbosity was initially coded by the third author and verified by the first two researchers using the following criteria: • low—required ongoing prompts during the think aloud; • moderate—required few or no prompts to think aloud, but did not elaborate on their thoughts; and • high—required no prompts to think aloud and provided unprompted elaborations on their thoughts.
All transcripts were uploaded into NVivo 12, qualitative data analysis computer software, for analysis. The third author conducted initial level hypothesis coding for the transcripts using the Model of Graphic Interpretation (MoGI; Zebehazy 2014; Zebehazy & Wilton, 2021) as the a priori coding framework to indicate instances related to confidence and motivation (self-perception, self-talk, and task perception), content knowledge and experience, quality of the graphic, and strategy use (prompted and unprompted). Hypothesis coding applies researcher-generated, predetermined codes to investigate hypotheses about the qualitative data “to confirm or disconfirm any assertions or theories developed thus far” (Saldaña, 2016, p. 171). A second-level coding procedure was then used by the first and second authors to check the initial level coding, come to consensus on discrepancies, and then examine the patterns that emerged under each code in relation to two of three performance levels (higher = 8–10 items correct and lower = 0–4 items correct), medium used (print or braille), and educational level of materials used (elementary, middle, or high school). Based on the patterns that emerged, the first two authors conducted additional analyses of high-performing tactile and print graphics users compared to lower performing graphics users in order to further differentiate qualitative nuances within categories of the MoGI, with particular focus on strategy use and metacognition.
Results
Verbosity Level
Students were rated as having high, moderate, or low verbosity levels in their think-aloud transcripts by medium, performance level, and level of materials. In general, tactile graphic users were more likely to have high or moderate verbosity levels than print graphic users. Low performers had a higher rate of low verbosity than high or moderate performers, and high school students were rated with higher verbosity levels overall. An intersection between medium, performance, and verbosity was noted: high-performing print graphic users were more likely to have lower verbosity levels than high-performing tactile graphic users.
Quality of Graphic
In general, medium type (print or tactile) did not affect the ability of students to elaborate on the quality of graphics when posed with specific questions after each type of graphic. However, older students (i.e., high school level) were more likely to provide suggestions rather than defaulting to responses like, “none” or “I don’t know”; higher performers, in general, supplied a greater volume of feedback and opinions related to the graphics. Although the majority of students felt the graphics were of high quality, suggestions for improvement focused most often on three elements. For graphic features (e.g., points and lines), students needed to have more blank space between them or more differentiation: I’d probably make it a little bigger diagram just so that you can tell definitely where the boundaries are of each circle . . . just to get a better idea of … this is for all the sports. These are for the separate ones because … you can figure it out but it seems a little small. So it gets all kind of together. It looks like it’s all one thing sometimes. Even if they were just a little bit thicker to feel the difference because some of them were thinner. They would kind of disappear a little bit once they came into contact with another one. So it was like, … which one do I follow now?
Students desired the option to experience graphics on paper rather than on embossed plastic sheets: It’s weird. Because since it’s like plastic-y. It’s like your fingers kind of slide over it and get stuck. It’s like when you are trying to sit on one of those seat plastic cover couches kind of thing.
Students wanted graphics that ensured clarity in terms of the relationship between a graphic feature and its label: Okay, again K-E is in a box of 4 lines slightly closer to the . . . right side of the box or whatever, but, like, it could easily be the top, the bottom or the left one (sighs). That’s what am I trying to say [there is] ambiguity.
Confidence, Motivation, and Self-Perception
In terms of coding instances around the students' confidence, motivation, and self-perception, a few general trends were noted in the transcripts. As performance increased, more students made comments that were related to a positive self-perception and were more likely to try or persevere when they perceived a question as challenging. Perseverance was also related to students who had a larger repertoire of strategies (see Strategy Use section). Older students were also more likely to make comments related to self-perception than younger students across media (i.e., tactile and print graphics). When a student was not confident, they were more likely to need strategy suggestions from the researcher to continue working with the graphic. For example, a low-performing elementary school student who used tactile graphics let negative self-perceptions based on past experience guide his engagement with the tasks: Researcher: So can you find what the coordinates are? Can you follow that across and down and see what the number and letter is? Participant: D, C, what? D, something, something. No, I cannot. Researcher: Okay. Keep trying both hands. See if that helps you. Participant: I am no good at reading with my left-hand braille. Researcher: That might help you with the graphics. Find a spot. Participant: E. D. (Sighs, “Aaahh.”) Researcher: Yeah, good.
By contrast, a high-performing tactile graphic student displayed greater optimism and confidence in her ability to problem solve using graphics: Participant: I didn’t realize that. I’m sure I probably would have figured it out if I had realized to look in the key again. I probably would have figured that out eventually. They had different, they were mentioned differently. I didn't really get that that’s what that was. Researcher: Have you used maps like this? Participant: A little bit, not as much, but I like them because it’s like a puzzle.
In addition, students who displayed less confidence in their ability to independently work with the graphics were more likely to elicit help from the researcher or require direct prompting to initiate and sustain engagement with the graphics: Researcher: Can you find the location of the house on the grid? Participant: I’ll try. Researcher: Okay, great. Participant: Is there a tactile on that? Researcher: M’mm-hmm, right below, right there. Participant: This? Researcher: M’mm-hmm. Participant: This one?
Experience and Content Knowledge
Across performance level, medium, and study-level groups, students were equally as likely to make comments related to experience and content knowledge. In particular, students made clear their lack of experience with the graphic type or questions. Students with more experience with a graphic type more frequently applied knowledge of the general features of that graphic type to the specific one being presented. For example, students who stated more experience with double bar graphs were more likely to search for a key or realize that there should be a key. A high-performing tactile graphic student, when commenting about the Venn Diagram, used prior knowledge of this graphic type: “I think what I am used to is kind of just three circles. And this was probably one more than that. So it wasn’t that far off.”
Some students used concepts from their own experiences to explore certain types of graphics. For example, one student likened the process of determining the height of bars in a bar graph by referring to the popular board game Battleship. Another student used their knowledge of a clock face to solve the rotation problem. Conversely, students who reported that they had less experience with a graphic type tended to report fewer connections between that graphic type and a salient experience that would help in analyzing the graphic.
Strategy Use
Whether prompted or unprompted, the following general strategy categories for working with graphics to answer multiple-choice questions emerged at some level from the transcripts: 1. stating the order of approach (e.g., making a stated plan), 2. previewing the graphic before beginning to answer questions, 3. reading the title of the graphic, 4. reading the question and options first, 5. using the process of elimination, 6. reviewing the question, 7. conducting self-checks or rechecking the answer for accuracy, and 8. making predictions.
Tactile graphic users
All tactile graphic users needed some level of prompting to engage or articulate strategies. Low performers required a greater variety of prompts. High performers were better able to articulate their approach to solving the graphic without prompting. In general, high performers employed a wider range of unprompted strategies, including reviewing the question as needed. Unprompted self-check, making predictions, and previewing questions was less frequently used across all performance groups.
Elementary and middle school–level tactile graphic users required prompting across a wider range of strategies than high school–level tactile learners. Middle school– and high school–level tactile graphic users were more likely to preview the graphic and state an approach without prompting compared to elementary school–level tactile learners. Across all levels, tactile graphic users made few unprompted predictions or self-checks.
This comparison between low and high performers who used tactile graphics illustrates the difference in ability to articulate strategies. High performers were able to articulate a strategy step-by-step and explicitly apply this strategy to the problem without prompting: Participant: So I see that it’s a bar graph. So what I will do is I will look at the bottom and it’s the list of the months so I will follow the month and it says, During what month. Okay. So I will look at the bars and see which one is the longest. And it looks like the second one in from the left is the longest. So I will follow that over to 70, but the month is April.
Conversely, this quote from a low performer demonstrates the interaction of the components to graphic interpretation as illustrated in the MoGI. Although motivated, the student’s lack of experience with the graphic type and difficulty with technical skills inhibited her ability to engage strategies or have enough experience with strategies to be able to articulate them; thus, she needed prompting and modeling: Participant: How many dogs is there? Researcher: So what do you have to do to figure that out? Participant: I know one thing. Researcher: What? Participant: It says “dogs.” So that means there’s not one dog. There’s more. Researcher: Okay. Good. … [R]emember when you followed those bars all the way up . . . nice and slow like you did? Then if you go across to the number, you can tell how many dogs there were. Yeah, just like that, but you have to do it the other way. Participant: This one? Researcher: Find the top of the dog bar. You’ve got the rectangle right at the top there. And now follow across from there. Participant: This one? Researcher: No, that’s a cross. Follow my finger. I’m going to do it with you this first one. Follow it up, up, up, . . . keep going. Oh, there’s the top of our bar. You feel it? Participant: Yeah. Researcher: Okay, so now there little tiny dotted lines help us get across. So we are going to follow towards your left this little tiny dotted line. Stay on it. Stay on it. . . . Good. And then we get to a number. What number is that? Participant: 16!
In another example, this low-performing user was able to engage a strategy but was not as “metacognitively” aware that they were explicitly selecting a strategy as compared to a high-performing tactile graphics user: Researcher: So when you were moving your hands together like that, what were you doing? Participant: Huh? Researcher: . . .[W]hat were you doing just then? Participant: I was trying to . . . I was finding the corner.
By comparison, this quote from a high-performing tactile graphic user illustrates greater independence in identifying and describing their metacognitive strategy for exploring the graphic: Participant: This one looks like it’s a complete almost reflection of this. So that wouldn’t [be it]. And then I think this one, it looks like more counter-clockwise, maybe. And then I’m trying to compare, like, I’m trying to picture how it would be like by just having one hand on the actual figure and then trying to relate it to the choices. . . . I feel like if I move this figure back up into this corner it would kind of be what I’m thinking like how I’m trying to picture it.
Print graphic users
As with the tactile graphic users, all print graphic users required prompts to some degree across performance levels, but they all employed unprompted strategies to some degree as well. Both elementary school– and middle school–level print graphic users were less likely to preview the graphic than the high school–level print users. Print graphic users across all levels were also unlikely to make unprompted predictions or self-checks. In terms of performance level, low-performing print users employed a narrower range of unprompted strategies. As with tactile graphic users, low-performing print graphic users did not articulate their strategy use as readily as high-performing print users. When they did articulate a strategy, they were less likely to provide clear, unprompted connections between their selected strategy and its application to the problem. For example, a low-performing print user selected an appropriate strategy, but the think aloud lacked clarity in articulating how it was being used. Participant: If the shaded figure in the box is rotated clockwise 90 degrees around point X, what figure would appear in the new position? I am not sure what it is called. Actually it would be B. Researcher: Okay. And how did you figure that out? Participant: Because I imagined a clock in my head and then I counted . . . the numbers on a clock and then I looked down and then I found out it was B. Researcher: Okay, so you moved the numbers that it would correspond on the clock to [figure it out]. Participant: Yeah.
Conversely, high-performing users required little-to-no prompting to articulate their strategy to the problem and apply it, in sequence: Researcher: Okay. . . . [T]alk to me about this map. Participant: Okay, so at first when I looked at it, it was like, “Whoa, a bunch of squiggly lines and words everywhere.” But I mean if you find a good starting point you can usually find . . . everything else on here without a problem. The city park was the best place to start and then just branch out and find everything else, because it’s number 2, and it’s big and dark, and the highway 13 is a good reference point too, because it’s dark and then you can just find everything else. But at first this one was really confusing.
Discussion
Connection with Performance
Analysis of the think-aloud transcripts in this study connect with the quantitative findings reported elsewhere (see Zebehazy & Wilton, 2021). In Zebehazy and Wilton (2021), greater frequency of engaging with graphics and higher teacher ratings of independence with graphics were associated with higher performance on problems featuring graphics. In this study, higher performing students were more likely, during the think-aloud process, to indicate that they were relying on previous experience with a graphic type when solving a given problem as compared with lower performing students. Further, higher performers provided more unprompted details into their thought processes around solving problems with graphics, implying a higher degree of metacognitive awareness of their own strategy use.
Connection with Verbosity Levels
Verbosity across participant level, performance, and medium were examined to provide perspective on the think-aloud analyses. Although the general patterns remain largely consistent (e.g., higher performing students and students in higher grade levels shared more details during the think-aloud process), there was a noteworthy difference when comparing verbosity across study medium. Print graphic users, given lower verbosity levels, were less likely than tactile graphic users to explicitly state what they were doing, which may be due to inherent differences in the sensory channels used to gather information from graphics. Print users were able to observe a larger portion of the graphic at one time and, as a result, did not require the same degree of purposeful strategic action as tactile users. Further, some strategies may not be applicable to the use of print graphics (e.g., detecting texture differences between zones). For example, the strategy of going back to the key would differ by medium because there may be more labels on the print graphic itself, or a print graphic user may be able to visually compare symbols in the key to the graphic with fewer explicit steps. The tactile graphic user may need to remember what was displayed in the key versus the graphic and return more frequently to the key, increasing the likelihood that the strategy would be recorded in the think-aloud process. Related, print graphic users were more likely to read the title of the graphic than tactile users without having to first overtly recognize that a title should be present.
MoGI
This study used the MoGI as the framework for coding the transcripts. What emerged from this process was further evidence of the interdependence of the aspects of the MoGI regardless of medium type (tactile or print). Based on students' think-aloud transcripts, rarely did only one component of the MoGI (i.e., confidence and motivation, content knowledge and experience, graphic quality, or skills) emerge as the sole factor underlying performance. For higher performing students demonstrating high levels of metacognitive awareness, multiple MoGI components were implicated. For example, students who displayed rich content knowledge, experience with the graphic type, confidence in their own skills, and strategies (both general and specific to the graphic type) interpreted the graphic more accurately and efficiently. In contrast, a student with a moderate performance level may articulate some strategies and experience, but lack confidence in their ability to apply them to new tasks or articulate less-sophisticated reasons for their strategy selection. While age likely plays a role in skill development and other aspects of the MoGI (e.g., strategy use and experience), statistical performance of students that controlled for age (by level taken) still revealed contributing factors to performance such as frequency with engaging with graphics (see Zebehazy & Wilton, 2021).
Use of Think Aloud for Assessment and Instruction
In the current study, the think-aloud process afforded the researchers a degree of in-vivo access to students’ metacognitive processes. Since verbosity alone was not a robust discriminator of high and low performance on graphic tasks, it was critical to examine the substance of students’ think-aloud processes. Translated to practice, qualitative analyses of transcripts generated from recordings of the think-aloud process would provide educators with insight into students’ self-regulated learning and metacognitive abilities. Further, use of the think-aloud process allows for the centering of student voices and communicates to the students that their thoughts and insights are a valuable part of the learning process.
IMPLICATIONS FOR PRACTITIONERS
The findings of this study hold several implications for practice. Individuals working with students with visual impairments should: 1. Create lessons that allow students to work on self-regulated learning and metacognition. 2. Provide think-aloud opportunities or opportunities for students to verbalize their plan of action and to evaluate success of their plans. Educators should model metacognition by applying think-aloud processes to their own strategic action. 3. Help students understand that taking risks and trying out strategies leads to learning. Emphasizing and providing feedback about the process that students were observed using and the strategies they tried or developed will support this understanding of learning. 4. Support strategy development for different graphic types before they are needed in the classroom. Lack of practice with reading graphics can inhibit a student’s ability to demonstrate content knowledge (e.g., mathematics ability). 5. Expand a student’s strategy repertoire and encourage them to try out strategies before asking the teacher for help right away. Think-aloud patterns of lower performing students indicated that asking for help before trying other strategies or exploring the page was common.
Limitations and Future Research
This study had some limitations. The varying levels of verbosity, as indicated, could mean that some students were using additional strategies that were not articulated and recorded. Further, the think-aloud process was unfamiliar to most students. Although practice was provided, more nuanced aspects of a student’s thinking may emerge as students become more adept in the think-aloud process. Finally, the use of the MoGI as an a priori framework in the development of the coding scheme for think-aloud transcripts may have limited the scope of what was coded.
Subsequent research using a different coding scheme could unlock other nuances in students’ think-aloud processes when engaged with graphics. For example, the effect of teacher–student interactions on students’ metacognitive development and strategy use would be an interesting and novel approach with this population of learners. In addition, there may be important data on strategy use that cannot be adequately captured in transcripts of the think-aloud process. For example, an analysis of eye-tracking data for print graphic users and hand-movement data for tactile graphic users would lend greater context to the findings of the current study.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.