Abstract
Introduction
Participants’ perspectives are valuable in evaluating the effectiveness of an intervention.
Methods
Interviews were conducted with teachers of students with visual impairments and students who completed an intervention designed to build graphics literacy skills.
Results
Six themes were identified with corresponding subthemes. The intervention was reported to build students’ graphics literacy skills.
Discussion
Some students were able to generalize the strategies they learned to academic classes. Higher-level thinking skills challenged some students.
Implications for practitioners
A systematic approach beginning early can increase students’ graphics literacy skills. Teachers should provide ongoing opportunities for thinking and regulation of learning.
Keywords
Students with visual impairments (i.e., those who are blind or have low vision; referred to as “students” for the remainder of the article) struggle to read graphics independently (see Rosenblum, Zebehazy, Gage, & Beal, 2021). As students move into higher education and the workforce, proficiency with interpreting information in graphics will allow them to be productive and competitive. To build independence with graphics, students need frequent, early, and systematic instruction (Zebehazy & Wilton, 2014b, 2014c, 2021). In a study by Rosenblum, Cheng, and Beal (2018) 11 teachers of students with visual impairments (referred to as “teachers” for the remainder of the article) recommended that students develop systematic exploration approaches for different types of graphic categories. Teachers reported, however, that they do not feel prepared to teach students graphics literacy skills, nor do they have the time to dedicate to preparing materials for their students (Zebehazy & Wilton, 2014a). Currently, curricula are not readily available to provide graphics literacy instruction. To address this scarcity, The Animal Watch VI: Building Graphics Literacy (AWVi: BGL) project developed an intervention to guide students to improve their exploration and interpretation of print and braille graphics. Significant differences in pre-and posttest performance of 41 students in grades 5–10 demonstrated the intervention was effective (Rosenblum et al., 2021).
To expand on the results of the Rosenblum et al. (2021) intervention, students and teachers participated in post-study interviews. Interviews served to gain the participants’ perceptions of the effectiveness of the intervention. Perceptions of the experience, or social validity, is an important component to understanding whether an intervention is effective and has the potential for wider use (Carter & Wheeler, 2019). The examination of social validity has traditionally been related to behavioral interventions (Wolf, 1978). Educational relevance examines if acceptance by both teachers and students is relatable to curriculum (Aaroe & Nelson, 1998; Carter & Wheeler, 2019). Callahan et al. (2008) suggested that lack of social validity of interventions might contribute to a research-to-practice gap. Interviews are one way to understand the social validity of a study by learning from participants about the positive and negative aspects of the intervention (Aaroe & Nelson, 1998; Gresham & Lopez, 1996). If participants perceive positive changes due to the intervention, then they will view the intervention positively, be more likely to use it in the future, and recommend it to others. In addition, motivation has been increasingly highlighted as a key factor in students’ learning (Pekrun et al., 2014). The more motivating materials, the more effective they will be in engaging students. The goal of interviewing both teachers and students who participated in the AWVi: BGL intervention was to determine the value, if any, they observed from the curriculum as well as strengths and recommendations they had for the materials.
Method
This study was approved by the Institutional Review Board at the University of Arizona. Informed consent was obtained from each participant. As part of the larger study (see Rosenblum et al., 2021), students and teachers participated in individual interviews to gather information about the effectiveness of the intervention and the students’ and teachers’ perceptions of the students’ graphics literacy skills (Figure 1). Sample graphics for coordinate plane 2: Grevy’s Zebra.
Intervention
Description of the graphs in the 10 instructional units.
Intervention procedure
Teachers received 1.5 hours of online training that included an orientation to the materials and research protocol prior to their students completing a pretest, the intervention, and posttest. The first author met with the student and teacher online using the Zoom videoconferencing application to complete a 6-item pretest (see Rosenblum et al., 2021), which was recorded. Following the pretest, the student completed the intervention as their schedule allowed. After completing the last unit (or end of school year nearing) the first author conducted the posttest online followed by recorded individual interviews with the student and teacher.
The first and fourth authors developed interview questions for the students (10) and teachers (8) around the unifying intent of evaluation of the intervention from a social validity perspective (Clarke & Wheeler, 2019).
The 10 student interview questions were: 1. Tell me about your experience working with our curriculum, iPad application (app), and graphics. 2. After working with our materials what are three things you’ve learned about using graphics to solve math word problems? 3. How did the “Getting Started” activity help prepare you for the session? 4. Tell me about using the iPad app and the graphics that accompanied it. 5. Think about how the information was paced, that is introduced, was it at a good pace for you? Explain why or why not. 6. How have you used what you’ve learned through these materials in your classes? 7. The iPad app had features such as adjusting the speed, background, and font colors. Tell me about your experience with these features. Tell me about any features we should change or add. 8. Tell me about the graphics you used. 9. How can we make the lessons you’ve done with these materials better? 10. Is there anything else you want to share?
The eight teacher interview questions were: 1. Tell me about your experience using the AWVI: BGL curriculum, app, and book of graphics with your student. 2. Did the training you received through the project prepare you for using the materials effectively with your student? Why or why not? 3. Tell me about your experience as you watched your student complete the pre- and posttest. What did you learn about your student through these? 4. Tell me about the layout of each lesson and what worked and did not work for you and your student. How can we improve the lessons? 5. What changes did you see in the student over time? For example, did you see any changes with understanding of graphics, types of questions or comments the student made, or greater independence? 6. When using the iPad, what accessibility features did the student use? Was the student successful with these features? If not, what were the challenges? Did you see improvement over time? What do you think is still needed for the student to be successful? 7. How can we improve the curriculum, app, and graphics? 8. Is there anything else you want to share?
Participants
Demographic data for the students and teachers were reported in Rosenblum et al. (2021). Students were in 5th to 10th grades, and the majority used braille as their primary literacy medium. All but four of the students attended public school programs where they received itinerant services. Following the posttest, 39 of the 41 students and 37 of the 38 teachers were interviewed.
Interview procedure and data coding
After completing the posttest described in Rosenblum et al. (2021), the first author met with each interviewee. The semi-structured interview questions were used to evaluate the intervention by learning about the interviewees’ perspectives on their: (a) experiences using the materials; (b) observed learning and transfer; (c) features and accessibility of the materials; and (d) recommendations for improvements. Semi-structured interviews were selected for their “ability to accommodate a range of research goals…[and] to draw the participant more fully into the topic of study” (Galleta, 2013, p. 45).
The first two authors coded the interview transcripts together using a thematic analysis process (Braun & Clarke, 2006). Given the evaluative purpose of the interviews, thematic analysis was considered an appropriate method as it is not theoretically bound but still seeks to describe patterns organized into themes (Clarke & Braun, 2017).
Conducting interviews with both teachers and students and looking for themes across the two groups supported a check on reliability since both groups would need to view the intervention as socially valid for it to ultimately be useful. The perceptions of both groups in comparison with quantitative performance data (Rosenblum et al., 2021) triangulated an overall judgment of the intervention potential for building students’ graphics literacy skills.
Results
Themes and subthemes.
Access to learning materials
Choices
Teachers and students appreciated the access choices the iPad app provided. This was especially true for dual media learners who often switched or combined more than one modality (e.g., tactual, visual, and auditory). A teacher reported, “Sometimes, if he was questioning the braille graph he would double tap the graph on the iPad and compare visually with what he was feeling on the graph…. He would listen to the questions with read aloud.” A student reflected, “I liked the multimedia approach with auditory, being able to look at the iPad as well as the book of graphs.” Some teachers noted the iPad app helped students evaluate and practice their learning media preferences, for example, type of braille display or VoiceOver (VO) speed. One teacher reflected, “It is wonderful that he can navigate with his Braille Note… [The iPad app] really gives him a lot of independence. This was a safe place to practice VoiceOver.”
App features
Students appreciated using one of four pre-set background-font combinations. Overall students who were VO users were positive about the app. They did note some challenges with pronunciation and not being able to stop within a problem to do a step before the audio continued. A student shared, “I liked how accessible with VO [the iPad app] was and that the images are described…. Just by reading the questions, I feel like I can see the graphs right away. It’s like magic.” Students who used the built-in speech, which emulated VO, noted that the multiple-choice answer choices were read as one string. They recommended that the letters A, B, and C be added to each choice. This change was implemented towards the end of the study. Students liked the ability to adjust the speed of the built-in screen reader.
Assessment
Self-reflection and monitoring
Some teachers noted their students’ ability to reflect on their own performance. A teacher who had two female students in the study commented, “[One girl] is able to explain her mistakes and come up with ideas of how she can do it better. She has really learned to self-analyze and to advocate for doing things in a way that work for her. Both girls are so shy that they don’t self-advocate and to see them doing it now is wonderful.” In contrast, some teachers noted students were not as aware of personal changes. A teacher reported, “He doesn't realize how much more efficient and accurate he is after doing this project.” Students made comments about how they were engaging with graphics because of the intervention. One shared, “We haven’t been using graphs in class. I think I am more aware of where the information is positioned now.”
Teachers’ understanding of students’ strengths and needs
The first opportunity teachers had to assess their students came during the pretest. One teacher noted, “As I watched my student complete the pretest, I became painfully aware how difficult it was for him to read graphs efficiently and successfully.” By the time the students reached the posttest, several teachers noted changes. One commented, “During the posttest, he breezed through many questions, answering correctly, without hesitation.” Yet another recognized, “While watching the posttest I also noticed a couple areas where he could benefit from going back and reviewing certain strategies taught in AnimalWatch[Vi: Building Graphics Literacy]. Now that we have completed the entire curriculum, we can use it as review in the future, focusing on particular areas of weakness for my student.”
During the intervention, teachers continued to learn about their students’ abilities. A teacher reported, “[The intervention] was very good because it helped me to focus on specific skills. I was able to identify some gaps that she had that I would not have been able to otherwise prior to her starting upper level math. It has given me data I need to set up follow up lessons.”
Content
Strengths of materials
Teachers and students reported strengths of the intervention including the animal content, high-quality graphics, layout of the units, and teacher curriculum notebook. Students who were blind appreciated picture descriptions and all students liked the animal sounds. A student shared, “Amazing descriptions of the images. They were spot on amazing. The sounds were amazing.” Yet some students were so focused on learning about the animals, they did not attend to the graphic instruction without teacher encouragement.
Teachers viewed the instructional sequence as appropriate. One commented, “I really like the explicit instruction in the warm-up and errorless teaching. Then he gets to branch on his own and he really liked the feature where he could check his answer. I liked at the end he had to put into words what he was thinking.” Many students liked the order of presentation. One shared, “The warm-up [questions] were a lot of help because they helped me get ready for the questions [in sets A and B], and get familiar with the graphs.”
The research team included color on braille graphics and tactile features on print graphics. Teachers and students appreciated these features and reported the graphics were of high quality. A teacher noted, “I feel like the graphics are detailed, have the right amount of information and whatever the production house is doing is spot on.” Students with low vision viewed graphics favorably. One shared, “I liked the color choice and having the feeling when I touched.”
Teachers believed the curriculum notebook was a strength of the intervention. A teacher commented, “I was able to look ahead at the lessons and really figure out what vocabulary and understanding I needed to address. I was able to pre-plan where I needed to spend additional time on the practice set.” Another teacher noted, “The [curriculum] notebook helped me guide them through the process and problem solve.”
Improvements to materials
Improvements teachers and students recommended often had to do with the students’ prior experience with the content covered in the intervention. For students with little content experience, it was recommended that there be additional warm-up problems and instruction while for those with more experience, the amount of practice and instruction provided was sufficient. Some teachers requested the inclusion of additional graphics for each unit so that teachers had material for extension activities or additional practice.
There were few recommendations made about the graphics. Those provided focused on clarity of line textures or simplification of maps and coordinate planes. Several teachers and students noted the challenges graphics on microcapsule paper presented. One teacher shared, “He did not like the graphics on microcapsule paper [for the line graphs and four quadrant coordinate planes] but a lot of textbooks are using them. So, it is good for him to have to try reading them.”
There were suggestions made for improving the iPad app including pauses between answer choices for multiple choice questions, pronunciation of words, consistent use of Roman numerals for coordinate plane quadrants, and glitches related to technical issues with data being pushed to the server.
Promoting thinking
The research team designed the intervention to promote thinking skills. We combined open-ended and multiple-choice questions throughout each unit. Open-ended questions included: two getting started questions, the last questions in set A and set B, and the final reflection question. A teacher shared the value of the open-ended questions, “I really liked that he had to do the getting started because he had to plan what to say.” A student commented, “Hearing myself talking about the graph out loud made me focus and think about what I was doing.” For many students, interpreting and expressing information was a new skill. The opportunities available in the intervention to work on thinking seemed to promote teachers’ engagement of their students in a way that scaffolded their thinking. One teacher commented, “At the beginning of the study I'd have to remind her to take her time and examine [the graphic] before you answer. I’d ask her ‘What do you think you can do differently?’ if she got it wrong. I saw her spend more time slowing down and looking over the graphic.” In contrast, some teachers didn’t recognize the opportunity the intervention provided to have students develop their thinking skills. They reported the open-ended questions were difficult for their students and recommended the intervention should only focus on the use of explicit questions.
Generalization
General education classes
For some students, knowledge gained through the intervention, carried over to their general education classes. A student noted, “In history I had to navigate a map of South Africa. I knew I had to go with a system to find each country. It was easier to navigate [because of what I learned in the map unit].” Another student shared, “[Now I] feel more successful in the math class because I can…work with graphs more fluently. I think if I had gone in math class before with a bar graph [I wouldn’t know what to do,] and now I really can see the difference.” A teacher who attended geometry class with the student expressed, “I do think it has crossed over to geometry and having to interpret shapes and prisms.”
High-stakes testing
Teachers commented about generalization of content learned in the intervention to high-stakes testing. A teacher noted, “[Skills from the intervention] gives them more confidence in the classroom and on the state test. The verbal description on the state test made it challenging but [the intervention] helped.” A teacher who observed her student as he took a high-stakes test reflected, “He was looking at the graphs more methodically toward the end. I did see that on the state test for the math.” A student reflected, “On state testing this year I felt a lot better. Last year there was a bar graph with two different bars and I was [wondering] ‘How does this work?’ and this year I got it.”
Larger community
A few teachers and students gave explicit examples of the impact of the intervention on the student outside of school. A student commented, “At Braille Challenge we did graphics and I thought I knew better about these types of graphs. I think just understanding the graphs. I was a little confused on one of [the questions] but then I realized I hadn’t read the graph.” Several teachers also commented about their students’ confidence at the Braille Challenge. Another application of content learned was made by a student who reflected, “I went to [a specialized school] for a short-course on transit and we had to look at maps. I think I did it better.”
Graphics literacy
Strategy building
The focus of the intervention was to assist students in building their skills in locating and interpreting information presented in graphics. It was clear, both through comments from teachers and students, that this goal was achieved. A teacher shared, “Her organization before on almost anything was she just whole hand explored the page. So, we talked about having a methodology, top/bottom, left/right. That made a big difference for her. Her confidence prior to doing [the intervention] was low. She was not able to follow a point on a graph down to the x-axis. She wasn’t going straight. She wasn’t connecting that a line had value. Now she is able to do that. She can take two pieces of info and assimilate them.” A student noted, “I try to explore the graphics in the beginning when I first get them rather than waiting for a question.”
Strategies mentioned by students.
Hand use
For some students the intervention increased their skills using their hands to locate information in graphics. A teacher reported, “I noticed she increased her bilateral skills with the braille. She typically reads braille with one hand so [the intervention] got her to use two hands.” The intervention was effective for some dual media learners in promoting their growth in tactile skills. A teacher reflected, “She often makes errors because she is not looking at the whole page so this project encouraged her to pay more attention. She used her hands a lot more than she realized. I saw her using her fingers to scan. A lot of time in class, she gets lost in a graph. It was interesting to see her incorporate the tactile piece.”
Noticing attributes of the graphics
A benefit of the intervention was that some students became more aware of subtleties in textures within graphics, the location of labels, or the use of color as a way to assist them in locating key information.
Student characteristics
Motivation
Students and teachers reported that the intervention was motivating. A teacher reflected, “I have seen a great growth in her. She was excited each time we got ready to work on this. She wants to shut down when something is difficult. This project kept her motivated.” For students part of the motivation was the environmental science content about animals unfamiliar to them. A student reported, “I loved the set up... It was fun learning about the animals. It made the app more enjoyable because you're learning about an animal in the midst of all this graph stuff.”
Self-confidence and self-advocacy
Teachers perceived the level of student confidence increased after the intervention. They reported students were more decisive, more willing to ask questions, and more self-assured. Some teachers observed changes in their students’ self-advocacy skills around graphics, as expressed in the words of one teacher, “I feel like she is asking more questions and becoming self-aware. She'll stop and will say ‘I don't get this.’ or will ask a direct question. She is self-advocating more.”
Independence
Teachers and students observed that students’ independence improved as they moved through the intervention. A teacher commented, “[She developed] greater independence! In the beginning, I had to ask a lot of questions and prompt her. Once she got into it, her questions got more relevant and she built on knowledge as we built on the lesson. She would remember some of the terminology and apply it to the next unit.” The design of the intervention helped promote student independence as noted by this comment from a teacher, “It was so good to have the student have the experience of the independent learning because it was all written out so they can work on their own.”
There were few students who specifically commented on independence. An examination of what they shared under other subthemes indicated a movement for them towards more independence in how they accessed and interpreted data presented in graphics. In one of the few direct comments about independence a student shared, “If I have to use [graphics] in class I will do better with feeling where everything is. If it is a graph like a coordinate plane, I usually start in the middle where the quadrants meet. If it is another type of graph, I start on the left or at the top. This exploration is new [for me] with most of them [graphics].”
Discussion
From a social validity perspective, comments from the teachers and students indicated that overall, the intervention was well received and benefits were recognized by both teachers and students. In conjunction with the pre- and posttest changes in performance (Rosenblum et al., 2021), the intervention has potential as a useful tool for teachers to help students develop graphic literacy skills. The interviews also provided insight into instructional strategies and improvements to be made to the AWVi: BGL materials.
Content considerations
When learning is interesting and motivating, students will more likely engage (Pekrun et al., 2014). AWVi: BGL is based on the work of Beal who recognized in previous projects that middle school students were more likely to engage in learning mathematics with content tied to environmental science (Beal et al., 2010; Beal & Rosenblum, 2018). Based on comments from both teachers and students, the content selected for the intervention was similarly motivating. AWVi: BGL provided students with opportunities that went beyond the focus of building graphics literacy skills. Teachers reported that students benefited from opportunities to practice access technology skills and to increase tactile skills, in both braille reading and graphics interpretation. Given the limited time of teachers, it is helpful when instruction can provide secondary benefits in other areas of the expanded core (Allman & Lewis, 2014) and core curriculum.
Teaching considerations
The intent of the intervention was to support students to develop systematic approaches to locate and interpret information in graphics. The teachers were overall positive about the approach used in the intervention. However, designers of future curricula may want to include additional specific strategies for tactile learners to guide them in developing efficient hand use.
The inclusion of higher-level thinking skills was met with mixed reviews by the teachers. The majority of students and teachers appeared to appreciate the opportunity to have thinking challenges incorporated into the units. For those teachers who felt their students struggled with these questions, inclusion of how to scaffold thinking skills may be a useful addition to future curricula. In addition to thinking skills, the interviews highlighted the emerging ability of some students to self-reflect and monitor their progress. Development of these learning attributes promote self-regulated learning behaviors (Butler et al., 2016) is helpful outside the context of graphic literacy skills.
Through the intervention, many teachers recognized the need to increase student wait time and decrease prompting. The researchers saw these reflections by teachers as positive. Teachers were considering their own teaching and their need to provide opportunities for students to take the lead in their learning. Evidence suggests that reflective teachers become more competent in questioning practices that develop student thinking (Weiland et al., 2014). It became clear, through both the student and teacher interviews, that there is no one way that students must approach graphics to gather information. However, the need for students to develop their own effective systematic approach was clearly reported. Students recognized that when a key is present it provides a wealth of information, that they need to explore all parts of the graphic, and that knowing what the question is asking can guide them in seeking information. Future curricula that provides teachers and students with additional strategies to continue to build systematic approaches is warranted, not just in the area of graphics literacy, but also in other areas of instruction.
Some of the students who participated in the study used both print and braille as their literacy mediums. All the students used audition. Opportunities for students to explore how they most effectively take in information, especially new information or complex content, is important. When students can determine the most effective way for them to access information for a given task their ability to make effective choices increases. This need of self-understanding lends importance to conducting learning media and assistive technology assessments with students and reviewing the results with them.
Teachers also noted that there are additional types of graphics not taught in class nor in the intervention that are included on high-stakes tests. Students need more practice with a wide range of graphics.
Limitations
This study had limitations. Many of the interviews took place during the students’ and teachers’ last 2–3 weeks of school. This is traditionally a busy time in the school year and it is possible that participants did not answer the questions with as much detail as they would have done earlier in the year. The use of a set of questions during each interview may have inhibited participants sharing additional information that would have resulted in additional themes.
Implications for practitioners
Use of a curriculum like AWVi: BGL can help teachers target necessary graphics literacy skills in a systematic way as well as assess what aspects of students’ reading of graphics need refinement. Based on the teacher and student interviews, basic elements that should be maintained if a teacher chooses to make their own materials or embed the instruction within classroom activities would include: attending to motivation, providing adequate and frequent opportunities to practice and transfer specific skills, and developing student ability to self-monitor and regulate. The more students learn to question, compare, and evaluate their own strengths and needs, the more likely they will be able to recognize their improvements in working with graphics and apply appropriate strategies to different graphic types independently. These thinking and metacognitive skills are transferrable to other areas of learning. Although the AWVi: BGL curriculum is targeted at middle school level, prior research (Zebehazy & Wilton, 2014a, 2014b, 2021) and student performance data (Rosenblum et al., 2021) indicate a need to begin early with students in exploring graphics, learning about elements of graphics, and comparing different layout of graphics to avoid gaps in skills ability at later ages when students need to use and apply the skills to keep up with peers in class.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Author’s note
The research described here was supported by a grant from the Institute of Education Sciences (R324A160154). The views expressed are not necessarily those of the funding agency. The authors wish to thank the teachers of students with visual impairments and their students who participated in the pilot study during the 2018-2019 school year. In addition, we thank our consultants, vendors, and advisory board without whom the development of AnimalWatch Vi: Building Graphics Literacy would not have been possible.
Dr. Carole Beal, University of Florida (retired), was the primary investigator for the AnimalWatch Vi Building Graphics Literacy project. She died on July 28, 2021. Her strong intellect and commitment to increase the number of children who enter the STEM professions is commendable. Her passion lives on through her work.
