Evaluation of Qualitative Data Analysis Software by a Visually Impaired Researcher: An Autoethnographic Study

The Significance of the Article

The present article focuses on qualitative researchers who have completely or partially lost their sight. There are no accurate records about the number of visually impaired researchers globally; however, their contributions to quantitative and qualitative methods cannot be ignored. For example, researchers with visual impairment developed methods for teaching statistics to students with visual impairment (Godfrey & Loots, 2015), while other researchers assessed the accessibility of statistical packages for blind researchers (McCabe, 2020). Likewise, researchers with visual impairment suggest techniques for teaching chemistry to students with visual impairment (Cartlidge, 2002; Fantin et al., 2016). Academics with Visual impairment teach courses that are traditionally taught by sighted teachers such as Biomedical Engineering (Greenvall et al., 2021). In addition, software and systems have been developed to facilitate collaboration between sighted and non-sighted researchers in fields like Molecular Biology (Cordes et al., 2008).

Indeed, the involvement of researchers with visual impairment in designing and conducting qualitative research is of great importance, allowing them to investigate issues relevant to them and their communities (Watharow & Wayland, 2022). However, a lack of sight puts scholars with visual impairment at an acute disadvantage, as they encounter substantial difficulties when performing qualitative data analysis (manually or electronically). This is not to suggest that the problem is located within the body of researchers with visual impairment as opposed to what is proposed by the medical model of disability (Marks, 1997). Rather, the social model of disability, which is foregrounded in this paper, states that society is responsible for the problems researchers with visual impairment encounter when conducting their research (Oliver, 2013). Based on this view, companies, programmers, and research institutions are to be held accountable for not offering accessible tools and software that allow researchers with visual impairment to perform their qualitative research at the same speed, comfort, and quality as their sighted peers. Therefore, researchers with visual impairment come up with ‘qualitative workarounds’ to conduct qualitative research. Some examples of workarounds are given below.

Yet, the procedures that precede data analysis in the qualitative research methodology are considered easier for scholars with visual impairment owing to new technological developments. Take, for example, data collection, as researchers with visual impairment can conduct interviews or focus discussion groups either in-person or virtually, with the latter enabling researchers with visual impairment to tackle transportation difficulties. Also, researchers with visual impairment arrange interview dates and times independently using appointment scheduling software, e.g., Zoho Bookings, Square appointments, Setmore, Appointy, and Calendly (Leonard & Main, 2023). Likewise, researchers can use transcription software, e.g., Otter.ai, Fireflies.ai, Descript, or Grain (Zhang, 2022), to transcribe interviews into text. Since there is little information about qualitative data analysis by academics with visual impairment, this topic will be covered extensively in the following sections.

People with visual impairment encounter many challenges to get jobs in research institutions. On top of these challenges are the negative attitudes that university administrators have towards hiring researchers with visual impairment, as many university administrators prefer hiring able-bodied researchers – or what Garland-Thomson (1997) calls “normate”. Also, sighted researchers tend to work with sighted colleagues since academia is dominated by researchers who do not have deficiencies - “Compulsory Able-Bodiedness” in McRuer’s terms (2010). Sighted researchers have high expectations of what visually impaired colleagues can do; consequently, researchers with visual impairment work under heavy stress. Another barrier is that university administrators believe that the accommodations that enable people with visual impairment to perform research tasks are expensive. These accommodations include accessible buildings, assistive technology devices, research assistants, and reduced working hours. Therefore, a Canadian university asked disabled academics ‘not to ask too much’, forcing them to depend on themselves to obtain accommodations (Waterfield et al., 2018). Titchkosky (2010), a disabled scholar, considers ‘bureaucracy’ as the main hindrance to the inclusion of disabled researchers in academia. As a result, researchers with visual impairment such as Lourens (2018) kept “silent” in an academic ecosystem that discriminates against people with disabilities.

The question of who should investigate disability issues has elicited heated discussions in the literature. In the context of disability medicalization, issues of disabled people were examined mainly by non-disabled researchers. Oliver (2013) calls for the ‘emancipation’ of disability studies, urging disabled people to be research creators rather than research subjects. Perceiving disabled researchers as individuals who lack scientific and academic competencies, they were excluded or held inferior positions in the research community. Although the research team of Beazley et al., (1997) included a disabled researcher, the team was led by a non-disabled researcher who was more familiar with the research problem. Barton (2005) calls for creating an ‘inclusive research culture’ that fosters constructive dialogue between the two parties: disabled and non-disabled researchers. However, Duckett and Pratt (2001) point out that people with visual impairment must have control over the research on their issues, as they ‘know best’.

In this article, the author, a researcher with a visual impairment, reflects on his perspective on using computer software in qualitative data analysis. The study that inspired this article (Emara & Haller, 2023) involved interviews with nineteen bloggers with visual impairment. In that qualitative study, the researcher made all interview preparations independently, including preparing the interview guide, recruiting interviewees, arranging interview dates and times, and transcribing interviews. Ultimately, the researcher needed to code interviewees’ responses to identify the main themes. To do so, the author attempted to use Q-DAS, but this approach proved to be inaccessible. Consequently, the author had to analyze the data by using Microsoft Word (MS Word). Thus, examining Q-DAS problems and other methods of qualitative data analysis is the main goal of this article. The article does not only benefit researchers with visual impairment but also benefits elderly scientists, taking universal design principles into account (Goldsmith, 2007).

The author’s identity, the (visual impairment), will remain visible throughout the current article even though disclosing the author’s identity in scholarly articles has been a highly debated issue (Fox & Gasper, 2020; Kerschbaum, 2014; Lugosi, 2006). Some experts, however, have acknowledged the advantages of sharing the author’s personal experiences with academics, professionals, and the public as a whole. Drawing on Karl Heider’s concept of autoethnography, this article presents the researcher’s reflections on conducting qualitative data analysis using computer software (Heider, 1975). Autoethnography is a research method that uses personal experience (“auto”) to describe and interpret (“graphy”) cultural texts, experiences, beliefs, and practices (“ethno” (Adams et al., 2017). Richards (2008) observes that autoethnography is not just narrating the author’s personal experiences; rather, it enables researchers to understand people’s issues from a broad perspective. In line with this conjecture, Bochner (2012) unapologetically argues that researchers should be allowed to tell their living experiences in academic works instead of depending solely upon empirical research. Thus, Chang (2016) maintains that auto ethnographers tend to voice the views of their community (s). I, therefore, reveal my disability throughout this article. I am doing so, following other authors who elected to disclose their disabilities in scholarly articles (e.g., Deegan, 2010; Lourens, 2021; Michalko, 2001).

Qualitative Analysis

Smit defines qualitative analysis as “[taking] apart words, sentences, and paragraphs, which is an important act in the research project to make sense of, interpret and theorize such data” (Smit, 2002, p. 67). Qualitative research includes multiple tasks, i.e., collecting, revising, categorizing, and coding data. In the following sections, I will focus only on discussing qualitative data analysis by researchers with visual impairment (for more about other phases of qualitative research, see Willig & Rogers, 2017).

Generally speaking, qualitative data analysis can be carried out in many different ways. Historically, qualitative analysis is, for the most part, performed manually. In the manual analysis, researchers had to utilize various tools, such as card indexes, different-colored pens, scissors, and tapes (Seale & Rivas, 2012). In addition to the cost, analyzing qualitative data manually requires much time, money, and effort (Miles & Huberman, 1994). Using manual techniques, researchers struggle to identify complex relationships between data and draw a broad picture of the participants’ responses. In contrast, conducting analysis using Q-DAS packages ¹ is much easier, allowing researchers to perform functions such as searching, grouping, categorizing, and visualizing data (Weitzman & Miles, 1995). Likewise, Drisko (1998) suggests that computer-aided-qualitative analysis helps researchers handle many tasks, such as text segmenting, coding, annotating, and theory development. Hence, using computerized qualitative data analysis results in more effective outcomes, saving the researcher’s time, effort, and money.

Qualitative Analysis by Researchers with Visual Impairments

Generally, researchers with visual impairments are supposed to rely on interviews as well as other qualitative methods because conducting interviews and focus discussion groups does not require normal sight. This is not the case for surveys or observations which require sharp sight. Further, handling qualitative analysis appears to be more convenient for people with visual impairment than quantitative analysis. That is to say, people with visual impairment can interpret textual data, at some point, more quickly than numeric data. This is not meant to suggest that researchers with visual impairment cannot analyze quantitative data at all, as Foulke found that students with visual impairment understood mathematical expressions and numeric data clearly (cited in Mack, 1984). Despite that, researchers with visual impairment are confronted by many issues related to exploring and visualizing numeric data.

Data Exploration

People with visual impairment recognize and process data in a different way and size from those of sighted people. One factor affecting how people with visual impairment make sense of data is the amount of information their brain can process and/or retain. Contrary to sighted people who can identify relationships between large chunks of information (Cattaneo & Vecchi, 2011; Edmonds & Pring, 2006), those with visual impairment interpret small amounts of information at once as a result of scanning information slowly (cited in Krueger & Ward, 1983). In this context, people with visual impairment cannot count only on memory to analyze data (Gual et al., 2014; Mack, 1984; Raz et al., 2007), especially in studies that generate large datasets.

Another key factor that influences data exploration is the format in which information is presented. People with visual impairment access data in numerous modes, including hard-copy braille, audio files, or electronic files. There are significant differences between these forms, though people with visual impairment have to explore the full breadth of data in small amounts (Buzzi et al., 2010). This is due to the fact that hearing and touch have limited recognition capacity, and thus, for example, the average speed of braille readers is low (Simon & Huertas, 1998). For the same reason, the average reading speed of the screen reader software ² is low. To increase the amount of information people with visual impairment absorb, some individuals elect to slow their reading speed or re-read the same portion of the text several times. In addition to wasting a lot of time, people with visual impairment still struggle to detect the relationships between choppy waters of data. This requires people with visual impairment to exert greater mental effort, thereby resulting in experiencing mental stress. However, the more researchers with visual impairment revisit the data, the more ‘engaged’ they will be, reaching a higher level of inquiry.

Using Braille in Qualitative Data Analysis

To analyze the data manually, researchers with visual impairment should first transcribe interviews and focus discussion groups into braille ³ . People with visual impairment use conventional braille writing tools, such as slates, stylus, and Perkins Brailler (Heller et al., 1998). This approach is considered outdated, inefficient, and time-consuming. Alternatively, researchers with visual impairment could use braille embossers to convert electronic files into hard-copy braille easily, though braille embossers are prohibitively expensive ⁴ . This makes researchers reluctant to use Braille since this system has many problems, such as low redundancy, code complexity, cognitive factors, limited scanning strategies, and space and bulk requirements (Tobin & Hill, 2015). Despite the abovementioned shortcomings, people with visual impairment can recall information in braille easily and thereby explore the data efficiently (Goudiras et al., 2009). Researchers can reread portions of the data several times since it is printed in braille (Marshall & Moys, 2020). In contrast, researchers cannot do so if the data is available in audio (Hartley, 1990) or electronic format. Hence, governments should take effective actions to revive braille literacy and researchers should suggest ways to solve the problems of the braille system (Allban & Emara, 2017; Emara, 2022, 2022b).

Qualitative Data Analysis Software

Researchers use Q-DAS software to make sense of data gathered through interviews or focus discussion groups. Researchers utilize Q-DAS for data entering, code assigning, and memo writing. In Bazeley and Jackson’s (2013) terms, Q-DAS allows thinking, linking, writing, modeling, and graphing. Oswald (2019) underlines two advantages of QDA software, that is, protecting participants’ privacy and enabling the research team to work simultaneously on the same project. However, the merits of Q-DAS packages should not be overestimated, as Ose (2016) contends that Q-DAS is regarded as a tool for “structuring data.” In this respect, the software does not offer a deep analysis or explanation for the issue/phenomenon under investigation (Silver & Lewins, 2014), rather it offers a preliminary analysis of it.

Using Screen Reading Software in Qualitative Data Analysis

People with visual impairment use screen reading software to access the information shown on a computer screen. The most popular screen readers are JAWS, NVDA, and WindowEyes. Pascual et al. (2014) found that people with visual impairment completed a few computer tasks successfully and took time longer than sighted people. Given that people with visual impairment face many difficulties when using a computer in general (Goggin et al., 2019; Vanderheiden et al., 2022), the use of Q-DAS seems a useless technique for this group. The main problem facing researchers with visual impairment is how to pull the data into Q-DAS software. While sighted researchers enter the data into the software using a computer mouse “dragging,” researchers with visual impairment cannot utilize this method because they rely heavily on a computer keyboard to enter the data (Potluri et al., 2018). Unfortunately, Silver and Lewins (2014) found that Computer Assisted Q-DAS packages are always run by a mouse.

Another problem is that most commercial Q-DAS packages display information on popup screens which are inaccessible to users of screen readers (Lazar et al., 2007). Users with visual impairment cannot access popup screens because they appear and disappear quickly. Examples of the problems facing Q-DAS users with visual impairment include:

(1) Most programs used in qualitative data analysis are incompatible with screen reader software.

As a result of Q-DAS being inaccessible, researchers with visual impairment seek help from sighted friends and colleagues when analyzing qualitative data. In essence, a scholar with a visual impairment might ask a sighted person to perform the analysis using Q-DAS software or read the analysis outputs shown on the screen. Along with the social cost induced by help-seeking (Frank, 2003), this technique reduces the reliability of the analysis. Rather, the data should be coded and analyzed by the researcher (s) who conducted the study because they are familiar with the study’s main variables, objectives, and other relevant details. Hence, researchers with visual impairment could lose what Gilbert (2002, p. 215) calls the “closeness to the data.” Other problems encountered by users of screen reader software with Q-DAS will be discussed in detail in the following sections.

Data Visualization

Q-DAS packages display the data visually in the form of illustrations, graphs, charts, and diagrams, posing many problems to researchers with visual impairment (McDonald & Rodrigues, 2016). Yet, Q-DAS software does not offer a description of the visual information shown on the screen. Q-DAS software even does not present data in tables despite being the most accessible data visualization form for people with visual impairment (Sauter, 2015). In another vein, some Q-DAS packages enable researchers to mark segments of data with different colors, making the outputs of data analysis inaccessible to researchers with visual impairments. In addition, the Q-DAS software lacks adequate color contrast between foreground and background, posing great challenges for researchers with low vision (Hristov et al., 2022). Further, some researchers utilize formatting attributes (e.g. underline, italic, and bold) to differentiate between categories. This marking technique is considered hard to identify for people with visual impairment (Jackson & Bazeley, 2019). Other issues of using formatting attributes in highlighting segments of the data are addressed in the next section.

Using Microsoft Word in Qualitative Data Analysis

Researchers with visual impairment can use word processors to code data. Qualitative researchers utilize MS Word functions in data analysis, such as inserting a table, sorting a table, and inserting a comment (La Pelle, 2004). As researchers categorize data, they utilize the find function to search for specific words, phrases, or sentences (Drisko, 1998). However, Q-DAS packages have search capabilities better than MS Word, enabling researchers to retrieve associated codes based on word synonyms, word roots, and related words (Gibbs, 2014). Unlike MS Word, Q-DAS software enables researchers to search for two keywords simultaneously by using Boolean operators (e.g., and/or) (Seale & Rivas, 2012). But, is MS Word considered a feasible alternative to Q-DAS programs?.

Because MS Word has many problems, researchers with visual impairment are perhaps reluctant to utilize this software in different phases of qualitative analysis. In the data exploration phase, screen reader users take much time to read the information stored in MS Word documents (Baker et al., 2021). Moreover, people with visual impairment are not able to employ fast-reading techniques such as skimming (Machulla et al., 2018). Thus, MS Word users with visual impairment adjust the screen reader settings to read the text without its format. Users do so to save time, as reading each word and its format could take longer time. This sacrifices text comprehension even though it is germane to data exploration.

In the phase of data analysis, people with visual impairment might use MS Word formatting tools to make each category/theme distinguishable. Coding data with MS Word is considered an arduous task, though recognizing relationships between the data is rather harder. A practical solution is to transcribe each interview into a separate MS Word, and move between MS Word documents rather than scrolling up/down a lengthy MS Word document (Silver & Lewins, 2014). However, future research should examine the feasibility of this solution.

If sighted researchers analyze data using MS Word, reading the analysis outputs will be inaccessible for researchers with visual impairment (Morales et al., 2013). Sighted researchers highlight codes in bold, but researchers with visual impairment cannot rely on this method because the screen reader does not inform the user about the highlighted text. Sayama et al. (2023) suggested using different sound pitches to allow users with visual impairment to identify highlighted words, but this technique has yet to be widely adopted. The use of colors is another coding tool that is regarded as problematic for screen reader users. An established coding technique is to give the passages of each theme a different color, but blind researchers cannot detect color variations without checking the color of each word. Sighted researchers, in contrast, can identify paragraphs presented in different color schemas easily by looking at a computer screen.

Subsequently, communicating relationships between data through written notes or comments by/to researchers with visual impairment would solve the abovementioned problems. Track Changes ⁵ is another potential solution to this issue. A suggested data analysis technique that utilizes MS Word features is described below.

Using Microsoft Excel in Data Analysis

More often than not, people with visual impairment perhaps prefer using MS Word over Excel. Microsoft Excel may seem unfriendly to people with visual impairment since the user has to make sense of large amounts of data arranged in rows and columns. The main problem of Excel facing people with visual impairment is that information is presented in a grid rather than a list view. Data exploration, therefore, would be challenging, especially if Excel tables are not accompanied by headers (Abu Doush et al., 2009). Other accessibility problems of Excel tables include split cells, merged cells, or nested tables, and thus researchers with visual impairment have to spend a lot of time reading the data in Excel. Excel users with visual impairment may delete the cell content accidently by pressing any button because the cell content remains highlighted until the user moves to the next cell (by pressing the tap key). Sighted users can detect this error easily, while users with visual impairment will not be able to discover this error unless they read the entire Excel sheet again (Doush & Pontelli, 2013).

Researchers who use Excel encounter difficulties in identifying relationships between data, especially if data is scattered across many cells. Previous studies introduced methods for analyzing qualitative data using Microsoft Excel (e.g., Amozurrutia & Servós, 2011; Meyer & Avery, 2009; Ose, 2016), which, to some degree, are considered difficult to undertake by researchers with visual impairment. Most techniques suggested in previous studies require using a computer mouse. Even if these techniques allow users of screen readers to use the keyboard, they must memorize numerous keyboard shortcuts, as each Excel function is performed by a different keyboard shortcut. In addition to suffering acute cognitive load, users may confuse Excel keyboard shortcuts with those of MS Word or other software (Ampratwum et al., 2016).

A Suggested Technique for Qualitative Data Analysis

Some experts overestimate the importance of computational qualitative methods as if a person cannot be a perfect qualitative researcher without mastering Q-DAS software. However, Gibbs (2014) contends convincingly that Q-DAS might not be a feasible research method, but it is simply a useful tool for structuring data. In Gilbert’s terms, though, researchers with visual impairment who use Q-DAS may be vulnerable to the “coding trap” because they cannot scan large sets of codes simultaneously to identify recurrent patterns. Another setback of the Q-DAS is that researchers are supposed to have normal vision to be able to perform computerized analysis tasks (Gibbs, 2014, p. 284). Hence, researchers with visual impairment should rely on qualitative data analysis methods that depend on textual rather than visual elements. To group the data under subcategories, researchers with visual impairment can use MS Word features, such as hyperlinks, headings, and subheadings (Ose, 2016).

As for distinguishing between categories, I suggest the use of different punctuation marks which are easy to trace by people with visual impairment. For example, researchers may insert the passages of one category between two brackets and insert the passages of another category between two parentheses. The rest of the punctuation marks can be adopted in the same way. Punctuation marks allocated for specific grammatical usage should be excluded. One thing to consider is the need for the research team to agree on the assignment of specific punctuation marks to certain categories and define them in the codebook before analysis. Nevertheless, if researchers with visual impairment work individually on a research project, the definition of the assigned punctuation marks should be provided in the study protocol. To search for a particular category to analyze, users of screen reader software can use the assigned punctuation mark.

Alternatively, researchers can give the passages of each category a sequence of numbers, and then retrieve a particular category that has the same sequence of numbers. Although punctuation marks and numbers are difficult to identify by researchers with visual impairment compared to illustrations and colors used by sighted researchers, researchers with visual impairment will become familiar with the suggested method by repeated practice. The suggested method allows non-sighted researchers to collaborate with sighted researchers who use computerized data analysis tools. Thus, automated data analysis methods should be tailored to the needs of researchers with visual impairment if the inclusion of this group in academia is important.

In addition, the proposed method can be adapted to suit researchers with various eye problems. Undoubtedly, this technique should work for elderly scientists, who, due to many health issues, would lose sight partially or completely as they get older. Thus, the suggested method ensures the inclusion of visually impaired researchers in academic institutions and guarantees that qualitative scientists will not be hampered from doing their seminal research. Future empirical research should examine the validity and practicality of the suggested technique.

Discussion

To the best of the author’s knowledge, this is the first article to discuss the challenges faced by researchers with visual impairment who use computer-aided qualitative research methods. Drawing on the autoethnography method, I, as a researcher with visual impairment, explain why it is difficult for me and my peers to use Q-DAS software. As Q-DAS software displays the data visually, researchers with visual impairment are hindered from collaborating with sighted researchers or joining research teams made entirely of sighted scholars. In addition, the Q-DAS entry method (a mouse) is not as accessible as the MS Word entry method (a keyboard). This perpetuates the traditional view of people with visual impairment as a minority group, having a medical problem, and having to grapple with their problems (Siebers, 2008).

I suggest that MS Word is the most accessible tool for qualitative data analysis. Using punctuation marks and numbers, researchers with visual impairment could have a variety of tools to categorize data and identify overarching themes. With only a few minor accessibility problems, researchers with visual impairment can share comments with colleagues and track changes made by other researchers using MS Word. But, of course, using visual coding methods by researchers with visual impairment is considered a far-reaching task. In the future, researchers with visual impairment might have the possibility to grasp data displayed visually by relying on alternative text (alt-text) (Mack et al., 2021; Wu et al., 2017).

These scientific endeavors take time to give promising results. Thus, Q-DAS developers and companies should make significant modifications so that researchers with visual impairments can use the software easily. This undoubtedly is better than designing qualitative data analysis software catered specifically for the needs of researchers with visual impairment. As such, inclusive and universal design fundamentals should be taken into account when developing Q-DAS as well as other software packages (Persson et al., 2015).

Inclusion and inclusivity should not be an afterthought, as companies should hire programmers with visual impairment and allow them to participate in designing accessible Q-DAS software in the first place. Previous studies have shown that people with visual impairment are efficient programmers (Damasio et al., 2019; Kane & Bigham, 2014). This echoes James Charlton’s (1998) clarion words “Nothing about us without us”. Companies should also realize that if their programs become more accessible to people with visual impairment, more revenues can be generated (Haller & Ralph, 2006). Considering them as potential users of Q-DAS software, future research should investigate the views of researchers with visual impairment regarding common accessibility problems of Q-DAS software. Future research should also focus on evaluating the overall accessibility of the software used for collecting, storing, visualizing, and analyzing qualitative data.

Implications for Practitioners

The main goal of this article is to encourage students, teachers, and scholars with visual impairment to conduct qualitative research, ending years of exclusion and marginalization of this group from academic institutions. As universities move hesitantly toward recruiting researchers and instructors with visual impairment, this article showcases the measures undertaken by a researcher with a visual impairment to solve the problems of automated qualitative data analysis tools. Influenced by preconceived notions, people with visual impairment have been seen as unable to carry out qualitative research independently. With developing accessible research methods, more opportunities will be offered for the benefit of this community.

Conclusion

The present article sought to investigate the problems faced by researchers with visual impairment in using Q-DAS software. Based on the real experience of the author, it can be concluded that the Q-DAS is difficult to use because it is mainly designed for sighted people. The article calls for Q-DAS companies to take prompt action to improve the overall accessibility of their software. In the meantime, researchers with visual impairment can utilize the MS Word formatting features to code, categorize, visualize, and analyze qualitative data.