User experience of mainstream and assistive technologies for people with visual impairments

Abstract

BACKGROUND:

There is lack of understanding of “user experience”; i.e. how people with visual impairments use today’s emerging mainstream technologies along with assistive technologies, which would ultimately lead to less user-friendly designs.

OBJECTIVE:

The primary aim of this study is to advance knowledge of user needs and concerns about usability of mainstream and assistive technologies.

METHODS:

The archival research method was used to obtain data via online forum sites. After reviewing six forum sites, this study has found 92 messages that specifically discussed the use of technology, and reviewed the messages by comparing them with ten usability heuristics by Nielsen.

RESULTS:

The majority of usability problems were associated with flexibility and efficiency of use; help for users to recognize, diagnose, and recover from errors; and help documentation.

CONCLUSIONS:

It is imperative to focus on improving usability of both assistive and mainstream technologies in use by people with visual impairments.

Keywords

Visual impairments assistive technology usability accessibility

1. Introduction

Over 21 million adults in the United States are visually impaired [1] and every year 75,000 more Americans are predicted to be visually impaired [2]. Many of those with visual impairments were born with intact vision but lost their vision as a result of eye diseases or health problems [3]. It has been reported that more than 158,000 veterans in the United States are visually impaired, and each year approximately 7,000 veterans are likely to lose their vision, ultimately becoming blind [4]. As the population ages, age-related eye diseases would further increase the population of those with visual impairments over the next 30 years [5].

Today, many people with visual impairments take advantage of various assistive technology applications for their work and activities of daily living. For instance, 24.6% of people with visual impairments have a full-time job [6], and today’s corporations typically conduct their daily business affairs through computers and the Internet [7]. Likewise, it is anticipated that using the Internet is essential and critical for those with visual impairments to accomplish their jobs. Further, it is well documented that the Internet and assistive technology play a very important role in helping people with disabilities to achieve their personal and educational objectives [8, 9].

Numerous studies [7, 10, 11, 12] have made considerable efforts to advance the quality of assistive technology applications for users with visual impairments. By using the assistive technology applications, more people with visual impairments can be anticipated to use more various mainstream technology applications. Yet, while there are a good number of previous studies on assistive technology needs among people with visual disabilities, very little is known about their needs for today’s mainstream technology and its usability, especially when they use assistive technology applications in sync. When it comes to usability, designers typically tend to focus on users who have no disabilities. When it comes to users with disabilities, designers tend to think only of accessibility. Thus, designers are likely to disregard usability for users with disabilities, especially for mainstream products. Good accessibility does not guarantee good usability for users with disabilities.

Improving the usability of a target system for those with special needs is a critical design issue. If those users with special needs are dissatisfied, the affect does not simply end with users’ frustration because they will be more likely to abandon the unsatisfactory system and eventually fail to achieve independence in everyday life. In addition, their vocational opportunities and job security could also be negatively influenced. Many employers have reported that providing proper accommodations (e.g., user-friendly assistive technology) for employees with disabilities has resulted in multiple benefits, enabling their company to retain qualified employees, eliminate the costs of training new employees, and increase workers’ productivity [13]. Those with special needs should be given an equal opportunity to develop employable skills and obtain jobs. Properly designed technology applications can contribute greatly toward this goal.

Many disability studies on information technology have been conducted by inviting people with visual impairments. However, the research participants were often individuals who would not use (or were not much interested in using) technology in their daily lives. The International Organization for Standardization (ISO) 9241-11 has defined usability as the extent to which a product can be used by specific users to achieve specified goals with effectiveness, efficacy, and satisfaction in a specific context use. Based on the ISO definition of usability, it is essential to target the right user groups. There is a need to focus more on people with visual impairments who are using or interested in using information technology. Thus, such usability research would be able to identify more accurate user requirements, which would ultimately result in designing and developing a more sophisticated technology for those with special needs. This study aims to explore how users with visual impairments use mainstream technologies along with assistive technologies.

2. Methods

2.1 Data collection

Between January and December 2017, this study investigated online forum sites where people with visual impairments share their concerns and needs. The forum sites include American Foundation for the Blind, Vision Aware, Family Connect, Daily Strength, Action for the Blind People, and Royal National Institute of Blind People. The data were obtained and analyzed using the archival research method [14]. There was no interaction and interference with the forum users’ online activities. The following approaches were used for data collection: access to the forum sites was public; information was identifiable, but not private; and information gathering required no interaction with the person who posted it online. After reviewing the major forum sites (i.e., 1183 posts of American Foundation for the Blind, 360 posts of Vision Aware, 1072 posts of Family Connect, 509 posts of Daily Strength, 1360 posts of Action for the Blind People, and 193 posts of Royal National Institute of Blind People), we found 92 forum posts that specifically discussed the needs and concerns about using mainstream technology applications via assistive technology applications. The other posts, which were more related to healthcare (vision loss/eye care), education, daily life, social life, and so on, were excluded.

2.2 Data analysis

The present study referred to the Nielsen’s usability heuristic checklist [15] in conducting the content analysis of the Internet forum postings (see Table 1) [16, 17]. An initial post (e.g., questions, needs, or concerns about usability problems) and its threads (e.g., responses, answers or opinions) were compared with the heuristic checklist to discover what caused poor user experience. An inter-rater reliability analysis using Cohen’s kappa was performed to determine consistency between the two raters. There was substantial agreement among the raters as the inter-rater reliability was found to be $\kappa=$ 0.75 (95% CI: 0.553 to 0.970).

Table 1
Nielsen’s heuristic checklist [15]

Heuristics	Descriptions
Visibility of system status	The system keeps users informed about what is going on through appropriate feedback within reasonable time.
Match between system and the real world	The system should speak the user’s language, with words, phrases, and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
User control and freedom	Users often choose system functions by mistake and will need a clearly marked “emergency exit” to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
Consistency and standards	Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Error prevention	Even better than good error messages is a careful design that prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
Recognition rather than recall	Minimize the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
Flexibility and efficiency of use	Accelerators – unseen by the novice user – may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
Aesthetic and minimalist design	Dialogues should not contain information that is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
Help users recognize, diagnose, and recover from errors	Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
Help and documentation	Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large.

3. Results

In this paper, those who posted a message on the web forums that we investigated will be referred to as participants with visual impairments. Despite the visual challenges, participants with visual impairments used a variety of mainstream technologies such as smartphones, tablet computers, social media, cloud data storage, multimedia players, audio editors, and so on. Those with visual impairments using the forums have questioned, answered, and exchanged information and knowledge related to various usability issues, including “visibility of system status”; “user control and freedom”; “consistency and standards”; “error prevention”; “recognition rather than recall”; and “aesthetic and minimalist design.” Top three usability problems (71%) are associated with “flexibility and efficiency of use”; “help users recognize, diagnose, and recover from errors”; and “help documentation” (see Table 2).

Table 2
List of usability issues

Usability issues	Posting (%)
Help users recognize, diagnose, and recover	30 (33%)
from errors
Help and documentation	22 (24%)
Flexibility and efficiency of use	13 (14%)
Aesthetic and minimalist design	7 (8%)
Recognition rather than recall	5 (5%)
Error prevention	4 (4%)
Consistency and standards	3 (3%)
Match between system and the real world	4 (4%)
Visibility of system status	2 (2%)
User control and freedom	2 (2%)
Sum	92 (100%)

3.1 Help users recognize, diagnose, and recover from errors

The participants with visual impairments have used various technologies. Technologies include assistive technologies (e.g., optical character readers, screen readers, and Braille note takers), communication applications (e.g., email and chatting tool), music applications (e.g., Flash player, iTunes and MP3 players), networking systems (e.g., Bluetooth and Wi-Fi), office suites, online search/shopping, portable devices (smartphone, Kindle, and tablet PC), and operating systems (e.g., Windows and iOS). User interactions with those various technologies also brought various errors leading to poor user experience. The errors occurred “before”, “during”, and “after” the participants with visual impairments used the technologies.

3.1.1 Before using technology applications

The participants with visual impairments encountered several challenges even before using technology applications. For example, when those with visual impairments wanted to open an account (or register) to use a new application, they were asked to fill out some required forms (e.g., username, password, birthday, and location) so that they used a screen reader application to complete the form. However, the screen reader could not properly complete the form because its system could not often work with various data entry modules (e.g., a text field and a combo box). When those with visual impairments also tried to download and install a new software program onto their computers, they often failed to do so because system user guides have paid less attention to the situation in which a user who is visually impaired and has lack of technology literacy tries to install a new program and/or troubleshoots to recover from any relevant errors. The following excerpt shows a good example: “Hi everyone, I am using the NVDA screen reader for Gmail. I wanted to add an extra Gmail address to my Google account but cannot seem to satisfy the requirement for adding birthday information, as it is not in a field or a combo box. They cannot readily be heard. Is there a way to address this?” Those with visual impairments also had a difficult time in finding a way to turn on the accessibility features of mainstream technology: “How do I turn on the accessibility features on an iPhone?” Those mainstream technology designers and developers improved accessibility by providing alternative ways for those with disabilities to use the mainstream technology but incorrectly assumed that those with disabilities know how to turn on the accessibility features that are not accessible to those with disabilities.

3.1.2 While using technology

The participants with visual impairments were also frustrated with various errors while using technologies. For example, their assistive technologies were often not compatible with mainstream technologies due to delayed system updates; could not control the cursor of the screen reader while using MS Word; could not convert PDF to Word; could not remove password encryption from a document; and could not save an Office file as an earlier/later version. The following excerpt shows an example of errors: “For several weeks, Google mail has had a message advising one to upgrade their browser soon, as Gmail and other features may not be well supported with changes apparently being implemented by Google. Google Chrome does not seem to work with JAWS software as well. I am currently using IE8, and not sure whether it is compatible with JAWS. If anyone has recommendations for this issue, please pass them on.” In addition to applications, those with visual impairments had a poor user experience using an operating system (e.g., Windows). For example, they lost Wi-Fi connection but failed to reconnect or failed to sync files properly between a mobile device and a desktop computer.

3.1.3 After using technology

The participants with visual impairments often failed to close an application of mainstream technology via assistive technology such as screen readers after completing their tasks. For example, one participant complained about closing the Microsoft Word application using JAWS and NVDA: “The keyboard shortcut CTRL+F6 works sometimes, but other times, the only way is to close IE and simply reload the page. I think Microsoft is dropping the ball and the screen readers JAWS and NVDA keep introducing more bugs. All these toolbars and ribbons are not improving the experience for the screen reader user.”

3.2 Help and documentation

When the participants with visual impairments encountered usability problems, they sought detailed guidance to learn how to resolve a problem by themselves or learn more about new or advanced features. For example, those with visual impairments noted that “I was a JAWS and PC user for over 15 years. However, in the past week I switched to using NVDA on my PC. Now I am struggling to master its usage quickly. Can other new NVDA users please share your experiences? I’m feeling very discouraged!” and “I just got a VICTOR Stream. It is the new version with wireless connection to the Internet. I have to enter the password for my home network but I cannot figure out or find in the user manual how to enter capital letters in my password using the numeric keypad on the Stream application.” As there is lack of educational guides accessible to those with visual impairments, they have often learned via trial-and-error approach: “(original posting) Is there any tutorial on how to use the iPhone? (Responding to the original posting) Hi there, I am not sure if this helps, but I learned a lot of how to use the iPhone just through experimentation.”

3.3 Flexibility and efficiency of use

When the participants with visual impairments wanted to customize the features of technology based on their preferences and computer literacy, it was often not easy to change the default settings: “Reading with any assistive equipment is physically grueling. I am interested in software that will support use of the vision I have by letting me mark a box I want to read and then command the speech software to read that box only. I want to avoid skipping out and reading into other parts of the page that I am not interested in at that moment.” Those with visual impairments also appreciated the system that provided multiple options to choose based on the users’ various needs: “One of the real strengths of Android is the number of alternatives that are available to the end user. As an example, there are more than one screen-reader to choose from the Google-Play store, such as Talk Back and Shine Plus.” As many mainstream technology applications today are manufactured to be equipped with assistive technology features, those with visual impairments are more likely to use various mainstream technologies without purchasing any additional assistive technology applications: “In my opinion, one of the best new functions of the iOS 10 operating system for the Apple iPad and iPhone is the magnifier. This could be very useful if you forget a hand-held portable magnifier, and the iOS 10 operating system has lots of features including brightness, contrast, and color filters as well as the magnification.”

3.4 Additional usability problems

The other usability issues were associated with “inconsistency”, “mismatch between system and the real world”, “recognition and recall”, and “minimalist designs.” The participants with visual impairments also appreciated simplicity of designs: “My eyes are not very good and my ability to learn is thus reduced. This makes the new Excel very challenging. Are there any alternatives? I need simplicity and legibility more than extra functions.” Those with visual impairments were also concerned about inconsistent designs among screen reader applications, leading to a difficult time to learn the system: “My biggest issue seems to be transitioning from JAWS commands to NVDA commands and just have to keep reminding myself that JAWS may use control XYZ to initiate a command ABC; on the other hand, NVDA uses the insert key as well to execute the given command.” As most mainstream technology applications tend to be designed by targeting the majority of users, i.e., sighted people, the user interfaces (e.g., icons) and layouts are often designed to resemble real-life artifacts that are seen by the sighted people. Thus, the interfaces and layouts tend to be more familiar to sighted users than users with low vision or residual vision. Those with visual impairments are likely to encounter a challenge in understanding the system’s user interfaces and layouts when they first use the system: “I have trouble with things like ATMs – it’s the unfamiliarity of layout for me. It sounds ridiculous and a total breach of privacy in a way, but when withdrawing from an ATM, I have someone with me. I need time to play around with and examine things which you can’t really do out in the big wide world.”

While technology is more advancing, those with visual impairments feel being more excluded from the mainstream technology: “My mother is becoming completely blind and has minimal feeling in her hands. She has had analog cell phones for years, but as the buttons have become smaller and smaller, she has had an increasingly difficult time finding a phone that works for her. Also, she is concerned about how she will be able to plug in a micro-USB cord to charge the phone, so a Samsung phone has a wireless charging system that would be good for her.” Error prevention interventions such as making the font larger would be effective and efficient for those with visual impairments, particularly low vision.

4. Discussions

As a variety of assistive technologies have been advancing today, people with visual impairments have been able to use more diverse mainstream technologies, such as smartphones, tablets, cloud data storage, and multimedia players. While they use those technology applications, they often encountered not only accessibility but also usability problems and were clueless as to how to resolve the problems on their own. Those with visual impairments took advantage of the forum sites by asking and answering questions with regard to the use of mainstream and assistive technologies.

Based on the Nielsen’s heuristic checklist, the majority of usability problems (71%) were associated with “flexibility and efficiency of use”, “help users recognize, diagnose, and recover from errors”, and ”help documentation.” For instance, the screen reader application gave the participants with visual impairments flexibility to change various features (e.g., between browsing mode and data entering mode), but it was often not easy and inefficient for them to change the features. Simply providing the flexibility of changing features of an application does not mean that the application is user-friendly. Steinebach et al. [18] introduced a design principle of the flexibility-usability tradeoff; that is, as the flexibility of a system increases, its usability decreases. Too much flexibility without a careful consideration of the target users’ needs and abilities/disabilities could lead to complexity and usability problems.

The participants with visual impairments also did not know how to resolve an error or even understand what kind of errors they encountered. For example, although the mainstream technology is equipped with well-developed accessibility features, it is ironically difficult for users with visual impairments to find out how to turn on the accessibility features. Those with visual impairments did not even know whether an accessibility feature was available for the application that they were using. The latest empirical study by Bigham et al. [19] reported that users with blindness were likely to be confused between “not knowing” if information that they seek are available on a web page and “inaccessibility”, ultimately leading to user frustration. There is a need to develop a system (e.g., troubleshooting processes from a perspective of visually impaired users) that helps users obtain a better understanding of what kind of errors they have and how to recover from errors.

Another significant problem is that there is a lack of educational guides or self-help resources, accessible to those with visual impairments. The existing guides/manuals that contain a lot of figures were designed by targeting sighted users. Problem solving steps and explanations were also designed with the assumption that the system is used by users who can see. Due to such unusable help-resources, the participants with visual impairments have typically relied on a trial-and-error strategy. In education domain, instructors often also teach visually impaired students via the trial-and-error strategy, but the effectiveness is doubtful, as the strategy is sometimes helpful, but other times fails to facilitate the teaching and learning activities [20]. More user-centered designs would be needed.

5. Conclusion

This study found that those with visual impairments often encountered usability problems in using both assistive and mainstream technologies together. In general, while usability tends to focus on sighted people using mainstream technologies, accessibility focuses on people with visual impairments using assistive technologies. Such separate attention would likely result in poor usability of assistive and mainstream technologies for users with visual impairments. This study leads to the conclusion that there is a need to pay more attention to improving usability of both assistive and mainstream technologies in use by those with visual impairments. Based on the results of this study, recommendations of how to improve the usability for users with visual impairments are briefly summarized: Various text entry modes should be provided; A new software program should be easily downloaded and installed; It should be easy to find and turn on the accessibility features; User-friendly manuals in accessible formats should be available to facilitate self-learning; The default settings should be easily adjusted; It should be easy to manage the operating system (e.g., system updates, data protection, and password security); More consistent user interface designs should be offered across different screen reader applications; and Troubleshooting steps should be carefully designed for the visually impaired to easily understand and follow. User-centered design principles should equally be applied to sighted users and visually impaired users instead of merely providing assistive technology.

Footnotes

Conflict of interest

None to report.

References

Centers for Disease Control and Prevention, Summary Health Statistics for U.S. Adults: National Health Interview Survey, 2011. Vital and Health Statistics, 2012. 10(256): 1-218.

Lighthouse International. Prevalence of Vision Impairment. 2015 [cited 2015 June 15]; Available from: http://li129-107.members.linode.com/research/statistics-on-vision-impairment/prevalence-of-vision-impairment/.

Orrico,

K.B.

Caring for visually impaired patients. Pharmacy Today, 2013. 4: 80-88.

Blinded Veterans Association. Information for Veterans Coping with Vision Loss. 2016 [cited 2016 August 23]; Available from: http://www.visionaware.org/info/everyday-living/essential-skills/information-for-veterans-coping-with-vision-loss/123.

National Federation of the Blind. Blindness and Low Vision. 2015 [cited 2015 May 28]; Available from: https://nfb.org/fact-sheet-blindness-and-low-vision.

Erickson,

Lee,

and von Schrader,

2012 Disability Status Report: United States. 2014; Available from: http://disabilitystatistics.org/reports/2012/English/HTML/report2012.cfm?fips=2000000&html_year=2012&subButton=Get+HT.

Inan,

F.A.

, et al. Internet use and cybersecurity concerns of individuals with visual impairments. Journal of Educational Technology & Society, 2016. 19(1): 28-40.

Brady,

, et al. Visual Challenges in the Everyday Lives of Blind People. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2013. ACM.

Hersh,

and Johnson,

M.A.

Assistive Technology for Visually Impaired and Blind People. 2010: Springer Science & Business Media.

10.

Ari,

I.A.

and Fethi,

A.I.

Assistive technologies for students with disabilities: A survey of access and use in turkish universities. TOJET: The Turkish Online Journal of Educational Technology, 2010. 9(2).

11.

Kim,

H.N.

Smith-Jackson,

and Nam,

C.S.

Elicitation of haptic user interface needs of people with low vision. International Journal of Human-Computer Interaction, 2013. 29(7): 488-500.

12.

Michalska,

A.M.

, et al. Accessible web page design for the visually impaired: A case study. International Journal of Human – Computer Interaction, 2014. 30(12): 995.

13.

Hendricks,

D.J.

Batiste,

and Hirsh,

Cost and effectivenss of accomodations in the workplace: Preliminary results of a nationwide study. Disability Studies Quarterly, 2005. 25(4).

14.

Cuffaro,

M.A.

Archival Research, in: Encyclopedia of Child Behavior and Development Goldstein,

and Naglieri,

J.A.

Eds. 2011, Springer US: Boston, MA. p. 140-141.

15.

Nielsen,

10 Usability Heuristics for User Interface Design. 1995; Available from: https://www.nngroup.com/articles/ten-usability-heuristics/.

16.

Krippendorff,

Content Analysis: An Introduction to Its Methodology. 1980, London: Sage.

17.

Strauss,

and Corbin,

Basics of Qualitative Research: Grounded Theory Procedures and Techniques. 1990, Newbury Park, CA Sage.

18.

Steinebach,

Guhathakurta,

and Hagen,

Visualizing Sustainable Planning. 2009: Springer.

19.

Bigham,

J.P.

Lin,

and Savage,

The Effects of “Not Knowing What You Don’t Know” on Web Accessibility for Blind Web Users. In The 19th International ACM SIGACCESS Conference on Computers and Accessibility. 2017. Baltimore, MD.

20.

American Foundation for the Blind. Educating Students With Visual Impairments for Inclusion in Society. 2017 [cited 2017 October 31]; Available from: http://www.afb.org/info/teachers/inclusive-education/35.