Development of a Multisensory Nonvisual Interface to Computers for Blind Users

The advent of graphic-based display environments such as those found on the Macintosh, Windows, OS/2 Presentation Manager, and XWindows, has the potential for providing barriers for individuals with visual impairments, particularly those who are blind. The problems stem from three factors. First, the information being displayed on the screen has shifted from a character-based format that was stored in an ASCII text buffer to a pixel-based format. This makes it much more difficult for screen reading software to determine what characters are on the screen. Second, text-based systems used a single font and relatively few attributes (bold, underline). On the graphic displays, text can assume a very large variety of sizes, fonts, and attributes (bold, underline, italic, crossed out, etc.). When these font changes or attributes contain information, they complicate the process of presenting information via speech or braille. Third, the advent of the graphics-based system has led to much more widespread incorporation of graphic elements, such as charts, diagrams, and pictures, within the text.

The new systems also introduce a number of advantages or opportunities for individuals with severe visual impairments or who are blind. Both the consistency of the human interface among applications and the use of system tools in the display process hold the potential for providing access to a broader range of applications for persons who are blind, if an effective human interface to these operating systems can be developed.

The Systems × project is using a multi-sensory approach to explore techniques for providing an effective and efficient interface to graphic-based by people who are blind. A prototype system, dubbed “Systems 3,” has been developed which uses speech input/output, the keyboard, and a virtual tactile tablet to allow individuals to access both text and graphic elements in the system. Using the prototype, individuals who are completely blind have been able to use a Macintosh computer, access and read text documents, and handle simple to moderate graphic elements such as bar charts. A series of research studies is now ongoing to test the limits of the access and to quantify the relative efficacy of various approaches to providing a nonvisual interface to these graphic-based operating environments.