Abstract
The present study examines feedback mechanisms (FMs) to facilitate communication between the blind and visually impaired (BVI) and their surroundings or devices. Although previous research has led to the development of numerous accessible technologies for aiding BVI individuals in various scenarios, the widespread usage of these technologies has been limited by a lack of user acceptance. This is often due to the high associated cost and difficulty in processing information intuitively, causing a high cognitive load, resulting in the system being mostly unusable. The core of these issues lie within the FMs opted for the intended application. To address these issues, this literature review scrutinizes unique literature with various implementations of FMs, which are then classified based on their cognitive load requirements, associated cost, and applicable scenarios, thereby assisting in making informed decisions for optimal design.
Keywords
Introduction
Feedback mechanisms (FMs) aid BVI individuals in interacting with technology, accessing information, and performing tasks, providing means for independent navigation and device operation (Wang et al., 2017). Therefore, accessible, low-cost, user-friendly, and effective FMs are crucial for meeting the needs of this population (Ferati et al., 2014; Lise, 2021). This paper provides an overview of FMs for BVI individuals, emphasizing information delivery modality, usage scenarios, user cognitive load, and cost of ownership. Reviewing existing literature and synthesizing key findings, this paper will explore the different FMs available for BVI users and discuss their advantages and limitations. In addition, the paper will highlight the importance of designing FMs tailored to individual users' specific needs and preferences that are accessible and user-friendly across different types of devices and technologies. Ultimately, this paper aims to contribute to a greater understanding of FMs by providing insights into some rarely used but highly potent mechanisms for BVI individuals that can help promote inclusivity and independence regarding the real-world scenario where embodied information can be used to draw insights on the localization.
Method
We performed a thorough investigation of FMs deployed across various technologies through comprehensive searches on Google Scholar. We utilized various Boolean operators with relevant keywords and performed multiple iterations until new literature emerged. Based on our identifications, we have sub-grouped the implementations into five categories: Sensory FMs, Multi-modal, Auditory, Torque/Tension, and Brain-Computer Interfaces. In our findings, we also discuss core issues with implementing proposed technologies and propose a future direction for the research.
Overview
Audio-based information delivery
This classification includes FMs built into assistive technology devices, such as screen readers, magnification software, or speech recognition software. Some examples of these mechanisms are audio prompts, spoken feedback, intensity-adjusted sound, pitch-adjusted sounds, etc. One such study presents a face recognition system, (Krishna et al., 2005), that uses a text-to-speech converter to communicate with the user of the detected faces. The study used Microsoft speech Engine to generate the speech audio signal. In applications like these, where the time or length of speech does not make a significant difference, the audio-based systems are less cognitively demanding and serve the purpose. Studies have found that audible beacons and wearables can be a great way to enhance the independent movement and playing of young VI children (Freeman et al., 2017). This study identifies ways to integrate environmental sound with the sound of wearables that also helps in proximity estimation between objects. The user uses a bracelet that plays the audio based on the beacon's proximity that can be worn or placed in different places. The beacons also play sounds to let the user know the beacon's location, the so-called environmental sound. This mechanism can also be used to find a particular person or object based on who has the beacon as a bracelet or where the beacon is placed. The study proposed combining both mechanisms to enable information to be embodied in the environment and provide the user with remote bracelet feedback for more precise and independent navigation.
Autonomous vehicles (AV) have contributed to independent navigation for users with visual impairments (Brinkley, 2019; Brinkley et al., 2017, 2020). In these interaction scenarios, auditory information is utilized for situational awareness, while a voice user interface is employed to control autonomous vehicle systems. FMs for navigation must address cognitive load as its core challenge, especially as it is delivered in real-time. Spatialized audio (Blum et al., 2013) provides a more intuitive user experience and addresses high bandwidth and low latency information delivery. This approach copes majorly with real-time hands-free navigation by generating a 3D sound effect using headphones. Similar to this binaural sound reproduction with Bone-conducted sound, more efficient and shorter distance navigation was achieved (Asakura, 2021). These are explained in the 1st section of the table.
Multi-modal Feedback mechanisms
These feedback mechanisms include a combination of auditory and haptic feedback to deliver information. These technologies can be used in navigation technologies, screen readers, and others requiring users to interact with multiple senses.
Multi-modal FMs mostly represent visual information in haptic and auditory forms. For instance, an interactive audio-tactile map (Brock & Jouffrais, 2015) can help blind users conveniently feel various representations of visual information using audio and tactile signals. Technologies like haptic braille (Jayant et al., 2010) can also help users to interact with an autonomous vehicle human-machine interface (HMI) while using these technologies (Figure 2).
Modes of auditory information delivery (Hosoi et al., 2019).
Multi-modal interface (Manawadu et al., 2017).
In the context of spatial or situational awareness during navigation, there have been various implementations where piezo-electric buzzers are attached to each finger (Zelek et al., 2000) that are activated in binary mode to indicate obstacles in front. The buzzers were used in glove construction, and the tactile feedback conveyed the depth of information about the surroundings. Some of the addressed issues failed to detect these tactile feedbacks as the ones generated with piezo-electric buzzers are relatively lower than motor based, also given the compactness of the instrument. Another such study(Meers & Ward, 2005), provides a substitute for a 3d perception system and landmark identification via an electro-tactile interface. As mentioned in the paper, the electro-tactile interface enables the user to receive tactile or touch feedback through electric simulations. These technologies have also been used in VR, gaming, and prosthetics. The frequency and pattern of the electric pulses can be used to simulate different types of touch, such as pressure, vibration, and even texture. Studies have focused on using headphones as an auditory feedback mechanism to guide a person. One such study, (Vongsathorn et al., 2013), addressed the issue of participants focused on the individual's body movements and having different awareness of the space around them. This mode of information delivery through headphones can be highly effective in providing hands-free navigation, allowing the user to use both hands so that when a user interacts with a virtual object in the dark, the user is notified of the obstacles. In this specific study, the researchers used music and changed the intensity (increased when the hand gets closer to the object) of the music to provide them with proximity information from the virtual object. The user can then identify the objects' shape by figuring out which part in space has the sound played loud and which part it did not. This way of localization and object detection can also help in daily navigation. There have been various implementations of drones to help the user navigate using the natural sound and airflow of the drones. One such implementation (Avila et al., 2015) uses a drone-mounted bracelet that the user wears to be deployed, and the drone follows the path to the destination, and the user follows. This strategy also helps identify objects.
Coding information in temporal form increases the cognitive load in information processing and sometimes makes the information completely unusable. To address this issue, a vibrotactile waistband (Erp et al., 2017) was developed to balance the amount of information displayed and the comprehensibility of the information using compound tactile signals, contributing to hands-free navigation.
We have combined Haptic, Electro-tactile simulation, and Audio-Tactile Maps in the table into this category. These are often composed of two different feedback mechanisms or used in conjunction with others. These are explained in the 2nd section of the table.
Torque/Tension/Force
Guide dogs and humanitarian assistance have been the classical way of BVI navigation. A study used a similar approach with drones, (Avila Soto et al., 2017), by incorporating leashes into the drone, similar to a guide dog. The drone is attached to a leash and connected to the arrow in the handle that the user holds, and this helps convey the direction to move. A study by Lupu et al. evaluated the VI’s cognitive load and emotional stress in real-world mobility experiments in the case of navigation and found that white canes still perform better than audio and haptic interfaces in most of the scenarios(Lupu et al., 2020). These are explained in the 3rd section of the table.
Sensory Feedback Mechanisms
This classification includes FMs that provide information through inherent senses such as smell and lighting. In a study by Quinones et al., (Quinones et al., 2011), the participants could identify intersections based on the distinct smell of the coffee. However, none of these smell-based technologies have been explicitly implemented to aid location identification. Another such sensory information is using partial vision to aid daily interactions. One such study, (Freeman et al., 2016), explores the potential of adaptive lighting to assist visually impaired children with their daily activities. The study proposes using a prototype that detects children's movements and objects' motion to trigger changes in lighting, providing natural and intuitive information to visually impaired children with minimal cognitive demands. Future technologies could consider similar embodied information rendering mechanisms, such as artificial smell cues, to aid independence for BVI individuals without additional cost. This is discussed in the 4th section of the table.
Brain-Computer Interfaces (BCI)/ Brain-Machine Interface
BCIs utilize brain signals and signal-processing algorithms to communicate information with the brain. These can be invasive and non-invasive. Non-invasive BCI (Figure 3) uses external sensors to measure the brain's activity without implantation, while invasive interfaces might require implants on the skull with wires directly attached to the brain. Most non-invasive mechanisms acquire information from the brain signals, analyze them, and translate them to other feedback signals such as electro-tactile simulations, audio, haptic, and even signals that can be fed directly to the brain. BCIs are becoming revolutionary communication mechanisms for people with disabilities with advancements in hardware, validation, reliability (Shih et al., 2012), surgical robotics, and massive progress in ai-based signal encoding-decoding mechanisms. With these technologies, it is possible to perceive the environment directly from cameras and interact with them by just thinking. This research field has garnered significant attention from companies like Neuralink (Figure 3) to augment the information transmission bandwidth and make these systems more usable to users (Pisarchik et al., 2019). Some initial proven implementations include monkey-playing games and typing with brain implants. These interfaces are highlighted in the 5th section of the table.
Example of Non-invasive BCI in left (Faisal et al., 2023) and Neuralink’s BCI implant in right (Waltz, 2020).
Findings
With our review, we suggest appropriate mechanisms for specific scenarios and present our findings in Table 1. Our study reveals that despite being accessible and easy to interpret, technologies might be expensive for individuals to afford, resulting in a preference for devices with higher availability and lower cost. On the contrary, some low-cost and even popular systems, such as haptics, for example, might be very hard to interpret and can induce high cognitive demand in certain scenarios with improper information encoding and decoding, thus making the system less usable in real-world scenarios. Additionally, due to their complexity of operation, some of these systems might require extensive user training, which users might not be willing to undergo. This perpetuates persistent dependence and satisfaction with conventional technologies, consequently hindering the widespread acceptance of new technologies. Thus, with our classification, researchers can balance cost and usability while designing a new interface. We highlight the need for hands-free navigation in most cases, particularly for those with complete vision loss, to preserve other sensory inputs. We also observed that some implementations are bulky and heavy, making them less ergonomic and usable in the real world.
Forms of Implementations and Applicable scenarios, cognitive load and cost requirements.
Discussion
Our study offers a comprehensive overview of feedback mechanisms researchers can leverage to develop technologies for BVI users. Our findings underscore the increasing need for hands-free navigation, embodied information, and usable vision to enhance independence, which is underutilized in the field. We contextualize future technologies and present current FMs for various AV experiments, recognizing their potential to facilitate independent navigation for BVI individuals. To achieve this, selecting appropriate FMs is critical, especially in challenging scenarios such as navigating unfamiliar environments where assistive technologies may fail to communicate enough information or none sometimes (e.g., audio in crowded settings). These challenges often arise during ingress and egress, where individuals may need to navigate unfamiliar terrain or even with independent navigation.
Conclusion
Our study categorizes FMs based on their current implementations, cognitive load requirements, the cost associated with them, and their usage scenarios. Furthermore, we highlight the benefits of sensory information delivery methods, such as lighting and smell, that can assist certain BVI individuals in recognizing specific locations and enhancing independence. We emphasize the importance of hands-free navigation and embodied information delivery methods and recommend creating practical and ergonomic technologies to tackle the challenges encountered by current accessibility aids.