Abstract
Visual-to-auditory sensory substitution devices have various benefits over tactile counterparts (eg, less hardware limitations), but they also suffer from several drawbacks (eg, learning time, potentially unpleasant sounds). An ‘ideal’ device would be intuitive to learn, pleasant to listen to, and capture relevant visual information in sufficient detail. In this presentation, we outline the general problem of how to convert an image into sound, and we give an overview of some possible approaches to the problem. We then go on to describe our own recent explorations using Interactive Genetic Algorithms (IGAs). IGAs enable a highly dimensional problem space to be explored rapidly. Initially, a set of orthogonally varying settings need to be identified (eg, different levels of maximum and minimum pitch, different ways of mapping lightness-loudness, musical vs non-musical intervals), and a set of random permutations of these settings are chosen. Participants then evaluate the ‘fitness’ of these different algorithms (eg, by selecting what the correct image is for a given sound). The fittest algorithms are then ‘bred’ together over successive generations. Using this approach, we compare the performance of evolved devices against one of the main existing devices (the vOICe) in three tasks: audio-visual matching, aesthetic preference, and auditory discrimination.