Abstract
Design students who are blind or sight-impaired face distinct challenges when studying a visually centric discipline such as design practice. Students who are sighted use computer-aided design (CAD) which is presented via high definition using a PC mouse. However, design students who are blind or sight-impaired are not able to use visual display technology; therefore, this creates a barrier to access for this community. The aim of this study is to present a haptic prototype trial (Haptic Application Prototype Test [HAPT]) designed to assist design students who are blind/sight-impaired to interact with prototype assembly at the Open University (OU). The study specifically assessed the user feedback and the efficacy of access to CAD interface through the affordances of the haptic interface. The experiment included two groups of participants: one group included students who were blind and sight-impaired and the second group students who were classed fully sighted. Both groups were tested in two conditions of haptic engagement – manual and virtual. The parameters examined were (a) time – set at an industry-recognized time taken to assemble a ‘sketch model’ or prototype, and (b) ncollision – the number of collisions created by a collision algorithm which calculated any random collisions with the virtual environment or objects therein. Quantitative results showed that there was little statistical difference between time and a between-group test. From this we can imply that the haptic interface had offered equal access to CAD for people in the trial who were sighted and blind/sight-impaired indiscriminate of their sight acuity. Further future work using HAPT could be developed to a wider audience and a larger more diverse range of sight-impaired users. Future work will focus on new explorations of teaching using of haptics for greater immersion for distance learners at the OU science, technology, engineering and mathematics (STEM) labs.
Introduction
Since the creation of the graphical user interface (GUI, pronounced ‘gooey’) fully sighted, PC users have recognized the sophisticated pictorial and graphical displays as an effective visually led interface, which offers access to many different software applications. As much as a GUI is a usable and effective interface for fully sighted users, it holds barriers for people who are blind and sight-impaired due to the visually centric nature of the presentations of data. Globally, it is estimated that at least 2.2 billion people have a vision impairment or blindness (World Health Organization [WHO], 2019). This means that there is a potential that 2.2 billion people may have difficulties accessing GUI and therefore require alternative access to digital data. Alternative access is usually in the form of specialized software which transcribes text and imagery to sound or into Braille. Non-visual access to graphics is often provided by a verbal description through screen readers, which is not always a satisfying solution as it offers a large amount of text which can overload the working memory. Braille devices deliver a single line of Braille to transcribe and represent graphic detail but not all people who are blind or sight-impaired are Braille users. People with limited sight who wish to use computer-aided design (CAD) for design purposes can find the visual nature of the on-screen graphics difficult to access and to fully appreciate via a single line of Braille or complex speech transcriptions. To fully access graphics, many designers, with visual impairments, are reliant on enlarging the scale of the graphics on-screen, but this can be confusing as it can distort the image making it unrecognizable. And people who are completely blind are unable to make this shift. In response to this, multi-modal media, such as haptic (touch-led) devices, have the potential to afford designers who are blind or sight-impaired greater access to digital interface systems via touch and sound.
Haptics is a broad term and said to be derived from the new Latin hapticè (Science of Touch). Haptics is also known to be derived from the feminine Greek ‘able to come into contact with’, from haptós ‘tangible’ (verbal adjective of háptein ‘to fasten’, háptesthai ‘to fasten’ [Merriam-Webster, 1921]). Today, the term haptic is more readily related to mechanical and digital devices/interfaces which are engineered to offer users simulated tactile and kinaesthetic sensations. Through users’ contact with receptors placed on the body or within a haptic device, the haptic system can locate the distance between the user and the virtual object in virtual space. The haptic augmented interface responds by creating a force resistance felt by the user through skin or guidance force.
The application of haptics can be separated into three main categories:
Graspable: kinaesthetic devices in glove format which uses force to mimic grip or ground-based devices which can locate and interact with simulated surfaces and solid objects.
Wearable: body-worn devices like the haptic vest or bracelet.
Touchable: pertaining to touchscreens, smartphone screens, or tablets.
Previous studies by researchers working with haptics have shown that non-sighted and sight-impaired communities have been one of many communities who have been able to benefit from haptic systems to support their cognitive map of visual data (Brewster, 2005; Lévesque, 2008). The benefits of haptic technology are shown to set within human–computer interactions (HCI) context (Burdea, 2000; Corliss & Johnsen, 1968; Mosher, 1964; Stone, 2000; Thring, 1983). However, in more recent years torque feedback has become more refined, meaning that the user receives more sensitive and adaptable higher fidelity touch-based stimulus. The higher fidelity, in turn, increases the users’ realistic perceptions of being engaged and present in the virtual environment.
Overview – Haptic Application Prototype Test study
The Haptic Application Prototype Test (HAPT) study aimed to tackle the usability issues of art/design modelling systems for students who are blind or sight-impaired. As a proof-of-concept study, the HAPT system is aimed to respond to the difficulties of learners without sight or with sight impairments to engage with CAD systems using standard GUI frameworks. Using a high-specification haptic device (Geomagic Touch™) and an augmented haptic force interface, HAPT allows the student who is sight-impaired/blind to be guided to touch or ‘read’ on-screen objects and move them into an assembled form. The HAPT system provides a tactile version of standard CAD design prototype assembly procedures, which usually are only conducted via visual CAD systems. The proposed study goes some way to facilitate dual sensory access within a visually centric discipline such as design practice.
Research questions
This study will use the following two research questions:
RQ1. Is it feasible for a dual sensory digital system to mimic physical user interaction with manual haptic (MH) prototype assembly?
RQ2. Can people who are blind and sight-impaired be enabled to further understand and carry out specific shape assembly tasks using HAPT, within a standardized timeframe?
Background
Historical access to digital technology
In the early years of computing (1980s), people who were blind could easily take part in interactions with computers which used command-line systems. These systems were logical and mostly simple to translate into voice or Braille outputs. Since the introduction of GUI and the use of visual metaphors, people with sight issues have found it difficult to interpret and interact with this type of interface. Lévesque (2008) asserts that single directional input devices, for example, a computer mouse, are unusable by blind people due to the lack of bi-directional feedback and force-guided orientation. Lévesque goes on to assert that we need to reinterpret and redefine custom interfaces which will increase blind users’ access to GUI, to make it more appropriate and to enable users to readily interact with visual data alongside their sighted peers. Tactile feedback (haptics) was noted by Lévesque as something which blind people can readily exploit but he advises designers/researchers working with touch-led technology that they should maintain user input throughout the development process to assure the feedback is appropriate for use (Lévesque, 2008).
Digital haptics and culture
Haptics have previously been used in museums to facilitate visitors’ interaction with precious artefacts without damaging or risking the physical artefact. Haptics used in this way offer visitors the chance to gain a deeper understanding of the explorations of precious artefacts. Brewster (2005) and Zimmer (2007) have focused on a haptic interface as a tool to afford greater cultural access for non-sighted museum visitors. Before Brewster’s study, museums offered blind visitors physical facsimile models which they could tactually explore. The limitations of the physical facsimile artefacts was the expense of reproducing exact copies, thereby only offering visitors small collection of artefacts. Bergamasco et al. (2002) have also worked on tactile exhibits in museums using a virtual haptic (VH) system entitled ‘Museum of Pure Form’. Bergamasco et al.’s study was less dedicated as a tool for non-sighted visitors; instead, they focused on facilitating visitor interactions with a larger selection of sculptural forms, but unlike Brewster’s study, it is not limited to interactions purely with small-scale objects and can offer full-scale sculptures for visitors to interact.
Other approaches to cultural haptic applications focus on arts via haptic systems which are developed as a personalized creative tool. ArtNova (Lin et al., 2002) allows users to press buttons and alter dials to increase or decrease haptic sensitivities, while offering a full gamut of dynamic viewing, surface textural applications, and sculpting tools. Dynasculpt (Snibbe et al., 1998) creates new ways to present a mass of three-dimensional (3D) objects in virtual space. Dynasculpt enables users to draw lines which become tubular or pipelines. The 3D object mass can also be deformed and manipulated to create 3D model objects which feedback properties of the object, for example, weight, surface texture, and real-time dynamic manipulations, to users. While both the haptic applications (ArtNova and Dynasculpt) offer innovative ways to interact with 3D objects and recognize tactile surfaces in real time, these systems are not able to encompass people who are sight-impaired or blind. In this article, we introduce HAPT. In this study, the technology and interface have been designed to facilitate design students, who are blind or sight-impaired, a greater tactile interaction with virtual 3D objects to increase their sense of presence in the virtual environment.
Method
Subjects
A total of 20 Open University design undergraduates participated in this study. The participants were subdivided into two groups of 10: 4 blind students and 6 sight-impaired students were assigned to Group 1, and 10 fully sighted students were assigned to Group 2. No one in Group 2 wore spectacles to correct low-vision issues. The participants in both groups were mixed gender, with a mean age of 36 years (35.6). None of the participants had used any form of kinaesthetic haptic experience prior to the test. None of the participants informed that they had any other physical or mental disabilities which would need to be considered for further adaptions to be made or anything which would prevent them from participating in the test.
The ethical principles from the Open University were applied to this study. Due consent was given by all participants to take part in the study, thereby consent given for all coded data was collated and published. The participants for this study were advised that they could leave at any time and they could also have breaks when needed throughout the trials. Participants were not paid to take part in the trials, but all expenses were paid.
Both groups were recruited to the study via the Open University Student Research Project Panel (SRPP) data records. Participants were selected from a larger set of listings using the following selection criteria:
Level and experience of design concept generation (shown via their academic record and CVs);
Proven professional skills in design and prototype methodologies (established from a pre-trial interview by a selected research panel).
The fully sighted group (Group 2) was classed as a control group which would offer a comparable standard of data from which to compare Group 1 results. As this study was a ‘proof-of-concept trial’ using an inclusive design framework, the samples of group participant numbers were appropriately sized according to appropriate HCI and human factor studies which are commonly known to work with five to six participants at any one time (Table 1).
Participant demographics – sight acuity, gender, number, dominant hand and age.
None of the fully sighted (FS) participants had to correct minor eye constraints with spectacles and thereby classified as fully sighted.
Instruments
Kinaesthetic single-point force haptic device – Geomagic Touch
To select the correct form of haptic device for the HAPT study, the specific attributes for all users’ interaction were defined and analysed in conjunction with several commercial off-the-shelf (COTS) devices. Due to the inclusive nature of the study, reference was given to the sensory user interactions of people who are blind or sight-impaired. An audit of COTS haptic devices was carried out and a summary showed that the haptic device needs to be as follows:
Easily operated by a haptic novice;
Should offer participants who were blind or sight-impaired sufficient force to effectively guide them to specific home cue points and 3D objects in virtual space;
Should offer a higher range of degrees of freedom (DoF) to increase the users’ range of access into the virtual space via greater range and reach.
The Geomagic Touch device held a higher specification of Newton’s force feedback (per mm2) than most other commercial devices. This means that users would be offered enough notable force pressure to enable them to (a) trace the contours of 3D objects; (b) enable guidance between home port – object; (c) guide users, at speed, to assemble the prototype within a standard set time; and (d) offer a stylus, a known and common tool in the design industry. The Geomagic Touch device was finally justified to be the most appropriate device for this study. It was the only COTS device to offer all of the specified points and it offered users 6 DoF. A higher grade of DoF means that the haptic device can offer the user greater rotational and reach movement. For example, haptic device with 6 DoF, such as the Geomagic Touch, would be able to offer users a mimic of rotational movement to that of the human joints and tendons in the arm. Using Geomagic Touch, users can engage more readily with objects to feel and trace the contours of 3D virtual objects on-screen. Added to this, the Geomagic Touch software could be adapted to enable researchers to measure and track the 3D virtual positioning using 6 DoF (axis x, y, z + rotational axis – pitch, roll, and yaw). The tracked positioning data could then be recorded and stored within the interface. The positioning data could offer researchers fuller contact data to an exact point of contact between objects and user, meaning that a full spectrum of logged contact could be analysed post-test. This was used to ascertain the efficiency of the device and augmented interface. The cross-matched data were checked to ascertain whether the users were correctly force-guided to the pertinent programmed shape contact points (Figure 1).
The Geomagic Touch™ device and specification chart.
Materials and procedures
The HAPT test consisted of a pre-trial training application, followed by two further trials using two conditions. Condition 1 = manual haptic (MH) and Condition 2 = virtual haptic (VH). Both groups were instructed to complete both conditions starting with the MH condition and moving on to the VH condition. Each participant was requested to repeat each condition three times.
Prototype object
The prototype object was pre-selected for the HAPT study and in the form of a chair. A specific style of chair was selected by the researchers for its simplicity and modelled from simple forms inspired by the iconic Gropius Chair (Bauhaus Gropius, 1920–1949). The Gropius Chair is based on by cubic forms using rectangular block armrests and interconnected wooden loop legs which connect from back to front of the chair. The HAPT chair was modelled on similar geometric rectangular seats and arms but was designed without base chair legs, therefore more akin to a ‘chaise longue’, and this was done for brevity of assembly. The construction of the HAPT chair was 3× simple stocky rectangular block shapes (Blocks 1–3); each block was placed end to end. Finally, one arch block stacked on top of Block 1, thereby acting as the backrest of the chair.
Digital haptic familiarization (pre-trial) task
Prior to the main haptic condition tests, both groups were requested to complete a digital haptic familiarization task. Group 1 (people who are blind and sight-impaired) participants were permitted to touch and explore all manual 3D blocks and the haptic apparatus used prior to the commencement of the formal tasks in each condition. The digital blocks were examined within a bounded space using the Geomagic Touch device. This was permitted to allow for the lack of sight and to offer a full mental picture of the materials and technology used. In the pre-trial test (Figure 2), participants were requested to push and ‘pick up and put down’ the cube to the audio-linked floor markers. Both groups were asked to make full use of the audio and haptic guidance in practice to become used to dual sensory input.
Pre-trial screenshot – pick-up–put-down haptic tasks.
Manual HAPT set-up
The MH trial was inspired by the design industry standard prototype assembly process. The set-up consisted of 4× four 3D foam block shapes (palm-sized) measuring 25 mm × 23 mm, 1× yellow arch, 2× red cuboids, 1× yellow cuboid placed on a table. The foundation block (yellow cuboid) was magnetically fixed to the table which gave users a firm base on which to build. The participant sat comfortably in front of the block set-up. They were presented with three foam blocks placed inside a wooden box which was located by the participant’s dominant hand. The participant was requested to initially select one cuboid from the box and to push it until it connected to the foundation block. This process was then repeated until only the arch shape was left. The final arch shape was then picked up and placed on top of the foundation block which completed the assembly. The duration of the MH condition was set at 5 min (max). The time was recorded using a digital stopwatch and completion was counted when the final block was placed. All aspects of the MH condition were recorded as still imagery and dynamic video records. In MH condition, both groups were requested to use the same pre-defined haptic movements: (a) ‘push’ and (b) pick-up and put-down. All movements were practised by each participant in the pre-trial prior to commencing each condition. Participants were encouraged to complete the task as quickly as possible while trying to minimize their accidental collision contact between shapes, for example, between active shape in hand and dormant shapes about to be picked up.
Virtual HAPT set-up
The VH set-up consisted of 4× four 3D foam simulated block shapes (palm-sized) measuring 25 mm × 23 mm, 1× yellow arch, 2× red cuboids, 1× yellow cuboid placed on a table. The blocks were scanned from a 3D scanned image of the MH blocks. Similar to MH condition in VH condition the foundation block is a yellow cuboid, but in VH the foundation block was locked to the virtual floor of a bounded box space. Virtual condition = four foam block shapes (palm-sized) measuring 25 mm × 23 mm (1× yellow arch, 2× red cuboids, 1× yellow cuboid). The participant sat comfortably at a standard computer desk in front of the Geomagic Touch device. The 21″ screen was specifically aligned with the Geomagic Touch haptic device for ease of reach. On the start-up screen, a bounded virtual environment was presented. On start-up, the screen featured two blocks (25 mm × 4 mm × 23 mm; simulated coloured foam shapes). Participants were requested to use the single-point stylus in their dominant hand to push one shape until it connected to the foundation block. Each time they needed a new shape, they could summon it by depressing the spacebar on the keyboard. This process was repeated until the final arch block, which was simply picked up, was aided by magnetic force. The arch was then stacked on top of the foundation block. Each time a participant connected the stylus to a new block, that block became live and pinged and the rest of the blocks remained inactive and static. This element was to guard against users accidentally picking up the wrong sequence of blocks or more than one block at one time. The duration of the assembly task was again set as 5 min and the time of the trial was recorded on a digital stopwatch embedded in the virtual interface. The allotted time of 5 min would commence on connection to the first block. The VH was then designed to shut down on either the connection of the last block or when the time ran out.
HAPT additional assistive cues
One of the many advantages of using haptic device and haptic augmented interface is that we could add varied assistances designed to specifically enable people with sight disablements to use the same system as non-disabled people. Within the augmented interface, we were able to offer the following assistive cues:
Increased force to aid speed of task completion;
Additional alert sound cues (non-speech = beeps);
Guided push force for shape connection;
Guided drag force to aid location of shapes;
Bounded space to confine the working space and aid quicker shape location;
Real-world environment sounds used to alert sound indicators when stylus connected to virtual floor, wall, for example, sounds mimicked shiny rubber floor using squeak sounds.
The use of multi-sensory (multi-modal) and added forces was used to add assistances to aid the user’s assembly of the prototype. The sounds were designed to link users more clearly with the virtual environment and objects, therefore increasing the user’s presence within the environment and to offer guidance alerts, for example, the closer the individual pushed the shapes together, the faster the sound ‘beeps’. There was a sound connected to the call of a new shape and the sound stopped when the user connected to the shape, to denote that they had contact with the shape. There were sounds and added force to the exploration of the bounded room space in which the shapes were kept, for example, floors, walls, and ceilings of the space. The floor/wall offered a mimic force of a solid plane offering no pliability. All floors, walls, and ceiling were solid, which was different from the feeling of the spongy foam shapes; therefore, it offered another way to assist users to distinguish between surfaces and shapes (Figure 3).
Multi-modal shapes showing sound and force feedback.
For brevity, simplicity of interaction, three salient actions were selected for primary interaction process in MH and VH conditions. These actions were push, pick-up, and put-down (Figures 4 and 5).
Push action MH (manual haptic condition).
Push action VH (virtual haptic condition).
The ‘push action’ was selected to be used as the main action over the serial ‘pick-up’ and ‘put-down’ action. This was due to brevity and ease of the push action over pick-up and put-down. The push action was also considered more user-friendly to participants in Group 1, as it maintained the shapes to be in consistent contact with the solid base, for example, floor of the bounded space. This allowed people in Group 1 to understand the block location and using the audio they could understand the shapes’ proximity to other shapes. All participants confirmed that the action was easy to understand and completed it successfully throughout the assembly tasks.
Feedback evaluation procedures
During the MH and VH trials, a test facilitator encouraged all participants to use the ‘think-aloud’ technique during their trial interactions. The think-aloud technique was used throughout the whole trial in all conditions. The technique is a common user-centred tool often used by researchers wishing to understand the internal processing in the minds of the participants, often used in BETA testing and alongside technology probes within HCI fields. The feedback conversations were recorded, transcribed, and coded from each participant. The second set of feedback data was collated after the all trials were over. Participants were presented with a short multiple-choice questionnaire via a screen reader. A short Likert scale, 1 to 5 scale questions, was also used, set around the appropriation of haptics as an assistive tool (see Figure (6). For blind participants, there was the option of a trial facilitator to help them respond to the questions via a standard (non-Braille) keyboard (Figure 6).
Questionnaires: Likert scale, and multiple choice questions.
Data analysis
The metrics used for HAPT trial in each condition were time and number of collisions. Time was measured across all iterations, in each condition, for each participant (3 × 2 × 10 = n60 (×2 = 120)), thereby the data yield for time was 120 data sets; these were analysed using SPSS (v21). The time data average for both groups was then analysed in a between-group analysis using Mann–Whitney U non-parametric test. The second metric used was the number of collisions (ncollision); this was examined from a data yield of 120 data sets. Collision was defined as any moment when a participant accidentally collided an active shape (in-motion) with a dormant shape. Added to this, the number of collisions was recorded as any accidental collision with the bounded walls and floors in the virtual environment. The use of collision data was to highlight the efficacy of force guidance of the haptic device and to establish the operational abilities of the augmented interface in conjunction with the device.
Qualitative data
The qualitative data, taken prior, during, and after trials, were processed using NVivo (v9). The data for both groups were transcribed from recordings coded and reviewed, analysed for emergent themes. First-level data analysis showed that the most commonly repeated word was ‘easy’, followed by the words ‘understandable’ and ‘interesting’. Further analysis of the themes revealed some sub-theme headers on ‘tool use’: usability, understanding, and fit for purpose. Further analysis offered phrases such as ‘Oh this is easy’, ‘this holds real potential for use’, and ‘more fun than I expected’ when working in the VH condition. Analysis from VH pre-trial offered that some participants were initially wary of looking stupid by being unable to pick up or put down the objects in the desired locations; however, this did not extend to the main trial.
Agreements (moderate and extreme)
One day after the trials, all participants were interviewed to establish their reflective thoughts on the test. A quasi-structured interview was used and asked open questions about their overall experience of using the haptic, whether it aided their tasks, whether they could feel the shape mass clearly, whether the assistances had aided or hindered progress, and whether they would use it again. From the participant content analysis of interview data, there was a moderate agreement by Group 1 participants that they would use the tool again, and they could perceive the mass of the geometric shapes on the screen and that they could understand the assembly techniques as it was ascribed to them.
Participant Non-sighted NS9 stated, It was satisfying accomplishing something which I had thought impossible or at the least very difficult in a relatively easy manner.
He went on to say, Moving from a mental picture to creating a prototype was satisfying. Without the interface, I can’t conceive how the task could be accomplished on my own. Only [sic] another alternative would have been a sighted assistant to do all the work. (NS9, 2018)
There was also a shared agreement across all groups that specifically using the single-point kinaesthetic device (Geomagic Touch) made the process simple and intuitive to control. However, one participant, NS8, indicated an innate need to use both hands to interact with the virtual 3D objects on-screen; NS8 stated, It was also somewhat confusing at the cognitive level, that while holding the pen in the right hand and clearly feeling a virtual wall, the left hand did not feel anything.
The fully sighted group showed a lesser agreement with how well haptics offered a more refined sense of pressure and touch than 2D graphics tools. They noted specifically that when the probe connected to the virtual walls and objects, they could just perceive that they were touching the walls and floor but more from the sound changes. However, they stated that they could tell that the shapes were mimicking foam rubber through the tactile perceptions being softer in surface than the environment walls. They appeared surprised at the convincing noise feedback of the end-effector as it connected to the bounded wall; participant FS5 (2018) commented, ‘the device was easy to hold and intuitive to use. I was impressed by the feel of the boundaries in the interface when converted to resistance in the device’, ‘Being able to feel the weight of the object was also a pleasant surprise’ (FS6, 2018).
Questionnaire data (quantitative)
The Likert questionnaire showed a consensus of moderate and extreme agreement between Group 1 (blind and sight-impaired) on their initial perceptions of the haptic device and the actual perceptions from using the tool; 50% of Group 1 participants noted that the interface and device was easy to control which was contrary to their pre-trial statements. It was revealed that many participants believed the interface would require high-level mathematical or programming skill just to use and control the tool. Next, analysing the Likert scale data showed that both groups selected a positive scale of 4 to 5, for questions surrounding haptic force feedback, interactivity, and handling of the virtual prototype interface and device. It is interesting to note that the fully sighted Likert scale graph shows a slight decrease in scale for questions surrounding handling and force (−1, −3), thereby showing disagreement with more questions than blind and sight-impaired participants. The results of multiple-choice questions revealed that all 20× participants stated they would be interested in being involved in future developments of the HAPT system, and that they were relatively surprised at the ease of use of the HAPT system.
As shown in Figure 7, for people who are blind and sight-impaired, the stronger disagreements are indicated in the paler shades and high agreements with statements are shown in darker tones. All participants disagreed with the haptic causing fatigue, and there are mid to strong agreements with force and guidance affording assistance and support of task.
Likert scale presented as a stacked plot graphic showing agreement (blind and sight-impaired).
It is shown in Figure 8 that for people who are sighted, there were lesser strong agreements and more neutral agreements with the questions around the increase in haptic force and their ability to interact with objects or aided guidance to the objects. These results concur with the users’ feedback during trials and their overall reactions after the trials.
Likert scale presented as a stacked plot graphic showing agreement (fully sighted).
Condition results (quantitative)
For each group performance, an average of time taken and ncollision was taken in each condition for each group. The averages were then used to compare between groups, as shown in Figures 9 and 10. The recorded average (time/collision) results revealed that both groups that completed the tests were well within the standard time limit of 5 min and with a low-medium number of collisions.
Average time taken to complete manual and virtual conditions for Groups 1 and 2.
Average number of collisions for manual and virtual conditions for Groups 1 and 2.
Using descriptive statistics (parametric) showed that there was not a normal distribution of data for either Group 1 or 2 for virtual or manual conditions for time and ncollision. Therefore, we moved to a non-parametric test. The Mann–Whitney U test was used to establish whether there was any statistical differencing happening between groups, meaning that it was established that there was no significant difference in time or ncollision shown by the people in non-sighted or sighted groups. Using the test again showed that was there no difference in the results for both groups in both conditions (see Table 2) (duration and collision), as shown by the p and z values.
Mann–Whitney U test results.
VH: virtual haptic; MH: manual haptic.
For accuracy and quality, further analysis was taken for ncollision data. Cohen’s kappa (κ) was used as a measure of capturing actions accurately by two raters. To do this, initially, the recordings were transcribed for every participant in MH and VH modes. The location data offered by the programming in VH condition were used as an ultimate checklist. The film and the location data readouts were then recorded and analysed by two inter-raters; the second raters’ sample size was calculated at 80% of agreement with Rater 1. Cohen’s kappa (κ) was used to calculate agreements and resulted in a moderate agreement result (0.50). There was also an 85% agreement with both raters to the location data. Overall, this shows that between the two inter-raters, there was a moderate agreement of the number of collisions made in both conditions. The number of collision rates showed little statistical difference between groups and between conditions.
Discussion
From the HAPT trials with both groups, a great deal has been learned through observations of blind touch behaviours, unique touch adaptations, and novel tactile passive–active touch, observed in both haptic conditions. Many observed examples concur with the exploratory procedures (EPs) disseminated by Klatzky and Lederman (2003) with specific reference to observed occurrences of contour following in both manual and virtual conditions. However, many of the EPs in both conditions were practical in nature and used by blind and sight-impaired participants to aid their personal mental map of the object and environment layout. Other unique touch processes were used to add value to the basic mental map and aid the individuals understanding of properties and orientation, for example, pinching, tracing, and encircling the shapes. Within the pre-trial period, blind and sight-impaired users presented some of their unique forms of exploratory touch other than was shown in Lederman and Klatzky’s EPs (Klatzky & Lederman, 2003), for example, diagonal (opposite corner to corner) and linear edge tracing of one leading edge.
RQ1: Is it feasible for a dual sensory digital system to mimic physical user interaction with MH prototype assembly? This pilot study showed that it is feasible to adapt a dual system to mimic physical haptic interactions. The feasibility has been shown through ‘time and collision’ metrics that were gained from manual and digital haptic blocks of testing from quantitative results; it is worth mentioning that although there were some time differences between manual and digital trials, due to MHs being a known interaction for a number of years for all participants, the difference was minimal (±1 min). Qualitatively, most participants stated that although they were used to MHs, the digital haptic conditions were easy to use and easy to activate the tasks.
RQ2: Can people who are blind and sight-impaired be enabled to further understand specific shape assembly tasks using HAPT, within a standardized timeframe of 5 min? Time metrics showed all participants were able to complete the given haptic tasks within the timeframe of 5 min, except one blind outlier, who achieved it in exact 5 min. It is noteworthy that although there were collisions with the environment, these collisions did not prolong the testing time beyond the given timeframe.
Besides the results, we feel that there is a much richer take-home message to be disseminated. The main aim of this study was to extend the access of CAD to people who are blind and sight-impaired. It is striking to note that other than alternative text formats and screen readers, there are little assistive technologies to aid people who are blind or have sight impairments. There is even less adapted technology which had been adopted that enabled blind or sight-impaired communities access to visually led software such as CAD in an applied way. Using HAPT results was encouraging that this was made possible and it has been evidenced here that there was insignificant statistical difference between sighted and sight-impaired groups. This means that in the HAPT, test people who are blind and sight-impaired have been exposed to the same CAD tasks as people who are sighted, and the results have shown little difference.
Previously, the assembly of prototype shapes used single directional feedback, for example, PC mouse. The mouse would point a shape and could move the shape but the user would not gain any haptic feedback to confirm the action this was processed purely by sight, thereby the use of single directional feedback is inaccessible to people who are blind. To understand the CAD shape movement, the process must be translated either by alternative text or Braille. The other alternative would be to offer people who are blind or severely sight-impaired a human assistant to support CAD modelling assembly. Often, the introduction of another person can confuse the channels of a student’s creative agency and their creative handwriting in CAD.
As a bi-directional touch-led study, HAPT was the first of its type, at the Open University. It is expected that more like-minded haptic trials will be reviewed to aid people who are blind or sight-impaired. The study has shown that blind and sight-impaired participants specifically found the trials ‘easy to operate’, and they were ‘inspired and interested in the use of haptics to augment touch and to enhance the interactions of the design process in the future’.
Inclusivity was at the root of HAPT and as such the qualitative results were very encouraging, in that there is a great deal of agreement for each of the three themes – ‘usability, understanding, and fit for purpose’. There was a slightly greater agreement under the theme of usability by sight-impaired and blind participants over fully sighted participants. Throughout the test, the think-aloud feedback and the post-trial feedback both offered inspiring positive comments from all participants, but the most rewarding comments came from Group 1 participants who were interested in understanding how they as sight-impaired users could gain a similar digital haptic system to inculcate into their own design concept generation process.
Limitations
There are several limitations associated with this study. A longitudinal research study may help to understand the longer term impact of haptic benefits and allow more participants in larger groups to feedback on their own novel touch-led interactions, and how this may differ according to factors such as the degree of their visual impairment. Impacts such as learning increments may also play a factor in a longitudinal study and more user-led work would enable the HAPT system to be more flexible and enrich the users unique use of the digital haptic interaction.
Conclusion
Contribution to practice: recommendations for the training and teaching sector and wider use
The aim of this study was to gather qualitative, narrative data, and quantitative metrics to present the accuracy and experience by people who are blind and sight-impaired and to enhance their use of CAD. The literature offered a narrative through current literature on haptic adaptations to break down access to digital visual data. The findings enabled the research team at the Open University to gain a deeper understanding about the groups’ reactions to a novel haptic interaction and their personal approaches to tactually using the HAPT system. The most interesting, but unexpected results, were how blind and sight-impaired individuals worked with HAPT in a unique and personal tactual manner. Our approach was to offer novel ways to extend access to virtual space and virtual objects which are flexible enough to allow for interactions for people who are blind and partially sighted. The wider contribution of the Open University’s haptic systems to people whom are blind and sight-impaired may offer new ways to interact with teaching and learning materials in a more applied hands-on approach. This means that in future barriers to high-level applied technologies in the Open University could be lifted. Looking further forward, the HAPT system could also offer the potential to enhance how people who are blind or hold sight impairments may orientate around public buildings or public areas such as airports. The HAPT system allowed users to maximize the amount of contour following access but work would be needed on how to tag or label objects in a virtual map of a space to aid the users mental processing of the map and objects therein.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors received a small internal funding from the Open University eSTEeM funding resource as financial support for the research, authorship, and/or publication of this article.
