Unintended Biases due to Simulated Impairment within Inclusive Mobility Research and Design

Abstract

Simulated impairment refers to requiring persons without impairments to substitute persons with impairments in endeavors related to training, design creation, or usability testing. However, disability voices and research suggest limited effectiveness, greater distancing, and exclusion. Despite this, the simulation of physical limitations, including that of aging, continue to be used under the assumption that physical tasks and usability ratings do not significantly differ in simulation. In this experiment, age- and sex-matched older adults who use and do not use a manual wheelchair (MWC) are instructed to independently perform a self-paced parallel park using an MWC. The total clearance between obstacles required to perform a collision-free trial was recorded. Thirty-eight volunteers were recruited (MWC-user n=15; Simulated Impairment [SI] n=23). Higher clearance was required by the MWC group, suggesting the use of simulated impairment for motor tasks may result in bias. Open-ended questions revealed self-centered viewpoints, supporting literature that raises inclusion concerns regarding views of an “Other” group. Overall, our results support the direct engagement of target populations during early design to appropriately define user perspectives and needs. Designers should work with the community of people who face limitations rather than substituting their voices with those who may not accurately represent all of their consumer needs.

Keywords

Inclusive design Simulative impairment Accessibility

Introduction

Older adults with mobility disability

Within the rapidly growing older adult population, there are trends of increasing disability (JCHS, 2016; Freedman et al., 2013; Okoro et al., 2018). Among these disabilities, mobility disability is the most prevalent, affecting over a quarter of older adults with disabilities (Okoro et al., 2018). Therefore, in the fast approaching “Grey Wave”, there is a crucial need to provide support for independent mobility as it is a key aspect of healthy aging, social participation, and access to employment opportunities (Gallez et al., 2018; Kattenstroth et al., 2010).

Of concern, current accessibility standards have been shown to be insufficient for our quickly changing society and its expansion of mobility aid technology (IMRL, 2021; Hunter-Zaworski et al., 2013). Older adults with disabilities are at risk of being left behind, as made evident by the 22% of such individuals who sought employment but ceased efforts due to discouragement (BLS, 2022). If independent mobility in aging is not adequately supported, reliance on unpaid caregiving is anticipated to rapidly increase as caregiving jobs become less prevalent; such a reliance is expected to have significant negative impacts on patient and caregiver health and satisfaction as well as the economy (LaPlante, 2003; NAC, 2020; White et al., 2021).

Inclusive design holds promise in creating supportive products and environments for older adults with disabilities. However, some assumptions commonly used in user testing have not been validated in the context of inclusive mobility and design research. For example, recent studies have shown biases in traditional subjective evaluation methods when applied to populations that have been historically excluded (Hallet et al., 2020; Tabattanon et al; 2020; Tabattanon et al., 2023). One assumption that has been widely applied yet rarely tested is that relating to simulated impairment.

Simulated impairment

While design principles call for active communication for end users, literature shows that in many cases, a practice called simulated impairment is utilized. Simulated impairment refers to an interactive role-playing experience utilized towards the stated goal of understanding people with disabilities or aging adults by simulating capability limitations (Nario-Redmond et al., 2017). Examples of simulations include wearing restrictive gloves to mimic reduced joint range of motion, using a walker to increase one’s spatial footprint, or using special lenses to imitate vision loss. Simulated impairment is commonly used within design endeavors, for example to overcome time and budget conflicts from recruiting otherwise difficult to reach stakeholders or usability testers (Cardoso et al., 2012).

However, simulated impairment has been known to have numerous drawbacks. Foremost, the employment of simulations leads to the exclusion and diminished valuation of disability and aging voices (Bennett et al., 2019). It can also distort role-players’ attribution of the source of challenges (e.g., “blaming” a disability as the root cause of a challenge rather than an inaccessible design or system) and thus lead to the stereotyping of an “other” group as being less capable and generally different from oneself (Scullion, 1996; Silvermann et al., 2016). Despite these psychosocial concerns, it remains strongly assumed that simulated impairment holds validity as a pedagogical tool when applied to recreate the disability experience in design and empathy (Nario-Redmond et al., 2017). Investigations of validity and population differences have focused on the psychological, behavioral, or biomechanics outcomes, such as unintended consequences in stereotyping (Leo et al., 2016; Brown et al., 1990; Horvat et al., 1986). There is a knowledge gap regarding the validity in product and environmental design. Hence, the purpose of this study is to test the validity of simulated impairment for task measures that are largely physical in nature. Specifically, we examine the simulation of manual wheelchair usage, as it has been utilized in the literature towards accessible design (Bertocci et al., 2018; IMRL, 2021).

Methods

Participants

A summary is presented in Table 1 . The study was approved the University of Michigan Institutional Review Board. Online postings, internal contact databases, and advertisements at local senior and disability communities were utilized to reach volunteers. All participants were screened via either email or phone.

Table 1.

Mean participant characteristics (standard deviation). All characteristics except occupied depth are matched across groups (p>0.05).

	MWC (n = 15)	SI (n = 23)
Age (SD) (years)	62.4 (6.7)	60.2 (9.2)
Current biological sex	M: 5 F: 10	M: 8 F: 18
Grip strength (kg)	21.7 (8.0)	24.0 (8.0)
Occupied depth (cm) *(p=0.0009; d=1.04)	109.9 (13.1)	120.0 (2.9)

The recruitment criteria were:

Aged 50 years or older

MWC-user group (“MWC”): Uses a MWC for at least some mobility support

Simulated impairment group (“SI”): Does not use a MWC at all for mobility support

Able to independently move through a 10m path indoors while using a MWC

No significant cognitive, uncorrected visual impairment, spatial neglect, nor upper extremity amputation, loss, or injury

Participants who arrived in a MWC were asked to not perform any transfers. Participants who did not have or need a MWC used the same standard sized MWC provided by the study. Eight participants from the MWC group used the MWC we provided (e.g., they prefer to use a walker when driving is needed; in this case, all participants were informed that a researcher could meet them at their car with an MWC upon arrival). All MWC participants were right-handed; three SI participants were left-handed.

Study Procedure

Informed consent was received for all participants. Age and current biological sex were self-reported. Dominant hand grip strength was measured via a hand dynamometer and occupied depth in a manual wheelchair was measured by an experimenter while the participant adopted a self-selected posture for comfortable movement.

Then, one of the two experimenters set up two parallel blocks at a distance of the participant’s occupied depth plus 5cm. This was done in a way avoiding the participant to know how much initial clearance was allotted. In doing so, the study design relies on participants’ internal represen-tation of their occupied space and maneuverability.

Participants were then shown two parallel blocks, as illustrated in Figure 1 . A starting point was indicated slightly to the left and behind one of the blocks (i.e., similar to a car’s position as it approaches a parallel park spot). The participants were instructed to move in between the blocks, in whichever maneuver they preferred, such that their ending position was squarely within the blocks defined space and faced the same direction they started from ( Figure 2 ). The instructions emphasized that time was not limited, and speed was not a priority. The trial was repeated if collisions were noted by the experimenters standing opposite to the set-up. Participants were unaware the space was initially given (5cm) and the magnitude of the increments (2cm for each unsuccessful trial).

Figure 1.

Parallel parking task between two parallel cardboard walls. (Left)The participant begins slightly behind one block, facing forward; (Center) then moves at a self-paced, self-directed maneuver into the space; (Right) as to park between the blocks. No collision is depicted in any of these images. The participant shown was in the MWC group.

Figure 2.

Examples of collisions. The dash lines indicate the participant’s estimated gaze direction as head is masked. (Left) A collision with the front wall, the collision was anticipated but could not be avoided; (Right) An unanticipated collision with the back wall. The study MWC was used by both SI participants shown here.

Practice between the two parallel blocks was not permitted before test trials. However, a five-minute period of practice maneuvers in the large laboratory open space was provided. Both MWC and SI participants stated that they were envisioning the parallel blocks as they practiced in open space. All participants requested to move onto performing the task before the end of practice time.

Equipment

The parallel blocks were created from heavy-duty cardboard and weighted with wood blocks. Cardboard was selected to minimize risk to injury as during our previous work relating to inclusive mobility and design, some participants with disabilities pointed out that relatively gentle collisions with harder materials could cause excess pain due to conditions like nerve damage or joint pain.

A video camera was set-up perpendicular to the blocks, as in Figure 1 and 2, to allow for playback and analysis of strategy. At least one researcher observed from this location. Found surface electromyography (sEMG) electrode were placed on the participant’s dominant arm (upon their bicep, triceps, anterior deltoid, and posterior deltoid) and one inertial measurement unit (IMU) was placed on the MWC, however these physiological data and wheelchair movement data analysis is beyond the scope of the presented work.

Results

Minimum clearance required for a parallel park

A summary of results is provided in Figure 3 . All participants were able to complete the task without requesting assistance (i.e., withdrawing from the task). The average (± standard deviation) minimum clearance required to perform the parallel park without a collision was 10.7 (6.4) cm for the MCW group and 6.1 (2.5) cm for the SI group. One-way group mean comparison indicated a significant difference; p = 0.004; d = 1.04. Not all collisions were perceived by participants in either group, however a formal count of collision perception was not collected. The examples of collisions presented in Figure 2 were not perceived . The perception of collisions could potentially inform how and where environmental feedback can benefit motor action and will be included in future work. The average number of attempts taken by the MWC and SI groups were 3.8 and 1.5, respectively.

Figure 3.

Total clearance required by test groups in the parallel park task.

Group demographics and anthropometry were matched with the exception of occupied depth. On average, the occupied depth was smaller for the MWC compared to the SI group ( Table 1) despite the fact that over half of the MWC sample used the same device as the SI group.

Before, during, and after the task, many SI participants spoke freely on expectations that their performance would be ‘worse’ when compared to the MWC group. However, these comments were not included as a formal qualitative evaluation in this study.

Discussion

The results support the hypothesis that simulated impairment can potentially bias task measures commonly considered appropriate for disability simulation. Tasks in this category often include “purely physical” endeavors that may be practiced or simulated with a device, for example gloves to restrict range of motion or, as in our case, a MWC. Ongoing work aims to increase the sample size for both MWC and SI groups.

Range of MWC types

The type of MWCs owned and thus used by participants was not controlled for in this study. This was due to health and safety reasons such as personnel training for a range of transfers and consequential prolonged close contact with participants that conflict with COVID-19 precautions. While it may be speculated that MWC type could affect results, we assume that our conclusions with regards to simulated MWC usage holds in validity due to the following reasons:

Both user groups used our MWC. While some participants in the MWC group did arrive in their own device, eight of 15 (53%) used the same MWC that SI participants did. In addition, not all MWC users rely on MWC for total mobility support (e.g., using a walker when vehicular driving is needed). Hence, the performance of the SI group is not strongly dependent of wheelchair type.

Smaller MWC, larger depth. The occupied depth was smaller for the MWC than the SI group. Intuitively, one would expect smaller devices to bias results towards smaller required clearances. This, however, was not the case, suggesting that within the range of tested occupied space, this characteristic may not have a significant influence.

More custom MWC, larger depth. The smaller MWCs used by the MWC group were also more customed than the standard MWC offered by the study. Intuitively, one would expect these custom devices designed for daily living have tighter turn radiuses and higher maneuverability than a standard MWC. Although our study did not collect the model of each personal device, two MWC participants stated that their device was lightweight and of high maneuverability. However, our results suggest that the SI group showed greater maneuverability as a group.

Viewing disability by the ICF framework, these results suggest that factors other than the device used are impacting the maneuvering measures. Hence, it is postulated that the observed performance of MWC users is most likely influenced by lived experience, training, and personal factors such as self-efficacy rather than MWC type. Therefore, simulation of wheelchair maneuvers by non-impaired individuals may not be representative of a population affected by disability. The difference between impaired and non-impaired may stem from differences in internal task representation and assumed physical capabilities. This hypothesis is tested in an ongoing study.

Quantitative implications

In sum, the use of simulated impairment can lead to biased results. For example, if the study presented here had been a design study, then an age-matched simulated impairment user test trial would have suggested that a smaller space is acceptable for MWC navigation. Other tasks where MWC simulation is used include ramp exertion studies, vehicle design, and public transportation usability (e.g., Bertocci et al., 2018; Cardoso et al., 2012). As maneuverability may be biased by simulated impairment, measures such as time, satisfaction and difficulty of use, and effort may also be biased. Ongoing analysis of electromyography (EMG) data will compare muscular exertion between MWC and SI groups.

Quantitative recommendations provided by such standards as the Americans with Disabilities Act (ADA) have been shown to be insufficient for the changing and aging population (e.g., minimum clearance of on-vehicle accessibility spots; maximum weight requirements of ramps; maximum ramp slopes in buildings and vehicles) (IMRL, 2021). Further, standards like the ADA which were intended to be a minimum guideline are commonly treated as the maximum accommodation provided (Salmen, 2011). As we approach a large shift in the population’s working and consumer age, it would benefit businesses and regulators to actively include these populations rather than simulating their involvement.

Qualitative implications

Multiple SI participants freely expressed their thoughts before, during, and after performing the parallel parking maneuver. Themes noted include an expectation of performing “worse” compared to actual wheelchair users; overall frustration (one SI participant stated, “this is not fun”, and considered withdrawing); “feeling grateful” or “appreciation” for not requiring a wheelchair themselves; and pondering or doubting whether lateral movement was reflective of a real-life maneuver that wheelchair users have to perform. In contrast, multiple MWC participants reflected on times they have had to perform a lateral movement or maneuver into a tight space (e.g., on public transportation, moving as close to a wall as possible; in specific dining or office settings). MWC participants also noted potential benefits of using MWCs with greater maneuverability; one participant stated they owned such a device but did not always use it due to personal reasons.

These noted thematic differences support the literature with the field of disability studies. Specifically, SI participants understood the temporary nature of the simulation and viewed ambulation from self-centered lenses (Silverman et al., 2015). This has the potential to impact design as ideation can lean towards accessibility patches that support participation rather than inclusive systems that support equity. Such pitfalls may occur when inclusion is not actively addressed in early design processes or when “able-bodied” capabilities, in this case ambulation, is viewed as a norm rather than one of many means to complete a task. While an examination of the behavioral and psychological impact of simulated impairment is beyond the scope of this project, we note an importance for designers and researchers alike to understand and consider these aspects when analyzing qualitative data.

Participants in both the SI and MWC group noted the similarity between the task and a vehicular parallel park; in future iterations of this study, a questionnaire regarding driving experience and skill may be included. Likewise, subjective evaluations of the task were not formally collected due to the overall limited validation of subjective measures in the context of historically excluded populations; this study limitations may be addressed in our future work by incorporating subjected user evaluation scales such as the NASA TLX or Environmental Utility Measure scales (Tabattanon et al, 2020; Hart, 2006; Damle et al., 2001). Our results suggest that subjective ratings or open-ended comments stemming from simulated impairment perspectives may not reflect the lived experiences and learned strategies of the disability population. This aligns with disability studies literature (Leo et al., 2014; Flower et al., 2007).

Conclusions

The results strongly suggest that simulated impairment, although still popularly utilized in domains of design and empathy, may bias even “purely physical” outcomes. The constraints that lead a designer to employing simulated design are understandable, however they must be aware of biases and implications to the resulting products. Our study demonstrated the potential bias in the simulation of mobility disability. Towards avoiding unintended consequences, design processes ought to actively include disability and aging voices, as lived experiences have been shown to inform pain points, user needs, and learned skills that may be unseen within a simulation. Future work may also recruit younger adults to expand upon generalizability.

Footnotes

Acknowledgements

This research was funded by grants from the University of Michigan, Rackham Research Grant U077156, Education and Research Center award 5 T42 OH008455-17-00. The contents of this manuscript do not necessarily represent the policy of nor endorsement by grant agencies.

ORCID iD

Kamolnat Tabattanon

References

Bennett

Rosner

D.K.

(2019). The Promise of Empathy: Design, Disability, and Knowing the “Other”. Proceedings of the 2019 Conference in Human Factors in Computing Systems, pp. 1-13.

Bertocci

Smalley

Page

Digiovine

(2018). Manual Wheelchair Propulsion on Ramp Slopes Encountered When Boarding Public Transit Buses, Disability and Rehabilitation: Assistive Technology, 14(6), pp. 561-565.

Brown

D.D.

Knowlton

R.G.

Hamill

Schneider

T.L.

Hetzler

R.K.

(1990). Physiological and Biomechanical Differences Between Wheelchair-Dependent and Able-Bodied Subjects During Wheelchair Ergometry. European Journal of Applied Physiology and Occupational Physiology, 60, pp. 179-182.

Bureau of Labor Statistics (2022). Persons with a Disability: labor Force Characteristics - 2021. Available at: https://www.bls.gov/news.release/pdf/disabl.pdf

Cardoso

Clarkson

P.J.

(2012). Simulation in User-Centered Design: Helping Designers to Empathise with Atypical Users. Journal of Engineering Design, 23(1), pp. 1-22.

Damle

Danford

G.S.

Steinfeld

(2001). Environmental Utility Measure (EUMTM): Difficulty Rating Scale (DRSTM), Acceptability Rating Scale (ARS TM). Center for Inclusive Design & Environmental Access.

Flower

Burns

M.K.

Bottsford-Miller

N.A.

(2007). Meta-Analysis of Disability Simulation Research. Remedial and Special Education, 28(2), pp. 72-79.

Freedman

V.A.

Spillman

B.C.

Andreski

P.M.

Cornman

J.C.

Crimmins

E.M.

Kramarow

Lubitz

Martin

L.G.

Merkin

S.S.

Schoeni

R.F.

Seeman

T.E.

Waidmann

T.A.

(2013) Trends in Late-Life Activity Limitations in the United States: An Update From Five National Surveys. Demography, 50, pp. 661-671.

Gallez

Motte-Baumvol

(2017). Inclusive Mobility or Inclusive Accessibility? A European Perspective. Governing Mobility in Europe: Interdisciplinary Perspectives, 56, pp.79-104.

10.

Hallet

(2020). Designing with Extraordinary People: Stories of Creating Accessible Technology for the Five Percent. Proceedings of the 64th HFES Annual Meeting, pp: 981 - 984.

11.

Hart

S. G.

(2006). NASA-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, pp. 904-908.

12.

Horvat

French

Henschen

(1986). A Comparison of the Psychological Characteristics of Male and Female Able-Bodied and Wheelchair Athletes. Paraplegia, 24, pp. 115-122.

13.

Hunter-Zaworski

K. M. U.

Rutenberg

Shurland (2013). Recommendations for the PRIIA: Specifications on the Next Generation of Accessible Passenger Rail Cars. Proceedings of the 2013 Joint Rail Conference, pp. V001T02A015–V001T02A015.

14.

Inclusive Mobility Research Lab (IMRL) (2021). Design Guidelines for Wheeled Mobility Accessibility (Updated 2021). Inclusive Mobility Research Lab. Accessed online at: https://includable.org/aav/index.php

15.

Joint Center for Housing Services of Harvard University (2016). Projections and Implications for Housing a Growing Population: Older Households 2015-2035. Available at jchs.hardvard.edu.

16.

Kattenstroth

Kolankowska

Kalisch

To.

Dinse

H.R.

(2010). Superior Sensory, Motor, and Cognitive Performance in Elderly Individuals with Multi-year Dancing Activities. Frontiers in Aging Neuroscience, 2.

17.

Kaye

H.S.

Taewoon

LaPlante

M.P.

(2000). Mobility Device Use in the United States. Disability Statistics Report (14). US Department of Education, National Institute on Disability and Rehabilitation Research.

18.

LaPlante

M.P.

(2003). Demographics of Wheeled Mobility Device Users. 2003 Space Requirements for Wheeled Mobility: International Workshop.

19.

Leo

Goodwin

(2014). Negotiated Meanings of Disability Simulations in an Adapted Physical Activity Course: Learning from Student Reflections. Adapted Physical Activity Quarterly, 31, pp. 144-161.

20.

Leo

Goodwin

(2016). Simulating Others’ Realities: Insiders Reflect on Disability Simulations. Adapted Physical Activity Quarterly, 33, pp. 156-175.

21.

Okoro

C.A.

Hollis

N.D.

Cyrus

A.C.

Griffin-Blake

(2018). Prevalence of Disabilities and Health Care Access by Disability Status and Type Among Adults – United States, 2016. U.S. Department of Health and Human Services, Morbidity and Mortality Weekly Report, 67 (23), pp. 882-887.

22.

Nario-Redmond

M.R.

Gospodinov

Cobb

(2017). Crip for a Day: The Unintended Negative Consequences of Disability Simulations. Rehabilitation Psychology, 62(3), pp. 324-333.

23.

National Association for Caregiving (2020) CAREGIVING in the US 2020. Available at: https://www.caregiving.org/caregiving-in-the-us-2020/

24.

Salmen

J.P.S.

(2011) U.S. Accessibility Codes and Standards: Challenges for Universal Design. Universal Design Handbook.

25.

Scullion

(1996). Quasidisability experiences using simulation. British journal of therapy and rehabilitation, 4, 292-293.

26.

Silverman

A.M.

Gwinn

J.D.

Van Boven

(2015). Stumbling in Their Shoes: Disability Simulations Reduce Judged Capabilities of Disabled People. Social Psychological and Personality Science, 6(4), pp. 464-471.

27.

Tabattanon

Schuler

D’Souza

(2020). Investigating Inclusive Design of Shared Automated Vehicles with Full-Scale Modeling. Proceedings of the 64th Human Factors and Ergonomics Annual Meeting, pp: 965-969.

28.

Tabattanon

Martin

B.J.

(2023, Feb.). Impact of disability incidence and simulation on inclusive mobility design. At the February Education and Research Center: Ergonomics Research Update, Virtual Webinar Series.

29.

White

DeAntonio

Ryan

Colyar

(2021). The Economic Impact of Caregiving. Report prepared for the Blue Cross Blue Shield Association.