Feasibility of utilizing clinical and driving simulator assessments to indicate driving performance deficits in adults with multiple sclerosis

Abstract

BACKGROUND:

Drivers with Multiple Sclerosis (MS) experience visual-cognitive impairment that impact on-road driving performance.

OBJECTIVE:

This study examines the feasibility of utilizing visual-cognitive and driving simulator assessments to indicate driving performance deficits (operational, tactical, and strategic maneuvers) in drivers with MS.

METHODS:

Through an evidence-informed feasibility framework, we evaluated recruitment capability and resulting sample characteristics, data collection procedures and outcome measures, participants’ acceptability and suitability of the driving simulator, the resources and ability to implement the study, and clinical and driving simulator assessment results.

RESULTS:

Thirty-eight persons with MS (median age $=$ 43 years, IQR $=$ 19) and 21 persons without MS (median age $=$ 41 years, IQR $=$ 14) participated. Missing data on the driving simulator resulted from scenario complexity (13 with MS, 4 without MS) or the onset of simulator sickness (1 with MS, 1 without MS). Seven participants with MS and two participants without MS reported symptoms of simulator sickness. Participants with MS (vs without MS) made more adjustment to stimuli errors (tactical maneuvers). For participants with MS, immediate verbal/auditory recall or divided/selective attention correlated with simulated driving maneuvers.

CONCLUSIONS:

Study findings identified challenges (missing data, simulator sickness), but established feasibility for executing a full-scale study to predict driving simulator performance in drivers with MS.

Keywords

Assessment cognitive dysfunction automobile driving feasibility studies multiple sclerosis high fidelity simulation training

ï»¿

1. Introduction

Drivers with Multiple Sclerosis (MS) experience visual-cognitive impairment that may compromise their fitness to drive [1]. Fitness to drive is determined via a comprehensive driving evaluation, with clinical and on-road driving assessments [2]. However, the comprehensive driving evaluation may be restrictive for some medically at-risk drivers, as it may be expensive, inaccessible, and may pose a crash risk during the on-road assessment, which can occur in real-world traffic conditions. Driving simulators offer a safe and secure assessment of driving performance for medically at-risk drivers [3]. Accordingly, a need exists to examine whether driving simulators can be used to indicate the underlying driving performance impairments of drivers with MS and inform decisions about a person’s fitness to drive.

Based on the extant literature, little congruency exists for which clinical tests indicate deficits in on-road and driving simulator outcomes in drivers with MS [1]. On-road studies show that impairment in binocular visual acuity, attention, information processing (visual), executive functioning, and visuospatial ability indicate failing outcomes [4, 5, 6, 7, 8, 9, 10, 11]. The driving errors that predict pass/fail outcomes include adjustment to stimuli errors (operational, tactical) and strategic errors in suburban/urban environments [9, 12, 13]. Conversely, driving simulator studies show that impairment in information processing (auditory) and divided attention indicate adjustment to stimuli errors (operational) of highway drives with secondary attention tasks in drivers with MS, as compared to those without MS [14, 15, 16, 17, 18]. Overall, these findings make it difficult to understand whether the evidence-based visual-cognitive indicators of on-road outcomes can indicate driving simulator outcomes in drivers with MS. As driving simulator studies mostly report on operational maneuvers of highway drives [14], it is difficult to understand whether driving simulators can be utilized to assess tactical and strategic maneuvers in suburban and urban environments – the maneuvers and environments previously shown to indicate failing an on-road assessment [9, 12, 13].

Based on prior on-road study findings [8, 9, 10, 12, 13], this study examined the feasibility of utilizing visual-cognitive and driving simulator assessments to indicate deficits in operational, tactical, and strategic driving maneuvers in drivers with and without MS. Feasibility was examined via using an evidence-informed framework, developed by Orsmond and Cohn who synthesized research studies on feasibility and developed a conceptual framework with five objectives to inform the guidance and development of future interventions [19]. Our study did not include an intervention but addressed the foundational components of feasibility through evaluating: 1) recruitment capability and resulting sample characteristics; 2) data collection procedures and outcome measures; 3) acceptability and suitability of the driving simulator; 4) resources and ability to implement the study; and 5) clinical and driving assessment results [19].

Examining the study’s feasibility provided information pertaining to: recruiting eligible participants with similar demographic characteristics to the MS population-based sample within the planned timeframe; the adequacy of data collection procedures and outcome measures; participants’ perceptions toward using the driving simulator and whether the onset of simulator sickness affected completing the simulated scenarios; obtaining the resources and ability to conduct the study successfully as per the protocol; and preliminary results that identified potential clinical indicators of driving performance in drivers with MS. Study findings that confirm feasibility will lay the foundation for executing a full-scale study to quantify the clinical tests that predict driving performance. Understanding if driving performance deficits that contribute to on-road outcomes can be detected on a driving simulator is important for making decisions about one’s driving performance.

2. Methods

The study’s methods, procedures, and participants’ demographic findings are documented in publications examining predictors of driving simulator performance in drivers with MS [20, 21].

2.1 Participants

From January 2019 to February 2020, we recruited participants 18 to 59 years with a valid graduated driver’s license via advertising in the London (Ontario) MS Clinic, Western University, MS Society of Canada’s Research Portal, social media sites (i.e., Facebook), network sites (i.e., Kijiji, Craigslist), and online editions of local newspapers. Inclusion criteria ensured that participants met Ontario’s legal vision standards to drive (i.e., corrected/uncorrected, far-sighted binocular visual acuity of at least 20/50 and binocular horizontal field of view of at least 120 degrees continuous) [22], and those with MS had a physician-verified diagnosis [23] and low to moderate physical disability (Expanded Disability Status Scale [EDSS] score $\leqslant$ 6.5) [24]. Exclusion criteria were those: with other physician-verified medical, neurological, or psychiatric diagnoses; who took medications/illicit drugs that could affect cognition or driving ability; and/or who experienced relapses, had corticosteroid treatment three months preceding study enrolment, and/or experienced severe fatigue (Fatigue Severity Scale mean score $>$ 5) [25] and/or severe depression (Beck Depression Index Fast Screen total score $\geqslant$ 14) [26].

2.2 Study protocol

Participants attended one in-person visit at Western University’s i-Mobile Driving Research Lab. Upon obtaining written informed consent and screening participants to confirm eligibility, participants completed a demographic and medical intake form [27], Driver Behaviour Questionnaire [28], and a clinical assessment of visual and cognitive ability [29, 30, 31, 32, 33], administered by the trained research student.

2.2.1 Clinical assessment

The OPTEC ${}^{\@setsize{\scriptsize}{8pt}{\viipt}{\@viipt}\textregistered}$ 5000 Peripheral Glare Vision Screener measured far-sighted binocular visual acuity (furthest row with ${<}$ 2 errors) [29]. The Useful Field of View ${}^{\text{TM}}$ (UFOV) measured central visual processing speed (UFOV1), divided attention (UFOV2), and selective attention (UFOV3) via responding to items presented on a computer screen (mean response accuracy in milliseconds) [30]. The Symbol Digit Modalities Test-Oral Version (SDMT) measured visual information processing by interpreting and reading the symbols’ paired numbers (no. correct responses/90 seconds) [31]. The California Verbal Learning Test-Second Edition, Immediate Recall Measure (CVLT2-IR), measured immediate verbal/auditory recall through five trials of recalling words on an itemized list (no. correct responses/80) [32]. The Brief Visuospatial Memory Test-Revised Version, Immediate Recall Measure (BVMTR-IR), measured immediate visuospatial recall through three trials of recalling and drawing six geometric figures accurately and in their locations on a display (no. correct responses/36) [33]. The BVMTR Delayed Recall Measure (BVMTR-DR), measured delayed visuospatial recall through one trial of recalling and drawing the geometric figures accurately and in their locations on the display 20 minutes after the BVMTR-IR (no. correct responses/12) [33].

We selected the visual and cognitive tests for this study, as they previously indicated deficits in on-road outcomes in drivers with MS [8, 9, 10]. Notably, losses in far-sighted binocular visual acuity correlated with adjustment to stimuli errors ( $r_{s}=$ 0.5, $p=$ 0.006), slower central visual processing speed correlated with gap acceptance errors ( $r_{s}=$ 0.4, $p=$ 0.03), and deficits in delayed visuospatial recall correlated with strategic errors ( $r_{s}=-$ 0.4, $p<$ 0.05) [8, 9]. Furthermore, impaired information processing and immediate visuospatial recall predicted failing an on-road assessment ( $x^{2}$ ( $df=$ 1, $N=$ 36) $=$ 7.3, $p=$ 0.007) [10].

2.2.2 Driving simulator assessment

After the clinical assessment, participants completed a driving assessment on the CDS 200 Simulator (DriveSafety Inc., Murray, Utah), administered by the trained research student. As displayed in Fig. 1, the computer desktop simulator had a fixed-based platform, three 19-inch LCD screens that covered 110-degrees horizontal field of view, and driver controls including a steering wheel, signal indicators, accelerator/brake pedals, and rear and side view wide-angle mirrors.

Figure 1.

CDS 200 Simulator by DriveSafety, Inc. (https://drivesafety.com). Note. This figure is reprinted with permission.

Figure 2.

Description of Events in Driving Simulator Assessment by DriveSafety, Inc. (https://drivesafety.com). Note. This figure is adapted and printed with permission. The descriptions in the figure are documented in prior studies examining the predictors of driving simulator performance in drivers with MS [20, 21].

The driving assessment included a protocol to mitigate symptoms of simulator sickness [34, 35]. We asked participants to refrain from alcohol, caffeine, high-fat, or any mind-altering substance intake 24 hours preceding the study visit. The simulator lab had an exclusive air conditioning unit with a room temperature of 72 degrees Fahrenheit (22 degrees Celsius), a tower fan to ensure consistent air flow throughout the room, and dim light settings to reduce glare from the simulator’s screen monitors. The research student calibrated and adjusted the simulator’s driver controls to meet participants’ anthropometric measures. Participants reported measures of simulator sickness (Modified Motion Sickness Assessment Questionnaire, i.e., 0–10, not at all to severely for sweaty, queasy, dizzy, nauseous) at baseline, before and after each drive [34]. A cut-point $\geqslant$ 5 indicated to terminate the driving simulator assessment due to experiencing symptoms of simulator sickness [34]. Soda crackers, ginger ale, and water were available if participants experienced any discomfort. Participants were offered breaks, or to walk outside, allowing them to return to baseline conditions prior to resuming simulated driving or to quit if discomfort continued.

Participants completed two pre-driving exercises to practice operating and controlling the driver controls and three adaptation drives to practice driving up to 50 kilometres per hour (31 miles per hour), stopping, and turning left/right on residential, suburban, or urban roads with other road users [36]. Participants completed a driving scenario, previously developed, refined, and validated to detect adjustment to stimuli errors and visual scanning errors of youth drivers [36]. The driving scenario took 10 minutes to maneuver, with pre-recorded auditory cues, through 12 intersections, four left turns, three right turns, four hazardous events and a navigational driving task, when driving 40–60 kilometres per hour (25–37 miles per hour) [36]. Figure 2 provides a detailed description of the operational, tactical, and strategic driving maneuvers and environments of hazardous events and the navigational driving task.

Based on Michon’s hierarchical model of driving behavior, operational maneuvers required automatic and habitual physical abilities to respond to the environment (e.g., pressing foot pedals in response to traffic signs) [37]. Tactical maneuvers required intermittent problem-solving and decision-making to navigate through the environment (e.g., judging the time/space required to overtake road users) [37]. Strategic maneuvers required higher-level reasoning, decision-making, and problem-solving to navigate a route beforehand or when driving, while considering traffic laws, flow of traffic, and the risks and challenges in the environment [37].

We selected this scenario to assess participants’ adjustment to stimuli (operational, tactical) and strategic driving maneuvers during the events, as adjustment to stimuli errors in suburban (OR $=$ 0.4, $p=$ 0.01, 95% CI $=$ [0.2, 0.8]) and urban environments (OR $=$ 0.3, $p=$ 0.03, 95% CI $=$ [0.1, 0.9]), and lane maintenance (OR $=$ 0.2, $p=$ 0.009, 95% CI $=$ [0.0, 0.7]) and speed regulation errors (OR $=$ 0.04, $p=$ 0.009, 95% CI $=$ [0.0, 0.4]) of a strategic driving maneuver previously predicted drivers with MS failing an on-road assessment [9, 12, 13].

2.2.2.1 Driving outcomes

We quantified participants’ adjustment to stimuli (operational, tactical) via reaction time, in seconds from when the event started until initial accelerator/brake pedal contact; response time, in seconds from when the event started until participants either completely stopped or drove past the road user/object (e.g., pedestrian); mean speed, in meters per second from when the event started until participants completely stopped or drove past the road user/object; and response type, categorized as participants who stopped, failed to stop, or experienced a collision. If participants experienced a collision, the driving scenario stopped until the research student advanced the scenario so participants could proceed with the drive after the collision. We quantified participants’ strategic maneuvers via a correct decision (i.e., turned toward London, Ontario, Canada) or incorrect decision (i.e., turned left toward Toronto, Ontario, Canada or drove straight through the intersection). The DriveSafety ${}^{\text{TM}}$ engineer team verified all driving simulator measures (email, 2020 May 8 from S Hallmark, K Melnrick, D Evans).

2.2.2.2 Video recordings

To optimize data collection, we video-recorded participants’ driving scenario with two Logitech C922 Pro Stream cameras. The cameras recorded high-definition videos with 1080 pixels at 30 frames per second. One camera was located behind the driver to record the driving scenario on the simulator’s monitor screens, while the other camera was located on the simulator’s middle monitor screen to record the driver’s eyes and face. We used Logitech Capture software to simultaneously video record the driving scenario (e.g., entire screen) with participants’ eyes and face (e.g., bottom left screen) as one mp4 file. Video recordings were stored as mp4 files on a password protected and encrypted folder on the i-Mobile Driving Research Lab’s computer drive. We viewed each participant’s video recordings once to verify the data collected on the driving simulator testing form and the metrics collected by the driving simulator. For example, the video recordings verified participants’ response type (i.e., stopped vs. failed to stop) or the occurrence of collisions (i.e., collided vs. did not collide) in each event and the navigational driving task decision (i.e., correct vs. incorrect).

2.3 Technology acceptance

To understand whether participants would accept or intend to use a driving simulator to assess their driving performance, we asked them to complete the Perceived Usefulness and Ease of Use Questionnaire [38] and the System Usability Scale [39]. The Perceived Usefulness and Ease of Use Questionnaire included 12 questions (1–7, strongly disagree to strongly agree) about the driving simulator’s usefulness and usability [38]. The System Usability Scale included 10 questions (1–5, strongly disagree to strongly agree) about the driving simulator’s usefulness and satisfaction [39].

2.4 Data analysis

Data analyses were performed with SPSS Statistics 26 (IBM Corporation, 2019), using two-sided tests with a significance level ${\alpha}=$ 0.05. Shapiro-Wilk tests of continuous variables showed that most data were not normally distributed in both participants with and without MS. Accordingly, summary statistics (e.g., medians, interquartile range, range, frequencies, percentages) were computed to describe findings in each of the five objectives. Mann-Whitney U, Chi-square ( ${{x}}^{{2}}$ ), or Fisher’s exact tests quantified if differences in demographic characteristics (objective 1), clinical test scores (objective 1), reported symptoms of simulator sickness (objective 3), or simulated driving maneuvers (objective 5) existed between participants with and without MS (and between those experiencing and not experiencing symptoms of simulator sickness for objective 3). For participants with MS, spearman rho ( $r_{s}$ ) or rank biserial correlations ( $r_{rb}$ ) quantified the strength and direction of correlations between demographic or clinical characteristics and reported symptoms of simulator sickness (objective 3), and between clinical test scores and simulated driving maneuvers (objective 5). Correlations $<$ 0.3 were weak, 0.3 to $<$ 0.7 were moderate, and 0.7 to 1.0 were strong to perfect [40].

Table 1
Sample characteristics ( $N=$ 59)

Characteristics	Participants		Between-group difference test
	With MS ( $n=$ 38)	Without MS ( $n=$ 21)
Demographic
Age (years)	43 $\pm$ 19 (36)	41 $\pm$ 14 (36)	$U=$ 332, $p=$ 0.29
Sex			$\chi^{2}$ ( $df=$ 1) $=$ 0.06, $p=$ 1.00
Male	12 (32%)	6 (29%)
Female	26 (68%)	15 (71%)
Ethnicity			Fisher’s $=$ 2.8, $p=$ 0.17
Caucasian	36 (95%)	17 (81%)
Other	2 (5%)	4 (19%)
No. medications	3 $\pm$ 4 (14)	0 $\pm$ 0.5 (2)	$U=$ 62.5, $p<$ 0.0001
No. years education	16 $\pm$ 3 (13)	18 $\pm$ 5 (12)	$U=$ 277.5, $p=$ 0.053
Employment status			Fisher’s $=$ 0.8, $p=$ 0.47
Employed/student	31 (82%)	19 (91%)
Unemployed/disabled	7 (18%)	2 (9%)
No. years driving	26 $\pm$ 19 (36)	24 $\pm$ 17 (35)	$U=$ 366, $p=$ 0.61
No. days/week driven	7 $\pm$ 2 (7)	7 $\pm$ 2 (7)	$U=$ 374, $p=$ 0.65
No. kilometres driven/day	20 $\pm$ 48 (188)	20 $\pm$ 27 (150)	$U=$ 315.5, $p=$ 0.19
Driver Behaviour Questionnaire (Mean score, 1–6)
Slips	1.6 $\pm$ 0.3 (1.5)	1.6 $\pm$ 0.5 (1.2)	$U=$ 312.0, $p=$ 0.17
Violations	1.4 $\pm$ 0.4 (1.2)	1.5 $\pm$ 0.5 (1.4)	$U=$ 336.0, $p=$ 0.32
Mistakes	1.6 $\pm$ 0.4 (1.6)	1.7 $\pm$ 0.6 (1.4)	$U=$ 359.5, $p=$ 0.54
Clinical
Peripheral fields (degrees)	170 $\pm$ 0 (10)	170 $\pm$ 0 (30)	$U=$ 381, $p=$ 0.45
Visual acuity			Fisher’s $=$ 1.0, $p=$ 0.41
$\leqslant$ 20/40	33 (87%)	20 (95%)
$\geqslant$ 20/50	5 (13%)	1 (5%)
Useful Field of View ${}^{\text{TM}}$ (milliseconds)
Subtest 1	16.8 $\pm$ 5.3 (13.2)	15.1 $\pm$ 5.4 (10.7)	$U=$ 344.5, $p=$ 0.39
Subtest 2	20.8 $\pm$ 9.1 (336.6)	19.1 $\pm$ 7.8 (16.1)	$U=$ 339.0, $p=$ 0.35
Subtest 3	33.7 $\pm$ 41.5 (350.5)	28.9 $\pm$ 50.7 (140.1)	$U=$ 369.0, $p=$ 0.64
SDMT (/90s)	62.0 $\pm$ 17.8 (46)	64.0 $\pm$ 14.0 (39)	$U=$ 287.5, $p=$ 0.08
CVLT2-IR (/80)	56.0 $\pm$ 14.0 (48)	56.0 $\pm$ 17.5 (33)	$U=$ 369.5, $p=$ 0.65
BVMTR-IR (/36)	28.5 $\pm$ 11.0 (27)	28.0 $\pm$ 8.0 (21)	$U=$ 391.5, $p=$ 0.91
BVMTR-DR (/12)	11.0 $\pm$ 2.3 (8)	11.0 $\pm$ 3.0 (8)	$U=$ 331.0, $p=$ 0.27

Note. MS $=$ Multiple Sclerosis; SDMT $=$ Symbol Digit Modalities Test-Oral Version; CVLT2 $=$ California Verbal Learning Test-Second Edition; IR $=$ Immediate Recall; DR $=$ Delayed Recall; BVMTR $=$ Brief Visuospatial Memory Test-Revised Version. Continuous data are summarized with medians $\pm$ interquartile range (range); categorical data are summarized with frequencies (percentages). The table includes findings that are documented in studies that examined the predictors of driving simulator performance in drivers with MS [20, 21]. $p\leqslant$ 0.05, two-tailed.

3. Results

3.1 Objective 1: Recruitment capability and resulting sample characteristics

Out of 180 individuals (84 with MS, 96 without MS) who expressed interest in the study, 38 individuals with MS (95% of recruitment goal) and 21 individuals without MS (105% of recruitment goal) participated. Participants were recruited via the London (Ontario) MS Clinic (33 with MS, 3 without MS), Western University (4 with MS, 18 without MS), and the MS Society of Canada’s Research Portal (1 with MS). Of individuals who declined participating, most did not provide a reason (26 with MS, 17 without MS). For those who provided reasons, eleven individuals with MS said it was too much of a time commitment, nine individuals (5 with MS, 4 without MS) declined due to the study location, and three individuals (1 with MS, 2 without MS) did not agree or follow through with the study requirements (e.g., did not attend study visit). For individuals who were excluded from the study, 49 individuals without MS inquired about taking part after recruitment was complete, and six individuals did not meet the inclusion criteria: two individuals with MS and two individuals without MS had medical diagnoses, one individual with MS took medication that could have affected cognition or driving ability, and one individual without MS did not meet the age criterion. We did not exclude any participants once they enrolled in the study.

Table 1 summarizes the demographic and clinical characteristics of participants with and without MS. As documented in prior studies [20, 21], participants with MS, compared to those without MS, took significantly more medications. In the MS group, 35 participants had relapsing MS (92%; ages 22–58, 24 female) and 3 had progressive MS (8%; ages 47–58, 2 female) [20, 21]. Diagnosis occurred a median of 8 years preceding the study (IQR $=$ 17.3; range $=$ 30): 0–30 years for those with relapsing MS and 5–25 years for those with progressive MS. Participants had a low to moderate physical disability (median EDSS $=$ 2.0, IQR $=$ 1.5): between 1 and 4 for participants with relapsing MS and between 3 and 5 for participants with progressive MS. Lastly, participants with MS had a median score of 4.2 (IQR $=$ 2.4) on the Fatigue Severity Scale, and a median score of 1.0 (IQR $=$ 2.3) on the Beck Depression Index Fast Screen [20, 21].

Table 2
Participant Responses on the Perceived Usefulness and Ease of Use Questionnaire ( $N=$ 54) and the System Usability Scale ( $N=$ 53)

Statement	Participants
	With MS	Without MS
Perceived Usefulness and Ease of Use Questionnaire	$n=$ 33	$n=$ 21
1. Using a driving simulator would enable me to accomplish driving tasks more effectively	4 $\pm$ 3 (6)	4 $\pm$ 3 (5)
2. Using a driving simulator would improve my driving performance	4 $\pm$ 3 (6)	4 $\pm$ 3 (5)
3. Using a driving simulator would increase my driving skills	4 $\pm$ 3 (6)	3 $\pm$ 3 (4)
4. Using a driving simulator would enhance my effectiveness in driving	4 $\pm$ 3 (5)	3 $\pm$ 3 (5)
5. Using a driving simulator would make it easier to drive	3 $\pm$ 3 (6)	3 $\pm$ 3 (5)
6. I would find a driving simulator to be useful for my driving	4 $\pm$ 2 (5)	3 $\pm$ 4 (5)
7. Learning to operate a driving simulator would be easy for me	4 $\pm$ 3 (6)	5 $\pm$ 3 (5)
8. I would find it easy to get a driving simulator to do what I want it to do	4 $\pm$ 3 (5)	4 $\pm$ 3 (6)
9. My interaction with a driving simulator would be clear and understandable	5 $\pm$ 2 (6)	5 $\pm$ 2 (5)
10. I would find a driving simulator flexible to interact with	5 $\pm$ 3 (6)	4 $\pm$ 1 (5)
11. It would be easy for me to become skillful at using a driving simulator	5 $\pm$ 3 (6)	5 $\pm$ 2 (4)
12. I would find a driving simulator easy to use	5 $\pm$ 2 (6)	5 $\pm$ 2 (5)
System Usability Scale	$n=$ 32	$n=$ 21
1. I think that I would like to use this system frequently	2 $\pm$ 2 (3)	2 $\pm$ 2 (3)
2. I found the system unnecessarily complex	1 $\pm$ 1 (2)	1 $\pm$ 1 (2)
3. I thought the system was easy to use	4 $\pm$ 2 (4)	4 $\pm$ 2 (3)
4. I think that I would need the support of a technical person to be able to use this system	2 $\pm$ 3 (4)	2 $\pm$ 2 (4)
5. I found the various functions in this system were well integrated	4 $\pm$ 2 (4)	3 $\pm$ 1 (3)
6. I thought that there was too much inconsistency in this system	2 $\pm$ 2 (4)	2 $\pm$ 2 (4)
7. I would imagine that most people would learn to use this system very quickly	4 $\pm$ 2 (4)	4 $\pm$ 1 (3)
8. I found the system to be very cumbersome to use	2 $\pm$ 1 (3)	2 $\pm$ 1 (3)
9. I felt very confident using the system	4 $\pm$ 1 (4)	4 $\pm$ 1 (4)
10. I needed to learn a lot of things before I could get going with this system	2 $\pm$ 2 (4)	2 $\pm$ 2 (3)

Note. MS $=$ Multiple Sclerosis. Participants’ ratings are summarized as medians $\pm$ interquartile range (range). Item ratings for the Perceived Usefulness and Ease of Use Questionnaire: 1 $=$ strongly disagree; 2 $=$ moderately disagree; 3 $=$ slightly disagree; 4 $=$ neutral; 5 $=$ slightly agree; 6 $=$ moderately agree; 7 $=$ strongly agree [38]. Item ratings for the System Usability Scale: 1 $=$ strongly disagree; 2 $=$ disagree; 3 $=$ neutral; 4 $=$ agree; 5 $=$ strongly agree [39]. Reasons for missing data on these questionnaires are discussed under 3.2 Objective 2: Data collection procedures and outcome measures.

3.2 Objective 2: Data collection procedures and outcome measures

All participants completed the clinical and driving simulator assessment; however, up to seventeen (28.8%) participants had missing data on the simulator. Reasons for missing data included: participants missing the landmark to trigger event 1 to occur (11 with MS, 4 without MS); the driving simulator not loading the driving scenario (1 with MS) or event 4 to occur (3 with MS); and experiencing symptoms of simulator sickness (1 with MS, 1 without MS). Furthermore, five participants with MS did not complete the Perceived Usefulness and Ease of Use Questionnaire and System Usability Scale because they completed the study before we added the questionnaires to the study protocol. When testing the participants, we observed different driving performance responses to events (e.g., stopped, failed to stop, collision), including the onset of simulator sickness. Accordingly, we added both questionnaires into the study protocol to examine participants’ intention to undergoing a driving simulator assessment. Lastly, one participant with MS did not complete the System Usability Scale during the study visit.

3.3 Objective 3: Acceptability and suitability of the driving simulator

Table 2 presents participants’ responses on the Perceived Usefulness and Ease of Use Questionnaire and System Usability Scale. On the Perceived Usefulness and Ease of Use Questionnaire, participants agreed the most on statements about the driving simulator being easy to interact with or use (e.g., item 12, median response $=$ 5 “slightly agree”) and the least on statements about using the driving simulator to improve their driving (e.g., item 5, median response $=$ 3 “slightly disagree”). On the System Usability Scale, participants agreed the most on statements about the driving simulator being functional, well-integrated, or easy to use (e.g., item 9, median response $=$ 4 “agree”) and the least on statements about the driving simulator being unnecessarily complex or technical to use (e.g., item 2, median response 1 $=$ “strongly disagree”).

Table 3
Participant Responses on the Modified Motion Sickness Assessment Questionnaire

Time of assessment	Participants		$p$	Experienced symptoms of simulator sickness		$p$
	With MS	Without MS		Yes	No
Baseline
Sweaty 0 1	0 $\pm$ 0 (1)97.3%2.7%	0 $\pm$ 0 (1)95.2%4.8%	0.68	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (1)95.9%4.1%	0.54
Queasy 0 3	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (3)95.2%4.8%	0.18	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (3)98.0%2.0%	0.67
Dizzy 0 1 3	0 $\pm$ 0 (3)94.6%2.7%2.7%	0 $\pm$ 0 (0)100%0%0%	0.28	0 $\pm$ 0 (0)100%0%0%	0 $\pm$ 0 (3)95.9%2.0%2.0%	0.54
Nauseous 0 1	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (1)90.5%9.5%	0.06	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (1)95.9%4.1%	0.54
Pre-drive 1
Sweaty 0 1	0 $\pm$ 0 (1)96.9%3.1%	0 $\pm$ 0 (1)94.7%5.3%	0.71	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (1)95.5%4.5%	0.86
Queasy 0	0 $\pm$ 0 (0)100%	0 $\pm$ 0 (0)100%	1.00	0 $\pm$ 0 (0)100%	0 $\pm$ 0 (0)100%	1.00
Dizzy 0 1 3	0 $\pm$ 0 (3)93.8%3.1%3.1%	0 $\pm$ 0 (0)100%0%0%	0.27	0 $\pm$ 0 (0)100%0%0%	0 $\pm$ 0 (3)95.5%2.3%2.3%	0.86
Nauseous 0 1	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (1)94.7%5.3%	0.19	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (1)97.7%2.3%	0.93
Pre-drive 2
Sweaty 0 1	0 $\pm$ 0 (1)96.8%3.2%	0 $\pm$ 0 (1)94.7%5.3%	0.72	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (1)95.5%4.5%	0.87
Queasy 0	0 $\pm$ 0 (0)100%	0 $\pm$ 0 (0)100%	1.00	0 $\pm$ 0 (0)100%	0 $\pm$ 0 (0)100%	1.00
Dizzy 0 1 3	0 $\pm$ 0 (3)93.5%3.2%3.2%	0 $\pm$ 0 (0)100%0%0%	0.26	0 $\pm$ 0 (0)100%0%0%	0 $\pm$ 0 (3)95.5%2.3%2.3%	0.87
Nauseous 0	0 $\pm$ 0 (0)100%	0 $\pm$ 0 (0)100%	1.00	0 $\pm$ 0 (0)100%	0 $\pm$ 0 (0)100%	1.00
Adaptation drive 1
Sweaty 0 1 2	0 $\pm$ 0 (1)97.3%2.7%0%	0 $\pm$ 0 (2)95.2%0%4.8%	0.66	0 $\pm$ 0 (0)100%0%0%	0 $\pm$ 0 (2)95.9%2.0%2.0%	0.54
Queasy 0 1	0 $\pm$ 0 (0)100%0%	0 $\pm$ 0 (1)90.5%9.5%	0.06	0 $\pm$ 0 (1)88.9%11.1%	0 $\pm$ 0 (1)98.0%2.0%	0.17
Dizzy 0 1 2 3	0 $\pm$ 0 (3)86.5%8.1%2.7%2.7%	0 $\pm$ 0 (0)100%0%0%0%	0.08	0 $\pm$ 0 (0)100%0%0%0%	0 $\pm$ 0 (3)89.8%6.1%2.0%2.0%	0.32
Nauseous 0	0 $\pm$ 0 (0)100%	0 $\pm$ 0 (0)100%	1.00	0 $\pm$ 0 (0)100%	0 $\pm$ 0 (0)100%	1.00

Table 3, continued
Time of assessment	Participants		$p$	Experienced symptoms of simulator sickness		$p$
	With MS	Without MS		Yes	No
Adaptation drive 2
Sweaty 0 1 3	0 $\pm$ 0 (1)97.3%2.7%0%	0 $\pm$ 0 (3)95.2%0%4.8%	0.66	0 $\pm$ 0 (0)100%0%0%	0 $\pm$ 0 (3)95.9%2.0%2.0%	0.54
Queasy 0 1 2	0 $\pm$ 0 (2)81.1%13.5%5.4%	0 $\pm$ 0 (2)85.7%9.5%4.8%	0.67	1 $\pm$ 2 (2)44.4%22.2%33.3%	0 $\pm$ 0 (1)89.8%10.2%0%	$<$ 0.001
Dizzy 0 1 2 3 4	0 $\pm$ 1.5 (4)67.6%8.1%13.5%8.1%2.7%	0 $\pm$ 0.5 (3)76.2%14.3%4.8%4.8%0%	0.38	1 $\pm$ 1.5 (2)44.4%33.3%22.2%0%0%	0 $\pm$ 0.5 (4)75.5%6.1%8.2%8.2%2.0%	0.16
Nauseous 0 1 2	0 $\pm$ 0 (0)100%0%0%	0 $\pm$ 0 (2)85.7%9.5%4.8%	0.02	0 $\pm$ 0.5 (1)77.8%22.2%0%	0 $\pm$ 0 (2)98.0%0%2.0%	0.02
Adaptation drive 3
Sweaty 0 1 3	0 $\pm$ 0 (1)97.2%2.8%0%	0 $\pm$ 0 (3)95.2%0%4.8%	0.68	0 $\pm$ 0 (0)100%0%0%	0 $\pm$ 0 (3)95.8%2.1%2.1%	0.54
Queasy 0 1 2 3	0 $\pm$ 0 (3)80.6%13.9%2.8%2.8%	0 $\pm$ 0.5 (2)76.2%19.0%4.8%0%	0.74	0 $\pm$ 1 (3)55.6%33.3%0%11.1%	0 $\pm$ 0 (2)83.3%12.5%4.2%0%	0.06
Dizzy 0 1 2 3 4	0 $\pm$ 1.8 (4)61.1%13.9%8.3%8.3%8.3%	0 $\pm$ 1 (3)66.7%19.0%9.5%4.8%0%	0.47	0 $\pm$ 3 (4)55.6%11.1%0%22.2%11.1%	0 $\pm$ 1 (4)64.6%16.7%10.4%4.2%4.2%	0.40
Nauseous 0 1 3	0 $\pm$ 0 (1)97.2%2.8%0%	0 $\pm$ 0 (3)81.0%14.3%4.8%	0.04	0 $\pm$ 0.5 (1)77.8%22.2%0%	0 $\pm$ 0 (3)93.8%4.2%2.1%	0.14
Driving scenario
Sweaty 0 1 2 4 9	0 $\pm$ 0 (9)89.2%2.7%2.7%2.7%2.7%	0 $\pm$ 0 (2)85.7%9.5%4.8%0%0%	0.78	0 $\pm$ 3 (9)55.6%11.1%11.1%11.1%11.1%	0 $\pm$ 0 (2)93.9%4.1%2.0%0%0%	$<$ 0.001
Queasy 0 1 2 3 4 5 7 8	0 $\pm$ 2 (8)67.6%5.4%8.1%5.4%5.4%2.7%2.7%2.7%	0 $\pm$ 1 (8)66.7%14.3%9.5%4.8%0%0%0%4.8%	0.83	4 $\pm$ 5 (8)11.1%11.1%22.2%11.1%11.1%11.1%11.1%22.2%	0 $\pm$ 0 (4)77.6%10.2%8.2%2.0%2.0%0%0%0%	$<$ 0.001

Table 3, continued
Time of assessment	Participants		$p$	Experienced symptoms of simulator sickness		$p$
	With MS	Without MS		Yes	No
Dizzy 0 1 2 3 4 5 7	0 $\pm$ 3 (7)56.8%10.8%2.7%13.5%5.4%5.4%5.4%	1 $\pm$ 2 (7)47.6%14.3%28.6%4.8%0%0%4.8%	0.96	5 $\pm$ 7 (7)33.3%0%0%11.1%0%22.2%33.3%	0 $\pm$ 2 (4)57.1%14.3%14.3%10.2%4.1%0%0%	0.009
Nauseous 0 1 2 3 4 5 6 10	0 $\pm$ 0 (5)86.5%2.7%5.4%2.7%0%2.7%0%0%	0 $\pm$ 1.5 (10)57.1%19.0%9.5%0%4.8%0%4.8%4.8%	0.02	2 $\pm$ 5.5 (10)33.3%11.1%22.2%0%0%11.1%11.1%11.1%	0 $\pm$ 0 (4)83.7%8.2%4.1%2.0%2.0%0%0%0%	$<$ 0.001

Note. MS $=$ Multiple Sclerosis. Continuous data are summarized with medians $\pm$ interquartile range (range). $p\leqslant$ 0.05, two-tailed.

Computing the percentage of participants who reported symptoms of simulator sickness on the Modified Motion Sickness Assessment Questionnaire (i.e., cut point $\geqslant$ 5) showed that seven participants with MS (19%) and two participants without MS (10%) reported symptoms of simulator sickness (i.e., 2% sweatiness, 7% queasiness, 9% dizziness, 5% nausea) either during (1 with MS before event 1, 1 without MS after event 2) or after completing the driving scenario (6 with MS, 1 without MS), which did not significantly differ between groups (Fisher’s exact $=$ 0.90, $p=$ 0.47). Table 3 presents participant responses to the Modified Motion Sickness Assessment Questionnaire throughout the driving simulator assessment. Participants without MS had higher responses to symptoms of nausea than those with MS for adaptation drives 2 and 3 and for the driving scenario. However, participants who experienced symptoms of simulator sickness (e.g., 7 with MS, 2 without MS) had higher responses to queasy and nausea for adaptation drive 2, and in all symptoms for the driving scenario.

For participants with MS, correlations between reported symptoms of simulator sickness and demographic or clinical characteristics indicated that females (vs. males, $r_{rb}=$ 0.4, $p=$ 0.03), greater fatigue (Fatigue Severity Scale, $r_{s}=$ 0.4, $p=$ 0.02), more years since last relapse ( $r_{s}=$ 0.3, $p=$ 0.05), and more medications ( $r_{s}=$ 0.4, $p=$ 0.03) correlated moderately with increased symptoms of dizziness.

3.4 Objective 4: Resources and ability to implement the study

This objective determined if the study had the resources to conduct the study successfully as per the protocol. We covered all fees (e.g., participant compensation, $1,475 CAD) through undesignated funds, and cost-share of faculty time. The research team already had all the screening and assessment administration manuals, assessment forms, software, and equipment needed to conduct the study. Without these resources, the study would not have been possible to conduct without securing funding.

Recruiting participants involved setting up collaborations with the London MS Clinic and the MS Society of Canada. Other recruitment methods involved using the University’s poster distribution services for eight months to post recruitment advertisements around campus for a fee of $170 CAD. On a monthly basis, we requested to post recruitment advertisements via online networks (i.e., Craigslist, Kijiji). Participants’ study visit took a median of 120 minutes to complete the entire study protocol (IQR $=$ 105–150, Range $=$ 90–180).

3.5 Objective 5: Clinical and driving simulator assessment results

Table 4 summarizes the driving simulator maneuvers in participants with and without MS. Participants with MS, compared to those without MS, made more adjustment to stimuli errors (tactical) via a significantly slower response time in the pedestrian event (i.e., event 3) and more collided with the vehicle that cut across the lane (i.e., event 4). Experiencing a collision (yes/no) in event 4 did not significantly impact whether a correct/incorrect decision was made in the navigational driving task (i.e., strategic driving maneuver; Fisher’s exact $=$ 0.005, $p=$ 1.0).

Table 4
Driving simulator maneuvers ( $N=$ 59)

Driving simulator maneuvers	Participants		Between-group difference test
	With MS ( $n=$ 38)	Without MS ( $n=$ 21)
Event 1: Car Pulls Out in Front	$n=$ 25	$n=$ 17
Reaction time	1.4 $\pm$ 0.5 (1.6)	1.3 $\pm$ 0.4 (2.2)	$U=$ 208.5, $p=$ 0.92
Response time	3.8 $\pm$ 0.8 (3.5)	3.9 $\pm$ 0.9 (2.2)	$U=$ 187.5, $p=$ 0.53
Mean speed	8.4 $\pm$ 1.3 (7.1)	8.7 $\pm$ 2.4 (4.6)	$U=$ 190.0, $p=$ 0.58
Event 2: Traffic Light Changes Colour	$n=$ 36	$n=$ 21
Reaction time	0.7 $\pm$ 1.3 (1.8)	0.7 $\pm$ 1.3 (1.6)	$U=$ 342.5, $p=$ 0.56
Response time	2.0 $\pm$ 0.5 (1.4)	2.4 $\pm$ 0.7 (1.4)	$U=$ 289.5, $p=$ 0.15
Mean speed	11.9 $\pm$ 4.6 (9.0)	9.8 $\pm$ 5.0 (11.3)	$U=$ 312.0, $p=$ 0.28
Response type	$n=$ 36	$n=$ 21	$\chi^{2}$ ( $df=$ 1) $=$ 3.6, $p=$ 0.10
Stop	13 (36%)	13 (62%)
Failed to stop	23 (64%)	8 (38%)
Event 3: Pedestrian Walks Out in Front	$n=$ 36	$n=$ 20
Reaction time	1.2 $\pm$ 0.7 (2.6)	1.4 $\pm$ 0.6 (2.4)	$U=$ 342.0, $p=$ 0.76
Response time	3.8 $\pm$ 0.7 (3.3)	3.6 $\pm$ 0.8 (1.5)	$U=$ 220.0, $p=$ 0.02
Mean speed	6.5 $\pm$ 1.7 (12.4)	6.4 $\pm$ 1.8 (9.8)	$U=$ 322.0, $p=$ 0.53
Response type			Fisher’s $=$ 0.5, $p=$ 0.70
Stop	30 (83%)	18 (90%)
Failed to stop	6 (17%)	2 (10%)
Event 4: Vehicle Cuts in Front	$n=$ 34	$n=$ 20
Reaction time	0.9 $\pm$ 0.4 (1.8)	0.8 $\pm$ 0.3 (2.3)	$U=$ 296.0, $p=$ 0.44
Response time	2.4 $\pm$ 1.8 (5.0)	2.5 $\pm$ 2.8 (5.6)	$U=$ 328.0, $p=$ 0.84
Mean speed	7.5 $\pm$ 2.9 (11.0)	6.7 $\pm$ 6.1 (11.6)	$U=$ 318.0, $p=$ 0.70
Response type			Cramer’s V $=$ 0.3, $p=$ 0.04
Stop	19 (56%)	14 (70%)
Failed to stop	2 (6%)	4 (20%)
Crashed	13 (38%)	2 (10%)
Navigational Driving Task	$n=$ 35	$n=$ 20	Fisher’s $=$ 0.0, $p=$ 1.00
Correct decision	28 (80%)	16 (80%)
Incorrect decision	7 (20%)	4 (20%)

Note. MS $=$ Multiple Sclerosis. Continuous data are summarized with medians $\pm$ interquartile range (range); categorical data are summarized with frequencies (percentages). Reaction time and response time are measured in seconds; Mean speed is measured in meters per second. Reasons for missing data are discussed under 3.2 Objective 2: Data collection procedures and outcome measures. The table includes findings that are documented in studies that examined the predictors of driving simulator performance in drivers with MS [20, 21]. $p\leqslant$ 0.05, two-tailed.

For participants with MS, correlations between clinical test scores and driving simulator maneuvers showed that a slower divided attention correlated with slower response time ( $r_{s}=$ 0.4, $p=$ 0.03), and deficits in immediate verbal/auditory recall ( $r_{s}=$ 0.4, $p=$ 0.02) and slower selective attention ( $r_{s}=-$ 0.4, $p=$ 0.03) correlated with slower mean speed in the traffic light event (operational maneuver). Deficits in immediate verbal/auditory recall correlated with slower reaction time in the pedestrian event (tactical maneuver, $r_{s}=-$ 0.4, $p=$ 0.02). Furthermore, slower selective attention correlated with incorrect decisions in the navigational driving task (strategic maneuver, $r_{rb}=$ 0.4, $p=$ 0.03).

4. Discussion

This study examined the feasibility of utilizing visual-cognitive and driving simulator assessments to indicate driving performance in drivers with and without MS.

4.1 Objective 1: Recruitment capability and resulting sample characteristics

As we reached 95% of our proposed recruitment goal for participants with MS, meeting the proposed sample size ( $N=$ 40) would require interest from at least two to three times as many individuals and would take 13 to 20 months to complete. Participants with MS showed some differences in clinical characteristics when compared to the population-based sample of those with MS. For example, 92% of participants had relapsing MS (vs. 85% of MS population), while 8% had progressive MS (vs. 15% of MS population) [41]. These differences may be due to recruiting participants with low to moderate physical disability, as those with progressive MS tend to have higher levels of physical disability [42]. Consistent with the literature, we recruited most participants with MS through an MS Clinic that treated or assessed patients, and most demographic and clinical characteristics did not significantly differ between participants with and without MS [4, 8, 10, 14, 18]. These non-significant differences might have been mitigated if we recruited more participants through the community in addition to institutions where patients are assessed or treated. As such, we suggest continuing to use a variety of recruitment strategies, in addition to recruiting through institutions that assess or treat individuals with MS.

4.2 Objective 2: Data collection procedures and outcome measures

The missing data during the driving simulator scenario might have been mitigated via pilot-testing the drive multiple times prior to data collection. Furthermore, collaborating with simulator industry partners about the process of triggering hazardous events to start might have minimized the number of bypassed events. However, such consultations are costly, especially given that the study had no external funding to offset the cost of specialty consultations. Nevertheless, we suggest four strategies to minimize missing data on a driving simulator: 1) pilot test the drives and driving performance measures to ensure proper functioning and data collection of the simulator; 2) ensure that computer programming occurs as to “hit” the landmark that cues hazardous events to start; 3) video record the main drives to supplement failure of the driving simulator to record such data; and 4) include additional practice drives with turns so that participants become more accustomed to turning with the restricted field of view on this simulator (e.g., half the screen, 55 out of the 110-degree horizontal field of view) as compared to real-world driving (e.g., 180-degree horizontal field of view and 135-degrees vertical field of view when stationary) [43].

4.3 Objective 3: Acceptability and suitability of the driving simulator

Participants’ acceptability toward the driving simulator suggested that those with and without MS perceived the driving simulator as easy to use, but they would not intend to use a driving simulator in their daily lives. In the literature, one study by Lamargue-Hamel et al. reported that participants with MS rated a driving simulator assessment as pleasant (0–5, very unpleasant to very pleasant; mean score $=$ 4.1, SD $=$ 0.9) [14, 17]. Based on the prior and our study’s findings, whether drivers with MS perceive the driving simulator as an adequate and suitable technology for measuring driving performance, and whether they are willing to undergo a driving simulator assessment, is not fully understood [14, 17]. Considering issues that may impact participants’ responses may help understand acceptability toward using a driving simulator to assess driving performance.

Consistent with findings in prior driving simulator studies [14, 17, 44], participants’ suitability toward the driving simulator showed that some participants with MS experienced symptoms of simulator sickness. For example, in Lamargue-Hamel et al., five of 30 participants with MS (17%) experienced motion sickness during a driving simulator assessment [17]. In Akinwuntan et al. six of 42 participants with relapsing MS (14%) discontinued a simulated training program in the first simulator familiarization session and continued as control participants due to experiencing simulator sickness [44]. Overall, the prior and our study’s findings indicate that some drivers with MS may experience symptoms of simulator sickness that can affect completing a driving scenario during an assessment or intervention [14]. However, since these studies’ findings derive from one session on a driving simulator, more research is needed to determine the number of repeated sessions (if any) and period of expositions that are most suitable for drivers with MS.

Profiling participants who experienced and did not experience symptoms of simulator sickness showed that turns may indicate symptoms of simulator sickness (e.g., adaptation drives 2–3). While dizziness did not significantly differ between groups before the driving scenario, ratings increased after adaptation drive 2. These preliminary findings suggest that increasing dizziness and turns may contribute to experiencing increasing symptoms of simulator sickness (e.g., sweaty, queasy, dizzy, nausea).

The demographic and clinical characteristics that were correlated with increased dizziness while driving the simulator correspond with at least one of the underlying factors that contribute to the occurrence of simulator sickness (i.e., female sex) [45]. One reason for why individuals with MS may be more susceptible to experiencing simulator sickness is that MS is more prevalent in women than men, and some may experience vestibular instability [14, 46]. However, the occurrence of simulator sickness is under-reported in driving simulator studies for drivers with MS. Understanding the physiological mechanisms will be a plausible next step to develop mitigation protocols to reduce simulator sickness during driving assessment or intervention.

4.4 Objective 4: Resources and ability to implement the study

Though the available resources enabled us to conduct the study, some were not optimal for detecting the underlying driving performance impairments in drivers with MS. For instance, the navigational driving task in the driving scenario did not provide an adequate assessment of participants’ strategic driving maneuvers, which depend on high-level reasoning, planning, judging, or problem-solving. Optimizing the navigational driving task would have required an added expense, and as such, we chose to use an existing driving scenario with a strategic maneuver component – but not to the extent required to make a targeted assessment.

4.5 Objective 5: Clinical and driving simulator assessment results

Preliminary results showed that participants with MS (vs. without MS) made more adjustment to stimuli errors (tactical) via taking longer to respond to the pedestrian who walked in front of them, or by not responding in enough time to the vehicle that cut across the lane in front of them and experiencing a collision. These findings suggest that tactical maneuvers may target driving performance deficits in drivers with MS which is consistent with prior on-road studies [12, 13], and builds on prior driving simulator studies which mostly show that adjustment to stimuli errors (operational) target driving performance [14, 15, 16, 17, 18].

As deficits in immediate verbal/auditory recall, slower divided attention, and slower selective attention correlated with deficits in adjustment to stimuli (i.e., operational, tactical) or strategic maneuvers, the CVLT2-IR, UFOV2, and/or UFOV3 may be useful for identifying drivers with these driving performance issues. Accordingly, we considered immediate verbal/auditory recall and divided/selective attention as predictors of driving performance in a follow up study [21].

4.6 Limitations

Study findings may only be generalized to individuals who meet the sample’s characteristics for individuals with and without MS (e.g., selection bias, spectrum bias) [20, 21]. While screening criteria ensured that severe fatigue would not be a confounding factor of driving performance in drivers with MS, we did not assess for whether fatigue severity changed throughout the study visit; though participants took breaks whenever they felt necessary. Although driving simulators can be used to assess operational and tactical driving maneuvers, the pre-existing driving scenario [36] used in this study did not allow us to perform an adequate or comprehensive assessment of gap acceptance or strategic driving maneuvers for drivers with MS. In this study, more drivers with MS than without MS experienced a collision on the driving simulator. Consistent with prior studies [14, 16, 18], these findings show that drivers with MS (and without MS) can experience collisions during driving simulator scenarios. However, since the absolute validity of driving simulators cannot yet be inferred, the occurrence of simulated collisions cannot be directly related to on-road collisions. The missing data during the driving simulator assessment may have underestimated actual results (e.g., type II error). However, due to the exploratory nature of this study, we did not correct for multiple comparisons (e.g., type I error).

4.7 Strengths

The research team consisted of multidisciplinary professionals with expertise in driver rehabilitation science, biostatistics, MS, neurology, occupational therapy, driving simulation, and transportation engineering. Though we knew participants’ diagnoses and clinical test scores, the kinematic data obtained on the driving simulator ensured the collection of objective data. Furthermore, video-recording the driving scenario and documenting driving outcomes on the standardized assessment form [36] enabled us to cross-reference metrics obtained by the driving simulator and those observed from the drive.

4.8 Implications

4.8.1 Future research

As the acceptability toward a driving simulator is not fully understood in drivers with MS [14], research that focuses on acceptability may help identify factors that influence participants’ perceptions on the usability, usefulness, practicality, and representativeness of driving simulators measuring driving performance. In addition to adjustment to stimuli maneuvers, gap acceptance and strategic driving maneuvers have been shown to predict failing an on-road assessment in drivers with MS [9, 12, 13]. However, gap acceptance maneuvers were not assessed in this study and the navigational driving task may not have adequately challenged the high-level reasoning, planning, judging, or problem-solving of drivers with MS. Developing and validating scenarios that adequately and comprehensively assess gap acceptance and strategic driving maneuvers for drivers with MS may provide insight to demographic, clinical, and driving characteristics that impact driving performance.

4.8.2 Policymakers

Canada’s fitness to drive standards state that occupational therapists or other specialists with expertise in driver rehabilitation can use driving simulators to assess whether drivers can compensate for motor deficits [43]. Based on prior [14, 15, 16, 17, 18, 21] and our study’s findings, visual-cognitive impairments may be indicators of driving performance deficits. Policymakers may consider implementing the use of driving simulators to identify, assess, or monitor visual-cognitive deficits to help inform decisions about one’s fitness to drive.

4.8.3 Practitioners

Tracking participants’ acceptability of a driving simulator over time may help identify factors that influence their perceptions on whether the driving simulator is a representative, useful, and practical measure of driving performance. Likewise, tracking the occurrence of simulator sickness over time may help identify factors that impact simulator sickness. Clinical tests that measure immediate verbal/auditory recall and divided attention may be useful for screening driving performance deficits in drivers with MS. Assessing operational, tactical, and strategic driving maneuvers in all driving environments (i.e., residential, suburban, urban, and highway) may provide an overall picture of a person’s driving performance.

5. Conclusion

This study indicated the feasibility of utilizing clinical and driving simulator assessments to indicate driving performance deficits in drivers with and without MS. Overall, the lower than proposed recruitment rates highlighted the importance of ensuring feasible planning, realistic timelines, and using a variety of recruitment methods to reach recruitment goals and enhance the generalizability of study findings. The missing data on the driving simulator emphasized the importance of understanding its data collection procedures and outcome measures. Participants’ acceptability toward the driving simulator indicated that participants perceived the simulator as a useful driving performance assessment, but they would not use one in their daily lives. Participants’ suitability toward the driving simulator showed that some drivers experienced symptoms of simulator sickness that affected their ability to complete the assessment. Lastly, clinical and driving assessment results showed that deficits in immediate verbal/auditory recall and divided/selective attention may underlie driving performance deficits on a driving simulator. Clinical tests that predict driving performance deficits may be useful for informing driving performance decisions in drivers with MS.

Author contributions

CONCEPTION: Sherrilene Classen, Sarah A. Morrow and Sarah Krasniuk.

PERFORMANCE OF WORK: Sarah Krasniuk.

INTERPRETATION OR ANALYSIS OF DATA: Sherrilene Classen, Sarah A. Morrow and Sarah Krasniuk.

PREPARATION OF THE MANUSCRIPT: Sarah Krasniuk.

REVISION FOR IMPORTANT INTELLECTUAL CONTENT: Sherrilene Classen, Sarah A. Morrow and Sarah Krasniuk.

SUPERVISION: Sherrilene Classen and Sarah A. Morrow.

Ethical considerations

This study received ethics approval from Lawson’s Health Research Institute (R-18-631) and Western University’s Health Sciences Research Ethics Board (112525), and all procedures were performed in compliance with relevant laws and institutional guidelines. The study took place from January 2019 to February 2020. Participants consented in writing prior to taking part and received a $25 CAD gift card.

Footnotes

Acknowledgments

Western University’s i-Mobile Driving Research Lab, the London (Ontario) Multiple Sclerosis Clinic, and the University of Florida’s Institute for Mobility, Activity, and Participation Lab provided infrastructure and support to conduct this research study and to disseminate findings. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

The authors have no conflicts of interest to report.

References

Krasniuk

Classen

Morrow

Tippett

Knott

Akinwuntan

. Clinical determinants of fitness to drive in persons with multiple sclerosis: Systematic review. Arch Phys Med Rehabil. 2019; 100(8): 1534-55. doi: 10.1016/j.apmr.2018.12.029.

Classen

Dickerson

Justiss

. Occupational therapy driving evaluation: Using evidence-based screening and assessment tools. In: McGuire MJ, Davis ES, eds. Driving and community mobility: Occupational therapy across the lifespan. Bethesda, MD: American Occupational Therapy Association Press; 2012. pp. 221-77.

Campos

Bedard

Classen

Delparte

Hebert

Hyde

, et al. Guiding framework for driver assessment using driving simulators. Front Psychol. 2017; 8(1428): 1-4. doi: 10.3389/fpsyg.2017.01428.

Akinwuntan

Backus

Grayson

Devos

. Validation of a short cognitive battery to screen for fitness-to-drive of people with multiple sclerosis. Eur J Neurol. 2018; 25(10): 1250-4. doi: 10.1111/ene.13690.

Akinwuntan

Devos

Stepleman

Casillas

Rahn

Smith

, et al. Predictors of driving in individuals with relapsing-remitting multiple sclerosis. Mult Scler. 2012; 19(3): 344-50. doi: 10.1177/1352458512451944.

Akinwuntan

O’Connor

McGonegal

Turchi

Smith

Williams

, et al. Prediction of driving ability in people with relapsing-remitting multiple sclerosis using the stroke driver screening assessment. Int J MS Care. 2012; 14(2): 65-70. doi: 10.7224/1537-2073-14.2.65.

Devos

Ranchet

Backus

Abisamra

Anschutz

Allison

, et al. Determinants of on-road driving in multiple sclerosis. Arch Phys Med Rehabil. 2017; 98(7): 1332-8. doi: 10.1016/j.apmr.2016.10.008.

Classen

Krasniuk

Morrow

Alvarez

Monahan

Danter

, et al. Visual correlates of fitness to drive in adults with multiple sclerosis. OTJR (Thorofare N J). 2018; 38(1): 15-27. doi: 10.1177/1539449217718841.

Krasniuk

Classen

Monahan

Danter

Rosehart

, et al. A strategic driving maneuver that predicts on-road outcomes in adults with multiple sclerosis. Transp Res F: Traffic Psychol Behav. 2019; 60: 147-56. doi: 10.1016/.j.trf.2018.10.014.

10.

Morrow

Classen

Monahan

Danter

Taylor

Krasniuk

, et al. On-road assessment of fitness-to-drive in persons with MS with cognitive impairment: A prospective study. Mult Scler. 2018; 24(11): 1499-506. doi: 10.1177/.1352458517723991.

11.

Lincoln

Radford

. Cognitive abilities as predictors of safety to drive in people with multiple sclerosis. Mult Scler. 2008; 14(1): 123-8. doi: 10.1177/1352458507080467.

12.

Krasniuk

Classen

Morrow

. Driving environments that influence on-road outcomes in persons with multiple sclerosis. Transp Res F: Traffic Psychol Behav. 2020; 70: 191-8. doi: 10.1016/j.trf.2020.03.003.

13.

Krasniuk

Classen

Morrow

Monahan

Danter

Rosehart

, et al. Driving errors that predict on-road outcomes in adults with multiple sclerosis. OTJR (Thorofare N J). 2017; 37(3): 115-24. doi: 10.1177/1539449217708554.

14.

Krasniuk

Knott

Bagajati

Azizderouei

Sultania

Alvarez

. Utilizing driving simulators for persons with multiple sclerosis: A scoping review. Transp Res F: Traffic Psychol Behav. 2022; 85: 103-118. doi: 10.1016/j.trf.2021.12.006.

15.

Devos

Brijs

Alders

Wets

Feys

. Driving performance in persons with mild to moderate symptoms of multiple sclerosis. Disabil Rehabil. 2013; 35(16): 1387-93. doi: 10.3109/09638288.2012.731468.

16.

Harand

Mondou

Chevanne

Bocca

Defer

. Evidence of attentional impairments using virtual driving simulation in multiple sclerosis. Mult Scler Relat Disord. 2018; 25: 251-7. doi: 10.1016/j.msard.2018.08.005.

17.

Lamargue-Hamel

Deloire

Saubusse

Ruet

Taillard

Philip

, et al. Cognitive evaluation by tasks in a virtual reality environment in multiple sclerosis. J Neurol Sci. 2015; 359(1–2): 94-9. doi: 10.1016/j.jns.2015.10.039.

18.

Kotterba

Orth

Eren

Fangerau

Sindern

. Assessment of driving performance in patients with relapsing-remitting multiple sclerosis by a driving simulator. Eur J Neurol. 2003; 50(3): 160-4. doi: 10.1159/000073057.

19.

Orsmond

Cohn

. The distinctive features of a feasibility study: Objectives and guiding questions. OTJR (Thorofare N J). 2015; 35(3): 169-77. doi: 10.1177/1539449215578649.

20.

Krasniuk

Classen

Morrow

. Driving errors that predict simulated rear-end collisions in drivers with multiple sclerosis. Traffic Inj Prev. 2021; 22(3): 212-7. doi: 10.1080/.15389588.2021.1883008.

21.

Krasniuk

Classen

Morrow

Alvarez

Srinivasan

, et al. Clinical predictors of driving simulator performance in drivers with multiple sclerosis. Mult Scler. 2021; 27(13): 2085-2092. doi: 10.1177/1352458521992507.

22.

Ministry of Transportation of Ontario. Medical Review for Ontario Drivers. 2018. [cited 2021 Nov 24]. Available from: http//www.mto.gov.on.ca/english/dandv/driver/medical-review/standards.shtml.

23.

Lublin

Reingold

Cohen

Cutter

Sorensen

Thompson

, et al. Defining the clinical course of multiple sclerosis: The 2013 revisions. Neurology. 2014; 83(3): 278-86. doi: 10.1212/WNL.0000000000000560.

24.

Kurtzke

. Rating neurologic impairment in multiple sclerosis: An expanded disability status scale (EDSS). Neurology. 1983; 33(11): 1444-52. doi: 10.1212/wnl.33.11.1444.

25.

Krupp

Larocca

Muir Nash

Steinberg

. The Fatigue Severity Scale: Application to patients with multiple sclerosis and systemic lupus erythematosus. Arch Neurol. 1989; 46(10): 1121-3. doi: 10.1001/archneur.1989.00520460115022.

26.

Beck

Steer

Brown

. BDI

{}^{\@setsize{\scriptsize}{8pt}{\viipt}{\@viipt}\textregistered}

-Fast Screen for Medical Patients (BDI

{}^{\@setsize{\scriptsize}{8pt}{\viipt}{\@viipt}\textregistered}

). San Antonio (TX): Psychological Corporation; 2000.

27.

Classen

Horgas

Awadzi

Shechtman

Joo

. Clinical predictors of older driver performance on a standardized road test. Traffic Inj Prev. 2008; 9(5): 456-62. doi: 10.1080/.15389580802260026.

28.

Cordazzo

STD

Scialfa

Bubric

Ross

. The Driver Behaviour Questionnaire: A North American analysis. J Safety Res. 2014; 50: 99-107. doi: 10.1016/j.jsr.2014.05.002.

29.

Stereo Optical Inc. Stereo Optical Company Vision Tester Slide Package: Driver Rehabilitation Slide Package. Chicago (IL): Stereo Optical Inc.; 2018 [cited 2021 Nov 24]. Available from: https//www.stereooptical.com/wp-content/uploads/2018/01/Driver-Rehab-Manual-56219.pdf.

30.

Brain

. UFOV Assesment Manual. [cited 2021 Nov 24]. Available from: https//www.visualawareness.com/wp-content/uploads/2019/04/UFOV-Assessment-Manual.pdf.

31.

Rao

. A manual for the brief, repeatable battery of neuropsychological tests in multiple sclerosis. New York, United States: National Multiple Sclerosis Society; 1991.

32.

Delis

Kramer

Kaplan

KJH

Ober

. California Verbal Learning Test – Second Edition. Toronto, Canada: Pearson Canada Assessment, Inc.; 2000.

33.

Benedict

RHB

. Brief Visuospatial Memory Test-Revised: Professional Manual. Odessa, Florida: Psychological Assessment Resources; 1997.

34.

Brooks

Goodenough

Crisler

Klein

Alley

Koon

, et al. Simulator sickness during driving simulation studies. Accid Anal Prev. 2010; 42(3): 788-96. doi: 10.1016/j.aap.2009.04.013.

35.

Stern

Akinwuntan

Hirsch

. Simulator sickness: strategies for mitigation and prevention. In: Classen S, ed. Driving Simulation for Assessment, Intervention, and Training A guide for occupational therapy and health care professionals. Bethesda, MD: American Occupational Therapy Association Press; 2017. pp. 107-20.

36.

Alvarez

Classen

Medhizadah

Knott

Asantey

, et al. Feasibility of DriveFocusTM and driving simulation interventions in young drivers. OTJR (Thorofare N J). 2018; 38(4): 245-53. doi: 10.1177/1539449218787495.

37.

Michon

. A critical view of driver behavior models: What do we know, what should we do? In: Evans L, Schwing RC, eds. Human Behavior and Traffic Safety. Boston, MA: Springer; 1985. pp. 485-520. doi: 10.1007/978-1-4613-2173-6_19.

38.

Davis

. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989; 13(3): 319-40. doi: 10.2307/249008.

39.

Brooke

Jordan

. SUS-A quick and dirty usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland AL, eds. Usability Evaluation in Industry. London, UK: Taylor and Francis; 1996. pp. 189-194.

40.

Jackson

. Research Methods and Statistics: A Critical Thinking Approach. 3 ed. Belmont, CA: Wadsworth Cengage Learning; 2009.

41.

Public Health Agency of Canada. Multiple Sclerosis in Canada 2018 [cited 2021 Feb 5] Available from: https://wwwcanada.ca/en/public-health/services/publications/diseases-conditions/multiple-sclerosis-infographic.html.

42.

Doshi

Chataway

. Multiple sclerosis, a treatable disease. Clin Med (Lond). 2017; 16(Suppl 6): s54-s59. doi: 10.7861/clinmedicine.16-6-s53.

43.

Canadian Council of Motor Transport Administrators. National Safety Code Standard 6: Determining Driver Fitness in Canada Part 1: A Model for the Administration of Driver Fitness Programs Part 2: CCMTA Medical Standards for Drivers. 2021. [cited 2021 Nov 26]. Available from: https//ccmta.ca/en/national-safety-code.

44.

Akinwuntan

Devos

Baker

Phillips

Kumar

Smith

, et al. Improvement of driving skills in persons with relapsing-remitting multiple sclerosis: A pilot study. Arch Phys Med Rehabil. 2014; 95(3): 531-7. doi: 10.1016/j.apmr.2013.08.294.

45.

Classen

Bewernitz

Shechtman

. Driving simulator sickness: An evidence-based review of the literature. Am J Occup Ther. 2011; 65(2): 179-88. doi: 10.5014/ajot.2011.000802.

46.

Akinwuntan

Chong

Daniel

Fleming

Lokey

. Simulator adaptation syndrome (SAS) and the vestibular system: An exploratory study. Adv Transp Stud. 2014; 2(Special Issue): 29-38. doi: 10.4399/97888548735374.

Feasibility of utilizing clinical and driving simulator assessments to indicate driving performance deficits in adults with multiple sclerosis

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSIONS:

Keywords

1. Introduction

2. Methods

2.1 Participants

2.2 Study protocol

2.2.1 Clinical assessment

2.2.2 Driving simulator assessment

2.2.2.1 Driving outcomes

2.2.2.2 Video recordings

2.4 Data analysis

Table 1 Sample characteristics ( N = 59)

3.1 Objective 1: Recruitment capability and resulting sample characteristics

Table 2 Participant Responses on the Perceived Usefulness and Ease of Use Questionnaire ( N = 54) and the System Usability Scale ( N = 53)

3.3 Objective 3: Acceptability and suitability of the driving simulator

Table 3 Participant Responses on the Modified Motion Sickness Assessment Questionnaire

3.5 Objective 5: Clinical and driving simulator assessment results

Table 4 Driving simulator maneuvers ( N = 59)

4.1 Objective 1: Recruitment capability and resulting sample characteristics

4.2 Objective 2: Data collection procedures and outcome measures

4.3 Objective 3: Acceptability and suitability of the driving simulator

4.4 Objective 4: Resources and ability to implement the study

4.5 Objective 5: Clinical and driving simulator assessment results

4.6 Limitations

4.7 Strengths

4.8 Implications

4.8.1 Future research

4.8.2 Policymakers

4.8.3 Practitioners

5. Conclusion

Author contributions

Ethical considerations

Footnotes

Acknowledgments

Conflict of interest

References

Table 1
Sample characteristics ( $N=$ 59)

Table 2
Participant Responses on the Perceived Usefulness and Ease of Use Questionnaire ( $N=$ 54) and the System Usability Scale ( $N=$ 53)

Table 3
Participant Responses on the Modified Motion Sickness Assessment Questionnaire

Table 4
Driving simulator maneuvers ( $N=$ 59)