Decisions on driving after brain injury/disease: Feasibility and construct validity of a new simulator assessment tool

Abstract

Introduction

Driving is a complex activity involving a high level of cognitive abilities and thus might be affected after a brain injury/disease. The aim of this research was to evaluate the feasibility and construct validity of a driving simulator tool as a complement to existing driving assessments of patients with cognitive dysfunctions after a brain injury/disease.

Method

A descriptive and prospective research design was achieved. For construct validation, decisions were based on results from the Useful Field of View, Nordic Stroke Driver Screening Assessment, Trail Making Test and, when necessary for the decision, an on-road observation. Results from the simulator tool were not included in the clinical decision process.

Results

A total of 129 patients from four different rehabilitation services were included. The results showed a significant difference in test results between those who were considered medically fit versus unfit to drive. A factor analysis revealed four components, all including attention in combination with processing speed, visuospatial function, simultaneous capacity and executive function; these are all represented in the simulator tool. A correlation analysis showed that simulator subtest 3 (response/divergent response to stimuli) had the strongest correlation with most of the other tests included.

Conclusions

The simulator was found to be feasible and valid and found to include components other than those measured in the other tests.

Keywords

Cognition driving assessment evaluation occupational therapy rehabilitation

Introduction

Driving is essential for most adults to manage work and other daily activities. In addition, the ability to drive can be a crucial building block for self-esteem, wellbeing and social contact (Adler and Rottunda, 2006). However, driving requires the interaction of multiple higher-level cognitive skills that might be affected by medical conditions such as an acquired brain injury/disease. Impairment of cognitive abilities of importance to driving, such as attention, information processing, memory and executive function, may lead to increased safety risk in traffic (Roca et al., 2013). Based on the psychosocial value of continued driving, one of the most challenging decisions facing rehabilitation services is to decide whether an individual who has suffered an injury or disease is able to continue driving or not. Because driving is such an intrinsically complex task, combining well-learned routines with a requirement for the driver to respond flexibly and safely to unpredictable events, making decisions on continued driving based on different assessment tools is often regarded as a major challenge by health care professionals.

There are several models describing prerequisites for driving. One of the most commonly cited and used is that of Michon (1985). Michon describes driving as a hierarchical structure with three concurrent and interdependent levels of decision-making and performance. First, the strategic level is considered to be the most important component; this includes strategic planning and decision-making about the route, time of day, traffic complexity and risks. Decisions on this level are usually made before the actual driving and are usually not included in a driving assessment about continued driving after injury or disease. Second, the tactical level involves specific decisions and tasks associated with mastering the vehicle according to information from ongoing traffic situations, referred to as behaviour. Finally, the operational level consists of direct vehicle actions for which most behaviour is automatic and consists of direct decisions on immediate control actions responding to the changing conditions. Decisions made at each of these three control levels require different information and occur in different timeframes (Shinar and Oppenheim, 2011). Strategic decisions are made before driving commences, whereas tactical and operational decisions require information about the immediate driving situation and are thus made in real time. The interaction of several cognitive abilities is required, especially at the tactical level, for which the most important factors for predicting motor vehicle collisions have been found to be executive function; focused, sustained, divided and selective attention; visuospatial function; decision-making; processing speed and memory (Field and Unsworth, 2017; Hird et al., 2016; Lafont et al., 2008; Seong-Youl et al., 2014). The planning or strategic level is the most difficult level to assess in a standardised way, because it concerns mainly evaluation of the risks involved and avoiding risks before the driver sits behind the wheel. In addition, information processing models have been used to describe the interaction between different stages, including perception, decision and response selection, and execution. Each of those stages takes time due to the assumed transformation of information (Shinar and Oppenheim, 2011). An on-road assessment is considered to be the only method to assess all three levels simultaneously. However, because on-road assessment lacks standardisation, is time consuming, expensive and stressful for the patient, and cannot identify which cognitive areas may be impaired, driving assessment of people who may have cognitive impairments is primarily based on various cognitive tests (Korner-Bitensky et al., 2005).

Occupational therapists within rehabilitation services are often involved in cognitive screening to support decisions made by a physician on continued driving or not. In order to get as much information as possible about the cognitive skills and behaviour important for driving, information from several cognitive test results as well as observed behaviour in simulated or on-road driving are recommended (Brouwer et al., 2011; Dragos and Rothkrantz, 2017; Samuelsson et al., 2018).

The aim of this study was to evaluate the feasibility and construct validity of a driving simulator tool as a complement to existing driving assessment in patients with cognitive dysfunctions after a brain injury/disease.

Method

This was a multicentre, descriptive and prospective study involving four rehabilitation services in four different counties in Sweden, all specialising in driving assessment. Patient inclusion criteria were ≥18 years of age, referred for driving assessment due to suspected cognitive dysfunction and having a valid driving licence. All participants who fulfilled the inclusion criteria and who were referred to any of the participating services from September 2016 to December 2017 were invited to participate. No exclusion criteria were defined. A study-specific protocol was developed and used by all participating services to ensure that all data were collected and documented in the same order (descriptive information, Useful Field of View (UFOV), Trail Making Test A and B (TMT-A and TMT-B), Nordic Stroke Driver Screening Assessment (NorSDSA), simulator test and, for some, an on-road observation) and in the same way. The protocol included descriptive information on diagnosis, age, gender, educational level (≤ 9 years, 10–12 years, >12 years), estimated driving/year in Swedish miles (<1000, 1000–2000, >2000), and self-estimated driving skills (worse than others, same as others, better than others). Before starting the data collection, all participating personnel (physicians and occupational therapists from the different services) took part in a workshop where the study process was presented, the cognitive tests were reviewed and discussed (all tests were well known to the personnel), and the therapists learned to handle the new simulator equipment (CyberSim). All tests related to one particular patient were performed by the same occupational therapist on one occasion, except for the on-road observation, which was sometimes done on a separate occasion. The simulator test results were not included in the final decision (which was also explained to the participants on the informed consent form and verbally at the assessment) because this new equipment was not yet validated.

As a basis for decisions on continued driving or not, the rehabilitation services routinely used the following tests: UFOV (Marshall et al., 2007; Visual Awareness Research Group, 2009), the TMT (Patel, 2014; Tombaugh, 2004) and the NorSDSA (Lundberg, 2003; Lundberg et al., 2003). The newly developed simulator tool CyberSim (Samuelsson et al., 2019) was used by all services. The methods used in this study, including the simulator tool but not the on-road observation, have been used in an earlier study that included healthy controls. For a detailed description of the different assessment tools, see Samuelsson et al. (2019).

Useful field of view

UFOV is a computer-based test recommended as a screening measure for fitness to drive (Marshall et al., 2007; Visual Awareness Research Group, 2009). UFOV consists of three subtests that assess the accuracy of visual processing under increasingly complex tasks. The participant must detect, identify and localise briefly presented targets and respond to them by pointing to the right spot on a touch screen or by using a mouse. The accuracy of each response is measured. UFOV provides one score, reported in milliseconds, for each of the three subtests as well as a total score. The three subtests measure different types of attention: processing speed, divided attention and selective attention.

Trail making test

TMT-A and TMT-B are tests of visual conceptual and visuomotor tracking. TMT has been found useful for predicting unsafe older drivers (Seong-Youl et al., 2014).

TMT-A includes 25 scattered numbers; the participant has to draw lines sequentially connecting the numbers as quickly as possible without lifting the pencil. TMT-B is more complex than TMT-A, because the participant must alternate between 13 numbers and 12 letters in two sequences. Results are expressed as the time taken to complete each subtest correctly (Tombaugh, 2004).

According to Tombaugh (2004), the TMT assesses visual search, scanning, speed of processing, mental flexibility and executive functions. TMT-B has been found to be useful in dementia screening as a means to assess driving concerns. Patel (2014) strongly argued that results from the TMT should be combined with other driving assessments and not be used as a standalone tool.

Nordic Stroke Driver Screening Assessment

The NorSDSA (Lundberg, 2003) is a paper-based test revised and further developed from the British Stroke Driver Screening Assessment (SDSA) (Nouri and Lincoln, 1992). The Nordic version of the SDSA has been adapted for right-hand traffic, and some road signs in the original English version have been replaced. The NorSDSA consists of six subtests, of which four are included in the analysis for a final score: the dot cancellation test, the direction test, the compass test and the road sign recognition test (Lundberg et al., 2003). For the final score, results for each patient are calculated according to classification functions derived from the original discriminant function including the four variables (Lundberg et al., 2003). There is a lack of studies on the validity and reliability of the NorSDSA.

Simulator test: CyberSim

CyberSim consists of specially developed software installed on a laptop attached to a steering wheel (Samuelsson et al., 2019). The steering wheel is used for steering and for responding to visual stimuli on the screen by pressing paddles located on the left- and right-hand side of the steering wheel. The participants are asked to ‘drive on the road’ while steering and reacting to stimuli shown on the left or right or both sides of the screen, both one at a time and simultaneously (Figure 1); the driving speed is constant (40 km/h). A short exercise before each subtest was used to inform each participant and check that they understood each subtest. He/she was told to steer on the right side of the road and steer around obstacles such as parked cars at the side of the road. The participants were also asked to respond each time a visual stimulus (a traffic sign in terms of an arrow pointing to the left or right) popped up on the screen, by pressing one of the two paddles on the steering wheel as quickly as possible (Figure 1). The CyberSim includes three subtests representing increasing cognitive challenges. The entire session takes about 30–40 minutes.

The first subtest includes a simple stimuli-response task where the participant is asked to press the right paddle every time an arrow appears on the screen while ‘driving’.

The second subtest shows one arrow pointing to the left or right on the left or right side of the screen. The person is asked to respond by pressing the corresponding (right or left) paddle depending on the direction of the arrow.

The last subtest shows two arrows at the same time, one on the left and one on the right side of the screen. When the arrows are pointing in opposite directions, the person is informed not to press any paddle (inhibited response); when both arrows points in the same direction, the person is expected to press the paddle representing the direction of the two arrows.

Figure 1.

CyberSim: the steering wheel with two paddles and the screen.

Reaction time (in milliseconds) and the number of missed and wrong responses are registered and reported in the protocol. An algorithm was developed to illustrate how well the person had managed to keep the car on the right side of the road (that is, how well the person managed to avoid driving on or over the road markings to the right or left where no obstacles were seen); this algorithm is presented as the wobbling factor.

On-road observation

When more information was necessary, an on-road observation was performed with specially trained occupational therapists working at any one of the rehabilitation services. The observation was performed together with a local, specially educated and experienced driving instructor. The participants drove on a standardised road trip, identified at each rehabilitation service, lasting for about 60 minutes. The road trip included different traffic situations and demands, and the occupational therapist documented risk situations and misbehaviour in relation to actions taken. The on-road observation was included in the final clinical decisions as a complement to the standard test kit. The qualitative information received from the on-road observation was not included in the study analysis.

Results from the overall decision (pass or fail) and results from the different cognitive tests constituted the bases for the analyses. The on-road observation was performed after the cognitive tests, when the professionals needed additional or confirmatory information for making a decision on continued driving.

Professionals’ feedback on CyberSim for feasibility

After completion of data collection, a second workshop took place with all professionals who had experience of using the CyberSim as a tool for making decisions on continued driving or not (eight occupational therapists and two physicians). This workshop aimed to gather information about experiences from the study in general and from using the CyberSim in particular. The group discussion (n = 10) was based on predefined question areas focusing on experience from the simulator tool in general, including patient opinions, experience and thoughts about feasibility and how to interpret and explain the results. All participants had extensive experience of driving assessment in patients with neurological disorders.

Statistics

Statistical analyses were conducted using the Statistical Package for the Social Sciences (SPSS) version 23.0. Descriptive statistics were used for the sociodemographic descriptions. Parametric and non-parametric statistical methods were used to analyse data based on the results from normal distribution analyses and the type of variables analysed.

Student’s t test was used to compare age in the pass versus fail groups. The Mann–Whitney U-test was used for all comparisons of results between groups. Due to the number of analyses, Bonferroni's adjustment was used (p < 0.0025). An additional analysis adjusting for age differences between groups was performed.

Construct validity was analysed using a factor analysis (principal component analysis) with a direct oblimin rotation to describe variability among observed correlated variables in terms of a potential lower number of unobserved variables called factors. Eigenvalues ≥1.0 and loadings ≥0.40 were considered. In addition, Spearman’s rank order correlation test was used. Interpretation of the correlation coefficient as described by Mukaka (2012) was used; thus, a correlation coefficient r_s > 0.5 and a p value <0.001 was considered moderate to acceptable and r_s > 0.7 was considered high.

Ethical considerations

No medical or other risks related to participation in the study were identified. All participants referred to one of the rehabilitation services during the defined period received information about the study and an invitation letter together with an informed consent form. The informed consent was signed by the participants. All data protocols were coded by the occupational therapists, and no participant could be identified by the researchers. All procedures were in accordance with the ethical standards of the Declaration of Helsinki.

Results

In all, 129 participants (95 men and 34 women) agreed by signing the informed consent form and were thus included. All participants were informed about the study aim and what participation in the study implied. The mean age was 61.3 ± 14 years (range 23–88 years). For further details on the participants see Table 1.

Table 1.

Patients’ demographic data (n = 129).

	n (%)
Gender
Men	95 (74)
Women	34 (26)
Diagnoses
Stroke	65 (50)
Mild cognitive impairment	18 (14)
Traumatic brain injury	15 (12)
Multiple sclerosis/Parkinson’s disease	10 (8)
Subarachnoid haemorrhage	5 (4)
Brain tumour	4 (3)
Other	12 (9)
Education
≤ 9 years	35 (27)
10–12 years	59 (46)
> 12 years	30 (23)
Missing	5 (4)
Estimated driving per year
<10,000 kilometres/year	42 (33)
10,000–20,000 kilometres/year	57 (44)
> 20,000 kilometres/year	28 (22)
Missing	2 (2)
Self-perceived driving skill
Worse than others	13 (10)
Same as others	95 (74)
Better than others	4 (3)
Missing	17 (13)

In addition to the cognitive tests and the simulator test, 67% of the patients (n = 86) underwent on-road observation due to a need to confirm and complete results from the cognitive tests. There was no difference in the test results based on gender, education, estimated driving/year, or self-perceived driving skills on any of the tests or the new simulator tool: UFOV 1–3/total, TMT-A, TMT-B, NorSDSA, CyberSim 1–3 (p > 0.05). Thus, these variables were not analysed further. Age had a correlation coefficient >r_s 0.5 for UFOV II, UFOV III and UFOV total (r_s, 0.57–0.61).

There were significant differences in all test results between those patients who were considered as pass versus fail for continued driving (after Bonferroni correction, p < 0.0025) except for CyberSim 1 misses and faults and CyberSim 2 faults, which could be ignored because few misses or faults were made (Table 2). Patients in the fail group were significantly older than those in the pass group (66 ± 15.4 versus 59 ± 12.6 years; p = 0.006). Thus, an additional analysis adjusting for the age difference between the groups was performed still showing significant differences for UFOV 2, UFOV 3, UFOV Total, TMT-A, TMT-B, NorSDSA, CyberSim all variables except CyberSim 1 (misses and faults) and CyberSim 2 (faults). Even if the analyses showed significant differences between groups, there were large standard deviations in every item, indicating overlapping results.

Table 2.

Results from the clinical cognitive tests and the simulated driving tool (CyberSim) categorised by pass or fail to continued driving (n = 129).

	Pass n = 85 mean (SD)	Fail n = 44 mean (SD)	P value
UFOV 1 processing speed	20.2 (27)	43.5 (51)	0.001
UFOV 2 divided attention	76.9 (100)	210.2 (161)	<0.001
UFOV 3 selective attention	174.1 (121)	282.9 (150)	<0.001
UFOV total	271.2 (221)	536.6 (333)	<0.001
TMT-A, sec.	47.6 (43)	62.5 (35)	<0.001
TMT-B, sec.	108 (60)	181 (80)	<0.001
NorSDSA total score	1.66 (1.54)	–0.406 (1.76)	<0.001
CyberSim 1 reaction time msec	700 (200)	1014 (822)	<0.001
CyberSim 1 wobbling	0.6 (0.3)	0.9 (0.6)	<0.001
CyberSim 1 misses	0.4 (0.7)	1.5 (3.0)	n.s.
CyberSim 1 faults	0.3 (1.0)	0.6 (1.2)	n.s.
CyberSim 2 reaction time msec	1034 (176)	1266 (282)	<0.001
CyberSim 2 wobbling	0.7 (0.3)	1.0 (0.8)	0.001
CyberSim 2 misses	0.1 (0.4)	1.3 (2.3)	<0.001
CyberSim 2 faults	2.1 (2.0)	3.2 (2.9)	n.s.
CyberSim 3 reaction time msec	1381 (294)	1698 (372)	<0.001
CyberSim 3 wobbling	0.7 (0.3)	1.0 (0.5)	<0.001
CyberSim 3 misses	0.2 (0.8)	0.9 (1.7)	<0.001
CyberSim 3 faults	1.1 (1.4)	2.5 (2.6)	0.001

The Mann–Whitney U-test was used for all comparisons. Accepted level of significance p < 0.0025.

Feasibility

The second workshop was arranged after study completion to get feedback on the feasibility of the new simulator tool. The group discussion (n = 10) was based on predefined question areas focusing on experience from the simulator tool in general, including patient opinions, experience and thoughts about feasibility, and how to interpret and explain the results. The occupational therapists considered the simulator tool to be simple and easy to learn and use, and to understand the results and how to interpret them. Subtest 3, including inhibition, was considered very useful, providing information that was impossible to observe in the other tests. The simulator protocol was considered easy to read and understand, and easy in terms of giving feedback on the results to the patient. The wobbling factor (the algorithm results) was considered hard to interpret but useful during the observation. All patients were able to perform the simulator task and no one experienced any simulator sickness.

For patients whose native language was not Swedish and/or with motor impairments, CyberSim was considered easy to understand and to use.

Construct validity

Construct validity defines how well a test measures up to its claims. In this study we are referring mainly to the tactical level in Michon’s model (1985).

Results from the factor analysis

The factor analysis resulted in a value of 0.807 for the Kaiser-Meyer-Olkin measure of sampling adequacy, and Bartlett’s test of sphericity was significant (<0.001). Different kinds of attention were present in all four components in addition to different cognitive abilities as identified by the authors based on the demands of each task. The first component (attention and processing speed) included tests based on responsiveness (UFOV and CyberSim) and/or performing a task within a defined timeframe (NorSDSA and TMT-A/-B). The second component, attention and visuospatial function, refers to the ability to keep the car within the road markings while driving in the CyberSim test. The third component (attention and simultaneous capacity) requires the participant to react to stimuli shown on different spots on the computer screen while driving in the CyberSim test. The fourth component (attention and executive function) includes inhibition in terms of not responding to certain stimuli (Table 3). The results from the factor analysis showed an explained variance of 73.3% (Table 4), including the four components with an eigenvalue ≥1.0 and loadings ≥0.4.

Table 3.

Factor analysis result with loadings.

Components
	Processing speed	Visuospatial function	Simultaneous capacity	Executive function
UFOV 1 processing speed	0.615
UFOV 2 divided attention	0.744
UFOV 3 selective attention	0.749
TMT-A, sec.	0.727
TMT-B, sec.	0.753
NorSDSA, total score	–0.695
CyberSim subtest 1	0.583
CyberSim 1 misses			0.585
CyberSim 1 wobbling		–0.728
CyberSim subtest 2	0.550
CyberSim 2 misses			0.416
CyberSim 2 wobbling		–0.653
CyberSim subtest 3	0.673
CyberSim 3 misses				0.555
CyberSim 3 wobbling		–0.695

UFOV: Useful Field of View; TMT: Trail Making Test; NorSDSA: Nordic Stroke Driver Screening Assessment.

Table 4.

Total variance explained according to the factor analysis (n = 129).

Component	Eigenvalue	PoV	Cumulative PoV
Processing speed	6.402	42.7	42.7
Visuospatial function	2.068	13.8	56.5
Simultaneous capacity	1.428	9.5	66.0
Executive function	1.096	7.3	73.3

PoV: percentage of variance.

Construct validity for CyberSim was supported because this was the only assessment tool represented in all components, and thus covered additional aspects not included in the traditional cognitive tests.

Results from the correlation analysis

The correlation analysis showed a moderate/acceptable coefficient, r_s, > 0.5 (Mukaka, 2012) for results on CyberSim 2 and 3 compared with the results for some of the other cognitive tests, indicating that the cognitive aspects measured are intercorrelated. For CyberSim 2, accepted correlations were identified for TMT-A and TMT-B. For CyberSim 3, an accepted correlation was identified for UFOV 2, UFOV Total, TMT-A, TMT-B and NorSDSA (Table 5). All accepted correlations (r_s > 0.5) were significant (p < 0.001).

Table 5.

Correlation analysis. Spearman’s rank correlation test was used for the analysis.

	UFOV I	UFOV 2	UFOV 3	UFOV total	TMT-A	TMT-B	NorSDSA	CyberSim 1	CyberSim 1 wobbling	CyberSim 2	CyberSim 2 wobbling	CyberSim 3	CyberSim 3 wobbling
UFOV 1 proc speed	1
UFOV 2 div attention	0.66	1
UFOV 3 sel attention	0.54	0.77	1
UFOV total	0.62	0.89	0.96	1
TMT-A sec	0.26	0.53	0.46	0.53	1
TMT-B sec	0.32	0.53	0.52	0.57	0.73	1
NorSDSA total	−0.43	−0.65	−0.70	−0.72	−0.53	−0.66	1
CyberSim 1 msec	0.27	0.38	0.34	0.38	0.39	0.48	–0.37	1
CyberSim1 wobbling	0.26	0.40	0.31	0.36	0.32	0.28	−0.39	.21	1
CyberSim 2 msec	0.26	0.41	0.38	0.42	0.52	0.57	−0.46	0.53	0.22	1
CyberSim 2 wobbling	0.26	0.40	0.34	0.38	0.27	0.22	−0.39	0.17	0.84	0.25	1
CyberSim 3 msec	0.45	0.58	0.45	0.53	0.58	0.61	–0. 51	0.46	0.26	0.67	0.26	1
CyberSim 3 wobbling	0.26	0.48	0.39	0.45	0.36	0.28	−0.44	0.23	0.82	0.31	0.87	0.33	1

Bold numbers r_s > 0.5 and p < 0.001

UFOV: Useful Field of View; TMT: Trail Making Test; NorSDSA: Nordic Stroke Driver Screening Assessment; proc: process; div: divided; sel: selective; msec: milliseconds.

Discussion

There is no existing consensus on how to assess cognitive abilities related to driving in patients diagnosed with an acquired brain injury/disease. Many assessment tools exist and there are many studies from different countries, diagnostic groups and settings, reporting varying results and recommendations. All differences might have an impact on how to interpret validity and feasibility into a new setting and thus should be implemented with care. In addition, studies following outcomes (risks in terms of traffic incidents) from assessments over time are lacking, probably due to difficulties in conducting such studies. Thus, we do not know enough about the sensitivity and specificity of existing tools based on actual risk factors. It is not justifiable to design a study where we let patients drive, even though we do not believe it is safe, just to look at what happens over time. In addition, because an assessment on driving behaviour includes aspects of individual abilities and conditions, a person-centred approach is required. In clinical practice, there are differences in resources and professional experience as well as knowledge, making driving assessment even more challenging (Larsson and Falkmer, 2014). A simulator as a complement to existing cognitive tests might allow the assessor to observe behaviours in situations that are similar to a traffic environment and thus should add valuable information for the final decision on continued driving. Before using a new assessment tool in clinical practice, it is necessary to look at its validity as well as its feasibility. Driving assessment is a common clinical question for many occupational therapists, especially for those who are working within geriatrics, neurology or rehabilitation.

The aim of the study was to evaluate the feasibility and construct validity of a new driving simulator tool as a complement to cognitive tests in a patient group referred for medical risk assessment related to continued car driving after an injury or disease affecting the brain. A validation process was performed to ensure the clinical usefulness of the new equipment. Construct validity was evaluated by analysing data collected from four different rehabilitation services dealing with driving assessment, thus enhancing reliability and reducing assessor bias. Thoughts and experiences related to the feasibility of the new equipment were gathered and summarised based on information from the second workshop, which included all participating occupational therapists.

Positive experiences of using the new equipment support the feasibility of CyberSim for the future. Therapists and physicians considered the new equipment to be a valuable contribution to other methods/tests used and a tool of relevance to the patient group.

Differences in the test results when comparing groups (pass/fail to continued driving) were significant for most variables, including the simulator subtests, supporting its feasibility. Because age has been shown to have an effect on cognitive test results in several studies (Selander et al., 2020; Tombaugh, 2004) and because there was a difference in age between the pass and fail groups, an analysis adjusting for the age difference was performed. The results showed that the difference between groups remained for most variables except for UFOV 1, CyberSim 1 (misses and faults) and CyberSim 2 (faults).

Construct validity was evaluated using a factor analysis showing a high level of explained variance for the four components identified. The results support that the simulator tool contributes to explaining variance in a significant way. Results from CyberSim were represented in all four components, indicating that CyberSim, in addition to attention, contributes with information on other cognitive abilities identified: processing speed, visuospatial function, simultaneous capacity and executive function. Attention has been found to be one of the most important cognitive abilities for driving. Attention is an overall concept, and multiple types are necessary to maintain safe driving (Fawcett et al., 2015). Focused, divided, selective and sustained attention are some of the types described in the literature. Focused attention allows drivers to react to unpredictable events, which is considered to be assessed in all subtests of the CyberSim. Divided and selective attention (ignore distractors and attend to hazardous stimuli) are considered to be assessed in subtests 2 and 3. Sustained attention is required to keep the driver engaged in the driving task, which seems to be required in every subtest of CyberSim but also in every other cognitive test used (Roca et al., 2011).

An analysis of construct validity showed that the different subtests and criteria evaluated for CyberSim had varying correlation with the other tests. CyberSim subtest 3 had the strongest correlation values when comparing the results with several of the other tests (Table 5). The wobbling factor had a correlation of r_s < 0.5 when comparing the results with all the other test results. These results indicate that CyberSim measures other components than the other tests, as confirmed in the factor analysis (Tables 3 and 5).

In contrast to other studies (for example Hird et al., 2016), there were more men participating in this study than women. The difference in gender cannot be explained by the number of licensed drivers in the Nordic countries or the incidence of stroke or dementia, which were the main diagnostic groups in this study. However, it does reflect the proportion of men versus women referred to clinical services for driving assessment in Sweden; this phenomenon requires further study.

By using different cognitive tests especially developed and used within driving assessment, therapists collect information on patients’ cognitive abilities necessary at the tactical level of behaviour (Michon, 1985). The operational level as defined by Michon (1985) could be observed using a simulator driving tool and/or on-road observation. This allows inclusion of other components of cognition necessary for safe driving behaviour, and thus the quality and safety of medical assessment should increase. Despite the fact that the assessment process has been improved by using a simulator tool, it still needs to be further improved by adding information on strategic levels and behaviour, as described by Michon (1985) and Shinar and Oppenheim (2011). Driving behaviour is based on conscious decisions, judgement of traffic situations and risk avoidance. In addition, long-terms decisions, such as which route to choose or at what time of the day to drive, also affect driving behaviour; cognitive tests do not cover these decisions. Schanke et al. (2008) did show the importance of including information on driving behaviour, especially for participants with traumatic brain injury, because they were found to have an increased risk for traffic accidents compared with norms. Thus, therapists need to collect information from other sources and with other methods, especially regarding the strategic level of behaviour.

Results from the comparisons of data for individuals who got a pass versus fail to continued driving showed overlapping standard deviations on all variables. Based on these results, no sharp boundaries, such as cut-off values, are relevant. Instead, a pragmatic and person-centred approach is required.

Several complementary assessment tools are often needed to get a nuanced basis for a decision on this complex question. On-road observation might help in getting a more holistic basis for the decision, because it adds information on performance-related, tactical control and the strategic level of behaviour by observing individuals in a real traffic environment (Shinar and Oppenheim, 2011). However, on-road observation has its shortcomings because it cannot be standardised, it is not always accessible, it is expensive and it includes risks. An on-road observation also does not give any answers about what cognitive abilities are impaired. A simulator tool such as the CyberSim includes standardised tasks and identifiable and measurable cognitive challenges. Thus, simulator results could contribute to the final decision on continued car-driving.

Study limitations and strengths

The final decision on continued driving, which was used as the gold standard, is always a subjective decision, although based on cognitive tests and, when needed, an on-road observation. The therapists, departments and physicians included in this study might have different routines and experiences in observing car driving after a brain injury or disease, which could have affected the reliability of the final decisions. The lack of analysable results from the on-road observation is a limitation of the study; it is recommended that future studies include this information. This study used the second workshop, including occupational therapists and physicians with experiences from using the CyberSim, to ensure the construct validity. A focus group methodology with external experts could be useful to evaluate feasibility more thoroughly. However, for the present study this was not possible because of the limited number of professionals with experience of the new tool.

The strengths of the study concerned the data collection process: it was standardised as well as how the new simulator tool was used in order to improve reliability. The results from the simulator tool were not used in the final decision process.

Conclusion

The CyberSim was found to be valid, to have high feasibility and to add new knowledge of interest to the occupational therapy community. In addition, the new simulator tool added valuable information not included in the other cognitive tests.

The simulator tool was reported to be easy to use and of value when looking at attention as well as behaviour. It was also considered useful in giving feedback to the patients because the results are easily presented in the protocol from each subtest. Therapists found it relevant for evaluating the competence needed for car driving.

The CyberSim includes components other than those measured in the other tests. Wobbling as well as misses and faults in the CyberSim simulator test formed separate factors, indicating cognitive aspects (such as visuospatial function, simultaneous capacity and executive function) other than attention and processing speed. CyberSim contributes to increasing the reliability of driving assessment and should be used together with cognitive tests.

Key findings

Construct validity for the CyberSim was confirmed.

The CyberSim was found feasible by participating professionals (occupational therapists and physicians) as well as included participants.

What the study has added

The study showed that the CyberSim is a valuable complement to traditional cognitive tests within driving assessment.

Footnotes

Acknowledgements

We would like to thank all the occupational therapists at the participating units who contributed to the data collection for the present study; the Department of rehabilitation medicine Jönköping, Sweden; the Department of rehabilitation medicine Linköping, Sweden; the Mobility Centre Gothenburg, Sweden and SamRehab Västervik, Sweden.

Research ethics

Ethical approval for the simulator participant study was obtained from the Ethics Regional Board in Linköping (Dnr 2016-181-31 and 2016-271-32).

Declaration of conflicting interests

The authors report no conflicts of interest.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the County Council of Östergötland, Sweden (LIO-582071; SC-2017-00091-26) and the Medical Research Council of Southeast Sweden (FORSS – 605951; 654091; 755541; 850271).

Contributorship

Both authors have planned the study, analysed data and composed the manuscript.

ORCID iD

Kersti Samuelsson

References

Adler

Rottunda

(2006) Older adults’ perspectives on driving cessation. Journal of Aging Studies 20(3): 227–235.

Brouwer

Busscher

Davidse

, et al. (2011) Traumatic brain injury: Tests in a driving simulator as part of the neuropsychological assessment of fitness to drive. In: Fisher DL, Rizzo M, Caird J, et al. (eds) Handbook of Driving Simulation for Engineering, Medicine and Psychology, 1st ed. Boca Raton, FL: CRC Press, pp. 659–669.

Dragos

Rothkrantz

(2017) Affective computing and augmented reality for car driving simulators. Acta Polytechnica CTU Proceedings 12: 13–23.

Fawcett

Risko

Kingstone

(eds) (2015) The Handbook of Attention. London: MIT Press.

Field

Unsworth

(2017) Revision of the competency standards for occupational therapy driver assessors: An overview of the evidence for the inclusion of cognitive and perceptual assessments within fitness-to-drive evaluations. Australian Occupational Therapy Journal 64(4): 328–339.

Hird

Egeto

Fischer

, et al. (2016) A systematic review and meta-analysis of on-road simulator and cognitive driving assessment in Alzheimer’s disease and mild cognitive impairment. Journal of Alzheimer’s Disease 53(2): 713–729.

Korner-Bitensky

Gélinas

Man-Son-Hing

, et al. (2005) Recommendations of the Canadian Consensus Conference on driving evaluation in older drivers. Physical & Occupational Therapy in Geriatrics 23(2–3): 123–144.

Lafont

Laumon

Helmer

, et al. (2008) Driving cessation and self-reported car crashes in older drivers: The impact of cognitive impairment and dementia in a population-based study. Journal of Geriatric Psychiatry and Neurology 21(3): 171–182.

Larsson

Falkmer

(2014) A Swedish survey of occupational therapists' involvement and performance in driving assessments. Scandinavian Journal of Occupational Therapy 14(4): 215–220.

10.

Lundberg

(2003) Manual for the Nordic Stroke Driver Assessment (NorSDSA). NorSDSA Produktion Görel Caneman and Catarina Lundberg [in Swedish]. Blidö, Sweden.

11.

Lundberg

Caneman

Samuelsson

, et al. (2003) The assessment of fitness to drive after a stroke: The Nordic Stroke Driver Screening Assessment. Scandinavian Journal of Psychology 44(1): 23–30.

12.

Marshall

Molnar

Man-Son-Hing

, et al. (2007) Predictors of driving ability following stroke: A systematic review. Topics in Stroke Rehabilitation 14(1): 98–114.

13.

Michon

(1985) A critical view of driving behaviour models: What do we know, what should we do? In: Evans

Schwing

(eds) Human Behavior and Traffic Safety. New York: Plenum Press, pp. 485–520.

14.

Mukaka

(2012) Statistics corner: A guide to appropriate use of correlation coefficient in medical research. Malawi Medical Journal 24(3): 69–71.

15.

Nouri

Lincoln

(1992) Validation of a cognitive assessment: Predicting driving performance after stroke. Clinical Rehabilitation 6: 275–281.

16.

Patel

(2014) Neuropsychological factors associated with Useful Field of View. MSc Thesis, College of Arts and Sciences, Ohio University.

17.

Roca

Castro

López-Ramón

, et al. (2011) Measuring vigilance while assessing the functioning of the three attentional networks: The ANTI-Vigilance task. Journal of Neuroscience Methods 198(2): 312–324.

18.

Roca

Lupiáñez

López-Ramón

, et al. (2013) Are drivers’ attentional lapses associated with the functioning of the neurocognitive attentional networks and with cognitive failure in everyday life? Transportation Research Part F: Traffic Psychology Behaviour 17: 98–113.

19.

Samuelsson

Modig-Arding

Wressle

(2018) Driving after an injury or disease affecting the brain: An analysis of clinical data. British Journal of Occupational Therapy 81(7): 376–383.

20.

Samuelsson

Tropp

Lundqvist

, et al. (2019) Development, concurrent validity and internal consistency of a simulator tool for assessing continued car driving after a brain injury/disease. British Journal of Occupational Therapy 82(9): 544–552.

21.

Schanke

Rike

Mølmen

, et al. (2008) Driving behaviour after brain injury: A follow-up of accident rate and driving patterns 6–9 years post-injury. Rehabilitation Medicine 40(9): 733–736.

22.

Selander

Wressle

Samuelsson

(2020) Cognitive prerequisites for fitness to drive: Norm values for the TMT, UFOV and NorSDSA tests. Scandinavian Journal of Occupational Therapy 27(3): 231–239.

23.

Seong-Youl

Jae-Shin

A-Young

(2014) Cognitive test to forecast unsafe driving in older drivers: Meta-analysis. NeuroRehabilitation 35(4): 771–778.

24.

Shinar

Oppenheim

(2011) Review of models of driver behavior and development of a unified driver behavior model for driving in safety critical situations. In: Cacciabue

(ed.) Human Modelling in Assisted Transportation. Milan: Springer, 215–223.

25.

Tombaugh

(2004) Trail Making Test A and B: Normative data stratified by age and education. Archives of Clinical Neuropsychology 19(2): 203–214.

26.

Visual Awareness Research Group (2009) UFOV User’s Guide Version 6.1.4. Birmingham, AL: Visual Awareness.