Abstract
Return to driving after a change in functional ability is a growing area of focus in rehabilitation. The decision of whether resumption of driving is appropriate for a person has fallen on physicians, and they often rely on a comprehensive driving assessment when making such recommendations. In Canada and the United States, these assessments are primarily completed by occupational therapists (Dickerson, 2013; Vrkljan et al., 2011). In 2009, the Canadian Association of Occupational Therapists (CAOT) created a position statement related to occupational therapy’s role in driver rehabilitation that acknowledges occupational therapists’ expertise in supporting safe driving, including evidence-based evaluation (CAOT, 2009).
It is well recognized in the scholarly literature that no single assessment tool can predict whether a person is safe to drive (Dickerson et al., 2014). Driving assessment is multidimensional, and specific off-road and on-road assessment practices vary greatly, although semistandardized on-road assessment is used most often, along with off-road assessments, including the Trail Making Test (TMT), Parts A and B (TMTA and TMTB; Bowie & Harvey, 2006); the Motor-Free Visual Perception Test (MVPT; Colarusso & Hammill, 2003); and a brake reaction time test (Korner-Bitensky et al., 2006). A combination of clinical and on-road assessments is considered the gold standard for evaluation of fitness to drive, but not all clinical assessment tools are widely available.
The MVPT has become a standard component of the driver screening process because it is easily accessible. However, there are concerns regarding its utility in driving assessment. Mazer et al. (1998) used the original version of the MVPT and found that people who scored ≤30 were more likely to fail an on-road assessment and would benefit from a more comprehensive driving evaluation. Vrkljan et al. (2011) recommended using the MVPT in conjunction with other assessment tools because the cutoff score does not predict whether the client will fail an on-road evaluation. Oswanski et al. (2007) suggested the optimal MVPT cutoff score for drivers older than age 55 yr is 32. Nevertheless, the MVPT’s ability to predict on-road performance has rarely been addressed.
Whether the TMTA and TMTB reflect a person’s cognitive ability, including basic and complex attention, has been well addressed in the scholarly literature. Roy and Molnar (2013) completed a systematic review of the literature in an attempt to establish acceptable driving norms for the TMT. They found that people who took more than 180 s to complete the assessment or who made three or more errors on the TMTB were more likely to fail an on-road assessment.
In this study, we used data extracted from patients’ clinical charts to examine the ability of two clinical assessments, the MVPT and the TMT, to predict on-road driving performance. We examined these two tools because they are most commonly administered in clinical driving assessments and have previously been shown to reflect driving performance. Our specific objective was to compare MVPT and TMT scores of patients who passed and patients who failed the on-road driving assessment to investigate these tools’ ability to predict on-road driving performance. We also examined their predictive ability in relation to specific diagnoses.
Method
A retrospective chart review was completed according to the methodological guidelines outlined in Gearing et al. (2006). The study was conducted in accordance with the Health Research Ethics Board and approved by the Northern Alberta Clinical Trials and Research Centre. Data were collected by Ana Holowaychuk and Yolan Parrott from clinical charts associated with a Tier 3 driver evaluation program at Glenrose Rehabilitation Hospital (Edmonton, Alberta, Canada) from 2015 to 2016. The data extracted were age, gender, primary diagnosis, years of driving experience, MVPT and TMTA–TMTB scores, and outcome of the on-road evaluation. Completion of both clinical and on-road assessments was required. Patients with a primary diagnosis of stroke; general neurological condition such as traumatic brain injury, multiple sclerosis, and epilepsy; or spinal cord injury were included. Patients who were new drivers (<5 yr experience) were excluded.
Clinical Assessments
Motor-Free Visual Perception Test (Original Version)
The MVPT assesses the accuracy and speed of visual–perceptual (VP) processing (Brown & Elliott, 2011). It consists of 36 questions that explore different domains, including figure ground, hidden figures, visual closure, visual memory, and visual discrimination. Higher scores indicate better VP abilities. VP speed (in seconds) was recorded as a component of the assessment and was included in the study.
Trail Making Test, Parts A and B
The TMT consists of two parts, A and B. The TMTA is used to evaluate basic processes of visual attention and sequencing; the TMTB, to evaluate higher level cognitive skills, such as executive control, divided and alternating attention, and cognitive flexibility (Oosterman et al., 2010). On the TMTA, test takers are required to connect numbers consecutively; on the TMTB, they are required to connect numbers and letters in alternating order. The number of errors and completion time (in seconds) taken to complete each part are recorded and were included in our study.
On-Road Driving Evaluation
The on-road evaluation took place on a semistandardized route that included a combination of residential and main roads. The assessment was completed by an occupational therapist and a certified driving instructor in a vehicle equipped with a passenger-side safety brake. The driving instructor was seated in the front passenger seat and provided all instructions for the assessment. The occupational therapist was seated in the back seat on the passenger side to allow for observation of physical operation of the vehicle along with driving performance.
After the assessment, the occupational therapist and the driving instructor reviewed the observations and made a recommendation: (1) pass—appropriate to proceed with community standard-based road test (if recommended by licensing authority); (2) fail, not appropriate at present—individual has potential to improve, and therefore a reassessment would be considered; and (3) fail, not fit to drive—individual is not appropriate to drive and has limited potential for improvement.
Data Analysis
Demographic variables such as age, mental illness, driving experience, language, and speech problems are presented using descriptive statistics. One-way analysis of variance (ANOVA; for age) and χ2 tests (for other demographic variables) were applied. Raw scores on clinical assessments and categorical data from the on-road driving performance (i.e., pass or fail) were used in the statistical analysis. Group differences on clinical assessments were examined using one-way ANOVA followed by post hoc comparisons using the Bonferroni test, with statistical significance set at p < .05. Differences between test scores of those who passed and those who failed the on-road evaluation were examined using independent-samples t tests.
To examine the clinical assessments’ predictive ability for driving performance, we performed logistic regression analyses using the enter method with driving performance (pass or fail) as the outcome variable and age, clinical assessment test scores, and driving training as predictor variables. Most patients had stroke as their primary diagnosis; therefore, we computed two separate logistic regression models, one including all patients (the all-patient model) and the other including only patients with stroke (the stroke-only model), for interpretation.
Results were reported with −2 log-likelihood (−2LL) to reflect the accuracy of prediction, Cox and Snell R 2 and Nagelkerke R 2 to reflect the percentage of variance explained by the predictors, and Wald statistics to reflect the significance of the predictors. Data quality was determined by insignificant results for Hosmer and Lemeshow’s goodness-of-fit statistic, as well as tolerance and variance inflation factor (VIF) values >0.1 and <10, respectively, on the collinearity statistics. All statistical analyses were performed with IBM SPSS Statistics (Version 22; IBM Corp., Armonk, NY). Power analysis using NCSS/PASS 2002 software (NCSS, LLC, Kaysville, UT) showed that a sample size of 82 patients would achieve 90% of power at the p < .05 level to detect a change in the outcome variable at an odds ratio of 4.75.
Results
In total, 89 charts were reviewed, and 7 were eliminated because of incomplete data. Patients’ mean age was 54.27 yr (SD = 12.92; range = 25–84 yr). The patients included 65 men and 17 women, 58 with stroke, 13 with a general neurological condition, and 11 with spinal cord injury (SCI). Demographic characteristics of the sample are provided in Table 1; χ2 tests showed insignificant differences among diagnoses. Patients had significant differences on age (F[2, 79] = 8.08, p = .001), and those with stroke were significantly older than those with a general neurological condition (p = .008) or SCI (p = .009).
Patient Demographics by Diagnosis
Note. M = mean; SCI = spinal cord injury; SD = standard deviation.
p < .005.
Clinical assessment results are shown in Table 2. Overall, VP speed, TMTA, TMTB, and MVPT were scored in the high range. Significant differences were found among the three groups on VP speed (F[2, 79] = 7.09, p = .001), time to complete the TMTA (F[2, 79] = 4.08, p = .021), and time to complete the TMTB (F[2, 79] = 3.34, p = .041). Post hoc comparisons revealed that patients with stroke performed significantly worse than patients with SCI on all three variables (ps = .002, .020, and .035, for VP speed, TMTA time to complete, and TMTB time to complete, respectively).
Test Performance by Diagnosis
Note. M = mean; MVPT = Motor-Free Visual Perception Test; SCI = spinal cord injury; SD = standard deviation; TMTA = Trail Making Test, Part A; TMTB = Trail Making Test, Part B; VP = visual–perceptual.
p < .05. **p < .005.
Clinical assessment results by on-road driving performance are shown in Table 3. Thirty-six patients passed the on-road evaluation, and 46 patients failed. Patients who failed the on-road evaluation made significantly more errors on the TMTA (t[65] = 2.34, p = .023) and took longer to complete the TMTB (t[67] = 2.66, p = .010).
Test Performance by On-Road Evaluation Results
Note. df = degree of freedom; M = mean; MVPT = Motor-Free Visual Perception Test; SD = standard deviation; TMTA = Trail Making Test, Part A; TMTB = Trail Making Test, Part B; VP = visual–perceptual.
p < .05.
The results of the logistic regression are shown in Table 4. The all-patient model predicted 66% of on-road driving performance with a significant drop of −2LL to 93.47 (χ2[N = 82] = 18.99, p = .015) and an explained variance of 27% and 20% assessed using Nagelkerke R 2 and Cox and Snell R 2, respectively. Among all the predictors, only TMTB time to complete was significant (Wald statistics = 4.93, β = −0.024, p = .026; exp β = 0.98, 95% confidence interval [CI] [0.96–0.99]), meaning that the odds of passing the on-road evaluation increased with decreased TMTB time to complete. The stroke-only model predicted 76% of on-road driving performance, with a significant drop of −2LL to 59.61 (χ2[N = 58] = 20.18, p = .010) and an explained variance of 39% and 29% assessed using Nagelkerke R 2 and Cox and Snell R 2, respectively. Similar to the all-patient model, only time to complete the TMTB was significant (Wald statistics = 6.19, β = −0.036, p = .013; exp β = 0.97, 95% CI [0.94–0.99]). In both models, MVPT score was not significant in predicting on-road driving performance.
Logistic Regression Results
Note. LL = log-likelihood; MVPT = Motor-Free Visual Perception Test; TMTA = Trail Making Test, Part A; TMTB = Trail Making Test, Part B; VP = visual–perceptual.
p < .05.
Compared with the all-patient model, the stroke-only model yielded a lower −2LL (a decrease of 36% [{93.47 − 59.61}/93.47 × 100]) and greater explained variance (an increase of 10% [76% − 66%]). Data quality was supported; Hosmer and Lemeshow’s goodness-of-fit statistic was insignificant for both models (χ2[N = 82] = 10.07, p = .260, for the all-patient model; χ2[N = 58] = 12.31, p = .138, for the stroke-only model), and the tolerance and VIF values of all the predictors ranged from 0.35 to 0.88 and 1.13 to 2.82, respectively.
Discussion
In this study, we used a retrospective chart review approach and found that performance on an on-road driving assessment significantly differed by TMTA and TMTB scores but not by MVPT score and VP speed. The findings showed that the TMTB was a significant predictor of on-road driving performance for patients with a neurological condition. It is notable that the MVPT, which has been thought to be a necessary tool in the assessment of VP abilities and driving performance, did not appear to be sensitive in predicting on-road driving performance.
One of this study’s key findings is that TMTB completion time significantly differed between patients who passed and patients who failed the on-road evaluation, and it significantly predicted the outcome of an on-road driving evaluation. The relationship of TMTB completion time to driving performance has been demonstrated in past studies (Emerson et al., 2012; Papandonatos et al., 2015). In addition, Classen et al. (2013) showed that the TMTB was as accurate as the Useful Field of View (UFOV), a tool used with people with visual impairment (Tatham et al., 2015) to predict driving performance. Hird et al. (2018) found that the TMT and UFOV were significantly associated with lane maintenance among people with ischemic stroke. However, it is worth noting that in some studies, the TMTA but not the TMTB predicted driving performance, especially among drivers with stroke (Barco et al., 2014). In addition, Vaucher et al. (2014) have reported that the TMT alone was not specific enough for clinicians to determine people’s driving outcome.
Previous studies that have analyzed the component of cognitive processing underlying the TMT have found that although scores on the TMTA and TMTB are highly correlated, they reflect different cognitive processes (Fellows et al., 2017). The TMTA is related more closely to attention, whereas the TMTB is better at assessing more complex cognitive processes such as inhibitory control and visual–spatial sequencing (Fellows et al., 2017). The TMTB could closely reflect driving ability; neural research has found that the TMTB induces heightened low-level visual search demands, switching between elements, and the need for speeded performance (MacPherson et al., 2017; Varjacic et al., 2018). In addition, neuroimaging studies have found an association between the neural activities of the executive control network and TMTB completion time (MacPherson et al., 2017; Varjacic et al., 2018), suggesting the sensitivity of the time domain in TMTB. This research further supports our findings that TMTB completion time but not error scores predicted on-road driving performance.
The MVPT’s lack of predictive ability could be attributed to two reasons. First, our sample’s MVPT scores appeared to be on the high end, with an average of about 34.06 for those who passed the on-road evaluation and 33.52 for those who failed. Unlike other studies that have reported substantially lower scores of <30 on the MVPT (Korner-Bitensky et al., 2000), scores in this study might reflect a ceiling effect. They might be attributable to patients’ education level, which was not included in their charts. Another plausible reason could be that the results of MVPT were inconsistent. In the past, some studies with older adults and people with stroke or traumatic brain injury have reported that the MVPT did not have sufficient predictive validity for on-road evaluation (e.g., Bouillon et al., 2006; Korner-Bitensky et al., 2000). However, other studies have reported that the MVPT was a significant predictor of driving capacity in older adults (Ball et al., 2006; Oswanski et al., 2007).
Moreover, the explained variance of the predictors was quite low for both models—about 28% in the all-patient model and about 39% in stroke-only model. This low level of explained variance was not unexpected because the study included only a few parameters (i.e., tests of the cognitive domain, driving training, and patient age). About one-third of the variance was explained by the cognitive domain, that is, the TMTB. The remaining variance may have been explained by other domains or factors, such as problem solving, inhibitory response, and psychomotor abilities, which we did not examine in this study.
The accuracy of prediction was moderate—66% for the all-patient model and 76% for the stroke-only model. It is notable that although the sample size for the stroke-only model was smaller than that for the all-patient model, the regression statistics for the stroke-only model yielded better results—an increase of 10% for accuracy of prediction and a decrease of 36% for −2LL. A plausible explanation could be that our prediction is sensitive to patients’ diagnoses, which are defined by specific cognitive and noncognitive characteristics. Future studies should avoid grouping patients with different neurological diagnoses together even though they fall within the wide scope of a neurological condition.
Although the MVPT has been updated (MVPT–4), we used the norms for the original version because that was the version documented in patient charts. We are uncertain whether the MVPT was administered with technical limitations that influenced the results. For example, the MVPT does not include norms for people older than age 80 yr. Published research on the MVPT–4, however, is limited, and future studies using the MVPT–4 would be of interest.
Limitations
This study had several limitations. First, the sample size was small, which posed some restrictions on data analysis. With a larger sample size, separate regression models could be constructed, corresponding to different diagnoses. Second, the study’s retrospective nature limited our ability to control for certain variables (e.g., age) and include other potential predictors. This is a disadvantage of retrospective studies, in which the composition of factors is often guided by clinical needs. Third, education level, which might influence performance on standardized assessment, was not known. Fourth, charts were retrieved from one local hospital, which limits the generalizability of the findings. Last, the driver evaluation program primarily completed assessments of patients with a neurological condition; patients with other conditions were not assessed.
Future Research
Further research using a prospective study approach to control for other clinical factors is recommended. Including predictive variables from different domains (cognitive and noncognitive) would provide a comprehensive examination of attributes of on-road performance and help in the development of assessment protocols for driving rehabilitation. Several driver evaluation programs currently use the MVPT–4 as a component of their clinical assessment; further research comparing the MVPT–4 scores and on-road driving performance would be beneficial in guiding decision making among clinicians.
Implications for Occupational Therapy Practice
Our results reinforce the need to consider multiple assessment tools in evaluating patients’ driving performance. Occupational therapists should revisit the use of MVPT as a predictor of driving performance because our results suggest that the TMTB might better reflect driving ability. Occupational therapists should also explore alternate tests or versions of assessment tools to accurately reflect VP function. More important, although many assessment tools are available for formal testing of cognition, certain tools are not in occupational therapists’ scope of practice. Therefore, occupational therapists should identify key components of assessing and predicting on-road driving performance, which can ultimately help develop a functional assessment protocol for driving evaluation.
This study has the following implications for occupational therapy practice:
The TMTB, but not the MVPT, predicted the outcome of on-road evaluation.
Occupational therapists should revisit the use of the MVPT in driving evaluations.
Prospective studies with a wider array of predictive variables are needed to identify proper assessment tools and key components of evaluating on-road driving performance in patients with a medical change.
Conclusion
In this study, we examined the ability of the MVPT and TMT to predict on-road driving performance. Results showed significant differences in TMTA and TMTB results between those who passed or failed the on-road evaluation. Only TMTB completion time predicted the outcome of the on-road evaluation, and these results were identical whether all patients or only patients with stroke were included. However, the MVPT, which has in the past been reported to be a significant indicator of driving performance, was not a significant predictor in this study. The results shed light on the importance of identifying proper clinical assessments to predict on-road driving performance. Future prospective studies with a wider array of predictive variables are recommended to support the present findings.
Footnotes
Acknowledgment
This study was funded by a Glenrose Rehabilitation Hospital Clinical Research Grant.