Screening for Lifetime History of Traumatic Brain Injury Among Older American and Irish Adults at Risk for Dementia: Development and Validation of a Web-Based Survey

INTRODUCTION

Traumatic brain injury (TBI) is common across the lifecourse and is an established risk factor for dementia [1, 2], increasing risk by two- to four-fold in some studies [3, 4]. While TBI in young adult athletes and veterans has received much recent attention, the highest incidence of TBI in civilian populations is in the elderly: One in 50 US adults age 75 years or older received medical care for a TBI in 2013, mostly due to ground-level falls [5]. The total lifetime prevalence of TBI among community-dwelling adults has been estimated at more than 40% for all-severity TBI [6] and more than 20% for TBI with loss of consciousness (LOC) [7]. Recently, several large epidemiological studies of administrative data have established that even mild TBI is an important risk factor for dementia in both civilians and military veterans [4, 8–12]. Mild TBI comprises the vast majority of TBIs in the US and globally, affecting more than an estimated 42 million people worldwide every year [8]. Several studies have also found that increasing TBI severity or older age at injury is associated with increasing risk for dementia [4, 8–12].

Mechanisms and prevention of post-TBI dementia, however, remain uncertain [13]. Thus, accurate measurement of total burden of lifetime TBI exposure, including number, severity, and timing of TBIs is of paramount importance in longitudinal cohort studies of cognitive aging that seek to unravel mechanisms and develop interventions for post-TBI cognitive decline and dementia. Relying solely on medical record documentation of TBI, while less prone to recall bias, under-estimates lifetime history of TBI exposure [14, 15]. Medical records are particularly inadequate for capturing lifetime history of TBI in older adults and those with milder TBIs as these sub-groups are the least likely to present to medical attention [14, 16]. Additionally, in aging cohorts, medical records may have never or may no longer exist for earlier-life exposures. For all of these reasons, it is usually necessary to rely on self-report in order to capture the lifetime history of TBI in aging cohorts. However, brief TBI exposure screens, comprised of one to three questions, may miss up to 80% of injuries that are identified using more comprehensive screens [16, 17]. In a large cohort study, a poorly sensitive screen will lead to substantial exposure mis-classification, thereby reducing power and limiting discovery of TBI-related sequelae.

Our goal in this study was to directly facilitate high-quality international cohort studies of post-TBI cognitive decline and dementia by developing and validating a low-cost, scalable, web-based, self-administered, comprehensive TBI exposure survey. Self-administration was considered important for scalability. Conducting a comprehensive TBI exposure interview requires staff training and may be prohibitively time-intensive or costly for many large longitudinal cohort studies of cognitive aging. First, we used qualitative methods to adapt an existing well-validated comprehensive semi-structured TBI exposure interview—the Ohio State University TBI Identification Method (OSU TBI-ID)—into an age-appropriate and culturally-appropriate interview and self-administered web-based survey appropriate for use in older adults residing in the US and Ireland. Next, we assessed test-retest reliability of the web-based survey against the ‘gold-standard’ interview in an international cohort of community-dwelling American and Irish older adults with normal cognition and mild cognitive impairment (MCI). Finally, to investigate the added value of using a comprehensive TBI screen versus a brief screen in a longitudinal cohort study of cognitive aging, we retrospectively evaluated the sensitivity and specificity of the brief TBI exposure screen currently used in the National Alzheimer’s Coordinating Center (NACC) Uniform Data Set (UDS) Version 3 [18] versus a comprehensive OSU TBI-ID interview.

METHODS

Survey validation methods

Study design

This was a two-site prospective cohort study with a within-participant repeated measures design.

Participants

Inclusion criteria for participants at both sites were as follows: age 50 years or older, English-speaking, have access to a computer or tablet with internet connectivity, and ability to participate in both the in-person interview and web-based survey within two to four weeks of each other. Exclusion criteria at both sites included being a current prisoner or patient in custody, any pre-existing disabling medical, neurological, or psychiatric condition that would impair ability to complete the interview or survey (e.g., major stroke, multiple sclerosis, cancer, schizophrenia), diagnosis of dementia (specified further below), or non-English speaking.

As described above, all participants at the US site had a diagnosis of normal cognition based on a comprehensive neurobehavioral assessment; participants at the Irish site had a diagnosis of normal cognition or MCI based either on a comprehensive neurobehavioral assessment (if recruited from memory disorders clinics) or the MoCA (if recruited from the community). Participants with MCI were encouraged to complete the interview and web-based survey with the assistance of an informant/caregiver; however, this was not a requirement for participation and was not considered in the analysis. This was done to approximate how this instrument would likely be used in a research study of cognitive aging where a participant might elect to request help from a caregiver to complete the online survey either for technology support or for help recalling details of an injury.

Procedures

Upon providing written informed consent, baseline demographic and clinical variables were collected: age, sex, race/ethnicity, years of education, employment status, and cognitive diagnosis (normal versus MCI as described above). Each participant completed the OA OSU TBI-ID twice; once via the Web-Based Survey (self-administered on personal computer or tablet off-site) and once via the Interview (interviewer-administered, in-person at each site). The two instruments were spaced a minimum of seven days and a maximum of 28 days apart to minimize practice/recall effects and reduce likelihood of interim injury. Participants completing the second instrument outside of the 7–28 window were excluded from further analysis. The order of administration was randomized to balance practice/recall effects of one instrument on the other. The Web-Based Survey was accessible from any web-browser on any device with internet capability after navigating to the REDCap website and entering a unique pass code. This pass code was provided to participants via email.

Target enrollment was 50 participants at the US site (all with normal cognition) and 50 participants at the Irish site (half with normal cognition, half with MCI). Enrollment was capped upon enrolling N = 113 participants, of whom N = 97 met all inclusion criteria (See Fig. 1). Of those enrolled at the Irish site, N = 15 were recruited from memory disorders clinics (of whom, all were categorized as MCI) and N = 30 were recruited from the community (of whom N = 6 were categorized as MCI).

OPEN IN VIEWER

Fig.1

Study flow chart showing validation cohort participants included in the reliability analysis. Participants were included in the reliability analysis if they completed both the OA OSU TBI-ID interview and the OA OSU TBI-ID web-based survey spaced 7–28 days apart.

RESULTS

DISCUSSION

We successfully adapted the OSU TBI-ID into an age- and culturally-appropriate TBI exposure interview and web-based survey for use in older community-dwelling adults in the US and Ireland who are at risk for dementia. We established reliability of our web-based survey versus the gold standard interview for comprehensively measuring lifetime history of TBI in this population. We demonstrated the added value of using a comprehensive OSU TBI-ID screen versus a brief screen in longitudinal cohort studies of cognitive aging. Specifically, we found that our web-based OSU TBI-ID survey is reliable as compared to the gold standard OSU TBI-ID interview for gathering all types of TBI history variables among older community dwelling adults in the US and Ireland with the exception of variables pertaining to age at injury among individuals with MCI.

Reliability of our web-based OA OSU TBI-ID survey was similar or superior to that of prior test-retest reliability studies of the OSU TBI-ID. This includes studies investigating test re-test reliability of the OSU TBI interview in US prison inmates [23], adults with moderate to severe TBI participating in the TBI Model Systems study [26], and the computer-assisted telephone interview (CATI) version of the OSU TBI-ID among adults residing in Colorado [35]. Recently, preliminary feasibility and predictive validity of a similar web-based self-administered OSU TBI-ID survey was demonstrated in a sample of US adults of all ages [37]. In this study, adults age 18 years and older (mean age 47 years, range 18–82 years) electively completed a web-based OSU TBI-ID via SurveyMonkey. Nearly all participants completed the survey in two to eight minutes. TBI severity was significantly associated with worse self-reported neurobehavioral symptoms, thereby establishing predictive validity of the survey [37]. Our study adds to this prior work by demonstrating reliability of a web-based OSU TBI-ID survey versus an in-person ‘gold standard’ OSU TBI-ID interview specifically in an older adult population with normal cognition and MCI. Our study thereby paves the way for use of this instrument in large cohort studies of cognitive aging.

In our survey reliability study, reliability was better among those who took the web-based survey before the interview as compared to those who were interviewed first. One possible explanation for this order effect is that self-administration of the web-based survey may have been more memorable to participants than the interview leading to greater recall effect (and higher reliability for survey-first participants). A related explanation is that respondents completing the web-based survey at home might have received assistance from another person which could further prime recall and carry over to the interview. An alternative explanation is that the semi-structured interview inadvertently informed the participants’ opinion about what should be considered a reportable injury thereby leading to modified/different reporting of injuries in the web-based survey (and lower reliability in interview-first participants). This finding therefore raises the testable hypothesis that test-retest reliability of the web-based survey may be superior than test-retest reliability of the interview.

Our screen comparison study found that while specificity of the NACC-UDS TBI screen was high, sensitivity was poor. This brief screen missed 50% of all-severity TBI exposures and 54% of TBI with LOC exposures reported via the OSU TBI-ID interview in a cohort of older cognitively normal adults. While it is surprising that sensitivity of the NACC-UDS was not higher among individuals with TBI with LOC (compared to those without LOC), one explanation for this finding is that most of the injuries reported in our study were extremely mild with only brief LOC. Thus, it remains possible that the NACC-UDS screen might perform better in a population with a higher prevalence of moderate to severe TBI with more prolonged periods of LOC. Overall, however, our findings are consistent with what has been reported in prior studies in predominantly younger adult populations of prisoners and veterans—who have a higher prevalence of moderate to severe TBI compared to our study population—which have reported that single-item TBI screens may miss up to 80% of TBI exposures identified via a comprehensive interview [16, 17].

Prevalence of TBI as measured by the OSU TBI-ID interview, differed slightly across our Survey Validation cohort (44%) versus our Screen Comparison cohort (34%). There are several likely contributors to this difference including chance and the relatively small sample size of these studies. Differential participant burden across the two studies (e.g., those with a history of TBI might have been more likely to complete the web-based OA OSU TBI-ID and therefore be included in the Survey Validation cohort), differential cognitive status across the two studies (normal cognition plus MCI versus normal cognition only), and differential nationalities across the two studies (US plus Irish versus US only) are also likely to have influenced results.

Exposure mis-classification may be non-differential (e.g., occurs at equal rates among those with and without the outcome of interest) or differential (e.g., occurs at a different rate among those with versus without the outcome of interest) [38]. If non-differential, the analysis may be biased toward the null thereby reducing power to detect an association between exposure and outcome if one is present. If differential, the analysis may be falsely biased away from the null. For example, if patients with dementia are less likely to remember and report their lifetime history of TBI in response to a brief screen as compared to cognitively normal controls, then a case-control study using a brief TBI screen may be biased toward identifying a protective effect of TBI on risk of dementia (e.g., a negative association). Importantly, our reliability study demonstrated that our comprehensive web-based OA OSU TBI-ID survey had excellent reliability even among individuals with MCI.

There have been several prior published studies using NACC-UDS TBI data to investigate associations of TBI exposure with various dementia-related outcomes [39–44]. A number of these prior studies have failed to identify statistically significant associations between history of TBI and risk of the dementia outcomes of interest, although several had identified non-significant trends [40–42]. Our findings suggest that more accurate exposure classification in NACC-UDS data, as could be achieved with the OA OSU TBI-ID interview or web-based survey, would likely have increased power and possibly reduced bias of these prior epidemiological analyses.

Our study has many strengths including combining qualitative methods for survey development and quantitative methods for survey validation and screen comparison. Additional strengths include being a two-site international study and inclusion of older adults with both normal cognition and MCI, which is important for generalizability to large longitudinal cohort studies of cognitive aging. Our survey validation study is limited by the relatively small size of our cohort and lack of measurement of time needed to complete the survey. Future studies could assess feasibility of the OA OSU TBI-ID web-based survey in a large population of older English speaking adults and determine whether completion time is indeed eight minutes or less as reported for the SurveyMonkey web-based OSU TBI-ID [37]. Future studies could additionally investigate whether the OA OSU TBI-ID is valid and reliable in younger populations. None of the participants in our survey validation study endorsed a history of moderate or severe TBI, potentially limiting generalizability to populations with more severe TBI. Prior studies of reliability of the OSU TBI-ID, however, have found somewhat higher reliabilities among populations with more severe TBI (Table 3), likely because milder injuries may be less memorable. These prior studies suggest that reliability of our OA OSU TBI-ID survey in older adults with a history of moderate to severe TBI would be expected to be as high if not higher than what we found in this study of older adults with and without mTBI.

In conclusion, we have demonstrated that older community-dwelling adults with MCI and normal cognition are capable of providing reliable self-reports of lifetime history of TBI via a web-based survey. We further demonstrate that the OSU TBI-ID interview is substantially more sensitive for both history of all severity TBI and history of TBI with LOC than the brief NACC UDS TBI screen. The web-based OA OSU TBI-ID survey and interview may enhance current and future longitudinal cohort studies of cognitive aging that seek to unravel mechanisms of TBI-related cognitive decline and dementia.