Traumatic Brain Injury Diagnostic Interview: Development,Interrater Reliability,and 2-Week Post-Injury Clinical Profiles

Abstract

Because most traumatic brain injuries (TBIs) do not present with objective indicators (e.g., neuroimaging findings) to confirm the diagnosis, clinicians often rely on self- or observer-reporting of alteration of consciousness (AOC; e.g., loss of consciousness [LOC], amnesia, other signs of altered mental status), and symptoms to make diagnoses. Moreover, there is no universal agreement on signs and symptoms to sufficiently diagnose TBI, which leads to variability and ambiguity in how TBI is diagnosed in clinical and research settings. The lack of standardized procedures for the diagnosis of acute TBI is a major challenge that hampers the ability to evaluate and compare TBI studies and advance the science and treatment of TBI. We present a new semi-structured TBI Diagnostic Interview (TBI-DI), developed for prospective TBI research to collect injury information important to verifying eligibility for the diagnosis of TBI. Specifically, the TBI-DI collects patient (and/or witness) reports of head trauma, AOC (including LOC and amnesia), and TBI-related symptomology. We describe the protocol, interrater reliability of the TBI-DI items to the same audio-recorded interview, and observed injury characteristics for interviews conducted at 2 weeks post-injury. The sample comprised 335 interviews (320 self-reported, 10 informant-reported, and 5 both) collected on individuals with TBI who were prospectively recruited from 4 U.S. level 1 trauma centers from 2019 to 2023. Cohen’s kappa was calculated to summarize interrater reliability n = 288 interviews. UpSet plots were created to illustrate the prevalence of distinct profiles of signs of AOC and symptom reporting. Overall, there was a near-perfect agreement between raters for all AOC descriptors (κ = 0.85–0.92) and symptom items (κ ranging from 0.92 to 0.99). We observed diverse profiles of AOC, with 45% manifesting witnessed LOC, post-traumatic amnesia, or other altered mental status. Patients (n = 325) self-reported 256 different combinations of the 14 acute symptoms included in the interview (most commonly experiencing headache, dizziness, fatigue, and difficulty concentrating). The TBI-DI and associated SOP appear well-suited for use in a multicenter prospective study of TBI. Future research should examine the stability of reporting by respondents and the alignment between interview and objective clinical information. The TBI-DI solicits diverse acute diagnostic information that, when combined with clinical information (including confounding factors) and objective injury indicators, may inform more rigorous scientific reporting and evidence-based TBI diagnostic practices.

Introduction

Traumatic brain injury (TBI) is a public health crisis that causes considerable morbidity throughout the world, with at least 55 million people living with TBI-related disability.¹ TBI is an injury to the brain due to outside forces, most commonly caused by falls and motor vehicle crashes.² Because objective indicators of TBI (e.g., acute intracranial findings on neuroimaging) are absent in most cases, TBI diagnoses often rely on evaluating alterations of consciousness (AOC) such as loss of consciousness (LOC), post-traumatic amnesia (PTA), and other signs of TBI-related altered mental status (AMS; e.g., disorientation and confusion). While these clinical signs are important indicators of TBI, they are non-specific to TBI and can be present due to confounding factors such as peripheral (non-brain) injuries, intoxication, medication side effects, and pre-existing conditions.^3,4

Moreover, in many cases of suspected or diagnosed TBI, patients present without observable AOC and instead report subjective impressions of AOC (e.g., a gap in memory), AMS (e.g., confusion), and/or other temporally related symptoms (e.g., headache, fatigue, and sensory sensitivity).⁵ As there is no universal agreement on what signs and symptoms suffice to indicate TBI, clinicians and research studies likely vary widely in their threshold for diagnosing TBI and typically do not document their approach explicitly. As a step toward improving the documentation of the clinical signs of TBI, the National Institute of Neurological Disorders and Stroke (NINDS) TBI common data elements (CDEs) recommend coding the level of evidence for LOC as “yes,” “no,” “suspected,” and “unknown,” although the utility of these categories and the uniformity with which they are applied across studies is unclear.^6,7 Acknowledging that TBI diagnoses are often uncertain, others suggest formally considering the diagnosis of TBI in a probabilistic way as “possible, probable, or definite.”⁸ Relatedly, the CDC has created a tiered system to categorize whether a self-reported TBI is “probable, possible, or delayed possible” to surveil population estimates of TBI more effectively.^9,10

Due to the ambiguity surrounding the assessment and diagnosis of TBI, multiple organizations have proposed their own diagnostic criteria. Since 2017, at least nine organizations have established their own diagnostic criteria for TBI that differ in their emphasis on varying signs of altered brain function versus subjective TBI-related symptoms.³ Given uncertainty in how altered brain function in particular should be assessed, definitions have generally avoided specifying operational definitions and assessment methods for AOC diagnostic criteria. Consequently, there has been widespread use of vague, conceptual definitions across studies, for example, see.^11
–13 As a major advance in this area, a more detailed American Congress of Rehabilitation Medicine (ACRM) consensus definition of mild TBI (mTBI) was published in 2023, although studies are needed to validate and determine how to apply the criteria.¹⁴ The lack of an evidence-based definition of TBI, and agreement on how it should be operationalized, is a major challenge that hampers the ability to evaluate and compare TBI studies and thereby advance the science and treatment of TBI.

To address gaps in evidence-based approaches to assess the diagnostic criteria of TBI, we developed a semi-structured interview for determining and documenting the clinical signs and symptoms of TBI. The TBI Diagnostic Interview (TBI-DI) collects information regarding the patient’s (and/or witness’s) report of head trauma, AOC (including LOC and amnesia), and TBI-related symptoms. Unlike other interviews developed for retrospective reporting of remote TBI histories,¹⁵ the TBI-DI was designed to assess for TBI in-hospital or shortly after hospitalization (e.g., within the first 2 weeks of injury). The interview was designed to serve as a standalone patient/informant interview to supplement clinical information and support TBI eligibility decisions in the context of multicenter prospective longitudinal studies. In this article, we detail the development of the TBI-DI, report the interrater reliability of TBI-DI items to the same audio-recorded interview, and describe the prevalence of distinct diagnostic criteria profiles in a large, prospectively recruited sample of level 1 trauma center patients who have sustained a TBI.

Materials and Methods

Participants

Adults with TBI (N = 509) were prospectively recruited into the Improving Patient Classification and Outcome Measurement in TBI (IPCOM) study from four U.S. level I trauma centers between September 14, 2019, and September 27, 2023.¹⁶ The institutional review board at the principal investigator’s institution approved the study and oversaw study activities across sites. Participants in the parent study completed informed consent (those in the phone sub-study gave verbal consent) and received financial compensation for completion of study visits. The original study design required enrolling participants in person in the hospital within 24 h of injury and conducting follow-up visits at 2 weeks, 3 months, and 6 months post-injury (parent study n = 311). Prompted by temporary restrictions on in-person research arising during the COVID-19 pandemic, the study opened a phone-based substudy that recruited participants at 2 weeks post-injury and completed abbreviated 2-week, 3-month, and 6-month outcome assessments. The present study focused on interviews conducted at 2 weeks post-injury (n = 335) because these interviews were audio recorded to enable inspection of interrater reliability (see Supplementary Fig. S1 for a recruitment and retention flow diagram).

Inclusion/exclusion criteria

Inclusion criteria for participation were age 18 years or older, adequate vision and hearing for clinical assessments, English speaking, presenting to a participating institution within 24 h of injury (2–4 weeks for the substudy), and meeting the study definition of TBI. In particular, we required participants to meet the following criteria, operationalized by a combination of chart review and TBI-DI, which is described in more detail below:

1. Head trauma or acceleration/deceleration event (e.g., whiplash)

AND

2a. Objective evidence of structural brain injury (i.e., acute intracranial findings on clinical head computed tomography [CT] scans)

2b. Any evidence of AOC (e.g., witnessed or reported unconsciousness, peritraumatic amnesia, or other signs of AMS such as feeling dazed or confused)

2c. One or more TBI-related symptoms (e.g., headache, nausea, vomiting, fatigue, dizziness, and double vision) reported soon after injury.

Because the primary aim of the study was to evaluate differing operational definitions of TBI, these criteria were developed to be consistent with an array of widely used diagnostic criteria, while maintaining a low floor to enable future data-driven approaches to evaluate varying definitions of TBI. While we anticipated that the majority of enrollments would meet a lenient interpretation of the 1993 ACRM definition of mTBI (i.e., would have either objective neuroimaging evidence of structural brain injury or at least subjective report of AOC;¹⁷), we aimed to also enroll participants with a probable mechanism of TBI and TBI symptoms-only (i.e., no evidence of structural brain injury or AOC) to align injury characteristics of the sample with common cases of community sport-related concussion.¹⁸ Although individuals could meet criterion 2 with only one mTBI symptom, it was uncommon to report no AOC and only one symptom (n = 4).

Exclusion criteria were active incarceration or under police custody; pregnancy; history of serious psychiatric; neurological disorder or other pre-existing conditions that would interfere with follow-up and outcome assessment; active alcohol dependence; current meth, cocaine, or heroin use; any past abuse of illicit drugs; low likelihood of follow-up; penetrating TBI; or concurrent spinal cord injury with American Spinal Injury Association scores of A to C. These inclusion/exclusion criteria paralleled those of the multicenter Transforming Research and Clinical Knowledge in TBI TRACK-TBI study,¹⁹ while eliminating the TRACK-TBI study requirement that a head CT was ordered by the treating physician and relaxing time constraints for enrollment for participants in the phone-based substudy. Moreover, to minimize confounds for the TBI diagnostic criteria, enrolling staff were trained to recognize and exclude non-traumatic neurological causes of AOC (e.g., syncope and seizures). They were also generally advised to exclude acute intoxication. Of 335 participants in this sample, only 7 had acute intoxication and were negative for acute intracranial signs of TBI.

Study design

Assessment of participants’ ability to complete assessments

Using procedures adapted from the TRACK-TBI study,²⁰ we used a decision tree based on a formal assessment to determine if patient participants would complete self-report clinical assessments, including the TBI-DI. In particular, to self-report individuals needed to display intelligible speech and perform adequately on an orientation assessment (the Galveston Orientation and Amnesia Test [GOAT]). Participants performing below the threshold on the GOAT needed to score in the non-confused range on the Confusion Assessment Protocol (CAP-Cog) to be deemed able to self-report. In this sample, only 10 individuals were not yet able to meet the criteria for self-reporting at the 2-week assessment (i.e., completed it via informant report).

Traumatic brain injury diagnostic interview

The TBI-DI is a semi-structured interview created to establish patient/witness perception of the clinical signs and symptoms of TBI, which may be used both to help establish diagnoses and to establish prognostic indicators (e.g., duration of AOC). Patients and/or witnesses are asked standard questions regarding TBI diagnostic criteria, including head trauma or acceleration/declaration event, LOC, PTA, retrograde amnesia (RGA), and other AMS. The interviewer asks standardized questions regarding these domains and provides follow-up questions to obtain information about the duration and existence of potential witnesses. For perceived “other AMS,” the patient is asked follow-up discrete (Yes/No) and free-response questions to determine the patient’s experience of “other AMS.” In light of the descriptive aims of the present article, we left these responses as is. Patients are also asked about TBI-related symptoms they may have experienced shortly after injury, such as headache, nausea, and balance problems. The interview form and standard operating procedure used in the study have been made publicly available (see Transparency, Rigor, and Reproducibility).

Interviewers underwent a standardized training protocol proctored by the lead site that comprised three main steps: (a) watching pre-recorded presentations explaining the study aims, inclusion/exclusion criteria, and interview process; (b) reviewing the interview SOP (e.g., study definitions of NINDS TBI CDEs such as yes, suspected, no, and unknown LOC, amnesia, and other AMS); and (c) demonstrating mastery in applying the study definitions of these codes to case examples via formal testing. The study PI or lead coordinator then held meetings with site staff to review and discuss the procedures and competency testing. Lead coordinators at each site observed enrollments as part of the check-out process for new enrolling staff. Of note, for the purposes of this research, the TBI-DI was conducted as a standalone interview. Interviewers, while not formally blinded to the medical chart, generally had limited access to the medical chart and were trained not to record information collected outside the interview on the case report form or to query participants about discrepancies between their report and other documented information. Moreover, second raters were trained to code interview variables as “unknown” if they were not adequately discussed in the interview recording.

Data curation of the TBI-DI

TBI-DIs conducted at 2 weeks post-injury by eight different interviewers were audio recorded when possible to facilitate data curation. Another research assistant (author A.M.) completed an independent second rating of all interviews. Of the 335 interviews that were conducted at 2 weeks, 47 did not have audio recordings, the majority (n = 36) of which were due to the interview taking place during the patient’s enrollment screening. A small number (n = 11) were missing due to administrative and technical errors. Thus, interviews of 288 participants were included in the reliability analysis. Discrepancies in codes identified between the original interviewer and the second rater were identified, and a third rater reconciled discrepancies and determined the final code to use in data analyses. Analyses of interrater reliability reported below compare codes recorded for the original interviewer to those of the second rater; other data reported reflect the final curated dataset.

Data analysis

Statistical analyses were conducted using IBM SPSS Statistics version 27, Armonk, NY;²¹ UpSet plots were produced in R Statistical Software using the UpSet R package (version 1.4.0). Demographic information and injury characteristics were summarized as means (standard deviations) or frequencies (percentages). Cohen’s kappa was calculated to summarize interrater reliability of 2-week TBI-DI audio-recorded interviews between the interviewer and second-rater. Kappa statistics were interpreted using the following guidelines: fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), and near perfect (0.81–1).²² We also calculated the total percentage agreement between raters. Reliability and agreement were computed using the n = 288 with audio-recorded interviews, whereas only the curated interviews conducted with the subject were included in these analyses for symptoms (n = 279). UpSet plots were created to illustrate the prevalence of distinct profiles of signs of AOC (full sample). Lastly, symptom reporting was tabulated through frequencies of each symptom reported, the number of symptoms reported, and the number of unique symptom profiles. Informant-reported data were excluded from analyses of symptoms due to the likely limited accuracy of an informant recounting a patient’s symptoms.

Results

Sample characteristics of the enrolled sample, 2-week interviews, and interviews included in the reliability analysis are presented in Table 1. The sample who completed the 2-week TBI-DI (n = 335) ranged from 18 to 88 years old (M [SD] = 42.7 [19.1]) and was 52% male, 58% White (34% Black, 2.5% Asian, 0.5% American Indian/Alaska Native, 5% Other/unknown race), and 6% Hispanic or Latino. The distribution of the highest level of care was 62% emergency department, 20% inpatient admission, and 18% intensive care unit. The majority of participants (93%) had admission GCS scores of 13–15. Thirty-two percent of patients had acute intracranial findings on head CT scans (CT+), 44% had none (CT−), and 24% had no head CT performed. Sample characteristics of the subsample included in the analysis were similar to the enrolled sample (Table 1).

Table 1.

Sample Characteristics, M (SD) or n (%)

	Allenrolled	2-week follow-up	Included in the reliability analysis
N	509	335	288
Age	43.4 (19.8)	42.7 (19.1)	43.2 (19.0)
Range	[18–91]	[18–88]	[18–87]
Gender
Female	239 (47%)	162 (48%)	137 (48%)
Male	258 (53%)	173 (52%)	151 (52%)
Race
Black	169 (33%)	114 (34%)	93 (33%)
White	280 (55%)	193 (58%)	171 (59%)
Asian	13 (3%)	8 (2.5%)	7 (2%)
American Indian/Alaskan Native	4 (1%)	2 (0.5%)	2 (1%)
Other/not reported	43 (8%)	18 (5%)	15 (5%)
Ethnicity
Hispanic/Latino	36 (7%)	21 (6%)	18 (6%)
Not Hispanic/Latino	454 (89%)	312 (93%)	268 (93%)
Unknown/missing	19 (4%)	2 (1%)	2 (1%)
Years of education	13.9 (2.3)	14.0 (2.4)	14.0 (2.4)
Range	[9–20]	[9–20]	[9–20]
Parent study (vs. substudy)	311 (61%)	201 (60%)	193 (67%)
Cause of injury, n (%)
Motor vehicle-traffic crash	277 (54%)	192 (57%)	162 (56%)
Fall	160 (32%)	108 (32%)	95 (33%)
Assault	25 (5%)	13 (4%)	11 (4%)
Other	47 (9%)	22 (7%)	20 (7%)
Highest level of care
Emergency department	274 (54%)	208 (62%)	177 (62%)
Inpatient floor	121 (24%)	68 (20%)	58 (20%)
Intensive care unit	105 (20%)	59 (18%)	53 (18%)
Missing	9 (2%)	0 (0%)	0 (0%)
CT (+/−)
CT+	194 (38%)	107 (32%)	96 (33%)
CT−	214 (42%)	147 (44%)	134 (47%)
Not ordered/missing	101 (20%)	81 (24%)	58 (20%)
GCS 13–15	462 (91%)	313 (93%)	269 (93%)
9–12	13 (3%)	5 (2%)	3 (1%)
3–8	29 (5%)	15 (4%)	14 (5%)
Missing	5 (1%)	2 (1%)	2 (1%)

CT, computed tomography; GCS, Glasgow Coma Scale; SD, standard deviation.

Table 2 provides interrater reliability statistics for AOC descriptors and symptom ratings on the TBI-DI. Overall, there was a near-perfect level of agreement for all AOC descriptors (κ ranging from 0.85 to 0.92). Other AMS descriptors demonstrated a similarly near-perfect level of agreement (κ ranging from 0.92 to 0.97), with the exception of the “other AMS-Free Response” (reflecting participants’ unprompted report beyond the aforementioned discrete options provided; κ = 0.75), which still demonstrates a substantial level of agreement. Lastly, symptom items demonstrated near-perfect agreement (κ ranging from 0.92 to 0.99). Full cross-tabulation tables can be found in Supplementary Tables S1, S2, and S3. Disagreement between the first and second rater was most commonly related to whether an acute injury characteristic was witnessed or suspected (i.e., unwitnessed).

Table 2.

Interrater Reliability and Total Agreement Between Raters (N = 288)

	Kappa	Total agreement
Loss of consciousness^a	0.92	94%
Post-traumatic amnesia^a	0.87	92%
Retrograde amnesia^a	0.85	93%
Other altered mental status (AMS)^a	0.92	95%
Discrete choices (Y/N)
Confused	0.96	98%
Dazed	0.97	98%
Stunned	0.95	97%
Saw Stars	0.92	97%
Felt like a bell rung	0.95	98%
Additional free responses (Y/N)	0.75	92%
Symptoms^b
Headache	0.98	99%
Nausea	0.98	99%
Vomiting	0.96	99%
Dizziness	0.98	99%
Fatigue	0.99	99%
Blurred vision	0.95	97%
Double vision	0.99	99%
Drowsiness	0.97	99%
Memory difficulties	0.94	97%
Difficulty sleeping	0.92	96%
Difficulty concentrating	0.96	98%
Balance problems	0.96	98%
Sensitivity to light	0.99	99%
Sensitivity to noise	0.96	98%

The subsample with an audio recording (n = 288) was included in the reliability analysis. Additionally, we restricted the analysis of symptoms to interviews performed with patient participants (n = 279).

Coded as Yes, Suspected, No, Unknown.

Coded as Yes or No.

AMS, altered mental status.

Figure 1 highlights the response frequencies of the TBI-DI AOC descriptors of all 2-week follow-up data (n = 335). This sample most frequently experienced witnessed or suspected Other AMS (n = 284; 86%), followed by LOC (n = 165; 49%), PTA (n = 160; 48%), and RGA (n = 73; 22%). With regard to Other AMS descriptors, participants most commonly displayed or experienced being confused (n = 192; 57%), followed by dazed (n = 187; 56%), stunned (n = 175; 52%), felt like the bell was rung (n = 149; 44%), and saw stars (n = 69, 21%). Nineteen percent of the sample (n = 63) volunteered additional Other AMS descriptors when asked this question, providing responses such as feeling “in shock,” “shaken up,” “head ringing,” “ears ringing,” “out of it,” and “groggy.” Since the goals of this article were to describe responses to the interview, all responses were retained regardless of whether they appeared to clinically reflect a brain injury-related condition. Overall, 152 (45%) of patients experienced or displayed one or more witnessed signs of AOC.

FIG. 1.

Stacked bar chart of individual responses on the TBI diagnostic interview (N = 335). TBI, traumatic brain injury. Y-axis depicts number of individuals.

Figure 2 contains UpSet plots that highlight the most common patterns of acute injury characteristics and symptoms (defined as one or more TBI-related symptoms endorsed). Figure 2a combines witnessed and suspected as a positive indicator of LOC, PTA, and RGA; Figure 2b restricts a positive indication to witnessed only. When witnessed/suspected were combined, the most common profiles were Other AMS and symptoms (n = 84, 26%); followed by (b) LOC, PTA, Other AMS, and symptoms (n = 57, 18%); LOC, Other AMS, and symptoms (n = 42, 13%); PTA, Other AMS, and symptoms (n = 32, 10%), and all AOC descriptors and symptoms (n = 31, 10%). When injury characteristics were restricted to witness, the majority of participants had symptoms only (n = 168, 52%), followed by Other AMS and symptoms (n = 27, 8%), LOC and symptoms (n = 26, 8%), and PTA, Other AMS, LOC, and symptoms (n = 24, 7%). The most common symptoms reported in this sample were headache (n = 249; 77%), dizziness (n = 212; 65%), fatigue (n = 183, 56%), and difficulty concentrating (n = 171, 53%); the least common symptoms reported were double vision (n = 48; 15%) and vomiting (n = 47; 14%; Table 3). Overall, the median number of symptoms endorsed was 6/14 (IQR = 3, 9; see Table 4 for descriptive statistics on the number of symptoms endorsed). Investigation into profiles of symptom reporting found 256 unique symptom profile combinations among the 325 patient participants.

FIG. 2.

Upset plot of the combinations of acute injury characteristics (LOC, PTA, RGA, other AMS, symptom reporting). Top: Considers both witnessed and unwitnessed/suspected characteristics to be present. A total of n = 325 (n = 10 informant interviews excluded) patients were included in the analysis. Bottom: Considers only witnessed characteristics to be present. Given the restriction to witnessed only, N = 11 patients did not have any acute injury characteristics or symptoms and thus were not included in the figure. Percentages reported are based on the n = 325 sample. Of note, when a patient experienced a gap in memory that was unwitnessed, the interview SOP provided consistent scoring criteria (e.g., unknown LOC, suspected PTA; see SOP linked from the Transparency, Rigor, and Reproducibility section, table at bottom of page 2). AMS, altered mental status; LOC, loss of consciousness; PTA, post-traumatic amnesia; RGA, retrograde amnesia.

Table 3.

Frequencies and Percentages of TBI Symptoms Endorsed on the TBI-DI

	N (%)
Headache	249 (77%)
Dizziness	212 (65%)
Fatigue	183 (56%)
Difficulty concentrating	171 (53%)
Difficulty sleeping	165 (51%)
Drowsiness	158 (49%)
Memory difficulties	153 (47%)
Balance problems	146 (45%)
Sensitivity to light	142 (44%)
Nausea	129 (40%)
Blurred vision	118 (36%)
Sensitivity to noise	116 (35%)
Double vision	48 (15%)
Vomiting	47 (14%)

A total of N = 325 patients reported 256 different combinations of the 14 symptoms (N = 10 are excluded from this analysis due to their interview being completed by an informant).

Table 4.

Frequencies (Percentages) and Cumulative Percentage of Number of Symptoms Endorsed on the TBI-DI

Number of symptoms	Frequency (%)	Cumulative percentage
0 symptom	14 (4%)	4%
1 symptom	23 (7%)	11%
2 symptoms	23 (7%)	18%
3 symptoms	23 (7%)	26%
4 symptoms	25 (8%)	33%
5 symptoms	31 (10%)	43%
6 symptoms	33 (10%)	53%
7 symptoms	32 (10%)	63%
8 symptoms	26 (8%)	71%
9 symptoms	29 (9%)	80%
10 symptoms	17 (5%)	85%
11 symptoms	18 (6%)	91%
12 symptoms	16 (5%)	96%
13 symptoms	11 (3%)	99%
14 symptoms	3 (1%)	100%

A total of N = 325 patients were included in this analysis (N = 10 are excluded due to their interview being completed by an informant).

TBI-DI, Traumatic Brain Injury Diagnostic Interview.

Discussion

TBI is a heterogeneous injury with multiple definitions and diagnostic standards; therefore, it is critical to standardize the way TBI is assessed and defined in both research and clinical settings. This research substantially adds to standardization efforts in TBI research by creating the semi-structured TBI-DI based on NINDS CDEs, providing operational definitions for applying the CDEs to the interview, documenting training procedures, and demonstrating strong interrater reliability of audio-recorded interview items. Second, our investigation of the prevalence of AOCs and symptoms assessed by the TBI-DI provides descriptive information regarding patients’ perception of their acute post-head trauma experience at 2 weeks post-injury. We opted to keep the inclusion criteria for the TBI-DI very broad to enable research into diverse profiles that could potentially be consistent with TBI. However, one could use the framework of the TBI-DI while applying more stringent thresholds for diagnosis, in accordance with other consensus definitions.^3,14

The TBI-DI is, to our knowledge, the first published structured interview designed to inform whether or not patients meet the criteria for head injury mechanism, acute signs, and acute symptoms of TBI based on their report or an informant report, during the acute and subacute post-injury period, and in a civilian trauma population. There have been several TBI screenings and DIs created to assess the history of TBI in military veterans, including the Veterans’ Affairs (VA) TBI Identification Interview,²³ Virginia Commonwealth University Retrospective Concussion Diagnostic Interview-Blast Version VCU RCDI-B,²⁴ VA Traumatic Brain Injury Screening Tool VATBIST,²⁵ and the Boston Assessment of Traumatic Brain Injury-Lifetime BAT-L.²⁶ Each interview exhibits unique features; for example, the VA TBI Identification Interview primarily relies on free recall and thus avoids non-shared terminology and may minimize over/under reporting. The VCU RCDI-B was designed to specifically assess exposure to military blasts. The VATBIST includes ratings of injury probability and severity, and the BAT-L includes injuries sustained before, during, and after military service. While each of these interviews has been investigated in military populations, their function in civilian populations is unknown and their purpose in establishing lifetime/remote injury histories is not ideally suited to establishing whether a single recent injury event qualifies one for a prospective study of TBI.

The most widely investigated diagnostic interview used in civilian populations, the Ohio State University TBI Identification Method OSU-TBI-ID,¹⁵ has become the gold standard for retrospective lifetime assessment to TBI history in civilian studies.^27
–29 Although the OSU-TBI-ID assesses lifetime history of TBI, the TBI-DI fills an important gap in the literature by providing a standardized measure that is completed in the post-injury period and assesses multiple potential signs of altered consciosusness and symptoms of acute TBI. By collecting data on a wide range of possible signs and symptoms of TBI, the TBI-DI provides a foundation from which to develop a strong evidence base for which self- and informant-reported signs and symptoms are most and least useful to detect TBI. The creation of this interview paves the way for clarity in operationally defining TBI, thus allowing for transparent research protocols and higher replicability of research.

Our TBI-DI underwent a rigorous curation process in which two raters independently rated a single participant interview; any differences found were reconciled by a third rater. We found near-perfect agreement for all discrete variables (κ ranging from 0.85 to 0.99) between the first and second raters. Strong interrater reliability of TBI-DI items demonstrates the standardization of the TBI-DI and training protocol, which allows researchers to employ multiple interviewers to conduct the TBI-DI to determine research eligibility in prospective studies. However, it is not yet known whether participants would report their histories consistently over time and how self- and informant-reported TBI indicators align with clinical or objective evidence of TBI. Future research should investigate additional forms of reliability with the TBI-DI, including the stability of participant responses over time and the concordance between participant reports and clinical information, such as clinician ratings.

The main objective of the TBI-DI was to assess for head trauma, signs of AOC, and TBI-related symptoms in a standardized fashion consistent with the NINDS TBI CDE. Since the development of the interview, expert consensus criteria for mTBI have been published that emphasize the importance of establishing that any signs and symptoms of TBI are not better accounted for by confounding factors, such as polytrauma, emotional reactions to trauma, or other medical comorbidities.¹⁴ Although this important step to diagnosis was not embedded into the TBI-DI, it was partially addressed in other ways in the study. First, examiners were trained to recognize and exclude for clinical suspicion that non-traumatic neurological issues (e.g., syncope, seizures) caused any peritraumatic AOC. Second, data on other confounds (e.g., peripheral injuries, non-accidental injury) were recorded where possible to enable future investigation of their contributions to participants’ clinical profiles. Because the literature on how to disentangle TBI from confounding factors is still maturing, there is a need for further research into how to best identify TBI in the presence of potentially confounding factors, as well as guidance on what confounding factors can be assessed by clinical interviews versus other data sources.

Analyzing the frequencies and patterns of patient responses in this interview highlights the ambiguity of TBI diagnosis, as the majority of patients in this sample did not demonstrate witnessed, observable signs of TBI. Most patients (86%) experienced “other” signs of AMS such as feeling “stunned” or “confused.” While these signs of AMS can be due to a head injury, they can also be indicative of shock and dissociation experienced following a psychological trauma.³⁰ Given the context surrounding TBI (e.g., motor vehicle collision), it can be difficult to decipher if signs of AMS such as “stunned” or “confused” are related to psychological trauma, TBI, or both, which may lead to misdiagnosis.³¹ Moreover, only 45% of the sample had one or more witnessed signs of AOC. The UpSet plots examining the patterns of AOC and symptom reporting demonstrated that profiles differ substantially when signs of AOC require a witness (e.g., a symptom-only profile shifts from 4% of the sample to 52%). In addition, UpSet plots further revealed that RGA never occurred without PTA; this builds upon other findings of this pattern and supports assertions by the authors of the 2023 ACRM consensus definition that RGA occurring in the absence of PTA might signify another diagnosis.^14,32,33

Differences in how various TBI diagnostic interviews label signs of TBI such as “stunned” or “confused” further demonstrate ambiguity in diagnosis and diagnostic terminology. For example, the VCU TBI Identification Interview includes “dazed,” “confused,” and “saw stars” as symptoms of TBI, whereas the VATBIST, BAT-L, and our novel TBI-DI include these as signs of AMS. Notably, the 2023 ACRM definition of TBI includes these descriptors under criterion 3 “symptoms” and describes them as “acute subjective alteration in mental status.” Moreover, DIs vary in whether they specify if an AOC is witnessed or unwitnessed; the VCU RCDI-B, VATBIST, and our TBI-DI specify if an AOC is witnessed or suspected; however, the OSU-TBI-ID and VCU TBI Identification Interview do not. These differences highlight that 1) it is important for researchers to be clear about the criteria used to diagnose TBI in their sample and 2) there is a continued need to develop evidence-based standards for the clinical diagnosis of TBI. We expect the TBI-DI data collected in this study, when paired with other clinical and objective biomarker data, will be useful to improve the empirical basis for revisions to consensus definitions of TBI diagnostic criteria and the guidance for how to operationalize definitions in research and clinical settings. For example, although for our study the TBI-DI was treated as a standalone interview independent of other data sources, in clinical practice, it would be useful to combine information from multiple sources or to use the TBI-DI to try to resolve any ambiguity in the medical record about one’s injury. An expert survey, implemented to help develop the new ACRM definition of mTBI, found that most experts agree that objective signs are more important for diagnosis than subjective signs; however, subjective signs of TBI should be captured by the diagnostic criteria (e.g., descriptors of AMS), as well as accompanying symptoms.⁵ The experts rated specific symptoms as differing in importance for diagnosis; for example, “confused,” “disoriented,” “dazed,” “balance problems,” “dizziness,” and “difficulty remembering” were widely regarded as important to determine TBI diagnosis, whereas other symptoms such as “pressure in head” and “don’t feel right” were widely seen as less helpful to diagnose TBI.⁵

In developing the TBI-DI, we took a relatively neutral approach aiming for a detailed collection of both witnessed and unwitnessed AOCs, multiple signs potentially consistent with AMS, and multiple symptoms of TBI (including the symptoms widely regarded as important above with the exception of “disoriented”) using either patient or informant report. While the new criteria put forth by the ACRM demonstrate significant progress toward unified, evidence-based criteria for mTBI, studies are needed to validate and standardize data collection. Therefore, the broad data collected through the TBI-DI might be useful to empirically validate or evolve expert-consensus-based conceptions of what signs and symptoms warrant a TBI diagnosis. In the majority of cases of TBI where objective injury signs are not available (e.g., CT−), the TBI-DI could provide substantial standardized information to support diagnostic decisions and to grade injury severity (e.g., estimates of LOC/PTA duration). In more severe injuries, where diagnosis is more certain from available clinical information, collecting the TBI-DI through informant reports could potentially provide additional information to add to estimates of injury severity, particularly in settings where such information is not well documented in medical charts. On the contrary, there are limitations to collecting the TBI-DI through informant reports, including the inability for informants to report on patient symptoms and the possibility that estimates (e.g., of PTA) duration based on family or other reports could diverge from objective assessments of PTA (e.g., serial GOAT or Westmead PTA Scale).³⁴

The TBI-DI is a novel diagnostic interview that demonstrated strong interrater reliability of audio-recorded interview items; however, there are some limitations to note. Interrater reliability was assessed by both raters listening to the same diagnostic interview, rather than each rater conducting their own diagnostic interview, which removes potential variance attributable to differences in interview technique and instability in participant report over time. Moreover, acute post-injury interrater reliability was unable to be assessed due to a lack of audio recordings for these interviews, which is a limitation given that the acute post-injury timepoint is potentially the most ideal time to gather self-reported TBI diagnostic information. Future research should continue to assess the reliability and validity of this interview across populations, time points, and administration methods to compile more evidence regarding the utility of this interview. Interrater reliability was investigated at the item level; therefore, the overall interrater reliability of the TBI-DI’s ability to determine whether TBI is present or absent has not yet been investigated. Future research should investigate the utility of using the TBI-DI as a tool to diagnose or rule out a TBI. Lastly, we applied exclusion criteria to minimize signs and symptoms of TBI that may be explained by confounding factors (e.g., history of psychiatric or neurological disorder, intoxication), yet this limits the generalizability of the TBI-DI to these populations. This was a necessary first step to create and validate the TBI-DI; however, future research should investigate the utility of the TBI-DI in these populations once the interview has demonstrated adequate reliability and validity in non-confounded samples.

Conclusions

In summary, we presented the protocol, interrater reliability of TBI-DI items, and clinical profiles of the novel TBI-DI intended to rigorously collect self- or informant-reported signs and symptoms pertinent to making TBI diagnostic decisions and supporting prognosis for prospective research studies. The findings of high interrater reliability support the feasibility of reliably collecting interview-based TBI diagnostic information at 2 weeks post-injury in a multicenter study. The TBI-DI provides a model to encourage more standardized and generalizable collection and reporting of TBI diagnostic information in other research studies, while the detailed data collected through the TBI-DI set the stage for additional work to build empirically supported diagnostic criteria for TBI.

Transparency, rigor, and reproducibility

The study was not pre-registered as it was not required by the funder or other parties. Developing the diagnostic interview was an objective of the grant that funded this work, in order to rigorously document the manner in which individuals met inclusion criteria. However, the analyses reported in this article were not explicitly planned at the time of study design. The initial grant aimed to recruit 350 individuals within 24 h of injury and expected 20% attrition at each follow-up timepoint. The study was powered based on hypothesized latent variable models of data not pertinent to this article. Because of our decision in 2020 to open a phone-based substudy—which was more efficient in design than the parent study—final enrollment exceeded that initially planned (N = 509 recruited between September 14, 2019, and September 27, 2023). A recruitment and retention flowchart is provided in the online (Supplementary Figure S1). Second ratings of audio-recorded interviews were performed by individuals who were blind to the ratings of the initial interviewer. The TBI-DI case report form and study manual are available at the following website: https://www.mcw.edu/departments/neurosurgery/research/nelson-lab/. For other inquiries about the data or methods, please contact the corresponding author.

Footnotes

Acknowledgments

The study was funded by the National Institute for Neurological Disorders and Stroke (NINDS) grant # R01 NS110856. The REDCap database used for the study was supported by The Clinical and Translational Science Institute (NIH grant #2UL1TR001436). The first author’s time was supported by the Medical College of Wisconsin Department of Neurosurgery. The article’s contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. The authors declare no further conflicts of interest with this work.

Authors’ Contributions

M.U.S.: Conceptualization, methodology, formal analysis, and writing—original draft; D.L.H.: Data curation, formal analysis, visualization, and writing—review and editing; J.D.C.: Conceptualization and writing—review and editing; N.T.: Conceptualization, methodology, and writing—review and editing; M.D.: Project administration and writing—review and editing; R.K.: Project administration and writing—review and editing; J.W.: Supervision and writing—review and editing; J.T.G.: Supervision and writing—review and editing; M.B.S.: Supervision and writing—review and editing; G.T.M.: Supervision and writing—review and editing; M.A.M.: Supervision and writing—review and editing; L.D.N.: Conceptualization, funding acquisition, methodology, project administration, data curation, formal analysis, and writing—original draft.

Author Disclosure Statement

The authors have no direct conflicts of interest with the submitted work. L.D.N., N.T., R.K., and M.D. received salary support from the National Institute of Neurology Disorders and Stroke grant # R01 NS110856 for the submitted work. L.D.N. also recently received support for other research from the U.S. Department of Defense, Medical College of Wisconsin Advancing a Healthier Wisconsin Endowment, and U.S. Centers for Disease Control and Prevention; we well as consulting stipends for unrelated work from the American College of Emergency Physicians (through a grant from Abbott) and Resolys Bio, Inc. Dr. Temkin was supported by NINDS grant# R01 NS110856 and Department of Defense award W81XWH-15-9-001. She also receives funding for other work from the US federal government. Dr. Corrigan was supported by a grant from the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR; 90DPTB0001). NIDILRR is a Center within the Administration for Community Living (ACL), Department of Health and Human Services (HHS). The contents of this publication are solely the responsibility of the authors and do not necessarily represent the policy or official views of NIDILRR, ACL, or HHS, and you should not assume endorsement by the Federal Government. J.T.G. receives funding support from the National Institute on Neurological Disorders and Stroke, the National Institute on Disability, Independent Living, and Rehabilitation Research, the Department of Defense, the SameYou Foundation, and the Barbara Epstein Foundation. He occasionally receives honoraria for conducting training seminars on severe TBI and disorders of consciousness for academic medical centers and institutes of higher education. M.A.M. receives research funding to the Medical College of Wisconsin from the National Institutes of Health, Department of Veterans Affairs, Centers for Disease Control and Prevention, Department of Defense, National Collegiate Athletic Association, National Football League, and Abbott Laboratories. He receives book royalties from Oxford University Press. He serves as a clinical consultant to Milwaukee Bucks, Milwaukee Brewers, and Green Bay Packers and is Co-Director of the NFL Neuropsychology Consultants without compensation. He receives travel support and speaker honorariums for professional activities.

Funding Information

The study was funded by National Institutes of Neurological Disorders and Stroke grant # R01 NS110856. The article’s contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health.

Supplementary Material

Supplementary Figure S1

Supplementary Table S1

Supplementary Table S2

Supplementary Table S3

References

GBD 2016 Traumatic Brain and Spinal Cord Injury Collaborators. Global, regional, and national burden of traumatic brain injury and spinal cord injury, 1990–2016: A systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol, 2019; 18(1):56–87.

Taylor

, Bell

, Breiding

, et al. Traumatic brain injury-related emergency department visits, hospitalizations, and deaths - United States, 2007 and 2013. MMWR Surveill Summ, 2017; 66(9):1–16.

Mayer

, Quinn

, Master

. The spectrum of mild traumatic brain injury: A review. Neurology, 2017; 89(6):623–632.

Teasdale

, Maas

, Lecky

, et al. The glasgow coma scale at 40 years: Standing the test of time. Lancet Neurol, 2014; 13(8):844–854.

Silverberg

, Iverson

, Arciniegas

, et al.; Group AMTDEC. Expert panel survey to update the American Congress of Rehabilitation Medicine definition of mild traumatic brain injury. Arch Phys Med Rehabil, 2021; 102(1):76–86.

Hicks

, Giacino

, Harrison-Felix

, et al. Progress in developing common data elements for traumatic brain injury research: Version two–the end of the beginning. J Neurotrauma, 2013; 30(22):1852–1861.

Maas

, Harrison-Felix

, Menon

, et al. Common data elements for traumatic brain injury: Recommendations from the interagency working group on demographics and clinical assessment. Arch Phys Med Rehabil, 2010; 91(11):1641–1649.

Kutcher

, Giza

. Sports concussion diagnosis and management. Continuum (Minneap Minn), 2014; 20(6 Sports Neurology):1552–1569.

Daugherty

, Peterson

, Waltzman

, et al. Rationale for the development of a traumatic brain injury case definition for the pilot national concussion surveillance system. J Head Trauma Rehabil, 2024; 39(2):115–120.

10.

Daugherty

, Waltzman

, Breiding

, et al. Refinement of a preliminary case definition for use in traumatic brain injury surveillance. J Head Trauma Rehabil, 2024; 39(2):121–139.

11.

Carroll

, Outtrim

, Forsyth

, et al. Mild traumatic brain injury recovery: A growth curve modelling analysis over 2 years. J Neurol, 2020; 267(11):3223–3234.

12.

Mikolic

, Polinder

, Steyerberg

, et al. Prediction of global functional outcome and post-concussive symptoms after mild traumatic brain injury: External validation of prognostic models in the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) study. J Neurotrauma, 2021; 38(2):196–209.

13.

Stenberg

, Haberg

, Follestad

, et al. Cognitive reserve moderates cognitive outcome after mild traumatic brain injury. Arch Phys Med Rehabil, 2020; 101(1):72–80.

14.

Silverberg

, Iverson

, Cogan

, et al.; members ABISIGMTTF. The American congress of rehabilitation medicine diagnostic criteria for mild traumatic brain injury. Arch Phys Med Rehabil, 2023; 104(8):1343–1355.

15.

Corrigan

, Bogner

. Initial reliability and validity of the Ohio State University TBI Identification Method. J Head Trauma Rehabil, 2007; 22(6):318–329.

16.

Kinney

, Bahrani

, Brenner

, et al. Relationship between neighborhood disadvantage and mild traumatic brain injury symptoms. J Head Trauma Rehabil, 2023; 2(38):165–174.

17.

Kay

. Neuropsychological treatment of mild traumatic brain injury. The Journal of Head Trauma Rehabilitation, 1993; 8(3):74–85.

18.

Patricios

, Schneider

, Dvorak

, et al. Consensus statement on concussion in sport: The 6th International Conference on Concussion in Sport-Amsterdam, October 2022. Br J Sports Med, 2023; 57(11):695–711.

19.

Nelson

, Temkin

, Dikmen

, et al.; and the TRACK-TBI Investigators. Recovery after mild traumatic brain injury in patients presenting to US level I trauma centers: A Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACK-TBI) study. JAMA Neurol, 2019; 76(9):1049–1059.

20.

Bodien

, Barber

, Taylor

, et al. Feasibility and utility of a flexible outcome assessment battery for longitudinal traumatic brain injury research: A TRACK-TBI study. J Neurotrauma, 2023; 40(3–4):337–348.

21.

Corp I. IBM SPSS Statistics for Windows 27.0 ed: IBM CORP. 2020.

22.

McHugh

. Interrater reliability: The kappa statistic. Biochem Med (Zagreb), 2012; 22(3):276–282.

23.

Vanderploeg

, Groer

, Belanger

. Initial developmental process of a VA semistructured clinical interview for TBI identification. J Rehabil Res Dev, 2012; 49(4):545–556.

24.

Walker

, Cifu

, Hudak

, et al. Structured interview for mild traumatic brain injury after military blast: Inter-rater agreement and development of diagnostic algorithm. J Neurotrauma, 2015; 32(7):464–473.

25.

Donnelly

, Donnelly

, Dunnam

, et al. Reliability, sensitivity, and specificity of the VA traumatic brain injury screening tool. J Head Trauma Rehabil, 2011; 26(6):439–453.

26.

Fortier

, Beck

, Werner

, et al. The Boston assessment of traumatic brain injury-lifetime semistructured interview for aassessment of TBI and subconcussive injury among female survivors of intimate partner violence: Evidence of research utility and validity. J Head Trauma Rehabil, 2022; 37(3):E175–E185.

27.

Gardner

, Rivera

, O’Grady

, et al. Screening for lifetime history of traumatic brain injury among older American and Irish adults at risk for dementia: Development and validation of a web-based survey. J Alzheimer’s Disease, 2020; 74(2):699–711.

28.

Schneider-Cline

, Bush

, McKelvey

. Using the OSU TBI-ID method for screening rural, older adults: A mixed methods feasibility study. Brain Inj, 2019; 33(7):899–915.

29.

Smith

, Rothschild

. Use of the Ohio state university TBI identification method (OSU-TBI) in community settings. Arch Phys Med Rehabil, 2023; 104(3):e60–e61.

30.

Schauer

, Elbert

. Dissociation following traumatic stress: Etiology and treatment. Zeitschrift Für Psychologie/Journal of Psychology, 2010; 218(2):109–127.

31.

Rosen

, Ayers

. An update on the complexity and importance of accurately diagnosing post-traumatic stress disorder and comorbid traumatic brain injury. Neurosci Insights, 2020; 15:2633105520907895; doi: 10.1177/2633105520907895

32.

Paniak

, MacDonald

, Toller-Lobe

, et al. A preliminary normative profile of mild traumatic brain injury diagnostic criteria. J Clin Exp Neuropsychol, 1998; 20(6):852–855.

33.

Luoto

, Iverson

, Losoi

, et al. Clinical correlates of retrograde amnesia in mild traumatic brain injury. Brain Inj, 2015; 29(5):565–572.

34.

Sherer

, Sander

, Maestas

, et al. Accuracy of self-reported length of coma and posttraumatic amnesia in persons with medically verified traumatic brain injury. Arch Phys Med Rehabil, 2015; 96(4):652–658.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB

0.02 MB