Ability of Serum Glial Fibrillary Acidic Protein,Ubiquitin C-Terminal Hydrolase-L1,and S100B To Differentiate Normal and Abnormal Head Computed Tomography Findings in Patients with Suspected Mild or Moderate Traumatic Brain Injury

Abstract

Head computed tomography (CT) imaging is still a commonly obtained diagnostic test for patients with minor head injury despite availability of clinical decision rules to guide imaging use and recommendations to reduce radiation exposure resulting from unnecessary imaging. This prospective multicenter observational study of 251 patients with suspected mild to moderate traumatic brain injury (TBI) evaluated three serum biomarkers' (glial fibrillary acidic protein [GFAP], ubiquitin C-terminal hydrolase-L1 [UCH-L1] and S100B measured within 6 h of injury) ability to differentiate CT negative and CT positive findings. Of the 251 patients, 60.2% were male and 225 (89.6%) had a presenting Glasgow Coma Scale score of 15. A positive head CT (intracranial injury) was found in 36 (14.3%). UCH-L1 was 100% sensitive and 39% specific at a cutoff value >40 pg/mL. To retain 100% sensitivity, GFAP was 0% specific (cutoff value 0 pg/mL) and S100B had a specificity of only 2% (cutoff value 30 pg/mL). All three biomarkers had similar values for areas under the receiver operator characteristic curve: 0.79 (95% confidence interval; 0.70–0.88) for GFAP, 0.80 (0.71–0.89) for UCH-L1, and 0.75 (0.65–0.85) for S100B. Neither GFAP nor UCH-L1 curve values differed significantly from S100B (p = 0.21 and p = 0.77, respectively). In our patient cohort, UCH-L1 outperformed GFAP and S100B when the goal was to reduce CT use without sacrificing sensitivity. UCH-L1 values <40 pg/mL could potentially have aided in eliminating 83 of the 215 negative CT scans. These results require replication in other studies before the test is used in actual clinical practice.

Introduction

Computed tomography (CT) is a frequently used diagnostic test for patients with mild and moderate traumatic brain injury (TBI), but it is considered to be unnecessary for many patients.^1,2 Overuse of CT for evaluating patients with mild TBI (mTBI) has continued despite the availability of clinical decision rules,^3,4 increasing awareness of potential hazards of ionizing radiation exposure,^5,6 and some professional society's recommendations to reduce CT imaging.⁷ This has led to a search for an easily obtained and objective test that can, in the early post-injury period and with high sensitivity, predict which patients with mTBI will have an acute intracranial lesion found on head CT. A highly sensitive test, even with only moderate specificity, would provide clinicians with a new tool that would be useful in reducing CT use for patients with suspected mTBI.

Over the last two decades, there has been increasing interest in several serum proteins that could potentially predict the presence of a brain injury from head trauma. These proteins are released as a result of blunt or rapid deceleration forces to the head that cause injury to the neurons and supporting glial cells. The most widely studied biomarker for this purpose is S100B,^8

–15 a protein that is released after astroglial injury.¹⁶ It has been suggested that a S100B biomarker level below a threshold level can safely eliminate the need to obtain a CT scan in patients with mTBI.¹⁷

The Scandinavian Neurotrauma Committee 2013 guidelines for the management of minimal, mild, and moderate head injuries in adults included the use of S100B as a blood sample derived screening test for CT-proven traumatic intracranial hemorrhage (ICH).¹⁷ In the United States, however, this has not yet been widely accepted and has not seen broad clinical use. The American College of Emergency Physicians (ACEP) clinical policy for neuroimaging and decision making for adults with mTBI in 2008 gave a “Level C” recommendation for the use of S100B as a screening tool to assess the need for CT imaging in patients with mTBI and no other extra-cranial injuries,⁷ but the 2012 Eastern Association for the Surgery of Trauma guidelines stated that there was not enough evidence to warrant reliance on S100B to determine the need for head CT.¹⁸

More recently, other biomarkers have begun to receive more attention for a potential role in detecting brain injury. These include glial fibrillary acidic protein (GFAP) and ubiquitin carboxyl-terminal hydrolase-L1 (UCH-L1). GFAP is a protein expressed in astroglia cells, but unlike S100B, GFAP is not found in significant amounts in extracerebral cells.¹⁶ Previous studies have demonstrated that elevated levels of GFAP in patients with mTBI are associated with abnormal findings on imaging of the brain,¹² can predict the need for surgical intervention,¹⁹ and are able to differentiate between patients in motor vehicle crashes with orthopedic injuries and those who had mTBI.¹⁹ McMahon and associates²⁰ recently reported that elevated levels of GFAP measured within the first 24 h after injury could help reduce unnecessary CT scans without sacrificing sensitivity in cohorts including patients with mild, moderate, and severe TBI and may outperform S100B for the detection of intracranial lesions on CT for patients with mild TBI.¹³

UCH-L1 is a deubiquitinase present in neurons.^14,21 Elevated serum levels of this protein have been correlated with brain injury and clinical outcome.²² There is some evidence for GFAP but only little evidence for UCH-L1 regarding the early diagnostic utility of these biomarkers in patients with mTBI, and these studies include only small numbers of patients with head CT positive acute intracranial lesions.²³

Given the limitations of S100B as an ideal single biomarker for pre-CT screening of patients with mTBI, it was postulated that other biomarkers, such as GFAP and UCH-L1, could be useful in screening for acute intracranial lesions among patients with mTBI. Therefore, the goal of this pilot study was to conduct the first evaluation of the early (within 6 h of injury) diagnostic characteristics of GFAP and UCH-L1 to exclude the presence of acute intracranial lesions on head CT in adult patients with mTBI. We also compared the diagnostic characteristics of UCH-L1 and GFAP with S100B. These results will be used to derive potential threshold values for screening patients with mTBI to determine the need for a head CT scan.

Methods

Patients and study procedures

This prospective multicenter observational study included patients 18–80 years of age who were evaluated and treated at one of seven study site hospital emergency departments (EDs) for a blunt closed head injury and potential mild to moderate TBI. The hospitals were composed of Level 1 and 2 trauma centers, a nontrauma center, and included both U.S. and European sites (Table 1). Eligible patients were those with an initial Glasgow Coma Scale (GCS) score of 9–15 who underwent emergency head CT scan for evaluation of the head injury as deemed necessary by the attending ED physician.

Table 1.

Description of Study Site Hospitals

Study site	Location	Annual ED volume	Designation
Washington University, Barnes-Jewish Hospital	St. Louis, MO	95,000	Level I
University of Florida, Shands Hospital	Gainesville, FL	66,000	Level I
Gwinnett Medical Center–Lawrenceville	Atlanta, GA	100,000	Level II
Dekalb Medical–North Decatur	Atlanta, GA	92,000	Non-trauma
Wayne State–Detroit Receiving Hospital	Detroit, MI	90,000	Level I
University of Pécs Medical Center	Pecs, Hungary	25,000^Ω	Level 1^*
Albert Szent-Györgyi Medical Center–University of Szeged	Szeged, Hungary	90,000	Level 1^*

ED, emergency department.

The ED volume is >25,000/year, and the facility cares for more than 2000 neurotrauma cases/year.

This is equivalent to a Level 1 facility in the United States.

No formal rules for obtaining a CT were used so as to best reflect current practice in the United States or Europe, but all sites were aware of and considered available clinical decision rules to guide the need for brain imaging. Included patients presented within 4 h of injury, completed the required CT scan as part of routine care, and had blood drawn for analysis within 6 h of injury. Table 2 details the full inclusion and exclusion criteria.

Table 2.

Inclusion/Exclusion Criteria

Inclusion criteria

• The subject was ≥18 years of age and no more than 80 years of age.

• Acceleration or deceleration closed injury to the head that was either self-reported or witnessed.

• Presented to an emergency department (ED) within 4 h of injury.

• An initial Glasgow Coma Scale score of 9–15 in the ED performed by the Principal Investigator (PI) or trained study personnel.

• ED workup included a head computed tomography (CT) scan (based on standard practice and/or decision rules).

• Informed consent was obtained from the subject or his or her legal representative; oral consent for the initial blood draw and/or deferred consent to 24 h was allowed for patients who were unable to consent at initial evaluation or exception from the informed consent requirement by use of “community consent” if approved by an Institutional Review Board.

• The PI deemed the subject to be an appropriate study candidate.

Exclusion criteria

• Participation in another clinical study that may affect the results of either study.

• Time of injury was not able to be accurately determined.

• Head CT not done as part of clinical emergency care.

• Primary diagnosis of ischemic or hemorrhagic infarct.

• Not available for 35 day follow-up visit.

• Venipuncture not feasible.

• Blood donation within 1 week of screening.

• The subject was otherwise determined medically unsuitable for study participation.

Institutional Review Board approval was obtained at each study site, and each site also obtained approval by the ethics board of the U.S. Army Medical Research and Materiel Command (USAMRMC) Office of Research Protections (ORP) Human Research Protection Office (HRPO) Department of Defense. Baseline patient data collected included demographics, medical history, substance use, GCS scores, and circumstances related to and the mechanism of injury. This was used to describe the study patient's general characteristics only.

Serum sampling and handling

Blood samples were collected at time of study enrollment and every 6 hours up to the time of discharge (either ED or hospital) or up to 24 h (maximum of five samples during index visit). Patients who were seen at follow-up (Day 35 ± 5 days) had another sample obtained when feasible. Blood samples were processed and the resulting serum was stored at −80°C and then shipped on dry ice to a central repository for storage until time of testing as per a pre-defined specimen handling procedure.

Serum analysis for GFAP, UCH-L1, and S100B

Serum samples were analyzed for UCH-L1 and GFAP concentration by the Banyan enzyme-linked immunosorbent assay at a later time by technicians blinded to clinical data and CT results. The assay was performed as follows: the test sample was added to the well of a 96-well microtiter plate that had been coated with a capture antibody specific to the antigen of interest (UCH-L1 or GFAP). Following a set incubation period during which any antigen present in the sample binds to the capture antibody, unbound material was removed by washing with buffer. Horseradish peroxidase (HRP)-conjugated detection antibody was then added to the well, binding to a second site on the antigen. Following a set incubation period, excess detection antibody was removed by washing with buffer.

A chemiluminescent substrate was then added to the well. The HRP enzyme catalyzes a specific reaction with the chemiluminescent substrate, which produces light that was detected with a 96-well plate-based luminometer. The amount of light generated is proportional to the amount of detection antibody and thus UCH-L1 or GFAP in the test sample and was compared with light generated from wells containing known antigen concentrations representing a calibration curve. Samples were tested in duplicate, and high and low positive controls were included with each plate. The lower limit of quantification (LLOQ) and lower limit of detection (LLOD) for UCH-LI are ≤30 pg/ml and ≤10 pg/mL and for GFAP ≤30 pg/ml and ≤20 pg/mL, respectively.

S100B concentrations were determined using an electrochemiluminescence immunoassay designed for in-vitro diagnostic testing (Roche, Cobas 6000). Results are given as S-S100 in μg/L with the standard normal reference intervals of 0.00–0.09 μg/L. Although various cutoff values have been proposed, for the purposes of this study, an S100B ≥0.10 μg/L (100 pg/mL) was considered to be abnormal and could indicate a traumatic abnormality on head CT.^17,24

Assay results were not available to the treating clinician and were not used to guide treatment. For this main analysis, we only considered results for the first sample collected within 6 h of injury. For all biomarkers, results were reported in picograms/milliliter (pg/mL). Because other studies used different units of measurement—we provided the units in the original referenced manuscripts and the equivalent in pg/mL for the above S100B sample measurement section and in the following results and discussion section where needed.

Head CT scans

Each subject's head CT images were reviewed by an independent committee consisting of three blinded board-certified neuroradiologists. The neuroradiologists determined whether a CT scan was positive—defined as the presence of an acute trauma-related intracranial lesion. Table 3 lists the criteria for what was deemed a positive finding. A procedure outlining the criteria and process to be followed for scoring the CT scans was developed before the reading and interpretation of any of the CT images. Two of the neuroradiologists who had no access to any other clinical or laboratory data, except subject age and sex, reviewed all of the study subjects' CT scans. Any discrepancy with respect to CT-positive or CT-negative was adjudicated by a third blinded radiologist and, in those instances, was the final interpretation. Interrater reliability between the two primary radiologists was determined using the Cohen Kappa statistic.

Table 3.

Definition of Acute Intracranial Lesion

Acute intracranial lesion is defined as any trauma induced or related finding. Acute lesions may include the following and the number of each finding:
	N
Extra-axial lesions
• Acute epidural hematoma (EPH)	3
• Acute subdural hematoma (SDH)	21
Cortical contusion	10
Ventricular compression	3
Ventricular trapping	0
Brain herniation	0
Intraventricular hemorrhage	1
Hydrocephalus	1
Subarachnoid hemorrhage	27
Petechial hemorrhagic or bland sheer injury	0
Brain edema	0
Post-traumatic ischemia	0
Intracerebral hematoma	0
Dural venous sinus injury and/or thrombosis	0

Outcomes

For this study, the primary clinical outcome of interest was the results of the head CT scan (positive/negative) among patients with the first blood sample drawn within 6 h of injury.

Data analysis and reporting

A descriptive analysis for all subjects was performed. Patient groups (CT positive and CT negative) were described using proportions and means or medians where appropriate. For data analyses, any biomarker value below the detectible limit was coded as 0 pg/mL. The exception to this was for the scatter plots that used a log linear y-axis for which a value of 0.01 pg/mL was used (0 pg/mL is not a valid value using the log scale). The Spearman correlation coefficients were calculated for each biomarker pairing. Logistic regression was used to determine area under the receiver operator characteristic (AUROC) curve for all three biomarkers independently and also for a single model with all three biomarkers collectively. Probability output from the logistic regression models and graphical displays were used to evaluate clinically relevant cutoff values for positive and negative values for GFAP, UCH-L1, and S100B.²⁵ A nonparametric approach was used to compare the correlated ROC curves without adjustments for pairwise comparisons.²⁶

Sensitivity and specificity were calculated for the primary and secondary outcomes. Because there would be little acceptance of missed injury on CT scan, our goal was to maximize sensitivity to 100% at the expense of specificity. This allowed for determination of the number of CT scans that could potentially be safely eliminated when screening using each biomarker. We also examined threshold values that would result in sensitivity ≥95% and ≥90% for comparison of specificity.²⁵ For GFAP and UCH-L1, we calculated individual sensitivity and specificity values. We also implemented a combined biomarker strategy: if either GFAP was ≥100 or UCH-L1 ≥40 (chosen based on our data for optimal UCH-L1 sensitivity), the test was considered positive. For the reporting of S100B, we used the Scandinavian recommendation of >0.10 μg/L (100 pg/mL) for patients injured within 6 h of serum sampling as the standard cutoff value.¹⁷

A sensitivity analysis (in this instance meaning varying the assumptions of the diagnostic tests) that examined patients with an initial sample obtained within 4 h (ACEP guidelines) of injury was performed to evaluate for diagnostic capability of the biomarkers at times closer to injury and times that may be more favorable to the standard S100B values.⁷ Because the cut points for UCH-L1 and GFAP were optimized to our data, we also re-examined the AUROC curves and graphical data to determine whether a different cut point value for S100B would improve the diagnostic capability of that test for our data. This was done so not to “favor” the new diagnostic markers (GFAP and UCH-L1) over S100B.

There were no formal hypotheses for this study, and it was not designed to provide a definitive assessment of the studied biomarkers' test characteristics. The study was, rather, descriptive and exploratory in nature, and the results will be used as an adjunct to help derive biomarker cutoff values for a future validation trial. The analysis included proportions with associated exact Clopper-Pearson 95% confidence intervals (CIs), means with standard deviation (SD), and medians with 25th and 75th percentiles where appropriate. The p values were only used for comparing AUROC curve data.

Recommended guidelines for reporting results of diagnostic tests were followed.^27,28 These results are not to be considered as evidence to reject a null hypothesis. All data analyses were performed using SAS^® 9.4 and R 3.1.0.

Results

The study enrolled 290 patients; 26 were initially excluded for a variety of reasons, leaving a total of 264 patients who had data for at least one blood sample and the required CT results. Excluded were 13 more subjects because the initial blood sample draw was past the 6-h time window, leaving a study sample size of 251 patients for this main analysis. Of those, only 231 subjects had an adequate sample volume for the analysis of all three markers (20 patients did not have enough blood sample volume to test S100B). Figure 1 outlines the study patient selection process.

FIG. 1.

Flow diagram describing excluded and included patients with mild to moderate traumatic brain injury. CT, computed tomography; UCH-L1, ubiquitin carboxyl-terminal hydrolase-L; GFAP, glial fibrillary acidic protein.

The mean age of all patients was 45.6 ± 18.4 years and 60.2% (95% CI; 53.8%–66.3%) were male. The main mechanisms of injury were falls and/or motor vehicle crashes. Of the 251 observations, 206 observations were classified as negative by both primary radiologists, 33 were classified as positive by both primary radiologists, and 12 received discrepant positive and negative classifications from the two primary radiologists (95% agreement; Kappa = 0.82, 95% CI 0.71–0.91). After final diagnosis, adjudicated by a third independent radiologist, 36 patients (14.3%; 95% CI 10.3%–19.3%) had a CT scan that was positive for an acute intracranial lesion. Table 3 shows the findings among patients with positive head CT scan results. Note that the number of findings add up to more than 36 because many patients had more than one CT abnormality.

All except one of the head CTs were positive for some type of blood (ICH); one patient had a nonhemorrhagic contusion. Of the 251 study patients, 225 (89.6%; 95% CI 85.2%–93.1%) had an initial GCS of 15 of whom 24 (10.7%) had a positive CT scan. Among patients with a GCS <15 (n = 26), 12 (46.2%) had a positive CT scan. Median values for all biomarkers were higher among CT positive patients. Table 4 details these and other patient characteristics.

Table 4.

Patient Characteristics

	CT negative 85.7% (total n = 215)	CT positive 14.3% (total n = 36)	All subjects (total n = 251)
Age (mean ± SD)	44.3 ± 18.1	53.7 ± 17.7	45.6 ± 18.4
Sex
Female	41.4% (89)	30.6% (11)	39.8% (100)
Male	58.6% (126)	69.4% (25)	60.2% (151)
Race
White	66.5% (143)	91.7% (33)	70.1% (176)
Black	27.4% (59)	2.8% (1)	23.9% (60)
Other	6.1% (13)	5.6% (2)	6.0% (15)
Employment status (n = 243)
Employed	47% (98)	41% (14)	46% (112)
Unemployed	23% (49)	18% (6)	23% (55)
Student	9% (19)	0% (0)	8% (19)
Homemaker	1% (3)	0% (0)	1% (3)
Retired	19% (40)	41% (14)	22% (54)
Alcohol
Yes	24.2% (52)	33.3% (12)	25.5% (64)
No	75.8% (163)	66.7% (24)	74.5% (187)
Drugs
Yes	15.4% (33)	5.6% (2)	13.9% (35)
No	84.7% (182)	94.4% (34)	86.1% (216)
Smoke
Yes	31.6% (68)	33.3% (12)	31.9% (80)
No	68.4% (147)	66.7% (24)	68.1% (171)
Mechanism of injury (n = 250)^Ω
MVC	39% (84)	17% (6)	35.9% (90)
Assault	12% (25)	11% (4)	11.6% (29)
Fall	50% (107)	75% (27)	53.4% (134)
Sports	3% (7)	0% (0)	2.8% (7)
Loss of consciousness (LOC)
Yes	63.7% (137)	75.0% (27)	65.3% (164)
No	32.6% (70)	19.4% (7)	30.7% (77)
Unknown	3.7% (8)	5.6% (2)	4.0% (10)
LOC information (n = 250)
Self-reported	57.0% (122)	36.1% (13)	53.8% (135)
Witnessed	43.0% (92)	63.9% (23)	45.8% (115)
GCS
15	93.5% (201)	66.7% (24)	89.6% (225)
14	5.1% (11)	16.7% (6)	6.8% (17)
13	0.5% (1)	11.1% (4)	2.0% (5)
9–12	0.9% (2)	5.6% (2)	1.6% (4)
GFAP (median; 25th, 75th)	7.8 (2.7, 22.1)	110.5 (20.4, 431.8)	10.3 (3.5, 37.4)
UCH-L1 (median; 25th, 75th)	56.2 (24.4, 104.3)	132.3 (98.2, 269.2)	65.8 (39.6, 125.2)
S100B (median; 25th, 75th)	100 (70, 190)	215 (160, 410)	120 (70, 230)

CT, computed tomography; SD, standard deviation; MVC, motor vehicle collision; GCS, Glasgow Coma scale; GFAP, glial fibrillary acidic protein; UCH-L1, ubiquitin carboxyl-terminal hydrolase-L.

The Spearman correlation coefficients were 0.49 for GFAP and UCH-L1, 0.37 for GFAP and S100B, and 0.77 for UCH-L1 and S100B. There were 45 subjects who had GFAP levels below detectable levels (4 had a positive CT scan), 2 patients had UCH-L1 levels below detection (none had a positive CT scan), and all S100B values obtained were in the detectible range.

Figure 2A shows scatter plots of the three biomarker results. The plots include all individual patients stratified by CT result. So the reader can better examine for potential false-negative values, only those patients with values less than or equal to 1200 pg/mL (Fig. 2B) and 400 pg/mL (Fig. 2C) for each biomarker are shown. This effectively reduces the scale on the y-axis for the corresponding biomarker also stratified by CT scan result. Figures 3A–C show similar scatter plots for all three biomarkers, but in this case stratified by the Glasgow Coma Scale score; with all values capped at 400 pg/mL for better visual resolution. For these figures, only biomarker values less than or equal to 400 pg/mL were included because the lower values are of most importance.

FIG. 2.

Scatter plots for glial fibrillary acidic protein (GFAP), ubiquitin carboxyl-terminal hydrolase-L (UCH-L1), and S100B stratified by computed tomography (CT) results. (A) All biomarker values; (B) biomarker values ≤1200 pg/mL; (C) biomarker values ≤400 pg/mL. Color image is available online at www.liebertpub.com/neu

FIG. 3.

Scatter plots for glial fibrillary acidic protein (GFAP), ubiquitin carboxyl-terminal hydrolase-L (UCH-L1), and S100B stratified by presenting Glasgow Coma Scale (GCS) score (all values ≤400 pg/mL for resolution). (A) GFAP; (B) UCH-L1; (C) S100B. CT, computed tomography. Color image is available online at www.liebertpub.com/neu

All three biomarkers had similar values for the AUROC curves: 0.75 (95% CI; 0.65–0.85) for S100B, 0.79 (0.70–0.88) for GFAP, and 0.80 (0.71–0.89) for UCH-L1. When a direct comparison was performed, neither GFAP nor UCH-L1 differed significantly from S100B (p = 0.21 and p = 0.77, respectively; see Fig. 4A–D). The slight difference in AUROC curve values for the biomarkers when calculated individually was a result of the logistic regression procedure excluding 20 patients with missing values for S100B (complete case analysis), but the results were nearly identical. We also examined the predictive values of two biomarkers compared with one marker alone and, regardless of the pairing (for example, GFAP and UCH-L1 compared with either marker alone), the AUROC curves were no more predictive than the AUROC curve of a single biomarker.

FIG. 4.

Area under the receiver operating characteristic (ROC) curve comparison for all (A) biomarkers in the model (n = 231) and for each individual marker (n = 251 for glial fibrillary acidic protein [B] and ubiquitin carboxyl-terminal hydrolase-L [C] and n = 231 for S100B [D]). Color image is available online at www.liebertpub.com/neu

Threshold values for the biomarkers were determined for our study population that provided for high sensitivity at the expense of specificity. For our primary outcome, UCH-L1 was 100% sensitive and 39% (95% CI 33%–46%) specific at a value ≥40 pg/mL (specificity was 40%, 95% CI 33%–47% when using a cutoff of 41 pg/mL; Table 5). In contrast, GFAP was 100% sensitive and 0% specific at a cutoff of 0 pg/mL, indicating that using the GFAP value associated with 100% sensitivity within 6 h of injury, the test could not reliably determine which patients had negative head CTs. At a sensitivity of 100%, S100B had a specificity of only 2%. In addition, S100B was only 91% sensitive and 44% specific at the recommended 100 pg/mL threshold. The combined biomarker strategy (both GFAP and UCH-L1) resulted in the same sensitivity and specificity estimates as that of UCH-L1 alone.

Table 5.

Sensitivity, Specificity, and 95% Confidence Intervals for Biomarkers at 6 and 4 Hours Post-Injury

	Biomarker	Threshold for a positive test (pg/mL)	Sample size	Sensitivity (95% CI)	Specificity (95% CI)
6 hours^Ω	GFAP & UCH-L1^∞	100 GFAP40 UCH-L1	251	1.00 (0.90, 1.00)	0.39 (0.33, 0.46)
4 hours^Ω	GFAP & UCH-L1^∞	100 GFAP40 UCH-L1	251	1.00 (0.89, 1.00)	0.37 (0.30, 0.44)
6 hours^Ω	GFAP^¥ (=100%)	0	251	1.00 (0.90, 1.00)	0.00 (0.00, 0.02)
	GFAP (≥ 95%)	0	251	1.00 (0.90, 1.00)	0.00 (0.00, 0.02)
	GFAP (≥90%)	0	251	1.00 (0.90, 1.00)	0.00 (0.00, 0.02)
	GFAP	15	251	0.81 (0.64, 0.92)	0.67 (0.61, 0.74)
	UCH-L1^¥ (=100%)	41	251	1.00 (0.90, 1.00)	0.40 (0.33, 0.47)
	UCH-L1 (≥ 95%)	44	251	0.97 (0.85, 1.00)	0.42 (0.36, 0.49)
	UCH-L1 (≥ 90%)	67	251	0.92 (0.78, 0.98)	0.58 (0.51, 0.65)
	UCH-L1	35	251	1.00 (0.90, 1.00)	0.34 (0.28, 0.41)
	S100B^¥ (=100%)	30	231	1.00 (0.89, 1.00)	0.02 (0.00, 0.04)
	S100B (≥95%)	80	231	0.97 (0.84, 1.00)	0.30 (0.24, 0.37)
	S100B (≥90%)	120	231	0.91 (0.75, 0.98)	0.54 (0.47, 0.61)
	S100B^α	100	231	0.91 (0.75, 0.98)	0.44 (0.37, 0.51)
4 hours^Ω	GFAP^¥ (=100%)	0	251	1.00 (0.89, 1.00)	0.00 (0.00, 0.02)
	GFAP (≥95%)	0	251	1.00 (0.89, 1.00)	0.00 (0.00, 0.02)
	GFAP (>90%)	4	251	0.90 (0.74, 0.98)	0.30 (0.24, 0.37)
	UCH-L1^¥ (=100%)	44	251	1.00 (0.89, 1.00)	0.40 (0.33, 0.48)
	UCH-L1 (≥95%)	64	251	0.97 (0.83, 1.00)	0.54 (0.47, 0.62)
	UCH-L1 (≥90%)	74	251	0.90 (0.74, 0.98)	0.60 (0.53, 0.67)
	S100B^¥ (=100%)	80	231	1.00 (0.88, 1.00)	0.28 (0.22, 0.35)
	S100B (>95%)	90	231	0.96 (0.82, 1.00)	0.36 (0.29, 0.44)
	S100B (≥90%)	120	231	0.93 (0.76, 0.99)	0.53 (0.45, 0.60)
	S100B^α	100	231	0.93 (0.76, 0.99)	0.42 (0.34, 0.49)

CI, confidence interval; GFAP, glial fibrillary acidic protein; UCH-L1, ubiquitin carboxyl-terminal hydrolase-L1

∞

If either GFAP UCH-L1 was above the threshold, it was considered to be a positive test. If both were below the threshold, the test was negative.

Time from reported injury to blood sample obtained.

Sensitivity and specificity results for each target sensitivity level (100%, ≥95%, and ≥90%) and resulting biomarker threshold values.

This is at the recommended threshold of 100 pg/mL for S100B.

Table 5 provides a summary of the sensitivity and specificity at a variety of biomarker threshold values (with associated 95% CIs) and details the threshold biomarker values determined for sensitivity ≥95% and ≥90% and the same resulting test characteristics for patients evaluated at 4 h post-injury.

Of the three patients with abnormal CT findings that would have been missed by S100B (<100 pg/mL), one had a subdural hematoma; the second patient had a subdural hematoma with a cortical hemorrhagic contusion and subarachnoid hemorrhage, and the third had a hemorrhagic contusion with subarachnoid hemorrhage and depressed skull fracture. For two patients, the mechanism of injury was a fall and the third was physically assaulted. All three presented with a GCS score of 15.

When implementing a combined strategy of all three markers (positive if GFAP ≥100, UCH-L1 ≥40, or S100B ≥100), the resulting sensitivity remained at 100%, but specificity decreased to 0.29 and 0.26 at 6 h and 4 h, respectively (not in table).

Finally, we examined the results for and GFAP and UCH-L1 when only considering the 231 patients having all biomarker data. For GFAP, specificity was 0.00, but for UCH-L1, 100% sensitivity was obtained at a concentration of 40.9 pg/mL (95% CI 0.89–1.00) and sensitivity was 0.40 (95% CI 0.33 to 0.47), neither reflecting a selection bias.

Discussion

This prospective observational study evaluated three biomarkers for potential usefulness to exclude an acute intracranial injury as determined by CT scan in patients with mild to moderate TBI. Both S100B and GFAP have been studied in patients with mild to moderate TBI with the former being extensively evaluated. For UCH-L1, however, only limited information in patients with mTBI is available, particularly related to early screening for CT abnormalities in patients with mild to moderate TBI.

Therefore, we conducted the first study designed to evaluate test characteristics of the three biomarkers in the same patient cohort with the goal of determining optimal testing strategies and applying the results to a larger cohort of patients with mild to moderate TBI. Our results demonstrate that early biomarker screening for patients with mild to moderate TBI potentially can provide the clinician with objective evidence needed to reduce CT use.

This study was not designed to determine biomarker capability to differentiate patients with or without any brain injury but rather to differentiate patients with head injury who have a negative versus positive head CT scan. Therefore, it is likely patients enrolled in this study who had a negative head CT scan could still have had neuronal and glial cellular damage that resulted in elevated biomarker levels. This and other factors may have led to the high degree of overlap in biomarker levels (particularly with GFAP and S100B) between CT negative and CT positive patients. In fact, a recent study reported that MRI during the subacute phase (1–2 weeks) post-injury detected injuries in more than 20% of patients who had negative CT finding on initial evaluations. This indicates that blood biomarkers coupled with advanced MRI imaging are complimentary in the diagnosis of mTBI cases.^20,29

Because there is no true “gold-standard” for the early diagnosis of mTBI, there still may be utility in considering the possibility that patients with elevated biomarkers may have sustained actual brain injury despite having a negative CT scan. This concept requires further study.

Of the three biomarkers evaluated in this study, UCH-L1 had the best test performance when used to differentiate between subjects with normal versus abnormal CTs. Within 6 h of injury, a UCH-L1 level above 40 pg/mL detected all 36 patients with an acute intracranial lesion as determined by a reference CT scan. Given the estimate of 39% specificity, we could have potentially eliminated 83 (95% CI 70–98 scans) of the 215 negative CT scans performed in our study population. The sensitivity and specificity when combining GFAP and UCH-L1 was the same as UCH-L1 alone, indicating that UCH-L1 was the driving biomarker at the given 6-h time interval. The value of these tests appears to be in the ability of a specified threshold value to reduce the need to obtain a head CT (“rule-out” an abnormal CT) among patients with mild to moderate TBI as opposed to using a high value to “rule-in” a positive scan.

Three patients with abnormal CT findings would have been missed by S100B (<100 pg/mL) at the 6-h time limit. These three patients presented with a GCS score of 15, had important injuries, and would require at least observation. Because this study was not designed to determine need for surgery, we do not know whether the patients required surgery or other invasive neurosurgical procedures.

Analysis of biomarker test characteristics using AUROC curve analysis did not reveal any significant difference between S100B and either UCH-L1 or GFAP. AUROC curves, however, reflect the combination of sensitivity and specificity across the entire range of each. Rather than focusing on a single summary statistic such as the AUROC curve, it is often more useful to define the most important aspect of the test being studied.^27,28,30

For our study, we determined that test sensitivity was the critical test characteristic, and lower levels of specificity would be tolerated. This was because of the a priori decision that the defined screening tests should miss little to no cases of acute intracranial lesions. It was hoped that with optimal sensitivity, that the specificity of the biomarker would still be high enough to safely reduce the number of negative CT scans and the tests would be acceptable to practicing physicians. Although on this derivation cohort, UCH-L1 had 100% sensitivity with a fairly good (39%) specificity, the study sample size was not large enough to have a high level of certainty, and a validation study on a larger patient cohort is warranted.

The Scandinavian guidelines for treating adult patients with mild to moderate head injury recommend S100B as an alternative to head CT when evaluating low-risk patients with either a presenting GCS of 14 or a GCS of 15 with suspected or confirmed loss of consciousness or persistent vomiting.¹⁷ For our patients, S100B was 91% sensitive when using the recommended threshold value of 100 pg/mL at 6 h post-injury.

Lowering the threshold for S100B to 30 pg/mL at 6 h allowed 100% sensitivity (95% CI 90–100) but had a specificity of only 2% (95% CI 0–4), making the test ineffective for decreasing the use of head CT in this setting. At 80 pg/mL, however, the sensitivity at 6 h was 97% (with a specificity of 30%, and at 4 h, this same value had a sensitivity of 100%, with a specificity of 28%. Both of these specificities are less than the 39% specificity of UCH-L1 at the cutoff value of 40 pg/mL.

Given the results of previous studies,^13,20 it was somewhat surprising that GFAP did not perform as well, but this may be because the samples were collected before the bigger rise in serum GFAP levels (data not shown). One recent study suggested that elevated GFAP levels provided a diagnostic benefit above clinical screening alone, but this study included GFAP results that were obtained up to 24 h after mild, moderate, and severe TBI.²⁰ Our study included only patients with mild and moderate TBI who were evaluated much earlier (within 6 h of injury), making our results more applicable and useful for ED screening for abnormal CT scans in patients with head injuries.

Results of biomarker testing performed within 4 h of injury showed a similar sensitivity for UCH-L1 with specificity that remained reasonably good (37%). For S100B at the same 4 h time window, sensitivity was 93% (lower 95% CI 76%) and specificity was 42%. Although this specificity would reduce the number of unnecessary CT scans, the lower sensitivity is unlikely to be acceptable to justify the elimination of head CTs in these patients.

Experts and professional societies have suggested that the increasing use of head CT to evaluate patients with mTBI is unjustified.^1,2,7,17 Clinical decision rules that can guide physicians on the use of head CT are well developed and studied^3,4,31,32 but have not seemed to result in decreased utilization of inappropriate CT examinations among patients with mTBI.^1,33

A simple, sensitive, and rapid screening test that could be used in conjunction with validated clinical decision rules has the potential to reduce the use of CT scans for patients with mTBI and the associated radiation exposure and financial costs. It has been suggested that radiation has the potential to cause a small increased cancer risk to humans.⁵ Recognition of these potential risks has resulted in an initiative by the Food and Drug Administration to reduce radiation exposure as well as publish practice guidelines to minimize unnecessary CT examinations.³⁴

The results of the current study provide evidence that assessments of serum levels of GFAP and UCH-L1 may achieve this goal by predicting, with a high sensitivity, acute intracranial lesions detected by CT while still preserving sufficient specificity to provide an important reduction in CT use.

Limitations

There were a number of limitations that must be considered when evaluating this study and its results. Because this study was designed to evaluate potentially optimal cutoff values for the biomarkers, performance of the biomarkers may be inflated. Derived CIs for specificity (and proportion of eliminated unnecessary CT scans) was conditioned on both the subjective decision of the CT scan and on the choice of an appropriate threshold with 100% sensitivity, and do not reflect the variability inherent in those two processes. External validation of these potential thresholds is needed to assess the robustness of these findings.

Our study found that patients presenting with an initial GCS <15 had a high proportion of positive head CTs. Over a third of patients (6 of 17) with a GCS of 14 had positive scans. This is likely because of our small number of patients and the relatively high percentage of positive CT scans and may not represent the usual group of patients with mTBI who present with an initial GCS of 14. Generalizability of these results is limited to the study population—i.e., patients with an initial GCS of 9–15 who would typically be recommended by the ED physician for emergency head CT scan.

The definition of a positive CT included findings other than just traumatic ICH, but we did exclude isolated skull fracture from the definition. Practice variation led to the inclusion of a relatively high percent of patients with positive head CTs relative to that of patients enrolled in U.S. sites. Given patients were enrolled in the United States and in Europe, selection of patients varied across sites. The majority of subjects had a GCS of 15, so these results may not apply to patients with a GCS score less than 15.

Finally, this study was not designed to determine clinical outcomes such as need for neurosurgical intervention or death but rather just to determine the association of specific biomarker values with a positive finding on head CT—an important first step in reducing CT use.

Conclusion

Our data suggest that when obtained within 6 h of injury, UCH-L1 and the combination of GFAP and UCH-L1 (at our pre-defined cutoff values) were very sensitive for a positive CT scan of the head among patients with mild to moderate TBI. If confirmed in a large trial with greater numbers of positive CT scans, UCH-L1 and the combination of GFAP and UCH-L1 could provide the objective evidence clinicians desire to reduce the use of CT scans among patients with mTBI. Further study related to other times post-injury and the utility of repeated test measurements will help further clarify the utility of these biomarkers.

Footnotes

Acknowledgment

This work is supported by the U.S. Army Medical Research and Material Command under Contract N. W81XWH-06-1-0517 and Contract N. W81XWH-10-C-0251. The views, opinions and/or findings contained in this report are those of the authors and should not be construed as an official Department of the Army position, policy or decision unless so designated by other documentation.

ClinicalTrials.gov Identifier: NCT01295346.

Banyan Biomarkers, Inc. personnel include Art Weber, Anna Shoshoo, Mike Catania, and Aarti Chawla.

Author Disclosure Statement

No competing financial interests exist.

References

Melnick

E.R.

, Szlezak

C.M.

, Bentley

S.K.

, Dziura

J.D.

, Kotlyar

, and Post

L.A.

(2012). CT overuse for mild traumatic brain injury. Jt. Comm. J. Qual., 38, 483–489.

Korley

F.K.

, Morton

M.J.

, Hill

P.M.

, Mundangepfupfu

, Zhou

, Mohareb

A.M.

, and Rothman

R.E.

(2013). Agreement between routine emergency department care and clinical decision support recommended care in patients evaluated for mild traumatic brain injury. Acad. Emerg. Med., 20, 463–469.

Stiell

I.G.

, Wells

G.A.

, Vandemheen

, Clement

, Lesiuk

, Laupacis

, McKnight

R.D.

, Verbeek

, Brison

, Cass

, Eisenhauer

M.E.

, Greenberg

, and Worthington

(2001). The Canadian CT Head Rule for patients with minor head injury. Lancet, 357, 1391–1396.

Haydel

M.J.

, Preston

C.A.

, Mills

T.J.

, Luber

, Blaudeau

, and DeBlieux

P.M.

(2000). Indications for computed tomography in patients with minor head injury. N. Engl. J. Med., 343, 100–105.

Smith-Bindman

, Lipson

, Marcus

, Kim

K.P.

, Mahesh

, Gould

, Berrington de Gonzalez

, and Miglioretti

D.L.

(2009). Radiation dose associated with common computed tomography examinations and the associated lifetime attributable risk of cancer. Arch. Intern. Med., 169, 2078–2086.

Brenner

D.J.

, and Hall

E.J.

(2007). Computed tomography—an increasing source of radiation exposure. N. Engl. J. Med., 357, 2277–2284.

Jagoda

A.S.

, Bazarian

J.J.

, Bruns

J.J.

Jr ., Cantrill

S.V.

, Gean

A.D.

, Howard

P.K.

, Ghajar

, Riggio

, Wright

D.W.

, Wears

R.L.

, Bakshy

, Burgess

, Wald

M.M.

, and Whitson

R.R.

(2008). Clinical policy: neuroimaging and decisionmaking in adult mild traumatic brain injury in the acute setting. Ann. Emerg. Med., 52, 714–748.

Jeter

C.B.

, Hergenroeder

G.W.

, Hylin

M.J.

, Redell

J.B.

, Moore

A.N.

, and Dash

P.K.

(2013). Biomarkers for the diagnosis and prognosis of mild traumatic brain injury/concussion. J. neurotrauma, 30, 657–670.

Egea-Guerrero

J.J.

, Revuelto-Rey

, Murillo-Cabezas

, Munoz-Sanchez

M.A.

, Vilches-Arenas

, Sanchez-Linares

, Dominguez-Roldan

J.M.

, and Leon-Carrion

(2012). Accuracy of the S100beta protein as a marker of brain damage in traumatic brain injury. Brain Inj. 26, 76–82.

10.

Zongo

, Ribereau-Gayon

, Masson

, Laborey

, Contrand

, Salmi

L.R.

, Montaudon

, Beaudeux

J.L.

, Meurin

, Dousset

, Loiseau

, and Lagarde

(2012). S100-B protein as a screening tool for the early assessment of minor head injury. Ann. Emerg. Med., 59, 209–218.

11.

Biberthaler

, Linsenmeier

, Pfeifer

K.J.

, Kroetz

, Mussack

, Kanz

K.G.

, Hoecherl

E.F.

, Jonas

, Marzi

, Leucht

, Jochum

, and Mutschler

(2006). Serum S-100B concentration provides additional information for the indication of computed tomography in patients after minor head injury: a prospective multicenter study. Shock, 25, 446–453.

12.

Metting

, Wilczak

, Rodiger

L.A.

, Schaaf

J.M.

, and van der Naalt

(2012). GFAP and S100B in the acute phase of mild traumatic brain injury. Neurology, 78, 1428–1433.

13.

Papa

, Silvestri

, Brophy

G.M.

, Giordano

, Falk

J.L.

, Braga

C.F.

, Tan

C.N.

, Ameli

N.J.

, Demery

J.A.

, Dixit

N.K.

, Mendes

M.E.

, Hayes

R.L.

, Wang

K.K.

, and Robertson

C.S.

(2014). GFAP out-performs S100beta in detecting traumatic intracranial lesions on computed tomography in trauma patients with mild traumatic brain injury and those with extracranial lesions. J. Neurotrauma, 31, 1815–1822.

14.

Yokobori

, Hosein

, Burks

, Sharma

, Gajavelli

, and Bullock

(2013). Biomarkers for the clinical differential diagnosis in traumatic brain injury—a systematic review. CNS Neurosci Ther. 19, 556–565.

15.

Muller

, Townend

, Biasca

, Unden

, Waterloo

, Romner

, and Ingebrigtsen

(2007). S100B serum level predicts computed tomography findings after minor head injury. J. Trauma, 62, 1452–1456.

16.

Zetterberg

, Smith

D.H.

, and Blennow

(2013). Biomarkers of mild traumatic brain injury in cerebrospinal fluid and blood. Nat. Rev. Neurol., 9, 201–210.

17.

Unden

, Ingebrigtsen

, Romner

; Scandinavian Neurotrauma Committee. (2013). Scandinavian guidelines for initial management of minimal, mild and moderate head injuries in adults: an evidence and consensus-based update. BMC Med., 11, 50.

18.

Barbosa

R.R.

, Jawa

, Watters

J.M.

, Knight

J.C.

, Kerwin

A.J.

, Winston

E.S.

, Barraco

R.D.

, Tucker

, Bardes

J.M.

, and Rowell

S.E.

(2012). Evaluation and management of mild traumatic brain injury: an Eastern Association for the Surgery of Trauma practice management guideline. J. Trauma Acute Care Surg., 73, Suppl 4, S307–S314.

19.

Papa

, Lewis

L.M.

, Falk

J.L.

, Zhang

, Silvestri

, Giordano

, Brophy

G.M.

, Demery

J.A.

, Dixit

N.K.

, Ferguson

, Liu

M.C.

, Mo

, Akinyi

, Schmid

, Mondello

, Robertson

C.S.

, Tortella

F.C.

, Hayes

R.L.

, and Wang

K.K.

(2012). Elevated levels of serum glial fibrillary acidic protein breakdown products in mild and moderate traumatic brain injury are associated with intracranial lesions and neurosurgical intervention. Ann. Emerg. Med., 59, 471–483.

20.

McMahon

P.J.

, Panczykowski

D.M.

, Yue

J.K.

, Puccio

A.M.

, Inoue

, Sorani

M.D.

, Lingsma

H.F.

, Maas

A.I.

, Valadka

A.B.

, Yuh

E.L.

, Mukherjee

, Manley

G.T.

, Okonkwo

D.O.

, Casey

S.S.

, Cheong

, Cooper

S.R.

, Dams-O'Connor

, Gordon

W.A.

, Hricik

A.J.

, Lawless

, Menon

, Schnyer

D.M.

, and Vassar

M.J.

(2015). Measurement of the glial fibrillary acidic protein and its breakdown products GFAP-BDP biomarker for the detection of traumatic brain injury compared to computed tomography and magnetic resonance imaging. J. Neurotrauma, 32, 527–533.

21.

Wilkinson

K.D.

, Lee

K.M.

, Deshpande

, Duerksen-Hughes

, Boss

J.M.

, and Pohl

(1989). The neuron-specific protein PGP 9.5 is a ubiquitin carboxyl-terminal hydrolase. Science, 246, 670–673.

22.

Mondello

, Muller

, Jeromin

, Streeter

, Hayes

R.L.

and Wang

K.K.

(2011). Blood-based diagnostics of traumatic brain injuries. Expert Rev. Mol. Diagn., 11, 65–78.

23.

Papa

, Lewis

L.M.

, Silvestri

, Falk

J.L.

, Giordano

, Brophy

G.M.

, Demery

J.A.

, Liu

M.C.

, Mo

, Akinyi

, Mondello

, Schmid

, Robertson

C.S.

, Tortella

F.C.

, Hayes

R.L.

, and Wang

K.K.

(2012). Serum levels of ubiquitin C-terminal hydrolase distinguish mild traumatic brain injury from trauma controls and are elevated in mild and moderate traumatic brain injury patients with intracranial lesions and neurosurgical intervention. J. Trauma Acute Care Surg., 72, 1335–1344.

24.

Bazarian

J.J.

, Blyth

B.J.

, He

, Mookerjee

, Jones

, Kiechle

, Moynihan

, Wojcik

S.M.

, Grant

W.D.

, Secreti

L.M.

, Triner

, Moscati

, Leinhart

, Ellis

G.L.

, and Khan

(2013). Classification accuracy of serum Apo A-I and S100B for the diagnosis of mild traumatic brain injury and prediction of abnormal initial head computed tomography scan. J. Neurotrauma, 30, 1747–1754.

25.

Florkowski

C.M.

(2008). Sensitivity, specificity, receiver-operating characteristic (ROC) curves and likelihood ratios: communicating the performance of diagnostic tests. Clin. Biochem. Rev., 29, Suppl 1, S83–S87.

26.

DeLong

E.R.

, DeLong

D.M.

, and Clarke-Pearson

D.L.

(1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44, 837–845.

27.

Bossuyt

P.M.

, Reitsma

J.B.

, Bruns

D.E.

, Gatsonis

C.A.

, Glasziou

P.P.

, Irwig

L.M.

, Moher

, Rennie

, de Vet

H.C.

, and Lijmer

J.G.

(2003). The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann. Intern. Med., 138, W1–W12.

28.

Sauerbrei

, Abrahamowicz

, Altman

D.G.

, le Cessie

, and Carpenter

(2014). STRengthening analytical thinking for observational studies: STRATOS initiative. Stat. Med., 33, 5413–5432.

29.

Kou

, Gattu

, Kobeissy

, Welch

R.D.

, O'Neil

B.J.

, Woodard

J.L.

, Ayaz

S.I.

, Kulek

, Kas-Shamoun

, Mika

, Zuk

, Tomasello

, and Mondello

(2013). Combining biochemical and imaging markers to improve diagnosis and characterization of mild traumatic brain injury in the acute setting: results from a pilot study. PloS One, 8, e80296.

30.

Korevaar

D.A.

, van Enst

W.A.

, Spijker

, Bossuyt

P.M.

, and Hooft

(2014). Reporting quality of diagnostic accuracy studies: a systematic review and meta-analysis of investigations on adherence to STARD. Evid. Based Med., 19, 47–54.

31.

Smits

, Dippel

D.W.

, de Haan

G.G.

, Dekker

H.M.

, Vos

P.E.

, Kool

D.R.

, Nederkoorn

P.J.

, Hofman

P.A.

, Twijnstra

, Tanghe

H.L.

, and Hunink

M.G.

(2005). External validation of the Canadian CT Head Rule and the New Orleans Criteria for CT scanning in patients with minor head injury. JAMA, 294, 1519–1525.

32.

Stiell

I.G.

, Clement

C.M.

, Rowe

B.H.

, Schull

M.J.

, Brison

, Cass

, Eisenhauer

M.A.

, McKnight

R.D.

, Bandiera

, Holroyd

, Lee

J.S.

, Dreyer

, Worthington

J.R.

, Reardon

, Greenberg

, Lesiuk

, MacPhail

, and Wells

G.A.

(2005). Comparison of the Canadian CT Head Rule and the New Orleans Criteria in patients with minor head injury. JAMA, 294, 1511–1518.

33.

Stiell

I.G.

, Clement

C.M.

, Grimshaw

J.M.

, Brison

R.J.

, Rowe

B.H.

, Lee

J.S.

, Shah

, Brehaut

, Holroyd

B.R.

, Schull

M.J.

, McKnight

R.D.

, Eisenhauer

M.A.

, Dreyer

, Letovsky

, Rutledge

, Macphail

, Ross

, Perry

J.J.

, Ip

, Lesiuk

, Bennett

, and Wells

G.A.

(2010). A prospective cluster-randomized trial to implement the Canadian CT Head Rule in emergency departments. CMAJ, 182, 1527–1532.

34.

Initiative to Reduce Unnecessary Radiation Exposure from Medical Imaging-February. 2010. Center for Devices and Radiological Health-U.S. Food and Drug Administration. Available at: http://www.fda.gov/downloads/Radiation-EmittingProducts/RadiationSafety/RadiationDoseReduction/UCM200087.pdf. Accessed: October 17, 2015 .