Clinical Utility of the Child and Adolescent Memory Profile (ChAMP) After Pediatric Traumatic Brain Injury

Abstract

Sixty-one children and adolescents with traumatic brain injury completed the Child and Adolescent Memory Profile (ChAMP; Sherman & Brooks, 2015) within 1 to 12 months post injury. Most of the ChAMP index scores demonstrated statistically significant negative correlations with time to follow commands following traumatic brain injury. Compared with demographically matched neurologically healthy controls, selected from the ChAMP standardization sample, participants with traumatic brain injury had statistically significantly lower scores on all ChAMP index scores but sensitivity and specificity were suboptimal. We conclude that the ChAMP has modest clinical utility as part of a more comprehensive evaluation of sequelae of traumatic brain injury in children and adolescents.

Keywords

learning memory brain injury neuropsychological assessment

Traumatic brain injury (TBI) is a major public health concern for children and adolescents, accounting for approximately 640,000 emergency department visits, 18,000 hospitalizations, and 1,500 deaths annually (Taylor et al., 2017). Especially with increasing injury severity, TBI negatively affects children and adolescents’ learning and memory, which in turn may lead to difficulties meeting educational demands in school and poor occupational attainment later in life (for reviews, see, Catroppa et al., 2016; Roebuck-Spencer et al., 2018). Uncomplicated milder injuries are typically associated with relatively better outcomes unless there are premorbid complicating factors (Babikian et al., 2013; Hung et al., 2014). The purpose of this investigation was to assess the clinical utility of a relatively new test of learning and memory, the Child and Adolescent Memory Profile (ChAMP; Sherman & Brooks, 2015) after pediatric TBI.

Deficits in memory following pediatric TBI have been linked to injury characteristics as well as educational outcomes (Donders & Nesbitt-Greene, 2004; Donders & Woodward, 2003; Miller & Donders, 2003; Salorio et al., 2005; Thaler et al., 2011; Viot et al., 2019). These studies used standardized instruments, including the California Verbal Learning Test–Children’s Version (CVLT-C; Delis et al., 1994), the Children’s Memory Scale (CMS, Cohen, 1997), the Test of Memory and Learning (Reynolds & Bigler, 1994) and the Wide Range Assessment of Memory and Learning (Sheslow & Adams, 1990). However, all of these instruments have normative samples that are now more than two dozen years old. In contrast, the ChAMP was published just a few years ago.

The ChAMP is measure of learning and memory that was designed to assess a wide range of examinees from school-age to college-age, and to be usable with children who have motor impairments unrelated to memory. However, although limited information on some clinical samples was included in the test manual, there has been very little independent research on the ChAMP. Specifically, at this time, we know of no published peer-reviewed study of how the instrument performs across the range of severity of pediatric TBI. This is important because tests are supposed to be specifically validated for the clinical populations with which they are used (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014). We wanted to determine the clinical utility of the ChAMP in a pediatric sample with TBI. We decided a priori that, in order for the ChAMP to be considered clinically useful in the evaluation of learning and memory in children and adolescents with TBI, a number of criteria would need to be met. In this context, we considered not only statistical significance but also magnitude of effect size, consistent with conventional criteria (Murphy et al., 2014). Specifically, these criteria were that

ChAMP index scores should demonstrate at least small (ρ ≥ .20) and statistically significant (p < .05) negative correlations with injury severity, as indexed by time to follow verbal commands postinjury.

Average ChAMP index scores of children and adolescents with TBI should be statistically significantly below those of demographically matched, neurologically healthy controls, with at least small univariate effect sizes (η² ≥ .05) .

Classification accuracy of the ChAMP in terms of discriminating between participants with TBI and demographically matched controls should be associated with an Area Under the Curve of at least .70.

Method

We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study.

Participants

This study included two groups of participants: a clinical group of children and adolescents with TBI, and a demographically matched control group. We first retrieved data on 61 clinical participants from the electronic archives on referred pediatric patients with TBI who were evaluated over a period of about 4 years between 2015 and 2019. We then matched 61 control participants to these clinical patients on age, birth sex, parental level of education and (when possible) parent-reported racial background. We selected the latter group, with permission from the publisher, from the ChAMP standardization sample.

We used the following criteria for selecting the clinical participants: (1) ≥6 years and ≤16 years old; (2) diagnosis of TBI, defined as an external blow to the head with alteration of consciousness; and (3) neuropsychological assessment with inclusion of the ChAMP completed within 30 to 360 days postinjury. The ChAMP had been routinely included in comprehensive outpatient neuropsychological evaluations of children and adolescents with TBI during the 4-year time frame. The only exceptions were if there were personal limitations that would make results of this instrument invalid (e.g., child not being fluent in English [n = 1] or having severe uncorrected visual impairment [n = 1]). Most often, the entire instrument had been administered. In some cases (about 8% of all children with TBI seen during this data collection period), only the screening version of the instrument was used because of time constraints, patient fatigue, or other barriers. For purposes of this study, we selected only cases where pediatric patients had completed the entire ChAMP.

We excluded from this investigation the data from three of originally 67 potential participants who met the above inclusion criteria but who did not pass a formal performance validity test (i.e., either the Test of Memory Malingering [Tombaugh, 1996; n = 2] or the Memory Validity Profile [Sherman & Brooks, 2015]; n = 1). In addition, we excluded data from children with a premorbid chronic neurological condition (hydrocephalus, n = 1), pervasive developmental disorder (autism, n = 1), or psychiatric disorder requiring hospitalization (bipolar disorder, n = 1).

In cases where clinical participants received multiple assessments, we only used results from the original evaluation. We obtained premorbid histories through a combination of review of academic and medical records, and semistructured interviews with parents or guardians. We determined duration of time to follow commands following injury (also known as coma) and findings from neuroimaging on the basis of review of acute care medical records. These included daily entries from attending physicians with regard to the patient’s ability to respond at least twice within the same day to a one-step instruction such as to squeeze the examiner’s hand or to open and close the eyes. We classified injury severity on the basis of the absence or presence of any acute intracranial neuroimaging findings, as documented in reports by board-certified radiologists, in combination with time to follow verbal commands. Glasgow Coma Scale scores were not included in this consideration because they were not available for about a fifth (n = 12) of the sample. Measures of posttraumatic amnesia were not routinely collected at children who were initially treated at remote hospitals and were not available for more than a third (n = 23) of the sample. The moderate–severe injury group (n = 32; 53%) included all children who had evidence of an acute intracranial lesion on neuroimaging; typically established within the first 24 to 72 hours after injury. This group also included all 18 children who had duration of time to follow verbal commands ≥ 24 hours (complete sample M = 1.66, Mdn = 0, SD = 4.01, range = 0-25). All children in the uncomplicated mild injury group (n = 29; 47%) had time to follow commands <30 minutes and no intracranial lesions on neuroimaging. All children with complicated mild TBI (i.e., time to follow commands >30 minutes but with acute intracranial lesions on neuroimaging) were assigned to the moderate–severe injury group. Table 1 presents characteristics of the subgroups of the final clinical sample and of the control group.

Table 1.

Characteristics of Children With TBI and Control Participants.

	Uncomplicated	Moderate–severe	All TBI	Controls
	Mild TBI (n = 29)	TBI (n = 32)	(n = 61)	(n = 61)
Sex, n (%)
Female	13 (44.83)	9 (28.13)	22 (36.07)	22 (36.07)
Male	16 (55.17)	23 (71.88)	39 (63.93)	39 (63.93)
Racial background, n (%)
Caucasian	23 (79.13)	23 (71.88)	46 (75.41)	41 (67.21)
African, Asian or, Latino/a	6 (20.87)	9 (28.12)	15 (24.59)	20 (32.79)
Age, years, M (SD)*	13.92 (2.63)	12.42 (3.12)	13.13 (3.05)	13.18 (2.96)
Parental education, years, M (SD)	13.79 (2.62)	13.41 (2.20)	13.59 (2.40)	13.64 (2.60)
Prior LD or ADHD, n (%)	12 (41.38)	11 (34.38)	23 (37.71)	—
	Uncomplicated	Moderate–severe	All TBI	Controls
	Mild TBI (n = 29)	TBI (n = 32)	(n = 61)	(n = 61)
Injury mechanism, n (%)**
Motor vehicle	10 (34.48)	19 (59.38)	29 (47.45)	—
Falls, sports, and other	19 (65.52)	13 (40.62)	32 (52.45)	—
Days since injury, M (SD)	134.41 (62.74)	131.03 (77.26)	132.64 (70.17)	—

Note. Control group selected from the standardization sample from the Child and Adolescent Memory Profile (ChAMP). ©2015 Psychological Assessment Resources. Used with permission. All rights reserved. TBI = traumatic brain injury; LD = learning disability; ADHD = attention-deficit/hyperactivity disorder.

p < .05 for difference between uncomplicated mild TBI and moderate–severe TBI. ^**p < .006 for difference between uncomplicated mild TBI and moderate–severe TBI.

Procedure

Neuropsychological evaluations of clinical participants were completed on clinical referral with informed parental consent and child assent on an outpatient basis. These evaluations were deferred until children and adolescents were medically stable and could recall meaningful information from day to day. The latter was typically established informally by asking about who came to visit or what was for dinner the previous day. The ChAMP and other tests (which varied across the age range) were administered and scored by experienced psychometrists with Master’s degrees or postdoctoral residents under the supervision of board-certified clinical neuropsychologists, or by the neuropsychologists themselves. This research was conducted with approval from the Institutional Review Board at Mary Free Bed Rehabilitation Hospital, and in compliance with the Helsinki Declaration.

Measurements

The ChAMP is a comprehensive instrument for assessing the learning and memory of persons between the ages of 5 and 21 years. It provides index scores of Verbal Memory, Visual Memory, Immediate Memory, and Delayed Memory; as well as a Total Memory composite that reflects global memory functioning across these domains, and a Screening index that is based on only the first two subtests. All these scores are expressed as standard scores (M = 100, SD = 15), with higher scores reflecting better performance. The 10 ChAMP subtests are based on four tasks, two of which involve verbal stimuli and two that involve visual stimuli. Lists requires the learning of 16 concrete nouns. It also has delayed recall (Lists Delayed) and 3-choice recognition (Lists Recognition) trials. Instructions is a paragraph prose recall task. It, too, has delayed recall (Instructions Delayed) and 3-choice recognition (Instructions Recognition) trials. Objects involves recognition memory of difficult-to-verbalize visual stimuli, based on characteristics such as shape, texture, and dimensionality. A delayed (Objects Delayed) trial with the same format is also included. Finally, Places measures recognition memory of visual scenes, based on spatial configuration and contextual detail. It also has a delayed (Places Delayed) trial that uses the same format. All subtests are expressed as scaled scores (M = 10, SD = 3), with higher scores reflecting better performance.

For the ChAMP index scores, internal consistencies range from .83 (Screening) to .93 (Total Memory), and test–retest reliabilities from .56 (Screening) to .77 (Total Memory). Internal consistencies of the subtests range from .54 (Lists Delayed) to .87 (Instructions). Test–retest reliabilities of the subtests range from .43 (Lists Recognition) to .70 (Instructions and Places).

Statistical Analyses

We evaluated background differences between the two TBI subgroups with t tests for continuous variables and with χ² tests for discrete variables. For the ChAMP data, we focused on the index standard scores because they are more reliable than the subtest scores. To investigate the relationship between ChAMP index scores and time to follow verbal commands in the complete clinical sample, we used Spearman correlations instead of Pearson ones because the distribution of time to follow verbal commands was positively skewed. We used analysis of variance to evaluate differences between the complete clinical sample and the control sample in mean performance on the six ChAMP index scores. Finally, we used logistic regression to determine how well findings on the ChAMP could classify group membership in terms of TBI versus control. For this purpose, we used only the Total Memory index because there would be serious problems with collinearity if we were to include multiple index scores in this analysis. Total Memory reflects learning and memory across both verbal and visual domains, and has one of the highest internal consistency and test–retest reliability coefficients of all ChAMP scores.

Results

We first wanted to make sure that injury severity and time since injury were not conflated in the clinical sample. The correlation between time to follow verbal commands and days since injury was not statistically significant, ρ = 0.11, p > .40. Furthermore, the uncomplicated mild and moderate–severe injury groups did not differ meaningfully in time since injury, t(59) = 0.19, p > .85. For these reasons, potential conflation of injury severity and time since injury was not a concern.

We then compared the two injury groups on other variables. There were no statistically significant differences between these two TBI groups in terms of sex, χ²(61) = 1.84, p > .17, proportion of racial minorities, χ²(61) = 0.45, p > .49, parental education, t(59) = 0.63, p > .53, or proportion of prior histories of either learning disability or attention-deficit/hyperactivity disorder, χ²(61) = 0.32, p > .56. However, the group with uncomplicated mild injuries was, on average, slightly older than the group with moderate–severe injuries, t(59) = 2.02, p < .05, d = 0.52. Since ChAMP scores are based on age-based norms, we did not consider this to be problematic. Finally, there was a notable difference in terms of injury circumstances, with the moderate–severe injury group being much more likely to have been injured as the result of accidents involving a motor vehicle than the uncomplicated mild group, χ²(61) = 7.72, p < .006, odds ratio (OR) = 4.59, confidence interval [CI: 1.52, 13.86]. This made sense, given the risk of more severe injuries when collisions occur at high speed. In contrast, uncomplicated mild injuries occurred more often as the result of sports or other recreational activities.

Table 2 presents the ChAMP index scores for the two TBI subgroups and the control group, along with the correlations with length to follow verbal commands in the complete TBI group. The associated ChAMP subtest scores are presented in Figure 1 for illustrative purposes. The pattern of the subtest scores suggested that the clearest and most consistent group difference was on the Places and Places Delayed subtests, where the control group outperformed the moderate–severe TBI group by about a standard deviation.

Table 2.

ChAMP Index Scores, M (SD), of Children With TBI and Demographically Matched Controls, and Spearman Correlations (ρ) With Time to Follow Commands in the Complete TBI Group.

	Uncomplicated	Moderate–severe	All TBI	Controls	ρ
	Mild TBI (n = 29)	TBI (n = 32)	(n = 61)	(n = 61)	ρ
Verbal Memory index^a	97.79 (14.47)	94.22 (17.29)	95.92 (15.98)	103.18 (13.13)	−0.19
Visual Memory index^b	96.00 (15.25)	90.31 (15.02)	93.02 (15.27)	103.82 (15.24)	−0.34**
Immediate Memory index^b	96.38 (12.89)	92.53 (14.16)	94.36 (13.60)	103.84 (14.62)	−0.27*
Delayed Memory index^b	96.59 (15.45)	90.47 (15.76)	93.38 (15.79)	103.85 (12.25)	−0.24
Total Memory index^b	96.31 (14.20)	91.19 (15.12)	93.62 (14.79)	104.10 (13.24)	−0.27*
Screening index^a	96.78 (13.85)	94.61 (16.63)	95.33 (15.24)	102.92 (14.62)	−0.29*

All TBI < control, p < .05. ^bAll TBI < control, p < .01.

Correlation p < .05. ^**Correlation p < .01.

Figure 1.

ChAMP subtest scores for clinical and control participants.

The uncomplicated mild group consistently had higher ChAMP index scores than the moderate–severe group, but these differences consistently fell short of statistical significance (all p > .10) and were also consistently associated with minimal effect sizes (all η² < .05). For this reason, we combined both groups for all further analyses in comparison with the demographically matched control group.

Several but not all of the ChAMP index scores correlated statistically significantly with length to follow verbal commands. Since we had decided a priori that the Total Memory index was of relatively greatest interest, it was relevant that this variable did have a statistically significant negative correlation with length to follow verbal commands.

We then compared the performance of the complete TBI sample with that of the control group. Even when using a Benjamini–Hochberg adjustment to correct for the effect of multiple comparisons, the performance of the control group was statistically significantly better than that of the children and adolescents with TBI on all of the ChAMP index scores. This included Verbal Memory, F(1, 121) = 7.52, p < .04, η² = 0.06; Visual Memory, F(1, 121) = 15.30, p < .002, η² = 0.11; Immediate Memory, F(1, 121) = 13.74, p < .003, η² = 0.10; Delayed Memory, F(1, 121) = 16.77, p < .001, η² = 0.12; Total Memory, F(1, 121) = 16.99, p < .001, η² = 0.12; and Screening, F(1, 121) = 7.88, p < .04, η² = 0.06. The associated effect sizes ranged from small to medium by conventional standards (Murphy et al., 2014).

Finally, we explored the classification accuracy of the ChAMP Total Memory index when distinguishing between the complete TBI group and the control group. The logistic regression was statistically significant, Wald χ²(1) = 13.25, p < .003. Sensitivity was 61% and specificity was 66%, for an overall classification accuracy of 64% with an Area Under the Curve of 0.70 (90% CI [0.65, 0.75]. Although the latter value just met the a priori criterion, the associated likelihood ratio (LR) was only 1.80, which was below the conventional standard of 2 (Grimes & Schulz, 2005). When we did a post hoc analysis where we ran logistic regressions separately by injury severity group, the results showed relatively greater discrimination of the participants with moderate–severe TBI from controls, Wald χ²(1) = 12.24, p < .0004, sensitivity 72%, specificity 69%, Area Under the Curve .74, 90% CI [0.68, 0.80], LR = 2.32, than of the participants with uncomplicated mild TBI from controls, Wald χ²(1) = 5.77, p < .02, sensitivity 48%, specificity 61%, Area Under the Curve .65, 90% CI [0.59, 0.71], LR = 1.23.

Finally, we conducted an analysis of what really predicted Total Memory scores in the complete TBI group. For this purpose, we used linear regression analysis, with Total Memory as the dependent variable and the following as independent variables: time to follow commands, presence, or absence of acute intracranial findings on neuroimaging, parental education, presence or absence of premorbid learning disability or attention-deficit/hyperactivity disorder, and time since injury. This regression model, which is presented in Table 3, was statistically significant, F(5, 55) = 2.97, p < .02, and explained a moderate proportion of the variance, R² = .21, with no concerns about collinearity (all variance inflation factors were <1.30). The only statistically significant contributors within this model were parental education and time to follow verbal commands. The former variable was associated with higher Total Memory scores, whereas the latter variable was associated with lower ones.

Table 3.

Regression Model for Total Memory in the Complete TBI Group.

Variable	Β	SE	β	t	p
Time to follow commands	−1.09	0.48	−0.30	−2.25	<.03
Neuroimaging findings	−1.67	3.89	−0.06	−0.43	<.67
Parental education	1.60	0.75	0.26	2.13	<.04
Prior LD or ADHD	−3.83	3.57	−0.13	−1.07	<.29
Time since injury	0.02	0.03	−0.09	−0.77	<.45

Note. TBI = traumatic brain injury; SE = standard error; LD = learning disability; ADHD = attention-deficit/hyperactivity disorder.

Discussion

The purpose of this evaluation was to determine the clinical utility of the ChAMP in the evaluation of learning and memory of children and adolescents with TBI. Criterion (1) was partially met in that most ChAMP index scores had statistically significant correlations with time to follow verbal commands, where greater injury severity was associated with lower ChAMP index scores. This was true for the Visual Memory, Immediate Memory, Total Memory, and Screening Index scores, although not for the Verbal Memory and Delayed Memory scores. Criterion (2) was unequivocally met: all average ChAMP index scores of children and adolescents with TBI were statistically significantly worse than those of demographically matched controls. However, with regard to Criterion (3), the results were somewhat disappointing. Although the classification accuracy met the minimum criterion for Area Under the Curve, the sensitivity and specificity of the ChAMP Total Memory index were modest, at best.

It was somewhat surprising that Verbal Memory did not perform as convincingly as Visual Memory because past research has suggested strong sensitivity of tests of verbal learning and memory, such as the CVLT-C, to the impact of TBI (Johnson & Donders, 2018; Salorio et al., 2005). At the same time, previous findings with other tests have been mixed regarding visual learning and memory after pediatric TBI. For example, Thaler et al. (2011) reported that of all the subtests on the Test of Memory and Learning, the visually based Object Recall task was relatively most sensitive to TBI. In contrast, Viot et al. (2019) reported that visual memory, as assessed with the CMS, was relatively preserved and improved most over time after severe TBI. It is possible that differences in test format (e.g., three learning trials on ChAMP Lists as compared with five on the CVLT-C) or item content (e.g., faces on the CMS versus environmental scenes on the ChAMP) may have contributed to this.

Our findings also indicated that Total Memory was affected positively by parental education and negatively by time to follow verbal commands. The former influence likely reflects the protective effect of cognitive reserve, which has also been demonstrated with other instruments (Donders & Kim, 2019; Johnson & Donders, 2018). The influence of time to follow commands supports the sensitivity of Total Memory to severity of pediatric TBI, even though its classification accuracy is fairly modest.

Although the current findings offer some support for the clinical utility of the ChAMP in the evaluation of novel learning and memory after pediatric TBI, it is also important to appreciate that the total classification accuracy of the Total Memory index in differentiating between participants with TBI on the one hand, and demographically matched controls on the other hand, was only 64%. That indicates that clinicians should not rely exclusively on that or any other ChAMP index to rule in or out sequelae of pediatric TBI. Like with any other test, findings from the ChAMP need to be considered in the context of premorbid history, other test results and behavioral observations in order to come to the most accurate diagnosis.

The findings from this investigation require further follow-up in terms of criterion or predictive validity. Correlating ChAMP findings with future outcomes after TBI, such as special education placement (e.g., Miller & Donders, 2003) or academic achievement (e.g., Fulton et al., 2012) will be important. More research is also needed with regard to the ChAMP’s sensitivity to severity of TBI because we did not find statistically significant differences between participants with, respectively, uncomplicated mild TBI and relatively more severe TBI, which was somewhat unusual (Fay et al., 2010; Papoutsis et al., 2014). In addition, the ChAMP requires further independent validation in other clinical samples where there is a high risk of memory impairment, such as pediatric epilepsy (Menlove & Reilly, 2015), because the current investigation was limited to participants with TBI.

We must also acknowledge some limitations of this investigation. This was a referred convenience sample and as such included relatively more persons with moderate–severe injuries than would be found with consecutive emergency room visits. However, this also guarded against effects of restriction of range in the dependent variables. The sample was too small to allow detailed subtyping (e.g., through cluster analysis), and it is possible that we might have found statistically significant differences between the uncomplicated mild versus moderate–severe TBI groups with larger subsample sizes. In addition, we did not have the technology to do more advanced neuroimaging analyses that have shown promise in the evaluation of children and adolescents with TBI, such as diffusion tensor imaging or magnetic resonance spectroscopy (Ashwal et al., 2014; Königs et al., 2018). The most significant limitation was that we were not able to do analyses of subgroups of participants with TBI who did versus did not have memory impairment, as evidenced by performance on independent tests because those were typically not administered as part of routine clinical care. This is a distinct avenue for future research.

With these consideration in mind, we conclude that the ChAMP has modest clinical utility in the evaluation of learning and memory of children with TBI. As part of a more comprehensive evaluation, it can assist clinicians to determine relative strengths and weaknesses that could potentially be addressed in the context of rehabilitation and/or special education. Application of the results of ChAMP findings, using the guidelines suggested on the basis of a recent systematic review of effective treatments of children with various forms of brain injury (Laatsch et al. 2019), is a specific goal for future research.

Footnotes

Acknowledgements

We thank the publisher of the ChAMP for allowing us access to the standardization data. The control group for this study was selected from the standardization sample from the Child and Adolescent Memory Profile (ChAMP). ©2014 Psychological Assessment Resources. Used with permission. All rights reserved.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jacobus Donders

References

American Educational Research Association, American Psycho-logical Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.

Ashwal

Tong

K. A.

Ghosh

Bartnik-Olson

Holshouser

B. A.

(2014). Application of advanced neuroimaging modalities in pediatric traumatic brain injury. Journal of Child Neurology, 29(12), 1704-1717. https://doi.org/10.1177/0883073814538504

Babikian

McArthur

Asarnow

R. F.

(2013). Predictors of 1-month and 1-year neurocognitive functioning from the UCLA longitudinal mild, uncomplicated, pediatric traumatic brain injury study. Journal of the International Neuropsychological Society, 19(2), 145-154. https://doi.org/10.1017/S135561771200104X

Catroppa

Anderson

Beauchamp

M. H.

Yeates

K. O.

(2016). New frontiers in pediatric traumatic brain injury: An evidence base for clinical practice. Routledge. https://doi.org/10.4324/9780203868621

Cohen

M. J.

(1997). Children’s Memory Scale. Psychological Corporation.

Delis

D. C.

Kramer

J. H.

Kaplan

Ober

B. A.

(1984). California Verbal Learning Test–Children’s Version. Psychological Corporation.

Donders

Kim

(2019). Effect of cognitive reserve on children with traumatic brain injury. Journal of the International Neuropsychological Society, 25(4), 355-361. https://doi.org/10.1017/S1355617719000109

Donders

Nesbit-Greene

(2004). Predictors of neuropsychological test performance after pediatric traumatic brain injury. Assessment, 11(4), 275-284. https://doi.org/10.1177/1073191104268914

Donders

Woodward

H. R.

(2003). Gender as a moderator of memory after traumatic brain injury in children. Journal of Head Trauma Rehabilitation, 18(2), 106-115. https://doi.org/10.1097/00001199-200303000-00002

10.

Fay

T. B.

Yeates

K. O.

Taylor

H. G.

Bangert

Dietrich

Nuss

Rusin

Wright

(2010). Cognitive reserve as a moderator of postconcussive symptoms in children with complicated and uncomplicated mild traumatic brain injury. Journal of the International Neuropsychological Society, 16(1), 94-105. https://doi.org/10.1017/S1355617709991007

11.

Fulton

J. B.

Yeates

K. O.

Taylor

H. G.

Walz

N. C.

Wade

S. L.

(2012). Cognitive predictors of academic achievement in young children 1 year after traumatic brain injury. Neuropsychology, 26(3), 314-322. https://doi.org/10.1037/a0027973

12.

Grimes

D. A.

Schulz

K. F.

(2005). Refining clinical diagnoses with likelihood ratios. Lancet, 365(9469), 1500-1505. https://doi.org/10.1016/S0140-6736(05)66422-7

13.

Hung

Carroll

L. J.

Cancelliere

Côté

Rumney

Keightley

Donovan

Stålnacke

B.-M.

Cassidy

J. D.

(2014). Systematic review of the clinical course, natural history, and prognosis for pediatric mild traumatic brain injury: Results of the international collaboration on mild traumatic brain injury prognosis. Archives of Physical Medicine and Rehabilitation, 95(3), S174-S191. https://doi.org/10.1016/j.apmr.2013.08.301

14.

Johnson

V. M.

Donders

(2018). Correlates of verbal learning and memory after pediatric traumatic brain injury. Applied Neuropsychology: Child, 7(4), 298-305. https://doi.org/10.1080/21622965.2017.1330688

15.

Königs

Pouwels

P. J. W.

van Heurn

E. L. W.

Bakx

Vermeulen

J. R.

Goslings

J. C.

Poll-The

B. T.

van der Wees

Catsman-Berrevoets

C. E.

Oosterlaan

(2018). Relevance of neuroimaging for neurocognitive and behavioral outcome after pediatric traumatic brain injury. Brain Imaging and Behavior, 12(1), 29-43. https://doi.org/10.1007/s11682-017-9673-3

16.

Laatsch

Dodd

Brown

Ciccia

Connor

Davis

Doherty

Linded

Locascio

Lundine

Murphy

Nagele

Niemeier

Politis

Rode

Slomine

Smetana

Yaeger

(2019). Evidence-based systematic review of cognitive rehabilitation, emotional, and family treatment studies for children with acquired brain injury literature: From 2006 to 2017. Neuropsychological Rehabilitation, 30(1), 130-161. https://doi.org/10.1080/09602011.2019.1678490

17.

Menlove

Reilly

(2015). Memory in children with epilepsy: A systematic review. Seizure, 25, 126-135. https://doi.org/10.1016/j.seizure.2014.10.002

18.

Miller

L. J.

Donders

(2003). Prediction of educational outcome after pediatric traumatic brain injury. Rehabilitation Psychology, 48(4), 237-241. https://doi.org/10.1037/0090-5550.48.4.237

19.

Murphy

K. R.

Myors

Wolach

(2014). Statistical power analysis (4th ed.). Routledge. https://doi.org/10.4324/9781315773155

20.

Papoutsis

Stargatt

Catroppa

(2014). Long-term executive functioning outcomes for complicated and uncomplicated mild traumatic brain injury sustained in early childhood. Developmental Neuropsychology, 39(8), 638-645. https://doi.org/10.1080/87565641.2014.979926

21.

Reynolds

C. R.

Bigler

E. D.

(1994). Test of Memory and Learning. PRO-ED.

22.

Roebuck-Spencer

Désiré

Beauchamp

(2018). Traumatic brain injury. In Donders

Hunter

S. J.

(Eds.), Neuropsychological conditions across the lifespan (pp. 139-161). Cambridge University Press. https://doi.org/10.1017/9781316996751.009

23.

Salorio

C. F.

Slomine

B. S.

Grados

M. A.

Vasa

R. A.

Christensen

J. R.

Gerring

J. P.

(2005). Neuroanatomic correlates of CVLT-C performance following pediatric traumatic brain injury. Journal of the International Neuropsychological Society, 11(6), 686-696. https://doi.org/10.1017/S1355617705050885

24.

Sherman

E. M. S.

Brooks

B. L.

(2015). Child and Adolescent Memory Profile. Psychological Assessment Resources.

25.

Sheslow

Adams

(1990). Wide Range Assessment of Memory and Learning. Jastak.

26.

Taylor

C. A.

Bell

J. M.

Breiding

M. J.

(2017). Traumatic brain injury-related emergency department visits, hospitalizations, and deaths: United States, 2007 and 2013. Morbidity and Mortality Weekly Report: Surveillance Summaries, 66(9), 1-16. https://doi.org/10.15585/mmwr.ss6609a1

27.

Thaler

N. S.

Barney

S. J.

Reynolds

C. R.

Mayfield

Allen

D. M.

(2011). Differential sensitivity of TOMAL subtests and index scores to pediatric traumatic brain injury. Applied Neuropsychology, 18(3), 168-178. https://doi.org/10.1080/09084282.2011.595443

28.

Tombaugh

T. N.

(1996). Test of Memory Malingering. Multi-Health Systems.

29.

Viot

Câmara-Costa

Laurence

Francillette

Toure

Brugel

Laurent-Vannier

Dellatolas

Gillibert

Meyer

Chevignard

(2019). Assessment of memory functioning over two years following severe childhood traumatic brain injury: Results of the TGE cohort. Brain Injury, 33(9), 1208-1218. https://doi.org/10.1080/02699052.2019.1631485