Sex- and Age-Related Differences in Post-Concussive Symptom Reporting Among Children and Their Parents

Abstract

Pediatric mild traumatic brain injury (pmTBI) has received increased public attention over the past decade, especially for children who experience persistent post-concussive symptoms (PCS). Common methods for obtaining pediatric PCS rely on both self- and parental report, exhibit moderate test-retest reliability, and variable child–parent agreement, and may yield high false positives. The current study investigated the impact of age and biological sex on PCS reporting (Post-Concussion Symptom Inventory) in patients with pmTBI (n = 286) at retrospective, 1 week, 4 months, and 1 year post-injury time points, as well as reported symptoms in healthy controls (HC; n = 218) at equivalent assessment times. HC and their parents reported higher PCS for their retrospective rating relative to the other three other study visits. Child–parent agreement was highest for female adolescents, but only approached acceptable ranges (≥ 0.75) immediately post-injury. Poor-to-fair child/parental agreement was observed for most other study visits for pmTBI and at all visits for HC. Parents rated female adolescents as being more symptomatic than their male counterparts in spite of small (pmTBI) or no (HC) sex-related differences in self-reported ratings, suggestive of a potential cultural bias in parental ratings. Test-retest reliability for self-report was typically below acceptable ranges for both pmTBI and HC groups, with reliability decreasing for HC and increasing for pmTBI as a function of time between visits. Parental test–retest reliability was higher for females. Although continued research is needed, current results support the use of child self-report over parental ratings for estimating PCS burden. Results also highlight the perils of relying on symptom self-report for diagnostic and prognostic purposes.

Introduction

Post-concussive symptoms (PCS) following pediatric mild traumatic brain injury (pmTBI; used synonymously herein with concussion) remain a large public health concern because of the number of children affected annually (∼ 750,000), and are especially problematic when they become persistent in nature (i.e., more than 1 month post-injury).^1,2 Several clinical^3,4 and research-based protocols exist for diagnosing persistent PCS,^5
–7 with the majority requiring patients and/or their parents to report on multi-dimensional PCS (cognitive, emotional, and physical), typically in reference to either the patient's pre-injury baseline (i.e., simple change methods) or a normative sample.⁸ Large studies have suggested that approximately one third of patients with pmTBI exhibit persistent PCS when classification occurs in a binary fashion.^5,6,9 However, the true incidence rate may depend on the diagnostic method employed, the reporting source (parent vs. child), point of care for the sample (emergency room vs. specialty clinic), or individual host factors (e.g., age and sex). PCS are also non-specific and therefore frequently observed in healthy children and those without concussions in the general population.

Parental report forms the basis for commonly used clinical risk scores (e.g., 5P risk score)⁹ and is frequently used in lieu of child ratings because of concerns about the validity/reliability of child self-report.^10,11 However, child–parent agreement on PCS burden may be dependent on injury chronicity,^6,11
–13 PCS domain,^6,11

–14 and assessment methodology.^15
–17 Further confusing the landscape, child–parent PCS agreement may also vary based on patient sex and age.^15,16,18 Similarly, female sex and adolescence represent known risk factors for persistent PCS, with females recovering more slowly than males up to 4 months post-injury.^5,9,19,20 Finally, multiple instruments are used to assess pediatric PCS (e.g., Post-Concussion Symptom Inventory, Health and Behavior Inventory, Post-Concussion Symptom Scale, Rivermead Post-Concussion Symptoms Questionnaire), with slight variations in each instrument in terms of administration and item content.²¹

To date, few studies have systematically examined how host factors (i.e., sex and age at injury) affect the magnitude of PCS presentation, psychometric properties of PCS (inter-rater [i.e., child–parent agreement] and test-retest reliability), or binary classification rates of symptomatic versus non-symptomatic patients at multiple stages of recovery.¹⁸ Our primary hypothesis was that the psychometric properties of PCS would generally fall below acceptable ranges (< 0.75) for independent cohorts of pmTBI and healthy controls (HC), but would be lower for females than for males, and lower for middle childhood than for adolescents.^6,15,18 We also predicted opposing temporal trajectories for test-retest reliability in HC (higher to lower reliability over time) relative to pmTBI (lower to higher reliability over time because of recovery from symptoms). These hypotheses were tested in large cohorts of pmTBI (n = 286) and HC (n = 218) at three time points up to 1 year post-injury.

Methods

Participants

Individuals with pmTBI (8–18 years) were diagnosed by clinicians independent of the study and were consecutively recruited from local emergency and urgent care departments in this ongoing study. Participants were evaluated at ∼1 week (sub-acute assessment [SA] at 7.3 ± 2.2 days), 4 months (early chronic [EC] assessment at 133.0 ± 19.7 days) and 1 year (late chronic [LC] assessment at 371.3 ± 31.7 days) post-injury. Inclusion criteria represented a blend of American Congress of Rehabilitation Medicine (upper limits: Glasgow Coma Scores ≥13, maximum loss of consciousness [LOC] = 30 min, and maximum post-traumatic amnesia [PTA] = 24 h) and Zurich Concussion in Sport Group (lower limits: at least two new symptoms) guidelines.⁸ HC, recruited from the local community through fliers and word of mouth, were evaluated at equivalent time points. Both parents and children provided informed consent (parents of children <18 years old or adolescents 18 years old) and/or assent (children <18 years old) according to institutional guidelines at the University of New Mexico School of Medicine.

Both pmTBI and HC were excluded for history of major neurological diagnoses, moderate or severe TBI (> 30 min LOC), developmental disorders (autism spectrum disorder or intellectual disability), psychiatric disorders other than adjustment disorder, substance abuse/dependence, or non-English fluency. Additional exclusion criteria for HC included attention-deficit/hyperactivity disorder or a learning disability. Urine-based drug screens were conducted for all participants at all three visits. Positive results resulted in study exclusion. Any new brain injury during the protocol also resulted in exclusion from subsequent visits. A total of 286 pmTBI (n = 162 males; 13.9 ± 2.8 years) participated in the study along with 218 sex- and age-matched HC (n = 121 males, 13.4 ± 2.9 years). See Figure 1, Supplementary text, and Table S1 for full details on enrollment, quality assurance, and retention rates across both cohorts and subgroups. The current study represents a partial replication and extension of findings from a previous article, with approximately half of the sample (total n = 279) up to 4 months post-injury.⁶

FIG. 1.

Participant flowchart. Flow chart of participant enrollment, inclusion, and data quality assurance from the sub-acute (SA), early chronic (EC), and late chronic (LC) phase visits for patients with pediatric mild traumatic brain injury (pmTBI) as well as healthy controls (HC). The mean number of days post-injury (DPI) at each assessment are included for pmTBI, whereas the mean number of days between SA and subsequent visits (days between visits; DBV) are included for both cohorts.

Procedures

All participants and their parents completed a modified version of the Post-Concussion Symptom Inventory (PCSI)²² at each visit as well as providing a retrospective (R) rating at the SA visit. The PCSI was modified in the following ways with author permission: (1) the version of the PCSI for 13–18-year-olds was also utilized for 12-year-old participants to equate the response scale, and (2) all references to an injury were removed from both retrospective and concurrent questionnaires and instructions to avoid bias in HC. The retrospective (i.e., 1 month prior to initial visit) and concurrent (day of assessment) reporting instructions were also specified in the modified version.

All PCSI data were normalized to a percentage value (sum of individual rating/maximum score × 100) because of different scale ratings for different age ranges.¹¹ The methods used to calculate persistent PCS are similar to in our previous publication⁸ and were based on a standardized rather than a simple change method because of superior psychometric properties. Specifically, normalized PCSI total scores were first converted to z-scores using the mean and standard deviation from the HC data separately for child and parental reports. Individuals from both pmTBI and HC groups were then operationally classified as being either highly symptomatic or non-symptomatic in a binary fashion using a distributional corrected z-score cutoff of z > 1.64.²³

Data for self-reported Tanner stage of development were also acquired along with a semi-structured interview of pediatric TBI history to confirm injury severity.²⁴ Additional clinical and neuropsychological measures were collected as part of this ongoing study, but have been previously presented in separate manuscripts.^25,26

Statistical analysis

All analyses were conducted in SPSS (IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY) separately for HC and pmTBI. Generalized estimating equations (GEE) compared self- and parental ratings on total (primary results; percent of scale maximum) and sub-domain (supplemental results; percent of sub-scale maximum) PCSI scales using the negative binomial distribution. Results were Bonferroni corrected for the child and parental measures within pmTBI and HC cohorts (0.05/2 = 0.025). Each group was further stratified based on biological sex and developmental stage (middle childhood [MC]: 8–12 years old; adolescence [AD]: 13–18 years old). Age cutoffs were determined based on a combination of general developmental theory^27
–29 and reports of differential outcomes in persistent PCS for these two groups.⁹ Intraclass correlation coefficients³⁰ were used to examine both inter-rater reliability (ICC[2,k]; self-report vs. parental report) and test-retest reliability (ICC[2,1]). Gwet's AC1 estimation³¹ evaluated agreement between child–parent dyads for binary classification of significant PCS symptoms. All reliability estimates were categorized as poor (≤ 0.39), fair (0.40–0.59), good (0.60–0.74), or excellent (≥ 0.75) based on published guidelines, with excellent (≥ 0.75) reliability deemed to be the minimal/acceptable number to establish good psychometric properties.³² All negative ICC values were replaced with a value of zero for graphical purposes only (i.e., Tables and Figures). The R module ggplot2 3.3.5 was used to generate figures.

Results

Demographics and PCSI magnitude results

Demographic data for HC and pmTBI groups are presented separately in Table 1. Raw PCSI scores are reported in Table S2, whereas all analyses were based on normalized values (percentages). Results from a 2 × 2 × 4 (sex × age × visit) GEE with child self-report normalized PCSI scores for HC (Fig. 2A) indicated a significant sex × visit interaction (Wald-χ² = 9.87; p = 0.020), as well as main effects associated with both age (Wald-χ² = 10.18; p = 0.001; Cohen's d = 0.23; MC > AD) and visit (Wald-χ² = 23.03; p < 0.001; R > SA ≈ EC ≈ LC) after Bonferroni correction. Follow-up analyses indicated that female HC had a higher normalized PCSI score than males at EC visit (p = 0.004), with non-significant sex-related differences at all other visits (all ps > 0.05). Results from the HC parental report (Fig. 2B) were similar, with a significant sex × visit interaction (Wald-χ² = 11.25; p = 0.010), as well as main effects for sex (Wald-χ² = 6.63; p = 0.010; Cohen's d = 0.186; F > M) and visit (Wald-χ² = 30.76; p < 0.001; R > SA & LC). Follow-up analyses for the interaction indicated that parents of females rated a higher PCSI magnitude than parents of males at the SA (p = 0.013), EC (p = 0.013), and LC (p = 0.005) visits, with non-significant sex differences for retrospective ratings (p > 0.05).

FIG. 2.

Total Post-Concussive Symptom Inventory (PCSI) ratings. Normalized PCSI ratings (sum of PCSI ratings/total possible points) are separately displayed for self- (Panel A) and parental report (Panel B) from healthy controls (HC), as well as for self- (Panel C) and parental (Panel D) report from patients with pediatric mild traumatic brain injury (pmTBI). Scatter-box plots are presented separately for retrospective (R), sub-acute (SA), early chronic (EC), and late chronic (LC) assessments with male (M) and female (F) middle childhood (MC) participants depicted in light color shades (M_MC: red; F_MC: cyan) and adolescent (AD) participants depicted in darker shades (M_AD: maroon; F_AD: blue).

Table 1.

Demographics and Injury Characteristic Data

	SA pmTBI (n = 286)	SA HC (n = 218)
Age	14(12-16)	13(11-16)
Sex (% female)	43.36%	44.50%
Tanner stage of development	3.4 ± 1.1	3.2 ± 1.3
Handedness (% right)	90.21%	89.00%
Parent BSI-18	5.5 ± 8.3	3.1 ± 5.0
pmTBI Hx	18.25%	5.96%
Injury characteristics
Loss of consciousness	32.98%	-
Post-traumatic amnesia	36.75%	-
Mechanism of injury
Struck by object	15.85%	-
Struck by person	20.42%	-
Fall	27.11%	-
MVC	27.82%	-
Assault	4.23%	-
Bicycle	4.23%	-
Other	0.35%	-
Sport or recreation related	59.86%	-

Data are either formatted as mean ± standard deviation or median (interquartile range) based on distribution properties.

SA, sub-acute; HC, healthy control; pmTBI, pediatric mild traumatic brain injury; BSI, Brief Symptom Inventory-18; MVC, motor vehicle crash; Hx, history.

Self-reported PCSI scores for the pmTBI cohort (Fig. 2C) indicated a significant main effect associated with visit (Wald-χ² = 75.80; p < 0.001; SA > R ≈ EC > LC), with a non-significant main effect of sex following Bonferroni correction (Wald-χ² = 4.86; p = 0.027; Cohen's d = 0.14; F > M). The pmTBI parental report PCSI scores (Fig. 2D) indicated a significant sex × age interaction (Wald-χ² = 8.02; p = 0.005), as well as main effects associated with both age (Wald-χ² = 8.97; p = 0.003; Cohen's d = 0.206; AD > MC) and visit (Wald-χ² = 112.33; p < 0.001; SA > (R|EC|LC); EC > LC). Follow-up analyses of the sex × age interaction indicated that parents of females rated significantly higher PCSI scores than parents of male adolescents (p < 0.001) in conjunction with similar male/female rating during middle childhood (p > 0.05).

Secondary analyses: Parent–child dyad PCSI magnitude results and sub-domain analyses

The next series of analyses directly compared child versus parent normalized PCSI scores using 2 × 2 × 2 × 4 (dyad × sex × age × visit) GEE analyses. Only the effects associated with dyad are reported to reduce duplication of results from primary analyses. Results for HC indicated a significant dyad × age interaction (Wald-χ² = 8.38; p = 0.004) and main effect for dyad (Wald-χ² = 38.71; p < 0.001; Cohen's d = 0.32; child > parent). Follow-up testing of the interaction suggested that the magnitude of difference between normalized child and parental PCSI ratings was greater for MC (p < 0.001; difference = 5.46%) than for AD (p = 0.018; difference = 1.69%). Full results from sub-domain (physical, cognitive emotional and fatigue) analyses for HC are presented in the Supplementary Text and Figure S1 (upper row). However, sub-domain analyses were largely similar to the results from the total score.

Results from the pmTBI cohort indicated significant dyad × age (Wald-χ² = 6.78; p = 0.009) and dyad × visit (Wald-χ² = 9.63; p = 0.022) interactions, as well as a main effect for dyad (Wald-χ² = 29.53; p < 0.001; Cohen's d = 0.24; child > parent). Similar to HC, follow-up testing of the dyad × age interaction suggested that child/parent differences in normalized PCSI ratings were larger for MC (p < 0.001; difference = 7.18%) than for AD (p = 0.017; difference = 3.16%) dyads. Subsequent testing for the dyad × visit interaction indicated that differences between child and parent dyads were highest for the retrospective PCSI rating (p < 0.001; difference = 6.80%), non-significant for the SA visit (p > 0.05), and then decreased in magnitude for both the EC (p = 0.007; difference = 4.00%) and LC (p = 0.035; difference = 2.99%) visits. Results from sub-domain analyses (see Supplementary Text and Fig. S1, lower row) were generally similar to the patterns observed with the total score.

Binary PCSI classification results

Figure 3 and Table S3 detail the percentage of HC and pmTBI categorized as being symptomatic at each study visit based on z-scores for self- and parental ratings, further stratified by age and sex. Statistical testing of data was not possible because of the large number of zero values (i.e., asymptomatic) following binary classification for HC (∼ 95% across sample) within multiple cohorts, as well as the presence of zero values for some pmTBI cohorts. From a qualitative perspective, MC females were generally more likely to be categorized as symptomatic than MC males based on self-report across all study visits (Fig. 3A). Although female HC were more likely than males to be categorized as symptomatic based on parental report, the differential was much higher for adolescent females. Similar results were obtained for self-report pmTBI data (Fig. 3B), indicating a higher percentage of symptomatic females at SA and EC visits relative to males. Parental binary classification results also indicated a high percentage of female adolescents classified as being symptomatic, extending all the way up to 1-year post-injury.

FIG. 3.

Percent symptomatic rates. Percent symptomatic rates based on binary classification are displayed for healthy controls (HC; Panel A) and patients with pediatric mild traumatic brain injury (pmTBI; Panel B). Percent symptomatic rates are separately calculated for sub-acute (SA), early chronic (EC), and late chronic (LC) assessments with male (M) and female (F) middle childhood (MC) participants depicted in light color shades (M_MC: red; F_MC: cyan) and adolescent (AD) participants depicted in darker shades (M_AD: maroon; F_AD: blue). Data for all child (All [C]: gray) and parental (All [P]: black) samples are presented in gray scale shades. The x-axis denotes the percentage of children binarily classified as symptomatic based on self or parent report ratings on the Post-Concussion Symptom Inventory (PCSI). Retrospective ratings were collected at the SA visit. The expected false positive rate for HC, defined as 5%, is noted in all panels with a dotted vertical line.

Inter-rater agreement³¹ for the binary classification of highly symptomatic individuals (Fig. 4 lower panels; Table S4) was excellent for the majority of HC parent–child dyads across all study visits, likely a result of the high proportion (∼ 95%) of asymptomatic individuals. In contrast, agreement for pmTBI dyads was generally only in the fair-to-good range, but varied greatly by age and sex within each visit. Specifically, inter-rater agreement improved from the SA to LC visits for all pmTBI dyads, with the exception of female adolescents (declined from good to fair). Notably, binary classification agreement was much higher than PCSI magnitude agreement among HC dyads across all study visits (compare top and bottom panels of Fig. 4A), whereas agreement varied for pmTBI dependent on visit (SA and EC visits: magnitude > binary; LC visit: binary > magnitude).

FIG. 4.

Inter-rater reliability and symptomatic status agreement. Inter-rater reliability between child–parent dyads on symptom magnitude (intraclass correlation coefficient [ICC; 2,k]; upper panels) and agreement on binary symptomatic status (Gwet's AC1 coefficient; lower panels) were calculated for healthy controls (HC; Panel A) and patients with pediatric mild traumatic brain injury (pmTBI; Panel B) using normalized scores from the Post-Concussion Symptom Inventory. ICC and AC1 values are presented separately for retrospective (R; ICC only), sub-acute (SA), early chronic (EC), and late chronic (LC) assessments. Male (M) and female (F) middle childhood (MC) participants are depicted in light color shades (M_MC: red; F_MC: cyan) whereas adolescent (AD) participants are depicted in darker shades (M_AD: maroon; F_AD: blue). A thicker black line also depicts values from the full sample. Bands representing inter-method agreement ranges (poor: red; fair: orange; good: yellow; excellent: green) run through the back of panels. All negative ICC values were transformed to a value of zero. Inter-rater agreement was generally in the poor range for HC, whereas increased agreement was noted for pmTBI dyads as a function of injury proximity (inverted U shape). In contrast, dyad agreement for classification of highly symptomatic individuals was generally in the excellent range for HC, but varied by sex and age between visits for pmTBI (SA and EC visits: magnitude > binary; LC visit: binary > magnitude).

Inter-rater reliability estimates

Inter-rater reliabilities (Fig. 4A; Table S5) for child–parent HC dyads were in the poor range for both MC and AD males for all study visits. Inter-rater reliability was higher (fair to good) for female than for male dyads for retrospective ratings in both age ranges, but then generally decreased into the poor range as a function of study visit. The only exception to this was the SA visit, for which inter-rater reliability was in the good range for the younger female dyads. Inter-rater reliabilities across the physical, cognitive, emotional, and fatigue sub-domains were similar to the total score for HC (see Supplementary Text and Fig. S2).

For the pmTBI cohort (Fig. 4B and Table S5), all dyads displayed a stereotypical pattern in which inter-rater reliability was poor at the retrospective visit, increased to the fair-to-excellent range for the SA visit, and then slowly declined to the fair or poor range across EC and LC visits. Global sex- or age-related differences were not as evident for pmTBI dyads. Inter-rater reliability was the highest for female AD dyads and the worst for female MC dyads. The general pattern of inter-rater reliabilities across the four sub-domains were generally similar to the total score (see Supplementary Text and Fig. S2).

Test-retest reliability of symptom measurements

Test-retest reliability results (Fig. 5) were calculated separately for both child and parent PCSI total ratings. Test-retest reliabilities for self-reported PCSI rating were higher for HC than for pmTBI, and generally displayed inverse time-dependent gradients across patients and controls. Within HC, test-retest reliability was generally in the good-to-excellent range between retrospective and SA ratings, and then generally declined to the poor-to-fair range. The only exception to this general trend was for adolescent males, who displayed good-to-excellent reliability between LC and retrospective/SA visits, respectively. A clear sex-based distinction was present for the parental PCSI ratings, in which test-retest reliability was higher for females (majority in the good-to-excellent range) than for males (all in the poor-to-fair range). Surprisingly, there was also minimal evidence for a time-dependent gradient (decreasing reliability as a function of visit) for parental ratings among males.

FIG. 5.

Test-retest reliability. Test-retest reliability for the Post-Concussion Symptom Inventory (PCSI) was quantified separately for healthy controls (HC; Panel A) and patients with pediatric mild traumatic brain injury (pmTBI; Panel B). The normalized total scores were compared in a pairwise fashion among retrospective (R), sub-acute (SA), early chronic (EC), and late chronic (LC) assessments. Within each table, test-retest reliability for child (C) self-report is presented above the diagonal whereas reliability for parental (P) report is presented below the diagonal. Separate tables are presented for males (M) and females (F), and middle childhood (MC) and adolescent (AD) cohorts, as well as for the total samples (All) of HC and pmTBI. All values are color-coded according to established test-retest reliability ranges (poor: red; fair: orange; good: yellow; excellent: green). All negative ICC values were transformed to a value of zero. As expected, test-retest reliability was highest for HC between R and SA visits, whereas pmTBI displayed the highest reliability between EC and LC visits.

Test-retest reliabilities were generally in the poor-to-fair ranges for self-reported PCSI ratings for all pmTBI cohorts (Fig. 5B). The only exception was reliability for MC females between the EC and LC visits, which was in the good range. As expected, there was slightly greater test-retest reliability observed between the EC and LC visits when symptoms were more likely to be stable. Parental test-retest reliabilities were similarly in the poor-to-fair range, exhibited minimal evidence of sex- or age-related differences, and were generally higher between EC and LC visits. The only exception was that parental test-retest reliability was slightly higher for MC females.

Discussion

Self- and parental report of PCS remain the diagnostic and prognostic gold standards for the field of pmTBI.^5
–7 The current study investigated the influence of host factors (sex and age at injury) on both self-report and parental ratings of PCS in separate cohorts of pmTBI and HC up to 1 year post-injury, as well as the putative impact of these same host factors on psychometric properties of test-retest and inter-rater (i.e., parent-child agreement) reliability. Current results suggest that both sex and age have significant impacts on concussive symptom reporting that are highly dependent on both the source (child vs. parent) and injury context (HC vs. pmTBI) in which they are obtained. In general, there was poor agreement between parental and child ratings across both cohorts, with the exception of the magnitude of symptoms during the sub-acute injury phase, as well as evidence of a possible bias toward the perception of increased PCS burden in females by parents. Both HC and their parents also demonstrated bias in the retrospective recall of PCS. Sub-domain (physical, cognitive, emotional, and fatigue) analyses generally followed the trends evident in the total score results for both self- and parental report. Test-retest reliability for PCS reporting was typically in the poor-to-fair range for both cohorts, but worse for males and for child self-report relative to parental ratings.

Scoping and systematic reviews of the literature^5,6 suggest that most studies^9,33

–36 rely on “simple change” scores to define persistent PCS rather than “standardized” methods such as normative approaches, regression-based indices, or reliable change indices.^37
–39 Importantly, simple change scores are inherently dependent on the retrospective recollection of a previous “baseline” state. Current findings highlight several potential issues with simple change approaches. First, retrospective recall is likely to be subject to multiple biases, and the nature of these biases may vary across groups. A bias toward increased PCS during retrospective ratings relative to SA, EC, and LC visits was observed for both self- and parental report among HC, which complicates the interpretation of “change.” In contrast, mTBI has been associated with the so-called “good-old days” bias⁴⁰ in which recollection of pre-morbid symptom burden is lower than typically reported. Collectively, these contrasting reporting biases may decrease the presence of “new” symptoms during simple change calculations for HC while increasing the likelihood of “new” symptom presentation for mTBI patients. Importantly, biases are also likely affected by how retrospective and concurrent symptoms are assessed, which varies across the instruments that are frequently used to measure PCS in pediatric samples.²¹ Second, it is possible that retrospective ratings may capture more trait-like characteristics of PCS reporting, whereas current ratings (as obtained at SA, EC, and LC visits) may be subject to state-like fluctuations in symptom burden caused by multiple environmental and health-related factors. Specifically, state-like fluctuations may affect test-retest reliability over time,⁶ and therefore may not be stable enough to be considered a true “new symptom” based on “simple change.”

Unlike adult studies, it is standard practice to obtain PCS ratings from both child and proxy (most typically parents) sources secondary to concerns about validity/reliability of child self-report.^10,11 However, multiple differences existed between parent and child PCS ratings. First, similar to in previous studies,^13
–15,18 both HC and pmTBI consistently self-reported their PCS symptom burden as being higher than their parents did. Second, parent–child inter-rater reliability agreement exhibited an inverted U-shaped function for pmTBI based on injury acuity, in which the highest agreement occurred immediately post-injury, partially replicating previous work.^6,12,13 Similar to in previous studies,¹⁸ inter-rater agreement was higher for female adolescents sub-acutely (excellent range), but was generally below acceptable ranges for all other visits regardless of age or sex. Parent–child inter-rater reliability for HC was also generally poor to fair when assessing symptom magnitude, with the exception of retrospective and sub-acute ratings for females (see subsequent discussion of binary PCS classification). Third, parents of both HC and pmTBI consistently rated adolescent females as having more PCS symptoms than males across both continuous and binary classification schemes. Although adolescent female pmTBI patients also self-reported increased PCS relative to males in both current and previous studies,^9,19,20 the magnitude of these sex-related differences was much smaller for self-report than for parental report for both pmTBI and HC.

These results collectively suggest that the level of parent–child agreement is strongly influenced by symptom magnitude, potentially a result of increased parent–child communication immediately post-injury. However, current results also indicate that parents and uninjured/recovered children are not rating the same psychological construct (i.e., PCS symptom load in children), which is paradoxically then used to monitor recovery in both clinical and research practice.⁹ This is best exemplified by parents rating 20% more female adolescents as being persistently symptomatic at 1 year post-injury relative to adolescent self-report (Fig. 3 and Table S3). Other research has demonstrated that parental ratings may also be influenced by cultural biases in which females are perceived to be more sensitive to pain⁴¹ and more symptomatic than their male counterparts based on self-report. Therefore, utilization of parental ratings in the studied age range (8–18 years old) may introduce more questions than value. Future research regarding utilization of self-report versus parental ratings of PCS is warranted.

Although early work suggested a monotonic increase in parent–child inter-rater reliability as a function of age,^12,17 more recent studies suggest that an inverted U shape may better characterize this relationship.^15,16 In contrast, increased parent–child agreement as a function of age only occurred for female patients with pmTBI in the current study, with no consistent age-related effects on parent–child agreement for HC. In terms of symptom magnitude, HC MC participants self-reported significantly higher symptom burdens than their AD counterparts, but parental ratings were equivalent across age groups. In contrast, there were no age-related differences in pmTBI self-report, with the exception of parents rating older females as being more symptomatic. It is possible that females may communicate more with their parents, resulting in generally higher agreement about PCS post-injury as well as a perception that females experience a higher PCS load. From a psychometric perspective, the PCSI uses an age-dependent rating scale for MC (smaller range on rating scale) relative to AD (larger range). Although this theoretically should have minimal effect on PCS scores after normalization, different scale ranges could bias age-related reporting results. In contrast, parental ratings are performed on the same scale regardless of the child's age, and no significant age-related differences were observed in parental PCS ratings. Therefore, studies that use the PCSI across different age ranges should also consider the reporting scale as a potential confounding psychometric explanation for age-related differences in PCS self-report.

Several studies have now systematically examined the prevalence of persistent PCS in children ≥1 year post-injury,^19,36,42,43 with a few reports suggesting that PCS may remain elevated up to 5 years post-injury.^36,43 Persistent PCS is typically classified in a binary fashion, and current findings suggest only a 5.5% drop in the number of pmTBI categorized as symptomatic at 1 year (14.4%) relative to 4 months (19.9%) based on self-report. Somewhat surprisingly, parents reported that an almost equivalent number of adolescent pmTBI females exhibited persistent symptoms at 1 year and 4 months post-injury. Although binary classifications seemingly improved child–parent agreement for HC, this is more likely representative of a statistical artifact caused by the large number of 0 values (∼ 95%) that increase the likelihood of high parent–child agreement (i.e., always guess 0) as demonstrated in previous studies.²⁴ In contrast to magnitude results, a similar trend was observed for pmTBI in which binary parent–child dyad agreement increased for most groups as a function of time as a result of recovery (i.e., an increased number of 0 ratings).

A priori hypotheses regarding opposite temporal gradients for test-retest reliability between the HC and pmTBI were generally supported. Within HC, the highest reliability was achieved between retrospective and sub-acute visits, which were rated in a temporally congruent fashion. Pairwise reliability then declined with increasing temporal gaps between ratings for females, whereas adolescent male test-retest reliability remained in the good-to-excellent range up to 1 year. Female test-retest reliability was markedly lower for self-report at all visits, suggestive of greater variation in symptom burden, whereas paradoxically, parental test-retest reliability was higher for females, suggestive of the perception of more stable/increased symptom burden. This further suggests that parents (i.e., perceived symptom burden) and children (i.e., experienced symptom burden) may be rating different psychological constructs, especially for females.

An opposing pattern was exhibited for pmTBI patients, with increased test-retest reliability between the early and late chronic follow-up visits. This pattern is expected because of increasing symptom stability following the resolution of the brain injury during the typical recovery process. There were no real sex-related differences in test-retest reliability for the pmTBI cohort. Collectively, the relative stability of overall symptom burden in HC on a sample/population level (i.e., no difference among SA, EC, and LC visits) coupled with poor-to-fair reliability at the individual level strongly argue for the use of normative rather than simple change methods⁶ to determine which individuals exhibit abnormal PCS following pmTBI.

Several limitations of the current study should be noted. Foremost, the sample size of each cohort was not consistent when stratified by sex and age (i.e., smaller for females and for MC). Although this is generally reflective of TBI epidemiology,² it may have limited our ability to detect smaller effects. This is especially true for more complex interactions, and the majority of significant findings in the current article were observed to be in the small effect size range (all Cohen's d < 0.35). Second, neither the sex of the reporting parent nor the number of siblings were used in analyses. Future studies should determine how parental sex interacts with child self-report of PCS symptoms. Third, we did not require the same parent to fill out the PCSI at each study visit, which may have introduced additional variability. Fourth, the current study slightly modified the PCSI (e.g., removal of references to injury) to make it more applicable to the general population, which may result in more general symptoms being reported by HC. Finally, PCS data is typically non-normally distributed (zero inflated; negative binomial distribution), which complicates the interpretation of statistical results for most psychometric properties.²⁴

Conclusion

In summary, although methodological concerns about defining PCS are not limited to children,⁴⁴ several unique challenges exist when assessing children. Current findings suggest that there is generally poor-to-fair test-retest reliability for PCS reporting in 8–18-year-old HC and pmTBI, as well as lower than acceptable levels of parent–child dyad agreement (i.e., <0.75), with the exception of adolescent females immediately post-injury. Parental ratings of perceived symptom burden appear to be relatively greater in female adolescents even when compared with experienced (i.e., self-report) symptoms, potentially secondary to cultural biases regarding the female sex or increased verbal communication in females. Several biases were also noted (i.e., higher retrospective ratings for both HC and their parents, lower parental relative to child ratings). These biases likely confound the implementation of persistent PCS diagnosis based on simple change methods when considered in conjunction with relatively low levels of test-retest reliability. The combination of these results supports the use of child self-report over parent ratings for estimating PCS burden, although continued research in this area is indicated. In conclusion, current results highlight the perils of relying on symptom self-report for diagnostic and prognostic purposes, as well as the importance of developing fluid⁴⁵ and imaging⁴⁶ biomarkers that can more objectively quantify injury severity.

Transparency, Rigor, and Reproducibility Summary

The current study was not pre-registered but was funded through two R01 awards from the National Institute of Neurological Disorders and Stroke. Portions of the abstract were previously published in Abstracts from The 40th Annual Symposium of the National Neurotrauma Society held between June 25 and 28, 2023 in Austin, Texas, with the abstract being entitled “Sex and age-related differences in child and parental reporting of persistent post-concussive symptoms” (doi: 10.1089/neu.2023.29130.abstracts). A total sample size of 291 patients with pmTBI and 224 HC were consecutively recruited from local emergency rooms and seen at SA, EC, and LC or equivalent time points. Study attrition occurred for 53 pmTBI and 19 HC for the EC visit. Study attrition occurred for an additional 29 pmTBI and 15 HC for the LC visit. A total of 48 participants were not yet eligible for their LC visit at the time of manuscript preparation during this ongoing study. Inclusion criteria for pmTBI was based on published criteria (American Congress of Rehabilitation Medicine [ACRM] and Zurich) and was independently verified by at least two clinicians prior to study enrollment. Participants were provided with a copy of their radiological findings, but were blinded to the results of their clinical assessments. Data were analyzed separately for HC and pmTBI using SPSS version 20.0 using information criterion tested distributions. Effect sizes are reported to enhance transparency, with primary results corrected for multiple comparisons. All surveys and questionnaires used in the current study are publicly available. The current study replicated previous findings in a subset of the cohort while also examining the effect of host factors (biological sex and age at injury) on key psychometric properties of the post-concussive symptom reporting. All data from this study will be publicly available in the Federal Interagency Traumatic Brain Injury Research (FITBIR) database upon completion of the study, or can be obtained sooner by requesting it from the study principle investigator. The results of this publication will be freely available through the journal Web site as part of funding requirements.

Footnotes

Authors' Contributions

Andrew R. Mayer was responsible for conceptualization, methodology, investigation, supervision, funding acquisition, data curation, formal analysis, and writing – original draft. Jessica R. McQuaid was responsible for conceptualization, data curation, formal analysis, and writing – original draft. Tracey V. Wick and Andrew B. Dodd were responsible for data curation, formal analysis, and writing – original draft. Cidney R. Robertson-Benta, David D. Stephenson, Harm J. van der Horn, Davin K. Quinn, William A. Davis, Anne K. Hittson Robert E. Sapien, John P. Phillips, and Richard A. Campbell were responsible for formal analysis, and writing – review and editing.

Funding Information and Data Availability

This research was supported by grants from the National Institutes of Health (NIH) (https://www.nih.gov; grant numbers NIH 01 R01 NS098494-01A1, R01 NS098494-03S1A1, and P30 GM122734) to Andrew R. Mayer. The NIH had no role in study review, data collection and analysis, decision to publish, or preparation of the manuscript. The data that support the findings of this study will be openly available in FITBIR at fitbir.nih.gov, reference number FITBIR-STUDY0000339 at the conclusion of this study.

Author Disclosure Statement

No competing financial interests exist.

Supplementary Material

Supplementary Figure S1

Supplementary Figure S2

Supplementary Table S1

Supplementary Table S2

Supplementary Table S3

Supplementary Table S4

Supplementary Table S5

Supplementary Text

References

Rausa

, Anderson

, Babl

, et al. Predicting concussion recovery in children and adolescents in the emergency department. Curr Neurol Neurosci Rep, 2018; 18(11):78; doi: 10.1007/s11910-018-0881-z

Taylor

, Bell

, Breiding

, et al. Traumatic brain injury-related emergency department visits, hospitalizations, and deaths – United States, 2007 and 2013. MMWR Surveill Summ, 2017; 66(9):1–16. 10.15585/mmwr.ss6609a1

American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). American Psychiatric Association: Arlington, VA; 2013.

World Health Organization. International statistical classification of diseases and related health problems. World Health Organization: Geneva, Switzerland; 2004.

Chadwick

, Sharma

, Madigan

, et al. Classification criteria and rates of persistent postconcussive symptoms in children: a systematic review and meta-analysis. J Pediatr, 2022; 246:131–137; doi: 10.1016/j.jpeds.2022.03.039

Mayer

, Stephenson

, Dodd

, et al. Comparison of methods for classifying persistent post-concussive symptoms in children. J Neurotrauma, 2020; 37(13):1504–1511; doi: 10.1089/neu.2019.6805

Quinn

, Mayer

, Master

, et al. Prolonged postconcussive symptoms. Am J Psychiatry, 2018; 175(2):103–111; doi: 10.1176/appi.ajp.2017.17020235

Dennis

, Baron

, Bartnik-Olson

, et al. ENIGMA brain injury: framework, challenges, and opportunities. Hum Brain Mapp, 2020; 43(1):149–166; doi: 10.1002/hbm.25046

Zemek

, Barrowman

, Freedman

, et al. Clinical risk score for persistent postconcussion symptoms among children with acute concussion in the ED. JAMA, 2016; 315(10):1014–1025; doi: 10.1001/jama.2016.1203

10.

Gioia

, Schneider

, Vaughan

, et al. Which symptom assessments and approaches are uniquely appropriate for paediatric concussion?. Br J Sports Med, 2009; 43(Suppl 1):i13–i22; doi: 10.1136/bjsm.2009.058255

11.

Sady

, Vaughan

, Gioia

. Psychometric characteristics of the postconcussion symptom inventory in children and adolescents. Arch Clin Neuropsychol, 2014; 29(4):348–363; doi: 10.1093/arclin/acu014

12.

Gagnon

, Teel

, Gioia

, et al. Parent–child agreement on postconcussion symptoms in the acute postinjury period. Pediatrics, 2020; 146(1):e2019231; doi: 10.1542/peds.2019-2317

13.

Rowhani-Rahbar

, Chrisman

, Drescher

, et al. Agreement between high school athletes and their parents on reporting athletic events and concussion symptoms. J Neurotrauma, 2016; 33(8):784–791; doi: 10.1089/neu.2015.4100

14.

Hajek

, Yeates

, Taylor

, et al. Agreement between parents and children on ratings of post-concussive symptoms following mild traumatic brain injury. Child Neuropsychol, 2011; 17(1):17–33; doi: 10.1080/09297049.2010.495058

15.

Patsimas

, Howell

, Potter

, et al. Concussion-symptom rating correlation between pediatric patients and their parents. J Athl Train, 2020; 55(10):1020–1026; doi: 10.4085/1062-6050-200-19

16.

Vollmer

, Kirkwood

, Comstock

, et al. Assessing the clinical utility of the question, “Is your child/are you back to normal?” in pediatric concussion symptom resolution. Clin Pediatr (Phila), 2018; 57(2):146–151; doi: 10.1177/0009922817693300

17.

Elbin

, Knox

, Kegel

, et al. Assessing symptoms in adolescents following sport-related concussion: a comparison of four different approaches. Appl Neuropsychol Child, 2016; 5(4):294–302; doi: 10.1080/21622965.2015.1077334

18.

Johnson

, McCarty

, Marcynyszyn

, et al. Child-compared with parent-report ratings on psychosocial measures following a mild traumatic brain injury among youth with persistent post-concussion symptoms. Brain Inj, 2021; 35(5):574–586; doi: 10.1080/02699052.2021.1889663

19.

Ewing-Cobbs

, Cox

Jr ., Clark

, et al. Persistent postconcussion symptoms after injury. Pediatrics, 2018; 142(5):e20180939; doi:10.1542/peds.2018-0939

20.

Rosenbaum

, Locandro

, Chrisman

SPD

, et al. Characteristics of pediatric mild traumatic brain injury and recovery in a concussion clinic population. JAMA Netw Open, 2020; 3(11):e2021463; doi: 10.1001/jamanetworkopen.2020.21463

21.

Zemek

, Farion

, Sampson

, et al. Prognosticators of persistent symptoms following pediatric concussion: a systematic review. JAMA Pediatr, 2013; 167(3):259–265; doi: 10.1001/2013.jamapediatrics.216

22.

Gioia

, Collins

, Isquith

. Improving identification and diagnosis of mild traumatic brain injury with evidence: psychometric support for the acute concussion evaluation. J Head Trauma Rehabil, 2008; 23(4):230–242; doi: 10.1097/01.HTR.0000327255.38881.ca

23.

Mayer

, Bedrick

, Ling

, et al. Methods for identifying subject-specific abnormalities in neuroimaging data. Hum Brain Mapp, 2014; 35(11):5457–5470; doi: 10.1002/hbm.22563

24.

Hergert

, Sicard

, Stephenson

, et al. Test-retest reliability of a semi-structured interview to aid in pediatric traumatic brain injury diagnosis. J Int Neuropsychol Soc, 2022; 28(7):687–699; doi: 10.1017/S1355617721000928

25.

Mayer

, Ling

, Dodd

, et al. Multicompartmental models and diffusion abnormalities in paediatric mild traumatic brain injury. Brain, 2022; 145(11):4124–4137; doi: 10.1093/brain/awac221

26.

Mayer

, Meier

, Dodd

, et al. Prospective study of gray matter atrophy following pediatric mild traumatic brain injury. Neurology, 2023; 100(5):e516–e527; doi: 10.1212/WNL.0000000000201470

27.

Child Development – Positive Parenting Tips. Centers for Disease Control and Prevention: 2021. Available from: https://www.cdc.gov/ncbddd/childdevelopment/positiveparenting/index.html.

28.

Berk

Development through the Lifespan. Pearson: Boston; 2014.

29.

Berk

, Meyers

. Infants, children, and adolescents. Pearson: Boston; 2016.

30.

Shrout

, Fleiss

. Intraclass correlations: uses in assessing rater reliability. Psychol Bull, 1979; 86(2):420–428; doi: 10.1037/0033-2909.86.2.420

31.

Gwet

. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol, 2008; 61(Pt 1):29–48; doi: 10.1348/000711006X126600

32.

Cicchetti

. The precision of reliability and validity estimates re-visited: distinguishing between clinical and statistical significance of sample size requirements. J Clin Exp Neuropsychol, 2001; 23(5):695–700; doi: 10.1076/jcen.23.5.695.1249

33.

Hearps

, Takagi

, Babl

, et al. Validation of a score to determine time to postconcussive recovery. Pediatrics, 2017; 139(2):e20162003; doi: 10.1542/peds.2016-2003

34.

Bressan

, Takagi

, Anderson

, et al. Protocol for a prospective, longitudinal, cohort study of postconcussive symptoms in children: the Take C.A.Re (Concussion Assessment and Recovery Research) study. BMJ Open, 2016; 6(1):e009427; doi: 10.1136/bmjopen-2015-009427

35.

Grubenhoff

, Deakyne

, Brou

, et al. Acute concussion symptom severity and delayed symptom resolution. Pediatrics, 2014; 134(1):54–62; doi: 10.1542/peds.2013-2988

36.

Fried

, Balla

, Catalogna

, et al. Persistent post-concussive syndrome in children after mild traumatic brain injury is prevalent and vastly underdiagnosed. Sci Rep, 2022; 12(1):4364; doi: 10.1038/s41598-022-08302-0

37.

Iverson

, Lovell

, Collins

. Interpreting change on ImPACT following sport concussion. Clin Neuropsychol, 2003; 17(4):460–467; doi: 10.1076/clin.17.4.460.27934

38.

Merritt

, Bradson

, Meyer

, et al. Evaluating the test-retest reliability of symptom indices associated with the ImPACT post-concussion symptom scale (PCSS). J Clin Exp Neuropsychol, 2018; 40(4):377–388; doi: 10.1080/13803395.2017.1353590

39.

Barker-Collo

, Theadom

, Jones

, et al. reliable individual change in post concussive symptoms in the year following mild traumatic brain injury: data from the Longitudinal, Population-based Brain Injury Incidence and Outcomes New Zealand in the Community (Bionic) Study. JSM Burns and Trauma, 2016; 1(1):1006.

40.

Gunstad

, Suhr

. “Expectation as etiology” versus “the good old days”: postconcussion syndrome symptom reporting in athletes, headache sufferers, and depressed individuals. J Int Neuropsychol Soc, 2001; 7(3):323–333; doi: 10.1017/s1355617701733061

41.

Myers

, Riley

, 3rd, Robinson

. Psychosocial contributions to sex-correlated differences in pain. Clin J Pain, 2003; 19(4):225–232; doi: 10.1097/00002508-200307000-00005

42.

Barlow

, Crawford

, Stevenson

, et al. Epidemiology of postconcussion syndrome in pediatric mild traumatic brain injury. Pediatrics, 2010; 126(2):e374-81; doi: 10.1542/peds.2009-0925

43.

Starkey

, Jones

, Case

, et al. Post-concussive symptoms after a mild traumatic brain injury during childhood and adolescence. Brain Inj, 2018; 32(5):617–626; doi: 10.1080/02699052.2018.1439533

44.

Karaliute

, Saksvik

, Smevik

, et al. Methodology matters: comparing approaches for defining persistent symptoms after mild traumatic brain injury. Neurotrauma Rep, 2021; 2(1):603–617; doi: 10.1089/neur.2021.0028

45.

Mannix

, Levy

, Zemek

, et al. Fluid biomarkers of pediatric mild traumatic brain injury: a systematic review. J Neurotrauma, 2020; 37(19):2029–2044; doi: 10.1089/neu.2019.6956

46.

Mayer

, Kaushal

, Dodd

, et al. Advanced biomarkers of pediatric mild traumatic brain injury: progress and perils. Neurosci Biobehav Rev, 2018; 94:149–165; doi: 10.1016/j.neubiorev.2018.08.002

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.05 MB