How Consistent Is Sluggish Cognitive Tempo Across Occasions,Sources,and Settings? Evidence From Latent State

Abstract

Research has yet to determine how much of the variance in sluggish cognitive tempo (SCT) symptom ratings is consistent across occasions, sources, and settings versus specific to occasion, source, and setting. Our first objective was to determine the amount of variance in SCT ratings that was consistent (trait consistency) across three occasions of measurement over 12 months versus specific to the occasion (occasion-specificity) with ratings by mothers, fathers, primary teachers, and secondary teachers of 811 Spanish children. Our second objective was then to determine the convergent validity for trait consistency and occasion-specificity variance components within and across settings. SCT ratings reflected mostly trait consistency for mothers, fathers, and primary teachers (less so for secondary teachers) with the convergent validity for trait consistency also being strong for mothers with fathers and for primary teachers with secondary teachers. Across home and school, however, convergent validity for trait consistency was low and even lower for occasion-specificity. SCT appears to be more trait-like rather than state-like, with similar levels of trait consistency across occasions and convergent validity within settings as attention-deficit/hyperactivity disorder (ADHD) symptoms in a prior study. However, SCT symptoms had slightly weaker convergent validity for trait consistency across settings relative to ADHD symptoms.

Keywords

attention problems sluggish cognitive tempo longitudinal research latent state–trait models

Sluggish cognitive tempo (SCT) is characterized by inconsistent alertness, slow thinking/behavior, and drowsiness. A recent meta-analysis identified SCT symptoms that loaded higher on an SCT factor than an attention-deficit/hyperactivity disorder–inattention (ADHD-IN) factor with preliminary evidence that SCT is also distinct from anxiety, depression, and daytime sleepiness (Becker, Leopold, et al., 2016). In addition, the meta-analysis found higher levels of SCT predicted higher levels of anxiety, depression, academic impairment, and social impairment.

SCT also has trait-like qualities (high test–retest correlations and moderate to high interrater correlations). In terms of test–retest correlations, SCT showed 1- to 12-month stability coefficients from .80 to .92 (Becker, Leopold, et al., 2016, Table 4; Khadka, Burns, & Becker, 2016), 2-year stability coefficients of .60 for mothers’, .65 for fathers’, and .45 for teachers’ ratings (different teachers across occasions; Bernad, Servera, Becker, & Burns, 2016), and a 10-year stability coefficient of .43 for mothers’ ratings (Leopold et al., 2016). In terms of interrater correlations, correlations for raters within the same setting ranged from .71 to .80 (Becker, Leopold, et al., 2016; Burns, Becker, Servera, Bernad, & García-Banda, 2016) with average home-to-school interrater correlations being .54 (Becker, Leopold, et al., 2016, Table 4). The test–retest and interrater correlations for SCT are also similar to those of ADHD-IN (Willcutt et al., 2012).

Utility of Latent State–Trait Measurement Models

Although there is some indication that family and other environmental stressors are associated with SCT (Becker, Leopold, et al., 2016; Becker, Withrow, et al., 2016), test–retest and interrater correlations suggest that SCT is a more trait-like (i.e., consistent across occasions, sources, and settings) than state-like (i.e., specific to the occasion, source, or setting) construct. Latent state–trait (LST) measurement models, however, can better test if SCT is a more trait- or state-like construct. We now describe LST models.

Single-Source LST Models

A single-source LST model allows a direct test of the consistency across measurement occasions question by decomposing the observed score variance in SCT symptom ratings into true score variance and error variance and then further decomposing the true score variance into trait (trait consistency [consistency across occasions]) and state residual (occasion-specificity) variance (Steyer, Geiser, & Fiege, 2012). Trait variance represents variance that is person-specific and independent of occasion and/or the person–occasion interaction. In contrast, occasion-specific variance represents occasion-specific influences and/or person–occasion interactions.

If trait variance (consistency across occasions) in SCT symptom ratings is higher than occasion-specific variance for various raters, such findings would indicate that SCT symptoms are more trait- than state-like across occasions of measurement for each source. If occasion-specific variance is larger than trait variance, then such would indicate that SCT symptoms represented more of an occasion-specific state for the individual sources. Whether SCT is more state- (occasion-specific) or trait-like (consistent across occasions) has important theoretical and clinical implications. If SCT is more trait-like, it will be important for future research to examine linkages with individual differences that are generally stable, such as temperament and personality, as well as the familial aggregation and heritability of SCT. If SCT is more state-like, it will be important to assess and understand situational stressors that may contribute to the expression of SCT.

To summarize, the single-source LST model can determine if SCT ratings are more trait- or state-like across multiple occasions of measurement for an individual source. Figure 1 shows this LST model. Supplemental Table S1 (available online at http://journals.sagepub.com/home/asm/supplemental) shows the equations for this model.

Figure 1.

Single source latent state-trait (LST) model with parcel-specific trait factors measured at three occasions. This model was applied to each of the four sources separately. T = trait factor; O = state (occasion-specific) residual factor. This figure was adapted with permission from Litson et al. (2016).

Multiple-Source LST Models

Although moderate to high inter-rater correlations among different sources for SCT ratings indicate convergent validity, it is not known if such consistency reflects convergent validity at the trait level (i.e., shared trait variance), convergent validity at the occasion-specific level (i.e., shared occasion-specific variance), or a combination of both. A multiple-source LST measurement model allows an answer to this question by determining the amount of trait (trait consistency) and occasion-specific variance that is either shared or not shared between a reference source (e.g., mothers) and nonreference sources (e.g., fathers, primary teachers, and secondary teachers; Courvoisier, Nussbeck, Eid, Geiser, & Cole, 2008). Figure 2 shows our multiple-source LST model and Supplemental Table S1 the equations for this model. We now describe the variance components of this model.

Figure 2.

Multiple source latent state-trait (LST) model with parcel-specific trait factors. In order to reduce the complexity of the figure, only three sources (mothers, fathers, and primary teachers) and two occasions are shown whereas the actual model involved four sources (mothers, fathers, primary teachers, and secondary teachers) and three occasions. T = reference trait factors that are defined by mother reports and are parcel-specific across sources; TS = residual trait factors that pertain to the non-reference sources and are source- and parcel-specific; O = reference state residual factors that are defined by mother reports and are parcel-specific across sources; OS = state residual factors that pertain to the non-reference sources and are source-specific across parcels. This figure was adapted with permission from Litson et al. (2016).

The first variance component is shared trait consistency, which is the proportion of trait variance in SCT ratings that a nonreference source shares with the reference source (i.e., convergent validity for trait variance [trait consistency]). The second is unique trait variance, which is the proportion of trait variance in SCT ratings that a nonreference source does not share with the reference source. Unique trait variance represents the degree of source specificity for trait aspects of SCT ratings (discrepancy in trait variance). The third is shared occasion-specificity, which is the proportion of occasion-specific variance in SCT ratings that a nonreference source shares with the reference source (the convergent validity for the occasion-specific aspects of SCT). The fourth is unique occasion-specificity. This component represents occasion-specific variance in SCT ratings that is unique to a nonreference source, thus not shared with the reference source.

The multiple-source LST model also allows one to determine convergent validity of residual (source-specific) trait variance among the nonreference sources (i.e., fathers with primary and secondary teachers, primary teachers with secondary teachers). Residual trait variance is trait variance for nonreference sources (fathers, primary teachers, and secondary teachers) that is independent of trait variance for the reference source (mothers) and is reflected in trait residual factors (TS correlations in Figure 2). Correlations between these factors allow one to determine the convergent validity between fathers and teachers as well as primary teachers with secondary teachers after partialing out what these three sources share with mothers, thus providing additional information on whether the nonreference sources share a common perspective that deviates from the reference source (mothers). Similar analyses can be applied with the source-specific occasion-specific variance for the nonreference sources (OS correlations in Figure 2).

If nonreference sources share a high proportion of trait variance (trait consistency) in SCT ratings with the reference source, then such findings would indicate convergent validity of trait variance within (i.e., fathers with mothers) and across (i.e., primary and secondary teachers with mothers) settings. If nonreference sources share a high proportion of occasion-specific variance in SCT ratings with the reference source, then such would provide support for convergent validity of the occasion-specific variance within and across settings. The lack of shared trait or occasion-specific variance between nonreference sources and the reference source would indicate a lack of convergent validity for both trait and occasion-specific variance (i.e., consistent [trait] variance was source-specific and occasion-specific variance was source-specific at a specific occasion). To summarize, the multiple-source LST model provides a novel way to study informant agreements and disagreements for trait and occasion-specific variance in symptom ratings as well as to identify correlates of the shared trait consistency and unique trait consistency (see also De Los Reyes, 2013; De Los Reyes, Alfano, Lau, Augenstein, & Borelli, 2016; De Los Reyes et al., 2015). We elaborate on this in the Discussion.

Trait and Occasion-Specific Variance in ADHD-IN, ADHD-HI, and Oppositional Defiant Disorder (ODD) Symptoms

Two recent studies applied single and multiple-source LST models to ADHD-IN, ADHD-hyperactivity/impulsivity (HI), and ODD symptom ratings in Spanish children (Litson, Geiser, Burns, & Servera, 2016; Preszler, Burns, Litson, Geiser, & Servera, 2016). Each study involved the same four sources (mothers, fathers, primary teachers, and secondary teachers) rating Spanish children in the first and second grades. ADHD-IN, ADHD-HI, and ODD symptom ratings contained a moderate (53%) to substantial (78%) amount of trait variance for the four sources. The results also showed strong convergent validity for trait variance for ADHD-IN, ADHD-HI, and ODD ratings within settings (68% to 80%) along with low (ODD: approximately 32%) to moderate (ADHD-IN and ADHD-HI: approximately 50%) across-setting convergent validity for trait variance. It is currently unknown how SCT symptom ratings will compare with the results on ADHD and ODD symptom ratings. It is especially important to determine if SCT symptom ratings are as trait-like as the ADHD-IN symptom ratings (i.e., Do SCT symptom ratings represent a trait as much as ADHD-IN symptom ratings?). Given the analyses on the SCT symptoms involved the same four sources and Spanish children, a direct comparison of the earlier results on ADHD-IN symptoms to SCT symptoms was possible.

Study Objectives

This study used ratings of SCT symptoms by mothers, fathers, primary teachers, and secondary teachers of Spanish children in the first and second grades to address two objectives. The first objective was to determine the proportion of trait variance (trait consistency) and occasion-specific variance (occasion-specificity) in SCT ratings for mothers, fathers, primary teachers, and secondary teachers. This objective required that a single-source LST model be fit to the ratings for each source separately (Figure 1). It was predicted that SCT symptoms would show larger amount of trait (consistency over occasions) than occasion-specific variance for each source (approximately 70% trait variance if the findings are similar to the ADHD-IN symptom ratings; Litson et al., 2016). This outcome would indicate that the SCT symptoms were more trait- than state-like for each source over the three occasions of measurement for the one-year interval.

The second objective was to determine the proportion of trait and occasion-specific variance in SCT ratings that fathers, primary teachers, and secondary teachers either shared or did not share with mothers. This objective required that a multiple source LST model be fit to the ratings of the four sources simultaneously (Figure 2). Eight predictions were made for this analysis. The first prediction was that fathers would share a large amount of trait variance with mothers (approximately 80% if similar to ADHD-IN results). The second prediction was that fathers would share a much smaller amount of occasion-specific variance with mothers (approximately 40% if similar to ADHD-IN results). The third prediction was that primary and secondary teachers would only share a moderate amount of trait variance with mothers (approximately 50% if similar to ADHD-IN results) while the fourth prediction was that primary and secondary teachers would share no occasion-specific variance with mothers (approximately 0% if similar to ADHD-IN results).

The fifth and sixth predictions dealt with correlations among residual trait factors for fathers, primary teachers, and secondary teachers. (Residual trait correlations indicate whether there is convergent validity for residual trait variance among father, primary teacher, and secondary teacher reports after accounting for trait variance that these sources share with mother reports.) The fifth prediction was that fathers would share approximately 0% residual trait variance with primary and secondary teachers, whereas the sixth prediction was that primary and secondary teachers would share substantial residual trait variance with each other (approximately 80% if similar to ADHD-IN results).

The seventh and eighth predictions dealt with the correlations among the residual occasion-specific factors for fathers, primary teachers, and secondary teachers. Residual occasion-specific correlations indicate whether there is convergent validity for the residual occasion-specific variance among father, primary teacher, and secondary teacher reports after accounting for variance that these sources share with mother reports. The seventh prediction was that fathers would share approximately 0% residual occasion-specific variance with primary and secondary teachers, and the eighth prediction was that primary and secondary teachers would share a very small amount of residual occasion-specific variance with each other. Support for these eight predictions would indicate strong convergent validity of trait variance within the home and school settings, moderate convergent validity of trait variance across the two settings, low convergent validity for occasion-specific variance within settings, and zero convergent validity for occasion-specific variance across settings.

This is the first study to apply single- and multiple-source LST models to SCT ratings. The findings will indicate if SCT symptoms reflect a more trait- or state-like construct for mothers, fathers, primary teachers, and secondary teachers as well as the degree of convergent validity of trait and occasion-specific variance within and across settings. These results will also us to determine if SCT symptoms are as trait-like as ADHD-IN symptoms by a comparison with the earlier results for the ADHD-IN symptoms, an important question given the recent interest in whether SCT represents a unique attention disorder (Barkley, 2016; Becker, Leopold, et al., 2016).

Method

Participants and Procedures

Forty-three of 46 elementary schools on Majorca (Spain) volunteered with 22 of these 43 schools randomly selected to participate (resources on Majorca only allowed for data collection at 22 schools) with eight additional schools recruited from Madrid (eight were asked and eight agreed to participate). The eight schools were recruited to increase sample size. Potential participants were mothers, fathers, primary teachers (children’s main classroom teacher), and secondary teachers (children’s teachers of special subjects such as English, Catalan language, music, visual arts, and physical education) of 1,045 first grade children. The study involved 811 unique children (54% boys) across the three assessments (i.e., [a] spring semester of the first grade, [b] 6-weeks after the first assessment, and [c] spring semester of the second grade). There were 758 unique children at the first assessment, 746 at the second assessment, and 718 at the third assessment. Children had the same primary teachers for first and second grades (first and second grades were together in the same room) with each child being rated by the same primary teacher across the three assessments. Secondary teachers were the same for the first and second assessments yet could have been different for the third assessment. A cover letter that explained the purpose was given to parents, and, with parental written consent, a similar cover letter was given to teachers. Teachers also provided written consent. The institutional review board approved the protocol.

For the first grade assessment, 723 mothers and 603 fathers returned the measures with the numbers for the second assessment being 667 mothers and 584 fathers and for the third assessment, 604 mothers and 540 fathers. At the first assessment, 61 primary teachers and 49 secondary teachers from 28 of 30 schools participated with primary teachers rating an average of 11.76 (SD = 5.09, n = 743) and secondary teachers, an average of 9.02 (SD = 6.58, n = 574) children. For the second assessment, 58 primary teachers and 51 secondary teachers participated with primary teachers rating an average of 10.93 (SD = 5.27, n = 679) and secondary teachers, an average of 9.93 (SD = 6.45, n = 623) children. For the third assessment, teachers from 29 of 30 schools participated with 62 primary teachers rating an average of 11.19 (SD = 4.43, n = 701) and 59 secondary teachers, an average of 10.45 (SD = 5.18, n = 664) children. At the first assessment, the average age of the children was 7 years with little variation. Although ethnicity was not collected for individual children, school-level data indicated that approximately 90% of the children were Caucasian and 10% were North African. Socioeconomic status information was not available. Although the ADHD symptom ratings were not the focus of the current study, 7.5% of the sample (range = 6% to 9% across the four sources) showed ADHD-IN and/or ADHD-HI symptoms in the clinical range (i.e., six of nine symptoms). A rating of “often” or higher was used to consider an ADHD symptom as qualitatively present. Given these procedures do not include the other ADHD diagnostic criteria (i.e., symptoms across settings, functional impairment, symptoms not better accounted for by another disorder), the actual number of children in the sample meeting full Diagnostic and Statistical Manual of Mental Disorders criteria for ADHD would likely be less than these percentages. Supplemental Table S2 shows the descriptive information for the ADHD-IN and ADHD-HI scales.

Measure

Child and Adolescent Disruptive Behavior Inventory (CADBI; Burns, Lee, Becker, Servera, & McBurnett, 2014)

The study used the eight SCT items from the CADBI. Table 1 shows the eight SCT symptoms.

Table 1.

Parcel Assignment of Sluggish Cognitive Tempo Symptoms.

Parcel 1

1. Loses train of thought during homework or home activities such as chores (e.g., forgets what he or she was about to say, forgets what he or she was looking for, loses his or her place when working on an activity)

2. Easily confused during homework or home activities such as chores (e.g., gets mixed up trying to follow the instructions for an activity, becomes confused when the directions for an activity are a little complicated)

3. Looks drowsy during homework or home activities such as chores even when he or she has had a good night sleep (e.g., seems sleepy and tired, yawns, i.e., drowsiness is NOT due to sleep problems at night)

Parcel 2

1. Daydreams during homework or home activities such as chores (e.g., stares off into space during activities, lost in his or her own thoughts during activities rather than paying attention to the activity)

2. Absent-minded during homework or home activities such as chores (e.g., puts something down during activities and then immediately forgets where he or she puts it; becomes so absorbed in one thought that he or she does not attend to the current activity)

3. Behavior is slow during homework or home activities such as chores (e.g., moves at a “snail’s pace,” behavior is as “slow as molasses,” last to finish the activity, slow at routine activities)

Parcel 3

1. Alertness changes from moment to moment during homework or home activities such as chores (e.g., spaces or zones out during activities, brief lapses of attention during activities)

2. Thinking seems slow during homework or home activities as chores (e.g., takes longer than you expect to respond to your questions even when you have his or her attention, slow to make decisions or choices)

Note. The eight Sluggish Cognitive Tempo items on the teacher rating scale were the same except classroom activities was substituted for during homework or home activities.

SCT symptoms were rated on a 6-point scale (almost never [never or about once per month], seldom [about once per week], sometimes [several times per week], often [about once per day], very often [several times per day], and almost always [many times per day]). Mothers and fathers were asked to make their ratings independently on the children’s behavior in home and community and not consider reports of behavior at school. Primary teachers and secondary teachers were instructed to make their ratings independently on school behavior. Earlier studies support the reliability and validity of scores on the SCT scale with ratings by mothers, fathers, and teachers from Chile, Korea, Nepal, Spain, and the United States (Belmar, Servera, Becker, & Burns, 2015; Burns et al., 2016; Khadka et al., 2016; Lee, Burns, & Becker, 2016a, 2016b; Lee, Burns, Snell, & McBurnett, 2014). Table 2 shows the descriptive information for the SCT scale in this study.

Table 2.

Descriptive Information for Sluggish Cognitive Tempo Scale.

	n	α	M	SD	Range
Occasion 1
Mothers	723	.87	0.90	0.79	0.00, 4.75
Fathers	603	.87	0.84	0.74	0.00, 4.50
Primary teachers	743	.95	1.02	1.12	0.00, 5.00
Secondary teachers	574	.95	0.85	1.06	0.00, 5.00
Occasion 2
Mothers	667	.89	0.86	0.75	0.00, 4.00
Fathers	584	.89	0.85	0.75	0.00, 3.75
Primary teachers	679	.95	0.90	1.02	0.00, 4.63
Secondary teachers	623	.95	0.89	1.09	0.00, 5.00
Occasion 3
Mothers	604	.88	0.89	0.77	0.00, 3.63
Fathers	540	.88	0.84	0.74	0.00, 4.38
Primary teachers	701	.94	0.84	1.03	0.00, 5.00
Secondary teachers	664	.94	0.83	1.00	0.00, 5.00

Note. The range for the scale was from 0.00 to 5.00. α = Cronbach’s alpha.

Analyses

Analytic Strategy

For item-level analyses, items were treated as ordered categories, using the robust weighted least squares estimator (WLSMV estimator, Mplus version 7.4). For the parcel-level analyses, the study used robust maximum likelihood estimation (MLR estimator). The MLR estimator uses all the available information for the analyses and also accounts for any nonnormality. Parcels were used in the more complex LST models to reduce the number of indicators (i.e., 36 indictors rather than 96). Measurement models were evaluated with comparative fit index (CFI; ideal study criterion ≥.95), the Tucker–Lewis Index (TLI; ideal study criterion ≥.95), and root mean square error of approximation (RMSEA; ideal study criterion ≤.05). Decrease in CFI was used to evaluate the invariance analyses for like-item loadings and thresholds as well as like-item parcels and intercepts. If the decrease in CFI was ≤.01, then the invariance constraints imposed at the step were assumed to be tenable (Little, 2013, chap. 5). Given children were clustered within classes, the Mplus type = complex option was used to correct the standard errors.

Preliminary Analyses on Individual Items to Justify Creation of Parcels

A multiple-indicator (eight SCT symptoms) by multiple-occasion (three assessments) by multiple-source (mothers, fathers, primary teachers, and secondary teachers) confirmatory factor analysis (CFA) was used to evaluate the invariance of like-item loadings and thresholds across sources and occasions. If constraints on like-symptom loadings and thresholds hold across the three occasions and four sources, then this outcome yields one set of eight unstandardized loadings rather 12 sets of eight unstandardized loadings. The single set of eight SCT symptom factor loadings was then used to create parcels for LST analyses. As noted earlier, the use of parcels reduces the number of indicators in the LST models from 96 to 36, thus making these complex analyses possible.

The SCT items were assigned to parcels in this manner. The eight unstandardized loadings were first ranked from highest to lowest. These rank-ordered loadings were then arranged into parcels using a procedure recommended by Little (2013), which involves assigning the two items with highest and lowest loadings to parcel one, the two items with second highest and lowest loadings to parcel two, and the two items with third highest and lowest loadings to parcel three. This process was repeated until all items were assigned to three parcels. This procedure increases likelihood of homogeneous parcels (for justification on the use of parcels, see Little, Rhemtulla, Gibson, & Schoemann, 2013; Rhemtulla, 2016).

Psychometric Properties of SCT Parcels

A multiple-indicator (three SCT parcels) by multiple-occasion (three assessments) by multiple-source (mothers, fathers, primary teachers, and secondary teachers) CFA was used to determine the invariance of like-parcel loadings and intercepts across sources and occasions. Invariance of like-indicator loadings and intercepts would establish measurement invariance of SCT across occasions and sources at the parcel level, thus providing justification for the multiple-source LST analysis with parcels.

Single-Source Latent State–Trait Analyses

A single-source LST analysis with parcel-specific trait factors was used to determine the proportion of trait (trait consistency) and state residual variance (occasion-specificity) for each source separately. The Mplus code for this model is available from the corresponding author.¹

Multiple-Source Latent State–Trait Analyses

A multiple-source LST analysis was used to determine the proportion of trait consistency, unique trait consistency, shared occasion-specificity, and unique occasion-specificity for the three comparison sources (fathers, primary teachers, and secondary teachers) relative to mothers. This analysis used mothers as the reference source due to the use of mothers as the reference source in the earlier studies (Litson et al., 2016; Preszler et al., 2016). The Mplus code for this model is available from the corresponding author.

Results

Measurement Models

Item-Level CFA

The item-level invariance analysis indicated a close fit for the four sources and three occasions model, χ²(4,302) = 7,181, CFI = .975, TLI = .974, and RMSEA = .029 (.028, .031). The constraints on like-item loadings and thresholds across the four sources and three occasions did not result in a decrement in the CFI value greater than .01, χ²(4,799) = 7,909, CFI = .973, TLI = .975, and RMSEA = .028 (.027, .029). The single set of eight unstandardized item–factor loadings from the final model was used to assign the eight SCT items to the three parcels in the manner described earlier. Table 1 shows the items in each parcel.

Parcel-Level CFA

The parcel-level invariance analysis indicated a close fit for the four-sources and three-occasions model, χ²(438) = 583, CFI = .993, TLI = .990, and RMSEA = .020 (.016, .024). The model with constraints on like-parcel loadings and the model with constraints on like-parcel loadings and intercepts also resulted in close fits along with no decrements in CFI greater than .01, χ²(460) = 636, CFI = .991, TLI = .988, and RMSEA = .022 (.01, .024); χ²(482) = 681, CFI = .990, TLI = .987, and RMSEA = .023 (.019, .026), respectively.

Latent Factor Means and Correlations

Mothers, fathers, and secondary teachers did not show a significant change in factor means across the three occasions (ps >.26). Primary teachers, however, showed a statistically significant decrease in factor means across occasions (ps <.05, latent Cohen d values from .06 to .14). Strong measurement invariance across sources and occasions along with stable means for three of the four sources and only trivial mean decreases for the fourth source indicate LST analyses were appropriate (Geiser et al., 2015). Supplemental Table S3 shows the SCT factor correlation within (.65 to .81) and across settings (.33 to .47) and occasions (.53 to .86).

Single-Source Latent State–Trait Analyses

The four single-source LST analyses resulted in close fit (CFIs ≥.993, TLIs ≥.992, and RMSEAs ≤.037). Table 3 shows the estimated trait variance (trait consistency), occasion-specific variance, and reliability coefficients for the four analyses. SCT ratings for mothers, fathers, and secondary teachers contained substantially more trait variance (Ms = 75%, 77%, and 70%, respectively) than occasion-specific variance (Ms = 25%, 23%, and 30%, respectively) with the ratings for secondary teachers containing slightly more trait than occasion-specific variance (Ms = 54% and 46%, respectively).

Table 3.

Average Trait Consistency, Occasion-Specificity, and Reliability Estimates From Single-Source Latent State–Trait Analyses on the Sluggish Cognitive Tempo Symptoms.

Source	Trait consistency	Occasion-specificity	Reliability
Mothers	.75 (.72, .81)	.25 (.19, .28)	.80 (.76, .88)
Fathers	.77 (.68, .83)	.23 (.17, .32)	.81 (.76, .88)
Primary teachers	.70 (.66, .75)	.30 (.25, .34)	.92 (.88, .95)
Secondary teachers	.54 (.51, .60)	.46 (.40, .49)	.92 (.88, .96)

Note. Entries indicate averages across parcels with ranges in parentheses.

Multiple-Source Latent State–Trait Analyses

The multiple-source LST model resulted in a close fit, χ²(561) = 819, CFI = .987, TLI = .986, RMSEA = .024 (.020, .027). Table 4 shows the shared trait consistency, unique trait consistency, shared occasion-specificity, and unique occasion-specificity for fathers, primary teachers, and secondary teachers with mothers. In terms of shared and unique trait variance, fathers shared a high-proportion trait consistency with mothers (M = 80%) with a corresponding low amount of unique trait consistency (M = 20%). This indicates strong convergent validity for trait variance within the home. In contrast, primary and secondary teachers shared a low proportion of trait consistency with mothers (Ms = 29% and 22%, respectively) with a correspondingly high proportion of unique trait consistency (Ms = 71% and 78%, respectively). These results indicate low convergent validity for trait variance for primary and secondary teachers with mothers. For the occasion-specific variance (shared occasion-specificity and unique occasion-specificity), fathers, primary teachers, and secondary teachers all showed high unique occasion-specificity (Ms = 72%, 100%, and 100%, respectively) relative to shared occasion-specificity with mothers (Ms = 37%, 0%, and 0%, respectively).

Table 4.

Average Shared Trait Consistency, Unique Trait Consistency, Shared Occasion-Specificity, and Unique Occasion-Specificity for a Multisource Latent State–Trait Analysis on the Sluggish Cognitive Tempo Symptoms.

Source	Shared trait consistency	Unique trait consistency	Shared occasion-specificity	Unique occasion-specificity
Fathers	.80 (.78, .81)	.20 (.19, .22)	.37 (.32, .44)	.63 (.56, .68)
Primary teachers	.29 (.27, .31)	.71 (.69, .73)	.00 (.00, .00)	1 (1, 1)
Secondary teachers	.22 (.20, .24)	.78 (.76, .80)	.00 (.00, .01)	1 (.99, 1)

Note. Values indicate the average across parcels with the range in parentheses. Mothers were used as the reference source.

The residual trait factor correlations among fathers, primary teachers, and secondary teachers provide additional information on whether the trait-like aspects of SCT ratings showed convergent validity within and across settings. The residual trait correlations indicate whether father and teacher reports share trait variance that these reports do not share with mothers. Primary and secondary teachers shared a high proportion of residual trait variance (M = 75%), whereas fathers did not share a meaningful amount of residual trait variance with primary and secondary teachers (Ms = 1%). These results indicate strong convergent validity within settings (between primary and secondary teachers) but nonexistent convergent validity across settings (fathers with primary and secondary teachers) for residual trait aspects of SCT.

The occasion-specific residual factor correlations for fathers, primary teachers, and secondary teachers indicate relationships among occasion-specific residual factor scores pertaining to nonreference sources after occasion-specific variance that nonreference sources share with mother reports have been partialled out. The average amount of shared occasion-specific residual variance for fathers with teachers was less than 1% with the average amount for primary with secondary teachers being 15% across the three occasions. These results indicate no shared occasion-specific residual variance for fathers with primary and secondary teachers and only a small amount for primary with secondary teachers.

Discussion

This is the first study to determine the amount of trait (consistency across occasions) and occasion-specific variance in SCT symptoms, a question examined with Spanish children with four different sources (mothers, fathers, primary teachers, secondary teachers) over three measurement occasions. SCT symptoms showed a substantial amount of trait variance relative to occasion-specific variance for the four sources, especially mothers, fathers, and primary teachers. These findings provide the strongest evidence to date of the trait-like nature of SCT in children (Barkley, 2016; Becker, Leopold, et al., 2016).

This study also determined the amount of trait and occasion-specific variance in SCT symptoms that fathers, primary teachers, and secondary teachers either shared or did not share with mothers as well as with each other. This question deals with the convergent validity and the source specificity of the trait and occasion-specific variance within and across settings (i.e., agreement and disagreement for trait and occasion-specific variance). Strong convergent validity occurred for trait variance within settings (mothers with fathers, primary with secondary teachers) and weak convergent validity for trait variance across settings with no convergent validity for occasion-specific variance across settings. How do these results for SCT symptoms compare with the results for ADHD and ODD symptoms? We now turn to these questions.

Trait and Occasion-Specific Variance in SCT, ADHD-IN, ADHD-HI, and ODD Symptoms

Given earlier studies used the same four sources, children, and 1-year interval to determine trait and occasion-specific variance in ADHD and ODD symptom ratings (Litson et al., 2016; Preszler et al., 2016), it was possible to compare the trait-like nature of ADHD and ODD symptoms to SCT symptoms. SCT symptoms contained similar amounts of trait variance (consistency across occasions) as the ADHD-IN (69% vs. 69%, averages for four sources), ADHD-HI (69% vs. 69%), and ODD symptoms (69% vs. 63%). SCT symptoms were thus as trait-like as ADHD-IN, ADHD-HI, and ODD symptoms for the four sources across the three measurement occasions.

Convergent validity of trait variance in SCT, ADHD-IN, ADHD-HI, and ODD ratings across the four sources was more complex. The amount of trait variance that fathers shared with mothers for SCT, ADHD-IN, and ADHD-HI symptom ratings was substantial and identical (80%, 79%, and 80%, respectively) with these values slightly larger than trait variance fathers shared with mothers for ODD (68%). Convergent validity, however, varied substantially across home and school. Primary and secondary teachers shared 46% to 55% of their trait variance with mothers for ADHD-IN symptoms, 42% to 55% for ADHD-HI symptoms, 31% to 32% for ODD symptoms, and 22% to 29% for SCT symptoms. Trait variance in SCT ratings showed less consistency across home and school than trait variance in ADHD symptom ratings. This across-setting difference in trait consistency in SCT and ADHD symptoms, especially ADHD-IN symptoms, requires additional study. However, it should also be noted that the average convergent validity of the residual trait variance for primary with secondary teachers was 75%, 75%, and 75% for SCT, ADHD-IN, and ADHD-HI, respectively, thus indicating strong and consistent convergent validity for the residual trait variance within the school setting for the three symptom dimensions.

These three sets of findings—(a) SCT symptoms showed the same level of trait consistency across occasions as ADHD-IN symptoms, (b) SCT symptoms showed the same level of convergent validity for trait consistency within home and school settings as ADHD-IN symptoms, and (c) SCT symptom ratings showed less convergent validity for trait consistency across home and school settings than ADHD-IN symptoms—have important implications. Taken together, these findings provide strong support for the consistency of SCT symptoms over time, suggesting that SCT symptoms in children are not likely to be transient. This is important to note given research linking SCT to functional impairment (Becker, Leopold, et al., 2016). In addition, our findings suggest that it is especially important that research on SCT obtain information within both the home and school settings. Although SCT symptoms are invariant across parent and teacher ratings (Burns et al., 2016), it is still possible that certain features of SCT are more clearly observed in certain contexts and, when observed, especially linked to impairment. For example, Burns et al. (2016) found teacher-rated SCT was a clearer predictor of parent-rated impairment than vice versa. It may be that teachers are optimal raters when assessing SCT. This is not to say that parents (and even youth themselves; Becker, Luebbe, & Joyce, 2015) are not good raters of SCT, but rather that it is especially important to gather teacher ratings of SCT whenever possible. Of course, gathering both parent and teacher ratings of ADHD is also recommended (American Academy of Pediatrics, 2011), but findings from the present study indicate that SCT symptoms may be more context-specific than ADHD symptoms. Additional research supporting this possibility would point to the need to better uncover the unique context-dependent aspect(s) of SCT that may be useful for informing etiological models as well as intervention efforts.

Use of Multiple-Source LST Models to Study Informant Discrepancies

Understanding the extent to which different informants agree or disagree in their perspectives of psychopathology and behavior is an important research area (Achenbach, 2016; Burns et al., 2008; De Los Reyes, 2013; De Los Reyes et al., 2015; De Los Reyes et al., 2016). One focus of informant discrepancy research is to quantify the level of consistency (convergence) and inconsistency (discrepancy) for multiple sources within and across settings, and a second focus is to evaluate the extent to which discrepancies are theoretical and clinically meaningful as opposed to representing bias or measurement error (De Los Reyes, 2013). Our study focused on the first issue of whether there is convergence or discrepancy within and across settings for parent and teacher ratings of SCT. The multiple-source LST model applied in this study allows for a more fine-grained analysis of informant discrepancies than previously applied methods by considering both trait (temporally stable) and state (more fluctuating) aspects of behavior and examining informant discrepancies for both aspects separately. With a minimum of two sources and two occasions of measurement, it is possible to determine the amount of trait variance (consistency across occasions) that is either shared (convergence or shared trait consistency) or not shared (discrepant or unique trait consistency) among sources. We found a high degree of convergence within home and school contexts but not between home and school contexts. It will be important for future research to evaluate why cross-setting discrepancies exist in ratings of SCT, as well as how such discrepancies relate to functioning. It is possible that the different contexts in which parents and teachers observe children completing academic work and engaging in social relationships may contribute to not only differences in ratings of SCT but also in the magnitude to which SCT is associated with impairment. The multiple-source LST model provides a useful approach to these and other questions that can advance the understanding of agreements and disagreements among multiple sources in relation to SCT and other psychopathology symptom dimensions.

Limitations and Future Directions

Limitations include the 1-year longitudinal span and narrow age range of participants in this study. It would also be useful to include self-ratings of SCT symptoms by children and adolescents in order to determine the convergent validity for trait variance of self-ratings with trait variance in ratings by parents and teachers. Likewise, in this study, a unidimensional measure of SCT was used, but there is some evidence that SCT may have a hierarchical structure with lower-order factors of the higher-order, general factor (Smith et al., 2016). It is possible that SCT subfactors differ in their etiologies and the extent to which they are more state-like or trait-like, and this too is another area for future research. Finally, it would be important to investigate these questions in clinical samples of children and adolescents (e.g., children with a diagnosis of ADHD), particularly since SCT symptom ratings may have more occasion-specific variance in youth who are experiencing acute stressors (Becker, Withrow, et al., 2016) as well as more severe levels of symptoms.

Conclusions

Single- and multiple-source LST models were used to determine the amount of trait and occasion-specific variance in SCT symptom ratings for mothers, fathers, primary teachers, and secondary teachers. SCT was strongly trait-like across time and within settings but showed low consistency across settings, suggesting that SCT may be a more setting-specific trait than ADHD-IN. These findings indicate that SCT symptoms should not be dismissed as transient or inconsequential; rather, these findings point to the importance of developing and evaluating clinical interventions that aim to modify the presence and developmental trajectory of SCT. This is particularly important given a growing body of research linking SCT to a range of functional impairment outcomes, both concurrently and over time (Becker, Leopold, et al., 2016).

Footnotes

Acknowledgements

We thank Cristina Trias and Cristina Solano for their help in data collection. We also thank the parents and teachers whose kind assistance made the research possible.

Authors’ Note

The content is solely the responsibility of the authors and does not necessarily represent the official views of the Spanish Government or the U.S. National Institutes of Health, the funding agencies.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Two Ministry of Economy and Competitiveness grants, PSI2011-23254 and PSI2014-52605-R (Spanish Government), and a predoctoral fellowship cofinanced by the European Social Fund and the Balearic Islands Government (FPI/1451/2012) supported this research. Christian Geiser is supported by award number R01DA034770-01 from the National Institutes on Drug Abuse. Stephen Becker is supported by award number K23MH108603 from the National Institute of Mental Health.

Notes

References

Achenbach

T. M.

(2016). Future directions for clinical research, services, and training: Evidence-based assessments across informants, cultures, and dimensional hierarchies. Journal of Clinical Child & Adolescent Psychology. Advance online publication. doi:10.1080/15374416.2016.1220315

American Academy of Pediatrics. (2011). ADHD: Clinical practice guidelines for the diagnosis, evaluation, and treatment of attention-deficit/hyperactivity disorder in children and adolescents. Pediatrics, 128, 1007-1022.

Barkley

R. A.

(2016). Sluggish cognitive tempo: A (misnamed) second attention disorder? Journal of the American Academy of Child & Adolescent Psychiatry, 55, 157-158.

Becker

S. P.

Leopold

D. R.

Burns

G. L.

Jarrett

M. A.

Langberg

J. M.

Marshall

S. A.

. . . Willcutt

E. G.

(2016). The internal, external, and diagnostic validity of sluggish cognitive tempo: A meta-analysis and critical review. Journal of the American Academy of Child & Adolescent Psychiatry, 55, 163-178. doi:10.1016/j.jaac.2015.12.006

Becker

S. P.

Luebbe

A. M.

Joyce

A. M.

(2015). The Child Concentration Inventory (CCI): Initial validation of a child self-report measure of sluggish cognitive tempo. Psychological Assessment, 27, 1037-1052. doi:10.1037/pas0000083

Becker

S. P.

Withrow

A. R.

Stoppelbein

Luebbe

A. M.

Fite

P. J.

Greening

(2016). Sluggish cognitive tempo is associated with suicide risk in psychiatrically hospitalized children. Journal of Child Psychology and Psychiatry, 57, 1390-1399. doi:10.1111/jcpp.12580

Belmar

Servera

Becker

S. P.

Burns

G. L.

(2015). Validity of sluggish cognitive tempo in South America: An initial examination using mother and teacher ratings of Chilean children. Journal of Attention Disorders. Advance online publication. doi:10.1177/1087054715597470

Bernad

Servera

Becker

S. P.

Burns

G. L.

(2016). Sluggish cognitive tempo and ADHD inattention as predictors of externalizing, internalizing, and impairment domains: A 2-year longitudinal study. Journal of Abnormal Child Psychology, 44, 771-785. doi:10.1007/s10802-015-0066-z

Burns

G. L.

Becker

S. P.

Servera

Bernad

M. M.

García-Banda

(2016). Sluggish cognitive tempo and ADHD inattention in the home and school contexts: Parent and teacher invariance and cross-setting validity. Psychological Assessment. Advance online publication. doi:10.1037/pas0000325

10.

Burns

G. L.

Lee

Becker

S. P.

Servera

McBurnett

(2014). Child and Adolescent Disruptive Behavior Inventory—Parent and Teacher Versions 5.0. Pullman: Washington State University.

11.

Burns

G. L.

Moura

M. A.

Walsh

J. A.

Desmul

Silpakit

Sommers-Flanagan

(2008). Invariance and convergent and discriminant validity between mothers’ and fathers’ ratings of oppositional defiant disorder toward adults, ADHD-HI, ADHD-IN, and academic competence factors within Brazilian, Thai, and American children. Psychological Assessment, 20, 121-130.

12.

Courvoisier

D. S.

Nussbeck

F. W.

Eid

Geiser

Cole

D. A.

(2008). Analyzing the convergent validity of states and traits: Development and application of multimethod latent state-trait models. Psychological Assessment, 20, 270-280. doi:10.1037/a0012812

13.

De Los Reyes

. (2013). Strategic objectives for improving understanding of informant discrepancies in developmental psychopathology research. Development and Psychopathology, 25, 669-682. doi:10.1017/S0954579413000096

14.

De Los Reyes

Alfano

C. A.

Lau

Augenstein

T. M.

Borelli

J. L

. (2016). Can we use convergence between caregiver reports of adolescent mental health to index severity of adolescent mental health concerns? Journal of Child and Family Studies, 25, 109-123. doi:10.1007/s10826-015-0216-5

15.

De Los Reyes

Augenstein

T. M.

Wang

Thomas

S. A.

Drabick

D. A. G.

Burgers

Rabinowitz

. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141, 858-900. doi:10.1037/a0038498

16.

Geiser

Keller

B. T.

Lockhart

Eid

Cole

D. A.

Koch

(2015). Distinguishing state variability from trait change in longitudinal data: The role of measurement (non)invariance in latent state-trait analyses. Behavior Research Methods, 47, 172-203. doi: 10.3758/s13428-014-0457-z

17.

Khadka

Burns

G. L.

Becker

S. P.

(2016). Internal and external validity of sluggish cognitive tempo and ADHD inattention dimensions with teacher ratings of Nepali children. Journal of Psychopathology and Behavioral Assessment, 48, 433-442. doi:10.1007/s10862-015-9534-6

18.

Lee

S. Y.

Burns

G. L.

Becker

S. P.

(2016a). Can sluggish cognitive tempo be distinguished from ADHD inattention in very young children? Evidence from a sample of Korean preschool children. Journal of Attention Disorders. Advance online publication. doi:10.1177/1087054716680077

19.

Lee

S. Y.

Burns

G. L.

Becker

S. P.

(2016b). Towards establishing the transcultural validity of sluggish cognitive tempo: Evidence from a sample of South Korean children. Journal of Clinical Child & Adolescent Psychology. Advance online publication. doi:10.1080/15374416.2016.1144192

20.

Lee

S. Y.

Burns

G. L.

Snell

McBurnett

(2014). Validity of the sluggish cognitive tempo symptom dimension in children: Sluggish cognitive tempo and ADHD-Inattention as distinct symptom dimensions. Journal of Abnormal Child Psychology, 42, 7-19. doi:10.1007/s10802-013-9714-3

21.

Leopold

D. R.

Christopher

M. E.

Burns

G. L.

Becker

S. P.

Olson

R. K.

Willcutt

E. G.

(2016). ADHD and sluggish cognitive tempo throughout childhood: Temporal invariance and stability from preschool through ninth grade. Journal of Child Psychology and Psychiatry, 57, 1066-1074. doi:10.1111/jccp.12505

22.

Litson

Geiser

Burns

G. L.

Servera

(2016). Trait and state variance in multi-informant assessments of ADHD and academic impairment in Spanish first-grade children. Journal of Clinical Child & Adolescent Psychology. Advance online publication. doi:10.1080/15374416.2015.1118693

23.

Little

T. D.

(2013). Longitudinal structural equation modeling. New York, NY: Guilford.

24.

Little

T. D.

Rhemtulla

Gibson

Schoemann

A. M.

(2013). Why the items versus parcels controversy needn’t be one. Psychological Methods, 18, 285-300. doi:10.1037/a0033266

25.

Preszler

Burns

G. L.

Litson

Geiser

Servera

(2016). Trait and state variance in oppositional defiant disorder symptoms: A multi-source investigation with Spanish children. Psychological Assessment. Advance online publication. doi:10.1037/pas0000313

26.

Rhemtulla

(2016). Population performance of SEM parceling strategies under measurement and structural model misspecification. Psychological Methods, 21, 348-368. doi:10.1037/met0000072.

27.

Smith

Z. R.

Becker

S. P.

Garner

A. A.

Rudolph

C. W.

Molitor

S. J.

Oddo

L. E.

Langberg

J. M.

(2016). Evaluating the structure of sluggish cognitive tempo using confirmatory factor analytic and bifactor modeling with parent and youth ratings. Assessment. Advance online publication. doi:10.1177/1073191116653471

28.

Steyer

Geiser

Fiege

(2012). Latent state-trait models. In Cooper

(Ed.), Handbook of research methods in psychology (pp. 209-308). Washington, DC: American Psychological Association.

29.

Willcutt

E. G.

Nigg

J. T.

Pennington

B. F.

Solanto

M. V.

Rohde

L. A.

Tannock

. . . Lahey

B. B.

(2012). Validity of DSM-IV attention-deficit/hyperactivity disorder symptom dimensions and subtypes. Journal of Abnormal Child Psychology, 121, 991-1010. doi:10.1037/a0027347.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB

How Consistent Is Sluggish Cognitive Tempo Across Occasions,Sources,and Settings? Evidence From Latent State–Trait Modeling

Abstract

Keywords

Utility of Latent State–Trait Measurement Models

Single-Source LST Models

Multiple-Source LST Models

Trait and Occasion-Specific Variance in ADHD-IN, ADHD-HI, and Oppositional Defiant Disorder (ODD) Symptoms

Study Objectives

Method

Participants and Procedures

Measure

Child and Adolescent Disruptive Behavior Inventory (CADBI; Burns, Lee, Becker, Servera, & McBurnett, 2014)

Analyses

Analytic Strategy

Preliminary Analyses on Individual Items to Justify Creation of Parcels

Psychometric Properties of SCT Parcels

Single-Source Latent State–Trait Analyses

Multiple-Source Latent State–Trait Analyses

Results

Measurement Models

Item-Level CFA

Parcel-Level CFA

Latent Factor Means and Correlations

Single-Source Latent State–Trait Analyses

Multiple-Source Latent State–Trait Analyses

Discussion

Trait and Occasion-Specific Variance in SCT, ADHD-IN, ADHD-HI, and ODD Symptoms

Use of Multiple-Source LST Models to Study Informant Discrepancies

Limitations and Future Directions

Conclusions

Footnotes

Acknowledgements

Authors’ Note

Declaration of Conflicting Interests

Funding

Notes

References

Supplementary Material