Abstract
Objective:
The Aberrant Behavior Checklist (ABC) is a standardized rating scale used for assessing problematic behavior of individuals with developmental disabilities. It has five subscales: Irritability, Social Withdrawal, Stereotypic Behavior, Hyperactivity, and Inappropriate Speech. A previous study in individuals with fragile X syndrome (FXS) reported six factors, with the Social Withdrawal factor bifurcating into Socially Unresponsive and Social Avoidance factors, suggesting a different factor structure in people with FXS.
Methods:
We assessed the ABC's factor structure (with both exploratory and confirmatory analyses) in 797 people with FXS and we compared these findings with exploratory factors derived from an independent sample of 357 individuals with FXS. In an ancillary analysis, we compared the overlap of the traditional ABC's Social Withdrawal scores with the Social Avoidance scores from the FXS-derived newer scale to determine whether overlap between these was very high and essentially redundant. Finally, we computed norms using both the traditional and the FXS-specific algorithms.
Results:
In confirmatory factor analyses, the FXS-specific algorithm produced the most consistent factor structure for the sample of 797 participants, but model fit was only marginally better than that derived by the original ABC scoring algorithm. Comparisons of factor structures from separate exploratory analyses revealed no consistent advantage of the FXS algorithm over the traditional algorithm. While a Social Avoidance subscale did emerge in some analyses, in other analyses, this was accompanied by loss of coherence on other domains of interest, such as the Socially Unresponsive/Social Withdrawal subscale.
Conclusion:
We question whether the newer FXS scoring algorithm contributes data that are consistently helpful in evaluating behavior of people with FXS. In general, we recommend continued use of the original ABC algorithm for scoring behavior of clients with FXS. However, we acknowledge that there may be circumscribed times when the new algorithm may be appropriate for scoring, namely when anxiety and/or social avoidance constructs are the central and unequivocal domains of interest.
Introduction
The Aberrant Behavior Checklist (ABC) is an empirically derived, standardized behavior rating scale developed for assessing treatment effects in people with developmental disabilities, including intellectual disability and autism (Aman et al., 1985a, 1985b). The ABC has traditionally been scored on five subscales, labeled as Irritability (15 items), Social Withdrawal (16 items), Stereotypic Behavior (7 items), Hyperactivity/Noncompliance (16 items), and Inappropriate Speech (4 items). We shall refer to this as the ABC/Original scoring algorithm (or simply ABC/Original) throughout this study.
Besides its extensive use in pharmacological trials, the ABC/Original also has been used in evaluating behavioral interventions, nutritional interventions, effects of environmental changes, for psychometric studies, for subject characterization, and for multiple other purposes (Aman and Singh, 2017). The scale has been translated into at least 35 languages other than English, and the manual estimates that it has been used in over 450 studies to date.
Fragile X syndrome (FXS) is the most common inherited form of developmental disability and the most common single-gene cause of autism spectrum disorder (ASD), impacting 1 in 4000 males and 1 in 6000 females (Chonchaiya et al., 2009). FXS results from a triplet repeat expansion in the fragile X mental retardation gene, resulting in gene methylation and transcriptional silencing and deficit fragile X mental retardation protein expression. As a single-gene disorder with increasingly understood pathophysiology, FXS has been the focus of intense translational medicine efforts in the last 15 plus years. Many preclinical success stories, in particular efforts focused on attenuating metabotropic glutamate receptor type 5 activity, have not been met with similar success in large-scale clinical trials (Erickson et al., 2018).
During this wave of past and ongoing treatment development, outcome measure selection has been a key focus of understanding the lack of success in human studies (Budimirovic et al., 2017). ABC items cover many behavioral concerns that FXS stakeholders have reported as treatment priorities (Weber et al., 2019). Following U.S. Food and Drug Administration approval of the ABC/Original as a primary outcome measure of ASD trials in youth exhibiting irritability (Aman and Singh, 2017), the ABC/Original was initially, and is to this day, a recommended and approvable endpoint in pivotal FXS clinical trials. Given this, the ABC has been included in all large-scale FXS clinical trials in recent years. When declared a primary or key secondary outcome measure, subscales of the ABC/Original scoring algorithm (Aman, 2012; Aman and Singh, 2017) have not been associated with treatment-linked improvement across many studies in youth and adults with FXS.
Some investigators in FXS drug research felt that items of the ABC deviated from others in the same subscales in response to pharmacotherapy, leading them to wonder if the “correct” algorithm might be different for individuals with FXS (Berry-Kravis et al., 2008). Others sought an FXS-specific version of the ABC based solely on a sample diagnosed with FXS (Sansone et al., 2012). In 2012, Sansone et al. (2012) conducted exploratory and confirmatory parcel factor analytic studies with 630 individuals having FXS. This resulted in a novel six-factor scoring algorithm that we shall refer to as the ABCFXS throughout this study.
The ABCFXS has six subscales: Irritability (18 items), Socially Unresponsive (13 items; similar to Social Withdrawal in ABC/Original), Stereotypy (6 items), Hyperactivity (10 items), Inappropriate Speech (4 items), and Social Avoidance (4 items, derived from Social Withdrawal in ABC/Original). Sansone et al. (2012) focused particular attention on the Social Avoidance subscale because they felt that it contained items that were especially indicative of FXS traits commonly seen across large samples of such individuals. Three items in the ABC/Original algorithm were removed from the ABCFXS because of failure to load consistently on a single factor in the study.
In 2014, Wheeler et al. (2014) also conducted confirmatory factor analyses (CFAs) in 292 males with FXS; there were 58 females in the sample, but they were not included in the CFAs. Wheeler et al. concluded that “both the 5-factor [ABC/Original] and the 6-factor solutions [ABCFXS] for the ABC-C were appropriate based on the fit indices” (italics added; p.144). They also stated that the 6-factor solution ABCFXS produced “a marginally better fit with our sample” (p. 145) than the 5-factor ABC/Original solution. Wheeler et al. (2014) were interested in conducting clinical comparisons with the Sansone et al.'s (2012) sample, so they used the ABCFXS data for several other comparisons.
With this background in mind, we had one ancillary question and three principal hypotheses and objectives, as follows: Ancillary: Correspondence of Social Withdrawal (in ABC/Original) and Social Avoidance (in ABCFXS) Subscales. The objective was to determine the strength of association between the entire 16-item Social Withdrawal subscale (i.e., content reported in ABC/Original) and the 4-item ABCFXS Social Avoidance subscale. We hypothesized that the two sets of items would be highly correlated (r ≥ 0.75), possibly enabling them to be used in a complementary manner. This could have important implications for the design and interpretation of FXS clinical drug trials. CFA. Second, we wanted to assess the robustness of six independent factor solutions with the ABC, which were explored previously by Kaat et al. (2014) in ASD. These factor solutions and the rationale for their selection are provided in the Methods. Given that the sample examined in this study comprised individuals with FXS, special attention was ultimately devoted to the 6-factor ABCFXS (Sansone et al., 2012) compared with the 5-factor ABC/Original (Aman et al., 1985a). We adopted the more conservative hypothesis, that the ABC/Original factor structure (including the original Lethargy/Social Withdrawal factor) would provide adequate fit in our CFAs and that they would reemerge in our later exploratory factor analyses (EFAs). However, if none of the previously proposed factor structures provided adequate statistical fit, we planned to employ post hoc EFAs to evaluate alternative item assignment. This would determine whether an exploratory assignment more closely approximated one of the two solutions of primary focus (the ABC/Original or ABCFXS). EFAs of ABCFXS. We eventually determined that EFAs were necessary to explore potential causes for misfit in the CFA models. We conducted these in two sets of analyses, first with all 58 items, and then with only the 55 algorithm items included in the final ABCFXS advocated by Sansone et al. (2012). For these analyses only, we included data from both the Wheeler et al.'s (2014) study and from the sample described here. Insofar as this step was employed only after the CFA models failed to provide adequate fit, our primary aim was to evaluate item assignment plus factor composition and integrity. We closely compared the EFA item assignment to the proposed ABCFXS and ABC/Original item assignments. Of particular interest was whether the Social Avoidance and Socially Unresponsive items would emerge together. Norms Development for Subjects with FXS. Finally, we presented normative data on the ABC for the large sample of people with FXS in our FORWARD sample (described in the Normative data section). Depending on CFA and EFA results, either ABC/Original or ABCFXS, or both scoring algorithms were to be used for norms development.
Methods
Project FORWARD and subject enrollment
Much of this study was launched through a project hereafter called FORWARD (Fragile X Online Registry With Accessible Research Database, a multisite observational study), which receives funding through the U.S. Centers for Disease Control and Prevention (Sherman et al., 2017, for details). At the time of this study, FORWARD consisted of 27 clinics in the United States, which specialized in the care of people with FXS and their families. Although FORWARD gathers both acute and longitudinal data, only baseline information from the longitudinal database is presented in this study. The longitudinal database is constrained to people with the full FXS mutation, and the large majority is from individuals between 4 and 24 years.
Recruitment for FORWARD occurred through these specialty clinics. All clinics and the data coordinating center (DCC) were approved by their respective institutional review boards for this work. All participants in FORWARD and/or their legally authorized representatives provided signed informed consents on entry into the study (more often, if required by institutional policy).
The activities of FORWARD are guided by a Membership/Infrastructure Committee, a Clinical Practices Committee, a Research Committee, the DCC, and a National Coordinator. Clinics participating within FORWARD are compensated for maintaining Institutional Review Board approvals and for data collection. Individual sites and the DCC interact to maintain accurate data. Sites meet annually to coordinate and plan activities; the national coordinator conducts site visits and assists sites with procedures to maintain consistent data collection, including annual follow-ups. According to the “EXPLAIN” group based in Germany (Haessler et al., 2013), FORWARD is the single largest longitudinal data repository in the world with detailed prospective clinical data on individuals with FXS.
This study used data from the FORWARD longitudinal database, including results from 797 individuals. Full mutation status in all cases was verified by the treating clinician and included date of testing, methylation status if known, and whether additional genetic anomalies were tested for in the past.
Our fragile X world sample
This was an archival sample, reported previously (Wheeler et al., 2014), comprising 357 individuals with FXS, ages 2 years and older, and their caregivers. These were children, adolescents, and adults whose caregivers completed the full ABC as part of an online survey about the individual's psychiatric and behavioral phenotype. Although some of their data were reported earlier, EFA findings were not. Because of the relevance of EFA results to this study, we included the Our Fragile X World (OFXW) sample to supplement results from the FORWARD sample. As with the FORWARD data, all 58 ABC items were captured, although only the 55 advocated by Sansone et al. (2012) were analyzed in some of our EFA comparisons.
ABC and periodic deviations in factor content
As stated above, the ABC/Original scores all 58 items onto five subscales: (1) Irritability (15 items), (2) Social Withdrawal (16 items), (3) Stereotypic Behavior (7 items), (4) Hyperactivity/Noncompliance (16 items), and (5) Inappropriate Speech (4 items). In non-FXS samples, most studies have found substantial concordance with the originally reported factor structure (ABC/Original), both in children and adults (Aman and Singh, 2017). Exceptions have included Brown et al., (2002), who did not find an Inappropriate Speech subscale, and Brinkley et al. (2007), who found that three self-injury items formed their own factor instead of loading with the Irritability subscale. In addition, various items from the Irritability and Hyperactivity/Noncompliance subscales occasionally crossed to the alternative factor, although the exact items crisscrossing have been inconsistent across studies (Aman and Singh, 2017). Finally, as noted in the Introduction, Sansone et al. (2012) studied children and young adults with FXS and reported that Lethargy/Social Withdrawal split into separate factors; the ABCFXS used 55 of the original 58 original items. Importantly, FORWARD collects all 58 items for all participants, allowing us to compare results with both the ABC/Original and ABCFXS algorithms in this study.
Analytic design
Subscale overlap
The agreement in rank-ordering of the ABC/Original (Aman and Singh, 2017) Social Withdrawal subscale (16 items) and ABCFXS (Sansone et al., 2012) Social Avoidance subscale (4 items) was calculated by a Spearman correlation coefficient.
Confirmatory factor analysis
Analyses were conducted in MPLUS version 8. Diagonally weighted least squares estimation with robust standard errors was performed on the polychoric correlation matrix. Missing data were addressed by listwise deletion (i.e., participants with missing data were excluded). Consistent with best practice recommendations (Brown, 2015), model fit was evaluated using several measures, including the mean-and-variance-adjusted chi-squared test (MV-χ2), absolute fit (root mean square error of approximation [RMSEA]), and incremental fit (comparative fit index [CFI]). While rules of thumb have been proposed for interpreting these fit indices with continuous data, debate exists regarding appropriate cutoffs for categorical data (Xia and Yang, 2019). In general, nonsignificant MV-χ2, lower RMSEA, and higher CFI are preferred.
For CFAs, the following six models were tested: (1) the ABC/Original five-factor model presented by Aman and Singh (1986); (2) the six-factor ABCFXS model proposed by Sansone et al. (2012); (3) the five-factor model from the Kaat et al. (2014) EFA; (4) the four-factor model found by Brown et al. (2002); and both the (5) four- and (6) five-factor models proposed by Brinkley et al. (2007). We were especially focused on differences between the standard ABC/Original and ABCFXS algorithms.
Exploratory factor analyses
In the end, due to model misfit and other reasons that will become apparent, we conducted EFAs to examine ABC item realignment and whether that corresponded better to the ABC/Original or ABCFXS algorithm. Ordinary least squares estimation with oblique (Geomin) rotation was conducted on the polychoric correlation matrix. EFA comparisons included results from the FORWARD and OFXW samples with particular reference to interpretability of five-factor and six-factor solutions and consistency with either the ABC/Original or ABCFXS algorithms.
Although methods have been proposed to suggest number of factors to extract (e.g., the Kaiser criterion, examination of a scree plot, parallel analysis), we used a theoretical approach, considering EFAs with five, six, seven, and eight factors. We chose this approach because number of factors to extract was of secondary importance to item alignment and concordance with the ABC/Original or ABCFXS algorithms.
ABC/original and ABCFXS norms
Means and standard deviations (SDs) by sex and age, and percentiles by sex alone for this large sample with FXS were calculated. Our plan was to present these norms for the scoring algorithm (ABC/Original or ABCFXS), which was more compelling based on CFAs and EFAs.
Results
Sample compositions
FORWARD sample
In all, 797 individuals with FXS and their caregivers comprised the FORWARD sample. The mean age was 11.44 years, with an SD of 8.14 (range 1–49 years). The majority of participants (about 77%) were males (Table 1). The sample was primarily White, with about 8% African American and 3% Asian participants. Eleven percent were Hispanic. Most caregivers had more than a high school education, with technical school, college degree, and postgraduate education quite common. Annual household income was rather diverse, with similar percentages reporting incomes spanning $25,000–$49,999 through >$150,000 per annum. Caregivers reported that about 41% of the sample was diagnosed with ASD at some time in their lives. More information on demographics is provided in Table 1.
Demographic Characteristics of Samples
OFXW sample, described by Wheeler et al. (2014).
Individuals were allowed to choose more than one racial category. Thus, the number of designations (825) is greater than the sample size (797) due to multiracial individuals.
$100,000–$149,999 and ≥$150,000 captured together on Carolina survey, making differentiation impossible.
Includes technical school, some college, and associate of arts degree.
ASD, autism spectrum disorder; N/A, not applicable; OFXW, Our Fragile X World.
OFXW sample; EFAs only
Three hundred fifty-seven individuals with FXS and caregivers who completed the OFXW survey were involved in EFA analyses. Mean age of the probands was 19.13 years, with an SD of 11.04 (range 3–66 years). About 83% of the sample was male. The sample was primarily White; about 2% were African American and 1% were Asian. Ethnically, 4% were Hispanic. Among the caregivers, technical, college, and postgraduate degrees were all common. Annual household income was diverse, with similar percentages of the sample reporting incomes spanning $25,000–$49,999 through $75,001–$100,000 per annum and over one-third reporting incomes >$100,000 per annum. More information appears in Table 1.
Subscale overlap
The correlation between ABC/Original Social Withdrawal scores and ABCFXS Social Avoidance scores was 0.58, which fell well below our hypothesis that these would show high agreement in rank ordering of individuals. Therefore, although these subscales were moderately related, they could not be used as “clinical substitutes” for one another.
Confirmatory factor analyses
Summaries of fit indices can be found in Table 2. The data deviated significantly from all of the models (all MV-χ2 < 0.05), with broadly comparable fit across all other indices (MV-χ2/df, RMSEA, and CFI). The ABCFXS model proposed by Sansone et al. (2012) yielded the best fit based on RMSEA and CFI criteria, although both the ABC/Original and five-factor Brinkley et al. models also yielded reasonable fit. As the ABC/Original model and the ABCFXS model are of particular interest, we examined their structure closely.
Model Fit Comparisons for Confirmatory Factory Analyses
Five-factor model.
Four-factor model.
Model does not include all 58 ABC items; SB-X2 are not comparable.
ABC, Aberrant Behavior Checklist; CFI, comparative fit index; CI, confidence interval; MV-χ2, mean-and-variance-adjusted chi-squared test; RMSEA, root mean square error of approximation.
Within the ABC/Original model, all but two items loaded strongly (using a 0.50 threshold) onto their assigned factors. The two items that did not were Listless, sluggish, inactive (Social Withdrawal factor, loading of 0.29), and Depressed mood (Irritability factor, loading of 0.43). Mean factor loadings for the ABC/Original were as follows: Irritability = 0.80; Lethargy/Social Withdrawal = 0.71; Stereotypic Behavior = 0.83, Hyperactivity = 0.81, and Inappropriate Speech = 0.86. Factor intercorrelations generally were moderate (range of 0.41–0.77).
Similarly, within the ABCFXS algorithm [i.e., Sansone et al.'s (2012) model], all but two items, both from the Socially Unresponsive/Lethargic factor, loaded strongly onto their assigned factors: Does nothing but sit and watch others (loading of 0.47) and Depressed mood (loading of 0.46). Mean loading for the ABCFXS factors were as follows: Irritability = 0.82; Hyperactivity = 0.83, Socially Unresponsive = 0.64, Social Avoidance = 0.92, Stereotypic Behavior = 0.85, and Inappropriate Speech = 0.86. Factor intercorrelations were generally moderate to high, with the exception of the correlation between the Social Avoidance and Hyperactivity factors (0.21). All parameters (item loadings, thresholds, and factor intercorrelations) for both the ABC/Original and ABCFXS are available online (Supplementary Tables S1 and S2).
Exploratory factor analyses
Although our initial plan was to confirm either the ABC/Original or ABCFXS algorithm through CFAs, neither model provided unambiguous support for or against the validity of the Socially Unresponsive and Social Avoidance factors. Therefore, we entered into a series of EFAs designed to determine the robustness of item assignment for the Socially Unresponsive, Social Avoidance, and other factors. All analyses were conducted with the 55 items included on the ABCFXS algorithm [i.e., with removal of the 3 items that Sansone et al. (2012) deleted], but comparable analyses with the full 58-item ABC produced very similar results. EFAs with all 58 items did not support removal of the 3 items excluded from the ABCFXS algorithm. They all had loadings >0.50 on one of the factors for the 5- through 8-factor EFAs, although the loadings in the CFA were less convincing for item 3 (listless).
In all of the EFA analyses that we conducted, without exception, the three self-injury items (nos. 2, 50, and 52) emerged as a separate factor, which we address in the Discussion. Note that this detracted equally from factor fit in all cases, irrespective of whether we discuss the ABC/Original or the ABCFXS scoring algorithms.
EFAs: item alignment as function of number of factors, scoring algorithm, and sample
Primary item assignments for the EFA models were determined based on the absolute magnitude of the factor loading and the size of potential cross-loadings. With two samples (FORWARD and OFXW), multiple EFAs with different numbers of extracted factors, and different items comprising factors, there is no simple method for comparing the integrity of the ABC/Original and ABCFXS scoring methods. However, there are still compelling ways to compare the products of these different algorithms and their likely clinical implications. In the analyses that follow, we compare ABC item assignment (i.e., whether the item was the most highly endorsed for a given factor) as a function of scoring method (ABC Original vs. ABCFXS), sample (FORWARD; OFXW), and number of factors (5–8).
FORWARD data
Summaries for the FORWARD EFAs are presented in Table 3. The content of the Stereotypic Behavior and the Inappropriate Speech subscales did not differ in any of these analyses (including for the OFXW data), so their results were not presented in Tables 3 and 4. Suffice it to say that they were consistently endorsed and assigned to their usual factors in all of these analyses, indicating very robust factor content.
Summary of Exploratory Factor Analysis Item Assignment Based on Data from FORWARD Sample
Note: ABC/Original and ABCFRX refer to the EFA models used to conduct the analyses. “Soc. Withdrawal” and “Soc. Unresponsive” refer to the Social Withdrawal and Socially Unresponsive subscales from the ABC/Original and ABCFRX, respectively. Decimals on each line refer to the percentage of prespecified items found to emerge in that respective analysis. Parentheses indicate the total number of items in preassigned algorithms. “HA” refers to HA factor.
Items factored with Socially Unresponsive items. If these items are excluded from the comparison, then the rate of “correct” preassignment increases to 63.4% (for five-factor analysis) and 90.2% (for six-factor analysis). Such correction allows for the fact that it is not possible to derive six factors with EFAs specifying five factors.
EFA, exploratory factor analysis; FXS, fragile X syndrome; HA, hyperactivity.
Summary of Exploratory Factor Analysis Item Assignment Based on Data from Our Fragile X World Sample
Note: ABC/Original and ABCFRX refer to the EFA models used to conduct the analyses. “Soc. Withdrawal” and “Soc. Unresponsive” refer to the Social Withdrawal and Socially Unresponsive subscales from the ABC/Original and ABCFRX, respectively. Decimals on each line refer to the percentage of prespecified items found to emerge in that respective analysis. Parentheses indicate the total number of items in preassigned algorithms. “HA” refers to Hyperactivity factor.
Items factored with Socially Unresponsive items. If these items are excluded from the comparison, then the rate of “correct” preassignment increases to 63.4% (for five-factor analysis) and 90.2% (for six-factor analysis). Such correction allows for the fact that it is not possible to derive six factors with EFAs specifying five factors.
In Table 3, the results are listed for ABC/Original and for ABCFXS, respectively. When possible, findings are presented for the various scoring algorithms (ABC/Original; ABCFXS) as predetermined by algorithm assignment (i.e., for the Hyperactivity, Irritability, Social Withdrawal or Socially Unresponsive [mutually exclusive], and Social Avoidance subscales). In the two rightmost columns, the overall combined rate of “correct” item assignment is summarized. Let us use the findings from the seven-factor analyses as a concrete example. For the ABC/Original, 60.0% of items were assigned to the Irritability subscale, 68.8% to the Hyperactivity subscale, 100% to the Social Withdrawal subscale, and 75.6% of items were aligned overall for all three subscales. Continuing with the seven-factor example for ABCFXS, 72.2% of items as determined by EFA were assigned to Irritability, 100% to the Hyperactivity subscale, 92.3% to Social Unresponsiveness, and no items were assigned to Social Avoidance. In this example, no Social Avoidance items emerged separately for ABCFXS algorithm.
As noted earlier, there has been a tendency in past studies for Irritability and Hyperactivity items to crisscross from one subscale to the other. In this study, the Irritability and Hyperactivity items were coupled together, and they did not decouple until seven-factor solutions were specified. To simplify the data for comparison, we assigned these data to undifferentiated “Combined factors” for the five- and six-factor results, and these combined factors were positioned under Hyperactivity to conserve space.
In general, alignment was better for ABCFXS than for ABC/Original when the combined Irritability/Hyperactivity items were decoupled. For the eight-factor comparison with the ABCFXS, we observed perfect alignment for Social Avoidance, but no items emerged separately for the Socially Unresponsive subscale. That is, the emergence of Social Avoidance seemed incompatible with Socially Unresponsive behavior appearing in this sample.
Brief overall analysis
Briefly, the seven-factor ABC/Original algorithm did a fairly good job of characterizing the results; both the Irritability and the Hyperactivity items emerged as independent factors, although there was a fair degree of cross-loadings among component items. All items encompassed within Social Withdrawal emerged. The Stereotypic Behavior and Inappropriate Speech subscales emerged fully intact (results not shown).
In contrast, the ABCFXS algorithm did not perform as well with these data. None of the first three ABCFXS factors included content from the Social Avoidance factor; when the Social Avoidance factor finally did emerge, it did so to the detriment of a Socially Unresponsive factor whose items completely disappeared (i.e., did not align with each other independent of several Hyperactivity items relating to deployment of attention, one that overlapped with Irritability, and others loading with an “Activity Valence” factor containing several negatively loaded items). Only 46.7% of the prespecified content from the eight-factor solution aligned with the ABCFXS algorithm (Table 3).
OFXW data
As with the FORWARD data, the OFXW analyses merged the Irritability and Hyperactivity factor content when five or six factors were specified (Table 4). With five- and six-factor EFAs, the content for Irritability and Hyperactivity again emerged as a single factor (87.7% endorsement for both five and six factors on ABC/Original and 90.3% and 89.3% assignment for ABCFXS [five and six factors, respectively]). The number of “correctly” prespecified ABC/Original items declined from 88.9% overall (five-factor solution) to 57.8% overall (seven- and eight-factor solutions) as factor complexity increased. For the Social Withdrawal subscale (ABC/Original), the number of correctly prespecified items declined from 92.9% (five-factor solution) to only 63.4% (with six, seven, and eight factors).
Conversely, alignment ranged from 38.5% (six-factor solution) to 84.6% (eight-factor solution) with the ABCFXS. For Social Avoidance items, concordance for the Socially Unresponsive factor was poor for five and six factors and very good for seven and eight factors. As a general observation, a “law of diminishing returns” seemed to apply, where fewer factors performed better for the ABC/Original algorithm and more factors performed better with the ABCFXS algorithm. Hence, the performance of the OFXW data depended, in large part, on the number of factors accepted; this was not the case for the FORWARD data.
Brief overall analysis
In short, preassignment of items resulting from the five- and six-factor analyses tended to capture traditional ABC content (Aman and Singh, 2017) when fewer factors were employed. However, this favorable characterization is only true if we are comfortable with both Irritability and Hyperactivity being bundled together. That is, if a lack of discrimination between the disruptive behavior subscales is acceptable, then little is lost; otherwise bundling is unacceptable. Conversely, the ABCFXS performed quite well when the target sample was specified with various degrees of Irritability, Hyperactivity, and especially Social Unresponsiveness and Social Avoidance.
Normative data
Normative data using the ABC/Original scoring algorithm appear in Table 5, with means and SDs presented separately for males and females. Percentile norms, again based on the ABC/Original scoring algorithm, appear in Supplementary Table S3. Norms for the ABCFXS scoring algorithm appear in Supplementary Table S4. These are broken out by age and sex; means are presented for males and females separately. Percentile norms for the ABCFXS algorithm appear in Supplementary Table S5.
Normative Data Using Aberrant Behavior Checklist/Original Algorithm for 678 Subjects with Fragile X Syndrome
Uses scoring algorithm specified in latest ABC Manual (Aman and Singh, 2017).
SD, standard deviation.
Discussion
Overlap between ABC/original social withdrawal and ABCFXS social avoidance
Whereas we predicted that the two subscale scores would result in highly comparable rank ordering of individual scores (i.e., r ≥ 0.75), the correlation between ABC/Original Social Withdrawal and ABCFXS Social Avoidance was only modest to moderate. As the content of the two clearly is significantly different, it would not be legitimate to let one (Social Withdrawal) serve as a proxy for the other (Social Avoidance). This methodological note indicates that the different names clearly signify constructs that are sufficiently different to place a limit on how we use and interpret these item clusters.
Summary of CFA results in FXS
Ultimately, CFAs did not resolve one of our primary questions: which of the structures of interest—the ABC/Original or ABCFXS—is preferable. Although analyses indicated a slight superiority of model fit for the ABCFXS algorithm over the ABC/Original and Brinkley et al.'s (2007) five-factor solutions, differences were only marginal and did not indicate a substantial advantage of the ABCFXS. This echoes findings of Wheeler et al. (2014). Simple visual inspection of the findings (from the EFAs) supports this impression. In the section that follows, we hope to demonstrate that any advantage was undermined by several other considerations.
Summary of EFA results in FXS
When we examined the simple structure of the ABCFXS and ABC/Original, we observed three important phenomena. First, the self-injury items tended to cluster independent of other factors that emerged in our analyses (irrespective of algorithm used). Second, as in past factor analyses with the ABC (Aman and Singh, 2017), there were considerable cross-loadings between the Irritability factor items and the Hyperactivity/Noncompliance items. The Irritability factor, as prespecified by the ABCFXS algorithm, performed slightly better on average than the ABC/Original assignment. Third, evidence of a Social Avoidance factor did not occur until late in the analyses (i.e., after extracting the maximum number of factors that seemed reasonable). For the FORWARD sample, this coincided with complete loss of the Socially Unresponsive items. Conversely, for the OFXW sample, this resulted in a fairly straightforward Social Avoidance factor as posited by Sansone et al. (2012). Although fit indices were fairly good for most ABC scoring algorithms, we would like to offer several observations why we believe it is prudent to stick with the original scoring algorithm, even among individuals with FXS.
Ambiguity of factor structure with social avoidance
We readily acknowledge that a more elaborate analysis of the OFXW data produced the predicted Social Avoidance factor separate from the Socially Unresponsive factor. However, when we “overfactored” to produce a Social Avoidance factor in the FORWARD sample, we also caused two undesirable occurrences. First, the solution that produced the Social Avoidance factor also eliminated the Socially Unresponsive items. Thus, forcing the analysis to produce the Social Avoidance factor eliminated all 13 Socially Unresponsive items in the FORWARD sample. Clinically, these “Socially Unresponsive” items seem highly relevant to individuals with FXS.
It is also important that this likely affects internal consistency of any Social Avoidance subscale. With only four items, the Social Avoidance factor is intrinsically unreliable and, because of this, it likely is a risky outcome measure, in that it may not capture subtle group differences or treatment effects. Furthermore, we wonder what is gained if Socially Unresponsive and Social Avoidance are split apart. One could argue that Socially Unresponsive and Social Avoidance are independent constructs that undermine signal from one another. However, this seems counter intuitive clinically and we know of no empirical data supporting the position. We suggest that the items are more useful if maintained intact.
Lack of parsimony
All other things being equal, the simplest explanation for a phenomenon is usually the best one. The original scoring algorithm for the ABC posits five subscales and results in only minor loss of items, even in this replication with individuals having FXS. Conversely, the ABCFXS is, by virtue of its greater complexity, more difficult to replicate.
Hampering/undermining communication
The purpose of standardized instruments is to compare and contrast clinical conditions and/or results of interventions in consistent ways. As the large majority of studies with the ABC have used the ABC/Original scoring system, introducing other scoring algorithms may well complicate communication. Let us highlight the issue by posing a hypothetical question. Would it make sense to develop an “ASD version” of the very widely used Child Behavior Checklist (CBCL)? We would argue no, as the purpose of administering the CBCL is to see how a given clinical group differs from the normative group. Having an “ASD CBCL” may fulfill no useful clinical role.
Factor analysis is a remarkable tool, and it has the virtue of impartiality (i.e., statistical programs do not intrinsically favor one solution or another). Due to sampling error, consecutive factor analyses rarely render exactly the same results on all occasions, even for the same clinical condition. That certainly was the case in our analyses, where additional factoring (i.e., with the OFXW data) produced results that differed from our other data (from Project FORWARD). Therefore, it is prudent to build a science of consensus around our measures of outcome within our broader field. This will enable comparisons to be made within clinical conditions and across clinical conditions. How else can we compare findings from studies of people with multiple etiologies, multiple ages, both sexes, and so forth?
Other issues raised by EFAs
Items numbers 2, 50, and 52 all address forms of self-injury, and all persisted in forming a unique factor of their own. This also occurred in Sansone et al.'s (2012) study (FXS) and in Brinkley et al.'s (2007) study conducted with children and adolescents having ASDs. This raises the question of whether these clinical populations display disruptive behaviors that are unique to them or whether this was a chance occurrence. Aman and Singh (2017) examined other factor analytic studies with the ABC and did not observe parallel outcomes, but we should remain vigilant for future recurrences.
As noted in the Introduction, replications of the ABC have found substantial agreement with the originally reported factor structure (ABC/Original). However, when disagreements have occurred, they mostly have appeared with the Irritability and the Hyperactivity/Noncompliance items showing some tendency to cross-load on the alternative factor. Likewise, in this trial, item content prespecified by Sansone et al. (2012) for Irritability and Hyperactivity/Noncompliance came closer to caregiver reports than that predicted by Aman and Singh (1986). The senior author (M.G.A.) intends to conduct a thorough examination of these disruptive behavior items to determine if an alternative subscale assignment will prove more consistent in the future.
Normative data
Means and SDs (Table 5) are provided in this publication for professionals who prefer to use the ABC/Original scoring algorithm. Because of their bulk, percentile norms are presented in the Supplementary Tables. Supplementary Table S3 contains percentile norms using the ABC/Original algorithm with 678 subjects having FXS. Supplementary Table S5 contains percentile norms using the ABCFRX algorithm with 678 subjects having FXS. Supplementary Table S4 contains means and SD norms for the ABCFRX algorithm with the same 678 subjects.
Of course, norms are a practical means to an end. Norms presented in this study do not convey the same meaning as in the general population. For example, clinicians will be focused on far more individuals having potentially problematic behavior than would be contained in the extreme 5% of the typically developing population. Also, because of distributional skew, we often encourage use of percentile norms in clinical practice. As we are often dealing with individuals having significant disabilities, clinicians may determine that it makes sense to focus on a high proportion of individuals, perhaps including even those whose scores fall in the upper 25th–40th percentiles.
Limitations and strengths
Two study limitations were as follows: first, the representativeness of our samples is unknown, although it is likely that the families who became involved in Project FORWARD and the OFXW study differed in some ways (e.g., parental education and motivation) from a truly random sample of individuals with FXS. Similarly, they likely differed from a truly random sample of individuals with other neurodevelopmental disorders. However, as FXS is a common population in which the ABC will be administered, it is helpful to evaluate evidence of its validity for such individuals. Second, there was some unknown level of overlap between families who were engaged in FORWARD and those from the OFXW study, which (based on interactions with subjects) we believe to be low.
One strength of this study is the large sizes of the samples (n1 = 797 and n2 = 357) involved, especially for a rare condition such as FXS. A second strength is the fact that we were able to independently compare the essential findings, across samples, using identical analyses. These comparisons, of course, revealed important inconsistencies across samples. One wonders if the composition of samples contributed to the different results across samples. The FORWARD dataset comprised clinically referred individuals only, the OFXW data comprised research participants only, and the Sansone sample largely comprised research participants.
Conclusions
At this stage, we feel that it is prudent to rely on the ABC/Original for scoring FXS data and to treat the ABCFXS algorithm as only experimental. We encourage use of the original algorithm for all populations of people with developmental disabilities, including those with FXS. There are numerous reasons for preferring the original algorithm, even among people with FXS. However, there may be very limited occasions when it is reasonable to use the ABCFXS, but only if diminished social anxiety or social avoidance is the clear and unequivocal goal.
Clinical Significance
The ABC has a long history and proven record for assessing behavior in clinical trials of both pharmacological and behavioral interventions. This study supports the psychometric properties of the original scoring algorithm among individuals with FXS. For consistency with previous studies and parsimony, we encourage other investigators to use the ABC/Original scoring algorithm in future research. With time, we hope that this will lead to establishing successful interventions in individuals with FXS and other syndromes alike.
Footnotes
Acknowledgments
We are grateful to the families who participated in FORWARD. We also acknowledge the contributions made to FORWARD by many staff members at each of the clinics participating in FORWARD. Without their efforts, this project would not have been feasible. We also acknowledge the contributions of the National Coordinator, Amie Milunovich, and members of the FORWARD Steering Committee: Elizabeth Berry-Kravis, MD, PhD, Milen Velinov, MD, PhD, Nicole Tartaglia, MD, Craig Erickson, MD, Howard Andrews, PhD, Walter Kaufmann, MD, Stephanie Sherman, PhD, and Centers for Disease Control and Prevention representatives. The authors would like to thank Gabrielle Basinski for preparing many of the tables in this article, as well as Melissa Raspa and Amanda Wylie for assisting with data management.
Disclaimers
Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services.
Disclosures
M.G.A. has received research contracts, consulted with, served on advisory boards, or done investigator training for Bracket Global; CogState Clinical Trials, Ltd.; J&J Pharmaceuticals; MedAvante-Prophase; Otsuka Pharmaceutical Development and Commercialization, Inc., Ovid Therapeutics; Supernus Pharmaceuticals; and Zynerba Pharmaceuticals. He receives royalties from Slosson Educational Publications. M.N. has received research contracts, consulted with, served on advisory boards, or done investigator training for the Simons Foundation, National Institutes of Mental Health, Department of Defense, Autism Treatment Network/Autism Intervention Research Network on Physical Health, and Cognoa. A.K. receives funding support from multiple National Institutes of Health grants to his university, all unrelated to this project. A.W. has received research contracts, consulted with, served on advisory boards, or done investigator training for Ovid Therapeutics, Roche Pharmaceuticals, Ionis Pharmaceuticals, and Sanofi Pharmaceuticals. C.E. has received current and/or past research support from the National Institutes of Health, the United States Department of Defense, the United States Centers for Disease Control, the John Merck Fund, Autism Speaks, the Simons Foundation, Cincinnati Children's Hospital Research Foundation, the FRAXA Research Foundation, the National Fragile X Foundation, the Roche Group, Seaside Therapeutics, Novartis, Neuren, Alcobra, the State of Ohio, Indiana University School of Medicine, and the Cincinnati Children's Hospital Research Foundation. He is a past consultant to Alcobra, the Roche Group, Fulcrum Therapeutics, and Novartis. He is a current consultant to Lenire Bioscience and Stalicla. He holds equity interest in and is a consultant for Confluence Pharmaceuticals. He is the inventor on intellectual property held by Cincinnati Children's Hospital Research Foundation and Indiana University describing methods for diagnosis and treatment methods in autism spectrum disorder and fragile X syndrome. H.A., T.-H.C., C.C., and C.B. report no conflicts of interest.
Supplementary Material
Supplementary Table S1
Supplementary Table S2
Supplementary Table S3
Supplementary Table S4
Supplementary Table S5
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
