The Factor Structure and Gender Invariance of ADHD Symptoms in College Students

Abstract

Previous studies examining the factor structure of attention-deficit/hyperactivity disorder (ADHD) symptoms in adults using self-report measures have shown mixed results, supporting two-, three-, and bifactor solutions. The current study further investigated the structure of ADHD symptoms in adults using the Current Symptoms Scale and rigorous model evaluation in a sample of 892 college students. Confirmatory factor analysis was used to analyze and compare five-factor structures; a single-factor model, a two-factor model, a three-factor model, and two bifactor models. A single-factor model with correlated residuals best fit the data. Factor correlations with nearly all related constructs (i.e., symptoms of oppositional defiant disorder, depression, impairment, previous ADHD diagnosis, grades, and substance use) were significant in the expected directions and the model was invariant across gender. These findings contribute to a growing body of work suggesting a unidimensional factor may best represent ADHD symptoms in adults. Implications are discussed.

Keywords

ADHD Barkley Current Symptoms Scale factor structure college students adults

Attention-deficit/hyperactivity disorder (ADHD) is a common neurological disorder of childhood characterized by pervasive and impairing patterns of inattention and/or hyperactivity–impulsivity symptoms that often persist into adulthood (American Psychiatric Association [APA], 2013). Previous findings of temporal instability in symptom presentation across development (Asherson et al., 2018) have raised questions about the structure of ADHD symptoms in adults and increased awareness of the need for a life span approach to ADHD. In response, numerous studies have examined the structure of ADHD symptoms using self-report measures developed to assess symptoms in adults (e.g., Gibbins et al., 2012; Glutting et al., 2005; Proctor & Prevatt, 2009; Span et al., 2002). Although some studies have supported a two-factor model of ADHD symptoms consistent with the Diagnostic and Statistical Manual of Mental Disorders (DSM; APA, 2013; Smith & Johnson, 1998), others have not supported this conclusion (Glutting et al., 2005; Park et al., 2018; Proctor & Prevatt, 2009; Span et al., 2002). Over the past decade, studies have increasingly advocated a more complex bifactor model to represent symptom presentation in adults (e.g., Gibbins et al., 2012; Stanton et al., 2018). However, recent criticisms concerning the application and interpretation of the bifactor model in studies evaluating the structure of psychopathology have called attention to the need for more rigorous evaluation of this approach (Bonifay et al., 2017). Heeding this call, several studies have revisited the contribution of the bifactor model and found little evidence to indicate this structure underlies ADHD symptoms in adults (Arias et al., 2018; Park et al., 2018). The purpose of the current study is to further investigate the structure of ADHD symptoms in adulthood by examining different factor structures in a college student sample as assessed by the Current Symptoms Scale (CSS; Barkley & Murphy, 2006), a widely used self-report measure of present symptoms (P. D. Rodriguez & Simon-Dack, 2013). Comparing one-, two-, and three-factor models as well as bifactor models of ADHD symptoms, the current study considers multiple indices to inform more rigorous model evaluation.

A Brief History of ADHD in the DSM

The hallmark symptoms of ADHD have been included in the DSM for over 50 years, first appearing in Diagnostic and Statistical Manual of Mental Disorders–Second edition (DSM-II; APA, 1968) as a brief description of “hyperkinetic disorder of childhood.” However, the name, the diagnostic criteria, and underlying structure of what is now known as ADHD have since undergone numerous revisions. The Diagnostic and Statistical Manual of Mental Disorders–Third edition (DSM-III; APA, 1980) gave prominence to inattentive symptoms and identified the syndrome as attention deficit disorder, which could present with or without symptoms of hyperactivity. The disorder was first identified as ADHD in the DSM-III-R (APA, 1987). This name change reflected a theoretical shift in diagnostic approach and in the understanding of the underlying structure of the disorder. Unlike the DSM-III, which listed attention deficit disorder symptoms as three separate factors (i.e., inattention, impulsivity, and hyperactivity), the DSM-III-R combined symptoms into a single list and did not include diagnostic subtypes to differentiate symptom presentations. Ensuing efforts to empirically define ADHD called into question the DSM-III-R conceptualization of a unidimensional disorder with heterogeneous symptom presentations and provided a basis for diagnostic criteria in subsequent editions of the DSM. The DSM-IV (APA, 1994), DSM-IV-TR (APA, 2000), and DSM-5 (APA, 2013) identify three subtypes of ADHD, with each indicating symptom elevations in inattention and/or hyperactivity/impulsivity dimensions.

As this brief history demonstrates, efforts to define ADHD and its underlying symptom structure have spanned decades and are ongoing as new evidence continues to emerge. A critical advancement in the understanding of ADHD occurred in the 1990s when it was finally recognized that the disorder did not subside with age as previously thought. The APA acknowledged ADHD in adulthood and included age-appropriate examples of workplace difficulties in the characterization of symptoms in the DSM-IV (APA, 1994). However, the two-factor model of ADHD presented in the DSM-IV was based on field trials conducted with only children and adolescents up to age 17 years (Lahey et al., 1994), which prompted concerns about validity in adults. Although ADHD was not assessed in the DSM-5 field trials in adults (Batstra & Frances, 2012), the DSM-5 (APA, 2013) retained the same symptoms and the two-factor structure of ADHD presented in the DSM-IV. A number of studies have examined the structure of ADHD in adults using confirmatory factor analyses; however, results have been mixed and questions remain about how to best conceptualize adult ADHD.

ADHD in Adults: Prevalence and Assessment

In response to growing awareness of ADHD in adulthood and concerns regarding utility of DSM criteria for identifying ADHD in adults (Fischer & Barkley, 2007), there has been a growth of research in recent decades examining ADHD in adults, including its prevalence, assessment, and factor structure (Biederman et al., 1994). Prevalence estimates of ADHD range from 2% to 8% among college students (DuPaul et al., 2009) and are around 3% in the general adult population (Fayyad et al., 2007; Kessler et al., 2006). A meta-analysis of past research found that ADHD persists into adulthood in about two thirds of cases (Faraone et al., 2006), though a higher estimate of 80% has also been reported (Barkley et al., 2002). Fewer symptoms of hyperactivity/impulsivity are generally endorsed from childhood into adulthood (Willcutt, 2012), suggesting that a two-factor model of ADHD may be vulnerable to developmental change and may not be an adequate representation of the adult phenotype (Hart et al., 1995; Larsson et al., 2011). Furthermore, although DSM criteria characterize ADHD as a disorder of childhood, recent population studies suggest that a sizeable proportion of adults report the onset of symptoms and impairment outside this period of development (Caye et al., 2016). Therefore, it is important to examine the patterning of symptom presentation in adults to characterize the ADHD phenotype beyond childhood.

College students may constitute a special population of interest with regard to ADHD symptom presentation and impairment given the unique features of this developmental period. The intersection of emerging adulthood as a transitional period of development and the unique demands of the college context, including increased academic, organizational, and social demands, may pose a particular challenge for students with ADHD (Arnett, 2000, 2016). Nationally, over a quarter of students who register with university disability services due so because of a diagnosis of ADHD, suggesting that college-age individuals are highly affected by ongoing concerns related to ADHD symptomology (DuPaul et al., 2009). Thus, advancing what is known about the conceptualization and assessment of ADHD in this population is an important goal.

Efforts to characterize and detect ADHD beyond childhood have resulted in the development of multiple measures specifically designed to assess symptoms of ADHD in adults, with many based on criteria established in the DSM-IV (APA, 1994). The CSS (Barkley & Murphy, 2006) is a self-report measure developed to assesses the presence of 18 ADHD symptoms within the past 6 months in adults, with items based on criteria defined in the DSM-IV-TR (APA, 2000). An earlier iteration of the widely used Barkley Adult ADHD Rating Scale–IV (BAARS-IV; Barkley, 2011), the CSS remains a common choice for clinicians and is recommended as an effective and efficient self-report scale to assess ADHD symptoms in adults (Knouse & Safren, 2010; P. D. Rodriguez & Simon-Dack, 2013; Taylor et al., 2011). Given the broad reliance on the CSS and other self-report measures to assess symptoms in adults, it is important that these measures adequately represent the structure of ADHD in adulthood. Furthermore, given that the presentation of symptoms has been shown to differ in children and adults (Willcutt, 2012), it is important to evaluate the structure of ADHD symptoms in adults as assessed by these self-report measures and in comparison to the two-factor structure conceptualized in the DSM (APA, 1994, 2000, 2013).

Factor Structures of ADHD

A number of previous studies have applied confirmatory factor analysis (CFA) to examine the factor structure of self-reported ADHD symptoms in adult samples using instruments based on DSM criteria, including the CSS (Barkley & Murphy, 2006). These studies have often compared traditional CFA models testing one-factor and two- and three-correlated factor structures of ADHD symptoms. More recently, studies have examined more complex bifactor models of ADHD symptoms. Results have been mixed, possibly reflecting differences in methodology and assessment approaches across studies. For example, although ADHD symptoms are heterogeneous in both presentation and severity (Nigg et al., 2005), patterns of symptom endorsement are likely to differ based on sample composition, with clinical samples generally demonstrating greater symptom severity and comorbidity compared with community samples (Wood et al., 2019). Another important issue concerns the criteria applied in the selection of a final model. Inadequate selection criteria, such as exclusive reliance on model fit without consideration of other important indices, may contribute to interpretational challenges and problematic conclusions regarding the structure of ADHD symptoms (Arias et al., 2018; Bonifay et al., 2017; Park et al., 2018; A. Rodriguez et al., 2016a). Identifying these issues is critical to ensure that the models promoted in previous research and in future studies provide the best representation of ADHD symptoms.

Previous studies have tested the one-, two-, and three-factor models of ADHD proposed in various editions of the DSM using traditional CFA but have yielded little consensus. For example, a CFA of self-reported symptoms in a large undergraduate student sample (N = 1,503) indicated support for a two-factor structure of ADHD, consistent with the conceptualization of ADHD in the DSM-IV and in the current edition, DSM-5 (Smith & Johnson, 1998). However, other studies have indicated a three-factor correlated model of inattention, hyperactivity, and impulsivity provides the best fit of self-reported symptoms in adult samples (Glutting et al., 2005; Park et al., 2018; Proctor & Prevatt, 2009; Span et al., 2002). Two of these studies (Park et al., 2018; Proctor & Prevatt, 2009) assessed self-reported ADHD symptoms using the CSS. Park et al. (2018) examined the factor structure of self-reported symptoms as assessed by either the CSS or its newer counterpart, the BAARS, in a pooled sample comprising parents of children with ADHD from three data sets. Results in that study indicated that one- and two-factor models demonstrated inadequate fit to the data, whereas a three-factor model showed adequate and comparatively better fit, had good reliability and construct replicability, and evidenced invariance across data set and gender. Proctor and Prevatt (2009) assessed self- and other-reported current or childhood CSS symptoms in a clinically referred sample of college students, the majority of whom were diagnosed with ADHD following a standardized evaluation. Results in that study indicated that although the three-factor model was a better fit than the one- or two-factor models, none of the models fit the data particularly well

Beyond examining model fit indices, results in Park et al. (2018) and Proctor and Prevatt (2009) point to the importance of examining interfactor correlations to guide model selection. As suggested by Brown (2015), interfactor correlations exceeding .85 may indicate that factors are not sufficiently independent and should be combined. Park et al. (2018) reported that correlations between inattention, hyperactivity, and impulsivity factors were high (ranging from .67 to .76), but did not exceed this suggested cutoff. Therefore, the three-factor model was retained in that study. Proctor and Prevatt (2009) also reported high correlations between impulsivity and hyperactivity for self-reported current symptoms (.82) and for self- and other-reported childhood symptoms (.83 and .88, respectively). Nonetheless, the authors supported a three-factor model for self- and other-reported current and childhood symptoms, noting that high interfactor correlations should be interpreted with caution given that models in that study demonstrated overall poor fit to the data. In other previous studies (e.g., Gomez et al., 2005; Wolraich et al., 2003), high interfactor correlations have supported the selection of more parsimonious two-factor models. Taken together, these studies demonstrate that decisions guiding model selection should take into consideration the extent to which factors in a CFA model are independent by examining interfactor correlations. However, reporting and application of interfactor correlations have been inconsistent and likely contribute to the lack of consensus across previous factor analytic studies of ADHD symptoms in college students and other adult populations.

In addition to traditional CFA models, bifactor models have recently become a focus of studies examining self-reported ADHD symptom in adults. In a bifactor analysis, a general ADHD factor captures common variance across all symptoms and is orthogonal to any specific factors (e.g., inattention, hyperactivity/impulsivity). This approach allows the variance shared by all items to be partitioned from the variance associated with each specific factor. This is in contrast to a two-correlated factor model in which item response variance is partitioned into either error variance or the combined direct plus indirect effects for each factor (Arias et al., 2018).Thus, a potential benefit of a bifactor model is that the identification of reliable specific factors would indicate the utility of subscales for prediction and outcome measurement, and more broadly, address the distortion that occurs when attempting to fit a unidimensional model to multidimensional data (Reise et al., 2007).

A number of studies comparing traditional CFA and bifactor models have concluded that ADHD symptoms in adults are best represented by a bifactor structure, with model selection often based on comparisons of fit indices. For example, based on the magnitude of fit indices, a bifactor model with two specific factors of inattention and hyperactivity/impulsivity was found to be the best model of ADHD symptoms in both children and adults (Martel et al., 2012). Fit indices also indicated support of a bifactor model with three specific factors of inattention, motor impulsivity/hyperactivity, and verbal impulsivity/hyperactivity as the best solution in a sample of noncollege student adults (Gibbins et al., 2012) and in a clinical sample of adults (Stanton et al., 2018).

Taken together, results in studies testing bifactor models of ADHD have been interpreted as evidence to support this factor structure of ADHD in adults. Importantly, however, the interpretation of bifactor models when applied to validate the latent structure of psychopathology rather than to assess psychometric properties of an assessment tool has been the focus of recent criticism. As outlined by Bonifay et al. (2017), causes for concern in the use of bifactor models focus on key issues of interpretability, model fit, and validation. In brief, concerns about model interpretation highlight the challenges presented in interpreting group factors that are orthogonal to a general factor and the importance of construing these factors as substantively unique and as representing meaningful variance exclusive of that accounted for by the general factor. Concerns about model fit point to the tendency of the bifactor model to fit all possible data, including nonsense response patterns, and as a result, to demonstrate superior goodness-of-fit in model comparisons. This tendency for bifactor models to over fit data renders fit indices an inadequate basis of model selection and insufficient evidence to promote an underlying bifactor structure of psychopathology (Bonifay et al., 2017). Thus, reliance on statistical fit indices raises concerns about the validity of the bifactor model as applied in previous studies. To address these concerns, bifactor models should be interpreted with caution using more rigorous criteria to ensure that support for this model is indicated not only by better fit indices but also by statistical evidence that both the general and specific factors of ADHD meaningfully contribute to the presentation of symptoms in adults.

Heeding concerns about the methodological challenges of bifactor models, recently published studies have revisited the structure of ADHD symptoms using recommended statistical approaches to inform more rigorous model evaluation (A. Rodriguez et al., 2016a, 2016b). For example, a meta-analysis of 31 bifactor models in previously published studies found that the general ADHD factor explained most of the common variance and that the specific factors were not interpretable as indicated by low reliability estimates (Arias et al., 2018). However, that study found that, in clinical samples, inattention had adequate specificity and stability to support interpretation beyond the general factor. Bifactor models presented similar challenges to interpretation in a study by Park et al. (2018). That study, also described above, compared five possible factor structures of ADHD symptoms as assessed by either the CSS or its newer counterpart, the BAARS in a pooled sample of adults. Although bifactor models demonstrated expected superior fit, reliability and construct replicability statistics indicated that only the general factor was interpretable. Therefore, Park and colleagues concluded that the more parsimonious three-factor model of inattention, hyperactivity, and impulsivity provided the better solution.

Together, findings in these recent studies suggest that reliance on fit indices for model comparisons in previous studies likely inflated conclusions regarding the contribution and usability of specific factors derived from bifactor models. Given that reliability is an aspect of construct validity, poor reliability of specific factors suggests that dimensional subtypes may have relatively limited utility in research and clinical practice (e.g., as modifiers of an ADHD diagnosis) in adults. Instead, given that reliability has been high for a general factor but poor for specific factors in previous bifactor models, overall ADHD symptoms appear to be a better indicator of ADHD risk. A general factor of ADHD symptoms is consistent with etiological theories of ADHD that emphasize the role of a unifying construct such as Barkley’s (1997) theory of ADHD, which proposes that deficits in behavioral inhibition and associated secondary impairments in executive functioning underpin the structure of ADHD and unify the symptom dimensions of inattention and hyperactivity/impulsivity. A unidimensional model of ADHD is also consistent with the assessment of total ADHD symptoms as a common metric in clinical research examining impairment and/or negative outcomes (e.g., substance use) associated with ADHD symptomatology (e.g., Flory et al., 2003; Upadhyaya & Carpenter, 2008).

The Current Study

The purpose of the current study was to investigate and clarify the structure of ADHD symptoms in college students. Five CFA models were conducted and compared by examining fit indices and other decision criteria (e.g., independence and reliability of factors). Correlates of the final factor model with multiple constructs known to be strongly associated with ADHD symptoms, including gender, academic achievement (i.e., GPA), alcohol, tobacco, and marijuana use, depression symptoms, oppositional defiant disorder (ODD) symptoms, and stimulant medication usage (DuPaul et al., 2001) were also examined. We also investigated the measurement invariance of the final model across gender, given ADHD is more commonly diagnosed in males of all ages than females (Nussbaum, 2012).

Models tested included a single-factor model of total ADHD symptoms, a two-factor model using the ADHD symptom dimensions of inattention and hyperactivity/impulsivity as defined in the DSM-IV-TR (APA, 2000) and the CSS manual (Barkley & Murphy, 2006), as well as a three-factor model of inattention, hyperactivity, and impulsivity. To address recent methodological concerns regarding the application of bifactor models, the current study also tested two bifactor models with either two (inattention and hyperactivity/impulsivity) or three (inattention, motor hyperactivity/impulsivity, and verbal hyperactivity/impulsivity) factors guided by prior research (Gibbins et al., 2012; Martel et al., 2012; Stanton et al., 2018) and using recommended statistical indices in model comparisons.

Given evidence indicating that specific factors derived from bifactor models are not interpretable given poor reliability and a highly reliable general factor that accounts for the majority of variance as well as findings of high interfactor correlations in traditional two- and three-factor models, the structure of ADHD symptoms in college students was expected to be best represented by a one-factor solution. We expected the one-factor model of ADHD symptoms to be strongly associated with lower GPA, greater alcohol, tobacco, and marijuana use, greater depression symptoms and ODD symptoms, and more stimulant medication usage. Finally, we expected the one-factor model of ADHD symptoms to be invariant to gender, consistent with prior research (DuPaul et al., 2001; Gomez, 2016; Park et al., 2018).

Method

We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study.

Participants and Procedure

Participants were 900 undergraduate students aged 18 years or older at a large, public university in the southeastern United States. The sample was predominantly female (75.8%). Participants indicated their race as predominantly Caucasian (80.3%) or African American (9.1%). There was a relatively even spread across the classes: 28.6% first year students, 28.0% second year, 23.5% third year, 16.9% fourth year, and 3.0% fifth year or beyond. In terms of age, the median age among the participants was 20 years; 97.4% of the participants reported their age between 18 and 24 years. Among the participants, 11.7% reported a previous diagnosis of ADHD and 11.6% of participants reported having a prescription for stimulant medication. This is slightly higher than previous prevalence estimates of ADHD in college students (i.e., 2% to 8%; DuPaul et al., 2009), though it is noted that our figures are based on self-reported ADHD and not the percentage of students with a documented medical diagnosis of ADHD.

The study was advertised on class Facebook pages and on the psychology department participant pool website. Professors of large nonpsychology classes also shared the link with their students. An online survey was administered to participants through Qualtrics during the 2014 spring semester. After consenting, students took 15 to 60 minutes to complete the survey, which included questions on demographic characteristics, prior ADHD diagnosis and medication status, grades, and the measures described below. They received course credit (if relevant) for participating in the study and were entered into a drawing to win a gift card. The university institutional review board approved the study protocol.

As a validity check, we included the same question about alcohol use twice in the survey, spaced 17 questions apart. The question asked, “In the past 12 months, how often did you drink beer, wine, wine coolers, or liquor?” Possible responses were “not at all,” “1-3 times,” “4-7 times,” “8-11 times,” “once a month,” “2-3 times a month,” “once a week,” “2-3 times a week,” “4-6 times a week,” “once a day,” “twice a day,” and “several times a day,” with each assigned a number in order from 1 to 12. Participants were removed from the sample if their responses on the two identical questions were more than two answers apart. For example, an invalid response would be “2-3 times a month” for the first question and “4-6 times a week” for the second. The original data set had 936 participants. Thirty-six participants were removed from the original data set for such invalid responses.

Across all variables used for data analyses, the percentage of missing data was very low, ranging from 0% to 2.5%. Eight participants were not included as they had missing data on all used variables. All remaining participants (N = 892) were used in data analyses. As we were fitting ordinal factor analysis models with unweighted least squares, pairwise deletion (default method in Mplus) was used for handling missing data.¹

Measures

ADHD and ODD Symptoms and Impairment

The CSS (Barkley & Murphy, 2006) was used to assess for current ADHD and ODD symptoms. Based on the DSM-IV-TR (APA, 2000), the CSS includes 18 items describing behavioral manifestations of the two ADHD symptom dimensions, inattention and hyperactivity/impulsivity (e.g., “Lose things necessary for tasks or activities”), and eight items describing behavioral manifestations of ODD (e.g., “Blame others for my mistakes or misbehavior”). Participants rated the frequency with which they had experienced each behavior in the previous six months on a 4-point scale ranging from (never or rarely) to (always). Behaviors rated as having occurred “often” or “always” were considered a current symptom endorsement (Barkley & Murphy, 2006). The CSS was an early precursor of the BAARS-IV (Barkley, 2011). The CSS and BAARS-IV are nearly identical, the only difference being that the BAARS-IV includes more detailed wording for select items (e.g., “Easily distracted” on the CSS vs. “Easily distracted by extraneous stimuli or irrelevant thoughts” on the BAARS-IV). In the current study, the Cronbach’s alpha coefficients for the measures of ADHD and ODD were 0.939 and 0.851, respectively.

The CSS (Barkley & Murphy, 2006) was also used to measure impairment through 10 items that asked participants to what extent any of the ADHD behaviors endorsed interfere with functioning in different life domains (e.g., work, educational activities, and driving a vehicle). Response options were also on a 4-point scale ranging from (never or rarely) to (always). The Cronbach’s alpha of the measure of impairment from the current sample was .932.

Depression Symptoms

Six items from the Center for Epidemiological Studies Depression Scale (CESD; Radloff, 1977) were used to assess symptoms of depression. Respondents were asked to rate the symptoms during the last 2 weeks on a 5-point Likert-type scale ranging from 1 (not at all or less than 1 day a week) to 5 (nearly every day). We administered a longer depression scale in the survey (the 20-item CESD-R; Eaton et al., 2004), but for the current study elected to use only the six items that were identical in wording to items from the original CESD. Importantly, the CESD-R has demonstrated strong reliability in other studies (α = .93; Van Dam & Earleywine, 2011) and the 20 items of the CESD-R correlated strongly (r = .95) with the six-item short form we utilized. The internal consistency (Cronbach’s alpha) of the six-item short form for depression in our sample was .876.

Alcohol, Tobacco, and Marijuana Use

Alcohol usage was measured by two indicators: “In the past 12 months, how often did you drink beer, wine, wine coolers, or liquor” (1[not at all] to 12 [several times per day]), and “Think of all the times you drank in the past 12 months. On average, how much did you drink each time” (1[less than one can or glass] to 13 [more than 25 drinks]). Participants also reported how often they smoked cigarettes or marijuana (1[not at all] to 12 [several times per day]). For the current study, we treated the outcomes as continuous to represent participants’ degree of alcohol, tobacco, and marijuana use.

Data Analytic Plan

We first examined the factor structure of ADHD symptoms measured by the CSS using CFA. Given that the number of response categories was small (i.e., four), ordinal factor analyses were conducted using unweighted least squares estimation method (ULSMV), as is recommended by previous studies (Forero et al., 2009; Shi, DiStefano, et al., 2018). Specifically, we fitted five ordinal factor analysis models: (1) one-factor model (general ADHD symptoms), (2) two-factor model (inattention + hyperactivity–impulsivity), (3) three-factor model (inattention + hyperactive + impulsive), (4), bifactor model with two specific factors (general ADHD + inattention + hyperactivity–impulsivity), and (5) bifactor model with three specific factors (general ADHD + inattention + motor hyperactivity–impulsivity + verbal hyperactivity–impulsivity). In Figure 1, we showed the diagrams of all models we fitted in the current study. The model fit was evaluated using the root mean square error of approximation (RMSEA), the comparative fit index (CFI), the Tucker–Lewis index (TLI), and the standardized root mean square residual (SRMR). We also examined the individual residual correlations (RC) to assess local fit. For acceptable fit, we used the following reference values: (1) RMSEA ≤ .06, (2) CFI ≥ .95, (3) TLI ≥ .95, (4) SRMR ≤ .10 × ${\bar{R}}^{2}$ , and (5) RC ≤ .15 (Hu & Bentler, 1999; Shi, Maydeu-Olivares, et al., 2018). The robust chi-square difference test (Asparouhov & Muthén, 2010) was also conducted to compare nested models.

Figure 1.

Diagrams of fitted models.

We also computed the model-based reliability coefficients for each model we tested. Specifically, for the one-factor, two-factor and three-factor models, the coefficient omega (ω, McDonald, 1999) was computed. When fitting bifactor models, following the guidelines by A. Rodriguez et al. (2016a, 2016b), we computed the omega hierarchical (ω_H) for the general factor and the omega hierarchical subscale (ω_HS) for each specific factor. In addition, to measure the degree of unidimensionality of the ADHD symptoms, we computed the explained common variance (ECV) index (Reise, 2012).

Once the factor structure of ADHD symptoms was confirmed, we examined the correlations between ADHD symptoms and related latent constructs (i.e., ODD symptoms, impairment, and depression symptoms) using a CFA model. We also tested the associations between ADHD symptoms and two covariates (i.e., gender and history of ADHD diagnosis) by including the covariates as predictors for ADHD symptoms. In addition, we examined the predictive utility of ADHD symptoms for several outcomes of interests, including GPA and the usage of alcohol, tobacco, and marijuana.

Finally, we investigated if the ADHD symptoms are measured in the same way across gender groups. Using multiple group CFA models, measurement invariance tests were conducted between female and male participants (Millsap, 2012). Specifically, following the procedure recommended by Van de Schoot et al. (2012), we first examined configural invariance by checking whether the same measurement model fits in both groups. Given that the same factor structure held for both females and males, we further tested if the parameters in the measurement model were equal across genders by comparing the fit of a series of nested models. For the baseline model, all model parameters were freely estimated (except for the constraints for the purpose of model identification). Then, a series of more constrained models were fitted by fixing the parameters of interest to be equal across groups. Since factor analysis models were fitted with ordinal data, weak invariance was tested by constraining the factor loading of each item equivalent across genders; strong invariance implied that both the factor loading and thresholds of each item were equal across gender (Millsap & Yun-Tein, 2004). To test invariance at different levels of constraints, likelihood ratio (chi-square difference) test was employed. As the unweighted least squares with mean- and variance-adjustment estimation method (ULSMV) was used, robust chi-square difference tests were conducted using the procedure described in Asparouhov and Muthén (2006, 2010). A nonsignificant chi-square difference test indicates that measurement invariance holds. Additional tests of invariance were conducted by examining the differences in the goodness-of-fit indices (Chen, 2007; Cheung & Rensvold, 2002). Following the recommendation by Chen (2007), a change of < −.010 in CFI, a change of <.015 in RMSEA or a change of < .030 in SRMR would be used as evidence for invariance. Provided that strong invariance holds, we compared the latent means of ADHD symptoms across genders (Shi et al., 2017). All data analyses were conducted using Mplus 8.0 (Muthén & Muthén, 1998-2012).

Results

Factor Structure of ADHD Symptoms

Descriptive statistics for all items used in the study are provided in Supplementary Tables S1 and S2 (available online). The fit indices and the standardized factor loadings for the five CFA models are summarized in Table 1 and Table 2, respectively. In Table 2, we also reported the model-based reliability coefficients for each factor fitted in the models. We can see that the one-factor model fit the data adequately well (CFI = 0.97, TLI = 0.97, RMSEA = 0.06, SRMR = 0.05), except that sizable local misfit was found with the largest RC (LR) of .17. The coefficient omega was 0.96. The fits of the two-factor model (CFI = 0.98, TLI = 0.98, RMSEA = 0.06, SRMR = 0.04, LR = 0.15) and three-factor model (CFI = 0.98, TLI = 0.98, RMSEA = 0.05, SRMR = 0.03, LR = 0.13) were better than the one-factor model. The robust chi-square difference test (Asparouhov & Muthén, 2010) indicated that the two-factor model fit significantly better than the one-factor model (i.e., Δχ² = 51.35, degrees of freedom [df] = 1, p < .001). The three-factor model yielded a significantly better fit than the two-factor model (i.e., Δχ² = 39.95, df = 2, p < .001). The reliability coefficients of the subscales were adequately high, with coefficient omega ranging from .92 to .94 (two-factor model) and from 0.87 to 0.99 (three-factor model), respectively. However, for both the two-factor and three-factor models, the interfactor correlations among the factors were very high (i.e., ranging from .89 to .93).² Therefore, although a two-factor model or three-factor model fit the data significantly better (i.e., the interfactor correlations are not perfect), from a practical viewpoint, given interfactor correlations of .90 or higher, the ADHD symptoms can be treated as unidimensional.

Table 1.

Goodness of Fit Indices Across Models.

	Models
	One-factor	Two-factor	Three-factor	Bifactor (two specific factors)	Bifactor (three specific factors)	One-factor (with correlated residuals)
χ²(df)	616.534 (135)	504.744 (134)	456.720 (132)	340.041 (117)	315.144 (117)	502.909 (132)
CFI	0.974	0.980	0.982	0.988	0.989	0.980
TLI	0.971	0.977	0.980	0.984	0.986	0.977
RMSEA	0.063	0.056	0.053	0.046	0.044	0.056
SRMR	0.040	0.036	0.033	0.027	0.026	0.036
LR	0.171	0.150	0.131	0.099	0.095	0.118

Note. CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; LR = largest residual correlation. All chi-square values are significant.

Table 2.

Standardized Factor Loadings and Reliability Estimates for Each Model.

Items	1F	2F		3F			Bifactor (two specific factors)			Bifactor (three specific factors)			1F (corr. res.)
Items	G	IN	H/Imp	IN	H	Imp	G	IN	H/Imp	G	IN	ImpM	ImpV	G+
Fails to give close attention to details	.73	.74		.74			.67	.39		.69	.35			.73
Fidgets with hands	.67		.68		.70		.68		.28	.66		.34		.65
Has difficulty sustaining attention	.82	.83		.83			.78	.25		.80	.19			.82
Leaves seat in classroom	.79		.80		.82		.81		−.05	.81		−.06		.79
Doesn’t listen	.77	.78		.78			.73	.25		.75	.19			.77
Runs about or climbs excessively	.71		.73		.74		.73		.48	.70		.52		.69
Does not follow instructions	.81	.83		.83			.75	.40		.77	.37			.81
Has difficulty playing quietly	.83		.84		.86		.84		.13	.83		.13		.83
Has difficulty organizing tasks	.80	.82		.82			.75	.38		.77	.34			.81
Acts as if driven by motor	.54		.55		.56		.55		.29	.53		.33		.52
Avoids tasks that require mental effort	.73	.74		.74			.66	.40		.69	.37			.73
Talks excessively	.66		.68		.69		.68		.02	.66			.22	.66
Loses things	.82	.84		.84			.79	.22		.81	.14			.82
Blurts out answers	.80		.82			.85	.82		−.09	.79			.33	.80
Is easily distracted	.83	.84		.84			.79	.25		.81	.18			.83
Has difficulty awaiting turn	.85		.87			.92	.89		−.22	.85			.28	.86
Is forgetful in daily activities	.82	.83		.83			.78	.27		.80	.19			.82
Interrupts or intrudes on others	.79		.81			.85	.82		−.19	.78			.42	.79
Reliability	ω	ω		ω			ω_H	ω_HS		ω_H	ω_HS			ω
	.96	.94	.92	.99	.87	.91	.93	.14	.01	.92	.10	.10	.13	.96

Note. Six confirmatory factor analysis models tested the following alternative structures of attention-deficit/hyperactivity disorder: IF = one general factor (G); 2F = two general factors of (1) inattention (IN) and (2) hyperactivity (H)/impulsivity (Imp); 3F = three general factors of (1) IN, (2) H, and (3) Imp; bifactor (two specific factors) = one general factor and two specific factors of (1) IN and (2) H/Imp; bifactor (three specific factors) = one general factor and three specific factors of (1) IN, (2) motoric impulsivity (ImpM), and (3) verbal impulsivity (ImpV); 1F(corr. res.) = one general factor with correlated residuals (G+).

The bifactor model with a general ADHD factor and three specific factors yielded the best fit among all fitted models (CFI = 0.99, TLI = 0.99, RMSEA = 0.04, SRMR = 0.03, LR = 0.10). A similar model fit was observed for the bifactor model with two specific factors (CFI = 0.99, TLI = 0.99, RMSEA = 0.05, SRMR = 0.03, LR = 0.10). The results were not surprising; as noted by methodologists (e.g., Bonifay et al., 2017), the bifactor model tends to show superior goodness of fit when compared with other models, and thus the better fit of a bifactor model may be a symptom of “overfitting.” Therefore, researchers are cautioned against accepting a bifactor solution based only on the criteria of improved fit indices (Murray & Johnson, 2013; Samuel, 2019). For both bifactor models we fitted, very low or negative factor loadings were observed for the specific factors. We further examined the model-based reliability coefficients. For the bifactor model with three specific factors, the reliability due to the general factor was high (ω_H = .92). However, the reliability of the subscale scores was low; the omega hierarchical subscale (ω_HS) for the subscales of inattention, motor hyperactivity–impulsivity and verbal hyperactivity–impulsivity were .10, .10 and .13, respectively. Similar results were observed for the bifactor model with two specific factors. Therefore, the specific factors from both bifactor models do not appear to be meaningful subconstructs. In addition, the ECV for both bifactor models we fitted were high (i.e., ECV = .88 for the bifactor model with two specific factors and ECV = .86 for the bifactor model with three specific factors). These results suggested that the general factor of ADHD symptoms explained more than 85% of the common variance extracted, supporting that ADHD symptoms are best represented by a unidimensional model.

By comparing the five individual factor models, we found that a single factor of ADHD symptoms globally fit the data well, but with some nonignorable local misfit. To capture the remaining covariances exclusive to the general factor, the bifactor model seemed not applicable as the specific factors did not have meaningful interpretations. We thereby adopted the one-factor model with correlated residuals as the final model. Specifically, guided by the individual RC (Maydeu-Olivares & Shi, 2017),³ we added correlated residuals among three items (i.e., “fidgets with hands,” “shifts around excessively,” and “acts as if driven by motor”). It is noted that theoretically the correlated residuals are interpretable. That is, the residuals among the three items over and above the ADHD symptoms factor are correlated because the wordings of the three items all reflect certain motor activities. The final model (one-factor with correlated residuals) fit the data well (CFI = 0.959; TLI = 0.932; RMSEA = 0.06; SRMR = 0.04; LR = 0.12); the size of standardized factor loadings ranged from .52 to .86 ( ${\bar{R}}^{2} = . 58$ ). For the final model, the model-based reliability was high (ω = .96).

Relation Between ADHD Symptoms and Other Variables of Interest

After confirming the factor structure of ADHD symptoms, we investigated the relation between ADHD symptoms and three related constructs, including ODD symptoms, impairment, and depression symptoms. Since these three constructs were measured and modeled as latent variables, we examined measurement models before running the structural model. For all three constructs, a single-factor fit the data adequately well. The final measurement model including CSS and the other three correlated constructs also fit the data well (CFI = 0.96, TLI = 0.95, RMSEA = 0.05, SRMR = 0.05). The correlations among the four latent variables are summarized in Table 3. Significant positive associations were found among all four latent variables. Also, according to Cohen’s (1988) conventions for interpreting correlation effect sizes, the associations were large, with correlation coefficients ranging from .60 (depression with impairment) to .88 (ADHD with impairment).

Table 3.

The Interfactor Correlation Between ADHD and Other Variables of Interest.

	ADHD	ODD	IM	DEP
ADHD	1.00
ODD	0.63	1.00
IM	0.88	0.61	1.00
DEP	0.61	0.61	0.60	1.00

Note. ADHD = attention-deficit/hyperactivity disorder; ODD = oppositional defiant disorder symptoms; IM = impairment; DEP = depression symptoms.

Table 4 exhibits the effects (standardized coefficients) of gender and diagnostic history on ADHD symptoms. By checking the direct path from gender to ADHD symptoms (as a latent factor under a CFA model), we did not find a significant gender difference in ADHD symptoms ( $\hat{β} < . 01$ , p =.94).⁴ However, for students who had been diagnosed with ADHD, their ADHD symptom scores were higher by approximately 1 standard deviation than those who had not been diagnosed ( $\hat{β} = 1.1$ , p < .00). Finally, we used total ADHD symptoms (factor) to predict the academic and behavioral outcomes of interests. As shown in Table 4, ADHD symptoms were positively associated with alcohol, tobacco, and marijuana use, with standardized effect sizes of .20, .21, and .17, respectively. After controlling for number of years at college, higher ADHD symptoms were associated with lower college GPA ( $\hat{β} = - . 15$ , p < .00).

Table 4.

The Relations Between ADHD Symptoms (as Latent Factor) and Other Variables of Interest.

	Variables	β^	SE	p
ADHD as outcome	Gender	<0.01	0.080	.937
ADHD as outcome	Diagnostic	1.08	0.095	<.001
ADHD as predictor	Alcohol	0.20	0.042	<.001
	Tobacco	0.21	0.033	<.001
	Drug	0.17	0.033	<.001
	GPA^a	−0.15	0.031	<.001

Note. ADHD = attention-deficit/hyperactivity disorder; SE = standard error. All the effects reported are standardized coefficients. Diagnostic indicates if a student is diagnosed with ADHD (1) or not (0). Alcohol = Alcohol use; Tobacco = Tobacco use; Drug = Marijuana use.

The effect of ADHD symptoms on GPA after controlling for number of years at college.

Measurement Invariance of ADHD Symptoms Across Gender

The results of the measurement invariance tests are reported in Table 5. As shown in the table, the baseline model with minimum parameter constraints fit the data well (CFI = 0.98, TLI = 0.98, RMSEA = 0.04, SRMR = 0.04), which provided support that configural invariance held. The weak invariance with equal factor loadings across gender was also supported by both the chi-square difference test (Δχ²= 16.62 [df = 17], p = .481) and the changes in goodness of fit indices (ΔCFI = 0.006, ΔRMSEA = −0. 010, ΔSRMR = 0.004). For strong invariance with both equal factor loadings and equal thresholds, the chi-square difference test was significant at the alpha level of .05, but not at the alpha level of .01 (Δχ² = 76.88 [df = 51], p = .011). By using the test based on changes in goodness of fit indices (ΔCFI < 0.0001, ΔRMSEA = −0. 003, ΔSRMR = 0.001), strong invariance was supported. Finally, provided that strong invariance held, we comparted the latent means of ADHD symptoms across gender. The difference of levels of ADHD symptoms between females and males was not statistically significant (difference = −.055, p = .524).

Table 5.

Results From Measurement Invariance Tests.

Model	χ²(df)	Δχ² (Δdf)	CFI	TLI	RMSEA	SRMR
Configural invariance	469.925 (266)	—	0.984	0.982	0.042	0.039
Weak invariance (loadings)	413.756 (283)	16.615 (17), p = .481	0.990	0.989	0.032	0.043
Strong invariance (loadings+ thresholds)	460.034 (334)	76.876 (51), p = .011	0.990	0.991	0.029	0.044

Note. CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; df = degrees of freedom. The Δχ2 (Δdf) column reports the robust chi-square difference test (Asparouhov & Muthén, 2010).

Discussion

The principal aim of this investigation was to examine the unique factor structure of ADHD symptoms based on the CSS self-report measure in a nonclinical college student sample. Five CFA models were conducted and evaluated, including traditional one-, two-, and three-factor models as well as two bifactor models with either two (inattention and hyperactivity/impulsivity) or three (inattention, motor hyperactivity/impulsivity, and verbal hyperactivity/impulsivity) factors. Results indicated that, as expected, both bifactor models demonstrated superior fit to the data relative to the other models tested, reflecting the tendency of bifactor models to overfit data (e.g., Bonifay et al., 2017). The results further revealed that the bifactor models did not reliably contribute any meaningful information beyond that accounted for by a general factor of ADHD symptomology. Therefore, there was insufficient evidence to promote an underlying bifactor structure of ADHD symptoms. Evaluating fit indices and other parameter estimates for the one-, two, and three-factors models supported the selection of a one-factor model with correlated residuals as the final model of ADHD symptoms, suggesting that overall symptoms may be an adequate indicator of ADHD risk as assessed in the present study.

The conceptualization of ADHD in adults is a relatively new area of exploration yet an important one given that a large majority of cases of ADHD persist into adulthood and many are newly identified during this period of development (Barkley et al., 2002; Caye et al., 2016; Faraone et al., 2006). Findings in previous studies have not yielded broad consensus. Some studies have supported the traditional two-dimensional DSM conceptualization (inattentive and hyperactive/impulsive symptoms). However, studies have increasingly found that this model of ADHD symptoms may not adequately represent symptom presentations beyond childhood and adolescence. Instead, many studies have promoted a three-factor model with separate but related factors of inattention, hyperactivity, and impulsivity to represent ADHD in college students and other adult populations. The present investigation contributes to this body of work by finding that a one-factor model with correlated residuals best represented CSS self-reported current ADHD symptoms in a nonclinical sample of college students. Therefore, as assessed in this study, the unidimensional structure of ADHD symptoms was adequately represented by overall symptoms rather than dimensional symptoms of inattention, hyperactivity, and/or impulsivity.

The selection of the final model in the present investigation was informed by comparison of one-, two-, and three-factor models as well as bifactor models of ADHD and guided by consideration of multiple indices to ensure rigorous evaluation of each model. Initial evaluation of the one-factor model revealed adequate fit to the data but with sizable local misfit that could be improved with model modifications. In comparison, the two- and three-factor models demonstrated superior fit to the data, but high interfactor correlations (i.e., ≥ .89) indicated that factors in these models were not sufficiently independent and that symptoms could instead be treated as unidimensional. The modification of the one-factor model was guided by individual RC (Maydeu-Olivares & Shi, 2017), which indicated that adding correlated residuals among three items was warranted. The residuals among these items over and above the ADHD symptoms factor were likely correlated because each reflected motor activities. Thus, the correlated residuals are theoretically interpretable. In addition to fitting the data well and representing the most parsimonious solution, the unidimensional factor of ADHD symptoms with correlated residuals was related in expected ways to a number of constructs known to be associated with ADHD symptoms, including academic achievement, substance use, and impairment. Additionally, we found that the unidimensional factor structure was unrelated to gender, which is consistent with previous research that has found factor structure to be invariant to gender (DuPaul et al., 2001; Gomez, 2016; Park et al., 2018) despite higher prevalence rates for ADHD in males among individual of all ages (Nussbaum, 2012).

Evaluation of the bifactor models in the present investigational lend support to recent work questioning whether a bifactor model approach should be utilized when attempting to describe the overall structure of a particular set of symptomologies. Bonifay et al. (2017) delineate methodological challenges presented by bifactor models, including limitations concerning model contributions to the interpretability and validation of factors. Bonifay et al. (2017) also warn that, while bifactor models tend to show superior goodness of fit in model comparison studies, this fit may be artificially inflated due to the model capturing random noise. Therefore, fit indices are an inadequate basis of model selection and do not provide sufficient evidence to promote an underlying bifactor structure of psychopathology (Bonifay et al., 2017). Considering these methodological concerns, reliance on statistical fit indices raises concerns about the validity of the bifactor model as applied in previous studies (e.g., Gibbins et al., 2012; Martel et al., 2012; Stanton et al., 2018). As with recently published studies evaluating a bifactor structure of ADHD, the current investigation applied recommended statistical approaches to inform more rigorous model evaluation (A. Rodriguez et al., 2016a, 2016b). However, as in previous studies, further psychometric analyses on bifactor models did not reveal improved interpretability or validity in this investigation, suggesting the bifactor model should be used with caution. Bifactor models presented similar challenges to interpretation in a study by Park et al. (2018). The bifactor model showed the best fit for self-reported CSS symptoms in their adult sample but also poor reliability and construct replicability. Therefore, the bifactor model was rejected in that study in favor of a more parsimonious three-factor structure (inattention, hyperactivity, and impulsivity). Although the present investigation also rejected bifactor models of ADHD symptoms, unlike Park et al. (2018), our results indicated the selection of a unidimensional model of ADHD symptoms.

The unidimensional model of ADHD symptoms observed in the present study suggests that ADHD symptoms in a nonclinical sample of college students may be best represented by an overall score on a DSM-based assessment. Interpretation of this conceptualization of ADHD symptoms warrants careful consideration. One approach is theoretical—a one-factor model is consistent with primary core deficit theories of ADHD (e.g., Barkley, 1997), which suggests that endogenous dysfunction underlies ADHD symptomatology. This approach is supported by findings in a recent review of studies examining neuropsychological performance patterns associated with ADHD subtypes in adults (LeRoy et al., 2018). Findings in that review indicated that four neuropsychological domains (i.e., executive functioning, attention, working memory, and memory) reliably differentiated adults with and without ADHD but did not differentiate ADHD subtypes (with the possible exception of memory). However, other research has not supported a primary core deficit theory of ADHD (e.g., Sonuga-Barke & Castellanos, 2005). An alternative explanation for the findings in the present study highlight important methodological considerations in the interpretation of factor analytic results. Although ADHD symptoms are heterogeneous in both presentation and severity (Nigg et al., 2005), patterns of symptom endorsement are likely to differ based on sample composition with clinical samples generally demonstrating greater symptom severity and comorbidity compared with community samples (Wood et al., 2019). The implications of the clinical composition of a participant sample in evaluating the factor structure of ADHD were recently demonstrated in review by Arias et al. (2018). Although the objective of that review was to critically evaluate published bifactor models of ADHD, Arias and colleagues concluded that, consistent with the present investigation, ADHD symptoms may be best represented by the overall score on DSM-based measures in nonclinical samples. In clinical samples, however, the inattentive factor distinguished itself from the general factor with more statistical specificity and replicability, suggesting potential psychometric utility of a specific inattention factor in clinical settings. As noted by Arias et al. (2018), differences in the structure of ADHD symptoms between clinical and nonclinical samples may be explained by differences in response variability. Among nonclinical samples, it is likely that symptom ratings will be made at the lower end of the scale. This homogenous response pattern could confer higher correlations among symptoms and support the use of an overall score in the assessment of nonclinical samples. In contrast, greater heterogeneity of symptom endorsement in clinical samples may result in more clearly defined specific factors. Thus, although the present study supports a unidimensional model of ADHD symptoms in a nonclinical college student sample, additional research is needed to clarify the extent to which this finding should be interpreted as evidence that ADHD symptoms may be best represented as a continuum in a nonclinical sample rather than in support of a primary core deficit approach.

Differences in the clinical composition of participant samples may account for inconsistent findings observed in the present investigation and other factor analytic studies. For example, although Park et al. (2018) did not find support for a bifactor model with specific factors, the results did indicate that a three-factor model best represented the structure of ADHD. The parents of children with ADHD who comprised the sample in that study did not constitute a clinical sample per se but did present more elevated symptoms of ADHD than the general population, likely reflecting the high heritability of ADHD (Larsson et al., 2014). Future research should clarify the conceptualization of ADHD by examining the factor structure of ADHD symptoms in college students with a diagnostic history or who meet diagnostic criteria to evaluate whether the one-factor model with correlated residuals holds in a clinical sample.

Other methodological differences between the present investigation and previous factor analytic studies warrant discussion. The current study examined the structure of self-reported CSS symptoms of ADHD in a normative sample of college students. Although Park et al. (2018) also assessed symptoms using the CSS, the participant sample in that study was relatively smaller in size (n = 430) as compared with the current study (n = 892) and was composed of parents who were not only older than the college student population (M = 41.11 years of age), but whose day-to-day responsibilities were likely distinct from that of college students. Thus, these samples differed both in terms of developmental stage of the participants, and degree of symptom risk. College students and their same-aged noncollege peers represent a transitional period of development that has been described as emerging adulthood (Arnett, 2000). The roles, responsibilities, and social experiences that are characteristic of this period of development differentiate emerging adults from those who more fully and routinely engage in the types of independent, self-sufficient activities of daily living that are typical of later adulthood. Furthermore, the emerging adults in the current study were college students, a status which presents its own unique social, cognitive, and organizational demands that may require greater attentional and self-monitoring resources than is required in more routinized day-to-day activities (Arnett, 2016). The intersection of emerging adulthood as a transitional period of development and the unique demands of the college context, including increased academic, organizational, and social demands, may pose a particular challenge for students with significant symptoms of ADHD (Arnett, 2000, 2016). Indeed, over a quarter of students who register with university disability services do so because of a diagnosis of ADHD, suggesting that college-age individuals are highly affected by ongoing concerns related to ADHD symptomology (DuPaul et al., 2009).

Another critical consideration in the present investigation and the broader body of work examining the structure of ADHD symptoms in adult populations is that factor analytic results are highly dependent on the measure used to assess symptoms. Assessments based on DSM criteria presume that symptoms manifest similarly across development. However, there is evidence to suggest that symptom presentations differ in children and adults. Fewer symptoms of hyperactivity/impulsivity are generally endorsed from childhood into adulthood (Willcutt, 2012), suggesting that a two-factor model of ADHD symptoms may be vulnerable to developmental change and may not be an adequate representation of the adult phenotype (Hart et al., 1995; Larsson et al., 2011). Furthermore, although DSM criteria characterize ADHD as a disorder of childhood, recent population studies suggest that a sizeable proportion of adults report the onset of symptoms and impairment outside this period of development (Caye et al., 2016). Therefore, it is important to examine the patterning of symptom presentation in adults to characterize the ADHD phenotype beyond childhood. Additional research is needed to better understand the extent to which measures based on DSM-IV and DSM-5 criteria adequately capture the symptom dimensions of ADHD in college students and other adult populations. Future studies could address this need by further evaluating the ADHD experience in adults, with the goal of identifying alternatives to DSM symptom presentations and developing more age-appropriate items.

Limitations of the current study include issues of generalizability related to extrapolating from a college sample to older adults and same-age peers who are not attending college. As previously noted, the unique demands of the college context may increase the salience of ADHD symptomatology. Therefore, the structure of ADHD as observed in college students may not be the best representation in other adult samples. Furthermore, we did not gather data regarding the participants’ history of learning disability, which may co-occur with ADHD and may be associated with distinct symptom presentations. Additionally, there was an overrepresentation of female participants in the sample, which could limit generalizability; however, more women attend college than do men (U.S. Department of Education, National Center for Education Statistics, 2012). Another limitation was the reliance on self-report data. However, evidence suggests that young adults are reliable reporters of their own ADHD symptoms and related behaviors (e.g., substance use; Kooij et al., 2008; Levy et al., 2004). Nonetheless, is worth noting that common test and method invariance is a limitation in understanding the extent to which self-reported symptoms of ADHD, depression, impairment, and ODD reflect true associations.

Despite these limitations, our finding that a unidimensional factor of total ADHD symptoms with correlated residuals is the best representation of ADHD symptoms among college students has important implications for how the disorder is conceptualized across research and potentially in clinical settings. Specifically, this indicates that a general symptom count is the most useful and parsimonious way to capture the presentation of ADHD symptoms among nonclinical college students. Based on the findings of the current study, to get a well-fitting and interpretable measurement model for ADHD symptoms, empirical researchers could consider a one-factor model with correlated residuals (among the three motor activities-related items). However, when clinically assessing ADHD, particularly in nonclinical samples, or using ADHD symptoms as a predictor or outcome when not considering a measurement model, a single score of total ADHD symptoms can be calculated and interpreted. For future methodological studies investigating the psychometric properties of adult ADHD scales, closer attention should be paid to the proposed one-factor structure. Importantly, while the current study supports a general factor of ADHD symptoms for a nonclinical population, various levels of symptom severity may be better represented by different factor structures in a clinical setting. Future research should determine if this factor structure holds across other common measures of ADHD symptoms in college students/young adults besides the CSS, and further examine how the factor structure of measures such as the CSS differs in clinical versus nonclinical samples.

Supplemental Material

Supplementary_material – Supplemental material for The Factor Structure and Gender Invariance of ADHD Symptoms in College Students

Supplemental material, Supplementary_material for The Factor Structure and Gender Invariance of ADHD Symptoms in College Students by Kate Flory, Dexin Shi, E. Rebekah Siceloff, Alex M. Roberts, Rebeca Castellanos, Emily Neger, Stephen Taylor and Kari Benson in Assessment

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was partially supported by a grant from the University of South Carolina Magellan Scholar Program awarded to Kari Benson.

ORCID iDs

Kate Flory

Dexin Shi

Supplemental Material

Supplemental material for this article is available online.

Notes

References

American Psychiatric Association. (1968). Diagnostic and statistical manual of mental disorders (2nd ed.). Author.

American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). Author.

American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed. Rev.). Author.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Author.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., Text Rev.). Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Author. https://doi.org/10.1176/appi.books.9780890425596

Arias

V. B.

Ponce

F. P.

Núñez

D. E.

(2018). Bi-factor models of attention-deficit/hyperactivity disorder (ADHD): An evaluation of three necessary but underused psychometric indexes. Assessment, 25(7), 885-897. https://doi.org/10.1177/1073191116679260

Arnett

J. J.

(2000). Emerging adulthood: A theory of development from the late teens through the twenties. American Psychologist, 55(5), 469-480. https://doi.org/10.1037/0003-066X.55.5.469

Arnett

J. J.

(2016). College students as emerging adults: The developmental implications of the college context. Emerging Adulthood, 4(3), 219-222. https://doi.org/10.1177/2167696815587422

10.

Asherson

Ramos-Quiroga

J. A.

Young

(2018). Adult ADHD: Clinical presentation and assessment. In Banaschewski

Coghill

Zuddas

(Eds.), Oxford textbook of attention deficit hyperactivity disorder (pp. 178-189). Oxford University Press.

11.

Asparouhov

Muthén

(2006). Robust chi square difference testing with mean and variance adjusted test statistics (Mplus Web Notes No. 10). Muthén & Muthén. http://www.statmodel.com/download/webnotes/webnote10.pdf

12.

Asparouhov

Muthén

B. O.

(2010). Simple second order chi-square correction scaled chi-square statistics (Technical Appendix). Muthén & Muthén.

13.

Barkley

R. A.

(1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121(1), 65-94. https://doi.org/10.1037/0033-2909.121.1.65

14.

Barkley

R. A.

(2011). Barkley Adult ADHD Rating Scale–IV (BAARS-IV). Guilford Press.

15.

Barkley

R. A.

Fischer

Smallish

Fletcher

(2002). The persistence of attention-deficit/hyperactivity disorder into young adulthood as a function of reporting source and definition of disorder. Journal of Abnormal Psychology, 111(2), 279-289. https://doi.org/10.1037/0021-843X.111.2.279

16.

Barkley

R. A.

Murphy

K. R.

(2006). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (3rd ed.). Guilford Press.

17.

Batstra

Frances

(2012). DSM-5 further inflates attention deficit hyperactivity disorder. Journal of Nervous and Mental Disease, 200(6), 486-488. https://doi.org/10.1097/NMD.0b013e318257c4b6

18.

Biederman

Faraone

S. V.

Spencer

Wilens

Mick

Lapey

K. A.

(1994). Gender differences in a sample of adults with attention deficit hyperactivity disorder. Psychiatry Research, 53(1), 13-29. https://doi.org/10.1016/0165-1781(94)90092-2

19.

Bonifay

Lane

S. P.

Reise

S. P.

(2017). Three concerns with applying a bi-factor model as a structure of psychopathology. Clinical Psychological Science, 5(1), 184-186. https://doi.org/10.1177/2167702616657069

20.

Brown

T. A.

(2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Press.

21.

Caye

Swanson

Thapar

Sibley

Arseneault

Hechtman

Arnold

L. E.

Niclasen

Moffitt

Rohde

L. A.

(2016). Life span studies of ADHD: Conceptual challenges and predictors of persistence and outcome. Current Psychiatry Reports, 18(12), Article 111. https://doi.org/10.1007/s11920-016-0750-x

22.

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464-504. https://doi.org/10.1080/10705510701301834

23.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5

24.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

25.

DuPaul

G. J.

Schaughency

E. A.

Weyandt

L. L.

Tripp

Kiesner

Ota

Stanish

(2001). Self-report of ADHD symptoms in university students: Cross-gender and cross-national prevalence. Journal of Learning Disabilities, 34(4), 370-379. https://doi.org/10.1177/002221940103400412

26.

DuPaul

G. J.

Weyandt

L. L.

O’Dell

S. M.

Varejao

(2009). College students with ADHD: Current status and future directions. Journal of Attention Disorders, 13(3), 234-250. https://doi.org/10.1177/1087054709340650

27.

Eaton

W. W.

Muntaner

Smith

Tien

Ybarra

(2004). Center for Epidemiologic Studies Depression Scale: Review and revision (CESD and CESD-R). In Maruish

M. E.

(Ed.), The use of psychological testing for treatment planning and outcomes assessment (3rd ed., pp. 363-377). Lawrence Erlbaum. https://doi.org/10.1037/t29280-000

28.

Faraone

S. V.

Biederman

Mick

(2006). The age-dependent decline of attention deficit hyperactivity disorder: A meta-analysis of follow-up studies. Psychological Medicine, 36(2), 159-165. https://doi.org/10.1017/S003329170500471X

29.

Fayyad

De Graaf

Kessler

Alonso

Angermeyer

Demyttenaere

De Girolamo

Haro

J. M.

Karam

E. G.

Lara

Lépine

J. P.

Ormel

Posada-Villa

Zaslavsky

A. M.

Jin

(2007). Cross-national prevalence and correlates of adult attention-deficit hyperactivity disorder. British Journal of Psychiatry, 190(5), 402-409. https://doi.org/10.1192/bjp.bp.106.034389

30.

Fischer

Barkley

R. A.

(2007). The persistence of ADHD into adulthood: (Once again) it depends on whom you ask. ADHD Report, 15(4), 7-16. https://doi.org/10.1521/adhd.2007.15.4.7

31.

Flory

Milich

Lynam

D. R.

Leukefeld

Clayton

(2003). Relation between childhood disruptive behavior disorders and substance use and dependence symptoms in young adulthood: Individuals with symptoms of attention-deficit/hyperactivity disorder are uniquely at risk. Psychology of Addictive Behaviors, 17(2), 151-158. https://doi.org/10.1037/0893-164X.17.2.151

32.

Forero

C. G.

Maydeu-Olivares

Gallardo-Pujol

(2009). Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS estimation. Structural Equation Modeling, 16(4), 625-641. https://doi.org/10.1080/10705510903203573

33.

Gibbins

Toplak

M. E.

Flora

D. B.

Weiss

M. D.

Tannock

(2012). Evidence for a general factor model of ADHD in adults. Journal of Attention Disorders, 16(8), 635-644. https://doi.org/10.1177/1087054711416310

34.

Glutting

J. J.

Youngstrom

E. A.

Watkins

M. W.

(2005). ADHD and college students: Exploratory and confirmatory factor structures with student and parent data. Psychological Assessment, 17(1), 44-55. https://doi.org/10.1037/1040-3590.17.1.44

35.

Gomez

(2016). ADHD and hyperkinetic disorder symptoms in Australian adults: Descriptive scores, incidence rates, factor structure, and gender invariance. Journal of Attention Disorders, 20(4), 325-334. https://doi.org/10.1177/1087054713485206

36.

Gomez

Burns

G. L.

Walsh

J. A.

Hafetz

(2005). A multitrait–multisource confirmatory factor analytic approach to the construct validity of ADHD and ODD rating scales with Malaysian children. Journal of Abnormal Child Psychology, 33(2), 241-254. https://doi.org/10.1007/s10802-005-1831-1

37.

Hart

E. L.

Lahey

B. B.

Loeber

Applegate

Frick

P. J.

(1995). Developmental change in attention-deficit hyperactivity disorder in boys: A four-year longitudinal study. Journal of Abnormal Child Psychology, 23(6), 729-749. https://doi.org/10.1007/BF01447474

38.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118

39.

Kessler

R. C.

Adler

Barkley

Biederman

Conners

C. K.

Demler

Faraone

S. V.

Greenhill

L. L.

Howes

M. J.

Secnik

Spencer

Ustun

T. B.

Walters

E. E.

Zaslavsky

A. M.

(2006). The prevalence and correlates of adult ADHD in the United States: Results from the National Comorbidity Survey Replication. American Journal of Psychiatry, 163(4), 716-723. https://doi.org/10.1176/ajp.2006.163.4.716

40.

Knouse

L. E.

Safren

S. A.

(2010). Adult attention-deficit hyperactivity disorder. In Baer

Blais

M. A.

(Eds.), Handbook of clinical rating scales and assessment in psychiatry and mental health (pp. 195-208). Humana Press. https://doi.org/10.1007/978-1-59745-387-5_9

41.

Kooij

J. J. S.

Boonstra

A. M.

Swinkels

S. H. N.

Bekker

E. M.

de Noord

Buitellar

J. K.

(2008). Reliability, validity, and utility of instruments for self-report and informant report concerning symptoms of ADHD in adult patients. Journal of Attention Disorders, 11(4), 445-458. https://doi.org/10.1177/1087054707299367

42.

Lahey

B. B.

Applegate

McBurnett

Biederman

Greenhill

Hynd

G. W.

Barkley

R. A.

Newcorn

Jensen

Richters

Garfinkle

Kerdyk

Frick

P. J.

Ollendick

Perez

Hart

E. L.

Waidman

Shaffer

(1994). DMS-IV field trials for attention deficit hyperactivity disorder in children and adolescents. American Journal of Psychiatry, 151(11), 1673-1685. https://doi.org/10.1176/ajp.151.11.1673

43.

Larsson

Chang

D’Onofrio

B. M.

Lichtenstein

(2014). The heritability of clinically diagnosed attention deficit hyperactivity disorder across the lifespan. Psychological Medicine, 44(10), 2223-2229. https://doi.org/10.1017/S0033291713002493

44.

Larsson

Dilshad

Lichtenstein

Barker

E. D.

(2011). Developmental trajectories of DSM-IV symptoms of attention-deficit/hyperactivity disorder: Genetic effects, family risk and associated psychopathology. Journal of Child Psychology and Psychiatry, 52(9), 954-963. https://doi.org/10.1111/j.1469-7610.2011.02379.x

45.

LeRoy

Jacova

Young

(2018). Neuropsychological performance patterns of adult ADHD subtypes. Journal of Attention Disorders, 23(10), 1136-1147. https://doi.org/10.1177/1087054718773927

46.

Levy

Sherritt

Harris

S. K.

Gates

E. C.

Holder

D. W.

Kulig

J. W.

Knight

J. R.

(2004). Test-retest reliability of adolescents’ self-report of substance use. Alcoholism: Clinical & Experimental Research, 28(8), 1236-1241. https://doi.org/10.1097/01.ALC.0000134216.22162.A5

47.

Martel

M. M.

Von Eye

Nigg

(2012). Developmental differences in structure of attention-deficit/hyperactivity disorder (ADHD) between childhood and adulthood. International Journal of Behavioral Development, 36(4), 279-292. https://doi.org/10.1177/0165025412444077

48.

Maydeu-Olivares

Shi

(2017). Effect sizes of model misfit in structural equation models: Standardized residual covariances and residual correlations. Methodology, 13(Suppl. 1), 23-30. https://doi.org/10.1027/1614-2241/a000129

49.

McDonald

R. P.

(1999). Test theory: A unified approach. Lawrence Erlbaum.

50.

Millsap

R. E.

(2012). Statistical approaches to measurement invariance. Routledge. https://doi.org/10.4324/9780203821961

51.

Millsap

R. E.

Yun-Tein

(2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479-515. https://doi.org/10.1207/S15327906MBR3903_4

52.

Muthén

L. K.

Muthén

B. O.

(1998-2012). Mplus user’s guide (7th ed.). Muthén & Muthén.

53.

Murray

A. L.

Johnson

(2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence, 41(5), 407-422. https://doi.org/10.1016/j.intell.2013.06.004

54.

Nigg

J. T.

Willcutt

E. G.

Doyle

A. E.

Sonuga-Barke

E. J.

(2005). Causal heterogeneity in attention-deficit/hyperactivity disorder: Do we need neuropsychologically impaired subtypes? Biological Psychiatry, 57(11), 1224-1230. https://doi.org/10.1016/j.biopsych.2004.08.025

55.

Nussbaum

N. L.

(2012). ADHD and female specific concerns: A review of the literature and clinical implications. Journal of Attention Disorders, 16(2), 87-100. https://doi.org/10.1177/1087054711416909

56.

Park

J. L.

Silveira

Elliott

Savalei

Johnston

(2018). Confirmatory factor analysis of the structure of adult ADHD symptoms. Journal of Psychopathology and Behavioral Assessment, 40(4), 573-585. https://doi.org/10.1007/s10862-018-9698-y

57.

Proctor

B. E.

Prevatt

(2009). Confirming the factor structure of attention-deficit/hyperactivity disorder symptoms in college students using student and parent data. Journal of Learning Disabilities, 42(3), 250-259. https://doi.org/10.1177/0022219408331043

58.

Radloff

L. S.

(1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385-401. https://doi.org/10.1177/014662167700100306

59.

Reise

S. P.

(2012). The rediscovery of bi-factor measurement models. Multivariate Behavioral Research, 47(5), 667-696. https://doi.org/10.1080/00273171.2012.715555

60.

Reise

S. P.

Morizot

Hays

R. D.

(2007). The role of the bi-factor model in resolving dimensionality issues in health measurement. Quality of Life Research, 16(1), 19-31. https://doi.org/10.1007/s11136-007-9183-7

61.

Rodriguez

Reise

S. P.

Haviland

M. G.

(2016a). Applying bi-factor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98(3), 223-237. https://doi.org/10.1080/00223891.2015.1089249

62.

Rodriguez

Reise

S. P.

Haviland

M. G.

(2016b). Evaluating bi-factor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137-150. https://doi.org/10.1037/met0000045

63.

Rodriguez

P. D.

Simon-Dack

S. L.

(2013). Factor analysis of five adult ADHD self-report measures: Are they all the same? Journal of Attention Disorders, 17(1), 64-69. https://doi.org/10.1177/1087054711423627

64.

Samuel

D. B.

(2019). Incoming editorial. Assessment, 26(2), 147-150. https://doi.org/10.1177/1073191118822760

65.

Saris

W. E.

Satorra

Van der Veld

W. M.

(2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling: A Multidisciplinary Journal, 16(4), 561-582. https://doi.org/10.1080/10705510903203433

66.

Shi

DiStefano

McDaniel

H. L.

Jiang

(2018). Examining chi-square test statistics under conditions of large model size and ordinal data. Structural Equation Modeling: A Multidisciplinary Journal, 25(6), 924-945. https://doi.org/10.1080/10705511.2018.1449653

67.

Shi

Maydeu-Olivares

Distefano

(2018). The relationship between the standardized root mean square residual and model misspecification in factor analysis models. Multivariate Behavioral Research, 53(5), 676-694. https://doi.org/10.1080/00273171.2018.1476221

68.

Shi

Song

Lewis

(2017). The impact of partial factorial invariance on cross-group comparison. Assessment, 26(7), 1217-1233. https://doi.org/10.1177/1073191117711020

69.

Smith

E. V.

Jr. Johnson

B. D.

(1998). Factor structure of the DSM-IV criteria for college students using the Adult Behavior Checklist. Measurement and Evaluation in Counseling and Development, 31(3), 164-183. https://doi.org/10.1080/07481756.1998.12068963

70.

Sonuga-Barke

E. J.

Castellanos

(2005). A common core dysfunction in attention-deficit/hyperactivity disorder: A scientific red herring? Behavioral and Brain Sciences, 28(3), 443-444. https://doi.org/10.1017/S0140525X05390078

71.

Span

S. A.

Earleywine

Strybel

T. Z.

(2002). Confirming the factor structure of attention deficit hyperactivity disorder symptoms in adult, nonclinical samples. Journal of Psychopathology and Behavioral Assessment, 24(2), 129-136. https://doi.org/10.1023/A:1015396926356

72.

Stanton

Forbes

M. K.

Zimmerman

(2018). Distinct dimensions defining the Adult ADHD Self-Report Scale: Implications for assessing inattentive and hyperactive/impulsive symptoms. Psychological Assessment, 30(12), 1549-1559. https://doi.org/10.1037/pas0000604

73.

Taylor

Deb

Unwin

(2011). Scales for the identification of adults with attention deficit hyperactivity disorder (ADHD): A systematic review. Research in Developmental Disabilities, 32(3), 924-938. https://doi.org/10.1016/j.ridd.2010.12.036

74.

Upadhyaya

H. P.

Carpenter

M. J.

(2008). Is attention deficit hyperactivity disorder (ADHD) symptom severity associated with tobacco use? American Journal on Addictions, 17(3), 195-198. https://doi.org/10.1080/10550490802021937

75.

U.S. Department of Education, National Center for Education Statistics. (2012). Degrees conferred by degree-granting institutions, by level of degree and sex of student: Selected years, 1869-70 through 2021-22. https://nces.ed.gov/programs/digest/d12/tables/dt12_310.asp

76.

Van Dam

N. T.

Earleywine

. (2011). Validation of the Center for Epidemiologic Studies Depression Scale–Revised (CESD-R): Pragmatic depression assessment in the general population. Psychiatry Research, 186(1), 128-132. https://doi.org/10.1016/j.psychres.2010.08.018

77.

Van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486-492. https://doi.org/10.1080/17405629.2012.686740

78.

Whittaker

T. A.

(2012). Using the modification index and standardized expected parameter change for model modification. Journal of Experimental Education, 80(1), 26-44. https://doi.org/10.1080/00220973.2010.531299

79.

Willcutt

E. G.

(2012). The prevalence of DSM-IV attention-deficit/hyperactivity disorder: A meta-analytic review. Neurotherapeutics, 9(3), 490-499. https://doi.org/10.1007/s13311-012-0135-8

80.

Wolraich

M. L.

Lambert

E. W.

Baumgaertel

Garcia-Tornel

Feurer

I. D.

Bickman

Doffing

M. A.

(2003). Teachers’ screening for attention deficit/hyperactivity disorder: Comparing multinational samples on teacher ratings of ADHD. Journal of Abnormal Child Psychology, 31(4), 445-455. https://doi.org/10.1023/A:1023847719796

81.

Wood

W. L.

Lewandowski

L. J.

Lovett

B. J.

(2019). Profiles of diagnosed and undiagnosed college students meeting ADHD symptom criteria. Journal of Attention Disorders. Advance online publication. https://doi.org/10.1177/1087054718824991

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.10 MB