The Validity of the World Health Organization Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale for Diagnostic and Statistical Manual of Mental Disorders,Fifth Edition in Adolescence

Abstract

Objectives:

Short, self-report screening measures for adolescent and adult attention-deficit/hyperactivity disorder (ADHD) would greatly enhance the likelihood of ADHD subjects to be correctly diagnosed and treated. This study aimed at testing the reliability, factor structure, convergent validity, external validity, and diagnostic accuracy of the official Italian translation of the ADHD Self-Report Screening Scale for DSM-5 (ASRS-5) in a sample of community-dwelling adolescents, extending previous data on adult participants to adolescent participants.

Methods:

Five hundred sixty-four community-dwelling male adolescents (mean age ≅15) were administered the ASRS-5, the Adult ADHD Self-Report Scale 18-item and 6-item versions (ASRS-18 and ASRS-6), the Wender Utah Rating Scale (WURS), and the Structured Clinician Interview for DSM-5-Clinician Version ADHD Module (SCID-5-CV-ADHD). School performance variables were also collected.

Results:

The item response theory (IRT) reliability of ASRS-5 was adequate. Dimensionality analyses strongly supported the unidimensional structure of ASRS-5 items; confirmatory factor analysis fit indices supported the adequacy of the one-factor model of ASRS-5. In terms of convergent validity, the ASRS-5 total score was significantly and positively associated with self-report and interview-based ADHD dimensional scores, as well as with school performance variables. Roughly 8.0% of our male adolescents met SCID-5-CV-ADHD criteria for categorical ADHD diagnosis. Ten-fold cross-validated receiver operating curve value was 0.843; precision-recall curve analysis suggests that an ASRS-5 total score >12 may be preferred for screening purposes in adolescence.

Conclusions:

Our data showed that the ASRS-5 may represent a psychometrically sound self-report instrument to reliably screen for DSM-5 ADHD, extending the range of application of ASRS-5 from adulthood to adolescence, suggesting that the ASRS-5 could be safely used for screening purposes also in community-dwelling adolescents, at least in its official Italian translation.

Introduction

Attention-deficit/hyperactivity disorder (ADHD) is a childhood-onset neurodevelopmental disorder, defined by the presence of developmentally inappropriate and impairing levels of inattention, hyperactivity, and impulsivity (American Psychiatric Association 2013; Asherson and Agnew-Blais 2019). In childhood, ADHD is among the most common psychiatric disorders (Polanczyk et al. 2015), and it often persists into adulthood and old age. Sex differences in ADHD diagnosis are well documented (Kooji et al. 2019), with female-to-male ratios ranging between 1:5 and 1:9 (Rucklidge 2010). Indeed, ADHD has been recognized as a disorder affecting individuals across the lifespan (e.g., Faraone et al. 2006); about 80%–85% of preteens continue to experience symptoms into the adolescent years and 60% into adulthood (e.g., Barkley et al. 1990; Biederman et al. 1996). In the transition from childhood to adolescence, the burden associated with the disorder increases dramatically because adolescents are known to experience age-specific problems associated with ADHD (e.g., Brahmbhatt et al. 2016), and begin to experience impairments that characterize adults suffering from this disorder, such as substance use (Molina et al. 2007). Accordingly, the Diagnostic and Statistical Manual of Mental Disorders (5th edition; DSM-5) ADHD diagnostic criteria have been adapted by adding some examples describing how ADHD symptoms are expressed across the lifespan, and the symptom threshold required has been reduced to five symptoms instead of six for older adolescents and adults in either the inattention or hyperactive/impulsive domain.

Diagnosing ADHD requires a careful and systematic assessment of a lifetime history of symptoms and impairment (Kooji et al. 2019); nonetheless, screening for ADHD symptoms may be particularly important, and should be part of routine mental health assessment (Brahmbhatt et al. 2016). For instance, Murray et al. (2019) carried out a longitudinal study on ADHD symptoms from age 7 to age 15 in a community-ascertained cohort (n = 1571) and suggested that more attention may need to be focused on adolescence in terms of detection of ADHD symptoms. Indeed, their findings showed that adolescence is a potential period of vulnerability for symptom onset and escalation (Murray et al. 2019). Moreover, although perceptions of ADHD as a childhood disorder have been changing, many ADHD diagnostic indicators refer to childhood-specific behaviors and settings, and the inclusion of developmentally appropriate indicators across the lifespan will be beneficial for identifying and monitoring ADHD symptoms (Murray et al. 2019). From this perspective, although ADHD adolescent and adults are known to underreport their symptoms (Asherson et al. 2016), the availability of short, self-report screening measures would greatly enhance the likelihood of adolescent and adult ADHD subjects to be correctly diagnosed and treated.

Recently, Ustun et al. (2017) presented a new version of the widely used Adult ADHD Self-Report Screening Scale (ASRS; Kessler et al. 2005; Kessler et al. 2007) updated for DSM-5 criteria (ASRS-5). The ASRS-5 is a 6-item, easily scored self-report measure developed and tested in national household, managed care subscribers, and specialty clinical samples (Ustun et al. 2017); it has been shown to detect the vast majority of general population cases at a threshold that also has high sensitivity (91.4%), specificity (96.0%), and positive predictive value (PPV = 67.3%). Despite these positive findings, the psychometric properties of ASRS-5 as a screening tool have been tested exclusively among adult participants (Ustun et al. 2017). Of course, ADHD may (and should) be easily diagnosed in childhood; however, detecting ADHD in adolescence may be relevant to reduce the burden that is associated with adult ADHD, as well as to identify late-onset ADHD subjects (Murray et al. 2019; Asherson and Agnew-Blais 2019). Thus, extending Ustun et al.'s (2017) findings to community-dwelling adolescents may be relevant.

Notably, screening campaigns usually involve subjects living in the community. In light of what was known about Berkson's bias (Berkson 1946) in the epidemiology literature, it should be observed that clinic/hospital patients may be unrepresentative of the ADHD population (e.g., showing more severe ADHD impairment, higher rates of psychiatric/medical comorbidity, etc.); thus, community-based studies may be particularly helpful for screening and identifying ADHD in adolescence (e.g., Buitelaar 2017; Murray et al. 2019). Finally, Ustun et al. (2017) provided excellent data on the diagnostic accuracy of ASRS-5 with respect to interview-based adult ADHD diagnoses. Nevertheless, providing data on the ASRS-5 reliability and factor structure may represent a useful addition to the existing literature on the ASRS-5.

Starting from these considerations, we aimed at testing the internal consistency reliability, factor structure, convergent validity, external validity, and diagnostic accuracy of the official Italian translation of ASRS-5 in a sample of community-dwelling Italian adolescents. Indeed, researchers and clinicians who work adolescents would benefit from a short rating scale for assessing ADHD symptoms (e.g., Keeley et al. 2018). Male adolescents with ADHD showed increased levels of real-life risk-taking behavior like substance abuse and reckless behavior (Dekkers et al. 2020). Adolescent participants were attending public professional schools in the Northern Italy, and were personally interviewed with a validated semistructured interview for diagnosing DSM-5 ADHD, namely the Structured Clinician Interview for DSM-5-Clinician Version ADHD Module (SCID-5-CV-ADHD). Notably, 95.8% of the Italian general population adolescents are high school students (ISTAT 2017). In this study, we relied only on male adolescents who were attending schools, which does not require adolescents to be attentive and remain seated for a long time period. Consistent data documented that ADHD should be conceived as a dimension of psychopathology (Coghill and Sonuga-Barke 2012; Demontis et al. 2019); however, practitioners are used to consider categorical ADHD diagnoses for their clinical decision making. Thus, in this study, we computed both dimensional (i.e., number of symptoms met) and categorical (i.e., absence vs. presence) interview-based ADHD assessment.

Methods

Participants

The sample was composed of 564 community-dwelling male adolescents (mean [M] age = 15.5 and standard deviation [SD] = 1.6 years), who were receiving professional education at public professional schools in the North of Italy. After obtaining Institutional Review Board approval from the university and the principals of the schools, researchers recruited adolescents from classrooms (data were collected in Winter 2018–Spring 2019). Written informed parent consent and adolescent assent were obtained before study participation. To participate in this study, participants were required to speak Italian as their first language to avoid cultural and lexical bias in questionnaire responses; to maximize the likelihood of including adolescents potentially expressing the ADHD phenotype in our sample, only male adolescents were allowed to participate in this study. In our sample, 184 adolescents (32.6%) reported at least one school failure in the preceding years.

Measures

World Health Organization adult ASRS-5

The ASRS-5 was developed as a screening scale for adult ADHD with the same form as its DSM-IV counterpart: that is, a simple additive scale with a limited number of variables and integer scoring for quick calculation (Ustun et al. 2017). Using Risk-Calibrated Super-sparse Linear Integer Model (RiskSLIM) and 10-Fold Cross-Validated Receiver Operating Characteristic Curve (10-F ROC) analyses, Ustun et al. (2017) derived the six items that best discriminated adults with an interview-based DSM-5 ADHD diagnosis from adults who were not diagnosed as suffering from ADHD from an original pool of 29 items. The six items in the final scale included (1) DSM-5 symptom of inattention (Criterion A1c: does not listen when spoken to directly), (2) non–DSM-5 symptoms of executive dysfunction (puts things off to last minute, depends on others to keep life in order), and (3) DSM-5 symptoms of hyperactivity and impulsivity (CriterionA2b, leaves seat inappropriately; Criterion A2e, acts as if “driven by a motor”; and Criterion A2g, blurts out answers). The ASRS-5 may be scored either using a fixed 0–5 range for all items or relying on the RiskSLIM algorithm; for the latter, a 0 score is assigned to all “never” responses, but scores for higher responses vary, with the possible range of 0 to 24 (Ustun et al. 2017). Using the RiskSLIM scoring, Ustun et al. 2017 reported excellent diagnostic efficiency data for the ASRS-5 in two samples of adult subjects. After obtaining the permission to use the ASRS-5 proprietary scoring rules developed by the New York University and Harvard University, we relied on the RiskSLIM algorithm for assessing the diagnostic accuracy of ASRS-5 items.

Consistent with the translation procedure for the previous version of the scale, the ASRS-5 has been translated into Italian using the standard World Health Organization (WHO) translation and back-translation protocol (Somma et al. 2019). The reliability data of ASRS-5 in this sample are reported in the Results.

Adult ADHD Self-Report Scale 18-item and 6-item versions

The Adult ADHD Self-Report Screening Scale-18 item version (ASRS-18) includes 18 items, which were explicitly designed to assess ADHD in adults based on the DSM-IV criteria. Consistent with the other existing translations of ASRS-18, the scale has been translated into Italian using the standard WHO translation and back-translation protocol (National Comorbidity Survey 2005). The response options of the 18 items are arranged on a 5-point Likert scale. Kessler et al. (2007) showed that 5-point scores rather than dichotomous scores should be used in rating ASRS-18 items. The ASRS-18 items are summed to yield a total score; the higher the ASRS-18 score, the higher the likelihood of adult ADHD diagnosis. The first six items of the scale, which are thought to best predict the presence or absence of ADHD, compose the Adult ADHD Self-Report Screening Scale-6 item version (ASRS-6). Both the ASRS-18 and ASRS-6 were recently validated in Italian community-dwelling adolescents (Somma et al. 2019). In this study, the Cronbach's α values were 0.82 (average inter-item r = 0.21) and 0.54 (average inter-item r = 0.21) for the ASRS-18 and ASRS-6, respectively.

Wender Utah Rating Scale

The Wender Utah Rating Scale (WURS) is a self-report questionnaire designed to retrospectively assess the severity of ADHD symptoms during childhood (Ward et al. 1993). Adequate reliability and validity were reported for the WURS (Taylor et al. 2011). The Italian translation of the WURS showed adequate reliability and validity (Fossati et al. 2001). In this study, the Cronbach's α value of the WURS was 0.89 (average inter-item r = 0.24).

Structured Clinician Interview for DSM-5-Clinician Version ADHD Module

The SCID-5-CV-ADHD is a semistructured interview for assessing DSM-5 ADHD criteria. It provides at least one question for each DSM-5 ADHD criterion and impairment and exclusion criteria are explicitly tested. The assessment for ADHD begins with two screening questions that are designed to determine whether or not to proceed with the full assessment of the 18 ADHD items; then, questions concerning the nine inattention symptoms and the nine hyperactive/impulsive symptoms are asked (First et al. 2016). The psychometric properties of the Italian translation of the SCID-5-CV-ADHD in Italian adolescents have been recently reported (Somma et al. 2019).

In this study, the inter-rater reliability of both SCID-5-CV-ADHD dimensional and categorial assessment was assessed using a pairwise interview design in the first 35 adolescents. Four raters were included in the inter-rater reliability assessment; raters were randomly paired, and each rater acted roughly the same number of times as interviewer and as independent observer. The SCID-5-CV-ADHD interview was independently scored by the interviewer and the independent observer. In our study, the intraclass r coefficient for absolute agreement based on one-way random-effect ANOVA for the SCID-5-CV-ADHD dimensional score (i.e., total number of symptoms) was 0.94, p < 0.001. Rather, in our sample, the Cohen's κ value for the categorial SCID-5-CV ADHD diagnosis was 0.77, p < 0.001.

School performance variables

The current year school grade averaged across literature subjects (i.e., literature, history, etc.), the current year school grade averaged across scientific subjects (i.e., mathematics, biology, chemistry, etc.), current year behavior grade, and current year number of disciplinary notes were used as adolescent's school performance variables. Schools in Italy use a 10-point scale that can be divided into failing (0 to 5) and passing (6 to 10) grades. A behavior grade lower than 8 indicates severe problem behavior at school and in the case of 6 grade, or even 7 grade in behavior, failure may occur. School performance variables were evaluated directly by the adolescents' teachers.

Procedures

All participants were assessed anonymously by trained clinical psychologists during school time; an alphanumeric code was used to allow matching adolescent's, graded with his/her corresponding test scores and school performance variable values. All measures were administered individually in random order. The SCID-5-CV-ADHD was administered blind to self-reports and school performance variable scores by clinical psychologist who were extensively trained (i.e., 40 hours or more) in the SCID-5-CV-ADHD assessment. In this study, time considerations allowed for testing only SCID-5-CV ADHD module. All participants were personally interview at school during school time at lesson break; consistent with previous studies (Kumar et al. 2011), our adolescents were administered only the SCID-5-CV ADHD module to obtain DSM-5 dimensional (i.e., total number of symptoms) and categorical diagnoses of ADHD.

Data analyses

Considering that Cronbach's α may yield suboptimal results in the case of short scales like the ASRS-5, in this study, we relied on item response theory (IRT) to obtain an estimate of ASRS-5 internal consistency reliability. The convergent validity of ASRS-5 total score was assessed by computing the Pearson r coefficients with both self-report and interview measures of ADHD. To provide further evidence of ASRS-5 validity, we computed Spearman r coefficients between the ASRS-5 total score and the school performance variable ordinal/non-normal scores. Quasi-inferential parallel analysis (PA; Buja and Eyuboglu 1992) was used to assess the dimensionality of ASRS-5 item polychoric correlation matrix (Zwick and Velicer 1986).

Since the factor analysis of ordinal data is still a controversial topic (Wirth and Edwards 2007), in this study, we relied both on weighted least square mean and variance adjusted (WLSMV) confirmatory factor analysis (CFA), and on IRT CFA based on graded response model (Samejima 1969). Based on the unidimensional nature of ASRS-5 and on the PA results, in this study we tested only a one-factor model of ASRS-5 item polychoric correlation matrix. In WLSMV CFA, we used several measures to identify model fit, including the χ² goodness-of-fit statistic, the root mean square error of approximation (RMSEA), the Tucker–Lewis index (TLI), the comparative fit index (CFI), and the standardized root mean square residual (SRMSR). Following Hu and Bentler's (1999) (Maydeu-Olivares 2013) suggestions, TLI and CFI values ≥0.95, RMSEA values close to 0.06, and SRSMR <0.08 were considered indicating good model fit, whereas TLI and CFI values of 0.90 and higher, and an RMSEA of 0.08 and lower are indications of an adequate fit. M₂* statistic and its associated RMSEA₂, and SRMSR were used to assess model fit in IRT CFA (Maydeu-Olivares 2013).

In this study, Student's t-test was computed to compare the mean ASRS total score that was computed according to the optimal RiskSLIM response scoring (Coghill and Sonuga-Barke 2012) between ADHD adolescents and non-ADHD adolescents; Cohen's d was used as an effect size measure.

In this study, the area under the curve (AUC; i.e., the probability that a randomly selected clinical case would score higher on the ASRS-5 than a randomly selected noncase; Kessler et al. 2005) was used as a measure of the overall ability of ASRS-5 total score computed according to the RiskSLIM algorithm to predict the presence or absence of ADHD diagnosis. Specifically, we relied on AUC based on 10-F ROC analysis (LeDell et al. 2015); the AUC of a perfect measure is expected to be 1.0, whereas it is expected to be 0.5 for a useless instrument. Precision-recall curve analysis (i.e., a curve showing the relationship between precision and recall for every possible cutoff score) was used to identify the ASRS-5 total score cutoff value with respect to SCID-5-CV ADHD diagnosis. For all possible cutoff scores provided by precision-recall curve analysis, ASRS-5 cutoff scores providing the minimum positive predictive value (PPV) to justify more in-depth assessment of ADHD and adequate specificity were selected for detection purposes. In this study, sensitivity (i.e., recall), specificity, positive predictive value (PPV; i.e., precision), negative predictive value (NPV), and likelihood ratio values were computed (e.g., Grimes and Schulz 2005). Finally, decision curve analysis (Vickers and Elkin 2006) was used to identify the proportion of adolescents meeting criteria for DSM-5 ADHD, which could be cost effectively treated relying on the ASRS-5 as the basis for selection based on various cost-benefit ratios (which were expressed as net benefits).

Results

The descriptive statistics, and results of the WLSMV CFA one-factor model and IRT CFA unidimensional model of ASRS items, as well as the quasi-inferential PA results, are summarized in Table 1. Based on IRT analyses of ASRS-5 items, the total amount of information in the ASRS-5 was 12.21, whereas the information in the in (−3, 3) range was 7.06 (i.e., 57.8% of the total information). This yielded an internal consistency reliability estimate of 0.714 (i.e., ) for the ASRS-5 total score, whereas the average inter-item polychoric r value for the ASRS-5 items was 0.18 (median polychoric r = 0.19, SD = 0.07). The ASRS-5 test information curve is displayed in Figure 1.

FIG. 1.

Test information function of the adult ASRS-5. ADHD, attention-deficit/hyperactivity disorder; ASRS-5, ADHD Self-Report Screening Scale for DSM-5.

Table 1.

Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale for DSM-5 Items: Descriptive Statistics, Dimensionality Analysis (i.e., Quasi-Inferential Parallel Analysis Based on 1000 Random Permutations of the Original Data), Weighted Least Square Mean and Variance Adjusted Confirmatory Factor Analysis, and Item Response Theory Confirmatory Factor Analysis Results (n = 564)

			Parallel analysis		WLSMV CFA	IRT factor analysis parameters
ASRS-5 items	M	SD	Eigen.	95 p	λ	a (SE)	d1 (SE)	d2 (SE)	d3 (SE)	d4 (SE)
(1) How often do you have difficulty concentrating…	1.23	0.93	1.93	1.23	0.47^***	0.95 (0.15)	1.62 (0.13)	−0.89 (0.11)	−2.59 (0.18)	−4.21 (0.32)
(2) How often do you leave your seat…	0.73	1.01	1.03	1.28	0.55^***	1.11 (0.18)	−0.23 (0.11)	−1.97 (0.16)	−2.75 (0.21)	−4.53 (0.37)
(3) How often do you have difficulty unwinding…	1.32	1.27	0.84	1.06	0.29^***	0.52 (0.12)	0.72 (0.09	−0.54 (0.09)	−1.40 (0.11)	−2.60 (0.17)
(4) When you're in a conversation, how often…	1.33	1.02	0.80	1.01	0.40^***	0.77 (0.13)	1.46 (0.12)	−0.55 (0.10)	−2.02 (0.14)	−3.82 (0.27)
(5) How often do you put things…	2.25	1.21	0.76	0.96	0.36^***	0.68 (0.13)	2.57 (0.17)	0.93 (0.10)	−0.13 (0.09)	−1.73 (0.13)
(6) How often do you depend on others…	1.26	1.05	0.64	0.89	0.50^***	1.01 (0.16)	1.30 (0.13)	−0.68 (0.11)	−2.36 (0.17)	−3.60 (0.26)
	WLSMV CFA goodness-of-fit statistics
	χ²		df	RMSEA	90% CI for RMSEA		TLI	CFI	SRMSR
One-factor model	20.64^*		9	0.048	0.020	0.075	0.92	0.95	0.026
	IRT factor analysis goodness-of-fit statistics
	M2			df	RMSEA		90% CI for RMSEA		SRMSR
Unidimensional model	7.65			9	0.000		0.000	0.045	0.045

Eigen., real data eigenvalues; 95 p, 95th percentile of random eigenvalues; λ, standardized factor loadings; a, Item discrimination parameter; d1–d4, item threshold parameters; bold highlights the correct number of factors to be extracted according to quasi-inferential parallel analysis results; χ² , goodness-of-fit chi-square test.

90% CI, 90% confidence interval; ASRS-5, Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale for DSM-5; CFI, comparative fit index; df, degrees of freedom; IRT, item response theory; M, mean; RMSEA, root mean square error of approximation; SD, standard deviation; SE, standard error; SRMSR, standardized root mean square residual; TLI, Tucker-Lewis index; WLSMV CFA, weighted least square mean and variance adjusted confirmatory factor analysis.

p < 0.05; ^*** p < 0.001.

The correlations (i.e., Pearson r values) between the ASRS-5 total score and the ASRS-6, ASRS-18, and SCID-5-CV-ADHD total number of symptoms are listed in Table 2, as well as the Spearman r values for the associations between the ASRS-5 total score and the adolescent's school performance variables.

Table 2.

Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale for DSM-5 Total Score: Pearson r Coefficients for the Associations with the Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale-6 Item Total Score, Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale-18 Item Total Score, Wender Utah Rating Scale Total Score, and Structured Clinical Interview for DSM-5-Clinician Version Adult Attention-Deficit/Hyperactivity Disorder Total Number of Symptoms, and Spearman r Coefficients for the Associations with the Adolescents' School Performance Indicators (n = 564)

ADHD measures	ASRS-5 Total score
ADHD measures	r	M	SD
ASRS-6 total score	0.58^***	8.94	3.82
ASRS-18 total score	0.68^***	26.80	9.79
WURS total score	0.62^***	29.40	15.09
SCID-5-CV-ADHD total number of symptoms	0.51^***	7.61	3.90
School performance variables
Literature subject average grade (current year)	−0.19^***	6.81	0.96
Scientific subject average grade (current year)	−0.22^***	6.79	1.19
Behavior grade (current year)	−0.26^***	7.76	1.28
Number of disciplinary notes (current year)	0.26^***	2.50	7.09
Number of previous school failures	0.13^*	0.63	0.78
M	8.12
SD	3.56

ADHD, attention-deficit/hyperactivity disorder; ASRS-5, Adult ADHD Self-Report Screening Scale for DSM-5; ASRS-6, Adult ADHD Self-Report Screening Scale-6 item version; ASRS-18, Adult ADHD Self-Report Screening Scale-18 item version; M, mean; SCID-5-CV-ADHD: Structured Clinical Interview for DSM-5-Clinician Version Adult ADHD Module; SD, standard deviation; WURS, Wender Utah Rating Scale.

p < 0.05; ^*** p < 0.001.

Based on SCID-5-CV-ADHD assessment, 46 adolescents (8.2%) received a DSM-5 ADHD diagnosis. When we scored the ASRS-5 using the optimal RiskSLIM response scoring (Coghill and Sonuga-Barke 2012), participants who received a SCID-5-CV ADHD diagnosis (n = 46; M = 17.00, SD = 3.32) scored significantly higher than participants who did not receive a SCID-5-CV ADHD diagnosis (n = 518; M = 11.50, SD = 4.22), t(562) = 8.61, p < 0.001, d = 1.32. In ROC analyses, the AUC was 0.843, 95% confidence interval = 0.744–0.941. Based on the optimal scoring of ASRS-5, precision-recall curve analysis suggested that when the cutoff score was set at 14, sensitivity was 0.75, with a PPV of 0.16, and specificity value was 0.65, whereas NPV value was 0.97. Finally, LR+ and LR− values were 2.14 and 0.38, respectively. Setting the threshold at 16 resulted in a worse PPV (0.28) with the same sensitivity (75.0%), whereas considering an ASRS-5 threshold of 12 led to a PPV value of 0.13, with an increase in sensitivity (0.94); specificity value was 0.45, whereas NPV value was 0.99. Finally, LR+ and LR− values were 1.71 and 0.14, respectively. These results seemed to suggest that the 12 or higher operating threshold would most likely be preferred for screening purposes.

The decision curve plotting net benefit against threshold probability is depicted in Figure 2. As it can be observed in Figure 2, we showed the decision curve for a range of threshold probability ranging from 0 through 0.50.

FIG. 2.

Decision curve plotting net benefit against threshold probability for the adult ASRS-5 optimal RiskSLIM response scoring. ADHD, attention-deficit/hyperactivity disorder; ASRS-5, ADHD Self-Report Screening Scale for DSM-5; RiskSLIM, Risk-Calibrated Super-sparse Linear Integer Model.

Discussion

Extending Ustun et al.'s (2017) findings, our data seemed to show that ASRS-5 is likely to represent a psychometrically sound self-report instrument to reliably screen for DSM-5 ADHD in community-dwelling adolescents, at least in its Italian translation. The relevance of this cross-cultural finding should not be underscored, since WHO measures are intended to be used worldwide.

According to our findings, the limited length (i.e., six items) of ASRS-5 did not prevent its total score to yield a reliable self-report assessment of the ADHD dimension. Dimensionality analyses strongly supported the unidimensional structure of ASRS-5 items; indeed, in our study, both WLSMV and IRT CFA fit indices supported the adequacy of the one-factor model of ASRS-5. Although previous symptom-level exploratory factor analysis studies (e.g., Adler et al. 2017) showed that adult ADHD symptoms may be organized across four factors representing executive dysfunction/inattention, hyperactivity, impulsivity, and emotional dyscontrol, it should be observed that the unidimensional factor structure of ASRS-5 was expected and desired. Indeed, the factor analysis of ASRS-5 items was not intended to assess the factor structure of ADHD symptoms per se. Rather, its purpose was to empirically test the hypothesis that six indicators (i.e., items) could be safely used to identify a latent ADHD dimension for screening purposes. This result supported previous indications (Ustun et al. 2017), suggesting that the ASRS-5 represented an improved version of the original ASRS (see also Somma et al. 2019). Our WLSMV CFA findings showed that all ASRS-5 item standardized factor loadings were positive, significant, and of non-negligible size; IRT CFA results showed that the ASRS-5 item 1, item 2, and item 6 showed discrimination parameter values that suggested strong relation with the latent dimension (Samejima 1969), although the remaining ASRS-5 items showed significant and moderate discrimination parameter estimates.

In terms of convergent validity, in our study, the ASRS-5 total score showed r coefficient values with both self-report and interview-based ADHD dimensional scores, which were all positive, significant, and suggesting a large effect size by conventional standards (Cohen 1988). Interestingly, the ASRS-5 total score showed positive, significant, although modest rank-order correlations with school performance variables. Notwithstanding the relatively small size of the correlation coefficients, the relevance of this finding should not be underscored, because it seemed to suggest that the adolescent's ASRS-5 self-reports were significantly associated with teacher's ratings of school performance, particularly with the current year behavior grade and number of disciplinary notes. It should be observed that although ADHD is usually associated with poor school performance, poor school performance does not necessarily require the presence of ADHD to occur (Daley and Birchwood 2010).

Relatively, in line with previous SCID-5-CV-ADHD-based estimates of the ADHD base rate among Italian adolescents (Somma et al. 2019), in this study, roughly 8.0% of our male adolescent sample met SCID-5-CV-ADHD criteria for categorical ADHD diagnosis. As a whole, our SCID-5-CV-ADHD data suggested that ADHD is not likely to represent a rare condition in male adolescent students attending professional schools, and it may be reliably assessed using a semistructured interview based on the DSM-5 criteria. Consistent with previous data (Ustun et al. 2017), in our study, adolescents who received a DSM-5 ADHD categorical diagnosis based on the SCID-5-CV-ADHD interview scored significantly and substantially larger on the ASRS-5 than adolescents who did not receive a DSM-5 ADHD categorical diagnosis. It should be observed that a Cohen's d value of 1.32 suggests that 90.7% of our ADHD adolescent group will be above the mean of our non-ADHD adolescent group, and there is a 82.5% chance that a male adolescent picked at random from the ADHD group has a higher score on the ASRS-5 than a male adolescent picked at random from the non-ADHD group.

In our sample, 10-fold cross-validated ROC analysis results showed that an adolescent picked up at random from the ADHD group had 84.3% the probability of scoring higher on the ASRS-5 than an adolescent picked up at random from the non-ADHD group (i.e., AUC = 0.843). Using the ASRS-5 optimal RiskSLIM response scoring to compute the scale total score, precision-recall curve suggested that an ASRS-5 total score >12 yielded an acceptable PPV (i.e., 0.13) with a sensitivity of 0.94, which may be considered adequate for screening purposes. In our sample of community-dwelling male adolescents who were attending professional schools, the probability that an adolescent did not have ADHD, given a negative test result (NPV) was 0.99, suggesting a negative ASRS-5 result (i.e., an ASRS-5 total score of 16 or lower), may be effective in actually excluding ADHD in male adolescents, at least when ADHD was assessed using the SCID-5-CV-ADHD. However, it should be observed that adopting the consistently validated ASRS-5 > 14 cut score for adults (Ustun et al. 2017) in our adolescent sample yielded values of 75.0% for sensitivity and 0.16 for PPV. In other terms, we feel that both cutoff scores may work adequately for identifying adolescents who received a SCID-5-CV ADHD diagnosis, although the ASRS-5 > 12 cut score seemed to work better than the ASRS-5 > 14 cut score in identifying subjects who received an ADHD diagnosis among community-dwelling male adolescents. Although we have no definitive explanations for this difference in the ASRS-5 cut score values between adult and adolescent subjects, we feel that differences in ADHD base rate in our community-dwelling sample and in the US ASRS-5 standardization adult samples (Ustun et al. 2017) may have influenced our findings.

Although, in this study, we do not know what the net benefit of any given intervention would be, we think that the results of decision curve analysis provided useful preliminary information. The ASRS-5 added value with threshold probabilities ranging from 0.00 to 0.40. In our study, the net benefit curve showed that at the 10% risk threshold, the net benefit was 0.02, which is equivalent to detecting two ADHD adolescents and suggesting zero unnecessary treatment (i.e., three true positives and zero false positives) per 100 patients. Of course, future studies focused on identifying risk models that can help making better clinical decisions are needed.

As a whole, extending previous findings (Ustun et al. 2017), our data showed that the ASRS-5 may be used as a screening instrument also among community-dwelling adolescents. Although ASRS-5 predictive values were somewhat lower than in the original validation study (Ustun et al. 2017), it should be observed that our data were collected among younger participants (i.e., male adolescents) who were not asking for treatment at the time the study was carried out. Thus, our results seemed to suggest that ASRS-5 may be helpful to detect ADHD in adolescence.

Limitations

Of course, our finding should be considered in light of several limitations. Our sample was composed of male adolescents who volunteered to participate in the study, rather than of adolescents who were randomly selected from the Italian adolescent population. Thus, our sample was more akin to a convenience study group than to a random sample representative of the adolescents in the Italian general population. Clinical assessment of ADHD requires careful consideration of multiple sources of information and indicators. Although we tried to increase the validity of our SCID-5-CVADHD diagnoses by testing their associations with objective indicators of school performance, our ADHD diagnoses were far from clinically sound ADHD assessment. Although the SCID-5-CV-ADHD was validated in Italian community-dwelling adolescents (Somma et al. 2018), other interviews exist to diagnose ADHD in adolescents, and their use might lead to different findings. Diagnostic agreement statistics may be influenced by the disorder base rate estimates; thus, the ASRS-5 may yield different diagnostic accuracy estimates in samples that are characterized by different ADHD base rates. These considerations inherently limit the generalizability of our finding and stress the need for further studies before accepting our conclusions.

It should be observed that, in our sample of male adolescent participants, a cutoff score of 16 on the ASRS-5 seemed to be appropriate for adolescents, whereas a cutoff score of 14 has been validated for adult participants (Ustun et al. 2017). Of course, future studies on the use of ASRS-5 in adolescents across different languages (e.g., English) and cultures are badly needed before accepting our findings. Indeed, the differing cutoff score might be due to age (i.e., adolescence vs. adult participants), sample (pure community-dwelling adolescence), language (Italian vs. English), or potentially cultural differences (Italian vs. US).

Conclusions

Even keeping these limitations in mind, our findings suggested that the ASRS-5 could be safely used in adolescent participants, with scores higher than 16 suggesting the need for accurate assessment of ADHD. These findings may be relevant in extending the range of application of ASRS-5 from adulthood to adolescence, suggesting that the ASRS-5 could be safely used for screening purposes also in community-dwelling adolescents.

Clinical Significance

ADHD is a well-studied condition in children; however, far less is known about ADHD in adolescence. The generalization of results from research using children or adult samples to adolescent samples may be inherently problematic because there are many differences between ADHD in children and adolescents, and adolescents and adults (e.g., Nichols et al. 2017). From this perspective, the availability of data on the reliability and construct validity of ASRS-5 in a sample of adolescents is crucial to make this self-report instrument available for clinicians and researchers working with adolescents. Indeed, the persistence of ADHD over adolescence into adulthood is not matched by continuity of care in this important phase of life (Buitelaar 2017). Identifying adolescents who need a throughout assessment of ADHD characteristics through the administration of an easy-to-use short self-report screening measure for ADHD may represent the first step in preventing the negative sequelae of ADHD (e.g., Pollak et al. 2016), particularly when undiagnosed in childhood.

Footnotes

Disclosures

A.S., G.G., M.A., E.C., and A.F. have nothing to disclose. In the past 3 years, L.A.A. has received grant/research support from Sunovion Pharmaceuticals, Enzymotec, Shire Pharmaceuticals, Otsuka and Lundbeck; has served as a consultant to Bracket, Sunovion Pharmaceuticals, Shire Pharmaceuticals, Otsuka Pharmaceuticals, SUNY, the National Football League, and Major League Baseball; and has received royalty payments (as inventor) since 2004 from NYU for license of adult ADHD scales and training materials. He has no conflicts in regard to stock ownership or speakers bureaus.

References

Adler

, Faraone

, Spencer

, Berglund

, Alperin

, Kessler

: The structure of adult ADHD. Int J Methods Psychiatr Res, 26:e1555, 2017.

American Psychiatric Association: Diagnostic and Statistical Manual of Mental

Disorders

, 5th ed. Washington, DC: American Psychiatric Association; 2013.

Asherson

, Agnew-Blais

: Annual Research Review: Does late-onset attention-deficit/hyperactivity disorder exist?. J Child Psychol Psychiatry, 60:333–352, 2019.

Asherson

, Buitelaar

, Faraone

, Rohde

: Adult attention-deficit hyperactivity disorder: Key conceptual issues. Lancet Psychiatry, 3:568–578, 2016.

Barkley

, Fischer

, Edelbrock

, Smallish

: The adolescent outcome of hyperactive children diagnosed by research criteria. I: An 8-year prospective follow-up study. J Am Acad Child Adolesc Psychiatry, 29:546–557, 1990.

Berkson

: Limitations of the application of fourfold table analysis to hospital data. Biometrics, 2:339–343, 1946.

Biederman

, Faraone

, Milberger

, Curtis

, Chen

, Marrs

, Ouellette

, Moore

, Spencer

: Predictors of persistence and remission of ADHD into adolescence: Results from a four-year prospective follow-up study. J Am Acad Child Adolesc Psychiatry, 35:343–351, 1996.

Brahmbhatt

, Hilty

, Hah

, Han

, Angkustsiri

, Schweitzer

: Diagnosis and treatment of attention deficit hyperactivity disorder during adolescence in the primary care setting: A concise review. J Adolesc Health, 59:135–143, 2016.

Buitelaar

: Optimising treatment strategies for ADHD in adolescence to minimise “lost in transition”. to adulthood. Epidemiol Psychiatr Sci, 26:448–452, 2017.

10.

Buja

, Eyuboglu

: Remarks on parallel analysis. Multivariate Behav Res, 27:509–540, 1992.

11.

Coghill

, Sonuga-Barke

: Annual Research Review: Categories versus dimensions in the classification and conceptualisation of child and adolescent mental disorders—implications of recent empirical study. J Child Psychol Psychiatry, 53:469–489, 2012.

12.

Cohen J (eds): Statistical Power Analysis for the Behavioral Sciences. Hillsdale, (New Jersey), L. Erlbaum Associates, 1988.

13.

Daley

, Birchwood

: ADHD and academic performance: Why does ADHD impact on academic performance and what can be done to support ADHD children in the classroom?. Child Care Health Dev, 36:455–464, 2010.

14.

Dekkers

, Huizenga

, Popma

, Bexkens

, Zadelaar

, Jansen

: Decision-making deficits in adolescent boys with and without attention-deficit/hyperactivity disorder (ADHD): An experimental assessment of associated mechanisms. J Abnorm Child Psychol, 48:495–510, 2020.

15.

Demontis

, Walters

, Martin

, Mattheisen

, Als

, Agerbo

, Baldursson

, Belliveau

, Bybjerg-Grauholm

, Bækvad-Hansen

, Cerrato

, Chambert

, Churchhouse

, Dumont

, Eriksson

, Gandal

, Goldstein

, Grasby

, Grove

, Gudmundsson

, Hansen

, Hauberg

, Hollegaard

, Howrigan

, Huang

, Maller

, Martin

, Moran

, Pallesen

, Palmer

, Pedersen

, Poterba

, Poulsen

, Ripke

, Robinson

, Satterstrom

, Stefansson

, Stevens

, Turley

, Walters

, Won

, Wright

; ADHD Working Group of the Psychiatric Genomics Consortium (PGC); Early Lifecourse & Genetic Epidemiology (EAGLE) Consortium; 23 and Me Research Team, Andreassen

, Asherson

, Burton

, Boomsma

, Cormand

, Dalsgaard

, Franke

, Gelernter

, Geschwind

, Hakonarson

, Haavik

, Kranzler

, Kuntsi

, Langley

, Lesch

, Middeldorp

, Reif

, Rohde

, Roussos

, Schachar

, Sklar

, Sonuga-Barke

EJS

, Sullivan

, Thapar

, Tung

, Waldman

, Medland

, Stefansson

, Nordentoft

, Hougaard

, Werge

, Mors

, Mortensen

, Daly

, Faraone

, Børglum

, Neale

: Discovery of the first genome-wide significant risk loci for attention deficit/hyperactivity disorder. Nat Genet, 51:63–75, 2019.

16.

Faraone

, Biederman

, Mick

: The age-dependent decline of attention deficit hyperactivity disorder: A meta-analysis of follow-up studies. Psychol Med (Paris), 36:159–165, 2006.

17.

First

, Williams

, Kang

, Spitzer

: User's Guide for the Structured Clinical Interview for DSM-5 Disorders-Clinician Version (SCID-5-CV). Arlington, VA: American Psychiatric Association; 2016.

18.

Fossati

, Di Ceglie

, Acquarini

, Donati

, Donini

, Novella

, Maffei

: The retrospective assessment of childhood attention deficit hyperactivity disorder in adults: Reliability and validity of the Italian version of the Wender Utah Rating Scale. Compr Psychiatry, 42:326–336, 2001.

19.

Grimes

, Schulz

: Refining clinical diagnosis with likelihood ratios. Lancet, 365:1500–1505, 2005.

20.

, Bentler

: Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling, 6:1–55, 1999.

21.

ISTAT: Studenti e scuole dell'istruzione primaria e secondaria in Italia [Primary and Secondary Schools and Students in Italy]. Roma, Istituto Nazionale di Statistica, 2017.

22.

Keeley

, Makol

, Qasmieh

, Deros

, Karp

, Lipton

, Augenstein

, Truong

, Racz

, De Los Reyes

: Validity of adolescent and parent reports on the six-item ADHD Self-Report Scale (ASRS-6) in clinical assessments of adolescent social anxiety. J Child Fam Stud, 27:1041–1053, 2018.

23.

Kessler

, Adler

, Ames

, Demler

, Faraone

, Hiripi

, Howes

, Jin

, Secnik

, Spencer

, Ustun

, Walters

: The World Health Organization Adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychol Med, 35:245–256, 2005.

24.

Kessler

, Adler

, Gruber

, Sarawate

, Spencer

, Van Brunt

: Validity of the World Health Organization Adult ADHD Self-Report Scale (ASRS) screener in a representative sample of health plan members. Int J Methods Psychiatr Res, 16:52–65, 2007.

25.

Kooij

JJS

, Bijlenga

, Salerno

, Jaeschke

, Bitter

, Balázs

, Thome

, Dom

, Kasper

, Nunes Filipe

, Stes

, Mohr

, Leppämäki

, Casas

, Bobes

, Mccarthy

, Richarte

, Kjems Philipsen

, Pehlivanidis

, Niemela

, Styr

, Semerci

, Bolea-Alamanac

, Edvinsson

, Baeyens

, Wynchank

, Sobanski

, Philipsen

, McNicholas

, Caci

, Mihailescu

, Manor

, Dobrescu

, Saito

, Krause

, Fayyad

, Ramos-Quiroga

, Foeken

, Rad

, Adamou

, Ohlmeier

, Fitzgerald

, Gill

, Lensing

, Motavalli Mukaddes

, Brudkiewicz

, Gustafsson

, Tani

, Oswald

, Carpentier

, De Rossi

, Delorme

, Markovska Simoska

, Pallanti

, Young

, Bejerot

, Lehtonen

, Kustow

, Müller-Sedgwick

, Hirvikoski

, Pironti

, Ginsberg

, Félegyházy

, Garcia-Portilla

, Asherson

: Updated European Consensus Statement on diagnosis and treatment of adult ADHD. Eur Psychiatry, 56:14–34, 2019.

26.

Kumar

, Faden

, Steer

: Screening for attention-deficit/hyperactivity disorder in adult inpatients with psychiatric disorders. Psychol Rep, 108:815–824, 2011.

27.

LeDell

, Petersen

, van der Laan

: Computationally efficient confidence intervals for cross-validated area under the ROC curve estimates. Electron J Stat, 9:1583–1607, 2015.

28.

Maydeu-Olivares

: Goodness-of-fit assessment of item response theory models. Measurement, 11:71–101, 2013.

29.

Molina

, Pelham

, Gnagy

, Thompson

, Marshal

: Attention-deficit/hyperactivity disorder risk for heavy drinking and alcohol use disorder is age specific. Alcohol Clin Exp Res, 31:643–654, 2007.

30.

Murray

, Booth

, Eisner

, Auyeung

, Murray

, Ribeaud

: Sex differences in ADHD trajectories across childhood and adolescence. Dev Sci, 22:e12721, 2019.

31.

National Comorbidity Survey: Adult ADHD Self-Report Scale (ASRS) Version 1.1: Background information,. 2005. Available at www.hcp.med.harvard.edu/ncs/asrs.php (accessed May 1, 2021 ).

32.

Nichols

JQV

, Shoulberg

, Garner

, Hoza

, Burt

, Murray-Close

, Arnold

: Exploration of the factor structure of ADHD in adolescence through self, parent, and teacher reports of symptomatology. J Abnorm Child Psychol, 45:625–641, 2017.

33.

Polanczyk

, Salum

, Sugaya

, Caye

, Rohde

: Annual Research Review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. J Child Psychol Psychiatry, 56:345–365, 2015.

34.

Pollak

, Oz

, Neventsal

, Rabi

, Kitrossky

, Maeir

: Do adolescents with attention-deficit/hyperactivity disorder show risk seeking?. Disentangling probabilistic decision making by equalizing the favorability of alternatives. J Abnorm Psychol, 125:1–12, 2016.

35.

Rucklidge

: Gender differences in attention-deficit/hyperactivity disorder. Psychiatr Clin North Am, 33:357–373, 2010.

36.

Samejima

: Estimation of Ability Using a Response Pattern of Graded Scores. Psychometrika Monograph No. 17. Richmond,, VA: Psychometric Corporation, 1969.

37.

Somma

, Borroni

, Fossati

: Construct validity and diagnostic accuracy of the Italian translation of the 18-Item World Health Organization Adult ADHD Self-Report Scale (ASRS-18) Italian translation in a sample of community-dwelling adolescents. Psychiatry Res, 273:753–758, 2019.

38.

Somma

, Carlotta

, Boni

, Arlotta

, Masci

, Busso

, Cerioli

, Manini

, Borroni

, Fossati

: Reliability and validity of the Structured Clinical Interview for DSM-5-Clinician Version (SCID-5-CV) Attention Deficit/Hyperactivity Disorder Criteria: Preliminary evidence from a sample of 217 Italian adolescents. J Psychopathol, 25:3–9, 2018.

39.

Taylor

, Deb

, Unwin

: Scales for the identification of adults with attention deficit hyperactivity disorder (ADHD): A systematic review. Res Dev Disabil, 32:924–938, 2011.

40.

Ustun

, Adler

, Rudin

, Faraone

, Spencer

, Berglund

, Gruber

, Kessler

: The World Health Organization Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale for DSM-5. JAMA Psychiatry, 74:520–526, 2017.

41.

Vickers

, Elkin

: Decision curve analysis: A novel method for evaluating prediction models. Med Decis Making, 26:565–574, 2006.

42.

Ward

, Wender

, Reimherr

: The Wender Utah Rating Scale: An aid in the retrospective diagnosis of childhood attention deficit hyperactivity disorder. Am J Psychiatry, 150:885–890, 1993.

43.

Wirth

, Edwards

: Item factor analysis: Current approaches and future directions. Psychol Methods, 12:58–79, 2007.

44.

Zwick

, Velicer

: Comparison of five rules for determining the number of components to retain. Psychol Bull, 99:432–442, 1986.