The Quantified Behavioral Test Failed to Differentiate ADHD in Adolescents With Neurodevelopmental Problems

Abstract

Objective: The Quantified Behavioral Test (QbTest) is a computerized diagnostic test for ADHD, used in clinical psychiatric care, but its validity may be questioned. We analyzed the QbTest’s diagnostic validity and its relation to cognitive ability and psychosocial factors in an adolescent population with a high occurrence of neurodevelopmental disorders. Method: In total, 340 participants aged 15 years, completed the QbTest, along with questionnaires, clinical and intelligence quotient (IQ) assessments. Results: The clinical assessment resulted in 89 (26%) participants with ADHD. Area under curve (AUC) scores indicated a random to poor validity of the QbTest (AUC range = 0.48-0.64). QbTest scores of inattention and impulsivity correlated with IQ. Conclusion: The QbTest was insufficient as a diagnostic test for ADHD, and was not able to differentiate ADHD from other neurodevelopmental conditions. Clinicians should be aware of the dubious discriminating power of the QbTest.

Keywords

ADHD assessment diagnostic validity neurodevelopment intelligence

Introduction

ADHD is characterized by symptoms of hyperactivity, inattention, and impulsivity with a prevalence in children of 3% to 4% (American Psychiatric Association, 2013; Polanczyk, Salum, Sugaya, Caye, & Rohde, 2015; Thapar & Cooper, 2016). There is a lack of objective laboratory tests to verify a diagnosis of ADHD. In clinical settings, the continuous performance task (CPT) that measures inattention (through vigilance) and impulsivity (through response inhibition) has been used as a part of the diagnostic evaluation, but previous meta-analyses have shown that the CPT does not have a stand-alone capacity to function as a diagnostic test for ADHD (Hervey, Epstein, & Curry, 2004; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). The Quantified Behavioral Test (QbTest) is a further development of the CPT by adding simultaneous measurements of motor activity (Ulberstad, 2017), and it aims to measure the three core symptoms of ADHD: hyperactivity, inattention, and impulsivity.

A smaller number of studies have validated the QbTest as a diagnostic tool for ADHD in children (Ulberstad, 2012b) and in adults (Edebol, Helldin, & Norlander, 2012, 2013; Ulberstad, 2012a). In those studies, the ability of the QbTest to confirm the ADHD diagnosis (i.e., sensitivity) as well as the ability to correctly identify individuals without ADHD (i.e., specificity) ranged from 86% to 90%, when patients with ADHD were compared with not affected individuals from the general population. Also, the factorial validity of the estimates produced by the QbTest, that is, activity level, inattention, and impulsivity, has been verified in adults (Hirsch & Christiansen, 2017) and in children (Reh et al., 2015), and the QbTest factor of inattention was found to correlate with the total intelligence quotient (IQ; Reh et al., 2015).

In clinical settings, the ability of the QbTest to differentiate adults or children with ADHD from patients referred for ADHD assessments has been explored (Hirsch & Christiansen, 2017; Hult, Kadesjo, Kadesjo, Gillberg, & Billstedt, 2015; Söderström, Pettersson, & Nilsson, 2014; Zelnik, Bennett-Back, Miari, Goez, & Fattal-Valevski, 2012). In some of those studies, the QbTest did not appear as an effective and diagnostic instrument for ADHD. For example, the sensitivity ranged from 47% to 68% and the AUCs ranged from 0.62 to 0.74 when ADHD was compared with clinically referred non-ADHD patients with a high proportion of autism spectrum disorder (ASD; Hult et al., 2015) or other psychiatric disorders (Söderström et al., 2014). Yet, in another study, the QbTest was able to discriminate between ADHD and ASD with an overall classification accuracy of 90% (Groom, Young, Hall, Gillott, & Hollis, 2016).

Studies on clinical populations raise the question whether the QbTest is able to discriminate ADHD from other neurodevelopmental or psychiatric conditions. Previous studies suffer from limitations such as low number of participants, and some studies were published by the manufacturer of the test (Ulberstad, 2012a, 2012b) or had received contributions from the manufacturer (Groom et al., 2016), which is problematic as financial ties in clinical studies have been previously associated with positive results (Ahn et al., 2017). Further validity studies of the QbTest’s ability to classify ADHD are required, and in particular, studies on populations with neurodevelopmental disorders other than ADHD, because this would mirror common clinical practice. In addition, there is a lack of knowledge on the relationship between QbTest performances, psychosocial factors, and cognitive ability.

Aims of the Study

Our objective was to explore the QbTest’s ability to correctly classify ADHD in a sample of individuals with a high occurrence of neurodevelopmental disorders. Furthermore, with an exploratory approach, we analyzed associations between psychiatric symptoms, cognitive functions, and the three QbTest parameters: hyperactivity, inattention, and impulsivity.

Method

The Child and Adolescent Twin Study in Sweden (CATSS)

The participants were recruited from the CATSS, which consists of twins born from 1992 and onward, recruited from the Swedish twin registry (STR; Anckarsater et al., 2011). CATSS is ongoing and has, to date, included approximately 30,000 twins. From 2004 and onward, the parents of 9- or 12-year-old twins were interviewed with the Autism-Tics, ADHD, and Other Comorbidities (A-TAC) inventory, which is a screening tool for neurodevelopmental symptoms. The A-TAC inventory targets neurodevelopmental problems and has been previously validated presenting good test–retest measures, excellent construct validity, and interrater reliability (Cubo et al., 2011; Hansson et al., 2005; Larson et al., 2010; Larson et al., 2013).

The Developmental Outcomes in a Genetic Twin Study in Sweden (DOGSS)

From CATSS, a total number of 860 same-sex twins born from 1993 to 1995 were invited to participate in the DOGSS, Supplementary Figure 1. Twins were invited to participate if at least one of the twins in a pair had screened positive in the A-TAC inventory for ASD, ADHD, tic disorder (TD), language disorder (LD), developmental coordination disorder (DCD), or behavioral disorders with known neurodevelopmental comorbidities, such as obsessive compulsive disorder (OCD), oppositional defiant disorder (ODD), conduct disorder (CD), or eating disorder (ED). From the 1995 cohort, the inclusion criteria were narrowed to include ASD and ADHD only. The number of invited twins represented approximately 15% of the population from CATSS in those age categories at the time for inclusion. In addition, a number of randomly selected controls were included representing 5% of the CATSS cohort.

Diagnostic Procedures in the DOGSS

Of the 860 invited twins for DOGSS, 457 twins went through an extensive assessment procedure around age 15. Trained psychologists, blind to all previous information and to the results of the examination of the co-twin, performed separate clinical interviews with each teenager and one or both parents. IQ was assessed using the neuropsychological battery of Wechsler Intelligence Scale for Children–IV (WISC-IV; A. S. Kaufman, Flanagan, Alfonso, & Mascolo, 2006). For the diagnostic assessment, the psychologists used the diagnostic interview Schedule for Affective Disorders and Schizophrenia in School-Age Children (K-SADS; J. Kaufman et al., 1997), and the global functioning level measured with Children’s Global Assessment Scale (C-GAS) was determined (Lundh, Kowalski, Sundberg, Gumpert, & Landen, 2010; Shaffer et al., 1983). The diagnoses were later verified by a clinical doctor specialized in child and adolescent psychiatry, who was “blind” to the presumptive status according to the A-TAC interview (Selinus et al., 2015). Questionnaires regarding psychosocial information such as school performance, peer problems, internalizing problems, antisocial behavior, misuse of alcohol and illicit drugs were administered to the participants. The participants and their parents also completed the Strengths and Difficulties Questionnaire (SDQ; Goodman, 2001).

Of the 457 participants in DOGSS, 356 individuals completed the QbTest. No major differences were observed between those who completed and those who did not complete the QbTest regarding gender, ADHD diagnosis, psychosocial factors, global functioning, and total IQ score, Supplementary Table 1. Those who completed the QbTest were more likely to have an anxiety diagnosis, and the proportion of fathers who had completed secondary school was lower. In total 10 individuals, all diagnosed with ADHD, were excluded from the final analysis because they had taken ADHD medication prior to the testing procedure. Another six individuals were removed due to incomplete diagnostic information, leaving us with 340 participants for the final analysis, Supplementary Figure 1.

The QbTest Procedure and Measures

The QbTest is a computer-administered test that combines a CPT, measuring inattention and impulsivity, with motor activity measurements during a 20-min period. The purpose of the test is to provide objective information on hyperactivity, inattention, and impulsivity. In this study, the adolescent and adult version of the QbTest delivered by Qbtech AB (http://www.qbtech.com) was used. Motor activity was measured with an infrared camera following a reflective marker attached to a headband worn by the participant. The CPT involves presentations of symbols on a computer screen and the participant is instructed to press a button only when the same stimulus is presented subsequently after another. The QbTest provides raw scores on five parameters from the measurement of movements (time active, distance, area, microevents, and motion simplicity), and from the CPT part, raw scores of three inattention parameters (omission errors [%], reaction time, and reaction time variation) and two impulsivity parameters (Commission Errors [%] and Anticipations). From those raw scores, three age- and sex-scaled parameters are calculated designated as Q scores—QbActivity, QbInattention, and QbImpulsivity—corresponding to z scores of normative data previously collected by the test provider. In addition, we constructed a total outcome QbTest score (QbTotal) comprising of the mean of the three cardinal outcomes. Higher test scores indicate worse performances (i.e., more activity, inattention, or impulsivity). In 15 cases, the QbActivity scores were set to missing due to episodes of missing movement recordings. In one case, the omission error scores were set to missing due to interruption in the end of the testing procedure.

Differences Between Examination Sites

The participants were enrolled between the years 2008 and 2011 at four different examination sites: two sites located in Stockholm, one in Gothenburg, and one in Malmoe. To be able to preclude effects from systematic errors due to the equipment used, we analyzed differences of the QbTest scores between the sites. No differences were found regarding the main parameters, QbTotal, QbActivity, QbInattention, and QbImpulsivity (data not shown). The raw score of motion simplicity was significantly higher at the Gothenburg site compared with the sites Malmoe and Stockholm (p = .023); however, after Bonferroni correction, this difference was not significant.

Statistical Analyses

All analyses were performed in STATA 14.0. Demographics presented as percentages or means, and standard deviations (SDs). For group comparisons, we used logistic or linear regression with a cluster robust sandwich estimator to adjust the precision of the estimates for the twin relationships. To evaluate the diagnostic ability of the QbTest, we applied receiver operating characteristic (ROC) curve analysis and discriminant analysis. First, we compared individuals with ADHD according to the diagnostic interview (K-SADS) with all the not ADHD-affected individuals. Second, we restricted the control group step by step by removing (a) the co-twins of the ADHD individuals, (b) the twin pairs in which one or both twins had any neurodevelopmental diagnosis other than ADHD according to the K-SADS interview (ASD, CD, ODD, TD, or LD), and (c) the twin pairs in which one or both twins had a psychiatric condition according to the K-SADS interview. ROC analysis was applied on QbTotal and the separate parameters of the QbTest (QbActivity, QbInattention, and QbImpulsivity), adjusted by gender and twin relationships, to get AUC values. For interpretation of the AUC values, we used the following criteria: 0.5 indicates a random prediction; 0.6 to 0.7, a poor validity of the test; 0.7 to 0.8, a fair validity; and values above 0.8 indicate good or excellent diagnostic validities (Tape, 2004). In addition, we performed a discriminant analysis to determine how well the three continuous parameters of the QbTest (QbActivity, QbInattention, and QbImpulsivity) and QbTotal predicted ADHD in relation to each of the control group compositions.

Associations between the QbTest parameters and social factors, neurodevelopmental or psychiatric conditions, total IQ, and the SDQ (including scores from the hyperactivity scales alone and the internalizing and externalizing scales) were analyzed using linear regression, with a cluster robust sandwich estimator for the twin pairs. The QbTest parameters entered the model as outcome variables in all analyses. Gender was adjusted for in all analysis, and adjustments were made for ADHD and the total IQ score in some analyses.

Ethical Considerations

The study was approved by the Ethical Review Board, Stockholm. The participants and their caregivers had consented orally and in writing for participating.

Results

In total, 340 individual twins participated in the study of which 55.3% were males and the median age at the QbTest procedure was 15 years (range = 14-16 years; Table 1). In total, 89 individuals were classified as having ADHD according to the clinical interview, and five (5.6%) of those individuals had an ongoing treatment with psychostimulants. Coexisting psychiatric conditions were common in the participants with ADHD (77.5%) as well as in those who did not have ADHD (59.0%). The most common neurodevelopmental diagnosis was LD affecting 27.0% of the participants, followed by TD affecting 13.8% and ASD affecting 1.7%. Anxiety disorder was the most common psychiatric condition affecting 27.3% of the participants, followed by stress-related disorders (10.4%) and previous depression (7.9%). In the participants with ADHD, all neurodevelopmental conditions except for ASD were significantly more frequent compared with the non-ADHD participants.

Table 1.

Demographic and Diagnostic Information of the Participants.

	Total sample	No ADHD	ADHD	p ^a
	n = 340	n = 251	n = 89
	N (%)	N (%)	N (%)
Sex
Male	188 (55.29)	125 (49.8)	63 (70.79)	Reference
Female	152 (44.71)	126 (50.2)	26 (29.21)	.003
Zygosity
Monozygotic	150 (44.12)	106 (42.23)	44 (49.44)	Reference
Dizygotic	190 (55.88)	145 (57.77)	45 (50.56)	.98
Parental education
Mother
Elementary school	30 (8.82)	22 (8.76)	8 (8.99)	Reference
Secondary school	168 (49.41)	115 (45.82)	53 (59.55)	.73
High school	126 (37.06)	100 (39.84)	26 (29.21)	.057
Unknown	16 (4.71)	14 (5.58)	2 (2.25)	—
Father
Elementary school	48 (14.12)	38 (15.14)	10 (11.24)	Reference
Secondary school	132 (38.82)	94 (37.45)	38 (42.7)	.26
High school	93 (27.35)	72 (28.69)	21 (23.6)	.85
Unknown	67 (19.71)	47 (18.73)	20 (22.47)	—
Psychosocial outcomes
Major school problems^b	66 (19.41)	30 (11.95)	36 (40.45)	<.001
Antisocial behavior^c	56 (16.47)	43 (17.13)	13 (14.61)	.23
Alcohol use past month	192 (56.47)	139 (55.38)	53 (59.55)	.93
Drug use past month	47 (13.82)	33 (13.15)	14 (15.73)	.38
Questionnaires
SDQ total score,^d M (SD)	33.1 (4.1)	32.7 (4.0)	34.2 (4.2)	<.001
ADHD (clinical interview)
All ADHD subtypes	89 (26.18)	0 (0)	89 (100)	—
ADHD	65 (19.12)	0 (0)	65 (73.03)	—
Inattentive subtype, ADD	22 (6.47)	0 (0)	22 (24.72)	—
Hyperactive-impulsive subtype, HDD	2 (0.59)	0 (0)	2 (2.25)	—
Global functioning
C-GAS, M (SD)	75.5 (13.5)	78.4 (13.1)	67.6 (11.6)	<.001
Cognitive function
WISC-III total score, M (SD)	94.4 (17.5)	96.9 (17.3)	87.4 (16.5)	<.001
Total IQ < 70	16 (4.71)	12 (4.78)	4 (4.49)	.502
Medication
Methylphenidate only	3 (0.88)	0 (0)	3 (3.37)	—
Atomoxetine only	2 (0.59)	0 (0)	2 (2.25)	—
Examination site
Stockholm Site 1	207 (60.88)	157 (62.55)	50 (56.18)	.72
Stockholm Site 2	6 (1.76)	4 (1.59)	2 (2.25)	—
Gothenburg	74 (21.76)	56 (22.31)	18 (20.22)	.73
Malmoe	53 (15.59)	34 (13.55)	19 (21.35)	.93
Psychiatric conditions^e
Any psychiatric condition	217 (63.8)	148 (59.0)	69 (77.5)	.002
Neurodevelopmental diagnosis
Language disorder	93 (27.35)	60 (23.9)	33 (37.08)	.003
Tic disorder	48 (14.12)	28 (11.16)	20 (22.47)	.010
Oppositional defiant disorder	14 (4.12)	3 (1.2)	11 (12.36)	<.001
Conduct disorder	9 (2.65)	2 (0.8)	7 (7.87)	.004
Autism spectrum disorder	6 (1.76)	5 (1.99)	1 (1.12)	.56
Other psychiatric diagnoses
Anxiety	73 (21.47)	52 (20.72)	21 (23.6)	.76
Stress-related disorder	35 (10.29)	27 (10.76)	8 (8.99)	.80
Depression life time	28 (8.24)	20 (7.97)	8 (8.99)	.26
OCD	14 (4.12)	8 (3.19)	6 (6.74)	.64
Substance/alcohol abuse	10 (2.94)	6 (2.39)	6 (6.74)	.049
Eating disorder	12 (3.53)	8 (3.19)	2 (2.25)	.94
Bipolar disorder	2 (0.59)	2 (0.8)	0 (0)	—
Psychosis	0 (0)	0 (0)	0 (0)	—

Note. Significant p-values marked as bolded. SDQ = Strengths and Difficulties Questionnaire; C-GAS = Children’s Global Assessment Scale; IQ = intelligence quotient; WISC = Wechsler Intelligence Scale for Children; OCD = obsessive compulsive disorder.

Group comparisons between participants with and without ADHD.

Major school problems defined as failing to receive grades or repeating a school year.

Antisocial behavior defined as criminal or violent behavior.

Total SDQ excluding the prosocial subscore.

Psychiatric conditions other than ADHD.

The QbTest as a Diagnostic Test for ADHD

We compared the ability of the QbTest to separate ADHD participants from all participants who were not diagnosed with ADHD, followed by analyses using gradually restricted control groups (Table 2). The results showed that the ability of the QbTest to correctly classify ADHD was poor in all analyses. The highest AUC value reached 0.64 for the QbImpulsivity parameter, when participants with ADHD were compared with non-ADHD individuals who were not ADHD co-twins, or were diagnosed with or had a co-twin with any neurodevelopmental condition (Table 3). No improvements were observed when the inattentive subtype (ADD) of ADHD was analyzed separately (data not shown). Similarly, the discriminative analysis showed that the sensitivity and specificity of the QbTest parameters were unsatisfactory (Table 2).

Table 2.

The Ability of the Quantified Behavioral Test (QbTest) to Correctly Classify Participants With and Without ADHD.

Type of control group	ADHD/not affected	QbTotal	QbActivity	QbInattention	QbImpulsivity	Correct classifications	PPV/NPV
Type of control group	ADHD/not affected	AUC	AUC	AUC	AUC	Sensitivity/specificity	PPV/NPV
All participants included	n = 89/	0.58	0.49	0.59	0.58	67.1%	36.2%
All participants included	n = 248	0.58	0.49	0.59	0.58	58.4%	83.4%
Excluding ADHD co-twins	n = 89/	0.58	0.48	0.59	0.58	69.5%	42.5%
Excluding ADHD co-twins	n = 204	0.58	0.48	0.59	0.58	59.7%	82.0%
Excluding ADHD co-twins and twin pairs with neurodevelopmental conditions	n = 89/	0.59	0.48	0.53	0.64	64.6%	60.9%
	n = 84	0.59	0.48	0.53	0.64	58.5%	62.3%
Excluding ADHD co-twins and twin pairs with neurodevelopmental or psychiatric conditions	n = 89/	0.60	0.55	0.55	0.59	58.5%	84.2%
	n = 27	0.60	0.55	0.55	0.59	65.4%	33.3%

Note. Presented as the AUC values from the ROC analysis, and sensitivity, specificity, PPV, and NPV from the discriminative analysis. AUC = area under curve; PPV = positive predictive value; NPV = negative predictive value; ROC = receiver operating characteristic.

Table 3.

QbTest Results in Relation to Psychosocial Factors and Neurodevelopmental Conditions Other Than ADHD.

	Model 1			Model 2
	QbActivity	QbInattention	QbImpulsivity	QbActivity	QbInattention	QbImpulsivity
	n = 314	n = 334	n = 335	n = 312	n = 330	n = 331
	β [95% CI]	β [95% CI]	β [95% CI]	β [95% CI]	β [95% CI]	β [95% CI]
Psychosocial factors
Global function (C-GAS)	−0.13 [−0.32, 0.06]	−0.15 [−0.32, 0.02]	−0.37 [−0.58, −0.16]***	0.04 [−0.16, 0.24]	0.01 [−0.15, 0.16]	−0.10 [−0.27, 0.08]
Repeated school year/failure of grades	0.55 [0.12, 0.97]*	0.22 [−0.14, 0.59]	0.40 [−0.06, 0.86]	0.45 [0.06, 0.85]*	0.12 [−0.19, 0.43]	0.25 [−0.15, 0.65]
Criminal/violent behavior	−0.32 [−0.71, 0.08]	−0.05 [−0.44, 0.33]	−0.30 [−0.76, 0.17]	−0.28 [−0.68, 0.12]	0.11 [−0.25, 0.46]	−0.13 [−0.57, 0.31]
Alcohol intake ever	−0.06 [−0.32, 0.21]	0.11 [−0.17, 0.39]	−0.06 [−0.39, 0.27]	−0.02 [−0.29, 0.24]	0.15 [−0.12, 0.42]	0.001 [−0.30, 0.31]
Drug intake ever	0.20 [−0.1, 0.55]	0.49 [0.14, 0.85]**	−0.13 [−0.59, 0.33]	0.19 [−0.17, 0.56]	0.39 [0.04, 0.73]*	−0.22 [−0.64, 0.20]
Neurodevelopmental conditions
Any ND condition other than ADHD	−0.02 [−0.30, 0.27]	0.13 [−0.16, 0.42]	0.34 [0.01, 0.66]*	−0.09 [−0.39, 0.20]	0.01 [−0.26, 0.28]	0.13 [−0.18, 0.45]
Language disorder	0.14 [−0.15, 0.42]	0.24 [−0.07, 0.55]	0.33 [−0.04, 0.71]	0.04 [−0.27, 0.35]	0.01 [−0.29, 0.30]	0.02 [−0.33, 0.37]
Tic disorder	−0.30 [−0.72, 0.11]	−0.38 [−0.73, −0.03]*	0.25 [−0.18, 0.67]	−0.33 [−0.76, 0.10]	−0.22 [−0.57, 0.13]	0.34 [−0.06, 0.73]
Autism	−0.32 [−1.09, 0.45]	0.05 [−0.60, 0.70]	−0.01 [−0.80, 0.77]	−0.33 [−1.05, 0.39]	0.02 [−0.55, 0.59]	−0.03 [−0.68, 0.63]
Opposite defiant disorder	−0.09 [−0.70, 0.51]	0.20 [−0.38, 0.78]	−0.02 [−0.85, 0.81]	−0.05 [−0.66, 0.56]	0.22 [−0.31, 0.74]	0.07 [−0.82, 0.95]
Conduct disorder	0.91 [0.28, 1.54]**	0.58 [−0.35, 1.51]	0.41 [−0.61, 1.43]	0.91 [0.27, 1.55]**	0.48 [−0.36, 1.32]	0.35 [−0.85, 1.56]

Note. Model 1: adjusted by gender and ADHD diagnosis. Model 2: adjusted by gender, ADHD diagnosis and total IQ score. Results presented as regression coefficients (β) and 95% CI. Note, Global Function Score (C-GAS) is analyzed as a standardized variable (M = 0, SD = 1). QbTest = Quantified Behavioral Test; CI = confidence interval; C-GAS = Children’s Global Assessment Scale; ND = neurodevelopmental; IQ = intelligence quotient.

p < .05. **p < .01. ***p < .001.

The comparison of the QbTest parameters and QbTest raw scores, in individuals with and without ADHD, showed that QbTotal was significantly associated with ADHD, and so were QbInattention and QbImpulsivity. QbActivity was not significantly associated with ADHD, Supplementary Table 2. The raw scores for omission errors and variation in reaction time were significantly higher in ADHD, both scores representing the QbInattention score. After adjustment for the total IQ score, all associations disappeared (data not shown).

Psychosocial Factors and Neurodevelopmental Conditions in Relation to the QbTest Performance

With an exploratory approach, we analyzed putative associations between the QbTest performances and various psychosocial factors as well as neurodevelopmental disorders other than ADHD (as presented in Table 3). A one standard deviation change in C-GAS or a positive score in the binary variables implies a change as given by the regression coefficients. We found statistically significant associations between higher QbActivity scores and severe school problems (failure to get grades or repeating a school year), and CD (Table 3). The relationship remained after adjusting for the total IQ score. We also found a significant association between a worse performance on QbInattention and any self-reported use of illicit drugs, which also remained after controlling for total IQ.

Associations were found between worse QbImpulsivity scores and the global functioning level, and the presence of any neurodevelopmental diagnosis other than ADHD (including LD, TD, ASD, ODD, or CD). Finally, a better performance on the QbInattention parameter was associated with TD. However, those relationships were no longer statistically significant after controlling for total IQ score (Table 3).

IQ Scores in Relation to QbTest Performances

The IQ scores in relation to the QbTest scores are presented in Figure 1. Significant associations were found between lower total IQ scores and worse performances on QbInattention and QbImpulsivity that remained after adjusting for a diagnosis of ADHD. There was also a weaker association observed between lower total IQ and QbActivity, but only after adjusting for ADHD (Table 4).

Figure 1.

Mean scores of the Quantified Behavioral Test (QbTest) parameters (QbActivity, QbInattention, QbImpulsivity) in relation to the IQ total scores categorized into age intervals.

Table 4.

Quantified Behavioral Test (QbTest) Results in Relation to the Total IQ Scores, and the Separate Indexes of WISC-IV.

	Model 1			Model 2
	QbActivity	QbInattention	QbImpulsivity	QbActivity	QbInattention	QbImpulsivity
	β [95% CI]	β [95% CI]	β [95% CI]	N = 311	N = 331	N = 332
	β [95% CI]	β [95% CI]	β [95% CI]	β [95% CI]	β [95% CI]	β [95% CI]
Total IQ	−0.01 [−0.02, 0.00]	−0.02 [−0.03, −0.02]***	−0.03 [−0.04, −0.02]***	−0.01 [−0.02, −0.00]*	−0.02 [−0.03, −0.01]***	−0.03 [−0.04, −0.02]***
Verbal index	−0.01 [−0.01, 0.00]	−0.01 [−0.02, −0.01]***	−0.02 [−0.02, −0.01]***	−0.01 [−0.01, −0.00]*	−0.01 [−0.02, −0.00]**	−0.02 [−0.02, −0.01]***
Performance index	−0.01 [−0.02, 0.00]	−0.02 [−0.03, −0.01]***	−0.04 [−0.05, −0.03]***	−0.01 [−0.02, −0.00]*	−0.02 [−0.03, −0.01]***	−0.03 [−0.05, −0.02]***
Working memory index	−0.00 [−0.02, 0.01]	−0.03 [−0.04, −0.02]***	−0.03 [−0.04, −0.02]***	−0.01 [−0.02, 0.01]	−0.03 [−0.04, −0.02]***	−0.03 [−0.04, −0.02]***
Processing speed index	−0.01 [−0.02, 0.00]	−0.03 [−0.04, −0.02]***	−0.03 [−0.04, −0.02]***	−0.01 [−0.02, 0.00]	−0.03 [−0.04, −0.02]***	−0.03 [−0.04, −0.02]***

Note. Model 1: adjusted for gender. Model 2: adjusted for gender and ADHD diagnosis. Results presented as regression coefficients (β) and 95% CI. IQ = intelligence quotient; WISC-IV = Wechsler Intelligence Scale for Children–IV; CI = confidence interval.

p < .05. **p < .01. ***p < .001.

SDQ and QbTest Performance

No associations were found between the self-reported or parent-reported hyperactivity scores of the SDQ and QbTest performances. Also, there were no associations found between the internalizing scale, including scores from the emotional and peer problems questions, or the externalizing scale, including scores from the conduct and hyperactivity scales (data not shown).

Discussion

This study aimed to investigate the ability of the QbTest to function as a diagnostic test for ADHD and to explore putative associations to neurodevelopmental symptoms, cognitive ability, and psychosocial outcomes. We found that the ability of the QbTest to separate individuals with ADHD from individuals without ADHD and from groups with various constellations of neurodevelopmental and psychiatric conditions were in most cases unsatisfactory.

The AUC values as well as the sensitivity and specificity results indicated that the QbTest in general has a poor ability to correctly classify ADHD, even when the control group was restricted to individuals without any neurodevelopmental or psychiatric condition. The result is consistent with two previous studies by Söderström et al. and Hult et al. that investigated the validity of the QbTest in psychiatric populations (Hult et al., 2015; Söderström et al., 2014), and similar AUCs results for the CPT measures (inattention and impulsivity) were found by Teicher and coworkers (Teicher, Polcari, Fourligas, Vitaliano, & Navalta, 2012). Yet, some previous studies reported sensitivity values around 90%, when patients with ADHD were compared with non-ADHD individuals who had been referred for ADHD assessment (Hirsch & Christiansen, 2017; Zelnik et al., 2012), although the specificity values were 45% or lower in those reports.

Studies show that ADHD patients have a higher level of locomotor activity (Garcia Murillo, Cortese, Anderson, Di Martino, & Castellanos, 2015), and most previous studies on the QbTest have observed higher QbActivity scores in the participants with ADHD (Edebol, Helldin, & Norlander, 2013; Hult et al., 2015; Lis et al., 2010; Söderström et al., 2014; Teicher et al., 2012). In this study, however, we were not able to replicate previous findings, and the results did not change after eliminating the participants with the ADD subtype from the analysis. Instead, we found that higher QbActivity scores were associated with a diagnosis of CD and school problems (failure to get grades or repeating a school year) that remained after adjusting for ADHD and the total IQ score. Our results do not preclude previous associations between ADHD and increased motor activity, but indicates that the activity measurement of the QbTest may reflect other factors than ADHD alone.

Our most robust finding was the relationship between lower IQ scores and worse results on the QbInattention and QbImpulsivity parameters, and the relationship remained after adjusting for the presence of ADHD. The association between lower IQ and a worse CPT performance has been shown previously (Milioni et al., 2017; Reh et al., 2015), suggesting that a high IQ may obscure ADHD symptoms (Milioni et al., 2017). The relationship between QbTest performances and IQ, which seems to be independent from ADHD, raises the question whether the QbTest may serve as a tool to measure cognitive ability rather than as a diagnostic instrument for ADHD. The cognitive profiles of ADHD also include emotional and behavioral aspects (A. Thapar, Cooper, & Rutter, 2017), but in this study, we did not find any relations between QbTest performances and self-reported emotional symptoms (internalizing or externalizing symptoms), violent or criminal behavior. The results support previous research indicating that cognitive functions alone are not sufficient to identify ADHD (Willcutt et al., 2005).

Strengths of this study include that the participants had gone through careful diagnostic procedures, based on validated clinical instruments, assigned by trained psychologists, and in conjunction with a specialist in child and adolescent psychiatry. As the participants were not specifically referred for ADHD assessment and were unaware of the use of the QbTest, they were blinded to its purpose. Other strengths include the cognitive testing and the completion of parental and self-assessment questionnaires with good psychometric properties.

One limitation is that the study was performed on a twin sample, which may have introduced biases, as the participants are related. To compensate for this, we used statistical methods accounting for the clustered nature of the data. The different examination sites may have introduced systematic errors due to different equipment, although we did not find any major differences of the QbTest parameters between the sites.

This study emphasizes the complexities behind the symptomatology of ADHD and related coexisting disorders, and the difficulty to perform objective diagnostics by measuring normally distributed traits and cognitive functions. We may conclude that the QbTest is dubious as a diagnostic test for ADHD, and, in particular, to separate ADHD from other neurodevelopmental conditions. The QbTest may reflect cognitive abilities rather than the actual ADHD diagnosis.

Supplemental Material

Supplement_figure_1_Flow-chart – Supplemental material for The Quantified Behavioral Test Failed to Differentiate ADHD in Adolescents With Neurodevelopmental Problems

Supplemental material, Supplement_figure_1_Flow-chart for The Quantified Behavioral Test Failed to Differentiate ADHD in Adolescents With Neurodevelopmental Problems by Viktoria Johansson, Eva Norén Selinus, Ralf Kuja-Halkola, Sebastian Lundström, Natalie Durbeej, Henrik Anckarsäter, Paul Lichtenstein and Clara Hellner in Journal of Attention Disorders

Supplemental Material

Supplement_table_1 – Supplemental material for The Quantified Behavioral Test Failed to Differentiate ADHD in Adolescents With Neurodevelopmental Problems

Supplemental material, Supplement_table_1 for The Quantified Behavioral Test Failed to Differentiate ADHD in Adolescents With Neurodevelopmental Problems by Viktoria Johansson, Eva Norén Selinus, Ralf Kuja-Halkola, Sebastian Lundström, Natalie Durbeej, Henrik Anckarsäter, Paul Lichtenstein and Clara Hellner in Journal of Attention Disorders

Supplemental Material

Supplement_table_2 – Supplemental material for The Quantified Behavioral Test Failed to Differentiate ADHD in Adolescents With Neurodevelopmental Problems

Supplemental material, Supplement_table_2 for The Quantified Behavioral Test Failed to Differentiate ADHD in Adolescents With Neurodevelopmental Problems by Viktoria Johansson, Eva Norén Selinus, Ralf Kuja-Halkola, Sebastian Lundström, Natalie Durbeej, Henrik Anckarsäter, Paul Lichtenstein and Clara Hellner in Journal of Attention Disorders

Footnotes

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Paul Lichtenstein has served as a speaker for Medice outside this work. The remaining authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by grants from Center for Psychiatry Research at the Department of Clinical Neuroscience, Karolinska Institutet, and Stockholm Health Care Services, and Vetenskapsrådet. The study funders had no role in the design, collection, analysis, or interpretation of the data, or in the writing or decision to submit the article.

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Viktoria Johansson, MD, PhD, is a consultant psychiatrist currently working with ADHD patients at the Northern Stockholm Psychiatry clinic. She is a postdoctoral researcher at Center for Psychiatric research and Karolinska Institutet in Stockholm with an interest in autism and ADHD.

Eva Norén Selinus, MD, PhD, is a child and adolescent consultant psychiatrist currently working at the Child and Adolescent Psychiatric clinic in Stockholm. She is a postdoctoral researcher at the Center for Psychiatric Research, Karolinska Institutet in Stockholm.

Ralf Kuja-Halkola, PhD, is a postdoctoral researcher and statistician working at Karolinska Institutet in Stockholm. He specializes in design and analyses of twin- and family-based studies.

Sebastian Lundström, PhD, is a licensed psychologist and a senior researcher at the University of Gothenburg.

Natalie Durbeej, PhD, is a behavioral scientist and a researcher at Uppsala University.

Henrik Anckarsäter, MD, PhD, is professor/senior consultant of forensic psychiatry at the Sahlgrenska’s Academy, University of Gothenburg. He has previously been visiting professor at the Universities of Lund and Paris XII, and is an honorary fellow of the International Academy of Law and Mental Health.

Paul Lichtenstein, PhD, is professor in genetic epidemiology at Karolinska Institutet in Stockholm.

Clara Hellner, MD, PhD, is professor in child and adolescent psychiatry and currently director of Research and Education at Stockholm Health Care Services. Her research interests are externalizing and internalizing behavior among youth and their association with neuropsychiatric conditions.

References

Ahn

Woodbridge

Abraham

Saba

Korenstein

Madden

. . . Keyhani

(2017). Financial ties of principal investigators and randomized controlled trial outcomes: Cross sectional study. BMJ, 356, Article i6770. doi:10.1136/bmj.i6770

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Anckarsater

Lundstrom

Kollberg

Kerekes

Palm

Carlstrom

. . . Lichtenstein

(2011). The Child and Adolescent Twin Study in Sweden (CATSS). Twin Research and Human Genetics, 14, 495-508.

Cubo

Saez Velasco

Delgado Benito

Villaverde

V. A.

Gabriel

Y. G. J. M.

Santidrian

A. M.

. . . Benito-Leon

(2011). Validation of screening instruments for neuroepidemiological surveys of tic disorders. Movement Disorders, 26, 520-526. doi:10.1002/mds.23460

Edebol

Helldin

Norlander

(2012). Objective measures of behavior manifestations in adult ADHD and differentiation from participants with bipolar II disorder, borderline personality disorder, participants with disconfirmed ADHD as well as normative participants. Clinical Practice and Epidemiology in Mental Health, 8, 134-143. doi:10.2174/1745017901208010134

Edebol

Helldin

Norlander

(2013). Measuring adult attention deficit hyperactivity disorder using the quantified behavior test plus. PsyCh Journal, 2, 48-62. doi:10.1002/pchj.17

Garcia Murillo

Cortese

Anderson

Di Martino

Castellanos

F. X

. (2015). Locomotor activity measures in the diagnosis of attention deficit hyperactivity disorder: Meta-analyses and new findings. Journal of Neuroscience Methods, 252, 14-26. doi:10.1016/j.jneumeth.2015.03.001

Goodman

(2001). Psychometric properties of the Strengths and Difficulties Questionnaire. Journal of the American Academy of Child & Adolescent Psychiatry, 40, 1337-1345. doi:10.1097/00004583-200111000-00015

Groom

M. J.

Young

Hall

C. L.

Gillott

Hollis

(2016). The incremental validity of a computerised assessment added to clinical rating scales to differentiate adult ADHD from autism spectrum disorder. Psychiatry Research, 243, 168-173. doi:10.1016/j.psychres.2016.06.042

10.

Hansson

S. L.

Svanstrom Rojvall

Rastam

Gillberg

Anckarsater

(2005). Psychiatric telephone interview with parents for screening of childhood autism—Tics, attention-deficit hyperactivity disorder and other comorbidities (A-TAC): Preliminary reliability and validity. The British Journal of Psychiatry, 187, 262-267. doi:10.1192/bjp.187.3.262

11.

Hervey

A. S.

Epstein

J. N.

Curry

J. F.

(2004). Neuropsychology of adults with attention-deficit/hyperactivity disorder: A meta-analytic review. Neuropsychology, 18, 485-503. doi:10.1037/0894-4105.18.3.485

12.

Hirsch

Christiansen

(2017). Factorial structure and validity of the quantified behavior test plus, Qb+). Assessment, 28, 1037-1049. doi:10.1177/1073191116638426

13.

Hult

Kadesjo

Gillberg

Billstedt

(2015). ADHD and the QbTest: Diagnostic validity of QbTest. Journal of Attention Disorders. doi:10.1177/1087054715595697

14.

Kaufman

A. S.

Flanagan

D. P.

Alfonso

V. C.

Mascolo

J. T.

(2006). Test review: Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV). Journal of Psychoeducational Assessment, 24, 278-295. doi:10.1177/0734282906288389

15.

Kaufman

Birmaher

Brent

Rao

Flynn

Moreci

. . . Ryan

(1997). Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): Initial reliability and validity data. Journal of the American Academy of Child & Adolescent Psychiatry, 36, 980-988. doi:10.1097/00004583-199707000-00021

16.

Larson

Anckarsater

Gillberg

Stahlberg

Carlstrom

Kadesjo

. . . Gillberg

(2010). The autism–tics, AD/HD and other comorbidities inventory (A-TAC): Further validation of a telephone interview for epidemiological research. BMC Psychiatry, 10, Article 1. doi:10.1186/1471-244X-10-1

17.

Larson

Lundstrom

Nilsson

Selinus

E. N.

Rastam

Lichtenstein

. . . Kerekes

(2013). Predictive properties of the A-TAC inventory when screening for childhood-onset neurodevelopmental problems in a population-based sample. BMC Psychiatry, 13, Article 233. doi:10.1186/1471-244X-13-233

18.

Lis

Baer

Stein-en-Nosse

Gallhofer

Sammer

Kirsch

(2010). Objective measurement of motor activity during cognitive performance in adults with attention-deficit/hyperactivity disorder. Acta Psychiatrica Scandinavica, 122, 285-294. doi:10.1111/j.1600-0447.2010.01549.x

19.

Lundh

Kowalski

Sundberg

C. J.

Gumpert

Landen

(2010). Children’s Global Assessment Scale (CGAS) in a naturalistic clinical setting: Inter-rater reliability and comparison with expert ratings. Psychiatry Research, 177, 206-210. doi:10.1016/j.psychres.2010.02.006

20.

Milioni

A. L.

Chaim

T. M.

Cavallet

de Oliveira

N. M.

Annes

DosSantos

. . . Cunha

P. J.

(2017). High IQ may “mask” the diagnosis of ADHD by compensating for deficits in executive functions in treatment-naive adults with ADHD. Journal of Attention Disorders, 21, 455-464. doi:10.1177/1087054714554933

21.

Polanczyk

G. V.

Salum

G. A.

Sugaya

L. S.

Caye

Rohde

L. A.

(2015). Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. Journal of Child Psychology and Psychiatry, 56, 345-365. doi:10.1111/jcpp.12381

22.

Reh

Schmidt

Lam

Schimmelmann

B. G.

Hebebrand

Rief

Christiansen

(2015). Behavioral assessment of core ADHD symptoms using the QbTest. Journal of Attention Disorders, 19, 1034-1045. doi:10.1177/1087054712472981

23.

Selinus

E. N.

Molero

Lichtenstein

Larson

Lundstrom

Anckarsater

Gumpert

C. H.

(2015). Childhood symptoms of ADHD overrule comorbidity in relation to psychosocial outcome at age 15: A longitudinal study. PLoS ONE, 10(9), e0137475. doi:10.1371/journal.pone.0137475

24.

Shaffer

Gould

M. S.

Brasic

Ambrosini

Fisher

Bird

Aluwahlia

(1983). A Children’s Global Assessment Scale (CGAS). Archives of General Psychiatry, 40, 1228-1231.

25.

Söderström

Pettersson

Nilsson

K. W.

(2014). Quantitative and subjective behavioural aspects in the assessment of attention-deficit hyperactivity disorder (ADHD) in adults. Nordic Journal of Psychiatry, 68(1), 30-37. doi:10.3109/08039488.2012.762940

26.

Tape

T. G.

(2004). Interpreting diagnostic tests. Retrieved from http://darwin.unmc.edu/dxtests/

27.

Teicher

M. H.

Polcari

Fourligas

Vitaliano

Navalta

C. P.

(2012). Hyperactivity persists in male and female adults with ADHD and remains a highly discriminative feature of the disorder: A case-control study. BMC Psychiatry, 12, Article 190. doi:10.1186/1471-244X-12-190

28.

Thapar

Cooper

(2016). Attention deficit hyperactivity disorder. The Lancet, 387, 1240-1250. doi:10.1016/S0140-6736(15)00238-X

29.

Thapar

Cooper

Rutter

(2017). Neurodevelopmental disorders. The Lancet Psychiatry, 4, 339-346. doi:10.1016/S2215-0366(16)30376-5

30.

Ulberstad

(2012a). Discriminant validity for the Quantified Behavioral Test (QbTest) in adolescents and adults. Internal document written by Qbtech AB info@qbtech.com, www.qbtech.com

31.

Ulberstad

(2012b). Discriminant validity for the Quantified Behavioral Test (QbTest) in children. Internal document written by Qbtech AB info@qbtech.com, www.qbtech.com

32.

Ulberstad

(2017). Clinical documentation for QbTest in patients with ADHD. Available from http://www.qbtech.com

33.

Willcutt

E. G.

Doyle

A. E.

Nigg

J. T.

Faraone

S. V.

Pennington

B. F.

(2005). Validity of the executive function theory of attention-deficit/hyperactivity disorder: A meta-analytic review. Biological Psychiatry, 57, 1336-1346. doi:10.1016/j.biopsych.2005.02.006

34.

Zelnik

Bennett-Back

Miari

Goez

H. R.

Fattal-Valevski

(2012). Is the test of variables of attention reliable for the diagnosis of attention-deficit hyperactivity disorder (ADHD)? Journal of Child Neurology, 27, 703-707. doi:10.1177/0883073811423821

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.30 MB

0.15 MB