Behavioral Assessment of Core ADHD Symptoms Using the QbTest

Abstract

Objective: Hyperactivity, one of the core symptoms of ADHD, has been mostly neglected in neuropsychological assessment of childhood ADHD. The neuropsychological Quantified behavior Test (QbTest) separately assesses all three core symptoms of ADHD on a behavioral level. Factor structure of the QbTest and its concurrent and discriminant validity are presented. Method: An exploratory factor analysis (n = 828 children) was performed. In a second sample (n = 102 children) a Multi-Trait-Multi-Method (MTMM) approach was used for validity analyses. Results: A three factorial model explained 76 % of the total variance, with the resulting QbTest factors significantly influenced by age and gender. The MTMM approach yielded promising results for discriminant, yet inconsistent findings for concurrent validity between the QbTest and another attention test as well as for Conners’ Parent and Teacher Rating Scales. Conclusion: Results indicate that the QbTest may be helpful for the behavioral assessment of childhood ADHD, yet further studies on its psychometric quality and clinical utility are needed.

Keywords

ADHD assessment continuous performance test (CPT)Quantified behavior Test (QbTest)

Introduction

There is a lack of well-validated, objective, and easy-to-administer tests that separately assess all three core symptoms of ADHD (i.e., age-inadequate levels of inattention, hyperactivity, and impulsiveness), one of the most common and highly impairing childhood disorders (Polanczyk, de Lima, Horta, Biederman, & Rohde, 2007; Stein, Blum, & Barbaresi, 2011). During the past decades, guidelines for the assessment and diagnosis of ADHD have been developed (e.g., American Academy of Child & Adolescent Psychiatry, 2007; American Academy of Pediatrics, 2000; Taylor et al., 2004). They all recommend the use of a variety of methods and informational sources, including child behavior observation, parent and teacher rating scales, standardized clinical interviews for parents and children, and physical examinations. Although these sources play an important role in the diagnosis of ADHD, they have been criticized because of their subjective nature. Self-report and observer-rating scales as well as clinical interviews are vulnerable to both clinician and informant biases (Edwards et al., 2007). Moreover, findings of reduced reliability for monitoring symptoms over time (Rabiner et al., 2010) and influences of children’s gender, ethnicity, and socioeconomic status (SES) on symptom ratings (Bussing et al., 2008) have further supported objections against rating scale procedures. Therefore, objective and reliable laboratory-based measures of ADHD symptoms are highly desirable, and considerable effort has been put into the development and evaluation of more direct assessment methods of core ADHD symptoms.

Over the past 20 years, computer-administered neuropsychological attention tests have become a popular means for behavioral assessment of attention processes, providing a direct observational, norm-referenced measure (Hasson & Fine, 2012). The continuous performance test (CPT) is the most commonly used neuropsychological test for ADHD evaluation in both research and clinical settings (Ballard, 1996; Corkum & Siegel, 1993; Epstein et al., 2003; McGee, Clark, & Symons, 2000; Nichols & Waschbusch, 2004; Riccio, Waldrop, Reynolds, & Lowe, 2001). The CPT is a computer-based vigilance test that aims at assessing executive functions (EFs) like sustained attention as well as selective attention processes and behavioral inhibition. These EFs have been shown to be closely related to symptoms of ADHD (for a review, see Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). When performing a CPT, participants are generally requested to react as fast as possible to target stimuli by pressing a key and to refrain from pressing it for nontarget stimuli. Failure to respond to the target stimuli is usually interpreted as a result of inattention, while responses to nontarget stimuli are interpreted as results of impulsivity. It is important to note that the term CPT in fact refers to a test paradigm, with many different versions that vary in duration from 6 to 22 min, target-to-nontarget ratio, and other test features (Riccio et al., 2001).

Although CPTs have excellent face validity and a great intuitive appeal as an objective measure for ADHD symptoms, research concerning the diagnostic utility of CPTs for ADHD remains controversial (Barkley, 1991; Halperin et al., 1990; McGee et al., 2000). While a large number of studies have reported differences in CPT performance measures between ADHD children and healthy controls (for a review, see Nichols & Waschbusch, 2004), only few have been successful in finding those differences for ADHD children and other clinical groups (O’Brien et al., 1992). Forbes (1998) stated that a diagnostic instrument must be able to distinguish between clinical groups to be of clinical utility. As it is true for behavior rating scales, interviews, standardized observation methods, or any other diagnostic tool, CPTs by themselves up to today have not been shown to have sufficient discriminative validity to determine a diagnosis of ADHD. There is agreement on the fact, though, that the use of CPTs as part of a larger neuropsychological battery can improve diagnostic precision and may be highly important for the reduction of gender bias in the diagnostic process (Hasson & Fine, 2012). Moreover, CPTs provide a quick and relatively cost-effective laboratory-based measure with the potential of being suitable for medication monitoring (Gualtieri & Johnson, 2005; Riccio et al., 2001; Wehmeier, Dittmann, Banaschewski, & Schacht, 2012; Wehmeier et al., 2011).

The Quantified behavior Test (QbTest©; see description below) is a commercial neuropsychological test that combines the CPT paradigm with apparative measurement of motor activity (for similar measurement techniques, see Teicher, Ito, Glod, & Barber, 1996) and aims at assessing all three core ADHD symptoms (i.e., inattention, hyperactivity, and impulsivity) separately. Two different QbTest versions are provided targeting two different age groups. The first version can be used for children aged 6 to 12, and the second version can be applied for participants aged 12 to 60. As described above, standard CPTs appear to have an insufficient ability to discriminate between ADHD and other clinical conditions. Thus, additional use of motor assessment might enhance test validity because hyperactivity is a core symptom in many ADHD children and has been frequently neglected in neuropsychological attention tests so far. In addition, the QbTest may be helpful in reducing gender, age, and SES biases in the diagnostic process, which are often observed when applying rating scale measures, as described above.

Despite these potential advantages, neither the factorial validity (i.e., “Does the test capture the three core ADHD symptoms?”) nor its convergent validity with other measures (i.e., “Do the QbTest results correlate with corresponding questionnaire measures?”) has been investigated. This is particularly noteworthy, as the QbTest (6-12)¹ is being marketed and widely used as a diagnostic tool for ADHD (Vogt & Williams, 2011) and even for titration of stimulant medication by a growing number of practitioners in European and North American countries (Wehmeier et al., 2011; Wehmeier et al., 2012). Moreover, the test has been incorporated in numerous studies concerning different aspects of ADHD (Brocki, Tillman, & Bohlin, 2008; Günther, poster presentation; Oades, Dauvermann, Schimmelmann, Schwarz, & Myint, 2010; Scholtens, Diamantopoulou, Tillman, & Rydell, 2011; Vogt & Williams, 2011; Wehmeier et al., 2011; Wehmeier et al., 2012). As the factorial validity of the test has not yet been investigated, it remains unclear how the different QbTest variables (a total of 17) relate to one another and whether they reflect ADHD symptoms in children. Reporting and using a large number of measures is problematic because it will lead to an increase of Type I error (false positive, that is, risk of diagnosing a healthy child with ADHD) since repeated measurement of the same latent construct dramatically decreases the threshold for a significant finding. Using standard corrections to control for the many measures (e.g., Bonferroni correction) would lead to higher risk of committing a Type II error though (false negative, that is, risk of overlooking a child who does in fact have ADHD and could profit from adequate treatment) because it drastically raises the threshold for a significant finding. These methodological problems could be attenuated by integrating variables that presumably measure the same latent constructs with single factors.

Besides the methodological issues, reporting a multitude of measures is highly inefficient for clinical practice. For practitioners, it is much more convenient to have few main parameters to consider and interpret than to observe 10 or more variables from one single test. This is especially true for diagnoses in ADHD where already a multitude of different informational sources and measures must be applied and integrated (see Taylor et al., 2004).

Factor analysis (FA) is the methodological procedure of grouping variables together and reducing redundant information by producing factor scores that are easier to interpret and thus a major benefit for practitioners. The primary aim of this study, therefore, is to explore the factorial structure of the QbTest and its conceptual accordance with core ADHD symptoms.

In addition to the open question regarding factorial validity, it is unknown whether the symptom dimensions measured with the QbTest overlap with questionnaire-based measures. Particularly, it is unknown whether the specific measurement of motor activity as assessed with the motion-tracking system incorporated in the QbTest is related to parent or teacher ratings of hyperactive behavior. The secondary aim of the study, therefore, is to examine the concurrent and divergent validity of the evolving QbTest factors.

Method

Procedure and Participants

Two separate samples were used to (a) analyze the structure of the QbTest (Sample I) and (b) analyze concurrent and discriminant validity of the evolving factors (Sample II). Sample I consisted of 901 German children who were referred to a practice for Child and Adolescent Psychiatry and Psychotherapy for ADHD assessment. Assessment was based on diagnostic standards as formulated in the guidelines (see Taylor et al., 2004). Clinical and psychological assessments were performed by a multiprofessional team. As part of the assessment process, children also completed the QbTest. As Sample I consisted of convenience sampling in the private pediatric practice, some children who were presented in the practice had been previously examined for ADHD symptoms elsewhere. Irrespective of preexisting diagnostic results though, all children had to complete routine assessment to assure well-founded diagnosis. Children who received ADHD-specific medication were off medication for at least 24 hr prior to performing the QbTest. Assessment took place in the practice and was either performed by a senior physician (LL) or by well-trained medical staff. All children received pharmacological and/or psychological treatment after ADHD diagnosis was confirmed.

Sample II consisted of 102 strictly diagnosed German ADHD children who were diagnosed and treated for ADHD at the University Hospital of Child and Adolescent Psychiatry, Essen, and at the Department of Clinical Psychology and Psychotherapy at the University of Marburg. A standardized Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR; American Psychiatric Association [APA], 2000)–based clinical interview with the parents (Parental Account of Childhood Symptoms [PACS]; Chen & Taylor, 2006; Delmo, Weiffenbach, Gabriel, Stadler, & Poustka, 2000/2001) and the Conners’ third Parent and Teacher Rating Scales (Conners, 2008), intelligence testing (Wechsler Intelligence Scale for Children–Fourth edition [WISC-IV], German version; Petermann & Wechsler, 2011; Sattler, 1992), and the QbTest were part of routine assessment. Part of Sample II (n = 32) additionally completed another laboratory test for attention assessment, the children’s test battery of attention assessment, with the subtests sustained attention, Go/No-Go, and divided attention (KITAP; Zimmermann, Gondan, & Fimm, 2002). Assessment was performed by well-trained medical staff, and again, all children received pharmacological and/or psychological treatment after ADHD diagnosis was confirmed.

Study protocols in accordance with the criteria of the Declaration of Helsinki were reviewed and approved by the local institutional review boards. Informed consent was obtained from all parents or guardians and children prior to the assessment, and their confidentiality was assured.

Deletion of univariate outliers in Sample I (3 SDs above mean in any of the relevant QbTest variables) left 829 cases for analysis. Because only one person with an age of 12 was available in the data set, this case was also excluded from further analysis for homogeneity of the sample. The final Sample I thus consisted of 828 cases: 588 males (M age = 8.5 years, SD = 1.6 years) and 240 females (M age = 8.5 years, SD = 1.5 years). According to Comrey and Lee (1992), this sample size is very good to excellent for exploratory factor analysis (EFA). Distribution of age and gender in the remaining sample did not significantly differ from the total sample. The gender distribution reflects the frequently reported distribution of gender in ADHD, with boys outnumbering girls about 3 to 1 (APA, 2000). Sample II consisted of 102 cases: 79 males (M age = 8.9 years, SD = 1.7 years) and 23 females (M age = 9 years, SD = 1.5 years). Table 1 presents the distribution of age and gender in the final Sample I and in Sample II.

Table 1.

Frequency Distribution of Sample I and Sample II by Age and Gender.

Sample I							Sample II
			Gender						Gender
			Female		Male				Female		Male
Age	n	%	n	%	n	%	n	%	n	%	n	%
6	110	13.3	27	3.2	83	10.0	6	5.8	0	0	6	5.8
7	146	17.6	46	5.6	100	12.1	15	14.7	3	2.9	12	11.8
8	153	18.5	41	5.0	112	13.5	26	25.5	6	5.9	20	19.6
9	172	20.8	59	7.1	113	13.6	18	17.7	7	6.8	11	10.8
10	140	16.9	42	5.1	98	11.8	17	16.7	3	2.9	14	13.7
11	107	12.9	25	3.0	82	9.9	11	10.8	2	1.9	9	8.8
12	0	0	0	0	0	0	9	8.8	2	1.9	7	6.8
Total	828	100	240	29.0	588	71.0	102	100	23	22.5	79	77.5

The Quantified Behavior Test for Children Aged 6 to 12 Years

The QbTest is a combined CPT and activity test for children aged 6 to 12 years (Ulberstad, 2012), which aims to assess all three core symptoms of ADHD in one test.² While performing a standardized CPT on a computer, the movements of the participant are recorded with an infrared camera following a reflective marker attached to a headband that the participant wears while performing the test. The infrared camera is placed about 1 m away from the participant, who is sitting in front of a computer screen. Participants are seated on a stool with no back support or armrest, to assure that they do not adopt a reclining posture. The QbTest CPT involves presentation of two different stimuli: a gray circle (target) and a gray circle with a cross (nontarget). The stimuli are presented on the screen for 100 ms per stimulus with an interstimulus interval (ISI) of 1,900 ms. The total number of stimuli presented in QbTest is 450 with an equal number of target and nontarget stimuli appearing in random order. Over the course of the test (15 min), participants are asked to press a button once in response to every target signal as fast as possible and to refrain from responding to nontargets. The test instructions thus emphasize both speed and accuracy. Participants’ activities during the test are recorded by reading the coordinates (X and Y) of the headband marker. The position of the marker is sampled 50 times per second, with a spatial resolution of 1/27 mm per camera unit (Ulberstad, 2012). QbTech© provides separate norms for boys and for girls, as well as for all age groups included in QbTest 6-12 (age groups are per year, that is, separate norms for ages 6, 7, 8, 9, 10, 11, and 12). According to the test manual (Ulberstad, 2012), normative data from a control group of healthy children are based on a total of n = 576, including n = 262 males and n = 314 females.

The QbTest reports a total of 17 parameters. Those can be divided into activity and CPT measures. The reported activity measures include 5 parameters: (a) Time Active, which reflects the percentage of time the subject has moved more than 1 cm/s; (b) Distance, which reflects the distance traveled by the reflective headband marker and is measured in meters; (c) the score Area, measured as the surface covered by the headband reflector during the test and is presented in square centimeters; (d) Total Number of Microevents that are small movements of the reflective marker that occur when a position change since the last microevent is greater than 1 mm; and (e) Motion Simplicity, a measure of complexity of the motion pattern that is being reported in percentage.

Twelve CPT measures are reported, including (f) Reaction Time (RT) as the average time of all correct responses. This score indicates latency in information processing and motor response speed. (g) The score Outliers represents RTs that are very slow compared with the overall RT performance during the test. (h) RT Variation (RTVar) is calculated by the standard deviation of the mean of correct response times. It is a measure of the participant’s inconsistency in response times. (i) The score Normalized Variation (NormVar) is the RTVar expressed in terms of RT. (j) The total number of missed targets is represented in the score Omission Errors, while the total number of false hits is depicted by (k) Commission Errors. (l) The score Normalized Commission Errors displays the proportion or ratio of commission errors to correct responses to the target stimulus. Too fast responses to a stimulus (less than 150 ms after presentation of the stimulus) are reported by the score (m) Anticipatory. When there is more than one button press per stimulus presentation, this is measured and reported by (n) Multiresponse. (o) D-Prime Modified (d′) is a measure of signal detectability. It reflects accuracy of target (signal) to nontarget (noise) discrimination and is calculated from commission and omission errors. (p) Longest Passivity is the maximum number of consecutive omission errors and gives information about the longest time the participant has been passive during the CPT. Finally, the total number of incorrect responses during the test is represented in the score (q) Error Rate that is calculated from commission errors plus omission errors divided by the total number of stimuli.

From all of the 17 above described QbTest parameters, 6 are secondary measures and measures of test involvement rather than providing direct performance information. Those secondary measures are calculated from performance information assessed by 1 of the 11 primary QbTest variables. Using those secondary variables for factor analysis would imply double counting the direct information contained in the primary variables. The secondary variables were thus excluded from further analyses (Variables 7, 9, 12, 15, 16, 17). Consequently, a total of 11 QbTest variables consisting of 5 activity measures and 6 CPT measures were included in the following analyses.

Statistical Analysis

Data reduction and analyses were carried out using the statistical package SPSS 19.0. Prior to analyses, all QbTest variables were examined for accuracy of data entry, missing values, and outliers. No missing values were found, and outliers were identified as being 3 SDs above mean for each of the QbTest variables.

As a first step, a series of EFA were performed for data reduction and to obtain the factor structure of the test. Variables were included in the factor analysis if the following criteria were met: (a) variables loaded significantly (>.30) on at least one factor and (b) conceptual coherence was evident. In case of double loadings (>.30 on more than one factor), variables were attributed to the factor with the highest existing factor loading for this variable as well as when conceptual coherence was given. The scree test and the number of eigenvalues above 1.0 were used to select the number of factors for extraction. Because it is unlikely that the underlying dimensions are totally unrelated, we did not restrain our analyses to varimax rotation, but used oblique rotation (δ = 0) instead. Also, each factor had to receive salient loadings (>.30) from more than one variable.

As a second step, Cronbach’s α was computed to report internal consistency for the emerging factors. Finally, influences of age and gender on the QbTest factors were analyzed with a MANOVA. Effect sizes for differences between gender and age groups were reported when appropriate and interpreted according to Cohen (1977; small: .01 ≥ η², medium: .06 ≥ η², and large: η² ≥ .14).

To further validate the QbTest, a multitrait–multimethod (MTMM) approach was used in Sample II, comparing the established factors (e.g., Hyperactivity, Inattention, Impulsivity) with the results of another test for attention assessment (KITAP; Zimmermann et al., 2002) and the Conners’ third Parent and Teacher Rating Scales (Conners, 2008). Association of standardized KITAP results in the subtest sustained attention (comprising variables RT, Omission, and Commission errors) as well as standardized parent and teacher behavior ratings (Conners’ DSM-Inattention subscale and DSM-Hyperactivity/Impulsivity subscale) with the QbTest factors were examined. According to the MTMM approach, we would expect significant, positive correlations between variables or factors representing the same construct (e.g., inattention) measured with different methods (e.g., rating scale vs. attention test). A significant positive correlation between those same constructs would be regarded as an indicator of concurrent validity. However, no significant correlations would be expected between different constructs (e.g., inattention vs. peer problems) assessed with different methods (e.g., rating scale vs. attention test). A low correlation between different constructs would give information on discriminant validity of the QbTest factors. The Conners’ subscale Peer Relations as well as IQ scores from the German short version of the WISC-IV (Petermann & Wechsler, 2011; Sattler, 1992) were included in the correlational analysis for this reason. Except for the Inattention factor, we would expect the other QbTest factors not to be associated with either IQ or with the Conners’ subscale of Peer Relations. Since Attention is a precondition for cognitive testing, we would expect the Inattention factor to be negatively correlated with IQ, but not to be correlated with the Peer Relation subscale. Finally, Pearson correlations were computed and interpreted according to Cohen (1988; small: r ≥ .1, medium: r ≥ .3, and large: r ≥ .5).

Results

Exploratory Factor Analyses

The Kaiser–Mayer–Olkin Measure (KMO) of sampling adequacy was .74, indicating a good sampling adequacy (Kaiser, 1974). No QbTest variables were excluded from the analysis since only one variable showed low communality of .11 (see Table 2). According to Bühner (2006), this can be tolerated if the total sample size is high and conceptual coherence is given. The correlation matrix was subjected to principal axis factoring with oblique rotation, yielding a three-factor solution according to the scree test (factor eigenvalues: Factor 1 = 5.40, Factor 2 = 1.59, Factor 3 = 1.33).

Table 2.

Rotated Factor Loadings and Communality Values (h²) From a Principal Axis Factor Analysis of QbTest Variables Using Oblimin Rotation (N = 828).

QbTest variables	1	2	3	h ²
Factor 1: Hyperactivity
Time active	0.92	0.47	0.36	0.85
Distance	0.97	0.44	0.39	0.94
Area	0.97	0.44	0.39	0.93
Microevents	0.99	0.46	0.38	0.99
Motion simplicity	0.52	0.30	0.18	0.27
Factor 2: Inattention
Omission errors	0.52	0.61	0.46	0.51
Reaction time	0.40	0.92	−0.08	0.92
Reaction time var.	0.54	0.93	0.44	0.94
Factor 3: Impulsivity
Commission errors	0.27	0.04	0.77	0.60
Multiresponse	0.23	0.12	0.31	0.11
Anticipatory	0.27	0.21	0.75	0.57
Eigenvalue	5.40	1.59	1.33
% total variance	49.13	14.43	12.11

Note. QbTest = quantified behavior test. Highest loadings are boldface.

Table 2 presents the factor loadings and communalities for the 11 QbTest variables as well as eigenvalues and percentages of explained variance for each factor in this analysis. The resulting three factors explained 76% of the total variance. The first factor accounted for 49.13% of the total variance with five QbTest variables conceptually related to motor activity/motion (i.e., Time Active, Distance, Area, Microevents, Motion Simplicity). The second factor explained 14.43% of the variance, with three variables conceptually related to inattention (i.e., Omission Errors, RT, RTVar). Finally, the third factor accounted for 12.11% of the total variance with variables conceptually related to impulsivity (i.e., Commission Errors, Multiresponse, Anticipatory). Thus, factor names were proposed according to ADHD core symptoms: Hyperactivity, Inattention, and Impulsiveness. Table 2 shows the rotated factor loadings and communality values of the QbTest.

Internal consistency of all three factors was acceptable with the following Cronbach’s alpha values: Hyperactivity (α = .95), Inattention (α = .76), and Impulsivity (α = .60). Correlations between factors were moderate (.18 ≥ r ≤ .48) with highest correlations occurring between Hyperactivity and Inattention (r = .48) and Hyperactivity and Impulsivity (r = .39).

Influences of Age and Gender

A MANOVA for the obtained three QbTest subscales resulted in significant main effects for gender, Wilks’s Lambda = .95, F(5, 814) = 14.49, p = .001, η² = .051, and age, Wilks’s Lambda = .63, F(15, 2248) = 26.93, p = .001, η² = .14, with medium to large effect sizes according to Cohen. Thus, interpretation of these subscales is dependent on age and gender influences. Means and standard deviations for the QbTest subscales are presented separately for gender and age groups in Table 3.

Table 3.

Means (and SDs) for the Three QbTest Factor Scales by Gender and Age.

Gender	Age	Hyperactivity	Inattention	Impulsivity	n
Males	6	0.88 (1.14)	1.19 (0.93)	0.42 (1.16)	83
Females	6	0.60 (0.87)	1.13 (0.91)	−0.13 (0.83)	27
Males	7	0.52 (0.95)	0.50 (0.79)	0.42 (1.21)	100
Females	7	0.28 (1.10)	0.64 (0.85)	−0.31 (0.75)	46
Males	8	0.05 (0.87)	−0.05 (0.81)	−0.04 (0.83)	112
Females	8	−0.23 (0.68)	0.18 (0.73)	−0.23 (0.82)	41
Males	9	−0.02 (0.96)	−0.25 (0.82)	0.11 (1.17)	113
Females	9	−0.46 (0.71)	−0.17 (0.67)	−0.36 (0.82)	59
Males	10	−0.45 (0.77)	−0.57 (0.78)	−0.10 (0.77)	98
Females	10	−0.55 (0.73)	−0.54 (0.69)	−0.47 (0.71)	42
Males	11	−0.46 (0.78)	−0.85 (0.62)	0.02 (0.98)	82
Females	11	−0.66 (0.78)	−0.89 (0.49)	−0.35 (0.79)	25
Males	Total	0.08 (1.02)	−0.02 (1.02)	0.13 (1.05)	588
Females		−0.20 (0.91)	0.05 (0.94)	−0.32 (0.79)	240

Concurrent and Discriminant Validity of the QbTest Factors

Table 4 shows the Pearson correlations of the established QbTest factors with Conners’ parent and teacher ratings, KITAP results, and IQ. There was some evidence supporting convergent validity of the QbTest factors. First of all, there was a significant positive correlation between the QbTest factor Hyperactivity and teacher ratings of hyperactive behavior (r = .27**, p < .01) on the Conners’ DSM-Hyperactivity/Impulsivity subscale. Thus, the more motor activity was measured by the QbTest, the more children were rated as being hyperactive-impulsive in classroom situations by their teachers. Moreover, the QbTest factor Impulsivity was significantly correlated with low RT in the KITAP (r = −.42*, p < .05). Children who tended to react faster on the KITAP also scored high on the Impulsivity factor in the QbTest. Finally, as expected, the QbTest factor Inattention showed a significant negative correlation with IQ (r = −.27*, p < .05), meaning that children with higher IQ scores have lower scores on the QbTest Inattention factor. Despite those convergent correlations, however, there were no other significant correlations between KITAP variables and QbTest factors. Also, QbTest factors did not significantly correlate with Conners’ parent ratings of inattentive or hyperactive/impulsive behavior.

Table 4.

Multitrait–Multimethod (MTMM) Matrix for Sample II.

		QbTest factors			Conners’ Parent Rating Scale			Conners’ Teacher Rating Scale			KITAP			WISC−IV
Method	Trait/variable	1	2	3	4	5	6	7	8	9	10	11	12	13
QbTest factors	1 Hyperactivity	1^a
	2 Inattention	.38**^a	1^a
	3 Impulsivity	.38**^a	.22*^a	1^a
Conners’ Parent Rating Scale	4 DSM−Scale Inattention	−.06^b	−.02^b	−.04^b	1^b
	5 DSM−Scale Hyperactive/Impulsive	.09^b	.10^b	.05^b	.44**^b	1^b
	6 Peer Relations	−.05^b	−.07^b	−.02^b	.22*^b	.32**^b	1^b
Conners’ Teacher Rating Scale	7 DSM−Scale Inattention	−.04^b	.03^b	.15^b	.47**^b	.23*^b	.21*^b	1^b
	8 DSM−Scale Hyperactive/Impulsive	.27**^b	.17^b	.03^b	.03^b	.40**^b	.35**^b	.39**^b	1^b
	9 Peer Relations	−.12^b	−.02^b	−.06^b	.03^b	.13^b	.39**^b	.32**^b	.35**^b	1^b
KITAP	10 Sustained attention/reaction time	.07^c	−.20^c	−.42*^c	.26^c	−.04^c	.05^c	−.41*^c	−.02^c	−.06^c	1^c
	11 Sustained attention/commission errors	−.04^c	−.25^c	−.25^c	−.17^c	−.19^c	.10^c	.00^c	.01^c	.22^c	.12^c	1^c
	12 Sustained attention/omission errors	−.01^c	.07^c	−.21^c	−.21^c	.08^c	−.06^c	−.33^c	.06^c	−.14^c	.19^c	.44*^c	1^c
WISC−IV	13 Total IQ score	−.14^d	−.27*^d	−.13^d	.08^d	.08^d	.02^d	.18^d	−.10^d	.06^d	.02^d	−.14^d	.16^d	1^d

Note. MTMM = multitrait–multimethod; KITAP = Test of Attentional Performance for Children; WISC−IV = Wechsler Intelligence Scale for Children–Fourth edition; DSM = Diagnostic and Statistical Manual of Mental Disorders. Significant correlations indicating concurrent or discriminant validity are boldface.

n = 102. ^bn = 94. ^cn = 32. ^dn = 87.

p < .05 level (two−tailed). **p < .01 level (two−tailed).

Concerning discriminant validity, no significant correlations between QbTest factors Hyperactivity and Impulsivity and WISC-IV results were found that can be interpreted as an indicator of discriminant validity for those factors. Furthermore, as expected, Conners’ parent and teacher ratings of Peer Relations showed no significant correlation with any of the three QbTest factors.

Discussion

The QbTest is a behavioral assessment tool in ADHD. We explored the factor structure of the QbTest children’s version (6-12) in a large sample of German children referred for assessment of ADHD. An exploratory principal factor analysis yielded a three-factorial model that explained 76% of the total variance in the data. Validity analyses in a second sample of German ADHD children revealed mixed findings regarding the convergent and divergent validity of the established QbTest factors. Although the Hyperactivity factor was significantly correlated with teacher ratings of hyperactive behavior, the other two QbTest factors showed less overlap with rating measures as well as with another laboratory test for attention assessment. Since other CPTs do not separately assess the participant’s motor activity, the three emerging factors and initial evidence of their concurrent validity constitute a major advantage of the QbTest. Given that validity results are heterogeneous though, further studies exploring psychometric quality and clinical utility of the QbTest are needed.

The factor structure in the presented study shows that there is one factor explaining a large amount of variance and two more factors each explaining additional unique parts of variance. From this finding, we can conclude that a participant’s performance on the QbTest cannot be sufficiently described by one overall measure of performance, but instead all three factor scores must be considered. While the first factor, Hyperactivity, contained the five motor activity variables, including Time Active, Distance, Area, Microevents, and Motion Simplicity, the second factor, Inattention, consisted of the three variables Omission Errors, RT, and RT Variation, which have been frequently linked with inattention in other studies (McGee et al., 2000; Nichols & Waschbusch, 2004). The third factor, Impulsivity, contained the three variables Commission Errors, Multiresponse, and Anticipatory, which clearly show conceptual coherence with behavioral impulsivity. Particularly, commission errors have been used as an indicator for impulsivity in many studies applying CPTs (Egeland & Kovalik-Gran, 2010a, 2010b; McGee et al., 2000; Nichols & Waschbusch, 2004; Willcutt et al., 2005). Hyperactivity was the factor explaining the largest amount of variance in this sample. Apparently, the five motor activity variables that show extremely high factor loadings on Hyperactivity are best described in one single factor due to their high conceptual coherence. Variables constituting the Inattention and Impulsivity factor are more heterogeneous, as also shown in their lower factor loadings. Internal consistency values for all three factors were adequate to excellent. This overall result is satisfactory.

Results from the MTMM analyses yielded mixed findings for convergent and discriminant validity of the established QbTest factors. First of all, the QbTest factor Hyperactivity was significantly correlated with Conners’ teacher ratings of hyperactive/impulsive behavior, indicating convergent validity for this factor. There seems to be correspondence between hyperactive behavior as measured by the QbTest and hyperactive behavior rated by teachers. Since teacher ratings have been shown to be influenced by children’s gender, ethnicity, and SES (Bussing et al., 2008), and have low reliability when monitoring symptoms over time (Rabiner et al., 2010), a valid and reliable laboratory measure for hyperactive behavior would be a welcome addition to ADHD assessment methods. Existing laboratory measures of ADHD, including former CPT versions, have limited to no ability to assess unique hyperactivity symptoms. Therefore, the factor structure presented in this study and the accordance of QbTest Hyperactivity and teacher ratings of hyperactive behavior support the combined measurement of CPT performance and motor activity as implemented in the QbTest.

While convergent validity of the established QbTest factors was partially supported by the significant correlation of the Hyperactivity factor with Conners’ teacher ratings, no associations were found with Conners’ parent ratings of inattentive or hyperactive-impulsive behavior. This result is in line with previous research examining differences in parent versus teacher ability to detect ADHD behaviors, showing teacher ratings to be more accurate (Tripp, Schaughency, & Clarke, 2006).

Furthermore, the Inattention factor showed a significant negative correlation with IQ meaning that children with higher IQ results had lower inattention values. Since attention is the basic behavior necessary to perform well on almost any kind of cognitive test, this result was expected. The result is interesting because it raises the question whether highly inattentive children (i.e., score high on the Inattention factor) may be underestimated by intelligence testing. Moreover, it brings up the question whether children with a higher IQ might be able to compensate deficits on CPTs. Future studies should examine whether these children often end up as “false negatives,” meaning they show normal CPT results while they actually do fulfill diagnostic criteria for ADHD. Contrary to our expectations concerning the association with rating scale measures, the QbTest Inattention factor did not show significant correlations with either Conners’ parent or teacher ratings of inattentive behavior. Maybe, teachers are better able to detect externalizing behavior (i.e., hyperactivity) that is highly visible in classroom situations than internalizing behavior (i.e., inattention), that normally does not disturb classroom proceedings.

We hypothesized that QbTest factors would be associated with KITAP results indicating convergent validity for the three QbTest factors. Results from the MTMM however showed no significant, positive correlations between QbTest factors and KITAP variables. Perhaps the differing levels of aggregation (factors vs. single variables) influenced the correlations. Since KITAP only reports results on single variable level, it was not possible to explore correlations on factor/trait level.

Results regarding discriminant validity were promising since QbTest factors did not significantly correlate with either Conners’ parent or teacher ratings of peer problems. Also, as expected, the QbTest factors Impulsivity and Hyperactivity did not correlate with IQ in the WISC-IV.

Overall, results regarding the validity of the established QbTest factors were heterogeneous. Reports of low to nonexistent correlations between laboratory measures and behavior rating measures of ADHD symptomatology are not unusual. Previous studies examining the relationship of CPTs and behavior ratings for ADHD have repeatedly failed to find significant intercorrelations for parent as well as teacher ratings (DuPaul, 1991; Edwards et al., 2007). Among other investigators, Barkley (1991) has challenged the ecological validity of laboratory measures of ADHD symptoms primarily because of the low to moderate correlations CPT measures have been shown to have with ratings of behavior problems. Others who have studied activity levels in children engaged in a CPT have stated that the CPT setting in fact imitates a classroom situation in which children are for most of the time required to remain seated and to engage in a given task (Reichenbach, Halperin, Sharma, & Newcorn, 1992; Teicher et al., 1996). It has been speculated that one reason for those inconsistent findings might be low correspondence between ratings of behavior and constructs measured by CPTs (Edwards et al., 2007). Behavior ratings on one hand can be seen as an impression that is based on the accumulation of behavior in a certain period of time that occurs in real-life situations (i.e., classroom or home). CPTs on the other hand explore and report behavior in a very specific moment in a laboratory setting. It may be that low correlations repeatedly found for these measures can be explained by the fact that these very different methods simply measure different aspects of behavior. We would expect similar measures like two different CPT versions to show significant, positive correlations then. However, as described above, results from the MTMM showed no significant, positive correlations between QbTest factors and KITAP variables.

In a current study that is in preparation, the predictive value of the QbTest together with the other variables in this study (i.e., Conners’ Rating Scales, KITAP, IQ Scores) will be examined in a set of ADHD patients and healthy matched controls to further evaluate the diagnostic utility of the QbTest.

Additional results of this study are in accordance with key findings of the ADHD literature. First, as expected, gender significantly influenced ADHD symptom severity in Sample I. Moderating effects of gender on ADHD symptomatology have been repeatedly reported in the literature. Girls are more likely to be inattentive and show more internalizing problems but less disruptive behavior compared with boys and are therefore at risk of being under-identified (Berry, Shaywitz, & Shaywitz, 1985; Gershon, 2002). In addition, in a meta-analytic review, Hasson and Fine (2012) found gender to be a significant moderating factor when using CPTs for ADHD assessment with gender effects being more pronounced for impulsivity than for inattention. We replicated those findings from Hasson and Fine (2012), as in our study, gender effects were also most evident for the Hyperactivity and the Impulsiveness subscales and less so for the Inattention subscale. Behavioral assessment measures with separate norms for boys and girls may be an option to reduce gender bias in the diagnostic process.

Second, consistent with the literature (Brocki et al., 2008), age also affected ADHD symptom scores measured with the QbTest. Across all three subscales, QbTest scores decreased with age, highlighting the necessity of age-specific norms as provided by the QbTest.

While assessment of the factorial structure and its concurrent and discriminant validity is an essential first step in evaluating the overall validity of the QbTest, several further issues need to be addressed before the test can be regarded as a well-validated screening and/or diagnostic tool for childhood ADHD. In particular, further research needs to clarify whether the test adequately distinguishes between children with and without ADHD and between children with ADHD and other disorders such as anxiety, depression, and autism spectrum disorder. This is particularly important given the high comorbidity rates between ADHD and these disorders (APA, 2000). Also, it would be desirable for future studies to compare different ADHD subtypes with regard to their QbTest factor scores. As ADHD children constitute a highly heterogeneous group, a significant task for future research is to match different CPT versions with different ADHD subtypes. Altogether the factor-analytic and correlational results presented in this study may indicate that the QbTest is particularly fit to assess children from the predominantly hyperactive as well as the combined hyperactive/impulsive subtype. This will have to be further explored in subsequent psychometric studies on the QbTest. Finally, although some research on QbTest’s sensitivity to detect treatment effects is available (Vogt & Williams, 2011; Wehmeier et al., 2011; Wehmeier et al., 2012), further research regarding this topic is needed.

Limitations

Although the results presented in this study show initial evidence for the utility of the QbTest, some important limitations have to be considered. First, the sample used for factor analyses in this study lacks cases in the 12-year-old category, resulting in an age range that did not include all age groups the QbTest is targeting at. Moreover, since the sample analyzed here consisted of a convenience sample of children referred for ADHD assessment, the sample included children with a wide range of severity of ADHD symptoms. Also, information on the existing comorbidity as well as the ADHD subtype was not available for analysis.

Second, several variables in our factor analysis showed high loadings on more than one factor. The variable Omission Errors for instance loaded high on Inattention but also significantly on Hyperactivity. This variable does not seem to differentiate well between the two different symptom clusters. The variable Multiresponse also had a low communality value and showed moderate loadings on all three factors with the highest loading on Impulsivity. Although the question arises whether these two variables should be removed from the test, the variable Omission Errors is conceptually important and has been consistently interpreted as a measure of inattention in previous CPT research (Halperin, Wolf, Greenblatt, & Young, 1991; Nichols & Waschbusch, 2004). Omission errors have been shown to be related to aspects of attention but significantly less to hyperactive/impulsive behavior and thus comprise differential information (Egeland & Kovalik-Gran, 2010b). We therefore believe that the variable Omission errors should remain in the test, and its classification within the Inattention factor is justifiable. The low factor loadings of Multiresponse can be explained by limited variance of this variable. Since only the most impulsive children press more than one time to a given stimulus, this leads to a restriction of variance and thus low factor loadings. Despite those mathematical issues, the variable Multiresponse could be of clinical use to identify extreme cases of impulsivity and should therefore remain in the test as well.

Finally, a diagnosis is always made by a clinician who must interpret and integrate different diagnostic results. Therefore it is important to note that as far as construction and implementation of rigorously designed and reliable tests can help us avoid subjective biases in the diagnostic process, the role of subjective interpretation of test results must be considered.

Conclusion

The QbTest is a behavioral assessment tool for ADHD that is increasingly being used in research and clinical settings across many different Western countries. This study is the first to examine the factorial structure and validity of the QbTest. Overall, the results show that the single QbTest variables meaningfully group together and that there is initial evidence for concurrent validity of each of the three emerging factors. However, low correlations with the Conners’ Parent Rating Scales and another laboratory test for attention assessment point to the need for extended research on the psychometric quality of the QbTest. Also additional research needs to further clarify the underlying constructs captured by CPTs in general and whether QbTest may be particularly beneficial in the behavioral assessment of the predominantly hyperactive as well as the hyperactive/impulsive ADHD subtype.

Footnotes

Acknowledgements

We thank all children and their parents for participation. We also thank the anonymous reviewer for helpful and constructive comments on our manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

Verena Reh is a clinical psychologist (Dipl. -Psych.) and PhD student whose main research interests are behavioral assessment methods for ADHD and new psychological treatment options for childhood ADHD and comorbid disorders.

Martin Schmidt is a psychologist (Dipl. -Psych.) and PhD student whose main research interests are assessment methods for ADHD in childhood and in adulthood, and new psychological treatment methods for children, adolescents, and adults with diagnosis of ADHD.

Le Lam, MD, is a child and adolescent psychiatrist, psychotherapist, pediatrician, former clinical and research assistant in child and adolescent psychiatry at University of Marburg, clinical and psychological assessment, and treatment of ADHD and comorbid disorders.

Benno G. Schimmelmann is an assistant professor of child and adolescent psychiatry at the University of Bern, chief physician and head of research at the University Hospital of Child and Adolescent Psychiatry in Bern. His main research interestests are ADHD, juvenile psychoses, and early detection of psychosis.

Johannes Hebebrand, MD, is a professor of child and adolescent psychiatry and psychotherapy, director of the Department of Child and Adolescent Psychiatry and Psychotherapy, University of Duisburg-Essen, and president of the German Society of Child and Adolescent Psychiatry, Psychosomatics, and Psychotherapy.

Winfried Rief, PhD, is a professor of clinical psychology and psychotherapy (chair) at University of Marburg, head of the outpatient clinic for psychological interventions, and head of the postgraduate training program in cognitive-behavior therapy at University of Marburg, Germany.

Hanna Christiansen, PhD, is a clinical child and adolescent psychologist whose main research interests are neuropsychology and treatment of ADHD, children of mentally ill parents, and prevention of mental disorders.

References

American Academy of Child & Adolescent Psychiatry. (2007). Practice parameter for the assessment and treatment of children and adolescents with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 894-921.

American Academy of Pediatrics. (2000). Clinical practice guideline: Diagnosis and evaluation of the child with attention-deficit/hyperactivity disorder. Pediatrics, 105, 1158-1170.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

Ballard

J. C.

(1996). Computerized assessment of sustained attention: A review of factors affecting vigilance performance. Journal of Clinical and Experimental Neuropsychology, 18, 843-863.

Barkley

R. A.

(1991). The ecological validity of laboratory and analogue assessment methods of ADHD symptoms. Journal of Abnormal Child Psychology, 19, 149-178.

Berry

C. A.

Shaywitz

S. E.

Shaywitz

B. A.

(1985). Girls with attention deficit disorder: A silent minority? A report on behavioral and cognitive characteristics. Pediatrics, 76, 801-809.

Brocki

K. C.

Tillman

C. M.

Bohlin

(2008). CPT performance, motor activity, and continuous relations to ADHD symptom domains: A developmental study. European Journal of Developmental Psychology, 7, 178-197.

Bühner

(2006). Einführung in die Test- und Fragebogenkonstruktion [An Introduction to Test- and Questionnaire Construction] (2nd ed.). München, Germany: Pearson Studium.

Bussing

Fernandez

Harwood

Wei

Garvan

C. W.

Eyberg

S. M.

Swanson

J. M.

(2008). Parent and teacher SNAP-IV ratings of attention deficit hyperactivity disorder symptoms. Assessment, 15, 317-328.

10.

Chen

Taylor

E. A.

(2006). Parental account of children’s symptoms (PACS), ADHD phenotypes and its application to molecular genetic studies. In Oades

R. D

(Ed.) Attention-deficit/hyperactivity disorder and hyperkinetic syndrome: current ideas and ways forward (pp. 3-20). Hauppauge, NY: Nova Science Publisher.

11.

Cohen

(1977). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.

12.

Cohen

(1988). Statistical power and analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

13.

Comrey

A. L.

Lee

H. B.

(1992). A first course in factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

14.

Conners

C. K.

(2008). Conners 3rd edition: Manual. Toronto, Ontario, Canada: Multi-Health Systems.

15.

Corkum

P. V.

Siegel

L. S.

(1993). Is the continuous performance task a valuable research tool for use with children with attention-deficit-hyperactivity disorder? Journal of Child Psychology and Psychiatry, 34, 1217-1239.

16.

Delmo

Weiffenbach

Gabriel

Stadler

Poustka

(2001). Übersetzung und Adaptation des Kiddie-Schedule for Affective Disorders and Schizophrenia-Present and Lifetime (K-SADS-PL). Retrieved from http://www.adhs-legasthenie.de/PDF/K-SADS_Fragebogen.pdf (Original work published 2000)

17.

DuPaul

G. J.

(1991). Parent and teacher ratings of ADHD symptoms: Psychometric properties in a community-based sample. Journal of Clinical Child Psychology, 20, 245-253.

18.

Edwards

Gardner

Chelonis

Schulz

Flake

Diaz

(2007). Estimates of the validity and utility of the Conners’ Continuous Performance Test in the assessment of inattentive and/or hyperactive-impulsive behaviors in children. Journal of Abnormal Child Psychology, 35, 393-404.

19.

Egeland

Kovalik-Gran

(2010a). Validity of the factor structure of Conners’™ CPT. Journal of Attention Disorders, 13, 347-357.

20.

Egeland

Kovalik-Gran

(2010b). Measuring several aspects of attention in One Test. Journal of Attention Disorders, 13, 339-346.

21.

Epstein

J. N.

Erkanli

Conners

C. K.

Klaric

Costello

J. E.

Angold

(2003). Relations between Continuous Performance Test Performance Measures and ADHD behaviors. Journal of Abnormal Child Psychology, 31, 543-554.

22.

Forbes

G. B.

(1998). Clinical utility of the test of variables of attention (TOVA) in the diagnosis of attention-deficit/hyperactivity disorder. Journal of Clinical Psychology, 54, 461-476.

23.

Gershon

(2002). A meta-analytic review of gender differences in ADHD. Journal of Attention Disorders, 5, 143-154.

24.

Gualtieri

C. T.

Johnson

L. G.

(2005). ADHD: Is objective diagnosis possible? Psychiatry (Edgmont), 2, 44-53.

25.

Günther

(2012). Questionnaires versus computer-based assessments to examine attentional processes. Poster presentation during the 3rd UK Paediatric Neuropsychology Symposium: Early Brain-Behaviour Relationships & Prognostic Indicators, London, UK.

26.

Halperin

J. M.

Newcorn

J. H.

Sharma

Healey

J. M.

Wolf

L. E.

Pascualvaca

D. M.

Schwartz

(1990). Inattentive and noninattentive ADHD children: Do they constitute a unitary group? Journal of Abnormal Child Psychology, 18, 437-449.

27.

Halperin

J. M.

Wolf

Greenblatt

E. R.

Young

(1991). Subtype analysis of commission errors on the continuous performance test in children. Developmental Neuropsychology, 7, 207-217.

28.

Hasson

Fine

J. G.

(2012). Gender differences among children with ADHD on Continuous Performance Tests. Journal of Attention Disorders, 16, 190-198.

29.

Kaiser

(1974). An index of factorial simplicity. Psychometrika, 39, 31-36.

30.

McGee

R. A.

Clark

S. E.

Symons

D. K.

(2000). Does the Conners’ Continuous Performance Test Aid in ADHD Diagnosis? Journal of Abnormal Child Psychology, 28, 415-424.

31.

Nichols

S. L.

Waschbusch

D. A.

(2004). A review of the validity of laboratory cognitive tasks used to assess symptoms of ADHD. Child Psychiatry & Human Development, 34, 297-315.

32.

Oades

R. D.

Dauvermann

M. R.

Schimmelmann

B. G.

Schwarz

M. J.

Myint

A. M.

(2010). Attention-deficit hyperactivity disorder (ADHD)and glial integrity: S100B, cytokines and kynurenine metabolism—Effects of medication. Behavioral and Brain Functions, 6, 29.

33.

O’Brien

J. D.

Halperin

J. M.

Newcorn

J. H.

Sharma

Wolf

Morganstein

(1992). Psychometric differentiation of conduct disorder and attention deficit disorder with hyperactivity. Journal of Developmental & Behavioral Pediatrics, 13, 274-277.

34.

Petermann

Wechsler

(2011). Wechsler Intelligence Scale for Children (WISC-IV) (4th ed., German Version). Frankfurt/M, Germany: Pearson.

35.

Polanczyk

de Lima

M. S.

Horta

B. L.

Biederman

Rohde

L. A.

(2007). The worldwide prevalence of ADHD: A systematic review and metaregression analysis. American Journal of Psychiatry, 164, 942-948.

36.

Rabiner

D. L.

Murray

D. W.

Rosen

Hardy

Skinner

Underwood

(2010). Instability in teacher ratings of children’s inattentive symptoms: Implications for the assessment of ADHD. Journal of Development & Behavioral Pediatrics, 31, 175-180.

37.

Reichenbach

L. C.

Halperin

J. M.

Sharma

Newcorn

J. H.

(1992). Children’s motor activity: Reliability and relationship to attention and behavior. Developmental Neuropsychology, 8, 87-97.

38.

Riccio

C. A.

Waldrop

J. J.

Reynolds

C. R.

Lowe

(2001). Effects of stimulants on the continuous performance test (CPT): Implications for CPT use and interpretation. Journal of Neuropsychiatry and Clinical Neurosciences, 13, 326-335.

39.

Sattler

J. M.

(1992). Assessment of children: behavioral and clinical applications. San Diego, CA: Sattler Publisher.

40.

Scholtens

Diamantopoulou

Tillman

C. M.

Rydell

A.-M.

(2011). Effects of symptoms of ADHD, ODD, and cognitive functioning on social acceptance and the positive illusory bias in children. Journal of Attention Disorders. Advance online publication.

41.

Stein

D. S.

Blum

N. J.

Barbaresi

W. J.

(2011). Developmental and behavioral disorders through the life span. Pediatrics, 128, 364-373.

42.

Taylor

Döpfner

Sergeant

Asherson

Banaschewski

Buitelaar

Zuddas

(2004). European clinical guidelines for hyperkinetic disorder—First upgrade. European Child & Adolescent Psychiatry, 13, i7-i30.

43.

Teicher

M. H.

Ito

Glod

C. A.

Barber

N. I.

(1996). Objective measurement of hyperactivity and attentional problems in ADHD. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 334-342.

44.

Tripp

Schaughency

E. A.

Clarke

(2006). Parent and Teacher Rating Scales in the evaluation of attention-deficit hyperactivity disorder: Contribution to diagnosis and differential diagnosis in clinically referred children. Journal of Developmental & Behavioral Pediatrics, 27, 209-218.

45.

Ulberstad

(2012). QbTest Technical Manual (rev. ed.). Stockholm, Sweden: Qbtech AB.

46.

Vogt

Williams

(2011). Early identification of stimulant treatment responders, partial responders and non-responders using objective measures in children and adolescents with hyperkinetic disorder. Child and Adolescent Mental Health, 16, 144-149.

47.

Wehmeier

P. M.

Dittmann

R. W.

Banaschewski

Schacht

(2012). Does stimulant pretreatment modify atomoxetine effects on core symptoms of ADHD in children assessed by quantitative measurement technology? Journal of Attention Disorders. Advance online publication.

48.

Wehmeier

P. M.

Schacht

Wolff

Otto

W. R.

Dittmann

R. W.

Banaschewski

(2011). Neuropsychological outcomes across the day in children with attention-deficit/hyperactivity disorder treated with atomoxetine: Results from a placebo-controlled study using a computer-based Continuous Performance Test combined with an infra-red motion-tracking device. Journal of Child and Adolescent Psychopharmacology, 21, 433-444.

49.

Willcutt

E. G.

Doyle

A. E.

Nigg

J. T.

Faraone

S. V.

Pennington

B. F.

(2005). Validity of the executive function theory of attention-deficit/hyperactivity disorder: A meta-analytic review. Biological Psychiatry, 57, 1336-1346.

50.

Zimmermann

Gondan

Fimm

(2002). Testbatterie zur Aufmerksamkeitsprüfung für Kinder (KITAP) [Test of Attentional Performance for Children]. Herzogenrath, Germany: Psytest.