Cognitive Testing to Identify Children With ADHD Who Do and Do Not Respond to Methylphenidate

Abstract

Objective: To explore the utility of cognitive measures for predicting response of children and adolescents to methylphenidate (MPH). Method: Participants from the International Study to Predict Optimized Treatment–in ADHD (iSPOT-A) completed a cognitive test battery prior to receiving 6 weeks of MPH. The responder criterion was a 25% reduction in ADHD-Rating Scale-IV scores. Receiver Operator Characteristics (ROC) classified non-responders from responders with maximal sensitivity and specificity. Results: Overall, 62% of participants responded to MPH. Response rates for ROC-identified groups ranged from 18% to 85%. Non-responders showed compromised cognition related to switching of attention, sustained attention, planning, and impulsivity. One group of responders were 10 years of age or older and had impaired switching of attention and impulsivity; a second group had enhanced switching of attention, normal or higher Continuous Performance Task (CPT) scores, and above average scores on digit span. Conclusion: Cognitive tests may provide a simple, low-cost tool for treatment planning for children and adolescents with ADHD.

Keywords

ADD/ADHD cognitive performance receiver operating characteristics methylphenidate

Introduction

ADHD adversely affects 3% to 10% of school-aged children (American Psychiatric Association, 2000; Polanczyk, Lima, Horta, Biederman, & Rohde, 2007). In a longitudinal study in the United States known as the National Survey of Children’s Health (NSCH), parent-reported ADHD diagnoses (lifetime) increased from 7.8% to 11% during 2003 to 2011 (Visser et al., 2013).

Children with ADHD experience deleterious cognitive problems, including increased risk of learning disability and communication disorders (Strine et al., 2006). The neurocognitive consequences of ADHD may account for the fact that children with ADHD are at increased risk of experiencing major physical injury and hospitalization (Strine et al., 2006). Data from international studies have indicated that attention deficit problems result in increased involvement in a motor vehicle crash, drinking and driving, and traffic violations (Barkley & Cox, 2007). Given the individual and social burden of ADHD and the availability of effective interventions, facilitating the identification of who will and will not respond to a specific treatment could speed the process of initiating effective intervention (Bernet et al., 2007).

Although a variety of interventions can reduce the key symptoms of ADHD, stimulant medications, including methylphenidate (MPH), are universally endorsed as the mainstay treatment for most children, adolescents, and adults (Sharma & Couture, 2014). Efficacy of MPH has been consistently demonstrated and replicated in multiple large-scale randomized controlled trials (RCTs; Greenhill et al., 2006; Greenhill, Findling, & Swanson, 2002; The MTA Cooperative Group, 1999) and community-based studies (Swanson et al., 1998). Between 2011 and 2012, the prescription of MPH in primary care settings in the United States increased by 1% (Visser et al., 2013). Between 2007 and 2013, prescriptions for ADHD among children between ages 4 and 17 increased by 28% (Visser et al., 2013).

Although MPH is effective in about two third of children and adolescents, individual response varies, with about one third not achieving adequate symptom control with an acceptable level of side effects (Greenhill et al., 1996; Kooij et al., 2010). The identification of individual children unlikely to respond to MPH has received little attention. Presently, these non-responders are identified through direct trial of a medication, an approach that fails to reduce their symptoms, exposes them to unnecessary side effects, and delays trials of other, potentially more beneficial interventions, often resulting in discouragement for the child and families. Using brief cognitive tests to identify persons who are quite unlikely to respond to MPH could speed up the process of their finding an effective treatment, whereas tests that identify those very likely to respond would provide much-needed assurance to patients and families. Hermens, Rowe, Gordon, and Williams (2006) described the value of finding ways to predict which children and adolescents would be stimulant-responsive. However, presently, no objective measures have been shown to identify, prior to medication usage, which children will or will not respond to MPH.

The availability of data from large, ongoing international, multi-site effectiveness trial of MPH for the treatment of ADHD (target sample N = 672) provides an opportunity to use signal detection analyses to explore whether non-responders can be pre-identified. The goal of this report was to use baseline and outcome data from the first cohort (50%) of International Study to Predict Optimized Treatment–in ADHD (iSPOT-A) to explore which participant characteristics, including baseline scores on objective tests of cognitive and emotional processes, would most efficiently identify individuals who would and would not respond to MPH.

Method

Participants were from iSPOT-A, a larger ongoing effectiveness study (target N = 672), and comprised the first cohort of children and adolescents (n = 336; 50%).

Overview of the iSPOT-A Study

The iSPOT-A is a Phase IV, multi-site, international, open-label effectiveness trial that aims to identify (a) characteristics that differentiate ADHD compared with healthy controls and (b) objective biological and cognitive markers that best predict treatment response to MPH. Using standardized objective measures of cognitive, brain activity and structure, and genetics, treatment naïve or washed-out children and adolescents were assessed un-medicated (at baseline) and again 6 weeks later on treatment. Seven research sites were selected to take part in the study; one specialized in academic research and six in a clinical practice setting. The trial has global study Principal Investigator (from Sponsor) and each site has its own Principal Investigator. Sites were located in the United States, Australia, and the Netherlands, with a Global Coordinating Center and Data Center located in Sydney, Australia.

While iSPOT-A is being conducted as a single study (ACTRN 12609000271202), it has a two-step data analytic procedure. Specifically, findings emerging from tests of the first half of the sample will be assessed for replication in the second half. The current study analyzed study participants from the first cohort who met criteria for ADHD (n = 336). Study enrollment commenced in September 2009; the first 50% of the sample was completed in April 2012.

The study complies with the principles of the “Declaration of Helsinki 2008,” the international Conference on Harmonization (ICH) guidelines and the laws and regulations of the country in which the research is conducted, including the principles of “Good Clinical Practice” as outlined in the U.S. Code of Federal Regulations. All sites received approval from their Institutional Review Board (IRB)/Independent Ethics Committee (IEC) prior to participant enrollment and each participant (and/or guardian) provided their informed consent to be involved in the study.

Study Participants

Enrollment and screening

Potential participants are recruited via pediatrician, other clinical referrals, or through general community advertisements (e.g., from university or clinical trial websites, flyers, word-of-mouth). The ADHD participant (and guardian/parent when less than 16 years) complete an initial telephone screening interview to see if the child or adolescent meets the inclusion and exclusion criteria (see Table 1). Those appearing to meet the inclusion/exclusion criteria are invited to attend a baseline visit where their eligibility is confirmed.

Table 1.

Inclusion and Exclusion Criteria.

Inclusion criteria
• Meet DSM-IV criteria for ADHD (primary diagnosis), as determined by a psychiatrist, physician, or clinical psychologist, in conjunction with the clinical workup undertaken at study entry using the MINI Kid • Score 6 or more items >1 on the Inattentive or Hyperactive/Impulsive subscales of the ADHD-RS-IV • No stimulant medication in the 7 days prior to baseline testing • Six to 17 years of age • Fluent and literate in English (and/or Dutch in The Netherlands)
Exclusion criteria
• Known contra-indication or intolerance to the use of methylphenidate as defined in the product package insert (including previous treatment failure at the highest recommended dose) • Pregnancy or females of child bearing potential not using contraception • Known medical condition, disease or neurological disorder which might, in the opinion of investigator(s), interfere with assessments in the study or put ADHD participants at increased risk when exposed to optimal doses of the drug treatment • History of physical brain injury or blow to the head that resulted in loss of consciousness for 5 min within the last 2 years, or for at least 15 min longer than 2 years ago • Known past or present substance dependence, including alcohol as determined by the MINI Kid • Participation in an investigational study within 4 months of the baseline visit in which participants have received an experimental drug/device that could affect the primary end points of this study. • Use of any psychological or counseling therapy, or central nervous system (CNS) medication that cannot be washed out prior to baseline visit, or use of any psychological or counseling therapy between baseline and follow-up Week 6 visit. • Participants who, in the opinion of the investigator, have a severe impediment to vision, hearing, and/or hand movement, which is likely to interfere with their ability to complete the testing batteries. • Participants who, in the opinion of the investigator, are unable and/or unlikely to comprehend and follow the study procedures and instructions. • Presence of any other co-morbid primary DSM-IV disorder.

Note. DSM-IV = Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994); MINI Kid = Mini International Neuropsychiatric Interview for Children and Adolescents; ADHD-RS-IV = ADHD-Rating Scale-IV.

The broad inclusion and minimal exclusion criteria were used to recruit a representative sample of the general ADHD population who typically receive MPH in routine practice, via pediatrician, family practitioner, internist, or other physician.

At baseline, after providing a signed consent and prior to enrollment, the inclusion/exclusion criteria were re-confirmed, including a face-to-face assessment of Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) diagnoses (using the Mini International Neuropsychiatric Interview for Children and Adolescents [MINI Kid]), and ADHD-Rating Scale-IV (ADHD-RS-IV). In addition (at the site investigator’s discretion), at baseline a QuickTox® Drug Screen and SAS Pregnancy Urine One Test are used to rule out potential illicit drug use or pregnancy. If all criteria are met (as per Table 1), the participant is enrolled and further clinical and demographic data are collected as well as the baseline standardized objective measures of cognition brain and genetics.

Procedure

Treatment delivery and compliance monitoring

ADHD participants are either treatment naïve or washed out before baseline, following recommendations on the package insert. ADHD participants are prescribed open-label MPH by their treating pediatrician. The open-label treatment design is intended to maximize participant safety and is consistent with the study’s naturalistic methodology.

Participants continue treatment with MPH until Week 6 and are required to have a minimum duration of treatment of 4 weeks), while refraining from any other ADHD treatments, including other stimulants, non-stimulant ADHD drugs, and non-pharmacological ADHD therapies such as counseling or behavior therapies.

Measures

Demographic variables were assessed at baseline, and clinical and cognitive measures were evaluated at Baseline (Week 0) and Week 6 (post-treatment). The current analysis uses only the demographic and cognitive measures at baseline and the change in clinical symptoms between baseline and Week 6.

ADHD diagnosis, comorbidities, and symptom severity

ADHD diagnosis was assessed using the ADHD-RS-IV and the MINI Kid. The MINI Kid was used to identify other current and lifetime psychiatric comorbidities. ADHD symptom severity was assessed using the ADHD-RS-IV (clinician rated, with higher numbers indicating greater pathology) and the Conners’ Parent Rating Scale–Revised: Long Version (CPRS-R: L; parent rated with higher numbers indicating greater pathology). Both scales were administered at baseline and at Week 6. Inter-rater reliability training for administration of the ADHD-RS-IV was provided by the Global Trial Manager to all principal investigators and research staff performing assessments. Participating clinicians received training and were assessed using video-recordings of a simulated consultation between a clinician and a parent describing a child’s symptoms. Symptom response by Week 6 was defined a priori as a reduction in symptoms on the ADHD-RS-IV of 25% or greater.

Cognitive measures

Ten cognitive tasks, summarized in Table 2, were administered to participants at study baseline and at study endpoint (Week 6) using the validated computer-based, touch-screen test battery, “IntegNeuro^TM” (Brain Resource Ltd.; see Williams et al., 2009, for a detailed description). In this 30- to 45-min test battery, standardized, pre-recorded task instructions are concurrently presented visually on the screen and audibly through headphones. Motor Tapping assessed basic motor function, fine movement speed, and manual dexterity (Gill, Reddon, Stefanyk, & Hans, 1986). Choice Reaction Time assessed basic sensory-motor functions, visual-motor coordination, information processing speed, speed-accuracy trade-off, and mapping of stimulus identification to the appropriate response (Adam et al., 1999). Memory Recall tested immediate and delayed recall scores (i.e., the number of words correctly recalled across the learning trials and the delayed trial), assessing verbal learning, memory recall, and verbal self-monitoring (Crossen & Wiens, 1994). Digit Span evaluated immediate recall and assessed the ability to hold, retain, and manipulate new verbal information online (Groth-Marnat & Baker, 2003). Verbal Interference (Word, Color) measured the ability to inhibit inappropriate, well-learned, impulsive, automatic responses (Sacks, Clark, Pols, & Geffen, 1991). Switching of Attention (Digits + Letters) evaluated the ability to sustain and control the direction of attention and to flexibly switch attention from one over-learned task to another (O’Donnell, MacGregor, Dabrowski, Oestreicher, & Romero, 1994). Continuous Performance Test (n-back) assessed the ability to maintain sustained attention and inhibit impulsive responding over an extended period. In addition, the task assessed target detection, and the ability to update information held in short-term memory (Borgaro et al., 2003). Go/No-Go assessed executive functioning and cognitive inhibition, or the ability to suppress well-learned, automatic responses (Logan & Cowan, 1984). Maze task measured how quickly a participant learned the route through the maze and their ability to remember that route. This task involves executive functioning and planning; the ability to choose, try, reject, and adapt alternative courses of thought and action; and visuospatial learning and memory (Bowden, 1989). Emotion Identification measured emotional recognition and discrimination between emotions (Mathersul et al., 2009; Williams et al., 2009).

Table 2.

Forty-Five Minute Battery of Computerized Tests of Cognitive and Emotional Processes.

Task name	Cognitive capacity assessed	Task description
Motor Tapping	Motor coordinations	Participants repeatedly tap a circle on the screen with the index finger of their dominant hand as fast as possible for 60 sec, then repeat with their non-dominant hand. Total task duration is approximately 2 min.
Choice Reaction Time	Processing speed	Participants touch one of four circles that lights up, as fast as possible. There are 20 pseudo-random trials with a random delay of 2 to 4 sec following the previous response. Task duration is approximately 2 min
Memory Recall	Recall memory	Words are presented aurally through headphones, and participants recall the words by speaking into a microphone. Immediate recall is assessed over 4 consecutive repeated trials of a list of 12 words being presented then recalled. Interference is assessed through a fifth trial with new words and a sixth trial of the original 12 word list immediately following. Delayed recall of the original list is assessed after 20 min. Total task duration is approximately 11 min.
Digit Span	Working memory	Participants recall in sequential order a series of digits that are presented one at time on the screen, using a 9-digit touchpad that appears after the list is presented. The sequence length begins at 3 digits and increases progressively throughout the task to a maximum of 9 digits. Maximum task duration is approximately 6 min.
Verbal Interference	Controlled attention	Colored words with incongruent color–word combinations are presented on the screen. Participants are required to first identify the name (Part 1) and then the color (Part 2) of each word. Assesses aspects of inhibition and interference comparable with those indexed by the Stroop test. Task duration is approximately 2 min.
Switching of Attention	Flexibility	13 digits (1-13) and 12 letters (A-L) are presented spatially across the screen. Participants select responses in ascending sequential order, alternating between digits and letters (e.g., 1-A-2-B). Task duration is approximately 3 min.
Continuous Performance Test	Sustained attention	An n-back continuous performance task in which one of 4 letters (B, C, D, G) is presented one at time in pseudo-random sequence, and participants respond when the same letter appears twice in row (tapping a circle on the screen). Task duration is approximately 6 min.
Go/No-Go	Inhibition	Participants respond as quickly as possible to the word “press” when it is presented in GREEN, and inhibit this response when presented in RED. Participants tap a circle on the screen to respond. Task duration is approximately 4 min.
Maze	Executive function	Participants are required to identify by trial and error a hidden path within an 8 × 8 grid of circles, with a total of 24 correct moves required to move from start to finish. Task ends with two consecutive error-free path completions, or times out after 10 min.
Explicit Emotion Identification	Emotion identification	Participants identify the emotional expression of faces presented on the screen, selecting 1 of 6 word labels presented below the face (Happy, Fear, Sad, Anger, Disgust, Neutral), with 48 faces in total (8 of each emotion). Task duration is approximately 4 min.

A total of 48 individual measures derived from these 10 tasks were included in the current analysis, using standardized z-scores for each task measure, derived from the normative population scores that form part of the test.

Responder criterion

The primary outcome variable was non-response, based on change scores from baseline to Week 6 on the ADHD-RS-IV. Participants who had less than 25% reduction in baseline ADHD-RS-IV total score were deemed to be non-responders.

Data Analysis

Receiver operator characteristics (ROC) analyses, based on signal detection methods (Kraemer et al., 1999), were used to identify which variables, and at what level or score, optimally discriminate non-responders and responders. In this recursive partitioning procedure, cut-points on tests are identified that discriminate non-responders from responders at p < 0.01. A kappa statistic is calculated for each cut-point, and the largest kappa coefficients correspond to cut-points with maximum sensitivity and specificity (Kraemer et al., 1999; QROC freeware available at mirecc.stanford.edu). This recursive partitioning method uses the patients’ cognitive test results to identify subgroups with significantly increased or decreased probability of MPH response. Subgroups wherein the proportion of responders differed from the overall response rate by at least 10 percentage points either direction (i.e., 72% or greater = elevated response likelihood; 52% or lower = decreased response likelihood) were further described. This threshold was chosen as it represents a clinically meaningful difference from the overall group rate with a number needed to treat of 10 or less (i.e., a moderate effect size). Participants were included if they had missing data on any of the cognitive tests but were excluded if they did not have observed outcome data on the ADHD-RS-IV.

Results

Table 3 summarizes the features of the sample (n = 336). Of the total sample who completed the Week 6 follow-up assessment (n = 284), 62% (n = 176) met the criterion for response to MPH (i.e., >25% reduction in baseline ADHD-RS-IV scores by Week 6 endpoint assessment). Details of the reasons for those who did not complete the follow-up assessment are provided in Figure 1.

Table 3.

Baseline Characteristics of the Study Sample (Total N = 284).

Baseline characteristic	M	SD
Age	11.9	3.2
ADHD-RS-IV Scale Total score (sum of Items 1-18)	36.9	10.2
Conners’ Parent CGI Total Subscale (T-score)	73.2	12.6
Baseline characteristic	n	% of sample
Gender (number of males)	208	73
ADHD-RS-IV combined subtype	161	567
ADHD-RS-IV inattentive subtype	98	34
ADHD-RS-IV hyperactive subtype	7	2
Below criteria on ADHD-RS-IV	18	6

Note. Data above reflect the sample of, who completed a Week 6 follow-up visit.

Figure 1.

Consort diagram.

Among all participants, 57% reported prior (lifetime) use of stimulant medication to treat ADHD (although washed out at baseline if they had been recently using stimulant mediation). Previous use of stimulant medication was not related to response status at Week 6 (χ² = 1.8, df = 1, p > .05).

Classification of Responders and Non-Responders via ROC

Of the 11 tasks (46 scores) in the cognitive test battery and 2 demographic measures of gender and age, 5 cognitive tasks were identified in the ROC tree as significant discriminators of responders and non-responders: the Switching of Attention accuracy, reaction time on the Continuous Performance Test (n-back), time to complete first Executive Maze run without error, Verbal Interference reaction time, and Digit Span number of correct trials. As shown in Figure 2, seven total subgroups of patients were identified in the ROC classification tree, and these groups varied in their response to MPH. Among three groups of patients (n = 28, n = 25, n = 20), defined by their profile of responses on the cognitive test battery, the proportions of responders to MPH were significantly lower (18%, 44%, and 40%) than the overall response rate of 62% (see Figure 2, gray boxes with dashed outline). Among two groups of patients (n = 20 and n = 70), the proportions of responders to MPH were significantly higher (85% and 83%) than the overall response rate (see Figure 2, boxes with bolded outline). Another two patient groups were identified that had responder rates comparable with the overall sample (69% and 60%; n = 13 and n = 78).

Figure 2.

Classification of responders and non-responders by scores on the cognitive test battery at baseline.

Patients With Lower Response Rates to MPH

Based on patient responses to the cognitive battery, ROC identified three groups of participants (n = 28, n = 25, and n = 20) with specific cognitive profiles and a significantly low proportion of response to MPH. Of the three groups, two had poor accuracy in switching of attention; their normative scores on the switching of attention accuracy test were less than the 22th percentile. Among these children and adolescents (n = 88), those below 10 years old had a low proportion of responders (i.e., 33%). This response rate was further attenuated among these younger patients who had poor planning implementation, indicated by Maze score results below the 14th percentile. Among all study participants, this group had the lowest proportion of responders—18% (see Figure 2). Children and adolescents who had poor switching of attention and were above 10 years old were identified as having lower likelihood of response to MPH—if they also scored in the normal to high range on the verbal interference task. Children older than 10 with verbal interference reaction times ≥34th percentile relative to the normative population had a response rate of 44%. The third group of patients identified as low responders using ROC was identified among the patients with normal or high switching of attention accuracy: but very low sustained attention scores (n = 20; Continuous Performance Test [CPT] scores <.5th percentile). Only 40% met responder criterion to MPH (see Figure 2, right side). For the three groups with lower proportion of responders, the three distinct cognitive profiles are shown in boxes (dashed outlines) in Figure 2.

Patients With Higher Response Rates to MPH

The ROC signal detection identified two groups of children and adolescents with a significantly greater response rate to MPH. Eighty-five percent of participants responded to MPH if they were above 10 years of age, had low cognitive flexibility as measured on the switching of attention task—plus poor verbal interference reaction times, a measure of impulsivity. A second profile was identified that corresponded with a high response rate (83%): participants with normal to above-normal switching of attention scores, normal to above normal CPT scores, and above average scores on the digit span task. Among these patients (n = 70), being accurate, fast, and having good working memory was associated with high response to MPH.

The resulting ROC classification tree identified two group profiles with significantly greater than average response rates: these two groups comprised 32% (98/284) of the sample. The ROC also identified 3 groups with significantly less than average or significantly less than average response rates: these three groups comprised 24% (73/284) of the total sample. Using the cut-points yielded in the ROC to identify non-responders, the specificity of the cognitive tests was 77% and a negative predictive value of 67%. For responders, the model yielded a sensitivity value of 76% and positive predictive value of 83%.

Table 4 provides a comparison of the low response groups and high response groups on other parameters measured in the study.

Table 4.

Low and High Response Groups: Demographic and Clinical Characteristics.

	Responder group			Low vs. high		Average vs. high
	Low responders^a	Average responders	High responders^a	t or χ²	p	t or χ²	p
N	74	91	90
M age in years (SD)	11.3 (3.1)	11.8 (3.2)	12.9 (2.9)	−3.22	.002	−2.23	.02
% male	73.0	75.8	71.1	0.07	.79	0.52	.47
Baseline M ADHD-RS-IV^b (SD)	37.4 (11)	35.6 (10)	36.6 (10)	0.53	.60	−0.63	.53
Baseline M Conners’ CGI^c (SD)	72.4 (13)	71.9 (13)	74.7 (12)	−1.18	.24	−1.55	.12
Combined or hyperactive subtype	64%	54%	56%	1.06	.30	0.53	.82
Inattentive subtype	30%	37%	39%	1.50	.22	0.45	.83
M dose at 6 weeks (SD)	21.6 (16)	22.8 (12)	25.1 (17)	−1.36	.18	−1.04	.30
Proportion of responders	32%	62%	83%		n/a		n/a

Low responders = children and adolescents with response rates 10% or less than the base response rate (62%); high responders = children and adolescents with response rates 10% or higher than the base response rate (62%).

ADHD-RS-IV = ADHD-Rating Scale-IV.

Conners’ CGI = Conners’ Global Index (Conners’ Parent Rating Scale).

Conclusion

In this large effectiveness study of the effects of MPH on ADHD symptoms, the response to MPH was 62%, which is comparable with the literature (Kooij et al., 2010). This current exploratory data analysis was conducted with data from a large, ongoing effectiveness study, in a planned first phase to develop hypotheses as to which cognitive measures might play a pivotal role in identifying individual participants who would fare particularly well or poorly with MPH. Signal detection methods and ROC analyses were chosen for this planned hypothesis-generation that will guide subsequent, formal hypothesis testing.

Specifically, this study revealed that a select number of baseline responses to a computerized cognitive test battery identified individuals who would and would not respond to a 6 week trial of MPH. Responses to the tasks of switching of attention, working memory, verbal interference, and mazes efficiently assorted the treatment response to MPH for a large percentage of children and adolescents. Non-responders had compromised cognition, including switching of attention, sustained attention, planning, and impulsivity. Responders showed intact cognition, particularly in speed and accuracy, switching of attention, sustained attention, and working memory. Response profiles yielded response rates ranging from 18% to 85%. Altogether, more than half of the sample could be placed into the much more or much less likely to respond groups defined a priori as entailing an effect size (Number Needed to Treat) of 10 or less.

Four of the most common findings in previous ADHD research have shown that individuals with ADHD exhibit (a) deficits in sustained attention, measured herein using the CPT (Barkley, 1997; Williams et al., 2010); (b) working memory (via digit span; Barkley, 1997); (c) impulsivity (reflected in verbal interference reaction times; Lansbergen, Kenemans, & van Engeland, 2007; Williams et al., 2010); and (d) frontal executive inefficiencies (Barkley, 1997), exemplified by maze errors as well as switching-of-attention inaccuracies. The current results add to literature by showing that these symptoms may be predictors of MPH response and may extend our ability to formulate and apply this information so as to better inform treatment planning. They also offer clues to the neurocircuitry or neuropathphysiology that underlies ADHD.

The switching of attention task, which was the first variable identified to discriminate responders and non-responders, requires holding “on-line” two sets of information at the same time and alternately switching between them, utilizing abilities of cognitive flexibility and executive function, both of which are deficits associated with ADHD (Brown, 2008; Brown, Reichel, & Quinlan, 2009). Greater errors on this task reflect a deficit in this ability, reflecting the continuation of using the same set of information, instead of switching to the alternate set of information.

The CPT (n-back) assesses the capacity to sustain attention over a prolonged period of time, and requires watching a series of letters appear on the screen, remembering each time what the previous letter was, to be able to respond when the same letter appears twice in a row. This is a classic task on which ADHD patients perform poorly in comparison with healthy controls (Williams et al., 2010). Lower accuracy and slower response times reflect poorer attention skills.

The verbal interference task assesses controlled attention constructs similar to the Stroop test. It requires reading a color name word, and responding with the name of color font the word is written in, while inhibiting the more automatic response of the word being read. Lower accuracy and slower response times on correct responses reflect more difficultly inhibiting automatic responses. Inhibiting automatic behavior is a core deficit in ADHD.

Regarding the maze, higher scores reflect more time before successful completion on the maze, suggesting compromised executive functioning and planning—cognitive functions that have not been shown to be especially stimulant-responsive. Furthermore, higher scores indicate difficulty in choosing, trying, rejecting, and adapting to alternative courses of thought and action. In addition to the obvious social and academic consequences of such problems, the current study suggests that these participants have a markedly lower probability of responding to MPH, currently the mainstay treatment for ADHD.

The study has several limitations. Most important is the inherent limitation to exploratory data analyses—namely, the mandate for replication of observed results. To generate the optimal classification of non-responders and responders, multiple tests were conducted, inflating Type 1 error risk. The use of alpha of .01 for each test provided some, albeit limited, protection from this concern. On the other hand, a strength of this study is that the initial cohort was used herein to generate and hone hypotheses. With completion of the larger iSPOT-A trial (target N = 672), formal hypothesis testing can be conducted. Additional limitations include generalizability, which although addressed by inclusive criteria or study entry, could still be challenged on the limited number and types of enrolling sites. A third issue is a non-blinded outcome assessment. Although pretreatment test results were not provided to the treating persons or raters, raters knew that all participants were being treated and thus may have been biased to call overrate the benefits of treatment. This measurement error, however, might be expected to introduce noise and thus limit the precision of the predictors being evaluated. Despite this limitation, there was a selection of a limited number of baseline features with an effect that was clinically meaningful. The inclusion of a placebo control was considered but rejected, as we were not asking how to differentiate placebo from MPH responders, which is an important question in itself. Rather, we wanted to determine whether, in a representative sample of patients, we could identify measures that would inform decision making by either identifying those with a higher or lower likelihood of responding, and in so doing create the hypotheses to be tested in the next phase of the trail.

Other study strengths include the application of ROC, which can evaluate a large number of potential predictors of any binary outcome such as responder/non-responder. In this study, 75 variables were tested. Unlike standard analytic tools (e.g., general linear models, logistic regression), ROC methods are not mathematically compromised by issues such as multi-collinearity (i.e., highly correlated predictor variables) or the distributions of the residuals (Kiernan, Kraemer, Winkleby, King, & Taylor, 2001). Future researchers, particularly those wanting to test multiple predictors, moderators, and mediators of outcomes from big data studies, could utilize ROC or other classification tools as a transparent method for reducing and honing hypotheses that can be tested in other cohorts or samples.

These results, if replicated in the next phase in this study, have important clinical implications. Availability of a low-cost, brief, easily administered test battery may help identify children and adolescents who are markedly less likely to have a positive response to MPH or possibly other similar medications. These children could avoid unnecessary side effects in the absence of therapeutic benefit and might benefit from exploration of alternative treatments. Conversely, identification of those with a rather high likelihood of benefit provides treatment selection guidance and reassurance to families facing these treatment choices.

In conclusion, this article has identified four brief feasible cognitive tests that appear to reliably identify more than half of the children and adolescents who have either a far higher or far lower than average chance of responding acutely to MPH. If replicated, these findings provide an important step in personalizing the treatment of ADHD.

Footnotes

Acknowledgements

We gratefully acknowledge the contributions of the site PIs and collaborators in this study, with particular acknowledgment to Martijn Arns, Kamran Fallahpour, Lea Williams, Roger DeBeus, Barbara Cohen, Harbans Multani, and Mona Ismail. We also acknowledge the monitoring support of the central iSPOT-A management team and PhaseForward.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was funded entirely by Brain Resource Group (BRG). With the exception of the first author, the authors are employees or consultants to BRG.

Author Biographies

Glen R. Elliott, PhD, MD, is chief psychiatrist for the Children’s Health Counsel and associate training director for Child and Adolescent Psychiatry at Stanford School of Medicine. He has a longstanding interest in the appropriate use of psychoactive medications in children and adolescents with severe psychiatric disorders.

Christine Blasey is a research psychologist and consulting biostatistician at Stanford University School of Medicine, Department of Psychiatry and professor of research design and statistics at Palo Alto University.

William Rekshan is a senior biostatistician at Brain Resource Ltd.

A. John Rush is emeritus professor at Duke-National University of Singapore. He serves as a scientific adviser to Brain Resource Ltd and to other commercial and governmental entities.

Donna M. Palmer is science manager at Brain Resource Ltd and a postdoctoral research affiliate of the Brain Dynamics Center, Westmead Millennium Institute and Sydney Medical School, University of Sydney.

Simon Clarke is a clinical associate professor of pediatrics and child health and is the head of the Department of Adolescent Medicine, Westmead Hospital. He is also an affiliate of the Centre for Research Into Adolescents’ Health, Westmead.

Michael Kohn is clinical associate professor of pediatrics and child health at The Children’s Hospital at Westmead and the Centre for Research Into Adolescents’ Health, Westmead.

Craig Kaplan is a statistician at Brain Resource Ltd.

Evian Gordon is founder and CEO of Brain Resource Ltd (2002), the company that created the first international database on the human brain across clinical, cognitive, imaging, and genetic information, in thousands of subjects aged six to 100 and in over 10 disorders. He also established the Brain Dynamics Center in 1982 and this interdisciplinary approach at Westmead Hospital, Sydney, Australia. The standardized and integrative methodology and database are also used in , a not-for-profit foundation based in California that has over 350 scientists from 28 countries who have access to freely publish this data in any manner.

References

Adam

J. J.

Paas

F. G.

Buekers

M. J.

Wuyts

I. J.

Spijkers

W. A.

Wallmeyer

(1999). Gender differences in choice reaction time: Evidence for differential strategies. Ergonomics, 42, 327-335.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

Barkley

R. A.

(1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121, 65-94.

Barkley

R. A.

Cox

(2007). A review of driving risks and impairments associated with attention-deficit/hyperactivity disorder and the effects of stimulant medication on driving performance. Journal of Safety Research, 38, 113-128.

Bernet

Bukstein

Walter

H. J.

Arnold

Beitchman

Benson

R. S.

. . .Jaselskis

(2007). Practice parameter for the assessment and treatment of children and adolescents with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 894-921.

Borgaro

Pogge

D. L.

DeLuca

V. A.

Bilginer

Stokes

Harvey

P. D.

(2003). Convergence of different versions of the Continuous Performance Test: Clinical and scientific implications. Journal of Clinical and Experimental Neuropsychology, 25, 283-292.

Bowden

S. C.

(1989). Maze learning: Reliability and equivalence of alternate pathways. Clinical Neuropsychologist, 3, 137-144.

Brown

T. E.

(2008). ADD/ADHD and impaired executive function in clinical practice. Current Psychiatry Reports, 10, 407-411.

10.

Brown

T. E.

Reichel

P. C.

Quinlan

D. M.

(2009). Executive function impairments in high IQ adults with ADHD. Journal of Attention Disorders, 13, 161-167.

11.

Crossen

J. R.

Wiens

A. N.

(1994). Comparison of the Auditory Verbal Learning Test (AVLT) and California Verbal Learning Test (CVLT) in a sample of normal participants. Journal of Clinical and Experimental Neuropsychology, 16, 190-194.

12.

Gill

D. M.

Reddon

J. R.

Stefanyk

W. O.

Hans

H. S.

(1986). Finger tapping: Effects of trials and sessions. Perceptual & Motor Skills, 62, 675-678.

13.

Greenhill

L. L.

Abikoff

H. B.

Arnold

L. E.

Cantwell

D. P.

Conners

C. K.

Elliott

. . .Wells

(1996). Medication treatment strategies in the MTA Study: Relevance to clinicians and researchers. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 1304-1313.

14.

Greenhill

L. L.

Findling

R. L.

Swanson

J. M.

(2002). A double-blind, placebo-controlled study of modified-release methylphenidate in children with attention-deficit/hyperactivity disorder. Pediatrics, 109, e39.

15.

Greenhill

Kollins

Abikoff

McCracken

Riddle

Swanson

. . .Cooper

(2006). Efficacy and safety of immediate-release methylphenidate treatment for preschoolers with ADHD. Journal of the American Academy of Child amd Adolescent Psychiatry 45, 1284-1293.

16.

Groth-Marnat

Baker

(2003). Digit span as a measure of everyday attention: A study of ecological validity. Perceptual & Motor Skills, 97, 1209-1218.

17.

Hermens

D. F.

Rowe

D. L.

Gordon

Williams

L. M.

(2006). Integrative neuroscience approach to predict ADHD stimulant response. Expert Reviews in Neurotherapy, 6, 753-763.

18.

Kiernan

Kraemer

H. C.

Winkleby

M. A.

King

A. C.

Taylor

C. B.

(2001). Do logistic regression and signal detection identify different subgroups at risk? Implications for the design of tailored interventions. Psychological Methods, 6, 35-48.

19.

Kooij

S. J.

Bejerot

Blackwell

Caci

Casas-Brugué

Carpentier

P. J.

. . .Asherson

(2010). European consensus statement on diagnosis and treatment of adult ADHD: The European network adult ADHD. BMC Psychiatry, 10, Article 67.

20.

Kraemer

H. C.

Kazdin

A. E.

Offord

D. R.

Kessler

R. C.

Jensen

P. S.

Kupfer

D. J.

(1999). Measuring the potency of risk factors for clinical or policy significance. Psychological Methods, 4, 257-271.

21.

Lansbergen

M. M.

Kenemans

J. L.

van Engeland

(2007). Stroop interference and attention-deficit/hyperactivity disorder: A review and meta-analysis. Neuropsychology 21, 251-262.

22.

Logan

G. D.

Cowan

W. B.

(1984). On the ability to inhibit thought and action: A theory of an act of control. Psychological Review, 91, 295-327.

23.

Mathersul

Palmer

D. M.

Gur

R. C.

Gur

R. E.

Cooper

Gordon

Williams

L. M.

(2009). Explicit identification and implicit recognition of facial emotions: II. Core domains and relationships with general cognition. Journal of Clinical and Experimental Neuropsychology, 31, 278-291.

24.

O’Donnell

J. P.

MacGregor

L. A.

Dabrowski

J. J.

Oestreicher

J. M.

Romero

J. J.

(1994). Construct validity of neuropsychological tests of conceptual and attentional abilities. Journal of Clinical Psychology, 50, 596-600.

25.

Polanczyk

Lima

M. S.

Horta

B. L.

Biederman

Rohde

L. A.

(2007). The worldwide prevalence of ADHD: A systematic review and metaregression analysis. American Journal of Psychiatry, 164, 942-948.

26.

Sacks

T. L.

Clark

C. R.

Pols

Geffen

L. B.

(1991). Comparability and stability of performance on six alternate forms of the Dodrill-Stroop Color-Word Test. The Clinical Neuropsychologist, 5, 220-225.

27.

Sharma

Couture

(2014). A review of the pathophysiology, etiology, and treatment of attention-deficit hyperactivity disorder (ADHD). The Annals of Pharmacotherapy, 48, 209-215.

28.

Strine

T. W.

Lesesne

C. A.

Okoro

C. A.

McGuire

L. C.

Chapman

D. P.

Balluz

L. S.

Mokdad

A. H.

(2006). Emotional and behavioral difficulties and impairments in everyday functioning among children with a history of attention-deficit/hyperactivity disorder. Preventing Chronic Disease, 3(2), Article 52.

29.

Swanson

Sergeant

Taylor

Sonuga-Barke

Jensen

Cantwell

(1998). Attention-deficit hyperactivity disorder and hyperkinetic disorder. The Lancet, 351, 429-433.

30.

The MTA Cooperative Group. (1999). A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder. Archives of General Psychiatry, 56, 1073-1086.

31.

Visser

S. N.

Danielson

M. L.

Bitsko

R. H.

Holbrook

J. R.

Kogan

M. D.

Ghandour

R. M.

. . .Blumberg

S. J.

(2013). Trends in the parent-report of health care provider-diagnosed and medicated attention-deficit/hyperactivity disorder: United States, 2003–2011. Journal of the American Academy of Child & Adolescent Psychiatry, 53, 34-46.

32.

Williams

L. M.

Hermens

D. F.

Thein

Clark

C. R.

Cooper

N. J.

Clarke

S. D.

. . .Kohn

M. R.

(2010). Using brain-based cognitive measures to support clinical decisions in ADHD. Pediatric Neurology, 42, 118-126.

33.

Williams

L. M.

Mathersul

Palmer

D. M.

Gur

R. C.

Gur

R. E.

Gordon

(2009). Explicit identification and implicit recognition of facial emotions: I. Age effects in males and females across 10 decades. Journal of Clinical and Experimental Neuropsychology, 31, 257-277.