Abstract
Background and purpose:
Attention-deficit/hyperactivity disorder (ADHD) is a highly prevalent disorder that can respond dramatically to medication, if dose is appropriately titrated. Studies suggest that computer measures of attention cannot be used for titration as they show improvement on doses too low to produce clinical benefits. We assessed whether measures of motor activity and attention using the McLean Motion Attention Test (M-MAT™) could identify doses associated with optimal clinical response.
Methods:
Eleven boys (9.6 ± 1.8 years), receiving treatment with methylphenidate, and meeting DSM-IV criteria for ADHD, participated in this triple-blind (parent, child, rater), within-subject, efficacy study. Subjects received 1 week each of placebo, low (0.4 mg/kg), medium (0.8 mg/kg), and high (1.5 mg/kg) daily doses of methylphenidate. Parents rated response using an index of clinical global improvement.
Results:
In 9/11 subjects, the dose that produced the best improvement on M-MAT™ measures was also the dose that produced the best clinical outcome (p < 10−5). Parents rated response to this dose significantly better than response to previously prescribed treatment. Objective measures of primarily activity and secondarily attention responded to treatment in a manner concordant with clinical ratings, suggesting that these measures have ecological validity, and the potential to facilitate medication management and titration.
Introduction
A potential consequence of the mediocre response obtained with conventional community care is poor compliance. Based on pharmacy records, Sanchez et al. found that 50% to 63% of children with ADHD (n = 9549) ceased receiving treatment during a single school year (Sanchez et al. 2005). Similarly, Perwien and colleagues reported that 85% of children (n = 735) and 88% of adults (n = 162) maintained good adherence for less than 2 months (Perwien et al. 2004). Capone and colleagues presented data from 5659 patients receiving pharmacotherapy for ADHD. They found that within the first 3 months of treatment 50% were no longer taking medication and, by the end of 18-months 80% had ceased refilling prescriptions (Capone et al. 2005). Problems with compliance occurred to the same degree regardless of medication type (i.e., extended release mixed amphetamine salts, extended release methylphenidate, long acting methylphenidate, or atomoxetine) (Capone et al. 2005). Clinicians may not be aware of compliance problems in their practice. Studies that evaluate ADHD medication use based on parental report or adolescent self-report provide rosy estimates of 70%–80% continuous compliance (Hugtenburg et al. 2006; Ibrahim el 2002). In reality, fewer than 10% of patients with ADHD comply with and persist with treatment long-term (Weiss et al. 2006).
The MTA approach of medication adjustments based on monthly contact with school teachers markedly improved outcome (Jensen et al. 2001), however this approach seems unlikely to gain traction in the community. Both the American Academy of Child and Adolescent Psychiatry (Dulcan and Benson 1997) and the American Academy of Pediatrics (AAP 2000) strongly advocate use of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) criteria and collection of parent and teacher ratings in the assessment and management of children with ADHD. However, only about 38% of pediatricians use DSM-IV when assessing children for attention problems (Gardner et al. 2004), and only 4% follow guidelines for collecting ratings from parents and teachers along with use of DSM-IV (Olson et al. 2005). One major barrier to the implementations of this standard is the high degree of discrepancy observed between parent and teacher ratings in primary practice (greater than 40%), which creates a conundrum that the practice guidelines fail to address (Leslie et al. 2004).
Theoretically, laboratory-based tests of attention and activity provide another means of gauging response to treatment. Previous research has shown that laboratory measures of activity and attention are highly responsive to the effects of methylphenidate (e.g., Greenberg 1987; Porrino et al. 1983; Teicher et al. 2004; Teicher et al. 2003; Teicher et al. 2006). However, some studies suggest that continuous performance attention tests (CPT) cannot be used for drug titration as CPT performance improves on doses that are too low to produce clinical benefits (Matier et al. 1992; O'Toole et al. 1997); calling into question their ecological validity. Hence, office–based attention testing has never become a standard of care or a recognized alternative.
In this study, we sought to ascertain whether objective assessment of seated motor activity, captured using infrared motion analysis during performance of a monotonous but demanding attention test (Teicher et al. 1996), could identify MPH doses associated with optimal clinical response as identified by parents. The potential benefit of an office-based measure is that the effectiveness of a given dose of stimulant can be assessed in a single session, and can be used to minimize time children with ADHD spend receiving doses or types of medication that are likely to provide little value. A second benefit is that performance of a child with ADHD on and off medication can be compared to a normative database, so that one can tell not only their degree of improvement, but whether medication enables them to sit-still and pay attention to the same degree as their non-ADHD classmates.
The McLean Motion and Attention Test (MMAT™) was used as the objective assessment device in this report. This test was previously called OPTAx™, and it was found in a recent open-assessment study of children with ADHD or hyperkinetic disorder, to provide a high degree of concordance with clinician ratings of improvement (Tabori–Kraft et al. 2007). The advantage of the present study is that it was triple-blind, with clinicians, parents, and children unaware of the dose of methylphenidate or placebo received each week, and the results of any of the MMAT™ assessments.
Results reported herein derive from a reanalysis of a study designed to assess dose-dependent effects of MPH on objective indices of hyperactivity, inattention, and resting regional cerebral blood flow. Results of these measures have been previously reported (Anderson et al. 2002; Teicher et al. 2000; Teicher et al. 2003). Comparisons between clinical ratings of methylphenidate response and M-MAT™ measures were not included in any of the previous reports.
Evaluation of therapeutic response in this study was based on parental report in which the degree of benefit was expressed using seven-point Clinical Global Impressions (CGI) guideposts (Guy 2005). Teacher ratings were not obtained. However, recent meta-analyses by Biederman and colleagues found that parent reports are at least as sensitive as teachers' reports of medication effects across three major randomized clinical trials of long-acting stimulants (n = 1445) (Biederman et al. 2004), and two major trials of atomoxetine (n = 318) (Biederman et al. 2006). Similarly, Bohnstedt et al. (Bohnstedt et al. 2005) compared teachers and parents as raters of a child's response to atomoxetine. Overall, parent/teacher agreement was moderately correlated for improvement on drug (baseline to endpoint change score r = 0.53, p < 0.04). However, parent ratings were more sensitive to symptom changes over the individual study weeks with effect sizes 1.5 to 4 times greater than teachers (Bohnstedt et al. 2005). These studies demonstrate the utility of parent ratings in evaluation of response to atomoxetine or long-acting stimulant preparations. Parents may not be as perceptive in their evaluation of response to morning and afternoon administration of immediate release MPH, as medication effects will generally wear off by late afternoon or early evening. Weekends or vacation periods provide the principle opportunity for parents to observe the full effect of short-acting drug on their child. Teacher ratings may provide more definitive data on the efficacy of short-acting medications.
Methods
Subjects
Subjects were recruited via regional newspaper advertisement for a study of hyperactive children medicated with stimulants. The McLean Hospital IRB reviewed and approved the study, which was conducted within a clinical research program at a university-affiliated, major psychiatric hospital located in a suburban city. The study was described in detail to potential participants. Parent(s) provided written informed consent, and each child gave verbal assent. To enter the study, children needed to meet the DSM-IV criteria for ADHD (APA 1994) assessed by means of structured interview with the Kiddie Schedule for Affective Disorder and Schizophrenia for School-Age Children, Epidemiologic Version, 5th revision (Orvaschel and Puig–Antich 1994). The children could not have major mood disorder, psychosis, tic disorder, a major anxiety disorder, or mental retardation. Children with oppositional defiant disorder, mild anxiety, or reported learning disorders could participate. Eleven Caucasian boys (9.6 ± 1.8 years of age; range: 6–12) who met DSM-IV criteria for ADHD, and who were currently receiving treatment with methylphenidate, participated in this study.
Procedure
Subjects took part in a 4-week triple-blind treatment trial in which they received 1 week of treatment with placebo and 1 week of treatment with low (0.4 mg/kg), medium (0.8 mg/kg), and high (1.5 mg/kg) daily doses of methylphenidate administered b.i.d. (morning and afternoon). Treatment order was randomized with the exception that the week of high dose treatment always followed the week of medium dose treatment to avoid possible untoward effects of an abrupt jump to high dose. This provided six possible dosing sequences. Individual medication doses were formulated at McLean Hospital, and packaged into identical gelatin capsules, to keep children, parents, and raters blind. During the last day of treatment on a given regimen, children returned to the laboratory and were tested for degree of movement and performance using M-MAT™ (Teicher et al. 1996). Testing occurred approximately 2 hours after their last methylphenidate dose. They also underwent functional MRI imaging in a 1.5T GE scanner (Anderson et al. 2002; Teicher et al. 2000).
Weekly parent ratings were obtained by interview with a clinician blind to treatment and M-MAT™ results using an index of clinical global improvement (CGI), making it possible to ascertain if beneficial effects of methylphenidate on objective measures of activity and attention were in concordance with parental ratings of efficacy. The CGI is rated on a 7-point scale from Very Much Improved to Very Much Worse (Guy 2005).
M-MAT™ is a 15-min challenge test designed primarily to quantify the hyperactivity or ‘fidgeting’ of children with ADHD during performance of a monotonous but demanding continuous performance task (CPT) attention task (Teicher et al. 1996). The CPT was a simple Go/No-Go task, similar to Greenberg's Minnesota Computer Assessment and Test of Visual Attention (Greenberg 1987), which used two easily discriminated geometric shapes. Stimuli were displayed on a computer screen for 200 milliseconds with a 2 second interstimulus interval (Teicher et al. 2004). Half of the stimuli were targets to which subjects were instructed to respond with a key press. No response was to be made to nontargets. During the M-MAT™, an infrared motion analysis system (Qualysis, Gothenburg, Sweden) tracked the precise two-dimensional location of a small reflective marker worn on a cap. The infrared motion analysis system collected and recorded movement data 50 times per second with a resolution of 0.04 mm [for full description see (Teicher et al. 1996)].
Improvement on M-MAT™ was determined by change in two a priori selected response measures. The primary measure of efficacy was the number of microevents emitted, which is essentially a measure of position changes. A new microevent occurs whenever the reflective head marker moves more than 1 mm from the previous microevent location. Microevents were selected a priori out of convenience, as they serve as the basis for the determination of all other activity measures (displacement, movement area, immobility duration, temporal and spatial scaling), and the activity measures are highly correlated. Microevents correlate 0.967, 0.966, 0.699, and −0.599 with measures of displacement, temporal scaling, movement area, and spatial complexity, respectively (n = 144, all p < 0.0001). Immobility duration is equal to the reciprocal number of microevents (one/microvents). The standard deviation of correct response latency was selected as a secondary measure. We previously found that hypervariability in response latency on M-MAT™ differentiated ADHD children from healthy controls better than other standard measures of attentional performance (Teicher et al. 1996). High intra-individual variability may be a defining characteristic of ADHD (Castellanos et al. 2005), and presumably corresponds to momentary lapses in attention (Weissman et al. 2006).
Data analysis
A clinical investigator highly familiar with the methodology and blind to results of parent ratings reviewed objective measures of activity and attention on each dose. To select the dose producing the most beneficial response on M-MAT™ measures, he first looked for the dose that normalized or markedly improved activity (microevents) based on previously reported values for normal controls on this test (Teicher et al. 1996; Teicher et al. 2004; Teicher et al. 2003). Activity was examined first, as this measure is more sensitive to and discriminative of differences between children with ADHD and healthy controls than the attention measures (Teicher et al. 1996). If more than one dose normalized activity than the dose that normalized activity and produced the smallest variability (S.D.) in response latency was selected as optimal. All of the remaining doses were ranked as well for efficacy. Second best, third best, and worst responses were ranked by levels of motor activity, except in one subject who had the same number of microevents (± 0.1%) on two different doses.
Practically, the most important question was whether M-MAT™ measures could identify the dose found to be most efficacious by parental CGI ratings. The next most important question was whether M-MAT™ measures could identify the dose associated with poorest response. As each subject was assessed on four different doses (including placebo), probability of correspondence across dose for best response on M-MAT™ and CGI was 0.25 by random chance. Departure from chance level correspondence was assessed using the Test for Significance of a Proportion (Bruning and Kintz 1977).
An overall degree of concordance was also determined based on correspondence between M-MAT ranking of a given dose for each subject (from best = 1 to worst = 4) and the corresponding within subject dose ranking based on CGI score. Pearson's Product Moment was used to assess degree of association as Pearson's Correlation of ranked data is equivalent to Spearman's rho, and is more suited to analysis of data with tied rankings.
Repeated measure analysis of variance was used to evaluate differences in M-MAT™ measures between weeks rated best and worst by parents, to compare CGI ratings between weeks rated best and worst by M-MAT™, and to examine the effect of dose on M-MAT™-derived and CGI-derived rankings. Sphericity was assessed and no corrections were required.
Results
In 9 of 11 subjects, the dose that produced the best overall improvement on objective measures of activity and attention was also the dose that parents indicated produced the best clinical outcome (expected proportion 0.25, observed proportion 0.82; Z = 4.35, p < 10−5). Similarly, 7 of 11 parents indicated that their child's worst week occurred when they were receiving the treatment that produced the worst objective outcome on M-MAT™ measures (expected proportion 0.25, observed proportion 0.64; Z = 2.96, p < 0.002).
Performance on M-MAT™ measures was much better during the week parents rated best versus the week rated worst (microevents: 40% improvement, F 1,10 = 28.8, p = 0.0003; variability in response latency: 36% improvement, F 1,10 = 9.6, p = 0.01). Parents rated the week that produced the best improvement in M-MAT™ response as significantly better than the child's response to previously prescribed treatment (CGI: +1.46, 95% CI: 0.44–2.36). Children received an average daily dose of 23.4 ± 20.4 mg MPH (0.7 ± 0.4 mg/kg) prior to enrollment, and an average daily dose of 38.2 ± 15.6 mg MPH (1.25 ± 0.35 mg/kg), during the week yielding the best M-MAT™ response. The week with the worst scores on the objective measures was rated as significantly worse than customary treatment (CGI: −1.55, 95% CI −0.73—2.36), and the differences in CGI scores between these weeks was highly significant (F 1,10 = 27.52, p < 0.0005). Age did not exert a significant effect on the mg/kg dose rated best by parents (r = 0.097, p > 0.7) or by M-MAT™ (r = 0.048, p > 0.8).
There was a strong tendency for M-MAT™ measures and parents to rank order response from best to worst as a function of dose (high > medium > low > placebo). M-MAT™ derived rankings had a very strong and somewhat more consistent association to dose (F 3,40 = 35.85, p < 10 −10 ; eta-squared effect size = 0.729) than CGI-derived rankings (F 3,40 = 17.27, p < 10−6; eta-squared = 0.564). As illustrated in Figure 1, there was considerable concordance between M-MAT™ and parental CGI in ranking of response to each dose (r = 0.655, p < 0.001). In 10 of 11 subjects, M-MAT™ indicated that the worst week occurred on placebo, and in the remaining case on the lowest dose. Parents indicated that the worst week occurred on placebo in seven cases, on the lowest dose in two cases, and on moderate dose in two cases. M-MAT™ indicated that the best response occurred in seven subjects on high dose and in four subjects on moderate dose (mean daily dose = 1.25 ± 0.35 mg/kg). Parents indicated that best response occurred in nine subjects on the highest dose, and in two subjects on the moderate dose (mean daily dose = 1.37 ± 0.09 mg/kg).

Relationship between placebo and doses of methylphenidate administered and ranking of response to each treatment (from best = 1 to worst = 4) according to M-MAT™ measures and CGI ratings from parental report.
Discussion
The MTA multimodal treatment study showed that children receiving conventional community care were often under-treated, and rarely received the full benefit that medications could provide if carefully titrated to maximal effectiveness (MTA Cooperative Group 1999). The protocol used in this study lead to the identification of doses of methylphenidate that were much better (two cases), or very much better (five cases), than the dose prescribed by the subject's community doctor, confirming the tendency for children with ADHD to receive suboptimal treatment. Titration depends on accurate and timely information. This study provides evidence that objective measures of primarily activity and secondarily attention performance show patterns of response to different doses of methylphenidate and placebo that are in good agreement with blind placebo-controlled parental ratings of efficacy. In 9 of 11 cases, M-MAT™ measures identified the dosage parents rated as most beneficial. This provides preliminary evidence that office-based assessment of clinical response with objective measures of activity and attention have ecological validity, and the potential to facilitate rapid and accurate dose titration. While these results are interesting and supportive of this hypothesis, we emphasized that these findings are preliminary and derived from only 11 subjects.
Much has been debated about the role of various informants in assessing clinical response to drug treatment. Farone et al (Faraone et al. 2005) found that teachers and parents agree better about when a child is improving with medication treatment than when worsening. They found a high probability that parents would confirm teacher reports of therapeutic improvement and a lower probability that teachers would confirm parent reports of improvement. In contrast, neither reporter was likely to confirm the other reporter's rating of no improvement or worsening. In this study, there was very good agreement between M-MAT™ measures and parenting ratings regarding doses that produced therapeutic improvement. However, there was also reasonable agreement between M-MAT and parents regarding doses that produced no improvement or worsening.
In general, M-MAT™ measures perform as one would expect for a bioassay. They appear to be unaffected by placebo, to mirror plasma levels and pharmacokinetics (Teicher et al. 2006), and to follow a dose-response relationship that corresponds to clinical experience. It is hard to imagine that ratings of clinical response would substantially differ from M-MAT™ measures, but it is reassuring to have it confirmed in the present preliminary study and in a previous report by Tabori-Kraft et al., (Tabori-Kraft et al. 2007).
Given that titration by M-MAT™ would have produced a very similar outcome to titration by parent ratings, one may then ask if M-MAT™ measures have any potential utility over and above parent ratings. First, it is important to note that in the present study, and in the numerous other studies that reveal the value of parental ratings, that blind placebo-controlled protocols were used (Biederman et al. 2004; Biederman et al. 2006; Bohnstedt et al. 2005; Faraone et al. 2005). We do not know how much parental ratings may be affected by knowledge of dose. It is conceivable that parents may unconsciously exaggerate the benefits of moderate dose if they are reluctant to have their child titrated to what they perceive will be a high dose, and to disparage the effects of a low dose that they believe to be insufficient. We do not know if parents would rate response in the same manner if they had not been informed, in writing, that their child could receive a placebo. M-MAT™ measures are not susceptible to parental preconceptions regarding type and dose of medication. Practically, it may be much easier to confirm parental reports using an objective laboratory measures in the course of routine care than for a clinician to endeavor to obtain impartial parental ratings by organizing a blind placebo-controlled trial for each patient.
A second potential difference between M-MAT™ measures and ratings by parents or teachers relates to time frame. How many days are required for a treatment to be in place to obtain a stable and reliable clinical rating? Several days, or weeks, might be necessary to counter habitual modes of interaction or behavior that a child with ADHD may have developed. The intrusion of random life events (e.g., dog runs away, bullied at school, not invited to a classmate's party) can also wreck havoc with behavioral ratings. Brief (5 min) M-MAT™ measures, suitable for titration, have been obtained up to eleven times per day (Teicher et al. 2006). We have found that M-MAT™ can accurately gauge response to a 0.4 mg/kg probe dose of immediate release methylphenidate within 90 min (Teicher et al. 2004), and we use this approach to effectively jump start the titration process. We are currently exploring the utility of a single-visit pharmacokinetic/pharmacodynamic test protocol in which multiple small doses are administered to produce accumulating blood levels, with brief M-MAT measures obtained at specific times to gauge response (Teicher et al. 2006). Theoretically, it should be possible to determine whether an individual responds best to low, medium, moderate, or high doses of methylphenidate in as little as 3–5 hours. If this proves successful, it would be very interesting to see if rapid titration to optimal dose could enhance compliance and improve overall clinical outcome.
This study is limited by small sample size, exclusive focus on boys between 6 and 12 years of age, and evaluation of immediate release methylphenidate as the sole treatment. It provides initial triple-blind placebo-controlled confirmation of results of a larger open study demonstrating the ecological validity of this technology (Tabori–Kraft et al. 2007). However, these findings must be viewed as preliminary and limited. A multicenter study of the utility of M-MAT™ for medication management is in progress. It will include both boys and girls with ADHD, and will evaluate response to methylphenidate, mixed amphetamine salts, and atomoxetine using parent, teacher, and clinician ratings. Given the concern surrounding the treatment of ADHD, and concern regarding undue influence of pharmaceutical manufacturers on prescribing practices, it may be valuable to have an impartial means of monitoring response to guide or confirm therapeutic decisions. This study indicates that laboratory-based assessment of motor activity and attention may have more potential to provide measures that accurately reflect clinically-relevant outcomes than previously realized.
Footnotes
Disclosures
M-MAT™ technology is licensed by McLean Hospital to BioBehavioral Diagnostic Company (BDC). Dr. Teicher has the potential to receive a portion of the royalties that might be paid to McLean from the use of M-MAT™, in compliance with guidelines established by Harvard Medical School to minimize conflict of interest in clinical research. Dr. Teicher has no equity interest in BDC and holds no management position. Dr. Teicher has been reimbursed by BDC for travel expenses incurred to present results of research on M-MAT™, is currently receiving funding from BDC for new research relating to M-MAT™, and has received consulting fees that fall within the de minimis guidelines established by Harvard Medical School and Partners Health Care. BDC did not support this work and had no input into the design, analysis or reporting of results.
Dr. Teicher also receives research support, as a component of an NIMH SBIR award, from Ambulatory Monitoring Inc., to investigate use of a feedback actigraph in treatment of children with ADHD. Dr. Teicher does not receive any funding, consulting or speaking fees from pharmaceutical companies.
Dr. Ann Polcari and Ms. Cynthia McGreenery have no conflicts of interest or financial ties to disclose.
Support for this project was provided, in part, by National Institute of Mental Health (NIMH) Grant MH-48343 to MHT.
