An Alternative Approach to Scoring the MTA-SNAP-IV to Guide Attention-Deficit/Hyperactivity Disorder Medication Treatment Titration towards Symptom Remission: A Preliminary Consideration

Abstract

Objective:

The Multimodal Treatment Study for Attention-Deficit/Hyperactivity Disorder Swanson, Nolan, and Pelham, Version IV (MTA-SNAP-IV) is a common rating scale to measure attention-deficit/hyperactivity disorder (ADHD) symptoms during medication treatment. Relying on the traditional scoring approach for this instrument to identify symptom remission, however, may leave a child with significant residual symptoms. The objective of this study was to examine an alternative scoring approach for this instrument to identify the extent of residual symptoms for children completing ADHD medication treatment.

Methods:

Parent and teacher ratings on the ADHD symptom component of the MTA-SNAP-IV were extracted from medical records of 80 children completing medication treatment at a specialty clinic in Canada. Data were scored in two ways. 1) Traditional scoring based on assigning a value ranging from 0 to 3 for response options: “Not at all,” “Just a little,” “Pretty much,” or “Very much,” for each symptom and then determining a mean across items, and 2) alternative scoring based on assigning values of 0, 0, 0.5, and 1 across the same response options and summing the total across items. Symptom remission based on the former is defined as a mean value ≤1, and for the latter it is defined as a summed value equal to 0.

Results:

Children were significantly less likely to be classified as symptom remitted under the alternative scoring method based on parent, teacher, and combined parent–teacher ratings. Using the alternative scoring approach, residual symptoms were identified for 25%, 39%, and 70% of children classified as symptom remitted (under traditional scoring rules) by parents, teachers, and parents/teachers combined, respectively.

Conclusions:

Potential “residual” ADHD symptoms were identified in many children attaining symptom remission using the traditional scoring approach; however, further scrutiny of this alternative scoring approach is required. Although it may improve the ability to detect residual symptoms that could signal the need for further intervention to achieve symptom remission, it may increase the risk of over treatment.

Introduction

In the approach to treating children with attention-deficit/hyperactivity disorder (ADHD) there are calls to pursue the attainment of symptomatic remission and even “normalization” rather than accept any positive treatment response. Within these calls, one proposed definition of remission is a loss of diagnostic status, with minimal or no symptoms (Steele et al. 2006a; Ramos-Quiroga and Casas 2011). This explicit treatment objective may serve as an outcome goal for clinicians in the management of children with ADHD.

In order to track progress toward remission, proponents recommend the utilization of psychometrically sound instruments to capture parent and teacher ratings of ADHD symptom severity. This is already a common best practice recommendation during medication treatment as described in several treatment guidelines (Pliszka and AACAP Work Group on Quality Issues 2007; Subcommittee on Attention-Deficit/Hyperactivity Disorder and Steering Committee on Quality Improvement and Management 2011). However, there is not yet consensus as to how data collected from these sources may be translated to signal achievement of symptomatic remission.

The Multimodal Treatment Study for ADHD Swanson, Nolan, and Pelham, Version IV (MTA-SNAP-IV) is one of the more frequently used instruments for measuring ADHD symptom severity. It was used in the Multimodal Treatment Study for ADHD (MTA) and was derived from the longer Swanson, Nolan, and Pelham, Version IV instrument (SNAP-IV) (Ramos-Quiroga and Casas 2011). It is composed of all the key symptoms for ADHD and oppositional defiant disorder (ODD) as described in the Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV) (American Psychiatric Association 1994). Each symptom is rated on a four point scale with the response options of “Not at all,” “Just a little,” “Pretty much,” or “Very Much,” which are assigned weights of 0, 1, 2, and 3, respectively. A mean value across items, with a minimum of 0 and maximum of 3, can be generated for the whole scale as well as subscales (e.g., inattentive symptom cluster). We label this the “traditional” scoring approach.

It has been proposed that one definition of symptomatic remission may be operationalized as a value ≤1 on the MTA-SNAP-IV (Swanson et al. 2001; Ramos-Quiroga and Casas 2011). Using this threshold, 68%, 56%, 34%, and 25% of participants in the MTA-study achieved symptom remission after 14 months of treatment in the combined, medication only, behavioral only, and community comparison groups, respectively (Jensen et al. 2001). A recent ADHD medication treatment service evaluation published in this journal used this same threshold, although only for ADHD symptoms, and observed remission rates of 83% for patients with combined parent and teacher ratings at discharge (or 63% if those with incomplete data are assumed to have not achieved remission) (Wagner et al. 2014).

Although a symptom remission threshold value of 1 is appealing and maps onto the mean value from a normative school-based sample (Bussing et al. 2008), further scrutiny is required to assess its utility as a measure of symptomatic remission. On the full 26 item instrument (all ADHD and ODD items), children with mean values ≤1 could still have up to 13 items at the “pretty much” level or up to 8 items at the “very much” level if all others items were rated at the “not at all” level. These “residual” symptoms may be problematic. Even a single symptom at the “pretty much” or “very much” level could, at least in some cases, lead to substantial functional impairment. Therefore, additional scoring strategies are necessary to address this potential measurement deficiency.

One approach is to assume that any individual symptom rating at the “pretty much” or “very much” level could lead to functional impairment, and as such, would not represent a remitted or normative state. This methodology was utilized in a ADHD medication study in Taiwan in which symptomatic remission was defined by no ADHD symptom ratings at the “pretty much” or “very much” level as rated by parents using a Chinese version of the SNAP-IV (Chou et al. 2012). Using this outcome threshold, 66.1% of children in their study would be classified as remitted at the end-point (Chou et al., 2012). A similar approach was also used in a 2006 randomized controlled methylphenidate trial conducted in Canada (Steele et al. 2006b). However, these studies did not leverage the use of this strategy to contrast resulting patterns with the traditional scoring approach.

The objective of this study was to examine outcomes from an ADHD medication treatment service using an alternative scoring approach, and to contrast this with the traditional scoring on the MTA-SNAP-IV. Specific objectives included determining (i) whether the proportion of children who attained symptom remission differed between the two scoring approaches, and (ii) the extent of “residual” symptoms among those children who met the criteria for symptom remission as defined by the traditional scoring approach.

Methods

Setting

All children included in this study were enrolled in the Child Development – Medication Assessment Service (CD-MAS). This is an outpatient treatment service within the developmental pediatric section of the Alberta Children's Hospital in Calgary, Alberta, Canada. CD-MAS aims to systematically evaluate whether a given child's attentional and/or behavioral difficulties are responsive to psychotropic medication and, if so, to determine the optimal medication and dose based on a balance of maximal improvement with minimal emergence of adverse effects. The approach used in CD-MAS is guided by parameters recommended by the Texas Children's Medication Algorithm Project for ADHD including medication sequencing and dosing, and use of systematically collected parent- and teacher-rated ADHD symptom severity scores throughout treatment to inform medication titration (Pliszka et al. 2000a,b; 2006). If an optimal medication and dose is identified for a given child, this same medication and dose is designated as the maintenance treatment and the child is discharged from CD-MAS and transferred to their primary care physician for long-term management. Details of resulting medication treatment from this service have been previously reported (Wagner et al. 2014).

A separate program called the Community Outreach of Pediatrics and Psychiatry in Education (COPE), a school-health partnership based in the same city, is the major referral source for CD-MAS. COPE provides child psychiatric and pediatric consultations for children referred from schools in the city and an adjacent periurban school district for whom the schools have concerns about emotional, behavioral, and/or development problems (McLennan et al. 2008). Referral to CD-MAS requires an ADHD diagnosis by a pediatrician or child psychiatrist, and physician–parent agreement on proceeding to medication treatment.

Although services offered by CD-MAS are limited to medication treatment, children and families are not restricted from receiving additional services, such as behavior therapy, from other providers. Information on the extent to which CD-MAS patients utilized other services was not available for this study. However, given the limited availability of evidence-based behavioral interventions in this same district, it is assumed that most of the children in this study would not be receiving concurrent intensive behavioral interventions.

Sample

An electronic database of patients who were no longer receiving care at CD-MAS, stripped of identifying information, was provided to the research team. The time period from March 1, 2007 to December 31, 2012 was covered by the database. Eligibility criteria included referral from the COPE program, admission and subsequent discharge from CD-MAS within the time period described, as well as completed outcome ratings from parents and teachers. Of the total 132 children in the database, 52 were dropped because of missing data. Information contrasting those with and without missing data is available in a previous report (Wagner et al. 2014).

Measures

The outcome measure for this study was a modified version of the MTA-SNAP-IV, termed the M-SNAP, which was completed by parents and teachers and utilized to measure symptom severity throughout treatment. The M-SNAP differs from the MTA-SNAP-IV in that it only contains the 18 items measuring ADHD symptom severity, not ODD symptoms. Outcome data collected from parents and teachers in this study were those closest to the discharge date when on the final/discharge dose.

Two approaches were used to generate outcome values to evaluate the attainment of symptom remission. The first approach, labelled “Traditional” scoring, employs the typical approach to calculating symptom severity on the MTA-SNAP-IV as described in the Introduction (Swanson et al. 2001), and in this case extended to the M-SNAP. The second approach, labelled “Alternative” scoring, is newly proposed for this analysis. With this approach, response options of “Not at all,” “Just a little,” “Pretty much,” and “Very Much” are assigned values of 0, 0, 0.5, and 1 respectively, and are summed across items. The resulting score can range from 0 to 18 for the M-SNAP (0–26 for the MTA-SNAP-IV). Symptom remission is defined by a value equal to 0, representing no symptoms reported at the “Pretty Much” and “Very Much” levels.

Analysis

First, the proportion of children attaining symptom remission at discharge as perceived by different informant types (parent, teacher, and parent and teacher combined), using the two scoring methods, was determined. The McNemar test of paired proportions was utilized to assess whether a significant difference in symptomatic remission rates was observed between the two scoring approaches. All available data were used; no a priori power equation was conducted. Second, the extent of “residual” symptoms was identified by summarizing the alternative scores of children who met the traditional scoring criteria (≤ 1) for symptomatic remission.

This study was approved by the Conjoint Health Research Ethics Board (CHREB) at the University of Calgary.

Results

A total of 80 children met the inclusion criteria, of which 88.8% (n = 71) were boys. Mean age was 8.3 (SD 5.0) years. Symptomatic remission rates as defined by each scoring method are summarized in Table 1 by different informants and for total ADHD symptoms and the inattentive and hyperactive/impulsive symptom clusters. In all cases, the proportion of children who achieved symptomatic remission under the traditional approach was significantly higher than with the alternative scoring method. Using the alternative scoring approach, the largest difference occurred for the combined parent–teacher scoring because of the requirement that for the child to be defined as in remission, neither the teacher nor parent could score the child as having any ADHD symptoms above the “Just a little” level.

Table 1.

ADHD Symptom Status at Discharge Using Two Different Scoring Approaches on the M-SNAP (N = 80)

	Symptomatic remission rate		Overall scores
	Traditional scoring % (n)	Alternate scoring % (n)	Traditional scoring Mean (SD)	Alternate scoring Mean (SD)
All ADHD symptoms
Parent ratings	73.8 (59)	55.0 (44)^***	0.75 (0.48)	1.11 (2.20)
Teacher ratings	80.0 (64)	48.8 (39)^***	0.70 (0.41)	1.02 (1.82)
Combined ratings	82.5 (66)	25.0 (20)^***	0.72 (0.31)	1.07 (1.45)
Inattentive symptoms only
Parent ratings	78.8 (63)	70.0 (56)^*	0.79 (0.51)	0.51 (1.19)
Teacher ratings	73.8 (59)	58.8 (47)^**	0.83 (0.45)	0.61 (1.13)
Combined ratings	76.3 (61)	41.3 (33)^***	0.81 (0.33)	0.56 (0.80)
Hyperactive-impulsive symptoms only
Parent ratings	72.5 (58)	63.8 (51)^*	0.71 (0.51)	0.61 (1.21)
Teacher ratings	85.0 (68)	65.0 (52)^***	0.56 (0.46)	0.41 (0.83)
Combined ratings	86.3 (69)	41.3 (33)^***	0.64 (0.36)	0.51 (0.78)

p < 0.05; ^** p < 0.005; ^*** p < 0.001 (using McNemar test for paired proportions).

ADHD, attention-deficit/hyperactivity disorder; M-SNAP, version of the Multimodal Treatment Study for ADHD Swanson, Nolan, and Pelham, Version IV that contains only ADHD symptoms not oppositional defiant disorder symptoms.

Figure 1 represents the frequency distributions of alternative symptom scores among those children who met the criteria for remission based on the traditional scoring approach for each respondent type. Mean residual symptoms were 0.2 (S.D. 0.4), 0.3 (S.D. 0.5), and 0.6 (S.D. 0.7), as rated by parents and teachers, and for the combined parent-teacher ratings respectively.

FIG. 1.

Residual ADHD symptoms for the full scale using alternative scoring for children with a traditional score ≤1.

Discussion

A significantly greater proportion of children were classified as symptom remitted using the traditional scoring approach than using the proposed alternative scoring approach. The proportion of children achieving symptomatic remission using the alternative scoring approach could not exceed that for the traditional scoring given the different scoring rules and remission criterion; however, the overlap could be substantial or complete. For this sample, the proportional differences were large and significant. Nevertheless, the residuals were on average small, with mean values equating to approximately one symptom at the “Pretty much” level for the parent–teacher combined ratings.

A critical issue is whether or in what situations these “residual symptoms” cause functional impairment. In situations in which they are not functionally impairing, there may be no clinical significance, as “normalization” may have been achieved without eliminating all symptoms. The vast majority of children attaining scores ≤1 using the traditional scoring approach will no longer meet diagnostic thresholds, with possible exceptions including children with a predominant inattentive presentation for whom the low values on hyperactive-impulsive items may bring down their overall ADHD score to ≤1. Some have addressed this by using the subscale score specific to inattention items (Stein et al. 2003). Further work is required to explore in more detail the relationship between residual symptoms and functional impairment, which will likely be complicated by variation in the relationship by context.

A second concern, in addition to the point that some residual symptoms may not be clinically significant, is the potential that this lower threshold may lead to overtreatment. This may occur in two situations. First, some children may have attained good function despite “residual symptoms.” In such situations, if the treatment amount (e.g., medication) is increased in an attempt to move all symptom ratings to “just a little” or “none at all,” a given child might be overmedicated. The second situation may occur when a child's residual symptoms, although functionally impairing, are not responsive to further increases in treatment. However, best practice requires that the child's functional status and individual response patterns inform treatment, and that there is not an exclusive focus on symptom counts.

The alternative scoring approach, as with the traditional approach, need not be relegated solely to indexing the proportions that are remitted. If an instrument like the MTA-SNAP-IV is used repeatedly throughout treatment as a measurement feedback system tool (Bickman 2008), the resulting values can indicate the extent to which symptoms are approaching a remitted state even if full symptom remission is not achieved. It is proposed that the resulting metric from the alternative scoring approach may be more clinically interpretable than the traditional scoring approach for this purpose. For example, a score of 2 with alternative scoring would reflect a pattern of two ADHD symptoms at the “Very much” level or four symptoms at the “Pretty much” level, whereas using the traditional scoring approach, the corresponding values would be 0.3 and 0.4, respectively.

The same rationales for using the alternative scoring approach could be extended to similarly scored instruments. For example, the ADHD-RS is a clinician-completed ADHD measure that uses the same weighting (i.e., 0, 1, 2, 3) across four response options for each ADHD symptom as the MTA-SNAP-IV. ADHD symptoms are then summarized through a summation of the weighted responses (DuPaul et al. 1998). The proposed symptom remission threshold of ≤18 for that instrument is essentially the same as a ≤1 mean value using traditional scoring for the MTA-SNAP-IV, and hence may have the same limitations for indicating symptom remission.

Limitations

There are a number of limitations to this study. First, the absence of a functional measure prevented assessment of the clinical significance of the identified residual symptoms. Second, data obtained for this study were from a single site and represent a modest sample size. Third, the low symptom counts at the end of treatment for this sample may be a function of the relatively high intensity of treatment in this service versus treatment as usual (Wagner et al. 2014), and hence the specific symptom patterns observed in this sample may not be generalizable to more typically obtained clinical outcomes. Fourth, although the measurement approach taken includes some aspects of parent/caregiver perspective, given that ratings by this stakeholder group are used, this does not adequately capture the range of dimensions consistent with patient (or family) centered medicine (Sacristan 2013). For example, a parent might not agree with setting the threshold at symptom resolution (using either scoring approach), and find some residual symptoms as completely acceptable. Within this clinic service, the parent/caregiver had to approve each titration and can, and sometime does, decline a physician's recommendation to trial a higher dose targeting residual symptoms. However, this study did not systematically document such declines or obtain measures of caregiver's preferred level of symptom resolution. Fifth, the measure in this study only included ADHD symptoms from the MTA-SNAP-IV and not the ODD items. Further examination would be required to determine whether the inclusion of ODD symptoms would substantially alter the resulting patterns using the alternative scoring approach, as well as allow for more direct comparison with previous publications using the full MTA-SNAP-IV.

Conclusions

Findings from this study suggest that the proposed alternate scoring method for the MTA-SNAP-IV may have some advantages over the traditional scoring approach in monitoring symptom resolution and remission status, and hence identify situations in which additional treatment may be required. However, further research is required to determine the relationship between these residual symptoms and functional impairment and to avoid contributing to potential overtreatment. Before such a new scoring and resulting threshold should be considered for adoption, additional studies are required. Two particular areas that need further examination include 1) determining score distribution using the alternative scoring approach in a normal comparison population, and 2) determining the extent to which the pursuit of further symptom resolution may be offset by the emergence of more adverse effects.

Clinical Significance

Currently proposed indicators of ADHD symptom remission may leave some children with residual symptoms that require clinical attention. The proposed alternate scoring approach offers one strategy to better measure residual ADHD symptoms during clinical treatment which may facilitate intervention titration closer to more complete symptomatic remission and normalization.

Footnotes

Disclosures

No competing financial disclosures exist.

References

American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, 4th ed. Washington, DC: American Psychiatric Association; 1994.

Bickman

: A Measurement Feedback System (MFS) is necessary to improve mental health outcomes. J Am Acad Child Adolesc Psychiatry, 47:1114–1119, 2008.

Bussing

, Fernandez

, Harwood

, Hou

, Garvan

, Eyberg

, Swanson

: Parent and teacher SNAP-IV ratings of attention deficit hyperactivity disorder symptoms psychometric properties and normative ratings from a school district sample. Assessment, 15:317–328, 2008.

Chou

W-J

, Chen

S-J

, Chen

Y-S

, Liang

H-Y

, Lin

C-C

, Tang

C-S

, Huang

Y-S

, Yeh

C-B

, Chou

M-C

, Lin

D-Y

, Hou

P-H

, Wu

Y-Y

, Liu

H-J

, Huang

Y-F

, Hwang

K-L

, Chan

C-H

, Pan

C-H

, Chang

H-L

, Huang

C-F

, Hsu

J-W

: Remission in children and adolescents diagnosed with attention-deficit/hyperactivity disorder via an effective and tolerable titration scheme for osmotic release oral system methylphenidate. J Child Adolesc Psychopharmacol, 22:215–225, 2012.

DuPaul

, Power

, Anastopoulos

, Reid

: ADHD Rating Scale-IV Checklists, Norms, and Clinical Interpretation. New York: Guilford Press; 1998.

Jensen

, Hinshaw

, Swanson

, Greenhill

, Conners

, Arnold

, Abikoff

, Elliot

, Hechtman

, Hoza

, March

, Newcorn

, Severe

, Vitello

, Wells

, Wigal

: Findings from the NIMH Multimodal Treatment Study of ADHD (MTA): Implications and applications for primary care providers. J Dev Behav Pediatr, 22:60–73, 2001.

McLennan

, Reckord

, Clarke

: A mental health outreach program for elementary schools. J Can Acad Child Adolesc Psychiatry, 17:122–130, 2008.

Pliszka

, AACAP Work Group on Quality Issues: Practice parameter for the assessment and treatment of children and adolescents with attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry, 46:894–921, 2007.

Pliszka

, Crismon

, Hughes

, Corners

, Emslie

, Jensen

, McCracken

, Swanson

, Lopez

: The Texas Children's Medication Algorithm Project: Revision of the algorithm for pharmacotherapy of attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry, 45:642–657, 2006.

10.

Pliszka

, Greenhill

, Crismon

, Sedillo

, Carlson

, Conners

, McCracken

, Swanson

, Hughes

, Llana

, Lopez

, Toprac

: The Texas Children's Medication Algorithm Project: Report of the Texas Consensus Conference Panel on Medication Treatment of Childhood Attention-Deficit/Hyperactivity Disorder. Part 1. J Am Acad Child Adolesc Psychiatry, 39:908–919, 2000a.

11.

Pliszka

, Greenhill

, Crismon

, Sedillo

, Carlson

, Conners

, McCracken

, Swanson

, Hughes

, Llana

, Lopez

, Toprac

: The Texas Children's Medication Algorithm Project: Report of the Texas Consensus Conference Panel on Medication Treatment of Childhood Attention-Deficit/Hyperactivity Disorder. Part II: Tactics. J Am Acad Child Adolesc Psychiatry, 39:920–927, 2000b.

12.

Ramos-Quiroga

, Casas

: Achieving remission as a routine goal of pharmacotherapy in attention-deficit hyperactivity disorder. CNS Drugs, 25:17–36, 2011.

13.

Sacristan

: Patient-centered medicine and patient-oriented research: improving health outcomes for individual patients. BMC Med Inform Decis Mak, 13:6, 2013.

14.

Steele

, Jensen

, Quinn

DMP

: Remission versus response as the goal of therapy in ADHD: A new standard for the field?. Clin Ther, 28:1892–1908, 2006a.

15.

Steele

, Weiss

, Swanson

, Wang

, Prinzo

, Binder

: A randomized, controlled effectiveness trial of OROS-methylphenidate compared to usual care with immediate-release methylphenidate in attention deficit-hyperactivity disorder. Can J Clin Pharmacol, 13:e50–62, 2006b.

16.

Stein

, Sarampote

, Waldman

, Robb

, Conlon

, Pearl

, Black

, Seymour

, Newcorn

: A dose-response study of OROS methylphenidate in children with attention-deficit/hyperactivity disorder. Pediatrics, 112:e404–e404, 2003.

17.

Subcommittee on Attention-Deficit/Hyperactivity Disorder, Steering Committee on Quality Improvement and Management: ADHD: Clinical Practice Guideline for the Diagnosis, Evaluation, and Treatment of Attention-Deficit/Hyperactivity Disorder in Children and Adolescents. Pediatrics, 128:1007–1022, 2011.

18.

Swanson

, Kraemer

, Hinshaw

, Arnold

, Conners

, Abikoff

, Clevenger

, Davies

, Elliot

, Greenhill

, Hechtman

, Hoza

, Jensen

, March

, Newcorn

, Owens

, Pelham

, Schiller

, Severe

, Simpson

, Vitiello

, Wells

, Wigal

, Wu

: Clinical relevance of the primary findings of the MTA: Success rates based on severity of ADHD and ODD symptoms at the end of treatment. J Am Acad Child Adolesc Psychiatry, 40:168–179, 2001.

19.

Wagner

, Vallerand

, McLennan

: Treatment receipt and outcomes from a clinic employing the attention-deficit/hyperactivity disorder treatment guideline of the Children's Medication Algorithm Project. J Child Adolesc Psychopharmacol, 24:472–480, 2014.