Abstract
This study examined the association of pharmacological treatments and academic achievement among children with attention-deficit/hyperactivity disorder (ADHD). Results examining the association of pharmacological treatments and academic achievement among children with ADHD are mixed. Our objective was to examine this association using structural equation modeling (SEM) techniques, which may be considered more sophisticated and advanced over traditional regression techniques. To achieve the purpose, we employed a sample of children with ADHD derived from the Early Childhood Longitudinal Study-Kindergarten (ECLS-K) data. The ECLS-K provides a large, community-based, nationally representative sample of children to examine across time with respect to academic achievement outcomes. The present study reveals a statistically nonsignificant association between pharmacological treatment and academic achievement among children with ADHD. These results derived from a large, community-based, nationally representative sample, using SEM techniques, may be considered highly generalizable.
Introduction
Given the prevalence of negative educational outcomes associated with this special population of children, the efficacy of treatments for ADHD and its related symptoms merits investigation. The most commonly utilized treatments for children with ADHD are behavioral, pharmacological, and a combination of behavioral and pharmacological treatments (Olfson et al. 2003). The use of pharmacological treatments such as Ritalin®, Adderall®, Strattera®, Vyvanse®, and Cylert® in treating the symptoms of ADHD in children has increased fivefold over the last 20 years (Bhatara et al. 2004; Thomas et al. 2006). From 1990 to 1998, the U.S. Drug Enforcement Administration has reported an eightfold increase in the production of prescription methylphenidate stimulants used in the treatment of children with ADHD (Feussner 1998).
Scheffler et al. examined the association between reported pharmacotherapy and academic achievement scores in both reading and mathematics with a sample of children with ADHD across time using the Early Childhood Longitudinal Study-Kindergarten cohort (ECLS-K) as a nationally representative, community-based sampling frame (Scheffler et al. 2009). In examining the efficacy of these pharmacological treatments, Scheffler et al. indicated a modest, positive association between medication use and academic achievement among elementary school-aged children with ADHD using the ECLS-K sample. Barnard et al., however, indicated a statistically nonsignificant association between pharmacological treatment and academic achievement for children with ADHD during elementary school-aged years when examining as an aggregate without respect to subtype symptoms in the special Education Elementary Longitudinal Study (SEELS) sample (Barnard et al. 2010). Additionally, Barnard et al. (2010) indicated a negative association between pharmacological treatment and academic achievement for children diagnosed with ADHD-not otherwise specified (NOS) symptoms when disaggregating the sample according to subtypes in the SEELS sample.
Based upon these mixed results, examining the efficacy of pharmacological treatment with respect to academic achievement is especially important. The purpose of the present study was to examine the association between pharmacotherapy and both reading and mathematics academic achievement among children with ADHD, with duration of medication treatment as the predictor variable. To achieve this purpose, we utilized the same ECLS-K data as from Scheffler et al., however, now performing structural equation models with these data (Barnard et al. 2010). Specifically, structural equation modeling (SEM) may be considered an advanced and sophisticated means of analyzing complex models simultaneously as “common approaches, such as regression, do not control for measurement error and, therefore may yield biased results” (Nelson et al. 2008) (p. 679). In addition, SEM approaches, such as latent growth modeling, may be considered particularly powerful when examining longitudinal data as this technique is able to discern an underlying trend or trajectory among individuals across time, as demonstrated in the present study (Hancock et al. 2001; Nelson et al. 2008). Using latent growth modeling techniques, we did not assume a linear trend across time but rather tested for different trends of intra-individual growth in achievement.
Methods
Sample
The present study consisted of a sample of 783 children with professional diagnoses of ADHD as reported by the parent from the Early Childhood Longitudinal Study-Kindergarten (ECLS-K) cohort. The ECLS-K followed a community-based, nationally representative sample of approximately 17,000 children from kindergarten to fifth grade as used in the present study beginning the 1998–1999 school year (NCES 2006). ECLS-K collected information on the cognitive, social, emotional, and physical development of these children from their families and schools through the use of trained evaluators who assessed children and/or through questionnaires completed by parents and teachers (NCES 2006). The sample consisted of children, with approximately 73.9% (n=579) male and the remaining 26.1% (n=204) female. With regard to ethnicity, 72.7% (n=589) of the sample were identified as White and non-Hispanic, 9.8% (n=77) described themselves as African American, 10.6% (n=83) described themselves as Hispanic, 1.0% (n=8) described themselves as Asian, 1.3% (n=10) described themselves as Native Hawaiian or other Pacific Islander, 1.4% (n=11) described themselves as American Indian or Alaska Native, and 3.2% (n=25) described themselves as being more than one race. Of these children with ADHD, approximately 66.9% (n=524) reported taking a prescription medicine and 33.1% (n=259) reported not as of the spring semester of fifth grade.
Measures
All measures were obtained from the ECLS-K. To measure the duration of pharmacological treatment, the parent interview variable of time taking prescription medicine for the treatment of ADHD was utilized. This variable (ECLS-K Naming Convention: P6 CHQ760) was measured as of the spring semester of fifth grade, with values ranging from 1 referring to “less than 1 month,” 2 referring to “less than 1 year,” 3 referring to “1–2 years,” 4 referring to “3–4 years,” and 5 referring to “more than 5 years.” Approximately 33.1% (n=259) of the sample was not taking any prescription medicine for the treatment of ADHD as of the spring semester of fifth grade. We should note that although these 259 children with ADHD were not receiving pharmacological treatment, this does not mean that these children did not receive pharmacological treatment at some other time point before the study. No data were provided concerning pharmacological treatment prior to the collection of the dataset study points. As a result, these 259 children were coded as zero (as not receiving pharmacological treatment). Approximately 1.9% (n=15) of the sample had received pharmacological treatment for less than 1 month, approximately 12.0% (n=94) had received such treatment for less than a year, approximately 19.3% (n=151) had received treatment for 1–2 years, and approximately 21.1% (n=165) had received treatment for 3–4 years. Finally, approximately 12.6% (n=99) of the sample had received pharmacological treatment for more than 5 years as of the spring semester of fifth grade. To measure academic achievement, reading and mathematics achievement scores were examined separately. The Item Response Theory-scaled scores for each of the five time points were utilized from the fall kindergarten semester to the spring fifth grade semester. Further information regarding the psychometric properties of these achievement tests can be obtained from ECLS-K in the form of psychometric reports. Table 1 provides means and standardized deviations for reading and mathematics achievement at each time point.
Procedure
Analyses were performed with MPlus (v. 5.20) (Muthén and Muthén 2008). Upon considering incomplete data as missing completely at random, missing at random, or missing not at random (Little and Rubin 2002), values for missing data were handled using multiple imputation techniques with a Bayesian estimation procedure that produced 10 iterated datasets. Final imputed values were derived from the 10 datasets, hence the term multiple imputation. Approximately 12%–28% of the score data were considered missing at each time point. This analytical decision, however, was dependent upon the amount of missing data and whether it may be considered ignorable as per the guidelines of Widaman (2006), as he asserts that multiple imputation techniques should be employed when there is a moderate or large amount of missing data. The guidelines are applied in longitudinal cohort designs. We estimated our models with and without values imputed for missing data and results appeared to be similar, supporting the use of multiple imputation techniques. As the ECLS-K is a complex dataset, weights were applied and design effects were adjusted to compensate for the underestimation of standard errors. This underestimation of standard errors can lead to a higher likelihood of a type I error rate (e.g., the likelihood of rejecting a true null hypothesis). Thus, weights must be applied and design effects were adjusted to yield more accurate estimates of standard errors, thereby decreasing the discrepancy between the population and the sample achieved (Hahs-Vaughn 2005, 2006). From consulting the ECLS-K user's manual, the panel or parent weight applied in each model was C1_6SCO and the cluster or primary sampling unit variable was C16FCPSCU.
Analysis
SEM techniques were utilized to achieve the purpose of the present study. To measure the growth trajectory for both reading and mathematics achievement, latent growth analyses were employed. Latent growth analyses may be considered a SEM technique of modeling growth or change across time, where growth parameters of initial status and slope are estimated as latent variables. From Figures 1 and 2, latent variables are represented by circles and observed variables are represented by squares. In latent growth analyses, the estimation of those parameters (e.g., intercept and slope) that represent growth as a function of a repeatedly measured variable (or variables) across time is conducted (Meredith and Tisak 1990). Latent growth modeling may be considered the estimation of “… systematic inter-individual differences in intra-individual change …” (Stoel et al. 2003) (p. 1). In assessing these latent growth trajectories, we utilized the same ECLS-K time points as Scheffler et al. (2009) and tested for a linear trend in these data. In latent growth modeling, we adjust for the time intervals among data collection time points by setting the path from the slope factor to each achievement data time point to some value to reflect this spacing of time. In the present study, these unstandardized path values were set to 0, 1, 3, 7, and 11, respectively, to reflect the interval of time among the waves of data collection from the fall semester of kindergarten to the spring semester of fifth grade. Tables 2 and 3 also reflect these values. As different children have different developmental trajectories, especially in considering children with special needs such as those with ADHD, the latent intercepts and slopes for reading and mathematics achievement were considered as random effects. All other variables entered into the model were considered as fixed or without measurement error. After establishing the measurement model for each achievement trajectory, the association between pharmacological treatment and academic achievement among children with ADHD was then examined. In these analyses, we statistically controlled for many of the same variables as Scheffler et al. (2009): Child gender, race, age, Individualized Education Program (IEP) participation, parent marital status, household size, household income, and mother's highest education. Race was entered as a series of dummy-coded variables along with gender, IEP participation, and parental marital status as nominal variables. Other covariates such as age, household size, and mother's highest education were semicontinuous with measurement at the ordinal level at least. Unlike the analyses of ECLS-K data by Scheffler et al. (2009), we could not control for intelligence in our model as the ECLS-K does not appear to contain data with respect to any measure of intelligence in reading the user's manual. It appears that Scheffler et al. considered intelligence to be time-invariant in children by using a fixed effects approach. This appears to imply a purely nature or genetic origin of intelligence and does not allow for fluctuation of intelligence due to development. Upon estimating our models for reading and mathematics achievement, five measures indicating goodness of fit were reported: Chi-square (χ 2) goodness-of-fit, χ 2/df ratio, the root mean square error of approximation (RMSEA), the Tucker Lewis Index (TLI), and the Comparative Fit Index (CFI). In comparing nonnested measurement models, values for the Bayesian Information Criterion (BIC) were compared, wherein smaller values indicate better fit. Standardized path coefficients values are also reported.

Reading achievement model.

Mathematics achievement model.
I=intercept; S=slope; IEP=Individualized Education Program; K=kindergarten.
Results
Before estimating the reading and mathematics achievement trajectories across time, we observed a significant degree of autocorrelation in the mathematics and reading achievement score data. Autocorrelation refers to “when the same measurement instruments are used over two or more occasions, [thus] there is a tendency for the measurement errors to correlate” (Schumacker and Lomax 2004) (p. 397). Sivo et al. (2005) indicated that not adjusting for autocorrelation can “diminish the ability of a researcher to detect growth if not explicitly modeled” (p. 215). Thus, to adjust for this autocorrelation, we utilized a model that adjusted for autoregressive disturbances as first order (or one panel lag) constrained to some number according to interval spacing of data between consecutive time points.
Reading achievement model
In examining the measurement model for the reading achievement trajectory being linear, results indicated that the data may fit the model (χ 2(6)=18.84, p<0.05, CFI=0.994, TLI=0.989, RMSEA=0.048, and BIC=24,008.73). We also estimated and compared other nonlinear trends, which did not fit the data as well as the linear trajectory. We specifically examined BIC values, as a logarithmic trend is nonnested within polynomial functions (e.g., quadratic: BIC=24,027.89; cubic: BIC=24,061.06; logarithmic: BIC=24,075.96). Thus, upon deciding that the model estimating the linear reading achievement trajectory may be considered as best fitting the data, the full model examining the association of pharmacological treatment and reading achievement among children with ADHD may then be considered. In examining the association between pharmacological treatment and academic achievement among children with ADHD, the χ 2 goodness-of-fit statistic was significant, indicating that the model may not fit the data (χ 2(34)=90.45, p<0.05). The χ 2 statistic, however, has been indicated as being sensitive to sample size; thus, an adjunct discrepancy-based fit index may be used as the ratio of χ 2 to degrees of freedom (χ 2/df). A χ 2/df ratio value less than 5 has been suggested as indicating an acceptable fit between the hypothesized model and the sample data (MacCallum et al. 1996). With a χ 2/df ratio value of 3.23 for the proposed model, the ratio value indicates an acceptable fit to the data. In addition, the RMSEA as compensating for the effects of model complexity was 0.06, which indicates close to an acceptable fit of the model, as the estimate was less than or close to 0.05 (Browne and Cudek, 1989). The value of TLI, also known as the Nonnormed Fit Index (NNFI), was 0.95 and the value of CFI was 0.96. Hu and Bentler (1999) noted that fit index values of 0.95 or values close to 0.95 are indicative of good fit. Thus, the model appears to fit the data well as represented in Figure 1.
After establishing model fit, the model can then be examined with respect to individual path values. In performing a post hoc power analysis using an RMSEA value of 0.05 or less (Browne and Cudek, 1989), results indicate a sufficient level of statistical power (1−β=0.99) (Preacher and Coffman 2006). The association between pharmacological treatment and reading achievement across time was statistically nonsignificant with a standardized path coefficient value of −0.12. This path value indicates no statistically significant relationship between pharmacological treatment and reading achievement across time among children with ADHD. Table 2 contains the standardized path coefficients from the latent variable constructs of intercept and slope to the observed reading achievement scores and all other paths.
Mathematics model
In examining the measurement model for the mathematics achievement trajectory being linear, results indicated that the data may fit the model (χ
2(6)=9.03
After establishing model fit, the model can then be examined with respect to individual path values. In performing a post hoc power analysis using an RMSEA value of 0.05 or less (Browne and Cudek, 1989), results indicate a sufficient level of statistical power (1−β=0.99) (Preacher and Coffman 2006). The association between pharmacological treatment and mathematics achievement across time was statistically nonsignificant with a standardized path coefficient value of 0.16. This path value indicates no statistically significant relationship between pharmacological treatment and mathematics achievement across time among children with ADHD. Table 3 contains the standardized path coefficients from the latent variable constructs of intercept and slope to the observed mathematics achievement scores and all other paths.
Discussion
Parents continue to be conflicted about the issue of whether to medicate their child with ADHD. As a result, research regarding this issue must be particularly vigilant before concluding that there is a positive association between medication use and academic achievement among children with ADHD. Although the results of the present study indicate a statistically nonsignificant association between pharmacological treatment and academic achievement among children with ADHD, the variability and typology of ADHD as a disorder should be considered. Barnard et al. (2010) also indicated a statistically nonsignificant association between pharmacological treatment and academic achievement among children with ADHD, but when acknowledging subtype symptoms in their analyses, Barnard et al. found statistically significant results. With a sample of 2,844 elementary-aged children with ADHD, the results of Barnard et al. showed that among children who exhibited symptoms of the inattentive, hyperactive-impulsive, and combined subtypes there was a positive association between pharmacological treatment and academic achievement. However, it should be noted that there was a negative association between pharmacological treatment and academic achievement among children who exhibited a lower level of symptoms of ADHD classified as NOS. Barnard et al. concluded that these children may be misdiagnosed or may have some other intervening co-morbid disorder that may be considered more prevalent. Future research should consider replicating the results of Barnard et al. with another sample wherein the influence of symptoms may be acknowledged. Discussing the influence of symptoms, however, is beyond the scope of the present study as the ECLS-K does not contain these data but their importance should not be overlooked.
In contrast to the results of Scheffler et al. (2009), the results of the present study indicate a statistically nonsignificant association between pharmacological treatment and academic achievement among children with ADHD when treated as an aggregate. In the present study, SEM, an advanced and sophisticated set of techniques, was utilized to examine the association of pharmacological treatment and academic achievement among children with ADHD over traditional regression techniques. SEM techniques are not necessarily superior to traditional regression techniques but rather may be considered more appropriate to the modeling of complex processes across time and by taking into account measurement error. However, using regression techniques to examine some of these complex and longitudinal processes invites a higher likelihood of error and, thus, bias in results. We suggest that the results of Scheffler et al., although statistically significant, were small in effect. Thus, issues of measurement error may account for statistically significant gains in academic achievement among children with ADHD.
Several limitations emerged as part of conducting the present study. We consider the primary limitation of the current study as concerning the diagnosis and medication status of the ECLS-K sample of children with ADHD. Our first limitation would be that many of the children with ADHD in this sample had received medication for less than 2 years, which may not be an adequate amount of time in order for statistically significant results in academic achievement to appear. Additionally, the potential side effects of medication used to treat ADHD may also have an unknown impact on the academic achievement of children with ADHD. Our second limitation would be that information regarding the diagnosis and medication status of these children was obtained by parental report of a professional diagnosis, which could be considered as not reliable. From this self-report, it appears that children who are male and white may be more likely to have a diagnosis of ADHD. However, parents of children from non-White ethnic groups have been indicated to be less inclined to acknowledge a diagnosis or medicated status, but this issue did not appear with regard to gender (Barnard-Brak and To 2009). It appears though that children who were male are more likely to have a diagnosis of ADHD in general and thus also more likely to receive medication (Faraone et al. 1995). Another limitation concerns the lack of in-depth information regarding the diagnostic and clinical histories of these children, especially with regard to possible coexisting or co-morbid disorders, which may also have treatments that interact with medications used in the treatment of ADHD. However, a lack of sufficient information regarding diagnostic procedures is not altogether uncharacteristic for samples of children with ADHD as there is no agreement upon set of assessments or procedures of how these assessments are to be utilized in diagnosing ADHD (Tobin et al. 2008). There have been attempts to establish such procedures regarding assessment of ADHD (Pelham and Waschbusch 2006). The large and complex nature of datasets, such as the ECLS-K, however, precludes the collection of such in-depth data regarding participants. Despite these limitations, the ECLS-K provides a large, community-based, nationally representative sample of children, and thus these results may be considered highly generalizable given these characteristics.
Conclusion
The present study presents an examination of the data from the study by Scheffler et al. (2009) using SEM techniques, which may be considered superior given their estimation of measurement error. Additionally, Scheffler et al. (2009) considered intelligence in children to be time-invariant, which contradicts literature, indicating a high degree in variability in intelligence (or cognitive ability) in children across time even with appropriate measurement (Breslau et al. 2001; Ment et al. 2003). There are of course researchers who argue that intelligence is fixed in children from birth, but these researchers advocate a solely nature or genetic origin of intelligence excluding the influence of nurture factors such as environmental and developmental processes. Although the nature versus nurture debate continues, current knowledge is summed up as “there is massive evidence that IQ is far from being immutable” (Howe 1998) (p. 71). Thus, the present study provides an examination of the ECLS-K data using advanced statistical techniques such as SEM without assuming that intelligence is time-invariant.
Clinical Significance
We suggest that in examining the association between pharmacological treatment and academic achievement among children with ADHD, there was a statistically nonsignificant association that may be potentially due to examining children with ADHD as an aggregate without acknowledging the influence of subtype symptoms as delineated by the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) (American Psychiatric Association 1994). We suggest that future research should pay attention to subtype symptoms given the current typology of ADHD as consisting of a subtype framework. With the emergence of the upcoming DSM-V, this subtype framework, however, may change and thus future research should perhaps explore DSM-IV and DSM-V subtype frameworks that emerge. The overarching message of the present study is to note the importance of not treating children with ADHD as an aggregate without acknowledging the influence of subtype symptoms.
Footnotes
Disclosures
Barnard-Brak and Brak have nothing to disclose.
Acknowledgment
The authors thank Ms. Tracey Sulak, M.S., for her invaluable assistance with this manuscript.
