Abstract
Background:
Childhood mental health problems represent a significant public health concern globally. There is a converging discussion among researchers and practitioners alike that the research results of effectiveness studies are not fully generalizable and applicable to ethnoracial minority groups in real-world practice settings.
Methods:
Archival data on discharges from eight residential programs for children and youth aged 5–18 and operated by a large, child and family human services agency were analyzed to identify the relationship between client demographics and treatment outcomes.
Results:
Minority status was associated with lower odds of treatment success and higher odds of disengagement from treatment.
Discussion:
The results found that minority youth in residential programs had less favorable outcomes, in terms of both disengagement from treatment and a successful discharge, than White youth. This study calls into question whether current treatments are reaching and best serve ethnoracial minority groups in real-world practice settings.
Introduction
Childhood mental health problems represent a significant public health concern globally (Merikangas, Nakamura, & Kessler, 2009). Cumulatively, these problems account for a sizable amount of the estimated nonfatal burden of disease and disability-adjusted life years at the national level and include externalized and internalized clinical diagnoses (Kieling et al., 2011; Murray et al., 2012; Whiteford et al., 2013). In the United States, multiple public health surveillance systems have revealed that an estimated 13–20% of children experience a mental disorder annually, with the prevalence of these conditions increasing in recent decades (Olfson, Blanco, Wang, Laje, & Correll, 2014; Perou et al., 2013). A range of negative outcomes are associated with limited and chronic childhood mental health problems and impede functionality across the life span as well as impacting families and society generally. These outcomes include the risk of onset for additional comorbid mental health problems, during both childhood (Briggs-Gowan, Carter, Bosson-Heenan, Guyer, & Horwitz, 2006; Meagher, Arnold, Doctoroff, Dobbs, & Fisher, 2009) and adulthood (Klein et al., 2012; Mordre, Groholt, Kjelsberg, Sandstad, & Myhre, 2011; Reef, Diamantopoulou, van Meurs, Verhulst, & van der Ende, 2011); psychiatric hospitalization, criminal justice involvement, poor physical health, and lower socioeconomic status during adulthood (Klein et al., 2012); expulsion and suspension in primary and secondary schooling and decreased educational attainment overall (Gilliam & Shahar, 2006; Myer et al., 2009; Sagatun, Heyerdahl, Wentzel-Larsen, & Lien, 2014); and finally, financial burden among families and overlapping human service systems providing care (Busch & Barry, 2007; Foster, Qaseem, & Connor, 2004; Kogan et al., 2008; Marks et al., 2009). Given the extent and accompanying negative outcomes of childhood mental health problems, viable treatment options are needed and one such response in recent decades has been empirically supported treatment (EST; Herschell, McNeil, & McNeil, 2004).
EST
EST can be conceptualized as a specific treatment that has, at a minimum, demonstrated efficacy across controlled research with a defined clinical population (Chambless & Hollon, 1998). Implementing EST involves the clinician choosing a best practice and that single practice is implemented for the duration of client treatment (Patterson, 2014). Guided by the EST movement and effectiveness studies, the identification, promotion, and dissemination of ESTs into treatment plans for clinical populations have been widespread since the mid-1990s (Chambless & Ollendick, 2001; Task Force on Promotion and Dissemination of Psychological Procedures, 1995).
Despite widespread EST use, there is converging discussion among researchers and practitioners alike that the research results of effectiveness studies are not fully generalizable and applicable to ethnoracial minority groups in real-world practice settings because of methodological, epistemological, and cultural competency issues (Aisenberg, 2008; Gone & Alcántara, 2007; McBeath, Briggs, & Aisenberg, 2010; Whitley, Rousseau, Carpenter-Song, & Kirmayer, 2011). Ethnoracial minority groups have been historically underrepresented in EST effectiveness studies, even among federally funded controlled clinical trials (Lau, Chang, & Okazaki, 2010; Mak, Law, Alvidrez, & Pérez-Stable, 2007; Yancey, Ortega, & Kumanyika, 2006), and as a result, analyses for ethnicity or race as a moderating factor in treatment outcome have been largely absent from such studies (Huey, Stanley, & Polo, 2008; Weisz, Jensen-Doss, & Hawley, 2006; Weisz, McCarty, & Valeri, 2006). Moreover, as a product of positivist epistemology, ESTs eliminate the subjectivity inherent within the culture and context in which ethnoracial minority groups reside and, in doing so, disregard distinct factors (e.g., indigenous healing practices) that could interact with potential treatment effects (Conner & Grote, 2008). Consequently, this narrow epistemology erodes the legitimacy of ESTs and further alienates the historically oppressed ethnoracial minority groups ESTs are intended to serve (Aisenberg, 2008; Echo-Hawk, 2011; McBeath et al., 2010; Whitley et al., 2011). In response to such divergence, there has been support for increased cultural competency among ESTs through cultural adaptation (Aisenberg, 2008; Bernal, Jiménez-Chafey, & Domenech Rodríguez, 2009; Bernal & Sáez-Santiago, 2006; Conner & Grote, 2008; Huey et al., 2008; Whitley et al., 2011). Culturally adaptive ESTs have been shown to be efficacious; however, the extent of this efficacy in comparison to other treatments has yet to be fully assessed (Cardemil, 2010; Miranda et al., 2005).
EST and Mental Health Outcomes
Effectiveness studies demonstrate a range of treatment modalities for addressing mental health problems in children (Eldevik et al., 2009; Fossum, Handegård, Martinussen, & Mørch, 2008; Leenarts, Diehle, Doreleijers, Jansma, & Lindauer, 2013). Nonetheless, effectiveness studies with child samples retain the same aforementioned issues of generalizability and applicability for ethnoracial minority groups. Both analyses for ethnicity or race as a moderating factor in treatment outcome have been largely absent from such studies (Harvey & Taylor, 2010; Weisz, Jensen-Doss, et al., 2006; Weisz, McCarty, et al., 2006), and if they were present, sample sizes may have been too small to detect a treatment-ethnicity or race effect (Huey et al., 2008).
Not surprisingly, the ethnicity or race of sample participants has been underreported among these effectiveness studies (Weisz, Jensen-Doss, et al., 2006). Finally, identified treatments lack cultural adaptability for ethnoracial minority children (Huey et al., 2008; McBeath et al., 2010). To this end, this study seeks to investigate the impact of ESTs on ethnoracial minority client outcomes of inpatient mental health services for children.
Methods
Data Source
The data for this study consisted of the deidentified discharge statuses and demographic data for 165 children and youths, hereinafter clients, discharged between July 1, 2009, and December 31, 2010, inclusive, from eight residential treatment programs operated by the Hillside Family of Agencies (HFA), a large child and family human services agency headquartered in Rochester, New York (see Patterson, Dulmus, & Maguin, 2012, for a complete description of HFA; see Table 1 for sample demographics). The July 1–December 31 time period was used, as this was a stable period in the agency’s history. The eight programs provide mental health services to children and youths between the ages of 5 and 18 years, although youths may exceed the upper age limit to complete their treatment programs. All programs used one or more cognitive or behavioral ESTs, the range of which included applied behavioral analysis, aggression replacement therapy, dialectical behavior therapy, positive behavioral interventions and supports, and trauma-focused cognitive behavior therapy. The provided data did not identify which children and youths received which ESTs. The deidentified data did not permit determining whether any of the 165 clients had been discharged more than once from one of the eight residential programs during the follow-up period.
Demographic Characteristics of Sample.
Note. N = 161.
Variables
At discharge, HFA clinical staff in each residential program coded the single reason for discharge from an agency-wide list of reasons. In consultation with HFA, four categories of reasons were developed. Reasons of having increased skills, accomplished goals, reduced risk behaviors, or requiring less intensive services were categorized as successful (n = 118). Reasons such as time limits in the program reached, transferred to adult services (aged out), or was no longer eligible or lost funding were categorized as program limitations (n = 9). Reasons such as crisis discharge or more intensive services required were categorized as program not meeting needs (n = 11). Reasons such as absent without official leave (AWOL), client lost contact or moved, client withdrew, and failed to attend program were categorized as disengaged from treatment (n = 18). All other reasons were categorized as other (n = 5). One variable, successful discharge, contrasted clients categorized as successful with the other three categories of clients. The second, disengaged, contrasted clients categorized as disengaged from treatment with the other three categories of clients.
Clients whose ethnicities were coded as Black, biracial, Hispanic, or other were coded as being minority and were contrasted with clients coded as White. Clients were divided into an 8- to 12-year age-group and a 13- to 20-year age-group based on plots of discharge reason category percentages against age.
Statistical Analysis
Records with missing data were excluded from all analyses. Cross-tabulations were used to summarize bivariate relationships between the client demographic variables and the discharge code variables. Significance was evaluated by Fisher’s exact test, two-tailed. A .05 significance level was used. Logistic regression was used to investigate multivariate relationships. No correction for multiple tests was made. SPSS version 21 was used for all analyses.
Results
The left-hand panel of Table 2 presents the separate cross-tabulations of minority, female, and age-group by successful discharge. The results show that a significantly lower percentage of minority clients were discharged as successful as were White clients (57.5% vs. 86.4%, respectively; odds ratio [OR] = 0.21, 95% confidence interval [CI] [0.10, 0.46]). Also, older clients were less likely to be successful than were younger clients (69.7% vs. 89.7%, respectively; OR = 0.27, 95% CI [0.08, 0.93]). Female, however, was not related to successful discharge, p = .59. A follow-on logistic regression of successful discharge on minority and age-group revealed that both minority (b = −1.624 ± 0.400, OR = 0.20, 95% CI [0.09, 0.43]) and age-group (b = −1.509 ± 0.660, OR = 0.22, 95% CI [0.06, 0.81]) retained significant relationships with successful discharge.
Cross-Tabulation of Client Demographics by Successful Discharge and by Disengagement.
Note. N = 161.
The right-hand panel of Table 2 presents the cross-tabulations of minority, female, and age-group by disengaged. The overwhelming majority of clients (88.8%) did not disengage from treatment. Thus, the numbers in “Yes” cells were small. A significantly larger percentage of minority clients (21.9%) disengaged from treatment than did White clients (2.3%; OR = 12.1, 95% CI [2.67, 54.5]). In addition, a significantly larger percentage of older clients (13.6%) than younger clients (0.0%) disengaged. Again, female was unrelated to disengagement, p = .62. A review of the underlying discharge reason codes for both minority and White clients reveals that the most frequent (n = 12) code was AWOL. Of the remaining seven clients, equal numbers were coded as “not responsive to service” or “withdrew.” A logistic regression was unable to be computed because of the zero frequency for 8- to 12-year-olds who disengaged.
Discussion and Application to Practice
The results show that the percentage of minority clients discharged as successful was about two thirds that of White clients and that the percentage of minority clients disengaging from treatment was about 9.5 times that of White clients. Taken together, the results indicate that the residential treatment programs analyzed here are more favorable for White clients than for minority clients. Incoming minority clients had a greater likelihood of leaving treatment, mainly by being/going AWOL or by not having a favorable discharge, than did Whites. In addition, the data revealed that a lower percentage of older, adolescent-aged clients were discharged as successful and that a higher percentage of the same disengaged from treatment, principally by going AWOL. Finally, both minority and age-group remained significant when successful discharge was regressed on them.
The discharge data also can be viewed as a crude log of progression through the programs. Clients are admitted and either remain in the program or disengage, as the data for disengagement in Table 2 show. Clients who remain in treatment are either discharged favorably or not. Of the minority clients staying in the programs, 74% are discharged as successful. The corresponding percentage for White clients is 88%. Thus, even minority clients who remain engaged in their programs have a lower likelihood of overall success.
Some limitations apply to these results. Most important is that these data are from a single agency and as such cannot be construed necessarily to estimate what would be found had similar data from a random sample of child and family agencies been analyzed. Although discharge decisions are made by the clinical staff relative to attainment of documented treatment goals, the reliability of judgments of treatment goal attainment is unknown, both within and across the eight residential programs. Intake information other than demographic data was unobtainable; thus, it is possible that the diagnostic profiles or the severity of presenting problems differed between minority and White clients or between the younger and older age-groups.
Despite these limitations, this study should be a cause for concern, as empirically supported treatments are not researching minority youth … thus science is not reaching minority youths. A contributing factor to this might well be that treatment outcome studies have not included adequate numbers of minority clients, so findings are not generalizable beyond the sampling frame. Most ESTs have been designed, tested, and validated with minimal input from minority clients (Bernal & Scharrón-Del-Río, 2001; Hussain-Gambles, Atkin, & Leese, 2004; Nagayama-Hall, 2001). Furthermore, in the few studies that have included minorities, the numbers are too small for any reliable comparisons or suggestions for clinical practice (Rubin & Parrish, 2007). With minorities often being overrepresented in many of our mental health facilities (Lasser, Himmelstein, Woolhandler, McCormick, & Bor, 2002; U.S. Department of Health and Human Services, 2001), it is unacceptable to fund randomized controlled trials that do not strive to include adequate numbers of minorities in the sample as available.
If African Americans and Latinos have low enrollment in randomized controlled trials, Native Americans can be characterized as nonexistent in our literature. If a Native American, a minority within the minority, walked into any mental health clinic in America, workers would have minimal empirical guidance for treatment.
Recently, a new science industry has emerged, implementation science, which helps workers choose from the overabundance of ESTs and implement them in practice settings. Having too many ESTs to pick from and implementing them correctly in real-world practice settings are very challenging. Implementing ESTs in real-world practice is vital and this focus is needed to bring science to health and wellness services. However, what is equally important is developing minority-specific ESTs. For instance, there are minimal ESTs for substance-abusing Native American youths (National Institute of Drug Abuse, 2003). Thus, in supporting ethnoracial minority participation in randomized controlled trials and fostering subsequent dissemination of culturally competent ESTs for mental health problems, those overseeing the peer-review process should ensure researchers report upon the number of minority study participants and whether this size allowed for subgroup comparisons and moderational analyses. This suggestion is notable, given ethnicity information is rarely reported in treatment outcome studies (Weisz, Jensen-Doss, et al., 2006). Finally, in supporting minority mental health treatment, dissemination committees across mental health disciplines should include a gauge of cultural competency, so that service providers are aware of the applicability of listed ESTs for these populations. For instance, the American Psychological Association, Division 12, Society of Clinical Psychology, Dissemination Subcommittee of the Committee on Science and Practice webpage states “[t]he information on the site is intended to educate mental health consumers and providers about scientifically supported psychological interventions and is not a formal treatment guideline” (Society of Clinical Psychology, 2013); however, the cultural competency of listed interventions is not presented for the mental health consumer or provider to review.
There should be a cease and desist order directed to all peer-reviewed journals that are considering publishing an article indicating the efficacy of an EST without discussing its enrollment, outcomes, and implications for minority groups or reason for exclusion. Most importantly, until empirically proven otherwise, ESTs that have not been validated with any assistance from minority populations should come with a very clear warning label indicating this significant limitation. Upholding our profession’s standards of cultural competencies demands that we prioritize the funding and publishing of studies that flip the enrollment numbers. New science would then allow us to stop relying on finding ways to adapt current ESTs to serve minority populations. There are still too many articles being published with the quote, “ … limited data … ” when it comes to understanding and treating minorities (Dabelea et al., 2014).
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this study has been provided by the National Institute of Alcohol Abuse and Alcoholism– 7K23AA017684-04.