Implementing a Modular Research-Supported Treatment in Child Welfare

Abstract

Objective: Increasing rates of mental health needs in child welfare clients highlight the importance of training child welfare workers in effective mental health interventions. This pilot study evaluates the effects of training public child welfare workers and interns in modularized research-supported treatments (RSTs). Methods: We compared knowledge of and attitudes toward RSTs and barriers and facilitators to their use in a sample of 26 field instructors and 11 master of social work students trained in modularized RSTs with an untrained sample of 55 field instructors and 36 students. Results: Trained field instructors had higher knowledge scores at follow-up and were more open to adopting RSTs, as compared with untrained field instructors. Fewer barriers and more facilitators were associated with more positive attitudes toward RSTs across all groups. Conclusions: Interventions to address attitudes, barriers, and facilitators may facilitate the uptake of RSTs in child welfare.

Keywords

research-supported treatments child welfare field instructor common elements

Due to the prevalence of emotional and behavioral problems among child welfare–involved youth, child welfare workers need to have knowledge and training in mental health treatment (Burns et al., 2004). Despite the increasing importance of research-informed mental health practice, child welfare service workers are infrequently delivering research-supported treatments (RSTs; Barth, 2008). Barriers to implementing treatments supported by research are numerous, including access to training (Wike et al., 2014), lack of time (Edmond, Megivern, Williams, Rochman, & Howard, 2006; Rubin & Parrish, 2007; Wiechelt & Ting, 2012), and lack of flexibility in meeting clients’ needs (Rubin & Parrish, 2007; Wiechelt & Ting, 2012).

Common Elements

To fulfill a system need for increasing workforce capacity in using research-supported clinical techniques, Chorpita, Daleiden, and Weisz (2005) analyzed existing treatment literature to distill discrete common practices from the most effective therapeutic interventions. This analysis produced a summary of practices common to the efficacious treatments, called practice elements. Examples of practice elements include time out, goal setting, practicing exposure to anxiety-provoking situations, and cognitive restructuring for depression and anxiety. These practice elements informed the design of treatment approaches that systematically combined these procedures in a guided, modular architecture. A matching technique can then be applied to identify practice elements relevant to effective treatment delivery for individual clients through a process of matching client characteristics (such as problem area, age, and gender) to the characteristics of the youth found in the treatment literature underlying the distillation process (Chorpita, Daleiden, & Weisz, 2005). Mental health practitioners in several states have since been trained in this approach (Barth et al., 2012; Chorpita et al., 2002).

Online resources developed by PracticeWise support knowledge uptake and use of this common elements approach in practice (PracticeWise, 2014). PracticeWise resources include (1) a searchable database with summaries of randomized clinical trials of child mental health treatments (which can be used to automate the matching process described previously, among other searchable queries), (2) Practice Guides, which are codified descriptions of individual practice elements and their specific steps, (3) an Excel-based clinical dashboard that helps to track individual client progress and practice data, and (4) flowcharts that guide and coordinate treatment decisions across various problem behaviors.

Weisz and colleagues (2012) showed that practitioners trained in one example of this “common elements” approach (which involved the delivery of practice elements in a structured yet flexible framework) achieved better client outcomes than those achieved by practitioners trained in standard manualized interventions for the problem areas of anxiety, depression, and disruptive behavior. This modular approach also allowed practitioners to feel more involved in the clinical decision-making process by providing the opportunity to assess complementarity between treatment options and presenting client needs (Borntrager, Chorpita, Higa-McMillan, & Weisz, 2009); in contrast, manualized interventions generally require strict adherence to a particular sequencing and pace of treatment. The common elements model was initially designed for mental health practitioners; however, considering the increasing prevalence rates of mental disorders among child welfare–involved youth (Burns et al., 2004), this pilot study assessed the effects of this common elements approach for child welfare workers.

Barriers to RSTs

Traditional RSTs can have limitations in a service setting like child welfare, where practitioners must typically take a holistic approach to managing the dynamics of their clients. Child welfare services often require social workers to engage in timely, urgent decision making (Collins-Camargo, 2007), which is a commonly identified barrier to implementing treatments supported by research (Edmond et al., 2006; Rubin & Parrish, 2007; Wiechelt & Ting, 2012). Constrained access to relevant and recent literature is also a significant barrier to training on treatments with scientific backing as well as knowledge uptake among social workers (Edmond et al., 2006; Leathers & Strand, 2013; Wiechelt & Ting, 2012; Wike et al., 2014). Due to the lack of empirically supported approaches, there are limited models available to guide social workers through the process of implementing RSTs in child welfare settings (Bledsoe-Mansori et al., 2013). Further, insufficient training may contribute to caseworkers’ fear of selecting the wrong intervention or implementing it inaccurately (Barratt, 2003). To improve uptake, focus groups have shown that field instructors and students would benefit from training on treatments supported by research (Wiechelt & Ting, 2012).

A modular approach is theorized to address these commonly noted barriers of manualized RST implementation. In contrast to the inflexibility of manualized treatments (Rubin & Parrish, 2007; Wiechelt & Ting, 2012) that restrict the level of individualization professionals can use in working with specific cases (Borntrager et al., 2009), a modular approach allows for a more flexible, tailored intervention (Barth, Kolivoski, Lindsey, Lee, & Collins, 2014). Similarly, increasing access to modular interventions that do not require a comprehensive understanding of an entire treatment manual could help streamline child welfare workers’ efforts in the selection and application of research-informed practice.

Attitudes Toward RSTs

Understanding practitioners’ attitudes toward RSTs and providing training strategies and techniques to best meet their needs is essential for enhancing uptake (Aarons et al., 2010). Practitioner buy-in is an essential element of implementation; workers have to be interested and have confidence in the evidence-based practice process (Cunningham & Duffee, 2009; Proctor, 2004), which is commonly defined as a five-step process that includes formulating a question, identifying evidence, critically assessing the evidence, applying in practice, and evaluating the application for improvements to future practice (Barth et al., 2012; Gibbs, 2003). More positive attitudes have been linked to better knowledge of RSTs among mental health practitioners (Nakamura, Higa-McMillan, Okamura, & Shimabukuro, 2011). Also, with leadership that supports its uptake, attitudes toward RSTs were found to improve for child welfare workers (Aarons & Sommerfeld, 2012). Studies have found that practitioners trained in a modular approach have better attitudes toward RSTs in general compared to practitioners trained in standard RSTs (Borntrager et al., 2009).

Infusing Research-Supported Techniques Through Field Instruction

Social work field placements provide an opportunity to reach both field instructors and students to promote RST knowledge and skills (Berger, 2013; Bledsoe-Mansori et al., 2013) and thus are an ideal setting for introducing this content (Grady, 2010). Training early career practitioners is a crucial method of engaging the emerging workforce with an empirically supported treatment approach (Berger, 2013). Several studies have examined the implementation of empirically supported treatment approaches among field instructors and found that there are still barriers to use, regardless of the highly rated utility of these treatments (Edmond et al., 2006; Wiechelt & Ting, 2012), and that allowing students to practice skills learned in the classroom through field instruction could help students select and apply appropriate RSTs (Leathers & Strand, 2013; Wiechelt & Ting, 2012). Rather than training students only in discrete manualized RSTs, providing instruction on the common elements of child mental health can be an opportunity to train a broader menu of implementation techniques.

Despite the importance of positive attitudes and practitioner buy-in (Aarons et al., 2010; Cunningham & Duffee, 2009; Proctor, 2004) toward RSTs, the relationship between knowledge uptake, attitudes, and adoption of these treatments has been mixed. Leathers and Strand (2013) found that training field instructors who work with children or adolescents on modular RSTs resulted in significantly more negative attitudes toward RSTs overall, but negative attitudes did not impact the knowledge uptake or implementation of RSTs. Further, although social work field instructors have expressed positive attitudes toward the use of treatments with empirical support, research has shown that many field instructors newly trained in the EBP process only partially implement the EBP process (Edmond et al., 2006; Wiechelt & Ting, 2012), such as formulating a research question and identifying relevant practice information (Edmond et al., 2006). However, appraising empirically supported interventions, applying relevant interventions in practice, and evaluating treatment outcomes were reported much less frequently (Edmond et al., 2006). These results warrant the need for further research on understanding how to promote full implementation of RSTs among field instructors.

The Current Study

The purpose of this pilot study was to examine field instructors’ and students’ knowledge of and attitudes toward RSTs in the context of service or treatment implementation as well as to assess the barriers and facilitators that influence the use of such treatments. Specifically, the following research questions are examined:

Research Question 1: Does training in the common elements approach lead to improvements in knowledge of and attitudes toward RSTs?

Research Question 2: How do the barriers and facilitators to using RST reported by field instructors and students trained in the common elements compare to those who have not been trained?

Research Question 3: Does the amount of reported barriers and facilitators to using RSTs correlate with trained and untrained participants’ knowledge of or attitudes toward these treatments?

Method

Study Design

This study used a two-group design to compare child welfare field instructors and master of social work (MSW) students who were trained on the common elements approach with those who were not trained. We compared trained and untrained groups on their knowledge of and attitudes toward RSTs as well as how the number of barriers and facilitators identified by participants related to their knowledge and attitudes. The trained group provided data at two time points, that is, pretest and 6-month follow-up. Those who did not choose to participate in the training were recruited into a comparison group and completed a 1-time survey. All procedures received approval from the university’s institutional review board.

Sampling and Recruitment

The sample was recruited using contact information provided by the Field Education Office and Child Welfare Training program at a Mid-Atlantic University. All field instructors in public child welfare (n = 166) and MSW students in child welfare field placements (n = 76) were invited by e-mail to participate in the study (see Figure 1). Of the 166 field instructors invited, 26 (16%) participated in the training and met eligibility criteria for the study , 19 (73%) of which completed 6-month follow-up surveys. Among the remaining 139 untrained field instructors, 55 (40%) completed the comparison group survey. Of the 76 students invited to the training, 11 (14%) participated and met eligibility criteria for the study; of these, 10 (91%) completed the 6-month follow-up. Of the 63 students who did not participate in the training, 36 (57%) completed the comparison group survey with usable data.

Figure 1.

Participant flowchart.

Training

Trained participants received an introduction to the common elements approach as well as an interactive training on the PracticeWise online resources. Both the field instructor and student training formats included hands-on practice experience with the resources; the student training included additional exercises and graded assignments that required navigating the online resources. Field instructors were offered three continuing education units for participation in the half-day training; students received one graduate-level credit for completion of the 2-day training.

Procedure

For the trained group, all participants completed a pretest prior to the training that assessed knowledge of and attitudes toward RSTs and demographics. Six months after the training, participants were sent a follow-up survey that reassessed their knowledge of and attitudes toward RSTs, as well as barriers and facilitators to use of the resources, and updated workplace description. The untrained group completed a survey measuring the same constructs as the trained group (knowledge of and attitudes toward RSTs, barriers and facilitators to using these treatments, and demographics). Both trained and untrained participants were compensated US$20 for survey completion.

Measures

Knowledge of RSTs

To assess participants’ knowledge of RSTs in child mental health, both the trained and the untrained participants were administered the Knowledge of Evidence-Based Services Questionnaire (KEBSQ) that describes 40 therapeutic techniques, like “teaching the child to measure his or her thoughts, emotions, and/or behavior repeatedly” and “encouraging the child to participate in pleasurable activities on a regular basis” (Stumpf, Higa-McMillan, & Chorpita, 2009). For each therapeutic technique, participants identify whether research evidence supports its application for anxious/avoidant behavior, depressed/withdrawn symptoms, disruptive behavior, attention/hyperactivity problems, or none of these. Participants can receive a total of 4 points per item and 160 total possible points for the questionnaire, with higher scores indicating more knowledge. Internal consistency of the KEBSQ was not assessed as each item represents individual techniques that are not expected to correlate and the test–retest reliability resulted only in moderate scores (r = .56; Stumpf et al., 2009). However, formative psychometric testing showed sufficient validity to discriminate between graduate students and practitioners as well as sensitivity to change in pre–posttest designs (Stumpf et al., 2009).

Attitudes toward research-supported practices

To assess attitudes toward research-supported techniques, the Evidence-based Practice Attitudes Scale (EBPAS) was administered to all participants (Aarons, 2004; Aarons et al., 2010). The total scale includes 15 items that measure the extent to which participants agree with each item from 0 (not at all) to 4 (to a very great extent). There are four subscales, that is, (1) appeal (4 items), which includes items such as the extent to which a RSTs made sense and was intuitively appealing; (2) requirements (3 items), which is the extent to which RSTs would be used if required by an agency, supervisor, or state; (3) openness to using RSTs (4 items), which includes items such as willingness to try new types of interventions; and (4) divergence from positive attitudes toward RSTs (4 reverse-scored items), which includes items such as “research-based treatments are not useful.” The EBPAS is scored by computing the average of each subscale; scores can range from 0 to 4 and higher scores indicate more positive attitudes for all subscales aside from divergence, in which lower scores indicate more positive attitudes. The total scale score is calculated by computing an average across all items. The EBPAS was found to have strong Cronbach’s α scores, ranging from .72 to .94 for subscales and the total score (Aarons, 2004; Aarons et al., 2010).

Barriers and facilitators

We identified 17 individual-level and 4 agency-level items in existing literature and measures to serve as an inventory to assess barriers and facilitators for using RSTs. A total of 8 barrier items were suggested by the Workshop EVALuation barrier subscale, which was developed to assess obstacles to implementing training materials (Bartholomew, Joe, Rowan-Szal, & Simpson, 2007). Existing literature on RST implementation suggested additional barriers, such as having access to computers or empirical literature (Edmond et al., 2006), confidence in selecting and implementing RSTs (Bledsoe-Mansori et al., 2013), and flexibility of the approach in meeting client needs (Borntrager et al., 2009; Rubin & Parrish, 2007; Wiechelt & Ting, 2012). As such, 13 additional items were added including the amount of time available, the amount of training you have received on RSTs, how applicable the content is to your clients, and your ability to track long-term client outcomes after case completion. Agency-level barriers included items such as the amount of time or resources your agency has and the amount of flexibility you have in choosing intervention methods. Each item was rated on a 5-point scale: 1 = very much a barrier, 2 = somewhat a barrier, 3 = neither a barrier nor a facilitator, 4 = somewhat a facilitator, and 5 = very much a facilitator. Reliability of the instrument in this study resulted in a Cronbach’s α of .87.

Trained respondents rated the degree to which items were a barrier or facilitator toward using the PracticeWise resources, and comparison groups focused more broadly on the use of RSTs. Of the 21 items, trained and untrained groups were queried about the same18 items for field instructors and 19 items for students; these items were included in the analysis. In addition to calculating an overall scale mean, scores were calculated by summing the total number of barriers (items rated as 1 or 2) and the total number of facilitators (items rated as 4 or 5) identified which yielded a total barrier score and total facilitator score. Items rates as 3 (neither a barrier nor a facilitator) were excluded in either calculation.

Demographic and workplace factors

Participants provided demographic information including age category (18–30, 31–40, 41–50, or 51–60), race (Caucasian, African American, Hispanic or Latino, Asian/Pacific Islander, more than one race, or other), gender (male, female, or transgender), and service unit (out-of-home care, in-home care, adoption, child protective services, older youth or independent living, or other). Responses on the service unit “other” category were reclassified to include administration and school-based work categories. Participants were asked to provide their current caseload size. Responses were categorized into those who have client contact (at least one case) and those who do not (no caseload).

Data Analysis

Data analyses were conducted using SPSS Statistics v.20. Results for descriptive statistics, bivariate analyses comparing trained and untrained groups, and multiple regression models are reported. Missing data on the EBPAS and total scores for barriers and facilitators were minimal (less than 3%). Consistent with methods used in prior research, KEBSQ scores were retained if no more than 20% of items were missing (Nakamura et al., 2011; Okamura, Nakamura, Mueller, Hayashi, & Higa-McMillan, 2014). A total of 14 (12%) participants were excluded for missing more than 20% of data. Remaining missing responses (n = 14, 13%) were determined to be Missing Completely at Random (MCAR), as per Little’s MCAR test (Little & Rubin, 1987), and were then imputed based on expectation maximization algorithms. Due to an error in survey administration of the KEBSQ, eight trained field instructors were not able to select more than one response for 25 of the 40 items that had more than one correct response. In order to ensure that participants were given proper credit for their correct answers, the maximum possible score (4 points) was given for each affected item if the respondent had endorsed one of the correct responses and received no points if the endorsed response was not correct.

Results

Descriptive Statistics and Bivariate Analysis

Descriptive statistics of the sample are provided in Table 1. Most field instructors were between 31 and 40 years (26%), 41 and 50 (32%), or 51 and 65 (32%), and most students were younger, falling in the 18–30 (77%) age bracket. The sample was primarily female (86%) and Caucasian (65%). Bivariate analysis showed a significant association between race and training participation in that there was a higher percentage of Caucasian (71% vs. 49%) and a lower percentage of African American (24% vs. 35%) participants in the untrained group (χ² = 12.35, p = .03). There were no other significant differences between trained and untrained groups on demographics. In relation to service unit, most participants worked in either out-of-home care (32%) or in-home care (32%); 67% of the sample reported having direct client contact.

Table 1.

Sample Descriptives.

	Trained n (%)		Untrained n (%)		χ²
	Field Instructors	Students	Field Instructors	Students	χ²
Age
18–30	2 (7.7)	10 (90.9)	5 (9.1)	26 (72.2)	3.85
31–40	9 (34.6)	1 (7.7)	12 (21.8)	5 (13.9)
41–50	5 (19.2)	—	21 (38.2)	3 (8.3)
51–65	10 (38.5)	—	16 (29.1)	2 (5.6)
Prefer not to say	—	—	1 (1.8)	—
Race/ethnicity
Caucasian	12 (46.2)	6 (54.5)	44 (80.0)	21 (58.3)	12.35*
African American	10 (38.5)	3 (27.3)	9 (16.4)	13 (36.1)
Hispanic/Latino	—	—	—	1 (2.8)
Asian/Pacific Islander	—	1 (9.1)	—	—
More than one race	1 (3.8)	1 (9.1)	—	—
Prefer not to say	3 (11.5)	—	2 (3.6)	1 (2.8)
Gender
Male	4 (15.4)	—	7 (13.5)	3 (8.3)	2.40
Female	21 (80.8)	11 (100.0)	45 (86.5)	33 (91.7)
Transgender	1 (3.8)	—	—	—
Service unit
Out-of-home care	10 (38.5)	2 (18.2)	16 (29.6)	13 (36.1)	3.12
In-home care	8 (30.8)	2 (18.2)	17 (31.5)	13 (36.1)
Child protective services	2 (7.7)	2 (18.2)	4 (7.4)	7 (19.4)
Administrative	2 (7.7)	1 (9.1)	5 (9.10)	—
Older youth/Independent living	2 (7.7)	1 (9.1)	3 (5.6)	1 (2.8)
Adoption	—	—	2 (3.7)	1 (2.8)
School-based work	—	1 (9.1)	1 (1.9)	1 (2.8)
Other	2 (7.7)	2 (18.2)	6 (11.1)	—
Client contact
Yes	13 (68.4)	9 (81.8)	23 (45.1)	33 (91.7)
No	6 (31.6)	2 (18.2)	28 (54.9)	3 (8.3)

Note. χ² values represent the bivariate association between trained or untrained groups and demographic variables.

*p < .05.

Knowledge of RSTs

Field instructors

KEBSQ knowledge scores for field instructors ranged from 16 to 113 of a possible 160 points. Overall, mean KEBSQ scores for child welfare field instructors were low (M = 77.84, SD = 17.65), compared to other studies of social work field instructors (Leathers & Strand, 2013), mental health clinicians (Nakamura et al., 2011), and school-based behavioral health practitioners, psychologists, counselors, and administrators (Stumpf et al., 2009), which ranged from 89 to 116. Trained field instructors’ KEBSQ scores increased but not significantly from pretest (M = 79.62, SD = 10.31) to 6-month follow-up (M = 84.00, SD = 10.54, n = 16). Also, there was no significant difference between trained pretest scores (M = 79.61, SD = 8.50) and untrained scores (M = 75.85, SD = 19.13). When considering all the KEBSQ scores for field instructors, trained field instructors’ scores (n = 17) were significantly higher at follow-up (M = 83.94, SD = 10.21) than those in the untrained comparison group (M = 75.85, SD = 19.13; t = −2.23, p = .03, n = 52) by approximately 8 points, on average. However, when removing one extremely low score among the untrained group, this difference is only a trend for significance (t = −2.00, p = .052).

Students

KEBSQ knowledge scores for students ranged from 19 to 103. Similar to field instructors, mean KEBSQ scores for students were low (M = 78.71, SD = 15.49), compared to other studies (Leathers & Strand, 2013; Nakamura et al., 2011; Stumpf et al., 2009). There were no significant differences in KEBSQ scores from pretest to follow-up for the trained group (M = 72.67, SD = 18.65; M = 84.90, SD = 13.19, respectively), although there was a 12-point average increase. Also, there were no significant differences between trained students at follow-up (M = 84.90, SD = 13.19) and untrained students (M = 77.51, SD = 15.80) on the KEBSQ.

Attitudes Toward RSTs

Field instructors

EBPAS scale scores for field instructors ranged from 1.93 to 3.87 on a 4-point scale, and average scores showed that overall attitudes were high (M = 2.94, SD = 0.44). There were no significant differences in overall attitudes (EBPAS) toward RSTs from pretest to follow-up for trained field instructors or between trained field instructors at follow-up and untrained field instructors (Table 2).

Table 2.

Comparison of Evidence-Based Practice Attitude Scale (EBPAS) Scores.

Scale/Subscale	Trained Pretest, M (SD)	Trained Follow-Up, M (SD)	t ^a	Untrained, M (SD)	t ^b
Field instructors
EBPAS total	3.03 (0.45)	2.94 (0.34)	1.21	2.93 (0.47)	−0.02
EBPAS subscales
Will use if required (requirements)	2.80 (0.84)	2.76 (0.68)	0.28	3.22 (0.81)	2.19*
Open to using (openness)	3.13 (0.68)	2.85 (0.44)	1.97	2.43 (0.80)	−2.80**
Find EBPs appealing (appeal)	3.24 (0.63)	2.94 (0.52)	2.50	3.05 (0.66)	0.59
Disinclination toward EBPs (divergence)	1.10 (0.77)	0.85 (0.68)	2.24*	0.85 (0.61)	0.03
Students
EBPAS total	2.85 (0.47)	2.82 (0.63)	0.28	3.74 (0.59)	−0.37
EBPAS subscales
Will use if required (requirements)	2.93 (0.90)	3.03 (1.24)	−0.36	3.90 (1.91)	−0.46
Open to using (openness)	2.43 (0.78)	2.28 (0.86)	0.61	3.51 (0.77)	0.85
Find EBPs appealing (appeal)	3.13 (0.77)	2.90 (0.75)	1.06	3.86 (0.87)	−0.15
Disinclination toward EBPs (divergence)	1.05 (0.56)	1.86 (0.50)	0.89	2.25 (0.86)	1.37

Note. EBPAS = Evidence-Based Practice Attitude Scale; EBPs = evidence-based practices. Bold-faced font represents significant findings. ^aTest statistic compares the trained group at pretest with the trained group at follow-up. ^bTest statistic compares the trained group at follow-up with the untrained group.

*p < .05. **p < .01.

In considering the four subscales of the EBPAS, the divergence subscale showed significant improvement (i.e., lower scores for reverse scale items) from pretest to follow-up for the trained group, indicating that after training, participants more strongly endorsed RSTs as clinically useful and more important than clinical experience. At follow-up, the trained group differed from the untrained group on two subscales (openness and requirements). The trained group had significantly higher scores on the openness subscale (M = 3.85) compared to the untrained group (M = 3.43, p = .007), and the trained group had significantly lower scores on the requirements subscale (M = 3.76) compared to the untrained group (M = 4.22, p = .03). In other words, trained field instructors were generally open to trying new interventions to a greater extent than those who were untrained; untrained field instructors indicated that they would adopt an RST if it were required for use to a greater extent than trained field instructors.

Students

Similar to field instructors, there were no significant differences in overall attitudes (EBPAS) toward RSTs for trained students from pretest to follow-up or between trained students at follow-up and untrained students. Further, there were no significant differences observed in EBPAS subscales between trained and untrained students at either time point.

Barriers and Facilitators

Field instructors

Using a count of the total number of barriers identified among 18 items, field instructors reported 7.65 barriers (SD = 3.45) on average, which ranged from 0 to 17. For reported frequencies of facilitators to using RSTs, field instructors reported an average of 4.38 (SD = 3.67), which also ranged from 0 to 17.

An analysis of individual items yielded a number of barriers and facilitators that were differentially rated by trained and untrained field instructors. Table 3 provides the average score and results of analyses for each of the 18 items; items that were significantly different across trained and untrained groups are in bold font. Although several items had higher average scores (i.e., more likely to be rated as facilitators) by the trained group compared to the untrained, only a few of these differences were significant. Specifically, the trained group reported their level of training, the simplicity of implementing an intervention, the flexibility of the intervention to meet clients’ needs, and their ability to track long-term client outcomes as greater facilitators to using RSTs when compared to the untrained group. In addition, the trained respondents reported higher ratings of their confidence to select and implement RSTs.

Table 3.

Barriers and Facilitators to Use of Research-Supported Treatments.

	Field Instructors			Students
	Trained, M (SD)	Untrained, M (SD)	t	Trained, M (SD)	Untrained, M (SD)	t
Individual-level items
The amount of time you have	1.63 (0.60)	1.73 (0.85)	0.45	1.89 (1.05)	1.91 (1.14)	0.05
Their fit with your counseling style	3.16 (0.77)	2.96 (0.88)	−0.86	3.00 (0.71)	3.18 (0.90)	0.54
The amount of training you have to use them	2.95 (0.97)	2.20 (1.04)	−2.74**	3.33 (1.32)	2.38 (1.18)	−2.10*
Your access to a computer when information is needed	3.47 (1.26)	3.82 (1.04)	1.18	3.33 (1.41)	3.76 (1.13)	0.97
Your access to empirical articles or journals	3.00 (1.20)	2.91 (1.11)	−0.30	3.44 (0.88)	3.56 (1.13)	0.28
How clear the methods are to use the tools	3.05 (1.03)	2.55 (0.88)	−2.08*	4.11 (0.60)	3.06 (1.07)	−3.87***
How applicable the content is to your clients	3.21 (0.86)	3.00 (1.02)	−0.80	3.44 (1.01)	3.15 (1.16)	−0.70
How relevant the content is to your work	3.32 (1.06)	3.09 (1.06)	−0.80	3.78 (0.97)	3.09 (1.19)	−1.60
How flexible it is to your clients’ needs	3.42 (0.77)	2.89 (0.97)	−2.17*	3.89 (0.78)	2.85 (1.13)	−2.58*
Your confidence in selecting the appropriate approach/intervention	3.42 (0.96)	2.85 (0.88)	−2.37*	3.67 (0.87)	2.74 (1.02)	−2.50*
Your confidence in implementing it	3.61 (0.92)	2.63 (0.96)	−3.81***	3.56 (1.01)	2.76 (1.10)	−1.94
Your coworker’s opinion	3.05 (0.41)	3.17 (0.64)	0.90	3.11 (0.33)	3.21 (0.88)	0.51
Your supervisor’s opinion	3.05 (0.71)	3.24 (0.82)	0.88	3.89 (0.93)	3.41 (1.21)	−1.10
Your field instructor’s opinion	—	—	—	3.78 (0.97)	3.50 (1.21)	−.63
Your ability to track client outcomes (after case completion/transfer)	2.53 (1.02)	2.04 (0.86)	−2.04*	3.44 (1.24)	2.24 (1.21)	−2.66*
Agency-level items
The amount of time or resources your agency has	2.21 (0.79)	1.89 (0.92)	−1.36	2.38 (0.92)	2.24 (1.10)	−0.33
Access to alternative resources	2.37 (0.90)	2.28 (0.92)	−0.37	2.88 (0.64)	2.74 (1.19)	−0.46
How they relate to the treatment philosophy at your agency	2.63 (0.96)	2.56 (1.09)	−0.24	3.25 (0.71)	3.15 (1.11)	−0.25
The amount of flexibility you have in choosing intervention methods	2.53 (0.84)	2.60 (1.26)	0.24	3.13 (0.84)	2.71 (1.03)	−1.07

Note. Range: 1= very much a barrier, 5 = very much a facilitator. Bold-faced font represents significant findings.

*p ≤ .05. **p ≤ .01. ***p ≤ .001.

Students

Of a possible 19 total items, students reported a similar average number of total barriers (M = 6.84, SD = 4.79) and facilitators (M = 6.26, SD = 5.43) and both ranged from 0 to 18. After examining total barriers and facilitators, an analysis of individual items for students was assessed and found to be not significantly different from field instructors. However, trained students rated the following items more highly compared to untrained students: level of training, simplicity of implementing an intervention, the flexibility of the intervention to meet clients’ needs, their ability to track long-term client outcomes, and confidence in selecting the appropriate RST approach. In contrast to field instructors, there were no significant differences between trained and untrained students in regard to their confidence in their ability to implement a RSTs approach.

Regression Analysis

To predict factors associated with knowledge of and attitudes toward RSTs, all trained and untrained field instructors and students with complete data (n = 101) were included in four multiple regression models. The first two multiple regression models assessed factors associated with RST knowledge (KEBSQ). Model 1 included the number of total barriers, attitudes toward RSTs (EBPAS), direct client contact, and whether the participant was trained in the common elements. The overall model was not significant. Model 2 included total facilitators in place of barriers and was also not significant.

The next two regression models assessed factors associated with attitudes toward RSTs; results are provided in Table 4. Model 3 included total barriers, client contact, training in the common elements, and knowledge score (KEBSQ). This model was significant, F(4, 100) = 2.81, p = .03, with a small effect size, R ² = .11 (Cohen, 1988). The number of total barriers was significantly associated with attitudes toward RSTs (B = −0.03, t = −2.20, p = .03), but client contact, training in common elements, and knowledge of RSTs were not significant predictors of attitudes. On average, for each additional barrier identified, attitudes toward RSTs decreased by .03 points.

Table 4.

Multiple Regression Predicting Attitudes Toward Research-Supported Treatments.

	Model 3: EBPAS			Model 4: EBPAS
Variable	B	SE	β	B	SE	β
(Constant)***	2.89	.28		3.48	.24
Client Contact	−.17	.11	−.16	−.23	.10	−.21*
Trained	.06	.12	.05	.07	.12	.06
Knowledge	.004	.003	.12	.004	.003	.14
Total Barriers	−.03	.01	−.22*
Total Facilitators				.04	.01	.21**
R ²	.11*			.16*

Note. EBPAS = Evidence-based Practice Attitudes Scale; SE = standard error. Bold-faced font represents significant findings.

*p < .05. *p < .01. ***p < .001.

Like Model 3, Model 4 included direct client contact, training in the common elements, and knowledge of RSTs but included total facilitators instead of total barriers. This model was significant, F(4, 100) = 4.40, p = .003, with a small effect size, R ² = .16 (Cohen, 1988). The number of total facilitators was significantly associated with attitudes toward RSTs (B = 0.04, t = 3.29, p = .001). Also, having client contact was associated with a significant decrease in attitudes toward RSTs (B = −0.23, t = −2.20, p = .03). Training in common elements and knowledge of RSTs, however, were not significant predictors of attitudes. Results from this regression model suggested that an increasing number of identified facilitators was a significant predictor of more positive attitudes, when controlling for client contact, training in common elements, and knowledge of RSTs (KEBSQ). The coefficient for total facilitators showed that, on average, for each additional facilitator, attitudes toward RSTs increased .04 points. Further, those without direct client contact had attitude scores that were .23 points higher, on average, than those who do have direct client contact.

Discussion and Applications to Social Work

Knowledge of and Attitudes Toward RSTs

Our findings suggest that child welfare field instructors who were trained in the common elements model performed better on a test of their knowledge of research support for various treatment techniques related to child mental health, compared to those who were not trained. This finding may reflect that the common elements resources provided opportunities to learn more about what practice techniques are supported by evidence for various problem areas. Alternately, these higher scores could be explained at least in part by selection bias: Individuals who participated in the common elements training may have had greater interest in mental health treatments and greater eagerness to learn about RSTs.

Similar to prior research on the common elements approach (Leathers & Strand, 2013), results of this study did not find significant increases in knowledge or overall attitudes from pretest to 6-month follow-up for trained field instructors or students. Widespread positive attitudes for most participants at pretest (e.g., mean of 2.89 on a 4-point scale) may have allowed little room for improvement. Although an earlier study (Leathers & Strand, 2013) of MSW field instructors provided with the common elements resources showed worsening attitudes toward RSTs after training, negative attitudes diminished from pretest to posttest for our trained field instructors. Specifically, attitudes on the divergence subscale, which measures the extent to which a practitioner perceives RSTs as not clinically useful, improved significantly at follow-up for field instructors. Training in the modular approach may have helped participants recognize the value of RSTs by making the techniques of efficacious treatments more accessible.

Other attitude subscale results indicate that there may be differences between trained and untrained field instructors in various aspects of RST attitudes. Trained field instructors had attitudes that were generally more open to adopting RSTs, when compared to untrained field instructors. It is possible that the training helped field instructors understand the utility of a flexible approach (Barth et al., 2014), leading field instructors to be more open to trying new interventions. Untrained participants may have a more narrow view of RSTs due to limited training or exposure, and thus their openness to using RSTs may be more guarded. Field instructors and students in the untrained group were more likely to adopt RSTs if required. As such, agency, supervisor, or state mandates may be instrumental in increasing the adoption of RSTs in practice.

Participating in the common elements training had no impact on attitudes toward RSTs for students. Similarities in knowledge of and attitudes toward RSTs for trained and untrained students may be due to having a more open-minded perspective of these treatments. Further, a number of students in both the trained and untrained sample participated in an unrelated training of a RST, an evidence-based practice (motivational interviewing) early in their MSW program. Being trained in a RST may have provided the untrained student comparison group with knowledge about RSTs and positive attitudes about their use.

Barriers and Facilitators of RSTs

Surveying child welfare field instructors and field students about the barriers and facilitators to using RSTs may help shed light on how uptake of RSTs can be enhanced. Results of the regression analysis showed that knowledge of RSTs was not predictive of attitudes, but the number of barriers and facilitators respondents identified was related to their attitudes. Specifically, having identified fewer barriers and more facilitators was predictive of better attitudes toward RSTs. Further, those with client contact had lower attitudes toward RSTs, indicating that being faced with applying RSTs in practice may lead to increased skepticism toward their utility.

Both trained field instructors and trained students identified several items as more facilitative to using RSTs compared to untrained groups. For example, trained field instructors were more likely to identify their level of training as a facilitator to using RSTs. This finding is supported in prior research that has also emphasized the importance of training in uptake of empirically based treatment approaches (Berger, 2013; Bledsoe-Mansori et al., 2013; Wike et al., 2014). Other items rated higher for trained participants included the simplicity of implementing an intervention, the flexibility of the intervention to meet clients’ needs, their ability to track long-term client outcomes, and their confidence in selecting RSTs for use.

Several important aspects of the common elements approach may have contributed to the trained participants’ identification of more facilitators. The modular approach uses a structure that is more applicable to individual client needs, which allows a greater connection between fit, as well as the social worker’s ability to make informed, tailored decisions, which may contribute to increased confidence in the selection and implementation of appropriate RSTs. Specifically, the common elements approach provides structure to client sessions, uses guides for evidence-informed decision making in the treatment process, and allows the practitioner to monitor practice delivery and client outcomes across treatment (Barth et al., 2014; Chorpita, Bernstein, Daleiden, & the Research Network on Youth Mental Health, 2008; Daleiden, Chorpita, Donkervoet, Arensdorf, & Brogan, 2006). In addition, the common elements approach has been noted to provide flexibility for the diverse, individual needs of clients (Barth et al., 2014), which supports findings of this study in which trained participants reported the item “flexibility of the intervention to meet clients’ needs” as more of a facilitator, compared with those who were not trained.

Similarities in how items were rated between students and field instructors indicate that both groups may experience the same challenges and supports. Further, trained field instructors also identified their confidence in implementing RSTs as more of a facilitator than the untrained group, but students did not. This finding may be related to the fact that students have limited caseloads and thus limited opportunities to practice new skills in the field (Berger, 2013), as opposed to field instructors who likely have more practice experience. However, further research is warranted to explore how these findings hold in a larger, representative sample of child welfare workers and across child welfare practice settings.

Strengths and Limitations

This pilot study was the first to examine the effects of the common elements approach in the child welfare sector. In addition, the study included early career student interns and practitioners who regularly interact with these students. Another advantage was our ability to compare those who were trained in the common elements to those who did not participate in the training. By surveying students and field instructors who were not trained, we were able to explore differences in knowledge of and attitudes toward RSTs and variation in identified barriers and facilitators.

However, there are several limitations of this study. As this was a pilot study, the sample was small and response rates were low, which may limit the generalizability of results to the larger child welfare field. As a nonexperimental study, the causal relationship cannot be established, and selection bias due to a nonrandomized sample may have led to differences in training and comparison groups. For example, attitudes toward RSTs may have been inflated, as those with more favorable attitudes may have been more likely to participate in the training. Scores may have also been inflated due to an experimenter demand effect, in which participants may have responded more favorably toward research-informed treatments as this was the focus of the study. There are also limitations to the instruments used in the study, as the KEBSQ had poor test–retest reliability (Stumpf et al., 2009), and the barrier and facilitator questionnaire was developed for use in this study and does not have prior validation; however, 8 of the 21 items have been used in an established measure (Bartholomew et al., 2007). In addition, because test–retest reliability of the EBPAS is unknown, it is possible that the variation in EBPAS between time points could be at least partially explained due to chance variation. In addition, an error in KEBSQ survey administration resulted in estimated total scores for eight field instructors, as they were not able to select multiple response options. This could potentially have led to an inaccurate estimation of their true scores. Finally, many students had already been trained in a manualized RST. As a result, differences between students who were trained and untrained may have been diminished.

Implications for Social Work and Future Directions

Training in a flexible, modular approach to research-informed treatments use may be instrumental in helping practitioners increase their openness to the use of these treatments. The relationship between knowledge of and attitudes toward RSTs and barriers and facilitators to their use may indicate that it is important to not only focus on increasing social workers’ understanding of RSTs but also training should address attitudes toward RSTs as well as common barriers. Bridging the gap between learning about and implementing RSTs regularly in practice with clients remains a challenge. For child welfare workers, whose primary role is not solely to provide mental health services, equipping these professionals to be familiar with research-informed techniques could improve clinical referrals as well as potentially increased effectiveness of their own work with clients. Expanding practitioners’ access to and knowledge of research-informed techniques may increase their uptake.

As this was a pilot study, future research should examine the applicability of the common elements approach in a larger, more representative sample of child welfare workers. Likely, some child welfare units or roles may benefit more than others from training in mental health techniques. Research is also needed to go beyond knowledge and attitudes; actual RSTs use and the effect on children and families must be better understood.

Conclusion

Increasing our understanding of factors that prevent and facilitate the use of RSTs for child welfare workers is essential to increase the uptake of research-informed practice in a public service system. Modular RSTs like the common elements model provide flexibility and allow for high levels of customization, which are valuable traits in child welfare practice. However, training in a modular approach did not increase practitioner knowledge about RSTs techniques even though participants endorsed the importance of RSTs in practice. Understanding how to connect the dots between openness to and implementation of RSTs remains a challenge in child welfare. Findings from this study suggest that training efforts that address attitudes toward RSTs as well as barriers and facilitators may be more useful than efforts solely focused on building knowledge of research-informed practices.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a sub-award to the University of Maryland School of Social Work from a grant from the Center for Medicare and Medicaid Services 1915(c) Community Alternatives to Psychiatric Residential Treatment Facilities National Demonstration Grant Waiver to the Innovations Institute of the School of Medicine, University of Maryland (#10-10226G/M00B040011).

References

Aarons

G. A.

(2004). Mental health provider attitudes towards adoption of evidence-based practice: The evidence-based practice attitude scale (EBPAS). Mental Health Services Research, 6, 61–74. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/15224451

Aarons

G. A.

Hoagwood

Landsverk

Glisson

Kelleher

Cafri

(2010). Psychometric properties and U.S. national norms of the evidence-based practice attitude scale (EBPAS). Psychological Assessment, 22, 356–365. doi:10.1037/a001918

Aarons

G. A.

Sommerfeld

D. H.

(2012). Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. Journal of the American Academy of Child and Adolescent Psychiatry, 51, 423–431. doi:10.1016/j.jaac.2012.01.018

Barratt

(2003). Organizational support for evidence-based practice within child and family social work: A collaborative study. Child and Family Social Work, 8, 143–150. doi:10.1046/j.1365-2206.2003.00276.x

Barth

R. P.

(2008). The move to evidence-based practice: How well does it fit child welfare services? Journal of Public Child Welfare, 2, 145–171. doi:10.1080/ 15548730802312537

Barth

R. P.

Kolivoski

K. M.

Lindsey

M. A.

Lee

B. R.

Collins

(2014). Translating common elements into social work research and education. Journal of Clinical Child and Adolescent Psychology, 43, 301–311. doi: 10.1080/15374416.2013.848771

Barth

R. P.

Lee

B. R.

Lindsey

M. A.

Collins

K. S.

Streider

Chorpita

B. F.

Sparks

J. A.

(2012). Evidence-based practice at a crossroads: The timely emergence of Common Elements and Common Factors. Research on Social Work Practice, 22, 108–119. doi:10.1177/1049731511408440

Bartholomew

N. G.

Joe

G. W.

Rowan-Szal

G. A.

Simpson

D. D.

(2007). Counselor assessments of training and adoption barriers. Journal of Substance Abuse Treatment, 33, 193–199. doi:10.1016/j.jsat.2007.01.005

Berger

(2013). Incorporating EBP in Field Education: Where we stand and what we need. Journal of Evidence-Based Social Work, 10, 127–135. doi:10.1080/15433714.2012.663663

10.

Bledsoe-Mansori

S. E.

Manuel

J. I.

Bellamy

J. L.

Fang

Dinata

Mullen

E. J.

(2013). Implementing Evidence-Based Practice: Practitioner assessment of an agency-based training program. Journal of Evidence-Based Social Work, 10, 73–90. doi:10.1080/15433714.2011.581545

11.

Borntrager

C. F.

Chorpita

B. F.

Higa-McMillan

Weisz

J. R.

(2009). Provider attitudes towards evidence-based practices: Are the concerns with the evidence or with the manuals? Psychiatric Services, 60, 677–681. doi:10.1176/appi.ps.60.5.677

12.

Burns

B. J.

Phillips

S. D.

Wagner

R. H.

Barth

R. P.

Kolko

D. J.

Campbell

Landsverk

(2004). Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 960–970. doi:10.1097/01.chi.0000127590.95585.65

13.

Chorpita

B. F.

Bernstein

A. D.

Daleiden

E. L.

, & the Research Network on Youth Mental Health. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35, 114–123. doi:10.1007/s10488-007-0151-x

14.

Chorpita

B. F.

Daleiden

E. L.

Weisz

J. R.

(2005). Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research, 7, 5–20. doi:10.1007/s11020-005-1962-6

15.

Chorpita

B. F.

Yim

M. L.

Donkervoet

J. C.

Arnsdorf

Amundsen

M. J.

McGee

Morelli

(2002). Toward large-scale implementation of empirically supported treatments for children: A review and observations by the Hawaii Empirical Basis to Services Task Force. Clinical Psychology: Science and Practice, 9, 165–190. doi:10.1111/ j.1468-2850.2002.tb00504.x

16.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

17.

Collins-Comarago

(2007). Administering research and demonstration projects aimed at promoting evidence-based practice in child welfare: Challenges and rewards. Journal of Evidence-Based Social Work, 4, 21–38. doi:10.1300/J394v04n03_03

18.

Cunningham

W. S.

Duffee

D. E.

(2009). Styles of evidence-based practice in the child welfare system. Journal of Evidence-Based Social Work, 6, 176–197. doi:10.1080/15433710802686732

19.

Daleiden

E. L.

Chorpita

B. F.

Donkervoet

C. M.

Arensdorf

A. A.

Brogan

(2006). Getting better at getting them better: Health outcomes and evidence-based practice within a system of care. Journal of the American Academy of Child and Adolescent Psychiatry, 45, 749–756. doi:10.1097/01.chi.0000215154.07142.63

20.

Edmond

Megivern

Williams

Rochman

Howard

(2006). Integrating evidence-based practice and social work field education. Journal of Social Work Education, 42, 377–396. doi: 10.5175/JSWE.2006.200404115

21.

Gibbs

L. E.

(2003). Evidence-based practice for the helping professions: A practical guide with integrated multimedia. Pacific Grove, CA: Brooks/Cole-Thomas Learning.

22.

Grady

M. D.

(2010). The missing link: The role of social work schools and evidence-based practice. Journal of Evidence-Based Social Work, 7, 400–411. doi:10.1080/15433711003591101

23.

Leathers

S. J.

Strand

T. C.

(2013). Increasing access to evidence-based practices and knowledge and attitudes: A pilot study. Research on Social Work Practice, 23, 669–679. doi:10.1177/1049731513491635

24.

Little

R. J. A.

Rubin

D. B.

(1987). Statistical analysis with missing data. New York, NY: John Wiley.

25.

Nakamura

B. J.

Higa-McMillan

C. K.

Okamura

K. H.

Shimabukuro

(2011). Knowledge of and attitudes towards evidence-based practices in community child mental health practitioners. Administration and Policy in Mental Health and Mental Health Services Research, 38, 287–300. doi:10.1007/s10488-011-0351-2

26.

Okamura

K. H.

Nakamura

B. J.

Mueller

Hayashi

Higa-McMillan

C. K.

(2014). An exploratory factor analysis of the Knowledge of Evidence-Based Services Questionnaire. Journal of Behavioral Health Services & Research. Advance online publication. doi:10.1007/s11414-013-9384-5

27.

PracticeWise. (2014). Practice wise evidence based services database. Satellite Beach, FL: PracticeWise, LLC.

28.

Proctor

E. K.

(2004). Leverage points for the implementation of evidence-based practice. Brief Treatment and Crisis Intervention, 4, 227–242. doi:10.1093/brief-treatment/mhh020

29.

Rubin

Parrish

(2007). Views of evidence-based practice among faculty in master of social work programs: A national survey. Research on Social Work Practice, 17, 110–122. doi:10.1177/1049731506293059

30.

Stumpf

R. E.

Higa-McMillan

C. K.

Chorpita

B. F.

(2009). Implementation of evidence-based services for youth: Assessing provider knowledge. Behavior Modification, 33, 48–65. doi:10.1177/0145445508322625

31.

Weisz

J. R.

Chorpita

B. F.

Palinkas

Schoenwald

Miranda

Bearman

S. K.

, … the Research Network on Youth Mental Health. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69, 274–282. doi:10.1001/archgenpsychiatry.2011.147

32.

Wiechelt

S. A.

Ting

(2012). Field instructors’ perceptions of evidence-based practice in BSW field placement sites. Journal of Social Work Education, 48, 577–593. doi:10.5175/JSWE.2012.201000110

33.

Wike

T. L.

Bledsoe

S. E.

Manuel

J. I.

Despard

Johnson

L. V.

Bellamy

J. L.

Killian-Farrell

(2014). Evidence-based practice in social work: Challenges and opportunities for clinicians and organizations. Clinical Social Work Journal, 42, 161–170. doi:10.1007/s10615-014-0492-3