Development and Implementation of a Child Welfare Workforce Strategy to Build a Trauma-Informed System of Support for Foster Care

Abstract

Effective strategies that increase the extent to which child welfare professionals engage in trauma-informed case planning are needed. This study evaluated two approaches to increase trauma symptom identification and use of screening results to inform case planning. The first study evaluated the impact of training on trauma-informed screening tools for 44 child welfare professionals who screen all children upon placement into foster care. The second study evaluated a two-stage approach to training child welfare workers on case planning for children’s mental health. Participants included (a) 71 newly hired child welfare professionals who received a 3-hr training and (b) 55 child welfare professionals who participated in a full-day training. Results from the first study indicate that training effectively increased knowledge and skills in administering screening tools, though there was variability in comfort with screening. In the second study, participants self-reported significant gains in their competency in identifying mental health needs (including traumatic stress) and linking children with evidence-based services. These findings provide preliminary evidence for the viability of this approach to increase the extent to which child welfare professionals are trauma informed, aware of symptoms, and able to link children and youth with effective services designed to meet their specific needs.

Keywords

child welfare workers foster care mental health services assessment child trauma program evaluation

Hundreds of thousands of children and youth across the United States come into contact with the child welfare system due to child maltreatment and neglect every year (U.S. Department of Health and Human Services, 2014), and despite recent downward trends in substantiated child maltreatment rates, there continues to be a large number of children and youth who experienced such significant abuse and/or neglect that they were removed from their home and placed into foster care (e.g., 254,904 children and youth entered foster care in fiscal year 2013; U.S. Department of Health and Human Services, 2014). Many of these youth experienced trauma. Trauma is defined as an event or set of events in which an individual experiences or witnesses injury or death, feels threatened for injury or death (self or others), or has an experience that violates his or her personal integrity. This may happen once (acute trauma) or chronically over a period of time, such as in cases of long-standing sexual abuse (The National Child Traumatic Stress Network [NCTSN], 2015).

Children and youth in foster care have higher prevalence rates of emotional and behavioral disorders compared to the general population (e.g., Casanueva et al., 2012; Horwitz et al., 2012; McMillen et al., 2004), including posttraumatic stress symptoms (e.g., 19.2% compared to 10.7% for those that remain in their home; Kolko et al., 2010). The implications of untreated mental health problems are substantial, such as increased risk for multiple placement moves and decreased likelihood of reunification (Landsverk, Davis, Ganger, Newton, & Johnson, 1996; Newton, Litrownik, & Landsverk, 2000) and heightened risk for ongoing physical and mental health problems (Karr-Morse & Wiley, 2012).

Within the child welfare system, having a trauma-informed system of care is paramount for achieving well-being and positive child welfare-related outcomes. As such, the Department of Health and Human Services, Children’s Bureau Administration for Children and Families (ACF), Centers for Medicare & Medicaid Services, and Substance Abuse and Mental Health Services Administration (SAMHSA) released jointly crafted federal guidelines urging states to adopt a comprehensive approach to screening, functional assessment, and linking children and youth with mental health needs to trauma-informed and evidence-based practices (EBPs; Sheldon, Tavenner, & Hyde, 2013). SAMHSA conceptualizes organizations or systems that are “trauma-informed” as those that understand the impact of trauma on children, youth, and families; are able to identify the signs and symptoms; are equipped to respond through policy, procedures, or practice; and resist retraumatization (SAMHSA, 2015). Further, NCTSN emphasizes the importance of screening, assessment, evidence-based treatment; the provision of educational resources for children, families, and service providers; use of a resilience framework; addressing caregiver trauma and its impact on the family system; ensuring continuity of care across child-serving systems; and minimizing secondary trauma of staff (NCTSN, 2015). However, as is being explored by this special issue, there is little guidance on how to go from a core set of principles or guiding frameworks toward actionable and measurable steps that create a trauma-informed system of care.

In alignment with the above guidance, we focus on operationalizing three important components of a trauma-informed system by ensuring that child welfare professionals (a) are trauma-informed (i.e., understand the impact of trauma on the children and families in their care); (b) engage in trauma identification; and (c) are treatment-informed (i.e., aware of effective, trauma-specific interventions in their community, and advocate for effective treatment). Foundational to these three components is strong communication and collaboration between child welfare and mental health partners.

Trauma-Informed Systems: Understanding Mental Health and the Impact of Trauma

Placing mental health symptoms within a developmental and trauma-informed context is important for ensuring that the well-being of children and youth in the child welfare system is comprehensively addressed (Berliner et al., 2015). Child welfare professionals’ basic knowledge about symptoms of traumatic stress, anxiety, depression, and behavior problems is critical for effectively supporting families (whether biological or foster) to meet the child’s well-being needs (Berliner et al., 2015). While curricula for parents, mental health professionals, the legal system, and others have been developed to meet this need, child welfare professionals typically receive limited training in child development and child mental health (Rakovshik & McManus, 2010). The NCTSN developed a Child Welfare Trauma Training Toolkit (Child Welfare Committee National Child Traumatic Stress Network, 2008). The curriculum is comprehensive and covers many aspects of trauma-informed practice in child welfare. To date it has not been evaluated for impact on changing child welfare systems of practice. Another promising curriculum is Project Focus, a training and consultation strategy designed to increase child welfare professionals’ ability to identify mental health and trauma-related needs and link children to effective services (Dorsey, Kerns, Trupin, Conover, & Berliner, 2012). This is the only mental health curriculum for child welfare professionals of which we are aware that has been evaluated in two small randomized controlled studies (Dorsey et al., 2012; Fitzgerald et al., 2015; Kerns, Dorsey, Trupin, & Berliner, 2010). Findings from these two pilot studies indicated that the curriculum was effective in increasing knowledge about effective practices and appropriately matching mental health needs to services.

Trauma Identification

Identification of traumatic stress symptoms is a necessary step to connecting children and youth with services. Using trauma screens can aid family members and professionals alike to know the types of experiences and impacts of traumatic events experienced by a child. Yet, many jurisdictions do not routinely use such screening tools (Children’s Bureau, 2005). There are numerous high-quality, well-validated tools for trauma exposure and traumatic stress that can be administered by child welfare professionals. For a review of commonly used tools, see Conradi, Wherry, and Kisiel (2011). Currently, there are no research-validated tools in the public domain for young children (aged 3–7). A post-traumatic stress disorder (PTSD) caregiver screen for young children is available but is yet to be validated (Scheeringa & Haslett, 2010).

Treatment-Informed Systems: Linking With Effective Treatment

It is critical that, once needs are identified, children and youth are connected with culturally competent and trauma-informed services that match those needs. Although the use of mental health services by children and youth in foster care is higher than in the general population (Farmer et al., 2001), they may not be matched with services that meet their clinical needs (Garland, Landsverk, & Lau, 2003; J. A. Landsverk, Burns, Stambaugh, & Rolls-Reutz, 2006).

Challenges exist in effectively bridging between mental health screening and referral to provision of services within the community. There are several possible reasons for this, ranging from lack of awareness of available treatments, limited availability of evidence-based treatments, and possible difficulty matching youth with specific treatment providers who are able to effectively address the unique needs of children and youth in care while delivering a culturally competent and evidence-based intervention (e.g., including biological families in treatment, addressing behavioral concerns associated with visitation; Kerns et al., 2014). It is important to attend to each of these challenges as part of a comprehensive approach. Since 2011, the ACF has been funding a series of, typically, large-scale initiatives to provide resources to tackle just this series of challenges (in 2011: integrating trauma-informed and trauma-focused practice in Child Protective Service delivery HHS-2011-ACF-ACYF-CO-0169; in 2012: initiative to improve access to needs-driven, evidence-based/evidence-informed mental and behavioral health services in child welfare HHS-2012-ACF-ACYF-CO-0279; and in 2013: promoting well-being and adoption after trauma HHS-2013-ACF-ACYF-CO-0637).

Washington State Context and Approach

The present study was conducted as part of Creating Connections, a 5-year project in Washington State, funded by the 2012 ACF initiative mentioned above. Consistent with the aims of the federally funded initiative, the focus of Creating Connections is to increase the access to and connection with evidence-based treatments for children and youth in the foster care system through screening, case planning, ongoing progress monitoring, and, if needed, service array reconfiguration. Here, we describe two studies that evaluate strategies to enhance the extent to which the child welfare system is identifying trauma symptoms and is trauma- and treatment-informed. In Study 1, we describe incorporation of a trauma symptoms screen. In Study 2, we describe a two-tier workforce training effort to support use of screening results in case planning.

Study 1

Training and Implementation of Trauma Symptoms Screen

Screening tool

Several screening tools were in place prior to this project, including the Pediatric Symptoms Checklist-17 (PSC-17; Gardner et al., 1999) for youth between 7 and 17 years old and the Global Assessment of Individual Needs—Short Screen (Dennis, Chan, & Funk, 2006) for those 13 to 17 years old. Younger children (through 65 months of age) were screened using the Ages & Stages Questionnaire—Social/Emotional (ASQ-SE; Jee et al., 2010; Squires, Bricker, Heo, & Twombly, 2001). These tools collectively provide information on internalizing symptoms (anxiety, depression); externalizing symptoms (conduct); attentional difficulties; and, for adolescents, substance abuse and suicidal ideations.

As part of the Creating Connections project, an additional screening tool specific for trauma symptoms was added, the Screen for Child Related Anxiety Emotional Disorders (SCARED)–Brief. This measure included a 5-item anxiety subscale and a 4-item PTSD-specific subscale (Muris, Merckelbach, Korver, & Meesters, 2000). It was identified as the most acceptable option to implement statewide because it met the following criteria: (a) research validated for trauma symptoms (Birmaher et al., 1999), (b) short in length (9 items total), and (c) available in the public domain. The PTSD subscale does not screen for specific traumas. The 4 items reference an event that was a “bad thing” and produced specifically connected symptoms such as bad dreams, distressing or intrusive memories, and avoidance. We adopted this approach primarily because, while a majority of children in child welfare are exposed to traumatic events, only a minority develops clinical trauma-specific distress (Kolko et al., 2010). This is the group that needs to be identified for referral to treatment.

Screening approach

Washington State DSHS Children’s Administration (DSHS-CA) employs a robust screening strategy called the Child Health and Education Tracking (CHET) program. This program is designed to assess strengths and areas of need related to health, education, and well-being (including mental health) for children and youth who are expected to be in foster care for 30 days or longer across five life domains (physical/medical, developmental, educational, emotional/ behavioral, and social connections and relationships). A specialized professional (i.e., CHET screener) completes the screens. The screeners are employees of the state child welfare agency; their job responsibilities are solely to conduct the screenings, they do not have an ongoing caseload. There are approximately 45 CHET screeners distributed across the state. The CHET is completed by accessing existing reports, administering screening tools and, when appropriate, conducting interviews with the child/youth and/or supportive people in the child/youth’s life. CHET screeners then complete a report that reflects findings and make individualized service recommendations. This report, designed to support case planning, is shared with the social worker, family, and caregivers. Historically, CHET is administered with high rates of completion (approximately 90% within 30 days of placement into care).

Implementation strategy

The statewide implementation of the SCARED within CHET was initiated in July 2014. In addition, a new program was implemented to provide ongoing screening (called Ongoing Mental Health [OMH] screening program) for a cohort of identified children and youth (aged 3–17) in foster care. Initiated in May 2014, three OMH screeners (located at the state headquarters office) readminister with caregivers by telephone one or more of the following mental health tools depending on the age of the child: ASQ-SE, PSC-17, and SCARED. Screeners have the option of administering the screening directly with youth over the age of 11. These screens are conducted every 6 months.

DSHS-CA revised the CHET database to incorporate the additional trauma screening data and created a complimentary OMH database. The following types of information are available for each child: (a) which tool(s) were administered, (b) who completed the tool, (c) if not complete, the reason why, (d) date of administration, (e) if there were concerns based on the screening scores, (f) if a referral was suggested, and (g) when the child welfare professional was notified. Both databases are able to be uploaded into the statewide SACWIS system for the child welfare professional. Screening results are provided to the caregiver in paper form. Ongoing technical assistance to CHET/OMH screeners is provided primarily by a DSHS-CA supervising program manager and, when needed, faculty from the University of Washington.

Training approach

Prior to statewide implementation of the SCARED, two steps were taken. First, an initial training was conducted with CHET/OMH supervisors. Second, a 2-hr training was provided to all statewide CHET/OMH staff. Training included how to gather, interpret, and share screening data with the child welfare professional. This training occurred within a 2-day conference in which participants also received refresher trainings on the existing screening tools, a session on compassion fatigue, and other topics (e.g., confidentiality).

Study 2

Child Welfare Workforce Training

The project was committed to minimizing the “file drawer problem,” as it relates to screening—where results of screenings are placed in child records but otherwise are rarely reviewed or examined (some reasons for this are discussed by Levitt, 2009). Because child welfare professionals regularly facilitate linkages with services, ensuring that they have a strong understanding of how to interpret the results of screening and use it effectively for case planning is essential (Dorsey et al., 2012; Stiffman et al., 2000; Stiffman, Pescosolido, & Cabassa, 2004). A two-stage training model, based heavily on a previously developed curriculum with demonstrated effectiveness in increasing referrals to EBPs (Project Focus; Dorsey et al., 2012; Fitzgerald et al., 2015; Kerns et al., 2010), includes a 3-hr training required for newly hired child welfare professionals (regional core training [RCT]) and a six-hr, in-depth, skills-focused training for any child welfare professional (In-Service Training [IST]). The trainings are embedded within the state’s child welfare training academy, the Alliance for Child Welfare Excellence, located at the University of Washington, School of Social Work. Curriculum content for the RCT and IST includes: assessing child mental health strengths and needs, trauma-specific content (i.e., prevalence, key signs, symptoms, and impacts of trauma), characteristics and behavior indicators of developmental and mental health concerns, psychotropic medications, use of screening tool results, coordinating mental health services within the community, understanding the elements and criteria of evidence-based and promising practices, matching mental health treatment with specific client needs, and ways to incorporate trauma-informed care into day-to-day work.

Hypotheses

Study 1: Training and implementation of trauma screen

Hypothesis 1: The CHET/OMH training will be associated with an increase in knowledge and skills on administering the PSC-17 and the SCARED.

Hypotheses 2a and b: After training and implementation, CHET screeners will report that trauma screening is effective (the SCARED will improve their perceived ability to identify trauma impact) and acceptable (they will report being confident in using the SCARED).

Hypothesis 3: Administering the trauma screening will be feasible (i.e., burden rated low).

Study 2

Child Welfare Workforce Training

Hypothesis 4: The RCT and IST will be associated with an overall increase in self-reported competency on each item and on total scores.

Hypotheses 5a and b: Compared to child welfare professionals with more experience, those with fewer years of experience will have lower scores on self-reported competency at pretraining and will show significantly more gain in competency from pre- to posttraining.

Hypotheses 6a and b: When compared to those who had previously worked as a mental health professional, child welfare professionals who had never worked as a mental health professional will have lower scores on self-reported competency at pretraining and will show significantly more gain in competency from pre- to posttraining.

Method

The evaluation activities were developed as part of a larger formative evaluation for the Creating Connections project and were intended to provide timely feedback to inform implementation. All activities were approved by the Washington State Institutional Review Board (IRB).

Study 1

Training and Implementation of Trauma Screen

Procedure

Training participants were asked to complete a paper survey at the end of each day of the 2-day training. Six months after the training, a link to an online survey was e-mailed to the CHET program supervisors who forwarded the link to the CHET/OMH screeners in their respective regions, and participants responded within 3 weeks. The 6-month follow-up survey was anonymous, so responses are not linkable to initial responses. Demographics were not collected to ensure anonymity and to keep the measure as brief as possible.

Participants

Fifty participants were invited to the CHET/OMH 2-day training conference and 44 attended. Of those that attended, 44 (100%) participants completed the survey at the end of Day 1, and 43 (97.7%) at the end of Day 2. Thirty-one participants completed the 6-month follow-up survey (70.5% of the 44 who attended at least 1 day of the training). Respondents represented all geographic regions of Washington State.

Measures

Perceived knowledge and skill in administering, scoring, and interpreting the PSC-17 and the SCARED were scored from 0 = complete beginner to 10 = fully expert. A final item collected ratings of the extent participants felt that the SCARED would allow them to identify trauma-related concerns from 0 = not at all to 10 = very much. Posttraining surveys included items about the self-reported amount of knowledge gained (from 0 = not at all to 10 = very much) as a result of the presentations related to adverse childhood experiences/trauma and brain development, compassion fatigue, and administering the ASQ/ASQ-SE.

The 6-month follow-up survey included an item rating current level of knowledge and skills in administering the PSC-17 and SCARED from 0 = complete beginner to 10 = fully expert; items rating the extent to which they used the knowledge and skills gained from the CHET/OMH conference training on the PSC-17 and the SCARED from 0 = never to 10 = every day; an item on how the SCARED has improved their ability to identify trauma-related concerns from = 0 not at all to 10 = very much; an item on their level of confidence in continuing to use the tool from 1 = reticent to 3 = comfortable to 5 = confident, and items asking estimates for the average number of minutes the SCARED adds to each CHET interview for administration of the measure, discussing results with caregivers and youth, data entry, and report writing. These items were asked separately for using the measure with caregivers/teachers versus youth. The survey had space for open-ended feedback about the SCARED measure.

Study 2

Child Welfare Workforce Training

Procedure

RCT and IST trainers explained the purpose of the survey, provided informed consent, and distributed paper surveys to at the beginning and end of each training.

Participants

Between June 2014 and April 2015, a total of 71 participants in 10 separate trainings completed the RCT and 55 participants in 5 separate trainings completed an IST; for both there was a 100% response rate (see Table 1 for demographics).

Table 1.
Study 2: Demographic Characteristics of Participants.

Variable Regional Core Training In-Service Training

Characteristic n Percentage n Percentage

Gender

Female 57 80.3 47 85.5

Male 13 18.3 8 14.5

Missing 1 1.4 0 0

Race/ethnicity

African American 5 7.0 3 5.5

Caucasian 39 54.9 38 69.1

Hispanic/Latino(a) 8 11.3 2 3.6

Asian/Pacific Islander 1 1.4 1 1.8

American Indian/Alaska native 1 1.4 0 0

Mixed 9 12.7 3 5.5

Other 2 2.8 0 0

Missing 5 7.0 8 14.5

Ever worked as a mental health provider

Yes 21 29.6 15 27.3

No 49 69.0 39 70.9

Missing 1 1.4 1 1.8

Current position

Social service specialist^a 64 90.2 22 40.0

Other^b 6 8.4 20 36.1

Missing 1 1.4 13 23.6

Regional core training In-service training

Mean SD n Mean SD n

Years in current position 0.3 1.2 70 4.7 4.3 54

Years worked in child welfare 3.2 5.4 70 10.2 9.8 54

^aSocial service specialist includes “social worker,” “social service specialist in training,” “social service specialist II,” and “social service specialist III.” ^bPositions with less than two participants were grouped into “Other.”

Measures

The survey instruments were developed in collaboration with the Alliance for Child Welfare Excellence to reflect the comprehensive list of competencies that child welfare professionals are expected to obtain within the first 3 months (for the RCT) or similar questions reflecting a deeper level of understanding and skills (for the IST). In addition to competency-specific items, project-specific items were added (e.g., “use and interpret the new trauma screening tool [SCARED]”). A root question asked for participants to “rate your comfort with your ability to do the following,” was followed by the questionnaire items. Ratings were from 0 = not at all comfortable to 10 = completely comfortable. A final question gathered self-reported knowledge about mental health, in general, from 0 = complete beginner to 10 = fully expert. For the first 9 months of training, the competencies were still under development. Thus, proposed competencies were used for the evaluation between March and November 2014 (24 items for RCT, 17 items for IST) and a finalized list of competencies was used from November 2014 onward (21 items for RCT, 23 items for IST). Specific information about the competencies and changes during this time period is available from the first author. We created a dichotomous measure of “experience in child welfare.” For ease of interpretation, “more experience” in the RCT was defined as having worked in child welfare above the sample median of 8 months for the RCT and 108 months for the IST (alternative analyses operationalizing experience as a continuous variable found results fully consistent with those described below).

Analysis

Study 1

Training and Implementation of Trauma Screening

Descriptive statistics were run for all items. Paired t-tests were used to examine changes in self-reported knowledge and skills from pre- to posttraining on administering the PSC-17 and SCARED. Thematic coding was used to summarize open-ended comments.

Study 2

Child Welfare Workforce Training

Paired samples t-tests explored changes in reported competency for every item and for total scale z-score from pre- to posttraining. Separate analyses were conducted on RCT and IST. Identical items from the proposed and finalized competency versions were analyzed by pooling data. Participants with missing data were excluded from item-level analyses. The amount of missing data was low; 100% of items were complete for at least 89% of the sample. Total scores were created by taking the average of the item scores and, for items with missing data, imputing participant-level item means, if participants completed 85% or more of the items.

Repeated measures analysis of variances with Experience × Time interaction terms were used for moderator analyses. Total measure z-scores from pooled pre- and postresponses and combined proposed and finalized competency versions were used to maximize statistical power to detect interactions by including all available data, despite changes in item wording. Differences between the groups on these moderators (previous experience as a mental health professional and amount of experience in child welfare) at pre- and posttraining were tested using independent samples t-tests. A Bonferroni correction was used due to the risk of familywise error, which sets the significance value for the RCT at p < .00135 and the IST at p < .0025.

Results

Study 1

Training and Implementation of Trauma Screening

Hypothesis 1 was supported. CHET/OMH worker self-reported knowledge and skills on administering screening measures significantly increased from pre- to posttraining for the PSC-17 (M_pre = 7.5, SD = 2.1; M_post = 8.6, SD = 1.5; t₍₄₁₎ = −4.4, p < .001) and the SCARED (M_pre = 3.0, SD = 3.4; M_post = 7.0, SD = 2.0; t₍₃₈₎ = −6.9, p < .001). Hypothesis 2a received some support. CHET/OMH workers reported that they believed that the SCARED would moderately improve their ability to identify trauma impact (M = 6.2, SD = 2.8). Participants reported being comfortable incorporating the training on compassion fatigue into their day-to-day work (M = 7.7, SD = 1.9), and that they gained a moderate level of knowledge during training sessions about adverse childhood experiences and trauma (M = 6.3, SD = 2.5), compassion fatigue (M = 6.3, SD = 2.4), and administering the ASQ/ASQ-SE (M = 4.9, SD = 3.2).

At the 6-month follow-up CHET/OMH training survey, participants reported high levels of knowledge and self-reported skills on administering the PSC-17 (M = 9.2, SD = 1.1) and the SCARED (M = 8.3, SD = 1.4). Participants reported that they used the knowledge and skills they gained from the training at a mean of 6.3 (SD = 3.7) for the PSC-17 and a mean of 7.0 (SD = 2.5) for the SCARED. Hypothesis 2a received some qualified support. Participants reported that integrating the SCARED into the CHET moderately improved their ability to identify trauma-related concerns (M = 4.9, SD = 3.3). However, these responses varied dramatically, with a trimodal distribution: 37% of participants rating this item as low with a 0–3, 27% of participants rating this exactly in the middle range with a 5, and 36% rating this item as high with 7–10. Hypothesis 2b also received qualified support. Participants were asked what they thought about continuing to use the SCARED (on a 1–5 scale). The mean score was 2.7 (SD = 1.7) but the distribution was bimodal, with 57% of participants responding with a 1 or 2, and 40% of participants responding with a 4 or 5. Due to the nature of exclusively using self-reported data, further exploration of the bimodal and trimodal distribution is limited.

An examination of open-ended comments on the survey uncovered some possible explanations for this mixed response. Many participants expressed their enthusiasm for the new tool; strengths that were described included a belief that the trauma tool helps caregivers understand more about the child’s emotional health and needs: “I am so glad we are using the trauma tool, so we are not missing a child’s inner experience of trauma, just because they outwardly appear to be coping.” Other participants noted using the trauma tool helped to initiate sensitive discussions “The trauma tool appears to be doing well for me and the families/children I work with … The questions do help open up some children into asking more questions to assess if a mental health evaluation is needed.” And lastly: “… I think SCARED is simple and quick.”

Some participants expressed concerns about the implementation of the training and the use of the SCARED measure. Several noted wanting additional training time, and others reported that the concerns and apprehensions regarding the tool were not fully addressed at the time of the training. Concerns were noted about some items seeming confusing or repetitive—especially when administered with other screens, and that the tool appeared to lack face validity for the child welfare population. For example, one respondent indicated “Some children have disclosed that ‘the very bad thing’ that happened to them are trivial things such as getting a shot (from the doctor) or having the car door hit their finger.” Other screeners indicated that the tool does not improve their ability to gather new information: “… the use of the trauma tool has not yielded any useful information that cannot be gathered through skilled interviewing and the tools we are already using.” Finally, concerns were expressed about asking directly about trauma. One responded, “For some children, mention of trauma at the home visit elevates their anxiety and requires de-escalation for the child to return to baseline status prior to the home visit ending.”

Participants reported that the administration, discussion of, and report writing for the SCARED added a mean of 32.6 min (Median = 30.5, SD = 18.0) to the CHET with adults and a mean of 34.3 min (Median = 27.0, SD = 21.5) with children or youth. The median numbers of minutes for each component were administration (Median_adult = 5.0, Median_youth = 7.5), discussing results (Median_adult = 5.0, Median_youth = 5.0), data entry (M_adult = 5.0, Median_youth = 5.0), and report writing (Median_adult = 7.8, Median_youth = 6.8).

Study 2a

Child Welfare Workforce Training (RCT)

Hypothesis 4 was supported. For the RCT, self-reported competency scores on nearly all items, including the total item score, significantly improved from pre- to posttraining, as did the mean z-score combined across proposed and finalized versions of the competencies (.99 SD change from pre- to posttraining). The only item that did not show significant improvement (p = .058), “collaborate with mental health and related resources to coordinate care on behalf of children” also had the highest average pretraining score (7.93 of 10), potentially creating a ceiling effect. The largest improvement was related to using information from the CHET and the SCARED (mean change scores ranged from 3.15 to 3.3). The second largest improvements were found for knowing about the elements of EBPs, how to refer to EBPs, and the knowledge of EBPs in their region (mean change scores 1.87–3.23).

Study 2b

Child Welfare Workforce Training (IST)

Self-reported competency scores on all items on the IST significantly improved from pre- to posttraining, including total item means. The mean z-score of combined data from participants completing the proposed and final versions of the competencies had a .98 SD improvement from pre- to posttraining. The largest change was the reported ability to use and interpret the SCARED (mean change score = 3.75); other items with large change scores included reported knowledge of which EBP’s best match client needs, ability to identify and describe the elements and criteria of EBPs/promising practices, basic knowledge of EBPs in their region, and knowledge of how to provide detailed information about expected outcomes to service providers (mean change scores ranged from 2.5 to 2.12). Details on the specific competencies for the RCT and IST are available from the first author.

Study 2

Moderators

Hypotheses 5a and b received partial support. Participants in the RCT (and with borderline significance, those in the IST) who indicated having more experience in child welfare reported significantly higher preassessment knowledge of training competencies compared to those who had less experience (see Table 2 and Figure 1). More child welfare experience was a significant moderator for the RCT trainings: Interaction tests found that those with less experience had significantly sharper improvement rates from pre- to posttraining, when compared to those participants with this experience. For both RCT and ISTs, there were no significant mean differences in knowledge remaining between the groups at posttest.

Table 2.
Pre–post Change Over Time in Knowledge and Skills Moderated by Prior Mental Health and Child Welfare Experience.

Regional Core Trainings

Measures Pre Post F Difference in slopes p value

No Yes t-test p value No Yes t-test

Mean SD n Mean SD n Mean SD n Mean SD n

Ever worked as a mental health professional −0.72 1.04 49 0.05 0.74 20 .004 0.38 0.73 49 0.77 0.52 20 0.015 4.06 .048

Less experience More experience t-test p value Less experience More experience t-test F Difference in slopes p value

Mean SD n Mean SD n Mean SD n Mean SD n

Experience working in child welfare −0.73 1.04 35 −0.25 0.94 34 .04 0.40 0.73 35 0.59 0.66 34 0.265 3.098 .044

In-Service Training

Measures Pre Post F Difference in slopes p value

No Yes t-test p value No Yes t-test p value

Mean SD N Mean SD N Mean SD n Mean SD n

Ever worked as a mental health professional −0.50 0.86 37 0.29 0.93 11 .001 0.25 0.92 37 0.73 0.47 11 0.24 1.33 .26

Less experience More experience t-test p value Less experience More experience t-test p value F Difference in slopes p value

Mean SD n Mean SD n Mean SD n Mean SD n

Experience working in child welfare −0.45 0.99 24 –0.19 0.88 24 .096 0.32 0.89 24 0.39 0.85 24 0.81 0.65 .42

Figure 1.
Pre- and posttraining total item z-scores, stratifying by high versus low experience in child welfare and whether the participant had ever worked as a mental health professional for regional core trainings (left) and in-service trainings (right).

Hypotheses 6a and b were also partially supported. Participants in both the RCT and the IST who indicated having previously worked as a mental health professional reported significantly higher preassessment knowledge of training competencies compared to those who had not worked as a mental health provider (see Table 2 and Figure 1). Prior experience as a mental health professional was a significant moderator for the RCT training: Interaction tests found that those without this experience had a significantly sharper improvement rates from pre- to posttraining when compared to those participants with this experience. For the RCT, those who had not worked as a mental health professional still had significantly lower scores posttraining but for the IST there were no significant differences at posttraining.

Discussion

We explored strategies to enhance the extent to which the child welfare system in Washington State is trauma- and treatment-informed, including strategies to enhance trauma identification. Results from Study 1, the training and implementation of the SCARED trauma screening tool, were overall positive, indicating the feasibility and general acceptability of the approach. However, particularly for training on screening tool administration, opinions were mixed on the satisfaction with and willingness to continue using the SCARED. At 6 months after the training was conducted, roughly a third reported that the SCARED did not significantly improve their ability to identify trauma symptoms, a third reported that it was very helpful at improving their ability, and a third were “on the fence.” Over half reported either reticence or low enthusiasm for continuing to use the SCARED. Some of the reasons cited in open-ended responses by those with lower enthusiasm included feeling like it did not provide new information beyond what was already known, questions around the ability of children to understand and respond appropriately to the measure, and concerns about asking directly about trauma. Perceived barriers were practical (e.g., time, technology, and resources) and perceptual (e.g., concern about “opening Pandora’s box”). These concerns are not unique to this study or this professional group. Similar concerns have been raised by primary care practitioners (e.g., Lee, Coles, Lee, & Kulkarni, 2012) and juvenile justice staff (Kerig 2013). Monitoring attitudes and opinions is important for tailoring ongoing support and assistance. When difficulties are noted, having strategies to mitigate concerns must be employed (e.g., targeted technical assistance, additional training opportunities). A year from the start date, anecdotal feedback from CHET staff indicates that there may be substantially less concern around the administration of the SCARED tool. While some concerns remained (i.e., children aren’t always able to discern between a traumatic event and a minor mundane event), other concerns appear to have diminished (e.g., a fear that children may decompensate when asked about trauma).

Results from Study 2, which evaluated the impact of two workforce training approaches, found that workers significantly increased their self-reported competencies across nearly the entire range of critical areas for addressing the mental health need for children and youth in child welfare. Furthermore, there was a significant interaction effect, whereby workers with less experience in the field or no prior experience providing mental health services had much greater gains in knowledge as a result of the trainings than their more experienced counterparts. For the more intensive, skills-focused training opportunity, the two groups were not significantly different at posttraining. This is a feasible approach to workforce training that could be readily adopted by training academies.

The workforce training approaches undertaken by this project incorporated a number of adult learning strategies to enhance transfer of knowledge. The RCT, followed within a year by the IST, represents opportunities for scaffolding knowledge—child welfare professionals first gain exposure to the concepts and then follow with a more in-depth skills focused learning opportunity. We used case-based vignettes based on actual child welfare cases to ensure applicability to the families being served and opportunities to apply learning principles to different ages, cultural backgrounds, and emotional and behavioral concerns. These approaches are aligned with those strategies outlined by Steinberg and Vinjamuri (2014) about how to achieve Council on Social Work Education competencies (e.g., relevance, problem-focused, and scaffolding). For the IST, a range of teaching strategies were incorporated, including use of videos, small and large group discussions, and case presentations. Information presented was practical and based on maximizing use of tools readily available (e.g., demonstrating where information about EBPs is on the administration’s intranet; using examples from existing screening tools). Our approach, however, currently does not include a formal coaching component, which is considered a critical strategy to support generalization of knowledge (Beidas et al., 2013; Dorsey et al., 2012).

Study Limitations

The results presented in these two studies are all self-report. While perceived improvements in self-confidence and skills are critical to establish, the studies could be further enhanced through case reviews, skill demonstrations, and longer term follow-up to establish maintenance of gains. The self-report data further limited our ability to examine possible other factors associated with the different distributions of satisfaction with the SCARED measure. We did not have a comparison group for either study. Therefore, we cannot definitively state that the changes observed were attributable to the training and not some other factor.

There are a number of considerations that may impact generalizability. The SCARED measure was embedded within a highly functioning screening system that is unique to Washington State. The implementation team leveraged an existing database infrastructure, thus avoiding the need to manage challenges associated with creation of a new system from scratch. Therefore, the results from our efforts may represent a “best case” scenario for other sites.

Summary and Future Directions

Identifying feasible strategies that enhance system responsiveness to trauma-related distress and posttraumatic stress symptoms is critical. These studies describe two such strategies that can be embedded within the child welfare system. However, to better understand the impacts of such efforts, connecting these efforts to documented changes in the mental health care and course of child welfare involvement will be paramount. In the future, we plan to document system-level changes in the extent to which the process from identification to referral to treatment is influenced. We also plan to examine systematic differences in the pathways between screening, referral, and receipt of services, such as presence of a developmental disability, extent of symptoms, and other demographic information.

While the current paper focused on the mental health and child welfare systems, future work should also involve medical providers. According to Zlotnik and colleagues (2015), collaboration across the health-care system and child welfare system may be more limited, once the child is involved with the child welfare system. Given this time of health system transformation and combined physical and behavioral health purchasing, it is critical that this linkage be made and maintained. Like child welfare, challenges related to screening exist in the medical office such as time to screen, how to screen using a trauma-informed approach, who should have the results of the screening, and where to refer or how existing screening should interface with the medical provider.

Variable	Regional Core Training	In-Service Training
Characteristic	n	Percentage	n	Percentage
Gender
Female	57	80.3	47	85.5
Male	13	18.3	8	14.5
Missing	1	1.4	0	0
Race/ethnicity
African American	5	7.0	3	5.5
Caucasian	39	54.9	38	69.1
Hispanic/Latino(a)	8	11.3	2	3.6
Asian/Pacific Islander	1	1.4	1	1.8
American Indian/Alaska native	1	1.4	0	0
Mixed	9	12.7	3	5.5
Other	2	2.8	0	0
Missing	5	7.0	8	14.5
Ever worked as a mental health provider
Yes	21	29.6	15	27.3
No	49	69.0	39	70.9
Missing	1	1.4	1	1.8
Current position
Social service specialist^a	64	90.2	22	40.0
Other^b	6	8.4	20	36.1
Missing	1	1.4	13	23.6
	Regional core training	In-service training
Mean	SD	n	Mean	SD	n
Years in current position	0.3	1.2	70	4.7	4.3	54
Years worked in child welfare	3.2	5.4	70	10.2	9.8	54

Regional Core Trainings
Ever worked as a mental health professional	−0.72	1.04	49	0.05	0.74	20	.004	0.38	0.73	49	0.77	0.52	20	0.015	4.06	.048
	Less experience	More experience	t-test p value	Less experience	More experience	t-test	F	Difference in slopes p value
Mean	SD	n	Mean	SD	n	Mean	SD	n	Mean	SD	n
Experience working in child welfare	−0.73	1.04	35	−0.25	0.94	34	.04	0.40	0.73	35	0.59	0.66	34	0.265	3.098	.044
In-Service Training
Measures	Pre	Post	F	Difference in slopes p value
No	Yes	t-test p value	No	Yes	t-test p value
Mean	SD	N	Mean	SD	N	Mean	SD	n	Mean	SD	n
Ever worked as a mental health professional	−0.50	0.86	37	0.29	0.93	11	.001	0.25	0.92	37	0.73	0.47	11	0.24	1.33	.26
	Less experience	More experience	t-test p value	Less experience	More experience	t-test p value	F	Difference in slopes p value
Mean	SD	n	Mean	SD	n	Mean	SD	n	Mean	SD	n
Experience working in child welfare	−0.45	0.99	24	–0.19	0.88	24	.096	0.32	0.89	24	0.39	0.85	24	0.81	0.65	.42

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding was provided by the Children’s Bureau, Administration of Children, Youth, and Families, Grant #90C01103/03.

References

Beidas

Edmunds

Cannuscio

Gallagher

Downey

Kendall

(2013). Therapists perspectives on the effective elements of consultation following training. Administration and Policy in Mental Health and Mental Health Services Research, 40, 507–517. doi:10.1007/s10488-013-0475-7

Berliner

Fitzgerald

M. M.

Dorsey

Chaffin

Ondersma

S. J.

Wilson

(2015). Report of the APSAC task force on evidence-based service planning guidelines for child welfare. Child Maltreatment, 20, 6–16. doi:10.1177/1077559514562066

Birmaher

Brent

D. A.

Chiappetta

Bridge

Monga

Baugher

(1999). Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): A replication study. Journal of the American Academy of Child & Adolescent Psychiatry, 38, 1230–1236. doi:10.1097/00004583-199910000-00011

Casanueva

Wilson

Smith

Dolan

Ringeisen

Horne

(2012). NSCAW II wave 2 report: Child well-being (OPRE Report No. 2012-38). Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, US Department of Health and Human Services. Retrieved from http://www.acf.hhs.gov/sites/default/files/opre/nscaw_report_w2_ch_wb_final_june_2014_final_report.pdf

Children’s Bureau. (2005). General findings from the federal child and family services review. U.S. Department of Health and Human Services. Retrieved from http://www.acf.hhs.gov/sites/default/files/cb/summary_of_the_results_of_the_2001_2004_cfsr.pdf. doi:10.1177/088626000015011007

Child Welfare Committee National Child Traumatic Stress Network. (2008). Child welfare trauma training toolkit: Comprehensive guide (2nd ed.). New York, NY: National Center for Child Traumatic Stress.

Conradi

Wherry

Kisiel

(2011). Linking child welfare and mental health using trauma-informed screening and assessment practices. Child Welfare, 90, 129–147. Retrieved from http://www.cwla.org/child-welfare-journal/

Dennis

M. L.

Chan

Y. F.

Funk

R. R.

(2006). Development and validation of the GAIN short screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults. American Journal on Addictions, 15, 80–91. doi:10.1080/10550490601006055

Dorsey

Kerns

S. E. U.

Trupin

E. W.

Conover

K. L.

Berliner

(2012). Child welfare caseworkers as service brokers for youth in foster care: Findings from project focus. Child Maltreatment, 17, 22–31. doi:10.1177/1077559511429593

10.

Farmer

E. M.

Burns

B. J.

Chapman

M. V.

Phillips

S. D.

Angold

Costello

E. J.

(2001). Use of mental health services by youth in contact with social services. Social Service Review, 75, 605–624. doi:10.1086/323165

11.

Fitzgerald

M. M.

Torres

M. M.

Shipman

Gorrono

Kerns

S. E. U.

Dorsey

(2015). Child welfare caseworkers as brokers of mental health services: A pilot evaluation of project focus Colorado. Child Maltreatment, 20, 37–49. doi:10.1177/1077559514562448

12.

Gardner

Murphy

Childs

Kelleher

Pagano

Jellinek

… Chiappetta

(1999). The PSC-17: A brief pediatric symptom checklist with psychosocial problem subscales. A report from PROS and ASPN. Ambulatory Child Health, 5, 225. Retrieved from http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1467-0658

13.

Garland

A. F.

Landsverk

J. A.

Lau

A. S.

(2003). Racial/ethnic disparities in mental health service use among children in foster care. Children and Youth Services Review, 25, 491–507. doi:10.1016/S0190-7409(03)00032-X

14.

Horwitz

S. M.

Hurlburt

M. S.

Goldhaber-Fiebert

J. D.

Heneghan

A. M.

Zhang

Rolls-Reutz

… Stein

R. E. K.

(2012). Mental health services use by children investigated by child welfare agencies. Pediatrics, 130, 861–869. doi:10.1542/peds.2012-1330

15.

Jee

S. H.

Conn

A.-M.

Szilagyi

P. G.

Blumkin

Baldwin

C. D.

Szilagyi

M. A.

(2010). Identification of social-emotional problems among young children in foster care. Journal of Child Psychology and Psychiatry, 51, 1351–1358. doi:10.1111/j.1469-7610.2010.02315.x

16.

Karr-Morse

Wiley

M. S.

(2012). Scared sick: The role of childhood trauma in adult disease. New York, NY: Basic Books.

17.

Kerig

P. K.

(2013). Trauma-informed assessment and intervention. Los Angeles, CA: National Center for Child Traumatic Stress.

18.

Kerns

S. E. U.

Dorsey

Trupin

E. W.

Berliner

(2010). Project focus: Promoting emotional health and well-beng for youth in foster care through connections to evidence-based practices. Report on Emotional and Behavioral Disorders of Youth, 10, 30–38. doi:10.1177/1077559511429593

19.

Kerns

S. E. U.

Pullmann

M. D.

Putnam

Buher

Holland

Berliner

… Trupin

E. W.

(2014). Child welfare and mental health: Facilitators of and barriers to connecting children and youths in out-of-home care with effective mental health treatment. Children and Youth Services Review, 46, 315–324. doi:10.1016/j.childyouth.2014.09.013

20.

Kolko

D. J.

Hurlburt

M. S.

Zhang

Barth

R. P.

Leslie

L. K.

Burns

B. J.

(2010). Posttraumatic stress symptoms in children and adolescents referred for child welfare investigation: A national sample of in-home and out-of-home care. Child Maltreatment, 15, 48–63. doi:10.1177/1077559509337892

21.

Landsverk

Davis

Ganger

Newton

Johnson

(1996). Impact of child psychosocial functioning on reunification from out-of-home placement. Children and Youth Services Review, 18, 447–462. doi:10.1016/0190-7409(96)00014-X

22.

Landsverk

J. A.

Burns

B. J.

Stambaugh

L. F.

Rolls-Reutz

J. A.

(2006). Mental health care for children and adolescents in foster care: Review of research literature. Retrieved from http://www.casey.org/Resources/Publications/pdf/MentalHealthCareChildren.pdf

23.

Lee

Coles

Lee

Kulkarni

(2012). Primary healthcare practitioners’ screening practices and attitudes towards women survivors of child abuse. Mental Health in Family Medicine, 9, 181–189.

24.

Levitt

J. M.

(2009). Identification of mental health service need among youth in child welfare. Child Welfare: Journal of Policy, Practice, and Program, 88, 27–48.

25.

McMillen

J. C.

Scott

L. D.

Zima

B. T.

Ollie

M. T.

Munson

M. R.

Spitznagel

(2004). Use of mental health services among older youths in foster care. Psychiatric Services, 55, 811–817. doi:10.1176/appi.ps.55.7.811

26.

Muris

Merckelbach

Korver

Meesters

(2000). Screening for trauma in children and adolescents: The validity of the traumatic stress disorder scale of the screen for child anxiety related emotional disorders. Journal of Clinical Child Psychology, 29, 406–413. doi:10.1207/S15374424JCCP2903_11

27.

Newton

R. R.

Litrownik

A. J.

Landsverk

J. A.

(2000). Children and youth in foster care: Disentangling the relationship between problem behaviors and number of placements. Child Abuse & Neglect, 24, 1363–1374. doi:10.1016/S0145-2134(00)00189-7

28.

Rakovshik

S. G.

McManus

(2010). Establishing evidence-based training in cognitive behavioral therapy: A review of current empirical findings and theoretical guidance. Clinical Psychology Review, 30, 496–516. doi:10.1016/j.cpr.2010.03.004

29.

Scheeringa

Haslett

(2010). The reliability and criterion validity of the diagnostic infant and preschool assessment: A new diagnostic instrument for young children. Child Psychiatry & Human Development, 41, 299–312. doi:10.1007/s10578-009-0169-2

30.

Sheldon

G. H.

Tavenner

Hyde

P. S.

(2013, 7 11). Letter to state director. Department of Health and Human Services. Retrieved from http://www.medicaid.gov/Federal-Policy-Guidance/Downloads/SMD-13-07-11.pdf

31.

Squires

Bricker

Heo

Twombly

(2001). Identification of social-emotional problems in young children using a parent-completed screening measure. Early Childhood Research Quarterly, 16, 405–419. doi:10.1016/S0885-2006(01)00115-6

32.

Steinberg

D. M.

Vinjamuri

M. K.

(2014). Activating adult-learning principles through small groups in preparing social work students to achieve CSWE research competencies. Journal of Teaching in Social Work, 34, 363–383. doi:10.1080/08841233.2014.937890

33.

Stiffman

A. R.

Hadley-Ives

Doré

Polgar

Horvath

V. E.

Striley

Elze

(2000). Youths’ access to mental health services: The role of providers’ training, resource connectivity and assessment of need. Mental Health Services Research, 2, 141–154. doi:10.1023/A:1010189710028

34.

Stiffman

A. R.

Pescosolido

Cabassa

L. J.

(2004). Building a model to understand youth service access: The gateway provider model. Mental Health Services Research, 6, 189–198. doi:10.1023/B: MHSR.0000044745.09952.33

35.

Substance Abuse and Mental Health Services Administration. (2015, 11 25). Trauma-informed approach and trauma-specific interventions. Retrieved from http://www.samhsa.gov/nctic/trauma-interventions

36.

The National Child Traumatic Stress Network. (2015, 11 25). Creating trauma-informed systems. Retrieved from http://www.nctsn.org/resources/topics/creating-trauma-informed-systems

37.

U.S. Department of Health and Human Services, Administration on Children, Youth, and Families, Children’s Bureau. (2014). The AFCARS report perliminary FY 2013 estimates as of July 2014 (Vol. No. 21). Washington, DC: U.S. Department of Health and Human Services.

38.

Zlotnik

Wilson

Scribano

Wood

J. N.

Noonan

(2015). Mandates for collaboration: Health care and child welfare policy and practice reforms create the platform for improved health for children in foster care. Adolescent Health Care, 45, 316–322. doi:10.1016/j.cppeds.2015.08.006