Building Capacity for Trauma-Informed Care in the Child Welfare System

Abstract

Exposure to childhood trauma is a major public health concern and is especially prevalent among children in the child welfare system (CWS). State and tribal CWSs are increasingly focusing efforts on identifying and serving children exposed to trauma through the creation of trauma-informed systems. This evaluation of a statewide initiative in Connecticut describes the strategies used to create a trauma-informed CWS, including workforce development, trauma screening, policy change, and improved access to evidence-based trauma-focused treatments during the initial 2-year implementation period. Changes in system readiness and capacity to deliver trauma-informed care were evaluated using stratified random samples of child welfare staff who completed a comprehensive assessment prior to (N = 223) and 2 years following implementation (N = 231). Results indicated significant improvements in trauma-informed knowledge, practice, and collaboration across nearly all child welfare domains assessed, suggesting system-wide improvements in readiness and capacity to provide trauma-informed care. Variability across domains was observed, and frontline staff reported greater improvements than supervisors/managers in some domains. Lessons learned and recommendations for implementation and evaluation of trauma-informed care in child welfare and other child-serving systems are discussed.

Keywords

trauma child welfare system trauma-informed implementation

Childhood exposure to maltreatment and other potentially traumatic events (PTEs) is a major public health concern. PTEs typically involve experiencing or witnessing serious threats or injury to the physical safety of oneself or others, including physical abuse, sexual assault, violence, and loss or separation from a caregiver. Approximately 85% of youth involved in the child welfare system (CWS) have been exposed to at least one PTE (Miller, Green, Fettes, & Aarons, 2011), and these children are nearly 4 times as likely to have experienced four or more PTEs and related adverse experiences than youth not involved in the CWS (Stambaugh et al., 2013). For many, the precipitating event leading to CWS contact involves a PTE (e.g., child abuse or neglect, exposure to family violence), and system contact itself (e.g., the investigation or placement process) may be perceived as traumatic (Kisiel, Fehrenbach, Small, & Lyons, 2009). Research also indicates youth in the CWS experience higher rates of post-traumatic stress disorder (PTSD) symptoms and other mental health problems compared to the general population (Ai, Foster, Pecora, Delaney, & Rodriguez, 2013; Burns et al., 2004; Pecora, Jensen, Romanelli, Jackson, & Ortiz, 2009).

Exposure to PTEs can disrupt brain development (Nemeroff et al., 2006) and increases the risk of traumatic stress reactions, a broad range of emotional, behavioral, cognitive, and social impairments. For example, PTE exposure has been linked to the onset of 28% of all psychiatric disorders in adolescents (McLaughlin et al., 2012) and is specifically associated with PTSD, delinquency, substance abuse, and academic problems (Kilpatrick et al., 2003; Saunders, 2003). These traumatic stress reactions threaten the CWS goals of safety, permanency, and well-being and likely mediate the relationship between exposure to adverse child experiences (including PTEs and other stressful experiences) and chronic health and mental health problems through adulthood (Felitti et al., 1998). The lifetime costs associated with child maltreatment alone have been estimated at $210,012 per child; all maltreatment cases in a single year result in an estimated $124 billion economic burden to society (Fang, Brown, Florence, & Mercy, 2012).

The call for action to prevent and to ameliorate the effects of PTE exposure through the creation of trauma-informed service systems is not new (Bloom & Farragher, 2013; Harris & Fallot, 2001; Ko et al., 2008). Burgeoning research on the prevalence, adverse effects, and costs of PTE exposure has prompted increased focus on making child-serving systems more “trauma-informed” and “trauma-sensitive.” However, the field has yet to reach consensus on what constitutes a trauma-informed system or how it is measured. The Substance Abuse and Mental Health Services Administration (SAMHSA, 2014) defines a trauma-informed system as one that

realizes the widespread impact of trauma and understands potential paths for recovery; recognizes the signs and symptoms of trauma in clients, families, staff, and others involved with the system; and responds by fully integrating knowledge about trauma into policies, procedures, and practices, and seeks to actively resist re-traumatization. (p. 9)

A similar and widely used definition specific to child welfare from the Chadwick Trauma-Informed Systems Project (CTISP, 2013) is that

a trauma-informed child welfare system is one in which all parties involved recognize and respond to the varying impact of traumatic stress on children, caregivers, and those who have contact with the system. Programs and organizations within the system infuse this knowledge, awareness, and skills into their organizational cultures, policies, and practices. They act in collaboration, using the best available science, to facilitate and support resiliency and recovery. (p. 11).

Addressing exposure to PTEs through appropriate screening and primary, secondary, or tertiary prevention efforts and evidence-based interventions are critical components of these definitions. The CWS is an important venue for addressing PTE exposure and accompanying traumatic stress reactions, but little is known about how to best operationalize these trauma-informed principles into day-to-day casework practice or the best methods for implementing trauma-informed care broadly in a CWS.

In 2001, SAMHSA established the National Child Traumatic Stress Network (NCTSN), a coalition of academic institutions and community providers, to improve services across the country for child trauma victims. Several NCTSN-related initiatives have begun to pilot strategies for developing more trauma-informed CWSs, including the use of multidisciplinary teams (Conradi et al., 2011b); identification of key drivers of CWS capacity for trauma-informed care (Hendricks, Conradi, & Wilson, 2011) and identification of “trauma champions”; and implementation of trauma screening and assessment procedures, trauma-focused evidence-based practices (EBPs), and trauma-informed decision making (Henry et al., 2011).

The Administration for Children and Families (ACF), the division of the U.S. Department of Health and Human Services that oversees federal child welfare policy, recently prioritized funding to support the development of trauma-informed CWSs (Sheldon, Tavenner, & Hyde, 2013). ACF has released multiple funding opportunities to improve trauma-informed care and place child well-being on equal standing with the traditional child welfare emphasis on safety and permanency. Since 2011, ACF funded 20 state and tribal CWSs to implement trauma-informed care through competitive discretionary awards.

To date, there is little research on system-level outcomes of efforts to enhance the capacity of child-serving systems to provide trauma-informed care or whether such changes are associated with improved outcomes for children and youth. Two of the 2011 ACF grantees have published descriptions of their systems approaches. Caringi and Lawson (2014), for example, describe a model for bringing trauma-informed care to Indian CWSs, and Fraser et al. (2014) describe Massachusetts’ cross-system child welfare and behavioral health approach. In addition, there is limited evidence that training child welfare staff about trauma can result in knowledge and practice change based on worker self-report (Conners-Burrow et al., 2013; Kramer, Sigel, Conners-Burrow, Savary, & Tempel, 2013) and that dissemination of a trauma-focused EBP to community providers can result in improved child outcomes (Lang, Franks, Epstein, Stover, & Oliver, 2015). However, very little is known about whether efforts to create trauma-informed systems using an array of strategies aligned with the SAMHSA and Chadwick definitions result in systemic changes in staff knowledge and practice or, more importantly, child outcomes.

The current article describes the initial implementation activities and system-level outcomes of a multiyear initiative to develop a trauma-informed CWS in Connecticut through a multipronged effort centered on workforce development, screening, dissemination of EBPs, and policy-focused changes. This strategy is based upon the SAMHSA and Chadwick definitions of a trauma-informed system and is consistent with previous NCTSN initiatives to promote trauma-informed CWSs. In 2011, the Connecticut Department of Children and Families (DCF) was awarded one of the five ACF grants to support development of trauma-informed CWSs. The Connecticut Collaborative on Effective Practices for Trauma (CONCEPT), formed through the grant, is a collaboration between DCF, the Child Health and Development Institute of Connecticut (which serves as the CONCEPT coordinating center), and The Consultation Center at Yale University (which is conducting the evaluation). The grant provided an initial planning year and a 4-year implementation period. This article describes (1) the overall approach of CONCEPT to system development and (2) initial evaluation results of system-level change for the initiative. Although some process data on individual components will be presented to indicate scope, the primary focus of results is on changes in system-level capacity to provide trauma-informed care during the first 2 years of implementation.

Methods

This section briefly summarizes the context for the initiative, describes the key implementation activities, and details the methods used to evaluate system-wide change.

State Context

DCF is an integrated state agency with five mandates for Connecticut’s children: child welfare, behavioral health, prevention, juvenile justice, and education. DCF employs approximately 3,500 staff across 14 area offices in six regions as well as psychiatric residential treatment facilities, a children’s psychiatric hospital, and a secure juvenile detention facility. Administrative data reviewed during the CONCEPT planning year revealed that DCF completed 44,158 investigations on more than 37,000 children in 2011; nearly one quarter (10,650 investigations) resulted in a finding of substantiated abuse or neglect. During the same period, DCF-contracted community-based providers served over 20,000 children, 28% (N = 5,633) of whom were involved with DCF for child protective services (CPS).

Spurred by a federal consent decree for failing to adequately meet the behavioral health needs of children and supervision by a court-appointed monitor since 1989, DCF has made significant improvements in the state’s behavioral health service array including dissemination of in-home EBPs beginning in the early 2000s. Prior to the current ACF grant, DCF began exploring trauma-informed care through a statewide dissemination of trauma-focused cognitive–behavioral therapy (TF-CBT; Cohen, Mannarino, & Deblinger, 2006), an EBP for children experiencing symptoms following exposure to PTEs, and piloting the NCTSN Child Welfare Trauma Training Toolkit (Child Welfare Collaborative Group & National Child Traumatic Stress Network, 2013) with approximately 900 child welfare staff. Additionally, the commissioner of DCF identified trauma-informed care as one of the seven cross-cutting themes to all of the department’s work in 2010, setting the stage for the creation of a trauma-informed system through CONCEPT.

Connecticut’s Trauma-Informed CWS Implementation Plan

Organizational structure/governance

A multidisciplinary core team reporting directly to DCF leadership provided oversight of planning and implementation for CONCEPT. The core team is comprised of child welfare managers, administrators, a family partner, the project coordinator, child trauma/implementation experts, and leads from the evaluation team. Several subcommittees reported to the core team including data/evaluation, screening/workforce development, policy, and trauma-focused EBP implementation.

Child welfare workforce development

Workforce development activities included three components central to the SAMHSA definition of “realizing” the impact of trauma on children and staff: (1) development of a cohort of “trauma champions” to serve as “early adopter” liaisons to local area offices, (2) implementation of a trauma-informed preservice or in-service training for the full child welfare workforce, and (3) efforts to address worker wellness and secondary traumatic stress.

From two to four trauma champions were selected in each area office and facility on a volunteer basis. Trauma champions represented diverse roles within the department, including frontline workers, supervisors, managers, and clinical coordinators. The cohort of approximately 40 staff was asked to provide at least one monthly in-service training focused on trauma in their local office. Example activities included circulating newsletters about trauma, holding brown bag lunches to discuss trauma, inviting local mental health providers of trauma-focused treatment to present, and providing the NCTSN Resource Parent Trauma Training to foster parents.

The second component, statewide mandatory preservice and in-service trauma training for child welfare staff, involved implementation of the NCTSN Child Welfare Trauma Training Toolkit (Child Welfare Colaborative Group & National Child Traumatic Stress Network, 2013). The Toolkit curriculum was developed to improve knowledge about child trauma and promote trauma-informed practice change across organizational levels within the child welfare workforce. To promote sustainability, CONCEPT engaged the DCF Training Academy to implement the Toolkit through a “train-the-trainer” approach. Training was provided to 487 managers and supervisors in the spring of 2013 and to 1,164 caseworkers and clinical staff in the fall of 2013. The Toolkit was added as an ongoing preservice training requirement for all new hires in 2014.

The third component addressed was worker wellness and secondary traumatic stress. Child welfare workers’ daily contact with the physical, sexual, and emotional abuse of children makes them highly susceptible to secondary traumatic stress, which can lead to decreased job performance, increased errors, impaired personal and professional relationships, and turnover (Bride, Jones, & Macmaster, 2007; Middleton & Potter, 2015). Each area office was required to maintain a worker support and wellness team and to participate in a quarterly statewide meeting. Each team addressed worker support and wellness through locally developed strategies that were shared and discussed at the statewide meetings. Modest funding from CONCEPT (∼ $60,000 annually) was split across all area offices and facilities to support local activities. For example, one team repurposed a small office into a designated staff wellness room where a worker can go when experiencing distress or a crisis; others held workshops on secondary traumatic stress, physical health, stress management, and nutrition or supported office-wide events to promote peer supports and improved morale.

Trauma screening

Screening children for PTEs and traumatic stress reactions is a key component of trauma-informed care (Conradi et al., 2011a) and is the primary strategy in the SAMHSA and Chadwick definitions for “recognizing” children suffering from trauma. The long-term goal of CONCEPT is to implement trauma screening for all children receiving in-home or out-of-home CPS and to embed screening data into the Statewide Automated Child Welfare Information System (SACWIS). The initial phase of this work included a brief trauma screening measure (Lang, Stover, Cloud, & Connell, 2011) that was developed and piloted with child welfare workers participating in TF-CBT learning collaboratives along with behavioral health providers (described in more detail below). The 28 participating child welfare workers, representing 11 area offices, screened at least 145 children and referred 61 (42%) for trauma-focused assessment or treatment.

Trauma-informed policy and practice guide revisions

DCF sought to align department policy with its practice efforts to create a trauma-informed CWS. DCF’s work is governed by brief policies (typically 1–2 pages of legislative and administrative mandates) and longer practice guides (procedures for practice that are specific to each program area). There are 38 overarching policy areas; while most are relevant to trauma-informed care (e.g., Child Protective Investigations and Foster and Adoption Services), others are not at all or minimally relevant (e.g., Fiscal and Engineering Services). Designated program leads are typically responsible for the content and revision of policy and practice guides within their area of oversight. Of note, prior to implementation, a key word search of all DCF policies and practice guides revealed no use of the terms “trauma” or “trauma-informed.” While existing policies and practice guides did address PTEs such as those typically addressed within the CWS (e.g., abuse, neglect, maltreatment, separation and removal from caregivers), the absence of the word trauma was consistent with a limited focus on trauma-informed care.

The DCF commissioner mandated that policy and practice guides be revised via a multidisciplinary CONCEPT policy workgroup, which reviewed content with the goal of supporting trauma-informed care. The policy workgroup developed a standardized policy review tool based on the Chadwick Center’s essential elements of trauma-informed CWSs and consistent with the SAMHSA definition of a trauma-informed approach (CTISP, 2012). Eventually, the group became familiar enough with trauma-informed principles that it modified policies and practice guides without formal use of the review tool. Policy modifications included identifying child and family strengths to cope with trauma, highlighting behavioral and emotional manifestations of trauma for caregivers, and obtaining trauma assessment results from behavioral health providers. For example, prior to review by the workgroup, the Family Assessment and Response (differential response) practice guide listed key considerations for a social worker’s first contact with the family that included addressing concerns raised in the suspected child abuse report, defining services available, interviewing the parent and child, and completing a risk assessment. The modified trauma-informed practice guide added additional considerations including identifying the child’s and caregiver’s exposure to PTEs, assessing common signs of traumatic stress in children, and assessing the impact of the parent’s own trauma on his or her ability to care for the child. Nine policies (out of 20 relevant policy areas) and 10 accompanying practice guides have been formally approved and disseminated to staff.

EBP dissemination

Increasing availability of trauma-focused EBPs has been a significant focus of DCF under its mandate for children’s behavioral health and is one of the primary strategies for “responding” to children suffering from trauma in the SAMHSA and Chadwick definitions. CONCEPT disseminated TF-CBT to 13 new community-based agencies, bringing the total number of agencies providing TF-CBT in Connecticut to 29. TF-CBT was disseminated through two learning collaborative cohorts, each lasting 11 months, based on the Breakthrough Series Collaborative quality improvement methodology (Ebert, Amaya-Jackson, Markiewicz, Kisiel, & Fairbank, 2012; Kilo, 1998). Learning collaboratives bring together teams from different agencies, comprised of staff at different organizational levels (e.g., management, supervisory, clinical) to support dissemination of a particular service model. Teams meet face-to-face throughout the year and receive ongoing consultation from trainers and implementation consultants; adult learning principles and the use of data to track improvements are key strategies.

The CONCEPT Learning collaboratives were adapted to include child welfare staff together with behavioral health providers in order to promote cross-system collaboration and to improve access to TF-CBT for children in the CWS, based upon South Carolina’s Community-Based Learning Collaborative model (Saunders & Hanson, 2014). Child welfare workers participating in the collaboratives were trained to screen children for trauma, to provide psychoeducation about trauma to children and families, and to engage and refer children who screened positive for a TF-CBT assessment at their partner agency. Behavioral health providers were trained to complete TF-CBT assessments using standardized measures, provide TF-CBT with fidelity, and to embed the model into the agency’s service array to promote sustainability. Child welfare staff and behavioral health providers also worked together to improve coordination and collaboration across systems, including procedures for referral and feedback, sharing information, and aligning treatment and case plans. Teams from 13 outpatient children’s behavioral health clinics (87 staff) and their local child welfare offices (78 staff) participated in the learning collaboratives and provided TF-CBT to 191 children in the CWS (as well as 533 additional children not in the CWS).

System-Level Evaluation Procedures

A major focus of the CONCEPT evaluation is to assess system-level change in the overall capacity of the CWS to deliver trauma-informed care, and a primary means of assessing this change has been through a survey of CWS personnel at regular intervals throughout the initiative. A stratified random sample of DCF staff in Years 1 and 3 of the grant were identified to represent current system-level perspectives on trauma-informed capacity, since the interest was not in individual-level change but in overall perceptions across the system at a point in time. The samples were based on a roster of all agency staff in leadership, supervisory, and caseworker/clinical roles, stratified by DCF area office or residential facility setting. The sampling rate was approximately 20% of all eligible DCF personnel (493 in Year 1 and 497 in Year 3); staff were excluded in Year 3 if they were active in learning collaborative activities to minimize survey burden among DCF staff. Selected staff received an e-mail invitation to complete a 30- to 45-min anonymous survey of the agency’s capacity to deliver trauma-informed care; the e-mail linked participants to a web-based consent and survey administered using the Qualtrics survey platform (http://www.qualtrics.com). Individuals completing the survey were entered into random drawings to receive a $20 gift card in exchange for their participation. Weekly e-mail prompts were sent to invited staff over a 5- to 7-week period following the initial invitation to promote participation.

Participants

A total of 223 DCF staff (45.2% response rate) completed the survey in Year 1 (preimplementation), and 231 staff (46.5% response rate) completed the survey in Year 3. Follow-up analyses compared response rates by role and region across survey years, although analyses were limited as a result of anonymous survey administration procedures. Individuals whose job description reflected program director, manager, or supervisory responsibilities were more likely to complete the survey than those in caseworker, frontline, and other roles (54.4–37.2%; t = 4.7, p < .01), but response rates by role did not differ statistically over survey time points. Response rates also varied across regions within each year, and regional patterns of response shifted from Year 1 to Year 3 for three regions (two regions had a decrease in response rate, and one had an increase).

The two samples represented a cross section of the CWS workforce, with approximately 3% in director/administrator roles (Year 1: 2.7%, Year 3: 3.0%), approximately one quarter of participants in manager or supervisory roles (Year 1: 27.8%, Year 3: 24.2%), over half in caseworker or clinical roles (Year 1: 57.8%, Year 3: 56.7%), and the remaining participants represented other roles (Year 1: 11.7%, Year 3: 16.0%; e.g., case aide, consultant, or medical/nursing services). Participants had worked in child welfare for a relatively long period (Year 1: 13.5 years, Year 3: 14.8 years, t = −2.04, p = .04)—the only statistically significant difference in cross-cohort comparisons of characteristics, likely due to the stability of the workforce over the 2-year survey period. As would be expected, caseworkers and other staff had worked in the field for less time than managers and directors (Year 1: 11.5 years, Year 3: 13.5 years vs. Year 1: 18.1 years, Year 3: 18.4 years). Child welfare sectors were also broadly represented, with approximately 20% involved in intake and disposition roles (e.g., Careline, CPS, Family Assessment Response), between 35% and 40% in ongoing services, about 10% in foster or adoption services, under 10% at facilities (e.g., residential, juvenile, or medical settings), and over 15% in other units or divisions; staff could select multiple units as applicable. Other characteristics are summarized in Table 1.

Table 1.

Respondent Characteristics (Child Welfare Trauma System Capacity Survey in Years 1 and 3).

CWS Staff Characteristics	Year 1 (N = 223)		Year 3 (N = 231)
CWS Staff Characteristics	Mean	SD	Mean	SD	Comparison
Mean years in Child Welfare	13.5	6.7	14.8	6.9	t(450) = −2.04, p = .04
Mean Caseload (if applicable)	17.1	18.8	16.3	19.2	t(322) = 0.39, p = .70
	n	%	n	%
Agency role					X²(3) = 2.18, p = .54
Director/administrator	6	2.7	7	3.0
Manager/supervisor	62	27.8	56	24.2
Caseworker/clinical staff	129	57.8	131	56.7
Other	26	11.7	37	16.0
Child Welfare Sector (check all)
Intake and disposition (CPS/DRS)	46	20.6	50	21.6	X²(1) = 0.07, p = .79
Ongoing services	87	39.0	81	35.1	X²(1) = 0.76, p = .38
Area resource group	16	7.2	14	6.1	X²(1) = 0.23, p = .63
Foster care and adoption services	16	7.2	26	11.3	X²(1) = 2.25, p = .13
Administration and quality improvement	13	5.8	13	5.6	X²(1) = 0.01, p = .93
Child welfare facility	21	9.4	17	7.4	X²(1) = 0.63, p = .43
Other	42	18.8	39	16.9	X²(1) = 0.30, p = .59
Education					X²(3) = 0.96, p = .81
Doctoral (MD, PhD)	7	3.1	4	1.7
Licensed masters	30	13.5	31	13.4
Masters	79	35.4	83	35.9
Bachelors or lower	107	48.0	113	48.9
Gender (female)	171	77.0	178	78.4	X²(1) = 0.12, p = .72
Race/ethnicity					X²(3) = 2.25, p = .52
African American	53	24.4	47	20.5
Caucasian	124	57.1	129	56.3
Hispanic	25	11.5	36	15.7
Other	15	6.9	17	7.4

Note. CPS = child protective services; DRS = differential response system.

Measures

The Trauma System Readiness Tool (TSRT; Hendricks et al., 2011) is a measure developed by the Chadwick Trauma-Informed Systems Project (CTISP) to assess CWS personnel perceptions of the agency and agency staff understanding of and capacity to use trauma-informed principles and practices to support children, families, and the child welfare workforce. Development of the initial instrument was completed by CTISP’s expert panel and was designed to align with the Essential Elements of a trauma-informed CWS outlined by the NCTSN. Sullivan, Preisler, Ake, Potter, and Beck (2012) developed an adaptation of the TSRT that reduced items, streamlined the response set, and maintained coverage of key domains.

The primary scales of the revised TSRT, administered to all participants, included 90 items covering a number of domains. Items were rated on a 5-point Likert-type scale from strongly disagree to strongly agree, with scales created by averaging ratings within each domain; higher scores were indicative of more positive or favorable ratings of the individual- or agency-level capacity in that aspect of trauma-informed care. A preliminary exploratory factor analysis of responses from Year 1 supported the hypothesized factor structure of the instrument, with a few minor exceptions. First, two scales hypothesized to reflect ratings of other staff’s trauma-related knowledge and trauma-related practice appeared to reflect a single domain and were combined. In addition, 9 items were dropped for either excessive cross-loading or failure to load on any factor. A total of 12 TSRT domains were included in the present study based on this preliminary analysis. Table 2 summarizes information about the TSRT subscales and reliability.

Table 2.

Trauma System Readiness Tool Domains.

Scale Label	Scale Content (Sample Item)	Cronbach’s α
Trauma training and education (8 items)	Personal exposure to key aspects of the effects of trauma on children (“I have received training and education in how trauma affects a child’s attachment to his or her caregivers.”)	Year 1: .95 Year 3: .96
Staff trauma knowledge and practice (15 items)	Agency-level staff understanding and use of trauma-related principles and practices (“The child welfare staff at my agency are aware of how trauma history can affect a parent’s response to workers and children welfare system activities/mandates.” “The child welfare staff at my agency work as a team with each family, empowering them to make decisions about their services.”)	Year 1: .94 Year 3: .94
Individual trauma knowledge and practice (5 items)	Personal understanding and use of trauma-related principles and practices (“I have a clear understanding of what trauma-informed practice means in my professional role.” “In practice, I am utilizing what I believe to be trauma-informed interactions with children and families.”)	Year 1: .83 Year 3: .83
Trauma supervision and support (7 items)	Agency-level supports for dealing with trauma among children and families as well as vicarious trauma exposure (“My agency provides regular supervision from a supervisor who is trained in understanding trauma.”)	Year 1: .90 Year 3: .92
Staff supports child relationships (4 items)	Staff efforts to support for maintenance of key child relationships (“The child welfare staff at my agency arrange frequent contact between a child and his or her siblings [except in cases where contact is harmful to the child.]”)	Year 1: .88 Year 3: .87
Birth family trauma support (9 items)	Staff use of trauma-informed practices with children’s birth families (“When working with birth families, the child welfare staff at my agency use a standardized measure or set of questions to assess birth parents’ trauma symptoms.”)	Year 1: .91 Year 2: .91
Resource family trauma support (7 items)	Staff use of trauma-informed practices with resource caregivers (“The child welfare staff at my agency provide resource families with information about the trauma histories and related problems of the children in their care.”)	Year 1: .91 Year 3: .91
Staff addresses child psychological safety (7 items)	Staff practices to address psychological safety (i.e., the sense of “feeling safe”) in children (“The child welfare staff at my agency make psychological safety a priority when making case decisions.”)	Year 1: .91 Year 3: .94
Agency trauma assessment (4 items)	Accessibility and trauma-focused content of behavioral assessments for children and youth in the area (“A structured mental health/developmental assessment includes a child’s trauma history.”)	Year 1: .94 Year 3: .94
Access to trauma-informed services (6 items)	Staff- and community-level access to trauma-informed services and supports for children in their caseloads (“Trauma-focused treatment for children is widely available in my community.”)	Year 1: .87 Year 3: .86
Local agency collaboration (general practices; 6 items)	Collaboration around general practice (“The child welfare staff at my agency and our local mental health community communicate openly with one another.”)	Year 1: .88 Year 3: .92
Local agency collaboration (trauma practices; 3 items)	Joint commitment to providing trauma-informed care (“The child welfare staff at my agency and our local mental health community have leadership committed to implementing trauma-focused treatments.”)	Year 1: .82 Year 3: .81

Results

Correlations among TSRT subscales are presented in Table 3 separately for Years 1 and 3 administrations. In general, correlations among TSRT subscales were small to moderate in size for Year 1 and slightly larger (i.e., moderate to large) in Year 3. Exposure to trauma-focused training and education was most highly correlated with individual trauma practice (Year 1: r = .58, p < .01; Year 3: r = .55, p < .01). Ratings of staff-level knowledge and practice were most highly correlated with other indicators of agency and staff-level practice (e.g., support of child relationships, support of birth and resource family trauma-related needs, and focus on child psychological sense of safety). Finally, the trauma-focused collaboration with local behavioral health agencies was highly correlated with general collaboration (Year 1: r = .59, p < .01; Year 3: r = .69, p < .01) and with greater access to trauma-informed services (Year 1: r = .51, p < .01; Year 3: r = .59, p < .01).

Table 3.

Correlations Among TSRT Subscales (Years 1 and 3).

Year 1	Year 3
Year 1	1	2	3	4	5	6	7	8	9	10	11	12
1.Trauma training and education	—	0.44**	0.55**	0.26**	0.35**	0.14*	0.23**	0.20**	0.20**	0.17*	0.13*	0.16*
2.Staff trauma knowledge and practice	0.31**	—	0.37**	0.58**	0.60**	0.56**	0.56**	0.59**	0.32**	0.45**	0.39**	0.48**
3.Individual trauma knowledge and practice	0.58**	0.15*	—	0.37**	0.28**	0.25**	0.38**	0.35**	0.35**	0.35**	0.31**	0.36**
4.Trauma supervision and supports	0.33**	0.51**	0.38**	—	0.40**	0.54**	0.61**	0.54**	0.33**	0.58**	0.43**	0.44**
5.Staff supports child relationships	0.13	0.54**	0.14*	0.35**	—	0.36**	0.43**	0.44**	0.19**	0.32**	0.22**	0.33**
6.Birth family trauma support	0.18**	0.52**	0.22**	0.57**	0.37**	—	0.64**	0.60**	0.25**	0.37**	0.37**	0.46**
7.Resource family trauma support	0.24**	0.46**	0.25**	0.50**	0.34**	0.61**	—	0.65**	0.36**	0.55**	0.49**	0.52**
8.Staff addresses child psychological safety	0.17*	0.50**	0.24**	0.43**	0.41**	0.52**	0.50**	—	0.32**	0.54**	0.41**	0.54**
9.Agency trauma assessment	0.29**	0.26**	0.36**	0.35**	0.24**	0.35**	0.38**	0.32**	—	0.47**	0.44**	0.50**
10.Access to trauma-informed services	0.17*	0.33**	0.28**	0.51**	0.29**	0.47**	0.42**	0.40**	0.39**	—	0.47**	0.59**
11.Local agency collaboration (general)	0.24**	0.28**	0.31**	0.35**	0.32**	0.36**	0.42**	0.49**	0.33**	0.33**	—	0.69**
12.Local agency collaboration (trauma)	0.26**	0.34**	0.33**	0.47**	0.28**	0.40**	0.47**	0.49**	0.41**	0.51**	0.59**	—

*p < .05. **p < .01.

Changes in System Readiness and Capacity

To assess for changes in child welfare staff perceptions of individual- and system-level readiness and capacity to deliver trauma-informed care, a series of regression models was run in Stata Version 14.0 using robust standard errors (Verardi & Croux, 2009) to account for clustering by area office or facility, as well as potential clustering effects associated with survey year. Multilevel modeling was also considered, but estimates of intraclass correlation (ICC) accounting for clustering by area office were extremely low (generally less than .001). Further, no Level-2 effects or cross-level effects were being examined, so robust standard error models were determined to be sufficient to model specified effects. Table 4 summarizes changes in key TSRT domains from Year 1 to Year 3.

Table 4.

Ratings of Trauma System Readiness and Capacity Domains in Years 1 and 3.

TSRT Domain	Year 1		Year 3		Time Effect [95% CI]	Time Comparison^a	Role Effect [95% CI]	Role Comparison^b
TSRT Domain	Mean	Robust SE	Mean	Robust SE	Time Effect [95% CI]	Time Comparison^a	Role Effect [95% CI]	Role Comparison^b
1. Trauma training and education	3.78	0.06	4.16	0.04	0.39** [0.23–0.54]	F = 28.31, p < .001	−0.11 [−0.29–0.07]	F = 14.25, p < .001
2. Staff trauma knowledge and practice	3.65	0.04	3.90	0.04	0.26** [0.11–0.40]	F = 14.05, p = .002	0.10 [−0.02–0.22]	F = 12.96, p < .001
3. Individual trauma knowledge and practice	3.65	0.04	3.98	0.04	0.33** [0.20–0.46]	F = 28.98, p < .001	−0.08 [−0.29–0.14]	F = 15.39, p < .001
4. Trauma supervision and supports	2.54	0.06	3.06	0.06	0.52** [0.33–0.71]	F = 33.75, p < .001	0.05 [−0.10–0.20]	F = 16.66, p < .001
5. Staff supports child relationships	3.62	0.05	3.87	0.06	0.24** [0.09–0.41]	F = 10.26, p = .005	0.24** [0.11–0.37]	F = 21.69, p < .001
6. Birth family trauma support	2.90	0.06	3.36	0.06	0.46** [0.31–0.61]	F = 40.68, p < .001	0.28** [0.12–0.44]	F = 56.64, p < .001
7. Resource family trauma support	3.08	0.05	3.50	0.04	0.42** [0.28–0.56]	F = 39.15, p < .001	0.20** [0.06–0.35]	F = 22.98, p < .001
8. Staff addresses child psychological safety	3.57	0.05	3.73	0.05	0.18* [0.01–0.34]	F = 5.26, p = .03	0.13 [−0.01–0.27]	F = 4.48, p = .03
9. Agency trauma assessment	3.29	0.07	3.68	0.06	0.39** [0.20–0.58]	F = 18.75, p < .001	0.00 [−0.22–0.23]	F = 9.72, p = .001
10. Access to trauma-informed services	2.89	0.07	3.40	0.05	0.51** [0.33–0.69]	F = 34.81, p < .001	0.10 [−0.02–0.23]	F = 16.83, p < .001
11. Local agency collaboration (general)	3.36	0.04	3.41	0.04	0.05 [−0.06–0.16]	F = 0.91, p = .35	0.13 [−0.04–0.31]	F = 2.03, p = .16
12. Local agency collaboration (trauma)	3.33	0.06	3.65	0.04	0.31** [0.15–0.48]	F = 15.52, p < .001	0.06 [−0.06–0.18]	F = 12.00, p < .001

Note. ^aTime comparison assessed using regression with robust standard errors to adjust for clustering by area office/facility; df is based on the number of setting clusters (1, 18).

^bRole effect controls for time and is assessed using regression with robust standard errors to adjust for clustering by area office/facility; df is based on the number of setting clusters (2, 18).

*p < .05. **p < .01.

Ratings of Year 1 readiness and capacity were generally in the positive range (between 3 and 4 on a 5-point scale). Areas rated highest were trauma training and education (M = 3.78, SD = 0.83) and both staff- and individual-level knowledge and practice domains (M = 3.65, SD = 0.63; M = 3.65, SD = 0.73, respectively). Three scales were rated in the generally unfavorable direction—trauma supervision and supports (M = 2.54, SD = 0.82), access to trauma-focused services (M = 2.89, SD = 0.75), and support for birth family trauma-related needs (M = 2.90, SD = 0.73).

Mean scores for nearly all domains increased significantly from Year 1 to Year 3 (see Table 4; the “Time Effect” column). The greatest increases were observed for trauma supervision and supports (β = 0.52, p < .001), access to trauma-informed services (β = 0.51, p < .001), and trauma-related supports for birth and resource families (β = 0.46, p < .001; β = 0.42, p < .001, respectively). Moderate gains also were observed for all other scales with the exception of general collaboration with local mental health agency staff (though trauma-focused collaboration did improve significantly; β = 0.31, p < .001).

Follow-up analyses were also conducted to see whether staff role was associated with differences in ratings of system readiness and capacity after controlling for time-related effects. Due to the limited number of managers and directors who completed the survey, this comparison contrasted ratings for managers and supervisors (combined) relative to caseworkers and other clinical staff. Results are also depicted in Table 4. As indicated, after controlling for time effects, only three scales evidenced significant differences between managerial/supervisory-level staff and caseworkers and frontline clinical staff. These scales all pertained to key support roles among caseworkers and their clients: support for helping the child maintain connections with key relationships (β = 0.24, p = .001), support for birth family trauma-related needs (β = 0.28, p = .002), and support for resource parent trauma-related needs (β = 0.20, p = .007). In each instance, caseworker ratings were higher than those of managers and supervisors, indicating a more favorable rating of capacity in these domains among caseworkers and clinical staff.

Finally, post hoc analyses (not included in the table) were run to test for interactions between role status and time. Only one significant effect was found for the rating of exposure to trauma training and education, F(3, 18) = 21.91, p < .001. Inclusion of the interaction effect revealed that caseworkers and clinical staff initially rated this domain lower in Year 1 (β = −0.26, p = .04) but showed significantly greater increases in this rating over time (β = 0.30, p = .05), accounting for the time effect, initially observed, but no longer significant when role differences in improvement were accounted for (β = 0.17, p = .25).

Discussion

The broad goal of CONCEPT is to create a more trauma-informed CWS that integrates research and best practices on childhood trauma and ultimately results in improved identification, case planning, and service delivery for children and families. The primary strategies CONCEPT has used to achieve these changes, based upon the SAMHSA and Chadwick definitions of a trauma-informed system, are workforce development, screening, policy change, and dissemination of trauma-focused EBPs. Systemic change also requires leadership support from the highest levels and an organizational culture that emphasizes the importance of trauma-informed care in all aspects of child welfare practice.

Initial results from the first 2 years of CONCEPT implementation activities show that significant improvements in ratings of trauma-informed knowledge, attitudes, and practice have been made across Connecticut’s child welfare workforce. These improvements were found across nearly all domains assessed (e.g., knowledge about trauma, foster care, supervision, birth families, and access to trauma-focused EBPs for behavioral health) and across all levels of child welfare staff, suggesting that an organizational and cultural shift to embrace trauma-informed care is occurring. Overall ratings of trauma-informed care at follow-up, while generally in the positive range, still suggest that there is room for additional improvement and that 2 years of a multipronged implementation was not sufficient for becoming a “trauma-informed system.”

Post hoc analyses revealed that improvements in trauma training and education were significantly greater for caseworkers than for supervisors/managers, a pattern that was not observed in other domains. This result suggests that trauma training may be especially important for frontline staff in terms of boosting overall system capacity to recognize and respond to trauma. Frontline staff also reported greater capacity than supervisors and managers for helping children maintain relationships and supporting birth and resource parents from a trauma-informed perspective. One interpretation of this finding could be that frontline staff were better able to apply knowledge about trauma to their work with children and families. Similarly, there may be somewhat different training needs for caseworkers and supervisors/managers, and some aspects of training curricula and targeted skills might be best tailored to job function. While additional research is needed, perhaps supplementing core training curricula that address issues of trauma-informed care relevant to all staff with tailored content and skills specific to role and job function would better strengthen capacity across service sectors and roles within CWS. For example, the practice application of trauma-informed care (and associated training content) might vary for CPS investigators (conducting a trauma-informed investigation), foster care caseworkers (working with resource parents from a trauma-informed perspective), supervisors (trauma-informed supervision and addressing staff wellness/secondary traumatic stress), leadership (policy change and resource allocation), and behavioral health/clinical staff (trauma-focused assessment and EBPs). Finally, agency support for dealing with trauma among children as well as vicarious trauma among staff was the lowest rated domain at both time points, suggesting that this is an area that needs additional focus.

Several important limitations to this evaluation should be noted. First, the response rate among child welfare staff was below 50%, and it is possible that a response bias exists (e.g., those interested in trauma were more likely to respond). However, the response rate was consistent for each administration period, and the rates of response among manager/supervisor and caseworkers/frontline staff remained consistent over time. Thus, it does not seem likely that the potential for introducing bias differs across waves, so the observed effect of time would appear to be independent of this limitation. Participants in this sample also, generally, had worked in the CWS for relatively long periods of time—on average 13.5 years. National data indicate this is atypical, with annual child welfare turnover rates as high as 23–60%; however, it is consistent with other reports of Connecticut’s child welfare workforce due to the state’s very low turnover rate (8% or less annually) attributed to high salaries and relatively low caseloads compared with other regions (Strand, Spath, & Bosco-Ruggiero, 2010). Understanding the implications of trauma-informed care for other CWSs with high turnover rates needs further exploration. Second, it was not possible to analyze the relative effects of individual program components (e.g., trauma screening, workforce development, and dissemination of EBPs) on system-level outcomes. We believe that the synergy of the entire initiative, combined with leadership support and the promulgation of attention to trauma into other programs and initiatives, contributed to an improved culture of trauma-informed care observed in the system-wide improvements. Future analyses, incorporating individual component-level evaluation findings, may help to tease apart the particular effects on system-level change. Third, evaluation data were self-reported by child welfare staff and did not include independent measures of practice change (e.g., the number of referrals) or data on children to determine whether improvements in trauma-informed care resulted in improved child and family outcomes. There is evidence that a more positive organizational climate in the CWS—suggested by improvements demonstrated in trauma-informed care—results in improved child and family outcomes (Glisson & Green, 2011). However, research on the effects of trauma-informed systems on child and family outcomes, and objective measures of worker practice change, is sorely needed. Fourth, because measures of trauma-informed care are relatively new, no norms or standards for interpreting the TSRT, for example, are available. Thus, while improvements were made over time, it is not clear how Connecticut compares to other state or tribal CWSs. Finally, and related to the issue of measurement, the TSRT measures self-reported perceptions of agency- and staff-level understanding and capacity to use trauma-informed principles, rather than an independent assessment of trauma-related skills and behaviors among CWS staff. It is important to note, however, that measures of system capacity to deliver trauma-informed care are relatively new, and this study is among the first to utilize measures such as the TSRT to demonstrate system-level change associated with a systemic initiative. Future research is needed to demonstrate how such changes are associated with independent ratings of worker and organizational behaviors as well as how such changes impact assessments of well-being among the CWS service population.

Further research on, and development of, measures of trauma-informed systems is needed to provide guidance to child welfare and other child-serving systems. As interest in trauma-informed care grows, there is a risk that “receiving some trauma-related training” becomes equivalent to “being trauma-informed.” However, we believe that trauma training is an essential but not nearly sufficient element of a trauma-informed system. For example, a onetime training on trauma is unlikely to result in improved access to trauma-specific services or improved collaboration with behavioral health providers, both of which were found in this evaluation; we hypothesize that other implementation activities such as the child welfare/behavioral health staff TF-CBT learning collaboratives and the broader culture shift were more likely to effect change in these domains. Given the significant financial resources necessary for system change, economic evaluations will also be important for identifying potential cost savings of trauma-informed systems. The significant costs of child maltreatment (Fang et al., 2012) and the established benefits of early intervention and trauma-focused EBPs (Lee, Aos, & Pennucci, 2015) suggest that cost savings are certainly plausible. However, there is not yet research to support this common assumption of cost savings underlying the economic case for trauma-informed systems.

A number of challenges to further improvements in trauma-informed care were identified during implementation. The most common overarching barrier was initiative fatigue and a reluctance among staff at all levels to add “one more thing”—whether a new training, a new screening measure, or a new meeting with behavioral health providers. A significant competing demand in Connecticut has been that DCF has focused resources on meeting the requirements of a federal consent decree it has been under for more than 25 years. The planned procurement of a new SACWIS over the next several years has delayed embedding screening in the current system and practice model, thus limiting the number of children who are screened for trauma. Concerns about EBP dissemination included resources for sustainment, how to sufficiently monitor treatment fidelity in a large-scale dissemination, and how to scale up improved collaboration and service planning between child welfare and EBP providers.

Recommendations

We make several recommendations for implementing trauma-informed care in CWSs. First, sufficient resources for the scope of work should be identified in advance to include more than a onetime training. While many states will not have the benefit of a 5-year grant to support these efforts, resources for project staff (including a lead implementation team), training, infrastructure, information technology, quality assurance, and evaluation should be identified in advance. Second, support and buy in from leadership at the highest levels is critical for systemic change. If this is not initially possible, grassroots or pilot programs could be tested and used as models to consider for broader dissemination (an approach we are taking with trauma screening). Third, a readiness and capacity assessment of the system’s strengths and needs is recommended along with sufficient time to plan implementation activities. For CONCEPT, this process resulted in major implementation changes, including recommendations by staff to address secondary traumatic stress and wellness. Fourth, identification and development of formal or informal “champions” can be an important strategy for systems change.

Finally, state and federal policy changes and funding opportunities that support trauma-informed care would accelerate these activities. For example, policy changes could promote, incentivize, or require trauma screening and access to EBPs, similar to how The Child and Family Services Improvement and Innovation Act of 2011 (P.L. 112-34) requires state child welfare agencies to report how they address trauma experienced by children in foster care. Funding could further support implementation activities, including EBPs. EBPs typically cost providers more than “treatment as usual” to deliver because of training, consultation, and data reporting requirements but are not typically reimbursed at higher rates, thus impeding increased capacity and expansion (Aarons, Wells, Zagursky, Fettes, & Palinkas, 2009). Ultimately, a trauma-informed CWS led by a supportive administration, comprised of healthy and knowledgeable staff, who routinely screen children for trauma, integrate knowledge about trauma into their case plans, and have access to a range of trauma-focused and other EBPs, will likely result in significant improvements in the lives of the children and families they serve.

Footnotes

Acknowledgments

We would like to thank the Connecticut Department of Children and Families, Marilyn Cloud, Tracy Davis, Michelle Delaney, Tiffany Franceschetti, Robert Franks, Maegan Genovese, Karen Grayson, Jodi Hill-Lily, Tanisha Mair, Emily Melnick, Carol O’Connor, Rita Pellagi, Bert Plant, Kristina Stevens, Laurie Valentine, Jeffrey Vanderploeg, Doriana Vicedomini, and all of the provider organizations, staff, and families who participated.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This initiative was funded by the Department of Health and Human Services, Administration for Children and Families, Children’s Bureau, Grant #0169.

References

Aarons

G. A.

Wells

R. S.

Zagursky

Fettes

D. L.

Palinkas

L. A.

(2009). Implementing evidence-based practice in community mental health agencies: A multiple stakeholder analysis. American Journal of Public Health, 99, 2087–2095. doi:10.2105/AJPH.2009.161711

A. L.

Foster

L. J. J.

Pecora

P. J.

Delaney

Rodriguez

(2013). Reshaping child welfare’s response to trauma: Assessment, evidence-based intervention, and new research perspectives. Research on Social Work Practice, 23, 651–668. doi:10.1177/1049731513491835

Bloom

S. L.

Farragher

(2013). Restoring sanctuary: A new operating system for trauma-informed organizations. New York, NY: Oxford University Press.

Bride

B. E.

Jones

J. L.

Macmaster

S. A.

(2007). Correlates of secondary traumatic stress in child protective service workers. Journal of Evidence-based Social Work, 4, 69–80.

Burns

B. J.

Phillips

S. D.

Wagner

H. R.

Barth

R. P.

Kolko

D. J.

Campbell

Landsverk

(2004). Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 960–970. doi:10.1097/01.chi.0000127590.95585.65

Caringi

J. C.

Lawson

H. A.

(2014). Conceptualizing a trauma informed child welfare system for Indian country. Journal of Family Strengths, 14, 1–25.

Chadwick Trauma-Informed Systems Project. (2012). Creating trauma-informed child welfare systems: A guide for administrators. San Diego, CA: Chadwick Center for Children and Families.

Chadwick Trauma-Informed Systems Project. (2013). Creating trauma-informed child welfare systems: A guide for administrators (2nd ed.). San Diego, CA: Chadwick Center for Children and Families.

Child Welfare Collaborative Group & National Child Traumatic Stress Network. (2013). Child welfare training toolkit: 3rd Edition: Trainer’s Guide. Los Angeles, CA: National Center for Child Traumatic Stress.

10.

Cohen

J. A.

Mannarino

A. P.

Deblinger

(2006). Treating trauma and traumatic grief in children and adolescents. New York, NY: Guilford Press.

11.

Conners-Burrow

N. A.

Kramer

T. L.

Sigel

B. A.

Helpenstill

Sievers

McKelvey

(2013). Trauma-informed care training in a child welfare system: Moving it to the front line. Children and Youth Services Review. doi:10.1016/j.childyouth.2013.08.013

12.

Conradi

Agosti

Tullberg

Richardson

Langan

Wilson

(2011a). Linking child welfare and mental health using trauma-informed screening and assessment practices. Child Welfare, 90, 129–147.

13.

Conradi

Agosti

Tullberg

Richardson

Langan

Wilson

(2011b). Promising practices and strategies for using trauma-informed child welfare practice to improve foster care placement stability: A breakthrough series collaborative. Child Welfare, 90, 207–225.

14.

Ebert

Amaya-Jackson

Markiewicz

J. M.

Kisiel

Fairbank

J. A.

(2012). Use of the breakthrough series collaborative to support broad and sustained use of evidence-based trauma treatment for children in community practice settings. Administration and Policy in Mental Health, 39, 187–199. doi:10.1007/s10488-011-0347-y

15.

Fang

Brown

D. S.

Florence

C. S.

Mercy

J. A.

(2012). The economic burden of child maltreatment in the United States and implications for prevention. Child Abuse and Neglect, 36, 156–165. doi:10.1016/j.chiabu.2011.10.006

16.

Felitti

V. J.

Anda

R. F.

Nordenberg

Williamson

D. F.

Spitz

A. M.

Edwards

… Marks . (1998). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The Adverse Childhood Experiences (ACE) Study. American Journal of Preventive Medicine, 14, 245–258.

17.

Fraser

J. G.

Griffin

J. L.

Barto

B. L.

Wenz-Gross

Spinazzola

… Bartlett

J. D.

(2014). Implementation of a workforce initiative to build trauma-informed child welfare practice and services: Findings from the Massachusetts Child Trauma Project. Children and Youth Services Review, 44, 233–242. doi:10.1016/j.childyouth.2014.06.016

18.

Glisson

Green

(2011). Organizational climate, services, and outcomes in child welfare systems. Child Abuse & Neglect, 35, 582–591. doi:10.1016/j.chiabu.2011.04.009

19.

Harris

Fallot

R. D.

(2001). Envisioning a trauma-informed service system: A vital paradigm shift. New Directions for Mental Health Services, 89, 3–22.

20.

Hendricks

Conradi

Wilson

(2011). The trauma system readiness tool. San Diego, CA: Chadwick Center for Children and Families.

21.

Henry

Richardson

Black-Pond

Sloane

Atchinson

Hyter

(2011). A grassroots prototype for trauma-informed child welfare system change. Child Welfare, 90, 169–186.

22.

Kilo

C.M

. (1998). A framework for collaborative improvement: Lessons from the Institute for Healthcare Improvement’s Breakthrough Series. Quality Management in Health Care, 6, 1–13.

23.

Kilpatrick

D. G.

Ruggiero

K. J.

Acierno

Saunders

B. E.

Resnick

H. S.

Best

C. L.

(2003). Violence and risk of PTSD, major depression, substance abuse/dependence, and comorbidity: Results from the National Survey of Adolescents. Journal of Consulting and Clinical Psychology, 71, 692–700.

24.

Kisiel

Fehrenbach

Small

Lyons

J. S.

(2009). Assessment of complex trauma exposure, responses, and service needs among children and adolescents in child welfare. Journal of Child & Adolescent Trauma, 2, 143–160.

25.

S. J.

Ford

J. D.

Kassam-Adams

Berkowitz

S. J.

Wilson

Wong

… Layne

C. M.

(2008). Creating trauma-informed systems: Child welfare, education, first responders, health care, juvenile justice. Professional Psychology: Research and Practice, 39, 396–404. doi:10.1037/0735-7028.39.4.396

26.

Kramer

T. L.

Sigel

B. A.

Conners-Burrow

N. A.

Savary

P. E.

Tempel

(2013). A statewide introduction of trauma-informed care in a child welfare system. Children and Youth Services Review, 35, 19–24.

27.

Lang

J. M.

Franks

R. P.

Epstein

Stover

Oliver

J. A.

(2015). Statewide dissemination of an evidence-based practice using Breakthrough Series Collaboratives. Children and Youth Services Review, 55, 201–209.

28.

Lang

J. M.

Stover

C. S.

Cloud

Connell

(2011). Connecticut Trauma Screen. Unpublished measure, Child Health and Development Institute.

29.

Lee

Aos

Pennucci

(2015). What works and what does not? Benefit-cost findings from WSIPP (Doc. No 15-02-4101). Olympia, WA: Washington State Institute for Public Policy.

30.

McLaughlin

K. A.

Greif Green

Gruber

M. J.

Sampson

N. A.

Zaslavsky

A. M.

Kessler

R. C.

(2012). Childhood adversities and first onset of psychiatric disorders in a national sample of US adolescents. Archives of General Psychiatry, 69, 1151–1160. doi:10.1001/archgenpsychiatry.2011.2277

31.

Middleton

J. S.

Potter

C. C.

(2015). Relationship between vicarious traumatization and turnover among child welfare professionals. Journal of Public Child Welfare, 9, 195–216.

32.

Miller

E. A.

Green

A. E.

Fettes

D. L.

Aarons

G. A.

(2011). Prevalence of maltreatment among youths in public sectors of care. Child Maltreat, 16, 196–204. doi:10.1177/1077559511415091

33.

Nemeroff

C. B.

Bremner

J. D.

Foa

E. B.

Mayberg

H. S.

North

C. S.

Stein

M. B.

(2006). Posttraumatic stress disorder: A state-of-the-science review. Journal of Psychiatric Research, 40, 1–21.

34.

Pecora

P. J.

Jensen

P. S.

Romanelli

L. H.

Jackson

L. J.

Ortiz

(2009). Mental health services for children placed in foster care: An overview of current challenges. Child Welfare, 88, 5–26.

35.

Saunders

B. E.

(2003). Understanding children exposed to violence: Toward an integration of overlapping fields. Journal of Interpersonal Violence, 18, 356–376. doi:10.1177/0886260502250840

36.

Saunders

B. E.

Hanson

R. F.

(2014). Innovative methods for implementing evidence-supported interventions for mental health treatment of child and adolescent victims of violence. In Reece

R. M.

Hanson

R. F.

Sargent

(Eds.), Treatment of Child Abuse: Common Ground for Mental Health, Legal, and Medical Practitioners (2nd ed., pp. 235–245) Baltimore, MD: Johns Hopkins University Press.

37.

Sheldon

G. H.

Tavenner

Hyde

P. S.

(2013). Tri-agency letter on trauma. Retrieved from http://www.medicaid.gov/Federal-Policy-Guidance/Downloads/SMD-13-07-11.pdf

38.

Stambaugh

L. F.

Ringeisen

Casanueva

C. C.

Tueller

Smith

K. E.

Dolan

(2013). Adverse childhood experiences in NSCAW (OPRE Report #2013-26). Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

39.

Strand

V. C.

Spath

Bosco-Ruggiero

(2010). So you have a stable child welfare workforce—What’s next. Children and Youth Services Review, 32, 338–345.

40.

Substance Abuse and Mental Health Services Administration. (2014). SAMHSA’s concept of trauma and guidance for a trauma-informed approach. HHS Publication No. (SMA) 14-4884. Rockville, MD: Substance Abuse and Mental Health Services Administration.

41.

Sullivan

Preisler

Ake

G. S.

Potter

Beck

(2012). North Carolina child welfare trauma readiness assessment. Raleigh, NC: North Carolina Department of Health and Human Services, Division of Social Services, Funded by the Department of Health and Human Services, Administration for Children, Youth and Families, Children’s Bureau Grant #90CO1058.

42.

Verardi

Croux

(2009). Robust regression in Stata. The Stata Journal, 9, 438–453.