Uncovering determinants of perceived feasibility of TF-CBT through coincidence analysis

Abstract

Introduction

A mental health provider's perception of how well an intervention can be carried out in their context (i.e., feasibility) is an important implementation outcome. This article aims to identify determinants of feasibility of trauma-focused cognitive behavioral therapy (TF-CBT) through a case-based causal approach.

Method

Data come from an implementation-effectiveness study in which lay counselors (teachers and community health volunteers) implemented a culturally adapted manualized mental health intervention, TF-CBT, delivered to teens who were previously orphaned and were experiencing posttraumatic stress symptoms and prolonged grief in Western Kenya. The intervention team identified combinations of determinants that led to feasibility among teacher- and community health volunteer-counselors through coincidence analysis.

Results

Among teacher-counselors, organizational-level factors (implementation climate, implementation leadership) determined moderate and high levels of feasibility. Among community health volunteer-counselors, a strong relationship between a clinical supervisor and the supervisee was the most influential determinant of feasibility.

Conclusion

Methodology and findings from this article can guide the assessment of determinants of feasibility and the development of implementation strategies for manualized mental health interventions in contexts like Western Kenya.

Plain Language Summary

A mental health provider's perception of how easy a therapy is to use in their work setting (i.e., feasibility) can impact whether the provider uses the therapy in their setting. Implementation researchers have recommended finding practices and constructs that lead to important indicators that a therapy will be used. However, limited research to our knowledge has searched and found practices and constructs that might determine feasibility of a therapy. This article uses existing data from a large trial looking at the continued use of a trauma-focused therapy to find practices and constructs that lead to moderate and high levels of feasibility. We found that in settings with a strong organizational structure that organization and leadership support for the therapy led to teachers in Kenya to perceive the therapy as easy to use. On the other hand, in settings with a weaker organizational structure, outside support from a clinical supervisor led to community health volunteers in Kenya perceiving the therapy as easy to use. The findings from this article can guide context-specific recommendations for increasing perceived therapy feasibility at the provider-, organization-, and policy levels.

Keywords

feasibility evidence-based determinants global global mental health interventions community-based children/child and adolescent/youth/family mental health treatment task shifting coincidence analysis configurational comparative method

Introduction

Feasibility, or the degree to which an intervention can be successfully carried out in a given context, is an important implementation outcome and conceptually, a close determinant of implementation success (Karsh, 2004; Proctor et al., 2011). Feasibility should be—and in some cases has been—considered and prioritized during treatment development and adaptation (e.g., Seitz et al., 2014; Stewart et al., 2020; Tutty et al., 2005). If an evidence-based treatment (EBT) is perceived to be effective and appropriate but is not feasible in a certain context, adoption may be impeded. Feasibility may be particularly important to direct service providers who make the final decision to implement an EBT in their setting (Proctor et al., 2011). In low-resource settings where there may be fewer external implementation supports, decisions to implement an EBT likely rely more heavily on the providers’ perception of whether the EBT can be feasibly implemented (Brooke-Sumner et al., 2015; Mendenhall et al., 2014). Thus, understanding what leads a provider to consider an EBT feasible in their context—and specifically in low-resource settings—has the potential to guide both the implementation process and the identification of strategies to bolster implementation success.

Recent reviews have called for the exploration and identification of determinants of implementation outcomes, like feasibility, to better guide the implementation of EBTs (e.g., Williams & Beidas, 2019). Williams and Beidas (2019) synthesized determinants of implementation success of evidence-based practice within child psychology and psychiatry in their review guided by existing theory and literature. They found most determinants of implementation success to fall within two levels: provider and organizational levels. Given the limited research connecting implementation determinants to feasibility, we followed Williams and Beidas’s (2019) suggestions and their integrated organizational culture-self-determination theory of evidence-based practice (see Figure 1 for a conceptual diagram of possible organizational- and provider-level determinants of feasibility) and literature outlined below.

Figure 1

Organizational-Level and Provider-Level Determinants of Feasibility

Provider-Level Determinants

Self-efficacy is a provider-level construct that refers to a provider's belief that they are able and capable of providing an EBT. Research indicates that self-efficacy in providing an EBT can impact EBT use (David & Schiff, 2017; Finch et al., 2020; Harned et al., 2013; Shapiro et al., 2012), EBT quality (Schiele et al., 2014), EBT efficacy (Caron et al., 2022; Rohrbach et al., 1993), and EBT attitudes (Schiele et al., 2014). Given the connection between self-efficacy and implementation success across various contexts, self-efficacy is often targeted throughout the implementation of EBTs (e.g., Cashwell & Dooley, 2001; Greason & Cashwell, 2009).

Work-related stressors, including burnout, fatigue, and low job satisfaction, are also important determinants of implementation success. Burnout can include increased emotional exhaustion while working, increased feelings of disconnection from one's role, and decreased feelings of accomplishment at work (Yang & Hayes, 2020). Yang and Hayes (2020) found that burnout impacts physical and psychological well-being, job satisfaction, client engagement, and clinical outcomes. In addition, burnout can lead to less empathy, poor therapeutic alliance, poor communication, and decreased attention in the therapy context (Salyers et al., 2015). On the other hand, decreasing burnout among therapists can increase perceived effectiveness of an intervention as shown in a study of trauma-focused EBT training that incorporates self-care education (Harrison et al., 2023). Therapist fatigue can occur during or after a therapy session and has been shown to negatively impact client outcomes and therapy quality (Chui & Hill, 2020). Satisfaction with one's job, on the other hand, may buffer against negative therapeutic outcomes (Samios et al., 2013). Research in the medical field indicates that job satisfaction is related to a provider's belief in evidence-based practice (Melnyk et al., 2010), which is closely related to implementation success (Eslinger et al., 2020; Jensen-Doss et al., 2009). Although it is clear that work-related stressors are important factors in determining implementation success, it remains unclear how these variables may differentially impact feasibility in different contexts.

EBT knowledge refers to the level of knowledge and understanding that EBT providers have of a certain EBT that extends beyond general familiarity of the intervention. Behavior change theories indicate that the level of knowledge impacts the use and adoption of EBTs (Higa & Chorpita, 2008). Roudbarani et al. (2022) recently found that clinician knowledge of mental health-related topics (e.g., symptoms of mental health disorders and treatment plan development) predicted the clinician's intentions to use an EBT with children diagnosed with developmental disorders. Other research highlights that EBT knowledge may not necessarily influence their attitude toward or use of an EBT (Nakamura et al., 2011; Walrath et al., 2006). The mixed results may be in part due to the differential impact EBT knowledge may have on various implementation outcomes. Thus, exploring how the level of knowledge of the intervention itself can impact whether an intervention is implemented and considered feasible is important.

Another important determinant of successful implementation is the extent to which a provider intends to use an intervention (Huijg et al., 2015). The theory of planned behavior presents intentions as a predictor of behavior (Ajzen, 1991). A provider will not implement an intervention if they have no intention to do so, yet a better understanding of the degree to which someone intends to implement the EBT may be important. For example, a health services-based meta-analysis found provider intentions to be associated with provider behaviors (Eccles et al., 2006; Godin et al., 2008). Mental health research—albeit limited—found similar connections between a greater degree of a provider's intention to utilize an EBT and EBT utilization (Fishman et al., 2018; Williams, 2015b). A more recent study that assessed the predictive validity of teachers’ behavioral intentions in using evidence-based interventions found that behavioral intentions explained the variance in implementation of the interventions (Fishman et al., 2020). In addition to EBT utilization and behaviors, a provider's intention to use an EBT is likely related to provider-perceived EBT feasibility.

Clinical supervision includes the provision of ongoing support by an EBT expert to EBT providers (Bearman et al., 2017) and is commonly used as a post-EBT training strategy to improve EBT implementation (Falender & Shafranske, 2004; Hoge et al., 2011; Schoenwald et al., 2013). Supervision has been shown to decrease turnover (Jovanović et al., 2016), increase self-efficacy (Gibson et al., 2009; Lohani & Sharma, 2023), and increase fidelity in providing EBTs (Bearman et al., 2017). Supervision may also play a role in increasing providers’ perceptions of feasibility, as the supervisor supports providers in their skill development and ability to address barriers to implementing an EBT in their setting. In addition to the practical skills and strategies providers learn in supervision, the relationship between a supervisor and their supervisee can play an important role in implementation success. A strong supervisory relationship and alliance have been associated with increased self-efficacy (Morrison & Lent, 2018; Park et al., 2019), increased therapeutic alliance (Park et al., 2019), increased perceived EBT effectiveness (Teichman et al., 2023), and enhanced clinical care (Patton & Kivlighan, 1997). Thus, both the skills learned in supervision and the relationship between a supervisor and supervisee may be important determinants of feasibility.

Organizational-Level Determinants

Implementation climate is the shared perception that a specific intervention is expected, supported, and rewarded within an organization (Klein et al., 2001; Klein & Sorra, 1996). Although there is limited research investigating the impact of implementation climate on EBT implementation outcomes (Williams et al., 2018), Williams et al. (2020) examined data from 30 outpatient children's mental health clinics and found that improvements in implementation climate predicted wider EBT utilization. In a separate study, Williams, Becker-Haimes, Schriger, et al. (2022) similarly found that implementation climate predicted increased adherence and competence in providing EBTs. In a cross-sectional analysis, which may be limited in its ability to detect a temporal relationship, Becker-Haimes et al. (2017) failed to find a significant association between implementation climate and EBT utilization within community mental health settings. Given the possible connection between implementation climate and implementation success, exploration of the differential impact of implementation climate on implementation outcomes, and specifically, feasibility is warranted.

Implementation leadership, which is operationalized as leadership behaviors that support the implementation of an EBT (Aarons et al., 2014), is an important determinant of implementation success (Meza et al., 2021). Powell et al. (2017) examined leadership within 19 mental health agencies in Pennsylvania and found that implementation leadership was associated with provider attitudes toward an EBT. Similarly, a study of community mental health clinics in California found a significant relationship between implementation leadership and other implementation constructs (organizational climate, burden, and EBT use; Brimhall et al., 2016). Other studies have connected implementation leadership to provider attitudes toward an EBT (Aarons, 2006), implementation climate (Williams et al., 2020), and organizational culture (Aarons, Ehrhart, Torres, et al., 2017). Williams, Hugh, Cooney, et al. (2022) found that implementation leadership is indirectly associated with fidelity through implementation climate. Thus, implementation leadership may positively impact feasibility as increased leadership support can make it easier for providers to implement an intervention in their organization.

Differential Impact of Determinants

Provider- and organizational-level determinants may have a differential impact on feasibility depending on the context in which an EBT is implemented. For example, in a context with a more diffuse organizational structure, certain provider-level determinants may influence feasibility the most. Alternatively, in a setting with high levels of implementation climate, determinants like self-efficacy may be less important. Powell et al. (2020) conducted a systematic review of determinants of implementing a trauma-focused EBT for children. The authors found determinants at various levels that may contribute to implementation success, yet ultimately highlight the need to explore how determinants may work together to influence implementation outcomes (Powell et al., 2020). To address the likely complexity in determinants of feasibility, different analytic approaches are needed to uncover these complex patterns and variations in important determinants across settings.

Study Aims

This present study aims to identify provider- and organizational-level determinants of feasibility of an adapted version of trauma-focused cognitive behavioral therapy (TF-CBT). Given the lack of existing literature on determinants of feasibility, this project is largely exploratory. To address our main aim of identifying determinants of feasibility, we conducted a configurational comparative method, coincidence analysis (CNA), utilizing data from an Implementation-Effectiveness trial of a culturally adapted version of TF-CBT (Pamoja Tunaweza [PT]; Dorsey, Gray, Wasonga, et al., 2020).

Method

Parent Study

Participants

The data from this secondary data analysis come from a Hybrid Type II Implementation-Effectiveness trial of PT, delivered by teachers (n = 125) and community health volunteers (CHVs; n = 123) in 40 primary schools in Bungoma, Kenya (Building and Sustaining Interventions for Children [BASIC]: Task Sharing Mental Health Care in Low-Resource Settings; Dorsey, Gray, Wasonga, et al., 2020). Bungoma is the second most populous county in Kenya and includes 137 public primary schools in total. At each school, children ages 11–14 who experienced parental death, posttraumatic stress symptoms, and/or prolonged grief were randomized to receive PT by either teacher-counselors or CHV-counselors. Thus, as a part of the PT delivery, each school included three teacher-counselors, one site leader (head teacher), three CHV-counselors, and one site leader (community health extension worker [CHEW]) for a planned total of 240 counselors and 80 site leaders across the 40 schools. There was some teacher- and CHV-counselor turnover, so the final sample included 248 counselors. There was also some overlap for CHEWs who provided oversight to CHV-counselors at multiple schools. Each counselor triad conducted two sequential, 8-week PT groups, one with girls and one with boys. 849 children received PT as a part of BASIC. Five Kenyan trainers and supervisors supported the implementation of PT by teacher- and CHV-counselors. Data for the secondary data analysis come from the 248 teacher- and CHV-counselors that provided PT to the 40 schools (see Table 1 counselor demographics). All participants consented to their participation in the study.

Table 1

Counselor Demographics

Demographic variable	CHVs (n = 123)	Teachers (n = 125)
Age [M (SD)]	43.1 (9.9)	42.1 (7.6)
Female (%)	70.0%	62.4%
Time at job (years)	7.2 (4.4)	7.0 (6.1)
Highest education (%)
Diploma certificate	5.7%	77.6%
Primary education	24.4%	0.0%
Secondary education	69.1%	4.8%
Master's degree	0.0%	17.6%
Prior training in MH care (yes)	52.0%	48.8%
Prior provision of MH care (yes)	67.5%	80.0%

Note. CHVs = community health volunteers; M = mean; SD = standard deviation; MH = mental health.

Procedures

We follow the six practical recommendations for improved implementation outcome reporting in this article (Lengnick-Hall et al., 2022). BASIC follows an incomplete stepped wedge cluster randomized design in that schools are randomly assigned to sequences in which counselors receive PT training and provide PT to their site (Dorsey, Gray, Wasonga, et al., 2020). The incomplete stepped wedge design includes seven sequences with different study start times. At a given point, one cluster of sites will begin PT training and implementation, and another cluster of sites acts as the control group. However, the secondary data analysis does not use data from the control groups. There are a total of seven sequences such that there are seven rounds of training and implementation of PT. Upon completion of the first sequence, the study team designed coaching to help support counselors in the following sequences. The implementation phase ends once the counselors deliver two PT groups (and sites move into the sustainment phase). All PT groups were held in the children's schools. At the time of writing this article, the sustainment phase is still ongoing. All procedures were reviewed and approved by the Duke University Institutional Review Board (IRB) and the Kenya Medical Research Institute (IRB).

Pamoja Tunaweza

PT (Together We Can in Kiswahili) was designed based on TF-CBT, an EBT for children who experience psychological distress following a traumatic event (Cohen et al., 2016). The Kenyan team and Tanzanian counselors and supervisors from a pilot study (O'Donnell et al., 2014) and randomized control trial (RCT; Dorsey, Lucid, Martin, et al., 2020) adapted TF-CBT to create PT. Some adaptations include leveraging lay providers, conducting the treatment in a group setting, calling the intervention a class, and tailoring terminology and stories to be appropriate in the given context (Woods-Jaeger et al., 2017). PT originally included 12 group sessions in the pilot and RCT but was shortened for BASIC. The EBT includes 8 group sessions and 2–4 individual sessions. Within each PT group, two counselors conduct the children's sessions, and one counselor conducts the guardians’ sessions. Children are eligible to participate in the group if they experienced parental death, are between 11 and 14 years of age, and report prolonged grief and/or posttraumatic stress. For more information regarding the parent study, please review the protocol (Dorsey, Gray, Wasonga, et al., 2020).

Secondary Data Analysis

This secondary data analysis includes data from a total of 248 counselors in BASIC (125 teachers and 123 CHVs) during the implementation phase of the parent study.

Measures

Measurement Approach. Given the Kenyan context of this study and the high-income country (HIC) origin of many implementation measures, the Kenyan and U.S.-based research teams dedicated substantial time to measurement selection. We prioritized existing measures of implementation constructs that were developed in low-to-middle-income countries (LMICs) (Haroz et al., 2019). However, when those measures were not available, we selected implementation measures developed in HICs and worked with the Kenyan research teams to adapt, translate, and back-translate each measure. In adapting measures for the study context, the U.S.-based and Kenyan research teams first discussed the implementation construct definitions (Proctor et al., 2011) and then edited each measure item to be best understood in the study context. When a measure developed in an LMIC or HIC that was appropriate for the study context was unavailable, we created measures following the theoretical domains framework (TDF; Atkins et al., 2017; Huijg et al., 2014). The TDF outlines different domains and determinants of behavior to help guide the development of behavior-based measures. We calculated the inter-item correlations (IIC) for each measure included in these analyses and required a minimum of 0.20 for each measure item (Piedmont, 2014). Counselors who were enrolled in the parent study were invited to complete quantitative surveys that included each measure described below by the study team over the phone and through WhatsApp. Trained interviewers met with each counselor in-person to verbally conduct the quantitative survey after the PT training and after PT implementation (and associated time points among the control participants). The trained interviewers recorded counselor reports via tablets and an online survey collection platform, Qualtrics.

Measures. Our primary outcome is perceived feasibility, and we assessed several provider- and organizational-level determinants of perceived feasibility. The outcome and all potential determinants, including definitions, scale, and psychometric information are in Table 2.

Table 2

Outcome and Determinant Measures

Measure	Definition		Scale	Measure properties	Time point
Outcome			Scale
The Perceived Feasibility Intervention (Weiner et al., 2017)	Counselor's perception of PT's feasibility in their context. This measure delineates the core construct of feasibility and differentiates it from both appropriateness and acceptability.		Four items (e.g., “Pamoja Tunaweza is easy to use in this community”) on a 5-point Likert scale from agree to strongly disagree.	Weiner et al. (2017) found that the measure had acceptable internal consistency (α = 0.89) and test–retest reliability (r = 0.88). Mean IIC was acceptable (r = 0.51) for this study. Higher scores indicate higher levels of feasibility.	Post-implementation
Provider-level determinants
Perceived self-control and self-efficacy	Created using the TDF beliefs about the capabilities domain (Huijg et al., 2014) to capture counselors’ perceived control and self-efficacy in providing PT in the future.		Six items with a mix of self-control-specific (e.g., “I have control over delivering a Pamoja Tunaweza group in my school”) and self-efficacy-specific statements (e.g., “I am confident that I can deliver Pamoja Tunaweza”) rated on a 5-point Likert scale. Some responses are rated from very difficult to very easy and others are rated from strongly disagree and strongly agree.	This measure consists of mean IICs for self-control and self-efficacy subscales were acceptable (r = 0.47, r = 0.41, respectively). Higher scores indicate more perceived self-control and self-efficacy.	Post-training, pre-implementation
Pamoja Tunaweza Knowledge (Dorsey, Lucid, Martin, et al., 2020)	Counselor's knowledge of PT based on the TF-CBT knowledge test created for TF-CBT training and certification. The Kenyan and U.S.-based research teams adapted the measure to align with the modifications made to TF-CBT in PT.		19 items with either True/false or multiple-choice responses.	Higher scores indicate more PT knowledge.	Post-training, pre-implementation
Intentions to use Pamoja Tunaweza	Counselors’ intentions to implement PT in the future. The measure uses TDF domain items (Huijg et al., 2014), the Evidence-Based Treatment Intentions Scale (Williams, 2015a), and the John Hopkins University (JHU) Implementation Science scale (Haroz et al., 2019).		Four items (e.g., “I intend to provide Pamoja Tunaweza/TF-CBT to orphaned children in my community next month”) on a 4-point Likert scale (1 = not at all, 4 = a lot) and one item on a 3-point Likert scale (1 = not at all likely, 3 = very likely).	The final measure included mean IIC was acceptable (r = 0.27). Higher scores indicate more intentions to use PT.	Post-implementation
The Professional Quality of Life	Counselor's satisfaction, fatigue, and burnout related to providing PT (Stamm, 2010).		This measure included 13 items on a 5-point Likert scale from strongly disagree to strongly agree and never to always. We separated the measure by subscales: satisfaction (five items such as “I am proud of what I can do to help as a Pamoja Tunaweza/TF-CBT counselor”), fatigue (three items, such as “I am not as productive at work because I am losing sleep over traumatic experiences of a child/guardian I counseled in Pamoja Tunaweza/TF-CBT”), and burnout (five items, such as “I am tired of working with Pamoja Tunaweza/TF-CBT children/guardians”).	Mean IICs for burnout, fatigue, and satisfaction subscales were acceptable (r = 0.45, r = 0.41, r = 0.43, respectively). Higher scores indicate more satisfaction, fatigue, and burnout.	Post-implementation
Supervisory Relationship	The adapted version of the Short Supervisory Relationship Questionnaire (Cliffe et al., 2016). This measure includes 14 items across three subscales: safe-base, reflective education, and structure. The safe-base subscale measures the extent to which counselors found their supervisor to provide a collaborative and safe environment to discuss clinical cases. The reflective education subscale measures to what extent counselors find their supervisor to provide reflective and educational support during supervision. The structure subscale refers to whether the supervisors utilized the prescribed elements of supervision.		Seven-point Likert scale (1 = strongly disagree, 7 = strongly agree). The safe-base subscale includes eight items (e.g., “My supervisor was approachable”), the reflective education subscale includes three items (e.g., “My supervisor encouraged me to self-evaluate my delivery of PT”), and the structure subscale includes three items.	All items are rated on a Cliffe et al. (2016) found that each subscale had good internal consistency: safe-base α = 0.97, reflective education α = 0.89, and structure subscale α = 0.88. The overall measure had strong test–retest reliability (r = 0.94). In this study, the subscales had acceptable mean IICs (r = 0.42, 0.39, 0.21). Higher measure scores indicate a stronger perceived supervisory relationship by counselors.	Post-implementation
Organizational-level determinants
The Implementation Climate Scale (ICS) (Ehrhart et al., 2014; Jacobs et al., 2014).		Counselor's perception of whether PT was expected, supported, and rewarded in their organization (i.e., strategic implementation climate).	10 statements (e.g., “I get the support I need to use Pamoja Tunaweza”) rated on a 5-point Likert scale from strongly disagree to strongly agree (Ehrhart et al., 2014; Jacobs et al., 2014).	Fernandez et al. (2018) found the original ICS to have acceptable internal consistency (α = 0.72) and reliable organizational-level means (ICC = 0.22). We found the scale to have an acceptable mean IIC (r = 0.39). Higher scores indicate a more positive implementation climate as perceived by counselors.	Post-implementation
Implementation Leadership (Aarons et al., 2014)		An adapted version of the implementation leadership scale which measures counselors’ perception of leadership support for PT.	12 items (e.g., “leader is knowledgeable about Pamoja Tunaweza/TF-CBT”) rated on a 5-point Likert scale (0 = not at all, 4 = very great extent).	Aarons et al. (2014) found excellent internal consistency (ranging from α = 0.95–0.98) and acceptable reliability of organizational-level means (ICC = 0.29) for the original measure. We found the scale to have an acceptable mean IIC (r = 0.53). Higher scores indicate stronger implementation leadership. Counselors completed this measure for their designated leader (teachers: headteacher, CHVs: CHEW).	Post-implementation

Note. The Time Point column represents when the measures were administered during the parent study: BASIC. Post-training and pre-implementation measurements occurred after counselors were trained in delivering PT, but before they implemented PT. Post-implementation occurred after counselors completed the course of two PT groups. PT = Pamoja Tunaweza; IIC = inter-item correlations; TDF = theoretical domains framework; TF-CBT = trauma-focused cognitive behavioral therapy; CHVs = community health volunteers; CHEW = community health extension worker; BASIC = Building and Sustaining Interventions for Children.

Analytic Method

To examine implementation factors associated with the feasibility of PT, we utilized CNA, a configurational comparative method. As opposed to regression-based analytic methods, CNA relies on the regularity theory of causation (i.e., variables can be “difference-makers” such that their presence when certain other variables are present lead to a certain outcome; Whitaker et al., 2020). CNA identifies the minimum set of configurations (i.e., combinations) of conditions (i.e., determinants) that are necessary and sufficient for the presence or absence of a specific outcome (i.e., feasibility; Baumgartner & Thiem, 2015). Necessary conditions must be present for the outcome, while sufficient conditions are ones that alone can lead to the outcome (Gwayi-Chore et al., 2022).

Our reasons for selecting CNA are fourfold. First, CNA's inductive approach can address an exploratory research aim. Second, a case-based approach allows researchers and implementers to better consider the context in which the data are collected. Given that the purpose of this study is to guide implementation strategy selection for future implementing sites, it is critical to consider the research context and iteratively refer to the cases that inform the results. Third, CNA provides rich information on the differential and complex impact determinants may have on an outcome depending on the setting. We expect that determinants of feasibility and the configuration of those determinants will present differently depending on context. Lastly, we selected CNA over other configurational comparative methods because CNA is inherently redundancy-free (Baumgartner & Falk, 2021; Baumgartner & Thiem, 2015) and thus will identify the most parsimonious model. In low-resource settings, it is important to find the fewest determinants of feasibility possible given limited resources to address determinants. We conducted the analytic steps outlined in Baumgartner and Ambühl's (2020) guide for the “cna” function in R (Ambühl et al., 2020; R Core Team, 2021). First, we explored theoretical models (e.g., Williams & Beidas, 2019) and existing literature presented in the introduction that outline implementation determinants that lead to implementation success. Second, we engaged stakeholders with expertise in the implementation of the adapted TF-CBT to guide determinant selection (i.e., met with counselors, supervisors, and study team members to discuss possible determinants of feasibility). Third, we applied a practical lens in which we considered whether each determinant could be impacted by certain implementation strategies. Fourth, we conducted a statistical approach to further refine the determinant selection (i.e., the “msc” function) in which we first set consistency to 1.00 and coverage to 0.25 and lowered consistency by 0.05 until we found configurations that met the consistency and coverage levels. Upon completion of determinant selection, we conducted CNA to identify implementation determinants of feasibility setting our consistency to .75 and coverage to .50 (for more details, see the Supplemental Materials).

Results

Descriptive Statistics

The Kenyan counselors in our sample represented both the health (i.e., CHVs) and education sectors (i.e., teachers) in Bungoma, Kenya. Participating CHVs (n = 123) were predominantly female (70.0%), had a mean age of 43.1 (SD = 9.9), and their highest level of education was most commonly secondary education. Participating teachers (n = 125) were predominantly female (62.4%), had a mean age of 42.1 (SD = 7.6), and their highest level of education was most commonly a diploma certificate (Table 1). Among CHV-counselors, zero reported low levels of feasibility, 65 (53%) reported moderate levels of feasibility (i.e., moderate—more than or equal to 3.5, and less than 4.5), and 58 (47%) reported high levels of feasibility (i.e., more than or equal to 4.5). There were no missing data for CHV-counselors. Among teacher-counselors, three reported low levels of feasibility, 77 (62%) reported moderate levels of feasibility, and 45 (36%) reported high levels of feasibility. We were missing data from 3% of teacher-counselors. However, missing data were determined to be missing at random. See Table 3 for additional descriptive statistics.

Table 3

Descriptive Statistics

	M (SD)
Variable	Total sample	CHVs	Teachers
Outcome
Feasibility	4.33 (0.46)	4.40 (0.42)	4.25 (0.49)
Determinants
Self-control	4.32 (0.45)	4.44 (0.42)	4.19 (0.44)
Self-efficacy	4.08 (0.71)	4.18 (0.76)	3.99 (0.65)
PT knowledge	26.61 (3.28)	25.14 (3.54)	27.99 (2.28)
Intentions to use PT	3.85 (0.27)	3.88 (0.23)	3.81 (0.30)
Professional quality of life
Satisfaction	4.59 (0.48)	4.63 (0.45)	4.55 (0.52)
Fatigue	1.40 (0.54)	1.25 (0.46)	1.56 (0.56)
Burnout	1.53 (0.52)	1.49 (0.55)	1.56 (0.48)
Supervisory relationship
Safe base subscale	6.59 (0.44)	6.63 (0.38)	6.59 (0.49)
Reflective education subscale	6.43 (0.58)	6.39 (0.63)	6.47 (0.51)
Structure subscale	4.86 (0.43)	4.91 (0.45)	4.81 (0.40)
Implementation climate	4.36 (0.46)	4.45 (0.41)	4.26 (0.49)
Implementation leadership	2.82 (0.73)	3.04 (0.66)	2.59 (0.73)

Note. Only the outcome and determinants included in the final CNA models are included in this table. SD = standard deviation; CHVs = community health volunteers; PT = Pamoja Tunaweza; CNA = coincidence analysis.

Determinants of Feasibility Results: CNA Analysis

In total, we conducted four CNA models: moderate feasibility among CHVs, high feasibility among CHVs, moderate feasibility among teachers, and high feasibility among teachers. We conducted separate condition reductions per model, so we report the results split by model. All results are displayed in Tables 4 and 5. Each model includes at least one solution that may have at least one pathway to that solution. More CNA analytic details and results in Boolean algebra are in the Supplemental Materials. We maintain the same consistency and coverage levels for each model (consistency ≥ .75, coverage ≥ .50).

Table 4

CNA Solution for Moderate and High Feasibility Among CHVs

Moderate feasibility
Solution	Pathway	Conditions					Outcome	Consistency	Coverage
		Self-efficacy			Reflective education supervisory relationship	Implementation Climate	Feasibility
1	1	–			Moderate	–	Moderate	.75	.70
High feasibility
Solution	Pathway	Conditions					Outcome	Consistency	Coverage
		Burnout	Fatigue	Self-efficacy	Reflective education supervisory relationship	Structure supervisory relationship	Feasibility
1	1	Low	–	–	–	–	High	.77	.61
1	2	–	–	–	High	Low/moderate	High	.77	.61

Note. – indicates that the condition was not significant in the final model. CNA = coincidence analysis; CHVs = community health volunteers.

Table 5

CNA Solution for Moderate and High Feasibility Among Teachers

Moderate feasibility
Solution	Pathway	Conditions						Outcome	Consistency	Coverage
		Self-efficacy	Intentions to use PT	Reflective education supervisory relationship	Safe base supervisory relationship	Implementation climate	Implementation leadership	Feasibility
1	1	–	–	–	–	Moderate	–	Moderate	.77	.54
1	2	Moderate	–	–	–	–	Moderate	Moderate	.77	.54
High feasibility
Solution	Pathway	Conditions						Outcome	Consistency	Coverage
		Burnout	Intentions to use PT	Reflective education supervisory relationship		Implementation climate	Implementation leadership	Feasibility
1	1	–	High	–		High	–	High	.79	.62
2	1	Moderate	–	–		High	–	High	.79	.52
3	1	–	–	High		High	High	High	.81	.50
3	2	–	–	Low/moderate		High	Low/moderate	High	.81	.50
4	1	–	Moderate	–		High	High	High	.81	.50
4	2	–	–	Low/moderate		High	Low/moderate	High	.81	.50
4	3	–	High	High		–	High	High	.81	.50

Note. – indicates that the condition was not significant in the final model. CNA = coincidence analysis; PT = Pamoja Tunaweza.

CHVs

Moderate Feasibility. Among CHVs, our analysis indicated one solution that led to moderate levels of feasibility: (1) if and only if CHV-counselors reported moderate levels of the reflective education supervisory relationship subscale, feasibility was moderate (Table 4).

High Feasibility. Among CHVs, we found one solution with two pathways that led to high levels of feasibility: (1) if and only if burnout was low or (2) the reflective education supervisory relationship subscale was high and the structure supervisory relationship subscale was moderate or low, feasibility was high among CHVs (Table 4).

Teachers

Moderate Feasibility. Among teachers, we found one solution with two pathways that led to moderate levels of feasibility: (1) if and only if the implementation climate was moderate or (2) self-efficacy was moderate and implementation leadership was moderate, feasibility was moderately rated among teachers (Table 5).

High Feasibility. Among teachers, we found four solutions that led to high feasibility (Table 5). Given similar consistency and coverage scores across all four solutions, we followed previous literature to explore model ambiguity (Gwayi-Chore et al., 2022). We found high levels of implementation climate across all four solutions, with high implementation climate (i.e., positive implementation climate) a necessary component of six of the seven possible pathways for high feasibility among teachers (only the third pathway in solution 4 did not include high implementation climate).

Discussion

Results from this secondary data analysis suggest that determinants of feasibility differ based on the context in which counselors are embedded. Overall, counselors in this sample relied on some sort of external support either alone or in combination with an internal provider-level construct to guide the perception of feasibility. However, we found differences in determinants of feasibility among CHVs and teachers that align with the context of our study and the organizational structure in which lay counselors work.

Community Health Volunteers. Among CHVs, who engage in community-based work and are less embedded in an organizational structure, provider-level variables (i.e., burnout and the supervisory relationship) impacted feasibility. CHVs do the majority of their work outside of the physical health facility to which they are connected. Other than some regular reporting and in-facility time, CHVs spend most of their time traveling within a community to provide health services. Within the context of the BASIC study, the CHV-counselors provided PT to children within the schools in their community, to which they were otherwise not connected. In addition, based on conversations with our research partners and counselors in Western Kenya, the CHEWs who were CHVs’ leaders within the health facilities were less involved in the day-to-day implementation of PT. Thus, CHV's perception of whether they could provide PT in their community likely relied more on their external intervention support such as clinical supervisors rather than internal organizational support.

Strategies that improve the relationship between counselors and their supervisors may be particularly important for CHVs. Enlow et al. (2019) present 14 recommendations to clinical supervisors that can improve the supervisory relationship. These recommendations include strategies that impact the differential between a supervisor's style and the supervisee's developmental level, the effect of contextual factors on the supervisee's performance, and the type of supervisor feedback (Enlow et al., 2019). The strategies recommended by Enlow et al. (2019) can be done with little to no cost and within routine supervision meetings. For example, a supervisor can provide positive feedback, use supervision contracts that are collaboratively developed, and focus on self-care and effective coping. Other scholars have investigated how to fix a ruptured supervisory relationship, which may also be useful in improving perceived feasibility if ruptures occur. For example, Friedlander (2015) describes relational strategies that can be enlisted to increase a supervisee's responsiveness to supervision. Strategies that address low burnout may similarly be important in enhancing feasibility among CHVs given the presence of low burnout as a condition for high feasibility among CHVs. Incorporating self-care into the training of EBTs may be a protective factor against burnout (Harrison et al., 2023). For example, Harrison et al. (2023) studied a TF-CBT training in which providers were encouraged to personally use the clinical strategies they would teach their clients (e.g., incorporate new behaviors in their routines and praising others). The emphasis on self-care within an EBT training is likely low cost and can have high impact on providers’ experience providing an EBT (Harrison et al., 2023). Despite the long list of recommendations, it is important to present multiple strategies to counselors and supervisors to tailor the strategies to their context.

Teachers. CNA results indicated that among teachers, organizational-level variables (i.e., implementation climate) when paired with other provider- and organizational-level variables were almost always necessary for moderate and high levels of feasibility. Teachers provided PT within their roles in a more specified organizational structure and physically provided PT at their own organizations. Teachers reported having stronger leadership support for the implementation of PT. Thus, we see implementation leadership and implementation climate rising to the top as determinants of feasibility among teachers.

Among teachers, it appears important to select strategies to enhance or harness high implementation leadership and climate. Williams et al. (2020) found that improved implementation leadership in an organization indirectly led to improvements in EBT use through a stronger implementation climate. Thus, finding strategies that target either implementation leadership or implementation climate may be promising for improving feasibility. Although few formal studies have identified strategies to improve implementation leadership and climate, the leadership and organizational change for implementation strategy (LOCI; Aarons et al., 2015; Aarons, Ehrhart, Moullin, et al., 2017; Skar et al., 2022) and the training in implementation practice leadership (TRIPLE; Proctor et al., 2019) strategy have preliminary supporting evidence and could be considered by organizations. The LOCI strategy includes training in an EBT and organizational-level supports for the EBT (Skar et al., 2022). Specifically, clinical organizational leaders received LOCI training sessions in which trainers taught principles of leadership and implementation climate, provided organizational feedback, and encouraged leaders to create personalized leadership plans to improve implementation leadership. Leaders participated in weekly coaching calls with the LOCI trainers to support the implementation of the leadership plan. Additional research may be needed to adapt LOCI for low-resource settings and incorporate LOCI fundamentals in routine leadership meetings. The TRIPLE strategy focuses on training clinical leaders in knowledge, skills, and tools needed to lead and evaluate the implementation of an EBT in an organization (Proctor et al., 2019). TRIPLE follows a similar format as LOCI and may require similar adaptations for low-resource settings.

Importantly, the results indicated that strategic support for an EBT was important for teachers rather than a molar organizational climate. Molar organizational climate differs from a strategic implementation climate in that it is the general support for the workforce and their well-being (Williams et al., 2018). Organizational climate was included as an initial determinant and was deemed unnecessary in each teacher pathway. Thus, future research and practice should focus on strategies that can be used to support the specific intervention within school-based settings. Despite the identification of some strategies to improve implementation leadership and climate, researchers and stakeholders should examine for whom and for what these strategies work best.

Practice and Policy Implications. Among providers, these findings can guide future implementation efforts to improve feasibility within similar contexts and motivate researchers to continue identifying strategies that address critical determinants (e.g., supervisory relationship, implementation climate, and implementation leadership). At the policy level, these results indicate that policy makers who wish to implement EBTs in their sector will need to consider the unique characteristics of the sector to determine the support needed for implementation of any given EBT. For example, CHVs have a less formal day-to-day supervisory structure whereas teachers arrive and work each day in a structured system of work and supervision that is less variable. Policy makers and implementors need to consider the structural day-to-day work setting and the expectations of work, supervision, and support to maximize the effectiveness of implementing within different work settings. In this case, the supports needed for CHVs are different from the supports needed for teachers because of the difference in work structures.

Strengths. A major strength of this article is the utilization of CNA to explore the causal chains of implementation determinants of feasibility in different contexts. CNA is a novel approach that is gaining traction within the implementation science field because it can uncover causal complexity—a common occurrence in real-world implementation (Whitaker et al., 2020). CNA allows researchers to better contextualize results within the cases and thus produces more practical findings that can be disseminated throughout the context in which the research was conducted. We are currently collaborating with our Kenyan research team to disseminate our findings to counselors and guardians who participated in the research to further contextualize our results. In these conversations, we hope to generate solutions to improve the feasibility of PT. The first author and principal investigators of this secondary data analysis do not have lived experience in Western Kenya or providing PT to children in Western Kenya, so it is critical to privilege the voices of those with lived experience in generating these solutions. The authors utilized CNA after the implementation of PT in this study, yet CNA can also be used to inform different stages of the implementation process. For example, CNA may be used to identify determinants of provider-perceived implementation and clinical outcomes prior to training to identify which strategies should be incorporated into the EBT training. In addition, CNA may be used during implementation to guide the selection of implementation strategies needed to augment implementation efforts (e.g., after a newly trained counselor provided one group of PT, CNA could be used to identify the strategies that should be harnessed before counselors provide additional PT groups).

Limitations. Despite the major strengths of the secondary data analysis, there are important limitations. First, CNA and the reporting of CNA are still developing. Second, we encountered model ambiguity in the results for teachers who reported high feasibility. Following previous research, we identified conditions that appeared in most solutions to find an important condition. However, other strategies can be used to identify the best model (e.g., exhaustiveness and faithfulness). Further research is needed to develop consistent and universal guidelines for assessing model fit within CNA. Third, we reduced the initial 15 variables that were included in the models to the variables described above using the “msc” function in R (see the Supplemental Materials) which helps to identify the minimally sufficient conditions for an outcome. The variables that were not included in the final models (e.g., organizational climate), were not described in this article given space restrictions. Fourth, other important determinants of feasibility may also exist that were not collected as a part of this study (e.g., level of funding within a school or the school's funding sponsor). Fifth, on average, counselors reported high feasibility and thus we were unable to identify determinants of low levels of feasibility. It is important to identify what determines low levels of feasibility in other contexts to better guide the implementation process. Nonetheless, high levels of feasibility indicate that the intervention was doable in the study context, and thus is an important implementation outcome of PT in Western Kenya. Lastly, this secondary data analysis did not include formal qualitative analysis that could better inform the results of CNA. We encourage future researchers who employ CNA and/or study feasibility in the context of teachers and CHVs to conduct additional qualitative measures to better understand the complex configurations of conditions and feasibility.

Conclusion

Using CNA, we identified organizational- and provider-level determinants of the feasibility of PT as perceived by lay counselors. Among CHV-counselors, the supervisory relationship was the most influential determinant of feasibility. Among teacher-counselors, organizational-level factors such as implementation climate and implementation leadership determined moderate and high levels of feasibility. Implementors and policy makers need to take into consideration the structures of the organizations in which they propose to implement EBTs and recognize that such structures are critical to understanding how to support implementation. Methodology and findings from this secondary data analysis can guide the assessment of determinants of feasibility and the development of implementation strategies in contexts like those in our investigation.

Supplemental Material

sj-docx-1-irp-10.1177_26334895231220277 - Supplemental material for Uncovering determinants of perceived feasibility of TF-CBT through coincidence analysis

Supplemental material, sj-docx-1-irp-10.1177_26334895231220277 for Uncovering determinants of perceived feasibility of TF-CBT through coincidence analysis by Clara Johnson, Rashed AlRasheed, Christine Gray, Noah Triplett, Anne Mbwayo, Andrew Weinhold, Kathryn Whetten and Shannon Dorsey in Implementation Research and Practice

Footnotes

Acknowledgments

We thank the Ace Africa leadership: Augustine I. Wasonga and Cyrilla Amanya. We also thank the Ace Africa PT supervisors and interviewers: Elijah Agala, Lilian Nandutu Aluka, Rose Chemuku, Oliver Hillary, Brigid Karakacha, Moses Malaba, Emmanuel Muli, Patrick Musa Masika, Micah Nalianya, Bernard Wafula Nambafu Nabalia, Michael Nangila, Sheila Wambui Nderitu, Daisy Anyango Okoth, Omariba Anne Nyaboke, Victor Shalakha, Annette Sulungai, Sylvia Wafula, and Nelly Wandera. We are grateful to the Ministry of Health, Ministry of Education, Teachers Service Commission, participating schools, communities, children, and families who are taking part in BASIC.

Declaration of Conflicting Interests

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Shannon Dorsey is an international TF-CBT trainer and has received compensation for TF-CBT training and consultation. All other authors declare no conflicting interests.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this research project was supported by a grant to the first and fifth authors by the National Institute of Mental Health (R01 MH112633; Shannon Dorsey and Kathryn Whetten, Multiple Prinicpal Investigators; P50 MH126219-02; Shannon Dorsey, Multiple Principle Investigator; F31 MH130041; Clara Johnson, Principal Investigator). The Center for Evidence and Implementation provided additional funding to support BASIC. The funders/sponsors have no role in study design or analysis.

ORCID iDs

Clara Johnson

Noah Triplett

Shannon Dorsey

Supplemental Material

Supplemental material for this article is available online.

References

Aarons

G. A.

(2006). Transformational and transactional leadership: Association with attitudes toward evidence-based practice. Psychiatric Services, 57(8), 1162–1169. https://doi.org/10.1176/ps.2006.57.8.1162

Aarons

G. A.

Ehrhart

M. G.

Farahnak

L. R.

(2014). The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9(1), 45. https://doi.org/10.1186/1748-5908-9-45

Aarons

G. A.

Ehrhart

M. G.

Farahnak

L. R.

Hurlburt

M. S.

(2015). Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 1–12. https://doi.org/10.1186/s13012-014-0192-y

Aarons

G. A.

Ehrhart

M. G.

Moullin

J. C.

Torres

E. M.

Green

A. E.

(2017). Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: A cluster randomized trial study protocol. Implementation Science, 12(1), 1–11. https://doi.org/10.1186/s13012-017-0562-3

Aarons

G. A.

Ehrhart

M. G.

Torres

E. M.

Finn

N. K.

Beidas

R. S.

(2017). The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health. Psychiatric Services, 68(2), 115–122. https://doi.org/10.1176/appi.ps.201600062

Ajzen

(1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T

Ambühl

Baumgartner

Epple

Parkkinen

Thiem

(2020). cna: Causal modeling with Coincidence Analysis (R Package Version 3.0. 0) [Computer software]. https://cran.r-project.org/package=cna

Atkins

Francis

Islam

O’Connor

Patey

Ivers

Foy

Duncan

E. M.

Colquhoun

Grimshaw

J. M.

(2017). A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implementation Science, 12(1), 1–18. https://doi.org/10.1186/s13012-017-0605-9

Baumgartner

Ambühl

(2020). cna: An R Package for configurational causal inference and modeling.

10.

Baumgartner

Falk

(2021). Configurational causal modeling and logic regression. Multivariate Behavioral Research, 1–19. https://doi.org/10.1080/00273171.2021.1971510

11.

Baumgartner

Thiem

(2015). Identifying complex causal dependencies in configurational data with coincidence analysis. The R Journal, 7(1), 176. https://doi.org/10.32614/RJ-2015-014

12.

Bearman

S. K.

Schneiderman

R. L.

Zoloth

(2017). Building an evidence base for effective supervision practices: An analogue experiment of supervision to increase EBT fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 44(2), 293–307. https://doi.org/10.1007/s10488-016-0723-8

13.

Becker-Haimes

E. M.

Okamura

K. H.

Wolk

C. B.

Rubin

Evans

A. C.

Beidas

R. S.

(2017). Predictors of clinician use of exposure therapy in community mental health settings. Journal of Anxiety Disorders, 49, 88–94. https://doi.org/10.1016/j.janxdis.2017.04.002

14.

Brimhall

K. C.

Fenwick

Farahnak

L. R.

Hurlburt

M. S.

Roesch

S. C.

Aarons

G. A.

(2016). Leadership, organizational climate, and perceived burden of evidence-based practice in mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 43(5), 629–639. https://doi.org/10.1007/s10488-015-0670-9

15.

Brooke-Sumner

Petersen

Asher

Mall

Egbe

C. O.

Lund

(2015). Systematic review of feasibility and acceptability of psychosocial interventions for schizophrenia in low and middle income countries. BMC Psychiatry, 15(1), 1–12. https://doi.org/10.1186/s12888-015-0400-6

16.

Caron

Drake

K. L.

Stewart

C. E.

Muggeo

M. A.

Ginsburg

G. S.

(2022). Intervention adherence and self-efficacy as predictors of child outcomes in school nurse-delivered interventions for anxiety. The Journal of School Nursing, 38(3), 249–258. https://doi.org/10.1177/1059840520925522

17.

Cashwell

T. H.

Dooley

(2001). The impact of supervision on counselor self-efficacy. The Clinical Supervisor, 20(1), 39–47. https://doi.org/10.1300/J001v20n01_03

18.

Chui

Hill

C. E.

(2020). Keep calm and alert and carry on: Therapist calmness and fatigue in relation to session process. Counselling Psychology Quarterly, 33(1), 66–78. https://doi.org/10.1080/09515070.2018.1502160

19.

Cliffe

Beinart

Cooper

(2016). Development and validation of a short version of the supervisory relationship questionnaire. Clinical Psychology & Psychotherapy, 23(1), 77–86. https://doi.org/10.1002/cpp.1935

20.

Cohen

J. A.

Mannarino

A. P.

Deblinger

(2016). Treating trauma and traumatic grief in children and adolescents. Guilford Press.

21.

David

Schiff

(2017). Self-Efficacy as a mediator in bottom-up dissemination of a research-supported intervention for young, traumatized children and their families. Journal of Evidence-Informed Social Work, 14(2), 53–69. https://doi.org/10.1080/23761407.2017.1298072

22.

Dorsey

Gray

C. L.

Wasonga

A. I.

Amanya

Weiner

B. J.

Micha Belden

Martin

Meza

R. D.

Weinhold

A. K.

Soi

Murray

L. K.

Lucid

Turner

E. L.

Mildon

Whetten

(2020). Advancing successful implementation of task-shifted mental health care in low-resource settings (BASIC): Protocol for a stepped wedge cluster randomized trial. BMC Psychiatry, 20, 10. https://doi.org/10.1186/s12888-019-2364-4

23.

Dorsey

Lucid

Martin

King

K. M.

O'Donnell

Murray

L. K.

Wasonga

A. I.

Itemba

D. K.

Cohen

J. A.

Manongi

Whetten

(2020). Effectiveness of task-shifted trauma-focused cognitive behavioral therapy for children who experienced parental death and posttraumatic stress in Kenya and Tanzania: A randomized clinical trial. JAMA Psychiatry, 77(5), 464–473. https://doi.org/10.1001/jamapsychiatry.2019.4475

24.

Eccles

M. P.

Hrisos

Francis

Kaner

E. F.

Dickinson

H. O.

Beyer

Johnston

(2006). Do self-reported intentions predict clinicians’ behaviour: A systematic review. Implementation Science, 1(1), 1–10. https://doi.org/10.1186/1748-5908-1-28

25.

Ehrhart

M. G.

Aarons

G. A.

Farahnak

L. R.

(2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9(1), 1–11. https://doi.org/10.1186/s13012-014-0157-1

26.

Enlow

P. T.

McWhorter

L. G.

Genuario

Davis

(2019). Supervisor–supervisee interactions: The importance of the supervisory working alliance. Training and Education in Professional Psychology, 13(3), 206. https://doi.org/10.1037/tep0000243

27.

Eslinger

Sprang

Ascienzo

Silman

(2020). Fidelity and sustainability in evidence-based treatments for children: An investigation of implementation determinants. Journal of Family Social Work, 23(2), 177–196. https://doi.org/10.1080/10522158.2020.1724581

28.

Falender

C. A.

Shafranske

E. P.

(2004). Clinical supervision: A competency-based approach. American Psychological Association. https://doi.org/10.1037/10806-000

29.

Fernandez

M. E.

Walker

T. J.

Weiner

B. J.

Calo

W. A.

Liang

Risendal

Friedman

D. B.

S. P.

Williams

R. S.

Jacobs

(2018). Developing measures to assess constructs from the inner setting domain of the consolidated framework for implementation research. Implementation Science, 13(1), 1–13. https://doi.org/10.1186/s13012-018-0736-7

30.

Finch

Ford

Grainger

Meiser-Stedman

(2020). A systematic review of the clinician related barriers and facilitators to the use of evidence-informed interventions for post traumatic stress. Journal of Affective Disorders, 263, 175–186. https://doi.org/10.1016/j.jad.2019.11.143

31.

Fishman

Beidas

Reisinger

Mandell

D. S.

(2018). The utility of measuring intentions to use best practices: A longitudinal study among teachers supporting students with autism. Journal of School Health, 88(5), 388–395. https://doi.org/10.1111/josh.12618

32.

Fishman

Lushin

Mandell

D. S.

(2020). Predicting implementation: Comparing validated measures of intention and assessing the role of motivation when designing behavioral interventions. Implementation Science Communications, 1(1), 1–10. https://doi.org/10.1186/s43058-020-00050-4

33.

Friedlander

M. L.

(2015). Use of relational strategies to repair alliance ruptures: How responsive supervisors train responsive psychotherapists. Psychotherapy, 52(2), 174. https://doi.org/10.1037/a0037044

34.

Gibson

J. A.

Grey

I. M.

Hastings

R. P.

(2009). Supervisor support as a predictor of burnout and therapeutic self-efficacy in therapists working in ABA schools. Journal of Autism and Developmental Disorders, 39(7), 1024–1030. https://doi.org/10.1007/s10803-009-0709-4

35.

Godin

Bélanger-Gravel

Eccles

Grimshaw

(2008). Healthcare professionals’ intentions and behaviours: A systematic review of studies based on social cognitive theories. Implementation Science, 3(1), 1–12. https://doi.org/10.1186/1748-5908-3-36

36.

Greason

P. B.

Cashwell

C. S.

(2009). Mindfulness and counseling self-efficacy: The mediating role of attention and empathy. Counselor Education and Supervision, 49(1), 2–19. https://doi.org/10.1002/j.1556-6978.2009.tb00083.x

37.

Gwayi-Chore

M.-C.

Aruldas

Avokpaho

Chirambo

C. M.

Kaliappan

S. P.

Houngbégnon

Togbevi

C. I.

Chabi

Nindi

Simwanza

(2022). Defining optimal implementation packages for delivering community-wide mass drug administration for soil-transmitted helminths with high coverage. BMC Health Services Research, 22(1), 1–10. https://doi.org/10.1186/s12913-022-08080-5

38.

Harned

M. S.

Dimeff

L. A.

Woodcock

E. A.

Contreras

(2013). Predicting adoption of exposure therapy in a randomized controlled dissemination trial. Journal of Anxiety Disorders, 27(8), 754–762. https://doi.org/10.1016/j.janxdis.2013.02.006

39.

Haroz

E. E.

Bolton

Nguyen

A. J.

Lee

Bogdanov

Bass

Singh

N. S.

Doty

S. B.

Murray

(2019). Measuring implementation in global mental health: Validation of a pragmatic implementation science measure in eastern Ukraine using an experimental vignette design. BMC Health Services Research, 19(1), 1–11. https://doi.org/10.1186/s12913-019-4097-y

40.

Harrison

J. P.

Deblinger

Pollio

Cooper

Steer

R. A.

(2023). TF-CBT training augmented with a self-care focus: Understanding facilitators and barriers to treatment implementation. Community Mental Health Journal, 59(7), 1409–1421. https://doi.org/10.1007/s10597-023-01130-0

41.

Higa

C. K.

Chorpita

B. F.

(2008). Evidence-based therapies: Translating research into practice. In Steele

R. G.

Elkin

T. D.

Roberts

M. C.

(Eds.), Handbook of evidence-based therapies for children and adolescents (pp. 45–61). Springer. https://doi.org/10.1007/978-0-387-73691-4_4

42.

Hoge

M. A.

Migdole

Farkas

M. S.

Ponce

A. N.

Hunnicutt

(2011). Supervision in public sector behavioral health: A review. The Clinical Supervisor, 30(2), 183–203. https://doi.org/10.1080/07325223.2011.604276

43.

Huijg

J. M.

Gebhardt

W. A.

Dusseldorp

Verheijden

M. W.

van der Zouwe

Middelkoop

B. J.

Crone

M. R.

(2014). Measuring determinants of implementation behavior: Psychometric properties of a questionnaire based on the theoretical domains framework. Implementation Science, 9(1), 33. https://doi.org/10.1186/1748-5908-9-33

44.

Huijg

J. M.

Gebhardt

W. A.

Verheijden

M. W.

van der Zouwe

de Vries

J. D.

Middelkoop

B. J.

Crone

M. R.

(2015). Factors influencing primary health care professionals’ physical activity promotion behaviors: A systematic review. International Journal of Behavioral Medicine, 22(1), 32–50. https://doi.org/10.1007/s12529-014-9398-2

45.

Jacobs

S. R.

Weiner

B. J.

Bunger

A. C.

(2014). Context matters: Measuring implementation climate among individuals and groups. Implementation Science, 9(1), 1–14. https://doi.org/10.1186/1748-5908-9-46

46.

Jensen-Doss

Hawley

K. M.

Lopez

Osterberg

L. D.

(2009). Using evidence-based treatments: The experiences of youth providers working under a mandate. Professional Psychology: Research and Practice, 40(4), 417. https://doi.org/10.1037/a0014690

47.

Jovanović

Podlesek

Volpe

Barrett

Ferrari

Kuzman

M. R.

Wuyts

Papp

Nawka

Vaida

(2016). Burnout syndrome among psychiatric trainees in 22 countries: Risk increased by long working hours, lack of supervision, and psychiatry not being first career choice. European Psychiatry, 32, 34–41. https://doi.org/10.1016/j.eurpsy.2015.10.007

48.

Karsh

(2004). Beyond usability: Designing effective technology implementation systems to promote patient safety. BMJ Quality & Safety, 13(5), 388–394. https://doi.org/10.1136/qshc.2004.010322

49.

Klein

K. J.

Conn

A. B.

Sorra

J. S.

(2001). Implementing computerized technology: An organizational analysis. Journal of Applied Psychology, 86(5), 811. https://doi.org/10.1037/0021-9010.86.5.811

50.

Klein

K. J.

Sorra

J. S.

(1996). The challenge of innovation implementation. Academy of Management Review, 21(4), 1055–1080. https://doi.org/10.5465/amr.1996.9704071863

51.

Lengnick-Hall

Gerke

D. R.

Proctor

E. K.

Bunger

A. C.

Phillips

R. J.

Martin

J. K.

Swanson

J. C.

(2022). Six practical recommendations for improved implementation outcomes reporting. Implementation Science, 17(1), 16. https://doi.org/10.1186/s13012-021-01183-3

52.

Lohani

Sharma

(2023). Effect of clinical supervision on self-awareness and self-efficacy of psychotherapists and counselors: A systematic review. Psychological Services, 20(2), 291–299. https://doi.org/10.1037/ser0000693

53.

Melnyk

B. M.

Fineout-Overholt

Giggleman

Cruz

(2010). Correlates among cognitive beliefs, EBP implementation, organizational culture, cohesion and job satisfaction in evidence-based practice mentors from a community hospital system. Nursing Outlook, 58(6), 301–308. https://doi.org/10.1016/j.outlook.2010.06.002

54.

Mendenhall

De Silva

M. J.

Hanlon

Petersen

Shidhaye

Jordans

Luitel

Ssebunnya

Fekadu

Patel

Tomlinson

Lund

(2014). Acceptability and feasibility of using non-specialist health workers to deliver mental health care: Stakeholder perceptions from the PRIME district sites in Ethiopia, India, Nepal, South Africa, and Uganda. Social Science & Medicine, 118, 33–42. https://doi.org/10.1016/j.socscimed.2014.07.057

55.

Meza

R. D.

Triplett

N. S.

Woodard

G. S.

Martin

Khairuzzaman

A. N.

Jamora

Dorsey

(2021). The relationship between first-level leadership and inner-context and implementation outcomes in behavioral health: A scoping review. Implementation Science, 16(1), 1–21. https://doi.org/10.1186/s13012-021-01104-4

56.

Morrison

M. A.

Lent

R. W.

(2018). The working alliance, beliefs about the supervisor, and counseling self-efficacy: Applying the relational efficacy model to counselor supervision. Journal of Counseling Psychology, 65(4), 512. https://doi.org/10.1037/cou0000267

57.

Nakamura

B. J.

Higa-McMillan

C. K.

Okamura

K. H.

Shimabukuro

(2011). Knowledge of and attitudes towards evidence-based practices in community child mental health practitioners. Administration and Policy in Mental Health and Mental Health Services Research, 38(4), 287–300. https://doi.org/10.1007/s10488-011-0351-2

58.

O’Donnell

Dorsey

Gong

Ostermann

Whetten

Cohen

J. A.

Itemba

Manongi

Whetten

(2014). Treating maladaptive grief and posttraumatic stress symptoms in orphaned children in Tanzania: Group-based trauma-focused cognitive-behavioral therapy. Journal of Traumatic Stress, 27(6), 664–671. https://doi.org/10.1002/jts.21970

59.

Park

E. H.

Lee

Y. Y.

Lee

S. M.

(2019). Relationship between the supervisory working alliance and outcomes: A meta-analysis. Journal of Counseling & Development, 97(4), 437–446. https://doi.org/10.1002/jcad.12292

60.

Patton

M. J.

Kivlighan

D. M.

Jr. (1997). Relevance of the supervisory alliance to the counseling alliance and to treatment adherence in counselor training. Journal of Counseling Psychology, 44(1), 108. https://doi.org/10.1037/0022-0167.44.1.108

61.

Piedmont

R. L.

(2014). Inter-item correlations. Encyclopedia of Quality of Life and Well-Being Research, 1, 3303–3304. https://doi.org/10.1007/978-94-007-0753-5_1493

62.

Powell

B. J.

Mandell

D. S.

Hadley

T. R.

Rubin

R. M.

Evans

A. C.

Hurford

M. O.

Beidas

R. S.

(2017). Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study. Implementation Science, 12(1), 1–13. https://doi.org/10.1186/s13012-017-0593-9

63.

Powell

B. J.

Patel

S. V.

Haley

A. D.

Haines

E. R.

Knocke

K. E.

Chandler

Katz

C. C.

Seifert

H. P.

Ake

Amaya-Jackson

(2020). Determinants of implementing evidence-based trauma-focused interventions for children and youth: A systematic review. Administration and Policy in Mental Health and Mental Health Services Research, 47(5), 705–719. https://doi.org/10.1007/s10488-019-01003-3

64.

Proctor

Ramsey

A. T.

Brown

M. T.

Malone

Hooley

McKay

(2019). Training in Implementation Practice Leadership (TRIPLE): Evaluation of a novel practice change strategy in behavioral health organizations. Implementation Science, 14(1), 1–11. https://doi.org/10.1186/s13012-019-0906-2

65.

Proctor

Silmere

Raghavan

Hovmand

Aarons

Bunger

Griffey

Hensley

(2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7

66.

R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

67.

Rohrbach

L. A.

Graham

J. W.

Hansen

W. B.

(1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22(2), 237–260. https://doi.org/10.1006/pmed.1993.1020

68.

Roudbarani

Tablon Modica

Maddox

B. B.

Bohr

Weiss

J. A.

(2022). Clinician factors related to the delivery of psychotherapy for autistic youth and youth with attention-deficit hyperactivity disorder. Autism, 27(2), 415–427. https://doi.org/10.1177/13623613221106400

69.

Salyers

M. P.

Flanagan

M. E.

Firmin

Rollins

A. L.

(2015). Clinicians’ perceptions of how burnout affects their work. Psychiatric Services, 66(2), 204–207. https://doi.org/10.1176/appi.ps.201400138

70.

Samios

Abel

L. M.

Rodzik

A. K.

(2013). The protective role of compassion satisfaction for therapists who work with sexual violence survivors: An application of the broaden-and-build theory of positive emotions. Anxiety, Stress & Coping, 26(6), 610–623. https://doi.org/10.1080/10615806.2013.784278

71.

Schiele

B. E.

Weist

M. D.

Youngstrom

E. A.

Stephan

S. H.

Lever

N. A.

(2014). Counseling self-efficacy, quality of services and knowledge of evidence-based practices in school mental health. The Professional Counselor, 4(5), 467–480. https://doi.org/10.15241/bes.4.5.467

72.

Schoenwald

S. K.

Mehta

T. G.

Frazier

S. L.

Shernoff

E. S.

(2013). Clinical supervision in effectiveness and implementation research. Clinical Psychology: Science and Practice, 20(1), 44. https://doi.org/10.1111/cpsp.12022

73.

Seitz

D. C.

Knaevelsrud

Duran

Waadt

Goldbeck

(2014). Internet-based psychotherapy in young adult survivors of pediatric cancer: Feasibility and participants’ satisfaction. Cyberpsychology, Behavior, and Social Networking, 17(9), 624–629. https://doi.org/10.1089/cyber.2014.0066

74.

Shapiro

C. J.

Prinz

R. J.

Sanders

M. R.

(2012). Facilitators and barriers to implementation of an evidence-based parenting intervention to prevent child maltreatment: The Triple P-Positive Parenting Program. Child Maltreatment, 17(1), 86–95. https://doi.org/10.1177/1077559511424774

75.

Skar

A.-M. S.

Braathu

Peters

Bækkelund

Endsjø

Babaii

Borge

R. H.

Wentzel-Larsen

Ehrhart

M. G.

Sklar

(2022). A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate. BMC Health Services Research, 22(1), 1–15. https://doi.org/10.1186/s12913-022-07539-9

76.

Stamm

(2010). The concise manual for the professional quality of life scale . ProQOL.org.

77.

Stewart

R. W.

Orengo-Aguayo

Young

Wallace

M. M.

Cohen

J. A.

Mannarino

A. P.

de Arellano

M. A.

(2020). Feasibility and effectiveness of a telehealth service delivery model for treating childhood posttraumatic stress: A community-based, open pilot trial of trauma-focused cognitive–behavioral therapy. Journal of Psychotherapy Integration, 30(2), 274. https://doi.org/10.1037/int0000225

78.

Teichman

Berant

Shenkman

Ramot

(2023). Supervisees’ perspectives on the contribution of supervision to psychotherapy outcomes. Counselling and Psychotherapy Research, 23(2), 516–529. https://doi.org/10.1002/capr.12540

79.

Tutty

Ludman

E. J.

Simon

(2005). Feasibility and acceptability of a telephone psychotherapy program for depressed adults treated in primary care. General Hospital Psychiatry, 27(6), 400–410. https://doi.org/10.1016/j.genhosppsych.2005.06.009

80.

Walrath

C. M.

Sheehan

A. K.

Holden

E. W.

Hernandez

Blau

(2006). Evidence-based treatments in the field: A brief report on provider knowledge, implementation, and practice. The Journal of Behavioral Health Services & Research, 33(2), 244–253. https://doi.org/10.1007/s11414-005-9008-9

81.

Weiner

B. J.

Lewis

C. C.

Stanick

Powell

B. J.

Dorsey

C. N.

Clary

A. S.

Boynton

M. H.

Halko

(2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12(1), 1–12. https://doi.org/10.1186/s13012-017-0635-3

82.

Whitaker

R. G.

Sperber

Baumgartner

Thiem

Cragun

Damschroder

Miech

E. J.

Slade

Birken

(2020). Coincidence analysis: A new method for causal inference in implementation science. Implementation Science, 15(1), 1–10. https://doi.org/10.1186/s13012-020-01070-3

83.

Williams

N. J.

(2015a). Assessing mental health clinicians’ intentions to adopt evidence-based treatments: Reliability and validity testing of the evidence-based treatment intentions scale. Implementation Science, 11(1), 1–13. https://doi.org/10.1186/s13012-016-0417-3

84.

Williams

N. J.

(2015b). Mechanisms of change in an organizational culture and climate intervention for increasing clinicians’ evidence-based practice adoption in mental health [PhD dissertation]. University of Tennessee. https://trace.tennessee.edu/utk_graddiss/3482

85.

Williams

N. J.

Becker-Haimes

E. M.

Schriger

S. H.

Beidas

R. S.

(2022). Linking organizational climate for evidence-based practice implementation to observed clinician behavior in patient encounters: A lagged analysis. Implementation Science Communications, 3(1), 1–14. https://doi.org/10.1186/s43058-022-00309-y

86.

Williams

N. J.

Beidas

R. S.

(2019). Annual research review: The state of implementation science in child psychology and psychiatry: A review and suggestions to advance the field. Journal of Child Psychology and Psychiatry, 60(4), 430–450. https://doi.org/10.1111/jcpp.12960

87.

Williams

N. J.

Ehrhart

M. G.

Aarons

G. A.

Marcus

S. C.

Beidas

R. S.

(2018). Linking molar organizational climate and strategic implementation climate to clinicians’ use of evidence-based psychotherapy techniques: Cross-sectional and lagged analyses from a 2-year observational study. Implementation Science, 13(1), 1–13. https://doi.org/10.1186/s13012-018-0781-2

88.

Williams

N. J.

Hugh

M. L.

Cooney

D. J.

Worley

J. A.

Locke

(2022). Testing a theory of implementation leadership and climate across autism evidence-based interventions of varying complexity. Behavior Therapy, 53(5), 900–912. https://doi.org/10.1016/j.beth.2022.03.001

89.

Williams

N. J.

Wolk

C. B.

Becker-Haimes

E. M.

Beidas

R. S.

(2020). Testing a theory of strategic implementation leadership, implementation climate, and clinicians’ use of evidence-based practice: A 5-year panel analysis. Implementation Science, 15(1), 1–15. https://doi.org/10.1186/s13012-020-0970-7

90.

Woods-Jaeger

B. A.

Kava

C. M.

Akiba

C. F.

Lucid

Dorsey

(2017). The art and skill of delivering culturally responsive trauma-focused cognitive behavioral therapy in Tanzania and Kenya. Psychological Trauma: Theory, Research, Practice, and Policy, 9(2), 230–238. https://doi.org/10.1037/tra0000170

91.

Yang

Hayes

J. A.

(2020). Causes and consequences of burnout among mental health professionals: A practice-oriented review of recent empirical literature. Psychotherapy, 57(3), 426. https://doi.org/10.1037/pst0000317

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB