Abstract
The prevention of child maltreatment via parenting interventions requires implementation on a broad scale, which is facilitated by drawing on a multidisciplinary array of service workers located in multiple settings. This underscores the importance of understanding factors that impact worker implementation of evidenced-based parenting and family support interventions. This study involved structured interviews with 174 service providers from several disciplines who had been trained previously in the delivery of the Triple P-Positive Parenting Program. These follow-up interviews, conducted an average of about 2 years after professional in-service training, provided the basis for examining predictors of sustained program use. Predictors examined included facilitators and barriers to program use, as well as organizational and provider-level characteristics such as attitudes toward evidence-based interventions. Highlighting the importance of a systems–contextual perspective on implementation, several provider and organization-level characteristics significantly predicted program use including provider self-confidence after training, fit of program with ongoing duties, availability of posttraining support, and perceived benefit of intervention for children and families. Implications for prevention and implementation science are discussed in view of the challenges inherent in the field of child maltreatment.
Improved parenting is a cornerstone of child maltreatment prevention (Barth, 2009). Thus, strengthening parenting and family interactions and improving child behavior across entire populations is a preventive approach that should ultimately reduce the prevalence rates of child maltreatment. One primary barrier to this approach is reaching large numbers of parents with effective, evidence-based programs for improving parent–child interactions. Such evidence-based parenting approaches are available; however, the reach of these services is quite poor and only a minority of parents avail themselves of parenting support services (Prinz & Sanders, 2007). Improving the accessibility of parenting support requires a public health approach involving media and communication strategies to support positive parenting, along with a range of services of increasing intensity for families that need and want more support (Sanders, 2008). Provision of broadly accessible parenting support requires training a wide range of service providers in multiple settings in evidence-based parenting approaches (Sanders, Cann, & Markie-Dadds, 2003). Training a large, multidisciplinary workforce of existing service providers supports program delivery at a population level, increasing the likelihood that parents will have the opportunity to benefit from evidence-based parenting approaches. Importantly, prior research has demonstrated population-level impact on child mental health and parenting outcomes from systematic, coordinated implementation of parenting support across agencies and service sectors (Sanders et al., 2008). To provide the broadest reach, training providers who work in settings that have access to the largest number of families, such as child care, education, or primary care, would be necessary. However, these providers are not routinely trained or supported in implementation of evidence-based parenting programs. The reach of such programs would be greatly expanded by training such a large, existing workforce (Shapiro, Prinz, & Sanders, 2010).
While training a workforce of existing service providers to implement evidence-based parenting approaches is a necessary step, it is not sufficient to produce population-level changes in parenting or child outcomes. Service providers must then actually implement the intervention for desired impact. A wide range of variables impacting implementation have been identified (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005; Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). These include aspects of the intervention, as well as provider, organizational, and contextual factors, acting alone and in combination, that serve as facilitators or barriers to implementation.
Implementation of evidence-based practices (EBPs), or the process of technology transfer, has been conceptualized as occurring in stages, involving exploration, adoption, program installation, implementation, full operation, and maintenance or sustainability (Fixsen et al., 2005; Simpson, 2002). Research examining the influences on these stages has highlighted the importance of organizational variables such as motivation, resources, climate, and openness to change; provider characteristics such as skills and attitudes toward EBPs; features of the program itself such as relative advantage over other programs, as well as aspects of the posttraining environment including implementation support and dedicated time or resources (Greenhalgh et al., 2004; Sanders & Murphy-Brennan, 2010). Given the wide variety of variables that can impact implementation, a conceptual framework that captures this complexity is required. The systems–contextual approach represents such a framework that captures the variety of provider, client, and organizational variables that must be considered when examining the process of implementation and dissemination (Beidas & Kendall, 2010). This approach takes into account contextual variables that can influence the relationship between provider training and subsequent provider behavior (e.g., program use). Contextual variables including those operating at the level of systems, organizations, and providers create a complex set of facilitators and barriers that interact to impact program use by providers. This is especially true within child welfare settings, in which significant barriers to the use of EBPs have been identified (Horwitz, Chamberlain, Landsverk, & Mullican, 2010). Potential barriers include lack of fit within the child welfare system and concerns about acceptability by client populations. However, research is beginning to emerge about implementation of evidence-based interventions within child welfare systems. For example, Chagnon, Pouliot, Malo, Gervais, and Pigeon (2010) examined research knowledge utilization among a large sample of administrators and practitioners within child welfare. Among these groups, relatively few (less than 20%) reported use of research-based knowledge in practice. Organization culture, relationships with researchers, and methods of knowledge dissemination were identified as particularly important. In a more direct assessment of use of an evidence-based parenting intervention within a child welfare context, Petra and Kohl (2010) conducted a pilot study of an evidence-based parenting intervention, Pathways Triple P, in collaboration with child welfare partners in one Midwestern city. Administrators, case managers, and parents found the intervention to be both useful and acceptable (Petra & Kohl, 2010).
Examination of factors that impact implementation must be accompanied by efforts to define and measure program use. Program use can be measured directly (through observation of providers delivering the program) or indirectly (through provider or client reports or case records). Direct measurement provides the most accurate data but is labor-intensive and time-consuming; indirect measurement methods are feasible and practical but may yield less accurate information. These issues become particularly salient in primary prevention of maltreatment, which requires broad reach of evidence-based parenting interventions across multiple service settings and a large number of providers (Sanders, Cann, & Markie-Dadds, 2003). Thus, reliance on provider self-reports of program use remains the most practical option in examining large-scale dissemination efforts that cross service systems and organizations.
At present, little is known about implementation of evidence-based parenting interventions among service providers from multiple disciplines. One evidence-based parenting intervention that is designed specifically as a public health approach to supporting parenting that is designed for use by a wide variety of providers from multiple disciplines is the Triple P-Positive Parenting Program (Triple P). Triple P is a multilevel intervention designed to improve parenting confidence and competence by introducing a range of parenting strategies using a self-regulatory framework (Sanders, 1999). Triple P interventions have been evaluated in a wide a range of service delivery settings (e.g., home, primary care, and school settings; Markie-Dadds & Sanders, 2006a, 2006b; Turner & Sanders, 2006) with a wide variety of populations (e.g., preschoolers, adolescents, and children with conduct problems, attention deficit hyperactivity disorder, or developmental disabilities; Bor, Sanders, & Markie-Dadds, 2002; Hoath & Sanders, 2002; Morawska & Sanders, 2006; Roberts, Mazzucchelli, Studman, & Sanders, 2006; Sanders, Markie-Dadds, Tully, & Bor, 2000). These evaluations have encompassed both brief and more extensive forms of program delivery. Primary Care Triple P, typically delivered in 2–4 sessions, has resulted in positive outcomes including improvements in child behavior problems, dysfunctional parenting styles, parent stress, and partner support (Crisante, 2003; Turner & Sanders, 2006). Common outcomes from more intensive Triple P interventions (e.g., 8–10 sessions) include reductions in parent-reported child behavior problems, reductions in aversive parenting practices, and improvements in parental self-efficacy (Bor et al., 2002; Hoath & Sanders, 2002; Sanders et al., 2000; Sanders, Cann, & Markie-Dadds, 2003). Meta-analyses have documented the positive effects of Triple P (e.g., deGraaf, Speetjens, Smit, de Wolff, & Tavecchio, 2008). Triple P has also been used to effectively train a broad range of providers to deliver parenting interventions (Sanders, Markie-Dadds, & Turner, 2003; Shapiro et al., 2008).
In order to examine factors related to implementation of EBPs, the current descriptive study examines use of an evidence-based parenting intervention in real-world settings by a large, diverse group of providers. Our goal is to examine the relationship of specific provider attributes including training history, perceived barriers and facilitators of program use, and attitudes toward use of EBPs, with self-reported implementation of an evidence-based parenting intervention (Triple P).This study also represents a partial replication of an earlier study designed to examine posttraining program use and perceived facilitators and barriers to that use with a different group of providers (Sanders, Prinz, & Shapiro, 2009).
Method
Participants
Service providers in this study were trained in Triple P as part of the U.S. Triple P System Population Trial, a study focusing on use of the Triple P system of interventions as a child maltreatment prevention strategy in one state in the Southeast region of the United States (South Carolina). The current study drew from a pool of 343 service providers from a variety of disciplines, who underwent Triple P training over a 3-year period in the latter part of the larger population trial. Service providers were working with parents in a variety of settings (primarily public) including education (schools), mental health (community mental health centers), child care, social services (county social services departments), and not-for-profit organizations (organizations providing home visitation or parent education services), were invited to participate in a telephone survey. See Table 1 for a summary of background characteristics of these providers. Of the 343 participants, 51 (14.8%) were not eligible to participate in the survey because they were no longer working in a setting in which they were able to use parenting or family interventions, or supervise others in these activities. Of the 292 remaining possibly eligible participants, 174 (59.5%) completed the survey; 31 (10.6%) indicated initial interest but could not be reached to complete the survey, and 21 (7.2%) declined to be interviewed. A total of 66 (22%) of the potential participants could not be located or were not interviewed due to retirement, job elimination (service position or work unit closed), relocation, or death (1 person).
Provider Characteristics
Most of the 174 providers who completed the survey (94%) were female. Disciplinary backgrounds included counseling professionals (e.g., counselors, social workers, psychologists; 34.5%), parent educators (22.4%), education professionals (21.3%), child care professionals (9.8%), administrators (7.5%), and other professionals (e.g., nurses, interns, law enforcement; 4.6%).
Procedures
As part of the U.S. Triple P System Population Trial, Triple P training was implemented in nine counties that were randomly assigned to receive the Triple P System of Interventions; nine other counties served as comparisons. The rationale, goals, and outcomes of this randomized controlled population trial have been previously described (Prinz, Sanders, Shapiro, Whitaker, & Lutzker, 2009). Briefly, the initial outcome data from the population trial indicated evidence of positive impact on three population-level indicators related to child maltreatment (substantiated child maltreatment cases, out-of-home placements, and child-maltreatment related injuries requiring hospital treatment; Prinz et al., 2009).
Training of service providers occurred in the context of implementing the full Triple P system, which includes five levels of intervention that together form a public health approach to parenting and family support (Sanders, 2008). The core Triple P system includes five levels of intervention of increasing intensity. Level 1 is a coordinated set of universal media and communication strategies designed to provide broad access to parenting information and support and to destigmatize the process of seeking parenting support. Level 2 involves parenting seminars or brief, one-session consultation for individual families on specific parenting challenges. Level 3 interventions are designed as brief, 3–4 session family-based interventions targeting a range of common parenting situations. More intensive parenting/family interventions delivered in a group or individual family format are available for families that have children evidencing more significant behavioral challenges (Level 4). Level 5 involves additional programming to boost the intensiveness of Level 4 with families that require it. Only Levels 2 through 5 of Triple P are germane to this report because Level 1 did not involve training of service providers.
Service providers working in the intervention counties who learned of and were interested in receiving Triple P training registered for training courses in the county in which they worked. Service providers chose the level of intervention and type of training to attend in consultation with a training consultant (Shapiro et al., 2010). The vast majority of service providers were trained at the lower level of intervention (i.e., Primary Care Triple P), as this level is the most flexible and can thus be accessed by the widest range of providers.
All service providers who completed Triple P training during a 3-year span and who at the time were employed in a setting providing services to parents or families or who were supervising staff working with parents or families were eligible to participate in the 20-min structured telephone interview. Service providers were told that the survey was for individuals who had participated in Triple P professional training courses in the last 3–4 years, and the survey focused on their work with children and families. Providers were informed that they would be asked about their opinions regarding issues related to services for children. Service providers were interviewed an average of 1.89 years after completion of training (range 0.3–3.7 years). Four female project staff (three with master’s degrees, one with a doctorate) were trained to administer the interviews by the first author and conducted the interviews.
Measures
Descriptive background information
Information on provider characteristics was collected, including gender, profession, work setting, level of education, and level and type of Triple P training completed. Eleven professional job categories were classified into six profession types; 19 work settings were classified into 7 types of settings. Background information also included a provider rating of self-efficacy in conducting parent consultations about child behavior completed at the end of Triple P training. Providers responded on a 7-point scale ranging from 1 (no, definitely not) to 7 (yes, definitely) to the question: “How confident are you in conducting parent consultations about child behavior?”
Structured Telephone Interview
Attitudes toward EBPs
Provider attitudes toward EBP were assessed using the Evidence-Based Practice Attitude Scale (EBPAS; Aarons, 2004; Aarons, McDonald, Sheehan, & Walrath-Greene, 2007). The EBPAS is a 15-item measure developed to assess mental health provider attitudes toward EBPs. The items are organized into four subscales reflecting attitudes toward adoption of EBPs: Appeal (appeal of EBPs), Openness (open to new practices), Requirements (likelihood of use given requirements to do so), and Divergence (degree to which a provider’s current practice diverges from research-based interventions; Aarons et al., 2007). Aarons and colleagues report adequate internal consistency for the measure as a whole and for each of the four subscales (Cronbach’s α ranges from .66 to .93 for the subscales; α = .794 for the EBPAS total score; Aarons et al., 2007, p. 469).
Self-efficacy in parent consultation
Self-efficacy in conducting parent consultations about child behavior was assessed by a single item on the provider survey: “How confident are you in conducting parent consultations about child behavior?” Providers responded on a 7-point scale ranging from 1 (no, definitely not) to 7 (yes, definitely).
Program use
Program use was measured by a series of questions during the structured telephone interview. Questions were asked about whether or not providers were using Triple P with families through their work (yes/no) or outside of their work (yes/no). Providers were also asked about how many families in the last 12 months they had used Triple P with (amount of use). Fidelity of use was not captured by these questions and was not assessed as part of the larger Triple P System Trial in which this study was embedded.
Facilitators and barriers to program use
A total of 15 facilitators and 18 barriers to program use were assessed during the structured telephone interview. Facilitators included items such as “Your organization welcomes change and innovation;” “Staff in your workplace provide support for Triple P.” Barriers included items such as “An obstacle to using Triple P is Triple P not being integrated with caseload or other responsiblities at work;” “An obstacle to using Triple P is difficulty applying Triple P to the needs of the child or family.” Providers were read a series of statements about potential facilitators and barriers or obstacles to their use of Triple P. Providers indicated their level of agreement or disagreement with each statement by responding on a 4-point scale (strongly agree, agree, disagree, or strongly disagree).
Results
Provider Characteristics
The characteristics of the providers who completed the survey are summarized in Table 1. Service providers were primarily female (93.7%) and relatively well educated, with the majority having received a college or graduate/professional degree (82.18%). Just over one third (34.5%) were counseling professionals, 22.4% were parent educators, and 21.3% classified themselves as educators. More than half were located in schools or other educational settings (62.6%); the remainder were located in private practice, not-for-profit organizations supporting school readiness, mental health or substance abuse treatment settings, or child care settings. A smaller number were located in social service or health settings (4.0% and 2.3%, respectively). The majority of providers had undergone Triple P Training at the Level 2/3 or Primary Care level (n = 134; 77.0%). A smaller number had undertaken Level 4 Standard or Group Triple P. Of the 174 providers completing the survey, 148 (85.1%) reported use of Triple P with families while 26 (14.9%) reported no use of Triple P.
Chi-square analyses were conducted to determine if the subsample of providers who completed the survey differed from the larger sample of providers eligible for the survey. These analyses revealed no significant differences between survey completers and noncompleters in provider gender (χ2 = .247, df = 1, p = .619), education level (χ2 = 3.872, df = 3, p = .276), work setting (χ2 = 5.961, df = 6, p = .428), or in the type of Triple P training that they had undertaken (χ2 = .1.739, df = 1, p = .187). Survey completers did differ significantly from noncompleters in type of profession (χ2 = 13.897, df = 5, p = .016); however, this significant finding is accounted for by a small group of providers within the “other” category who were volunteers at the time of training and who were less likely to have completed the survey. No significant differences were apparent for the remaining groups of professionals.
Provider Characteristics and Program Use
Logistic regression analyses were undertaken to ascertain the extent to which provider background characteristics were associated with their use of Triple P. Logistic regression analyses results indicated no relationship between background characteristics and use of Triple P. Use of Triple P was not significantly related to provider education (Wald statistic = 3.743, p = .809), profession (Wald statistic = 3.495, p = .982), or work setting (Wald statistic = 2.291, p = 1.00). Use of Triple P was not significantly related to level of Triple P provider training (Primary Care B = −.716, odds ratio [OR] = .489, Wald statistic = 961, p = .327; Level 4 Group B = −1.010, OR = .364, Wald statistic = 2.471, p = .116; Level 4 Standard B = −.078, OR = .925, Wald statistic = .012, p = .914).
Linear regression analyses examined the extent to which provider characteristics predicted the amount of Triple P used. Amount of use was defined as the number of families that providers reported having used Triple P within the past 12 months. Because the distribution for amount of use was significantly skewed, a square root transformation was applied to this variable, which significantly reduced the skew. Linear regression analyses using the transformed dependent variable and independent variables of gender, education, profession, work setting, and level of Triple P training revealed that amount of Triple P use with families was not significantly associated with gender, education, profession, work setting, and level of Triple P training, F(7, 125) = .861, p = .539.
Provider Attitudes Toward EBP and Program Use
Provider attitudes were assessed using the EBPAS. In order to assess the reliability of the EBPAS in the current sample, Cronbach’s αs of the EBPAS subscales and EBPAS total scales were calculated. Cronbach’s α = .920 for the requirements subscale, .815 for the appeal subscale, .725 for the openness subscale, .386 for the divergence subscale, and .751 for the total EBPAS. These internal consistency reliability estimates are quite similar to those found by Aarons (2004). The Divergence subscale, in particular, appears to have weaker internal consistency than the other subscales. Factor analyses were also conducted with the EBPAS in the current sample; the factor structure for the requirements, appeal, and openness subscales were identical to that found by Aarons. However, the divergence subscale did not emerge as a unitary subscale in this sample. Given the low internal consistency for this subscale, caution is recommended regarding use of the Divergence subscale.
Logistic regression analyses were conducted to explore the association of provider attitudes toward evidence-based interventions and use of Triple P. The EBPAS subscale of requirements (the likelihood of adopting and EBP given requirements to do so) significantly predicted program use (B = .850, OR = 2.341, Wald statistic = 4.081, p = .043), while the remaining subscales of openness (B = −.602, OR = .548, Wald statistic = 2.683, p = .101), divergence, (B = −.174, OR = .841, Wald statistic = .189, p = .664), and appeal (B = −.238, OR = .788, Wald statistic = .214, p = .644), did not. The total EPBAS score was not significantly predictive of use of Triple P (B = −.108, OR = .897, Wald statistic = .038, p = .846).
Linear regression analyses were conducted to examine the impact of provider attitudes toward EBP, as assessed using the EPBAS, and amount of program use. Provider attitudes as assessed by the EBPAS did not significantly predict the amount of use of Triple P with families in the last year, F(5, 141) = .599, p = .701.
Self-Efficacy in Conducting Parent Consultations About Child Behavior
Provider self-efficacy or confidence in conducting parent consultations about child behavior as rated at the completion of training (M = 5.95, SD = .943) was significantly predictive of program use (B = −.534, OR = .586, Wald statistic = 6.034, p = .014). However, confidence rated at the time of the survey (M = 5.71, SD = .994) was not significantly predictive of program use (B = −.309, OR = .734, Wald statistic = 2.112, p = .146). Further examination of the relationship between these two measures revealed a moderate and significant correlation (Spearman’s ρ = .308; Pearson chi-square = 42.370, p = .002). Mean confidence scores did decrease significantly over time, t = 2.634, df = 169, p = .009.
Linear regression analyses were conducted to examine the impact of provider self-efficacy and amount of program use. Provider self-efficacy did not significantly predict the amount of use of Triple P in the last year, F(2, 140) = 2.76, p = .067.
Provider Perceptions of Facilitators and Barriers to Program Use
Logistic regression analyses were undertaken to examine the association of facilitators on use of Triple P. Three of 15 facilitators to use of Triple P significantly predicted program use. These include supervision and case consultation (B = 1.202, OR = 3.338, Wald statistic = 3.692, p = .05); Triple P producing change in children and families (B = 1.949, OR = 7.022, Wald statistic = 4.558, p = .03) and provider knowledge and skills in behavioral family intervention (B = 2.467, OR = 11.791, Wald statistic = 6.394, p = .01). See Table 2 for data on provider perceptions of facilitators to program use.
Provider Perception of Facilitators to Program Use
p < .05.
One of 18 perceived barriers, lack of integration of Triple P with a provider’s caseload or other responsibilities, was significantly related to program use (B = −.1099, OR = .333, Wald statistic = 5.970, p =.015). Table 3 provides information on provider’s perceived barriers to use.
Provider Perception of Barriers to Program Use
*p < .05.
A linear regression analysis was conducted to examine the potential contribution of facilitators and barriers on the number of families providers reported having used Triple P within the last 12 months. Facilitators and barriers did not significantly predict the amount of use of Triple P, F(33, 102) = 1.39, p = .111.
To test the stability of the variables that were significant in the exploratory logistic regression analyses, an additional set of regression analyses was conducted. The significant variables included the EBPAS subscale of requirements, three facilitators (supervision/consultation on cases, Triple P producing change in children and families, and provider knowledge and skills in behavioral family intervention [BFI]), one barrier (Triple P not being integrated with caseload or other responsibilities), and provider self-efficacy in parent consultations about child behavior at the conclusion of Triple P training. These variables collectively predicted use of Triple P (χ2 = 19.724, df = 6, p = .003).
A linear regression analysis was conducted to examine the impact of these variables on the amount of Triple P used in the last 12 months. The results of the linear regression were significant, F(6, 132) = 2.664, p = .018; R 2 = .108. Taken together, provider self-efficacy and BFI skills, likelihood of program use given requirements to do so, supervision, seeing change in families, and (negatively) lack of integration of Triple P with caseload or other responsibilities significantly predicted the amount of Triple P use with families.
Discussion
The ability to engage and train a large, multidisciplinary workforce in the use of EBPs is a necessary but not sufficient condition to produce population-wide reduction of child-maltreatment prevalence rates. It is further necessary for the workforce, once trained, to sustain use of the EBP with families. One of the most practical and feasible methods for assessing program use with a large, multidisciplinary sample of providers working in multiple service sectors is through provider self-report. Provider self-reports are an important source of information in implementation research, as factors operating at the provider level, as well as at the organization and systems level, have an impact on program use. The results of this study highlight the importance of both individual-provider and organizational–contextual factors in implementation of EBPs. This study also extends the literature by documenting factors related to ongoing use of an EBP nearly 2 years after training in a large, multidisciplinary work force. Importantly, 85% of the surveyed providers reported that they were using Triple P with families 2 years after training on average. This finding attests to the robustness of Triple P interventions and the potential for providers in real-world settings to incorporate EBPs into their work with families.
The robustness of Triple P interventions is further underscored by the result that the use of Triple P was not significantly related to gender, education, profession, or work setting of providers. This highlights the accessibility of Triple P for a diverse workforce employed across a range of work settings, including social services agencies. Accessibility of an evidence-based parenting program by a wide range of providers in a variety of real-world work settings is a necessary step in population-level implementation of EBPs. Use of Triple P was not significantly related to the level of Triple P training that providers had undergone; this finding is not surprising given that providers actively selected the level of Triple P training that was most feasible given their methods of service delivery (Shapiro et al., 2010).
Underscoring the value of adopting a systems–contextual approach (Beidas & Kendall, 2010) to training of service providers and consistent with implementation research highlighting the impact of a range of variables operating at multiple levels on program use, in this study the variables that significantly impacted provider reports of program use occurred at both the provider and the organizational levels.
Organizational Characteristics
Implementation of Triple P occurs in a context of organizational supports and structures (Sanders & Murphy-Brennan, 2010). The balance of facilitators and barriers in the posttraining environment may, in fact, be one of the most important aspects that determine the success of implementation efforts. In this study, the ability to discuss cases and receive consultation or supervision significantly predicted program use. Such support is necessary early in implementation to integrate new skills and programs into existing services (Fixsen et al., 2005). The one organizational barrier identified as a significant (negative) predictor of program use involved Triple P not being integrated with caseload or other responsibilities at work. These findings underscore the importance of a priori examination of elements necessary to support program delivery as well as fit of the program within the organization and with provider work duties before a decision is made to adopt a program. For example, if delivery of an evidence-based intervention requires time to prepare for sessions, time for supervision, or additional measures to monitor client progress, organizations will need to consider whether or not providers have the ability to accommodate these activities into their current schedule.
Provider Characteristics
Provider characteristics such as needs, motivation, attitudes, and skills can impact program implementation (Greenhalgh et al., 2004). In this study, confidence in conducting parent consultations about child behavior assessed at training completion was significantly related to program use. This finding is consistent with prior research in identifying self-efficacy in program delivery as an important predictor of training completion and program use (Turner, Nicholson, & Sanders, 2011). However, self-efficacy in program delivery as measured at the time of the survey was not related to program use. While the lack of a significant relationship between program use and self-efficacy at the time of the survey may initially be puzzling, further inspection of the data revealed that provider self-efficacy remained relatively high but had decreased slightly by the time of the survey. It is possible that self-confidence in program delivery may fade over time if the rate of program use is relatively low, or as the length of time since training increases. This hypothesis is a plausible explanation, given that the survey was conducted an average of 2 years posttraining and that a large number of providers used Triple P with a relatively small number of families (which may impact self-efficacy in program delivery).
In addition to provider confidence in conducting parent consultations about child behavior, provider’s perception of their knowledge of behavioral family intervention skills was significantly related to program use. Skill acquisition in BFIs presumably occurred as a function of Triple P training as few service providers receive preservice or inservice training on evidence-based parent consultation or in behavioral approaches more generally. Knowledge of BFI skills may also influence provider confidence; in this study both constructs emerged as significantly related to implementation of Triple P with families.
Provider perceptions of the reasons for change in family functioning appeared in this study as an important indicator of program use. Triple P was seen as producing change in families, which served to facilitate program implementation. Feedback from client progress thus represents a potential positive reinforcement mechanism for providers who implement EBPs in real-world settings. As noted by Greenhalgh, Robert, Macfarlane, Bate, and Kyriakidou (2004), providers are not passive recipients of innovations (p. 598). Use of EBPs is associated with activities to evaluate and make meaning of these experiences; if providers perceive themselves and the interventions they use as successful, they are more likely to continue to use the intervention in the future.
Service provider attitudes toward adoption of EBPs may also impact implementation. This study adds to the growing literature on the impact of provider attitudes on use of EBPs as assessed using the EBPAS.
In this sample, the EBPAS total score and three of the four subscales (requirements, appeal, and openness) demonstrated good reliability. The divergence subscale internal consistency was low (as was noted by Aarons, 2004). The requirements subscale did significantly predict program use; providers were more likely to use Triple P with families if they believed that use of EBPs was required by their supervisor, agency, or state. This finding supports the importance of management, organization, and systems-level buy-in prior to implementation of EBPs and highlights the need for clear communication about expectations for use. How such expectations are conveyed, and the activities that are engaged in at a systems–contextual level to support use of EBPs, become important considerations to support implementation. Other provider attitudes as assessed by the EBPAS related to use of EBPs were not significant predictors of program use or amount of use.
The lack of impact on the variables examined on the amount of program use may be explained by the finding that most providers used the intervention with a limited number of families (58% of providers used the program with 10 or fewer families in the prior 12 months). However, what is not captured are the approximately 40% of providers who reported program use with much larger groups of families (from 20 to 80 families in the last year). A minority of providers (n = 4) reported extremely high use (i.e., more than 100 families in the prior 12 months). However, given the small number of these extraordinarily high users (n = 4), further analyses were not possible; however, such a high degree of use by a few providers has the potential to positively impact program reach at a population level. While the most desirable outcome would be moderate-to-high use by all trained providers, understanding the factors that influence unusually high rates of use are important. Follow-up qualitative surveys with these providers are planned.
While examination of variables related to program implementation by a real-world sample of providers from multiple settings is a strength of this study, several study limitations are worthy of mention. One primary limitation of this study is reliance on provider self-report of program implementation. In the context of a population trial, collection of data from individual client records across multiple agencies or observational methods in multiple settings was simply not feasible. A second limitation is the high rate of use among those providers completing the survey; this may have constrained the potential predictions achievable with logistic regression. A third limitation is the exploratory and descriptive nature of this study; the number of analyses conducted and the lack of a comparison group limits the strength of inferences that can be made. The current findings should be considered preliminary and caution is recommended in interpretation of the variables that emerged as significant predictors, given the number of variables examined. A fourth limitation is the survey was reliance on provider self-report, which captured variables primarily at the level of the provider. Measuring program use through provider self-report is feasible and practical in large-scale studies but does not provide objective data about the quality of implementation or fidelity to the intervention model. Current theoretical models focus on the multiple steps in the process of technology transfer that are impacted by provider, organizational, and systems/contextual-level variables (Simpson, 2002). Future research efforts should include additional measures of organizational and contextual-level variables, as well as measures of fidelity of implementation.
Implications for Prevention of Child Maltreatment
These findings underscore the importance of taking a systems–contextual approach to training, implementation, and dissemination of evidence-based parenting interventions on a broad scale. Significant attention to program fit with current methods of service delivery and provider caseloads is important to address prior to training. Once a decision is made to adopt an evidence-based parenting program, high-quality training that results in increase in provider skills and self-confidence in program delivery is necessary to increase the likelihood of program implementation. This study also underscores the critical importance of the posttraining environment, such as supervision and case consultation, which appear to influence program use. Further research is needed to examine more specifically which models of supervision are most likely to be useful (i.e., a peer-supported self-regulatory approach as compared to an expert supervision model). Changing parenting on a broad scale through use of evidence-based parenting programs is possible; however, we must move well beyond a “train and hope” model in order for the investment in training to pay off.
Footnotes
The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This research was supported by grants U17/CCU422317 and R18CE001340 to Prinz and Sanders from the National Center for Injury Prevention Control, Centers for Disease Control and Prevention.