The Dating Matters ® Toolkit: Approaches to increase adoption,implementation,and maintenance of a comprehensive violence prevention model

Abstract

Purpose:

This practical implementation report describes a comprehensive teen dating violence prevention model—Dating Matters^®—and the approach taken to maximize its potential for widespread dissemination through development of the Dating Matters Toolkit. Dating Matters has evidence of effectiveness for preventing teen dating violence and other adolescent risk behaviors from a multi-site randomized controlled trial. Identifying strategies that reduce barriers to the dissemination and implementation of evidence-based comprehensive prevention models, like Dating Matters, is critical to their widespread adoption. Lessons learned from creating the Toolkit can inform the development and dissemination of similar comprehensive prevention strategies and speed their adoption and use in the field.

Approach:

We engaged in a multi-pronged, data-driven approach to maximize adoption, implementation, and maintenance of the Dating Matters model based on multiple data sources and lessons learned from the demonstration project, while retaining core components and content of the evaluated model. Potential barriers to the national dissemination of Dating Matters were identified in four key areas: training, technical assistance, model flexibility, and accessibility. A series of modifications were made to the implementation model to address these challenges and facilitate scale-up prior to national dissemination. We outline these challenges and describe solutions implemented through the development of the Dating Matters Toolkit.

Outcomes:

The Dating Matters Toolkit includes web-based facilitator training, enhanced implementation support through a new staff role and community of practice, online comprehensive implementation guidance and resources, and added flexibility to improve feasibility and adoption in communities during capacity-building. Findings from an initial evaluation of the Toolkit suggest users perceived the comprehensive model as feasible and accessible with enough implementation support. Most expressed interest in adopting the model. Users described cost and stakeholder buy-in as remaining barriers. Ongoing improvements to the Toolkit to address these challenges and future research to evaluate its effectiveness are planned.

Plain Language Summary:

Research suggests that comprehensive, multi-component prevention models may be more effective than single programs. However, they may also have more barriers to adoption, implementation, and maintenance. This article describes development of the Dating Matters^® comprehensive teen dating violence prevention model, and the approach the Centers for Disease Control and Prevention (CDC) took to make it easier for communities to adopt the model, implement it with fidelity, and maintain their prevention efforts. Based on lessons learned from a multi-site demonstration project, CDC identified four potential barriers to implementing comprehensive prevention: training, technical assistance, model flexibility, and accessibility. To address these challenges, CDC created the Dating Matters Toolkit, an implementation guidance package that incorporates new tools, resources, and modifications to the original model to encourage uptake. The Toolkit includes web-based training, multiple implementation supports for program facilitators, access to free program materials, online implementation guidance, and increased model flexibility to improve feasibility and adoption in more communities. An initial evaluation suggests users generally view the Toolkit favorably in terms of adoption and implementation; however, they described cost and stakeholder buy-in as remaining challenges. Lessons learned from this project may help program developers, implementers, and communities identify promising approaches to improve uptake of comprehensive prevention efforts. Addressing the factors that help or hinder a community’s ability to carry out these efforts is a critical step toward increasing use of coordinated, multiple component prevention approaches.

Keywords

Dating Matters dating violence comprehensive prevention implementation dissemination

The field of violence prevention continues to evolve and expand its impact by developing new prevention approaches based in research and practice evidence to address risk for violence and prevent it before it begins (Basile et al., 2016; Fortson et al., 2016; Niolon et al., 2017). Although many prevention programs have achieved positive outcomes, they often demonstrate small or time-limited effects (Community Preventive Services Task Force, 2018; DeGue et al., 2014; Whitaker et al., 2006). As such, the field of violence prevention increasingly recognizes the importance of building comprehensive prevention strategies that include multiple, coordinated prevention approaches across the social ecology to strengthen outcomes beyond the effects of single-component interventions (Basile et al., 2016; DeGue et al., 2016; Frieden, 2010; Niolon et al., 2017). Strategies that address risk and protective factors across relationship types and contexts by pairing evidence-based prevention programming with, for example, social and environmental change approaches that reinforce positive norms and behaviors may be more effective in increasing healthy behaviors and reducing rates of violence—the ultimate goal of the public health approach to violence prevention (DeGue et al., 2016; Rothman et al., 2015).

Research from the field of youth violence prevention suggests that comprehensive prevention strategies can be more effective than single-component approaches (David-Ferdon & Simon, 2014 ; Matjasko et al., 2012). Yet, these multi-component, integrated prevention models may be more costly and challenging to implement (Elliott & Mihalic, 2004; O’Connell et al., 2009; Rothman et al., 2015). Thus, identifying strategies that reduce barriers to the dissemination and implementation of comprehensive prevention models is critical to their widespread adoption. To advance these efforts, this practical implementation report describes one comprehensive prevention model—Dating Matters®: Strategies to Promote Healthy Teen Relationships (Dating Matters)¹—and the approach taken by the Centers for Disease Control and Prevention (CDC) to maximize its potential for adoption, implementation, and maintenance for national dissemination while reducing costs to communities through development of the Dating Matters Toolkit. Lessons learned from this process can inform efforts to develop and disseminate similar comprehensive prevention strategies to speed their adoption and use in the field.

The Dating Matters comprehensive teen dating violence prevention model

CDC-developed Dating Matters, starting in 2009, to address growing concerns from the public and policymakers about the risks of teen dating violence, and the need for evidence-based prevention strategies that advanced the field beyond the individual-level, school-based programs that were standard practice at the time (Teten Tharp et al., 2011). Dating Matters built on existing evidence to create multiple prevention components that address risk and protective factors for youth as well as their parents, schools, and neighborhoods. The model sought to reach adolescents at younger ages—11 to 14 years old—in middle school to build healthy relationship skills early and prevent violence before it begins. Furthermore, it was designed to leverage the infrastructure and capacity of local health departments to bring together multiple sectors (e.g., education, government, business, health, nonprofit) and stakeholders (e.g., educators, youth, parents, public health, youth-serving and violence prevention organizations) for implementation.

Dating Matters includes several core components: school-delivered programs for youth in sixth, seventh, and eighth grades; community-based programs for parents of sixth, seventh, and eighth graders; online training for school staff on their role in preventing dating violence; a youth communications program implemented by high school-age “brand ambassadors”; and activities that build prevention capacity, inform local policy, and track relevant data at the community level (see Figure 1). Each of the multi-session youth and parent programs provide interactive opportunities to build healthy relationship skills (e.g., communication, making healthy choices, emotion regulation, help-seeking and victim support, positive parenting) and develop positive norms and behaviors. Because the model was developed to build on existing evidence-based strategies, eighth-grade youth and their parents receive evidence-based teen dating violence prevention programs that pre-dated the development of Dating Matters (i.e., Safe Dates; Families for Safe Dates; Foshee et al., 2004, 2012), and parents of sixth graders receive an evidence-based sexual health promotion program adapted for Dating Matters to also address dating violence (i.e., Parents Matter!; Dittus et al., 2004). CDC developed the remaining youth and parent programs and other intervention components specifically for Dating Matters.

Figure 1.

The Dating Matters® comprehensive teen dating violence prevention model.

CDC implemented and rigorously evaluated Dating Matters in a multi-site demonstration project from 2011 to 2016. Forty-six middle schools across four sites were randomly assigned to implement either the Dating Matters comprehensive model or a standard-of-care youth program (Safe Dates) in eighth-grade only. Findings indicated that the Dating Matters comprehensive model was more effective in preventing teen dating violence perpetration and victimization as well as a range of other risk outcomes during middle school than an evidence-based, single-program approach (DeGue et al., 2020; Estefan et al., 2020; Niolon et al., 2019; Vivolo-Kantor et al., 2019). A long-term follow-up of these youth through high school (in progress; see Niolon et al., 2016 for a discussion) will provide more information on whether these effects are sustained over time, and replication of these findings overall is needed. However, current evidence suggests that Dating Matters is an effective violence prevention strategy.

Key factors in bringing effective prevention to scale

Disseminating evidence-based strategies for widespread adoption is the final, and perhaps most critical, phase of the public health approach² to violence prevention (Mercy et al., 1993). Yet there are multiple challenges involved in bringing evidence-based prevention strategies to scale in “real-world” settings where resources, capacity, and other key supports may be more limited than in controlled and typically well-funded research settings (Bumbarger et al., 2010). This may be particularly true of comprehensive strategies that require coordination of several activities, potentially involving multiple organizations, funding sources, populations, partnerships, and stakeholders.

Several frameworks exist to guide the design, translation, dissemination, and implementation of evidence-based interventions (Bartholomew et al., 2011; Glasgow et al., 1999; Green & Kreuter, 2005; Klesges et al., 2005; Proctor et al., 2011). One such framework, RE-AIM, identifies five dimensions for evaluating implementation efforts: reach (i.e., extent of individual-level participation), efficacy (i.e., intended health outcomes), adoption (i.e., extent of uptake in relevant settings and organizations), implementation (i.e., intervention delivered as intended or with fidelity), and maintenance (i.e., sustainability of intervention effects, implementation, and “institutionalization” within the organization or community) (Glasgow et al., 1999). Here, we focus on three key factors in implementation success that are common across existing frameworks—adoption, implementation, and maintenance—applying terminology and definitions from the RE-AIM framework, where possible, for consistency (Glasgow et al., 1999). Attention to these critical factors may help overcome the challenges unique to comprehensive prevention and boost their potential for broad dissemination and effective implementation in the field.

Adoption

Despite their potential for greater impact, comprehensive prevention strategies are inherently more complex and more resource-intensive to implement than single programs (Barton et al., 1997; DeGue et al., 2016; Nation et al., 2003). Communities often have limited resources and capacity to address multiple needs. Decisions regarding the adoption of prevention strategies are influenced by a number of factors, including perceived feasibility, intervention complexity, acceptability, availability of implementation supports, community readiness, and compatibility with the existing infrastructure (Bumbarger et al., 2010; Proctor et al., 2011; Rogers, 1995). Attention to these factors during intervention planning and development can increase the likelihood of adoption and facilitate broader uptake in the field.

The perceived feasibility of an intervention—its simplicity, accessibility, flexibility, ease of use, and cost—is a critical factor that can either facilitate or impede adoption decisions (Dusenbury & Hansen, 2004; Orlandi, 1986). The more clearly and coherently an intervention can be communicated to decision-makers, the lower its perceived complexity and the more likely it is to be adopted (Orlandi, 1986). Some evidence also suggests that more complex programs are less likely than their simplified counterparts to be implemented completely (Smith et al., 1993). Offering effective implementation supports (e.g., accessible training, clear guidance) or flexible implementation options that reflect the diverse capacities of communities may increase perceived feasibility and adoption (Bartholomew et al., 2011). Furthermore, approaches to implementation (e.g., training model, implementation mode, setting, or supports) that minimize costs without sacrificing effectiveness can place the intervention within reach of more communities. Thus, efforts to improve actual and perceived feasibility are critical to promoting adoption and may also improve the quality of implementation and potential for maintenance.

Implementation

Research has demonstrated the importance of high-quality implementation when carrying out evidence-based programs—higher fidelity results in better outcomes (Bumbarger et al., 2010; Elliott & Mihalic, 2004; Glasgow et al., 1999). However, in real-world, practice settings, maintaining fidelity can be a challenge affected by many factors, including school or community capacity, training quality or consistency, implementer supports and resources, intentional adaptations to address community needs, or unplanned adaptations in response to implementation barriers (Botvin, 2004; Bumbarger et al., 2010; Elliott & Mihalic, 2004; Kemp, 2016). Thus, addressing these potential challenges during intervention development and dissemination, and ensuring access to implementation guidance and supports, is a critical consideration when disseminating evidence-based interventions.

The importance of highly trained facilitators is well-documented (Bumbarger et al., 2010; Nation et al., 2003). Interactive training that goes beyond the “how-to’s” to help facilitators understand the theory behind an intervention, along with the provision of ongoing support, can help facilitators engage fully with their role and navigate barriers to fidelity (Bumbarger et al., 2010). For example, providing supplementary training for teacher-implementers, beyond manualized guidance, has been shown to improve fidelity and effectiveness in school-based prevention programs (Wolfe et al., 2009). Balancing fidelity with the potential need for adaptations is a well-recognized challenge in the field; unplanned adaptations can create “drift” and threaten effectiveness, while strict adherence to an “out-of-box” intervention can threaten acceptability and restrain useful innovations in real-world settings (Bumbarger et al., 2010; Kemp, 2016). Guidance on the use of planned adaptations to address the needs of special populations or respond to other challenges (e.g., time limitations) can allow implementers to make necessary adjustments without sacrificing effectiveness (Castro et al., 2004). Identifying common barriers to fidelity can also inform improvements to materials or training that reduce unplanned adaptations (Melde et al., 2006). Furthermore, continual efforts to assess and build capacity (i.e., the resources—knowledge, skills, staff, partners, tools—an organization needs to successfully implement and sustain a prevention model) can help prevent or address challenges with technical capacities, resources, or community buy-in that affect fidelity (Elliott & Mihalic, 2004). Implementing multi-component, comprehensive prevention models with fidelity poses unique challenges but addressing them during program development and refinement with an effective, efficient implementation model may improve uptake, implementation quality, and outcomes.

Maintenance

Without planning for long-term maintenance, interventions—and their effects—may be short-lived (Cooper et al., 2015; Lyon et al., 2011). Maintenance planning early in the process of intervention development is critical (Bumbarger et al., 2010). Sustainable interventions require sufficient organizational capacity and infrastructure, effective partnerships, strong leadership support; “fit” between the intervention and community needs, and access to resources and expertise to sustain the intervention (Cooper et al., 2015; Fixsen et al., 2005; Johnson et al., 2004). To be maintained, interventions must continue to meet the needs of stakeholders over time with the potential for integration into the ongoing operations, or infrastructure, of an organization (Bumbarger et al., 2010; Johnson et al., 2004). Comprehensive prevention strategies may be more challenging to maintain but attention to maintenance planning, continuously building organizational and community capacity for prevention, keeping stakeholders engaged and involved, adapting to the needs and capacity of the community, and integrating the necessary staffing into the organizational infrastructure may increase the potential for long-term maintenance.

Developing the Dating Matters Toolkit

CDC developed the Dating Matters Toolkit,³ a comprehensive package of implementation tools, training, and guidance based on lessons learned from the demonstration project. Development of the Toolkit focused on supporting adoption, implementation, and maintenance in real-world settings through the creation of new resources as well as modifications to the implementation model. While consideration of these factors also occurred during intervention development, revisiting and attending to these concerns while planning for dissemination is an important step in bringing evidence-based interventions to scale (Bartholomew et al., 2011).

Qualitative review of several existing data sources informed the development of the Toolkit for national dissemination. First, in-person focus groups were conducted at each of the four evaluation sites with youth (N = 61; four groups) and parents (N = 27; four groups) who participated in Dating Matters programs during the demonstration project. Focus groups with youth assessed recall of the program content and the extent to which students used or shared its messages with others, as well as program strengths and areas for improvement. Parent focus groups sought to understand parent recall and use of the program content, program strengths and areas for improvement, and ways to improve parent participation for future implementation. Second, structured 1-hr interviews were completed with five individuals who implemented Dating Matters during the demonstration project. Participants provided feedback on their experience with youth and parent programs, the technical assistance model, the community-level components, and general recommendations for improvements. For both the focus groups and key informant interviews, the authors reviewed key findings summarized by federal contractors based on their qualitative analysis of participant feedback. Third, the authors reviewed implementation data collected during the demonstration project, including 4 years of session logs completed by program facilitators after each session of the youth programs (sixth grade, N = 404; seventh grade, N = 401) and parent programs (sixth grade, N = 76; seventh grade, N = 70) to identify common challenges with fidelity. Session logs contained both quantitative and qualitative data. We did not review implementation data for eighth-grade youth and parent programs as both are existing evidence-based programs distributed by a private publisher and, as such, the program content could not be modified as part of the Toolkit development (i.e., Safe Dates, Families for Safe Dates; Hazelden Publishing). However, the Toolkit was designed to support and improve implementation of the entire prevention model, including these eighth-grade programs. Fourth, two experienced program facilitators from the demonstration project reviewed the CDC-developed sixth- and seventh-grade youth and parent programs in detail and provided specific feedback based on their experience implementing the programs. They provided recommendations for revisions or improvements to the activities, messaging, and implementation guidance with the goal of improving acceptability and relevance to participants and ease of use for facilitators.

Through qualitative review of these data sources and lessons learned from the demonstration project, we identified several potential challenges for the national dissemination of Dating Matters. These challenges fell into four key areas aligned with the goals of improving adoption, implementation, and maintenance: training, technical assistance, model flexibility, and accessibility. With an intentional focus on retaining core content and hypothesized mechanisms of change, a series of modifications, updates, and improvements were made to the Dating Matters implementation model to address these challenges and facilitate scale-up prior to national dissemination. To inform implementation practice, we outline these challenges and solutions implemented to address them here as examples of strategies that could be used to improve the dissemination potential of future comprehensive prevention models (see Table 1 for a summary).

Table 1.

Summary of key modifications to the Dating Matters^® implementation model for national dissemination.

Challenge	Demonstration project (original model)	Dating Matters Toolkit^a (current model)
Training	● Train-the-trainer model ● Training for master and local trainers (40–50 hr, in person) ● Program facilitator training conducted by local trainer (40–50 hr, in person)	● Free, web-based program facilitator training (~4 hr) ● In-person training activities (~2 hr) and ongoing supervision provided by a Dating Matters Coach^b ● Coaches’ Playbook^c provides training for Dating Matters Coaches
Technical assistance	● Technical assistance contractor, federally funded	● Dating Matters Coach provides supervision and support to program facilitators and partner organizations (e.g., schools) ● Team Up! for Dating Matters online community of practice^d ● Dating Matters Guide to Implementation^e ● Coaches’ Playbook guides Dating Matters Coaches in providing supervision and support to program facilitators
Model flexibility	● All model components required by federal funding	● Implementation guidance supports phased implementation of selected components while building capacity for comprehensive prevention ● Adaptation guidance for youth and parent programs
Accessibility	● Accessible to communities receiving federal funding for implementation ● Many program materials requiring high-cost professional printing	● Free online access to all program, training, and implementation materials and tools through the Toolkit website^a ● At-a-Glance briefs provide decision-makers with easy to access, high-level information on the model ● Most materials available for free download and low-cost self-printing Select professionally printed materials available at no cost to communities^f

The Dating Matters Toolkit is available at https://vetoviolence.cdc.gov/apps/dating-matters-toolkit/

Dating Matters Coaches provide training, supervision, and support to facilitators of the youth and parent programs, and are typically employed (one Coach per community) by the health department or community-based organization leading Dating Matters implementation. Coaches complete the free, online facilitator training with additional guidance provided in the Coaches’ Playbook, at no additional cost to communities.

The Coaches’ Playbook is available free online: https://vetoviolence.cdc.gov/sites/vetoviolence.cdc.gov.apps.dating-matters-toolkit/themes/dmh_ng_bootstrap/assets/pdf/DM-Coaches-Playbook-RGB-508-8-2.pdf

Team Up! is a free, peer-led, CDC-moderated online community of practice accessible through a free mobile or desktop application: https://vetoviolence.cdc.gov/apps/dating-matters-toolkit/team-up#/

The Dating Matters Guide to Implementation provides an overarching roadmap for lead organizations interested in implementing the Dating Matters comprehensive model, and is available free online: https://vetoviolence.cdc.gov/sites/vetoviolence.cdc.gov.apps.dating-matters-toolkit/themes/dmh_ng_bootstrap/assets/pdf/DM-Guide-to-Implementation-RGB-508-8-2.pdf

Some materials included in the i2i: What R U Looking 4? youth communications program were designed for professional printing to increase their appeal to youth. These materials can be ordered from CDC in print at no cost. See: https://vetoviolence.cdc.gov/apps/dating-matters-toolkit/i2i-program-facilitator#/. In addition, select additional materials that benefit from professional printing, such as spiral-bound facilitator manuals, can also be ordered for free from CDC in addition to being available for free download and self-printing. All CDC-developed program and implementation materials are available at no cost to communities in print or as a free download.

Training

Accessible, efficient, and effective training is critical to supporting the adoption, implementation, and maintenance of an intervention. Training for facilitators of the Dating Matters youth and parent programs during the demonstration project occurred in 5- or 6-day in-person sessions. Using a train-the-trainer approach, CDC trained a small group of master trainers at the national level who then trained staff at each site to serve as local trainers for facilitators throughout implementation. Continuing this training model posed several challenges for scale-up. First, it was time-intensive, costly, and required an infrastructure of certified master trainers available to train new communities. Furthermore, interviews with key staff suggested there was substantial variation in fidelity to the training curricula when carried out by local trainers at each site, indicating potential for drift in the quality or content of facilitator training over time. Finally, the potential for high turnover of program facilitators (common in most community-based interventions) posed a challenge for training to new facilitators as they are hired.

To address these challenges for national dissemination, CDC developed a free, web-based program facilitator training that substantially reduces the time and cost involved in training and ensures all facilitators receive consistent training content. Although substantially condensed, the training is interactive, covers core content from the in-person trainings, and includes videos of select sessions and skills demonstrated by experienced facilitators from the demonstration project, with actors portraying youth and parent participants. The total training time for a youth or parent program facilitator is about 6 hr. This includes about 2 hr of online material, 2 hr of self-directed “homework” activities, and 2 hr of in-person, one-on-one coaching.

To supplement the online training, a new staff role—the Dating Matters Coach—was developed to ensure facilitators receive the training, supervision, and support needed to implement the programs with fidelity. Coaches are local staff employed by the implementing organization to provide ongoing implementation support and technical assistance to facilitators and oversee youth and parent program implementation, including management tasks such as coordination with sites and data collection for program evaluation. Coaches will ideally be experienced and skilled health educators with the ability to supervise and support youth and parent program facilitators. A full-time Coach position is recommended for large-scale implementation (e.g., multiple schools and organizations), but the role can also be filled by part-time staff or a full-time staff member holding multiple roles (e.g., Coach and program facilitator) in small-scale implementation, providing communities with greater flexibility in staffing. With more time devoted to Dating Matters than a typical program facilitator, a Coach can engage more deeply with the program content, consider the need for adaptations, connect with Coaches in other communities, problem-solve, and develop best practices for their community that can be shared with facilitators to improve implementation. Coaches complete the same free online training as facilitators and have access to a Coaches’ Playbook⁴ with detailed guidance for carrying out their training and supervisory roles.

By moving to an online training model with support provided by a Coach, communities have low-cost access to flexible, feasible training that is high-quality and consistent—overcoming important barriers to adoption, implementation, and maintenance. Although staffing a Coach position may require significant resources, these resources not only facilitate the initial training needed to carry out multiple programs but provide ongoing support to ensure quality implementation. Program facilitators often provide services through in-kind partnerships with local organizations, as part of their regular work as teachers or school staff, or through short-term contracts. Thus, integrating key prevention staff, like a Coach, into the infrastructure of the lead organization overseeing implementation can help build capacity and support maintenance.

Technical assistance

Beyond training, a range of implementation supports are needed to ensure organizations have the capacity, knowledge, and resources needed to carry out a multi-component prevention strategy. Technical assistance (e.g., answering questions, problem-solving, creating supplemental guidance, providing additional training or instruction) was provided to sites in the Dating Matters demonstration project through a federally funded contracting organization. Continuing this model for national dissemination would limit the number of communities that could access these supports to the capacity of a contract. In lieu of this model, CDC built alternate sources of technical assistance into the dissemination plan to ensure that all communities would have access to the guidance and support needed to implement Dating Matters with fidelity. Together, these resources aim to make technical assistance freely available to all communities—decreasing the perceived complexity of the intervention, increasing the potential for greater adoption, and improving the ability of organizations to implement the model with greater fidelity and impact.

One key source of technical assistance, as noted above, is the Dating Matters Coach, a role created to ensure program facilitators have access to an experienced, well-trained supervisor to address questions, challenges, and other issues that arise during program implementation. Rogers (2002) also highlighted the importance of peer support and activating peer networks to diffuse preventive interventions by encouraging peer communication and changing system norms. Thus, to promote peer-to-peer support and learning, CDC also created an online community of practice, Team Up! for Dating Matters,⁵ moderated by CDC and accessible through a free mobile app and online. Communities implementing Dating Matters are encouraged to share questions, tips, challenges, solutions, and success stories on a secure and accessible platform. Participants can also interact with and receive assistance from CDC staff and experienced Dating Matters facilitators from the demonstration project engaged in Team Up! as consultants. Everyone involved in implementation—from lead staff to program facilitators—is encouraged to join.

The Dating Matters Toolkit also includes a comprehensive package of implementation guidance and resources for each component of the prevention model as well as for the model itself. Lessons learned from the demonstration project indicated that a key challenge in standing up a comprehensive prevention model like Dating Matters, is knowing where to start and how to build and maintain the necessary infrastructure. The Dating Matters Guide to Implementation⁶ (CDC, 2019) was developed to provide complete guidance for carrying out the comprehensive prevention model, including information on building capacity and partnerships, planning a budget, hiring staff, recruiting schools and participants, adaptation, program evaluation, and sustainability. It was designed to answer common questions and challenges experienced by communities in the demonstration project, and includes detailed information to help lead staff navigate the challenges of standing up a multi-component prevention strategy.

Model flexibility

Communities vary in their initial capacity to carry out evidence-based prevention efforts in terms of expertise, resources, and stakeholder support. Building flexibility into an intervention or implementation model allows communities to start where they are and grow their capacity over time, increasing the potential for adoption and maintenance (Bartholomew et al., 2011). This could be particularly important for comprehensive prevention strategies that may require more resources and partnerships to implement than a single-program approach.

Dating Matters was designed and evaluated as a comprehensive, multi-level prevention strategy comprising seven programs and three community-level prevention activities. To achieve the preventive effects demonstrated in the evaluation (e.g., Niolon et al., 2019), implementation of the entire model is recommended. However, to increase its feasibility and adoption potential for national dissemination, the Toolkit was designed to support communities interested in implementing only select components of the model as they build capacity for comprehensive prevention. For example, communities interested in implementing only the youth programs to start can easily access the training, program materials, and implementation guidance that is specific to those programs. Staffing recommendations for the model were also designed to be flexible, allowing smaller organizations or communities to begin these efforts with existing resources and expand their infrastructure, staff, and expertise as their capacity and resources grow. The Dating Matters Capacity Assessment and Planning Tool⁷ (DM-CAPT) is available to help communities assess and improve their prevention capacity over time, with the goal of integrating additional components into their efforts as capacity increases and achieving comprehensive prevention as designed. By building flexibility into the implementation model for this comprehensive prevention strategy, Dating Matters can reach more youth, families, and neighborhoods sooner while leveraging successes from those efforts to engage additional partners, secure necessary resources, and build an infrastructure that can be maintained and is ready for expansion. Providing opportunities for pre-implementation work to assess and address community readiness (e.g., needs, capacity, infrastructure) may create “fertile ground” to facilitate strategy adoption and increase the likelihood of quality implementation and maintenance (Bumbarger et al., 2010).

Accessibility

Adoption decisions are influenced by several factors, including perceived feasibility and complexity (Bumbarger et al., 2010; Orlandi, 1986). Accessible interventions—those that require fewer resources to obtain, understand, or assess for fit—may have greater potential for adoption and, thus, widespread dissemination. A key modification of the Dating Matters implementation model for national dissemination involved the creation of an online hub—the Dating Matters Toolkit website⁸—that provides free access to all CDC-developed resources and materials necessary to implement the prevention model. The interactive website is organized by staff role and prevention component to help users identify the materials they need most quickly and reduce the perceived complexity of the intervention. In addition, brief “At-a-Glance” documents are included to provide interested communities and decision-makers with a high-level introduction to the overall model and each component, including its goals and the resources required. During the demonstration model, CDC provided professionally printed program materials to each site. To reduce costs and increase access, most program materials were re-designed prior to national dissemination to allow free downloads from the Toolkit website and low-cost self-printing. Select materials that still require professional printing can be ordered from CDC for free.

In the first 6 months that the Toolkit was available, the website was viewed thousands of times and more than 100 unique organizations requested access to the program materials and facilitator training by completing a required user agreement. Requestors most often represented violence prevention or response organizations (e.g., domestic violence/sexual violence prevention, shelters, victim services), 56%; state and local governments, 12%; or K–12 schools, 9%. Although there is not currently a mechanism to track uptake and implementation of the Dating Matters model nationwide, these early indicators of interest and adoption suggest substantial potential for the reach of this comprehensive prevention approach.

Evaluating the Dating Matters Toolkit

CDC conducted a mixed methods evaluation of the Dating Matters Toolkit in Spring 2020 to examine usability and impacts on the adoption, implementation, and maintenance potential of the model. Individuals were invited to complete an anonymous 18-item survey online through a public link on the Toolkit website as well as emails sent to registered users and key CDC partners in dating violence prevention. Survey respondents were asked to provide contact information if they would be willing to participate in a 30 min follow-up interview. No incentives were provided. Interviews were administered by trained facilitators through a federal contract with informed consent from participants. Interview data was analyzed in MAXQDA 2018 for patterns relating to website navigation, program and material comprehension, interest and perceived feasibility of programmatic implementation, and user satisfaction.

Results

Survey participants (N = 70) had diverse professional backgrounds ranging from youth-serving professionals and victim advocates to law enforcement; 84% reported working in violence-related fields. Interview participants (N = 16) also represented a diverse set of stakeholders from front-line prevention staff to organizational leadership and technical advisors to federal grant recipients serving youth. Among interview participants, 19% were currently implementing Dating Matters, 44% were considering integrating a Dating Matters component into their current work, and 38% had never implemented Dating Matters and had no current plans to start. Notably, very few participants had experience with Dating Matters implementation at the time of data collection due to the recent release of the Toolkit and program materials (about 6 months prior) and the COVID-19 pandemic that closed most US schools in the Spring of 2020, side-tracking implementation planning for school-based and in-person interventions. Thus, these data reflect, primarily, the experience of Toolkit users as they utilized the website to learn about the prevention model, assess its feasibility and potential acceptability in their community, and begin planning for future implementation.

Survey and interview data provide a glimpse at perceptions of adoption and implementation feasibility. Survey data indicated that three-quarters of respondents agreed (54%) or strongly agreed (21%) that the Toolkit contained all of the implementation supports tools and materials needed for adoption and implementation. When asked whether it would be possible to implement all seven components of Dating Matters with the right resources and partnerships in place, the vast majority of respondents indicated it was possible (51%) or definitely possible (31%), with only two respondents indicating it was impossible (2.8%). A subset of 28 survey respondents were asked about potential barriers to adopting and implementing the comprehensive model and reported insufficient financial or human resources or a lack of community buy-in as concerns most often (see Table 2). Likewise, most adoption challenges that participants discussed during interviews were related to gaining buy-in from key stakeholders (including teachers, school officials, leadership in their organizations, student ambassadors, and parents) and having time and funding to carry out the programmatic activities in their entirety. Yet, responses in both the survey and interviews reflected strong interest in adopting and implementing Dating Matters. Most survey respondents indicated that they were interested (37%) or very interested (47%) in adopting and implementing one or more components of Dating Matters their community or organization.

Table 2.

Barriers to Dating Matters implementation using the Dating Matters Toolkit, reported by survey respondents (n = 28).

Barrier reported	% (n)
Too expensive/lack funding	32 (9)
Insufficient staffing or expertise	25 (7)
Lack of community support or “buy-in”	25 (7)
Lack of necessary community partnerships	18 (5)
Already implementing other complementary strategies	18 (5)
Too difficult	14 (1)
Do not see the need or potential benefit of comprehensive prevention	4 (1)

These findings suggest that the Dating Matters Toolkit and modified model described here may support improved adoption and implementation, as intended. However, anticipated challenges related to resources and engaging stakeholders may continue to be important barriers to adoption of the entire comprehensive model as well as maintenance of those components that are adopted. The Toolkit may provide a useful model for developers of multi-component prevention strategies engaged in dissemination planning. However, additional research and continual improvements to the Toolkit are needed to meet the challenges that communities face and to bring effective prevention to scale.

Considerations for future research and dissemination

Not all communities interested in dating violence prevention will have the initial capacity to carry out comprehensive prevention and may only have the resources or partnerships needed for select components (e.g., school-based programs). As such, the Toolkit was designed to support adoption and implementation of individual components, when needed, to meet the needs of more communities. However, implementation that deviates from the model, as evaluated, has implications for effectiveness. While the complete, comprehensive model has evidence of effectiveness for preventing teen dating violence and related outcomes (Niolon et al., 2019; Vivolo-Kantor et al., 2019), many of the individual components are also either evidence-based or evidence-informed (i.e., youth and parent programs) and/or have no existing evidence-based equivalents in the field (i.e., training for educators, youth communications program, policy and indicator data activities). The components have also been implemented successfully with middle school students in underserved communities where limited research is available. Thus, the individual components may still represent the best available options for some communities investing in dating violence prevention (Niolon et al., 2016, 2019). Ongoing program evaluation in the field will be critical in assessing the effects of subsets of the comprehensive model on key outcomes, as implemented using the Toolkit resources. Program evaluation guidance, including suggested process and outcome measures, is provided in the Dating Matters Guide to Implementation.

Future research that examines the impact of the Toolkit on adoption decisions, implementation quality, and maintenance is needed. Although Dating Matters has demonstrated effectiveness in preventing teen dating violence, it is possible that the Toolkit may result in strengthened outcomes through improved feasibility, fidelity, support, and maintenance planning in the future. Conversely, it is possible that some modifications could have unexpected negative impacts on effectiveness and outcomes. Implementation research conducted in the field where communities have attempted to implement Dating Matters utilizing only the Toolkit—without the additional financial and technical supports provided by CDC in the demonstration project—would greatly improve understanding of this dissemination model, its efficacy, and future improvements. Such research could also assess the extent to which communities are able to adopt the entire comprehensive model as designed or choose to focus their resources on a subset of components. Although we would anticipate reductions in effectiveness as the intervention becomes less comprehensive—and closer to a standard single-program approach—opportunities to evaluate subsets of the model as implemented in the field, particularly those components with the greatest stand-alone uptake, would be valuable. These evaluations could help inform decision-makers considering investment in comprehensive prevention by identifying core components, the unique effectiveness of subsets of interventions within the model, or quantifying reductions in effectiveness as the model is scaled back. It is possible that such research could also help identify opportunities to “trim the fat” and create additional efficiencies for implementation without sacrificing effectiveness, further improving potential for dissemination (Bumbarger et al., 2010).

It is important to balance the need for interventions to “evolve” over time in response to new evidence with the risks of modifying them in ways that could impact efficacy demonstrated in prior evaluations. As more is learned from implementation research and practice, the need for further modifications or adaptations to the Dating Matters intervention model or implementation guidance may arise. CDC is applying lessons learned from the user surveys and interviews described here to continually improve and expand the content and delivery of the Toolkit through an iterative, rapid feedback loop. For example, CDC is currently developing detailed program evaluation guidance to increase the capacity of communities to take on this crucial activity as part of their implementation planning. The design of the Toolkit website itself is also being streamlined to ensure greater usability and ease of access. In addition, to meet the needs of communities during the COVID-19 pandemic, CDC is creating adaptation guidance in 2020 to help schools and organizations implement Dating Matters programs online. New research on teen dating violence might suggest ways to strengthen outcomes by building on or revising the existing strategies in the future. It is important to balance the need for interventions to “evolve” over time in response to new evidence with the risks of modifying them in ways that could impact efficacy demonstrated in prior evaluations. Researchers have pointed to the benefits of using iterative, rapid research designs to continually optimize—rather than concretize—interventions as an innovative shift toward improving the timeliness and responsivity of research (Riley et al., 2013). Such approaches could be useful for assessing the effects of model modifications on key outcomes as those changes are implemented over time, and would be consistent with recommendations within the field that adaptations to evidence-based interventions necessitate additional evaluation (Bartholomew et al., 2011; Elliott & Mihalic, 2004).

Given the unique challenges and potential benefits of implementing comprehensive prevention approaches, learning more about what works—and what does not—in real-world settings is critical to promoting scale-up of these prevention strategies in the future. Attending to adoption, quality implementation, and maintenance during intervention development, planning for dissemination, and supporting an iterative process with ongoing evaluation, may help make the promise of multi-level, coordinated, and evidence-based prevention strategies a reality for more communities.

Footnotes

Acknowledgements

The authors acknowledge Lianne Estefan, Beverly Fortson, Jeffrey Herbst, Phyllis Niolon, Jennifer Dills, and Henrietta Kuoh in the National Center for Injury Prevention and Control at CDC; Mauro Sifuentes; Aimee Wood; and Banyan Communications, Inc. for their input and feedback on the development of the Toolkit. The authors also acknowledge the public health departments (Alameda County, CA; Baltimore, MD; Broward County, FL; and Chicago, IL) and contracting organizations (Edge Research; Research Triangle Institute; Ogilvy Public Relations; NORC at the University of Chicago) who contributed to the Dating Matters demonstration project. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Sarah DeGue

Notes

References

Bartholomew

L. K.

Parcel

G. S.

Kok

Gottleib

N. H.

Fernandez

M. E.

(2011). Planning health promotion programs: An intervention mapping approach. Jossey-Bass.

Barton

Watkins

Jarjoura

(1997). Youths and communities: Toward comprehensive strategies for youth development. Social Work, 42(5), 483–493. https://doi.org/10.1093/sw/42.5.483

Basile

K. C.

DeGue

Jones

Freire

Dills

Smith

S. G.

Raiford

J. L.

(2016). STOP SV: A technical package to prevent sexual violence. National Center for Injury Control and Prevention, Centers for Disease Control and Prevention. http://www.cdc.gov/violenceprevention/pdf/sv-prevention-technical-package.pdf

Botvin

G. J.

(2004). Advancing prevention science and practice: Challenges, critical issues, and future directions. Prevention Science, 5(1), 69–72. https://doi.org/10.1023/B:PREV.0000013984.83251.8b

Bumbarger

B. K.

Perkins

D. F.

Greenburg

M. T.

(2010). Taking effective prevention to scale. In Pfohl

D. B., W.

Yoon

(Eds.), Handbook of youth prevention science (pp. 433–444). Routledge

Castro

F. G.

Barrera

Martinez

C. R.

(2004). The cultural adaptation of prevention interventions: Resolving tensions between fidelity and fit. Prevention Science, 5(1), 41–45. https://doi.org/10.1023/B:PREV.0000013980.12412.cd

Centers for Disease Control and Prevention. (2019). Dating Matters®: Strategies to promote healthy teen relationships, guide to implementation.

Community Preventive Services Task Force. (2018). Primary prevention interventions to reduce perpetration of intimate partner violence and sexual violence among youth. https://www.thecommunityguide.org/findings/violence-primary-prevention-interventions-reduce-perpetration-intimate-partner-violence-sexual-violence-among-youth

Cooper

B. R.

Bumbarger

B. K.

Moore

J. E.

(2015). Sustaining evidence-based prevention programs: Correlates in a large-scale dissemination initiative. Prevention Science, 16(1), 145–157. https://doi.org/10.1007/s11121-013-0427-1

10.

David-Ferdon

Simon

(2014). Preventing youth violence: Opportunities for action. National Center for Injury Prevention and Control, Centers for Disease Control and Prevention.

11.

DeGue

Hipp

T. N.

Herbst

J. H.

(2016). Community-level approaches to prevent sexual violence. In Jeglic

E. L.

Calkins

(Eds.), Sexual violence: Evidence based policy and prevention (pp. 161–179). Springer.

12.

DeGue

Niolon

P. H.

Estefan

L. F.

Tracy

A. J.

V. D.

Vivolo-Kantor

A. M.

Little

T. D.

Latzman

N. E.

Tharp

A. T.

Lang

K. M.

Taylor

B. G.

(2020). Effects of Dating Matters® on sexual violence and sexual harassment outcomes among middle school youth: A cluster-randomized controlled trial. Prevention Science. Advance online publication. https://doi.org/10.1007/s11121-020-01152-0

13.

DeGue

Valle

L. A.

Holt

M. K.

Massetti

G. M.

Matjasko

J. L.

Tharp

A. T.

(2014). A systematic review of primary prevention strategies for sexual violence perpetration. Aggression and Violent Behavior, 19(4), 346–362. https://doi.org/10.1016/j.avb.2014.05.004

14.

Dittus

Miller

K. S.

Kotchick

B. A.

Forehand

(2004). Why parents matter! The conceptual basis for a community-based HIV prevention program for the parents of African American youth. Journal of Child and Family Studies, 13(1), 5–20. https://doi.org/10.1023/b:jcfs.0000010487.46007.08

15.

Dusenbury

Hansen

W. B.

(2004). Pursuing the course from research to practice. Prevention Science, 5(1), 55–59. https://doi.org/10.1023/B:PREV.0000013982.20860.19

16.

Elliott

D. S.

Mihalic

(2004). Issues in disseminating and replicating effective prevention programs. Journal of Prevention Science, 5(1), 47–53. https://doi.org/10.1023/B:PREV.0000013981.28071.52

17.

Estefan

L. F.

Vivolo-Kantor

A. M.

Niolon

P. H.

V. D.

Tracy

A. J.

Little

T. D.

DeGue

Latzman

N. E.

Tharp

A. T.

Lang

K. M.

McIntosh

W. L.

(2020). Effects of the Dating Matters® comprehensive prevention model on health- and delinquency-related risk behaviors in middle school youth: A cluster-randomized controlled trial. Prevention Science. Advance online publication. https://doi.org/10.1007/s11121-020-01114-6

18.

Fixsen

D. L.

Naoom

S. F.

Blase

K. A.

Friedman

R. M.

Wallace

(2005). Implementation research: A synthesis of the literature (FMHI publication #231). The National Implementation Research Network, Louis de la Parte Florida Mental Health Institute, University of South Florida.

19.

Fortson

B. L.

Klevens

Merrick

M. T.

Gilbert

L. K.

Alexander

S. P.

(2016). Preventing child abuse and neglect: A technical package for policy, norm, and programmatic activities. Centers for Disease Control and Prevention.

20.

Foshee

V. A.

Bauman

K. E.

Ennett

S. T.

Linder

G. F.

Benefield

Suchindran

(2004). Assessing the long-term effects of the safe dates program and a booster in preventing and reducing adolescent dating violence victimization and perpetration. American Journal of Public Health, 94(4), 619–624. https://doi.org/10.2105/AJPH.94.4.619

21.

Foshee

V. A.

McNaughton Reyes

H. L.

Ennett

S. T.

Cance

J. D.

Bauman

K. E.

Bowling

J. M.

(2012). Assessing the effects of Families for safe dates, a family-based teen dating abuse prevention program. Journal of Adolescent Health, 51(4), 349–356. https://doi.org/10.1016/j.jadohealth.2011.12.029

22.

Frieden

T. R.

(2010). A framework for public health action: The health impact pyramid. American Journal of Public Health, 100(4), 590–595. https://doi.org/10.2105/AJPH.2009.185652

23.

Glasgow

R. E.

Vogt

T. M.

Boles

S. M.

(1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework, 89(9), 1322–1327.

24.

Green

Kreuter

(2005). Health program planning: An educational and ecological approach. McGraw-Hill.

25.

Johnson

Hays

Center

Daley

(2004). Building capacity and sustainable prevention innovations: A sustainability planning model. Evaluation and Program Planning, 27(2), 135–149. https://doi.org/10.1016/j.evalprogplan.2004.01.002

26.

Kemp

(2016). Adaptation and fidelity: A recipe analogy for achieving both in population scale implementation. Prevention Science, 17(4), 429–438. https://doi.org/10.1007/s11121-016-0642-7

27.

Klesges

L. M.

Estabrooks

P. A.

Dzewaltowski

D. A.

Bull

S. S.

Glasgow

R. E.

(2005). Beginning with the application in mind: Designing and planning health behavior change interventions to enhance dissemination. Annals of Behavioral Medicine, 29(2), 66–75. https://doi.org/10.1207/s15324796abm2902s_10

28.

Lyon

A. R.

Frazier

S. L.

Mehta

Atkins

M. S.

Weisbach

(2011). Easier said than done: Intervention sustainability in an urban after-school program. Administration Policy in Mental Health and Mental Health Services Research, 38(6), 504–517. https://doi.org/10.1007/s10488-011-0339-y

29.

Matjasko

Vivolo-Kantor

Massetti

Holland

Holt

Dela Cruz

(2012). A systematic meta-review of evaluations of youth violence prevention programs: Common and divergent findings from 25 years of meta-analyses and systematic reviews. Aggression and Violent Behavior, 17(6), 540–552. https://doi.org/https://dx-doi-org.web.bisu.edu.cn/10.1016/j.avb.2012.06.006

30.

Melde

Esbensen

F.-A.

Tusinski

(2006). Addressing program fidelity using onsite observations and program provider descriptions of program delivery. Evaluation Review, 30(6), 714–740. https://doi.org/10.1177/0193841X06293412

31.

Mercy

J. A.

Rosenberg

M. L.

Powell

K. E.

Broome

C. V.

Roper

W. L.

(1993). Public health policy for preventing violence. Health Affairs, 12(4), 7–29. https://doi.org/10.1377/hlthaff.12.4.7

32.

Nation

Crusto

Wandersman

Kumpfer

K. L.

Seybolt

Morrissey-Kane

Davino

(2003). What works in prevention: Principles of effective prevention programs. American Psychologist, 58(6–7), 449–456. https://doi.org/10.1037/0003-066X.58.6-7.449

33.

Niolon

P. H.

Kearns

Dills

Rambo

Irving

Armstead

Gilbert

L. K.

(2017). Preventing intimate partner violence across the lifespan: A technical package of programs, policies, and practices. https://www.cdc.gov/violenceprevention/pdf/ipv-technicalpackages.pdf

34.

Niolon

P. H.

Taylor

B. G.

Latzman

N. E.

Vivolo-Kantor

A. M.

Valle

L. A.

Tharp

A. T.

(2016). Lessons learned in evaluating a multisite, comprehensive teen dating violence prevention strategy: Design and challenges of the evaluation of Dating Matters: Strategies to promote healthy teen relationships. Psychology of Violence, 6(3), 452–458. https://doi.org/10.1037/vio0000043

35.

Niolon

P. H.

Vivolo-Kantor

A. M.

Tracy

A. J.

Latzman

N. E.

Little

T. D.

DeGue

Lang

K. M.

Estefan

L. F.

Ghazarian

S. R.

McIntosh

W. L.

Taylor

Johnson

L. L.

Kuoh

Burton

Fortson

Mumford

E. A.

Nelson

S. C.

Joseph

Valle

L. A.

Tharp

A. T.

(2019). An RCT of Dating Matters: Effects on teen dating violence and relationship behaviors. American Journal of Preventive Medicine, 57(1), 13–23. https://doi.org/10.1016/j.amepre.2019.02.022

36.

O’Connell

M. E.

Boat

Warner

K. E.

(2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities (Vol. 7). National Academic Press.

37.

Orlandi

M. A.

(1986). The diffusion and adoption of worksite health promotion innovations: An analysis of barriers. Preventive Medicine, 15(5), 522–536. https://doi.org/10.1016/0091-7435(86)90028-9

38.

Proctor

Silmere

Raghavan

Hovmand

Aarons

Bunger

Griffey

Hensley

M. J. A.

Health

Research

M. H. S.

(2011). Outcomes for implementation research: Conceptual distinctions. Measurement Challenges, and Research Agenda, 38(2), 65–76.

39.

Riley

W. T.

Glasgow

R. E.

Etheredge

Abernethy

A. P.

(2013). Rapid, responsive, relevant (R3) research: A call for a rapid learning health research enterprise. Clinical and Translational Medicine, 2(1), Article e10. https://doi.org/10.1186/2001-1326-2-10

40.

Rogers

E. M.

(1995). Diffusion of Innovations (4th ed.). Free Press.

41.

Rogers

E. M.

(2002). Diffusion of preventive innovations. Addictive Behaviors, 27(6), 989–993. https://doi.org/10.1016/S0306-4603(02)00300-3

42.

Rothman

E. F.

Bair-Merritt

M. H.

Tharp

A. T.

(2015). Beyond the individual level: Novel approaches and considerations for multilevel adolescent dating violence prevention. American Journal of Preventive Medicine, 49(3), 445–447. https://doi.org/10.1016/j.amepre.2015.05.019

43.

Smith

D. W.

McCormick

L. K.

Steckler

A. B.

McLeroy

K. R.

(1993). Teachers’ use of health curricula: Implementation of growing healthy, project SMART, and the teenage health teaching modules. Journal of School Health, 63(8), 349–354. https://doi.org/10.1111/j.1746-1561.1993.tb07151.x

44.

Teten Tharp

Burton

Freire

Hall

Harrier

Latzman

Luo

Niolon

Ramirez

Vagi

. (2011). Dating Matters™: Strategies to promote healthy teen relationships. Journal of Women’s Health, 20, 1–5. https://doi.org/10.1089/jwh.2011.3177

45.

Vivolo-Kantor

A. M.

Niolon

P. H.

Estefan

L. F.

V. D.

Tracy

A. J.

Latzman

N. E.

Little

T. D.

Lang

K. M.

DeGue

Tharp

A. T.

(2019). Middle school effects of the Dating Matters^® comprehensive teen dating violence prevention model on physical violence, bullying, and cyber-bullying: A cluster-randomized controlled trial. Prevention Science. Advance online publication. https://doi.org/10.1007/s11121-019-01071-9

46.

Whitaker

D. J.

Morrison

Lindquist

Hawkins

S. R.

O’Neil

J. A.

Nesius

A. M.

(2006). A critical review of interventions for the primary prevention of perpetration of dating violence. Aggression and Violent Behavior, 11, 151–166. https://doi.org/10.1016/j.avb.2005.07.007

47.

Wolfe

D. A.

Crooks

C. V.

Jaffe

Chiodo

Hughes

Ellis

Stitt

Donner

(2009). A school-based program to prevent adolescent dating violence: A cluster randomized trial. Archives of Pediatric and Adolescent Medicine, 163, 692–699. https://doi.org/10.1001/archpediatrics.2009.69