Implementing Sustainable Evidence-Based Interventions in the Community: A Fidelity-Focused Training Framework for the Savvy Caregiver Program

Abstract

Evidence-based (EB) programs provide a tested means for addressing serious public health concerns. While establishing and maintaining fidelity to these programs is critical for high-quality outcomes, unfortunately, implementation reports rarely describe the strategies used to train and monitor fidelity in community-based implementations. Thus, an essential translational ingredient is a clearly articulated model for establishing a systemic, sustainable fidelity framework. This article provides a detailed description of the fidelity-focused framework developed for Michigan’s statewide implementation of the Savvy Caregiver Program. This framework, titled Creating Confident Caregivers® (CCC), instilled conceptual understanding of fidelity, practice adherence, and competence via ongoing progressive practitioner training, peer observation, participant evaluation, and experiential assessments. Practitioners maintained very high evaluations from participants, and a select cadre of practitioners achieved master trainer status to monitor regional fidelity. This framework promotes sustainable fidelity to EB interventions and provides a replicable platform for other community-based implementations.

Keywords

Alzheimer’s disease evidence-based program fidelity implementation intervention training

Community-based implementations of evidence-based (EB) programs provide a tested means for addressing serious public health concerns (Breitenstein et al., 2010; McCurry et al., 2015). In response to rising prevalence of Alzheimer’s disease and other dementias, for example, the Administration on Aging’s Alzheimer’s disease Supportive Services Program (ADSSP) shifted its funding priorities to support broad scale translations of EB programs for family caregivers of persons with dementia. Since 2007, ADSSP grants have provided states with multiyear funding to implement and evaluate EB dementia caregiver programs via community partners (seehttp://www.aoa.acl.gov/AoA_Programs/HPW/Alz_Grants).

One hallmark of successful implementation is upholding fidelity to the original EB program (Frank, Coviak, Healy, Belza, & Casado, 2008; National Registry of Evidence-based Programs and Practices [NREPP], 2012). Obtaining fidelity to program components is neither obvious nor inherent to program delivery, and reports of positive outcomes alone are insufficient if implementation fidelity has not been assessed (Borrelli, 2011; Carroll et al., 2007). Moreover, failure to achieve fidelity can result in a range of unintended effects for participants, practitioners, and organizations. Unfortunately, a paucity of detailed fidelity training or monitoring strategies hinders community-based EB program implementation efforts. Indeed, a review found that treatment integrity, or fidelity, was adequately addressed in only 3.5% of evaluated interventions (Perepletchikova, Treat, & Kazdin, 2007). This article describes a progressive fidelity-focused training framework developed as part of Michigan’s ADSSP statewide implementation of Savvy Caregiver Program (SCP), an EB dementia caregiver intervention (Hepburn, Lewis, Sherman, & Tornatore, 2003). This fidelity framework integrated multiple-modality training and assessment sources to deliver SCP successfully to 4,732 community-based family caregivers, far exceeding original project goals.

“Creating Confident Caregivers®”: A Fidelity-Focused Training Framework

Michigan’s statewide implementation of SCP was branded as “Creating Confident Caregivers®” (CCC). This special branding signified the oversight and fidelity-focused training framework developed for the implementation, and to differentiate it from any potential unmonitored offering of SCP. The CCC framework upheld EB program elements. At the same time, each participating Area Agencies on Aging (AAAs) was empowered to establish its own regional work plan and goals for the program. All 16 AAAs in Michigan participated in the CCC-SCP implementation, and each agency took their particular resources, organizational capacities, regional size, and population base into account when planning their CCC budget and anticipated training and service goals. Moreover, each AAA selected program sites, local partners for the program, and determined the appropriate number of practitioners and programs needed for their region. They were responsible for their region’s marketing, registration, and eligibility determination of program participants.

As Figure 1 illustrates, CCC’s fidelity-focused framework uses an iterative approach to training and implementation. Each component of the training process and program documentation contribute to embedding fidelity and consistent implementation. Similar to the quality assurance process presented by McCallion, Ferretti, and Brick (2008), CCC established a maintenance loop that integrated skill and program development into the cycle of training opportunities. In turn, as practitioners gained more experience delivering SCP, these higher order skills were integrated into subsequent training workshops and fidelity discussions.

Figure 1.

The sustainable fidelity implementation cycle.

EB Program Review and Selection: Laying the Foundation for Implementation

Every EB program has distinct provider and service elements, and its characteristics are key elements that contribute to a successful implementation in the community (Menne et al., 2015). The selection of an EB program should align conceptually and programmatically with the needs, values, and resources of the participating service network, and is a critical first step to establishing a fidelity-focused implementation.

Michigan’s Adult and Aging Services Agency (AASA) determined the SCP was well-suited to its aging network from among the Administration for Community Living (ACL) approved EB dementia caregiver interventions. AASA committed to the AAAs as the optimal project partners due to their service expertise and statewide coverage. SCP’s person-centered approach aligns with Michigan’s commitment to person-centered services. Its 6-weekly session format is time-limited and structured, yet not scripted. As Michigan’s AAAs offer many EB health promotion programs (i.e., Stanford Chronic Disease Self-Management program), AASA felt the network’s familiarity with a weekly group format for a defined audience would benefit the SCP implementation.

The SCP Intervention

SCP was designed explicitly as a psychoeducational intervention for community-based delivery, and has been proven to enhance family caregiver knowledge of dementia, caregiving skills, and attitudes that support them in their care role (Hepburn, Lewis, Tornatore, Sherman, & Bremer, 2007; Samia, Aboueissa, Halloran, & Hepburn, 2014; Smith & Bell, 2005). Participating in SCP empowers caregivers to feel more competent and confident in their ability to provide high-quality care for their family member with dementia. SCP practitioners are trained to “coach” family caregivers to employ their new knowledge and skills in their care role and interactions. The six weekly session content includes (a) information about Alzheimer’s disease and other dementia, (b) caregiver self-care, (c) the anchors of enjoyable (contented) involvement, (d) stages of dementia and impacts on performance, (e) strengthening support resources (family, others), and (f) review and integration.

Fidelity Criteria

Once an appropriate EB program is selected, fidelity criteria must be explained to host service networks, agencies, and practitioners (Estabrooks & Glasgow, 2006; McCallion et al., 2008; Mowbray, Holter, Teague, & Bybee, 2003). Core fidelity criteria have been delineated by the National Council on Aging (2006) and the NREPP (2012) to include serving the target population; adhering to the defined program structure, time frame; maintaining staff skills; utilizing specified equipment and materials; and evaluating participant and program quality. Table 1 presents fidelity criteria, SCP’s components, and how the CCC implementation framework addressed each criterion.

Table 1.

Overview of Core Fidelity Criteria, SCP Components, and CCC Implementation.

Core fidelity criteria	SCP components	CCC implementation of SCP
Serving target audience	SCP is designed and tested to serve informal caregivers of persons with dementia living in the community	Eligibility CCC promotional materials include target audience, eligibility criteria Eligibility screening part of class registration AAA registers participants
Program structure	Psychoeducational training to enhance dementia caregiver knowledge, skills, and attitudes. In-person group format One leader per group No fee to participants	Upholds psychoeducational training approach, format, and structure of SCP Approved SCP materials shared with all participating AAAs, aging network After team-delivery during initial training period, all CCC-SCP led by one practitioner Group and materials free of charge
Program time frame	Six 2-hr weekly sessions	Upholds six 2-hr weekly sessions of SCP ADSSP defined “completion” as attendance of at least four of six SCP sessions
Maintain staff skills	SCP training manual Trainer manual provides the conceptual basis of SCP, and explains the goal, content, and sequence of each session. Manual stresses adopting coaching approach SCP sessions are not scripted, but suggested wording and explicit exercises are presented in detail.	TTT Workshops AAAs select regional practitioners Annual 2-day, in-person training in SCP Led by SCP co-developer Required attendance to deliver SCP Project-wide fidelity conference calls Regularly scheduled to review and support fidelity and implementation Refresher classes
		Annual one-day session to review SCP content, structure, CCC implementationLearning sessions for master trainersAnnual 1-day development sessions
Utilizing specified program materials	SCP materials:SCP trainer’s manualSCP course slides, handouts, videosCaregiver manual	Upholds use of only SCP-produced materialsMinor adaptations in class delivery approved in consultation with SCP developers
Evaluating participant and program quality	SCP participant evaluation survey	CCC-SCP evaluation materials produced, managed, and analyzed by CCC project teamPractitioners collect CCC attendance logs, demographic formsParticipant evaluations based on SCP collected at sixth session of each programSurveys measuring participant outcomes collected during ADSSP grant period

Source. National Council on Aging (2006).

Note. SCP = Savvy Caregiver Program; CCC = Creating Confident Caregivers®; AAA = Area Agencies on Aging; ADSSP = Alzheimer’s disease Supportive Services Program; TTT= Train the Trainer.

It cannot be assumed that service agency staff will instinctively recognize what is required to deliver an EB program with fidelity. As Table 1 outlines, each fidelity and EB program element and the corresponding implementation plan can be described and outlined to stakeholders and participating agencies, and measured to ensure that fidelity is achieved and upheld across the implementation.

Practitioner Training

While Michigan’s aging network is familiar with training and educational programs, delivering an EB program like SCP with fidelity requires special fluency and skill (Maslow, 2012). One has to adhere to SCP curriculum, assume a coaching approach with caregivers, and maintain its psychoeducational tone and tenor. In recognition of these skills, and the treatment and intervention aspect of SCP, AASA designates CCC-SCP program providers as practitioners.

The fidelity cycle requires investment in fidelity-focused training, coaching, and technical assistance for new practitioners. This investment establishes a capable and committed cadre of intervention practitioners, who become the “face” of the EB program implementation. Such practitioners garner support and respect for the program in the community (Borrelli, 2011; Breitenstein et al., 2010). Real-world translations of community-based EB programs, however, must conform to available resources, including staff. In Michigan, for example, the AAAs selected staff to provide the CCC program in their regions, necessitating the preparation of a diverse group of practitioners.

Train-the-Trainer Workshops

CCC established a rigorous 2-day train-the-trainer workshop, offered annually, as a requirement for prospective practitioners. These workshops were conducted by Sherman, a co-developer of SCP, and project director Steiner. They were held in a central part of the state and included a comprehensive review of all EB content, materials, and resources. Each train-the trainer workshop began with an overview of the development of the SCP, its theoretical basis, relevant outcome research, and guidance regarding best practice implementation models and process. This was followed by a comprehensive review and interactive discussion of all SCP curricula and materials. The program’s core components and program implementation, as illustrated on Table 1, were described explicitly throughout each training (NREPP, 2012). As ADSSP projects are required to measure participant outcomes, each training workshop stressed the importance of program data collection of participant demographics, program documentation, and reporting requirements.

The CCC’s practitioner training emphasized the central goals of fidelity, especially with respect to practitioner adherence, competence, and differentiation (Perepletchikova & Kazdin, 2005). Adherence is the degree to which practitioners deliver a program as intended by the program developer, and competence signifies the interpersonal, group management skills used in delivering the intervention (Breitenstein et al., 2010). Differentiation captures how the intervention differs from other programs along critical dimensions (Dobson & Singer, 2005). While adherence is understood as a priority for high-quality implementation, practitioner competence is equally critical to ensuring participant retention, participation, and engagement. Competence may be especially important for action and change-oriented components of an intervention such as SCP. For CCC, differentiating SCP from other caregiver groups has been essential, as many practitioners and participants have considerable experience with support groups, either as a leader or caregiver.

Program Provision

The SCP is designed to be delivered by a solo practitioner, an approach that provides internal consistency over 6 weeks, and is cost-effective for implementation. The dual responsibilities of adherence and competence, however, was supported by CCC, which encouraged all new practitioners to form peer partnerships to review SCP content and prepare for their first delivered programs. Newly trained practitioners met with experienced practitioners in pairs or regional teams to share delivery techniques and address fidelity issues or other problems. CCC instituted a system of in vivo peer observation and mentorship. This system of local peer support among practitioners embedded a valuable and feasible means of “front line” fidelity monitoring at the regional level (Breitenstein et al., 2010). Peer observation, fidelity checklists, and discussions of self-report reinforce EB training and encourage practitioners’ adherence through real-time, strengths-based observation of skills and potential areas for further development and training.

Sustaining Fidelity During Program Provision

Fidelity must be upheld at every phase of an EB program implementation. Following training workshops, the project director and lead SCP trainer [authors] facilitated monthly fidelity-focused teleconferences with practitioners. These calls addressed issues and questions regarding adherence, competence, differentiation and implementation. The conference call format provided a timely and efficient method to reach geographically dispersed CCC practitioners across the state. Review of SCP content, delivery and fidelity were provided by [author] as needed. Project requirements, such as marketing and program branding were discussed by [coauthor]. These discussions often covered technical aspects of program delivery, but practitioners were also encouraged to share their insights into the effects of the EB program on participants, and their own experiences as service and program providers.

As the project’s scope expanded, teleconferences engaged specific regions, new practitioners, or addressed topics requested by practitioners, such as recruitment, eligibility, reporting, and implementation. Constructive, program-building discussions among practitioners provided a platform to resolve fidelity issues and other barriers to success. A frequent topic involved recruitment of eligible caregivers. AAAs do not publicize the services they support. Moreover, the target audience for SCP is broader than many aging programs: family caregivers of a person with dementia living in the community, regardless of age or income. Thus, project-wide discussions focused on alternative forms of publicity, new to the AAAs, to recruit eligible participants for the CCC program, including radio, television, and various print media (McCurry et al., 2015). The teleconferences supported crucial real-time discussions of fidelity concerns, and provided an opportunity for more experienced practitioners to share and discuss their skills regarding time and group management. Overall, these conversations enhanced the collective skill set of the CCC practitioners, and provided an additional indicator of practitioner engagement and skill development.

Accommodating Program Adaptations

Adaptations often arise in the delivery of an EB program and thus require serious consideration regarding their adoption (O’Connor, Small, & Cooney, 2007). Implementation with fidelity requires managers and practitioners to differentiate between changes to the EB content that compromise program fidelity, known as “drift,” from adaptations that enhance an EB implementation. Any adaptation proposed by CCC practitioners was assessed based on feedback from participants, seasoned practitioners, and approval from SCP co-developers. Adaptations deemed to enhance comprehension or engagement were incorporated into the ongoing training and discussions with active practitioners. Project-wide discussions provided opportunities to promote a coherent understanding and discernment of fidelity across the statewide CCC implementation of SCP.

For example, in the first phase of implementation, some practitioners wanted to add inspirational poems to the SCP sessions, while others wanted to omit exercises in the SCP curriculum that they deemed cumbersome. These alterations to the EB program were recognized as program drift and were strongly discouraged in all CCC fidelity trainings and discussions. On the other hand, when practitioners proposed methods for gathering participant input during an exercise that highlighted the larger theme and impact of the exercise, the method was adopted as a valuable adaptation that upheld content and enhanced caregiver participation.

Progressive Training for Experienced Practitioners

A sustainable program fidelity framework benefits from ongoing participation and leadership development of experienced practitioners. Over time, experienced practitioners assume greater responsibility for fidelity at the regional level. As statewide implementation became established, CCC provided specialized training and professional development opportunities for practitioners to enhance their delivery and mentoring skills. Practitioners at these events generally raised higher order concerns and expressed greater understanding and commitment to program fidelity.

Refresher training workshops

Refresher workshops were held annually to maintain program delivery skills among practitioners who had become inactive or were seeking to improve their delivery of SCP with fidelity. These meetings provided a shared platform for practitioners to remain current with respect to evolving implementation questions and to learn from other, more experienced practitioners. Practitioners who had not offered SCP for 1 year were required to attend a refresher workshop prior to providing the program again.

Master trainer certification

The master trainer certification was developed to recognize practitioner skill development. As practitioners gained expertise in delivery of SCP and implementation, they were able to apply to become certified as CCC master trainers. Master trainers status represented practitioners’ achievement of skill development, program fidelity, and empowered them to serve as regional leadership for the implementation.

As part of the certification process, the program documentation, program observation, and participant evaluations for a prospective master trainer were reviewed. If this initial review was satisfactory with respect to fidelity and upholding program requirements, prospective master trainer candidates completed a series of essays on SCP content and CCC implementation. Essays were reviewed and evaluated for knowledge and fluency of SCP, ability to explicate the role of fidelity, detail threats to fidelity, and whether or not they reflected the person-centered, empowerment, and coaching perspectives of SCP. All reviewers of the essays were blinded to identities of the candidates. Approximately a quarter of all applicants did not pass the master trainer certification the first time. In these cases, practitioners were given additional training support and were able to apply a second time after providing more SCP programs. The review was initially conducted by the project team and a panel of commissioners from the Michigan Commission on Services to the Aging, the certifying authority. As a cadre of master trainers expanded, they participated in the reviews of prospective candidates as well. The multiple layers of reviewers provided distinct perspectives to the certification process. At present, CCC master trainers are certified for 2 years, and the Commission reviews new certifications twice a year.

Master trainer professional development

A key aspect of being a master trainer is to assume a leadership role in the CCC. To promote leadership skills and consistency across the state, all master trainers are invited to annual professional development events with the AASA executive director and administrators. These meetings have featured presentations by leading academics, and professional development sessions, as well as program discussions. Topics often include reaching diverse caregivers, current research on Alzheimer’s disease and related dementias, and professional coaching. These workshops enable program leaders and practitioners to engage in higher level discussion of current issues and help promote a coherent understanding and commitment to providing SCP with fidelity across the state.

As of 2016, the CCC trained a total of 106 practitioners, 85 of whom were active throughout the ADSSP-funded implementation, and 65 of whom are currently providing the SCP program as part of Michigan’s CCC. Fifteen practitioners are currently certified as CCC Master Trainers. Twelve of these master trainers are AAA staff, seven of whom have CCC in their job description. This level of agency adoption reflects the value AAAs place on the program.

Program Data and Results

Between 2008 and 2015, AASA provided more than 500 ADSSP-funded SCP programs to 4,732 caregiver participants (see Table 2). This table offers a useful descriptive summary of the number of trainers, programs offered, and participants by each AAA regions involved in the CCC implementation. Clearly, there is variation between the regions. As stated earlier, AAAs set their own program and service goals based on their capacity and need. The number of programs offered varied based on a number of factors, including population base, capacity of the region’s aging services, and number of dedicated practitioners. In real-world implementation, such variation exists. Thus, consistent and integrated fidelity training is required to assist regions to meet their own goals.

Table 2.

Descriptive Summary of Michigan CCC-SCP Program Implementation (2008-2015).

AAA regions	Trainers	Master trainers	Programs (n)	Participants (n)	Completers (4+ sessions)
1-A	10		30	303	213
1-B	14	2	72	820	615
2	8		23	186	157
3-A	2	1	7	61	51
3-B	3		4	31	26
3-C	4		6	45	37
4	6	2	35	328	277
5	6		40	391	305
6	8	2	44	406	315
7	16		58	497	405
8	3		8	53	50
9	11	2	62	517	427
10	8	2	46	350	271
11	5	4	67	526	428
14	5		28	314	254
Total	109	15	530	4,722	3,831 (81%)

Note. CCC = Creating Confident Caregivers®; SCP = Savvy Caregiver Program; AAA = Area Agencies on Aging.

Monitoring program fidelity through data on each program component helps measure, achieve, and sustain desired outcomes (Breitenstein et al., 2010). All ADSSP-funded projects were required to incorporate program documentation and participant outcomes. Thus, CCC gathered participant demographic information, service delivery information, and participant outcomes via pre- and post-surveys, and evaluations of program content, materials, and practitioner group management skill.

The CCC database, managed by AASA, records each program by participant, practitioner, and region. Using a process similar to practice based research (Epstein, 2001), data gathered through demographic forms, attendance logs, and participant program evaluations are reviewed for evidence of program fidelity and practitioner skill in delivering the EB program as intended. Both the project director and data coordinator review all documents and read written comments prior to data entry. Program information includes the date and location of sessions, attendance logs, practitioner, and group size. Demographic information provides caregiver and person with dementia data (e.g., age, gender, and relationship to the care recipient, duration of caregiving). Participant evaluations assess program site, materials, leader, relevance, and overall satisfaction with the program. Fidelity monitoring and maintenance occurs through the integrated and iterative review of program data, and helps guide ongoing EB delivery training.

Participant Evaluations

The participant’s voice is an important component in the fidelity framework. As a statewide project serving nearly 5,000 participants, the CCC project team has reviewed thousands of participant data elements. Careful attention to both program ratings and participant comments is warranted, as these data offer valuable confirmation of, or challenge, the degree of program consistency and fidelity from across the implementation.

In CCC, nearly all participants (98.8%) stated they would recommend the SCP to another dementia caregiver. Such enthusiastic endorsement of an intervention is gratifying for practitioners and policy makers alike; positive word-of-mouth recommendation from a trusted peer is a leading source of referral. Program approval among participants was consistently high, with an average score of 4.8 on a 5-point scale (1 = never to 5 = always). Specific items pertaining to engagement and program delivery received similarly high rating (4.52-4.73). High scores, however, are not a given. Lower scores given for any practitioner, especially if seen in more than two programs, were often a “red flag” that alerted regional fidelity monitors to address problems of skill or program delivery.

Administrative Review

Program documents were carefully reviewed, and any data trends were analyzed and noted as performance measures. CCC programs have served caregivers from all 83 counties and all 16 AAA regions. Women represented 80% of the participants, with adult children exceeding spouses as caregiver participants. The population served by the CCC implementation of SCP mirrors the larger diverse population of the state.

Participant evaluations often included written comments, addressing program impact, practitioner competence, and program delivery. This qualitative database was analyzed separately for evidence of SCP content and group dynamics. Comments often referenced core concepts from the SCP materials or the “coaching” approach that SCP embodies. Caregivers acknowledged positive changes in their caregiving as a result of participating in SCP. As one caregiver wrote, “After just two sessions, I had the best week with my mother!” Another comment illustrates how program adherence to the intervention goals, that is, enhanced knowledge of dementia, caregiver skill, and confidence, can be determined from participant comments:

Before I started, I was worried, upset and unsure of how I was going to help and take care of (my husband). I now feel relieved and confident. I am relieved because I have knowledge of what I need to do and a better understanding of what he is going through. (Emphasis added)

Another caregiver wrote:

I have found that understanding dementia and learning about contented involvement were a great impact on my continuing caregiving. Taking care of myself and knowing confusion and fear impact the relationship between me and my person. (Emphasis added)

The trainer presented the material clearly, made sure to engage us in the exercises, and encouraged us to practice our new skills at home with our family members.

Comments referring to SCP concepts and objectives, that is, increased caregiver self-care, understanding of dementia, and activities of contented involvement for the person with dementia, are indicators that core concepts have been transmitted.

Of course, a certain number of comments signaled concerns regarding fidelity. For example, comments revealed the participant is not in the program’s target group or that the practitioner “understood that we didn’t really want to discuss the decision making exercise.” Reviewing the entire database of written comments, blind to practitioner, allowed the project team to discern variations and identify normative comments for all practitioners. Some comments, while positive in tone, can signal an approach that indicates lack of fidelity.

Participant Outcomes

Measuring an EB program’s effectiveness offers validation of the implementation’s level of fidelity. During the entire ADSSP grant period, participants received project demographic forms before the start of their program and program attendance logs were maintained; 88% of attendees (4,185) provided a demographic form and 79% (3,753) attended at least 4+ sessions of the total six SCP sessions. While these forms indicate participant engagement, participants in programs held from January 2009 to December 2013 also received a caregiver pre-survey designed to collect caregiver outcome measures. Completion of these surveys was encouraged but not required, and no compensation was provided for survey completion. The parallel caregiver post-surveys were mailed directly to eligible participants 6 months post-program. Post-survey eligibility included attendance at 4+ sessions and a completed pre-survey. During the time frame, AASA received 3,133 participant forms, of which 2,403 were then sent post-surveys. Of the 730 ineligible participants, 406 had completed only the demographic form (i.e., not the entire pre-survey), 231 had not attended 4+ sessions, and 93 had other factors, such as inability to leave their person alone at home (13%), care recipient illness (13%) or facility placement or death of the care recipient (13%). In all, AASA received 1,815 caregiver post-surveys, a return rate of 76% from among eligible participants.

Participants in the CCC implementation who completed both pre- and post-surveys reported enhanced caregiver skills, outlook, and self-care, the intended goals of SCP. Participants reported significantly increased life satisfaction and reduced sense of feeling overwhelmed. They reported significant improvement in their ability to meaningfully engage with their relative with dementia and respond calmly and appropriately to disruptive behaviors. With respect to enhanced outlook or attitudes, caregivers reported reduced expectations of the person with dementia, and significantly increased acceptance of the disease, ability to focus on the present, and sense of having learned life lessons as a result of their caregiving experiences (Steiner & Sherman, 2015).

Managing a chronic, degenerative condition, such as Alzheimer’s disease or dementia, is a challenging task. Documenting participants’ gains in caregiver well-being, skills, and attitude is encouraging given disease progression and that postintervention data were collected a full 6 months post-intervention. That these outcomes mirror the original SCP randomized control trial findings, affirms CCC’s implementation fidelity to SCP.

Sustaining Practitioner Engagement

Providing a variety of practitioner engagement opportunities and administrative review all contribute to establishing and upholding program fidelity. Nonetheless, challenges will arise in any large-scale community-based implementations of an EB program. For example, turnover of practitioners or community partners is likely to occur and may challenge an agency’s ability to meet its intended service goals. To address this issue, CCC’s annual train-the-trainer workshops replenish the pool of active practitioners and annual refresher workshops, introduced in 2012, help maintain cohesion, engagement, and adherence among established practitioners. Stakeholders receive timely updates on project findings via project briefs and presentations. Together, these measures have cultivated broad statewide awareness and support, and maintain the project’s progress and evolution over time.

As noted in Table 2, several regions trained many practitioners while a few regions have highly stable program teams. Providing an EB program over 6 weeks requires support and dedication. In Michigan, we found two unexpected sources of reward: practitioners who adhered to the EB guidelines reported seeing improvements repeatedly among the caregiver participants over the course of the 6-week sessions. In addition, trainers reported that the project provided a level of professional enhancement that recognized their skills and contribution (Author, 2015). Participants in many aging programs present with difficult, chronic, and comorbid issues. Since SCP’s content is designed to empower caregivers with knowledge and skills, practitioners noted improvements in participants’ appearance, posture, and social interactions over the 6 weeks of SCP. Such notable positive changes may be unexpected and hard to quantify, but practitioners’ observations of visible and marked changes in participants has been seen as a “reward” for following fidelity. As one master trainer states, “Once you trust the program, positive results are evident.” Master Trainers have become regional leaders on dementia care issues, and have been asked to contribute to statewide policy and programs on dementia care. The ability to witness caregiver improvement and be recognized as a leader in dementia care and an EB psychoeducational intervention enhances practitioner engagement.

Training Modifications: Incorporating Lessons Learned Into the Fidelity Cycle

CCC uses an iterative approach to program implementation, with the EB program elements as the foundation and draws on practitioners’ and participants’ input to further define and refine fidelity. Throughout 7 years of SCP project implementation, “lessons learned” were incorporated into subsequent trainings, meetings, or teleconference calls. All elements of program delivery were included, from marketing techniques, eligibility determination, to practitioner group management skills. Successes and failures were shared during training events, further deepening an understanding and shared sense of ownership of the program’s structure and fidelity at all levels of the implementation.

Fidelity begins with who is in the room. As with other EB programs, SCP sets clear guidelines regarding who qualifies as the target audience. In CCC, the AAAs are responsible for their programs, practitioners, and attracting eligible participants. The shift of these responsibilities resulted in registration of 50 participants (e.g., professional caregivers) who did not meet the target audience, violating an element of fidelity. Moreover, given their very different role and relationship to the people in their care compared to family members providing care for a relative with dementia, CCC practitioners counseled the ineligible participants out of the program and referred them to more appropriate resources.

Addressing Fidelity Concerns

As mentioned earlier, “drift,” or diversion from an EB intervention protocol or content, is a common threat to fidelity (Bellg et al., 2004; Breitenstein et al., 2010). Broad community-based implementation efforts may be especially vulnerable to such issues, given the range and number of practitioners and remote program settings. In addition, fidelity can be challenged when practitioners allow themselves or participants to shift into a different modality, such as support group mode, rather than SCP’s EB’s psychoeducational approach. Yet another form of drift can arise when practitioners, trained as nurses or social workers, default to their professional expertise and methodologies to solve problems for the caregiver participants. Practitioners who revert to familiar modalities, such as case management, do not deliver SCP as intended and deprive participants of the opportunity to develop their own caregiving skills and strategies.

To avoid or address such concerns, CCC training workshops and fidelity-focused discussions incorporated exercises that enabled practitioners to identify their own personal and/or professional attitudes, biases, and “blind spots” that might compromise their delivery of the EB program. CCC provided additional consultation with experienced practitioners when a practitioner’s training or personal experience, including their own family caregiving, affected delivery of the EB content.

If and when additional training and assistance failed to improve fidelity among a trainee, the AAA staff worked, with project support, to inform the practitioner that he or she did not appear suited to deliver this particular EB program. Actions to “counsel out” a practitioner in this way occurred once in six of the 14 AAA regions that offered CCC-SCP. Although every attempt should be made to avoid this outcome, counseling out practitioners, when necessary, signals commitment to upholding overall fidelity among all practitioners and partners (Borrelli, 2011; Breitenstein et al., 2010; Carroll et al., 2007; Dobson & Singer, 2005; McCallion et al., 2008).

Sustainability of CCC’s Implementation

A desired outcome of community-based EB program implementation is that the program is sustained beyond the original funding is completed. In Michigan, AASA has sustained CCC after the conclusion of ADSSP funding in response to AAAs request. In a critical step toward sustainability, the Governor-appointed Commission on Services to the Aging approved the CCC service definition for AAAs in 2015. This service definition allows AAAs to use federal funds for the program (e.g., Title III-E, III-D) and the service standard retains the fidelity elements and documentation of the program. As evidence of the sustainability of CCC in 2015-2016, 10 AAAs provided CCC programs to serve more than 500 participants. Nine master trainers attended the 2016 learning session to further enhance and expand their support of SCP fidelity. AASA continues to collect all program and participant documentation and monitor fidelity for state and federal caregiving reports.

Adapting the CCC Framework for Other EB Programs

While every implementation effort will have its own resources and barriers, EB programs can be effective if fidelity is defined, discussed, and monitored. In the CCC community-based implementation of SCP, Michigan has been able to offer a proven intervention to family caregivers and engaged a cadre of committed and thoughtful practitioners. In the Michigan implementation, we found the 2-day in-person SCP trainings and project-wide fidelity monitoring conference calls to be especially critical to establish a solid implementation team that continues to be invested in upholding their deep understanding and appreciation for fidelity. The time invested in development of the CCC practitioners was, much like SCP, person-centered and empowerment oriented. As such, the training cycle resulted in a shared sense of responsibility for fidelity across the project. Although we found the refresher and other training opportunities to be enormously fruitful, these two aspects of practitioner training created both a solid foundation and opportunities for ongoing and deepening discussion of fidelity in theory—and in practice. Project-wide or regional discussion of challenges and questions regarding fidelity could be addressed in a timely manner and across all project levels. The level of shared understanding and commitment to fidelity was surely been one of the key components of success in CCC’s implementation.

Others have noted that an iterative loop of program information is vital to developing and maintaining program fidelity (McCallion et al., 2008). As such, the CCC fidelity-focused training cycle relies on clear expectations, ongoing training, and skill development, supported by the feedback loop of program information. While program data provides valuable insight into practitioner and content variation, fidelity is multimodal. The practitioner must be engaged in the process of skill improvement to enhance fidelity and outcomes. Throughout this process, practitioners also need to be respected and supported. Overall, this fidelity training framework is amenable to specific adaptations to suit the training requirements of an EB program and to meet the needs of participating agencies or organizations, and enable agencies and practitioners to be explicit, proactive, and intentional in its effort to deliver an EB program with fidelity.

Finally, direct and continuous engagement of regional and state stakeholders who appreciated the benefits to caregivers across the state has garnered support for utilization of federally allocated funding to sustain the CCC implementation of SCP in Michigan. CCC has integrated all of these methods to embed SCP as an additional EB program, and has established an accepted standard and mechanism of best practice throughout the state.

Conclusion

An essential translational ingredient that is rarely explicated is the establishment of a systemic, sustainable fidelity framework (Frank et al., 2008). The explicit aim of the CCC fidelity-focused training framework is to ensure the highest caliber delivery of the EB intervention, SCP, enhance dementia capacity within the aging network, and support family members providing dementia care. As such, the CCC’s integrated and sustainable program serves as a successful and viable example of a sustainable fidelity implementation model.

Footnotes

Authors’ Note

Grantees carrying out projects under government sponsorship are encouraged to express freely their findings and conclusions. Therefore, points of view or opinions do not necessarily represent official Administration on Aging (AoA), Administration for Community Living (ACL), or U.S. Department of Health and Human Services (DHHS) policy.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported in part by cooperative agreements (#90AE0322 and #90AE0341) from the Administration on Aging, Administration for Community Living, U.S. Department of Health and Human Services.

Author Biographies

Carey Wexler Sherman, PhD, is an assistant research scientist at the Institute for Social Research, The University of Michigan. Her research program focuses on dementia family caregiving, remarriage and stepfamily in later life, and translational research. She is a co-developer of the evidence-based Savvy Caregiver Program.

Sally C. Steiner, LMSW, is a departmental specialist at the State of Michigan’s Aging & Adult Services Agency, Michigan Department of Health and Human Services. She specializes in dementia services, dementia caregiving, and mental health. She was project director for the federal Alzheimer’s Dementia Supportive Services Program grants and currently administers the Creating Confident Caregivers® program.

References

Bellg

A. J.

Borrelli

Resnick

Hecht

Minicucci

D. S.

Ory

. . . Czajkowski

(2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH Behavior Change Consortium. Health Psychology, 23, 443-451.

Borrelli

(2011). The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. Journal of Public Health Dentistry, 71(Suppl. 1), S52-S63.

Breitenstein

S. M.

Gross

Garvey

C. A.

Hill

Fogg

Resnick

(2010). Implementation fidelity in community-based interventions. Research in Nursing & Health, 33, 164-173.

Carroll

Patterson

Wood

Booth

Rick

Balain

(2007). A conceptual framework for implementation fidelity. Implementation Science, 2. Retrieved from http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-2-40

Dobson

K. S.

Singer

A. R.

(2005). Definitional and practical issues in the assessment of treatment integrity. Clinical Psychology: Science and Practice, 12, 384-387.

Epstein

(2001). Using available clinical information in practice-based research: Mining for silver while dreaming of gold. Social Work in Health Care, 33(3-4), 15-32.

Estabrooks

P. A.

Glasgow

R. E.

(2006). Translating effective clinic-based physical activity interventions into practice. American Journal of Preventive Medicine, 31(4, Suppl.), S45-S56.

Frank

J. C.

Coviak

C. P.

Healy

T. C.

Belza

Casado

B. L.

(2008). Addressing fidelity in evidence-based health promotion programs for older adults. Journal of Applied Gerontology, 27, 4-33.

Hepburn

Lewis

Sherman

C. W.

Tornatore

(2003). The Savvy Caregiver Program: Developing and testing a transportable dementia family caregiver training program. The Gerontologist, 43, 908-915.

10.

Hepburn

Lewis

Tornatore

Sherman

C. W.

Bremer

K. L.

(2007). The Savvy Caregiver Program: The demonstrated effectiveness of a transportable dementia caregiver psychoeducation program. Journal of Gerontological Nursing, 33(3), 30-36.

11.

Maslow

(2012). Translating innovation to impact: Evidence-based interventions to support people with Alzheimer’s disease and their caregivers at home and in the community. Retrieved from http://www.aoa.acl.gov/AoA_Programs/HPW/Alz_Grants/docs/TranslatingInnovationtoImpactAlzheimersDisease.pdf

12.

McCallion

Ferretti

L. A.

Brick

(2008). Effective practice standard (EPS): Program fidelity as quality assurance/Quality improvement (NCOA/ASA Joint Conference Presentation). Washington, DC. Retrieved from http://www.nacddarchive.org/nacdd-initiatives/arthritis/McCallion_Ferretti_July28_Fidelity_EPSrevised.pdf

13.

McCurry

S. M.

Logsdon

R. G.

Mead

Pike

K. C.

La Fazia

D. M.

Stevens

Teri

(2015). Adopting evidence-based caregiver training programs in the real world: Outcomes and lessons learned from the STAR-C Oregon translation study. Journal of Applied Gerontology, 1-18. Retrieved from: http://jag.sagepub.com/content/early/2015/04/10/0733464815581483.full. doi:10.1177/0733464815581483

14.

Menne

H. L.

Bass

D. M.

Johnson

J. D.

Kearney

K. R.

Bollin

Teri

(2015). Program components and outcomes of individuals with dementia: Results from the replication of an evidence-based program. Journal of Applied Gerontology,1-16. Retrieved from: http://jag.sagepub.com/content/early/2015/05/28/0733464815591212.full. doi:10.1177/0733464815591212

15.

Mowbray

C. T.

Holter

M. C.

Teague

G. B.

Bybee

(2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315-340.

16.

National Council on Aging. (2006). About evidence-based programs. Retrieved from https://www.ncoa.org/center-for-healthy-aging/basics-of-evidence-based-programs/about-evidence-based-programs/

17.

National Registry of Evidence-based Programs and Practices. (2012). A road map to implementing evidence-based programs. Retrieved from http://www.samhsa.gov/nrepp

18.

O’Connor

Small

S. A.

Cooney

S. M.

(2007). Program fidelity and adaptation: Meeting local needs without compromising program effectiveness. University of Wisconsin–Extension. Retrieved from http://fyi.uwex.edu/whatworkswisconsin/files/2014/04/whatworks_04.pdf

19.

Perepletchikova

Kazdin

A. E.

(2005). Treating integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12(4), 365-383.

20.

Perepletchikova

Treat

T. A.

Kazdin

A. E.

(2007). Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75, 829-841.

21.

Samia

L. W.

Aboueissa

Halloran

Hepburn

(2014). The Maine Savvy Caregiver Project: Translating an evidence-based dementia family caregiver program within the RE-AIM framework. Journal of Gerontological Social Work, 57, 640-661.

22.

Smith

S. A.

Bell

P. A.

(2005). Examining the effectiveness of the Savvy Caregiver Program among rural Colorado residents. Rural and Remote Health, 5, Article 466. Retrieved from http://www.rrh.org.au

23.

Steiner

Sherman

C. W.

(2015). Creating Confident Caregivers: Michigan’s Expansion Project. Final ADSSP Performance Report to Administration for Community Living.