Abstract
Introduction:
Yoga for treatment of worry in older adults is an intervention that is especially likely to translate into real-world practice. Assessing treatment fidelity improves confidence that effective interventions can be consistently applied and allows researchers to explore if any null results for effectiveness are indeed the result of a lack of intervention efficacy or lack of proper intervention implementation.
Methods:
This study describes treatment fidelity of a yoga intervention in a randomized preference trial that compared cognitive–behavioral therapy (CBT) and yoga for the treatment of worry, anxiety, and sleep in worried older (≥60 years) adults. Established methods for assessing treatment fidelity of CBT guided the procedure for ensuring that the yoga intervention was delivered as intended. The yoga intervention consisted of 20, 75-min, in-person, group, gentle yoga classes held twice weekly.
Results:
Six female instructors (mean age = 64 years) taught 660 yoga classes that were videotaped. Ten percent of these classes, stratified by instructor, were randomly selected for review. The average adherence score for yoga instructors was 6.84 (range 4–8). The average competency scores were consistently high, with an average score of 7.24 (range 6–8). Teaching content not included in the protocol occurred in 26 (38.1%) sessions and decreased over time. Observed ratings of instructor adherence were significantly related to ratings of competency. Instructor adherence was also significantly associated with lower participant attendance, but not with any of the other process or outcome measures.
Conclusions:
The larger range found in adherence relative to competence scores demonstrated that teaching a yoga class according to a protocol requires different skills than competently teaching a yoga class in the community, and these skills improved with feedback. These results may foster dialog between the yoga research and practice communities.
Clinical Trial Registration No.: NCT 02968238.
Introduction
Older adults prefer psychotherapy over pharmacotherapy for treatment of anxiety. 1 Thus, alternatives to medication are needed. Worry has been conceptualized as a coping strategy to reduce the experience of anxiety. 2 Cognitive–behavioral therapy (CBT) is the most efficacious nonpharmacological treatment for worry 2 and currently has the strongest evidence base for treating late-life worry. 3 An emerging evidence base 4,5 suggests that yoga is another nonpharmacological intervention that may reduce late-life anxiety.
Although the research evidence is still emerging, yoga is already widely used by 13.2% of the general population in the United States, and this number is increasing. 6 Furthermore, yoga is generally easy to practice and maintain. 4 Therefore, yoga as an intervention to reduce worry should be evaluated because in addition to promising data, yoga can be translated into real-world practice. 7,8 Yoga is affordable, commonly implemented in group classes that take limited instructor time, requires little additional instructor training to be customized for the target population, and is already accessible in most communities. Taken together, these elements increase the likelihood that yoga can be a self-sustaining practice.
One outcome used to evaluate successful intervention implementation into practice is treatment fidelity. 9 Treatment fidelity is “the degree to which an intervention was implemented as it was described in the protocol or as it was intended…” (p. 69). 9 Assessing treatment fidelity when translating interventions to real-world settings allows researchers to determine if a null result for effectiveness is the result of a lack of intervention efficacy or lack of proper implementation. 9,10
The current study focuses on establishing the treatment fidelity of a yoga intervention delivered in a randomized preference trial that compared the effectiveness of CBT with yoga for treatment of worry, anxiety, and sleep in older (≥60 years) adults with elevated worry symptoms. 11 The objectives are to (1) describe the results of treatment fidelity assessments from this trial (n = 250 in the yoga groups); (2) summarize the topics reviewed with instructors to enhance fidelity over time; and (3) examine if treatment fidelity was associated with participants' process measures (expectancy ratings, satisfaction, and adherence) and the primary study outcome (worry).
Methods
This study was part of a trial approved by the Wake Forest School of Medicine Institutional Review Board and registered as a clinical trial. Participants provided written informed consent for the study procedures.
A treatment fidelity plan was developed to ensure that the yoga intervention was robust and delivered as intended. 12 Recommendations by the Treatment Fidelity Workgroup of the NIH Behavior Change Consortium 10,13 and other investigators were incorporated, 14,15 including those related to treatment design, training, treatment delivery, and treatment receipt. Yoga dose and delivery; yoga style/components; modifications; instructor selection; home practice; and fidelity measurement were also systematically considered. 12
Dose and delivery of yoga
The yoga intervention consisted of 20, 75-min, in-person, group, gentle yoga classes held twice weekly. Participants were given options of class times and locations and entered yoga classes on a rolling basis. Instructors generally taught at the same times, which gave participants the option to have a consistent instructor. Participants were also asked to complete a brief home practice for at least 15 min ≥5 days per week. Additional protocol details have been published. 11
Style, components of the yoga intervention, and specific class sequences
Study yoga classes were based on the Relax into Yoga approach, which is designed to be safe and effective for seniors 16 and is not affiliated with a specific yoga style. 17 Regarding the class environment, instructors avoided background music for those who may have difficulty hearing and incense/scented candles for those who may have breathing conditions. 18 Furthermore, students were able to participate either on the floor (on a mat) or a chair based on comfort and physical ability. Adequate support and time were given for transitions between physical postures. 18 Yoga classes began the practice more actively to match a worried state of mind and then gradually slowed down the practice, ending with meditation to facilitate relaxation.
The main components of the yoga intervention included yogic principles, centering and breathing, physical movements, and meditation/relaxation (Table 1).
Yoga Session Self-Evaluation and Observation Rating Form
Specific movements to Relax into Yoga; gentle standing twist added as an optional pose when teaching for this study; italic text indicates optional practices.
Yogic principles
Krucoff et al. described practice principles for yoga with older adults (e.g., create a safe environment and emphasize feeling over form). 16,17
Centering and breathing (10 min)
In addition to a centering and breathing practice, all movements were taught with awareness of breath.
Physical movements (50 min)
Instructors were asked to begin with the simplest version of a pose and suggest that students could work up to more complicated options only if they were interested and able to do so. See Table 1 for specific movements (also described in a published book). 16
Meditation/relaxation (15 min)
Experientially, yoga experts report that people with worry and anxiety symptoms can find meditation challenging initially and physical postures can help to release some of the agitation to set the stage for meditation and relaxation. 19 Instructors included awareness cues (e.g., pause and notice) throughout the class. In addition, guided meditation was used to focus attention on the present moment and away from worrying thoughts. 19
The yoga intervention also intentionally excluded the following, as recommended by a consensus statement for yoga components essential to avoid when instructing those with anxiety: breath holding after inhalation or rapid breathing techniques; meditation practices without a specific focus; yoga done in heated, crowded, or enclosed spaces; and practices that emphasize ability, accomplishment, or competition or require complex instructions. 20
Dealing with modifications
Yoga instructors were trained to offer modifications. Class size was capped at 10 so that instructors could provide adequate attention to participants. To ensure safety for older adults with osteoporosis, seated and standing forward bending and extreme twisting were avoided. 17
Selection of instructors
Proper training by a skilled and qualified yoga instructor is essential for the safe and effective use of yoga. 12,18,21 All study yoga instructors were required to have ≥200 h of yoga training and ≥1 year of yoga teaching experience. Eligible instructors were provided with 16 h of study-specific training. C.K. was the primary trainer. Each yoga instructor demonstrated her knowledge of content by teaching in front of the instructor at the training. They were also given the trainer's book, 16 a recording of a sample study class, and online materials to support consistent implementation of the curriculum.
Facilitation of home practice
Participants were given home practice guidelines such as wearing comfortable clothing and practicing in a quiet place. Home practice recordings were provided on a CD, and questions about home practice were discussed at each class.
Measurement of intervention fidelity over time
In addition to the study-specific training, instructors also completed a self-evaluation checklist for each class (Table 1), tracked class attendance, and reported any deviations from the protocol. Furthermore, all yoga classes were videotaped and 10% were randomly selected, stratified by instructor, and reviewed by an investigator (Dr. Sohl) who was not involved in other study activities. Videos that were cut off before the end of a class were replaced with the next closest dated video for that instructor. Dr. Sohl is a trained yoga instructor in addition to her doctoral degree in health psychology. She used the self-evaluation checklist to inform scoring of adherence and competence measures adapted from CBT interventions. 22,23 Yoga instructors then met regularly with Dr. Danhauer to discuss the evaluation and facilitate consistency among classes.
Measures
Instructor adherence and competence
A measure was adapted to assess both instructor competence and adherence in the delivery of intervention skills and an overall adherence rating, “What was the yoga teacher's overall adherence to the treatment manual during this session?” (0 no adherence to 8 optimal adherence), and competency, “What was the overall competency of the yoga teacher during this session?” (0 none to 8 excellent). Adherence ratings were informed by behavioral anchors and rater completion of the checklist (Table 1). Example behaviors to consider were also provided to inform competency ratings (e.g., the yoga teacher focused on students or the yoga teacher maintained a balance between meeting the students' needs and structuring the session within the protocol guidelines). If any mean adherence scores were <6 instructors would have undergone retraining. 22,23
Participant process measures
Expectancy ratings were assessed with the Expectancy Rating Scale, 24 a measure of beliefs (e.g., how logical does the treatment seem and confidence in undergoing treatment) assessed after the first yoga class attended by the participants. Satisfaction with treatment was measured with the 8-item Client Satisfaction Questionnaire (higher scores indicate higher satisfaction) immediately after completing the 10-week intervention. 25 Attendance was also assessed (the number of completed classes, max = 20).
Participant primary study outcome
Worry was assessed with the Penn State Worry Questionnaire-Abbreviated, an 8-item measure of worry frequency and severity as the primary endpoint, immediately upon completing the intervention. Higher scores represent more severe worry. 26,27
Participant demographics
Age, gender, and race were assessed by self-report to describe study participants.
Analyses
The number and percent of videos selected; videos reviewed by observation, with reasons why they were not viewed; and self-reported evaluation forms completed by instructors were described. The average instructor adherence and competency scores, variability in scores over time, frequency of ratings <6, and frequency of teaching of skills that were not included in the protocol were also calculated. Strengths and weaknesses reviewed at regular meetings to enhance fidelity and prevent skill drift over time were qualitatively summarized.
Ordinal logistic regression was conducted, accounting for repeated measures on instructors, to examine whether instructor adherence was associated with instructor competence and participant characteristics. Linear models that account for repeated measures on instructors and participants were used to evaluate associations between instructor adherence and participant process and outcome measures (expectancy ratings, satisfaction, attendance, and worry). All quantitative analyses were run using SAS software (version 9.4, Cary, NC).
Results
Treatment fidelity over time
Six female instructors (mean age = 64 years) taught 660 study yoga classes (May 2017–February 2019). A total of 63 videos across six instructors were reviewed [N (%) of sessions taught that were rated: instructor 1: 11 of 132 sessions (8.3%); instructor 2: 1 of 20 sessions (5.0%); instructor 3: 28 of 253 sessions (11.1%); instructor 4: 10 of 119 sessions (8.4%); instructor 5: 11 of 109 sessions (10.1%); and instructor 6: 2 of 18 sessions (11.1%)]. Of the 660 self-reported evaluation forms expected, 14 were not completed (2.1%; n = 10 of 14 missing from the same instructor). Quarterly meetings with yoga instructors started out in person and then moved to conference call or in-person based on instructors' preference. Feedback was initially sent out exclusively as a group summary and then also sent individually through e-mail (per instructor's request).
The average adherence score was 6.84 (range 4–8), which was slightly higher at the last review session than the first (Table 2). Only seven adherence ratings were <6 (all from the same instructor; n = 1 for a score of 4 and n = 6 for a score of 5), and no competence ratings were <6. Feedback and scheduling changes were made to address this issue. The competence scores were consistently high with an average score of 7.24 (range 6–8). Additional skills not included in the protocol were taught in 26 of the 68 sessions reviewed (38.2%). Teaching of skills not included in the protocol decreased from the first to the last review (Table 2).
Treatment Fidelity Ratings Over Time
Numbers of classes reviewed in preparation for regular meetings with Dr. Danhauer were supplemented by a final round of review of 10 classes (Review 5) due to initially selected videos being cut off.
Topics reviewed with instructors
Strengths and weaknesses observed by the fidelity reviewer and any other issues that arose were discussed at regular meetings to enhance fidelity and prevent skill drift. Common instructor strengths included the following: provided clear guidance for adapting the postures safely; prompted awareness cues to encourage an appropriate level of effort; demonstrated kind presence and excellent rapport; used nonjudgmental language and great pacing; and coordinated movements with breath. Common instructor weaknesses included the following: were occasionally too directive; did not consistently lead transitions (e.g., from floor to chair); did not discuss home practice; instructed components out of order per protocol; and exhibited moments of distraction (e.g., completed documentation during relaxation).
Additional skills taught, which were not included in the protocol, were also discussed. Lessons learned from observing classes included that it was necessary to be needed to be more explicit about the importance of only teaching what was provided in the study material. For example, some instructors included guided imagery during relaxation and different breathing practices (e.g., pausing after inhaling, which is contraindicated for anxiety) or provided education on expected benefits of practices. An example of other issues reviewed with instructors was that some participants came late to class or left early, which was disruptive. Because of this discussion, letters were sent to participants asking them to arrive on time and not leave early and created signs to use once class started.
Association of instructor adherence and patient measures
There were 337 participants in the 63 classes reviewed (173 distinct participants). These 173 participants had a mean age of 66.2 (4.8), 88% were female and 88% white, which is similar to the demographic breakdown of the main trial. Instructor adherence and competence ratings were significantly associated (p < 0.0001; Table 3). There were no significant associations between instructor adherence to the protocol and participant demographics. As shown in Table 4, of the process and outcome measures examined here, only participant attendance was significantly associated with instructor adherence (p = 0.0382). Interestingly, lower instructor adherence was associated with better attendance, although there was only a slight difference in the number of classes attended (16 classes vs. 17.5 classes).
Associations of Instructor Adherence to the Protocol with Instructor Competence and Participant Characteristics
p-Values are from ordinal logistic regression models that account for repeated measures on instructors, with adherence as the dependent variable. Participants with multiple classes were allocated to the instructor from their initial class.
Association of Instructor Adherence with Participant Process Measures and Outcomes
p-Values are from linear models investigating the association between instructor adherence and participant process measures and outcomes (dependent variables) that account for repeated measures on instructors and participants.
LS, least squares; SE, standard error.
Discussion
This study demonstrated the adoption of guidelines for implementation of yoga interventions 12 in a real-world context while matching the rigor of studying treatment fidelity of a more established evidence-based intervention. 22,23 All of the yoga instructors were highly competent and competence ratings were significantly related to adherence ratings. Yet, the instructor adherence scores were more variable (i.e., they were observed to have a wider range) than competence scores.
These differences in scores suggest that teaching a yoga class with high adherence to a protocol requires different skills than competently teaching a community yoga class. For example, it was important that study yoga instructors be willing to be videotaped and observed, attend to the level of detail required for research reporting, and adhere to the structure required for teaching a manualized yoga intervention. Yoga instructors also needed to have some level of comfort with technology (e.g., video camera and study website) and be able to handle different work conditions at the community sites selected (e.g., churches, medical offices, and senior living facilities).
Therefore, it is important to convey the additional skills and flexibility needed when hiring yoga instructors to teach in a research study since some instructors may prefer to teach with more creativity, be uninterested in the additional documentation needed, or be more comfortable in a consistent physical location. It was helpful to explain the purpose of each of these research requirements to enhance engagement from instructors. Study staff frequently communicated with yoga instructors to collect class recordings and were available for any issues that arose, which likely contributed to the high level of adherence to fidelity documentation. Future studies could consider additional ways to ensure completeness of fidelity ratings from instructors (e.g., submitting documents electronically).
Additional training topics that emerged throughout the study included a standardized plan for handling disruptions and a discussion of professional boundaries (e.g., level of personal contact outside of class). Adherence to the protocol was significantly associated with lower participant attendance, but not with any of the other process or outcome measures studied. The association of adherence with attendance could have been because instructor adherence was lower at the beginning of the study before instructors received feedback, while participant engagement in behavioral interventions often drops off over time. 28
A meta-analysis that examined the association of interventionist adherence and competence scores with a variety of study outcomes also found no significant relationships. 29 Yet, there was significant heterogeneity in the studies assessed, and moderator analyses suggested that there may be a stronger association between competence and study outcomes, especially in studies that targeted major depressive disorder. 29 The lack of relationship with outcomes in this study and prior studies may have been due to the overall high level of adherence and competence, limiting the variability for assessing these associations. The current investigation is the first yoga study to assess the relationship between treatment fidelity and participant outcomes.
The larger, randomized preference trial revealed that one could not conclude that there was a difference between yoga and CBT. 30 The current study elaborates on how the yoga intervention was delivered as intended in a real-world setting. The knowledge that yoga and CBT were properly implemented raises more confidence in the results of the randomized preference trial. 30
Limitations and future directions
Since the current treatment design was based on an existing program, 16 elements listed in a consensus statement of components to include when teaching yoga to reduce anxiety were included in various degrees. 20 For example, abdominal breathing was considered very important or essential in this consensus statement and the current program instructed the three-part breath, which has abdominal breathing as a foundational component for this practice. Regarding two other very important or essential components, (1) focusing on exhalation was an optional breathing practice in the intervention and (2) humming bee breath (bhramari) was not included. In addition, some modifications made (e.g., option to teach a gentle standing twist) were not in the book. These limitations were perceived as minimal in the context of the high quality of the published Relax into Yoga program and relative ease for future dissemination. 16 Furthermore, observing video recordings is a time-intensive practice for ensuring ongoing treatment fidelity, and more cost-effective methods may be validated for evaluating fidelity in future studies. 31
Conclusions
Our results, which revealed a larger range in adherence versus competence scores, demonstrated that teaching a yoga class according to a research protocol requires different skills than competently teaching yoga in the community, and these skills improved with feedback. These data may foster dialog between the yoga research and practice communities by providing insight into the hiring, study management, and training practices needed to support yoga instructors' optimal intervention fidelity. This study also established a rigorous methodology for ensuring yoga fidelity in real-world settings. Future studies may consider more cost-effective and sustainable methods for fidelity evaluation.
Footnotes
Acknowledgments
The authors gratefully acknowledge the participants and yoga instructors for their contributions to the implementation of this study. Some results from this publication were accepted for presentation at the 41st Annual Meeting and Scientific Sessions of the Society of Behavioral Medicine planned for April 2020 in San Francisco, CA (conference canceled).
Authors' Contributions
S.J.S. was involved in conceptualization, methodology, data curation, project administration, and writing—original draft; G.A.B. was involved in conceptualization, methodology, funding acquisition, data curation, supervision, project administration, and writing—review and editing; C.K. was involved in methodology and writing—review and editing; G.H. was involved in data curation, project administration, and writing—review and editing; M.E.M. was involved in methodology, formal analysis, and writing—review and editing; A.A. was involved in formal analysis and writing—review and editing; and S.C.D. was involved in conceptualization, methodology, funding acquisition, data curation, supervision, project administration, and writing—review and editing.
Author Disclosure Statement
As coauthor of the book, “Relax into Yoga for Seniors,” C.K. receives royalties from the publisher, New Harbinger. The other authors have no conflicts of interest to report.
Funding Information
This study was funded by the Patient-Centered Outcomes Research Institute Program Award (CER-1511-33007; MPIs Brenes and Danhauer) and the Dean's Office of Wake Forest School of Medicine. The sponsor provided funds to complete the research. The funder was not involved in study design; collection, management, analysis, and interpretation of data; writing of the report; or the decision to submit the report for publication.
