Treatment fidelity in Model of Human Occupation research

Abstract

Introduction

Treatment fidelity is an important consideration in occupational therapy efficacy research. It involves methodological strategies that enhance quantitative rigor and improves confidence in research outcomes. This study examines treatment fidelity in Model of Human Occupation efficacy literature because it is a widely used, occupation-focused theory with a substantial evidence base.

Method

Relevant articles (n = 17) were coded for treatment fidelity with the Treatment Fidelity Measure. Percentages of articles reporting treatment fidelity strategies and mean adherence to fidelity strategies were calculated. One-way analysis of variance was used to examine fidelity over time.

Results

Only one study reported provider training strategies and measured nonspecific treatment effects. Overall mean adherence to treatment fidelity strategies for Model of Human Occupation research was 0.57; one study demonstrated high fidelity with a score of ≥0.80. Model of Human Occupation efficacy literature trends toward higher levels of treatment fidelity over time.

Conclusion

Model of Human Occupation research demonstrates a moderate level of treatment fidelity overall; it has stronger treatment fidelity from 2001 to the present. As the model progresses toward more outcomes studies, Model of Human Occupation scholars should report provider training and strategies that monitor treatment delivery to improve the quality and applicability of research outcomes.

Keywords

Treatment fidelity Model of Human Occupation research design evidence-based treatment occupational therapy

Introduction

Treatment fidelity is the degree to which an experimental treatment is delivered as intended and how well that treatment differs from the control condition (Gearing et al., 2011; Hildebrand et al., 2012). Treatment fidelity includes a variety of methodological strategies that reliably and validly test the outcomes of clinical treatment. These strategies enhance quantitative rigor and promote accurate interpretation of research results (Bellg et al., 2004; Borrelli, 2011; Borrelli et al., 2005; Cochrane, 1971; Gearing et al., 2011; Waltz et al., 1993). When outcomes are clearly linked to treatment in a manuscript, research consumers can identify solid, evidence-based treatment techniques to implement in practice (see Holm, 2000; Murphy and Gutman, 2012). Treatment fidelity should be considered throughout the research process and reported in resulting manuscripts (Bellg et al., 2004; Borrelli, 2011; Boutron et al., 2008; Murphy and Gutman, 2012). However, this critical component of outcomes studies is infrequently and inconsistently reported in occupational therapy literature (Murphy and Gutman, 2012).

Purpose statement and research questions

The current study explores the fidelity of occupational therapy treatment based on the Model of Human Occupation (MOHO) to promote occupation-focused practice instead of a commonly used impairment-focused approach (see National Board for Certification in Occupational Therapy, Inc., 2004) among researchers and clinicians. The MOHO was chosen as the occupation-focused theory of interest because it is a widely known, occupation-focused theory with a large evidence base and standardized assessments to examine its concepts; it is one of only two occupation-focused theories that explore the effectiveness of treatment strategies developed with its tenets (Lee, 2010; Lee et al., 2008; Taylor, 2017). Specific research questions are as follows:

What level of treatment fidelity exists in efficacy studies for MOHO-based treatments?

How well do therapists adhere to core elements of the MOHO during the intervention phase of the research study?

Has treatment fidelity in MOHO research changed over time?

Research and clinical significance

Results describe the current state of fidelity in MOHO outcomes literature and reveal growth areas for future research. Findings may strengthen the validity of future MOHO efficacy studies and improve the quality of research available for occupational therapists to apply in clinical practice (Bellg et al., 2004; Borrelli, 2011; Borrelli et al., 2005; Gearing et al., 2011; Hildebrand et al., 2012; Holm, 2000; Parham et al., 2007; Waltz et al., 1993).

Review of literature

Treatment fidelity addresses whether treatment is delivered according to the research plan and whether the control and experimental treatments differ as intended (Bellg et al., 2004; Borrelli 2011; Borrelli et al., 2005; Gearing et al., 2011). The concept is well-established in health behavior change research and emerging in occupational therapy literature. Treatment fidelity is important for clinical research and practice, as it enables scholars to capture the complexities of multifaceted, client-centered treatment and tease out the effects of treatment versus confounding variables (Gearing et al., 2011; Hildebrand et al., 2012; Murphy and Gutman, 2012). Additionally, reports of treatment fidelity in published manuscripts allow research consumers to appraise the quality of evidence and implement effective treatment strategies in the clinic (Holm, 2000; Murphy and Gutman, 2012). The following section highlights work involving treatment fidelity in health behavior change and occupational therapy research.

Treatment fidelity and public health

The concept of treatment fidelity evolves over time. The importance of scientific control for studies that measure the effectiveness and efficiency of medical treatments is cited in original treatment fidelity works; these techniques provide quantifiable evidence for effective treatments and how efficiently they are utilized (Cochrane, 1971). Over time, treatment fidelity broadens to the idea of treatment integrity, or whether treatment is delivered as intended, and expands to include whether experimental and control conditions differ as intended (treatment differentiation), whether the recipient understands and uses treatment skills (treatment receipt), and whether the recipient applies treatment skills to daily life (treatment enactment) (Borrelli, 2011; Borrelli et al., 2005, Gearing et al., 2011). Expansion of the term occurred in response to an increasing prevalence of treatment efficacy studies.

In more recent years, members of a Treatment Fidelity Workgroup established as part of the National Institutes of Health (NIH) Behavioral Change Consortium (BCC) developed the five-part treatment fidelity framework to operationally define treatment fidelity for clinical research. In this framework, treatment fidelity comprises five core categories: treatment design, provider training, treatment delivery, treatment receipt, and skill enactment post treatment (Bellg et al., 2004; Borrelli et al., 2005; Gearing et al., 2011; Murphy and Gutman, 2012). Treatment design includes characteristics of the treatment that should be reported for evaluation and replication of the study (for example, content, dosage, and provider credentials). Provider training pertains to the details surrounding knowledge and skill acquisition of research clinicians (for example, standardization, outcome measures, and provider retention). Treatment delivery consists of techniques to ensure that content and dosage are achieved (for example, treatment manual, records of contact time, and observation). Treatment receipt addresses how well participants understand and use treatment skills (for example, pre- and post-tests, role plays, and homework). Treatment enactment is a measure of participants' ability to apply new knowledge or skills in everyday life (for example, written materials and self-report of home behavior).

This NIH BCC Treatment Fidelity Framework has been used to develop a fidelity measure and evaluate treatment fidelity in health behavior change outcomes studies (see Bellg et al., 2004; Borrelli et al., 2005). The fidelity measure examines a research study based on all categories of treatment fidelity (Borrelli et al., 2005). When evaluated with this fidelity measure, behavior change treatments in public health and psychology literature show the following results: percent use of fidelity strategies varies by fidelity category; studies demonstrate low levels of adherence to training and treatment delivery strategies; and overall adherence varies between categories, with only 15.5% of articles meeting the criteria for high treatment fidelity (≥0.80; Borrelli et al., 2005).

Treatment fidelity and occupational therapy

Occupational therapy addresses treatment fidelity in efficacy studies over time. However, the concept is more overtly addressed in the literature around 2007. Parham et al. (2007) retrospectively reviewed sensory integration outcomes studies based on core structural and process elements of treatment. They found weak fidelity to those elements in current literature on sensory integration outcomes. Later, the Sensory Integration Research Collaborative developed the Ayres Sensory Integration Fidelity Measure, which formally assesses adherence, replicability, and differentiation of sensory integration treatment to increase the likelihood that sensory integration treatment in research, education, and practice is faithful to Ayres Sensory Integration principles (Parham et al., 2011).

Hildebrand et al. (2012) developed a model for treatment fidelity in rehabilitation research. They detail methods for provider training and supervision as well as monitoring treatment delivery (such as the Enhanced Medical Rehabilitation Adherence and Competency Rating Form). These methods aim to demonstrate adherence to key components of treatment and therapist competency with the experimental treatment; they also serve as a model for reliably and validly examining the outcomes of rehabilitation services.

Three significant publications may be the catalyst for increased attention to treatment fidelity in occupational therapy literature. The Consolidated Standards of Reporting Trials (CONSORT) guidelines were released in 2001 and expanded in 2008; the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) guidelines were released in 2004. Both sets of guidelines provide detailed instructions on the ways to report clinical trials and treatment fidelity for efficacy studies (Altman et al., 2001; Boutron et al., 2008; Des Jarlais, 2004). Additionally, former editors of the American Journal of Occupational Therapy published an editorial citing the importance of treatment fidelity in treatment efficacy studies in 2012. The article references the NIH BCC Treatment Fidelity Framework as a guide for designing and implementing efficacy studies in the field of occupational therapy. The piece indicates that fidelity is often missing from occupational therapy outcomes studies, defines treatment fidelity, and highlights strategies to improve the quality of occupational therapy research (Murphy and Gutman, 2012).

Although treatment fidelity appears in select occupational therapy literature, it has not been applied to studies that explore the outcomes of treatment guided by an overarching occupation-focused theory (such as the MOHO). This type of application conceptualizes standardization of treatment and change processes rather than steps within a manual or protocol. Thus, therapists have the freedom to tailor treatment to the client within a theory-based framework. This application aligns well with occupation-focused models and effectively captures the complexities of client-centered practice (Hawe et al., 2004, 2009; Hildebrand et al., 2012).

Methodology

Computer and hand-based searches were completed to identify MOHO outcomes literature for further analysis of treatment fidelity practices. Please note that the majority of MOHO literature focuses on the development and validation of theory and standardized assessments. First, the Cumulative Index to Nursing and Allied Health Literature and Medline databases were searched for applicable publications dated 1980 to October 2016. The year 1980 was chosen as a starting point because it marks the inception of the MOHO. Key search terms include the Model of Human Occupation and occupational therapy combined with efficacy or effectiveness, as well as the Model of Human Occupation and intervention combined with efficacy or effectiveness. The terms efficacy and effectiveness were utilized to identify articles with an experimental manipulation of treatment, a necessary criterion for the analysis of treatment fidelity (see Borrelli et al., 2005). Next, a review of articles included in Table 25.21 MOHO Research on Outcomes: Intervention Studies in Kielhofner's Model of Human Occupation, 5th Edition was performed (Bowyer and Kramer, 2017). Articles listed in the table represent evidence for the outcomes of MOHO-based services. Finally, the evidence briefs section of the University of Illinois Chicago MOHO Clearinghouse website https://www.moho.uic.edu/resources/scholarship.aspx was searched for articles relevant to the outcomes of MOHO-based treatment published 1980 to October 2016 (University of Illinois Chicago, 2016).

Articles were included in the current study if they were published in a peer-reviewed journal, examined the outcomes of treatment explicitly developed with the MOHO or its concepts with descriptive or quantitative analysis, and measured changes in participants' perceptions, abilities, or behaviors as a result of treatment. Only core studies were analyzed as one study produced multiple articles. Studies that examined the outcomes of general occupational therapy treatment techniques with MOHO assessments were not included because the treatment was not explicitly developed with the MOHO. Studies published in other languages, editorials, essays, and qualitative studies were also excluded.

The initial search yielded 49 publications. Only 17 articles fit study inclusion and exclusion criteria. Although this number may seem small, the MOHO has more evidence on effectiveness of its treatment strategies than any other occupation-focused theory (Lee, 2010). See Table 1 for a complete list of articles included in the study and their treatment fidelity scores. Relevant studies included MOHO-based treatment for individuals with dementia, HIV/AIDS, mental health conditions, multiple sclerosis, spinal cord injury, stroke, and well older adults across acute, inpatient, and outpatient settings. Multiple diagnoses and settings were included to examine general MOHO treatment principles that are tailored to the client during service provision. This approach assumes that overarching treatment and change processes are standardized across MOHO-based treatments without having to ascribe to a specific treatment protocol for all participants (see Hawe et al., 2004, 2009). For example, in the MOHO, therapists carry out a specific therapeutic reasoning process that is tailored to each client and his or her interests versus a one-size-fits-all treatment. Similarly, the client achieves change through a unique combination of occupational participation and engagement versus execution of the same steps as every participant in a specific treatment regimen or protocol (Taylor, 2017).

Table 1.

Model of Human Occupation articles and their treatment fidelity scores.

	Mean proportion of adherence
Article	Design	Training	Delivery	Receipt	Enactment	Overall
Brown and Carmichael (1992)	1.00	0.00	0.20	0.75	1.00	0.57
Cohen and Schemm (2007)	0.82	0.25	0.20	0.75	1.00	0.62
Corcoran and Gitlin (2001)	0.64	0.25	0.40	0.75	1.00	0.58
Gitlin et al. (2003)	0.73	0.50	0.40	0.75	1.00	0.65
Graff et al. (2003)	0.83	0.00	0.20	0.75	1.00	0.52
Ingvarsson and Theodórsdóttir (2004)	0.83	0.00	0.00	0.75	1.00	0.48
Josephsson et al. (1993)	0.83	0.00	0.00	0.75	1.00	0.48
Josephsson et al. (1995)	1.00	0.00	0.00	0.75	1.00	0.52
Kielhofner and Brinson (1989)	0.45	0.00	0.20	0.75	1.00	0.42
Kielhofner et al. (2004)	0.67	0.00	0.00	0.75	1.00	0.43
Kielhofner et al. (2008)	1.00	0.00	0.00	0.75	1.00	0.62
Metzelthin et al. (2010, 2013)	0.64	0.75	1.00	1.00	1.00	0.81
Peirce (2001)	1.00	0.00	0.20	0.75	1.00	0.65
Schaaf and Mulrooney (1989)	1.00	0.00	0.00	0.75	1.00	0.52
Shinohara et al. (2012)	0.91	0.00	0.00	0.75	0.50	0.54
Turner and Lydon (2008)	0.83	0.00	0.40	1.00	1.00	0.62
Yamada et al. (2010)	0.91	0.00	0.20	1.00	1.00	0.65

Instrument

The Treatment Fidelity Checklist was used to code relevant articles for treatment fidelity. The Treatment Fidelity Checklist consists of 25 items that encompass the five domains of treatment fidelity: treatment design, provider training, treatment delivery, treatment receipt, and enactment of treatment skills (Borrelli et al., 2005). Items are rated based on the following categorical scale: 0 = not applicable, 1 = absent, 2 = present, and 3 = found in companion article. A score of 0 is assigned if the treatment fidelity strategy is not appropriate for the article (studies without a control group were not penalized for nonadherence in the design category), a score of 1 is given if the item is not represented in the manuscript, a score of 2 is issued if the item is detailed in the manuscript, and a score of 3 is assigned when the item can be answered with content from a related article. The Treatment Fidelity Checklist demonstrates adequate inter-rater reliability with 84% agreement between raters (Borrelli et al., 2005).

Coding procedure

Articles were coded between October 2016 and January 2017. Three members of the research team participated in the coding procedure, which was adapted from Borrelli et al. (2005). Coders met by phone to gain familiarity with the concept of treatment fidelity, the Treatment Fidelity Measure, and the coding procedure. A coding guideline was developed to ensure adherence to the established procedure. Researchers individually coded each article with the Treatment Fidelity Measure and entered responses into a Microsoft Excel form. The Microsoft Excel form was obtained from Belinda Borrelli, one of the original authors of the tool, and included examples of each item to improve understanding and accuracy in coding. Monthly contact was established for 4 months to resolve coding issues. All coders met by phone at the end of the coding period to discuss and resolve discrepancies in responses.

Characteristics of coders

The coding procedure was implemented under the supervision of a professor of occupational therapy at Texas Woman's University. All coders were affiliated with the school of occupational therapy at Texas Woman's University. One coder works as a part-time clinician and is currently pursuing a doctoral degree in occupational therapy. The remaining two coders are full-time occupational therapy students pursuing a master's degree in occupational therapy.

Intercoder reliability

Percent agreement is identified in the literature as the most appropriate indicator of intercoder reliability for the Treatment Fidelity Measure (Borrelli et al., 2005). For the purposes of the current study, the cutoff score for acceptable intercoder reliability was predetermined as ≥80%, in accordance with Lombard et al. (2002). Percent agreement was determined by dividing the number of items agreed upon by the number of possible items for each article in the study. Those values were averaged to determine overall percent agreement. Percent agreement across articles ranged from 56% to 100% (mean, 81.2%).

Analytic plan

Frequency counts and percentages were calculated for each item on the Treatment Fidelity Measure. The number of articles where an item or treatment fidelity strategy was documented was divided by the number of articles where the strategy was appropriate. For example, length of the experimental treatment was documented in 14 out of 17 possible articles, yielding 82.35% adherence for that item. Calculations were modified for items related to a control group because only nine out of 17 articles included this type of comparison. Specifically, percent adherence of items marked “present” for the control group were calculated out of nine possible articles versus 17. For example, length of the control intervention was documented in four out of nine articles, with a control group yielding 44.44% adherence to that item.

Mean proportion of adherence to treatment fidelity strategies was computed for each article based on the calculations in Borrelli et al. (2005): items marked as present were summed and then divided by the number of possible strategies for the article in question. Means for each article were summed and divided by the total number of articles for an overall index of treatment fidelity. Similar calculations were made for each category of treatment fidelity (design, training, delivery, receipt, and enactment) within and across articles. Comparisons of the overall mean proportion of adherence between time periods (1980–1990, 1991–2000, and 2001–2017) were examined with one-way analysis of variance.

Results

Frequency of reporting treatment fidelity strategies

See Figure 1 for the frequency counts of treatment fidelity strategies reported in the MOHO outcomes literature as itemized by the Treatment Fidelity Measure. Adherence to individual items ranges from 0% to 100%. Ranges of adherence to fidelity strategies in each category are as follows: design 44.44–100%, training 5.88–17.64%, delivery 5.88–52.94%, receipt 17.65–100%, and enactment 94.12–100%.

Figure 1.

Frequency of adherence to items on the treatment fidelity measure for Model of Human Occupation literature. Control frequencies are tallied out of nine total articles, as only nine articles contained control groups. All other frequencies are tallied out of 17 total articles.

Mean proportion of adherence to treatment fidelity strategies: categorically and overall

Table 2 shows the mean proportion of adherence to treatment fidelity strategies overall and for each treatment fidelity category. The overall index of treatment fidelity strategies is 0.57. Mean proportion of adherence for individual categories is as follows: design 0.83, training 0.10, delivery 0.20, receipt 0.79, and enactment 0.97.

Table 2.

Proportion of adherence to treatment fidelity strategies in Model of Human Occupation literature.

Category	Mean proportion	SD
Design	0.83	0.82
Training	0.10	0.30
Delivery	0.20	0.40
Receipt	0.79	0.40
Enactment	0.97	0.17
Overall mean	0.57	0.09

Note. n = 17. Table adapted from Borrelli et al. (2005).

Mean proportion of adherence to treatment fidelity strategies over time

A one-way analysis of variance was conducted to compare overall mean adherence with treatment fidelity strategies over time. Articles were grouped into three time frames: 1980–1990, 1991–2000, and 2001–2017. There is no significant difference in mean adherence to treatment fidelity strategies in the MOHO literature over time at the p < 0.05 level (F(2,14) = 2.16; p = 0.15). However, these values do trend upward. Although the data largely show inconsistent changes in mean adherence to categories of treatment fidelity over time, small improvements are noted in the training and receipt categories. Table 3 illustrates both the upward trend in overall mean adherence to treatment fidelity and the differences in adherence to treatment fidelity categories in the MOHO literature over time.

Table 3.

Proportion of adherence to treatment fidelity strategies over time.

	Mean proportion of adherence
Category	1980–1990 (n = 2)	1991–2000 (n = 3)	2001–2017 (n = 12)
Design	0.73	0.94	0.82
Training	0.00	0.00	0.15
Delivery	0.10	0.07	0.25
Receipt	0.75	0.75	0.81
Enactment	1.00	1.00	0.96
Overall	0.47	0.52	0.60

High levels of treatment fidelity

For the purposes of this study, high treatment fidelity is defined as a value of ≥0.80 for mean proportion of adherence to the Treatment Fidelity Measure (Borrelli et al., 2005; Gearing et al., 2011). The MOHO efficacy literature as a whole does not meet criteria for high treatment fidelity (0.57). However, it demonstrates high fidelity in the design (0.83) and enactment categories (0.97). One individual article shows high treatment fidelity overall, with a mean adherence value of 0.81 (Metzelthin et al., 2013). A majority of individual articles demonstrate high fidelity in the design (75%, n = 12) and enactment (94.12%, n = 16) categories. One article demonstrates high treatment fidelity in the delivery category (Metzelthin et al., 2013). Three articles demonstrate high treatment fidelity in the receipt category (Metzelthin et al., 2013; Turner and Lydon, 2008; Yamada et al., 2010).

Discussion

The current study shows moderate levels of treatment fidelity in the MOHO outcomes research. However, treatment fidelity for individual categories is high (design, and enactment). Overall treatment fidelity in the MOHO literature is weakened by the provider training and treatment delivery categories. These results are similar to those found in health behavior change research (Borrelli et al., 2005). Neither body of literature met the criteria for high fidelity overall (≥0.80) but demonstrated high levels of fidelity in at least one individual category.

The MOHO outcomes literature demonstrates lowest mean adherence to strategies in the provider training and delivery categories, with values of 0.10 and 0.20, respectively. The majority of MOHO outcomes studies include detailed interventions, which suggest provider training is needed to carry out the treatment. However, only a few articles report provider training procedures in the methodology section. Only one study references standardization of provider training in a companion article. In terms of delivery, MOHO outcomes studies incorporate specific intervention guidelines with descriptions, time and/or session allotments, and, in some cases, a decision tree to ensure intervention content is delivered as intended. However, few articles mention strategies that ensure dose, provider adherence, and nonspecific treatment effects.

Our results align with previous research studies on treatment fidelity as well as trends noticed by former editors of the American Journal of Occupational Therapy when reviewing occupational therapy outcomes studies (see Borrelli et al., 2005; Murphy and Gutman, 2012; Parham et al., 2007). These findings may result from failure to report use of fidelity strategies within these categories or failure to consider these aspects of treatment fidelity when designing and implementing the experimental treatment. Either way, weak treatment fidelity for provider training and delivery affects the validity, generalizability, and replicability of both the treatment and research results (Borrelli et al., 2005; Gearing et al., 2011; Parham et al., 2007).

The mean overall adherence to treatment fidelity of MOHO efficacy studies is not significantly different over time. However, there is a positive trend toward higher levels of treatment fidelity from 1980 to 2017. These results differ from health behavior change research, which shows a decrease in the use of strategies over time (Borrelli, et al., 2005).

The trend towards higher levels of treatment fidelity in MOHO research may be attributed to model development and significant treatment efficacy publications. First, early MOHO research is dedicated to scientific testing of the model and its constructs as well as the development of MOHO-based standardized assessments. MOHO efficacy studies have only become more prominent in recent years, as evidenced by the number of articles per time period in the current study (1980–1990: n = 2, 1991–2000: n = 3, 2001–2017: n = 12). Second, release of the CONSORT guidelines in 2001 and the TREND guidelines in 2004 may have prompted MOHO scholars to pay more attention to treatment fidelity in efficacy studies moving forward. Approximately 12 out of the 17 MOHO outcomes studies included in the current study were published in the year 2001 or later, and eight of the MOHO outcomes studies were published in the year 2004 or later.

Limitations

The current study may be limited in several ways. Although a comprehensive review of the MOHO literature was conducted, the search method may have missed relevant articles, including those currently undergoing the publication process. Weak areas of fidelity in MOHO efficacy studies were identified but reasons for those weaknesses were not determined. For example, few articles documented provider training before treatment implementation despite very detailed and structured programs. Fidelity strategies for training may have been implemented satisfactorily in these cases yet not reported for a variety of reasons (such as space limitations and common reporting practices within the discipline at the time of publication). Additionally, we used a framework and tool used to examine treatment fidelity that was developed for psychologically focused interventions. It is possible that the concepts and language may not coincide with occupational therapy interventions perfectly. However, many of the MOHO-based interventions in this review included a psychological component or were carried out with mental health populations, whereas others aimed to facilitate a health behavior change in the participant group. Furthermore, areas of weak fidelity identified in this review (training and adherence to protocol) are preserved in a more recent treatment fidelity model for rehabilitative interventions proposed by Poltawski et al. (2014).

Implications

Occupational therapy researchers should familiarize themselves with treatment fidelity and use fidelity strategies in treatment efficacy studies. Both the NIH BCC Treatment Fidelity Framework (Borrelli et al., 2005) and the model of intervention fidelity for rehabilitation research (Poltawski et al., 2014) provide an excellent foundation in treatment fidelity as well as guidance for selecting appropriate fidelity strategies. Knowledge and use of treatment fidelity strategies will improve the rigor of occupational therapy research (Borrelli et al., 2005; Gearing et al., 2011, Murphy and Gutman, 2012).

Occupational therapy researchers should report treatment fidelity strategies used in efficacy research, especially with regards to provider training. The CONSORT (Boutron et al., 2008) and TREND (Des Jarlais et al., 2004) guidelines provide direction for effectively reporting research studies. The CONSORT guidelines have been expanded for nonpharmacological trials and include specific instructions on how to include treatment fidelity in the published report. Comprehensive reports of treatment fidelity in efficacy studies will improve replicability of the intervention in future research studies and in the clinic (Borrelli et al., 2005; Gearing et al., 2011, Murphy and Gutman, 2012).

More specifically, researchers who focus on the outcomes of MOHO-based treatment should consider strategies and tools to monitor intervention delivery. These strategies and tools would ensure adherence to and competence with MOHO-based treatment as well as differentiation of experimental and control groups while the treatment is administered. Successful implementation strategies to monitor intervention delivery would bolster confidence in conclusions related to treatment efficacy (Borrelli et al., 2005; Gearing et al., 2011, Murphy and Gutman, 2012).

Conclusion

MOHO efficacy studies demonstrate high levels of treatment fidelity for design and enactment categories and weakest fidelity for training and delivery categories. They have moderate treatment fidelity overall. Future MOHO outcomes studies should report methods for provider training and monitor treatment delivery to improve the overall fidelity of MOHO-based interventions. This will improve confidence in the effects of MOHO-based treatment and translation of evidence-based treatments to the clinic.

Key findings

Model of Human Occupation research demonstrates moderate treatment fidelity overall.

This literature implements design and enactment fidelity strategies well, but provider training and treatment delivery strategies poorly.

What the study has added

This study provides basic quantitative evaluation of treatment fidelity in Model of Human Occupation outcomes research and identifies ways to improve the fidelity of future Model of Human Occupation studies.

Footnotes

Acknowledgments

We acknowledge Brittney Burton and Daisy Lim for their assistance with the coding process.

Research ethics

Ethics approval and consent were not required as this study of literature involves no human subjects.

Declaration of conflicting interests

None declared.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Contributorship

Patricia Bowyer and Melanie Morriss Tkach collaborated on study design and the analysis plan. Melanie Morriss Tkach researched literature and gathered and coded data. Patricia Bowyer and Melanie Morriss Tkach conducted statistical analysis and interpreted the results. Melanie Morriss Tkach wrote the first draft of the manuscript. Both authors reviewed and edited the manuscript. Both authors approved the final version of the manuscript.

References

Altman

Schulz

Moher

et al. (2001) The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Annals of Internal Medicine 134(8): 663–694.

Bellg

Borrelli

Resnick

et al. (2004) Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology 23(5): 443–451.

National Board for Certification in Occupational Therapy, Inc. (2004) A practice analysis study of entry-level occupational therapist registered and certified occupational therapy assistant practice. Occupational Therapy Journal of Research 24(Supplement 1): S1–S31.

Borrelli

(2011) The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. Journal of Public Health Dentistry 71(s1): S52–S63.

Borrelli

Sepinwall

Ernst

et al. (2005) A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. Journal of Consulting and Clinical Psychology 73(5): 852–860.

Boutron

Moher

Altman

et al. (2008) Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: Explanation and elaboration. Annals of Internal Medicine 148(4): 295–309.

Bowyer

Kramer

(2017) Evidence for practice from the model of human occupation. In: Taylor

(ed.) Kielhofner's Model of Human Occupation, 5th ed. Philadelphia: Wolters Kluwer, pp. 418–465.

Brown

Carmichael

(1992) Assertiveness training for clients with a psychiatric illness: A pilot study. British Journal of Occupational Therapy 55(4): 137–140.

Cochrane

(1971) Effectiveness and Efficiency: Random Reflection on Health Services, London: Nuffield Provincial Hospitals Trust.

10.

Cohen

Schemm

(2007) Client-centered occupational therapy for individuals with spinal cord injury. Occupational Therapy in Health Care 21(3): 1–15.

11.

Corcoran

Gitlin

FNL

(2001) Family caregiver acceptance and use of environmental strategies provided in an occupational therapy intervention. Physical & Occupational Therapy in Geriatrics 19(1): 1–20.

12.

Des Jarlais

Lyles

Crepaz

et al. (2004) Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: The TREND statement. American Journal of Public Health 94(3): 361–366.

13.

Gearing

El-Bassel

Ghesquiere

et al. (2011) Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clinical Psychology Review 31(1): 79–88.

14.

Gitlin

Belle

Burgio

et al. (2003) Effect of multicomponent interventions on caregiver burden and depression: The REACH multisite initiative at 6-month follow-up. Psychology and Aging 18(3): 361–374.

15.

Graff

Vernooij-Dassen

Hoefnagels

et al. (2003) Occupational therapy at home for older individuals with mild to moderate cognitive impairments and their primary caregivers: A pilot study. OTJR: Occupation, Participation and Health 23(4): 155–164.

16.

Hawe

Shiell

Riley

(2004) Complex interventions: How “out of control” can a randomised controlled trial be?. BMJ (Clinical research ed.) 328(7455): 1561–1563.

17.

Hawe

Shiell

Riley

(2009) Theorising interventions as events in systems. American Journal of Community Psychology 43(3–4): 267–276.

18.

Hildebrand

Host

Binder

et al. (2012) Measuring treatment fidelity in a rehabilitation intervention study. American Journal of Physical Medicine and Rehabilitation 91(8): 715–724.

19.

Holm

(2000) Our mandate for the new millennium: Evidence-based practice. American Journal of Occupational Therapy 54(6): 575–585.

20.

Ingvarsson

Theodórsdóttir

(2004) Vocational rehabilitation at Reykjalundur rehabilitation center in Iceland. Work 22(1): 17–19.

21.

Josephsson

Bäckman

Borell

et al. (1993) Supporting everyday activities in dementia: An intervention study. International Journal of Geriatric Psychiatry 8(5): 395–400.

22.

Josephsson

Bäckman

Borell

et al. (1995) Effectiveness of an intervention to improve occupational performance in dementia. The Occupational Therapy Journal of Research 15(1): 36–49.

23.

Kielhofner

Brinson

(1989) Development and evaluation of an aftercare program for young chronic psychiatrically disabled adults. Occupational Therapy in Mental Health 9(2): 1–25.

24.

Kielhofner

Braveman

Finlayson

et al. (2004) Outcomes of a vocational program for persons with AIDS. American Journal of Occupational Therapy 58(1): 64–72.

25.

Kielhofner

Braveman

Fogg

et al. (2008) A controlled study of services to enhance productive participation among people with HIV/AIDS. American Journal of Occupational Therapy 62(1): 36–45.

26.

Lee

Taylor

Kielhofner

et al. (2008) Theory use in practice: A national survey of therapists who use the model of human occupation. American Journal of Occupational Therapy 62(1): 106–117.

27.

Lee

(2010) Achieving best practice: A review of evidence linked to occupation-focused practice models. Occupational Therapy in Healthcare 24(3): 206–222.

28.

Lombard

Snyder-Duch

Bracken

(2002) Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human Communication Research 28(4): 587–604.

29.

Metzelthin

van Rossum

de Witte

et al. (2010) The reduction of disability in community-dwelling frail older people: Design of a two-arm cluster randomized controlled trial. BMC Public Health 10(1): 511.

30.

Metzelthin

van Rossum

de Witte

et al. (2013) Effectiveness of interdisciplinary primary care approach to reduce disability in community dwelling frail older people: Cluster randomized controlled trial. BMJ 347(7926): 12.

31.

Murphy

Gutman

(2012) Intervention fidelity: A necessary aspect of intervention effectiveness studies. American Journal of Occupational Therapy 66(4): 387–388.

32.

Parham

Cohn

Spitzer

et al. (2007) Fidelity in sensory integration intervention research. American Journal of Occupational Therapy 61(2): 216–227.

33.

Parham

Roley

May-Benson

et al. (2011) Development of a fidelity measure for research on the effectiveness of the Ayres Sensory Integration® intervention. American Journal of Occupational Therapy 65(2): 133–142.

34.

Peirce CC (2001) The effectiveness of two different occupational therapy interventions with individuals with multiple sclerosis: A randomized controlled trial. PhD Thesis, Nova Southwestern University, USA.

35.

Poltawski

Norris

Dean

(2014) Intervention fidelity: Developing an experience-based model for rehabilitation research. Journal of Rehabilitation Medicine 46(7): 609–615.

36.

Schaaf

Mulrooney

(1989) Occupational therapy in early intervention: A family-centered approach. American Journal of Occupational Therapy 43(11): 745–754.

37.

Shinohara

Yamada

Kobayashi

et al. (2012) The model of human occupation-based intervention for patients with stroke: A randomised trial. Hong Kong Journal of Occupational Therapy 22(2): 60–69.

38.

Taylor

(2017) Kielhofner's Model of Human Occupation, 5th ed. Philadelphia: Wolters Kluwer.

39.

Turner

Lydon

(2008) Psychosocial programming in Ireland based on the model of human occupation: A program evaluation study. Occupational Therapy in Health Care 22(2–3): 105–114.

40.

University of Illinois at Chicago (2016) The model of human occupation evidence briefs. Available at: https://www.moho.uic.edu/resources/scholarship.aspx (accessed 7 October 2016).

41.

Waltz

Addis

Koerner

et al. (1993) Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology 61(4): 620–630.

42.

Yamada

Kawamata

Kobayashi

et al. (2010) A randomised clinical trial of a wellness programme for healthy older people. British Journal of Occupational Therapy 73(11): 540–548.