An Assessment of Intervention Fidelity in Published Social Work Intervention Research Studies

Abstract

Objectives:

Intervention fidelity is a critical strategy to help advance the usefulness and integrity of social work research. This study assessed the extent to which a selected sample of published social work intervention researchers reported its intervention protocols.

Methods:

Six core social work journals were reviewed in this analysis. The authors reviewed every scholarly article within each journal published from 2009 through 2013 (N = 1,380). A total of N = 32 (2.3%) published intervention research articles met the inclusion criteria. All N = 32 studies were evaluated using 11 criteria developed by the research team related to assessing the study’s treatment fidelity.

Results:

The finding suggests that although attention to treatment fidelity in social work research has increased, methodological inconsistencies still remain.

Conclusion:

Effective social work research of an intended intervention is largely dependent on the intervention’s fidelity. Implications for social work practice and research are discussed.

Keywords

intervention fidelity treatment fidelity systematic assessment

In the past 25 years, social work has made considerable improvements in many key areas of research and evaluation (R&E). Social work researchers and evaluators are (a) better attending to issues concerning ethics (Holosko, Thyer, & Danner, 2009; Rubin & Parrish, 2007), (b) more focused on outcome specificity (Thyer & Myers, 2007), (c) creating a culture that supports and encourages research in agencies and academia (McRoy, Flanzer, & Zlotnik, 2012), (d) using more precise and consistent R&E terms (Holosko & Thyer, 2011), (e) increasing the quality of their research writings through the use of uniform stylistic guidelines (e.g., American Psychological Association’s Journal Article Reporting Standards; Holosko, 2006), and (f) assessing their research designs more thoroughly. Holosko (2006) furthered this “laundry list” of improvements when he outlined evidence of social work’s evolution in (R&E based on the following additional benchmarks:

we are conducting more R&E than ever before,

we are publishing more R&E studies,

we have staked claim to certain R&E methods that suit our practice realities, and

we have expanded our R&E methodological repertoire to include nontraditional approaches such as meta-analyses, participatory program evaluations, systematic reviews, and so on (p. 449).

However, as Wodarski (2011) suggested, although social work has made significant progress in research, there continues to be ambivalence associated with it. As such, social workers must “vigorously pursue further knowledge on how to measure and evaluate data to ensure that best practices used, live up to that label” (p. 577).

There is a mounting emphasis for social work professionals to ethically integrate evidence-informed techniques into their day-to-day practice (Holosko et al., 2009). Yet, despite this, major gaps still exist between what are known as effective practices and what is actually done (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). While evidence-based practice provides a starting point in ensuring clients receive appropriate interventions, it does not direct or ensure how our interventions are actually being implemented. As such, effective social work research on intervention outcomes will likely not only depend on the science and theory endorsing the use of an intended intervention but also on its intervention fidelity.

The importance of intervention fidelity has gained considerable momentum over the past 20 years (Borrelli et al., 2005). Broadly defined as the extent to which the delivery of an intervention is administered as was initially planned (Dane & Schneider, 1998; Mowbray, Holter, Teague, & Bybee, 2003), intervention fidelity is an important factor in intervention research across a variety of disciplines (Gearing et al., 2011). Given social work’s research unique client focus (Holosko & Leslie, 1998) and because approximately 25% of published studies in our top empirical journals are intervention studies (Holosko, 2009), intervention fidelity becomes a critical strategy to help advance the usefulness and integrity of our research to, not only better understand and serve client populations, but also guide practitioners in the field. Shaw (2005) championed the use of intervention fidelity in social work research when he insisted social work research will be “distinctively good” when it (among other things) “aims for methodological excellence in whatever it does” (p. 663).

Based on a review of extant literature in this area, the primary components of intervention fidelity include (a) the design and operationalized treatment procedures that is, clear definitions of the required intervention content as well as processes; (b) intervention training; (c) monitoring of intervention delivery and receipt; (d) verifying adherence to protocols (e.g., outside sources such as videotape or observation; (e) self-report assessments from research participants or implementers; and (f) treatment enactment (Borrelli et al., 2005; Gearing et al., 2011; Moncher & Prinz, 1991). Despite increasing awareness of the importance of intervention fidelity, using these strategies in intervention research has been rather limited (Bellg et al., 2004). Fraser and Galinsky (2010) defined intervention research as “the systematic study of purposive change strategies” (p. 459). Different from evaluation research, which contributes to assessing processes and outcomes of programs, intervention research “is distinguished by its emphasis on the design and development of the intervention” (p. 460). These authors further contended that implementing an intervention with low fidelity has the potential of undermining even a well-designed study.

The quality and practice effectiveness of all interventions are largely dependent on the degree to which they can be replicated and can accurately account for their outcomes. As such, it has been posited that fidelity is an important methodological practice to enhance the reliability and validity of behavioral interventions (Bellg et al., 2004). As such, Carroll et al. (2007) suggested that intervention fidelity “acts as a potential moderator of the relationship between interventions and their intended outcomes” (p. 1). That is, measurement of a program’s adherence to an intended model helps prevent potentially false conclusions from being drawn about an intervention’s effectiveness (Carroll et al., 2007; H. Chen, 1990; Gresham, Gansle, and Noell, 1993). Overall, enhancing the intervention’s fidelity has the added resulting effect of increasing the external and internal validity of the intervention, which are critical aspects for the replicability and generalizability of any study (Borrelli et al., 2005). Fidelity also has significant implications to an intervention’s statistical power, further impacting the integrity of the intervention’s results (Moncher & Prinz, 1991).

In the past decades, social and behavioral science disciplines have considered treatment fidelity in a wide range of contexts including psychosocial interventions (McArthur, Riosa, & Preyde, 2012), health behavior interventions (Borrelli et al., 2005), school-based behavioral interventions (Dusenbury, Brannigan, Falco, & Hansen, 2003), evaluation of psychoanalytic therapies (Silberschatz, Fretter, & Curtis, 1986), clinical psychology, behavioral therapy, marital and family therapy (Moncher & Prinz, 1991), special education programs (Salend, 1984), multisite program evaluation (Gidycz et al., 2001), and psychiatric rehabilitation (Bond, Evans, Salyers, Williams, & Kim, 2000).

Taken together, their findings suggest that many studies do not use strategies to enhance intervention, and/or measure or report the extent of treatment fidelity. For example, McArthur, Riosa, and Preyde (2012) found in their review of the subject that only 1 in 10 (10%) of the articles assessed specially measured and reported treatment fidelity in psychosocial intervention research. Whereas 21% of the N = 342 of studies in health behavior intervention research reviewed by Borrelli et al. (2005) did the same. Similarly, Gresham, Gansle, and Noell (1993) reviewed applied behavior analysis studies, and reported that 16% of N = 168 articles measured fidelity, but nearly 66% of these articles did not define intervention fidelity as an independent variable.

This lack of intervention fidelity is also endemic in social work research. Tucker and Blythe (2008) illustrated this when they reviewed a decade of social work research N = 128 published articles on psychosocial outcomes to assess treatment fidelity. Their research revealed that 39% used and reported on some strategies to enhance fidelity. A similar study conducted by Naleppa and Cagle (2010) reviewed N = 63 published outcome studies. Their findings suggested that overall standard measures of treatment fidelity were lacking in published social work research.

Examining the degree to which social work researchers are addressing fidelity is critical to the effectiveness of an intervention. As Holosko and Leslie (1998) contended, at its core social work research seeks to answer the following three questions, namely,

How does this study help us to better understand our clients and their problems?

How does it better help us to treat clients?

How does it help in developing theories or knowledge to guide our practice? (p. 434)

To effectively answer these questions and attend to the best interest of our clients, social work researchers and evaluators must address intervention fidelity better than we have in the past. With the growing ethical imperative promoting the use of interventions that have been informed by current best available research (Gambrill, 2011; Gibbs & Gambrill, 2002; McNeece & Thyer, 2004), it is critical that social workers attend to fidelity assessments more thoroughly to ensure that the maximum utility of the intervention is being realized—to affect change and achieve desired outcomes. Assessing the extent to which studies are addressing intervention fidelity is an important step to guide future research of social work interventions and in turn, their eventual outcomes.

The purpose of this study is to assess how published social work intervention research studies describe and note issues related to intervention fidelity. The study has two objectives: (1) to assess the extent to which a selected sample of published social work intervention research report their intervention protocols and (2) to develop an intervention fidelity checklist for social service practitioners and social work intervention researchers to use. Implications of this study will ideally inform practitioners and researchers about the importance of using an intervention fidelity protocol to ensure that this issue be treated with the importance it warrants. The review of this important topic of intervention research not only encourages social service researchers to treat intervention fidelity more seriously in published research but will hopefully also offset the noticeable void in literature about this important topic.

Method

Sample

Six core social work journals were reviewed in this secondary analysis. These were selected because they were used in reviews of research related comparisons and were also social work journals in which highly cited articles were published. The journals included British Journal of Social Work (BJSW), Journal of Social Service Research (JSSR), Social Service Review (SSR), Social Work (SW), Social Work Research (SWR), and Research on Social Work Practice (RSWP). Similarly, Rubin and Parrish (2007) and Holosko (2009) selected JSSR, RSWP, and SWR in their review of the types of designs used in social work R&E. Hodge, Lacasse, and Benson (2012) conducted an examination of social work’s most influential articles and listed BJSW, RSWP, SSR, and SW in which 14 of the most influential social work articles were published.

Data Collection

The authors first reviewed the abstracts of every scholarly article within each journal published from 2009 through 2013 (N = 1,380) to identify potential intervention studies. Only intervention studies were included in this assessment as intervention research is prominent in social work (Holosko, 2009), and it was the focus of this entire special edition journal. Using the framework initially outlined by Fraser and Galinsky (2010) that distinguished intervention research “by its emphasis on the design and development of interventions (p. 460),” the authors with consultation from two social work authors who conduct and publish intervention studies in social work, subsequently developed a set of minimal intervention research criteria to assist in identifying potential intervention studies. These mutually exclusive criteria included (1) specification of a defined intervention; (2) at minimum, the intervention had a one group pre/posttest design; (3) outcomes (rather than outputs) were clearly specified; (4) an assessment of behavioral change outcomes was foremost (not cognitive or attitudinal); (5) comparison change statistics were evident; and (6) all studies were primarily guided by theory. Studies that used archival data were excluded from this analysis.

All studies were then evaluated using eight criteria related to assessing their overall treatment fidelity. These criteria items were developed using work by Moncher and Prinz (1991) and Tucker and Blythe (2008), and they included (a) the intervention was pretested; (b) ethics was mentioned, (c) treatment manuals/protocols were mentioned; (d) the use of treatment manuals/protocols (no training) were noted, (e) training and supervision was mentioned; (f) partial/full description of intervention was offered; (g) any consideration of consistency of treatment implementation was mentioned; (h) was actual term treatment/intervention fidelity used; (i) the tenets of treatment fidelity were discussed without the actual use of the term; (j) strategies used to enhance treatment fidelity were mentioned; and (k) treatment fidelity quantitatively assessed or not.

In order to establish interrater reliability for assessing the fidelity for the articles selected, a sample of eight studies (not included in the study sample) from the same set of journals were independently appraised on these criteria by each researcher. Approximately, 86.7% of the articles were seen and rated in the same way. The other 14% were discussed between the raters using the criteria framework, until 100% congruence was determined.

Results

A total of N = 32 (2.3%) published articles met all six of these inclusion criteria. These studies included experimental, preexperimental, and minimally quasi-experimental designs. Ranked in descending order: (a) Research on Social Work Practice accounted for 68.75% of the studies analyzed, (b) Social Work Research contributed 12.5%, (c) Social Work and Journal of Social Service Research each contributed 6.25%, and (d) the British of Journal of Social Work and Social Service Review had 3.13% each.

Sample sizes ranged from N = 40 to 974. The number of specified outcomes in the studies ranged from one to seven and included outcomes related to child-rearing practices in a parenting, substance abuse treatment, and sexual risk behaviors for HIV, to name a few. Various statistical methods were used to assess change statistics typically including analysis of variance, multivariate analysis of variance, analysis of covariate, hierarchy linear model, paired sample t-tests, χ² tests, and effect sizes. Table 1 shows the frequency of selected fidelity criteria among the 32 reviewed articles in ranked order. Of the 32 studies examined, 81.3% offered either a partial or full description of intervention.

Table 1.

Frequency of Selected Criteria to Assess Treatment Fidelity in Six Empirical Journals.

Selected Intervention Fidelity Criteria	Yes (%)	No (%)
1. Description of intervention	26 (81.3)	6 (18.7)
2. Use of treatment fidelity term	21 (65.6)	11 (34.4)
3. Manuals/protocols mentioned	20 (62.5)	21 (37.5)
4. Training offered	19 (59.4)	13 (40.6)
5. Use of manuals/protocols	16 (50.0)	16 (50.0)
6. Strategies used to monitor intervention delivery	16 (50.0)	16 (50.0)
7. Supervision offered	13 (40.6)	19 (59.4)
8. Discussion of the tenets of treatment fidelity without using term	3 (9.4)	29 (90.6)

Note. N = 32.

The mention of manuals for the interventions ranked second (62.5%) in the reviewed articles; however, 50% of them stated they actually used the manual in training, supervision, or throughout intervention’s implementation. These articles explicitly indicated that the intervention is both manualized and the manual is being used during interventions by practitioners in order to minimize personal styles to affect results (e.g., see Hall et al., 2009; Kayser, Feldman, Borstelmann, & Daniels, 2010; Striley, Nattala, Abdallah, Dennis, & Cottler, 2013).

In our sample, 59.4% offered training and 40.6% offered supervision to practitioners. A much lower percentage (37.5%) provided both training and supervision.

More than half (65.6%) of the studies reviewed actually used the term intervention fidelity term (or a similar term, such as treatment integrity or adherence). A much smaller proportion (9.4%) of the articles discussed tenets without using these terms. For example, Jones (2009) mentioned that in order to maintain consistency, the principal investigator co-facilitated all group interventions; however, they did not specifically mention fidelity as the reason to ensure consistency.

Finally, 16 out of the 32 articles reviewed utilized strategies to monitor consistency of intervention delivery. The strategies used included using a fidelity checklist, monitoring by an outside research team or audiotapes, and completing standardized fidelity measurements. The most frequently used strategy was quantitatively assessing the intervention using a fidelity checklist by either practitioners or outside members. Some studies reported that intervention practitioners completed the treatment fidelity checklist after each session and discussed it (Conner et al., 2011; Langhorst, Choi, Keyser-Marcus, & Svikis, 2012; Leitch, Vanslyke, & Allen, 2009; Leung et al., 2009), and others reported the intervention was monitored by outside specialists and researchers using a fidelity checklist (Coulter, 2011; Jacobson, Osteen, Sharpe, & Pastoor, 2012; Kayser et al., 2010; Painter, 2009; Timmer et al., 2010; Valdez, Cepeda, Parrish, Horowitz, & Kaplan, 2013). Painter (2009) used standardized scales, Therapist Adherence Measure and Supervisor Adherence Measure, to ensure consistency of delivery intervention. Although quantitative assessments were the most utilized strategy to assess the intervention, few studies reported their results. For example, after conducting a fidelity analysis, Painter (2009) and Parish, Rose, Luken, Swaine, and O’Hare (2012) noted the varying levels to which their respective studies adhered to fidelity.

The top five ranked articles appear in Table 2. All of the listed articles addressed all eight of the selected fidelity criteria.

Table 2.

Top Five Ranked Articles of Eight Selected Fidelity Intervention Criteria in Six Empirical Social Work Journals.

Authors (Years)	Title and Journal Source
Zajicek-Farber (2010)	Building Practice Evidence for Parent Mentoring Home Visiting in Early Childhood, RSWP
Conner and Fraser (2011)	Preschool Social–Emotional Skills Training: A Controlled Pilot Test of the Making Choices and Strong Families Programs, RSWP
Kayser, Feldman, Borstelmann, and Daniels (2010)	Effects of a Randomized Couple-based Intervention on Quality of Life of Breast Cancer Patients and Their Partners, SWR
Lanier et al. (2011)	Parent–Child Interaction Therapy in a Community Setting: Examining Outcomes, Attrition, and Treatment Setting, RSWP
Painter (2009)	Multisystemic Therapy as Community-Based Treatment for Youth With Severe Emotional Disturbance, RSWP

Note. N = 32. RSWP* = Research on Social Work Practice; SWR* = Social Work Research.

Discussion

Adhering to fidelity protocols assists researchers from simply knowing if the intervention worked, to understanding the nuanced particulars (i.e., who, what, when, where, and why) of the intervention’s implementation (Century, Rudnick, & Freeman, 2010). As social work professionals continue to embrace evidence-informed practice, this study’s findings suggested that intervention fidelity is still an area in social work research that requires attention. This research extends previous work in this area by providing a more recent assessment of published social work intervention studies.

Overall, the results from this study report slight improvements in attention to fidelity in social work research. Out of all 8 fidelity items examined in this assessment, studies reported a “description of the intervention” most frequently. Although the majority of the studies (81.3%) did provide description of the intervention, the amount of detail given varied. As H. T. Chen and Rossi (1983) noted, more specificity about the intervention allows for a better understanding and interpretation of its effects. The articles that did provide a description of the intervention only offered scant detail, making it nearly impossible to interpret the intervention’s effects (Summerfelt, 2003). Very few articles went as far to give a weekly detailed description of the intervention. Kayser, Feldman, Borstelmann, and Daniels (2010; see Table 2) and Shin (2009; see Table 1) studies offered clear and well-stated examples about how to provide more clear and concise treatment details using a table. Providing ample details not only assists researchers determine what was actually tested and how the intervention’s components contributed to the outcomes but also assists in replicating the intervention (Maynard, Peters, Vaughn, & Sarteschi, 2013). Social work researchers should strive to include as much detail possible about the intervention in their studies. Doing so helps ensure that the future implementation of the intervention is conducted as effectively and successfully as possible (Summerfelt, 2003).

Training personnel responsible for implementing the intervention was mentioned in 59.4% of studies examined. Adequately training providers to deliver interventions is a critical component in intervention fidelity, as the implementers’ level of education, competence, and skills directly effects if the intervention is being executed as was initially planned (Bellg et al., 2004). Ideally, implementer training, use of manuals, and supervision work synchronically to ensure the intervention is being delivered according to uniform protocols (Naleppa & Cagle, 2010; Waltz, Addis, Koerner, & Jacobson, 1993). Interestingly, however, only 50% of these studies reported an explicit description of how the manual was used. As noted by other authors, the extent to which a treatment manual outlines the study’s purpose, goals, session objectives, instructions for conduct, guidelines for specific problems, and other critical components contributes to the enrichment of provider training and standardization of training procedures (Bellg et al., 2004; Moncher & Prinz, 1991). While 62.5% of the assessed studies mentioned manuals, they did not specify if (or how) the manual was actually used in the treatment’s interventions delivery. It is important that social work researchers report as many nuanced aspects of how the intervention was approached, since variations in manual use can alter the effectiveness of the intervention.

The majority of articles (59.4%) in this assessment did not report supervision of treatment providers. Supervision of providers helps facilitate compliance and adherence to the treatment and reduces deviation (drift) from the original model (Bellg et al., 2004). As Perepletchikova and Kazdin (2005) acknowledged, intervention training should be supplemented with ongoing supervision to ensure its treatment consistency. An intervention is likely to be more successful when trainers are provided with regular feedback, support, and encouragement during implementation (Fixsen et al., 2005; Perepletchikova & Kazdin, 2005). As indicated in the literature, in order to ensure adherence to treatment fidelity accurately, it is critical to provide both training and supervision. The extent to which training and supervision was offered varied considerably, as few authors mentioned detailed context, hours, and times of training and supervision (e.g., see Hall et al., 2009).

A “description of the methods used” to monitor the delivery and progress of treatment was reported in 50% of these articles. The assessed studies used a variety of techniques to address fidelity to the intervention, such as fidelity checklists (or treatment integrity forms) adherence to the treatment manual, quality assurance checks, and review of audio-taped sessions, to name a few. Moncher and Prinz (1991) suggested utilizing outside sources, participant reports, and implementer reports as primary steps for verifying fidelity. As Gearing et al. (2011) noted, intervention monitoring is the “heart” of fidelity, because it involves measuring treatment adherence throughout the course of the intervention. Thus, it is important for researchers to give considerable attention to the specific techniques they will use to help ensure that the treatments are being administered consistently and appropriately.

We also assessed how many articles actually used the term “treatment fidelity” (or a similar term) in their methods section. It was found that 65.6% of the studies used the precise term “treatment fidelity” (or a similar term). This finding was an improvement from Tucker and Blythe’s (2008) study, which noted 10% of their N = 128 studies actually used the term “fidelity.” Similar to their study, we also assessed if the tenets of fidelity were mentioned without specifically using the term fidelity. Here, 9.4% discussed these tenets without specifically using the term. Not using the term treatment fidelity (or its synonyms) may suggest that social work researchers are possibly unaware of how paramount fidelity is to the reliability and validity of their study. Consideration of treatment fidelity prior to the implementation of the study should direct researchers to explicitly include the term in their study. We recommend social work researchers include in their study a subsection of their methods content specifically designated to identifying treatment fidelity. As an example, Conner and Fraser (2011) provides a “treatment fidelity” subsection in their article that clearly reports the methods they used for fidelity. Directly addressing this may assist researchers, practitioners, and educators pinpoint the actual methods employed by the researchers to ensure the intervention was delivered as initially planned.

In addition to attending to the aforementioned fidelity protocols, we suggest that social work researchers pretest the intervention with a subset of the study’s sample of participants, to ensure that the intervention is appropriate for their study’s population. Social work interventions are often designed for use with “vulnerable” populations and should aim to “preserve the dignity, worth, and uniqueness of the individual” (Naleppa & Cagle, 2010, p. 679). Surprisingly, none of the studies in this assessment reported pretesting their interventions on a sample before implementation. While the intervention may have internal reliability and validity without pretesting, one cannot generalize the effects of the intervention from one population to another. As indicated by Wodarski (2011), one of our so-called social research conundrums is applying homogenous interventions to heterogeneous populations. Administering a pretest to a similar sample prior to intervention administration may also assist researchers refine their intervention to meet the needs of that specific group.

Additionally, among this data set, we assumed that 100% of the studies would mention procedures relate to ethical issues and concerns. Surprisingly however, only 75% of the reviewed articles mentioned ethics. Institutional Review Board (IRB) approval, receiving informed consent/assent from participants, or other ethical considerations is “standard operating procedures” for most studies with human subjects and should be reported in the study. We were concerned to find that 25% of the studies were published without providing these assurances, although they actually may have done so, but did not mentioned them.

The intent of this study was to assess how social work intervention research studies noted issues related to intervention fidelity. Most important is to encourage the social work research community to treat intervention fidelity more seriously than it has. While our study does suggest the need for more attention to fidelity in social work research, the data may indicate the underreporting of fidelity measures by authors. In that light, we urge all social work journals to mandate this as a necessary requirement for publication. We conclude by offering a suggested authors’ checklist Table 3, to assist the social work authors in attending to fidelity in their research endeavors. Establishing fidelity protocols should be outlined with as much specificity as possible before intervention delivery, and we hope that using the checklist helps make that process easier.

Table 3.

Recommended Authors’ Brief Checklist for Assessing Treatment Fidelity.

The following guidelines are presented as a recommended checklist of 8 items for assessing intervention fidelity. Items answered with a “No” warrant authors carefully reviewing of intervention fidelity of their treatments
	Yes	No
1. Was the intervention pretested on a subset of the sample population?	□	□
2. Was ethics mentioned?	□	□
3. Were treatment manuals or protocols used?	□	□
4. Was training for practitioners offered?	□	□
5. Was supervision for practitioners offered?	□	□
6. Was a detailed description of intervention provided?	□	□
7. Was the term intervention fidelity used?	□	□
8. Were specific details provided about how intervention fidelity was monitored?	□	□

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

*References marked with an asterisk indicate studies included in the meta-analysis. The in-text citations to studies selected for meta-analysis are not preceded by asterisks.

Bellg

A. J.

Borrelli

Resnick

Hecht

Minicucci

Ory

Czajkowski

(2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology, 23, 443–451. doi:10.1037/0278-6133.23.5.443

Bond

G. R.

Evans

Salyers

M. P.

Williams

Kim

H. W.

(2000). Measurement of fidelity in psychiatric rehabilitation. Mental Health Services Research, 2, 75–87.

Borrelli

Sepinwall

Ernst

Bellg

A. J.

Czajkowski

Breger

Orwig

(2005). A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. Journal of Consulting and Clinical Psychology, 73, 852–860. doi:10.1037/0022-006X.73.5.852

Carroll

Patterson

Wood

Booth

Rick

Balain

(2007). A conceptual framework for implementation fidelity. Implementation Science, 35, 85–101. doi:10.1186/1748-5908-2-40

Century

Rudnick

Freeman

(2010). A framework for measuring fidelity of implementation: A foundation for shared language and accumulation of knowledge. American Journal of Evaluation, 31, 199–218.

Chen

(1990). Theory-driven evaluations. Thousand Oaks, CA: Sage.

Chen

H. T.

Rossi

P. H.

(1983). Evaluating with sense: The theory driven approach. Evaluation Review, 7, 283–302.

*Conner

N. W.

Fraser

M. W.

(2011). Preschool Social–Emotional Skills Training a Controlled Pilot Test of the Making Choices and Strong Families Programs. Research on Social Work Practice, 21, 699–711. doi:10.1177/1049731511408115

10.

*Coulter

(2011). Systemic family therapy for families who have experienced trauma: A randomised controlled trial. British Journal of Social Work, 41, 502–519.

11.

Dane

Schneider

(1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45.

12.

Dusenbury

Brannigan

Falco

Hansen

(2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.

13.

Fixsen

D. L.

Naoom

S. F.

Blase

K. A.

Friedman

R. M.

Wallace

(2005). Implementation research: A synthesis of the literature (FMHI Publication #231). Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network. Retrieved from http://ctndisseminationlibrary.org/PDF/nirnmonograph.pdf

14.

Fraser

M. W.

Galinsky

M. J.

(2010). Steps in intervention research: Designing and developing social programs. Research on Social Work Practice, 20, 459–466.

15.

Gambrill

(2011). Evidence-based practice and the ethics of discretion. Journal of Social Work, 11, 26–48. doi:http://dx.doi.org.proxy-remote.galib.uga.edu/10.1177/1468017310381306

16.

Gearing

R. E.

El-Bassel

Gesquiere

Baldwin

Gillies

Ngeow

(2011). Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clinical Psychology Review, 31, 79–88.

17.

Gibbs

Gambrill

(2002). Evidence-based practice: Counterarguments to objections. Research on Social Work Practice, 12, 452–476.

18.

Gidycz

C. A.

Lynn

S. J.

Rich

C. L.

Marioni

N. L.

Loh

Blackwell

L. M.

Mario

N. L.

(2001). The evaluation of a sexual assault risk reduction program: A multisite investigation. Journal of Consulting and Clinical Psychology, 69, 1073–1078.

19.

Gresham

Gansle

Noell

(1993). Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis, 2, 257–263.

20.

*Hall

J. A.

Sarrazin

M. S. V.

Huber

D. L.

Vaughn

Block

R. I.

Reedy

A. R.

Jang

(2009). Iowa case management for rural drug abuse. Research on Social Work Practice, 19, 407–422. doi:10.1177/1049731509331925

21.

Hodge

D. R.

Lacasse

J. R.

Benson

(2012). Influential publications in social work discourse: The 100 most highly cited articles in disciplinary journals: 2000–09. British Journal of Social Work, 42, 765–782.

22.

Holosko

M. J.

(2006). A suggested authors’ checklist for submitting manuscripts to Research on Social Work Practice. Research on Social Work Practice, 16, 449–454.

23.

Holosko

M. J.

(2009). What types of designs are we using in social work research and evaluation? Research on Social Work Practice, 20, 665–673.

24.

Holosko

M. J.

Leslie

(1998). Obstacles to conducting empirically based practice. In Wodarski

Thyer

(Eds.), Handbook of empirical social work practice issues (Vol. 2, pp. 433–453). New York, NY: John Wiley.

25.

Holosko

M. J.

Thyer

(2011). Pocket glossary for commonly used research terms. Thousand Oaks, CA: Sage.

26.

Holosko

M. J.

Thyer

Danner

J. E.

(2009). Ethical guidelines for designing and conducting evaluations of social work practice. Journal of Evidence-Based Social Work, 6, 348–360.

27.

*Jacobson

J. M.

Osteen

P. J.

Sharpe

T. L.

Pastoor

J. B.

(2012). Randomized trial of suicide gatekeeper training for social work students. Research on Social Work Practice, 22, 270–281. doi:10.1177/1049731511436015

28.

*Jones

L. V.

(2009). Claiming your connections: A psychosocial group intervention study of black college women. Social Work Research, 33, 159–171.

29.

*Kayser

Feldman

B. N.

Borstelmann

N. A.

Daniels

A. A.

(2010). Effects of a randomized couple-based intervention on quality of life of breast cancer patients and their partners. Social Work Research, 34, 20–32.

30.

*Langhorst

D. M.

Choi

Y. J.

Keyser-Marcus

Svikis

D. S.

(2012). Reducing sexual risk behaviors for HIV/STDs in women with alcohol use disorders. Research on Social Work Practice, 22, 367–379. doi:10.1177/1049731512441683

31.

*Lanier

Kohl

P. L.

Benz

Swinger

Moussette

Drake

(2011). Parent-child interaction therapy in a community setting: examining outcomes, attrition, and treatment setting. Research on Social Work Practice, 21, 689–698. doi:10.1177/1049731511406551

32.

*Leitch

M. L.

Vanslyke

Allen

(2009). Somatic experiencing treatment with social service workers following Hurricanes Katrina and Rita. Social Work, 54, 9–18.

33.

*Leung

Tsang

Heung

Yiu

(2009). Effectiveness of Parent-child Interaction Therapy (PCIT) among Chinese families. Research on Social Work Practice, 19, 304–313. doi:10.1177/1049731508321713

34.

Maynard

Peters

Vaughn

Sarteschi

(2013). Fidelity in after-school program intervention research: A systematic review. Research on Social Work Practice, 23, 613–623. doi:10.1177/1049731513491150

35.

McArthur

Riosa

Preyde

(2012). Treatment fidelity in psychosocial intervention for children and adolescents with comorbid problems. Child & Adolescent Mental Health, 17, 139–145.

36.

McNeece

Thyer

B. A.

(2004). Evidence-based practice and social work. Journal of Evidence-Based Social Work, 1, 5–23. doi:10.1300/J394v01n0102

37.

McRoy

R. G.

Flanzer

J. P.

Zlotnik

J. L.

(2012). Building research culture and infrastructure. New York, NY: Oxford University Press.

38.

Moncher

F. J.

Prinz

R. J.

(1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247–266.

39.

Mowbray

C. T.

Holter

M. C.

Teague

G. B.

Bybee

(2003). Fidelity Criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340. doi:10.1016/S1098-2140(03)00057-2

40.

Naleppa

M. J.

Cagle

J. G.

(2010). Treatment fidelity in social work intervention research: A review of published studies. Research on Social Work Practice, 20, 674–681.

41.

*Painter

(2009). Multisystemic therapy as community-based treatment for youth with severe emotional disturbance. Research on Social Work Practice, 19, 314–324. doi:10.1177/1049731508318772

42.

Perepletchikova

Kazdin

A. E.

(2005). Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice, 12, 365−383.

43.

Rubin

Parrish

(2007). Problematic phrases in the conclusions of published outcome studies: Implications for evidence-based practice. Research on Social Work Practice, 17, 334–347.

44.

Salend

S. J.

(1984). Integrity of treatment in special education research. Mental Retardation, 22, 309–315.

45.

Shaw

(2005). Is social work research distinctive? Social Work Education, 26, 659–669.

46.

*Shin

S. K.

(2009). Effects of a solution-focused program on the reduction of aggressiveness and the improvement of social readjustment for Korean youth probationers. Journal of Social Service Research, 35, 274–284.

47.

Silberschatz

Fretter

P. B.

Curtis

J. T.

(1986). How do interpretations influence the process of psychotherapy? Journal of Consulting and Clinical Psychology, 54, 646–652.

48.

*Striley

C. W.

Nattala

P. P.

Abdallah

A. B.

Dennis

M. L.

Cottler

L. B.

(2013). Enhanced case management versus substance abuse treatment alone among substance abusers with depression. Social Work Research, 37, 19–25.

49.

Summerfelt

W. T.

(2003). Program strength and fidelity in evaluation. Applied Developmental Science, 7, 55–61.

50.

Thyer

B. A.

Myers

L. L.

(2007). A social worker’s guide to evaluating practice outcomes. Alexandria, VA: Council on Social Work Education.

51.

*Timmer

S. G.

Zebell

N. M.

Culver

M. A.

Urquiza

A. J.

(2010). Efficacy of adjunct in-home coaching to improve outcomes in parent—child interaction therapy. Research on Social Work Practice, 20, 36–45. doi:10.1177/1049731509332842

52.

Tucker

A. R.

Blythe

(2008). Attention to treatment fidelity in social work outcomes: A review of the literature from the 1990s. Social Work Research, 32, 185–190.

53.

*Valdez

Cepeda

Parrish

Horowitz

Kaplan

(2013). An adapted brief strategic family therapy for gang-affiliated Mexican American adolescents. Research on Social Work Practice, 23, 383–396. doi:10.1177/1049731513481389

54.

Waltz

Addis

M. E.

Koerner

Jacobson

N. S.

(1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620–630. doi:10.1037/0022-006X.61.4.620

55.

Wodarski

J. S.

(2011). The social work practice research conundrum. Journal of Human Behavior in the Social Environment, 21, 577–600. doi:10.1080/10911359.2011.580249

56.

*Zajicek-Farber

M. L.

(2010). Building practice evidence for parent mentoring home visiting in early childhood. Research on Social Work Practice, 20, 46–64. doi:10.1177/1049731509333172