Notes From the Field: Applying the Consolidated Framework for Implementation Research in a Qualitative Evaluation of Implementation of the Queensland Aboriginal and Torres Strait Islander Brief Intervention Training Program

Abstract

This paper describes the applicability of the Consolidated Framework for Implementation Research (CFIR) to the qualitative evaluation of the implementation of the Queensland Aboriginal and Torres Strait Islander Brief Intervention Training Program, the B.strong Program. Interviews were conducted with 20 B.strong Program trainees and four health service managers from eight purposively sampled Indigenous primary health care services in Queensland to collect their perceptions of the B.strong Program implementation. The 26 constructs of the CFIR were used to guide data collection and analysis. Additional constructs were developed for two program implementation aspects, “quality improvement” and “cultural suitability.” Findings are presented from the application of the CFIR to the evaluation of the implementation of a brief intervention training program in the Australian Indigenous context. While demonstrating the applicability of the CFIR in this evaluation, this study also highlights that it may require modification, to ensure identification of the different contextual factors that influence program implementation.

Keywords

consolidated framework for implementation research Australian indigenous brief intervention training program evaluation chronic disease

Background

The Consolidated Framework for Implementation Research (CFIR) was developed specifically for conducting systematic assessment of multilevel implementation contexts to identify factors that might influence intervention implementation and effectiveness (Damschroder et al., 2009). It is composed of five domains: the intervention characteristics, outer setting, inner setting, characteristics of the individuals, and the process of implementation (CFIR Research Team-Center for Clinical Management Research, 2020). The five domains incorporate 26 constructs and 13 subconstructs that reflect the factors most likely to influence implementation of interventions.

The Queensland Aboriginal and Torres Strait Islander Brief Intervention (BI) Training Program (B.strong Program) funded by Queensland Health and implemented in the state of Queensland, Australia, aimed to build the capacity, skills and confidence of Queensland’s frontline health and community workers to deliver smoking cessation, nutrition and physical activity BIs to their Indigenous clients. From June 2017 to June 2020, two Indigenous trainers delivered 107 workshops in urban, rural and remote areas of Queensland. More than 1,250 Indigenous health professionals and other health and community professionals were trained in the workshops and a series of online modules.

Australia has implemented several BI training programs specifically designed for Aboriginal and Torres Strait Islander (referred to as Indigenous in this paper) health professionals and other health professionals to deliver BIs to Indigenous clients in the primary health care (PHC) sector (Martin et al., 2019; NSW Department of Health, 2010; Queensland Health, 2007). Whilst complex inter-dependent factors are known to impact program implementation in health care (Coles et al., 2017), little knowledge exists about what factors impact on the uptake and adoption by health professionals and health services of programs that train health workers in BI delivery (Sebastian et al., 2020). This study assessed the applicability of the CFIR framework to the qualitative evaluation of the implementation of the B.strong Program and the usefulness of the framework in identifying the facilitators or barriers to program implementation.

Methods

A purposive sampling approach was used in the study (Tolley et al., 2016). To ensure geographic representation, we purposively selected 33 PHC services that were representative of metropolitan, rural and remote locations in Queensland whose staff had attended B.strong workshops from eight of the 15 Queensland Hospital and Health Service regional geographic health administration areas. As of 28 February 2020, all B.strong workshop trainees from the 33 PHC services and their supervisors (n = 152) were invited, through their manager, to participate in the study. Twenty-four participants consented and completed interviews. Twelve participants declined, and 116 did not respond to three emails, either individually or through their manager. Most high-level themes relating to the study aims were identified after coding the initial in-depth interviews. Eventually, little new information was provided, and it was apparent that data saturation was reached after 24 interviews, and recruitment ceased.

Interviews were conducted using separate semi-structured interview guides developed for the two interview groups: health professionals, and managers of the trainees. All interviews were conducted by the first author from June 2019 to April 2020. The interviews were conducted either face-to-face (n = 8) or via video conference (n = 14) and teleconference (n = 2). The duration of the interviews ranged from 30 to 40 minutes. Interviews were voice-recorded with the participants’ permission. Ethics approvals for the study were granted from two Human Research Ethics Committees each in Northern Territory and Queensland. Additionally, site-specific approvals were obtained from four Queensland Hospital and Health Services.

Assessing Applicability of CFIR

For each of the constructs in the five domains of the CFIR, a valence rating of the strength of the reference (0, 1 or 2), and the relative influence (positive or negative) was created. The valence rating criteria, as used in previous CFIR studies (Damschroder & Lowery, 2013; Soi et al., 2018), considered the nature, numbers and strength of the comments.

One of the study objectives was to identify the use of continuous quality improvement (CQI) processes and strategies at the level of PHC services to support BIs with clients. The construct of “reflecting and evaluating” in the CFIR domain of “process,” and the sub-construct, “goals and feedback” within the construct of implementation climate (inner setting) incorporated some elements of quality improvement. However, the efforts at the organizational level to consistently apply and sustain the program were not deemed to be fully captured by these constructs. Similarly, none of the CFIR constructs were identified as comprehensively incorporating the influence of cultural suitability in the program’s acceptance, uptake and ongoing application. Although the construct of “culture” in the “inner setting” domain referred to the norms and values of the organization, it did not reflect how the program addressed cultural suitability. Hence, another construct, “Cultural Suitability” was added to the domain of “intervention characteristics.”

Results

Of the eight sites participating in the study, three were in metropolitan locations, four in rural areas and one in a remote location. Out of 24 participants,12 (50%) were Indigenous. A summary of the CFIR constructs and their strength and influence is given in Table 1. Eight of the main 26 CFIR constructs were identified as having little relevance to the implementation of the B.strong program as they generated no participant feedback. All the important themes identified from the interviews could be coded against the remaining 20 constructs, including the two additional constructs.

Table 1.

Rating of Strength and Influence of CFIR Constructs.

Domain	Constructs^a	Plain Language Definition	Valence Rating (Strength and Influence)
Intervention Characteristics	Intervention source	Whether the intervention is externally or internally developed.	+ 2
	Evidence, strength and quality	Belief that the intervention will generate the desired outcomes.	+ 2
	Relative advantage	The advantage of implementing the intervention versus an alternative solution.	+ 2
	Design, quality and packaging	Perceptions on how the intervention is bundled and presented.	+ 2
	Cultural suitability^b	The training, resources and supports of the program acknowledge Indigenous ways of living and reflect on their experiences, environment and world view.	+ 2
Outer Setting	Patient needs & resources	The extent to which patient needs, as well as barriers and facilitators to meet those needs are accurately known and prioritized by the organization.	+ 2
Outer Setting	Cosmopolitanism	The degree to which an organization is networked with other external organizations.	−1
Inner Setting	Structural characteristics	The social architecture, age, maturity, and size of the organization.	+ 1
	Culture	Norms and values of the organization.	+ 2
	Implementation climate (Sub-themes: Compatibility, Relative priority, Incentives, Goals and feedback, Learning climate)	The organization’s capacity for change (the extent to which use of that intervention will be rewarded, supported, and expected), and the individual’s involvement in the intervention.	+ 2
	Readiness for implementation (Sub-themes: Leadership engagement, Available Resources)	The organization’s commitment to its decision to implement the intervention.	+ 2
	Quality improvement (Implement, Audit, Feedback and Review)^b	The organization’s ability to monitor the implementation of the program, review, feedback and refine to improve processes and outcomes	−2
Characteristics of Individuals	Knowledge and beliefs about the intervention	Individual’s attitudes to and knowledge about the intervention.	+ 1
	Self-efficacy	Individual’s belief in their own capabilities to apply the intervention to achieve the desired outcomes.	+ 1
	Individual stage of change	The ability of the individual to sustain use of the intervention.	+ 1
	Other personal attributes	Personal and professional attributes of the individual that may help with implementation of the intervention.	+ 2
Intervention Process	Planning	The tasks for implementing an intervention are developed in advance, and the quality of those schemes or methods.	+ 2
	Engaging (Sub-themes: Opinion leaders, Champions)	Attracting and involving appropriate individuals in the implementation.	+ 2
	Executing	Carrying out or accomplishing the implementation according to plan.	+ 2
	Reflecting and evaluating	Quantitative and qualitative feedback about the progress and quality of implementation accompanied with regular personal and team reflections.	+ 2

^a Eight of the CFIR constructs were identified as having little relevance as they generated no participant feedback, and are not included: Adaptability, Complexity, Trialability & Cost (from Intervention Characteristics), Peer pressure & External policies and incentives (Outer Setting), Networks and communications (Inner Setting), Individual identification with organization (Characteristics of Individuals).

^b Additional constructs created for this study.

Only two constructs had a negative influence on implementation: quality improvement and cosmopolitanism (which refers to the degree to which an organization is networked with other external organizations). The former reflected health services’ poor efforts in ongoing quality improvement in relation to B.strong program implementation in their health service. Four constructs, structural characteristics (inner setting), participants’ knowledge and beliefs about the intervention, individual stage of change, and self-efficacy (characteristics of individuals) had weaker positive influence, rated as +1. All other constructs were rated as having stronger positive influence (+2). The CFIR domains that were perceived as having the most influence on the implementation of the B.strong Program were: the characteristics of the program, implementation processes and inner setting, which all received +2 or −2 valence ratings for four or more constructs.

The domain of the characteristics of the program was perceived to be an important influence on implementation, with all constructs rated as +2. Probes under different constructs of this domain enabled identification of the importance of having a credible source, individual tailoring of the program to suit the characteristics of the organizations, client and trainee populations, and the design, quality and relative advantage of the program.

The newly introduced construct of ‘cultural suitability’ allowed for deeper probing into the versatility of the different training components in ensuring their suitability for the Indigenous trainee population and the client population. The feedback from Indigenous health professionals generally highlighted the importance of visuals in the training materials, the mode of delivery of the workshop and the language used in printed client materials. The cultural appropriateness of the program was found equally important by non-Indigenous health professionals. The creation of the new construct “quality improvement” helped with understanding measures taken by the PHC services to maintain BI skills among their workforce and to improve the quality and quantity of BIs delivered.

Discussion

The development and application of two additional constructs, “quality improvement” and “cultural suitability” proved valuable. Our experience shows that the CFIR may not cover all the factors that can impact on program implementation, particularly when a strong cultural specificity underlies the program, and there are specific elements to the implementation of new programs/evidence in the setting. The development of the new construct of “quality improvement” addressed the latter issue, reflecting the recent focus on CQI in Australian Indigenous PHC (Gardner et al., 2018). This was similar to the approach adopted in a study conducted on the implementation of a ‘Screening, Brief Intervention, and Referral to Treatment’ (SBIRT) program in South African emergency center settings (van der Westhuizen et al., 2019).

The B.strong Program was developed as a culturally appropriate training program. This was a priori information the researchers had, which led to the development of the new construct “cultural suitability.” A similar approach was taken in an evaluation by Green (2018) using the CFIR. The analysis focused on examining the cultural relevancy of SBIRT implementation for an African American community (Green, 2018). Across the stakeholder groups, Green identified two emergent themes, “trust” and “communication” as addressing conditions that cut across more than one of the CFIR domains. In our study, if CFIR constructs had been applied as initially developed, important data on the influence of cultural suitability of the program would have been missed.

Adding to the evidence from previous studies, this study highlights that the application of the CFIR may require adaptation to match the characteristics of the program studied and the population targeted. If tailored appropriately, the CFIR is a very efficient tool for systematically teasing out the different contextual factors that influence program implementation. This study highlights the importance of evaluating the cultural suitability of programs that are designed for use with Indigenous clients. Applying CFIR in all areas of the qualitative evaluation including the development of the data collection tools and analysis of the findings was found to be advantageous. This study provides an example of how to tailor CFIR for use in evaluating the implementation of capacity development programs, particularly of training programs for health professionals in the Australian Indigenous context.

A limitation of the study is that, as there was a relatively small study sample size, it may not reflect the views of all trainees in the B.strong training program across all Queensland regions. We ensured adequate representation of men and women, different types of health service organization, and participants from metropolitan, rural and remote locations. This study is valuable in providing insights into the application of the CFIR to the evaluation of BI training program implementation, and findings could be applicable to other programs for Indigenous population groups.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by an Australian Government Research Training Program (RTP) scholarship for the lead researcher through the Charles Darwin University, Darwin, Australia, and by a doctoral scholarship for the lead researcher provided by the Centre for Research Excellence in Integrated Quality Improvement, funded by the National Health and Medical Research Council (ID 1078927).

ORCID iD

Saji Sebastian

References

CFIR Research Team-Center for Clinical Management Research. (2020). Constructs. https://cfirguide.org/constructs/

Coles

Wells

Maxwell

Harris

F. M.

Anderson

Gray

N. M.

Milner

MacGillivray

(2017). The influence of contextual factors on healthcare quality improvement initiatives: What works, for whom and in what setting? Protocol for a realist review. Systematic Reviews, 6(1), 168.

Damschroder

L. J.

Aron

D. C.

Keith

R. E.

Kirsh

S. R.

Alexander

J. A.

Lowery

J. C.

(2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. https://doi.org/10.1186/1748-5908-4-50

Damschroder

L. J.

Lowery

J. C.

(2013). Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implementation Science, 8(1), 51.

Gardner

Sibthorpe

Chan

Sargent

Dowden

McAullay

(2018). Implementation of continuous quality improvement in Aboriginal and Torres Strait islander primary health care in Australia: A scoping systematic review. BMC Health Services Research, 18(1), 541.

Green

H. D.

(2018). A community-based evaluation of screening, brief intervention, and referral to treatment (SBIRT) for the black community. Qualitative Health Research, 28(3), 418–432.

Martin

Dono

Stewart

H. B.

Sparrow

Miller

Roder

Bowden

(2019). Evaluation of an intervention to train health professionals working with Aboriginal and Torres Strait Islander people to provide smoking cessation advice. Australian and New Zealand Journal of Public Health, 43(2), 156–162. https://doi.org/10.1111/1753-6405.12879

NSW Department of Health. (2010). The Final report of the NSW SmokeCheck aboriginal tobacco prevention project 2007-2008. NSW Department of Health.

Queensland Health. (2007). SmokeCheck evaluation report 2006. Queensland Government.

10.

Sebastian

Thomas

D. P.

Brimblecombe

Majoni

Cunningham

F. C.

(2020). Factors impacting on development and implementation of training programs for health professionals to deliver brief interventions, with a focus on programs developed for indigenous clients: A literature review. International Journal of Environmental Research and Public Health, 17(3), 1094.

11.

Soi

Gimbel

Chilundo

Muchanga

Matsinhe

Sherr

(2018). Human papillomavirus vaccine delivery in Mozambique: Identification of implementation performance drivers using the consolidated framework for implementation research (CFIR). Implementation Science, 13(1), 1–12.

12.

Tolley

Ulin

P. R.

Mack

Robinson

E. T.

Succop

S. M.

(2016). Qualitative methods in public health. A field guide for applied research (2nd ed.). Wiley.

13.

van der Westhuizen

Myers

Malan

Naledi

Roelofse

Stein

D. J.

Lahri

Sorsdahl

(2019). Implementation of a screening, brief intervention and referral to treatment programme for risky substance use in South African emergency centres: A mixed methods evaluation study. PLoS One, 14(11), e0224951.