Statistical Synthesis of Contextual Knowledge to Increase the Effectiveness of Theory-Based Behaviour Change Interventions

Introduction

It is widely accepted that the uptake of new, research-based innovations into health care can be slow and variable and that implementation efforts need to be expended in order for the anticipated benefit for patients to be achieved. Guidance has been developed for increasing the adoption of new innovations through the development and delivery of behaviour-change interventions. ¹ This guidance emphasizes the importance of conducting an exploration of the local context in order to develop tailored interventions, piloting and modelling intervention components before rolling them out, and using mixed methods of evaluation. Studies testing the efficacy of behaviour-change interventions give mixed findings. A systematic review which explored the effectiveness of a variety of interventions, such as audit and feedback, computerized reminders, and educational outreach found that whilst all interventions have some impact, this is typically small and variable. ² At least some of this variability can be explained by the mediating effects of different settings and contexts for implementation.

Our aim was to describe current practice in the development of behaviour-change interventions and outline a component in the development of tailored behaviour-change interventions that has been largely overlooked to date: the synthesis of contextual data to identify targets for change. We will illustrate how we have attempted to address this oversight with reference to our research programme (summarized in Box 1).

Factors influencing the use of evidence-based recommendations

There are many factors that influence the uptake of evidence-based recommendations in health care. For example, 41 different factors were identified from one review of the literature, ³ which were classified according to whether they were characteristics of: the innovations themselves; the health professionals expected to adopt the innovation into practice; or the organization in which the innovation is to be introduced.

The importance of considering the characteristics of innovations is often over-looked. ⁴ An innovation's complexity, trialability, observability, relative advantage over current practice, and compatibility with existing norms and standards, have been suggested as features that influence the diffusion of innovations into practice. ⁵ Research exploring the characteristics of health professionals has identified psychological constructs, including fear of losing professional autonomy ⁶ and self-efficacy regarding the skills required to adopt the innovation. ⁷ Organizational influences include concepts, such as organizational culture, ⁸ and time and fiscal constraints, including scarcity of resources available to implement change in practice. ²

This focus on different characteristics has been expanded upon by other researchers. For example, economic and administrative factors, as well as patient beliefs or behaviour, have been suggested to be important. ⁹ The impact of social influences upon health professionals, such as social networks,^10–12 team work and team culture has also received attention, ¹³ taking the scope of coverage and interest beyond purely characteristics of the health professionals'.

Exploring key factors influencing innovation adoption: current practice

Given the many factors influencing the adoption of innovations into practice, commentators often recommend that implementation strategies be tailored to the local context, ideally with the exploration of the local context guided by theory. ¹ If a theoretical approach to exploring the local context is adopted, selecting a theory-base can be a difficult decision, with no systematic way of choosing one theory over another. ¹⁴ Often there is little justification provided for why one particular theory has been used over another. ¹⁵ Resolving these uncertainties presents a considerable barrier to developing tailored and theory-based implementation strategies. Some researchers have used multiple theoretical constructs, domains, or frameworks, rather than single theories, to underpin exploration of the local context.^16,17

Our research programme is an example of an approach to research implementation that uses a theoretical framework rather than a single theory. The programme is built around the operationalization of key facets of a theoretical framework developed from a synthesis of research on dissemination and implementation. The framework places particular emphasis on the influence of health professionals' perceptions of and attitudes towards the innovation, their degree of engagement in social networks relating to clinical practice, and the culture of the setting in which they work. ¹¹ In addition, it highlights the importance of broader (and harder to influence) variables such as the sociopolitical climate and the size, maturity and decentralization of the organization. Frameworks such as this encourage an approach that recognises hierarchical levels of influence. If a single theory was used, our focus would more likely fall upon characteristics of the health professionals' alone (e.g. through use of social cognitive theory), ¹⁸ social networks (e.g. through use of diffusion of innovation theory), ¹⁰ or the importance of organizational culture (e.g. through use of, for example, competing values framework). ¹⁹

In the area of patient safety it has long been recognized that health care behaviours exist within a multidimensional and systemic context – with ‘upstream’ and ‘latent’ variables impacting on the ‘downstream’ behaviours, processes and outcomes manifest in adverse events and the quality of care.^20,21 This view is not unique to patient safety. The theoretical framework that we have adopted encompasses both theoretical (rationale) and empirical (effects) components and offers some key advantages. It is not overtly prescriptive; ¹¹ requires testing through primary research; is multidimensional, recognizing the differing components of change (adoption, diffusion, implementation and sustainability); has the ‘transformative’ ²² and temporal dimensions of change designed into the review that underpins it; and is characterised by a breadth of focus in, and evidence for, the factors that influence change.

Synthesizing data

Typically in an implementation study, having explored the local context in which the innovation is to be implemented, researchers then synthesise the information collected to identify which factors to target with tailored implementation strategies. This synthesis of data will sometimes involve combining qualitative and quantitative data, requiring skills in mixed methods approaches. However, even if only quantitative data have been collected, it may be necessary to synthesize data (e.g. health professionals' attitudes towards the targeted innovation, and data about team or organizational culture). Synthesis of data is an integral component in the development of tailored implementation strategies. If the data collected is analysed inappropriately, important factors influencing the adoption of innovations into practice may be overlooked and the tailored implementation strategies may fail to target the main factors influencing adoption of the innovation.

Possible reasons for unsuccessful behaviour-change interventions include the difficulty of selecting a suitable theory-base where a theoretical approach to exploring the local context has been adopted^23,24 lack of attention into how to link intervention techniques with theoretical constructs,^24,25 and unsystematic development of the intervention. ² Techniques such as intervention mapping, ²⁶ guidance on developing behaviour-change interventions, ²⁷ and taxonomies of behaviour-change techniques ²⁸ can provide a useful framework for developing tailored and systematic behaviour-change interventions.

We do not currently know whether introducing change at one particular hierarchical level is likely to be more influential than introducing change at another level. It has been suggested that changes introduced at one level may lead to changes at another level, ²⁷ adding to the complexity of evaluating the impact of tailored interventions. With the current lack of evidence regarding the cost effectiveness of delivering tailored interventions and the suggestion that as a consequence of this, lower cost tailored interventions may be most feasible, ²⁹ there is a strong rationale for exploring how to tailor interventions in the most cost effective way. Interventions tailored to different hierarchical levels of influence may be beneficial.

Recommendations for data synthesis using multilevel modelling

Multilevel modelling enables separation of the effects of influences at different levels within hierarchically organized data. As a tool for efficiently synthesizing quantitative data, it can be used to analyse data generated from theory-based explorations of the local context. However, it can also be used to synthesise data from a-theoretical diagnostic analyses too. In our research programme, which has adopted a theory-based approach, survey data are to be collected about health professionals' perceptions of and attitudes towards a specified innovation, their degree of engagement in social networks around the clinical area of the innovation (the individual level), and the culture of their health care team (the team level).

The analysis will enable us to identify the degree of variation in our dependent variable attributable to individual differences as well as that attributable to team differences. Due to the difficulty of obtaining an objective measure of health professionals' adoption of the innovation at the individual level, we are using two self-reported adoption measures: adoption and intention to adopt the innovation. The degree of co-variation between the two will be explored as past behaviour is often the best predictor of future behaviour. ³⁰ We acknowledge that self-report measures are subject to social desirability bias, with the tendency for health professionals to over-estimate their adoption levels. ³¹ However, our focus is on the degree of variation in self-reported adoption accounted for in the model, rather than whether health professionals adopt the innovation or not as a categorical outcome measure. Therefore, assuming that responses to our two self-report measures are normally distributed, the planned multilevel model can explore the relative influence of individual and team differences upon adoption rates. Objective data on adoption is available though only at an aggregated level, a difficulty acknowledged by other researchers. ³² We will make use of these data when evaluating the effectiveness of the intervention, using interrupted time series analysis to estimate intervention effectiveness and duration of effect at an aggregated level. This will be in addition to individual level evaluation measures provided by re-administering the survey to explore changes in attitudes, social networks and team culture, and by conducting qualitative interviews with health care professionals to explore the perceived impact and effectiveness of the intervention. ³³

Subsidiary analyses within the multilevel model will enable us to control for, and compare, fixed characteristics, such as the influence of professional group, with the option of controlling for key sociodemographic variables, such as number of years since qualifying. The information derived from such a multilevel model will enable us to tailor the behaviour-change intervention more specifically, knowing at which hierarchical level to focus the intervention, and to which groups. ²⁷

Multilevel modelling has the potential to bridge the gap to some extent between qualitative and quantitative methods through taking contextual information into account in quantitative analyses. ³⁴ Through taking account of the hierarchies in the data simultaneously, multilevel modelling enables the context of health care to be taken into account more realistically. As a data synthesis technique, it has been used in other implementation studies of the adoption of electronic health records in Canada ³⁵ and lifeguards' adoption of recommendations for a skin cancer prevention programme. ³⁶ However, their focus is upon evaluating the impact of a theory-based implementation strategy compared with a non theory-based strategy, rather than synthesizing data collected from a diagnostic analysis.

An important strength of multilevel modelling, particularly when used to synthesise contextual information, is its robustness and flexibility. It can handle both continuous and categorical outcome measures, ³⁴ multiple outcome measures (multivariate multilevel modelling), can cope with missing data, and can be used to control for key social demographic variables. Each of these strengths is particularly pertinent to implementation studies, with recommendations for multiple evaluation measures, ¹ the potential for participant attrition and subsequent missing data at different phases of the research programme, and the importance of controlling for social demographic variables in analyses. The strengths of using multilevel modelling, however, have to be offset against the complexity of the technique. The technique also requires a larger sample size than more commonly used techniques, such as multiple regression. The choice of technique for data synthesis should not act as a barrier to selecting certain innovations (for instance those that apply to a smaller sample of health professionals) to focus implementation efforts upon.

Future research

Future research in this area should aim to explore further the potential for multilevel modelling to synthesise contextual information collected during a diagnostic analysis, as well as other synthesis techniques, including those for combining qualitative and quantitative data. Such research will be critical in bringing a much needed focus on the process of data synthesis in implementation research. Future research focussed on such data synthesis techniques, combined with research that develops and evaluates behaviour-change taxonomies could make a significant contribution to enhancing the effectiveness of tailored behaviour-change interventions whilst furthering understanding of what makes some behaviour-change interventions more effective than others.