Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Abstract

Background

Many cluster randomised trials of individual-level interventions are at risk of bias, mostly due to identification and recruitment biases, which would not feature under individual randomisation. This bias arises when participants are recruited into the trial with knowledge of the treatment arm they’ve been allocated to. These trials are also at risk of other biases including lack of clear documentation of primary outcome and apparent unconcealed randomisation. Many of these other risks are easily surmountable, and might reflect poor reporting rather than poor practice, or signal lack of knowledge.

Objectives

To determine whether investigators are aware of the common sources of risks of bias in cluster trials, and methods to mitigate these risks. We will explore why these biases occur in light of what is known about preventing them and what enables these risks to be mitigated. We will also explore the reasons for adopting cluster randomisation.

Setting

Principal investigators, statisticians and trialists with experience in conducting cluster randomised trials identified via an existing sampling frame of 104 contemporary cluster trials of individual-level interventions.

Methods

A realist approach will be used to underpin this study. We will conduct semi-structured interviews over Zoom to identify the rationale behind using a cluster trial when the intervention is at the level of the individual, and knowledge of the importance of blinding for those identifying and recruiting participants. Data collected from the interviews will be analysed using thematic analysis. Themes within the data will be mainly captured with the research question in mind but will remain flexible.

Anticipated results

We aim to understand the reasons why cluster trials are being conducted when they are at risk of bias. It is hoped that understanding these reasons will provide useful information so future cluster trialists are aware of these risks, and provided with practical solutions.

Keywords

risk of bias semi-structured interview recruitment cluster trials

Introduction

In individually randomised trials, patients are randomly allocated to different interventions, henceforth referred to as treatment or control conditions. Rather than randomising individual patients, cluster randomised trials (CRTs) randomise entire clusters (such as wards, schools or social groups) to treatment or control conditions (Eldridge & Kerry, 2012; Murray, 1998). Cluster randomisation is the only feasible randomised design when the intervention is delivered to the entire cluster (for example, installing a new computer system). When the intervention is delivered directly to the individual (hence referred to as an individual-level intervention), such as a drug to prevent post-partum haemorrhage, both cluster and individual randomisation are theoretically possible.

We recently identified that all but a few of a random-sample of 40 trials of individual-level interventions evaluated using a cluster randomised design were assessed at high risk of bias (Easter et al., 2021). The causes of these biases were multi-factorial, but one prominent cause was due to identification and recruitment biases. We identified that the vast majority of cluster trials of individual-level interventions identify or recruit research participants after randomisation of clusters to treatment conditions. In most it was deemed possible that selection of individual participants could be affected by knowledge of the intervention; with some showing evidence of baseline imbalance on individual-level characteristics across treatment arms. These risks pose a serious threat to the validity of the findings from cluster trials.

We identified other possible risks of bias, not necessarily specific to the use of cluster randomisation. However, some of these apparent risks of bias might reflect poor reporting rather than poor practice. For example, many trials were assessed as not implementing randomisation in a way that is clearly concealed. This is something which is easily correctable by use of an independent statistician, or other acceptable concealed randomisation method; but might also indicate a feature that is simply not being reported well. Other risks of biases included a failure to clearly specify or document the primary outcome or primary assessment time: a small minority of trials neither publish a protocol paper, nor pre-register the trial on a trial registration database. In these trials, there is no possible way to verify any pre-specified primary outcome and these trials will be at risk of selective reporting. Some studies were assessed as being at risk of bias due to measurement of the outcome, which might be overcome by using blind outcomes assessors of subjective outcomes.

This assessment of risk of bias confirms to a large extent what is already known – that cluster trials are at risk of identification and recruitment bias (Brierley et al., 2012; Diaz-Ordaz et al., 2013; Eldridge et al., 2009; Giraudeau & Ravaud, 2009; Puffer et al., 2003). This most recent research confirms that the risk is particularly important in studies with post randomisation recruitment, a feature which is common in cluster trials of individual-level interventions (Eldridge et al., 2008; Froud et al., 2012). Moreover, being particularly prevalent in cluster trials of individual-level interventions raises questions around why these evaluations are using cluster randomisation, when individual randomisation would have been theoretically possible.

Objectives

Whilst there are recommendations to help investigators mitigate these risks of bias (including recruitment by someone blind to the allocation), it appears these recommendations are rarely followed. There is therefore an urgent need to understand why this is happening, so as to actively identify solutions that are amenable in practice. We focus on individual-level interventions, as these are a specific type of intervention which are associated with a particularly high risk of bias when using cluster randomisation, and which in theory would appear to be amenable to evaluation using individual randomisation. Our objectives are therefore to understand, and explore:

1. The rationale behind using the cluster randomised design when the intervention is delivered at the level of the individual.

2. The knowledge of the importance of the timing of the identification and recruitment of participants with respect to randomisation of clusters.

3. The knowledge of the importance of the blinding of those who identify and recruit participants.

4. The barriers and facilitators for a successful uptake of recommended practice.

Methods

Overview

In brief, we propose to conduct semi-structured interviews with investigators who have been involved in cluster trials which have evaluated individual-level interventions. Eligible interviewees will be identified using a sampling frame. The semi-structured interviews will be completed virtually through a video calling system, Zoom, using a semi-structured topic guide, conducted by one interviewer and will be recorded and transcribed verbatim. Prior to the main interviews we plan to conduct a pilot assessment of the recruitment and interview process and approach to data analysis. A separate set of similar trials and corresponding participants will be used for this pilot phase. A realist approach will be adopted throughout the study and thematic analysis will be undertaken for interpretation of the data. An inductive approach will be taken to allow emergent themes to develop, and without using preconceived themes to drive the assessment. Further details are given in the following sections.

Identification of the Sampling Frame

A sampling frame of contemporary cluster randomised trials of individual-level interventions will be constructed. We will use a convenience sample of trials (Marshall, 1996) ascertained from another review of cluster randomised trials published between 2014 and 2019 (Zhang et al., 2021). From this review we will include only those with individual-level therapeutic interventions and to help focus on trials with active individual recruitment, we will limit inclusion to only those trials for which consent was taken at the individual level. Focusing on individual-level interventions aligns with our motivation to study interventions which could in theory be evaluated using individual level randomisation; focusing on trials with active participant recruitment targets our assessment for which the risk of identification and recruitment bias is largest; and focusing on “therapeutic interventions” targets our finding to the evaluation of interventions intended to bring about improvements in health. We have defined therapeutic interventions broadly as medicinal, clinical or surgical based interventions. Using these criteria, 104 eligible studies formed our sampling frame (Figure 1). For the pilot phase we will use the six most recent trials identified in an earlier review (Easter et al., 2021), limited to studies with the same inclusion criteria as the main sampling frame and ensuring there is no overlap with the main sample. These sampling frames will be used as the basis from which to identify researchers with experience in conducting cluster randomised trials of individual-level interventions.

Figure 1.

Identification of trials for inclusion.

Pilot Phase

A pilot phase will be completed prior to undertaking the formal semi-structured interviews, using a set of the six most recent cluster trials of individual-level interventions with active recruitment, identified from a previous review (Easter et al., 2021). Participants will be approached using the same methods as described below. The focus of the pilot phase is primarily to refine the topic guide, but to also improve the processes involved in recruiting participants and, to gain experience in conducting and completing the data analysis. It is also an opportunity for the interviewer (CE) to refine her interview technique to gain the most out of the interviews given the time allocated.

Participants will be informed that they will be participating in a pilot study and will be specifically requested to provide feedback about the process. Pilot interviews will be timed to last approximately 15 min longer than the main interviews so as to allow specific time set aside at the end of the interview for feedback. Participants will also be able to give further feedback after completion of this interview via email should they wish to. This feedback is anticipated to include all aspects in which the participant was involved, from the invitation email, to how the interview was conducted. Additionally, feedback will be used to refine aspects within the process for future recruitment of participants and conducting the semi-structured interviews. It will also be stressed to these participants that provided that there is no substantial change in these processes the data collected from the interview will also be used in the analysis for the main study. A pilot email (Supplemental Material 1) and pilot participant information sheet (Supplemental Material 2) have been developed to reflect this phase.

Recruitment

For the main study we plan to recruit a mixture of professionals with experience in conducting cluster randomised trials including principal investigators, statisticians, and clinical trialists from our pool of 104 included studies. Recruitment will occur in batches of between 5 to 10 participants. Subsequent recruitment processes will be refined depending on who has been interviewed previously and we will try as far as possible to achieve a diverse sample of participants in relation to the levels of experience that they have in cluster trials. This will ensure a representation of views across varying degrees of experience within these types of professionals. As we plan to include a mixture of professionals, we will allow recruitment for more than one investigator from each trial.

A contact list of eligible participants will be created including names of all corresponding authors of the 104 trials; and in the first instance the email address of the named corresponding author. Participants will be recruited initially by contacting the named corresponding author directly by email, with a request that our email is forwarded to other relevant investigators of the same study (Supplemental Material 3). This first email contact will contain a short summary of the objectives of the study and a participant information sheet outlining the expectations and time commitments of accepting to participate (Supplemental Material 4), including how the outputs of this research study are expected to contribute to improving cluster trials in the future.

Depending on the nature of the recruitment to date, invitation letters might be modified to emphasise recruitment of particular professionals (for example statisticians, if recruitment of statisticians has to date been low). In cases of no response, follow-up emails will be sent approximately 2 weeks later (Supplemental Material 5). Where no response is obtained after the second attempt, we will attempt to contact other relevant investigators from the same trial, using a similar approach. To contact the other investigators, we will search initially via their corresponding affiliation to identify a current email address. If we are unable to find the author using this method, we will use a search engine (e.g., google) to find the author’s current email address.

Initially, we will approach investigators from trials for which the corresponding author is affiliated with an English-speaking country (n = 55), starting with the most recently published trials (Figure 2). If the data saturation point is not reached (Saunders et al., 2018) we will continue to approach researchers from the remaining trials indicating that participation requires good use of English language, again beginning with the most recently published trials. The saturation point is defined as when no new themes emerge from the interview responses, within the types of professionals we are including. If saturation has not been reached after we have invited all eligible participants from all 104 studies in our sampling frame (for example, if recruitment is very low), further participants will be identified through snowball sampling (Tenny et al., 2021). This will be undertaken by way of asking participants who are initially approached if they know of colleagues who meet the inclusion criteria for this study for their contact details, as detailed in both the email and the participant information sheet.

Figure 2.

Strategy for inviting participants from included trials.

Incentivising Invited Participants

Recruitment to qualitative research studies, such as surveys and interviews, is known to be problematic as this is something that is entirely voluntary (Clark, 2010). If the research being conducted benefits the public and there is a clear understanding of the aims and objectives of the research, participants are more likely to take part for altruistic reasons (Islam & Tanasiuk, 2012). However, this may not be sufficient to ensure an optimal response rate to take part in a qualitative interview lasting up to 60 min.

There is a wealth of literature on how survey participation rates are affected by incentives (Church, 1993; Singer & Ye, 2013), and where the quality of the response will not be negatively affected by using such incentives (Medway & Tourangeau, 2015). Kelly et al. (2017) demonstrated that when conducting interviews for the purpose of research, a monetary incentive could increase participant willingness in comparison to no incentive or a non-monetary incentive, thus resulting in higher response rates. Therefore, to gain a high response rate and incentivise invited participants, it will be clearly communicated (within the initial correspondence and participant information sheet) that upon completion of the interview, the participant will receive a £50 Amazon voucher.

Preparation for the Interview Process

Where a response is obtained stating that the invitee would like to take part in the study, a set of dates and times for arranging the interview will be sent. After confirmation of an interview date, a consent form will be sent to be completed prior to the interview (Supplemental Material 6). It will be made clear to the interviewees that any direct quotes used in the write up of the findings will be anonymous and we will not use quotes that directly link the participant to the trial for which they have been identified from. Additionally, if quotes were to be used from the interview we would seek further permission to ensure that the participant is happy with the quote being used. Prior to the interview taking place and upon the return of the consent form, a questionnaire will be sent to the interviewee (Supplemental Material 7). This is to gain information about the interviewee’s role as a researcher and their general experience of conducting cluster randomised trials. Furthermore, an illustrative case study (Supplemental Material 8) will also be sent to the participant to read in advance. This is to ensure that the participant is familiar with the case study which the main part of the interview will focus on. A planned timeline for when interviews will be taking place will be devised, to ensure there is adequate timing between each interview to undertake data analysis. We anticipate that the data collection from these interviews will take place over roughly 6 months.

Data Collection

Data will be collected from participants using semi-structured interviews (Adams, 2015). At the beginning of the interview, the interviewer will disclose their name, job title and the overarching aim and rationale of the study. The interviewee will be asked that they have their cameras on for the duration of the interview (as outlined in the information sheet). This is to ensure that there is a good level of engagement and interaction between both the interviewee and interviewer throughout the interview reflecting an in-person interview (Krouwel et al., 2019). The interview will generally follow the topic guide but will remain flexible (Supplemental Material 9). The areas that are outlined in the topic guide are recruitment (focused around selection bias) and design justification. The content of this topic guide has been informed by the risks of bias observed within the review of 40 cluster trials of individual-level interventions (Easter et al., 2021). The topic guide will be refined based on feedback from the pilot phase. If there are areas the participant has not covered in their response, a more direct questioning approach will be taken to prompt further in-depth discussion. Should the interviewee refer to a specific study or certain unique features of a study during the interview, they will be asked in the concluding questions if they are happy with this detail being included in the transcript. Checking with the interviewee will allow them to decide if they are happy with this specific information being disclosed to the wider study team, also with the understanding that trial names will be omitted and replaced with, for example, trial A.

The length of time and content of the topic guide was determined through a number of practice semi-structured interviews with colleagues within the Institute of Applied Health Research at the University of Birmingham. Ultimately, to meet the study objectives it was decided that a focused semi-structured interview would be required and this would be feasible to achieve within 60 min. Therefore, each interview is expected to last up to 60 min with the pilot study requiring a further 15 min to allow for verbal feedback. These semi-structured interviews will be conducted by CE who is a female, part-time PhD student and medical statistician with no formal experience in conducting qualitative research.

All interviews will be conducted virtually (Bertrand & Bourdeau, 2010) through Zoom (Zoom Video communications inc, 2016), which will be recorded and a real time transcript generated. In the case that the recorder fails or the transcript is not generated, the interviewer will use the written field notes taken during the interview and also immediately after the interview (Phillippi & Lauderdale, 2018). The interviewer will also be reflective upon completion of each interview by recording their thoughts and feelings on the interview so as to capture if the interviewer’s characteristics (CE) could influence the answers from the interviewee (Dodgson, 2019). When analysing the data this reflective summary will be considered.

Data Analysis Overview

All interviews, including the pilot phase interviews, will be transcribed during the interview through the transcribe tool in Zoom. This will be followed-up by a checking process (by CE) to ensure an accurate representation of the content of the interview using the field notes made and the audio recording.

A one-page summary will be completed using the ‘one sheet of paper’ method (Ziebland & McPherson, 2006) after each interview has been transcribed. This one-page summary will document the main areas of discussion and corresponding themes to provide the researcher with an overview of the interview content. The interviewee will be contacted to check that the one-page summary of their interview is representative and accurate. When the summaries are returned to participants they will be asked to provide any comments within 2-weeks. Participants will be informed that if we do not receive a response from them within this time frame, it will be presumed that they are happy with the summary. If a participant gives comments on the one-page summary such as additions, amendments or removing of areas that were discussed in the interview, this will be amended accordingly to respect the participant’s revisions (Hagens et al., 2009). The rationale for undertaking this checking process is to ensure that the opinions of the participant have not been misunderstood. The data from the interviews completed in the pilot phase will be included in the data analysis, provided that there are no substantial changes in the topic areas included for discussion that are outlined in the guide.

The data analysis will be conducted from a realist perspective. This enables us to view “the world the way it is” by understanding and acknowledging that there could be multiple different correct ways many of us perceive reality in the world (Lakoff, 1987). A realist perspective has been chosen to approach the data analysis as this aligns with how we perceive achieving our research objectives. Therefore, enabling us to think and view in this realistic way can give a representative understanding of the reality which is perceived from each individuals’ data (Allmark & Machaczek, 2018). Therefore, the conclusions made from this data will reflect the realist approach that has been implemented throughout each stage of this study to give a true view from this specific perspective.

We will conduct an inductive thematic analysis, meaning that the data will drive the identification of patterns and themes (‘bottom up’) (Braun & Clarke, 2006), and so findings emerge from the data through the interactions we have with the data (Patton, 1990). The data management tool NVivo will aid the data management process.

Throughout this analysis process themes within the data will be captured primarily with the research question in mind, but allowing for other emergent themes not directly pertaining to the research question to be captured, noting that themes will not be rigorously defined but will remain flexible. The analysis will be completed using the following coding process, with codes derived from the data, rather than being driven by our preconceived ideas: (1) identifying codes, (2) sorting codes into categories, and (3) synthesizing the categories into relevant themes. This will begin when the transcript of the first interviews become available.

The interviews, audio transcripts and coding will happen simultaneously and in parallel. Therefore, as the interviews progress, the analysis of the data from these interviews will be completed, followed by the coding and hence themes will evolve and accumulate over this phase of the study, this is known as an iterative process (Srivastava & Hopwood, 2009). When all interviews have been conducted and each interview has been formally analysed, the coding and themes to come out of this process will then be re-evaluated. So, the coding will therefore be refined further given the collective set of themes to come from these interviews and re-applied to each of the interviewees’ data.

Furthermore, a set of themes will be identified for each participant and comparisons of these themes will be completed between groups of participants. These comparisons will be assessed across the different groups of professionals (principal investigators, statisticians, trialists) and also the level experience in undertaking cluster trials.

Structure for Completing the Data Analysis

When the first three interviews have been completed members of the research team (SG: Professor of Medical Sociology and qualitative methodologist; KH: Professor of Biostatistics and expert in cluster randomised trials; CK: Biomedical statistician with experience in qualitative research; CE: PhD student and Medical statistician) will independently complete a quick analysis to identify potential themes. A meeting will then be held to discuss the themes found independently by these members of the research team, discussing each interview in turn. Consequently, an agreement will be reached on the coding approach which will be used for the remaining analysis. This step is to minimise the risk that the team allow their own pre-conceived opinions to bias the data analysis.

Additionally, within this meeting the research team will critically assess the content of the interview. This will be undertaken to identify where further questions are required to explore areas not sufficiently covered or to address questions which are not working well. In turn this will inform on changes to the topic guide as to improve future interviews.

Reporting of Results

In the first instance the characteristics of the participants who have been interviewed will be summarised (Supplemental Table 2) and the trials for which they have been identified from which meet our inclusion criteria (Supplemental Table 1). Summaries of participants will be reported including the proportion of professionals invited that accepted the invitation for an interview and, the average number of participants interviewed per included study, using an appropriate summary measure.

The themes that have been identified from the collected interview data will be displayed in the form of a diagram (Verdinelli & Scagnoli, 2013), for example, a map to show similar themes and link together under an overarching theme. Additionally, the comparisons across different groups of interviewees will be reported to assess the similarities and differences in themes. Further quotes from these interviews will help depict the range of opinions and thoughts from these participants and enhance the notion of the themes deduced from the data. These quotes will be reported to ensure that anonymity is maintained for both the participant and the trial for which they have been identified from to participate using appropriate guidance (Corden & Sainsbury, 2006; Saunders et al., 2015). Furthermore, these specific quotes from participants that are to be used for disseminating the findings of this study will require further permission from the participant ensuring that they have the final decision on if they are happy for their quote to be included.

Ethical Consideration

It is expected that the risks of taking part in this study are minimal, although there are some important concerns. First and foremost, we will be interviewing professionals who have conducted cluster randomised trials for which we believe, in many instances, the study might have been conducted at risk of bias. It will therefore be important not to appear to be critical. To this end, focus will be drawn to the possibility that these biases simply reflect a poor reporting process and to the very real possibility that logistical issues somehow mean that these biases were inevitable. This will be portrayed through the use of an illustrative case study that has been created purely for using in these interviews and will form the basis for the discussion. The use of an illustrative case study will also provide a platform for which participants can give their opinions on without the need to refer to the cluster trial for which they have previously been a part of, with the view that they can talk openly without the worry of repercussions on their career.

When reporting results, all quotes will be anonymous and so will not have any link to the participant; and we will take steps to ensure the chosen quotes do not link directly to the relevant paper. Direct permission will be obtained for use of all quotes through the consent form. Additionally, CE will be the only person conducting the interviews and the data will be anonymised before the research team have access to the data. This has been put in place to ensure that the participant understands that the content of the interview will not be related back to them, especially given there is a prominent cluster trial researcher within this research team. To ensure the source data remains confidential, all recordings of interviews and transcripts will be stored securely on The University of Birmingham’s secure server. This study was approved by the Science, Technology, Engineering and Mathematics Ethical review committee at The University of Birmingham (ERN_21-1699).

Qualitative Rigor

Methods have been put in place to maintain qualitative rigor throughout the study. In the first instance the research team is comprised of researchers with multidisciplinary backgrounds; KH (expert in cluster randomised trials and a medical statistician), CK (medical statistician and experience in conducting qualitative interviews) and SG (medical sociologist and expert in qualitative methods), to help guide the conduct of the study and ensure the study is well focused and thorough. Appropriate background reading, including the conduct of a literature review, has contributed to the understanding of the context and also the practicalities of planning and completing a semi-structured interview study.

To guide the analysis and hence to improve rigor, the six-step process outlined in Braun and Clarke (2006) will be used to ensure that there is consistency during this process. In addition, the research team will independently conduct their own assessment and coding of the transcripts for the first few interviews. The team will then meet to have an in-depth discussion of their findings and perspective on the interviews so to allow for possible biases to be acknowledged and noted and reach agreement to take the analysis forward (Noble & Smith, 2015). The interviewer will also keep a reflexive journal detailing field notes, feelings during and after the interview was conducted and areas which need to be approached in subsequent interviews (Phillippi & Lauderdale, 2018). In addition, the COREQ guidelines (Tong et al., 2007) have been utilised and served as a checking tool for designing this study. For example, under the subsection of data collection, detailing how the interview topic guide is developed and tested, documenting the decision process that led to finalising this guide prior to commencing the interviews. Using the approaches outlined will ensure the reflexivity, credibility and also the transferability of the study findings, all important aspects to maintain the rigor of this study.

Discussion

Cluster randomised trials lend themselves by design to being at an increased risk of bias in comparison to individual randomised trials. Although there is a wealth of research in this area (Bolzern et al., 2018; Brierley et al., 2012; Diaz-Ordaz et al., 2013; Easter et al., 2021; Eldridge et al., 2008; Froud et al., 2012; Hahn et al., 2005; Puffer et al., 2003; Yang et al., 2017), alluding to the fact that it is important to address these areas at risk of bias, by designing the bias out, or implementing methods to minimise bias, to name a few, it is now even more paramount to gain a deeper level of understanding. Questions such as, “How were decisions made within the research team on designing the study?”, “What approaches were considered to minimise bias?” and “How important is it to address risk of bias in this study?” are not usually answered in such reviews of the literature. Hence a need to delve further to gain richer information from the professionals directly involved in conducting cluster randomised trials.

Our best source of understanding of how a cluster trial was conducted and the decisions that were made is the published results paper. Although there are guidelines on the number of words used, figures and tables and included to conform to the journal’s publishing remit, it is not clear whether these trials do not give bias enough weight when planning and conducting a cluster trial or if the true reason is the authors do not feel it is important enough to clearly detail the steps they have undertaken in the trial. Acknowledging these areas for which methods can be put in place to help minimise the risk of bias is important and an understanding of how they are being discussed within research teams will give insight on how to support cluster trials in the future.

The overarching focus of the interviews is the interviewee’s opinions and perceptions of bias within the cluster trials given their previous experience of being involved in cluster trials. It is particularly important to understand beyond what was reported in these cluster trial papers and the decision processes made. Additionally, it is important to understand the experiences of the participants of the practical challenges of undertaking a cluster trial. We propose to include a diverse range of disciplines that form the necessary skills to undertake a cluster trial, this will include the principal investigators, statisticians and trialists. To this end, we hope to build information from a range of different perspectives.

Challenges

We anticipate facing a number of challenges. The lead researchers in this project (KH and CE) are statisticians and not qualitative researchers. Nonetheless, they have a long-standing interest in the practicalities of running trials and their research spans both the technical and the practical. KH has previous experience in working in qualitative studies trying to understand how trials are conducted. Furthermore, they will be supported by a senior qualitative researcher (SG) and have day to day support from CK, who has direct experience in conducting qualitative research in trial methodology. We have built in some support mechanisms so that in the early stages of this project CE will have extensive support. To this end, a small number of interviews will be conducted as part of a pilot phase.

We might also face the challenge of a relatively junior researcher interviewing senior investigators. Use of the topic guide should help focus the interview in these settings, along with the pilot phase, whereby substantial practice within this environment will lessen this challenge. Interviewing a range of different professionals should also help.

Additionally, there may be a challenge when recruiting participants to take part in this study. Although there is a monetary incentive and studies have shown that this can increase the rate of uptake to such qualitative studies (Church, 1993; Kelly et al., 2017; Singer & Ye, 2013), there could be other decisions that are simply not in our control. As we are approaching the corresponding author of the studies that we have identified for inclusion, they could, for example, advise the other more junior members of the research team not to take part in this study. The reason for this could be that the corresponding author is worried that this interview could reflect badly on the study team, even with all of the measures that we have added to ensure that anonymity is maintained and a priority. Ultimately for our study this could cause an imbalance in the types of participants included in our study as we are aiming to include a broad spectrum of professional abilities to ensure that our findings truly reflect the reality of what is occurring. Therefore, to try and mitigate this as much as possible, we plan to recruit participants in batches to allow targeting of specific professionals if there is a lack of recruitment. For example, recruit more statisticians in the next batch if there has only been one interview conducted with a statistician and the rest have been with trialist and principal investigators.

Application of Findings in Practice

A summary of all the findings from this qualitative research will be reported, highlighting the particularly prevalent issues facing researchers conducting cluster randomised trials. The importance of the findings from this study will be discussed through the dissemination of these results to give context within the cluster trial setting. From this, recommendations will be suggested, addressing each of these issues to come out of this study, and so, will help support and aid researchers who are designing and conducting a cluster randomised trial.

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material for Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study by Christina Easter, Caroline Kristunas, Karla Hemming and Sheila Greenfield in Qualitative Study Protocol

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Supplemental Material

Supplemental Material - Risk of Bias in Cluster Randomised Controlled Trials of Individual-Level Interventions: Protocol for a Semi-Structured Interview Study

Footnotes

Author Contributions

KH is leading the research development of the project. SG is leading on the qualitative methodology and integrity of the project, with day to day support from CK. CE will design and develop the tools, conduct the interviews and undertake the analysis. All authors will provide important critical input into the project. All authors helped develop the data abstraction tools, provided critical insight and commented on the draft paper.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Karla Hemming is funded by a NIHR Senior Research Fellowship SRF-2017-10-002. Christina Easter is funded by the NIHR (SRF-2017-10-002). This research was also partly funded by the UK NIHR Collaborations for Leadership in Applied Health Research and Care West Midlands initiative.

ORCID iD

Christina Easter

Supplemental Material

Supplemental Material for this article is available online.

References

Adams

W. C.

(2015). Conducting semi-structured interviews. In Newcomer

K. E.

Hatry

H. P.

Wholey

J. S.

(Eds.), Handbook of practical program evaluation (pp. 492–505). John Wiley & Sons.

Allmark

Machaczek

(2018). Realism and Pragmatism in a mixed methods study. Journal of Advanced Nursing, 74(6), 1301–1309. https://doi.org/10.1111/jan.13523

Bertrand

Bourdeau

(2010). Research interviews by Skype: A new data collection method. In Esteves

(Ed.), Proceedings from the 9th European conference on research methods (pp. 70–79). IE Business School: Academic Publishing.

Bolzern

Mnyama

Bosanquet

Torgerson

D. J.

(2018). A review of cluster randomized trials found statistical evidence of selection bias. Journal of Clinical Epidemiology, 99, 106-112. https://doi.org/10.1016/j.jclinepi.2018.03.010.

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Brierley

Brabyn

Torgerson

Watson

(2012). Bias in recruitment to cluster randomized trials: A review of recent publications. Journal of Evaluation in Clinical Practice, 18(4), 878–886. https://doi.org/10.1111/j.1365-2753.2011.01700.x

Church

A. H.

(1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly, 57(1), 62–79. https://doi.org/10.1086/269355

Clark

(2010). On ‘being researched’: Why do people engage with qualitative research? Qualitative Research, 10(4), 399–419. https://doi.org/10.1177/1468794110366796

Corden

Sainsbury

(2006). Using verbatim quotations in reporting qualitative social research: Researchers’ views. https://www.york.ac.uk/inst/spru/pubs/pdf/verbquotresearch.pdf

10.

Diaz-Ordaz

Froud

Sheehan

Eldridge

(2013). A systematic review of cluster randomised trials in residential facilities for older people suggests how to improve quality. BMC Medical Research Methodology, 13(1), 127. https://doi.org/10.1186/1471-2288-13-127

11.

Dodgson

J. E.

(2019). Reflexivity in qualitative research. Journal of Human Lactation, 35(2), 220–222. https://doi.org/10.1177/0890334419830990

12.

Easter

Thompson

J. A.

Eldridge

Taljaard

Hemming

(2021). Cluster randomized trials of individual-level interventions were at high risk of bias. Journal of Clinical Epidemiology, 138, 49-59. https://doi.org/10.1016/j.jclinepi.2021.06.021.

13.

Eldridge

Ashby

Bennett

Wakelin

Feder

(2008). Internal and external validity of cluster randomised trials: Systematic review of recent trials. Bmj, 336(7649), 876–880. https://doi.org/10.1136/bmj.39517.495764.25

14.

Eldridge

Kerry

(2012). A practical guide to cluster randomised trials in health services research. Wiley.

15.

Eldridge

Kerry

Torgerson

D. J.

(2009). Bias in identifying and recruiting participants in cluster randomised trials: What can be done? Bmj, 339, b4006. https://doi.org/10.1136/bmj.b4006.

16.

Froud

Eldridge

Diaz Ordaz

Marinho

V. C.

Donner

(2012). Quality of cluster randomized controlled trials in oral health: A systematic review of reports published between 2005 and 2009. Community Dentistry and Oral Epidemiology, 40(suppl 1), 3–14. https://doi.org/10.1111/j.1600-0528.2011.00660.x

17.

Giraudeau

Ravaud

(2009). Preventing bias in cluster randomised trials. PLOS Medicine, 6(5), Article e1000065. https://doi.org/10.1371/journal.pmed.1000065

18.

Hagens

Dobrow

M. J.

Chafe

(2009). Interviewee transcript review: Assessing the impact on qualitative research. BMC Medical Research Methodology, 9(1), 47. https://doi.org/10.1186/1471-2288-9-47

19.

Hahn

Puffer

Torgerson

D. J.

Watson

(2005). Methodological bias in cluster randomised trials. BMC Medical Research Methodology, 5(1), 10. https://doi.org/10.1186/1471-2288-5-10

20.

Islam

Tanasiuk

(2012). Differential response on pre- and post-disclosed committed inducements in a face to face interview. University Library of Munich. https://ideas.repec.org/p/pra/mprapa/107276.html

21.

Kelly

Margolis

McCormack

LeBaron

P. A.

Chowdhury

(2017). What affects people’s willingness to participate in qualitative research? An experimental comparison of five incentives. Field Methods, 29(4), 333–350. https://doi.org/10.1177/1525822x17698958

22.

Krouwel

Jolly

Greenfield

(2019). Comparing Skype (video calling) and in-person qualitative interview modes in a study of people with irritable bowel syndrome – an exploratory comparative analysis. BMC Medical Research Methodology, 19(1), 219. https://doi.org/10.1186/s12874-019-0867-9

23.

Lakoff

(1987). Women, fire, and dangerous things: What categories reveal about the mind: The University of Chicago Press.

24.

Marshall

M. N.

(1996). Sampling for qualitative research. Family Practice, 13(6), 522–526. https://doi.org/10.1093/fampra/13.6.522

25.

Medway

R. L.

Tourangeau

(2015). Response quality in telephone surveys: Do prepaid cash incentives make a difference? The Public Opinion Quarterly, 79(2), 524–543. http://www.jstor.org/stable/24546375

26.

Murray

D. M.

(1998). Design and analysis of group randomized trials. Oxford University Press Inc.

27.

Noble

Smith

(2015). Issues of validity and reliability in qualitative research. Evidence Based Nursing, 18(2), 34–35. https://doi.org/10.1136/eb-2015-102054

28.

Patton

M. Q.

(1990). Qualitative evaluation and research methods (2nd ed.). Sage.

29.

Phillippi

Lauderdale

(2018). A guide to field notes for qualitative research: Context and conversation. Qualitative Health Research, 28(3), 381–388. https://doi.org/10.1177/1049732317697102

30.

Puffer

Torgerson

Watson

(2003). Evidence for risk of bias in cluster randomised trials: Review of recent trials published in three general medical journals. Bmj, 327(7418), 785–789. https://doi.org/10.1136/bmj.327.7418.785

31.

Saunders

Kitzinger

(2015). Anonymising interview data: Challenges and compromise in practice. Qualitative Research, 15(5), 616–632. https://doi.org/10.1177/1468794114550439

32.

Saunders

Sim

Kingstone

Baker

Waterfield

Bartlam

Burroughs

Jinks

(2018). Saturation in qualitative research: Exploring its conceptualization and operationalization. Quality and Quantity, 52(4), 1893–1907. https://doi.org/10.1007/s11135-017-0574-8

33.

Singer

(2013). The use and effects of incentives in surveys. The ANNALS of the American Academy of Political and Social Science, 645(1), 112–141. https://doi.org/10.1177/0002716212458082

34.

Srivastava

Hopwood

(2009). A practical iterative framework for qualitative data analysis. International Journal of Qualitative Methods, 8(1), 76–84. https://doi.org/10.1177/160940690900800107

35.

Tenny

Brannan

G. D.

Brannan

J. M.

Sharts-Hopko

N. C.

(2021). Qualitative study. StatPearls [Internet]. StatPearls Publishing.

36.

Tong

Sainsbury

Craig

(2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357. https://doi.org/10.1093/intqhc/mzm042

37.

Verdinelli

Scagnoli

N. I.

(2013). Data display in qualitative research. International Journal of Qualitative Methods, 12(1), 359–381. https://doi.org/10.1177/160940691301200117

38.

Yang

Carter

B. L.

Gums

T. H.

Gryzlak

B. M.

Levy

B. T.

(2017). Selection bias and subject refusal in a cluster-randomized controlled trial. BMC Medical Research Methodology, 17(1), 94. https://doi.org/10.1186/s12874-017-0368-7

39.

Zhang

J. Z.

Nicholls

S. G.

Carroll

Nix

H. P.

Goldstein

C. E.

Hey

S. P.

Brehaut

J. C.

McLean

P. C.

Weijer

Fergusson

D. A.

Taljaard

(2021). Informed consent in pragmatic trials: Results from a survey of trials published 2014-2019. Journal of Medical Ethics. . Online ahead of print. https://doi.org/10.1136/medethics-2021-107765

40.

Ziebland

McPherson

(2006). Making sense of qualitative data analysis: An introduction with illustrations from DIPEx (personal experiences of health and illness). Journal of Medical Education, 40(5), 405–414. https://doi.org/10.1111/j.1365-2929.2006.02467.x

41.

Zoom Video communications inc . (2016) Security guide. Zoom Video Communications Inc. https://d24cgw3uvb9a9h.cloudfront.net/static/81625/doc/Zoom-Security-White-Paper.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.20 MB

0.22 MB

0.20 MB

0.27 MB

0.21 MB

0.33 MB

0.20 MB

0.21 MB

0.33 MB

0.36 MB

0.69 MB