Understanding the functions and operations of data monitoring committees: Survey and focus group findings

Introduction

Randomized clinical trials, especially large trials with important clinical outcome measures, have often been overseen by independent data monitoring committees (DMCs), external to the sponsor, whose role was to monitor trial conduct and participant safety, and recommend trial changes or termination whenappropriate.^1,2 Since the original concept for DMCs was introduced nearly 50 years ago, ³ the use of DMCs appears to have increased^4,5 and evolved. ⁶ Today, DMCs may oversee small trials with symptomatic endpoints, groups of related trials, or entire research portfolios related to a product. Depending on the nature of the trial, they may be external, internal to the trial sponsor, or have a mixture of internal and external members. Responsibilities, including review of efficacy, safety, data quality, and trial operations, also vary.

Because the scope and use of DMCs has widened, variability in roles and responsibilities across DMCs can contribute to unclear expectations about the DMC role, both among DMCs and other trial stakeholders (e.g. statistical data analysis center (SDAC), sponsors, regulatory bodies, and institutional review boards (IRBs)). Guidelines describing DMC roles, as well as DMC interactions and communications with other stakeholders, are needed to maintain DMC independence and optimize DMC functions. The overall increased use of DMCs has also resulted in a mismatch between the need for and availability of qualified DMC members. ⁷ The varying functions of DMCs require clarification of the appropriate training and composition of DMCs in order to develop a plan for preparing the next generation of DMC members. The Clinical Trials Transformation Initiative (CTTI), a public–private partnership that aims to develop and drive adoption of practices that will increase the quality and efficiency of clinical trials, undertook a project to address the challenges associated with DMC use. The goal of the DMC project is to promote DMC independence and ensure that DMCs are composed of qualified members who are convened for appropriate trials. Additionally, DMCs should conduct their activities in a clear manner that is understood by all stakeholders and enhances the overall quality of trial oversight. ⁸

Initial project activities revealed that there is limited published information about the rationale for and operations of DMCs. The literature does not adequately discuss when DMCs are needed beyond the classical, large phase 3 clinical trials, and there is a lack of information on DMC member training and qualifications. Therefore, CTTI conducted a survey to assess the current use and conduct of DMCs, as well as the training practices for DMC members. In addition, key focus groups were convened to gain an in-depth understanding of the needs and best practices related to DMC use in the modern context. The results of these evidence-gathering activities, described here, combined with input from multi-stakeholder groups of experts, were used by the CTTI project team to develop recommendations and tools for DMCs and trial sponsors. ⁹ Here we summarize the findings from the survey and focus groups.

Methods

The project consisted of a survey and six moderated focus group discussions. The online survey was designed to gather opinions on current DMC issues, as described by the project objectives. The survey was targeted at DMC members, sponsor or contract research organization representatives involved in the organization of DMCs, and SDAC representatives. Respondents were sought via CTTI members, project team–recommended contacts, and DMC-related LinkedIn groups. The survey was initially sent to 629 individuals. However, recipients were encouraged to forward it to others; therefore, no reliable data are available regarding the number of potential respondents who had access to the survey and could have completed it. The survey had 143 total respondents, which included DMC members (n = 76), sponsors involved with organization of DMCs (n = 52), and SDAC representatives (n = 15). Questions posed to respondents were tailored to each type of respondent. The complete survey is available on the CTTI website. ¹⁰

Survey questions were created and reviewed by members of the CTTI DMC project team (see Acknowledgements). The questions were developed based on team members’ expert knowledge of the topics being surveyed, with support from the Duke Center for Learning Healthcare. No formal survey validation was performed. Survey completion required approximately 20 minutes. The survey was configured to require a response to initial questions regarding the respondent’s role and interaction with DMCs. Subsequent survey questions permitted non-response. “None” or other negative responses were available where appropriate.

Participation was voluntary, anonymous, and uncompensated. The survey was distributed via Constant Contact software (Waltham, Massachusetts) and completed using Qualtrics software (Provo, Utah). Survey data were collected from 24 October 2014 to 1 December 2014. Data were then aggregated by the Duke Center for Learning Health Care and distributed to project team members for descriptive analysis.

Six focus groups were convened to gather additional qualitative, in-depth, information related to DMCs. The focus groups were composed of (1) DMC members (n = 10), (2) patients and/or patient advocate DMC members (n = 7), (3) IRB and US Food and Drug Administration (FDA) representatives (n = 7), (4) industry sponsors (n = 6), (5) government and non-profit sponsors (n = 6), and (6) SDAC representatives (n = 6). The CTTI DMC project team nominated focus group participants considered to have extensive knowledge of and experience with DMCs and representing a diverse set of stakeholders (n = 37). Survey respondents were also able to volunteer for focus group participation (n = 5). Focus group meetings were conducted by the Duke Center for Learning Healthcare between December 2014 and January 2015. The 1 hour web conferences were conducted using a pre-specified interview guide (Supplementary Appendix). Transcribed interviews were analyzed for themes with the goal of identifying consensus opinions, important insights, and recommendations.

Both the survey and focus groups were designated as exempt research by the Duke University IRB.

Results

The survey and focus group results are organized and presented in four topic areas: (1) DMC purpose and rationale, (2) formation and organization, (3) communication, and (4) qualifications and training. The demographics of survey respondents are shown in Table 1. DMC members who completed the survey represented a variety of scientific disciplines and were experienced DMC members, about half of whom had served as a DMC chair. As noted in the “Methods” section, a response rate is not reportable because recipients of the survey were allowed to forward it to others.

OPEN IN VIEWER

Table 1.

Characteristics of survey respondents.

Characteristics	n (%)
DMC organizers	52
Pharmaceutical industry	23 (44)
Government	12 (23)
Academic	7 (13)
CRO	5 (10)
Device industry	4 (8)
ARO	1 (2)
DMC members	76
Non-patient advocate DMC members	63 (83)
Patient advocate DMC members	13 (17)
Scientific discipline a
Clinical research	28 (37)
Physician	25 (33)
Biostatistics	25 (33)
Patient advocacy	13 (17)
Epidemiology	7 (9)
Medical ethics	6 (8)
Pharmacology/pharmacotherapy	6 (8)
Medical ethics	6 (8)
Basic science	5 (7)
Non-MD clinician	2 (3)
Number of DMCs served on
<5	36 (47)
About 5–10	21 (28)
About 11–20	8 (11)
>20	11 (14)
Years of DMC service b
<1	4 (5)
About 1–3	9 (12)
About 4–6	15 (20)
About 7–10	16 (21)
>11	31 (41)
Experience as a DMC chair
Yes	36 (47)
No	40 (53)
SDAC representatives	15

ARO: academic research organization; CRO: contract research organization; DMC: data monitoring committee; SDAC: statistical data analysis center.

a

Respondents could select more than one discipline.

b

N = 75.

Purpose and rationale

Role of DMCs

The overall survey findings regarding the role of DMCs were generally consistent across the types of survey respondents and with the views expressed in the focus groups. The ranking of DMC activities from most (1) to least (7) important as rated by DMC members and organizers who answered this question is shown in Figure 1. The activity rated as most important was reviewing summary adverse events and primary outcomes by treatment arm to assess the risk-to-benefit relationship for study participants. Focus group participants recognized that DMCs hold a unique place in trial oversight. They indicated that the primary responsibility of a DMC is to be an independent advisory body representing the interests of trial participants. Specifically, it was noted that DMCs should have access to unmasked study data (i.e. study treatment assignment is disclosed) in order to periodically review the accumulating safety and efficacy findings and advise the sponsor on whether to continue, modify, or terminate a trial based on an assessment of risks and benefits. According to the focus group discussion, unmasked interim analyses should be identified in the charter and agreed upon beforehand. An FDA representative in the focus groups noted that DMCs historically have done pre-specified analyses to see if there are reasons to stop the trial early, but they must also react to unanticipated adverse events and trends.

OPEN IN VIEWER

Figure 1.

Importance of DMC activities as rated by DMC members and various types of DMC organizers.

Survey respondents and focus group participants also highlighted the role of DMCs in assessing the validity and integrity of the trial to enhance its potential to generate reliable findings. A general consensus was that DMC roles should be clearly defined in the trial charter. With regard to whether DMC recommendations should be binding, the focus group discussion indicated that historically the DMC has almost universally been an advisory body, but that DMCs could have executive authority in some cases if this is agreed upon with the sponsor and clearly articulated in the charter.

A theme that emerged from the focus group discussion was that in order to minimize bias, DMCs should have a very limited role in trial design. The concern was that if the DMC is too involved in the design, they might become vested in the outcome of the trial. However, focus group participants felt that the DMC should already be organized and have the opportunity to review the study protocol and offer suggestions before the study is launched.

Determining whether a DMC is needed

No clear consensus emerged from the focus groups on specific criteria for requiring a DMC for a given trial. Participants indicated that the decision to use a DMC often rests with the sponsor and IRBs, and historically, DMCs have most commonly been used for large, randomized, controlled outcomes studies. However, it was also noted that DMCs can be used regardless of study size or study phase. Factors cited by focus groups in support of DMC use include high overall study risks, the complexity of the intervention, and the vulnerability of the study population.

A theme that emerged in the focus groups was that external independent DMC oversight is not needed in all cases. Some participants expressed a belief that DMC use is increasing and that in some cases they are not needed and are perhaps over-utilized. Generally, participants felt that all trials require a comprehensive monitoring plan, but not all require monitoring by a DMC.

Formation and organization

DMC composition

The survey indicated the typical size of a DMC ranged from 3 to 7 members. The focus groups emphasized that DMC membership should include someone who understands complex data and the challenges of data analysis and interpretation. Overall, the focus groups identified the following membership needs of a DMC:

Medical experts;

Clinical researchers with specific scientific expertise;

Statisticians with clinical trial experience;

Ethicist under certain circumstances;

Patient advocate, especially for trials including patient-reported outcomes, quality-of-life measures, or a comparative effectiveness research design.

DMC member identification

At least three-quarters (38/49) of sponsors surveyed indicated they have standard operating procedures governing the conduct of DMCs and identification of prospective DMC members. The focus groups recognized that identifying DMC members with relevant expertise and experience is challenging, and several noted that balancing conflicts of interest is another challenge. In the survey, sponsors reported that they most commonly identified potential DMC members based on prior service on DMCs or by recommendations based on their reputation and expertise. Occasionally, DMC members themselves assist in identification of prospective DMC members with whom they previously interacted or served on other DMCs.

DMC member contracts and remuneration

In the survey, approximately 65% of the sponsors (48/52) and DMC members (48/75) indicated there was a formal contract with DMC members always or most of the time. There was high concordance between the sponsors and DMC members on the typical contract-specific items except for the following that were reported less often by the DMC members: pay for unscheduled meetings, pay for work done outside the regular meetings, and Sunshine Act reporting requirements ¹¹ (Figure 2).

OPEN IN VIEWER

Figure 2.

Items typically contained in a DMC contract, as reported by DMC organizers (i.e. sponsors) and members.

Almost 90% of the DMC members (47/54) and two-thirds of the sponsors (22/34) indicated the initial contract was customized specifically for each DMC. In the contract, remuneration was based on per-meeting or per-teleconference rates, or hourly rates. Government sponsors reported setting the remuneration rate, while industry sponsors generally negotiated with the DMC members.

DMC charter

Over 80% (48/53) of sponsors surveyed reported that a DMC charter is always used, while less than 60% (44/75) of the DMC members indicated the same. Nevertheless, there was a high concordance between the sponsors and DMC members on the typical charter-specific items. Approximately three-quarters (41/52) of the sponsors indicated that charters are very useful in providing the information needed to carry out DMC responsibilities, compared with 60% (44/74) of the DMC members. The DMC charter originated from multiple authors, including the sponsor and/or the SDAC. Over three-quarters (50/63) of the DMC members indicated they had input into the charter all or most of the time.

The focus groups identified a number of proposed improvements to the typical charter:

Use a standard template that is modified for individual studies;

Include an agreed-upon outline of the DMC report including efficacy data review;

Indicate that ad hoc analyses may be needed beyond the pre-planned analyses;

Fully address whether the DMC will have access to unmasked data (i.e. study treatment assignment is disclosed) at the subject level and aggregate level;

Address if, and when, study leadership is notified of aggregate event rates;

Include clear decision guidelines but indicate that judgment needs to be exercised;

Describe (in detail) communication practices between the DMC, SDAC, and sponsor.

DMC meeting schedule and conduct

Based on the survey, DMC meetings were commonly a combination of regularly scheduled meetings (quarterly or semi-annually) and ad hoc meetings. These meetings were typically conducted as a mix of remote (i.e. teleconference or web conference) and in-person meetings. The schedule of planned meetings was set by the sponsor and the DMC. Over 75% of the sponsors (39/49) and DMC members (54/72) indicated the DMC had the authority to conduct additional ad hoc meetings.

Communication

Findings of the DMC may require communication to a variety of stakeholders in the course of their work on a trial, including the SDAC, sponsor, regulatory bodies, and/or IRBs. Focus group participants indicated that DMCs typically will communicate directly with the SDAC and the sponsor, with the sponsor relaying information to the other stakeholders as needed. However, they noted that communication can be complicated by a desire to keep some DMC activities confidential from the trial sponsor or to ensure that regulators and/or IRBs are receiving timely and candid presentations of information. Focus group participants suggested that guidance for various lines of communication can be clarified in the charter.

DMC and SDAC communication

Survey responses from the SDAC representatives indicated that a biostatistician with expertise in interim analysis methods who prepared the DMC report is present at DMC meetings all or most of the time, DMC members typically see table shells in advance of the first interim analysis, and DMC members have the opportunity to suggest changes as needed (Figure 3). These respondents also indicated that flexibility exists in planned analyses for the DMC and the DMC meeting schedule, and that the sponsor is not necessarily aware of these changes when they are suggested by the DMC.

OPEN IN VIEWER

Figure 3.

Statistical data analysis center representatives’ (n = 15) survey responses regarding analyses and reports for DMCs.

Additionally, suggested best practices from the SDAC representatives included the following:

To effectively communicate with the DMC, the SDAC statistician should be experienced with DMCs and fully understand the trial’s protocol and procedures;

The SDAC report to the DMC should provide an overview of current study status;

Delivery of the SDAC report to the DMC should be performed through a secure electronic system;

Use of figures and graphics in the SDAC report to the DMC should be encouraged.

DMC and sponsor communication

In the survey, 93%(67/72) of DMC members indicated that DMCs have been able to function independently from the trial sponsor all or most of the time. Interaction between the DMC and sponsor has typically been greatest during charter development, and then minimal during trial conduct except during the open session of DMC meetings. Overall, 24% (17/72) of DMC member respondents had experienced some pressure to make a particular decision, but any perceived pressure on decision-making primarily came from other DMC members (65% (11/17)). DMC member respondents (97% (70/72)) also indicated having not suffered adverse consequences from the sponsor for making a particular decision.

Survey findings suggest that while the DMC is recognized as an advisory body, with the sponsor maintaining final decision-making authority, DMC recommendations are nearly always accepted by the sponsor. Of DMC member survey respondents, 97% (70/72) indicated that DMC recommendations are accepted by the sponsor always or most of the time.

DMC, regulator, and IRB communication

IRB and FDA focus group participants indicated they have the authority to request a DMC, or modify a planned DMC (e.g. if there are concerns about lack of DMC independence), but this rarely occurs. While the FDA may review the DMC charter prior to study initiation, the IRB typically does not, and neither body prospectively evaluates the qualifications of proposed DMC members.

IRB members reported no direct communication with the DMC, although they may review recommendations generated from DMC meetings. FDA representatives reported that direct communication with DMCs is relatively rare, with most having occurred when a DMC recommended stopping a trial.

Discussion and conclusion

Use of DMCs has expanded over the past four decades from highly targeted applications at their inception to what could now be considered routine use in the conduct of a broader range of modern clinical trials. In observing the increasing breadth of DMCs, CTTI became aware of an opportunity to address a number of challenges related to DMC conduct. Many of the results summarized in this article are not surprising. We found there is variation in when DMCs are used, how DMCs are constituted, how meetings are conducted, and how data are summarized and presented. Some of this variation is driven by the breadth of trials that utilize DMCs, and thus is expected and needed.

Overall, participants agreed that independence from the study sponsor and transparency in disclosure of potential conflicts of interest are both critical to the optimal functioning of DMCs. Although the majority of DMC members reported that they generally are able to function independently without undue influence from trial sponsors, proposed areas for potential enhancement include the development of procedures for ongoing reporting of conflicts of interest and establishment of clear communication practices between sponsors, DMCs, and other key stakeholders.

Additional themes that emerged from the survey and focus groups included training for DMC members and statistical support provided by the SDAC. Based on the CTTI survey of DMC members, sponsors, and statistical centers supporting DMCs, as well as input from the focus groups, the overwhelming majority confirmed that DMC members may well be operating with little or no formal training in DMC best practices. Despite the growing number of DMCs, there is no known central registry of individuals who have participated as DMC members. Also, there are no formal requirements for DMC training by sponsors or regulatory agencies, nor are there any certified training programs for DMC membership. However, there are textbooks,^2,12 publications of case histories, and web-based material^13,14 that could be used to augment a training program if a mechanism was developed to sustain such a program. Use of existing professional meetings as a forum for DMC training holds promise and has on occasion been done informally in the past.

In addition to training for DMC members, consensus from the survey and focus groups is that statistical centers that support DMCs in their activity to monitor trial progress and assess the ongoing benefits and risks to trial participants would themselves benefit from training in how best to meet the needs of DMCs and provide clear, accurate, and timely reports. Many statistical centers may have inadequate understanding of the protocol, the data management and flow, and the importance of preparing DMC interim reports that contain effective summary data displays. This is critical as optimal performance of DMCs is in large part dependent on the quality of the interim analysis reports DMCs must periodically review. Thus, some level of training must be included for statistical centers. Finally, sponsors themselves may need to incorporate some level of DMC training with their trial leadership, and there needs to be more uniformity in how DMCs are constructed and organized and in the statistical support they receive.

A limitation of this research is the survey sampling method we used, which allowed recipients to forward the survey to an unlimited number of individuals (chain-referral sampling), thereby precluding our ability to ascertain a true denominator and to calculate a response rate. This method was selected in order to reach a larger number of individuals who had experience with DMCs. Referral sampling has the potential to introduce bias because the initial survey recipients may be more likely to forward the survey to colleagues who share their views, experiences, and characteristics. We attempted to mitigate these biases by starting with a large initial sample of diverse individuals. Although our data may not be entirely generalizable to the larger population, they nonetheless help characterize DMCs and reveal important insights about their function and critical role in trial monitoring.

The insights gathered from the survey and focus groups, along with further input from expert stakeholder meetings, were used to guide the CTTI project team in developing recommendations and tools to improve the conduct of DMCs and enhance the quality of trial oversight. ⁹