Clinical Trials

The MsFLASH (Menopause Strategies: Finding Lasting Answers for Symptoms and Health) Network recruited into five randomized clinical trials (n = 100–350) through mass mailings. The fifth trial tested two interventions for postmenopausal vulvovaginal symptoms (itching, pain, irritation, dryness, or pain with sex) and thus required a high level of sensitivity to privacy concerns. For this trial, in addition to mass mailings we pilot tested a social media recruitment approach. We aimed to evaluate the feasibility of recruiting healthy midlife women with bothersome vulvovaginal symptoms to participate in the Vaginal Health Trial through Facebook advertising.

As part of a larger advertising campaign that enrolled 302 postmenopausal women for the 12-week randomized, double-blind, placebo-controlled Vaginal Health Trial from April 2016 to February 2017, Facebook advertising was used to recruit 25 participants. The target population for recruitment by mailings and by Facebook ads included women aged 50–70 years and living within 20 miles of study sites in Minneapolis, MN and Seattle, WA. Design of recruitment letters and Facebook advertisements was informed by focus group feedback. Facebook ads were displayed in the “newsfeed” of targeted users and included a link to the study website. Response rates and costs are described for both online ads and mailing.

Facebook ads ran in Minneapolis for 28 days and in Seattle for 15 days, with ads posted and removed from the site as needed based on clinic flow and a set budget limit. Our estimated Facebook advertising reach was over 200,000 women; 461 women responded and 25 were enrolled at a cost of US$14,813. The response rate per estimated reach was 0.22%; costs were US$32 per response and US$593 per randomized participant. The social media recruitment results varied by site, showing greater effectiveness in Seattle than in Minneapolis. We mailed 277,000 recruitment letters; 2166 women responded and 277 were randomized at a cost of US$98,682. The response rate per letter sent was 0.78%; costs were US$46 per response and US$356 per randomized participant. Results varied little across sites.

Recruitment to a clinical trial testing interventions for postmenopausal vaginal symptoms is feasible through social media advertising. Variability in observed effectiveness and costs may reflect the small sample sizes and limited budget of the pilot recruitment study.

In this study, we compared two research consent techniques: a standardized video plus usual consent and usual consent alone.

Individuals who completed 24-month outcomes (completers) in the Operations and Pelvic Muscle Training in the Management of Apical Support Loss study were invited to participate in an extended, longitudinal follow-up study (extended Operations and Pelvic Muscle Training in the Management of Apical Support Loss). Potential participants who were (1) able to provide consent and (2) not in long-term care facilities were randomized 1:1 to a standardized video detailing the importance of long-term follow-up studies of pelvic floor disorders followed by the usual institutional consent process versus the usual consent process alone. Randomization, stratified by site, used randomly permuted blocks. The primary outcome was the proportion of participants who enrolled in the extended study and completed data collection events 5 years after surgery. Secondary outcomes included the proportion enrolled in the extended study, completion of follow-up at each study year, completion of data collection points, completion of in-person visits, and completion of quality of life calls. Motivation and barriers to enrollment (study-level and personal-level) and satisfaction with the study consent process were measured by questionnaire prior to recruitment into extended Operations and Pelvic Muscle Training in the Management of Apical Support Loss. Groups were compared using an intention-to-treat principle, using unadjusted Student’s t-test (continuous) and chi-square or Fisher’s exact (categorical) test. A sample size of 340 (170/group) was estimated to detect a 15% difference in enrollment and study completion between groups with p < 0.05.

Of the 327 Operations and Pelvic Muscle Training in the Management of Apical Support Loss completers, 305 were randomized to the consent process study (153 video vs 152 no video). Groups were similar in demographics, surgical treatment, and outcomes. The overall rate of extended study enrollment was high, without significant differences between groups (video 92.8% vs no video 94.1%, p = 0.65). There were no significant differences in the primary outcome (video 79.1% vs no video 75.7%, p = 0.47) or in any secondary outcomes. Being “very satisfied” overall with study information (97.7% vs 88.5%, p = 0.01); “strong agreement” for feeling informed about the study (81.3% vs 70.8%, p = 0.06), understanding the study purpose (83.6% vs 71.0%, p = 0.02), nature and extent (82.8% vs 70.2%, p = 0.02), and potential societal benefits (82.8% vs 67.9%, p = 0.01); and research coordinator/study nurse relationship being “very important” (72.7% vs 63.4%, p = 0.03) were better in the video compared to the no video consent group.

The extended study had high enrollment; most participants completed most study tasks during the 3-year observational extension, regardless of the use of video to augment research consent. The video was associated with a higher proportion of participants reporting improved study understanding and relationship with study personnel.

The postmarket research goal is to assess “generalizability” or “external validity” to see if the early results of clinical trials with investigational devices are reproducible in everyday practice in the real world and the longer term. Registries have an important but ambivalent role in achieving this goal.

Although registries are common, in practice they follow the regulatory processes that appear designed primarily for pharmaceutical clinical trials and confirmatory studies. We review the literature to assess different definitions and the role of registries in the hierarchy of scientific evidence. We analyze common characteristics affecting registry design, implementation, and governance as well as safety reporting and off-label use while describing the experience of setting up an international, prospective registry for an endovascular device used to treat abdominal aortic aneurysms.

Key areas in which to distinguish registries from trials are as follows: eligibility, setting (patients and institutions), device configurations and iterations, the use of design and quality “spaces,” a focus on systematic quality checks (rather than source data monitoring), open-ended follow-up, flexibility in the definition of end points and sample sizes, data sharing, and publishing commitments.

Both clinical trials and registries are essential and complementary research methods and the strengths and weaknesses of each need to be recognized. The specific characteristics of registry research deserve to be acknowledged and safeguarded in the regulations governing clinical investigations with medical devices.

The ICON6 trial (ISRCTN68510403) is a phase III academic-led, international, randomized, three-arm, double-blind, placebo-controlled trial of the addition of cediranib to chemotherapy in recurrent ovarian cancer. It investigated the use of placebo during chemotherapy and maintenance (arm A), cediranib alongside chemotherapy followed by placebo maintenance (arm B) and cediranib throughout both periods (arm C). Results of the primary comparison showed a meaningful gain in progression-free survival (time to progression or death from any cause) when comparing arm A (placebo) with arm C (cediranib). As a consequence of the positive results, AstraZeneca was engaged with the Medical Research Council trials unit to discuss regulatory submission using ICON6 as the single pivotal trial.

A relatively limited level of on-site monitoring, single data entry and investigator’s local evaluation of progression were used on trial. In order to submit a license application, it was decided that (a) extensive retrospective source data verification of medical records against case report forms should be performed, (b) further quality control checks for accuracy of data entry should be performed and (c) blinded independent central review of images used to define progression should be undertaken. To assess the value of these extra activities, we summarize the impact on both efficacy and safety outcomes.

Data point changes were minimal; those key to the primary results had a 0.47% error rate (36/7686), and supporting data points had a 0.18% error rate (109/59,261). The impact of the source data verification and quality control processes were analyzed jointly. The conclusion drawn for the primary outcome measure of progression-free survival between arm A and arm C was unchanged. The log-rank test p-value changed only at the sixth decimal place, the hazard ratio does not change from 0.57 with the exception of a marginal change in its upper bound (0.74–0.73) and the median progression-free survival benefit from arm C remained at 2.4 months. Separately, the blinded independent central review of progression scans was performed as a sensitivity analysis. Estimates and p values varied slightly but overall demonstrated a difference in arms, which is consistent with the initial result. Some increases in toxicity were observed, though these were generally minor, with the exception of hypertension. However, none of these increases were systematically biased toward one arm.

The conduct of this pragmatic, academic-sponsored trial was sufficient given the robustness of the results, shown by the results remaining largely unchanged following retrospective verification despite not being designed for use in a marketing authorization. The burden of such comprehensive retrospective effort required to ensure the results of ICON6 were acceptable to regulators is difficult to justify.

A risk-based approach to clinical research may include a central statistical assessment of data quality. We investigated the operating characteristics of unsupervised statistical monitoring aimed at detecting atypical data in multicenter experiments. The approach is premised on the assumption that, save for random fluctuations and natural variations, data coming from all centers should be comparable and statistically consistent. Unsupervised statistical monitoring consists of performing as many statistical tests as possible on all trial data, in order to detect centers whose data are inconsistent with data from other centers.

We conducted simulations using data from a large multicenter trial conducted in Japan for patients with advanced gastric cancer. The actual trial data were contaminated in computer simulations for varying percentages of centers, percentages of patients modified within each center and numbers and types of modified variables. The unsupervised statistical monitoring software was run by a blinded team on the contaminated data sets, with the purpose of detecting the centers with contaminated data. The operating characteristics (sensitivity, specificity and Youden’s J-index) were calculated for three detection methods: one using the p-values of individual statistical tests after adjustment for multiplicity, one using a summary of all p-values for a given center, called the Data Inconsistency Score, and one using both of these methods.

The operating characteristics of the three methods were satisfactory in situations of data contamination likely to occur in practice, specifically when a single or a few centers were contaminated. As expected, the sensitivity increased for increasing proportions of patients and increasing numbers of variables contaminated. The three methods showed a specificity better than 93% in all scenarios of contamination. The method based on the Data Inconsistency Score and individual p-values adjusted for multiplicity generally had slightly higher sensitivity at the expense of a slightly lower specificity.

The use of brute force (a computer-intensive approach that generates large numbers of statistical tests) is an effective way to check data quality in multicenter clinical trials. It can provide a cost-effective complement to other data-management and monitoring techniques.

A major goal of the National Institutes of Health’s Clinical and Translational Science Award program is to facilitate clinical research and enhance the transition of basic to clinical research. As such, a number of Clinical and Translational Science Award centers have developed services to facilitate the conduct of clinical research, including support with fulfilling regulatory requirements.

The University of Kentucky sought to establish an institutional semi-independent monitoring committee to provide oversight for clinical research studies per National Institutes of Health requirements and recommendations. Our semi-independent monitoring committee was initiated in 2010.

Since the inception of our semi-independent monitoring committee we have restructured its operations and protocols to improve efficiency. This article discusses our experiences with semi-independent monitoring committee creation and growth.

This article summarizes our experience in creating and maturing an institutional data monitoring committee.

For single arm trials, a treatment is evaluated by comparing an outcome estimate to historically reported outcome estimates. Such a historically controlled trial is often analyzed as if the estimates from previous trials were known without variation and there is no trial-to-trial variation in their estimands. We develop a test of treatment efficacy and sample size calculation for historically controlled trials that considers these sources of variation.

We fit a Bayesian hierarchical model, providing a sample from the posterior predictive distribution of the outcome estimand of a new trial, which, along with the standard error of the estimate, can be used to calculate the probability that the estimate exceeds a threshold. We then calculate criteria for statistical significance as a function of the standard error of the new trial and calculate sample size as a function of difference to be detected. We apply these methods to clinical trials for amyotrophic lateral sclerosis using data from the placebo groups of 16 trials.

We find that when attempting to detect the small to moderate effect sizes usually assumed in amyotrophic lateral sclerosis clinical trials, historically controlled trials would require a greater total number of patients than concurrently controlled trials, and only when an effect size is extraordinarily large is a historically controlled trial a reasonable alternative. We also show that utilizing patient level data for the prognostic covariates can reduce the sample size required for a historically controlled trial.

This article quantifies when historically controlled trials would not provide any sample size advantage, despite dispensing with a control group.

Travel burden often substantially limits the ability of individuals to participate in clinical trials. Wide geographic dispersion of individuals with rare diseases poses an additional key challenge in the conduct of clinical trials for rare diseases. Novel technologies and methods can improve access to research by connecting participants in their homes and local communities to a distant research site. For clinical trials, however, understanding of factors important for transition from traditional multi-center trial models to local participation models is limited. We sought to test a novel, hybrid, single- and multi-site clinical trial design in the context of a trial for Juvenile Neuronal Ceroid Lipofuscinosis (CLN3 disease), a very rare pediatric neurodegenerative disorder.

We created a “hub and spoke” model for implementing a 22-week crossover clinical trial of mycophenolate compared with placebo, with two 8-week study arms. A single central site, the “hub,” conducted screening, consent, drug dispensing, and tolerability and efficacy assessments. Each participant identified a clinician to serve as a collaborating “spoke” site to perform local safety monitoring. Study participants traveled to the hub at the beginning and end of each study arm, and to their individual spoke site in the intervening weeks.

A total of 18 spoke sites were established for 19 enrolled study participants. One potential participant was unable to identify a collaborating local site and was thus unable to participate. Study start-up required a median 6.7 months (interquartile range = 4.6–9.2 months). Only 33.3% (n = 6 of 18) of spoke site investigators had prior clinical trial experience, thus close collaboration with respect to study startup, training, and oversight was an important requirement. All but one participant completed all study visits; no study visits were missed due to travel requirements.

This study represents a step toward local trial participation for patients with rare diseases. Even in the context of close oversight, local participation models may be best suited for studies of compounds with well-understood side-effect profiles, for those with straightforward modes of administration, or for studies requiring extended follow-up periods.