Clinical Trials

Clinical trials for Alzheimer’s disease have been aimed primarily at persons who have cognitive symptoms at enrollment. However, researchers are now recognizing that the pathophysiological process of Alzheimer’s disease begins years, if not decades, prior to the onset of clinical symptoms. Successful intervention may require intervening early in the disease process. Critical issues arise in designing clinical trials for primary and secondary prevention of Alzheimer’s disease including determination of sample sizes and follow-up duration. We address a number of these issues through application of a unifying multistate model for the preclinical course of Alzheimer’s disease. A multistate model allows us to specify at which points during the long disease process the intervention exerts its effects.

We used a nonhomogeneous Markov multistate model for the progression of Alzheimer’s disease through preclinical disease states defined by biomarkers, mild cognitive impairment and Alzheimer’s disease dementia. We used transition probabilities based on several published cohort studies. Sample size methods were developed that account for factors including the initial preclinical disease state of trial participants, the primary endpoint, age-dependent transition and mortality rates and specifications of which transition rates are the targets of the intervention.

We find that Alzheimer’s disease prevention trials with a clinical primary endpoint of mild cognitive impairment or Alzheimer’s disease dementia will require sample sizes of the order many thousands of individuals with at least 5 years of follow-up, which is larger than most Alzheimer’s disease therapeutic trials conducted to date. The reasons for the large trial sizes include the long and variable preclinical period that spans decades, high rates of attrition among elderly populations due to mortality and losses to follow-up and potential selection effects, whereby healthier subjects enroll in prevention trials. A web application is available to perform sample size calculations using the methods reported here.

Sample sizes based on multistate models can account for the points in the disease process when interventions exert their effects and may lead to more accurate sample size determinations. We will need innovative strategies to help design Alzheimer’s disease prevention trials with feasible sample size requirements and durations of follow-up.

Baseline covariate imbalance (between treatment groups) is a common problem in randomized clinical trials which often raises questions about the validity of trial results. Answering these questions requires careful consideration of the statistical implications of covariate imbalance. The possibil ity of having covariate imbalance contributes to the marginal variance of an unadjusted treatment difference estimator, which can be reduced by making appropriate adjustments. Actual observed imbalance introduces a conditional bias into an unadjusted estimator, which may increase the conditional size of an unadjusted test.

This article provides conditional estimation and inference procedures to address the conditional bias due to observed imbalance. Interestingly, it is possible to use the same adjusted treatment difference estimator to address the marginal variance issue and the conditional bias issue associated with covariate imbalance. Its marginal variance estimator tends to be conservative for conditional inference, and we propose a conditionally appropriate variance estimator. We also provide an estimator of the conditional bias in an unadjusted treatment difference estimator, together with a conditional variance estimator.

The proposed methodology is illustrated with real data from a stroke trial and evaluated in simulation experiments based on the same trial. The simulation results show that covariate imbalance can result in a substantial conditional bias and that the proposed methods deal with the conditional bias quite effectively.

We recommend that the proposed methodology be used routinely to address the observed covariate imbalance in randomized clinical trials.

The multi-arm multi-stage framework uses intermediate outcomes to assess lack-of-benefit of research arms at interim stages in randomised trials with time-to-event outcomes. However, the design lacks formal methods to evaluate early evidence of overwhelming efficacy on the definitive outcome measure. We explore the operating characteristics of this extension to the multi-arm multi-stage design and how to control the pairwise and familywise type I error rate. Using real examples and the updated nstage program, we demonstrate how such a design can be developed in practice.

We used the Dunnett approach for assessing treatment arms when conducting comprehensive simulation studies to evaluate the familywise error rate, with and without interim efficacy looks on the definitive outcome measure, at the same time as the planned lack-of-benefit interim analyses on the intermediate outcome measure. We studied the effect of the timing of interim analyses, allocation ratio, lack-of-benefit boundaries, efficacy rule, number of stages and research arms on the operating characteristics of the design when efficacy stopping boundaries are incorporated. Methods for controlling the familywise error rate with efficacy looks were also addressed.

Incorporating Haybittle–Peto stopping boundaries on the definitive outcome at the interim analyses will not inflate the familywise error rate in a multi-arm design with two stages. However, this rule is conservative; in general, more liberal stopping boundaries can be used with minimal impact on the familywise error rate. Efficacy bounds in trials with three or more stages using an intermediate outcome may inflate the familywise error rate, but we show how to maintain strong control.

The multi-arm multi-stage design allows stopping for both lack-of-benefit on the intermediate outcome and efficacy on the definitive outcome at the interim stages. We provide guidelines on how to control the familywise error rate when efficacy boundaries are implemented in practice.

In the era of targeted therapies, clinical trials in oncology are rapidly evolving, wherein patients from multiple diseases are now enrolled and treated according to their genomic mutation(s). In such trials, known as basket trials, the different disease cohorts form the different baskets for inference. Several approaches have been proposed in the literature to efficiently use information from all baskets while simultaneously screening to find individual baskets where the drug works. Most proposed methods are developed in a Bayesian paradigm that requires specifying a prior distribution for a variance parameter, which controls the degree to which information is shared across baskets.

A common approach used to capture the correlated binary endpoints across baskets is Bayesian hierarchical modeling. We evaluate a Bayesian adaptive design in the context of a non-randomized basket trial and investigate three popular prior specifications: an inverse-gamma prior on the basket-level variance, a uniform prior and half-t prior on the basket-level standard deviation.

From our simulation study, we can see that the inverse-gamma prior is highly sensitive to the input hyperparameters. When the prior mean value of the variance parameter is set to be near zero ( ≤ 0 . 5 ) , this can lead to unacceptably high false-positive rates ( ≥ 40 % ) in some scenarios. Thus, use of this prior requires a fully comprehensive sensitivity analysis before implementation. Alternatively, we see that a prior that places sufficient mass in the tail, such as the uniform or half-t prior, displays desirable and robust operating characteristics over a wide range of prior specifications, with the caveat that the upper bound of the uniform prior and the scale parameter of the half-t prior must be larger than 1.

Based on the simulation results, we recommend that those involved in designing basket trials that implement hierarchical modeling avoid using a prior distribution that places a majority of the density mass near zero for the variance parameter. Priors with this property force the model to share information regardless of the true efficacy configuration of the baskets. Many commonly used inverse-gamma prior specifications have this undesirable property. We recommend to instead consider the more robust uniform prior or half-t prior on the standard deviation.

Diagnostic errors contribute to the large burden of healthcare-associated harm experienced by children. Primary care settings involve high diagnostic uncertainty and limited time and information, creating ideal conditions for diagnostic errors. We report on the design and conduct of Project RedDE, a stepped-wedge, cluster-randomized controlled trial of a virtual quality improvement collaborative aimed at reducing diagnostic errors in pediatric primary care.

Project RedDE cluster-randomized pediatric primary care practices into one of three groups. Each group participated in a quality improvement collaborative targeting the same three diagnostic errors (missed diagnoses of elevated blood pressure and adolescent depression and delayed diagnoses of abnormal laboratory studies), but in a different sequence. During the quality improvement collaborative, practices worked both independently and collaboratively, leveraging general quality improvement strategies (e.g. process mapping) in addition to error-specific content (e.g. pocket guides for blood pressure norms) delivered during the intervention phase for each error. The quality improvement collaborative intervention included interactive learning sessions and webinars, quality improvement coaching at the team level, and repeated evaluation of failures via root cause analyses. Pragmatic data were collected monthly, submitted to a centralized data aggregator, and returned to the practices in the form of run charts comparing each practice’s progress over time to that of the group. The primary analysis used patients as the unit of analysis and compared diagnostic error proportions between the intervention and baseline periods, while secondary analyses evaluated the sustainability of observed reductions in diagnostic errors after the intervention period ended.

A total of 43 practices were recruited and randomized into Project RedDE. Eleven practices withdrew before submitting any data, and one practice merged with another participating practice, leaving 31 practices that began work on Project RedDE. All but one of the diverse, national pediatric primary care practices that participated ultimately submitted complete data. Quality improvement collaborative participation was robust, with an average of 63% of practices present on quality improvement collaborative webinars and 85% of practices present for quality improvement collaborative learning sessions. Complete data included 30 months of outcome data for the first diagnostic error worked on, 24 months of outcome data for the second, and 16 months of data for the third.

Contamination across study groups was a recurring concern; concerted efforts were made to mitigate this risk. Electronic health records played a large role in teams’ success.

Project RedDE, a virtual quality improvement collaborative aimed at reducing diagnostic errors in pediatric primary care, successfully recruited and retained a diverse, national group of pediatric primary care practices. The stepped-wedge, cluster-randomized controlled trial design allowed for enhanced scientific efficiency.

More than half of the 40,000 incident rectal cancer patients in the United States each year are diagnosed at clinical stage II and III (locally advanced stage). For this group, high rates of cure can be achieved with the combination of pelvic radiation and sensitizing 5-fluorouracil (chemoradiation), surgery and chemotherapy, but treatment is long, arduous and toxicities are substantial. The PROSPECT trial (N1048, NCT01515787) was designed to determine whether neoadjuvant chemotherapy with 5-fluorouracil and oxaliplatin (FOLFOX) could be used as an alternative to neoadjuvant chemoradiation without compromising treatment outcomes and to spare these patients excess toxicity. The statistical design balanced the twin co-primary goals of achieving low local and distant recurrence rates. Study design features contended with the need for stringent safeguards given limited phase II data, the need for straightforward criteria to facilitate both accrual and protocol fidelity and the importance of patients’ perspectives on symptom burden and treatment toxicity.

PROSPECT is an ongoing multi-site two-group seamless phase II/III randomized trial comparing standard neoadjuvant chemoradiation versus neoadjuvant chemotherapy with selective use of chemoradiation for patients with locally advanced rectal cancer. Challenges addressed in the design and conduct of PROSPECT have included the following: (1) setting safety thresholds given limited single-center phase II data, (2) establishing workable eligibility criteria, (3) balancing competing time to local and distant recurrence as co-primary endpoints and (4) obtaining reliable and complete data for patients’ symptom burden. The design and implementation challenges, choices, modifications and their implications for the design of future national cooperative group clinical trials are presented.

PROSPECT incorporated stringent thresholds for both complete surgical resection (R0) and the time to local recurrence as early stopping rules. When predetermined stopping criteria were not met after evaluation of the first 366 participants in the randomized phase II, the study transitioned seamlessly to phase III with cumulative accrual of over 1000 participants. Eligibility criteria stipulating rectal tumor location based on distance from the anal verge were unworkable, and the protocol was amended to a more pragmatic approach that assigned surgeons with primary responsibility for determining eligibility. Central radiology review was feasible and in some cases prompted discontinuation of protocol treatment. Participation in toxicity reporting using the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events was uniformly high and was well accepted by participants from over 200 sites in the United States, Canada and Switzerland.

The strategies used to overcome these obstacles may inform the design of other studies that involve multi-modality treatment interventions, particularly trials where implementation of consistent criteria for eligibility and outcomes across hundreds of practice settings is necessary.

A significant barrier to conducting clinical trials is their high cost, which is driven primarily by the time and resources required to activate trials and reach accrual targets. The high cost of running trials has a substantial impact on their long-term feasibility and the type of clinical research undertaken.

A scoping review of the empirical literature on the costs associated with conducting clinical trials was undertaken for the years 2001–2015. Five reference databases were consulted to elicit how trials costs are presented in the literature. A review instrument was developed to extract the content of in-scope papers. Findings were characterized by date and place of publication, clinical disease area, and network/cooperative group designation, when specified. Costs were captured and grouped by patient accrual and management, infrastructure, and the opportunity costs associated with industry funding for trials research. Cost impacts on translational research and health systems were also captured, as were recommendations to reduce trial expenditures. Since articles often cited multiple costs, multiple cost coding was used during data extraction to capture the range and frequency of costs.

A total of 288 empirical articles were included. The distribution of reported costs was: patient management and accrual costs (132 articles), infrastructure costs (118 articles) and the opportunity costs of industry sponsorship (72 articles). 221 articles reported on the impact of undertaking costly trials on translational research and health systems; of these, the most frequently reported consequences were to research integrity (52% of articles), research capacity (36% of articles) and running low-value trials (34% of articles). 254 articles provided recommendations to reduce trial costs; of these, the most frequently reported recommendations related to improvements in: operational efficiencies (33% of articles); patient accrual (24% of articles); funding for trials and transparency in trials reporting (18% of articles, each).

Key findings from the review are: 1) delayed trial activation has costs to budgets and research; 2) poor accrual leads to low-value trials and wasted resources; 3) the pharmaceutical industry can be a pragmatic, if problematic, partner in clinical research; 4) organizational know-how and successful research collaboration are benefits of network/cooperative groups; and 5) there are spillover benefits of clinical trials to healthcare systems, including better health outcomes, enhanced research capacity, and drug cost avoidance. There is a need for more economic evaluations of the benefits of clinical research, such as health system use (or avoidance) and health outcomes in cities and health authorities with institutions that conduct clinical research, to demonstrate the affordability of clinical trials, despite their high cost.

Recruitment is an important aspect of clinical research, as poor recruitment could undermine the scientific value of a trial or delay the development process of new treatments. The development of electronic medical records provides a new way to identify potential participants for trials by matching the eligibility criteria with patients’ data within electronic medical records.

A literature search was performed to examine the effectiveness and efficiency of the electronic medical record recruitment method using MEDLINE, PubMed, PubMed Central, CINAHL Plus with Full Text, ScienceDirect and Cochrane Library databases. These searches generated 11 articles that met the eligibility criteria, and handsearching reference lists generated two additional articles bringing the total number of articles to 13. These articles were subjected to critical appraisal utilising the Effective Public Health Practice Project tool.

Out of the 13 included articles, 11 provided quantitative data on recruitment effectiveness while seven articles provided quantitative data on recruitment efficiency. The automation in screening and patient identification by using alerts, a notification system, to notify research staff of a potential participant, was observed to contribute to higher recruitment yield and reduced workload due to its specificity on participant screening. The use of electronic medical record alerts was found to be associated with better recruitment outcomes when they were sent to dedicated research staff rather than physicians. Using electronic medical records for recruitment was found to be effective due to its capability for patient identification outside working hours and fast processing time, which was particularly useful for clinical trials in acute conditions. Several challenges may hinder the impact of the electronic medical record recruitment method, including the lack of conformity of clinical trial eligibility criteria and electronic medical record data structure and missing data. ‘Alert fatigue’ could also impact on the effectiveness of this method in the long term.

The results from this review supports electronic medical record being an effective and efficient method for clinical trial recruitment. Recommendations were made in order to maximise the potential of the electronic medical record recruitment method and also for future research in order to improve the quality of evidence to support this strategy for recruitment.

In comparison with commercial drugs, there are few regulations concerning the labeling of investigational drugs. This leads to variability in their content and layout. This increases the risk of errors during storage, validation, compounding, dispensing and administration. The aim of this study was to evaluate the conformity and variability of investigational drug labels. Additional exploratory aims were to evaluate the use of an automated script to describe the labels and to identify the factors associated with the ease of finding a kit number.

An 87-criterion list was developed to evaluate content, format and readability. It included eight criteria to evaluate the conformity to the Canadian Food and Drugs Regulation. A systematic cross-sectional evaluation of all investigational drug labels in our 500-bed mother–child center was performed. All active protocols during the period of 14–22 February 2018 were included. Labels from drugs that were sourced locally were excluded. Labels affixed to the outside (external) and inside (internal) containers, as well as labels from American and European sponsors, were compared with the chi-square and Student’s t tests. A script was developed in Python to automatically determine key information (number of words, main colors and their proportion). A short survey was conducted with a convenience sample of pharmacists to rate the ease of finding the kit number on labels. Correlation was evaluated with different factors.

A total of 27 protocols were included (24 internal and 34 external labels). The majority (33/34) of external labels were compliant with the Regulation. Some internal labels did not state the expiry date (9/13), the sponsor address (2/13) or storing conditions (1/13). A total of 10 criteria were different between internal and external labels, for instance, the number of languages was higher on external labels (median 3 (2–14) vs 10 (2–50); p = 0.013). Five criteria were different depending on the sponsors’ location, for instance, European sponsors were more prone to use bold characters (25% vs 61%, p = 0.034). There was a mean of 146 ± 111 words and 78.3% ± 7.3% empty space per label. These were positively correlated (p < 0.001). The proportion of free space on a label was also correlated with the ease of finding the kit number (p = 0.002).

We measured a high variability in the labeling of investigational drugs. Key information was missing from labels affixed to internal containers, despite the use of a high number of pages. The automation worked well and further work is needed to identify criteria that may improve readability and reduce error risk. Detailed and harmonized international guidelines are needed.