Abstract
Background:
Drug development in Alzheimer’s disease (AD) over the past two decades has had high rates of failure. Novel trial designs, such as adaptive designs, have the potential to improve the efficiency of drug development in AD.
Objective:
To evaluate the design characteristics, temporal trends, and differences in design between sponsor types in phase II trials of investigational agents in AD.
Methods:
Phase I/II, II, and II/III trials for AD with drug or other biological interventions registered from December 1996 to December 2021 in ClinicalTrials.gov were included. Descriptive statistics were used to summarize trial characteristics. Linear, logistic, and multinomial regression models assessed temporal trends and differences between sponsor types in design characteristics.
Results:
Of N = 474 trials identified, randomized parallel group design was the most common design (72%). Only 12 trials (2.5%) used an adaptive design; adaptive features included early stopping rules, model-based dose-finding, adaptive treatment arm selection, and response adaptive randomization. The use of non-randomized parallel-group and open-label single arm designs increased over time. No temporal trend in the use of adaptive design was identified. Trials sponsored by industry only were more likely to use a randomized parallel-group design and have a larger estimated sample size than trials with other sponsor types.
Conclusion:
Our systematic review showed that very few phase II trials in AD used an adaptive trial design. Innovation and implementation of novel trial designs in AD trials can accelerate the drug development process.
INTRODUCTION
Alzheimer’s disease (AD) is a neurodegenerative disease that causes loss of memory and other cognitive abilities [1]. More than 55 million people are living with dementia worldwide, and an estimated 6.5 million people in United States age 65 and older are living with AD in 2022 [2, 3]. The worldwide prevalence of AD is expected to triple from 50 to 150 million by 2050 [3 –5]. To date, therapeutic options are limited [5]. There are currently only seven drugs, which may delay clinical decline or help lessen the AD symptoms, approved by the U.S. Food and Drug Administration (FDA) for AD [5]. The two most recently approved drugs, lecanemab and aducanumab, are the first agents to address the underlying biology of AD [5, 6]. However, none of these are curative or can stop its progression [1, 3]. It is crucial to identify and develop new treatments for AD, and to test efficacy and safety rigorously, through well-designed clinical trials.
A variety of trial designs have been used in AD trials. The traditional trial design used to assess the efficacy of new treatments is the double-blind, placebo-controlled, randomized parallel group design [4 , 8]. Participants are randomized to the placebo group or treatment group. Other designs used in AD trials include: futility designs, staggered start/delayed withdrawal designs, time-to-onset designs, and others [7]. Nonetheless, the standard design remains the most commonly used. Given the high failure rate of AD drug development, innovations in trial designs are critical to shortening the development cycle for AD treatments [9].
Novel adaptive designs have also been used in neurodegenerative disorders to increase the flexibility of a trial and efficiency of clinical development [8 , 10–12]. According to FDA guidance on adaptive trials, an adaptive design is defined as “a clinical trial design that allows for prospectively planned modifications to one or more aspects of the design based on accumulating data from subjects in the trial” [13]. Adaptive features may include: 1) early termination for efficacy, futility or both; 2) adaptations to treatment arm selection (including adaptive dose finding); 3) adaptations to the sample size; 4) adaptations to the patient population (including adaptive enrichment); 5) adaptations to patient allocation; and 6) adaptations to endpoint selection (Box 1) [13]. van Eijk et al. (2018) [11] reviewed clinical trials in neurodegenerative disorders using group sequential trial designs and identified the reduction in sample size and the total trial duration by using the adaptive design. For the phase II trial of BAN2401, a monoclonal antibody targeting amyloid protofibrils, a Bayesian adaptive proof-of-concept trial design was used to identify optimal dose. The trial’s adaptive features included response adaptive randomization and early termination of the study for futility or a signal for success [14]. This design allowed investigators to gain the trial efficiency by avoiding unnecessary patient exposures to an ineffective treatment.
Between 2004 to 2022, drug development in AD had only a 2% success rate in achieving regulatory approval [15]. Only two agents (aducanumab and oligomannate) were approved compared to 98 unique compounds that failed in phase II or phase III testing. (Lecanemab was recently approved in June 2023.) One potential reason for failure could be due to lack of clear results in phase II to initiate phase III testing [15]. Adaptive designs in clinical research became more attractive to investigators with the aims of increase in flexibility and efficiency of trials [16]. It has been shown that the use of adaptive designs can be more reliable and efficient in the early phase of clinical drug development [11 , 16]. Hatfield et al. [17] also identified that recent clinical trials registered on ClinicalTrials.gov used adaptive designs most frequently in phase II trials in oncology. Increasing the use of adaptive designs in phase II development can improve efficiency to make accurate “go/no-go” decisions to phase III testing.
In this study, we conducted a systematic review of trial designs focusing on phase II trials in AD registered on ClincialTrials.gov. This systematic review will: 1) describe the trial designs and characteristics in phase II interventional trials in AD, 2) identify temporal trends in the design of AD trials, and 3) compare the trial characteristics among sponsor types in AD trials.
MATERIALS AND METHODS
Data source
ClinicalTrials.gov is a database of privately and publicly funded clinical studies conducted around the world, with over 413,591 registered trials from 220 countries. Trials conducted in the U.S. must be registered on ClinicalTrials.gov; trials conducted outside of the U.S. may but are not required to be registered. Trial records were accessed on August 20, 2021. Search filters were used to select the trials which met our inclusion/exclusion criteria. Trials were included if they were classified as “interventional” trials and the conditions or diseases were reported as “Alzheimer’s Disease”. The study phase was required to be reported as Phase I/II, Phase II, or Phase II/III, and the first posted date on ClinicalTrials.gov was before July 1, 2021. Only “drug” (e.g., small molecules) or “biological” interventions were included; trials of other intervention types such as behavioral, combination product, device, dietary supplement, procedure, and radiation were excluded. Our study focused on reviewing phase II drug or biologic agents given the high failure rate of past AD drug trials; future work will expand our investigation to phase III trials and other intervention types. Open-label extension trials were identified by manually reviewing trials with the study status “enrolling by invitation” or the trial record contains the term “extension” on ClinicalTrials.gov. These open-label extension trials were excluded from analysis.
Data collection
For each trial, available discrete data fields were directly downloaded from ClinicalTrials.gov including NCT ID (primary key), trial title, study phase (phase I/II, phase II, or phase II/III), sponsor (industry only/other), study status, first posted date, and trial completion date. Additional trial characteristics were manually abstracted from the clinicaltrials.gov record including study design (randomized parallel, randomized crossover, open-label single arm, etc.), use of adaptive design, adaptive feature(s) (early stopping for futility/efficacy, adaptive dose finding, etc.), number of arms, use of randomization, randomization ratio, blinding, and estimated sample size. If the requisite trial characteristics were not listed or were unclear on the trial record, a PubMed search for publications related to the trials based on their NCT ID was performed. The manually abstracted dataset was merged with the downloaded dataset to generate the dataset for analysis.
Data were abstracted by two investigators (A.L. and D.S.) and then cross-checked for accuracy by a third investigator (C.M.). In addition, cross-tabulations were used to identify inconsistencies between fields. Inconsistencies were settled through discussion.
Statistical methods
Trial characteristics were summarized using descriptive statistics. Proportions and frequencies were used for categorical variables, while medians and ranges were used for continuous variables. To investigate temporal trends in trial characteristics, trials were categorized into five time periods based on the year of trial registration (first posted date): (1 = 1999–2004, 2 = 2005–2009, 3 = 2010–2014, 4 = 2015–2019, 5 = 2020–2021). To test the temporal trend in the estimated sample size, linear regression was used, treating year groups of trial registration as a continuous predictor. For binary trial characteristics (Adaptive Design (Y/N), randomization (Y/N), study sponsor (industry only versus other [NIH, other federal, all others]), logistic regression was used treating year groups of trial registration as a continuous predictor. For multi-category trial characteristics, multinomial logistic regression was used treating year groups as a continuous predictor. Comparison of characteristics between trial sponsor types (industry only versus other) was conducted with chi-squared tests or two-sample t-tests. Two-sided p-values <0.05 were considered statistically significant. Statistical analyses were performed using the nnet package version 7.3–17 [18] in R version 4.0.2 [19].
RESULTS
Figure 1 presents the flow diagram of the trial search. Among the 2,155 trials that were identified, 1,681 trials did not meet our inclusion criteria, yielding a total of N = 474 trials eligible to be analyzed. A majority of trials (81%) were single phase II studies, while 11% and 8% were dual phase I/II and phase II/III studies, respectively (Table 1). Over half (56%) of the studies were complete at the time of data collection; 65 (13%) trials were actively recruiting participants. The year of trial registration ranged from 1999 to 2021. Fifty-three percent of trials were sponsored by industry only; the remaining trials had other sponsors.
Flow diagram for trials included from clinicaltrials.gov.
Characteristics of N = 474 phase II clinical trials in Alzheimer’s disease from clinicaltrials.gov
*Other designs include: Factorial design (4 trials), dose finding design (2), non-randomized crossover design (1), and unknown design (2). **The calculation of time duration was only restricted to trials with completed, terminated, and withdrawn study status. Trials with missing trial completion date or negative calculated time duration were excluded.
Design characteristics
The most common design was the standard randomized parallel-group design (n = 339/474 trials; 72%) (Table 1). Other frequently used designs included the open-label single arm design (15%), randomized crossover (6%), and non-randomized parallel-group design (5%). A majority of trials (80%) were randomized and most (n = 393/459; 86%) had two or more arms. Among 380 randomized trials with available information regarding the randomization ratio, a majority (33%) had equal allocation across arms. The median target sample size was 106.5 participants (range = 5–2,935).
Only 12 out of 474 (2.5%) trials used an adaptive design (Tables 1 and 2). Of these, four included early stopping rules for efficacy or futility; three used model-based dose finding designs; two were platform trials which allowed adaptations to treatment arm selection; one trial with response adaptive randomization allowed adaptations to patient allocation based on outcome data; and two trials had unspecified adaptive features.
Listing of N = 12 trials with adaptive features
Temporal trends in trial design
The use of non-randomized parallel-group (Odds Ratio [OR] = 1.76; 95% confidence interval [CI] = 1.15–2.71; p = 0.010) and open-label single arm designs increased over time (OR = 1.58; 95% CI = 1.23–2.03; p≤0.001), relative to the standard randomized parallel-group design (Fig. 2A). The use of randomization decreased over time (OR = 0.64; 95% CI = 0.51–0.80; p≤0.001; Fig. 2B). The proportion of trials with industry only sponsors decreased over time (OR = 0.71; 95% CI = 0.59–0.84; p≤0.001), relative to other sponsor types (Fig. 2C). Relative to standalone phase II studies, dual phase I/II studies increased over time (OR = 1.60; 95% CI = 1.19–2.15; p = 0.002), whereas dual phase II/III studies showed no clear trend (OR = 1.09; 95% CI = 0.80–1.49; p = 0.58; Fig. 2D). The estimated sample size per trial also showed no clear temporal trend (p = 0.58; Fig. 2E). Of the 12 trials employing adaptive designs, there was no clear temporal trend for use thereof (OR = 1.00; 95% CI = 0.59–1.72; p = 1; Fig. 2F).
Temporal trends in trial characteristics. A) Study design; B) Use of randomization; C) Sponsor type; D) Study phase; E) Estimated sample size per trial; and F) Number of adaptive trials. * N = 27, 115, 131, 154, and 47 trials were registered in 1999–2004, 2005–2009, 2010–2014, 2015–2019, and 2020–2021, respectively.
Design characteristics by sponsor type
Industry-only sponsored trials were more likely to use a randomized parallel-group design, compared to trials with other sponsor types (p = 0.004; Table 3). Industry only sponsored trials had larger estimated sample sizes on average compared to trials with other sponsors (p≤0.001), and single phase II studies were more likely to be used in industry only sponsored trials (p = 0.0065). Use of randomization differed significantly by sponsor type (p = 0.04). There was no significant difference in the use of adaptive features by sponsor type (p = 0.34).
Comparison of trial characteristics by sponsor type
DISCUSSION
Drug development for AD has faced high rates of failure, with only three new agents [15] for primary disease progression approved by the U.S. Food and Drug Administration (FDA) since 2003. Current agents under clinical development target a wide range of novel disease mechanisms, including amyloid, tau, inflammation, and synaptic plasticity [4]. Of the 143 agents in 172 ongoing trials for AD in 2022, over half the agents (82/143; 57%) and trials (94/172; 55%) are in middle (phase II) development. Our analysis of temporal trends in trial characteristics suggest more AD are testing novel mechanisms of action in earlier phases of development. More recently registered trials were more likely to use a non-randomized parallel assignment or open-label single arm design, incorporate dual phase I/phase II designs, and include non-industry (academia, government) sponsors. However, the increasing use of non-randomized designs may be a worrying trend as these designs provide less rigorous evidence and may be prone to bias.
Overall, we confirmed that a large majority of trials still use the “standard” randomized parallel-group design, though their relative use is decreasing over time. However, only a very small proportion (2.5%) of phase II AD trials employed adaptive designs. Adaptive designs are most commonly used in oncology trials [17]. A 2020 systematic review identified that 15% of pediatric oncology phase I trials in the United States used model-based dose finding designs [20]. Reviews from other therapeutic areas also showed infrequent use of adaptive designs. For randomized trials for kidney dialysis, for example, only an average of 1.82% trials used adaptive designs.
We did not find a significant temporal trend in the use of adaptive designs in AD trials. This contrasts with evidence from certain other disease entities suggesting that the use of adaptive designs have increased over time. Across all disease areas, the number of adaptive trials increased from 11 to 38 per 10,000 registered trials from 2001 to 2013 [17]. More recently, the use of adaptive master protocol designs have increased dramatically, particularly in cancer and infectious disease trials [21]. However, these trends are not uniform across all therapeutic areas. No significant trend in the use of adaptive designs was observed in pediatric oncology phase I trials [20]. For dialysis trials, the proportion of adaptive trials actually decreased from 6.12% in 2009 to 0.43% in 2019 [22].
We identified only 12 AD trials with adaptive features; the type of adaptation varied between trials. Early termination for futility or efficacy was the most common (n = 4/12; 33%), followed by adaptive dose finding (25%), adaptations to treatment arm selection (25%), and adaptations to patient allocation based on outcome data (8%). Notably, none of the 12 AD trials included adaptations to the patient population (adaptive enrichment) or sample size re-estimation. Hatfield et al. reported that for phase II trials across all diseases, group sequential designs (early stopping), and adaptive dose selection designs were the most common adaptation [17]. A more recent review of 142 adaptive phase II and phase III trials identified the most common adaptations were: seamless phase II/III (57% of trials), group sequential (21%), biomarker adaptive (20%), and adaptive dose-finding (16%) [23].
Overview of common adaptive features in clinical trials based on U.S. Food and Drug Administration (FDA) guidance
These findings motivate the need to design more trials with adaptive features to improve the efficiency of AD drug development (Box 1). Trials with early stopping rules for futility can reduce trial resources spent on agents that are unlikely to succeed. Conversely, trials with adaptive sample size re-estimation can allow investigators to modify the overall sample size based on observed interim estimates of design parameters (e.g., effect size) to ensure the final analysis will be well-powered [24, 25]. Given the heterogeneity of AD, trials that can adaptively enrich for patients more likely to respond to treatment will be an effective strategy [26, 27]. Model-based dose finding methods [28, 29] can improve the accuracy of identifying the optimal dose compared to traditional designs [30], thus improving the chance for the agent to achieve a therapeutic effect in phase II development. Adaptive platform designs can streamline the AD drug development pipeline by establishing common a trial infrastructure to test multiple agents [31, 32]. Finally, response-adaptive randomization can adaptively assign more patients to more promising treatment arms [33].
However, a number of challenges and barriers to conducting more adaptive trials need to be addressed [13, 34]. Adaptive trials are more complex to design and conduct. Rapid data entry is required for interim analyses. Adaptive modifications require careful control of the type 1 error rate to account for multiple testing. Prior to conducting adaptive trials, pre-planned modifications need to be carefully specified in the trial protocol. Funding, regulatory, and communication challenges can also occur when conducting adaptive trials [34]. A recent mixed methods study examining the attitudes and opinions towards adaptive trials among different trial stakeholders identified that there is variable understanding of adaptive trials among stakeholders (more limited understanding by clinicians and greater perceived understanding by regulators [35]). Participants identified that adaptive trials have the potential for improved efficiency depending the situation and that more education on adaptive designs is needed.
In order to increase the use of adaptive designs in AD trials, we first need to raise the awareness of adaptive designs among the AD clinical research community, including clinicians, statisticians, regulators, pharmaceutical companies, research ethics boards, funding agencies, and patient support groups. By sharing the benefits of adaptive designs over traditional designs with all stakeholders, we can encourage study teams to implement these designs, where appropriate and feasible.
Second, we need to develop user-friendly, accessible software to aid the design and conduct of these complex trials. Often, to design adaptive trials, investigators and statisticians need to perform statistical simulations to assess the operating characteristics of the design across a wide range of potential parameters. While there are free, open-source packages available in R, these programs generally require programming knowledge and may not be accessible by non-statisticians [36 –38]. Commercial software include a point-and-click, interactive interface, but these programs are typically expensive to license and may not be accessible to all investigators and institutions [39, 40]. There are relatively few interactive, web-based platforms available to design adaptive trials [36, 41].
Third, we need to develop designs tailored to the needs to AD trials. A majority of adaptive designs (master protocols, group sequential designs, response-adaptive randomization) were developed by cancer statisticians for oncology trials. For example in cancer trials, response-adaptive randomization works well for a binary endpoint (e.g., best overall response) measured over relatively short time-periods (e.g., 1–2 months) post-randomization. This allows for sufficient efficacy data to be accrued in order to update the randomization ratio on a frequent basis. In comparison, AD trials typically measure continuous participant cognitive, behavioral, or symptom outcome scores repeatedly over a longer follow-up period, making response-adaptive randomization more challenging to implement. We thus need to develop new designs or modify existing designs to address the distributional and logistical challenges for conducting AD trials.
Our study is constrained by the completeness and accuracy of trial records reported on clinicaltrials.gov. Data elements from structured fields in clinicaltrials.gov were generally well-populated; data missingness ranged from 0% to 7.6% (Table 1). As prior studies have identified, relying solely on the information available in these records may lead to an underestimation of the rate of adaptive designs [17]. To ensure accurate estimates, we searched for publications referencing the trial’s NCT ID to confirm and/or augment the trial design characteristics for each trial. We also performed searches by keyword, such as “adaptive” or “Bayesian”, in clinicaltrials.gov, to ensure that we accurately captured all potential adaptive designs. Further, our study focused on trials registered on U.S. based clinicaltrials.gov; results based on trials registered in other trial registries (e.g., European Union Drug Regulating Authorities Clinical Trials Database [EuDRACT]) may differ.
In summary, our study identifies the urgent need to modernize the design of AD trials. Adaptive designs have already improved the efficiency of clinical development in cancer, infectious diseases, and other therapeutic areas. The current AD drug development pipeline is looking for early signals of efficacy among candidate drugs primarily in Phase II trials. This presents an opportunity to introduce adaptive features to increase the efficiency of phase II trials. All stakeholders must work together to add adaptive designs to our clinical trials “toolbox” in order to more rapidly bring new AD treatments from bench to bedside.
Footnotes
ACKNOWLEDGMENTS
The authors would like to acknowledge the Centre for Complex Interventions team and Biostatistics Core Services team at the Centre for Addiction and Mental Health (CAMH) for their thoughtful discussion and contributions for this study.
FUNDING
Funding for this study was provided by the CAMH Discovery Fund and the Centre for Complex Interventions at CAMH.
CONFLICT OF INTEREST
Tarek K. Rajji has received research support from Brain Canada, Brain and Behavior Research Foundation, BrightFocus Foundation, Canada Foundation for Innovation, Canada Research Chair, Canadian Institutes of Health Research, Centre for Aging and Brain Health Innovation, National Institutes of Health, Ontario Ministry of Health and Long-Term Care, Ontario Ministry of Research and Innovation, and the Weston Brain Institute. Tarek K. Rajji also received for an investigator-initiated study in-kind equipment support from Newronika, and in-kind research online accounts from Scientific Brain Training Pro, and participated in 2021 and 2022 in an advisory activity for Biogen Canada Inc. Tarek K. Rajji is also an inventor on the United States Provisional Patent No. 17/396,030 that describes cell-based assays and kits for assessing serum cholinergic receptor activity.
All other authors have no conflict of interest to report.
DATA AVAILABILITY
The datasets were directly downloaded from ClinicalTrials.gov including NCT ID (primary key), trial title, study phase, sponsor, study status, first posted date, and trial completion date. The data supporting the findings of this study are available upon reasonable request from the corresponding author.