Clinical Trials

In June 2013, a 1-day workshop on Dynamic Treatment Strategies (DTSs) and Sequential Multiple Assignment Randomized Trials (SMARTs) was held at the University of Pennsylvania in Philadelphia, Pennsylvania. These two linked topics have generated a great deal of interest as researchers have recognized the importance of comparing entire strategies for managing chronic disease. A number of articles emerged from that workshop.

The purpose of this survey of the DTS/SMART methodology (which is taken from the introductory talk in the workshop) is to provide the reader the collected articles presented in this volume with sufficient background to appreciate the more detailed discussions in the articles.

The way that the DTS arises naturally in clinical practice is described, along with its connection to the well-known difficulties of interpreting the analysis by intention-to-treat. The SMART methodology for comparing DTS is described, and the basics of estimation and inference presented.

The DTS/SMART methodology can be a flexible and practical way to optimize ongoing clinical decision making, providing evidence (based on randomization) for comparative effectiveness.

The DTS/SMART methodology is not a solution for unstandardized study protocols.

The DTS/SMART methodology has growing relevance to comparative effectiveness research and the needs of the learning healthcare system.

Recent advances in medical research suggest that the optimal treatment rules should be adaptive to patients over time. This has led to an increasing interest in studying dynamic treatment regime, a sequence of individualized treatment rules, one per stage of clinical intervention, which maps present patient information to a recommended treatment. There has been a recent surge of statistical work for estimating optimal dynamic treatment regimes from randomized and observational studies. The purpose of this article is to review recent methodological progress and applied issues associated with estimating optimal dynamic treatment regimes.

We discuss sequential multiple assignment randomized trials, a clinical trial design used to study treatment sequences. We use a common estimator of an optimal dynamic treatment regime that applies to sequential multiple assignment randomized trials data as a platform to discuss several practical and methodological issues.

We provide a limited survey of practical issues associated with modeling sequential multiple assignment randomized trials data. We review some existing estimators of optimal dynamic treatment regimes and discuss practical issues associated with these methods including model building, missing data, statistical inference, and choosing an outcome when only non-responders are re-randomized. We mainly focus on the estimation and inference of dynamic treatment regimes using sequential multiple assignment randomized trials data. Dynamic treatment regimes can also be constructed from observational data, which may be easier to obtain in practice; however, care must be taken to account for potential confounding.

A dynamic treatment regime (DTR) comprises a sequence of decision rules, one per stage of intervention, that recommends how to individualize treatment to patients based on evolving treatment and covariate history. These regimes are useful for managing chronic disorders, and fit into the larger paradigm of personalized medicine. The Value of a DTR is the expected outcome when the DTR is used to assign treatments to a population of interest.

The Value of a data-driven DTR, estimated using data from a Sequential Multiple Assignment Randomized Trial, is both a data-dependent parameter and a non-smooth function of the underlying generative distribution. These features introduce additional variability that is not accounted for by standard methods for conducting statistical inference, for example, the bootstrap or normal approximations, if applied without adjustment. Our purpose is to provide a feasible method for constructing valid confidence intervals (CIs) for this quantity of practical interest.

We propose a conceptually simple and computationally feasible method for constructing valid CIs for the Value of an estimated DTR based on subsampling. The method is self-tuning by virtue of an approach called the double bootstrap. We demonstrate the proposed method using a series of simulated experiments.

The proposed method offers considerable improvement in terms of coverage rates of the CIs over the standard bootstrap approach.

In this article, we have restricted our attention to Q-learning for estimating the optimal DTR. However, other methods can be employed for this purpose; to keep the discussion focused, we have not explored these alternatives.

Subsampling-based CIs provide much better performance compared to standard bootstrap for the Value of an estimated DTR.

A behavioral intervention is a program aimed at modifying behavior for the purpose of treating or preventing disease, promoting health, and/or enhancing well-being. Many behavioral interventions are dynamic treatment regimens, that is, sequential, individualized multicomponent interventions in which the intensity and/or type of treatment is varied in response to the needs and progress of the individual participant. The multiphase optimization strategy (MOST) is a comprehensive framework for development, optimization, and evaluation of behavioral interventions, including dynamic treatment regimens. The objective of optimization is to make dynamic treatment regimens more effective, efficient, scalable, and sustainable. An important tool for optimization of dynamic treatment regimens is the sequential, multiple assignment, randomized trial (SMART). The purpose of this article is to discuss how to develop optimized dynamic treatment regimens within the MOST framework.

The article discusses the preparation, optimization, and evaluation phases of MOST. It is shown how MOST can be used to develop a dynamic treatment regimen to meet a prespecified optimization criterion. The SMART is an efficient experimental design for gathering the information needed to optimize a dynamic treatment regimen within MOST. One signature feature of the SMART is that randomization takes place at more than one point in time.

MOST and SMART can be used to develop optimized dynamic treatment regimens that will have a greater public health impact.

Due to the cost and complexity of conducting a sequential multiple assignment randomized trial (SMART), it is desirable to pre-define a small number of personalized regimes to study.

We proposed a simulation-based approach to studying personalized dosing strategies in contexts for which a therapeutic agent’s pharmacokinetic and pharmacodynamics properties are well understood. We take dosing of warfarin as a case study, as its properties are well understood. We consider a SMART in which there are five intervention points in which dosing may be modified, following a loading phase of treatment.

Realistic SMARTs are simulated, and two methods of analysis, G-estimation and Q-learning, are used to assess potential personalized dosing strategies.

In settings where outcome modelling may be complex due to the highly non-linear nature of the pharmacokinetic and pharmacodynamics mechanisms of the therapeutic agent, G-estimation provides for which the more promising method of estimating an optimal dosing strategy. Used in combination with the simulated SMARTs, we were able to improve simulated patient outcomes and suggest which patient characteristics were needed to best individually tailor dosing. In particular, our simulations suggest that current dosing should be determined by an individual’s current coagulation time as measured by the international normalized ratio (INR), their last measured INR, and their last dose. Tailoring treatment only based on current INR and last warfarin dose provided inferior control of INR over the course of the trial.

The ability of the simulated SMARTs to suggest optimal personalized dosing strategies relies on the pharmacokinetic and pharmacodynamic models used to generate the hypothetical patient profiles. This approach is best suited to therapeutic agents whose effects are well studied.

Prior to investing in a complex randomized trial that involves sequential treatment allocations, simulations should be used where possible in order to guide which dosing strategies to evaluate.

Cancer affects millions of people worldwide each year. Patients require sequences of treatment based on their response to previous treatments to combat cancer and fight metastases. Physicians provide treatment based on clinical characteristics, changing over time. Guidelines for these individualized sequences of treatments are known as dynamic treatment regimens (DTRs) where the initial treatment and subsequent modifications depend on the response to previous treatments, disease progression, and other patient characteristics or behaviors. To provide evidence-based DTRs, the Sequential Multiple Assignment Randomized Trial (SMART) has emerged over the past few decades.

To examine and learn from past SMARTs investigating cancer treatment options, to discuss potential limitations preventing the widespread use of SMARTs in cancer research, and to describe courses of action to increase the implementation of SMARTs and collaboration between statisticians and clinicians.

There have been SMARTs investigating treatment questions in areas of cancer, but the novelty and perceived complexity has limited its use. By building bridges between statisticians and clinicians, clarifying research objectives, and furthering methods work, there should be an increase in SMARTs addressing relevant cancer treatment questions. Within any area of cancer, SMARTs develop DTRs that can guide treatment decisions over the disease history and improve patient outcomes.

Clinical trials frequently spend considerable effort to collect data on patients who were assessed for eligibility but not enrolled. The Consolidated Standards of Reporting Trials (CONSORT) guidelines’ recommended flow diagram for randomized clinical trials reinforces the belief that the collection of screening data is a necessary and worthwhile endeavor. The rationale for collecting screening data includes scientific, trial management, and ethno-socio-cultural reasons.

We posit that the cost of collecting screening data is not justified, in part due to inability to centrally monitor and verify the screening data in the same manner as other clinical trial data.

To illustrate the effort and site-to-site variability, we analyzed the screening data from a multicenter, randomized clinical trial of patients with transient ischemic attack or minor ischemic stroke (Platelet-Oriented Inhibition in New Transient Ischemic Attack and Minor Ischemic Stroke (POINT)).

Data were collected on over 27,000 patients screened across 172 enrolling sites, 95% of whom were not enrolled. Although the rate of return of screen failure logs was high overall (95%), there were a considerable number of logs that were returned with ‘no data to report’ (23%), often due to administrative reasons rather than no patients screened.

In spite of attempts to standardize the collection of screening data, due to differences in site processes, multicenter clinical trials face challenges in collecting those data completely and uniformly. The efforts required to centrally collect high-quality data on an extensive number of screened patients may outweigh the scientific value of the data. Moreover, the lack of a standardized definition of ‘screened’ and the challenges of collecting meaningful characteristics for patients who have not signed consent limits the ability to compare across studies and to assess generalizability and selection bias as intended.

Phase II clinical trials are important milestones to determine whether a dose-effect exists and to decide on future doses to use in confirmatory studies. To take into account the overall shape of the dose–response curve, modeling the relationship by linear or non-linear models is preferable to the classical pair-wise comparisons of the effect of each dose versus the placebo or the comparator. The multiple comparisons and modeling approach has been developed within the last 10 years to address this important question in the clinical development of drugs. Despite some recent publications referring to this methodology, few detailed applications have been shown so far and several practical questions remain to be addressed.

Starting from a set of candidate models, model selection using classical methods criteria is possible. However, it suffers some limitations, not taking into account the uncertainty of the selection process itself. An attractive solution is to use model averaging, which applies appropriate weights to the parameters (e.g., the minimum effective dose) obtained from each model.

A discussion of the selection criteria is first presented. Through two real examples, how to proceed with model selection and model averaging is presented and discussed.

The first multiple comparisons and modeling approach papers addressed normal responses. More recently, an extension of this methodology has been proposed to deal with other types of responses, in particular binary, time-to-event and longitudinal data. Questions that remain are concerned with the choice of the candidate models and of their parameters’ guesstimates.

The analysis of clinical dose-finding studies using a modeling of the entire curve offers a promising alternative as compared with the classical multiple comparisons methods, while not compromising the necessary rigor of the analysis.

Bayesian predictive probabilities can be used for interim monitoring of clinical trials to estimate the probability of observing a statistically significant treatment effect if the trial were to continue to its predefined maximum sample size.

We explore settings in which Bayesian predictive probabilities are advantageous for interim monitoring compared to Bayesian posterior probabilities, p-values, conditional power, or group sequential methods.

For interim analyses that address prediction hypotheses, such as futility monitoring and efficacy monitoring with lagged outcomes, only predictive probabilities properly account for the amount of data remaining to be observed in a clinical trial and have the flexibility to incorporate additional information via auxiliary variables.

Computational burdens limit the feasibility of predictive probabilities in many clinical trial settings. The specification of prior distributions brings additional challenges for regulatory approval.

The use of Bayesian predictive probabilities enables the choice of logical interim stopping rules that closely align with the clinical decision-making process.

Missing data are unavoidable in most randomized controlled clinical trials, especially when measurements are taken repeatedly. If strong assumptions about the missing data are not accurate, crude statistical analyses are biased and can lead to false inferences. Furthermore, if we fail to measure all predictors of missing data, we may not be able to model the missing data process sufficiently. In longitudinal randomized trials, measuring a patient’s intent to attend future study visits may help to address both of these problems. Leon et al. developed and included the Intent to Attend assessment in the Lithium Treatment – Moderate dose Use Study (LiTMUS), aiming to remove bias due to missing data from the primary study hypothesis.

The purpose of this study is to assess the performance of the Intent to Attend assessment with regard to its use in a sensitivity analysis of missing data.

We fit marginal models to assess whether a patient’s self-rated intent predicted actual study adherence. We applied inverse probability of attrition weighting (IPAW) coupled with patient intent to assess whether there existed treatment group differences in response over time. We compared the IPAW results to those obtained using other methods.

Patient-rated intent predicted missed study visits, even when adjusting for other predictors of missing data. On average, the hazard of retention increased by 19% for every one-point increase in intent. We also found that more severe mania, male gender, and a previously missed visit predicted subsequent absence. Although we found no difference in response between the randomized treatment groups, IPAW increased the estimated group difference over time.

LiTMUS was designed to limit missed study visits, which may have attenuated the effects of adjusting for missing data. Additionally, IPAW can be less efficient and less powerful than maximum likelihood or Bayesian estimators, given that the parametric model is well specified.

In LiTMUS, the Intent to Attend assessment predicted missed study visits. This item was incorporated into our IPAW models and helped reduce bias due to informative missing data. This analysis should both encourage and facilitate future use of the Intent to Attend assessment along with IPAW to address missing data in a randomized trial.

When randomizations are assigned at the cluster level for longitudinal cluster randomized trials (longitudinal-CRTs) with a continuous outcome, formulae for determining the required sample size to detect a two-way interaction effect between time and intervention are available.

To show that (1) those same formulae can also be applied to longitudinal trials when randomizations are assigned at the subject level within clusters and (2) this property can be extended to 2-by-2 factorial longitudinal-CRTs with two treatments and different levels of randomization for which testing a three-way interaction between time and the two interventions is of primary interest.

We show that slope estimates from different treatment arms are uncorrelated, regardless of whether randomization occurs at the third or second level and also regardless of whether slopes are considered fixed or random in the mixed-effects model for testing two-way or three-way interactions. Sample size formulae are extended to unbalanced designs. Simulation studies were applied to verify the findings.

Sample size formulae for testing two-way and three-way interactions in longitudinal-CRTs with second-level randomization are identical to those for trials with third-level randomization. In addition, the total number of observations required for testing a three-way interaction is demonstrated to be four times as large as that required for testing a two-way interaction, regardless of the level of randomization for both fixed- and random-slope models.

The findings may be only applicable to longitudinal-CRTs with normally distributed continuous outcome.

All of the findings are validated by simulation studies and enable the design of longitudinal clinical trials to be more flexible in regard to the level of randomization and allocation of clusters and subjects.