Clinical Trials

A surrogate endpoint is an endpoint observed earlier than the true endpoint (a health outcome) that is used to draw conclusions about the effect of treatment on the unobserved true endpoint. A prognostic marker is a marker for predicting the risk of an event given a control treatment; it informs treatment decisions when there is information on anticipated benefits and harms of a new treatment applied to persons at high risk. A predictive marker is a marker for predicting the effect of treatment on outcome in a subgroup of patients or study participants; it provides more rigorous information for treatment selection than a prognostic marker when it is based on estimated treatment effects in a randomized trial.

We organized our discussion around a different theme for each topic.

“Fundamentally an extrapolation” refers to the non-statistical considerations and assumptions needed when using surrogate endpoints to evaluate a new treatment. “Decision analysis to the rescue” refers to use the use of decision analysis to evaluate an additional prognostic marker because it is not possible to choose between purely statistical measures of marker performance. “The appeal of simplicity” refers to a straightforward and efficient use of a single randomized trial to evaluate overall treatment effect and treatment effect within subgroups using predictive markers.

The simple themes provide a general guideline for evaluation of surrogate endpoints, prognostic markers, and predictive markers.

A surrogate marker is a variable commonly used in clinical trials to guide treatment decisions when the outcome of ultimate interest is not available. A good surrogate marker is one where the treatment effect on the surrogate is a strong predictor of the effect of treatment on the outcome. We review the situation when there is one treatment delivered at baseline, one surrogate measured at one later time point, and one ultimate outcome of interest and discuss new issues arising when variables are time-varying.

Most of the literature on surrogate markers has only considered simple settings with one treatment, one surrogate, and one outcome of interest at a fixed time point. However, more complicated time-varying settings are common in practice. In this article, we describe the unique challenges in two settings, time-varying treatments and time-varying surrogates, while relating the ideas back to the causal-effects and causal-association paradigms.

In addition to discussing and extending popular notions of surrogacy to time-varying settings, we give examples illustrating that one can be misled by not taking into account time-varying information about the surrogate or treatment. We hope this article has provided some motivation for future work on estimation and inference in such settings.

The validation of intermediate markers as surrogate markers (S) for the true outcome of interest (T) in clinical trials offers the possibility for trials to be run more quickly and cheaply by using the surrogate endpoint in place of the true endpoint.

Working within a principal stratification framework, we propose causal quantities to evaluate surrogacy using a Gaussian copula model for an ordinal surrogate and time-to-event final outcome. The methods are applied to data from four colorectal cancer clinical trials, where S is tumor response and T is overall survival.

For the Gaussian copula model, a Bayesian estimation strategy is used and, as some parameters are not identifiable from the data, we explore the use of informative priors that are consistent with reasonable assumptions in the surrogate marker setting to aid in estimation.

While there is some bias in the estimation of the surrogacy quantities of interest, the estimation procedure does reasonably well at distinguishing between poor and good surrogate markers.

Some of the parameters of the proposed model are not identifiable from the data, and therefore, assumptions must be made in order to aid in their estimation.

The proposed quantities can be used in combination to provide evidence about the validity of S as a surrogate marker for T.

In studies with surrogate outcomes available for all subjects and true outcomes available for only a subsample, survival analysis methods are needed that incorporate both endpoints in order to assess treatment effects.

We develop a semiparametric estimated likelihood method for the proportional hazards model with discrete time data and a binary covariate of interest. Our proposed method allows for real-time validation of surrogate outcomes and flexible censoring mechanisms.

Our proposed estimator is consistent and asymptotically normal. Through numerical studies, we showed that our proposed method for estimating a covariate effect is unbiased compared to the naïve estimator that uses only surrogate endpoints and is more efficient with moderate missingness compared to the complete-case estimator that uses only true endpoints. We further demonstrated the advantages of our proposed method in comparison with existing approaches when there is real-time validation. We also illustrated the use of our proposed method by estimating the effect of gender on time to detection of Alzheimer’s disease using data from the Alzheimer’s Disease Neuroimaging Initiative.

The proposed method is able to account for the uncertainty of surrogate outcomes using a validation subsample of true outcomes in estimating a binary covariate effect. The proposed estimator can outperform standard semiparametric survival analysis methods and can therefore save on costs of a trial or improve power in detecting treatment effects.

Choosing to replace or maintain an existing cancer staging system is a difficult task. The system plays a critical role in patient counselling and treatment decision making because the staging system conveys prognosis.

Many issues may be considered when deciding the preferred system (i.e. old or new), such as the level of evidence for one or more factors included in the system or the general opinions of expert clinicians. However, given the major objective of estimating prognosis on an ordinal scale, we argue that the rival staging system candidates should be compared on their ability to predict outcome. We sought to outline an algorithm that would compare two rival ordinal systems on their predictive ability.

We devised an algorithm based largely on the concordance index, which is appropriate for comparing two models in their ability to rank observations. We demonstrate our algorithm with a prostate cancer staging system example.

We have provided an algorithm for selecting the preferred staging system based on prognostic accuracy. It appears to be useful for the purpose of selecting between two ordinal prediction models.

Biomarkers associated with treatment-effect heterogeneity can be used to make treatment recommendations that optimize individual clinical outcomes. To accomplish this, statistical methods are needed to generate marker-based treatment-selection rules that can most effectively reduce the population burden due to disease and treatment. Compared to the standard approach of risk modeling to derive treatment-selection rules, a more robust approach is to directly minimize an unbiased estimate of total disease and treatment burden among a pre-specified class of rules. This problem is one of minimizing a weighted sum of 0–1 loss function, which is computationally challenging to solve due to the nonsmoothness of 0–1 loss. Huang and Fong, among others, proposed a method that uses the Ramp loss to approximate the 0–1 loss and solves the minimization problem through repetitive constrained optimizations. The algorithm was shown to have comparable or better performance than other comparative estimators in various settings. Our aim in this article is to further extend the algorithm to allow for variable selection in the presence of a large number of candidate markers.

We develop an alternative method to derive marker combinations to minimize the weighted sum of Ramp loss in Huang and Fong, based on data from randomized trials. The new algorithm estimates treatment-selection rules by repetitively minimizing a smooth and differentiable objective function. Through the use of an L1 penalty, we expand the method to allow for feature selection and develop an algorithm based on the coordinate descent method to build the treatment-selection rule.

Through extensive simulation studies, we compared performance of the proposed estimator to four existing approaches: (1) a logistic regression risk modeling approach, and three other “direct optimizing” approaches including (2) the estimator in Huang and Fong, (3) the weighted support vector machine, and (4) the weighted logistic regression. The proposed estimator performs comparably to that of Huang and Fong, and comparably or better than other estimators. Allowing for variable selection using the proposed estimator in the presence of a large number of markers further improves treatment-selection performance. The proposed estimator is also advantageous for selecting variables relevant to treatment selection compared to L1 penalized logistic regression and weighted logistic regression. We illustrate the application of the proposed methods in host-genetics data from an HIV vaccine trial.

The proposed estimator is appealing considering its effectiveness and conceptual simplicity. It has significant potential to contribute to the selection and combination of biomarkers for treatment selection in clinical practice.

There is currently much interest in generating more individualized estimates of treatment effects. However, traditional statistical methods are not well suited to this task. Post hoc subgroup analyses of clinical trials are fraught with methodological problems. We suggest that the alternative research paradigm of predictive analytics, widely used in many business contexts, can be adapted to help.

We compare the statistical and analytics perspectives and suggest that predictive modeling should often replace subgroup analysis. We then introduce a new approach, cadit modeling, that can be useful to identify and test individualized causal effects.

The cadit technique is particularly useful in the context of selecting from among a large number of potential predictors. We describe a new variable-selection algorithm that has been applied in conjunction with cadit. The cadit approach is illustrated through a reanalysis of data from the Randomized Aldactone Evaluation Study trial, which studied the efficacy of spironolactone in heart-failure patients. The trial was successful, but a serious adverse effect (hyperkalemia) was subsequently discovered. Our reanalysis suggests that it may be possible to predict the degree of hyperkalemia based on a logistic model and to identify a subgroup in which the effect is negligible.

Cadit modeling is a promising alternative to subgroup analyses. Cadit regression is relatively straightforward to implement, generates results that are easy to present and explain, and can mesh straightforwardly with many variable-selection algorithms.

Primary care providers often fail to identify patients who are overweight or obese or discuss weight management with them. Electronic health record-based tools may help providers with the assessment and management of overweight and obesity.

We describe the design of a trial to examine the effectiveness of electronic health record-based tools for the assessment and management of overweight and obesity among adult primary care patients, as well as the challenges we encountered.

We developed several new features within the electronic health record used by primary care practices affiliated with Brigham and Women’s Hospital in Boston, MA. These features included (1) reminders to measure height and weight, (2) an alert asking providers to add overweight or obesity to the problem list, (3) reminders with tailored management recommendations, and (4) a Weight Management screen. We then conducted a pragmatic, cluster-randomized controlled trial in 12 primary care practices.

We randomized 23 clinical teams (“clinics”) within the practices to the intervention group (n = 11) or the control group (n = 12). The new features were activated only for clinics in the intervention group. The intervention was implemented in two phases: the height and weight reminders went live on 15 December 2011 (Phase 1), and all of the other features went live on 11 June 2012 (Phase 2). Study enrollment went from December 2011 through December 2012, and follow-up ended in December 2013. The primary outcomes were 6-month and 12-month weight change among adult patients with body mass index ≥25 who had a visit at one of the primary care clinics during Phase 2. Secondary outcome measures included the proportion of patients with a recorded body mass index in the electronic health record, the proportion of patients with body mass index ≥25 who had a diagnosis of overweight or obesity on the electronic health record problem list, and the proportion of patients with body mass index ≥25 who had a follow-up appointment about their weight or were prescribed weight loss medication.

We encountered challenges in our development of an intervention within the existing structure of an electronic health record. For example, although we decided to randomize clinics within primary care practices, this decision may have introduced contamination and led to some imbalance of patient characteristics between the intervention and control practices. Using the electronic health record as the primary data source reduced the cost of the study, but not all desired data were recorded for every participant.

Despite the challenges, this study should provide valuable information about the effectiveness of electronic health record-based tools for addressing overweight and obesity in primary care.

Intermittent exotropia is the most common form of divergent strabismus (squint) in children. Evidence regarding its optimum management is limited. A pilot randomised controlled trial has recently been completed (Surgery versus Active Monitoring in Intermittent Exotropia trial) to determine the feasibility of a full randomised controlled trial.

To identify drivers for and barriers against parents’ participation in Surgery versus Active Monitoring in Intermittent Exotropia and to seek their views on information received, the need for randomisation, and enhancing acceptability.

Multiple method qualitative study using semi-structured telephone interviews to explore parents’ motivations and trial screening logs to provide an indication of common barriers. Exploratory thematic analysis identified key themes.

A total of 48 interviews were conducted (14 participants; 34 non-participants). Barriers included no desire for surgery/preference to ‘wait and see’, wanting surgery immediately, feeling uncomfortable about ‘surrendering control’ over decision-making/being managed ‘at random’, lack of confidence in the effectiveness of surgery, believing the risks outweighed the benefits, and lack of trust. Drivers included desiring surgery, ‘nothing to lose’, benefits offsetting the risks, and being in a trial would result in better care. Some also mentioned ‘doing their bit’ for research. Suggestions for enhancing acceptability included allowing choice of treatment group, giving more time for decision-making, expanding on information given, and improving communication. Many felt the necessity of randomisation was adequately explained, but there was some indication that it was misunderstood. Information extracted from the screening logs of 80/89 eligible non-participants indicated the most prevalent barrier was not wanting surgery/preferring to observe (56%), followed by desiring surgery straightaway (15%). Opposition to randomisation/wanting to retain control was recorded in 9% of cases as was the belief that the child’s squint was not severe enough to warrant surgery.

Interviews were not audio-recorded. Not all who consented to interview could be contacted, although the response/contact rate was good (48/62). A few parents did not provide reasons for refusing the trial.

Opposition to surgery and concerns about surrendering control were common obstacles to participation, whereas parents keen for their child to undergo the operation but happy to defer tended to embrace a ‘nothing to lose’ attitude. Many non-participants would have consented if allowed to choose group, although most of these would have chosen observation. While most parents felt happy with information given and that randomisation was adequately explained, it is of concern that there may be some misunderstanding, which should be addressed in any trial. These findings will inform future trials in childhood exotropia, for example, consideration of preference arms and improving communication. Lessons learnt from the Surgery versus Active Monitoring in Intermittent Exotropia trial could prove valuable to paediatric and surgical trials generally.

The field of HIV prevention research has recently experienced some mixed results in efficacy trials of pre-exposure prophylaxis, vaginal microbicides, and HIV vaccines. While there have been positive trial results in some studies, in the near term, no single method will be sufficient to quell the epidemic. Improved HIV prevention methods, choices among methods, and coverage for all at-risk populations will be needed. The emergence of partially effective prevention methods that are not uniformly available raises complex ethical and scientific questions regarding the design of ongoing prevention trials.

We present here an ethical analysis regarding inclusion of pre-exposure prophylaxis in an ongoing phase IIb vaccine efficacy trial, HVTN 505. This is the first large vaccine efficacy trial to address the issue of pre-exposure prophylaxis, and the decisions made by the protocol team were informed by extensive stakeholder consultations. The key ethical concerns are analyzed here, and the process of stakeholder engagement and decision-making described.

This discussion and analysis will be useful as current and future research teams grapple with ethical and scientific study design questions emerging with the rapidly expanding evidence base for HIV prevention.

There is some consensus among authors of reports of clinical studies that a measure of follow-up time is informative for the interpretation of the Kaplan–Meier estimate of the survivor function of the event time of interest. Previous authors have suggested that length of follow-up is important to report because the findings of a study should be extracted from the time frame in which most of the subjects have had the event or have remained under observation. This time frame is where the Kaplan–Meier estimate is most stable. This concept of stability is relative to the potential maximum information about the event time distribution contained in the sample; it is not relative to the true, population survivor function. A measure of stability is useful for the interpretation of an interim analysis in which an immature survivor function is presented. Our interest in this article lies in characterizing the unobserved, complete follow-up Kaplan–Meier estimate based on the observed, partial follow-up estimate. Our focus is not on characterizing the true event time distribution relative to its estimate. The concept of stability has not been well-defined in the literature, which has led to inconsistency and lack of transparency across trials in their attempts to capture it through a variety of measures of follow-up.

We report the results of a survey of recent literature on cancer clinical trials and summarize whether follow-up is reported and if so, whether it is well-defined. We define commonly used measures of follow-up in clinical studies.

We explain how each measure should be assessed to evaluate the stability of the Kaplan–Meier estimate for the event, and we identify relationships among measures. We propose a new measure that better conveys the desired information about the stability of the current Kaplan–Meier estimate relative to one based on complete follow-up. We apply the proposed measure to a meningioma study for illustration.

It is useful for reports of clinical studies to supplement Kaplan–Meier estimates with quantitative assessments of the stability of those estimates relative to the potential follow-up of study participants. We justify the use of one commonly used measure, and we propose a new measure that most directly accomplishes this goal.

This study aimed to (1) develop a decision aid for women considering participation in the Sentinel Node Biopsy versus Axillary Clearance 2 (SNAC-2) breast cancer surgical trial and (2) obtain evidence on its acceptability, feasibility, and potential efficacy in routine trial clinical practice via a two-stage pilot.

The decision aid was developed according to International Patient Decision Aid Standards. Study 1: an initial pilot involved 25 members of the consumer advocacy group, Breast Cancer Network Australia. Study 2: the main pilot involved 20 women eligible to participate in the SNAC-2 trial in New Zealand. In both pilots, a questionnaire assessed: information and involvement preferences, decisional conflict, SNAC-2 trial-related understanding and attitudes, psychological distress, and general decision aid feedback. A follow-up telephone interview elicited more detailed feedback on the decision aid design and content.

In both pilots, participants indicated good subjective and objective understanding of SNAC-2 trial and reported low decisional conflict and anxiety. The decision aid was found helpful when deciding about trial participation and provided additional, useful information to the standard trial information sheet.

The development and two-stage piloting process for this decision aid resulted in a resource that women found very acceptable and helpful in assisting decision-making about SNAC-2 trial participation. The process and findings provide a guide for developing other trial decision aids.

The intention-to-treat principle states that all randomised participants should be analysed in their randomised group. The implications of this principle are widely discussed in relation to the analysis, but have received limited attention in the context of handling errors that occur during the randomisation process. The aims of this article are to (1) demonstrate the potential pitfalls of attempting to correct randomisation errors and (2) provide guidance on handling common randomisation errors when they are discovered that maintains the goals of the intention-to-treat principle.

The potential pitfalls of attempting to correct randomisation errors are demonstrated and guidance on handling common errors is provided, using examples from our own experiences.

We illustrate the problems that can occur when attempts are made to correct randomisation errors and argue that documenting, rather than correcting these errors, is most consistent with the intention-to-treat principle. When a participant is randomised using incorrect baseline information, we recommend accepting the randomisation but recording the correct baseline data. If ineligible participants are inadvertently randomised, we advocate keeping them in the trial and collecting all relevant data but seeking clinical input to determine their appropriate course of management, unless they can be excluded in an objective and unbiased manner. When multiple randomisations are performed in error for the same participant, we suggest retaining the initial randomisation and either disregarding the second randomisation if only one set of data will be obtained for the participant, or retaining the second randomisation otherwise. When participants are issued the incorrect treatment at the time of randomisation, we propose documenting the treatment received and seeking clinical input regarding the ongoing treatment of the participant.

Randomisation errors are almost inevitable and should be reported in trial publications. The intention-to-treat principle is useful for guiding responses to randomisation errors when they are discovered.

Participants of early-phase intervention trials for serious conditions provide high estimates of likelihood of benefit, even when informed consent forms do not promise such benefits. However, some technically correct, negatively stated benefits statements—such as “it is not guaranteed that you will benefit”—could play a role in raising expectations of benefit because in ordinary English usage such statements denote a likely but not a certain-to-occur event.

An experimental online survey of 584 English-speaking adults recruited online. They were randomized to receive one of two benefit statements (“not guaranteed” vs “some but very small chance”), using a hypothetical scenario of an early-phase clinical trial testing an intervention to treat amyotrophic lateral sclerosis. We assessed respondents’ willingness to consider participating in the amyotrophic lateral sclerosis trial, their estimates of likelihood of benefit, and their explanations for those estimates.

The two arms did not differ in willingness to consider participation in the amyotrophic lateral sclerosis trial. Those receiving “not guaranteed” benefit statement had higher estimates of benefit than those receiving “some but very small chance” statement (35.7% (standard deviation 20.2) vs 28.3% (standard deviation 22.0), p < 0.0001). A total of 43% of all respondents chose expressions of positive sentiment (hope and need to stay positive) as explanations of their estimates; these respondents’ estimates of benefit were higher than others but similar between the two arms. The effect of benefit statements was greatest among those who chose “Those are just the facts” as the explanation for their estimate (31.0% (standard deviation 22.4%) in “not guaranteed” arm vs 18.9% (standard deviation 21.0%) in comparison arm, p = 0.008).

The use of “not guaranteed” language in benefit statements, when compared to “small but very small chance” language, appeared to increase the perception of likelihood of benefit of entering an early-phase trial, especially among those who view their estimates of benefits as “facts.” Such “no guarantee” benefit statements may be misleading and should not be used in informed consent forms.