Underpowered trials at trial start and informed consent: action is needed beyond the COVID-19 pandemic

Underpowered clinical trials: frequent and unethical

Otherwise methodologically correct but underpowered clinical trials, i.e. those aiming to recruit a limited number of participants to reasonably provide valuable estimates of treatment effects, are considered unethical. ¹ Underpowered trials cannot yield valid scientific knowledge and cannot justify exposing participants to risks or burdens that any trial entails. ¹ Although this type of trial was distressingly common 20 years ago, ² a remarkable surge has been observed during the coronavirus disease 2019 (COVID-19) pandemic. A meta-epidemiological study assessing 91 COVID-19 randomised controlled trials (RCTs) published up to October 2020 showed that the median number of participants was 84; 25% of these trials enrolled less than 50 participants. ³ An analysis by two Food and Drug Administration officials showed that among 2895 individual treatment cohorts – belonging to 2024 COVID-19 drug trials – registered on ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform up to November 2020, 95% of all cohorts of trials were underpowered. ⁴ How could this happen?

Underpowered clinical trials: how they can be useful

Research ethics committees (RECs) often identify lack of an adequate sample size calculation as undermining trials’ scientific validity. ⁵ Many investigators point out that underpowered trials are easier to complete; they justify their conduct by the fact that data from underpowered RCTs can be used in meta-analyses. ² But for this to be methodologically correct, comparable research methods must have been used among all the RCTs that are included in each meta-analysis. ² Since underpowered RCTs are prone to negative results, they are less likely to be published. This introduces publication bias in future meta-analyses based on published reports. To prevent this, results of underpowered trials should be posted on freely accessible registers.

Thus, an adequate meta-analysis requires that unpublished RCTs are included and for this to happen this must be pre-specified in the protocols of underpowered RCTs. This would entail a data-sharing agreement to conduct an individual data network meta-analysis. ⁶ When investigators were not originally organised as a network, they should try to pool the data with the same treatment by creating a minimal dataset and a central data repository for analysis ⁷ ; this can be appropriate for data even from under-enrolled single-centre RCTs. ⁸ A sensible approach in this regard is for investigators to have a database ready to pool relevant data with those of other trialists, and by including sufficient meta-data and making sure that data are interoperable. These post hoc approaches are, however, considered to be unethical, since pooling of data was not considered at the inception of these RCTs, ¹ and, therefore, could not be predicted or communicated to potential participants. Yet, this has not prevented the conduct of underpowered trials and, in some cases, their contribution to meta-analyses ⁹ that have provided useful information for clinical practice.

Information to be provided to trial participants and the role of RECs

We consider that the informed consent forms for underpowered RCTs must include information on the trial’s limited capacity of yielding scientifically valid findings. The basic ethical requirement is to inform about everything that would possibly affect participants’ decisions and clearly being able to balance risks and burdens versus the potential for important scientific results is absolutely central. If investigators know (or presume) at trial start that their research will produce no valid data and do not share this with potential participants, they are in fact cheating them. ² Non-disclosure may happen because investigators may fear that including this information will reduce recruitment. ² In other cases, investigators did not conduct a power analysis before submitting the protocol to the relevant REC. This was the case of, for example, two large adaptive RCTs assessing drugs for hospitalised COVID-19 patients, UK RECOVERY and WHO Solidarity, where scientifically and ethically correct power calculations to estimate sample sizes could not be performed at the start of the trials that aimed to enrol ‘thousands’ of participants. ¹⁰ Another correct but seldom used approach is to run underpowered trials as part of a ‘meta-trial’ in which investigators prospectively agreed to pool the data from small trials conducted under the same protocol. This was the case of a meta-trial assessing awake positioning for COVID-19 to prevent intubation and death that pooled data from five trials run in six different countries. ¹¹

When the trial protocol does not include a reason for not calculating the sample size, the REC should ask the investigators to conduct such an estimation or to explain why they do not know how to conduct a sample size calculation before resubmitting the trial protocol for their consideration. There is presently a concern that RECs could be reluctant to flag issues concerning trial design, because of an attitude that they must focus on ethics, ¹² while scientific quality must be judged by funders. Here, however, such a stance seems like the abandoning of a major ethical issue and RECs (known as institutional review boards – IRBs – in the US) should follow the US regulations, which specify that an IRB is supposed to assess scientific validity so that research participants are not subject to risk or inconvenience pointlessly. ¹³ The EU clinical trials regulation requires trials to comply with the principles of good clinical practice, ¹⁴ which establishes the need for RECs to review the adequacy of the sample size – including trial power calculations – to be included in the study protocol. ¹⁵

Conclusion

RECs should very carefully examine the sample size calculations of the trials they evaluate and be reasonably sure that the data obtained will have scientific and social value. We believe that in trials with no formal sample size calculation, participants should be informed that the results obtained may not be scientifically valid by themselves, but, if this happens, they are planned to be pooled with the results of similar trials. This will require that the investigators make the necessary arrangements for this to take place. With this approach, investigators will ensure that participants are appropriately informed, and that the trial will be useful for society. Underpowered trials whose investigators have not reached agreements with others to pool the data before submitting the protocol to the relevant REC should not be approved. Ethical and scientific standards should be respected in all circumstances, even in pandemics. ¹⁶