Centre for Statistical and Methodological Excellence (CESAME): A Consortium Initiative for Improving Methodology in Randomised Clinical Trials

Abstract

When conducting randomised clinical trials, the choice of methodology and statistical analyses will influence the results. If the planned methodology is not of optimal quality and predefined in detail, there is a risk of biased trial results and interpretation. Even though clinical trial methodology is already at a very high standard, there are many trials that deliver biased results due to the implementation of inadequate methodology, poor data quality and erroneous or biased analyses. To increase the internal and external validity of randomised clinical trial results, several international institutions within clinical intervention research have formed The Centre for Statistical and Methodological Excellence (CESAME). Based on international consensus, the CESAME initiative will develop recommendations for the proper methodological planning, conduct and analysis of clinical intervention research. CESAME aims to increase the validity of randomised clinical trial results which will ultimately benefit patients worldwide across medical specialities. The work of CESAME will be performed within 3 closely interconnected pillars: (1) planning randomised clinical trials; (2) conducting randomised clinical trials; and (3) analysing randomised clinical trials.

Keywords

Randomised clinical trials methodology biostatistics centre of excellence consortium

Introduction

During the last decades, there has been an increased focus on transparency in randomised clinical trials and medical research.^1,2 This positive movement was enhanced and made accessible by international collaborations and guidelines, such as the CONSORT statement and the EQUATOR network.^3,4 These international consensus standards for reporting guidelines of research studies, including randomised clinical trials, have improved the quality of reporting and enhanced the ability to assess the risk of bias in medical research. Several other guidelines related to planning, conducting and analysing clinical trials currently exist.⁵ Cochrane has also published reviews evaluating methodology and statistics for clinical trials.^6,7 However, several specific methodological aspects of conducting randomised clinical trials are still not optimised and systematised.

For decades, targeted temperature management was the choice of treatment worldwide to prevent brain injury and death in patients after out-of-hospital cardiac arrest.^8,9 Targeted temperature management was included in international guidelines based on the results of 2 small trials; however, both trials had methodological limitations.^8,10,11 Years later, 2 major randomised trials conducted with high methodological quality showed that targeted temperature management did not result in any beneficial effects on survival or neurological outcomes,^12,13 and now clinical practice is changing.¹⁴ This example is just one of many demonstrating that evidence based on just randomising and comparing groups is not sufficient–trials need to be conducted with high methodological quality in all phases to provide reliable results. Unfortunately, most clinical research is not helpful or even misleading due to poor methodological quality, poor data quality and erroneous or biased analyses.^15-20 There is an urgent need for trials of higher methodological quality to improve clinical care of patients worldwide.

Although randomised clinical trials have improved treatment of patients worldwide, such trials may still have critical limitations caused by inadequate use and type of methodology, poor data quality and insufficient or biased analyses. There are several decisions to make when planning, conducting and analysing randomised clinical trials. These choices of methodology and statistics will influence the results. If the planned methodology is not predefined in detail, there is a risk of data-driven biased trial results. Therefore, conducting a randomised clinical trial requires strong methodological expertise and transparency, and poorly conducted trials may lead to biased results and wrong conclusions, compromising patients’ well-being.

Clinical trials may be affected by different errors, and we have chosen to categorise them as systematic errors (‘bias’), random errors (‘play of chance’) and trial design and conduct errors.²¹

Systematic errors

Systematic errors may be described as biases disproportioning weight in favour of or against one intervention over another.^21-23 In clinical research, systematic errors may cause investigators to draw erroneous conclusions about beneficial and harmful effects of an experimental intervention, damaging the validity of research results and therefore, they must be avoided.^21,22,24 Systematic errors can be reduced by addressing sources of bias in trials, such as selection, performance, detection bias and attrition bias.²⁵

Random errors

Random errors may be described as errors caused by the ‘play of chance’.²⁶ Random errors have an impact on the results reliability. Since they are inherent in all research requiring an inference process, they cannot be eliminated.^21,26 However, they may among other things be reduced by limiting the number of outcome comparisons and by using adequate sample sizes with sufficient power to estimate intervention effects in randomised clinical trials.^21,26

Trial design and conduct errors

Trial design and conduct errors may be described as all types of errors related to the design and conduct of trials. These errors can be inappropriate methodological choices that may lead to overestimating beneficial effects and underestimating harmful effects.²¹ There are several types of design and conduct errors including inappropriate use of surrogate outcomes without a demonstration of surrogacy, inappropriate use of composite outcomes, the choice of suboptimal comparator or a non-relevant clinical question to investigate.^21,27 As a consequence of these errors, the trials results may be hard to interpret clinically and without relevance to the patients.²¹

A common example of inadequate methodology is how most statistical methods require validation of underlying statistical assumptions to ensure the validity of results of randomised clinical trials and under some circumstances to optimise statistical power.²⁸ Our group has previously conducted a review of randomised clinical trials published in major medical journals to investigate whether the underlying statistical assumptions were predefined, assessed and reported.²⁸ Based on the identified trials, we concluded that trialists rarely report if or how underlying statistical assumptions are validated.²⁸ We also discovered that there were no clear recommendations or guidelines for this issue. We therefore formed a group of experienced statisticians and trialists and reached a consensus on detailed recommendations for how to assess the underlying statistical assumptions.²⁹

Evidence-based medicine has gained interest over the past decades with its integration of the best available evidence, clinical expertise, clinical practice and values of the patients.^30,31 However, poor methodology is a serious concern for medical research resulting in inadequate treatments and waste of resources.^16-20 Poor methodology may be especially pronounced in academic trials.³² Several of these methodological issues are caused by the lack of international recommendations for planning, conducting and analysing randomised clinical trials. We believe that developing recommendations for statistical and methodological issues based on international consensus will ultimately increase the validity of randomised clinical trial results.

This article describes the aim of The Centre for Statistical and Methodological Excellence (CESAME) focusing on developing and optimising statistical and methodological issues within randomised clinical trials. CESAME will develop and communicate statistical and methodological guidelines for specific issues to minimise the risks of errors in randomised clinical trials. This will ultimately benefit patients worldwide. The CESAME approach and this article are partially inspired by the STRATOS initiative.³³

The Centre for Statistical and Methodological Excellence

CESAME aims to facilitate the premise that randomised clinical trials should adhere to the highest methodological and statistical standards for the benefit of patients. The primary focus is to heighten the methodological quality of randomised clinical trials. The work of CESAME will be within 3 closely interconnected pillars (please see Figure 1):

Planning randomised clinical trials

Conducting randomised clinical trials

Analysing randomised clinical trials

Figure 1.

The 3 CESAME pillars.

As the processes in the 3 pillars are strongly linked, we plan to include all pillars in most future projects. Implementation will be a part of all 3 pillars and include publishing new recommendations and guidelines, developing training tools when appropriate and making coding publicly available for transparency.

Why the centre?

CESAME is based on an international consortium. We expect that the international collaboration will produce better generalisability of findings and a more precise understanding of relevant issues. An international group of experts will guarantee projects of high quality, and recommendations will be based on consensus reached by all relevant members of the consortium. A large collaboration may also increase the impact and probability of changing research traditions worldwide.

The CESAME approach

Projects within CESAME will follow the same approach to ensure pertinent results. This approach includes the following steps:

I. to conduct systematic reviews to create an overview of the current knowledge within the specific topic.

II. to conduct new relevant studies, if necessary. New studies may be both simulation studies and empirical studies designed specifically for the research topic in question.

III. to develop recommendations and practical instructions based on the new knowledge gained from the previous steps.

All CESAME projects will include at least one trialist, methodologist and biostatistician to ensure a diverse author group with different competencies. The protocols and statistical analysis plans for all studies and systematic reviews will be predefined in detail and made publicly available (registered or published) before initiation. All source codes will be made publicly available (eg, at GitHub, San Francisco, CA, USA) for the immediate benefit of researchers worldwide.

The CESAME guidance documents will primarily be written for experienced trialists, data managers and data analysts.³³ Based on these documents, further congruent guidance documents may be developed for researchers with less experience as well as for experts within the specific fields.³³

There are several relevant issues that CESAME will try to improve and standardise. Please see Table 1 for 4 specific examples.

Table 1.

Examples of CESAME projects.

Pillar 1 (Planning RCTs):
Quantifying anticipated intervention effects in randomised clinical trials:
Sample size and power estimations are pivotal elements when planning and designing randomised clinical trials. To calculate a sample size, it is necessary to quantify an anticipated intervention effect. Large, anticipated intervention effects lead to relatively small sample sizes and small anticipated intervention effects lead to relatively large sample sizes. These anticipated intervention effects may be estimated based on previous trial results, that is, realistic intervention effects. We hypothesise that trials published in major medical journals show similar effect sizes across medical specialities, and it may be possible to present general recommendations on how to quantify anticipated intervention effects in randomised clinical trials. To confirm or reject our hypothesis, we plan to systematically assess and quantify previous intervention effects in published randomised clinical trials.

Pillar 1 (Planning RCTs):
The sequential trial design versus the adaptive trial design
The sequential trial design has for decades been regarded at the top of the hierarchy of evidence. However, the adaptive trial design is becoming frequently used. Adaptive trials have the advantage of adapting the necessary sample size per the results at interim analyses. The changes of adaptive trials include continuous reestimation of sample size, allocation ratio, and addition of new interventions to a running trial (platform trials). These new innovative designs and the results thereof still need further investigation and validation. The aim of this project is to compare findings of adaptive trials and sequential trials through systematic reviews and simulation studies of already collected data, and, if necessary, new simulation studies will be conducted. This project will provide insight into the advantages and disadvantages of each trial design and provide recommendations as to which design is optimal when planning a specific trial.

Pillar 2 (Conducting RCTs):
Central data monitoring
Data monitoring of clinical trials is a tool aiming to reduce the risks of random errors (eg, clerical errors) and systematic errors, which include misinterpretation, misunderstandings, and fabrication.³⁴ Traditional ‘good clinical practice data monitoring’ with on-site monitors checking collected data against source data is often not sufficient to ensure optimal data quality. It also increases trial costs and is time-consuming for the local investigators. The present central data monitoring project aims to optimise data monitoring during the conduct of randomised trials by automation of central monitoring of data completeness and data quality. An electronic data capture system will be developed and made publicly available to facilitate audit trail, data separation between sites, and a range of data validation checks.

Pillar 3 (Analysing RCTs):
Adjusting for centre in multicentre randomised trials
Multicentre randomised clinical trials are particularly prone to problems with analysis and reporting. It is generally recommended to stratify the randomisation by centre and to perform a stratified analysis by adjusting for centre in the analysis. However, even in trials published in high-impact journals, adjustment for centre is sub-optimally performed, which may lead to unnecessary loss of power and subsequently increased risk of type II error. Several methods are proposed for adjusting for centre (eg, fixed-effect analysis, mixed-effects analysis, and generalised estimating equations), but different methods might lead to different conclusions. There is currently no international consensus on when to use which method. This research project will focus on the influence of different methodologies for centre-adjustments and aim to identify the optimal methodology without unnecessary loss of power.

International collaboration

The CESAME initiative is taken by an international group of experienced trialists, methodologists and biostatisticians from the Copenhagen Trial Unit, Centre for Clinical Intervention Research, Department of Regional Health Research at University of Southern Denmark, the Biostatistical Department at University of Copenhagen and Centre for Anaesthesiological Research at Zealand University Hospital (Denmark), Istituto di Ricerche Farmacologiche Mario Negri (Italy), Lund University (Sweden), Department of Data Driven Medicine, Medical Center at University of Freiburg (Germany), McMaster University (Canada), ECRIN (France) and George Institute (Australia). Members of this group have previously collaborated in various projects and made important contributions to the statistical and methodological aspects of conducting randomised clinical trials. This includes developing recommendations for assessing assumptions of statistical analyses,²⁹ recommendations regarding statistical analyses for different types of outcomes,³⁵ practical recommendations for handling missing data³⁶ and developing Trial Sequential Analysis.³⁷ The latter can control risks of type I and type II errors in frequentist statistical analyses of randomised clinical trials and systematic reviews of such trials.³⁷ The international collaboration within the CESAME initiative will boost the quality of all methodological, operational and statistical aspects of randomised clinical trials (Figure 2).

Figure 2.

The CESAME organisational structure.

Organisational structure

An executive board will govern the CESAME initiative. Each institution in CESAME will appoint one person to be part of the executive board and one person as an alternate. The executive board will focus on setting priorities and providing broad oversight of CESAME, and the board will have regular virtual meetings. The meetings will also encourage further collaboration on new projects within the group. A coordinating committee will support the executive board by providing administrative services and coordinate activities for CESAME. The coordinating committee will be established at the Copenhagen Trial Unit, but other institutions will take part in the work on a rotational basis. The coordinating committee will also organise funding applications.

Conclusion

In this article, we have shortly presented the CESAME initiative, a collaborating centre focusing on the improvement of the statistical and methodological quality of randomised clinical trials. We have defined 3 linked pillars that encompass future projects within CESAME: (1) planning randomised clinical trials; (2) conducting randomised clinical trials; and (3) analysing randomised clinical trials. CESAME encourages international collaboration between its participating institutions to develop and implement statistical and methodological guidelines for randomised clinical trials.

There are presently highly influential and widely accepted guidelines for designing and reporting randomised clinical trials, that is, the SPIRIT and CONSORT guidelines.^4,38 Our intentions are not to replace or replicate these. Instead, we aim to fill the methodological gaps found in the planning, conducting and analysing of randomised clinical trials. There is a need for a comprehensive and detailed series of guidance documents dealing with the variety of topics relevant to these essential processes, written and endorsed by panels of renowned specialists and researchers, and developed through a transparent and integrated approach.

The ambition of the CESAME initiative is to increase the validity of trial results and ultimately benefit patients worldwide.

Footnotes

Acknowledgements

None.

Funding:

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Declaration Of Conflicting Interests:

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Author Contributions

CKJ and JCJ wrote the first draft. All authors commented on the manuscript and approved the final version.

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

ORCID iD

Caroline Kamp Jørgensen

Availability of Data and Materials

Not applicable.

References

Skoog

Saarimäki

Gluud

, et al. Transparency and Registration in Clinical Research in the Nordic Countries. Nordic Trial Alliance, NordForsk; 2015:1-108. Accessed May 15, 2022. https://www.nordictrialalliance.org/wp-content/uploads/2021/09/nta_transparency_report.pdf

AllTrials.net. Sense about Science. Updated 2014. Accessed May 15, 2022. https://www.alltrials.net/

EQUATOR Network. Enhancing the QUAlity and Transparency Of health Research. 2022. Accessed May 15, 2022. https://www.equator-network.org/

Schulz

Altman

Moher

; CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010;152(11):726-732.

European Medicines Agency. Biostatistics. 2022. Accessed 23 May, 2022. https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-guidelines/clinical-efficacy-safety/biostatistics

Akl

Oxman

Herrin

, et al. Using alternative statistical formats for presenting risks and risk reductions. Cochrane Database Syst Rev. 2011;2011(3):CD006776.

Hopewell

Loudon

Clarke

Oxman

Dickersin

. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009;2009(1):MR000006.

Nolan

Morley

Vanden Hoek

, et al. Therapeutic hypothermia after cardiac arrest: an advisory statement by the advanced life support task force of the International Liaison Committee on Resuscitation. Circulation. 2003;108(1):118-121.

Nolan

Sandroni

Böttiger

, et al. European Resuscitation Council and European Society of Intensive Care Medicine Guidelines 2021: post-resuscitation care. Resuscitation. 2021;161:220-269.

10.

Hypothermia after Cardiac Arrest Study Group. Mild therapeutic hypothermia to improve the neurologic outcome after cardiac arrest. N Engl J Med. 2002;346(8):549-556.

11.

Bernard

Gray

Buist

, et al. Treatment of comatose survivors of out-of-hospital cardiac arrest with induced hypothermia. N Engl J Med. 2002;346(8):557-563.

12.

Dankiewicz

Cronberg

Lilja

, et al. Hypothermia versus normothermia after out-of-hospital cardiac arrest. N Engl J Med. 2021;384:2283-2294.

13.

Nielsen

Wetterslev

Cronberg

, et al. Targeted temperature management at 33°C versus 36°C after cardiac arrest. N Engl J Med. 2013;369(23):2197-2206.

14.

Sandroni

Nolan

Andersen

, et al. ERC-ESICM guidelines on temperature control after cardiac arrest in adults. Intensive Care Med. 2022;48(3):261-269.

15.

Ioannidis

JPA

. Why most clinical research is not useful. PLoS Med. 2016;13(6):e1002049.

16.

Chalmers

Bracken

Djulbegovic

, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383:156-165.

17.

Chan

A-W

Song

Vickers

, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383:257-266.

18.

Glasziou

Altman

Bossuyt

, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383:267-276.

19.

Ioannidis

Greenland

Hlatky

, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166-175.

20.

Al-Shahi Salman

Beller

Kagan

, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383(9912):176-185.

21.

Garattini

Jakobsen

Wetterslev

, et al. Evidence-based clinical practice: overview of threats to the validity of evidence and how to minimise them. Eur J Intern Med. 2016;32:13-21.

22.

Gluud

. Bias in clinical intervention research. Am J Epidemiol. 2006;163(6):493-501.

23.

Tripepi

Jager

Dekker

Zoccali

. Selection bias and information bias in clinical research. Nephron Clin Pract. 2010;115:c94-c99.

24.

Smith

Noble

. Bias in research: Table 1. Evid Based Nurs. 2014;17:100-101.

25.

Spieth

Kubasch

Penzlin

Illigens

Barlinn

Siepmann

. Randomized controlled trials - a matter of design. Neuropsychiatr Dis Treat. 2016;12:1341-1349.

26.

Barraza

Arancibia

Madrid

Papuzinski

. General concepts in biostatistics and clinical epidemiology: Random error and systematic error. Medwave. 2019;19(7):e7687.

27.

Hilal

Sonbol

Prasad

. Analysis of control arm quality in randomized clinical trials leading to anticancer drug approval by the US Food and Drug Administration. JAMA Oncol. 2019;5(6):887-892.

28.

Nielsen

Nørskov

Lange

, et al. Assessing assumptions for statistical analyses in randomised clinical trials. J Evid Based Med. 2019;24(5):185-189.

29.

Nørskov

Lange

Nielsen

, et al. Assessment of assumptions of statistical analysis methods in randomised clinical trials: the what and how. J Evid Based Med. 2021;26(3):121-126.

30.

Masic

Miokovic

Muhamedagic

. Evidence based medicine - new approaches and challenges. Acta Inform Med. 2008;16:219-225.

31.

Sackett

Rosenberg

WMC

Gray

JAM

Haynes

Richardson

. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71-72.

32.

Laterre

François

. Strengths and limitations of industry vs. Academic randomized controlled trials. Clin Microbiol Infect. 2015;21(10):906-909.

33.

Sauerbrei

Abrahamowicz

Altman

le Cessie

Carpenter

initiative

. STRengthening analytical thinking for observational studies: the STRATOS initiative. Stat Med. 2014;33(30):5413-5432.

34.

Olsen

Hansen

Safi

Jakobsen

Greisen

Gluud

. Central data monitoring in the multicentre randomised SafeBoosC-III trial – a pragmatic approach. BMC Med Res Methodol. 2021;21(1):160.

35.

Jakobsen

Tamborrino

Winkel

, et al. Count data analysis in randomised clinical trials. J Biom Biostat. 2015;6(1):2.

36.

Jakobsen

Gluud

Wetterslev

Winkel

. When and how should multiple imputation be used for handling missing data in randomised clinical trials - a practical guide with flowcharts. BMC Med Res Methodol. 2017;17(1):162.

37.

Copenhagen Trial Unit. Trial sequential analysis. 2022. Accessed May 1, 2022. https://ctu.dk/tsa/

38.

Chan

Tetzlaff

Altman

, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200-207.