Statistical Fragility in Minimally Invasive Colorectal Surgery Studies: A Review of Randomized Trials

Abstract

Purpose:

The P value has been used as a statistical tool in randomized controlled trials (RCTs) to establish significance but does not provide information on the robustness of a study when used alone. The fragility index (FI) provides a supplemental approach for demonstrating robustness in RCTs that report dichotomous outcomes. This study aims to determine the statistical fragility of RCTs that compare minimally invasive techniques with open techniques in managing benign and malignant colorectal diseases.

Methods:

Dichotomous outcomes of minimally invasive surgery versus open surgery in RCTs from 2000 to 2023 were assessed. The overall FI and fragility quotient (FQ) of each study were calculated.

Results:

Of the 1377 screened studies, 50 met the inclusion criteria. In total, 820 outcomes were recorded with 747 outcomes reported as not significant (P ≥ .05) and 73 as significant (P < .05). The overall FI for all studies including all outcomes was 5 (interquartile range [IQR] 4–7) with a FQ of 0.031 (IQR 0.014–0.062). Of the 50 RCTs, 6 (12%) reported a loss to follow-up that was greater than the overall FI of 5.

Conclusions:

As RCTs are judged increasingly beyond just the P value, practicing colorectal surgeons will benefit from using and interpreting the FI, FQ, and the P value of studies both in analyzing future RCTs and in determining whether or not to make a change in their clinical practice if there is an efficiently true discovery.

Introduction

The management of diseases involving the colon and rectum has evolved over the years and many of these diseases are definitively treated with surgery. To address this category of colorectal diseases, methods of management are moving toward minimally invasive techniques including laparoscopy and robotic surgery. Despite the growing use of these modalities, several techniques have been described in the surgical literature. For example, methods such as side-to-end versus end-to-end techniques are used for colorectal anastomosis¹ and isoperistaltic versus antiperistaltic ileocolic anastomosis.² The decision to choose one surgical approach over another is often unclear. To investigate which surgical approach produces superior outcomes when managing surgical diseases, randomized controlled trials (RCTs) are considered the gold standard of research evidence. Replicability and reproducibility are what physicians count on when referring to RCT evidence and statistics to their patients.³ However, it has recently come to attention that attempts to replicate randomized control studies result in contradictory evidence.⁴ This begs the question of the applicability of the results from RCT studies. The reason for disparate evidence has been attributed to several causes such as biases in the reporting of data, problems in study design, and reliance on a P value to determine statistical significance.⁵ The latter has been considered a major contributor to irreproducibility.

The P value has been used as a statistical tool in RCTs to establish significance. Although the value can vary, the P value is often set to an alpha value of .05 implying that the collected data is significant if there is a 5% or less probability that the difference observed is owing to chance. This threshold of .05 is arbitrarily used to reject or accept the null hypothesis. However, this is misleading because the P value does not provide information on the strength or robustness of a study when used alone.⁶ To address this shortcoming, the fragility index (FI) was proposed by researcher and epidemiologist Alvan Feinstein in 1990 and was first implemented by Walsh et al. in 2014.^7,8 The FI provides a supplemental approach to demonstrating fragility, or in converse, the stability of the value of a P value in a RCT reporting dichotomous outcomes. In other words, the FI is the minimum number of patients required to switch from an event to a nonevent to deem the P value no longer significant. The fragility quotient (FQ) divides the FI by the sample size to provide a standardized measure of the fragility of the study. Therefore, the FQ standardizes the fragility to the sample size of the study to be compared across studies. An FI analysis with a large value indicates that the study is robust, can withstand many changes in patient outcomes, and remains significant. However, a study is fragile if the FI is small indicating that a few changes in the outcomes can result in a loss of statistical significance. For instance, if a study has an FI analysis of 2, this would mean that two patients from the experimental group would need to change their outcome from an event to a nonevent for the outcome to no longer be significant. Therefore, the larger the FI the more robust the study. Similarly, when comparing FQ, the larger the value the more robust the study and the smaller the value, the less robust the study. As it stands right now, there is no current cutoff value for an FI or an FQ to definitively define a study as robust or fragile. Rather, on the continuum the higher the FI/FQ value the better.

With the increased use of minimally invasive techniques in the management of colorectal cancers and disease, it is vital to understand dichotomous outcomes in the literature before deciding to implement a new technique. To be specific, minimally invasive techniques include the use of laparoscopic or robotic approaches in the management of both benign and malignant colorectal disease processes. The purpose of this study is to determine the statistical fragility of RCTs that compare minimally invasive techniques versus open techniques in the management of colorectal cancers and diseases.

Methods

This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Fig. 1). The PubMed database was queried from January 1, 2000 to March 15, 2023, for all RCTs relating to minimally invasive surgeries for colorectal benign and malignant diseases. To be included for the analysis, RCTs needed to report dichotomous outcomes with associated P values. The included studies were from a variety of journals listed in Table 1. Each study was manually evaluated to determine if they met inclusion criteria for studies comparing minimally invasive surgical techniques to open techniques for both benign and malignant diseases of the colon and rectum. Studies were subsequently excluded if they were non-RCT studies, not a surgical intervention, post hoc analysis RCTs, animal model studies, cadaveric studies, studies with anything other than 1:1 randomization, and studies reporting non-dichotomous outcomes. For each included study, the following data points were extracted, the journal name, study design, authors, publication year, PubMed Identifier, loss to follow-up (LTF), dichotomous outcomes (specified as primary or secondary), and associated P values for each outcome, if provided.

FIG. 1.

PRISMA diagram of included studies. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Table 1.

Overall Fragility Results

Characteristic	Outcomes	Fragility index (IQR)	Fragility quotient (IQR)
All trials	820	5 (4–7)	0.031 (0.014–0.062)
Outcomes
Primary	278	5 (4–7)	0.035 (0.020–0.065)
Secondary	542	5 (4–7)	0.029 (0.013–0.061)
Journals (number of articles)
Annals of Internal Medicine (1)	15	6 (3–7)	0.072 (0.036–0.084)
Annals Surgery (5)	89	5 (4–7)	0.041 (0.024–0.067)
Archives of Surgery (1)	7	1 (1–4)	0.021 (0.021–0.083)
British Journal of Surgery (7)	177	5 (3–8)	0.028 (0.013–0.0425)
Colorectal Disease (4)	42	5 (4–6)	0.114 (0.091–0.120)
Disease Colon Rectum (2)	34	5 (4–6)	0.053 (0.042–0.063)
European Journal of Pediatric Surgery (1)	15	6 (2–8)	0.030 (0.010–0.039)
International Journal of Colorectal Disease (1)	18	5 (4–6)	0.034 (0.028–0.041)
International Journal of Surgery (2)	27	5 (3–6)	0.077 (0.053–0.092)
Laparoendoscopic & Advanced Surgical Techniques (2)	13	6 (5–7.5)	0.020 (0.015–0.046)
Journal of Surgical Oncology (3)	36	5 (4–7)	0.014 (0.012–0.022)
JAMA (4)	109	5 (4–9)	0.019 (0.011–0.041)
JAMA Surgery (1)	29	5 (3–9)	0.060 (0.032–0.085)
Lancet (1)	13	4 (2–5.5)	0.018 (0.014–0.023)
Lancet Gastroenterology and Hepatology (2)	27	7 (5.5–10)	0.008 (0.005–0.010)
Lancet Oncology (4)	69	7 (4–12)	0.008 (0.004–0.015)
Surgical Endoscopy (6)	69	6 (5–7)	0.049 (0.038–0.067)
Surgical Laparoscopy Endoscopy & Percutaneous Techniques (1)	11	6 (5–6.5)	0.038 (0.031–0.041)
Surgery Today (1)	14	5 (5–6)	0.083 (0.083–0.100)
World Journal of Surgery (1)	6	5 (4–5)	0.128 (0.103–0.128)
Reported P value
P > .05	747	5 (4–7)	0.034 (0.015–0.065)
P < .05	73	2 (1–5)	0.020 (0.007–0.030)
Lost to follow-up
FI < LTF	77	6 (3–8)	0.023 (0.01–0.042)
FI > LTF	743	5 (4–7)	0.032 (0.015–0.063)
Year of publication
2017–2023	449	5 (4–8)	0.036 (0.014–0.065)
2009–2016	273	5 (4–7)	0.028 (0.014–0.062)
2000–2008	98	5 (4–7)	0.023 (0.015–0.042)

IQR, interquartile range; LTF, loss to follow-up.

The obtained outcomes were labeled as significant (P < .05) or nonsignificant (P ≥ .05). The FI calculation for each outcome was achieved using a two-by-two contingency table that included the dichotomous outcomes from each trial and calculated as previously reported by Walsh et al.⁸ Each outcome event was manipulated until the reversal of significance was achieved. The number of manipulations required to change the P value from significant to nonsignificant or vice versa was determined to be the FI for each outcome. An example of this process can be found in Table 2. The median FI as well as the interquartile range for all outcomes within a study were reported as the overall FI for that study. To standardize the FI value, the FI was divided by the associated total sample size to give the FQ for each study. In addition, the FI and the FQ along with their interquartile ranges were stratified for outcome type (primary versus secondary), reported significance (P < .05 vs P ≥ .05), and year (Table 1). Finally, the overall FI and FQ were determined with the incorporation of all outcomes.

Table 2.

Demonstration of Reversal of Significance with a Fragility Index (FI) = 1

	Outcome A	Outcome B	P value
Scenario 1			.031
Treatment A	15	5
Treatment B	75	85
Scenario 2			.061
Treatment A	14	6
Treatment B	75	85

Each study was evaluated and a Cochrane risk bias of assessment was also performed for each of the individual studies (Table 3). Seven items were used to assess bias risk: random sequence generation (selection bias), allocation concealment (selection bias), blinding of participants and personnel (performance bias), blinding of outcome assessment (detection bias), complete outcome data (attrition bias), selective reporting (reporting bias), and other bias. A series of Cochrane signaling questions were applied to each article and a score was provided via the Cochrane algorithm, with each category scored as having a risk of bias that was low, high, or unclear.

Table 3.

Cochrane Risk of Bias Assessment

Authors	Random sequence generation (selection bias)	Allocation concealment (selection bias)	Blinding of participants and personnel (performance bias)	Blinding of outcome assessment (detection bias)	Incomplete outcome data (attrition bias)	Selective reporting (reporting bias)	Other bias
Agrawal et al.⁹	Low	Low	Unclear	Low	Unclear	Low	Low
Altomare et al.¹⁰	Low	Low	Low	Low	Low	Low	Low
Angenete et al.¹¹	Low	Low	Low	Low	Low	Low	Low
Azhar et al.¹²	Low	Low	Low	Unclear	Low	Low	Low
Bollo et al.¹³	Low	Low	Unclear	Low	Low	Low	Low
Braga et al.¹⁴	Low	Low	Low	Low	Low	Low	Low
Braga et al.¹⁵	Low	Low	High	Unclear	Low	Low	Low
Chen et al.¹⁶	Low	Low	Low	Unclear	Unclear	Low	Low
Cheung et al.¹⁷	Low	Low	High	Unclear	Low	Low	Low
Colon Cancer Laparoscopic or Open Resection Study Group¹⁸	Low	Low	High	Low	Low	Low	Low
Dohrn et al.¹⁹	Low	Low	Low	Low	Low	Low	Low
Emile et al.²⁰	Unclear	Low	Low	Unclear	Low	Low	Low
Feng et al.²¹	Low	Low	Low	Low	Low	Low	Low
Feng et al.²²	Low	Low	High	Unclear	Low	Low	Low
Feng et al.²³	Low	Low	High	Unclear	Unclear	Low	Low
Feng et al.²⁴	Low	Low	Low	Low	Low	Low	Low
Ferrer-Márquez et al.²⁵	Low	Low	Low	Low	Low	Low	Low
Fleshman et al.²⁶	Low	Low	High	High	Unclear	Low	Low
Fujii et al.²⁷	Low	Low	High	Unclear	Unclear	Low	Low
Gervaz et al.²⁸	Unclear	Low	Low	Low	Low	Low	Low
Hong et al.²⁹	Unclear	Low	Low	Unclear	Low	Low	Low
Ibáñez et al.²	Unclear	Low	Low	Low	Low	Low	Low
Ishibe et al.³⁰	Low	Low	High	High	Low	Low	Low
Jayne et al.³¹	Unclear	Low	High	Low	Low	Low	Low
Kang et al.³²	Low	Low	Low	Low	Low	Low	Low
Kang et al.³³	Low	Low	High	Unclear	Low	Low	Low
Kitano et al.³⁴	Low	Low	High	Unclear	Low	Low	Low
Klarenbeek et al.³⁵	Low	Low	Low	Low	Low	Low	Low
Kohl et al.³⁶	Low	Low	High	Low	Low	Low	Low
Lacy et al.³⁷	Low	Low	High	Low	Low	Low	Low
Lascarides et al.³⁸	Low	Low	Low	Low	Low	Low	Low
Lezoche et al.³⁹	Low	Low	High	Low	Low	Low	Low
Li et al.⁴⁰	Low	Low	High	Low	Low	Low	Low
Liang et al.⁴¹	Low	Low	Low	Low	Unclear	Low	Low
Liang et al.⁴²	Low	Low	Low	Low	Low	Low	Low
Madbouly et al.⁴³	Low	Low	High	High	Low	Low	Low
Marcello et al.⁴⁴	Low	Low	High	Low	Unclear	Low	Low
Neudecker et al.⁴⁵	Low	Low	High	Unclear	Low	Low	Low
Ng et al.⁴⁶	Low	Low	High	Unclear	Unclear	Low	Low
Okkabaz et al.⁴⁷	Low	Low	Low	Low	Low	Low	Low
Planellas et al.⁴⁸	Low	Low	Low	Low	Low	Low	Low
Planellas et al.¹	Low	Low	Low	Low	Low	Low	Low
Pucciarelli et al.⁴⁹	Low	Low	Low	Low	Unclear	Low	Low
Schultz et al.⁵⁰	Low	Low	Low	Unclear	Low	Low	Low
Schultz et al.⁵¹	Low	Low	Low	Low	Low	Low	Low
Serra-Aracil et al.⁵²	Low	Low	Low	Low	Low	Low	Low
Stevenson et al.⁵³	Low	Low	Low	Unclear	Low	Low	Low
Thornell et al.⁵⁴	Low	Low	Low	Unclear	Low	Low	Low
Veldkamp et al.⁵⁵	Low	Low	High	Unclear	Low	Low	Low
Xu et al.⁵⁶	Low	Low	Low	Low	Unclear	Low	Low

Results

Of the 1377 screened studies, 50 met the inclusion criteria (Fig. 1). Characteristics of the included trials are listed in Table 1. In total, 820 outcomes were recorded with 747 of them reported as not significant (P ≥ .05) and 73 as significant (P < .05). Of the 747 nonsignificant outcomes, the median FI was 5 (interquartile range [IQR] 4–7) with an FQ of 0.034 (IQR 0.015–0.065). Of the 73 significant outcomes, the median FI was 2 (IQR 1–5) with an FQ of 0.020 (IQR 0.007–0.030). From the 820 total outcomes, 278 (33.9%) were primary and 542 (66.1%) were secondary. The median FI for both primary and secondary outcomes was the same with an FI of 5 (IQR 4–7) with an associated FQ of 0.035 (IQR 0.020–0.065) and 0.029 (IQR 0.013–0.061), respectively. Of the 50 RCTs, 6 had more patients LTF than the overall FI of 5. Therefore, 12% of studies reported an LTF value that was greater than the overall FI (Fig. 2).

FIG. 2.

Distribution of the number of patients lost to follow-up.

The overall FI for all studies including all outcomes was 5 (IQR 4–7) with an FQ of 0.031 (IQR 0.014–0.062) indicating that the reversal of 3 of 100 outcomes may change the study significance of the included RCTs. The FI stratified by year of publication identified an FI of 5 (IQR 4–7) from 2000 to 2008, an FI of 5 (IQR 4–7) from 2009 to 2016, and an FI of 5 (IQR 4–8) from 2017 to 2023 demonstrating fragility over the 23-year period (Table 1).

Discussion

This study finds that the overall median FI was 5 with an associated FQ of 0.031 for RCTs evaluating minimally invasive techniques in colorectal surgery in the past 23 years. An FI of 5 tells us that the reversal of five patients’ outcomes would be enough to change the significance of the outcome. Standardizing the studies for sample size, the FQ of 0.031 means that about 3 out of 100 patients would need to reverse their outcomes to alter the significance of each of the studies. Of the included RCTs, 12% (6) of studies presented an LTF value greater than the overall FI. This suggests that the unknown outcomes of the patients LTF may have been enough to reverse the significance of the study if their outcomes favored a reversal of significance. The remaining studies either had less than five patients LTF or no LTF reported. A low FI and FQ, in addition to higher LTF than overall FI, suggests that the data reported in the literature on RCTs for minimally invasive techniques within colorectal surgery may be fragile and not as robust as we thought.

These results coincide with previous studies done analyzing FI in other specialties.^57–63 A study by Nelms et al in 2021 looked at all colorectal surgical randomized control studies between the years of 2016 and 2018 and calculated an FI median of 3 and 57% of trials had an LTF greater than the FI.⁵⁹ Our study examining FI in minimally invasive techniques over the past 23 years is slightly higher at 5 but still overall low. Of the reported studies, a median FI has been reported as high as 8 in its first application of the FI analysis by Walsh et al.⁸ and even up to an FI of 12 in a study examining clinical practice guidelines for acute coronary syndrome.⁶¹ Our results add to a growing body of literature examining the quality of reported data in RCTs especially as it relates to the delivery of clinical care. Clear and informative objective data from RCTs provide physicians with the necessary tools required to make an informed decision about whether or not to utilize results from published studies in their day-to-day practice. Objective data that provides solely a P value of significance does not give information about the robustness of a study and whether or not it is replicable. The use of the FI and FQ gives additional information than a P value alone in that it is correlated with statistical power and sample size.⁵⁷ A high FI can mean that the P value of a trial is far from .05 and/or has a high power and conversely a low FI can mean a P value is near .05 or that the sample size is low. The latter leads to a high likelihood that the study’s findings are not replicable and likely not a true discovery. Therefore, a study with a low FI demonstrates that only a few individuals are required to change the outcome of significance and may not be a reliable study to change clinical practice. As statistical power refers to the likelihood of detecting a statistically significant effect if it indeed exists, a low power signifies a heightened risk of false negatives, implying that significant effects might remain undetected. By juxtaposing the FI with statistical power, researchers and clinicians can gauge whether statistically significant results stem from adequately powered studies or if they are precarious and susceptible to data variations. Furthermore, replication and reproducibility play pivotal roles in corroborating the credibility of research findings in clinical contexts. Fragile findings pinpointed through low FI values may necessitate replication to establish robustness. Moreover, considering factors such as heterogeneity among studies and the presence of similar studies aids in assessing the necessity for replication, highlighting the importance of a standardized approach like an FQ. The interplay between FI and statistical power, coupled with other statistical parameters, enables researchers to evaluate the resilience, validity, and reproducibility of clinical research findings. The identification of fragile findings and comprehension of study power are critical strides toward ensuring the reliability of evidence-based medicine.

At this time, there are still no accepted FI or FQ targeted numbers to define whether or not a study is robust or fragile. Therefore, it is still not certain what FI and FQ are acceptable to determine which RCTs are better used to govern clinical practice by physicians. Studies currently evaluating FI/FQ values are doing so using a composite of the current published literature; however, FI and FQ values should be used in future individual RCT studies as part of the analysis to contribute to the growing body of literature. The goal is to be able to use this study among the growing literature in the FI to help determine what the acceptable cutoff will be. Whether that cutoff should be a generalized value in the literature or stratified between specialties will also need to be determined. Nonetheless, despite the hopes of utilization, the fragility analysis comes with its own limitations. First, they can only be utilized in RCTs that have dichotomous outcomes. Another limitation is that studies with continuous variables and outcomes cannot use the fragility analysis to determine robustness or fragility and excludes these studies from further inquiry. In addition, this study does not adjudicate the quality of colorectal surgery literature but rather brings to the forefront the need to critically analyze objective data before adjusting clinical practices as well as discuss the issue of replicability and how understanding and utilizing statistics is vital in clinical practice. Over time, as more fragility analysis studies are conducted on the current published data, and hopefully its incorporation into future RCT data reporting, practicing physicians can be provided with a more well-rounded understanding of the provided data to make a more informed decision prior to changing clinical practice.

Conclusion

There are several minimally invasive techniques for alleviating diseases within colorectal surgery. Some have been adopted and others still are contested owing to published evidence in RCTs providing P values of significance. However, several studies have shown that the P value may not be enough to change clinical significance. This study reports fragility within minimally invasive colorectal surgeries showing that a median number of five patients is all that would be needed to change the significance of the current reported data. Patients’ LTFs have the potential to alter significance in up to 12% of the current literature examining RCTs for minimally invasive colorectal surgeries. The need for more than a P value is increasingly being recognized across specialties in the literature including colorectal. As RCTs are judged increasingly beyond just the P value, practicing colorectal surgeons will benefit from using and interpreting the FI, FQ, and the P value of studies both in analyzing future RCTs and in determining whether or not to make a change in their clinical practice if there is an efficiently true discovery.

Footnotes

Acknowledgments

The author expresses gratitude to Michael Megafu and Emmanuel Megafu for their insights into statistical analysis for fragility studies.

Authors’ Contributions

The author confirms sole responsibility for the following: study conception and design, data collection, analysis and interpretation of results, and article preparation.

Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

References

Anonymous. Randomized clinical trial comparing side to end vs end to end techniques for colorectal anastomosis. Int J Surg, 2020; 83:220–229.

Ibáñez

, Abrisqueta

, Luján

, et al. Isoperistaltic versus antiperistaltic ileocolic anastomosis. Does it really matter? Results from a randomised clinical trial (ISOVANTI). Surg Endosc, 2018; 33(9):2850–2857.

National Academies of Sciences, Engineering, Medicine, Policy. Replicability. In: Reproducibility and Replicability in Science. National Academies Press (US); 2019.

Ioannidis

JPA

. Contradicted and initially stronger effects in highly cited clinical research. Jama, 2005; 294(2):218–228.

Sterne

JAC

, Cox

, Smith

. Sifting the evidence—what’s wrong with significance tests? Another comment on the role of statistical methods. Bmj, 2001; 322(7280):226–231.

Samargandi

, Al-Taha

, Moran

, et al. Why the p value alone is not enough: The need for confidence intervals in plastic surgery research. Plast Reconstr Surg, 2018; 141(1):152e–162e.

Feinstein

. The unit fragility index: An additional appraisal of “statistical significance” for a contrast of two proportions. J Clin Epidemiol, 1990; 43(2):201–209.

Walsh

, Srinathan

, McAuley

, et al. The statistical significance of randomized controlled trial results is frequently fragile: A case for a Fragility Index. J Clin Epidemiol, 2014; 67(6):622–628.

Agrawal

, Sulya

, Acharya

, et al. Impact of “lateralizing” the sigmoid colostomy on port ergonomics in laparoscopic anorectoplasty for high anorectal malformations. Eur J Pediatr Surg, 2022; 32(2):177–183.

10.

Altomare

, Delrio

, Shelgyn

, et al. Transanal reinforcement of low rectal anastomosis versus protective ileostomy after total mesorectal excision for rectal cancer. Preliminary results of a randomized clinical trial. Colorectal Dis, 2021; 23(7):1814–1823.

11.

Angenete

, Thornell

, Burcharth

, et al. Laparoscopic lavage is feasible and safe for the treatment of perforated diverticulitis with purulent peritonitis: The first results from the randomized controlled trial DILALA. Ann Surg, 2016; 263(1):117–122.

12.

Azhar

, Johanssen

, Sundström

, et al. Laparoscopic lavage vs primary resection for acute perforated diverticulitis: Long-term outcomes from the Scandinavian diverticulitis (SCANDIV) randomized clinical trial. JAMA Surg, 2021; 156(2):121–127.

13.

Bollo

, Turrado

, Rabal

, et al. Randomized clinical trial of intracorporeal versus extracorporeal anastomosis in laparoscopic right colectomy (IEA trial). Br J Surg, 2020; 107(4):364–372.

14.

Braga

, Frasson

, Vignali

, et al. Open right colectomy is still effective compared to laparoscopy: Results of a randomized trial. Ann Surg, 2007; 246(6):1010–1014; discussion 1014–5.

15.

Braga

, Vignali

, Gianotti

, et al. Laparoscopic versus open colorectal surgery: A randomized trial on short-term outcome. Ann Surg, 2002; 236(6):759–766; disscussion 767.

16.

Chen

, Cao

, Huang

, et al. End-to-end intestinal anastomosis using a novel biodegradable stent for laparoscopic colonic surgery: A multicenter study. Surg Today, 2019; 49(12):1003–1012.

17.

Cheung

HYS

, Chung

, Tsang

WWC

, et al. Endolaparoscopic approach vs conventional open surgery in the treatment of obstructing left-sided colon cancer: A randomized controlled trial. Arch Surg, 2009; 144(12):1127–1132.

18.

Buunen

, Veldkamp

, et al.. Survival after laparoscopic surgery versus open surgery for colon cancer: Long-term outcome of a randomised clinical trial. Lancet Oncol, 2009; 10(1):44–52.

19.

Dohrn

, Yikilmaz

, Laursen

, et al. Intracorporeal Versus extracorporeal anastomosis in robotic right colectomy: A multicenter, triple-blind, randomized clinical trial. Ann Surg, 2022; 276(5):e294–e301.

20.

Emile

, Elbanna

, Youssef

, et al. Laparoscopic ventral mesh rectopexy vs Delorme’s operation in management of complete rectal prolapse: A prospective randomized study. Colorectal Dis, 2017; 19(1):50–57.

21.

Feng

, Yuan

, Li

, et al. Robotic versus laparoscopic surgery for middle and low rectal cancer (REAL): Short-term outcomes of a multicentre randomised controlled trial. Lancet Gastroenterol Hepatol, 2022; 7(11):991–1004.

22.

Feng

, Tang

, Zhang

, et al. Robotic versus laparoscopic abdominoperineal resections for low rectal cancer: A single-center randomized controlled trial. J Surg Oncol, 2022; 126(8):1481–1493.

23.

Feng

, Zong

, Zhao

, et al. High versus low ligation of the inferior mesenteric artery during laparoscopic rectal cancer surgery: A prospective study of surgical and oncological outcomes. J Surg Oncol, 2021; 123(Suppl 1):S76–S80.

24.

Feng

, Ling

T-L

, Lu

A-G

, et al. Completely medial versus hybrid medial approach for laparoscopic complete mesocolic excision in right hemicolon cancer. Surg Endosc, 2014; 28(2):477–483.

25.

Ferrer-Márquez

, Rubio-Gil

, Torres-Fernández

, et al. Intracorporeal versus extracorporeal anastomosis in patients undergoing laparoscopic right hemicolectomy: A multicenter randomized clinical trial (The IVEA-study). Surg Laparosc Endosc Percutan Tech, 2021; 31(4):408–413.

26.

Fleshman

, Branda

, Sargent

, et al. Effect of laparoscopic-assisted resection vs open resection of Stage II or III rectal cancer on pathologic outcomes: The ACOSOG Z6051 Randomized Clinical Trial. Jama, 2015; 314(13):1346–1355.

27.

Fujii

, Ishibe

, Ota

, et al. Long-term results of a randomized study comparing open surgery and laparoscopic surgery in elderly colorectal cancer patients (Eld Lap study). Surg Endosc, 2021; 35(10):5686–5697.

28.

Gervaz

, Mugnier-Konrad

, Morel

, et al. Laparoscopic versus open sigmoid resection for diverticulitis: Long-term results of a prospective, randomized trial. Surg Endosc, 2011; 25(10):3373–3378.

29.

Hong

H-J

, Zhao

, Yu

C-R

, et al. Comparative study of oncologic efficacy of cephalomedial to lateral dissection versus medial to lateral dissection in laparoscopic total mesorectal excision for rectal cancer: An RCT study. J Surg Oncol, 2021; 123(Suppl 1):S65–S75.

30.

Ishibe

, Ota

, Fujii

, et al. Midterm follow-up of a randomized trial of open surgery versus laparoscopic surgery in elderly patients with colorectal cancer. Surg Endosc, 2017; 31(10):3890–3897.

31.

Jayne

, Pigazzi

, Marshall

, et al. Effect of robotic-assisted vs conventional laparoscopic surgery on risk of conversion to open laparotomy among patients undergoing resection for rectal cancer: The ROLARR randomized clinical trial. Jama, 2017; 318(16):1569–1580.

32.

Kang

, Park

, Lee

, et al. Single-port laparoscopic surgery can be performed safely and appropriately for colon cancer: Short-term results of a pilot randomized controlled trial. J Laparoendosc Adv Surg Tech A, 2017; 27(5):501–509.

33.

Kang

S-B

, Park

, Jeong

S-Y

, et al. Open versus laparoscopic surgery for mid or low rectal cancer after neoadjuvant chemoradiotherapy (COREAN trial): Short-term outcomes of an open-label randomised controlled trial. Lancet Oncol, 2010; 11(7):637–645.

34.

Kitano

, Inomata

, Mizusawa

, et al. Survival outcomes following laparoscopic versus open D3 dissection for stage II or III colon cancer (JCOG0404): A phase 3, randomised controlled trial. Lancet Gastroenterol Hepatol, 2017; 2(4):261–268.

35.

Klarenbeek

, Veenhof

, Bergamaschi

, et al. Laparoscopic sigmoid resection for diverticulitis decreases major morbidity rates: A randomized control trial: Short-term results of the Sigma Trial. Ann Surg, 2009; 249(1):39–44.

36.

Kohl

, Rosenberg

, Bock

, et al. Two-year results of the randomized clinical trial DILALA comparing laparoscopic lavage with resection as treatment for perforated diverticulitis. Br J Surg, 2018; 105(9):1128–1134.

37.

Lacy

, García-Valdecasas

, Delgado

, et al. Laparoscopy-assisted colectomy versus open colectomy for treatment of non-metastatic colon cancer: A randomised trial. Lancet, 2002; 359(9325):2224–2229.

38.

Lascarides

, Buscaglia

, Denoya

, et al. Laparoscopic right colectomy vs laparoscopic-assisted colonoscopic polypectomy for endoscopically unresectable polyps: A randomized controlled trial. Colorectal Dis, 2016; 18(11):1050–1056.

39.

Lezoche

, Baldarelli

, Lezoche

, et al. Randomized clinical trial of endoluminal locoregional resection versus laparoscopic total mesorectal excision for T2 rectal cancer after neoadjuvant therapy. Br J Surg, 2012; 99(9):1211–1218.

40.

JC-M

, Leung

, Ng

SS-M

, et al. Laparoscopic-assisted versus open resection of right-sided colonic cancer–a prospective randomized controlled trial. Int J Colorectal Dis, 2012; 27(1):95–102.

41.

Liang

J-T

, Shieh

M-J

, Chen

C-N

, et al. Prospective evaluation of laparoscopy-assisted colectomy versus laparotomy with resection for management of complex polyps of the sigmoid colon. World J Surg, 2002; 26(3):377–383.

42.

Liang

, Hou

, Liu

, et al. Effectiveness and safety of laparoscopic resection versus open surgery in patients with rectal cancer: A randomized, controlled trial from China. J Laparoendosc Adv Surg Tech A, 2011; 21(5):381–385.

43.

Madbouly

, Mohii

. Laparoscopic ventral rectopexy versus stapled transanal rectal resection for treatment of obstructed defecation in the elderly: Long-term results of a prospective randomized study. Dis Colon Rectum, 2019; 62(1):47–55.

44.

Marcello

, Fleshman

, Milsom

, et al. Hand-assisted laparoscopic vs. laparoscopic colorectal surgery: A multicenter, prospective, randomized trial. Dis Colon Rectum, 2008; 51(6):818–826; discussion 826–8.

45.

Neudecker

, Klein

, Bittner

, et al. Short-term outcomes from a prospective randomized trial comparing laparoscopic and open surgery for colorectal cancer. Br J Surg, 2009; 96(12):1458–1467.

46.

LWC

, Tung

, Cheung

HYS

, et al. Hand-assisted laparoscopic versus total laparoscopic right colectomy: A randomized controlled trial. Colorectal Dis, 2012; 14(9):e612–7.

47.

Okkabaz

, Haksal

, Atici

, et al. J-pouch vs. side-to-end anastomosis after hand-assisted laparoscopic low anterior resection for rectal cancer: A prospective randomized trial on short and long term outcomes including life quality and functional results. Int J Surg, 2017; 47:4–12.

48.

Planellas

, Salvador

, Farrés

, et al. A randomized clinical trial comparing the initial vascular approach to the inferior mesenteric vein versus the inferior mesenteric artery in laparoscopic surgery of rectal cancer and sigmoid colon cancer. Surg Endosc, 2019; 33(4):1310–1318.

49.

Pucciarelli

, Del Bianco

, Pace

, et al. Multicentre randomized clinical trial of colonic J pouch or straight stapled colorectal reconstruction after low anterior resection for rectal cancer. Br J Surg, 2019; 106(9):1147–1155.

50.

Schultz

, Wallon

, Blecic

, et al. One-year results of the SCANDIV randomized clinical trial of laparoscopic lavage versus primary resection for acute perforated diverticulitis. Br J Surg, 2017; 104(10):1382–1392.

51.

Schultz

, Yaqub

, Wallon

, et al. Laparoscopic lavage vs primary resection for acute perforated diverticulitis: The SCANDIV randomized clinical trial. Jama, 2015; 314(13):1364–1375.

52.

Serra-Aracil

, Zarate

, Bargalló

, et al. Transanal versus laparoscopic total mesorectal excision for mid and low rectal cancer (Ta-LaTME study): Multicentre, randomized, open-label trial. Br J Surg, 2023; 110(2):150–158.

53.

Stevenson

ARL

, Solomon

, Lumley

, et al. Effect of laparoscopic-assisted resection vs open resection on pathological outcomes in rectal cancer: The ALaCaRT randomized clinical trial. Jama, 2015; 314(13):1356–1363.

54.

Thornell

, Angenete

, Bisgaard

, et al. Laparoscopic lavage for perforated diverticulitis with purulent peritonitis: A randomized trial. Ann Intern Med, 2016; 164(3):137–145.

55.

Veldkamp

, Kuhry

, Hop

WCJ

, et al. Laparoscopic surgery versus open surgery for colon cancer: short-term outcomes of a randomised trial. Lancet Oncol, 2005; 6(7):477–484.

56.

, Su

, He

, et al. Short-term outcomes of complete mesocolic excision versus D2 dissection in patients undergoing laparoscopic colectomy for right colon cancer (RELARC): a randomised, controlled, phase 3, superiority trial. Lancet Oncol, 2021; 22(3):391–401.

57.

Reito

, Raittio

, Helminen

. Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery. PeerJ, 2019; 7:e6813.

58.

Mian

, Megafu

, et al. The statistical fragility of the distal fibula fracture literature: A systematic review of randomized controlled trials. Injury, 2023; 54(6):1809–1814.

59.

Nelms

, Vargas

, Bedi

, et al. When the p value doesn’t cut it: The fragility index applied to randomized controlled trials in colorectal surgery. Dis Colon Rectum, 2022; 65(2):276–283.

60.

Narayan

, Gandhi

, Chrouser

, et al. The fragility of statistically significant findings from randomised controlled trials in the urological literature. BJU Int, 2018; 122(1):160–166.

61.

Gonzalez-Del-Hoyo

, Mas-Llado

, Blaya-Peña

, et al. The fragility index in randomized clinical trials supporting clinical practice guidelines for acute coronary syndrome: measuring robustness from a different perspective. Eur Heart J Acute Cardiovasc Care, 2023; 12(6):386–390.

62.

Lee

, Samarasinghe

, Javidan

, et al. The fragility of significant results from randomized controlled trials in esophageal surgeries. Esophagus, 2023; 20(2):195–204.

63.

Megafu

, Megafu

, Nguyen

, et al. The Statistical Fragility of Orbital Fractures: A Systematic Review of Randomized Controlled Trials. J Oral Maxillofac Surg, 2023; 81(6):752–758.