Editorial Comment for Bucuras et al.

T he primary objective of any research study is to answer some important question. As researchers, we are commonly limited to reporting on our own experience, but haven't we all wished for more? More patients, more centers, just plain more data with the hope that the old saying, “There is power in numbers” rings true and provides us with the answers we have been seeking. This is the foundation on which the Clinical Research Office of the Endourological Society (CROES) was created. The CROES initiative “aims to promote and support high quality international patient-centered research in a transparent way and to facilitate the implementation of research projects by creating a global network.” ¹ The CROES mission relies on worldwide collaboration and its stated vision is clear: “To apply rigorous scientific evaluation to the field of endourology with the intent to bring patients globally the most effective and efficient care possible.” The CROES Percutaneous Nephrolithotomy (PCNL) study was the first to test the feasibility of a project this ambitious. It is as a part of this larger PCNL study that this report on patients with solitary kidneys was conducted.

While there is no question that the efforts of the CROES PCNL study group should be applauded, there are inherent challenges to a project of this magnitude that can impact the conclusions gathered from the data. I think we can categorize these strengths and limitations into those that relate to study size and those that result from study design. CROES data are collected in a prospective manner through a remote data management system (DMS). The CROES PCNL study collects data from 96 centers in 26 countries and includes 5803 patients. The power of a study this size is immediately evident when you consider the clinical entity described in this article. Solitary kidney is uncommon enough that single institutions would be limited to small numbers of patients and be hard-pressed to draw any significant conclusions, but the CROES PCNL Global study includes 189 patients worldwide. In fact, this article suggests that outcomes related to morbidity and stone-free rate were less favorable for the solitary kidneys. The logistical challenge of collecting data from nearly 100 separate centers, however, makes it such that CROES is reliant on self-reporting and data entry into the DMS. This type of self-reported data can impact the quality of the data gathered by introducing the potential for missing key data points, unreliable data, infrequent data entry, etc. Furthermore, some countries have stricter Institutional Review Board (IRB) regulations than others; therefore, the ability to include all cases in a prospective manner for 1 year will be different in each center, because the centers that require IRB approval and patient consent for inclusion will be unlikely to have every patient consent to participate. Items that are even more difficult to control include the heterogeneity of the surgical selection, the approach, and the follow-up of patients across centers. For example, true stone-free rates are difficult to determine because ultrasonography, CT, and other forms of radiography were all used based on the availability or local practice, and recording of operative times was not standardized.

As reviewers, we are asked to be critical in our assessment of manuscripts submitted, and any author can attest to the unavoidable request from the reviewers for some data point that “should” have been collected. Within the CROES database, the desire to collect important data points but make the data entry manageable such that centers would not only agree to participate, but be able to commit the resources to do so, makes it inevitable that some desired data points are missing. For example, stone analysis was not collected in the database for the CROES PCNL study, a data point that has considerable influence on clinical decision making, complications, outcomes, and recurrence rates. In this article focusing on solitary kidney, the data collected in the database did not allow the cause of solitary kidney to be elucidated.

While there are inherent limitations to the data collected, this does not discount the ability of this large multicenter real-life database initiative to provide exceptionally important clinical data. As mentioned, the CROES PCNL study was really the pilot study to see if a project of this size was even possible. I think the CROES researchers should be commended for the self-assessment of their initial efforts and for taking steps to continuously improve the quality and reliability of the data collected. One excellent example of this is the creation of the CROES Audit Committee. This committee was established “to conduct clinical audits within the individual studies to ensure that data collected by CROES are of high clinical standards, reliable, and validated within the possibilities of the audit process.” ² The Audit Committee conducts an assessment of each center and gives it a quality score (QS) that is based on number of missing data points, total number of patients included by the center, rate of inclusion, missing or not available data, and number of key data deemed as outliers. The QS helps determine which centers should be audited, and centers are also selected at random to undergo an audit. The goal of this audit process is, of course, to ensure that the data contained in the final database for each study is the best quality that is available.

The CROES Global Study group is a fine example of the power of a worldwide research collaborative. While there are challenges to a project of this scale, the CROES researchers have shown commitment to ensuring the quality of the data collected. We live in an era of globalization, and this is undoubtedly a model for how we will explore important clinical problems in the future.