Abstract
Dear Editor:
We read with interest the recent article by Withington and colleagues, Assessment of Stone Complexity for PCNL: A Systematic Review of the Literature, How Best Can We Record Stone Complexity in PCNL? published in your journal. In the interest of standardized reporting of stone complexity, these instruments for measuring stone complexity are very important if they perform as expected, and several recent articles have assessed and externally validated these tools. Furthermore, to better understand the true utility of these instruments, a thorough systematic review of the individual studies provides a much more comprehensive evaluation. However, the quality of any output is dependent upon the quality of the input, and this is particularly true with systematic reviews. For results of a systematic review to be meaningful, a primary requirement is that data from the included articles are correctly retrieved. Interpretations based on information that is incorrectly harvested from the articles raise the risk of erroneous conclusions.
We noted in reading the article that our article, External validation of a preoperative renal stone grading system: reproducibility and inter-rater concordance of the Guy's stone score using preoperative computed tomography and rigorous postoperative stone-free criteria, published in Urology in 2014, was cited nine times by Withington et al. Among these, we noted three instances of incorrect data abstraction. These include our country (cited in table 3 as Lebanon, although we are located in the United States), the number of included patients (cited in table 3 as 66 patients, whereas we reported 166), and a statement noting missing method (page 5, lines 7–8), although these are clearly outlined in our article's method section.
From these three examples, it is clear that at least some of the input data are incorrect, and hence the summary results from the systematic review itself are jeopardized. Although these issues may in fact be completely irrelevant to their final conclusions, the reader nevertheless must be cautious in drawing any conclusions from this report. It may be worthwhile for the authors to consider repeating their data abstraction, and given that these are contemporary studies they should be able to contact the corresponding authors of each of their included studies to address any remaining uncertainties.