Reliability and Validity of the Triple D Score to Predict Stone-Free Status After Extracorporeal Shockwave Lithotripsy: Methodological Mistake

Dear Editor:

I was interested in reading the paper by Ozgor and colleagues that was published in the January 2017 edition of the Journal of Endourology. ¹ The purpose of the authors was to evaluate the accuracy of the Triple D scoring system in predicting shockwave lithotripsy (SWL) success rates. ¹ They reported that stone-free status was achieved in 115 out of 200 patients (57.5%), and 85 patients had one or more residual fragments (42.5%). Differences in stone characteristics, including stone location, density, and volume, were statistically significant in patients, regardless of whether SWL achieved stone-free status or not (p < 0.001, p < 0.001, and p < 0.001, respectively). ¹

These results have nothing to do with external validation, reliability, and validity. ^{2 –6} Reproducibility (precision) and validity (accuracy) as two completely different methodological issues should be assessed by using appropriate tests. Regarding reliability, for quantitative variables, intraclass correlation coefficient should be used; for qualitative variables, weighted kappa should be used. However, to assess validity, for quantitative variables, interclass correlation coefficient (Pearson r) and for qualitative variables, sensitivity, specificity, positive predictive value, negative predictive value, likelihood ratio positive and likelihood ratio negative, as well as diagnostic accuracy and odds ratio are among the most appropriate tests. Moreover, for prediction, we need at least two different cohort datasets or at least one cohort dataset, splitting them for development and validation of our model. ^5,6

Their multivariate analyses revealed that Triple D score and stone location were identified as independent factors affecting SWL success (p < 0.001 and p = 0.008, respectively). The mean number of SWL sessions was significantly higher in patients with SWL failure (p = 0.003), concluding the external validity of the Triple D scoring system and its association with SWL success in the treatment of renal and ureteral stones. Such a conclusion is just a misleading message, because none of the earlier methodological issues regarding reliability, validity, and prediction have been taken into account.