Heterogeneity in the Reporting of Disease Characteristics and Treatment Outcomes in Studies Evaluating Treatments for Nephrolithiasis

Introduction

U se of radiographic imaging in patients with nephrolithiasis is variable in clinical practice and typically at the discretion of the provider. There are no formal guidelines for either preoperative assessment of stone size or postoperative evaluation of stone clearance. CT, ultrasonography (US), plain radiography of the kidneys, ureters, and bladder (KUB), and intravenous urography (IVU) are all used variably in imaging patients with stone disease. While CT is the most sensitive modality for stone detection and provides the most anatomic information, ¹ US and KUB have lower associated cost, radiation exposure, and are generally more readily obtained. All of these modalities are characterized by variable sensitivity and specificity for stone detection, as well as margins of error in evaluation of stone size. ^{2 –4}

While there is no evidence that this variation in practice adversely affects patient care, it nonetheless contributes to heterogeneity in the literature regarding preoperative and postoperative evaluations of patients who are undergoing treatment. These studies are frequently retrospective and may include nonuniform initial evaluations as well as follow-up regimens. While there is no consensus regarding “optimal” treatment end points (eg, radiographic vs clinical, strict stone free vs presence of residual fragments [RF]), standardization of postoperative evaluations would nonetheless enable more reliable and accurate comparisons of treatment efficacy.

We have anecdotally observed heterogeneity in the literature regarding evaluations of patients who were undergoing treatment for stone disease. We performed a systematic literature search to evaluate the variability of these parameters in published studies.

Materials and Methods

A systematic PubMed search was performed of the literature from 2006 to 2008. Search terms included kidney stones, ureteral stones, nephrolithiasis, ureterolithiasis, ureteroscopy (URS), shockwave lithotripsy (SWL), and percutaneous nephrolithotomy (PCNL). Only articles that evaluated outcomes of stone treatment in adults were included. Excluded studies included those that evaluated the feasibility or safety of a specific treatment as well as those that evaluated specific imaging modalities.

The following parameters from each study were compiled in a prospectively determined database: Journal name, date of publication, retrospective vs prospective study, modality used for preoperative imaging, method of calculating stone burden (maximum diameter vs surface area vs volume), type of stone treatment, modality used for postoperative imaging, whether strict stone free vs presence of RF was reported, definition of treatment “success” if mentioned, other treatment outcomes mentioned (stone related events or stone related surgery), and timing of postoperative imaging used to evaluate stone clearance.

Results

One hundred and fifty-four studies were included in the analysis. Forty-seven (31%) studies were prospective and 107 (69%) were retrospective. There was wide variability in both type of imaging modality used preoperatively and method of calculating preoperative stone burden (Table 1). Treatment modalities included URS (46), PCNL (71), SWL (50), and multiple modalities (9).

There was also wide variability in imaging modalities used postoperatively and parameters used to evaluate stone clearance (Table 2). Radiographic outcomes included strict stone free (114) and presence of RF (64). There was wide variation in size of RF reported (<2 mm through <7 mm) (Table 3).

The “stone free” definition included the presence of RF in 18 studies. Treatment success was defined variably to include RFs of various sizes (Table 3). Reported clinical outcomes included stone-related events (46) and stone-related surgery (90).

There was a range of timing to postoperative imaging for assessment of stone clearance (mean 49 days; range 1–365). Time to imaging was ≤1 month in 47% (73), between 1 and 3 months in 27% (42), >3 months in 1% (2), and no mention in 24% (37).

Discussion

Our study confirms that there is wide variability in the literature in the preoperative and postoperative evaluation of patients who are undergoing treatment for stone disease. While this is primarily the result of variability in actual practice, it hinders our ability to compare the efficacy of treatments for specific disease subgroups.

Comparison of efficacy data for different therapies requires some assurance that preoperative and postoperative evaluations are similar. Improved standardization is needed, and future studies will need both more rigorous study design (eg, prospective studies with uniform preoperative and postoperative imaging) and guidance from professional societies regarding relevant treatment end points. Although logistical issues will prohibit total uniformity in the literature, some movement toward standardization is necessary to improve our ability to apply comparative data to clinical practice.

Indeed, variable use of imaging modalities in the literature complicates our ability to compare treatment results. While CT, US, and KUB/IVU all have viable roles in clinical practice, they may not be directly comparable because of differential sensitivity and specificity for stone detection as well as accuracy in stone measurement. CT is recognized as the most sensitive modality for stone detection (particularly in the ureter) and yields the most detailed anatomic information. ^3,5 Unlike KUB/IVU, it is useful for the vast majority of radiopaque and radiolucent stones. CT, however, is more costly than alternative modalities and involves greater radiation exposure. In addition, CT can have significant discrepancies in evaluation of stone size based on window setting (eg, abdomen vs bone) and stone composition or location. ^6,7

Eisner and colleagues ⁶ reported recently that the most accurate determination of stone size is obtained using magnified bone windows. Also, there can be variability in measurements of stone diameter using axial vs reconstructed coronal imaging, which may or may not be performed on an institutional and individual basis. Finally, CT protocol and slice thickness may affect stone measurements; in our analysis, only 3 of the 52 studies that used CT imaging preoperatively and/or postoperatively reported a uniform protocol and/or cut size.

US is an alternative modality that is used frequently based on its low cost, logistical ease, and lack of radiation exposure, although it is notoriously operator dependent. It may also provide indirect evidence of ureteral stones, including signs of obstruction—eg, absent ureteral jet, presence of hydronephrosis, and elevated resistive index. It has a wider margin of error for estimation of stone size compared with other modalities.

KUB is also readily available and inexpensive, but it has lower sensitivity for small renal stones and is less useful for radiolucent stones and stones that are overlying bony structures. Nonetheless, it correlates well with CT size measurements for medium to large stones and is particularly useful when following a known radiopaque stone. ⁸

IVU is less frequently used, but it can provide additionally both functional and anatomic information. ³

Regardless of which imaging modalities are used, comparable and accurate estimations of stone size are necessary so that practitioners can apply the results of a study to the correct subgroup of patients. Stone size is the central criterion for many practitioners in determining which treatment to recommend—eg, SWL/URS vs PCNL for stones <2 cm or >2 cm. Furthermore, detection and size measurements of RF vary based on the imaging modality and may confound algorithms for staged treatment based on size or presence of these fragments. ⁹ Finally, it is critical that urologists and radiologists have active dialogue regarding how stones are measured and reported to ensure consistency in translation of these results to clinical practice.

We also demonstrate that radiographic end points are variable in the literature, specifically the size of RF. There is no strict size criterion for “clinical significance” of RF, because risks of stone growth and future events may vary between populations (eg, patients with recurrent or infection stones or those on medical therapy). Most literature on RF pertains to SWL and demonstrates an increased risk of future stone events in a significant minority of patients (20%–40%) with RF ≤4 mm. ^{10 –12}

Regarding RF after PCNL, Raman and associates ⁹ demonstrated that RF ≤2 mm correlate with a significantly decreased risk of future stone events. Overall, the literature reflects that the majority of RF ≤4 mm will remain clinically silent, at least through reported follow-up. Other factors may affect the likelihood of passage of small RF, including stone position, previous stone surgery, metabolic abnormality, and placement of a ureteral stent. ¹³ Use of medical therapy also may enhance passage of small fragments, as well as modulate risk of stone growth and future events. ^{14 –17}

Based on the multitude of patient and disease factors that can affect stone passage, there are no clearly established radiographic criteria for success or failure of a procedure. Until we improve our understanding of the natural history of these fragments for specific disease subgroups, we suggest that clinical end points always be reported (eg, stone-related events or surgery) and radiographic end points include a range of RF sizes (eg, ≤2 mm, ≤4 mm, and strict stone free). While strict stone free is indeed most desirable, small RF may also be an acceptable treatment end point, depending on the clinical scenario.

Timing of postoperative imaging for assessment of stone clearance is also variable in the literature and, regardless of what definitions of success are used, may affect the presence or absence of RF. Passage of small RF may take at least several months. ¹³ There are different strategies during URS for addressing small fragments, including allowing tiny fragments to pass adjacent to a stent or after stent removal vs active extraction with or without an access sheath. These may variably affect the presence of RF on early postoperative imaging, and it is not clear whether these strategies differentially impact long-term outcomes. In addition, early postoperative evaluation of stone clearance may not reflect longer term stone growth or stone events that may be important sequelae not captured with early or even medium term follow-up. It is important that future studies of treatment efficacy report long-term clinical follow-up when possible.

Limitations of our study include using 3 recent years of literature rather than a larger sample. This was thought to be an adequately reflective sample size to evaluate variability in the literature, however, and represents the literature from the most recent period. Also, we pooled articles that pertained to adult subjects only, so the heterogeneity may not apply in exact proportions to pediatric literature.

Conclusion

There is heterogeneity in the literature in both the preopertive and postoperative assessment of patients who are undergoing treatment for stone disease. More uniformity is essential to ensure comparable preoperative staging and postoperative evaluation of stone clearance/clinical success to ensure a fair comparison of treatment efficacy. This increased rigor is particularly important in reporting outcomes for more controversial disease subsets (eg, lower pole stones) or treatment approaches (eg, URS vs PCNL for larger renal stones or SWL vs URS for ureteral stones in different locations based on size criteria). More rigorous analysis will allow physicians to counsel patients more accurately regarding risks, benefits, and success rates of different treatment options.