Abstract
Background
Detection of small renal calculi has benefitted from recent advances in computed tomography (CT) scanner design. Information regarding observer performance when using state-of-the-art CT scanners for this application is needed.
Purpose
To assess observer performance and the impact of radiation dose for detection and size measurement of <4 mm renal stones using CT with integrated circuit detectors and iterative reconstruction.
Material and Methods
Twenty-nine <4 mm calcium oxalate stones were randomly placed in 20 porcine kidneys in an anthropomorphic phantom. Four radiologists used a workstation to record each calculus detection and size. JAFROC Figure of Merit (FOM), sensitivity, false positive detections, and calculus size were calculated.
Results
Mean calculus size was 2.2 ± 0.7 mm. The CTDIvol values corresponding to the automatic exposure control settings of 160, 80, 40, 25, and 10 Quality Reference mAs (QRM) were 15.2, 7.9, 4.2, 2.7, and 1.3 mGy, respectively. JAFROC FOM was ≥ 0.97 at ≥ 80 QRM, ≥ 0.89 at ≥ 25 QRM, and was inferior to routine dose (160 QRM) at 10 QRM (0.72, P < 0.05). Per-calculus sensitivity remained ≥ 85% for every reader at ≥ 25 QRM. Mean total false positive detections per reader were ≤ 3 at ≥ 80 QRM, but increased substantially for two readers ( ≥ 12) at ≤ 40 QRM. Measured calculus size significantly decreased at ≤ 25 QRM (P ≤ 0.01).
Conclusion
Using low dose renal CT with iterative reconstruction and ≥ 25 QRM results in high sensitivity, but false positive detections increase for some readers at very low dose levels (≤ 40 QRM). At very low doses with iterative reconstruction, measured calculus size will artifactually decrease.
Introduction
The prevalence of urinary calculi in the US population has been increasing over the past several decades and now affects roughly 8.8% of the US population (1). Accurate detection of urinary calculi, secondary signs of ureterolithiasis, and alternative diagnoses have been demonstrated with computed tomography (CT) even when low dose technique is used (2–6). Most studies examining low dose CT for calculus detection have been based on detection of calculi larger than 3 mm in size, and there is relatively little information regarding the specific detection of tiny (<2.5 mm) renal calculi (7). Detecting tiny calculi is clinically important for comparing treatment efficacy and for calculus surveillance (8).
The task of detecting objects with intrinsically high contrast to background tissue facilitates CT dose reduction (9). While CT performs well for detection of larger calculi, accurate detection is negatively impacted independently by decreasing dose and decreasing calculus size (7,10). The difficulty in detecting small calculi is compounded by low dose technique as statistical noise can obscure calculi resulting in a false negative exam (11). Conversely, images with high degree of noise could also result in a false positive exam, which can potentially lead to misdiagnosis and/or unnecessary additional or follow-up examinations. Dedicated assessment of CT performance for detection of small calculi at low dose is needed to justify the use of either a low or high dose protocol.
Calculus composition influences attenuation and subsequent detection at CT. Calcium containing calculi are the most common calculus types, representing over 80% of urinary calculi (12). These calculi are the most attenuating with Hounsfield values referenced between 1200 and 2800, compared with non-calcium containing calculi referenced between 200 and 1100 (12). Previous research using single slice CT has shown decreased rates of detection for both lower attenuating calculi and decreasing radiation dose (7). Modern CT scanning equipment takes advantage of advancements in detector design, image reconstruction algorithms, automatic tube current modulation, optimal kVp selection, and CT noise reduction methods such as iterative reconstruction (13,14). Traditional CT detectors allow the introduction of electronic noise when transmitting analog signal to a distant analog to digital convertor. Integrated circuit detectors have been shown to significantly decrease signal degradation and improve image quality by incorporating the detector and converter within the same electronic board (15). Improvements such as these can provide improved spatial resolution, temporal resolution, contrast, and image quality while reducing radiation dose.
The principal goal of this study is to estimate the lowest dose level at which detection accuracy and calculus size remain unchanged at renal calculus CT using modern CT equipment and iterative reconstruction.
Material and Methods
Phantom construction
A phantom was constructed using human calcium oxalate calculi. Calculus composition was confirmed to be calcium oxylate monohydrate by infrared (Fourier transform infrared) spectroscopy. Calculi were mechanically fragmented until less than 4 mm in size (n = 29). Zero to four calculi were inserted into each of 20 porcine kidneys (i.e. 10 pairs of kidneys) based on a pre-determined randomization scheme for each pair of kidneys. The calculus insertion procedure was performed using an 11-gauge introducer needle with an inner diameter of 2.39 mm and a water bath to avoid introducing air bubbles. Kidneys were arranged in pairs within a plastic bag containing oil to simulate a retroperitoneal fat-soft tissue interface. The paired kidneys were then suspended in an acrylic water bath. The water bath was wrapped in radiation therapy bolus material, which possesses attenuation simulating soft tissue (Fig. 1). The cross-sectional diameter of the finished phantom was 31 cm anterior-posterior and 40 cm transverse, which would correspond to a large patient size.
Phantom used in study. (a) Cross-sectional CT image from study shows pairs of porcine kidneys containing the inserted calculi and submerged in a water bath. Subsequently, the water bath was wrapped in tissue-equivalent radiation therapy bolus material, which possesses attenuation simulating soft tissue. (b) Radiograph of the phantom showing the porcine kidneys and small calculi (left kidney) and (c) image of the phantom in our study prior to wrapping in tissue-equivalent radiation therapy bolus material.
Image acquisition
CT acquisition was performed using a 192-slice multidetector CT system (Somatom Force; Siemens Healthcare, Malvern, PA, USA). The system was equipped with integrated circuit detectors (Stellar detector; Siemens Healthcare) and iterative reconstruction (advanced modeled iterative reconstruction; ADMIRE; Siemens Healthcare). CT acquisition parameters included 120 kV, 0.8 mm focal spot, Br36d kernel, 30 cm field of view, 3 mm slice thickness, and 2 mm reconstruction. Tube current was varied for each acquisition by changing the Quality Reference mAs (QRM) using the CT system’s automatic exposure control. The first acquisition for each phantom was performed using 400 QRM; this high dose scan was used to create the reference standard. Subsequent acquisitions were performed using 160 QRM (our clinical tube current setting for our “low dose renal CT”), as well as: 80 QRM, 40 QRM, 25 QRM, and 10 QRM. The CTDIvol values corresponding to the automatic exposure control settings were 15.2, 7.9, 4.2, 2.7, and 1.3 mGy, respectively. ADMIRE iterative reconstruction strength was 2, 3, 4, 5, and 5 for the 160, 80, 40, 25, and 10 QRM scans, respectively.
Dose estimations
Effective dose for CT exams were calculated for each dose level (or QRM) using the CTDIvol reported by the scanner, and the assumption that the scan would be 40 cm in length if an abdomen and pelvis was being scanned. The following formula was used: Effective Dose (mSv) = CTDIvol × Scan Length × K Factor, where scan length = 40 cm and K Factor = 0.015.
Image evaluation
Four blinded genitourinary subspecialty-trained radiologists with 11, 18, 18, and 24 years of experience as staff radiologists participated independently as readers for detection of renal calculi. The workstation used for review allowed simultaneous viewing of the axial and coronal images of each kidney pair with standard image manipulation capability (i.e. window/level, zoom). Readers circumscribed a spherical region of interest (ROI) about the longest linear dimension of each identified calculus using the axial plane. The workstation automatically recorded the location and size of each circumscribed calculus. For each calculus detection, readers entered a confidence score (1 to 100), representing the perceived likelihood that a detection represented a true calculus (1 = absolutely negative to 100 = absolutely positive for the presence of calculus), to be used for jackknife alternative free-response receiver operating characteristic (ROC) 1 calculations (JAFROC1, described below). Readers were permitted to examine CT datasets in a single or multiple sessions.
The reference standard was created by the unblinded principal investigator and a staff GU radiologist not participating in the observer study (Fig. 2). These radiologists marked the presence and size of each calculus by tightly circumscribing them with the ROI tool in a manner similar to the test readers using the axial plane. The diameter obtained by the reference readers was used as the reference size for each calculus.
Depiction of overall study schema. The reference standard was created by two non-reader GU radiologists using high dose CT images. Routine and lower dose images were interpreted by four GU radiologists using a computer workstation, with readers circling small renal calculi. Reader detections were automatically matched to reference markings by a computer using an overlapping spheres method.
To match reference and reader detections for CT images, the computer workstation automatically matched reference and reader detections using an overlapping spheres method. Each reader detection was characterized by the computer as either a true positive or false positive detection.
Statistical analysis
Clinically relevant comparison of calculus detection methods requires correct detection as well as spatial localization of each calculus, permitting multiple calculi detections per kidney, and penalizing for false positive detections. The confidence scores for each detection were used to estimate the JAFROC1 figure of merit (FOM) (16), which is similar to ROC analysis but requires correct spatial localization of all detections and penalizes readers for false positives. To further explore diagnostic performance, we also calculated the sensitivity of each CT dose level by reader using a generalized estimating equation regression to account for multiple calculi per image (17) and counted false positive detections for each reader for every exam. Specificity was not estimable with the study design on account of no structured evaluation of negative (calculus-free) regions within the images.
We computed the mean size of each reader marking on each calculus at each of the CT dose levels. We calculated the difference between the reference size and average measured size across the readers and summarized median, interquartile range by dose. Wilcoxon signed rank test was used for comparison. Descriptive analyses were conducted using the SAS System (version 9.4, Cary, NC, USA). JAFROC1 FOM was estimated using JAFROC 4.2.
Results
Nine of ten kidney pairs had at least 1 calculus (range, 1–6) in 14 kidneys, with six kidneys not having any calculi. Median longest linear dimension of the calculi measured on the reference dose images was 2.3 mm (range, 1.2–3.7 mm; mean, 2.2 ± 0.7 mm, with only 4 calculi measured larger than 3 mm).
Accuracy of calculus detection
JAFROC1 FOM, indicating performance of readers, for lower-dose renal CT, along with estimated differences between routine clinical renal calculus CT and lower doses, including 95% CI.
QRM is the automatic exposure control setting for the CT system and adjusts tube current depending on patient attenuation and would be used to scan patients of any size. The CTDIvol is specific for the large anthropomorphic phantom used in our study.

Cropped CT images showing one kidney with a small calculus over a range of CT dose levels. Top row shows marking by the reference readers (green circle), as well as reader marking (red circle) by one of the GU radiologist readers. Reference detections were marked by reference readers only on the 400 QRM dataset, with the location recorded and superimposed on reader markings automatically by the computer workstation. Note that true positive reader detections are made down to the automatic exposure control setting of 25 QRM. Values in parentheses represent CTDIvol in mGy.
Reader sensitivity for the detection of 29 small renal calculi for low-dose renal CT. Confidence intervals have been adjusted for multiple calculi per dataset.
False positive detections at CT were low for two reviewers at all CT dose levels (Readers 1 and 2, Figs. 4 and 5). False positive detections at CT for the remaining two reviewers were also low using automatic exposure control settings of 80 and 160 QRM, but increased substantially at the very low doses (Fig. 4).
False positive detections per reader by CT dose level. (a) Cropped CT images showing a false positive reader marking using CT images obtained at an automatic exposure control setting of 40 QRM (CTDIvol 4.2 mGy; red circle, left-most image). Middle image shows the corresponding kidney without reader markings. Right-most image shows high-dose reference images obtained using 400 QRM. (b) Axial images show a single calculus at different tube current settings and iterative reconstruction strengths. The calculus appears substantially smaller at the lower dose levels and stronger strengths of iterative reconstruction.

Radiation dose
Estimated low-dose CT effective dose at each CTDIvol for the large anthropomorphic phantom used in this experiment.
Accuracy of calculus size measurement
Descriptive summary of the reference calculus size vs. measured calculus size at each dose level.
For each calculus at every dose level, the measured calculus size was the mean across the four readers. The table shows the median difference and interquartile ranges for this difference in size at each dose level relative to the reference measurement (obtained by reference readers using CT images obtained using 120 kV and 400 QRM).
Discussion
Our results in a dedicated experimental phantom study of small calcium containing calculi demonstrate that modern CT technique maintains a high sensitivity even when detecting these calculi at low doses. At very low doses, image noise may result in increased false positive rates, even for experienced radiologists. At low radiation dose and increasing strength of iterative reconstruction, the size of calculi measurably decreases.
High sensitivity was maintained even at very low automatic exposure control settings of 25 QRM, corresponding to a CTDIvol of 2.7 mGy and 1.6 mSv effective dose for our phantom. It is important to note that the high level of detection was maintained in our phantom, which was relatively large in size. Increasing patient size has previously been shown to have a detrimental effect on detection of detecting renal and ureteral calculi (18,19). This knowledge may be especially important in the emergency room, where known calculus formers tend to undergo repeated CT imaging, and there is a great desire to decrease the radiation dose as long as it does not compromise the detection of small calculi (4,20,21). As a result of this study, our clinical practice has lowered the automatic exposure control settings to 80 QRM for renal calculus CT performed on systems with integrated circuit detectors and iterative reconstruction.
Our results also suggest an adverse reader effect on false positive rate at very low dose levels. Two readers demonstrated an increased false positive rate with decreasing dose. At very low dose levels, CT images appear noisy even with strong strengths of iterative reconstruction and increased familiarity with the appearance of noisy very low dose images may be needed prior to their routine clinical use.
An important and notable aspect of our study is that the measured diameter of tiny (<4 mm) renal calculi decreased with automatic exposure control settings of 25 QRM and less when using strong levels of iterative reconstruction. Solomon et al. using semi-automated segmentation measures recently reported that model-based iterative reconstruction had a similar impact on the measured size of kidney calculi and lung nodules at 50% dose reduction (22). Our study is the first to confirm this finding using human observers making manual measurements and the methodology we employed permitted direct observation of both the artifactual “shrinking” of the calculus as well as the reader outlining of the calculus (Figs. 3 and 5). Prior reports did not find a significant impact of CT dose on calculus size, but these studies included mainly larger calculi (10,23). Because iterative reconstruction results in contrast-dependent spatial resolution (resulting in low contrast objects appearing smaller), as the contrast is lowered, the spatial resolution also declines (22,24). In our study, calculi appeared markedly smaller at doses employing high levels of iterative reconstruction.
Our study is limited by several factors. The phantom used in the study has limited applicability to different clinical populations given its large size. Furthermore, our phantom modeled calculi that are most commonly seen in clinical practice, i.e. calcium oxalate calculi, and hence may not be applicable to calculi of other compositions. Our phantom mimicked calculi within the renal calyces and did not assess calculus detection in clinically important locations such as the ureter or pelvis. The phantom also did not contain sources of anatomic noise and artifacts such as bone or bowel gas. The phantom size also has important effects on effective dose for CT (25). Medium and small patients would necessarily have even lower effective doses.
In conclusion, our results confirm that modern CT equipment can detect tiny urinary calculi at very low doses in an in vitro pig kidney model. At very low doses, we observed that some radiologists interpret CT exams with an increased number of false positive detections, and that the size of renal calculi can be artifactually decreased with strong levels of iterative reconstruction, resulting in size underestimation. Further evaluation of renal calculus detection in humans would be helpful to expand upon the findings of our study.
Footnotes
Acknowledgements
The authors thank Kristina Nunez for her assistance with manuscript preparation and Dr. Isaac Francis from the University of Michigan for his expert review and suggestions. They thank Maria Shiung for her coordination of reader efforts and overseeing the archival and distribution of image data, and Dr. Beth Schuler for her expertise in providing comparative radiation dose data.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The CT scanner used for this research is provided to Mayo Clinic by Siemens Healthcare as part of the Mayo Clinic CT Innovation Center. Other authors have no potential conflicts of interest.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Dr. Joel G Fletcher and Dr. Cynthia H McCollough received grant support from Siemens Healthcare, which manufactures the CT scanner used in this work.
