Abstract
Purpose:
To determine the extent to which radiologists and urologists can predict histology using multiphasic CT imaging.
Methods:
Patients with a preoperative multiphasic CT undergoing surgery for a renal mass were identified between 2003 and 2013. Tumors >10 cm, locally advanced or metastatic disease, and patients managed by reviewers were excluded. A survey and deidentified scans were provided to reviewers. Sensitivity and accuracy in predicting histology was calculated for each reviewer. Correlation was assessed by the Fleiss kappa coefficient. Multivariable logistic regression determined factors associated with predictive accuracy for final pathology.
Results:
There were 120 patients who met criteria. Mean tumor size was 3.3 cm; there were 102 (85%) that were malignant, and 73% of these were clear-cell renal-cell carcinoma (RCC). The most common benign histology was angiomyolipoma (n=10, 56%) followed by oncocytoma (n=5, 28%). Correlation among reviewers was statistically fair for predicting malignant (κ=0.25) and final pathology (κ=0.22). Sensitivity for predicting malignant masses was 90%. Reviewers accurately predicted malignant pathology in 82% of cases and predicted final pathology in 58% of cases. Adjusted for size, scan type, and reviewer, clear-cell RCC vs benign histology was associated with 21 times increased odds of accurate pathologic identification (P<0.001).
Conclusions:
Urologists and radiologists were able to accurately identify malignant histology in 82% of cases, although sensitivity for malignant histology was 90%. Developing a preoperative nomogram for identification of clear-cell RCC may be feasible and should be further explored.
Introduction
T
Cross-sectional imaging is nearly universal in the evaluation of a patient with a newly diagnosed renal mass. Current American Urological Association guidelines recommend either MRI or CT with and without contrast enhancement to assess tumor size, location, presence or absence of renal vein, or inferior vena cava involvement, lymph node involvement, and vascular anatomy. 6,7 Over the past decade, multiple studies have attempted to define imaging-associated factors that could be used to predict both tumor histology and tumor grade. 8 –15 Some tumors, such as angiomyolipoma, can be reliably predicted through renal mass imaging. 16 Other tumor types—clear-cell renal-cell carcinoma (RCC), papillary RCC, and oncocytoma—have demonstrated characteristic radiographic patterns; however, the accuracy of histologic reporting based on CT imaging has not been clearly evaluated. 15,17 –19
We therefore chose to investigate the interpretative accuracy of urologists and radiologists using CT imaging to predict renal tumor histology. We hypothesized that there would be no difference in the predictive accuracy between urologists and radiologists.
Methods
A retrospective review was performed to identify consecutive adult patients undergoing partial or radical nephrectomy between 2003 and 2013 (n=2389) with an available preoperative multiphasic CT scan (dual phasic, triphasic, or CT angiogram) from our institution or a referring institution within the previous year. There was no standard scan protocol. Patients whose preoperative multiphasic CT scans were no longer available (n=1989), with known metastatic or locally advanced disease (n=2 3), tumor diameter >10 cm (n=14), and polycystic kidney disease (n=9) were excluded. The majority of patients without available scans had uniphasic CT, MRI, ultrasonography, or a no longer existing scan from a referring institution. Patients receiving care from study reviewers were excluded (n=234). The study reviewers included two urologic oncologists (TAM, RSB) and two abdominal radiologists (AAP, MT). All reviewers had substantial experience interpreting renal mass CT scans.
After identification of cases for inclusion, scans were deidentified and provided to reviewers. Reviewers were able to access all sequences and to use standard radiographic tools including measurement of Hounsfield units (HU). A standardized computer questionnaire was generated for each reviewer to individually complete while assessing each case (see Supplementary Questionnaire; Supplementary Data are available online at
The Fleiss kappa correlation coefficient assessed interrater correlation. Significance of kappa scores was defined as <0.01: poor; 0.01–0.2: slight; 0.21–0.4: fair; 0.41–0.6: moderate; 0.61–0.8: substantial; and 0.81–1.0: almost perfect correlation. 21 Sensitivity and accuracy in predicting histology were calculated. Multivariable logistic regression assessed factors associated with accurate prediction of malignant final histology based on CT imaging. Variables in the model were determined a priori as factors likely to be associated with accurate prediction. These included pathology (benign, nonclear-cell, clear-cell), scan type (dual or triphasic vs CT angiogram), and radiologist vs urologist.
Prediction plots from the regression model were generated to evaluate the predicted accuracy for identification of malignancy based on increasing tumor size when comparing urologists with radiologists when adjusting for scan type and pathology. A priori, P values <0.05 were considered statistically significant. Stata version 12.1 (Stata Corp., College Station, TX) was used for statistical analyses. Institutional Review Board approval was granted for the conduct of this study by Indiana University.
Results
One hundred twenty patients met all criteria for inclusion in the study. Patient characteristics are shown in Table 1. Eighty-five percent of patients had malignant histology (n=102/120), with the most common malignant histology being clear-cell RCC (n=74/102, 73%). The most common benign histology was angiomyolipoma (n=10/18, 56%) followed by oncocytoma (n=5/18, 28%). Tumors were fairly evenly distributed between the left and right kidneys, and the mean (standard deviation [SD]) tumor diameter was 3.3 (1.9) cm. The mean (SD) tumor diameter for clear-cell RCC was 3.3 (1.9) cm, for papillary RCC was 3.2 (1.9) cm, and for oncocytoma was 3.6 (2.1) cm.
SD=standard deviation; IQR=interquartile range.
Among the 102 with malignancy, 71 (70%) were accurately identified as malignant by all four reviewers (71 cases=284 scans). The most commonly cited reason for assignment of malignant histology was enhancement pattern (n=271/284 scans). For 28 (38%) of the 74 clear-cell RCC cases, all reviewers accurately predicted the histology. In 95% (n=106/112 scans) of these accurately predicted clear-cell RCC cases, reviewers marked enhancement pattern as a significant contributor to their decision (Fig. 1A). For 55% (n=61/112 scans), the reviewers marked the presence of tumor necrosis. Finally, for 43% (n=48/112 scans), reviewers cited tumor margin as the reason for their selection. Among the 19 papillary RCC tumors, on four occasions the majority of reviewers (≥3) accurately identified the tumor histology (Fig. 1B). For these cases, the most commonly cited radiographic indicator of the tumor type was the enhancement pattern. All angiomyolipoma cases were identified by the reviewers from the presence of macroscopic fat in the tumor (Fig. 1C).

Examples of accurately predicted clear-cell renal-cell carcinoma (RCC) (
There was fair correlation among reviewers for predicting benign vs malignant (κ=0.25) and for predicting final pathology (κ=0.22). Overall, average sensitivity for identifying malignant histology was 90% (range 80%–99%) (Table 2). The average accuracy in predicting malignant histology was 82% (range 75%–93%). Sensitivity and accuracy for predicting clear-cell RCC histology were 74.3% (range 58%–92%) and 62.0% (range 53%–81%), respectively. Overall accuracy in predicting the final pathology was 57.5% (range 47%–77%) (Table 2). There was a 10% false negative rate.
On multivariable logistic regression, when compared with scans containing benign histology, both nonclear-cell RCC malignant histology and clear-cell RCC histology were independently associated with accurate prediction of malignancy (P<0.001 each) (Table 3). Of note, the odds of accurate prediction of malignancy were three times higher for clear-cell RCC histology when using nonclear-cell RCC as the comparison (odds ratio 2.68, 95% confidence interval 1.4–5.2, P=0.004). Radiologists were at 59% increased odds of accurate prediction of malignancy, which did not attain statistical significance (P=0.100). With increasing tumor size, there was a nonstatistical improvement in accurate predictive ability. Radiologists demonstrated nonstatistically significantly better accuracy at prediction compared with urologists (Supplementary Fig. 1; Supplementary Data are available online at
Discussion
The relevance of this study is incumbent on emerging data that, similar to other tumor types such as breast and lung, there is value to diagnosing histology before implementing treatment plans for kidney masses. Surgery for benign renal tumors may be unrewarding for the surgeon and nonbeneficial to the patient. Biopsy alone carries inherent limitations including nondiagnostic biopsies and procedure morbidity. In addition, certain patient and tumor characteristics pose logistical challenges—large body habitus, small tumor size, anterior or upper-pole location—that may preclude biopsy altogether. Because imaging, especially CT, is ubiquitous in the workup for newly diagnosed renal masses, the potential to predict histology or differentiate malignant from benign disease would be both useful and cost-effective.
Multiple studies have examined the predictive role of imaging in the evaluation of renal tumors, the majority of which attempt to separate malignant from benign histology to delineate which tumors need surgical intervention. Recently, Kutikov and associates 12 created a nomogram for the prediction of malignant histology based on nephrometry scores, a tool commonly used by urologists to classify the complexity of tumors that is slowly being integrated into the armementarium of radiologists. 22 In validation studies, the area under the curve for the nomogram's ability to identify malignant histology suggested good predictive ability. 12,23
Additional models for predicting malignant histology have been proposed with varying success. 10,11,13 Although current models may be reasonably predictive for tumor complexity and/or the presence of malignancy, there may be room to improve on their clinical utility. Specifically, a model's ability to identify specific tumor histology could be particularly useful in determining treatment plans, especially for small renal masses.
In our study, the average sensitivity for malignant histology was 90%. This indicates that despite occasional false positives, evident by the 82% accuracy rate, 90% of malignant cases were successfully identified from multiphasic CT scan. We also report that clear-cell RCC histology on final pathology was associated with increased odds of accurate histologic prediction while other previously suggested variables such as tumor size, patient age, and sex did not improve results. Interestingly, if a reviewer had chosen “malignant” for all cases, this would have a 100% sensitivity and an 85% accuracy, both better results than we report. It is important to note, however, that the false negative rate for choosing nonmalignant in the setting of malignant disease was only 10% and it would have been significant higher if “malignant” were chosen universally. It is imperative to minimize false negatives—true malignant disease incorrectly diagnosed as benign—when determining if a test is useful or not.
Studies examining cross-sectional imaging features associated with subsequent diagnosis of malignancy are numerous. Contrast enhancement is suggested as a method of differentiating histologic subtypes 8,15,19,24,25 and was reported by our reviewers as a useful method of predicting tumor histology. Alshumrani and colleagues 8 reported that median nephrographic phase enhancement for clear-cell RCC was 65 HU, for papillary RCC it was 16 HU, and for oncocytoma it was 80 HU. 8 In addition, Zhang and coworkers 15 report that clear-cell RCC scans often show mixed enhancement patterns while papillary RCC has the least degree of enhancement.
Tumor characteristics, mainly size and tumor depth as defined by nephrometry scores, have also been mentioned as useful predictors of renal tumor malignancy. 12,13,23 In the nomogram from Kutikov and associates, 12 tumor size was the strongest predictor of malignant status. They further reported that large, interpolar tumors had the highest odds of malignancy. In addition, a study by Lane and colleagues 13 implicated size as the strongest predictor of malignancy for T1 tumors. Neither study reported a cut-point for tumor size most strongly associated with malignancy.
In addition to tumor size and location, Okhunov and coworkers 26 reported that presence of perirenal fat on CT imaging increases the odds of a tumor having clear-cell RCC histology. The reviewers in our study cited that the enhancement pattern, presence of necrosis, tumor size, and tumor margin aided in assigning histology, which is in line with these previous studies. Similar to a report by Remzi and colleagues, 27 our study found that size was unreliable in being an independent predictor of renal malignancy, however.
Interest in predicting malignant histology from imaging exists among radiologists, as evidenced by numerous studies published in the radiologic literature on the topic. 15,17,24,28 Our study is, however, the first to compare predictive abilities between radiologists and urologists. Within the confines of the present study, radiologists were not significantly more accurate in predictive ability. Interestingly, one urologist and one radiologist were subjectively better at identifying histology, which leads us to believe that there is nothing inherent to the individual specialties the enables physicians to be more accurate at predicting tumor subtype. We had hypothesized that there would be no difference in ability between specialties to predict histology; however, we had not expected to find such significant variation between reviewers. We believe the differences warrant further investigation into how radiologists and urologists examine films when evaluating for renal mass histology.
Novel technologies to identify specific histology are being evaluated. One promising option is the chimeric G250 ligand on positron emission tomography (PET)-CT scan, which is touted as highly specific for clear-cell histology. 29,30 In recently published results from the REDECT trial, 195 patients underwent PET-CT and contrast enhanced CT after receiving the ligand. Three reviewers predicted whether the mass was clear-cell RCC. The average sensitivity for identifying clear-cell in the setting of the PET-CT with ligand was 86% compared with 76% by contrast CT alone. Specificity was significantly lower for the contrast CT than PET-CT. We similarly found an average sensitivity of 74% for the identification of clear-cell RCC, which increased to 90% for our ability to predict malignant vs benign histology.
The potential for novel technologies in the preoperative identification of clear-cell RCC is exciting; however, the costs associated will likely be high. Because nearly all patients undergo CT scan as part of their renal mass workup, continuing to investigate better ways to use this ubiquitous tool to predict renal tumor histology is worth pursuing.
Overall, we were disappointed in our ability to accurately identify histology. Despite having 90% sensitivity for malignant histology, our sensitivity for clear-cell RCC was only 74% and even lower for papillary RCC. Certain characteristics were evident and helpful in the accurate assignment of pathology, as evidenced by Figure 1; however, these were not applicable in all scans. One radiologist achieved 92% sensitivity for clear-cell, which suggests that it is possible to accurately identify clear-cell RCC from preoperative multiphasic CT scan. Radiologist 1 primarily relied on the enhancement pattern and presence of necrosis or calcification in his assignment of clear-cell. This reviewer also accurately identified 15 of the 19 papillary RCC cases and demonstrated 92.5% predictive accuracy for histology in patients with malignant tumors.
It should be mentioned that before conducting our study, we elected not to discuss as a group which pathologic features might predict particular histology. Although this may have improved our final results, we were intending to provide a snapshot of our current individual predictive abilities. Based on our preliminary findings, we hope to generate a predictive tool using the combined approaches of the reviewers to improve our predictive accuracy. We anticipate this future nomogram could be used by both urologists and radiologists to aid therapeutic decision making and potentially obviate the need for renal core biopsy for some newly diagnosed renal masses.
Recent studies suggest that differentiating between malignant and benign tissue by percutaneous renal mass biopsy is nearly 100% when compared with final tumor pathology. 31,32 The ability to accurately identify specific histology is lower at 94% to 98%, and the nondiagnostic biopsies occur in 3% to 22% of biopsies depending on published series. 31,32 Because one of our reviewers was able to achieve 93% predictive accuracy for the tumor histology based on preoperative cross-sectional imaging, we think that it may be feasible to create a predictive tool using specific radiographic tumor characteristics that may reach an acceptable level of accuracy, especially when renal mass biopsy is high risk or not a reasonable option.
Generating a reliable and accurate nomogram to predict malignant—specifically clear-cell RCC—may significantly impact renal tumor management. Patients with benign tumors may obviate surgical intervention or surveillance imaging. Small low-grade papillary or clear-cell RCC tumors may be better selected for active surveillance. Finally, a reliable nomogram would aid in identification of patients who might benefit from enucleation (minimal margin) partial nephrectomy based on tumor capsule properties inherent to various tumor subtypes. 33
This study is not without limitations. By excluding cases managed by study reviewers, many cases were eliminated. Although including cases managed by the reviewers would have allowed for more scans to be examined, it would have introduced bias because the reviewers would be more likely to recall the pathology, particularly for challenging cases. Because only four reviewers participated in the study, our results may be less generalizable. Despite these limitations, this study is the first to compare the ability of radiologists and urologists to predict renal mass histology, specifically being able to differentiate malignant tumors using CT.
Histologic specific management of localized renal masses is expanding, and using preoperative imaging for predictive purposes would be advantageous. At the current time, we do not believe that the majority of urologists and radiologists are able to reliably predict renal mass histology using standard CT imaging. The generation of a prediction tool, designed by urologists and radiologists, may enable improved histologic identification in the future.
Conclusions
Urologists and radiologists accurately identified malignant histology in 82% of cases, although sensitivity for malignant histology was 90%. Development of predictive models and general education of urologists and radiologists in patterns of radiographic tumor characteristics may improve the sensitivity and specificity of preoperative imaging to determine renal mass histology.
Footnotes
Disclosure Statement
No competing financial interests exist.
Abbreviations Used
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
