Abstract
Background
A solitary pulmonary nodule (SPN) may represent early stage lung cancer. Lung cancer is a devastating disease with an overall 5-year mortality rate of approximately 84% but with early detection and surgery as low as 47%. Currently a contrast-enhanced multiple-row detector CT (MDCT) scan is the first examination when evaluating patients with suspected lung cancer.
Purpose
To apply an additional high resolution CT (HRCT) to SPNs to test whether certain morphological characteristics are associated with malignancy, to assess the diagnostic accuracy of HRCT in the characterization of SPNs, and to address the reproducibility of all measures.
Material and Method
Two hundred and thirteen participants with SPNs were included in a follow-up study. Blinded HRCT images were assessed with regard to margin risk categories (MRCs), calcification patterns and certain other characteristics and overall malignancy potential ratings (MPRs) were given. Morphological characteristics were tested against reference standard and ROC methodology was applied to assess diagnostic accuracy. Reproducibility was measured with Kappa statistics and 95% confidence intervals were computed for all results. Histopathology (90%) and CT follow-up (10%) were used as reference standard.
Results
MRCs (P < 0.001), calcification patterns (P = 0.003), and pleural retraction (P < 0.001) were all statistically significantly associated to malignancy. Reproducibility was moderate to substantial. Sensitivity, specificity, and overall diagnostic accuracy of HRCT were 98%, 23% and 87%, respectively. Reproducibility was substantial.
Conclusion
Statistically significant associations between SPN MRCs, calcification patterns, pleural retraction and malignancy were found. HRCT yielded a very high sensitivity and a somewhat lower specificity for malignancy. Reproducibility was high.
Keywords
A solitary pulmonary nodule (SPN) is defined as a round or oval lesion smaller than 3 cm, completely surrounded by lung parenchyma and not associated with lymphadenopathy, atelectasis or pneumonia (1). Work-up of an SPN is important as it may represent early stage lung cancer, a disease with an overall 5-year mortality rate of approximately 84%. However, with early detection and surgery the mortality rate can be as low as 47% (2).
The search for lung cancer is one of the premier uses of chest imaging and the now many years of computed tomography (CT) imaging have seen a steady evolution of methods used to evaluate pulmonary nodules and mass lesions. Today a contrast-enhanced multiple-row detector CT (MDCT) scan is the first examination when evaluating patients with suspected lung cancer.
Previous studies have shown that an additional high resolution CT (HRCT) scan using thin slices is beneficial in the work-up of an SPN due to the enhanced ability to study morphological characteristics. Especially margin characteristics, calcification patterns, and certain other characteristics or signs have been associated with malignancy (3–9). However, most of these studies were carried out using single-slice scanners and were methodologically weakened by either failure to summarize and analyze participant demographics, failure to apply proper statistics and compute confidence intervals, or failure to assess the reproducibility of the results. In the Vancouver guidelines as well as the more recent STARD initiative these issues are all addressed (10, 11).
This study, inspired by the STARD initiative, explored the specific HRCT characteristics of SPNs, and evaluated the reproducibility of all observations.
Material and Methods
Subjects
The study conformed to the Danish legal requirements. As all subjects received standard patient care, institutional review board approval was waived (The Central Denmark Research Committees on Biomedical Research Ethics, case no. 1-15-0-72-2-09).
A follow-up study was designed. In a 68-month long study period all adult patients with no previous malignancies referred from their general practitioner to the Department of Pulmonology at our hospital for the evaluation of suspected lung cancer were clinically evaluated. Based on individual symptoms and/or signs they were referred to an MDCT scan of the thorax and upper abdomen. If an SPN was found on the MDCT, an HRCT scan of the thorax was performed. Consecutive patients with SPNs sized 5–30 mm that fulfilled the general SPN criteria were eligible for inclusion. However, only nodules sized 10 mm or more were biopsied. A total of 1988 patients were referred to an MDCT scan for the evaluation of a suspected lung cancer in the study period, but only 249 patients fulfilled the defined SPN-criteria and were eligible for inclusion. However, 18 participants were excluded as the readers could not agree on which nodule to describe (if any at all), and 18 participants were excluded due to insufficient follow-up time, leaving 213 participants for analysis (Fig. 1). Participants were only included once.
Study flow diagram (modified from
Procedures
After regular MDCT scans, spiral HRCT scans were performed. HRCT scans were planned to include nodules only, using an MDCT scanner (Philips MX 16-channel CT scanner or Philips Brilliance CT 64-channel scanner; Philips Healthcare, Best, The Netherlands), and following a standard protocol with the settings: slice thickness 1.00 mm, scan mode helical, tube voltage 120 kV, tube current automatic, filter L. No contrast material was administered.
Crude HRCT data were copied from our hospital's fully digitalized Picture Archiving and Communication System (PACS) onto a dedicated Philips Brilliance CT workstation ver. 4.02 and blinded in preparation for the study using the standard software application ‘Anonymize’.
Collection and validation of the data
Two senior consultant radiologists (FR and HTM) reviewed the blinded HRCT scans of SPNs. No patient data were visible to the readers and they had no access to each other's results. The readers were granted access to raw scanning data and were allowed to present and manipulate these as preferred.
The readers categorized each SPN as solid, partly solid or non-solid. Previous studies have shown that, although abundant, solid nodules do not represent the same risk of malignancy as partly solid or nonsolid nodules (12). Thus, in the first part of the study, focus was on solid nodules only. Previously, these nodules have been categorized into one of six margin categories based on HRCT appearance (smooth, polygonal, lobulated, spiculated, ragged, halo) (7). However, as some margin categories are hard to discriminate from another (Fig. 2), the margin categories were organized into only three margin risk categories (MRCs) based on their HRCT appearance: 3 high risk (spiculated, ragged); 2 intermediate risk (lobulated); 1 low risk (smooth, polygonal). Nodules previously described as having a ‘halo’ were considered partly solid nodules. If more than one category was applicable, the readers chose the dominant. Calcification patterns were rated on a scale of 4 to 1: 4 malignant (dystrophic, amorphous); 3 indeterminate (eccentric); 2 benign (central, lamellar, chondroid); 1 none; and other characteristics (fat inclusion to bronchus sign to vessel sign to pleural retraction) as being present or absent. The bronchus sign was defined as a bronchus leading to the SPN or an air bronchogram (4), and the vessel sign as a pulmonary vessel leading to the SPN (5).
(a and b) Same SPN as in Fig. 3b as it appears on two different slices 4 mm apart. One of the readers characterized the nodule as ‘spiculated’ while the other characterized it as ‘ragged’. Consensus was ‘ragged’; (c and d) Another SPN on two different slices 5 mm apart. One reader referred to it as ‘smooth’ and the other as ‘polygonal’. Consensus was ‘polygonal’. Rather than being mere examples of inter-rater disagreement these images illustrate that the same nodule may fit into more than one margin category, and thus explains the need to further organize margin categories into margin risk categories (MRCs). (All: C-425 to W1400)
In the second part of the study, the readers assigned an integrative malignancy potential rating (MPR) on a scale of 5 to 1 to all nodules based on a weighting of nodule attenuation, margin risk category, calcifications and other characteristics: 5 definitely malignant; 4 probably malignant; 3 indeterminate; 2 probably benign; 1 definitely benign. All readings were done by both radiologists individually followed by consensus (Figs. 2 and 3).
(a) Smooth SPN with benign calcifications and the vessel sign. The nodule was rated ‘definitely benign’ and biopsy revealed a hamartoma. The patient was a-78-year-old woman; (b) Ragged SPN with no calcifications. Both the vessel sign and pleural retraction was present. The nodule was rated ‘definitely malignant’ and surgical resection revealed a squamous cell carcinoma. The patient was a 54-year-old man; (c) Partly solid SPN. The nodule was rated ‘definitely malignant’ and surgical resection revealed an adenocarcinoma. The patient was an 82-year-old woman; (d) Lobulated SPN with no calcifications. Both the vessel sign and pleural retraction was present. The nodule was rated ‘definitely malignant’ and surgical resection revealed an adenocarcinoma. The patient was a 78-year-old man. (All: C-425 to W1400)
By means of mainly transthoracic fine or coarse needle aspiration biopsy or surgical resection a definitive histopathological diagnosis was obtained in 90% (191/213) of the cases. In the majority of the procedures material was obtained by biplane fluoroscopy-guided fine needle aspiration biopsies. Ultrasound-guided biopsies were performed when appreciable and CT-guided biopsies in a very few cases. All malignant diagnoses were based on cytology or histopathology and all non-malignant diagnoses were verified by additional coarse needle aspiration biopsies. In this manner three temporally distinct non-malignant diagnoses were accepted.
A senior consultant pathologist examined and characterized tissue specimens with regard to malignancy as being either present or absent. Whereas only a single positive specimen was required for a nodule to be considered malignant, three consecutive negative specimens were required for a nodule to be considered benign. For the remaining 10% of the patients (22/213) it was not appropriate to do an invasive procedure to achieve a definitive diagnosis for various reasons (i.e. nodule size <10 mm, poor compliance, severe co-morbidity, etc.). These patients were instead followed by regular interval CT scans according to international standard. Follow-up was done at 3, 6, 12, 24 months or longer if considered necessary (13), but ceased prematurely if lesions resolved entirely during the follow-up period. Median follow-up time (25th–75th percentile) was 12 (8–28) months.
Statistical methods
The study results were analyzed using the consensus readings. Likelihood ratios of positive test results (LR+) were computed with 95% confidence intervals, and Fisher's exact test was used to test variables against reference standard. P values of ≤0.05 were considered statistically significant.
MPRs were applied on an ordinal scale of 5 to 1 with increasing values being indicative of disease. Based on the receiver-operating characteristics (ROC) methodology MPRs were tabulated against reference standard outcome at multiple cut-offs. Considering the severity of the disease, a cut-off prioritizing sensitivity was chosen, and thus MPRs 5 to 3 were treated as malignant test results. Sensitivity, specificity, positive and negative predictive value were tabulated and 95% confidence intervals were computed with binomial exact methods. The overall diagnostic accuracy was defined as the area under the ROC curve and calculated using a maximum-likelihood ROC model assuming bivariate normal distribution (14). A 95% confidence interval was computed with the bootstrap method.
Reproducibility of ordinally scaled measures (MRCs, calcifications, MPRs) was assessed with weighted kappa statistics with quadratic weights, and reproducibility of dichotomous measures (other characteristics) was assessed with Cohen's unweighted kappa. 95% confidence intervals were computed using exact methods for binomial outcomes and the bootstrap method for all other outcomes. Kappa values reflected 0.00–0.20 slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and 0.81–1.00 almost perfect reproducibility.
The licensed statistical software package STATA/IC 10 (StataCorp LP, College Station, TX, USA) was used.
Results
Ninety-nine men and 114 women were included. Mean age (range) was 65 (32–87) years, and mean SPN size (standard deviation) was 18 (6) mm.
There was a distribution of 58% (124/213) malignant and 42% (89/213) benign nodules. Of the 124 malignant nodules the distribution was 4% (5/124) small cell carcinomas, 21% (26/124) squamous cell carcinomas, 52% (65/124) adenocarcinomas, 2% (3/124) large cell carcinomas, 3% (4/124) other non-small cell carcinomas, 3% (4/124) lung metastases, and 14% (17/124) unclassified malignant nodules. Furthermore, the readers agreed on a distribution of 92% (196/213) solid nodules, 7% (15/213) partly solid nodules and 1% (2/213) non-solid nodules [κ: 0.73 (0.54–0.87)].
Only solid nodules were focused on for morphologic characteristics. MRCs were highly associated with malignancy (P < 0.001). Thus 53% of the malignant nodules were categorized as high-risk nodules [LR+ : 5.5 (2.8–11)], 39% as intermediate-risk nodules [LR+ : 2.0 (1.6–2.6)], and only 9% as low-risk nodules [LR+ : 1.0]. Reproducibility of the results was substantial [κ: 0.67 (0.58–0.75)] (Table 1).
Cross-tabulation of margin risk categories (MRCs) and reference standard (n = 196)
MRC cross-tabulated against reference standard. For each row the number of subjects within the category with either malignant or benign nodules is shown. Also shown is the likelihood ratio of a positive test result (LR+). Results are given with 95% confidence interval
Calcifications were present in 5% (6/112) of the malignant and 6% (5/84) of the benign SPNs, and there was a statistically significant association between calcification patterns and malignancy (P = 0.003). In fact all calcification patterns characterized as ‘indeterminate’ or ‘malignant’ were malignant and all calcification patterns characterized as ‘benign’ were benign. Reproducibility was substantial [κ: 0.74 (0.44–0.94)] (Table 2).
Cross-tabulation of calcification patterns and other characteristics and reference standard (n = 196)
Calcification patterns and other characteristics cross-tabulated against reference standard. For each row the number of subjects showing a specific trait with either malignant or benign nodules is shown. Also shown is the likelihood ratio of a positive test result (LR+), P value, and kappa value. Results are given with 95% confidence intervals
Calc = calcification patterns, Indeterm = indeterminate; N/A = not applicable
In 58% of the malignant SPNs and 31% of the benign SPNs pleural retraction was recognized, making this a highly significant predictor of malignancy with moderate reproducibility [LR+ : 1.9 (1.3–2.7); P < 0.001; κ: 0.56 (0.44–0.68)]. Likewise, the vessel sign was present in 54% of the malignant SPNs and 31% benign SPNs, making this an equally highly significant predictor of malignancy with fair reproducibility [LR+ : 1.7 (1.2–2.5); P = 0.002; κ: 0.25 (0.12–0.37)]. Fat was present in 5% of the benign SPNs but none of the malignant SPNs, making this a statistically significant predictor of benignity with moderate reproducibility [LR+ : 0; P = 0.03; κ: 0.43 (0.02–0.84)]. The bronchus sign was not associated with malignancy (Table 2).
For the integrative approach all nodules were in focus. Prevalence of disease was 58% (51–65%). According to the ROC table corresponding positive likelihood ratios were found ranging from 1.0 for definitely benign nodules, to 1.1 (1.0–1.2) for probably benign nodules, to 1.3 (1.1–1.4) for indeterminate nodules, to 2.9 (2.1–4.1) for probably malignant nodules, and to 8.3 (4–17) for definitely malignant nodules, respectively. The ROC table was analyzed according to our predefined cut-off. Sensitivity was 98% (94–100%), specificity was 23% (14–33%), false-positive rate was 78% (67–86%), and false-negative rate was 2% (0–6%). The positive predictive value was 64% (57–71%) and the negative predictive value was 91% (71–99%). From the fitted ROC curve the overall diagnostic accuracy was estimated to 87% (83–92%) (Fig. 4). Reproducibility of the MPRs and thus also of the diagnostic test results was substantial [κ: 0.76 (0.69–0.82)] (Tables 3 and 4).
Parametric ROC curve of true-positive rate (sensitivity) and false-positive rate (1-specificity). When applicable, the area under the parametric ROC curve represents a measure of the unbiased overall diagnostic accuracy of a test. The curve was constructed assuming bivariate normal distribution
ROC tabulation of malignancy potential ratings (MPRs) and reference standard (n = 213)
MPR cross-tabulated against reference standard. For each MPR the number of subjects with either malignant or benign nodules is shown. Also shown is the likelihood ratio of a positive test result (LR+). Results are given with 95% confidence intervals
Prevalence of disease, classification probabilities, predictive values and diagnostic likelihood ratios (n = 213)
Diagnostic test variables with estimates and 95% confidence intervals. Derived from Table 3
Discussion
Two hundred and thirteen participants with SPNs were included in a follow-up study. Blinded HRCT images were assessed with regard to margin risk categories (MRCs), calcification patterns and certain other characteristics and overall malignancy potential ratings (MPRs) were given. Morphological characteristics were tested against reference standard and ROC methodology was applied to assess diagnostic accuracy. Reproducibility was measured with kappa statistics. Histopathology (90%) and CT follow-up (10%) were used as reference standard. MRCs (P < 0.001), calcification patterns (p = 0.003), and pleural retraction (P < 0.001) were all statistically significantly associated to malignancy. Reproducibility was moderate to substantial. Sensitivity, specificity and overall diagnostic accuracy of HRCT were 98%, 23% and 87%, respectively. Reproducibility was substantial.
The first part of the study focused on morphology, assigned MRCs to the solid nodules and found a highly significant association between MRCs and malignancy with increasing positive likelihood ratios for malignancy corresponding to increasing margin risk category. Reproducibility of this measure was substantial. Next there was a significant association between calcification patterns and malignancy with substantial reproducibility. With regard to other characteristics, pleural retraction was a highly significant predictor of malignancy with a positive likelihood ratio of two and moderate reproducibility. No other characteristics proved useful as a predictor of malignancy.
In the second part of the study an integrative approach was applied. A 5-category MPR was assigned to all included nodules and a highly significantly association between MPRs and malignancy was established, with increasing positive likelihood ratios for malignancy corresponding to increasing ratings, and with a very high sensitivity and a somewhat more fair specificity for malignancy. Overall diagnostic accuracy was high reflecting a good if not perfect test, and reproducibility of the MPR was substantial.
In a previous study researchers cross-tabulated six different margin characteristics categories with reference standard and came to highly significant results (7), which have, however, later been questioned (9). Inspired by these studies we organized solid nodules according to patient risk, and added kappa statistics. On that basis the highly significant test results were reproduced. At the same time reproducibility was clinically acceptable.
Some calcification patterns have traditionally been associated with either benignity or malignancy (3, 6), although at least one report has contradicted this (8). It was interesting, that based on ‘known’ patterns of either benignity or malignancy, a strong association between calcifications and malignancy was found, but due to the limited sample size reproducibility became uncertain.
Pleural retraction was highly significantly associated with malignancy in close accordance with a previous publication (8). Although the vessel sign was also highly significantly associated to malignancy, the interobserver reproducibility by two experienced radiologists was only fair. It is important to acknowledge that even though a measurement parameter may come up with a significant result such result is useless if the reproducibility is poor. The remaining other characteristics were non-significant, non-reproducible or too hampered by limited sample size to be of any use.
This study focused on the validity of HRCT morphologic features in the initial characterization of SPNs. If contrast enhancement and evaluation of lymph nodes had been included our results would probably be different (Figs. 5 and 6). We are aware that morphologic features do not stand alone in the radiological work-up of SPNs. However, while certain morphological features will alert radiologists other, such as fat inclusion, may be considered benign. Although to our knowledge no previous publications have dealt with the diagnostic accuracy of HRCT, in at least one publication ROC methodology was applied to compute sensitivity and specificity. The authors of this publication ended up with results similar to ours (8).
(a–d) Four out of a total of seven SPNs falsely classified as ‘definitely malignant’. The readers characterized the first three nodules as ‘spiculated’ and the fourth nodule as ‘partly solid’. The vessel sign and pleural retraction was present in relation to the first two nodules whereas the bronchus sign, the vessel sign, and pleural retraction was present in relation to the third nodule. All nodules were characterized as ‘definitely malignant’ but whereas the first three nodules pathologically were consistent with inflammation, clinical data and CT follow-up proved the last nodule to be a hemorrhagic lung contusion. The remaining false-positive SPNs showed a multitude of margin characteristics and histopathology was consistent with inflammation. (All: C-425 to W1400)
Two false-negative SPNs imaged by HRCT and MDCT, respectively. (a and b) The readers characterized the first nodule as ‘lobulated’, found no other characteristics, and categorized it as ‘probably benign’. It turned out to be an adenocarcinoma; (c and d) The readers characterized the second nodule as ‘smooth’, and found no other characteristics. The nodule was categorized as ‘probably benign’ but turned out to be a carcinoid tumor. Both tumors did in fact show enhancement on the regular MDCT. Thus, they might have been categorized otherwise, if the readers had had access to these images a forehand. (HRCT: C-425 to W1400. MDCT: C60 to W360)
A number of different approaches are used when confronted with the risk of lung cancer or the discovery of an SPN. Some institutions engage in screening studies, typically applying low dose CT to a large patient population with varying degrees of success (15–17), and debatable benefit for the patients (18).
Also the initial use of F-18-FDG PET/CT for the characterization of SPNs is growing. However, this may be hazardous to the patients. F-18-FDG PET/CT struggles with both high false-positive and high false-negative rates, and whereas false-positive nodules may be considered annoying, false-negatives are detrimental. Thus, whereas inflammatory diseases are often false-positive, small nodules (<15 mm), highly differentiated adenocarcinomas, bronchiolo-alveolar carcinomas, and carcinoid tumors are often false-negative (19).
In daily clinical practice the work-up of an SPN is a challenge, and a regular MDCT scan is often the first examination. When performed as an adjunct to the regular MDCT, HRCT of an SPN is a fast and inexpensive procedure with a low radiation dose and high cost-effectiveness.
MRCs are easy to apply in clinical practice, although margin characterization is not an exact science and certain basic margin categories are hard to discriminate (Fig. 2). Our data show, however, that the clinical impact of these discrepancies is of minor importance, because they ultimately end up in the same MRC. Calcification patterns and pleural retraction are also easy to apply in the daily clinical practice.
The study recommends the use of a 5-category MPR. The prevalence of disease in this non-selected group is comparable to the prevalence in many multidisciplinary teams that deal with lung cancer work-up, and the results are comparable with F-18-FDG-PET and 99mTc-depreotide SPECT (20).
However, an issue still remains. We applied a strategy intended to favor sensitivity, and thus naturally observed a high false-positive rate of 78% (69/89), of which 44 cases were characterized as ‘indeterminate’, 18 cases as ‘probably malignant’ and seven cases as ‘definitely malignant’. Also, we observed a 2% (2/124) false-negative rate, of which both cases were characterized as ‘probably benign’ and none as ‘definitely benign’. This study was performed to test HRCT characterization of SPNs solely, but we are convinced though, that if the readers had had access to patient data, as well as the regular MDCT scans, those numbers would have been lower, and especially there would have been no false-negatives (Figs. 5 and 6).
A specific limitation of our study was the risk of selection bias when all referred patients were examined at the Department of Pulmonology but only some had a chest X-ray performed. Although this could potentially be of some concern, most of the patients were smokers (>90%), and as members of a high-risk group even a normal chest X-ray would not lead to omission of an MDCT. Thus we believe our study population and disease prevalence closely reflect those of any university hospital dealing with the work-up of lung cancer. However, this also means our results cannot be transferred to a screening population.
In conclusion, considering the performance characteristics, HRCT of a solitary pulmonary nodule using morphological features is fast, widely available, and seems to be a very sensitive and accurate method in the work-up of lung cancer, and when combined with MDCT the number of invasive procedures or further imaging in patients with suspected lung cancer can be reduced.