Diagnosis of lymph node metastasis in uterine cervical cancer: usefulness of computer-aided diagnosis with comprehensive evaluation of MR images and clinical findings

Lymph node (LN) status is an important parameter for determining treatment strategy and for predicting the prognosis for patients with uterine cervical cancer (1–4). Although the size criterion, i.e. the short-axis diameter greater than 10 mm, has traditionally been used for diagnosing the LN status on magnetic resonance imaging (MRI), previous studies have shown a wide range of sensitivity (30–72%) (5–13); the sensitivity was as low as 30% in a study which strictly applied lesion-by-lesion analysis and was 55% in a recent study (14). PET-CT is also limited as its sensitivity was 51% for overall LNs (15) and as low as 17% for LNs less than 4 mm or less in diameter (16).

The apparent diffusion coefficient (ADC) determined on diffusion-weighted imaging (DWI), which reflects the random thermal motion of protons, has been recently applied to oncologic imaging as it was suggested that cancerous tissue with high cell densities and abundant intra- and intercellular membranes restricts water diffusion. In terms of differentiating metastatic from non-metastatic LNs in uterine cervical cancer, investigators have shown the superiority of ADC to the use of size criteria (17–19).

Despite the improvement of the diagnostic performance of MR by using ADC for detecting pelvic metastatic LNs in patients with uterine cervical cancer, there is a considerable overlap of the ADC in metastatic and non-metastatic LNs, which may cause inaccurate diagnosis. This potential limitation may be related to the restricted ability of ADC to differentiate malignant from benign tissue as ADC can only indicate water diffusion in tissue and this is not a cancer-specific parameter. Therefore, we suggest that the combination of MR and clinical findings, which are related to the pelvic lymph node status in uterine cervical cancer, can overcome this limitation of ADC as previous study showed improved diagnostic accuracy by comprehensive evaluation of MR and clinical findings (20–25).

In this regard, the computer-aided diagnosis (CAD) system is anticipated to improve the diagnostic performance for detecting pelvic LN metastasis as CAD uses an out-performing artificial intelligence for medical decision-making based on the comprehensive evaluation of various MR and clinical findings (26–28).

The purpose of our study was to estimate the value of the Bayesian classifier-based CAD system in helping to achieve a comprehensive evaluation of the MR and clinical findings for obtaining the diagnosis of pelvic LN metastasis in patients with uterine cervical cancer.

Material and Methods

This retrospective study was approved by our institutional review board for human investigation, and informed consent was waived.

Patients

The primary criteria for patient enrollment in our study were as follows: (a) each patient had undergone hysterectomy and lymphadenectomy for biopsy-proven uterine cervical cancer and had undergone MR examination within 20 days before lymphadenectomy between January 2005 and March 2009, at our institution; (b) the surgical maps which depicted the location of the dissected LNs, were available for each patient; and (c) none of the patients had undergone medical or radiation treatment for uterine cervical cancer before surgery or had another malignancy at the time of their participation in this study. According to these criteria, a total of 143 patients (mean age ± SD 48 years ±11, range 24–73 years) were included.

MR examination

All MR examinations were performed on a 1.5-T unit (Gyroscan Intera, Philips Medical System, Best, The Netherlands) with a four-element body coil (Philips Medical System, Best, The Netherlands). Although the MR acquisition protocol was changed during the study period, our basic protocols for transverse, sagittal, and coronal T2-weighted fast spin-echo images and DWI covering the pelvis using phased array coil were constant. T2-weighted images were obtained with the following parameters: TR/TE (msec) 3100–3400 /90–100, slice thickness 4 mm, interslice gap 1 mm, field of view 18–22 cm, matrix 256 × 256 which was interpolated to 512 × 512, echo-train length 32, and number of acquisitions 4.

Transverse DWIs using a single-shot, spin-echo, echo planar imaging sequence were obtained during free breathing using the following imaging parameters: repetition time of 8500–8800 ms; echo time of 69 ms; number of excitations, four; field of view of 360 × 360 mm; matrix of 256 × 256; sensitivity encoding factor, two; echo planar imaging factor of 87; slice thickness 4 mm without the interslice gap; and b values of 0 and 1000 s/mm². Fat suppression was applied using a short TI inversion recovery technique with an inversion time of 160 msec. Images included the lower abdomen and entire pelvis with an acquisition time of 9–10 min.

Node-by-node correlation between MRI and histology

Pelvic LN stations were classified as left or right internal (including obturator and parametrial), external, and common iliac stations and were constant for both MRI and the surgical map (5). The consensus and accurate assignment for the pelvic LN station were made by a weekly panel discussion in which the radiologist, gynecologic surgeon, and pathologist participated. The histologic report included the number of metastatic LNs of all the harvested LNs in each station.

To identify LNs concordant between the histologic report and MRI, the following strategy was used: (a) when there was no metastatic LN in a station, the largest LN was included; (b) when there were metastatic LNs in a station, only metastatic LNs were included from the station; (c) when a metastatic LN was noted in the histologic report but not on MRI, this node was considered to be a false-negative LN; (d) when a non-metastatic LN was noted in the surgical field but not on MRI or when a non-metastatic LN was too small to allow measurement of ADC, this LN was excluded from the data analysis; and (e) when a LN was noted on MRI but not in the histologic report or in the surgical field, this LN was also excluded from the data analysis.

MR and clinical findings

Lymph nodes

Two board-certificated radiologists with 5 and 10 years of clinical experience with pelvic MR imaging, measured the mean ADC, maximum ADC, minimum ADC, and short-axis and long-axis diameters of pelvic LNs in a consensus fashion. ADC was measured using commercial software (Diffusion lab, Clinical imaging solution, Seoul, Korea) which displays both DWI and ADC maps. A free-hand region of interest (ROI) was drawn on the ADC maps as large as possible within the limits of the LN margins. When drawing an ROI on the DWI, the software automatically registered the location of the ROI such that its exact location was consistent across all images with different b values. From each ROI, the mean, maximum, and minimum ADC were measured. The short-axis and long-axis diameters of each LN were measured on transverse, sagittal, and coronal T2WIs.

Primary tumor

The presence or absence of parametrial and vaginal invasion, the tumor volume, and the greatest tumor diameter have been known to be related to the lymph node status in patients with uterine cervical cancer (29–33). The parametrial and vaginal invasion were evaluated on T2WI according to the following criteria (34): (a) parametrial invasion was indicated when T2WI showed disruption of the hypointense stroma and there was tumor protrusion through the disrupted stroma into the parametrium; (b) when the tumor protruded through the defect of the hypointense thin vaginal fornix and parametrial invasion was also indicated; and (c) vaginal invasion was diagnosed when disruption of a low-signal-intensity vagina or a thick hyperintense vagina was identified on T2WI.

Tumor volume was estimated by free-hand ROI drawing of the entire volume of the tumor; the greatest tumor diameter was also measured.

Clinical findings

Clinical findings including patient age, serum squamous cell carcinoma antigen, hemoglobin, the presence of human papilloma virus infection, cancer cell type, and cancer cell differentiation were taken from the patients' medical records. All of these parameters were proven to be related to lymph node status in patients with uterine cervical cancer (29–33).

CAD system using the Bayesian classifier

For CAD regarding the probability of LN metastasis, the Bayesian classifier was used. A detailed explanation of the Bayesian classifier is presented in Appendix 1 (to be found online only). The flow of our study design in terms of the Bayesian classifier is illustrated in Fig. 1.

OPEN IN VIEWER

Fig. 1

Flow diagram showing study design

Based on the training set of cases that included all of the LNs, primary tumors, and clinical findings as well as the final pathology diagnosis of each LN (metastatic or non-metastatic), the Bayesian classifier was implemented. A sequential forward selection method was used in order to eliminate irrelevant features. To assess the cross-validation when selecting features, a leave-one-out method was used which split the data into a training set (n = N – 1) and a test set (n = 1) that mutually exclusively and alternatively repeated the learning process until all data had been tested.

Statistical analysis

In order to determine the incremental value of CAD to the MR findings alone (ADC and size criteria) in the diagnosis of LN metastasis in uterine cervical cancer, receiver-operating characteristics (ROC) analysis was performed. Using the t-test, the statistically contributable findings used to differentiate metastatic and non-metastatic LNs, were selected.

The areas under the ROC curves (Az) of the selected findings and the CAD system were then compared using the Z-test. ROC study was performed using Medcalc for Windows (version 7.4.1.0; MedCalc Software; Mariakierke, Belgium). From the ROC curves of the selected findings and the CAD system, the optimal cut-off point which showed the best separation (minimal false-negative and false-positive results) for the diagnosis of LN metastasis in uterine cervical cancer, was extracted and used as a criterion for differentiating metastatic from non-metastatic LNs. The sensitivity, specificity, and overall accuracy were then compared between the CAD system and selected findings according to the Generalized Estimating Equations by using SPSS software (SPSS for Windows, version 15.0, 2006; SPSS, Chicago, IL, USA). In every statistical analysis, a P value less than 0.05 was considered to indicate a significant difference.

Results

There were 680 LNs in 143 patients, and a total of 70 metastatic LNs were noted in 34 of these patients. Single metastatic LN per patient was noted in 16 patients, two metastatic LNs in seven, three metastatic LNs in eight, four metastatic LNs in two, and eight metastatic LNs in one patient. Twenty-four metastatic LNs were identified in the right internal iliac station, 21 in the left internal iliac station, 12 in the left external iliac station, six in the right external iliac station, four in the left common iliac station, and three in the right common iliac station. As there was no metastatic LN detected by surgery but not noted on MRI, all metastatic LNs were therefore included in the data analysis.

The mean values of the mean ADC (0.839 × 10⁻³ vs. 1.022 × 10⁻³), maximum ADC (0.918 × 10⁻³ vs. 1.105 × 10⁻³), and minimum ADC (0.770 × 10⁻³ vs. 0.942 × 10⁻³) were lower in metastatic LNs than in non-metastatic LNs (P < 0.0001 for all comparisons). The mean values of both short- (6.69 mm vs. 4.11 mm) and long-axis diameters (10.56 mm vs. 7.65 mm) were greater in metastatic LNs than in non-metastatic LNs (P < 0.0001 for all comparisons).

The mean value of the tumor volume (13.79 cm³ vs. 4.29 cm³), tumor diameter (3.25 cm vs. 1.87 cm), and SCC-Ag (5.87 ng/mL vs. 1.31 ng/mL) were greater in patients with metastatic LNs than in those without metastatic LNs (P < 0.0001 for all comparisons). Parametrial and vaginal invasion were more frequently noted in patients with metastatic LNs than in those without metastatic LNs (P = 0.0001 for the parametrial invasion and P = 0.0058 for the vaginal invasion). The frequency of cell type, cell differentiation, hemoglobin, and patient age were not different in patients with and without LN metastasis (P > 0.05).

The Bayesian classifier from the training set extracted nine significant parameters which significantly influenced the LN status, including the mean ADC, short-axis and long-axis diameters, tumor volume, parametrial and vaginal invasion, cancer cell type, cancer cell differentiation, and human papilloma virus infection. Therefore, these nine parameters were used to determine the LN status by CAD which then calculated the probability of metastasis in each LN.

The Az values of CAD, the mean ADC, the maximum ADC, the minimum ADC, and the short-axis and long axis diameters are shown in Table 1. The Az value of CAD is greater than those of the other parameters (P = 0.006 for the CAD vs. the mean ADC; P < 0.0001 for the CAD vs. the maximum ADC; P = 0.004 for the CAD vs. the minimum ADC; P = 0.026 for the CAD vs. short-axis diameter; and P < 0.0001 for the CAD vs. long-axis diameter). The mean ADC and short-axis diameter showed greater Az values than the maximum ADC and long-axis diameter (P < 0.05). The Az value did not differ between the mean and minimum ADC or between the short-axis diameter and minimum ADC (P > 0.05) (Fig. 2).

OPEN IN VIEWER

Fig. 2

Receiver-operating-characteristics curves for CAD, mean ADC, and minimum ADC, and the short-axis lymph node diameter. CAD = computer-aided diagnosis, ADC = apparent diffusion coefficient

OPEN IN VIEWER

Table 1

Az value of CAD, mean ADC, maximum ADC, and minimum ADC, and the short- and long-axis lymph node diameters

	Az value (95% CI)
CAD	0.924 (0.901–0.943)
Mean ADC	0.854 (0.826–0.880)
Max ADC	0.827 (0.796–0.854)
Min ADC	0.849 (0.820–0.875)
Short-axis diameter	0.856 (0.827–0.881)
Long-axis diameter	0.753 (0.719–0.785)

CAD = computer-aided diagnosis, Max = maximum, Min = minimum, ADC = apparent diffusion coefficient

According to the ROC curve, the optimal cut-off values for differentiating metastatic from non-metastatic LNs included 0.19 for the probability of CAD using the Bayesian classifier, 0.911 × 10⁻³ for the mean ADC, 0.970 × 10⁻³ for the maximum ADC, 0.860 × 10⁻³ for the minimum ADC, 4.42 mm for the short-axis diameter, and 7.61 mm for the long-axis diameter. We then used these values to determine the LN status.

The sensitivity, specificity, and accuracy of CAD, the mean ADC, the maximum ADC, the minimum ADC, and the short-axis and long-axis diameters are shown in Table 2. The specificity and accuracy of CAD (86%, 86%) were greater than those of the mean ADC (77%, 77%), maximum ADC (77%, 77%), minimum ADC (68%, 70%), and the short-axis diameter (65%, 67%) (P < 0.05). The sensitivity did not differ in CAD (84%), the mean ADC (83%), maximum ADC (79%), minimum ADC (84%), the short-axis diameter (91%) or the long-axis diameter (81%) (P > 0.05) (Figs. 3–5).

OPEN IN VIEWER

Fig. 3

A 53-year-old female patient with uterine cervical cancer without metastatic lymphadenopathy. Transverse T2-weighted image (a), DWI with a b value of 1000 s/mm² (b), and color mapping of ADC (c) show an enlarged lymph node in the left obturator/internal iliac station. The short- and long-axis diameters of this lymph node were 8.6 mm and 18.3 mm, respectively. The mean ADC of this lymph node was 0.778 × 10⁻³ mm²/s. The probability of metastasis as determined by CAD was 0.01. The volume and diameter of the uterine cervical cancer were 0.9 cc and 1.2 cm, respectively. There was no parametrial or vaginal invasion noted on the T2-weighted image (not shown). According to the criteria of each parameter, this lymph node was misdiagnosed as a metastasis according to the mean ADC and size-based criteria. However, this lymph node was determined by CAD to be a non-metastasis

OPEN IN VIEWER

Fig. 4

A 41-year-old female patient with uterine cervical cancer with a metastatic lymph node in the left obturator/internal iliac station. A transverse T2-weighted image (a), DWI with a b value of 1000 s/mm² (b), and color mapping of ADC (c) show an enlarged lymph node in the left obturator/internal iliac station. The short- and long-axis diameters of this lymph node were 13.37 mm and 16.53 mm, respectively, The mean ADC of this lymph node was 0.794 × 10⁻³ mm²/s. The probability of metastasis as determined by CAD was 1.0. The volume and diameter of the uterine cervical cancer were 52.8 cc and 4.2 cm, respectively. There was vaginal invasion without parametrial invasion noted on the T2-weighted image (not shown). According to the criteria of each parameter, this lymph node was determined to be metastasis according to the mean ADC, size-based criteria, and CAD

OPEN IN VIEWER

Fig. 5

A 42-year-old female patient with uterine cervical cancer with a metastatic lymph node in the right obturator/internal iliac station. A transverse T2-weighted image (a), DWI with a b value of 1,000 s/mm² (b), and color mapping of ADC (c) show an enlarged lymph node in the right obturator/internal iliac station. The short- and long-axis diameters of this lymph node were 15.6 mm and 24.8 mm, respectively. The mean ADC of this lymph node was 1.031 × 10⁻³ mm²/s. The probability of metastasis as determined by CAD was 1.0. The volume and diameter of the uterine cervical cancer were 57.36 cc and 5.7 cm, respectively. There was parametrial invasion without vaginal invasion noted on the T2 weighted image (not shown). According to the criteria of each parameter, this lymph node was misdiagnosed to be non-metastasis by mean ADC criteria. However, this lymph node was determined to be metastasis according to CAD and size-based criteria

OPEN IN VIEWER

Table 2

Comparison of the diagnostic performance of CAD and various single parameters of DWI and MRI

	CAD	Mean ADC	Max ADC	Min ADC	SD	LD
True-positive	59	58	55	59	64	57
False-positive	85	143	142	193	216	264
True-negative	525	467	468	417	394	346
False-negative	11	12	15	11	6	13
Sensitivity (%)	84	83	79	84	91	81.4
Specificity (%)	86	77	77	68	65	56.7
Accuracy (%)	86	77	77	70	67	59

CAD = computer-aided diagnosis, Max = maximum, Min = minimum, ADC = apparent diffusion coefficient, SD = short-axis diameter, LD = long-axis diameter

Discussion

Compared with the human decision-making process, a CAD system using artificial intelligence seems to be advantageous as the artificial intelligence can arrive at a more consistent decision when multiple parameters are required for reaching the diagnosis. Decision-making by the human brain, which depends on an observer's experience, may be inconsistent as humans cannot completely remember all previous cases, and various physical or emotional states can interfere with the human decision-making process.

Our study showed a greater Az value with CAD than with each single imaging finding, such as ADC or the LN diameter. Moreover, our CAD system showed greater specificity and accuracy for detecting metastatic LNs than any single imaging finding. We therefore suggest that the improved diagnostic performance using the CAD system results from the benefit of the comprehensive evaluation of all significant parameters. Given this advantage, the CAD system can reduce the negative influence caused by a deviated or abnormally measured parameter in decision-making. For example, in patients with findings suggesting a very low risk of LN metastasis, i.e. small LN diameter, small cancer volume confined to the uterine cervix, and low squamous cell carcinoma antigen, a low ADC alone is not likely to suggest a high probability of LN metastasis. Given this circumstance, application of a single ADC threshold may lead to a false-positive diagnosis. In contrast, the CAD system which considers the other parameters favoring non-metastasis would show a low probability of LN metastasis given the same circumstance. Such a case may be encountered in clinical situations where most diagnoses are made by collecting all related but not confirming indicators for the final diagnosis. In this regard, as consistent decision-making based on comprehensive evaluation of all parameters is necessary, CAD with artificial intelligence would therefore seem to be useful.

In terms of cross-validation, our study used the leave-one-out method as our CAD system was tested using the leave-one-out cross-validation method (35) in which all but one case were used to train the Bayesian classifier which was then applied to the single, excluded case. This procedure was repeated for each case so that each case was left out only once, i.e. used as the test case for a Bayesian classifier trained on the remaining cases. This statistical method has been used in previous CAD-related research and was proven to be valid in limited, available data-sets (36–39).

In studies evaluating the diagnostic accuracy of an imaging modality for detecting metastatic LNs, the accurate correlation of LN location between the surgical and imaging findings is an important issue. Despite the limitations of a retrospective study, our study attempted to correlate the location of LNs noted in the surgical and MR findings according to the consensus and accurate assignment based on a weekly panel discussion in which the radiologist, gynecologic surgeon, and pathologist participated. Moreover, in order to prevent the potential limitations of inaccurate registration between the surgical and MR findings, we applied strict criteria for including LNs in this study. Nevertheless, we admit that complete correlation of LNs in the surgical and imaging findings is hardly possible as small LNs may not be concordantly detected in both the surgical and imaging findings.

Measurement of the ADC may vary according to the observer, as Klerkx et al. recently suggested (40); they noted that the reproducibility for measuring ADC ranged between 42% and 65% in the two observers and in the two imaging tests. We suggest that this limitation not only occurs in ADC measurement but also when measuring the LN diameter. Actually, inter- or intra-observer variation always occurs when measuring image parameters. When considering the measurement of LN diameters, these variations may significantly influence the results. Given that the short-axis diameter of many metastatic LNs may lie within a narrow range of 8–10 mm, even seemingly small, 1-mm alterations in size criteria may lead to significant shifts in sensitivity and specificity. Although the results of CAD system are affected less than that of single measurement, negative effect of inaccurate measurement on diagnostic accuracy cannot be neglected. Moreover, as ADC and diameter are very important factors influencing the probability of LN metastasis, the measurement of these parameters should be carefully performed.

Despite the advantages of the CAD system, potential limitations also exist. When the training set for machine learning inaccurately represents the patient population, the test results based on the training set are inaccurate. Therefore, input of adequate and accurate training data is mandatory. In this regard, our study may potentially be limited as erroneous measurement of the ADC in small LNs on DWI with low spatial resolution may result in an inaccurate training data-set.

In a previous study (14), the Az value of the minimum ADC (0.864) was greater than that of the minimum ADC (0.849) in our study. We assume that different results for the Az value of the minimum ADC may be caused by different methods for measuring the ADC value. We draw as large ROIs as possible within the limits of the LN margins; the previous study manually placed at least five, circular ROIs including five clustering voxels (approximated 8 mm²) and selected the lowest ADC value. Our method with selecting the lowest ADC value in each ROI may be influenced by unexpected lowering of the ADC by non-tissue-related conditions such as image noise or focal field inhomogeneity. In contrast, the methods of the previous study can reduce such an ADC fluctuation in a single voxel with unexpectedly low ADC as it measured the mean ADC value of clustering voxels. However, as manual drawing of five ROIs within an LN is laborious and time-consuming, it is therefore difficult to use in clinical practice.

In the clinical application of DWI for diagnosing metastatic LNs, there has been no consensus regarding the b value. Our study applied b values of 0 and 1000 s/mm². DWI with a b value of 0 s/mm² may be influenced by perfusion effects, thereby potentially reducing the accuracy of the ADC values. However, as lymph nodes generally do not have a large amount of perfusion, such effects should not have significantly impacted our ADC results.

DWI has an inherent limitation with regard to the general application of a specific threshold. As ADC depends on a multitude of factors affecting tissue diffusivity and regional gradient, i.e. body temperature, tissue pressure, perfusion rate, and the magnetic environment of each individual subject (41), the threshold ADC for differentiating metastatic LNs may vary according to patient or MR protocol. Moreover, it may be challenging to achieve reproducible and accurate ADC value from intrinsically small LNs, although our study attempted to draw ROIs as large as possible without including extra nodal tissue.

In conclusion, in our study, CAD which comprehensively evaluated the MR imaging and clinical information using the Bayesian classifier, showed greater specificity and accuracy for differentiating metastatic from non-metastatic pelvic LNs in patients with uterine cervical cancer. Given the value of the comprehensive evaluation of significant findings in order to make optimal clinical decisions, the CAD system can be used successfully for diagnosing metastatic LNs.