Performance evaluation of breast cancer diagnosis with mammography,ultrasonography and magnetic resonance imaging

1 Introduction

The incidence of breast cancer has increased globally in women over the last several decades. Despite progress in identifying risk factors and genetic markers for breast cancer, most cases occur without known major predictors [1, 2]. Tumour stage at detection is the key factor for survival. Breast imaging has been recommended as a method of early diagnosis and confer benefits of reducing breast cancer mortality in the population [3, 4]. Mammography, ultrasonography and MRI have been widely utilized as noninvasive diagnostic imaging in clinical practice for evaluation of breast lesions.

Accurate classification using medical images not only helps the physicians make the appropriate clinical decision on further examination and treatment, but also can reduce the economic burden of patients. Performance evaluation of the three modalities can provide references for radiologists in clinical diagnosis [5, 6]. Hongda Shao et al. evaluated the three modalities performance and MRI was best with a sensitivity of 90.9%, a specificity of 82.7%, a PPV of 89.3%, a NPV of 85.3 and AUC of 0.88 [6]. Juan Wang et al. proved MRI sensitivity (87.2%) were significantly higher than mammography (76.9%) or ultrasonography (82.1) and MRI can detect most of small breast tumor (< = 2 cm) [7]. Berg WA, et al. compared the advantages and disadvantages of three modalities in diagnosing different types of breast diseases [8]. Strobel K, et al. calculated parameters to evaluated the three modalities performance as BI-RADS category 4 and gave the results that MRI was useful for the noninvasive work-up of lesions classified at mammography or ultrasonography [9, 10].

In this article, we conducted a systematic comparative cohort study, investigating the diagnostic accuracy, PPV, NPV, sensitivity, specificity, AUC and ROC space of mammography, ultrasonography and MRI in the diagnosis of breast lesions. Through our study, the diagnosis performance of clinical suspected breast tumor by using these three modalities will be quantitatively evaluated. Moreover, our dataset has the following characteristics comparing to the previous works. Firstly, the age range of patients in our study is wide, from 19 to 62. Secondly, the dataset includes as many as 15 types of breast diseases, i.e., breast adenosis, mastitis, abscess, granulomatous lobular mastitis(GLM), plasma cell mastitis, fibroadenoma, phyllodes tumors, hyperplasia, intraductal papilloma, epidermal cysts, invasive ductal carcinoma (IDC), ductal carcinoma in situ (DCIS), invasive lobular adenocarcinoma, tubular carcinoma, and mucinous adenocarcinoma.

2 Materials and methods

2.2 Imaging methods

Full-field digital mammography including two standard views, mediolateral oblique and craniocaudal had been performed by FDR-3000AWS of FUJIFILM. All images were acquired for each breast through fully automated exposure control. Breast ultrasonography had been performed by voluson e8 of GE. All of the cases had undergone the breast MRI in the device Signa HDxt, GE. A conventional breast MRI protocol was utilized using a standard breast coil in the prone position. Major parameters of the different imaging modalities are shown in Table 1.

OPEN IN VIEWER

Fig.1

Histogram of participants ages statistics.

OPEN IN VIEWER

Table 1

Image acquisition parameters of mammography, ultrasonography and MRI

Modality	Device	Major parameters
mammography	FUJIFILM	KVP 31
	FDR-3000AWS	Field of View Dimension(s) 296/236
		Exposure Time 810
		Tube Current 103
		Image Pixel Spacing 0.05/0.05
ultrasonography	GE	probe: LA523
	voluson e8	frequency: 5–10 MHz
MRI	GE,	T1: TR 360 ms, TE 7.43 ms, section thickness 4.5 mm, FOV 340 mm, matrix 512×512, NEX 0.5
	Signa HDxt	T2: TR 5080 ms, TE 100.8 ms, section thickness 4.5 mm, FOV 340 mm, matrix 512×512, NEX 3
		DWI: TR 4000 ms, TE 82.1 ms, section thickness 4.5 mm, FOV 340 mm, matrix 256×256, NEX 4, b values 800 s/mm2
		MASK
		DCE-MRI (eight sequences,)
		DCE-subtracted images: DCE-MRI subtracting MASK
		SCREENSAVE: increasing, plateau, washout

Imaging assessment score had been given based on BI-RADS criterion. Diagnosis were divided into five categories: 1 (negative), 2 (benign finding), 3 (probably benign finding, short– term sustained monitor), 4A (2–10% malignancy, biopsy recommended), 4B (10–50% malignancy, biopsy recommended), 4C (50–95% malignancy, biopsy recommended) and 5 (highly suggestive of malignancy) [11, 12].

According to the pathological reports, among 107 cases detected with the suspected breast cancer, 63 cases were categorized as benign lesions, including hyperplasia (22), breast adenosis (21), mastitis (6), fibroadenoma (5), GLM (3), phyllodes tumors (2), abscess (1), plasma cell mastitis (1), intraductal papilloma (1), epidermal cysts (1); while 44 cases were categorized as malignant lesions, including IDC (36), DCIS (4), invasive lobular adenocarcinoma (2), tubular carcinoma (1), mucinous adenocarcinoma (1). Diagnosis results of BI-RADS category with mammography, ultrasonography and MRI modalities compared to the pathological diagnosis are shown in Fig. 2.

OPEN IN VIEWER

Fig.2

Comparison between three imaging modalities and pathological diagnosis in 107 cases.

Data analysis of case diagnosis results shows that each modality has its strengths and weaknesses. Figure 3 shows some examples. Mammography occurred misdiagnosis and overdiagnosis on the lesions with extremely dense or heterogeneously dense breasts, which were correctly diagnosed by ultrasonography and MRI. For a 45-year-old woman with adenosis in left breast (Fig. 3(a)), mammography misdiagnosed as BI-RADS category 4B, but ultrasonography and MRI diagnosed as BI-RADS category 2 correctly. Relatively, the second example is a rare case. For a 27-year-old woman with breast adenosis in left breast (Fig. 3(b)), the ultrasonography misdiagnosed as BI-RADS category 4B and 4A, but mammography correctly diagnosed as BI-RADS category 1 because of extremely dense. Study showed that ultrasonography diagnosis of fibroadenoma is better than mammography and MRI. However, ultrasonography may misdiagnose when lesions emerged glandular structural disorder. In the last example (Fig. 3(c)), pathological diagnosis was IDC, ultrasonography deemed the lesion as BI-RADS category 2 because this case was glandular structural disorder, however, MRI diagnosed as BI-RADS category 4A and mammography diagnosed as BI-RADS category 4B.

OPEN IN VIEWER

Fig.3

Row from left to right: medial lateral oblique mammography, craniocaudal mammography, DCE-MRI, ultrasonography. Three cases of each column from top to bottom: (a) the 45-year-old woman with adenosis in left breast; (b) the 27-year-old woman with adenosis in left breast; (c) the 60-year-old woman with IDC in left breast.

To illustrate the diagnosis performance of three modalities more clearly, the number of cases which were diagnosed correctly, misdiagnosis and overdiagnosis by all three modalities, two modalities and only one modality were systematically analyzed (Table 3).

The ROC curve and the ROC space of three modalities was shown respectively in Fig. 4. The diagnostic accuracy, PPV, NPV, sensitivity, specificity and AUC values of different imaging methods in breast cancer diagnosis were shown in Table 4. It can be seen that the sensitivity of MRI was significantly higher than mammography and ultrasonography. The AUC value of MRI can reach 0.948, which was higher than that of mammography and ultrasonography. However, the NPV value of MRI was higher than that of mammography and ultrasonography, because the overdiagnosis rate of MRI was the highest.

OPEN IN VIEWER

Fig.4

ROC curve and ROC space of mammography, ultrasound and MRI.

Figure 4 shows and compares three unsmoothed ROC curves generated using three imaging modalities in classifying between malignant and benign breast tumors. It shows that using MRI yielded highest AUC value, while using mammography generated smallest AUC value. In addition, by applying an operating threshold to three imaging modalities, three performance markers are also plotted in Fig. 4, which show that MRI yields the significantly higher TPR (68.1%) than mammography (53.8%) and ultrasonography (59.7%) in the diagnosis of breast cancer, but the FPR value of MRI (10.0%) is the middle of the three modalities namely, the FPR values are 10.7% and 9.4% for mammography and ultrasonography, respectively.

Although many new investigative imaging modalities have been developed and tested in breast cancer detection and diagnosis (i.e., different types of CT-guided imaging modalities [14, 15]), mammography remains the standard examination imaging modality used in breast cancer screening of the general population in current clinical practice [16]. However, our study shows that using mammography cannot easily detect non-calcified lesions especially when the lesions are located within extremely dense or heterogeneously dense breast tissue, so misdiagnosis can happen. Relatively, mammography can accurately diagnose when breast tissue belongs to entirely fatty and scattered areas of fibroglandular density.

Ultrasonography has been commonly used as an imaging procedure where mammography and MRI are not available in some clinics. It can identify small nonpalpable masses in dense breast tissue whereas mammography cannot detect it well. Among all of the lesions in our dataset, two cases of benign lesions in dense breasts were misdiagnosed by mammography but correctly diagnosed by ultrasonography. Meanwhile, one case of malignant lesions in dense breasts were misdiagnosed by mammography but correctly diagnosed by ultrasonography. However, the sonogrsphic features of benign and malignant breast lesions have some overlaps, so ultrasonography diagnostic specificity is low [17, 18]. In our dataset, ultrasonography misdiagnosed two cases as malignant which pathological results were benign. In addition, ultrasonography is an operator-dependent technique, so it has never been used as the only modality for breast cancer screening [19–21].

MRI includes many scan protocols so that it can yield significant improvement in cancer diagnosis. Our results indicated that both the sensitivity and the AUC of MRI were higher than those of mammography and ultrasonography, which is consistent with most previous reported work [22–24]. MRI has a higher accuracy than mammography and ultrasonography in detecting malignant tumor, but it misdiagnosed as BI-RADS category 4 easily on some benign lesions, such as breast adenosis, phyllodes tumors, hyperplasia and lobular inflammation. In our dataset, twelve cases were overdiagnosed malignant by MRI, but their pathological results were benign. It indicated that MRI reduced the specificity performance. Moreover, high examination cost may limit its use widely [6, 26].

Comparing with previous work from Hongda Shao et al. [6], Juan Wang, et al. [7], Berg WA, et al. [8] and Strobel K, et al. [9], our study presented more performance evaluation values. Firstly, the patients in our study have a wide range of age, and there were 65 cases under 35 years old. Secondly, our study involved more types of diseases (10 types of benign lesions and 5 types of malignant tumor), which makes the results more representative. Thirdly, our evaluation method was comprehensive, and seven measures were employed. The ROC curve could present the results intuitively and the ROC space was proposed.

From our study, both mammography and MRI have high clinical value in diagnosis of breast tumor. Furthermore, MRI can provide more information and its diagnosis sensitivity is higher than mammography. So MRI, with a higher clinical value, is worth promoting. Ultrasonography plays important role for the early diagnosis of dense breasts. In particular, ultrasonography check-up doesn’t involve radiation. Besides images, radiologists may need additional information like age, family history, menopause and so on to finish their diagnosis. For high-risk populations including the subjects with the family history of breast cancer or suffered from cervical cancer, it suggested that the combined MRI and mammography can be the choice for early detection of breast cancer. Even so, none of ultrasonography, mammography and MRI can replace pathological surgery in the detection of breast carcinoma now, because none of those three modalities reaches the criterion of 100% sensitivity and 100% specificity. That means although measures can be taken to improve the accuracy, the gap between medical imaging methods and pathological golden standard will still exist for some time.