Performance evaluation of breast cancer diagnosis with mammography,ultrasonography and magnetic resonance imaging

Abstract

OBJECTIVE:

Various imaging modalities have been used to diagnose suspicious breast lesions. Purpose of this study is to compare the diagnostic accuracy for breast cancer using mammography, ultrasonography and magnetic resonance imaging (MRI).

METHODS:

Total 107 patients aged from 19 to 62 years are included in this retrospective study. Mammography, ultrasonography and MRI scans were performed for each patient detected with suspected breast tumor within a month. In addition, the tumor diversity (10 types of benign and 5 types of malignant) was confirmed by pathological findings of tumor biopsy. To compare the diagnosis performance of the three imaging modalities, the overall fraction correct (accuracy), positive predict value (PPV), negative predict value (NPV), sensitivity and specificity were calculated. Meanwhile, the receiver operating characteristic (ROC) analysis was also performed.

RESULTS:

The diagnostic accuracy ranged from 78.5% to 86.9% among three imaging modalities. All modalities yielded a PPV lower than 77.8% and a NPV higher than 90.0% in identifying the presence of malignant tumors. MRI presented a diagnostic accuracy of 86.9%, as well as a sensitivity of 95.5% and an area under curve (AUC) of 0.948, which are higher than mammography and ultrasonography.

CONCLUSION:

By using a diverse dataset and comparing the diagnostic accuracy of three imaging modalities commonly used in breast cancer detection and diagnosis, this study also demonstrated that mammography, ultrasonography and MRI had different diagnostic performance in breast tumor identification. Among them, MRI yielded the highest performance even though the unexpected specificity may lead to over-diagnosis, and ultrosonography is slightly better than mammography.

Keywords

Mammography ultrasonography breast magnetic resonance imaging (MRI)breast cancer diagnosis diagnostic performance assessment

1 Introduction

The incidence of breast cancer has increased globally in women over the last several decades. Despite progress in identifying risk factors and genetic markers for breast cancer, most cases occur without known major predictors [1, 2]. Tumour stage at detection is the key factor for survival. Breast imaging has been recommended as a method of early diagnosis and confer benefits of reducing breast cancer mortality in the population [3, 4]. Mammography, ultrasonography and MRI have been widely utilized as noninvasive diagnostic imaging in clinical practice for evaluation of breast lesions.

Accurate classification using medical images not only helps the physicians make the appropriate clinical decision on further examination and treatment, but also can reduce the economic burden of patients. Performance evaluation of the three modalities can provide references for radiologists in clinical diagnosis [5, 6]. Hongda Shao et al. evaluated the three modalities performance and MRI was best with a sensitivity of 90.9%, a specificity of 82.7%, a PPV of 89.3%, a NPV of 85.3 and AUC of 0.88 [6]. Juan Wang et al. proved MRI sensitivity (87.2%) were significantly higher than mammography (76.9%) or ultrasonography (82.1) and MRI can detect most of small breast tumor (< = 2 cm) [7]. Berg WA, et al. compared the advantages and disadvantages of three modalities in diagnosing different types of breast diseases [8]. Strobel K, et al. calculated parameters to evaluated the three modalities performance as BI-RADS category 4 and gave the results that MRI was useful for the noninvasive work-up of lesions classified at mammography or ultrasonography [9, 10].

In this article, we conducted a systematic comparative cohort study, investigating the diagnostic accuracy, PPV, NPV, sensitivity, specificity, AUC and ROC space of mammography, ultrasonography and MRI in the diagnosis of breast lesions. Through our study, the diagnosis performance of clinical suspected breast tumor by using these three modalities will be quantitatively evaluated. Moreover, our dataset has the following characteristics comparing to the previous works. Firstly, the age range of patients in our study is wide, from 19 to 62. Secondly, the dataset includes as many as 15 types of breast diseases, i.e., breast adenosis, mastitis, abscess, granulomatous lobular mastitis(GLM), plasma cell mastitis, fibroadenoma, phyllodes tumors, hyperplasia, intraductal papilloma, epidermal cysts, invasive ductal carcinoma (IDC), ductal carcinoma in situ (DCIS), invasive lobular adenocarcinoma, tubular carcinoma, and mucinous adenocarcinoma.

2 Materials and methods

2.1 Participants

The retrospective study was conducted on 107 patients with suspected breast cancer (based on findings such as pain, breast nipple discharge, palpable breast lumps and so on) between September 2010 and July 2017. These data were obtained from cooperative hospital. The patients who had completed the examinations of all three imaging modalities during the period were collected. Statistics show that 90 patients finished the three inspections in a randomized order within one month, and 17 patients finished more than one month. The age of the participants ranged from 19 to 62 (mean 37.3±9.3 years old).

2.2 Imaging methods

Full-field digital mammography including two standard views, mediolateral oblique and craniocaudal had been performed by FDR-3000AWS of FUJIFILM. All images were acquired for each breast through fully automated exposure control. Breast ultrasonography had been performed by voluson e8 of GE. All of the cases had undergone the breast MRI in the device Signa HDxt, GE. A conventional breast MRI protocol was utilized using a standard breast coil in the prone position. Major parameters of the different imaging modalities are shown in Table 1.

Fig.1

Histogram of participants ages statistics.

Table 1

Image acquisition parameters of mammography, ultrasonography and MRI

Modality	Device	Major parameters
mammography	FUJIFILM	KVP 31
	FDR-3000AWS	Field of View Dimension(s) 296/236
		Exposure Time 810
		Tube Current 103
		Image Pixel Spacing 0.05/0.05
ultrasonography	GE	probe: LA523
	voluson e8	frequency: 5–10 MHz
MRI	GE,	T1: TR 360 ms, TE 7.43 ms, section thickness 4.5 mm, FOV 340 mm, matrix 512×512, N_EX 0.5
	Signa HDxt	T2: TR 5080 ms, TE 100.8 ms, section thickness 4.5 mm, FOV 340 mm, matrix 512×512, N_EX 3
		DWI: TR 4000 ms, TE 82.1 ms, section thickness 4.5 mm, FOV 340 mm, matrix 256×256, N_EX 4, b values 800 s/mm²
		MASK
		DCE-MRI (eight sequences,)
		DCE-subtracted images: DCE-MRI subtracting MASK
		SCREENSAVE: increasing, plateau, washout

Imaging assessment score had been given based on BI-RADS criterion. Diagnosis were divided into five categories: 1 (negative), 2 (benign finding), 3 (probably benign finding, short– term sustained monitor), 4A (2–10% malignancy, biopsy recommended), 4B (10–50% malignancy, biopsy recommended), 4C (50–95% malignancy, biopsy recommended) and 5 (highly suggestive of malignancy) [11, 12].

2.3 Data analysis

Common diagnostic parameters such as sensitivity, specificity, accuracy, PPV, NPV, AUC and ROC space values were calculated by using 2×2 contingency tables, which were constructed by comparing diagnostic outcome from three modalities with biopsy result which can be regarded as the golden standard. ROC curve illustrates the performance of a binary classifier model as its discrimination threshold is varied [13]. When describing the ROC curve, we mapped the BI-RADS category into an exact prediction either as positive or negative based on a certain threshold.

Based on BI-RADS criterion, each category corresponds to probability range of benign and malignant (Table 2). According to TP, FP, TN and FN calculation methods, the median of the probability range is multiplied by the number of cases on a certain category, then the calculated values of all categories are added, and we can get the sensitivity and specificity values for different imaging modalities to draw the ROC space.

Table 2
Probability range for the different imaging modalities in the entire cohort

BI-RADS Benign Probability (%) Malignant Probability (%)

category 1–2 100 0

category 3 99 (98–100) 1 (0–2)

category 4A 94 (90–98) 6 (2–10)

category 4B 70 (50–90) 30 (10–50)

category 4C 27.5 (5–50) 72.5 (50–95)

category 5 2.5 (0–5) 97.5 (95–100)

BI-RADS	Benign Probability (%)	Malignant Probability (%)
category 1–2	100	0
category 3	99 (98–100)	1 (0–2)
category 4A	94 (90–98)	6 (2–10)
category 4B	70 (50–90)	30 (10–50)
category 4C	27.5 (5–50)	72.5 (50–95)
category 5	2.5 (0–5)	97.5 (95–100)

B denotes benign breast lesion, M denotes malignant breast lesion.

3 Results

According to the pathological reports, among 107 cases detected with the suspected breast cancer, 63 cases were categorized as benign lesions, including hyperplasia (22), breast adenosis (21), mastitis (6), fibroadenoma (5), GLM (3), phyllodes tumors (2), abscess (1), plasma cell mastitis (1), intraductal papilloma (1), epidermal cysts (1); while 44 cases were categorized as malignant lesions, including IDC (36), DCIS (4), invasive lobular adenocarcinoma (2), tubular carcinoma (1), mucinous adenocarcinoma (1). Diagnosis results of BI-RADS category with mammography, ultrasonography and MRI modalities compared to the pathological diagnosis are shown in Fig. 2.

Fig.2

Comparison between three imaging modalities and pathological diagnosis in 107 cases.

Data analysis of case diagnosis results shows that each modality has its strengths and weaknesses. Figure 3 shows some examples. Mammography occurred misdiagnosis and overdiagnosis on the lesions with extremely dense or heterogeneously dense breasts, which were correctly diagnosed by ultrasonography and MRI. For a 45-year-old woman with adenosis in left breast (Fig. 3(a)), mammography misdiagnosed as BI-RADS category 4B, but ultrasonography and MRI diagnosed as BI-RADS category 2 correctly. Relatively, the second example is a rare case. For a 27-year-old woman with breast adenosis in left breast (Fig. 3(b)), the ultrasonography misdiagnosed as BI-RADS category 4B and 4A, but mammography correctly diagnosed as BI-RADS category 1 because of extremely dense. Study showed that ultrasonography diagnosis of fibroadenoma is better than mammography and MRI. However, ultrasonography may misdiagnose when lesions emerged glandular structural disorder. In the last example (Fig. 3(c)), pathological diagnosis was IDC, ultrasonography deemed the lesion as BI-RADS category 2 because this case was glandular structural disorder, however, MRI diagnosed as BI-RADS category 4A and mammography diagnosed as BI-RADS category 4B.

Fig.3

Row from left to right: medial lateral oblique mammography, craniocaudal mammography, DCE-MRI, ultrasonography. Three cases of each column from top to bottom: (a) the 45-year-old woman with adenosis in left breast; (b) the 27-year-old woman with adenosis in left breast; (c) the 60-year-old woman with IDC in left breast.

To illustrate the diagnosis performance of three modalities more clearly, the number of cases which were diagnosed correctly, misdiagnosis and overdiagnosis by all three modalities, two modalities and only one modality were systematically analyzed (Table 3).

Table 3

Correctly diagnosed, misdiagnosis and over-diagnosis numbers by three imaging modalities

	Correctly diagnosed	Misdiagnosis	Overdiagnosis
mammography	84	3	20
ultrasonography	90	4	13
MRI	96	0	11
mammography &ultrasonography	72	1	2
mammography &MRI	78	0	4
ultrasonography &MRI	81	0	1
mammography &ultrasonography &MRI	69	0	2

The ROC curve and the ROC space of three modalities was shown respectively in Fig. 4. The diagnostic accuracy, PPV, NPV, sensitivity, specificity and AUC values of different imaging methods in breast cancer diagnosis were shown in Table 4. It can be seen that the sensitivity of MRI was significantly higher than mammography and ultrasonography. The AUC value of MRI can reach 0.948, which was higher than that of mammography and ultrasonography. However, the NPV value of MRI was higher than that of mammography and ultrasonography, because the overdiagnosis rate of MRI was the highest.

Fig.4

ROC curve and ROC space of mammography, ultrasound and MRI.

Table 4

Diagnostic value of different imaging modalities in the entire cohort

Imaging methods	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Accuracy (%)	AUC
mammography	88.6	71.4	68.4	90.0	78.5	0.877
ultrasonography	90.9	79.4	75.5	92.6	84.1	0.919
MRI	95.5	81.0	77.8	96.2	86.9	0.948

Note: p < 0.0001 is accepted as statistically significant. All results based on 95% CI.

Figure 4 shows and compares three unsmoothed ROC curves generated using three imaging modalities in classifying between malignant and benign breast tumors. It shows that using MRI yielded highest AUC value, while using mammography generated smallest AUC value. In addition, by applying an operating threshold to three imaging modalities, three performance markers are also plotted in Fig. 4, which show that MRI yields the significantly higher TPR (68.1%) than mammography (53.8%) and ultrasonography (59.7%) in the diagnosis of breast cancer, but the FPR value of MRI (10.0%) is the middle of the three modalities namely, the FPR values are 10.7% and 9.4% for mammography and ultrasonography, respectively.

4 Discussion

Although many new investigative imaging modalities have been developed and tested in breast cancer detection and diagnosis (i.e., different types of CT-guided imaging modalities [14, 15]), mammography remains the standard examination imaging modality used in breast cancer screening of the general population in current clinical practice [16]. However, our study shows that using mammography cannot easily detect non-calcified lesions especially when the lesions are located within extremely dense or heterogeneously dense breast tissue, so misdiagnosis can happen. Relatively, mammography can accurately diagnose when breast tissue belongs to entirely fatty and scattered areas of fibroglandular density.

Ultrasonography has been commonly used as an imaging procedure where mammography and MRI are not available in some clinics. It can identify small nonpalpable masses in dense breast tissue whereas mammography cannot detect it well. Among all of the lesions in our dataset, two cases of benign lesions in dense breasts were misdiagnosed by mammography but correctly diagnosed by ultrasonography. Meanwhile, one case of malignant lesions in dense breasts were misdiagnosed by mammography but correctly diagnosed by ultrasonography. However, the sonogrsphic features of benign and malignant breast lesions have some overlaps, so ultrasonography diagnostic specificity is low [17, 18]. In our dataset, ultrasonography misdiagnosed two cases as malignant which pathological results were benign. In addition, ultrasonography is an operator-dependent technique, so it has never been used as the only modality for breast cancer screening [19 –21].

MRI includes many scan protocols so that it can yield significant improvement in cancer diagnosis. Our results indicated that both the sensitivity and the AUC of MRI were higher than those of mammography and ultrasonography, which is consistent with most previous reported work [22 –24]. MRI has a higher accuracy than mammography and ultrasonography in detecting malignant tumor, but it misdiagnosed as BI-RADS category 4 easily on some benign lesions, such as breast adenosis, phyllodes tumors, hyperplasia and lobular inflammation. In our dataset, twelve cases were overdiagnosed malignant by MRI, but their pathological results were benign. It indicated that MRI reduced the specificity performance. Moreover, high examination cost may limit its use widely [6 , 26].

Comparing with previous work from Hongda Shao et al. [6], Juan Wang, et al. [7], Berg WA, et al. [8] and Strobel K, et al. [9], our study presented more performance evaluation values. Firstly, the patients in our study have a wide range of age, and there were 65 cases under 35 years old. Secondly, our study involved more types of diseases (10 types of benign lesions and 5 types of malignant tumor), which makes the results more representative. Thirdly, our evaluation method was comprehensive, and seven measures were employed. The ROC curve could present the results intuitively and the ROC space was proposed.

From our study, both mammography and MRI have high clinical value in diagnosis of breast tumor. Furthermore, MRI can provide more information and its diagnosis sensitivity is higher than mammography. So MRI, with a higher clinical value, is worth promoting. Ultrasonography plays important role for the early diagnosis of dense breasts. In particular, ultrasonography check-up doesn’t involve radiation. Besides images, radiologists may need additional information like age, family history, menopause and so on to finish their diagnosis. For high-risk populations including the subjects with the family history of breast cancer or suffered from cervical cancer, it suggested that the combined MRI and mammography can be the choice for early detection of breast cancer. Even so, none of ultrasonography, mammography and MRI can replace pathological surgery in the detection of breast carcinoma now, because none of those three modalities reaches the criterion of 100% sensitivity and 100% specificity. That means although measures can be taken to improve the accuracy, the gap between medical imaging methods and pathological golden standard will still exist for some time.

Footnotes

ACKNOWLEDGMENTS

The scientific guarantor of this publication is Prof. Wei Qian. The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article. This study was funded by the National Key Research and Development Program Sub – topics under Grant (No. 2016YFC1303005), the Natural Science Foundation of Liaoning Province of China under Grant (No. 201404186), the Fundamental Research Funds for the Central Universities under Grant (No. N150408001), the National Natural Science Foundation of China under Grant (No.81671773, 61672146). The ethics committee of Shengjing Hospital of China Medical University (Number: 2014PS67K). Methodology: retrospective, diagnostic or prognostic study, performed at one institution.

References

Oeffinger

K.C.

, Fontham

E.T.

, Etzioni

et al., Breast cancer screening for women at average risk: guideline update from the American Cancer Society, JAMA 314(15) (2015), 1599–1614.

Nelson

H.D.

, Pappas

, Cantor

et al., Harms of breast cancer screening: Systematic review to update the US preventive services task force recommendationharms of breast cancer screening, Annals of Internal Medicine 164(4) (2016), 256–267.

Kelly

K.M.

, Dean

, Comulada

W.S.

, Lee

S-J.

Breast cancer detection using automated whole breast ultrasound and mammography in radiographically dense breasts, European Radiology 20(3) (2010), 734–742.

Lord

, Lei

, Craft

et al., A systematic review of the effectiveness of magnetic resonance imaging (MRI) as an addition to mammography and ultrasound in screening young women at high risk of breast cancer, European Journal of Cancer 43(13) (2007), 1905–1917.

Elmore

J.G.

, Longton

G.M.

, Carney

P.A.

et al., Diagnostic concordance among pathologists interpreting breast biopsy specimens, JAMA 313(11) (2015), 1122–1132.

Shao

, Li

, Zhang

et al., Comparison of the diagnostic efficiency for breast cancer in Chinese women using mammography, ultrasound, MRI, and different combinations of these imaging modalities, Journal of X-ray Science and Technology 21(2) (2013), 283–292.

Wang

, Chen

, Wu

, Tang

Comparison of Diagnostic Efficiency of Breast Cancer Imaging in Chinese Women: Digital Mammography, Ultrasound, MRI, and Combinations of These Modalities, Journal of Medical Imaging & Health Informatics 5(7) (2015).

Berg

W.A.

, Gutierrez

, Nessaiver

M.S.

et al., Diagnostic accuracy of mammography, clinical examination, US, and MR imaging in preoperative assessment of breast cancer, Radiology 233(3) (2004), 830–849.

Strobel

, Schrading

, Hansen

N.L.

et al., Assessment of BI-RADS category 4 lesions detected with screening mammography and screening US: Utility of MR imaging, Radiology 274(2) (2015), 343.

10.

Yang

Q.X.

, Ji

, Feng

L.L.

et al., Significant MRI indicators of malignancy for breast non-mass enhancement, Journal of X-ray Science and Technology 25(6) (2017), 1033–1044.

11.

Yabuuchi

, Matsuo

, Okafuji

et al., Enhanced mass on contrast-enhanced breast MR imaging: Lesion characterization using combination of dynamic contrast-enhanced and diffusion-weighted MR images, Journal of Magnetic Resonance Imaging 28(5) (2008), 1157–1165.

12.

Collettini

, Martin

, Diekmann

et al., Diagnostic performance of a Near-Infrared Breast Imaging system as adjunct to mammography versus X-ray mammography alone, European Radiology 22(2) (2012), 350–357.

13.

Fawcett

An introduction to ROC analysis, Pattern Recognition Letters 27(8) (2006), 861–874.

14.

Baikejiang

, Zhang

, Li

Diffuse optical tomography for breast cancer imaging guided by computed tomography: A feasibility study, Journal of X-ray Science and Technology 25(3) (2017), 341–355.

15.

Shah

J.P.

, Mann

S.D.

, Tornai

M.P.

Characterization of X-ray scattering for various phantoms and clinical breast geometries using breast CT on a dedicated hybrid system, Journal of X-ray Science and Technology 25(3) (2017), 373–389.

16.

Le-Petross

H.T.

, Shetty

M.K.

Magnetic resonance imaging and breast ultrasonography as an adjunct to mammographic screening in high-risk patients, Seminars in Ultrasound, CT and MRI. 32. Elsevier; (2011), 266–272.

17.

Nothacker

, Duda

, Hahn

et al., Early detection of breast cancer: Benefits and risks of supplemental breast ultrasound in asymptomatic women with mammographically dense breast tissue. A systematic review, BMC Cancer 9(1) (2009), 335.

18.

Sohn

Y-M.

, Kim

M.J.

, Kwak

J.Y.

et al., Breast ultrasonography in young Asian women: Analyses of BI-RADS final assessment category according to symptoms, Acta Radiologica 52(1) (2011), 35–40.

19.

Zonderland

H.M.

, Coerkamp

E.G.

, Hermans

et al., Diagnosis of breast cancer: Contribution of US as an adjunct to mammography, Radiology 213(2) (1999), 413–422.

20.

Hwang

K-H.

, Lee

J.G.

, Kim

J.H.

et al., Computer aided diagnosis (CAD) of breast mass on ultrasonography and scintimammography. Enterprise networking and Computing in Healthcare Industry, 2005. HEALTHCOM 2005. Proceedings of 7th International Workshop on. IEEE; (2005), 187–189.

21.

Guo

, Cheng

, Huang

et al., Breast ultrasound image enhancement using fuzzy logic, Ultrasound in Medicine & Biology 32(2) (2006), 237–247.

22.

Mann

R.M.

, Kuhl

C.K.

, Kinkel

, Boetes

Breast MRI: Guidelines from the European society of breast imaging, European Radiology 18(7) (2008), 1307–1318.

23.

Warner

, Messersmith

, Causer

et al., Systematic Review: Using Magnetic Resonance Imaging to Screen Women at High Risk for Breast CancerUsing MRI to Screen Women at High Risk for Breast Cancer. Annals of Internal Medicine 148(9) (2008), 671–679.

24.

King

, Brooks

J.D.

, Bernstein

J.L.

et al., Background parenchymal enhancement at breast MR imaging and breast cancer risk, Radiology 260(1) (2011), 50–60.

25.

Weinstein

S.P.

, Localio

A.R.

, Conant

E.F.

et al., Multimodality screening of high-risk women: A prospective cohort study, Journal of Clinical Oncology 27(36) (2009), 6124–6128.

26.

Cheng

Y-C.

, Wu

N-Y.

, Ko

J.S.

et al., Breast cancers detected by breast MRI screening and ultrasound in asymptomatic Asian women: 8 years of experience in Taiwan, Oncology 82(2) (2012), 98–107.