Exploring value of CT coronary imaging combined with machine-learning methods to predict myocardial ischemia

Abstract

PURPOSE:

To establish a machine-learning (ML) model based on coronary computed tomography angiography (CTA) images for evaluating myocardial ischemia in patients diagnosed with coronary atherosclerosis.

METHODS:

This retrospective analysis includes CTA images acquired from 110 patients. Among them, 58 have myocardial ischemia and 52 have normal myocardial blood supply. The patients are divided into training and test datasets with a ratio 7 : 3. Deep learning model-based CQK software is used to automatically segment myocardium on CTA images and extract texture features. Then, seven ML models are constructed to classify between myocardial ischemia and normal myocardial blood supply cases. Predictive performance and stability of the classifiers are determined by receiver operating characteristic curve with cross validation. The optimal ML model is then validated using an independent test dataset.

RESULTS:

Accuracy and areas under ROC curves (AUC) obtained from the support vector machine with extreme gradient boosting linear method are 0.821 and 0.777, respectively, while accuracy and AUC achieved by the neural network (NN) method are 0.818 and 0.757, respectively. The naive Bayes model yields the highest sensitivity (0.942), and the random forest model yields the highest specificity (0.85). The k-nearest neighbors model yields the lowest accuracy (0.74). Additionally, NN model demonstrates the lowest relative standard deviations (0.16 for accuracy and 0.08 for AUC) indicating the high stability of this model, and its AUC applying to the independent test dataset is 0.72.

CONCLUSION:

The NN model demonstrates the best performance in predicting myocardial ischemia using radiomics features computed from CTA images, which suggests that this ML model has promising potential in guiding clinical decision-making.

Keywords

Heart machine learning computed tomography angiography (CTA)myocardial ischemia

1 Introduction

As the average age of the population has continued to increase, the incidence and mortality of coronary heart disease have likewise risen [1 –3]. Currently, the most commonly used method for the clinical assessment of coronary artery stenosis is computed tomography angiography (CTA). This technology is highly sensitive for identifying the degree of coronary artery stenosis and plaque type [4, 5], but the tissue contrast offered by CTA is limited, and so the feasibility of using CTA images to estimate myocardial ischemia is low [6]. The current gold standard modalities for the clinical diagnosis of myocardial ischemia include single-photon emission computed tomography (SPECT), positron emission tomography (PET), and fractional flow reserve (FFR) [1]. However, these tests are expensive and invasive, and so they cannot be widely used in clinical practice. An economical and practical method for assessing myocardial ischemia is therefore needed.

Texture analysis can be used on medical images to noninvasively extract a large number of parameters that cannot be visualized with the naked eye or quantified by conventional analysis, and to then construct predictive or prognostic models for disease treatment [7 –9]. Recently, research has shown that texture analysis can be used for the preliminary evaluation of myocardial ischemia [10]. However, this technique does have limitations. For instance, many texture features are extremely sensitive to the acquisition protocol; even if the same acquisition protocol is used for multiple analyses, the reproducibility of the features is uncertain, and such reproducibility is required for both accurate follow-up research and clinical applications [11, 12].

Any model based on machine learning (ML) must therefore demonstrate its ability to produce reproducible results in regard to texture parameters. To this end, we sought to build a high-performance ML model that could be used to evaluate myocardial ischemia in patients with coronary heart disease, and we assessed both the number of texture features and the effect of various ML methods.

2 Methods

2.1 Patient information

We obtained approval for this study from the research ethics committee of Changzhou No.2 People’s Hospital, the Affiliated Hospital of Nanjing Medical University, with a waiver of informed consent. We retrospectively analyzed data from patients with myocardial ischemia and patients with normal myocardial blood supply confirmed by FFR or digital subtraction angiography (DSA) at our hospital between September 2018 and January 2021. Patients were included in the study if they had no history of other heart disease, heart bypass surgery, or coronary stent implantation, and if the interval between CTA image acquisition and DSA or FFR diagnosis of myocardial ischemia was < 2 weeks. Patients were excluded from the analysis if they had liver or kidney dysfunction or if the interval between CTA image acquisition and myocardial ischemia diagnosis was < 1 week (Fig. 1).

Fig. 1

Flowchart for inclusion and exclusion of patients.

2.2 CTA scanning

All patients underwent CTA scans on a multislice spiral CT device (64 slices; Siemens AG, Munich, Germany). The scan range was from 1 to 2 cm below the tracheal crest to the bottom of the heart. The following parameters were used = automatic tube current modulation; tube voltage, 120 kVP; and slice thickness, 7.5 mm. The contrast agent iohexol (350 mg I/mL, Omnipaque, GE Healthcare) was intravenously injected. The duration of contrast medium injection was 12s, and the injection rate is 6 ml/s; heart rate≥75 beats/min; for every increase of 5 beats/min, the injection rate was elevated by 0.2 mL/s, and the contrast medium dose was similarly increased. Figures 2 and 3 show the CCTA images of patient with and without myocardial ischaemia, respectively.

Fig. 2

A 57-year-old male patient with myocardial ischaemia. A: Axial image, B: Sagittal image, and C: Coronal image.

Fig. 3

A 58-year-old female patient with myocardial ischaemia. A: Axial image, B: Sagittal image, and C: Coronal image.

2.3 Automated segmentation and feature extraction

The CTA images were imported into CQK (CT Coronary Artery Quantitative Analysis Kit, GE Healthcare, China) software for automated segmentation and feature extraction. Myocardial segmentation results were validated by a radiologist in the cardiovascular subgroup with at least 10 years of experience. The image analysis software package was imported on the CQK platform to extract texture features, which complied with Image Biomarker Standardisation Initiative (IBSI). Fig. 4 showed the workflow of the construction of the myocardial ischemia model.

Fig. 4

Workflow of the construction of the myocardial ischemia model.

2.4 Feature selection

The extracted texture features were preprocessed to remove texture features with zero variance. A Mann-Whitney U test was then used to identify texture features with significant difference (P values < 0.01) between the two groups (myocardial ischemia group and normal myocardial blood supply group), and these features were retained. To avoid unnecessary complexities and redundancies of the models, we then performed a minimum redundancy maximum relevance (MRMR) analysis on the selected texture features. Considering to the number of features affecting the final results, after MRMR, 4 feature subsets were obtained with 5, 10, 15, 20 features, respectively.

2.5 Model construction and comparison

The patients were divided into the training and test datasets with the ratio of 7 : 3, the training dataset was used to train the model, and independent test dataset was used to validate the final model. Seven ML methods were used to construct models = naive Bayes (NB), random forest (RF), extreme gradient boosting linear (xgbLinear), neural network (NN), k-nearest neighbors (KNN), support vector machines with the radial basis function kernel (svmRadial), and support vector machines with linear kernel (svmLinear). The optimal feature subset was identified based on the area under the curve (AUC) values of the 7 ML methods trained with each subset of features. The mean AUC value of the 7 models was determined, and the subset of texture features with the largest mean AUC was selected as the final subset.

After determining the optimal subset of texture features, a nested cross-validation scheme was performed for each ML method, and the model was repeatedly trained in the training data set. Each ML approach eventually has 100 models and each indicator (ie, accuracy, sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) would have 100 values. In addition, the robustness of each model was evaluated by calculating the relative standard deviation (RSD) values [13]. The smaller the RSD value, the higher robust of ML model. The following equation was used for RSD calculation: $RSD % = σ_{< Metric >} / μ_{< Metric >} \times 100 %$ where σ_<Metric> and μ_<Metric> are the average value of 100 specific metric values and the standard deviation of the values of 100 specific indicators (AUC and accuracy in this study), respectively.

2.6 Model validation and data analysis

According to the performance and stability of 7 ML models, the optimal model was determined, and the performance of the optimal model was validated in the independent test dataset using ROC analysis.

Then, in the statistical data analysis, continuous variables with a normal distribution were expressed as mean±standard deviation (X±S), and Student’s t-test was used for comparing the difference between the two patient groups. Continuous variables with non-normal distribution were expressed as median (interquartile range), and the Mann-Whitney U test was employed for comparing the difference between the groups. Categorical variables were presented as frequency (percentage) and were analyzed using a chi-square test. All statistical tests were conducted with SPSS v26.0 and R v3.6.1 (https://www.r-project.org) software. The two-tailed P-value of < 0.05 was deemed statistically significant.

3 Results

In this study, 58 patients with normal myocardial blood supply and 52 patients with myocardial ischemia were enrolled in this study. Among them, 77 patients were assigned in training dataset, and 33 patients were in test dataset. There were no obvious differences between the groups with respect to sex; age; presence of hypertension, hyperlipidemia or diabetes; and history of alcohol use or smoking (Table 1).

Table 1
Clinical characteristics of study patients

Variable Patients with normal myocardial blood supply Patients with myocardial ischemia P value

(n = 52) (n = 58)

Mean age±SD, y 60.8±9.8 61.2±10.1 0.945

Sex 0.951

Male 34 (65.3) 38 (65.5)

Female 18 (34.6) 20 (34.5)

Hypertension 0.884

Yes 30 (57.7) 32 (55.2)

No 22 (42.3) 26 (44.8)

Hyperlipidemia 0.829

Yes 15 (28.8) 16 (27.6)

No 37 (71.2) 42 (72.4)

Diabetes 0.213

Yes 9 (17.3) 16 (27.6)

No 43 (82.7) 42 (72.4)

History of alcohol use 0.365

Yes 12 (23.1) 18 (31.0)

No 40 (76.9) 40 (68.9)

History of smoking 0.996

Yes 16 (30.8) 18 (31.0)

No 36 (69.2) 40 (68.9)

Variable	Patients with normal myocardial blood supply	Patients with myocardial ischemia	P value
(n = 52)	(n = 58)
Mean age±SD, y	60.8±9.8	61.2±10.1	0.945
Sex			0.951
Male	34 (65.3)	38 (65.5)
Female	18 (34.6)	20 (34.5)
Hypertension			0.884
Yes	30 (57.7)	32 (55.2)
No	22 (42.3)	26 (44.8)
Hyperlipidemia			0.829
Yes	15 (28.8)	16 (27.6)
No	37 (71.2)	42 (72.4)
Diabetes			0.213
Yes	9 (17.3)	16 (27.6)
No	43 (82.7)	42 (72.4)
History of alcohol use			0.365
Yes	12 (23.1)	18 (31.0)
No	40 (76.9)	40 (68.9)
History of smoking			0.996
Yes	16 (30.8)	18 (31.0)
No	36 (69.2)	40 (68.9)

We used 4 subsets with different numbers of features (5, 10, 15, and 20 features) to train the model. The results showed that the model using 5 features demonstrated the highest average AUC. A heat map was used to identify the top 5 differences between the patients in normal myocardial blood supply group and those in myocardial ischemia group (Fig. 5).

Fig. 5

Heatmap of the 5 selected features for each patient. The larger the value, the color was closer to red, and the smaller the value, the color was closer to blue.

As shown in Fig. 6 and Table 2, we found that among the 7 ML models, xgbLinear had the highest accuracy (0.821) and AUC value (0.777), followed by NN (accuracy = 0.818; AUC = 0.757). NB had the highest sensitivity value (0.942), and RF had the highest specificity value (0.85). KNN had the lowest accuracy (0.74), svmLinear had the lowest AUC value (0.732), RF had the lowest sensitivity value (0.775), and NB had the lowest specificity value (0.68).

Fig. 6

Performance metrics of 7 predictive models built with different machine-learning methods. RF, random forest; symRadial, support vector machines with the radial basis function kernel; symLinear, support vector machines with linear kernel; NB, naive Bayes; NN, neural network; xgbLinear, extreme gradient boosting linear; KNN, k-nearest neighbors; AUC, area under the curve; NPV, negative predictive value; PPV, positive predictive value. “•” indicates outliers.

Table 2

Performance metrics for seven ML models

Metric	KNN	NB	NN	RF	svmLinear	svmRadial	xgbLinear
Accuracy	0.74±0.14	0.80±0.12	0.82±0.06	0.82±0.08	0.81±0.14	0.81±0.19	0.82±0.11
AUC	0.75±0.15	0.76±0.16	0.75±0.12	0.75±0.11	0.73±0.19	0.75±0.12	0.78±0.17
NPV	0.84±0.15	0.95±0.11	0.93±0.12	0.85±0.12	0.92±0.14	0.88±0.13	0.88±0.15
PPV	0.76±0.21	0.74±0.13	0.80±0.15	0.88±0.16	0.80±0.19	0.82±0.17	0.83±0.15
Sensitivity	0.80±0.19	0.94±0.12	0.89±0.18	0.78±0.17	0.89±0.18	0.83±0.19	0.82±0.21
Specificity	0.76±0.21	0.68±0.21	0.76±0.19	0.85±0.24	0.73±0.28	0.78±0.22	0.83±0.27

KNN, k-nearest neighbors; NB, naive Bayes; NN, neural network; RF, random forest; symLinear, support vector machines with linear kernel; symRadial, support vector machines with the radial basis function kernel; xgbLinear, extreme gradient boosting linear; AUC, area under the curve; NPV, negative predictive value; PPV, positive predictive value

The most robust classifier was NN (RSD = 0.16 for accuracy, 0.08 for AUC), followed by RF (RSD = 0.14 for accuracy, 0.10 for AUC), svmRadial (RSD = 0.16 for accuracy, 0.11 for AUC), xgbLinear (RSD = 0.22 for accuracy, 0.13 for AUC), NB (RSD = 0.21 for accuracy, 0.15 for AUC), KNN (RSD = 0.20 for accuracy, 0.17 for AUC), and svmLinear (RSD = 0.26 for accuracy, 0.18 for AUC) (Fig. 7).

Fig. 7

Model performance metrics (Left: accuracy, Right: area under the curve [AUC]) vs model stability. KNN, k-nearest neighbors; NB, naive Bayes; NN, neural network; RF, random forest; symLinear, support vector machines with linear kernel; symRadial, support vector machines with the radial basis function kernel; xgbLinear, extreme gradient boosting linear; RSD, relative standard deviation.

In general, we observed that NN had the best performance of the ML models in the training dataset. The performance of the NN model in the independent test dataset demonstrated an AUC value of 0.721 (95% confidence interval = 0.529 –0.882), with accuracy, sensitivity, specificity, PPV, NPV of 0.67, 0.69, 0.65, 0.65 and 0.69, respectively (cutoff = 0.396) (Fig. 8).

Fig. 8

Receiver operating characteristic curve of the model built using the neural network machine learning method. AUC, area under the curve.

4 Discussion

In this study, we extracted and selected the optimal myocardial radiomic features from CTA images and then used these features to construct 7 ML models for predicting and evaluating myocardial ischemia. We found that among these 7 models, NN model demonstrated relatively good predictive performance and the best stability. These results suggest that the NN method should be the model of choice for the prediction of myocardial ischemia.

Previous studies have shown that myocardial ischemia can cause changes in myocardial function and myocardial tissue, and these early changes cannot be observed with the naked eye. However, these subtle changes can be identified by assessing myocardial texture [14]. As in our study, the final selected feature set consisted of high-order texture features calculated from wavelet-transformed image. Similarly, Hu and co-workers [15] evaluated 1,409 radiomics features from CTA images from patients with myocardial ischemia and constructed a logistic regression model that demonstrated AUC values of 0.762 and 0.671 for the training and test cohorts, respectively. Shu et al. [10] similarly used ML to develop a model that could predict chronic myocardial ischemia. From CTA images, 378 textural radiomic features were captured based on 3-dimensional myocardial segmentations, and the resulting model demonstrated accuracy values of 0.839, 0.832 and 0.816 for the training, test and validation cohorts, respectively. In another study, Zhao and colleagues [16] extracted 385 radiomics features from target lesions on CTA images and constructed a logistic regression model, which demonstrated AUCs for predicting myocardial ischemia of 0.835 and 0.717 for the training and test cohorts, respectively.

In order to obtain optimized machine learning model, performance and stability were both to be considered. Performance determines the capability of the model, while stability determines the reliability of the model, and the latter is more important. In our study, we found 7 model had similar performance as shown in Table 2, but their stabilities were different, so we chose the optimized model which had relative high performance but most stable one. Several studies have also assessed and compared various ML-based models for predicting disease. For instance, Li et al. [17] evaluated the ability of 9 ML methods to predict local and distant metastasis in advanced nasopharyngeal cancer. They observed that the Adaboost (AUC = 0.82) and RF (AUC = 0.85) methods had excellent predictive performance and stability. Yin et al. [13] studied the ability of 7 ML models to estimate the pathological grades of clear cell renal cell carcinoma, and svmRadial (AUC = 0.86; RSD = 0.17 for accuracy, 0.13 for AUC) demonstrated an excellent prognostic performance. These findings suggest that there is no “one-size-fits-all” prognostic model for tumor prediction [18], and we believe that this also holds true for predicting myocardial ischemia based on radiomics features.

Many studies of prognostic model use random resampling to split the data set into training and test cohorts, which may lead to bias due to unreasonable partitioning schemes. Additionally, many elements can affect the performance of these models, such as the quantity of texture features and ML models; however, many studies ignore these potential differences and simply use an ML method to train and test the final model. In this study, a nested cross-validation was employed to train the prediction model, the data set was divided 100 times repeatedly, and the average performance indicators were derived to demonstrate the robustness of the model. In addition, we used different numbers of texture feature sets and ML models to train and identify optimal models.

This study did have some limitations. First, the data used to train and test the model were from a single institution, which may lead to selection bias. Our results must be verified with a large-scale, multicenter, prospective study. Second, the model included only radiomic features from CTA images. To improve the predictive performance of the model, laboratory indicators should be included. Our focus in this study was the effectiveness of radiomics features; in future studies, these features plus laboratory indicators will need to be assessed.

In summary, we studied the role of myocardial texture features in predicting myocardial ischemia and compared the effectiveness of 7 ML models. Among these models, the NN method using 5 features had the most outstanding diagnostic performance and highest stability. This radiomics-based model can further improve the clinical performance of CTA by allowing clinicians to more accurately diagnose myocardial ischemia and formulate more effective treatment plans.

References

Nagel

, Berry

Magnetic resonance perfusion or fractional flow reserve in coronary disease. Reply, New England Journal of Medicine 381 (2019)2277–8.

Abbara

, Blanke

, Maroules

. et al, SCCT guidelines for the performance and acquisition of coronary computed tomographic angiography: A report of the society of Cardiovascular Computed Tomography Guidelines Committee: Endorsed by the North American Society for Cardiovascular Imaging (NASCI), . Journal of Cardiovascular Computed Tomography 10 (2016)435–49.

Shay

C.M.

, Ning

, Daniels

S.R.

et al,Status of cardiovascular health in US adolescents: Prevalence estimates from the National Health and Nutrition Examination Surveys (NHANES) –. Circulation 127 (2013)1369–76.

Zhou

, Yang

et al,Impact of clinical guideline recommendations on the application of coronary computed tomographic angiography in patients with suspected stable coronary artery disease, . Chinese Medical Journal 129 (2016)135–41.

Budoff

M.J.

, Li

, Kazerooni

E.A.

et al,Diagnostic accuracy of noninvasive 64-row computed tomographic coronary angiography (CCTA) compared with myocardial perfusion imaging (MPI): The PICTURE study. A prospective multicenter trial, . Academic Radiology 24 (2017)22–9.

La Grutta

Toia

Maffei

et al,Infarct characterization using CT, . Cardiovascular Diagnosis and Therapy 7 (2017)171–88.

Yang

, Wu

, Sun

et al,Radiomics of small renal masses on multiphasic CT: accuracy of machine learning-based classification models for the differentiation of renal cell carcinoma and angiomyolipoma without visible fat, . European Radiology 30 (2020)1254–63.

Yao

, Ye

, Wei

et al,Radiomics in hepatocellular carcinoma: A state-of-the-art review, . World Journal of Gastrointestinal Oncology 13 (2021)1599–615.

Thawani

, McLane

, Beig

et al,Radiomics and radiogenomics in lung cancer: A review for the clinician. Lung Cancer 115 (2018)34–41.

10.

Shu

, Cui

, Zhang

et al,Predicting chronic myocardial ischemia using CCTA-based radiomics machine learning nomogram. Journal of Nuclear Cardiology 29 (2022)262–74.

11.

, Li

, Zhang

et al, Radiomics analysis of multicenter CT images for discriminating mucinous adenocarcinoma from nomucinous adenocarcinoma in rectal cancer and comparison with conventional CT values, . Journal of X-ray Science and Technology 28 (2020)285–97.

12.

Shao

, Mao

, Liu

et al, Epithelial salivary gland tumors: Utility of radiomics analysis based on diffusion-weighted imaging for differentiation of benign from malignant tumors, . Journal of X-ray Science and Technology 28 (2020)799–808.

13.

Yin

, Yang

, Tang

et al, Enhanced computed tomography radiomics-based machine-learning methods for predicting the Fuhrman grades of renal clear cell carcinoma, . Journal of X-ray Science and Technology 29 (2021)1149–60.

14.

Leipsic

, Weir-McCall

, Blanke

FFR for complex coronary artery disease treatment planning: new opportunities, . Interventional Cardiology 13 (2018)126–8.

15.

, Wu

, Dong

et al, Novel radiomics features from CCTA images for the functional evaluation of significant ischaemic lesions based on the coronary fractional flow reserve score, . International Journal of Cardiovascular Imaging 36 (2020)2039–50.

16.

Zhao

, Yuan

, Chen

et al, Exploring the diagnostic effectiveness for myocardial ischaemia based on CCTA myocardial texture features, . BMC Cardiovascular Disorders 21 (2021)416.

17.

, Liu

, Dong

et al,Multiparametric MRI radiomic model for preoperative predicting WHO/ISUP nuclear grade of clear cell renal cell carcinoma, . Journal of Magnetic Resonance Imaging 52 (2020)1557–66.

18.

Goyal

, Razik

, Kandasamy

et al, Role of MR texture analysis in histological subtyping and grading of renal cell carcinoma: a preliminary study. Abdominal Radiology 44 (2019)3336–49.