Applying a nomogram based on preoperative CT to predict early recurrence of laryngeal squamous cell carcinoma after surgery

Abstract

PURPOSE:

To identify the value of a computed tomography (CT)-based radiomics model to predict probability of early recurrence (ER) in patients diagnosed with laryngeal squamous cell carcinoma (LSCC) after surgery.

MATERIALS AND METHOD:

Pre-operative CT scans of 140 LSCC patients treated by surgery are reviewed and selected. These patients are randomly split into the training set (n = 97) and test set (n = 43). The regions of interest of each patient were delineated manually by two senior radiologists. Radiomics features are extracted from CT images acquired in non-enhanced, arterial, and venous phases. Variance threshold, one-way ANOVA, and least absolute shrinkage and selection operator algorithm are used for feature selection. Then, radiomics models are built with five algorithms namely, k-nearest neighbor (KNN), logistic regression (LR), linear support vector machine (LSVM), radial basis function SVM (RSVM), and polynomial SVM (PSVM). Clinical factors are selected using univariate and multivariate logistic regressions. Last, a radiomics nomogram incorporating the radiomics signature and clinical factors is built to predict ER and its efficiency is evaluated by receiver operating characteristic (ROC) curve and calibration curve. Decision curve analysis (DCA) is also used to evaluate clinical usefulness.

RESULTS:

Four features are remarkably associated with ER in patients with LSCC. Applying to test set, the area under the ROC curves (AUCs) of KNN, LR, LSVM, RSVM, and PSVM are 0.936, 0.855, 0.845, 0.829, and 0.794, respectively. The radiomics nomogram shows better discrimination (with AUC: 0.939, 95% CI: 0.867–0.989) than the best radiomics model and the clinical model. Predicted and actual ERs in the calibration curves are in good agreement. DCA shows that the radiomics nomogram is clinically useful.

CONCLUSION:

The radiomics nomogram, as a noninvasive prediction tool, exhibits favorable performance for ER prediction of LSCC patients after surgery.

Keywords

Laryngeal squamous cell carcinoma nomogram early recurrence radiomics

1 Introduction

As an aggressive malignancy, laryngeal squamous cell carcinoma (LSCC) is one of the most prevalent cancers in the head and neck region and accounts for 85% –95% of all laryngeal cancers [1]. The main risk factors for LSCC are tobacco smoking and alcohol consumption, and the roles of these risk factors have been reported by many studies [2]. Currently accepted treatment modalities for LSCC management include surgery, radiation therapy (RT), and chemotherapy. Treatment scheme is based on tumor location, histology, staging, baseline function, and, in some cases, patient preference [3]. Despite multimodal therapy, more than 50% of patients with locoregionally advanced squamous cell carcinoma of the head and neck have recurrence or develop metastases (or both) within 2 years of treatment [4 –6]. The early prediction of recurrence and the early implementation of effective intervention measures are the key to improve the curative effect of LSCC [7, 8]. Therefore, a model for estimating the postoperative early recurrence (ER) of LSCC should be built.

Radiomics represents a noninvasive, high-throughput post-processing technique, which has emerged as a way of quantifying the tumor heterogeneity captured by radiological scans to estimate the features in the computed tomography (CT) images of tumors [9]. Radiomics models based on a wide range of imaging features are considered powerful prognostic biomarkers for predicting cancer recurrence [10]. Research interest in radiomics nomogram has been growing, as radiomics nomogram has shown promise in improving models for postoperative recurrence in patients with cancer [11 –13]. However, whether clinical radiomics nomogram can assess ER after LSCC surgery is unclear. ER refers to recurrence or metastasis within 2 years after surgery; it often originates from residual tumor and is related to the size of the primary tumor and vascular invasion [14]. Therefore, the present study was designed to explore radiomics nomogram to identify the value of CT-based radiomics model to predict ER in patients with LSCC after surgery.

This study aimed to develop and validate a nomogram for predicting the risk of ER in patients with LSCC who underwent surgical resection. In addition, the prediction performances and clinical usefulness of the nomogram, a radiomics model, and a clinical model were compared. Published studies have suggested that the CT-based radiomics nomogram may improve preoperative identification of nodal status and TP53 status and help in clinical decision-making in LSCC [15, 16]. Contrast-enhanced CT radiomics signature was independently associated with overall survival in LSCC patients. The previous study [17] has not discussed whether radiomics nomogram can well predict the early recurrence of laryngeal squamous cell carcinoma after surgery. This will provide a basis for standardized treatment and follow-up of laryngeal squamous cell carcinoma after surgery. Notably, the nomogram allows the tailoring of treatment to individual patients and guides plans for follow-up and surveillance in the long term.

2 Materials and methods

2.1 Patients

The ethics committee of Yantai Yuhuangding Hospital Affiliated to Qingdao University approved this retrospective study and waived patient informed consent. This study enrolled 145 patients with histologically confirmed LSCC from April 2013 to July 2017. All participants met the following inclusion criteria: (a) underwent surgical resection; (b) underwent enhanced CT scan within 1 week before surgery; (c) diagnosed with LSCC by pathologists; and (d) had follow-up for at least 2 years [14]. The exclusion criteria were as follows: (a) patients with other multiple primary tumors; (b) indeterminate or incomplete clinicopathologic information; (c) CT image quality that does not meet the analysis criteria. Union for International Cancer Control (8th edition) was used to evaluate the disease stage.

The 140 patients who met the criteria were divided into the training and test sets with a ratio of 7:3 using random stratified grouping. Demographic and clinicopathologic data, including age, gender, smoking, drinking, anatomical partition, T stage, N stage, clinical stage, histological grade, surgical approach, pathological metastasis and ER, were derived from medical records. In this study, ER is defined as recurrence within 2 years after operation [14]. During follow-up, patients were encouraged to have a follow-up in the first 1 month after surgery and then approximately every 3 months for the first 2 years. Recurrence was defined as the recurrence of LSCC revealed by CT, MRI, or laryngoscopy and confirmed via pathological biopsy at follow-up [18, 19].

The characteristics of the 140 enrolled patients are shown in Table 1. The training and test sets had 97 and 43 patients, respectively. The training set had 21 ER patients and 76 non-ER patients. The test set had 10 ER patients and 33 non-ER patients. No significant differences in age, gender, smoking, drinking, anatomical partition, T stage, N stage, clinical stage, surgical approach, pathological metastasis and ER emerged between the two groups (P > 0.05), but there was a significant difference in histological grade. The postoperative ER rate of patients with LSCC is 22.1%.

Table 1
Clinical characteristics of patients in the training and test sets

Characteristic Training set (n = 97) Test set (n = 43) P value

Age^a 60.2±7.4 60.6±8.4 0.686

Gender 0.802

Male 94(96.9) 42(97.7)

Female 3(3.1) 1(2.3)

Smoking 0.837

Yes 89(91.8) 39(90.7)

No 8(8.2) 4(9.3)

Drinking 0.217

Yes 73(75.3) 28(65.1)

No 24(24.7) 15(34.9)

Anatomical partition 0.568

Supraglottic 27(27.8) 15(34.9)

Subglottic 5(5.2) 1(2.3)

Glottic 65(67) 27(62.8)

T stage 0.245

1 45(46.4) 13(30.2)

2 26(26.8) 12(27,9)

3 16(16.5) 12(27.9)

4 10(10.3) 6(14.0)

N stage

0 82(84.5) 33(76.7) 0.198

1 7(7.2) 4(9.3)

2 8(8.2) 4(9.3)

3 0(0) 2(4.7)

Clinical stage 0.171

I 42(43.3) 13(30.2)

II 20(20.6) 10(23.3)

III 19(19.6) 11(25.5)

IV 16(16.5) 9(20.9)

Histological grade 0.017

Well 47(48.5) 14(32.6)

Moderate 32(33.0) 23(53.5)

Poor 18(18.6) 6(14.0)

Surgical approach 0.727

Partial 81(83.5) 35(81.4)

Total 16(15.5) 8(18.6)

Pathological metastasis 10(10,3) 4(9.3) 0.541

Early recurrence Yes 21(21.6) Yes 10(23.3) 0.833

No 76(78.4) No 33(76.7)

Characteristic	Training set (n = 97)	Test set (n = 43)	P value
Age^a	60.2±7.4	60.6±8.4	0.686
Gender			0.802
Male	94(96.9)	42(97.7)
Female	3(3.1)	1(2.3)
Smoking			0.837
Yes	89(91.8)	39(90.7)
No	8(8.2)	4(9.3)
Drinking			0.217
Yes	73(75.3)	28(65.1)
No	24(24.7)	15(34.9)
Anatomical partition			0.568
Supraglottic	27(27.8)	15(34.9)
Subglottic	5(5.2)	1(2.3)
Glottic	65(67)	27(62.8)
T stage			0.245
1	45(46.4)	13(30.2)
2	26(26.8)	12(27,9)
3	16(16.5)	12(27.9)
4	10(10.3)	6(14.0)
N stage
0	82(84.5)	33(76.7)	0.198
1	7(7.2)	4(9.3)
2	8(8.2)	4(9.3)
3	0(0)	2(4.7)
Clinical stage			0.171
I	42(43.3)	13(30.2)
II	20(20.6)	10(23.3)
III	19(19.6)	11(25.5)
IV	16(16.5)	9(20.9)
Histological grade			0.017
Well	47(48.5)	14(32.6)
Moderate	32(33.0)	23(53.5)
Poor	18(18.6)	6(14.0)
Surgical approach			0.727
Partial	81(83.5)	35(81.4)
Total	16(15.5)	8(18.6)
Pathological metastasis	10(10,3)	4(9.3)	0.541
Early recurrence	Yes 21(21.6)	Yes 10(23.3)	0.833
	No 76(78.4)	No 33(76.7)

Note —The data are displayed as n (%) except otherwise noted. a, mean±standard deviation.

2.2 Image acquisition

CT images were retrieved from the picture archiving and communication system (PACS; Carestream, Langley, British Columbia, Canada). The exposure parameters for the CT scan were as follows: voltage, 120 kV; effective mAs, 300 mAs; scanning slice thickness, 1.25 mm; pitch, 0.97; and matrix, 512×512.

2.3 Image segmentation

All patients underwent plain CT scan and two-phase enhancement scans, namely, arterial- and venous-phase scans. All contrast-enhanced CT Digital Imaging and Communications in Medicine images from PACS were exported for segmenting on ITK-SNAP version 3.8.0 (www.itksnap.org). The region of interest (ROI) of the lesion was annotated by a junior radiologist (Dr. A, with 5 years of experience) and confirmed by an expert radiologist (Dr. B, with 10 years of experience). The 3D volume of interest (VOI) covering the whole tumor was acquired by stacking up ROIs delineated slice by slice on CT images. Most of the tumor images can be segmented, and a small number of images that are difficult to segment are drawn according to the combination of arterial-phase, venous-phase, and plain CT scan images. A sample of the segmentation process is shown in Fig. 2A and 2B. The edge of the tumor is sketched layer by layer, and the final VOI is shown in Fig. 2C.

Fig. 2

Flowchart of CT image segmentation. VOI segmentation is performed on unenhanced and contrast-enhanced computed tomography images (A). Features are extracted from the VOI of primary tumor (B), including tumor shape, intensity, and texture. The VOI of primary tumor can be shown in a 3D status (C). VOI, volume of interest.

2.4 Radiomics feature extraction

Before feature extraction, VOIs were prepossessed by gray value standardization, gray level discretization, and image resampling. The pretreatment process is completed on PyRadiomics. Normalization of the extracted features is the first step before we do feature selection. We replaced the outliers with the median of the particular variance vector once the values reached beyond the range of the mean and standard deviation. Furthermore, we standardized the data in a specific interval. The standardized formula is as follows: (fi-u)/std, where fi represents a single characteristic data, u is the average value of the data column, and std pertains to the standard deviation of the data column. PyRadiomics in Python (version 3.7) was used to extract 3×1409 radiomics features from the arterial-phase, venous-phase, and plain CT scan images of each patient. “3” means the arterial-phase, venous-phase, and plain CT scan images of each case. Each phase has 1409 radiomics features. We computed 4227 radiomics features in all. Radiomics features included three categories: shape-based features, first-order statistics features, and texture features. Three months later, Dr. A and Dr. C (with 20 years of experience) segmented the images of 30 patients who were randomly selected to assess the intra- and inter-observer reproducibility of radiomics features by intra- and inter-class correlation coefficients (ICCs). ICC >0.8 indicated an excellent agreement of radiomics features [20].

2.5 Radiomics feature selection

Three methods were used to reduce the dimensions of features and effectively identify the most significant features. First, the variance threshold method was applied to remove the eigenvalues of the variance smaller than 0.8. Variance reflects the degree to which a group of data deviates from its average. The variance threshold is used to remove the feature values whose variance is less than 0.8. The formula of variance: $S^{2} = \frac{1}{n \times [{(x_1 - \bar{x})}^{2} + {(x_2 - \bar{x})}^{2} + \dots + {(x_n - \bar{x})}^{2}]}$ (1)

Total 4227 radiomics features are used to compute variance. Second, the Select K Best method was used to analyze the correlation between the features and the classification results, and the features with P < 0.05 were selected [18, 19]. Finally, this study adopted the least absolute shrinkage and selection operator (LASSO) method [21] for feature selection (Fig. 3). Radiomics signature (Rad-score) was calculated in each patient through the linear combination of the selected features with their respective coefficients.

Fig. 3

Radiomics feature selection. (A) Selection of tuning parameter (alpha) in the least absolute shrinkage and selection operator (LASSO) model using fivefold cross-validation with minimum criteria for feature selection to single out optimal features. (B) LASSO coefficient profiles of 1234 features. Vertical line was plotted at the selected value via fivefold cross-validation, where optimal alpha resulted in 4 non-zero coefficients.

2.6 Building and testing of Radiomics models

Radiomics-based models were built using five classifiers, namely, k-nearest neighbor (KNN), logistic regression (LR), linear support vector machine (LSVM), radial basis function ranking SVM (RSVM), and polynomial SVM (PSVM). These 5 models are commonly used algorithms in machine learning [22, 23]. Logistic regression (LR) model is the earliest computer model-aided diagnosis system for benign and malignant adnexal masses, whose advantage lies in its simple calculation and strong interpretability. For each feature in the data, its influence on the classification label can be measured by the coefficient of the linear part. Support vector machine (SVM) algorithm has good versatility, which does not have the risk of falling into local minimum and the problem of over-fitting. It can play a better effect in dealing with regression problems, classification and discrimination problems. SVM is considered as a general machine learning algorithm, which avoids the disadvantage that the result error is difficult to control when the number of samples is very large. K-nearest neighbor (KNN) is simple and convenient to operate, easy to implement and has good adaptability, which can be classified without any training. The prediction performances of the radiomics-based classifiers were assessed in terms of receiver operating characteristic (ROC) curve, area under the ROC curve (AUC), sensitivity, specificity, and accuracy in the training and test sets.

2.7 Construction and assessment of radiomics nomogram

In the training set, univariate and multivariate logistic regressions were used to select clinical risk factors. The clinical risk factors are N. stage, surgical approach and pathological metastasis. Rad-score equation was built based on radiomics features and corresponding regression coefficients. The significant variables, namely, clinical risk factors and Rad-score, were employed to develop a radiomics nomogram. The performance was evaluated through accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1 score, ROC curve, and AUC, and the corresponding 95% confidence interval (CI) was reported. The calibration curve was used to graphically investigate the agreement of ground truth and the probability predicted by the nomogram in the training and test sets. Decision curve analysis (DCA) was conducted to assess the clinical application of nomogram in the training and test sets [24].

2.8 Statistical analysis

All statistical analyses were conducted in R 4.0.2 and Python 3.6 software. All the levels of statistical significance were two-sided, and P < 0.05 was considered significant. Categorical clinical variables were compared using chi-square test or Fisher’s exact test, and continuous clinical variables were analyzed by two-sample t test. “SelectKBest” and “LassoCV” in Scikit-learn were used for selecting radiomics features. The “glm” function was used in multivariate logistic regression analysis. The “glm” package in R software was used to fit generalized linear models, specified by giving a symbolic description of the linear predictor and a description of the error distribution. The “rms” package was used to draw the radiomics nomogram and calibration curves. The “rms” means recession modeling strategy. Professor Harrell provides us with many easy-to-use functions for building, evaluating and testing model. The most famous is that “rms “ package is the first to provide the function of drawing the nomogram. The “sklearn and matplotlib” package was used to plot the ROC curves and measure the AUCs. The “rmda” package was used to perform DCA.

3 Results

3.1 Radiomics features and prediction performance of radiomics models

Radiomics workflow is illustrated in Fig. 1. Intra- and inter-observer ICCs were in the ranges of 0.857–0.968 and 0.842–0.959, which indicates an excellent agreement in the radiomics features. Firstly, 3603 radiomics features were screened out according to the threshold of 0.8 by low variance selection method. And then, 1234 features (P < 0.05) were selected based on select_KBest method (Table 2). Finally, four features with non-zero coefficients were selected with 5-fold cross validation method (Table 3). 5 machine learning models are built using the same features.

Fig. 1

Flowchart of the study design. ANOVA, analysis of variance; LASSO, least absolute shrinkage and selection operator

Table 2

Three feature selection steps of 5 machine leaning models.

The feature selection step	Number of radiomics features
low variance selection method	3603
Select_KBest	1234
LASSO	4

Note —LASSO, least absolute shrinkage and selection operator.

Table 3

The optimal predictive radiomic features screened by the LASSO

Names of radiomics features	Phase	Weighted coefficient
wavelet-LLH_glszm_GrayLevelNonUniformity	Venous phase	0.031944254
original_shape_LeastAxisLength	Venous phase	0.020917856
logarithm_firstorder_Kurtosis	Arterial phase	0.0000345
wavelet-HHL_glszm_SizeZoneNonUniformity	Non-contrast	0.000345871

Note —LASSO, least absolute shrinkage and selection operator.

The predictive models built using KNN, LR, LSVM, RSVM, and PSVM showed favorable discriminatory abilities with AUCs of 0.829 (95% CI: 0.644–0.964), 0.845 (95% CI: 0.657–0.993), 0.794 (95% CI: 0.616–0.976), 0.936 (95% CI: 0.845–1.000), and 0.855 (95% CI: 0.648–0.995), respectively, in the test set (Fig. 4B, Table 4). RSVM demonstrated the best performance among the five models (Fig. 4A-B). There is significant difference (P < 0.05) between KNN and PSVM in the training set. The significant difference is showed between LSVM and RSVM (P < 0.05) in the training set (Fig. 5).

Fig. 4

Receiver operating characteristic (ROC) curves of K-Nearest Neighbor (KNN), Logistic Regression (LR), Linear Support Vector Machine (LSVM), Radial basis function Support Vector Machine (RSVM), and Polynomial Support Vector Machine (PSVM) classifiers in the (A) training set and (B) test set.

Table 4

Performance of models built with different classifiers and features from different modalities

Method	AUC	95% CI	SEN	95% CI	SPE	95% CI	ACC	95% CI	Cutoff
Training set
KNN	0.932	0.892–0.973	0.952	0.741–0.998	0.776	0.664–0.861	0.814	0.723–0.886	0.2
RSVM	0.887	0.825–0.980	0.810	0.574–0.937	0.921	0.830–0.967	0.897	0.819–0.949	0.212
LR	0.846	0.753–0.957	0.810	0.574–0.937	0.921	0.830–0.967	0.897	0.819–0.949	0.557
LSVM	0.829	0.746–0.952	0.810	0.682–0.983	0.868	0.767–0.932	0.856	0.770–0.919	0.234
PSVM	0.828	0.722–0.934	0.714	0.477–0.878	0.908	0.814–0.959	0.866	0.782–0.927	0.180
Test set
RSVM	0.936	0.845–1.000	0.900	0.541–0.995	0.939	0.784–0.989	0.930	0.809–0.985	0.210
PSVM	0.855	0.648–0.995	0.800	0.442–0.965	0.970	0.825–0.998	0.930	0.809–0.985	0.188
LR	0.845	0.657–0.993	0.900	0.541–0.995	0.788	0.606–0.904	0.814	0.666–0.916	0.482
KNN	0.829	0.644–0.964	0.800	0.442–0.965	0.788	0.606–0.904	0.791	0.640–0.900	0.2
LSVM	0.794	0.616–0.976	0.700	0.354–0.919	0.818	0.639–0.924	0.791	0.640–0.900	0.185

Note —KNN, K-Nearest Neighbor; LR, Logistic Regression; LSVM, Linear Support Vector Machine; RSVM, Radial basis function Support Vector Machine; PSVM, Polynomial Support Vector Machine; AUC, area under the receiver operating characteristic; SEN, sensibility; SPE, specificity; ACC, accuracy; 95% CI, 95% confidence interval.

Fig. 5

The comparison 5 machine learning models (with P-values) in the (A) training set and (B) test set. KNN, K-Nearest Neighbor; LR, Logistic Regression; LSVM, Linear Support Vector Machine; RSVM, Radial basis function Support Vector Machine; PSVM, Polynomial Support Vector Machine.

3.2 Radiomic signature and nomogram construction

The Rad-score was calculated using the formula: $RadScore = 0.031944254 \times f_{1} + 0.020917856 \times f_{2} + 0.0000345 \times f_{3} + 0.000345871 \times f_{4} + 0.216$ (2) where f₁ is wavelet-LLH_glszm_GrayLevelNonUniformity, f₂ is original_shape_LeastAxisLength, f₃ is logarithm_firstorder_Kurtosis, and f₄ is wavelet-HHL_glszm_SizeZoneNonUniformity. The Rad-scores in the training and test sets showed no significant difference (P = 0.482).

The results of univariate and multivariate logistic regression analyses showed that surgical approaches, postoperative radiotherapy, pathological N-stage, and pathological metastasis were the risk factors for predicting LSCC ER (Table 5).

Table 5

Logistic regression analyses of early recurrence for training set and test set

Variable	Univariate logistic		Multivariate logistic
	regression		regression
	Odds ratio (95% CI)	P value	Odds ratio (95% CI)	P value
Age	0.002(0.005–0.422)	0.674	NA	NA
Gender	1.128(0.700–1.82)	0.622	NA	NA
Smoking	0.964(0.714–1.303)	0.813	NA	NA
Drinking	1.068(0.882–1.294)	0.500	NA	NA
Anatomical partition	0.842(0.773–0.918)	0.000	NA	NA
T stage	1.040(0.976–1.107)	0.228	NA	NA
N stage	1.220(1.091–1.365)	0.001	0.846(0.721–0.993)	0.044
Clinical stage	1.194(1.120–1.273)	0.000	NA	NA
Histological grade	0.848(0.765–0.941)	0.002	NA	NA
Surgical approach	1.272(1.147–1.410)	0.000	1.119(1.020–1.227)	0.020
Pathological metastasis	1.685(1.392–2.040)	0.000	1.955(1.458–2.623)	0.000
Rad score	64.727(15.021–278.926)	0.000	23.356(5.994–91.008)	0.000

Note —Data are odds ratios, with 95% CI in parentheses.

A radiomics nomogram was constructed based on the Rad-score, pathological metastasis, pathological N-stage, and surgical approaches (Fig. 6). In the training set, the radiomics nomogram achieved an AUC of 0.892 (95% CI: 0.789–0.981), which was better than that of the clinical model (LR: 0.791, 95% CI: 0.688–0.881) and the radiomics signature (AUC: 0.773, 95% CI: 0.682–0.900; Fig. 7A, Table 6). In addition, the radiomics nomogram achieved a sensitivity of 0.905 (95% CI: 0.682–0.983), a specificity of 0.842 (95% CI: 0.736–0.912), a PPV of 0.613 (95% CI: 0.423–0.776), a NPV of 0.967 (95% CI: 0.885–0.995), an accuracy of 0.856 (95% CI: 0.770–0.919), and an F1 score of 0.773 in the training set (Table 6). In the test set, the radiomics nomogram achieved a better AUC (0.921; 95% CI: 0.72–0.89) than the clinical model (LR: 0.835; 95% CI: 0.59–0.81; P = 0.2) and the radiomics signature (AUC: 0.815; 95% CI: 0.689–0.991; P = 0.2) as illustrated in Fig. 7B and Table 6. P-values of training set and test set was 0.083 and 0.483. The nomogram had a sensitivity of 0.900 (95% CI: 0.541–0.995), a specificity of 0.879 (95% CI: 0.709–0.960), a PPV of 0.692 (95% CI: 0.389–0.896), a NPV of 0.967 (95% CI: 0.809–0.998), and an F1 score of 0.783 in the test set (Table 6). Calibration curves for the radiomics nomogram in the training and test sets are shown in Fig. 8. The calibration curve showed good calibration in the training and test sets.

Fig. 6

Radiomics nomogram for laryngeal squamous cell carcinoma (LSCC) early recurrence (ER) prediction based on the training set. Surgical approach: 0, Partial laryngectomy; 1, Total laryngectomy; 2, Resection of epiglottic tongue root carcinoma. Pathological metastasis: “0” means “no”; “1” means “yes.”

Fig. 7

Receiver operating characteristic (ROC) curves of radiomics nomogram, radiomics model and clinical model in the (A) training set and (B) test set.

Table 6

Prognostic performances of models on the training and test sets

	Training set							Test set
	AUC	Accuracy	Sensitivity	Specificity	PPV	NPV	F1 score	AUC	Accuracy	Sensitivity	Specificity	PPV	NPV	F1 score
	(95% CI)	(95% CI)	(95% CI)	(95% CI)	(95% CI)	(95% CI)		(95% CI)	(95% CI)	(95% CI)	(95% CI)	(95% CI)	(95% CI)
Radiomic nomogram	0.892	0.856	0.905	0.842	0.613	0.967	0.773	0.921	0.884	0.900	0.879	0.692	0.967	0.783
	(0.789–0.981)	(0.770–0.919)	(0.682–0.983)	(0.736–0.912)	(0.423–0.776)	(0.885–0.995)		(0.836–0.987)	(0.749–0.961)	(0.541–0.995)	(0.709–0.960)	(0.389–0.896)	(0.809–0.998)
Radiomics signature	0.846	0.897	0.810	0.921	0.739	0.946	0.731	0.845	0.814	0.900	0.788	0.563	0.963	0.692
	(0.753–0.957)	(0.819–0.949)	(0.574–0.937)	(0.830–0.967)	(0.513–0.889)	(0.860–0.983)		(0.657–0.993)	(0.666–0.916)	(0.541–0.995)	(0.606–0.904)	(0.306–0.792)	(0.791–0.998)
Clinical model	0.791	0.814	0.714	0.842	0.556	0.914	0.625	0.835	0.837	0.800	0.848	0.615	0.933	0.696
	(0.688–0.881)	(0.723–0.886)	(0.477–0.878)	(0.736–0.912)	(0.356–0.740)	(0.816–0.965)		(0.691–0.959)	(0.693–0.932)	(0.442–0.965)	(0.673–0.943)	(0.323–0.849)	(0.765–0.988)

Note —Data are AUC, accuracy, sensitivity, specificity, PPV, NPV and F1 score with 95% CI in parentheses. AUC, area under the receiver operating characteristic; PPV, positive predictive value; NPV, negative predictive value.

Fig. 8

Calibration curves of radiomics nomogram in the training and test sets. The diagonal dotted line represents perfect prediction by an ideal model, and the solid line represents the performance of the nomogram. The close fit between the diagonal dotted lines and solid lines shows the good predictive ability of the nomogram for laryngeal squamous cell carcinoma (LSCC) early recurrence (ER).

The DCA results showed that the radiomics nomogram could provide more benefit than the clinical model and the radiomics model with a threshold probability range of 0–1.0 each in the training and test sets (Fig. 9). Decision curve analysis was conducted to determine the clinical usefulness of the radiomics nomogram by quantifying the net benefits at different threshold probability in the training and test sets. The decision curve analysis for the radiomics nomogram, clinical model and rad-score is presented in Fig. 9. The decision curve showed that if the threshold probability of a patient or doctor is >10%, using the radiomics nomogram to predict early recurrence of LSCC after surgery adds more benefit than either the treat-all-patients scheme or the treat-none scheme. There are 29 true negative (TN) cases, 4 false positive (FP) cases, 1 false negative (FN) cases and 9 true positive (TP) cases in the test set (Fig. 10).

Fig. 9

Decision curve analysis (DCA) of radiomics nomogram, radiomics model and clinical model in the (A) training set and (B) test set.

Fig. 10

Confusion matrix for the test set, 29 true negative (TN) cases, 4 false positive (FP) cases, 1 false negative (FN) cases and 9 true positive (TP) cases. TP, true positive; FP, false positive; TN, true negative; FN, false negative.

4 Discussion

This study is the first to develop a radiomics nomogram based on radiomics analysis that incorporates key clinical risk factors to quantitatively assess the risk of ER in patients with LSCC after surgery. Compared with the clinical and radiomics models, the radiomics nomogram had enhanced performance and could differentiate ER from non-ER with a sensitivity of 90.5% and a specificity of 84.2% in the test set. Therefore, we considered that the radiomics nomogram will be helpful to predict ER for patients with LSCC after surgery and guide in clinical decision making.

The radiomics features of primary tumors can be used to evaluate and predict the recurrence of gastric adenocarcinoma, breast cancer, larynx and hypopharynx squamous cell carcinoma, and other tumors [25 –28]. As a noninvasive preoperative examination, CT has been extensively used in cancer diagnosis and in guiding postoperative treatment. In a recent study, radiomics signature and nomogram based on CT were used to predict the overall survival of patients after the surgical resection of LSCC [17]. Unlike this study, we predicted the ER of patients with LSCC after surgery based on radiomics nomogram.

In this retrospective study, five predictive models were established to verify whether radiomics could be used to predict ER. Our results revealed that all the five models had good predictive performance in feature classification (accuracy: 0.791–0.930, AUC: 0.794–0.936). In our study, combining radiomics with independent clinical factors to build a radiomics nomogram achieved a favorable effect in predicting ER in patients with LSCC after surgery. The results indicated that the radiomics nomogram could serve as a noninvasive and reliable clinical adjuvant diagnosis method.

We believe that this is the first study of its kind to predict ER in LSCC based on radiomics nomogram. Cui et al. established a nomogram to predict survival probability in Asian patients with LSCC [29]. A nomogram with accessible clinicopathologic factors generates a more precise estimation of survival probability. Nevertheless, the experimental design is not comprehensive lack of radiomics signature. To some extent, this study made up the scarcity of previous research studies in this topic.

Our analyses have several predictors that are worth noting. First, we revealed T stage and tumor–node–metastasis (TNM) stage were correlated with the risk of recurrence through univariate analysis, but we cannot confirm the findings in multivariate analysis because tumor size or TNM stage undertook a relatively lower stratification value [30, 31]. Currently, the American Joint Committee on Cancer’s TNM stage system [32], which is based on anatomical information, is widely adopted when estimating recurrence risk. However, in clinical practice, the predictive efficacy of this system is limited. Patients with LSCC within the same stage are assumed to have homogenous outcomes, but, in fact, the outcomes are quite heterogeneous on account of variability in clinicopathologic features and tumor biology. The small proportion of N stages in our radiomics nomogram also seems to confirm this statement. The prevalences of metastatic tumors of the larynx are as follows: 39% transglottic, 27% supraglottic, 27% subglottic, and 7% glottic tumors [33]. Compared with other laryngeal cancer subsites, subglottic carcinoma has lower incidence and poorer prognosis [34]. The prognosis of supraglottic laryngeal carcinoma is worse than that of glottic carcinoma possibly because of the predisposing lymph node metastasis [35]. However, in the present study, anatomical partition was not integrated into the clinical risk factors, which might be relate to the small sample size.

Our study has several limitations. First, the segmentation of cancer tissues was performed manually, which was time consuming even for an experienced clinician. The automated method based on multi-view spatial aggregation framework was used for tracking and segmenting head and neck cancer in serial CT scans [36]. Thus, developing a rapid, reliable, fully automated segmentation method and cost-saving assay for LSCC is needed. Second, the retrospective study limited the variables available for analysis because some selection bias or unidentified confounding bias may influence the results. The impact of potential confounders needs to be addressed in prospective studies. On the basis of dividing the error into deviation and variance, machine learning creatively uses the prediction error (rather than goodness of fit) as the standard to measure the model, thus preventing over-fitting to a large extent [37, 38]. However, machine learning cannot eliminate over-fitting, and there is no way to do it at present. Third, the relatively small sample size and single-center design remarkably limit our study. Multicenter studies with a larger sample size need to be conducted in the future. The sample size is small and further studies with larger sample sizes should be carried out to improve the prediction efficiency of the nomogram. The problem of imbalanced image dataset is inevitable. There are a certain number of cases during the study period, and follow-up studies will continue to collect sample sizes, thus increasing the predictive ability of the radiomics nomogram. No cross-validation is used to evaluate the performance of a machine learning model due to the lack of randomness in data grouping. Finally, molecular factors may help predict the recurrence probability of LSCC with the advent of modern gene array and immunohistochemistry technology. LSCC is a heterogeneous disease, and our ability to assess its prognosis and predict the risk of recurrence based on clinical and radiomics findings is limited. Better decision-making tools are needed to help patients and their providers choose among therapeutic options. A combination of radiomics, clinical, and molecular factors may ultimately uncover more powerful and robust measures for disease risk classification than any one modality alone.

5 Conclusion

The radiomics nomogram should be recommend being used in this field on predicting early recurrence of laryngeal squamous cell carcinoma after surgery. The radiomics nomogram showed favorable prediction accuracy for early recurrence of laryngeal squamous cell carcinoma after surgery, which might facilitate the individualized risk stratification and clinical decision-making in LSCC patients. The radiomics nomogram, which combines radiomics features and clinical factors, had good predictive ability for LSCC ER based on its predictive value and clinical usefulness. Although the nomogram should not be used as a substitute for clinical practice, it may aid in the decision-making process for patients with LSCC. More accurate prediction models are expected to be applied in clinical practice in the era of big data with the continuous development of industrial intelligence in the future.

List of abbreviations

Computed tomography

LSCC

Laryngeal squamous cell carcinoma

Early recurrence

KNN

K-nearest neighbor

Logistic regression

LSVM

Linear support vector machine

RSVM

Radial basis function support vector machine

ROC

Receiver operating characteristic

DCA

Decision curve analysis

AUCs

The area under the ROC curves

PPV

Positive predictive value

NPV

Negative predictive value

Declarations

Ethics approval and consent to participate: The ethics committee of Yantai Yuhuangding Hospital Affiliated to Qingdao University approved this retrospective study and waived patient informed consent. All methods were performed in accordance with the relevant guidelines and regulations in Ethics Approval. This study is not involved with any compliance with ethical issues.

Consent for publication: Not applicable.

Availability of data and material: The datasets during and/or analysed during the current study available from the corresponding author.

Competing interests

The authors declare that they have no competing interests.

Funding

This study was supported in part by grants from Taishan Scholar Foundation of Shandong Province (No.ts20190991).

Authors’ contributions

Yao Yao: Methodology, Software, Formal analysis, Investigation, Resources, Writing - Original Draft.

Chuanliang Jia: Conceptualization, Resources, Writing - Review & Editing.

Haicheng Zhang: Methodology, Software, Data Curation.

Yakui Mou: Project administration, Supervision, Investigation.

Cai Wang: Investigation, Validation, Resources.

Xiao Han: Investigation, Validation, Resources.

Pengyi Yu: Investigation, Validation, Resources.

Ning Mao: Writing - Review & Editing, Conceptualization, Methodology.

Xicheng Song: Conceptualization, Supervision, Project administration.

Footnotes

Acknowledgments

We thank all members of the Big Data and Artificial Intelligence Laboratory, Yantai Yuhuangding Hospital, Yantai, Shandong, China.

References

Leoncini

, Vukovic

, Cadoni

, et al., Tumour stage and gender predict recurrence and second primary malignancies in head and neck cancer: A multicentre study within the INHANCE consortium, European Journal of Epidemiology 33(12) (2018), 1205–1218.

Zuo

J.J.

, Tao

Z.Z.

, Chen

, et al., Characteristics of cigarette smoking without alcohol consumption and laryngeal cancer: Overall and time-risk relation: A meta-analysis of observational studies, European Archives of Oto-rhino-laryngology 274(3) (2017), 1617–1631.

Gamez

M.E.

, Blakaj

, Zoller

, et al., Emerging concepts and novel strategies in radiation therapy for laryngeal cancer management, Cancers 12(6) (2020), 1651.

Licitra

and Felip

, Squamous cell carcinoma of the head and neck: ESMO clinical recommendations for diagnosis, treatment and follow-up, Annals of Oncology 20(Suppl 4) (2009), 121–122.

Pfister

D.G.

, Spencer

, Adelstein

, et al., Head and neck cancers, version 2.NCCN Clinical Practice Guidelines in Oncology, Journal of the National Comprehensive Cancer Network 18(7) (2020), 873–898.

Blanchard

, Baujat

, Holostenco

, et al., Meta-analysis of chemotherapy in head and neck cancer (MACH-NC): A comprehensive analysis by tumour site, Radiotherapy and Oncology 100(1) (2011), 33–40.

Zhang

and Wu

, Fasudil inhibits proliferation and migration of Hep-2 laryngeal carcinoma cells, Drug Design, Development and Therapy 12 (2018), 373–381.

Shen

, Gao

, Li

, et al., TIMP-3 Increases the chemosensitivity of laryngeal carcinoma to cisplatin via facilitating mitochondria-dependent apoptosis, Oncology Research 27(1) (2018), 73–80.

Gillies

R.J.

, Kinahan

P.E.

and Hricak

, Radiomics: Images are more than pictures, They are data, Radiology 278(2) (2016), 563–577.

10.

Yin

, Mao

, Liu

, et al., Can clinical radiomics nomogram based on 3D multiparametric MRI features and clinical characteristics estimate early recurrence of pelvic chondrosarcoma? Journal of Magnetic Resonance Imaging 51(2) (2020), 435–445.

11.

Huang

, Liu

, He

, et al., Radiomics signature: A potential biomarker for the prediction of disease-free survival in early-stage (I or II) non-small cell lung cancer, Radiology 281(3) (2016), 947–957.

12.

Ren

, Jing

, Wang

, et al., Automatic recognition of laryngoscopic images using a deep-learning technique, The Laryngoscope 130(11) (2020), E686–e693.

13.

Zhang

, Jiang

, Chen

, et al., Hepatocellular carcinoma: Radiomics nomogram on gadoxetic acid-enhanced MR imaging for early postoperative recurrence prediction, Cancer Imaging 19(1) (2019), 22.

14.

Kulbe

, Klein

, Wu

, et al., Discovery of prognostic markers for early-stage high-grade serous ovarian cancer by Maldi-imaging, Cancers 12(8) (2020), 2000.

15.

Wei

, Zhang

, Shi

, et al., The value of CT radiomics features to predict visceral pleural invasion in ≤3cm peripheral type early non-small cell lung cancer, Journal of X-Ray Science and Technology 30(6) (2022), 1115–1126.

16.

Tian

, Li

, Jia

, et al., Radiomics model for predicting TP53 status using CT and machine learning approach in laryngeal squamous cell carcinoma, Frontiers in Oncology 12 (2022), 823428.

17.

Chen

, Wang

, Zeng

, et al., Evaluation of CT-based radiomics signature and nomogram as prognostic markers in patients with laryngeal squamous cell carcinoma, Cancer Imaging 20(1) (2020), 28.

18.

Shu

, Tang

, Cui

, et al., Clear cell renal cell carcinoma: CT-based radiomics features for the prediction of Fuhrman grade, European Journal of Radiology 109 (2018), 8–12.

19.

Zhang

, Tian

, Dong

, et al., Radiomics features of multiparametric MRI as novel prognostic factors in advanced nasopharyngeal carcinoma, Clinical Cancer Research 23(15) (2017), 4259–4269.

20.

Berenguer

, Pastor-Juan

M.D.R.

, Canales-Vázquez

, et al., Radiomics of CT features may be nonreproducible and redundant:Influence of CT acquisition parameters, Radiology 288(2) (2018), 407–415.

21.

Friedman

, Hastie

and Tibshirani

, regularization paths for generalized linear models via coordinate descent, Journal of Statistical Software 33(1) (2010), 1–22.

22.

Mao

, Ma

, Duan

, et al., Preoperative classification of primary and metastatic liver cancer via machine learning-based ultrasound radiomics, European Radiology 31(7) (2021), 4576–4586.

23.

, Wu

, Mao

, et al., Computed tomography-based radiomics model to predict central cervical lymph node metastases in papillary thyroid carcinoma: A multicenter study, Frontiers in Endocrinology 12 (2021), 741698.

24.

Vickers

A.J.

and Elkin

E.B.

, Decision curve analysis: A novel method for evaluating prediction models, Medical Decision Making 26(6) (2006), 565–574.

25.

Bahig

, Lapointe

, Bedwani

, et al., Dual-energy computed tomography for prediction of loco-regional recurrence after radiotherapy in larynx and hypopharynx squamous cell carcinoma, European Journal of Radiology 110 (2019), 1–6.

26.

Mao

, Jiao

, Duan

, et al., Preoperative prediction of histologic grade in invasive breast cancer by using contrast-enhanced spectral mammography-based radiomics, Journal of X-Ray Science and Technology 29(5) (2021), 763–772.

27.

Park

, Lim

, Ko

E.S.

, et al., Radiomics signature on magnetic resonance imaging: Association with disease-free survival in patients with invasive breast cancer, Clinical Cancer Research 24(19) (2018), 4705–4714.

28.

, Huang

, Sun

, et al., Effect of laparoscopic vs open distal gastrectomy on 3-Year disease-free survival in patients with locally advanced gastric cancer: The CLASS-01 Randomized Clinical Trial, JAMA 321(20) (2019), 1983–1992.

29.

Cui

, Wang

, Zhong

, et al., Development and validation of nomogram to predict risk of survival in patients with laryngeal squamous cell carcinoma, Bioscience Reports 40(8) (2020), BSR20200228.

30.

Brandstorp-Boesen

, Sørum Falk

, Folkvard Evensen

, et al., Risk of recurrence in laryngeal cancer, PloS One 11(10) (2016), e0164068.

31.

Haapaniemi

, Koivunen

, Saarilahti

, et al., Laryngeal cancer in Finland: A 5-year follow-up study of 366 patients, Head & Neck 38(1) (2016), 36–43.

32.

Berger

D.M.S.

, Wassenberg

R.M.

, Jóźwiak

, et al., Inter-observer variation in the histopathology reports of head and neck melanoma; a comparison between the seventh and eighth edition of the AJCC staging system, European Journal of SurgicalOncology 45(2) (2019), 235–241.

33.

Zenga

, Mehrad

and Bradley

J.P.

, Metastatic cancer to the larynx: A case report and update, Journal of Voice 30(6) (2016), 774.e779–774.e712.

34.

MacNeil

S.D.

, Patel

, Liu

, et al., Survival of patients with subglottic squamous cell carcinoma, Current Oncology 25(6) (2018), e569–e575.

35.

Jin

, Hu

W.H.

, Guo

L.B.

, et al., Treatment results and prognostic factors of patients undergoing postoperative radiotherapy for laryngeal squamous cell carcinoma, Chinese Journal of Cancer 30(7) (2011), 482–489.

36.

Liang

, Thung

K.H.

, Nie

, et al., Multi-view spatial aggregation framework for joint localization and segmentation of organs at risk in head and neck CT images, IEEE Transactions on Medical Imaging 39(9) (2020), 2794–2805.

37.

Abdelwahed

N.M.

, El-Tawel

G.S.

and Makhlouf

M.A.

, Effective hybrid feature selection using different bootstrap enhances cancers classification performance, Bio Data Mining 15(1) (2022), 24.

38.

Charilaou

and Battat

, Machine learning models and over-fitting considerations, World Journal of Gastroenterology 28(5) (2022), 605–607.