Machine learning for lung texture analysis on thin-section CT: Capability for assessments of disease severity and therapeutic effect for connective tissue disease patients in comparison with expert panel evaluations

Abstract

Background

The need for quantitative assessment of interstitial lung involvement on thin-section computed tomography (CT) has arisen in interstitial lung diseases including connective tissue disease (CTD).

Purpose

To evaluate the capability of machine learning (ML)-based CT texture analysis for disease severity and treatment response assessments in comparison with qualitatively assessed thin-section CT for patients with CTD.

Material and Methods

A total of 149 patients with CTD-related ILD (CTD-ILD) underwent initial and follow-up CT scans (total 364 paired serial CT examinations), pulmonary function tests, and serum KL-6 level tests. Based on all follow-up examination results, all paired serial CT examinations were assessed as “Stable” (n = 188), “Worse” (n = 98) and “Improved” (n = 78). Next, quantitative index changes were determined by software, and qualitative disease severity scores were assessed by consensus of two radiologists. To evaluate differences in each quantitative index as well as in disease severity score between paired serial CT examinations, Tukey's honestly significant difference (HSD) test was performed among the three statuses. Stepwise regression analyses were performed to determine changes in each pulmonary functional parameter and all quantitative indexes between paired serial CT scans.

Results

Δ% normal lung, Δ% consolidation, Δ% ground glass opacity, Δ% reticulation, and Δdisease severity score showed significant differences among the three statuses (P < 0.05). All differences in pulmonary functional parameters were significantly affected by Δ% normal lung, Δ% reticulation, and Δ% honeycomb (0.16 ≤r² ≤0.42; P < 0.05).

Conclusion

ML-based CT texture analysis has better potential than qualitatively assessed thin-section CT for disease severity assessment and treatment response evaluation for CTD-ILD.

Keywords

Lung computed tomography interstitial lung disease connective tissue disease machine learning texture analysis

Introduction

Connective tissue diseases (CTD) are a group of autoimmune diseases, sometimes also referred to as collagen vascular diseases, which consists of systemic sclerosis (SS), rheumatoid arthritis (RA), systemic lupus erythematosus (SLE), Sjögren syndrome, inflammatory myositis including polymyositis (PM), dermatomyositis (DM) and antisynthetase syndrome, and mixed connective tissue diseases (MCTD) (1). Moreover, patients with CTD are frequently affected by disease-related interstitial lung disease (ILD). ILD is a shared name for several types of diffuse parenchymal abnormalities of lung tissue and is characterized by inflammation or progressive scarring and fibrosis of lung parenchyma (1). Since ILD is associated with severe morbidity and early mortality for patients with CTD (2), early diagnosis and quantification of the severity of ILD is important for treatment initiation, monitoring, and prognostication (3,4).

Thin-section computed tomography (CT) of the chest has long been utilized for clinical assessment and physiological measures for diagnosing ILD, assessing disease severity and progression of disease, and assessing efficacy of treatment, and thus plays an important role in the management of patients with CTD. Interstitial abnormalities visible on thin-section CT include reticulation, consolidation, traction bronchiectasis, ground glass opacity (GGO), architectural distortion, volume loss, and honeycombing (5). Currently, evaluation of interstitial lung involvement on thin-section CT is performed by means of reader-based visual assessment or by subjective semi-quantitative scoring methods (6). However, these assessments are subjective and time-consuming while even in a research setting, substantial inter- and intra-observer variability have been reported, which may affect disease management for ILD (7). The need for quantitative assessment of interstitial lung involvement on thin-section CT has therefore arisen, and several investigators have been testing commercially available or proprietary software for CT texture analyses with different methods based on histogram analysis or artificial intelligence (4,8 –14). With these systems, thin-section CT findings can be categorized as follows: (i) normal lung, (ii) GGO, (iii) reticulation, (iv) emphysema, (v) nodular lesion, (vi) consolidation, and (vii) honeycomb, and moderate or good agreements among radiologists have been demonstrated (8,10,12).

Under these circumstances, we have developed and investigated the capability of a new system for computer-aided detection and computer-aided volumetry based on non-artificial intelligence (AI) and AI in basic and clinical trials (15 –17). For this system, we developed new machine learning (ML)-based software for fully automated CT texture analysis on thin-section chest CT for patients with various pulmonary diseases (18). However, no studies have been reported so far on this system's capability for evaluation of quantitative disease severity and treatment response for patients with CTD. We hypothesized that our newly developed ML-based algorithm has a better potential than qualitatively assessed disease severity for disease severity assessment and treatment response evaluation for patients with CTD. The aim of the present study was thus to prospectively and directly evaluate the capability of ML-based CT texture analysis software for disease severity and treatment response assessments in comparison with qualitatively assessed thin-section CT for patients with CTD.

Material and Methods

Protocol, support, and funding

The training and validation cases in this study were retrospectively obtained, and all test cases were gathered as a prospective study, which was approved by our institutional review board of Kobe University Hospital. Written informed consent was obtained from each individual. This study was financially and technically supported by Canon Medical Systems Corporation, Smoking Research Foundation, and Grants-in-Aid for Scientific Research from the Japanese Ministry of Education, Culture, Sports, Science and Technology (JSTS.KAKEN; No. 18K07675 and No. 20K08037). Three of the authors are employees of Canon Medical Systems (K.A., Y.F., and N.S.) but did not have control over any of the data used in this study.

Participants

Training set

A total of 45 cases were retrospectively collected from our institution between April 2015 and October 2015. The inclusion criteria for test cases were: (i) CT images reconstructed as 1-mm section thickness; (ii) diagnosis of ILD based on updated diagnostic criteria through multidisciplinary involvement of experienced clinical experts, radiologists, and pathologists (19); (iii) diagnosis of COPD based on Global Initiative for Chronic Obstructive Lung Disease (20); (iv) diagnosis of infectious disease based on radiological and microbacterial examinations from sputum or specimens obtained by transbronchial, CT-guided, or video-assisted thoracic surgery; (v) availability of baseline and >1-year follow-up CT scans; (vi) availability of baseline and results of >1-year follow-up pulmonary function tests and microbacterial examinations; (vii) no combined complications such as acute exacerbation in ILD or cardiopulmonary disease patients as well as pneumonia in patients with COPD and ILD; (viii) no evidence of other diseases such as thoracic malignancies, pneumothorax, and pulmonary thromboembolism at the time of baseline CT acquisition; and (ix) no specific treatment for pulmonary diseases at the time of baseline CT acquisition.

In the present study, 17 cases were excluded because they were lost to follow up (n = 5) or due to insufficient follow-up CT studies during a <2-year follow-up period (n = 12). Eventually, 28 training cases were included as the training set for this study. A total of 28 training cases consisting of 15 men (mean age = 66 ± 12 years; age range = 48–85 years) and 13 women (mean age = 63 ± 14 years; age range = 43–87 years) with usual interstitial pneumonia (UIP; n = 5), non-specific interstitial pneumonia (NSIP; n = 5), rheumatoid arthritis (RA; n = 3), dermatomyositis (DM; n = 2), polymyositis (PM; n = 2), mixed connective tissue disease (MCTD; n = 2), COPD (n = 2), non-tuberculous mycobacteriosis infection (NTM; n = 2), tuberculosis (TB; n = 2), sarcoidosis (n = 2), and mycoplasma pneumonia (n = 1). Data from a total of 11,250 slices obtained from these patients were used as the training dataset.

Validation set

A total of 27 cases were retrospectively collected as the validation set from our institution between November 2015 and December 2015, by applying the same inclusion criteria for selecting the test cases. Ten cases were excluded because they were lost to follow up (n = 6) or due to insufficient follow-up CT studies during a <2-year follow-up period (n = 4). Eventually, 17 cases with ten thin-section CT data were included as the validation set for this study. For the validation set, 17 cases were selected, consisting of ten men (mean age = 63 ± 11 years; age range = 48–71 years) and seven women (mean age = 61 ± 12 years; age range = 45–76 years) with UIP (n = 3), NSIP (n = 3), COPD (n = 3), NTM (n = 3), RA (n = 2), TB (n = 2), and sarcoidosis (n = 1). Data from a total of 6875 slices obtained from these patients were used as validation dataset.

Test set

Between January 2016 and December 2017, a total of 206 patients with CTD-related ILD (CTD-ILD) who met the various inclusion criteria and had been prospectively examined at our institution by means of unenhanced initial and follow-up CT scans using 320- and 64-detector row CT systems and had undergone pulmonary function and serum krebs von den Lungen-6 (KL-6) tests were originally included in this study. The inclusion criteria for this study were: (i) diagnosis of CTD-ILD based on updated diagnostic criteria through multidisciplinary involvement of experienced clinical experts, radiologists, and pathologists (21); (ii) availability of baseline and at least two years of follow-up CT examinations; (iii) availability of baseline and at least two years of follow-up pulmonary function tests (PFTs); (iv) availability of baseline and at least two years of follow-up laboratory tests such as IL-6; and (v) no combined complications such as pneumonia or acute exacerbation at the time of initial CT acquisition. Next, 57 cases were excluded due to specific treatment for pneumonia or acute exacerbation at the time of had not baseline CT acquisition (n = 28), insufficient follow-up studies (n = 19), or because they were lost to follow up (n = 10). Finally, 149 consecutive patients with CTD-ILD consisting of 56 men (mean age = 60 ± 11 years; age range = 39–83 years) and 93 women (mean age = 58 ± 9 years; age range = 30–85 years) were included as test cases. During at least two years of follow-up, each patient underwent ≥2 follow-up CT examinations after initial CT examination. Therefore, the total number of paired serial CT scans in this study was determined to be 364 examinations. For the test set, 149 consecutive patients with CTD-ILD were included, consisting of 54 cases of progressive scleroderma (PSS), 34 cases of dermatomyositis (DM), 29 cases of rheumatoid arthritis (RA), 14 cases of polymyositis (PM), seven cases of mixed connective tissue disease (MCTD), seven cases of anti-neutrophil cytoplasmic antibody (ANCA)-associated vasculitis, and four cases of Sjögren syndrome. After integration of clinical, laboratory, and paired serial CT data in each patient, all paired serial CT examinations (n = 364) were categorized as “Stable” (n = 188), “Worse” (n = 98), or “Improved” (n = 78) groups by a multidisciplinary team consisting of board-certified chest radiologists, pulmonologists, and immunologists with >10 years of experience who were not included for the image analyses in the study.

Thin-section CT examination

All CT data were obtained with a 320-detector row CT scanner (Aquilion ONE; Canon Medical Systems, Otawara, Tochigi, Japan) and two 64-detector row CT scanners (Aquilion 64; Canon Medical Systems). In each patient, baseline and follow-up CT examinations were performed with the same scanner in this study. For the 320-detector row CT, wide volumetric scanning (also known as step-and-shoot scanning) with the following parameters was used: collimation = 320 × 0.5 mm; 270 mA; 120 kVp; gantry rotation time = 0.5 s; matrix = 512 × 512, field of view (FOV) = 300–350 mm; 3 steps. For the 64-detector row CT, helical scanning with the following parameters was used: collimation = 64 × 0.5 mm; 270 mA; 120 kVp; beam pitch = 0.83; gantry rotation time = 0.5 s; matrix = 512 × 512; and FOV = 300–350 mm. All thin-section CT data were then reconstructed with the filtered back projection method in contiguous section thicknesses of 1 mm and used for generating the reconstruction standard lung kernel (FC51, Canon).

Pulmonary functional test

Pulmonary function testing was performed with an automatic spirometer (System 9, Minato Ikagaku) and according to American Thoracic Society standards (22,23). We measured forced vital capacity (FVC), forced expiratory volume in 1 s (FEV₁), vital capacity (VC), lung volume, and diffusing capacity of the lung for carbon monoxide (DL_CO), and then evaluated the ratio of FEV₁ to FVC (FEV₁/FVC%), the percentage predicted VC (%VC), and the percentage predicted DL_CO corrected for alveolar volume (%DL_CO/V_a) according to the ATS-ERS guidelines (24). DL_CO was estimated for 10-s breath-holding.

ML-based texture analysis software algorithm

Details of the ML-based texture analysis software was described in the previous literature (18), and this section is briefly mentioned the algorithm. Fig. 1 shows a schematic diagram of the ML-based texture analysis software used in this study. Given a set of chest CT images as input, it classifies every single voxel into seven radiological texture patterns: (i) normal lung; (ii) GGO; (iii) reticulation; (iv) emphysema; (v) nodular lesion; (vi) consolidation; and (vii) honeycomb. It utilizes three-dimensional (3D) ML methods described in the following sections. It consists of four stages: (i) preprocessing; (ii) feature extraction; (iii) classification; and (iv) postprocessing.

Fig. 1.

Flow chart of machine learning for computed tomography texture analysis. Flow chart of the proposed method. At the feature extraction stage, likelihood of each texture pattern's occurrence on every voxel is calculated. At the classification stage, probability of each texture pattern is calculated from the features extracted on each voxel. Finally, each voxel is labeled with a specific texture pattern showing the maximum posterior probability.

Preprocessing

Given a set of chest CT images as input, software converts it to an isotropic volume with 0.6-mm spacing. The lung region in the volume is then automatically segmented by means of the segmentation method implemented in the Lung Nodule Analysis application of Vitrea (Vital Images, Inc., Minnetonka, MN, USA). Only the voxels in the resulting lung mask are then processed in the following steps.

Feature extraction

In the feature extraction stage, the likelihood of each texture pattern on each voxel is calculated. For the six texture patterns not including nodular, the extremely randomized trees (ERT) method (25) is used to calculate the likelihoods. ERT is a tree-based ensemble method for supervised classification and randomly selects the cut-point. To capture texture patterns on multiple view scales, we use three sizes of 3D region of interests as inputs for ERT. For the nodular texture pattern, we employ the radial structure tensor (RST) (26) to calculate the likelihood of occurrence. RST is a filter that enhances blob-like structures by correlating position and direction of the gradient vectors in a local neighborhood. This is followed by applying average pooling with multiple local window sizes to the extracted features by means of both ERT and RST.

Classification

In the classification stage, probability of each texture pattern is calculated from the extracted features for each voxel by using the multi-class support vector machine (SVM) (27), which is a set of supervised learning methods used for classification. The output probabilities of SVM are then corrected by using conditional random field (CRF) (28), which is a discriminative model for machine learning. It provides optimal probabilities for a whole volume by considering differences in both location and voxel values between adjacent voxels.

Postprocessing

Finally, each voxel is labeled with a specific texture pattern with the maximum posterior probability. The voxels with a Hounsfield unit (HU) below −950 are relabeled with emphysema. Note that the voxels with a honeycomb label are excluded for this simple thresholding, as this texture pattern may contain voxels below −950 HU.

Image analysis

Quantitative assessment of disease severity using ML-based CT texture analysis software

All measurements by means of the newly developed ML-based CT texture analysis software were performed by a board-certified radiologist (T.Y.) using a commercially available workstation (Vitrea; Vital Images, Inc., Minnetonka, MN, USA), although the software used in this study was proprietary software (CT Lung Parenchyma Analysis, Prototype v. 3) provided by Canon Medical Systems and installed on the same workstation provided by Vitrea. With our proprietary software, thin-section CT findings for each patient with ILD were divided into the following seven different radiological findings-based categories derived from the glossary terms for thoracic imaging published by the Fleischner Society (29): (i) normal lung; (ii) emphysema; (iii) nodular lesion; (iv) consolidation; (v) GGO; (vi) reticulation; and (vii) honeycomb. Each lesion volume, normalized by the lung volume determined from CT data, was then automatically calculated. All radiological finding volumes (%normal lung, % emphysema, %nodular lesion, %consolidation, %GGO, %reticulation, and %honeycomb) were determined as a percentage of total lung volume. In addition, volume changes (Δs) between paired serial CT scans of each lesion were also automatically calculated for each patient.

Qualitative disease severity assessment based on thin-section CT findings

To evaluate the disease severity of ILD in patients with CTD, qualitatively assessed disease severity was independently scored by two chest radiologists (Y.O. and D.T.) with, 24 and 26 years of experience, respectively, with a picture archiving and communication system (PACS) (Shade Quest; Yokogawa Electric, Tokyo, Japan). Both reviewers assessed disease severity without having access to any information about disease severity based on serum KL-6 levels or to pulmonary function test results for any of the individuals.

As previously described (29 –31), visual evaluation involved both a severity and an extent score. The former is based on assessment of the parenchymal abnormalities assumed to reflect increasing severity of lung involvement: GGO (score = 1); irregular pleural margins (score = 2); septal and subpleural lines (score = 3); honeycombing (score = 4); and subpleural cysts (score = 5) (30 –32). The overall severity score thus ranged from 0 (no abnormality) to 15 (all abnormalities present). The extent score was obtained by counting the bronchial pulmonary segments in which any of the previous abnormalities are observed: involvement of 1–3 segments rated a score of 1; of 4–9 segments a score of 2; and of >9 segments a score of 3. The overall extent score thus ranged from 0 (no abnormality in any segment) to 15 (all five abnormalities in more than nine segments). The severity and extent scores were then added to obtain a total disease score (range = 0–30) (29 –31). In this study, each investigator evaluated total disease severity score at two different times. The final value for each investigator was then determined as the average value of the two assessments. In addition, change in total disease severity (Δ) between the paired serial CT scans was also calculated for each patient.

Statistical analysis

To determine reproducibility of ML assessment of all differences in radiologically detected findings between the first and second measurements with ML-based CT texture analysis software on CT examination at each time point, correlation and reproducibility of each finding concerning differences between the two measurements were statistically assessed by means of Pearson's correlation and Bland–Altman analysis (33).

For evaluating the intra-observer agreements on total disease severity scores for CT examination at each time point, correlation and reproducibility coefficients for the first and second evaluations by each investigator were determined by using the same statistical analyses.

To assess inter-observer agreement on total disease severity score for the two investigators on CT examination at each time point, correlation between the scores by the two investigators was also evaluated by means of Pearson's correlation. Further, the limits of agreement between both investigators were calculated by Bland–Altman analysis.

For determination of the associations between each pulmonary functional parameter and quantitatively and qualitatively assessed indexes on CT examination at each time point, univariate regression analyses were performed for each parameter and all quantitative indexes and disease severity scores. In addition, stepwise regression analyses were also performed to determine associations between each pulmonary functional parameter and all quantitative indexes.

To evaluate the differences between paired serial CT examinations for each quantitative index as well as for each disease severity score, Tukey's honestly significant difference (HSD) test was performed among three statuses used in this study.

Finally, all changes in pulmonary functional parameter between the paired serial CT scans were correlated with each quantitative index and disease severity score change by means of univariate regression analysis. In addition, stepwise regression analyses were also performed to determine associations between each pulmonary functional parameter and all quantitative index changes observed on the paired serial CT scans.

Results

Representative cases are shown in Figs. 2 and 3.

Fig. 2.

A 62-year-old male patient with dermatomyositis (first line, L to R: thin-section CT scans at baseline, acute exacerbation phase, and after treatment phase; second line, L to R: CT texture analysis results obtained by means of machine learning-based software at baseline and the same two phases.). A comparison of CT scans at baseline (i.e. “Stable” group) and at acute exacerbation phase (i.e. “Worse” group), shows an increase in GGO, reticulation, and honeycomb area, and a decrease in normal lung area. Δ%normal lung, Δ%GGO, Δ%reticulation and Δ%honeycomb were determined as −5.5%, 4.4%, 6.2%, and 7.1%¸ respectively, while Δdisease severity score was 10. A comparison of CT scans obtained at the acute exacerbation phase (i.e. “Worse” group) and at the after treatment phase (i.e. “Improved” group), shows a decrease in GGO, reticulation and honeycomb area, and an increase in normal lung area. Δ%normal lung, Δ%GGO, Δ%reticulation and Δ%honeycomb were determined as 3.3%, −1.5%, −2.8%, and −1.1%¸ respectively, while Δdisease severity score was −4. CT, computed tomography; GGO, ground glass opacity.

Fig. 3.

A 65-year-old female patient with progressive scleroderma (first line, L to R: thin-section CT scans at baseline, acute exacerbation phase, and after treatment phase; second line, L to R: CT texture analysis results by means of machine learning-based software at baseline and the same two phases.). A comparison of CT scans obtained at baseline (i.e. “Stable” group) and at the acute exacerbation phase (i.e. “Worse” group) shows an increase in GGO and consolidation area and a decrease in normal lung area. Δ%normal lung, Δ%GGO, and Δ%consolidation were determined as −16.9%, 13.2%, and 2.5%¸ respectively, while Δdisease severity score was 6. A comparison of CT scans obtained at the acute exacerbation phase (i.e. “Worse” group) and the after treatment phase (i.e. “Worse” group) shows an increase in GGO, reticulation, and honeycomb area and a decrease in normal lung area. Δ%normal lung, Δ%GGO, Δ%reticulation, and Δ%honeycomb were determined as −19.5%, 14.9%, 4.2%, and 0.2%, while Δdisease severity score was 15. CT, computed tomography; GGO, ground glass opacity.

On correlations between first and second assessments of ML evaluation of each radiological finding were significantly excellent or perfect for all radiological findings for volume between the first and second measurements (0.99 ≤ r; P < 0.0001), while their reproducibility coefficients were determined as 0.0 ± 0.0%.

Correlations and intra-observer agreements between first and second assessments by the two investigators were significant and excellent for qualitative disease severity score between first and second evaluations (first investigator: r = 0.99; P < 0.0001; second investigator: r = 0.99; P < 0.0001). Moreover, reproducibility coefficients were = 0 ± 2 for both investigators.

Correlation and inter-observer agreements between the two investigators were significant and excellent for qualitative disease severity scores (r = 0.98; P < 0.0001). The limits of agreement for qualitative disease severity scores between the two investigators were determined as 0 ± 2.

Table 1 shows the results of univariate regression analysis of differences between each pulmonary functional parameter and all radiological finding volumes derived from ML-based CT texture analysis as well as qualitative disease severity scores obtained in this study. There were significant correlations between FEV₁/FVC% and all quantitative indexes except % nodular lesion (%normal lung: r = 0.51; P < 0.0001; other indexes: −0.39 ≤ r ≤ −0.17; P ≤ 0.0001) and disease severity index (r = −0.46; P < 0.0001). There were also significant correlations between %VC and all quantitative indexes except %nodular lesion (%normal lung: r = 0.59; P < 0.0001; other indexes: −0.45 ≤ r ≤ −0.22; P < 0.0001) and disease severity index (r = −0.56; P < 0.0001). Further, there were significant correlations between %DL_CO/V_A and all quantitative indexes except %nodular lesion (%normal lung: r = 0.51; P < 0.0001; other indexes: −0.39 ≤ r ≤ −0.17; P < 0.0001) and disease severity index (r = −0.46; P < 0.0001). Finally, there were significant correlations between KL-6 level and all quantitative indexes except %emphysema, %nodular lesion, and %consolidation (%normal lung: r = −0.24; P < 0.0001; other indexes: 0.10 ≤ r ≤ 0.20; P ≤ 0.01) and disease severity index (r = 0.22; P < 0.0001).

Table 1.

Univariate regression analysis of differences between each pulmonary functional parameter and all radiological finding volumes derived from ML-based CT texture analysis as well as qualitative disease severity scores on CT examination at each time point.

Method	Evaluated index	FEV₁/FVC%		%VC		%DL_CO/V_A		Serum KL-6 level
Method	Evaluated index	r	P value	r	P value	r	P value	r	P value
Quantitative assessment	%normal lung (%)	0.51	<0.0001	0.59	<0.0001	0.51	<0.0001	−0.24	<0.0001
	%emphysema (%)	−0.17	0.0001	−0.22	<0.0001	−0.17	<0.0001	−0.01	0.79
	%nodular lesion (%)	−0.04	0.31	−0.05	0.28	−0.06	0.13	0.06	0.13
	%consolidation (%)	−0.33	<0.0001	−0.36	<0.0001	−0.29	<0.0001	0.10	0.01
	%GGO (%)	−0.33	<0.0001	−0.35	<0.0001	−0.29	<0.0001	0.15	0.0009
	%reticulation (%)	−0.39	<0.0001	−0.45	<0.0001	−0.39	<0.0001	0.20	<0.0001
	%honeycomb (%)	−0.27	<0.0001	−0.37	<0.0001	−0.30	<0.0001	0.17	0.0001
Qualitative assessment	Disease severity score	−0.46	<0.0001	−0.56	<0.0001	−0.46	<0.0001	0.22	<0.0001

CT, computed tomography; ML, machine learning.

On stepwise regression analysis of differences between all quantitative radiological indexes and each pulmonary functional parameter as well as serum KL-6 level, FEV₁/FVC% was significantly affected by %normal lung and %reticulation (r² = 0.27; P < 0.05), %VC was significantly affected by %normal lung, %reticulation, and %GGO (r² = 0.36; P < 0.05), while % DL_CO/V_A was significantly affected by %normal lung, %reticulation, and %GGO (r² = 0.27; P < 0.05). In addition, the serum KL-6 level was significantly affected by %normal lung and %reticulation (r² = 0.27; P < 0.05).

The results of a comparison of differences in pulmonary functional parameter, serum KL-6, each radiological index, and disease severity score among “Stable,” “Worse,” and “Improved” statuses for all patients with CTD-ILD are shown in Table 2. ΔFEV1/FVC%, Δ%VC, Δ%DLCO/VA, Δserum KL-6 level, Δ%normal lung, Δ%consolidation, Δ%GGO, Δ%reticulation and Δdisease severity score showed significant differences among the three statuses (P < 0.05). In addition, Δ%honeycomb had significant difference between “Stable” and “Worse” groups (P<0.05) and between “Worse” and "Improved" groups (P<0.05).

Table 2.

Comparison of differences in pulmonary functional parameter, serum KL-6, each radiological index and disease severity score among “Stable,” “Worse,” and “Improved” statuses for paired serial CT examinations in all patients with CTD-ILD.

	Status determined from paired serial CT scans
	Stable (n = 188)	Worse (n = 98)	Improved (n = 78)
ΔFEV₁/FVC%	0.3 ± 6.5	−6.0 ± 6.9*	5.2 ± 9.3*^†
Δ%VC	0.2 ± 6.0	−8.5 ± 8.2*	6.3 ± 9.7*^†
Δ%DL_CO/V_A	−0.2 ± 5.3	−6.4 ± 6.7*	3.7 ± 7.4*^†
ΔKL-6	−5.6 ± 289.0	190.2 ± 383.6*	−165.6 ± 632.7*^†
Δ% normal lung	−0.1 ± 1.4	−10.5 ± 9.9*	9.7 ± 8.0*^†
Δ% emphysema	0.0 ± 1.3	0.8 ± 4.4	0.0 ± 5.7
Δ% nodular lesion	0.0 ± 0.6	0.0 ± 1.6	0.0 ± 1.4
Δ% consolidation	0.0 ± 0.3	0.5 ± 1.1*	−0.3 ± 0.8*^†
Δ% GGO	0.0 ± 1.9	5.7 ± 9.7*	−5.4 ± 8.2*^†
Δ% reticulation	0.0 ± 1.4	1.8 ± 5.8*	−1.7 ± 4.5*^†
Δ% honeycomb	0.2 ± 1.2	1.5 ± 3.4*	0.1 ± 2.0^†
Δdisease severity score	0.2 ± 1.4	4.0 ± 5.1*	−2.3 ± 3.9*^†

Values are given as mean ± SD.

*Significant difference with “Stable” group (P < 0.05).

^†

Significant difference with “Worse” group (P < 0.05).

CT, computed tomography; CTD-ILD, connective tissue disease-related interstitial lung disease.

Table 3 shows results of univariate regression analysis of differences between each radiological index and each pulmonary functional parameter as well as serum KL-6 level for all patients with CTD-ILD. All quantitative radiological indexes except Δ% emphysema and Δ% nodular lesion and Δdisease severity score showed significant correlations with ΔFEV₁/FVC% (Δ% normal lung: r = 0.51; P < 0.0001; other quantitative indexes: 0.34≤r ≤ 0.21; P ≤ 0.0001; Δdisease severity score: r = −0.37; P < 0.0001) and Δserum KL-6 level (Δ% normal lung: r = −0.38; P < 0.0001; other quantitative indexes: −0.33≤r ≤ 0.20; P < 0.0001; Δdisease severity score: r = −0.33; P < 0.0001). All quantitative radiological indexes and Δdisease severity score showed significant correlations with Δ%VC (Δ% normal lung: r = 0.64; P < 0.0001; other quantitative indexes: −0.45≤r ≤ −0.11; P < 0.05; Δdisease severity score: r = −0.52; P < 0.0001) and ΔDL_CO/V_A (Δ% normal lung: r = 0.58; P < 0.0001; other quantitative indexes: −0.36≤r ≤ −0.12; P < 0.05; Δdisease severity score: r = −0.51; P < 0.0001).

Table 3.

Univariate regression analysis of differences between each radiological index and each pulmonary functional parameter as well as serum KL-6 level for paired serial CT examinations in all patients with CTD-ILD.

Method	Evaluated index	ΔFEV₁/FVC%		Δ%VC		Δ%DL_CO/V_A		ΔKL-6
Method	Evaluated index	r	P value	r	P value	r	P value	r	P value
Quantitative assessment	Δ% normal lung	0.51	<0.0001	0.64	<0.0001	0.58	<0.0001	−0.38	<0.0001
	Δ% emphysema	−0.11	0.05	−0.11	0.03	−0.12	0.02	−0.04	0.46
	Δ% nodular lesion	−0.09	0.09	−0.15	0.003	−0.16	0.002	0.06	0.25
	Δ% consolidation	−0.21	0.0001	−0.21	0.0002	−0.22	<0.0001	−0.20	<0.0001
	Δ% GGO	−0.24	<0.0001	−0.29	<0.0001	−0.28	<0.0001	−0.22	<0.0001
	Δ% reticulation	−0.34	<0.0001	−0.36	<0.0001	−0.36	<0.0001	−0.27	<0.0001
	Δ% honeycomb	−0.25	<0.0001	−0.45	<0.0001	−0.36	<0.0001	−0.23	<0.0001
Qualitative assessment	Δdisease severity score	−0.37	<0.0001	−0.52	<0.0001	−0.51	<0.0001	−0.33	<0.0001

CT, computed tomography; CTD-ILD, connective tissue disease-related interstitial lung disease.

Results of stepwise regression analysis of differences among all quantitative radiological indexes and each pulmonary functional parameter as well as serum KL-6 level for all patients with CTD-ILD are shown in Table 4. All differences in pulmonary functional parameters as well as serum KL-6 level were significantly affected by Δ% normal lung, Δ% reticulation, and Δ% honeycomb (0.16≤r²≤0.42; P < 0.05).

Table 4.

Stepwise regression analysis of differences among all quantitative radiological indexes and each pulmonary functional parameter as well as serum KL-6 level for paired serial CT examinations in all CTD-ILD patients.

	First step	Second step	Third step	r²	P value
ΔFEV₁/FVC%	Δ% normal lung	Δ% reticulation	Δ% honeycomb	0.31	0.03
Δ%VC	Δ% normal lung	Δ% reticulation	Δ% honeycomb	0.42	0.01
Δ%DL_CO/V_A	Δ% normal lung	Δ% reticulation	Δ% honeycomb	0.36	0.03
ΔSerum KL-6 level	Δ% normal lung	Δ% reticulation	Δ% honeycomb	0.16	0.04

CT, computed tomography; CTD-ILD, connective tissue disease-related interstitial lung disease.

Discussion

Our findings demonstrate that ML-based CT texture analysis can quantitatively and reproducibly assess pulmonary functional loss in patients with CTD-ILD and evaluate disease status using correlations between quantitatively assessed radiological findings and pulmonary functional parameters as well as serum KL-6 level. In addition, ML-based CT texture analysis software is potentially more effective, in terms of reproducibility, correlations among evaluated radiological and pulmonary functional parameters, and follow-up examinations at various disease stages, than qualitatively assessed disease severity score by board-certified chest radiologists. To the best of our knowledge, no other studies have compared the capability of ML-based CT texture analysis for disease assessment with disease severity qualitatively assessed by board-certified chest radiologists in this setting.

Assessment of the reproducibility coefficients of ML-based CT texture analysis for each radiological finding evaluation and each board-certified chest radiologist as well as inter-observer agreement between two investigators, showed that the correlation and reproducibility coefficient of the radiological finding assessments using this software were superior to those of qualitatively assessed disease severity scores for patients with CTD-ILD. Therefore, ML-based CT texture analysis had better potential for fewer discrepancies than qualitatively assessed disease severity scores for the assessment of patients with CTD-ILD.

On the basis of the results of univariate and stepwise regression analyses of quantitatively and qualitatively assessed radiological indexes, pulmonary functional parameters, and serum KL-6 level, it can be concluded that %normal lung correlates better than disease severity score with pulmonary functional parameters and serum KL-6 level, while %GGO and %reticulation proved to be significant factors for improving correlations using stepwise regression analyses. Moreover, these findings can be considered compatible with the past literature, even though the techniques used were different (8 –14). Therefore, quantitative assessment using ML-based CT texture analysis may be potentially superior to qualitative assessment for disease severity assessment based on thin-section CT in routine clinical practice.

When differences between paired serial CT scans in terms of each quantitative radiological index and disease severity score as well as pulmonary functional parameters and serum KL-6 level were assessed, Δ% normal lung, Δ% consolidation, Δ% GGO, Δ% reticulation, Δ% honeycomb, and Δdisease severity score as well as each pulmonary functional parameter and serum KL-6 level differences could be used to identify differences among “Stable,” “Worse,” and “Improved” statuses. In addition, these radiological differences correlated significantly with changes in pulmonary functional parameters and serum KL-6 level between paired serial CT scans. Furthermore, Δ% normal lung, Δ% reticulation, and Δ% honeycomb significantly influenced differences in all pulmonary functional parameters as well as serum KL-6 level. Finally, these quantitative radiological indexes obtained with ML-based CT texture analysis showed better correlations with disease status changes than did qualitatively determined disease severity scores in routine clinical practice. Therefore, our results indicate that ML-based CT texture analysis can be expected to be useful for answering various questions in not only academic, but also clinical studies of CTD-ILD as well as other patients with diffuse lung disease.

The present study has some limitations. First, this was a retrospective study, and the training and validation data were obtained from a single institution and only a few CT systems from only one CT vendor were used. Moreover, the number of training and validation cases were limited, and no external test set was used for confirmation of the results. A single hardware vendor's CT images were used, while other vendors CT data were not tested in this study. Therefore, it is necessary that data from other vendors will need to be used in a future study to validate the usefulness of this software and apply it in routine clinical practice. Second, neither the CT acquisition protocol nor the reconstruction algorithm was modified by means of state-of-the art CT techniques. Therefore, the diagnostic performance of our proprietary software might have been affected by the above-mentioned shortcomings and should thus be considered one of the limitations of this study. Third, only two radiologists assessed all radiological findings. As previously mentioned, however, increasing the number of investigators from different institutions and specialties may reduce inter-observer agreement for radiological finding evaluation, especially where honeycombing is concerned (34 –36). Moreover, diagnoses of ILD were not evaluated, nor was the effect of the software in this setting determined in this study. Further investigations of the application of this software in routine clinical practice are therefore warranted.

In conclusion, newly developed ML-based CT texture analysis shows better potential than qualitatively assessed thin-section CT for disease severity assessment and treatment response evaluation of patients with CTD.

Footnotes

Acknowledgements

The authors thank Shinichiro Seki (Division of Functional and Diagnostic Imaging Research, Department of Radiology, Kobe University Graduate School of Medicine); Yuji Kishida (Department of Radiology, Kobe University Graduate School of Medicine); Shintaro Tokunaga, Shuya Hori, Motoko Tachihara, Kazuyuki Kobayashi, Yoshihiro Nishimura (all from the Division of Respiratory Medicine, Department of Internal Medicine, Kobe University Graduate School of Medicine); Wakiko Tani, Noriyuki Negi, and Takamichi Murakami (all from the Center for Radiology and Radiation Oncology, Kobe University Hospital) for their valuable contributions to this study.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: KA, YF, and NS are employees of Canon Medical Systems Corporation, who had no control over any data or information submitted for publication nor any control over any parts of data or information included in this study.

Funding

The author(s) received the following financial support for the research, authorship, and/or publication of this article: This work was financially or technically supported by Canon Medical Systems Corporation, the Smoking Research Foundation, and Grants-in-Aid for Scientific Research from the Japanese Ministry of Education, Culture, Sports, Science and Technology (JSTS.KAKEN; No. 18K07675 and No. 20K08037).

ORCID iDs

Yoshiharu Ohno

Kota Aoyagi

Daisuke Takenaka

References

Fischer

du Bois

. Interstitial lung disease in connective tissue disorders. Lancet 2012;380:689–698.

Khanna

Mittoo

Aggarwal

, et al. Connective tissue disease-associated interstitial lung diseases (CTD-ILD) - report from OMERACT CTD-ILD working group. J Rheumatol 2015;42:2168–2171.

Hansell

Goldin

King

Jr , et al. CT Staging and monitoring of fibrotic interstitial lung diseases in clinical practice and treatment trials: a position paper from the Fleischner Society. Lancet Respir Med 2015;3:483–496.

van Royen

Moll

van Laar

, et al. Automated CT quantification methods for the assessment of interstitial lung disease in collagen vascular diseases: a systematic review. Eur J Radiol 2019;112:200–206.

Ahuja

Arora

Kanne

, et al. Imaging of pulmonary manifestations of connective tissue diseases. Radiol Clin North Am 2016;54:1015–1031.

Goh

Desai

Veeraraghavan

, et al. Interstitial lung disease in systemic sclerosis: a simple staging system. Am J Respir Crit Care Med 2008;177:1248–1254.

Aziz

Wells

Hansell

, et al. HRCT Diagnosis of diffuse parenchymal lung disease: inter-observer variation. Thorax 2004;59:506–511.

Iwasawa

Asakura

Sakai

, et al. Assessment of prognosis of patients with idiopathic pulmonary fibrosis by computer-aided analysis of CT images. J Thorac Imaging 2009;24:216–222.

Yoon

Seo

Kim

, et al. Quantitative assessment of change in regional disease patterns on serial HRCT of fibrotic interstitial pneumonia with texture-based automated quantification system. Eur Radiol 2013;23:692–701.

10.

Iwasawa

Kanauchi

Hoshi

, et al. Multicenter study of quantitative computed tomography analysis using a computer-aided three-dimensional system in patients with idiopathic pulmonary fibrosis. Jpn J Radiol 2016;34:16–27.

11.

Humphries

Yagihashi

Huckleberry

, et al. Idiopathic pulmonary fibrosis: data-driven textural analysis of extent of fibrosis at baseline and 15-month follow-up. Radiology 2017;285:270–278.

12.

Lee

Seo

, et al. Prediction of survival by texture-based automated quantitative assessment of regional disease patterns on CT in idiopathic pulmonary fibrosis. Eur Radiol 2018;28:1293–1300.

13.

Jacob

Bartholmai

Rajagopalan

, et al. Predicting outcomes in idiopathic pulmonary fibrosis using automated computed tomographic analysis. Am J Respir Crit Care Med 2018;198:767–776.

14.

Iwasawa

Okudela

Takemura

, et al. Computer-aided quantification of pulmonary fibrosis in patients with lung cancer: relationship to disease-free survival. Radiology 2019;292:489–498.

15.

Matsumoto S, Ohno Y, Aoki T, et al. Computer-aided detection of lung nodules on multidetector CT in concurrent-reader and second-reader modes: a comparative study. Eur J Radiol. 2013;82:1332–1337.

16.

Ohno Y, Aoyagi K, Chen Q, et al. Comparison of computer-aided detection (CADe) capability for pulmonary nodules among standard-, reduced- and ultra-low-dose CTs with and without hybrid type iterative reconstruction technique. Eur J Radiol. 2018;100:49–57.

17.

Ohno Y, Aoyagi K, Yaguchi A, et al. Differentiation of Benign from Malignant Pulmonary Nodules by Using a Convolutional Neural Network to Determine Volume Change at Chest CT. Radiology. 2020;296:432–443.

18.

Ohno Y, Aoyagi K, Takenaka D, et al. Machine learning for lung CT texture analysis: Improvement of inter-observer agreement for radiological finding classification in patients with pulmonary diseases. Eur J Radiol. 2021;134:109410.

19.

Raghu

Remy-Jardin

Myers

, et al. American Thoracic Society, European Respiratory Society, Japanese Respiratory Society, and Latin American Thoracic Society. Diagnosis of idiopathic pulmonary fibrosis. An official ATS/ERS/JRS/ALAT clinical practice guideline. Am J Respir Crit Care Med 2018;198:e44–e68.

20.

Global Initiative for Chronic Obstructive Lung Disease. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease 2019 report. Available at: https://goldcopd.org/wp-content/uploads/2018/11/GOLD-2019-v1.7-FINAL-14Nov2018-WMS.pdf (accessed 22 March 2019).

21.

Fischer

Strek

Cottin

, et al. Proceedings of the American college of rheumatology/association of physicians of Great Britain and Ireland connective tissue disease-associated interstitial lung disease summit: a multidisciplinary approach to address challenges and opportunities. QJM 2019;112:81–93.

22.

[No authors listed]. Standardization of spirometry--1987 update. Statement of the American thoracic society. Am Rev Respir Dis 1987;136:1285–1298.

23.

[No authors listed]. Lung function testing: selection of reference values and interpretative strategies. American thoracic society. Am Rev Respir Dis 1991;144:1202–1218.

24.

[No authors listed]. Single breath carbon monoxide diffusing capacity (transfer factor). Recommendations for a standard technique. Statement of the American thoracic society. Am Rev Respir Dis 1987;136:1299–1307.

25.

Geurts

Ernst

Wehenkel

. Extremely randomized trees. Mach Learn 2006;63:3–42.

26.

Wiemker

Klinder

Bergtholdt

, et al. A radial structure tensor and its use for shape-encoding medical visualization of tubular and nodular structures. IEEE Trans Vis Comput Graph 2013;19:353–366.

27.

Fan

Chang

Hsieh

, et al. LIBLINEAR: a library for large linear classification. J Mach Learn Res 2008;9:1871–1874.

28.

Krahenbuhl

Koltun

. Efficient inference in fully connected CRFs with gaussian edge potentials. Available at: https://arxiv.org/abs/1210.5644 (accessed 22 March 2019).

29.

Hansell

Bankier

MacMahon

, et al. Fleischner society: glossary of terms for thoracic imaging. Radiology 2008;246:697–722.

30.

Camiciottoli

Orlandi

Bartolucci

, et al. Lung CT densitometry in systemic sclerosis: correlation with lung function, exercise testing, and quality of life. Chest 2007;131:672–681.

31.

Ohno Y, Koyama H, Nogami M, et al. Dynamic perfusion MRI: capability for evaluation of disease severity and progression of pulmonary arterial hypertension in patients with connective tissue disease. J Magn Reson Imaging. 2008;28:887–899.

32.

Ohno Y, Nishio M, Koyama H, et al. Oxygen-enhanced MRI for patients with connective tissue diseases: comparison with thin-section CT of capability for pulmonary functional and disease severity assessment. Eur J Radiol. 2014;83:391–397.

33.

Bland

Altman

. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–310.

34.

Watadani

Sakai

Johkoh

, et al. Interobserver variability in the CT assessment of honeycombing in the lungs. Radiology 2013;266:936–944.

35.

Walsh

Calandriello

Sverzellati

, et al. UIP Observer consort. Interobserver agreement for the ATS/ERS/JRS/ALAT criteria for a UIP pattern on CT. Thorax 2016;71:45–51.

36.

Tominaga

Bankier

Lee

, et al. Study group of diffuse interstitial lung disease in Japan. Inter-observer agreement in identifying traction bronchiectasis on computed tomography: its improvement with the use of the additional criteria for chronic fibrosing interstitial pneumonia. Jpn J Radiol 2019;37:773–780.