A Deep Learning-Based Artificial Intelligence Model Assisting Thyroid Nodule Diagnosis and Management: Pilot Results for Evaluating Thyroid Malignancy in Pediatric Cohorts

Abstract

Purpose:

Artificial intelligence (AI) models have shown promise in predicting malignant thyroid nodules in adults; however, research on deep learning (DL) for pediatric cases is limited. We evaluated the applicability of a DL-based model for assessing thyroid nodules in children.

Methods:

We retrospectively identified two pediatric cohorts (n = 128; mean age 15.5 ± 2.4 years; 103 girls) who had thyroid nodule ultrasonography (US) with histological confirmation at two institutions. The AI-Thyroid DL model, originally trained on adult data, was tested on pediatric nodules in three scenarios axial US images, longitudinal US images, and both. We conducted a subgroup analysis based on the two pediatric cohorts and age groups (≥14 years vs. < 14 years) and compared the model’s performance with radiologist interpretations using the Thyroid Imaging Reporting and Data System (TIRADS).

Results:

Out of 156 nodules analyzed, 47 (30.1%) were malignant. AI-Thyroid demonstrated respective area under the receiver operating characteristic (AUROC), sensitivity, and specificity values of 0.913–0.929, 78.7–89.4%, and 79.8–91.7%, respectively. The AUROC values did not significantly differ across the image planes (all p > 0.05) and between the two pediatric cohorts (p = 0.804). No significant differences were observed between age groups in terms of sensitivity and specificity (all p > 0.05) while the AUROC values were higher for patients aged <14 years compared to those aged ≥14 years (all p < 0.01). AI-Thyroid yielded the highest AUROC values, followed by ACR-TIRADS and K-TIRADS (p = 0.016 and p < 0.001, respectively).

Conclusion:

AI-Thyroid demonstrated high performance in diagnosing pediatric thyroid cancer. Future research should focus on optimizing AI-Thyroid for pediatric use and exploring its role alongside tissue sampling in clinical practice.

Introduction

Thyroid cancer is the most common endocrine malignancy in children and adolescents.^1
–3 Although it is less frequent in young children and adolescents than adults, its incidence is increasing worldwide; thus, thyroid cancer is the second most common cancer in adolescents aged 15–19 years.^4,5 The increases in early and advanced forms of thyroid cancer highlight the need for prompt detection and appropriate management in pediatric populations.^6,7 Although several Thyroid Imaging Reporting and Data Systems (TIRADS) assist in the evaluation of thyroid nodules on ultrasonography (US) in adults, these classification systems are not designed for the pediatric population and their performance in pediatric populations has been inconsistent.^8

–12 The adult-based TIRADS biopsy criteria have not demonstrated the sensitivity required for pediatric use.^8

–12 Therefore, clinicians could benefit from tools that more accurately identify pediatric patients at high risk of malignancy to prioritize surgery, while limiting risks of surgery in patients with benign nodules.

Artificial intelligence (AI), particularly deep learning (DL), has shown promise in the evaluation of thyroid nodules among adults.^13,14 Increased computational power and large datasets have facilitated the development, external validation, and web-based accessibility of several AI-based systems.^13,15 However, pediatric thyroid nodules present distinct characteristics, including a higher risk of malignancy (22–26% vs. 5–10%) and distant metastases (30% vs. 5%), compared with adults in previous studies.^16

–21 Therefore, AI-based systems require validation with pediatric datasets to reliably guide decisions regarding cases that require clinical monitoring with repeat US versus cases that require fine-needle aspiration (FNA) biopsy. However, few studies have investigated the use of DL to assess thyroid nodules in pediatric populations.²² We evaluated the applicability of a DL-based model for the evaluation of thyroid nodules in pediatric populations.

Methods

Study design and two datasets

Datasets were obtained from two tertiary hospitals: Ajou University Medical Center (AUMC) in South Korea and Stanford Lucile Packard Children’s Hospital (LPCH) in the United States. The institutional review boards at both hospitals approved the study protocol, and the requirement for informed consent was waived due to the retrospective study design (AUMC: AJOUIRB-DB-2024-124, LPCH: IRB-68847). A research Collaboration agreement (RRA 527653) was established between the 2 institutions.

We included 150 consecutive pediatric patients (aged ≤18 years) with thyroid nodules ≥5.0 mm who underwent US and FNA at two institutions (2013.1-2022.12 at AUMC and 2019.1-2022.12 at LPCH). The majority were performed for either abnormal physical exam or follow-up of an incidentaloma discovered on imaging performed for other indications. A minority were done for surveillance of specific predispositions (e.g., cancer predisposition syndrome). Pediatric thyroid experts (pediatricians, radiologists, endocrinologists, or surgeons) reviewed the patient’s clinical presentation and US images before recommending FNA. Of these patients, 22 were excluded due to the lack of a definitive diagnosis (Bethesda category I, III–V) on biopsy without surgical confirmation (n = 18) or suboptimal image quality (n = 4). Patients with definitive benign or malignant diagnoses based on surgical specimens or biopsies (Bethesda category II or VI) were included.²³ We finally analyzed 156 thyroid nodules (benign, n = 109; malignant, n = 47) from 128 consecutive patients (mean age 15.5 ± 2.4 years; 103 girls) (Fig. 1).

FIG. 1.

Flowchart showing the study participants. FNA, fine-needle biopsy; CNB, core-needle biopsy.

US examination and image analysis

All US examinations were performed using a 10–12 or 5–14 MHz linear probe. US images were obtained from four different manufacturers: Samsung (Seoul, South Korea), Philips (Amsterdam, The Netherlands), GE Healthcare (Chicago, IL, USA), and Siemens (Munich, Germany). All US examinations at AUMC were performed by radiologists. At LPCH, they were performed by ultrasound technicians under the supervision of radiologists. A staff radiologist (E.J.H.), who had 17 years of clinical experience in performing and evaluating thyroid US images, retrospectively reviewed all US images and assessed nodule features, including composition, echogenicity, margin, orientation, calcification, and spongiform appearance (presence or absence). The radiologist was blinded to the outcome. In total, 156 thyroid nodules were assessed according to the relevant adult guidelines (American College of Radiology [ACR]- and Korean [K]-TIRADS).^2,3

Evaluation of system performance using a pediatric dataset

We used the AI-Thyroid software, a DL model trained on an adult population, to identify malignant thyroid nodules. The software is freely available online (http://cdss.co.kr/). The AI algorithm was trained on 19,711 US images and validated on 11,185 US images from multicenter adult datasets (24 hospitals).¹³ The algorithm is based on a VGG19-based convolutional neural network and uses binary cross-entropy as its loss function. It can process Digital Imaging and Communication in Medicine or JPEG images (both axial and longitudinal) with a manual annotation box. The algorithm outputs a gradient-weighted class activation mapping and an abnormality score (0–1.00) indicating the probability of malignancy. A predefined cut-off value of 0.472 was used to determine the presence of malignancy.¹³

We tested AI-Thyroid on two pediatric cohorts across three scenarios, including (1) nodules with axial US images, (2) nodules with longitudinal US images, and (3) nodules with both axial and longitudinal US images. We used the higher value between the two images in scenario 3. We performed a subgroup analysis according to the two different pediatric cohorts (AUMC vs. LPCH) and pediatric age groups (≥14 years vs. <14 years).

Comparison of system performance with adult-based TIRADS classification systems

We compared AI-Thyroid performance with adult-based TIRADS classification systems.^2,3 We used biopsy criteria (TIRADS category and nodule size) to evaluate the performances of ACR- and K-TIRADS. Nodules that met biopsy criteria were classified as malignant; nodules that did not meet such criteria were considered benign.¹³

Statistical analyses

Patient demographics and US grayscale features were compared using the chi-square test or Fisher’s exact test. The Kruskal–Wallis rank-sum test was used to compare quantitative variables. The frequency and malignancy risk of each nodule category were expressed as percentages. The diagnostic performances of the adult-based TIRADS and AI-Thyroid in detecting thyroid cancer were evaluated using sensitivities, specificities, positive predictive values (PPVs), and negative predictive values (NPVs), and the area under the receiver operating characteristic (AUROC) curves. AUROC curves were calculated with 95% confidence intervals (CIs). Diagnostic performances were compared using a proportion test with Yates’ continuity correction for proportions and the DeLong test within the R package “pROC” (version 4.1.2; R Foundation for Statistical Computing, Vienna, Austria). p-values <0.05 were considered statistically significant.

Results

Table 1 summarizes the clinical and radiological characteristics of patients with thyroid nodules. The overall prevalence of malignancy was 30.1% (47/156). Malignant nodules included papillary thyroid carcinomas (n = 40, including 3 follicular variants), follicular carcinomas (n = 3), medullary carcinomas (n = 3), and poorly differentiated thyroid carcinoma (n = 1). The prevalence of malignancy was significantly higher at AUMC (41.0%; 34/83) than at LPCH (17.8%; 13/73) (p = 0.003).

Table 1.

Clinicoradiological Characteristics of Thyroid Malignancies vs. Benign Nodules

Variables	Total (n = 156)	Benign (n = 109)	Cancer (n = 47)	p value
Age (years)				0.009
Mean ± standard deviation	15.5 ± 2.4	15.2 ± 2.5	16.2 ± 1.7
Range	6–18	6–18	12–18
Sex				1.000
Female	127 (81.4)	89 (81.8)	38 (80.9)
Male	29 (18.6)	20 (18.2)	9 (19.1)
Diameter (mm)				<0.001
Mean ± standard deviation	24.0 ± 15.1	21.3 ± 13.3	30.7 ± 16.9
Range	5.0–71.3	5.0–71.3	7.0–65.0
Location				0.170
Right	79 (50.6)	50 (45.5)	29 (61.7)
Left	73 (46.8)	56 (51.8)	17 (36.2)
Isthmus	4 (2.6)	3 (2.7)	1 (2.1)
Composition				<0.001
Solid	84 (53.8)	41 (37.3)	43 (91.5)
Partially cystic	72 (46.2)	68 (62.7)	4 (8.5)
Echogenicity				<0.001
Hypoechogenicity	53 (34.0)	15 (14.5)	38 (80.9)
Iso- or hyper-echogenicity	103 (66.0)	94 (85.5)	9 (19.1)
Calcifications				<0.001
Absence	121 (77.6)	104 (95.5)	17 (36.2)
Microcalcification	33 (21.2)	4 (3.6)	29 (61.7)
Macrocalcifications	2 (1.3)	1 (0.9)	1 (2.1)
Margin				<0.001
Smooth	115 (73.7)	96 (88.2)	19 (40.4)
Irregular/spiculated	28 (17.9)	5 (4.5)	23 (48.9)
Ill-defined	13 (8.3)	8 (7.3)	5 (10.6)
Orientation				<0.001
Parallel	124 (79.5)	99 (90.9)	25 (53.2)
Nonparallel	32 (20.5)	10 (9.1)	22 (46.8)
Cohort				0.003
AUMC	83 (53.2)	49 (45.5)	34 (72.3)
LPCH	73 (46.8)	60 (54.5)	13 (27.7)

The numbers in parentheses are percentages.

AUMC, Ajou University Medical Center; LPCH, Stanford Lucile Packard Children’s Hospital.

Patients with thyroid cancer (mean age 16.2 ± 1.7 [12–18] years) were significantly older than patients with benign nodules (mean age 15.2 ± 2.5 [6–18] years) (p = 0.009). Sex distribution did not significantly differ between groups (p = 1.000). The mean size of the nodules was 24.0 ± 15.1 mm; malignant nodules were significantly larger (30.7 ± 16.9 mm) than benign nodules (21.3 ± 13.3 mm; p < 0.001). US features revealed significant between-group differences in composition, echogenicity, orientation, margins, and calcifications (all p < 0.001). Malignant nodules were more likely to exhibit solid composition, hypoechogenicity, nonparallel orientation, spiculated margins, and microcalcifications.

Diagnostic performance of AI-Thyroid applied to the pediatric datasets

Table 2 and Supplementary Table S1 summarizes the diagnostic performances of AI-Thyroid according to different scenarios. AI-Thyroid demonstrated respective AUROC, sensitivity, specificity, and accuracy values of 0.913 (CI: 0.867–0.958), 78.7% (CI: 66.7–90.5), 85.3% (CI: 77.8–91.7), and 83.3% (CI: 76.9–89.1) in scenario 1; 0.929 (CI: 0.885–0.974), 78.7% (CI: 66.7–90.2), 91.7% (CI: 85.7–96.3), and 87.8% (CI: 82.1–92.3) in scenario 2; and 0.927 (CI: 0.891–0.954), 89.4% (CI: 79.5–97.6), 79.8% (CI: 72.1–86.9), and 82.7% (CI: 76.3–88.5) in scenario 3. No statistically significant differences in AUROC values of AI-Thyroid were observed across the image planes (all p > 0.05, Fig. 2).

Table 2.

Diagnostic Performance of AI-Thyroid in Pediatric Cohorts for Discriminating Malignant from Benign Thyroid Nodules

Cohort	AUROC	Accuracy (%)	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Total (n = 156)
Scenario 1	0.913 (0.867–0.958)	83.3 (76.9–89.1)	78.7 (66.7–90.5)	85.3 (77.8–91.7)	69.8 (56.9–82.3)	90.3 (84.2–95.7)
Scenario 2	0.929 (0.885–0.974)	87.8 (82.1–92.3)	78.7 (66.7–90.2)	91.7 (85.7–96.3)	80.4 (68.2–90.5)	90.9 (85.5–96.2)
Scenario 3	0.927 (0.891–0.954)	82.7 (76.3–88.5)	89.4 (79.5–97.6)	79.8 (72.1–86.9)	65.6 (53.7–77.1)	94.6 (89.1–98.9)
AUMC (n = 83)
Scenario 1	0.918 (0.859–0.977)	84.3 (77.1–91.6)	76.5 (62.1–91.2)	89.8 (81.0–97.7)	83.9 (70.0–95.8)	84.6 (75.5–94.3)
Scenario 2	0.925 (0.867–0.984)	85.5 (77.1–92.8)	73.5 (58.1–87.5)	93.9 (85.7–100.0)	89.3 (76.3–100.0)	83.6 (72.7–92.4)
Scenario 3	0.933 (0.882–0.963)	86.8 (78.3–94.0)	88.2 (75.8–97.2)	85.7 (75.0–94.9)	81.1 (67.6–93.2)	91.3 (82.4–97.9)
LCPH (n = 73)
Scenario 1	0.949 (0.891–1.000)	83.6 (75.3–91.8)	84.6 (63.6–100.0)	83.3 (73.3–91.8)	52.4 (31.9–73.7)	96.2 (90.0–100.0)
Scenario 2	0.954 (0.895–1.000)	89.0 (80.8–95.9)	84.6 (60.0–100.0)	90.0 (82.3–96.8)	64.7 (38.9–86.7)	96.4 (91.4–100.0)
Scenario 3	0.945 (0.905–0.991)	79.5 (69.9–89.0)	92.3 (75.0–100.0)	76.7 (65.6–87.5)	46.2 (25.9–66.7)	97.9 (92.9–100.0)

The numbers in parentheses are 95% confidence intervals.

AUROC, area under the receiver operating characteristic curve; NPV, negative predictive value; PPV, positive predictive value.

FIG. 2.

Diagnostic performance of AI-Thyroid applied to pediatric datasets across different image planes (A: Total cohort, B: AUMC, C: LPCH). AUMC, Ajou University Medical Center; AUROC, area under the receiver operating characteristic; CI, confidence interval; LPCH, Stanford Lucile Packard Children’s Hospital.

Table 2 and Supplementary Table S2 summarizes the diagnostic performances of AI-Thyroid according to different pediatric cohorts (AUMC vs. LPCH). When the overall diagnostic performance of AI-Thyroid applied to the two pediatric datasets, AI-Thyroid had similar performances in two cohorts, with respective AUROC, sensitivity, specificity, and accuracy values of 0.945–0.954, 84.6–92.3%, 76.7–90.0%, and 79.5–89.0% for the LPCH dataset; these values were 0.918–0.933, 73.5–88.2%, 85.7–93.9%, and 84.3–86.8% for the AUMC dataset. No statistically significant differences in AUROC values were observed between the cohorts (p = 0.804; Fig. 3).

FIG. 3.

Diagnostic performance of AI-Thyroid applied to two pediatric datasets. AUMC, Ajou University Medical Center; AUROC, area under the receiver operating characteristic; CI, confidence interval; LPCH, Stanford Lucile Packard Children’s Hospital.

Diagnostic performance of AI-Thyroid according to different age groups

Figure 4 visualizes the distribution of benign and malignant thyroid nodules according to age in the pediatric population. The prevalence of thyroid cancer showed significant differences even within the pediatric population, and it was higher in pediatric patients aged ≥14 years than in pediatric patients aged <14 years (36.1% vs. 8.8%, p = 0.002). Table 3 summarizes the diagnostic performances of AI-Thyroid according to different age groups. AI-Thyroid demonstrated respective AUROC, sensitivity, specificity, and accuracy values of 0.899–0.926, 77.3–88.1%, 75.8–92.3%, and 80.6–86.9% among pediatric patients aged ≥14 years, 1.000, 100.0%, 83.3–93.5%, and 85.2–94.1% among pediatric patients aged <14 years. The AUROC values were higher for patients aged <14 years compared to those aged ≥14 years (all p < 0.01). However, no statistically significant differences were observed between the different age groups in terms of sensitivity, specificity, PPV, NPV, and accuracy (all p > 0.05).

FIG. 4.

Diagnostic performance of AI-Thyroid across different age groups and distribution of benign and malignant thyroid nodules according to age. AUROC, area under the receiver operating characteristic; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

Table 3.

Diagnostic Performance of AI-Thyroid by Age Group for Discriminating Malignant from Benign Thyroid Nodules

Cohort	AUROC	Accuracy (%)	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Scenario 1
Age <14	1.000 (1.000–1.000)	94.1 (85.3–100.0)	100.0 (100.0–100.0)	93.5 (83.9–100.0)	60.0 (0.0–100.0)	100.0 (100.0–100.0)
Age ≥14	0.899 (0.844–0.953)	81.1 (73.8–87.7)	77.3 (64.1–90.2)	83.3 (74.3–90.7)	72.3 (58.0–8.38)	86.7 (78.7–94.4)
p value	0.002	0.068	0.352	0.163	0.572	0.099
Scenario 2
Age <14	1.000 (1.000–1.000)	94.1 (85.3–100.0)	100.0 (100.0–100.0)	93.5 (83.9–100.0)	60.0 (0.0–100.0)	100.0 (100.0–100.0)
Age ≥14	0.926 (0.877–0.975)	86.9 (80.3–92.6)	77.3 (63.9–89.1)	92.3 (85.7–98.4)	85.0 (71.8–96.1)	87.8 (80.3–94.5)
p value	0.004	0.243	0.352	0.823	0.173	0.270
Scenario 3
Age <14	1.000 (1.000–1.000)	85.2 (70.4–96,3)	100.0 (100.0–100.0)	83.3 (66.7–96.2)	42.9 (0.0–83.3)	100.0 (100.0–100.0)
Age ≥14	0.906 (0.846–0.965)	80.6 (73.1–88.0)	88.1 (78.0–97.1)	75.8 (64.7–86.1)	69.8 (56.9–82.0)	90.9 (82.7–98.1)
p value	0.002	0.580	0.526	0.445	0.317	0.238

The numbers in parentheses are 95% confidence intervals.

AUROC, area under the receiver operating characteristic curve; NPV, negative predictive value; PPV, positive predictive value.

Comparison of the diagnostic performance with the TIRADS-based classification

Table 4 summarizes the diagnostic performances of AI-Thyroid and radiologists using TIRADS-based classifications. The respective AUROC, sensitivity, specificity, and accuracy values of the radiologists were 0.684–0.831, 87.2%, 49.5–78.9%, and 60.9–81.4%. AI-Thyroid yielded the highest AUROC values, followed by ACR-TIRADS and K-TIRADS (p = 0.016 and p < 0.001, respectively). AI-Thyroid demonstrated comparable sensitivity and specificity to ACR-TIRADS (p = 0.748 and p = 0.991, respectively), while it showed comparable sensitivity (p = 0.755) but higher specificity (p < 0.001) to K-TIRADS.

Table 4.

Comparison of Diagnostic Performance Between AI-Thyroid and Radiologists Using TIRADS-Based Classifications

Cohort	AUROC	Accuracy (%)	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
AI-Thyroid	0.927 (0.880–0.973)	82.7 (76.3–88.5)	89.4 (79.5–97.6)	79.8 (72.1–86.9)	65.6 (53.7–77.1)	94.6 (89.1–98.9)
K-TIRADS	0.684 (0.616–0.751)	60.9 (53.8–68.6)	87.2 (76.3–95.9)	49.5 (40.2–59.5)	42.7 (33.7–51.8)	90.0 (81.6–96.7)
ACR-TIRADS	0.831 (0.769–0.892)	81.4 (75.0–87.2)	87.2 (77.3–95.8)	78.9 (70.6–86.2)	64.1 (51.6–75.9)	93.5 (88.2–97.9)
p value^a	<0.001	<0.001	0.755	<0.001	0.008	0.319
p value^b	0.016	0.930	0.748	0.991	0.954	0.764

p values indicate the comparison results between AI-Thyroid and K-TIRADS.

p values indicate the comparison results between AI-Thyroid and ACR-TIRADS.

AUROC, area under the receiver operating characteristic curve; NPV, negative predictive value; PPV, positive predictive value.

Discussion

Our study demonstrated that AI-Thyroid performs well in diagnosing pediatric thyroid cancer. AI-Thyroid achieved AUROC values ranging from 0.913 to 0.929, with no significant differences observed across different image planes or between the two pediatric cohorts. Notably, AI-Thyroid yielded the highest AUROC values and demonstrated comparable or even higher sensitivity and specificity than radiologist interpretations using TIRADS. These findings suggest that the diagnostic performances of the adult-based AI-Thyroid model are acceptable in the pediatric population.

Thyroid nodules are becoming increasingly prevalent in pediatric US examinations.^5
–7 A recent population-based study conducted in the United States showed the rates for both smaller, early-stage tumors and larger, later-stage tumors increased significantly over time.⁶ Although US and FNA cytology offer primary assessments of thyroid nodule malignancy risk, both carry a significant risk of indeterminate results and may fail to accurately exclude malignancy. A recent meta-analysis of a pediatric cytopathology series revealed that 5–28% of FNA specimens were nondiagnostic, whereas 3.3–38% were cytologically indeterminate.²⁴ Several recent pediatric case series have reported a malignancy rate of 51–64% among surgically managed thyroid nodule patients, highlighting the high rate of unnecessary surgery and the limitations of current preoperative diagnostics in terms of accurately identifying high-risk patients.^18,25,26 In addition, the lack of specific US-based risk stratification systems for pediatric populations complicates the assessment and management of these lesions.^8

–11,27 In this respect, we consider that AI-based systems may have a potential role to identify nodules at high risk of malignancy while reducing unnecessary biopsies by identifying nodules with acceptably low risk. This study evaluated the diagnostic performance of AI-Thyroid on two pediatric datasets. AI-Thyroid yielded AUROC values of 0.913–0.929 without significant differences across different image planes (axial vs. longitudinal planes) and between the two cohorts, despite variations in malignancy prevalence (41.0% vs. 17.8%) and geographic settings (South Korea vs. the United States). We anticipate that the AUMC and LPCH cohorts may have different baseline environmental factors, epigenetic/genetic backgrounds, and US protocols. Nonetheless, this study highlights that the model’s consistent performance across both settings is a significant strength. The results were also similar to those from studies involving adult populations, which reported AUROC values ranging from 0.922 to 0.938 ¹³. These findings suggest that AI-Thyroid, validated in adults using large-scale multicenter data, may also be applicable to pediatric populations.¹³

Even within pediatric populations, thyroid cancers can exhibit different characteristics depending on the age at diagnosis.^28,29 Harach et al. reported distinct characteristics between thyroid cancers in children under and over the age of ten, including variations in histological types, a male predominance, and a higher mortality rate in younger children compared to older ones.²⁸ Sugino et al. suggested that patients aged under 15 years present with distinct clinical manifestations, indicating that an age cutoff of <15 years might be more appropriate than <19 years for pediatric thyroid cancers.²⁹ In this study, we found a higher prevalence of malignant nodules among pediatric patients aged ≥14 years (36.1%) compared to those aged <14 years (8.8%). Despite the lower prevalence of malignant nodules among patients aged <14 years, no significant differences in sensitivity and specificity were observed between age groups. However, we consider that increasing the diagnostic cutoff value for patients aged <14 years may enhance diagnostic performance, given the lower prevalence of malignant nodules in this younger group. Future studies with larger, age-based pediatric datasets are needed to assess the clinical implications of AI-Thyroid for this population and to guide optimal management strategies for thyroid nodules, which may differ between young children and adolescents.

The lack of pediatric-specific risk stratification systems has led to the frequent use of adult-based TIRADS for assessing pediatric thyroid nodules. However, this practice is controversial due to insufficient evidence supporting these systems in children, particularly concerning biopsy indications. A recent study applying ACR-TIRADS to 404 thyroid nodules in 314 pediatric patients found that biopsy was not indicated for 22% of the 77 malignant thyroid nodules.³⁰ Another study evaluated five adult-based US risk stratification systems across 277 nodules in 221 children, revealing diagnostic performance that varied from 70% to 78% for sensitivity and from 42% to 78% for specificity.³¹ Similarly, a meta-analysis indicated that ACR-TIRADS, the American Thyroid Association guidelines, and European-TIRADS demonstrated moderate diagnostic performance for pediatric thyroid nodules, while K-TIRADS’s diagnostic efficacy was lower than expected.¹¹ Consequently, there is ongoing confusion regarding the use of adult-based TIRADS for managing pediatric thyroid nodules.³² Although the source of heterogeneity may mainly be the size threshold for suggesting biopsy, it remains unclear whether this is due to malignant nodules exhibiting different US morphology, carrying a higher risk on a per-nodule basis, a combination of both, or some other factor in the pediatric population. In our study, we found that the sensitivity and specificity of adult-based TIRADS were 87.2% and 49.5–78.9%, respectively. AI-Thyroid demonstrated comparable sensitivity and either comparable or higher specificity compared to adult-based TIRADS and achieved the highest AUROC values, followed by ACR-TIRADS and K-TIRADS. These findings suggest that AI-Thyroid could serve as a valuable decision-making aid by accurately identifying pediatric patients at high risk of malignancy, thus prioritizing FNA for those cases while reducing unnecessary biopsies for patients with benign nodules.

Few studies have explored the application of AI for predicting malignancy in pediatric thyroid nodules. Radebe et al. reported the first attempt at developing an interpretable machine learning-based clinical tool using 67 pediatric thyroid nodules.³³ Yang et al. developed a DL algorithm to differentiate between benign and malignant thyroid nodules using 139 pediatric thyroid nodules, reporting a sensitivity of 87.5% and a specificity of 36.1%.²² Although these studies demonstrated the potential of AI applications in pediatric thyroid nodules, the models were trained on relatively small patient populations due to the rarity of pediatric thyroid cancer. In this study, we utilized AI-Thyroid, a DL model originally trained on an adult population, to identify malignant thyroid nodules in a pediatric population.¹³ Given the limited sample size of pediatric thyroid cancer cases for developing new models, we assessed the applicability of an adult-based DL model for evaluating thyroid nodules in the pediatric populations. Our study demonstrated the high performance of AI-Thyroid in diagnosing pediatric thyroid cancer. Based on these results, we consider that this DL-based model is applicable for assessing thyroid nodules in children.

Our study had several limitations. First, the sample size was relatively small, particularly in the younger age group (<14 years). Nevertheless, this study represents one of the largest cohorts to date evaluating the diagnostic performance of an AI system in pediatric thyroid nodules. Further validation in larger cohorts is warranted to confirm our findings. Additionally, all patients underwent pathological confirmation, which may have introduced selection bias. Second, the data selection process, including the exclusion of low-quality images and the selection of the best representative axial and longitudinal images for the AI-Thyroid test, could have introduced bias. Third, we did not directly compare AI-Thyroid outcomes with physician-based outcomes. Additionally, comparing AI-Thyroid performance with adult-based TIRADS classification systems using biopsy criteria may not be entirely appropriate. Clinical diagnosis is a complex process, often informed by the real-time assessment of multiple images, rendering a direct comparison with AI data inappropriate. This algorithm did not incorporate clinical data (such as any pertinent family history), but a future state incorporating clinical factors may improve the robustness of the neural network and is a potential future direction. Similarly, this algorithm is not intended to produce a patient-facing summary or generate treatment recommendations, but those may also be future directions. Fourth, the assignment of US features has repeatedly been shown to exhibit considerable inter-observer variability. It would have been preferable if the images had been reviewed by at least two experts in consensus, possibly with a third expert as a tie-breaker in cases of disagreement. Fifth, the Bethesda classification categorizes FNA cytology into 6 different groups of varying malignant potential. Fusing the AI-Thyroid results with Bethesda III, IV, and V nodules could be a direction for future research.

In conclusion, we demonstrated the robust performance of AI-Thyroid in diagnosing thyroid cancer. Future investigations should focus on optimizing AI-Thyroid as a valuable adjunct to tissue sampling in clinical practice and drive the development of AI-based and/or conventional systems tailored to children.

Footnotes

Acknowledgment

A preliminary version of this study was presented in abstract form at the 2023 Annual Meeting of the American Thyroid Association.

Authors’ Contributions

E.J.H., K.W.Y., and K.D.M.: Acquisition of imaging data and data collection; E.J.H.: Article writing; J.H.L.: Statistical analysis and article review; K.W.Y., K.D.M., N.M., A.K.D., and E.T.: Article review. All authors read and approved the article.

Author Disclosure Statement

The authors of this article declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Funding Information

This work was supported by the National Research Foundation of Korea (NRF) grant by the Korea government (MSIT) (#2021R1C1C100698711).

Supplementary Material

Supplementary Tables

References

Haugen

, Alexander

, Bible

, et al. 2015 American Thyroid Association Management Guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: The American Thyroid Association Guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid, 2016; 26(1):1–133; doi: 10.1089/thy.2015.0020

, Chung

, Na

, et al. 2021 korean thyroid imaging reporting and data system and imaging-based management of thyroid nodules: Korean Society of Thyroid Radiology Consensus Statement and Recommendations. Korean J Radiol, 2021; 22(12):2094–2123; doi: 10.3348/kjr.2021.0713

Tessler

, Middleton

, Grant

, et al. ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White paper of the ACR TI-RADS Committee. J Am Coll Radiol, 2017; 14(5):587–595; doi: 10.1016/j.jacr.2017.01.046

Vergamini

, Frazier

, Abrantes

, et al. Increase in the incidence of differentiated thyroid carcinoma in children, adolescents, and young adults: A population-based study. J Pediatr, 2014; 164(6):1481–1485; doi: 10.1016/j.jpeds.2014.01.059

Lee

, Yun

, Lee

, et al. Trends in pediatric thyroid cancer incidence, treatment, and clinical course in Korea during 2004–2016: A Nationwide Population-Based Study. Thyroid, 2021; 31(6):902–911; doi: 10.1089/thy.2020.0155

Bernier

, Withrow

, Berrington de Gonzalez

, et al. Trends in pediatric thyroid cancer incidence in the United States, 1998-2013. Cancer, 2019; 125(14):2497–2505; doi: 10.1002/cncr.32125

, Tai

, Moon

, et al. Update on pediatric thyroid cancer incidence and mortality trends in the United States, 2000-2018. Cancer Invest, 2024; 42(4):357–360; doi: 10.1080/07357907.2024.2353313

Kim

, Yoon

, Hwang

, et al. Diagnostic performance of adult-based ATA and ACR-TIRADS ultrasound risk stratification systems in pediatric thyroid nodules: A systematic review and meta-analysis. Eur Radiol, 2021; 31(10):7450–7463; doi: 10.1007/s00330-021-07908-8

Scappaticcio

, Maiorino

, Iorio

, et al. Exploring the performance of ultrasound risk stratification systems in thyroid nodules of pediatric patients. Cancers (Basel), 2021; 13(21); doi: 10.3390/cancers13215304

10.

Richman

, Benson

, Doubilet

, et al. Assessment of American College of Radiology Thyroid Imaging Reporting and Data System (TI-RADS) for pediatric thyroid nodules. Radiology, 2020; 294(2):415–420; doi: 10.1148/radiol.2019191326

11.

Xing

, Qiu

, Zhu

, et al. Diagnostic performance of adult-based ultrasound risk stratification systems in pediatric thyroid nodules: A systematic review and meta-analysis. Front Endocrinol (Lausanne), 2023; 14:1187935; doi: 10.3389/fendo.2023.1187935

12.

Daniels

, Shaffer

, Garbin

, et al. Validity of the American College of Radiology Thyroid Imaging Reporting and Data System in children. Laryngoscope, 2023; 133(9):2394–2401; doi: 10.1002/lary.30425

13.

, Lee

, et al. Artificial intelligence model assisting thyroid nodule diagnosis and management: A multicenter diagnostic study. J Clin Endocrinol Metab, 2024; 109(2):527–535; doi: 10.1210/clinem/dgad503

14.

Peng

, Liu

, Lv

, et al. Deep learning-based artificial intelligence model to assist thyroid nodule diagnosis and management: A multicentre diagnostic study. Lancet Digit Health, 2021; 3(4):e250–e259; doi: 10.1016/S2589-7500(21)00041-8

15.

, Zhang

, et al. Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: A retrospective, multicohort, diagnostic study. Lancet Oncol, 2019; 20(2):193–201; doi: 10.1016/S1470-2045(18)30762-9

16.

Francis

, Waguespack

, Bauer

, et al.; American Thyroid Association Guidelines Task Force. Management guidelines for children with thyroid nodules and differentiated thyroid cancer. Thyroid, 2015; 25(7):716–759; doi: 10.1089/thy.2014.0460

17.

Lebbink

, Links

, Czarniecka

, et al. 2022 European Thyroid Association Guidelines for the management of pediatric thyroid nodules and differentiated thyroid carcinoma. Eur Thyroid J, 2022; 11(6); doi: 10.1530/ETJ-22-0146

18.

Gupta

, Ly

, Castroneves

, et al. A standardized assessment of thyroid nodules in children confirms higher cancer prevalence than in adults. J Clin Endocrinol Metab, 2013; 98(8):3238–3245; doi: 10.1210/jc.2013-1796

19.

Jarzab

, Handkiewicz-Junak

. Differentiated thyroid cancer in children and adults: Same or distinct disease? Hormones (Athens), 2007; 6(3):200–209.

20.

Cherella

, Angell

, Richman

, et al. Differences in thyroid nodule cytology and malignancy risk between children and adults. Thyroid, 2019; 29(8):1097–1104; doi: 10.1089/thy.2018.0728

21.

Burgwardt

, Healy

, Menendez

, et al. Validating the modified McGill thyroid nodule score for assessment of preoperative risk of pediatric thyroid malignancy. J Pediatr Surg, 2024; 59(7):1394–1398; doi: 10.1016/j.jpedsurg.2024.03.037

22.

Yang

, Page

, Wagner

, et al. Thyroid nodules on ultrasound in children and young adults: Comparison of diagnostic performance of radiologists’ impressions, ACR TI-RADS, and a deep learning algorithm. AJR Am J Roentgenol, 2023; 220(3):408–417; doi: 10.2214/AJR.22.28231

23.

Ali

, Baloch

, Cochand-Priollet

, et al. The 2023 Bethesda system for reporting thyroid cytopathology. Thyroid, 2023; 33(9):1039–1044; doi: 10.1089/thy.2023.0141

24.

Vuong

, Chung

DGB

, Ngo

, et al. The use of the Bethesda system for reporting thyroid cytopathology in pediatric thyroid nodules: A meta-Analysis. Thyroid, 2021; 31(8):1203–1211; doi: 10.1089/thy.2020.0702

25.

Amirazodi

, Propst

, Chung

, et al. Pediatric thyroid FNA biopsy: Outcomes and impact on management over 24 years at a tertiary care center. Cancer Cytopathol, 2016; 124(11):801–810; doi: 10.1002/cncy.21750

26.

Canadian Pediatric Thyroid Nodule Study G. The Canadian Pediatric Thyroid Nodule Study: An evaluation of current management practices. J Pediatr Surg, 2008; 43(5):826–830; doi: 10.1016/j.jpedsurg.2007.12.019

27.

Kim

, Suh

, Baek

, et al. Unnecessary thyroid nodule biopsy rates under four ultrasound risk stratification systems: A systematic review and meta-analysis. Eur Radiol, 2021; 31(5):2877–2885; doi: 10.1007/s00330-020-07384-6

28.

Harach

, Williams

. Childhood thyroid cancer in England and Wales. Br J Cancer, 1995; 72(3):777–783; doi: 10.1038/bjc.1995.410

29.

Sugino

, Nagahama

, Kitagawa

, et al. Cutoff age between pediatric and adult thyroid differentiated cancer: Is 18 years old appropriate? Thyroid, 2022; 32(2):145–152; doi: 10.1089/thy.2021.0255

30.

Shapira-Zaltsberg

, Miller

, Martinez-Rios

, et al. Comparison of the diagnostic performance of the 2017 ACR TI-RADS guideline to the Kwak guideline in children with thyroid nodules. Pediatr Radiol, 2019; 49(7):862–868; doi: 10.1007/s00247-019-04385-6

31.

Kim

, Yoon

, Baek

, et al. Diagnostic performance of five adult-based US risk stratification systems in pediatric thyroid nodules. Radiology, 2022; 305(1):190–198; doi: 10.1148/radiol.212762

32.

Sammer

MBK

, Hollingsworth

, Stanescu

, et al. Employing caution when applying the American College of Radiology Thyroid Imaging Reporting and Data System for pediatric thyroid nodule management. Pediatr Radiol, 2021; 51(8):1290–1293; doi: 10.1007/s00247-021-05083-y

33.

Radebe

, van der Kaay

DCM

, Wasserman

, et al. Predicting malignancy in pediatric thyroid nodules: Early experience with machine learning for clinical decision support. J Clin Endocrinol Metab, 2021; 106(12):e5236–e5246; doi: 10.1210/clinem/dgab435

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.14 MB

0.00 MB