Prospective Validation of ThyroSPEC Molecular Testing of Indeterminate Thyroid Nodule Cytology Following Diagnostic Pathway Optimization

Abstract

Background:

Molecular testing for cytologically indeterminate thyroid nodules (ITNs) is often reported with incomplete data on clinical assessment and ultrasound malignancy risk (USMR) stratification. This study aimed to clinically validate the diagnostic accuracy of a novel molecular test, assess the incremental preoperative malignancy risk of other clinical factors, and measure the impacts of introducing molecular testing at the population level.

Methods:

Comprehensive clinical data were collected prospectively for the first 615 consecutive patients with ITNs in a centralized health care system following implementation of a reflexive molecular test. Clinical data include patient history, method of nodule discovery, clinical assessment, USMR, cytology, molecular testing, and surgery or follow-up along with surgeon notes on surgical decision-making. Accuracy of molecular testing and the impact of the introduction of molecular testing were calculated. A multivariable regression model was developed to identify which clinical factors have the most diagnostic significance for ITNs.

Results:

A locally developed, low-cost molecular test achieved a negative predictive value (NPV) of 76–91% [confidence interval, CI 66–95%] and a positive predictive value (PPV) of 46–65% [CI 37–75%] in ITNs using only residual material from standard liquid cytology fine-needle aspiration (FNA). Sensitivity was highest (80%; [CI 63–92%]) in the American Thyroid Association (ATA) intermediate-suspicion ultrasound category, and lowest (46%; [CI 19–75%]) in the ATA high-suspicion ultrasound category. Following implementation of molecular testing, diagnostic yield increased by 14% (p = 0.2442) and repeat FNAs decreased by 24% (p = 0.05). Mutation was the primary reason for surgery in 76% of resected, mutation-positive patients. High-risk mutations were associated with a 58% (p = 0.0001) shorter wait for surgery. Twenty-six percent of patients with a negative molecular test result underwent surgery. Multivariable regression highlighted molecular testing and USMR as significantly associated with malignancy.

Conclusions:

Molecular testing improves preoperative risk stratification but requires further stratification for intermediate-risk mutations. Incorporation of clinical factors (especially USMR) with molecular testing may increase the sensitivity for detection of malignancy. Introduction of molecular testing offers some clinical benefits even in a low resection rate setting, and directly influences surgical decision-making. This study illustrates the importance of the local diagnostic pathway in ensuring appropriate integrated use of molecular testing for best outcomes.

Introduction

With neck ultrasound, thyroid nodules are found in more than 20% of the population,¹ but have a low risk of malignancy. Cytologically indeterminate thyroid nodules (ITNs) present an inherent diagnostic problem because capsular or vascular invasion cannot be assessed in fine-needle aspiration cytology (FNAC) specimens. Since 2012, molecular testing has increasingly become an important part of the diagnostic pathway,^2
–4 but cannot be used in isolation. However, published evaluations of molecular test performance lack details about additional pretest malignancy risk assessment based on clinical presentation, history, palpation, and ultrasound, which are needed to define the additional diagnostic value of molecular testing.^5
–7

Most commercially available molecular tests for ITNs are marketed based on high negative predictive value (NPV) and benign call rate (BCR), assuming that patients with test negative nodules avoid surgery because of the molecular test result.^8
–10 However, before introduction of molecular testing, average resection rates are ∼39% for Bethesda III (atypia or follicular lesion of undetermined significance, AUS/FLUS) and 70% for Bethesda IV (follicular neoplasm/suspicious for follicular neoplasm, FN/SFN) nodules,¹¹ which is higher than the 21% for AUS/FLUS and 56% for FN/SFN nodules in our setting before implementation of molecular testing.¹² In fact, if each positive molecular test result led to surgery and each negative molecular test result prevented surgery, the resection rate would have increased in our setting causing “overtreatment” as reported elsewhere.^13,14

The different resection rates and the idiosyncrasies of local thyroid nodule diagnostic pathways, which are incompletely described in most FNAC validation studies of thyroid nodule molecular tests, contribute to the significant interinstitution discrepancies in test performance that have been reported for commercially available molecular tests.¹⁵

This study assesses outcomes for thyroid nodule molecular testing on FNAC specimens in Southern Alberta, Canada, with government-funded health care and centralized cytopathology and thyroid nodule triage for the entire population. We previously determined the local resection and malignancy rates in each Bethesda cytology category in this setting, namely 21% and 26% for AUS/FLUS and 56% and 43% for FN/SFN, respectively.¹² We also previously implemented ultrasound malignancy risk (USMR) assessment in January 2018.¹⁶ Building on these initial steps, we locally developed and analytically validated ThyroSPEC, a low-cost ($700 USD for out-of-province referrals) molecular diagnostic test for ITN FNACs. We subsequently adopted reflex reimbursed molecular testing for all AUS/FLUS and FN/SFN residual FNAC specimens along with a local thyroid nodule diagnostic pathway implemented with the Calgary Primary Care Networks (Supplementary Fig. S1).

The primary aim of this study is to prospectively clinically validate the accuracy of a novel molecular test in indeterminate FNAC, and to determine the incremental impact of additional diagnostic factors in the thyroid nodule diagnostic pathway with a multivariable regression model. The secondary aim is to measure the impact of the introduction of molecular testing into an entire health care system with a previously optimized thyroid nodule diagnostic pathway.⁴ This will be performed by comparing FNAC diagnoses and surgical overtreatment in consecutively diagnosed patients, before and after the introduction of molecular testing.

Methods

Study population

A prospective observational study was conducted. All patients in Southern Alberta with an AUS/FLUS or FN/SFN thyroid nodule diagnosed from July 30, 2020, until July 31, 2022, were included. All patients underwent ultrasound-guided fine-needle aspiration (FNA) processed by Hologic liquid-based method, according to thyroid nodule USMR assessment. When nodules were diagnosed as AUS/FLUS or FN/SFN, DNA and RNA were extracted from residual FNA material (not a further dedicated pass) using Qiagen's AllPrep DNA/RNA/miRNA Universal Kit using the residual ThinPrep preservation liquid (CytoLyt^®) from the original FNAC.

Data collection

Electronic health records were used for this study (IRB approval: HREBA.CHC-20-0068) to determine demographics, laboratory results, clinical history, surgical decision-making, ultrasound findings, cytology diagnoses, molecular test results, outcomes, surgical procedure, histopathologic findings based on standardized synoptic reporting, and follow-up (Fig. 1). In five cases, the reasons for surgery were unclear, and so, the responsible surgeon was contacted. Data were aggregated in a custom REDCap database with a data cutoff of July 31, 2022, for this study, and the database continues to be updated on an ongoing basis.

FIG. 1.

Flowchart study population summary.

This study uses real-world histology diagnoses with standardized routine histopathologic reports, including review by a second pathologist, to reflect the existing real-world diagnostic setting. Pathologists had access to the molecular test result through electronic health records.

Molecular testing

Of 615 nodules, 580 (94%) passed quality control (Fig. 1) and were analyzed by ThyroSPEC, a locally developed MassARRAY test (validation summary provided in the Supplementary Data) that interrogates the 139 most prevalent mutations and gene fusions in thyroid cancer (Supplementary Table S1 in the Supplementary Data) using residual material from liquid FNAC or air-dried smears.¹⁷ Mutation profiling was performed with Agena iPlex Pro chemistry according to the manufacturer's (Agena) protocol. This was performed in a College of Physicians and Surgeons of Alberta (CPSA)-accredited laboratory (Alberta Precision Laboratories). Reporting of ThyroSPEC results was performed according to the specific alterations identified and grouped in one of five risk categories with an associated treatment recommendation (Supplementary Table S2 in the Supplementary Data).

Impact of molecular testing on the thyroid nodule diagnostic pathway

Cytologic features, resection rates, diagnostic yield, and surgical decision-making were reported according to data from electronic health records.

Two historical cohorts were used for baseline comparisons to assess the impact of molecular testing on resection rates, diagnostic yield, and cytology diagnoses. The 2017–2020 cohort consists of 1006 consecutive ITNs collected from May 2017 until February 2020 in the same health care region, and 282 of 314 resected ITNs passed quality control and were tested retrospectively on ThyroSPEC using residual cytologic material when histologic diagnosis was available due to surgical resection (IRB approval: HREBA.CC 2000421). The 2017–2020 cohort was used as a baseline for comparison of resection rates and diagnostic yield. The 2010–2013 cohort¹² consisted of 1029 consecutive ITNs diagnosed from January 2010 until December 2013 in the same health care region, with no molecular testing. The 2010–2013 cohort was used as a baseline for comparison of cytology diagnoses.

Although demographic data are not available for all cohorts, the cohorts are large and consecutively collected from the same population and therefore expected to be clinically similar. The COVID-19 pandemic caused a general reduction of surgery capacity but was not cited in clinician notes as a factor in triage or surgical decision-making.

Statistical analyses

ThyroSPEC test performance was calculated in two ways for sensitivity, specificity, NPV, and positive predictive value (PPV). Method 1 used mutation status as the test result and the histopathologic diagnosis (noninvasive follicular thyroid neoplasm with papillary-like nuclear features [NIFTP] considered malignant) as the gold standard. Method 2 used mutation status as the test result and histopathologic diagnosis or an assumed benign diagnosis after more than 1-year follow-up without resection as gold standards. Method 1 necessarily overestimates the PPV and underestimates the NPV (i.e., many benign nodules are excluded since they are resected at a lower rate). Conversely, method 2 necessarily overestimates the NPV and underestimates the PPV.¹⁸ Therefore, we provide ThyroSPEC test performance as a range for each metric, bookending the ranges with methods 1 and 2.

All statistical analyses were conducted using the R statistical software package.¹⁹ Continuous variables were expressed using medians and interquartile ranges (IQRs), while nominal variables were presented as frequency counts and percentages. ThyroSPEC test sensitivity, specificity, PPV, NPV, and success rate, as well as the ITN resection rate and malignancy rate, were reported with confidence intervals (CIs). The significance of variables was evaluated using t-test for continuous variables and Fisher's exact test or chi-square test for proportions.

A multivariable logistic regression model was built using the R package MASS to identify clinical factors that are highly associated with malignancy of ITNs. The data set was randomly split into training (80%) and testing (20%) sets using 10-fold cross-validation (R package caret). The model's performance was evaluated using receiver operating characteristic (ROC) curve analysis (R package ROCR), and the area under the curve (AUC) was calculated for both the training and testing sets. The optimal threshold point on the ROC curve, along with its corresponding CI, was provided for the test set. A p-value of <0.05 was considered statistically significant.

Results

Study population

Median nodule volume was 6.1 mL (IQR 2.6–20.2 mL), median largest dimension was 2.4 cm (IQR 1.6–3.5 cm), median patient age was 54 years (IQR 43–65 years), and median length of follow up was 13 months (IQR 7–19 months). Genomic variants were detected in NRAS (83), HRAS (30), KRAS (17), BRAF (16), SPOP (14), and PAX8/PPARG (7) as well as less common variants (Supplementary Fig. S2). Common histologic diagnoses included 104 benign, 49 papillary thyroid carcinomas, 13 NIFTPs, 10 follicular thyroid carcinomas (FTCs), and 6 Hurthle cell carcinomas (Supplementary Fig. S3).

ThyroSPEC clinical validation

Only including resected nodules, ThyroSPEC had a sensitivity and specificity of 74% [CI 60–84%] and 67% [CI 56–77%] for AUS/FLUS and 67% [CI 43–85%] and 79% [CI 60–92%] for FN/SFN, respectively (Table 1). Specificity was 78% [CI 73–83%] in AUS/FLUS and 79% [CI 60–92%] in FN/SFN when unresected nodules were assumed to be benign after following the patient for >1 year. Prevalence of malignancy in resected nodules increased monotonically in each of five classes of mutation (defined in Supplementary Table S2 in the Supplementary Data) from 29% (n = 90, [CI 20–38%]) for no mutation to 100% (n = 4) for high-risk mutations (Table 2).

Table 1.

ThyroSPEC Test Performance Range from Resected Nodules Only, to All Resected Nodules and Unresected Nodules Assumed Benign with At Least 1 Year of Data Following the Patient

Cytology	Prevalence of malignancy	n	Sensitivity	Specificity	NPV	PPV
All indeterminate	21–44%	179–377	72% [CI 60–81%]	70–78% [CI 60–83%]	76–91% [CI 66–95%]	46–65% [CI 37–75%]
Bethesda III (AUS/FLUS)	17–41%	139–327	74% [CI 60–84%]	67–78% [CI 56–83%]	79–93% [CI 67–96%]	42–61% [CI 32–72%]
Bethesda IV (FN/SFN)	42–53%	40–50	67% [CI 43–85%]	79–84% [CI 60–97%]	70–77% [CI 47–90%]	70–82% [CI 46–96%]
ATA low suspicion, TR3	20–35%	52–88	72% [CI 47–90%]	56–83% [CI 38–91%]	79–92% [CI 58–97%]	46–52% [CI 28–72%]
ATA intermediate suspicion, TR4	22–49%	71–161	80% [CI 63–92%]	69–79% [CI 52–86%]	78–93% [CI 60–97%]	52–72% [CI 38–85%]
ATA high suspicion, TR5	17–41%	32–78	46% [CI 19–75%]	83–84% [CI 72–97%]	70–89% [CI 47–95%]	35–67% [CI 14–93%]

NIFTP is treated as cancer because NIFTP is a surgical diagnosis. The first row includes all indeterminates, the rows below stratify the cohort by cytology or USMR (USMR category). Forty-one nodules not risk stratified by ACR-TIRADS or ATA classifications were retrospectively classified as per ACR-TIRADS according to the radiology report as described previously.

ACR-TIRADS, American College of Radiology Thyroid Imaging Reporting and Data Systems; ATA, American Thyroid Association; AUS/FLUS, atypia or follicular lesion of undetermined significance; CI, confidence interval; FN/SFN, follicular neoplasm/suspicious for follicular neoplasm; NIFTP, noninvasive follicular thyroid neoplasm with papillary-like nuclear features; NPV, negative predictive value; PPV, positive predictive value; USMR, ultrasound malignancy risk.

Table 2.

Proposal for Revised Standardized Management Options for Thyroid Nodules with Atypia or Follicular Lesion of Undetermined Significance and Follicular Neoplasm/Suspicious for Follicular Neoplasm Cytology in Our Setting Based on the Significant Results (p < 0.05) for Ultrasound Malignancy Risk and ThyroSPEC in the Multivariable Regression Model for This Study

Category	Prevalence of malignancy in this study	Management options
Category	Prevalence of malignancy in this study	ATA benign—low risk, ACR-TIRADS 1–3	ATA intermediate—high risk, ACR-TIRADS 4–5
Failed quality control	40% (n = 5)	While noting the lower risk in this USMR category, management should also account for cytology diagnosis and clinical assessment.	While noting the higher risk in this USMR category, management should also account for cytology diagnosis and clinical assessment.
Benign molecular marker	33% (n = 3)	Molecular testing has not changed management recommendations based on ultrasound result, cytology diagnosis and clinical assessment. Surveillance preferred based on ultrasound result.	Molecular testing has not changed management recommendations based on ultrasound result, cytology diagnosis and clinical assessment.
No mutation detected	29% (n = 90, [CI 20–38%])
Intermediate-risk mutation	60% (n = 72, [CI 48–71%])	Refer to endocrinology to discuss diagnostic lobectomy or ongoing surveillance suggested by ultrasound result.	Ultrasound and molecular results argue for diagnostic lobectomy.
Malignant molecular marker	90% (n = 10)	Total thyroidectomy or lobectomy depending on size or observation in a clinical study depending on localization.	Total thyroidectomy or lobectomy depending on size or observation in a clinical study depending on localization.
High-risk mutations	100% (n = 4)	Total thyroidectomy or lobectomy depending on size or observation in a clinical study depending on localization. For TP53, diagnostic lobectomy may be considered.	Total thyroidectomy or lobectomy depending on size or observation in a clinical study depending on localization.

Less aggressive treatment is recommended for nodules 1 cm or less. See Supplementary Table S2 in the Supplementary Data for the predefined standardized recommendations for each genetic alteration detected by ThyroSPEC in this study.

Multivariable regression model

Method of nodule discovery

Based on a post hoc analysis, the source of initial nodule discovery significantly correlates with resection rates and malignancy risk (Supplementary Fig. S4). Nodules discovered by palpation have a 56% (39/70) malignancy rate compared with 40% (46/114) for nodules discovered by other means, which is 38% higher in relative terms (p = 0.0424).

Regression model accuracy

The regression analysis results are provided in Table 3. ThyroSPEC class and USMR category are significantly associated with malignancy, whereas Bethesda IV diagnosis and palpation as the method of nodule discovery show an insignificant trend toward positive association with malignancy, but with odds ratios of 2.64 [CI 0.89–7.84] and 2.19 [CI 0.90–5.33], respectively, when compared with Bethesda III and nodule discovery without palpation. The regression model had an AUC of 0.763 for the training set and an AUC of 0.718 for the test set. At the optimal threshold point on the ROC curve for the test set, the model demonstrates a sensitivity of 90% [CI 55.5–99.8%] and a specificity of 55% [CI 31.5–76.9%]. The ROC curve is depicted in Figure 2.

FIG. 2.

Multivariable regression model: ROC curve for (A) train set and (B) test set. ROC, receiver operating characteristic.

Table 3.

Multivariable Regression Model Evaluating the Risk of Malignancy Based on ThyroSPEC Test Positive Versus Test Negative, Ultrasound Malignancy Risk Category, Bethesda Category, and Palpation Discovery

Discretized variable	Coefficient ± SE	OR [CI]	p
ThyroSPEC test positive (vs. test negative)	1.77 ± 0.48	5.90 [2.30–15.09]	<0.001
USMR Category 4–5 (vs. 1–3)	1.35 ± 0.51	3.86 [1.43–10.41]	0.008
Bethesda Category BIV (vs. BIII)	0.97 ± 0.55	2.64 [0.89–7.84]	0.079
Palpation discovery yes (vs. no)	0.78 ± 0.45	2.19 [0.90–5.33]	0.085

OR, odds ratio; SE, standard error.

Impact of molecular testing on the thyroid nodule diagnostic pathway

Cytology

Between July 2020 and July 2022, the percentage of total FNAC diagnosed indeterminate decreased to 10%, compared with 2010–2013 with 18% ITNs (p < 0.0001). Cytologic atypia and architectural/nuclear features are described in 140/439 (36%) and 15/439 (4%) of ITNs.

Repeat FNAs decreased 24% (p = 0.05) following implementation of molecular testing compared with the 2010–2013 cohort, although the time gap between these cohorts means that implementation of the 2017 Bethesda guideline between those cohorts is a confounding factor. We note that the local cytopathology practice discontinued the explicit recommendation to repeat FNAC in the cytology reports for AUS/FLUS nodules when molecular testing was implemented, which likely reduced repeat FNAs. More than 50% of repeat FNAs clarify an initial AUS/FLUS diagnosis (Fig. 3A), before and after ThyroSPEC implementation. Furthermore, the resection rate for Bethesda III and IV nodules both increased since 2017–2020, although not significantly (Fig. 3B).

FIG. 3.

(A) Diagnosis of repeat FNACs of initially AUS/FLUS nodules. Second diagnosis when a nodule initially diagnosed AUS/FLUS undergoes repeat FNACs. Six of sixty-one FNACs with molecular testing had an intermediate-risk mutation, the rest were test negative on ThyroSPEC. (B) Resection rates before and after introduction of molecular testing. AUS/FLUS, atypia or follicular lesion of undetermined significance; FNAC, fine needle aspiration cytology.

Surgical decision-making

The mutation was the primary reason for surgery in 76% (78/103) of resected molecular test-positive patients according to surgeon notes, demonstrating direct influence of molecular testing on clinical decision-making. Whereas the primary reason 12/42 (29%) for deciding against surgery (despite a positive molecular test result) is patient preference, with a stable or decreasing USMR stratification category on follow-up ultrasound nearly as common at 11/42 (26%). Less common reasons for deciding against surgery despite a positive molecular test result were 7/42 (17%) patients with small nodule size or low-risk US features and 7/42 (17%) patients not being surgical candidates due to age, comorbidities, or more serious medical problems. Of the 106 resected molecular test-negative patients, the primary reason for surgery 38/106 (36%) was ultrasound or cytology risk categories (FN/SFN or high-risk ultrasound features), with patient preference the primary reason for surgery in 28/106 (26%) patients.

Of 99 histologically benign nodules, 31 (31%) were resected for indications other than risk of malignancy and 68 (69%) were resected for suspected malignancy. In this study, only 11/615 (2%) patients received completion thyroidectomies following a malignant result in a diagnostic hemithyroidectomy.

The resection rate for ITNs increased a relative 13% (p = 0.09) compared with the 2017–2020 cohort (before implementation of molecular testing), from 314/1006 (31% [CI 28–34%]) to 217/615 (35% [CI 32–39%]) of all patients with ITNs who underwent surgery or were scheduled for surgery. The diagnostic yield (histologic malignancy rate of resected nodules) increased a relative 14% (p = 0.2442) compared with the 2017–2020 cohort, from 71/193 (37% [CI 30–44%]) in AUS/FLUS and 34/83 (41% [CI 30–52%]) in FN/SFN to 59/143 (41% [CI 33–49%]) in AUS/FLUS and 21/41 (51% [CI 36–67%]) in FN/SFN.

Patients with high-risk mutations had surgery 78 days (median, n = 15, IQR 58–116 days) after molecular testing versus 184 days (median, n = 97, IQR 112–264 days) for ThyroSPEC-negative patients, a 58% (p = 0.0001) shorter wait for high-risk patients. Average turnaround time for ThyroSPEC was 7.6 days.

Of the patients undergoing ongoing surveillance, 123/291 (42%) had a follow-up ultrasound, 31/291 (11%) had a follow-up FNA, 40/291 (14%) had both a follow-up ultrasound and follow-up FNA, and 97/291 (33%) had no follow-up FNA or ultrasound by the study cutoff.

Discussion

ThyroSPEC clinical validation

ThyroSPEC achieved a 94% [CI 92–96%] success rate for analysis of liquid FNAC specimens despite the use of only residual material without a dedicated FNA pass or custom preservative.^20,21

Inter- and intraobserver histopathologic diagnosis variability is known to be high, especially for the differential diagnosis of minimally invasive FTC versus benign follicular cell-derived tumors. This may create opportunities for genomic data to inform or even revise the histologic diagnoses.^22

–27 Also, the prevalence and malignancy risk of specific mutations such as BRAF and RAS vary by geography.²⁸ With these complexities in mind, we recommend that patients and clinicians take into account the whole diagnostic picture, including ultrasound, cytology, clinical assessment, and patient preferences, in addition to mutations, when deciding to undergo or avoid surgery (Supplementary Table S2 in the Supplementary Data).²⁹

ThyroSPEC relies on local mutation-specific risk of malignancy when available for more precise management,¹² but there remains a need to estimate PPV of infrequent variants from pooled cohorts because the local mutation-specific PPV for variants beyond BRAF and RAS requires enormous ITN sample sizes.^5,30,31 A mutation in RAS (NRAS/HRAS/KRAS) has a prevalence of 22% (130/580) and malignancy risk of 58% (36/62) in ITNs in this study, well within the broad range found in the literature.³² A RAS mutation increases the malignancy risk of an ITN, but resecting every RAS ⁺ nodule would result³³ in an increase in resections and overtreatment of benign nodules in our setting.²

Although the RAS ⁺ malignancy risk is within the Bethesda V range where surgery is recommended by the guidelines,² Lupo et al.³⁴ demonstrated that RAS ⁺ indeterminate nodules can be further risk stratified.³⁵ Therefore, further risk stratification methods should be explored in our setting to improve treatment of nodules with intermediate-risk mutations.

Valderrabano et al. showed that commercially available molecular tests have higher rates of false negatives and a lower NPV in higher prevalence of malignancy institutions.³⁶ Using ThyroSeq v3 as a benchmark for a more expensive test with 92% sensitivity and 41% specificity,³⁷ the NPV and PPV at our 46% prevalence of malignancy in indeterminates would be 86% and 57%, respectively. Sensitivity of 92% would proportionately increase test-positive calls compared with ThyroSPEC; we estimate 221 test positives compared with the 173 found in this study (based on the ratio of ThyroSeq v3 sensitivity to ThyroSPEC sensitivity). Therefore, local prevalence of malignancy and resection rates for each cytologic category are needed for appropriate interpretation of molecular test results.

Multivariable regression model

Ranked variable importance (Table 3) demonstrates how crucial molecular testing is for preoperative malignancy risk stratification of ITNs. USMR categories had a significant incremental risk stratification benefit beyond molecular testing alone, in line with Hu et al.,³⁸ which evaluated USMR categories to inform molecular testing, but contrasting with the results of Figge et al.,³⁹ which applied USMR categories in the context of a regression model as performed here. The American Thyroid Association (ATA) and Thyroid Imaging Reporting and Data Systems (TIRADS) categories are assessed independently in Figge et al., diluting the positive trends found in each USMR classification system.

However, the method of nodule discovery and the difference between Bethesda categories III and IV have an insignificant association with malignancy risk. Nodule volume, sex, and age had little to no association with malignancy risk according to the multivariable regression analysis. Molecular test results and USMR have a significant association with malignancy risk and can be used to personalize surgical decision-making according to locally developed standardized management options (Table 2).

Contrary to the Bethesda guidelines and our previous study,¹² in this study, there was only a 10% absolute or 24% relative difference between the AUS/FLUS and FN/SFN categories in the prevalence of malignancy in resected nodules. Moreover, the lack of reporting on cytologic atypia in this study unfortunately prevents comparison with published analyses showing architectural and nuclear features in 1162/3532 (33%) of ITNs as having an incremental diagnostic benefit.⁴⁰ This emphasizes to our local cytopathologists the importance of stringency in separating the AUS/FLUS and FN/SFN categories, and systematically reporting all cytopathologic features, for accurate risk stratification.¹⁶ A greater discrepancy between indeterminate categories could be an important incremental risk stratification tool for determining preoperative malignancy risk alongside molecular markers, USMR, and potentially method of detection.

Impact of molecular testing on the thyroid nodule diagnostic pathway

Unlike the increase in indeterminate diagnoses reported in some centers following implementation of molecular testing,⁴¹ our setting has observed a significant decrease in the proportion of thyroid nodules diagnosed cytologically indeterminate since 2010–2013. Therefore, this study avoided the potential flaw of washing out the advantages of molecular testing with an increased incidence of indeterminate FNACs because of molecular testing availability.

Stewart et al.⁴² reported a large, single-center series of ITNs with cytology data without molecular testing and without reporting USMR stratification. Our study had less overtreatment at 68/615 (11% compared with 42% in Stewart et al., p < 0.0001) overtreated benign nodules with no other indication for surgery. Of 99 total histologically benign nodules, the rate of surgery for other indications was 5% (31/615) in our study versus 42% in Stewart et al. (p < 0.0001). Our study also had less undertreatment, with 11/615 (2%) patients receiving completion thyroidectomies following a malignant result in a diagnostic hemithyroidectomy compared with 11% in Stewart et al. (p < 0.0001). In total, 13% of patients in our study received suboptimal initial treatment compared with 54% in the Stewart et al. study (p < 0.0001); the difference is likely attributable to molecular diagnostics, USMR stratification, and specific lobectomy criteria⁴³ in our setting.

Future directions

In our low resection rate setting, higher sensitivity at the price of lower specificity would induce overtreatment. This consideration, along with cost, is an impediment to improving the sensitivity of ThyroSPEC by adding less specific biomarkers such as copy number alterations, which would increase the number of true and false positives.

This study suggests that reflexive testing of all ITNs may be a suboptimal allocation of limited health care resources. We propose targeting molecular testing to subsets of ITNs in three settings, each is incorporated into a flowchart (Fig. 4) that bridges the diagnostic pathway between the end of the upstream flowchart (Supplementary Fig. S1) and the clinical options recommended in Table 2.

FIG. 4.

Proposed clinical algorithm for allocation of molecular testing via checkpoints to limit testing of nodules where management may be determined decisively through other means. Comorbidities or patient preferences are examples of reasons unrelated to malignancy for surgery or no surgery. ROM, risk of malignancy.

Repeat FNAs of AUS/FLUS nodules are recommended by the ATA guideline² and in our setting provide a definitive diagnosis on cytology in ∼50% of initially AUS/FLUS nodules (Fig. 3A), thereby diverting half of the nodules from molecular testing. Repeat FNAs as a first step before molecular testing⁴⁴ have been reported to reduce overtreatment while missing only rare, low-risk tumors.⁸

Patient preferences and other diagnostic criteria resulted in 42/172 (24%) patients avoiding surgery despite a positive molecular test result and 106/408 (26%) patients undergoing surgery despite a negative molecular test result. Long-term follow-up is needed to confirm the outcomes of these patients. Based on clinical guidelines, molecular testing should not be performed where the result will not affect management.^2,3

In accordance with published data,³⁸ NPV of molecular testing is highest in ATA intermediate-suspicion/TR4 nodules where testing is most likely to prevent surgeries, but has minimal incremental risk stratification benefit in ATA high-suspicion/TR5 nodules, which are often already indicated for surgery pretest (Table 1). We note the wide CIs as a limitation of subgroup analysis, particularly for surprising results such as the low sensitivity in ATA high-suspicion/TR5 nodules. If confirmed in a larger sample, our results suggest that ATA intermediate-suspicion/TR4 nodules should be prioritized for molecular testing.

Strengths and limitations

The strengths of this study include a consecutive, population-based cohort, rather than limited by center, which reduces the risk of selection bias and mitigates verification bias.³⁷ The centralization of cytopathology, and thyroid nodule referral and implementation of USMR assessment in this 1.5 million population health care region ensures standardization.

The weaknesses of this study include limited follow-up for many patients, limited statistical validity for some subset analyses, low molecular test sensitivity, a low resection rate limiting generalizability,¹¹ and a possible bias in that molecular testing was available to pathologists during histopathologic diagnosis.

Conclusion

This study clinically validated a novel molecular test and assessed the incremental benefit of reimbursed molecular testing of ITNs as part of an optimized thyroid nodule diagnostic pathway for a health care region with 1.5 million inhabitants. The results of this study illustrate the necessity of evaluation and optimization of local diagnostic pathways to ensure appropriate, successful, and integrated use of molecular testing for best outcomes.

Footnotes

Acknowledgments

Craig Ivany, Chris Naugler, and Alberta Precision Laboratories for implementing ThyroSPEC clinically soon after development and validation. Paula Seal for improving the local radiology practice for thyroid nodule malignancy risk assessment. Sana Ghaznavi for the determination of Bethesda FNA malignancy rates.

Authors' Contributions

P.S.: conceptualization, data curation, formal analysis, investigation, methodology, validation, visualization, and writing (original draft). M.E.: conceptualization, investigation, methodology, supervision, validation, and writing (review and editing). J.W.: data curation, formal analysis, software, and writing (review and editing). M.K.: investigation, resources, and writing (review and editing). A.B.: investigation, supervision, resources, and project administration. M.P.: investigation and project administration. Z.P.: investigation, formal analysis, and validation. B.Z.: investigation and validation. R.S.: investigation. D.J.D.: supervision, project administration, and resources. R.P.: conceptualization, funding acquisition, investigation, methodology, project administration, supervision, validation, and writing (review and editing).

Author Disclosure Statement

M.E. and R.P. receive licensing fees for ThyroSPEC. All other authors have no disclosures.

Funding Information

Grant funding was provided by The Thyroid Foundation of Canada, Genome Alberta, and the Alberta Cancer Foundation. Molecular testing costs postimplementation of ThyroSPEC were borne by Alberta Precision Laboratories (a crown corporation).

Supplementary Material

Supplementary Figure S1

Supplementary Figure S2

Supplementary Figure S3

Supplementary Figure S4

Supplementary Data

References

, Ming

, Tian

, et al. Mapping global epidemiology of thyroid nodules among general population: A systematic review and meta-analysis. Front Oncol, 2022; 12:1029926; doi: 10.3389/fonc.2022.1029926

Haugen

, Alexander

, Bible

, et al. 2015 American Thyroid Association Management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid, 2016; 26(1):1–133; doi: 10.1089/thy.2015.0020

Patel

, Yip

, Lubitz

, et al. The American Association of Endocrine Surgeons guidelines for the definitive surgical management of thyroid disease in adults. Ann Surg, 2020; 271(3):e21–e93; doi: 10.1097/SLA.0000000000003580

Durante

, Hegedus

, Czarniecka

, et al. 2023 European Thyroid Association clinical practice guidelines for thyroid nodule management. Eur Thyroid J, 2023; 12(5):e230067; doi: 10.1530/ETJ-23-0067

Stewardson

, Eszlinger

, Paschke

. Diagnosis of endocrine disease: Usefulness of genetic testing of fine-needle aspirations for diagnosis of thyroid cancer. Eur J Endocrinol, 2022; 187(3):R41–R52; doi: 10.1530/EJE-21-1293

Valderrabano

, Eszlinger

, Stewardson

, et al. Clinical value of molecular markers as diagnostic and prognostic tools to guide treatment of thyroid cancer. Clin Endocrinol (Oxf), 2023; 98(6)753–762; doi: 10.1111/cen.14882

Paschke

, Eszlinger

, Kopp

Euthyroid and Hyperthyroid Nodules and Goiter. In: DeGroot's Endocrinology. (Robertson RP. ed.). Elsevier: Netherlands, 2022; pp. 1277–1297.

Nishino

, Mateo

, Kilim

, et al. Repeat fine needle aspiration cytology refines the selection of thyroid nodules for Afirma gene expression classifier testing. Thyroid, 2021; 31(8):1253–1263; doi: 10.1089/thy.2020.0969

Steward

, Carty

, Sippel

, et al. Performance of a multigene genomic classifier in thyroid nodules with indeterminate cytology: A prospective blinded multicenter study. JAMA Oncol, 2019; 5(2):2374–2445; doi: 10.1001/jamaoncol.2018.4616

10.

Patel

, Angell

, Babiarz

, et al. Performance of a genomic sequencing classifier for the preoperative diagnosis of cytologically indeterminate thyroid nodules. JAMA Surg, 2018; 153(9):2168–6262; doi: 10.1001/jamasurg.2018.1153

11.

Bongiovanni

, Spitale

, Faquin

, et al. The Bethesda System for Reporting Thyroid Cytopathology: A meta-analysis. Acta Cytol, 2012; 56(4):333–339; doi: 10.1159/000339959

12.

Ghaznavi

, Clayton

, Eszlinger

, et al. Accuracy of thyroid fine-needle aspiration cytology: A cyto-histologic correlation study in an integrated Canadian health care region with centralized pathology service. Acta Cytol, 2022; 66(3):171–178; doi: 10.1159/000521562

13.

Sacks

, Bose

, Zumsteg

, et al. Impact of Afirma gene expression classifier on cytopathology diagnosis and rate of thyroidectomy. Cancer Cytopathol, 2016; 124(10):722–728; doi :10.1002/cncy.21749

14.

Khan

, Zeiger

. Thyroid nodule molecular testing: Is it ready for prime time?. Front Endocrinol (Lausanne), 2020; 11:590128; doi: 10.3389/fendo.2020.590128

15.

Marcadis

, Valderrabano

, Ho

, et al. Interinstitutional variation in predictive value of the ThyroSeq v2 genomic classifier for cytologically indeterminate thyroid nodules. Surgery, 2019; 165(1):17–24; doi: 10.1016/j.surg.2018.04.062

16.

, Wu

, Seal

, et al. Improvement in thyroid ultrasound report quality with radiologists' adherence to 2015 ATA or 2017 TIRADS: A population study. Eur Thyroid J, 2022; 11(3):e220035; doi: 10.1530/ETJ-22-0035

17.

Stewardson

, Eszlinger

, Wu

, et al. Prospective evaluation of the ThyroSPEC™ mutation panel for the diagnosis of indeterminate thyroid fine needle aspiration cytologies (FNAC) in the Southern Alberta Health Care Region. Thyroid, 2019; 29(S1):A–133.

18.

Tee

, Lowe

, Brand

, et al. Fine-needle aspiration may miss a third of all malignancy in palpable thyroid nodules: A comprehensive literature review. Ann Surg, 2007; 246(5):714–720; doi: 10.1097/SLA.0b013e3180f61adc

19.

R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing: Vienna, Austria; 2016. Available from: https://www.R-project.org [Last accessed: January 12, 2023 ].

20.

Partyka

, Randolph

, Lawrence

, et al. Utilization of direct smears of thyroid fine-needle aspirates for ancillary molecular testing: A comparison of two proprietary testing platforms. Diagn Cytopathol, 2018; 46(4):320–325; doi: 10.1002/dc.23902

21.

Nikiforova

MA-O

, Lepe

MA-O

, Tolino

, et al. Thyroid cytology smear slides: An untapped resource for ThyroSeq testing. Cancer Cytopthol, 2021; 129(1):33–42; doi: 10.1002/cncy.22331

22.

Eszlinger

, Piana

, Moll

, et al. Molecular testing of thyroid fine-needle aspirations improves presurgical diagnosis and supports the histologic identification of minimally invasive follicular thyroid carcinomas. Thyroid, 2015; 25(4):401–409; doi: 10.1089/thy.2014.0362

23.

Hirokawa

, Carney

, Goellner

, et al. Observer variation of encapsulated follicular lesions of the thyroid gland. Am J Surg Pathol, 2002; 26(11):1508–1514; doi: 10.1097/00000478-200211000-00014

24.

Lloyd

, Erickson

, Casey

, et al. Observer variation in the diagnosis of follicular variant of papillary thyroid carcinoma. Am J Surg Pathol, 2004; 28(10):1336–1340; doi: 10.1097/01.pas.0000135519.34847.f6

25.

Franc

, de la Salmonière

, Lange

, et al. Interobserver and intraobserver reproducibility in the histopathology of follicular thyroid carcinoma. Hum Pathol, 2003; 34(11):1092–1100; doi: 10.1016/s0046-8177(03)00403-9

26.

Cibas

, Baloch

, Fellegara

, et al. A prospective assessment defining the limitations of thyroid nodule pathologic evaluation. Ann Intern Med, 2013; 159(5):325–332; doi: 10.7326/0003-4819-159-5-201309030-00006

27.

Craig

, Khalil

, Eszlinger

, et al. Malignancy is in the eye of the beholder: Pathologic diagnosis of challenging follicular neoplasms in the era of noninvasive follicular thyroid neoplasms with papillary-like nuclear features and immunohistochemical and molecular adjuncts. Surgery, 2021; 169(1):22–26; doi: 10.1016/j.surg.2020.04.004

28.

Kakudo

Different threshold of malignancy for RAS-like thyroid tumors causes significant differences in thyroid nodule practice. Cancers (Basel), 2022; 14(3):812; doi: 10.3390/cancers14030812

29.

Medici

, Kwong

, Angell

, et al. The variable phenotype and low-risk nature of RAS-positive thyroid nodules. BMC Med, 2015; 13:184; doi: 10.1186/s12916-015-0419-z

30.

Yip

Thoughtful utilization of molecular testing in refining thyroid nodule risk assessment: Do not throw out the baby. Thyroid, 2020; 30(4):474–475; doi: 10.1089/thy.2020.0148

31.

Goldner

, Angell

, McAdoo

, et al. Molecular variants and their risks for malignancy in cytologically indeterminate thyroid nodules. Thyroid, 2019; 29(11):1594–1605; doi: 10.1089/thy.2019.0278

32.

Nabhan

, Porter

, Lupo

, et al. Heterogeneity in positive predictive value of RAS mutations in cytologically indeterminate thyroid nodules. Thyroid, 2018; 28(6):729–738; doi: 10.1089/thy.2017.0635

33.

Guan

, Toraldo

, Cerda

, et al. Utilities of RAS mutations in preoperative fine needle biopsies for decision making for thyroid nodule management: Results from a single-center prospective cohort. Thyroid, 2020; 30(4):536–547; doi: 10.1089/thy.2019.0116

34.

Lupo

, Walts

, Sistrunk

, et al. Multiplatform molecular test performance in indeterminate thyroid nodules. Diagn Cytopathol, 2020; 48(12):1254–1264; doi: 10.1002/dc.24564

35.

Finkelstein

, Sistrunk

, Malchoff

, et al. A retrospective evaluation of the diagnostic performance of an interdependent pairwise microRNA expression analysis with a mutation panel in indeterminate thyroid nodules. Thyroid, 2022; 32(11):1362–1371; doi: 10.1089/thy.2022.0124

36.

Valderrabano

, Leon

, Centeno

, et al. Institutional prevalence of malignancy of indeterminate thyroid cytology is necessary but insufficient to accurately interpret molecular marker tests. Eur J Endocrinol, 2016; 174(5):621–629; doi: 10.1530/EJE-15-1163

37.

DiGennaro

, Vahdatzad

, Jalali

, et al. Assessing bias and limitations of clinical validation studies of molecular diagnostic tests for indeterminate thyroid nodules: Systematic review and meta-analysis. Thyroid, 2022; 32(10):1144–1157; doi: 10.1089/thy.2022.0269

38.

, Nguyen

, Patel

, et al. The effect modification of ultrasound risk classification on molecular testing in predicting the risk of malignancy in cytologically indeterminate thyroid nodules. Thyroid, 2022; 32(8):905–916; doi: 10.1089/thy.2021.0659

39.

Figge

, Gooding

, Steward

, et al. Do ultrasound patterns and clinical parameters inform the probability of thyroid cancer predicted by molecular testing in nodules with indeterminate cytology?. Thyroid, 2021; 31(11):1673–1682; doi: 10.1089/thy.2021.0119

40.

Valderrabano

, Khazai

, Thompson

, et al. Cancer risk associated with nuclear atypia in cytologically indeterminate thyroid nodules: A systematic review and meta-analysis. Thyroid, 2018; 28(2):210–219; doi: 10.1089/thy.2017.0419

41.

Fazeli

, Zehr

, Amraei

, et al. ThyroSeq v2 testing: Impact on cytologic diagnosis, management, and cost of care in patients with thyroid nodule. Thyroid, 2020; 30(10):1528–1534; doi: 10.1089/thy.2019.0191

42.

Stewart

, Leang

, Bhatt

, et al. Quantifying the differences in surgical management of patients with definitive and indeterminate thyroid nodule cytology. Eur J Surg Oncol, 2020; 46(2):252–257; doi: 10.1016/j.ejso.2019.10.004

43.

, Hu

, Ghaznavi

, et al. The prospective implementation of the 2015 ATA guidelines and modified ATA recurrence risk stratification system for treatment of differentiated thyroid cancer in a Canadian tertiary care referral setting. Thyroid, 2022; 32(12):1509–1518; doi: 10.1089/thy.2022.0055

44.

Bayona

, Benavent

, Muriel

, et al. Outcomes of repeat fineneedle aspiration biopsy for AUS/FLUS thyroid nodules. Eur J Endocrinol, 2021; 185(4):497–506; doi: 10.1530/EJE-21-0330