Repeat Fine Needle Aspiration Cytology Refines the Selection of Thyroid Nodules for Afirma Gene Expression Classifier Testing

Abstract

Background:

Molecular testing (MT) refines risk stratification for thyroid nodules that are indeterminate for cancer by fine needle aspiration (FNA) cytology. Criteria for selecting nodules for MT vary and remain largely untested, raising questions about the best strategy for maximizing the usefulness of MT while minimizing the harms of overtesting. We used a unique data set to examine the effects of repeat FNA cytology-based criteria for MT on management decisions and nodule outcomes.

Methods:

This was a study of adults (age 25–90 years; 281 women and 72 men) with cytologically indeterminate (Bethesda III/IV) thyroid nodules who underwent repeat FNA biopsy and Afirma Gene Expression Classifier (GEC) testing (N = 363 nodules from 353 patients) between June 2013 and October 2017 at a single institution, with follow-up data collected until December 2019. Subgroup analysis was performed based on classification of repeat FNA cytology. Outcomes of GEC testing, clinical/sonographic surveillance of unresected nodules, and histopathologic diagnoses of thyroidectomies were compared between three testing approaches: (i) Reflex (MT sent on the basis of the initial Bethesda III/IV FNA), (ii) SemiRestrictive (MT sent if repeat FNA is Bethesda I–IV), and (iii) Restrictive (MT sent only if repeat FNA is Bethesda III/IV) testing approaches.

Results:

Restricting MT to nodules that remain Bethesda III/IV on repeat FNA would have missed 4 low-risk cancers and 3 noninvasive follicular thyroid neoplasms with papillary-like nuclear features (NIFTP) (collectively 2% of the test population) but would have avoided diagnostic surgery for 42 benign nodules (12% of the test population). The Restrictive testing strategy was more specific (delta 0.126 confidence interval [CI 0.093 to 0.159] and 0.129 [CI 0.097 to 0.161], respectively) but less sensitive (delta −0.339 [CI −0.424 to −0.253] and −0.340 [CI −0.425 to −0.255], respectively) than the Reflex and SemiRestrictive approaches for detecting NIFTP or cancer.

Conclusions:

Repeat FNA cytology can guide the selection of cytologically indeterminate thyroid nodules that warrant MT. The Restrictive model of performing Afirma GEC only on nodules with two separate biopsies showing Bethesda III/IV cytology would reduce the rate of diagnostic surgery for histologically benign nodules while missing only rare low-risk tumors. Given the low but nontrivial risks of thyroidectomy, the higher specificity of the Restrictive testing approach disproportionately outweighs the potential harms.

Introduction

Molecular tests (MT) help refine the preoperative cancer risk assessment of cytologically indeterminate thyroid nodules (1). The Afirma Gene Expression Classifier (GEC; Veracyte, Inc., South San Francisco, CA), an early version of such a test, used gene expression profiling and machine-learning algorithms to stratify cytologically indeterminate nodules as having either low- (GEC-Benign [GEC-B]) or intermediate-risk (GEC-Suspicious [GEC-S]) profiles, warranting either clinical observation or diagnostic lobectomy, respectively (2). Current guidelines provide a general framework for MT of cytologically indeterminate thyroid nodules and do not offer specific recommendations about the role of repeat biopsy in selecting nodules for testing (3,4). This flexibility allows clinicians to adapt MT for their practice setting but has also led to inconsistent reports of test utilization and performance (5).

Veracyte endorses a Reflex testing model (Fig. 1A), where separate samples for cytology and molecular testing are collected from every nodule during the initial fine needle aspiration (FNA). The cytology is interpreted by a centralized laboratory, and aspirates resulted as Bethesda III or IV are reflexed to Afirma using the concurrent MT sample (6 –11). Reflex testing is simple but not suitable for all settings. This approach requires two additional FNA passes for potential MT from every nodule, even though the majority of those nodules ultimately do not require Afirma testing. Further, for nodules classified as Bethesda III due to atypia in a suboptimal sample, cytologic evaluation of a repeat FNA alone may suffice in some cases to resolve the indeterminate findings in the initial FNA (12 –14).

FIG. 1.

Comparison of Reflex, Restrictive, and SemiRestrictive strategies for selecting nodules for molecular testing. (A) In the Reflex testing approach, samples for both cytology and Afirma testing are collected from all nodules at the time of initial FNA, and samples classified as Bethesda III or IV are reflexed for Afirma testing. (B) In the Restrictive testing approach, the initial FNA of a nodule is for cytology only. Nodules classified as Bethesda III or IV undergo repeat FNA, at which time samples for cytology and Afirma testing are collected. Afirma testing is performed only if the repeat FNA is classified as Bethesda III or IV. (C) In the SemiRestrictive testing approach, Afirma testing is performed unless the repeat FNA is classified as Bethesda V or VI. AUS/FLUS, atypia of undetermined significance/follicular lesion of undetermined significance; FN/SFN, follicular neoplasm/suspicious for follicular neoplasm; FNA, fine needle aspiration; NonDx, nondiagnostic; SUS, suspicious for malignancy.

These considerations have prompted institutions to develop their own criteria for selecting nodules for Afirma testing. Some have adopted a two-staged Restrictive testing approach (Fig. 1B), where nodules classified as Bethesda III/IV on initial FNA undergo a repeat FNA, at which time material is collected for both Afirma and repeat cytology. Under this model, Afirma testing is confined to nodules that remain Bethesda III/IV on repeat FNA (15 –19). Afirma testing would not be performed if the repeat FNA is Bethesda V (Suspicious for Malignancy) or VI (Malignant), given the low negative predictive value (NPV) of Afirma in these categories (2). Likewise, nodules classified as Bethesda I (Nondiagnostic) or II (Benign) on repeat FNA would forego Afirma testing in favor of additional biopsy or surveillance, respectively.

Despite the appeal of this Restrictive strategy, its effectiveness compared with other Afirma testing strategies remains without an evidence base. Indeed, among surgical cohorts of thyroid nodules classified as Bethesda III, a benign repeat FNA did not necessarily confer a low cancer risk (20,21), posing a quandary for the clinician and patient: Should such nodules be treated as benign or indeterminate? If the latter, would the Restrictive model miss cancers that could have been detected using more permissive testing criteria?

In recognition of these concerns, we prospectively implemented a two-staged SemiRestrictive testing approach (Fig. 1C), in which Afirma testing was performed for cytologically indeterminate nodules classified as Bethesda I–IV on repeat FNA. Although our SemiRestrictive model had been devised to exclude nodules classified as Bethesda V/VI on repeat FNA from Afirma testing, we found that most (seven of nine) such cases had been sent for Afirma off-protocol during the study period. Therefore, with very few exceptions, Afirma GEC had been performed regardless of repeat FNA cytology. This population provided us with a unique opportunity to compare how different cytologic selection criteria (i.e., Reflex, SemiRestrictive, or Restrictive) for MT would have influenced test performance at our institution. The goal of this study was to correlate the Bethesda classification of the repeat FNA with Afirma GEC results, histologic outcomes of resected nodules, and surveillance outcomes of unresected nodules. We investigated the harms versus benefits of foregoing the Afirma test when the repeat FNA is interpreted as Bethesda I or II, as per the Restrictive strategy.

Materials and Methods

Study population and data collection

Institutional review board approval was obtained for this analysis of all thyroid FNA samples from Beth Israel Deaconess Medical Center that were submitted for Afirma GEC testing between June 2013 and October 2017. Nodules with an initial Bethesda III/IV cytology and repeat FNA (for Afirma testing and cytology) were identified. Afirma results, Bethesda classification (22) of initial and repeat FNAs based on cytology report review, sonographic nodule size, patient age, and sex were recorded for each nodule. Nodules under surveillance were classified as “concerning” if they developed 20% growth in at least two dimensions, >50% volume increase, or suspicious sonographic features, as per the American Thyroid Association guidelines (4); nodules lacking those findings were classified as “stable.” Follow-up data were collected until December 2019. Histopathologic diagnoses and surgery type were recorded for resected nodules. Histologic review was performed to confirm the subclassification of papillary carcinoma and to determine whether nodules diagnosed as follicular variants of papillary carcinoma warranted reclassification as noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP), as per the 2018 NIFTP criteria (23). Nodule size, laterality, and location were used to correlate the Afirma-tested nodule to the nodule followed by surveillance and/or resection.

Thyroid ultrasound, FNA, and Afirma GEC testing

Thyroid ultrasound was performed by Endocrine Certification in Neck Ultrasound (ECNU) certified endocrinologists and radiologists with thyroid expertise. The FNA was performed under continuous ultrasound guidance by using a 25-gauge needle. Initial FNA of a nodule typically consisted of four passes collected into CytoLyt for preparation of a ThinPrep (Hologic, Inc.) slide. For nodules classified as Bethesda III/IV on initial FNA, repeat FNA typically consisted of two passes collected into a vial of nucleic acid preservative for Afirma GEC testing and four passes collected into CytoLyt for ThinPrep cytology. Samples for Afirma GEC were submitted to Veracyte, Inc. as per our SemiRestrictive testing policy (Fig. 1C), with the addition of seven samples classified as Bethesda V/VI on repeat FNA that were submitted for Afirma testing off-protocol.

Data analysis

Fisher's exact test and Wilcoxon rank-sum test were used to compare categorical and continuous variables, respectively (GraphPad Prism v.8.4.1). To compare performance characteristics of the three testing strategies, we considered repeat FNA cytology and Afirma GEC results as a combined testing modality and defined “positive” or “negative” verdicts (regarding indication for surgical referral on the basis of the FNA) as follows:

Reflex:

(Any cytology result) + GEC-B = negative

(Any cytology result) + GEC-S = positive

SemiRestrictive:

(Bethesda I, II, III, or IV) + GEC-B = negative

(Bethesda I, II, III, or IV) + GEC-S = positive

Bethesda V or VI = positive

Restrictive

Bethesda I or II = negative

(Bethesda III or IV) + GEC-B = negative

(Bethesda III or IV) + GEC-S = positive

Bethesda V or VI = positive

We assumed 37.5% positive predictive value (PPV) and 94.5% NPV of Afirma testing based on the clinical validation study (2). We estimated sensitivity and specificity for the three different testing strategies by using conditional probability formulae. To account for the uncertainty due to sampling variation in this study as well as those used to estimate the PPV and NPV of Afirma testing, we computed the standard error and 95% confidence intervals (CIs) of the performance measure estimates by using the bootstrap method (24). The analyses described earlier were performed by using SAS 9.4 (SAS, Inc., Cary NC).

Results

Study population characteristics

Figure 2 shows exclusion criteria and case selection for this study. During the study, 383 FNAs from 382 thyroid nodules (373 patients) were submitted for Afirma GEC. Nine patients underwent Afirma GEC testing for two separate nodules. For one patient, the same nodule was tested twice over a three-year period. We excluded 12 nodules from analysis due to test cancellation (2 cases) or absence of prior indeterminate cytology (10 cases) (Fig. 2). Seven nodules had nondiagnostic Afirma results (GEC-ND); features of these nodules are summarized in Tables 1, 2, and 3 but have been excluded from subsequent analysis. The final cohort consisted of 363 nodules with (i) initial FNA classified as Bethesda III/IV, (ii) repeat FNA, and (iii) diagnostic Afirma GEC result; 215/363 (59%) nodules from 210 patients were classified as GEC-B, while 148/363 (41%) nodules from 143 patients were GEC-S. Table 1 summarizes demographic data and nodule size for the study population. Table 2 summarizes the Bethesda classifications of the initial and repeat FNAs. In accordance with the study design, the repeat FNAs were predominantly interpreted as Bethesda I–IV. Although our institutional policy was to forego Afirma testing when the repeat FNA cytology was Bethesda V/VI, seven of nine such aspirates were sent for Afirma GEC off-protocol. One was classified as GEC-B, while the remaining six were classified as GEC-S. The remaining two cases (both Bethesda V) were resected without Afirma testing.

FIG. 2.

Flow diagram demonstrating identification of the study population from the pool of thyroid nodules that were sampled for Afirma GEC testing. One nodule was tested twice with the Afirma GEC during the study period, as detailed in footnote a of Table 3. GEC, Gene Expression Classifier.

Table 1.

Patient and Nodule Characteristics for Samples Sent for Afirma Gene Expression Classifier Testing

	GEC-B	GEC-S	GEC-ND
Nodules, N	215	148	7
Patients, N	210	143	7
Sex, N (%)^a
Male	46 (22)	26 (18)	1 (14)
Female	164 (78)	117 (82)	6 (86)
Age, years^b
Range	25–87	23–90	35–69
Mean ± SD	56.0 ± 13.4	53.6 ± 15.0	53.9 ± 11.1
Nodule size, cm^c
Range	0.5–5.5	0.5–8.6	0.9–2.2
Mean ± SD	2.2 ± 1.0	2.3 ± 1.1	1.5 ± 0.5

Nodules with indeterminate cytology (Bethesda III or IV) on initial FNA underwent repeat FNA to collect samples for cytology and Afirma GEC testing. Patient sex, age, and nodule size data are presented based on the Afirma GEC-B, GEC-S, or GEC-ND.

Sex distribution between GEC-B and GEC-S groups showed no difference (p = 0.42, Fisher's exact test).

Patient age between GEC-B and GEC-S groups showed no difference (p = 0.11, Mann–Whitney unpaired t-test).

Nodule size between GEC-B and GEC-S groups showed no difference (p = 0.74, Mann–Whitney unpaired t-test).

FNA, fine needle aspiration; GEC, Gene Expression Classifier; GEC-B, Benign Gene Expression Classifier result; GEC-ND, nondiagnostic Gene Expression Classifier result; GEC-S, suspicious Gene Expression Classifier result; N, number; SD, standard deviation.

Table 2.

Fine Needle Aspiration Classification of Nodules That Were Sent for Afirma Gene Expression Classifier Testing

	GEC-B (N = 215)	GEC-S (N = 148)	GEC-ND (N = 7)
Initial FNA cytology, N (%)^a
Bethesda III	167 (78)	112 (76)	6 (86)
Bethesda IV	48 (22)	36 (24)	1 (14)
Repeat FNA cytology, N (%)
Bethesda I	29 (13)	16 (11)	4 (57)
Bethesda II	85 (40)	45 (30)	3 (43)
Bethesda III	63 (29)	52 (35)	0
Bethesda IV	37 (17)	29 (20)	0
Bethesda V^b	1 (0.5)	4 (3)	0
Bethesda VI^b	0	2 (1)	0

Nodules with indeterminate cytology (Bethesda III or IV) on initial FNA underwent repeat FNA to collect samples for cytology and Afirma GEC testing. If the repeat FNA cytology was Bethesda I, II, III, or IV, Afirma testing was performed on the concurrent sample collected for molecular testing as per our institutional SemiRestrictive policy for Afirma testing.

Distribution of Bethesda III and Bethesda IV interpretations on initial FNA did not differ between GEC-B and GEC-S groups (p = 0.70, Fisher's exact test).

The seven nodules that were classified as Bethesda V (N = 5) or Bethesda VI (N = 2) and submitted for Afirma testing represent deviations from institutional protocol; these samples are included in this table for the purposes of comparing Reflex testing protocols (i.e., Afirma test sent only on the basis of initial FNA cytology of Bethesda III or IV) with the SemiRestrictive and Restrictive testing approaches described herein.

Outcomes of nodules following Afirma GEC testing

Surveillance or surgical follow-up data were available from 189 of 215 (88%) GEC-B nodules and 134 of 148 (91%) GEC-S nodules (Table 3). As expected, far fewer GEC-B nodules than GEC-S nodules were resected (15% vs. 86%, respectively, of nodules with available follow-up, p < 0.0001). Surgical and surveillance outcomes of GEC-B nodules are summarized in Table 4 and Supplementary Table S1, respectively. Of the 28 resected GEC-B nodules, 25 (89%) were histologically benign (hyperplastic/adenomatoid nodules or adenomas), 1 was NIFTP, and 2 were carcinomas. Although diagnostic surgery was generally recommended for GEC-S nodules, several patients opted for surveillance, the records of which were available for 19 GEC-S nodules (Supplementary Table S2). Of the 115 GEC-S nodules that were resected (Table 5), 80 (70%) were benign (17 with Hurthle-cell features), 15 (13%) were NIFTP, and 20 (17%) were carcinomas.

Table 3.

Follow-Up of Nodules Sent for Afirma Gene Expression Classifier Testing

	GEC-B (N = 215), N (%)	GEC-S (N = 148), N (%)	GEC-ND (N = 7), N (%)
Medical/ultrasound surveillance^a	161 (75)	19 (13)	4 (57)
Partial thyroidectomy	19 (9)	67 (45)	1 (14)
Partial+completion thyroidectomy^a,b	3 (1)	12 (8)	0
Total thyroidectomy	6 (3)	35 (24)	0
Thyroidectomy (type N/A)	0	1 (1)	0
Follow-up data N/A	26 (12)	14 (9)	2 (29)

One of these tumors underwent Afirma testing twice, separated by 35 months. Ultrasound surveillance after the first Afirma test (GEC-B) revealed nodule growth, prompting repeat biopsy and Afirma testing 35 months after the initial FNA. The second Afirma test was GEC-S, prompting hemithyroidectomy. The tumor was diagnosed as a follicular variant of PTC (well-circumscribed/partially encapsulated tumor with bluntly invasive growth), prompting completion thyroidectomy.

One nodule was initially monitored by ultrasound after a GEC-B result. Hemithyroidectomy was performed 3 years after the initial FNA because of sonographic evidence of nodule growth (3.4 cm to 3.9 cm). The tumor was diagnosed as follicular carcinoma (encapsulated with extensive angioinvasion).

N/A, not available; PTC, papillary thyroid carcinoma.

Table 4.

Histologic Classification of Afirma Gene Expression Classifier-Benign Nodules That Were Resected (N = 28)

Type of surgery	Histopathologic classification (N)
Partial thyroidectomy	Hyperplastic/adenomatoid nodule (14)
	Follicular adenoma (4)
	NIFTP (1)
Partial with completion thyroidectomy	Hyperplastic/adenomatoid nodule (1)^a
	Follicular adenoma (1)^a
	Follicular carcinoma with angioinvasion (1)^b
Total thyroidectomy	Hyperplastic/adenomatoid nodule (4)^c
	Hurthle cell adenoma (1)
	Infiltrative FV-PTC (1)^d

Nodules with initial indeterminate cytology (Bethesda III/IV) and repeat cytology classified as Bethesda I, II, III, IV, or V.

Partial thyroidectomy for these two benign cases (hyperplastic/adenomatoid nodule and follicular adenoma) was followed by completion thyroidectomy because of multiple incidental papillary microcarcinomas (separate from the index nodule aspirated for Afirma testing).

After the GEC-B result, this 3.4 cm nodule was monitored by ultrasound for 3 years. Nodule growth to 3.9 cm prompted surgery.

For one of these cases, the repeat FNA cytology (concurrent to the Afirma sample) was interpreted as Bethesda V, prompting total thyroidectomy.

In spite of the benign GEC result, total thyroidectomy was prompted in this case by a combination of the cytology report (Bethesda III interpretation that included papillary carcinoma in the differential diagnosis) and sonographic features.

FV-PTC, follicular variant of papillary thyroid carcinoma; NIFTP, noninvasive follicular thyroid neoplasm with papillary-like nuclear features.

Table 5.

Histologic Classification of Afirma Gene Expression Classifier-Suspicious Nodules That Were Resected (N = 115)

Type of surgery	Histopathologic classification (N)
Partial thyroidectomy	Hyperplastic/adenomatoid nodule (38)
	Hyperplastic/adenomatoid nodule with Hurthle cell features (11)
	Follicular adenoma (4)
	Hurthle cell adenoma (3)
	PTC, classical (2)
	PTC, encapsulated follicular variant (1)
	NIFTP (8)
Partial with completion thyroidectomy	Hurthle cell carcinoma, minimally invasive (2)
	Hurthle cell carcinoma with capsular and angioinvasion (1)
	PTC, classical (3)
	PTC, encapsulated follicular variant (2)^a
	PTC, infiltrative follicular variant (1)
	NIFTP (3)^b
Total thyroidectomy	Hyperplastic/adenomatoid nodule (15)
	Hyperplastic/adenomatoid nodule with Hurthle cell features (5)
	Follicular adenoma (2)
	Hurthle cell adenoma (1)
	Follicular carcinoma, minimally invasive (1)
	Medullary thyroid carcinoma (1)
	PTC, classical (3)
	PTC, encapsulated follicular variant (1)
	PTC, infiltrative follicular variant (1)
	PTC, not otherwise specified (1)^c
	NIFTP (4)
Surgery type unknown	Benign nodule, not otherwise specified (1)

Selected for nodules with initial indeterminate cytology (Bethesda III/IV) and repeat cytology classified as Bethesda I, II, III, IV, V, or VI.

One of these tumors underwent Afirma testing twice, separated by 35 months (as described in footnote a, Table 3). Ultrasound surveillance after the first Afirma test (GEC-B) revealed nodule growth, prompting repeat biopsy and Afirma testing. The second Afirma test was GEC-S, prompting partial thyroidectomy. Histologic examination revealed a well-demarcated/partially encapsulated tumor with nuclear atypia of papillary carcinoma and bluntly invasive growth, best classified as encapsulated follicular variant of papillary carcinoma.

Three patients with NIFTP underwent completion thyroidectomy because of papillary microcarcinoma (separate from the index nodule tested by Afirma) with lymphovascular invasion and/or multifocality.

Review of pathology report and medical records indicates that the index nodule was a 1.1 cm follicular variant of papillary carcinoma with BRAF mutation (specific genotype not stated), present in a background of multifocal papillary microcarcinomas ranging from <0.1 to 0.7 cm.

Comparison of Afirma GEC performance between different testing strategies

We stratified nodule outcomes by Bethesda classification of the repeat FNA and GEC result (Fig. 3). For this analysis, Bethesda III/IV nodules were pooled together based on their similar recommendations for MT. Itemization of outcomes by each Bethesda category is provided in Supplementary Tables S3 and S4. Nodules classified as Bethesda III/IV on repeat FNA comprise the Restrictive testing group, those classified as Bethesda I–IV on repeat FNA comprise the SemiRestrictive group, and further inclusion of the seven nodules classified as Bethesda V/VI on repeat FNA approximate the Reflex testing group. Table 6 summarizes the benign-call rate, cancer prevalence, and cancer/NIFTP prevalence for these three testing approaches. The Restrictive approach would have missed four cancers (all Bethesda II on repeat FNA) and three NIFTPs (two Bethesda I and one Bethesda II) that would have otherwise been flagged as GEC-S under the SemiRestrictive and Reflex approaches. The clinicopathologic features of these seven cancer/NIFTP cases are summarized in Table 7; all four cancers were American Joint Committee Cancer (AJCC) stage I and considered low risk for structural disease recurrence. Conversely, Afirma testing of nodules classified as Bethesda I or II on repeat FNA, as per the Reflex and SemiRestrictive models, led to 42 cases where GEC-S results prompted diagnostic surgeries for histologically benign tumors. Altogether, the Restrictive strategy was more specific (delta 0.126 [CI 0.093 to 0.159] and 0.129 [CI 0.097 to 0.161], respectively) and it was less sensitive (delta −0.339 [CI −0.424 to −0.253] and −0.340 [CI −0.425 to −0.255], respectively) than the Reflex and SemiRestrictive approaches for detecting NIFTP or cancer.

FIG. 3.

Stratification of nodule outcomes based on repeat FNA cytology and Afirma GEC results. ¹Four nodules with initial cytology of Bethesda III/IV and repeat cytology of Bethesda I had nondiagnostic Afirma results. Two of these nodules were stable on surveillance. Two nodules were lost to follow-up. ²Three nodules with initial cytology of Bethesda III/IV and repeat cytology of Bethesda II had nondiagnostic Afirma results. One nodule was resected by hemithyroidectomy; final histologic diagnosis was benign. The other two nodules were monitored and clinically/sonographically stable. One nodule was tested by Afirma twice, as described in Table 3 footnote a: initially with a GEC-B result (classified above as “Surveillance: Concerning” in the GEC-B column) and subsequently with a GEC-S result 35 months later (classified above as “Surgery: Malignant” in the GEC-S column). GEC-B, Gene Expression Classifier-Benign; GES-S, Gene Expression Classifier-Suspicious.

Table 6.

Calculation of Benign-Call Rate, Cancer Prevalence, and Cancer+Noninvasive Follicular Thyroid Neoplasm with Papillary-Like Nuclear Features Prevalence for the Different Testing Approaches Analyzed in This Study

	Reflex	SemiRestrictive^a	Restrictive^b
Benign-call rate^c	215/363 (59%)	214/356 (60%)	100/181 (55%)
Cancer prevalence^d	22/323 (7%)	18/317 (6%)	14/158 (9%)
Cancer+NIFTP prevalence^d	38/323 (12%)	33/317 (10%)	25/158 (16%)

The differences between the SemiRestrictive and Reflex approaches were not statistically significant (p = 0.31 for benign-call rate, p = 0.24 for cancer prevalence, and p = 0.10 for cancer+NIFTP prevalence. Fisher's exact test).

The differences between the Restrictive and Reflex approaches were not statistically significant (p = 0.41 for benign-call rate, p = 0.46 for cancer prevalence, and p = 0.25 for cancer+NIFTP prevalence. Fisher's exact test).

Benign-call rate is calculated by dividing the number of Benign Afirma results (GEC-B) by the total number of Afirma tests that would have been performed by using each testing strategy. The cases that comprise each testing approach (Reflex, SemiRestrictive, or Restrictive) are designated in Figure 3. Nodules with nondiagnostic GEC results are not included from benign-call rate calculations.

Prevalence calculations are made only among nodules with available clinical/sonographic or surgical follow-up data.

Table 7.

Clinicopathologic Features of Seven Cancers/Noninvasive Follicular Thyroid Neoplasm with Papillary-Like Nuclear Features That Were Detected by the Reflex and SemiRestrictive Approaches But Would Not Have Undergone Afirma Testing by the Restrictive Testing Approach

Case	Age, years/sex	Index nodule size and histology	ETE, LVI, surgical margins	Incidental microPTC	Pathologic stage	AJCC 8th edition stage group	AMES risk/MACIS score
1	55/F	3.6 cm encapsulated follicular variant of papillary carcinoma	Negative	2 foci, 0.9 cm each	(m)pT2 N0	I	Low risk/5.48
2	62/F	2.8 cm Hurthle cell carcinoma, minimally invasive	Negative	None	pT2 N0	I	Low risk/5.8
3	47/F	1.1 cm follicular variant of papillary carcinoma, not further specified	Negative	Multiple foci (<0.1–0.7 cm)	(m)pT1b NX	I	Low risk/4.09
4	59/F	1.5 cm classical papillary carcinoma	Negative	None	pT1b N0	I	Low risk/5.17
5	78/F	1.5 cm NIFTP	N/A	2 foci, 0.5 cm and smaller; LVI noted	(m)pT1a NX	I	Low risk/6.39
6	67/F	0.8 cm NIFTP	N/A	None	N/A	N/A	N/A
7	48/F	2.5 cm NIFTP	N/A	None	N/A	N/A	N/A

In all four patients with carcinoma, preoperative ultrasound was performed and showed no evidence of abnormal lymph nodes. Central lymph node dissection was not performed in these four cases, but peri-thyroidal lymph nodes were identified (all negative for tumor) in the thyroidectomy specimens from cases 1, 2, and 4. Cases 1–3 received radioactive iodine ablation; post-treatment scans have all been negative for disease. Case 4 was treated by hemithyroidectomy without radioactive iodine; postoperative ultrasound revealed prominent submandibular lymph nodes, which were benign on biopsy.

AJCC, American Joint Committee Cancer; AMES (age, metastasis, extent, size) prognostic scoring system; ETE, extrathyroidal extension; F, female; LVI, lymphovascular invasion; (m), multifocal primary tumors; MACIS, (metastasis, age, completeness of resection, local invasion, size) prognostic scoring system; microPTC, papillary thyroid microcarcinoma; pT, pathologic T-stage.

Discussion

We had a unique opportunity to examine how different cytology-based strategies (Reflex, SemiRestrictive, or Restrictive) for selecting thyroid nodules for Afirma testing would have influenced management decisions and outcomes at our institution. By foregoing Afirma testing for nodules classified as Bethesda I or II on repeat FNA, the Restrictive approach would have missed 7 NIFTPs and cancers (2%, or 7/363 of the test population) that would have been detected by the Reflex and SemiRestrictive approaches. These seven tumors were all low-risk neoplasms. Based on the generally slow progression of such tumors, current recommendations (4) for nodules classified as Bethesda I (repeat biopsy, close observation, or resection depending on sonographic pattern) or Bethesda II (surveillance) would likely have sufficed to catch these tumors before the development of adverse outcomes. At the same time, the Restrictive strategy would have averted diagnostic surgeries, otherwise prompted by GEC-S results in the Reflex and SemiRestrictive approaches, in up to 42 histologically benign nodules (12%, or 42/363 of the test population). Given the low but nontrivial risks of thyroidectomy, we conclude that this benefit of the Restrictive strategy disproportionately outweighs both the potential harms of missing rare, low-risk tumors and the uncertainty that may be caused by disparate initial (Bethesda III/IV) and repeat (Bethesda I/II) FNA cytology interpretations.

The financial costs of a permissive Afirma testing strategy is also a consideration, particularly for nodules classified as Bethesda II on repeat FNA. The low prevalence of cancer and NIFTP in this group diminishes the Afirma GECs contribution to risk stratification, making the GEC an expensive confirmation of the low risk already conveyed by the repeat FNA cytology. Taken together, this analysis supports adoption of the Restrictive testing strategy at our institution. Although this study was limited to the Afirma GEC, the role of the Restrictive approach in regulating the pretest probability of NIFTP/cancer in the test population should be broadly applicable to other MT platforms, including those with a genotyping component that can help rule in disease.

The resection rate of cytologically indeterminate nodules can serve as one measure of an MT's effect on clinical decision making. In previous reports from our institution before the implementation of MT for thyroid FNAs, the resection rate of indeterminate nodules ranged from 37% (in 2010–2011) to 53% (2006–2008) (25,26). In the current study, GEC-tested nodules had an overall resection rate of 39%, raising questions about the impact of MT on reducing surgical rates among cytologically indeterminate nodules. The prevalence (12%) of NIFTP/cancer (Table 6) and PPV (26%) of the GEC (Fig. 3) in our overall study population provides insight into this matter. These values are low compared with those of the GEC clinical validation study (24–25% prevalence and 37–38% PPV) (2) as well as the pooled values for disease prevalence (20%) and PPV (37%) among post-validation studies for the GEC (27).

We considered several reasons for the low NIFTP/cancer prevalence in this study. First, given the high inter-observer variability among cytopathologists for the Bethesda III category (28), it is possible that our institutional criteria for Bethesda III during the study period were more lenient and, thus, included more benign nodules compared with those selected for Afirma testing at other institutions. The availability of MT itself may have been a contributing factor. The implementation of MT has been reported to coincide with an increase in indeterminate cytologic interpretations and a concomitant decrease in benign diagnoses (29,30). It is also likely that our relatively permissive policy for Afirma testing contributed to a greater share of histologically benign nodules sent for Afirma testing. A Restrictive testing strategy would have increased the prevalence of NIFTP/cancer in the GEC-tested population to 16% (Table 6) and PPV to 32% (23 NIFTP/cancer out of 72 GEC-S in the Restrictive group with available clinical or surgical follow-up), more closely approximating the corresponding values from the GEC validation study (2) and meta-analysis of post-validation reports (27).

Taken together, our findings highlight the importance of monitoring disease prevalence among cytologically indeterminate nodules and applying rigorous criteria for selecting those nodules that warrant MT. Although this study examined the role of repeat FNA cytology and Restrictive testing criteria for guiding the selection of nodules for MT, other strategies for mitigating the overuse of MT may also be effective. Second-opinion reviews of cytologically indeterminate aspirates have been shown to improve diagnostic accuracy (31 –34) and would likely have reclassified a subset of Bethesda III FNAs in our study as benign. Limiting the Afirma test to those nodules with indeterminate cytology on second-opinion review would be expected to further increase NIFTP/cancer prevalence among tested nodules. Integration of a nodule's sonographic features with clinical and cytologic findings may offer additional opportunities for the judicious selection of nodules for MT.

With a few exceptions (35 –37), most studies of Afirma GECs performance in the real-world setting have limited their outcome appraisal to resected nodules (9,11,16,18,38 –46). In contrast, our study includes longitudinal sonographic/clinical follow-up as reference outcomes for unresected GEC-tested nodules, as recommended by Duh et al., thus providing a more comprehensive and representative study population for evaluating test performance (47).

This study has several limitations. First, this analysis is limited to our institutional experience with the Afirma GEC, a test that has been superseded by Veracyte's Afirma Gene Sequencing Classifier (GSC) since 2017. We restricted our analysis to the years that the GEC was used at our institution to permit longitudinal follow-up of nodules. This design also allowed us to compare the cytologic criteria for ancillary molecular testing without introducing the additional variable of an updated version of the Afirma test. Compared with the GEC, the Afirma GSC showed superior specificity (68%, compared with 52% for GEC) and similar sensitivity (91%, compared with 90% for GEC) in its clinical validation study, with early reports from real-world practice settings supporting this finding (2,48 –50). Therefore, it is possible that the relative harms of a permissive testing approach that we observed for the GEC would be lower for the more technologically advanced GSC. Second, the Reflex testing model in this study approximates but is not equal to Reflex testing in practice, in which the sample for MT is collected at the time of initial FNA. We further recognize that testing strategies that favor nodule surveillance over diagnostic lobectomy are not necessarily more cost-effective; the economic cost of each of the testing strategies was not calculated in the current study. Finally, there are regional and global differences in the management of cytologically indeterminate nodules, based, in part, on contrasting perspectives regarding clinical surveillance, diagnostic surgery, and risk tolerance (27,51). Strategies for selecting nodules for MT described here are likely to be different for Asian practices where active surveillance, rather than diagnostic surgery, is the mainstay for indeterminate nodules (52).

In conclusion, this is the first study, to our knowledge, that provides data about the role of repeat FNA cytology in selecting nodules for molecular testing. The different cytologic criteria for selecting nodules for Afirma GEC testing present trade-offs between sensitivity and specificity for detecting NIFTP and cancer. Reflex testing does not involve a repeat biopsy. This strategy, as well as the relatively permissive SemiRestrictive testing strategy, detected rare cases of NIFTP and low-risk cancer (representing 2% of the test population) that would not have undergone Afirma testing under the Restrictive testing approach. However, the higher sensitivity of these permissive testing strategies is offset by their lower specificity: Afirma testing of nodules classified as Bethesda I or II on repeat FNA prompted the resection of 42 histologically benign nodules (representing 12% of the test population) by virtue of a suspicious GEC result. Considering the low mortality and relatively slow rate of tumor progression of low-stage differentiated thyroid cancers, the Restrictive testing approach offers a better balance between the harms of overtesting versus missing NIFTP or cancer at the time of repeat FNA.

Footnotes

Acknowledgments

The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard Catalyst, Harvard University and its affiliated academic health care centers, or the National Institutes of Health.

Authors' Contributions

All authors have significantly contributed.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This work was conducted with support from Harvard Catalyst | The Harvard Clinical and Translational Science Center (National Center for Advancing Translational Sciences, National Institutes of Health Award UL 1TR002541) and financial contributions from Harvard University and its affiliated academic health care centers.

Supplementary Material

Supplementary Table S1

Supplementary Table S2

Supplementary Table S3

Supplementary Table S4

References

Nishino

, Nikiforova

. 2018. Update on molecular testing for cytologically indeterminate thyroid nodules. Arch Pathol Lab Med, 142:446–457.

Alexander

, Kennedy

, Baloch

, et al. 2012. Preoperative diagnosis of benign thyroid nodules with indeterminate cytology. N Engl J Med, 367:705–715.

Patel

, Yip

, Lubitz

, et al. 2020. The American Association of Endocrine Surgeons guidelines for the definitive surgical management of thyroid disease in adults. Ann Surg, 271:e21–e93.

Haugen

, Alexander

, Bible

, et al. 2016. 2015 American Thyroid Association Management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on thyroid nodules and differentiated thyroid cancer. Thyroid, 26:1–133.

Valderrabano

, Hallanger-Johnson

, Thapa

, et al. 2019. Comparison of postmarketing findings vs the initial clinical validation findings of a thyroid nodule Gene Expression Classifier: a systematic review and meta-analysis. JAMA Otolaryngol Head Neck Surg, 18:e191449.

Ali

, Siperstein

, Sadow

, et al. 2019. Extending expressed RNA genomics from surgical decision making for cytologically indeterminate thyroid nodules to targeting therapies for metastatic thyroid cancer. Cancer Cytopathol, 127:362–369.

Harrell

, Bimston

. 2014. Surgical utility of Afirma: effects of high cancer prevalence and oncocytic cell types in patients with indeterminate thyroid cytology. Endocr Pract, 20:364–369.

Abeykoon

, Mueller

, Dong

, et al. 2016. The Effect of implementing Gene Expression Classifier on outcomes of thyroid nodules with indeterminate cytology. Horm Cancer, 7:272–278.

, Young

, Hung

, et al. 2016. Clinical factors influencing the performance of Gene Expression Classifier testing in indeterminate thyroid nodules. Thyroid, 26:916–922.

10.

Al-Qurayshi

, Deniwar

, Thethi

, et al. 2017. Association of malignancy prevalence with test properties and performance of the Gene Expression Classifier in indeterminate thyroid nodules. JAMA Otolaryngol Head Neck Surg, 143:403–408.

11.

Hang

, Westra

, Cooper

, et al. 2017. The impact of noninvasive follicular thyroid neoplasm with papillary-like nuclear features on the performance of the Afirma Gene Expression Classifier. Cancer Cytopathol, 125:683–691.

12.

Baloch

, LiVolsi

, Jain

, et al. 2003 Role of repeat fine-needle aspiration biopsy (FNAB) in the management of thyroid nodules. Diagn Cytopathol, 29:203–206.

13.

Chen

, Pace

, Chen

, et al. 2012. Yield of repeat fine-needle aspiration biopsy and rate of malignancy in patients with atypia or follicular lesion of undetermined significance: the impact of the Bethesda System for Reporting Thyroid Cytopathology. Surgery, 152:1037–1044.

14.

Yassa

, Cibas

, Benson

, et al. 2007. Long-term assessment of a multidisciplinary approach to thyroid nodule diagnostic evaluation. Cancer, 111:508–516.

15.

Zhu

, Faquin

, Samir

. 2015. Relationship between sonographic characteristics and Afirma Gene Expression Classifier results in thyroid nodules with indeterminate fine-needle aspiration cytopathology. AJR Am J Roentgenol, 205:861–865.

16.

Chaudhary

, Hou

, Shen

, et al. 2016. Impact of the Afirma Gene Expression Classifier Result on the surgical management of thyroid nodules with category III/IV cytology and its correlation with surgical outcome. Acta Cytol, 60:205–210.

17.

Villabona

, Mohan

, Arce

, et al. 2016. Utility of ultrasound versus Gene Expression Classifier in thyroid nodules with atypia of undetermined significance. Endocr Pract, 22:1199–1203.

18.

Baca

, Wong

, Strickland

, et al. 2017. Qualifiers of atypia in the cytologic diagnosis of thyroid nodules are associated with different Afirma Gene Expression Classifier results and clinical outcomes. Cancer Cytopathol, 125:313–322.

19.

Harrison

, Sosa

, Jiang

. 2017. Evaluation of the Afirma Gene Expression Classifier in repeat indeterminate thyroid nodules. Arch Pathol Lab Med, 141:985–989.

20.

VanderLaan

, Marqusee

, Krane

. 2011. Clinical outcome for atypia of undetermined significance in thyroid fine-needle aspirations: should repeated FNA be the preferred initial approach?. Am J Clin Pathol, 135:770–775.

21.

Renshaw

. 2010. Does a repeated benign aspirate change the risk of malignancy after an initial atypical thyroid fine-needle aspiration?. Am J Clin Pathol, 134:788–792.

22.

Ali

, Cibas ES 2018 The Bethesda System for Reporting Thyroid Cytopathology: Definitions

Criteria

, and Explanatory

Notes

. Second edition. Springer International Publishing, Cham, Switzerland.

23.

Nikiforov

, Baloch

, Hodak

, et al. 2018. Change in diagnostic criteria for noninvasive follicular thyroid neoplasm with papillarylike nuclear features. JAMA Oncol, 4:1125–1126.

24.

Efron

, Tibshirani

. 1993. An Introduction to the Bootstrap. Chapman & Hall/CRC, New York .

25.

Singh

, Wang

. 2011. Eliminating the “Atypia of Undetermined Significance/Follicular Lesion of Undetermined Significance” category from the Bethesda System for Reporting Thyroid Cytopathology. Am J Clin Pathol, 136:896–902.

26.

Brister

, Singh

, Wang

. 2015. Reporting thyroid FNA before and after implementation of the Bethesda system-one institution's experience. Diagn Cytopathol, 43:28–31.

27.

Ngo

HTT

, Nguyen

TPX

, Vu

, et al. 2020 Impact of molecular testing on the management of indeterminate thyroid nodules among Western and Asian countries: a systematic review and meta-analysis. Endocr Pathol [Epub ahead of print]; DOI: 10.1007/s12022-020-09643-0.

28.

Cibas

, Baloch

, Fellegara

, et al. 2013. A prospective assessment defining the limitations of thyroid nodule pathologic evaluation. Ann Intern Med, 159:325–332.

29.

Sacks

, Bose

, Zumsteg

, et al. 2016. Impact of Afirma Gene Expression Classifier on cytopathology diagnosis and rate of thyroidectomy. Cancer Cytopathol, 124:722–728.

30.

Fazeli

, Zehr

, Amraei

, et al. 2020. ThyroSeq v2 testing: impact on cytologic diagnosis, management, and cost of care in patients with thyroid nodule. Thyroid, 30:1528–1534.

31.

Bellevicine

, Migliatico

, Vigliar

, et al. 2017. Intra-institutional second opinion diagnosis can reduce unnecessary surgery for indeterminate thyroid FNA: a preliminary report on 34 cases. Cytopathology, 28:254–258.

32.

Davidov

, Trooskin

, Shanker

, et al. 2010. Routine second-opinion cytopathology review of thyroid fine needle aspiration biopsies reduces diagnostic thyroidectomy. Surgery, 148:1294–1299; discussion 1299–1301.

33.

Gerhard

, Boerner

. 2014. The value of second opinion in thyroid cytology: a review. Cancer Cytopathol, 122:611–619.

34.

Tan

, Kebebew

, Reiff

, et al. 2007. Does routine consultation of thyroid fine-needle aspiration cytology change surgical management?. J Am Coll Surg, 205:8–12.

35.

Deaver

, Haugen

, Pozdeyev

, et al. 2018. Outcomes of Bethesda categories III and IV thyroid nodules over 5 years and performance of the Afirma Gene Expression Classifier: a single-institution study. Clin Endocrinol (Oxf), 89:226–232.

36.

Angell

, Frates

, Medici

, et al. 2015. Afirma benign thyroid nodules show similar growth to cytologically benign nodules during follow-up. J Clin Endocrinol Metab, 100:E1477–E1483.

37.

Sipos

, Blevins

, Shea

, et al. 2016. Long-term nonoperative rate of thyroid nodules with benign results on the Afirma Gene Expression Classifier. Endocr Pract, 22:666–672.

38.

Brandler

, Liu

, Cho

, et al. 2018. Does noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP) have a unique molecular profile?. Am J Clin Pathol, 150:451–460.

39.

Jug

, Datto

, Jiang

. 2018. Molecular testing for indeterminate thyroid nodules: performance of the Afirma Gene Expression Classifier and ThyroSeq panel. Cancer Cytopathol, 126:471–480.

40.

Livhits

, Kuo

, Leung

, et al. 2018 Gene Expression Classifier vs targeted next-generation sequencing in the management of indeterminate thyroid nodules. J Clin Endocrinol Metab103:2261–2268.

41.

Song

, LiVolsi

, Montone

, et al. 2017. Pre-operative features of non-invasive follicular thyroid neoplasms with papillary-like nuclear features: an analysis of their cytological, Gene Expression Classifier and sonographic findings. Cytopathology, 28:488–494.

42.

Walts

, Sacks

, Wu

, et al. 2018. A retrospective analysis of the performance of the RosettaGX((R)) Reveal thyroid miRNA and the Afirma Gene Expression Classifiers in a cohort of cytologically indeterminate thyroid nodules. Diagn Cytopathol, 46:901–907.

43.

Roychoudhury

, Klein

, Souza

, et al. 2017. How “suspicious” is that nodule? Review of “suspicious” Afirma Gene Expression Classifier in high risk thyroid nodules. Diagn Cytopathol, 45:308–311.

44.

Celik

, Whetsell

, Nassar

. 2015. Afirma GEC and thyroid lesions: an institutional experience. Diagn Cytopathol, 43:966–970.

45.

Samulski

, LiVolsi

, Wong

, et al. 2016. Usage trends and performance characteristics of a “gene expression classifier” in the management of thyroid nodules: an institutional experience. Diagn Cytopathol, 44:867–873.

46.

Yang

, Sullivan

, Zhang

, et al. 2016. Has Afirma Gene Expression Classifier testing refined the indeterminate thyroid category in cytology?. Cancer Cytopathol, 124:100–109.

47.

Duh

, Busaidy

, Rahilly-Tierney

, et al. 2017. A systematic review of the methods of diagnostic accuracy studies of the Afirma Gene Expression Classifier. Thyroid, 27:1215–1222.

48.

Patel

, Angell

, Babiarz

, et al. 2018. Performance of a genomic sequencing classifier for the preoperative diagnosis of cytologically indeterminate thyroid nodules. JAMA Surg, 153:817–824.

49.

Angell

, Heller

, Cibas

, et al. 2019. Independent comparison of the Afirma Genomic Sequencing Classifier and Gene Expression Classifier for cytologically indeterminate thyroid nodules. Thyroid, 29:650–656.

50.

Vuong

, Nguyen

TPX

, Hassell

, et al. 2021. Diagnostic performances of the Afirma Gene Sequencing Classifier in comparison with the Gene Expression Classifier: a meta-analysis. Cancer Cytopathol, 129:182–189.

51.

Vuong

, Ngo

HTT

, Bychkov

, et al. 2020. Differences in surgical resection rate and risk of malignancy in thyroid cytopathology practice between Western and Asian countries: a systematic review and meta-analysis. Cancer Cytopathol, 128:238–249.

52.

Kakudo

. 2020. Asian and Western practice in thyroid pathology: similarities and differences. Gland Surg, 9:1614–1627.