Performance of the Afirma Gene Expression Classifier in Hürthle Cell Thyroid Nodules Differs from Other Indeterminate Thyroid Nodules

Introduction

The Afirma gene expression classifier (AGEC) is a proprietary 142-gene mRNA expression panel developed by Veracyte (South San Francisco, CA). The test provides a binary “suspicious” or “benign” result for triaging thyroid nodules that have indeterminate cytologic features. The assay has been validated on fine-needle aspiration (FNA) material for patients older than 21 years of age with a >1 cm thyroid nodule and a cytologic diagnosis of atypia of undetermined significance (AUS)/follicular lesion of undetermined significance (FLUS) or suspicious for follicular neoplasm (SFN) (1,2) according to The Bethesda System for Reporting Thyroid Cytopathology (TBSRTC) (3). Based on the initial validation study, the AGEC is not recommended for cytologic diagnoses of benign or suspicious for malignancy (1). A benign AGEC result carries a negative predictive value of 94–95% (sensitivity 90%, specificity 49–53%), allowing these patients to be followed clinically and spared surgery (1,4 –6). In contrast, patients with suspicious AGEC results have a malignancy rate of 37–38% (1), and are typically managed by thyroid lobectomy or, in some cases, total thyroidectomy (TT) (7,8).

Based on these test characteristics, AGEC results are being incorporated into surgical decision making for patients with cytologically indeterminate thyroid nodules. The forthcoming American Thyroid Association's updated clinical practice guidelines for thyroid nodule management are expected to include molecular testing as an acceptable adjunct to thyroid FNA. Similarly, the National Comprehensive Cancer Network's 2014 guidelines for thyroid carcinoma integrate molecular testing for optional triage of indeterminate FNAs, with observation being acceptable if the predicted risk of malignancy is equivalent to benign cytology (≤5%) (7). In routine practice, patients with indeterminate cytology and benign AGEC results have experienced a dramatic reduction in surgery rates, resulting in similar management for these patients as cytologically benign cases (8,9).

Following the integration of AGEC testing into clinical practice, further studies have indicated that the AGEC's performance for Hürthle cell (oncocytic) nodules (HCNs) may differ from the published data for other AUS/FLUS and SFN nodules (10). The initial validation data for the AGEC suggested a tendency for benign HCNs to yield suspicious AGEC results (1). These data are not surprising, given previous literature on the clustering of benign and malignant Hürthle cell tumors using various gene expression profiling methods (11 –15). Follow-up studies by several groups have also identified a trend of suspicious AGEC results in FNAs of nodules with Hürthle cell features (16 –18).

To further investigate and characterize the performance of the AGEC in thyroid nodules with Hürthle cell features, this study evaluated a multi-institutional cohort of 134 patients. This study is the largest series reported to date of HCNs that have undergone AGEC testing with detailed surgical and pathologic follow-up. The study aimed to describe the management and outcome of these HCNs before and after introduction of the AGEC and characterize the test's performance in this tumor subtype.

Materials and Methods

Results

Before the AGEC became available (during reference periods 2010 at BWH and 2012 at MGH), 66 patients were diagnosed with SHCN or AFHCN. Four (6%) patients were excluded due to being lost to follow-up (n=2), lack of data after the cytology result (n=1), or a diagnosis of suspicious for malignancy in another nodule, which dictated treatment decisions (n=1), leaving a reference population of 62 patients for further evaluation (Table 1). As illustrated in Figure 1A, 19% (n=12) of patients in this cohort were followed clinically, with documented reasons including repeat FNA diagnosed as benign or lacking oncocytic features (n=5), patient preference (n=3), and a separate malignancy in another organ (n=1), with no documented reason in three cases. Eighty-one percent (n=50) of patients underwent surgery, with 34% (n=17) of resections yielding a malignancy in the index nodule (9 Hürthle cell carcinoma, 4 papillary carcinoma, 3 follicular carcinoma, and 1 medullary carcinoma). Fifty percent (n=25) of patients underwent hemithyroidectomy (HT), and 50% (n=25) underwent TT or near-TT.

OPEN IN VIEWER

FIG. 1.

Study population characterized from three tertiary care facilities from 2010 to 2014, including (A) patients treated prior to the availability of the Afirma gene expression classifier (AGEC) and (B) patients managed using the AGEC.

After the introduction of the AGEC, 171 patients were diagnosed with SHCN or AFHCN cytology (2011–2014 at BWH and 2013–2014 at MGH). Twenty-one (12%) patients were excluded due to being lost to follow-up (n=6), awaiting surgery at the time of data acquisition (n=5), a lack of a documented treatment plan following cytology or AGEC results (n=4), awaiting repeat FNA for AGEC testing (n=3), a diagnosis of suspicious for malignancy in another nodule, which dictated treatment decisions (n=2), and aborted surgical procedure due to complications (anaphylaxis, n=1). The excluded patients were 81% female (n=17, p=0.76) with a mean age of 62 years, similar to the reference and AGEC populations (p=0.11).

Ninety-seven patients were clinically managed without using AGEC testing, despite molecular testing being available. For these patients, the decision for surgery or clinical follow-up was based on clinical and sonographic features alone. Statistical analysis did not reveal any difference in demographics or nodule size on ultrasound between this group and either the pre-AGEC cohort or the AGEC cohort (Table 1), although detailed reports of ultrasound characteristics that may have played a role in management decisions were not available. Eighty percent (n=78) of these patients had surgery, and 29% (n=23) were diagnosed with a malignant index nodule. Thus, the rates of surgical intervention and malignancy in the index nodule did not change significantly after AGEC implementation (p=0.9 and p=0.6, respectively), suggesting a relatively constant malignancy rate of 31% throughout the study period. Documented reasons for electing clinical follow-up instead of surgery (n=19, 20%) in this cohort included repeat FNA with benign cytology (n=10), a separate malignancy in another organ (n=6), prior neck surgery limiting future surgical approaches (n=1), a physician's decision to treat the cytology as if benign (n=1), and unknown (n=1).

From 2011 to 2014, 72 patients with an AFHCN or SHCN diagnosis underwent AGEC testing (Fig. 1B). Sixty-three percent (n=45) of AGEC results were classified as suspicious for malignancy, 36% (n=26) were classified as benign, and 1% (n=1) were non-diagnostic.

Surgery was performed in 96% (43/45) of patients with suspicious AGEC results. Two patients were clinically followed despite suspicious AGEC results due to metastatic lung cancer in one and refusal of surgical intervention in the other. Of the resected nodules, 14% (n=6) of patients were found to have a malignant index nodule (Table 2). Examining the 43 surgical cases by TBSRTC category, 18 patients had a cytologic diagnosis of AFHCN, 11% (n=2) of whom were diagnosed with a malignancy (1 papillary carcinoma, 1 Hürthle cell carcinoma), while 25 were cytologically SHCN, 16% (n=4) of whom were diagnosed with a malignancy (3 Hürthle cell carcinoma, 1 papillary carcinoma). Table 3 summarizes the AGEC's performance in light of the final pathology for each TBSRTC category in our study.

Benign AGEC results were associated with a decision for clinical follow-up in 88% of patients (23/26 benign AGEC cases), while 12% (n=3) of patients had surgery, all with benign final pathology results. The documented reasons for electing surgery in the setting of benign AGEC results included nodule growth during follow-up (n=1), the presence of two conflicting AGEC and FNA results for the same nodule (n=1), and a symptomatic nodule (n=1).

As the predictive value of a test in clinical practice depends heavily on the prevalence of the abnormality in the population being tested (19), a graph was determined showing the relationship between PPV and NPV over a range of possible pretest probabilities. Sensitivity and specificity of the AGEC in this specific cytologic setting were extracted using data gathered from four previous investigations and the current study (Table 4). A sensitivity of 95% and a specificity of 10% were assumed for the AGEC, and using Bayes' formula, a PPV was estimated across a range of pretest probabilities (Fig. 2). The prevalence of malignancy in the subset of AGEC patients, assuming the two AGEC suspicious patients who were not operated were diagnosed with malignant nodules on final pathology, was 18%, corresponding to a PPV of 19%. The minimal possible prevalence based on only known resected malignancies was 14%, corresponding to a PPV of 15%. In order to reflect the true prevalence of malignancy for patients undergoing AGEC testing, a 22% prevalence rate was calculated using the combined average of the present study and the other four published studies, which corresponds to a PPV of 23% (Fig. 2).

OPEN IN VIEWER

FIG. 2.

The relationship between positive predictive value (PPV) and negative predictive value (NPV) over a range of possible pretest probabilities. A sensitivity and specificity of 95% and 10%, respectively, were used. The dotted line represents 18% prevalence as reported in our current study. The dashed line represents a 22% prevalence of malignancy, averaged from data gathered from four previous studies and the current study.

Prior to the introduction of the AGEC, surgical cases were evenly divided between HT and TT or near-TT. Similarly, for AGEC suspicious cases, 53% (n=23) of patients underwent HT, and 47% (n=20) underwent TT. These two groups were further analyzed for index nodule size, the presence of contralateral nodules >1 cm, multinodular goiter, and the presence of clinical Hashimoto thyroiditis or hypothyroidism (Table 5). Patients in the TT group were more likely to have clinical Hashimoto thyroiditis or hypothyroidism (p=0.04), and tended to have a slightly larger nodule size, although this trend was not statistically significant (p=0.53). No correlation was found between the cytologic diagnosis and the type of surgery that was elected. In addition, papillary thyroid microcarcinoma (<1 cm) was incidentally discovered at the same rate in the TT and HT groups.

Discussion

To facilitate surgical decision making, it is important for clinicians to have accurate data regarding the AGEC's performance for HCNs. The baseline malignancy rate in resected nodules of 34% prior to the introduction of the AGEC is consistent with the prevalence of malignancy in the validation study (25%; p=0.62), facilitating comparisons of test performance and PPV (1).

Thirty-two percent of patients in the present study avoided surgery based on a benign AGEC result. While this rate is lower than the previously reported 41% for all indeterminate nodules in the original AGEC study (1), it still supports the alteration of HCN management based on the AGEC, deferring surgery in a significant number of patients. These results are consistent with prior reports, which have demonstrated an AGEC benign rate ranging from 28% to 53% (8,16 –18).

The current findings confirm a trend identified by earlier studies (Table 4). The original validation set contained 21 benign Hürthle cell adenomas, of which 81% yielded suspicious AGEC results (1). An independent follow-up study reported that 12/13 SHCN FNAs had suspicious AGEC results, but only 17% (2/12) of AGEC-suspicious nodules were malignant on resection (16). Similarly, Lastra et al. found that 68% of SHCN FNAs yielded suspicious AGEC results, but only 15% (2/13) harbored malignancy (17). Harrell et al. noted that 53% of suspicious AGEC cases had prominent Hürthle cell features, and 90% of AFHCN and SHCN FNAs yielded suspicious AGEC results (18). Together, these data demonstrate an increased rate of suspicious AGEC results in HCNs, despite a relatively low risk of malignancy.

While the low malignancy rate in the present AGEC cohort compared to the baseline malignancy rate raises concern that this may be a selected population with low clinical suspicion, the consistency between the current study and previous reports confirms the reproducibility of this finding. Since the majority of AGEC benign patients did not undergo surgery, this study cannot entirely exclude the possibility that some of these thyroid nodules harbored malignancies, accounting for the decrease in malignancy rate in AGEC suspicious cases. However, the present study reveals that in a sizable group of patients, clinicians are proceeding directly to surgery, even when AGEC testing is available, and that these patients have a similar rate of malignancy to individuals treated before the availability of the AGEC but a higher rate of malignancy on final pathology compared with the AGEC suspicious group (29% vs. 18%).

While the data show no statistically significant differences between the groups in the analyzed clinical and radiographic parameters, unfortunately due to incompletely recorded ultrasound data, it was not possible to study ultrasound characteristics such as central vascularity or irregular borders known to be associated with a higher malignancy rate in indeterminate nodules (20,21). It is plausible that clinicians opting for their patients to go directly to surgery may have had access to sonographic imaging/reports showing ultrasound findings that influenced this decision. Thus, while those patients who underwent AGEC testing may not be fully representative of the entire population of patients with HCNs, the substantial proportion of suspicious AGEC results despite a decreased malignancy rate suggests altered test performance in HCNs relative to other indeterminate nodules.

Compared with the pre-AGEC cohort in which 66% of resected HCNs were benign, 86% of resected nodules were benign in the AGEC suspicious cohort. If the two patients with suspicious AGEC results who did not undergo surgery are assumed to harbor malignancies, the maximum prevalence of malignancy would be 18%. These results demonstrate that applying clinical judgment before using the AGEC with a pretest probability of 18% will yield a PPV of 19%, suggesting decreased specificity of the AGEC when applied to HCNs. Moreover, if the AGEC had been implemented on all AFHCN/SHCN patients, a suspicious AGEC result would not increase the likelihood of malignancy for an index nodule (30% pretest probability, corresponding to 31% PPV).

The reason for the AGEC's overestimation of suspicious oncocytic nodules is unclear. Given the rarity of HCNs, the AGEC may not be sufficiently trained to differentiate benign from malignant samples. Although not confirmed, it has been suggested that accumulation of mitochondria in oncocytic thyroid cells is indirectly responsible for a distinct miRNA profile, potentially increasing the AGEC's false-positive rate (22). A review of the proprietary Veracyte algorithm, described briefly in the validation study's supplementary material, reveals an alternate possibility (1). The algorithm first analyzes data from six rare subtype classifiers comprised of 25 genes designed to identify melanoma, renal cell carcinoma, breast carcinoma, parathyroid tissue, medullary thyroid carcinoma, and Hürthle cell adenoma/carcinoma. Each of these independent classifiers is analyzed sequentially, and a suspicious result from any individual classifier halts progression through the remaining analysis, including the 142-gene benign versus suspicious classifier. Thus, it is possible that some AFHCN and SHCN FNAs are identified as suspicious by the initial rare subtype classifier's Hürthle cell cassette and are not analyzed by the main 142-gene classifier. However, the proprietary nature of the Veracyte algorithm precludes a more rigorous analysis of the workflow to confirm this hypothesis.

These observations regarding the unique nature of HCNs are supported by recent advances in the understanding of the molecular biology of Hürthle cell tumors. Unlike papillary thyroid carcinomas, which often have BRAF activating mutations or RET/PTC gene rearrangements, these molecular alterations are either not associated with Hürthle cell tumors (23,24) or are only seen in a minority of cases (25). Similarly, while RAS mutations and PAX8/PPARγ gene rearrangements most often characterize follicular carcinomas, a minority of Hürthle cell tumors harbor such genetic changes (24 –27). Instead, widely invasive Hürthle cell carcinomas demonstrate changes in the β-catenin and mTOR pathways with large regions of amplification in chromosomes 5, 7, 12, and 17 (24). Other studies have identified further targets that appear unique to Hürthle cell carcinoma, including PVALB (28). Although these results await confirmation and further study, they suggest that Hürthle cell neoplasms constitute a distinct molecular class of tumors, where genetic differentiation of benign from malignant appears not to be well described at this time.

As the molecular profile of thyroid neoplasms continues to be defined, the possibility of incorporating other gene panels in addition to the AGEC may improve specificity and facilitate clinical management of HCNs. Kloos et al. demonstrated that the addition of a single-gene test with low sensitivity failed to contribute diagnostic information due to inability to exclude carcinoma in AGEC-suspicious cases (29). Thus, multi-gene panels appear to be needed to increase specificity. Clinical validation data for a competing multi-gene panel using next-generation sequencing have recently been reported (30). It remains to be seen how this panel's performance will compare to the AGEC in multisite trials and in distinct tumor subtypes such as HCNs.

Similar to other post-validation studies, the present study was limited in determining the number of true-negative AGEC cases due to lack of surgical resection of AGEC-benign nodules. Additionally, the retrospective study design limits the ability to capture data fully regarding the decision-making process as it unfolds between the patient and physician. For example, Aragon Han et al. found that, in general, molecular testing did not contribute meaningfully to surgical decision making in a majority of thyroid FNAs, since the combination of clinical and sonographic features was sufficient to determine the appropriate treatment when reviewed retrospectively in light of the final pathology (31). Future prospective studies that examine factors driving a decision for surgery versus clinical follow-up are needed to determine when additional molecular information can optimally contribute to the decision-making process and improve patient outcomes. In this regard, when no other clinical parameters leading to an indication for TT are present, and as long as the false-positive rate of AGEC for HCNs remains high, it appears reasonable to pursue HT for AGEC-suspicious HCN patients.

As data regarding the most clinically relevant approach to using the AGEC accumulate, evaluation of the test's performance in biologically distinct subsets of thyroid lesions such as HCNs is essential to maximize the benefit of the AGEC. This study demonstrates an increased rate of suspicious AGEC results in HCNs compared to other indeterminate thyroid nodules, with a correspondingly low PPV. Thus, the findings confirm previously published reports indicating that a suspicious AGEC result does not warrant a more aggressive surgical approach. The decreased number of benign AGEC results limits the utility of the assay to spare many patients from surgery, though a significant number of patients may still avoid surgery when the AGEC is utilized as one tool in the evaluation of HCNs. This study reports the largest series of HCNs subjected to AGEC testing with surgical and pathologic follow-up, and it confirms that while a third of patients may be spared surgery, those with HCNs and suspicious AGEC findings need to be counseled accordingly, given the low malignancy rate on final pathology.