Analytical Performance of a Gene Expression Classifier for Medullary Thyroid Carcinoma

Abstract

Background:

The aim of this study was to demonstrate the analytical validity of an RNA classifier for medullary thyroid carcinoma (MTC).

Methods:

Fresh-frozen tissue specimens were obtained from commercial sources, and MTC diagnoses were confirmed by histopathology review. De-identified patient fine-needle aspiration biopsies (FNABs) and whole blood from normal donors were obtained. Total RNA was extracted, amplified, and hybridized to custom microarrays for gene expression analysis. Gene expression data were normalized and classified via a machine learning algorithm. Positive control materials were produced from MTC tissues and tested across multiple experiments and laboratories. Twenty-seven MTC tissue specimens were used to evaluate the sensitivity of the MTC classifier. Gene expression data from tissues and FNABs were used to model classifier response to mixtures of MTC samples with normal thyroid tissue, a benign thyroid nodule, a Hürthle cell adenoma, and whole blood. Select mixture conditions were confirmed in vitro. Assay tolerance to RNA input variation (5–25 ng) and genomic DNA contamination (30% by mass) was evaluated. The intra- and inter-run reproducibility and inter-laboratory accuracy of MTC classifier results were characterized.

Results:

The MTC classifier sensitivity of 96.3% [confidence interval 81.0–99.9%] was determined retrospectively using 27 MTC confirmed tissue specimens. One false-negative result in a necrotic tissue implicated sample necrosis in reduced classifier sensitivity. Dilution modeling of MTC samples with normal or benign tissues showed consistent detection of MTC down to 20% sample proportions, with in vitro confirmation of 20% analytical sensitivity. Classifier tolerance to RNA input variation (5–25 ng), genomic DNA contamination (30% by mass), and an interfering substance (blood) was demonstrated with 100% accurate classifier results under all tested conditions. The maximum observed run-to-run score difference for a single FNAB sample was ∼1 unit compared with the average score difference between 38 MTC and non-MTC FNABs of ∼32 units. MTC classifier results for 20 tissues processed from total RNA in two different laboratories showed 100% concordance.

Conclusions:

The MTC classifier, offered as part of the routine molecular testing of cytology-indeterminate thyroid nodules, demonstrates robust analytical sensitivity, specificity, accuracy, and reproducibility.

Introduction

Medullary thyroid carcinoma (MTC) is an uncommon but aggressive neoplasm arising from the parafollicular cells of the thyroid. MTC represents approximately 2.2% of all thyroid malignancies in the United States (1,2), with an incidence of 0.3–1.4% in thyroid nodules (3,4). Current guidelines for the treatment of MTC recommend a total thyroidectomy in conjunction with central neck dissection as the minimum procedure for most patients. The low prevalence and aggressive behavior of MTC with early metastases poses a challenge for the detection and effective treatment of this disease. Early diagnosis is critical due to the low rate of cure once disease is present outside of the thyroid gland (4).

Fine-needle aspiration is an accurate and well-established technique for the preoperative evaluation of thyroid nodules (4,5). However, MTC is not always suspected or accurately diagnosed by cytology, as the cytologic features of MTC can show significant overlap with those of other thyroid neoplasms, leading to low sensitivity and specificity (3,6 –13). In a recent international multicenter study, less than half of sporadic MTCs were specifically diagnosed preoperatively by cytology on fine-needle aspiration biopsies (FNAB) (14). While sensitive, the use of serum calcitonin screening (basal or stimulated) for the preoperative detection of MTC remains controversial because of a variety of factors, including differences in assay concordance and detection limits, a wide range of normal calcitonin levels in the population resulting in low test specificity (false-positives), and uncertain cost-effectiveness related to the low prevalence of MTC (3 –5,15 –19). Thus, a more accurate method for preoperative diagnosis of MTC is needed so that the optimal preoperative evaluation and surgery necessary to maximize patient outcome can be planned (4,20). As not all MTCs harbor RET or RAS point mutations, current gene mutations tests in thyroid FNAB samples are not highly sensitive for MTC (21,22).

The Veracyte Afirma Gene Expression Classifier (GEC) is a molecular test that reclassifies cytology indeterminate (Bethesda III–IV) thyroid nodules as either benign or suspicious (23,24). FNAB specimens from a patient are collected from a single nodule. Cytology specimens are prepared, and the corresponding GEC specimen is stored in nucleic acid preservative. When cytology identifies a nodule as indeterminate, the corresponding sample collected for genomic analysis is tested via a gene expression assay, the GEC, performed in the Veracyte CLIA-certified clinical laboratory. The GEC uses a series of “cassettes” to evaluate the expression of 25 genes associated with specific neoplasms potentially encountered in thyroid FNABs, upstream of a support-vector machine (SVM) algorithm that classifies the sample as benign or suspicious based on the normalized expression of 142 additional genes (23,24). The cassettes were developed through an unbiased, whole-transcriptome interrogation of histopathologically confirmed thyroid lesions, which identified distinct gene expression signatures specific to each type of lesion. One of these cassettes comprises an MTC classifier; the five classifier genes, CALCA, CEACAM5, SCG3, SCN9A, and SYT4, are involved in membrane structure, vesicular transport, and calcium-dependent exocytosis (23 –25). All cassettes were clinically validated against blinded expert histopathology review as part of the GEC on a large independent test set of consecutively collected and operated cytology samples (23). In particular, the MTC cassette correctly called 2/2 MTC positive samples (100% sensitivity), and generated no false calls of MTC on 318 non-MTC samples (Bethesda III–VI; 100% specificity) (23). Both of the MTC cases in this cohort had indeterminate FNAB cytology, and MTC was not suspected preoperatively. Subsequently, the MTC classifier identified 42/43 instances of surgically confirmed MTC in a series of >53,000 thyroid nodules screened by cytology, with a positive predictive value (PPV) of 98% (26).

This study presents the analytical validation of the MTC classifier following guidelines provided by the Evaluation of Genomic Applications in Practice and Prevention Working Group (EGAPP), the Centers for Disease Control ACCE Project: Evaluation of Genetic Testing, and Emergency Care Research Institute (ECRI) Evidence-based Practice Center (27 –29). The performance of the MTC classifier was evaluated across a diverse set of histopathologically confirmed MTC tissue specimens and thyroid FNABs. The analytical sensitivity of the MTC classifier to limiting assay RNA input is examined, and the dilution of an MTC positive sample with benign thyroid tissues is computationally simulated (24). The study demonstrates analytical specificity through assessment of assay performance in the context of potentially interfering genomic DNA, and simulates the effect of blood contamination on classifier performance. It also determines the reproducibility of MTC classifier results across technical replicates, across sample processing batches, and across laboratories.

Materials and Methods

Specimens

Fresh-frozen MTC tissue specimens were obtained from tissue bank sources (Cooperative Human Tissue Network [CHTN], Asterand, Inc., and ProteoGenex). Tissue specimens were shipped frozen and stored at −80°C upon receipt. The supplied clinical records were reviewed to confirm the surgical pathology diagnosis of MTC for each specimen. Prior to use in analytical studies, the MTC diagnosis for each tissue specimen was verified histopathologically using tissue sections prepared in-house. Briefly, frozen tissues were mounted for sectioning with Tissue-Tek O.C.T. medium (Sakura Finetek), and thick sections (1.5–3 mm) were generated on a CM1800 cryostat (Leica Biosystems). Thick sections were immersed in 10% buffered formalin, paraffin embedded, sectioned at 5 μm thickness, and separate sections were processed for hematoxylin and eosin (H&E) staining and calcitonin immunohistochemistry (IHC) using standard procedures (Histo-Tec Laboratory). Slides were read and specimens confirmed as MTC by a board-certified anatomic pathologist at Veracyte. Tumor content is expressed as percent tumor by area from review of representative H&E sections (Table 1).

Table 1.

Evaluation of MTC Classifier Performance on Histology Confirmed MTC Tissues

				Histopathologic confirmation of MTC
Patient	Age	Sex	Reference surgical pathology diagnosis	H&E	Ct IHC	Tumor content	MTC classifier call
1	48	F	MTC	+	+	∼100%	+
2	61	F	MTC	+	+	∼100%	−
3	50	M	MTC	+	+	∼100%	+
4	28	M	MTC	+	+	∼100%	+
5	60	M	MTC	+	+	∼100%	+
6	46	F	MTC	+	+	∼100%	+
7	89	F	MTC	+	+	∼100%	+
8	34	F	MTC	+	+	∼100%	+
9	50	M	MTC	+	+	∼100%	+
10	44	F	MTC	+	+	∼100%	+
11	60	M	MTC	+	+	∼100%	+
12	39	F	MTC	+	+	∼100%	+
13	23	F	MTC	+	+	∼100%	+
14	31	M	MTC	+	+	∼100%	+
15	21	F	MTC	+	+	∼100%	+
16	27	M	MTC	+	+	∼100%	+
17	59	M	MTC	+	+	∼100%	+
18	34	M	MTC	+	+	∼100%	+
19	19	F	MTC	+	+	∼100%	+
20	75	M	MTC	+	+	∼100%	+
21	45	F	MTC	+	+	∼100%	+
22	67	F	MTC	+	+	∼100%	+
23	43	M	MTC	+	+	∼90%	+
24	52	F	MTC	+	+	∼90%	+
25	52	M	MTC	+	+	∼80%	+
26	65	M	MTC	+	+	∼75%	+
27	57	M	MTC	+	+	∼40%	+

Tissue sections with <100% tumor content included non-malignant components comprised predominantly of normal thyroid follicles and intervening stroma.

MTC, medullary thyroid carcinoma; H&E, hematoxylin and eosin; Ct IHC, calcitonin immunohistochemistry.

Following review and waiver by an Institutional Review Board, FNABs referred to the Veracyte Clinical Laboratory Improvement Amendments (CLIA) certified laboratory for GEC testing, which activated the MTC cassette (23), were selected for physician follow-up by the Veracyte Medical Director. Cytology preparations (ThinPrep, Hologic) and smears (Diff-Quick, VWR International) were performed following standard procedures. Patients with documented MTC histopathology or clinical sequelae consistent with MTC (e.g., elevated serum calcitonin) were assigned a clinical truth label of MTC (26). Forty-one MTC FNAB samples (24 Bethesda III–IV, 12 Bethesda V, 5 Bethesda VI by cytology) were de-identified following standard procedures prior to use in analytical studies. The normal thyroid tissue, benign follicular nodule, Hürthle cell adenoma, and blood specimens used in this study were described previously (23,30).

RNA extraction, amplification, and microarray hybridization

For tissue specimens, a 20 μm frozen section taken immediately adjacent to the thick sections processed for histology were prepared, and RNA was isolated using the AllPrep Micro Kit (QIAGEN). Total RNA yield and quality was determined as described previously (30). Total RNAs from clinical FNAB samples and tissue specimens were amplified and hybridized to a custom microarray, and gene expression was analyzed using the GEC as described (30). Each expression analysis processing batch contained GEC benign and malignant thyroid tissue controls (30) as well as a no template (water) negative control. RNA from histopathologically confirmed MTC samples served as positive controls for the MTC classifier in each processing batch. Preanalytic test requirements, including sample collection and processing procedures and MTC classifier training and development, were described and validated previously (23,24). Confidence intervals (CI) were calculated as exact, two-sided Clopper–Pearson binomial confidence intervals.

In silico mixture modeling

Mixtures of gene expression data were modeled in silico from single sample data as previously described (24,30). Briefly, 200 simulations were performed for each mixture proportion, resulting in 200 independent classifier results (calls) per condition. The frequency of correct calls, that is, calls of MTC in a mixture containing an MTC-positive sample, are expressed as proportions. Each simulation incorporates known technical variability for in vitro mixing and gene expression intensity measurements. The statistical methods used in this study were reviewed and approved by a biomedical statistician.

Results

Tissue sensitivity

The prospective clinical collection of a diversity of confirmed MTC specimens remains a challenge due to the low prevalence of this neoplasm. Therefore, the performance of the MTC classifier was evaluated using a set of commercially available, histology verified MTC tissue specimens representing 27 different patients (Table 1). Out of 22 tissues with 100% estimated tumor content, 21 were correctly identified as MTC by the MTC classifier. Two heterogeneous MTC specimens containing normal thyroid follicles and intervening stroma and an estimated 90% tumor content were correctly classified, as were three additional specimens with 80%, 75%, and 40% estimated tumor content, respectively (Table 1). Additionally, three different MTC confirmed specimens from patient 22 were independently evaluated; all three were called MTC by the classifier, including one specimen with estimated blood contamination comprising 75% of the tissue section (data not shown). The false-negative in this tissue set (Patient 2, Table 1) is MTC by standard pathology criteria (H&E morphology in combination with calcitonin immunoreactivity). Analysis of adjacent, sequential sections from this tissue identified a gradient of decreasing MTC classifier gene expression and classifier score. The H&E morphology of these adjacent sections showed evidence of coagulative necrosis, with proportions of non-viable tumor in some sections of up to 60% (data not shown). Overall, the MTC classifier correctly identified MTC in 26/27 cases, providing a sensitivity estimate on tissues of 96.3% [confidence interval (CI) 81.0–99.9%].

Analytical sensitivity

Variability in routine quantitation of total RNA concentration for FNAB samples can result in modest variation around the nominal 15 ng total RNA assay input requirement. To characterize the effect of varying total RNA input on MTC classifier sensitivity, an assay input titration was performed on total RNA extracted from FNABs. Total RNA from three different nodules with confirmed MTC was assayed in triplicate at total RNA input quantities of 5, 10, 15, and 25 ng. GEC results were obtained at all input amounts, and all replicates for each nodule were classified correctly as MTC at all tested input levels (Fig. 1A). As previously reported (30), four similarly tested MTC negative FNABs did not trigger the MTC classifier at any of these input amounts, demonstrating classifier robustness to variation in assay input quantities (Fig. 1A).

FIG. 1.

Analytical sensitivity. (A) Three medullary thyroid carcinoma (MTC)-positive and four MTC-negative fine-needle aspiration biopsy (FNAB) samples were assayed in triplicate at RNA inputs ranging from 5 to 25 ng. (B) An MTC-positive FNAB and (C) an MTC-positive tissue were mixed in silico with normal tissue, a benign nodule, and a Hürthle adenoma. The proportion of MTC-positive calls (y-axis) at each mixture decrement (x-axis) is shown.

The FNAB procedure has the potential to introduce material from tissues adjacent to a nodule, which could in turn dilute the molecular signature of the nodule being sampled. To examine the effect of sample dilution on MTC classifier performance, dilution was modeled using an in silico simulation previously demonstrated to show agreement with in vitro microarray data (31) and for the GEC (24,30). Independently, an MTC positive FNAB sample and an MTC positive tissue sample were mixed in silico with ex vivo FNABs from a normal tissue sample and a benign thyroid nodule, and an FNAB derived from a Hürthle cell adenoma with gold-standard histopathologic confirmation, respectively (23). Mixture proportions from 100% MTC specimen to 0% MTC specimen were modeled in 10% decrements (Fig. 1B and C). In simulated mixtures, both the MTC tissue and FNAB sample were correctly classified a majority of the time in mixtures representing up to fivefold (80%) dilution of the MTC sample (Fig. 1B and C). To confirm the modeling result, RNA mixtures of an MTC FNAB with benign nodule RNA were tested in vitro in triplicate at mixture proportions of 40%, 30%, and 20% MTC FNAB RNA by mass. All nine technical replicates were classified as MTC by the classifier, confirming the in silico modeling result and demonstrating a robust limit of detection for MTC in RNA mixtures representing up to 80% dilution of the MTC sample.

Analytical specificity

The presence of an interfering substance with the potential to inhibit a molecular assay could cause a false-negative result. If the substance in question generates a positive signal through a nonspecific mechanism, a false-positive may result. Deviation in the RNA extraction process can result in sporadic genomic DNA contamination of the purified total RNA. During GEC testing, the routine in-process quality control of the RNA extraction process using the Bioanalyzer chip (Agilent) identifies and excludes samples with genomic DNA contamination >30% by total nucleic acid mass. To test the effect of 30% genomic DNA contamination by mass on the classification of MTC, 6.4 ng of genomic DNA extracted from an MTC positive tissue sample was added to 15 ng of total RNA isolated from the same sample, and the mixture assayed in triplicate. All three replicates of the MTC positive tissue were correctly called MTC by the classifier in the presence of 30% genomic DNA contamination (Fig. 2A). Three MTC negative samples assayed similarly with 30% genomic DNA added (30) were not identified as MTC by the classifier, demonstrating that incidental genomic DNA contamination from the sample extraction process will not generate false MTC classifier calls. FNAB can also contain varying quantities of blood (32). It is therefore important to determine whether blood itself, or the presence of blood in an FNAB, interferes with the detection of gene expression specific to MTC. To examine the effect of blood on the MTC classification, dilution was modeled using in silico simulation. An MTC positive FNAB and an MTC positive tissue sample were mixed in silico with nine different blood samples collected from nine different individuals as described previously (30). Mixture proportions spanning the range from 100% MTC sample to 0% MTC sample were modeled in 10% decrements (Fig. 2B and C). Both the MTC tissue and FNAB were correctly classified a majority of the time in simulated mixtures representing up to 80% blood (Fig. 2B and C). None of the nine pure blood samples were called MTC, suggesting that the gene expression signature of blood alone is unlikely to result in a nonspecific false call of MTC.

FIG. 2.

Analytical specificity. (A) Total RNAs from an MTC-positive and MTC-negative tissue were mixed with same-sample genomic DNA (30% by mass) and assayed in triplicate. (B) An MTC-positive FNAB and (C) an MTC-positive tissue were mixed in silico with normal blood from nine different donors. The proportion of MTC-positive calls (y-axis) at each mixture decrement (x-axis) is shown.

Assay control material

MTC-positive control materials were produced from tissue specimens with histology confirmed MTC. An RNA control material derived from patient 22 (Tables 1 and 2) was tested in triplicate in the Veracyte Research and Development (R&D) laboratory as part of the interfering substances study, and in singlicate in the R&D and CLIA clinical laboratories as part of the accuracy study described below. An RNA control material derived from patient 13 (Tables 1 and 2), in use as an analytic control for routine clinical testing, shows a classifier score SD of 0.507 across 235 technical replicates, versus an inter-run score SD of 0.321 estimated using three MTC FNABs (Fig. 3). All positive control replicates processed to date have classified correctly as MTC (data not shown).

FIG. 3.

MTC classifier score variation. Intra-run (R1, R2, R3) and inter-run (FNAB 1–3) score variation, illustrated as per-replicate score differences relative to the mean score for each grouping, is shown for 27 technical replicates of three MTC-positive FNABs. Score differences for an additional 38 MTC-positive and 38 MTC-negative FNABs are shown within and across diagnostic categories (MTC+ vs. MTC−).

Table 2.

Inter-Laboratory Concordance of MTC Classifier Calls on 20 Tissues

	Tissue identifier
	3	4	6	7	12	13	14	19	20	22
R&D Laboratory	+	+	+	+	+	+	+	+	+	+
Clinical Laboratory	+	+	+	+	+	+	+	+	+	+

	Tissue identifier
	28	29	30	31	32	33	34	35	36	37
R&D Laboratory	−	−	−	−	−	−	−	−	−	−
Clinical Laboratory	−	−	−	−	−	−	−	−	−	−

Tissue identifiers for MTC samples correspond to the patient identifiers used in Table 1.

+, MTC classifier call of positive; −, MTC classifier call of negative.

Reproducibility

The Afirma GEC was previously demonstrated to show high reproducibility for technical replicates within a processing run, across processing runs, and between the Veracyte R&D and CLIA laboratories (30). This study characterizes the intra- and inter-run reproducibility of MTC classifier calls. FNABs from three different patients with histopathologically confirmed MTC were each processed in triplicate from total RNA in a CLIA-certified laboratory as part of one sample processing batch. Processing of these samples in triplicate was repeated twice more in the clinical laboratory, utilizing different assay operators and assay reagent lots for the two subsequent processing batches. In each of three processing batches, all nine replicates for each FNAB, starting from total RNA (27 independent technical replicates in total) were correctly classified as MTC by the MTC classifier (Fig. 3 and data not shown). Thus, the MTC classifier demonstrates high reproducibility in the context of the multiple different operators and reagent lots that are typical of routine clinical laboratory operations.

The study also characterized variation in the MTC score underlying the classifier calls. Score differences were calculated for the 27 FNAB technical replicates within and across processing runs by grouping replicates by sample either within a run (R1, R2, R3) or across runs (FNAB 1–3), and calculating score differences relative to the mean score for each grouping (Fig. 3). Calculated thus, the estimated intra-run and inter-run SD of MTC scores for FNAB samples is 0.246 and 0.321, respectively, with a maximum observed score difference across runs for a single sample of ∼1 unit on an ∼40-unit scale, or ∼2.5% of the score range. To place this degree of technical variation in context, Figure 3 also illustrates score differences for 38 MTC positive (21 Bethesda III–IV, 12 Bethesda V, 5 Bethesda VI cytology) and 38 MTC negative (19 Bethesda V, 19 Bethesda VI cytology) FNAB samples encountered in routine clinical practice. Scores for MTC positive and MTC negative samples routinely differ by up to ∼35 units (Fig. 3). The MTC classifier thus demonstrates high reproducibility of calls and scores within and across processing runs, with a bimodal separation of scores observed between MTC positive and negative sample groups.

Accuracy

Inter-laboratory concordance of test results is an important measure of test accuracy (29). To characterize the accuracy of the MTC classifier, 10 tissue specimens positive for MTC and 10 GEC control tissues negative for MTC were processed through classification in the laboratory where the test was developed (Veracyte R&D laboratory) and independently in the CLIA laboratory on a different day using different operators, reagent lots, and equipment. The MTC classifier results for all 20 specimens were accurate and 100% concordant (Table 2), thus demonstrating MTC classifier accuracy.

Discussion

The development and validation of molecular tests for neuroendocrine lesions is challenging due to the low prevalence of these malignancies. To extend the body of evidence demonstrating the external validity of the MTC classifier (23,26), sensitivity was assessed on a set of 27 MTC tissue samples with associated reference standard diagnosis, and the correct call of MTC was observed in 26 samples, resulting in 96.3% sensitivity. The single false-negative tissue displayed clear MTC pathology confounded by the presence of coagulative necrosis. Elevated expression of CALCA, the calcitonin precursor gene, was noted in the array data for this sample, but expression of other MTC classifier genes was not meaningfully elevated above baseline levels (data not shown). While classifier error remains a formal possibility, the available evidence strongly suggests a negative association between necrosis and the ability to detect the expression of genes characteristic of MTC.

Thyroid FNABs can represent complex and potentially dilute mixtures of follicular and parafollicular cells with stroma, lymphocytes, colloid, cyst contents, and blood (24). Therefore, a molecular test intended for use on thyroid FNAB should demonstrate resistance to analyte dilution and potential interference (30). The MTC classifier demonstrates high analytical sensitivity, with consistent detection of MTC at one-third of nominal input requirements, and no evidence of false calls in benign or malignant FNAB samples across a range of input levels. Mixing simulations on array data consistently detected MTC in mixtures representing up to fivefold dilutions of MTC samples, demonstrating the robustness of the MTC classifier to sample dilution. Interference of test performance due to genomic DNA or blood contamination was not evident, with no false-positive calls observed in the presence of 30% genomic DNA or at any level of blood mixture. False-negative calls are anticipated only at mixtures representing ≥80% blood contamination by gene expression, as supported by mixture modeling and the correct classification of an MTC tissue from patient 22 containing ∼75% blood contamination.

During routine clinical use of the GEC, a FNAB from a nodule subsequently identified as a paraganglioma arising within an intrathyroidal parathyroid gland gave rise to a false-positive classifier call of MTC (26). This paraganglioma sample, and a second surgically confirmed paraganglioma that was not called MTC by the classifier, showed elevated expression of SCG3, SYT4, and SCN9A, but nominal levels of CALCA and CEACAM5, consistent with the absence of calcitonin and CEA expression reported for thyroid paraganglioma (33). By histology, paraganglioma shares a differential diagnosis with MTC, Hürthle cell carcinoma, and other neuroendocrine tumors; the Multiple Endocrine Neoplasia type 2 syndrome resulting from germline mutations in the RET proto-oncogene is characterized by related tumors in multiple endocrine organs, including medullary carcinomas of the thyroid and intra-adrenal paragangliomas, known as pheochromocytomas (4,33). No instances of MTC classifier cross-reactivity to a variety of benign or malignant thyroid nodules was observed, including Hürthle cell adenomas and carcinomas (100% specificity), during the prospective validation of the GEC (23). The very low incidence of paraganglioma (33) does not substantially change the overall negative predictive value and PPV of the MTC classifier. Recently, a 15-gene subset of a 92-gene molecular classifier for tumor origin showed promise in distinguishing pheochromocytoma and paraganglioma from MTC by retrospective principal component analysis on formalin-fixed, paraffin-embedded tissues (34). None of these 15 genes is represented in either the Afirma GEC or MTC classifiers.

In line with EGAPP, ACCE, and ECRI technical evaluation criteria for novel molecular tests, the analytical reproducibility of the MTC classifier was evaluated within processing runs, across processing runs within a laboratory, and across laboratories. High concordance is demonstrated of MTC classification and score within and across three processing runs in a CLIA-certified laboratory, using three surgically confirmed patient FNABs, and absolute concordance of MTC calls between the CLIA laboratory and the laboratory in which the test was developed using 10 histopathology confirmed MTC tissues. EGAPP level I analytic validity is therefore demonstrated for the MTC classifier as an index test using reference standard samples processed under a spectrum of laboratory conditions.

The MTC classifier is an accurate and robust tool for the preoperative identification of MTC in Bethesda III–VI thyroid FNAB, and may be used in conjunction with cytopathology to increase the detection of this rare but potentially aggressive malignancy. Clinical implementation of this test may provide for earlier diagnosis of MTC, ensuring not only more timely treatment but also the proper preoperative workup and the correct surgical management of the disease (35).

Footnotes

Acknowledgments

The authors thank James Diggans for analytical support, Sharlene Velichko for technical assistance with tissue sectioning, Ambika Sopory and Maryam Hosseini for sample processing, Lyssa Friedman for sample procurement, and Moraima Pagan for manuscript assistance. This study was funded by Veracyte, Inc. Select tissue samples were provided by the Cooperative Human Tissue Network, which is funded by the National Cancer Institute.

Author Disclosure Statement

D.G.P., Z.H., S.Y.K., R.J.M., M.G.M., R.T.K., P.S.W., and G.C.K. are Veracyte employees and equity owners. S.T.T. is an equity owner in Veracyte. G.C.K. has U.S. Patent 8,541,170 issued. R.T.K. is a consultant to Novo Nordisk.

References

Hundahl

, Fleming

, Fremgen

, Menck

. 1998. A National Cancer Data Base report on 53,856 cases of thyroid carcinoma treated in the U.S., 1985–1995 [see comments]. Cancer, 83:2638–2648.

Aschebrook-Kilfoy

, Ward

, Sabra

, Devesa

. 2011. Thyroid cancer incidence patterns in the United States by histologic type, 1992–2006. Thyroid, 21:125–134.

Costante

, Meringolo

, Durante

, Bianchi

, Nocera

, Tumino

, Crocetti

, Attard

, Maranghi

, Torlontano

, Filetti

. 2007. Predictive value of serum calcitonin levels for preoperative diagnosis of medullary thyroid carcinoma in a cohort of 5817 consecutive patients with thyroid nodules. J Clin Endocrinol Metab, 92:450–455.

Wells

Jr. , Asa

, Dralle

, Elisei

, Evans

, Gagel

, Lee

, Machens

, Moley

, Pacini

, Raue

, Frank-Raue

, Robinson

, Rosenthal

, Santoro

, Schlumberger

, Shah

, Waguespack

; American Thyroid Association Guidelines Task Force on Medullary Thyroid Carcinoma. 2015. Revised American Thyroid Association guidelines for the management of medullary thyroid carcinoma. Thyroid, 25:567–610.

Haugen

, Alexander

, Bible

, Doherty

, Mandel

, Nikiforov

, Pacini

, Randolph

, Sawka

, Schlumberger

, Schuff

, Sherman

, Sosa

, Steward

, Tuttle

, Wartofsky

. 2016. 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid, 26:1–133.

Bugalho

, Santos

, Sobrinho

. 2005. Preoperative diagnosis of medullary thyroid carcinoma: fine needle aspiration cytology as compared with serum calcitonin measurement. J Surg Oncol, 91:56–60.

Kudo

, Miyauchi

, Ito

, Takamura

, Amino

, Hirokawa

. 2007. Diagnosis of medullary thyroid carcinoma by calcitonin measurement in fine-needle aspiration biopsy specimens. Thyroid, 17:635–638.

Dustin

, Jo

, Hanley

, Stelow

. 2012. High sensitivity and positive predictive value of fine-needle aspiration for uncommon thyroid malignancies. Diagn Cytopathol, 40:416–421.

Lew

, Snyder

, Sanchez

, Solorzano

. 2011. Fine needle aspiration of the thyroid: correlation with final histopathology in a surgical series of 797 patients. J Am Coll Surg, 213:188–194; discussion 194–185.

10.

Choi

, Moon

, Lee

, Baek

, Kim

, Park

. 2011. Ultrasonographic findings of medullary thyroid cancer: differences according to tumor size and correlation with fine needle aspiration results. Acta Radiol, 52:312–316.

11.

Elisei

, Bottici

, Luchetti

, Di Coscio

, Romei

, Grasso

, Miccoli

, Iacconi

, Basolo

, Pinchera

, Pacini

. 2004. Impact of routine measurement of serum calcitonin on the diagnosis and outcome of medullary thyroid cancer: experience in 10,864 patients with nodular thyroid disorders. J Clin Endocrinol Metab, 89:163–168.

12.

Wirth

, Ross

, Randolph

, Cunnane

, Sadow

. 2013. Case records of the Massachusetts General Hospital. Case 5-2013. A 52-year-old woman with a mass in the thyroid. N Engl J Med, 368:664–673.

13.

Trimboli

, Treglia

, Guidobaldi

, Romanelli

, Nigri

, Valabrega

, Sadeghi

, Crescenzi

, Faquin

, Bongiovanni

, Giovanella

. 2015. Detection rate of FNA cytology in medullary thyroid carcinoma: a meta-analysis. Clin Endocrinol (Oxf), 82:280–285.

14.

Essig

Jr , Porter

, Schneider

, Debora

, Lindsey

, Busonero

, Fineberg

, Fruci

, Boelaert

, Smit

, Meijer

, Duntas

, Sharma

, Costante

, Filetti

, Sippel

, Biondi

, Topliss

, Pacini

, Maciel

, Walz

, Kloos

. 2013. Fine needle aspiration and medullary thyroid carcinoma: the risk of inadequate preoperative evaluation and initial surgery when relying upon FNAB cytology alone. Endocr Pract, 19:920–927.

15.

NCCN Clinical Practice Guidelines in Oncology. Thyroid carcinoma. Available at: www.nccn.org/professionals/physician_gls/pdf/thyroid.pdf (accessed April 12, 2016 ).

16.

Ross DS 2015 Diagnostic approach to and treatment of thyroid nodules. In: Cooper

, Mulder

(eds) UpToDate. UpToDate, Waltham, MA. Available at: https://www.uptodate.com/contents/diagnostic-approach-to-and-treatment-of-thyroid-nodules (accessed October 17, 2016 ).

17.

Rosario

, Ward

, Carvalho

, Graf

, Maciel

, Maia

, Vaisman

. 2013. Thyroid nodules and differentiated thyroid cancer: update on the Brazilian consensus. Arq Bras Endocrinol Metabol, 57:240–264.

18.

Perros

, Boelaert

, Colley

, Evans

, Gerrard Ba

, Gilbert

, Harrison

, Johnson

, Giles

, Moss

, Lewington

, Newbold

, Taylor

, Thakker

, Watkinson

, Williams

. 2014. Guidelines for the management of thyroid cancer. Clin Endocrinol (Oxf), 81:1–122.

19.

Costante

, Durante

, Francis

, Schlumberger

, Filetti

. 2009. Determination of calcitonin levels in C-cell disease: clinical interest and potential pitfalls. Nat Clin Pract Endocrinol Metab, 5:35–44.

20.

Shah

, Faquin

, Izquierdo

, Khurana

. 2009. FNA of misclassified primary malignant neoplasms of the thyroid: Impact on clinical management. Cytojournal, 6:1.

21.

Moura

, Cavaco

, Leite

. 2015. RAS proto-oncogene in medullary thyroid carcinoma. Endocr Relat Cancer, 22:R235–252.

22.

Pagan

, Kloos

, Lin

, Travers

, Matsuzaki

, Tom

, Kim

, Wong

, Stewart

, Huang

, Walsh

, Monroe

, Kennedy

. 2016. The diagnostic application of RNA sequencing in patients with thyroid cancer: an analysis of 851 variants and 133 fusions in 524 genes. BMC Bioinformatics, 17:6.

23.

Alexander

, Kennedy

, Baloch

, Cibas

, Chudova

, Diggans

, Friedman

, Kloos

, LiVolsi

, Mandel

, Raab

, Rosai

, Steward

, Walsh

, Wilde

, Zeiger

, Lanman

, Haugen

. 2012. Preoperative diagnosis of benign thyroid nodules with indeterminate cytology. N Engl J Med, 367:705–715.

24.

Chudova

, Wilde

, Wang

, Rabbee

, Egidio

, Reynolds

, Tom

, Pagan

, Rigl

, Friedman

, Wang

, Lanman

, Zeiger

, Kebebew

, Rosai

, Fellegara

, LiVolsi

, Kennedy

. 2010. Molecular classification of thyroid nodules using high-dimensionality genomic data. J Clin Endocrinol Metab, 95:5296–5304.

25.

Ashburner

, Ball

, Blake

, Botstein

, Butler

, Cherry

, Davis

, Dolinski

, Dwight

, Eppig

, Harris

, Hill

, Issel-Tarver

, Kasarskis

, Lewis

, Matese

, Richardson

, Ringwald

, Rubin

, Sherlock

. 2000. Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nat Genet, 25:25–29.

26.

Kloos

, Monroe

, Traweek

, Lanman

, Kennedy

. 2016. A genomic alternative to identify medullary thyroid cancer preoperatively in thyroid nodules with indeterminate cytology. Thyroid, 26:785–793.

27.

ACCE Model Process for Evaluating Genetic Tests. Available at: www.cdc.gov/genomics/gtesting/ACCE/index.htm (accessed May 4, 2016 ).

28.

Sun

, Bruening

, Uhl

, Ballard

, Tipton

, Schoelles

. 2010. Quality Regulation and Clinical Utility of Laboratory-Developed Molecular Tests. AHRQ Technology Assesment Program, USA Department of Health and Human Services.

29.

Teutsch

, Bradley

, Palomaki

, Haddow

, Piper

, Calonge

, Dotson

, Douglas

, Berg

, Group

. 2009. The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group. Genet Med, 11:3–14.

30.

Walsh

, Wilde

, Tom

, Reynolds

, Chen

, Chudova

, Pagan

, Pankratz

, Wong

, Veitch

, Friedman

, Monroe

, Steward

, Lupo

, Lanman

, Kennedy

. 2012. Analytical performance verification of a molecular diagnostic for cytology-indeterminate thyroid nodules. J Clin Endocrinol Metab, 97:E2297–2306.

31.

Robinson

, Speed

. 2007. A comparison of Affymetrix gene expression arrays. BMC Bioinformatics, 8:449.

32.

Cibas

, Ali

, Conference NCITFSotS. 2009. The Bethesda System For Reporting Thyroid Cytopathology. Am J Clin Pathol, 132:658–665.

33.

Baloch

, LiVolsi

. 2001. Neuroendocrine tumors of the thyroid gland. Am J Clin Pathol, 115:S56–67.

34.

Kerr

, Schnabel

, Sullivan

, Zhang

, Huang

, Erlander

, Brachtel

, Dry

. 2014. A 92-gene cancer classifier predicts the site of origin for neuroendocrine tumors. Mod Pathol, 27:44–54.

35.

Panigrahi

, Roman

, Sosa

. 2010. Medullary thyroid cancer: are practice patterns in the United States discordant from American Thyroid Association guidelines?. Ann Surg Oncol, 17:1490–1498.