Heterogeneity in Positive Predictive Value of RAS Mutations in Cytologically Indeterminate Thyroid Nodules

Abstract

Background:

RAS mutations are common in the available mutational analysis of cytologically indeterminate (Cyto-I) thyroid nodules. However, their reported positive predictive value (PPV) for cancer is widely variable. The reason for this variability is unknown, and it causes clinical management uncertainty. A systematic review was performed, evaluating the PPV for cancer in RAS mutation positive Cyto-I nodules, and variables that might affect residual heterogeneity across the different studies were considered.

Methods:

PubMed was searched through February 22, 2017, including studies that evaluated at least one type of RAS mutation in Cyto-I nodules, including any (or all) of the Bethesda III/IV/V categories or their equivalents and where the histological diagnosis was available. The PPV residual heterogeneity was investigated after accounting for Bethesda classification, blindedness of the histopathologist to the RAS mutational status, Bethesda category-specific cancer prevalence for each study, and which RAS genes and codons were tested. This was studied using five meta-regression models fit to different sets of Bethesda classification categories: Bethesda III, IV, or V (III/IV/V); Bethesda III or IV (III/IV); Bethesda III only; Bethesda IV only; and Bethesda V only.

Results:

Of 1831 studies, 23 were eligible for data inclusion. Wide ranges of PPV were found at 0–100%, 28–100%, and 0–100% in Bethesda III, IV, and V, respectively. Residual heterogeneity remained moderately high for PPV after accounting for the above moderators for Bethesda III/IV/V (21 studies; I ² = 59.5%) and Bethesda III/IV (19 studies; I ² = 66.0%), with significant Cochran's Q-test for residual heterogeneity (p < 0.001). Among individual Bethesda categories, residual heterogeneity was: Bethesda III (eight studies; I ² = 89.0%), IV (12 studies; I ² = 53.5%), and V (10 studies; I ² = 34.4%), with significant Cochran's Q-test for Bethesda III (p < 0.001) and IV (p = 0.04).

Conclusion:

The PPV of RAS mutations in Bethesda III and IV categories is quite heterogeneous across different studies, creating low confidence in the accuracy of a single estimate of PPV. Clinicians must appreciate this wide variability when managing a RAS-mutated Cyto-I nodule. Future studies should seek to resolve this unexplained variability.

Introduction

Cytologically indeterminate (Cyto-I) thyroid nodules remain a challenge for physicians and patients, and occur in about 15–25% of thyroid nodules when fine-needle aspiration (FNA) biopsy is performed (1). Based on the Bethesda classification, Cyto-I nodules include atypia of undetermined significance (AUS) or follicular lesion of undetermined significance (FLUS; Bethesda III), follicular neoplasm (FN) or suspicious for a follicular neoplasm (SFN; Bethesda IV), and suspicious for malignancy (SM; Bethesda V) (2). The risk of cancer in these categories was estimated to be 5–15%, 15–30%, and 60–75%, respectively (2), although subsequent studies have suggested that the prevalence of cancer among Bethesda III nodules is higher than initially estimated (3). Mutational analysis, available in many countries from commercial and non-commercial molecular diagnostic laboratories, of Cyto-I thyroid nodules has been developed to identify genomic variants associated with thyroid cancer that may inform the need for surgery and perhaps the extent of thyroid surgery. Such panels typically include selected H-, N-, and K-RAS mutations, and collectively these RAS mutations are the most common abnormality identified among Cyto-I nodules (4 –9). As such, the overall positive predictive value (PPV) of the panel is largely driven by the PPV of the RAS mutations. Yet, a broad range of PPVs for RAS mutations has been reported, as some histologically benign nodules are also mutated. The predictive value of RAS mutations for cancer has ranged from low to high, and has raised questions about the utility of RAS mutations to inform the need for and extent of surgery.

In this study, a systematic literature review was performed with the primary objective of estimating the percentage of unexplained heterogeneity in PPV across different studies after accounting for study-level variables that could contribute to this heterogeneity.

Methods

A systematic literature review was performed in PubMed through February 22, 2017. The results of four searches were combined using different search words: (i) RAS mutation or molecular diagnostics and thyroid nodules, (ii) RAS and thyroid cancer, (iii) indeterminate thyroid nodules, and (iv) mutational panel and thyroid nodules. For the study to be included, at least one type of RAS mutation should have been tested in Cyto-I thyroid nodules, including any (or all) of the indeterminate Bethesda III/IV/V categories or their equivalents (4,9), and there must have been surgical follow-up to determine the histological diagnosis. When multiple studies were published from one institution, the study dates of inclusion were evaluated, and the largest non-overlapping cohorts were included and those that were redundant were excluded to avoid nodules being potentially counted more than once. The search generated 1831 references. After reviewing titles and/or abstracts, 1796 references were excluded. The remaining 35 papers were reviewed, and 12 were excluded: eight (10 –17) due to inability to extract the necessary data fully, and four (18 –21) due to potential overlap of patient cohorts with included studies (Fig. 1). Given the period when the search was done, histologies now considered noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP) were likely considered the follicular variant of papillary thyroid cancer (fvPTC) in the earlier studies. To be consistent in the pathological readings between earlier and more recent studies, NIFTP were considered as malignant tumors in this analysis. Studies that specifically stated that the histopathology diagnosis was established when the pathologist was unaware of the RAS mutational status were considered blinded. Otherwise, the study was considered unblinded. The study did not control for whether the treating physician was blinded to the mutational status.

FIG. 1.

Summary of literature search strategy and results.

To measure the range of PPVs reported in the literature in each Bethesda category on a per medical center basis, different studies were assigned to a single center when possible as follows: (i) University of Pittsburgh Medical Center (UPMC) (6,7,22 –24), recognizing that one study (22) included several contributing centers; (ii) University of Leipzig (8,9); (iii) University of California at San Francisco (5); (iv) University of Sienna (25); (v) University of Ferrara (26); (vi) University of Iowa (27); (vii) Konkuk University School of Medicine (28); (viii) Xi'an Jiaotong University Health Science Center (29); (ix) McMaster University (30); (x) University of Sao Paulo (31); (xi) Institute of Pathology, Locarno (32); (xii) University of Pisa (33); (xiii) University of Minnesota (34); (xiv) Moffitt Cancer Center (35); (xv) Erasmus University Hospital (4); and (xvi) Brigham and Women's Hospital (36). Two studies had multiple centers participating to a degree that precluded assigning them to one center (37,38).

Subsequently, mixed-effect meta-regression models were used to estimate unexplained heterogeneity after accounting for study-level moderator variables. The “mixed-effect” designation indicates that (i) random-effect models were used rather than fixed-effect models and (ii) study-level moderator variables were included in the models (39). In the meta-analysis context, a random-effects model indicates that the parameterization allows the underlying “true” PPV to vary across studies rather than assuming a constant underlying PPV for all studies (40,41). The random-effects formulation is also used to allow generalization of results beyond the limited set of studies included in the meta-analysis. The “moderator variable” portion of the mixed-effect meta-regression indicates that study-level variables are included in the model, and parameter estimates are made for these variables, analogous to variables in a linear regression model. The moderator variables can account for some of the between-study variability. A moderator variable to account for medical center of each study was not included, as most centers reported only one study, and some studies could not be reasonably assigned to a single center.

In a random-effect meta-analysis, there are two sources of variability: s ², a measure of the “typical” within-subject variance, and \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\usepackage{upgreek}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document}$${ \hat \tau ^2}$$ \end{document} , a measure of between-study variability. The between-study variability is referred to as heterogeneity. The heterogeneity measure I ² is defined as: \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\usepackage{upgreek}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*}{ I^2 } = 100 \% \; \times \; { \frac { { { \hat \tau } ^2 } } { { { \hat \tau } ^2 } + \; { s^2 } } } \end{align*} \end{document}

and represents the percentage of total variability due to heterogeneity (42). In random-effect models that include study-level moderator variables (i.e., mixed-effect meta-regression models), the between-study variance \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\usepackage{upgreek}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document}$${\hat \tau ^2}$$ \end{document} is typically reduced relative to the model not including moderator variables, since some of the between-study variability is explained by the moderators. The formula for I ² is the same, but the value now represents the percentage of residual heterogeneity, which is the percentage of total variability due to heterogeneity after accounting for moderators. Although there are no standard thresholds for unacceptable levels of heterogeneity, Higgins et al. (43) provide general descriptors of “low,” “moderate,” and “high” for I ² values of 25%, 50%, and 75%, respectively.

For the current analysis, mixed-effect meta-regression models were fit to estimate the percentage of residual heterogeneity for the primary outcome PPV, as well as secondarily for sensitivity, using the R package metafor (44). For each outcome, meta-regression models were fit to different sets of indeterminate nodules based on Bethesda classifications: Bethesda III/IV/V, which included all available data (Bethesda categories III, IV, V, III/IV combined when data for III and IV were inseparable, and IV/V combined when data for IV and V were inseparable); Bethesda III/IV, which included Bethesda categories III, IV, and III/IV inseparable (in other words excluding category V); Bethesda III only; Bethesda IV only; and Bethesda V only.

Each PPV and sensitivity model contained the following moderators: histopathology blindedness to RAS status, Bethesda category-specific cancer prevalence for each study, and RAS mutations tested (specifically NRAS 12 and/or 13, HRAS 12 and/or 13, and KRAS 61, as these mutations had the highest variability of inclusion across studies). Regardless, a mutation in any of the three codons for any of the three RAS genes was considered positive. For the two model sets that included multiple Bethesda classifications (III/IV/V and III/IV models), the Bethesda classification was included as a moderator variable, and a multilevel effect was included to account for the correlation among results from the same study. The I ² measure for these multilevel models was calculated by a generalization of the I ² formula described above (45,46). All I ² values are reported with corresponding exact confidence intervals (47). As a supplemental analysis, for each model, the statistical test of heterogeneity is also presented based on Cochran's Q statistic, although this test is known to be underpowered unless many studies are included.

For all models, the arcsin square root transformation was applied to each outcome to fit model assumptions better. Sensitivity analyses using logit transformations were also performed, and the results were consistent. Pooled estimates of PPV and sensitivity are not reported because this was not the purpose of the meta-analysis and, due to the large amount of heterogeneity observed, we would not want a pooled estimate to be interpreted as the “true” PPV or sensitivity established by this meta-analysis.

Results

Table 1 reports the studies included in the meta-analysis, including blinded status and whether NRAS 12/13, HRAS 12/13, and KRAS 61 mutations were included in mutations tested. For each study, the total number of indeterminate nodules, the number of RAS-positive nodules, and the number of cancer-positive nodules are reported for each Bethesda classification level. For five studies (8,25,29,30,35), Bethesda III and IV were not reported separately, and for one study (4), Bethesda IV and V were not reported separately. The totals for these “inseparable” results are reported across the relevant columns. It could not be determined whether NRAS 12/13, HRAS 12/13, and KRAS 61 were included in one study (34); these mutations are considered to have values of “No” as moderators in the meta-regression models.

Table 1.

Included Studies with Descriptions of Blinded Status, Types of RAS Mutations Tested, and Number of Indeterminate Nodules, Including their Bethesda Classification Categories

					Number of indeterminate nodules by Bethesda classification ^a
Publication	Blinded	NRAS 12+ or 13	HRAS 12+ or 13	KRAS 61	III	IV	V
Beaudenon et al. (37)	Yes	No	Yes	No	22 (5) (7)	19 (5) (6)	12 (2) (8)
Bongiovanni et al. (32)	No	No	No	No	None	32 (5) (6)	None
Cantara et al. (25)	No	Yes	Yes	Yes	41 (3) (7)^b		54 (10) (46)
Danilovic et al. (31)	No	Yes	Yes	Yes	116 (0) (23)	20 (0) (7)	36 (1) (29)
De Napoli et al. (33)	Yes	Yes	No	No	None	258 (31) (90)	None
Eszlinger et al. (8)	No	No	No	No	164 (15) (25)^b		None
Eszlinger et al. (9)	No	No	No	No	None	158 (21) (45)	38 (1) (37)
Eszlinger et al. (38)	Yes	No	No	No	56 (10) (12)	169 (18) (30)	39 (2) (33)
Gill et al. (30)	No	No	No	No	60 (16) (23)^b		None
Hwang et al. (28)	No	Yes	Yes	Yes	81 (27) (78)	None	None
Le Mercier et al. (4)	Yes	Yes	Yes	Yes	None	34 (6) (6)^b
Liu et al. (29)	No	No	No	Yes	50 (5) (8)^b		13 (1) (6)
Medici et al. (36)	Yes	No	Yes	No	(n/a)^c (2) (1)	(n/a)^c (3) (2)	3 (3) (3)
Moses et al. (5)	Yes	Yes	No	Yes	None	110 (8) (29)	27 (2) (14)
Nikiforov et al. (7)	No	Yes	Yes	Yes	None	143 (24) (39)	None
Nikiforov et al. (22)	No	No	No	No	21 (2) (3)	23 (3) (12)	7 (0) (5)
Nikiforov et al. (23)	No	No	No	No	247 (19) (35)	214 (34) (58)	52 (8) (28)
Nikiforov et al. (6)	No	Yes	Yes	Yes	95 (14) (22)	None	None
Patel et al. (24)	No	Yes	Yes	Yes	(n/a)^c (51) (40)	(n/a)^c (30) (21)	(n/a)^c (6) (5)
Rossi et al. (26)	No	Yes	Yes	Yes	23 (4) (10)	17 (2) (8)	22 (1) (20)
Shrestha et al. (34)	No	NA	NA	NA	44 (15) (13)	12 (8) (5)	12 (1) (9)
Stence et al. (27)	No	Yes	No	Yes	11 (0) (4)	25 (9) (12)	5 (0) (5)
Valderrabano et al. (35)	No	Yes	Yes	Yes	102 (16) (20)^b		None

In addition, all studies tested for NRAS 61, HRAS 61, and KRAS 12/13, except for one (33) that did not test for HRAS 61 and KRAS 12/13 and two (5,29) that did not test for HRAS 61.

Total indeterminate nodules (RAS-positive nodules; cancerous nodules).

Numbers reflect a combination of two different Bethesda categories, as it was not possible to discern numbers in each indeterminate category.

n/a, not available: total indeterminate nodules not available, as only RAS-positive nodules were reported.

All studies tested for NRAS 61, HRAS 61, and KRAS 12/13, except for one (33) that did not test for HRAS 61 and KRAS 12/13 and two (5,29) that did not test for HRAS 61. Those RAS mutations (NRAS 61, HRAS 61, and KRAS 12/13) that were included in all or nearly all studies are not reported in Table 1 or included as moderators in the meta-regression models.

Residual heterogeneity in PPV models

Figures 2 –4 shows the unadjusted PPV values of different centers per Bethesda categories III, IV, and V, respectively. The PPV varied between 0% and 100% in Bethesda III, between 28% and 100% in Bethesda IV, and between 0% and 100% in Bethesda V. Residual heterogeneity remained moderately high for PPV, even after accounting for Bethesda classification, blinded status, Bethesda category-specific cancer prevalence for each study, and RAS mutations for Bethesda III/IV/V (I ² = 59.5%) and somewhat higher for Bethesda III/IV, with Bethesda V excluded (I ² = 66.0%). For both combined Bethesda classification models, Cochran's Q test for residual heterogeneity was statistically significant (p < 0.001; Table 2).

FIG. 2.

Positive predictive value (PPV) of Bethesda category III across different studies. PPVs for all studies are displayed as circles with confidence intervals. ^‡Five institutions include: Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts; Santa Monica Thyroid Center, Santa Monica, California; Endocrinology and Diabetes Division, West Los Angeles Veterans Affairs Medical Center, Los Angeles, California; Texas Diabetes and Endocrinology, Austin, Texas; and The Austin Diagnostic Clinic, Austin, Texas. ^§This included University of Cincinnati and University of Colorado.

FIG. 3.

PPVs of Bethesda Category IV across different studies. PPVs for all studies are displayed as circles with confidence intervals. †This included: Arcispedale Santa Maria Nuova-IRCCS, Reggio Emilia, Italy. ‡Five institutions include: Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts; Santa Monica Thyroid Center, Santa Monica, California; Endocrinology and Diabetes Division, West Los Angeles Veterans Affairs Medical Center, Los Angeles, California; Texas Diabetes and Endocrinology, Austin, Texas; and The Austin Diagnostic Clinic, Austin, Texas. ^§This included University of Cincinnati and University of Colorado.

FIG. 4.

PPV of Bethesda category V across different studies. PPVs for all studies are displayed as circles with confidence intervals. †This included Arcispedale Santa Maria Nuova-IRCCS, Reggio Emilia, Italy. ‡Five institutions include: Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts; Santa Monica Thyroid Center, Santa Monica, California; Endocrinology and Diabetes Division, West Los Angeles Veterans Affairs Medical Center, Los Angeles, California; Texas Diabetes and Endocrinology, Austin, Texas; and The Austin Diagnostic Clinic, Austin, Texas.

Table 2.

Meta-Regression Models for Positive Predictive Value: Percentage of Unexplained Heterogeneity

Bethesda categories included	Number of studies ^a	I² : Percentage of unexplained heterogeneity [CI]	Test for residual heterogeneity: QE (df), p-value
III, IV, V^b	21	59.5% [31.4–83.6%]	55.6 (26), p < 0.001
III and IV^c	19	66.0% [10.2–89.5%]	48.4 (17), p < 0.001
III only	8	89.0% [63.9–99.2%]	26.9 (3), p < 0.001
IV only	12	53.5% [0–90.3%]	13.5 (6), p = 0.04
V only	10	34.4% [0–87.4%]	4.9 (4), p = 0.30

All models included variables for blinded versus non-blinded status, RAS mutations tested (NRAS 12/13, HRAS 12/13, and KRAS 61), and Bethesda category-specific cancer prevalence for each study. Models including multiple Bethesda classification levels also included variables for Bethesda classification.

Two studies reporting only RAS-positive nodules were excluded because study-specific cancer prevalence was not calculable and could not be included in the model (24,36). Results for models not including the prevalence variable and including these two studies produced similar results.

Includes inseparable cases of combined III/IV (8,25,29,30,35) and IV/V (4).

Includes inseparable cases of combined III/IV (as above).

For models fit specifically to each single Bethesda level, after accounting for blinded status, Bethesda-category-specific cancer prevalence for each study, and RAS mutations included, the residual heterogeneity was highest for Bethesda III (I ² = 89.0%), decreased for Bethesda IV (I ² = 53.5%), and further decreased for Bethesda V (I ² = 34.4%). Cochran's Q-test was significant for the residual heterogeneity for levels III (p < 0.001) and IV (p = 0.04) but not for level V (p = 0.30; Table 2). Two studies (24,36) were not included in the PPV models because only RAS-positive nodules were reported, and therefore study-specific cancer prevalence could not be determined and constituted a missing value for the modeling.

Moderator effects in PPV models

None of the moderator variables were statistically significantly associated with PPV in any of the models at the 0.05 significance level. The variables closest to this threshold were HRAS 12/13 (p = 0.07) and cancer prevalence (p = 0.08) in Bethesda III/IV/V.

Residual heterogeneity in sensitivity models

Residual heterogeneity for sensitivity models followed a similar pattern as those for PPV models. After accounting for Bethesda classification, blinded status, Bethesda category-specific cancer prevalence for each study, and RAS mutation, residual heterogeneity remained moderately high in Bethesda III/IV/V (I ² = 65.1%) and Bethesda III/IV (I ² = 70.2%). For both combined Bethesda classification models, Cochran's Q-test for residual heterogeneity was statistically significant (p < 0.001; Table 3).

Table 3.

Meta-Regression Models for Sensitivity: Percentage of Unexplained Heterogeneity

Bethesda categories included	Number of studies ^a	I² : Percentage of unexplained heterogeneity [CI]	Test for residual heterogeneity: QE (df), p-value
III, IV, V^b	21	65.1% [41.7–85.5%]	92.3 (31), p < 0.001
III and IV^c	20	70.2% [48.8–96.0%]	69.8 (20), p < 0.001
III only	10	78.7% [41.4–95.8%]	27.1 (5), p < 0.001
IV only	13	49.8% [0–86.0%]	13.9 (7), p = 0.052
V only	12	15.5% [0–71.3%]	5.4 (6), p = 0.49

All models included variables for blinded versus non-blinded status, RAS mutations tested (NRAS 12/13, HRAS 12/13, and KRAS 61), and, Bethesda category-specific cancer prevalence for each study. Models including multiple Bethesda classification levels also included variables for Bethesda classification.

Two studies reporting only RAS-positive nodules were excluded because sensitivity could not be calculated without the count of false negatives (24,36).

Includes inseparable cases of combined III/IV (8,25,29,30,35) and IV/V (4).

Includes inseparable cases of combined III/IV (as above).

For models fit specifically to each single Bethesda category, the residual heterogeneity in descending order was Bethesda III (I ² = 78.7%), Bethesda IV (I ² = 49.8%), and Bethesda V (I ² = 15.5%) after accounting for blinded status, RAS mutations included, and Bethesda category-specific cancer prevalence for each study. Cochran's Q-test was significant for the residual heterogeneity for level III (p < 0.001) but not for levels IV (p = 0.052) or V (p = 0.49) at the 0.05 significance level (Table 3). Two studies (24,36) were not included in the sensitivity models because only RAS-positive nodules were reported and therefore sensitivity could not be calculated.

Moderator effects in sensitivity models

In the Bethesda III/IV/V model, sensitivity was significantly lower for Bethesda V nodules than for Bethesda IV nodules (p < 0.001 and p = 0.07 for Bethesda V versus Bethesda III) after adjusting for blinded status, RAS mutations included, and Bethesda category-specific cancer prevalence for each study, and accounting for within-center correlation. No other moderator variables were significantly associated with sensitivity in any of the models at the 0.05 significance level.

Discussion

The clinical significance of a RAS mutation in a Cyto-I thyroid nodule is a challenge, as this mutation is found in both benign and malignant thyroid nodules. The significant between-study heterogeneity seen in this study is similar to a recent meta-analysis of diagnostic value of RAS mutation in indeterminate thyroid nodules (48). The inclusion criteria of the previous study differed from the present study, as it excluded studies if mutational status of the three RAS genes was not done, and if they only represented one or two cytological classes of the indeterminate Bethesda categories. Additionally, the literature search was done in January 2016 in that study (48) versus February 2017 in the present study. While the two studies agree that between-study heterogeneity was significant, the between-study variability was then measured after accounting for multiple moderators, and moderate or high persistent (residual) heterogeneity was found in Bethesda III, Bethesda IV, Bethesda III + IV, and Bethesda III–V. As a result, this significant residual heterogeneity is uniquely interpreted as precluding a clinically useful point estimate of the pooled PPV of RAS mutation, and it is suggested that its cause should be the focus of future investigation.

In this analysis, the PPV of RAS mutations had substantial variability across different studies, especially among Bethesda III and IV categories. The cause of such wide heterogeneity is not fully known. Moderate and high heterogeneity remained after accounting for the cytology types included in each study according to the Bethesda classification, blinded status of the histopathologist to the RAS mutational status, Bethesda category-specific cancer prevalence for each study, and types of RAS mutations tested. In addition, similar residual heterogeneity was detected in RAS sensitivity to detect cancer.

This persistent (residual) heterogeneity after accounting for these moderators suggests the existence of other modifying factors, such as differences in the RAS mutation detection methodology or the threshold of pathology to differentiate benignity from malignancy at an individual level or at an institutional level, given the possibility of conformation toward the patterns of institutional peers. The differentiation of benign from malignant lesions on surgical histopathology is imperfect. If RAS-mutated clonal benign nodules were over-represented in a study, or if study histopathologists tended to classify RAS-mutated clonal nodules benign instead of cancer, then the PPV of a RAS mutation would be reduced. Lloyd et al. examined the observer variation among 10 experienced pathologists in the diagnosis of fvPTC. A concordant diagnosis of fvPTC was made by all 10 reviewers with a cumulative frequency of only 39% (49). Hirokawa et al. (50) investigated inter-observer variation among eight pathologists in assessment of encapsulated follicular lesions. Only in 10% of the cases was there complete agreement. El Sheikh et al. (51) assessed inter- and intra-observer variability in distinguishing fvPTC, follicular adenoma, and follicular thyroid cancer among six experts. Complete agreement occurred only in 13% of cases. In that study (51), even intra-observer agreement was quite variable and ranged from 17% to 100%. Similarly, Cibas et al. (52) reported high levels of disagreement between benign and malignant diagnoses by expert histopathologists in the evaluation of follicular lesions.

It is well known that RAS mutations can be present in cytologically and/or histologically benign nodules (53). The significance of a RAS mutation in thyroid nodules in predicting the transformation from adenoma to carcinoma, or from carcinoma in situ to invasive carcinoma, is unknown. Transgenic mouse models have demonstrated progressive changes from hyperplasia to adenoma and carcinoma in some animals (54). Supporting this progressive transformation model is the clinical finding that small follicular thyroid carcinomas are uncommon (55), which suggests that larger follicular thyroid carcinomas evolve from small follicular adenomas that have grown and transformed. Conversely, it appears that RAS alone is unable to transform a cell from benign to malignant, and that additional factors are needed for this event (54), and the field is at an early stage of gaining insights into what other factors may contribute to transformation (56). Even less is known regarding the potential rate at which an invasive transformation may occur. Overall, the relatively high prevalence of follicular adenomas and the low prevalence of follicular carcinomas suggest that the rate of transformation is low, and the presence of a RAS mutation is of questionable clinical significance for most patients. This is supported by the observation that cytologically benign thyroid nodules that did not undergo surgery were retrospectively found to harbor RAS mutations and were stable for a mean follow-up of 8.3 years (36).

The introduction of NIFTP (57) as a non-cancerous (but not necessarily benign) neoplasm may add more uncertainty to the PPV of RAS mutation in predicting clinically significant thyroid cancer, as 38% of NIFTP harbored RAS mutations. Wong et al. (58) reported that among GEC suspicious operated Bethesda III/IV nodules that a histological review following the introduction of NIFTP resulted in 64% of all cancer diagnoses being reclassified to NIFTP (including 88% of fvPTC diagnoses). The risk of cancer diminishes in Bethesda III/IV nodules when NIFTP is not considered cancer (59). Therefore, the predictive value of a RAS mutation toward predicting cancer could be lower when NIFTP is not counted as a true positive. Yet, this calculation may not be the most clinically relevant approach, since the diagnosis of NIFTP will not be known before surgery (as it requires a pathological diagnosis) to inform the decision to proceed with surgery. Additionally, some advocate that their surgical resection is necessary to avoid their potential progression to a carcinoma. From this perspective, the PPV of a RAS mutation is unchanged by the introduction of NIFTP when one considers true positives to include nodules warranting surgery (NIFTP and carcinomas). Still, the argument to perform a total thyroidectomy in the setting of a RAS mutation alone, and without other factors suggesting a high risk of malignancy, is even less compelling with the adoption of NIFTP nomenclature, as a hemithyroidectomy for them is considered sufficient.

The strengths of this study include its modest diversity of institutional experiences and number of individual studies and the statistical methods that account for different factors that may influence the results. Its weaknesses include the relatively small number of RAS-mutated nodules assigned with blinded histopathology. The requirement that the authors specifically state that the pathologist was blinded to the mutational status may have erroneously mis-categorized some blinded studies/nodules as unblinded. The blindedness of the treating physician was not included in the models, although it is possible that this information influenced which nodules underwent surgery in some studies.

In conclusion, this meta-analysis demonstrates that there is substantial residual heterogeneity in the PPV of a RAS mutation across different studies. Additional investigation is needed to explain this heterogeneity further and the degree to which inter-observer variability among pathologists and other factors contribute to this variability. Given this heterogeneity, the modest PPV for cancer in nodules with RAS-only mutations in many studies, and the trend toward a more conservative treatment of low-risk malignancies, hemithyroidectomy may be favored over total thyroidectomy in the management of these nodules in the absence of compelling findings in the contralateral lobe. Future studies are needed to understand the potential pathway(s) from RAS-mutated benign neoplasm to invasive carcinoma, and to clarify whether all RAS-mutated nodules are best served by surgery or whether other safe and more conservative approaches may be appropriate for some patients.

Footnotes

Author Disclosure Statement

M.A.L. reports receiving research funds from Veracyte for an unrelated project as well as speaker fees. M.A.L. also reports receiving research funds from Pathway Genomics, and previously was on the advisory board of Rosetta Genomics, from which he has also received research funds. K.N.P. reports receiving speaker fees from Veracyte. R.T.K. is a Veracyte employee and equity owner. No competing financial interests exist for the remaining authors.

References

Alexander

. 2008. Approach to the patient with a cytologically indeterminate thyroid nodule. J Clin Endocrinol Metab, 93:4175–4182.

Cibas

, Ali

. 2009. The Bethesda System for Reporting Thyroid Cytopathology. Thyroid, 19:1159–1165.

Krauss

, Mahon

, Fede

, Zhang

. 2016. Application of the Bethesda classification for thyroid fine-needle aspiration: institutional experience and meta-analysis. Arch Pathol Lab Med, 140:1121–1131.

Le Mercier

, D'Haene

, De Neve

, Blanchard

, Degand

, Rorive

, Salmon

. 2015. Next-generation sequencing improves the diagnosis of thyroid FNA specimens with indeterminate cytology. Histopathology, 66:215–224.

Moses

, Weng

, Sansano

, Peng

, Khanafshar

, Ljung

, Duh

, Clark

, Kebebew

. 2010. Molecular testing for somatic mutations improves the accuracy of thyroid fine-needle aspiration biopsy. World J Surg, 34:2589–2594.

Nikiforov

, Carty

, Chiosea

, Coyne

, Duvvuri

, Ferris

, Gooding

, LeBeau

, Ohori

, Seethala

, Tublin

, Yip

, Nikiforova

. 2015. Impact of the multi-gene ThyroSeq next-generation sequencing assay on cancer diagnosis in thyroid nodules with atypia of undetermined significance/follicular lesion of undetermined significance cytology. Thyroid, 25:1217–1223.

Nikiforov

, Carty

, Chiosea

, Coyne

, Duvvuri

, Ferris

, Gooding

, Hodak

, LeBeau

, Ohori

, Seethala

, Tublin

, Yip

, Nikiforova

. 2014. Highly accurate diagnosis of cancer in thyroid nodules with follicular neoplasm/suspicious for a follicular neoplasm cytology by ThyroSeq v2 next-generation sequencing assay. Cancer, 120:3627–3634.

Eszlinger

, Krogdahl

, Munz

, Rehfeld

, Precht Jensen

, Ferraz

, Bosenberg

, Drieschner

, Scholz

, Hegedus

, Paschke

. 2014. Impact of molecular screening for point mutations and rearrangements in routine air-dried fine-needle aspiration samples of thyroid nodules. Thyroid, 24:305–313.

Eszlinger

, Piana

, Moll

, Bosenberg

, Bisagni

, Ciarrocchi

, Ragazzi

, Paschke

. 2015. Molecular testing of thyroid fine-needle aspirations improves presurgical diagnosis and supports the histologic identification of minimally invasive follicular thyroid carcinomas. Thyroid, 25:401–409.

10.

Krane

, Cibas

, Alexander

, Paschke

, Eszlinger

. 2015. Molecular analysis of residual ThinPrep material from thyroid FNAs increases diagnostic sensitivity. Cancer Cytopathol, 123:356–361.

11.

Yoon

, Kwon

, Lee

, Kim

, Moon

, Kwak

. 2015. RAS mutations in AUS/FLUS cytology: does it have an additional role in BRAFV600E mutation-negative modules?. Medicine (Baltimore), 94:e1084.

12.

Park

, Sun

, Hong

, Kwak

, Kim

, Chung

, Choi

. 2013. Application of BRAF, NRAS, KRAS mutations as markers for the detection of papillary thyroid cancer from FNAB specimens by pyrosequencing analysis. Clin Chem Lab Med, 51:1673–1680.

13.

Park

, Kim

, Hwang

, Lee

, Kim

, Han

, Lim

, Kim

, Yoo

, Park

. 2013. BRAF and RAS mutations in follicular variants of papillary thyroid carcinoma. Endocr Pathol, 24:69–76.

14.

Lee

, Jung

, Kim

, Bae

, Jung

, Choi

, Kang

. 2013. Molecular genotyping of follicular variant of papillary thyroid carcinoma correlates with diagnostic category of fine-needle aspiration cytology: values of RAS mutation testing. Thyroid, 23:1416–1422.

15.

Mathur

, Weng

, Moses

, Steinberg

, Rahbari

, Kitano

, Khanafshar

, Ljung

, Duh

, Clark

, Kebebew

. 2010. A prospective study evaluating the accuracy of using combined clinical factors and candidate diagnostic markers to refine the accuracy of thyroid fine needle aspiration biopsy. Surgery, 148:1170–1176; discussion 1176–1177.

16.

Filicori

, Keutgen

, Buitrago

, AlDailami

, Crowley

, Fahey

3rd , Zarnegar

. 2011. Risk stratification of indeterminate thyroid fine-needle aspiration biopsy specimens based on mutation analysis. Surgery, 150:1085–1091.

17.

Aragon Han

, Olson

, Fazeli

, Prescott

, Pai

, Schneider

, Tufano

, Zeiger

. 2014. The impact of molecular testing on the surgical management of patients with thyroid nodules. Ann Surg Oncol, 21:1862–1869.

18.

Gupta

, Dasyam

, Carty

, Nikiforova

, Ohori

, Armstrong

, Yip

, LeBeau

, McCoy

, Coyne

, Stang

, Johnson

, Ferris

, Seethala

, Nikiforov

, Hodak

. 2013. RAS mutations in thyroid FNA specimens are highly predictive of predominantly low-risk follicular-pattern cancers. J Clin Endocrinol Metab, 98:E914–922.

19.

Ohori

, Nikiforova

, Schoedel

, LeBeau

, Hodak

, Seethala

, Carty

, Ogilvie

, Yip

, Nikiforov

. 2010. Contribution of molecular testing to thyroid fine-needle aspiration cytology of “follicular lesion of undetermined significance/atypia of undetermined significance.”. Cancer Cytopathol, 118:17–23.

20.

Radkay

, Chiosea

, Seethala

, Hodak

, LeBeau

, Yip

, McCoy

, Carty

, Schoedel

, Nikiforova

, Nikiforov

, Ohori

. 2014. Thyroid nodules with KRAS mutations are different from nodules with NRAS and HRAS mutations with regard to cytopathologic and histopathologic outcome characteristics. Cancer Cytopathol, 122:873–882.

21.

, Song

, Kim

, Park

, Yoo

, Yang

, Hwang

, Kim

. 2015. RAS mutations in indeterminate thyroid nodules are predictive of the follicular variant of papillary thyroid carcinoma. Clin Endocrinol (Oxf), 82:760–766.

22.

Nikiforov

, Steward

, Robinson-Smith

, Haugen

, Klopper

, Zhu

, Fagin

, Falciglia

, Weber

, Nikiforova

. 2009. Molecular testing for mutations in improving the fine-needle aspiration diagnosis of thyroid nodules. J Clin Endocrinol Metab, 94:2092–2098.

23.

Nikiforov

, Ohori

, Hodak

, Carty

, LeBeau

, Ferris

, Yip

, Seethala

, Tublin

, Stang

, Coyne

, Johnson

, Stewart

, Nikiforova

. 2011. Impact of mutational testing on the diagnosis and management of patients with cytologically indeterminate thyroid nodules: a prospective analysis of 1056 FNA samples. J Clin Endocrinol Metab, 96:3390–3397.

24.

Patel

, Carty

, McCoy

, Ohori

, LeBeau

, Seethala

, Nikiforova

, Nikiforov

, Yip

. 2017. Preoperative detection of RAS mutation may guide extent of thyroidectomy. Surgery, 161:168–175.

25.

Cantara

, Capezzone

, Marchisotta

, Capuano

, Busonero

, Toti

, Di Santo

, Caruso

, Carli

, Brilli

, Montanaro

, Pacini

. 2010. Impact of proto-oncogene mutation detection in cytological specimens from thyroid nodules improves the diagnostic accuracy of cytology. J Clin Endocrinol Metab, 95:1365–1369.

26.

Rossi

, Buratto

, Tagliati

, Rossi

, Lupo

, Trasforini

, Lanza

, Franceschetti

, Bruni

, Degli Uberti

, Zatelli

. 2015. Relevance of BRAF(V600E) mutation testing versus RAS point mutations and RET/PTC rearrangements evaluation in the diagnosis of thyroid cancer. Thyroid, 25:221–228.

27.

Stence

, Gailey

, Robinson

, Jensen

, Ma

. 2015. Simultaneously detection of 50 mutations at 20 sites in the BRAF and RAS genes by multiplexed single-nucleotide primer extension assay using fine-needle aspirates of thyroid nodules. Yale J Biol Med, 88:351–358.

28.

Hwang

, Kim

, Han

, Lim

, Kim

, Yoo

, Park

, Oh

, Kim

, Yang

. 2015. Preoperative RAS mutational analysis is of great value in predicting follicular variant of papillary thyroid carcinoma. Biomed Res Int, 2015:697068.

29.

Liu

, Gao

, Zhang

, Zhao

, Chen

, Ji

, Hou

, Shi

. 2014. Assessment of molecular testing in fine-needle aspiration biopsy samples: an experience in a Chinese population. Exp Mol Pathol, 97:292–297.

30.

Gill

, Nayan

, Kocovski

, Cutz

, Archibald

, Jackson

, Young

, Gupta

. 2015. Local molecular analysis of indeterminate thyroid nodules. J Otolaryngol Head Neck Surg, 44:52.

31.

Danilovic

, Lima

, Domingues

, Brandao

, Hoff

, Marui

. 2014. Pre-operative role of BRAF in the guidance of the surgical approach and prognosis of differentiated thyroid carcinoma. Eur J Endocrinol, 170:619–625.

32.

Bongiovanni

, Molinari

, Eszlinger

, Paschke

, Barizzi

, Merlo

, Giovanella

, Fasolini

, Cattaneo

, Ramelli

, Mazzucchelli

, Frattini

. 2015. Laser capture microdissection is a valuable tool in the preoperative molecular screening of follicular lesions of the thyroid: an institutional experience. Cytopathology, 26:288–296.

33.

De Napoli

, Bakkar

, Ambrosini

, Materazzi

, Proietti

, Macerola

, Basolo

, Miccoli

. 2016. Indeterminate single thyroid nodule: synergistic impact of mutational markers and sonographic features in triaging patients to appropriate surgery. Thyroid, 26:390–394.

34.

Shrestha

, Evasovich

, Amin

, Radulescu

, Sanghvi

, Nelson

, Shahi

, Burmeister

. 2016. Correlation between histological diagnosis and mutational panel testing of thyroid nodules: a two-year institutional experience. Thyroid, 26:1068–1076.

35.

Valderrabano

, Khazai

, Leon

, Thompson

, Ma

, Chung

, Hallanger-Johnson

, Otto

, Rogers

, Centeno

, McIver

. 2017. Evaluation of ThyroSeq v2 performance in thyroid nodules with indeterminate cytology. Endocr Relat Cancer, 24:127–136.

36.

Medici

, Kwong

, Angell

, Marqusee

, Kim

, Frates

, Benson

, Cibas

, Barletta

, Krane

, Ruan

, Cho

, Gawande

, Moore

Jr , Alexander

. 2015. The variable phenotype and low-risk nature of RAS-positive thyroid nodules. BMC Med, 13:184.

37.

Beaudenon-Huibregtse

, Alexander

, Guttler

, Hershman

, Babu

, Blevins

, Moore

, Andruss

, Labourier

. 2014. Centralized molecular testing for oncogenic gene mutations complements the local cytopathologic diagnosis of thyroid nodules. Thyroid, 24:1479–1487.

38.

Eszlinger

, Bohme

, Ullmann

, Gorke

, Siebolts

, Neumann

, Franzius

, Adam

, Molwitz

, Landvogt

, Amro

, Hach

, Feldmann

, Graf

, Wefer

, Niemann

, Bullmann

, Klaushenke

, Santen

, Tonshoff

, Ivancevic

, Kogler

, Bell

, Lorenz

, Kluge

, Hartenstein

, Ruschenburg

, Paschke

. 2017. Evaluation of a two-year routine application of molecular testing of thyroid fine-needle aspirations using a seven-gene panel in a primary referral setting in Germany. Thyroid, 27:402–411.

39.

Cooper

, Hedges

, Valentine

. 2009. The Handbook of Research Synthesis and Meta-Analysis. Second edition. Russell Sage Foundation, New York, New York.

40.

Berkey

, Hoaglin

, Mosteller

, Colditz

. 1995. A random-effects regression model for meta-analysis. Stat Med, 14:395–411.

41.

Borenstein

, Hedges

, Higgins

, Rothstein

. 2010. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods, 1:97–111.

42.

Higgins

, Thompson

. 2002. Quantifying heterogeneity in a meta-analysis. Stat Med, 21:1539–1558.

43.

Higgins

, Thompson

, Deeks

, Altman

. 2003. Measuring inconsistency in meta-analyses. BMJ, 327:557–560.

44.

Viechtbauer

. 2010. Conducting meta-analyses in R with the metafor package. J Stat Softw, 36:1–48.

45.

Nakagawa

, Santos

ESA

. 2012. Methodological issues and advances in biological meta-analysis. Evol Ecol, 26:1253–1274.

46.

Viechtbauer

. I² for Multilevel and Multivariate Models. Available at: www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate (accessed August 1, 2017 ).

47.

Viechtbauer

. 2007. Confidence intervals for the amount of heterogeneity in meta-analysis. Stat Med, 26:37–52.

48.

Clinkscales

, Ong

, Nguyen

, Harruff

, Gillespie

. 2017. Diagnostic value of RAS mutations in indeterminate thyroid nodules. Otolaryngol Head Neck Surg, 156:472–479.

49.

Lloyd

, Erickson

, Casey

, Lam

, Lohse

, Asa

, Chan

, DeLellis

, Harach

, Kakudo

, LiVolsi

, Rosai

, Sebo

, Sobrinho-Simoes

, Wenig

, Lae

. 2004. Observer variation in the diagnosis of follicular variant of papillary thyroid carcinoma. Am J Surg Pathol, 28:1336–1340.

50.

Hirokawa

, Carney

, Goellner

, DeLellis

, Heffess

, Katoh

, Tsujimoto

, Kakudo

. 2002. Observer variation of encapsulated follicular lesions of the thyroid gland. Am J Surg Pathol, 26:1508–1514.

51.

Elsheikh

, Asa

, Chan

, DeLellis

, Heffess

, LiVolsi

, Wenig

. 2008. Interobserver and intraobserver variation among experts in the diagnosis of thyroid follicular lesions with borderline nuclear features of papillary carcinoma. Am J Clin Pathol, 130:736–744.

52.

Cibas

, Baloch

, Fellegara

, LiVolsi

, Raab

, Rosai

, Diggans

, Friedman

, Kennedy

, Kloos

, Lanman

, Mandel

, Sindy

, Steward

, Zeiger

, Haugen

, Alexander

. 2013. A prospective assessment defining the limitations of thyroid nodule pathologic evaluation. Ann Intern Med, 159:325–332.

53.

Najafian

, Noureldine

, Azar

, Atallah

, Trinh

, Schneider

, Tufano

, Zeiger

. 2017. RAS mutations, and RET/PTC and PAX8/PPAR-gamma chromosomal rearrangements are also prevalent in benign thyroid lesions: implications thereof and a systematic review. Thyroid, 27:39–48.

54.

Kim

, Zhu

. 2009. Lessons from mouse models of thyroid cancer. Thyroid, 19:1317–1331.

55.

Kamran

, Marqusee

, Kim

, Frates

, Ritner

, Peters

, Benson

, Doubilet

, Cibas

, Barletta

, Cho

, Gawande

, Ruan

, Moore

Jr , Pou

, Larsen

, Alexander

. 2013. Thyroid nodule size and prediction of cancer. J Clin Endocrinol Metab, 98:564–570.

56.

Jung

, Kim

, Jung

, Park

, Kim

, Liu

, Bae

, Lee

, Kim

, Lee

, Chung

. 2016. Mutational burdens and evolutionary ages of thyroid follicular adenoma are comparable to those of follicular carcinoma. Oncotarget, 7:69638–69648.

57.

Nikiforov

, Seethala

, Tallini

, Baloch

, Basolo

, Thompson

, Barletta

, Wenig

, Al Ghuzlan

, Kakudo

, Giordano

, Alves

, Khanafshar

, Asa

, El-Naggar

, Gooding

, Hodak

, Lloyd

, Maytal

, Mete

, Nikiforova

, Nose

, Papotti

, Poller

, Sadow

, Tischler

, Tuttle

, Wall

, LiVolsi

, Randolph

, Ghossein

. 2016. Nomenclature revision for encapsulated follicular variant of papillary thyroid carcinoma: a paradigm shift to reduce overtreatment of indolent tumors. JAMA Oncol, 2:1023–1029.

58.

Wong

, Angell

, Strickland

, Alexander

, Cibas

, Krane

, Barletta

. 2016. Noninvasive follicular variant of papillary thyroid carcinoma and the Afirma gene-expression classifier. Thyroid, 26:911–915.

59.

Cibas

, Ali

. 2017. The 2017 Bethesda System for Reporting Thyroid Cytopathology. Thyroid, 27:1341–1346.