Real-World Performance of the American Thyroid Association Risk Estimates in Predicting 1-Year Differentiated Thyroid Cancer Outcomes: A Prospective Multicenter Study of 2000 Patients

Abstract

Background:

One of the most widely used risk stratification systems for estimating individual patients' risk of persistent or recurrent differentiated thyroid cancer (DTC) is the American Thyroid Association (ATA) guidelines. The 2015 ATA version, which has increased the number of patients considered at low or intermediate risk, has been validated in several retrospective, single-center studies. The aims of this study were to evaluate the real-world performance of the 2015 ATA risk stratification system in predicting the response to treatment 12 months after the initial treatment and to determine the extent to which this performance is affected by the treatment center in which it is used.

Methods:

A prospective cohort of DTC patients collected by the Italian Thyroid Cancer Observatory web-based database was analyzed. We reviewed all records present in the database and selected consecutive cases that satisfied inclusion criteria: (i) histological diagnosis of DTC, with the exclusion of noninvasive follicular thyroid neoplasm with papillary-like nuclear features; (ii) complete data of the initial treatment and pathological features; and (iii) results of 1-year follow-up visit (6–18 months after the initial treatment), including all data needed to classify the estimated response to treatment.

Results:

The final cohort was composed of 2071 patients from 40 centers. The ATA risk of persistent/recurrent disease was classified as low in 1109 patients (53.6%), intermediate in 796 (38.4%), and high in 166 (8.0%). Structural incomplete responses were documented in only 86 (4.2%) patients: 1.5% in the low-risk, 5.7% in the intermediate-risk, and 14.5% in the high-risk group. The baseline ATA risk class proved to be a significant predictor of structural persistent disease, both for intermediate-risk (odds ratio [OR] 4.67; 95% confidence interval [CI] 2.59–8.43) and high-risk groups (OR 16.48; CI 7.87–34.5). Individual center did not significantly influence the prediction of the 1-year disease status.

Conclusions:

The ATA risk stratification system is a reliable predictor of short-term outcomes in patients with DTC in real-world clinical settings characterized by center heterogeneity in terms of size, location, level of care, local management strategies, and resource availability.

Introduction

Most cases of differentiated thyroid cancer (DTC) currently diagnosed display indolent clinical behavior and are associated with very low mortality rates. Therefore, a more conservative approach in DTC management, with less extensive surgery, more selective use of radioactive iodine and less intensive follow-up protocols, is being recommended (1).

The aim is to avoid subjecting low-risk patients to unnecessary diagnostic procedures and overtreatment without reducing the chances of identifying those rare cancers that are likely to require more aggressive management (2). To facilitate this process, several scientific societies have developed tools for the prognostic stratification of patients with DTC. One of the most widely used is that developed by the American Thyroid Association (ATA), which aids clinicians in estimating individual patients' risk of persistent or recurrent disease (3). Its usefulness in these settings has been demonstrated by several retrospective, single-center studies (4).

A revised version of this system was included in the ATA guidelines published in 2016 (1). The criteria for classifying the risk of recurrence as high in this version are substantially more restrictive compared with the previous ones and, as a result, the number of patients who will be considered at low or intermediate risk has increased. The updated stratification system has also been validated in several retrospective cohort studies in different parts of the world. Most of these studies, however, were conducted at single health care facilities that served as a referral center for patients with thyroid cancer (5 –7), and it is unclear whether their findings reflect the performance of the system in real-world, heterogeneous clinical settings. New data, ideally from prospective, multicenter studies, are needed to better define the impact of the recent revisions on the system's ability to predict post-treatment DTC evolution, fundamental information for developing cost-effective follow-up strategies.

In this prospective cohort study, we analyzed data of over 2000 cases of DTCs managed in 40 diverse health care settings in Italy. Our aims were as follows: (i) to evaluate the performance of the 2015 ATA risk stratification system in predicting the response to treatment documented ∼12 months after the initial disease treatment; and (ii) to determine the extent to which this performance is affected by the treatment center itself.

Methods

The Italian Thyroid Cancer Observatory (ITCO) web-based database was opened in 2013 at the Thyroid Cancer Center of the Sapienza University of Rome (the network's coordinating center). Since then, it expanded to 49 other thyroid cancer centers in the country that joined the network (8). The database now includes prospectively collected data on nearly 7000 patients with histologically confirmed diagnoses of differentiated, medullary, poorly differentiated, or anaplastic thyroid cancer. Cases are inputted in the database at the time of the initial treatment in the reporting ITCO center, or when the patient begins follow-up in the reporting center within 12 months after undergoing initial treatment in a non-ITCO center.

Each case record contains information on patient demographics and biometrics, circumstances of the diagnosis, tumor pathology, surgical and radioactive iodine (RAI) treatments, as well as the results of periodic follow-up examinations. The ITCO provides no guidance or restrictions in terms of patient management to the participating centers, since the database is designed to provide a picture of real-world practices. Sensitive patient data are encrypted, and the database is anonymously managed for statistical analysis.

For the purposes of the present study, we reviewed all records present in the database and selected consecutive cases that satisfied the following criteria: (i) histological diagnosis of DTC—papillary thyroid cancer (PTC), follicular thyroid cancer (FTC), and poorly DTCs and their variants (with the exclusion of noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP); (ii) availability of all information on the initial treatment and pathological characteristics of the tumor required for the ATA recurrence risk assessment; and (iii) results of the 1-year follow-up visit (carried out 6–18 months after the initial treatment), including all data needed to classify the estimated response to treatment.

For each case, we recorded the following information.

Initial treatment

Treatment of the primary tumor was classified as thyroid lobectomy or total thyroidectomy. The latter category also included patients who had had a completion thyroidectomy following thyroid lobectomy. For all patients who had total thyroidectomy, we also recorded the use of radioactive iodine remnant ablation (RRA) as performed or not performed. Cervical lymph node dissection, when performed, was described as central compartment dissection, lateral compartment dissection, or both.

Risk of persistent or recurrent disease

The estimated level of risk was determined by the study team in accordance with the 2009 ATA guidelines (3) and relevant modifications in the 2015 update (1). Classification was based on the data available immediately after the initial treatment ( Fig. 1 ). When treatment consisted of lobectomy followed by completion thyroidectomy, we considered pathologic data on tissues collected during both surgical procedures. PTCs regarded as “aggressive” were those with (i) solid, insular, tall-cell, columnar-cell, hobnail-cell, sclerosing, or poorly differentiated histologic subtypes, or (ii) evidence of intratumoral blood vessel invasion (regardless of histologic subtype).

FIG. 1.

Risk of persistent or recurrent disease according to the ATA risk stratification system. ATA, American Thyroid Association; ETE, extrathyroidal extension; PTC, papillary thyroid cancer; RAI, radioactive iodine.

Responses to the initial treatment

These were classified as excellent, biochemical incomplete, structural incomplete, or indeterminate on the basis of data collected during the clinical evaluation carried out at the 1-year follow-up visit. These data included imaging findings (cervical ultrasound in all patients, and RAI scintigraphy in selected individuals), basal or stimulated serum thyroglobulin (Tg) levels, and anti-Tg antibody (TgAb) levels. Additional imaging studies were performed at the clinicians' discretion. The results were classified as specified in the ATA guidelines (1) for patients who had undergone thyroidectomy followed by RRA, and as advocated by the European Society for Medical Oncology (9) for those whose initial treatment consisted of surgery alone (thyroidectomy or lobectomy) (Supplementary Table S1).

Cervical lymph nodes with highly suspicious features on ultrasonography, as defined by the European Thyroid Association guidelines (10), were considered imaging evidence of persistent disease; those displaying low-suspicion features were classified as nonspecific imaging findings (11). Suspicious findings of other imaging studies were classified by the treating physicians. The presence of structural disease at the 1-year evaluation was considered persistent disease (1,12).

Statistical analyses

In our descriptive analysis, continuous variables were expressed as medians with interquartile ranges and nominal variables in terms of frequency counts and corresponding percentages. To model the response to treatment, we chose a cumulative link model, which is routinely used to describe the distribution of ordinal categorical response data. The ordinal response was categorized in descending order of desirability as excellent, indeterminate, biochemical incomplete, or structural incomplete. In this framework, the link used was the log of a probability ratio, calculated as the probability of not exceeding a given category versus the probability of exceeding it (in a logit model perspective), and predictors were inserted on a linear scale.

Given the hierarchical structure of the data, with patients nested within treatment centers, we used a mixed-effect model specification, with a center-specific intercept summarizing unobserved center-specific characteristics. To account for potential overlap or association between the center-specific features that were observed, and the unobserved features described by the random intercepts, we inserted in the model the average for each covariate on patients from a given center. We approximated the integral defining the likelihood via the Laplace approximation, using the R library ordinal (13). We also evaluated a binary response to treatment consisting of excellent versus structural incomplete responses only, in a mixed logit model. This was estimated by the R library lme4 (14).

All statistical analyses were performed with the R statistical software package, R Core Team (2017) (15).

Results

Out of 6867 case records in the database at data lock (2019), 1452 (21.1%) were excluded because the histological diagnosis was incomplete or failed to meet the inclusion criteria (i.e., tumors diagnosed as medullary thyroid cancer, anaplastic thyroid cancer, NIFTP, or tumors with unknown malignant potential), and 148 additional cases were excluded because one or more items essential to estimate the risk of recurrence were missing. Subsequent exclusions consisted of 3158 cases in which the patient had not yet undergone a 1-year follow-up assessment, and 38 others in which 1-year follow-up data needed to classify the treatment response were lacking. Thus, the final cohort consisted of 2071 patients who have been followed in 40 of the ITCO centers (Table 1 ).

Table 1.

Clinical and Demographic Features of the Study Cohort

No. of patients	2071
Age in years, median (IQR)	48 (38–59)
Sex, n (%)
Female	1546 (74.6)
Male	525 (25.4)
ITCO reporting center, n (%)	40
Academic center	29 (72.5)
Location
Northern Italy	19 (47.5)
Central Italy	13 (32.5)
Southern Italy	8 (20)
Time of thyroid cancer diagnosis, n (%)
Not specified	79 (3.8)
Postsurgical	813 (39.3)
Presurgical	1179 (56.9)
Initial treatment of primary tumor, n (%)^a
Total thyroidectomy + RRA	1191 (57.5)
Total thyroidectomy	807 (39)
Lobectomy	73 (3.5)
Neck dissection, n (%)
Not done	1283 (61.9)
Central neck dissection	546 (26.4)
Central and lateral neck dissection	196 (9.5)
Lateral neck dissection	46 (2.2)
Histology, n (%)
Papillary thyroid cancer	1958 (94.5)
Aggressive variants	99 (4.8)
Classical and other variants	1862 (89.9)
Follicular thyroid cancer	70 (3.4)
Hurthle cell carcinoma	37 (1.8)
Tumor size (mm), median (IQR)	11 (6 –18)
Tumor foci, n (%)
Not specified	17 (0.8)
Unifocal	1291 (62.3)
Multifocal, laterality not specified	19 (0.9)
Multifocal, unilateral	229 (11.1)
Multifocal, bilateral	515 (24.9)
Extrathyroidal extension, n (%)
None	1439 (69.5)
Microscopic	563 (27.2)
Strap muscle invasion
Yes	31 (5.5)
No	367 (65.2)
Not specified^b	165 (29.3)
Macroscopic (T4a)	63 (3.0)
Macroscopic (T4b)	6 (0.3)
Lymph node status, n (%)
cN0/Nx	619 (29.9)
N0	972 (46.9)
N1a	277 (13.4)
N1b	203 (9.8)

The categories “total thyroidectomy + RRA” and “total thyroidectomy,” each includes some cases of “completion thyroidectomy” performed after thyroid lobectomy.

Many pathology reports did not include details of muscle invasion before the publication of 8th edition of AJCC TNM staging system.

IQR, interquartile range; ITCO, Italian Thyroid Cancer Observatory; RRA, radioactive iodine remnant ablation.

The ATA risk of persistent/recurrent disease was classified as low in 1109 patients (53.6%), intermediate in 796 (38.4%), and high in 166 (8.0%). Treatment responses observed at the 1-year follow-up visit are summarized in Table 2. Overall, structural incomplete responses were documented in 86 (4.2%) patients. The frequency of structural incomplete responses increased progressively with the level of risk estimated at baseline from 1.5% in the low-risk group, to 5.7% in those at intermediate risk, and 14.5% in the small subset of patients considered at high risk for persistent/recurrent disease (Fig. 2).

FIG. 2.

Prevalence of patients classified as low, intermediate, and high risk, and their rates of structural persistent disease.

Table 2.

Responses to Treatment at 1-Year Evaluation

No. of patients	All	TT + RRA	TT	TL
No. of patients	2071	1191	807	73
Excellent response	1576 (76.1%)	921 (77.3%)	655 (81.1%)	—
Indeterminate response	376 (18.2%)	168 (14.1%)	137 (17%)	71 (97.3%)^a
Biochemical incomplete response	33 (1.6%)	33 (2.8%)	—	—
Structural incomplete response	86 (4.2%)	69 (5.8%)	15 (1.9%)	2 (2.7%)

Not possible to document stability of thyroglobulin values at first evaluation.

RRA, radioactive iodine remnant ablation; TL, thyroid lobectomy; TT, total thyroidectomy.

As shown in Table 3, the ATA risk class assigned at baseline proved to be a significant predictor of the response to treatment observed at the 1-year follow-up visit. It was able to predict the presence of structural disease as opposed to an excellent response. Furthermore, in cases classified as intermediate or high risk, there was a significantly higher probability of a “less-than-excellent response” (i.e., indeterminate or biochemical incomplete or structural incomplete, in decreasing order) (Table 4).

Table 3.

Subgroup Analysis of 1662 Patients with Excellent or Structural Complete Responses at 1 Year

Estimated risk of persistent disease	Likelihood of structural incomplete response at 1 year^a
Estimated risk of persistent disease	Coefficient ± SE	OR (95% CI)	p
Low (reference)	—	—
Intermediate	1.54 ± 0.3	4.67 (2.59–8.43)	<0.0001
High	2.8 ± 0.38	16.48 (7.87–34.5)	<0.0001

Likelihood of structural disease according to the estimated risk of persistent disease calculated at baseline.

Versus excellent response.

CI, confidence interval; OR, odds ratio; SE, standard error.

Table 4.

Likelihood of Less-Than-Excellent Response (All Four Classes; Ordinal Analysis) According to the Estimated Risk of Persistent Disease Calculated at Baseline

ATA risk	Coefficient ± SE	OR (95% CI)	p
Low (reference)	—
Intermediate	0.52 ± 0.12	1.68 (1.34–2.10)	<0.0001
High	1.17 ± 0.19	3.23 (2.23–4.67)	<0.0001

ATA, American Thyroid Association.

We also assessed whether the performance of the initial persistent disease risk estimate was significantly influenced by the practices of individual reporting centers, which included both academic and nonacademic health care facilities distributed throughout Italy (Table 1). Some potential biases (such as the case-mix of patients treated, surgical volumes, and different tools used) are difficult to document but can potentially influence both the initial risk estimation and the subsequent assessment of the response to treatment. The mixed-effect model we used took these into account with a center-specific intercept summarizing unobserved center-specific features. The practices of individual recruiting centers did not influence the prediction of the 1-year status by the ATA risk stratification system (coefficient −0.88 ± 1.53, p = 0.57, in intermediate-risk patients, and −0.77 ± 2.19, p = 0.72, in high-risk patients, for the prediction of structural disease).

Discussion

A reliable estimate of the post-treatment risk of persistent or recurrent disease in a patient with DTC on the basis of clinical, histopathological, and perioperative data provides valuable prognostic information. Importantly, it supports clinicians' efforts to develop personalized treatment and follow-up strategies (4,16). Most cases can be safely managed with less extensive surgery, more selective use of RAI therapy, and relatively relaxed follow-up schedules. The expected benefits are substantial and include reduced health care costs, lower treatment-related morbidity rates, and improved quality of life for patients. These expected benefits must, however, be weighed against the risk of missing those thyroid cancers that warrant intensive therapeutic efforts and close post-treatment surveillance due to their intrinsic biologic aggressiveness.

The ATA risk stratification system was validated in different cohorts around the world (17 –20). Additional features were published in 2016 to include evaluation of the number of vascular invasion foci, number and size of involved lymph nodes, presence of extranodal extension, and (if available) BRAF and TERT promoter mutational status. This updated version has been already validated (5 –7,21 –23). However, all of these validation studies were based on retrospective review and conducted in a few high-volume thyroid cancer referral centers. The current study is based on a large, contemporary cohort of patients with prospectively collected data in many thyroid centers across Italy, including academic and nonacademic institutions, to validate the ATA risk stratification system in predicting persistent disease at the 1-year follow-up visit.

Interinstitutional and interobserver variabilities have been reported in the diagnosis of histological subtypes (24), the detection and quantification of extrathyroidal extension (25), neck ultrasonographic examination (26), and various aspects of RAI administration, including indications, the amount of activity administered, and the method used to ensure appropriately elevated TSH levels at the time of RAI therapy (withdrawal of thyroid hormone replacement therapy vs. rhTSH injections) (27). Treatment centers also vary widely in the number of thyroidectomies performed by their staff each year and this factor is a well-established predictor of outcome (28,29). Furthermore, different assays for Tg and TgAb are used in different institutions.

All of these confounding factors could not be systematically documented on the basis of the information available in our database, but it was reasonable to suspect that some of them could potentially influence both the initial risk estimates in our cohort and the subsequent assessment of the response to treatment. By the use of mixed-effect model, we took into account unobserved center-specific features, documenting that the performance of ATA risk stratification system is not affected by the center in which it is applied.

Our findings demonstrate that the ATA risk stratification system for recurrent/persistent disease is indeed a reliable predictor at the 1-year follow-up evaluation, independent of treatment centers. This is true despite the fact that the likelihood of a “less-than-excellent” response varies across treatment centers, probably as a result of between-center differences in surgical volumes, case mixes, the availability of diagnostic tools, and/or other factors.

It should be stressed that our findings apply exclusively to the prediction of the response to initial treatment documented at the 1-year visit. Risk stratification is in fact a dynamic, ongoing process in which the likelihood of recurrence is periodically reassessed and the management strategy modified as needed (30). Our current findings cannot provide any indication on how the system will perform in predicting the longer term evolution of a DTC. It is important to note, however, that most DTC recurrences are identified within the first 5 years of follow-up after initial treatment (31). Moreover, recent evidence suggests that persistent disease observed at the 1-year follow-up visit is associated with worse outcomes than “recurrences” identified later (12). Prediction of this early outcome may thus have particular clinical relevance.

A limitation of our study is related to the inclusion of non-PTC cases in our cohort, in particular FTCs and Hurthle-cell thyroid carcinomas. There is indeed growing evidence that these tumors behave differently from each other and also from PTCs (24). These tumors represented only 5.2% of the DTCs in our cohort. Therefore, our current findings can shed no light on the specific performance of the ATA risk stratification system in patients with these less common thyroid cancer histologic subtypes.

In conclusion, the ATA risk stratification system is a reliable predictor of short-term outcomes in patients with DTC in real-world clinical settings characterized by appreciable treatment-center heterogeneity in terms of size, location, level of care, diagnostic resources, and local management strategies.

Footnotes

Acknowledgments

We thank all the collaborators of the ITCO Network: Rosa Falcone, Valeria Ramundo, Marco Biffoni, Laura Giacomelli (Department of Translational and Precision Medicine, and Department of Surgical Sciences, Sapienza University of Rome); Michela Massa (Department of Medical Sciences, Fondazione IRCCS Casa Sollievo della Sofferenza, S. Giovanni Rotondo); Efisio Puxeddu (Department of Medicine, University of Perugia); Paolo Limone (Division of Endocrinology, Diabetology, and Metabolism, Mauriziano Umberto I Hospital, Turin); Armando Patrizio (Department of Clinical and Experimental Medicine, University of Pisa); Poupak Fallahi (Department of Translational Research of New Technologies in Medicine and Surgery, University of Pisa); Michela Marina (Department of Medicine and Surgery, Azienda Ospedaliero-Universitaria and University of Parma); Ilaria Messuti (Department of Oncology, Division of Endocrinology and Metabolism, Humanitas-Gradenigo Hospital, University of Turin); Giovanni Savoia, Piernicola Garofalo (Division of Endocrinology Cervello Hospital, Palermo); Maria Grazia Deiana, Federica Presciuttini (Department of Endocrinology, AOU Sant'Andrea, Sapienza University of Rome); Marco Centanni, Camilla Virili (Department of Medico-surgical Sciences and Biotechnologies, Sapienza University of Rome); Valeria Calzolaro (Geriatrics Unit, Department of Clinical and Experimental Medicine, University of Pisa); Maria Beatrice Panarotto (ASST-Spedali Civili, Brescia); Ezio Ghigo (Division of Endocrinology, Diabetology and Metabolism, Department of Medical Sciences, University of Turin); Andrea Palermo (University Hospital Campus Bio-Medico, Rome); Salvatore Tumino (Department of Clinical and Experimental Medicine, University of Catania); Gianluca Aimaretti (Department of Translational Medicine, Università del Piemonte Orientale, Novara); Maria Grazia Chiofalo, Vincenzo Marotta (Istituto Nazionale dei Tumori, IRCCS Fondazione Pascale, Naples); Annamaria D'Amore (Division of Endocrine Surgery, Fondazione Policlinico Gemelli, Catholic University, Rome); Alice Nervo, Marco Gallo, Alessandro Piovesan, Alberto Ragni, Francesco Felicetti (Department of Medical Sciences, Molinette Hospital, University of Turin); Luca Chiovato, Martina Molteni, Giulia Bendotti (Unit of Internal Medicine and Endocrinology, Laboratory for Endocrine Disruptors, Department of Internal Medicine and Therapeutics, Istituti Clinici Scientifici Maugeri IRCCS, University of Pavia); Lorenzo Bresciani, Laura Locati (Istituto Nazionale dei Tumori di Milano).

Author Disclosure Statement

No competing financial interests exist.

Funding Information

Writing support was provided by Marian Everett Kent, BSN, and funded by the Fondazione Umberto Di Mario. The study was supported by the Sapienza University of Rome Research Grant (RM11916B83A211FC) to C.D.

Supplementary Material

Supplementary Table S1

References

Haugen

, Alexander

, Bible

, Doherty

, Mandel

, Nikiforov

, Pacini

, Randolph

, Sawka

, Schlumberger

, Schuff

, Sherman

, Sosa

, Steward

, Tuttle

, Wartofsky

. 2016. 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid, 26:1–133.

Lamartina

, Grani

, Durante

, Filetti

. 2018. Recent advances in managing differentiated thyroid cancer. F1000Res, 7:86.

Cooper

, Doherty

, Haugen

, Hauger

, Kloos

, Lee

, Mandel

, Mazzaferri

, McIver

, Pacini

, Schlumberger

, Sherman

, Steward

, Tuttle

. 2009. Revised American Thyroid Association management guidelines for patients with thyroid nodules and differentiated thyroid cancer. Thyroid, 19:1167–1214.

Pitoia

, Jerkovich

. 2019. Dynamic risk assessment in patients with differentiated thyroid cancer. Endocr Relat Cancer, 26:R553–R566.

Park

, Kim

, Song

, Oh

, Kim

, Kwon

, Jeon

, Kim

, Shong

, Kim

. 2017. Dynamic risk stratification for predicting recurrence in patients with differentiated thyroid cancer treated without radioactive iodine remnant ablation therapy. Thyroid, 27:524–530.

Pitoia

, Jerkovich

, Urciuoli

, Schmidt

, Abelleira

, Bueno

, Cross

, Tuttle

. 2015. Implementing the modified 2009 American Thyroid Association risk stratification system in thyroid cancer patients with low and intermediate risk of recurrence. Thyroid, 25:1235–1242.

Momesso

, Vaisman

, Yang

, Bulzico

, Corbo

, Vaisman

, Tuttle

. 2016. Dynamic risk stratification in patients with differentiated thyroid cancer treated without radioactive iodine. J Clin Endocrinol Metab, 101:2692–2700.

Lamartina

, Durante

, Lucisano

, Grani

, Bellantone

, Lombardi

, Pontecorvi

, Arvat

, Felicetti

, Zatelli

, Rossi

, Puxeddu

, Morelli

, Torlontano

, Crocetti

, Montesano

, Giubbini

, Orlandi

, Aimaretti

, Monzani

, Attard

, Francese

, Antonelli

, Limone

, Rossetto

, Fugazzola

, Meringolo

, Bruno

, Tumino

, Ceresini

, Centanni

, Monti

, Salvatore

, Spiazzi

, Mian

, Persani

, Barbaro

, Nicolucci

, Filetti

. 2017. Are evidence-based guidelines reflected in clinical practice? An analysis of prospectively collected data of the Italian thyroid cancer observatory. Thyroid, 27:1490–1497.

Filetti

, Durante

, Hartl

, Leboulleux

, Locati

, Newbold

, Papotti

, Berruti

. 2019. Thyroid cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol, 30:1856–1883.

10.

Leenhardt

, Erdogan

, Hegedus

, Mandel

, Paschke

, Rago

, Russ

. 2013. 2013 European thyroid association guidelines for cervical ultrasound scan and ultrasound-guided techniques in the postoperative management of patients with thyroid cancer. Eur Thyroid J, 2:147–159.

11.

Lamartina

, Grani

, Biffoni

, Giacomelli

, Costante

, Lupo

, Maranghi

, Plasmati

, Sponziello

, Trulli

, Verrienti

, Filetti

, Durante

. 2016. Risk stratification of neck lesions detected sonographically during the follow-up of differentiated thyroid cancer. J Clin Endocrinol Metab, 101:3036–3044.

12.

Sapuppo

, Tavarelli

, Belfiore

, Vigneri

, Pellegriti

. 2019. Time to separate persistent from recurrent differentiated thyroid cancer: different conditions with different outcomes. J Clin Endocrinol Metab, 104:258–265.

13.

Christensen RHB 2019 ordinal—Regression Models for Ordinal Data. R package version 2019. 12–10. Available at https://CRAN.R-project.org/package=ordinal (accessed January 10, 2020 ).

14.

Bates

, Mächler

, Bolker

, Walker

. 2015. Fitting linear mMixed-effects models using lme4. J Stat Softw, 67:1–48.

15.

Team

, 2014. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.

16.

Lamartina

, Grani

, Durante

, Borget

, Filetti

, Schlumberger

. 2018. Follow-up of differentiated thyroid cancer—what should (and what should not) be done. Nat Rev Endocrinol, 14:538–551.

17.

Castagna

, Maino

, Cipri

, Belardini

, Theodoropoulou

, Cevenini

, Pacini

. 2011. Delayed risk stratification, to include the response to initial treatment (surgery and radioiodine ablation), has better outcome predictivity in differentiated thyroid cancer patients. Eur J Endocrinol, 165:441–446.

18.

Tuttle

, Tala

, Shah

, Leboeuf

, Ghossein

, Gonen

, Brokhin

, Omry

, Fagin

, Shaha

. 2010. Estimating risk of recurrence in differentiated thyroid cancer after total thyroidectomy and radioactive iodine remnant ablation: using response to therapy variables to modify the initial risk estimates predicted by the new American Thyroid Association staging system. Thyroid, 20:1341–1349.

19.

Vaisman

, Momesso

, Bulzico

, Pessoa

, Dias

, Corbo

, Vaisman

, Tuttle

. 2012. Spontaneous remission in thyroid cancer patients after biochemical incomplete response to initial therapy. Clin Endocrinol (Oxf), 77:132–138.

20.

Pitoia

, Bueno

, Urciuoli

, Abelleira

, Cross

, Tuttle

. 2013. Outcomes of patients with differentiated thyroid cancer risk-stratified according to the American Thyroid Association and Latin American thyroid society risk of recurrence classification systems. Thyroid, 23:1401–1407.

21.

Lee

, Lee

, Moon

, Lee

, Kang

, Jeong

, Nam

, Chung

, Jo

, Lee

. 2017. Practical performance of the 2015 American Thyroid Association guidelines for predicting tumor recurrence in patients with papillary thyroid cancer in South Korea. Thyroid, 27:174–181.

22.

van Velsen

EFS

, Stegenga

, van Kemenade

, Kam

BLR

, van Ginhoven

, Visser

, Peeters

. 2019. Evaluating the 2015 American Thyroid Association risk stratification system in high-risk papillary and follicular thyroid cancer patients. Thyroid, 29:1073–1079.

23.

Albano

, Bertagna

, Bonacina

, Durmo

, Cerudelli

, Gazzilli

, Panarotto

, Formenti

, Mazziotti

, Giustina

, Giubbini

. 2018. Possible delayed diagnosis and treatment of metastatic differentiated thyroid cancer by adopting the 2015 ATA guidelines. Eur J Endocrinol, 179:143–151.

24.

Grani

, Lamartina

, Durante

, Filetti

, Cooper

. 2018. Follicular thyroid cancer and Hurthle cell carcinoma: challenges in diagnosis, treatment, and clinical management. Lancet Diabetes Endocrinol, 6:500–514.

25.

, Wenig

, Haser

, Rowe

, Asa

, Baloch

, Du

, Faquin

, Fellegara

, Giordano

, Ghossein

, LiVolsi

, Lloyd

, Mete

, Ozbek

, Rosai

, Suster

, Thompson

, Turk

, Urken

. 2016. Inter-observer variation in the pathologic identification of minimal extrathyroidal extension in papillary thyroid carcinoma. Thyroid, 26:512–517.

26.

Grani

, Lamartina

, Cantisani

, Maranghi

, Lucia

, Durante

. 2018. Interobserver agreement of various thyroid imaging reporting and data systems. Endocr Connect, 7:1–7.

27.

Rachinsky

, Rajaraman

, Leslie

, Zahedi

, Jefford

, McGibbon

, Young

, Pathak

, Badreddine

, De Brabandere

, Fong

, Van Uum

. 2016. Regional variation across Canadian Centers in Radioiodine Administration for thyroid remnant ablation in well-differentiated thyroid cancer diagnosed in 2000–2010. J Thyroid Res, 2016:2867916.

28.

Liang

, Liu

, Mok

, Shi

. 2016. Associations of volume and thyroidectomy outcomes: a nationwide study with systematic review and meta-analysis. Otolaryngol Head Neck Surg, 155:65–75.

29.

Al-Qurayshi

, Robins

, Hauch

, Randolph

, Kandil

. 2016. Association of surgeon volume with outcomes and cost savings following thyroidectomy: a national forecast. JAMA Otolaryngol Head Neck Surg, 142:32–39.

30.

Tuttle

, Alzahrani

. 2019. Risk stratification in differentiated thyroid cancer: from detection to final follow-up. J Clin Endocrinol Metab, 104:4087–4100.

31.

Durante

, Montesano

, Torlontano

, Attard

, Monzani

, Tumino

, Costante

, Meringolo

, Bruno

, Trulli

, Massa

, Maniglia

, D'Apollo

, Giacomelli

, Ronga

, Filetti

, Group

. 2013. Papillary thyroid cancer: time course of recurrences during postsurgery surveillance. J Clin Endocrinol Metab, 98:636–642.