International Phase IV Field Study for the Reliability and Validity of the European Organisation for Research and Treatment of Cancer Thyroid Cancer Module EORTC QLQ-THY34

Abstract

Purpose:

The aim of this study was to validate the new European Organisation for Research and Treatment of Cancer Quality of Life Thyroid Cancer Module (EORTC QLQ-THY34).

Methods:

We enrolled 437 thyroid cancer patients from 17 countries. One group (n = 303), undergoing treatment or best supportive care, completed the questionnaires at three time points (before therapy [t1], 6 weeks later [t2], and 6 months after t2 [t3]). A second group (survivors ≥2 years after diagnosis, n = 134) completed it at a random baseline time point and a second time 1 week later. We determined internal consistency (using Cronbach's alpha), the scale structure (with confirmatory factor analysis), and discriminant validity (using known-group comparisons). Group 1 data were used to assess responsiveness and group 2 data to determine test-retest reliability using intra-class correlations (ICC).

Results:

All 34 items fulfilled the criteria to be kept in the questionnaire. Cronbach's alpha was >0.70 in 8 of the 9 multi-item scales. All standardized factor loadings exceeded 0.40, confirming the proposed scale structure. The ICC was >0.70 in all scales expressing good test-retest reliability. Differences in scale scores between patients with different histology were >5 points in all scales. In all but one of the pre-specified scales (Dry Mouth), changes over time were ≥|4| points between at least two time points.

Conclusion:

The EORTC QLQ-THY34 with its 9 multi-item and 8 single-item scales is a reliable and valid tool to measure quality of life in thyroid cancer patients and can be used in future trials and studies.

Introduction

When evaluating the effect and potential benefit of new treatments in oncology, the patients' perspective should be included. This can be done by administering well-tailored questionnaires asking about their health-related quality of life (QoL) in a structured way. For this purpose, a number of reliable and valid instruments exist, for example, the European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Core Instrument (EORTC QLQ-C30)¹ or the Functional Assessment of Cancer Therapy² that is used worldwide.^3,4

However, according to a recent systematic review,⁵ there are only a handful of disease-specific instruments for thyroid cancer patients,^6
–8 and, to date, none has been tested for responsiveness (i.e., the instrument's ability to detect changes in QoL over time). This can lead to an under-reporting of problems that are specific to this group of patients, for example, side-effects of radioactive iodine (RAI) treatment or hormonal imbalances.^9

–12

Therefore, the EORTC Quality of Life Group (QLG) decided to develop a new instrument for this purpose, starting with an extensive phase I ensuring content validity,¹³ followed by phase II in which items were generated with a response format (4-point Likert scale) and time frame (1 week), same as the EORTC QLQ-C30. In phase III, the new thyroid cancer module (EORTC QLQ-THY34) was pilot-tested internationally.¹⁴

Finally, in the present work, its psychometric properties were tested prospectively in a large group of thyroid cancer patients that was clinically, culturally, and linguistically diverse. This phase IV validation study included two different groups of patients for different purposes: The first one, aiming at measuring the instrument's sensitivity to change (“responsiveness”), included patients who were about to undergo initial or further treatment or best supportive care. The second group, aiming at assessing test-retest reliability, included survivors in whom no rapid changes in QoL were expected based on their clinical situation. A patient representative was involved in the entire module development process. The results of the final phase IV are presented herein.

Methods

Target population and patient selection criteria

The target population for the Thyroid Cancer Module is patients with all types of thyroid cancer histology (papillary, follicular, Hurthle-cell, poorly differentiated, medullary, anaplastic, and mixed).

Two groups of patients were enrolled into the validation study: those just about to receive initial or further treatment or best supportive care (Group 1) and those whose diagnosis dated back at least 24 months without structural evidence of disease (based on imaging) and without anti-neoplastic treatment within the past 12 months (Group 2, survivors).

Patients had to fulfill the following criteria to be eligible: cytologically or histologically verified thyroid cancer (ICD-10, C73), ability to understand and complete the questionnaires (language proficiency and cognitive functioning as judged by the local study coordinator on inclusion), age 16 years or older, and written informed consent. Patients with second malignancies (i.e., any ICD-10 C diagnosis except non-melanoma skin cancer) and patients who had been included in phase I–III of the module development could not participate.

Sampling

Participants were enrolled following a sampling matrix capturing histology and treatment type to ensure that the most frequent tumor types and treatment protocols were represented in the study, and that each cell of the matrix had enough observations for reliable statistical tests.¹⁵ Participants could be approached at the inpatient or outpatient setting.

Data collection and study design

Eligible patients were informed about the study and given a Patient Information Sheet in the local language, which included written information and an invitation to participate in accordance with ethical and governance requirements of each participating center. Institutional Review Board Approval: The Ethics Committee of Rhineland-Palatinate Medical Association was responsible for the study protocol review for the principal investigator and approved it with reference number 837.406.17 (11240).

The patients in Group 1 were approached three times. The first data collection occurred in the 4 weeks before treatment or best supportive care started (t1). “Before treatment” was defined as one of the following: before surgery (initial surgery, completion thyroidectomy, additional neck surgery, or distant site surgery), before the initiation of non-surgical treatment (RAI, tyrosine kinase inhibitors [TKI], external beam radiation therapy, radiofrequency ablation, or other systemic or local anti-tumor treatment), or before the initiation of best supportive care (this excluded the aforementioned treatment modalities).

Patients could be enrolled either after initial diagnosis or after the diagnosis of residual or recurrent disease. They were approached again 6 weeks after the first day of treatment (t2), and 6 months after the second time point (t3).

Data collection took place before the patient was informed about any new results of tests or clinical examinations. A window of 1 week before and 1 week after the exact date of t2 and a window of 2 weeks before and 2 weeks after the exact date of t3 were allowed for data collection. The exact dates of t1, t2, and t3 were documented on the case report forms.

The survivors in Group 2 could be enrolled at any given time point as long as it was at least 24 months after the thyroid cancer diagnosis. They participated a second time 1 week (time window: −1 day/+3 days) after the first administration of the questionnaire to measure test-retest reliability. This time window was shorter compared with the interval in Group 1 to make clinical changes in QoL even more unlikely.

Data collection at t1 was usually performed in person. At the follow-up time points, whenever required, questionnaires could be mailed out to participants or could be completed via telephone interviews or online. A system for electronic data capture was established using the Computer Based Health Evaluation System.¹⁶ The method of questionnaire administration was documented on the case report form.

Instruments

Patients received two questionnaires at each measurement point, the EORTC QLQ-C30¹ and the EORTC QLQ-THY34.¹⁴ The latter is protected by copyright by the EORTC (It is included in the Supplementary Data for reference.).

At t1, they also completed a debriefing questionnaire assessing patient acceptability.

Statistical analysis

Data analysis was conducted in accordance with the EORTC Module Development Guidelines¹⁵ and with the recommendations of the International Society of Quality of Life.¹⁷

Cronbach's alpha was computed for the hypothesized scales, elaborated in phase III of this module's development.¹⁴ If indicated by this step, the scale structure was changed. The threshold for Cronbach's alpha was set at 0.70.¹⁷

We had defined a priori a list of criteria for items to be included or excluded from the questionnaire. An item should be kept if five out of the following seven criteria are fulfilled: mean score >1.5 over all groups of patients, neither floor nor ceiling effects (no floor effect defined as responses three and four >10%, no ceiling effect defined as responses one and two >10%), full range of possible responses used, percentage of patients saying the item is difficult to understand <5%, percentage of patients saying the item is upsetting <5%, percentage of patients completing the item >95%, Cronbach's alpha of hypothesized scale >0.65.

We performed confirmatory factor analysis to examine the scale structure of the EORTC QLQ-THY34 and calculated standardized factor loadings for each item with regard to the corresponding scale, whereby loadings >0.40 were considered adequate.¹⁸ We evaluated the model-data-fit with the Tucker-Lewis Index (TLI) and the Comparative Fit Index (CFI), with both indices considered to signpost good fit if they exceed 0.95¹⁹ as well as with the Root Mean Squared Error of Approximation (RMSEA), which should be below 0.06.¹⁹

According to the guidelines of the EORTC QLG,¹⁵ scale scores were constructed by summing up all item responses of that scale, and then this raw score was transformed from 0 to 100. If at least half of the items in a particular scale were not available, no score was calculated for that scale. We then calculated the range, mean and standard deviation, median and interquartile range for all scales, and the percentage of missing values per scale.

Convergent and divergent validity were investigated by correlating the scales of the EORTC QLQ-C30 and the EORTC QLQ-THY34.

To measure discriminant validity, the method of known-group comparison was used, that is, the mean scores of the EORTC QLQ-THY34 scales of relevant clinical subgroups (different histology; survivors vs. patients undergoing treatment) were compared by calculating the delta between the groups.

Responsiveness (sensitivity to change) was determined by calculating the mean changes between t1, t2, and t3 in Group 1, stratified by treatment group and focusing on the following issues: Voice, Shoulder Functioning, and/or Discomfort in Head and Neck due to surgery; Dry Mouth due to RAI; Cramps, Exhaustion, and/or Voice due to TKI. Our assumption was that QoL would decrease between t1 and t2 and increase between t2 and t3 in the pre-specified domains. However, changes were only to be described with no defined thresholds.

Test-retest reliability was ascertained by computing intra-class correlation (ICC) coefficients between t1 and t2 in Group 2.

Data entry, cleaning, and analysis were conducted using STATA (StataCorp. 2017. Stata Statistical Software: Release 16; StataCorp LP, College Station, TX).

Ethical approval

All procedures performed were in accordance with the ethical standards of the institutional research committees and with the 1964 Helsinki declaration and its later amendments of comparable ethical standards. All sites obtained ethical approval in accordance with regional and national requirements. Approval number from the principal investigator's institution: 837.406.17 (11240).

Consent to participate

Written informed consent was obtained from all individual participants included in the study.

Results

Sample characteristics

Overall, 459 patients were contacted to participate (324 in Group 1, 135 in Group 2). Of them, 21 had follicular cytology that turned out to be adenomas on histological examination after surgery and their questionnaires were thus excluded, according to the study protocol, leaving 438 patients (303 in Group 1, 135 in Group 2) of whom 437 participated at least once, one patient declined at t1 (see flowchart in Fig. 1).

FIG. 1.

Patient flow through the study.

In Group 1, 220 participated at all time points, 55 twice, and 20 only once (at t2, 4 patients had died already, 10 declined, and 19 could not be contacted; at t3, another 10 patients had died, another 10 declined, and another 24 could not be contacted). In Group 2, 124 participated at all time points and 10 only once (6 declined and 4 could not be contacted any more).

The 437 participants came from 21 institutions in 17 countries (Australia n = 21, Austria n = 23, Belgium n = 3, Brazil n = 18, Cyprus n = 13, Germany n = 21, Greece n = 28, Italy n = 70, Japan n = 15, Jordan n = 46, Norway n = 26, Portugal n = 17, Spain n = 20, Sweden n = 28, Switzerland n = 67, The Netherlands n = 15, and the UK n = 6). The EORTC QLQ-THY34 was thereby validated in 12 languages, sometimes with country-specific variations: Arabic for Jordan, Dutch, English, French for Europe, German, Greek, Italian, Japanese, Norwegian, Portuguese for Portugal, Portuguese for Brazil, Spanish for Spain, and Swedish.

Clinical and demographic characteristics of the sample are displayed in Table 1.

Table 1.

Demographic and Clinical Characteristics of the Participants (n = 437)

	Total	Group 1 (n = 303)	Group 2 (n = 134)
Sex
Male	118 (27%)	93 (31%)	25 (19%)
Female	319 (73%)	210 (69%)	109 (81%)
Age (in years)
<40	131 (30%)	102 (34%)	29 (22%)
40–49	81 (19%)	53 (17%)	28 (21%)
50–59	84 (19%)	53 (17%)	31 (23%)
60–69	69 (16%)	51 (17%)	18 (13%)
70–79	57 (13%)	38 (13%)	19 (14%)
80+	15 (3%)	6 (2%)	9 (7%)
Education
<10 years	68 (16%)	61 (20%)	7 (5%)
10 years	45 (10%)	36 (12%)	9 (7%)
>10 years	311 (71%)	203 (67%)	108 (81%)
Missing information	13 (3%)	3 (1%)	10 (7%)
Histology
Papillary	304 (70%)	213 (70%)	91 (68%)
Follicular	45 (10%)	23 (8%)	22 (16%)
Hurthle-cell	10 (2%)	10 (3%)	0 (0%)
Poorly differentiated	7 (2%)	7 (2%)	0 (0%)
Medullary	47 (11%)	27 (9%)	20 (15%)
Anaplastic	19 (4%)	19 (6%)	0 (0%)
Mixed	5 (1%)	4 (1%)	1 (1%)
UICC stage^a
UICC I	245 (56%)	169 (56%)	76 (57%)
UICC II	59 (14%)	41 (14%)	18 (13%)
UICC III	31 (7%)	20 (7%)	11 (8%)
UICC IV	68 (16%)	56 (18%)	12 (9%)
Unknown	34 (8%)	17 (6%)	17 (13%)
ATA response to therapy^b
No evidence of disease	143 (33%)	44 (15%)	99 (74%)
Indeterminate	22 (5%)	15 (5%)	7 (5%)
Biochemically incomplete	21 (5%)	7 (2%)	14 (10%)
Structural disease	160 (37%)	153 (50%)	7 (5%)
Unknown to the collaborator	84 (19%)	77 (25%)	7 (5%)
Missing information	7 (2%)	7 (2%)	0 (0%)
Hypoparathyroidism
No	296 (68%)	204 (67%)	92 (69%)
Transient (or too early to know)	53 (12%)	40 (13%)	13 (10%)
Permanent (≥6 months)	39 (9%)	18 (6%)	21 (16%)
Unknown to the collaborator	47 (11%)	41 (14%)	6 (4%)
Treatment scheme
No treatment	4 (1%)	4 (1%)	0 (0%)
Mono-modal	164 (38%)	103 (34%)	61 (46%)
Multi-modal	268 (61%)	196 (65%)	72 (54%)
Missing information	1 (0.2%)	0 (0%)	1 (1%)
Type of surgery
None	29 (7%)	24 (8%)	5 (4%)
Less than total thyroidectomy	44 (10%)	25 (8%)	19 (14%)
Total thyroidectomy	364 (83%)	254 (84%)	110 (82%)
Other treatment
Radiotherapy for local control	23 (5%)	18 (6%)	5 (4%)
Radiotherapy for distant metastases	25 (6%)	23 (8%)	2 (1%)
TKI ever received	36 (8%)	35 (12%)	1 (1%)
Chemotherapy ever received	14 (3%)	13 (4%)	1 (1%)
Radioiodine for ablation	148 (34%)	121 (40%)	27 (20%)
Radioiodine for therapy	127 (29%)	84 (28%)	43 (32%)

Group 1, patients about to start treatment; Group 2, survivors.

“Less than total thyroidectomy” refers to lobectomy with or without resection of the thyroid isthmus, as well as to more limited surgery, such as resection of only the isthmus or of a thyroglossal duct remnant, and others.

“Hypoparathyroidism,” “Type of surgery,” and “Other treatment” all time points combined.

Usually taken from t2. If not available, then t1 or t3.

At t1.

ATA, American Thyroid Association; TKI, tyrosine kinase inhibitors; UICC, Union Internationale Contre le Cancer.

Item characteristics

Table 2 describes the various item characteristics. For each item, it is reported how many of the pre-defined criteria were fulfilled. Based on the tests for internal consistency, the scale structure was slightly changed in comparison to the hypothesized scale structure developed in phase III. Item 46 (“Have you felt restless or agitated?”) worked better with the other items of the scale Treatment- and Disease-related Worry than with the item 47 “Have you had a rapid heartbeat?” Hence, it was decided to use item 47 as a single-item scale and combine item 46 with the worry scale.

Table 2.

Item Characteristics

No.	Item	Cronbach's alpha of hypothesized scale (average over t1–t3)	Mean (average over t1–t3)	Proportion scores 3/4	Proportion scores 1/2	Proportion upsetting	Proportion difficult	Proportion complete	Number of criteria fulfilled
q 31	Have you had sudden attacks of tiredness?	0.85	1.9	0.19	0.81	0.00	0.01	1.00	7
q 32	Have you felt mentally exhausted?	0.85	1.9	0.23	0.77	0.00	0.01	1.00	7
q 33	Have you felt physically exhausted?	0.85	2.1	0.26	0.74	0.00	0.00	1.00	7
q 34	Have you had pain in your throat?	0.72	1.5	0.08	0.92	0.00	0.00	1.00	5
q 35	Have you had any discomfort in your neck?	0.72	1.8	0.22	0.78	0.00	0.00	1.00	7
q 36	Have you had problems with hoarseness?	0.89	1.6	0.16	0.84	0.00	0.01	0.99	7
q 37	Has your voice sounded different as a result of your disease or treatment?	0.89	1.7	0.19	0.80	0.00	0.01	1.00	7
q 38	Have you had a tired voice?	0.89	1.7	0.17	0.83	0.00	0.01	1.00	7
q 39	Have you had thin or lifeless hair as a result of your disease or treatment?	0.85	1.6	0.16	0.84	0.00	0.00	1.00	7
q 40	Have you lost any hair?	0.85	1.6	0.15	0.85	0.00	0.00	1.00	7
q 41	Have you had problems swallowing solid food?	0.72	1.4	0.07	0.93	0.00	0.00	1.00	5
q 42	Have you choked when swallowing?	0.72	1.3	0.04	0.96	0.00	0.00	1.00	5
q 43	Have you had a dry mouth?	n.a.	1.8	0.19	0.81	0.00	0.00	1.00	7
q 44	Have you had problems tolerating heat or cold?	n.a.	1.8	0.21	0.79	0.00	0.01	1.00	7
q 45	Have you felt physically less attractive as a result of your disease or treatment?	n.a.	1.7	0.20	0.80	0.00	0.00	1.00	7
q 46	Have you felt restless or agitated?	0.72	1.9	0.23	0.77	0.00	0.00	1.00	7
q 47	Have you had a rapid heartbeat?	n.a.	1.5	0.09	0.91	0.00	0.00	1.00	5
q 48	Have you had problems raising your arm or moving it sideways?	n.a.	1.4	0.10	0.90	0.00	0.00	1.00	5
q 49	Have you felt as though there was a knot in your throat?	0.72	1.7	0.15	0.85	0.00	0.00	1.00	7
q 50	Have you worried about a possible recurrence of the disease?	0.72	2.2	0.34	0.65	0.02	0.00	0.99	7
q 51	Have you worried about having to come off your thyroid hormone replacement tablets to prepare for a radioiodine body scan or radioiodine treatment?	0.72	1.7	0.17	0.77	0.00	0.05	0.94	5
q 52	Have you worried about having to take drugs for the rest of your life?	0.72	1.7	0.16	0.84	0.01	0.00	1.00	7
q 53	Have you had pain in your joints?	n.a.	1.8	0.23	0.77	0.00	0.00	1.00	7
q 54	Have you had tingling or numbness in your fingers or toes?	0.54	1.6	0.16	0.84	0.00	0.01	1.00	6
q 55	Have you had tingling or numbness around your mouth?	0.54	1.2	0.04	0.96	0.00	0.00	1.00	4
q 56	Have you had any muscle cramps?	n.a.	1.6	0.11	0.89	0.00	0.00	1.00	7
q 57	Have you found it upsetting to see those close to you distressed as a result of your disease or treatment?	0.88	2.0	0.25	0.74	0.02	0.01	0.99	7
q 58	Have you worried about the future of people who are important to you?	0.88	2.3	0.37	0.62	0.02	0.00	1.00	7
q 59	Have you worried how those close to you will cope with your illness and treatment?	0.88	2.1	0.29	0.71	0.00	0.01	1.00	7
q 60	Have you worried that you will be a burden to others?	0.88	2.0	0.29	0.70	0.01	0.00	1.00	7
q 61	Have you had any problems with your job or your education because of your disease or treatment?	n.a.	1.7	0.17	0.82	0.00	0.00	0.99	7
q 62	Have you felt supported by your doctors?	0.83	1.8	0.18	0.82	0.00	0.02	0.99	7
q 63	Have you felt supported by other health care professionals (e.g., nurse)?	0.83	1.9	0.21	0.77	0.00	0.03	0.98	7
q 64	Have you felt supported by your family members or friends?	0.83	1.4	0.07	0.93	0.00	0.01	1.00	5

n.a., not applicable (single item scale). If not further specified, the data presented are for t2.

All items but one fulfilled at least five of the seven criteria, and item 55 fulfilled four criteria. While overall, items 54 and 55 taken together as a scale had suboptimal Cronbach's alpha, it was adequate in certain subgroups of patients for whom this domain is most relevant (see next paragraph). It was, therefore, decided to keep item 55 and to keep it together with item 54 in one scale. In consequence, the EORTC thyroid cancer module consists of 34 items with 9 multi-item and 8 single-item scales.

Scale characteristics, reliability, and construct validity

The scales resulting from the items exhibit very good characteristics (Table 3). For all of them, the entire range was used and missing values were very few.

Table 3.

Scale Characteristics

Scale	Mean score	SD	Median score	IQR	Min	Max	Skewness	Missing
Group 1, t2
Exhaustion (EX)	32.1	26.0	33.3	33.3	0	100	0.8	0%
Discomfort in the head and neck (DI)	26.1	21.9	22.2	22.2	0	100	0.9	0%
Voice (VO)	26.5	28.6	22.2	44.4	0	100	0.9	0%
Hair problems (HA)	21.1	30.9	0.0	33.3	0	100	1.5	0%
Swallowing (SW)	11.7	19.7	0.0	16.7	0	100	2.0	0%
Treatment- and disease-related worry (TD)	31.8	23.8	25.0	38.9	0	100	0.7	0%
Tingling or numbness (TI)	15.8	21.0	8.3	33.3	0	100	1.6	0%
Worry about important others (WO)	40.6	28.2	33.3	50.0	0	100	0.5	0%
Lacking social support (SO)	21.0	23.2	16.7	33.3	0	100	1.1	0%
Dry mouth (DM)	26.3	29.9	33.3	33.3	0	100	0.9	0%
Altered temperature tolerance (TO)	25.3	30.3	0.0	33.3	0	100	0.9	0%
Body image (BI)	26.8	32.0	0.0	33.3	0	100	0.9	0%
Shoulder functioning (SH)	15.2	27.5	0.0	33.3	0	100	1.9	0%
Joint pain (JP)	28.6	31.7	33.3	33.3	0	100	0.8	0%
Cramps (CR)	17.7	25.3	0.0	33.3	0	100	1.4	0%
Impact on job or education (JE)	24.5	32.0	0.0	33.3	0	100	1.1	1%
Rapid heartbeat (RH)	14.7	22.7	0.0	33.3	0	100	1.5	0%

Group 2, t1
Exhaustion (EX)	27.0	25.9	22.2	33.3	100	0.9	0%
Discomfort in the head and neck (DI)	13.3	16.2	11.1	22.2	67	1.2	0%
Voice (VO)	14.1	21.5	0.0	22.2	100	2.1	0%
Hair problems (HA)	16.4	26.3	0.0	16.7	100	1.6	0%
Swallowing (SW)	9.2	16.7	0.0	16.7	83	2.0	0%
Treatment- and disease-related worry (TD)	21.8	21.4	16.7	25.0	83	1.1	0%
Tingling or numbness (TI)	15.2	20.5	0.0	16.7	100	1.6	0%
Worry about important others (WO)	27.1	29.3	16.7	41.7	100	1.0	1%
Lacking social support (SO)	25.4	30.8	11.1	44.4	100	1.1	1%
Dry mouth (DM)	21.9	29.5	0.0	33.3	100	1.2	0%
Altered temperature tolerance (TO)	26.1	33.0	0.0	66.7	100	0.9	0%
Body image (BI)	12.9	23.8	0.0	33.3	100	1.8	0%
Shoulder functioning (SH)	9.3	21.8	0.0	0.0	100	2.4	1%
Joint pain (JP)	33.3	33.2	33.3	66.7	100	0.5	0%
Cramps (CR)	24.1	30.8	0.0	33.3	100	1.2	1%
Impact on job or education (JE)	10.7	24.2	0.0	0.0	100	2.5	2%
Rapid heartbeat (RH)	18.0	25.5	0.0	33.3	100	1.5	1%

IQR, interquartile range; SD, standard deviation.

The internal consistency coefficients of the various scales were all above 0.70 except for one scale (Table 4). This scale with suboptimal Cronbach's alpha (Tingling and Numbness) had good values in certain groups of patients, for example, an average of 0.68 in patients with hypoparathyroidism, 0.70 in patients with medullary thyroid cancer, and 0.87 in patients with anaplastic thyroid cancer. The test-retest reliability was very good in all scales.

Table 4.

Internal Consistency, Test-Retest Reliability, and Construct Validity

Scale	N items	Items	Cronbach's alpha	Test-retest (ICC)	Standardized factor loadings
Exhaustion (EX)	3	31, 32, 33	0.85	0.83	0.76–0.87
Discomfort in the head and neck (DI)	3	34, 35, 49	0.72	0.80	0.65–0.71
Voice (VO)	3	36, 37, 38	0.89	0.88	0.81–0.88
Hair problems (HA)	2	39, 40	0.85	0.95	0.77–0.98
Swallowing (SW)	2	41, 42	0.72	0.87	0.73–0.79
Treatment- and disease-related worry (TD)	4	46, 50, 51, 52	0.72	0.92	0.51–0.69
Tingling or numbness (TI)	2	54, 55	0.54	0.90	0.52–0.78
Worry about important others (WO)	4	57, 58, 59, 60	0.88	0.93	0.77–0.88
Lacking social support (SO)	3	62, 63, 64	0.83	0.76	0.64–0.87
Dry mouth (DM)	1	43	—	0.91
Altered temperature tolerance (TO)	1	44	—	0.83
Body image (BI)	1	45	—	0.85
Shoulder functioning (SH)	1	48	—	0.88
Joint pain (JP)	1	53	—	0.85
Cramps (CR)	1	56	—	0.72
Impact on job or education (JE)	1	61	—	0.83
Rapid heartbeat (RH)	1	47	—	0.76

Cronbach's alpha presents the average over all time points.

ICC, intra-class correlation coefficient between t1 and t2 in Group 2.

Standardized factor loadings were all above 0.40 (Table 4). The CFI was 0.95, the TLI was 0.94, and the RMSEA was 0.05, supporting the assumption that the constructs underlying the scales are valid.

Convergent and divergent validity

The correlations between EORTC QLQ-C30 and EORTC QLQ-THY34 scales indicate that related concepts are, indeed, associated whereas others are not (Supplementary Tables S1–S5). For example, the Fatigue scale of the EORTC QLQ-C30 correlates highly (e.g., 0.79 at t1, 0.74 at t2, 0.84 at t3) with the Exhaustion scale of the EORTC QLQ-THY34 (convergent validity) but not entirely, which shows that the measured constructs are similar but not the same. In other words: the Exhaustion scale of the EORTC QLQ-THY34 captures partially different aspects than the fatigue scale of the EORTC QLQ-C30.

Evidence for divergent validity is, for example, that the correlations of different constructs, such as Shoulder Functioning (measured with the EORTC QLQ-THY34) and Nausea and Vomiting (measured with the EORTC QLQ-C30), are weak.

Discriminant validity

There are differences of ≥5 points in nearly all scales when we compare the scale scores of patients under treatment with survivors (Table 5). Similarly, the comparison of the scale scores regarding different histology types indicated differences of ≥5 points in all of the scales at least at one time point and one comparison between patient groups (Table 6), showing that the module is able to differentiate between clinically distinct groups.

Table 5.

Score Differences Between Patients Scheduled for Treatment (Group 1) and Survivors (Group 2)

	Group 1	Group 2	Difference (absolute values)
Exhaustion (EX)	32.1	25.9	6
Discomfort in the head and neck (DI)	26.1	12.3	14
Voice (VO)	26.5	12.8	14
Hair problems (HA)	21.1	13.7	7
Swallowing (SW)	11.7	8.9	3
Treatment- and disease-related worry (TD)	31.8	18.8	13
Tingling or numbness (TI)	15.8	12.0	4
Worry about important others (WO)	40.6	25.6	15
Lacking social support (SO)	21.0	22.8	2
Dry mouth (DM)	26.3	22.0	4
Altered temperature tolerance (TO)	25.3	24.2	1
Body image (BI)	26.8	11.6	15
Shoulder functioning (SH)	15.2	11.0	4
Joint pain (JP)	28.6	27.4	1
Cramps (CR)	17.7	15.6	2
Impact on job or education (JE)	24.5	8.9	16
Rapid heartbeat (RH)	14.7	15.4	1

Displayed are the mean values per group at t2 and the delta between them.

Table 6.

Score Differences (Delta) Regarding Histology

	Group 1						Group 2
	t1		t2		t3
	DTC vs. MTC	DTC vs. ATC	DTC vs. MTC	DTC vs. ATC	DTC vs. MTC	DTC vs. ATC	DTC vs. MTC
Exhaustion (EX)	7	1	8	2	8	17	11
Discomfort in the head and neck (DI)	6	6	2	7	6	9	4
Voice (VO)	5	22	1	17	3	10	3
Hair problems (HA)	11	2	14	9	5	8	9
Swallowing (SW)	1	15	3	6	4	8	1
Treatment- and disease-related worry (TD)	3	2	5	3	0	13	10
Tingling or numbness (TI)	2	7	5	9	6	9	7
Worry about important others (WO)	1	5	6	10	4	3	24
Lacking social support (SO)	1	2	6	7	7	11	15
Dry mouth (DM)	5	8	7	6	14	10	13
Altered temperature tolerance (TO)	2	17	0	8	15	22	5
Body image (BI)	6	9	2	7	2	9	2
Shoulder functioning (SH)	1	2	5	1	14	7	3
Joint pain (JP)	7	3	2	0	7	14	21
Cramps (CR)	11	5	4	7	1	15	8
Impact on job or education (JE)	7	5	2	13	10	7	3
Rapid heartbeat (RH)	1	0	7	3	7	8	11

In Group 2 (survivors ≥2 years after diagnosis), no patients with anaplastic thyroid cancer were enrolled because these patients rarely survive for longer than 6 months.

ATC, anaplastic thyroid cancer; DTC, differentiated thyroid cancer; MTC, medullary thyroid cancer;

Patients with total thyroidectomy indicated more treatment- and disease-related worry than those with less than total thyroidectomy. For example, at t1, the respective average scores were 35 versus 27 in Group 1; at t2, it was 33 versus 20.

Responsiveness

In all but one of the pre-specified scales (Dry Mouth), changes were ≥|4| points between at least two time points (Table 7), indicating the instrument's ability to detect changes over time.

Table 7.

Changes in Scores for Different Treatment Groups Between Time Points

	Patients with surgery between t1 and t2		Patients with RAI		Patients with TKI
	(n = 141)		(n = 168)		(n = 32)
	t1 to t2	t2 to t3	t1 to t2	t2 to t3	t1 to t2	t2 to t3
Exhaustion (EX)	3.1	3.2	−4.0	2.2	−3.1	14.3
Discomfort in the head and neck (DI)	2.4	−4.5	−3.5	−4.2	0.4	0.0
Voice (VO)	9.5	−6.1	−4.1	−3.1	−6.8	6.2
Hair problems (HA)	5.3	6.8	5.4	−2.5	2.1	5.4
Swallowing (SW)	3.5	0.7	−2.5	−0.1	−0.5	7.7
Treatment- and disease-related worry (TD)	−1.6	0.4	1.0	0.7	−1.9	6.2
Tingling or numbness (TI)	4.5	0.3	0.7	−2.1	−2.6	11.9
Worry about important others (WO)	−5.4	−2.5	−3.5	0.4	−1.8	8.9
Lacking social support (SO)	−1.3	3.3	0.1	0.5	−4.2	1.0
Dry mouth (DM)	0.7	0.9	−0.2	−0.7	3.1	4.8
Altered temperature tolerance (TO)	3.8	3.0	−2.8	1.1	−1.0	10.7
Body image (BI)	7.6	4.2	3.4	0.0	7.5	6.2
Shoulder functioning (SH)	4.3	4.7	1.0	0.0	0.0	10.7
Joint pain (JP)	4.3	2.9	3.2	−2.6	1.0	4.8
Cramps (CR)	1.4	4.5	−3.6	2.0	−3.1	6.2
Impact on job or education (JE)	−0.5	4.2	0.6	−1.7	−6.7	9.9
Rapid heartbeat (RH)	−1.9	3.2	−3.4	0.6	−2.1	9.5

RAI, radioactive iodine.

Time and help needed to complete the questionnaire

The time required to complete the questionnaire (core questionnaire and module together) was <10 minutes in 42% of all participants; 34% needed 11–15 minutes, 14% 16–20 minutes, 6% up to 30 minutes, and 1% more than 30 minutes. For the remaining 3% (n = 12), the time needed was not documented. The majority of the patients (87%) completed the questionnaire with paper and pencil, 13% electronically. Those who used paper and pencil were somewhat quicker in completing the questionnaire, with 45% versus 26% needing <10 minutes (p = 0.07).

Three-quarters (74%) of patients required no help to complete the questionnaire, 17% needed help with understanding the questions, and 7% needed other help (for example, reading the question out loud because they had forgotten their glasses). For the remaining 2%, no information was available about any help needed. The proportion of patients who needed help with understanding differed by histology group (32% in those with anaplastic cancer, 20% medullary, 16% differentiated). It was unrelated to the cognitive functioning of the participants, as measured with the EORTC QLQ-C30.

Some patients spontaneously commented on the questionnaire, for example, “easy questionnaire to answer,” or “the questions are clear.”

Discussion

The incidence of thyroid cancer has increased over the past years, mainly driven by papillary thyroid cancer,²⁰ while treatment varies considerably according to histology and stage. To capture patient-reported outcomes in clinical trials, reliable and valid measures are necessary. Such instruments should cover all relevant QoL issues but be as short as possible, and, ideally, be available in many languages.

The EORTC QLG together with the EORTC Head and Neck Cancer Group and the EORTC Endocrine Tumours Group decided to develop such a tool. This instrument was now field tested in the last phase of its development.

Results suggested retaining all the items that were developed in phases I through III.^13,14 The instrument is therefore more elaborate than the City of Hope QoL thyroid questionnaire,⁷ with 30 items. The latter was carefully developed with qualitative interviews but only in the United States, and all study participants had a college degree. Subsequently, Dagan et al. shortened that questionnaire to 15 items and named it TQOLI,²¹ but its response categories vary, making it more difficult for patients to complete it.

The Thyroid Module of the MD Anderson Inventory comprised of 25 items.²² It was developed in the United States, without the participation of other countries. Another questionnaire, the ThyPRO,²³ was designed for benign thyroid disease only; thyroid cancer patients were explicitly excluded during the development and validation.

The THYCA-QoL was developed to be used in conjunction with the EORTC QLQ-C30.⁶ It contains 24 items and is, therefore, somewhat shorter than our instrument. However, it was developed only in The Netherlands, making it more difficult to apply to other countries. Moreover, no patients with anaplastic thyroid cancer were included in its development.

The EORTC QLQ-THY34 scales exhibit very good characteristics. For all of them, the entire range was used and missing values were infrequent. The tests provide evidence for the reliability and validity of the instrument. However, it should be kept in mind that the internal consistency for the scale Tingling and Numbness is good only in certain groups of patients, for example, those with anaplastic thyroid cancer or those diagnosed with hypoparathyroidism. Researchers using the EORTC QLQ-THY34 are, therefore, advised to check whether internal consistency is sufficient in their sample, and, if not, then treat the items as independent.

Regarding responsiveness, it should be noted that some changes were not in the direction that we had expected. However, this is not evidence for poor psychometric properties of the EORTC QLQ-THY34 but rather indicates that the temporal evolution of QoL in patients with thyroid cancer is not yet fully understood. In hindsight, the results can be explained rationally from a clinical point of view.

For example, cramps related to treatment with TKI can occur several weeks or months after treatment initiation. Hence, the observed worsening at t3 as compared with t2 is plausible. In other words, these findings illustrate the need for an instrument such as the EORTC QLQ-THY34 to study changes in QoL over time in thyroid cancer patients in a systematic manner (which goes beyond the scope of the present study). The results of the test-retest reliability underline that the observed changes are not simply a matter of random fluctuation but that, most likely, real changes were captured. In conclusion, the EORTC QLQ-THY34 is responsive, that is, changes in thyroid cancer-specific QoL can be measured in future clinical studies using this instrument.

Regarding the target group of this module, we deliberately included patients with non-differentiated thyroid cancers. A disadvantage of this approach is that some items might not be relevant for all patients in a study; for example, questions related to RAI are not relevant for patients with medullary or anaplastic thyroid cancer. The advantage, however, is that the same instrument can be used for different studies and results can then be compared more easily.

This approach had also been used for the EORTC head and neck cancer module, which also includes various types of head and neck cancer, such as laryngeal, oropharyngeal, or salivary gland cancer.^24

–27 Conversely, for gynecological cancers, separate modules were developed for cervical,²⁸ ovarian,²⁹ endometrial,³⁰ and vulva³¹ cancer. Our decision to develop a single module was also influenced by the much lower prevalence of medullary and anaplastic thyroid cancer compared with differentiated thyroid cancer.

These differences in the disease occurrence also led to one of the limitations of our study, namely that the number of patients with certain histologies, for example, anaplastic thyroid cancer or Hurthle cell cancer, is relatively small, making the results in these groups of patients more uncertain.

Another limitation of our study is that patients of some countries with high incidence rates of thyroid cancer, such as Korea, Israel, Turkey, France, or China (https://gco.iarc.fr/), were not included. This was due to the fact the researchers from these countries who initially expressed their interest to participate were eventually unable to enroll patients.

However, there were several colleagues from China who approached us and asked to use the EORTC QLQ-THY34 for their own studies after phase III was completed. As the questionnaire is already translated into Mandarin and Cantonese, this was possible. Hence, it is likely that validation studies for these languages will be published in due course.

A challenge of our study had been the COVID-19 pandemic, which made it more difficult to approach patients and include them in the study. The accrual time period was lengthened due to this reason. However, it did not lead to a biased selection of patients.

In conclusion, the EORTC QoL module for thyroid cancer consisting of 34 items is reliable and able to differentiate between clinically relevant groups. It also captures changes over time, and its item and scale characteristics are very good. There are no items that patients or survivors found irrelevant.

The instrument is ready for use in clinical research and available in the following languages: Arabic (Egypt), Arabic (Israel), Arabic (Jordan), Bengali, Bulgarian, Chinese Cantonese (Hong Kong), Chinese Mandarin ( = simplified Chinese), Croatian, Danish, Dutch, English, Finnish, French (Canada), French (Europe), German, Georgian, Greek, Gujarati, Hebrew, Hindi, Hungarian, Italian, Japanese, Kannada, Korean, Malayalam, Marathi, Norwegian, Polish, Portuguese (Portugal), Portuguese (Brazil), Punjabi, Romanian Serbian, Russian, Spanish (Argentina), Spanish (Chile), Spanish (Mexico), Spanish (Spain), Spanish (United States), Swedish, Tamil, Telugu, Thai, Turkish, Ukrainian, and Urgu (India).

The module together with the scoring instructions can be obtained from the EORTC (https://qol.eortc.org/form/#1).

Data Availability Statement

The data of this study are stored in the EORTC data repository and can be accessed by other researchers.

Footnotes

Acknowledgments

The authors would like to acknowledge the work of colleagues who helped with data collection: Gisele da Rocha Santa, Micha Pilz, Hisayo Doi, and Emily Dickson; for setting up this study in local institutions: Cláudia Áraújo.

Authors' Contributions

S.S., G.S., A.A.-I., M.P., I.I., A.A.Ø., E.H., L.D.L., E.G., J.Ig., S.J.J., N.K., G.I., O.H., L.M., J.In., G.A., and H.R. contributed to the study conception and design. D.K. supervised the translation procedures for all languages. Data collection was performed by G.S., A.A.-I., M.P., I.I., A.A.Ø., E.H., L.D.L., E.G., J.Ig., S.J.J., N.K., M.B., R.C., G.I., O.H., R.R.G., G.F., L.M., J.In., G.A., and D.F. Data entry, plausibility checks, and data cleaning were done by D.E. Data analysis was performed by S.S. following a statistical analysis plan that had been drafted by S.S. und discussed with G.S., A.A.-I., M.P., I.I., A.A.Ø., E.H., L.D.L., E.G., N.K., M.B., G.I., O.H., R.R.G., G.F., and J.In. The first draft of the article was written by S.S., and all authors commented on it and on subsequent versions of the article. All authors read and approved the final article.

Author Disclosure Statement

S.S. has received honoraria from Lilly for reviewing papers for their Quality of Life Award and from Eisei for advice in writing a paper, outside of the submitted work. N.K. reports honoraria from ONO PHARMACEUTICAL, Bristol Meyers Squibb, Merck Biopharma, Astra-Zeneca, Merck Sharp & Dohme, Eisai, Bayer, and Chugai Pharmaceutical, all outside the submitted work. M.P. reports personal fees from Meeting&Words and from Hinovia S.r.l., outside the submitted work. G.S. has received research support in the form of donations from Merck Sharp & Dohme, IBSA, and Alpha-Sigma, which partially defrayed costs associated with recruiting patients in this study. M.B. has received speaker fees from Lilly and Takeda outside of the submitted work. D.F. reports speaker fees and honoraria for advisory boards of Roche, Lilly, Ipsen, Eisai, Sanofi, and Bayer all outside the submitted work.

All authors have a special interest in the quality of life in thyroid cancer patients and survivors. All other authors declare that they have no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.

Funding Information

This study was funded by the European Organisation for Research and Treatment of Cancer, Quality of Life Group (No. 002/2017). The EORTC Quality of Life Group business model involves charges for commercial companies using EORTC instruments. Academic use of EORTC instruments is free of charge.

Supplementary Material

Supplementary Data

Supplementary Tables S1

Supplementary Tables S2

Supplementary Tables S3

Supplementary Tables S4

Supplementary Tables S5

References

Aaronson

, Ahmedzai

, Bergmann

, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst, 1993; 85(5):365–376.

Cella

, Tulsky

, Gray

, et al. The Functional Assessment of Cancer Therapy scale: Development and validation of the general measure. J Clin Oncol, 1993; 11(3):570–579.

Velikova

, Coens

, Efficace

, et al. Health-Related Quality of Life in EORTC clinical trials—30 years of progress from methodological developments to making a real impact on oncology practice. Eur J Cancer Suppl, 2012; 10(1):141–149.

Giesinger

, Efficace

, Aaronson

, et al. Past and current practice of patient-reported outcome measurement in randomized cancer clinical trials: A systematic review. Value Health, 2020; 24(4):585–591; doi: 10.1016/j.jval.2020.11.004

Roth

, Lubitz

, Swan

, et al. Patient-reported quality-of-life outcome measures in the thyroid cancer population. Thyroid, 2020; 30(10):1414–1431; doi: 10.1089/thy.2020.0038

Husson

, Haak

, Mols

, et al. Development of a disease-specific health-related quality of life questionnaire (THYCA-QoL) for thyroid cancer survivors. Acta Oncol, 2013; 52(2):447–454.

Dow

, Ferrell

, Anello

. Quality-of-life changes in patients with thyroid cancer after withdrawal of thyroid hormone therapy. Thyroid, 1997; 7(4):613–619.

Aschebrook-Kilfoy

, James

, Nagar

, et al. Risk factors for decreased quality of life in thyroid cancer survivors: Initial findings from the North American Thyroid Cancer Survivorship Study. Thyroid, 2015; 25(12):1313–1321.

Walshaw

, Smith

, Kim

, et al. Systematic review of health-related quality of life following thyroid cancer. Tumori J, 2021; 108(4):291–314; doi: 10.1177/03008916211025098

10.

Altuntas

, Hocaoglu

. Effects of chronic suppression or oversuppression of thyroid-stimulating hormone on psychological symptoms and sleep quality in patients with differentiated thyroid cancer. Horm Metab Res, 2021; 53(10):683–691; doi: 10.1055/a-1639-1024

11.

Massolt

, van der Windt

, Korevaar

TIM

, et al. Thyroid hormone and its metabolites in relation to quality of life in patients treated for differentiated thyroid cancer. Clin Endocrinol (Oxf), 2016; 85(5):781–788.

12.

Husson

, Haak

, Oranje

, et al. Health-related quality of life among thyroid cancer survivors: A systematic review. Clin Endocrinol (Oxf), 2011; 75(4):544–554.

13.

Singer

, Husson

, Tomaszewska

, et al. Quality of life priorities in patients with thyroid cancer: Results from a multi-national EORTC phase I study. Thyroid, 2016; 26(11):1605–1613.

14.

Singer

, Jordan

, Locati

, et al. The EORTC module for quality of life in patients with thyroid cancer: Phase III. Endocr Relat Cancer, 2017; 24(4):197–207.

15.

Wheelwright

, Bjordal

, Bottomley

, et al. EORTC Quality of LIfe Group Guidelines for Developing Questionnaire Modules, 5th Edition. European Organisation for Research and Treatment of Cancer: Brussels; 2021.

16.

Erharter

, Giesinger

, Kemmler

, et al. Implementation of computer-based quality-of-life monitoring in brain tumor outpatients in routine clinical practice. J Pain Symptom Manage, 2010; 39(2):219–229.

17.

Reeve

, Wyrwich

, Wu

, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res, 2013; 22(8):1889–1905.

18.

Ferguson

, Cox

. Exploratory factor analysis: A user's guide. Int J Sel Assess, 1993; 1(2):84–94.

19.

, Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling, 1999; 6:1–55.

20.

Miranda

, Lortet-Tieulent

, Bray

, et al. Thyroid cancer incidence trends by histology in 25 countries: A population-based study. Lancet Diabetes Endocrinol, 2021; 9(4):225–234; doi: 10.1016/s2213-8587(21)00027-9

21.

Dagan

, Bedrin

, Horowitz

, et al. Quality of life of well-differentiated thyroid carcinoma patients. J Laryngol Otol, 2004; 118(7):537–542.

22.

Gning

, Trask

, Mendoza

, et al. Development and initial validation of the thyroid cancer module of the M. D. Anderson Symptom Inventory. Oncology, 2009; 76(1):59–68.

23.

Watt

, Hegedus

, Groenvold

, et al. Validity and reliability of the novel thyroid-specific quality of life questionnaire, ThyPRO. Eur J Endocrinol, 2010; 162(1):161–167.

24.

Singer

, Araújo

, Arraras

, et al. Measuring quality of life in head and neck cancer patients: Update of the EORTC QLQ-H&N module, phase III. Head Neck, 2015; 37:1358–1367.

25.

Singer

, Amdal

, Hammerlid

, et al. International validation of the revised European Organisation for Research and Treatment of Cancer Head and Neck Cancer Module, the EORTC QLQ-HN43: Phase IV. Head Neck, 2019; 41(6):1725–1737.

26.

Bjordal

, Ahlner-Elmqvist

, Tollesson

, et al. Development of an European Organization for Research and Treatment of Cancer (EORTC) questionnaire module to be used in quality of life assessments in head and neck cancer patients. EORTC Quality of Life Study Group. Acta Oncol, 1994; 33(8):879–885.

27.

Bjordal

, de Graeff

. A 12 country field study of the EORTC QLQ-C30 (version 3.0) and the head and neck cancer specific module (EORTC QLQ-H&N35) in head and neck patients. Eur J Cancer, 2000; 36:1796–1807.

28.

Greimel

, Kuljanic

, Waldenstrom

, et al. The EORTC Quality of Life cervical cancer module (EORTC QLQ-CX24). Cancer, 2006; 107:1812–1822.

29.

Cull

, Howat

, Greimel

, et al. Development of a European Organization for Research and Treatment of Cancer questionnaire module to assess the quality of life of ovarian cancer patients in clinical trials: A progress report. Eur J Cancer, 2001; 37(1):47–53.

30.

Greimel

, Nordin

, Lanceley

, et al. Psychometric validation of the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire–Endometrial Cancer Module (EORTC QLQ-EN24). Eur J Cancer, 2011; 47:183–190.

31.

Jensen

, Froeding

, Greimel

, et al. Development and pre-testing of the EORTC Quality of Life Questionnaire for vulva cancer patients: The significance of patients involvement. Int J Gynecol Cancer, 2017; 27:1770–1770.