Assessing Health-Related Quality of Life Among Survivors of Testicular Cancer: A Systematic Review

Design

A systematic review was conducted to assess the methodological quality of studies carried out with the sole purpose of producing a survivors of TC HRQoL tool (Phase 1). Each identified study was assessed with the Appraisal Tool for Cross-Sectional Studies (AXIS) (Downes et al., 2016). This information was used to inform the development and execution of the subsequent Phase 2 critique of the existing HRQoL tools’ psychometric properties discovered and verified in the systematic search process. This Phase 1 reporting was conducted based on the PRISMA guidelines (Moher et al., 2009). This study was performed with approval was granted by the Ethics Committee of (information blinded for peer review purposes) .

Inclusion/Exclusion Criteria and Screening Procedure

Studies had to meet the following inclusion criteria to undergo AXIS assessment. First, there had to be a discussion of HRQoL assessment among a cohort of study participants within the study. Second, the study needed to have a discussion of the methods used to create a HRQoL tool (i.e., providing details on the development and implementation of such a tool). Lastly, the HRQoL tool had to be specifically designed for survivors of TC. The articles were screened in the following manner:

Data Sources and Search Strategy

The search strategy involved using the Ovid Medline (1950 to present), the Cumulative Index to Nursing and Allied Health Literature (1982 to present), PsycINFO (1806 to present), All EBM Reviews (2020), Ovid Healthstar (1966 to present), the Education Resources Information Center (1966 to present), and Google Scholar (2020) databases to locate relevant literature. Further, the review used ancestry (citation tracking) and grey literature (e.g., unpublished reports, PhD theses) searches to ensure full capture of relevant research. Keywords used for the search included several variants of TC, QoL, HRQoL, assessment, survey, scale, questionnaire, and survival.

Methodological Quality Assessment

Five raters independently appraised the methodological integrity of the qualified cross-sectional studies with the AXIS instrument (Downes et al., 2016). AXIS specifically appraises the quality of study design, sample size and characteristics, measures, internal consistency, results, analysis, and limitations. The AXIS checklist is comprised of 20 questions in three primary categories: quality of reporting (7 questions), study design quality (7 questions), and possible introduction of biases (6 questions). AXIS does not provide an established rule for determining the quality of each study. The tool instead provides a means for a rater to critique each portion of a study to make a judgment on overall quality, which, according to Downes et al. (2016), may provide for an element of subjective assessment. For the present review, five raters (all professionals in the health sciences field) individually scored each identified cross-sectional study. The 20 AXIS questions were each originally scored by assigning numeric values to two categorical responses: Yes (scored as 1) and No (scored as 0).

After all raters completed their scoring, a series of peer conferences were held to help establish consensus on each of the 20 items if there were any discrepancies. Consensus was defined as 80% of the raters reaching an agreement (i.e., 4 out of 5 raters). If there was no consensus on an item, the item was scored as Partially Fulfilled (scored as 0.5). After each item had a determination of consensus on its score, the values were then summed for a total appraisal score value to determine the level of methodological quality of the study. Fleiss’ kappa was calculated for assessing the consistency of observer agreement for each of the identified scales.

As observed in previous critical appraisals using AXIS, a percent value was predetermined at the beginning of this study to identify publications as high or low quality (Arab Alkabeya et al., 2019; Boxberger and Reimers, 2019; Henderson et al., 2019). For this study’s critical appraisal, the authors predetermined the value of a high-quality publication as total appraisal scores equal to or exceeding 70% of the total (i.e., at least 14 out of 20 questions scored as 1, or a score ≥14 to compensate for some items where there was no consensus scored at 0.5). If the publication score was between 60% and 69.9%, it was considered fair quality, and below 60% was considered low quality.

Tools Identified

Database searches identified 786 unique publications that passed primary screening (Figure 1). Articles were first excluded if the studies did not use specific HRQoL tools for survivors of TC (n = 743). For the remaining 43 articles, the full text was then reviewed, followed by exclusion of duplicates or different phases of the same HRQoL tool specific to survivors of TC (n = 41). Overall, two studies met all inclusionary and exclusory criteria out of the 786 total publications and were included in this critical appraisal review to be further examined for methodological rigor: (1) the European Organization for Research and Treatment of Cancer’s Quality of Life Questionnaire for Testicular Cancer, 26 items (EORTC QLQ-TC 26) and (2) the Cancer Assessment for Young Adults for Men with Testicular Cancer (CAYA-T).

OPEN IN VIEWER

Figure 1.

PRISMA flow chart.

Holzner et al.’s (2013) study produced the EORTC QLQ-TC26 via in-depth literature reviews and qualitative interviews with 28 experts in the field (ranging from urologists to psychologists to nurses from Austria, the Netherlands, Italy, Canada, and England) and 62 survivors of TC (from Austria, Canada, and the Netherlands). This tool is a supplemental module questionnaire that is TC-specific and must be used alongside the core questionnaire, the EORTC QLQ-C30. This self-administered tool was published in 2012 and comprises 26 items grouped into seven multi-domain scales (treatment side effects, treatment satisfaction, future perspective, communication, sexual activity, functioning, and enjoyment) and six single items (job and education problems, physical limitations, family problems, infertility, body image problems, testicular transplant satisfaction).

Hoyt et al.’s (2013) study produced the CAYA-T via in-depth literature reviews and qualitative interviews with 21 patients aged 18–29 recruited from the California Cancer Care Registry. The prototype tool was then pilot tested on 15 demographically similar survivors of TC. This tool assesses seven biopsychosocial domains specific to survivors of TC. This questionnaire was published in 2013, it is self-administered, and compromises of 90 items grouped into 17 HRQoL domains: physical, sexual confidence, sexual functioning, body image strength, positive masculine self-image, positive adult self-image, cognitive–emotional regulation, disclosure ability, relationship maintenance, social connectedness, health-care confidence, goal navigation, goal facility, financial maintenance, recreational pursuit, spiritual stability, and finding meaning.

Critical Appraisal

The critical appraisal of the EORTC QLQ-TC 26 included the following scores: a total appraisal score of 10.5/20 (52.5%), which indicates a low-quality study, a 95% inter-rater agreement score, and a Fleiss kappa statistic of 0.779, which suggests substantial agreement among raters (Sim & Wright, 2005) (see Table 1). There was a discrepancy among the raters on whether the data were adequately described or not by the authors.

OPEN IN VIEWER

Table 1.

Study Quality Scores Obtained for the Appraisal Tool for Cross-Sectional Studies (AXIS).

Items		Consensus scores *
		Holzner et al. (EORTC QLQTC-26)	Hoyt et al. (CAYA-T)
1	Were the aims/objectives of the study clear?	1	1
2	Was the study design appropriate for the stated aim(s)?	1	1
3	Was the sample size justified?	0	0.5
4	Was the target/reference population clearly defined? (Is it clear who the research was about?)	1	1
5	Was the sample frame taken from an appropriate population base so that it closely represented the target/reference population under investigation?	0	0.5
6	Was the selection process likely to select subjects/participants that were representative of the target/reference population under investigation?	0	0
7	Were measures undertaken to address and categorize non-responders?	0	0.5
8	Were the risk factor and outcome variables measured appropriate to the aims of the study?	1	1
9	Were the risk factor and outcome variables measured correctly using instruments/measurements that had been trialed, piloted or published previously?	1	1
10	Is it clear what was used to determine statistical significance and/or precision estimates? (e.g., p-values, confidence intervals)	0	1
11	Were the methods (including statistical methods) sufficiently described to enable them to be repeated?	0	1
12	Were the basic data adequately described?	0.5	0
13	Does the response rate raise concerns about non-response bias?	0	0
14	If appropriate, was information about non-responders described?	0	0.5
15	Were the results internally consistent?	1	1
16	Were the results presented for all the analyses described in the methods?	1	1
17	Were the authors’ discussions and conclusions justified by the results?	1	1
18	Were the limitations of the study discussed?	1	0
19	Were there any funding sources or conflicts of interest that may affect the authors’ interpretation of the results?	0	0
20	Was ethical approval or consent of participants attained?	1	1
	Total appraisal score	10.5/20(52.5%)	13/20(65%)
	Study quality	Low	Fair
	Inter-rater agreement	19/20(95%)	16/20(80%)
	Fleiss’ kappa	0.779	0.599

Note. *Score meaning: 1 = Yes; 0 = No; 0.5 = Partial.

The critical appraisal of the CAYA-T resulted in the following comprehensive scores: a total appraisal score of 13/20 (65%), which indicates a fair-quality study, an 80% inter-rater agreement score, and a Fleiss kappa statistic of 0.599, which suggests moderate agreement among raters (Sim & Wright, 2005). There was a discrepancy on the discussion primarily concerning the sampling methodologies employed by the authors, including the representativeness of the study population and possible non-response biases.

Key Findings

Improved treatment and prognosis of TC is increasing not only the number of survivors of the disease, but also the length of their lives. The literature outlines unique challenges survivors of TC have in improving and maintaining HRQoL outcomes post-treatment. There are some concerns that existing tools may not to capture the most comprehensive HRQoL information possible among the population. It was, therefore, imperative to initiate a formal two-phased process to determine the methodological rigor of the original studies that produced survivors of TC HRQoL tools. Phase 1’s critical appraisal of the seminal study from which these TC-specific HRQoL tools stemmed provided an opportunity for us to begin the process of determining if the tools are built upon the actuality and involvedness of the survivor experience and the overall methodological quality of existing tools. This information provides a necessary foundation to conduct a critique of each tool’s applicability in the field.

This research identified two survivors of TC HRQoL tools (the EORTC QLQ-TC26 and the CAYA-T) and provided a critical appraisal to determine the soundness of their seminal cross-sectional study design. Phase 1 findings suggest that there are significant methodological concerns for each reviewed seminal study. The Hoyt et al. (2013) (CAYA-T) and the Holzner et al. (2013) (EORTC QLQ TC-26) appraisals indicate a failure to: (1) recruit a diverse sample that represents a spectrum of perceptions and attitudes to inform the development of the scale, (2) provide an explanation for the possible presence of non-response and ascertainment biases, nor provide ways these limitations were mitigated, and/or (3) report on the qualitative nature of how interviews/focus groups were conducted (i.e., there was a lack of any comprehensive discussion on the qualitative methods used to establish a thematic structure). Each study had unique limitations in each of the AXIS tool’s categories: quality of reporting, study design quality, and possible introduction of biases. Hoyt et al.’s (CAYA-T) study was the more rigorous of both tools as it scored higher in each of the three categories, but it did have its fair share of limitations.

For Category 1: Quality of reporting, both the Hoyt et al. (2013) and Holzner et al. (2013) studies discussed the aims of their research, clearly stated the target population, and reported on the results that were outlined in their methods. There was some variation related to the reporting of significance testing, data, and limitations. Unlike Hoyt et al.’s (2013) study, there was a lack of in-depth discussion in Holzner et al.’s (2013) study on the qualitative methods used to establish a thematic structure from the semistructured interviews conducted with patients. This is problematic as it serves as a very important part of the scale-developmental process. Proper, meaningful qualitative data collection, reporting, and analysis serve as fundamental building blocks for any assessment tool. We support Nassar-McMillan and Borders (Nassar-McMillan & Borders, 2002) assertion that qualitative methods in survey design “provide an efficient means for the purposes of both item generation and refinement.” (p. 2) Not providing that information in any significant detail is detrimental to fully understanding the conceptual framework of the tool in question, which casts doubt on its applicability in the field.

For Category 2: Study design quality, the primary concerns stemmed from each study’s sample demographics. Holzner et al. (2013) and Hoyt et al.’s (2013) studies both recruited non-diverse demographic samples, which affect the generalizability of their produced tools. Holzner et al.’s (2013) sample (n = 62 in Phase 1 and n = 156 in Phase 3) consisted of what appeared to be upper-middle class, nearing-middle aged males from Australia, Austria, Canada, Italy, the Netherlands, and Spain. There was no mention of racial/ethnic diversity, which lets us assume there was a lack of it within the sample. The authors, interestingly, made an emphasis to mention which language each country primarily spoke. Language diversity does not guarantee a diversified sample based on race/ethnicity. In fairness, we acknowledge this was designed via the European Organization for Research and Treatment of Cancer, which would help explain the use of their limited selection pool.

Hoyt et al. (2013) was more thorough in ensuring a representative sample (n = 171) than Holzner et al. (2013). Although the sample was recruited from the California Cancer Registry, which may lead to generalizability concerns, the authors made mention of race/ethnicity, sexual orientation, and more in-depth socioeconomic indicators as part of their recruitment methodology. Why is this important if TC manifests itself most of the time among White males? Minority males tend to be disproportionately affected by late-stage discovery and suffer from worse HRQoL outcomes as compared to their White counterparts (Ghazarian et al., 2017; Li et al., 2020). For example, Black males have nearly twice the mortality rate and Latinos are projected to have the highest incidence rates of TC among all race/ethnicities by the year 2026 (Ghazarian et al., 2017; Li et al., 2020). These data indicate a significant shift in the burden associated with TC now resides with minority males. The data on, and perspectives of, these men, however, are largely absent from the current peer-reviewed survivors of TC literature (Oldenburg et al., 2007), particularly pertaining to HRQoL. Hoyt et al. (2013), for example, had two Black/African Americans enrolled in their study, representing 1.2% of the total study population. So, although selecting from a demographically varied cancer registry has the best chances of soliciting a diverse sample, this sometimes fails to result in a true demographic representation.

The literature calls for research to expand into minority populations to understand this shift in TC incidence, prevalence, and mortality, yet such requests remain mostly unanswered (Klaassen et al., 2018). Bukavina et al.’s (2017) work, for example, is a clear call to action for HRQoL research to expand further into minority populations. As it pertains to survivors of TC HRQoL, we are failing to understand the importance of promoting more diversity in our samples, let alone achieving even the most modest of inclusion standards. The field must put forth a better effort at recruiting more inclusive samples.

Sztankay et al. (2018) attempted to expand the sampling pool in their attempt to validate the EORTC QLQ TC-26 but fell short. The authors followed what appeared to be Holzner et al.’s (2013) sampling methodology but swapped representation from Australia and Canada for Serbia and Poland. Hoyt et al., on the other hand, had a more representative sample but it came only from California, thus also questioning its ability to generalize to other populations, also at the global scale. For survivors of TC HRQoL tools to be truly generalizable, we need to broaden our sampling pools, especially into traditionally underserved communities, both globally and in the United States. These would include populations in which TC, historically, was not a concern but is now a rising point of interest due to emerging disparities in mortality and late-stage morbidity outcomes.

For Category 3: Introduction of biases, much of the concern for both the Hoyt et al. (2013) and the Holzner et al. (2013) studies stemmed from the presence of non-response and ascertainment biases, which relates to Category 2’s sampling methodology weaknesses. However, what is unclear in both studies is not necessarily the presence of non-response bias, but the influence. There is a lack of discussion about non-responders and how that may have affected the results. Both studies do, however, provide information on the inclusionary/exclusionary processes that narrowed the sample size down to its analyzed level. Hoyt et al. provided more information on this phenomenon than Holzner et al. (2013), thus helping legitimize their sample. Finally, there are reports of internal consistency on the part of Hoyt et al. (2013). Holzner et al. (2013) offers up a bit less but both studies allude to their scales being reliable and valid. Phase 2 of this two-phased critique of both scales will address this issue more in-depth.

Comment on HRQoL Assessment

The quality of general HRQoL tools has been debated for the past few decades. Gill and Feinstein (1994) conducted a critical appraisal of HRQoL and suggested that many, if not most, tools assess patient health status rather than actually measuring HRQoL. Guyatt (1997) indicated that most general measures of HRQoL (in this case, the SF-36) “focuses to a large extent on how patients are functioning” (p. 720) instead of HRQoL. These works indicate that HRQoL is more than physical factors of health and wellness, comprised of mental, functional, and emotional well-being. We cannot agree more.

This idea is supported in the TC-specific HRQoL literature by the works of Smith et al. (2016) and Alexis et al. (2020). Those specific studies offer a perspective of multidimensionality related to TC-related HRQoL instead of the traditional unidimensional approach to measuring QoL among survivors of TC, such as Luckett et al.’s (2008) study that called for continued use of the SF-36. Primarily assessing physical health status with minimal effort to measure any other dimension (i.e., mental, emotional, spiritual, environmental, financial) would be shortsighted.

One other concern is the incorporation, or lack thereof, of TC survivor voices in the development of TC-specific HRQoL tools. Gill and Feinstein (1994) asserted that HRQoL can only be appropriately assessed via incorporating the opinions of actual patients and not rely so much upon the word of, as they put it, “experts.” It appears that Holzner et al.’s (2013) EORTC QLQ TC-26 counters this notion almost completely. According to Holzner et al. (2013), “. . .comments by patients were very rare and therefore did not have a substantial impact on item selection and item wording.” (p. 373) Further, the authors state that “Overall, patients made only a low number of comments indicating good acceptance and understandability of the items.” (p. 373) This is a stretch. One cannot assume just because a participant does not comment on a tool or an individual item that it implies comprehension. This appears counterintuitive to a patient-centered approach necessary for measuring HRQoL.

Hoyt et al. (2013), on the other hand, reported a fairly inclusive method to incorporate the patient voice into the development of the CAYA-T. There was no indication of the role health and wellness professionals played in the creation and/or validation of the tool’s thematic structure. A more responsible approach to creating such a tool would be to strike a balance between the patients’ and providers’ voices in the tool’s developmental phases. We believe that the incorporation of more representative voices would increase the scope of the tool to include topic otherwise unknown if not for the expansion. For example, there are some important topics not covered by either tool, including fear of recurrence and body image concerns beyond the lack of a testicle, such as overweight/obesity, which many survivors of TC experience post-treatment.

Strengths and Limitations

This critical appraisal is not without its own limitations. For example, the use of AXIS is a limited measuring approach, especially when it refers to an unstandardized interpretation of the final score. AXIS does not provide an established rule of determining the quality of each study. The tool instead provides a means for a rater to critique each portion of a study to make a judgment on overall quality, which, according to Downes et al. (2016) may provide for an element of subjective assessment. However, our numeric rating system of percentage of agreement helped provide a more objective approach to assessing overall quality.

It was, in our collective opinion, essential to employ a critical appraisal analysis via the AXIS tool for this review, as Downes et al. (2016) suggests, due to its focus on an integration of the “best external evidence with clinical care” (p. 1). They further note that “when reading any type of evidence, being critical of all aspects of the study design, execution, and reporting is vital for assessing its quality before being applied to practice.” (p. 1) It bears repeating that tools/instruments/methods employed in a clinical setting, used to measure clinical outcomes, and/or inform clinical care decisions, demand rigorous testing to ensure quality data. This lends rationale for why we opted to employ our two-phased process in this review. Critiquing the design of a study that produced an assessment tool is just as vital to determining the overall quality of said tool by testing for its psychometric properties. Considering the information collected from these scales have clinical implications, we found the use of AXIS to be appropriate for this review.

The scope of this review, admittedly, is limited. However, this is due to the dearth of survivors of TC research in the field. There are some notable exceptions (e.g., Saab et al., 2014). There is an ever scarcer amount of literature on survivors of TC HRQoL scale development and use. This, unfortunately, is one of the inherent challenges of scale development, particularly with a relatively rarer disease in a select population.

Our consensus scoring technique method could have possibly introduced bias into the study. For example, we decided to conference rather than use raw ratings in the calculation of agreement, which could possibly inflate the % agreement. However, we had a 3-point scoring system to account for non-consensus instead of a compete forcing of compliance or noncompliance.

Implications for Cancer Survivors

The development of a multidimensional tool is needed to focus on these areas of concern to provide researchers, health-care professionals, and patients the highest and most relevant information regarding survivors of TC HRQoL. Having access to specific HRQoL information for this population will help us understand how to properly approach and address HRQoL setbacks for these individual’s post-treatment. Thus, allowing medical personnel and others the ability to ultimately increase the HRQoL for survivors of TC.

The limitations surrounding the ability to confidentially measure HRQoL specific to TC presents researchers and practitioners the opportunity to develop a true gold standard tool to accurately measure the HRQoL among survivors of TC. A HRQoL assessment created specifically for survivors of TC that has been thoroughly tested for methodological rigor (both the tool and its seminal study) can assist researchers and practitioners in understanding the needs and wants of this population from diagnosis through, potentially, the rest of their lives. The creation of such a tool should (1) lead to the understanding of both health function and health status and (2) capture the multidimensional factors of HRQoL among survivors of TC. This, in turn, will allow researchers and practitioners to collect the highest quality of data and provide the opportunity to improve the overall HRQoL of survivors of TC.