Distress Screening in Adolescents and Young Adults with Cancer: Development of Cut-Points for the Cancer Distress Scales-Adolescent and Young Adults

Abstract

Purpose:

Distress is an important issue facing adolescent and young adults (AYA) with cancer due to their stage of development. Metrics are necessary to help improve psychosocial outcomes in this population. This study determined cut-points for the newly developed Cancer Distress Scales (CDS)-AYA.

Methods:

The CDS-AYA is a new patient-reported outcomes measure that comprises five independently functioning scales, including the following: Impact of Cancer (12 items), Physical (12 items), Emotional (11 items), Cognitive (8 items), and Cancer Worry (5 items). Canadian AYA with cancer 15–39 years of age completed the CDS-AYA and the Hospital Anxiety and Depression Scale (HADS), as part of the CDS-AYA field test. Only patients who had completed responses to the CDS-AYA and HADS were included in these analyses. Receiver operating characteristic (ROC) curve analysis was used to generate cut-points for five CDS-AYA scales based on distress defined by the HADS anxiety and HADS depression scale.

Results:

In total 453 of 515 respondents had complete data for the CDS-AYA and HADS were included in analyses. Area under the curve (AUC) in the ROC analyses ranged from 0.75 to 0.85. The CDS-AYA Emotional scale had the greatest AUC. The cutoff value for the Emotional scale was 27 based on the HADS anxiety scores (78.3% and 78.9%).

Conclusions:

The five CDS-AYA scales had fair to good accuracy when classifying the none/low and moderate/severe distress categories based on HADS anxiety and depression scales. For screening purposes, it is recommended that the Emotional scale or Impact of Cancer scale be utilized.

Introduction

The National Comprehensive Cancer Network (NCCN) has defined distress as an unpleasant experience of psychological, social, and/or spiritual nature.¹ The experience of distress can range from mild feelings of fear and sadness to more severe conditions such as depression and anxiety.¹ For adolescent and young adults (AYA), a diagnosis of cancer comes at a time when they are reaching developmental milestones, including developing self-identity and values, obtaining autonomy from parents, preparing for a career, exploring intimacy and relationships, and developing strong peer relationships.^2–4 The physical changes, social isolation, loss of independence, and side effects of treatment that often occur with a cancer diagnosis can be particularly distressing for this age group.³ AYA-aged survivors may also experience distress as they regain independence, return to normalcy, and are faced with the late effects (e.g., infertility) of treatment.⁵

Clinical screening for distress is recommended throughout the cancer spectrum, from the time of diagnosis through to survivorship.^6–9 Owing to the unique aspects of distress experienced by AYA with cancers, patient-reported outcome measures used in screening should be tailored to this patient population.

The NCCN distress thermometer (DT)¹ and Edmonton Symptom Assessment Scale (ESAS)¹⁰ are commonly used screening tools for distress in cancer care. Some work has been done assessing the validity of the NCCN DT in the AYA population. Recklitis et al.¹¹ compared the widely used NCCN DT with a psychiatric diagnostic interview in a young adult cancer survivor population. This study concluded that the NCCN DT should not be used as a stand-alone screening tool in young adult survivors of cancer because it did not meet the criteria for acceptable sensitivity or specificity, failing to identify 31.8% of survivors who were diagnosed with distress using the gold standard interview process.¹¹ Although the ESAS is extensively used and validated, most work has been done in adult advanced cancer populations.^12,13

An Australian tool for psychosocial assessment of on-therapy patients and cancer survivors has been developed for the AYA cancer population.¹⁴ This tool is presented in a checklist format, which limits the ability to use this tool to assess severity, change over time, or for research applications. The Cancer Distress Scales for AYA (CDS-AYA) is a modification of the Australian distress screening tools.^14,15 The CDS-AYA has the potential to address the gap in the availability of cancer distress measures for the AYA population. Further definition of cut-points for distress screening need to be defined before the CDS-AYA is useful for clinical practice. The objective of this study was to determine cut-points for clinical application of the CDS-AYA to screen for distress in the AYA cancer population.

Methods

AYA were defined as individuals between the ages of 15 and 39 years. Age definitions for the AYA cohort vary globally, for the purposes of this study the broadest definition, of 15–39 years of age, was chosen. This age definition for AYA is used by the Canadian Partnership Against Cancer¹⁶ and other organizations.¹⁷

Data were collected as part of the CDS-AYA field-test study from both AYA undergoing cancer treatment and those who had completed treatment. This group is collectively referred to in this article as “AYA with cancer.”¹⁵ The survey was conducted at four institutions in Canada, including Princess Margaret Cancer Centre (PMCC) (Toronto, ON), McMaster Children's Hospital (Hamilton, ON), Alberta Children's Hospital (Edmonton, AB), and British Columbia (BC) Women's and Children's Hospital (Vancouver, BC). Approval was obtained from the Research Ethics Boards at each participating institution.

Survey

AYA with cancer were recruited at scheduled clinic visits and were asked to complete a questionnaire package. The self-completed survey contained demographic and clinical questions, along with the CDS-AYA, Hospital Anxiety and Depression Scale (HADS),^18,19 ESAS-revised (ESAS-r),^10,19 and NCCN DT.¹ Patients diagnosed with any type of cancer who were undergoing treatment, in aftercare, or in survivorship were eligible to participate. Questionnaires were completed using iPads or paper booklets. Detailed methods are described in Tsangaris et al.¹⁵

CDS-AYA

The CDS-AYA is a newly developed set of scales for measuring distress in AYA with cancer. The scales were based on two checklists for AYA-specific psychosocial assessment and care developed in Australia: the AYA oncology screening tool for on-treatment patients and the AYA survivorship oncology screening tool.¹⁴ Cognitive interviews with 45 AYA with cancer aged 15–39 years, and 25 experts, were used to refine the Australian distress screening tools to create the field-test version of the CDS-AYA. Field-test data from 515 participants led to the refinement of five scales that measure distress according to modern psychometric theory.¹⁵ Field-test data were also used to assess reliability and validity of CDS-AYA in the AYA cancer population.¹⁵

There is no overall score for the CDS-AYA. Instead the CDS-AYA consists of five independently functioning scales: Impact of Cancer (12 items), Physical (12 items), Emotional (11 items), Cognitive (8 items), and Cancer Worry (5 items). Developers recommend the use of the 12-item “Impact of Cancer” scale as the primary metric because its content relates to AYA-specific concepts of development (e.g., level of independence, identity, and romantic relationships). The CDS-AYA provides four response options (none, mild, moderate, and severe). For each item, respondents are asked to indicate “how much distress they experienced in the past week.” For each of the five scales, item scores are summed and transformed onto a scale of 0–100, with higher scores reflecting more distress.

HADS

The 14-item HADS has seven depression and seven anxiety items.^18,19 Respondents are asked to indicate based on how they felt for the past week. Each item consists of four response options and a total score is generated for each of the depression and anxiety subscales that can range from 0 to 21, with higher scores indicating more symptoms of depression or anxiety. The HADS has been described as a screening tool for patients with cancer, with scores of seven and below indicating the absence of anxiety or depression as measured by the two subscales.^18–20 In a review of instruments used in cancer to screen for emotional distress, the HADS received an overall rating of “Good,” compared with the other instruments used in the field-test study, ESAS^10,21 and NCCN DT,¹ which received a lower rating of “Fair.” Based on the superior rating in the review of screening instruments, HADS was selected to assess cut-points for distress screening in the CDS-AYA scales.²²

Analysis

All analyses were conducted using Statistical Package for Social Sciences (SPSS) 25.0 (IBM SPSS Statistics, version 25, IBM Corp.).²³ Cases were excluded from the analysis if there was missing data in the CDS-AYA. Missing data for the HADS were imputed using the half rule, with the mean of the scale imputed into missing items if at least half the items were completed in the scale.²⁴ Differences in the clinical and demographic characteristics between the field-test and the cut-point analysis participants were examined using Chi-square. Statistical significance was p < 0.05.

Cut-points CDS-AYA

Methods for determining cut-points for the CDS-AYA scales were adapted from Selby et al.¹² Distress was defined using both the HADS anxiety and depression subscales. Cut-points for defining distress were based on the HADS developer's recommended categories: <8 = normal, 8–10 = borderline abnormal, and 11–21 = abnormal.^18,19 The presence of anxiety or depression for this study was represented by a combination of the borderline abnormal and abnormal HADS categories (score ≥8). A receiver operating characteristic curve was generated for each of the five CDS-AYA scales to determine cut-points and suitability of the scales for detecting distress defined by the HADS anxiety and depression scores.

The smallest distance from optimum (DFO) was used to determine the cut-point for the CDS-AYA scale that best maximized sensitivity and specificity. In cases where DFO was equal across scale values, the value that maximized sensitivity was chosen. The smallest scale cutoff value based on the HADS anxiety and depression scores were selected as the final cutoff value for the CDS-AYA scale. This approach ensured that both anxiety and depression cases were captured. Values for area under the curve (AUC) were calculated and assessed using the following categories: 0.5 no discrimination, 0.7–0.8 acceptable; 0.8–0.9 excellent, and >0.9 outstanding.²⁵

Results

Participants

In the field test a total of 515 assessments were collected across five sites in Canada, of which 453 participants were included in this study. A total of 62 cases were excluded because either the CDS-AYA scales or HADS could not be scored due to missing data. There were no significant differences in the clinical and demographic characteristics between the field-test participants and cut-point analysis sample (p ≥ 0.373). All demographic and clinical statistics were self-report and summary statistics are provided in Table 1.

Table 1.

Participant Demographic and Clinical Statistics

	Field-test sample (n = 515)		Cut-point sample (n = 453)
	n	%	n	%
Gender
Male	289	56.1	251	55.4
Female	225	43.7	201	44.4
Not reported	1	0.2	1	0.2
Age at the time of survey (years)
15–19	127	24.7	116	25.6
20–29	226	43.9	205	45.3
30–39	162	31.4	132	29.1
Diagnosis
Carcinoma	115	22.3	98	21.6
Leukemia	139	27.0	122	26.9
Lymphoma	110	21.4	100	22.1
Sarcoma	28	5.4	27	6.0
Other	107	20.8	91	20.1
Not reported	16	3.1	15	3.3
Province
Alberta	4	0.8	4	0.9
British Columbia	21	4.1	17	3.8
Ontario	490	95.1	432	95.4
Treatment status
Active	264	51.3	208	45.9
Completed	213	41.4	212	46.8
No treatment required	16	3.1	15	3.3
Not reported	22	4.3	18	4.0

Cut-points

Mean and median CDS-AYA scores are displayed in Table 2 for each of the five scales, by distress status as defined by the normal, borderline abnormal/abnormal categories for the HADS anxiety and depression scores. For the CDS-AYA scales, AUC ranged from 0.75 to 0.85 for HADS anxiety and 0.74–0.81 for HADS depression (Table 3). Table 3 displays the sensitivity and specificity for scale scores, along with the three values associated with the lowest DFO.

Table 2.

Mean and Median CDS-AYA Scores by HADS Anxiety and Depression Severity Categories

CDS-AYA scale	HADS anxiety						HADS depression
	Normal (n = 334)			Borderline abnormal/abnormal (n = 178)			Normal (n = 433)			Borderline abnormal/abnormal (n = 79)
	Median	Valid n	Mean (SD)	Median	Valid n	Mean (SD)	Median	Valid n	Mean	Median	Valid n	Mean (SD)
Impact of Cancer	7	292	13 (16)	37	163	34 (21)	13	383	17 (18)	41	72	41 (21)
Physical	22	334	20 (14)	34	178	34 (13)	25	433	23 (15)	39	79	37 (14)
Emotional	11	332	15 (15)	38	177	39 (18)	17	430	20 (17)	46	79	43 (23)
Cognitive	7	332	11 (14)	28	177	30 (20)	7	431	15 (16)	32	78	32 (22)
Cancer Worry	21	334	22 (19)	48	178	46 (25)	28	433	27 (22)	48	79	48 (25)

Scales are scored on 0–100.

CDS-AYA, Cancer Distress Scales-adolescent and young adults; HADS, Hospital Anxiety and Depression Scale; SD, standard deviation.

Table 3.

Summary of Receiver Operating Characteristic Analysis for CDS-AYA Scales

CDS-AYA scale	Anxiety HADS-borderline abnormal/abnormal							Depression HADS-borderline abnormal/abnormal
	Scale value	Sensitivity (%)	Specificity (%)	DFO	AUC	95% CI for AUC		Scale value	Sensitivity (%)	Specificity (%)	DFO	AUC	95% CI for AUC
	Scale value	Sensitivity (%)	Specificity (%)	DFO	AUC	LB	LB	Scale value	Sensitivity (%)	Specificity (%)	DFO	AUC	LB	UB
Impact of Cancer	24	70.2	71.3	0.17	0.775	0.728	0.822	33	73.2	76.5	0.13	0.801	0.741	0.861
	27	67.1	75.1	0.17				34	71.8	78.9	0.12
	28	67.1	75.4	0.17				36	64.8	81.8	0.16
Physical	24	79.5	56.4	0.23	0.746	0.699	0.792	32	69.0	73.9	0.16	0.785	0.725	0.846
	27	72.7	63.3	0.21				34	69.0	74.1	0.16
	30	63.4	70.9	0.22				35	64.8	78.9	0.17
Emotional	24	82.6	71.6	0.11	0.846	0.807	0.884	30	80.3	69.7	0.13	0.809	0.747	0.872
	27	78.3	78.9	0.09				32	76.1	72.3	0.13
	30	73.9	81.7	0.10				34	76.1	72.6	0.13
Cognitive	11	78.9	64.7	0.17	0.782	0.736	0.827	18	76.1	61.2	0.21	0.742	0.673	0.812
	15	71.4	69.9	0.17				22	73.2	68.3	0.17
	18	70.8	69.9	0.18				26	64.8	76.0	0.18
Cancer Worry	32	72.7	65.4	0.19	0.773	0.725	0.821	36	67.6	66.8	0.22	0.739	0.677	0.802
	36	68.3	77.9	0.15				41	62.0	76.3	0.20
	41	59.6	86.9	0.18				46	56.3	82.1	0.22

Optimal cutoff based on smallest DFO and maximum sensitivity represented in bold.

AUC, area under the curve; CI, confidence interval; DFO, distance from optimum; LB, lower bound; UB, upper bound.

The CDS-AYA Emotional scale had the largest AUC of the five scales. Based on DFO and maximum sensitivity, the cutoff value for the Emotional scale was determined to be 27 for anxiety (78.3%, 78.9%) and 30 for depression (80.3%, 69.7%). The CDS-AYA Impact of Cancer scale had a cut-point of 24 (70.2%, 71.3%) for anxiety, and 34 (71.8%, 78.9%) for depression based on maximum sensitivity and smallest DFO. The values of 27 for the Emotional scale and 24 for the Impact of Cancer scale were chosen as the optimal screening cutoff values to capture all cases with any anxiety or depression.

Discussion

Initial field testing results by Tsangaris et al.¹⁵ provided evidence of validity and reliability of the CDS-AYA scales in the Canadian AYA cancer population. The cut-point for the identification of borderline abnormal/abnormal cases of distress in this population improves the clinical utility of this new instrument. The Emotional scale had good classification accuracy for both HADS depression and anxiety. The Impact of Cancer scale, which was recommended as the main CDS-AYA scale with content explicitly focused on AYA issues, had similar accuracy in the classification of depression, but had slightly less accuracy in the classification of anxiety than the Emotional Scale. The CDS-AYA cut-points identified in this study should be considered for screening purposes only. The diagnostic value of these cut-points requires further validation.

In general, an AUC of 0.5 or less is considered to be associated with a test that has no better accuracy than chance, and is unable to distinguish between groups. In this study, the AUC of the CDS-AYA based on HADS anxiety and HADS depression was >0.7 in all cases. The CDS-AYA Emotional scale had the greatest AUC of 0.85 considered to be excellent.²⁵ In comparison with other instruments sensitivity and specificity of cut-points for the CDS-AYA were within the range reported for other commonly used instruments, including the NCCN DT, ESAS, and HADS.²²

Cutoff values for the NCCN DT based on HADS criterion were found to have variable sensitivity (0.63–0.86) and specificity (0.59–0.81).²² There was similar variability for ESAS cutoffs also based on HADS with sensitivity ranging from 0.61 to 0.90, and specificity ranging from 0.55 to 0.76.²² For both ESAS and NCCN DT sensitivity and specificity tended to be lower for cutoff values that were based on HADS depression criterion.²² The HADS was also found to have high variability in the sensitivity and specificity of cutoff values.²² In the five studies that used a cutoff of eight for HADS anxiety, sensitivity ranged from 0.34 to 0.94, and specificity from 0.72 to 0.88. Two studies that used a cutoff of eight based on HADS depression criterion had a sensitivity of 0.23 and 0.71, and specificity of 0.95 in both cases.²²

In the Vodermaier et al.²² review, the validity of a cutoff value was determined by averaging the reported sensitivity and specificity values, if the average was >0.8 validity was considered high. The cut-point of 27 for the Emotional scale based on HADS anxiety found in this study almost met this metric with a sensitivity of 0.78 and specificity of 0.79.

Given the variability in the sensitivity and specificity for the HADS cutoffs, basing the CDS-AYA cut-points on HADS anxiety and HADS depression criteria could further compound any inaccuracies in the initial HADS cut-point definitions. In a study by Mitchell et al.,²⁶ the authors performed a meta-analysis to determine the diagnostic validity of the HADS in both cancer and palliative care settings. This study found that the overall HADS score could identify anxiety, depression, or mixed mental disorder in only 6 out of 10 cases, and recommended that the HADS should not be used for case findings.²⁶ This weak case finding utility of the HADS in the cancer and palliative care settings presents a limitation to applying HADS to identify CDS-AYA cut-points.

Cut-points for HADS scales were also developed primarily for adults, and not specifically for AYA. For HADS, White et al.²⁷ identified a lower cut-point for the depression scores and higher cut-point for anxiety scores for the 15- to 17-year-old age group than those recommended by the developer. Cut-points from the White et al.²⁷ study were not used in this study due to the unknown applicability to the 18- to 39-year-old age group. Further research to examine the association between the cut-points for CDS-AYA and clinical diagnosis of distress through gold standard interviews in AYA patients and survivors is warranted.

Psychosocial care is an important aspect of person-centered care, and a priority for AYA with cancer. It is recommended that screening focus on the use of the CDS-AYA Impact of Cancer or Emotional scales, the remaining scales can be administered in the event of a positive screen for higher levels of distress to gather further information. The CDS-AYA fills an important gap in AYA-specific distress screening tools for cancer that captures issues important to this population. To improve the applicability of the CDS-AYA as a screening tool in clinical practice, further study needs to focus on determining referral pathway based on scale scores to facilitate proper care of AYA after screening.

Footnotes

Acknowledgments

This study was conducted with support from C17 Council, and funded by the Childhood Cancer Canada Foundation and the Kids with Cancer Society. We thank Abha Gupta, Dilshan Pieris, Kate Wang, Seline Tam, Nivetha Ramachandran, Colleen Fitzgerald, Sarah McKillop, and Linda Churcher for their contributions. We also thank Pandora Patterson, Michael Osborne, Fiona McDonald developers of the Australian DT AYA on-treatment and survivorship tools.

Author Disclosure Statement

No competing financial interests exist.

References

National Comprehensive Cancer Network (NCCN). NCCN clinical practice guidelines in oncology: distress management, version 2.2018. Fort Washington, PA: NCCN; 2018.

Bellizzi

, Smith

, Schmidt

, et al. Positive and negative psychosocial impact of being diagnosed with cancer as an adolescent or young adult. Cancer. 2012; 118:5155–62.

D'agostino

, Penney

, Zebrack

. Providing developmentally appropriate psychosocial care to adolescent and young adult cancer survivors. Cancer. 2011; 117:2329–34.

Zebrack

. Psychological, social, and behavioral issues for young adults with cancer. Cancer. 2011; 117:2289–94.

Enskär

, von Essen

. Prevalence of aspects of distress, coping, support and care among adolescents and young adults undergoing and being off cancer treatment. Eur J Oncol Nurs. 2007; 11:400–8.

Bultz

, Groff

, Fitch

, et al. Implementing screening for distress, the 6th vital sign: a Canadian strategy for changing practice. Psychooncology. 2011; 20:463–9.

Bultz

, Johansen

. Screening for distress, the 6th vital sign: where are we, and where are we going?. Psychooncology. 2011; 20:569–71.

Carlson

, Waller

, Mitchell

. Screening for distress and unmet needs in patients with cancer: review and recommendations. J Clin Onc. 2012; 30:1160–77.

Howell

, Hack

, Green

, Fitch

. Cancer distress screening data: translating knowledge into clinical action for a quality response. Palliat Support Care. 2014; 12:39–51.

10.

Watanabe

, Nekolaichuk

, Beaumont

. The Edmonton Symptom Assessment System, a proposed tool for distress screening in cancer patients: development and refinement. Psychooncology. 2012; 21:977–85.

11.

Recklitis

, Blackmon

, Chang

. Screening young adult cancer survivors for distress with the distress thermometer: comparisons with a structured clinical diagnostic interview. Cancer. 2016; 122:296–303.

12.

Selby

, Cascella

, Gardiner

, et al. A single set of numerical cutpoints to define moderate and severe symptoms for the Edmonton Symptom Assessment System. J Pain Symptom Manag. 2010; 39:241–9.

13.

Hughes

, Holland

, Zanino

, et al. Prevalence and intensity of pain and other physical and psychological symptoms in adolescents and young adults diagnosed with cancer on referral to a palliative care service. JAYAO. 2015; 4:70–5.

14.

Palmer

, Patterson

, Thompson

. A national approach to improving adolescent and young adult (AYA) oncology psychosocial care: the development of AYA-specific psychosocial assessment and care tools. Palliat Support Care. 2014; 12:183–8.

15.

Tsangaris

, D'Agostino

, Rae

, et al. Development and psychometric evaluation of the cancer distress scales for adolescent and young adults (CDS-AYA). JAYAO. In press.

16.

Adolescent and young adults with cancer: technical report. Toronto, ON: Canadian Partnership Against Cancer; 2017.

17.

Department of Health and Human Services, National Institutes of Health, National Cancer Institute, LiveStrong Young Adult Alliance. Closing the gap; research and care imperatives for adolescents and young adults with cancer: a report of the Adolescent and Young Adult Oncology Progress Review Group; 2006. Accessed June 3, 2019 from: www.cancer.gov/types/aya/research/ayao-august-2006.pdf.

18.

Snaith

. The hospital anxiety and depression scale. Health Qual Life Out. 2003; 1:29.

19.

Zigmond

, Snaith

. The hospital anxiety and depression scale. Acta Psychiat Scand. 1983; 67:361–70.

20.

Carroll

, Kathol

, Noyes Jr

, et al. Screening for depression and anxiety in cancer patients using the Hospital Anxiety and Depression Scale. Gen Hosp Psychiat. 1993; 15:69–74.

21.

Vignaroli

, Pace

, Willey

, et al. The Edmonton Symptom Assessment System as a screening tool for depression and anxiety. J Palliat Med. 2006; 9:296–303.

22.

Vodermaier

, Linden

, Siu

. Screening for emotional distress in cancer patients: a systematic review of assessment instruments. J Natl Cancer Inst. 2009; 101:1464–88.

23.

IBM Corp. IBM SPSS statistics, version 25.0. Armonk, NY: IBM Corp; 2018.

24.

Bell

, Fairclough

, Fiero

, Butow

. Handling missing items in the Hospital Anxiety and Depression Scale (HADS): a simulation study. BMC Res Notes. 2016; 9:479.

25.

Mandrekar

. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol. 2010; 5(9):1315–6.

26.

Mitchell

, Meader

, Symonds

. Diagnostic validity of the Hospital Anxiety and Depression Scale (HADS) in cancer and palliative settings: a meta-analysis. J Affect Disord. 2010; 126(3):335–48.

27.

White

, Leach

, Sims

, et al. Validation of the Hospital Anxiety and Depression Scale for use with adolescents. Br J Psychiatry. 1999; 175:452–4.