Multiple Group Confirmatory Factor Analysis of the DASS-21 Depression and Anxiety Scales: How Do They Perform in a Cancer Sample?

Abstract

The DASS-21 is a public domain instrument that is commonly used to evaluate depression and anxiety in psychiatric and community populations; however, the factor structure of the measure has not previously been examined in oncologic settings. Given that the psychometric properties of measures of distress may be compromised in the context of symptoms related to cancer and its treatment, the present study evaluated the psychometric properties of the DASS-21 Depression and Anxiety scales in cancer patients (n = 376) as compared to noncancer control participants (n = 207). Cancer patients ranged in age from 21 to 84 years (mean = 58.3, standard deviation = 10.4) and noncancer control participants ranged in age from 18 to 81 years (mean = 45.0, standard deviation = 11.7). Multiple group confirmatory factor analysis supported the structural invariance of the DASS-21 Depression and Anxiety scales across groups; the factor variance/covariance invariance model was the best fit to the data. Cronbach’s coefficient alpha values demonstrated acceptable internal consistency reliability across the total sample as well as within subgroups of cancer patients and noncancer control participants. Expected relationships of DASS-21 Depression and Anxiety scale scores to measures of suicidal ideation, quality of life, self-rated health, and depressed mood supported construct validity. These results support the psychometric properties of the DASS-21 Depression and Anxiety scales when measuring psychological distress in cancer patients.

Keywords

DASS-21 depression anxiety psychometrics cancer

Introduction

Depression and anxiety are among the most commonly reported aspects of cancer and its treatment (American Cancer Society, 2014; Jacobsen & Andrykowski, 2015), and nearly all cancer patients experience at least some symptoms of distress (American Cancer Society, 2014). Given the prevalence of depression and anxiety in cancer, accrediting agencies now mandate routine distress screening for patients with cancer (American College of Surgeons Commission on Cancer, 2015; Andersen et al., 2014; National Comprehensive Cancer Network, 2013). Accordingly, accurately and appropriately assessing distress in cancer patients has garnered a great deal of research attention (Vodermaier, Linden, & Siu, 2009). Although multiple validated tools have been developed to assess depression and anxiety in psychiatric and primary care settings, the psychometric properties of many of these tools have rarely been explored specifically among patients with cancer. Given that cancer patients often experience pain, fatigue, concerns about mortality, and other symptoms related to their cancer and its treatment that may closely approximate symptoms of depression and anxiety, measures that accurately assess psychological distress in other populations may not function equivalently in cancer populations (Hopko et al., 2007). Without establishing the strength of a given measure’s psychometric properties in the population with which the measure will be used, it is impossible to know if the measure assesses what it aims and purports to assess, and that it can be used for its intended purposes (Groth-Marnat, 2009). As such, it is important to evaluate the psychometric properties of measures of psychological distress among individuals diagnosed with cancer (Kroenke, Spitzer, Williams, & Lowe, 2010; Waller, Compas, Hollon, & Beckjord, 2005; Zigmond & Snaith, 1983).

The Depression Anxiety Stress Scales (DASS; Lovibond & Lovibond, 1995) refers to a 42-item package of measures that are used clinically and in research (e.g., Antony, Orsillo, & Roemer, 2001; Osman et al., 2012) to evaluate and distinguish three areas of distress: depression symptoms, anxiety symptoms, and general stress. The measure is also available in a 21-item short form (DASS-21). Both the DASS and the DASS-21 can be administered flexibly to assess only specific symptoms (e.g., depression) if desired. Items on the depression scale evaluate symptoms of dysphoric mood (e.g., “I couldn’t seem to experience any positive feeling at all”). Items on the anxiety scale evaluate symptoms related to physiological hyperarousal, much like those seen in questionnaires evaluating panic (e.g., “I was aware of dryness of my mouth”). Finally, items on the stress scale reflect negative affectivity (e.g., “I felt that I was rather touchy”).

The psychometric properties of the DASS-21 have been evaluated in various populations, including adult psychiatric patients between the ages of 16 and 91 years (Antony, Bieling, Cox, Enns, & Swinson, 1998; Ng et al., 2007; Ronk, Korman, Hooke, & Page, 2013), adult community samples between the ages of 18 and 91 years (Henry & Crawford, 2005; Osman et al., 2012; Sinclair et al., 2012), and older adults in medical settings aged 61 years or older (Gloster et al., 2008; Wood, Nicholas, Blyth, Asghari, & Gibson, 2010). The DASS-21 has also been translated into multiple languages, such as Persian (Asghari, Saded, & Dibajnia, 2008), Spanish (Daza, Novy, Stanley, & Averill, 2002), Chinese (Taouk, Lovibond, & Laube, 2001), Nepali (Tonsing, 2014), and Vietnamese (Tran, Tran, & Fisher, 2013). Strong evidence has been found for the internal consistency reliability of the measure’s three scales (Antony et al., 1998; Asghari et al., 2008; Daza et al., 2002; Gloster et al., 2008; Henry & Crawford, 2005; Osman et al., 2012; Ronk et al., 2013; Sinclair et al., 2012; Tran et al., 2013; Wood et al., 2010). Construct validity has been demonstrated via significant positive correlations with other validated measures of depression and anxiety (Antony et al., 1998; Asghari et al., 2008; Daza et al., 2002; Gloster et al., 2008; Henry & Crawford, 2005; Osman et al., 2012; Sinclair et al., 2012), and significant negative correlations with measures of positive affectivity and quality of life (Gloster et al., 2008; Henry & Crawford, 2005; Sinclair et al., 2012; Tonsing, 2014). Moreover, the measure has successfully discriminated individuals with diagnosed psychological disorders from those without psychological disorders (Antony et al., 1998; Asghari et al., 2008; Gloster et al., 2008; Tran et al., 2013). Although the DASS-21 has evidenced strong psychometric properties in varied populations, it has not been validated for use specifically in cancer.

The strong psychometric support for the DASS-21 across psychiatric, medical, and community samples, the empirical basis for its development, and the measure’s ability to distinguish among multiple dimensions of psychological distress indicate that it could be an appropriate measure for use by oncology researchers and clinical providers. However, before such utility can be supported, it must be determined that the measure functions equivalently well among cancer patients as compared to populations in which the psychometric properties have already been shown to be strong (Groth-Marnat, 2009), such as community-based samples (Henry & Crawford, 2005; Osman et al., 2012; Sinclair et al., 2012). Thus, the present study evaluated the structural validity, internal consistency reliability, and construct validity of the DASS-21 Depression and Anxiety scales in a sample of cancer patients as compared to a noncancer sample. The primary study hypothesis was that the DASS-21 Depression and Anxiety scales would demonstrate measurement invariance across cancer and noncancer samples. It was further hypothesized that scores on the DASS-21 Depression and Anxiety scales would be (1) positively related to reports of suicidal ideation as measured by the ninth item of the Patient Health Questionnaire-9 (PHQ-9; Kroenke, Spitzer, & Williams, 2001), (2) positively correlated with depressed mood as measured by an item from the McGill Quality of Life Inventory (Cohen, Mount, Strobel, & Bui, 1995), (3) negatively correlated with quality of life as measured by other selected items from the McGill Quality of Life Inventory (Cohen et al., 1995), and (4) negatively correlated with self-rated health as measured by the SF-1 (Ware & Sherbourne, 1992).

Methods

Participants and procedure

The current study utilized an Internet-mediated survey design. Two separate protocols were administered simultaneously, one of which was available for patients currently receiving cancer treatment (Hoerger, Chapman, Mohile, & Duberstein, 2016) and the other for individuals who had no history of cancer. To be eligible to participate as part of the cancer patient subsample (n = 376), individuals had to be at least 18 years old, be able to read English, have a current diagnosis of cancer, and currently be receiving oncologic care. To be able to participate as part of the noncancer control subsample (n = 207), participants had to be at least 18 years old, be able to read English, and have no history of cancer.

Cancer patients were recruited through the NIH Clinical and Translational Science ResearchMatch (Harris et al., 2012) recruitment tool, which allows for the recruitment of cancer patients from an online database of over 80,000 patient volunteers. The current study’s website was advertised, with permission, through other health education websites, search engines, discussion forums, and relevant listservs. Noncancer control participants were recruited through the websites Psychological research on the net (http://psych.hanover.edu/Research/exponnet.html; Krantz, 2014) and Online social psychology studies (https://www.socialpsychology.org/expts.htm; Plous, 2014), two websites that advertise Internet-mediated academic research participation opportunities to the general public. The study website was also posted to other websites, search engines, discussion forums, and relevant listservs. The study website included an online consent form, contact information for study personnel, and all study survey materials. Participants were given automated, normed personality feedback based on responses to the Mini-IPIP (Donnellan, Oswald, Baird, & Lucas, 2006), which was completed as part of the study protocol, immediately following completion of study activities as a demonstration of appreciation for their time and effort. Participants were also given the opportunity to enter a lottery for $100. The study was approved by the sponsoring university’s Institutional Review Board prior to enrollment, and informed consent was obtained from all individual participants included in the study. All study procedures were in compliance with American Psychological Association ethical standards.

Measures

DASS-21 Depression and Anxiety Scales (Lovibond & Lovibond, 1995)

The DASS-21 Depression and Anxiety scales are two of the three scales that comprise the DASS-21. The Stress scale was not administered in the present study to streamline the assessment for use in oncology settings and to reduce participant burden. Each scale comprises seven items, and respondents rate the applicability of each item to the past week on a scale from 0 (Not at all) to 3 (Most of the time). Higher scores indicate more severe symptoms of psychological distress.

Validation measures

Measures assessing quality of life, depressed mood, self-rated health, and suicidal ideation were utilized for validation, as these constructs have been previously shown to be associated with psychological distress (Applebaum et al., 2014; Kotov, Gamez, Schmidt, & Watson, 2010; Meurer, Layde, & Guse, 2001).

Quality of life

Quality of life was measured by four items adapted from the McGill Quality of Life Inventory (Cohen et al., 1995). On a scale from 1 (Not at all) to 10 (Extremely), respondents were asked to rate (1) their current feelings of happiness, (2) their current perception of their life as purposeful and meaningful, and (3) their current perception that their days feel like a burden. In addition, on a scale from 1 (Very bad) to 10 (Excellent), respondents were asked to rate (4) their overall quality of life, including physical, emotional, social, spiritual, and financial quality of life. Responses were averaged to yield a composite indicator of quality of life (α = .86).

Depressed mood

Depressed mood was assessed with an item adapted from the McGill Quality of Life Inventory (Cohen et al., 1995). On a scale from 1 (Not at all) to 10 (Extremely), respondents were asked to rate their current feelings of depression. Higher scores indicated more severe depressed mood.

Self-rated health

Self-rated health was assessed using the first item from Ware and Sherbourne’s (1992) SF-36 health questionnaire. This item, which is often referred to as the SF-1, asks respondents to rate their overall health on a scale from 1 (Poor) to 5 (Excellent), wherein higher scores indicate better self-rated health. The item is commonly used in research to evaluate self-rated health, and ratings have been shown to be associated with mortality even after controlling for objective indicators of overall health (DeSalvo, Bloser, Reynolds, He, & Muntner, 2006).

Suicidal ideation

Suicidal ideation was evaluated with the ninth item of the PHQ-9 (Kroenke et al., 2001). This item asks respondents to rate the frequency with which they have had “thoughts that you would be better off dead or of hurting yourself in some way” over the prior two weeks. Response options range from 0 (Not at all) to 3 (Nearly every day). For the present analysis, responses were dichotomized to reflect presence or absence of reported suicidal ideation. In the current sample, 20.5% of cancer patients and 18.8% of noncancer control participants endorsed presence of suicidal ideation. As the present study was an anonymous online survey, participants were informed in advance that they would receive no clinical intervention as a result of their questionnaire responses. Participants experiencing suicidal ideation had the option of contacting the study investigators, and all participants had access to mental health resources at the end of the study.

Analytic plan

Multiple group confirmatory factor analysis was used to evaluate the structural invariance of the DASS-21 across cancer patients and noncancer control participants. Based on prior research (Henry & Crawford, 2005; Lovibond & Lovibond, 1995) and theory (Clark & Watson, 1991), both a one-factor model (general distress) and a two-factor model (depression and anxiety) were evaluated.

According to the guidelines set forth by Vandenberg and Lance (2000), three increasingly restrictive models were iteratively examined to determine the degree of model invariance across cancer patients and noncancer control participants. First, to establish configural invariance and evaluate if the same factors exist across groups, the baseline fit of the measurement model was evaluated in two separate confirmatory factory analyses with each group. Second, metric invariance was evaluated by constraining factor loadings to equivalence across groups while factor variances, factor covariance, and error variances were freely estimated. In the third and final model, factor variances and covariance were also constrained to equivalence across groups to explore factor variance/covariance invariance. Only the error variances were freely estimated in this most restrictive model. Each model was statistically compared to the less restrictive iteration using a χ² difference test to determine which model best fit the observed data.

Overall model fit was determined using Bentler’s (2007) recommendations. Multiple indicators of overall model fit were examined, including (a) the Comparative Fit Index (CFI; Bentler, 1990), an absolute index with values ≥ .95 indicating good model fit and values ≥ .90 indicating acceptable fit; (b) the Root Mean Square Error of Approximation (RMSEA; Steiger, 1990), a parsimony-adjusted index with values ≤ .08 indicating acceptable model fit and values ≤ .05 indicating good fit; and (c) the Standardized Root Mean Residual (SRMR; Hu & Bentler, 1999), an absolute index with values ≤ .08 indicating acceptable model fit and values ≤ .05 indicating good fit. A model was determined to fit well if two of these three criteria demonstrated at least acceptable model fit. The likelihood ratio χ² was also reported for completeness; however, it was not considered the only indicator of model fit because it is highly influenced by sample size and does not demonstrate degree of fit (Bentler, 2007).

Once the optimal structural model was identified, sensitivity analyses were conducted controlling for age, gender, education, marital status, and race to evaluate the potential impact of these covariates on model fit. Internal consistency reliabilities of the DASS-21 Depression and Anxiety scales were then evaluated using Cronbach’s alpha. Construct validity was explored via Pearson product-moment correlations of the DASS-21 Depression and Anxiety scale scores with scores on measures of quality of life, depressed mood, and self-rated health. Binary logistic regression analysis explored the relationship between DASS-21 Depression and Anxiety scores and reports of suicidal ideation. For the present study, the multiple group confirmatory factor analysis was conducted using MPlus version 5.21 (Muthén & Muthén, 1998–2011). All other analyses were completed in SPSS version 19 (IBM Corporation, Released 2010).

Results

Descriptive statistics and preliminary analysis

Sample characteristics can be found in Table 1. Total DASS-21 Depression scale scores ranged from 0 to 21 for both groups. Total DASS-21 Anxiety scale scores ranged from 0 to 15 for cancer patients and from 0 to 18 for control participants. Average DASS-21 Depression and Anxiety scale scores did not significantly differ across the two groups. Average scores on the scales in both groups were below previously identified cutoff values used to indicate clinical levels of distress (Eng & Chan, 2013). The data evidenced multivariate nonnormality, thus the Satorra-Bentler χ² (S-Bχ²; Satorra & Bentler, 1998) test statistic was used in all steps of the multiple group confirmatory factor analysis rather than a nonscaled estimate.

Table 1.

Sample characteristics.

	Cancer (n = 376)	Noncancer (n = 207)
DASS-21 Depression^a	4.3 (4.3)	4.2 (5.0)
DASS-21 Anxiety^a	2.5 (2.8)	2.8 (3.4)
Age^a,b	58.3 (10.4)	45.0 (11.7)
Gender^c,d
Female	123 (32.7)	152 (73.4)
Male	253 (67.3)	55 (26.6)
Education^c,d
Less than bachelor’s degree	123 (32.7)	100 (48.3)
Bachelor’s degree or higher	253 (67.3)	107 (51.7)
Marital status^c,d
Unmarried	82 (21.8)	108 (52.2)
Married	294 (78.2)	99 (47.8)
Race^c,d
White, non-Latino	354 (94.1)	159 (76.8)
Non-White	22 (5.9)	48 (23.2)
Cancer site^c
Prostate	212 (30.9)	—
Breast	57 (8.3)	—
Colon/rectal	51 (7.4)	—
Hematologic	28 (4.1)	—
Other	28 (4.1)	—
Cancer stage^c
Post-tx active monitoring	61 (10.5)	—
Localized	143 (24.5)	—
Metastatic	116 (19.9)	—
Uncertain about stage	56 (9.6)
Time since diagnosis (years)^a	2.46 (2.80)	—

M (SD).

Independent sample t test resulted in a significant difference at p < .01 (two tailed) between groups.

n (%).

Chi-squared analysis resulted in a significant difference at p < .01 (two tailed) between groups. Noncancer participants had no history of cancer.

Structural validity

Configural invariance

The one-factor model reflecting general psychological distress did not fit well statistically or descriptively for cancer patients (S-Bχ²= 272.99, p < .01, CFI = .87, RMSEA = .08, SRMR = .07) or for noncancer control participants (S-Bχ²= 222.77, p < .01, CFI = .86, RMSEA = .10, SRMR = .08). Thus, no further analyses were conducted evaluating the one-factor model.

The two-factor model fit well for both cancer patients and noncancer control participants, demonstrating configural invariance (Table 2). For responses provided by both groups, all estimated unstandardized factor loadings for both the Depression and Anxiety scales were statistically significant (p < .01; Table 3), as was the variance for the Depression factor (Patients: Φ = .33, p < .01; Controls: Φ = .45, p < .01). The variance for the Anxiety factor was not statistically significant for the noncancer control participants (Φ = .10, p = .09), although it was for the cancer patients (Φ = .08, p = .03), indicating less of a spread on responses to items assessing Anxiety among noncancer control participants as compared to cancer patients. The unstandardized covariance between the two factors was statistically significant for both groups (Patients: Φ = .11, p < .01; Controls: Φ = .15, p < .01), indicating that the Depression and Anxiety latent factors were positively related to one another in both groups.

Table 2

Fit statistics for configural invariance, metric invariance, and factor variance/covariance invariance models.

Model	S-Bχ²	df	p	CFI^a	RMSEA^b	SRMR^b	Reference model #	ΔS-Bχ²	Δdf	Δp
1. Configural	277.025	152	<.001
Cancer patients	158.993	76	<.001	.944	.054	.046	–	–	–	–
Control	118.032	76	.001	.960	.052	.048	–	–	–	–
2. Metric	312.060	176	<.001	.946	.051	.055	1	35.222	24	.065
3. Factor	321.432	179	<.001	.943	.052	.078	2	7.568	3	.055

CFI: comparative fit index; RMSEA: root mean square error of approximation; SRMR: standardized root mean square residual.

Acceptable fit ≥ .90, good fit ≥ .95.

Acceptable fit ≤ .08, good fit ≤ .05.

Table 3

Unstandardized factor loadings from baseline models of the DASS-21 Depression and Anxiety Scales among cancer patients and noncancer control participants.

DASS-21 item	Cancer (n = 376)	Noncancer (n = 207)
Depression
I couldn’t seem to experience any positive feeling at all	1.03*	1.19*
I found it difficult to work up the initiative to do things	0.97*	0.84*
I felt that I had nothing to look forward to	1.17*	1.22*
I felt down-hearted and blue	1.21*	1.18*
I was unable to become enthusiastic about anything	1.14*	1.16*
I felt I wasn’t worth much as a person	0.89*	0.91*
I felt that life was meaningless	0.72*	0.92*
Anxiety
I was aware of dryness of my mouth	1.01*	1.11*
I experienced breathing difficulty (e.g., excessively rapid breathing, breathlessness in the absence of physical exertion)	0.99*	0.90*
I experienced trembling (e.g., in the hands)	1.17*	1.25*
I was worried about situations in which I might panic and make a fool of myself	1.46*	1.85*
I felt I was close to panic	1.54*	1.58*
I was aware of the action of my heart in the absence of physical exertion (e.g., sense of heart rate increase, heart missing a beat)	0.83*	1.32*
I felt scared without any good reason	1.20*	1.56*

p < .05.

Metric invariance

Good model fit was found for the metric invariance model, indicating equivalence of the factor loadings across groups (Table 2). Furthermore, the metric invariance model was a superior fit to the data, when this model was statistically compared to the configural invariance models.

Factor variance/covariance invariance

This most restrictive model fit the data well (Table 2; Figure 1). When this model was statistically compared to the metric invariance model, it was found to be a superior fit to the data, demonstrating that the association between depression and anxiety, and the spread of DASS-21 Depression and Anxiety scale scores, was equivalent across groups. Sensitivity analyses controlling for age, gender, education, marital status, and race demonstrated no change in model fit as compared to the unadjusted analyses.

Figure 1.

Factor variance/covariance invariance structural model. All values presented were constrained to equivalence across groups. *p < .05.

Internal consistency reliability

Across the total sample, internal consistency reliability was good for both the Depression (α = .91) and Anxiety (α = .74) scales of the DASS-21. For the cancer patient subsample, internal consistency reliability was good for the Depression scale (α = .90) and acceptable for the Anxiety scale (α = .70). Internal consistency reliability was good for both scales for the control participants (Depression: α = .93; Anxiety: α = .80).

Construct validity

Scores on the DASS-21 Depression and Anxiety scales demonstrated significant bivariate correlations with measures of quality of life, depressed mood, and self-rated health in the expected directions (Table 4). Fisher’s r-to-z tests demonstrated that the relationships between quality of life and the Depression scale were significantly stronger than the relationships between quality of life and the Anxiety scale for the total sample, and within each group separately. The relationships between scores on the item querying depression and the DASS-21 Depression scale were also significantly stronger than the relationships with the Anxiety scale for the total sample and for each subsample. Correlations did not differ significantly across cancer and noncancer participants for any comparison. Additionally, among the total sample, the unadjusted odds of reporting suicidal ideation were higher for participants with higher DASS-21 Depression (OR = 1.39, 95% CI = 1.31–1.48) and Anxiety (OR = 1.31, 95% CI = 1.22–1.40) scores as opposed to those with lower scores. These relationships were also found when cancer patients (Depression: OR = 1.40, 95% CI = 1.29–1.51; Anxiety: OR = 1.28, 95% CI = 1.15–1.37) and control participants (Depression: OR = 1.39, 95% CI = 1.25–1.54; Anxiety: OR = 1.40, 95% CI = 1.24–1.57) were examined separately. These results provided support for the construct validity of the two-factor structure of the DASS-21 Depression and Anxiety scales.

Table 4

Correlations between scores on the DASS-21 Depression and Anxiety Scales and measures of quality of life, depression, and self-rated health.

	Total sample (N = 583)		Cancer (n = 376)		Noncancer (n = 207)
	DASS-21 Depression	DASS-21 Anxiety	DASS-21 Depression	DASS-21 Anxiety	DASS-21 Depression	DASS-21 Anxiety
Quality of life	−.76***^,a	−.36***^,a	−.73***^,b	−.36***^,b	−.79***^,c	−.35***^,c
Self-rated health	−.27***	−.26***	−.25***	−.31***	−.32***	−.23**
Single-item depression	.71***^,d	.41***^,d	.69***^,e	.39***^,e	.74***^,f	.45***^,f

p < .01, ***p < .001. Within each row, correlation coefficients with the same superscript were significantly different from one another according to Fisher’s r-to-z tests; p < .05.

Discussion

The primary goal of the present study was to evaluate the appropriateness of the DASS-21 Depression and Anxiety scales for use with cancer patients. Given that leading accrediting agencies now mandate routine distress screening for patients with cancer (American College of Surgeons Commission on Cancer, 2015; Andersen et al., 2014; National Comprehensive Cancer Network (NCCN), 2013), it is important to identify measures with sound psychometric properties for use in oncologic settings. Multiple group confirmatory factor analysis demonstrated that the DASS-21 Depression and Anxiety scales measured the constructs of depression and anxiety equivalently across cancer patients and control participants. Results of this analysis showed that the factor variance/covariance invariance model was the best fit to the data. The same two-factor structure was found across groups, with items contributing to the same scales at equivalent levels, and with the same spread of responses on the scales and the same relationship between the scales, across groups. Thus, results suggest that the DASS-21 Depression and Anxiety scales are a structurally valid assessment of psychological distress to be used with cancer patients. Additionally, consistent with long-standing evidence and theory (Lovibond & Lovibond, 1995), the Depression and Anxiety factors were significantly correlated in both cancer patients and control participants. This suggests that, while Depression and Anxiety are best measured and conceptualized as two distinct constructs, individuals experiencing symptoms of either depression or anxiety are likely to experience at least some symptoms of the other, regardless of cancer diagnostic history.

The internal consistency reliability of the DASS-21 Depression and Anxiety scales was also supported. Cronbach’s coefficient alpha values for both scales demonstrated acceptable reliability (Groth-Marnat, 2009) across the total sample as well as within subgroups of cancer patients and control participants. As would be expected, average DASS-21 Depression and Anxiety scale scores for the current sample were similar to those previously observed among community samples (Henry & Crawford, 2005; Osman et al., 2012; Sinclair et al., 2012), older adults in medical settings (Gloster et al., 2008), and chronic pain patients (Wood et al., 2010), and lower than those observed among psychiatric patients (Antony et al., 1998; Ng et al., 2007; Ronk et al., 2013). Given that participants in the present sample generally did not meet criteria for mental health diagnoses, but rather were eligible for inclusion because of presence or absence of a medical diagnosis, it follows that scores on the DASS-21 Depression and Anxiety scales were lower than those observed among psychiatric samples.

Evidence of the construct validity of the DASS-21 Depression and Anxiety scales was also found. Consistent with expectations and prior research (Applebaum et al., 2014; Meurer et al., 2001), increased reports of depression and anxiety symptoms were associated with increased odds of reporting suicidal ideation, poorer quality of life, and worse self-rated health. The relationships between scores on the item querying depression and the DASS-21 Depression scale were also significantly stronger than the relationships with the Anxiety scale for the total sample and for each subsample, as would be expected. The relationship between scores on the measure of quality of life and DASS-21 scale scores was also stronger for the Depression scale than for the Anxiety scale, as demonstrated by the correlations outlined in Table 4. Additionally, Cronbach’s alpha values were lower for the Anxiety scale than the Depression scale. The Depression scale is focused on affective symptoms, while the Anxiety scale targets somatic sensations. Participants across both groups endorsed more affective than somatic symptoms of distress. Similar patterns of responding have been observed in other studies using the DASS-21 in varied populations (Henry & Crawford, 2005; Norton, 2007).

The present findings must be interpreted within the context of relevant limitations. The present study was cross-sectional in design, precluding evaluation of the test–retest reliability of the DASS-21 Depression and Anxiety scales. Additionally, in an effort to diminish participant burden the DASS-21 Stress scale was not administered, and thus the psychometric properties of this scale could not be evaluated. Furthermore, there were a number of sociodemographic variables that significantly differed across cancer patients and non-cancer control participants. Although it is possible that these sociodemographic differences may have impacted how the DASS-21 Depression and Anxiety scales functioned in the two groups, tests of measurement invariance indicated that the scales functioned similarly across the two groups. Moreover, adjusting for these differences in sensitivity analyses did not impact model fit. Taken together, these findings indicate that the DASS-21 Depression and Anxiety scales operated similarly despite being administered to diverse samples, and therefore, similarly captured depressive and anxious symptoms across the demographic contexts of the present study.

Conclusion

In conclusion, this study investigated the psychometric properties of the DASS-21 Depression and Anxiety scales across a diverse sample of cancer patients and a sample of participants without a history of cancer. The present results suggest that the DASS-21 Depression and Anxiety scales represent a psychometrically viable option for health systems and clinicians when assessing distress in cancer patients.

Footnotes

Author Biographies

Rina S. Fox, PhD, MPH, is currently a postdoctoral fellow in the Department of Medical Social Sciences at the Northwestern University Feinberg School of Medicine. Her research interests are focused on developing and evaluating interventions to promote health-related quality of life among cancer survivors.

Teresa A. Lillis has a PhD in clinical psychology from the University of Kansas and is currently a postdoctoral fellow in the Department of Behavioral Sciences at the Rush University Medical Center. Her research interests are in women's health, traumatic stress, and sleep.

James Gerhart, PhD, is a clinical psychologist and directs the Psychosocial Oncology and Cancer Integrative Medicine Program at the Rush University Cancer Center. His research focuses on understanding how patient and provider distress impacts communication and medical decision-making.

Michael Hoerger, PhD, MSCR, is an assistant professor of psychology, psychiatry, and oncology at the Tulane University. His research aims to improve quality of life by anticipating, preventing, and alleviating the physical and emotional burden of cancer.

Paul Duberstein, PhD, is interested in leveraging advances in the psychological, behavioral, and social sciences to improve the delivery of health care to patients with advanced, life-limiting disease and their caregivers. He received a doctoral degree in clinical community psychology from the SUNY Buffalo, completed an internship in clinical psychology at the McLean Hospital/Harvard Medical School, and completed a research fellowship in geriatric mental health at the University of Rochester Medical Center.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the following grants: National Institute of Mental Health T32MH018911, National Cancer Institute R01CA168387, National Institute of General Medical Sciences U54GM104940, and the University of Rochester Medical Center Department of Psychiatry Leonard F. Salzman Research Award. Dr. Gerhart was supported by the Coleman Foundation and the Prince Charitable Trusts. Dr. Fox was supported through a National Cancer Institute Training Grant 5T32CA193193.

References

American Cancer Society (2014) Cancer treatment and survivorship facts & figures 2014–2015, Atlanta, GA: American Cancer Society.

American College of Surgeons Commission on Cancer (2015) Cancer program standards: Ensuring patient-centered care, Chicago, IL: American College of Surgeons.

Andersen

B. L.

DeRubeis

R. J.

Berman

B. S.

Gruman

Champion

V. L.

Massie

M. J.

Rowland

J. H.

(2014) Screening, assessment, and care of anxiety and depressive symptoms in adults with cancer: An American Society of Clinical Oncology guideline adaptation. Journal of Clinical Oncology 32: 1605–1619.

Antony

M. M.

Bieling

P. J.

Cox

B. J.

Enns

M. W.

Swinson

R. P.

(1998) Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychological Assessment 10: 176–181.

Antony

M. M.

Orsillo

S. M.

Roemer

(2001) Practitioner’s guide to empirically based measures of anxiety, New York, NY: Kluwer Academic/Plenum.

Applebaum

A. J.

Stein

E. M.

Lord-Bessen

Pessin

Rosenfeld

Breitbart

(2014) Optimism, social support, and mental health outcomes in patients with advanced cancer. Psychooncology 23: 299–306.

Asghari

Saded

Dibajnia

(2008) Psychometric properties of the Depression Anxiety Stress Scales-21 (DASS-21) in a non-clinical Iranian sample. International Journal of Psychology 2: 82–102.

Bentler

P. M.

(1990) Comparative fit indexes in structural models. Psychological Bulletin 107: 238–246.

Bentler

P. M.

(2007) On tests and indices for evaluating structural models. Personality and Individual Differences 42: 825–829.

10.

Clark

L. A.

Watson

(1991) Theoretical and empirical issues in differentiating depression from anxiety. In: Kleinman

J. B. A.

(eds) Psychosocial aspects of depression, Hillsdale, NJ: Lawrence Erlbaum Associates, Inc, pp. 39–65.

11.

Cohen

S. R.

Mount

B. M.

Strobel

M. G.

Bui

(1995) The McGill Quality of Life Questionnaire: A measure of quality of life appropriate for people with advanced disease. A preliminary study of validity and acceptability. Palliative Medicine 9: 207–219.

12.

IBM

Corporation.

(Released 2010) IBM SPSS Statistics for Windows, Version 19.0 , Armonk, NY: IBM Corp.

13.

Daza

Novy

Stanley

Averill

(2002) The Depression Anxiety Stress Scale-21: Spanish translation and validation with a Hispanic sample. Journal of Psychopathology and Behavioral Assessment 24: 195–205.

14.

DeSalvo

K. B.

Bloser

Reynolds

Muntner

(2006) Mortality prediction with a single general self-rated health question. A meta-analysis. Journal of General Internal Medicine 21: 267–275.

15.

Donnellan

M. B.

Oswald

F. L.

Baird

B. M.

Lucas

R. E.

(2006) The mini-IPIP scales: Tiny-yet-effective measures of the Big Five factors of personality. Psychological Assessment 18: 192–203.

16.

Eng

Chan

(2013) Depression Anxiety Stress Scale-21 (DASS-21). Spinal Cord Injury Research Evidence. Retrieved from https://www.scireproject.com/outcome-measures-new/depression-anxiety-stress-scale-21-dass-21.

17.

Gloster

A. T.

Rhoades

H. M.

Novy

Klotsche

Senior

Kunik

Stanley

M. A.

(2008) Psychometric properties of the Depression Anxiety and Stress Scale-21 in older primary care patients. Journal of Affective Disorders 110: 248–259.

18.

Groth-Marnat

(2009) Introduction. In: Handbook of psychological assessment, 5th ed. Hoboken, NJ: Wiley, pp. 9–23.

19.

Harris

P. A.

Scott

K. W.

Lebo

Hassan

Lighter

Pulley

(2012) ResearchMatch: A national registry to recruit volunteers for clinical research. Academic Medicine: Journal of the Association of American Medical Colleges 87: 66–73.

20.

Henry

J. D.

Crawford

J. R.

(2005) The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology 44: 227–239.

21.

Hoerger

Chapman

B. P.

Mohile

S. G.

Duberstein

P. R.

(2016) Development and psychometric evaluation of the Decisional Engagement Scale (DES-10): A patient-reported psychosocial survey for quality cancer care. Psychological Assessment 28: 1087–1100.

22.

Hopko

D. R.

Bell

J. L.

Armento

M. E. A.

Robertson

S. M. C.

Hunt

M. K.

Wolf

N. J.

Mullane

C. M.

(2007) The phenomenology and screening of clinical depression in cancer patients. Journal of Psychosocial Oncology 26: 31–51.

23.

L. T.

Bentler

P. M.

(1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6: 1–55.

24.

Jacobsen

P. B.

Andrykowski

M. A.

(2015) Tertiary prevention in cancer care: Understanding and addressing the psychological dimensions of cancer during the active treatment period. American Psychologist 70: 134–145.

25.

Kotov

Gamez

Schmidt

Watson

(2010) Linking “big” personality traits to anxiety, depressive, and substance use disorders: A meta-analysis. Psychological Bulletin 136: 768–821.

26.

Krantz

J. H.

(2014) Psychological research on the net. Retrieved from http://psych.hanover.edu/Research/exponnet.html.

27.

Kroenke

Spitzer

R. L.

Williams

J. B.

(2001) The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine 16: 606–613.

28.

Kroenke

Spitzer

R. L.

Williams

J. B.

Lowe

(2010) The patient health questionnaire somatic, anxiety, and depressive symptom scales: A systematic review. General Hospital Psychiatry 32: 345–359.

29.

Lovibond

P. F.

Lovibond

S. H.

(1995) The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy 33: 335–343.

30.

Meurer

L. N.

Layde

P. M.

Guse

C. E.

(2001) Self-rated health status: A new vital sign for primary care? Wisconsin Medical Journal 100: 35–39.

31.

Muthén

L. K.

Muthén

B. O.

(1998–2011) MPlus user’s guide (6th ed.), Los Angeles, CA: Muthén & Muthén.

32.

National Comprehensive Cancer Network. (2013). NCCN Clinical Practice Guidelines in Oncology: Distress Management (version 3.2013). 2013. Retrieved from www.nccn.org/professionals/physician_gls/f_guidelines.asp.

33.

Trauer

Dodd

Callaly

Campbell

Berk

(2007) The validity of the 21-item version of the Depression Anxiety Stress Scales as a routine clinical outcome measure. Acta Neuropsychiatrica 19: 304–310.

34.

Norton

P. J.

(2007) Depression Anxiety and Stress Scales (DASS-21): Psychometric analysis across four racial groups. Anxiety Stress and Coping 20: 253–265.

35.

Osman

Wong

J. L.

Bagge

C. L.

Freedenthal

Gutierrez

P. M.

Lozano

(2012) The Depression Anxiety Stress Scales-21 (DASS-21): Further examination of dimensions, scale reliability, and correlates. Journal of Clinical Psychology 68: 1322–1338.

36.

Plous

(2014) Online social psychology studies. Retrieved from https://www.socialpsychology.org/expts.htm.

37.

Ronk

F. R.

Korman

J. R.

Hooke

G. R.

Page

A. C.

(2013) Assessing clinical significance of treatment outcomes using the DASS-21. Psychological Assessment 25: 1103–1110.

38.

Satorra, A., & Bentler, P. M. (1998). Scaling corrections for chi-square statistics in covariance structure analysis. Proceedings of the American Statistical Association, 36, 308–313.

39.

Sinclair

S. J.

Siefert

C. J.

Slavin-Mulford

J. M.

Stein

M. B.

Renna

Blais

M. A.

(2012) Psychometric evaluation and normative data for the Depression, Anxiety, and Stress Scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Evaluation & the Health Professions 35: 259–279.

40.

Steiger

J. H.

(1990) Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research 25: 173–180.

41.

Taouk

Lovibond

P. F.

Laube

(2001) Psychometric properties of a Chinese version of the short Depression Anxiety Stress Scales (DASS21). Report for New South Wales Transcultural Mental Health Centre, Sydney, Australia: Cumberland Hospital.

42.

Tonsing

K. N.

(2014) Psychometric properties and validation of Nepali version of the Depression Anxiety Stress Scales (DASS-21). Asian Journal of Psychiatry 8: 63–66.

43.

Tran

T. D.

Tran

Fisher

(2013) Validation of the Depression Anxiety Stress Scales (DASS) 21 as a screening instrument for depression and anxiety in a rural community-based cohort of northern Vietnamese women. BMC Psychiatry 13: 24.

44.

Vandenberg

R. J.

Lance

C. E.

(2000) A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods 3: 4–70.

45.

Vodermaier

Linden

Siu

(2009) Screening for emotional distress in cancer patients: A systematic review of assessment instruments. Journal of the National Cancer Institute 101: 1464–1488.

46.

Waller

N. G.

Compas

B. E.

Hollon

S. D.

Beckjord

(2005) Measurement of depressive symptoms in women with breast cancer and women with clinical depression: A differential item functioning analysis. Journal of Clinical Psychology in Medical Settings 12: 127–141.

47.

Ware

J. E.,

Jr. Sherbourne

C. D.

(1992) The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care 30: 473–483.

48.

Wood

B. M.

Nicholas

M. K.

Blyth

Asghari

Gibson

(2010) The utility of the short version of the Depression Anxiety Stress Scales (DASS-21) in elderly patients with persistent pain: Does age make a difference? Pain Medicine 11: 1780–1790.

49.

Zigmond

A. S.

Snaith

R. P.

(1983) The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica 67: 361–370.