The Center for Epidemiologic Studies Depression Scale

Abstract

Objectives: For the Center for Epidemiologic Studies Depression Scale (CES-D) ratings, the study examined support for a bifactor model, and also the internal consistency reliability and external validity of the factors in this model. Method: Participants (N = 1,178) were older adults from the general community who completed the CES-D. Results: Confirmatory factor analysis of their ratings indicated support for the bifactor model. For this model, the general factor explained most of the covariance in the scores of the CES-D items for Depressed Affect, Somatic Symptoms and Retarded Activity, and Interpersonal Difficulties items. Most of the covariance in the scores of the Positive Affect (PA) scale was explained by its own specific factor. Additional analyses showed support for internal consistencies and external validities of general factors based on all the CES-D items, and when PA items were excluded, and also the PA-specific factor. Discussion: The findings support the use of a total CES-D score without the PA items and also the concurrent use of the PA scale score.

Keywords

Center for Epidemiologic Studies Depression Scale CES-D bifactor model internal consistency external validity older adults

The Center for Epidemiological Studies Depression Scale (CES-D) is a 20-item self-rating questionnaire developed by Radloff (1977) that is used widely in research and clinical settings for screening depressive symptoms. It has been used with various age groups, including older adults (e.g., Ros et al., 2011), adult community samples (e.g., Sheehan, Fifield, Reisine, & Tennen, 1995), and adolescents (e.g., Cheng, Yen, Ko, & Yen, 2012), and in different cultures and cross-cultural studies (e.g., Mackinnon, McCallum, Andrews, & Anderson, 1998). Although the CES-D was initially developed for epidemiological studies (Radloff, 1977) it has also been used in clinical populations (Weissman, Sholomskas, Pottenger, Prusoff, & Locke, 1977) and primary care settings (Thomas & Brantley, 2004).

The CES-D has empirically supported scales for Depressed Affect (DA), (lack of) Positive Affect (PA), Somatic Symptoms and Retarded Activity (SS), and Interpersonal Difficulties (ID). However, the recommended (Radloff, 1977) and common practice for screening depressive symptoms has been to use the total score based on all items, rather than the scores for the different scales (Edwards, Cheavens, Heiy, & Cukrowicz, 2010). If the use of the total score is to have credibility, it would be necessary to demonstrate, at the very least, that most of the covariance of the ratings on the CES-D items can be explained by a single general factor, and for the general factor to have good reliability and external validity. To date these have not been clearly demonstrated. For a multidimensional scale such as the CES-D, the support for a general factor can be evaluated using a bifactor model (an orthogonal factor model, with one general depression factor on which all the CES-D items load, and specific factors for DA, PA, SS, and ID items). For CES-D ratings provided by a group of older adults, the current study examined support for a bifactor model, and also the internal consistency reliability and external validity of the factors in this model.

In the original validation study, based on three independent groups, Radloff (1977) found support for a four-factor oblique model, with factors for DA, PA, SS, and ID (see Figure 1). These factors were derived from principal components analysis. This procedure involves the analysis of the variables in a data set (such as the ratings of the CES-D items) simultaneously to uncover how the items can be grouped together. While exploratory factor analysis (EFA) does the same, it differs from principal components analysis. Principal components analysis establishes the groups or more specifically the components via the linear relationships of the variables, whereas EFA establishes the groups or more specifically the factors in terms of the common variances shared between the variables. Variances that are not shared, called error variances, are not considered when producing the factors. Thus, EFA solutions are based on the common factor model. Confirmatory factor analysis (CFA) is also based on the common factor model. However, unlike EFA which is exploratory, the goal of CFA is to validate a proposed (a priori) factor model. CFA studies of the CES-D have confirmed the original oblique four-factor model (Knight, Williams, McGee, & Olaman, 1997; Sheehan et al., 1995), although a few studies have supported a three-factor model, with factors for combined DA and SS, PA, and ID (e.g., Dick, Beals, Keane, & Manson, 1994). There has been little support for a one-factor model. The four-factor model has also been supported in older adults (Chapleski, Lamphere, Kaczynski, Lichtenberg, & Dwyer, 1997; Foley, Reed, Mutran, & DeVellis, 2002; Hertzog, Van Alstine, Usala, Hultsch, & Dixon, 1990; Ros et al., 2011), the target group in the current study. CFA studies have also shown support for a second-order factor model, with the four-factor structure as primary factors and a single second-order factor of general depression (Hertzog et al., 1990; Knight et al., 1997; Sheehan et al., 1995; see Figure 1).

Figure 1.

CES-D models examined in the study.

For the second-order factor model, McCallum, MacKinnon, Simons, and Simons (1995) used Schmid–Leiman parameterization to compute the loadings of the items on the general factor and the primary (more accurately specific) factors for a group of older adults. The findings indicated that loadings on the DA and SS specific factors were generally low (mostly less than .20), loadings for the PA and ID factors were appreciable (around .50), and loadings on the general factor were high (all ≥0.578). The general factor accounted for 76.43% of the total common variance, whereas the specific factors for DA, PA, SS, and ID accounted for 4.98%, 5.67%, 5.29%, and 7.64%, respectively. These findings can be interpreted in terms of a dominant single general factor for the CES-D that accounts for most of the covariance in the CES-D item scores. However, as this study used only 16 of the 20 CES-D items and only two of the original four response categories, this study did not provide a test of the bifactor model for the standard 20-item CES-D.

Apart from using the Schmid–Leiman parameterization on a second-order factor model, another approach that can be used to evaluate the presence of a dominant general factor is the bifactor model (Reise, Moore, & Haviland, 2010). From a confirmatory perspective, a bifactor model is a latent structure with a general factor, on which all items load, and two or more domain specific factors. The general and specific factors are uncorrelated (orthogonal model). The general factor represents the covariance among all the scale items, whereas the specific factors represent domain specific item response covariance that is not accounted for by the general factor. There are also no cross-loadings of items across the specific factors. Figure 1 shows a conceptual path diagram of the bifactor model as applied to the CES-D. As shown, it is a confirmatory factor model with five orthogonal factors: a general factor on which all items load, and specific factors for the DA, PA, SS, and ID items. The general factor explains the covariance across all the DA, PA, SS, and ID items, whereas the specific factors explain the unique variance of the items within the specific factors.

The bifactor model is different from a conventional second-order factor model. The higher order factor in a second-order factor model represents a superordinate factor explaining the covariance of the primary factors, and not the items, as is the case of the general factor in a bifactor model (Reise et al., 2010). Another important difference is that unlike the second-order factor model, the bifactor model does not impose the proportionality constraint (i.e., for a given set of subtests, the ratios of variance attributable to the respective first-order factor to variance attributable to the general factor are constrained to be the same). Because of this constraint, the relationships of the general and the specific factors in this model with external variables can only be examined if the regression path of the general factor or one of the specific factors to the external variable is fixed at zero (Reise et al., 2010). Thus, the external validity of all factors cannot be examined simultaneously. Consequently, the bifactor model can be considered to be better than the Schmid–Leiman parameterization on the second-order factor for evaluating the factor loadings on the general and specific factor and the external validity of these factors (Reise et al., 2010). If it can be demonstrated that the CES-D general factor accounts for most of the variance for the CES-D ratings, and that it has good internal reliability and external validity, then it could provide good support for using the total CES-D score for research and clinical practice.

For a group of older adults, Yang and Jones (2008) examined the applicability of the bifactor model for a nine-item version of the CES-D, with specific factors for “dysphoria,” “psychosomatic,” and “lack of positive affect.” They found support for the bifactor model, with the loadings for all items being higher on the general factor than the specific factors. In a subsequent study involving the same group, Yang, Tommet, and Jones (2009) examined the applicability of the bifactor model for the 20-item version of the CES-D. Although there were specific factors for DA, PA, SS, and ID, based on EFA solution, one of the original DA items was allocated to the PA specific factor, and another one to the SS specific factor. This study found support for their bifactor model, with the factor loadings for most of the items being higher on the general factor than the specific factors. However, as the items allocated to the specific factors in this study were slightly different from how they would be allocated in terms of the original factor structure (Radloff, 1977), and as the study did not examine the internal reliability and external validity of the general and specific factors, the findings in the study can be seen as limited. There is clearly a need for more studies in this area.

In the current study, we examined and compared the fit of several CES-D models, using all 20 item ratings provided by older adults from the general community. Initially, CFA was used to examine the fit for the bifactor model. In this model there were five orthogonal factors: a general factor on which all items loaded, and specific factors for DA, PA, SS, and ID. The items for the specific factors were identical to those in the original four-factor model proposed by Radloff (1977). We compared the fit of this model with a one-factor model (all 20 items loading on a single factor), the four-factor model proposed by Radloff (1977), and the second-order factor model, with four lower order factors loading on a single higher order factor. The lower order factors were DA, PA, SS, and ID and the items for these factors were again identical to those in the original four-factor model proposed by Radloff (1977). The four models tested are depicted in Figure 1. We then examined the salience of the factor loadings, the amount of common variance explained, and the internal consistency reliability values of the general and specific factors in the bifactor model. Additionally, we examined the external validity of the factors of the bifactor model in terms of their associations with independent measures of depression and suicidal ideation. In this respect, there are data showing that CES-D total scores correlate well with clinical ratings of depression (Roberts & Vernon, 1983), and with suicidal ideation (McLaren, Gomez, Bailey, & Van der Horst, 2007). Based on past findings reported by Yang et al. (2009) and McCallum et al. (1995), we expected support for the bifactor model, with higher loadings, variance explained, and reliability value for the general factor than the specific factors. We also expected the general factor to have stronger associations with depression and suicidal ideation than the specific factors, with the association being higher for depression than for suicidal ideation (since depression is more aligned to the general factor than is suicidal ideation).

Method

Participants

A convenience sample of 1,178 older adults aged 65 years and older participated in the study. Apart from age, there was no other exclusionary criterion for participation. The sample consisted of 573 men aged from 65 to 96 years (M = 74.98, standard deviation [SD] = 7.48) and 605 women aged from 65 to 98 years (M = 75.01, SD = 7.65). The men and women did not differ significantly on age, t(1176) = −0.06, p > .05. The majority of men were married (n = 313, 54.6%), with the other men being widowered (n = 149, 26.0%), divorced (n = 56, 9.8%), or single (n = 55, 9.6%). The majority of men had completed secondary school or equivalent (n = 401, 70.0%), with the remaining having completed primary school (n = 87, 15.2%), or a university degree (n = 85, 14.8%). Almost half of the women were married (n = 284, 46.9%), with the remaining women being widowed (n = 180, 29.8%), divorced (n = 95, 15.7%), or single (n = 7.6%). Most of the women had completed secondary school or equivalent (n = 453, 74.9%), with the others having completed primary school (n = 73, 12.1%), or a university degree (n = 79, 13.0%). There was a significant association between gender and marital status, χ²(3, N = 1168) = 14.35, p = .002, with more men being married (n = 313; Haberman’s standardized adjusted residuals statistic or HAR = 2.6, p < .01), and more women being divorced (n = 95; HAR = 3.0, p < .01) than that expected by chance (n for married = 597, and n for divorced = 151). There was no relationship between gender and highest level of education, χ²(2) = 3.74, p > .05. Men (M = 11.10, SD = 10.31) and women (M = 10.54, SD = 9.44) scored similarly on the CES-D, t(1166) = 0.98, p > .05. Based on a total scale score of 16 or more that is generally used to identify older adults with a higher probability of clinical depression (Lewinsohn, Seeley, Roberts, & Allen, 1997), there was no relationship between gender and percentage scoring at a clinical level, χ²(2) = 3.74, p > .05. For the current sample, 24.3% of men and 22.7% of women had a CES-D total score of 16 or greater. Apart from this information, we had no other information on the mental health of participants.

Materials

The Center for Epidemiologic Studies Depression Scale

As noted in the introduction, the CES-D (Radloff, 1977) is a 20-item self-report measure which assesses the number of depressive symptoms that a person has experienced (e.g., I felt that I could not shake off the blues even with help from my family). This measure comprises scales for DA (seven items; Items 3, 6, 9, 10, 14, 17, and 18), PA (four items; Items, 4, 8, 12, and 16), SS (seven items; Items 1, 2, 5, 7, 11, 13, and 20), and ID (two items; Items 15 and 19). Participants rated each item on a 4-point scale that represented the number of days the participant experienced the emotion or thought in the statement (0 = less than 1 day, 1 = 1-2 days, 2 = 2-4 days, 3 = 5-7 days) over the preceding 7 days. The four PA items are reverse scored. Higher scores indicate more severe depressive symptoms. In terms of internal consistency, past studies have shown that the alpha coefficient of the full CES-D scale is high, with values usually above .85 (e.g., Knight et al., 1997; Radloff, 1977; Ros et al., 2011). Knight et al. reported values of .87, .78, .75, and .44 for DA, PA, SS, and ID, respectively. Cronbach’s alpha internal consistency values for DA, PA, SS, and ID were .87, .80, .80, and .59, respectively, in the current sample.

Suicide Subscale in the General Health Questionnaire

The Suicide Subscale of the General Health Questionnaire (GHQ; Goldberg & Hillier, 1979) has seven items. Although all seven items together are intended to measure suicidal ideation, three of the items (Items 1, 2, and 4) tap depression symptoms, and the other four items (Items 3, 5, 6, and 7) tap suicidal ideation (Watson, Goldney, Fisher, & Merritt, 2001). The three depression items and the four suicidal ideation items were used in the current study as indicators to model latent factors for depression and suicidal ideation, respectively. To avoid confusion with the factors in the CES-D, these factors are referred to henceforth as GHQ–Depression and GHQ–Suicidal Ideation. Consistent with previous research (Goldney, Winefield, Tiggemann, Winefield, & Smith, 1989; Hamilton & Schweitzer, 2000), the responses for items were scored from 0 to 3, with higher scores indicative of depression or suicidal ideation. Cronbach’s alpha internal consistency values for GHQ–Depression and GHQ–Suicidal Ideation were .66 and .76, respectively, for the current sample.

Procedure

Ethics approval was obtained from Federation University’s Human Research Ethics Committee. The sample was recruited for a variety of smaller studies, each assessing depressive experiences in older adults as one of the aims. All participants were from the general community, and they were recruited through several sources from the State of Victoria, Australia. Participants were recruited in person. At various venues (e.g., assisted living facilities, social and sporting clubs), one of the researchers gave a brief presentation about the study. They explained the procedure, and interested individuals were given an envelope with the questionnaires and a participant information letter. The questionnaires were completed by participants at the time and returned to the researcher. A total of 1,400 questionnaires were distributed, with 1,178 usable questionnaires being returned, indicating a response rate of 84%.

Analytical Procedure

Initially, the normality of the CES-D items was examined using PRELIS 2.51 (Joreskög & Sorböm, 1996). Table 1 shows the mean and standard deviation (SD) scores, and the skewness and kurtosis values of the items. All items showed significant skewness and kurtosis (p < .001). There was also significant multivariate skewness (110.665, z = 117.348, p < .001) and kurtosis (796.316, z = 51.020, p < .001). Given this, the study used maximum likelihood with robust estimation to ascertain statistical fit. This procedure, referred to as MLMVχ² in Mplus, tests the closeness of fit between the unrestricted sample covariance matrix and the restricted (model) covariance matrix, correcting for the lack of normality.

Table 1.

Mean and Standard Deviation, and Skewness, and Kurtosis of the CES-D Items.

Item No. Brief description	M	SD	Skewness	Kurtosis
Depressed Affect
3. Could not shake blues	0.318	0.728	2.482	5.506
6. I felt depressed	0.420	0.802	2.066	3.792
9. My life had been failure	0.236	0.662	3.114	9.223
10. I felt fearful	0.259	0.652	2.842	7.854
14. I felt lonely	0.504	0.903	1.693	1.644
17. I had crying spells	0.207	0.598	3.204	10.101
18. I felt sad	0.431	0.799	1.921	2.887
Positive Affect
4. Just as good as others	1.126	1.231	0.519	−1.376
8. Hopeful about future	1.124	1.190	0.511	−1.303
12. I was happy	0.916	1.075	0.815	−0.705
16. I enjoyed life	0.908	1.105	0.828	−0.774
Somatic Symptoms
1. Bothered by things	0.401	0.743	1.975	3.363
2. Poor appetite	0.363	0.770	2.196	3.952
5. Keeping mind on task	0.601	0.880	1.405	1.054
7. Everything was effort	0.626	0.915	1.385	1.045
11. My sleep was restless	0.873	1.018	0.871	−0.468
13. Talked less than usual	0.435	0.809	1.916	2.824
20. I could not get going	0.603	0.893	1.380	0.892
Interpersonal Distress
15. People unfriendly	0.259	0.666	2.840	7.625
19. Felt people disliked me	0.205	0.587	3.311	11.164

Note. CES-D = Center for Epidemiologic Studies Depression Scale. All skewness and kurtosis were significant (p < .001).

All the CFA models in the study were conducted using Mplus (Version 7) software (Muthén & Muthén, 2013). As all types of χ² values, including MLMVχ², are inflated by large sample sizes, the fit of the models was also examined using two commonly used practical fit indices: the root mean square error of approximation (RMSEA) and the comparative fit index (CFI). The guidelines suggested by Hu and Bentler (1998) are that RMSEA values of .06 or less be taken as good fit, values >.060 to .08 be considered moderate fit, values >.08 to .10 be considered marginal fit, and values >.10 be considered poor fit. For the CFI, values of .95 or higher are taken as indicating good model–data fit, values of .90 and <.95 are taken as acceptable fit, and values <.90 as poor fit. It is to be noted, however, that the appropriateness of these “benchmarks” has yet to be established for bifactor analyses (West, Taylor, & Wu, 2012). To determine statistical differences between models, the difference in MLMVχ² values was used. As this difference is not distributed as a chi-square, it is necessary to adjust for this difference. This was done using the scaling correction formula for MLMV on the Mplus website.

The internal consistency reliability of the general and specific factors was computed using McDonald’s omega hierarchical (ωH). Zinbarg, Revelle, Yovel, and Li (2005) have recommended this method for a scale that has a hierarchical factor structure, such as a bifactor structure. For a bifactor model, the ωH of the general factor is model-based reliability computed in terms of the proportion of variance for the general factor to the total observed variance for the scale. For a specific factor, the ωH is model-based reliability computed in terms of the proportion of variance for the relevant specific factor to the total observed variance for the subscale (Brunner, Nagy, & Wilhelm, 2012). According to Zinbarg et al. (2005), if the value of ωH for the general factor is high, then there is justification for summing the responses of the items in the scale. The converse of this argument is that if the value of ωH for a specific factor is high, then there is justification in using its scale score, independent of the support for a general factor.

In order to examine the association of GHQ–Depression and GHQ–Suicidal Ideation with the factors in the bifactor model, the bifactor model was extended to include the latent factors for GHQ–Depression and GHQ–Suicidal Ideation, and these factors were correlated with the general and specific factors. The GHQ–Depression and GHQ–Suicidal Ideation factors were correlated. According to Hemphill (2003), coefficients of <.2 are small effects, .2 to .3 are medium effects, and >.30 are large effects. These guidelines were used in the study to interpret the effect sizes of the correlations in these models.

Results

Fit of the Bifactor Model and Comparisons With Other Models

Table 2 shows the results of all the CFA models tested.¹ For the bifactor model, the factor loadings of the two items on the ID scale were set equal, and Items 17 and 10 of the DA scale were also constrained to be equal. Without these modifications the bifactor model showed nonidentification and inadmissible solution. Based on guidelines proposed by Hu and Bentler (1998), the goodness-of-fit values (RMSEA and CFI) for the one-factor model indicated poor fit. For the four-factor model, the second-order factor model, and the bifactor model, there was good fit in terms of their RMSEA values, and acceptable fit in terms of their CFI values. For the four-factor model, the correlations involving PA with DA, SS, and ID were .36, .31, and .32, respectively, whereas the correlations among the other factors ranged from .90 to .70. For the second-order factor model, the loadings for DA, PA, SS, and ID on the general factor were .99, .36, .90, and .77, respectively. Table 2 also shows that there was no significant difference in fit between the four-factor model and the second-order factor model, and both these models showed better fit than the one-factor model. The bifactor model had better fit than all the other models tested. Overall, therefore, the findings indicated that of the four models tested, the bifactor model was the optimum structural model to represent ratings on the CES-D.

Table 2.

Fit of the Factor Models of the CES-D.

Model (M)	MLMVχ²	df	RMSEA [90% CI]	CFI	Models compared
Model (M)	MLMVχ²	df	RMSEA [90% CI]	CFI	ΔM	Δdf	ΔMLMVχ²
One-factor (O)	1095.150***	170	.068 [.064, .072]	.766	—
Four-factor (F)	449.458***	164	.039 [.034, .043]	.928	O − F	6	169.333***
Second-order factor (S)	452.443***	166	.038 [.034, .043]	.927	O − S	4	119.333***
					F − S	2	2.017
Bifactor (B)	394.6274***	152	.037 [.032, .041]	.939	O − B	18	329.204***
					F − B	12	47.048***
					S − B	14	51.826***

Note. CES-D = Center for Epidemiologic Studies Depression Scale; CFI = comparative fit index; CI = confidence interval; RMSEA = root mean square error of approximation.

***

p < .001.

Salience of Factor Loadings of the General and Specific Factors of the Bifactor Model

Table 3 presents the factor loadings and the decomposed variance estimates for the bifactor model. In an absolute sense, with the exception of the four PA items, for all other items the loadings on the general factor were relatively higher than the loadings on the specific factors. As shown in the table, for the general factor, three of the four items belonging to the PA factor had nonsalient loadings, and all other items had salient loadings (based on Thurstone’s recommendation that values ≥ .30 are salient). For the specific factors, salient loadings were found for two (out of seven) DA items, three (out of four) PA items, one (out of seven) SS item, and both the ID items.

Table 3.

Completely Standardized Factor Loadings and Sources of Variance of the CES-D in the Bifactor Model.

Item No. brief description	Bifactor
	G		DA		PA		SS		ID		h ²	u ²
	λ	Variance	λ	Variance	λ	Variance	λ	Variance	λ	Variance	h ²	u ²
3. Could not shake blues	.756	.572	.039	.002							.627	.373
6. I felt depressed	.784	.615	.112	.013							.366	.634
9. My life had been failure	.599	.359	−.085	.007							.399	.601
10. I felt fearful	.627	.393	.078	.006							.458	.542
14. I felt lonely	.650	.423	.188	.035							.527	.473
17. I had crying spells	.595	.354	.416	.173							.752	.248
18. I felt sad	.761	.579	.416	.173							.276	.724
4. Just as good as others	.142	.020			.506	.256					.467	.533
8. Hopeful about future	.232	.054			.643	.413					.610	.390
12. I was happy	.315	.099			.715	.511					.707	.293
16. I enjoyed life	.286	.082			.791	.626					.396	.604
1. Bothered by things	.628	.394					.035	.001			.438	.562
2. Poor appetite	.635	.403					.186	.035			.427	.573
5. Keeping mind on task	.643	.413					.118	.014			.571	.429
7. Everything was effort	.644	.415					.395	.156			.209	.791
11. My sleep was restless	.433	.187					.147	.022			.269	.731
13. Talked less than usual	.517	.267					−.043	.002			.615	.385
20. I could not get going	.627	.393					.471	.222			.356	.644
15. People unfriendly	.432	.187							.411	.169	.513	.487
19. Felt people disliked me	.587	.345							.411	.169	.627	.373
Variance explained
Percent total variance	32.77		2.04		9.03		2.26		1.69		47.8	52.2
Percent common variance	68.57		4.27		18.89		4.72		3.53

Note. CES-D = Center for Epidemiologic Studies Depression Scale; G = General; DA = Depressed Affect; PA = Positive Affect; SS = Somatic Symptoms; ID = Interpersonal Distress. For each item the residual is 1 − (variance for the item on the general factor + variance for the item on the specific factor). As an example, the residual for Item 3 is 1 − (.572 + .002) = .426.

Variance Explained by the General and Specific Factors of the Bifactor Model

Table 3 also shows that the specific factors for DA, PA, SS, and ID accounted for 4.27%, 18.89%, 4.72%, and 3.53%, respectively, of the common variance. Thus, together, the specific factors accounted for 31.41% of the common variance, and the general factor accounted for the remaining 68.57% of the common variance. The PA specific factor accounted for 60.14% of the total common variance accounted for by all the specific factors.

Internal Consistency Reliability Values of the General and Specific Factors

The ωH value for the general factor was .85. The ωH values for DA, PA, SS, and ID were .049, .708, .075, and .225, respectively. Thus, the value for the general factor was high, the value for PA was moderately high, and the values for DA, SS, and ID were low, based on values between .70 and .79 being moderate, and values of .80 or more as being high (Clark & Watson, 1995).

Associations of Depression and Suicidal Ideation With the Factors in the Bifactor Models

Table 4 shows the correlations of GHQ–Depression and GHQ–Suicidal Ideation latent factors with the factors in the bifactor model. As shown, the general factor and the PA specific factor were significantly and positively associated with GHQ–Depression and GHQ–Suicidal Ideation, with large and medium effect sizes, respectively. None of the other correlations were significant. Using Steiger’s (1980) z-to-r transformation formula for comparing correlations measured on the same participants, the correlation for the general factor with GHQ–Depression was significantly higher than the correlation for the general factor with GHQ–Suicidal Ideation, z = 3.74, p < .001. The correlation for the PA specific factor with GHQ–Depression did not differ significantly from that for the PA specific factor with GHQ–Suicidal Ideation, z = 1.14, ns.

Table 4.

Correlations of the Factors in the Bifactor Model With the GHQ–Depression and GHQ–Suicide Ideation.

Factors	GHQ–Depression	GHQ–Suicidal Ideation
General	.579***	.471***
Specific Depressed Affect	.012	.053
Specific Positive Affect	.242***	.201***
Specific Somatic Symptoms	.058	.046
Specific Interpersonal Distress	.045	.017

Note. CES-D = Center for Epidemiologic Studies Depression Scale; GHQ = General Health Questionnaire.

***

p < .001.

Additional Analyses Based on Previous Results

For the PA items we found mostly nonsalient loadings on the general factor and salient loadings on its specific factor, relatively high variance accounted by the PA items to the PA specific factor, moderately high ωH value for the PA specific factor, and significant associations of the PA specific factor with GHQ–Depression and GHQ–Suicidal Ideation. Therefore, we examined the support for a modified bifactor model in which the PA items were not part of the general factor, and independent of the DA, SS, and ID specific factors. The goodness-of-fit values for the modified bifactor model were MLMVχ²(df = 156) = 454.32, p < .001; RMSEA = .040 (90% CI = .036-.045); CFI = .92. Although this model showed reduced statistical fit compared with the standard (original) bifactor model, ΔMLMVχ²(Δdf = 4) = 41.60, p < .001, like the standard bifactor model, its RMSEA value indicated good fit, and its CFI value indicated acceptable fit, thereby indicating somewhat comparable qualitative fit across them. Like the general factor in the standard bifactor model, the ωH value for the general factor in the modified bifactor model was high, at .89. Also, the general factor in the modified bifactor model was significantly and positively associated with GHQ–Depression (r = .40) and GHQ–Suicidal Ideation (r = .30), with large effect sizes.

Discussion

Initial analysis indicated good support for the original CES-D bifactor model, with this model showing better support than the four-factor model, the higher order factor model, and the one-factor model. The four-factor model and the second-order factor model showed mixed fit, with the RMSEA showing good fit and the CFI showing adequate fit. The one-factor model showed poor fit. Thus, there was some support for the four-factor model and the second-order factor model, but not the one-factor model.

The fit findings for the four-factor model, the second-order factor model, and the one-factor model are consistent with existing data on the CES-D (Knight et al., 1997; Sheehan et al., 1995), including data involving older adults (Chapleski et al., 1997; Foley et al., 2002; Hertzog et al., 1990; Ros et al., 2011). The support of the bifactor model in the current study is also consistent with that reported by McCallum et al. (1995), which used Schmid–Leiman parameterization on a second-order factor model to examine a bifactor conceptualization of the CES-D. However, unlike the current study that used all 20 items in the CES-D and the original four response categories, the study by McCallum et al. used only 16 items and only two response categories. The support here for the bifactor model can also be interpreted as consistent with the study by Yang et al. (2009). Indeed for a slightly modified bifactor model (in which one of the original DA items was allocated to the PA specific factor, and another one to the SS specific factor), they reported comparable fit (RMESA of .025, compared with .037 in the current study). However, as the current study provided an evaluation of the bifactor model of the standard CES-D items, response categories, and items-to-factors structure, the findings for the bifactor model in the current study extend existing data in this area.

The findings for the original bifactor model in the study showed that with the exception of three PA items, all other items had salient loadings on the general factor. With the exception of the four PA items, for all other items the loadings on the general factor were relatively higher than the loadings on the specific factor. For the specific factors, salient loadings were found for mainly PA and ID items. Overall, the general factor accounted for slightly more than twice the common variance of all the specific factors together. The specific factors for DA, PA, SS, and ID accounted for 4.27%, 18.89%, 4.72%, and 3.53%, respectively, of the common variance. Somewhat comparable with these findings, the study by McCallum et al. (1995) found that the general factor accounted for 76.43% of the total common variance, while the specific factors for DA, PA, SS, and ID accounted for 4.98%, 5.67%, 5.29%, and 7.64%, respectively. Taken together, these findings indicate that the general factor is relatively more dominant than the specific factors. While the general factor explains most of the covariance in the scores of the CES-D items, the findings in the current study showed that this is especially so for the DA, SS, and ID items.

The findings of the current study also showed that for the original bifactor model, the internal consistency reliability value (based on ωH) for the general factor was .848 and the internal consistency reliability values were .049, .708, .075, and .225 for DA, PA, SS, and ID, respectively. The value for the general factor was high, the value for PA was moderately high, and the values for DA, SS, and ID were low, based on values between .70 and .79 being moderate, and values of .80 or more as being high (Clark & Watson, 1995). Thus, there was weak support for the internal consistency reliabilities for DA, SS, and ID. In contrast, there was acceptable support for the internal consistency reliabilities for the general factor and the specific factor for PA.

The current study also examined the external validity of the factors of the original bifactor model in terms of their associations with GHQ–Depression and GHQ–Suicidal Ideation. The findings showed that the general factor and PA were significantly and positively associated with GHQ–Depression and GHQ–Suicidal Ideation, with large and medium effect sizes, respectively. None of the other correlations were significant. Also, the correlation for the general factor with GHQ–Depression was statistically significantly higher than the correlation for the general factor with GHQ–Suicidal Ideation. Taken together, the correlation findings can be interpreted as supportive of the external validity of the general factor and bifactor model. Also, the finding that the PA specific factor was associated with GHQ–Depression and GHQ–Suicidal Ideation can be taken as supporting the external validity of this factor.

The lack of support for the internal consistency reliabilities of the DA-, SS-, and ID-specific factors indicates that there is no justification for using their scale scores, as the general factor captures most of the variance in these scales. In contrast, the support for the internal consistency reliabilities for the general factor and the PA specific factor raises the possibility that the total and PA scale scores could be used. However, as the differential meaning of a general factor that includes the PA items from a PA factor with the same set of PA items as in the general factor score is unclear, this may not be prudent. Our findings showed that although the fit of a modified bifactor model in which the PA items were not part of the general factor and were independent of the DA, SS, and ID specific factors, was less than the standard bifactor model, the difference in fit between the modified and the standard model was low. Also, the ωH value for the general factor in the modified bifactor model was high, and it was significantly and positively associated with GHQ–Depression and GHQ–Suicidal Ideation, with large effect sizes. Thus, the general factor in the modified bifactor model had comparable support for its internal consistency reliability and external validity. Given this, it can be argued that despite the fact that the modified general factor had slightly lower fit than the standard general bifactor model, from a practical viewpoint, it would be simpler to use a total score without including the scores of the PA items (i.e., sum of DA, SS, and ID scores), and the PA scale score. We recommend this method of scoring the CES-D. It is to be noted in this respect, that our scoring preference is consistent with that proposed by other researchers (Edwards et al., 2010; Schroevers, Sanderman, van Sonderen, & Ranchor, 2000).

In summary, the findings in this study supported the original bifactor model for ratings of the CES-D. For this model, the general factor was relatively more dominant than the specific factors. The general factor explained most of the covariance in the scores of the CES-D items for DA, SS, and ID items. Most of the covariance in the scores of the PA scale was explained by the PA specific factor rather than the general factor. The findings also supported a modified bifactor model in which the PA items were not part of the general factor, and independent of the DA, SS, and ID specific factors Taken together, these findings were interpreted as indicating that the CES-D is best scored in terms of a total score using only the DA, SS, and ID items, and also a score for PA using the four PA items. Thus, our scoring recommendation is inconsistent with the use of the total score based on all the DA, SS, ID, and PA items, as recommended (Radloff, 1977) and used generally for screening depressive symptoms (Edwards et al., 2010). The use of a total score based on only the DA, SS, and ID items is, however, consistent with that suggested by some researchers (Edwards et al., 2010; Schroevers et al., 2000).

In concluding, the findings and interpretations made in this study have limitations. First, although we had an 84% return rate, we had no information on the 16% who did not respond. Thus, we do not know how the missing data from these individuals affect our findings. Second, because we examined older adults, the findings here could be unique to this age group, and not applicable to other age groups. Related to this limitation is that we used a convenience sample. Third, even for older adults, as the sample involved a normal community group of older adults, the findings may not be applicable to other groups of older adults, such as those with mental health or cognitive problems. Indeed past studies have shown that the CES-D is valid and reliable for identifying clinical levels of depression in clinical groups who are at high risk for developing major depression (Hann, Winter, & Jacobsen, 1999; Pandya, Metz, & Patten, 2005). Fourth, it is possible that factors such as socioeconomic, ethnicity, and marital status could influence CES-D ratings (Yang et al., 2009). The failure to control for these effects in the study could have confounded the results. Fifth, the findings reported here are based on a single study. As a consequence, there is a need for cross-validation of the findings before the findings can be generalized. Sixth, for the bifactor model, the factor loadings of both items on the ID scale, and Items 17 and 10 of the DA scale, were constrained to be equal. Without these modifications, the bifactor model showed nonidentification and inadmissible solution. It is uncertain if the findings were confounded by these modifications. Although we have highlighted a number of limitations, we believe that the findings in the current study add to the literature on the bifactor model of the CES-D, and the adequacy of using the total CES-D score for research and clinical practice, at least with older adults. It would be useful if more studies were conducted in this area, taking into consideration the limitations highlighted here.

Footnotes

Acknowledgements

The authors wish to thank Angelina Crea, Mitchell Hobbs, Simon Morris, Karyn Newnham, Jayne Turner, and Kellie Wilson for collecting the data.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Brunner

Nagy

Wilhelm

(2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796-846. doi:10.1111/j.1467-6494.2011.00749.x

Chapleski

E. E.

Lamphere

J. K.

Kaczynski

Lichtenberg

P. A.

Dwyer

J. W.

(1997). Structure of a depression measure among American Indian elders: Confirmatory factor analysis of the CES-D Scale. Research on Aging, 19, 462-485. doi:10.1177/0164027597194004

Cheng

C. P.

Yen

C. F.

C. H.

Yen

J. Y.

(2012). Factor structure of the Center for Epidemiologic Studies Depression Scale in Taiwanese adolescents. Comprehensive Psychiatry, 53, 299-307. doi:10.1016/j.comppsych.2011.04.056

Clark

L. A.

Watson

(1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319. doi:10.1037/1040-3590.7.3.309

Dick

R. W.

Beals

Keane

E. M.

Manson

S. M.

(1994). Factorial structure of the CES-D among American Indian adolescents. Journal of Adolescence, 17, 73-79. doi:10.1006/jado.1994.1007

Edwards

M. C.

Cheavens

J. S.

Heiy

J. E.

Cukrowicz

K. C.

(2010). A re-examination of the factor structure of the CES-D: Is a 1-factor model plausible? Psychological Assessment, 22, 711-715. doi:10.1037/a0019917

Foley

K. L.

Reed

P. S.

Mutran

E. J.

DeVellis

R. F.

(2002). Measurement adequacy of the CES-D among a sample of older African-Americans. Psychiatry Research, 109, 61-69. doi:10.1016/S0165-1781(01)00360-2

Goldberg

D. P.

Hillier

V. F.

(1979). A scaled version of the General Health Questionnaire. Psychological Medicine, 9, 139-145. doi:10.1017/S0033291700021644

Goldney

R. D.

Winefield

A. H.

Tiggemann

Winefield

H. R.

Smith

(1989). Suicidal ideation in a young adult population. Acta Psychiatrica Scandinavica, 79, 481-489. doi:10.1111/j.1600-0447.1989.tb10291.x

10.

Hamilton

T. R.

Schweitzer

R. D.

(2000). The cost of being perfect: Perfectionism and suicide ideation in university students. Australian and New Zealand Journal of Psychiatry, 34, 829-835. doi:10.1080/j.1440-1614.2000.00801.x

11.

Hann

Winter

Jacobsen

(1999). Measurement of depressive symptoms in cancer patients: Evaluation of the CES-D. Journal of Psychosomatic Research, 46, 437-443. doi:10.1016/S0022-3999(99)00004-5

12.

Hemphill

J. F.

(2003). Interpreting the magnitudes of correlation coefficients. American Psychologist, 58, 78-79. doi:10.1037/0003-066X.58.1.78

13.

Hertzog

Van Alstine

Usala

P. D.

Hultsch

D. F.

Dixon

(1990). Measurement properties of the Center for the Epidemiologic Studies Depression Scale (CES-D) in older populations. Psychological Assessment, 1, 64-72. doi:10.1037/1040-3590.2.1.64

14.

L. T.

Bentler

P. M.

(1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453. doi:10.1037/1082-989X.3.4.424

15.

Joreskög

K. G.

Sorböm

(1996). LISREL 8: User’s reference guide. Chicago, IL: Scientific Software.

16.

Knight

R. G.

Williams

McGee

Olaman

(1997). Psychometric properties of the Center for Epidemiologic Studies Depression Scale (CES-D) in a sample of women in middle life. Behaviour Research and Therapy, 35, 373-380. doi:10.1016/S0005-7967(96)00107-6

17.

Lewinsohn

P. M.

Seeley

J. R.

Roberts

R. E.

Allen

N. B.

(1997). Center for the Epidemiologic Studies Depression Scale (CES-D) as a screening instrument for depression among community-residing older adults. Psychology and Aging, 12, 277-287. doi:10.1037/0882-7974.12.2.277

18.

Mackinnon

McCallum

Andrews

Anderson

(1998). The Center for Epidemiological Studies Depression Scale in older community samples in Indonesia, North Korea, Myanmar, Sri Lanka, and Thailand. Journals of Gerontology Series B, Psychological Sciences and Social Sciences, 53, P343-P352.

19.

McCallum

MacKinnon

Simons

(1995). Measurement properties of the Center of Epidemiological Studies Depression Scale: An Australian community study of aged persons. Journal of Gerontology: Social Sciences, 50B, S182-S189. doi:10.1093/geronb/50B.3.S182

20.

McLaren

Gomez

Bailey

Van Der Horst

R. K.

(2007). The association of depression and sense of belonging with suicidal ideation among older adults: Applicability of resiliency models. Suicide and Life-Threatening Behavior, 37, 89-102. doi:10.1521/suli.2007.37.1.89

21.

Muthén

L. K.

Muthén

B. O.

(2007). Mplus user’s guide (5th ed.). Los Angeles, CA: Muthén & Muthén.

22.

Pandya

Metz

Patten

S. B.

(2005). Predictive value of the CES-D in detecting depression among candidates for disease-modifying multiple sclerosis treatment. Psychosomatics, 46, 131-134. doi:10.1176/appi.psy.46.2.131

23.

Radloff

L. S.

(1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385-401. doi:10.1177/014662167700100306

24.

Reise

S. P.

Moore

T. M.

Haviland

M. G.

(2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559. doi:10.1080/00223-891.2010.496477

25.

Roberts

R. E.

Vernon

S. W.

(1983). The Center for Epidemiologic Studies Depression Scale: Its use in a community sample. American Journal of Psychiatry, 140, 41-46.

26.

Ros

Latorre

J. M.

Aguilar

M. J.

Serrano

J. P.

Navarro

Ricarte

J. J.

(2011). Factor structure and psychometric properties of the Center for Epidemiologic Studies Depression Scale (CES-D) in older populations with and without cognitive impairment. International Journal of Aging and Human Development, 72, 83-110. doi:10.2190/AG.72.2.a

27.

Schroevers

M. J.

Sanderman

van Sonderen

Ranchor

A. V.

(2000). The evaluation of the Center for Epidemiologic Studies Depression (CES-D) Scale: Depressed and positive affect in cancer patients and healthy reference subjects. Quality of Life Research, 9, 1015-1029. doi:10.1023/A:1016673003237

28.

Sheehan

T. J.

Fifield

Reisine

Tennen

(1995). The measurement structure of the Center for Epidemiologic Studies Depression Scale. Journal of Personality Assessment, 64, 507-521. doi:10.1207/s15327752jpa6403_9

29.

Steiger

J. H.

(1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245-251. doi:10.1037/0033-2909.87.2.245

30.

Thomas

J. L.

Brantley

P. J.

(2004). Factor structure of the Centerfor Epidemiologic Studies Depression Scale in low-income women attending primary care clinics. European Journal of Psychological Assessment, 20, 106-115. doi:10.1027/1015-5759.20.2.106

31.

Watson

Goldney

Fisher

Merritt

(2001). The measurement of suicidal ideation. Crisis, 22, 12-14. doi:10.1027//0227-5910.22.1.12

32.

Weissman

Sholomskas

Pottenger

Prusoff

Locke

(1977). Assessing depressive symptoms in five psychiatric populations: A validation study. American Journal of Epidemiology, 106, 203-214.

33.

West

S. G.

Taylor

A. B.

(2012). Model fit and model selection in structural equation modelling. In Hoyle

R. H.

(Ed.), Handbook of structural equation modeling (pp. 209-231). New York, NY: Guilford Press.

34.

Yang

F. M.

Jones

R. N.

(2008). Measurement differences in depression: Chronic health-related and sociodemographic effects in older Americans. Psychosomatic Medicine, 70, 993-1004. doi:10.1097/PSY.0b013e31818ce4fa

35.

Yang

F. M.

Tommet

Jones

R. N.

(2009). Disparities in self-reported geriatric depressive symptoms due to sociodemographic differences: An extension of the bi-factor item response theory model for use in differential item functioning. Journal of Psychiatric Research, 43, 1025-1035. doi:10.1016/j.jpsychires.2008.12.007

36.

Zinbarg

R. E.

Revelle

Yovel

(2005). Cronbach’s α, Revelle’s β, and McDonald’s ωH: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123-133. doi:10.1007/s11336-003-0974-7