Abstract
This study assessed whether the Center for Epidemiological Studies Depression Scale (CES-D) functions equivalently in assessing depressive symptom severity in lesbian, bisexual, and heterosexual women. Using differential item functioning methods, the authors examined (a) whether there is a bias in CES-D total scores and in individual item scores and (b) whether there are differences across female sexuality groups in the construct of depression assessed by the CES-D. Data were collected anonymously online from 273 women. The CES-D total score was unbiased across groups, but there were biases and construct differences in some individual symptoms. Thus, the findings suggest not only that researchers and clinicians can feel secure in using the CES-D in women of different sexual orientations but also that there may be interesting group differences in the manifestation of particular depressive symptoms.
The American Psychological Association has repeatedly called on researchers to devote more attention to the potentially different ways gay, lesbian, and bisexual (GLB) individuals experience basic psychological processes (Herek & Garnets, 2007). The majority of clinical articles on GLB psychology focus on whether or not nonheterosexuals are at greater risk than heterosexuals for developing psychopathology. GLB individuals have been found, with varying degrees of replication, to be at greater risk than heterosexuals for developing depression (Fergusson, Horwood, & Beautrais, 1999; Hershberger, Pilkington, & D’Augelli, 1997; Lewis, Derlega, Griffin, & Krowinski, 2003). Depression research with female sexual minorities is particularly important given that women with depression outnumber men at least 2:1 (Nolen-Hoeksema, Grayson, & Larson, 1999) and that lesbians are extremely likely to seek therapy at least once in their lives, most often for depression (Cochran, Sullivan, & Mays, 2003; Meyer, 2003; Platzer, 1998).
Before one can compare lesbian, bisexual, and heterosexual women on the prevalence or severity of depression, it first must be established that the scales that are used to measure depression do so equivalently in the different groups. A standard method to address this question is differential item functioning (DIF; see, e.g., Teresi, Ramirez, Lai, & Silver, 2008; Zumbo, 2007). In the context of depression, DIF exists when the relationship between severity of a symptom (item scores) and severity of depression differs across groups, that is, the item score–severity relationship is moderated by the group variable. Although there have been notable recent DIF studies based on race and gender (Ayalon & Young, 2003; Gelin & Zumbo, 2003; Iwata, Turner, & Lloyd, 2002; Yang, Tommet, & Jones, 2009), this issue has not previously been considered for sexual minorities. The purpose of this study was to assess DIF for heterosexual, bisexual, and lesbian women using the Center for Epidemiological Studies Depression Scale (CES-D; Radloff, 1977).
Method
Recruitment and data collection were conducted entirely online and anonymously with ads posted in women’s and GLB group message boards and on Craigslist in eight geographically diverse U.S. cities. Of the 313 women who responded, 273 consented and completed the survey. Ages ranged from 18 to 76 years (M = 31.0, SD = 11.4). The sample was 84% Caucasian, 8% Latina/Hispanic, and 6% African American. Of the participants, 69% had at least a college degree, and 26% had some college. Overall, 45% of the participants identified themselves as heterosexual, 29% as bisexual, and 25% as lesbian. Heterosexuals and lesbians were significantly older and more educated than bisexuals, but there were no group differences in race distribution.
Severity of depressive symptoms was assessed using the CES-D (Radloff, 1977), a 20-item measure with 4-point responses on how frequently over the past week the participant had the experience (e.g., felt sad, from 0 = only rarely or never to 3 = most of the time). The reliability and validity of the CES-D have been demonstrated numerous times (e.g., Bush, Novack, Schneider, & Madan, 2004). In the current study, Cronbach’s alpha for the CES-D was .93.
DIF was investigated using matched moderated regression (MMR; Young, Fogg, & Choi, 2002). MMR is parallel to a factor-analytic DIF model in employing continuous-item responses but with a manifest depression-severity variable. Analyses compared pairs of sexual orientation groups. Each analysis regressed a CES-D item on sexual orientation (coded 0 or 1), CES-D total score, and an interaction term of the product of the two and hierarchically removed nonsignificant parameters until a final model was reached. A significant interaction term represents nonuniform DIF, indicating that the item is more strongly related to overall depression for one group than for another or, alternatively, that the magnitude of the group effect depends on the level of depression. A significant sexual orientation term represents uniform DIF, indicating that sexual orientation–related factors other than depression contribute to the item score, thereby introducing a bias when the item is used to assess depression severity. A significant total score effect indicates a relationship between the item score and depression severity; this, as expected, was observed for all items. For several analyses, the assumption of homogeneity of residual variance across groups was violated and was corrected for by creating and rerunning the weighted regressions (Aguinas, 2004). As described below, the total scale bias was very small, so that purification (i.e., reanalysis with a less biased, recalculated total score) was unnecessary. Based on the MMR procedures, individual participants were matched across groups on total CES-D score (within one point). To maximize the sample sizes, matching and analyses were done for pairs of sexual orientation groups. Consequently, the sample sizes for each comparison were heterosexual and lesbian, n = 130; heterosexual and bisexual, n = 132; and bisexual and lesbian, n = 100. 1
Results
In the heterosexual–lesbian comparison, no items showed nonuniform DIF, and four items showed uniform DIF (Table 1). Even at equal severities of depression, heterosexuals talked less (Item 13) and felt more sad (Item 18), and lesbians enjoyed life less (Item 16) and had more difficulty getting going (Item 20). However, the effect of the sum of these bias effects on the CES-D total score was very small (lesbians 0.05 points higher) because item biases in opposite directions cancelled out.
Results of the Final Differential Item Functioning Regression Models
Note. All shown effects were statistically significant (at least p < .05). For the two heterosexual comparisons, no nonuniform differential item functioning effects were significant, and so these columns are omitted.
In the heterosexual–bisexual comparison, no items showed nonuniform DIF, and two items showed uniform DIF. At equal severities of depression, heterosexuals felt more fearful (Item 10), and bisexuals had more difficulty getting going (Item 20). Again, the effect of these biases on CES-D total score was negligible (heterosexuals 0.09 points higher).
In the bisexual–lesbian comparison, one item (Item 18) showed uniform DIF. At equal severities of depression, bisexuals show greater sadness. In addition, one item (Item 19) showed nonuniform DIF, indicating that the relationship between feeling that people dislike them and depression severity was stronger for bisexuals than for lesbians. This effect also can be interpreted as the magnitude of bias differing as a function of depression severity. Below a CES-D total score of 20, the lesbians had higher scores on this item, and the difference decreased in magnitude as the depression severity increased. In contrast, for total scores greater than 20, the bisexuals had higher item scores, and this difference increased in magnitude as depression severity increased. The total scale bias from these two items ranged across depression severity from 0.34 scale points (lesbians higher) to −1.28 scale points (bisexuals higher).Thus, the effect on the total CES-D score in this comparison was quite small.
Discussion
The goals of this study were to assess whether the CES-D exhibits bias in assessing depression in different sexual orientation groups and whether the relationships between symptoms and depression severity differ in these groups. The results indicate minimal bias in CES-D total score. Thus, the findings support the use of the CES-D in heterosexual, lesbian, and bisexual women. Six individual items exhibited DIF in at least one of the three group comparisons, indicating that these items may be biased as indicators of depression severity. In all but one case, this effect was uniform DIF, indicating that the item score is influenced by factors other than depression that are related to sexual orientation. One item exhibited nonuniform DIF when comparing bisexuals and lesbians. Feeling that others dislike you was a larger component of depression severity in bisexuals than in lesbians. This result also could represent a complex bias pattern, in which the item score is influenced by orientation-related factors other than depression severity but the direction and magnitude of the bias vary as a function of depression severity.
It is important to recognize a number of limitations of this study. A major challenge in all research on DIF is explaining why DIF is occurring. Although there were statistically significant group differences in age and education, these differences were small and unlikely to be the source of the DIF observed. It would be useful for future research to examine variables such as degree of “outness” (Frost & Meyer, 2009), internalized heterosexism (Mays & Cochran, 2001; Szymanski, Chung, & Balsam, 2001), and community acceptance (Proescholdbell, Roosa, & Nemeroff, 2006). This study employed a convenience sample and one that was relatively homogeneous in terms of race (White) and education (high). Thus, it is not known how the results may generalize to other populations with different demographics. It also would be of interest to conduct a similar study in men. Nevertheless, the study also had a number of methodological strengths. Participants were recruited online, a method that provided anonymity and generated a larger sample size than is often seen in GLB research (Riggle, Rostosky, & Reedy, 2005). The study also included bisexual women as a distinct group (Balsam & Mohr, 2007; Diamond, 2008). In addition, the CES-D is a commonly used instrument for which information on DIF and scale equivalence is of practical importance.
In summary, this study provides evidence that the CES-D does function as a valid, unbiased measure of overall depression severity in women self-identified as heterosexual, lesbian, and bisexual. However, there were small biases identified in some individual symptom items that suggest areas for future research on how factors other than depression severity contribute to reports of depressive symptoms. Furthermore, this study demonstrates how DIF studies have the potential to address both psychometric and substantive psychopathology issues.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.