Abstract
This study examined the properties of the Beck Depression Inventory-II (BDI-II) in a sample of 111 patients with chronic fatigue syndrome (CFS). Exploratory factor analysis identified two factors. The mean score for the Somatic-Affective factor was significantly higher than the Cognitive factor. Convergent and discriminant validity were assessed for BDI-II total score, the two factor scores, and the BDI for Primary Care (BDI-PC). The BDI-PC and Cognitive factor demonstrated superior validity. Results suggest patients endorse BDI-II somatic items that overlap with CFS symptoms at a high rate. Factor scores should be evaluated separately, or the BDI-PC should be utilized with this population.
Keywords
Introduction
High rates of depression occur among patients with chronic fatigue syndrome (CFS), with one review reporting a 50–75% lifetime history of major depressive disorder (MDD; Afari & Buchwald, 2003). However, depression in CFS is challenging to measure due to overlapping symptoms. Some psychodiagnostic instruments are more reliable and valid than others for use with a CFS population as they allow for clinical judgment in attributing overlapping symptoms to illness as opposed to depression (Reeves et al., 2003). Consequently, some depression rating scales were found to overestimate depression in CFS samples (Henderson and Tannock, 2005). Self-report depression measures commonly used in CFS research, such as the Beck Depression Inventory-II (BDI-II; Beck et al., 1996), may be vulnerable to overestimation of depressive symptomatology. The BDI-II measures depression severity by assessing a range of depressive symptoms, several of which are also hallmark symptoms of CFS, including fatigue, concentration difficulties, and changes in sleeping patterns. Due to the self-report nature of the BDI-II, endorsement of items related to depression versus CFS symptomatology is difficult to differentiate.
The BDI-II has been validated with several populations including healthy adults (Dozois et al., 1998), primary care patients (Arnau et al., 2001), and people with MDD (Steer et al., 1999a; Steer et al., 1998). Initial validation studies with adult psychiatric outpatients using both exploratory and confirmatory factor analyses found two-factor solutions including Somatic-Affective and Cognitive factors (Beck et al., 1996; Steer et al., 1999a). However, prominent inconsistencies in factor structure exist across studies. In a review, Vanheule et al. (2008) reported that several two- and three-factor solutions have been found, but most have not been replicated. Although several studies revealed Somatic-Affective and Cognitive factors, some items loaded differently across studies. For example, in a primary care sample, Arnau et al. titled their factors Somatic-Affective and Cognitive, but three items (Sadness, Self-Criticalness, and Crying) loaded on factors opposite of the original Beck et al. solution. Among a college student sample, Dozois et al. acquired a two-factor model that was nearly identical to Beck et al.’s model, but the factors were termed Cognitive-Affective and Somatic-Vegetative. Taken together, the diversity of BDI-II studies suggests that there is a lack of consistency in factor structures, items contributing to factors, differences in the conceptualization of factors, and varying findings across populations (Vanheule et al., 2008).
It is possible that CFS samples may yield yet another unique response pattern due to the explicit overlap between CFS diagnostic criteria and BDI-II items. Some research suggests that somatic symptoms of depression do not significantly influence the validity of depression diagnoses in medical patients (Simon and von Korff, 2006). However, the appropriateness of using clinical judgment in attributing somatic symptoms to CFS versus psychiatric diagnoses is not well understood. For example, using a diagnostic interview, Johnson et al. (1996a) found rates of somatization disorder diagnoses in a CFS sample substantially increased when CFS diagnostic symptoms counted toward the psychiatric diagnosis compared to when CFS symptoms were omitted. Taylor and Jason (1998) found that the type of measure used to diagnose psychopathology in CFS does influence diagnostic reliability and rate of diagnosis, as some measures were developed for appropriate use with medical samples.
Differentiating CFS and depressive somatic symptoms poses a particular problem for self-report scales, as they do not allow for contextual assessment, such as whether the onset of symptoms occurred before, after, or concurrent with CFS onset. One study found that the original BDI was not a valid measure of depression in CFS populations when compared with a semi-structured psychiatric interview (Farmer et al., 1996). Upon closer examination of BDI symptom categories, Johnson et al. (1996b) found that patients with CFS showed a pattern of lower mood and self-reproach symptoms compared to those with depressive disorders. BDI mood symptoms were concluded to be indicators of comorbid depression in CFS (Johnson et al., 1996b). The structure and psychometric properties of the BDI-II have not been evaluated for CFS patients. Regardless, the BDI-II continues to be widely used as a measure of depressive symptomatology in this population.
An alternative approach to measuring depression among patients with CFS is to use a measure developed for use with medical populations. Beck et al. (1997) developed a seven-item version of the BDI for use in primary care settings (BDI-PC) that excludes somatic items. The BDI-PC includes the item Loss of Pleasure to assess anhedonia. This measure has the ability to discriminate depressed and non-depressed medical inpatients (Beck et al., 1997) and outpatients (Steer et al., 1999b). The BDI-PC has been used in few CFS research studies (e.g., Servaes et al., 2002; Servaes et al., 2000).
In order to identify potential considerations when using the BDI-II for assessing depression among patients with CFS in research or clinical practice, the present study evaluated the psychometric properties of the BDI-II in a CFS sample using exploratory factor analysis (EFA). A comparison of factor scores was carried out to determine whether patients with CFS were more likely to endorse somatic items of depression due to overlapping CFS and depressive symptomatology. Finally, convergent and discriminant validity was assessed for the BDI-II total score, factor scores, and the BDI-PC.
Method
Participants
The present investigation utilized baseline data derived from a larger longitudinal study of non-pharmacological treatments for CFS (Jason et al., 2007). Participants were recruited from physician referrals, media advertisements, and CFS support groups. Participants were age 18 and older, not pregnant, able to read and speak English, and physically capable of attending study appointments. Participants were included if they met the Fukuda et al. (1994) criteria for CFS. A total of 114 participants were enrolled in the original study. Three participants were excluded from the present investigation because they did not complete the BDI-II, leaving a total of 111 participants.
Materials
CFS Questionnaire
The CFS Questionnaire (Jason et al., 1997) was used to collect demographic and symptom data. This screening scale has demonstrated adequate interrater reliability (Kappa = .85) and an average five-day test-retest reliability of .77 for patients with CFS (Hawk et al., 2007). This scale was found to adequately differentiate patients with CFS from those with MDD and healthy controls with 90% specificity and 93% sensitivity (Hawk et al., 2007).
Beck Depression Inventory-II (BDI-II)
Depressive symptomatology was measured with the BDI-II (Beck et al., 1996), a 21-item self-report instrument. Items on the BDI-II are rated on four-point scales ranging from zero to three, with a maximum total score of 63. Higher scores indicate more severe depressive symptoms. The BDI-II has good internal consistency (α = .92) and one-week test-retest reliability (r = .93; Beck et al., 1996). The BDI-II demonstrated good convergent validity compared with the Beck Anxiety Inventory (r = .56), and good discriminant validity when compared with Sociotropy and Autonomy Independence Scale (r = −.10; Steer and Clark, 1997). The seven items of the BDI-PC (Sadness, Loss of Pleasure, Suicidal Thoughts or Wishes, Pessimism, Past Failure, Self-Dislike, and Self-Criticalness) were derived from the full BDI-II. Possible scores on the BDI-PC range from zero to 21.
Medical Outcomes Study-Short Form36-Item Health Survey (SF-36)
Participants completed the SF-36 (Ware and Sherbourne, 1992), a measure of health perceptions and disability demonstrating good psychometric properties (McHorney et al., 1993). This measure includes eight subscales measuring various aspects of physical and mental health. The following four subscales were used: Physical Functioning, Role-Physical, Mental Health, and Role- Emotional. These four subscales were selected because the Physical Functioning and Role-Physical subscales were designed to exclusively measure physical health outcomes, while the Mental Health and Role-Emotional subscales were designed to exclusively assess mental health functioning (Ware and Sherbourne, 1992). Scores on each subscale range from zero to 100, with higher scores indicating better health or functioning.
Procedure
Participants underwent a medical examination and structured psychiatric interview to establish CFS and psychiatric diagnoses and rule out exclusionary conditions (Fukuda et al., 1994). CFS diagnoses were determined by the study physician (for more details see Jason et al., 2007). All procedures were approved by the DePaul University Institutional Review Board. Written informed consent was given by all participants.
Results
Demographic outcomes
In regards to demographic characteristics, 82.9% of the 111 participants were female. The average age was 44.1 years (SD = 11.6). Regarding ethnicity, 88.3% were White, 4.5% were Latino, 3.6% were African American, and 3.6% were Asian American. For marital status, 50.4% were married/living with a partner, 31.5% were single, and 18.0% were divorced/separated. For work status, 60.4% were not working and 39.6 % were working or full-time students.
The average total score on the BDI-II was 17.66 (SD = 9.14), indicating a moderate level of depression. The BDI-II demonstrated good internal consistency, α = .89. One-way analyses of variance and Pearson correlations did not reveal a significant relationship of BDI-II score and sociodemographic variables. For the BDI-PC, the average score was 4.30 (SD = 3.18).
Principal axis factoring
Principal axis factoring (PAF) with Promax (oblique) rotation was used to evaluate the factor structure of the BDI-II in the CFS sample. PAF is suggested as an appropriate approach for EFA as it produces more robust, replicable factors than other methods such as principal components analysis (Gorsuch, 1997). Indices of factorability were examined as follows: Bartlett’s Test of Sphericity was significant (χ2 = 917.38, p < .001), the Kaiser-Meyer-Olkin statistic was .82, and the Measures of Sampling Adequacy all exceeded .7, indicating that the data were appropriate for PAF. Descriptive statistics and intercorrelations among the BDI-II items are reported in Table 1.
Means (M), Standard Deviations (SD), and Intercorrelations among Beck Depression Inventory-II Items (N = 111)
Two methods were used to determine the number of factors to retain. First, parallel analysis (Hayton et al., 2004) was used. Parallel analysis compares actual eigenvalues to a series of eigenvalues from randomly generated datasets with the same number of observations and variables as the original dataset. The suggested number of factors to retain is based on whether the actual eigenvalues are larger than the average and 95th percentile eignenvalues from the random datasets, and our data indicated retention of two factors. Second, we evaluated the extraction percentage of variance explained by the potential factors. Only the first two factors explained greater than 5% of the variance, indicating that the remaining factors accounted for a negligible amount of variance (Pett et al., 2003). Based on these two methods, two factors were retained.
Factors one and two accounted for 30.0% and 5.72% of the extracted variance, respectively, with a cumulative variance explained of 35.72%. The two factors were correlated (r = .63). The rotated factor pattern matrix and extraction communalities are reported in Table 2. The regression coefficient cutoff for meaningful factor loadings was set at .35, given guidelines for meaningful factor loadings generally range from .3 to .4 (Floyd and Widaman, 1995). Factor one was comprised of 11 items: Worthlessness, Guilty Feelings, Self-Dislike, Self-Criticalness, Crying, Sadness, Past Failure, Punishment Feelings, Pessimism, Suicidal Thoughts or Wishes, and Loss of Interest. Factor two was comprised of eight items: Tiredness or Fatigue, Concentration Difficulty, Loss of Energy, Changes in Sleeping Pattern, Loss of Interest in Sex, Indecisiveness, Irritability, and Loss of Pleasure. Two items, Agitation and Changes in Appetite, did not load above .35 on either factor. The factors most closely resembled Beck et al.’s (1996) structure of Cognitive and Somatic-Affective factors. Although both factors arguably include affective items, the hallmark depressive symptom, Loss of Pleasure, loaded on factor two. Thus, factors one and two were termed Cognitive and Somatic-Affective, respectively, consistent with Beck et al.’s conceptualization.
Promax Rotated Factor Pattern Matrix (N = 111)
Notes: h2 = communalities; salient coefficients (≥ .35) are bolded; Factor 1 = Cognitive; Factor 2 = Somatic-Affective
Evaluation of factor scores
Total scores on the two factors were computed and compared. The two low-loading items were excluded from the following analyses. Because the Cognitive factor had 11 items on a zero to 33 scale and the Somatic-Affective factor had eight items on a zero to 24 scale, the Cognitive factor was adjusted to match the zero to 24 scale of the Somatic-Affective factor by multiplying scores by 0.7272. A paired-samples t-test revealed a significant difference, t(110) = 11.95, p < .001, with Somatic-Affective scores being significantly larger (M = 10.07, SD = 4.56) than Cognitive scores (M = 5.21, SD = 4.12).
Convergent and discriminant validity
Convergent and discriminant validity were evaluated using Pearson product moment correlation coefficients. The BDI-II total, Cognitive factor, Somatic-Affective factor, and BDI-PC scores were correlated with the four SF-36 subscales. In order to demonstrate convergent validity, a depression measure is expected to correlate strongly with the Role-Emotional and Mental Health subscales. To demonstrate discriminant validity, a depression measure is expected to have a low correlation with the Physical Functioning and Role-Physical subscales. Significant differences in correlations between physical and mental health SF-36 subscales were evaluated using Fisher’s r to z tests to further assess discriminant validity. Correlation coefficients for the two SF-36 mental health subscales were compared with coefficients for the two physical health subscales, yielding four z tests for each of the four BDI scores.
The correlation coefficients and Fisher’s r to z test results used to assess convergent and discriminant validity are reported in Table 3. Regarding convergent validity, all four BDI measures significantly correlated with the Mental Health and Role-Emotional subscales. However, the BDI-II total scores and Somatic-Affective factor scores were also significantly correlated with the Physical Functioning and Role-Physical subscales, while the BDI-PC and Cognitive factor scores were not. The BDI-II total, BDI-PC, and Cognitive factor scores had comparable correlation coefficients for the Mental Health (r = −.66 to −.70) and Role-Emotional (r = −.44 to −.48) subscales, while the Somatic-Affective factor demonstrated somewhat lower convergent validity with the Mental Health and Role-Emotional subscales, with correlation coefficients of r = −.47, and r = −.37, respectively.
Correlation Coefficients (r) and Convergent and Discriminant Validity Results (Fisher’s r to z Tests) for BDI and SF-36 Scores
Notes: 1Higher scores mean less impairment; significant correlations at *p < .05, **p < .01, ***p < .001; for significantly different correlations a = Mental Health > Role-Physical, b = Mental Health > Physical Functioning, c = Role-Emotional > Role-Physical, d = Role-Emotional > Physical Functioning
According to results from the Fisher’s r to z tests, the BDI-PC and Cognitive factor demonstrated the best discriminant validity, as correlation coefficients for both mental health subscales were significantly higher than both of the physical health subscales (z = 2.33 to 5.52). Conversely, the four z scores for the Somatic-Affective factor score did not reveal any significantly different correlations across SF-36 mental health and physical health subscales (z = .002 to .89). The BDI-II total revealed somewhat better discriminant validity than the Somatic-Affective score, as three of the four z tests showed significantly different correlations (z = 2.04 to 4.29). However, the Role-Emotional correlation was not significantly different from the Physical Functioning correlation for the BDI-II (z = 1.65).
Discussion
This study used PAF to explore the structure of the BDI-II in a CFS sample. Results revealed two factors, with the first factor comprised of 11 items characteristic of Cognitive symptoms, and the second factor comprised of eight items characteristic of Somatic-Affective symptoms. This factor structure did not fully resemble any previous BDI-II solutions (Vanheule et al., 2008). With the exception of two items: Crying and Loss of Interest, and the two items that did not saliently load on either factor, our two factors most closely matched those found in Beck et al.’s (1996) original validation study using a clinical sample. Several items on the Somatic-Affective factor overlap with symptoms of CFS, such as Tiredness or Fatigue, Loss of Energy, Concentration Difficulty, and Changes in Sleeping Pattern. As expected, we found the average score on the Somatic-Affective factor was nearly twice as high as the Cognitive factor. Patients endorsed BDI-II items related to CFS at a higher degree than cognitive symptoms more commonly associated with depression, highlighting the difficulty in interpreting total BDI-II scores with this population.
Regarding convergent and discriminant validity, the BDI-II total score and the Somatic-Affective factor correlated significantly with both physical and mental health measures of disability on the SF-36. These two measures had higher correlations with the two SF-36 physical health subscales than the BDI-PC and Cognitive factor. The BDI-PC and the Cognitive factor both demonstrated better convergent and discriminant validity than the BDI-II total and Somatic-Affective factor.
The use of the BDI-II with medical patients has been questioned due to the prominence of somatic items on the measure. However, Arnau et al. (2001) reported that the BDI-II had adequate psychometric properties for measuring depression in primary care settings. Yet, the authors did not provide details of medical diagnoses among their sample. Due to the explicit overlap in diagnostic symptomatology of CFS and MDD, patients with CFS are a unique patient group within primary care settings. This suggestion is highlighted by comparing BDI-II item mean scores for the present CFS sample with the primary care sample used in Arnau et al.’s study. The key overlapping CFS and MDD items were substantially higher for our sample than Arnau et al.’s: Tiredness or Fatigue (M = 1.81 vs. 0.65), Loss of Energy (M = 1.72 vs. 0.70), Changes in Sleeping Pattern (M = 1.48 vs. 0.77), and Concentration Difficulty (M = 1.28 vs. 0.46).
One resolution to interpreting BDI-II scores among patients with CFS would be to evaluate the two factor scores separately. Discounting scores on the Somatic-Affective factor as depressive symptoms may not be appropriate given the combination of both affective and somatic symptoms on this factor. Importantly, a key item measuring anhedonia, Loss of Pleasure (Ward, 2006), is measured on the Somatic-Affective factor. Moreover, the controversy over assigning somatic symptoms to psychological versus organic disorders among those with CFS has not been resolved. Many patients experience comorbid depression. Further, increased severity of somatic symptoms was found to predict more severe depression in those with MDD (Maes, 2009), so excluding somatic symptoms from an evaluation of depression in CFS may not be indicated. Yet, the interpretation of symptoms has important treatment implications, as patients with CFS considered to have a primary depressive disorder based on symptom reporting may not receive proper medical interventions. The precise nature of somatic symptoms cannot be determined from a self-report measure like the BDI-II, and future research is needed to enhance interpretation of these symptoms. Consequently, examination of scores on both Cognitive and Somatic-Affective factors should be undertaken as part of a more intensive psychodiagnostic evaluation.
The BDI-PC is likely to produce fewer problems than the BDI-II in measuring depressive symptomatology in CFS populations due the omission of overlapping symptoms. Interestingly, average BDI-PC scores in this study were somewhat higher compared to previous studies using this measure with CFS samples (M = 3.3, SD = 2.6; Servaes et al., 2002; M = 2.63, SD = 1.82; Servaes et al., 2000). Differing recruitment sources and study enrollment criteria may account for differences in BDI-PC scores across studies.
A major limitation of the present study was the small sample size used for EFA. Stevens (2002) indicated that an adequate EFA sample size should produce factor loadings above 0.8 for at least three items on each factor. Our solution only had two items loading above 0.8, indicating that it may not be robust and should be interpreted with some reservation, and replication of these findings is needed. Of note, we did not assume independence among factors with the rotation method used, as oblique rotation has been suggested for use in social sciences research, as correlation among factors is expected (Costello and Osborne, 2005; Gorsuch, 1997). We conducted PAF using orthogonal and oblique rotation, and comparable results were obtained for both. Additional exploratory and confirmatory methods are needed to determine the stability of this factor structure among patients with CFS.
Some additional issues merit consideration when interpreting these findings. First, the total explained variance by our factors was low (35.7%). Our rate of explained variance was lower than that of Beck et al. (1996), Dozois et al. (1998), and Arnau et al. (2001). However, these authors used principal components analysis, which typically yields greater explained variance than PAF. Second, while most closely resembling the findings of Beck et al., our solution is also similar to other factor structures that have been found for the BDI-II (Vanheule et al., 2008). Interestingly, our solution using a CFS sample was more congruent with that of Beck et al. who used a clinical sample, but it was less congruent with the findings from Arnau et al.’s medical outpatient sample. This may suggest that patients with CFS have different response patterns on the BDI-II than general medical populations.
Differentiation between symptoms of CFS and depression may be complicated by current methods of evaluating psychiatric symptomatology among patients with CFS, necessitating future research to explore this complex issue. The BDI-II is commonly used to evaluate severity of depressive symptomatology in CFS research. However, our results suggest that patients endorse higher severity of Somatic-Affective items, which contain items overlapping with CFS symptoms. This has implications for using BDI-II total scores for measuring depression in CFS research or clinical practice as scores may be interpreted as reflecting a primary depressive disorder. Consequently, it is important to evaluate Somatic-Affective and Cognitive symptoms separately. Given the findings from this study as well as previous research showing inadequate validity of the BDI-II for patients with CFS (Farmer et al., 1996), future research should reconsider the use of the BDI-II total score for measuring and reporting depression in CFS samples. An alternative measure for consideration is the BDI-PC, which demonstrated good convergent and discriminant validity in this study.