Psychometric Properties of the Copenhagen Burnout Inventory in an Academic Healthcare Institution Sample in the U.S.

Abstract

The Copenhagen Burnout Inventory (CBI) has demonstrated good psychometric properties among respondents in many different countries, but minimal research exists using the CBI in a U.S.-based sample. The current study represents a secondary analysis of existing CBI data from 1,679 academic health center employees at one mid-size teaching hospital in the southeastern region of the U.S. Analyses assessed CBI scale reliability, confirmatory factorial validity, discriminant validity against a measure of meaningful work, and test invariance for professional role sub-groups (physicians, nurses/physician assistants, and other hospital staff), gender groups, and different age groups. Results provided evidence for good reliability and discriminant validity as well as construct validity supporting the CBI proposed three-factor structure. Configural and metric variance equivalence were demonstrated across the range of employee types, and across age and gender groups. Scalar invariance equivalence was not established, suggesting further research may be needed to support group mean comparisons using the CBI.

Keywords

burnout confirmatory factor analysis healthcare employees measurement invariance USA

Introduction/Background

Prevention of employee burnout is an ongoing concern in U.S.-based academic healthcare settings where trainees often work up to 80 hours a week, and clinical service demands often outstrip the bandwidth of existing human resources. Although burnout has been characterized somewhat differently by various authors over the years, it is widely considered a syndrome consisting of emotional and physical exhaustion caused by chronic work stress (Schaufeli et al., 2009). Similarly, a recent definition of burnout put forth by The World Health Organization in the ICD-11 states that, “burnout is a syndrome conceptualized as resulting from chronic workplace stress that has not been successfully managed” and that it is “characterized by three dimensions: feelings of energy depletion or exhaustion; increased mental distance from one’s job, or feelings of negativism or cynicism related to one’s job; and reduced professional efficacy” (World Health Organization, 2018).

The range and extent of negative outcomes associated with occupational burnout are concerning and have been shown to impact not only individuals (such as maladaptive coping strategies, depression and in some cases even suicide), but also institutions and systems (e.g., costly job turnover, increased medical errors; Aiken et al., 2013; McHugh et al., 2011; West et al., 2018). One approach organizations can use to address employee burnout is to occasionally assess the prevalence and extent of concerning levels of burnout via survey screening, in conjunction with implementing wellness support interventions (West et al., 2018). In order to do this well, organizations need a common understanding of burnout using well-designed instruments.

Most of the burnout research to date in the United States (U.S.) has relied on using the Maslach Burnout Inventory (MBI; Maslach et al., 1981), and despite acceptable psychometric support, not all agree with the MBI definition of burnout (Kristensen et al., 2005; Schaufeli et al., 2009); and many organizations, especially large ones, may find the fees for usage particularly prohibitive. A relatively newer non-commercial and freely available burnout survey, the Copenhagen Burnout Inventory (CBI; Kristensen et al., 2005), has been used in many countries across a wide variety of settings and samples and the tool has a growing evidence base demonstrating good psychometric properties for measuring occupational burnout (Borritz et al., 2006; Chin et al., 2018; Fiorilli et al., 2015; Fong et al., 2014; Kristensen et al., 2005; Lapa et al., 2018; Madsen et al., 2015; Mahmoudi et al., 2017; Molinero et al., 2013; Sestili et al., 2018; Winwood & Winefield, 2004; Yeh et al., 2007).

To date, very few studies have been done using the CBI in U.S.-based populations (Leake et al., 2017; Walters et al., 2018). It appears that the CBI would be appropriate for use with U.S. healthcare employees, nonetheless, it is still important to assess psychometric properties when surveys are used in other countries and samples different from the original validation sample. Moreover, the CBI was originally created and validated in Denmark which consistently ranks among the best countries worldwide in terms of employee life satisfaction and well-being (Biswas-Diener et al., 2010). Authors of the CBI have argued that validity of the instrument should be tested in different cultures (Kristensen et al., 2005). The current study sought to assess several psychometric properties of the CBI including construct validity, discriminant validity, reliability, and factorial test invariance in a sample of American healthcare employees.

Method

Data Sources and Measures

Permission from the Institutional Review Board of the authors’ institution was obtained to conduct the study analyses. The study design represents a secondary analysis of existing CBI data collected at a single, public, mid-sized academic health center located in the southeastern region of the U.S. in two different projects, one conducted 2016 and one in 2017. The two projects contributing to the dataset for this study have been previously described in detail (Messias et al., 2019; Thrush et al., 2019). In brief, the sample consisted of academic health center employees who were direct participants in patient care-related activities in 2017, and a sample of resident physicians at the same hospital in 2016. The response rates for these studies were 24.4% and 44%, respectively, and both samples provided a reasonable reflection of the institution’s population from which they were drawn (Messias et al., 2018; Thrush et al., 2019). That is, the sample distribution was similar to the total healthcare workforce distribution in the teaching hospital where data were collected, where 38% are physicians, 32% are nurses, and 30% are other healthcare employees. All survey respondents were invited to complete the CBI via an e-mailed link with informed consent described in writing and considered as accepted by the participants upon submitting their survey responses. The LimeSurvey electronic platform was used to host the survey and was managed by one person in the hospital’s information technology department. Of 1,679 respondents, 30% (n = 504) were physicians or resident physicians, 38% (n = 643) were nurses (including registered nurses-RNs, and licensed practical nurses—LPNs) or advanced practice staff (such as advanced practice registered nurses-APRNs, physician assistants—PAs), and 32% (n = 532) were all other hospital staff who were engaged in direct patient care (e.g., pharmacists, social workers, respiratory therapists, technologists, etc.).

The CBI is a 19-item survey designed to measure the degree of physical and psychological fatigue and exhaustion attributable to the personal, work, or patient-care related aspects of one’s daily life (Kristensen et al., 2005). Respective items are averaged to contribute to one of three burnout dimensions: personal burnout (6 items), work-related burnout (7 items), and patient-related burnout (6 items). For all items, responses are scored on a five-point scale where always = 100, often = 75, sometimes = 50, seldom = 25, and never/almost never = 0. For each dimension of burnout, a score of 0 reflects no burnout whereas, a score of 50 or greater is considered to reflect moderate to high degrees of burnout.

Meaningful work was measured using 4 items adapted from a previously validated measure, the Work as Meaning Inventory (WAMI; Steger et al., 2012), including: My work is fulfilling and satisfying; I have a good sense of what makes my work meaningful”; “The work I do serves a greater purpose; and I view my work as contributing to my personal growth. WAMI item responses were modified to match the same 5 point scaling approach used on the CBI, with items averaged to reflect a total score. Theoretically, meaningful work is considered as a state antithetical to that of burnout, thus we would expect inverse relationships between the constructs. The meaningful work items were not completed by respondents in the 2016 dataset, but were completed by a majority of respondents in the 2017 dataset (83%; n = 1,398).

Analyses

To assess factorial validity in a new sample, a confirmatory factor analysis (CFA) was performed to confirm the original three-factor model designed by the CBI developers. CFA analyses were conducted using R software, with the default maximum likelihood approach and variances on the latent constructs fixed to 1 for the first item in each subscale. Fit indices commonly used in CFA were reviewed including root mean square error of approximation (RMSEA), standardized root mean squared residual (SRMR), and comparative fit index (CFI), and judged for acceptability using criteria set forth by Hu and Bentler (1995, 1999) [RMSEA < 0.08, SRMR < 0.06, and CFI ≥ 0.90]. Test invariance analyses were also computed in R (R Core Team) using categorical multi-group confirmatory factor analysis procedures, with models estimated comparing the three health care worker groups sampled (physicians, nurses/advanced practice providers, and other hospital staff); as well as the two gender groups (male, female), and three broad age groups (age 20–35, 36–50, and 51+). We chose to examine test invariance for these categorical groupings because literature to date has shown mixed findings and significant differences in the expression of burnout among these categories with females, younger employees and nurses typically having higher rates of burnout (McHugh et al., 2011; West et al., 2018). These extant categories were also readily distinguished in the existing dataset with adequate sample sizes among these groupings for such analyses. Following recommendations by Cheung and Rensvold (2002), we used the change in CFI < 0.01 criteria to indicate equality (i.e., invariance) of the factor model across groups examined. SPSS 25.0 software was used to assess divergent validity in correlational analyses between the CBI scales and the WAMI mean score, as well as to assess scale reliability.

Results

Prior to CFA analyses, items were assessed for normality and indicated significant non-normal distributions for all items. However, as shown in Table 1, none of the items indicated problematic amounts of skewness or kurtosis, thus no transformations to correct for non-normality were utilized. Also shown in Table 1 are the survey item descriptive statistics and scale reliability analyses. Cronbach’s α for the survey overall was high, 0.946. For each scale individually, Cronbach’s α levels were also very high (personal burnout α = 0.892, work burnout α = 0.896, and patient burnout α = 0.897). Correlations between the meaningful work scale and CBI scales were significant (p < 0.001) and in negative directions as expected (r = −0.519 for personal burnout, r = −0.595 for work burnout, and r = −0.477 for patient-related burnout).

Table 1.

Descriptive and Reliability Statistics for Copenhagen Burnout Inventory Items (n = 1,679).

Item	Skewness	Kurtosis	Mean	SD	Corrected item to total correlation	α if item dropped
Personal Burnout
How often do you feel tired?	−0.341	0.098	66.0	20.7	0.680	0.943
How often are you physically exhausted?	−0.248	−0.262	53.4	23.6	0.680	0.943
How often are you emotionally exhausted?	−0.205	−0.292	54.4	24.5	0.700	0.942
How often do you think: “I can’t take it anymore”?	0.363	−0.817	32.3	26.9	0.748	0.941
How often do you feel worn out?	−0.288	−0.333	56.6	23.8	0.753	0.941
How often do you feel weak and susceptible to illness?	0.444	−0.328	32.6	24.7	0.623	0.943
Work Burnout
Is your work emotionally exhausting?	−0.365	−0.218	64.8	24.8	0.711	0.942
Do you feel burnt out because of your work?	0.073	−0.709	45.3	28.7	0.763	0.941
Does your work frustrate you?	0.440	−0.495	32.2	26.0	0.786	0.941
Do you feel worn out at the end of the working day?	0.159	−0.236	40.2	22.9	0.514	0.945
Are you exhausted in the morning at the thought of another day at work?	−0.144	−0.253	53.1	25.3	0.670	0.943
Do you feel that every working hour is tiring for you?	−0.131	−0.195	48.5	24.2	0.661	0.943
Do you have enough energy for family and friends during leisure time?	0.073	−0.670	43.0	27.7	0.792	0.940
Patient Burnout
Do you find it hard to work with patients?	0.400	−0.624	24.9	21.8	0.634	0.943
Do you find it frustrating to work with patients?	0.367	−0.485	29.0	23.7	0.676	0.943
Does it drain your energy to work with patients?	0.380	−0.633	26.2	22.3	0.625	0.943
Do you feel that you give more than you get back when you work with patients?	0.175	−0.816	41.3	29.2	0.512	0.946
Are you tired of working with patients?	0.996	0.213	18.5	22.7	0.628	0.943
Do you sometimes wonder how long you will be able to continue working with patients?	0.820	−0.250	24.8	27.3	0.633	0.943

Note: All items were non-normally distributed, per both Kolmogorov-Smirnov and Shapiro-Wilk tests, p < 0.001.

CFA analyses indicated the overall χ² for the model was significant [χ ²(149) = 1891.13, p < 0.0001], thus the null hypothesis of good fit is rejected, but this may be simply from having too much power due to a large sample size. With the exception of χ² parameter estimates, the goodness of fit indices for the overall model indicated mostly acceptable levels [CFI 0.922, RMSEA 0.083, SRMR 0.044] signifying fitness of the three-factor model as originally proposed by the CBI developers. The CFA path model is shown in Figure 1 with standardized regression weights displayed for each item (all above 0.40; ranging from 0.57 to 0.86). Also shown in Figure 1, the inter-scale coefficients were high, and in expected positive directions. The largest inter-scale coefficient observed was between CBI work and personal burnout scales (r = 0.96), followed by patient and work burnout (r = 0.67) and patient and personal burnout (r = 0.58).

Figure 1.

Confirmatory Factor Analysis Path Model with Standardized Parameter Estimates. Note: All items significant, p < 0 .001.

Test invariance results from the multi-group CFA are shown in Table 2. The CFI differences were lower than 0.01 in all cases for both configural and metric invariance across subgroup models, thus suggesting that the three-–factor structure of the CBI fit well across gender, professional status and age groups. Models could not be estimated by the R program for scalar or residual invariance.

Table 2.

Multi-Group Confirmatory Factor Analysis Tests for Measurement Invariance of the Copenhagen Burnout Inventory By Status, Age, And Gender Groups.

Model	(df) χ2*	RMSEA	SRMR	CFI	CFI
All groups	(149)1891.13	0.083	0.044	0.922	n/a
Status
MDs (n = 504)	(149)626.72	0.080	0.047	0.920	n/a
Others (n = 532)	(149)683.30	0.082	0.045	0.925	n/a
Nurses/Physician	(149)942.82	0.091	0.047	0.912	n/a
Assistants (n = 643)
Configural invariance	(447)2252.92	0.085	0.047	0.918	n/a
Metric invariance	(479)2324.42	0.083	0.055	0.917	−0.001
Age
20-35 (n = 396)	(149)596.93	0.087	0.039	0.933	n/a
36-50 (n = 538)	(149)670.36	0.081	0.045	0.925	n/a
51+ (n = 430)	(149)658.75	0.091	0.047	0.913	n/a
Configural invariance	(479)1941.29	0.086	0.047	0.921	n/a
Metric invariance	(479)1985.42	0.083	0.052	0.920	−0.001
Gender
Female (n = 1080)	(149)1365.84	0.087	0.043	0.919	n/a
Male (n = 280)	(149)372.12	0.073	0.050	0.939	n/a
Configural invariance	(298)1742.57	0.084	0.045	0.922	n/a
Metric invariance	(314)1753.90	0.082	0.049	0.923	0.001

*p < 0.0001.

Discussion

Overall, the results provide ample evidence indicating the CBI was a valid and reliable tool for measuring burnout in academic healthcare professionals who participated in this study. The CBI demonstrated good reliability, with moderately high correlations between the scales. Item-level reliability analyses indicated the importance of all items in measuring different dimensions of physical fatigue, tiredness, and exhaustion. The CBI also demonstrated adequate discriminant validity with significant negative correlations when tested against a subset of meaningful work items.

Construct validity was also demonstrated supporting the three-factor structure of the CBI, although it is worth noting that several of the fitness statistics were right on the border of cut-off levels proposed by statisticians, and scalar and residual invariance equivalence were not established suggesting further research may be needed to support group mean comparisons using the CBI. Nonetheless, these findings suggest the CBI demonstrated psychometric acceptability in our particular setting and sample. In addition, the factor structure and item loadings demonstrated measurement invariance across multiple sub-groups of survey respondents suggesting broad appropriateness across the range of direct healthcare providers surveyed.

Overall, the acceptable psychometric properties of the CBI we found are consistent with prior research in a sample of U.S. social workers and a sample of child welfare services employees (Leake et al., 2017; Walters et al., 2018). Nevertheless, conclusions of this study are limited to a single institution, a mid-sized academic health center in the southeastern region of the United States, thus may not be generalizable beyond healthcare workers in similar academic medical centers. Also, the design of the study was a secondary analysis of existing data from two studies that were originally conducted to evaluate the extent of burnout within a sample of academic healthcare employees. Future research using representative and diverse healthcare samples should be done to confirm the study results, and to extend analyses to examine more nuanced aspects of psychometric testing. Research is still needed to assess other important properties of the CBI such as concurrent and predictive validity. Given that there is some evidence to suggest system-related differences in the experience of burnout between private practice and academic healthcare employees (Deckard et al., 1992; Whippen & Canellos, 1991), additional research is also warranted to evaluate whether the CBI exhibits similarly acceptable properties when used with healthcare employees in other types of practice settings.

Given the very brief nature of the CBI which takes only a few minutes to complete, and the ease with which it can be administered by organizations, it may work well as a tool in U.S. academic healthcare organizations and other settings and samples to elicit information which could help guide burnout interventions and wellness support services.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Carol R. Thrush, EdD

References

Aiken

L. H.

Sloane

D. M.

Bruyneel

Van den Heede

Sermeus

, & Rn4cast Consortium. (2013). Nurses’ reports of working conditions and hospital quality of care in 12 countries in Europe. International Journal of Nursing Studies, 50(2), 143–153.

Biswas-Diener

Vittersø

Diener

(2010). The Danish effect: Beginning to explain high well-being in Denmark. Social Indicators Research, 97(2):229–246.

Borritz

Rugulies

Christensen

K. B.

Villadsen

Kristensen

T. S.

(2006). Burnout as a predictor of self-reported sickness absence among human service workers: Prospective findings from three year follow up of the PUMA study. Occupational and Environmental Medicine, 63(2), 98–106.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.

Chin

R. W. A.

Chua

Y. Y.

Chu

M. N.

Mahadi

N. F.

Wong

M. S.

Yusoff

M. S.

Lee

Y. Y.

(2018). Investigating validity evidence of the Malay translation of the Copenhagen burnout inventory. Journal of Taibah University Medical Sciences, 13(1), 1–9.

Deckard

G. J.

Hicks

L. L.

Hamory

B. H.

(1992). From the infectious diseases society of America. The occurrence and distribution of burnout among infectious diseases physicians. Journal of Infectious Diseases, 165(2), 224–228.

Fiorilli

De Stasio

Benevene

Iezzi

D. F.

Pepe

Albanese

(2015). Copenhagen burnout inventory (CBI): A validation study in an Italian teacher group. TPM: Testing, Psychometrics, Methodology in Applied Psychology, 22(4), 537–551.

Fong

T. C.

R. T.

S. M.

(2014). Psychometric properties of the Copenhagen burnout inventory—Chinese version. The Journal of Psychology, 148(3), 255–266.

L. T.

Bentler

P. M.

(1995). Evaluating model fit. In Hoyle

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76–99). Sage.

10.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.

11.

Kristensen

T. S.

Borritz

Villadsen

Christensen

K. B.

(2005). The Copenhagen burnout inventory: A new tool for the assessment of burnout. Work & Stress, 19(3), 192–207.

12.

Lapa

Carvalho

Viana

Ferreira

P. L.

Pinto-Gouveia

Belo-Cabete

(2018). Development and evaluation of a global burnout index derived from the use of the Copenhagen burnout inventory in Portuguese physicians. Acta Medica Portuguesa, 31(10), 534–541.

13.

Leake

Rienks

Obermann

(2017). A deeper look at burnout in the child welfare workforce. Human Service Organizations: Management, Leadership & Governance, 41(5), 492–502.

14.

Madsen

I. E.

Lange

Borritz

Rugulies

(2015). Burnout as a risk factor for antidepressant treatment—a repeated measures time-to-event analysis of 2936 Danish human service workers. Journal of Psychiatric Research, 65, 47–52.

15.

Mahmoudi

Atashzadeh-Shoorideh

Rassouli

Moslemi

Pishgooie

A. H.

Azimi

(2017). Translation and psychometric properties of the Copenhagen burnout inventory in Iranian nurses. Iranian Journal of Nursing and Midwifery Research, 22(2), 117.

16.

Maslach

Jackson

S. E

Leiter

M. P.

(1981). Maslach burnout inventory: MBI. Consulting Psychologists Press.

17.

McHugh

M. D.

Kutney-Lee

Cimiotti

J. P.

Sloane

D. M.

Aiken

L. H.

(2011). Nurses’ widespread job dissatisfaction burnout, and frustration with health benefits signal problems for patient care. Health Affairs (Millwood), 30, 202–210.

18.

Messias

Gathright

M. M.

Freeman

E. S.

Flynn

Atkinson

Thrush

C. R.

Clardy

J. A.

Thapa

(2019). Differences in burnout prevalence between clinical professionals and biomedical scientists in an academic medical centre: A cross-sectional survey. BMJ Open, 9(2), e023506. https://dx-doi-org.web.bisu.edu.cn/10.1136/bmjopen-2018-023506

19.

Molinero

E. R.

Basart

H. G. Q.

Moncada

S. L.

(2013). Validation of the Copenhagen burnout inventory to assess professional burnout in Spain. Revista Espanola de Salud Publica, 87(2), 165–179.

20.

Schaufeli

W. B.

Leiter

M. P.

Maslach

(2009). Burnout: 35 years of research and practice. Career Development International, 14(3), 204–220.

21.

Sestili

Scalingi

Cianfanelli

Mannocci

Del Cimmuto

De Sio

Chiarini

Muzio

M. D.

Villari

Giusti

M. D.

La Torre

(2018). Reliability and use of Copenhagen burnout inventory in Italian sample of university professors. International Journal of Environmental Research and Public Health, 15(8), 1708.

22.

Steger

M. F.

Dik

B. J.

Duffy

R. D.

(2012). Measuring meaningful work: The work and meaning inventory (WAMI). Journal of Career Assessment, 20(3), 322–337.

23.

Thrush

C. R.

Guise

J. B.

Gathright

M. M.

Messias

Flynn

Belknap

Thapa

P. B.

Williams

D. K.

Nada

E. M.

Clardy

J. A.

(2019). A one-year institutional view of resident physician burnout. Academic Psychiatry, 43(4), 361–368.

24.

Walters

J. E.

Brown

A. R.

Jones

A. E.

(2018). Use of the Copenhagen burnout inventory with social workers: a confirmatory factor analysis. Human Service Organizations: Management, Leadership & Governance, 42(5), 437–456.

25.

West

C. P.

Dyrbye

L. N.

Shanafelt

T. D.

(2018). Physician burnout: Contributors, consequences and solutions. Journal of Internal Medicine, 283(6), 516–529.

26.

Whippen

D. A.

Canellos

G. P.

(1991). Burnout syndrome in the practice of oncology: Results of a random survey of 1,000 oncologists. Journal of Clinical Oncology, 9(10), 1916–1920.

27.

Winwood

P. C.

Winefield

A. H.

(2004). Comparing two measures of burnout among dentists in Australia. International Journal of Stress Management, 11(3), 282–289.

28.

World Health Organization. (2018). International classification of diseases for mortality and morbidity statistics (11th Revision). https://icd.who.int/browse11/l-m/en#/http://id.who.int/icd/entity/129180281

29.

Yeh

W. Y.

Cheng

Chen

C. J.

P. Y.

Kristensen

T. S.

(2007). Psychometric properties of the Chinese version of Copenhagen burnout inventory among employees in two companies in Taiwan. International Journal of Behavioral Medicine, 14(3), 126–133.