Suitability of the Depression,Anxiety,and Stress Scale in Parkinson’s Disease

Abstract

Background: The Depression, Anxiety, and Stress Scale –21 (DASS-21) is a frequently used measure of emotional disturbance symptoms in Parkinson’s disease (PD). However, the factor structure of the DASS-21 in PD has yet to be explored.

Objective: To assess whether the scale is measuring these symptoms in PD in the same way as the general population.

Methods: The present study fit a series of established DASS-21 factor structures with both confirmatory factor analysis (CFA) and exploratory structural equation modelling (ESEM) using data from 251 participants with PD.

Results: The 3-factor ESEM provided the best fit. The depression and stress scales fit well, however, few items on the anxiety subscale loaded clearly, with several items significantly loading onto the depression or stress factors.

Conclusions: Whilst the depression and stress subscales appear suitable in PD, poor loadings and internal consistency indicate the anxiety subscale may not accurately assess anxiety symptomology in PD. This may be due to the scale’s reliance on physiological symptoms as indicators of anxiety, when many of these are present in PD. Thus, the anxiety subscale of the DASS-21 may not be a suitable measure of anxiety in PD.

Keywords

Parkinson’s disease DASS-21 ESEM structure

INTRODUCTION

Depression and anxiety affect a significant proportion of people with Parkinson’s disease (PD) [1, 2]. The Depression, Anxiety, and Stress Scale –21 (DASS-21) [3] is a frequently used measure of emotional disturbance in PD [4 –7]. The convergent and discriminant validity of the DASS-21 has been reported in community [8, 9] and clinical [10, 11] populations, but has not been explored in PD. Recent reviews identified that few measures were suitable for assessing depression or anxiety in PD, but did not assess the DASS-21 [2, 12]. Further, the DASS offers potential advantage in PD research and practice since, offering the potential to assess depression, anxiety and stress efficiently in just 21 items. Given its common use in PD [4 –7], and its established validity across several clinical populations [8 –11], the suitability of the DASS-21 in PD needs to be assessed. The DASS-21 contains three 7-item subscales, one each for depression, anxiety, and stress [3]. Previous studies have used these subscales to measure the severity of the respective symptoms [4 , 14]. However, the DASS-21’s factor structure has yet to be examined in PD. It therefore remains unclear whether the subscale items load onto their intended constructs (depression, anxiety, stress) when the measure is used in PD populations.

Several structures of the DASS-21 have been proposed. The 1-factor model, where all items are driven by a single factor, was tested by Crawford and Henry [15] in a large, non-clinical sample and found to fit poorly. Despite the poor model fit, all items significantly loaded onto a single factor, indicating a large amount of common variance [15]. A 2-factor model proposed by Duffy, Cunningham, and Moore[16] suggested that the items measured physiological hyperarousal and generalised negativity. Duffy et al.’s model demonstrated good fit when applied to community samples [17, 18]. Lovibond and Lovibond [3], the scale developers, proposed the commonly used 3-factor model of depression, anxiety, and stress. The 3-factor model has demonstrated good fit in both community [9, 19] and clinical samples [10, 11].

More recent approaches apply bifactor modelling, which purports that all items are influenced by a general factor (for example, general distress) as well as the specific factors (depression, anxiety, and stress) [20]. Henry and Crawford [9] proposed a bifactor model which included a general factor (in addition to depression, anxiety and stress). This general factor influenced all items and was thought to reflect ‘general psychological distress’. Henry and Crawford’s model has since demonstrated good fit in community samples [19, 21]. Tully, Zajac, and Venning [22] also proposed a bifactor model. In this model, the domain specific factors were anxiety and depression. Stress items were solely influenced by the general factor (negative affect) which influenced all items. Tully et al.’s model demonstrated good fit in a large community sample [23].

Given that the previous models all demonstrate good fit across both clinical and community adult samples (1-factor model aside), this study examined which solution provides the best fit in a PD population. Use of the DASS-21 typically assumes the measure aligns with a 3-factor model such as that proposed by Lovibond and Lovibond [3], such that subscale scores reflect depression, anxiety, and stress levels. The aforementioned models were tested using Confirmatory Factor Analysis (CFA). A key aspect of CFA is that each item loads uniquely onto the factor specified by the researcher, and cannot load onto others [24]. For example, when the Lovibond and Lovibond 3-factor model is fitted using CFA, items on the ‘Depression’ subscale cannot load on the ‘Anxiety’ or ‘Stress’ scales, and vice versa. This approach implies that a given behaviour is only indicative of a single emotional factor (e.g. that an increased heart rate is only indicative of anxiety, not stress). Since emotional distress behaviours are often associated with a range of symptoms, such an assumption may be inappropriate and misleading [25].

Bifactor modelling addresses some of the problems associated with this assumption [26]. Bifactor approaches propose a general factor that accounts for the variance common to all items, and two or more specific factors, that capture the remaining variance in each item (e.g. depression) [26]. However, this approach still assumes that items are only influenced by the specified factors [20]. While the bifactor model can account for those instances where all items load onto a general factor, it does not allow cross-loadings between specific factors - unless they are explicitly modelled by the researcher [20].

To address some of these issues, Asparouhov and Muthén developed exploratory structural equation modelling (ESEM) [24]. ESEM allows for the use of ‘Target’ rotation, which performs exploratory factor analysis (where items can load on any factor) while also rotating the solution to best fit a specified confirmatory model [24]. The analysis rotates the solution to maximise the loadings on the specified factors, whilst allowing items to load onto other factors if that best represents the data. An ESEM approach demonstrates clear utility with psychometric measurements where observed behaviours often represent multiple factors [25].

The present study fit a series of models to DASS-21 data collected from a large, non-demented, community-based PD cohort. These included: a) 1-factor model; b) Duffy et al.’s, 2-factor model [16]; c) Lovibond and Lovibond’s 3-factor model [3]; d) Henry and Crawford’s Bifactor Model [9]; e) Tully et al.’s Bifactor Model [22]. This study (i) explored the fit of each model, and (ii) tested whether an ESEM approach provided a better fit compared to CFA or bifactor models.

METHODS

Participants

Data were collected as part of the larger ParkC [27] study which has approval from Curtin University’s Human Research Ethics Committee (HR158/2013). All testing was conducted during the ‘on’ state of medication. The full testing protocol has previously been described in detail [27]. Inclusion required a formal diagnosis of idiopathic PD by a neurologist or geriatrician, in accordance with the United Kingdom Parkinson’s disease Society Brain Bank criteria [28]. Individuals were screened for dementia using the Mini Mental State Examination (score ≤24 resulting in exclusion) [29]. 251 participants met inclusion criteria (170 males, 81 females) and the mean age was 66.11 (SD = 9.77). Mean levodopa equivalent dose [30] was 568.57 mg (SD = 480.80) and mean disease duration was 64.03 months (SD = 60.46). Thirty-eight participants reported a diagnosis of depression, and 24 reported a diagnosis of anxiety at the time of assessment. Fifty-six were receiving antidepressants and 10 anxiolytics at the time of assessment.

Measures

The DASS-21 is a 21-item self-report measure assessing depressive, anxious, and stressful symptomology within the past week [3]. The DASS-21 is scored on a Likert scale from 0 (‘Did not apply to me at all’) to 3 (‘Applied to me very much, or most of the time’) [3]. Higher scores indicate more severe symptomology. The scale has demonstrated adequate internal consistency and diagnostic validity in a range of different samples [31].

Analysis

Data were analysed using Confirmatory Factor Analysis and Exploratory Structural Equation Modelling with Mplus version 7 [32]. Robust maximum likelihood (MLR) estimation was used, having demonstrated superior performance with ordinal data in small sample sizes [33].

Model fit was assessed using the Tucker & Lewis Index (TLI), threshold: >0.95 [34]; the Comparative Fit Index (CFI), threshold: >0.95 [35]; the Root Mean Square Error of Approximation (RMSEA), threshold: <0.05 [36]; and the Standardised Root Mean Square Residuals (SRMR), threshold: <0.08 [34]. The χ² fit statistic is also reported, but has been shown to over-reject models with ordinal indicators and should be interpreted with caution [33]; The parsimony ratio represents the complexity of the model relative to the number of observed variables; values closer to 1 indicate more parsimonious models [37]. The Akaike Information Criterion was used to compare models [38];

Factor determinacy scores represent the correlation between individuals’ estimated factor scores and the true latent factor, threshold: >0.9 [39]; Factor determinacy represents the quality of the estimated factor - the extent to which the estimated factor represents the true factor (free of error) [39]. Scores closer to 1 indicate greater quality of estimation. Factor determinacy scores were used to assess how well the specified model measured the latent constructs (e.g., depression).

The internal consistency of each factor was assessed using McDonald’s ω coefficient and Cronbach’s α. The ω coefficient is considered to be more accurate than Cronbach’s α, and can be used with bifactor models [40]. The coefficients and bias-corrected 95% confidence intervals were calculated using maximum likelihood estimation with 10 000 bootstrapped samples. For completeness, omega and alpha were calculated for all factor solutions.

Five established DASS-21 factor structures were fit first in CFA, and then in ESEM with ‘Target’ rotation:

1-factor Model.

2-factor Model –Oblique model, two correlated factors representing physiological hyperarousal and generalised negativity, Duffy et al. (2005).

3-factor Model –Oblique model, three correlated factors representing depression, anxiety, and stress, Lovibond and Lovibond (1995).

Bifactor Model A –Nested model, 3 independent factors representing depression, anxiety, and stress and a general negative affect factor, Henry and Crawford (2005).

Bifactor Model B –Nested model, 2 independent factors representing depression and stress and a general negative affect factor, Tully, Zajac, and Venning (2009).

RESULTS

Model fit

Of the five models, Lovibond and Lovibond’s 3-factor Model demonstrated the best overall fit in both CFA and ESEM. The ESEM of the Henry and Crawford Bifactor Model A showed slightly better fit than the 3-factor models, but the improvement was not sufficient to justify the loss of parsimony. The Lovibond and Lovibond 3-factor CFA and ESEM showed comparable fit. The ESEM fit was slightly better than the CFA fit, but was less precise (as it required estimating more parameters) [41].

Modification indices suggested correlating two pairs of item residuals: DASS1 (‘I found it hard to wind down’) with DASS12 (‘I found it difficult to relax’), and DASS4 (‘I experienced breathing difficulty’) with DASS19 (‘I was aware of the action of my heart in the absence of physical exertion’). These residual pairs have been correlated in previous factor analyses [9, 15]. The three leading models (3-factor CFA, 3-factor ESEM, and Bifactor Model A ESEM) were estimated again, with these residualscorrelated.

The Lovibond and Lovibond 3-factor ESEM fit better than the CFA, but fit was only slightly better than the ESEM of the Henry and Crawford Bifactor Model A. Given the Lovibond and Lovibond 3-factor ESEM required the estimation of fewer parameters than the Bifactor Model A, and still demonstrated comparable fit, it was considered the better model [41]. Fit statistics for all models are reported in Table 1. Factor loadings with means and standard deviations for items in the Lovibond and Lovibond 3-factor ESEM with the two residual pairs correlated are reported in Table 2.

Two DASS items showed cross-loadings. DASS9 (‘I was worried about situations in which I might panic and make a fool of myself’) had similar loadings on both anxiety and stress factors. DASS17 (‘I felt I wasn’t worth much as a person’) showed similar loadings on both Depression and Anxiety factors. A number of Anxiety items also loaded onto other factors. DASS2 (‘I found it difficult to relax’) loaded onto the Depression factor and DASS19 (‘I was aware of the action of my heart in the absence of physical exertion’) loaded onto the Stress factor. DASS4 (‘I experienced breathing difficulty’) did not clearly load onto any factor. The mean for DASS7 (‘I experienced trembling’) was also markedly higher than that of the other items.

Consistent with the factor loadings, the factor determinacy scores indicated the model was more accurately assessing depression and stress than anxiety, with scores of 0.96, 0.92, and 0.88, respectively. The R² values for all items were significant at the p < 0.01 level.

The three factors were also correlated at p < 0.005. As ESEM reduces overestimation of factor correlations [24], this intercorrelation is likely an accurate representation of the relationships between symptoms. This indicates that depression, anxiety, and stress were significantly related in our sample. However, given the poor coherence of the anxiety subscale, correlations with the anxiety factor may not be representative of true relationships with anxiety in the sample.

Internal consistency

As can be seen in Table 3, all scales except the Duffy et al. 2-factor model demonstrated acceptable internal consistency. The anxiety subscale of Lovibond and Lovibond’s 3-factor model showed markedly poorer internal consistency than the depression and stress subscales. This suggests that individuals did not indicate similar levels of severity across anxiety items, but they did indicate similar levels of severity for depression and stress items. The Henry & Crawford (Bifactor Model A) and Tully, Zajac, and Venning (Bifactor Model B) models showed high internal consistency. However, the larger confidence intervals for the ω coefficient suggest that these estimates were less precise than those of the other models, and as such the true value may be quite different to the estimate.

DISCUSSION

The present study used confirmatory factor analysis and exploratory structural equation modelling to determine the ideal factor structure of the DASS-21 in a PD sample. The 3-factor model proposed by Lovibond and Lovibond [3] demonstrated the best fit. As predicted, the best model fit was confirmed using ESEM. Depression, anxiety, and stress were significantly correlated. The depression and stress items clearly loaded onto their intended factors and displayed good internal consistency. In contrast, the anxiety subscale demonstrated poor internal consistency, and more than half of the items loaded onto other factors. The clear identification of the depression and stress factors, but not the anxiety factor, suggests that the presentation of anxiety in PD may be quite different to that of other clinical or non-clinical samples. Anxiety items would show clearer loadings if the DASS-21 were accurately assessing anxiety in PD in the same manner as for these other populations.

It is possible that anxiety subscale items did not load clearly onto the anxiety factor in PD as they assess the presence of behaviours or physiological symptoms that individuals with PD are more likely to experience, regardless of anxiety. The DASS-21 anxiety subscale asks about the experience of dryness of mouth, breathing difficulties, tremor, awareness of the heart, and worry about social situations [3]. PD itself is associated with problems with excess saliva and drooling [42], breathing difficulties, pulmonary dysfunction [43, 44], changes in cardiac functioning [45, 46], and increased levels of perceived social stigma due to motor symptoms [47]. Many items of the anxiety subscale thus assess some of the actual symptoms of PD. For example, participants in the present study scored highly for DASS7 (‘I experienced trembling’), which assesses one of the cardinal motor symptoms of PD. The anxiety subscale may not discriminate between individuals with and without anxiety in PD, as both groups may experience the symptoms to a similar extent due to PD symptomology. The clear way in which the depression and stress items factored into their respective subscales suggests that these are indeed distinct factors. As the depression and stress subscales do not assess the experience of physiological symptoms, they may have been less influenced by PD-related, physiological changes.

Lovibond and Lovibond suggest that the presentation of anxiety is consistent across clinical and non-clinical populations, with only the severity of symptoms differing [48]. The results of this study suggest that for people who experience physiological changes associated with PD progression, somatic symptoms may not be a reliable indicator of a comorbid anxiety disturbance. Most anxiety scales used in PD rely on somatic indicators, which may not be appropriate. For example; the Hamilton Anxiety Rating Scale (HARS) [49], the Beck Anxiety Scale (BAI) [50], and the newly-developed Parkinson’s Anxiety Scale (PAS) [51] assess heart palpitations and respiratory difficulties. These are somatic items that were highly unreliable in the present sample. A promising alternative is the Geriatric Anxiety Inventory, which is recommended for use in PD [12]. The GAI primarily assesses cognitive symptoms of anxiety (e.g. worrying, catastrophising, etc.), and thus may be less affected by physiological symptoms [52]. The GAI does, however, assess the experience of gastrointestinal distress (‘upset stomach’), a common symptom of PD [53]. Although the accuracy of the GAI is affected by a single physiological symptom of PD, it is likely not as affected as other measures. Clinicians must consider the comorbid impact of physical symptoms on their assessment of anxiety, and we suggest that a focus on cognitive indicators such as worry and catastrophising may provide a more accurate approach for those with PD.

The benefits of the ESEM approach are clear when considering that the CFA model showed acceptable fit. The unidimensionality of the depression and stress subscales may have sufficiently aided fit in the CFA approach such that the model appeared acceptable. The CFA, however, would not have revealed the significant cross-loadings of several anxiety items identified by the ESEM. These cross-loadings indicate that not only are some items not consistently measuring anxiety, but that they also covary with depression or stress. Allowing items to load freely while fitting a specific model to the data provided a more thorough investigation of the DASS-21 in PD. The ESEM approach indicates that, while the Lovibond and Lovibond [3] 3-factor model shows good fit, it does not adequately assess anxiety symptoms in PD.

These conclusions should be considered in the context of this study’s limitations. The present study is the first application of exploratory structural equation modelling to the DASS-21. It is not known whether this pattern of loadings is specific to the DASS-21, to this participant cohort, or to PD in general. Further replication in other PD samples is required, including samples with more advanced PD in whom the expression of anxiety symptoms may be more severe.

Overall, this study demonstrates a need for further investigation of the structure of the DASS-21 in PD. The subscales of the DASS-21 are currently used in both research and clinical contexts. This study suggests that this may not be completely appropriate. Evidence presented here suggests that the depression and stress subscales of the DASS-21 may be suitable for use in PD, but the anxiety subscale may not be a valid measure of anxiety symptomology in PD. This may have significant implications for clinical use, as individuals may be incorrectly identified as having clinically significant levels of anxiety symptoms based on the DASS-21. Further exploration is required to determine whether the DASS-21 factor structure identified in this study is consistent in other PD cohorts, and to assess whether alternative measures of anxiety would be more appropriate for use in PD.

AUTHOR CONTRIBUTIONS

First draft of manuscript written by ARJ, reviewed by LB, EJC, BJL, MGT, RSB, NG, & AML. Research design developed and implemented by MGT, RSB, NG, & AML. Statistical approach developed and implemented by ARJ, critiqued by RSB. All authors gave final approval of the manuscript prior to submission.

FINANCIAL DISCLOSURES

ARJ, LB, EJC, BJL: Scholarship awarded by Curtin University - Australian Postgraduate Award

EJC, BJL: Received funding from Parkinson’s Western Australia for other projects

RSB: Salary (University of Western Australia)

AML: Salary (Curtin University)

NG: Salary (Curtin University), Salary (School Curriculum and Standards Council)

MGT: None

RELEVANT CONFLICTS OF INTEREST

There are no conflicts of interest for any author.

RELEVANT FUNDING SOURCES

No funding sources relevant to current article for any author.

References

Gallagher

& Schrag

(2012) Psychosis, apathy, depression and anxiety in Parkinson’s disease. Neurobiol Dis, 46, 581–589.

Torbey

& Pachana

& Dissanayaka

NNW

(2015) Depression rating scales in Parkinson’s disease: A critical review updating recent literature. J Affect Disord, 184, 216–224.

Lovibond

& Lovibond

(1995) Manual for the Depression Anxiety Stress Scales, Psychology Foundation, Sydney.

Harris

, McNamara

& Durso

(2015) Novelty seeking in patients with right- versus left-onset Parkinson disease. Cogn Behav Neurol, 28, 11–16.

Wegelin

, McNamara

, Durso

, Brown

& McLaren

(2005) Correlates of excessive daytime sleepiness in Parkinson’s disease. Parkinsonism Relat Disord, 11, 441–448.

Troeung

, Gasson

& Egan

(2015) Patterns and predictors of mental health service utilization in people with Parkinson’s disease. J Geriatr Psychiatry Neurol, 28, 12–18.

Whitworth

, Loftus

, Skinner

, Gasson

, Barker

, Bucks

& Thomas

(2013) Personality affects aspects of health-related quality of life in Parkinson’s disease via psychological coping strategies. J Parkinsons Dis, 3, 45–53.

Norton

(2007) Depression Anxiety and Stress Scales (DASS-21): Psychometric analysis across four racial groups. Anxiety Stress Coping, 20, 253–265.

Henry

& Crawford

(2005) The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. Br J Clin Psychol, 44, 227–239 213p.

10.

, Trauer

, Dodd

, Callaly

, Campbell

& Berk

(2007) The validity of the 21-item version of the Depression Anxiety Stress Scales as a routine clinical outcome measure. Acta Neuropsychiatr, 19, 304–310.

11.

Wood

, Nicholas

, Blyth

, Asghari

& Gibson

(2010) The utility of the short version of the Depression Anxiety Stress Scales (DASS-21) in elderly patients with persistent pain: Does age make a difference?Pain Med, 11, 1780–1790.

12.

Dissanayaka

NNW

, Torbey

& Pachana

(2015) Anxiety rating scales in Parkinson’s disease: A critical review updating recent literature. Int Psychogeriatr, 27, 1777–1784.

13.

Chong

TT-J

, Bonnelle

, Manohar

, Veromann

K-R

, Muhammed

, Tofaris

, Hu

& Husain

(2015) Dopamine enhances willingness to exert effort for reward in Parkinson’s disease. Cortex, 69, 40–46.

14.

McNamara

, Durso

& Harris

(2006) Life goals of patients with Parkinson’s disease: A pilot study on correlations with mood and cognitive functions. Clin Rehabil, 20, 818–826.

15.

Crawford

& Henry

(2003) The Depression Anxiety Stress Scales (DASS): Normative data and latent structure in a large non-clinical sample. Br J Clin Psychol, 42, 111–131.

16.

Duffy

, Cunningham

& Moore

(2005) Brief report: The factor structure of mood states in an early adolescent sample. J Adolesc, 28, 677–680.

17.

Szabó

(2010) The short version of the Depression Anxiety Stress Scales (DASS-21): Factor structure in a young adolescent sample. J Adolesc, 33, 1–8.

18.

Willemsen

, Markey

, Declercq

& Vanheule

(2011) Negative emotionality in a large community sample of adolescents: The factor structure and measurement invariance of the short version of the depression anxiety stress scales (DASS-21). Stress Health, 27, e120–e128.

19.

Osman

, Wong

, Bagge

, Freedenthal

, Gutierrez

& Lozano

(2012) The depression anxiety stress scales— 21 (DASS-21): Further examination of dimensions, scale reliability, and correlates. J Clin Psychol, 68, 1322–1338.

20.

Wiesner

& Schanding

(2013) Exploratory structural equation modeling, bifactor models, and standard confirmatory factor analysis models: Application to the BASC-2 Behavioral and Emotional Screening System Teacher Form. J Sch Psychol, 51, 751–763.

21.

Gomez

(2013) Depression Anxiety Stress Scales: Factor structure and differential item functioning across women and men. Pers Individ Dif, 54, 687–691.

22.

Tully

, Zajac

& Venning

(2009) The structure of anxiety and depression in a normative sample of younger and older Australian adolescents. J Abnorm Child Psychol, 37, 717–726.

23.

Vasconcelos-Raposo

, Fernandes

& Teixeira

(2013) Factor structure and reliability of the Depression, Anxiety and Stress Scales in a large Portuguese community sample. Span J Psychol, 16, E10.

24.

Asparouhov

& Muthén

(2009) Exploratory structural equation modeling. Struct Equ Modeling, 16, 397–438.

25.

Booth

& Hughes

(2014) Exploratory structural equation modeling of personality data. Assessment, 21, 260–271.

26.

Reise

, Morizot

& Hays

(2007) The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Qual Life Res, 16, 19–31.

27.

Loftus

, Bucks

, Thomas

, Kane

, Timms

, Barker

& Gasson

(2015) Retrospective assessment of Movement Disorder Society criteria for mild cognitive impairment in Parkinson’s disease. J Int Neuropsychol Soc, 21, 137–145.

28.

Hughes

, Daniel

, Kilford

& Lees

(1992) Accuracy of clinical diagnosis of idiopathic Parkinson’s disease: A clinico-pathological study of 100 cases. J Neurol Neurosurg Psychiatry, 55, 181–184.

29.

Folstein

, Folstein

& McHugh

(1975) “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res, 12, 189–198.

30.

Tomlinson

, Stowe

, Patel

, Rick

, Gray

& Clarke

(2010) Systematic review of levodopa dose equivalency reporting in Parkinson’s disease. Mov Disord, 25, 2649–2653.

31.

Antony

, Bieling

, Cox

, Enns

& Swinson

(1998) Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychol Assess, 10, 176–181.

32.

Muthén

& Muthén

, (1998-2015), Mplus User’s Guide, Muthén & Muthén, Los Angeles, CA.

33.

(2015) Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behav Res Methods, 1–14. doi: 10.3758/s13428-015-0619-7

34.

& Bentler

(1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling, 6, 1–55.

35.

Bentler

(1990) Comparative fit indexes in structural models. Psychol Bull, 107, 238–246.

36.

Steiger

(2007) Understanding the limitations of global fit assessment in structural equation modeling. Pers Individ Dif, 42, 893–898.

37.

Raykov

, Tomer

& Nesselroade

(1991) Reporting structural equation modeling results in Psychology and Aging: Some proposed guidelines. Psychol Aging, 6, 499–503.

38.

Akaike

(1974) A new look at the statistical model identification. IEEE Trans Automat Contr, 19, 716–723.

39.

Gorsuch

(1983) Factor analysis, L. Erlbaum Associates, Hillsdale, NJ.

40.

Revelle

& Zinbarg

(2009) Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma. Psychometrika, 74, 145–154.

41.

Bentler

& Mooijaart

(1989) Choice of structural model via parsimony: A rationale based on precision. Psychol Bull, 106, 315–317.

42.

Martinez-Martin

, Rodriguez-Blazquez

, Kurtis

, Chaudhuri

& Group oBotNV (2011) The impact of non-motor symptoms on health-related quality of life of patients with Parkinson’s disease. Mov Disord, 26, 399–406.

43.

Herer

, Arnulf

& Housset

(2001) Effects of levodopa on pulmonary function in Parkinson’s disease. CHEST J, 119, 387.

44.

Sabaté

, Rodriguez

, Méndez

, Enríques

, & Gonzélez

(1996) Obstructive and restrictive pulmonary dysfunction increases disability in Parkinson disease. Arch Phys Med Rehabil, 77, 29–34.

45.

Malek

, Grosset

, Stewart

, Macphee

GJA

& Grosset

(2013) Prescription of drugs with potential adverse effects on cardiac conduction in Parkinson’s disease. Parkinsonism Relat Disord, 19, 586–589.

46.

Orimo

, Uchihara

, Nakamura

, Mori

, Kakita

, Wakabayashi

& Takahashi

(2008) Axonal α-synuclein aggregates herald centripetal degeneration of cardiac sympathetic nerve in Parkinson’s disease. Brain, 131, 642–650.

47.

Chapuis

, Ouchchane

, Metz

, Gerbaud

& Durif

(2005) Impact of the motor complications of Parkinson’s disease on the quality of life. Mov Disord, 20, 224–230.

48.

Lovibond

& Lovibond

(1995) The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther, 33, 335–343.

49.

Hamilton

(1959) The assessment of anxiety states by rating. Br J Med Psychol, 32, 50–55.

50.

Beck

, Epstein

, Brown

& Steer

(1988) An inventory for measuring clinical anxiety: Psychometric properties. J Consult Clin Psychol, 56, 893–897.

51.

Leentjens

AFG

, Dujardin

, Pontone

, Starkstein

, Weintraub

, & Martinez-Martin

(2014) The Parkinson Anxiety Scale (PAS): Development and validation of a new anxiety scale. Mov Disord, 29, 1035–1043.

52.

Pachana

, Byrne

, Siddle

, Koloski

, Harley

& Arnold

(2007) Development and validation of the Geriatric Anxiety Inventory. Int Psychogeriatr, 19, 103–114.

53.

Fasano

, Visanji

, Liu

LWC

, Lang

& Pfeiffer

(2015) Gastrointestinal dysfunction in Parkinson’s disease. Lancet Neurol, 14, 625–639.