Abstract
There have been proposals to expand definitions for categorical disorders and dimensionally conceptualized syndromes (e.g., psychopathy) to include negative mood lability and dysregulation (NMD). Factor analytic results are often presented in support of these proposals, and we provide factor analytic demonstrations across clinically oriented samples showing that NMD indicators load strongly onto factors with a range of psychopathology. This is unsurprising from a transdiagnostic perspective but shows that factor analysis could potentially be used to justify expanding definitions for specific constructs even though NMD indicators show strong, nonspecific loadings on psychopathology factors ranging widely in nature. Expanding construct definitions and assessment approaches to emphasize NMD also may negatively impact discriminant validity. We agree that targeting NMD is essential for comprehensive assessment, but our demonstrative analyses highlight a need for using factor analysis and other statistical methods in a careful, theoretically driven manner when evaluating psychopathology structure and developing measures.
Negative mood lability and negative emotional dysregulation are listed as criteria for many disorders described in the Diagnostic and Statistical Manual of Mental Disorders (5th ed., text rev.; DSM-5-TR; American Psychiatric Association [APA], 2022), and both are linked strongly to many other disorders as well (Aldao et al., 2016; Brandes et al., 2019; Carver et al., 2017). Recognizing different viewpoints on definitions of constructs such as emotion dysregulation and the extent of its overlap with affective lability and negative mood symptoms (Carpenter & Trull, 2013; Thompson, 2019), we collectively refer to these interrelated constructs as negative mood lability and dysregulation (NMD) for the sake of convenience. In addition to NMD’s pervasive psychopathology associations, NMD dimensions are key treatment targets and associate robustly with relationship satisfaction, general functioning, and other factors relevant to treatment planning (Bodalski et al., 2019; Weiss et al., 2015).
In addition to NMD already being described as a feature of many disorders, researchers have proposed expanding the criteria for various DSM disorders to include or greater emphasize NMD features to ensure comprehensive assessment and treatment planning. This includes different research groups proposing that emotion dysregulation facets are core aspects of attention-deficit/hyperactivity disorder (ADHD) across childhood and adulthood (e.g., Hirsch et al., 2018; Vidal et al., 2014; Yue et al., 2022). Other examples include irritability and NMD being identified as core features of autism, and assessment measures have been refined and developed for assessing NMD in the context of ADHD and autism specifically (e.g., Mazefsky et al., 2018; Ward et al., 1993; Weibel et al., 2022). In the personality disorders (PD) literature, there is ongoing debate regarding the lack of representation of NMD features such as affective lability and irritability in the narcissistic PD criteria (Dimaggio, 2022). Vulnerable narcissistic dimensions representing different NMD elements already are widely studied across the clinical, personality, and social literatures, even though they do not feature prominently within the DSM NPD criteria (Miller et al., 2021). Related to narcissism research, research conducted outside of a DSM-based framework on dimensional externalizing syndromes suggests a need to better integrate NMD assessment in psychopathy research (Garofalo et al., 2020; Guo et al., 2022).
Factor Analysis: Applications in Negative Mood Lability and Dysregulation Research
Factor analytic results provide a significant portion of the basis for proposals focused on expanding disorder and syndrome definitions to capture NMD (e.g., Hirsch et al., 2018; Retz et al., 2012). Factor analytic methods also feature prominently in research on psychopathology classification, measure development, and assessment more broadly (Sellbom & Tellegen, 2019). In studies providing support for expanding construct definitions, items assessing NMD are often factor analyzed concurrently with items traditionally used to assess a domain. Both NMD and traditional item ratings often load strongly onto a common factor or load onto factors that are differentiable but still strongly correlated (Stanton & Zimmerman, 2018).
From a transdiagnostic perspective (Aldao et al., 2016; Carver et al., 2017), it is unsurprising that NMD indicators correlate and load strongly onto common factors with indicators of many different symptoms and traits. For instance, transdiagnostic studies from the perspective of the Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017) consistently indicate that NMD overlaps strongly with other psychopathology spanning HiTOP spectra ranging from detachment, to externalizing, to thought disorder, to internalizing, the last of which is unsurprising due to content overlap across NMD and other internalizing measures (Kotov et al., 2017). At the broadest level of analysis, general psychopathology factors modeled in studies often are heavily saturated with NMD-related variance (Southward et al., 2022), and one possible substantive interpretation is that a general psychopathology or “p-factor” represents tendencies toward responding impulsively to negative emotional experiences (Carver et al., 2017; also see Levin-Aspenson et al., 2021 for discussion of p-factor interpretation).
Description of Study Goals and Demonstrative Analyses
In this study, we demonstrate that NMD items load strongly onto common factors with a range of psychopathology to illustrate how factor analytic results potentially could be used to provide a rationale for broadening definitions of many different clinical constructs. Factor analysis is useful for identifying shared variance and for mapping psychopathology structure to be clear. However, interpretating the results of factor analyses of NMD and other individual constructs in isolation could lead to an increasing number of proposals focused on adding NMD to construct definitions. As reviewed, proposals to expand definitions already span DSM chapters and the clinical and nonclinical literatures. Expanding construct definitions may be warranted in some cases, as broadened disorder definitions may be helpful for increasing assessment comprehensiveness (e.g., adding NMD criteria to ADHD could be used to explicitly account for ADHD-borderline PD overlap; Hirsch et al., 2018; Retz et al., 2012). However, significant research indicates that boundaries between various clinical disorders and syndromes are already blurred, and adding nonspecific, transdiagnostic NMD features to different disorders may further complicate differential diagnosis. It also may result in discriminant validity issues more generally if measures used to assess different clinical constructs share similar, overlapping NMD item content (Evans et al., 2017; Stanton, 2020).
Acknowledging the complexities related to expanding construct definitions, we present a series of straightforward factor analytic demonstrations using data spanning sample types and assessment methods. This includes (a) interview ratings from adult outpatients (N = 2,149), and (b) self-ratings from adults (N = 429) recruited online who self-identified as having significant mental health histories. We refer to these samples going forward as Sample 1 (adult outpatients) and Sample 2 (adults recruited online), respectively.
We show that NMD indicators load strongly onto common factors with ratings relevant to domains already proposed to be expanded (e.g., various externalizing psychopathology). We also conduct factor analyses showing that NMD indicators load strongly onto common factors with other psychopathology in addition to those reviewed. Collectively, joint factor analyses of NMD with other symptoms across samples include commonly studied forms of psychopathology spanning ADHD, externalizing, thought disorder, and detachment. Although NMD has not been proposed as a key feature of some specific domains that we examine per se, these analyses provide the basis for discussing the importance of studying a range of psychopathology concurrently to permit strong examinations of specificity and discriminant validity. We also provide recommendations for using factor analysis in a theoretically driven manner when evaluating psychopathology structure and developing measures.
In addition, we present a second series of analyses reinforcing these recommendations. These analyses focus on examining external correlates for what we refer to as “standard” and “expanded” factors from our factor analyses with other forms of psychopathology already described as having NMD as a core feature (e.g., borderline PD and mania). Standard factors are based on factor analyses of traditional rating sets not including NMD indicators, and expanded factors are based on models including both traditional indicators and NMD ratings. These correlational analyses show that expanded factors correlate robustly with other external psychopathology measures in many cases. Such associations with these external measures assessing NMD-based forms of psychopathology are not surprising given content overlap across measures (e.g., both expanded factor and mania measures including NMD content). Still, they provide another demonstration of NMD’s lack of specificity and show how expanding construct definitions could negatively impact discriminant validity.
Method
Adherence to Ethical Standards and Data Availability for Study Samples
Procedures for data collection for both samples received Institutional Ethics Board approval, and all participants across samples provided their informed consent. Ethics board permission was not obtained to share the Sample 1 data on open-source forums, but these data are available upon request. Permission was obtained to share data from Sample 2, which are available here: https://osf.io/n67fd/?view_only=8a7a527f68d74779a51bedd525a643d6.
Sample 1: Adult Outpatient Sample
Sample 1 Procedure and Description
Interview data were drawn from a sample of 2,149 adult participants from the Rhode Island Methods to Improve Diagnostic and Assessment Services (MIDAS) Project (Zimmerman, 2016). These participants were seeking treatment at an outpatient psychiatry practice affiliated with an academic medical center, and interviews were administered at treatment intake. Mean participant age was 38.5 years (SD = 12.8). Most participants were women (61.0%; 39% men). The majority were White or European American (90.7%; 4.4% Black or African American; 1% Asian American; very small percentages of other identities endorsed), and 2.7% were Hispanic or Latino/x. Sample breakdown for highest level of education was 40.5% associate’s degree or some college; 21.9% high school degree or equivalent; 15.1% 4-year undergraduate degree; 14.2% advanced degree (e.g., master’s, PhD) or some graduate-level coursework; and 8.3% with less than a high school education.
Item Sets Used for Factor Analyses Based on the Sample 1 Interview Data
All items used in our factor analyses were item-level interview ratings of PD criteria from the Structured Interview for DSM-IV Personality (SIDP-IV; Pfohl et al., 1997). Interviewers rated participants on each criterion using a scale ranging from 0 (not present) to 3 (strongly present). Frequencies for all items are provided in Online Supplemental Table S1. The borderline PD ratings of affective instability (criterion 6) and intense anger/difficulty controlling anger (Criterion 8) were included as NMD markers. We included the anger rating in addition to affective instability because it captures both experiencing and difficulties regulating a specific form of negative affect. These two items were jointly analyzed with criterion-level ratings of (a) paranoid PD (seven items), (b) histrionic PD (eight items), and antisocial PD (seven items representing criterion A ratings). We focused on ratings of these three disorders to efficiently demonstrate that NMD ratings load strongly onto common factors with disorders differing in their nature and primary features (e.g., suspiciousness/mistrust vs. disinhibition and antagonism).
Other Measures Used for Examining External Factor Correlates in Sample 1
We examined factor correlations with current diagnostic ratings from the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-IV; First et al., 1995). These SCID ratings were not included in our factor analyses because they were coded as present or not present at the diagnostic level, whereas the SIDP ratings represent item-level data (one exception was that SCID item-level data were collected for major depression, which we do not examine here because NMD already is a central major depression feature).
Diagnostic ratings from the SCID used as external variables included (sample prevalence rates in parentheses): Major depression (43.0%), persistent depression (8.3%), social anxiety (26.8%), generalized anxiety (19.3%), panic disorder (17.7%), obsessive-compulsive disorder (6.4%), any bipolar spectrum disorder (6.1%), and intermittent explosive disorder (3.4%). We also included diagnostic ratings of borderline PD (9.5%) based on SIDP-IV scores. The MIDAS data set includes many other ratings that would have been difficult to integrate, and we focused on the disorders listed that are defined by NMD or linked strongly to it. All study interviews were conducted by PhD-level psychologists or research assistants with at least a bachelor’s level of education in psychology or a related field. Other publications using the MIDAS data set provide detailed information about the training process for interviewers and for interrater reliability, which was strong across a range of interview ratings (see Zimmerman, 2016).
Sample 2: Adults With a Self-Identified History of Mental Health Issues
Sample 2 Procedure and Description
Demonstrative analyses using self-ratings were based on data from 429 adults recruited online using the Prolific platform. Prolific provides screening filters for participant selection, and only participants who responded “yes” to the screening question “Do you have—or have you had—a diagnosed, on-going mental health/illness/condition?” were eligible to participate. With use of this recruitment approach, over three-fifths (62.2%) of the sample reported currently accessing psychiatric treatment (52.9% current medication; 32.4% current psychotherapy). Mean participant age was 32.8 years (SD = 11.7). Most participants were cisgender women (63.9%; 32.6% cisgender men; 3.5% nonbinary; 0.7% transgender women, 1.2% transgender men). Most participants identified as White or European American 82.8% (9.8% Black or African American; 7.2% Asian American; small percentage of other racial identities also were endorsed); 6.5% were Hispanic or Latino/x. The modal response for highest education level was having some college, an associate’s, or vocational degree (39.9%; 29.8% bachelor’s degree; 15.4% high school diploma or less; 14.5% post-bachelor’s level education; two missing responses).
Item Sets Used for Factor Analyses Based on the Self-Report Data From Sample 2
Features of NMD were assessed using the Affective Lability scale (six items) from the Comprehensive Assessment of Traits Relevant to Personality Disorder (CAT-PD; Wright & Simms, 2014). Item scores from this scale capture tendencies toward having (a) frequent mood swings and (b) behavioral disinhibition related to experiencing negative affect (e.g., “lose control when emotional”), which have been proposed as features to be added to various psychopathology domains. Items from these CAT-PD scales and others described subsequently were administered using a 0 (very untrue of me) to 4 (very true of me) response format.
The CAT-PD Affective Lability items were jointly analyzed with ratings of (a) ADHD symptoms and (b) behavioral rigidity. Symptoms of ADHD were assessed using the 18-item Adult ADHD Self-Report Scale (ASRS; Kessler et al., 2005), and participants rated their symptom experiences over the past year using a 5-point scale ranging 0 (never) to 4 (very often). Next, we assessed rigidity with the 12-item Rigidity scale (e.g., “others get frustrated by my inflexibility”) of the Broad Autism Phenotype Questionnaire (BAPQ; Hurley et al., 2007). The BAPQ is a widely used measure for assessing dimensions relevant to autism, and participants responded to BAPQ items using a 6-point scale ranging from 1 (very rarely) to 6 (very often). Finally, we conducted joint analyses of NMD ratings with ratings from the CAT-PD (a) Unusual Experiences (seven items: e.g., “feel outside of my body at times”), (b) Manipulativeness (six items), and (c) Social Withdrawal (six items) scales. Analyses based on these additional CAT-PD item sets demonstrate that NMD indicators show strong, nondiscriminant factor loadings onto common factors with indicators of other psychopathology differing in nature, in addition to loading strongly onto common factors with ADHD and rigidity as reviewed.
Additional Measures Used for Analyses Examining External Factor Correlates in Sample 2
Participants also completed other questionnaires assessing NMD and related dimensions (e.g., distress and fear). These measures were expected to correlate strongly with expanded factors as discussed, and we examined their correlates with factors to show potential negative impacts on discriminant validity if construct definitions are broadened to include NMD. Most of these measures assessed current symptoms rather than trait ratings, which was another reason for using them as external variables rather than including items from these measures in factor analyses. These additional measures included the Social Anxiety (six items), Mania (five items; “thoughts raced rapidly”), and Dysphoria (10 items; for example, “felt depressed”) scales from Expanded Version of the Inventory of Depression and Anxiety Symptoms (IDAS-II; Watson et al., 2012). Participants responded to IDAS-II items on a 1 (not at all) to 5 (extremely) scale in reference to the past 2 weeks. We used dimensional scores for each scale representing the sum of their respective item scores. Total Dysphoria scores exceeding a cutoff of a 28 indicate an increased likelihood of meeting criteria for an internalizing disorder (e.g., major depression; Stasik-O’Brien et al., 2019), and we also integrated this categorical variable. Roughly half of the sample participants (50.8%) exceeded this cutoff.
Analyses also included two other CAT-PD scales assessing internalizing psychopathology: Anxiousness (7 items) and Depressiveness (6 items). Finally, participants were administered the 13-item Mood Disorder Questionnaire (MDQ; Hirschfeld et al., 2000) to assess their lifetime histories of experiencing hypomania symptoms. Participants responded to the MDQ symptom items using a “yes” or “no” response format, and we summed all 13 MDQ item scores to create a dimensional score. Furthermore, endorsement of seven or more MDQ symptoms plus participants reporting that (a) several or more symptoms co-occurred and (b) symptoms caused at least moderate impairment indicates a positive hypomania screening history. Analyses also included this categorical variable, and 19.8% of participants had a positive MDQ screen. Mean values, standard deviations, and coefficient omega values (which all exceeded .85) for all study scales are presented in Online Supplemental Table S2.
Results
Mood Lability and Dysregulation Ratings Lack Specificity in Their Factor Loadings
Overview of Factor Analytic Approach
All factor analyses presented are exploratory factor analyses (EFAs) conducted using principal axis factoring. These EFAs were based on polychoric correlation matrices to account for using categorical item-level data. We focused on single-factor models in all instances to evaluate the extent to which NMD ratings loaded strongly onto common factors with other psychopathology, and use of confirmatory factor analysis would have yielded isomorphic results given our focus on single-factor models. For example, in Sample 2, we examined a single-factor model of NMD and ADHD ratings, a single-factor model of NMD and Rigidity ratings, and so on. We recognize that we could have examined multifactor models based on item sets spanning different measures and then adjudicated among different structural configurations (e.g., jointly analyzing NMD, ADHD, rigidity, and other ratings from Sample 2). However, examination of multifactor models spanning all forms of psychopathology assessed here may be better suited to address questions concerning NMD’s optimal classification within broader structural frameworks such as the HiTOP, which was not our study goal (for examples of studies addressing this research question, see Forbes et al., 2021; Stanton et al., 2023). Examination of multifactor structures spanning different forms of psychopathology also would have been complicated and unwieldy. For example, in Sample 2, this would have involved factor analyzing over 50 item ratings providing coverage of some psychopathology domains much more so than others (i.e., 18 ADHD items, 12 assessing Rigidity, over 20 CAT-PD items). In addition to being straightforward, our analyses focused on single-factor models parallel cases where factor analytic results are used as evidence for expanding a specific psychopathology domain, an approach we review critically following presentation of our demonstrative analyses.
Factor Analytic Results in Sample 1
Prior to examining joint factor models, we examined initial models using only the ratings traditionally used to score each PD as discussed (e.g., examining a one-factor model of the seven antisocial PD ratings). These standard models based on ratings of paranoid, antisocial, and histrionic PDs performed as expected: All items loaded >.50 on their respective factors, with the exception of the criterion 7 rating for histrionic PD loading .38 on its respective factor. When examining expanded factor models, affective lability and anger ratings had strong loadings (all loadings ≥ .56) on common factors with ratings of each PD as shown in Table 1.
Factor Loadings for Expanded Single Factor Models Using the Personality Disorder Interview Data from Sample 1
Note. N = 2,149. Affective lability and anger items are bolded. All items are from the Structured Interview for DSM-IV Personality. PD = Personality Disorder. Paraphases for the criterion ratings shown above are based on criteria descriptions provided in the fourth and fifth editions of the Diagnostic and Statistical Manual of Mental Disorders.
Factor Analytic Results in Sample 2
Initial standard factor models conducted without NMD ratings also conformed to expectations in the Sample 2 self-report data. All ADHD items and all CAT-PD items (Unusual Experiences, Manipulativeness, and Social Withdrawal) had strong loadings on their respective factors (all loadings ≥ .55). All 12 BAPQ Rigidity items loaded ≥ .48 on a common factor.
Factor loadings for analyses including NMD item ratings are shown in Table 2 (ADHD and rigidity examples) and Table 3 (other examples based on CAT-PD scale items). Across the results presented in Tables 2 and 3, the NMD items “unpredictable moods,” “frequent mood swings,” “lose control when emotional,” and “overreact to everything” loaded >.50 on all factors. Factor loadings were substantially stronger in many cases, as items assessing unpredictable moods and having frequent mood swings loaded .74 and .73, respectively, on a common with the CAT-PD Unusual Experiences items, for example.
Factor Loadings for Expanded Single Factor Models Using Self-Report Data from Sample 2
Note. N = 429. All affective lability items are bolded and are from the Comprehensive Assessment of Traits Relevant to Personality Disorder. All items with an asterisk (*) are reverse-keyed items.
Factor Loadings for Additional Single Factor Models Using Self-Report Data From Sample 2
Note. N = 429. All items are from the Comprehensive Assessment of Traits Relevant to Personality Disorder. All affective lability items are bolded, and items with an asterisk (*) are reverse-keyed items.
It is worth acknowledging that reverse-keyed NMD items including “know how to cope” and “remain calm when stressed out” that index positive coping skills tended to have weaker loadings across analyses than did other NMD items. Consistent with this, reverse-keyed items from other scales also showed weaker loading patterns. For example, reverse-keyed BAPQ Rigidity items such as “am comfortable with unexpected changes” also had comparatively weaker loadings than most other BAPQ items, as shown in Table 2. Regardless of these issues concerning reverse-keyed items, these analyses provide strong evidence overall that NMD items load strongly on joint factors spanning a wide range of psychopathology.
External Factor Associations: Reduced Specificity for Expanded Factors
External Factor Correlates in the Sample 1 Interview Data
All standard and expanded factors were modeled using regression-based factor scores in subsequent analyses examining their external correlates across samples. First, we examined Sample 1 standard and expanded factor associations with diagnostic ratings, which are provided in Table 4. 1 Correlations for both standard and expanded factors were relatively weak in some cases (e.g., correlations with depressive disorder diagnoses). Examples of stronger expanded factor correlations included the expanded Histrionic PD factor correlating .55 with borderline PD, whereas the traditional Histrionic PD factor correlated only .37 with borderline PD. Similar increases in correlation magnitude were observed for the expanded Paranoid PD and Antisocial PD factors compared to their corresponding standard factors. These increases in correlation magnitude with borderline PD are not surprising because the borderline PD affective lability and anger ratings were included in expanded factor models. Still, this pattern of results illustrates how adding nonspecific features to different disorders potentially could increase disorder overlap and have an adverse impact on discriminant validity. Other examples not involving borderline PD included the correlation for expanded Paranoid PD with intermittent explosive disorder being .10 greater in magnitude than the correlation for standard Paranoid PD and intermittent explosive disorder (.30 vs. .20; see Table 4).
Personality Disorder Factor Correlations With Diagnostic Ratings in Sample 1
Note. N = 2,149. All correlations are polyserial correlations. Bolded correlations are significant at a p < .001 level, and correlations > .25 are underlined. OCD = obsessive–compulsive disorder; PD = personality disorder.
External Factor Correlates in the Sample 2 Self-Report Data
Sample 2 correlations for both standard and expanded factors with other study variables are shown in Table 5 (ADHD and Rigidity correlations) and Table 6 (correlations for other factors). These results indicated that standard factors generally showed weak-to-moderate external correlations, although some exceptions involved standard ADHD correlating strongly with IDAS-II Dysphoria, IDAS-II Mania, and MDQ scores. In contrast, expanded factors showed strong correlates in many cases, demonstrating how inclusion of NMD inflates the magnitude—and thereby the nonspecificity—of external correlations.
Correlations for Attention-Deficit/Hyperactivity and Rigidity Factors With Other Study Variables in Sample 2
Note. N = 429. Correlations that differed by ≥.10 across columns being compared are bolded, and correlations differing by ≥.15 across columns are bolded and underlined. All correlations ≥.20 were significant at a p < .001 level. Correlations for factors with continuous variables are Pearson correlations; factor correlations with categorical variables are polyserial correlations. ADHD = Attention-Deficit/Hyperactivity Disorder; CAT-PD = Comprehensive Assessment of Traits Relevant to Personality Disorder; IDAS-II = Expanded Version of the Inventory of Depression and Anxiety Symptoms; MDQ = Mood Disorder Questionnaire.
Correlations for Thought Disorder, Antagonism, and Social Withdrawal Factors With Other Study Variables in Sample 2
Note. N = 429. Correlations that differed by ≥ .20 across columns being compared are bolded. Correlations for factors with continuous variables are Pearson correlations; factor correlations with categorical variables are polyserial correlations. Manipulate = Manipulativeness; CAT-PD = Comprehensive Assessment of Traits Relevant to Personality Disorder; IDAS-II = Expanded Version of the Inventory of Depression and Anxiety Symptoms; MDQ = Mood Disorder Questionnaire.
Focusing on Table 5 specifically, correlation increases for expanded versus standard ADHD were notable in some cases (e.g., .10 differences with CAT-PD Anxiousness). Differences for expanded versus standard ADHD were minor in other instances, however, which may have been due in part to the standard ADHD factor already correlating strongly with a number of external variables as reviewed. The differences for expanded versus standard Rigidity (also shown in Table 5) were more consistent and pronounced: The expanded Rigidity factor had several strong correlates (e.g., .61 with CAT-PD Anxiousness), and seven of the eight external correlates for expanded Rigidity were ≥.15 stronger in magnitude than corresponding correlations for standard Rigidity.
Differences in the magnitudes of correlates shown in Table 6 for expanded and standard factors were even more drastic in many cases. Salient examples include standard Social Withdrawal correlating .39 with CAT-PD Anxiousness but expanded Social Withdrawal correlating .63 (.24 increase). More generally, expanded Social Withdrawal showed strong, nonspecific associations with many other Table 6 variables (e.g., CAT-PD Depressiveness and IDAS-II Dysphoria). In contrast, standard Social Withdrawal showed relative specificity in a theoretically consistent manner via its strong associations with IDAS-II Social Anxiety and CAT-PD Depressiveness (rs = .50 and .47, respectively), but very weak associations with other variables such as IDAS-II Mania (r = .11). Another prominent example included expanded Manipulativeness correlating .49 with categorical IDAS-II Dysphoria screening scores, which nearly doubled the correlation for standard Manipulativeness (r = .25).
Discussion
Review of Study Goals and Demonstration Results
Our demonstrative analyses showed that ratings of NMD features loaded strongly onto common factors with items used to assess an array of other clinical constructs across samples and assessment methods. Acknowledging that some expanded factor correlates in the Sample 1 data were quite weak in magnitude, expanded factors including NMD correlated moderately to strongly with other external study measures in many cases. Specific examples that may be problematic theoretically and indicative of discriminant validity issues included expanded Manipulativeness correlating robustly with dysphoric mood and anxiousness. Examples such as this from our correlational analyses demonstrate how expanded assessment approaches targeting NMD may result in a loss of specificity, with these results complementing our factor analyses showing that NMD loads strongly onto common factors with ratings spanning externalizing traits, paranoia, thought disorder, rigidity, ADHD symptoms, and detachment.
As discussed, these findings are not surprising given systematic evidence showing that NMD features correlate strongly with many symptoms and disorders (Aldao et al., 2016; Carver et al., 2017). Expanding construct definitions and emphasizing NMD in assessment may result in different forms of psychopathology being challenging to differentiate, which could exacerbate long-standing difficulties concerning diagnostic comorbidity and differential diagnosis. However, we recognize differing viewpoints on the costs and benefits of expanding construct definitions to include NMD, as doing so may be necessary for ensuring NMD is adequately addressed in assessment and treatment. Next, we provide recommendations for interpreting factor analytic results, designing studies, and clinical assessment intended to be useful to researchers from different backgrounds and with differing viewpoints on NMD proposal expansions.
Although our recommendations focus on NMD, many of the recommendations shared subsequently would also potentially apply to studying other nonspecific features. For example, relationship difficulties and low levels of conscientiousness, which might represent having difficulty navigating daily goals, also are linked to many psychological disorders to varying degrees (Carver et al., 2017; Watts et al., 2021). Therefore, we hope that our recommendations will be useful for drawing attention to discriminant validity issues that are often neglected more generally across substantive clinical research areas (Stanton, 2020; Watts et al., 2021).
Considerations When Using Factor Analysis in Studies of Negative Mood Dysregulation
Interpreting Factor Analytic Results in Context
Before providing recommendations about interpreting factor analytic results carefully, we would like to note that our goal is not at all to suggest that factor analysis is not useful for informing assessment and advancing understanding of psychopathology structure. We commonly apply factor analysis in our own work and agree with perspectives that measurement problems often arise due to neglecting to explicate domain and measure structure prior to using new assessment tools (Flake et al., 2017). Rather, we build from other recent reviews describing the need to use factor analysis in a thoughtful, theoretically driven manner (e.g., Fried, 2020; Watts et al., 2020) to offer ideas when using factor analysis to study NMD specifically.
First, one primary consideration centers on the interpretation of NMD items loading strongly onto common factors with other psychopathology, especially because NMD shows such nonspecific factor loadings as demonstrated. Keeping this lack of specificity in mind raises the question of how clearly and compellingly a latent variable representing a single disorder entity accounts for shared variance across NMD and other items (also see Fried, 2020; Watts et al., 2021 for discussion on distinctions between theoretical and statistical models). Furthermore, items may load strongly onto common factors and/or a model may fit reasonably well even in the absence of having a clear psychological explanation for why those specific patterns of results occurred (Fried, 2020; Sellbom & Tellegen, 2019; Watts et al., 2021). This was the case in some of our analyses. Although plausible interpretations could be generated, we examined joint factor models of NMD and some other forms of psychopathology such as detachment even when we did not specify clear theoretical reasons for doing so; despite this, factor loadings for all items generally were quite strong in magnitude. Therefore, we encourage researchers to connect data analysis and theory by clearly describing upfront how specific analytic techniques will be used to test predictions about NMD’s placement with domain structures (see Fried, 2020; Greene et al., 2022 for more detailed recommendations).
To be clear, some proposals already provide theoretical explanations for why NMD is important to study in relation to specific disorders or syndromes (e.g., some proposals on narcissism and ADHD). For example, some proposals focused on expanding narcissistic PD definitions describe that intense negative emotional experiences may occur due to interpersonal sensitivity and when unrealistic expectations of others are not met, among other factors (Caligor et al., 2015; Dimaggio, 2022). Researchers from different backgrounds (e.g., psychiatry vs. personality psychology) still disagree on narcissism definitions in some ways, but theoretically driven narcissism research has facilitated meaningful dialogue (Ackerman et al., 2019).
The Value in Studying a Range of Psychopathology Concurrently
Even if researchers have clear theoretical reasons for studying NMD features in relation to a specific disorder or set of symptoms, we still believe it is beneficial to examine NMD’s overlap with a range of psychopathology concurrently when possible. For example, even if researchers predict that NMD aligns particularly strongly with symptoms of one disorder theoretically, it may be possible that the NMD features in question still show moderate to strong loadings on common factors with other symptoms based on our demonstrations and the extensive NMD literature to date (Carver et al., 2017). When designing studies from a dimensional perspective, this could include jointly analyzing NMD with a specific construct of interest along with internalizing and disinhibition ratings. Most NMD features are described as internalizing in nature within models such as the HiTOP, and disinhibition also is strongly linked to some aspects of NMD (Forbes et al., 2021; Stanton et al., 2023); therefore, including measures of these psychopathology within analyses permits strong tests of specificity (e.g., do proposed NMD features overlap more strongly with a targeted construct of interest or with internalizing).
From a DSM-focused lens, assessing a range of symptoms spanning multiple disorders, and preferably multiple diagnostic chapters, provides data analytic opportunities not otherwise possible. For instance, if symptoms spanning ADHD, PDs, major depression, and bipolar spectrum disorders are assessed concurrently, the comparative degree of overlap for specific NMD features across disorders can be determined as reviewed. This allows researchers to potentially draw clearer conclusions regarding the extent to which specific NMD features are important characteristics of some disorders more than others (e.g., a specific NMD manifestation may be more characteristic of ADHD than internalizing disorders). In contrast, studying NMD in relation to other individual disorders or narrow sets of symptoms makes it challenging to determine the extent to which NMD is a core feature of one type of psychopathology more so than another. These concepts presented are not new (e.g., Forbes et al., 2021; Gros et al., 2011; Stanton et al., 2020), but they are worth emphasizing given that transdiagnostic features such as NMD continue to be the target of proposals focused on expanding the boundaries of individual syndromes.
Implications for Using Factor Analysis to Inform Measurement Approaches
Possible Impacts of Expanding Assessment Measure Content
Given that factor analytic results commonly are used inform measure development, our next series of considerations focus on how increasing an emphasis on NMD content within measures may affect assessment in research and practice. First, if disorder or syndrome definitions are expanded to include NMD, a key consideration focuses on how much NMD will be emphasized within specific disorder definitions and measures used to assess expanded constructs. For example, correlations for the expanded Social Withdrawal factor were much stronger than the correlations for standard Social Withdrawal, whereas differences for expanded versus standard ADHD were less pronounced. Items assessing NMD constituted six of 12 items used for the joint factor analysis with standard Social Withdrawal items, but only six of 24 total items in the joint analysis with ADHD.
Thus, the addition of NMD to the expanded model for ADHD may have had less of an impact due at least in part to NMD items representing a much smaller portion of the overall item set than in the case of expanded Social Withdrawal. Evidence supporting this interpretation includes the standard ADHD factor already having moderate to strong correlations with many other psychopathology variables as reviewed (e.g., correlations > .50 and .60 with MDQ and IDAS-II Dysphoria scores, respectively). In contrast, other standard factors based on smaller item sets in Sample 2 generally had weak correlates with external variables, allowing more range for correlations to inflate when adding NMD content to expanded factors. Still, these patterns of findings could be interpreted as providing more support for adding NMD-based features to ADHD more so than other domains. There may be a stronger empirical basis for adding NMD to some construct definitions than others, and testing different methods for expanding disorder conceptualizations would be helpful for informing possible criteria changes (e.g., adding a single versus multiple NMD symptoms to diagnostic criteria).
Related to these points, the degree to which NMD additions impact differential diagnosis and discriminant validity also could depend meaningfully on the sequencing of assessment approaches. For example, if NMD was described as a core feature or “gateway” criterion of a disorder, that may have significantly more impact than if NMD features were added as symptom criteria but were not required to be present for diagnostic criteria to be met. Expanded factors from Sample 1 included only two NMD ratings, and correlations for expanded factors in this sample with internalizing diagnoses were weak in many cases. Therefore, relatively minor NMD additions may not necessarily have a substantial impact on diagnosis.
Unfortunately, analyses focused on evaluating different symptom scoring configurations for making diagnoses and determining clinical cutoff scores were not possible to conduct here. Sample 2 participants only completed self-report measures, precluding examination of how different self-report cutoff scores were linked to diagnoses made by clinicians. In regard to the Sample 1 procedures, clinicians did not have access to alternative disorder descriptions when assigning diagnoses, such that we could not examine how use of expanded disorder descriptions including NMD would influence clinician perceptions (e.g., if affective lability was described as core to antisocial PD, this could influence how clinicians scored antisocial PD interview ratings).
Assessment Approaches Recognizing Both Specific and Nonspecific Features
One possible method for accounting for NMD in assessment would be to guide measurement approaches using “hybrid” diagnostic models balancing comprehensiveness and specificity (Gros et al., 2011). Our use of the term “hybrid” here differs from its use in PD research on the Alternative Model for Personality Disorders; in this case, “hybrid” here refers to identifying both disorder-specific and general, nonspecific features when assessing a specific syndrome (Allsopp et al., 2019; Gros et al., 2011). For example, from this hybrid perspective, affective lability could be added as a criterion to antisocial PD but recognized as a non-specific, transdiagnostic criterion or specifier that is not sufficient in and of itself for diagnosis. Thus, antagonism and disinhibition would remain primary to antisocial PD, but NMD could still be recognized if comprehensiveness was the goal.
Frameworks such as the HiTOP are intended to address these issues concerning disorder overlap and poor interrater reliability for many diagnoses (Kotov et al., 2017), and NMD features are classified primarily within the internalizing spectrum of the HiTOP as discussed. When using the HiTOP, diagnosticians would determine the extent to which scores on subscales assessing NMD dimensions such as affective lability are elevated, rather than assessing NMD as a characteristic of different disorders. However, evidence directly showing that dimensional models improve clinical outcome to a sufficient degree to warrant a paradigm shift remains limited (Zimmerman, 2021), and HiTOP-based research approaches continue to be criticized by some for being atheoretical in nature, among other reasons (Haeffel et al., 2022; see DeYoung et al., 2022 in response).
Limitations, Future Directions, and Conclusion
Several limitations and related future directions in addition to those already discussed are important to consider. First, although NMD features consistently show non-specific, robust associations with many other forms of psychopathology in child and adolescent samples (Brandes et al., 2019), our demonstrative analyses focused on adult samples. It would have been interesting to address these issues in other sample types, as there has been significant controversy around the creation of NMD-based disorders such as disruptive mood dysregulation disorder that are diagnosed in childhood (Evans et al., 2017).
Longitudinal research would be useful for understanding how NMD influences other symptoms (and vice versa) and for testing different interpretations of spectra dimensions possibly accounting for NMD’s overlaps with other forms of psychopathology (e.g., the p-factor at the broadest level; the internalizing spectrum as a “liability” for experiencing negative mood symptoms). Relatedly, longitudinal methods also could inform understanding of the extent to which NMD trajectories are similar over time and across contexts for individuals with different diagnoses. For example, some studies indicate potential differences on measures of dispositional affective intensity and lability across diagnostic groups, such that expanding construct definitions to emphasize specific NMD features may actually be useful for disorder differentiation (e.g., elevated emotional lability observed in the context of ADHD could help to distinguish ADHD from other disorders; Vidal et al., 2014). In constrast, other research using ecological momentary assessment methods indicates that mood experiences and regulation are similar overall across ADHD and other disorders (Moukhtarian et al., 2021). Regarding other study design limitations, all factor and external measure associations were monomethod (i.e., only self-ratings were used in Sample 1; only interview ratings in Sample 2). Self-report assessment may have advantages for assessing internal negative emotional experiences, and observational and informant methods offer incremental information complementary to self-report, particularly when assessing outward expressions of NMD (e.g., expressions of anger and frustration; Ali et al., 2022). Therefore, integrating multiple assessment methods in addition to incorporating longitudinal designs would be optimal for informing NMD conceptualizations and examining NMD’s specificity.
Acknowledging these future directions, our factor analytic and correlational demonstrations highlight a need to be mindful of the non-specific nature of NMD when designing studies and interpreting results. Assessing NMD in treatment planning and research studies provides key information that would be costly to ignore. However, focusing heavily on NMD when assessing different disorders and syndromes could negatively impact differential diagnosis and discriminant validity, and study designs continue to vary substantially in the extent to which NMD and related dimensions are studied as disorder-specific versus transdiagnostic. We hope that our recommendations will be useful for guiding research design and data analysis when studying NMD and other nonspecific psychopathology features.
Supplemental Material
sj-docx-1-asm-10.1177_10731911231174471 – Supplemental material for Negative Mood Dysregulation Loads Strongly Onto Common Factors With Many Forms of Psychopathology: Considerations for Assessing Nonspecific Symptoms
Supplemental material, sj-docx-1-asm-10.1177_10731911231174471 for Negative Mood Dysregulation Loads Strongly Onto Common Factors With Many Forms of Psychopathology: Considerations for Assessing Nonspecific Symptoms by Kasey Stanton, Kennedy M. Balzen, Christine DeFluri, Peyton Brock, Holly F. Levin-Aspenson and Mark Zimmerman in Assessment
Footnotes
Authors’ Note
Please note that the ideas appearing in this manuscript have not been disseminated previously.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
