Comparing Measures of Criterion A to Better Understand Incremental Validity in the Alternative Model of Personality Disorders

Abstract

The Alternative Model of Personality Disorders distinguishes between the severity of personality dysfunction (Criterion A) and individual differences in personality disorder expression (Criterion B). Several Criterion A measures exist, but few studies have compared these measures with each other. Moreover, debates about whether the constructs of Criteria A and B are redundant (i.e., weak incremental validity) should be framed around how different Criterion A measures perform relative to others. This study of 204 undergraduate students evaluated multiple measures of Criterion A. These measures were strongly correlated with Criterion B, but evidenced incremental validity (39% of outcomes, 5% average additional variance explained) with outcomes of psychopathology and interpersonal impairments, and less consistent incremental validity with suicidality, aggression, and mental health utilization. We discuss how these results inform the construct of Criterion A relative to Criterion B and evaluate strengths/weaknesses of Criterion A measures.

Keywords

Criterion A Level of Personality Functioning Scale personality disorders multi-method

The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5; American Psychiatric Association [APA], 2013) includes an alternative model for personality disorders (AMPD). This alternative model distinguishes between the severity of personality dysfunction (Criterion A) and the stylistic expressions in which personality disorder manifests (Criterion B). Criterion A is conceived as a single dimension reflecting self and interpersonal impairment, a description that accords with psychodynamic, interpersonal, personological, and attachment theories (Bender et al., 2011; Mulay et al., 2018; Pincus & Roche, 2019). Criterion B is conceived as a pathological trait model, usually represented through five pathological personality factors that link well with conceptualizations from other pathological trait models (Van den Broeck et al., 2014; Wright & Simms, 2014), normal personality trait models (Helle et al., 2017; Suzuki et al., 2015), and Research Domain Criteria (RDoC) constructs (Harkness et al., 2014) and also appear to capture the larger metastructure of psychopathology (Krueger, 2013; Wright & Simms, 2015). In the DSM-5, Criterion B is operationalized using the Personality Inventory for DSM-5 (PID-5; APA, 2013) or the brief form (PID-5-BF; APA, 2013).

The AMPD has received increasing research interest (Milinkovic & Tiliopoulos, 2020; Widiger et al., 2019; Zimmermann et al., 2019) and the World Health Organization accepted a similar model in the International Classification of Diseases (11th edition; ICD-11; World Health Organization, 2020). Yet, fundamental questions still remain in this new conceptualization of personality disorder. Specifically, how to best assess these dimensions of psychopathology and whether Criterion A is redundant with Criterion B. As such, this research examines several measures of Criterion A to determine their reliability, overlap, and validity, including whether any measure of Criterion A demonstrates sustained incremental validity over Criterion B in relation to important outcomes.

Assessment of Criterion A

Criterion A is operationalized in the DSM-5 with the Level of Personality Functioning Scale (LFPS; APA, 2013). It was originally designed to be a clinician-rated instrument and, for several years, there was not a self-report version directly operationalizing Criterion A, which led to research support of Criterion A lagging behind Criterion B (Roche et al., 2016). Now there are at least five self-report measures and two interview-rated measures of Criterion A (each reviewed below). Some of these measures have just been developed in recent years, and as such, there are few studies examining these measures, and even fewer comparing them. Yet this work is crucial in determining how choices of measurement may impact the validity of Criterion A reflected in the literature.

Incremental validity addresses the question of redundancy in the AMPD model as many have noted that there is conceptual and empirical overlap within constructs in Criteria A and B (e.g., Widiger et al., 2019). This is typically accomplished by adding Criterion B predictors in first, and then Criterion A predictor(s) in second, and vice versa, to examine additional variance explained in the second step. Across many studies, the trend is that Criterion A has less consistent evidence of incremental validity compared with Criterion B.

Measurement matters when assessing for overlap, as the choice of outcomes and assessment instruments for Criteria A and B factors into how redundant a set of constructs appear. Research that uses Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV; APA, 1994) personality disorders as the key outcome finds less consistent evidence of incremental validity for Criterion A (Few et al., 2013; Hentschel & Pukrop, 2014; Hopwood et al., 2012), compared with outcomes that assess global impairment (e.g., Zimmermann et al., 2014).

For Criterion B, several studies have shown that using a normal range personality measure increases the likelihood of finding incremental validity of Criterion A, whereas using pathological traits decreases this likelihood (e.g., Berghuis et al., 2012; Hentschel & Pukrop, 2014; but see Sleep et al., 2020). Morey et al. (2020) demonstrated that pathological personality traits are saturated with global personality pathology that is not trait-specific, which may be why it is difficult to find incremental validity with measures of Criterion A. How to best address this overlap (e.g., only use Criterion B, replace Criterion B with normal range trait measures, or tolerate conceptual redundancy within a model) is an ongoing debate in the field.

For Criterion A, some measures may result in stronger criterion/incremental validity compared with other measures. When this occurs, it may reflect differences in the measures (e.g., better psychometrics) and/or it may reflect concerns about the clarity of the underlying construct. The choice of measurement tools can be based on a variety of factors (e.g., lab preferences, measure length, popularity of a measure, and translations available), which may lead to a situation where researchers fail to find incremental validity for Criterion A due to the measure that was used (Type II error). Alternatively, a research team may report incremental validity for the construct of Criterion A due to the measure used, when that finding may not replicate across most other Criterion A measures (Type I error). Studies designed with multiple measures of Criterion A are well positioned to examine these discrepancies and facilitate the synthesis of accumulating research on the validity of Criterion A as a construct. Next, we review the research to date on the validity and incremental validity of several Criterion A measures, organized by each measure, as well as the few studies that measure multiple Criterion A measures together.

LPFS–Self Report (LPFS-SR)

The LPFS-SR is an 80-item self-report measure that is strongly correlated with other global measures of personality impairment (Morey, 2017). The LPFS-SR is associated with interpersonal problems and personality disorders, and is also strongly correlated with normal range personality traits (rs ranging around .5) and pathological personality traits (rs = .6–.7; Hopwood et al., 2018; see also Hemmati et al., 2020). Higher LPFS-SR at baseline was significantly associated with greater daily experiences of negative affect, stress, and invalidation (Heiland et al., in press). Sleep and colleagues (2020) examined incremental validity for the LPFS-SR, finding that the LPFS-SR was able to increment normal range and pathological personality traits (PID-5) for the outcomes of personality disorder. The additional variance explained by Criterion A ranged from 3% to 13%, although the authors of this study viewed this additional variance as small and argued that it was evidence against meaningful incremental validity of Criterion A. In an ecological momentary assessment study, the LPFS-SR was associated with higher negative affect, lower positive affect, cold/submissive behavior, and perceptions of coldness in others, along with variability in most of these variables. After including the PID-5 dimensions, the LPFS-SR remained significant for lower dominant behavior and higher variability in dominant behavior, higher negative affect, and lower positive affect with higher variability in positive affect (Ringwald et al., 2021).

LPFS–Self-Report of Criterion A (LPFS-SRA)

The LPFS-SRA is a 12-item self-report measure that uses the verbatim language of the LPFS in the DSM-5 and asks participants to rate themselves. The LPFS-SRA has been related to interpersonal problems (Dowgwillo et al., 2018) and pathological personality traits (rs = .2–.5), along with identity diffusion, defensive styles, and attachment insecurity (Roche et al., 2018). In a 14-day diary study, the LPFS-SRA evidenced small (3%–5% additional variance) incremental validity over the PID-5 in explaining variance in daily ratings of self and interpersonal dysfunction (Roche et al., 2016). In a different 14-day diary study measuring Criterion A (LPFS) and Criterion B (PID-5-BF) daily, there was evidence of incremental validity (5%–11%) for Criterion A across daily outcomes of self-functioning, emotionality, thinking, awareness/regulation, and interpersonal domains (Roche, 2018). In particular, Criterion A was associated with all of these outcomes reflecting broad dysfunction, whereas the PID-5-BF domains were related to specific impairments (e.g., disinhibition and daily thinking difficulties, detachment/antagonism, and daily interpersonal problems), which reflects the goal of the AMPD to capture global impairment (Criterion A) and more specific manifestations of dysfunction (Criterion B).

LPFS–Brief Form 2 (LPFS-BF2)

The LPFS-BF2 is a 12-item self-report measure. The LPFS-BF2 was positively associated with a number of personality disorders, pathological personality styles (using the severity indices of personality problems), and global psychopathology symptoms (brief symptom inventory total score; Weekers et al., 2019). The LPFS-BF2 was also positively associated with measures of well-being, global symptom distress, and dysfunctional schema modes, and there was evidence that the LPFS-BF2 incremented the PID total score for well-being (.08 additional variance), symptom distress (.07 additional variance), and dysfunctional schemas (ranging from .03 to 0.10 additional variance when a given schema outcome was significantly incremented by Criterion A relative to Criterion B; Bach & Hutsebaut, 2018).

Self and Interpersonal Functioning Scale (SIFS)

The SIFS (Gamache et al., 2019) is a relatively new 24-item self-report measure. In this original study, the SIFS was related to low self-esteem, low satisfaction with life, lower personality organization, low empathy, pathological narcissism, anger, poorer mental and general health, and was correlated strongly (rs = .49–.81) with the PID-5-BF. In community (n = 280) and clinical (n = 194) samples, incremental validity (over the PID-5) ranged from 0.02 to 0.16 (M = 0.08), with larger increments found for low perspective-taking (0.16), low satisfaction with life (0.10), low self-esteem (0.09), and motor impulsivity (0.09), for the SIFS (Gamache & Savard, 2019).

DSM-5 Levels of Personality Functioning Questionnaire (DLOPFQ)/DLOP-SF

The DLOPFQ is a 132-item measure (Huprich et al., 2018), with a shorter 24-item form (DLOP-SF; Siefert et al., 2020). The DLOPFQ was significantly related to insecure attachment, dependency, and lower well-being, correlated around 0.6 with pathological personality traits, but found incremental validity (additional variance explained ranging from 0.03 to 0.14) compared with the PID-5 in its association with attachment, dependency, and well-being (Huprich et al., 2018). The shorter form was associated with interpersonal problems, lower well-being, and insecure attachment style (Siefert et al., 2020).

Informant Measures

The original informant/clinician-rated measure is the LPFS, which has been used by clinicians with higher scores relating to greater PD symptoms, lower social functioning, risk composite, and level of care rating (Morey et al., 2013). Lay raters have also been able to use this measure with adequate inter-rater reliabilities (average rater intraclass correlation coefficient [ICC] = .7–.9), and it similarly relates to PD diagnosis, Axis I diagnoses, and psychodynamic and interpersonal variables (Roche et al., 2018; Zimmermann et al., 2014). Regarding incremental validity, Few and colleagues (2013) examined clinician-rated measures of the LPFS and pathological traits, finding that Criterion A did not increment Criterion B in associations with personality disorders.

Interviews have also been designed to specifically address the Criterion A construct. The semi-structured interview for personality functioning scale of DSM-5 (STiP-5.1; Hutsebaut et al., 2014) is a 45-minute interview, with adequate average inter-rater reliability (.81–.92), and is related to personality disorders, pathological personality traits, global psychopathology symptoms as assessed by the brief symptom inventory total score, and severity of personality problems (Berghuis et al., 2013; Hutsebaut et al., 2017). Zettl and colleagues (2019) found positive associations with personality disorder symptoms and that scores differentiated whether a patient was diagnosed with a PD or not. The STiP was correlated with pathological personality traits (PID-5) in around the 0.2 range except for disinhibition (r = .45; Weekers et al., 2021).

The Structured Interview for Personality Disorders Module I (SCID-AMPD) is a 45- to 90-minute interview (Bender et al., 2018), with good inter-rater reliability (average inter-rater reliability = .93) and associations with number of suicide attempts, psychiatric hospitalizations, and severity of diagnoses (Kampe et al., 2018). The interview effectively distinguishes between patients diagnosed with a PD or not (Buer Christensen et al., 2020). We are unaware of any research examining the incremental validity of these Criterion A interviews over and above Criterion B traits to predict outcomes.

Comparing Criterion A Measures

McCabe et al. (2021) noted that no previous research had compared self-report measures of Criterion A, and then they examined subscales of the LPFS-BF2, DLOPFQ, and LPFS-SR, finding these subscales shared substantial overlap (rs > .5) and were also strongly associated with the five-factor model. The Criterion A subscales were more strongly related within a measure than between different measures purportedly measuring the same construct (e.g., empathy), which they evaluated as poor discriminate validity at the subscale level. A few studies have examined cross-method comparisons of Criterion A. Specifically, the STiP was positively related to the LPFS-BF2 (r = 0.49; Weekers et al., 2021) and the LPFS-SRA was positively related to the LPFS–observer-rated (LPFS-OR) measure (r = .27; Roche et al., 2018). This study also published subscale-level data, and consistent with McCabe and colleagues (2021), subscales were not aligning well across method, evidencing poor discriminant validity.

A comparative content analysis for self-reported Criterion A measures (e.g., LPFS-SR, DLOP, LPFS-BF, and SIFS) found that graduate student and PhD-level raters viewed the content of these measures as very similar (associations higher than .8, many higher than .9), especially for the content measuring identity and intimacy difficulties (Waugh et al., 2021).

These raters were also asked to rate each Criterion A measure for content reflected in the Criterion B domains, thus comparing content overlap of Criterion A and Criterion B. They found that many Criterion A instruments appeared to contain more overlapping content for negative affectivity and detachment, relative to the other Criterion B domains. The authors reported the total amount of Criterion A versus Criterion B construct representation (higher scores indicating less Criterion B overlap). They reported effect sizes for LPFS-SR (d = 0.94), LPFS-BF2 (d = 0.84), SIFS (d = 0.63), and DLOP (d = 1.15). These results may suggest that, based on content overlap, the domains of Negative Affect (NA) and Detachment (DET) will be strongly associated with Criterion A measures, and that the DLOP may be more distinct, and the SIFS more overlapping, with the PID-5-BF. These findings are also largely consistent with Zimmermann and colleagues (2020) who found that several self-report Criterion A measures (e.g., LPFS-SR, LPFS-BF) and the average of PID-5-BF domains submitted to factor analysis form a single severity factor, indicating potential redundancy between Criterion A and Criterion B.

In summary, very few measures have compared Criterion A instruments, and even fewer have compared how Criterion A measures perform relative to each other in predicting important outcomes within the same sample.

Summary

Several self-report and interview measures are available to assess for Criterion A. These measures are often evaluated by examining reliability, and demonstrate criterion validity by examining associations with global impairment (such as a total score from a broad measure of psychopathology) and distinguishing level of severity or belonging to a diagnostic group or severity indicator (e.g., history of hospitalizations, number of suicide attempts).

Incremental validity is also examined to determine how distinct Criteria A and B are as constructs. The majority of this literature uses mono-method self-report frameworks, and inconsistently finds small but significant (less than 0.10) additional variance explained by Criterion A measures. Some measures have performed better (DLOPFQ) and using outcomes related to global impairments, or temporally dynamic data, tend to increase incremental validity coefficients.

The Present Research

This research compares five self-report measures of Criterion A to examine their reliability, criterion validity, overlap, and incremental validity with a Criterion B measure. More specifically, we hypothesized that all Criterion A measures will be sufficiently reliable, and that all Criterion A measures will be positively associated with each other. We chose outcome variables that were related to mental health utilization (e.g., history of psychiatric and psychotherapy utilization) as a proxy for levels of severity used in other studies. We also included a measure of interpersonal problems to capture interpersonal dysfunction. We employed a general symptom inventory appropriate for the college-level sample, meant to represent general dysfunction, and also included so-called “critical items” used by that measure to mark suicidality and aggression.

We assessed Criterion B through the PID-5-BF and hypothesized substantial (rs = .5–.6) associations with Criterion A measures, as has been found in previous research. We further hypothesize that measures of Criterion A will increment Criterion B for the aforementioned outcomes. We did not have a priori hypotheses about which Criterion A measures would be more strongly associated with outcomes, but we report these findings to help elucidate the similarities and differences among these measures of the same purported construct.

We also included a new interview-based measure of Criterion A and similarly evaluate its reliability, criterion validity, association with self-reported Criterion A measures, overlap with Criterion B, and incremental validity. We present these results separately because all of the other variables in the study are self-report and more easily comparable. For instance, one would expect a lower association with outcome measures due to unshared method variance and, if the interview-based Criterion A measure does not increment, it may be due to a variety of reasons that are difficult to disentangle (e.g., the measure is weak, the construct is redundant, and/or the self-reported outcomes may not have been significant with any interview-rated scale).

Method

Participants and Procedure

We collected data for this project across two semesters. In the spring of 2019, participants completed self-report surveys online in exchange for course credit (n = 108). In the fall of 2019, participants (n = 96) were interviewed by undergraduate research assistants (RAs) using the brief personality interview (see the appendix) and then were asked to complete self-report surveys on a lab computer. The interviews were audiotaped so that multiple RAs could later code the interview using the LPFS (APA, 2013). The research was institutional review board (IRB)-approved and followed appropriate ethical guidelines.

The total combined sample was 204 participants. Participant’s age ranged from 18 to 24 years (M = 18.90, SD = 1.47) and predominately identified as heterosexual (89%), non-Hispanic (93%), single (43%), and female (69%). The sample largely identified as Caucasian (74%), with smaller portions identifying as Asian (13%), African American (9%), Other (4%), American Indian or Alaskan Native (2%), and Native Hawaiian or Pacific Islander (<1%). The brief personality interview was only collected in the fall of 2019, and three participant interviews were not coded due to an oversight, resulting in a sample size of 93 for that variable. The SIFS and DLOP-SF were also only collected in fall 2019 (the authors became aware of these measures after spring 2019), and due to some missing data, the sample sizes were 91 (DLOP-SF) and 92 (SIFS).

Measures

Self-Report Criterion A Measures

The five self-report surveys of Criterion A used for this study all assess for personality dysfunction and all can be calculated with an average score representing the single dimension of Criterion A. The LPFS self-report (LPFS-SR; Morey, 2017) is an 80-item self-report questionnaire employing a 4-point Likert-type scale ranging from 1 (totally false) to 4 (very true). The LPFS-SR is scored using unique weights for items that represent more severe personality dysfunction content. The LPFS-SRA (APA, 2013; Roche et al., 2016) is a 12-item self-report questionnaire created by taking the LPFS verbatim from the DSM-5 and asking participants to use the coding scheme about themselves. As such, a 5-point scale ranging from 0 to 4 is used (with unique anchor points for each number across each item). The LPFS-BF2 (Bach & Hutsebaut, 2018) is a 12-item self-report measure employing a 4-point Likert-type scale ranging from 0 (very false or often false) to 3 (very true or often true). The SIFS (Gamache et al., 2019) is a 24-item self-report questionnaire employing a 5-point Likert-type scale ranging from 0 (not at all) to 4 (totally). The DSM-5 Level of Personality Functioning Questionnaire short-form (DLOP-SF; Siefert et al., 2020) is a 24-item self-report questionnaire employing a 6-point Likert-type scale from 1 (strongly disagree) to 6 (strongly agree).

LPFS–Interview-Rated

The Brief Personality Interview was developed specifically for this study to target difficulties with identity, self-direction, empathy, and intimacy (see the appendix). This roughly 20-minute interview was coded for Criterion A personality dysfunction using the LPFS (APA, 2013). The LPFS is a 12-item coding instrument, with a 5-point scale ranging from 0 (healthy) to 4 (impaired). This instrument captures elements of personality dysfunction described in the DSM-5. Once inter-rater reliability was established, we calculated the average score across the raters for each of the 12 LPFS items. Then, we averaged these items to comprise the four subscales of identity (ID), self-direction (SD), empathy (EPY), and intimacy (INT) along with the average score of all subscales (LPFSavg). These scores are denoted as LPFS-IR. Descriptive and reliability information for all Criterion A measures is listed in Table 7, and reviewed in the “Results” section.

Criterion B

The PID-5-BF (APA, 2013) is a 25-item instrument that was derived from its parent measure, the 220-item PID-5 (Krueger et al., 2012). Items are rated on a 4-point scale, ranging from 0 (very false) to 3 (very true), and is scored to represent the five pathological trait domains of Negative Affect (NA, α = .74), Detachment (DET, α = .64), Antagonism (ANT, α = .74), Disinhibition (DIS, α = .69), and Psychoticism (PSY, α = .79).

Mental Health Utilization

Participants were asked items from the 2016 National Survey on Drug Use and Health survey. Items assessed for history of inpatient treatment (Have you ever stayed overnight or longer in a hospital or other facility to receive treatment or counseling for any problem you were having with your emotions, nerves, alcohol/drug addiction, or mental health?), outpatient treatment (Have you ever received any outpatient treatment or counseling for any problem you were having with your emotions, nerves, alcohol/drug addiction, or mental health at any of the places listed below?), and psychiatric medication history (Have you ever taken any prescription medication that was prescribed for you to treat a mental or emotional condition?). The rate of any mental health utilization for this sample was 35.3%, which included inpatient treatment (6.4%), psychiatric medication (17.6%), and outpatient treatment (30.4%). The percentage of those endorsing inpatient treatment without endorsing outpatient treatment was 44%, while the percentage of those endorsing psychiatric medication without endorsing outpatient treatment was 20%.

Interpersonal Problems

Interpersonal problems were measured using the Inventory of Interpersonal Problems–Short Circumplex (IIP-SC; Soldz et al., 1995). The IIP-SC measures interpersonal problems that an individual does too much, or has difficulty doing (e.g., I do . . . too much, It is hard for me to). Items from the IIP-SC span themes of (low/high) agentic and (low/high) communal interpersonal problems. The 32-item measure is rated using a 5-point Likert-type scale ranging from 0 (not at all) to 4 (extremely). We used the total (elevation) score, calculated by averaging the subscales of the IIP-SC once they were z scored using college student sample norms (Hopwood et al., 2008; α = .84, M = −0.08, SD = 0.69, skewness = 0.51, kurtosis = −0.11).

Psychological Distress

The Counseling Center Assessment of Psychological Symptoms (CCAPS) 34 was released in September 2009 and updated in 2012 (Locke et al., 2012). It is a 34-item instrument on a 5-point Likert-type scale, ranging from 0 (not at all like me) to 4 (extremely like me). Participants endorse symptoms experienced over the past 2 weeks, which organizes into the subscales of Depression, Generalized Anxiety, Social Anxiety, Academic Distress, Eating Concerns, Hostility, and Alcohol Use. For this study, we use the distress index, which includes items from each subscale to capture a more general aspect of psychological distress (α = .93, M = 1.19, SD = 0.81, skewness = 0.75, and kurtosis = 0.14). We also examined three individual “critical items” that the CCAPS-34 identifies as important clinical markers. This includes suicidal ideation (I have thoughts of ending my life, M = 0.32, SD = 0.59, skewness = 1.53, and kurtosis = 0.80), aggressive ideation¹ (I have thoughts of hurting others, M = 0.16, SD = 0.43, skewness = 2.54, and kurtosis = 5.29), and aggressive non-control (I’m afraid I may lose control and act violently, M = 0.25, SD = 0.53, skewness = 1.88, and kurtosis = 2.24). These three items were square root transformed to reach acceptable levels of skewness (<2) and kurtosis (<7; West et al., 1995), and descriptive information was reported after those transformations.

Data Analysis

Convergent and Criterion Validity

For convergent validity, we calculated the correlations between Criterion A measures. For criterion validity, we calculated the correlations between each Criterion A measure and the outcome variables. To examine whether certain Criterion A measures were more strongly associated with a given outcome, we compared correlation magnitude strengths using Steiger’s Z test for dependent correlations (Steiger, 1980). This method tests the difference between two dependent correlations that share one variable in common. We used online software to calculate Steiger’s Z (Lee & Preacher, 2013). Steiger’s Z is sensitive to sample size, so we used the lowest n for each comparison to be more conservative. For most comparisons, this was n = 198; however, any comparison using either the SIFS or DLOP-SF was computed with n = 88 due to these measures only being collected in fall 2019, resulting in a smaller sample size.

Associations and Incremental Validity With Criterion B Measures

We calculated correlations between each Criterion A measure and the Criterion B domains, comparing magnitude strengths using Steiger’s Z. We also entered the five PID-5-BF domains into a regression to examine the amount of variance the Criterion B dimensions were able to explain for each Criterion A measure.

For incremental validity, we first report a series of multiple regressions, where each outcome was regressed on the PID-5-BF domains, to report the relationship between Criterion B and the outcomes. Then, we used a series of hierarchical linear regressions for each outcome model. We first entered all five PID-5-BF domains into Step 1 and then added each Criterion A measure separately into the regression to examine the amount of additional variance explained by a Criterion A measure once Criterion B domains were accounted for. We then reversed this process by adding in each Criterion A measure separately into Step 1, followed by all five PID-5-BF domains in Step 2. We then evaluated the relative additional variance explained in Step 2 of Criterion A versus Criterion B.

We replicated these incremental validity analyses while using correlations corrected for unreliability (see Supplemental Table 1). We also replicated these incremental validity analyses using the Predicted Residual Sum of Squares (PRESS) effect size estimates that are free from over-fitting models with more predictors (see Supplemental Table 2).

Analysis of Interview-Based Criterion A Measure

In the spring of 2020, eight undergraduate RAs were given the deidentified audio files for each participant interview and were asked to rate the interviews using the LPFS (APA, 2013). Prior to this, RAs were given introductory information about the LPFS and a journal reading on the LPFS was completed. Due to the impact of coronavirus disease 2019 (COVID-19), several RAs were unable to complete all of the ratings for the 96 participants. Specifically, five RAs rated at least 92 of participant interviews. The remaining three RAs completed 65, 43, and 42 interviews, respectively (essentially stopping once the university ended in-person learning). Reliability scores cannot be calculated with incomplete data, and the correlation between the LPFS average score of eight raters versus five raters was 0.97. Given this, we retained the ratings for the five RAs who completed most of the ratings, and we use these five RAs ratings moving forward. Inter-rater reliability was evaluated using a two-way, random ICC calculations, to quantify the reliability of a single rater (ICC [2,1]) and reliability across the mean of the five raters (ICC [2,5]).

Next, we evaluated the LPFS-IR in terms of convergent validity (correlations with other Criterion A measures), criterion validity (correlations with the outcomes), overlap with Criterion B domains (correlations), and incremental validity (regression).

Results

Descriptive Information and Intercorrelations Among Criterion A Measures

Descriptive information for all Criterion A self-report measures is presented in Table 1. Across measures, the mean level for Criterion A is low compared with the range of the scales, consistent with prior research using Criterion A measures in a sample of college students (Roche et al., 2018). The reliability across measures is adequate for the average Criterion A score. Reliability can increase with a larger number of items per scale, and the Criterion A measures vary from 12 to 80 items. Therefore, we applied the Spearman–Brown correction and set the number of items at 80, resulting in all scales having reliabilities above .94. Associations among Criterion A measures are presented in Table 2. Criterion A measures were strongly correlated (mean r = .68, range = .54–.79).

Table 1.

Descriptive Information for Criterion A Measures.

Descriptive Details	LPFS-SRA	LPFS-SR	LPFS-BF2	SIFS	DLOP-SF
M	0.74	2.97	0.81	1.09	2.72
SD	0.53	0.74	0.59	0.50	0.75
Skewness	1.04	0.80	0.29	0.54	0.02
Kurtosis	1.19	0.31	−0.98	−0.18	−0.34
Theoretical range	0–4	0–5.475	0–3	0–4	0–5
Reliability avg.	0.83	0.94	0.89	0.84	0.90
Reliability ID	0.63	0.86	—	0.62	0.86
Reliability SD	0.69	0.82	—	0.42	0.88
Reliability EPY	0.68	0.75	—	0.74	0.80
Reliability INT	0.66	0.80	—	0.67	0.81
Reliability self	—	—	0.87	—	—
Reliability other	—	—	0.81	—	—
Spearman–Brown correction to 80 items
Reliability avg.	0.97	0.94	0.98	0.95	0.97
Reliability ID	0.92	0.86	—	0.84	0.95
Reliability SD	0.94	0.82	—	0.71	0.96
Reliability EPY	0.94	0.75	—	0.90	0.93
Reliability INT	0.93	0.80	—	0.87	0.93
Reliability self	—	—	0.98	—	—
Reliability other	—	—	0.97	—	—

Note. LPFS-SR theoretical range is based on unique weighting for individual items. The LPFS-BF2 has a two-factor structure, so reliability for the subscales is reported above. LPFS-SRA = Level of Personality Functioning Scale–Self-Report of Criterion A; LPFS-SR = Level of Personality Functioning Scale–Self-Report; LPFS-BF2 = LPFS–Brief Form 2; SIFS = Self and Interpersonal Functioning Scale; DLOP-SF = DSM-5 Levels of Personality–Short Form; SD = self-direction; ID = identity; EPY = empathy; INT = intimacy.

Table 2.

Interrelationships Among Criterion A Measures.

Criterion A measures	LPFS-SRA	LPFS-SR	LPFS-BF2	SIFS	DLOP-SF
LPFS-SRA	1
LPFS-SR	.67*	1
LPFS-BF2	.63*	.68*	1
SIFS	.66*	.79*	.77*	1
DLOP-SF	.54*	.66*	.69*	.66*	1

Note. LPFS-SRA = Level of Personality Functioning Scale–Self-Report of Criterion A; LPFS-SR = Level of Personality Functioning Scale–Self-Report; LPFS-BF2 = LPFS–Brief Form 2; SIFS = Self and Interpersonal Functioning Scale; DLOP-SF = DSM-5 Levels of Personality–Short Form.

p < .05.

Criterion A Measures and Outcomes

The correlation between Criterion A measures and outcomes are presented in Table 3. Consistent with expectations, all Criterion A measures were positively associated with history of outpatient therapy, history of medication, psychological distress, interpersonal problems, suicidal ideation, and aggressive non-control, and most were also associated with aggressive ideation. No Criterion A measures were associated with history of inpatient treatment.

Table 3.

Associations Among Criterion A Measures and Outcome Variables.

Outcome Variables	LPFS-SRA	LPFS-SR	LPFS-BF2	SIFS	DLOP-SF
Inpatient ever	0.07	0.00	0.00	0.11	−0.11
Outpatient ever	0.25*^ab	0.25*^ab	0.21*^b	0.38*^a	0.38*^a
Medication ever	0.14*	0.24*	0.19*	0.29*	0.31*
Interpersonal problems	0.61*^b	0.73*^a	0.63*^b	0.62*^b	0.60*^b
CCAPS distress	0.62*^b	0.72*^a	0.73*^a	0.74*^a	0.62*^ab
Suicidal ideation	0.40*^ab	0.45*^a	0.48*^a	0.49*^a	0.23*^b
Aggressive ideation	0.33*^a	0.25*^a	0.29*^a	0.19^ab	0.04^b
Aggressive non-control	0.37*^bc	0.43*^ab	0.50*^a	0.44*^ab	0.21*^c

Note. Superscripts denote significant differences among Criterion A measures for a given outcome variable, calculated using Steiger’s Z test for dependent correlations. Point biserial correlation values are reported for the inpatient, outpatient, and medication variables, which are dichotomous. LPFS-SRA = Level of Personality Functioning Scale–Self-Report of Criterion A; LPFS-SR = Level of Personality Functioning Scale–Self-Report; LPFS-BF2 = Level of Personality Functioning Scale–Brief Form 2; SIFS = Self and Interpersonal Functioning Scale; DLOP-SF = DSM-5 Levels of Personality–Short Form; CCAPS = Counseling Center Assessment of Psychological Symptoms.

p < .05.

Specific pair-wise correlation magnitude comparisons are noted in the superscripts of Table 3. For outpatient therapy, the SIFS and DLOP-SF were stronger in magnitude compared with the LPFS-BF2. For interpersonal problems, the LPFS-SR was of a stronger magnitude compared with all others. For psychological distress, the LPFS-SR, LPFS-BF2, and SIFS were stronger in magnitude compared with the LPFS-SRA. For suicidal ideation, the LPFS-SR, LPFS-BF2, and SIFS were stronger in magnitude compared with the DLOP-SF. For aggressive ideation, the LPFS-SRA, LPFS-SR, and LPFS-BF2 were stronger in magnitude compared with the DLOP-SF. For aggressive non-control, the LPFS-BF2 was stronger in magnitude compared with the LPFS-SRA, and the LPFS-BF2, LPFS-SR, and SIFS were stronger in magnitude compared with the DLOP-SF.

Evaluating Overlap Among Criterion A and Criterion B of the AMPD

The correlation between Criterion A measures and the Criterion B measure are presented in Table 4. The average association between Criterion A and Criterion B was 0.51. Correcting for attenuation resulted in a higher average association (0.65), with all associations for detachment in the .8–.9 range and the SIFS associated with negative affectivity at .85, and detachment at .96. We will next highlight significant magnitude differences.

Table 4.

Associations Among Criterion A Measures and Criterion B Measure.

Correlation	LPFS-SRA	LPFS-SR	LPFS-BF2	SIFS	DLOP-SF
NA	.48*^b	.55*^b	.52*^b	.67*^a	.59*^ab
DET	.60*	.65*	.65*	.70*	.65*
ANT	.56*^a	.54*^a	.42*^b	.43*^ab	.30*^b
DIS	.44*^a	.46*^a	.41*^a	.45*^a	.17^b
PSY	.50*^b	.65*^a	.50*^b	.49*^b	.41*^b
Correlation with correction for attenuation
NA	.61	.66	.64	.85	.72
DET	.82	.84	.86	.96	.86
ANT	.71	.65	.52	.55	.37
DIS	.58	.57	.52	.59	.22
PSY	.62	.75	.60	.60	.49
Regression
NA	.22*	.24*	.26*	.38*	.33*
DET	.28*	.26*	.41*	.42*	.46*
ANT	.32*	.23*	.12*	.07	.02
DIS	.13*	.07	.09	.21*	−.09
PSY	.05	.27*	.05	−.01	.07
Adj. R²	.52*	.63*	.50*	.64*	.49*

Note. The r values are reported in the correlation section and standardized betas are reported in the regression section. Superscripts denote significant differences among Criterion A measures for a given PID-5-BF variable, calculated using Steiger’s Z test for dependent correlations. Regression analyses included Criterion A measures (columns) that were regressed onto the five PID-5-BF scales (rows). LPFS-SRA = Level of Personality Functioning Scale–Self-Report of Criterion A; LPFS-SR = Level of Personality Functioning Scale–Self-Report; LPFS-BF2 = Level of Personality Functioning Scale–Brief Form 2; SIFS = Self and Interpersonal Functioning Scale; DLOP-SF = DSM-5 Levels of Personality–Short Form; NA = Negative Affectivity; ET = Detachment; ANT = Antagonism; DIS = Disinhibition; PSY= Psychoticism.

p < .05.

For NA, the SIFS had a stronger magnitude compared with the LPFS-SRA, LPFS-SR, and LPFS-BF2. For ANT, the LPFS-SRA and LPFS-SR had a stronger magnitude compared with the LPFS-BF2 and DLOP-SF. For DIS, all other measures were stronger in magnitude compared with the DLOP-SF, and, for Psychoticism, the LPFS-SR was stronger in magnitude compared with all other measures.

Table 4 also included multiple regressions to determine the amount of variance shared among Criterion A and Criterion B measures. The self-report measures of Criterion A had substantial overlap with Criterion B (range = 0.50–0.64). The consistent associations were for NA and DET. After this, specific measures appeared to be more related to certain PID-5-BF domains (e.g., LPFS-SRA and ANT, LPFS-SR and PSY, and SIFS and DIS).

Criterion B and Outcome Variables

Before examining incremental validity, we first demonstrate that Criterion B domains are related to the outcomes (see Table 5). Specifically, there were strong associations with the outcomes of interpersonal problems and psychological distress, and moderate associations with the other variables (except history of inpatient treatment).

Table 5.

Multiple Regression Associations for PID-5-BF Variables and Outcome Variables.

Outcome Variables	NA	DET	ANT	DIS	PSY	Adj. r²
Inpatient ever	0.69	2.26	2.55	0.46	0.06	.07
Outpatient ever	2.18*	0.53	1.48	1.78	1.64	.17*
Medication ever	2.12*	0.48	1.69	2.32*	1.31	.14*
Interpersonal problems	0.45*	0.21*	0.08	0.00	0.18*	.51*
CCAPS distress	0.42*	0.25*	0.11*	0.04	0.17*	.56*
Suicidal ideation	0.00	0.32*	0.14	0.00	0.11	.20*
Aggressive ideation	−0.15*	0.23*	0.28*	−0.07	0.11	.16*
Aggressive non-control	−0.04	0.27*	0.21*	0.09	0.10	.23*

Note. Coefficients are reported as odds ratios for binary outcomes and standardized beta coefficients for dimensional outcomes. Adj. r² = Nagelkerke’s R² for mental health utilization outcomes that were modeled using a binary logistic model. PID-5-BF = Personality Inventory for DSM-5–Brief Form; NA = Negative Affectivity; ET = Detachment; ANT = Antagonism; DIS = Disinhibition; PSY = Psychoticism; CCAPS = Counseling Center Assessment of Psychological Symptoms.

p < .05.

Incremental Validity of Criterion A Measures

For each outcome, the PID-5-BF was entered in Step 1, and then each Criterion A measure was entered into Step 2 in separate models to examine whether a Criterion A measure explains additional variance in the outcomes. Steps 1 and 2 were then reversed. These models are presented in Table 6. In all cases, when a Criterion A measure was significant, it was in the expected positive direction.

Table 6.

Incremental Validity of Criterion A and Criterion B for Outcomes.

Step 1	Step 2	Inp ever	Out ever	Med ever	Interp prob	CCAPS distress	Sui ide	Agg ide	Agg non-control
Adjusted R² for total models (Steps 1 and 2)
PID-5-BF	LPFS-SRA	.08	.18*	.14*	.55*	.58*	.21*	.17*	.23*
PID-5-BF	LPFS-SR	.07	.17*	.16*	.60*	.61*	.22*	.15*	.26*
PID-5-BF	LPFS-BF2	.06	.19*	.15*	.55*	.65*	.25*	.17*	.29*
PID-5-BF	SIFS	.08	.34*	.38*	.50*	.63*	.27*	.00	.20*
PID-5-BF	DLOP-SF	.09	.41*	.48*	.51*	.59*	.21*	.00	.18
ΔR² for Criterion A: PID-5-BF entered (Step 1), Criterion A entered (Step 2)
PID-5-BF	LPFS-SRA	.01	.01	<.01	.04*	.02*	.01	.02	.01
PID-5-BF	LPFS-SR	.00	<.01	.01	.08*	.06*	.02*	<.01	.01
PID-5-BF	LPFS-BF2	.00	<.01	.01	.04*	.09*	.05*	.01	.06*
PID-5-BF	SIFS	.04	<.01	.01	.02	.06*	.06*	.03	.02
PID-5-BF	DLOP-SF	.06	.08*	.11*	.03*	.02*	.01	.01	.01
ΔR² for Criterion B: Criterion A entered (Step 1), PID-5-BF entered (Step 2)
LPFS-SRA	PID-5-BF	.07	.09*	.11*	.20*	.21*	.07*	.08*	.12*
LPFS-SR	PID-5-BF	.07	.09*	.07	.09*	.10*	.05	.12*	.08*
LPFS-BF2	PID-5-BF	.06	.12*	.09*	.18*	.12*	.05*	.11*	.07*
SIFS	PID-5-BF	.05	.15*	.26*	.14*	.11*	.08	.03	.06
DLOP-SF	PID-5-BF	.07	.20*	.31*	.18*	.23*	.21*	.04	.19*

Note. Criterion B is entered into a step using all five PID-5-BF scales together. Criterion A scales entered into a step separately for each outcome model. Binary logistic regression used for inpatient, outpatient, and medication variables, which are dichotomous. For these variables, we report Nagelkerke’s R² for the Step 1 and Step 2 variance. For all other variables, we report the adjusted R². Inp = ever had inpatient therapy; Out = ever had outpatient therapy; Med = ever had psychiatric medication; Interp prob = interpersonal problems; CCAPS = Counseling Center Assessment of Psychological Symptoms; PID-5-BF = Personality Inventory for DSM-5–Brief Form; LPFS-BF2 = LPFS–Brief Form 2; Sui ide = suicidal ideation critical item; Agg ide = aggressive ideation critical item; Agg non-control = aggressive non-control; LPFS-SRA = Level of Personality Functioning Scale–Self-Report of Criterion A; LPFS-SR = Level of Personality Functioning Scale–Self-Report; SIFS = Self and Interpersonal Functioning Scale; DLOP-SF = DSM-5 Levels of Personality–Short Form.

p < .05.

We reported the adjusted r² for the total models (Steps 1 and 2) to document the amount of variance explained by Criteria A and B together.² We focus first on the additional variance explained by Criterion A (second set of rows in Table 6). Looking across the outcomes (columns), consistent support for incremental validity was found for psychological distress and interpersonal problems, with less consistent support for suicidal ideation. There was infrequent support for incremental associations of mental health utilization (DLOP-SF was the only measure that incremented Criterion B) and aggressive non-control (LPFS-BF2 was the only measure that incremented Criterion B).

In total, there were 40 possible incremental validity findings (8 outcomes × 5 measures). Criterion A incremented Criterion B 15 times (39% of findings). When significant, the average additional variance explained was 5.47 (the average additional variance from all 40 models, including nonsignificant findings, was 2.83). In contrast (looking at the third set of rows in Table 6), Criterion B incremented Criterion A 29 times (73% of findings). When significant, the average additional variance explained was 14.07 (the average from all 40 models was 11.83). It was rare to find incremental validity only for Criterion A (2 times), while finding incremental validity only for Criterion B was more common (17), and when both Criteria A and B evidenced incremental validity, in every case Criterion B had a stronger (11) or equal (1) magnitude compared with Criterion B. Overall, Criterion B evidenced incremental validity about twice as often (15 vs. 29), and the average magnitude of a significant finding was about 2.5 times as large for Criterion B (5.47 vs. 14.07).

We recalculated incremental validity in two other ways. First, we reran all analyses using the correlation matrix corrected for attenuation due to the unreliability of measures (see Supplemental Table 1). This was done because Criterion B measures had weaker reliability compared with Criterion A measures, which may have presented more favorable incremental validity score for Criterion A. Several models did not converge with Criterion A entered second due to multicollinearity with DET, and when PID-5-BF variables were entered second, DET could not fit in the model, so was excluded. This perhaps led to an underestimation of incremental validity for those outcomes as it evaluated the incremental validity of the PID-5-BF without DET, and DET was significantly associated with several outcomes (see Table 5). Standard errors of these estimates are not reliable, precluding significance testing.

We also recalculated incremental validity using the PRESS statistic, which derives R² values that are free from over-fitting models with more predictors. This was done because Criterion B measures had a higher number of scales compared with Criterion A, which may have presented a more favorable incremental validity score for Criterion B due to model overfitting. Full details for calculating the PRESS R² statistic are found in Supplemental Table 2, and results were only reported for dimensional outcomes as it was unclear how to implement this statistic using dichotomous outcomes.

Incremental validity for Criterion A measures on average was 2.83% (including nonsignificant findings) using the adjusted R² approach (reported already). Incremental validity values were lower in analyses that accounted for unreliability (1.79%) or used the PRESS R² approach (2.16%). Incremental validity for Criterion B measures on average was 11.83% (including nonsignificant findings). Incremental validity values were higher in analyses that accounted for unreliability (14.98%), but lower in analyses that used the PRESS R² approach (6.64%).

Inter-Rater Reliability of Criterion A Interview

The inter-rater reliability results for the LPFS-IR are presented in Table 7. Considering the average LPFS score, reliability of individual raters was moderate (.52), while the reliability averaged across five raters was good (.84). These magnitudes are comparable to Roche and colleagues (2018) who also found inter-rater reliability around .4–.5 for individual undergraduate raters and around .8 for the average across five undergraduate raters. Examining the reliability of subscales across raters, the identity subscale had the weakest reliability (.65), while the other subscales demonstrated adequate to good reliability (.75–.90).

Table 7

Inter-Rater Reliability for LPFS–Interview-Rated (LPFS-IR) Measure.

Name	Descriptive					ICC [95% intervals]
Name	M	SD	Skew	Kurt	α	Single rater	Average rater
LPFS-IR avg.	0.29	0.19	1.30	1.46	.79	0.52 [0.42, 0.62]	0.84 [0.79, 0.89]
LPFS-IR ID	0.34	0.19	1.67	3.76	.63	0.27 [0.18, 0.38]	0.65 [0.52, 0.75]
LPFS-IR SD	0.30	0.26	1.07	0.91	.38	0.38 [0.28, 0.48]	0.75 [0.66,0.82]
LPFS-IR EPY	0.21	0.24	1.90	4.09	.79	0.39 [0.29, 0.50]	0.76 [0.67, 0.83]
LPFS-IR INT	0.32	0.37	1.43	1.76	.85	0.64 [0.36, 0.73]	0.90 [0.87, 0.93]

Note. Average rater based on five raters. ICC = intraclass correlation coefficient; LPFS-IR = Level of Personality Functioning Scale–Interview-Rated; SD = self-direction; Skew = skewness; Kurt = kurtosis; alpha = Cronbach’s alpha (internal consistency of items comprising the scales); ID = identity; EPY = empathy; INT = intimacy.

The LPFS-IR evidenced small to medium correlations with other measures of Criterion A, such as the LPFS-SRA (r = .39, p <.001), LPFS-SR (r = .22, p = .037), LPFS-BF2 (r = .31, p = .003), SIFS (r = .38, p < .001), and DLOP-SF (r = .28, p = .009). The LPFS-SRA correlation was slightly larger compared with prior research (.27) using a different stimulus for LPFS coding (Roche et al., 2018). There was also small overlap with the Criterion B dimensions of NA (r = .29, p = .005), DET (r = .25, p = .018), ANT (r = .25, p = .018), DIS (r = .11, p = .307), and PSY (r = .27, p = 009). This magnitude is consistent with prior research using a different interview to correlate with the PID (Weekers et al., 2021). The LPFS-IR was significantly associated with history of outpatient treatment (r = .35, p < .001), interpersonal problems (r = .27, p = .013), and CCAPS distress (r = .30, p = .004), but not significantly associated with history of inpatient or medication, or the critical items of the CCAPS.

For incremental validity, the LPFS-IR only incremented the PID-5-BF traits in the outcome of history of outpatient therapy. Specifically, when entered second, the LPFS-IR explained an additional 7% of variance (compared with the PID-5-BF entered second, which explained 19% additional variance). However, when entered together, the LPFS-IR was the strongest association (p = .02) and none of the PID-5-BF dimensions remained significant at p values of < .05.

Discussion

This research examined how the construct/measures of Criterion A were related to each other and key outcomes indicative of psychopathology. We first focus on the broader picture of Criterion A as a construct, and then compare Criterion A measures.

The Construct of Criterion A

First, measures of Criterion A appear to be reliable, and intercorrelations suggest that most self-report measures capture a similar construct. Criterion A measures are significantly and consistently related to markers of mental health utilization, interpersonal problems, psychological distress, and more specific clinical concerns, such as suicidal ideation and markers of aggression.

Criterion A also shares significant variance with Criterion B (regression-based variance explained, ranging from 0.50 to 0.64 across self-report measures, many zero-order correlations in excess of 0.5), which increases when correcting for the low reliability of Criterion B measurement (correction for attenuation). Despite this, most Criterion A measures significantly incremented interpersonal problems and psychological distress, and over half incremented suicidal ideation. Thus, regardless of the measure chosen to capture the Criterion A construct, it seems that small but significant incremental validity is found for these key outcomes. Less consistent incremental validity was found for treatment utilization and aggressive non-control, which may be due to the strengths of a given Criterion A measure and should be replicated in future studies to increase confidence in the link between those outcomes and the construct of Criterion A.

These findings are in contrast to other studies that tend to find Criterion A does not increment when the outcome is defined narrowly as personality disorder diagnostic criteria. We agree with Morey (2019) that research on incremental validity should move beyond traditional PD constructs as outcomes. Interestingly, Sleep and colleagues (2020) also found incremental validity in Criterion A at around 6% in their study and concluded that this level of incremental validity was low. There does not seem to be a clear consensus in the field for what percentage is necessary to be meaningful and likely depends in part on the outcome assessed. Consistent with prior research, Criterion B is more frequently and more powerfully incrementing Criterion A than the reverse.

The PID-5-BF domains of NA and DET were the strongest and most consistent associations with the Criterion A measures (see Table 4), and these were also the domains most consistently associated with the outcomes (see Table 5). This is in line with other research that found NA is broadly associated with many negative outcomes and does not have the discriminative nuance that the other pathological trait domains possess (Roche, 2018). Indeed, Saulsman and Page (2004) also noted neuroticism to be at the core of many personality disorders. Thus, it may be that the construct of negative affectivity is appropriately conceptualized as a core PD impairment (Criterion A), even if it is also employed as a stylistic difference in how personality dysfunction is expressed (Criterion B). Detachment was even more strongly associated with Criterion A (adjusted for reliability, the associations were higher than .8) and shares a clear conceptual overlap with intimacy impairment contained within the Criterion A construct.

Conceptual overlap across PD criteria is not necessarily problematic and, in fact, is quite common in other assessment literatures such as cognitive assessment. For instance, on the WISC-V, the visual spatial index (VSI) and fluid reasoning index (FRI) both rely upon visual-spatial processing, have similar looking tasks to derive index scores, and are correlated. Yet assessment psychologists readily recognize the nuances in these scores and can make meaningful distinctions when discrepancies arise. Similarly, negative emotion content is found in both Criteria A and B, and it may be more fruitful to consider how these similarities/discrepancies in scores fit together, rather than viewing overlap as a problem to be eliminated.

Diagnostically, Criteria A and B function in different roles (Criterion A captures severity and commonality; criteria B captures individual differences in expression). But they also connect to different theoretical frameworks, as Criterion A emphasizes personological and psychodynamic themes, whereas Criterion B emphasizes multivariate and trait themes (Mulay et al., 2018). Thus, the decision for retaining Criteria A and B should also account for this added clinical utility in having the AMPD represent many different theoretical lenses with which to see a client.

Comparing Criterion A Measures

All Criterion A measures (self-report) were significantly correlated with one another and had similar (within .15) magnitudes of association with the outcomes. One exception is the DLOP-SF, which had weaker associations with the CCAPS critical items, compared with most other Criterion A measures.

The overlap between Criterion A and Criterion B was substantial, with NA and DET being the main associations. This may be partially explained by content overlap as a comparative content analysis study (Waugh et al., 2021) found that content overlap was strongest for NA and DET in all of the Criterion A measures (except for the LFPS-SRA, which was not included in their study). Also consistent with this research, the measure with the most content overlap with Criterion B was noted to be the SIFS, and the least was noted to be the DLOP-SF and indeed, in this study, the strongest regression variance explained was for the SIFS (64%) and the weakest was for the DLOP-SF (49%), suggesting the association is partially explained by shared content overlap.

Some Criterion A measures have weaker associations with certain domains. These associations matter because a study using one Criterion A measure may find a significant association, whereas using a different Criterion A measure may not find that association. For instance, the LPFS-SR had stronger associations with psychoticism, which may make it better positioned to find associations with thought disturbance. The SIFS had stronger associations with negative affectivity, which may make it better positioned to find associations with emotional distress, and the DLOP-SF had weaker associations with disinhibition, which may make it harder to find associations with externalizing outcomes. In each of these cases, researchers would not want to find a result (or miss a result) due to measurement-specific differences in how the construct of Criterion A is assessed. This inconsistency may reflect differences in measures, or it may reflect a lack of criterion validity for the construct of Criterion A and the associated outcomes. Studies employing multiple Criterion A measures may be well positioned to disentangle these different possibilities. We hope our research can be useful to synthesize such results into a better understanding of the nomological net of the Criterion A construct.

Similarly, the incremental validity findings are more robust if they are demonstrated across several measures. This was found for interpersonal problems, psychological distress, and somewhat for suicidal ideation. But there were also measure-specific effects (e.g., treatment utilization incremented by DLOP-SF, aggressive non-control incremented by LPFS-BF2). Researchers should therefore be cautious in claiming that Criterion A fails to increment Criterion B, as we demonstrated in this study that the measure of Criterion A can influence whether a particular outcome will be incremented. However, some outcomes are incremented more consistently than others and, when the incrementation appears measurement specific, we suggest caution in its interpretation and invite the need to replicate the findings across a new sample.

We cannot conclude that one Criterion A measure is superior to the others based on the totality of this research. However, one can consider trade-offs across different measures. The LPFS-SRA is taken verbatim from the DSM-5, making it easy to compare it with interview ratings using the LPFS. However, in this study, it demonstrated a range restriction compared with the other measures, and the items themselves have different scale point descriptions on each item, likely increasing completion time compared with other measures. The LPFS-SR is a comprehensive view of Criterion A, but is the longest measure (80 items) and has a complex scoring routine. The LPFS-BF2 is the briefest measure; however, it uses a two-factor subscale structure (unlike all others with a four-factor structure), so if subscale-level comparisons are desired this measure may be less precise. The SIFS and DLOP-SF are both 24-item measures, with a four-factor structure found in other measures, and represent the middle road in terms of completion length for the surveys.

Interview-Rated Measure of Criterion A

The LPFS-IR was designed to be a free and briefer interview measure (20 minutes vs. 45–90 minutes) to assess for Criterion A personality dysfunction. The LPFS-IR appears to be reliable across five raters (ICC average rater score 0.84), and the items comprising the average score also appear to have adequate reliability (α = .79). However, inter-rater reliability for a single rater is only moderate (ICC single rater = 0.52). Taken together, it suggests the LPFS-IR is sufficiently reliable to use when averaging across five raters, but may not be reliable enough when using a single rater (such as in clinical practice with a single clinician rating a client).

Of note, the reliability (internal consistency) for the LPFS-IR subscale of self-direction is poor (.38), while the ICC across raters was adequate (.75). This suggests that coders can reliably agree on elevations of self-direction problems, but the three items comprising this scale do not hang together well. However, the LPFS-SRA uses the exact same items in a self-report format, and the internal consistency reliability for that subscale is .65. This may further suggest that the poor reliability (internal consistency) from the LPFS-IR may not be the items themselves, but an inadequate set of interview questions to tap the construct of self-direction difficulties in a college sample. These comparisons highlight the usefulness of multi-method data to better understand strengths/limitations in how a construct is measured.

The LPFS-IR was associated with self-reported measures of Criterion A, establishing convergent validity. The weaker correlation magnitudes (especially compared with other Criterion A measures) is likely explained by unshared method variance with the self-reported outcome variables. The LPFS-IR was associated with some Criterion outcomes (outpatient therapy, interpersonal problems, and psychological distress), but was not related to the others, providing less consistent validity support for this measure. The LPFS-IR only incremented the PID-5-BF in explaining additional variance in the history of outpatient treatment variable. This is an important outcome, yet future research will need to establish a broader base of incremental validity, and this may be facilitated by considering outcomes beyond self-report.

Strengths and Limitations

This study contained some strengths including a multi-method assessment of Criterion A and a comprehensive examination of Criterion A self-report measures within the same study. This study also has notable limitations, which can be organized by participants, measures, and data analysis.

Participant Limitations

The participant sample size was small, although this study focused mainly on convergent validity and therefore a smaller n would be appropriate in expecting moderate to large effect sizes, and the sample size is comparable to other studies evaluating interview-based methods for Criterion A. The sample was also drawn from a university setting, which, while not precluding psychopathology, the range restriction in Criterion A measures suggest it was a relatively healthy sample. While structural relationships between variables do tend to generalize in clinical and nonclinical samples (Bach et al., 2018; O’Connor, 2002), future research should replicate these findings in a clinical sample. In addition, a more diverse sample (e.g., race/ethnicity, gender, and age range) would also aid in evaluating the generalizability of these findings.

Measure Limitations

While five measures of Criterion A are the most that anyone has compared to date, several other measures do exist or have been published since this research was collected. Future research should continue to integrate these measures within a single study to understand their overlap and relative performance. We chose in this study to only use the average Criterion A scores, rather than to use subscales. We chose this because extant research suggests the subscales evidence poor discriminant validity (McCabe et al., 2021), the construct of Criterion A was designed to be a single dimension (Morey, 2019), and factor analysis results suggest that many Criterion A measures have a reasonable one-factor structure (Bliton et al., in press; Hummelen et al., 2021). Nevertheless, examining subscale-level data may be fruitful in determining which aspects of the Criterion A construct are contributing to an association with a particular outcome. This level of detail was beyond the scope of this article. The level of impairment from the LPFS-IR was low, which in some ways is expected in a relatively healthy college sample. However, it may also signal an underestimated level of impairment by RA coders (e.g., Zimmermann et al., 2014), or inadequacies of the stimuli (brief personality interview) or coding tool (LPFS) to capture psychopathology.

Our outcome measures were exclusively self-report, and may underrepresent psychopathology that could be assessed through other sources (e.g., informant reports, life history data). It may be particularly important to take a multi-method approach toward outcomes in evaluating method-based differences of AMPD instruments. More generally, the criterion validity of Criterion A measures is still being established, and a more thorough assessment of key severity markers is needed.

Future research should also consider collecting longitudinal data to evaluate these measures since to date only two measures (LPFS-SRA, LPFS-SR) have been evaluated using this type of data, and research found more nuanced patterns for Criteria A and B when considering longitudinal outcomes (Ringwald et al., 2021; Roche, 2018). Indeed, many of the Criterion A constructs themselves reflect temporally dynamic processes (e.g., inconsistent sense of self, vacillating goals, emotional lability, and if-then contingencies for personality impairment worsening when perceiving rejection or difference of opinions) that would be more directly captured in a temporally dynamic data collection approach.

Data Analysis Limitations

With a larger sample, more advanced data analyses could have been performed, including structural equation modeling (SEM), to determine whether the one-factor structure presumed in the Criterion A measures was reasonable. Similarly, with SEM, we could have modeled Criterion A as a latent construct, and then examined path models toward the outcomes (or perhaps latent outcomes), which would have arguably been more parsimonious than running separate models. This was not possible, given the small sample size, and we also feel it was more useful to treat Criterion A measures separately to facilitate comparison among them.

Despite these limitations, the present research provides a comprehensive comparison of Criterion A measures, finds some evidence for modest incremental validity with Criterion B, and introduces a new interview to capture Criterion A efficiently. We hope this moves the conversation forward on how to conceptualize and assess personality dysfunction.

Supplemental Material

sj-docx-1-asm-10.1177_10731911211059763 – Supplemental material for Comparing Measures of Criterion A to Better Understand Incremental Validity in the Alternative Model of Personality Disorders

Supplemental material, sj-docx-1-asm-10.1177_10731911211059763 for Comparing Measures of Criterion A to Better Understand Incremental Validity in the Alternative Model of Personality Disorders by Michael J. Roche and Sarah Jaweed in Assessment

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Michael J. Roche

Supplemental Material

Supplemental material for this article is available online.

Notes

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.).

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.

Bach

Hutsebaut

(2018). Level of Personality Functioning Scale–Brief Form 2.0: Utility in capturing personality problems in psychiatric outpatients and incarcerated addicts. Journal of Personality Assessment, 100(6), 660–670.

Bach

Sellbom

Simonsen

(2018). Personality Inventory for DSM-5 (PID-5) in clinical versus nonclinical individuals: Generalizability of psychometric features. Journal of Assessment, 25, 815–825.

Bender

D. S.

Morey

L. C.

Skodol

A. E.

(2011). Toward a model for assessing level of personality functioning in DSM-5, part I: A review of theory and methods. Journal of Personality Assessment, 93, 332–346.

Bender

D. S.

Skodol

A. E.

First

M. B.

Oldham

J. M.

(2018). Structured Clinical Interview for the DSM-5 Alternative Model for Personality Disorders (SCID-5-AMPD) Module I Level of Personality Functioning Scale. APPI.

Berghuis

Hutsebaut

Kaasenbrood

De Saeger

Ingenhoven

(2013). Semi-structured interview for personality functioning DSM-5 (STiP-5). Trimbos Institute.

Berghuis

Kamphuis

J. H.

Verheul

(2012). Core features of personality disorder: Differentiating general personality dysfunctioning from personality traits. Journal of Personality Disorders, 26, 704–716.

Bliton

C. F.

Roche

M. J.

Pincus

A. L.

Dueber

(in press). Examining the structure and validity of self-report measures of DSM-5 Alternative Model for Personality Disorder Criterion A. Journal of Personality Disorders.

10.

Buer Christensen

Eikenaes

Hummelen

Pedersen

Nysæter

T.-E.

Bender

D. S.

Skodol

A. E.

Selvik

S. G.

(2020). Level of personality functioning as a predictor of psychosocial functioning—Concurrent validity of criterion A. Personality Disorders: Theory, Research, and Treatment, 11(2), 79–90.

11.

Dowgwillo

Roche

Pincus

(2018). Examining the interpersonal nature of Criterion A of the DSM–5 Section III Alternative Model for Personality Disorders using bootstrapped confidence intervals for the interpersonal circumplex. Journal of Personality Assessment, 100(6), 581–592.

12.

Few

L. R.

Miller

J. D.

Rothbaum

A. O.

Meller

Maples

Terry

D. P.

Collins

MacKillop

(2013). Examination of the Section III DSM-5 diagnostic system for personality disorders in an outpatient clinical sample. Journal of Abnormal Psychology, 122(4), 1057–1069.

13.

Gamache

Savard

(2019, October 17). Assessing incremental validity of Criterion A and B from the DSM-5 Alternative model for personality disorders using a new self-report measure of Criterion A [Conference session]. Conference paper presented at the International Society for the Study of Personality Disorders congress, Vancouver, British Columbia, Canada.

14.

Gamache

Savard

Leclerc

Côté

(2019). Introducing a short self-report for the assessment of DSM–5 level of personality functioning for personality disorders: The Self and Interpersonal Functioning Scale. Personality Disorders: Theory, Research, and Treatment, 10(5), 438–447.

15.

Harkness

A. R.

Reynolds

S. M.

Lilienfeld

S. O.

(2014). A review of systems for psychology and psychiatry: Adaptive systems, Personality Psychopathology Five (PSY-5), and the DSM-5. Journal of Personality Assessment, 96, 121–139.

16.

Heiland

A. M.

Veilleux

J. C.

(in press). Severity of personality dysfunction predicts affect and self-efficacy in daily life. Personality Disorders: Theory, Research, and Treatment.

17.

Helle

A. C.

Trull

T. J.

Widiger

T. A.

Mullins-Sweatt

S. N.

(2017). Utilizing interview and self-report assessment of the Five-Factor Model to examine convergence with the alternative model for personality disorders. Personality Disorders: Theory, Research, and Treatment, 8(3), 247–254.

18.

Hemmati

Morey

L. C.

McCredie

M. N.

Rezaei

Nazari

Rahmana

(2020). Validation of the Persian translation of the Level of Personality Functioning Scale–Self-Report (LPFS-SR): Comparison of college students and patients with personality disorders. Journal of Psychopathology Behavioral Assessment, 42, 546–559.

19.

Hentschel

A. G.

Pukrop

(2014). The essential features of personality disorder in DSM-5: The relationship between criteria A and B. Journal of Nervous and Mental Disease, 202(5), 412–418.

20.

Hopwood

C. J.

Good

Morey

L. C.

(2018). Validity of the DSM–5 Levels of Personality Functioning Scale–Self Report. Journal of Personality Assessment, 100(6), 650–659.

21.

Hopwood

C. J.

Pincus

A. L.

DeMoor

R. M.

Koonce

E. A.

(2008). Psychometric characteristics of the Inventory of Interpersonal Problems–Short Circumplex (IIP–SC) with college students. Journal of Personality Assessment, 90(6), 615–618.

22.

Hopwood

C. J.

Thomas

K. M.

Markon

K. E.

Wright

A. G.

Krueger

R. F.

(2012). DSM-5 personality traits and DSM-IV personality disorders. Journal of Abnormal Psychology, 121(2), 424–432.

23.

Hummelen

Braeken

Buer Christensen

Nysaeter

T. E.

Germans Selvik

Walther

Pedersen

Eikenaes

Paap

M. C.

(2021). A psychometric analysis of the structured clinical interview for the DSM-5 Alternative Model for Personality Disorders Module I (SCID-5-AMPD-I): Level of Personality Functioning Scale. Assessment, 28, 1320–1333.

24.

Huprich

S. K.

Nelson

S. M.

Meehan

K. B.

Siefert

C. J.

Haggerty

Sexton

Dauphin

V. B.

Macaluso

Jackson

Zackula

Baade

(2018). Introduction of the DSM-5 levels of Personality Functioning Questionnaire. Personality Disorders: Theory, Research, and Treatment, 9, 553–563.

25.

Hutsebaut

Berghuis

De Saeger

Kaasenbrood

Ingenhoven

(2014). Semi-Structured Interview for Personality functioning DSM–5 (STiP 5.1). Trimbos Institute.

26.

Hutsebaut

Kamphuis

J. H.

Feenstra

D. J.

Weekers

L. C.

De Saeger

(2017). Assessing DSM–5-oriented level of personality functioning: Development and psychometric evaluation of the semi-structured interview for Personality Functioning DSM–5 (STiP-5.1). Personality Disorders: Theory, Research, and Treatment, 8(1), 94–101.

27.

Kampe

Zimmermann

Bender

Caligor

Borowski

Ehrenthal

J. C.

Benecke

Hörz-Sagstetter

(2018). Comparison of the structured DSM–5 clinical interview for the Level of Personality Functioning Scale with the structured interview of personality organization. Journal of Personality Assessment, 100(6), 642–649.

28.

Krueger

R. F.

(2013). Personality disorders are the vanguard of the post-DSM-5.0 era. Personality Disorders: Theory, Research, and Treatment, 4(4), 355–362.

29.

Krueger

R. F.

Derringer

Markon

K. E.

Watson

Skodol

A. E.

(2012). Initial construction of a maladaptive personality trait model and inventory for DSM-5. Psychological Medicine, 42(9), 1879–1890.

30.

Lee

I. A.

Preacher

K. J.

(2013). Calculation for the test of the difference between two dependent correlations with one variable in common [Computer software]. http://quantpsy.org

31.

Locke

B. D.

McAleavey

A. A.

Zhao

Lei

P.-W.

Hayes

J. A.

Castonguay

L. G.

Tate

Lin

Y.-C.

(2012). Development and initial validation of the counseling center assessment of psychological symptoms–34. Measurement and Evaluation in Counseling and Development, 45(3), 151–169.

32.

McCabe

G. A.

Oltmanns

J. R.

Widiger

T. A.

(2021). Criterion A scales: Convergent, discriminant, and structural relationships. Journal of Assessment, 28, 813–828.

33.

Milinkovic

M. S.

Tiliopoulos

(2020). A systematic review of the clinical utility of the DSM-5 Section III Alternative Model of Personality Disorder. Personality Disorders: Theory, Research, and Treatment, 11, 377–397.

34.

Morey

L. C.

(2017). Development and initial evaluation of a self-report form of the DSM–5 Level of Personality Functioning Scale. Psychological Assessment, 29, 1302–1308.

35.

Morey

L. C.

(2019). Thoughts on the assessment of the DSM–5 alternative model for personality disorders: Comment on Sleep et al. (2019). Psychological Assessment, 31, 1192–1199.

36.

Morey

L. C.

Bender

D. S.

Skodol

A. E.

(2013). Validating the proposed diagnostic and statistical manual of mental disorders, 5th edition, severity indicator for personality disorder. Journal of Nervous and Mental Disease, 201, 729–735.

37.

Morey

L. C.

Good

E. W.

Hopwood

C. J.

(2020). Global personality dysfunction and the relationship of pathological and normal trait domains in the DSM-5 alternative model for personality disorders. Journal of Personality. Advance online publication. https://doi.org/10.1111/jopy.12560

38.

Mulay

A. L.

Cain

N. M.

Waugh

M. H.

Hopwood

C. J.

Adler

J. M.

Garcia

D. J.

Kurtz

J. E.

Lenger

K. A.

Skadberg

(2018). Personality constructs and paradigms in the Alternative 35 DSM-5 Model of Personality Disorder. Journal of Personality Assessment, 100, 593–602.

39.

O’Connor

B. P.

(2002). The search for dimensional structure differences between normality and abnormality: A statistical review of published data on personality and psychopathology. Journal of Personality and Social Psychology, 83(4), 962–982.

40.

Pincus

A. L.

Roche

M. J.

(2019). Paradigms of personality assessment and level of personality functioning (Criterion A) in the DSM-5 Alternative Model for Personality Disorders. In Hopwood

C. J.

Mulay

A. L.

Waugh

M. H.

(Eds.), The DSM-5 Alternative Model for Personality Disorders: Integrating multiple paradigms of personality assessment (pp. 48–59). Routledge.

41.

Ringwald

W. R.

Hopwood

C. J.

Pilkonis

P. A.

Wright

A. G. C.

(2021). Dynamic features of affect and interpersonal behavior in relation to general and specific personality pathology. Personality Disorders: Theory, Research, and Treatment, 12(4), 365–376. https://doi.org/10.1037/per0000469

42.

Roche

M. J.

(2018). Examining the alternative model for personality disorder in daily life: Evidence for incremental validity. Personality Disorders: Theory, Research, and Treatment, 9, 574–583.

43.

Roche

M. J.

Jacobson

N. C.

Phillips

J. J.

(2018). Expanding the validity of the Level of Personality Functioning Scale observer report and self-report versions across psychodynamic and interpersonal paradigms. Journal of Personality Assessment, 100(6), 571–580.

44.

Roche

M. J.

Jacobson

N. C.

Pincus

A. L.

(2016). Using repeated daily assessments to uncover oscillating patterns and temporally-dynamic triggers in structures of psychopathology: Applications to the DSM-5 alternative model of personality disorders. Journal of Abnormal Psychology, 125, 1090–1102.

45.

Saulsman

L. M.

Page

A. C.

(2004). The five-factor model and personality disorder empirical literature: A meta-analytic review. Clinical Psychology Review, 23, 1055–1085.

46.

Siefert

C. J.

Sexton

Meehan

Nelson

Haggerty

Dauphin

Huprich

(2020). Development of a short form for the DSM–5 Levels of Personality Functioning Questionnaire. Journal of Personality Assessment, 102, 516–526.

47.

Sleep

C. E.

Weiss

Lynam

D. R.

Miller

J. D.

(2020). The DSM–5 section III personality disorder criterion a in relation to both pathological and general personality traits. Personality Disorders: Theory, Research, and Treatment, 11(3), 202–212.

48.

Soldz

Budman

Demby

Merry

(1995). A short form of the inventory of interpersonal problems circumplex scales. Assessment, 2, 53–63.

49.

Steiger

J. H.

(1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87(2), 245–251.

50.

Suzuki

Samuel

D. B.

Pahlen

Krueger

R. F.

(2015). DSM-5 alternative personality disorder model traits as maladaptive extreme variants of the five-factor model: An item-response theory analysis. Journal of Abnormal Psychology, 124(2), 343–354.

51.

Van den Broeck

Bastiaansen

Rossi

Dierckx

De Clercq

Hofmans

(2014). Hierarchical structure of maladaptive personality traits in older adults: Joint factor analysis of the PID-5 and the DAPPBQ. Journal of Personality Disorders, 28, 198–211.

52.

Waugh

M. H.

McClain

C. M.

Mariotti

E. C.

Mulay

A. L.

DeVore

E. N.

Lenger

K. A.

Russell

A. N.

Florimbio

A. R.

Lewis

K. C.

Ridenour

J. M.

Beevers

L. G.

(2021). Comparative content analysis of self-report scales for level of personality functioning. Journal of Personality Assessment, 103, 161–173.

53.

Weekers

L. C.

Hutsebaut

Kamphuis

J. H.

(2019). The Level of Personality Functioning Scale–Brief Form 2.0: Update of a brief instrument for assessing level of personality functioning. Personality and Mental Health, 13(1), 3–14.

54.

Weekers

L. C.

Verhoeff

S. C. E.

Kamphuis

J. H.

Hutsebaut

(2021). Assessing Criterion A in adolescents using the Semistructured Interview for Personality Functioning DSM–5. Personality Disorders: Theory, Research, and Treatment, 12(4), 312–319. https://doi.org/10.1037/per0000454

55.

West

S. G.

Finch

J. F.

Curran

P. J.

(1995). Structural equation models with nonnormal variables: Problems and remedies. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 56–75). SAGE.

56.

Widiger

T. A.

Bach

Chmielewski

Clark

L. A.

DeYoung

Hopwood

C. J.

Thomas

K. M.

(2019). Criterion A of the AMPD in HiTOP. Journal of Personality Assessment, 7, 77–92.

57.

World Health Organization. (2020). International statistical classification of diseases and related health problems (11th ed.). https://icd.who.int/

58.

Wright

A. G. C.

Simms

L. J.

(2014). On the structure of personality disorder traits: Conjoint analyses of the CAT-PD, PID-5, and NEO-PI-3 trait models. Personality Disorders: Theory, Research, and Treatment, 5, 43–54.

59.

Wright

A. G. C.

Simms

L. J.

(2015). A metastructural model of mental disorders and pathological personality traits. Journal of Psychological Medicine, 45, 2309–2319.

60.

Zettl

Taubner

Hutsebaut

Volkert

(2019). Psychometric Evaluation of the German Version of the Semi-Structured Interview for Personality Functioning DSM-5 (STiP-5.1). Psychotherapie, Psychosomatik, Medizinische Psychologie, 69(12), 499–504.

61.

Zimmermann

Benecke

Bender

D. S.

Skodol

A. E.

Schauenburg

Cierpka

Leising

(2014). Assessing DSM–5 Level of Personality Functioning From Videotaped Clinical Interviews: A Pilot Study with untrained and clinically inexperienced students. Journal of Personality Assessment, 96(4), 397–409.

62.

Zimmermann

Kerber

Rek

Hopwood

C. J.

Krueger

R. F.

(2019). A brief but comprehensive review of research on the alternative DSM-5 model for personality disorders. Current Psychiatry Reports, 21(9), 92.

63.

Zimmermann

Müller

Bach

Hutsebaut

Hummelen

Fischer

(2020). A common metric for self-reported severity of personality disorder. Psychopathology, 53, 168–178. https://doi.org/10.1159/000507377

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB