Abstract
The DSM-5 proposal indicates that personality disorders (PDs) be defined as collections of maladaptive traits but does not provide a specific diagnostic method. However, researchers have previously suggested that PD constructs can be assessed by comparing individuals’ trait profiles with those prototypic of PDs and evidence from the five-factor model (FFM) suggests that these prototype matching scores converge moderately with traditional PD instruments. The current study investigates the convergence of FFM PD prototypes with interview-assigned PD diagnoses in a sample of 99 homeless individuals. This sample had very high rates of PDs, which extends previous research on samples with more modest prevalence rates. Results indicated that diagnostic agreement between these methods was generally low but consistent with the agreement previously observed between explicit PD measures. Furthermore, trait-based and diagnostic interview scores evinced similar relationships with clinically important indicators such as abuse history and past suicide attempts. These findings demonstrate the validity of prototype methods and suggest their consideration for assessing trait-defined PD types within DSM-5.
A considerable body of research has supported the hypothesis that the personality disorders (PDs) of the American Psychiatric Association’s (APA) Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR; APA, 2000) are reasonably well understood within a dimensional trait model (Clark, 2007; Samuel & Widiger, 2008). From this perspective, PDs as defined by DSM-IV-TR are not discrete entities that are distinct from each other and from normal personality but rather are specific constellations of maladaptive traits (Krueger et al., 2011; Widiger & Trull, 2007). In light of this research and theory, DSM-5 appears poised to include a dimensional trait model as part of the official diagnostic nomenclature (DSM-5 Personality and Personality Disorders Work Group, 2011). In fact, the most recent proposal indicates that PDs will be operationalized, in part, by a specific set of maladaptive traits (DSM-5 Personality and Personality Disorders Work Group, 2011; Skodol, 2012).
This proposed hybrid system recognizes that, although evidence suggests that traits are the most informative level of description for personality pathology (e.g., Simms et al., 2011), there might also be times when clinicians or researchers seek to identify or label the specific constellations of traits that reflect a given PD construct (Bornstein, 2011). The diagnostic labels associated with categorical PD constructs (e.g., borderline) are quite familiar to clinicians, so a method of recapturing these diagnostic constructs from traits might be quite helpful to assist in the transition to a dimensional model (Miller, in press). Another potential benefit of diagnostic categories was to stimulate research and generate specific treatment recommendations. Although, this has not occurred for several of the disorders (Blashfield & Intoccia, 2000; Boschen & Warner, 2009), there are certain PDs that are being studied actively (e.g., borderline, antisocial, schizotypal, obsessive–compulsive, avoidant, narcissistic, and perhaps dependent), and a straightforward method of assessing these constructs within a dimensional system would facilitate their continued evaluation and aid the translation of the existing research literature.
However, one challenge is determining precisely how to diagnose categorical PD constructs from a set of traits. In this regard, the current DSM-5 proposal is notably silent and has not proposed the method for recapturing the PDs. Nonetheless, there is an empirical literature that has examined methods for recovering PD constructs from dimensional trait systems, including the use of prototype matching techniques (e.g., Miller, Lynam, Widiger, & Leukefeld, 2001). Although this approach has been applied to other models (e.g., Benning, Patrick, Hicks, Blonigen, & Krueger, 2003), a majority of the research has used measures of the five-factor model of personality (FFM; McCrae & Costa, 2003). The FFM is a predominant model of general personality functioning that has received a great deal of support regarding its ability to conceptualize the DSM PDs (e.g., Samuel & Widiger, 2008). Although distinctions have been drawn between the FFM and the five-domain model proposed for DSM-5 (e.g., Krueger & Eaton, 2010), there are many more similarities than differences. Thus, research concerning the FFM has the potential to inform decisions about the DSM-5 diagnostic process.
Miller et al. (2001) first proposed that PD constructs could be estimated by comparing an individual’s FFM trait profile with that of a prototypic case of a PD. Prototypic descriptions were developed by Lynam and Widiger (2001), who collected FFM facet ratings for each PD from expert researchers and produced mean consensus profiles. There are two primary methods by which an individual’s trait profile has been compared with these prototypes. The current study extends previous work by evaluating the convergence of these methods with diagnoses assigned by a semistructured interview, as well as their relationships with concurrently assessed clinically relevant symptoms. In addition, unlike previous research, we evaluated these methods in a sample with a very high prevalence of PDs.
The method most commonly used for this comparison has been calculating double-entry (i.e., intraclass) correlations between an individual’s trait profile and a given PD’s profile. These prototype matching indices (PMIs) converge well with, and perform much like, instruments designed to assess DSM-IV PDs (Miller & Lynam, 2003; Stepp & Trull, 2007; Trull, Widiger, Lynam, & Costa, 2003). The PMI technique is robust across different methods of assessment including informant (Miller, Pilkonis, & Morse, 2004), semistructured interview (Miller, Bagby, & Pilkonis, 2005), and clinician rating (Miller et al., 2010; Samuel, Edmundson, & Widiger, 2011). In general, it appears that PMIs converge with measures of PDs about as well as any two PD measures do with each other (e.g., Miller, in press; Widiger & Boyd, 2009).
However, noting that the calculation of PMIs was rather complex and required statistical software, Miller, Bagby, Pilkonis, Reynolds, and Lynam (2005) proposed a simplified prototype count (PC) approach. This method considers only scores from FFM facets described as characteristically high (≥4) or characteristically low (≤2) by the raters surveyed by Lynam and Widiger (2001). Miller, Bagby, Pilkonis, Reynolds, and Lynam (2005) reported that the PC method correlated highly with the PMI approach (median r = .91) and converged significantly with a self-report measure for 9 of the 10 PDs (obsessive–compulsive PD was the exception). Subsequently, Decuyper, De Clercq, De Bolle, and De Fruyt (2009) replicated this convergent validity for the PC method among adolescents.
Nonetheless, a limitation of this previous research was low prevalence rates for the PDs. For example, Miller et al. (2008) specifically noted the relatively low prevalence rates for several PDs within their American psychiatric sample. In fact, only three PD categories (borderline, avoidant, and dependent) had at least 10 participants who met diagnostic criteria, and the low rates of the other 7 PDs precluded the investigation of diagnostic efficiency rates. In other words, research to date has suggested that the prototype matching techniques work reasonably well among samples with moderate rates of personality pathology, but examination of these methods in samples with severe personality pathology is needed.
Through work with the FFM, trait-based assessment methods have also been shown to approximate reasonably the nomological networks of more traditional assessments of specific PDs, including antisocial (Gudonis, Miller, Miller, & Lynam, 2008; Miller et al., 2001) and borderline (Stepp & Trull, 2007; Trull et al., 2003). For example, Trull and colleagues demonstrated that the FFM borderline (BPD) prototype correlated as highly with external validators, such as global dysfunction and childhood abuse history, as did traditional measures of the BPD. However, little research has specifically examined the FFM prototypes for the remaining eight DSM-IV-TR PDs in terms of their ability to recreate the nomological networks derived from diagnostic interviews.
The present study has two interrelated aims. First, we sought to investigate the relationship between the PMI and PC methods, as well as their convergence with an interview measure of the DSM-IV PDs, in a severely dysfunctional sample with high rates of PDs. This aim is not predicated on the belief that the interview measure represents a gold standard for the assessment of PDs, but simply that it reflects the current diagnostic system and is thus a useful point of comparison for these more experimental trait-based methods. With regard to this aim, we hypothesize that the two FFM methods will correlate very highly with one another. Additionally, although the two trait-based assessment methods might have more difficulty differentiating PDs in a sample with high co-occurrence, their convergence with diagnoses assigned by a PD interview should be relatively similar to previous research.
A second aim, which extends previous research, is to determine how PD scores derived via the two FFM methods and those derived from a semistructured diagnostic interview relate to external validators, such as indicators of general distress, Axis I diagnoses, and clinically relevant behaviors (e.g., past suicide attempts). These analyses will provide additional evidence that compares the nomological networks of the trait-based diagnostic methods to a more traditional approach. We suspect that although the FFM-based diagnostic approaches might be limited for specific PDs (e.g., schizotypal) due to instrumentation issues (i.e., limited coverage of maladaptive content), their overall concurrent validity will be comparable to what is obtained for the PD diagnostic interview.
Materials and Method
Participants
Participants were homeless adults recruited from two drop-in shelters, one in a large metropolitan area and one in a moderately sized urban area in the Northeastern United States. Previous analyses did not indicate any appreciable differences between the sites so results are collapsed (Ball, Connolly, Linares, & Cobb-Richardson, 2007). We excluded individuals only when drop-in center staff intended to refer for emergency psychiatric services due to acute psychosis, violence, or suicidality. Only two were excluded for these reasons and both were transported to the emergency room. Those who provided informed consent completed an assessment battery that included several interview and self-report measures and took approximately 3 hours with breaks taken as needed. Participants received a $30 debit card for public transportation as reimbursement for their time and effort. All procedures were approved by the appropriate institutional review boards.
Measures
Computer-assisted SCID-II Expert System (CAS-II)
The CAS-II (First, Gibbon, Williams, Spitzer, & Benjamin, 2000) is a computerized version of the Structured Clinical Interview for DSM-IV–Axis II and provides an assessment of each diagnostic criterion for the DSM-IV PD categories. An interviewer verbally administered a series of questions that screen for symptoms of the DSM-IV-TR PDs. When the participant endorsed a given item, the computer software prompted the interview to query further and determine whether the criterion was indeed present. Administration of items for a particular PD was discontinued when (a) the individual reached the diagnostic threshold or (b) the diagnosis was ruled out. For this reason, not all items were administered to each participant and internal consistency statistics cannot be computed. The CAS-II was administered by two interviewers with master’s degrees and significant clinical experience. Both interviewers received appropriate training on the administration of the CAS-II using materials available from the publisher. One was trained by the developers of the SCID-II and the other by the local research division’s SCID trainer. Initial calibration was conducted with an independent expert, and both interviewers received weekly supervision throughout the study from the primary investigator who had extensive experience in the administration of structured diagnostic interviews.
NEO Personality Inventory–Revised (NEO PI-R)
The NEO PI-R (Costa & McCrae, 1992) contains 240 statements assessing five broad domains, as well as 30 facets that underlie them. The NEO PI-R scores have strong temporal stability over a 7-year period (Costa, Herbst, McCrae, & Siegler, 2000) and have shown consistency across cultures (McCrae et al., 2005).
Brief Symptom Inventory (BSI)
The BSI (Derogatis, 1992) is a 53-item self-report inventory of psychiatric symptoms that asks participants to rate items on a five-point scale of distress. It yields three global severity measures and nine primary symptom dimensions. We report only the Global Severity Index, which quantifies overall severity in a single composite score.
Computer-Assisted SCID–Clinician Version (CAS-CV)
The CAS-CV (First, Spitzer, Gibbon, & Williams, 1996) is a computer-assisted version of the SCID-I and provides an assessment of the DSM-IV-TR Axis I diagnoses. In the current study, we used four categorical variables indicating whether the individual met current or lifetime criteria for a mood disorder, psychotic disorder, substance use disorder, and/or anxiety disorder. To provide external criterion variables that were maximally distinct from the diagnoses, we also used four variables from the CAS-CV overview module. These were violent arrests, which was the number of arrests they reported for violent crimes; longest job, which was the length of their longest period of employment in their lifetime; abuse history, which was a yes/no variable indicating whether they reported a history of physical or sexual abuse in childhood; and suicide attempt, which was a yes/no variable indicating whether they had previously attempted suicide.
Prototype Derivation
Prototype matching indices (PMIs)
PMIs were calculated by correlating a participant’s complete FFM profile with the mean consensus profile of Lynam and Widiger (2001) for each of the 10 DSM-IV PDs. For these comparisons we used intraclass correlations (ICCs), which have the advantage of being sensitive to profile shape and mean-level differences. Although Furr (2010) cautioned that ICCs conflate scatter differences and McCrae (2008) found that they obtain results similar to Pearson correlations, we used them in the current study to facilitate direct comparison with previous findings that have also used ICCs.
Prototype count method
Using the procedures outline by Miller, Bagby, Pilkonis, Reynolds, and Lynam (2005), we summed the raw scores for facets deemed characteristic by the experts in Lynam and Widiger (2001). Facets rated low for a PD were reverse-scored.
Results
Sample Characteristics
The sample was mostly male (57%), and predominantly African American (64%), but also had 23% European Americans and 12% Hispanic Americans. They had an average age of 41.4 (SD = 10.7). Education ranged from 7 to 16 years, with a mean of 12.1 (SD = 2.0), but most (64%) had completed high school. The self-reported duration of lifetime homelessness ranged from 1 month to 18 years, with a mean of 30.8 (SD = 42.9) months, whereas current homelessness averaged approximately 1 year (13.7 months, SD = 25.7). The sample demonstrated a history of impaired functioning as 59% reported previous psychiatric hospitalization and 44% reported a previous incarceration. CAS-CV interviews indicated that Axis I disorders were prevalent, with generalized anxiety disorder (33%), major depressive disorder (29%), drug abuse/dependence (28%), and posttraumatic stress disorder (25%) the most frequent diagnoses. Lifetime or current prevalence rates for broad Axis I categories were 67% for mood disorders, 66% for substance use disorders, 59% for anxiety disorders, and 19% for psychotic disorders. Eighty-eight percent of the sample met criteria for a PD diagnosis, with a mean of 3.5 (SD = 2.4) diagnoses per participant.
We calculated the validity scales developed by Schinka, Kinder, and Kremer (1997) to detect negative presentation management (NPM), positive presentation management (PPM), and inconsistent (INC) response styles on the NEO PI-R. These were converted to t scores based on the base rates provided by Schinka and colleagues. The mean t scores in our sample were 57.3 (SD = 16.3) for NPM, 46.3 (SD = 11.0) for PPM, and 61.7 (SD = 15.2) for INC. The NPM and PPM were comparable to values obtained in a clinical sample whereas the INC value was higher (Young & Schinka, 2001). Consistent with previous studies (Young & Schinka, 2001), we employed a t score ≥70 as a cutoff. A total of 61 participants were above this cut-point for at least one of the validity scales, with 31 (19%) for NPM, 4 (3%) for PPM, and 44 (28%) for INC. These 61 individuals were excluded from further analyses, yielding a sample of 99 participants. 1
Demographic variables for this retained sample were recalculated and compared with the values for those excluded to determine if there were significant differences using independent sample t tests and chi-square analyses. The final sample of 99 did not differ significantly in terms of gender (60% male) or race/ethnicity (63% African American, 24% European Americans, and 11% Hispanic) from those excluded. The retained group also did not differ significantly in terms of age (M = 42.1, SD = 10.7), length of current homelessness (M = 14.8 months, SD = 28.8), or total months of lifetime homelessness (M = 29.8, SD = 42.4). But the difference in total years of education between those retained (M = 12.5, SD = 2.0) and those omitted (M = 11.5, SD = 1.9) was significant, t(158) = 3.0, p = .003.
The groups did not differ in terms of lifetime or current prevalence for mood (64%), psychotic (15%), or anxiety disorders (65%). They also did not differ for the rate of any PD (86%) or any Axis I disorder (85%), but the selected group was less likely to obtain a substance use disorder diagnosis (57%), χ2(1) = 9.4, p = .002. Additionally, a lower proportion of the retained group reported a history of psychiatric hospitalization (51%), χ2(1) = 5.7, p = .02, or incarceration (36%), χ2(1) = 4.6, p = .03.
Descriptive Statistics
Because the validity of FFM profile matching techniques depends on the ability of NEO PI-R’s assessment of the facets, we first investigated the descriptive statistics of these scales. Table 1 provides the means and standard deviations for the 30 scales. In addition, it provides values for Cronbach’s alpha, some of which were low (i.e., four values were < .50). The median value across the facets was .63. This value was lower than the median value of .71 reported for the facets within the NEO PI-R manual (Costa & McCrae, 1992). Personality disorders were quite prevalent in this sample, according to the CAS-II. The most prevalent PDs were obsessive–compulsive personality disorder (OCPD; 58% of the sample), paranoid (55%), narcissistic (41%), and schizoid (39%). Eight of the 10 PD categories had frequencies greater than 20%, whereas histrionic and dependent were the least frequent with 9 (9%) and 5 (5%) cases, respectively. These values are quite a bit higher than was reported in previous research on the FFM prototype methods, such as the American sample used in Miller et al. (2008). In that sample of 84 psychiatric outpatients, the most prevalent PDs were borderline and avoidant (26%), whereas dependent (11%) was the only other PD with a rate greater than 10%. In contrast to the current sample, there were only two cases each of antisocial and histrionic PD and no one met criteria for schizoid or schizotypal PD.
NEO PI-R Descriptive Statistics
Note: N = 99. NEO PI-R = NEO Personality Inventory–Revised (Costa & McCrae, 1992).
Table 2 presents the correlations among the domain assessments from the NEO PI-R. Consistent with prior research, the domains were not strictly orthogonal and evinced significant correlations across domains. These magnitude of these correlations ranged from a .00 (openness and neuroticism) to −.55 (neuroticism and conscientiousness), with a median absolute value of .31. Table 3 presents a similar correlation matrix for the PD diagnoses assigned by the CAS-II. Several of these values also reached significance. The correlations ranged from a minimum of .01 (paranoid and histrionic) to .58 (paranoid and borderline) with a median value of .18.
Correlations Among the NEO PI-R Domain Scores
Note: N = 99. NEO PI-R = NEO Personality Inventory–Revised.
p < .01 (2-tailed). *p < .05 level (2-tailed).
Correlations Among CAS-II PD diagnoses
Note: N = 99. CAS-II = Computer Assisted SCID-II–Clinician Version; OCPD = obsessive–compulsive personality disorder.
p < .01 (2-tailed). *p < .05 level (2-tailed).
Diagnostic Convergence
The first column of Table 4 presents the convergence of the two FFM prototype methods in terms of Pearson correlations. With the possible exception of histrionic (r = .81) they agreed quite highly, with a median of .89. Table 4 also presents the point–biserial correlation for each of these methods with the CAS-II diagnoses. Perhaps not surprisingly, given their strong relation to one another, both metrics obtained very similar correlations with the CAS-II diagnoses. However, the magnitude of these correlations varied substantially across the PDs. The PMIs’ correlations ranged from .09 (obsessive–compulsive) to .62 (borderline), with a median of .34. Similarly, the PC method had correlations ranging from .02 (obsessive–compulsive) to .61 (borderline), with a median of .31. 2
Convergent Correlations Between Profile Matching Methods and CAS-II Diagnoses
Note: N = 99. CAS-II = Computer Assisted SCID-II Expert System; PMIs = FFM profile matching indices; Count = FFM count method.
Concurrent Validity
As the trait-based methods of diagnosing PDs agreed only moderately with the semistructured diagnostic interview, we tested how each correlated with additional variables that were theoretically linked. For this purpose we correlated the PD scores from all three methods with a measure of overall distress (i.e., the GSI from the BSI), lifetime Axis I diagnoses, and clinically relevant outcomes collected from specific items on the CAS-CV overview module. The first row for each PD in Table 5 presents the Pearson correlations between the dimensional PMI score and the criterion variables. The second and third rows for each PD presents the point–biserial correlations using the categorical PD diagnoses assigned by the FFM count method and the CAS-II, respectively. When interpreting this table, we eschewed significance tests of differences among the dependent correlations because of the rather large number of comparisons that were considered. Instead, we focused our interpretations on those criterion variables where the correlations for the FFM methods and the CAS-II differed by more than .20, as this suggested a meaningfully different effect size. This value was chosen because it represents the midpoint between a small and medium correlation (Cohen, 1992) and has been used previously to describe meaningful differences (Saulsman & Page, 2004). As these are differences among dependent correlations, there is not a direct link between magnitude and statistical significance. However, in each of the cases we detail below, differences between correlations larger than .20 were significant at p < .05.
Correlations With Criterion Variables
Note: N = 99. PMI = FFM Prototype Matching Index; Count = categorical diagnosis using FFM count method and cut-score of t ≥ 65; CAS-II = Categorical Diagnosis using Computer-Assisted SCID-II Expert System; BSI GSI = the Global Severity Index from the Brief Symptom Inventory (Derogatis, 1992); Lifetime Psychotic Dx = the presence of any current or lifetime psychotic disorder diagnosis via the CAS-CV; Lifetime SUD dx = the presence of any current or lifetime substance use disorder diagnosis via the CAS-CV; Violent Arrests = the number of lifetime violent arrests; Abuse History = a history of physical or sexual abuse in childhood; Suicide Attempt = a dichotomous variable reflecting if the individual had ever attempted suicide. Violent Arrests, Longest Job, Abuse History, and Suicide Attempts were all drawn from specific items within the CAS-CV interview module.
p < .01 (two tailed). *p < .05 (two-tailed).
Overall it appeared the nomological networks for the trait-based and semistructured interview methods were highly similar. The GSI from the BSI provides an indicator of general symptom severity. As such it is relevant to all 10 PDs but provides limited evidence of discriminant validity among the PD constructs. Narcissistic was the only diagnosis where the methods evinced a substantial difference, as the CAS-II correlated more strongly (.30) than did the FFM PMI (.05). A lifetime psychotic disorder diagnosis via the CAS-CV is a potentially relevant criterion for schizotypal diagnosis, and the CAS-II score correlated much more strongly (.48) with it than did the PMI (.02) or count (.12) methods. A CAS-CV-assigned lifetime substance use disorder (SUD) diagnosis is particularly relevant to the antisocial and borderline diagnoses and here the results were mixed. Although the FFM and CAS-II borderline diagnoses evinced very similar correlations with this criterion, the trait-based assessments of antisocial PD failed to correlate significantly with SUD diagnosis, whereas the CAS-II did.
We also evaluated four individual life experiences, drawn from the CAS-CV overview module, because of their potential relations with specific PDs. The CAS-II antisocial diagnosis correlated highly (.46) with the number of violent arrests, whereas the FFM-based antisocial PMI (.07) and count (.00) methods did not. The longest period of employment had similar, negative, correlations with both the PMI and CAS-II assessments of antisocial PD. In addition, the longest period of employment correlated positively with the FFM count method OCPD diagnosis, but not with the CAS-II. A history of physical, sexual, and/or mental abuse was significantly correlated with BPD assigned by the trait-based and diagnostic interview methods. A similar pattern was found for a history of suicide attempts as the BPD CAS-II diagnosis achieved a correlation (.56) that was comparable to the FFM PMI (.49) and count (.47) methods.
Discussion
Consistent with previous research, we found strong agreement between two different FFM prototype matching approaches. The PMI and PC methods obtained a median correlation of .89, which was quite similar to the .91 value reported by Miller, Bagby, Pilkonis, Reynolds, and Lynam (2005). Although histrionic PD obtained lower agreement, even this value was relatively high, suggesting that the two methods did not differ fundamentally. However, the convergence of these trait-based methods with a semistructured interview was even lower than reported in previous research. For example, Miller, Reynolds, and Pilkonis (2004) found the median convergence was .50 for the PMIs, and Miller, Bagby, and Pilkonis (2005) reported a median convergence of .45 for the FFM PC method. The relevant values in our study were .34 and .31, respectively. However, the comparisons were not equivalent as the previous studies considered dimensional PD symptom counts rather than categorical diagnoses. One would expect improved convergent correlations if dimensional scores were available for the CAS-II, as continuous measures of psychopathology have demonstrated appreciable validity advantages over discrete scoring approaches (Markon, Chmielewski, & Miller, 2011). Furthermore, the correlations were also likely attenuated by the somewhat lower measurement reliability noted for the NEO PI-R facets within this highly pathological sample.
Nevertheless, the current findings, along with those from previous research, provide important information about assessing PD constructs via a trait-based model. Although, the agreement between the dimensional PD scores assigned by the PMI method and the categorical diagnoses from the semistructured interview was relatively poor, it was consistent with values observed between measures specifically designed to assess the PDs. Widiger and Boyd (2009) reviewed 25 studies that reported the agreement between a dimensional scores on a self-report PD measure and a semistructured interview and found that the median agreement ranged from .26 (OCPD) to .53 (avoidant), with an overall median of .37. Thus, it appeared that the convergence of the FFM PMI method with the CAS-II in this highly dysfunctional sample (e.g., .34) was similar to the agreement typically observed between semistructured interviews and self-report measures specifically designed to assess the PDs. Although this reflects poorly on the validity of the PD diagnoses (Clark, 2007; Trull & Durrett, 2005), it suggests that when one chooses to assess the PD constructs, a trait-based diagnostic method performs about as well as measures explicitly designed for the purpose.
In this regard, an important contribution of the current study was the ability to look beyond convergence and compare trait-based and traditional diagnostic approaches in terms of their relationships with external criteria. It was apparent from these analyses that the FFM prototype approaches and the CAS-II diagnostic interview produced largely similar nomological networks. For example, the borderline prototype scores demonstrated correlations with past suicide attempts, history of abuse, substance use disorder diagnoses, and overall dysfunction that were comparable to those from a semistructured interview (Trull et al., 2003). In addition, the results suggested that the FFM assessments for the other PDs captured about as much general symptom severity as did the more traditional diagnostic approach.
The results for antisocial were not as favorable for the trait-based diagnostic approaches. Although the trait-based and interview assessments of antisocial PD evinced similar correlations with employment history, the CAS-II antisocial score demonstrated a stronger relationship with the number violent arrests and lifetime substance use diagnoses. These latter two findings are difficult to understand as they contrast markedly with prior research that demonstrated largely equivalent nomological networks for the antisocial PD prototypes (Gudonis et al., 2008; Miller et al., 2001). In this case, it does appear that the values obtained for the CAS-II may reflect the problem of criterion contamination in the current study. The CAS-II assessment of antisocial PD explicitly inquires about past illegal behaviors (e.g., drug use), violence, and criminal history. Although the criterion measures we selected were not drawn from the CAS-II, the fact that they were assessed by the same interviewer does complicate their use as validity criteria. Nonetheless, it is purely the case that the CAS-II validity was higher than normal. The magnitude of the relationship between the FFM antisocial count method and lifetime substance use diagnoses was particularly low (r = .05) in the current sample compared with the values reported by Gudonis et al. (2008) concerning the same FFM method and diagnoses of alcohol (r = .30) and marijuana abuse (r = .23). Given the study by Gudonis et al. used the same measure of the FFM (i.e., the NEO PI-R) and the identical count technique, our current findings do not appear to reflect a limitation of the NEO PI-R, FFM, or a difficulty of trait assessments, in general, but might instead be attributable to idiosyncrasies of this sample. To further probe these relationships, we examined the correlation of violent arrests and lifetime substance use diagnoses with NEO PI-R agreeableness. Contrary to expectations, these relationships were −.03 and −.06, respectively. This does support the idea that the trait of agreeableness relates somewhat differently to criminal arrests within this sample than is typical (e.g., Samuels et al., 2004).
In addition, the schizotypal prototype approaches did not appear to capture the presence of psychotic diagnoses as well as the CAS-II interview. It is perhaps debatable how strongly a measure of schizotypal PD should relate to frank psychotic disorders (e.g., Lenzenweger, 2010), but the current finding echoed previous research on the FFM prototype assessment of schizotypal (e.g., Miller, Reynolds, & Pilkonis, 2004). In this case, the low value is a function of the FFM measure and supports the contention that the NEO PI-R assessment of openness to experience is limited in its ability to adequately capture aspects of schizotypy (Watson, Clark, & Chmielewski, 2008; but see also Haigler & Widiger, 2001). Although it should be answered empirically, we expect alternative trait measures, such as the Psychoticism scale assessed by the Personality Instrument for DSM-5 (PID-5; Krueger, Derringer, Markon, Watson, & Skodol, in press) would perform much better in this regard.
The results for OCPD were muddied and difficult to interpret. The unexpectedly high prevalence of OCPD in the sample has been previously documented, yet remains puzzling (Connolly, Cobb-Richardson, & Ball, 2008). In addition, the FFM assessment of OCPD has obtained the weakest empirical support in previous research (Miller et al., 2008; Miller, in press). In the current study, OCPD identified by the CAS-II appeared to be strongly related to measures of dysfunction. In contrast, the FFM prototypes for OCPD were not related with dysfunction and even evinced a significant positive correlation with employment history. The historically inconsistent relationship between OCPD and functioning (Cramer, Torgersen, & Kringlen, 2007; Ryder, Costa, & Bagby, 2005; Ullrich, Farrington, & Coid, 2007) only complicates the ability to discern which method provides the more valid assessment.
Implications
The difficulties with the current PD categories are well documented (Trull & Durrett, 2005), and the problematic co-occurrence that characterizes the DSM-IV-TR PD model (Ball, 2001) was apparent in the current study with a mean of 3.5 PD diagnoses per participant according to the CAS-II. Clark (2007) has questioned the utility of reproducing such a flawed system, and even proponents of the FFM have suggested that prototype matching ignores the promise of a dimensional trait system to provide a valid and useful model of personality disorder in its own right (e.g., Widiger & Trull, 2007). Although the research does suggest that it is the traits themselves that are robust across samples (Eaton, Krueger, South, Simms, & Clark, 2011) and useful in clinical contexts (Samuel & Widiger, 2006), we agree with Miller (in press) that prototype matching has practical utility to bridge the DSM-IV PD categories with a dimensional model that might eventually replace them. The current results suggest that the prototype approaches proposed by Miller et al., at least when used with the FFM as assessed by the NEO PI-R, demonstrate convergent validity and produce nomological networks that are mostly equivalent to a traditional PD assessment instrument. Furthermore, we suggest that this literature might be useful for considering methods for trait-based PD diagnosis within DSM-5.
The current findings also bear on the more general issue concerning the relative validity of alternative assessment methods and sources. Although, semistructured diagnostic interviews are often considered the “gold standard” for the assessment of PDs (McDermut & Zimmerman, 2005), research has not demonstrated that they yield scores with greater predictive validity than other sources. For example, Hopwood et al. (2008) compared an interview and self-report measure of borderline PD and found that each method incremented the other in predicting functional impairment. Although research comparing the prediction of functional outcomes would be helpful, it is likely that the ideal assessment strategy would use information from multiple sources (Lenzenweger, Loranger, Korfine, & Neff, 1997; Widiger & Samuel, 2005; Zimmerman, 1994). Thus, the current study suggests that neither the trait-based prototype approach nor the semistructured diagnostic interview is necessarily superior. Rather, they are simply alternative methods of assessing the PD constructs.
Limitations
A notable limitation of the current study was that the diagnostic interview (CAS-II) did not provide dimensional scores that would have allowed a more precise assessment of the DSM-IV PD constructs and likely would have yielded larger convergent correlations with the FFM methods. Additionally, although the CAS-II interviewers were carefully trained and supervised, formal interrater reliability was not calculated. The use of a computer-prompted interview mitigates this concern, but it is nonetheless a limitation. The interpretation of the diagnostic agreement results was also complicated because the comparisons were across assessment methods. Whereas the CAS-II diagnoses were derived via semistructured interview, the FFM scores were self-reported. Research that compares the diagnostic agreement between FFM prototype matching scores and explicit PD assessments within, as well as across, assessment methods would be helpful.
Our sample of homeless persons was useful because it contained a very high rate of PD pathology. However, these individuals have complicated histories and were currently in extremely adverse life situations, such that it is difficult to disentangle the basic maladaptive personality patterns that define PDs from behaviors and attitude that reflect their current circumstances. Although this is not a limitation, per se, it does potentially complicate the assessment and interpretation of the current findings. We attempted to avoid such complications by our rigorous standards for excluding participants that demonstrated inconsistent responding. The excluded group did evince lower educational attainment, more frequent substance use disorder diagnoses, and greater rates of previous incarceration and psychiatric hospitalization than did the retained group. Of course, this strategy increases internal validity at the expense of external validity.
In addition, the present analyses concern only concurrent validity across methods. It would be of great interest for future research to investigate how well each of these methods prospectively predicts important functional outcomes. Although Stepp and Trull (2007) demonstrated that the FFM prototypes for borderline and antisocial reasonably predicted functioning prospectively over 6 years, that study did not compare them to a traditional PD assessment. For example, it would be important to know how well the CAS-II and FFM PD assessments of BPD predict future hospitalizations, suicide attempts, and/or relationship functioning.
Finally, our results are limited in that they consider the FFM prototypes as operationalized by the NEO PI-R. Although this FFM research provides an important base for considering trait-based assessments of PD constructs, the same techniques should ultimately be tested using the DSM-5 system. For example, it appears that the PID-5 (Krueger et al., in press) will explicitly operationalize and assesses the DSM-5 trait model, and research should demonstrate that trait profiles produced by this measure can also reproduce the nomological networks that characterize DSM-IV-TR PD measures.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Writing of this article was supported by the Office of Academic Affiliations, Advanced Fellowship Program in Mental Illness Research and Treatment, Department of Veterans Affairs. Research was funded by the National Alliance for Research on Schizophrenia and Depression (NARSAD) Independent Investigator Award (Samuel A. Ball).