Burdensomeness,Belongingness,and Capability: Assessing the Interpersonal–Psychological Theory of Suicide With MMPI-2-RF Scales

Abstract

Given the emerging body of literature demonstrating the validity of the interpersonal–psychological theory of suicide (IPTS), and the importance of increasing our understanding of the development of risk factors associated with suicidal behavior, it seems worthwhile both to expand IPTS research via Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI-2-RF) correlates and to expand the availability of methods by which to assess the constructs of the IPTS. The present study attempted to do so in a large adult outpatient mental health sample by (a) inspecting associations between the IPTS constructs and the substantive scales of the MMPI-2-RF and (b) exploring the utility of MMPI-2-RF scale–based algorithms of the IPTS constructs. Correlates between the IPTS constructs and the MMPI-2-RF scales scores largely followed a pattern consistent with theory-based predictions, and we provide preliminary evidence that the IPTS constructs can be reasonably approximated using theoretically based MMPI-2-RF substantive scales. Implications of these findings are discussed.

Keywords

MMPI-2-RF interpersonal–psychological theory of suicide suicide construct validity

The accurate assessment and prediction of suicidal behavior remains a quandary for clinicians and researchers alike. With an estimated one person dying by suicide every 12 minutes (Centers for Disease Control and Prevention, 2014), refining methods by which to assess for suicide risk is imperative. The interpersonal–psychological theory of suicide (IPTS; Joiner, 2005) is one promising framework through which to understand suicide risk. The IPTS proposes three jointly necessary and sufficient causes that lead to suicidal behavior (i.e., potentially lethal attempts and death): perceived burdensomeness, thwarted belongingness, and the acquired capability for suicide (Joiner, 2005; Van Orden et al., 2010). The goal of the present study is twofold: (a) examine the associations between the IPTS measures and the scales of the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008/2011) and (b) determine if the individual correlates can be improved by creating scale-based, theoretically derived algorithms. In doing so, we hope to link the MMPI-2-RF to a contemporary model of suicidal behavior, to promote research on these constructs, and to increase their utility in clinical settings.

Interpersonal–Psychological Theory of Suicide

The IPTS consists of three constructs: perceived burdensomeness, thwarted belongingness, and the acquired capability for suicide. Perceived burdensomeness describes a sense of self-hatred and a belief that one’s death is worth more than his or her life. Thwarted belongingness includes feelings of loneliness and the belief that one’s life is devoid of reciprocal caring relationships. Acquired capability for suicide is characterized by fearlessness about death/bodily harm and high tolerance for physical pain, which is acquired over time through experiencing and habituation to painful and fearful encounters (e.g., nonsuicidal self-injury, sexual abuse, physical abuse or fights, contact sports, combat experiences; Gradus et al., 2012; Joiner, 2005; Muehlenkamp & Gutierrez, 2007; Selby et al., 2010; Van Orden et al., 2010). Data also indicate that genetic factors contribute to the capability for suicide (Birklein et al., 2008; Smith et al., 2012).

The IPTS makes two main predictions about when the desire for death and serious or lethal suicidal behavior is likely to occur. First, the theory proposes that the serious desire for death arises from the cooccurrence of perceived burdensomeness and thwarted belongingness, as well as hopelessness that these situations will change (M. D. Anestis, Moberg, & Arnau, 2014; Joiner, 2005; Van Orden et al., 2010). There is evidence that perceived burdensomeness and thwarted belongingness are related, but distinct, constructs that can be reliably measured (Van Orden, Cukrowicz, Witte, & Joiner, 2012). Additionally, the IPTS describes perceived burdensomeness and thwarted belongingness as malleable, state (as opposed to trait) constructs. Thus, the degree to which an individual experiences each factor may fluctuate based on intrapersonal (e.g., mood, affect) and interpersonal (i.e., interpersonal environments) factors. According to the theory, each psychological state alone increases risk for passive suicidal ideation; however, active suicidal ideation develops when both perceived burdensomeness and thwarted belongingness are present and viewed as unchanging and stable (i.e., hopeless). Empirical studies have documented associations between heightened levels of these two interpersonal constructs and suicidal ideation (e.g., M. D. Anestis & Joiner, 2011; M. D. Anestis, Khazem, Mohn, & Green, 2015; Joiner et al., 2009; Van Orden, Witte, Gordon, Bender, & Joiner, 2008). Also consistent with predictions of the IPTS, Hagan et al. (2015) found that the interaction between thwarted belongingness and perceived burdensomeness only predicted current suicide risk when levels of hopelessness were high.

Second, the IPTS states that for a lethal or near-lethal attempt to occur, one must not only have a desire for death but one must also have the capability to carry out a suicide attempt (i.e., the capability for suicide). Thus, the theory predicts that perceived burdensomeness, thwarted belongingness, and the capability for suicide are jointly necessary and sufficient in the prediction of serious or lethal suicidal behavior. Unlike perceived burdensomeness and thwarted belongingness, which the IPTS describes as state constructs that fluctuate over time, the theory describes the capability for suicide as a factor that is more stable and based on genetic predispositions and acquired exposure to events that increase fearlessness about death and increase physical pain tolerance. Several studies to date have provided evidence for the theory’s main hypothesis; namely, that the interaction between perceived burdensomeness, thwarted belongingness, and acquired capability predicts suicide attempts (e.g., M. D. Anestis & Joiner, 2011; Joiner et al., 2009).

Several self-report measures have been developed to assess the three IPTS components. Perceived burdensomeness and thwarted belongingness are commonly measured using the Interpersonal Needs Questionnaire (INQ; Van Orden et al., 2012), as well as with proxy measures from personality inventories (J. C. Anestis et al., 2016; Cramer et al., 2012; Joiner et al., 2009; Smith et al., 2012). Capability for suicide is often measured through the Acquired Capability for Suicide Scale (ACSS; Van Orden et al., 2008) and ACSS–Fearlessness About Death (ACSS-FAD; Ribeiro, Witte, et al., 2014). It can also be assessed through measurements of physical pain tolerance (Witte, Gordon, Smith, & Van Orden, 2012). In both the sample of young adults and a sample of psychiatric inpatients, self-report measures of the capability for suicide have been found to have good convergent validity (Ribeiro, Witte, et al., 2014). Specifically, self-reported fearlessness about death was found be associated with fear of suicide, self-reported courage to attempt suicide, and a number of outcomes associated with pain perception (e.g., self-perceived ability to withstand physical discomfort, fear of physical pain, and a behavioral measure of pain tolerance). Nevertheless, a common problem with face valid, self-report measures of suicidality is the tendency of some high-risk populations (e.g., older adults, military personnel) to underreport suicidal ideation (e.g., M. D. Anestis & Green, 2015; Cukrowicz, Jahn, Graham, Poindexter, & Williams, 2013). The development of IPTS measures embedded within larger assessment instruments that are not solely focused on suicidality may thus allow for enhanced understanding of the development of risk factors for suicidal behavior.

Minnesota Multiphasic Personality Inventory–2–Restructured Form

The prediction and assessment of suicidal behavior using personality questionnaires, such as the Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1942), has been the topic of many previous studies. Several such studies have examined the profile differences of MMPIs between individuals who previously attempted suicide, those who died by suicide, and nonsuicidal control groups. These studies have produced inconsistent results and indicate that the original MMPI was not able to consistently differentiate suicidal from nonsuicidal individuals (see, e.g., Clopton & Baucom, 1979; Clopton & Jones, 1975; Clopton, Pallis, & Birtchnell, 1979; Clopton, Post, & Larde, 1983; Daigle, 2004; Devries & Farberow, 1967; Leonard, 1977; Watson, Klett, Walters, & Vassar, 1984). These inconsistent findings may be due to the authors examining scales which were not directly related to suicidal behavior. In fact, the original MMPI contained few items with suicidal content (Graham, 2011).

The limited research examining the prediction and assessment of suicidal behavior using the MMPI-2 (Butcher et al., 2001) shows slightly more promise. Specifically, the Suicidal Ideation (DEP4) Content Component Scale, which assesses recent suicidal ideation; the Depression Content Scale, which assesses depressive symptomology; and the Suicidal Potential Scale, which includes six critical items related to suicidal intent and behavior (Glassmire, Stolberg, Greene, & Bongar, 2001), have all shown relationships with suicidal thoughts and behaviors (Boone, 1994; Green, Handel, & Archer, 2006). Additionally, MMPI-2 Content Scales have been noted to incrementally contribute to the prediction of suicidal ideation and identified differences between genders (males: Type A [TPA], females: Anger [ANG]; Kopper, Osman, & Barrios, 2001).

The MMPI-2-RF (Ben-Porath & Tellegen, 2008/2011) was designed to address shortcomings of the MMPI-2 through development of psychometrically sound scales using a maximally efficient item pool. An additional goal was to tie the instrument to contemporary models of personality and psychopathology (e.g., Ben-Porath & Tellegen, 2008/2011; Sellbom & Ben-Porath, 2005; Tellegen & Ben-Porath, 2008/2011; Tellegen, Ben-Porath, & Sellbom, 2009; Tellegen et al., 2006). Major innovations with the MMPI-2-RF include the use of Restructured Clinical (RC) Scales in lieu of the original Clinical Scales and incorporation of a hierarchical interpretive structure with narrow breadth, symptom-focused scales, such as the Suicidal/Death Ideation (SUI) Scale, at the lower level of the hierarchy. Previous studies have reported the classification accuracy rate of the SUI scale to identify individuals in a medical setting who had a previous suicide attempt to be 84% (Tarescavage et al., 2013). Additionally, the SUI scale has been shown to have incremental validity in identifying individuals reporting previous suicide attempts (Gottfried, Bodell, Carbonell, & Joiner, 2014) and incremental variance to predictions of future suicidal behavior within 1 year of the MMPI-2-RF administration after accounting for information obtained during a clinical interview (Glassmire, Tarescavage, Burchett, Martinez, & Gomez, 2015). The focus of the SUI scale is on suicidal ideation, with one item pertaining to suicide attempts; thus, the content of the scale may not sufficiently account for the IPTS model, particularly the acquired capability for suicide, and its reliance on ideation misses variables that might be important in the transition from suicidal ideation to suicidal behavior (e.g., Klonsky & May, 2014).

The MMPI-2-RF Technical Manual (Tellegen & Ben-Porath, 2008/2011) includes additional correlational data with a number of suicide-related criterion variables (record review, therapist ratings, other self-report measures) in various settings and populations. Interested parties are referred to the MMPI-2-RF Technical Manual for more specific information about these samples and results. Notably, in both men and women and across samples, SUI consistently was the strongest correlate with record review variables (e.g., reported history of suicide attempts, being admitted for suicidality, intake notation regarding suicidal ideation or plan, and number of previous suicide attempts reported; rs ranged from .27 to .55); however, other emotional dysfunction scales (e.g., Demoralization [RCd], Self-Doubt [SFD], Helplessness/Hopelessness [HLP]) and behavioral dysfunction scales (e.g., Behavioral/Externalizing Dysfunction [BXD], Antisocial Behavior [RC4]) had noteworthy associations. Additionally, for SUI, effect sizes were smaller for suicide attempt variables and larger for suicidal ideation variables. Few MMPI-2-RF substantive scales demonstrated even small effect associations with the number of previous suicide attempts, with the female criminal defendants being a single exception (both Stress/Worry [STW] and Low Positive Emotions [RC2], r = .29). In the outpatient community mental health center sample, therapist-rated client suicidal ideation had the highest associations with SUI (.46), Emotional/Internalizing Dysfunction (EID; .44), and RCd (.41) in men and RCd (.33), SUI (.32), and EID (.32) in women. In the disability claimants sample, questionnaire-assessed suicidality had large effect correlates with SUI (r = .81 in men and women) and RCd (r = .50 in men and women) and smaller associations with other substantive scales. Thus, across measures and settings, numerous MMPI-2-RF substantive scales are associated with suicidal ideation, with less compelling data regarding attempts. Moving beyond single scale examinations to algorithmic-based analyses may prove more fruitful in determining the utility of the MMPI-2-RF in identifying risk factors for lethal self-harm.

The Current Study

Given the emerging body of literature demonstrating the validity of the IPTS model of suicidality (Ribeiro & Joiner, 2009), and the importance of increasing our understanding of risk factors associated with suicidal behavior, it seems worthwhile both to expand IPTS research via MMPI-2-RF correlates and to expand the availability of methods by which to assess the constructs of the IPTS. Furthermore, given the current drive to focus suicide research on an ideation to action framework (Klonsky & May, 2014), in which studies identify factors that differentially confer risk for suicidal ideation and behavior, the development of a theory-based MMPI interpretive strategy could have substantial value. Additionally, numerous, large MMPI-2 databases exist from a range of patient populations, and the MMPI-2-RF can be scored from these existing MMPI-2 databases; thus, the development of MMPI-2-RF proxy scales would afford more avenues through which to study the IPTS. Furthermore, the MMPI family of instruments are the measures of personality and psychopathology most commonly used in clinical practice (Archer, Buffington-Vollum, Stredny, & Handel, 2006; Camara, Nathan, & Puente, 2000) and the most popular personality instruments taught in clinical psychology training programs (Ready & Veague, 2014). Several studies have been published in which the authors use idiosyncratic methods of approximating the IPTS variables when the IPTS measures were not available (e.g., J. C. Anestis et al., 2016; Cramer et al., 2012; Joiner et al., 2009; Smith et al., 2012). Having an established and uniform method of approximation would increase the validity of studies using a proxy method. To this end, the current investigation utilized an adult outpatient mental health sample to address two goals: (a) examine the associations between the IPTS measures and the MMPI-2-RF and (b) determine if the individual correlates can be improved by creating scale-based algorithms.

Hypotheses regarding Goal 1 were based on IPTS empirical and theoretical studies (e.g., Van Orden et al., 2010) and the data from the MMPI-2-RF Technical Manual (Tellegen & Ben-Porath, 2008/2011). The IPTS proposes that thwarted belongingness and perceived burdensomeness in combination comprise suicidal desire. Indeed, the two constructs are typically highly correlated (rs range from .42 to .72; Hill et al., 2015) and share many external correlates. As such, we predicted that this shared variance would result in both perceived burdensomeness and thwarted belongingness being associated with scales assessing broad emotional dysfunction (e.g., EID, RCd, Dysfunctional Negative Emotions [RC7], Negative Emotionality/Neuroticism–Revised [NEGE-r]) as well as SUI.

Although there is considerable shared variance, the IPTS asserts that the two constructs are differentiable and have distinct associations (Joiner, 2005; Van Orden et al., 2010). Perceived burdensomeness has empirical and/or conceptual relations to poor physical health and chronic pain, feelings of low self-worth and helplessness, anxious rumination, and conflictual family relations (Kanzler, Bryan, McGeary, & Morrow, 2012; Van Orden et al., 2010; Van Orden et al., 2012). We predicted that perceived burdensomeness would be specifically related to measures of somatic complaints (Somatic Complaints [RC1], Gastrointestinal Complaints [GIC], Malaise [MLS], Head Pain Complaints [HPC], Neurological Complaints [NUC], Cognitive Complaints [COG]), internalizing scales related to low self-worth and helplessness (SFD, HLP, Inefficacy [NFC]), and problematic family interactions (Family Problems [FML]). Thwarted belongingness has empirical and/or conceptual ties to unhappiness, isolation, stormy relationships, introversion, social awkwardness, and poor self-confidence (Van Orden et al., 2010; Van Orden et al., 2012). We predicted that thwarted belongingness would be positively related to internalizing and interpersonal indices of introversion and social disengagement (RC2, Interpersonal Passivity [IPP], Social Avoidance [SAV], Shyness [SHY], Introversion/Low Positive Emotionality–Revised [INTR-r]), scales related to feelings of alienation (Cynicism [RC3], Ideas of Persecution [RC6]), and conflictual interpersonal relationships (FML, Juvenile Conduct Problems [JCP]). We also predicted thwarted belongingess would be negatively correlated with a measure of interpersonal assertiveness (Aggressiveness–Revised [AGGR-r]).

The acquired capability for suicide is developed through repeated exposure to painful and provocative experiences that result in decreased fear of pain and death and an increase in pain tolerance (Joiner, 2005). Thus, we predicted that the ACSS would be positively correlated with MMPI-2-RF indices assessing personality traits related to engagement in painful and provocative experiences (M. D. Anestis, Soberay, Gutierrez, Hernandez, & Joiner, 2014; DeWall, Anderson, & Bushman, 2011; Joiner, 2005; Ribeiro, Silva, & Joiner, 2014; Tellegen & Ben-Porath, 2008/2011; Van Orden et al., 2010): global measures of pain and somatic discomfort (RC1), a history of suicide attempts (SUI), overactivation (Hypomanic Activation [RC9], Activation [ACT]), externalizing scales related to impulsivity and aggression (BXD, RC4, Aggression [AGG], Juvenile Conduct Problems [JCP], Substance Abuse [SUB], Disconstraint–Revised [DISC-r]), and interests related to adventure and sensation seeking (Mechanical–Physical Interests [MEC]). We also predicted that the ACSS would be negatively correlated with indices of fearfulness (Behavior-Restricting Fears [BRF], Multiple Specific Fears [MSF]).

When considering development of proxy indices of the IPTS measures (Goal 2), a conscious emphasis was placed on capturing the nomological network of the construct in question, as opposed to relying on an empirical method for index development. This approach is consistent with goals to link the MMPI-2-RF to contemporary models and theory of personality and psychopathology (e.g., Ben-Porath & Tellegen, 2008/2011; Sellbom & Ben-Porath, 2005; Tellegen & Ben-Porath, 2008/2011; Tellegen, Ben-Porath, & Sellbom, 2009; Tellegen et al., 2006). The following scales were selected to approximate perceived burdensomeness: Malaise (MLS), Helplessness/Hopelessness (HLP), Self-Doubt (SFD), Inefficacy (NFC), Stress and Worry (STW), and Family Problems (FML). The following MMPI-2-RF scales were selected to approximate thwarted belongingness: Low Positive Emotions (RC2), Juvenile Conduct Problems (JCP), FML, Interpersonal Passivity (IPP), Social Avoidance (SAV), Shyness (SHY), and low scores on Aggressiveness–Revised (AGGR-r). Regarding the acquired capability for suicide, a number of events and personality traits are thought to result in lowered fear of death and increased tolerance for physical pain. Of these factors, the MMPI-2-RF contains scales assessing physical aggression toward others (Aggression [AGG], JCP, AGGR-r), history of suicide attempts (SUI), low anxiety and fear-related emotions (low scores on Behavior Restricting Fears [BRF] and Multiple Specific Fears [MSF]), overarousal (Activation [ACT]), engagement in physical activities that may result in physical injury (Mechanical–Physical Interests [MEC]), and impulsive sensation seeking (Disconstraint-Revised [DISC-r]).

Method

Participants

Participants were 1,029 adult outpatients (57.6% female) who presented for psychological services at the Florida State University (FSU) Psychology Clinic and produced a valid profile on the MMPI-2-RF (exclusionary criteria used: CNS ≥ 18, VRIN-r or TRIN-r ≥ 80T, or Fp-r ≥ 100T¹). Ages ranged from 18 to 66 (M = 27.08, SD = 10.14). Race/ethnicity was generally consistent with the representative ethnic breakdown of Tallahassee, Florida. The majority of the sample identified as White (76.0%), with 10.1% Hispanic, 9.9% Black, 2.3% Asian/Pacific Islander, 1.5% Alaskan/American Indian, and 0.3% Other. Regarding marital status, 76.4% reported being never married, 10.9% married, 10.4% divorced, 1.4% separated, and 0.8% widowed.

The FSU Psychology Clinic serves the community as well as university students and staff, and due to the clinic’s inexpensive sliding scale fee schedules, individuals receiving services at the clinic are generally of lower socioeconomic status. In the current sample, 51.6% reported being unemployed at the time of initial presentation at the clinic. In terms of exclusionary criteria based on symptomatology, arrangements for inpatient hospitalization are made for individuals presenting with florid mania, marked psychosis, or imminent suicide risk. All other presentations are readily accepted. As a result, outpatients served by the FSU clinic present with a variety of conditions that range in severity (see Cukrowicz et al., 2011, for a full description). The most prevalent diagnostic categories in this outpatient sample were mood disorders, alcohol use disorders, generalized anxiety disorder, cannabis use disorders, social phobia, and borderline personality disorder.

Procedure

All outpatients who present at the FSU Psychology Clinic complete a large battery of screening questionnaires prior to receiving psychological services. The data presented in this study were drawn from this large battery of screening questionnaires. All outpatients included in the sample provided informed consent to participate in the FSU Institutional Review Board–approved research.

Measures

Minnesota Multiphasic Personality Inventory–2–Restructured Form (Ben-Porath & Tellegen, 2008/2011)

The MMPI-2-RF is a 338-item self-report questionnaire designed to measure personality characteristics and psychopathology. All items use true/false responding and make up six sets of scales: nine Validity scales, three Higher Order (HO) Scales, nine RC Scales, 23 Specific Problems (SP) Scales, two Interest Scales, and a revised version of the Personality Psychopathology Five Scales (PSY-5; Harkness et al., 2014). Data on psychometric properties and external correlates are reported in the MMPI-2-RF: Technical Manual (Tellegen & Ben-Porath, 2008/2011).

In the present study, participants were administered the MMPI-2 on presentation for services. As the 338 items of the MMPI-2-RF come from the MMPI-2 item pool, MMPI-2 protocols were later rescored into MMPI-2-RF scales for research purposes. Evidence for the validity of this strategy has been reported in prior studies (Tellegen & Ben-Porath, 2008/2011; Van Der Heijden, Egger, & Derksen, 2010). Cronbach’s α ranged from .52 (DSF) to .93 (EID). Unless otherwise noted, raw scale scores were used in all analyses.

Interpersonal Needs Questionnaire (Van Orden et al., 2012)

The INQ is a 15-item self-report questionnaire designed to assess an individual’s feelings of thwarted belongingness (INQ-TB; nine items) and perceived burdensomeness (INQ-PB; six items). Each item is scored on a 7-point scale ranging from 0 (not at all true for me) to 7 (very true for me), with higher scores indicating higher levels of thwarted belongingness or perceived burdensomeness. Acceptable psychometric properties for the INQ have been reported, including convergent validity (Van Orden et al., 2012). There was evidence of good internal consistency in the current sample (α = .77 and .88 for thwarted belongingness and perceived burdensomeness, respectively).

Acquired Capability for Suicide Scale (Van Orden et al., 2008)

The ACSS is a self-report measure designed to assess one’s fearlessness about self-injury and death. Each item is scored on a 5-point scale ranging from 1 (not at all like me) to 5 (very much like me), with higher scores indicating greater acquired capability for suicide. The current study used the five-item version of this scale, which has been used in many previous studies (see Ribeiro, Witte, et al., 2014, for review) with reports of acceptable convergent and discriminant validity (Van Orden et al., 2008). Internal consistency of the ACSS for the current study was adequate (α = .67).

Beck Scale for Suicidal Ideation (Beck & Steer, 1996)

The BSS is a 21-item self-report questionnaire designed to assess current thoughts of suicide. Each item includes three responses scored from 0 to 2 with higher scores indicating greater severity of suicidal ideation. Items 1 to 19 are used to measure suicidal ideation. Internal consistency in the current study was good (α = .88).

Clinician-Rated Suicide Risk (Chu et al., 2015; Joiner, Walker, Rudd, & Jobes, 1999)

Clinician ratings of suicide risk were determined based on a standardized empirically informed suicide risk assessment interview developed by Joiner et al. (1999) and updated by Chu et al. (2015). Suicide risk is determined by: (a) the presence of previous suicidal behavior, (b) the presence of current resolved plans and preparations, (c) the presence of current suicidal ideation, and (d) any other notable risk factors. As indicated by Joiner et al. (1999) and Chu et al. (2015), these factors are used to categorize individuals into one of the following suicide risk levels: low, low-to-moderate, moderate, moderate-to-severe, severe, severe-to-extreme, and extreme. This framework has been used as a means of delineating empirically informed suicide risk levels in a number of past studies that provide evidence for their construct validity and reliability (e.g., Van Orden et al., 2008; Witte, Timmons, Fink, Smith, & Joiner, 2009). Each clinician in the FSU clinic is trained in the use of this framework by Dr. Joiner.

Beck Depression Inventory–II (Beck, Steer, & Garbin, 1998)

The BDI-II is a 21-item self-report questionnaire used to assess current symptoms and severity of depression. Each item is endorsed on a Likert-type scale ranging from 0 to 3, with higher scores indicating higher levels of depression. For the current study, responses on Item 7 (“I dislike myself”) were examined in the cross-validation sample for the purposes of convergent/discriminant validity.

Application for Services

All potential clients complete an application for services for the FSU Psychology Clinic. This application asks for a range of demographic, social, and life history variables. For the purposes of convergent/discriminant validity, we utilized client endorsement (or nonendorsement) to an item regarding if the client has ever experienced “Social isolation.” Client suicide attempt history was also gleaned from responses to the question “If you have ever attempted suicide, when did your most recent attempt occur?” Possible responses are “I have never attempted suicide”; “Within the last month”; “More than 1 month ago, but within the last year”; “More than 1 year ago, but less than 5 years ago”; “More than 5 years ago.” These responses were collapsed into “No history of attempt” or “History of attempts.”

Data Analytic Procedure

Zero-order correlations were calculated for the MMPI-2-RF, the ACSS, the two INQ scales, the BSS, and clinician-rated suicide risk.

To develop the proxy indices, the sample (n = 1,029) was randomly divided into a derivation sample (66%; n = 680) and a cross-validation sample (34%; n = 351). The derivation sample was used for scale construction. Following the example of Sellbom et al. (2012), we used Mosier’s (1951) double cross-validation procedure to construct IPTS proxy indices from the MMPI-2-RF substantive scales. The derivation sample was divided into two random halves (n = 340 in each). Multiple regression analyses were conducted in Subgroup 1, with selected RF scales predicting an IPTS construct. The standardized beta weights were then used to calculate a proxy scale in Subgroup 2. The same procedure was followed in the Subgroup 2: Multiple regression analyses with RF scales predicting an IPTS construct were conducted and the standardized beta weights were used to calculate a proxy scale in Subgroup 1. To examine the parity of constructs across the two subgroups, the zero-order correlation was examined between the proxy indices constructed in each subgroup. Once comparability was demonstrated, the same multiple regressions were generated using the full sample. Standardized beta weights from the regressions were then used to calculate the final proxy indices: RF-PB, RF-TB, RF-AC.

The cross-validation sample was used to test convergent, discriminant, and incremental validity of the MMPI-2-RF proxy indices, using actual IPTS scale scores along with relevant external correlates. Zero-order and partial correlations (prs) were conducted, along with multiple regression analyses using the PROCESS macro for SPSS (Hayes, 2013) for the purposes of replicating previous IPTS interaction findings with the proxy indices.

Results

Data Issues

As is expected for suicide-related variables, SUI, BSS, and clinician-rated suicide risk were each zero-inflated, resulting in significant skew and kurtosis. Transformations brought these values to accepted levels (square root transformation for BSS and SUI; Blom transformation for clinician-rated suicide risk).

Zero-Order Correlations

Initial zero-order correlations using the full sample are displayed in Table 1. Given the size of the sample and the number of correlational analyses conducted, we elected to focus on correlations ≥ .20. As predicted, both INQ-PB and INQ-TB displayed large positive correlations with EID, RCd, RC7, SUI, and NEGE-r. These results indicate that both INQ-PB and INQ-TB are associated with suicidal ideation, mood problems (e.g., depression, irritability, anger, anxiety, worry, fear), general unhappiness, and dissatisfaction. In examining zero-order correlations for MMPI-2-RF scales expected to differentiate between INQ-PB and INQ-TB, the expected pattern did not occur. We saw equivalent correlations between INQ-PB and INQ-TB for many MMPI-2-RF scales, including some that were not predicted. For example, it was hypothesized that INQ-PB, but not INQ-TB, would display significant correlations with indices related to poor physical health and chronic pain; yet both INQ subscales displayed large significant correlations (rs range from .37 to .63) with RC1, MLS, GIC, HPC, NUC, and COG. In other cases, there were larger correlations with INQ-TB relative to INQ-PB (but not vice versa). Some hypothesized relationships were not observed. For example, contrary to prediction, INQ-TB was minimally associated with IPP and JCP.

Table 1.

Correlations Between MMPI-2-RF Scales, Interpersonal Needs Questionnaire, Acquired Capability for Suicide Scale, Beck Scale for Suicidal Ideation, and Clinician-Rated Suicide Risk.

	INQ-PB	INQ-TB	ACSS	BSS	Risk^a
EID (Emotional/Internalizing Dysfunction)	.60	.76	−.00	.46	.40
THD (Thought Dysfunction)	.36	.31	.13	.21	.19
BXD(Behavioral/Externalizing Dysfunction)	.15	.18	.28	.12	.15
RCd (Demoralization)	.58	.73	.04	.45	.39
RC1 (Somatic Complaints)	.46	.45	.06	.33	.28
RC2 (Low Positive Emotions)	.52	.68	−.02	.42	.37
RC3 (Cynicism)	.30	.36	.11	.19	.18
RC4 (Antisocial Behavior)	.26	.32	.22	.19	.25
RC6 (Ideas of Persecution)	.39	.35	.11	.22	.25
RC7 (Dysfunctional Negative Emotions)	.49	.56	−.04	.32	.31
RC8 (Aberrant Experiences)	.37	.37	.18	.27	.25
RC9 (Hypomanic Activation)	.12	.14	.21	.08	.08
MLS (Malaise)	.50	.62	.00	.38	.34
GIC (Gastrointestinal Complaints	.34	.37	.02	.26	.23
HPC (Head Pain Complaints)	.40	.37	.02	.25	.21
NUC (Neurological Complaints)	.37	.35	.11	.27	.26
COG (Cognitive Complaints)	.39	.47	.07	.30	.25
SUI (Suicidal/Death Ideation)	.51	.44	.13	.70	.53
HLP (Helplessness/Hopelessness)	.49	.55	.02	.43	.32
SFD (Self-Doubt)	.54	.67	.03	.40	.33
NFC (Inefficacy)	.38	.52	−.06	.30	.28
STW (Stress/Worry)	.39	.50	−.06	.24	.26
AXY (Anxiety)	.43	.45	−.01	.28	.28
ANP (Anger Proneness)	.30	.35	−.00	.22	.21
BRF (Behavior-Restricting Fears)	.26	.25	−.15	.14	.18
MSF (Multiple Specific Fears)	.10	.05	−.26	.01	.04
JCP (Juvenile Conduct Problems)	.16	.17	.20	.10	.15
SUB (Substance Abuse)	.07	.16	.12	.09	.15
AGG (Aggression)	.27	.30	.18	.23	.25
ACT (Activation)	.11	.13	.11	.07	.07
FML (Family Problems)	.46	.55	.13	.28	.26
IPP (Interpersonal Passivity)	.15	.21	−.21	.14	.13
SAV (Social Avoidance)	.37	.49	.01	.29	.31
SHY (Shyness)	.35	.53	−.06	.26	.21
DSF (Disaffiliativeness)	.27	.38	.08	.23	.16
AES (Aesthetic–Literary Interests)	.13	.16	−.03	.05	.08
MEC (Mechanical–Physical Interests)	−.07	−.10	.28	−.09	−.09
AGGR-r (Aggressiveness–Revised)	−.07	−.14	.24	−.08	−.04
PSYC-r (Psychoticism–Revised)	.36	.34	.12	.23	.21
DISC-r (Disconstraint–Revised)	.04	.07	.31	.05	.06
NEGE-r (Negative Emotionality/Neuroticism–Revised)	.45	.54	−.06	.31	.30
INTR-r (Introversion/Low Positive Emotionality–Revised)	.43	.56	−.00	.34	.32

Note. INQ-PB = Interpersonal Needs Questionnaire Perceived Burdensomeness scale; INQ-TB = Interpersonal Needs Questionnaire Thwarted Belongingness scale; ACSS = Acquired Capability for Suicide Scale; BSS = Beck Scale for Suicidal Ideation; Risk = Clinician-Rated Suicide Risk. All r ≥ .07 were statistically significant. For ease of reading, we have highlighted those ≥.2 in bold.

Clinician-rated suicide risk is an ordinal rating; Spearman’s rho was calculated.

In order to further explore this pattern of associations, post hoc prs were calculated to examine the associations of INQ-PB without the effect of INQ-TB, and vice versa. Doing so resulted in a somewhat clearer picture of the unique associations of INQ-PB and INQ-TB, and the results are presented in Table 2. Specifically, when controlling for INQ-TB, INQ-PB displayed unique correlations (r > .20) with disordered thinking and suspiciousness (THD, RC6), head and neck pain (HPC), and thoughts of suicide (SUI). The finding regarding SUI was surprising, as we expected SUI to be associated with both INQ-PB and INQ-TB. When controlling for INQ-PB, INQ-TB displayed unique correlations (r > .20) with a lack of positive emotional experiences (RC2), cynicism (RC3), unusual thoughts and perceptions (RC8), gastrointestinal complaints (GIC), cognitive difficulties (COG), and feeling indecisive and inefficacious (NFC). Several MMPI-2-RF scales remained correlated with both INQ-PB and INQ-TB, even when controlling for the other INQ scale (e.g., EID, RCd, RC1, RC7, MLS, HLP, SFD, AXY, FML).

Table 2.

Partial Correlations for MMPI-2-RF Scales and Interpersonal Needs Questionnaire (Controlling for the Other INQ Scale).

	INQ-PB	INQ-TB
EID (Emotional/Internalizing Dysfunction)	.24	.62
THD (Thought Dysfunction)	.21	.13
BXD (Behavioral/Externalizing Dysfunction)	.07	.10
RCd (Demoralization)	.23	.59
RC1 (Somatic Complaints)	.26	.23
RC2 (Low Positive Emotions)	.16	.53
RC3 (Cynicism)	.10	.24
RC4 (Antisocial Behavior)	.10	.19
RC6 (Ideas of Persecution)	.21	.16
RC7 (Dysfunctional Negative Emotions)	.22	.38
RC8 (Aberrant Experiences)	.18	.20
RC9 (Hypomanic Activation)	.07	.07
MLS (Malaise)	.20	.45
GIC (Gastrointestinal Complaints)	.13	.23
HPC (Head Pain Complaints)	.25	.16
NUC (Neurological Complaints)	.19	.18
COG (Cognitive Complaints)	.15	.31
SUI (Suicidal/Death Ideation)	.34	.18
HLP (Helplessness/Hopelessness)	.21	.37
SFD (Self-Doubt)	.21	.51
NFC (Inefficacy)	.10	.38
STW (Stress/Worry)	.10	.36
AXY (Anxiety)	.20	.27
ANP (Anger Proneness)	.12	.22
BRF (Behavior-Restricting Fears)	.11	.14
MSF (Multiple Specific Fears)	.06	.01
JCP (Juvenile Conduct Problems)	.07	.09
SUB (Substance Abuse)	−.04	.14
AGG (Aggression)	.12	.17
ACT (Activation)	.08	.06
FML (Family Problems)	.20	.37
IPP (Interpersonal Passivity)	.01	.16
SAV (Social Avoidance)	.07	.37
SHY (Shyness)	.05	.42
DSF (Disaffiliativeness)	.05	.28
AES (Aesthetic–Literary Interests)	.02	.12
MEC (Mechanical–Physical Interests)	.02	−.09
AGGR-r (Aggressiveness–Revised)	.02	−.12
PSYC-r (Psychoticism–Revised)	.18	.17
DISC-r (Disconstraint–Revised)	.02	.04
NEGE-r (Negative Emotionality/Neuroticism–Revised)	.15	.39
INTR-r (Introversion/Low Positive Emotionality–Revised)	.08	.44

Note. INQ-PB = Interpersonal Needs Questionnaire Perceived Burdensomeness scale; INQ-TB = Interpersonal Needs Questionnaire Thwarted Belongingness scale; All pr ≥ .1 were statistically significant. For ease of reading, we have highlighted those ≥.2 in bold.

As predicted, ACSS scores were positively correlated with externalizing scales related to impulsivity and aggression (BXD, RC4, JCP, MEC, AGRR-r, DISC-r) and overactivation (RC9; see Table 1). As expected, ACSS was also negatively correlated with MSF, indicating that higher scores on the ACSS are associated with low numbers of specific fears and phobias. ACSS was also negatively correlated with IPP. Low scores on IPP are indicative of interpersonal assertiveness. Contrary to predictions, correlations between ACSS and both ACT and BRF were small (<.20; although in the predicted directions).

In Table 1, we also report correlations between the MMPI-2-RF scales, the BSS, and clinician-rated suicide risk level. A large number of significant, positive correlations were observed. For both measures, SUI demonstrated the largest associations (BSS r = .70, suicide risk level r = .53).

Development of Proxy Indices

Prior to the explication of the proxy indices, a decision was made to prioritize theory over statistics in the creation of the indices. One of the primary purposes in deriving the RC scales and restructuring the MMPI-2 was to link the instrument to contemporary models and theory of personality and psychopathology (e.g., Ben-Porath & Tellegen, 2008/2011; Sellbom & Ben-Porath, 2005; Tellegen & Ben-Porath, 2008/2011; Tellegen et al., 2006; Tellegen et al., 2009). By relying on theory, as opposed to empirical methods, for index development, we attempted to determine if the MMPI-2-RF could capture one such contemporary model (the IPTS) at the scale level. As a first step, four experts on the IPTS and/or the MMPI-2-RF (JCA, JAF, RCA, MDA) independently reviewed descriptions and correlates of the MMPI-2-RF (those more familiar with the MMPI-2-RF than the IPTS were given Van Orden et al., 2010, as a reference). Each evaluator selected candidate MMPI-2-RF scales for each of the three proxy IPTS indices based on each construct. The first author combined the responses and then facilitated feedback and discussion from each expert in turn. Through this process, responses across experts were compared and distilled into a consensus list of possible MMPI-2-RF contributors to the IPTS proxy indices. The HO, RC, and PSY-5 scales were removed from proxy index construction to reduce construct overrepresentation and excessive collinearity. There were three exceptions to this rule: RC2 and AGGR-r for the TB index and DISC-r for the AC index. RC2 was retained for the TB index as there are not lower order Specific Problems Scales covering the same construct. DISC-r is the best measure of impulsivity, an important aspect in the development of the AC. AGGR-r contributed meaningfully to the TB index and contains different items from AGG, and thus was allowed to remain. The resulting consensus list was used in the analyses presented below.²

Proxy Indices in the Derivation Sample

Perceived burdensomeness

In attempting to capture perceived burdensomeness, the following scales were utilized: MLS, HLP, SFD, NFC, STW, and FML. Using Subgroup 1, a regression was conducted with the selected MMPI-2-RF scales predicting INQ-PB (R = .64). Next, the standardized betas from Sample 1 were used to construct an RF-PB scale in subgroup 2. RF-PB in Subgroup 2 correlated with INQ-PB at .63. This process was repeated in Subgroup 2. A regression analysis was conducted with the selected RF scales predicting INQ-PB (R = .65). The standardized betas from Subgroup 2 were used to construct RF-PB in Subgroup 1. RF-PB in Subgroup 1 correlated with INQ-PB at .58. The correlation between RF-PB calculated in Subgroups 1 and 2 indicated the same constructs were being measured (r = .95). Given the comparability of the scale in each subgroup, the same regression analysis was conducted in the whole derivation sample (R = .64). Using the standardized betas from the full sample, RF-PB was constructed in the full derivation sample. The correlation between RF-PB and INQ-PB was large (r = .63). Standardized betas for the overall derivation sample are reported in Table 3. As predicted, MLS, HLP, SFD, and FML significantly and positively contributed to the prediction of INQ-PB. Contrary to expectation, NFC negatively predicted INQ-PB, despite demonstrating a positive correlation at the zero-order level. The behavior of NFC in the regression could indicate the presence of a suppressor. It could also be a result of shared variance with the other internalizing scales. Also unexpectedly, STW was not a significant predictor of INQ-PB. This suggests that the unique aspects of STW not shared by the other internalizing scales do not contribute appreciably to the model.

Table 3.

Standardized Beta Weights for Predicting INQ-Burdensomeness, INQ-Belongingness, and Acquired Capability for Suicide Scale in the Overall Derivation Sample and Intercorrelations Among the IPTS Measures and Their MMPI-2-RF Counterparts (n = 680).

INQ-PB		INQ-TB		ACSS
	β		β		β
MLS	.190***	RC2	.494***	SUI	.151***
HLP	.200***	JCP	−.033	BRF	−.157***
SFD	.344***	FML	.315***	MSF	−.208***
STW	−.001	IPP	−.005	AGG	.046
NFC	−.150**	SAV	.002	ACT	.055
FML	.176***	SHY	.192***	JCP	.090
—	—	AGGR-r	.098	MEC	.207***
—	—	—	—	DISC-r	.028
				AGGR-r	.118*
	r		r		r
RF-PB	.63***		.75***		.12**
RF-TB	.57***		.77***		.06
RF-AC	.02		−.02		.49***

Note. INQ-PB = Interpersonal Needs Questionnaire Perceived Burdensomeness scale; INQ-TB = Interpersonal Needs Questionnaire Thwarted Belongingness scale; ACSS = Acquired Capability for Suicide Scale; MLS = Malaise; HLP = Helplessness/Hopelessness; SFD = Self-doubt; STW = Stress and Worry; NFC = Inefficacy; FML = Family Problems; RC2 = Low Positive Emotions; JCP = Juvenile Conduct Problems; IPP = Interpersonal Passivity; SAV = Social Avoidance; SHY = Shyness; AGGR-r = Aggressiveness–Revised; SUI = Suicidal/Death Ideation; BRF = Behavior Restricting Fears; MSF = Multiple Specific Fears; AGG = Aggression; MEC = Mechanical–Physical Interests; DISC-r = Disconstraint–Revised.

p < .05. **p < .01. ***p < .001.

Thwarted belongingness

The following scales were identified as representing thwarted belongingness in the MMPI-2-RF: RC2, JCP, FML, IPP, SAV, SHY, and AGGR-r. Multiple Rs for RF-TB in the two subgroups were .77 and .78, respectively. Using the standardized betas derived in one subgroup to construct RF-TB in the other subgroup results in strong positive correlations between RF-TB and INQ-TB (rs = .77 and .75, respectively). Furthermore, the correlation between RF-TB calculated in each subgroup indicated the same constructs were being measured (r = .99). RF-TB was constructed in the overall derivation sample (R = .78), resulting in a large correlation with INQ-TB (r = .77). Standardized betas for the overall derivation sample are reported in Table 3. RC2, FML, and SHY positively contributed to the prediction of INQ-TB. Contrary to expectation, JCP, IPP, SAV, and AGGR-r did not significantly contribute to the predictive model, despite being correlated with INQ-TB at a zero-order level.

Acquired capability for suicide

The following scales were selected to represent RF-AC: SUI, BRF, MSF, AGG, ACT, JCP, MEC, DISC-r, and AGGR-r. Multiple Rs for RF-AC in the two subgroups were .48 and .56, respectively. When RF-AC was constructed in one sample using the standardized betas in the other subgroup, these scales were positively correlated with ACSS (rs = .48 and .45, respectively). The correlation between the RF-AC scales using Subgroups 1 and 2 indicated the same constructs were being measured (r = .92). RF-AC in the overall derivation sample (R = .51) was positively correlated with ACSS (r = .49). Standardized betas in the full sample are found in Table 3. SUI, MEC, and AGGR-r positively contributed to the model. As hypothesized, beta weights for BRF and MSF were statistically significant and negative. Contrary to expectation, AGG, ACT, JCP, and DISC-r were not significant predictors of ACSS, despite significant correlations at the zero-order level. Of note, JCP and DISC-r share items, suggesting that the shared variance does not contribute appreciably to the model.

We calculated a sum of the beta weights for each RF-IPTS scale in the derivation sample. These three indices (RF-PB, RF-TB, RF-AC) were used subsequent in analyses in the cross-validation sample.³

Incremental, Convergent, and Discriminant Validity in the Cross-Validation Sample

We next examined the constructed indices in the cross-validation sample. Zero-order correlations between the MMPI-2-RF proxy indices and the IPTS theory measures were in the expected direction across each construct. RF-TB demonstrated a large positive correlation with INQ-TB (r = .74, p < .001) and INQ-PB (r = .64, p < .001), and RF-PB demonstrated a large positive correlation with both INQ-PB (r = .61, p < .001) and INQ-TB (r = .70, p < .001). We note that these correlations indicate suboptimal discriminant validity between RF-PB and RF-TB; however, some previous studies have struggled to find clear discriminant validity between INQ-PB and INQ-TB (see Van Orden et al., 2012, and the correlations presented in Table 1 of this article). The correlation between RF-AC and ACSS demonstrated shrinkage and was smaller than expected (r = .33). Correlations between both RF-PB and RF-TB with the ACSS were not statistically significant (r = .07 and .04, respectively). Likewise, RF-AC was not associated with INQ-PB and INQ-TB (r = −.07 and −.04, respectively). These nonsignificant correlations are consistent with principles of the IPTS.

Three prs using RF-PB and RF-TB were conducted to further explore the convergent and discriminant validity of the proxy indices. As the theory would predict, we found that social isolation was related to RF-TB (pr controlling for RF-PB = .26, p < .001) but not RF-PB (pr controlling for RF-TB = .02, p = .682). We also found that self-dislike was related to RF-PB (pr controlling for RF-TB = .28, p < .001) but not RF-TB (pr controlling for RF-PB = .08, p = .320). Finally, RF-AC demonstrated a small, but positive, statistically significant relationship with a reported history of suicide attempts (r = .15, p = .009). While the zero-order correlations among RF-PB and RF-TB and the INQ subscales may suggest poor discriminant validity, these prs provide preliminary evidence that RF-PB and RF-TB are assessing two distinct constructs.

Table 4 displays results of zero-order and prs (controlling for SUI) examining the relationships between the MMPI-2-RF proxy indices, the IPTS theory measures, the BSS, and clinician-rated suicide risk level. Given that SUI had the largest correlations with the BSS and clinician-rate suicide risk level in this sample (Table 1), it was important to control for SUI to demonstrate the incremental validity of the proxy indices. As hypothesized, measures of perceived burdensomeness and thwarted belongingness were associated with the BSS at the zero-order level (e.g., RF-PB r = .52; RF-TB r = .53). When controlling for SUI, these relationships remained statistically significant, although the prs were small (RF-PB and RF-TB, prs = .16). While a stronger relationship was expected between acquired capability and suicide risk level, ACSS, but not RF-AC, was statistically significantly related to risk level at the zero-order level (r = .18). When controlling for SUI, this relationship was nonsignificant.

Table 4.

Zero-Order and Partial Correlations (Controlling for SUI) Between MMPI-2-RF Proxy Indices, Interpersonal Needs Questionnaire, Acquired Capability for Suicide Scale, Beck Suicide Scale, and Clinician-Rated Suicide Risk Level.

	BSS		Suicide Risk Level
	r	pr	r	pr
RF-PB	.52***	.16*	.38***	.05
RF-TB	.53***	.16*	.42***	.11
RF-AC	−.02	.000	−.04	−.03
INQ-PB	.62***	.30***	.51***	.21**
INQ-TB	.51***	.23**	.33***	.09
ACSS	.15*	.09	.18**	.12

Note. r = zero-order correlation, pr = partial correlation controlling for SUI. INQ-PB = Interpersonal Needs Questionnaire Perceived Burdensomeness scale; INQ-TB = Interpersonal Needs Questionnaire Thwarted Belongingness scale; ACSS = Acquired Capability for Suicide Scale; BSS = Beck Scale for Suicidal Ideation; SUI = Suicidal/Death Ideation.

p < .05. **p < .01. ***p < .001.

Finally, two multiple regression analyses were conducted to examine whether the proxy indices could reproduce previously reported findings with the IPTS constructs. Perceived burdensomeness and thwarted belongingness are theorized to comprise suicidal desire, and previous studies have reported that the interaction of perceived burdensomeness and thwarted belongingness predicts suicidal ideation (e.g., Hagan et al., 2015; Van Orden et al., 2008). To this end, we conducted a hierarchical multiple regression analysis in which the main effects of RF-PB and RF-TB were entered in Step 1 and the two-way interaction of RF-PB and RF-TB was entered in Step 2. All predictor variables were centered for this analysis. Total score on the Beck Suicide Scale served as the dependent variable. The overall model was significant (F_(3,324) = 51.96, p < .001), and the addition of the RF-PB × RF-TB interaction significantly improved the model (ΔR² = .02, p < .001, F² = .04). Analysis of simple slopes revealed that the relationship between RF-TB and suicidal ideation increased in magnitude from low (t = 1.19, p = .235) to mean (t = 3.33, p = .001) to high (t = 4.74, p < .001) levels of RF-PB⁴. Results from Step 2 are reported in Table 5.

Table 5.

MMPI-2-RF Proxy Indices for Perceived Burdensomeness and Thwarted Belongingness Predicting Suicidal Ideation.

	R ²	ΔR²	t	b	SE	p
Step 2	0.33	0.02
RF-PB			−0.65	−.10	0.15	.514
RF-TB			0.47	.03	0.06	.637
RF-PB*RF-TB			3.36	.67	0.02	<.001

Note. SE = standard error. Data from Step 2 of a hierarchical multiple regression is displayed. Main effects of RF-PB and RF-TB were entered in Step 1, and the interaction of RF-PB and RF-TB was added to Step 2. Total score on the Beck Suicide Scale served as the dependent variable. R² = R² for Step 2; ΔR² = change in R² with the addition of the interaction term.

A second multiple regression analysis strove to replicate findings that the three-way interaction between the theory variables represent the highest level of suicide risk (e.g., Joiner et al., 2009). In this regression, the main effect of each of the three MMPI-2-RF IPTS indices was added in Step 1, the two-way interaction between each of the MMPI-2-RF IPTS indices were added in Step 2, and the three-way interaction was added in Step 3. The dependent variable was clinician-rated suicide risk. The overall model was significant (F_(7,283) = 11.21, p < .001), and the addition of the three-way interaction resulted in statistically significant improvement to the model (ΔR² = .01, p =.044, F² = .01). Contrary to expectation, however, analysis of simple slopes revealed that the relationship between the RF-TB and RF-PB interaction and suicide risk decreased in magnitude from low (t = 3.24, p = .001) to mean (t = 3.01, p = .003) to high (t = .60, p = .546) levels of RF-AC⁵. Results from Step 3 are reported in Table 6.

Table 6.

MMPI-2-RF Proxy Indices of the Three IPTS Components Predicting Clinician-Rated Suicide Risk.

	R ²	ΔR²	t	b	SE	p
Step 3	0.20	0.01
RF-PB			−2.17	−.27	0.13	.031
RF-TB			−1.25	−.06	0.05	.212
RF-AC			−1.16	−.13	0.11	.246
RF-PB*RF-TB			3.40	.06	0.02	.001
RF-PB*RF-AC			1.39	.09	0.07	.166
RF-TB*RF-AC			1.35	.04	0.03	.177
IPTS			−2.00	−.02	0.01	.047

Note. SE = standard error. Data from Step 3 of a hierarchical multiple regression is displayed. Main effects of RF-PB, RF-TB, and RF-AC were entered in Step 1, the two-way interactions of RF-IPTS scales were entered in Step 2, and the three-way interaction of all RF-IPTS scales was added to Step 3. Clinician-rated suicide risk served as the dependent variable. R² = R² for Step 3; ΔR² = change in R² with the addition of the three-way interaction term.

Discussion

The current study is the first to formally examine the relationship between IPTS constructs and the MMPI-2-RF. We reported correlates between INQ and ACSS scores and MMPI-2-RF substantive scale scores, which largely followed a pattern consistent with predictions based on the theory. Furthermore, we provide preliminary evidence from a clinical sample that proxy indices for the IPTS constructs can be reasonably approximated using algorithms derived from MMPI-2-RF scales. Specifically, the RF-TB and RF-PB indices demonstrated associations with INQ indices in a theoretically consistent pattern. A similar association was found between the RF-AC scale and the ACSS; however, the relationship was not as strong or as consistent as predicted. Given that published studies have used idiosyncratic methods to approximate the IPTS constructs from personality measures (e.g., J. C. Anestis et al., 2016; Cramer et al., 2012; Joiner et al., 2009; Smith et al., 2012), the present study represents an important step toward establishing a uniform proxy procedure across future studies.

Suicidal Desire

Consistent with the IPTS and our hypotheses, large positive correlations were observed between both INQ-PB and INQ-TB and the MMPI-2-RF internalizing scales (EID, RCd, RC7, SUI, and NEGE-r). Both INQ-PB and INQ-TB were most strongly associated with EID and RCd, suggesting that as unpleasant emotional experiences and feelings of being overwhelmed increased so did the perceptions of burdensomeness and belongingness. These strong relationships between INQ-PB and INQ-TB and internalizing/emotional dysfunction are consistent with studies demonstrating strong association between suicidality and internalizing liability (e.g., Eaton et al., 2013; Naragon-Gainey & Watson, 2011; Sunderland & Slade, 2015).

The IPTS further asserts that perceived burdensomeness and thwarted belongingness are distinct constructs. Evidence for the discriminant validity of INQ-PB and INQ-TB was limited in this sample. In addition to the shared associations with the internalizing psychopathology mentioned above, both INQ-PB and INQ-TB were positively correlated with all RC scales except RC9, all Somatic/Cognitive and Internalizing Specific Problem scales, and all Interpersonal Specific Problems Scales except IPP. When examining prs to identify unique associations, the strongest unique associations were between INQ-PB and SUI and between INQ-TB and INTR-r and SHY. This suggests that as an individual’s report of suicidal ideation and history of suicide attempts increased so did the perception that she or he was a burden to others. Additionally, individuals with introverted, shy, and socially inhibited personality characteristics were more likely to report a decreased sense of belonging.

Initial examination of the MMPI-2-RF proxy indices of RF-PB and RF-TB suggest adequate construct validity (i.e., INQ-PB was strongly associated with the developed RF-PB proxy index, and INQ-TB was strongly associated with the developed RF-TB proxy index). Despite the inclusion of theoretically relevant (and in most cases, distinct) MMPI-2-RF scales in the construction of the proxy indices, limited discriminant validity between RF-PB and RF-TB was observed (e.g., r between INQ-TB and RF-TB = .77, r between INQ-TB and RF-PB = .75). Several other studies (though certainly not all) also report mixed discriminant validity evidence for particular correlates of INQ-PB and INQ-TB. For example, Van Orden et al. (2012) found that both INQ-PB and INQ-TB were significantly related to low social worth, as opposed to just INQ-PB as the theory would predict. Marty, Segal, Coolidge, Klebe (2012) found that loneliness and reciprocal care had stronger (large) correlations with INQ-PB, although the theory would predict a stronger association between these indices and INQ-TB. Both INQ subscales were equally correlated with self-esteem, instead of showing a stronger relationship with INQ-PB as predicted (Marty et al., 2012).

Although limited discriminant validity was apparent between the INQ subscales and RF-PB/RF-TB, preliminary evidence with external correlates in the cross-validation sample was more promising. Both RF-PB and RF-TB demonstrated positive correlations with a measure of suicidal ideation. Discriminant relations were observed in that RF-TB, but not RF-PB, was associated with an index of social isolation, and RF-PB, but not RF-TB, was associated with an index of self-dislike. According to the IPTS, the interaction between high thwarted belongingness and high perceived burdensomeness places an individual at greatest risk for elevated suicidal ideation; thus, testing this interaction was an important step in exploring the validity of the MMPI-2-RF proxy measures. Indeed, consistent with the theory, we found that the interaction between RF-PB and RF-TB predicted suicidal ideation, above and beyond each individual scale, such that those high on both RF-PB and RF-TB experienced the greatest level of suicidal ideation. Our results are consistent with previous findings that the presence of both interpersonal states contributes to the strongest desire for suicide (Van Orden et al., 2008).

These analyses were repeated controlling for SUI, in order to demonstrate the incremental validity of the proxy indices above and beyond SUI. When controlling for SUI, RF-PB and RF-TB both significantly predicted scores on the BSS. This finding indicates that both RF-PB and RF-TB have unique association with a measure of suicidal ideation, above and beyond the effects of the SUI construct. In the two-way interaction described above, when controlling for SUI, the interactive effect of RF-TB and RF-PB was not statistically significant.

In sum, consistent with others, we have presented evidence of the limited distinction between burdensomeness and belongingness as measured by the INQ. We also have presented evidence of the largely adequate convergent, discriminant, and incremental validity of MMPI-2-RF proxy indices of these constructs. We assert based on the current findings that the combination of the two constructs (as assessed by the MMPI-2-RF) reasonably approximate suicidal desire as proposed by Joiner (2005). Partial correlations with external correlates (both those controlling for the other aspect of suicidal desire and those controlling for SUI) support this assertion that the proxy indices offer something unique above and beyond SUI and also represent the constructs in question. The multiple regression analyses examining the interaction of PB and TB also support theory-based hypotheses about the proxy indices, though we acknowledge that this interaction became statistically nonsignificant when controlling for SUI. Caution, however, must be used when interpreting the analyses using SUI as a control variable. Although controlling for SUI was an important component of demonstrating incremental validity, doing so is problematic. Criterion contamination is a concern because use of an explicit measure of suicidality (i.e., SUI) to predict self-reported suicidal ideation (i.e., BSS) will leave little variance left to predict. SUI represents a high bar to overcome. Relatedly, SUI relies heavily on historical (trait) variables (e.g., past attempts), whereas perceived burdensomeness and thwarted belongingness are considered to be dynamic (state) constructs—distorted perceptions which change over time. The results of the current analyses suggest that RF-PB and RF-TB measure variance not accounted for by SUI.

Capability for Suicide

As expected, acquired capability, as measured by the ACSS, demonstrated small to moderate associations with indicators of impulsivity and aggression (BXD, RC4, JCP, MEC, AGGR-r, DISC-r) and overactivation (RC9). Higher levels of fearlessness about self-injury and death are associated with externalizing behavioral dysfunction and poor behavioral controls. In the context of the IPTS, the current results support the MMPI-2-RF manual’s recommendation that, if SUI is elevated, the risk for suicide may be exacerbated by elevations on BXD and DISC-r (Ben-Porath & Tellegen, 2008/2011). ACSS was negatively correlated with MSF, such that increased levels of acquired capability were associated with decreased fear. ACSS scores were also negatively correlated with IPP. Being interpersonally passive runs counter to acting-out behaviors and behaviors associated with risk of harm. Overall, although the effect sizes were smaller than expected, the pattern of associations was consistent with predictions based on the IPTS model.

The associations between ACSS and RF-AC scores, while moderate and acceptable, were weaker than anticipated (.48 in the derivation sample, .33 in the cross-validation sample). Neither RF-AC not ACSS were significantly related to clinician-rated suicide risk level with and without controlling for SUI. In the three-way interaction between the three MMPI-2-RF proxy indices, RF-AC performed contrary to expectation in the prediction of suicide risk in the cross-validation sample. The three-way interaction was not statistically significant when controlling for SUI.

The lower validity of RF-AC, relative to RF-TB and RF-PB, could be due to the difficulty capturing all facets of the acquired capability construct within the MMPI-2-RF item pool. Relative to burdensomeness and belongingness, the MMPI-2-RF contains less content that directly assesses capability for suicide, making approximating this construct potentially problematic. Scales selected for the RF-AC were based on theory and previous associations with correlates such as sensation seeking, aggressiveness, and low fearfulness; however, there is not content in the MMPI-2-RF to assess specific components such as fearlessness about death. In addition to possible construct underrepresentation, we did not have access to the ACSS-FAD (Ribeiro, Witte, et al., 2014), arguably a more valid indicator of the acquired capability. Future studies using the ACSS-FAD, as well as behavioral measurements of pain tolerance and fear, may be able to improve the RF-AC index. In addition to the possibility of measurement limitations, we must also acknowledge that the three-way interaction predicted in this study, as dictated by the IPTS, is not consistently demonstrated in studies. For example, Bryan, Morrow, Anestis, and Joiner (2010) found that the three-way interaction did not significantly predict suicidal history in a military sample, although an interaction between burdensomeness and acquired capability did. The nonsignificant three-way interaction findings, however, are not necessarily indicative of a problem with the proxy indices. They instead may highlight the use of imperfect outcome variable. The theory specifies that the three-way interaction predicts death or near-death by suicide (Joiner, 2005). The farther the outcome variable is from death or near-death by suicide, the weaker the prediction as specified by theory. In the case of the current study, the clinician-rated suicide risk variable may be much more laden with suicidal ideation than by imminent death by suicide, rendering it an imperfect outcome variable (the same critique can be made of Bryan et al., 2010). Future studies examining the three-way interaction would benefit from using a more proximal death-related index. Most likely, the multiplicative impact of these three methodological factors contributed to the unexpected acquired capability findings. Future studies refining the approximation of the IPTS constructs with the MMPI-2-RF are needed to determine if the RF-AC can be improved or if acquired capability simply cannot be approximated with the MMPI-2-RF.

Limitations and Future Directions

In addition to those addressed throughout, several other limitations also must be acknowledged. The sample was largely Caucasian; thus, it is unclear whether the results will generalize to more diverse samples. In constructing the proxy indices, two scales with overlapping item content were included (i.e., JCP and DISC-r); however, removing DISC-r diminished scale validity, indicating that the nonshared item content contributed meaningfully to the improved prediction. As we are presenting preliminary evidence, future studies are needed to replicate the associations between the IPTS constructs and the MMPI-2-RF scales and to further establish the validity of the MMPI-2-RF IPTS indices. Studies using participants from other settings (e.g., inpatient, veteran) and/or a longitudinal design to evaluate the utility of the RF-TB, RF-PB, and RF-AC indices to prospectively predict suicide attempts are needed.

Despite these limitations, the MMPI-2-RF IPTS proxy indices fill an important gap in the literature on suicide risk assessment. Recent work in suicidology has promoted the importance of considering variables that facilitate the transition from suicidal ideation to suicidal behavior (e.g., Klonsky & May, 2014), highlighting the fact that risk factors for thoughts of suicide have repeatedly proven to be poor predictors of suicidal behavior and that only a small portion of those who think about suicide go on to make an attempt (e.g., Nock et al., 2008). Theories such as the IPTS (Joiner, 2005), the Three Step Theory of Suicide (Klonsky & May, 2015), and the Motivational Volitional Model of Suicide (O’Conner, 2011) all emphasize this point, noting that certain factors facilitate thoughts of suicide whereas others enable individuals thinking about suicide to translate that ideation into action. On a more practical level, the MMPI-2-RF IPTS proxy scales have the potential to provide the clinician with an assessment of intra- and interpersonal factors that promote thoughts of suicide (thwarted belongingness and perceived burdensomeness) as well as a factor that helps differentiate between those who only think of suicide from those who think about and attempt suicide (capability for suicide). By addressing these mechanisms within the context of a broadband instrument that includes scales to assess self-report bias, the proxy indices can be interpreted more clearly in populations most at risk for suicide (e.g., older adults, military personnel) that have been shown to under report suicidal ideation when asked to respond directly to questions assessing IPTS-related variables (e.g., M. D. Anestis & Green, 2015; Cukrowicz et al., 2013). As such, for clinicians, these indices could serve as a more accurate method for assessing risk among individuals less likely to be forthright regarding current thoughts and plans for suicide than stand-alone measures of the IPTS model. As the MMPI-2/MMPI-2-RF are among the most commonly used measures in clinical practice, the continued validation of these theoretically derived IPTS indices from the MMPI-2-RF has the potential not only to expand the possibilities for theory-based research of suicidal behavior but also to increase the utility of these indices in clinical practice and improve the assessment and prediction of suicidal behavior.

Footnotes

Acknowledgements

We would like to thank Jessica D. Ribeiro for her contributions in the early stages of this project and Chrissy Gilder for her statistical consulting.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Anestis

J. C.

Anestis

M. D.

Rufino

K. A.

Cramer

C. J.

Miller

H. A.

Khazem

L. R.

Joiner

T. E.

(2016). Understanding the relationship between suicidality and psychopathy: An examination of the interpersonal-psychological theory of suicidal behavior. Archives of Suicide Research. Advance online publication. doi:10.1080/13811118.2015.1048399

Anestis

M. D.

Green

B. A.

(2015). The impact of varying levels of confidentiality on disclosure of suicidal thoughts in a sample of United States National Guard personnel. Journal of Clinical Psychology, 71, 1023-1030. doi:10.1002/jclp.22198

Anestis

M. D.

Joiner

T. E.

(2011). Examining the role of emotion in suicidality: Negative urgency as an amplifier of the relationship between components of the interpersonal-psychological theory of suicidal behavior and lifetime number of suicide attempts. Journal of Affective Disorders, 129, 261-269.

Anestis

M. D.

Khazem

L. R.

Mohn

R. S.

Green

B. A.

(2015). Testing the main hypotheses of the interpersonal-psychological theory of suicidal behavior in a large diverse sample of United States military personnel. Comprehensive Psychiatry, 60, 78-85. doi:10.1016/j.comppsych.2015.03.006

Anestis

M. D.

Moberg

F. B.

Arnau

R. C.

(2014). Hope and the interpersonal-psychological theory of suicidal behavior: Replication and extension of prior findings. Suicide and Life-Threatening Behavior, 44, 175-187. doi:10.1111/sltb.12060

Anestis

M. D.

Soberay

K. A.

Gutierrez

P. W.

Hernandez

T. D.

Joiner

T. E.

(2014). Reconsidering the link between impulsivity and suicidal behavior. Personality and Social Psychology Review, 18, 366-386. doi:10.1177/1088868314535988

Arbisi

P. A.

Polusny

M. A.

Erbes

C. R.

Thuras

Reddy

M. K.

(2011). Minnesota Multiphasic Personality Inventory–2 Restructured Form in National Guard soldiers screening positive for posttraumatic stress disorder and mild traumatic brain injury. Psychological Assessment, 23, 203-214. doi:10.1037/a0021339

Archer

R. P.

Buffington-Vollum

J. K.

Stredny

R. V.

Handel

R. W.

(2006). A survey of psychological tests use patterns among forensic psychologists. Journal of Personality Assessment, 87, 85-95. doi:10.1207/s15327752jpa8701_07

Beck

A. T.

Steer

R. A.

(1996). Manual for the Beck Scale for Suicidal Ideation. San Antonio, TX: Psychological Corporation.

10.

Beck

A. T.

Steer

R. A.

Garbin

M. G.

(1998). Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review, 8, 77-100. doi:10.1016/0272-7358(88)90050-5

11.

Ben-Porath

Tellegen

(2008/2011). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2-Restructured Form): Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press.

12.

Birklein

Depmeier

Rolke

Hansen

Rautenstrauss

Prawitt

Magerl

(2008). A family-based investigation of cold pain tolerance. Pain, 138, 111-118. doi:10.1016/j.pain.2007.11.012

13.

Boone

D. E.

(1994). Validity of the MMPI-2 Depression Content Scale with psychiatric inpatients. Psychological Reports, 74, 159-162. doi:10.2466/pr0.1994.74.1.159

14.

Bryan

C. J.

Morrow

C. E.

Anestis

M. D.

Joiner

T. E.

(2010). A preliminary test of the interpersonal-psychological theory of suicidal behavior in a military sample. Personality and Individual Differences, 48, 347-350.

15.

Butcher

J. N.

Graham

J. R.

Ben-Porath

Y. S.

Tellegen

Dahlstrom

W. G.

Kaemmer

(2001). MMPI-2 (Minnesota Multiphasic Personality Inventory-2): Manual for administration, scoring, and interpretation (Rev. ed.). Minneapolis: University of Minnesota Press.

16.

Camara

W. J.

Nathan

J. S.

Puente

A. E.

(2000). Psychological test usage: Implications in professional psychology. Professional Psychology: Research and Practice, 31, 141-154. doi:10.1037/0735-7028.31.2.141

17.

Centers for Disease Control and Prevention. (2014). Web-based injury statistics query and reporting system. Retrieved from http://www.cdc.gov/injury/wisqars/

18.

Chu

Klein

K. M.

Buchman-Schmitt

J. M.

Hom

M. A.

Hagan

C. F.

Joiner

T. E.

(2015). Routinized assessment of suicide risk in clinical practice: An empirically informed update. Journal of Clinical Psychology, 71, 1186-1200. doi:10.1002/jclp.22210

19.

Clopton

Baucom

(1979). MMPI ratings of suicide risk. Journal of Personality Assessment, 43, 296-299. doi:10.1207/s15327752jpa4303_12

20.

Clopton

Jones

(1975). Use of the MMPI in the prediction of suicide. Journal of Clinical Psychology, 31, 52-54. doi:10.1002/1097-4679(197501)31:1<52::aid-jclp2270310115>3.0.co;2-f

21.

Clopton

Pallis

Birtchnell

(1979). Minnesota Multiphasic Personality Inventory Profile Patterns of Suicide Attempters. Journal of Consulting and Clinical Psychology, 47, 135-139. doi:10.1037/0022-006X.47.1.135

22.

Clopton

Post

Larde

(1983). Identification of suicide attempter by means of MMPI profiles. Journal of Clinical Psychology, 39, 868-871. doi:10.1002/1097-4679(198311)39:6<868::aid-jclp2270390609>3.0.co;2-r

23.

Cramer

R. J.

Garza

M. J.

Henderson

C. E.

Ribiero

J. D.

Silva

Smith

A. R.

. . . White

(2012). A trait-interpersonal perspective on suicide risk in criminal offenders. Archives of Suicide Research, 16, 334-337. doi:10.1080/13811118.2013.722057

24.

Cukrowicz

K. C.

Jahn

D. R.

Graham

R. D.

Poindexter

E. K.

Williams

R. B.

(2013). Suicide risk in older adults: Evaluating models of risk and predicting excess zeros in a primary care sample. Journal of Abnormal Psychology, 122, 1021-1030.

25.

Cukrowicz

K. C.

Timmons

K. A.

Sawyer

Caron

K. M.

Gummelt

H. D.

Joiner

T. E.

(2011). Improved treatment outcome associated with the shift to empirically supported treatments in an outpatient clinic is maintained over a ten-year period. Professional Psychology: Research and Practice, 42, 145-152. doi:10.1037/a0021937

26.

Daigle

(2004). MMPI inmate profiles: Suicide completers, suicide attempters, and non-suicidal controls. Behavioral Sciences & the Law, 22, 833-842. doi:10.1002/bsl.618

27.

Devries

Farberow

(1967). A multivariate profile analysis of MMPIs of suicidal and nonsuicidal neuropsychiatric hospital patients. Journal of Projective Techniques and Personality Assessment, 31, 81-84. doi:10.1080/0091651x.1967.10120420

28.

DeWall

C. N.

Anderson

C. A.

Bushman

B. J.

(2011). The general aggression model: Theoretical extensions to violence. Psychology of Violence, 1, 245-258. doi:10.1037/a0023842

29.

Eaton

N. R.

Krueger

R. F.

Markon

K. E.

Keyes

K. M.

Skodol

A. E.

Wall

. . . Grant

B. F.

(2013). The structure and predictive validity of the internalizing disorders. Journal of Abnormal Psychology, 122, 86-92.

30.

Glassmire

D. M.

Stolberg

R. A.

Greene

R. L.

Bongar

(2001). The utility of MMPI-2 suicide items for assessing suicidal potential: Development of a suicidal potential scale. Assessment, 8, 281-290. doi:10.1177/107319110100800304

31.

Glassmire

D. M.

Tarescavage

A. M.

Burchett

Martinez

Gomez

(2015). Clinical utility of the MMPI-2-RF SUI items and scale in a forensic inpatient setting: Association with interview self-report and future suicidal behaviors. Psychological Assessment. Advance online publication. doi:10.1037/pas0000220

32.

Gottfried

Bodell

Carbonell

Joiner

(2014). The clinical utility of the MMPI-2-RF Suicidal/Death Ideation scale. Psychological Assessment, 26, 1205-1211. doi:10.1037/pas0000017

33.

Gradus

J. L.

Qin

Lincoln

A. K.

Miller

Lawler

Sørensen

H. K.

Lash

T. L.

(2012). Sexual victimization and completed suicide among Danish female adults. Violence Against Women, 18, 552-561. doi:10.1177/1077801212453141

34.

Graham

J. R.

(2011). MMPI-2: Assessing personality and psychopathology (5th ed.). New York, NY: Oxford University Press.

35.

Green

B. A.

Handel

R. W.

Archer

R. P.

(2006). External correlates of the MMPI-2 Content Component Scales in mental health inpatients. Assessment, 13, 80-97. doi:10.1177/1073191105284432

36.

Hagan

C. R.

Podlogar

M. C.

Chu

Joiner

T. E.

(2015). Testing the interpersonal theory of suicide: The moderating role of hopelessness. International Journal of Cognitive Therapy, 8, 99-113. doi:10.1521/ijct.2015.8.2.99

37.

Harkness

A. R.

McNulty

J. L.

Finn

J. A.

Reynolds

S. M.

Shields

S. M.

Arbisi

P. A.

(2014). The MMPI-2-RF Personality Psychopathology Five (PSY-5-RF) Scales: Development and validity research. Journal of Personality Assessment, 96, 140-150. doi:10.1080/00223891.2013.823439

38.

Hathaway

S. R.

McKinley

J. C.

(1942). The Minnesota Multiphasic Personality Schedule. Minneapolis: University of Minnesota Press.

39.

Hayes

A. F.

(2013). An introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: Guilford Press.

40.

Hill

R. M.

Rey

Marin

C. E.

Sharp

Green

K. L.

Petitt

J. W.

(2015). Evaluating the Interpersonal Needs Questionnaire: Comparison of the reliability, factor structure, and predictive validity across five version. Suicide and Life-Threatening Behavior, 45, 302-314. doi:10.1111/sltb.12129

41.

Joiner

(2005). Why people die by suicide. Cambridge, MA: Harvard University Press.

42.

Joiner

T. E.

Van Orden

K. A.

Witte

T. K.

Selby

E. A.

Ribeiro

J. D.

Lewis

Rudd

M. D.

(2009). Main predictions of the interpersonal-psychological theory of suicidal behavior: Empirical tests in two sample of young adults. Journal of Abnormal Psychology, 118, 634-646. doi:10.1037/a0016500

43.

Joiner

T. E.

Walker

Rudd

Jobes

(1999). Scientizing and routinizing the assessment of suicidality in outpatient practice. Professional Psychology: Research and Practice, 30, 447-453. doi:10.1037/0735-7028.30.5.447

44.

Kanzler

K. E.

Bryan

C. J.

McGeary

D. D.

Morrow

C. E.

(2012). Suicidal ideation and perceived burdensomeness in patients with chronic pain. Pain Practice, 12, 602-609. doi:10.1111/j.1533-2500.2012.00542.x

45.

Klonsky

E. D.

May

A. M.

(2014). Differentiating suicide attempters from suicide ideators: A critical frontier for suicidology research. Suicide and Life-Threatening Behavior, 44, 1-5.

46.

Klonsky

E. D.

May

A. M.

(2015). The Three-Step Theory (3ST): A new theory of suicide rooted in the “Ideation-to-Action” framework. International Journal of Cognitive Therapy, 8, 114-129.

47.

Kopper

B. A.

Osman

Barrios

F. X.

(2001). Assessment of suicidal ideation in young men and women: The incremental validity of the MMPI-2 Content Scales. Death Studies, 25, 593-607. doi:10.1080/07481180126578

48.

Leonard

(1977). The MMPI as a suicide predictor. Journal of Consulting and Clinical Psychology, 45, 367-377. doi:10.1037/0022-006X.45.3.367

49.

Marty

M. A.

Segal

D. L.

Coolidge

F. L.

Klebe

K. J.

(2012). Analysis of the psychometric properties of the Interpersonal Needs Questionnaire (INQ) among community-dwelling older adults. Journal of Clinical Psychology, 68, 1008-1018. doi:10.1002/jclp.21877

50.

Mosier

C. I.

(1951). Symposium: The need and means of cross-validation, I. Problems and designs of cross-validation. Educational and Psychological Measurement, 11, 5-11.

51.

Muehlenkamp

J. J.

Gutierrez

P. M.

(2007). Risk for suicide attempts among adolescents who engage in non-suicidal self-injury. Archives of Suicide Research, 11, 69-82. doi:10.1080/13811110600992902

52.

Naragon-Gainey

Watson

(2011). The anxiety disorders and suicidal ideation: Accounting for comorbidity via underlying personality traits. Psychological Medicine, 41, 1431-1447.

53.

Nock

M. K.

Borges

Bromet

E. J.

Alonso

Angermeyer

Beautrais

. . . Williams

(2008). Cross-national prevalence and risk factors for suicidal ideation, plans, and attempts. British Journal of Psychiatry, 192, 98-105.

54.

O’Conner

R. C.

(2011). The Integrated Motivational-Volitional Model of Suicidal Behavior. Crisis, 32, 295-298.

55.

Ready

R. E.

Veague

H. B.

(2014). Training in psychological assessment: Current practices of clinical psychology programs. Professional Psychology: Research and Practice, 45, 278-282. doi:10.1037/a0037439

56.

Ribeiro

J. D.

Joiner

T. E.

(2009). The interpersonal-psychological theory of suicidal behavior: Current status and future directions. Journal of Clinical Psychology, 65, 1291-1299. doi:10.1002/jclp.20621

57.

Ribeiro

J. D.

Silva

Joiner

T. E.

(2014). Overarousal interacts with a sense of fearlessness about death to predict suicide risk in a sample of clinical outpatients. Psychiatry Research, 218, 106-112. doi:10.1016/j.psychres.2014.03.036

58.

Ribeiro

J. D.

Witte

T. K.

Van Orden

K. A.

Gordon

K. H.

Bender

T. B.

Joiner

T. E.

(2014). Fearlessness about death: The psychometric properties and construct validity of the revision to the Acquired Capability for Suicide Scale. Psychological Assessment, 26, 115-126. doi:10.1037/a0034858

59.

Selby

E. A.

Anestis

M. D.

Bender

T. W.

Ribeiro

J. D.

Nock

M. K.

Rudd

M. D.

. . . Joiner

T. E.

(2010). Overcoming the fear of lethal injury: Evaluating suicidal behavior in the military through the lens of the Interpersonal–Psychological Theory of Suicide. Clinical Psychology Review, 30, 298-307. doi:10.1016/j.cpr.2009.12.004

60.

Sellbom

Ben-Porath

Y. S.

(2005). Mapping the MMPI-2 Restructured Clinical Scales onto normal personality traits: Evidence of construct validity. Journal of Personality Assessment, 85, 179-187. doi:10.1207/s15327752jpa8502_10

61.

Sellbom

Ben-Porath

Y. S.

Patrick

C. J.

Wygant

D. B.

Gartland

D. M.

Stafford

K. P.

(2012). Development and construct validation of MMPI-2-RF indices of global psychopathy, fearless-dominance, and impulsive-antisociality. Personality Disorders: Theory, Research, and Treatment, 3, 17-38. doi:10.1037/a0023888

62.

Smith

A. R.

Ribeiro

J. D.

Mikolajewski

Taylor

Joiner

T. E.

Iacono

W. G.

(2012). An examination of environmental and genetic contributions to the determinants of suicidal behavior among male twins. Psychiatry Research, 197, 60-65. doi:10.1016/j.cpr.2009.12.004

63.

Sunderland

Slade

(2015). The relationship between internalizing psychopathology and suicidality, treatment seeking, and disability in the Australian population. Journal of Affective Disorders, 171, 6-12. doi:10.1016/j.jad.2014.09.012

64.

Tarescavage

A. M.

Windover

Ben-Porath

Y. S.

Boutacoff

L. I.

Marek

R. J.

Ashton

. . . Heinberg

L. J.

(2013). Use of the MMPI-2-RF Suicidal/Death Ideation and Substance Abuse Scales in screening bariatric surgery candidates. Psychological Assessment, 25, 1384-1389. doi:10.1037/a0034045

65.

Tellegen

Ben-Porath

Y. S.

(2008/2011). Minnesota Multiphasic Personality Inventory–2–Restructured Form: Technical manual. Minneapolis: University of Minnesota Press.

66.

Tellegen

Ben-Porath

Y. S.

Sellbom

(2009). Construct validity of the MMPI-2 Restructured Clinical (RC) Scales: Reply to Rouse, Greene, Butcher, Nichols, and Williams. Journal of Personality Assessment, 91, 211-221. doi:10.1080/00223890902794192

67.

Tellegen

Ben-Porath

Y. S.

Sellbom

Arbisi

P. A.

McNulty

J. L.

Graham

J. R.

(2006). Further evidence on the validity of the MMPI-2 Restructured Clinical (RC) Scales: Addressing questions raised by Rogers, Sewell, Harrison, and Jordan and Nichols. Journal of Personality Assessment, 87, 148-171. doi:10.1207/s15327752jpa8702_04

68.

Van Der Heijden

P. T.

Egger

J. I. M.

Derksen

J. J. L.

(2010). Comparability of scores on the MMPI–2–RF scales generated with the MMPI–2 and MMPI–2–RF booklets. Journal of Personality Assessment, 92, 254-259. doi:10.1080/00223891003670208

69.

Van Orden

K. A.

Cukrowicz

K. C.

Witte

T. K.

Joiner

T. E.

(2012). Thwarted belongingness and perceived burdensomeness: Construct validity and psychometric properties of the Interpersonal Needs Questionnaire. Psychological Assessment, 24, 197-215. doi:10.1037/a0025358

70.

Van Orden

K. A.

Witte

T. K.

Cukrowicz

K. C.

Braithwaite

S. R.

Selby

E. A.

Joiner

T. E.

Jr. (2010). The interpersonal theory of suicide. Psychological Review, 117, 575-600. doi:10.1037/a0018697

71.

Van Orden

K. A.

Witte

T. K.

Gordon

K. H.

Bender

T. W.

Joiner

T. E.

(2008). Suicidal desire and the capability for suicide: Tests of the interpersonal-psychological theory of suicidal behavior among adults. Journal of Consulting and Clinical Psychology, 76, 72-83. doi:10.1037/0022-006X.76.1.72

72.

Watson

Klett

Walters

Vassar

(1984). Suicide and the MMPI: A cross-validation of predictors. Journal of Clinical Psychology, 40, 115-119.

73.

Witte

T. K.

Gordon

K. H.

Smith

P. N.

Van Orden

K. A.

(2012). Stoicism and sensation seeking: Male vulnerability for the acquired capability for suicide. Journal of Research in Personality, 46, 384-392. doi:10.1016/j.jrp.2012

74.

Witte

T. K.

Timmons

K. A.

Fink

Smith

A. R.

Joiner

T. E.

(2009). Do major depressive disorder and dysthymic disorder confer differential risk for suicide? Journal of Affective Disorders, 115, 69-78. doi:10.1016/j.jad.2008.09.003