A Review of Psychometrically Tested Instruments Assessing Suicide Risk in Adults

Abstract

Objective: Identify suicidal ideation and behavior screening instruments with the strongest psychometric properties, using the Interpersonal-Psychological Theory of Suicidal Behavior. Methods: Information databases PsycINFO and PubMed were systematically searched, and articles evaluating the psychometric properties of instruments assessing suicidal ideation and behavior (n = 2,238) were reviewed. International populations and articles with diverse methodologies were integrated. Results: Review of records resulted in the inclusion of 51 articles that assessed 16 instruments. The majority of studies used the English language version (68.6%) and included U.S. populations (65.7%). However, global populations and non-English language versions were also represented. Conclusion: More diverse population representation, and non-English versions of instruments, is required to improve generalizability of assessment measures. Including underrepresented groups and non-English instruments will promote culturally and linguistically sensitive instruments that may better assess suicide risk in diverse populations.

Keywords

psychometrics instrument screening suicidal ideation and suicide

Suicide occurs in all regions of the world, in both high- and low-income countries, making it a major public health concern worldwide (World Health Organization [WHO], 2014). Specifically, every 40 seconds, an individual somewhere in the world dies by suicide, which translates to over 800,000 annual global deaths (WHO, 2014). Suicide represents 50% and 71% of all violent global deaths in men and women, respectively (WHO, 2014). Life expectancy is an important indicator of a population’s health (Braveman, Egerter, & Mockenhaupt, 2011), and individuals who experience suicidal ideation have disproportionately lower life expectancies, compared with nonsuicidal individuals.

Suicide is responsible for these many years of potential life lost (Center for Disease Control and Prevention [CDC], 2013), in part, because it is a multifaceted public health crisis. Suicidal ideation involves a complex interplay between individual, relationship, community, and societal factors (CDC, 2014). These complex factors make predicting suicide risk onerous, and this further potentiates vulnerability. Suicide risk is dynamic, fluctuating over time concurrent with both external events and internal experiences, which can change rapidly (Bryan & Rudd, 2012). Although prediction is multifarious, using a theoretical framework to guide the identification of suicide assessment instruments with the strongest psychometric properties may partially increase the ability to recognize individuals at high risk of suicide.

Theoretical Framework

Joiner’s (2005) Interpersonal-Psychological Theory of Suicidal Behavior (IPTS) was used to guide this review. The theoretical definition of suicide involves desire and ability to enact lethal means. Specifically, the individual will not die by suicide unless she or he has both the desire to die by suicide and the ability to do so. Suicidal desire, according to the theory, emerges when two interpersonal states—perceived burdensomeness (i.e., belief that one’s existence burdens others, and the perception that one’s death is worth more than one’s life to others) and thwarted belongingness (i.e., sense of alienation, disconnection, and social isolation)—are perceived as hopeless and experienced simultaneously. The third component of the IPTS is acquired capability, which refers to habituation to pain and fear, enabling one to more readily engage in self-harm. Specifically, Joiner (2005) posits that in addition to experiencing simultaneous perceived burdensomeness and thwarted belongingness, the capability to initiate suicidal behavior is acquired via exposure to painful and fear-provoking events (e.g., self-injury, previous suicide attempt, physical violence, combat situation) that habituate individuals to the pain and fear associated with death. Repeated exposure to these painful experiences creates the ability for lethal self-injurious behavior, in part by increasing the tolerance for pain and potentiating a fearlessness of death. As a result, there is a three-way interaction between perceived burdensomeness, low belongingness, and the acquired capability for suicide.

Methods

The Whittemore and Knafl (2005) methodology guided the literature search and was used to enhance the rigor of this review. Identification of suicide risk assessment instruments with newly evaluated psychometric properties provided boundaries for the systematic literature search. Data reduction, data comparison, conclusion drawing, and verification permitted thorough interpretation and synthesis of recent psychometric property evidence.

Information databases PsycINFO and PubMed were systematically searched with AND in addition to OR, in combination with the following medical subject heading terms and keywords: suicide, suicidal ideation, self-injurious behavior, self-harm, self-injury, suicide attempted, suicidal behaviors, risk assessment, risk, instrument, scale, questionnaires, psychiatric status rating scales, psychological tests, mental status schedule, screen, mental health, interpersonal psychological theory, interpersonal theory of suicide, psychometric, psychometrics, measure, and validation. The search strategy is further detailed in Figure 1, using the modified PRIMSA flow diagram (Moher, Liberati, Tetzlaff, Altman, & The PRISMA Group, 2009).

Figure 1.

Modified PRISMA Flow Diagram (Moher et al., 2009).

Inclusion criteria for information databases included: peer-reviewed articles, suicidal ideation or suicidal behavior risk instrument, depression screen with a designated suicide item, sample with adults 18 years and older, and psychometric properties (i.e., reliability, validity, sensitivity, specificity, or factor analysis) evaluated within the past 6 years (2010 to 2015). Year limits were enacted to permit identification of current suicide risk screening trends and newly evaluated psychometrics. International articles with country- and language-specific instrument versions, which included sample psychometric properties, were included to promote evaluation of the populations screened by instrument. Additionally, articles with diverse methodologies were included and integrated into the review, as suggested by Whittemore and Knafl (2005).

Exclusion criteria for information databases included the following: articles not written in the English language, reviews and reports, articles published prior to 2010, and articles reiterating established psychometric properties but not reevaluating psychometric properties in new samples. Given our focus on newly evaluated suicide risk instrument psychometric properties in adult populations, adolescent and pediatric samples (i.e., <18 years) and adolescent-specific suicide risk instruments were excluded. Further, because pregnancy-related depression was not a particular focus, pregnancy-specific screening instruments were excluded, in part because the percentage of nonpregnant adults far exceeds the percentage of pregnant women at any given time. Finally, because the review focuses on screening during adulthood, geriatric-specific instruments were also excluded given that they are only applicable to a particular segment of the adult population only (i.e., older adults).

Results

Systematic searches resulted in the inclusion of 51 articles that assessed the psychometric properties of 16 suicide risk assessment instruments. Research participant descriptions, theoretical framework information, sample psychometric properties, and Oxford Center for Evidence-Based Medicine (2011) levels of evidence are detailed in Appendix A and synthesized below.

The Beck Scale for Suicide Ideation (BSSI; Beck & Steer, 1991) is a 21-item self-report measure with five screening items. Endorsement of any suicidal ideation screening items requires the administration of an additional 14 items to further assess the severity of suicidal ideation. The remaining 14 items identify passive ideation, active ideation, desire to die, suicidal plans, access to means, and willingness to share suicidal desire with others. The final two items assess the number of previous attempts and the degree of desire to die during the most recent attempt. Total scores assessing suicidal ideation range from 0 to 38.

During original scale development, in studies of psychiatric inpatients, the BSSI demonstrated high concurrent validity coefficients with the SSI (p < .001; Beck & Steer, 1991), was positively correlated with having made a suicide attempt (p < .001; Pinniti, Steer, Rissmiller, Nelson, & Beck, 2002), demonstrated high internal consistency (coefficient alpha .96 to .97; Beck & Steer, 1991; Pinniti et al., 2002), had 1-week test–retest reliabilities ranging from .54 to .88 (Beck & Steer, 1991; Pinniti et al., 2002), and total item-total correlations were significant beyond the .001 level (Pinniti et al., 2002).

In recent evaluations, in a sample of U.S. male psychiatric inpatients, convergent validities with the Reasons for Attempting Suicide Questionnaire (RASQ) internal subscale and Adult Suicide Ideation Questionnaire (ASIQ; p < .01; Horon, McManus, Schmollinger, Barr, & Jimenez, 2013) were established. Further, convergent validities were established with the Beck Hopelessness Scale (BHS) in primarily female Asian American young adult volunteers (p < .01; Miranda, Gallagher, Bauchner, Vaysman, & Marroquin, 2012) and incarcerated U.S. psychiatric inpatients (p < .01; Horon et al., 2013). Finally, a Chinese version of the BSSI demonstrated convergent validity with the total Three-Dimensional Psychological Pain Scale, Psychache Scale, and Beck Depression Inventory (BDI), in majority female Chinese outpatients with mood or depressive disorders (p < .01; Li et al., 2014).

Internal consistencies (Cronbach’s alpha) in U.S. samples ranged from .85 to .98, in studies that included incarcerated U.S. males (.85, Horon et al., 2013; .94, Smith, Wolford, Mandracchia, & Jahn, 2013), African American mothers with a history of suicide attempt (.91; Woods, Zimmerman, Carlin, Hill, & Kaslow, 2013), primarily Asian American women undergraduates (.96, Miranda, Valderrama, Tsypes, Gadol, & Gallagher, 2013; .98, Polanco-Roman & Miranda, 2013), and primarily female Asian American young adult volunteers (.95; Miranda et al., 2012). In two of these samples, which contained majority Asian American young women, follow-up internal consistencies (Cronbach’s alpha) ranged from .97 (Miranda et al., 2012) to .98 (Miranda et al., 2013).

In majority female Chinese samples, the internal consistency (Cronbach’s alpha) of the Chinese version of current ideation was .90 (Li et al., 2014) and .91 (Xie et al., 2014). In these Chinese samples, worst ideation internal consistencies (Cronbach’s alpha) were .94 (Xie et al., 2014) and .95 (Li et al., 2014). Further, Cronbach’s alpha of the Chinese version, in a sample of primarily female Taiwanese adults with obsessive-compulsive disorder (OCD), was .85 (Tzu-Chi et al., 2010).

Additionally, in a study including a majority of Pakistani women hospitalized after a suicide attempt, internal consistency (Cronbach’s alpha) of the Urdu translated version was .89 (Husain et al., 2014). Finally, the Korean version of the instrument, in a sample of majority South Korean women at high-risk for suicide, produced a Cronbach’s alpha of .92 (Kim, Ha, Yu, Park, & Ryu, 2014), and in a majority male Korean epileptic sample, Cronbach’s alpha was .82 (Lim et al., 2010).

The Suicidal Behaviors Questionnaire—Revised (SBQ-R; Osman et al., 2001) is a 4-item self-report measure of four suicidal constructs, each measuring a dimension of suicidal ideation. Item 1 evaluates lifetime ideation and attempt, Item 2 assesses frequency of ideation in the past 12 months, Item 3 explores suicide threats, and Item 4 evaluates the likelihood of future suicidal behavior.

In a study comparing psychiatric inpatients and nonclinical undergraduate students, during original scale development, the SBQ-R differentiated between suicidal and nonsuicidal groups (p < .001). A cutoff score of 2 in both clinical and nonclinical samples was most useful in identifying individuals with established suicide status, correctly identifying individuals as positive for suicide ideation or attempts (sensitivity: .80–1.0), and individuals identified as nonsuicidal were correctly identified as nonsuicide ideators or nonattempters (specificity: .96–1.0; Osman et al., 2001).

Recent sample internal consistencies (Cronbach’s alpha) in U.S. adults ranged from .79 to .90 in studies that included majority male deployed military personnel (.79; Bryan, Hernandez, Sybil, & Clemans, 2013) and university students and adults (.90; O’Riley & Fiske, 2012). In a sample of primarily female United Kingdom individuals with a history of traumatic event exposure, Cronbach’s alpha of the English version was .87 (Panagioti, Gooding, & Nicholas, 2012). The instrument was also translated to German in another study, which included primarily female German adults, resulting in a Cronbach’s alpha of .76 (Wagner, Klinitzke, Brahler, & Kersting, 2013).

The Interpersonal Needs Questionnaire (INQ; Van Orden, Witte, Gordon, Bender, & Joiner, 2008) is a 10-, 12-, 15-, 18-, or 25-item self-report measure, with subscales measuring the constructs of both thwarted belongingness and perceived burdensomeness. Using a 7-point Likert scale, individuals consider recent events and indicate the degree to which each item is true for them. The scores are coded, and some items are reversed scored, with higher numbers reflecting greater levels of thwarted belongingness and perceived burdensomeness. During scale development, in a sample containing a majority of undergraduate students, INQ-12 internal consistency (Cronbach’s alpha) belongingness subscale scores were .85 and burdensomeness subscale scores were .89 (Van Orden et al., 2008).

With regard to factor structure, in a study examining clinical and nonclinical young adults, the INQ-10, −12, and −15 demonstrated better fit than the INQ-18 and −25 during confirmatory factor analysis (p < .001; Hill et al., 2014). Additionally, factor analysis confirmed that the INQ consists of two distinct latent factors associated with burdensomeness and belongingness, in deployed U.S. military personnel samples (p < .001; Bryan, 2011), U.S. undergraduate students (p < .001; Freedenthal, Lamis, Osman, Kahlo, & Gutierrez, 2011), and in U.S. samples of younger and older adults (p < .001; Van Orden, Cukrowicz, Witte, & Joiner, 2012). Results indicated 10 (p < .001; Bryan, 2011), 12 (p < .001; Freedenthal et al., 2011), and 15 (p < .001; Van Orden et al., 2012) items provided reliable and acceptable fit.

In an American undergraduate sample, INQ-12 perceived burdensomeness subscale scores were positively correlated with the BDI-II (p < . 01), BHS (p < .01), Modified Scale for Suicide Ideation (p < .01), Life Attitudes Schedule-Short Form (p < .01), and the Acquired Capability for Suicide Scale (ACSS; p < .05), and negatively correlated with the Multidimensional Scale of Perceived Social Support (p < .01), and Reasons for Living Inventory for Young Adults (RFL-YA; p < .01; Freedenthal et al., 2011). Similarly, in this sample, INQ-12 thwarted belongingness subscale scores were positively correlated with the BDI-II, BHS, Modified Scale for Suicide Ideation, Life Attitudes Schedule-Short Form, and negatively correlated with Multidimensional Scale of Perceived Social Support and RFL-YA (p < .01; Freedenthal et al., 2011).

Higher sample BSSI scores, in a U.S. multisample study including undergraduates, young adults, and older adults, were also associated with greater thwarted belongingness and perceived burdensomeness subscale scores (p < .01; Van Orden et al., 2012). In this sample, the INQ also demonstrated predictive validity, as higher summed thwarted belongingness and perceived burdensomeness subscale scores were both associated with higher BSSI scores 1 month later (Belongingness, p < .05; Burdensomeness, p < .01; Van Orden et al., 2012).

In majority male deployed U.S. military samples, INQ-10 internal consistency (Cronbach’s alpha) of belongingness was .86 and burdensomeness was .81 (Bryan, 2011; Bryan et al., 2013). In a primarily female U.S. undergraduate sample, INQ-12 Cronbach’s alpha of belongingness was .92 and burdensomeness was .93 (Freedenthal et al., 2011). In a sample containing American Indian or Alaska Natives, INQ-18 Cronbach’s alpha of belongingness was .90 and burdensomeness was .90 (O’Keefe & Wingate, 2013). Finally, in a sample including U.S. undergraduates, young adult outpatients, and older adults, INQ-25 Cronbach’s alpha of belongingness was .85 and burdensomeness was .89 (Van Orden et al., 2012).

The ASIQ (Reynolds, 1991) is a 25-item self-report measure rated on 7-point item response scale. The instrument assesses frequency of suicidal thoughts, desire to die, suicidal plans, and suicidal behaviors occurring in the previous month. The potential range of total scores is 0 to 150. Higher scores indicate numerous suicidal cognitions occurring with regularity. During original scale development, in a study examining undergraduate students, the ASIQ had high reported reliability (coefficient alpha: .97), test–retest reliability (.86), contrasted groups validity (p < .001), and significant correlations (all p’s < .001) between the ASIQ and depression, hopelessness, anxiety, self-esteem, and history of prior suicide attempts (Reynolds, 1991).

In a recent sample including U.S. male incarcerated psychiatric patients, convergent validities with the BHS, BSSI, and RASQ Internal subscale were demonstrated (p < .01; Horon et al., 2013). In this study, which included many men from the lowest socioeconomic statuses, sample internal consistencies (Cronbach’s alpha) ranged from .85 (i.e., cutoff score of 31) to .95 (i.e., no cutoff score; Horon et al., 2013).

The Suicidal Ideation Scale (SIS; Rudd, 1989) is a 10-item self-report measure, with each item rated on a 5-point Likert scale ranging from 1 to 5. The instrument measures a continuum of suicidal thoughts, from ideation to attempts. The SIS has the ability to discriminate between those who have and have not attempted suicide. During scale development, in a study including undergraduate students, internal consistency (coefficient alpha) was .86 and interrater reliability was .95 (Rudd, 1989).

Recent sample convergent validity between the SIS and the Cultural Assessment of Risk for Suicide total and subscale scores was established (p < .001; Chu et al., 2013) in a study including majority U.S. women, with inclusion of homosexual, bisexual, and transgender, populations. Further, in a majority male U.S. clinical military sample, construct validity was established with the Behavior and Symptom Identification Scale-24 (p < .001; Luxton, Rudd, Reger, & Gahm, 2011). In these studies, internal consistency (Cronbach’s alpha) was .91 (Luxton et al., 2011) and .94 (Chu et al., 2013).

The Scale for Suicide Ideation (SSI-C; Beck, Kovacs, & Weissman, 1979) is a clinician-administered, 21-item scale with each item consisting of three alternate statements based on intensity from 0 to 2. The first 19 items are scored, resulting in a total score that ranges from 0 to 38. The instrument quantifies the intensity of current conscious suicidal ideation by scaling various dimensions of self-destructive thoughts or wishes. Items assess the extent of suicidal thoughts, the extent of the wish to die, the desire to make an actual suicide attempt, details of plans (if any), internal deterrents, and subjective feelings of control regarding proposed attempt. During scale development, in a study including inpatient psychiatric populations, internal consistency (Cronbach’s alpha) was .89, interrater reliability was .83, 16 of the 19 coefficients were significant (p < .01), and concurrent validity was high with clinical evaluations as well as with the BDI (p < .001; Beck et al., 1979).

An expansion of the SSI, which includes the SSI-W (worst ideation), uses the same format and scoring as the SSI-C, but focuses specifically on suicidal ideation at the worst point in one’s life (Beck, Brown, & Steer, 1997). During SSI-W scale development, in a study of psychiatric outpatients, internal consistency (Cronbach’s alpha) was .89 and the scale was associated with a history of suicide attempt (p < .001; Beck et al., 1997).

In recent studies, English version interrater reliability was .89, in samples including majority male Italian adults with panic disorder (De Baradis et al., 2013) and obsessive-compulsive disorder (De Baradis et al., 2014). Further, in a sample including Finnish adults with major depressive disorder (MDD), the English version produced cutoff score dependent sensitivities ranging from .76 to .81 and cutoff score dependent specificities ranging from .68 to .92 (Vuorilehto et al., 2014). Internal consistency (Cronbach’s alpha) of the Korean version, in a sample of South Korean psychiatric in-patients, was .95 (Jon, Lee, & Park, 2013).

The Suicide Probability Score (SPS; Cull & Gill, 1982) is a 36-item self-report measure, with each item being rated on a 4-point Likert scale. Individuals report frequencies of subjective experiences and past behaviors (none or little to most or all of the time), which aids in the assessment of suicide risk. The items are then weighed from 0 to 5 and totaled to determine a total weighted score, normalized t-score, suicide probability scores, and four subscale scores (i.e., 12 hopelessness items, 9 negative self-evaluation items, 8 suicide ideation items, and 7 hostility items). In an original standardization study comparing nonclinical adolescents, nonclinical adults, psychiatric inpatients, and individuals with a history of suicide attempt, internal consistency (coefficient alpha) for the total scale was .93 and subscale scores ranged from .62 to .89 (.62 for negative self-evaluation, .78 for hostility, .80 for hopelessness, and .89 for suicide ideation; Cull & Gill, 1982). During scale development, split-half reliability for the total scale was .93 (Cull & Gill, 1982).

Internal consistency (Cronbach’s alpha) of the English version, in a sample study examining a majority of hospitalized Scottish females after a recent suicide attempt, was .86 (O’Connor, Smyth, Ferguson, Ryan, & Williams, 2013). Another study used the English version for 15% of participants, and the French version for the remaining 85%, in a sample containing an all-male Canadian incarcerated population (Naud & Daigle, 2010). In this male-incarcerated sample, a receiver operating characteristic (ROC) analysis showed the area under the curve for detecting hopelessness was .63, suicidal ideation was .66, negative self-evaluation was .64, and hostility was .64 (p < .001; Naud & Daigle, 2010). The sensitivity of the SPS in this incarcerated Canadian sample was .36 and the specificity was .85 (Naud & Daigle, 2010).

The ACSS (Van Orden et al., 2008) is a 20-item self-report instrument, with scores rated on a 5-point Likert scale ranging from 0 to 4. Total scores can range from 0 to 80. The instrument assesses respondents’ fearlessness about death and acquired capability for suicide. Higher scores indicate less fear of death, greater pain tolerance, and exposure to painful and provocative events (creating acquired capability).

Factor analysis, in a study of U.S. undergraduate students, demonstrated that the strongest factor loading was associated with item 19 (i.e., I am not at all afraid to die, p = .03), but all items were reasonable indicators of each factor (Ribeiro et al., 2014). In studies examining undergraduates and individuals with a history of suicide attempt, there was a strong correlation between total scores and perceived courage to make a suicide attempt (p < .001; Ribeiro et al., 2014), positive associations with exposure to painful and provocative events (p < .001; Bender, Gordon, Bresin, & Joiner, 2011; Van Orden et al., 2008), and a strong negative correlation with fear of suicide (p < .001; Ribeiro et al., 2014).

Additional factor analysis, in a U.S. incarcerated male population, indicated that a four-factor model provided the best statistical and conceptual fit; three of the four factors were interpretable (i.e., general fearlessness and perceived pain tolerance, fearlessness of death, and spectator enjoyment of violence; p < .001; Smith et al., 2013). Sample internal consistencies (Cronbach’s alpha) ranged from .69 to .84 in studies including majority male U.S. deployed military personnel (.69; Bryan et al., 2013), primarily female American Indian or Alaska natives (.84; O’Keefe & Wingate, 2013), and German men exposed to violent video gaming (.84; Teismann, Fortsch, Baumgart, Het, & Michalak, 2014).

The Hopelessness Depression Symptom Questionnaire—Suicidality Subscale (HDSQ-SS; Joiner, Pfaff, & Acres, 2002) is a 4-item self-report measure (subscale of the HDSQ; Metalsky & Joiner, 1997). Items are rated on 4-point Likert scale ranging from 0 to 3, based on the severity and frequency of suicidal ideation in the past 2 weeks. Total scores range from 0 to 12. The instrument also assesses the number of past suicide attempts. During original scale development, in a study examining nonclinical adolescents and young adults, internal consistency (coefficient alpha) was .90, correlations between suicidal ideation and depression were high (p < .001), and interitem correlations were high with all items loading strongly (p < .001; Joiner et al., 2002).

In a more recent study consisting of majority female American Indian or Alaska natives from 27 different tribes, the sample internal consistency (Cronbach’s alpha) was .88 (O’Keefe & Wingate, 2013).

The BDI-II (Beck, Steer, Ball, & Ranieri, 1996) is a 21-item self-report inventory, with symptoms rated on a 4-point scale ranging from 0 to 3. Item 9 assesses suicidal ideation. Total scores range from 0 to 63. The instrument measures the severity of depression within the past 2 weeks. Four new symptoms (agitation, concentration difficulty, worthlessness, and loss of energy) were added to the BDI-II, and four symptoms (weight loss, body image change, work difficulty, and somatic preoccupation symptoms) were removed from the original BDI (Beck & Steer, 1984).

During scale development, the BDI-II had high internal consistency (coefficient alpha .91) and moderate to strong convergent validities with other self-report and clinical rating scales of depression in studies including psychiatric, nonclinical young adult, and undergraduate populations (p < .001; Ball & Steer, 2003; Beck et al., 1996). While validating the scale in a sample of clinically depressed outpatients, two factors representing Somatic-Affective and Cognitive dimensions were found (p < .05), and confirmatory factor analysis supported a model in which the BDI-II reflected one underlying second-order dimension composed for two first-order factors representing cognitive and noncognitive symptoms (p < .001; Steer, Ball, Ranieri, & Beck, 1999).

In a sample ROC analysis, including U.S. post-Myocardial Infarction patients, the area under the curve for diagnosing MDD was .962 (i.e., good discrimination between those with and without depression; p < .001; Huffman et al., 2010). Further, sample convergent validity with the BHS was established in a sample including primarily female U.S. young adult volunteers (p < .01; Miranda et al., 2012). Also, convergent validity with the Patient Health Questionnaire (PHQ)-9 was demonstrated in U.S. heart failure patients (p < .01; Hammash et al., 2012) and depressed Australian adults (p < .001; Titov et al., 2011). In a sample of primarily female Norwegian adults, using a cutoff score of 12, sensitivity was .85 and specificity was .88. (Kjaergarrd, Elisabeth, Wang, Waterloo, & Jorde, 2014). With a cutoff score of 14, however, in a sample of primarily male U.S. adults, sensitivity was .88 and specificity was .84 (Huffman et al., 2010. In this U.S. male sample, using a cutoff score of 16, sensitivity was .88 and specificity was .92 (Huffman et al., 2010).

In a sample study examining majority Norwegian women who experienced first stroke, person-separation reliability (i.e., ability of the scale to distinguish at least three distinct groups of depression) was 1.99 (Lerdal, Kottorp, Gay, Grov, & Lee, 2014). Internal consistency (Cronbach’s alpha) ranged from .89 to 90 in studies examining U.S. heart failure patients (.89; Hammash et al., 2012), U.S. young adult volunteers (.90; Miranda et al., 2012), primarily female Norwegian adults (.89; Kjaergarrd et al., 2014), Norwegian adults with a history of stroke (.90; Lerdal et al., 2014), and depressed Australian adults (.90; Titov et al., 2011).

The Arabic version of the instrument, in a sample including primarily female college students from Kuwait, demonstrated convergent validity with the Hopkins Symptoms Checklist-25 (p < .001; Al-Turkait & Ohaeri, 2010). The Arabic version, in this college sample from Kuwait, produced an internal consistency (Cronbach’s alpha) of .83 (Al-Turkait & Ohaeri, 2010). Finally, the Chinese version of the instrument, in a sample of Taiwanese outpatients with OCD, had a Cronbach’s alpha of .93 (Tzu-Chi et al., 2010).

The BDI—Fast Screen (BDI-FS; Scheinthal, Steer, Giffin, & Beck, 2001) contains seven items that assess the psychological symptoms of depression, all of which were drawn from the 21-item BDI-II. It is a self-report inventory and each item contains a 4-point rating scale ranging from 0 to 3; thus, total scores can range from 0 to 21. During scale development, internal consistency (Cronbach’s alpha) was .88 for medical adult outpatients being treated in family practice (Beck, Steer, Ball, Ciervo, & Kabat, 1997) and was .86 for medical adult outpatients being treated in internal medicine (Steer, Cavalieri, Leonard, & Beck, 1999).

The German version of the instrument, in a study examining majority women medical patients from the German general population, sample internal consistency (Cronbach’s alpha) was .84 (Kliem, Moble, Zenger, & Brahler, 2014). Further, in this study, the BDI-FS was positively correlated with the PHQ-9 (p < .001; Kliem et al., 2014).

The PHQ-9 (Kroenke, Spitzer, & Williams, 2001) is a 9-item self-report measure with each question rated on a scale from 0 to 3. Total scores can range from 0 to 27, with Item 9 inquiring about thoughts of self-harm. A major depression diagnosis is present if five of the nine depressive symptom criteria (depressed mood or anhedonia) have been present more than half the days in the past 2 weeks.

During original scale development in primary care and obstetrics-gynecology studies, there was a strong association between increasing PHQ-9 scores and worsening functional status, disability days, and symptom-related difficulty (p < .05; Kroenke et al., 2001). The PHQ-9 was also positively correlated with the Mental Health Inventory (p < .05; Kroenke et al., 2001). Internal consistency (Cronbach’s alpha) was .89 and .86 in primary care and obstetrics-gynecology studies, respectively, and 48-hour test–retest reliability was .84 (Kroenke et al., 2001).

In recent studies, PHQ-9 scores were strongly correlated with BDI-II scores, in samples including U.S. older adult heart failure patients (p < .01; Hammash et al., 2012) and depressed Australian adults (p < .001; Titov et al., 2011). There was also convergent validity with the BSSI in a sample of primarily female U.S. college students (p < .01; Polanco-Roman & Miranda, 2013).

Sensitivity for different cutoff scores ranged from .54 to .92, in studies including U.S. cardiac and stroke patients (.54; Razykov, Zieglestein, Whooley, & Thombs, 2012), depressed U.S. adults (.69; Uebelacker, German, Baudiano, & Miller, 2011), U.S. heart failure patients (.70; Hammash et al., 2012), U.S. older adults (.88; Phelan et al., 2010), and U.S. epilepsy patients (.92, Rathore et al., 2014). In these sample studies, specificity for varying cutoff scores ranged from .74 to .92 (.74, Rathore et al., 2014; .80, Phelan et al., 2010; .84, Uebelacker et al., 2011; .90 Razykov et al., 2012; .92, Hammash et al., 2012). Finally, in a sample including depressed U.S. adults, the sensitivity of suicide Item 9 was .69 and the specificity of Item 9 was .84 (Uebelacker et al., 2011).

Internal consistency (Cronbach’s alpha) ranged from .74 to .85 in samples including depressed Australian adults (.74; Titov et al., 2011), U.S. college students (.82, Polanco-Roman & Miranda, 2013; .83, Miranda et al., 2013), Hispanic American women (.84; Merz, Malcarne, Roesch, Riley, & Sadler, 2011), and U.S. heart failure patients (.85; Hammash et al., 2012). Follow-up internal consistency (Cronbach’s alpha) was .79 (Miranda et al., 2013) and .83 (Polanco-Roman & Miranda, 2013) in U.S. college student samples, and was .81 in a sample of depressed Australian adults (Titov et al., 2011). Interrater reliability was .81 in an international sample including advanced cancer patients (Lie et al., 2015).

While evaluating the Iranian and Dutch versions of the instrument, in a ROC analysis, the area under the curve for diagnosing MDD was .83 in the Iranian version (p < .001; Khamseh et al., 2011) and was .87 in the Dutch version (p < .001; Zuithoff et al., 2010). The Dutch version produced a sensitivity and specificity of .82 (Zuithoff et al., 2010). In contrast, the Iranian version produced a sensitivity of .73 and specificity of .76 (Khamseh et al., 2011). Internal consistency (Cronbach’s alpha) of the Dutch version was .88 (Zuithoff et al., 2010). Cronbach’s alpha of the Iranian version was .87 (Khamseh et al., 2011).

The Spanish version of the instrument, in a sample of Peruvian women, had an internal consistency (Cronbach’s alpha) of .81 (Zhong et al., 2014). Further, the Spanish version, in a sample of Spanish speaking U.S. women who emigrated from Mexico, had a Cronbach’s alpha of .85 (Merz et al., 2011).

Finally, the sensitivity and specificity of suicide Item 9 in the Japanese version of the instrument was .70 and .97, respectively (Inagaki et al., 2013). The sensitivity and specificity of detecting MDD in the Japanese version with cutoff points of 4/5 were .86 and .85, respectively (Inagaki et al., 2013).

The Telephone-Linked Communication Patient Health Questionnaire – 9 (TLC-PHQ-9; Farzanfar et al., 2014; Friedman, Stollerman, Mahoney, & Roznblyum, 1997) is an automated telephone-based system that speaks to the patient using digitized human speech and administers the 9-item instrument. Each question is rated on a scale from 0 to 3, with total scores ranging from 0 to 27. Item 9 inquires about thoughts of self-harm. If thoughts of self-harm are endorsed, a special alert immediately notifies the clinician. In contrast, if self-harm is not endorsed, patient responses are stored in a database and clinicians review reports at periodic intervals.

Internal consistency (Cronbach’s alpha) was .92 in a sample consisting of majority U.S. women with varying degrees of depression (Farzanfar et al., 2014) and was .84 in a study including majority male U.S. National Guard soldiers (Fine et al., 2013). Test–retest reliability (weighted Kappa) in the depressed U.S. sample was .76 (Farzanfar et al., 2014). Further, in the depressed U.S. sample, sensitivity was .82 and specificity was .90 (Farzanfar et al., 2014). In the U.S. National Guard sample, a cutoff score of 10 produced the optimal balance of sensitivity (.56) and specificity (.86; Fine et al., 2013).

The Sheehan Suicidality Tracking Scale (S-STS; Coric, Stock, Pultz, Marcus, & Sheehan, 2009), adapted from the Suicidality Module of the Mini International Neuropsychiatric Interview Structured Diagnostic Interview for DSM-IV (Sheehan et al., 1998), is an 8-item self-report or clinician administered rating scale. Each item in the scale is scored on a 5-point Likert scale ranging from 0 to 4. The instrument tracks spontaneous and treatment-emergent suicidal ideation and behaviors (self-injury, self-harm, ideation, and attempt). Data from the S-STS can be analyzed as individual item scores, suicidal ideation subscale score, suicidal behavior subscale score, and total score.

During scale development, in a randomized trial including female outpatients with generalized anxiety disorder, the sensitivity of the S-STS in prospectively identifying subjects with suicidal thoughts or behaviors was 100%, as compared with the Hamilton Rating Scale for Depression suicide Item 3, which had a sensitivity of 63% (Coric et al., 2009).

Criterion validity was reinforced in a more recent sample including majority Italian female undergraduate students, as individuals endorsing suicide ideation had higher S-STS global, suicide ideation subscale, and suicidal behavior subscale scores (p < .001; Preti et al., 2013). Further, in this sample, convergent validity with the General Health Questionnaire, and discriminative validity with the Rosenberg Self-Esteem Scale and Modified Social Support Survey was established (p < .001; Preti et al., 2013). Finally, in this sample, internal consistencies (Guttman’s lambda 2 for global scores, suicide ideation scores, and suicide behavior scores) ranged from .83 to .88 and test–retest consistency scores ranged from .46 to .88 (Preti et al., 2013).

The Self-Injury Implicit Association Test (SI-IAT; Nock & Banaji, 2007a; Nock & Banaji, 2007b) is an approximately 5-minute computer-based task that examines implicit thoughts without having to rely on self-report. It is a performance-based measure that assesses strengths of automatic associations (persons categorize stimuli into one of two groups). Participants independently classify self-injury related stimuli (e.g., pictures of skin that has been cut) or neutral stimuli (e.g., pictures of noninjured skin) as quickly as possible. Faster sorting during the test indicates stronger implicit links in those constructs for the subject. Suicide, death, and self-injury implicit associations can predict future self-harm events.

During original instrument development, in studies including adolescents (nonsuicidal controls, individuals with suicidal ideation, and individuals with a recent suicide attempt) and psychiatrically distressed individuals, there were large differences on the SI-IAT between nonsuicidal persons and suicide ideators (p < .001; Nock & Banaji, 2007b), and suicide attempters (p < .001; Nock & Banaji, 2007b; Nock et al., 2010), as well as between suicide ideators and suicide attempters (p = .009; Nock & Banaji, 2007b). Further, during instrument validation testing, nonsuicidal adolescents had a negative association between self-injury and oneself, suicide ideators showed a small positive association between self-injury and oneself, and suicide attempters had a large positive association between self-injury and oneself (p < .05; Nock & Banaji, 2007b). In studies also including adult U.S. psychiatric inpatient individuals, prediction of suicide ideation and suicide attempt was accurate despite age, mood and substance disorders, hopelessness, and total number of psychiatric disorders (p < .001, Ellis, Rufino, & Green, 2016; p < .001, Nock & Banaji, 2007b; p < .05, Nock et al., 2010).

Additionally, in this study of U.S. psychiatric inpatients, IAT scores were positively correlated with BSSI, BHS, and PHQ-9 scores (p < .01; Ellis et al., 2016). Moreover, in a sample including Canadian adults with suicidal ideation or recent self-harm, the Death or Life IAT significantly predicted self-harm (p = .02; Randall, Rowe, Dong, Nock, & Colman, 2013). In this Canadian sample, with a high cutoff, the Death or Life IAT sensitivity was 96.6% and specificity was 53.9%; with a low cutoff, the Death or Life IAT sensitivity was 58.6% and specificity was 96.2% (Randall et al., 2013).

The Reasons for Living Inventory (RFL; Linehan, Goodstein, Nielsen, & Chiles, 1983) is a 48-item self-report measure. Respondents indicate, on a 6-point Likert-type scale, the importance of each reason for not attempting suicide. Higher scores indicate stronger reasons for living. The instrument assesses a range of beliefs thought to be important in differentiating suicidal from nonsuicidal individuals.

The original instrument had 72-items; however, in a validation study including both clinical and nonclinical undergraduates and adults, principal-component factor analysis was applied and total items were reduced to 48 (Linehan et al., 1983). Subsequent factor analyses in nonclinical and psychiatric inpatient participants indicated that there were six primary reasons for living, encompassing the six RFL subscales (i.e., Survival and Coping Beliefs, Responsibility to Family, Child-Related Concerns, Fear of Suicide, Fear of Social Disapproval, and Moral Objections; Linehan et al., 1983). In this validation study, the RFL differentiated suicidal from nonsuicidal individuals (p < .001); specifically, in nonclinical individuals, the Fear of Suicide further differentiated between previous ideators and previous suicide attempters (p < .001; Linehan et al., 1983). Alternatively, in clinical individuals, Child-Related Concerns differentiated between current suicide ideators and current suicide attempters (p < .001; Linehan et al., 1983). However, in both clinical and nonclinical populations, Survival and Coping, Responsibility to Family, and Child-Related Concerns subscales were most useful in differentiating suicidal and nonsuicidal groups (Linehan et al., 1983).

In recent studies, internal consistency (Cronbach’s alpha) was .94 in a sample including majority female U.S. older adults (Segal, Marty, Meyer, & Coolidge, 2012) and was .96 in a sample containing African American mothers with a history of suicide attempt (Woods et al., 2013). In the study including U.S. older adults, Cronbach’s alpha for survival and coping beliefs was .94, responsibility to family was .86, child-related concerns was .78, fear of suicide was .78, fear of social disapproval was .81, and moral objections was .82 (Segal et al., 2012).

The English version was also translated to Malay in one study, and factor analysis confirmed six primary reasons for living (p < .001; Aishvarya et al., 2014). The Malaysian version was positively correlated with the Positive And Negative Suicide Ideation Inventory (PANSI-Positive), Rosenberg Self-Esteem Scale (RSE), Adult Trait Hope Scale (ATH), Provision of Social Relations (PSR), and Satisfaction with Life (SWL) (p < .001; Aishvarya et al., 2014). Further, in this study, the Malaysian version was negatively correlated with the Depression Anxiety Stress Scale (DASS), BHS, and PANSI-Negative (p < .001; Aishvarya et al., 2014). Internal consistency (Cronbach’s alpha) was .94 (Aishvarya et al., 2014).

Discussion

Approximately 90% of unplanned suicide attempts and 60% of planned first attempts occur within 1 year of the onset of suicidal ideation (American Psychiatric Association[APA], 2010). Thus, identifying psychometrically tested suicide risk assessment instruments with the strongest psychometric properties are paramount in recognizing individuals at high risk of suicide. There is not a universal set of strategies for suicidal ideation detection; however, the WHO (2014) recommends assessment of emotional distress, early identification of mental disorders and alcohol misuse, and reduction in access to the most prevalent means. In contrast, U.S. National Guideline Clearinghouse recommendations (NGC, 2014) state that suicide risk assessment involves a clinical interview with subsequent administration of Beck’s Hopelessness, Suicidal Ideation and Suicide Intent scales, BDI, and the Hamilton Rating Scale for Depression.

U.S. suicide risk assessment recommendations are based on the following levels of evidence: C (i.e., studies rated as 2+ , case control or cohort studies), D (i.e., evidence level 3 or 4), Q (i.e., qualitative studies with appropriate quality), and Good Clinical Practice (i.e., based on clinical experience; NGC, 2014). Sample articles support a portion of these national guidelines, as the SIS, SSI, and BDI-II had the first, second, and fourth highest internal consistencies, respectively, and administration of these instruments is equally feasible. Moreover, guideline levels of evidence and sample levels are similar, as 7.8% and 92.2% of included studies represent levels of Evidences 2 and 3, respectively.

Although there are conflicting theories regarding suicide, and several models were identified in this review, analysis of the sample population partially supports the IPTS. In studies that contained a majority of male participants (n = 11), over 36% of men were in the military and more than 27% were incarcerated, and it is likely these groups of men in particular had experienced or perpetrated traumatic events. Repeated provocative exposures, such as traumatic events, may create less fear of pain, injury, and death. Additionally, provocative exposures may potentiate feelings of low belongingness, particularly if the individual is removed from a familiar environment and placed in a combat or prison environment with others experiencing simultaneous stress.

Moreover, several sample participants indicated family members had attempted suicide or died by suicide. These psychologically painful experiences may also accelerate feelings of social isolation and decrease fear of death. Finally, over 30% of all participants sampled had made a previous suicide attempt, which is likely a conservative estimate because not all studies inquired about previous suicide attempts. Repeated self-injury supports gradual habituation toward increasingly lethal self-harm. This group of individuals, if expressing the desire to die by suicide, as endorsed by positive suicidal ideation on screening instruments, may increasingly develop the ability to die by suicide.

Although several countries have established suicide risk assessment screening instrument recommendations, current suicide risk assessment tools do not contain guidelines for mental health clinicians on how to tailor risk assessment for diverse patient populations, including sexual orientation, race or ethnicity, and religious diversity, which is problematic because willingness to report suicidal behavior varies by age, sex, race or ethnicity, and religion (American Psychological Association, 2012; WHO, 2014). The IPTS suggests that individuals gradually become more vulnerable, and insensitive screening practices may precipitate feelings of isolation and being misunderstood by others.

In the current sample, African American, Asian American, American Indian, Hispanic American, Latino(a) American, Pacific Islanders, Mestizos, Peruvian, Malay, Chinese, Korean, Japanese, Taiwanese, Pakistani, Australian, German, United Kingdom, Canadian, Norwegian, Dutch, Italian, Austrian, Finnish, Scottish, Arab, Iranian, and Indian populations were represented. Additionally, in one sample, transgender, homosexual, and bisexual populations were represented. Inclusion of diverse populations permitted preliminary evaluations of psychometric properties in these important populations.

However, although there was inclusion of diverse populations and non-English language versions (i.e., Chinese, Japanese, Korean, Urdu, Arabic, Iranian, Malay, Spanish, French, Dutch, and German), these diverse populations and language versions remain underrepresented. Further, some studies utilized validated non-English versions, whereas other studies used a translator to administer a translated version, and additional evaluations will more firmly establish psychometrics in these non-English versions.

The majority (68.6%) of studies used the English language version (n = 35). Of the 35 studies using the English version, 65.7% included U.S. populations (n = 23). This results in questionable usefulness and generalizability in clinical practice because many of the instruments are tested on nonrepresentative samples and have not been adequately tested in important subpopulations (APA, 2010). Generalizablity may further be limited if the English version of an instrument is applied in a country where several languages are spoken, and where cultural and religious perspectives are diverse. Thus, it is important to additionally test non-English versions of the instrument in studies that include more diverse populations, to validate psychometrics, and improve generalizability.

Comprehensive search strategies were employed in this review; however, the included suicide risk assessment instruments do not represent an exhaustive list. Although this work adds to the psychometric properties outlined in Brown’s (2001) review, includes additional screening instruments (i.e., INQ, ACSS, HDSQ-SS, BDI-FS, PHQ-9, TLC-PHQ-9, S-STS and Implicit Association Test), includes international populations, and integrates studies with varying methodologies, Brown provides a comprehensive summary of additional instruments. Particularly, one of the U.S. national guideline recommended instruments, the Hamilton Rating Scale for Depression, did not appear as a psychometrically tested suicide risk instrument during the search, but it is described in Brown’s (2001) review.

In addition, another potential limitation involves the low levels of evidence that support previously discussed results. Another obstacle involves relying so profoundly on the individual’s self-report in determining the effectiveness of a suicide risk assessment instrument. Only one sample instrument, the Implicit Association Test, did not involve participants relaying thoughts of self-harm and instead utilized computer-based assessments. Perhaps there is consistent underreporting of suicidal ideation in those who have acquired the ability and strongly desire to die by suicide. The resulting effects are practices and recommendations that are largely based on observational studies, validated primarily by participant self-report.

Conclusion

To confirm suicide risk assessment instruments’ psychometric properties and improve generalizability, more diverse population representation and additional representation of non-English versions in studies is required. Including underrepresented groups and non-English instruments will promote enhanced culturally and linguistically sensitive suicidal ideation and suicidal behavior instruments that may better predict risk. Additional research of underrepresented groups may also reduce the feelings of isolation and burdensomeness experienced by those with suicide ideation, if these groups are equally represented and included. Addressing existing research gaps may reduce morbidity (i.e., perceived burdensomeness and thwarted belongingness) and mortality (i.e., the acquired ability for lethal self-harm). Finally, addressing these research gaps will be important in understanding the social, cultural, economic, and political context of suicide.

Footnotes

Appendix A: Literature Table

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Elizabeth Kreuze, RN, is a PhD candidate in Nursing Science at the Medical University of South Carolina, College of Nursing. Her research focus includes mental health promotion and suicide prevention. As part of her dissertation research study, she is working collaboratively with her professors on a mixed methods analysis of a community suicide prevention project.

Dorian A. Lamis, PhD, is a licensed clinical psychologist and an assistant professor in the Department of Psychiatry and Behavioral Sciences at the Emory University School of Medicine. His research focuses on mood disorders, substance use, and suicidal behaviors in a variety of populations. He has edited 2 books and published over 100 peer reviewed articles and book chapters on these topics. He also has a strong interest in clinical work, especially with patients who have been diagnosed with a serious mental illness and are at-risk for suicide and/or other self-harm behaviors.

References

Aishvarya

Maniam

Karuthan

Sidi

Ruzyanei

Oei

T. P.

(2014) Psychometric properties and validation for reasons for living inventory in an outpatient clinical population in Malaysia. Comprehensive Psychiatry 55: S107–S113. doi:10.1016/j.comppsych.2013.06.010.

Al-Turkait

F. A.

Ohaeri

J. U.

(2010) Dimensional and hierarchical models of depression using the Beck depression inventory-II in an Arab college student sample. BMC Psychiatry 10: 1–14. doi:10.1186/1471-244X-10-60.

American Psychiatric Association. (2010). Practice guideline for the assessment and treatment of patients with suicidal behaviors. Arlington, VA. Retrieved from http://psychiatryonline.org/pb/assets/raw/sitewide/practice_guidelines/guidelines/suicide.pdf.

American Psychological Association. (2012). Diversity & suicidal behavior. American Psychological Association: Fact Sheet. Section VII. Retrieved from http://www.div12.org/wp-content/uploads/2012/10/Suicide-and-Diversity-Div12.pdf.

Ball

Steer

R. A.

(2003) Mean beck depression inventory-II scores of outpatients with dysthymic or recurrent episode major depressive disorders. Psychological Reports 93: 507–512. doi:10.2466/pr0.2003.93.2.507.

Beck

A. T.

Brown

G. K.

Steer

R. A.

(1997) Psychometric characteristics of the scale for suicide ideation with psychiatric outpatients. Behavior Research and Therapy 35: 1039–1046. doi:10.1016/S0005-7967(97)00073-9.

Beck

A. T.

Kovacs

Weissman

(1979) Assessment of suicidal intention: The scale for suicide ideation. Journal of Counseling and Clinical Psychology 4: 343–352. doi:10.1037/0022-006Z.47.2.343.

Beck

A. T.

Steer

R. A.

(1984) Internal consistencies of the original and revised beck depression inventory. Journal of Clinical Psychology 40: 1365–1367. ISSN: 0021-9762.

Beck

A. T.

Steer

R. A.

(1991) Manual for the Beck scale for suicide ideation, San Antonio, TX: Psychological Corporation.

10.

Beck

A. T.

Steer

R. A.

Ball

Ciervo

C. A.

Kabat

(1997) Use of the Beck anxiety and Beck depression inventories for primary care with medical outpatients. Assessment 4: 211–219.

11.

Beck

A. T.

Steer

R. A.

Ball

Ranieri

(1996) Comparison of Beck depression inventories –IA and –II in psychiatric outpatients. Journal of Personality Assessment 67: 588–597. ISSN: 0022-3891.

12.

Bender

T. W.

Gordon

K. H.

Bresin

Joiner

T. E.

(2011) Impulsivity and suicidality: A test of the mediating role of painful experiences. Journal of Affective Disorders 129: 301–307. doi:10.1016/j.jad.2010.07.023.

13.

Braveman

Egerter

Mockenhaupt

(2011) Broadening the focus: The need to address the social determinants of health. American Journal of Preventative Medicine 40: 4–18. doi:10.1016/j.amepre.2010.10.002.

14.

Brown, G. K. (2001). A review of suicide assessment measures for intervention research with adults and older adults. GK Brown. PDF retrieved from the Suicide Prevention Resource Center http://www.sprc.org/sites/sprc.org/files/library/BrownReviewAssessmentMeasuresAdultsOlderAdults.pdf.

15.

Bryan

C. J.

(2011) The clinical utility of a brief measure of perceived burdensomeness and thwarted belongingness for the detection of suicidal military personnel. Journal of Clinical Psychology 67: 981–992. doi:10.1002/jclp.20726.

16.

Bryan

C. J.

Hernandez

A. M.

Sybil

Clemans

(2013) Combat exposure and suicide risk in two samples of military personnel. Journal of Clinical Psychology 69: 64–77. doi:10.1002/jclp.21932.

17.

Bryan

C. J.

Rudd

D. M.

(2012) Life stressors, emotional distress, and trauma-related thoughts occurring in the 24 h preceding active duty U.S. soldiers’ suicide attempts. Journal of Psychiatric Research 46: 843–848. doi:10/1016/j.jpsychires.2012.03.012.

18.

Center for Disease Control and Prevention. (2013). Years of potential life lost (YPLL) reports, 1999–2013. Atlanta, Georgia. Retrieved from http://webappa.cdc.gov/sasweb/ncipc/ypll10.html.

19.

Centers for Disease Control and Prevention. (2014). The social-ecological model: A framework for prevention. Injury Prevention & Control: Division of Violence Prevention. Atlanta, Georgia. Retrieved from http://www.cdc.gov/violenceprevention/overview/social-ecologicalmodel.html.

20.

Chu

Floyd

Diep

Pardo

Goldblum

Bongar

(2013) A tool for the culturally competent assessment of suicide: The cultural assessment of risk for suicide (CARS) measure. Psychological Assessment 25: 424–434. doi:10.1037/a0031264.

21.

Coric, V., Stock, E. G., Pultz, J., Marcus, R., & Sheehan, D. V. (2009). Sheehan suicidality tracking scale (Sheehan-STS): Preliminary results from a multicenter clinical trial in generalized anxiety disorder. Psychiatry (Edgmont), 6, 26–31.

22.

Cull

J. G.

Gill

W. S.

(1982) Suicide probability scale, Los Angeles, CA: Western Psychological Services.

23.

De Baradis

Serroni

Marini

Rapini

Carano

Valchera

Di Giannantonio

(2014) Alexithymia, suicidal ideation, and serum lipid levels among drug-naïve outpatients with obsessive-compulsive disorder. Brazilian Journal of Psychiatry 36: 125–130. doi:10.1590/1516-4446-2013-1189.

24.

De Beradis

Campanella

Serroni

Moschetta

F. S.

Di Emidio

Conti

Di Giannantonio

(2013) Alexithymia, suicide risk and serum lipid levels among outpatients with panic disorder. Comprehensive Psychiatry 54: 517–522. doi:10.1016/j.comppsych.2012.12.013.

25.

Ellis, T. E., Rufino, K. A., & Green, K. L. (2016). Implicit measure of life/death orientation predicts response of suicidal ideation to treatment in psychiatric inpatients. Archives of Suicide Research. 20(1), 59–68. doi:10.1080/13811118.2015.1004483.

26.

Farzanfar

Hereen

Fava

Davis

Vachon

Friedman

(2014) Psychometric properties of an automated telephone-based PHQ-9. Telemedicine and e-Health 20: 115–121. doi:10.1089/tmj.2013.0158.

27.

Fine

T. H.

Contractor

A. A.

Tamburrino

Elhai

J. D.

Prescott

M. R.

Cohen

G. H.

Calabrese

J. R.

(2013) Validation of the telephone-administered PHQ-9 against in-person administered SCID-I major depression. Journal of Affective Disorders 150: 1000–1007. doi:10.1016/j.jad.2013.05.029.

28.

Freedenthal

Lamis

D. A.

Osman

Kahlo

Gutierrez

P. M.

(2011) Evaluation of the psychometric properties of the interpersonal needs quiestionnaire-12 in samples of men and women. Journal of Clinical Psychology 67: 609–623. doi:10.1002/jclp.20782.

29.

Friedman

R. H.

Stollerman

J. E.

Mahoney

D. M.

Rozenblyum

(1997) The virtual visit: Using telecommunications technology to take care of patients. Journal of the American Medical Informatics Association 4: 413–425. ISSN: 1067-5027.

30.

Hammash

M. H.

Hall

L. A.

Lennie

T. A.

Heo

Chung

M. L.

Lee

K. S.

Moser

D. K.

(2012) Psychometrics of the PHQ-9 as a symptom measure of depressive symptoms in patients with heart failure. European Journal of Cardiovascular Nursing 12: 446–453. doi:10.1177/1474515112468068.

31.

Hill

R. M.

Rey

Marin

C. E.

Sharp

Green

K. L.

Pettit

J. W.

(2014) Evaluating the interpersonal needs questionnaire: Comparison of the reliability, factor structure, and predictive validity across five versions. Suicide and Life-Threatening Behavior 45: 302–314. doi:10.1111/sltb.12129.

32.

Horon

McManus

Schmollinger

Barr

Jimenez

(2013) A study of the use and interpretation of standardized suicide risk assessment: Measures within a psychiatrically hospitalized correctional population. Suicide and Life-Threatening Behavior 43: 17–38. doi:10.1111/j.1943-278X.2012.00124.x.

33.

Huffman

J. C.

Doughty

C. T.

Januzzi

J. L.

Pirl

W. F.

Smith

F. A.

Fricchione

G. L.

(2010) Screening for major depression in post-myocardial infarction patients: Operating characteristics of the Beck depression inventory-II. International Journal of Psychiatry in Medicine 40: 187–197. doi:10.2190/PM.40.2.e.

34.

Husain

Afsar

Ara

Fayyaz

Rahman

R. U.

Tomenson

Chaudhry

I. B.

(2014) Brief psychological intervention after self-harm: Randomised controlled trial from Pakistan. British Journal of Psychiatry 204: 462–470. doi: 10.1192/bjp.bp.113.138370.

35.

Inagaki

Ohtsuki

Yonemoto

Kawashima

Saitoh

Oikawa

Yamada

(2013) Validity of the patient health questionnaire (PHQ)-9 and PHQ-2 in general internal medicine in primary care at a Japanese rural hospital: A cross-sectional study. General Hospital Psychiatry 35: 592–597. doi:10.1016/j.genhosppsych.2013.08.001.

36.

Joiner

T. E.

(2005) Why people die by suicide, Cambridge, MA: Harvard University Press.

37.

Joiner

T. E.

Pfaff

J. J.

Acres

J. G.

(2002) A brief screening tool for suicidal symptoms in adolescents and young adults in general health settings: Reliability and validity data from the Australian national general practice youth suicide project. Behavior Research and Therapy 40: 471–481. doi:10.1016/S0005-7967(01)00017-1.

38.

Jon

W. H.

Lee

E. J.

Park

J. S.

(2013) Effects of a suicide prevention programme for hospitalised patients with mental illness in South Korea. Journal of Clinical Nursing 23: 1845–1856. doi:10.1111/jocn.12417.

39.

Khamseh

M. E.

Baradaran

H. R.

Javanbakht

Mirghorbani

Yadollahi

Malek

(2011) Comparison of the CES-D and PHQ-9 depression scales in people with type 2 diabetes in Tehran, Iran. BMC Psychiatry 11: 1–6. doi:10.1186/1471-244X-11-61.

40.

Kim

J. H.

Park

D. H.

Ryu

S. H.

(2014) Path analysis of suicide ideation in older people. International Psychogeriatrics 26: 509–515. doi:10.1017/S1041610213002366.

41.

Kjaergarrd

Elisabeth

Wang

Waterloo

Jorde

(2014) A study of the psychometric properties of the Beck depression inventory-II, the Montgomery and Asberg depression rating scale, and the hospital anxiety and depression scale in a sample from a healthy population. Scandinavian Journal of Psychology 55: 83–89. doi:10.1111/sjop.12090.

42.

Kliem

Moble

Zenger

Brahler

(2014) Reliability and validity of the beck depression inventory-fast screen for medical patients in the general German population. Journal of Affective Disorders 156: 236–239. doi:10.1016/j.jad.2013.11.024.

43.

Kroenke

Spitzer

R. L.

Williams

J. B.

(2001) The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine 16: 606–613. doi:10.1046/j.1525-1497.2001.016009606.x.

44.

Lerdal

Kottorp

Gay

C. L.

Grov

E. K.

Lee

K. A.

(2014) Rasch analysis of the Beck depression inventory-II in stroke survivors: A cross-sectional survey. Journal of Affective Disorders 158: 48–52. doi:10.1016/j.jad.2014.01.013.

45.

Xie

Luo

Shi

Ying

Wang

(2014) Clarifying the role of psychological pain in the risks of suicidal ideation and suicidal acts among patients with major depressive episodes. Suicide and Life-Threatening Behavior 44: 78–88. doi:10.1111.sltb.12056.

46.

Lie

H. C.

Hjermstad

M. J.

Fayers

Finset

Kaasa

Loge

J. H.

(2015) Depression in advanced cancer—assessment challenges and associations with disease load. Journal of Affective Disorders 173: 176–184. doi:10.1016/j.jad.2014.11.006.

47.

Lim

H. W.

Song

H. S.

Hwang

Y. H.

Lee

H. W.

Suh

C. K.

Park

S. P.

Kwon

S. H.

(2010) Predictors of suicidal ideation in people with epilepsy living in Korea. Journal of Clinical Neurology 6: 81–88. doi:10.3988/jcn.2010.6.2.81.

48.

Linehan

M. M.

Goodstein

J. L.

Nielsen

S. L.

Chiles

J. A.

(1983) Reasons for staying alive when you are thinking of killing yourself: The reasons for living inventory. Journal of Consulting and Clinical Psychology 51: 276–286. doi:10.1037/0022-006X.51.2.276.

49.

Luxton

D. D.

Rudd

M. D.

Reger

M. A.

Gahm

G. A.

(2011) A psychometric study of the suicide ideation scale. Archives of Suicide Research 15: 250–258. doi:10.1080/13811118.2011.589720.

50.

Merz

E. L.

Malcarne

V. L.

Roesch

S. C.

Riley

Sadler

G. R.

(2011) A multigroup confirmatory factor analysis of the patient health questionnaire-9 among English- and Spanish-speaking Latinas. Cultural Diversity and Ethnic Minority Psychology 17: 309–316. doi:10.1037/a0023883.

51.

Metalsky

G. I.

Joiner

T. E.

(1997) The hopelessness depression symptom questionnaire. Cognitive Therapy and Research 21: 359–384.

52.

Miranda

Gallagher

Bauchner

Vaysman

Marroquin

(2012) Cognitive inflexibility as a prospective predictor of suicidal ideation among young adults with a suicide attempt history. Depression and Anxiety 29: 180–186. doi:10.1002/da.20915.

53.

Miranda

Valderrama

Tsypes

Gadol

Gallagher

(2013) Cognitive inflexibility and suicidal ideation: Mediating role of brooding and hopelessness. Psychiatry Research 1: 174–181. doi:10.1016/j.psychres.2013.02.033.

54.

Moher

Liberati

Tetzlaff

Altman

D. G.

The PRISMA Group (2009) Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine 6: e1000097. doi:10.1371/journal.pmed1000097.

55.

National Guideline Clearinghouse. (2014). Clinical practice guideline for the prevention and treatment of suicidal behavior. Rockville, MD: Agency for Healthcare Research and Quality. Retrieved from http://www.guideline.gov/content.aspx?id=48046&search=suicide+ideation.

56.

Naud

Daigle

M. S.

(2010) Predictive validity of the suicide probability scale in a male inmate population. Journal of Psychopathology and Behavioral Assessment 32: 333–342. doi:10.1007/s10862-009-9159-8.

57.

Nock

M. K.

Banaji

M. R.

(2007a) Assessment of self-injurious thoughts using a behavioral test. American Journal of Psychiatry 164: 820–823. doi:10.1176/ajp.2007.164.5.820.

58.

Nock

M. K.

Banaji

M. R.

(2007b) Prediction of suicide ideation and attempts among adolescents using a brief performance-based test. Journal of Consulting and Clinical Psychology 75: 707–715. doi:10.1037/0022-006X.75.5.707.

59.

Nock

M. K.

Park

J. M.

Finn

C. T.

Deliberto

T. L.

Dour

H. J.

Banaji

M. R.

(2010) Measuring the suicidal mind: Implicit cognition predicts suicidal behavior. Psychological Science 21: 511–517. doi:10.1177/0956797610364762.

60.

O’Connor

R. C.

Smyth

Ferguson

Ryan

Williams

J. M.

(2013) Psychological processes and repeat suicidal behavior: A four-year prospective study. Journal of Consulting and Clinical Psychology 81: 1137–1143. doi:10.1037/a0033751.

61.

O’Keefe

V. M.

Wingate

L. R.

(2013) The role of hope and optimism in suicide risk for American Indians/Alaska natives. Suicide and Life-Threatening Behavior 43: 621–633. doi:10.1111/sltb.12044.

62.

O’Riley

A. A.

Fiske

(2012) Emphasis on autonomy and propensity for suicidal behavior in younger and older adults. Suicide and Life-Threatening Behavior 42: 394–404. doi:10.1111/j.1943-278X.2012.00098.x.

63.

Osman

Bagge

C. L.

Guiterrez

P. M.

Konick

L. C.

Kopper

B. A.

Barrios

F. X.

(2001) The suicidal behaviors questionnaire-revised (SBQ-R): Validation with clinical and nonclinical samples. Psychological Assessment 8: 443–454. doi:10.1177.107319110100800409.

64.

Oxford Center for Evidence-Based Medicine. (2011). Levels of evidence. Oxford, England: Author. Retrieved from http://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf.

65.

Panagioti

Gooding

P. A.

Nicholas

(2012) Hopelessness, defeat, and entrapment in posttraumatic stress disorder: Their association with suicidal behavior and severity of depression. Journal of Nervous and Mental Disease 200: 676–683. doi:10.1097/NMD.0b013e3182613f91.

66.

Phelan

Williams

Meeker

Bonn

Frederick

LoGerfo

Snowden

(2010) A study of the diagnostic accuracy of the PHQ-9 in primary care elderly. BMC Family Practice 11: 1–9. doi:10.1186/1471-2296-11-63.

67.

Pinninti, N., Steer, R. A., Rissmiller, D. J., Nelson, S., & Beck, A. T. (2002). Use of the Beck Scale for suicide ideation with psychiatric inpatients diagnosed with schizophrenia, schizoaffective, or bipolar disorders. Behaviour Research and Therapy, 40, 1071–1079. doi:10.1016/S0005-7967(02)00002-5.

68.

Polanco-Roman

Miranda

(2013) Culturally related stress, hopelessness, and vulnerability to depressive symptoms and suicidal ideation in emerging adulthood. Behavior Therapy 44: 75–87. doi:10.1016/j.beth.2012.07.002.

69.

Preti

Sheehan

D. V.

Coric

Distinto

Pitanti

Vacca

Petretto

D. V.

(2013) Sheehan suicidality tracking scale (S-STS): Reliability, convergent and discriminative validity in young Italian adults. Comprehensive Psychiatry 54: 842–849. doi:10.1016/j.comppsych.2013.03.012.

70.

Randall

J. R.

Rowe

B. H.

Dong

K. A.

Nock

M. K.

Colman

(2013) Assessment of self-harm risk using implicit thoughts. Psychological Assessment 25: 714–721. doi:10.1037/a0032391.

71.

Rathore

J. S.

Jehi

L. E.

Fan

Patel

S. I.

Foldvary-Schaefer

Ramirez

M. J.

Tesar

G. E.

(2014) Validation of the patient health questionnaire-9 (PHQ-9) for depression screening in adults with epilepsy. Epilepsy & Behavior 37: 215–220. doi:10.1016/j.yebah.2014.06.030.

72.

Razykov

Zieglestein

R. C.

Whooley

M. A.

Thombs

B. D.

(2012) The PHQ-9 versus the PHQ-8 – is item 9 useful for assessing suicide risk in coronary artery disease patients? Data from the heart and soul study. Journal of Psychosomatic Research 73: 163–168. doi:10/1016/j.jpsychores.2012.06.001.

73.

Reynolds

W. M.

(1991) Psychometric characteristics of the adult suicidal ideation questionnaire in college students. Journal of Personality Assessment 56: 289–307. doi:10.1207/s15327752jpa5602_9.

74.

Ribeiro

J. D.

Witte

T. K.

Van Orden

K. A.

Selby

E. A.

Gordon

K. H.

Bender

T. W.

Joiner

T. E.

(2014) Fearlessness about death: The psychometric properties and construct validity of the revision to the acquired capability for suicide scale. Psychological Assessment 26: 115–126. doi:10.1037/a0034858.

75.

Rudd

M. D.

(1989) The prevalence of suicidal ideation among college students. Suicide & Life-Threatening Behavior 19: 173–183.

76.

Scheinthal, S. M., Steer, R. A., Giffin, L. & Beck, A. T. (2001). Evaluating geriatric medical outpatients with beck depression inventory-fast screen for medical patients. Aging & Mental Health, 5, 143–148. doi:10.1080/13607860120038320.

77.

Segal

D. L.

Marty

M. A.

Meyer

W. J.

Coolidge

F. L.

(2012) Personality, suicidal ideation, and reasons for living among older adults. Journal of Gerontology Psychological Society and Social Science 67B: 159–166. doi:10.1093/geronb/gbr080.

78.

Sheehan

D. V.

Lecrubier

Sheehan

K. H.

Amorim

Janavas

Weiller

Dunbar

G. C.

(1998) The mini-international neuropsychiatric interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry 59: 22–33.

79.

Smith

P. N.

Wolford

Mandracchia

J. T.

Jahn

D. R.

(2013) An exploratory factor analysis of the acquired capability for suicide scale in male prison inmates. Psychological Services 10: 97–105. doi:10.1037/a0030817.

80.

Steer

R. A.

Ball

Ranieri

W. F.

Beck

A. T.

(1999) Dimensions of the beck depression inventory-II in clinically depressed outpatients. Journal of Clinical Psychology 55: 117–128. doi:10.1002/(SICI)1097-4679(199901).

81.

Steer

R. A.

Cavalieri

T. A.

Leonard

D. M.

Beck

A. T.

(1999) Use of the Beck depression inventory for primary care to screen for major depression disorders. General Hospital Psychiatry 21: 106–111.

82.

Teismann

Fortsch

E. M.

Baumgart

Het

Michalak

(2014) Influence of violent video gaming on determinants of the acquired capability for suicide. Psychiatry Research 215: 217–222. doi:10.1016/j.psychres.2013.10.021.

83.

Titov

Dear

B. F.

McMillan

Anderson

Zou

Sunderland

(2011) Psychometric comparison of the PHQ-9 and BDI-II for measuring response during treatment of depression. Cognitive Behavioral Therapy 40: 126–136. doi:10.1080/16506073.2010.550059.

84.

Tzu-Chi

Hwa-Sheng

Chen-Huan

Ying-Yeh

Kuei-Ru

Hsien-Chich

Hsiu-Ju

(2010) Anxiety, depressive symptom and suicidal ideation of outpatients with obsessive compulsive disorders in Taiwan. Journal of Clinical Nursing 19: 3092–3101. doi:10.1111/j.1365-2702.2010.03378.x.

85.

Uebelacker

L. A.

German

N. M.

Baudiano

B. A.

Miller

I. W.

(2011) Patient health questionnaire depression scale as a suicide screening instrument in depressed primary care patients: A cross-sectional study. The Primary Care Companion For CNS Disorders 13: e1–e6. doi:10.4088/PCC.10m01027.

86.

Van Orden

K. A.

Cukrowicz

K. C.

Witte

T. K.

Joiner

T. E.

(2012) Thwarted belongingness and perceived burdensomeness: Construct validity and psychometric properties of the interpersonal needs questionnaire. Psychological Assessment 24: 197–215. doi:10.1037/a0025358.

87.

Van Orden

K. A.

Witte

T. K.

Gordon

K. H.

Bender

T. W.

Joiner

T. E.

(2008) Suicidal desire and the capability for suicide: Tests of the interpersonal-psychological theory of suicidal behavior among adults. Journal of Consulting and Clinical Psychology 76: 72–83. doi:10.1037/0022-006X.76.1.72.

88.

Vuorilehto

Valtonen

H. M.

Merlartin

Sokero

Suominen

Isometsa

E. T.

(2014) Method of assessment determines prevalence of suicidal ideation among patients with depression. European Psychiatry 29: 338–344. doi:10.1016/j.eurpsy.2013.08.005.

89.

Wagner

Klinitzke

Brahler

Kersting

(2013) Extreme obesity is associated with suicidal behavior and suicide attempts in adults: Results of a population-based representative sample. Depression and Anxiety 30: 975–981. doi:10.1002/da.22105.

90.

Whittemore

Knafl

(2005) An integrative review: Updated methodology. Journal of Advanced Nursing 52: 546–553. doi:10.1111/j.1365-2648.2005.03621.x.

91.

Woods

A. M.

Zimmerman

Carlin

Hill

Kaslow

N. J.

(2013) Motherhood, reasons for living, and suicidality among African American women. Journal of Family Psychology 27: 600–606. doi:10.1037/a0033592.

92.

World Health Organization. (2014). Preventing suicide: A global imperative. Geneva, Switzerland: Author. Retrieved from http://apps.who.int/iris/bitstream/10665/131056/1/9789241564779_eng.pdf.

93.

Xie

Luo

Ying

Wang

Shi

(2014) Anhedonia and pain avoidance in the suicidal mind: Behavioral evidence for motivational manifestations of suicidal ideation in patients with major depressive disorder. Journal of Clinical Psychology 70: 681–692. doi:10.1002/jclp.22055.

94.

Zhong

Gelaye

Rondon

Sanchez

S. E.

Garcia

P. J.

Sanchez

Williams

M. A.

(2014) Comparative performance of patient health questionnaire-9 and Edinburgh postnatal depression scale for screening antepartum depression. Journal of Affective Disorders 162: 1–7. doi:10.1016/j.jad.2014.03.028.

95.

Zuithoff

N. P.

Vergouwe

King

Nazerth

van Wezep

M. J.

Moons

K. G.

Geerlings

M. I.

(2010) The patient health questionnaire-9 for detection of major depressive disorder in primary care: Consequences of current thresholds in a crosssectional study. BMC Family Practice 11: 98. doi: 10.1186/1471-2296-11-98.