Allowing for Nondisclosure in High Suicide Risk Groups

Abstract

Inaccurate disclosure of information is a significant limitation when relying on self-report for suicide risk screening. However, allowing for an explicit nondisclosure response option in screening items (e.g., “prefer not to disclose”) may ultimately improve follow-up assessment validity. This study investigated explicit nondisclosure in suicide risk screening among military service members and veterans (n = 135), men older than 50 years, (n = 187), and LGBTQ (lesbian, gay, bisexual, transgender, or queer) young adults (n = 140); all higher risk and lower help-seeking populations. Results indicated that allowing for explicit nondisclosure minimally affected screening sensitivity and specificity. However, its selection was prevalent among higher risk participants, and was associated with experience of past unhelpful reactions and “avoidance of stigmatized consequences.” Allowing for nondisclosure creates potential psychometric and classification advantages, and may be a safe and effective way to encourage discussion of barriers, build trust, maximize patient autonomy, and ultimately facilitate accurate risk disclosure to improve assessment validity.

Keywords

selective disclosure risk assessment help seeking BSS mixed methods

Currently, direct self-report questioning is a universally accepted standard practice for clinical screening of suicide risk (Bryan & Rudd, 2006; Chu et al., 2015; Joiner, Walker, Rudd, & Jobes, 1999), and is a key first step to identifying suicidal individuals and connecting them with appropriate follow-up assessment and care (LeFevre, 2014; Mann et al., 2005; Suicide Prevention Resource Center, 2015). However, while a review of evidence indicates that more than 75% of suicide decedents had contact with a primary care provider in the year before their death (Luoma, Martin, & Pearson, 2002), another review found that almost an equivalent 70.5% of individuals who endorsed suicide ideation, plans, and/or attempts in the past year reported no use of mental health services (Hom, Stanley, & Joiner, 2015). Combined, these data suggest significant problems with patient identification and referral for suicide risk.

Even when suicide risk screening is faithfully implemented, there are significant limitations with reliance on the patient’s self-report. These include the patient’s stigma against help seeking for suicidal thoughts (Hom, Stanley, Podlogar, & Joiner, 2017; Niederkrotenthaler, Reidenberg, Till, & Gould, 2014; Vogel & Wade, 2009), past negative experiences with disclosure (Fulginiti, Pahwa, Frey, Rice, & Brekke, 2016), and uncertainty or ambivalence about suicidal thoughts (Harris, McLean, Sheffield, & Jobes, 2010)—which all may promote inaccurate or deceptive self-reporting of current suicide risk. Ilic et al. (2014) found, in a longitudinal study among people with mental illness, that the experience of stigma in past interactions predicted subsequent maladaptive responses of overcompensation (i.e., acting in counterstereotypic ways) and withdrawal (i.e., defensive avoidance of people or situations where discrimination may occur), which in turn, predicted worse mental health outcomes and decreased social support. With regard to suicidal thoughts and behaviors, these responses may translate to intentional underreporting of suicide risk and avoidance of help seeking, resulting in greater isolation and greater risk for suicide. Additional barriers to seeking care for suicidal thoughts or behaviors have been found to include a lack of perceived need for services, preference for self-management, fear of hospitalization, and structural factors such as time or finances (Drum, Brownson, Burton Denmark, & Smith, 2009; Hom et al., 2015). Lastly, social desirability bias and the potential for unintentional misunderstanding of specific screening terms (e.g., “suicidal behavior”) may confuse respondent reports (Brener, Billy, & Grady, 2003; Tourangeau & Yan, 2007; see Crosby, Ortega, & Melanson, 2011, for a discussion of standardizing suicide risk classification terms). Furthermore, when self-reported suicide risk is genuinely disclosed, suicidal ambivalence, uncertainty, and temporal instability still limit its predictive reliability. For example, a multinational survey of more than 1,000 participants found that nearly 40% of individuals at higher risk for suicide reported that they frequently engaged in an internal debate about “whether to live or die,” suggesting that even when self-report may be accurate at the time, it can likely change (Harris et al., 2010). Another study of hospital inpatients found that self-ratings of suicidal thoughts and intent often varied dramatically on an hourly basis (Kleiman et al., 2017).

Given all of these barriers, it may be unsurprising that approximately half of people who report lifetime suicidal thoughts deny ever telling someone about such thoughts (Drum et al., 2009; Encrenaz et al., 2012). Among primary-care patients who endorsed suicidal ideation on a confidential research questionnaire, only 7% initiated a conversation about such thoughts with their provider (Vannoy & Robins, 2011). Conversely, 78% of current or recently discharged inpatients who died by suicide had explicitly denied suicide risk during the final communications before their deaths (Busch, Fawcett, & Jacobs, 2003). Consequently, methods for suicide risk identification beyond self-reported questionnaires are beginning to be researched, including through the use of blood biomarkers, implicit behavioral task indicators, personal digital devices, and big data algorithms (Cha, Najmi, Park, Finn, & Nock, 2010; Glenn & Monteith, 2014; Niculescu et al., 2015; Nock et al., 2010; Simon et al., 2018). Nevertheless, despite all its limitations, self-report is unambiguously the cheapest, fastest, and most prevalent method of suicide risk screening in current accepted standard practice—and in many situations, it is the only available option. Therefore, identifying and directly addressing specific limitations of self-reported suicide risk screening without altogether discarding its clinical utility is a needed direction for research.

The Present Study

One specific aspect of self-reported suicide risk screening that may contribute to maladaptive response reactions is the frequent reliance on “forced” responding. Here, “forced” responding is defined as any screening method that either requests or requires the respondent to provide a quantitative or qualitative response to a screening question, as opposed to explicitly allowing for no evaluative answer to be given to one or more questions (i.e., “nondisclosure”). Typically, self-reported suicide risk screening surveys “force” respondents in this way to endorse, deny, or quantify various aspects of suicide risk, for example, through the use of dichotomous or Likert-type items (Brown, 2000; Range & Knott, 1997). Forced responding may be considered by many clinicians and health care systems to be necessary in suicide risk screening, particularly if one relies exclusively on using screening cutoff scores to inform the need for follow-up assessment (for a discussion of the difference between screening and assessment, see Suicide Prevention Resource Center, 2014). Forced responding may also satisfy the evaluator’s understandable concern for medicolegal negligence if suicide risk questions were skipped and the respondent is either involuntarily hospitalized without sufficient justification, or alternatively, dies by what may have been a preventable suicide (Packman, Pennuto, Bongar, & Orthwein, 2004; Pearson, Stanley, King, & Fisher, 2001). However, forced responding in suicide risk screening may possibly contribute to false or misleading risk classification due to the numerous limitations of self-reported suicide risk screening discussed above, counteracting its perceived clinical utility.

Selective disclosure (i.e., withholding information from unknown individuals until they are trusted) and information seeking (i.e., increasing one’s knowledge, discourse, and competency) are two adaptive coping responses found to be used by stigmatized individuals that were associated with improved mental health outcomes (Ilic et al., 2014). Selective disclosure to only trusted confidants was also described by 40 suicide attempt survivors as a central component of their decision-making process for seeking help (Frey, Fulginiti, Lezine, & Cerel, 2018), and the general prevalence of its use is supported by the fact that suicidal individuals are much more likely to report thoughts and behaviors to close family and friends over medical or mental health gatekeepers (Drum et al., 2009; Encrenaz et al., 2012).

Therefore, an alternative to relying on forced responding in self-reported suicide risk screening is to instead explicitly invite and allow for a nondisclosure response option to each question (e.g., “prefer not to disclose”). Although allowing for such an option may come at the cost of worsened screening specificity (i.e., more people may be flagged for follow-up assessment because their screening risk is undisclosed), it may ultimately improve the validity of the follow-up clinical assessment by signaling to the respondent an invitation for information seeking and trust building before forcing a definitive endorsement of some level of suicide risk. Other potential advantages of allowing for explicit nondisclosure in screening items include the possibility that the choice of nondisclosure in and of itself may improve screening measure predictions above and beyond explicitly reported data, and that the screening measure may be continuously improved psychometrically by collecting feedback from respondents of “rejected” items (e.g., due to confusing or disagreeable language). However, to the authors’ knowledge, no study of self-report suicide risk screening measures has investigated the potential costs and benefits of including a nondisclosure response option as a procedurally and psychometrically informative response choice for determining suicide risk.

This study aimed to measure the prevalence of nondisclosure in suicide risk screening among demographic groups at high risk for suicide when it was explicitly presented as a “prefer not to disclose” response option, investigate statistical correlates and respondent reasons for nondisclosure, and investigate whether allowing for and recording nondisclosure may inform follow-up risk assessment and measure development. It was hypothesized that (1) nondisclosure would be specific to higher suicide-risk individuals; (2) predictors and given reasons for nondisclosure would include aversion to help seeking, perceived stigma, avoidance of consequences, and/or suicidal ambivalence; and (3) nondisclosure would be positively associated with suicide risk indicators (e.g., hopelessness, past suicidal behavior) above and beyond explicit report.

Method

Participants

Participants were recruited from three high suicide risk demographic groups. Namely, military service members and veterans (Milvet, n = 135); men older than 50 years (M50, n = 187); and self-identified lesbian, gay, bisexual, transgender, or queer young adults (LGBTQ, n = 140). Each of these demographic groups have higher suicide rates and/or lower mental health help-seeking rates than the general population (Canetto, 2017; Denneson et al., 2016; Haas et al., 2010; Ribeiro et al., 2017).

See Tables 1 and 2 for descriptive statistics by recruitment source and demographic group. Participants were recruited via targeted advertisements online (i.e., Facebook and Reddit discussion boards for military, veteran, and LGBTQ communities), paper flyers on campus and in the local community, and by private Qualtrics research panels. All interested participants were directed to the study website, where they were provided with the study consent page and were informed that the study consisted of “an anonymous online survey of your history and current perspectives regarding mental health issues, such as depression, anxiety, and suicide; and may include a phone interview for selected participants.” Participants were further informed that the purpose of the study was “to improve online mental health survey techniques,” and only after the study, were participants debriefed that the researchers were specifically interested in level of suicide risk disclosure. Interested participants were also required to correctly answer comprehension questions, state their consent and commitment to provide accurate responses in a free-response text box, indicate which study inclusion criteria they met, and provide a valid U.S. phone number.

Table 1.

Sample Counts, Attrition, and Response Set Validity by Participant Source.

	Online recruitment		Qualtrics panel		University/community		F/χ²	df	p
	n/M	%/SD	n/M	%/SD	n/M	%/SD	F/χ²	df	p
1. Viewed page	2,496		577		37		809.32***	4	<.001
2. Gave consent	418_a	17%	558_b	97%	37_b	100%
3. Met study criteria	205	49%	263	47%	20	54%
4. Valid response set	186	91%	256	97%	20	100%
Minutes to completion^†	43	468	39	79	41	30	[median test]		.089
Measure inconsistency	0.23_a	0.09	0.20_b	0.08	0.22_a	0.08	9.32**	456	.002
Measure variance	1.75_a	0.60	1.56_b	0.58	1.76_a	0.35	6.15*	458	.022

Note. df = degree of freedom. Percentages are within column, relative to the next/previous count. Values within rows that do not share subscripts are significantly different from each other in pairwise tests. “Measure inconsistency” was calculated by taking each participant’s difference between straight-scored and reverse-scored items averaged across all survey measures and standardized to the total sample, such that 0 indicates perfectly consistent reporting across all items, and 1 indicates the most inconsistent reporting on average in the study sample. “Measure variance” was calculated by averaging the variance of each measure for each participant, such that 0 indicates straight-line responding.

†

Medians and standard deviations are presented due to large positive skew.

p < .05. **p < .01. ***p < .001.

Table 2.

Descriptives of Study Samples by Population Group.

	Milvet (n = 135)		M50 (n = 187)		LGBTQ (n = 140)		F/χ²	df	p
	n/M	%/SD	n/M	%/SD	n/M	%/SD	F/χ²	df	p
Age	51.46_a	16.65	59.45_b	6.41	20.37_c	3.41	2608.18***	252.52	<.001
DASS-21	26.24	12.29	26.14	11.73	25.18	10.62	0.37	459	.692
Male	122_a	90.4%	187_b	100.0%	38_c	27.1%	251.43***	4	<.001
Female	13_a	9.6%	0_b	0.0%	98_c	70.0%
Other	0_a	0.0%	0_a	0.0%	4_c	2.9%
White	108_a	80.0%	179_b	95.7%	86_c	61.4%	70.02***	8	<.001
Black	11_a	8.1%	3_b	1.6%	15_a	10.7%
Hispanic	6_a	4.4%	2_a	1.1%	27_c	19.3%
Asian	1	0.7%	1	0.5%	2	1.4%
Other	9_a	6.7%	2_b	1.1%	10_a	7.1%
AcPst	3.83	1.19	3.48	1.32	3.68	1.20	2.24	317	.108
HelpPst	3.91	0.80	3.21	0.92	3.81	0.85	2.06	133	.131
AcFtr	4.16	1.01	3.96	1.05	3.90	1.09	1.74	317	.177
Den	60_a	44.4%	80_a	42.8%	82_b	58.6%	10.16*	4	.038
Full D	58_a	43.0%	81_a	43.3%	40_b	28.6%
Non D	17	12.6%	26	13.9%	18	12.9%

Note. Milvet = military or veteran status; M50 = men older than 50 years; LGBTQ = identified as lesbian, gay, bisexual, transgender, or queer; DASS-21 = 21-item Depression Anxiety Stress Scales; AcPst = accuracy of past risk disclosure; HelpPst = helpfulness of responses to past risk disclosure; AcFtr = predicted accuracy of future risk reporting; Den = deniers (answered and denied all BSS-ND items); Full D = full disclosers (answered all and elevated one or more BSS-ND items); Non D = nondisclosers (did not answer some or all of the BSS-ND items); df = degree of freedom. Values within rows that do not share subscripts are significantly different from each other in pairwise tests.

p < .05. **p < .01. ***p < .001.

After providing informed consent, interested participants were asked to complete the 21-item Depression, Anxiety, and Stress Scale (DASS-21; Lovibond & Lovibond, 1995), which is a screening measure of current psychopathology or distress, and were next asked to call a phone number from the phone number that they previously reported, in order to prevent survey fraud (see Teitcher et al., 2015). Interested participants who scored below a cutoff of 11 on the DASS-21 (i.e., below the 70th percentile in a nonclinical sample, Henry & Crawford, 2005) or who did not call from a phone number they provided were excluded from the study. The cutoff score of 11 on the DASS-21 was implemented to oversample for higher suicide-risk participants, as determined from pilot testing on an undergraduate student sample.

Procedure

All procedures were reviewed and approved by the university institutional review board. Participants who passed inclusion and exclusion criteria completed an online battery of survey measures, including measures assessing suicide-associated risk factors, help-seeking associated factors, and explicit suicide risk. Survey measures were presented in random order across participants, with the exception of the direct suicide risk measure (i.e., Beck Scale for Suicide Ideation–Nondisclosure [BSS-ND], defined below) which was presented at the end to minimize order effects. Participants were allowed to skip any/all items in all survey measures, but were presented with a pop-up confirmation if they attempted to proceed to the next page without answering an item. All nonstudent, nonpanel participants received the equivalent of a $10 Amazon.com gift code for survey completion. University students were provided the option of receiving research participation class credit or the Amazon.com gift code. Qualtrics panel participants were compensated directly by Qualtrics, according to private Qualtrics panel contracts.

After study debriefing and compensation, participants who did not deny current intent for suicide (i.e., either reported current elevated intent or did not disclose intent) were notified to expect a follow-up phone call from the principal investigator and were called within 2 hours of survey completion. The study website was accessible only between the hours of 6:00 a.m. and 11:00 p.m. EST to allow for this immediate follow up. The principal investigator attempted up to 3 times to reach identified participants for a follow-up phone call, separated by 1-day intervals. The principal investigator left scripted voice messages when possible, and on the third and final message, offered an additional $15 Amazon.com gift code for follow-up compensation to promote response, given the high level of safety concerns (for a discussion of using targeted incentives, see Kay, Boggess, Selvavel, & McMahon, 2001). If a participant was successfully reached for the follow-up phone interview, the principal investigator followed a scripted risk assessment and asked a brief set of qualitative, open-ended survey questions regarding experiences with past disclosure and reasons for nondisclosure, as applicable. Participants were reminded at the beginning of the phone call follow-up interview, that just like in the survey, they did not have to answer any question that they did not want to. After the follow-up assessment, the principal investigator continued with safety planning, resource referral, and other interventions as appropriate, according to Chu et al. (2015).

Data Analysis

All analyses were conducted using IBM SPSS v. 21 (IBM Corp., 2012). Participant survey response sets were preliminarily analyzed and were considered invalid and excluded from analysis if they met either of the following: (1) had completion times less than 7 minutes (i.e., 2 minutes less than the fastest valid survey completion time in pilot testing), (2) had average straight-score/reverse-score inconsistencies across measures more than two standard deviations above the total sample mean, or (3) had average scale variances across measures more than two standard deviations below the total sample mean. These exclusions removed the outlying inattentive, random, and “straight-line” response sets from the data (n = 26).

Item skipping was rare (1.7% missing data), and due to the fact that some measures were contingent on previous responses (i.e., Past Experiences with Disclosure), statistical analyses for the study hypotheses were conducted using pairwise deletion for missing data. For the suicide risk screening measure (i.e., BSS-ND), data were technically complete (i.e., no data were missing due to item skipping), but the inclusion of nondisclosure responses prevented the ability to calculate total scale scores. To address this limitation, explicitly nondisclosed BSS-ND responses were temporarily treated as “missing,” and were recoded into one of the item’s three ordinal response options using 10 cycles of Markov chain Monte Carlo fully conditional specification multiple imputation. Data for imputation were drawn from the full survey data of participants who were presented with the full BSS-ND measure (n = 137), with 100 iterations between cycles. Exploratory tests showed no problems with convergence or proportion of missing data (i.e., 7.65% missing data including “nondisclosed” items; Bodner, 2008; Enders, 2010). Total BSS-ND scores for nondisclosers were then recomputed using imputed BSS-ND item data. In this way, predicted current suicide risk measure scores for nondisclosers were minimally affected by the number of measure items that were coded “not disclosed.”

Follow-up phone assessments to address reported suicide risk and collect qualitative reasons for nondisclosure were successful for 54% of participants identified as “at risk” (i.e., 65 out of 120). Participants who were unable to be reached for follow up were significantly older that participants whose follow-up data were included in analyses, ΔM = 6.51, t(118) = 2.11, p = .037 and were significantly less likely to have been queried about their suicide risk in the past, Δ $\hat{p}$ = −14.8%, χ² = 4.57, p = .033. However, no other significant differences were found between non–follow-up and follow-up participants, including by gender, χ² = 0.97, p = .617; race, χ² = 5.53, p = .237; population group, χ² = 5.60, p = .061; accuracy of past disclosure, ΔM = −0.21, t(94) = −0.84, p = .401; helpfulness of past disclosure, ΔM = −0.36, t(25) = −0.88, p = .390; predicted accuracy of future disclosure, ΔM = −0.25, t(94) = −1.12, p = .265; survey nondisclosure rates, Δ $\hat{p}$ = −2.8%, χ² = 0.10, p = .751; or imputed BSS-ND scores, ΔM = −0.57, t(118) = −0.41, p = .685. Free-response data in the survey and follow-up phone interviews were categorized and summarized according the study investigators’ judgment.

Measures

The following measures included in current analyses were drawn as a subset from the full assessment battery completed by study participants:

Beck Scale for Suicide Ideation–Nondisclosure (BSS-ND; Beck, Steer, & Ranieri, 1988)

The original BSS is composed of 19 scored items and 2 unscored items assessing presence and severity of current suicidal thoughts and behavior (e.g., wish to die, making of plans). Each item is scored on a 3-point ordinal scale, with a total score range from 0 to 38. Respondents either complete the first 7 or all 21 items depending on their initial responses. In this study, each item of the BSS-ND included an additional unscored choice option of “prefer not to disclose.” Deniers answered and denied all BSS-ND items, full disclosers answered all and elevated one or more BSS-ND items, and nondisclosers did not answer some or all of the BSS-ND items. Nondisclosed item scores were later predicted via Markov chain Monte Carlo multiple imputation. The imputed BSS-ND had good internal consistency (α = .850).

Suicidal Behaviors Questionnaire–Revised (SBQ-R; Osman et al., 2001)

The SBQ-R is a brief 4-item self-report measure of past suicidal behavior, including frequency and severity of past ideation and attempts, and self-prediction of future behavior. Responses are rated on item-specific ordinal scales, with a total score range from 3 to 18. Osman et al. (2001) found in a community sample that a total score greater than or equal to 7 effectively discriminated those at elevated suicide risk. The SBQ-R had adequate internal consistency (α = .750)

Barriers to Help Seeking Scale (BHSS; Mansfield, Addis, & Courtenay, 2005)

The BHSS is a 31-item measure designed to assess reasons for not seeking professional help for mental and physical health problems, including need for control and self-reliance, minimizing problem and resignation, concrete barriers and distrust of caregivers, privacy, and emotional control. In this study, The BHSS was shortened due to survey length constraints to 20 of the most intercorrelated items representative of each of the subscales. Therefore, BHSS subscales were not computed in this study, and all participants’ BHSS scores were converted to total mean scores, with a total score range from 0 to 4. The shortened BHSS measure had excellent internal consistency (α = .926)

General Help-Seeking Questionnaire (GHSQ; Wilson, Deane, Ciarrochi, & Rickwood, 2005)

The GHSQ is a modifiable measure, and in this study, consisted of the following probe, “If you were having thoughts about suicide, how likely is it that you would seek help from the following people?” followed by eight Likert-type items that each assess intention to seek help from a different source (e.g., psychologist/psychiatrist, religious leader, help/crisis line, nobody [reverse-coded]). Item scores were summed into a total measure score ranging from 8 to 48. With regard to suicidal thoughts, the GHSQ has demonstrated good internal and test–retest reliability, and modest but significant predictive validity of behavioral help seeking (Wilson et al., 2005). In this study, the GHSQ had adequate internal consistency (α = .798).

Beck Hopelessness Scale (BHS; Beck, Weissman, Lester, & Trexler, 1974)

The BHS is a 20-item, true–false measure that assesses the individual’s negative expectations for the future (e.g., “I might as well give up because I can’t make things better for myself”). The BHS has been shown to be a highly sensitive measure for predicting future suicidal behavior (Beck, Steer, Kovacs, & Garrison, 1985; McMillan, Gilbody, Beresford, & Neilly, 2007), and has been associated with nondisclosure of suicide risk (Podlogar et al., 2016). The BHS had excellent internal consistency (α = .928)

Brief Fear of Negative Evaluation Scale–Revised (BFNE-II; Carleton, McCreary, Norton, & Asmundson, 2006)

The BFNE-II is a 12-item Likert-type measure that assesses apprehension and distress arising from concerns about being judged disparagingly or hostilely by others (e.g., “I am concerned about other people’s opinions of me”), with a total score range from 0 to 48. The BFNE-II is different from the original BFNE by having all items straightforwardly worded, which improves its factor structure (Carleton et al., 2006). The BFNE-II was used in this study as a proxy measure of social desirability response bias, and had excellent internal consistency (α = .967)

Internalized Stigma of Mental Illness–10 Item (ISMI-10; Boyd, Otilingam, & Deforge, 2014)

The 10-item ISMI is a shortened form of the full 29-item ISMI (Ritsher, Otilingam, & Grajales, 2003), and contains the two strongest items from each of five factors: alienation, stereotype endorsement, discrimination experience, social withdrawal, and stigma resistance (reverse-coded). Items are Likert-type and averaged into a total score ranging from 1 to 4. The ISMI-10 has been shown to retain the essential properties of the ISMI-29, including adequate full-scale internal consistency, reliability, and external validity to measures of help seeking and stigma (Boyd et al., 2014). In this study, the ISMI-10 had good internal consistency (α = .813).

Interpersonal Needs Questionnaire (INQ; Van Orden, Cukrowicz, Witte, & Joiner, 2012)

The INQ is a 15-item self-report measure designed to assess the degree to which one experiences the psychological states of perceived burdensomeness (PB; e.g., “These days, the people in my life would be better off if I were gone,” total score range from 6 to 42) and thwarted belongingness (TB; e.g., “These days, I feel disconnected from other people,” total score range from 9 to 63), as described by the interpersonal theory of suicide (Van Orden et al., 2010). Both subscales have been shown to be independent predictors of suicidal ideation (Van Orden, Witte, Gordon, Bender, & Joiner, 2008), and have been associated with nondisclosure (Podlogar et al., 2016). Both subscales of the INQ had excellent internal consistency (α_PB = .938, α_TB = .914).

Suicidal Ambivalence (WTD/WTL; Harris et al., 2010)

This set of three items assesses frequency and intensity of internal suicidal debates. The first item asks, “Have you ever had an internal debate (in your head) about whether to live or die?” (WTD/WTL) and is rated on a 7-point Likert-type scale. The other two items assess current wish to live and wish to die, identical to BSS Items 1 and 2, but are rated on Likert-type scale from 0 to 10. Both frequency of suicidal debates (Harris et al., 2010) and the quantitative difference between wish to die and wish to live have been found to be associated with suicidal behavior (Brown, Steer, Henriques, & Beck, 2005; Bryan, Rudd, Peterson, Young-McCaughan, & Wertenberger, 2016).

Past Experiences With Disclosure (ESID; Hom et al., 2017)

This interview measure consists of eight modules that each assess whether one was asked about having thoughts of suicide from a different source (e.g., psychologist or psychiatrist, religious leader, friend, family, etc.). Endorsed modules are then followed up with questions assessing the participants’ disclosure experience, including accuracy of past risk disclosure (AcPst; i.e., “to what extent did you accurately express your current thoughts of suicide to this individual”), helpfulness of responses to past disclosure (HelpPst; i.e., “to what extent was their reply helpful vs. hurtful?”), and predicted accuracy of future reporting (AcFtr; i.e., “After this experience, how likely would you be to be honest if this individual asked you again if you were having thoughts of suicide?”). Each item is scored on a Likert-type scale from 1 to 5 and followed with space for free-response explanations. In this study, scores across all endorsed modules were averaged for overall past and future disclosure accuracy and helpfulness of responses. Free-response content was categorized and summarized by the study investigators. Most participants (79.6%) reported past experiences with suicide risk querying and disclosure, including 99% of the LGBTQ young adults, 88% of the military service members and veterans, and 64% of the men older than 50 years.

Results

Prevalence of Nondisclosure

As shown in Table 2, nondisclosure in suicide risk screening was indeed represented among military service members and veterans (12.6%), men older than 50 years (13.9%), and LGBTQ young adults (12.9%) who scored above the 70th percentile on the DASS-21. Furthermore, among the 240 participants who reported some level of current suicide risk, the overall rate of explicit nondisclosure was one in four. This same rate of nondisclosure was also present among the participants who met our follow-up criteria of not denying current suicide intent (i.e., 30 nondisclosers out of 120 flagged). Nondisclosure was highly specific to participants who indicated elevated current suicide risk (i.e., imputed BSS-ND scores above zero). Specifically, 57 of the 61 nondisclosers (93.4%) indicated elevated current suicide risk in their explicit report, and conversely, 222 of the 226 participants who otherwise did not indicate any suicide risk (98.2%) did not selectively disclose. The plurality of participants who selectively disclosed on the BSS-ND did so for only one item; however, the number of nondisclosed items ranged from 1 (n = 15, 28.8%) to 19 (n = 2, 3.8%), M = 4.21, SD = 3.89. Rates of nondisclosure by item are presented in Table 3. Overall, the most common items to be nondisclosed were BSS Item 11 (i.e., whether the reason for suicide is for influencing others, solving problems, or escape; 16.1%), followed by BSS Item 19 (i.e., whether one has before hidden suicidal desire; 15.3%) and BSS Item 13 (i.e., availability of method; 14.6%). The items with the greatest likelihood for nondisclosure relative to likelihood for full disclosure (i.e., greatest proportions) were BSS Item 7 (i.e., frequency of ideation; $\hat{p}$ = 0.39), BSS Item 18 (i.e., whether completed final arrangements; $\hat{p}$ = 0.38), and BSS Item 15 (i.e., expectancy to make a suicide attempt; $\hat{p}$ = 0.35). For these three items, more than one in three participants who did not deny risk chose “prefer not to disclose.”

Table 3.

Item-Level Counts of Nondisclosure.

Item no.	Item content	Non D		Full D		Den		Non D/Full D	95% CI
Item no.	Item content	n	%	n	%	n	%	Non D/Full D	95% CI
BSS 11	Reason for Suicide	22	16.1	108	78.8	7	5.1	.20	[.14, .27]
BSS 19	Whether Hidden Desire	21	15.3	81	59.1	35	25.5	.26	[.17, .34]
BSS 13	Availability of Method	20	14.6	76	55.5	41	29.9	.26	[.18, .35]
BSS 14	Courage to Kill Self	16	11.7	76	55.5	45	32.8	.21	[.13, .29]
BSS 15	Expectancy to Make Attempt	14	10.2	40	29.2	83	60.6	.35	[.22, .48]
BSS 4	Desire to Kill Self	14	3.1	87	19.2	352	77.7	.16	[.09, .23]
BSS 7	Frequency of Ideation	13	9.5	33	24.1	91	66.4	.39	[.25, .54]
BSS 6	Duration of Ideation	12	8.8	44	32.1	81	59.1	.27	[.16, .39]
BSS 20	Past Suicide Attempts	12	2.6	93	20.5	348	76.8	.13	[.06, .19]
BSS 18	Completed Final Arrangements	11	8.0	29	21.2	97	70.8	.38	[.23, .53]
BSS 10	Deterrents to Attempt	10	7.3	62	45.3	65	47.4	.16	[.08, .25]
BSS 12	Specificity of Planning	9	6.6	61	44.5	67	48.9	.15	[.06, .23]
BSS 8	Attitudes toward Ideation	9	6.6	83	60.6	45	32.8	.11	[.04, .17]
BSS 5	Desire to Save Self	9	2.0	84	18.5	360	79.5	.11	[.04, .17]
BSS 16	Actual Preparation	8	5.8	24	17.5	105	76.6	.33	[.17, .50]
BSS 9	Control over Action	8	5.8	33	24.1	96	70.1	.24	[.11, .37]
BSS 2	Wish to Die	8	1.8	129	28.5	316	69.8	.06	[.02, .10]
BSS 1	Wish to Live	7	1.5	86	19.0	360	79.5	.08	[.03, .14]
BSS 17	Written Suicide Note	4	2.9	15	10.9	118	86.1	.27	[.07, .47]
BSS 3	Reasons for Living vs. Dying	4	0.9	82	18.1	367	81.0	.05	[.00, .09]
BSS 21	Wish to Die During Last Attempt	3	3.2	69	74.2	21	22.6	.04	[.00, .09]

Note. BSS = Beck Scale for Suicide Ideation; Non D = nondisclosers (selected “prefer not to disclose” for item); Full D = full disclosers (elevated item); Den = deniers (did not elevate item); Non D/Full D = proportion of nondisclosers relative to full disclosers, with 95% confidence intervals (CI).

Statistical Correlates and Reasons Given for Nondisclosure

As shown in Table 4, analysis of variance tests indicated that nondisclosure, relative to explicit suicide risk denial, was significantly associated with more severe scores across nearly all hypothesized predictors, with the exception of fear of negative evaluation (i.e., social desirability). However, only accuracy of past disclosure, predicted accuracy of future disclosure, and past unhelpfulness of reactions to disclosure significantly differentiated nondisclosers from full disclosers of suicide risk. Perceived burdensomeness, internalized stigma, and suicidal ambivalence did not predict nondisclosure relative to full disclosure of suicide risk. Also, imputed BSS-ND scores were not significantly different between full disclosers and nondisclosers. Significance trends within demographic subsamples were concordant with the total sample, with the exception that among LGBTQ young adults, hopelessness was significantly higher among nondisclosers, M = 10.35, SE = 1.23, than full disclosers, M = 6.77, SE = 0.81, t(56) = 2.43, p = .016.

Table 4.

Analysis of Variance Tests Across Disclosure Levels.

			Den (n = 222)				Full D (n = 179)				Non D (n = 61)
	Welch	df	M	SE	t _{Non D}	p	M	SE	t _{Non D}	p	M	SE
BSS-ND	—	—	0.00	0.00	−10.01***	<.001	6.68	0.58	−1.13	.262	7.77	0.78
SBQ-R	143.42***	145.64	5.15	0.14	−7.74***	<.001	9.37	0.22	1.11	.269	8.82	0.45
BHSS	9.08***	168.34	1.77	0.05	−3.82***	<.001	2.00	0.06	−1.66^†	.099	2.18	0.10
GHSQ	28.44***	148.74	30.32	0.57	5.56***	<.001	24.77	0.63	1.36	.177	22.93	1.20
BHS	62.87***	150.30	5.51	0.36	−7.60***	<.001	11.44	0.47	−0.33	.740	11.73	0.74
BFNE-II	1.85	162.97	25.02	0.88	−1.92^†	.059	26.06	1.11	−1.33	.188	28.87	1.80
ISMI-10	26.32***	154.58	1.79	0.03	−5.08***	<.001	2.11	0.04	−1.19	.236	2.21	0.08
TB	60.39***	170.21	26.55	0.81	−7.37***	<.001	38.66	0.87	0.14	.891	38.43	1.39
PB	56.61***	142.00	9.31	0.37	−5.82***	<.001	17.11	0.72	0.23	.819	16.78	1.23
WTD-freq	91.21***	151.37	1.99	0.08	−6.68***	<.001	3.77	0.11	0.91	.366	3.54	0.22
WTD-diff	106.13***	134.59	−9.13	0.12	−8.34***	<.001	−4.55	0.34	0.08	.933	−4.61	0.53
AcPst	14.61***	114.55	3.98	0.10	5.33***	<.001	3.60	0.10	3.54**	.001	2.84	0.19
HelpPst	5.64**	51.85	3.93	0.11	3.31**	.002	3.85	0.11	2.90**	.007	3.25	0.18
AcFtr	16.49***	113.40	4.28	0.08	5.71***	<.001	3.98	0.09	4.03***	<.001	3.24	0.16

Note. BSS-ND = imputed BSS-ND score; SBQ-R = Suicidal Behaviors Questionnaire–Revised; BHSS = Barriers to Help Seeking Scale–total mean score; GHSQ = General Help-Seeking Questionnaire; BHS = Beck Hopelessness Scale; BFNE-II = Brief Fear of Negative Evaluation Scale–Revised; ISMI-10 = Internalized Stigma of Mental Illness Scale—10 Item; TB = Thwarted Belongingness; PB = Perceived Burdensomeness; WTD-freq = frequency of suicidal ambivalence; WTD-diff = wish to die minus wish to live; AcPst = accuracy of past risk disclosure; HelpPst = helpfulness of responses to past risk disclosure; AcFtr = predicted accuracy of future risk reporting; df = degree of freedom. Welch t tests were used.

†

p < .10. *p < .05. **p < .01. ***p < .001.

Nondisclosure was significantly correlated with severity of explicitly reported suicide risk via BSS-ND score, r_PB = 0.25, p < .001. This association conflated the relationships between nondisclosure and certain hypothesized predictors that were also associated with suicide risk (e.g., BHS, PB, WTD). After controlling for these intercorrelations, nondisclosure was significantly associated in partial correlations only with inaccuracy of past disclosure, pr = 0.22, p < .001, predicted future inaccuracy of disclosure, pr = 0.24, p < .001, unhelpfulness of reactions to disclosure, pr = 0.20, p = .023, and was negatively associated with help seeking (i.e., GHSQ), pr = −0.11, p = .018. Partial correlations in demographic subsamples showed the same trends, with the exception that among LGBTQ young adults, nondisclosure was also significantly associated with hopelessness (i.e., BHS), pr = 0.26, p = .003. No other significant partial correlations were found between nondisclosure and indicators of suicide risk (e.g., SBQ-R, TB, PB, etc.), above and beyond shared associations with explicit risk report.

As shown in Table 5, the most frequent free-response reasons participants gave for using nondisclosure included “to avoid consequences” (n = 14), “if felt dismissed” (n = 10), and “if the language does not capture my feelings” (n = 9). The most frequent reasons participants gave for providing full disclosure included “having a trusting relationship [with the screener]” (n = 7), “if felt listened to” (n = 6), and “if provided direction without force” (n = 6).

Table 5.

Qualitative Categories of Reported Reasons for Nondisclosure and Full Disclosure.

Reasons for nondisclosure	n	Reasons for full disclosure	n
To avoid consequences (e.g., will call police, hospitalize, block me, disrupt my business, discharge from military service, stigmatizing)	14	Trusting of relationship (e.g., good rapport, long known, lived experience, face to face)	7
If felt dismissed (e.g., accused of attention seeking, intervention not followed through, called “crazy,” told to stop)	10	If felt listened to (e.g., screener made effort to understand, took their time, showed sincerity)	6
Language does not capture my feelings (e.g., can’t express, unsure, did not want to lie)	9	If provided direction without force (e.g., treated “professionally, respectably,” “off the record,” always in control)	6
Felt it was not serious enough (e.g., would not actually attempt, under control, want to keep attention on other problems)	7	Felt it was necessary (e.g., serious problem, needed to relieve a “weight”)	5
Emotionally draining (e.g., uncomfortable, triggering, “don’t want to go there,” shameful)	5	I am an honest/truthful person	5
Mistrusting of relationship	4	To provide reassurance, destigmatize	4
Desire privacy	4	If can say in my own words	4
Hopeless (e.g., past treatment was unhelpful, no need to try)	4	Hopeful (e.g., past treatment was helpful, know that it can be better)	4
I always just say no	2	Personal experience with another’s disclosure	1

Note. Participant responses were recorded in free-response sections of the “Past Experiences with Suicidal Ideation Disclosure” measure (ESID), as well as in follow-up phone interviews. Responses similar to “I don’t know” were not included.

Discussion

The purpose of this study was to investigate nondisclosure in explicit suicide risk screening among three higher suicide-risk demographic groups, specifically military service members and veterans, men older than 50 years, and LGBTQ young adults. This was tested by including the response choice “prefer not to disclose” for each item of the suicide risk screening measure used in these groups (i.e., BSS-ND), and by investigating frequencies, quantitative, and qualitative predictors of this choice.

Results indicated that nondisclosure was represented across all three higher suicide-risk demographic groups included in analyses and was specific to individuals who reported at least some suicide risk. Namely, one quarter of the participants who explicitly indicated suicide risk also opted to be withholding in their disclosure, while more than 98% of risk-denying participants did not choose such an option. Furthermore, despite significant demographic differences and explicit suicide risk differences across study groups, rates of nondisclosure did not significantly differ. These results suggest that nondisclosure may be a somewhat common response in reaction to questions about suicide risk among those at higher risk—regardless of age, gender, or sexual orientation.

It is important to note that relative to past findings that simply allowed for item skipping (Podlogar et al., 2016), explicitly allowing for nondisclosure did not necessarily increase the proportion of missing data among participants who did not deny risk, and also did not decrease and only very minimally increased the number of participants identified for follow up, relative to reliance on their self-reported information (i.e., 4 of the 226 participants who otherwise denied risk). In other words, explicitly allowing for nondisclosure negligibly affected screening decisions. However, our results suggest that allowing for nondisclosure may add information in follow-up assessments regarding possible process factors for the assessor to address (e.g., mistrust, past experience of negative or unhelpful reactions, dislike of current screening terms), and may also reduce the likelihood for assessor uncertainty and respondent surprise or defensiveness regarding discussing the potential accuracy of responses in follow-up assessment. This transparency and opportunity for information seeking and trust building has the potential to improve both the clinical relationship and ultimately the validity of the follow-up risk assessment, with minimal cost to screening specificity. Conversely, if we had used forced responding in our screening measure, one quarter of our follow-up assessments would have relied on screening information that would have been inaccurate for one reason or another.

Nearly all of the hypothesized mechanisms (e.g., lower help seeking, internalized stigma, perceived burdensomeness, suicidal ambivalence) were significantly associated with nondisclosure relative to risk denial. However, this was likely due to significant intercorrelations between these factors, nondisclosure, and explicitly reported suicide risk. The association between nondisclosure and reported suicide risk suggests that respondents who report current suicide risk are also more likely to withhold some information—hence the crux of self-reported suicide risk screening. Accounting for this effect, one’s past experiences with disclosure, resistance to help seeking, and report of whether he or she selectively discloses were the only significant predictors of nondisclosure. Therefore, we did not find evidence in this study to support significant incremental utility of using explicit nondisclosure data in screening to predict suicide risk. However, we do not wish to rule out this possibility, for our models were limited to examining simple linear relationships with other self-reported cross-sectional risk indicators. Future research using complex machine learning algorithms and longitudinal behavioral outcomes may indeed find a predictive effect of nondisclosure, and may be strengthened by the inclusion of such response data in suicide risk screening.

Simply disagreeable item language was often given as a reason for nondisclosure, and likely contributed to the high nondisclosure rate for BSS Item 11 (i.e., reasons for suicidal ideation). Content of the other most commonly nondisclosed items supported the qualitative finding that avoidance of potential consequences and other anticipated or past-experienced unhelpful reactions by an assessor may be drivers for nondisclosure that are in need of further investigation. Screening items related to more proximal indicators of risk (e.g., frequency of ideation, completion of final arrangements, expectancy to make an attempt, actual preparation) were the most likely to be nondisclosed relative to fully disclosed. In a clinical situation, nondisclosure of such items is likely to be highly distressing for the assessor. However, the results of this study suggest that nondisclosure does not necessarily indicate higher suicide risk than someone who would have fully disclosed answers to all items, but instead is indicative of perceived unhelpfulness or undesirability of anticipated reactions.

Of course, avoiding presumed intervention responses is likely to increase one’s suicide risk. However, addressing nondisclosure in suicide risk screening by forcing a response, for instance, to determine whether the respondent is above or below a certain cutoff score, may not be the most therapeutic reaction. Instead, simply defaulting to follow up with respondents who selectively disclose critical screening items while providing trust, empathy, and open-ended support may be the most beneficial to the patient, and also may encourage full disclosure in future risk assessments. This philosophy is similar to those embraced by Dialectical Behavioral Therapy (see Coyle, Shaver, & Linehan, 2018) and the Collaborative Assessment and Management of Suicidality (Jobes, 2016), which emphasize a patient-centered approach to suicide risk assessment and management. For respondents who selectively disclose in risk screening, potential interventions to promote full disclosure in follow-up assessment may include (1) allowing for sufficient time and attention paid to inviting respondents to describe their risk in their own words; (2) providing hope, reassurance, and de-stigmatization before conducting risk assessment; and (3) affirming the participant’s agency in outcomes as best as possible. Notably, in qualitative responses, only two participants reported that they will always selectively disclose if given the choice, whereas all other participants indicated modifiable reasons for nondisclosure.

Limitations of this study include its restrictive inclusion and exclusion requirements, therefore potentially biasing the findings to be representative only of a certain type of person within each target population group. For example, participants in this study were limited to those who were computer/Internet literate and were willing to provide their phone number, and may have been more open and trusting than their counterparts in general. Second, the results were cross-sectional and dependent on self-reported and retrospective measures. Good internal consistency reliabilities, response set validity checks, and convergent intercorrelations between measures provide support for the validity of the self-reported data. However, prospective study designs are needed that assess the actual predictiveness of nondisclosure compared with forced response, as well as using other forms of measurement that may better capture selective disclosure beyond explicit nondisclosure. Third, we only investigated the specific nondisclosure choice of “prefer not to disclose,” which may be better at measuring desire for selective disclosure than at measuring self-uncertainty and temporal fluctuation/ambivalence. Other reasonable nondisclosure response choices that are perhaps more relevant to these latter barriers for accurate risk disclosure and that are worthy of investigation include “I don’t know,” “uncertain,” “it’s complicated,” or “none of the above.” Fourth, we did not use formal thematic analysis by multiple independent raters to code our qualitative data, and we did not assess for saturation. Although our qualitative findings offer support for the canonical literature and are an informative complement to our quantitative results, they should be interpreted as worthwhile hypotheses for future, more rigorous investigations. Last, because of the online availability and anonymity of this compensated study, faked or repeated results cannot be completely ruled out. However, phone number verification was a highly effective method for screening out computer bots and other survey scammers in pilot testing—although at the cost of creating selection effects mentioned above. Relatedly, due to its national online availability, participants likely completed the survey and potential follow-up interview at different points during the 24-hour day, depending on their local time zone. Whether a participant completed the study at 2:00 p.m. or 2:00 a.m. local time may have affected results; unfortunately, however, these data were not collected.

In conclusion, nondisclosure in suicide risk screening—whether explicitly reported or not—is likely often occurring among those at higher suicide risk. Nondisclosure in suicide risk screenings may be driven by many predictors of suicide risk itself, as well as by mistrust, avoidance of stigmatized consequences, and not wanting to be misunderstood. Suicide prevention research and assessment may be incrementally advanced by recording and analyzing this behavior. Further research is also needed on reducing stigma against reporting suicidal thoughts and behaviors, and on identifying ethically acceptable reactions that maximize the patient’s control over suicide risk disclosure.

Footnotes

Acknowledgements

The authors wish to acknowledge the significant contributions of Drs. E. Ashby Plant, Natalie Sachs-Ericsson, N. Brad Schmidt, John Taylor, and Peter Gutierrez.

Authors’ Note

The views in this article are those of the authors and do not necessarily represent the official policy or position of the Military Suicide Research Consortium, Department of Defense, Department of Veterans Affairs or the U.S. government.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article is based on data used in a doctoral dissertation and was supported in part by a grant from the American Psychological Foundation, and by a grant from the Military Suicide Research Consortium, an effort supported by the Office of the Assistant Secretary of Defense for Health Affairs under Award Nos. W81XWH-10-2-0181, W81XWH-10-2-0178. Writing of this article was supported by the Rocky Mountain MIRECC for Suicide Prevention and by the Office of Academic Affiliations, Advanced Fellowship Program in Mental Illness Research and Treatment, Department of Veterans Affairs.

ORCID iD

Matthew C. Podlogar

References

Beck

A. T.

Steer

R. A.

Kovacs

Garrison

(1985). Hopelessness and eventual suicide: A 10-year prospective study of patients hospitalized with suicidal ideation. American Journal of Psychiatry, 142, 559-563. doi:10.1176/ajp.142.5.559

Beck

A. T.

Steer

R. A.

Ranieri

W. F.

(1988). Scale for Suicide Ideation: Psychometric properties of a self-report version. Journal of Clinical Psychology, 44, 499-505.

Beck

A. T.

Weissman

Lester

Trexler

(1974). The measurement of pessimism: The Hopelessness Scale. Journal of Consulting and Clinical Psychology, 42, 861-865.

Bodner

T. E.

(2008). What improves with increased missing data imputations? Structural Equation Modeling, 15, 651-675. doi:10.1080/10705510802339072

Boyd

J. E.

Otilingam

P. G.

Deforge

B. R.

(2014). Brief version of the Internalized Stigma of Mental Illness (ISMI) scale: Psychometric properties and relationship to depression, self esteem, recovery orientation, empowerment, and perceived devaluation and discrimination. Psychiatric Rehabilitation Journal, 37, 17-23. doi:10.1037/prj0000035

Brener

Billy

Grady

(2003). Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: Evidence from the scientific literature. Journal of Adolescent Health, 33, 436-457.

Brown

G. K.

(2000). A review of suicide assessment measures for intervention research with adults and older adults. Retrieved from https://www.sprc.org/sites/default/files/migrate/library/BrownReviewAssessmentMeasuresAdultsOlderAdults.pdf

Brown

G. K.

Steer

R. A.

Henriques

G. R.

Beck

A. T.

(2005). The internal struggle between the wish to die and the wish to live: A risk factor for suicide. American Journal of Psychiatry, 162, 1977-1979. doi:10.1176/appi.ajp.162.10.1977

Bryan

C. J.

Rudd

M. D.

(2006). Advances in the assessment of suicide risk. Journal of Clinical Psychology, 62, 185-200.

10.

Bryan

C. J.

Rudd

M. D.

Peterson

A. L.

Young-McCaughan

Wertenberger

E. G.

(2016). The ebb and flow of the wish to live and the wish to die among suicidal military personnel. Journal of Affective Disorders, 202, 58-66. doi:10.1016/j.jad.2016.05.049

11.

Busch

K. A.

Fawcett

Jacobs

D. G.

(2003). Clinical correlates of inpatient suicide. Journal of Clinical Psychiatry, 64, 14-19. doi:10.4088/JCP.v64n0105

12.

Canetto

S. S.

(2017). Suicide: Why are older men so vulnerable? Men and Masculinities, 20, 49-70. doi:10.1177/1097184X15613832

13.

Carleton

R. N.

McCreary

D. R.

Norton

P. J.

Asmundson

G. J. G.

(2006). Brief Fear of Negative Evaluation Scale—Revised. Depression & Anxiety, 23, 297-303. doi:10.1002/da.20142

14.

Cha

C. B.

Najmi

Park

J. M.

Finn

C. T.

Nock

M. K.

(2010). Attentional bias toward suicide-related stimuli predicts suicidal behavior. Journal of Abnormal Psychology, 119, 616-622. doi:10.1037/a0019710

15.

Chu

Klein

K. M.

Buchman-Schmitt

J. M.

Hom

M. A.

Hagan

C. R.

Joiner

T. E.

(2015). Routinized assessment of suicide risk in clinical practice: An empirically informed update. Journal of Clinical Psychology, 71, 1186-1200. doi:10.1002/jclp.22210

16.

Coyle

T. N.

Shaver

J. A.

Linehan

M. M.

(2018). On the potential for iatrogenic effects of psychiatric crisis services: The example of dialectical behavior therapy for adult women with borderline personality disorder. Journal of Consulting and Clinical Psychology, 86, 116-124. doi:10.1037/ccp0000275

17.

Crosby

A. E.

Ortega

Melanson

(2011). Self-directed violence surveillance: Uniform definitions and recommended data elements (Version 1.0). Retrieved from www.cdc.gov/violenceprevention/pdf/self-directed-violence-a.pdf

18.

Denneson

L. M.

Kovas

A. E.

Britton

P. C.

Kaplan

M. S.

McFarland

B. H.

Dobscha

S. K.

(2016). Suicide risk documented during veterans’ last Veterans Affairs health care contacts prior to suicide. Suicide and Life-Threatening Behavior, 46, 363-374. doi:10.1111/sltb.12226

19.

Drum

D. J.

Brownson

Burton Denmark

Smith

S. E.

(2009). New data on the nature of suicidal crises in college students: Shifting the paradigm. Professional Psychology: Research and Practice, 40, 213-222. doi:10.1037/a0014465

20.

Encrenaz

Kovess-Masféty

Gilbert

Galéra

Lagarde

Mishara

Messiah

(2012). Lifetime risk of suicidal behaviors and communication to a health professional about suicidal ideation. Crisis, 33, 127-136. doi:10.1027/0227-5910/a000113

21.

Enders

C. K.

(2010). Applied missing data analysis. New York, NY: Guilford Press.

22.

Frey

L. M.

Fulginiti

Lezine

Cerel

(2018). The decision-making process for disclosing suicidal ideation and behavior to family and friends. Family Relations, 67, 414-427. doi:10.1111/fare.12315

23.

Fulginiti

Pahwa

Frey

L. M.

Rice

Brekke

J. S.

(2016). What factors influence the decision to share suicidal thoughts? A multilevel social network analysis of disclosure among individuals with serious mental illness. Suicide and Life-Threatening Behavior, 46, 398-412. doi:10.1111/sltb.12224

24.

Glenn

Monteith

(2014). New measures of mental state and behavior based on data collected from sensors, smartphones, and the Internet. Current Psychiatry Reports, 16(12), 523. doi:10.1007/s11920-014-0523-3

25.

Haas

A. P.

Eliason

Mays

V. M.

Mathy

R. M.

Cochran

S. D.

D’Augelli

A. R.

. . . Clayton

P. J.

(2010). Suicide and suicide risk in lesbian, gay, bisexual, and transgender populations: Review and recommendations. Journal of Homosexuality, 58, 10-51. doi:10.1080/00918369.2011.534038

26.

Harris

K. M.

McLean

J. P.

Sheffield

Jobes

D. A.

(2010). The internal suicide debate hypothesis: Exploring the life versus death struggle. Suicide and Life-Threatening Behavior, 40, 181-192. doi:10.1521/suli.2010.40.2.181

27.

Henry

J. D.

Crawford

J. R.

(2005). The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 44, 227-239. doi:10.1348/014466505X29657

28.

Hom

M. A.

Stanley

I. H.

Joiner

T. E.

(2015). Evaluating factors and interventions that influence help-seeking and mental health service utilization among suicidal individuals: A review of the literature. Clinical Psychology Review, 40, 28-39. doi:10.1016/j.cpr.2015.05.006

29.

Hom

M. A.

Stanley

I. H.

Podlogar

M. C.

Joiner

T. E.

(2017). Are you having thoughts of suicide? Examining experiences with disclosing and denying suicidal ideation. Journal of Clinical Psychology, 73, 1382-1392. doi:10.1002/jclp.22440

30.

IBM Corp. (2012). IBM SPSS Statistics for Windows. Armonk, NY: Author.

31.

Ilic

Reinecke

Bohner

Röttgers

H.-O.

Beblo

Driessen

. . . Corrigan

P. W.

(2014). Managing a stigmatized identity: Evidence from a longitudinal analysis about people with mental illness. Journal of Applied Social Psychology, 44, 464-480. doi:10.1111/jasp.12239

32.

Jobes

D. A.

(2016). Managing suicidal risk: A collaborative approach (2nd ed.). New York, NY: Guilford Press.

33.

Joiner

T. E.

Walker

R. L.

Rudd

M. D.

Jobes

D. A.

(1999). Scientizing and routinizing the assessment of suicidality in outpatient practice. Professional Psychology: Research and Practice, 30, 447-453. doi:10.1037//0735-7028.30.5.447

34.

Kay

W. R.

Boggess

Selvavel

McMahon

M. F.

(2001). The use of targeted incentives to reluctant respondents on response rate and data quality. Proceedings of the Survey Research Methods Section, American Statistical Association. Retrieved from https://nces.ed.gov/FCSM/pdf/IHSNG_wkayasa2000.pdf

35.

Kleiman

E. M.

Turner

B. J.

Fedor

Beale

E. E.

Huffman

J. C.

Nock

M. K.

(2017). Examination of real-time fluctuations in suicidal ideation and its risk factors: Results from two ecological momentary assessment studies. Journal of Abnormal Psychology, 126, 726-738. doi:10.1037/abn0000273

36.

LeFevre

(2014). Clinical guideline screening for suicide risk in adolescents, adults, and older adults in primary care. Annals of Internal Medicine, 160, 719-726.

37.

Lovibond

S. H.

Lovibond

P. F.

(1995). Manual for the Depression Anxiety Stress Scales (2nd ed.). Sydney, New South Wales, Australia: Psychology Foundation.

38.

Luoma

J. B.

Martin

C. E.

Pearson

J. L.

(2002). Contact with mental health and primary care providers before suicide: A review of the evidence. American Journal of Psychiatry, 159, 909-916. doi:10.1176/appi.ajp.159.6.909

39.

Mann

J. J.

Apter

Bertolote

Beautrais

Currier

Haas

. . . Hendin

(2005). Suicide prevention strategies: A systematic review. JAMA Journal of the American Medical Association, 294, 2064-2074. doi:10.1001/jama.294.16.2064

40.

Mansfield

A. K.

Addis

M. E.

Courtenay

(2005). Measurement of men’s help seeking: Development and evaluation of the Barriers to Help Seeking Scale. Psychology of Men & Masculinity, 6, 95-108. doi:10.1037/1524-9220.6.2.95

41.

McMillan

Gilbody

Beresford

Neilly

(2007). Can we predict suicide and non-fatal self-harm with the Beck Hopelessness Scale? A meta-analysis. Psychological Medicine, 37, 769-778. doi:10.1017/S0033291706009664

42.

Niculescu

A. B.

Levey

D. F.

Phalen

P. L.

Le-Niculescu

Dainton

H. D.

Jain

. . . Salomon

D. R.

(2015). Understanding and predicting suicidality using a combined genomic and clinical risk assessment approach. Molecular Psychiatry, 20, 1266-1285. doi:10.1038/mp.2015.112

43.

Niederkrotenthaler

Reidenberg

D. J.

Till

Gould

M. S.

(2014). Increasing help-seeking and referrals for individuals at risk for suicide by decreasing stigma: The role of mass media. American Journal of Preventive Medicine, 47(3 Suppl. 2), S235-S243. doi:10.1016/j.amepre.2014.06.010

44.

Nock

M. K.

Park

J. M.

Finn

C. T.

Deliberto

T. L.

Dour

H. J.

Banaji

M. R.

(2010). Measuring the suicidal mind: Implicit cognition predicts suicidal behavior. Psychological Science, 21, 511-517. doi:10.1177/0956797610364762

45.

Osman

Bagge

C. L.

Gutierrez

P. M.

Konick

L. C.

Kopper

B. A.

Barrios

F. X.

(2001). The Suicidal Behaviors Questionnaire–Revised (SBQ-R): Validation with clinical and nonclinical samples. Assessment, 8, 443-454. doi:10.1177/107319110100800409

46.

Packman

W. L.

Pennuto

T. O. C.

Bongar

Orthwein

(2004). Legal issues of professional negligence in suicide cases. Behavioral Sciences & the Law, 22, 697-713. doi:10.1002/bsl.613

47.

Pearson

J. L.

Stanley

King

C. A.

Fisher

C. B.

(2001). Intervention research with persons at high risk for suicidality: Safety and ethical considerations. Journal of Clinical Psychiatry, 62(Suppl. 25), 17-26.

48.

Podlogar

M. C.

Rogers

M. L.

Chiurliza

Hom

M. A.

Tzoneva

Joiner

T. E.

(2016). Who are we missing? Nondisclosure in online suicide risk screening questionnaires. Psychological Assessment, 28, 963-974. doi:10.1037/pas0000242

49.

Range

L. M.

Knott

E. C.

(1997). Twenty suicide assessment instruments: Evaluation and recommendations. Death Studies, 21, 25-58. doi:10.1080/074811897202128

50.

Ribeiro

J. D.

Gutierrez

P. M.

Joiner

T. E.

Kessler

R. C.

Petukhova

M. V.

Sampson

N. A.

. . . Nock

M. K.

(2017). Health care contact and suicide risk documentation prior to suicide death: Results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Journal of Consulting and Clinical Psychology, 85, 403-408. doi:10.1037/ccp0000178

51.

Ritsher

J. B.

Otilingam

P. G.

Grajales

(2003). Internalized stigma of mental illness: Psychometric properties of a new measure. Psychiatry Research, 121, 31-49. doi:10.1016/j.psychres.2003.08.008

52.

Simon

G. E.

Johnson

Lawrence

J. M.

Rossom

R. C.

Ahmedani

Lynch

F. L.

. . . Shortreed

S. M.

(2018). Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records. American Journal of Psychiatry, 175, 951-960. doi:10.1176/appi.ajp.2018.17101167

53.

Suicide Prevention Resource Center. (2014). Suicide screening and assessment. Retrieved from https://www.sprc.org/resources-programs/suicide-screening-and-assessment

54.

Suicide Prevention Resource Center. (2015). Zero suicide. Retrieved from http://zerosuicide.sprc.org/

55.

Teitcher

J. E. F.

Bockting

W. O.

Bauermeister

J. A.

Hoefer

C. J.

Miner

M. H.

Klitzman

R. L.

(2015). Detecting, preventing, and responding to “fraudsters” in Internet research: Ethics and tradeoffs. Journal of Law, Medicine & Ethics, 43, 116-133. doi:10.1111/jlme.12200

56.

Tourangeau

Yan

(2007). Sensitive questions in surveys. Psychological Bulletin, 133, 859-883.

57.

Van Orden

K. A.

Cukrowicz

K. C.

Witte

T. K.

Joiner

T. E.

(2012). Thwarted belongingness and perceived burdensomeness: Construct validity and psychometric properties of the Interpersonal Needs Questionnaire. Psychological Assessment, 24, 197-215. doi:10.1037/a0025358

58.

Van Orden

K. A.

Witte

T. K.

Cukrowicz

K. C.

Braithwaite

S. R.

Selby

E. A.

Joiner

T. E.

(2010). The interpersonal theory of suicide. Psychological Review, 117, 575-600. doi:10.1037/a0018697

59.

Van Orden

K. A.

Witte

T. K.

Gordon

K. H.

Bender

T. W.

Joiner

T. E.

(2008). Suicidal desire and the capability for suicide: Tests of the interpersonal-psychological theory of suicidal behavior among adults. Journal of Consulting and Clinical Psychology, 76, 72-83. doi:10.1037/0022-006X.76.1.72

60.

Vannoy

S. D.

Robins

L. S.

(2011). Suicide-related discussions with depressed primary care patients in the USA: Gender and quality gaps. A mixed methods analysis. BMJ Open, 1(2), 1-7. doi:10.1136/bmjopen-2011-000198

61.

Vogel

D. L.

Wade

N. G.

(2009). Stigma and help-seeking. The Psychologist, 22, 20-23.

62.

Wilson

C. J.

Deane

F. P.

Ciarrochi

Rickwood

(2005). Measuring help-seeking intentions: Properties of the General Help-Seeking Questionnaire. Canadian Journal of Counselling and Psychotherapy, 39, 15-28.