Measuring Suicide Risk Without Asking About Suicide? Psychache Scale Score Validation and Cut Score Development in a Military Sample

Abstract

This study evaluated the ability of scores on the self-report Psychache Scale (PAS) to subtly screen pre-existing suicide risk factors in a military population. For 525 Canadian military personnel, PAS normative data were established, and scores demonstrated high internal consistency reliability. Based on associations with a rationally constructed measure of suicidality, PAS scores possessed large effect sizes, convergent validity. The PAS also had significant diagnostic accuracy for identifying previous multiple suicide intent communications and previous multiple suicide attempts, behavioral risk factors for subsequent suicide. Preliminary cut scores for identifying at risk military personnel are provided.

Keywords

Suicidal behavior risk psychache mental pain cut scores military norms

Introduction

Suicide is a global health issue claiming over 700,000 lives annually (World Health Organization, 2021). Within the U.S. military, suicide is the second leading cause of mortality, representing 24.8% of fatalities, exceeded only by accidents (Mann & Fischer, 2019). In the Canadian Armed Forces (CAF), between 2002 and 2018, rates of suicide increased by over 40% (Boulos, 2021). VanTil et al. (2021) indicated that between 1975 and 2016 suicide risk for both male and female veterans consistently exceeded that of the Canadian general population. Further, male veterans overall had a 1.5 times higher risk of dying by suicide compared to the male Canadian general population and female veterans had a 2.0 times higher risk of dying by suicide compared to the female Canadian general population. They further reported that Canadian military trends are more consistent than those observed in the UK or the U.S. Overall, therefore, suicide constitutes a health issue both in the general population and within military contexts, and research to better identify those at risk is needed.

That said, identifying who will die by suicide is challenging (Franklin et al., 2017) because, although numerous risk factors have been specified, there is currently no single self-report scale whose scores can accurately predict, on an individual basis, who will attempt or die by suicide without producing excessive false positive predictions (Fowler, 2012). Psychological factors with empirical importance for suicide include depression and hopelessness (Troister et al., 2015), however, it is the construct of mental pain that recent meta-analyses (Ducasse et al., 2017; Verrocchio et al., 2016) have highlighted as a core component and predictor of suicidality. In articulating his model of suicide, Shneidman (1993), a pioneer of suicide research, termed the mental pain associated with suicide as “psychache” (p. 145), theoretically asserting that, based on his clinical experience, psychache causes suicide, and that all other factors only associate with suicide through psychache. Psychache is an individual difference variable consisting of overwhelming psychological pain that includes anguish, despair, grief, guilt, fear, and panic. It is a deeper, more primal, more savage mental pain than is found in depression or general distress (Campos et al., 2017). Although related to depression and hopelessness, the measurement of psychache is factor analytically distinct and its scores demonstrate discriminant validity relative to scores on measures of depression and hopelessness in the prediction of suicidality (DeLisle & Holden, 2009; Troister & Holden, 2013) suggesting that it can provide researchers and clinicians with unique information. Longitudinal studies have also supported the convergent validity and discriminant validity (relative to scores on measures of depression and hopelessness) of assessing psychache for predicting suicidal ideation in general and high-risk university students (Troister et al., 2013; Troister & Holden, 2012).

A promising measure of psychache is the Psychache Scale (PAS; Holden et al., 2001; see Table 1). The PAS is a brief (2–5 minute), subtle self-report measure. Importantly, the PAS contains no wording that directly references suicide. This can be a more acceptable approach with nonclinical populations than inquiring directly about suicidal behaviors, can ameliorate issues that some health professionals have in asking about suicide, and may overcome the concealment of suicidal urges in persons such as perfectionists (D’Agata & Holden, 2018; Flett et al., 2014; Friedman, 2006). Indeed, Shneidman (1994) suggested that 10% of individuals who die by suicide fail to display suicidal cues or mask their presentations, and Rudestam (1977) reported that, in interviewing persons having had a family member who died by suicide, almost 50% were surprised by the suicide. Further, Killian (2024) has indicated that inhibitions in responding to direct queries about suicide are based in shame, embarrassment, stigma, and fear of involuntary hospitalization.

Table 1.

The Psychache Scale (Holden et al., 2001). Copyright R. R. Holden and K. Mehta, 1998. Reproduced With Permission.

The following statements refer to your psychological pain, NOT your physical pain. By circling the appropriate number, please indicate how frequently each of the following occurs.
1 = never; 2 = sometimes; 3 = often; 4 = very often; 5 = always
1.	I feel psychological pain.	1	2	3	4	5
2.	I seem to ache inside.	1	2	3	4	5
3.	My psychological pain seems worse than any physical pain.	1	2	3	4	5
4.	My pain makes me want to scream.	1	2	3	4	5
5.	My pain makes my life seem dark.	1	2	3	4	5
6.	I can’t understand why I suffer.	1	2	3	4	5
7.	Psychologically, I feel terrible.	1	2	3	4	5
8.	I hurt because I feel empty.	1	2	3	4	5
9.	My soul aches.	1	2	3	4	5
Please continue this inventory using the following scale:
1 = strongly disagree; 2 = disagree; 3 = unsure; 4 = agree; 5 = strongly agree
10.	I can’t take my pain any more.	1	2	3	4	5
11.	Because of my pain, my situation is impossible.	1	2	3	4	5
12.	My pain is making me fall apart.	1	2	3	4	5
13.	My psychological pain affects everything I do.	1	2	3	4	5

Previous research with the PAS has shown that psychache scores outperform those of depression: in identifying suicide risk (Troister et al., 2015); in forecasting subsequent suicide attempts (Lambert et al., 2020); in associating with self-harm (Holden et al., 2001); and in mediating links between childhood trauma and self-harm (Holden et al., 2022). To date, the merits of the PAS scores as an alternative measure to assess suicide risk within a military population have not been investigated.

Aims and Hypotheses

The current study’s aims and contributions were to, within a military sample, establish PAS normative data, assess PAS score reliability, evaluate PAS score validity based on a criterion of suicidality, and develop PAS cut scores for indicating individuals at high risk for suicidal behavior. Our hypotheses were that PAS scale scores: 1) would have high internal consistency reliability, 2) would demonstrate appropriate levels of validity as evidenced by a large correlation with a suicidality criterion measure, and 3) would provide cut scores representing significant diagnostic accuracy.

Method

Participants

Participants were 525 Regular Force and Class B Reservist CAF members (407 men, 104 women, 13 other or declined to indicate) across elements (i.e., land, air, sea) and ranks. Mean age was 39.67 years (SD = 9.62). We employed stratified random sampling based on CAF members’ rank, element, and component. Specifically, we invited individuals via email from each element, and rank category, and component based on the overall representation in the CAF with the goal of obtaining a representative sample. Invitations were sent to 5000 individuals. Although 1234 participants initiated the study, only 694 completed data collection. Of those participants, 67 asked that their data be removed. Six participants were removed based on taking less than one third of the median completion time for participating in the study and 59 participants were removed for failing two validity check items. Finally, we removed 37 Class A and C Reservists from the final data set. Because Class B reservists are highly similar to Regular Force members, in that they are currently working full-time and are not currently deployed, they were retained in the final data set. In the analyzed sample, 86.3% were Regular Force and 13.3% were Primary Reserve Class B CAF members. For Regular Force personnel, our proportions matched within 3% for each CAF rank category except for Junior NCMs in the Army (18.3% in our sample, 30% of the CAF), which was slightly underrepresented.

Materials

The PAS (Holden et al., 2001) is a 13-item scale assessing Shneidman’s concept of psychache (e.g., “I can’t understand why I suffer”). Item responses (5-point ratings) sum to an overall score (between 13 and 65) of mental pain. Previous non-military research with student, community, and forensic samples attests to the PAS scores’ reliability and validity (Ducasse et al., 2017). For example, regarding internal consistency reliability, a scale score coefficient alpha of .94 has been reported both for undergraduates (Troister et al., 2015) and for incarcerated offenders (Pereira et al., 2010). As evidence for validity, PAS scores correlated .33 and .54 with suicide attempter status in undergraduate (D’Agata & Holden, 2018) and homeless men (Patterson & Holden, 2012) samples, respectively.

The Holden Suicidality Scale (HSS; Holden, 2021) is a rationally constructed, face valid, self-report measure (see Table 2) comprised of eight items (three response options per item) that assesses previous suicide ideation, suicidal communications, and suicide attempts, and predicted future suicidal behavior. Item responses are summed to total scores ranging between 8 and 24. Both past suicidal communications and previous attempts are historical behavior items. Because previous suicidal actions including communications (Osman et al., 2001; Robins et al., 1959) and multiple suicidal behaviors (Rudd et al., 1996) are primary suicide risk factors, the presence of either multiple lifetime communications (Item 3) or multiple lifetime attempts (Item 5) indicates elevated risk.

Table 2.

The Holden Suicidality Scale (Holden, 2021). Copyright R. R. Holden, 2021. Reproduced With Permission.

Please, for each question, circle the one response that best applies to you

1. At any point in your lifetime, have you thought about killing yourself? (circle only one)

Once

More than once

2. At any point in the past year, have you thought about killing yourself? (circle only one)

Once

More than once

3. At any point in your lifetime, have you told someone that you were going to kill yourself? (circle only one)

Once

More than once

4. At any point in the past year, have you told someone that you were going to kill yourself? (circle only one)

Once

More than once

5. At any point in your lifetime, have you attempted to kill yourself? (circle only one)

Once

More than once

6. At any point in the past year, have you attempted to kill yourself? (circle only one)

Once

More than once

7. In the future, how likely are you to attempt to kill yourself? (circle only one)

Unlikely

Unsure

Likely

8. In the future, if you attempt to kill yourself, how likely is it that you would die? (circle only one)

Unlikely

Unsure

Likely

Procedure

The data presented are a subset of a larger study which involved other measures; additional reports on the larger study are forthcoming. Participants were treated in accordance with American Psychological Association guidelines and were provided with debriefing including information regarding available counseling resources. Administration was standardized and online through a Qualtrics survey. Median participation time, including other measures, was 56.3 minutes. This research was approved by research ethics boards of Defence Research and Development Canada and a Canadian university. Participants provided informed consent, were not paid, and participation was anonymous.

Results

No data variable analyzed had more than one percent of cases missing. Thus, undertaken analyses used listwise deletion.

For normative data, the mean score on the PAS was 23.35 (SD = 11.87). In support of our first hypothesis, scale score internal consistency reliability was high (Cronbach’s α = .97; McDonald’s ω = .97). Further, for the PAS items, a diagonally weighted least squares confirmatory factor analysis offered support for a one-factor solution (RMSEA = .001, SRMR = .038, CFI = .999, TLI = .999) with all item factor loadings exceeding .71. For the HSS, the mean score was 10.04 (SD = 2.57) and scale score internal consistency reliability was acceptable (Cronbach’s α = .73; McDonald’s ω = .77).

Participant age was not related to scores on either the PAS, r(522) = −.06, p = .17, or HSS, r(522) = −.07, p = .11. Other demographic differences on PAS and HSS scores were also examined for sex, element, rank, and component using analysis of variance (Table 3). Nonsignificant effects were found for sex, element, and component and, thus, these were not considered further. For rank, significant, albeit small effect size differences, were present for scores on the PAS, F(3, 520) = 3.21, p = .02, η² = .02, and on the HSS, F(3, 520) = 3.67, p = .01, η² = .02.

Table 3.

Psychache Scale (PAS) and Holden Suicidality Scale (HSS) Scores as a Function of Demographics.

		n	PAS				HSS
		n	Mean	SD	F-Ratio	η ²	Mean	SD	F-Ratio	η ²
Demographic
Reported sex	Men	407	23.33	11.93	0.06	.00	2.07	2.56	0.98	.00
	Women	104	23.59	12.04			1.77	2.46
	Other	14	24.26	9.40			2.64	3.20
Element	Land	251	22.94	11.49	0.38	.00	2.01	2.62	2.66	.01
	Air	194	23.90	12.28			1.81	2.24
	Sea	80	23.65	12.17			2.59	3.02
Rank	Private to master corporal, sailor 3rd class to master sailor	177	25.50	12.96	3.21*	.02	2.51	2.99	3.76*	.02
	Sergeant to chief warrant officer, petty officer 2nd class to chief petty officer 1st class	159	21.96	11.56			1.84	2.55
	Second lieutenant to captain, acting sub-lieutenant to lieutenant	110	23.36	10.77			1.86	2.09
	Major and above, lieutenant-commander and above	78	21.66	10.91			1.51	1.90
Component	Primary reserve	70	22.62	10.76	0.34	.00	1.67	2.23	1.54	.00
	Regular force	452	23.51	12.05			2.08	2.60

*p < .05.

Note. η² values ≥.01, .06, and .14 correspond to small, medium, and large effect sizes, respectively.

In support of our second hypothesis, PAS score convergent validity, based on correlating PAS and HSS scores and evaluated using bootstrapping (1000 samples) to obviate concerns about non-normality and heteroscedasticity, was .50, p < .001, a large effect size. When rank was covaried out, the correlation between PAS and HSS scores altered minimally to .49, p < .001.

The usefulness of the PAS for identifying high risk was evaluated through receiver operating characteristic (ROC) curves. This incorporates sensitivity (i.e., true positive rate) and 1-specificity (i.e., false alarm rate) and is accepted for optimizing cut scores (Akobeng, 2007). Informational value is summarized as an AUC (area under curve) that is compared with a .50 chance value. An AUC is the probability that a pair of cases chosen randomly from two underlying distributions will be correctly identified. AUCs of .50–.70, .70–.90, or over .90 are low, moderate, or high accuracy, respectively (Streiner & Cairney, 2007). Table 4 reports ROC diagnostic efficiency statistics for a variety of cut scores in identifying at risk individuals using the PAS.

Table 4.

Diagnostic Efficiency Statistics for PAS Cut Scores for Identifying At-Risk Cases.

Cut Score	At-Risk Indicator
	Lifetime Multiple Communications of a Suicide Attempt					Lifetime Multiple Suicide Attempts
	Sensitivity	Specificity	Youden’s Index	PPV	NPV	Sensitivity	Specificity	Youden’s Index	PPV	NPV
PAS ≥ 33	.73	.84	.57	.18	.98	.69	.83	.52	.09	.99
PAS ≥ 34	.73	.85	.58	.19	.98	.69	.83	.53	.09	.99
PAS ≥ 35	.73	.87	.60	.20	.98	.69	.85	.54	.10	.99
PAS ≥ 36	.73	.88	.61	.22	.98	.69	.86	.56	.10	.99
PAS ≥ 37	.69	.89	.58	.24	.98	.69	.87	.56	.11	.99
PAS ≥ 38	.69	.90	.59	.24	.98	.69	.88	.58	.12	.98
PAS ≥ 39	.69	.90	.59	.26	.98	.69	.89	.58	.13	.99
PAS ≥ 40	.65	.91	.57	.27	.98	.62	.90	.51	.13	.99
PAS ≥ 41	.65	.92	.57	.28	.98	.62	.90	.52	.13	.99
PAS ≥ 42	.62	.92	.54	.29	.98	.62	.91	.53	.14	.99
PAS ≥ 43	.58	.93	.50	.30	.98	.62	.92	.53	.15	.99

Note. Within an at-risk indicator, largest value of Youden’s Index is in bold.

PPV = positive predictive value; NPV = negative predictive value.

For our sample, responses to HSS Item 3 (communicating suicide intent to others) indicated 26 at risk individuals based on reporting multiple lifetime suicide communications and 497 as not at risk. PAS scores differed significantly between those identified as being at risk (M = 42.33, SD = 12.05) and those not at risk (M = 22.40, SD = 11.02), t(521) = 8.95, p < .001, with a large effect size (Cohen, 1992), Cohen’s d = 1.80.¹ ROC curve analysis for predicting at-risk status from PAS scores had an area under the curve of .884, p < .001, 95% CI [.833, .936], a moderate level of diagnostic accuracy that supported our third hypothesis. For this aspect of suicidality, the optimal (maximizing Youden’s index) PAS cut score was 36.5 (.731 sensitivity, .879 specificity).

For our sample, responses to HSS Item 5 (suicide attempt) indicated 13 at risk individuals based on reporting multiple lifetime suicide attempts and 510 as not at risk. PAS scores differed significantly between those identified as being at risk (M = 42.23, SD = 15.25) and those not at risk (M = 22.91, SD = 11.40), t(521) = 5.98, p < .001, with a large effect size, Cohen’s d = 1.68. ROC curve analysis for predicting this at-risk status from PAS scores indicated an area under the curve of .842, p < .001, 95% CI [.730, .955], a moderate level of diagnostic accuracy that again supported our third hypothesis. For indicating a lifetime multiple suicide attempts, the optimal PAS cut score (Youden’s index) was 39.5 (.692 sensitivity, .886 specificity).

Discussion

Overall, PAS scores provided a brief, reliable, valid and significantly accurate screen for persons in suicide at-risk groups, based on multiple occurrences of previous suicidal behavior. For specifying accuracy with AUCs, the PAS ranged from .842 to .884. AUCs and cut scores combining sensitivity and specificity that maximize Youden’s index are one method for indicating diagnostic efficiency. Of note, high sensitivity can be an index for ruling out a risk if an individual tests negative, and high specificity can be an index for ruling in a risk if a person tests positive. In some contexts, other indices such as positive predictive power, negative predictive power, and relative risk ratios could be preferred and, depending on circumstances, the costs (including monetary) of different screening errors (both false positives and false negatives) could be weighted and combined with prevalence rates into a utility analysis incorporating more than sensitivity and specificity.

The current study provides several innovative contributions to the field. First, this research demonstrates that PAS scores have validity for indicating, within the military, personnel who are at risk for suicidal behavior. Second, the current investigation provides normative data and cut scores for identifying at risk members of the armed forces. Third, in contrast to many suicide risk screens, PAS items contain no reference to suicide. Thus, the PAS provides an indirect, subtle assessment that may circumvent dissembling or concealment present among some individuals. Indeed, underreporting has been shown to attenuate the association between self-reported suicide ideation and other (e.g., clinical interview) measures of suicidality (Khazem et al., 2021). Furthermore, use of the PAS may overcome the reluctance some health caregivers have in directly asking about suicide as well as provides a means of assessing an individual’s risk without necessarily bringing up past incidents of suicidal thoughts or behaviours, that may be upsetting or triggering. In addition to these contributions, our findings on the role of psychache provide additional empirical, non-experimental support for the theoretical model of Shneidman (1993, 1994) whereby unbearable mental pain is regarded as the causal agent for suicidal behavior.

Five issues with the present research merit further comment. First, the lack of sex differences on the PAS and HSS seems to be inconsistent with findings indicating that, in general, women have higher rates of suicidal ideation and attempts than men (Canetto & Sakinofsky, 1998). Although such findings replicate, they do vary as a function of age, culture, and nationality (Freeman et al., 2017), suggesting that sex differences (or their lack) in suicidal behavior within a military population require additional research to delineate their specific nature.

Second, this research does not assess subsequent suicidal ideation or behavior but rather focuses on self-reported concurrent suicidality, as assessed by the HSS, and previous suicidal behavior: lifetime suicidal communication and a lifetime previous attempt. However, as highlighted by the World Heath Organization (2014), “a prior suicide attempt is the single most important predictor of death by suicide in the general population: individuals who have made prior suicide attempts are at much higher risk of dying by suicide than individuals who have not made prior suicide attempts” (p. 25). Thus, the convergent validity for the PAS scores is with regard to self-reported suicide risk rather than death by suicide.

Third and related to the second issue, the present investigation focuses on psychache and not other antecedents to suicide ideation or behavior. More recent theories incorporate an idea to action framework (Klonsky & May, 2014) that distinguishes between suicidal ideation and suicidal behavior and incorporates risk factors such as capability to act, reduced pain sensitivity, fearlessness, impulsivity, and physical availability of lethal materials.

Fourth, might the observed associations between PAS scale scores and suicidality be spuriously high? Although the influence of mono-method bias (i.e., self-report) cannot be ruled out, influences such as content overlap are not present. Inspection of the items of the PAS and HSS (Tables 1 and 2) shows little or no redundancy in content between the two measures. The content of the two scales addresses distinct constructs that are statistically linked.

Fifth, in emphasizing that the PAS is subtle because of the avoidance of item content that mentions suicidality, it seems somewhat inconsistent to use the HSS (an obvious, face valid measure) as a criterion. It is important to note, however that the factors that impede disclosure include shame, embarrassment, stigma, and fear of involuntary hospitalization (Killian, 2024). These factors can exist in applied (e.g., clinical, employment, correctional) settings where scale score results can have an impact on subsequent outcomes for scale respondents. Mirichlis et al. (2023), however, have indicated that in research settings, where responding is anonymous and/or confidential as it is in this research, more candid responding occurs because respondents do not anticipate subsequent negative impacts, want to contribute to research, and are emotionally prepared to discuss sensitive topics.

Limitations to our study and sample also exist. First, participants were a moderate size Canadian military sample. Replication with other nationalities and samples can establish the generalizability of our results. Second, although random sampling was employed, participants were volunteers that were a subset of the CAF. Despite the sample’s similarity to the overall CAF in the distribution of rank categories, generalizability to other populations requires confirmation. As such, replication with other military samples is recommended. Third, measures used were all self-report. This introduces the possibility of mono-method bias including socially desirable responding. Using other data collection methods such as health practitioner interviews could reduce potential biases and be informative. Fourth, ROC curve analyses were based on single items of the HSS. Because single-item measures can have limited reliability, replication with multi-item scales of suicide communication or suicide attempts is recommended. Fifth, with this research being cross-sectional, change in suicide risk status was not examined. Follow-up outcomes (e.g., subsequent suicide attempts, deaths) remain for future investigation. Sixth, in focusing on psychache, other variables such as depression and hopelessness were not considered. As such, discriminant and incremental validity within the military remain to be established for PAS scale scores. Seventh, this research did not consider other brief, face valid, self-reports such as the Brief Suicide Cognitions Scale (Rudd & Bryan, 2021). The comparison among alternative, concise screens is a direction for future research.

From an applied aspect, although the PAS demonstrates diagnostic accuracy, it is a screen for suicide risk status not an instrument for a definitive designation. A single scale cannot yield a valid diagnosis but can be useful for indicating the need for additional assessment (e.g., as a screen for further detailed testing and clinical interviewing). In this regard, subject to further validation studies, the PAS could be an easy-to-administer (self-report), brief (2–5 minute), non-threatening (no suicide item content) instrument usable within a triage process of the mental health system for identifying individuals requiring more intensive and expensive mental health evaluation.

In conclusion, the PAS has merit as a brief, subtle, self-report screen for identifying pre-existing risk for suicide in armed forces members. Additional research replicating these findings with other populations and other criteria will establish the generalizability of the PAS as a suicide risk screen.

Footnotes

Author Contributions

Conceptualization: All authors, Data curation: M.T.P. and E.L.P., Formal analysis: R.R.H., Writing - original draft R.R.H, review & editing, All authors.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Ronald R. Holden is the author of the Psychache and the Holden Suicidality scales but has no existing commercial interest in either scale. There are no other potential conflicts of interest.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Considerations

ORCID iD

Ronald R. Holden

Data Availability Statement

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data are not available.*

Note

Author Biographies

Ronald R. Holden is a professor emeritus in the Department of Psychology, Queen’s University, Canada. His research focuses on suicide, faking, and test construction.

Madeleine T. D’Agata holds a Ph.D. in social-personality psychology from Queen’s University, Canada. She conducted research for National Defence (Canada) from 2016-2022 on mental health as well as susceptibility to online influence and currently serves as the Chief of Staff in the Enforcement Branch at Environment and Climate Change Canada.

Erika L. Peter holds a Ph.D. in social-personality psychology from Queen’s University, Canada. She currently is a research scientist for Defence Research and Development Canada.

References

Akobeng

A. K.

(2007). Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatrica, 96(5), 644–647. https://doi.org/10.1111/j.1651-2227.2006.00178.x

Boulos

(2021). 2020 report on suicide mortality in the Canadian Armed Forces (1995 to 2019). Surgeon General Report, Surgeon General Health Research Program Health Research Program SGR-2021-001.

Campos

R. C.

Gomes

Holden

R. R.

Piteira

Rainha

(2017). Does psychache mediate the relationship between general distress and suicide ideation in the general population? Death Studies, 41(4), 241–245. https://doi.org/10.1080/07481187.2016.1251510

Canetto

S. S.

Sakinofsky

(1998). The gender paradox in suicide. Suicide and Life-Threatening Behavior, 28(1), 1–23. https://doi.org/10.1111/j.1943-278X.1998.tb00622.x

Cohen

(1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155

D’Agata

M. T.

Holden

R. R.

(2018). Self-concealment and perfectionistic self-presentation in concealment of psychache and suicide ideation. Personality and Individual Differences, 125, 56–61. https://doi.org/10.1016/j.paid.2017.12.034

DeLisle

M. M.

Holden

R. R.

(2009). Differentiating between depression, hopelessness, and psychache in university undergraduates. Measurement and Evaluation in Counseling and Development, 42(1), 46–63. https://doi.org/10.1177/0748175609333562

Ducasse

Holden

R. R.

Boyer

Artero

Calati

Guillaume

Courtet

Olié

(2017). Psychological pain in suicidality: A meta-analysis. Journal of Clinical Psychiatry, 79(3), 16r10732. https://doi.org/10.4088/JCP.16r10732

Flett

G. L.

Hewitt

P. L.

Heisel

M. J.

(2014). The destructiveness of perfectionism revisited: Implications for the assessment of suicide risk and the prevention of suicide. Review of General Psychology, 18(3), 156–172. https://doi.org/10.1037/gpr0000011

10.

Fowler

J. C.

(2012). Suicide risk assessment in clinical practice: Pragmatic guidelines for imperfect assessments. Psychotherapy, 49(1), 81–90. https://doi.org/10.1037/a0026148

11.

Franklin

J. C.

Ribeiro

J. D.

Fox

K. R.

Bentley

K. H.

Kleiman

E. M.

Huang

Musacchio

K. M.

Jaroszewski

A. C.

Chang

B. P.

Nock

M. K.

(2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143(2), 187–232. https://doi.org/10.1037/bul0000084

12.

Freeman

Mergl

Kohls

Székely

Gusmao

Arensman

Koburger

Hegerl

Rummel-Kluge

(2017). A cross-national study on gender differences in suicide intent. BMC Psychiatry, 17(1), 234. https://doi.org/10.1186/s12888-017-1398-8

13.

Friedman

R. A.

(2006). Uncovering an epidemic - screening for mental illness in teens. New England Journal of Medicine, 355(26), 2717–2719. https://doi.org/10.1056/NEJMp068262

14.

Holden

R. R.

(2021). The Holden Suicidality Scale. Author.

15.

Holden

R. R.

Mehta

(1998). The Psychache Scale. Authors.

16.

Holden

R. R.

Mehta

Cunningham

E. J.

McLeod

L. D.

(2001). Development and preliminary validation of a scale of psychache. Canadian Journal of Behavioral Science, 33(4), 224–232. https://doi.org/10.1037/h0087144

17.

Holden

R. R.

Patterson

A. A.

Fekken

G. C.

(2022). Childhood trauma and non-suicidal self-injury: Mental pain and depression as mediators in university and community samples? Personality and Individual Differences, 187, 111414. https://doi.org/10.1016/j.paid.2021.111414

18.

Khazem

L. R.

Rufino

K. A.

Rogers

M. L.

Gallyer

A. J.

Joiner

T. E.

Anestis

J. C.

(2021). Underreporting on the MMPI-2-RF extends to extra-test measures of suicide risk. Psychological Assessment, 33(8), 789–794. https://doi.org/10.1037/pas0001034

19.

Killian

K. D.

(2024) (In press). Development of a subtle screening for suicidal ideation: Psychometric characteristics and implications for family therapists. Contemporary Family Therapy. https://doi.org/10.1007/s10591-024-09705-z

20.

Klonsky

E. D.

May

A. M.

(2014). Differentiating suicide attempters from suicide ideators: A critical frontier for suicidology research. Suicide and Life-Threatening Behavior, 44(1), 1–5. https://doi.org/10.1111/sltb.12068

21.

Lambert

C. E.

Troister

Ramadan

Montemarano

Fekken

G. C.

Holden

R. R.

(2020). Psychache predicts suicide attempter status change in students starting university. Suicide and Life-Threatening Behavior, 50(3), 668–678. https://doi.org/10.1111/sltb.12624

22.

Mann

C. T.

Fischer

(2019). Recent trends in active-duty military deaths (IF10899). In Focus. Retrieved from. https://crsreports.congress.gov/product/pdf/IF/IF10899

23.

Mirichlis

Boyes

Hasking

Lewis

S. P.

(2023). Why are individuals who have not disclosed self-injury comfortable discussing their experiences in research? The Journal of Nervous and Mental Disease, 211(7), 473–478. https://doi.org/10.1097/NMD.0000000000001648

24.

Osman

Bagge

C. L.

Gutierrez

P. M.

Konick

L. C.

Kopper

B. A.

Barrios

F. X.

(2001). The suicidal behaviors questionnaire-revised (SBQ-R): Validation with clinical and nonclinical samples. Assessment, 8(4), 443–454. https://doi.org/10.1177/107319110100800409

25.

Patterson

A. A.

Holden

R. R.

(2012). Psychache and suicide ideation among men who are homeless: A test of Shneidman’s model. Suicide and Life-Threatening Behavior, 42(2), 147–156. https://doi.org/10.1111/j.1943-278X.2011.00078.x

26.

Pereira

E. J.

Kroner

D. G.

Holden

R. R.

Flamenbaum

(2010). Testing Shneidman’s model of suicidality in incarcerated offenders and in undergraduates. Personality and Individual Differences, 49(8), 912–917. https://doi.org/10.1016/j.paid.2010.07.029

27.

Robins

Murphy

G. E.

Wilkinson

R. H.

Jr. Gassner

Kayes

(1959). Some clinical considerations in the prevention of suicide based on a study of 134 successful suicides. American Journal of Public Health, 49(7), 888–899. https://doi.org/10.2105/AJPH.49.7.888

28.

Rudd

M. D.

Bryan

C. J.

(2021). The brief suicide Cognitions scale: Development and clinical application. Frontiers in Psychiatry, 12, 737393. https://doi.org/10.3389/fpsyt.2021.737393

29.

Rudd

M. D.

Joiner

Rajab

M. H.

(1996). Relationships among suicide ideators, attempters, and multiple attempters in a young-adult sample. Journal of Abnormal Psychology, 105(4), 541–550. https://doi.org/10.1037/0021-843X.105.4.541

30.

Rudestam

K. E.

(1977). Physical and psychological responses to suicide in the family. Journal of Consulting and Clinical Psychology, 45(2), 162–170. https://doi.org/10.1037/0022-006X.45.2.162

31.

Shneidman

E. S.

(1993). Suicide as psychache. The Journal of Nervous and Mental Disease, 181(3), 145–147. https://doi.org/10.1097/00005053-199303000-00001

32.

Shneidman

E. S.

(1994). Clues to suicide, reconsidered. Suicide and Life-Threatening Behavior, 24(4), 395–397.

33.

Streiner

D. L.

Cairney

(2007). What's under the ROC? An introduction to receiver operating characteristics curves. Canadian Journal of Psychiatry, 52(2), 121–128. https://doi.org/10.1177/070674370705200210

34.

Troister

D’Agata

M. T.

Holden

R. R.

(2015). Suicide risk screening: Comparing the beck depression inventory-II, beck hopelessness scale, and psychache scale in undergraduates. Psychological Assessment, 27(4), 1500–1506. https://doi.org/10.1037/pas0000126

35.

Troister

Davis

M. P.

Lowndes

Holden

R. R.

(2013). A five-month longitudinal study of psychache and suicide ideation: Replication in general and high-risk university students. Suicide and Life-Threatening Behavior, 43(6), 611–620. https://doi.org/10.1111/sltb.12043

36.

Troister

Holden

R. R.

(2012). A two-year prospective study of psychache and its relationship to suicidality among high-risk undergraduates. Journal of Clinical Psychology, 68(9), 1019–1027. https://doi.org/10.1002/jclp.21869

37.

Troister

Holden

R. R.

(2013). Factorial differentiation among depression, hopelessness, and psychache in statistically predicting suicidality. Measurement and Evaluation in Counseling and Development, 46(1), 50–63. https://doi.org/10.1177/0748175612451744

38.

VanTil

Kopp

Heber

(2021). Veteran suicide mortality study; Follow-up period 1975 to 2016. Research Directorate Technical Report. Veterans Affairs Canada.

39.

Verrocchio

M. C.

Carrozzino

Marchetti

Andreasson

Fulcheri

Bech

(2016). Mental pain and suicide: A systematic review of the literature. Frontiers in Psychiatry, 7, 108. https://doi.org/10.3389/fpsyt.2016.00108

40.

World Health Organization . (2014). Preventing suicide: A global imperative. World Health Organization. https://apps.who.int/iris/handle/10665/131056

41.

World Health Organization . (2021). Suicide worldwide in 2019: Global health estimates. World Health Organization.