Factor Structure and Measurement Invariance of the Stigma of Suicide Scale–Short Form Among Adolescents and Early Adults in Hong Kong

Abstract

Suicide is a global public health concern. The Stigma of Suicide Scale–Short Form (SOSS-SF) is a brief measure on the stigma toward suicide. This study aimed to examine the psychometric properties of the SOSS-SF in a sample of 1,946 adolescents and early adults (M age = 23.3, 69% females) in Hong Kong. The participants completed the SOSS-SF, measures on suicidal ideation and perceived distress, and binary items on lifetime risk behaviors online. Exploratory structural equation modeling evaluated the factor structure and reliability of the SOSS-SF in split samples and its measurement invariance, convergent validity, and discriminant validity in the overall sample. Contrary to a poor fit for the three-factor structure in the original 16-item SOSS-SF, the four-factor model showed a good fit in the revised 12-item version in both subsamples. The four factors (Glorification, Isolation, Disgrace, and Selfishness) loaded significantly on three items each (λ = .52-.93) and had adequate reliability (ω = .78-.88). The SOSS-SF displayed scalar invariance across age group, gender, and suicidal ideation and significant associations with validating variables. These results demonstrate a valid and reliable four-factor structure for the SOSS-SF and support its use for assessment of multifaceted stigma toward suicide.

Keywords

exploratory structural equation modeling factor analysis measurement invariance psychometrics stigma suicidal ideation severity

Suicide is an important public health issue as a leading cause of death among adolescents worldwide (Hawton et al., 2012). Suicidal behavior is perceived to result from an interaction of psychological, psychiatric, environmental, and sociocultural factors (Bridge et al., 2006). A recent meta-analysis of 365 studies from the past 50 years (Franklin et al., 2017) summarizes current knowledge about the risk factors of suicidal ideation. Some common risk factors for suicide in adolescents include ethnic minorities (Forte et al., 2018), sexual minorities (Caputi et al., 2017), impulsivity (Liu et al., 2017), and presence of mental disorders such as depression (Y. B. Cheung et al., 2006; Hawton et al., 2013), psychosis (Pelizza et al., 2019), and substance abuse (Bilsen, 2018). Results of the meta-analysis suggest that predictive ability of the existing studies has not improved across 50 years and studies rarely examined the combined effects of multiple risk factors.

Stigmatizing attitudes toward suicide are perceived as psychological barriers on help-seeking for suicide attempt survivors (Corrigan et al., 2017). Suicide stigma has been associated with worse well-being and greater suicidal ideation in suicide attempt survivors (Mayer et al., 2020; Salway et al., 2018; Sheehan et al., 2017) and suicide loss survivors (Oexle et al., 2020; Pitman et al., 2018; Scocco et al., 2017). Psychological distress has been linked with greater perceived suicide stigma in suicide survivors (Scocco et al., 2017) and attempters (Scocco et al., 2016).

A valid and reliable measure of attitudes toward suicide could enable a better understanding of suicide stigma among adolescents and facilitate formulation of suicide prevention and intervention strategies. The Stigma of Suicide Scale (SOSS) was developed by Batterham et al. (2013b) as a measure of attitudes toward suicide. The SOSS consists of a comprehensive list of 58 descriptors of a prototypical person who died by suicide. The developers identified three factors (stigma, isolation, and glorification) and developed a 16-item Stigma of Suicide Scale–Short form (SOSS-SF) by selecting items with the strongest factor loadings. The SOSS-SF has displayed satisfactory reliability and convergent validity across studies (Batterham et al., 2013a; Chan et al., 2014; Ozturk et al., 2017; Williams et al., 2018). In the Chinese context, this scale has undergone initial validation in a sample of 224 university students in mainland China (Han et al., 2017).

Several methodological pitfalls could be noted in the previous psychometric studies on the SOSS-SF. First, the original three-factor structure was derived via principal component analysis, which was only tenable for the purpose of item reduction but not for examination of latent factor structure (Schmitt, 2011). Second, most of the psychometric studies (Chan et al., 2014; Han et al., 2017; Ozturk et al., 2017; Williams et al., 2018) were based on samples of university undergraduates. Student samples typically have a narrow age range and are considered more homogeneous (Henry, 2008), which raise concerns on the representativeness of these samples and associated external validity. Hanel and Vione (2016) demonstrate that results derived from student samples might not generalize to the general public in the case of attitudinal variables. Further large-scale studies are needed to evaluate the psychometric properties of the SOSS-SF in community samples of adolescents and early adults.

Third, the validation study by Han et al. (2017) used an obsolete method of orthogonal rotation for the SOSS-SF factors. The unrealistic assumption of uncorrelated factors could lead to incorrect number of factors and distorted factor structure. Fourth, establishing measurement invariance in terms of equal factor loadings and intercepts is crucial to ensure valid comparisons of the latent constructs across groups (Van de Schoot et al., 2012). Measurement invariance of the SOSS-SF has been overlooked in almost all of the psychometric studies except for an examination of gender invariance in Williams et al. (2018). From a developmental perspective, it is plausible that the SOSS-SF functions differently between individuals in different life stages (adolescence and early adulthood). It is of practical interest to evaluate the measurement invariance of the SOSS-SF across age. Besides, presence or absence of suicidal ideation could theoretically alter the views or attitudes of an individual toward suicide. However, the measurement invariance of the SOSS-SF across people with and without suicidal ideation is yet to be examined.

In light of the research gaps, the present study aimed to perform a systematic investigation of the psychometric properties of the Chinese SOSS-SF in a large-scale community sample of adolescents and early adults in Hong Kong. Confirmatory factor analysis (CFA) was a traditional factor analytic technique that could be overly restrictive with the assumption of exact zero cross-loadings (B. Muthén & Asparouhov, 2012). The present study utilized the exploratory structural equation modeling (ESEM) approach to analyze the factor structure of the SOSS-SF. Reliability of the SOSS-SF factors and measurement invariance of the SOSS-SF across gender, age group, and lifetime suicidal ideation were assessed. Finally, convergent and discriminant validity of the SOSS-SF factors were evaluated with constructs on perceived distress, suicidal ideation severity, and lifetime experience of risk behaviors (suicide ideation, suicide attempt, deliberate self-harm, and bullied others). For convergent validity, the SOSS-SF factors should demonstrate significant associations with perceived distress, suicidal ideation severity, lifetime suicidal ideation, deliberate self-harm, and suicide attempt. In terms of discriminant validity, the SOSS-SF factors should have marginal associations with the lifetime experience of having bullied others.

Method

Participants and Procedures

Participants of this study originated from the baseline wave of a large-scale panel online survey on mental well-being and suicide risks of adolescents and early adults in Hong Kong. Various promotion means such as bulk emails, promotion posters, and reminders were sent to the members of local university and three local nongovernmental organizations. This online survey recruited a total of 1,946 participants under convenience sampling from January 2018 to July 2018. Eligibility to participate in the survey included residence in Hong Kong, an age between 12 and 35 years old, and ability to understand Cantonese. Incentives were provided to improve the response rate of the survey.

Ethics approval was obtained from the human research ethics committee of the authors’ university (Reference number: EA1709039). The participants provided informed consent and completed an online questionnaire. Participation in the survey was fully voluntary and the participant could choose to stop at any time. Contact information of 24-hour emotional support hotlines and online emotional support services was included at the end of the survey and the participant was encouraged to seek help in case he or she felt discomfort and distressed during or after completing the survey. We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study.

Measures

Participants were asked about their demographic characteristics on gender, age, education level, student status, diagnosis of mental disorders (major depressive disorder, schizophrenia, social phobia, and avoidant personality disorder). They completed the Chinese version of the SOSS-SF and the following measures. Given the wide age range of the present study sample, participants aged 24 years or below were classified into the adolescents group and those aged 25 years or above were classified into the early adults group.

Stigma of Suicide

The SOSS-SF is a 16-item instrument that inquires respondents’ views on people who died by suicide (Batterham et al., 2013b). The SOSS-SF has been translated into Chinese and undergone preliminary validation by Han et al. (2017). The original version of the SOSS-SF consists of three subscales: (a) Stigma (eight items: immoral, pathetic, irresponsible); (b) Isolation (four items: lonely, lost, disconnected); and (c) glorification (four items: strong, brave, noble). The items are scored on a 5-point Likert-type format from 1 (strongly disagree) to 5 (strongly agree), with higher scores denoting higher levels of stigma in that aspect.

Suicidal Ideation Severity

The Suicidal Ideation Attributes Scale (van Spijker et al., 2014) is a brief measurement tool of the severity of suicidal ideation of the participants over the past month. This five-item scale measures the frequency of suicide thoughts, uncontrollability of the thoughts, closeness to suicide attempt, distress brought by the thoughts, and their interference on daily activities. The items are scored on an 11-point Likert-type format with higher scores indicating greater recent severity. The five items were summed to produce the total score on suicidal ideation severity with a theoretical range from 0 to 50. This scale has been validated in the Chinese context (Han et al., 2017). The one-factor model showed an adequate fit (comparative fit index [CFI] = 0.99, Tucker–Lewis index [TLI] = 0.98, root mean square error of approximation [RMSEA] = 0.051, standardized root mean square residual [SRMR] = 0.017) to the data and good reliability (Cronbach’s α = .87) in the present study.

Perceived Distress

Participants reported their perceived levels of distress from eight different sources on academic, job, financial, social life, physical well-being, mental well-being, relationship with family, and relationship with partner over the past month. The items on academic, job, and relationship with partner were only completed by respondents who were studying, working, and had a partner, respectively. The eight items are scored on a 5-point Likert-type format from 1 (not at all) to 5 (very serious), with higher scores denoting greater distress in that aspect. The eight items were averaged to give the total score on perceived distress with a theoretical range from 1 to 5. For this self-constructed scale, the one-factor model showed an acceptable fit (CFI = 0.98, TLI = 0.98, RMSEA = 0.059, SRMR = 0.025) to the data and good reliability (α = .82) in the present study.

Lifetime Risk Behaviors

Respondents were asked about their lifetime experience of the following four risk behaviors: bullied others, considered suicide, deliberate self-harm, and suicide attempt. The four items were answered on a yes or no format and modelled as binary variables in the analysis. In the present study, deliberate self-harm referred to whether or not the participant has injured himself or herself intentionally in the lifetime. The first item on bullying was considered a risk behavior that was marginal to the construct of suicide while the latter three items were considered to be conceptually relevant to suicide.

Data Analysis

Preliminary data screening did not reveal any outliers in the continuous variables (stigma of suicide, suicidal ideation severity, and perceived distress) in the study sample. The overall sample (N = 1,946) was split into two random subsamples for cross-validation of the factor structure. The ample sample size (N > 950) of the split-half subsamples provided adequate statistical power for the 16-item SOSS-SF. The demographic and descriptive profiles of the two subsamples were compared via chi-square (χ²) and independent t tests. The present study evaluated the psychometric properties of the SOSS-SF in four steps. In the first step, CFA was used to test the original 16-item, three-factor model in Subsample 1. The present study adopted the ESEM approach to examine the factor structure of the SOSS-SF using Mplus 8.4 (L. K. Muthén & Muthén, 2017). The ESEM approach is flexible in accounting for the cross-loadings of the latent factors (Fong et al., 2015). However, 12 of the 16 SOSS-SF items displayed substantial floor/ceiling effects with over one forth of the respondents endorsing the minimum or maximum values. The SOSS-SF items were modeled as ordinal variables using the robust weighted least squares estimator.

ESEM models with one to five factors were specified under the oblique Geomin rotation to allow for factor correlations. Evaluation of the model fit was based on the χ² test and cutoff criteria on the following approximate fit indices (Hu & Bentler, 1999): CFI ≥ 0.95, TLI ≥ 0.95, RMSEA ≤ 0.06, and SRMR ≤ 0.06. Eigenvalues, scree plot inspection, and model fit indices were used to determine the dimensionality of the SOSS-SF. Nested models were compared using the χ² difference test under the DIFFTEST function in Mplus. In case of an inadequate model fit, problematic items with double loadings (λs ≥ .30) or no significant loadings (all λs < .50) were iteratively removed from the model. The ESEM factor model generated in Subsample 1 was tested in Subsample 2. The second step assessed the reliability of the SOSS-SF factors using Mcdonard’s omega (ω) in both subsamples, with values of at least 0.70 indicating satisfactory reliability.

In the third step, the measurement invariance of the SOSS-SF across age group, gender, and lifetime suicidal ideation was evaluated in the overall sample. As a prerequisite for conducting invariance tests, the factor model was fitted in separate subgroups between adolescents and early adults, females and males, participants without and with lifetime suicidal ideation. Configural invariance model was specified under multigroup ESEM to allow for different factor loadings and thresholds. This was followed by testing the scalar invariance model with constrained equal factor loadings and thresholds. Given the potential oversensitivity of the χ² test to trivial deviations in large sample size (G. W. Cheung & Rensvold, 2002), tenability of scalar invariance was examined via the χ² difference test and changes in fit indices between the configural and scalar invariance models. The criteria of ΔCFI ≥ −0.005 and ΔRMSEA ≤ 0.01 were considered evidence in support of invariance with categorical indicators (Rutkowski & Svetina, 2017). The last step evaluated the structural invariance of factor variances, covariances, and means across age group, gender, and lifetime suicidal ideation.

The fourth step examined the convergent and discriminant validity of the SOSS-SF factors with reference to perceived distress, suicidal ideation severity, and lifetime risk behaviors. Perceived distress and suicidal ideation severity in the past month were modeled as continuous outcomes while the four risk behaviors were modeled as separate binary outcomes in the ESEM model. Demographic characteristics (age, gender, and student status) were controlled in the ESEM model. There were no missing data in the SOSS-SF and other measures in the present study. The significance level was set at 0.01 given the current large sample size.

Results

Participant Characteristics

The overall sample had a mean age of 23.3 years (SD = 5.2). The majority of the sample was adolescents (63.4%), female (68.6%), attained at least tertiary education (81.4%), and students (75.5%). A small proportion of the sample reported a diagnosis of major depressive disorder (7.7%), social phobia (2.8%), schizophrenia (1.5%), and avoidant personality disorder (1.4%). The lifetime prevalence of risk behaviors was 45.3%, 26.4%, 19.6%, and 7.8% for suicidal ideation, deliberate self-harm, bullying, and suicide attempt, respectively. The sample reported moderate levels of perceived distress (M = 2.62, SD = 0.79) and overall low severity of suicidal ideation (M = 5.1, SD = 9.3) over the past month. As shown in Table 1, the two subsamples did not differ significantly in most of the characteristics (p = .07-.78), except for a significant yet small difference (Cohen d = 0.13, p < .01) in suicidal ideation severity. Perceived distress was significantly and positively correlated with lifetime suicidal ideation, suicidal ideation severity, deliberate self-harm, and suicide attempt (r = .33-.39, p < .01) but not with bullying (r = .11, p > .01).

Table 1.

Demographic and Descriptive Profiles of the Two Split Samples.

Variables	Subsample 1 (n = 971), N (%)	Subsample 2 (n = 975), N (%)	χ²	p
Gender
Female	670 (69.0)	664 (68.1)	0.18	.67
Age group			1.12	.29
Adolescents (12-24 years)	604 (62.2)	629 (64.5)
Early adults (25-35 years)	367 (37.8)	346 (35.5)
Tertiary education or above	778 (81.6)	770 (81.1)	0.07	.78
Studying at school	730 (75.2)	739 (75.8)	0.10	.75
Diagnosis of mental disorder
Major depressive disorder	80 (8.2)	69 (7.1)	0.93	.34
Schizophrenia	16 (1.6)	14 (1.4)	0.14	.70
Social phobia	30 (3.1)	24 (2.5)	0.71	.40
Avoidant personality disorder	11 (1.1)	16 (1.6)	0.92	.34
Lifetime risk behavior
Bullying experience	185 (19.1)	197 (20.2)	0.41	.52
Suicidal ideation	458 (47.2)	423 (43.4)	2.81	.09
Deliberate self-harm	262 (27.0)	252 (25.8)	0.32	.57
Suicide attempt	86 (8.9)	65 (6.7)	3.26	.07
Variables	M (SD)	M (SD)	t	p	d
Age	23.5 (5.1)	23.1 (5.2)	1.72	.09	0.08
Perceived distress	2.63 (0.79)	2.61 (0.78)	0.82	.41	0.04
Suicidal ideation severity	5.71 (10.0)	4.49 (8.41)	2.91	<.01*	0.13

p < .01.

Factorial Validity

As shown in Table 2, the original three-factor CFA model showed a poor fit to the 16-item version of the SOSS-SF with CFI = 0.82, TLI = 0.79, RMSEA = 0.165, SRMR = 0.117. For the ESEM models, the one-factor, two-factor, and three-factor models failed to provide an adequate fit to the Subsample 1. The four-factor model provided a partially acceptable fit (CFI = 0.97 and SRMR = 0.038 but TLI = 0.94 and RMSEA = 0.09) and fitted the data significantly better than the three-factor model (Δχ² = 428, Δdf [degrees of freedom] = 13, p < .01). Despite a significantly better fit (Δχ² = 254, Δdf = 12, p < .01) for the five-factor model, the fifth factor only loaded significantly on two items, indicating possible overextraction of the factors. Investigation of the four-factor model in the 16 SOSS-SF items revealed four problematic items. Three of them (“Pathetic,” “Shallow,” and “Noble”) showed double factor loadings and the other item (“Disconnect”) did not load significantly on any factor. Removal of these four items led to a good fit for the four-factor model with CFI = 0.99, TLI = 0.98, RMSEA = 0.053, and SRMR = 0.013. These findings supported the presence of four factors for the revised 12-item SOSS-SF in Subsample 1.

Table 2.

Fit Indices of the CFA and ESEM Models in Subsample 1.

Model	χ²	df	Δχ²	CFI	TLI	RMSEA [90% CI]	SRMR
16-Item SOSS-SF
Original three-factor CFA	2786*	101		.82	.79	.165 [.160, .171]	.117
ESEM models
One-factor ESEM	6651*	104		.57	.50	.255 [.249, .260]	.214
Two-factor ESEM	3134*	89	2271*	.80	.73	.188 [.182, .193]	.124
Three-factor ESEM	1108*	75	1214*	.93	.89	.119 [.113, .125]	.056
Four-factor ESEM	550*	62	428*	.97	.94	.090 [.083, .097]	.038
Five-factor ESEM	209*	50	254*	.99	.98	.057 [.049, .065]	.021
12-Item SOSS-SF
Four-factor ESEM	89.5*	24		.99	.98	.053 [.042, .065]	.013

Note. n = 971. CFA = confirmatory factor analysis; ESEM = exploratory structural equation modeling; df = degrees of freedom; Δχ² = change in chi-square; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; CI = confidence interval; SRMR = standardized root mean square residual; SOSS-SF = Stigma of Suicide Scale–Short Form.

p < .01.

Factor Structure and Reliability

Table 3 presents the factor structure of the revised 12-item, four-factor model in Subsample 1. The eigenvalues of the four factors were 4.24, 2.50, 1.73, and 0.83, accounting for 77.5% of the total variance. All of the 12 items loaded significantly and substantially (λ > .50, p < .01) onto exactly one of the four factors: glorification, isolation, disgrace, and selfishness, with each factor consisting of three items. Six cross-loadings were statistically significant (p < .01) but were smaller than 0.30. The revised 12-item, four-factor model provided a good model fit to Subsample 2 with CFI = 0.99, TLI = 0.99, RMSEA = 0.048, and SRMR = 0.012. The four factors displayed acceptable reliability in both Subsample 1 (ω = .78-.88) and Subsample 2 (ω = .73- .86). In Subsample 1, glorification was found to be negatively correlated with the other three factors (r = −.44 to −.16, p < .01). Selfishness was positively associated with isolation and disgrace (r = .17 to .63, p < .01) but the latter two were not significantly associated with each other (r = .03, p = .53).

Table 3.

Factor Loadings, Reliability, and Correlations of the Four Factors in the 12-Item SOSS-SF in Subsample 1.

Item	Glorification	Isolation	Disgrace	Selfishness
1. Strong	.93*	.02	.01	−.05
2. Brave	.76*	.02	−.08	.07
16. Dedicated	.52*	−.16*	.27*	−.03
3. Lonely	.01	.91*	.02	−.11
4. Lost	.02	.76*	−.16*	.15*
5. Isolated	−.05	.80*	.08	−.01
6. Immoral	−.01	−.03	.76*	.12*
7. An embarrassment	−.05	.04	.80*	.02
8. Vengeful	.08	.03	.69*	.03
12. Irresponsible	−.05	.01	.12*	.72*
13. Stupid	.03	−.02	−.02	.95*
14. Cowardly	−.03	.04	.07	.78*
Mcdonald ω	.78	.85	.81	.88
Factor correlations
Isolate	−.18*
Disgrace	−.16*	.03
Selfishness	−.44*	.17*	.63*

Note. n = 971. Factor loadings with magnitude greater than .40 are bolded. SOSS-SF = Stigma of Suicide Scale–Short Form.

p < .01.

Measurement Invariance

Table 4 shows the fit indices of the measurement invariance models of the SOSS-SF in the overall sample. The four-factor ESEM model displayed adequate model fits in the separate subgroups of adolescents (n = 1,233) and adults (n = 713), females (n = 1,334) and males (n = 612), and participants without (n = 1,065) and with lifetime suicidal ideation (n = 881). Good model fit was found for both configural and scalar invariance models across age group, gender, and lifetime suicidal ideation (CFI = 0.991-0.995, TLI = 0.982-0.994, RMSEA = 0.032-0.054, SRMR = 0.012-0.022). Despite the significant changes (Δχ² = 110.6-180.3, Δdf = 64, p < .01) in the χ² difference test, the scalar invariance models showed comparable CFI (ΔCFI = −0.003-0.002) and improvements in TLI (ΔTLI = 0.006-0.012) and RMSEA (−0.011 to −0.022). These results supported scalar measurement invariance across age group, gender, and lifetime suicidal ideation.

Table 4.

Measurement Invariance Tests of the Four-Factor ESEM Models Across Age Group, Gender, and Lifetime Suicidal Ideation for the 12-Item SOSS-SF.

Model	N	χ²	df	CFI	TLI	RMSEA [90% CI]	SRMR
Age group
M1. Adolescents	1,233	104.1*	24	0.995	0.985	0.052 [0.042, 0.062]	0.012
M1. Adults	713	51.0*	24	0.996	0.989	0.040 [0.024, 0.055]	0.012
M2. Configural	1,946	150.0*	48	0.995	0.987	0.047 [0.038, 0.055]	0.012
M3. Scalar	1,946	221.2*	112	0.995	0.994	0.032 [0.025, 0.038]	0.020
Gender
M1. Female	1,334	123.1*	24	0.993	0.982	0.056 [0.046, 0.066]	0.013
M1. Male	612	61.9*	24	0.994	0.984	0.051 [0.035, 0.067]	0.013
M2. Configural	1,946	181.4*	48	0.994	0.983	0.053 [0.045, 0.062]	0.013
M3. Scalar	1,946	307.5*	112	0.991	0.989	0.042 [0.037, 0.048]	0.022
Lifetime suicidal ideation
M1. No ideation	1,065	94.3*	24	0.994	0.983	0.052 [0.042, 0.064]	0.012
M1. Have ideation	881	89.1*	24	0.993	0.980	0.055 [0.043, 0.068]	0.014
M2. Configural	1,946	182.7*	48	0.993	0.982	0.054 [0.046, 0.062]	0.013
M3. Scalar	1,946	221.4*	112	0.995	0.994	0.032 [0.026, 0.038]	0.019

Note. ESEM = exploratory structural equation modeling; SOSS-SF = Stigma of Suicide Scale–Short Form; df = degree of freedom; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = Root mean square error of approximation; CI = confidence interval; SRMR = standardized root mean square residual.

p < .01.

The four SOSS-SF factors showed invariant factor variances/covariances across gender and lifetime suicidal ideation (Δχ² = 16.5-20.5, Δdf = 10, p > .01). Invariance in factor variances/covariances was not found across age group (Δχ² = 27.8, Δdf = 10, p < .01), with adults displaying smaller variances (0.67-0.95) in the four SF factors than adolescents (variance fixed at 1 as reference). In terms of latent factor means, adults showed equivalent levels of glorification, disgrace, and selfishness (Δ = −0.036-0.043, SE = 0.06, p = .45-.50) but significantly higher levels of isolation (Δ = 0.17, SE = 0.06, p < .01) than adolescents. Male participants displayed significantly higher levels of disgrace and selfishness (Δ = 0.17 to 0.39, SE = 0.06, p < .01) and lower levels of isolation (Δ = −0.26, SE = 0.06, p < .01) than female participants. Presence of lifetime suicidal ideation was linked to higher latent means in glorification and isolation (Δ = 0.13 to 0.49, SE = 0.05, p < .01) but lower means in disgrace and selfishness (Δ = −0.36 to −0.45, SE = 0.06, p < .01).

Associations Between Demographic Characteristics and the Study Variables

As shown in Table 5, student status did not significantly (p > .01) predict any of the SOSS-SF factors. Consistent with results in the previous section, age positively predicted isolation and males were significantly associated with lower levels of isolation and higher levels of disgrace and selfishness. Regarding the relationships between the demographic characteristics and validating variables, age tended to be associated with higher perceived distress (β = .06, SE = .03, p = .017) and lower levels of suicidal ideation severity (β = −.05, SE = .03, p = .045) over the past month. Males were associated with a higher probability of bullying (β = .18, SE = .03, p < .01) and a lower probability of deliberate self-harm (β = −.07, SE = .03, p = .031). The likelihoods of lifetime suicidal ideation and suicide attempt were not significantly associated with age, gender, and student status (p = .46-.99).

Table 5.

Associations Among the Demographic Characteristics, SOSS-SF Factors, Validating Variables in the Overall Sample.

	SOSS-SF factors
	Glorification	Isolation	Disgrace	Selfishness
	β (SE)	β (SE)	β (SE)	β (SE)
Demographic characteristics
Age	−.00 (.03)	.08 (.03)*	−.00 (.03)	.01 (.03)
Male	−.03 (.03)	−.11 (.03)*	.18 (.03)*	.08 (.02)*
Student status	−.01 (.03)	.03 (.03)	−.06 (.03)	.00 (.03)
Validating variables
Perceived distress	.23 (.03)*	.13 (.02)*	.22 (.04)*	−.14 (.04)*
Bullying experience	.07 (.04)	.04 (.04)	.05 (.05)	−.03 (.06)
Lifetime suicidal ideation	.25 (.04)*	.13 (.03)*	−.10 (.05)	−.14 (.05)*
Suicidal ideation severity	.23 (.03)*	.15 (.02)*	.07 (.03)	−.20 (.03)*
Deliberate self-harm	.22 (.04)*	.11 (.03)*	.00 (.05)	−.10 (.05)
Suicide attempt	.19 (.05)*	.20 (.04)*	.13 (.06)	−.30 (.06)*

Note. N = 1,946. SOSS-SF = Stigma of Suicide Scale–Short Form; β = standardized regression estimates; SE = standard error.

p < .01.

Convergent and Discriminant Validity

The combined ESEM model with demographic characteristics (gender, age, and student status) and validating variables (perceived distress, suicidal ideation severity, lifetime suicidal ideation, bullying, deliberate self-harm, and suicide attempt) provided a good fit to the overall sample (Δχ² = 253.2, Δdf = 96, p < .01, CFI = 0.99, TLI = 0.99, RMSEA = 0.029, 90% confidence interval [0.025, 0.033], SRMR = 0.019). Glorification and isolation were significantly and positively associated with perceived distress, suicidal ideation severity, lifetime suicidal ideation, deliberate self-harm, and suicide attempt (β = .11 to .25, SE = .02-.05, p < .01). Disgrace was significantly and positively associated with perceived distress (β = .22, SE = .04, p < .01) but not with any of the lifetime risk behaviors (p > .01). Selfishness was significantly and negatively associated with perceived distress, suicidal ideation severity, lifetime suicidal ideation, and suicide attempt (β = −.14 to −.30, SE = .03-.06, p < .01). All of the four factors were not associated with bullying experience (β = −.03 to .07, SE = .04- .06, p = .12-.56).

Discussion

The present study evaluated the factor structure of the SOSS-SF using ESEM in a large sample of adolescents and early adults in Hong Kong. Neither the CFA nor ESEM results supported the original three-factor structure, which was derived via principal component analysis. The present study found a superior fit for the four-factor model over the three-factor model and the five-factor model was prone to overextraction of factors. Similar to a previous validation study in Chinese (Han et al., 2017), our study removed four items because of sizable cross-loadings. The four-factor structure demonstrated adequate model fits in the refined 12-item version across the two random subsamples. In line with recent findings (Williams et al., 2018), the SOSS-SF factors displayed good reliability and small intercorrelations (r < .20) in general. The modest intercorrelations suggest the existence of distinct stigma facets toward suicide. The use of bifactor modeling is not warranted given the lack of the general stigma factor.

The four-factor structure emerged for the SOSS-SF rather than the existing three-factor structure in the current analysis. This discrepancy could be attributed to the sample difference between the present study and Han et al. (2017) in terms of sample size (N = 1,946 vs. 224), setting (community vs. university), age range (12-35 vs. 18-28), geographical context (Hong Kong vs. mainland China). From an analytical aspect, the inconsistency in dimensionality might be explained by the modeling approach (ESEM vs. CFA), item nature (categorical vs. continuous; Williams et al., 2018) and factor rotation (oblique vs. orthogonal; Han et al., 2017). The original factors on isolation and glorification were preserved in the four-factor structure while the six items under the original stigma factor were found to load on two separate factors of disgrace and selfishness. Items under the disgrace factor appear to tap on judgments stemming from morality standard (Pridmore & McArthur, 2009) while items under the selfishness factor refer to critical appraisals of suicide from the cognitive aspect. The moderately strong correlation (r = .63, p < .01) between disgrace and selfishness implies that they are closely related but different stigma factors.

Measurement invariance of the SOSS-SF was examined across gender, age group, and lifetime suicidal ideation via multigroup ESEM. Consistent with Williams et al. (2018), the SOSS-SF demonstrated scalar invariance across gender with no gender differences in the loadings and thresholds. Besides, scalar invariance was also found for the SOSS-SF across age group and lifetime suicidal ideation. The presence of scalar invariance supports meaningful comparisons of the structural parameters of stigma across the demographic subgroups. The SOSS-SF factors displayed largely equivalent variances/covariances across the demographic subgroups. Males reported higher latent means of disgrace and selfishness and lower latent means of isolation compared with females. Such gender differences in the SOSS-SF factor means are consistent with the previous findings (Batterham et al., 2013a; Williams et al., 2018). Similar to previous research findings (Batterham et al., 2019; Williams et al., 2018), participants with lifetime suicidal ideation showed lower levels of suicide stigma but higher levels of glorification and isolation.

Practical Implications

The four SOSS-SF factors displayed expected convergent and discriminant validity with the validating variables in the present sample. All of the four factors were not significantly associated with the lifetime experience of having bullied others, which supports discriminant validity for the SOSS-SF. Higher stigma in glorification and isolation were positively linked with perceived distress and suicidal ideation severity over the past month, and lifetime experiences of suicidal ideation, deliberate self-harm, and suicide attempt. These two stigma factors appear to be potential risk factors of suicidal ideation and future suicide attempts. Participants who tended to label the suicide act as disgraceful showed only significantly greater perceived distress. This factor was not significantly associated with any of the risk behaviors and suicidal ideation severity. Consistent with previous literature (Scocco et al., 2016; Scocco et al., 2017), all four stigma factors showed significant associations with perceived distress.

Higher stigmatization of suicide as selfish was associated with lower levels of perceived distress and suicidal ideation severity over the past month together with lower likelihood of lifetime suicidal ideation and suicide attempt. The selfishness factor could potentially play a protective role in the development of suicidal ideation and subsequent suicide attempts. According to the ideation-to-action framework (Klonsky et al., 2016), on the development of suicidal ideation, the transition from suicidal ideation to suicide attempts becomes a critical step for the at-risk individuals. Future longitudinal studies are needed to explore the potential role of this stigma facet on strategies of suicide prevention targeting such a transition among the at-risk individuals.

It should be noted that only selfishness but not disgrace was found to be significantly associated with recent suicidal ideation severity, lifetime suicidal ideation and suicide attempt. Moreover, they were linked with perceived distress in the opposite direction. These findings demonstrate that these two factors are related but distinct types of stigma toward suicide. It is theoretically important to distinguish between them to avoid conflating their relationships with suicide ideation and behaviors. Since the two factors involve cognitive participation, psychoeducation interventions, and cognitive reframing programs may help improve the perceived stigma and its demoralizing effects on the adolescents (Kucukalic & Kucukalic, 2017). Particular subgroups of adolescents with distinct stigma profiles could be identified to allow targeted suicide prevention programs for individuals with diverse service needs (Fullagar et al., 2007).

Study Limitations

A few limitations should be noted in the present study. First, the majority of the present sample was female, belonged to the adolescents group, and had higher education level. This differed from the demographic profile of general population in Hong Kong. Cautions should be warranted before replication of the present four-factor structure in future surveys with random sampling. Second, despite the large sample size, the present data were cross-sectional in nature and could not assess the test–retest reliability and longitudinal invariance of the SOSS-SF. The causal direction of the relationships between the stigma factors and constructs such as perceived distress remains to be elucidated in future longitudinal studies. The small sample size for participants with suicide attempts (n = 151 < 200) does not allow a stable estimation of the SOSS-SF factor model in this subgroup. Examination of measurement invariance across those with and without suicide attempts was not feasible in this study. Third, the present study only assessed a small number of psychiatric disorders on depression, social phobia, schizophrenia, and avoidant personality disorder. Other common psychiatric disorders such as anxiety disorders and substance abuse disorders should also be inquired in future studies.

Fourth, the present study did not gather information about exposure to suicidality (Chan et al., 2014). Exposure to suicidality via social media platforms could contribute to various public perceptions of suicide (Nathan & Nathan, 2020). Future research should explore the potential role of exposure to suicidality and other risk behaviors such as alcohol intake, substance abuse, and internet addiction in shaping suicide stigma among adolescents and early adults. Fifth, the sample size of adolescents in early and middle adolescence (n = 145) was considerably smaller than those in late adolescence (n = 1,088). Such an imbalance in age distribution of the participants prevents meaningful classification of the adolescents into further subgroups. Further studies with similar subgroup sizes are needed to confirm the measurement invariance of the SOSS-SF across different stages of adolescence from middle school to college years. Finally, the present study did not utilize any procedures to control for random responding or inattention in the responses of this large online sample.

Conclusions

The present innovative study systematically examined the psychometric properties of the SOSS-SF via the ESEM approach and contributed to a deeper understanding of the stigma toward suicide among a large sample of adolescents and early adults in Hong Kong. The current findings demonstrated a robust four-factor structure for the SOSS-SF with satisfactory reliability and measurement invariance across gender, age group, and lifetime suicidal ideation. The four-factor structure provides a more nuanced view of various stigma attitudes by distinguishing between the disgrace and selfishness factors. Having a valid and reliable measure of the diverse stigma facets could facilitate evaluation of antistigma campaigns in suicide prevention research. The use of the SOSS-SF as a brief and invariant assessment tool of the multifaceted nature of stigma toward suicide is recommended.

Research Data

sj-csv-1-asm-10.1177_1073191120976857 – Research Data for Factor Structure and Measurement Invariance of the Stigma of Suicide Scale–Short Form Among Adolescents and Early Adults in Hong Kong

Research Data, sj-csv-1-asm-10.1177_1073191120976857 for Factor Structure and Measurement Invariance of the Stigma of Suicide Scale–Short Form Among Adolescents and Early Adults in Hong Kong by Ted C. T. Fong, Paul S. F. Yip, Milton Y. H. Chan and Rainbow T. H. Ho in Assessment

This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).

Footnotes

Acknowledgements

The authors would like to express their gratitude toward The Boys’ and Girls’ Clubs Association, Caritas, and The Hong Kong Federation of Youths Group for their help in recruitment of participants and all respondents for their participation in the online survey. The authors wish to bestow special gratitude to the generosity of the Charities Trust in supporting the project.

Authors’ Note

The raw data generated from this manuscript will be made available by the authors, without undue reservation, to any qualified researcher.

Author Contributions

TCT formulated the research question, performed literature review, conducted the data analysis, interpreted and wrote the results. PSF designed the study protocol, supervised data collection and interpreted the results. MC contributed to the literature review, data curation, and interpretation of the results. RTH contributed to the discussions and wrote the paper. All coauthors participated in critical editing of the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Hong Kong Jockey Club Charities Trust.

ORCID iD

Ted C. T. Fong

References

Batterham

P. J.

Calear

A. L.

Christensen

(2013a). Correlates of suicide stigma and suicide literacy in the community. Suicide and Life-Threatening Behavior, 43(4), 406-417. https://doi.org/10.1111/sltb.12026

Batterham

P. J.

Calear

A. L.

Christensen

(2013b). The Stigma of Suicide Scale: Psychometric properties and correlates of the stigma of suicide. Crisis: The Journal of Crisis Intervention and Suicide Prevention, 34(1), 13-21. https://doi.org/10.1027/0227-5910/a000156

Batterham

P. J.

Han

Calear

A. L.

Anderson

Christensen

(2019). Suicide stigma and suicide literacy in a clinical sample. Suicide and Life-Threatening Behavior, 49(4), 1136-1147. https://doi.org/10.1111/sltb.12496

Bilsen

(2018). Suicide and youth: Risk factors. Frontiers in Psychiatry, 9, Article 540. https://doi.org/10.3389/fpsyt.2018.00540

Bridge

J. A.

Goldstein

T. R.

Brent

D. A.

(2006). Adolescent suicide and suicidal behavior. Journal of Child Psychology and Psychiatry, 47(3-4), 372-394. https://doi.org/10.1111/j.1469-7610.2006.01615.x

Caputi

T. L.

Smith

Ayers

J. W.

(2017). Suicide risk behaviors among sexual minority adolescents in the United States, 2015. JAMA, 318(23), 2349-2351. https://doi.org/10.1001/jama.2017.16908

Chan

W. I.

Batterham

P. J.

Christensen

Galletly

(2014). Suicide literacy, suicide stigma and help-seeking intentions in Australian medical students. Australasian Psychiatry, 22(2), 132-139. https://doi.org/10.1177/1039856214522528

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5

Cheung

Y. B.

Law

C. K.

Chan

Liu

K. Y.

Yip

P. S. F.

(2006). Suicidal ideation and suicidal attempts in a population-based study of Chinese people: Risk attributable to hopelessness, depression, and social factors. Journal of Affective Disorders, 90(2-3), 193-199. https://doi.org/10.1016/j.jad.2005.11.018

10.

Corrigan

P. W.

Sheehan

Al-Khouja

M. A.

, & Stigma Suicide Research Team. (2017). Making sense of the public stigma of suicide factor analyses of its stereotypes, prejudices, and discriminations. Crisis: Journal of Crisis Intervention and Suicide Prevention, 38(5), 351-359. https://doi.org/10.1027/0227-5910/a000456

11.

Fong

T. C. T.

Chan

J. S. M.

Chan

C. L. W.

R. T. H.

Ziea

E. T. C.

Wong

V. C. W.

B. F. L.

S. M.

(2015). Psychometric properties of the Chalder Fatigue Scale revisited: An exploratory structural equation modeling approach. Quality of Life Research, 24(9), 2273-2278. https://doi.org/10.1007/s11136-015-0944-4

12.

Forte

Trobia

Gualtieri

Lamis

D. A.

Cardamone

Giallonardo

Fiorillo

Girardi

Pompili

(2018). Suicide risk among immigrants and ethnic minorities: A literature overview. International Journal of Environmental Research and Public Health, 15(7), Article 1438. https://doi.org/10.3390/ijerph15071438

13.

Franklin

J. C.

Ribeiro

J. D.

Fox

K. R.

Bentley

K. H.

Kleiman

E. M.

Huang

Musacchio

K. M.

Jaroszewski

A. C.

Chang

B. P.

Nock

M. K.

(2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143(2), 187-232. https://doi.org/10.1037/bul0000084

14.

Fullagar

Gilchrist

Sullivan

(2007). The construction of youth suicide as a community issue within urban and regional Australia. Advances in Mental Health, 6(2), 107-118. https://doi.org/10.5172/jamh.6.2.107

15.

Han

Batterham

P. J.

Calear

A. L.

Shou

van Spijker

B. A. J.

(2017). Translation and validation of the Chinese versions of the Suicidal Ideation Attributes Scale, Stigma of Suicide Scale, and Literacy of Suicide Scale. Death Studies, 41(3), 173-179. https://doi.org/10.1080/07481187.2016.1214633

16.

Hanel

P. H. P.

Vione

K. C.

(2016). Do student samples provide an accurate estimate of the general public? PLOS ONE, 11(12), Article e0168354. https://doi.org/10.1371/journal.pone.0168354

17.

Hawton

Comabella

C. C. I.

Haw

Saunders

(2013). Risk factors for suicide in individuals with depression: A systematic review. Journal of Affective Disorders, 147(1-3), 17-28. https://doi.org/10.1016/j.jad.2013.01.004

18.

Hawton

Saunders

K. E. A.

O’Connor

R. C.

(2012). Self-harm and suicide in adolescents. Lancet, 379(9834), 2373-2382. https://doi.org/10.1016/s0140-6736(12)60322-5

19.

Henry

P. J.

(2008). Student sampling as a theoretical problem. Psychological Inquiry, 19(2), 114-126.

20.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. https://doi.org/10.1080/10705519909540118

21.

Klonsky

E. D.

May

A. M.

Saffer

B. Y.

(2016). Suicide, suicide attempts, and suicidal ideation. In Cannon

T. D.

Widiger

(Eds.), Annual review of clinical psychology (Vol. 12, pp. 307-330). https://doi.org/10.1146/annurev-clinpsy-021815-093204

22.

Kucukalic

(2017). Stigma and suicide. Psychiatria Danubina, 29(Suppl. 5), S895-S899. http://www.psychiatria-danubina.com/UserDocsImages/pdf/dnb_vol29_noSuppl%205/dnb_vol29_noSuppl%205_895.pdf

23.

Liu

R. T.

Trout

Z. M.

Hernandez

E. M.

Cheek

S. M.

Gerlus

(2017). A behavioral and cognitive neuroscience perspective on impulsivity, suicide, and non-suicidal self-injury: Meta-analysis and recommendations for future research. Neuroscience & Biobehavioral Reviews, 83(December), 440-450. https://doi.org/https://doi.org/10.1016/j.neubiorev.2017.09.019

24.

Mayer

Rusch

Frey

L. M.

Nadorff

M. R.

Drapeau

C. W.

Sheehan

Oexle

(2020). Anticipated suicide stigma, secrecy, and suicidality among suicide attempt survivors. Suicide and Life-Threatening Behavior, 50(3), 706-713. https://doi.org/10.1111/sltb.12617

25.

Muthén

Asparouhov

(2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17(3), 313-335. https://doi.org/10.1037/a0026802

26.

Muthén

L. K.

Muthén

B. O.

(2017). Mplus user’s guide (8th ed.). Muthén & Muthén.

27.

Nathan

N. A.

Nathan

K. I.

(2020). Suicide, stigma, and utilizing social media platforms to gauge public perceptions. Frontiers in Psychiatry, 10, Article 947. https://doi.org/10.3389/fpsyt.2019.00947

28.

Oexle

Feigelman

Sheehan

(2020). Perceived suicide stigma, secrecy about suicide loss and mental health outcomes. Death Studies, 44(4), 248-255. https://doi.org/10.1080/07481187.2018.1539052

29.

Ozturk

Akin

Durna

(2017). Testing the psychometric properties of the Turkish version of the Stigma of Suicide Scale (SOSS) with a sample of university students. Journal of Psychiatric Nursing, 8(2), 102-109. https://doi.org/10.14744/phd.2016.38981

30.

Pelizza

Poletti

Azzali

Paterlini

Garlassi

Scazza

Chiri

L. R.

Pupo

Raballo

(2019). Suicidal thinking and behavior in adolescents at ultra-high risk of psychosis: A two-year longitudinal study. Suicide and Life-Threatening Behavior, 49(6), 1637-1652. https://doi.org/10.1111/sltb.12549

31.

Pitman

A. L.

Stevenson

Osborn

D. P. J.

King

M. B.

(2018). The stigma associated with bereavement by suicide and other sudden deaths: A qualitative interview study. Social Science & Medicine, 198(February), 121-129. https://doi.org/10.1016/j.socscimed.2017.12.035

32.

Pridmore

McArthur

(2009). Suicide and Western culture. Australasian Psychiatry, 17(1), 42-50. https://doi.org/10.1080/10398560802596843

33.

Rutkowski

Svetina

(2017). Measurement invariance in international surveys: Categorical indicators and fit measure performance. Applied Measurement in Education, 30(1), 39-51. https://doi.org/10.1080/08957347.2016.1243540

34.

Salway

Gesink

Ibrahim

Ferlatte

Rhodes

A. E.

Brennan

D. J.

Marchand

Trussler

(2018). Evidence of multiple mediating pathways in associations between constructs of stigma and self-reported suicide attempts in a cross-sectional study of gay and bisexual men. Archives of Sexual Behavior, 47(4), 1145-1161. https://doi.org/10.1007/s10508-017-1019-0

35.

Schmitt

T. A.

(2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29(4), 304-321. https://doi.org/10.1177/0734282911406653

36.

Scocco

Preti

Totaro

Ferrari

Toffol

(2017). Stigma and psychological distress in suicide survivors. Journal of Psychosomatic Research, 94(March), 39-46. https://doi.org/10.1016/j.jpsychores.2016.12.016

37.

Scocco

Toffol

Preti

., & The SOPRoxi Project Team. (2016). Psychological distress increases perceived stigma toward attempted suicide among those with a history of past attempted suicide. Journal of Nervous and Mental Disease, 204(3), 194-202. https://doi.org/10.1097/NMD.0000000000000457

38.

Sheehan

L. L.

Corrigan

P. W.

Al-Khouja

M. A.

, & Stigma Suicide Res, T. (2017). Stakeholder perspectives on the stigma of suicide attempt survivors. Crisis: Journal of Crisis Intervention and Suicide Prevention, 38(2), 73-81. https://doi.org/10.1027/0227-5910/a000413

39.

Van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486-492. https://doi.org/10.1080/17405629.2012.686740

40.

van Spijker

B. A. J.

Batterham

P. J.

Calear

A. L.

Farrer

Christensen

Reynolds

Kerkhof

A. J. F. M

. (2014). The Suicidal Ideation Attributes Scale (SIDAS): Community-based validation study of a new scale for the measurement of suicidal ideation. Suicide and Life-Threatening Behavior, 44(4), 408-419. https://doi.org/10.1111/sltb.12084

41.

Williams

C. L.

Cero

Gauthier

J. M.

Witte

T. K.

(2018). Examination of the latent factor structure and construct validity of the stigma of suicide scale-short form. Death Studies, 42(10), 616-626. https://doi.org/10.1080/07481187.2017.1416431

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.27 MB