Construction and Validation of a Self-report Subjective Emptiness Scale

Abstract

Subjective emptiness is a psychiatric symptom that is primarily assessed and studied as a criterion of borderline personality disorder, even though research suggests that it may have clinical importance beyond that diagnosis. The aim of this series of studies was to develop and validate a standalone self-report measure of subjective emptiness. A systematic, multistep approach to identifying test content was used to generate 88 items that were then trimmed to 53 via ratings of interviews with patients and experts. This preliminary scale was administered to a sample of 544 university students, and a trimmed version was given to two samples oversampled for clinical problems (n = 1,067; n = 1,016). A five-item measure fit a unidimensional model well and had satisfactory internal consistency across these samples. External validity analyses suggested that emptiness, as measured by the Subjective Emptiness Scale, is strongly related to a number of clinical constructs, particularly in the internalizing domain, indicating that emptiness is an important construct to consider in its own right, independent of its presence in the borderline criterion set.

Keywords

emptiness scale instrument measure construction validation

Subjective emptiness has typically been studied as a symptom of borderline personality disorder (BPD), although research suggests that it is closely related to a range of internalizing characteristics, particularly depression (Klonsky, 2008). Theory and research demarcate emptiness as a significant and distinct symptom with a wide range of clinical correlates. However, research has been constrained by the lack of a straightforward measure that is not embedded in BPD assessments and does not include content from related but different constructs. The goal of the three studies presented in this article was to validate a brief unidimensional measure of subjective emptiness, which we define as the experience of profound hollowness and disconnection from self and others, lack of fulfillment and an absence of meaning (Bateman & Krawitz, 2013; Freeman, 2007; Kernberg, 1975; LaFarge, 1989; Levy, 1984; Orbach et al., 2003; Singer, 1977a, 1977b).

Emptiness and Psychopathology

Relative to other borderline features, emptiness is associated with greater impairment across the broadest range of psychosocial domains and is one of the slowest remitting symptoms (Ellison et al., 2016; Zanarini et al., 2007). Research suggests that emptiness is also strongly associated with depression (Klonsky, 2008) and anxiety (Mann et al., 1989). Significant but somewhat weaker associations have also been found with narcissistic and antisocial personality disorders (Gunderson & Ronningstam, 2001; Zerach, 2016), schizophrenia (Zandersen & Parnas, 2018), and dissociation (Rallis et al., 2012). Preliminary research further suggests that emptiness is related to a number of clinically important outcomes, including self-harming behavior such as suicidality, substance abuse, and compulsive sexual behavior (Bandelow et al., 2010; Bandelow & Wedekind, 2015; Blasco-Fontecilla et al., 2013; Blasco-Fontecilla et al., 2015; Callahan, 1996; Delgado-Gomez et al., 2012; Ellison et al., 2016; Khantzian et al., 1990; Klonsky, 2008; Lloyd et al., 2007; Orbach et al., 2003; Rallis et al., 2012; Roos et al., 2015; Segal-Engelchin et al., 2015), emotion dysregulation (Hayes et al., 1996; LaFarge, 1989), social dysfunction (Ellison et al., 2016; Klonsky, 2008; Lamprell, 1994; Peteet, 2011; Richman & Sokolove, 1992), psychiatric hospitalization, absenteeism at work, and reliance on disability benefits (Ellison et al., 2016; Miller et al., 2018).

Despite this initial evidence that subjective emptiness represents a transdiagnostic risk factor for a host of clinically important issues, it has not been subjected to a sustained and focused program of research (Elsner et al., 2017). One consequence is that the boundaries between emptiness and other diagnostic constructs remain unclear. Hierarchical models of psychopathology help bring order to hypotheses about how different clinical constructs relate to one another (Krueger et al., 2018; Wright et al., 2012). In particular, any given construct can be compared and contrasted from others empirically, both in terms of which general domains it is associated with (e.g., internalizing vs. externalizing) and its level of abstraction or breadth (e.g., broad domains, trait spectra, specific symptom; Conway et al., 2019). From this perspective, empirical research suggests that emptiness is a relatively narrow symptom most strongly related to constructs in the broad internalizing domain of psychopathology, inclusive of negative affectivity and detachment spectra (Klonsky, 2008; Mann et al., 1989), whereas externalizing constructs such as aggression and impulsivity tend to be less strongly associated with emptiness (Koenigsberg et al., 2001).

Among internalizing constructs, emptiness has been most closely linked to depression. Klonsky (2008) found that emptiness, conceptualized as a BPD criterion, was strongly related to depression and suicidal ideation, whereas associations with anxiety were no longer significant with depression controlled. Structurally, depression tends to fall within the distress and detachment domains of psychopathology (Clark et al., 1994); we would anticipate emptiness falling in the same region of psychopathological space. However, emptiness can be conceptualized as more specific than depressed mood, and in particular as the experience of an absent or missing self (Kernberg, 1975; LaFarge, 1989; Levy, 1984).

Thus, we conceptualize emptiness as a symptom that is located within the internalizing domain of psychopathology (Wright et al., 2012), specifically representing a blend of negative affectivity and detachment spectra (Kotov et al., 2017), similar to but meaningfully distinct from depression (as articulated in the definition above). Ultimately, the nature, correlates, causes, and consequences of emptiness are empirical questions that will be best answered with the benefit of a valid tool for measuring the construct.

Existing Measures of Emptiness

Given that it is a diagnostic criterion of BPD, a common approach to studying emptiness has been to use a single item from instruments designed to measure BPD (Blasco-Fontecilla et al., 2015; Ellison et al., 2016; Gunderson & Ronningstam, 2001; Klonsky, 2008; Rallis et al., 2012; Stepp et al., 2009). Although this work has helped establish its clinical relevance, there are two significant limitations to this approach to conceptualizing emptiness. First, single items are generally less reliable than multi-item scales. Second, existing research suggests that emptiness is a transdiagnostic construct that merits investigation in its own right, independent of its functioning within the context of a BPD diagnosis.

A related approach has been to use multi-item emptiness scales that are embedded in measures of higher order constructs like BPD. For instance, Poreh et al. (2006) constructed the Borderline Personality Questionnaire using items based on Diagnostic and Statistical Manual of Mental Disorders–Fourth edition (DSM-IV) diagnostic criteria for BPD, including emptiness. The internal consistency of the emptiness subscale was acceptable, ranging from .73 to .81 across three university student samples. However, the correlates of the scale were only assessed at the level of the overall BPD construct, and the contents of the emptiness scale included items involving identity disturbance, loneliness, and boredom. Likewise, Palomares Mora (2017) and Choi et al. (2019) developed emptiness scales in the context of BPD measures; however, these measures are not available in English. The Mental Pain Questionnaire (Orbach et al., 2003) includes an emptiness subscale for which items were generated using transcripts from interviews with individuals in clinical and nonclinical samples. In initial validation work, the emptiness subscale showed acceptable internal consistency (α = .75) and was moderately correlated with both anxiety (r = .37) and depression (r = .43). However, the scale includes items assessing an absence of goals and desire, in addition to those measuring emptiness, suggesting that it is assessing a somewhat broader construct than what is intended by the clinical literature. Hazell (1984) developed the Experienced Levels of Emptiness Scale. Items for this scale were generated using existing theories of emptiness and transcripts from interviews with nonclinical adults and refined using factor analysis. The scale exhibited acceptable internal consistency (α = .92) and a strong relationship with depression (r = .69) in a sample of 265 college students and 5 psychiatric patients. However, validation of this measure beyond these initial results has been limited, and the scale includes content related to alexithymia and depersonalization, again suggesting that it is somewhat broad and possibly multidimensional.

In addition to the three aforementioned unidimensional emptiness scales, two multidimensional instruments exist. Buggs (1996) constructed the two-dimensional 28-item Emptiness Scale by generating items based on theory. Items were then refined via factor analysis using a sample of 201 clinical participants; however, principal components analysis revealed a single stable factor rather than the proposed two-factor structure. Reliability and construct validity were not reported. Moreover, the content of the scale appears to overlap with closely related yet conceptually distinct constructs including loneliness, sense of belonging, and boredom. Ermis-Demirtas (2018) also developed a four-factor multidimensional emptiness measure, the Sense of Emptiness Scale. Principal axis factoring using a sample of 405 college students was used to refine the item pool. All four dimensions showed acceptable reliability ranging from .97 to .98. Scale scores were associated with heightened suicide probability and reduced hope, meaning in life, and resilience. Discriminant validity was not examined.

Each of these multi-item scales has potential utility. However, they were developed in the context of overarching models of other constructs, and they often included heterogeneous item content that may not reflect a core emptiness construct, but other associated albeit distinct dimensions. We sought to focus more exclusively on the core features of emptiness, by eschewing content related to empirically similar but conceptually distinguishable concepts.

Overview of Present Studies

We approached this task by adopting the three steps of construct validation articulated in modern psychometric theory (Jackson, 1970; Loevinger, 1957; Simms & Watson, 2007; Tellegen & Waller, 2008). The first step in this process is to define the construct of interest and generate test content that matches that definition. This requires relatively precise assertions regarding what psychological features and behaviors are and are not aspects of that construct. This is challenging because many psychological features tend to be correlated in nature and in psychological assessment. This step corresponds to the psychometric principle of content validity and establishes the boundaries of the construct of interest. As described in greater detail below, we used a variety of procedures to establish content validity, including literature reviews, sampling of the experiences of people with clinically significant levels of emptiness, and item reviews by academic and experiential experts.

The second step involves testing this definition in terms of its internal structure. To the degree that the initial definition is correct, the contents thought to reflect a single construct should be homogeneous. If multiple dimensions are proposed, those dimensions should be apparent in factor analytic models. We note that emptiness may have different aspects, manifestations, and pathways. Thus, it is possible to develop multidimensional emptiness measures, as has been done by previous groups, particularly if content is heterogeneous. However, we sought to identify the singular core of emptiness, and thus to generate a unidimensional measurement model inclusive of the most central features of the construct. We reasoned that this would provide a useful tool for researchers interested specifically in emptiness, and that the creation of a brief instrument could have clinical utility. Thus, we sought a robust unidimensional model, as free as possible from the impact of correlated but independent factors.

The third step involves examining the associations of the resulting measure (Cronbach & Meehl, 1955). To the degree that the definition is accurate, scores on the measure should correlate in an anticipated pattern, with more closely related constructs being more strongly associated than less related constructs (Campbell & Fiske, 1959; Westen & Rosenthal, 2003). We administered a variety of personality and psychopathology measures to evaluate the convergent and discriminant validity of our subjective emptiness items. Given the nature of emptiness and previous evidence regarding its correlates, we generally expected emptiness to correlate most strongly with constructs that have internalizing and detachment components, such as borderline personality and depression, and to correlate more modestly with externalizing constructs such as antagonism and disinhibition.

Methods and results are presented for each stage of the construct validation process sequentially. Moreover, we report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study. Thus, the following three sections describe methods and results for studies in several samples designed to define emptiness (Study 1), determine its core features and create a unidimensional scale (Study 2), and establish its correlates (Study 3). This research was approved by local institutional review board and all participants were consented. Data, code, and other materials are available at https://osf.io/jqpby/.

Study 1: Defining Emptiness

The fact that subjective emptiness is, by definition, an absence of something makes it difficult to define precisely. We had three main goals in generating item content. The first was to capture the phenomenology of emptiness as closely as possible by basing items on actual personal experiences. The second was to avoid drifting away from the core of the experience by trimming items that seemed to tap closely related constructs such as identity problems or boredom (Hazell, 2003). The third was to maintain a connection to the existing theoretical and empirical literature, as described above.

We identified experience-based content from two data sources. The first data source was from an online search conducted in September, 2016 designed to locate public social media conversations with titles containing the word “feel” and “empty” or “emptiness” (Holtz et al., 2012). Content was included if it focused on the psychiatric symptom of emptiness, as indicated by terms such as “feel” and “describe,” whereas posts and threads focused on spiritual emptiness were excluded. Relevant posts and threads were identified on PsychForums.com, Crazyboards.org, MentalHealthForum.net, Yahoo Answers, Reddit, and Quora. After identifying initial quotes on these sites, each site was searched again for posts or threads that included the keywords. This process resulted in 473 quotes from anonymously posted online public message board threads. To explicate a set of features specifically characteristic of emptiness (Rier, 2007), we identified themes among these quotes using inductive thematic analysis (Boyatzis, 1998; Patton, 1990). Specifically, we generated codes denoting basic meaningful elements evident from a qualitative review of the quotes, then coded each quote into categories (Rier, 2007). We eliminated themes that seemed, based on the theoretical and empirical literature, related to neighbor constructs rather than emptiness per se. The remaining features were: “dead inside,” “like you are not human,” “like you are waiting for something,” “disconnected,” and “not present in your own life.” We used these features to construct an initial definition of the construct (as articulated in the first paragraph of this article).

The second data source was from transcripts of semistructured telephone interviews with 18 individuals with personal experiences of feeling empty (Vogt et al., 2004). This process began with an internet search for mental health influencers (Bloggers and YouTubers) who self-identified as having BPD diagnoses. Of the 66 individuals contacted using information they had posted on their blog or website, 37 responded. Eighteen of these individuals then completed an interview consisting of four parts (https://osf.io/vwk6p/). The first part entailed asking interviewees to describe their subjective experiences of emptiness in various contexts. This included prompts to discuss their internal experiences, the environments in which they had them, and how these experiences affected interpersonal relationships. Next, participants were asked to elaborate on how a metaphor they had previously used to describe emptiness captured their experience (Fainsilber & Ortony, 1987). Third, participants described whether emptiness was associated with the themes identified in the prior content analysis study. Participants were invited to elaborate on these experiences. Finally, participants were asked to distinguish the experience of emptiness from depression.

An initial pool of 88 unique items was generated based on the content from these two data sources (see Table S1 available online:https://osf.io/pnkax/). Our goal was to write unambiguous items that were easy to read, endorsable, and free from jargon and colloquialisms. We next asked two sets of experts to rate the content validity of these items based on the definition we provided them. Thirteen subject matter experts who were identified as experts in personality disorder research and 10 of the 37 individuals who posted publicly about their mental health problems completed content validity surveys adapted from Lawshe (1975). These experts rated each item according to its content relevance, with a focus on the degree to which it captured the core of emptiness as distinct from closely related concepts. They were invited to edit items, suggest new items, and provide qualitative feedback regarding ways to improve content validity. Calculation of Lawshe’s (1975) Content Validity Ratio, an index of agreement among judges, revealed that the expert and patient panels reliably rated 95.45% and 93.18% of the items as useful, respectively. Inclusion of construct irrelevant content was the most commonly cited reason for poor content validity among the experts. Based on the expert ratings and qualitative analyses of these interviews, we refined our initial item set to be more specific, distinct, and evocative. We again required that the items be straightforward, free from jargon or colloquialism, and easy to read. This process resulted in 53 items we expected to reflect the core features of subjective emptiness that we then used in the second stage of the construct validation process (see Table S2 available online: https://osf.io/nmd5v/). Items were rated on a 4-point Likert-type scale from 1 (not at all true) to 4 (very true).

Study 2: Determining the Core Features of Emptiness

We used data from three samples for Study 2, in which we used factor analytic methods to determine the core features of emptiness and trim items to create a unidimensional emptiness measure.

Participants

Sample 1 consisted of undergraduate students from California State University San Marcos who were recruited via the human participant pool website and compensated with course credit (n = 544). Sample 2 (n = 1,067) and Sample 3 (n = 1,016) consisted of adults who responded to online ads seeking participants with psychiatric diagnoses. We only analyzed data for participants who met inclusion criteria (older than 18 years of age, fluent in English) and completed at least 90% of the survey (99% of Sample 1, 75% of Sample 2, and 59% of Sample 3). Participants in Samples 2 and 3 did not receive compensation. Data from these three samples is available at https://osf.io/c3akx/.

Sample 1 included 417 women and 126 men, aged 18 to 46 years (M = 20.22, SD = 3.17), of whom 45% were Hispanic/Latinx, 27% White, 14% Asian/Pacific Islander, 11% Other, and 3% African American. Among participants, 6% reported currently taking psychiatric medication, and 7% reported attending therapy currently.

Sample 2 participants (n = 1,067) were recruited via an announcement (https://osf.io/ynujb/) posted to social media platforms (e.g., Reddit), classified websites (e.g., Craigslist), support groups (e.g., Meetup groups), and online research study listings websites (e.g., www.callforparticipants.com). The recruitment flyer requested participation from individuals with a psychiatric diagnosis who were older 18 years of age and fluent in English. The sample size of 1,067 represents the 75% of original respondents who completed at least 90% of the survey. The sample included 716 women, 342 men, aged 18 to 77 years (M = 29.78, SD = 11.49). The ethnic background of the sample was 82% White, 8% Other, 4% Hispanic/Latinx, 3% Asian/Pacific Islander, and 2% African American. Among participants in Sample 2, 42% completed some college, 25% possessed a bachelor’s degree, 15% earned a high school diploma/GED, 13% earned a postgraduate degree, and 4% did not graduate from high school. Although all participants were asked to participate if they had a psychiatric diagnosis, 94% reported having received a psychiatric diagnosis on the survey; 65% reported currently taking psychiatric medication, and 49% reported attending therapy currently. We did not collect specific information regarding psychiatric diagnoses.

Sample 3 participants were recruited using identical methods as Sample 2. The sample size of 1,016 represents the 59% of original respondents who completed at least 90% of the survey. The low completion rate in Sample 3, relative to Sample 2, likely resulted from the difference in survey length, as described below. It included 572 women, 403 men, and 39 other, aged 18 to 76 years (M = 27.50, SD = 10.13). The sample was 82% White, 7% Other, 5% Asian/Pacific Islander, 4% Hispanic/Latinx, and 2% African American; 41% had completed some college, 23% possessed a bachelor’s degree, 18% earned a high school diploma/GED, 13% earned a postgraduate degree, and 4% did not graduate from high school. Although all participants were asked to participate only if they had a psychiatric diagnosis, 84% reported having received a psychiatric diagnosis; 50% reported currently taking psychiatric medication, and 39% reported attending therapy currently. We did not collect information regarding specific psychiatric diagnoses.

Analyses and Results

As a first step, a single dimension was extracted via categorical exploratory factor analysis (EFA) with diagonally weighted least squares mean and variance-adjusted (WLSMV) estimation in Sample 1 using Mplus version 8 (Muthén & Muthén, 1998-2017). All items in the 53-item model loaded at least moderately onto this first factor (.55-.89), whose eigenvalue of 31.92 explained more than 60% of the item variance. The second eigenvalue was less than 2, suggesting the presence of a robust underlying dimension. Our focus was on refining this core dimension via item trimming to create a reliable measurement tool.

Of the 53 items from Study 1, 36 were discarded due to extreme means or limited variances, failure to meet the fourth grade and lower reading-level criterion, and relatively low item-total correlations (see Table S3 available online: https://osf.io/x95y2/). This resulted in a 17-item scale that we subjected to categorical EFA with WLSMV estimation in Sample 1. Loadings from this model ranged from .70 to .88. The eigenvalue of the first factor was 11.33; eigenvalues of all other factors were below 1.

Categorical confirmatory factor analysis with WLSMV estimation in Mplus version 8 (Muthén & Muthén, 1998-2017) was used in Sample 2 to cross-validate this model. Model fit was assessed using Tucker–Lewis index (TLI), comparative fit index (CFI), and root mean square error of approximation (RMSEA), with cutoff values recommended by Hu and Bentler (1999; TLI and CFI > .95, and RMSEA < .06.) The 17-item model identified in Sample 1 did not fit the data well (Table 1). Based on this first analyses, seven items were discarded due to local dependence and low factor loadings. The resulting 10-item model (Model 2) showed improved albeit suboptimal fit. Hence, three more items exhibiting local dependence were removed. However, removing these three items did not improve model fit as revealed by suboptimal fit statistics for the resulting seven-item model (Model 3). Two more items were discarded due to local dependence. The resulting five-item model (Model 4, see Appendix) fit the data well. We cross-validated this model in Sample 3, where it continued to fit the data well (Table 1). The internal consistencies of the items in this model were .88 and .89 across Samples 2 and 3, respectively, and standardized path coefficients ranged from .82 to .82 (Sample 2) and from .76 to .90 (Sample 3; Table 2).

Table 1.

Fit Indices of Unidimensional CFA Models for the Subjective Emptiness Scale in Two Clinical Samples.

Model	χ²(df)	TLI	CFI	RMSEA [95% CI]
Sample 2 (n = 1,067)
17-Item	2,606.14 (119)	.94	.95	.14 [.14, .15]
10-Item	374.88 (35)	.98	.98	.10 [.09, .10]
7-Item	259.60 (14)	.98	.98	.13 [.12, .14]
5-Item	4.67 (5)	1.00	1.00	.00 [.00, .04]
Sample 3 (n = 1,016)
5-Item	11.43 (5)	1.00	1.00	.04 [.01, .06]

Note. CFA = confirmatory factor analysis; df = degrees of freedom; TLI = Tucker–Lewis index; CFI = comparative fit index; RMSEA = root mean square error of approximation; CI = confidence interval.

Table 2.

Item Characteristics and Standardized Factor Loadings for the Subjective Emptiness Scale in Samples 1 and 2.

	Item	M			SD			CFA Loadings
	Item	S1	S2	S3	S1	S2	S3	S2	S3
1	I feel empty inside.	1.47	2.60	2.69	0.68	1.08	1.10	.82	.90
2	I feel absent in my own life.	1.33	2.47	2.58	0.65	1.15	1.14	.82	.87
3	No matter what I do, I still feel unfulfilled.	1.44	2.71	2.87	0.72	1.10	1.09	.82	.87
4	I feel like I am forced to exist.	1.26	2.41	2.62	0.65	1.24	1.23	.82	.84
5	I feel all alone in the world.	1.38	2.50	2.41	0.70	1.16	1.15	.82	.76

Note. CFA = confirmatory factor analysis; S = sample.

Study 3: Evaluating the External Validity of Emptiness

Having developed a robust, reliable, and unidimensional index of subjective emptiness, our third objective was to examine the convergent and discriminant correlates of the construct across all three samples. We expected patients to have substantially higher scores than students given our understanding of emptiness as a transdiagnostic clinical construct. In terms of correlates with other instruments, we anticipated that SES scores would exhibit strong positive correlations with scores on the emptiness item of a BPD measure. Moreover, we anticipated associations between SES scores and other BPD symptoms, lower meaning in life, and various traits related to the negative affectivity and detachment domains of psychopathology (e.g., negative affectivity, emotional lability, anxiousness, anhedonia, depression, restricted affectivity, and identity problems). We expected weaker correlates with constructs related to externalizing and psychoticism domains, such as impulsive behavior and psychotic thinking.

Our interpretation of effects was guided both by effect size recommendations in the literature (Cohen, 1969; Funder & Ozer, 2019) but also by cautions about applying such benchmarks too rigidly (Bosco et al., 2015). Given that all of our measures were administered by self-report and most of them refer to some sort of problems in living, we expected somewhat stronger correlations than might be expected in the general psychological literature. Our main interest was in distinguishing relatively strong from relatively weak effects. As such we interpreted correlations >.60 as indicating strong convergence, correlations between .30 and .60 as moderate, and correlations <.30 as relatively weak. Thus, we expected negative affectivity and detachment constructs such as borderline personality and depression to have correlations >.60, externalizing constructs such as antagonism and impulsivity to have correlations <.30, and constructs that blend or include some aspects of negative affectivity or detachment to have correlations in the middle range.

Samples 1 and 2 Measures

The Zanarini Rating Scale for Borderline Personality Disorder is a nine-item measure of DSM-IV BPD features (Zanarini et al., 2015). Items are rated on a 4-point Likert-type scale with higher ratings indicating higher levels of psychopathology. For example, the item measuring identity disturbance is rated from 0 (I have had a good idea of who I am) to 4 (I have felt that I had no idea of who I am most of the time). Internal consistency was .82 for Sample 1 and .83 for Sample 2.

The Purpose in Life test–short form is a four-item scale designed to assess the presence of meaning and purpose in life (Schulenberg et al., 2011). Items are rated on a 5-point Likert-type scale (e.g., 1 = I have no goals or aims; 5 = I have clear goals and aims). Internal consistency was .83 for Sample 1 and .85 for Sample 2.

The Barratt Impulsiveness Scale–Brief is an eight-item scale measuring trait impulsivity (Steinberg et al., 2013). Items are rated on a 4-point Likert-type scale from 1 (rarely/never) to 4 (almost always/always). Internal consistency was .72 for Sample 1 and .84 for Sample 2.

The Center for Epidemiologic Studies Short Depression Scale is a 10-item scale measuring depression (Andersen et al., 1994). Items are rated on a 4-point Likert-type scale from 1 = rarely or none of the time (less than 1 day) to 4 = all of the time (5-7 days). Internal consistency was .81 for Sample 1 and .83 for Sample 2.

Sample 3 Measures

The Self-concept and Identity Measure (SCIM) is a 27-item scale measuring clinically relevant identity disturbance (Kaufman et al., 2015). Items include “I have never really known what I believe or value,” and are rated on a 7-point Likert-type scale from 1 (strongly disagree) to 7 (strongly agree). The SCIM is correlated with emotion dysregulation, BPD depression, and other measures of psychopathology. Scores on the SCIM have been found to yield high internal consistency (α = .89), test–retest reliability (α = .93, r = .88; intraclass correlation coefficient = .88), and adequate construct validity. Internal consistency was .92 for the full scale and as follows for the subscales: identity disturbance (α = .88), identity consolidation (α = .83), lack of identity (α = .89).

The Personality Inventory for DSM-5–Short Form (PID-5-SF) is a 100-item scale measuring pathological personality traits (Maples et al., 2015). Items include “I’m good at conning people,” and are rated on a 4-point Likert-type scale from 1 = very false or often false to 4 = very true or often true.

Results

As hypothesized, independent samples t tests revealed large and statistically significant sample effects such that student participants (M = 6.89, SD = 2.78) had lower scores on the SES than participants in Sample 2 (M = 12.70, SD = 4.69, t = −26.45, p < .001, d = 1.43) and Sample 3 (M = 13.17, SD = 4.80, t = −27.94, p < .001, d = 1.52). The correlation between SES scores and age was .04 across all three samples. Gender differences were not statistically significant (t = .15). Ethnicity was examined separately in Sample 1 and Samples 2 and 3 given different rates across these samples. There were no ethnic differences in SES scores in Sample 1 (F = 1.14) or in combined Samples 2 and 3 (F = 1.60).

Correlations between the SES and criterion scales are displayed in Table 3 (Samples 1 and 2) and Table 4 (Sample 3). As expected, scores on the SES were strongly correlated (r > .60) with higher scores on the emptiness symptom of BPD, the overall BPD composite, anhedonia, depression, depressivity, and lack of identity. Notably, correlates were considerably higher with the BPD emptiness symptom (r = .64 in Sample 1 and .73 in Sample 2) than with all other symptoms (Mdn r = .42 in Sample 1 and .37 in Sample 2). Also consistent with our expectations, correlations with externalizing variables such as impulsivity, attention seeking, manipulativeness, callousness, deceitfulness, distractibility, grandiosity, and risk taking were low (r < .30). Other coefficients were spread between these effect sizes, suggesting that emptiness has some role to play in a wide variety of pathological constructs.

Table 3.

Pearson Correlations Between Seven-Item Subjective Emptiness Scale and Validation Measure Scores in Samples 1 and 2.

	Sample 1	Sample 2
Borderline personality composite	.69	.66
Borderline symptoms
Emptiness	.64	.73
Anger	.31	.30
Moodiness	.47	.45
Identity disturbance	.56	.56
Self-destructive behavior	.41	.37
Impulsivity	.33	.26
Suspiciousness or unreality	.39	.48
Efforts to avoid abandonment	.43	.36
Unstable relationships	.38	.30
Related constructs
Depression	.73	.74
Meaning in life	−.55	−.66
Impulsivity	.27	.21

Note. p < .01.

Table 4.

Pearson Correlations Between Subjective Emptiness Scale and Validating Variables in Sample 3.

PID-5 facets
Scale	Correlation	Scale	Correlation
Anhedonia	.79^*	Irresponsibility	.26^*
Anxiousness	.40^*	Manipulativeness	.04
Attention seeking	−.07	Perceptual dysregulation	.38^*
Callousness	.17^*	Perseveration	.38^*
Deceitfulness	.17^*	Restricted affectivity	.18^*
Depressivity	.80^*	Rigid perfectionism	.25^*
Distractibility	.26^*	Risk taking	.10^*
Eccentricity	.30^*	Separation insecurity	.28^*
Emotional lability	.35^*	Submissiveness	.19^*
Grandiosity	.07	Suspiciousness	.43^*
Hostility	.29^*	Unusual beliefs and experiences	.23^*
Impulsivity	.18^*	Withdrawal	.40^*
Intimacy avoidance	.28^*
Self-Concept and Identity Measure
Total	.61^*	Identity disturbance	.35^*
Lack of identity	.77^*	Identity consolidation	−.48^*

Note. PID-5 = Personality Inventory for DSM-5.

p < .01.

As discussed above, one way to situate emptiness in this pattern of correlations is through the use of hierarchical models of psychopathology (Conway et al., 2019; Krueger et al., 2018). In these models, relatively narrow dimensions such as emptiness can be placed within a broader hierarchy, which specifies its configuration in relation to dimensions such as negative affectivity, detachment, psychoticism, antagonism, and disinhibition (Wright et al., 2012). As expected, the pattern of correlates from this study generally suggests that emptiness is more related to internalizing constructs such as negative affectivity and anhedonia and less related to externalizing constructs such as antagonism and disinhibition. Specifically, emptiness was strongly correlated with traits at the intersection of negative affectivity and detachment spectra (Kotov et al., 2017), in particular withdrawal, anhedonia, and depressivity. This suggests that emptiness occupies a somewhat interstitial position between these two relatively broad domains.

Given strong correlations with depression in these data as well as in other research (Klonsky, 2008), we conducted follow-up analyses to evaluate discriminant validity. We compared the fit of models that treated emptiness and depression as a single construct (one factor) to the fit of models that treated emptiness and depression as separate constructs (two factors), again using weighted least squares means and variance adjusted estimation in Mplus in Samples 1 and 2. Chi-square difference tests revealed better fit for the two-factor models in both samples (Sample 1: chi-square difference = 22.10, degrees of freedom = 1, p < .01; Sample 2, chi-square difference = 185.04, degrees of freedom = 1, p < .01). Complete results of this analysis are available at https://osf.io/qetb3/. This supports the distinctness of emptiness and depression, despite the close similarity between these constructs.

Discussion

Subjective emptiness is a serious clinical symptom associated with a variety of mental health problems, significant psychosocial impairment, and high-risk complications (Blasco-Fontecilla et al., 2013; Ellison et al., 2016; Gunderson & Ronningstam, 2001; Klonsky, 2008; Stoffers et al., 2010; Zanarini et al., 2007). Despite its clinical significance, focused research on the identification, causes, and treatment of emptiness has been limited. We reasoned that one barrier to such research is the current lack of measurement tools specifically designed to assess subjective emptiness, independent of BPD.

The Nature of Subjective Emptiness

As such, our goal was to develop and validate a unidimensional self-report measure of subjective emptiness. We first sampled direct quotations from people with clinically significant emptiness, conducted interviews, and reviewed the literature. This process led to the generation of a definition of the construct with 88 candidate items that captured descriptions by clinical theorists and researchers, supplemented by accounts of clinical participants’ lived experiences. We next conducted extensive content validity analyses designed to identify the most salient, core features of the construct, and to trim candidate items to 53 that we would consider in further analyses. We collected data from an undergraduate sample and two groups oversampled for clinical problems in order to identify a robust unidimensional five-item scale, and used that scale to examine the correlates of subjective emptiness.

These studies revealed the core features and conceptual boundaries of subjective emptiness as reported by individuals who have experienced the condition. Core features include feelings of absence from one’s own life, lack of fulfillment, the experience of forced existence, and profound aloneness. As expected, individuals from the clinically saturated groups reported substantially higher rates of subjective emptiness than students. The correlates of emptiness imply a profoundly negative affective state associated with heightened detachment, negative affectivity, and disturbed identity. Specific correlates also offer two more insights regarding the nature of emptiness and its place within more general models of psychopathology.

These correlates were interpreted through the lens of hierarchical models of psychopathology (Clark et al., 2017; Krueger et al., 2018). At a relatively broad level, the emotional dysfunction characterizing emptiness may comprise a blend of features from both the internalizing and detachment domains (Kotov et al., 2017). Our results suggest that emptiness was strongly correlated to constructs from these domains, and more strongly so than more externalizing constructs such as hostility and impulsivity. Moreover, the pattern of validity correlations suggested that emptiness is, similar to depression, interstitial between the negative affectivity (internalizing) and detachment domains (Clark et al., 1994; Wright et al., 2012).

We note that, although the DSM-5 alternative model for personality disorders (American Psychiatric Association, 2013) includes references to emptiness as a specific form of Criterion A dysfunction, the concept is not represented among Criterion B traits. It is also not directly referenced in any other DSM-5 diagnosis. We surmise that the absence of emptiness in Criterion B likely occurred because factor analytic models will tend to collapse similar constructs into the same factor, as occurred in the process of developing Alternative Model of Personality Disorder traits (Krueger et al., 2012). While collapsing similar constructs into unitary dimensions can be a defensible decision based on factor analytic evidence, there is a risk that doing so eliminates clinically important variables that may be relatively rare or which commonly co-occur with other, more common issues. That being said, covariance modeling, and construct validation more generally, is a sufficiently flexible approach to make it possible to reintroduce such constructs in future models. The evidence from this work, if replicated and extended, suggests that future iterations of the Alternative Model of Personality Disorder Criterion B and other similarly focused hierarchical models should consider representing emptiness as a distinct dimension, similar to but distinct from depression and interstitial between negative affectivity and detachment.

These results also speak to a core hypothesis in psychoanalytic models of psychopathology, some of which have posited that emptiness results from conflicted identity (Kernberg, 1975; Kohut, 1977). The general idea is that the individual experiences an inner emptiness because of an inability to incorporate different aspects of the self. This dynamic leads to a dependence on others to experience wholeness and satisfaction. At the same time, the maladaptive sense of self can turn others away through hostility or self-loathing behavior. Thus, the individual will tend to vacillate through different self-states and ultimately experience emptiness as the end result of a vicious cycle. In contrast to this hypothesis, in the current study emptiness was more strongly associated with a lack of identity than identity conflict. This suggests that emptiness as measured by the SES may involve an impoverished identity more than dependence on others to feel complete or instability/disintegration in certain components of identity (e.g., values and opinions).

To replicate this work or further examine the breadth, definition, and hierarchical structure of emptiness across various populations, researchers, and clinicians are encouraged to utilize publicly available data, posted on OSF (https://osf.io/c3akx/). Moreover, they need not be limited to the five-item SES in all contexts. The five items comprising the SES are just a small selection of the content identified as relevant to this domain. Thus, researchers may use the original item set or some subset thereof to continue this research, for example by examining multidimensional models of emptiness.

Limitations and Future Directions

This study had several strengths, including large and clinically diverse samples and extensive content, internal, and external validity analyses including multiple approaches to the examination of construct structure and a wide array of validating measures. Nevertheless, several limitations may affect the interpretability and generalizability of these findings and point to important directions for further research. First, common method bias may have inflated parameter estimates since all constructs were measured using self-report questionnaires. Although self-report is the most intuitive approach to measuring subjective inner states, it would be informative to examine associations between subjective emptiness and outcomes assessed via different methods. We adjusted our interpretation of effect sizes with this effect in mind, and would expect lower and less general correlations between the SES and psychopathology indicators gathered with different methods. We had relatively limited information about participants in Samples 2 and 3; future studies are needed in patient populations with richer diagnostic information and attention checks to replicate these results.

Although we examined a wide array of validating constructs (see also Konjusha et al., in press), we left some important variables out for practical reasons. Examinations of the links between subjective emptiness and clinically important but low base rate phenomena such as self-harming behavior would be particularly informative. Similarly, the utility of the SES for predicting outcomes such as hospitalization and suicidal ideation should be assessed. The relatively high dropout rate in the third sample may have biased results, to the degree that study variables may have been related to dropout.

There was a relative lack of ethnic diversity within the samples. Future studies should examine whether findings from this study generalize to other kinds of samples (e.g., Konjusha et al., in press). To examine the stability and longitudinal correlates of subjective emptiness, longitudinal research at different timescales would be informative. Both of these approaches would also enable more causal inferences regarding the nature and effects of emptiness. Although we maintain that feeling empty is different than feeling anhedonically depressed, it may prove difficult to tease those feelings apart empirically. This is an important question for further research. Such research will benefit from the availability of the SES as a standalone measure of emptiness.

Finally, given its clinical relevance as suggested by this study and previous research, there is a need to better understand the effectiveness of therapeutic strategies that target emptiness. Thus far, efficacious pharmacological (Bellino et al., 2010; Bellino et al., 2011; Bellino et al., 2014; Pascual et al., 2008) or psychotherapeutic approaches (Scheel, 2000; Verheul et al., 2003) have not been identified for treating subjective emptiness, although some preliminary evidence supports group schema therapy (Farrell et al., 2009).

Conclusion

In conclusion, results from this series of studies supported the unidimensionality, internal consistency, and construct validity of the Subjective Emptiness Scale (SES) and identified the core features and correlates of subjective emptiness. This construct has been discussed extensively in the theoretical literature, but has been the subject of relatively limited empirical research. The findings reported here suggest that subjective emptiness is a clinically important and distinct symptom characterized by profound disconnection from self and others, chronic lack of fulfillment, and an absence of meaning, that is strongly associated with a variety of internalizing features involving negative affect, interpersonal detachment, risk for self-harm, and identity problems.

Footnotes

Appendix

Acknowledgements

Special thanks to the subject matter experts who evaluated the content validity of the Subjective Emptiness Scale and to Marie Thomas in the Department of Psychology at California State University San Marcos for suggestions regarding the content of this document.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Office for Training, Research and Education in the Sciences, San Marcos, CA (RISE Grant number GM-64783, 2016-2017)

ORCID iD

Adrian L. Price

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Author. https://doi.org/10.1176/appi.books.9780890425596

Andersen

E. M.

Malmgren

J. A.

Carter

W. B.

Patrick

D. L.

(1994). Screening for depression in well older adults: Evaluation of a short form of the CES-D. American Journal of Preventive Medicine, 10(2), 77-84. https://doi.org/10.1016/S0749-3797(18)30622-6

Bandelow

Schmahl

Falkai

Wedekind

(2010). Borderline personality disorder: A dysregulation of the endogenous opioid system? Psychological Review, 117(2), 623-636. https://doi.org/10.1037/a0018095

Bandelow

Wedekind

(2015). Possible role of a dysregulation of the endogenous opioid system in antisocial personality disorder. Human Psychopharmacology: Clinical and Experimental, 30(6), 393-415. https://doi.org/10.1002/hup.2497

Bateman

A. W.

Krawitz

(2013). Borderline personality disorder: An evidence-based guide for generalist mental health professionals. Oxford University Press. https://doi.org/10.1093/med:psych/9780199644209.001.0001

Bellino

Bozzatello

Rinaldi

Bogetto

(2011). Paliperidone ER in the treatment of borderline personality disorder: A pilot study of efficacy and tolerability. Depression Research and Treatment, 2011, Article 680194. https://dx-doi-org.web.bisu.edu.cn/10.1155/2011/680194

Bellino

Bozzatello

Rocca

Bogetto

(2014). Efficacy of omega-3 fatty acids in the treatment of borderline personality disorder: A study of the association with valproic acid. Journal of Psychopharmacology, 28(2), 125-132. https://doi.org/10.1177/0269881113510072

Bellino

Paradiso

Bozzatello

Bogetto

(2010). Efficacy and tolerability of duloxetine in the treatment of patients with borderline personality disorder: A pilot study. Journal of Psychopharmacology, 24(3), 333-339. https://doi.org/10.1177/0269881108095715

Blasco-Fontecilla

Baca-Garcia

Courtet

Nieto

R. G.

De Leon

. (2015). Horror vacui: Emptiness might distinguish between major suicide repeaters and nonmajor suicide repeaters: A pilot study. Psychotherapy and Psychosomatics, 84(2), 117-119. https://doi.org/10.1159/000369937

10.

Blasco-Fontecilla

de León-Martínez

Delgado-Gomez

Giner

Guillaume

Courtet

. (2013). Emptiness and suicidal behavior: An exploratory review. Suicidology Online, 4, 21-32. http://suicidology-online.com/pdf/SOL-2013-4-21-32.pdf

11.

Bosco

F. A.

Aguinis

Singh

Field

J. G.

Pierce

C. A.

(2015). Correlational effect size benchmarks. Journal of Applied Psychology, 100(2), 431-449. https://doi.org/10.1037/a0038047

12.

Boyatzis

R. E.

(1998). Transforming qualitative information: Thematic analysis and code development. Sage.

13.

Buggs

G. R.

(1996). Emptiness: Assessment, origins, sequelae, and relationship to abuse (Publication No. 9704787) [Doctoral dissertation, California Institute of Integral Studies]. ProQuest Dissertations and Theses Global.

14.

Callahan

(1996). A specific therapeutic approach to suicide risk in borderline clients. Clinical Social Work Journal, 24(4), 443-459. https://doi.org/10.1007/BF02190749

15.

Campbell

D. T.

Fiske

D. W.

(1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105. https://doi.org/10.1037/h0046016

16.

Choi

(2019). Development of the Emptiness Scale for Borderline Personality Disorder. Stress, 27, 98-106. https://doi.org/10.17547/kjsr.2019.27.1.98

17.

Clark

L. A.

Cuthbert

Lewis-Fernández

Narrow

W. E.

Reed

G. M.

(2017). Three approaches to understanding and classifying mental disorder: ICD-11, DSM-5, and the National Institute of Mental Health’s Research Domain Criteria (RDoC). Psychological Science in the Public Interest, 18(2), 72-145. https://doi.org/10.1177/1529100617727266

18.

Clark

L. A.

Watson

Mineka

(1994). Temperament, personality, and the mood and anxiety disorders. Journal of Abnormal Psychology, 103(1), 103-116. https://doi.org/10.1037/0021-843X.103.1.103

19.

Cohen

(1969). Statistical power analysis for the behavioural sciences. Academic Press.

20.

Conway

C. C.

Forbes

M. K.

Forbush

K. T.

Fried

E. I.

Hallquist

M. N.

Kotov

Mullins-Sweatt

S. N.

Shackman

A. J.

Skodol

A. E.

South

S. C.

Sunderland

Waszczuk

M. A.

Zald

D. H.

Afzali

M. H.

Bornovalova

M. A.

Carragher

Docherty

A. R.

Jonas

K. G.

Krueger

R. F.

. . . Eaton

N. R.

(2019). A hierarchical taxonomy of psychopathology can reform mental health research. Perspectives on Psychological Science, 14(3), 419-436. https://doi.org/10.1177/1745691618810696

21.

Cronbach

L. J.

Meehl

P. E.

(1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302. https://doi.org/10.1037/h0040957

22.

Delgado-Gomez

Blasco-Fontecilla

Sukno

Ramos-Plasencia

M. S.

Baca-Garcia

(2012). Suicide attempters classification: Toward predictive models of suicidal behavior. Neurocomputing, 92(1), 3-8. https://doi.org/10.1016/j.neucom.2011.08.033

23.

Ellison

W. D.

Rosenstein

Chelminski

Dalrymple

Zimmerman

(2016). The clinical significance of single features of borderline personality disorder: Anger, affective instability, impulsivity, and chronic emptiness in psychiatric outpatients. Journal of Personality Disorders, 30(2), 261-270. https://doi.org/10.1521/pedi_2015_29_193

24.

Elsner

Broadbear

J. H.

Rao

(2017). What is the clinical significance of chronic emptiness in borderline personality disorder? Australasian Psychiatry, 26(1), 88-91. https://doi.org/10.1177/1039856217734674

25.

Ermis-Demirtas

(2018, August). Sense of emptiness: Development and validation of a scale [Doctoral dissertation]. Texas A&M University, Corpus Christi, Texas. https://tamucc-ir.tdl.org/handle/1969.6/87100

26.

Fainsilber

Ortony

(1987). Metaphorical uses of language in the expression of emotions. Metaphor and Symbol Activity, 2(4), 239-250. https://doi.org/10.1207/s15327868ms0204_2

27.

Farrell

J. M.

Shaw

I. A.

Webber

M. A.

(2009). A schema-focused approach to group psychotherapy for outpatients with borderline personality disorder: A randomized controlled trial. Journal of Behavior Therapy and Experimental Psychiatry, 40(2), 317-328. https://doi.org/10.1016/j.jbtep.2009.01.002

28.

Freeman

(2007). Borderline personality disorder: A practitioner's guide to comparative treatments. Springer.

29.

Funder

D. C.

Ozer

D. J.

(2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2(2), 156-168. https://doi.org/10.1177/2515245919847202

30.

Gunderson

J. G.

Ronningstam

(2001). Differentiating narcissistic and antisocial personality disorders. Journal of Personality Disorders, 15(2), 103-109. https://doi.org/10.1521/pedi.15.2.103.19213

31.

Hayes

S. C.

Wilson

K. G.

Gifford

E. V.

Follette

V. M.

Strosahl

(1996). Experiential avoidance and behavioral disorders: A functional dimensional approach to diagnosis and treatment. Journal of Consulting and Clinical Psychology, 64(6), 1152-1168. https://doi.org/10.1037/0022-006X.64.6.1152

32.

Hazell

C. G.

(1984). A scale for measuring experienced levels of emptiness and existential concern. Journal of Psychology, 117(2), 177-182. https://doi.org/10.1080/00223980.1984.9923674

33.

Hazell

C. G.

(2003). The experience of emptiness. AuthorHouse.

34.

Holtz

Kronberger

Wagner

(2012). Analyzing internet forums: A pratical guide. Journal of Media Psychology, 24, 55-66. https://doi.org/10.1027/1864-1105/a000062

35.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118

36.

IBM Corp. (2013). IBM SPSS Statistics for Windows (Version 22.0) [Computer software].

37.

Jackson

D. N.

(1970). A sequential system for personality scale development. In Spielberger

(Eds.), Current topics in clinical and community psychology (Vol. 2, pp. 61-96). Elsevier. https://doi.org/10.1016/B978-0-12-153502-5.50008-4

38.

Kaufman

E. A.

Cundiff

J. M.

Crowell

S. E.

(2015). The development, factor structure, and validation of the Self-Concept and Identity Measure (SCIM): A self-report assessment of clinical identity disturbance. Journal of Psychopathology and Behavioral Assessment, 37(1), 122-133. https://doi.org/10.1007/s10862-014-9441-2

39.

Kernberg

(1975). The subjective experience of emptiness. In Borderline conditions and pathological narcissism (pp. 213-224). Rowman & Littlefield.

40.

Khantzian

E. J.

Halliday

K. S.

McAuliffe

W. E.

(1990). Addiction and the vulnerable self: Modified dynamic group therapy for substance abusers. Guilford Press.

41.

Klonsky

E. D.

(2008). What is emptiness? Clarifying the 7th criterion for borderline personality disorder. Journal of Personality Disorders, 22(4), 418-426. https://doi.org/10.1521/pedi.2008.22.4.418

42.

Koenigsberg

H. W.

Harvey

P. D.

Mitropoulou

New

A. S.

Goodman

Silverman

Serby

Schopick

Siever

L. J.

(2001). Are the interpersonal and identity disturbances in the borderline personality disorder criteria linked to the traits of affective instability and impulsivity? Journal of Personality Disorders, 15(4), 358-370. https://doi.org/10.1521/pedi.15.4.358.19181

43.

Kohut

, (1977). The restoration of the self. International Universities Press.

44.

Konjusha

Hopwood

C. J.

Price

A. L.

Zimmermann

(in press). Investigating the transdiagnostic value of subjective emptiness. Journal of Personality Disorders.

45.

Kotov

Krueger

R. F.

Watson

Achenbach

T. M.

Althoff

R. R.

Bagby

R. M.

. . . Eaton

N. R.

(2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454.

46.

Krueger

R. F.

Derringer

Markon

K. E.

Watson

Skodol

A. E.

(2012). Initial construction of a maladaptive personality trait model and inventory for DSM-5. Psychological Medicine, 42(9), 1879-1890. https://doi.org/10.1017/S0033291711002674

47.

Krueger

R. F.

Kotov

Watson

Forbes

M. K.

Eaton

N. R.

Ruggero

C. J.

Simms

L. J.

Widiger

T. A.

Achenbach

T. M.

Bach

Bornovalova

M. A.

Carpenter

W. T.

Chmielewski

Cicero

D. C.

Clark

L. A.

Conway

DeClercq

DeYoung

C. G.

Docherty

A. R.

. . . Bagby

R. M.

(2018). Progress in achieving quantitative classification of psychopathology. World Psychiatry, 17(3), 282-293. https://doi.org/10.1002/wps.20566

48.

LaFarge

(1989). Emptiness as defense in severe regressive states. Journal of the American Psychoanalytic Association, 37(4), 965-995. https://doi.org/10.1177/000306518903700405

49.

Lamprell

(1994). From emptiness to loss: A journey from the intrapersonal to the interpersonal. British Journal of Psychotherapy, 10(3), 331-343. https://doi.org/10.1111/j.1752-0118.1994.tb00665.x

50.

Lawshe

C. H.

(1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563-575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x

51.

Levy

S. T.

(1984). Psychoanalytic perspectives on emptiness. Journal of the American Psychoanalytic Association, 32(2), 387-404. https://doi.org/10.1177/000306518403200207

52.

Lloyd

Raymond

N. C.

Miner

M. H.

Coleman

(2007). Borderline personality traits in individuals with compulsive sexual behavior. Sexual Addiction & Compulsivity, 14(3), 187-206. https://doi.org/10.1080/10720160701480337

53.

Loevinger

(1957). Objective tests as instruments of psychological theory. Psychological Reports, 3(3), 635-694. https://doi.org/10.2466/pr0.1957.3.3.635

54.

Mann

D. B.

Laitman

L. B.

Davis

K. L.

(1989). Dementia with coexistent major depression. American Journal of Psychiatry, 146(11), 1472-1478. https://doi.org/10.1176/ajp.146.11.1472

55.

Maples

J. L.

Carter

N. T.

Few

L. R.

Crego

Gore

W. L.

Samuel

D. B.

Williamson

R. L.

Lynam

D. R.

Widiger

T. A.

Markon

K. E.

Krueger

R. F.

Miller

J. D.

(2015). Testing whether the DSM-5 personality disorder trait model can be measured with a reduced set of items: An item response theory investigation of the Personality Inventory for DSM-5. Psychological Assessment, 27(4), 1195-1210. https://doi.org/10.1037/pas0000120

56.

Miller

C. E.

Lewis

K. L.

Huxley

Townsend

M. L.

Grenyer

B. F.

(2018). A 1-year follow-up study of capacity to love and work: What components of borderline personality disorder most impair interpersonal and vocational functioning? Personality and Mental Health, 12(4), 334-344. https://doi.org/10.1002/pmh.1432

57.

Muthén

L. K.

Muthén

B. O.

(1998-2017). Mplus user’s guide (8th ed.). Muthén & Muthén.

58.

Orbach

Mikulincer

Gilboa-Schechtman

Sirota

(2003). Mental pain and its relationship to suicidality and life meaning. Suicide and Life-Threatening Behavior, 33(3), 231-241. https://doi.org/10.1521/suli.33.3.231.23213

59.

Orbach

Mikulincer

Sirota

Gilboa-Schechtman

(2003). Mental pain: A multidimensional operationalization and definition. Suicide and Life-Threatening Behavior, 33(3), 219-230. https://doi.org/10.1521/suli.33.3.219.23219

60.

Palomares Mora

. (2017). El sentimiento de vacío en el Trastorno Límite de la Personalidad: Construcción de un cuestionario sobre sentimiento de vacío en el TLP [Doctoral dissertation]. Universidad Complutense de Madrid, Madrid, Spain. https://eprints.ucm.es/41523/

61.

Pascual

J. C.

Soler

Puigdemont

Perez-Egea

Tiana

Alvarez

Perez

(2008). Ziprasidone in the treatment of borderline personality disorder: A double-blind, placebo-controlled, randomized study. Journal of Clinical Psychiatry, 69(4), 603-608. https://doi.org/10.4088/JCP.v69n0412

62.

Patton

M. Q.

(1990). Qualitative evaluation and research methods. Sage.

63.

Peteet

J. R.

(2011). Approaching emptiness: Subjective, objective and existential dimensions. Journal of Religion and Health, 50(3), 558-563. https://doi.org/10.1007/s10943-010-9443-7

64.

Poreh

A. M.

Rawlings

Claridge

Freeman

J. L.

Faulkner

Shelton

(2006). The BPQ: A scale for the assessment of borderline personality based on DSM-IV criteria. Journal of Personality Disorders, 20(3), 247-260. https://doi.org/10.1521/pedi.2006.20.3.247

65.

Rallis

B. A.

Deming

C. A.

Glenn

J. J.

Nock

M. K.

(2012). What is the role of dissociation and emptiness in the occurrence of nonsuicidal self-injury? Journal of Cognitive Psychotherapy, 26(4), 287-298. https://doi.org/10.1891/0889-8391.26.4.287

66.

Richman

N. E.

Sokolove

R. L.

(1992). The experience of aloneness, object representation, and evocative memory in borderline and neurotic patients. Psychoanalytic Psychology, 9(1), 77-91. https://doi.org/10.1037/h0079323

67.

Rier

D. A.

(2007). Internet social support groups as moral agents: The ethical dynamics of HIV + status disclosure. Sociology of Health and Illness, 29(7), 1043-1058. https://doi.org/10.1111/j.1467-9566.2007.01023.x

68.

Roos

C. R.

Kirouac

Pearson

M. R.

Fink

B. C.

Witkiewitz

(2015). Examining temptation to drink from an existential perspective: Associations among temptation. Purpose in Life, and Drinking Outcomes, 29(3), 716-724. https://doi.org/10.1037/adb0000063

69.

Scheel

K. R.

(2000). The empirical basis of dialectical behavior therapy: Summary, critique, and implications. Clinical Psychology: Science and Practice, 7(1), 68-86. https://doi.org/10.1093/clipsy.7.1.68

70.

Schulenberg

S. E.

Schnetzer

L. W.

Buchanan

E. M.

(2011). The Purpose in Life test-short form: Development and psychometric support. Journal of Happiness Studies, 12(5), 861-876. https://doi.org/10.1007/s10902-010-9231-9

71.

Segal-Engelchin

Kfir-Levin

Neustaedter

S. B.

Mirsky

(2015). Mental pain among female suicide attempt survivors in Israel: An exploratory qualitative study. International Journal of Mental Health and Addiction, 13(4), 423-434. https://doi.org/10.1007/s11469-015-9545-2

72.

Simms

L. J.

Watson

(2007). The construct validation approach to personality scale construction. In Spielberger

(Eds.), Handbook of research methods in personality psychology (pp. 240-258). Guilford Press.

73.

Singer

(1977a). The experience of emptiness in narcissistic and borderline states: I. Deficiency and ego defect versus dynamic-defensive models. International Review of Psycho-Analysis, 4(4), 459-469.

74.

Singer

(1977b). The experience of emptiness in narcissistic and borderline states: II. The struggle for a sense of self and the potential for suicide. International Review of Psycho-Analysis, 4(4), 471-479.

75.

Steinberg

Sharp

Stanford

M. S.

Tharp

A. T.

(2013). New tricks for an old measure: The development of the Barratt Impulsiveness Scale–Brief (BIS-Brief). Psychological Assessment, 25(1), 216-226. https://doi.org/10.1037/a0030550

76.

Stepp

S. D.

Pilkonis

P. A.

Yaggi

K. E.

Morse

J. Q.

Feske

(2009). Interpersonal and emotional experiences of social interactions in borderline personality disorder. Journal of Nervous and Mental Disease, 197(7), 484-491. https://doi.org/10.1097/NMD.0b013e3181aad2e7

77.

Stoffers

Völlm

B. A.

Rücker

Timmer

Huband

Lieb

(2010). Pharmacological interventions for borderline personality disorder. Cochrane Database of Systematic Reviews, 2018(2), Article CD012956. https://doi.org/10.1002/14651858.CD005653.pub2

78.

Tellegen

Waller

N. G.

(2008). Exploring personality through test construction: Development of the Multidimensional Personality Questionnaire. In Boyle

G. J.

Matthews

Saklofske

D. H.

(Eds.), The SAGE handbook of personality theory and assessment: Personality measurement and testing (Vol. 2, pp. 261-292). Sage. https://doi.org/10.4135/9781849200479.n13

79.

Verheul

Van Den Bosch

L. M.

Koeter

M. W.

De Ridder

M. A.

Stijnen

Van Den Brink

(2003). Dialectical behaviour therapy for women with borderline personality disorder: 12-Month, randomised clinical trial in The Netherlands. British Journal of Psychiatry, 182(2), 135-140. https://doi.org/10.1192/bjp.182.2.135

80.

Vogt

D. S.

King

D. W.

King

L. A.

(2004). Focus groups in psychological assessment: Enhancing content validity by consulting members of the target population. Psychological Assessment, 16(3), 231-243. https://doi.org/10.1037/1040-3590.16.3.231

81.

Westen

Rosenthal

(2003). Quantifying construct validity: Two simple measures. Journal of Personality and Social Psychology, 84(3), 608-618. https://doi.org/10.1037/0022-3514.84.3.608

82.

Wright

A. G. C.

Thomas

K. M.

Hopwood

C. J.

Markon

K. E.

Pincus

A. L.

Krueger

R. F.

(2012). The hierarchical structure of DSM-5 pathological personality traits. Journal of Abnormal Psychology, 121(4), 951-957. https://doi.org/10.1037/a0027669

83.

Zanarini

M. C.

Frankenburg

F. R.

Reich

D. B.

Silk

K. R.

Hudson

J. I.

McSweeney

L. B.

(2007). The subsyndromal phenomenology of borderline personality disorder: A 10-year follow-up study. American Journal of Psychiatry, 164(6), 929-935. https://doi.org/10.1176/ajp.2007.164.6.929

84.

Zanarini

M. C.

Weingeroff

J. L.

Frankenburg

F. R.

Fitzmaurice

G. M.

(2015). Development of the self-report version of the Zanarini Rating Scale for Borderline Personality Disorder. Personality and Mental Health, 9(4), 243-249. https://doi.org/10.1002/pmh.1302

85.

Zandersen

Parnas

(2018). Identity disturbance, feelings of emptiness, and the boundaries of the schizophrenia spectrum. Schizophrenia Bulletin, 45(1), 106-113. https://doi.org/10.1093/schbul/sbx183

86.

Zerach

(2016). The mediating role of emptiness and materialism in the association between pathological narcissism and compulsive buying. International Journal of Mental Health and Addiction, 14(4), 424-437. https://doi.org/10.1007/s11469-015-9591-9