Abstract
Self-compassion is a positive psychological construct associated with heightened well-being, but the construct is largely measured via self-report. In a study of divorcing adults (N = 120), we sought to replicate and extend prior research on the association between self-rated and observed self-compassion, the linguistic cues associated with self-rated and observed self-compassion, and the predictive utility of observed self-compassion. Untrained observers rated participants’ stream-of-consciousness recordings about their marriage and separation experience. We found adequate consensus among raters of observed self-compassion and a significant, positive association between self-rated and observed self-compassion. Greater self- and observer-rated self-compassion were associated with less distress at baseline; however, only observed self-compassion was associated with less distress at the final study assessment. Discussion centers on the cues observers use to perceive self-compassion in others and the extent to which behavioral manifestations of affect may shape such ratings.
Romantic breakups, from the end of a first love relationship to divorce after years of marriage, are emotionally challenging events (Sbarra & Manvelian, 2021). For some people, the difficulties of a separation abate rather quickly and the transition out of the relationship, though challenging, is relatively uncomplicated. For other people, roughly 10–15% of people experiencing a divorce (Sbarra et al., 2015), the end of the relationship is fraught with considerable emotional pain and suffering. Often, this pain can be a complicated amalgam of affective experience—yearning, longing, anger, conflict, jealousy, and rejection. How people interact with their own psychological experiences around the separation plays a key role in adjustment over time. This paper addresses the concept of self-compassion in the wake of a marital separation and we consider self-compassion as one psychological strategy people adopt for coping with the end of marriage (and difficult life experiences in general).
Self-compassion—defined as being kind to oneself, holding one’s pain in mindful awareness, and identifying one’s suffering from the perspective of a common humanity (Neff, 2003a)—is associated with improved psychological functioning and coping following stressful events (Allen & Leary, 2010; Neff et al., 2007; Sirois et al., 2015), as well as reduced risk of psychopathology (MacBeth & Gumley, 2012). The construct is generally measured via self-report using the widely employed Self-Compassion Scale (SCS; Neff, 2003b). Though self-compassion may largely be a private, internal psychological process (Neff, 2003b), Sbarra et al. (2012) demonstrated that untrained observers can reliably agree when people are expressing self-compassion via verbalizing their thoughts and feelings and that observer-rated self-compassion predicts psychological adjustment following marital separation.
The present study reports on a conceptual replication of the Sbarra et al. (2012) study and an important extension investigating the association between self- and observer-rated self-compassion. In addition to seeking to replicate the observer-rated self-compassion effect, we explore the linguistic and affective cues observers use to infer the presence of self-compassion and whether observer-rated self-compassion is associated with psychological distress, above and beyond self-reported self-compassion, following marital separation. The inclusion of linguistic and affective variables in this report allows us a deeper understanding of the cues observers may use to perceive self-compassion in others. To the extent that the association between self-reported and observed self-compassion are explained by cues reflecting affect, this would suggest that observers rate participants’ self-compassion as a function of general perceptions of distress (or its absence) and would raise new questions about differences between the observable manifestations of the construct and the internal experience of self-compassion.
Self-compassion and its correlates
Self-compassion, at least as it is conceptualized in modern Western psychology, has its roots in much older Eastern philosophical thought and religion, and most directly Buddhism (Neff, 2003a). Whereas Western society traditionally conceptualizes compassion as compassion for others, Buddhist psychology views compassion for the self and for others as both vitally important aims (Neff, 2003b). In this context, “compassion” is defined as being receptive to and touched by the suffering of oneself or others, such that one is moved to alleviate this suffering while regarding the sufferer non-judgmentally (in the event they have done wrong or failed in some way; Neff, 2003a, 2003b).
Starting in the late 1980s, through a growing dialogue between Buddhism and Western psychology (Neff, 2003a), the concept and practice of “mindfulness” began to be adopted by psychology researchers and clinicians to help treat mental illness (e.g., Kabat-Zinn & Chapman-Waldrop, 1988; Kabat-Zinn et al., 1992). Similarly, Neff (2003a) proposed that the Buddhist conceptualization of self-compassion, compared to self-esteem, could provide a healthier way of relating to the self. At the time, Western psychological research and popular media had largely promoted the advantages of high self-esteem (e.g., Rosenberg, 1979). However, some researchers have cautioned about potential downsides of self-reported high self-esteem, particularly when it is reflective of unstable, extrinsically based, or a component of narcissistic views of the self (Aberson et al., 2000; Baumeister et al., 1993; Damon, 1995; Feather, 1994; Seligman, 1995). Self-compassion, on the other hand, offers a way for people to be more accepting and non-judgmental toward their internal experiences and self-evaluations (Neff, 2003a).
To measure self-compassion, Neff, 2003b developed the SCS. Self-compassion evidences good test-retest reliability and is associated with other constructs as we would expect: positive correlations with self-esteem (but not narcissism), life satisfaction, and emotional processing (i.e., attempts to understand one’s emotions), and negative correlations with depression, anxiety, rumination, and thought suppression. In a systematic review, MacBeth and Gumley (2012) found large effect sizes for the relationship between self-compassion and psychopathology, with higher levels of self-compassion associated with lower levels of psychopathology (r = −.54), depression (r = −.52), anxiety (r = −.51), and stress (r = −.54). These findings were recently replicated in adolescent samples (Marsh et al., 2018). Intervention studies and mediational analyses suggest a causal relationship: Increasing self-compassion leads to improvements in mental health and well-being (Raque-Bogdan et al., 2011; Zessin et al., 2015).
Observer-rated self-compassion
To date, almost all studies have measured self-compassion via self-report with Neff, 2003b SCS having emerged as the most widely used instrument. Though subjective reports of self-compassion are reliable and exhibit construct validity, self-reports of any kind can be subject to response biases (Paulhus & Vazire, 2007). Moreover, and perhaps most importantly, self-reported self-compassion shares critical method variance with other self-report measures of interest, particularly measures that are often used as outcome measures (e.g., well-being). Therefore, exploring the psychometrics of self-compassion as assessed from an observer’s perspective can help reduce the influence of response sets, including social desirability, that can conflate empirical associations when both the predictor and dependent variable are measured via the same method (see Podsakoff et al., 2003). The use of multiple measures can help mitigate response bias and shared method variance artifacts and observational assessments can provide a valuable psychometric path toward accomplishing that.
Another reason we might use observer reports is to avail ourselves of the informational asymmetry between self and other reports and thus obtain more comprehensive and unbiased assessments (Vazire, 2010). Insofar as self-compassion is an observable trait, any differences in the degree to which we can know ourselves to be self-compassionate versus the degree to which others can know ourselves as such would be useful information. According to Vazire’s self-other knowledge (SOKA) model, the extent to which the self or other has access to or can detect or interpret accurately personality trait information depends on that trait’s observability and evaluativeness. For example, neuroticism is predominantly characterized by internalized behaviors (e.g., thoughts and feelings like rumination and sadness), which the self has more access to than the other; in contrast, extraversion is mostly characterized by externalized behaviors (e.g., gregariousness and talkativeness), which is more observable to another person. When trait evaluation has implications for a person’s ego, that person is more likely to engage in ego-protective distortions in self-perception compared to less emotionally invested observers (e.g., seeing themselves as more intelligent or creative than they really are). Though the observability and evaluativeness of self-compassion have not been studied extensively, we believe observer ratings may offer unique insight into the degree to which people treat themselves kindly.
Self-compassion has been characterized as an internal process not necessarily involving the outward expression of emotion (Neff, 2003b), but there is clear evidence it is a reliably observable trait. Neff et al. (2007) discovered that observer-rated self-compassion, assessed via therapists rating participants based on a single-item, global rating of self-compassion, was positively associated with self-ratings of self-compassion (r = .32, p = .05) based on the SCS. Additionally, Sbarra et al. (2012) asked participants, at an initial lab visit, to discuss their marital separation in four-minute stream-of-consciousness recordings that were then rated for self-compassion by independent observers. The observers were “naive” or untrained with respect to the concept of self-compassion and merely asked to complete a modified version of the SCS with the word stems oriented as third-person (rather than first-person) phrases. Using this modified, third-person version of the SCS, the observers were highly reliable in their assessment of participants’ self-compassion (interobserver consensus, ICC(2, k) = .77). Additionally, Sbarra et al. (2012) tested the psychometric utility of observer-rated self-compassion to predict psychological adjustment trajectories. Participants rated as higher in self-compassion at the beginning of the study experienced less divorce-related psychological distress at their first lab visit, as well as 9 months later. This finding adds theoretical nuance to Neff, 2003b suggestion: perhaps self-compassion is largely a private, internal psychological process when people are alone, but in certain interpersonal situations where an individual is openly discussing and expressing their feelings about a painful experience such as divorce, observers—including friends, family, and therapists—may detect clear indicators of self-compassion, some of which may be especially or uniquely detectable to observers alone. 1
A critical limitation of the Sbarra et al. (2012) study was the absence of self-reported self-compassion measurements, which made it impossible to determine the extent to which (a) observer-rated self-compassion was correlated with participants’ subjective reports of self-compassion and (b) observer-rated self-compassion predicted psychological adjustment over-and-above self-reported self-compassion. To investigate the convergent and unique predictive validity of observed self-compassion, the present study sought to replicate and extend the work of Sbarra et al. by measuring and analyzing the relationship between self-reported and observer-rated self-compassion, as well as assessing their unique contributions, if any, to predicting psychological adjustment.
As mentioned, not only is observed self-compassion reliable, but it may provide useful information beyond self-reported self-compassion. People may underreport self-compassion because they lack self-awareness or fear it leading to self-indulgence or weakness (Gilbert et al., 2011; Gilbert & Procter, 2006; Neff, 2004). As a putatively positive construct that is colloquially defined as “being kind to yourself,” self-compassion may be a trait high in evaluativeness—in the sense of being a socially desirable or undesirable trait—thus promoting ego-protective biases that may lead to its underreporting (Vazire, 2010). People may overreport their level of self-compassion, too. Self-accuracy regarding traits is likely higher than other-accuracy for deliberate behaviors and explicit aspects of personality, with this effect potentially mitigated or reversed for automatic behaviors and implicit aspects of personality (Back et al., 2009; Vazire, 2010; Vazire & Doris, 2009). There are likely some aspects of self-compassion that are more deliberate/explicit (e.g., our explicit attitudes about the fallibility of human nature) and some that are more automatic/implicit (e.g., the speed with which we berate ourselves when we make a mistake). Our self-perception of automatic/implicit self-compassion behaviors may be distorted because the situations likely to bring them online are usually cognitively “hot” and filled with intense, negative affect. For example, when we are recounting a recent romantic separation, we may be so consumed by memories and lamentations that we forget about self-kindness in the present moment. We need only reflect on our personal experience and remember how many times a friend has surprised and said to us at such times: “You’re being too harsh on yourself.” Taken together, the under or overreporting of self-compassion may mean observer ratings have the potential to offer unique insight into self-compassion relative to subjective experience; for this reason, we expect observed self-compassion to have predictive validity over-and-above self-reported self-compassion (Oh et al., 2011).
Brunswik Lens Model: cue utilization and validity
As mentioned, the observability of self-compassion is not studied extensively and it remains unclear exactly what cues observers use to detect self-compassion in others. To the extent that observers can again agree on their assessments of self-compassion, an additional goal of this study is to examine the specific verbal behaviors that can serve as indicators of observers’ assessments of self-compassion in others through a stream-of-consciousness paradigm aimed at eliciting such verbal behaviors. One way to approach this goal is via the Brunswik (1956) Lens Model. According to Brunswik, within a given environment, certain stimuli may function as a lens through which observers perceive or evaluate underlying psychological constructs (see Figure 1). Cue utilization is indicated by the extent to which linguistic or affective cues are associated with observer-rated self-compassion. In the absence of an objective, criterion measure of self-compassion, we measured cue validity as the extent to which linguistic or affective cues are associated with self-rated self-compassion. Brunswik Lens Model for Observer Cues of Self-Compassion. LIWC = Linguistic Inquiry Word Count (Pennebaker et al., 2015) software program. As described in the Results, blue “Lens” words are negative emotion words (including both anger and sadness-related words), first-person singular pronouns, words suggesting certainty, words focused on the present, as well as affective valence and intensity.
In the current report, we investigated cue validity and cue utilization by analyzing the linguistic behavior contained in recordings of participants narrating their thoughts and feelings about their marital separation—an action not too different from everyday discussions of painful experiences that people engage in with friends, family, and therapists, and contexts where verbalized expressions of self-compassion may be particularly germane. This approach has been used elsewhere in the study of divorce and romantic breakups. For example, Borelli et al. (2019) found that participants who report greater ongoing attachment to a former partner were more likely to use first-person plural words (“we-talk”) when describing their separation experiences; people who used more first-person plural words were, in turn, more likely to be rated by observers as evidencing greater ongoing attachment to a former partner. We follow a similar approach in this study and consider a broad swath of candidate linguistic cues derived from the Linguistic Inquiry Word Count (LIWC; Pennebaker et al., 2015) software program. Given that (a) first-person word use (e.g., me, myself, and I) reflects greater self-focus and is associated with higher trait-level negative affect (Tackman et al., 2018); (b) depression and rumination involve high levels of self-focus (Nolen-Hoeksema et al., 2008; Smith & Greenberg, 1981); and (c) one component of self-compassion is mindfulness (i.e., non-judgmental present moment awareness), or not becoming over-involved or immersed in one’s own psychological experiences, we believe that first-person singular pronoun use will be associated with less self-compassion. Similarly, Sbarra et al. (2012) discovered that negative emotion words (i.e., words reflecting anxiety, anger, and sadness) and positive emotion words (i.e., words reflecting positive affect and optimism) were associated with lower and higher observer-rated compassion, respectively, and we expect similar results in the current report. In addition, to examine the possibility that observers simply listen for the affective valence or intensity of participants’ responses—and thereby infer self-compassion or its absence as a function of expressed unsettledness—an independent set of observers rated the expressed emotion in the narrative to determine if these ratings serve as possible cues of observed self-compassion.
Present study
Our work on this study was guided by three broad aims. Although the hypotheses for this study were not pre-registered, Aim 1 was to replicate the findings of Sbarra et al. (2012) by examining the inter-rater reliability of self-compassion. Under Hypothesis 1 (H1), we hypothesized that observed self-compassion measured from stream-of-consciousness recordings (SCObserved) would evidence considerable agreement among observers as evidenced by strong interobserver intraclass correlations (>0.75; Cicchetti, 1994; Fleiss & Cohen, 1973). Aim 2 was to extend the prior work in this area and measure the strength and direction of association between SCObserved and self-reported self-compassion. Under Hypothesis 2 (H2), we hypothesized that SCObserved would be significantly positively correlated with self-reported self-compassion as measured via self-report questionnaire (SCSelf). Aim 3 was to understand which linguistic cues are used by observers to rate self-compassion in others and which are valid indicators of self-reported self-compassion. Under Hypothesis 3 (H3), based on previous observation regarding the kinds of language use associated with negative psychological states (Greenberg & Pyszczynski, 1986; Sbarra et al., 2012), we predicted that use of high levels of first-person pronouns (both single and plural; i.e., I, me, my; we, us, and ours), high levels of negative emotion words, and high levels of cognition words (e.g., know and understand) would be negatively associated with higher SCObserved ratings, whereas language associated with positive psychological states, such as high levels of positive emotion words, would be positively associated with higher SCObserved ratings. Under Hypothesis 4 (H4), we also predicted that participants’ expressed affective valence and intensity would be associated with SCObserved such that high levels of observer-rated expressed negative emotion and emotional intensity would be negatively associated with higher SCObserved ratings. We included these variables to explore the possibility that observers rely more holistically on the expressions of affective valence and intensity in their ratings of self-compassion rather than on the verbatim content of the language expressed in the SOC recordings. Under Hypothesis 5 (H5), we hypothesized that the pattern of associations between linguistic cues and SCSelf to be the same pattern of associations predicted under H3 and H4 for SCObserved. Finally, Aim 4 was to assess the predictive validity of observer-rated self-compassion. Under Hypothesis 6 (H6), replicating and extending the findings of Sbarra et al. (2012), we hypothesized that higher levels of SCObserved would be associated with better self-reported adjustment and that this effect would hold even when controlling for SCSelf.
Method
Participants
The present study involves data that were collected as part of a broader National Institutes of Health-funded investigation (R01HD069498) of adult adjustment following a romantic separation or divorce; other reports are published from this study (Bourassa, Hasselmo, & Sbarra, 2019; Manvelian et al., 2018; Milek et al., 2018; O’Hara et al., 2020; Tackman et al., 2018) but none have focused on self-compassion. Participants were 137 community-dwelling adults (41 male; age mean = 43.2, SD = 10.79, range: 21–65, median = 44) recruited in Tucson, Arizona, who reported a recent marital separation (mean months since separation = 3.73 months, SD = 2.08). Participants reported being in their relationships an average of 14.83 years before the separation (SD = 9.03, range: 2–38, median = 12.50) and were recruited through the local family and conciliation court, online and newspaper advertisements, and divorce recovery support groups. Sixty-one percent of the sample reported their race as White (non-Hispanic), 20.7% reported they were Hispanic, 5.0% reported being African American, 2.1% reported their race as Native American, 2.1% reported they were Asian, while 5.7% indicated their race was “Other.” Nine participants (6.6%) were students residing in the United States.
Power analyses
Bivariate correlations between self-compassion measures and linguistic and affective cues.
Note. SCSelf = self-reported self-compassion; SCObserved = observer-rated self-compassion from stream-of-consciousness recording; Pos. emotion = positive emotion words; Neg. emotion = negative emotion words; Cog. words = cognitive words; Pres. focus = present focus; Aff. valence = affective valence; Aff. intensity = affective intensity.
a p < .05.
b p < .01.
Attrition analyses
In this study, 137 participants comprised the sample at the initial visit. Of these, 120 participants provided stream-of-consciousness recordings about their relationship breakup (described below) that were then used for observer ratings of self-compassion. A comparison of those with stream-of-consciousness recordings (n = 120) to those without stream-of-consciousness recordings (n = 17) revealed no significant differences in age, sex, race, and initial levels of psychological distress (see composite index of psychological distress in Measures) at the first visit (all ps > .05).
Observers
Eight undergraduate students (7 female, 1 male) served as raters and ranged in age from approximately 19 to 22 years old. All observers were unacquainted with the participants whom they rated.
Procedure
As mentioned, these data were collected as part of a broader study investigating the impact of marital separation on physical and psychological health over time. Participants completed five study assessments, each separated by 1 month and involving a stream-of-consciousness (SOC) recording and completion of questionnaires assessing their adaptation to the separation experience. In an attempt to approximate Sbarra et al.'s (2012) repeated-measures study design within this broader study, the present study only coded and included in analyses the initial visit’s SOC recording, and the questionnaire packets included self-report measures of psychological adjustment, with the first laboratory visit packet also containing self-report measures of self-compassion, length of relationship and time since separation, self-esteem, optimism, and attachment style (see Measures).
At the first laboratory visit, after participant informed consent was reviewed and confirmed, a research assistant administered the instructions detailing how to complete the stream-of-consciousness (SOC) exercise (see Supplemental Materials). Specifically, the research assistant informed participants that they would be imagining their former partner for 30 seconds and then speaking into a recorder about their recent breakup in a stream-of-consciousness fashion for 4 minutes. Once the research assistant left the room, the participant closed their eyes and imagined a detailed image of their former partner for 30 seconds. After this 30-second period ended, the research assistant entered the room and placed a digital voice recorder on the table in front of the participant before exiting the room again. The participant then began speaking about his or her recent breakup or divorce experiences in a stream-of-consciousness fashion. After this four-minute period ended, the research assistant knocked on the door, which signaled to the participant to stop speaking, and entered the room to collect the recorder.
Eight raters coded the participants’ four-minute SOC recordings for self-compassion using the modified SCS-SF (see Measures). To minimize observer burden and maximize independence of ratings, we undertook two precautions. First, four observers rated only even participants’ SOC files and the other four observers rated only odd participants’ SOC files such that each participants’ SOC file was rated by four observers. To minimize artifacts from increasing familiarity with the concept (i.e., raters may become “more accurate” over time), the directionality of sound files was counterbalanced such that four raters rated assigned files in ascending order (e.g., participants 1 to 120) and the remaining four rated in descending order (e.g., participant 120 to 121). In both odd and even observer groups, two observers rated files in ascending order and two rated files in descending order.
Measures
Self-reported and observer-rated self-compassion
As mentioned above, the majority of previous research has focused on outcomes following measurement of self-rated self-compassion using the Self-Compassion Scale–Short Form (SCS-SF; Raes et al., 2011). Participants self-reported their levels of self-compassion using the SCS-SF prior to the initial visit (α = .86). Observer ratings of self-compassion were collected by observers rating each participant’s stream-of-consciousness recording from the initial visit using a modified version of the SCS-SF in which the first-person oriented, Likert-type items are reworded to be third-person statements (Raes et al., 2011; Sbarra et al., 2012). For example, “When something painful happens I try to take a balanced view of the situation” would be reworded as “When something painful happens s/he tries to take a balanced view of the situation.” Under Aim 1 (below), we evaluate the reliability of the observed self-compassion composite index using the standardized scores from the four pairs of raters; thus, the observed self-compassion composite consisted of four items derived from eight observers.
Self-reported separation-related psychological distress composite
We used a composite index to measure participants’ self-reported separation-related psychological distress (SRPD). The SRPD composite served as the primary outcome variable and included measures of depressive symptoms (BDI; Beck et al., 1996), separation-related emotional intrusion (IES-R; Creamer et al., 2003), self-concept disruptions (Lewandowski & Bizzoco, 2007), and grief (Prigerson et al., 1995). We used a percent of maximum possible (POMP) scoring system such that higher scores on the composite index indicated increased self-reported separation-related distress (Cohen et al., 1999; see Supplemental Materials for further details). We computed the composite index score separately for each of the five study assessments. This composite is a reliable and valid index of adults’ psychological responses to the end of marriage (Bourassa et al., 2017, 2019; O’Hara et al., 2020; Sbarra et al., 2013).
Linguistic categories
SOC recordings were transcribed and analyzed using the Linguistic Inquiry and Word Count program (LIWC; Pennebaker et al., 2015). The LIWC program detects and assigns each word to a linguistic category—such as first-person pronouns, positive emotion words, and negative emotion words—and reports the percentage of each linguistic category as a percentage of total words. In this way, it allows determining what proportion of a participant’s uttered words captured within the SOC recording falls within specific linguistic categories.
Affective valence and intensity
An independent set of six raters assessed the affective valence and intensity of participants’ SOC recordings on 100-point scales; these raters rated only affective valence and intensity, and they did not provide ratings of self-compassion. Raters evaluated affect valence (α = .80) by choosing a number that best represented how the participant felt when talking about their separation experience, with anchors being 0 = Negative, 50 = Neutral, and 100 = Positive. They also assessed affect intensity (α = .81) by choosing a number that best represented how intensely the participant experienced their emotions when talking about their separation experience, anchors of 0 = Neutral, 50 = Somewhat intense, and 100 = Very intense.
Covariates
Similar to Sbarra et al. (2012), we sought to examine the role of self-compassion after accounting for other individual differences, including time since relationship separation, length of relationship prior to separation, self-esteem, optimism, attachment anxiety and avoidance, and categories of language use as analyzed by the LIWC program (see Linguistic categories). Self-esteem was measured using a single item asking participants to rate whether they have high self-esteem on a 7-point Likert scale ranging from 1 (Strongly disagree) to 7 (Agree strongly; Robins et al., 2001). Optimism was measured using the 10-item Life Orientation Test-Revised (LOT-R; Scheier et al., 1994), which included three items measuring optimism, three items measuring pessimism, and four items serving as fillers, with each item rated on a 4-point scale ranging from 0 (Strongly disagree) to 4 (Strongly agree). Adult attachment style was measured using the 12-item Experiences in Close Relationships Scale–Short Form (ECR-S; Wei et al., 2007). The ECR-S measures individuals along two dimensions of attachment: anxious and avoidant attachment. Anxious individuals typically fear rejection and abandonment, whereas avoidant individuals tend to find intimacy uncomfortable and strive for independence.
Data analytic plan
Aim 1 was to replicate the work of Sbarra et al. (2012) by examining the inter-rater reliability of self-compassion. Under H1, we expected SCObserved to be a reliable measure of self-compassion as evidenced by strong interobserver agreement. To evaluate H1, we calculated the intraclass correlation coefficients for the four raters of each participant SOC recording at the first lab visit, specifically ICC(1,k). Aim 2 was to extend the prior work in this area and measure the strength and direction of association between SCObserved and self-reported self-compassion. Under H2, we predicted that self-reported self-compassion as measured via the SCS-Short Form (SCSelf) would be positively associated with observer-rated self-compassion based on stream-of-consciousness recordings (SCObserved). To evaluate H2, we analyzed the zero-order correlation between SCSelf and SCObserved. Aim 3 was to understand which linguistic cues were used by observers to rate self-compassion in others and which were valid indicators of self-reported self-compassion. Under H3, we predicted that participant use of language generally related to negative psychological states would be negatively associated with higher SCObserved ratings, whereas that language generally related to positive psychological states would be positively associated with higher SCObserved ratings. Under H4, we predicted that participants’ expressed affective valence and intensity would be associated with SCObserved such that high levels of observer-rated expressed negative emotion and emotional intensity would be negatively associated with higher SCObserved ratings. Under H5, we hypothesized that the pattern of associations between linguistic cues and SCSelf to be the same pattern of associations predicted under H3 and H4 for SCObserved. Aim 4 was to investigate the predictive validity of self-compassion. Under H6, we hypothesized that higher levels of SCObserved would be associated with better self-reported adjustment at the first study visit after accounting for SCSelf. To examine H6, and following the basic model examined by Sbarra et al. (2012), we specified a series of multilevel analyses examining the association between self- and observer-observed self-compassion and the POMP-scored separation-related psychological distress (SRPD) composite over time.
Results
Aim 1: consensus among raters of observed self-compassion
Aim 1 was to replicate the work of Sbarra et al. (2012) by examining the degree of consensus among raters in their assessments of participants’ levels of self-compassion. Hypothesis 1 (H1) sought to replicate the inter-rater agreement for SCObserved reported by Sbarra et al. In both the present study and Sbarra et al., each SOC file was rated by four observers, supporting the comparability of these two studies’ intraclass correlations. In this study, the intraclass correlation ICC(1, 3) = .65, 95% CI [.52, .74]. 2 Previous research suggests that an ICC of 0.65 indicates “good” inter-rater agreement (Cicchetti, 1994; Fleiss & Cohen, 1973) and the reliability observed here is comparable to the ICCs obtained by Holleran and Mehl (2008) in their analysis of studies involving observers rating SOC writings for Big Five personality traits (though Holleran and Mehl reported ICC(1, 1) instead of ICC(1, k)). Nevertheless, an ICC of 0.65 effectively means that more than half of the variance lies between participants.
Aim 2: self-other agreement in rated self-compassion
Aim 2 was to measure the strength and direction of association between SCObserved and SCSelf. Under Hypothesis 2 (H2), we predicted that SCObserved would be significantly positively correlated with SCSelf. This prediction was accurate, with participants rated highly in self-compassion by observers also reporting greater self-reported self-compassion, r = .25, p = .006, 95% CI [.12, .39]. For comparison, Holleran and Mehl (2008) reported an accuracy correlation between judges’ impressions of personality traits and participant personality self-reports of .40.
Aim 3: linguistic cues of self-compassion
Aim 3 (encompassing H3, H4, and H5) investigated the potential linguistic cues used by observers to rate self-compassion. As shown in Table 1 (and Figure 1), correlational analyses reveal that participants rated higher in self-compassion from their SOC recordings used fewer first-person singular pronouns, negative emotion words (including both anger and sadness-related words), words suggesting certainty, and words focused on the present. Only negative emotion words were significant predictors of SCSelf. We also observed that SCSelf was associated with observer ratings of affective valence and not intensity, whereas SCObserved was associated with both affective valence and intensity. Given these associations, we conducted a mediation analysis to examine whether the two markers closely associated with both SCSelf and SCObserved—negative emotion words and observer-rated affective valence—operated indirectly to explain the association between SCSelf and SCObserved (see Figure 2). The total indirect effect was reliably different from zero, b = .20, SE = .06, 95% CI [.09, .34], as were both of the individual mediators. The total effect was accounted for entirely by the two mediators. The positive indirect path for negative emotion words reflects the fact that people who reported greater self-compassion used fewer negative emotion words to describe their marital separation, and, in turn, observers rated people who used fewer negative emotion words as expressing greater self-compassion. People who reported greater self-compassion were also observed to express greater positive affective valence, and, in turn, the expression of positive affective valence (identified by a set of independent observers) was associated with greater observer-rated self-compassion. Thus, it appears that when observers use the absence of negative emotion words and overall positive affective valence to assess self-compassion, they are attending to expressions also indicative of self-rated self-compassion. Multiple mediation model predicting self-reported self-compassion from observer-rated self-compassion via the mediators of affective valence and negative emotion words. Standard errors are presented in parentheses. * p < .05; ** p < .01; *** p < .001.
Aim 4: self-compassion ratings and psychological adjustment
Results from multilevel models illustrating the effects of self-reported and observer-rated self-compassion on separation-related psychological distress.
Note. SCSelf = self-reported self-compassion; SCObserved = observer-rated self-compassion from stream-of-consciousness recording.
a p < .05.
b p < .01.
c p < .001.
Another goal of the Sbarra et al. (2012) report was to determine if the association between SCObserved and SRPD remained after accounting for relevant competing predictors. Insofar as self-reported “good outcomes” are associated with self-reported constructs that reflect positive coping, the simple inverse correlation between self-compassion and an indicator of separation-related distress is not especially interesting; however, if this effect persists once we account for a range of relevant covariates, this would provide stronger evidence for the specificity of self-compassion in separation-related adjustment. Table 2 (Model 2) displays the results of this analysis. Both self-compassion variables remained significant predictors of distress when controlling for gender, time since separation, relationship length, self-esteem, optimism, anxious and avoidant attachment, and expression of positive and negative emotion words. Female participants reported significantly less SRPD at baseline. None of the other variables were reliably different from zero. 3
Finally, Sbarra et al. (2012) also reported that people observed to be lower in self-compassion decreased in SRPD more quickly over time (likely due to regression to the mean), and evidenced greater SRPD at the final assessment period. As shown in Table 2 (Model 3), we found that neither self-reported self-compassion nor observer-rated self-compassion interacted with time, indicating that lower baseline self-compassion was not associated with faster declines in SRPD over time. Because SRPD declines over time, it is important to check whether the baseline (i.e., initial intercept effects) persist over time as well. We re-centered the model so the intercept represented the final occasion of measurement. At the final occasion, controlling for all the variables in Model 2, SCObserved remained a significant predictor of SRPD (p = .01), but SCSelf did not (p = .14). In other words, the unique portion of observer-rated self-compassion—that is, the portion of the construct that is unrelated to self-report self-compassion—measured at baseline was correlated with separation-related distress 4 months later. Participants who were observed to be higher in self-compassion at the baseline assessment reported less SRPD at the final study assessment.
Discussion
Divorce is an emotionally challenging event that can be difficult in even the best of circumstances. How people psychologically interact with their thoughts and feelings about these experiences can play a critical role in the course of their adjustment (Sbarra et al., 2015). One internal coping strategy in the face of stressful life events is self-compassion, and the present study sought to conceptually replicate and extend prior research on observed self-compassion among divorcing adults (Sbarra et al., 2012). To elicit observable self-compassion behaviors, we used a stream-of-consciousness paradigm with recently separated adults to foster a situation where verbalizations of self-compassion (or its absence—self-recrimination, low mindfulness, and a self-centered perspective on one’s distress) would be likely. We investigated whether independent observers could reliably discern self-compassion in these stream-of-consciousness recordings, and whether these ratings were associated with self-reported self-compassion. In addition, the current report explored the linguistic and affective cues associated with observed and self-reported self-compassion, as well as whether observed self-compassion was associated separation-related distress over-and-above self-reported self-compassion at the outset of the study and over time.
We expected to replicate the strong inter-rater reliability of self-compassion found in Sbarra et al. (2012) and predicted that observer ratings of self-compassion would agree to a strong degree, as evidenced by strong intraclass correlations (ICCs) above 0.75. However, the ICC found in the present study of ICC(1, k) = .65 was below 0.75 and the ICC reported in Sbarra et al. (2012) of ICC(2, k) = .77. Given that the 95% confident interval of the current [.52, .74] crosses the 95% confidence interval for observed self-compassion in Sbarra et al.’s sample [.73, .80], we conclude that we replicated the prior effect and that untrained observers can reliably discern variation in self-compassion. Previous research suggests that ICCs between 0.60 and 0.74 still indicate “good” inter-rater agreement (Cicchetti, 1994; Fleiss & Cohen, 1973), and that an ICC of 0.65 falls within the typical range achieved in similar types of studies (Holleran & Mehl, 2008). The slightly lower ICC in the current project, compared to Sbarra et al. (2012), may be explained by a small methodological difference between the studies: Half of the SOC recordings in the current project were rated by one set of four observers, whereas the other half were rated by another set of four observers, for a total of eight observers. In the prior report, each SOC recording was rated by the same set of four observers, hence reducing variability among observers and their ratings compared to the present study. Therefore, though the present study’s inter-rater reliability of self-compassion was slightly weaker than Sbarra et al.’s, observer consensus on self-compassion ratings in the present study was still good, and procedural differences may have contributed to this discrepancy. Finally, given the difference between self- and other-accuracy for personality traits low in observability (with the self generally being more accurate) is reduced to the degree the observer is acquainted with the target, future studies might explore the relative reliability of observer assessments of self-compassion as a function of acquaintance (e.g., strangers, friends, and romantic partners; see Vazire, 2010).
A limitation of the Sbarra et al. (2012) study was the absence of self-reported self-compassion measurement and the inability to determine how observer-rated self-compassion was related to self-reported self-compassion. To study the unique and predictive utility of observed self-compassion, the current report addressed this oversight and showed that participants who reported being more self-compassionate were also rated by others as more self-compassionate. This finding is consistent with Neff et al.’s (2007) discovery that observer-rated self-compassion, assessed via therapists rating participants based on a single-item, global rating of self-compassion, was positively associated with self-ratings of self-compassion based on the SCS. The present study extends Neff et al. (2007) by measuring observer-rated self-compassion using the short form version of the self-report scale and demonstrates that non-therapist laypersons can also reliably detect self-compassion in others. Together, these results suggest that people can indeed infer the presence of self-compassion in others and emphasizes the point that measurements of the same trait from multiple methods are related and yet assess unique variance (Oh et al., 2011).
To study the observability of self-compassion, the current project also explored which linguistic and affective cues observers use to assess self-compassion in others and which are potentially valid indicators of self-rated self-compassion. We found that participants rated by others as higher in self-compassion tended to use fewer first-person singular words, negative emotion words (including both anger and sadness-related words), words suggesting certainty, and words focused on the present; on the other hand, the only linguistic cue associated with self-reported self-compassion was use of fewer negative emotion words. Additionally, we discovered that participants judged to be more self-compassionate were also more likely to be judged as expressing more positive affect and less emotional intensity, but that only increased positive affect, and not affective intensity, was associated with both greater self-rated and observer-rated self-compassion. Our mediation analyses, which found that negative emotion words and observer-rated affective valence during the stream-of-consciousness task fully explained the association between self- and observer-rated self-compassion, suggests that both self and other may associate self-compassion with the expression of positive affect and lack of negative emotion word use when recounting difficult experiences. Indeed, these observable affective expressions may be part-and-parcel of the self-kindness dimension of self-compassion—it may well be that both self and other are unable to make fine-grained distinctions about the specific nature of self-compassion and observe its presence at the most global level in terms of broad expression of affect. Moreover, it is possible that observers are noting linguistic markers of self-compassion that are valid but not perceived by the self. This interpretation is in line with the self versus other informational asymmetry purported by the SOKA model: observers may be more attentive to the ways in which we convey difficult experiences that are indicative of self-compassion, ways that may be imperceptible to the self too occupied with recounting. On the other hand, it is also possible that self-compassion is a relatively hidden phenomenon and what we call observed self-compassion involves the detection of linguistic markers unrelated to actual self-compassion. These points may help us understand the relatively limited correlation between observed and self-reported self-compassion and strengthen the case for developing an objective measure of self-compassion.
Another goal of the present study was to explore whether observer-rated self-compassion predicted psychological adjustment over-and-above self-reported self-compassion. We found that observer-rated self-compassion was predictive of separation-related distress at baseline and up to 4 months later, and that this relationship remained after accounting for competing predictors (gender, optimism, etc.) and self-reported self-compassion. This finding is consistent with our hypothesis that observer-rated self-compassion may be tapping aspects of self-compassion that are associated with separation-related distress over time beyond self-reported self-compassion. Whether through ego-protective response biases or lack of self-awareness, people may under- or overreport aspects of self-compassion that meaningfully impact their psychological adjustment. Observers, on the other hand, who are not subject to these biases (but of course susceptible to other biases), may shine a light on these hidden aspects of self-compassion undetectable through self-report alone, strengthening the case for multimethod approaches to self-compassion (and personality) assessment. Self-reported self-compassion, though a reliable and valid measure, may not fully capture an individual’s degree of self-compassion. However, the alternative interpretation mentioned above applies here, too: Self-compassion may be an obscure process, and observer ratings of self-compassion may be detecting something other than self-compassion, such as participants’ general emotional well-being. Again, which interpretation is correct is difficult to determine without an unimpeachable criterion measure of self-compassion.
Exploring this alternative interpretation further, one strong possibility is that observers are simply cueing into or picking-up on participants’ emotional distress about the separation experience when making their ratings of self-compassion. This perspective is consistent with the fact that the affective valence and negative emotion word variables fully accounted for the association between self- and observer-rated self-compassion. In addition, as noted in Footnote 3, when we accounted for the affective intensity and valence variables in the multilevel models predicting separation-related psychological distress, the effect of observed self-compassion was eliminated. These facts suggest that the observers tune-in to the affective cues as a means of determining variation in self-compassion. But is this surprising? An absence of self-kindness would be an expression of emotional pain; the opposite of mindfulness is perseverative self-focus, which is highly associated with distress in the wake of a separation (Bourassa et al., 2019); the opposite of “common humanity” is self-blame and recrimination. These are all affective experiences, and the affective aspects of self-compassion may be the easiest for an observer to perceive. The present study’s objective was to determine whether observed self-compassion accounts for variation in emotional well-being following marital separation. We show this to be the case, and it is not particularly informative to explain away the effects of observed self-compassion via the more general construct of affective expression. Rather, it remains an open question as to which variables most explain resilience after divorce. The present study identified observed self-compassion as a candidate variable, and we look forward to future studies that identify other variables with the same or greater degree of explanatory power and specificity as observed self-compassion.
Future Directions
As mentioned, a key limitation in the present study is the lack of an objective, criterion measure of self-compassion. Though observers may offer a unique perspective and provide a fuller picture of a person’s degree of self-compassion when combined with self-rated self-compassion, they both nonetheless suffer from the fact that they are highly subjective measurements. To further establish the construct validity of self- and observer-rated self-compassion, we need a greater understanding of their real-world manifestations. The present study attempted to elicit self-compassion behaviors through participant narration of marital separation experiences and measured them via structured observer assessments, but the assessment of personality traits and their associated behaviors are different exercises and should not be confused (Vazire & Mehl, 2008). Future studies should develop more objective, behavioral measures of self-compassion, including self-compassion behavior coding protocols. For example, though the present study measured linguistic and affective correlates of self-compassion, a more specific (and arguably more useful) behavior coding protocol would be to count the number of times a participant berates (or forgives) themselves when recalling a difficult experience. Such a behavioral criterion could further demonstrate the predictive validity of self- and observer-rated self-compassion.
The development of more comprehensive measures of self-compassion has important implications for psychotherapeutic work with clients experiencing a marital separation and divorce. As mentioned, improving self-compassion is a goal for many interventions aimed at enhancing mental health and well-being (Raque-Bogdan et al., 2011; Zessin et al., 2015). Indeed, improving self-compassion is the primary intervention target of the increasingly popular and empirically supported Mindful Self-Compassion Program (Neff, 2020; Neff & Germer, 2013). To track client progress, then, clinicians may benefit from a range of tools that help them accurately assess their clients’ levels of self-compassion. As discussed, a multimethod approach to measuring self-compassion is advantageous, with the present study finding that both self-reported and other-reported self-compassion may contribute distinct, valuable information about a person’s self-compassion and psychological adjustment. Clinicians may administer the standard, self-report SCS developed by Neff, 2003b to their clients, but clinicians might also complete a structured instrument (like the one used in the current report), that allows them to assess their clients’ self-compassion. Additionally, our results suggest that clinicians may more accurately detect a client’s degree of self-compassion if they pay attention to the affective valence of their speech and their use of negative emotion words. The creation of a behavioral criterion for self-compassion may not only support the development of more construct-valid personality assessments, but it may also help clinicians look for more specific, telltale signs of self-compassion (or lack thereof) in the verbal expressions of their clients.
In addition to the lack of a behavioral criterion for self-compassion, the results of this study should be understood in terms of its other limitations. There were some procedural differences that hindered our ability to make exact comparisons between the current report and Sbarra et al. (2012). For example, when using observers to rate participant self-compassion, the present study used eight raters compared to Sbarra et al.’s four raters. Therefore, it is not obvious whether the slightly lower inter-rater reliability achieved in the present report provides an updated estimate of the inter-rater reliability of observed self-compassion, or whether it reflects mere procedural differences. Moreover, the present study tracked participants monthly over 4 months, whereas Sbarra et al. tracked participants every 3 months over a 9-month period. Although those lower in observed self-compassion began and ended each study higher in separation-related distress, it is unclear whether observer-rated self-compassion in the present study predicts psychological adjustment up to 9 months later, as found in Sbarra et al. In addition, the current study did not collect information about participant gender identity, sexual orientation, disability status, or socioeconomic or educational status. Future observed self-compassion studies should measure these demographic factors to enhance the generalizability and/or specificity of results, including exploring any variance in observed self-compassion within these group identities. Finally, it is important to remember that this is a study about self-compassion in the wake of a marital separation. Consequently, the results may not generalize to the experiences of self-compassion in general or to events other than the end of marriage, and it will be important for future studies to examine the overlap between self-report and observed self-compassion in other contexts.
Conclusion
The present study suggests people have some knowledge about their own self-compassion and the self-compassion of others, and that this fact has important implications for well-being. Self-reported and other-reported self-compassion are related but distinct, with each contributing unique information about a person’s psychological health. Furthermore, though Neff, 2003b points out the internalized nature of self-compassion, the current report provides a strong case for the observability of self-compassion, at least when observers have access to people’s stream-of-consciousness thoughts about their breakup. We highlighted the shared affective expressions consistent with both self-rated and observed self-compassion, but also their linguistic and affective differences. However, to further study the construct validity of self- and other-rated self-compassion, and to promote a truly comprehensive, multimethod approach to self-compassion measurement, we believe the development of a behavioral criterion for self-compassion is imperative. Combining self-, other-, and behavioral ratings of self-compassion may be an invaluable tool for clinicians who aim to improve the self-compassion of their clients and wish to track this progress over time. Nevertheless, the results of the present study suggest that others may indeed perceive how kindly we treat ourselves, and their insight is both one we are not privy to and yet may influence the course of our well-being. If we do not already, perhaps now is a good time to start paying attention to how others perceive how kindly we treat ourselves.
Supplemental Material
sj-pdf-1-spr-10.1177_02654075211047238 – Supplemental Material for Emotional recovery following divorce: Will the real self-compassion please stand up?
Supplemental Material, sj-pdf-1-spr-10.1177_02654075211047238 for Emotional recovery following divorce: Will the real self-compassion please stand up? by Ronald F Chau, Widyasita N Sawyer, Jeff Greenberg, Matthias R Mehl and David A Sbarra in Journal of Social and Personal Relationships
Footnotes
Author contributions
WNS, RFC, and DAS developed the paper concept. WNS, RFC, and DAS performed the analyses. RFC and DAS drafted the manuscript. JG and MRM provided critical revisions. WNS, MRM, and DAS organized and provided critical oversight of data collection and coding. WNS, MRM, and DAS designed the study. All authors approved the final version of the manuscript for submission.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the National Institute of Child Health and Human Development (R01HD069498). Ronald F. Chau’s work on this paper was supported in part by an American Australian Association Graduate Education Scholarship.
Open research statement
As part of IARR’s encouragement of open research practices, the authors have provided the following information. This research was not pre-registered. The data used in the research are available. The data can be obtained at: https://osf.io/6hrkg/?view_only=ef444cff9be9452781f834416619ec2e or by emailing:
or by emailing:
Open Science Practices
Supplemental material
Supplementary material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
