Prolonged Grief,Posttraumatic Stress,Anger,and Revenge Phenomena Following Homicidal Loss

Abstract

This study examined associations of symptoms of Prolonged Grief Disorder (PGD), Posttraumatic Stress Disorder (PTSD), anger, and revenge thoughts and feelings with different categories of negative cognitions and indices of anxious and depressive avoidance behaviors, among 331 homicidally bereaved individuals. Outcomes showed that symptom levels of PGD and PTSD were higher among individuals who had more negative cognitions about the self, were more inclined to catastrophically misinterpret their grief reactions, and tended to engage in depressive avoidance. Anger was strongly associated with negative cognitions about the self. Revenge thoughts and feelings were associated with pessimistic cognitions and anxious avoidance of loss-related stimuli.

Keywords

bereavement homicide cognitions avoidance

Introduction

The death of a loved one due to murder is one of the most stressful events that people can experience. Homicidally bereaved individuals have to deal with the pain related to the separation and the emotional consequences of the traumatic event that caused the death. The violent nature of the loss gives rise to both symptoms of grief (e.g., yearning, searching) and posttraumatic stress symptoms (e.g., intrusive thoughts about the homicide) that may be intertwined with one condition inflating the other (Kristensen, Weisæth, & Heir, 2012). In addition, more than other types of losses, homicidal losses can lead to media attention, contact with the criminal justice system, and stigmatization—constituting additional burdens for bereaved individuals (Armour, 2006; Gintner, 2001; Rynearson, 1984).

Research examining the emotional consequences of homicidal loss is scarce (Mastrocinque et al., in press; Nakajima, Masaya, Akemi, & Takako, 2012; van Denderen, de Keijser, Kleen, & Boelen, 2015). However, the few studies that have been conducted on this topic have provided some evidence that homicidally bereaved individuals experience higher levels of Posttraumatic Stress Disorder (PTSD) and Prolonged Grief Disorder (PGD)—a syndrome of persistent, debilitating grief reactions (e.g., yearning, difficulties accepting the death, and difficulties moving on in life; see Prigerson et al., 2009)—compared with individuals bereaved by accident, suicide, and natural and nonviolent deaths (Amick-McMullan, Kilpatrick, & Resnick, 1991; Currier, Holland, & Neimeyer, 2006; Murphy, Johnson, Wu, Fan, & Lohan, 2003). Furthermore, more than other losses, homicidal loss can give rise to anger and revenge thoughts and feelings directed toward the person held responsible for the death (cf. Rynearson, 1984; Stuckless & Goranson, 1992). Anger and revenge thoughts and feelings are likely to block grief resolution by maintaining a ruminative focus on why the loss occurred and how it could have been prevented, and by reducing engagement in social activities that could foster adjustment (cf. Gäbler & Maercker, 2011; Orth, Montada, & Maercker, 2006). These and other health consequences of homicidal loss result in many different mental, physical, emotional, and social needs of family and friends of the homicide victim providing a challenge to their coping resources (Mastrocinque et al., in press).

Cognitive Behavioral Perspective on Psychopathology After Homicidal Loss

Contemporary theories of bereavement-related psychopathology propose that negative cognitions and avoidance behaviors mediate the development and maintenance of emotional distress following loss (e.g., Boelen, Van den Hout, & Van den Bout, 2006; Maccallum & Bryant, 2013; Shear & Shair, 2005). For example, drawing from cognitive approaches of PTSD (e.g., Ehlers & Clark, 2000), Boelen et al.’s (2006) cognitive behavioral conceptualization of PGD highlights the importance of negative cognitions about the self, life, the future, and catastrophic misinterpretations of grief reactions, as well as two forms of avoidance behavior, termed “anxious avoidance” and “depressive avoidance” in the exacerbation and maintenance of acute grief responses. Negative cognitions about the self, life, and the future refer to explicit assumptions reflecting a negative self-view (“I am a worthless person since she or he died”), a negative view on life (“Life is meaningless since she or he died”), and a pessimistic outlook on the future (“I will never get happy again”). Catastrophic misinterpretations of grief reactions refer to negative, fear-evoking meanings that bereaved individuals may assign to their grief reactions (“If I would elaborate on the implications of this loss, I’d go mad,” “If I would allow my tears to come up, I would lose control”). Anxious avoidance refers to the avoidance of stimuli reminding of the irreversibility of the separation that is driven by the fear that confronting these reminders will be unbearable. Depressive avoidance concerns the withdrawal from social, occupational/educational, and recreational activities that could facilitate adjustment (Boelen et al., 2006; Eisma et al., 2013).

Several prior studies—in samples predominantly comprised of individuals confronted with a loss to illness and unintentional injury—have provided evidence that these six cognitive behavioral variables, that is, negative cognitions about (a) the self, (b) life, (c) the future, (d) catastrophic misinterpretations, (e) anxious avoidance, and (f) depressive avoidance behaviors, are associated with PGD both cross-sectionally (Boelen & Klugkist, 2011) and prospectively (e.g., Boelen, van den Hout, & van den Bout, 2013; Eisma et al., 2013; Van der Houwen, Stroebe, Schut, Stroebe, & Van den Bout, 2010). In addition, there is evidence that targeting these variables in the treatment of PGD alleviates distress (Boelen, de Keijser, van den Hout, & van den Bout, 2011).

The Current Study

The current study was designed to examine the relationship between these six cognitive behavioral variables and emotional distress among people confronted with homicidal loss, which has not been done before. In so doing, we used self-reported data from an ongoing research program on the psychological sequelae of homicidal loss (see van Denderen, de Keijser, Gerlsma, Huisman, & Boelen, 2014; van Denderen, de Keijser, Huisman, & Boelen, in press). In the paper by van Denderen et al. (in press), we reported detailed findings on the severity of PGD and PTSD in this sample, whereas the other article (van Denderen et al., 2014) focused on the associations of PGD and PTSD with dispositional and situational revenge. Building on these prior studies, the current study investigated cognitive behavioral variables in PGD, PTSD, anger, and revenge thoughts and feelings. Studying this issue was deemed important because there is little knowledge about the underlying mechanisms of emotional distress among homicidally bereaved individuals (Rynearson, Schut, & Stroebe, 2013). Enhancing this knowledge could help to inform methods to identify homicidally bereaved people at risk for elevated distress and to develop interventions to alleviate this distress. Our specific aims were as follows.

First, we aimed to examine to what extent the six cognitive behavioral variables distinguished in Boelen et al.’s (2006) cognitive behavioral conceptualization, that is, negative cognitions about (a) the self, (b) life, (c) the future, (d) catastrophic misinterpretations of grief reactions, (e) anxious avoidance, and (f) depressive avoidance were correlated with symptom levels of PGD, PTSD, and anger, as well as with revenge thoughts and feelings. Second, we aimed to examine which of these cognitive behavioral variables explained a unique proportion of variance in these four dependent variables when controlling for the shared variance between these cognitive behavioral variables and the variance explained by sociodemographic and loss-related variables. Third, we examined which of these cognitive behavioral variables had a unique association with these four dependent variables when also taking into account the shared variance between these dependent variables. This was expected to yield clues about the specific cognitive behavioral correlates of distinct characteristics of PGD, PTSD, anger, and revenge thoughts and feelings following homicidal loss.

Because—to our knowledge—no studies have yet examined the linkage between these cognitive behavioral variables and distress following homicidal loss, we had no particular hypotheses about the cognitive behavioral variables most strongly associated with the outcome variables. However, based on prior research (see Boelen et al., 2013), we could expect the severity of symptoms of PGD and PTSD to be positively associated with negative cognitions and avoidance behaviors. More speculatively, it was deemed conceivable that the two other dependent variables included in this study—anger and revenge thoughts and feelings—would be elevated among bereaved individuals with increased negative cognitions about themselves, their life, their future, and the “bearableness” of their grief reactions. It was also expectable that elevated fears to confront stimuli that reminded of the loss (implicated in increased anxious avoidance) and difficulties to continue usual roles and activities (implicated in increased depressive avoidance) would be associated with more severe anger and revenge thoughts and feelings. The expectation that anger and revenge were likely elevated among people with more negative cognitions and avoidance behaviors was based on the notion that these cognitions and behavior likely decrease the ability to turn attention away from thoughts about the violence, unfairness, and perpetrator of the homicidal loss (cf. Gäbler & Maercker, 2011). Yet, we had no specific hypotheses about which of the six cognitive behavioral variables would be uniquely associated with PGD, PTSD, anger, and revenge thoughts and feelings following homicidal loss.

Method

Participants and Procedure

Data were available from 331 homicidally bereaved individuals who participated in an ongoing research program on correlates and treatment of psychopathology following homicidal bereavement. In total, 136 participants were recruited via Victim Support the Netherlands—a governmental organization in which professionals, mostly social workers, offer practical and legal support to homicidally bereaved families. A further 195 participants were recruited via support organizations all providing low-threshold support and advice to homicidally bereaved individuals, offered by people who were homicidally bereaved themselves. Potential participants recruited from these sources were all required to be at least 6 months out from the homicide. During the period of data collection (mid-2011 to mid-2013), support staff from Victim Support the Netherlands handed out questionnaires to all people confronted with homicidal loss they came in contact with. We (the researchers) handed over 504 questionnaire packets to support staff from this organization and 136 packets were received back. Thus, the response rate was at least 27.2%. However, because not all 504 questionnaire packets were actually distributed among bereaved individuals, the true response rate (i.e., the number of returned questionnaires relative to the number of distributed questionnaires) is likely higher; the exact rate is unknown because the number of questionnaires that were actually distributed was not systematically registered. In addition, 333 questionnaire packets were distributed via support organizations, 195 (58.6%) of which were returned. All participants were offered written information about the goals of the research program and were informed that by completing and returning the questionnaires, they provided consent for the anonymous use of their data for research purposes. Victim Support the Netherlands and the support organizations participating in the recruitment approved this procedure.

Participants recruited from Victim Support the Netherlands (n = 136) and those recruited via support organizations (n = 195) differed such that the former group was younger and more recently bereaved and included more bereaved parents (p < .01). To increase statistical power for the current study, groups were combined to form the present study sample. Sociodemographic and loss-related variables were included as control variables in our main analyses to rule out the effect of differences between groups on the associations between cognitive behavioral variables and symptom scores. Table 1 summarizes sociodemographic and loss-related characteristics of the present study sample. As can be seen, most participants were women, aged 52 years on average, approximately 7 years removed from their loss. Almost half of the participants had suffered the loss of a child. In the majority of the cases (92.7%), a suspect had been identified and/or arrested and/or convicted; although the exact number of unsolved cases was unknown, participants confronted with such cases were a small minority (less than 100% minus 92.7% is 7.3%).

Table 1.

Characteristics and Symptom Scores of the Sample (N = 331).

Demographic characteristics
Gender, N (%)
Men	113 (34.1)
Women	218 (65.9)
Age (years), M (SD)	52.6 (15.5)
Loss characteristics
Deceased is, N (%)^a
Partner/spouse	25 (7.6)
Child	157 (47.4)
Parent	41 (12.4)
Sibling	52 (15.7)
Other relative	47 (14.2)
Article 1. Number of years since loss, M (SD)	6.9 (6.5)
Has a suspect been identified, arrested, and/or convicted? N (%)
Yes	307 (92.7)
No	24 (7.3)
Recruited via, N (%)
Victim Support the Netherlands	136 (41.1)
Support organizations	195 (58.9)
Article 11. Symptom scores, M (SD); range
Article 111. Prolonged Grief Disorder (ICG-r, 29 items version)	90.3 (23.5); 30-150
Posttraumatic Stress (PSS-SR)	19.3 (12.9); 0-51
Anger (SCL-90)	9.7 (4.8); 6-30
Article 1V. Revenge thoughts and feelings (TRIM)	22.0 (7.8); 5-35

Note. ICG-r = Inventory of Complicated Grief–Revised version; PSS-SR = PTSD Symptom Scale–Self-Report version; SCL-90 = Symptom Checklist–90; TRIM = Transgression Related Interpersonal Motivation.

Relationship to the deceased was unknown for nine participants.

Measures

Inventory of Complicated Grief–Revised (ICG-r)

The ICG-r is a 30-item self-report measure tapping symptoms of PGD and other markers of disturbed grief (e.g., “I feel myself longing and yearning for . . .”). Participants rate the presence of symptoms in the last month on 5-point scales ranging from 1 = never to 5 = always with summed item scores yielding an overall PGD severity score. The English version (Prigerson & Jacobs, 2001) and the 29-item Dutch version used in this study (Boelen, Van den Bout, De Keijser, & Hoijtink, 2003) have adequate psychometric properties. In the present study, 7 items representing cognitive grief reactions (i.e., Items 10, 14, 18, 21, 23, 25, and 26; see Boelen, Van den Bout, De Keijser, & Hoijtink, 2003, Table 1 for items) and 1 item that represents avoidance (Item 13) were excluded from the main analyses to avoid content overlap between the dependent variable (i.e., PGD) and the independent variables (negative cognitions and avoidance). This shortened ICG-r included 21 items representing emotional grief reactions only. Cronbach’s alpha of the 21-item ICG-r was .94.

PTSD Symptom Scale–Self-Report version (PSS-SR)

The PSS-SR is a 17-item measure of PTSD as defined in Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR; American Psychiatric Association [APA], 2000). Participants rate the frequency of symptoms on 4-point scales ranging from 0 = not at all to 3 = 5 or more times per week/almost always. The English (Foa, Riggs, Dancu, & Rothbaum, 1993) and Dutch versions (Engelhard, Arntz, & van den Hout, 2007) have good psychometric properties. Participants were asked to keep in mind an event from their past that they were bothered by most during the preceding month, to provide a brief description of the event, and then to rate the 17 items with this event in mind. In the present study, 2 items representing persistent avoidance associated with the loss (the first and second symptom of the DSM-IV-TR avoidance cluster) were excluded from the main analyses to avoid content overlap between the dependent variable (i.e., PTSD) and the independent variables (anxious and depressive avoidance). Cronbach’s alpha of this 15-item shortened PSS-SR was .93.

Subscales “Self,” “Life,” “Future,” and “Catastrophic Misinterpretations” from the Grief Cognitions Questionnaire (GCQ)

The GCQ is a 38-item measure of loss-related negative cognitions developed by Boelen, van den Bout, and van den Hout (2003). In the present study, four of its nine subscales were used, namely, three subscales tapping global negative beliefs about the Self (six items, for example, “Since . . . is dead, I feel less worthy”), Life (four items, “My life is meaningless since . . . died”), and the Future (five items, “I don’t have confidence in the future”), and one subscale representing Catastrophic Misinterpretations of one’s own grief reactions (four items, “Once I would start crying, I would lose control”). Participants rated their agreement with each item on 6-point scales with anchors disagree strongly and agree strongly. Internal consistencies of these four subscales were α = .92, α = .95, α = .93, and α = .92, respectively.

Depressive and Anxious Avoidance in Prolonged Grief Questionnaire (DAAPGQ)

The DAAPGQ is a nine-item measure comprising five items tapping Depressive Avoidance (“I avoid doing activities that used to bring me pleasure, because I feel unable to carry out these activities”) and four items tapping Anxious Avoidance (“I avoid situations and places that confront me with the fact that . . . is dead and will never return.”). Participants rated their agreement with each item on 6-point scales ranging from not at all true for me to completely true for me. Psychometric properties of the subscales are adequate (Boelen & van den Bout, 2010; Eisma et al., 2013). In the present study, their internal consistencies were α = .94 (Depressive Avoidance) and α = .76 (Anxious Avoidance).

Subscale “Hostility” from the Symptom Checklist–90 (SCL-90)

This 6-item subscale is part of the SCL-90, a 90-item self-report measure of psychopathology (Derogatis, 1983; Dutch version by Arrindell & Ettema, 2003). It assesses thoughts, feelings, and actions that are characteristic of the affect state of anger. Its items include “Feeling easily annoyed or irritated,” “Temper outbursts that you could not control,” “Having urges to beat, injure, or harm someone,” “Getting into frequent arguments,” “Shouting or throwing things,” and “Having urges to break or smash things.” Participants rate the occurrence of these symptoms in the preceding week, on 5-point scales ranging from 1 = not at all to 5 = very much. We used this SCL-90 Hostility subscale as an index of anger, acknowledging that its items represent a limited operationalization of anger. In the current study, the α was .89.

Subscale “Revenge” from the Transgression Related Interpersonal Motivation (TRIM)

The subscale “Revenge” from the TRIM (McCullough et al., 1998; Dutch version by Gerlsma, Lugtmeyer, Van Denderen, & De Keijser, 2013) was used to measure revenge thoughts and feelings. This five-item subscale taps the motivation to seek revenge and to see harm come to the offender following a transgression (e.g., “I wish that something bad would happen to him or her”). McCullough et al. (1998) used a 5-point response scale. In our study, items were rated on 7-point scales (1 = disagree strongly, 7 = agree strongly) with higher scores pointing at stronger revenge thoughts and feelings. Psychometric properties of the English (McCullough et al., 1998) and Dutch versions (Gerlsma et al., 2013) are adequate. In the current sample, the α was .84.

Results

Preliminary Analyses

Table 1 shows mean scores, together with standard deviations and score ranges, on the measures of PGD (ICG-r), posttraumatic stress (PSS-SR), anger (SCL-90 Hostility), and revenge thoughts and feelings (TRIM). In total, 163 participants (49.2%) scored above the cutoff score of 90 on the summed 29 items of the ICG-r, which is a threshold for “PGD caseness” (see Boelen, Van den Bout, De Keijser, & Hoijtink, 2003). In addition, 104 participants (31.4%) passed the threshold for “PTSD caseness” according to the DSM-IV-TR based scoring rule put forth by Brewin, Andrews, and Rose (2000) with symptom scores of at least 2 (“2-4 times a week/half of the time”) on at least one reexperiencing, three avoidance, and two hyperarousal symptoms. Mean scores on the SCL-90 Hostility subscale were significantly higher than the mean score of M = 7.2 found in a representative sample from the normal Dutch population studied by Arrindell and Ettema (2003); t(321) = 9.08, p < .001. As noted, TRIM items were scored on 7-point scales, whereas in other studies (e.g., McCullough et al., 1998), items are scored on 5-point scales. The mean score in our sample was M = 22 (SD = 7.8) and was M = 16.4 (SD = 5.2) when item scores were transformed to 5-point scales; this score was significantly higher compared with McCullough et al.’s (1998) sample of university undergraduates’ exposure to interpersonal transgressions (M = 8.7, SD = 4.5), t(302) = 25.6, and their sample of psychology students betrayed or hurt in a close relationship (M = 8.99, SD = 5.1), t(302) = 24.5, ps < .001.

Dependent and independent variables did not demonstrate unacceptable levels of skew or kurtosis (i.e., none had positive standardized skewness values that exceeded 2 or positive standardized kurtosis values that exceeded 7; Tabachnick & Fidell, 2007).

Correlational Analyses

Pearson correlations between the study measures are shown in Table 2. As can be seen, at the p < .001 level, the four dependent variables were all significantly associated with the six cognitive behavioral variables. This indicates that higher levels of PGD, PTSD, anger, and revenge thoughts and feelings were associated with stronger endorsement of negative cognitions about the self, life, the future, and catastrophic misinterpretations, and the tendency to engage in anxious avoidance and depressive avoidance. The measures of PGD (ICG-r) and PTSD (PSS-SR) both included one anger/irritability item. When the anger/irritability item was removed from the ICG-r, the correlation between PGD and anger (SCL Hostility subscale) remained r = .53 (as in Table 1); when the anger/irritability item was removed from the PSS-SR, the correlation between PTSD and anger went from r = .63 to r = .61. Thus, intercorrelations of anger with PGD severity and PTSD severity were not due to content overlap between the measures tapping these symptoms.

Table 2.

Correlations Among Study Measures.

	PGD (ICG-r)	PTSD (PSS-SR)	Anger (SCL-90)	Revenge (TRIM)	Negative self (GCQ)	Negative life (GCQ)	Negative future (GCQ)	Misinterpretations (GCQ)	Anxious avoidance (DAAPGQ)
PTSD (PSS-SR)	.76
Anger (SCL-90)	.53	.63
Revenge (TRIM)	.36	.24	.29
Negative self (GCQ)	.69	.69	.63	.24
Negative life (GCQ)	.72	.70	.54	.28	.82
Negative future (GCQ)	.73	.68	.53	.34	.72	.86
Misinterpretations (GCQ)	.72	.66	.51	.32	.65	.68	.74
Anxious avoidance (DAAPGQ)	.51	.48	.40	.31	.56	.51	.50	.58
Depressive avoidance (DAAPGQ)	.73	.71	.53	−.25	.70	.77	.77	.66	.52

Note. All correlations are significant at p < .001. PGD = Prolonged Grief Disorder; ICG-r = Inventory of Complicated Grief–Revised version; PTSD = Posttraumatic Stress Disorder; PSS-SR = PTSD Symptom Scale–Self-Report version. SCL-90 = Symptom Checklist–90. TRIM = Transgression Related Interpersonal Motivation; GCQ = Grief Cognitions Questionnaire; DAAPGQ = Depressive and Anxious Avoidance in Prolonged Grief Questionnaire.

Regression Analyses

In our next rounds of analyses, we examined the degree to which the cognitive behavioral variables explained variance in PGD, PTSD, anger, and revenge, when taking into account sociodemographic and loss-related background variables, and the shared variance between symptom scores. To this end, four series of hierarchical regression analyses were conducted with PGD, PTSD, anger, and revenge consecutively treated as dependent variables. Independent variables were entered sequentially in blocks. First, all sociodemographic and loss-related background variables that we assessed were entered in Block 1, that is, gender (0 = man, 1 = woman), age in years, the number of years passed since the loss occurred, dummy coded relationship to the deceased person (i.e., deceased was a partner [0 = no, 1 = yes], a child [0 = no, 1 = yes], a parent [0 = no, 1 = yes], a sibling [0 = no, 1 = yes]), whether a suspect had been identified and/or arrested and/or convicted (0 = no, 1 = yes), and, finally, recruitment source (0 = recruitment from support organizations, 1 = recruitment from Victim Support the Netherlands). Then, in Block 2, all six cognitive behavioral variables were entered. Finally, in Block 3, we entered concomitant symptoms—for example, PTSD, anger, and revenge, in the regression with PGD as dependent variable—to examine the association of the cognitive behavioral variables with each dependent variable, when controlling for the overlap between these dependent variables. Outcomes of these regression analyses are summarized in Table 3.

Table 3.

Summary of Regression Analyses Predicting Levels of PGD, PTSD, Anger, and Revenge Thoughts and Feelings.

	Block 1: Background variables				Block 2: Cognitive behavioral variables				Block 3: Concomitant symptoms
	PGD	PTSD	Anger	Revenge	PGD	PTSD	Anger	Revenge	PGD	PTSD	Anger	Revenge
	β	β	β	β	β	β	β	β	β	β	β	β
Background variables
Gender (1 = female)	.20***	.19***	.03	.06	.05	.03	−.09*	.03	.04	.01	−.10*	.04
Age	−.13	−.19*	−.25**	−.20*	.07	<.01	−.10	−.10	.09*	−.05	−.05	−.11
Time since loss	−.16**	−.21**	−.23***	−.08	<.01	−.03	−.11*	<.01	.01	−.01	−.07	−.02
Deceased was partner/spouse	.16**	.14*	−.02	.08	.09*	.06	−.07	.04	.03	.05	−.12*	.04
Deceased was child	.45***	.24**	.12	.12	.17***	−.02	−.12	.01	.13*	−.07	−.11	−.02
Deceased was parent	.15*	.04	.09	−.02	.10*	<.01	.02	−.04	.11*	−.04	.03	−.08
Deceased was sibling	.15*	.02	.06	<.01	.09*	−.05	−.03	<.01	.11**	−.09	<.01	−.03
Suspect identified/arrested/convicted	−.10	−.02	−.07	−.03	−.05	.02	−.03	−.04	−.06*	−.07	−.04	<.01
Recruited via Victim Support the Netherlands	−.08	−.02	.11	.09	−.02	.06	−.06	<.01	−.08*	.09*	−.08	.04
Cognitive behavioral variables
Negative self					.15*	.19**	.41***	−.10	.13	.02	.40***	−.23
Negative life					.07	.09	−.08	.09	.04	.16	−.12	−.09
Negative future					.06	.08	.19	.35*	<.01	.02	.14	.29*
Catastrophic misinterpretations					.33***	.25***	.13	.09	.22***	.08	.07	−.04
Anxious avoidance					<.01	−.06	−.04	.20**	<.01	−.07	−.05	.20**
Depressive avoidance					.29***	.30***	.08	−.09	.17**	.15**	.01	−.19
Concomitant symptoms
PGD									—	.45***	−.13	.47***
PTSD									.40***	—	.38***	−.19
Anger									−.06	.21***	—	.23**
Revenge									.15***	−.07	.14**	—
R ²	18.4%	15.1%	14.1%	4.5%	70.2%	64.1%	47.1%	15.8%	76.4%	73.4%	52.5%	25.2%
R² change	18.4%	15.1%	14.1%	4.5%	51.7%	49%	33.3%	11.3%	7.1%	8.5%	5.7%	7.9%
F change	7.80***	5.98***	5.56***	1.50	86.10***	66.33***	30.71***	6.15***	26.2***	27.94***	10.49***	9.03***

Note. PGD = Prolonged Grief Disorder; PTSD = Posttraumatic Stress Disorder.

p < .05. **p < .01. ***p < .001.

Block 1: Sociodemographic and loss-related variables

Sociodemographic and loss-related variables explained 18.4%, 15.1%, 14.1%, and 4.5% of the variance in PGD, PTSD, anger, and revenge, respectively. Female gender; (less) time since loss; and having lost a partner, child, parent, or sibling explained a unique proportion of the variance in symptom levels of PGD. Variables explaining unique variance in PTSD severity were female gender, (lower) age, (less) time since the loss, and having lost a partner or child. (Lower) age and (less) time explained unique variance in anger. Finally, (lower) age explained unique variance in revenge thoughts and feelings.

Block 2: Six cognitive behavioral variables

In Block 2, the six cognitive behavioral variables added 51.7%, 49%, 33.3%, and 11.3% to the explained variance in PGD, PTSD, anger, and revenge, respectively. The variables explaining a unique proportion of variance in PGD severity were having lost a partner, child, sibling, or parent, as well as with higher endorsement of items tapping negative cognitions about the self and catastrophic misinterpretations of grief reactions, and items tapping depressive avoidance. Similarly, negative cognitions about the self, catastrophic misinterpretations of grief reactions, and depressive avoidance explained unique proportions of variance in PTSD severity. The regression with anger as a dependent variable showed that elevated anger was associated with gender (higher scores among men), (lower) time since loss, and endorsement of negative cognitions about the self. Finally, the regression with revenge as the dependent variable showed that elevated revenge thoughts and feelings were associated with stronger endorsement of negative cognitions about the future and items tapping anxious avoidance of stimuli reminding of the loss.

Block 3: Concomitant dependent variables

In Block 3, levels of PTSD, anger, and revenge added 7.1% to the explained variance in PGD scores. Apart from concomitant PTSD and revenge thoughts and feelings and the background variables age, having lost a child, parent, or sibling, suspect being identified/arrested/convicted, and recruitment source, catastrophic misinterpretations and depressive avoidance explained unique variance in PGD in this final model. The regression predicting PTSD severity showed that, in Block 3, levels of PGD, anger, and revenge added 8.5% to the explained variance in PTSD scores. Depressive avoidance was the only cognitive behavioral variable explaining unique variance in PTSD severity in this final model, apart from recruitment source. The regression predicting anger severity showed that, in Block 3, levels of PGD, PTSD, and revenge thoughts and feelings added 5.7% to the explained variance in anger. Apart from these concomitant symptoms, (male) gender and having lost a partner, as well as negative cognitions about the self explained unique variance in anger levels in this final model. The regression predicting revenge thoughts and feelings showed that levels of PGD, PTSD, and anger added 7.9% to the explained variance in revenge. Apart from concomitant PGD scores and anger scores, stronger endorsement of items tapping negative cognitions about the future and items tapping anxious avoidance explained unique variance in revenge thoughts and feelings in this final model.

As noted, the measures of PGD (ICG-r) and PTSD (PSS-SR) both included one anger/irritability item. When we conducted this series of regression analyses (with background variables, cognitive behavioral variables, and concomitant symptoms entered in three consecutive blocks) while removing these items from both measures, outcomes were very similar to the outcomes summarized in Table 3 with, for instance, the same variables explaining a unique proportion of the variance in the four dependent variables.

As noted also, the measure of PTSD symptoms did not explicitly ask participants to keep in mind the homicide. The open-ended question where participants wrote down their most upsetting event was completed by 204 people, and 169 wrote down some aspect of the homicide as the most distressing event. When we conducted the regression analyses with PTSD severity as the dependent variable, only including these 169 respondents, outcomes very similar to the ones reported in Table 3 were found. That is, as in the analyses with the full sample, negative cognitions about the self, catastrophic misinterpretations, and depressive avoidance predicted variance in PTSD severity in Block 2 of the regression (where sociodemographic and loss-related background variables were controlled); in Block 3, depressive avoidance continued to be the only cognitive behavioral variable predicting PTSD severity, albeit at a trend level (p = .052) with this smaller sample of 169 participants.

Discussion

This study sought to enhance knowledge about the association between cognitive and behavioral variables and emotional distress following the death of a loved one due to homicide. In so doing, we aimed to contribute to knowledge about psychological mechanisms underlying the emotional impact of homicidal loss that could inform the development of interventions targeting this impact. This is relevant given the number of people confronted with homicidal loss (see Armour, 2006) and the emotional needs of this population that are not always met (Mastrocinque et al., in press). In our study, we focused on six cognitive behavioral variables—negative cognitions about (a) the self, (b) life, (c) the future, (d) catastrophic misinterpretations of one’s own grief reactions, (e) anxious avoidance, and (f) depressive avoidance—examining their association with four dependent variables, namely, symptom levels of PGD, PTSD, and anger and revenge thoughts and feelings. In a prior study (van Denderen et al., 2014), we found revenge thoughts and feelings to be associated with other emotional consequences of homicidal loss. Building on this work, the current study examined cognitive behavioral correlates of these emotional consequences.

Main Findings

Zero-order correlations showed that all six cognitive behavioral variables were significantly associated with the four dependent variables. Thus, elevated symptom levels of PGD, PTSD and anger, and revenge thoughts and feelings were associated with stronger endorsement of negative cognitions about the self, life, the future, and catastrophic misinterpretations of one’s own grief reactions, as well as the tendency to engage in anxious avoidance of reminders of the loss and depressive avoidance of activities that could foster recovery. These findings accord with prior research findings that negative cognitions and avoidance behaviors mediate the development and maintenance of emotional distress following loss (e.g., Boelen, de Keijser, & Smid, in press; Boelen et al., 2006; Maccallum & Bryant, 2013; Shear & Shair, 2005) and trauma (Ehlers & Clark, 2000).

Using hierarchical multiple regression analyses, we examined which of the six cognitive behavioral variables were associated with each outcome variable when controlling for the shared variance between these variables. Outcomes showed that negative cognitions about the self, catastrophic misinterpretations of grief reactions, and depressive avoidance (but not negative cognitions about life and the future, and anxious avoidance) explained a unique proportion of the variance in symptom levels of PGD and PTSD. These findings indicate that, in people confronted with homicidal loss, elevated grief and posttraumatic stress go hand in hand with lower self-esteem (“Since . . . is dead, I feel less worthy”), catastrophic misinterpretations of one’s own grief reactions (e.g., “Once I would start crying, I would lose control”), and a tendency to refrain from activities that could foster adjustment (e.g., “I avoid doing activities that used to bring me pleasure, because I feel unable to carry out these activities”). That depressive avoidance seemed more important in PGD and PTSD than anxious avoidance is consistent with prior research (Boelen & van den Bout, 2010); it suggests that the inability to engage in activities that could foster recovery plays a more important role in maintaining postloss distress than anxiously avoiding reminders of the loss. That negative views of the self seemed more important than negative views of life is consistent with outcomes of a study by Mancini, Prati, and Black (2011) showing that negative self-esteem, but not global views of life being meaningless, mediated the impact of violent loss on postloss distress. Outcomes also showed that negative cognitions about the self represented the only cognitive behavioral variable that had a unique association with anger. The findings that negative cognitions about the self were correlated with PGD, PTSD, and anger accord with prior evidence that low self-esteem is related with distress following loss (Golden & Dalgleish, 2012) and trauma (Foa, Ehlers, Clark, Tolin, & Orsillo, 1999).

The regression predicting revenge showed that elevated revenge thoughts and feeling were associated with a combination of negative views of the future (e.g., “I don’t have confidence in the future”) and anxious avoidance (e.g., “I avoid situations and places that confront me with the fact that . . . is dead and will never return”). This indicates that the preoccupation with ways to “get even” with the person(s) who caused the death is stronger when homicidally bereaved individuals feel less confident that they will be able to adjust to the event and to confront the pain associated with the event. This is reminiscent of prior theorizing that increased revenge thoughts and feelings reflect dysfunctional attempts to cope with the stress following a transgression (Gäbler & Maercker, 2011; Orth et al., 2006).

In the final steps of our regression analyses, concomitant symptom levels were added to the regression equations as independent variables. These analyses thus yielded clues about which of the six cognitive behavioral variables were associated with the distinct, defining features of PGD, PTSD, anger, and revenge thoughts and feelings, respectively. Outcomes of these analyses showed that catastrophic misinterpretations of grief reactions and depressive avoidance continued to have a unique linkage with PGD, whereas depressive avoidance was the only cognitive behavioral variable that continued to contribute to the explained variance in PTSD. This suggests that the cognitive behavioral variables distinguished in Boelen et al.’s (2006) cognitive behavioral conceptualization of PGD are, indeed, more specific to PGD than to bereavement-related posttraumatic stress.

Several notable findings emerged in the regression analyses predicting anger and revenge thoughts and feelings. For example, negative cognitions about the self emerged as the only cognitive behavioral variable associated with anger; revenge thoughts and feelings, in turn, were associated with higher endorsement of items tapping negative cognitions about the future and anxious avoidance. It is possible that negative views of self are associated with anger, whereas negative views of life and the future are not because, given the nature of the loss, the person feels responsible for not having prevented the death, with anger reflecting a way to externalize this self-reproach. The finding of a differential linkage of views of self with anger and revenge suggests that among homicidally bereaved individuals, a negative view of self coincides with more generic angry feelings that are not specifically targeted toward the source of the loss (e.g., angry outbursts, urges to smash things), whereas a negative view of self is not associated with angry feelings and behaviors directed toward the source of the loss (implicated in elevated revenge thoughts and feelings). The differential linkage of self-view with anger versus revenge phenomena is reminiscent of literature indicating that self-esteem has different effects dependent on the presence of moderating variables (e.g., Zeigler-Hill, 2006).

Limitations

It is important to note that the cross-sectional design of this study does not allow for drawing conclusions about the direction of causality between the cognitive behavioral variables and outcome variables assessed. There is some evidence for a prospective linkage between cognitive behavioral variables similar to the ones assessed in this study and bereavement-related distress (Boelen et al., 2013; Golden & Dalgleish, 2012; Mancini et al., 2011). Yet, the extent to which these negative cognitions and avoidance behaviors precede loss-related distress in groups exclusively comprised of homicidally bereaved persons still remains to be studied.

This study has several other limitations that must be taken into account. For instance, data were all based on self-report. Consequently, shared method variance may have inflated correlations between dependent and independent variables. Another possible limitation is that the majority (58.9%) of the participants were recruited via one of three support organizations all providing self-help based low-threshold support and advice to homicidally bereaved individuals; this may have caused a selection bias, with, for instance, an overrepresentation of bereaved individuals who were open to express thoughts and feelings and an underrepresentation of more severely distressed victims who were unable or unwilling to share their thoughts and feelings about their loss. Notable too is that recently bereaved individuals, that is, those who suffered loss within the previous 6 months were not included in this study. The implication of these limitations associated with our recruitment procedures is that caution must be applied in generalizing the current findings to nonassessed groups. A further limitation is that our measure of PGD was not totally consistent with criteria for “Persistent Complex Bereavement Disorder” (PCBD), a concept recently included in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; APA, 2013), whereas our measure of PTSD was based on criteria from DSM-IV-TR (APA, 2000) and not DSM-5 (APA, 2013). The current findings likely generalize well to recent DSM-5-based conceptualizations of disturbed grief and posttraumatic stress, given that differences between DSM-5-based and earlier criteria for disturbed grief and PTSD are small. Yet, pending replication of the current findings using such DSM-5-based criteria, some caution should be applied in generalizing these findings.

Conclusion

Notwithstanding these limitations, this study contributes to our understanding of mechanisms involved in prolonged grief, posttraumatic stress, anger, and revenge thoughts and feelings in people confronted with the homicidal death of a loved one. The current finding can have several clinical implications. For instance, the findings suggest that negative cognitions and maladaptive avoidance behaviors should be addressed in the treatment of emotional distress following homicidal loss. Different cognitions and avoidance behaviors should perhaps be targeted dependent on the core complaints of the person. For instance, among those with elevated PGD and PTSD, interventions should focus more strongly on negative views of self and depressive avoidance; negative views of self could be targeted using cognitive restructuring, and depressive avoidance by encouraging mourners to (re)engage in valued roles and activities in order to increase activity levels. As noted, revenge phenomena were associated with a tendency to anxiously avoid reminders of the loss as well as negative views of the future. Provided these findings are replicated in prospective studies, these findings imply that, for people with elevated revenge thoughts and feelings, interventions should focus on helping people to reduce their fear of confronting reminders of the loss (by means of cognitive restructuring or gradual exposure to such reminders in the presence of a supporting therapist), to obtain a more optimistic outlook on the future (e.g., by encouraging them to engage in activities that are potentially fulfilling), and, more broadly, to find ways to deal with the pain associated with the loss, other than by engaging in ruminative thoughts about the homicide and ways to retaliate (e.g., by sharing thoughts and feelings with friends). Cognitive behavioral interventions have proven effective in the treatment of PGD in diverse samples of bereaved individuals (Boelen, de Keijser, van den Hout, & van den Bout, 2007). It would be interesting for future research to examine the usefulness of these interventions to bereaved individuals with emotional difficulties following the homicidal death of a relative.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Paul Boelen is full professor of clinical psychology at the Department of Clinical and Health Psychology at Utrecht University.

Mariette van Denderen is PhD student at Groningen University.

Jos de Keijser holds an endowed professorship in the treatment of complex grief at Groningen University.

References

American Psychiatric Association. (2000). Diagnostic and Statistical Manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and Statistical Manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Amick-McMullan

Kilpatrick

D. G.

Resnick

H. S.

(1991). Homicide as a risk factor for PTSD among surviving family members. Behavior Modification, 15, 545-559. doi:10.1177/01454455910154005

Armour

(2006). Violent death: Understanding the context of traumatic and stigmatized grief. Journal of Human Behavior in the Social Environment, 14, 53-90. doi:10.1300/J137v14n04_04

Arrindell

W. A.

Ettema

(2003). Symptom checklist. Handleiding bij een multidimensionele psychopathologie-indicator [Symptom Checklist. Manual of a multidimensional psychopathology-measure]. Lisse, The Netherlands: Swets & Zeitlinger.

Boelen

P. A.

de Keijser

Smid

(in press). Cognitive behavioral variables mediate the impact of violent loss on post-loss psychopathology. Psychological Trauma: Theory, Research, Practice, and Policy.

Boelen

P. A.

de Keijser

van den Hout

M. A.

van den Bout

(2007). Treatment of Complicated Grief: A comparison between cognitive behavioral therapy and supportive counseling. Journal of Consulting and Clinical Psychology, 75, 277-284. doi:10.1037/0022-006X.75.2.277

Boelen

P. A.

de Keijser

van den Hout

M. A.

van den Bout

(2011). Factors associated with outcome of cognitive behavioral therapy for Complicated Grief: A preliminary study. Clinical Psychology & Psychotherapy, 18, 284-291. doi:10.1002/cpp.720

Boelen

P. A.

Klugkist

(2011). Cognitive behavioural variables mediate the associations of neuroticism and attachment insecurity with Prolonged Grief Disorder severity. Anxiety, Stress, & Coping, 24, 291-307. doi:10.1080/10615806.2010.527335

10.

Boelen

P. A.

van den Bout

(2010). Anxious and depressive avoidance and symptoms of prolonged grief, depression, and posttraumatic stress-disorder. Psychologica Belgica, 50, 49-67. doi:10.5334/pb-50-1-2-49

11.

Boelen

P. A.

van den Bout

de Keijser

Hoijtink

(2003). Reliability and validity of the Dutch version of the Inventory of Traumatic Grief. Death Studies, 27, 227-247. doi:10.1080/07481180302889

12.

Boelen

P. A.

van den Bout

van den Hout

M. A.

(2003). The role of cognitive variables in psychological functioning after the death of a first degree relative. Behaviour Research and Therapy, 41, 1123-1136. doi:10.1016/S0005-7967(02)00259-0

13.

Boelen

P. A.

van den Hout

M. A.

van den Bout

(2006). A cognitive-behavioral conceptualization of Complicated Grief. Clinical Psychology: Science and Practice, 13, 109-128. doi:10.1111/j.1468-2850.2006.00013.x

14.

Boelen

P. A.

van den Hout

M. A.

van den Bout

(2013). Prolonged Grief Disorder: Cognitive behavioral theory and therapy. In Stroebe

M. S.

Schut

van den Bout

(Eds.), Complicated Grief (pp. 221-234). New York, NY: Routledge.

15.

Brewin

C. R.

Andrews

Rose

(2000). Fear, helplessness, and horror in posttraumatic stress disorder: Investigating DSM-IV-TR criterion A2 in victims of violent crime. Journal of Traumatic Stress, 13, 499-509.

16.

Currier

J. M.

Holland

J. M.

Neimeyer

R. A.

(2006). Sense-making, grief and the experience of violent loss: Toward a mediational model. Death Studies, 30, 403-428. doi:10.1080/07481180600614351

17.

Derogatis

L. R.

(1983). SCL-90-R: Administration, scoring, and procedures—Manual II (2nd ed.). Towson, MD: Clinical Psychometric Research.

18.

Ehlers

Clark

D. M.

(2000). A cognitive model of posttraumatic stress disorder. Behaviour Research and Therapy, 38, 319-345. doi:10.1016/S0005-7967(99)00123-0

19.

Eisma

M. C.

Stroebe

M. S.

Schut

H. A. W.

Stroebe

Boelen

P. A.

van den Bout

(2013). Avoidance processes mediate the relationships between rumination and symptoms of psychopathology after loss. Journal of Abnormal Psychology, 122, 961-970. doi:10.1037/a0034051

20.

Engelhard

I. M.

Arntz

van den Hout

M. A.

(2007). Low specificity of symptoms on the Post-Traumatic Stress Disorder (PTSD) Symptom Scale: A comparison of individuals with PTSD, individuals with other anxiety disorders and individuals without psychopathology. British Journal of Clinical Psychology, 46, 449-456. doi:10.1348/014466507X206883

21.

Foa

E. B.

Ehlers

Clark

D. M.

Tolin

D. F.

Orsillo

S. M.

(1999). The Posttraumatic Cognitions Inventory (PTCI): Development and validation. Psychological Assessment, 11, 303-314. doi:10.1037/1040-3590.11.3.303

22.

Foa

E. B.

Riggs

Dancu

Rothbaum

(1993). Reliability and validity of a brief instrument for assessing Post-Traumatic Stress Disorder. Journal of Traumatic Stress, 6, 459-474. doi:10.1002/jts.2490060405

23.

Gäbler

Maercker

(2011). Revenge phenomena and posttraumatic stress disorder in former East German political prisoners. Journal of Nervous and Mental Disease, 199, 287-294. doi:10.1097/NMD.0b013e3182174fe7

24.

Gerlsma

Lugtmeyer

Van Denderen

De Keijser

(2013). Measuring revenge, avoidance and benevolence after victimization: Data from Dutch detainees and adolescents from the general community. Manuscript submitted for publication.

25.

Gintner

G. G.

(2001). Sudden and violent loss: Clinical guidelines for the screening and treatment of survivors. In Sandhu

(Ed.), Faces of violence: Psychological correlates, concepts and intervention strategies (pp. 355-376). New York, NY: Nova Science.

26.

Golden

Dalgleish

(2012). Facets of pejorative self-processing in Complicated Grief. Journal of Consulting and Clinical Psychology, 80, 512-524. doi:10.1037/a0027338

27.

Kristensen

Weisæth

Heir

(2012). Bereavement and mental health after sudden and violent losses: A review. Psychiatry, 75, 76-97. doi:10.1521/psyc.2012.75.1.76

28.

Maccallum

Bryant

R. A.

(2013). A cognitive attachment model of prolonged grief: Integrating attachments, memory, and identity. Clinical Psychology Review, 33, 713-727. doi:10.1016/j.cpr.2013.05.001

29.

Mancini

A. D.

Prati

Black

(2011). Self-worth mediates the effects of violent loss on PTSD symptoms. Journal of Traumatic Stress, 24, 116-120. doi:10.1002/jts.20597

30.

Mastrocinque

J. M.

Metzger

Madeira

Lang

Pruss

Sandys

. . . Cerulli

(in press). I’m still left here with the pain: Exploring the health consequences of homicide on families and friends. Homicide Studies.

31.

McCullough

M. E.

Rachal

K. C.

Sandage

S. J.

Worthington

E. L. J.

Brown

S. W.

Hight

T. L.

(1998). Interpersonal forgiving in close relationships: II. Theoretical elaboration and measurement. Journal of Personality and Social Psychology, 75, 1586-1603. doi:10.1037/0022-3514.75.6.1586

32.

Murphy

S. A.

Johnson

L. C.

Fan

J. J.

Lohan

(2003). Bereaved parents’ outcomes 4 to 60 months after their children’s deaths by accident, suicide, or homicide: A comparative study demonstrating differences. Death Studies, 27, 39-61.

33.

Nakajima

Masaya

Akemi

Takako

(2012). Complicated Grief in those bereaved by violent death: The effects of Post-Traumatic Stress Disorder on Complicated Grief. Dialogues in Clinical Neuroscience, 14, 210-214.

34.

Orth

Montada

Maercker

(2006). Feelings of revenge, retaliation motive, and posttraumatic stress reactions in crime victims. Journal of Interpersonal Violence, 21, 229-243. doi:10.1177/0886260505282286

35.

Prigerson

H. G.

Horowitz

M. J.

Jacobs

S. C.

Parkes

C. M.

Aslan

Goodkin

. . . Maciejewski

P. K.

(2009). Prolonged Grief Disorder: Psychometric validation of criteria proposed for DSM-V and ICD-11. PLoS Medicine, 6(8), e1000121. doi:10.1371/journal.pmed.1000121

36.

Prigerson

H. G.

Jacobs

S. C.

(2001). Traumatic grief as a distinct disorder: A rationale, consensus criteria, and a preliminary empirical test. In Stroebe

M. S.

Hansson

R. O.

Stroebe

Schut

H. A. W.

(Eds.), Handbook of bereavement research: Consequences, coping, and care (pp. 613-647). Washington, DC: APA Press.

37.

Rynearson

E. K.

(1984). Bereavement after homicide: A descriptive study. American Journal of Psychiatry, 141, 1452-1454.

38.

Rynearson

E. K.

Schut

Stroebe

(2013). Complicated Grief after violent death: Identification and intervention. In Stroebe

M. S.

Schut

van den Bout

(Eds.), Complicated Grief (pp. 278-292). New York, NY: Routledge.

39.

Shear

Shair

(2005). Attachment, loss, and Complicated Grief. Developmental Psychobiology, 47, 253-267. doi:10.1002/dev.20091

40.

Stuckless

Goranson

(1992). The vengeance scale: Development of a measure of attitudes toward revenge. Journal of Social Behavior & Personality, 7, 25-42.

41.

Tabachnick

B. G.

Fidell

L. S.

(2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon.

42.

van Denderen

de Keijser

Gerlsma

Huisman

Boelen

P. A.

(2014). Revenge and psychological adjustment after homicidal loss. Aggressive Behavior, 40, 504-511. doi:10.1002/ab.21543

43.

van Denderen

de Keijser

Huisman

Boelen

P. A.

(in press). Prevalence and correlates of self-rated Post-Traumatic Stress Disorder and Complicated Grief in a community-based sample of homicidally-bereaved individuals. Journal of Interpersonal Violence.

44.

van Denderen

de Keijser

Kleen

Boelen

P. A.

(2015). Murder-related psychopathology in bereaved: A systematic review. Trauma, Violence, & Abuse, 16, 70-80. doi:10.1177/1524838013515757

45.

Van der Houwen

Stroebe

Schut

Stroebe

van den Bout

(2010). Mediating processes in bereavement: The role of rumination, threatening misinterpretations, and deliberate grief avoidance. Social Science & Medicine, 71, 1669-1676. doi:10.1016/j.socscimed.2010.06.047

46.

Zeigler-Hill

(2006). Discrepancies between implicit and explicit self-esteem: Implications for narcissism and self-esteem instability. Journal of Personality, 74, 119-143. doi:10.1111/j.1467-6494.2005.00371.x