Problems with recall-based attachment style priming paradigms

Abstract

Attachment styles are often primed by having participants recall and describe a relationship that is prototypical of a given attachment style. Researchers may exclude participants who cannot recall such a relationship or who describe relationships that do not conform to the assigned prime. I suggest that excluding participants is untenable and may threaten a study’s validity. In the present research, I examine predictors of exclusion from an attachment-priming study. Priming insecure attachment resulted in greater odds of exclusion relative to a control condition. Female participants with greater sexual experience also had lesser odds of exclusion. These results suggest that attachment-priming procedures contribute to participant exclusion that compromise internal and external validity. Discussion focuses on directions for future attachment-priming research.

Keywords

Anxiety attachment avoidance priming security validity

Experimental approaches are becoming increasingly common in research on adult attachment (Mikulincer & Shaver, 2007a, 2007b). Although a variety of procedures are available to prime attachment, a common method involves having participants recall and describe a relationship with an attachment figure that conforms to the description of a prototypical attachment style (e.g., secure; Bartz & Lydon, 2004). Researchers using such recall-based attachment-priming paradigms, however, are often faced with a dilemma: What to do with participants who, for any number of reasons, are unable to properly complete their assigned prime? Including these participants might risk rendering priming effects as unreliable during statistical analyses. Thus, the alternative—excluding these participants from analyses—may seem like an attractive approach.

The appropriateness of this practice, however, relies on exclusion being a random process, as systematic differences between included and excluded participants would constitute a potential threat to the internal and external validity of studies using recall-based attachment-priming paradigms. This possibility has yet to be examined. In the present article, I therefore investigated predictors of exclusion due to not rendering an appropriate response to the priming cue and offer recommendations for how recall-based attachment-priming paradigms might be improved.

Attachment theory

Following Hazan and Shaver's (1987) conceptualization of romantic love as an attachment process, Bowlby’s attachment theory (1969/1982) has emerged as one of the dominant theories of adult relationships (see Mikulincer & Shaver, 2007a, for a review). According to the theory, human infants are born helpless and must rely on attachment figures—typically primary caregivers—to meet their needs and provide comfort and support during times of distress (Bowlby, 1969/1982). These early interactions with attachment figures are subsequently internalized into working models of self and other, with romantic partners serving as attachment figures in adulthood (Mikulincer & Shaver, 2007a; Shaver & Hazan, 1987). When attachment figures are responsive to an infant’s needs, the infant will develop positive working models of self and other, otherwise referred to as a secure attachment style. When attachment figures fail to provide the necessary support, alternatively, the infant will develop negative internal working models of self and/or other, otherwise referred to as an insecure attachment style.

Attachment insecurity is typically characterized as comprising two orthogonal dimensions: avoidance and anxiety (e.g., Brennan, Clark, & Shaver, 1998). Avoidance reflects the extent to which individuals attempt to evade emotional closeness in relationships and prefer to be self-reliant, whereas anxiety reflects the extent to which individuals are focused on rejection and abandonment by their attachment figures (Bartholomew & Horowitz, 1991; Mikulincer & Shaver, 2007a). Researchers interested in examining attachment processes in adult relationships have often assessed attachment style as an individual difference variable, using an assortment of self-report measures to measure the dimensions of attachment avoidance and anxiety (e.g., Brennan et al., 1998; Fraley, Heffernan, Vicary, & Brumbaugh, 2011; Wei, Russell, Mallinckrodt, & Vogel, 2007). Others, however, have found it fruitful to temporarily activate secure, avoidant, or anxious attachment styles, as a means of assessing the causal outcomes of these primed attachment styles.

Methods of attachment style priming

Forms of attachment style priming have been around for decades (e.g., Silverman, Lachmann, & Milich, 1982, as cited in Balwdin, 2007); however, this experimental approach has gained more traction in the last 10–15 years (e.g., Baldwin, Keedian, Fehr, Enns, & Koh-Rangarajoo, 1996; Bartz & Lydon, 2004; Carnelley & Rowe, 2010; Mikulincer & Shaver, 2007b). For example, in their review, Mikulincer and Shaver (2007b) found that priming a sense of attachment security was associated with a range of positive outcomes, such as reduced availability of trauma-related thoughts for participants with posttraumatic stress disorder (Mikulincer, Shaver, & Horesh, 2006) and increased willingness to help a distressed stranger (Mikulincer, Shaver, Gillath, & Nitzberg, 2005).

Attachment style priming paradigms can draw upon a number of different priming methods. One of the core distinctions in such paradigms is that attachment styles may be primed subliminally or supraliminally. Subliminal attachment style priming may involve presenting participants—outside of their conscious awareness—with the name of an individual they have indicated is a source of security (e.g., Mikulincer et al., 2005), pictures of attachment figure availability (e.g, Mikulincer et al., 2001), or words associated with security (e.g., comfort, love, and support; Mikulincer & Shaver, 2001) and insecurity (e.g., lonely, reject, and abandon; Canterberry & Gillath, 2013).

One of the limitations of subliminal priming methods is that it is difficult to know what exactly is being primed. For instance, subliminally presenting someone with the word “reject” might prime memories of a time they were rejected (i.e., priming anxiety), or alternatively, prime memories of a time they rejected someone else (i.e., priming avoidance). Frequently, subliminal security priming procedures employ other measures (e.g., the WHOTO; Fraley & Davis, 1997) that help to ensure that participants are able to bring attachment figures to mind in order to prime security. Recently, Milyavskaya and Lydon (2012) extended the use of the WHOTO measure to facilitate the recall of attachment figures to whom individuals were insecurely attached. This approach, however, has yet to be extended to facilitate the priming of anxious and avoidant attachment styles.

Supraliminal attachment style priming paradigms, alternatively, typically require participants to recall previous relationships in which their feelings/experiences matched descriptions of prototypically secure, anxious, or avoidant attachment styles.¹ Bartz and Lydon (2004), for example, primed security by asking participants to think and write about:

… a relationship you have had in which you have found that it was relatively easy to get close to the other person and you felt comfortable depending on the other person. In this relationship you didn’t often worry about being abandoned by the other person and you didn’t worry about the other person getting too close to you. (p. 1394)

Thus, in contrast to subliminal paradigms, recall-based supraliminial attachment style priming paradigms more clearly specify to participants what roles, experiences, and feelings are to be called to mind.

Problematic responses to recall-based attachment primes

Although results from recall-based attachment studies have proven captivating, recall-based procedures rely heavily on the premise that memories corresponding to multiple attachment styles are readily available for most people (Baldwin et al., 1996). In their first study, Baldwin, Keedian, Fehr, Enns, and Koh-Rangarajoo (1996) asked participants to list their 10 most significant relationships and indicate which of the three attachment styles best characterized each relationship. They found that “88% of participants listed relationships corresponding to at least two of the three attachment-style descriptions, and nearly half (47%) of the sample generated names for all three attachment-style descriptions” (p. 97) and that when considering all of their relationships, “91% reported that they had experienced all three styles” (p. 100). On the basis of these findings, Baldwin and colleagues reasoned that attachment styles could be primed readily in most individuals, and their third study provided further empirical support for this claim.

In spite of Baldwin et al.’s (1996) findings, recall-based attachment primes are not completed properly by 100% of participants. Unfortunately, researchers employing recall-based attachment-priming paradigms rarely report the frequency of problematic prime responses. Thus, estimating the prevalence of problematic attachment prime responses is difficult.

An article by Mikulincer et al. (2001) was one of two articles found employing recall-based attachment priming in which some estimate of problematic responses was provided. In Study 5, the researchers utilized recall-based priming to examine the effects of attachment on feelings of empathy and personal distress. Their sample consisted of 150 participants, of which, “four persons who failed to recall the targeted episode were dropped from the sample” (p. 1218). Although the prevalence of problematic responses in this study (<3%) does not strongly suggest that widespread concern regarding problematic responses is necessary, rates were much higher in Bartz and Lydon’s (2004) study. In Study 1, the authors used recall-based priming to examine the effects of attachment on the recall of agency and communion-related words. Of their sample of 293 participants, “48 participants in our study were unable to nominate someone with whom they had an attachment relationship to the one requested” (p. 1393) and were subsequently dropped from the sample (16.3%).

Taken together, these articles suggest that problematic prime responses may be given by anywhere from 2 to 16% of participants. I suggest that a prevalence of 16% problematic responses seems reasonable based on the findings of Baldwin and colleagues (1996); roughly 50% of their participants were not able to list relationships for each of the three attachment styles. Thus, in a hypothetical recall-based attachment-priming experiment, 50% of participants would have at least one attachment-priming condition that they would not be able to complete. This probability divided among three attachment-priming conditions yields a 16.66% chance of participants being assigned to a condition they would not be able to complete—a potential underestimate should some participants have more than one attachment style for which they are not able to recall a relevant memory. Although it would be arbitrary to prescribe a particular value for when prevalence of problematic responses should be deemed concerning, I submit that the potential risk of having 16% of participants (or greater) excluded from analyses is too high.

If problematic responses are commonplace, the question then becomes one of how to handle participants who yield a problematic response? The examples of Mikulincer et al. (2001), and Bartz and Lydon (2004) notwithstanding, researchers are also rarely explicit about these analytical decisions. One response would be to do nothing and to keep such participants in the sample for analyses. This approach, however, would likely undermine statistical detection of attachment-priming effects, particularly when participants assigned to one condition recall a relationship that is oppositional to the relationship type requested (e.g., recalling a relationship that would fit an anxiety prime when assigned to a security prime condition).

The alternative option, therefore, would be to exclude participants from the sample who—for any number of reasons—are unable to correctly respond to their assigned prime, lest their responses cloud the outcome of statistical analyses of the attachment-priming effects. The exclusion approach was used in the articles reporting problematic responses (i.e., Bartz & Lydon, 2004; Mikulincer et al., 2001), and I suspect is the norm for most articles employing recall-based attachment primes, as this approach would presumably give researchers the best chance of detecting a “clean” attachment prime effect. However, for exclusion to be a theoretically valid procedure, it must occur randomly. In other words, the validity—potentially both internal and external—of findings from recall-based attachment-priming studies would be compromised if exclusion rates were associated with particular method and individual difference variables.

The present research

Researchers now often cite Baldwin et al. (1996) as justifying their assumption that attachment-priming protocols are appropriate for their—and most—participants. However, given the importance of this assumption, and its implications for participant inclusion/exclusion in recall-based attachment-priming studies, it needs to be evaluated in greater depth, as the level of (un)availability of different attachment styles may vary across samples and according to theoretically important individual differences. Hence, findings from attachment style priming studies may be more limited in their generalizability than researchers anticipate.

In the present research, I explored whether theoretically relevant methodological, demographic, and sexual experience variables predicted whether participants were classified as warranting exclusion from analyses based on their responses to recall-based attachment primes. To do so, I analyzed data from a recently completed study on condom use, in which attachment style priming procedures were used. The number of participants who would be excluded from analysis was high (29.7% of the original sample); many participants could not, or did not, complete their assigned attachment style prime correctly and thus, would normally be excluded from the sample. Although the high frequency of problematic responses largely rendered the original purpose of data collection (to study attachment and condom use) untenable, I used the data set as an opportunity to explore and identify attributes that significantly predicted exclusion to determine whether attachment style priming samples are biased in theoretically relevant ways.

Classifying predictors of exclusion

Despite the present investigation of not having been originally designed to examine sample exclusion as an outcome, the study included several variables of potential interest for predicting exclusion. These predictors could be classified in a number of ways; I have opted to group them according to their implications for the internal and external validity.

Predictors impacting internal validity

In his classic article, Campbell (1957) described internal validity as a concern of whether “the experimental stimulus make some significant difference in this specific instance?” (p. 297) Thus, any factor associated with exclusion that undermines the ability to detect the possible effects of the attachment primes would compromise the internal validity of the study. To this end, the attachment-priming conditions themselves are important predictors of exclusion to investigate, as disparities in exclusion rates between conditions would be potentially indicative of different Type II error rates for each condition. Such differences would be somewhat analogous to Campbell’s (1957) “mortality” threat to internal validity, whereby conditions are (dis)advantaged because of differential participant exclusion. In other words, should particular attachment-priming conditions emerge as significant predictors of exclusion, it would mean that certain attachment primes (e.g., security) would have a better chance of successfully demonstrating an effect than others, not because the others effects aren’t there, but rather because those conditions (e.g., anxiety and/or avoidance) would be disadvantaged by reduced power vis-à-vis smaller sample sizes.

Predictors impacting external validity

Campbell (1957) described external validity, alternatively, as concerning “to what populations, settings, and variables can this effect be generalized?” (p. 297) To this end, a number of demographic criteria (e.g., age, education, socioeconomic status, etc.) might be interesting to examine as potential predictors of exclusion, simply for the sake of evaluating whether the external validity of recall-based attachment priming is more limited for certain populations. Even so, it is difficult to foresee a reason a priori why exclusion from recall-based attachment-priming studies would be associated with some of these variables. There is reason, however, to believe that demographic variables related to relational and sexual experience (e.g., gender, relationship status, and sexual experience) might be predictive of exclusion in recall-based attachment-priming studies.

In his early writings, Bowlby (1969/1982) posited that attachment was one of several behavioral systems—a “species-universal, biologically evolved neural program that organizes behavior in ways that increase the chances of an individual’s survival and reproduction, despite inevitable environmental dangers and demands” (Mikulincer & Shaver, 2007a, p. 10)—along with caregiving and sex. Hazan and Shaver (1987) later theorized that adult romantic relationships operate as an amalgam of the attachment, caregiving, and sexual behavioral systems. In other words, romantic relationships serve not only attachment functions (e.g., emotional support during times of distress) but also the opportunity to reciprocate giving emotional support and care and to engage in sex. Thus, although the early interactions with caregivers—typically parents—are certainly a cornerstone of attachment theory, as individuals reach emerging adulthood (i.e., Arnett, 2000; Tanner & Arnett, 2009), they begin to transfer primary attachment-related responsibilities to their romantic partners (Fraley & Davis, 1997; Hazan & Shaver, 1987; Mikulincer & Shaver, 2007a).

For the present study, this theoretical interplay between the attachment and sexual behavioral systems suggests that individual differences in romantic and sexual experience may be associated with the ability to recall attachment-related memories, as in the case of recall-based attachment primes. Subsequently, less relationally and sexually experienced individuals may be more likely to contribute exclusion-worthy responses. This possibility would present a problem for the external validity of attachment-priming studies, particularly those used to examine sexuality-related outcomes.

Indeed, attachment style priming has thus far proven provocative for researchers interested in relational or sexuality-related outcomes (see Dewitte, 2012; Mikulincer & Shaver, 2007a, for reviews). For example, Collins and Gillath (2012) found that among those high in trait (i.e., self-reported) attachment avoidance or anxiety, priming security decreased preferences for breakup strategies associated with negative outcomes (e.g., avoiding/withdrawing from one’s partner, using self-blame during a breakup). Priming attachment security also seems to encourage a preference for long-lasting relationships, whereas priming attachment avoidance results in a preference for short-term casual sex relationships (Gillath & Schachner, 2006). In other research, priming attachment insecurity resulted in sexual fantasies that were rife with themes of aggression and emotional distance compared to priming attachment security (Birnbaum, Simpson, Weisberg, Barnea, & Assulin-Simhon, 2012).

Thus far, the possibility that attachment-priming experiments such as these might be biased against participants with particular relational or sexual characteristics has gone unexplored. This possibility, however, requires empirical evaluation, as a perceived strength of attachment theory as a framework for examining sexuality is its broad applicability.

Method

Participants and procedure

Students from introductory psychology courses (n = 323, 195 female) at a large American/Midwestern university participated in the study in return for partial course credit. Students selected the study from an online portal that offered no description of study content; thus, at the time of recruitment, participants had no idea that the study was about relationships or sexuality. Based on questionnaires administered during the lab session, more than 90% of participants were between the ages of 18 and 20 years, and the sample was more or less evenly divided between single participants (54.3%) and those in a romantic relationship (45.7%). Most participants had experience with kissing (92.6%), stimulating a partner’s genitals (80%), having their own genitals stimulated by a partner (81.2%), performing oral sex (67.7%), receiving oral sex (73.2%), and penile–vaginal intercourse (64.9%).

Participants responded to these items before being randomly assigned to one of four attachment style priming conditions (see Bartz & Lydon, 2004, for similar attachment primes): security (n = 88), anxiety (n = 74), avoidance (n = 82), or a control condition (n = 79). The priming procedure required participants to recall a close relationship that conformed to a description that was prototypical of one of three attachment styles (secure, anxious, or avoidant). Participants were then asked to “describe it in detail. (You may refer to external events, behaviors of the parties involved, and internal feelings: thoughts, emotions, desires, and the like).”

Participants in the security condition were asked “to remember a close relationship in which you felt that the goal of getting close to your partner was achieved with relative ease, a relationship in which you felt comfortable being dependent on your partner or comfortable with your partner being dependent upon you, a relationship in which you did not worry that you would be abandoned or that your partner would get too close to you.”

Participants in the anxiety condition were asked “to remember a relationship in which you felt that your partner refused to create a connection as close as you wanted, in which you often feared that your partner did not really love you or did not wish to stay with you, in which you felt that you want to merge completely with your partner, and this desire sometimes distanced your partner from you.”

Participants in the avoidance condition were asked “to remember a close relationship in which you did not feel comfortable getting close to your partner, you had difficulty trusting in your partner completely and had difficulty being dependent on your partner, a relationship in which you felt tense when your partner got too close, and often felt as though your partner wanted a relationship more intimate than what you were ready for.”

Participants in the control condition were asked “to remember a relationship with someone you know, who is not very close to you (i.e., an acquaintance).”

Some participants did not write anything in response to their prime, or indicated that they had not experienced a relationship matching the description of their assigned prime. The rest of the participants in the security, anxiety, and avoidant conditions all wrote about previous romantic relationships, although this type of relationship was not explicitly requested (i.e., participants could have written about their relationship with a parent or another attachment figure). Most participants in the control condition, alternatively, wrote about acquaintances with whom they had no sexual experience (69.60%; n = 55) or previous casual sex partners (30.30%; n = 24).

Coding response exclusion

Two volunteer undergraduate research assistants, who were blind to the purpose of this examination, independently dummy-coded participants’ responses to the assigned priming condition as either acceptable (coded as 0) or warranting exclusion (coded as 1). Prime responses were coded as warranting exclusion for one of three reasons: participants did not write anything, participants wrote that they had not had a relationship that matched the description of their assigned attachment style prime, or participants described a relationship that did not conform to their assigned attachment style prime (e.g., being assigned to the security prime, but writing about a relationship that fit an anxiety prime description). Interrater agreement of exclusion coding was high (Cohen’s κ = .76, 95% confidence interval: .84, .68). The author then reviewed the research assistants’ coding; in the case of a dispute (n = 31) the author made the final decision as to whether a response was coded as acceptable (70.3%) or warranting exclusion (29.7%).

Analysis strategy

Patterns of prime response acceptance/exclusion were then analyzed using logistic regression, via the “Zelig” package in R (Imai, King, & Lau, 2008). Three blocks of variables were evaluated in terms of their ability to predict the odds of a participant’s prime response being coded as warranting exclusion. The blocks of variables were grouped according to their content (i.e., methodological, demographic, or sexuality), as well as the type of validity they were related to (i.e., internal or external). The first block included methodological variables related to the internal validity of the attachment primes. The second block included basic demographic variables of importance to the external validity of the primes. Finally, the third block included sexuality-related variables also of importance to the external validity of the primes.

Although grouping predictors based on such “structural properties” is a legitimate analytical approach, it is important to recognize that the ordering of these blocks is somewhat arbitrary as far as casual priority is concerned (see Cohen, Cohen, West, & Aiken, 2003, for a discussion of this issue). The current ordering, for example, privileges the importance of the internal validity-related predictors (vs. the external validity-related predictors). Although I think that it is more important to assess whether recall-based attachment priming is an internally valid procedure for any sample before worrying about the generalizability of this procedure, I recognize that others may feel that external validity is the more pressing issue. However, reanalyzing the data with the other possible orderings of blocks yielded essentially the same results, so the ordering in the present study does not seem to exert a concerning amount of influence over the outcome of analyses.²

The first block consisted of three dummy-coded variables comparing the attachment style primes (security, anxiety, and avoidance, each coded as 1 in their respective dummy code variable) to the control condition (coded as 0). Next, demographic variables for age, socioeconomic status, education, religion, and relationship status (dummy code: in a relationship = 1, not in a relationship = 0) were added to the model. Finally, an aggregate variable capturing participants’ levels of sexual experience was added to the model. This aggregate was created by summing responses regarding experience with kissing, stimulating a partner’s genitals, having their own genitals stimulated by a partner, performing oral sex, receiving oral sex, and experiencing penile–vaginal intercourse (have experienced the behavior = 1, have not experienced the behavior = 0). Participants’ scores on the sexual experience aggregate ranged from 0 to 6 (M = 4.61, SD = 2.00). Finally, given the evidence of gender differences for experience with some sexual behaviors (Petersen & Hyde, 2010), the role of participant gender (dummy code: male = 1, female = 0) and the interaction between participant gender and sexual experience were also evaluated in this block.

The importance of groups of predictors was evaluated using hierarchical likelihood ratio tests, followed by testing individual predictors within the group using Wald tests. Using likelihood ratio testing initially is useful, as exclusively relying on Wald tests in small-to-moderate sample sizes (as in the current study) can sometimes lead to less reliable analyses (Agresti, 2007). Likelihood ratio tests involve a χ² test to examine whether the addition of predictors helps to explain a significant additional amount of variability—a procedure somewhat analogous to the test of ΔR² in multiple regression. A significant χ² value indicates that the more complex model (i.e., the model with more predictors) is superior to the less complex model (i.e., the model with fewer predictors). As βs in these logistic regression models represent the log odds of exclusion, they must first be transformed (i.e., e ^β) before they can be clearly interpreted as the odds of exclusion.

Results

Adding the attachment-priming dummy-coded variables significantly improved the prediction of exclusion, χ²(3) = 33.09, p < .001. Compared with the participants in the control condition, participants in the anxiety (β = 1.25, p = .001) or the avoidance (β = 1.53, p < .001) priming conditions had significantly greater odds of having a noncompliant prime response. Adding the block of demographic predictors also significantly improved the prediction of noncompliance, χ²(5) = 12.25, p = .03. Of the demographic variables, only relationship status significantly predicted noncompliance (β = −.76, p = .006), as participants who were currently in relationships had significantly lesser odds of having a prime response warranting exclusion compared to participants who were not currently in a relationship. Thus, relationship status was the only variable retained from this block of predictors.

Finally adding the block of sexual experience, gender, and the interaction between participant gender and sexual experience significantly improved prediction of exclusion, χ²(3) = 11.58, p = .009. Specifically, there was a significant main effect of sexual experience (β = −.27, p = .001), which was qualified by an interaction with participant gender (β = .29, p = .04); the Wald test for the main effect of relationship status was rendered nonsignificant by the addition of the predictors in this block. The interaction is discussed in more detail subsequently.

Parameters for the final model predicting attachment prime noncompliance are presented in Table 1. Participants in the anxiety or avoidance priming conditions had more than triple and quintuple, the odds, respectively, of having a prime response worthy of exclusion compared to participants in the control condition. Participants who were more sexually experienced, conversely, had lesser odds of having an exclusion-worthy prime response compared to participants who were less sexually experienced. Probing the interaction between participant gender and sexual experience revealed that women who had higher levels of sexual experience had lesser odds of exclusion compared to women who had lower levels of sexual experience. Men, alternatively, had similar odds of exclusion, regardless of their level of sexual experience (see Figure 1).

Figure 1.

Interaction between participant gender and sexual experience for probability of exclusion. Female participants with greater sexual experience had a smaller probability of being excluded compared with female participants with greater sexual experience.

Table 1.

Final logistic regression model predicting participant exclusion based on prime response.

	β	SE	e ^β	95% CI (e ^β)
Security	−.08	.44	.92	.39, 2.17
Anxiety	1.25	.40	3.50**	1.59, 7.68
Avoidance	1.67	.39	5.30***	2.45, 11.45
Relationship status	−.45	.29	.64	.36, 1.13
Male	−1.08	.72	.34	.08, 1.39
Sexual experience	−.27	.08	.77**	.65, .90
Male × experience	.29	.14	1.34*	1.01, 1.77

CI: confidence interval.

Note. Security, anxiety, and avoidance are dummy-coded variables for the attachment style priming conditions compared with the control condition. Relationship status is a dummy-coded variable for whether participants are in a relationship or not. Male is a dummy-coded variable for whether participants are male or female. β = log of the odds of being excluded. e ^β = odds of exclusion.

*p < .05; **p < .01; ***p < .001.

Discussion

Participants in studies employing recall-based attachment-priming paradigms may not always be able to supply responses that match the attachment style described in the condition to which they are assigned. When this happens, researchers may be tempted to exclude such participants from their analyses. In the present article, I explored whether particular method and demographic variables were able to predict whether a participant’s response was coded as warranting exclusion or not. Participants assigned to conditions designed to prime insecure attachment styles (i.e., anxiety or avoidance) were more likely to offer exclusion-worthy responses. Further, although odds of exclusion were equivalent across most demographic variables, female participants who were less sexually experienced were more likely to offer exclusion-worthy responses than female participants who were more sexually experienced. I now turn to discussing the implications of these results for the internal and external validity of recall-based attachment-priming paradigms and conclude by offering suggestions for improving recall-based attachment primes and future research.

Compromised internal validity for recall-based primes of insecure attachment

Participants assigned to insecure attachment style priming conditions (i.e., avoidance or anxiety) were more than three times as likely to have a noncompliant prime response as those in the control condition. This is perhaps not altogether surprising, as dispositional insecure attachment styles seem to be somewhat less common than secure ones (Baldwin et al., 1996; Feeney & Noller, 1990; Hazan & Shaver, 1987). Although this bodes well for studies strictly limited to comparisons between security and control conditions, this finding suggests that studies intended to examine the causal influences of insecure attachment styles could come with a steeper cost in order to attain sufficient statistical power for analyses—a problem compounded by recent calls for larger samples in experimental psychological research (e.g., Simmons, Nelson, & Simonsohn, 2011, who advocated n > 20 per cell of a design). This finding suggests that studies attempting to examine the causal effects of priming anxiety and/or avoidance are burdened by a threat to internal validity (Campbell, 1957), whereby null effects of these manipulations may actually be a result of exclusion-induced depreciation in statistical power and thus higher Type II error rates.

External validity and the attachment–sex association for women

Results from this study suggest that current attachment-priming procedures are not appropriate for all participants and that traditional exclusion criteria may result in biased samples, as less sexually experienced participants—particularly women—were more likely to be removed from the samples because of problematic prime responses. Thus, results from studies using recall-based attachment primes might not generalize to less sexually experienced female populations. These findings are troubling, as researchers are often interested in examining the effects of attachment primes on relational (e.g., Collins & Gillath, 2012; Pierce & Lydon, 1998) or sexual (e.g., Birnbaum et al., 2012; Gillath & Schachner, 2006) outcomes. For example, a researcher who was interested in examining the effects of primed attachment styles on virgins’ beliefs about first intercourse (e.g., Humphreys, 2013) would likely be left with a sample of more highly sexually experienced female virgins, as more prototypical less sexually experienced virgins would be at higher risk of exclusion. This problem of sample bias would be relevant to a number of topics of study, such as studies regarding condom use, sexual risk taking, and relationship initiation/termination.

The importance of sexual experience for a female participant’s ability to successfully complete their assigned attachment prime may, at first, seem unintuitive. However, the importance of strong attachment-like bonds for female sexuality has been stressed by a number of theorists. Basson (2000, p. 54), for example, has persuasively argued for a model of female sexual response in which “the rewards of emotional closeness—the increased commitment, bonding, and tolerance of imperfections … all serve as the motivational factors” for female sexual arousal. A strong emotional bond to sexual partners is also prominently featured in women’s sexual strategies and reactions to infidelity (Buss, Larsen, Westen, & Semmelroth, 1992), as well as cultural expectations for women’s sexual conduct (Sakaluk et al., 2013). Taken alongside the present findings, this literature suggests that women’s attachment and sexual behavioral systems may be more closely intertwined, as those with less sexual experience struggled to think of relevant attachment-related memories.

Limitations

The foremost limitation of the present study is that it was not originally conceived to examine prime response-based exclusion as an outcome. Originally intended as a study on condom use, it was only once the high rate of exclusion was evident that predictors of exclusion were examined. One consequence of this change in focus is that some theoretically interesting predictors of exclusion were not included in the study. Self-report measures of trait attachment avoidance and anxiety (e.g., Brennan, Clark, & Shaver, 1998), for example, would have been relevant predictors of exclusion to include. Studies would be greatly compromised if those with particular attachment styles were more likely to be excluded from samples. Alternatively trait attachment style might interact with priming conditions, thereby leading to exclusion, if participants with particular attachment styles (e.g., insecure) do not have memories that were compatible with certain attachment primes (e.g., a security prime). As interactions between trait and primed attachment are sometimes of interest to researchers (e.g., Gillath & Schachner, 2006), examination of the possible associations between trait attachment dimensions and exclusion from attachment-priming samples should be a focus of further research.

Finally, participants in the attachment conditions may have been inadvertently primed to think of sexual/relationship partners—as opposed to other possible attachment figures like mothers and fathers—given the focus of the study on condom use (mentioned during the informed consent process), by the presence of the sexual experience questions that preceded the attachment-priming condition to which participants were assigned, and the use of the word “partner” in the attachment primes themselves. However, should this limitation have been the primary factor contributing to the exclusion rates observed in the present study, one would not have expected there to be differential exclusion by attachment condition. Indeed, the rates of exclusion for participants in the security condition were equitable to those in the control condition, in which more than half of participants did not recall a sexual partner. This limitation, therefore, although worth mentioning, may not have had as pervasive of an impact on the results of the present study as one might initially fear.

Future directions and conclusion

Given the significance of the relationship status and sexual experience variables, it is likely that exclusion from recall-based attachment-priming studies is partially a product of relational inexperience.³ In other words, attachment-priming protocols are currently worded in such a way that makes it less likely that less relationally experienced participants will be able to complete them correctly, if at all. The period of emerging adulthood is a critical development period during which young adults begin to explore romantic and sexual relationships (Arnett, 2000; Tanner & Arnett, 2009) and when attachment-related responsibilities begin to be transferred to romantic partners (Fraley & Davis, 1997). As the vast majority of participants were between 18 and 20 years, it is unlikely that participants had accumulated a large number of different romantic/sexual partners, thus making it difficult for them to recall and describe a relationship of a given description. Taken along with the increased odds of exclusion in insecure attachment-priming conditions, if participants have had few romantic/sexual relationships, they may not yet have experienced a relationship in which they predominantly felt insecurely attached.

One option for researchers to address these limitations of recall-based attachment-priming procedures is to avoid relying on undergraduate students—who are likely less sexually experienced than the rest of the adult population—as participants in their research (i.e., Sears, 1986). For studies that do not require direct observation or collection of behavioral measures, services like Amazon’s Mechanical Turk may prove useful (MTurk; Buhrmester, Kwang, & Gosling, 2011; Mason & Suri, 2012), as MTurk samples tend to be older than college-based samples, and hence, may have more relationship and sexual experience. In my experience, Mturk examples are, in fact, older (e.g., Mdn age = 28 years), more sexually experienced (e.g., ≥85% have experience with the performing/receiving oral sex and penile–vaginal intercourse), and have lower rates of exclusion-worthy responses to recall-based attachment primes (e.g., ≤10% in a previous sample of 500 male/female participants). This solution, however, does little to address the need to examine the causal influence of attachment styles during emerging adulthood (Arnett, 2000).

Therefore, researchers may be best served by considering alternative priming methods. In younger adult samples, for example, one approach might be to prime memories of particular attachment figures with whom experience is likely to be greater (e.g., best friends and parents). Indeed, self-report measurement of attachment styles has begun to recognize the need to distinguish between different attachment figures (i.e., mother, father, best friend, and romantic partner; Fraley et al., 2011). Future attachment style priming studies wishing to use undergraduate samples should therefore consider explicitly specifying a particular attachment figure in their priming protocols. Careful consideration, however, should be given to which target attachment figure researchers include in their primes, as attachment processes for one kind of attachment figure (e.g., relationship partner) may be more or less important for particular topics of study (e.g., romantic/sexuality-based outcomes) than other kinds of attachment figures (e.g., father).

A final concern is that, regardless of which attachment figure is specified, participants’ predominant attachment styles may be incongruent with the primes to which they are assigned (e.g., participants recalling a relationship in which they felt secure overall but were asked to respond to an anxiety prime). Thus, attachment-priming protocols may be improved by requesting that participants recall a particular moment within a specific relationship, in which their feelings conformed to an assigned attachment style, as opposed to an entire relationship characterized by the requested attachment style. As participants presumably have a limited number of mothers, fathers, best friends, and romantic partners, this procedural amendment should increase the probability that participants are able to respond to primes in a compliant manner, by shifting the unit of memory from relationships to experiences within relationships (see Appendix 1 for an updated Attachment Style Priming Protocol⁴).

Despite the problems with attachment style priming protocols that have been highlighted in this article, researchers should continue to adopt and update attachment-priming protocols, particularly in the domains related to romantic and/or sexual relationships. However, they should report exclusion rates in their articles and investigate and report predictors of exclusion when warranted (e.g., when >10% of participants give noncompliant prime responses). Such variables, if significant as predictors of exclusion, could indicate that results and samples for attachment style priming studies are biased in theoretically relevant ways, thereby requiring researchers to make concessions about the limitations of their studies’ generalizability.

Footnotes

Appendix 1 Acknowledgments

The author would like to thank Dr Monica Biernat and Matthew Baldwin for their helpful comments on earlier versions of this article, and Allegra Baxter, Sarah Kaminski, and Hunter Vore for their help with data collection and entry.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Notes

References

Agresti

(2007). An introduction to categorical data analysis (2nd ed.). Hoboken, NJ: John Wiley & Sons.

Arnett

J. J.

(2000). Emerging adulthood: A theory of development from the late teens through the twenties. American Psychologist, 55, 469–480. doi:10.1037/0003-066X.55.5.469

Baldwin

M. W.

(2007). On priming security and insecurity. Psychological Inquiry, 18, 157–196. doi: 10.1080/10478400701512703

Baldwin

M. W.

Keelan

J. P. R.

Fehr

Enns

Koh-Rangarajoo

(1996). Social-cognitive conceptualization of attachment working models: Availability and accessibility effects. Journal of Personality and Social Psychology, 71, 94–109. doi:10.1037/0022-3514.71.1.94

Bartholomew

Horowitz

L. M.

(1991). Attachment styles among young adults: A test of a four-category model. Journal of Personality and Social Psychology, 61, 226–244. doi:10.1037/0022-3514.61.2.226

Basson

(2000). The female sexual response: A different model. Journal of Sex & Marital Therapy, 26, 51–65. doi:10.1080/009262300278641

Bartz

J. A.

Lydon

J. E.

(2004). Close relationships and working self-concept: Implicit and explicit effects of priming attachment on agency and communion. Personality and Social Psychology Bulletin, 30, 1389–1401. doi:10.1177/0146167204264245

Birnbaum

G. E.

Simpson

J. A.

Weisberg

Y. J.

Barnea

Assulin-Simhon

(2012). Is it my overactive imagination? The effects of contextually activated attachment insecurity on sexual fantasies. Journal of Social and Personal Relationships, 29, 1131–1152. doi:10.1177/0265407512452978

Bowlby

. (1969/1982). Attachment and loss: Attachment. New York, NY: Basic Books.

10.

Brennan

K. A.

Clark

C. L.

Shaver

P. R.

(1998). Self-report measurement of adult attachment: An integrative overview. In Simpson

J. A.

Rholes

W. S.

(Eds.), Attachment theory and close relationships (pp. 46–76). New York, NY: Guilford Press.

11.

Buhrmester

Kwang

Gosling

S. D.

(2011). Amazon’s mechanical turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3–5. doi:10.1177/1745691610393980

12.

Buss

D. M.

Larsen

R. J.

Westen

Semmelroth

(1992). Sex differences in jealousy: Evolution, physiology, and psychology. Psychological Science, 3, 251–255. doi:10.1111/j.1467-9280.1992.tb00038.x

13.

Campbell

D. T.

(1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54, 297–312. doi:10.1037/h0040950

14.

Canterberry

Gillath

(2013). Neural evidence for a multifaceted model of attachment security. International Journal of Psychophysiology, 88, 232–240. doi:10.1016/j.ijpsycho.2012.08.013

15.

Carnelley

K. B.

Rowe

A. C.

(2010). Priming a sense of security: What goes through people’s minds? Journal of Social and Personal Relationships, 27, 253–261. doi:10.1177/0265407509360901

16.

Cohen

West

S. G.

Aiken

L. S.

(2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Earlbaum.

17.

Collins

T. J.

Gillath

(2012). Attachment, breakup strategies, and associated outcomes: The effects of security enhancement on the selection of breakup strategies. Journal of Research in Personality, 46, 210–222. doi:10.1016/j.jrp.2012.01.008

18.

Dewitte

(2012). Different perspectives on the sex-attachment link: Towards an emotion-motivational account. Journal of Sex Research, 49, 105–124. doi: 10.1080/00224499.2011.576351

19.

Feeney

J. A.

Noller

(1990). Attachment style as a predictor of adult romantic relationships. Journal of Personality and Social Psychology, 58, 281–291. doi:10.1037/0022-3514.58.2.281

20.

Fraley

R. C.

Davis

K. E.

(1997). Attachment formation and transfer in young adults’ close friendships and romantic relationships. Personal Relationships, 4, 131–144. doi:10.1111/j.1475-6811.1997.tb00135.x

21.

Fraley

R. C.

Heffernan

M. E.

Vicary

A. M.

Brumbaugh

C. C.

(2011). The experiences in close relationships-relationship structures questionnaire: A method for assessing attachment orientations across relationships. Psychological Assessment, 23, 615–625. doi:10.1037/a0022898

22.

Gillath

Schachner

D. A.

(2006). How do sexuality and attachment interrelate? Goals, motives, and strategies. In Mikulincer

Goodman

G. S.

(Eds.), Dynamics of romantic love: Attachment, caregiving, and sex (pp. 121–147). New York, NY: Guilford Press.

23.

Hazan

Shaver

(1987). Romantic love conceptualized as an attachment process. Journal of Personality and Social Psychology, 52, 511–524. doi:10.1037/0022-3514.52.3.511

24.

Humphreys

T. P.

(2013). Cognitive frameworks of virginity and first intercourse. Journal of Sex Research, 50, 664–675. doi:10.1080/00224499.2012.677868

25.

Imai

King

Lau

(2008). Toward a common framework of statistical analysis and development. Journal of Computational and Graphical Statistics, 17, 892–913. doi:10.1198/106186008X384898

26.

Mason

Suri

(2012). Conducting behavioral research on Amazon’s mechanical turk. Behavior Research Methods, 44, 1–23. doi:10.3758/s13428-011-0124-6

27.

Mikulincer

Gillath

Halevy

Avihou

Avidan

Eshkoli

(2001). Attachment theory and reactions to others’ needs: Evidence that activation of the sense of attachment security promotes empathetic responses. Journal of Personality and Social Psychology, 81, 1205–1224. doi:10.1037/0022-3514.81.6.1205

28.

Mikulincer

Shaver

P. R.

(2001). Attachment theory and intergroup bias: Evidence that priming the secure base schema attenuates negative reactions to out-groups. Journal of Personality and Social Psychology, 81, 97–115. doi:10.1037/0022-3514.81.1.97

29.

Mikulincer

Shaver

P. R.

(2007a). Attachment in adulthood: Structure, dynamics, and change. New York, NY: Guilford Press.

30.

Mikulincer

Shaver

P. R.

(2007b). Boosting attachment security to promote mental health, prosocial values, and inter-group tolerance. Psychological Inquiry, 18, 139–156. doi:10.1080/10478400701512646

31.

Mikulincer

Shaver

P. R.

Gillath

Nitzberg

R. A.

(2005). Attachment, caregiving, and altruism: Boosting attachment security increases compassion and helping. Journal of Personality and Social Psychology, 89, 817–839. doi:10.1037/0022-3514.89.5.817

32.

Mikulincer

Shaver

P. R.

Horesh

(2006). Attachment bases of emotion regulation and posttraumatic adjustment. In Snyder

D. K.

Simpson

J. A.

Hughes

J. N.

(Eds.), Emotion regulation in families: Pathways to dysfunction and health (pp. 77–99). Washington, DC: American Psychological Association. doi:10.1037/11468-004

33.

Milyavskaya

Lydon

J. E.

(2012). Strong but insecure: Examining the prevalence and correlates of insecure attachment bonds with attachment figures. Journal of Social and Personal Relationships, 30, 529–544. doi:10.1177/0265407512461200

34.

Petersen

J. L.

Hyde

J. S.

(2010). A meta-analytic review of research on gender differences in sexuality, 1993–2007. Psychological Bulletin, 136, 21–38. doi:10.1037/a0017504

35.

Pierce

Lydon

(1998). Priming relational schemas: Effects of contextually activated and chronically accessible interpersonal experiences on responses to a stressful event. Journal of Personality and Social Psychology, 75, 1441–1448. doi:10.1037/0022-3514.75.6.1441

36.

Sakaluk

J. K.

Todd

L. M.

Milhausen

Lachowsky

N. J.

, & the Undergraduate Research Group in Sexuality. (2013). Dominant heterosexual sexual scripts in emerging adulthood: Conceptualization and measurement. Journal of Sex Research. doi:10.1080/00224499.2012.745473

37.

Sears

D. O.

(1986). College sophomores in the lab: Influences of a narrow data base on social psychology’s view of human nature. Journal of Personality and Social Psychology, 51, 515–530. doi:10.1037/0022-3514.51.3.515

38.

Silverman

L. H.

Lachmann

Milch

R. H.

(1982). The search for oneness. New York: International Universities Press.

39.

Simmons

J. P.

Nelson

L. D.

Simonsohn

(2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366. doi:10.1177/0956797611417632

40.

Tanner

J. L.

Arnett

J. J.

(2009). The emergence of ‘emerging adulthood’: The new life stage between adolescence and young adulthood. In Furlong

(Ed.), Handbook of youth and young adulthood: New perspectives and agendas (pp. 39–45). New York, NY: Routledge.

41.

Wei

Russell

D. W.

Mallinckrodt

Vogel

D. L.

(2007). The experiences in close relationships scale (ECR)-short form: Reliability, validity, and factor structure. Journal of Personality Assessment, 88, 187–204. doi:10.1080/00223890701268041