Development and Initial Validation of the Sexual Minority Women Rejection Sensitivity Scale

Status-Based RS

Minority stress theory positions expectations of rejection, which are captured by the construct of RS, as a proximal stressor along with internalized negativity and sexual identity concealment (Meyer, 2003). Research has demonstrated that higher expectations of rejection and discrimination (also referred to as anticipated stigma, public regard, or stigma consciousness) ¹ are associated with higher internalizing symptomatology and internalized negativity (Lewis et al., 2003, 2006; Quinn & Chaudoir, 2009). While expectations of rejection capture the cognitive component of anticipation of rejection, RS captures both the cognitive and affective components of the anticipation of rejection and their interaction.

Downey and Feldman (1996) developed the RS model to explain the process by which an individual’s past experiences of rejection can lead to anxious expectations that others will reject the individual in current or future interactions. The original RS construct (referred to as personal RS [PRS]) focuses on general expectations of rejection that are not attributable to a specific component of one’s identity (Downey & Feldman, 1996). Research has demonstrated that higher PRS is associated with more frequent rumination following rejection (Pearson, Watkins, & Mullan, 2011) and more depressive symptoms (Ayduk, Downey, & Kim, 2001). The RS construct has since been expanded to capture expectations of rejection associated with various stigmatized social identities, including gender, race, and sexual orientation (London et al., 2012; Mendoza-Denton et al., 2002; Pachankis et al., 2008). Similar to the conceptualization of PRS, the social identity-based RS models theorize that past experiences of direct, indirect, or even vicarious discrimination can lead individuals to develop anxious expectations of similar marginalization in contexts in which the potential for discrimination or stigmatization on the basis of one’s social identity is present (London et al., 2012; London, Ahlqvist, Gonzalez, Glanton & Thompson, 2014; Mendoza-Denton et al., 2002). In turn, identity-based RS is associated with heightened vigilance for cues of rejection as well as emotional and behavioral reactions to perceived rejection (Downey & Feldman, 1996; Downey et al., 1998).

Previous research has demonstrated that identity-based RS plays an important role in psychosocial functioning among ethnic/racial minorities, women, and sexual minorities. For example, identity-based RS is associated with more frequent internalizing symptoms among Asian Americans (Chan & Mendoza-Denton, 2008), lesbians, and gay men (Feinstein et al., 2012) as well as poorer academic outcomes among African Americans and women (London et al., 2012; Mendoza-Denton et al., 2002). All of the identity-based RS models share a common focus on the anxiety that members of stigmatized groups experience in social contexts in which their identity may be threatened. Yet, identity-based RS models are also distinct, in that the specific cues and triggers that signal the potential for threat are different for each group and are based on the prevailing stereotypes about that group (London et al., 2012). For example, as many gay men may be aware of stereotypes linking being a gay male with having HIV/AIDS, medical situations may activate anxious expectations of rejection for gay men (e.g., giving blood; Herek & Capitanio, 1999). Yet, a similar medical context may not activate concerns of rejection among heterosexual men or even SMW as these groups are not stereotyped as being at high risk for HIV/AIDS. Thus, in order to assess a particular group’s expectations of rejection, it is critical that RS measures focus on the unique situations that are anxiety provoking for each group.

Sexual Orientation RS

Pachankis, Goldfried, and Ramrattan (2008) were the first to extend the construct of identity-based RS to sexual minorities. They developed the Gay-Related Rejection Sensitivity Scale (GRRSS) to capture the types of situations in which gay men experience anxiety and expect to be rejected, such as medical contexts and interactions with heterosexual men. They found that sexual orientation RS was positively associated with parental rejection, internalized negativity, and unassertiveness. Further, internalized negativity mediated the association between parental rejection and sexual orientation RS among a sample of gay men.

In an effort to use the GRRSS with lesbians as well as gay men, Feinstein, Goldfried, and Davila (2012) modified the measure by removing 2 items that were theorized to be specific to gay men’s experiences. They found that sexual orientation RS was positively associated with frequency of discrimination, internalized negativity, and severity of depression and social anxiety symptoms. Sexual orientation RS and internalized negativity mediated the associations between discrimination and both depressive and social anxiety symptoms. Although Feinstein and colleagues provided initial evidence that the GRRSS could be used with lesbians, the measure was developed and validated exclusively among gay men. The items were based on gay men’s reports of the situations in which they experience anxiety and expect to be rejected, and the measure is unlikely to capture the full range of situations that trigger expectations of rejection among SMW.

In addition to common concerns among sexual minority men and women, there are unique stressors that each group experiences, which result from different stereotypes and sources of rejection for each group (e.g., Friedman & Leaper, 2010; Hequembourg & Brallier, 2009; Kertzner et al., 2009; Ross, Dobinson, & Eady, 2010; Szymanski, 2005). Further, the intersection of their two stigmatized identities, sexual minority and female, creates a unique configuration of stereotypes about SMW (Brooks, 1981; Diamond, 2003; Purdie-Vaughns & Eibach, 2008). For instance, SMW face sexual objectification by heterosexual men, such that heterosexual men may proposition SMW to have sex with them and another female (Friedman & Leaper, 2010; Hequembourg & Brallier, 2009; Kertzner et al., 2009; Ross et al., 2010; Szymanski, 2005). Sexual minority men also face sexual objectification, however, the typical perpetrators of such objectification are likely to be other sexual minority men (Engeln-Maddox, Miller, & Doyle, 2011; Kozak, Frankenhauser, & Roberts, 2009). Additionally, as stereotypes do not link sexual minority status with having HIV/AIDS among women, as they do among men, it is unlikely that SMW will experience contagion-stereotype anxiety in medical situations. Given differences in the situations that elicit anxiety and expectations of rejection between sexual minority men and women, it is unlikely that the GRRSS captures the full range of unique situations that elicit concerns about rejection for SMW.

Current Studies

The goals of the current studies were to develop and provide preliminary validation for a measure of sexual orientation RS that focuses specifically on the experiences of SMW. In Study 1a, we identified specific types of situations that activated concerns of rejection on the basis of sexual identity among SMW. These descriptive narratives were used to generate rejection scenarios for the SMW Rejection Sensitivity Scale (SMW-RSS). In Study 1b, we identified the core scenarios and systematically reduced the list of scenarios by examining inter-item reliability and the underlying factor structure. In Study 2, we examined the measure’s convergent and discriminant validity.

Study 1a

In order to identify situations in which SMW typically anticipate or experience rejection on the basis of their sexual identity, a community sample of SMW were asked to recall situations in which they anticipated or experienced such rejection. These could be situations that they personally experienced or situations that they vicariously experienced (e.g., saw or knew of). Based on the most frequent themes generated by participants, we developed the scenarios for the SMW-RSS and questions assessing expectations of rejection and concern/anxiety regarding potential rejection for each scenario. In measures of RS, participants are first asked to read a scenario and imagine themselves in that situation, and then respond to two questions, one examining their anxiety/concern about rejection in the situation (affect) and the other examining their expectations of rejection in the situation (cognition). We attempted to stay as close as possible to the context of the situations detailed in the qualitative responses generated by participants in Study 1a. Procedures used to develop scenarios were similar to those used by Pachankis and colleagues (2008).

Method

Procedures

Data for this study were collected using an Internet-based survey administered via PsychData. E-mail announcements were sent to over 100 lesbian, gay, bisexual, and transgender (LGBT) community and university-based organizations throughout the United States. The contact person at each organization was asked to distribute the study invitation to all members and/or individuals on their listserv. Participants were offered the opportunity to enter a raffle to win one of the eight US$25 gift cards.

Participants

A sample of 150 self-identified SMW completed the survey, including 72 (48.0%) self-identified lesbian women, 45 (30.0%) self-identified bisexual women, 23 (15.3%) self-identified queer women, and 10 (6.6%) women who identified with other sexual orientation identity labels. Demographic information is reported in Table 1. Due to the distinct stigmatization experiences of transgender individuals and the intersectionality of their multiple stigmatized identities, we did not include transgender participants in these studies. Only participants who reported their gender to be female were included, while individuals who identified their gender as transgender or an unlisted other gender identity, such as genderqueer, were not included in analyses for any of the three studies (n = 7, 4, and 19 for Studies 1a, 1b, and 2, respectively). The sample demonstrated variability in the extent to which women were open to others about their sexual identity (M = 4.65 and SD = 1.36 on a 1–7 scale, with 7 indicating high sexual orientation disclosure [the Outness Inventory, OI]; Mohr & Fassinger, 2000). Thus, there was sufficient variability in sexual orientation identity and outness to capture the experiences of SMW with varying sexual orientation identity labels and outness.

OPEN IN VIEWER

Table 1.

Demographics.

Demographics Variable	Study 1a	Study 1b	Study 2
N	150	128	300
Age (M, SD)	26.3 (8.6)	27.1 (9.5)	26.8 (8.5)
Age range	18–62	18–63	18–60
Sexual identity
Lesbian	48.0%	34.4%	37.7%
Bisexual	30.0%	39.1%	35.3%
Queer	15.3%	13.3%	23.0%
Other	6.6%	13.3%	4.0%
Racial/ethnic group
Caucasian/White	76.0%	76.6%	76.3%
Latina	8.0%	7.0%	7.0%
Asian/Pacific Islander	4.7%	3.1%	1.3%
African American/Black	3.3%	3.1%	6.0%
Multiracial	7.3%	3.9%	6.7%
Other	0.7%	6.3%	2.6%
Regional area type
Urban	44.7%	56.3%	61.0%
Suburban	40.7%	32.8%	28.0%
Rural	14.7%	10.9%	11.0%
Country of residence
United States	100.0%	78.1%	83.7%
United Kingdom	—	18.8%	9.7%
Australia	—	—	3.0%
Canada	—	3.1%	3.7%
Region of United States
Northeast	16.7%	41.4%	48.0%
Midwest	49.3%	32.2%	21.4%
West	9.3%	5.1%	15.7%
Southwest	14.7%	11.1%	6.0%
Southeast	8.0%	10.1%	8.9%
College students	64.0%	55.1%	41.8%
Annual income
<US$10,000	50.7%	—	40.1%
US$10,000–24,999	22.0%	—	25.4%
US$25,000–49,999	16.0%	—	24.7%
US$50,000–74,999	6.7%	—	6.0%
>US$75,000	4.7%	—	3.7%
Education level
Some high school	—	—	2.3%
Graduate high school	4.0%	9.4%	8.3%
Some college	38.7%	35.9%	29.0%
Associate’s degree	2.0%	3.9%	9.4%
Bachelor’s degree	17.3%	16.4%	25.0%
Some graduate school	16.7%	14.8%	11.3%
Graduate/professional degree	21.3%	19.5%	14.7%

Note. Percentages for U.S. region were calculated by dividing by the number of participants in each group living in the United States.

Measures

OI (Mohr & Fassinger, 2000)

The OI assesses the extent to which individuals have disclosed their sexual orientation to a variety of people and groups, including family, heterosexual friends, coworkers, supervisors, religious community members and leaders, and strangers. Each person or group is rated on a scale of 1 (person definitely does NOT know about your sexual orientation status) to 7 (person definitely knows about your sexual orientation status and it is OPENLY talked about). A “not applicable” option is also provided if there is no such person or group in the respondent’s life. An overall score was computed by averaging the responses across the 11 items (α = .90). This measure has demonstrated high inter-item reliability predominantly among educated, European American samples of sexual minorities (α = .79–.95; Balsam, Beauchaine, Rothblum, & Solomon, 2008; Balsam & Szymanski, 2005; Lewis et al., 2005; Meidlinger & Hope, 2014; Mohr & Fassinger, 2003; Todosijevic, Rothblum, & Solomon, 2005). It has also demonstrated negative associations with discomfort with disclosure and need for privacy regarding sexual orientation as well as positive associations with single item measures of disclosure of sexual orientation (Mohr & Fassinger, 2000, 2003).

SMW-RSS–initial development

Participants were asked to respond to four open-ended questions about situations that made them feel concerned about being rejected, discriminated against, or treated differently because of their sexual orientation identity as part of an online survey. The first question asked about any types of situations in general that triggered concerns about rejection. The second question asked about situations that would elicit concerns of rejection that were unique to SMW, and the third and fourth asked about overt (referred to as obvious discrimination or rejection) and subtle (referred to as possible sexual orientation–based discrimination or rejection) situations. Specific instructions and questions are available upon request from the first author. These four questions were asked to capture a wider variety of situations that elicited concerns and expectations of rejection than a single item could capture. All 150 participants answered at least one of the four open-ended questions, 123 participants completed two or more, 121 completed three or more, and 107 completed all four questions. Participants were able to pause and continue the survey at a later time. We asked about both rejection and discrimination because there is individual variation in how people describe their negative sexual orientation–related experiences and we wanted to capture a broad range of these experiences. Participants were asked to provide detailed information about what happened that concerned them, what they thought and felt as a result of the experience, and the type of location it occurred in.

First, the first author downloaded the responses into a text file and read and re-read all of the responses, making note of the contexts, perpetrators, and themes reflected by responses. She then created a coding scheme for three variables: the global context that responses fell into, the perpetrator of rejection, and the specific theme within each context that each situation reflected. The coding scheme was based on the content of the responses and the first author’s expertise in the area of prejudice and discrimination against SMW. The author’s expertise allowed her to identify links between participants’ responses and the literature on the experiences of discrimination among SMW, ensuring that the themes covered the situations reported by participants and those described in the literature. Next, three independent coders read and coded each response from the online responses using the established coding scheme. Fleiss’ (1971) κ is used to estimate inter-item reliability with three or more raters, and values over .81 are considered near perfect agreement. Inter-rater reliability was high (Fleiss’ κ = .93). Then, the first and second authors created several scenarios to reflect each common theme. Themes reported by fewer than three participants were not developed into scenarios. Themes that were specific to bisexual or plurisexual (i.e., attracted to more than one gender) women (e.g., rejection by lesbians and gay men) were not developed into scenarios, because the measure was intended to be applicable to all SMW. Responses fell into five global contexts: family, work, health care, social (subdivided into general and interactions/dating setups with heterosexual men), and other. Perpetrators of rejection varied based on the domain into which an event fell. For example, in the family domain, perpetrators included a variety of specific family members (e.g., mother, father, sister, brother, aunt, and uncle), while perpetrators in the social domain included male and female strangers, male and female heterosexual friends and acquaintances, and service providers (e.g., waitress/waiter and real estate agents). A total of 36 scenarios were created, reflecting a wide range of contexts and perpetrators of rejection and representing 14 major themes, including (1) encounters with strangers, (2) physical safety concerns, (3) sexual objectification/harassment from heterosexual men, (4) housing discrimination, (5) being set up on dates with men, (6) interactions with heterosexual friends (homonegative slurs and drifting of friendships), (7) difficulties with heterosexual roommates, (8) strained social interactions at work, (9) being excluded from special occasions (e.g., weddings and birthday parties), (10) differential treatment by heterosexual females in female spaces (locker rooms and bathrooms), (11) rejection from members and leaders of one’s religious congregation, (12) concerns relating to differential treatment of one’s children due to one’s sexual identity, (13) bias from gynecologists, and (14) bias in school (college/university). After the scenarios were created, two general questions were tailored to capture the cognitive and affective components of RS activated in each scenario: “How anxious/concerned would you be that you would be treated differently or experience a negative outcome because of your sexual orientation?” and “To what extent would you expect to be treated unfairly based on your sexual orientation?”

Discussion

Potential scenarios for the SMW-RSS were developed based on situations in which SMW reported experiencing or being concerned about experiencing rejection based on their sexual minority identity. The coding of responses into contexts and themes led to the development of 36 potential scenarios for the SMW-RSS. The majority of the scenarios developed from the qualitative online responses (27 of the 36) reflected themes and/or contexts that were not included in the GRRSS. These unique scenarios represented 11 of the 14 themes that were identified from the qualitative responses. While not all of these themes were specific to SMW, they reflected concerns that were not addressed in the GRRSS. Seven scenarios reflected themes that were unique to SMW, such as sexual objectification/harassment from heterosexual men and bias from gynecologists. This measure appears to reflect the unique and varied rejection concerns of SMW.

Study 1b

The purpose of Study 1b was to pilot test the 36 scenarios on a second independent sample of SMW and to utilize exploratory factor analysis (EFA) to identify the best performing scenarios to be included in the final measure.

Method

Procedure

Data for this study were collected using an Internet-based survey administered via Qualtrics. Participants were recruited from websites that announced volunteer opportunities (e.g., Craigslist) as well as listservs and Facebook groups that target LGB individuals. Participants were offered the opportunity to enter a raffle to win one of the four US$25 gift cards.

Participants

A sample of 128 SMW completed the survey, including 44 (34.4%) self-identified lesbian women, 50 (39.1%) self-identified bisexual women, 17 (13.3%) self-identified queer women, and 17 (13.3%) women who identified with other sexual orientation identity labels. Demographic information is presented in Table 1. The sample demonstrated variability in the extent to which they were open to others about their sexual orientation (M = 4.64 and SD = 1.47 on the OI). This sample size is considered acceptable for EFAs on items that are highly determined (high ratio of items per factor, e.g., 6–7 items per factor; 36:1 in this analysis) and moderate to high communalities among indicators (high proportion of variance shared among items, e.g., communalities of .4–.8; among this sample, .39–.61; MacCallum, Widaman, Preacher, & Hong, 2001; MacCallum, Widaman, Zhang, & Hong, 1999; Sapnas & Zeller, 2002).

Measures

Sexual identity

Participants were asked to self-identify as heterosexual, lesbian, queer, or bisexual. A blank space was also provided for those who identified with an identity that was not listed. This question functioned as a check that participants identified as SMW. Participants who identified as heterosexual (n = 3) were not provided access to the survey.

The OI (Mohr & Fassinger, 2000)

See Study 1a for a description of this measure (α = .91 in this sample).

SMW-RSS–pilot scenarios

The SMW-RSS measures SMW’s anxiety/concern about and expectations of rejection on the basis of their sexual orientation. The 36 scenarios that were developed in Study 1a were presented individually online to 128 participants. Each scenario presented a situation in which there was potential for rejection, based on a woman’s sexual minority identity. Consistent with other measures of social identity-based RS (Chan & Mendoza-Denton, 2008; London et al., 2012; Mendoza-Denton et al., 2002; Pachankis et al., 2008), participants were asked to imagine themselves in each scenario; they were presented with two questions subsequent to each scenario. First, participants were asked, “How anxious/concerned would you be that you would be treated differently or experience a negative outcome because of your sexual orientation?” This question was rated on a 1 (not at all anxious) to 6 (very anxious) scale. Second, participants were asked, “To what extent would you expect to be treated unfairly based on your sexual orientation?” This question was rated on a 1 (very unlikely) to 6 (very likely) scale. The specific wording of each question reflected the content of the specific scenario it followed. Anxious Expectation scores were computed for each scenario by multiplying responses to the two questions, consistent with other measures of RS and with the conceptualization of RS as reflecting the interaction between anxiety about and expectations of rejection. Means and standard deviations for the SMW-RSS for each group of SMW (lesbian, bisexual, queer, and other-identified) for Studies 1b and 2 are presented in Table 2. Means and standard deviations were very similar across groups.

OPEN IN VIEWER

Table 2.

Means and Standard Deviations for SMW-RSS.

	Study 1b			Study 2
Sexual Identity	n	M	SD	n	M	SD
Lesbian	44	17.97	7.59	113	18.76	7.03
Bisexual	50	13.29	7.65	106	16.13	7.02
Queer	17	15.19	5.22	69	17.67	6.74
Other-identified	17	15.30	7.39	12	16.74	6.56

Note. SMW-RSS = Sexual Minority Women Rejection Sensitivity Scale.

Results

We conducted an EFA using maximum likelihood estimation of the Anxious Expectations scores for the 36 potential SMW-RSS scenarios using Mplus Version 7 (Muthén & Muthén, 2012). The Cattell scree test (Cattell, 1966; Tabachnick & Fidell, 2013) indicated that one factor should be extracted, because the first eigenvalue was followed by a substantial drop, and subsequent eigenvalues fell on a nearly straight line. The first four eigenvalues were 15.43, 2.31, 1.88, and 1.65. The first unrotated factor accounted for 42.86% of the total variance in the 36 items. A parallel analysis was conducted with 5,000 permutated samples drawn from the raw data using the parallel analysis function in the n Factors package for R Version 3.1.02 (R Core Team, 2013; Raiche, 2010). The first four 95th percentile eigenvalues for the parallel analysis were 2.38, 2.15, 2.01, and 1.90, indicating that the first two eigenvalues in our EFA were larger than those generated on a randomized set of samples. Because parallel analyses may suggest more factors than necessary, trivial factors indicated by small but statistically significant factor loadings should be trimmed (Buja & Eyuboglu, 1992; O’Connor, 2000). The second eigenvalue from the EFA was not substantially larger than its corresponding 95th percentile parallel analysis eigenvalues (2.31 compared to 2.15 [difference of .16]), it accounted for only 6.42% of the total variance, and the two-factor solution failed to yield a clearly interpretable simple structure. Therefore, we decided the one-factor model was optimal. Due to high overall factor loadings, a more restrictive factor loading cutoff was used for item reduction. Items that had a factor loading of less than .55 (and thus shared less than 30% of their variance with the underlying factor) were not included in the final measure (Comrey & Lee, 1992). When items reflected similar situations and were judged redundant, the item with the higher factor loading was retained. These reductions resulted in retaining 16 items for the final measure (see Table 3; Cronbach’s α = .93). Cronbach’s αs were also examined for the final 16 items scale, with one of each item deleted; the removal of any 1 item from the scale did not have a substantial impact on the Cronbach’s α. Cronbach’s αs with 1 item deleted ranged from .924 to .929.

OPEN IN VIEWER

Table 3.

Factor Loadings of Retained Items for Studies 1b and 2.

	Factor Loadings
Item	EFA	CFA
1. You and your female partner are having dinner together at a restaurant. A male customer approaches your table.	.69	.61
2. You and your female partner are leaving a store together holding hands. A car drives by and the driver honks the horn loudly several times.	.68	.65
3. You are on a date with a woman at a restaurant. Your waiter provides you and your date with poor service.	.72	.62
4. You and your female partner are walking together holding hands. Several men are gathered on a corner outside a bar.	.77	.70
5. You are at a bar with a female friend and an intoxicated male approaches you and attempts to pick you up. You turn him down and he reacts angrily, calling you a “dyke.”	.71	.67
6. You and your female partner are looking to buy a house. After looking at a house together with the realtor, the realtor fails to schedule an appointment to view another house that she represents.	.74	.62
7. A few of your female coworkers regularly try to set you up with men but they never try to set you up with women.	.68	.54
8. You are hanging out with a group of heterosexual female coworkers, and the subject turns to boyfriends and husbands.	.70	.69
9. A new female friend of yours makes negative remarks about lesbians	.72	.59
10. You disclose your sexual orientation to a new friend. Your friend doesn’t express concern about it, but your friendship soon drifts apart.	.73	.55
11. You walk into the locker room at the gym and begin to change. A woman near you moves to a different part of the locker room.	.63	.57
12. Your supervisor begins raising concerns about your performance at work for the first time after you bring your female partner to a company picnic.	.78	.64
13. You notice your relatives looking at you and your female partner at a family reunion, but they don’t come over to talk to you.	.64	.68
14. You and your female partner are getting married. Several of your coworkers do not come to the wedding ceremony.	.69	.66
15. The principal at your child’s elementary school has never spoken to you and your partner at school events, but you often see him speaking with other parents.	.71	.69
16. During a lecture on sexual orientation, your professor includes several stereotypes about lesbian and bisexual women as if they were facts. After the lecture, you approach the professor and politely point out the incorrect aspects of the lecture. You receive a lower grade than you expected in the class.	.71	.56

Note. CFA = confirmatory factor analysis; EFA = exploratory factor analysis.

To determine whether the factor analysis had adequate power to detect the hypothesized effects, a Monte Carlo simulation was run in which the results of the EFA were used to estimate the population parameters. The Monte Carlo simulation model was a one-factor model in which all variables loaded on a single factor. A total of 5,000 replications were run (5,000 samples of 128 participants were drawn) based on the population parameters derived from the actual EFA results. Results from the Monte Carlo simulation indicated that the exploratory analysis conducted had a power of 94–95% to detect significant factor loadings for all items.

Discussion

The purposes of Study 1b were to pilot test the 36 potential SMW-RSS scenarios, identify the best performing scenarios, and examine the factor structure of the selected scenarios for the SMW-RSS. Results of the EFA of the 36 potential scenarios indicated a unitary factor structure with high factor loadings for many scenarios. Scenarios with factor loadings below the threshold were excluded from the final measure. Additionally, among scenarios reflecting similar situations, the scenario with the highest factor loading was retained in the final measure.

Study 2

As a measure of sexual orientation RS among SMW, the SMW-RSS should demonstrate convergent and discriminant validity with constructs from the RS and minority stress literatures. Within the identity-based RS literature, measures of gender and race/ethnicity-based RS demonstrate moderate to high correlations with PRS, other identity-based RS measures, and fear of negative evaluation from others, as all of these constructs share a common component of sensitivity to rejection (Chan & Mendoza-Denton, 2008; London et al., 2012; Mendoza-Denton et al., 2002). In order to be useful, measures of identity-based RS must demonstrate that they outperform related measures of RS in predicting identity-specific outcomes (Chan & Mendoza-Denton, 2008; London et al., 2012; Mendoza-Denton et al., 2002). Therefore, we expected that the SMW-RSS would continue to predict mental health and minority stress outcomes, including generalized anxiety, depression, sexual identity acceptance concerns (concerns regarding the acceptance of one’s sexual orientation by others), concealment motivation (desire to conceal one’s sexual identity), disclosure of sexual orientation, difficulty developing a positive sexual identity, and internalized negativity, when other types of RS (gay-related, gender, and PRS) are controlled for.

While RS and anxiety share some common components, including heightened emotional arousal, they are also distinct. Pachankis and colleagues (2008) distinguished RS from social anxiety, noting that social anxiety is characterized by unreasonable or excessive fear in social situations, while RS focuses on the realistic concerns that sexual minorities have about being rejected by others because of their sexual orientations. Past research using the GRRSS (Feinstein et al., 2012; Pachankis et al., 2008) revealed a medium correlation between sexual orientation RS and social anxiety, indicating that social anxiety and RS are distinct constructs and that RS is positively associated with social anxiety symptoms (Feinstein et al., 2012; Pachankis et al., 2008). It is likely that RS and generalized anxiety are even more distinct, as generalized anxiety is not limited to social situations. Generalized anxiety is characterized by excessive or unrealistic anxiety in one or more aspects of one’s life (American Psychiatric Association, 2013). It is theorized that RS will lead to heightened vigilance for, and anxiety regarding, the potential for rejection, leading to chronic anxiety that may impact a variety of psychosocial outcomes (Downey & Feldman, 1996). Therefore, we expected that a measure of SMW’s RS concerns would predict generalized anxiety. We also expected that the SMW-RSS would outperform other measures of RS in predicting generalized anxiety symptoms among a sample of SMW. This measure should more accurately capture women’s concerns about rejection on the basis of their sexual orientation and thus should be more accurate in predicting generalized anxiety symptoms.

Both RS and minority stress literatures also link identity characteristics, such as centrality and salience of one’s stigmatized identity, with identity-based RS and expectations of rejection. Race-based RS among African Americans and Asian Americans has been linked with higher ethnic identification, a major component of which is centrality of one’s racial/ethnic identity (Chan & Mendoza-Denton, 2008; Mendoza-Denton et al., 2002). Additionally, the identity-based RS model posits that RS leads to sensitivity to and vigilance for rejection based on an identity, which increases the salience of that identity (Downey & Feldman, 1996; Mendoza-Denton et al., 2002). Similarly, minority stress theory postulates that centrality and salience of one’s sexual orientation impact the associations between stressors and psychosocial outcomes (Meyer, 2003), suggesting that stressors such as sexual orientation RS may be associated with the centrality and salience of one’s sexual identity.

In Study 2, we tested a number of hypotheses regarding convergent and discriminant validity in an independent sample of SMW.

Hypothesis 1: The SMW-RSS will be positively correlated with other measures of RS (gay related, gender, and PRS) and fear of negative evaluation.

Hypothesis 2a: The SMW-RSS will be positively correlated with discrimination, acceptance concerns, internalized negativity, concealment motivation, and difficulty developing a positive sexual identity.

Hypothesis 2b: The SMW-RSS will be negatively correlated with outness.

Hypothesis 3: The SMW-RSS will be positively associated with negative mental health outcomes, including anxiety and depression symptoms.

Hypothesis 4: The SMW-RSS will be positively correlated with salience and centrality of sexual orientation.

Hypothesis 5: The SMW-RSS will continue to significantly positively predict anxiety and depression symptoms, concealment motivation, acceptance concerns, difficulty developing a positive sexual identity, and internalized negativity, when gay-related RS (GRRS) is controlled for.

Hypothesis 6: The SMW-RSS will continue to significantly positively predict anxiety and depression symptoms, concealment motivation, acceptance concerns, difficulty developing a positive sexual identity, and internalized negativity when gender RS (GRS) and PRS are controlled for.

Method

Procedure

Data for this study were collected using an Internet-based survey administered via Qualtrics. Participants were recruited from websites that announced volunteer opportunities (e.g., Craigslist) as well as listservs and Facebook groups that target LGB individuals. Participants were offered the opportunity to enter a raffle for one of the six chances to win a US$50 gift card.

Participants

An independent sample of 300 SMW completed the survey, including 113 (37.7%) self-identified lesbian women, 106 (35.3%) self-identified bisexual women, 69 (23.0%) self-identified queer women, and 12 (4.0%) women who identified with other sexual identity labels. Demographic information is presented in Table 1. This sample size was considered adequate, because power analyses indicate that it has greater than .80 power to detect small effect sizes (r = .15) and is considered to be an appropriate sample size for conducting confirmatory factor analyses (CFAs) on models with highly determined factors (16:1 in this analysis) and moderate communalities among items (.29–.48 in this sample; MacCallum et al., 1999, 2001).

Measures

Sexual identity

Participants were asked to self-identify as heterosexual, lesbian, queer, or bisexual. A blank space was also provided for those who identified with an identity that was not listed. This question functioned as a check that participants identified as SMW. Participants who identified as heterosexual (n = 6) were not provided access to the survey.

SMW-RSS

The final SMW-RSS consisted of the 16 scenarios with the highest factor loadings in Study 1b (see Study 1b for a description of the questions that followed each scenario and how the measure was scored). The SMW-RSS demonstrated high inter-item reliability (α = .90). SMW-RSS scores were computed by averaging Anxious Expectation scores across scenarios (range of 1–36). This scoring method is also consistent with other measures of social identity-based RS (Chan & Mendoza-Denton, 2008; London et al., 2012; Mendoza-Denton et al., 2002; Pachankis et al., 2008). Means and standard deviations for the SMW-RSS for each group of SMW (lesbian, bisexual, queer, and other-identified) for Studies 1b and 2 are presented in Table 2. The scenarios and questions for the SMW-RSS are available as Online Supplemental Materials.

GRRSS–revised for lesbian women

The GRRSS (Pachankis et al., 2008; revised for lesbian women by Feinstein et al., 2012) measures anxiety/concern about and expectations of rejection on the basis of one’s sexual orientation. Participants are asked to imagine themselves in each of the 12 scenarios (e.g., “You go to donate blood and the person who is supposed to draw your blood turns to her coworker and says, ‘Why don’t you take this one?’”). The GRRSS is constructed similarly to the SMW-RSS, with anxiety and expectations questions following each scenario, and is scored in the same way, with a range of 1–36. The GRRSS demonstrated high inter-item reliability (α = .88) in this study. In previous research, the GRRSS has demonstrated similarly high inter-item reliability in a sample of predominantly educated European American gay men (α = .91; Pachankis et al., 2008) and a similar sample of lesbians and gay men (α = .92; Feinstein et al., 2012).

Gender Rejection Sensitivity Questionnaire–short (Gender RSQ)

The Gender RSQ (London et al., 2012) measures anxiety/concern, among women about and expectations of rejection on the basis of one’s gender. Participants are asked to imagine themselves in each of the 11 scenarios (e.g., “Imagine that you were just accepted into a graduate program. Your advisor/mentor for the program is a senior male professor. You meet your advisor for the first time on the first day of classes.”). The Gender RSQ is constructed similarly to the SMW-RSS, with anxiety and expectations questions following each scenario, and is scored in the same way, with a range of 1–36. The Gender RSQ demonstrated high inter-item reliability (α = .91) in our sample and has demonstrated similarly high inter-item reliability among ethnically diverse samples of undergraduate college students (α = .92; London et al., 2012).

Personal RSQ

The Personal RSQ (Downey & Feldman, 1996) measures anxiety/concern about, and expectations of, interpersonal rejection that are not attributable to a specific social identity. Participants are asked to imagine themselves in each of the eight scenarios (e.g., “You approach a close friend to talk after doing or saying something that seriously upset him/her.”). One of the eight Personal RSQ scenarios was excluded in the current study, because of the concern that it could measure sexual orientation RS, rather than PRS among sexual minorities (“You go to a party and notice someone on the other side of the room and then you ask them to dance.”). The Personal RSQ is constructed similarly to the SMW-RSS, with anxiety and expectations questions following each scenario, and is scored in the same way, with a range of 1–36. The Personal RSQ had high inter-item reliability (α = .80) in this study and has demonstrated similarly high inter-item reliability in mixed gender and female only samples of predominantly European American undergraduate college students (α ranging from .79 to .91; Downey et al., 1998; Downey & Feldman, 1996).

The Lesbian, Gay, and Bisexual Identity Scale–Revised (LGBIS-R)

The LGBIS (Mohr & Kendra, 2011) is a 27-item measure that assesses eight aspects of LGB identity, including internalized negativity, acceptance concerns, concealment motivation, difficult process of sexual identity development, identity uncertainty, identity superiority, identity affirmation, and identity centrality. Each item is rated on a scale of 1 (strongly disagree) to 6 (strongly agree) and subscale scores are computed by averaging the respective item scores, with subscale scores ranging from 1 to 6. This study focuses on five of these eight subscales: sexual orientation centrality (5 items; α = .89; e.g., “My sexual orientation is a central part of my identity.”), difficult process of sexual identity development (3 items; α = .80; e.g., “Admitting to myself that I’m an LGB person has been a very slow process.”), internalized negativity (3 items; α = .89; e.g., “I wish I were heterosexual.”), concealment motivation (3 items; α = .84; e.g., “I keep careful control over who knows about my same-sex relationships.”), and acceptance concerns (3 items; α = .82; e.g., “I often wonder whether others judge me for my sexual orientation.”). The measure has demonstrated high levels of reliability with Cronbach’s αs for subscales ranging from .72 to .94 in two samples of predominantly European American LGB undergraduate college students (Mohr & Kendra, 2011). In regard to validity, concealment motivation and acceptance concerns are negatively associated with outness, while concealment motivation, internalized negativity, acceptance concerns, and difficult process of sexual identity development are all positively associated with severity of depressive symptomatology (Mohr & Kendra, 2011). Centrality of sexual orientation has been linked with importance of sexual orientation (Mohr & Kendra, 2011).

The OI (Mohr & Fassinger, 2000)

See Study 1a for a description of this measure (α = .93 in this sample).

Salience of sexual orientation

This was operationalized as frequency of sexual orientation–related thoughts and it was measured with 1 item: “How often do you think about your sexual orientation/identity?” The item was rated on a scale of 1 (never) to 6 (very often). This measure is similar to the measure of salience used by Quinn and Chaudoir (2009).

Heterosexist Harassment, Rejection, and Discrimination Scale (HHRDS)

The HHRDS (Szymanski, 2006) is a 14-item measure of the frequency of experiences of discrimination, rejection, and harassment faced by SMW (e.g., “How many times have you been treated unfairly by your employers, bosses, and supervisors because of your sexual identity?”). The original items referred to one’s lesbian identity, so this was changed to sexual identity to be inclusive of various sexual minority identities (e.g., lesbian, bisexual, queer). Items are rated on a scale of 1 (never) to 6 (almost all of the time) and then summed to create the total score (range of 14–84; α = .92). The HHRDS has demonstrated high reliability and validity predominantly among samples of educated European American SMW, with inter-item correlations ranging from α = .87 to .90 (Friedman & Leaper, 2010; Szymanski, 2006). It has also been positively associated with severity of anxiety and depression symptoms and frequency of experiences of gendered-heterosexism (Friedman & Leaper, 2010; Szymanski, 2006).

Brief Fear of Negative Evaluation (BFNE)

The BFNE (Leary, 1983) is a 12-item measure of concerns about being negatively evaluated by others (α = .93; i.e., “I worry about what other people will think of me even when I know it doesn’t make any difference.”). Participants are asked to rate how characteristic of oneself each statement is on a scale of 1 (not at all characteristic of me) to 5 (extremely characteristic of me). Total scores are computed by summing responses to all items (range of 12–60). The BFNE has demonstrated high inter-item reliability among samples of predominantly educated European American sexual minority men (α = .90 and .92; Blashill & Vander Wal, 2010; Pachankis et al., 2008) and a strong positive association with GRRS in a sample of gay men (Pachankis et al., 2008).

Generalized Anxiety Disorder–7 item (GAD-7)

The GAD-7 (Spitzer, Kroenke, Williams, & Lowe, 2006) is a 7-item measure of generalized anxiety symptoms experienced over the past 2 weeks. Participants are asked to rate how often they experience each symptom (α = .94; i.e., “Feeling nervous, anxious, or on edge”) on a scale of 0 (not at all) to 3 (nearly every day). Total scores are computed by summing item responses (range of 0–21). The GAD-7 has demonstrated strong inter-item reliability in samples of predominantly educated European American SMW (αs = .90 and .92; Kaysen et al., 2014; Lehavot & Simoni, 2011). It has also demonstrated a strong positive association with depression symptoms (Kaysen et al., 2014).

The Center for Epidemiological Studies Depression Scale (CES-D)

The CES-D (Radloff, 1977) is a 20-item measure of the severity of depression symptoms over the past week (e.g., “I was bothered by things that usually don’t bother me.”). Items are rated on a scale of 0 (rarely or none of the time) to 3 (most or all of the time) and summed to compute total scores (range of 0–60). The measure demonstrated strong inter-rater reliability in this sample (α = .88) and in other samples of predominantly educated European American SMW (α = .90 – .94; Haines et al., 2008; Kaysen et al., 2014; Levitt, Puckett, Ippolito, & Horne, 2012).

Results

There was less than 1% missing data, which was handled using linear interpolation. Table 4 includes means, standard deviations, and correlations among variables.

OPEN IN VIEWER

Table 4.

Correlations Among Major Study Variables—Study 2.

Variable	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
1. SMW-RSS	—	.73**	.49**	.33**	.39**	.48**	.40**	−.18**	.37**	.13*	.28**	.28**	.21**	.31**	.31**
2. GRRSS		—	.45**	.24**	.23**	.31**	.40**	−.07	.28**	.13*	.18**	.22**	.18**	.17**	.21**
3. GRS			—	.39**	.33**	.39**	.31**	−.20**	.35**	.06	.26**	.22**	.21**	.22**	.20**
4. PRS				—	.29**	.23**	.25**	−.26**	.24**	.18**	.29**	.31**	.33**	.06	.05
5. BFNE					—	.54**	.02	−.15**	.31**	.15**	.22**	.45**	.37**	.26**	.18**
6. AC						—	.21**	−.23**	.48**	.36**	.39**	.26**	.27**	.30**	.21**
7. HHRDS							—	.15*	.14*	−.05	−.04	.21**	.25**	.09	.29**
8. Outness								—	−.28**	−.23**	−.57**	−.14*	−.18*	.02	.32**
9. Difficult process									—	.34**	.37**	.15**	.10	.21**	.10
10. IN										—	.35**	.06	.10	−.01	−.21**
11. CM											—	.16**	.12*	−.07	−.29**
12. Anxiety												—	.76**	.21**	.09
13. Depression													—	.10	.09
14. Salience														—	.48**
15. Centrality															—
M	17.54	15.89	10.15	9.01	3.32	3.50	47.17	4.66	3.32	1.77	3.17	9.30	23.59	4.20	4.24
SD	7.00	7.27	5.66	5.28	0.94	1.39	15.55	1.45	1.39	1.10	1.31	6.41	9.96	1.31	1.22
Range	1–36	1–36	1–36	1–36	12–60	1–6	14–84	1–7	1–6	1–6	1–6	0–21	0–60	1–6	1–6

Note. AC = acceptance concerns; IN = internalized negativity; CM = concealment motivation; RS = rejection sensitivity; SMW-RSS = Sexual Minority Women RS Scale; GRRSS = Gay-Related RS Scale; GRS = gender RS; PRS = personal RS; BFNE = Brief Fear of Negative Evaluation; HHRDS = Heterosexist Harassment, Rejection, and Discrimination Scale.

*p < .05. **p < .01.

CFA

A CFA was conducted on the 16 SMW-RSS items using Mplus Version 7 (Muthén & Muthén, 2012). The first unrotated factor accounted for 39.69% of the total variance in the 16 items. All items had factor loadings of .50 or higher (see Table 3), indicating a good fit for the unitary factor structure. Model fit was assessed by the comparative fit index (CFI), the Tucker–Lewis index (TLI), and the root mean square error (RMSEA), with good model fit indicated by a CFI and TLI > .90 and an RMSEA < .08 (Bentler, 1992; Bentler & Bonett, 1980; Browne & Cudeck, 1993; Kline, 2005). Analyses indicated that the hypothesized model fit the data marginally well (CFI = .85, TLI = .83, RMSEA = .09). This lower model fit is attenuated when the error terms of three pairs of related scenarios are allowed to covary (Scenarios 1 and 2, 2 and 4, and 9 and 10; CFI = .92, TLI = .90, RMSEA = .06; factor loadings remain above .5 for all items). These error term correlations were identified through inspection of modification indices and were thus associated with significant reductions in chi-square, χ²(1) = 30.89; 39.20; and 57.51, respectively, ps < .001.

To determine the power this CFA had to detect the significance of factor loadings, a Monte Carlo simulation was run in which the results of the CFA were used to estimate the population parameters. A total of 5,000 replications were run (5,000 samples of 300 participants were drawn) based on the population parameters derived from actual CFA results. Results indicated that the CFA had a power of 94–95% to detect significant factor loadings for all items.

Partial Correlations

A series of partial correlations were conducted to test Hypotheses 1–4, controlling for age (see Table 5). Age was controlled because it was significantly correlated with SMW-RSS (r = −.12, p < .05) and several other variables (e.g., outness r = .25, p < .001, anxiety symptoms r = −.23, p < .001). As hypothesized (Hypothesis 1), SMW-RSS was moderately to highly correlated with GRRS, GRS, PRS, and fear of negative evaluation. In line with Hypothesis 2, SMW-RSS was moderately, positively correlated with acceptance concerns, experiences of sexual orientation–based discrimination, difficulty developing a positive sexual identity, and had small positive correlations with internalized negativity and concealment motivation. SMW-RSS also had a small significant negative correlation with outness. Consistent with hypotheses, SMW-RSS demonstrated small significant positive correlations with anxiety and depression symptoms (Hypothesis 3) as well as moderate significant positive correlations with salience and centrality of sexual orientation (Hypothesis 4).

OPEN IN VIEWER

Table 5.

Partial Correlations Controlling for Age.

Variable	Partial Correlation With SMW-RSS
GRRS	.73**
GRS	.49**
PRS	.33**
BFNE	.39**
Acceptance concerns	.48**
HHRDS	.40**
Outness	−.18**
Difficult process	.37**
Internalized negativity	.13*
Concealment motivation	.27**
Anxiety	.28**
Depression	.21**
Salience	.31**
Centrality	.31**

Note. RS = rejection sensitivity; SMW-RSS = Sexual Minority Women RS Scale; GRRSS = Gay-Related RS Scale; GRS = gender RS; PRS = personal RS; BFNE = Brief Fear of Negative Evaluation; HHRDS = Heterosexist Harassment, Rejection, and Discrimination Scale.

*p < .05. **p < .01.

Hierarchical Regressions

Six hierarchical regression analyses were conducted to test our hypothesis that SMW-RSS would continue to predict mental health and minority stress variables when GRRS is controlled for (Hypothesis 5). In Step 1, age was entered as a control (results not reported) and GRRS was entered as a predictor. In Step 2, SMW-RSS was entered as a predictor. Standardized regression coefficients for Steps 1 and 2 are reported for the two predictors of interest (GRRS and SMW-RSS), the change in R² from Step 1 to Step 2 (addition of SMW-RSS), and Holm–Bonferroni adjusted p values are also reported in Table 6. Holm–Bonferroni corrections were made to reduce the family-wise error rate and were calculated using the p.adjust function in the stats package for R Version 3.1.02 (R Core Team, 2013), which are reported in Table 6. The Holm–Bonferroni correction is less conservative than the Bonferroni correction but continues to appropriately adjust for family-wise error, and this method is generally recommended over the Bonferroni correction (Aickin & Gensler, 1996; Holm, 1979; Levin, 1996; Wright, 1992). The inclusion of SMW-RSS as a predictor was associated with a significant change in R², indicating that SMW-RSS significantly predicted anxiety symptoms, concealment motivation, acceptance concerns, and difficulty developing a positive sexual identity over and above GRRS but not depression symptoms or internalized negativity. In contrast, GRRS was not significantly associated with any variable when SMW-RSS was also entered as a predictor. Thus, Hypothesis 5 was largely supported.

OPEN IN VIEWER

Table 6.

Divergent Validity Regression Analyses.

		Step 1 Regression Coefficients					Step 2 Regression Coefficients					Change in R2 for Step 2
Dependent Variable	Independent	β	t	p	Adjusted p	sr 2	β	t	p	Adjusted p	sr 2	Δ R2	F	p	Adjusted p
Anxiety	GRRS	.22	4.01	<.001	<.001	.052	.08	0.94	.347	1.000	.003
	SMW-RSS						.20	2.44	.015	0.045	.020	.018	5.98	.015	.045
Depression	GRRS	.18	3.17	.002	.006	.032	.09	1.01	.311	1.000	.003
	SMW-RSS						.13	1.55	.122	0.244	.008	.008	2.41	.122	.244
Concealment motivation	GRRS	.18	3.13	.002	.006	.032	−.04	−0.51	.614	1.000	.001
	SMW-RSS						.30	3.63	<.001	0.001	.043	.041	13.18	<.001	.001
Acceptance concerns	GRRS	.32	5.78	<.001	<.001	.102	−.04	−0.51	.612	1.000	.001
	SMW-RSS						.49	6.37	<.001	<0.001	.063	.107	40.52	<.001	<.001
Difficult process	GRRS	.29	5.16	<.001	<.001	.084	.06	0.76	.446	1.000	.002
	SMW-RSS						.31	3.83	<.001	<0.001	.048	.043	14.67	<.001	<.001
Internalized negativity	GRRS	.13	2.32	.021	.021	.018	.12	1.37	.171	1.000	.006
	SMW-RSS						.02	.27	.790	0.790	<.001	<.001	0.07	.790	.790
Anxiety	GRS	.09	1.43	.153	.459	.007	.02	.27	.791	1.000	<.001
	PRS	.27	4.62	<.001	<.001	.068	.25	4.14	<.001	<0.001	.056
	SMW-RSS						.16	2.60	.010	0.040	.023	.019	6.75	.010	.040
Depression	GRS	.07	1.23	.221	.459	.005	.04	.62	.536	1.000	.001
	PRS	.30	5.00	<.001	<.001	.079	.28	4.71	<.001	<0.001	.071
	SMW-RSS						.08	1.23	.219	0.332	.005	.005	1.52	.219	.332
Concealment motivation	GRS	.17	2.75	.006	.024	.025	.11	1.60	.112	0.448	.009
	PRS	.22	3.67	<.001	.001	.044	.20	3.23	.001	0.004	.035
	SMW-RSS						.15	2.30	.022	0.066	.018	.016	5.29	.022	.066
Acceptance concerns	GRS	.35	5.89	<.001	<.001	.107	.19	3.19	.002	0.012	.034
	PRS	.09	1.60	.111	.111	.009	.03	0.59	.559	0.559	.001
	SMW-RSS						.36	6.22	<.001	<0.001	.118	.097	38.71	<.001	<.001
Difficult process	GRS	.30	5.00	<.001	<.001	.079	.20	3.09	.002	0.012	.032
	PRS	.13	2.13	.034	.068	.015	.09	1.46	.144	0.288	.007
	SMW-RSS						.24	3.89	<.001	0.001	.050	.042	15.14	<.001	.001
Internalized negativity	GRS	−.01	−0.12	.904	.904	<.001	−.05	−0.68	.500	1.000	.002
	PRS	.18	2.91	.004	.012	.028	.17	2.63	.009	0.027	.023
	SMW-RSS						.09	1.39	.166	0.332	.007	.006	1.93	.166	.332

Note. Age was entered as a control in both levels. sr² = semi-partial correlation squared; RS = rejection sensitivity; SMW-RSS = Sexual Minority Women RS Scale; GRRS = gay-related RS; GRS = gender RS; PRS = personal RS.

An additional six hierarchical regression analyses were conducted to test our hypothesis that SMW-RSS would continue to significantly predict mental health and minority stress variables when GRS and PRS are controlled for (Hypothesis 6). In Step 1, age was entered as a control (results not reported), and GRS and PRS were entered as predictors. In Step 2, SMW-RSS was entered as a predictor, standardized regression coefficients for Steps 1 and 2 are reported for the three predictors of interest (GRS, PRS, and SMW-RSS), the change in R² from Step 1 to Step 2 (addition of SMW-RSS) and Holm–Bonferroni adjusted p values are also reported. Results are presented in Table 6. Hypotheses 5 and 6 were tested separately to ensure that moderate to high multicollinearity between the measures of sexual orientation RS and the two other measures of RS did not obscure differences in predictive validity between the SMW-RSS and the GRRS. The inclusion of SMW-RSS as a predictor resulted in a significant increase in R² for anxiety symptoms, acceptance concerns, and difficult process of sexual identity development. This indicates that SMW-RSS significantly predicted anxiety symptoms, acceptance concerns, and difficulty developing a positive sexual identity, over and above GRS and PRS, but not depression symptoms or internalized negativity. GRS was significantly associated with acceptance concerns and difficult process of sexual identity development when SMW-RSS was also a predictor, while PRS was significantly associated with five of the six variables when SMW-RSS was also a predictor. Thus, Hypothesis 6 was largely supported, as the SMW-RSS continued to significantly predict three of the six variables even when two other measures of RS were included in models.

Discussion

The purposes of Study 2 were to confirm the unitary factor structure of the SMW-RSS and to examine the validity of the SMW-RSS. The model fit indices indicated moderate model fit for the unitary factor structure of the SMW-RSS; however, a significant proportion of variance in each item remained unique. This remaining unique item variance is likely explained by the wide range of situations reflected in the measure. Because scenarios were chosen to reflect a variety of situations that trigger concerns about rejection based on sexual orientation among SMW, these items were expected to share some common variance but also, by definition, reflected unique situational aspects of sexual orientation rejection concerns. By allowing the error variance of three pairs of items to covary, model fit was significantly improved and all factor loadings remained high. The three pairs of items each shared common context, theme, and/or perpetrator characteristics, which explain their additional shared variance. The poor model fit indices of the unmodified CFA did not indicate that a one-factor model is a poor model of the measure due to (1) high factor loadings for all items on the single factor, (2) the measures’ aim to capture a wide range of situations which elicit concern about sexual orientation–based rejection among SMW, and (3) additional shared variance among scenarios with shared components.

Convergent validity was supported with correlations between the SMW-RSS and other measures of RS, fear of negative evaluation, measures of minority stress, sexual orientation salience and centrality, and internalizing symptoms. Divergent validity was supported by the findings that the SMW-RSS continued to predict several minority stress and mental health variables when other measures of RS were controlled for. The unique aspects of the SMW-RSS remained significant even when the shared variance among multiple measures of RS was held constant.

General Discussion

This article describes the development and preliminary validation of a measure of sexual orientation RS specifically for SMW. The SMW-RSS captures the range of unique situations in which SMW experience anxiety and expect to be rejected or discriminated against. The SMW-RSS demonstrated high inter-item reliability, a unitary factor structure, and adequate variance among participants in two separate samples. In addition, convergent and discriminant validity were supported with moderate to large correlations between the SMW-RSS and other measures of RS as well as constructs related to sexual orientation (e.g., centrality and salience), minority stress, and negative mental health outcomes.

The SMW-RSS also predicted most minority stress and mental health variables over and above the existing measure of sexual orientation RS that was developed for gay men (the GRRSS). This highlights the importance of group-specific measures to assess RS, given that the situations in which individuals expect to be rejected differ depending on the group. Although Feinstein and colleagues (2012) demonstrated that the GRRSS was associated with relevant psychosocial outcomes among lesbians, the current findings suggest that the GRRSS does not adequately capture the range of unique situations in which SMW expect to be rejected. This is consistent with recent work that has demonstrated that SMW experience unique concerns related to rejection compared to sexual minority men (Friedman & Leaper, 2010; Hequembourg & Brallier, 2009; Kertzner et al., 2009; Szymanski, 2005).

Contrary to our hypothesis, neither the SMW-RSS nor the GRRSS predicted internalized negativity or depression symptoms when both measures were included in models. There are several potential explanations for this finding. First, many of our participants were recruited from LGB organizations. They may have relatively low levels of internalized negativity and depression symptoms due to the support and sense of community provided by the organizations; high levels of internalized negativity and depression symptoms may prevent SMW from becoming involved in LGB community organizations. Second, it is noteworthy that internalized negativity and depression symptoms were associated with both sexual orientation RS measures when the other measure was not controlled for. This may indicate that the shared variance of the two measures of sexual orientation RS, rather than each measure’s unique variance, is responsible for the associations between sexual orientation RS and both internalized negativity and depression symptoms.

We also demonstrated that the SMW-RSS significantly predicted anxiety symptoms, acceptance concerns, and difficulty developing a positive sexual identity when GRS and PRS were controlled for. This suggests that sexual orientation RS has an influence on psychosocial outcomes over and above the influences of general expectations of rejection and those based on gender. Additionally, PRS, but not GRS, was a significant predictor of anxiety and concealment motivation in these analyses. This is consistent with the previous findings that PRS is associated with anxiety (Downey & Feldman, 1996; London et al., 2012), and PRS reflects general concerns about social rejection, which are closely related to the motivation to conceal one’s sexual identity. Finally, GRS was also a significant predictor of difficulty developing a positive sexual identity and acceptance concerns. The findings that both the SMW-RSS and the GRS were associated with difficulties in sexual identity development and acceptance concerns may reflect the importance of both stigmatized identities in understanding these constructs among SMW.

Contrary to our hypotheses, the SMW-RSS did not significantly predict internalized negativity or depression symptoms when GRS and PRS were controlled for. However, multicollinearity between the three measures may be responsible for the non-significant findings when all of the measures were entered simultaneously. In addition, range restriction in internalized negativity and depression symptoms may have attenuated the associations between SMW-RSS and these outcomes. Internalized negativity was particularly restricted in our sample; more than 50% of the women reported no internalized negativity and less than 5% reported scores above the midpoint of the scale. Depression symptoms were also restricted in our sample, with two thirds of participants in the 14–32 score range out of a possible range of 0–60. In regard to internalized negativity, only PRS was a significant predictor. This was surprising as PRS measures general concerns about social rejection, while internalized negativity focuses on internalized negative attitudes toward one’s sexual identity. However, the measure of PRS was administered after the SMW-RSS. The priming of sexual orientation may have led participants to think of PRS items in terms of sexual identity, potentially contributing to this association.

Practice Implications

The results of these studies indicated that SMW’s experiences of stigmatization and discrimination are often distinct from those of sexual minority men, although all sexual minorities do share some common stigmatization experiences. These distinct experiences lead sexual minority men and women to be concerned about social rejection based on their sexual orientations in some different contexts. Clinicians and researchers should be cognizant of the unique experiences and concerns of SMW.

These studies also indicate that sexual orientation RS is associated with a variety of negative sexual identity and mental health outcomes among SMW, including internalizing symptomatology, sexual identity concealment, internalized negativity, and difficulty developing a positive sexual identity. Theory indicates that sexual orientation RS may be a mechanism through which experiences of stigmatization and discrimination impact sexual identity development and mental health among sexual minorities. Although experiences of discrimination may be largely outside of an individual’s control, clinicians can help clients to reduce anxious expectations of rejection as a way to improve well-being. In line with this notion, Pachankis and colleagues recently developed the first treatment to specifically target minority stress and co-occurring health problems (depression, anxiety, substance use, and sexual risk behavior) for gay and bisexual men (Pachankis, 2014; Pachankis, Hatzenbuehler, Rendina, Safren, & Parsons, 2015). Their intervention includes techniques to restructure negative cognitions stemming from experiences of minority stress, such as expectations of rejection. Their intervention focuses on gay and bisexual men, but similar techniques could be used to help SMW cope with their own expectations of rejection. It will be important for future research to continue to examine the extent to which sexual orientation RS mediates the associations between experiences of discrimination and negative psychosocial outcomes.

Limitations and Future Directions

The current findings should be considered in light of several study limitations. First, our use of a non-probability sample of self-identified SMW affiliated with LGBT organizations and listservs limits the extent to which findings can be generalized to the broader population of SMW. Second, our data were cross-sectional and cannot support causal inference. Third, our samples were relatively young and had limited racial/ethnic diversity, so our findings may not generalize to older and racially/ethnically diverse SMW. Given the intersectionality of multiple stigmatized identities, our measure may not reflect all of the unique concerns of subgroups of SMW (e.g., older individuals and racial/ethnic minorities). Fourth, our samples predominately were educated women from urban and suburban regions of the United States, United Kingdom, Australia, and Canada. The extent to which findings generalize to less educated women and those from rural areas and non-Western countries is unknown. Fifth, we did not examine test–retest reliability of the SMW-RSS, so it will be important for future longitudinal studies to examine whether sexual orientation RS is stable over time. Finally, the measure developed and validated in this set of studies measures sexual orientation RS among SMW in general. As the minority stress experiences of lesbians and bisexual women differ (e.g., Dyar, Feinstein, & London, 2015), bisexuals may have specific rejection concerns that are not reflected in this measure. For example, bisexuality is often stereotyped as a phase or an unstable and invalid sexual orientation/identity, and bisexuals often face stigmatization from lesbians/gay men as well as heterosexuals (Brewster & Moradi, 2010). As a result, bisexuals may be concerned about rejection in situations that do not elicit concern for other SMW. In addition, we did not examine situations which evoke RS concerns among transgender and genderqueer individuals as a result of their distinct stigmatization experiences and the intersectionality of their multiple stigmatized identities. Future research should examine RS among transgender and genderqueer individuals.

Despite limitations, the current findings demonstrate the preliminary reliability and validity of a new measure of sexual orientation RS for SMW. We encourage researchers to consider the role of sexual orientation RS in studies of minority stress and mental health among sexual minorities and to use group-specific measures to capture the unique concerns of subgroups of sexual minorities. In sum, given that sexual orientation RS is associated with a variety of negative psychosocial outcomes, it is likely to be one of the mechanisms through which experiences of discrimination impact mental health and sexual identity development.