Examining the Measurement Invariance of the Conformity to Masculine Norms Inventory (CMNI-30) by Sexual Orientation

Abstract

An abbreviated version of the Conformity to Masculine Norms Inventory, the CMNI-30, was developed with several strengths. However, its measurement invariance across men with different sexual orientations has not been examined in a U.S. sample, so it is unclear whether these different populations understand the items similarly. In addition, no studies have compared conformity to masculinity norms across sexual orientations. This would be important to understand sexual minority men’s experience of masculinity and how it differs from heteronormative masculinity. This article explores the measurement invariance of the CMNI-30 among 882 heterosexual, gay, and bisexual+ men using confirmatory factor analysis, and compares their CMNI-30 subscale scores. Results indicated that the CMNI-30 demonstrated residual invariance between men of different sexual orientations, suggesting that men of different sexual orientations interpreted the items similarly. We also found higher levels of conformity to the masculine norms of Winning, Heterosexual self-presentation, and Power over women among heterosexual men relative to gay and bisexual+ men, and higher levels of Pursuit of status among gay men relative to the other groups. Results provide support for the use of the CMNI-30 in research with men of different sexual orientations.

Keywords

masculine norms psychometrics measure invariance sexual minority men

The Conformity to Masculine Norms Inventory (CMNI-30; Mahalik et al., 2003) has been a staple of research on masculinity informing practice including the development of the American Psychological Association’s guidelines for psychological practice with boys and men (American Psychological Association [APA], Boys and Men Guidelines Group, 2018). Research on the original version of the inventory has provided consistent support for its reliability, validity, and factor structure, but its 94-item length has been a concern for some researchers. Although several short forms have been developed (e.g., Hsu & Iwamoto, 2014; Owen, 2011; Parent & Moradi, 2011), the most recent is a promising 30-item version demonstrating a strong factor structure, internal consistency, and partial scalar and residual invariance across racial groups (Levant et al., 2020).

However, the measurement invariance across other groups, for example, heterosexual, gay, and bisexual men, has not been previously established in a U.S. sample. Exploring how this scale functions in these different populations is essential not only for our understanding of the construct and how sexual orientation might be related to perceptions of masculinity but also for practical purposes. If the instrument works differently among sexual minority men, it may lead to an inaccurate interpretation of the role of masculinity in the lives of these men, and thus to invalid findings examining issues commonly assessed in relation to the CMNI such as psychological distress, substance use, and help-seeking for mental and physical health concerns. The purpose of the current study is to test and establish the measurement invariance of the new, shorter CMNI-30 between heterosexual and sexual minority men. In addition, the study aims to explore differences in conformity to the 10 masculinity norms measured by the CMNI-30 among men of different sexual orientations.

Masculinity Norms

Masculinity norms, or those rules and standards that guide and constrain boys’ and men’s behavior (Mahalik et al., 2003), have been a long-studied issue in sociology and psychology (see Hearn et al., 2012; Smiler, 2004 for reviews). Not only are masculinity norms an important sociological phenomenon that affects society as a whole, but conformity to those norms is of particular interest to health practitioners due to its correlation. For example, studies have linked conformity to masculinity with important health risk behaviors (e.g., Mahalik et al., 2007; Mahalik et al., 2022a) and mental health outcomes such as depression, substance use, negative social functioning, body image problems, stress, and reluctance to seek help (see Wong et al., 2017 for a meta-analysis). Some elements of conformity to masculinity norms like not expressing one’s emotions and avoiding affectionate behaviors are reported to be especially important when predicting negative mental health outcomes. However, specific elements such as prioritizing work over other parts of one’s life have positive correlates like motivation to succeed, courage, and resilience (Gerdes & Levant, 2018; Hammer & Good, 2010). Thus, understanding conformity to masculinity norms and its facets is important when exploring men’s mental health, and can help mental health professionals to identify risk and resilience factors.

Although previous research notes that conformity to masculinity norms is associated with a range of psychological and physical health outcomes, traditional masculinity may also function to stratify men into groups. Hegemonic masculinity theory suggests that while there are many interpretations and enactments of masculinity, there is a single dominant form of masculinity that is positioned above femininity and subordinate types of masculinity (Connell & Messerschmidt, 2005). When men conform to traditional norms, they potentially enable their domination of others and align themselves with the power privileged to this social construction of masculinity. For men from historically marginalized groups, conforming to these masculine norms may grant them some power normally reserved for men from dominant groups (i.e., White, heterosexual, cisgender).

In the case of sexual minority men, certain masculine norms may be useful for shielding them from experiences of discrimination as well. For example, young sexual minority men who make their self-worth contingent on academic competence were more likely to conceal their sexual orientation (Pachankis & Hatzenbuehler, 2013). That is, sexual minority men may use success in traditionally masculine domains (i.e., one’s career) as a type of armor against stigma and discrimination due to their sexual orientation.

Yet, there are costs to this phenomenon. Sexual minority men who engage in such behaviors often suffer heightened social isolation and emotional distress (Pachankis & Hatzenbuehler, 2013). There is also evidence that dominant masculinity affects sexual minority men’s views of other sexual minority men, with previous studies noting that gay men rated masculine men as more desirable partners and as more likable than effeminate gay men (Bailey, 1996; Sánchez, 2016; Skidmore et al., 2006). Others have even said that “It seems ironic that gay men use the same masculine standards they were excluded by to exclude each other” (Pezzote, 2008, p. 63, as cited in Sánchez, 2016).

As such, masculinity norms have an important role in the life experiences of sexual minority men. This raises questions about whether scales that assess “traditional masculinity” perform similarly with heterosexual and sexual minority men, namely, whether these scales are invariant with respect to sexual orientation. Without exploring this issue, it is difficult to justify the use of such scales for comparing conformity to masculinity norms across sexual orientation groups, limiting our understanding of how masculinity constructs contribute to the psychological and physical health of all men, particularly sexual minority men.

Measures of Masculinity Norms and the CMNI

Although several quantitative measures have been developed to assess masculinity constructs (see Thompson & Bennett, 2015 for a discussion of existing measures’ strengths and weaknesses), Mahalik et al.’s (2003) CMNI has been one of the most cited and used measures of masculinity related constructs in the last 20 years. It assesses a broader array of masculine norms than comparable measures (e.g., the Male Role Norms Inventory; Levant et al., 1992), and has consistently shown robust psychometrics across studies and versions (e.g., Hammer et al., 2018; Parent & Moradi, 2011). In addition, while other instruments (e.g., the Brannon Masculinity Scale and the Male Role Attitudes Scale; Brannon & Juni, 1984; Pleck et al., 1993) focus on agreement with cognitive statements representing ideal masculine behavior (e.g., “It is best to keep your emotions hidden”), the CMNI also encompasses actual behaviors related to these ideals (e.g., “I frequently put myself in risky situations”) and feelings about the masculinity norms (e.g., “It bothers me to ask for help”). These strengths contribute to a more comprehensive assessment of the construct in comparison with similar measures.

Since its development in 2003, the CMNI has been used to identify how conformity to masculine norms is associated with an array of presenting concerns and help-seeking dynamics. The measure can be used with male-identifying clients to develop men’s awareness and understanding of masculinity issues as they relate to presenting concerns (Mahalik et al., 2005). Indeed, hundreds of studies using the CMNI document the connection between conforming to traditional masculine norms and men’s psychological development, mental health, and physical well-being (see APA Boys and Men Guidelines Group, 2018; Wong et al., 2017). As such, the CMNI has been a useful tool for gender-sensitive psychological practice with men. The CMNI has also been utilized in samples of sexual minority men, with previous work demonstrating links to body image concerns, internalized stigma, and health risk behaviors (Hamilton & Mahalik, 2009; Kimmel & Mahalik, 2005).

Mahalik et al.’s (2003) original instrument included 94 items that loaded on 11 factors: Winning, Emotional control, Risk-taking, Violence, Power over women, Dominance, Playboy, Self-reliance, Primacy of work, Disdain for homosexuals (now termed “Heterosexual self-presentation”), and Pursuit of status. However, the CMNI’s length has led multiple other authors to create shorter versions, such as Owen’s (2011) CMNI-55, Parent and Moradi’s (2011) CMNI-46, and Hsu and Iwamoto’s (2014) CMNI-29. Although shorter versions are useful as they mitigate the problem of respondents’ fatigue, Levant et al. (2020) argue that these shorter versions are either still too long or suffer from one or more methodological issues such as poor model-data fit.

As a result, Levant et al. (2020) proposed a new version of the CMNI containing 30 items. They slightly modified some of the original items and then used exploratory factor analysis to find low-loading or cross-loading items; following this step, the Dominance subscale was removed from the CMNI. This was expected given the poor psychometric properties of the Dominance scale as reported in other studies (Mahalik et al., 2003; Parent & Moradi, 2009). Then, confirmatory factor analysis (CFA) was applied to compare several competing models: a model with three items loading on each factor, a unidimensional model without factors, a second-order model (where three items load on each factor and the 10 factors load on a single, higher-order masculinity factor), and a bifactor model (where the items themselves and not the 10 factors load on the general masculinity factor). The results suggested that the 10-factor model had the best-fit statistics and so it was preferred over the others. In doing so, Levant et al. (2020) were able to address the length and fit issues reported in previous versions of the CMNI while also retaining key features of the original CMNI including its multidimensional assessment of masculinity norms and its behavioral, emotional, and cognitive features.

Measurement Invariance and the CMNI-30

Levant et al. (2020) also explored the CMNI-30’s measurement invariance with respect to race. Measurement invariance refers to a property of a scale that has the same psychometric structure across groups; if this is the case, this suggests that individuals understand the items similarly regardless of their group affiliation, and so their scores can be meaningfully interpreted and compared. For example, if a CFA of the CMNI in Group A shows that the item “It bothers me to ask for help” loads on the Self-reliance factor, but a similar analysis with members of Group B shows it loads on the Primacy of Work factor, this suggests that the item is not interpreted in the same way by the group members, rendering any comparison of their scores meaningless (e.g., one cannot say that group A is higher on the Self-reliance factor than Group B as the items do not carry the same meaning for members of these groups). Because of this, ignoring measurement invariance can pose a threat to the validity of an instrument’s scores (Guenole & Brown, 2014), as one cannot assume that the scores are appropriate for their intended uses (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014).

There are several levels of measurement invariance: configural, metric, scalar, and residual (Bowen & Masa, 2015; Putnick & Bornstein, 2016; van de Schoot et al., 2012). Configural invariance means that the scale shares a similar factor structure across groups. Metric invariance adds the equivalence of the factor loadings across groups (i.e., the items have a similar contribution to the latent variable). Scalar invariance adds an equivalence of the intercepts (i.e., for the same levels of the latent variable, the observed scores are the same across groups), and residual invariance means that the residual variances are also equal across groups (i.e., the factors also account for the same part of the observed variance across the groups). Scales with at least scalar invariance are considered psychometrically acceptable because scalar invariance means that any observed differences may be directly attributed to differences in the latent construct, so groups can be meaningfully compared. However, sometimes it is possible to establish partial invariance at each of these stages, where some level of invariance is achieved with the exception of differences in a small number of the items.

Levant et al. (2020) found that the CMNI-30 has a partial strong invariance when comparing White and non-White men, with some items having a larger intercept among White men and others among men of color. Since the differences in the intercepts were small, the authors concluded that the CMNI-30 is suitable for comparisons across these groups. However, other groups of men might not interpret the CMNI-30 in the same way. In particular, they did not test the instrument’s invariance with respect to sexual orientation.

Sexual minority men are exposed to unique stressors related to their stigmatized social status (Meyer, 1995, 2003) and are often viewed as breaking from traditional masculinity (Sánchez et al., 2009). So, it is necessary to explore whether they interpret the scales similarly to other men before comparing these groups. The question of whether men of different sexual orientations interpret the CMNI similarly is important as research reports the salience of masculinity constructs as a major contributor to the psychological and physical health of sexual minority men (Fischgrund et al., 2012; Hamilton & Mahalik, 2009; Mahalik et al., 2005; Sanchez, 2016).

Hegemonic masculinity theory may again offer some additional context to the potential invariance or non-invariance between heterosexual and sexual minority men on the CMNI-30. Since being heterosexual is central to the perceived ideal form of masculinity, heterosexual men may consider masculine norms to be aligned with an identity they view as proximal, while sexual minority men may view traditional masculinity as inapplicable to their sense of self. For example, where heterosexual men might consider Heterosexual self-presentation as an inherent part of masculinity, others might not, meaning that the way men of different groups understand masculinity may vary. In practice, this dynamic could cause them to respond in such a way that the same items load on different factors, resulting in measurement non-invariance between these two groups. However, other scholars have noted that very few men can achieve the social construction of ideal masculinity (Morris & Ratajczak, 2019) so heterosexual and sexual minority men may both consider conformity to masculine norms as unattainable such that they would interpret the CMNI items similarly.

We are aware of only one study exploring measurement invariance of the CMNI across different sexual orientation groups (Krivoshchekov et al., 2022), though notably, that study used a somewhat different version of the CMNI-30 and with a Russian sample. In their study, Krivoshchekov et al. (2022) tested several potential models with the items that survived Levant et al.’s (2020) exploratory factor analysis (i.e., without the Dominance scale and other items). Their tests resulted in a similar model to the one identified by Levant et al., though some of the items in Krivoshchekov et al.’s version were different from those identified in Levant et al.’s. For example, Levant et al.’s version included the item “I think that violence is sometimes necessary,” but Krivoshchekov et al.’s version included the item “Even if a person made me very angry, I would not use violence,” instead. Using their version of the CMNI-30, Krivoshchekov et al. (2022) established partial scalar invariance across heterosexual and sexual minority men, after allowing the intercepts to vary in one item (“I bring up my feelings when talking to others”). The authors concluded that overall, the two groups have similar understandings of the items in their version.

Krivoshchekov et al.’s work provides important evidence on the CMNI’s appropriateness for use among sexual minority men but is limited in several ways. Most importantly, their instrument is different from Levant et al.’s with some items written differently due to cultural differences. Second, as Russian legislation currently bans same-sex marriages (Reuters, 2020), and there have been reports of state-sponsored violence against gay people (Amnesty International, 2019), sexual minority men in Russia may be more inclined to conform to traditional masculinity norms to protect themselves relative to sexual minority men in the United States. This may affect how they interpret some of the CMNI items. For instance, as homosexuality is less acceptable in Russia, sexual minority men may be just as likely as heterosexual men to consider Heterosexual self-presentation as a central aspect of masculinity there. Conversely, heterosexual men in the United States might agree that Heterosexual self-presentation is an important aspect of masculinity, but sexual minority men in the United States may not, resulting in non-invariance when comparing these groups. Therefore, to make sure that the instrument is appropriate for the U.S. population, it is important to test the measurement invariance of Levant et al.’s (2020) CMNI-30 with respect to sexual orientation in the United States.

Conformity to Masculinity Norms Among Different Sexual Minority Groups

The main purpose of establishing measurement invariance of the CMNI-30 is to determine if differences among sexual orientations are due to actual differences between groups or whether they are an artifact of the way persons from different groups interpret or experience the items differently. If this is confirmed, it is then possible to explore group-based differences in the CMNI-30. This has two important implications. Theoretically, understanding patterns of conformity to masculinity norms can inform researchers on whether some norms are more important to some men than others, enhancing our understanding of what masculinity means for these men and how they may experience the world if their concept of masculinity diverges from the hegemonic one. Practically, as conformity to specific masculinity norms is associated with significant mental health outcomes (see Wong et al., 2017), comparing conformity to these norms among different sexual minority men can help identify potential risks or protective factors that can inform counselors.

In spite of the importance of identifying differences in certain norms among different men, previous studies did not discuss the heterogeneity within sexual minority populations and how it affects their conformity to masculinity norms. Instead, they grouped all sexual minority men together in their comparisons to heterosexual men. This approach neglects to address how the diversity of sexual minority men’s identities affects their enactments of masculinity. For example, gay men may experience Heterosexual self-presentation as a less important aspect of masculinity than bisexual+ men (an umbrella term for men attracted to more than one gender; see Davila et al., 2018), especially those in relationships with women. This may result in different item loadings on this subscale as well as different item scores, as bisexual+ men may find “passing” as heterosexual to be more central to their concept of masculinity than gay men (Sánchez et al., 2009). Therefore, it is important to examine the CMNI-30 measurement invariance across different sexual minority identities, and, if the CMNI-30 is found to be invariant, compare the scores of these groups. Doing so will reveal whether the use of the CMNI-30 is appropriate among sexual minority men and facilitate our understanding of whether and to what extent they conform to traditional masculinity norms.

The Current Study

The purpose of the current study was to test the CMNI-30’s measurement invariance with respect to sexual orientation in a American sample. We do so by first comparing heterosexual and sexual minority men, and then comparing heterosexual men with gay and bisexual+ men separately. We also relied on the invariance analysis results to compare the subscale scores across these groups of men. Doing so may inform researchers and practitioners about the utility and interpretability of the CMNI-30, particularly for comparing men based on their sexual orientation. It can also help them understand the different conceptions of masculinity in the United States and whether they vary by sexual orientation.

Method

Procedure

The data were collected as a part of two separate studies conducted within a few months of each other. The purpose of the first study was to test men’s attitudes toward wearing masks during COVID-19 (Mahalik et al., 2022b). The study included 589 participants (91.17% heterosexual, 3.56% bisexual, 4.24% gay, and 1.02% other). The participants completed a 10 to 15 minutes anonymous online survey through Prolific, an online platform well-suited for the social and behavioral sciences (Palan & Schitter, 2018). The pre-screening criteria for eligible participants included: (a) identifying as a man, either cisgender or transgender, (b) living in the United States, and (c) being at least 18 years of age. Prolific users that met these criteria were able to access the survey and choose to participate. In addition to the CMNI-30, the survey included other scales related to different COVID-19 attitudes. Those who completed the survey were compensated USD1.27, equating to an average rate of USD7.48 per hour as reported in Prolific’s fair wage calculation for the study. Researchers complied with APA ethical guidelines, and the procedures were approved by the Boston College Institutional Review Board (approval number 21.045.01e-1).

The second study explored intimate partner violence (IPV) among sexual minority men (Harris & Mahalik, under review). This study had 293 participants (46.07% gay, 43.00% bisexual, 4.09% heterosexual, and 6.82% other). They completed another survey via Prolific including the CMNI-30 and a scale about IPV experiences; this survey took about 15 to 20 minutes to complete. The survey focused on non-heterosexual men, and so our screening criteria were: (a) identifying as a man (cisgender and transgender), (b) being at least 18 years old, (c) having engaged in a sexual relationship in the last year, (d) currently residing in the United States, (e) identifying as a sexual minority (gay, bisexual, queer, questioning, pansexual, or other sexual minority identities). Given its purpose, heterosexual men were excluded from the original study; However, we decided to retain them here as they were relevant to the current study. The criterion of having a part of a relationship in the previous year was added as one of this study’s purposes involved men in a relationship. Although this criterion was not formally included in the study on heterosexual men, being in a sexual relationship in the previous year seems like a relatively lenient criterion and was probably met by most of our sample. Participants who completed the survey were compensated USD2.54, equating to an hourly rate of USD11.35. The Boston College Institutional Review Board (approval number 21.210.01e) approved all procedures.

Participants

From this point on, we discuss the aggregated data obtained in both studies (N = 882). When asked about their gender, 99.2% reported being men; four participants were gender non-conforming, and they were removed from the analysis. Three other participants reported their gender as “other” and specified being transgender men. Note that we did not ask directly about participants’ sex so it is possible that some of those identifying as men were also transgender. The participants ranged in age from 18 to 79 years old (M = 31.24, SD = 10.79). The participants were mostly heterosexual (62.53%; 18.22% gay; 16.51% bisexual; the rest self-identified as “other” and specified being queer, pansexual, or questioning, with no other sexual orientations mentioned). Most of the participants were White or European American (71.75%; 15.60% Asian or Asian-American; 5.69% Black or African American; 10.82% Latino/Hispanic of any race), never married (68.45%; 27.79% married; 2.28% divorced), with education ranging from no formal schooling to a doctoral degree, with the modal education representing completion of a bachelor’s degree. Annual income ranged from below USD9,525 to >USD200,000, with modal income between USD38,701 and USD60,000.

Measures

The participants filled out surveys including multiple instruments, as reported elsewhere (Harris & Mahalik, under review; Mahalik et al., 2022b). In the current work, we report the results of the demographic questionnaire (see above) and The CMNI-30 (Levant et al., 2020). The instruments were presented in random order.

The CMNI-30 is a 30-item short form of Mahalik et al.’s (2003) 94-item questionnaire that assesses conformity to 10 dominant cultural norms of masculinity in the United States (e.g., Risk-taking, Self-reliance, Emotional control). Each of the subscales has three items; all of the items employ a 6-point Likert-type response format ranging from 1 (strongly disagree) to 6 (strongly agree), with nine reverse-coded items. The subscales scores were calculated using a simple sum of the items’ scores. In this sample, the overall Cronbach’s alpha reliability was .83, and the subscale alphas ranged between .72 and .93.

Analyses

All of the data were analyzed using SPSS 27 and R (R core team, 2017), with the invariance analyses conducted using the lavaan R package (Rosseel, 2012). We conducted two measurement invariance analyses with respect to sexual orientation. First, we compared heterosexual men (n = 553) and all other sexual orientations (excluding two participants who reported they are “questioning”¹; n = 323). Second, to examine more specific groups, we compared the three major groups: heterosexual, gay (n = 160), and bisexual+ men (n = 163). Under the bisexual+ group, we included all participants who self-identified as being attracted to more than one gender (e.g., bisexual, pansexual, queer²), following Davila et al. (2018).

Comparing the three groups, we found differences among the groups in terms of age (F(2, 873) = 15.04, p < .001); a Tukey’s honestly significant difference (HSD) test revealed that the bisexual+ men in the sample were significantly younger (M = 27.37, SD = 7.99) than the gay (M = 33.51, SD = 12.20; p < .001) and heterosexual men (M = 31.74, SD = 10.82). Table 1 describes other demographic characteristics of the three groups. Using a χ² test and after correcting for multiple comparisons using the Bonferroni correction, we found some differences among the groups in race, marital status, education level, and income. With respect to race (α = .013), we found that heterosexual men were less likely to be Asian, χ²(2) = 11.66, p = .003, compared with gay and bisexual+ men, and that bisexual+ men were more likely to be Latino/Hispanic, χ²(2) = 12.40, p = .002. Given Levant et al.’s (2020) findings that the CMNI-30 is invariant with respect to race, we assumed that these differences should not affect our results.

Table 1.

Demographic Characteristics of Heterosexual, Gay, and Bisexual+ Men in the Sample.

Demographic variable	Categories	Heterosexual(n = 553)	Gay(n = 160)	Bisexual+ (n = 167)	χ²(2)	p-value
Race	% White	68.90	78.75	74.23	6.57	.037
	% Asian or Asian American	18.81	11.25	9.20	11.66	.003
	% Black or African American	5.60	5.00	6.75	0.49	.784
	% Latino/Hispanic of any race	9.58	7.50	18.40	12.40	.002
Relationship status	% Never married	63.65	76.25	76.69	15.50	< .001
	% Married	33.63	17.50	18.40	24.97	< .001
	% Divorced^a	1.81	3.75	2.45	2.12	.346
Education level	% High school diploma	13.92	8.12	14.72	4.20	.122
	% Bachelor’s degree	35.44	39.38	26.38	6.66	.036
	% Graduate degree	19.89	21.25	9.82	9.73	.008
Annual income	% USD24,000 or below	14.65	13.75	24.54	9.97	.007
	% USD24,001–USD60,000	30.74	36.25	38.65	4.36	.113
	% USD60,001–USD120,000	34.72	30.62	29.45	2.08	.354
	USD120,001 or above	19.89	19.38	6.75	15.72	< .001

Note. For brevity, only response categories with important content differences are presented in this table.

χ² values are approximated due to the small number of observations in each cell.

Regarding relationship status (α = .017), heterosexual men were more likely to be married (33.63%) than gay and bisexual+ men, 17.50% and 18.40%, respectively; χ²(2) = 24.97, p < .001, and less likely to never have been married, 63.65% vs. 76.25% and 76.69% for gay and bisexual+ men, respectively; χ²(2) = 15.50, p < .001. The differences in marital status are understandable given how gay marriage has only recently become legal in the United States. It is likely that the rates of sexual minority men in long-term relationships are higher than the rates of those who are married.

We also found that bisexual+ men were less likely to have graduate degrees, α = .017; χ²(2) = 9.73, p = .008. In addition, looking at income levels (α = .013), bisexual+ men were more likely to have a lower income than both heterosexual and gay men, χ²(2) = 9.97, p = .007 for income of USD24,000 or below, χ²(2) = 15.72, p < .001 for income of USD120,001 or above. We speculate that this is due to the bisexual+ group being significantly younger than the other groups, so they are less likely to have completed their education or to have reached their earning potential.

In both of the invariance analyses, we applied a CFA approach. To conduct a CFA analysis, the data must meet the assumption of multivariate normality (Bryant et al., 1999). However, our data did not meet the condition of multivariate normality (Mardia’s skewness = 12,082.02, p < .001, Mardia’s kurtosis = 71.71, p < .001). Therefore, all analyses were conducted using a robust maximum likelihood estimator. This estimation procedure corrects the standard errors and fit statistics such that they are less sensitive to violations of the normality assumption (see more details in Li, 2015).

The first stage under the CFA framework involves fitting a model for each of the groups separately to make sure that the structure of the scale is similar across groups and to identify potential differences in their best-fitting model (Bowen & Masa, 2015; Byrne et al., 1989). To determine which model to use, we tested several models on the general sample, all of which were also tested in the CMNI-30’s development process: a unidimensional model (all items loaded on a single factor), a 10-factor model with each item loading on its respective scale and the factors are independent, a second-order model where the items load on their respective scales and the scales load on a common higher-order factor of “conformity to masculinity norms,” and a bifactor model where the items load on their respective factors and also on a general “conformity to masculinity norms” factor that is orthogonal to the subscale factors. Once a well-fitting common model has been identified, a set of nested models (i.e., with stricter constraints as described above) was used to establish measurement invariance (van de Schoot et al., 2012). If any model does not fit the data that means the instrument does not meet that level of invariance.

To test for the first level of invariance, configural invariance, we conducted a CFA where all groups were set to have the same factor structure. If that model fits the data well, we moved on to examining metric invariance. In the metric invariance model, the factor loadings were set to be equal across groups. For the next step, testing for scalar invariance, we constrained the items’ intercepts to be equal across groups, in addition to the factor loadings. Finally, to test for residual invariance, we constrained the residual variances of the items as well as the factor loadings and intercepts to be equal across groups.

We tested these models’ fit by using several fit statistics (Hu & Bentler, 1999; Weston & Gore, 2006). These fit statistics generally represent the extent to which a model improves on a generic, baseline model (with no constraints) or how well data reproduced from a model are similar to the observed data; these interpretations of the fit statistics are similar regardless of the number of the groups in the model. The fit statistics we used were: a χ² test comparing each model with a baseline model, root mean square error of approximation (RMSEA; ≤ .08 indicating an acceptable fit, ≤ .06 indicating a strong fit), comparative fit index (CFI), Tucker–Lewis index (TLI; for both CFI and TLI, ≥ .90 indicating an acceptable fit, ≥ .95 indicating a strong fit), and standardized root mean squared residual (SRMR; ≤ .08 indicating an acceptable fit). Note that, while the results of χ² tests are commonly reported (D. L. Jackson et al., 2009) as they are relatively easy to interpret, they are very sensitive to even trivial model-data misfit in large samples (over 200 participants; Alavi et al., 2020), and so other fit statistics should be preferred when determining a model’s fit (Cheung & Rensvold, 2002). In addition to these measures of fit, we used ΔCFI ≥ -.01, and ΔRMSEA ≤ .015 as criteria for good relative model fit when comparing nested models (Chen, 2007; Cheung & Rensvold, 2002). Inconsistencies among the fit statistics should generally be examined thoroughly, and they do not necessarily mean that there is a problem with the model (Lai & Green, 2016). Nevertheless, for brevity, we treated the fit statistics as sources of evidence of a model’s fit and in cases of minor disagreements made decisions based on the majority of fit statistics.

Once the instrument’s measurement invariance has been established, we used a series of ANOVAs to compare the observed subscale scores of heterosexual, gay, and bisexual+ men. We preferred this approach over using latent scores to reflect the CMNI’s use in practice. Although subscale scores were originally calculated by summing up each subscale’s item scores, different researchers and practitioners also use the items’ means (e.g., Krivoshchekov et al., 2022), and no consistent guidelines or norms for the CMNI exist. Following Levant et al. (2020), we chose to report the sums of the items instead of the means. Given the correlations among the subscales, we used a Bonferroni correction to account for the multiple comparisons when conducting the ANOVA tests (α = .005). Where the results were significant, we applied a Tukey HSD test to find the source of the differences among the groups.

Results

Before testing the CMNI-30’s measurement invariance, we tested three models and chose the best-fitting one in the following steps. We found that a unidimensional model yielded a poor fit to the data, χ² (405) = 7,566.30, p < .001; RMSEA = .160 (90% CI = [.157, .163]); CFI = .33; TLI =.29; SRMR = .145. The 10-factor model yielded a good fit to the data: χ² (360) = 968.82 (p < .001); RMSEA = .050 (90% CI = [.046, .054]); CFI = .94; TLI =.93; SRMR = .050. The second-order model and the bifactor model did not converge. There are many potential reasons why the models did not converge (e.g., poor model-data fit, small sample size). Both models did not converge or had a very poor fit to the data in previous works, as well (Krivoshchekov et al., 2022; Levant et al., 2020). Although it is impossible to rule out these models as the best-fitting ones, their consistent issues even in relatively large-scale studies suggest that they are not useful in practice. We decided to continue with the 10-factor model. The standardized loadings of the model in the full sample are presented in Table S1.

Heterosexual Versus Sexual Minority Men

First, we tested whether the measurement model fit each group separately. For heterosexual men, the ten-factor model had an acceptable fit according to most criteria: χ² (360) = 838.50, p < .001; RMSEA = .056 (90% CI = [.051, .061]); CFI = .93; TLI =.91; SRMR = .066. The model for sexual minority men also fit the data well: χ² (360) = 525.98, p < .001; RMSEA = .043 (90% CI = [.035, .051]); CFI = .95; TLI =.94; SRMR = .048. Therefore, we moved on to testing the instrument’s measurement invariance.

Table 2 presents the relative fit statistics of each of the invariance models. First, we tested whether the factor structure was similar across groups, that is, configural invariance. Based on all fit measures except for χ², the model fit was acceptable. All items significantly loaded on their respective factors. This indicates that the factor structure was similar in both groups. Next, we tested the model’s metric invariance, namely, whether the item loadings are consistent among heterosexual and sexual minority men. We again found an acceptable fit using all fit statistics except for χ². Based on both ΔCFI and ΔRMSEA, the model was similar to the configural model in terms of its fit, supporting metric invariance.

Table 2.

Measurement Invariance of the CMNI (Heterosexual Versus Others).

Invariance level	χ²(df)	p-value	RMSEA (90% CI)	CFI	TLI	SRMR	Δχ²(df)	p-value	ΔRMSEA	ΔCFI
Configural	1,370.17 (720)	<.001	.052 [.048, .056]	.94	.92	.058	—	—	—	—
Metric	1,452.79 (740)	<.001	.054 [.050, .058]	.93	.92	.063	77.57 (20)	<.001	.002	−.006
Scalar	1,533.93 (760)	<.001	.055 [.051, .059]	.92	.91	.063	119.77 (20)	<.001	.001	−.005
Residual	1,686.00 (790)	<.001	.058 [.054, .062]	.91	.90	.064	185.30 (30)	<.001	.003	−.011

Note. RMSEA = root mean square error of approximation; CI = confidence interval; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean squared residual.

We then tested for scalar invariance (i.e., equality of intercept in addition to factor structure and factor loading). We again found an acceptable fit using all fit statistics except for χ² and concluded that the intercepts were equal across the groups. We found only small differences in fit between the scalar and the metric model. Finally, we explored the model’s residual invariance (i.e., whether the residuals were equal across groups). There was an acceptable fit using all measures except for χ². In spite of the good absolute fit and the good relative fit using ΔRMSEA, we found that the change in ΔCFI was slightly above our cutoff, suggesting that the scalar model fit better than the residual one. Since the other fit measures were acceptable, we concluded that the CMNI-30 is strictly invariant when comparing heterosexual and sexual minority men, meaning that the scale’s scores’ psychometric properties are consistent between those groups. However, given the discrepancies among the fit statistics, this result should be treated with caution.

Heterosexual Versus Gay Versus Bisexual+

We again started with testing the model’s fit for each group separately. We reported the fit for heterosexual men above. For gay men, the ten-factor model had an acceptable fit according to most criteria: χ² (360) = 495.11, p < .001; RMSEA = .055 (90% CI = [.043, .067]); CFI = .92; TLI =.91; SRMR = .065. The model for bisexual+ men also fit the data well: χ² (360) = 482.19, p < .001; RMSEA = .052 (90% CI = [.039, .064]); CFI = .93; TLI =.92; SRMR = .063. Therefore, we moved on to testing the instrument’s measurement invariance.

Table 3 presents the relative fit statistics of each of the invariance models. For the configural model, all fit measures except for χ² indicated that the model fit was acceptable. All items significantly loaded on their respective factors in all groups. For the metric invariance model, we also found a generally acceptable fit. Based on both ΔCFI and ΔRMSEA, the model was similar to the configural model in terms of its fit, supporting the instrument’s metric invariance. So, we concluded that the item loadings in both groups were similar. The scalar invariance model also had an acceptable fit, and so did the residual invariance model. All of the relative fit measures also indicated good fit relative to the less restrictive model, except for a large ΔCFI of the residual invariance model relative to the scalar invariance model, suggesting it did not fit the data as well as the scalar invariance model. However, as the other fit measures were acceptable, we determined that the CMNI-30 has a residual invariance with respect to sexual orientation. Again, given the large ΔCFI, this result should be interpreted carefully.

Table 3.

Measurement Invariance of the CMNI (Heterosexual vs. Gay vs. Bisexual+).

Invariance level	χ²(df)	p-value	RMSEA (90% CI)	CFI	TLI	SRMR	Δχ²(df)	p-value	ΔRMSEA	ΔCFI
Configural	1,826.76 (1,080)	<.001	.055 [.051, .060]	.93	.91	.063	—	—	—	—
Metric	1,932.75 (1,120)	<.001	.057 [.053, .061]	.92	.91	.068	101.11 (40)	<.001	.002	−.006
Scalar	2,059.97 (1,160)	<.001	.058 [.054, .062]	.91	.90	.069	214.63 (40)	<.001	.001	−.007
Residual	2,301.83 (1,220)	<.001	.062 [.058, .066]	.90	.89	.071	330.40 (60)	<.001	.004	−.015

Note. RMSEA = root mean square error of approximation; CI = confidence interval; CFI = comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean squared residual.

Comparing the Groups on Conformity to Masculine Norms

After finding that the instrument is invariant, that is, it is possible to make meaningful comparisons across sexual orientation groups, we now turn to interpret these score differences (see Table 4). After applying the Bonferonni correction, we found significant differences among the groups in Winning (p < .001, η² = 0.03), Violence (p < .001, η² = 0.04), Heterosexual self-presentation (p < .001, η² = 0.19), Pursuit of status (p < .001, η² = 0.02), Primacy of work (p < .001, η² = 0.02), and Power over women (p < .001, η² = 0.09).

Table 4.

Participants’ Subscales Scores (SD) by Sexual Orientation.

CMNI subscales	Heterosexuals	Gay	Bisexual+	F(df1, df2)	p-value
Emotional control	10.49 (3.83)	9.38 (3.79)	10.18 (3.98)	5.13 (2, 870)	.006
Winning	9.63 (3.42)	8.41 (3.16)	8.37 (3.32)	13.72 (2, 862)	<.001*
Playboy	7.86 (3.43)	8.37 (4.11)	8.61 (3.95)	3.25 (2, 871)	.039
Violence	9.19 (3.24)	7.33 (3.19)	9.07 (3.52)	20.19 (2, 868)	<.001*
Heterosexual self-presentation	8.33 (4.39)	4.20 (1.90)	5.05 (2.43)	101.80 (2, 869)	<.001*
Pursuit of status	10.59 (3.12)	11.28 (3.24)	9.96 (3.19)	7.02 (2, 869)	<.001*
Primacy of work	9.98 (3.60)	9.37 (3.64)	8.65 (3.60)	9.00 (2, 866)	<.001*
Power over women	7.49 (3.77)	4.87 (2.66)	5.62 (3.31)	42.95 (2, 868)	<.001*
Self-reliance	9.56 (3.06)	10.06 (3.46)	10.39 (3.16)	5.03 (2, 870)	.007
Risk taking	10.21 (3.42)	9.58 (3.57)	10.20 (3.28)	2.18 (2, 871)	.113

p < .005 (significant after applying the Bonferroni correction). The possible score range of each subscale is 3 to 18.

We then used a Tukey HSD test to identify specific differences among the groups. We found that heterosexual men scored higher than both gay and bisexual+ men in the Winning (both p < .001), Heterosexual self-presentation (both p < .001), and Power over women subscales (both p < .001), with no differences between gay and bisexual men in these subscales. Heterosexual men also scored higher than bisexual+ men in the Primacy of work subscale (p < .001) but not higher than gay men (p = .148; there were no significant differences between gay and bisexual+ men;p = .175). Gay men had higher scores in Pursuit of status relative to both heterosexual (p = .044) and bisexual+ men (p < .001); the difference between heterosexual and bisexual+ men was not significant (p = .062). Finally, gay men had lower scores in violence in comparison with both heterosexual and bisexual+ men (both p < .001), but there were no significant differences between these latter groups (p = .919).

Discussion

In the current study, we aimed to test the measurement invariance of Levant et al.’s (2020) CMNI-30 across groups of men with different sexual orientations. We also sought to compare these groups’ levels on the different masculinity norms as defined in the CMNI. The findings support the CMNI-30’s 10-factor structure reported by Levant et al. (2020) and the construct validity of the scale through evidence of full residual invariance between heterosexual, gay, and bisexual+ men. We also found significant differences across these groups in some subscales, and they were especially notable in Heterosexual self-presentation and Power over women.

Our initial CFA supports Levant et al.’s (2020) conclusion that the CMNI-30 should be used as a measure of specific masculine norms rather than as a measure of overall conformity to traditional masculine norms. This means that the CMNI-30 should not be used to assess overall conformity to masculinity norms but rather as a measure of conformity to specific norms, strengthening Mahalik et al.’s (2003) conceptualization of conformity to masculinity norms as a multidimensional construct. In addition, this finding has been consistent across other versions of the scale. For example, Hammer et al. (2018) also did not find support for the general factor using the CMNI-46 and recommended using only the subscale scores rather than an overall conformity to masculinity norms score.

In addition, the analysis of invariance provides critical information about the construct validity of the CMNI-30. We found that the CMNI-30 has residual invariance with respect to sexual orientation. That is, not only is the instrument suitable for comparison across heterosexual, gay, and bisexual+ men (providing evidence that the instrument measures the construct as expected), but the measure has similar reliability across groups.

These findings are different from Krivoshchekov et al.’s (2022) findings, which suggested that the CMNI-30 has only partial scalar invariance, namely, one of the items (“I bring up my feelings when talking to others”) did not have equal intercepts across the groups. This means that without applying a complex model allowing the item’s parameters to be estimated freely (or removing the item), it is harder to justify using the Russian version of the CMNI-30 for comparisons between heterosexual and sexual minority men. However, as only one of Krivoshchekov et al.’s items was problematic, this finding may be due to chance.

As mentioned, our findings suggest that comparisons among sexual orientation groups are meaningful. That the CMNI-30 was useful for all groups highlights the relevance of hegemonic masculinity in heterosexual, gay, and bisexual+ men’s lives. Indeed, while there was a notable variation in the level of conformity across men from dominant (i.e., heterosexual) and marginalized (i.e., gay, bisexual+) groups, the 10-factors solution was supported across sexual orientation. This may mean that even though they may emphasize different aspects of masculinity, sexual minority men generally view masculinity similarly to heterosexual men, showing this construct’s dominance in our culture.

We also believe that the differences we found among sexual orientation groups are noteworthy. We found that heterosexual men were more likely to endorse more masculinity norms than the other groups. Specifically, heterosexual men were more likely to endorse Winning, Heterosexual self-presentation, and Power over women. That heterosexual men have higher scores on the Heterosexual self-presentation scale seem to be self-explanatory; it is unlikely that gay or bisexual+ men, at least those whose status is known, would be upset if others thought viewed them as sexual minorities. Sexual minority men’s lower conformity to Power over women may indicate outgroup solidarity between women and sexual minority men, as they are both members of historically victimized groups (Ball & Branscombe, 2019). Another potential explanation is that as sexual minority men are less likely to be romantically involved with women, they are not as concerned with exerting power over them.

That heterosexual men scored significantly higher on the Winning subscale was notable. We speculate that being more likely to endorse items such as “I will do anything to win” may reflect heterosexual men’s privileged position in a heteronormative society that allows them to focus on competition rather than just survival. However, gay men were more likely to have higher Pursuit of status scores than heterosexual and bisexual+ men. Although this may be surprising given how both Winning and Pursuit of status are related to achieving a position of power, the finding reflects Pachankis and Hatzenbuehler’s (2013) results on how gay men strive toward academic success, competence, and competition. Our findings also add important context to Pachankis and Hatzenbuehler’s (2013) study on the outcomes of young sexual minority men whose self-worth is contingent on competition and achievement. Given that the average age was higher in this sample (M = 31.24) than in Pachankis and Hatzenbuehler’s (M = 20.56), it could be that gay men in this study view attaining status as a more effective shield against discrimination at a later life stage than besting others in competition. Finally, gay men also tended to have lower Violence scores in comparison with the other two groups. We speculate that their status as a minority caused them to reject violence that historically has been directed at them, though future studies should explore this suggestion.

As specific masculinity norms have different relationships with physical and mental health outcomes, these findings have some implications for professionals treating men of different sexual orientations. For example, Wong et al. (2017) found that several masculinity norms including Winning, Heterosexual self-presentation, and Violence are associated with negative mental health and lower levels of psychological help-seeking. In addition, greater conformity to masculinity norms is associated with negative health behaviors (Mahalik et al., 2007, Mahalik et al. 2022b), including negative attitudes toward mask-wearing during the pandemic (Mahalik et al., 2007, Mahalik et al. 2022b), especially the Winning subscale (Levant & Wimer, 2013). Indeed, heterosexual men are less likely than other men to engage in some health-promoting behaviors (Boehmer et al., 2012; Caceres et al., 2018; Horowitz et al., 2001). These findings, combined with ours, suggest that heterosexual men conform to masculine norms that have been associated with higher health risks, as well as less likelihood of seeking professional help.

Most of the existing literature suggests that sexual minority men are at a higher risk of various health issues (C. L. Jackson et al., 2016), perhaps attributable to the unique stresses they face (Herek & Garnets, 2007). Although the scholarly literature has not explored the link between masculinity and health outcomes among sexual minority men as extensively as it did among heterosexual men, previous work has identified the areas of body image, health risk behaviors, and internalized stigma as being related to sexual minority men’s conformity to masculine norms (Hamilton & Mahalik, 2009; Kimmel & Mahalik, 2005). Given the deleterious effects of heterosexist discrimination on sexual minority men’s well-being (Szymanski & Ikizler, 2013), future work should consider the role that conformity to masculine norms plays in the manifestation of minority stress.

Regarding limitations, though our national sample was heterogeneous across age, race, and employment, caution should be exercised in applying this validity evidence to all groups of men, as our sample is not random and hence not representative. In addition, we did not examine the intersection of race, age, sexual orientation, class, immigration status, and other variables that influence how men experience masculine norms. Future research on the properties of the CMNI-30 should explore a nationally representative sample as well as seek to examine how men’s intersectional identities and culturally bound meanings of masculinity affect the performance of the inventory. In addition, considering the differences between our findings and Krivoshchekov et al.’s (2022), future studies may find other results across different countries and cultures.

Many of our gay and bisexual+ samples were drawn from a study on sexual minority men in relationships, a condition not posed upon the heterosexual men sample. We assumed this condition’s effect on the instrument is negligible as a large proportion (34%) of the heterosexual sample was married, and a majority of Americans (69%) are partnered (Pew Research Center, 2020). We also speculate that the sexual minority men included in this study likely have higher levels of outness than the average sexual minority man in the United States, given that most of the sexual minority men in this study were currently or recently in a relationship. As we did not measure the outness of the sexual minority men included, future work that utilizes the CMNI-30 among sexual minority men should consider measuring outness and its potential relevance to their experiences of traditional masculinity.

Finally, we also consider the possibility of variance in the interpretation of the Heterosexual self-presentation norm, particularly as it applies to gay and bisexual+ men. Although the instrument was invariant between gay and bisexual+ men, the content of the items in the heterosexual self-presentation scale is about being perceived as gay, which may lead to different interpretations among these two groups. For example, a gay man may experience these items as straightforward questions about his feelings about his gay identity while a bisexual+ man may feel these questions do not capture the breadth of his feelings about his sexual orientation (Brewster & Moradi, 2010; Paul et al., 2014). Future qualitative studies could examine sexual minority men’s perceptions of their sexual orientation, especially among bisexual+ men.

In conclusion, this study enhances the confidence in the use of the CMNI-30 to assess conformity to an array of traditional masculine norms. Specifically, our results provide evidence for the construct validity of the scale in confirming the measurement invariance of the CMNI-30 across sexual orientation groups. In addition, we found differences in the CMNI-30 subscales by sexual orientation that can have implications for men’s physical and mental health. Taken together, the CMNI-30 demonstrates significant psychometric strengths for investigating conformity to traditional masculinity norms across men of different sexual orientations.

Supplemental Material

sj-docx-1-asm-10.1177_10731911221149085 – Supplemental material for Examining the Measurement Invariance of the Conformity to Masculine Norms Inventory (CMNI-30) by Sexual Orientation

Supplemental material, sj-docx-1-asm-10.1177_10731911221149085 for Examining the Measurement Invariance of the Conformity to Masculine Norms Inventory (CMNI-30) by Sexual Orientation by Ella Anghel, James R. Mahalik and Michael P. Harris in Assessment

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ella Anghel

Data Availability

All underlying material can be accessed directly from the authors.

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Alavi

Visentin

D. C.

Thapa

D. K.

Hunt

G. E.

Watson

Cleary

M. L.

(2020). Chi-square for model fit in confirmatory factor analysis. Journal of Advanced Nursing, 76, 2209–2211.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing.

American Psychological Association, Boys and Men Guidelines Group. (2018). APA guidelines for psychological practice with boys and men. http://www.apa.org/about/policy/psychological-practice-boys-men-guidelines.pdf

Amnesty International. (2019). Russia: Two years after Chechnya’s gay purge victims still seek justice as LGBTI defender receives death threats. https://www.amnesty.org/en/latest/news/2019/04/russia-two-years-after-chechnyas-gay-purge-victims-still-seek-justice-as-lgbti-defender-receives-death-threats/

Bailey

J. M.

(1996). Gender identity. In Savin-Williams

R. C.

Cohen

K. M.

(Eds.), The lives of lesbians, gays, and bisexuals: Children to adults (pp. 71–93). Harcourt Brace College.

Ball

T. C.

Branscombe

N. R.

(2019). When do groups with a victimized past feel solidarity with other victimized groups? In Mallet

R. K.

Monteith

M. J.

(Eds.), Confronting prejudice and discrimination (pp. 73–92). Academic Press. https://doi.org/10.3389/fpsyg.2019.01879

Boehmer

Miao

Linkletter

Clark

M. A.

(2012). Adult health behaviors over the life course by sexual orientation. American Journal of Public Health, 102(2), 292–300. https://doi.org/10.2105/ajph.2011.300334

Bowen

N. K.

Masa

R. D.

(2015). Conducting measurement invariance tests with ordinal data: A guide for social work researchers. Journal of the Society for Social Work and Research, 6, 229–249. https://doi.org/10.1086/681607

Brannon

Juni

(1984). A scale for measuring attitudes about masculinity. Psychological Documents, 14 (Doc. #2612).

10.

Brewster

M. E.

Moradi

(2010). Perceived experiences of anti-bisexual prejudice: Instrument development and evaluation. Journal of Counseling Psychology, 57(4), 451–468. https://doi.org/10.1037/a0021116

11.

Bryant

F. B.

Yarnold

P. R.

Michelson

E. A.

(1999). Statistical methodology: VIII. Using confirmatory factor analysis (CFA) in emergency medicine research. Academic Emergency Medicine, 6(1), 54–66. https://doi.org/10.1111/j.1553-2712.1999.tb00096.x

12.

Byrne

B. M.

Shavelson

R. J.

Muthén

(1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456–466. https://doi.org/10.1037/0033-2909.105.3.456

13.

Caceres

B. A.

Brody

A. A.

Halkitis

P. N.

Dorsen

Chyun

D. A.

(2018). Sexual orientation differences in modifiable risk factors for cardiovascular disease and cardiovascular disease diagnoses in men. LGBT Health, 5(5), 284–294. https://doi.org/10.1089/lgbt.2017.0220

14.

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834

15.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

16.

Connell

R. W.

Messerschmidt

J. W.

(2005). Hegemonic masculinity: Rethinking the concept. Gender and Society, 19(6), 829–859. https://doi.org/10.1177/0891243205278639

17.

Davila

Jabbour

Dyar

Feinstein

B. A.

(2018). Bi+ visibility: Characteristics of those who attempt to make their bisexual+ identity visible and the strategies they use. Archives of Sexual Behavior, 48(1), 199–211. https://doi.org/10.1007/s10508-018-1284-6

18.

Fischgrund

B. N.

Halkitis

P. N.

Carroll

R. A.

(2012). Conceptions of hypermasculinity and mental health states in gay and bisexual men. Psychology of Men & Masculinity, 13(2), 123–135. https://doi.org/10.1037/a0024836

19.

Gerdes

Z. T.

Levant

R. F.

(2018). Complex relationships among masculine norms and health/well-being outcomes: Correlation patterns of the Conformity to Masculine Norms Inventory subscales. American Journal of Men’s Health, 12(2), 229–240. https://doi.org/10.1177/1557988317745910

20.

Guenole

Brown

(2014). The consequences of ignoring measurement invariance for path coefficients in structural equation models. Frontiers in Psychology, 5, Article 980. https://doi.org/10.3389/fpsyg.2014.00980

21.

Hamilton

C. J.

Mahalik

J. R

. (2009). Minority stress, masculinity, and social norms predicting gay men’s health risk behaviors. Journal of Counseling Psychology, 56, 132–141. https://doi.org/10.1037/a0014440

22.

Hammer

J. H.

Good

G. E.

(2010). Positive psychology: An empirical examination of beneficial aspects of endorsement of masculine norms. Psychology of Men & Masculinity, 11, 303–318. https://doi.org/10.1037/a0019056

23.

Hammer

J. H.

Heath

P. J.

Vogel

D. L.

(2018). Fate of the total score: Dimensionality of the conformity to masculine norms inventory-46 (CMNI-46). Psychology of Men & Masculinity, 19(4), 645–651. https://doi.org/10.1037/men0000147

24.

Harris

M.P.

Mahalik

J.R

. (under review). Sexual stigma, masculinity, and physical intimate partner violence perpetration among sexual minority men.

25.

Hearn

Nordberg

Andersson

Balkmar

Gottzén

Klinth

Pringle

Sandberg

(2012). Hegemonic masculinity and beyond. Men and Masculinities, 15(1), 31–55. https://doi.org/10.1177/1097184x11432113

26.

Herek

G. M.

Garnets

L. D.

(2007). Sexual orientation and mental health. Annual Review of Clinical Psychology, 3(1), 353–375. https://doi.org/10.1146/annurev.clinpsy.3.022806.091510

27.

Horowitz

S. M.

Weis

D. L.

Laflin

M. T.

(2001). Differences between sexual orientation behavior groups and social background, quality of life, and health behaviors. Journal of Sex Research, 38(3), 205–218. https://doi.org/10.1080/00224490109552089

28.

Hsu

Iwamoto

D. K.

(2014). Testing for measurement invariance in the Conformity to Masculine Norms-46 across White and Asian American college men: Development and validity of the CMNI-29. Psychology of Men & Masculinity, 15, 397–406. https://doi.org/10.1037/a0034548

29.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. https://doi.org/10.1080/10705519909540118

30.

Jackson

C. L.

Agénor

Johnson

D. A.

Austin

S. B.

Kawachi

(2016). Sexual orientation identity disparities in health behaviors, outcomes, and services use among men and women in the United States: A cross-sectional study. BMC Public Health, 16(1), Article 807. https://doi.org/10.1186/s12889-016-3467-1

31.

Jackson

D. L.

Gillaspy

J. A.

Jr. Purc-Stephenson

(2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14(1), 6–23.

32.

Kimmel

S. B.

Mahalik

J. R

. (2005). Body image concerns of gay men: The roles of minority stress and conformity to masculine norms. Journal of Consulting and Clinical Psychology, 73, 1185–1190. https://doi.org/10.1037/0022-006X.73.6.1185

33.

Krivoshchekov

Gulevich

Ostroverkhova

(2022). The conformity to masculine norms inventory-30: Validity and measurement invariance of a Russian-language version. Psychology of Men & Masculinities, 23(1), 59–73. https://doi.org/10.1037/men0000379

34.

Lai

Green

S. B

. (2016). The problem with having two watches: Assessment of fit when RMSEA and CFI disagree. Multivariate Behavioral Research, 51, 220–239. https://doi.org/10.1080/00273171.2015.1134306

35.

Levant

R. F.

Hirsch

L. S.

Celentano

Cozza

T. M.

Hill

MacEachern

Marty

Schnedeker

(1992). Male role norms inventory [Database record]. PsycTESTS Dataset. https://doi.org/10.1037/t27377-000

36.

Levant

R. F.

McDermott

Parent

M. C.

Alshabani

Mahalik

J. R.

Hammer

J. H.

(2020). Development and evaluation of a new short form of the Conformity to Masculine Norms Inventory (CMNI-30). Journal of Counseling Psychology, 67(5), 622–636. https://doi.org/10.1037/cou0000414

37.

Levant

R. F.

Wimer

D. J.

(2013). Masculinity constructs as protective buffers and risk factors for men’s health. American Journal of Men’s Health, 8(2), 110–120. https://doi.org/10.1177/1557988313494408

38.

C.-H.

(2015). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. https://doi.org/10.3758/s13428-015-0619-7

39.

Mahalik

J. R.

Burns

S. M.

Syzdek

(2007). Masculinity and perceived normative health behaviors as predictors of men’s health behaviors. Social Science and Medicine, 64, 2201–2209. https://doi.org/10.1016/j.socscimed.2007.02.035

40.

Mahalik

J. R.

Di Bianca

Harris

M. P

. (2022a). Conformity to masculine norms and men’s responses to the COVID-19 pandemic. Psychology of Men & Masculinities, 23(4), 445–449. https://doi.org/10.1037/men0000401

41.

Mahalik

J.R.

Di Bianca

Harris

M.P

. (2022b). Men’s attitudes toward mask-wearing during COVID-19: Understanding the complexities of mask-ulinity. Journal of Health Psychology, 27(5), 1187–1204. https://doi.org/10.1177/1359105321990793

42.

Mahalik

J. R.

Locke

Ludlow

Diemer

Scott

R. P. J.

Gottfried

Freitas

(2003). Development of the Conformity to Masculine Norms Inventory. Psychology of Men and Masculinity, 4, 3–25. https://doi.org/10.1037/1524-9220.4.1.3

43.

Mahalik

J. R.

Talmadge

W. T.

Locke

B. D.

Scott

R. P

. (2005). Using the conformity to masculine norms inventory to work with men in a clinical setting. Journal of Clinical Psychology, 61(6), 661–674. https://doi.org/10.1002/jclp.20101

44.

Meyer

I. H.

(1995). Minority stress and mental health in gay men. Journal of Health and Social Behavior, 36(1), 38–56. https://doi.org/10.2307/2137286

45.

Meyer

I. H.

(2003). Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: Conceptual issues and research evidence. Psychological Bulletin, 129(5), 674–697. https://doi.org/10.1037/0033-2909.129.5.674

46.

Morris

E. W.

Ratajczak

(2019). Critical masculinity studies and research on violence against women: An assessment of past scholarship and future directions. Violence Against Women, 25(16), 1980–2006. https://doi.org/10.1177/1077801219875827

47.

Owen

(2011). Assessing the factor structures of the 55-and 22-item versions of the Conformity to Masculine Norms Inventory. American Journal of Men’s Health, 5(2), 118–128. https://doi.org/10.1177/1557988310363817

48.

Pachankis

J. E.

Hatzenbuehler

M. L.

(2013). The social development of contingent self-worth in sexual minority young men: An empirical investigation of the “Best Little Boy in the World” hypothesis. Basic and Applied Social Psychology, 35(2), 176–190. https://doi.org/10.1080/01973533.2013.764304

49.

Palan

Schitter

(2018). Prolific.ac—A subject pool for online experiments. Journal of Behavioral and Experimental Finance, 17, 22–27. https://doi.org/10.1016/j.jbef.2017.12.004

50.

Parent

M. C.

Moradi

. (2009). Confirmatory factor analysis of the Conformity to Masculine Norms Inventory and development of the Conformity to Masculine Norms Inventory-46. Psychology of Men & Masculinity, 10(3), 175–189. https://doi.org/10.1037/a0015481

51.

Parent

M. C.

Moradi

(2011). An abbreviated tool for assessing conformity to masculine norms: Psychometric properties of the Conformity to Masculine Norms Inventory-46. Psychology of Men & Masculinity, 12(4), 339–353. https://doi.org/10.1037/a0021904

52.

Paul

Smith

N. G.

Mohr

J. J.

Ross

L. E.

(2014). Measuring dimensions of bisexual identity: Initial development of the Bisexual Identity Inventory. Psychology of Sexual Orientation and Gender Diversity, 1(4), 452–460. https://doi.org/10.1037/sgd0000069

53.

Pew Research Center. (2020, October 5). Nearly half of U.S. adults say dating has gotten harder for most people in the last 10 years. https://www.pewresearch.org/social-trends/2021/10/05/rising-share-of-u-s-adults-are-living-without-a-spouse-or-partner/

54.

Pezzote

(2008). Straight acting: Gay men, masculinity, and finding true love. Kensington.

55.

Pleck

J. H.

Sonenstein

F. L.

L. C.

(1993). Masculinity ideology: Its impact on adolescent males’ heterosexual relationships. Journal of Social Issues, 49(3), 11–29. https://doi.org/10.1111/j.1540-4560.1993.tb01166.x

56.

Putnick

D. L.

Bornstein

M. H.

(2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004

57.

R Core Team. (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing.

58.

Reuters. (2020, July 15). Russian parliament begins legalising ban on same-sex marriage. https://www.reuters.com/article/us-russia-politics-gaymarriage-idUSKCN24G1CJ

59.

Rosseel

(2012). Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of Statistical Software, 48, 1–36. https://doi.org/10.18637/jss.v048.i02

60.

Sánchez

F. J.

(2016). Masculinity issues among gay, bisexual, and transgender men. In Wong

Y. J.

Wester

S. R.

Wong

Y. J.

Wester

S. R.

(Eds.), APA handbook of men and masculinities (pp. 339–356). American Psychological Association. https://doi.org/10.1037/14594-016

61.

Sánchez

F. J.

Greenberg

S. T.

Liu

W. M.

Vilain

(2009). Reported effects of masculine ideals on gay men. Psychology of Men and Masculinities, 10, 73–87. https://doi.org/10.1037/a0013513

62.

Skidmore

W. C.

Linsenmeier

J. A. W.

Bailey

J. M.

(2006). Gender nonconformity and psychological distress in lesbians and gay men. Archives of Sexual Behavior, 35, 685–697. https://doi.org/10.1007/s10508-006-9108-5

63.

Smiler

A. P.

(2004). Thirty years after the discovery of gender: Psychological concepts and measures of masculinity. Sex Roles, 50(1/2), 15–26. https://doi.org/10.1023/b:sers.0000011069.02279.4c

64.

Szymanski

D. M.

Ikizler

A. S

. (2013). Internalized heterosexism as a mediator in the relationship between gender role conflict, heterosexist discrimination, and depression among sexual minority men. Psychology of Men & Masculinity, 14(2), 211–219. https://doi.org/10.1037/a0027787

65.

Thompson

E. H.

Bennett

K. M.

(2015). Measurement of masculinity ideologies: A (critical) review. Psychology of Men & Masculinity, 16(2), 115–133. https://doi.org/10.1037/a0038609

66.

Van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486–492. https://doi.org/10.1080/17405629.2012.686740

67.

Weston

Gore

P. A.

(2006). A brief guide to structural equation modeling. The Counseling Psychologist, 34(5), 719–751. https://doi.org/10.1177/0011000006286345

68.

Wong

Y. J.

M. R.

Wang

Miller

S. K.

(2017). Meta-analyses of the relationship between conformity to masculine norms and mental health-related outcomes. Journal of Counseling Psychology, 64, 80–93. https://doi.org/10.1037/cou0000176