Reply to Gangestad’s (2016) Comment on Wood,Kressel,Joshi,and Louie (2014)

Accurate Fertile Windows: 6 Days is the Best Estimate

In both Wood et al.’s (2014) and Gildersleeve et al.’s (2014a) meta-analytic reviews, menstrual cycle shifts were evident primarily in studies that used less precise measures of the fertile phase. That is, women’s preferences did not shift across the cycle in studies that used hormonal assessments to identify time of ovulation. They only shifted across the cycle in studies that relied on women’s self-reports of cycle day, and only in studies that used longer, less accurate fertile phases spanning 9, 13, or 15 days (see Wood & Carden, 2014, Figure 1; Wood et al., 2014, Figures 2a and 2b).

OPEN IN VIEWER

Figure 1.

Wilcox, Dunson, and Baird’s (2000) probability of being in the 6-day fertile phase at each menstrual cycle day.

OPEN IN VIEWER

Figure 2.

Effects of width of fertile phase on women’s preferences for symmetric men, after excluding Gangestad and colleagues’ (Gangestad & Thornhill, 1998; Thornhill & Gangestad, 1999; Thornhill et al., 2003) research (Wood et al., 2014).

Gangestad (2016) speculated that larger fertile phases more accurately captured the 6-day fertile window, given that some women have short cycles and others long cycles, so that a larger window might capture the varying timing of fertility across women (see also Gildersleeve et al., 2014b). Wood et al. (2014) judged this unlikely given the lack of shifts in studies using hormonal assessments—the gold standard for estimating fertility. Also, direct evidence contradicting Gangestad’s speculation that 9 days are needed to capture the fertile phase given variability in the days that women ovulate comes from Wilcox, Dunson, and Baird (2000; see also Wilcox, Dunson, Weinberg, Trussell, & Baird, 2001). Using hormonal tests to identify women’s ovulatory day, Wilcox et al. (2000) estimated for each day of the month the likelihood of women being in the 6-day fertile window. Their findings are represented in Figure 1. Importantly, the curve averages across the naturally occurring variation in cycle length and date of ovulation. Because each cycle includes many days in which women are not fertile and only a few fertile days, increasing the length of the fertile window increases the number of women incorrectly classified as fertile. For example, extending the fertile phase to include Day 7 designates additional women as fertile—fully 82% of these would be incorrectly classified because they are not actually fertile, and only 18% would be correct. Thus, Wilcox et al.’s (2000) results demonstrate that, because fertility rises and falls steeply across the cycle, smaller fertile windows centered around the day of ovulation yield the most accurate measures of the fertile window. These estimates reflect the most reliable values after taking into account variability in the day of ovulation.

Wood et al. (2014) puzzled over why the preference shifts emerged only in studies with less precise measures of the fertile phase, noting that, “we can only speculate why studies with larger fertile windows produced larger preference shifts” (p. 242). This larger window/stronger effect pattern emerged from correlational analyses that were conducted at the meta-analytic level, and thus are open to multiple explanations. However, the pattern was not due to any particular research methods, given that it emerged across multiple paradigms involving preferences for masculinity as well as symmetry. Ultimately, Wood et al. postulated confirmatory hypothesis-testing procedures in which the original study authors began with a standard 6-day fertile window and successively widened this in exploratory analyses until they identified an effect—a strategy that would have capitalized on chance. Adding further plausibility to this artifactual account of mate preference findings, Gildersleeve et al.’s (2014a) review that claimed to find evidence of menstrual cycle shifts used a broader fertile window (8.61 days) in calculating effect sizes than Wood et al.’s review (6.85 days) that concluded cycle effects are unreliable.

Accuracy in Representing Gangestad and Colleague’s Research

Gangestad (2016) argued that the fertility calculations in his research were misrepresented by Wood et al. (2014) as varying across study, given that he used a standard 9-day fertile window as well as a continuous measure of conception risk associated with each cycle day. However, Gangestad and colleagues’ research actually varied widely in reporting format. Of the five studies Wood et al. located by this research group, two provided sufficient information for Wood et al. to calculate an effect size comparing the fertile versus nonfertile phase (Gangestad & Thornhill, 1998; Thornhill et al., 2003), one provided sufficient data to calculate an effect only for continuous estimates of conception risk (Thornhill & Gangestad, 1999), and two did not provide data sufficient to estimate an effect size (Gangestad et al., 2007; Gangestad et al., 2004). It might also seem surprising that, despite Gangestad’s endorsement of the 9-day fertile phase, two of these studies did not even report results comparing a fertile and nonfertile phase (Gangestad et al., 2007; Gangestad et al., 2004). Additionally suggesting flexibility in reporting, all five studies excluded substantial numbers of participants based on shifting sets of criteria, with exclusion rates ranging from 14% in Gangestad et al. (2004) to 46% in Gangestad and Thornhill (1998).

We do not know why Gangestad and colleagues varied their decision rules in these five studies with respect to sample selection, analytic strategy, and reporting procedures. However, it seems certain that the standard definition of p-hacking is not relevant. Gangestad and colleagues’ effect sizes are extremely large outliers in this literature and clearly were not tailored to a .05 reporting criterion. Instead, the results they reported were of sufficient magnitude to skew the overall findings of both reviews to show significant shifts in mate preferences across the cycle. Contrary to Gangestad’s (2016) claims, however, his research using a particular paradigm is not responsible for the finding that only studies with large fertile phases found that fertile women preferred symmetric men more than nonfertile ones. That is, this effect for length of the fertile phase holds whether or not Gangestad and colleagues’ research is included in the regression models (compare Figure 2 with Wood et al.’s [2014] Figure 2b).

In general, Gangestad (2016) attempted to justify variability in the length of the fertile window in this research literature, but he failed to address other forms of flexibility. Although citing Harris, Pashler, and Mickes (2014), Gangestad failed to acknowledge one of their main points: Variations in the literature include not just the number of days counted as fertile or infertile, but also the placement of these windows, the selection of moderators, the exclusion criteria of female participants, and the reported analyses (e.g., see Figure 1 of Harris, Chabot, & Mickes, 2013). Along with other researchers, Gangestad has studied the same variable (e.g., symmetry) using different criteria for categorization and analysis across different articles (as noted before). This flexibility in reporting format potentially enabled a variety of patterns to be interpreted as supporting evolutionary psychology predictions about mate preferences.

Decline Effect Over Time

Research findings that capitalize on chance tend to decline over time as subsequent research fails to document the initial pattern (Ioannidis, 2005; Schooler, 2011). In both Wood et al.’s (2014) and Gildersleeve et al.’s (2014a) data, cycle shifts approximated zero in more recent studies. Gangestad (2016) argued that this decline effect in studies of preference for symmetry was due to changes in the research paradigms used over time. However, this account is not plausible given that the decline effect also emerged in studies of masculinity in Wood et al.’s review and was found broadly across all studies in Gildersleeve et al.’s (2014a) review.

A more plausible account for the decline effect is that recent studies have improved methods for assessing fertility. Studies published more recently were in fact found to use more rigorous methods and to more precisely estimate the fertile phase (see Wood & Carden, 2014). Alternatively, Schooler (2011) tied decline effects to publication bias and regression toward the mean, speculating that “early published studies benefit from being at one statistical end of a larger body of (unpublished) findings” (p. 437). Whether the increasing precision in estimating the fertile phase or the increasing availability of (formerly) unpublished results is responsible, recent studies appear to be less susceptible to false-positive conclusions about the effects of cycle phase on women’s judgments.

Publication Bias

Publication bias proved to be a strong artifact influencing the size of effect in this literature. Only published studies revealed shifts in women’s preferences across the cycle in Wood et al.’s (2014) review of symmetry, masculinity, and dominance, and this same publication effect was evident in Gildersleeve et al.’s (2014a) review. Specifically, unpublished studies in both reviews revealed null effects.

Gildersleeve et al. (2014a) failed to report the null effects in their unpublished literature, but instead concluded no evidence for publication bias based on a funnel plot of the effect sizes and associated tests. Yet these tests provide only an indirect assessment of publication bias, by evaluating whether less precise, smaller studies report different results than more precise, larger studies. Because a variety of factors could produce or mask differences between small-sample and large-sample findings, Lau, Ioannidis, Terrin, Schmid, and Olkin (2006) concluded that, “asymmetry of the funnel plot, either visually interpreted or statistically tested, does not accurately predict publication bias” (p. 599). Especially in literatures with considerable heterogeneity, as is the case with studies on mate preferences, publication bias statistics are likely to be misleading (Ioannidis & Trikalinos, 2007). Additionally, under a variety of conditions, these tests have been shown to be insensitive to even large amounts of bias (Kicinski, 2014).

As Gangestad (2016) noted, Gildersleeve et al. (2014b) concluded that menstrual cycle shifts had evidential basis, given the distribution of statistically significant p values in this literature. However, their calculations were based on small numbers of studies (e.g., k = 14), and the results would likely have revealed little evidential value without the very large effects reported by Gangestad and colleagues (e.g., Gangestad et al., 2007; Gangestad et al., 2004; Gangestad & Thornhill, 1998; Thornhill & Gangestad, 1999; Thornhill et al., 2003).

Given the inaccuracy of existing tests for publication bias, Ioannidis and Trikalinos (2007) recommended that, “whenever both unpublished and published information is available, the results of these 2 types of evidence should be compared” (p. 1095). The results of this comparison in Gildersleeve et al.’s (2014a) and Wood et al.’s (2014) reviews were clear: Published studies overestimated the effects of menstrual cycles on mate preferences.

Future Directions

The controversy over whether menstrual cycles affect women’s preferences has focused the field’s attention on this issue, potentially increasing interest in reporting and publishing null findings, and in this way reducing the influence of publication bias. In fact, several studies that failed to find any evidence of menstrual cycle effects on women’s social preferences were recently accepted for publication in top journals (Hawkins, Fitzgerald, & Nosek, 2015; Scott & Pound, in press), including a particularly impressive investigation of women’s preferences for masculinity in 12 societies differing in economic development (Scott et al., 2014). These publications are likely beneficial consequences of the controversy over menstrual cycle effects, and they suggest a basis for optimism about the self-correcting nature of science.

The controversy over whether women’s preferences shift across the cycle also may have the benefit of spurring new evolutionary models of women’s reproductive activities. One such effort highlights women’s evolved capacities to regulate both their mate preferences and their menstrual cycles according to the demands of societal roles (Eagly & Wood, 2013; Wood et al., 2014). Thus, women’s preferences for a mate vary with gender roles in their society, as these roles influence the costs and benefits women attach to male attributes. Furthermore, the frequency and patterning of women’s menstrual cycles vary with women’s productive roles in a society, as these roles influence women’s capacity for extended lactation and frequent childbirth— thereby affecting the frequency of women’s cycles. More powerful evolutionary models are being developed that recognize the advent of cultural roles and complex group living in human’s evolutionary history, along with women’s development of the capacity to tailor their reproductive activities to a range of social roles.