Abstract
Social science is commonly used in debates about controversial issues, especially for those concerning human sexuality. However, caution must be exercised in interpreting such social science literature, because of a variety of methodological and theoretical weaknesses that are not uncommon. Families are complex structurally and over time; such data are not easily analyzed. Merely determining the number of, for example, sexual minority families has been a difficult task. While some new theories are popular with social scientists, for example, sexual minority theory, they are often used to the exclusion of other, equally valid theories and often are not well tested empirically. Some types of families remain relatively unexamined. Social scientists can be biased by their own values, which are reflected in weak use of theory and in a variety of methodological problems. Eight studies are presented as examples of probable confirmation bias, in which methods and theory were modified in unusual ways that may have affected the outcomes and conclusions. Suggestions for improving social science include greater attention to effect sizes rather than statistical significance per se, deliberately minimizing the politicization of science, developing a culture of humility with respect to social science, deliberately reducing common biases, and maintaining a deeper curiosity about social science than is often seen. Scientists must be open to seeing their best “sacred cow” ideas or theories disproven or modified with increases in research on such issues.
Summary
In controversial areas of social science, there can be numerous threats to the validity of science. Here, some of the more common risks for social science research and theory are examined, with several specific illustrations of how bias appears to have crept into social science, often as confirmation bias. Recommendations are made for reducing bias in future research.
Keywords
Introduction
Statistics can be very useful for working through the “fog of data” and detecting underlying patterns that can help us understand sociological conditions, trends, and applications. However, improper use of statistics and research methods may at best obscure facts and, at worst, distort facts. The risk of the latter use of statistics and methods may be increased in areas of greater controversy, when personal values and self-justification are at stake. Controversy and concurrent biases may also limit our use of scientific theories. The focus of this paper will be on ways in which controversy in the area of human sexuality seems to have led to bias in the use of statistics, research methods, and social science theory, bias that may be present from many background perspectives or theories. Understanding the types of errors that can be made in social science in such controversial areas is critically important to assessing the validity of scientific literature and its application to social policy. A lack of awareness of these biases could easily lead scholars and students to accept as valid, research that is of very low quality, even misleading in its conclusions. Suggestions are also made regarding the development of and testing of alternative theories in the area of human sexuality and several ways to guard against improper use of statistics in controversial areas of social science.
Background: Complexity of Families
One of the first facts to keep in mind is that families are complex units of study. It is tempting to try to boil it down to the “good” family versus “bad” families, but that would be an incorrect over-simplification. Family relationships and structures will change over time and sorting out how those impact child outcomes is very complicated (Jensen and Sanner 2021; Schumm 2014, 2020a). Of course, children change over time as part of those processes. For example, one might have to consider some of the following possibilities of how families might change over time and how those changes might influence their children: Heterosexual couple are chaste until marriage and then have two children born 20 years apart, parents did not divorce. Heterosexual couple cohabit before marriage and then have eight children over a span of 24 years, parents did not divorce. Heterosexual couple cohabit for 5 years before marriage, marry, and then have one child Heterosexual couple cohabit for 6 years, have a child after 3 years, then marry but divorce after 2 years of marriage. Heterosexual couple cohabits after engagement for 6 months, marry, and have three children over a 5 year span of time. Heterosexual couple cohabit for 4 years, have a child, cohabit for two more years, then break up Heterosexual couple cohabit for 2 years, have a child, cohabit for another year, then marry, divorce after another 5 years. A heterosexual woman marries a man, has a child, then divorces, after 5 years starts living with a lesbian social mother who has her own biological child by a gay man. Two lesbian women marry, each has a biological child through artificial insemination, break-up after 2 years, each of them has two or three new partners over the next 10 years, without marrying again. Two sisters are raising a foster child together; a mother and daughter are raising the daughter’s biological child.
Other scholars have observed the same complexity (Rosenfeld, 2017). Regnerus (2020) reported that “for example, a child in a single-parent household at one point may later find themselves in a same-sex coupled household, or a heterosexual stepfamily. Western households are becoming more dynamic, or ‘fluid’, to employ a term now commonly used to describe sexuality” (pp. 51–52). Likewise, Jensen and Sanner (2021) noted increases in sexual and gender minority families, never-married single-parent families, cohabiting families, stepfamilies, and half-sibling relationships (p. 464) in addition to finding a variety of other types of families in research, including grandparent families, foster families, monogamous families, polygamous or polygynous families, extended families, and same-sex parent families (p. 468). Jensen and Sanner (2021) also noted that parent gender in single-parent and stepfamilies is often ignored (p. 468). Recently, to their credit, Kabatek and Perales (2021) were careful to screen out unmarried couples who were probably not gay or lesbian.
Merely trying to pin down the numbers of each of the above types of families in society has proven to be difficult (Regnerus 2020 (pp. 44–46); Smock and Schwartz 2020), with estimates of some types of families differing by up to a factor of 50 or more times each other (Schumm et al. 2017; also, Schumm 2018, p. 74). Jensen and Sanner (2021) found that, in their review of different approaches scholars have taken to measuring family diversity, it was often “not clear which specific family structures have been either studied often or largely overlooked” (p. 465). In addition to the complexities of family structures, the impact of structural effects on outcome variables (divorce rates, effects on children) may be changing over time as well. Smock and Schwartz (2020) argue that family complexity may have reached a plateau in the past decade, but even so, plenty of complexity remains, and more research on that complexity is needed.
In addition to such structural changes, there are qualitative factors, such as quality of the relationships between the partners, with the children; levels of relationship conflict, severity of the conflict, frequency of the conflicts; levels of emotional or physical abuse between partners or toward the children. Certain qualitative factors may influence changes in the structures (Scott, Garibay, and Do 2021) while changes in the structures may influence the qualitative factors; the qualitative and structural factors may interact or mediate with themselves or with other factors in more complex models of family change over time (Jensen and Sanner 2021). Even if there were no bias in social science, we would be challenged by great complexities in our attempts to understand family life.
Weak Social Science Theories
Another problem for social science research involves weak use of theory. Jensen and Sanner (2021) found that, of their review of 283 studies, 63% were atheoretical. Research that does not appear to use any theory is limited in several ways. First, the research will be more difficult to place in context with research that has been based on theory. Second, providing avenues for future research will be more difficult, without having any theoretical guidance. Third, while a lack of theory may be apparent on the surface, it is always possible that an unstated or implicit theory is being used but possibly in a confusing manner. Finally, it is more difficult to build on science in a cumulative way without the help of explicit theory. Thus, we would argue that not using theory is perhaps the weakest form of using theory. But other issues remain. For example, when one hears the term “sexual minority” one may assume that members of such groups are oppressed by the “sexual majority” and accordingly suffer lower levels of education, income, per capita family income, etc. However, some research (Elwood et al. 2017, 2020a, 2020b) has found that some sexual minorities have reported higher levels of education, higher levels of income, fewer children per household, and higher levels of per capita income, as well as lower rates of racial minority statuses, than sexual majorities.
While some would classify such a situation as an example of intersectionality, defined here as complex conditions in which various demographic variables may interact with each other in predicting a variety of outcomes, including, for example, societal oppression, societal privilege (Letiecq 2019), or child outcomes associated with different family structures (Schumm 2016). Intersectionality is often used when minority status is compounded in multiple ways, rather than when a person has one minority status but several other majority statuses. Although it makes sense that minorities might suffer discrimination because of minority status, it is also possible, though seldom tested, that discrimination might occur because of illegal drug use, poor social skills, nonconventional attitudes or behaviors, or other factors only indirectly related to or merely correlated with (not caused by) minority status. Seldom are complex theoretical models evaluated empirically, even if they are developed (Schumm 2020b). Jensen and Sanner (2021: 472) noted the paucity of family science studies that had used complex mediation or moderation models or both together. For example, with a more complex model, it’s possible that sexual minority status might predict illegal drug use, which might predict aggressive behavior, which might predict rejection by others, which might be misinterpreted as rejection only because of minority status. In other words, all too often, research with sexual minorities is founded upon sexual minority theory and little else in the way of theory (Bailey 2020; Rosik and Van Mol 2021; Schumm 2020b).
Alternative explanations are often overlooked. For example, Elwood et al. (2020a, 2020b) reported that legal changes in California law improved the mental health of lesbian and gay adults but neglected the alternative explanation that selection effects associated with marriage (i.e., those who are more educated, have higher incomes, have fewer children before marriage, and have better mental health are more likely to be able to attract partners and enter into marriage) led to better outcomes for married lesbians and gay men rather than reduced discrimination from legal changes. Social desirability response bias (Regnerus 2020) is another concept that is seldom considered as a possible factor in how people might respond to questions about human sexuality. Sometimes social desirability is measured but then apparently not used in an article’s statistical analyses (Lefevor et al. 2020; Schumm 2015; Schumm and Crawford 2021a).
Newness/Weakness of New Research on Families
In addition to the complexity of family life and weak application of social science theories, there are many areas of family life that are relatively new to research and research may have been going on for only a few years in some areas. As Ioannidis (2005) has pointed out, early research is often incorrect because of smaller size studies, non-random studies, studies biased in favor of certain subgroups, poor quality measurement, and weaker statistical analyses, analyses that can be limited by poor quality measurement and/or small sample sizes, not to mention other biases. Research in new areas may seem exciting and groundbreaking, but it’s often misleading and is likely to be refuted by later more advanced research.
Paradoxically, at the same time, even if research has been going on for 50 years, some may say that it’s still in its infancy (Haden and Applewhite 2020) or in much need of methodological improvements (Carone et al. 2021; Jensen and Sanner 2021). For example, Carone et al. (2021, p. 1) state that most research on same-sex parenting continues to be focused on lesbian mothers, rather than gay fathers, bisexual mothers, or transgender parents. It may be easier to assume that gay, bisexual, or transgender parents are the same as lesbian mothers, but that assumption may not be correct. Another weakness of new research is that complex multivariate models are often not tested, sometimes because there is not sufficient data to permit such tests or insufficient theory to guide such tests. As an example, a test of a complex model predicting support for same-sex marriage and parenting found that the strongest predictor was not political party, political orientation, gender, or education but rather belief in casual sex outside of marriage (Schumm 2015). On the other hand, sometimes complex models may be used to obscure relevant information (Regnerus 2020).
Confirmation Bias in Social Science
However, there is bias in social science (and medical research, Baumgartner 2019), and it can play a role in how separate research articles and/or the wider research literature are understood and interpreted. Such confirmation bias is a serious threat in the social sciences (Schumm 2021). While it can occur elsewhere (Baumgartner 2019), it may be especially prevalent in controversial social science research, including that involving same-sex relationships, including parenting issues, as Stacey and Biblarz (2001) acknowledged 20 years ago. Regnerus (2020) has also raised the issue of confirmation bias. Recent work has highlighted publication bias, which is a form of confirmation bias (Baumgartner 2019; Schwartz 2021). But how would one detect it? Assertion is one thing, but providing solid evidence is another.
One detectable pattern is the omission of reported scientific findings, especially in literature reviews, that don’t fit the desired narrative (Regnerus 2020; Schumm and Crawford 2019a, 2019b). For example, most of the time scientists look for statistically significant findings and are eager to report them. Thus, it should be surprising if a scholar found both significant and non-significant findings but only published the latter, omitting the significant findings. A case in point were articles (Flaks 1994; Flaks et al. 1995) that reported non-significant findings from a dissertation (Flaks 1993) but omitted the significant finding that the same-sex parents in the study were substantially more likely than the heterosexual parents (67% vs. 27%, p < 0.01, d = 0.88) to be accepting of their children’s eventual sexual orientation, whether gay or straight.
Not only can such issues occur with individual studies (e.g., Flaks et al. 1995), but it can occur in reviews of the scientific literature, which can also be fed back to the public in misleading ways. Some scholars may feel that scientific “results that could be construed as threatening” should be ignored (Regnerus 2020, p. 57). However, a contrary view would suggest that reviews which ignore any contrary evidence should be questioned. For example, over 90% of the reviews on same-sex parenting published between 2001 and 2017 “found” no evidence/no research for children of same-sex parents being more likely to grow up to become lesbian, gay, or bisexual than for children of heterosexual parents (Schumm and Crawford 2019a). Even the minority of reviews that presented some contrary data usually cited only one or two studies to that effect. Yet, there is evidence from dozens of studies, summarized elsewhere, including meta-analyses (Schumm 2013, 2015, 2018, 2020c; Schumm and Crawford 2021a, 2021b; 2021c), that there the children of same-sex parents are more likely to become nonheterosexual. As a most recent example, Gartrell et al. (2019) found that over 70% of the daughters of lesbian mothers (when the daughters were 25 years old, having been studied since birth in a longitudinal study) reported same-sex sexual attractions. How can it be that over 60 recent scientific literature reviews could arrive at such an incorrect answer to such a basic question? Perhaps it’s largely a matter of confirmation bias creeping into the academic process.
One way that this occurs may be through a tendency for scholars to cite research findings favorable to their causes while ignoring/not citing contradictory evidence. It has been found that authors were far more likely to cite research that was favorable for same-sex families, even their own research (Schumm 2010a; Schumm and Crawford 2020a) than to cite research that was not favorable. As an example, Goldberg, Bos, and Gartrell (2011) found that nearly 60% of the offspring of lesbian mothers, by age 17, had used illegal drugs, compared to about a fifth of the children of heterosexual parents, a substantial and statistically significant difference. However, compared to other studies even by the same authors, that study has been cited less often (Schumm, Palakuk, and Crawford 2020). Journal editors can also succumb to confirmation bias, permitting superficial peer reviews, ignoring relevant conflicts of interest, and accepting misleading conclusions (Baumgartner 2019, p. 105; Brophy 2016, p. 225). As will be seen with some of the following examples of apparent confirmation bias, it may well be that in academia unpopular ideas are suppressed and “met with rhetorical violence or stonewalling rather than intelligible and rational analysis of the relevant information and public debate” (Baumgartner 2019, p. 110).
A cognitive bias codex, featuring 118 such biases, has been presented online by the visual capitalist (www.visualcapitalist.com/wp-content/uploads/2021/08/all-118-cognitive-biases.html, downloaded 21 April 2022). Confirmation bias is one of 13 biases classified under the section “that we are drawn to details the confirm our own existing beliefs.” Thus, is not confirmation bias a tendency to accept findings that one expected and to reject findings that were not expected? In other words, if one expected one’s research to lead to a rejection of the null hypothesis, one might be happier to find such; on the other hand, if one expected one’s research to not lead to a rejection of the null hypothesis, one might be happier to find that. This can occur subconsciously (Schwartz 2021, p. 104). Given an ever present limitation of resources in terms of time and funding, once one finds what one expected, there would always be a sound resource-based argument to conclude that research and move on to something else. Only if the findings were unexpected would one be tempted, perhaps, to dig deeper and see if there were a way to “save” one’s expectations, even at greater cost in resources. However, better science would involve challenging one’s results, even when expected results had been found, even assuming the role of one’s critics and trying to deconstruct the findings as those critics might.
If your expectation is to not reject the null hypothesis, certain methodological approaches may make finding that expectation more likely (Schumm 2010c, 2012a, 2021; Schumm and Crawford 2015). Thus, if one wanted to show that one group of children were no different than another group of children, you could use small samples; non-random, biased samples; low quality measurement, overlook social desirability issues (Regnerus 2020), use inappropriate statistical tests, break up your sample into even smaller groups, or overcontrol for confounding variables (Regnerus 2020), among many other approaches. One approach often used is to compare group A, not versus group B, but to a mixture of groups A and B, which will obscure true differences between groups A and B; for example, Sullins (2015) found that of a group of 44 allegedly same-sex (lesbian) families, only 17 of those 44 families involved true lesbian couples, rendering comparisons of the “lesbian” families with heterosexual families in previous research reports suspect at best.
On the other hand, if one wanted to reject the null hypothesis, one might take an opposite approach, using larger samples, higher quality measurement, random samples, and select a variety of control variables (Schumm 2021). In either approach, one would avoid discussion of effect sizes, the magnitude of the effects, because with smaller samples you might find medium to large effect sizes, even if your results were not statistically significant, while in larger samples, you might find significant results that involved trivially small effect sizes. In a general sense, an effect size (Cohen’s d) is a difference in the mean scores of two groups divided by a measure of their standard deviations. An effect size of about 0.50 should be large enough to be detected by the naked eye, if allowed (Cohen, 1992a; Cohen, 1992b), but small effect sizes can be meaningful (Funder and Ozer 2019), as they concluded that effect sizes of 0.20 (even 0.10 at times) or higher are important. If a result involves both a medium or larger effect size and is statistically significant, all the better. However, a very large effect size could be problematic if it only reflects a substantial degree of common methods variance, social desirability bias, or other research artifacts (Funder and Ozer 2019).
When it comes to research that might relate to policy changes, some scholars may present evidence that something makes life better for a subgroup of the population but discount the effect that same matter has on other subgroups or the total population (Elwood et al. 2020a, 2020b). For example, one might argue that legalization of same-sex marriage had helped those who got married; however, what was the effect on those who didn’t get married or on heterosexual marriages or rates of marriage?
We will try to illustrate the larger problem of potential confirmation bias with several examples, using cases representing both possible liberal and possible conservative biases. While liberal bias tends to dominate the field of family science, even to the “queering of family studies” (Regnerus 2020, p. 56), conservative bias is also common in, for example, assumptions that the traditional nuclear family is inherently superior to other family forms (Jensen and Sanner 2021, p. 477). Even if research (Sullins 2021) does support the benefits of mothers and fathers for children in general, that might lead scientists to overlook problems within nuclear families (e.g., family violence, emotional abuse).
Examples of Several Problematic Studies
It may be difficult to convey the seriousness of scholastic bias without delving deeply into methodological details. Here, eight studies will be examined in detail. The first five concern potential liberal/progressive bias, while the last three concern potential conservative/traditional bias. The overarching concern, regardless of the direction of bias, is that there is a risk that methods and findings used will slant in directions that favor specific outcomes (Fine 2022; Schumm 2021). The headings for these eight examples will be based on the subject matter content of the research under discussion.
The Mental Health of Transgender Children
Durwood, McLaughlin, and Olson (2017) compared the self-esteem of transgender children with that of cisgender children. Durwood et al. claimed to have found no differences. Perhaps in part because that finding was expected, even politically correct, the study has been cited 300 times since as of February 16, 2023 [the sister study, Olson et al. 2016b, has been cited 885 times]. In particular, they claimed that there were no significant differences in self-worth between the transgender and cisgender children.
But was that a correct finding? Durwood et al. did not report the overall means and standard deviations for the two groups but only the means/standard deviations for three subgroups, divided by age. Furthermore, they divided the required significance level by three since they had created three subgroups. However, if one were to calculate the overall means and standard deviations, as reported in Schumm and Crawford (2020b), the results (effect size of 0.31) would be statistically significant (p < 0.02), with lower levels of self-worth for the transgender children even when their parents were supportive; furthermore, one of the three subgroups (middle children) would also be significant in the same direction (effect size of 0.45, p < 0.03). Funder and Ozer (2019) have indicated that effect sizes of 0.20 or larger may be small at the level of single events but ultimately more consequential across further studies while effect sizes of 0.41 or greater are of medium size that are of practical use in the short run with individual studies and even more important in the long run (p. 166). Thus, the overall results changed from no differences if parents supported their children to significant differences in an adverse direction, even if their parents were supportive.
Compared to the nearly 1,100 citations for Durwood and Olson’s articles, only four studies have independently cited Schumm and Crawford (2020b) – Kirichenko (2020), Pilgrim (2021), Dangaltcheva, Booth, and Moretti (2021) and Vadevelu and Arunberkfa (2022). Kirichenko (p. 29) argued that we had noted some methodological issues (Schumm et al., 2019) but that the main conclusions of Durwood, McLaughlin, and Olson (2017) remained unaffected. Dangaltcheva et al. cited Schumm and Crawford (2020b) to note there was some debate in the area but the “larger literature consistently underscores” the role of parental support (p. 2). Pilgrim, however, stated that “Olson (2016b) argued that transitioning children with parental support have the same levels of psychopathology as controls. However, a review of her raw data undermines that claim” (p. 8). Vadevelu and Arunberkfa (2022) cited Schumm and Crawford (2020) in connection with their convenience sampling plan (p. 6) and with respect to future research (p. 12), along with many other articles that they cited. Thus, three of the four citations ignored the essential findings of Schumm and Crawford (2020b), which may either reflect confirmation bias or the difficulty in overturning incorrect but widely cited results.
Retaining their same methods, Gibson, Glazier, and Olson (2021) reported on an enlarged (from the same sample as before, although some younger children were not included (see Durwood, et al., 2021 for a larger sample) group of transgender and cisgender children, citing one significant result for parent-reported anxiety (d = 0.29, p = 0.04, though our reanalysis found p < 0.015). However, further analysis also found d = 0.20 for parent-reported depression and for child-reported anxiety, p < 0.10, two-tailed; in other words, if one had expected higher depression and anxiety for the transgender children and used a one-tailed test, both of those results would have been statistically significant as well. For the three groups, odds ratios ranged between 1.82 and 1.96 for the rates of transgender versus control children scoring in the clinical ranges of anxiety or depression. Gibson, Glazier, and Olson (2021) concluded that transgender children had at worst “only slightly higher” levels of anxiety and depression. To accept such a conclusion, one would have to reject Funder and Ozer’s (2019) conclusion that effect sizes of 0.20 or higher are important, as well as the merits of our earlier critique (Schumm and Crawford 2020b). One could have hypothesized that transgender children would have higher levels of anxiety or depression and lower levels of self-worth, either because of minority stress theory or issues inherent to transgenderism in general; but with this example, clear evidence in support of such a hypothesis is overruled by improper methodology, which suggests a bias against such a hypothesis in favor of a pro-transgender hypothesis. Furthermore, another hypothesis could have been that parental support is important but insufficient for sustaining the mental health of transgender children, which would fit with minority stress theory (negative peer and cultural influences are counteracting a positive role of parental support) but that hypothesis was not tested either.
The Impact on Mental Health of Same-Sex Marriage Legalization
Elwood et al. (2020a, 2020b) reported that the legalization of same-sex marriage in California had significantly improved the mental health of married same-sex partners. Although, they were using a large sample, they did not report effect sizes. But a deeper look into the research suggests some different results. First, in their review of the literature they cited Hatzenbuehler et al. (2018) as one of their key references. Elwood et al. (2020a) highlighted their findings by saying “Furthermore, lesbians, gay men, and other sexual minorities who live in American regions with greater levels of prejudice are more likely to die approximately 12 years prematurely from cardiovascular diseases, homicide, suicide, and other causes than sexual minorities who live in more tolerant areas” (p. 935).
Hatzenbuehler (2016) in a review of the literature in the area of structural stigma and health inequalities cited their 2014 paper’s findings as some of their key evidence. The problem is that Hatzenbuehler et al. (2014) was retracted (2018) because of a coding error that changed the results dramatically (Regnerus 2017). In other words, in that article, sexual minority theory didn’t work out as expected, and Elwood et al. (2020a) were not correct in citing a retracted article as if its findings were valid. Elwood et al. are not alone in citing a retracted article as if it were still valid. Among dozens of articles continuing to cite the retracted article in Google Scholars, Moagi et al. (2021), for example, analyzed several thousand articles on mental health challenges for LGBT persons and boiled them down to the 21 most relevant, including Hatzenbuehler et al. (2014) among them despite that article having been retracted for 3 years.
Furthermore, a careful study of the demographic differences between LG adults in the study compared to their heterosexual counterparts (Elwood et al. 2020b) shows that the LG adults were demographically less stressed, if not better off than the heterosexual adults: for their 2005–2007 data – more Whites (56.8% vs. 45.9%), less female (49.5% vs. 58.9%), less likely to have a child at home (22.0% vs. 49.6%), more likely to have a college degree (44.4% vs. 32.7%), more likely to be employed (77.0% vs. 74.3%), more likely to have an income 3.5 times the federal poverty level (60.4% vs. 50.9%), and more likely to be in excellent health (21.8% vs. 21.1%); and for their 2008–2015 data – more Whites (47.3% vs. 40.8%), less female (50.9% vs. 57.8%), less likely to have a child at home (24.8% vs. 44.5%), more likely to have a college degree (41.2% vs. 35.3%), as likely to be employed (68.7% vs. 69.1%), more likely to have an income 3.5 times the federal poverty level (49.1% vs. 44.6%), but less likely to be in excellent health (17.6% vs. 19.4%). Their findings of better demographic conditions for same-sex couples compared to heterosexual couples has been recognized for at least a few years (Smock and Schwartz 2020). Even if married same-sex couples had better mental health after legal marriage was extended to same-sex couples, that difference might have been the result of selection effects rather than a reduction in stigma (i.e., more mentally healthy persons being more likely to get married because of their greater mate selection value and attractiveness). Thus, there are alternative plausible theories to sexual minority theory (Bailey 2020; Rosik and Van Mol 2021; Schumm 2018, pp. 31–46; 2020b), while the theory’s predictions about the demographic problems that might be expected for a stigmatized minority were in the wrong direction for most comparisons.
Turning to the research itself, if one looks at the mental health (K6 Scale, higher score = greater distress) of the non-married lesbian or gay persons, it was worse after legalization, changing from 4.81 to 5.40, with an effect size of −0.07 [t (5438) = 2.54, p = 0.011]. The mental health of married lesbian or gay persons changed for the better, from 5.09 to 4.14, with an effect size of 0.12 [t (1553) = 1.88, p = 0.061]. But the outcome is more than just that—the overall mental health of all the lesbian or gay persons in the sample was worse (4.84–5.06, effect size of 0.03) but not statistically significant after California legalized same-sex marriage, because the married represent only 22.2% of the total sample of lesbian and gay adults surveyed—declines among the unmarried more than offset gains among the smaller group of married LG adults. In terms of the percentage of persons with only poor/fair health, there was a slight (20%–17%, effect size of 0.04) non-significant improvement for married lesbians and married gay men; while for the unmarried group there was a decline in health (18%–23%, t (5438) = 2.01, p = 0.045, effect size of 0.06). Overall poor/fair health increased for both nonheterosexuals (18%–21%, effect size of −0.05, p < 0.05) and heterosexuals (17%–19%, effect size of −0.01, not significant). Furthermore, the study found that mental health declined (not statistically significant) among all heterosexuals (3.57–3.60, effect size of 0.004) in the study. Comparing the mental health of heterosexuals versus nonheterosexuals, for the earlier surveys, t (76,977) = 11.53, p < 0.0001, with an effect size of 0.23 while in their later surveys, t (115,479) = 9.57, p < 0.0001, with an effect size of 0.15.
Before the change in law, there were 135 married same-sex parent couples with children with an average mental health score of 5.09; afterward, there were 362 married parent couples with an average mental health score of 4.14 (a substantial improvement possibly for the children of married same-sex couples as lower scores represented better mental health). For the unmarried parent couples, there were 331 couples with an average mental health score of 4.81 before and 475 with an average mental health score of 5.40 after the change in law. For the results in terms of the percentage of respondents who scored higher (at least a moderate level) on a scale of mental distress, rates dropped for married lesbian and gay persons from 50% to 33% but increased from 43% to 48% for unmarried lesbian and gay persons while rates were lower (29%) for heterosexuals both before and after (Elwood et al. 2020a, p. 9). The data were not broken down by whether the couples had children or not, but the overall impression seems to be that more children were living in families with worse mental health after the law change than before. What was found was that the mental health of married lesbians and married gay men improved over the years, with a modest but non-significant effect size; at the same time, the mental health of unmarried lesbians and unmarried gay men declined, with a smaller but statistically significant effect size. Furthermore, it seems that children may not be faring well, since it is possible that more children are living after the law change with same-sex parents with worse mental health. Smock and Schwartz (2020) found that “same-gender couples with college degrees were dramatically less likely to have children in the home” (p. 18), which may suggest that children with same-sex couples may be more likely to be living with parents with less socioeconomic resources and possibly worse mental health.
Yet, Elwood et al. (2020b) concluded that mental health improved “profoundly” (p. 938) for the married group and declined “slightly” (p. 938) for the unmarried group. Yet it appears that neither effect size reflected a “profound” difference, and what they termed “slightly” was more significant statistically than what they described as “profound.” Combining both the married and unmarried groups of lesbians and gay men, the net result was a smaller effect size (−0.03) that was not statistically significant. If one were to attribute all change in their results to the legal changes, the net result was not positive. But perhaps the most remarkable finding was that there were far greater effect sizes for mental health differences as a function of sexual orientation in spite of the numerous demographic advantages of the lesbians and gay men, not to mention favorable changes in their legal environment.
Reducing Suicidal Ideation Among LGBT High School Students
Hatzenbuehler et al. (2014) studied survey data from high school students across a variety of locations and developed a measure of protective school climate for LGBT students. Their plan was to assess whether more protective school climates reduced suicidal thoughts, plans, or attempts among LGBT students. While the results for plans and attempts were not statistically significant, the result for suicidal thoughts was significant (p < 0.05). On the surface that result seemed promising – a reduction in suicidal thoughts. However, their figure on page 284 shows that while the apparent reduction in suicidal thoughts was from 37% to 20% for lesbian and gay students, suicidal thinking seemed to increase from about 10% to 12% for their heterosexual youths (not significant). As with the second example above, there may be a situation of winners and losers. Hatzenbuehler et al. (2014) did not specify the exact number of missing cases for each of their groups, the sample sizes they reported for the original data set were used. The reduction of 17% for the 735 lesbian or gay youths might represent help for 125 lesbian or gay students, a positive outcome. However, the increase of 2% for the 51, 644 heterosexual youth could represent worse conditions for 1033 heterosexual youths, a net negative outcome for 908 youths. Suicide prevention programs should reduce suicide rates for all groups of students.
Stability of Same-Sex Marriages or Relationships
As a fourth example, Rosenfeld (2014) reported that same-sex marriages (with children) were just as stable as heterosexual marriages (with children), his article having been cited at least 125 times as of February 20, 2023. His multivariate analysis was published in a top-tier journal and played an important role in the court cases leading up to Obergefell in 2015. However, there were problems with the makeup of the study; nearly 100 of the couples in the sample that were counted as “stable” were couples for which one partner had died since the start of the study (Schumm 2015, 2018). That’s not how stability is typically measured. Second, there were only four same-sex couples who saw themselves as married and who had children—they were compared to just under 500 heterosexual couples who were married and had children (Schumm 2015). The breakup rates were 25% versus 7.8% but that large difference was not significant statistically because of the extremely small sample size for the nonheterosexual couples.
For example, if there were four lesbian parent couples and one of them broke up, while you had 96 heterosexual parent couples and only one of them broke up in the same time period (1%), even such a 25% versus 1% difference would not be statistically significant by a two-sided Fisher Exact Test, even though other tests might show a significant result. Even if none of the 96 heterosexual couples broke up, the results would barely reach statistical significance (p = 0.04) by a two-sided Fisher Exact Test. The bottom line is that if you want to compare a group of four cases against larger groups, it will be very difficult to obtain a statistically significant result, because of the small number of cases in the smaller of the two groups. That such research didn’t find any significant results may reflect the unusual structure of the two data sets more than any underlying social psychological reality.
Academic Achievement of Children of Same-Sex Parents in the Netherlands
Kabatek and Perales (2021) obtained a sample of 3006 same-sex families from the Netherlands, comparing them to between 1.5 million to nearly two million heterosexual families. They argued that when they controlled for enough variables, the children with lesbian mothers appeared to do better academically than did children with heterosexual parents. Their results are extensive and seem convincing at first glance. However, one might question whether academic achievement (or even labor force outcomes) is the most relevant outcome for children of same-sex parents. Past research has found stronger results for children’s sexual orientation, sexual behavior, drug use, and gender roles than for educational differences (Schumm 2018). In their conclusion, the authors state that their findings “contradict deficit models of same-sex parenting as well as claims that being raised in a same-sex family has an independent, detrimental effect on children” (p. 415). Of the dozens of outcomes possible for children, their research addressed one major area, not the host of other areas. In particular, selecting education as an outcome—especially when the effect sizes are small—may be misleading in that some students may find “refuge” in academia: in general, because teachers may treat them more kindly than their parents or peers and in particular, because academia may be one of the institutions that features the least hostility against same-sex parents or their children or LGBTQ students regardless of their parents. Notably, the same-sex parents had more college education than the other parents (effect size, d, = 0.83, a large effect). The families of same-sex parents had fewer children to raise (d = 0.46, a medium effect) and per capita household income was probably higher (d = 0.11), although the effect size for both parents being employed was small (0.03). Given those effects by themselves, it is not surprising that the children of same-sex parents scored higher in four areas, with effect sizes of 0.13, 0.17, 0.04, and 0.15, all less than “small.” The “surprise” might be that the children from same-sex parent families didn’t do much better academically. However, the authors did not report effect sizes for the demographic differences between the two groups of families. Since the authors controlled for their independent variables by blocks, I will discuss the effect sizes as calculated for each block. In terms of the block for socioeconomic background, the same-sex parents had advantages, as noted, for education and per capita income, and a slight advantage for both parents being employed (d = 0.03). For the block of demographic variables, same-sex parents were less likely to be married (d = 1.21), more likely to be registered partners (d = 1.64), less likely to be first-generation immigrants (d = 0.37), and, as noted, had fewer children (d = 0.46). The block for family dynamics included a larger number of variables. Same-sex parents were more likely to have made at least one residential move (d = 0.17), to have made more residential moves (d = 0.29), to have had a least one change in family structure (d = 1.35), and to have made more changes in family structure (d = 1.35). In terms of initial household composition, same-sex parents were less likely to have been together since having their child (d = 1.44), more likely to have changed parents from the initial composition (d = 1.42), were more likely to have started as a single-parent household (d = 0.82), or have been in a different household type (d = 0.87). Remarkably, the child’s share of life in a same-sex household featured an effect size of 36.71. Without the original data, it is difficult to predict how changing their analyses would change their results, but it would seem that many of their control variables were highly correlated, which creates analytical challenges, and that many of their control variables had very large effect sizes, far larger than the effects for their outcome variables. One would wonder if they used their control variables to predict which families were same-sex versus heterosexuals, if one would not obtain a very large explained amount of variance. In addition, if one uses, say, two variables to measure concept A and one variable to measure concept B, it is possible that the results for concept A will be minimized since the relevant variance is split up between highly correlated items.
Since they measured family dynamics with more variables than they used to measure socioeconomic status, it is possible that this artificially minimized the effects of family dynamics on educational outcomes. Furthermore, the authors reported that children of same-sex parents, although they were not more likely to fail a grade, were more likely to enroll in a college preparation curriculum, more likely to graduate from high school, and more likely to enroll in college. However, they were nearly twice as likely, relative to the children of heterosexual parents, to enroll in the college preparatory curriculum than to enroll in college, but that difference was not explained or discussed. In other words, if one were to predict college enrollment from having been in a college prep program in high school, the results might show that same-sex parenting might have had a negative effect. While the authors presented a figure that suggested an interaction effect between academic outcomes and a child’s tenure with same-sex parents, other research (Sullins 2016b) suggests an opposite interaction effect for non-academic variables such as depression, suicide ideation, and anxiety. A final limitation to note, common to other similar types of research, is that by restricting their analysis to intact couples (because they cannot determine the sexual orientation of single parents in their data set), the results may be biased by omitting the data from the children of single parents (whether lesbian, gay, bisexual, or heterosexual) whose children may be suffering academically due to their parents’ separation or divorce. Only studying the children of parents in currently intact relationships will not give us a true picture of any harms done to children through family structural changes over time.
Gay and Bisexual Men Who Have Experienced Sexual Orientation Change Efforts
Sullins, Rosik, and Santero (2021) reported results from a study of 125 gay and bisexual men who participated in some degree of sexual orientation change efforts (SOCE). An earlier version of the same study (Santero, Whitehead, and Ballesteros 2018), which was based on a dissertation (Santero, 2011) was retracted (Retraction Notice, 2020), followed by a protest from one of the original co-authors (Whitehead 2019). The study had a number of commendable aspects, but it also featured room for improvement. First, there was considerable opportunity to create multi-item measurement scales from many of the study’s questions, but results for some of those possible scales were not reported in Sullins, Rosik, and Santero (2021). Second, those providing SOCE need, if possible, help in understanding how pre-existing conditions for their clients may influence SOCE outcomes with respect to apparent sexual orientation changes as well as their evaluation of SOCE helpfulness. How would combinations of pre-existing conditions influence SOCE outcomes? Such answers were missing. Another important question unanswered by Sullins, Rosik, and Santero (2021) is whether men for whom SOCE led to stronger levels of homosexuality will report more or less satisfaction with their SOCE experience. Would men who became more homosexually oriented over time reject their SOCE experience and/or report high levels of dissatisfaction? Would a strong homosexual sexual identity before therapy reduce the chances of success in terms of reducing same-sex attraction? Would there be interaction effects among pre-test and post-test measures of sexual attraction, behavior, and identity? Would it be possible to develop a typology of satisfaction with therapy experience overall? Were younger clients more or less likely to report greater harms versus benefits from SOCE? Did congruence between sexual orientation attraction and identity before SOCE predict SOCE outcomes? These types of questions remained unanswered from the results presented in Sullins, Rosik, and Santero (2021), but some of these questions have been the focus of a recent report (Schumm, 2022).
Conservative Research on Adult Children of Same-Sex Parents
Regnerus (2012a, 2012b) received intense criticism for two articles on same-sex parenting (e.g., Kabatek and Perales, 2021; see further discussion in Schumm, 2015). An earlier commentary (Schumm 2012b) on his articles was deemed by some scholars as a “defense” of his research. However, the standard deviations for two of Regnerus’s binary variables were mathematically impossible for the sample size (Schumm, Crawford, and Lockett 2019a, 2019b). While his critics were correct that his sample lacked more than two or three (of nearly three thousand families represented) cases of children raised from birth by continuously intact lesbian (or gay) parent couples, they did not recognize that most other studies of lesbian or gay couples also featured large percentages of children from unstable lesbian mother or gay father parent couples (Regnerus 2020; Schumm 2018, 2020b, 2020c). Even so, Regnerus could have included in his requirements for the research company that conducted the research a requirement to provide data on at least 30 to 50 adult children whose same-sex parents had been in a stable relationship with each other from the birth of the focal child to when the child turned 18 years of age. If the company had been unable to provide such data, then the refunded funds could have been used to collect such data independently, because the lack of data from children raised from birth to adulthood in same-sex families has remained a major concern with the Regnerus (2012a, 2012b) study, as well as for many other studies with children from same-sex parent families.
More Recent Conservative Research on Children of Same-Sex Parents
Sullins (2016a) is another study of the children of same-sex parents, which was criticized by Frank (2016) and was issued a letter of caution (Depression Research and Treatment 2017). On the positive side, Sullins detected that a majority (61.4%) of the same-sex parents of the alleged 44 such couples (Wainright and Patterson 2006) were in fact heterosexual parents. However, removing the heterosexual couples from the sample reduced the sample size to only 20 families (three gay parent families added to the 17 lesbian parent families). It is not clear from his initial analyses (Sullins 2016a) how many of the same-sex families were stable over time or how many might have left their partners and gained new opposite sex partners; later, he added relevant material (Sullins 2016b). While his sample included 12,268 heterosexual families, it had only 20 gay or lesbian families. By predicting dependent variables from several independent with such a small sample, there is a great risk of increasing type II error—obtaining non-significant results for the independent variables (even though the non-adjusted R squared might seem large) more due to low statistical power (small sample size) than to factual underlying conditions (Perez et al. 2019; Schumm 2010c, 2012a, 2018; Schumm and Crawford 2015). The sample size difference shows up in large differences in standard errors for the key variable, such as 0.44 (larger sample) versus 19.0 (smaller sample) for suicidal ideation. The smaller sample is borderline too small to detect nonlinear effects or interaction effects, which might be involved in comparing the two samples (e.g., Allen and Price 2020). It’s not clear how many of the children in either sample reported same-sex sexual attraction, behavior, romantic involvement, or identity across the waves of the study or if the rates differed across the samples. In this article adding obesity, as a last addition to the list of predictors, reduced the overall results to non-significance. However, it is well known that lesbians (who made up 85% of the sample) tend to have higher BMIs (Boehmer, Bowen, and Bauer 2007); controlling for obesity is to some extent controlling for a key tendency of lesbianism itself. In other words, using large numbers of independent variables with small samples can lead to overcontrol statistically (Regnerus 2020). Nevertheless, Sullins (2016a) found that the adult children of same-sex parents were substantially more likely to report having been abused (sexually, physically, or emotionally) as children (d = 1.17), higher levels of depression (d = 0.85), to have considered suicide (d = 0.97), or being distant from one or both parents (d = 0.84) while as adolescents having considered suicide (d = 0.89), had anxiety (d = 1.09), been distant from one or both parents (d = 1.83), and been obese (d = 0.58), although reports of depression were lower (d = −0.13, not significant). Sullins’ evidence, however limited, contradicts assertions that there is absolutely no evidence that the children of same-sex parents experience mental health problems. Notably, the journal’s expression of concern did not dispute the facts, only that the facts that indicated that the children of same-sex parents were having problems that might be used to justify their harm (Regnerus 2020).
Discussion: Best Practices
General
Social science research in general can be subject to methodological bias. For example, Badovinac et al. (2021) list a number of sources of such bias, including failure to state the research questions, not defining the study population well, having a participation rate less than 50%, inconsistent recruiting parameters, not providing power or effect size estimates, not using interval or ratio measurements of outcomes, using invalid or unreliable predictor variables, using single-item measures of outcomes, not using blinded coders, retention rates less than 80%, not controlling for confounding variables, not describing gender distributions, not describing statistical methods used in sufficient detail, and not reporting actual p values (p. 574). While methodological biases may often occur because of other factors (e.g., lack of adequate training in methodology), sometimes they appear to be confounded with confirmation bias, although reasons for such confounding may remain obscure (Schumm 2021).
Effect Sizes
A first step toward lowering the risk of bias is to report effect sizes as well as significance levels. Significance levels depend heavily on sample size. With large samples, a small effect size can be statistically significant; with a small sample, a large effect size may not be statistically significant (Regnerus 2020). Significant tests indicate if a given result might have occurred by chance alone (or not); effect sizes tell us how large or substantial an effect may be. For those whose bias is in favor of the null hypothesis, reporting effect sizes (which might be large even if not significant statistically) may counteract too much emphasis on significance level by itself when small samples are used. For those whose bias is in favor of rejecting the null hypothesis, reporting effect sizes (which might be very small yet statistically significant) may counteract too much emphasis on significance level by itself when large samples are used.
Minimizing Politicization of Science
A second good practice would be to move away from any idea that everything is political, including science, because that approach, while partly realistic, can lead to a perverse justification of using science for political advantage rather than using science to help us discover facts that might not fit our preferred values or narratives. Honest scientists have been discriminated against when they refused to agree to only publish results that would please their employers or professional organizations. One interesting response to political pressures and/or bias is for scientists of opposing theories to join together to analyze, discuss, and report results, as has been done recently (Lefevor et al. 2019; Lefevor et al. 2020; Lefevor et al. 2020; Rosik, Lefevor, McGraw, and Beckstead, 2021). Lefevor et al. (2019) stated that “A politically, religiously, and sexual-identity diverse research team was constructed to reduce bias” (p. 462); likewise, Lefevor et al. (2019) said that they hoped that “respectful collaboration” and “collaborative inquiry of differing, even opposing, ideologies would increase critical thinking and reliability” (p. 357). Likewise, Watson (2019) argued that dialogue between opposing theories could “profoundly enhance thinking within both” (p. 4).
A Culture of Humility
A third good practice, which may seem to contradict the previous point but actually supports it, would be to approach our own research with humility, recognizing that anyone, including ourselves, can make mistakes, of omission or commission, in developing our theories or our research (Davis et al. 2021). Researchers should be glad if someone else discovers flaws in theories or research, even in already published materials. And science in general and the public should be glad for research or commentary that exposes such flaws.
It takes humility to look at both sides of an issue—or from a variety of theoretical perspectives—in conducting research. Sometimes research even presents itself as a contrast between two different theories, allowing for critical tests to compare the empirical validities of both theories (e.g., Schumm 2020d; Schumm and Goldstein 2021). Rather than assuming that one theory is correct and the other theory is wrong, perhaps they are both correct under certain conditions or historical times (Wicker 1985, p. 1099) or perhaps both are incorrect or both partially correct. Facts will be more helpful than non-facts for the public in general and for smaller groups in particular. If a scientist espouses a cause or worldview (Sunshine 2009) to such a degree that it biases their research, in the long run, their bias will be exposed to their shame. In today’s culture it takes more courage to try to be even handed as a scientist because cancel culture can be a real issue for anyone who challenges certain viewpoints or assumptions.
Another dimension of humility is that a superficial result may be just that. Did you find a positive correlation? It might be an artifact of an extraneous or lurking variable, even a distorter variable (Brase and Brase, 2015, p. 541; Rosenberg 1968). Is there a negative correlation? The same possibilities might present themselves. Is there a non-significant correlation? It might be non-significant from a small sample size rather than not being a substantial effect size. Did you find a significant correlation? It might be a trivially small effect if the sample is large enough. Did a group in therapy have worse outcomes? It’s possible that the therapy group went into therapy because their pre-treatment outcomes or conditions were substantially worse than those of the control group. Humility means that no matter what initial result you obtain, researchers ought to want to dig deeper, beyond a superficial level, even if the initial results seem promising.
Another example of humility is recognizing the limitations of cross-sectional survey research. Along those lines Lefevor et al. (2019) acknowledged “the impossibility of drawing causal comparisons from survey data” and that “we strongly caution against a causal interpretation of our data” (p. 367). In contrast, it’s remarkable how many times one will find an article, using survey research, that proceeds to make causal conclusions and draw policy implications, even after a wink and a nod to the limitations of cross-sectional survey research.
Reduce Bias
A fourth good practice would be to recognize that having a preferred perspective can lead to, consciously or unconsciously, the use of methods that bias research outcomes in the direction of our own values or political ideology, as has been discussed elsewhere (Badovinac et al. 2021; Regnerus 2020; Schumm 2010b; 2015, 2018, 2021). As scientists, we need to be aware of this and do our best to “catch” biases before they lead to poor research, policy decisions, and the inevitable victims of those poor policies or laws.
Combining Humility and Curiosity
A fifth practice is a combination of humility and curiosity. Too often scientists think exclusively in terms of main effects, that is, whether one factor influences another factor. However, life is complex, which takes humility to acknowledge and curiosity to explore. Two important aspects of such complexity may be found through testing for mediation and interaction effects (Jensen and Sanner 2021). Mediation means that one variable influences another via a third variable. For example, high levels of cholesterol might lead to (or reflect) damage in certain blood vessels, which might reduce a person’s cognitive capabilities. If having high blood pressure worsened the relationship between cholesterol and cognitive reductions, that would be an interaction effect. Even if we find that two variables are not related to each other, that null result could be an outcome of a suppressor variable (Bergstrom and West 2020). Distorter variables can change an apparent positive relationship between two variables into an apparent negative relationship. Another factor is that situations are dynamic as much as static. In one case, using a static variable (whether a state had ever adopted a law for same-sex marriage) to predict outcomes of the legalization of same-sex marriage yielded no significant results, but using a dynamic, time-based variable (how many years had passed since the state had adopted a law for same-sex marriage) did yield significant results (Schumm 2018, pp. 197–207). It is also possible to have linear and nonlinear components to statistical models; assuming that all relationships are linear without evidence can lead to mistakes with serious public health consequences (Schumm et al. 2021). The complexity of life demands humility from the scientist but also should create a vast degree of curiosity.
Rigorous Testing of Competing Theories
Without rigorous testing, it would be all too easy to run an analysis and as soon as it confirms one’s own theory stop any further analysis. Such a practice leaves it up to other scholars to retest your findings, which takes time and, at best, delays publication of contrary results. Rather, one should assume the role of one’s own critic and challenge one’s preferred results. As a recent example, there were competing theories about a historical event. One person favored theory A while another favored theory B. The discussion was acrimonious and unlikely to make much progress because neither person was subjecting their theory to statistical testing. After several years, other scholars came long and tested the two theories statistically. Rather than finding that theory A was right and theory B wrong, or vice versa, they found that both theories were right, just at different historical time periods. Theory A worked best earlier in history while theory B worked better later in history. Thus, neither theory was “wrong” per se, each just fit the data better at different time periods (Schumm 2020d; Schumm and Goldstein 2021). As long as the theories were seen as polar opposites where one had to be right and the other had to be wrong, the actual situation—that each worked better, just at different times—might not have been discovered. Having competing theories wasn’t a “bad” thing, but what was needed was for each side to give serious consideration to the possibility that the other side was correct, at least part of the time. As another example, two potentially contradictory theories—cognitive dissonance theory (that highly religious sexual minority Mormons would feel a conflict between their religion and their sexual orientations) and minority stress theory (that sexual minority Mormons would experience discrimination because of their sexual orientations)—were proposed in an article, but the research found support for both theories (Lefevor et al. 2020). Such academic flexibility is needed, especially when different sides seem to be polarized. Similar theoretical problems can occur if we try to fit a two-dimensional reality into a one-dimensional line. If reality involves two (or three or more) dimensions and we force theory to be only one-dimensional, there must be over-simplification and probably a great deal of error. Having no theoretical foundations for one’s research, of course, may be more problematic than theory that is too simplistic (Jensen and Sanner 2021). In addition to testing two theories with one sample, we may want to consider testing one theory in more than one very different samples rather than assuming that our theory is only relevant to one type of sample (Parent et al. 2018).
Conclusion
The practices we have tried to describe need to be lived, not merely claimed. We must be willing to have our pet ideas falsified, even our religious or political ideas, no matter how dear to us or others, and must avoid the use of methods that artificially favor our pet hypotheses. That type of humility must be adopted not only by scientists but by leaders at all levels of society. At the same time, we should not change scientific course because of a study here or there; there must be a pattern of reproducible results that gains our attention and leads eventually to changes in scientific theory. Theory should not stand by itself but should be subjected to scientific testing over the years to assess its validity or usefulness.
Footnotes
Declaration of Conflicting Interests
The authors declare no potential conflicts of interest for the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Austin Institute for the Study of Family and Culture (3206 Fairfax Walk, Austin, Texas 78705).
