Sex as a Biological Variable: A 5-Year Progress Report and Call to Action

Abstract

A little over 5 years ago, the U.S. National Institutes of Health (NIH) announced the intention to develop policies to require applicants to report plans to balance male and female cells and animals in preclinical investigations. Soon thereafter, the NIH issued a request for information from the scientific community and consulted with various stakeholders. The feedback received was considered during development of policy requiring the consideration of sex as a biological variable (SABV) in NIH-funded research on vertebrate animals and humans, which went into effect for applications due on or after January 25, 2016. We identified NIH programs related to SABV and reviewed SABV-relevant scientific literature. We find that the application of SABV throughout the research process can serve as a guiding principle to improve the value of biomedical science. The NIH is engaged in ongoing efforts to develop resources to help investigators consider SABV in their research. We also provide an update on lessons learned, highlight ways that different disciplines consider SABV, and describe the opportunities for scientific discovery that applying SABV offers. We call on NIH's various stakeholders to redouble their efforts to integrate SABV throughout the biomedical research enterprise. Sex- and gender-aware investigations are critical to the conduct of rigorous and transparent science and the advancement of personalized medicine. This kind of research achieves its greatest potential when sex and gender considerations are integrated into the biomedical research enterprise in an end-to-end manner, from basic and preclinical investigations, through translational and clinical research, to improved health care delivery.

The mission of the National Institutes of Health (NIH) is to seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability. As the single largest funder of biomedical research, NIH has a unique responsibility to continually examine and improve its effectiveness. By “turning discovery into health,” NIH fulfills its promissory note to the taxpayers who fund biomedical research. Making the most of this investment depends on a well-functioning research enterprise that continually earns the trust of scientists and laypeople alike.

We agree with colleagues who said, “Science isn't science if it isn't reproducible.”¹ NIH continues to focus on improving rigor and transparency to enhance scientific reproducibility through policy^2,3 and changes in business practices (e.g., the grant review process). NIH emphasizes that—like randomization, blinding, and other basic principles—appropriate accounting for the potential influence of sex on outcomes in preclinical research is a crucial component of rigorous experimental design.⁴

The pervasive effects of sex on biological functioning—and the need to consider sex for research to be reproducible and relevant for all people—warranted a separate NIH policy. Beginning January 2016, NIH has expected all applicants proposing to carry out studies in vertebrate animals and humans: (1) to factor sex as a biological variable (SABV) into research designs, analyses, and reporting or (2) to provide strong justification for single-sex investigations.⁵ Additional NIH guidance describes how SABV can be considered in the various stages of research.⁶

Applying SABV in biomedical research is the first step toward individualized medicine because sex profoundly influences health and disease across bodily systems and medical disciplines.^7

–10 Therefore, accounting for the potential influence of sex in research design, data analyses, and reporting (e.g., disaggregating results by sex) is crucial to rigorous and transparent science. Owing to the historical focus on male research animals and cells and the ongoing underreporting of sex in preclinical research, however, science still needs to better identify the health-relevant differences between males and females across each level of biological complexity^11

–16—building a more complete and solid foundation of knowledge that makes clinical and translational studies more relevant to all people.

SABV: Where Are We Now?

The SABV Policy complements and extends NIH's long-standing requirement, as per the 1993 NIH Revitalization Act, for the appropriate participation of women and underrepresented minorities in clinical studies. More than two decades after its enactment, we see that the NIH inclusion policy has been generally effective. Roughly half or more of the research participants in NIH-funded clinical studies are now women.^17,18 Nevertheless, there is ample scope for improving the extent to which scientists integrate sex (a biological variable defined genetically) and gender (a person's self-identity as expressed by choices and influenced by social, cultural, and psychological factors) into the design, analyses, and reporting of clinical research.^19
–21

Geller et al. reviewed NIH-funded randomized controlled trial reports published in 14 leading U.S. medical journals in 2015. They found that fewer than a third of the reports analyzed data by sex, included sex in their statistical analyses, or provided an explanation for not doing so.²² The combined proportion of studies reporting sex-based analyses or explaining why such analyses were not done was no greater than the proportions of such studies published in 2004 or 2009.^23,24

Another group of researchers at the Allen Institute for Artificial Intelligence recently carried out a much broader analysis of clinical studies, which were funded by NIH and other public and private entities around the world.²⁵ That group found significant overall underrepresentation of female participation in clinical research of diseases relative to the prevalence of those diseases among females, even in the most recent time period examined (2014–2018). The investigators observed substantial female underrepresentation for 7 of 11 disease categories, with no evidence of reduction in “enrollment sex bias” over time for digestive diseases, hepatitis, and chronic kidney diseases.²⁵

In alignment with the 21st Century Cures Act, a recent amendment²⁶ to NIH's inclusion policy²⁷ requires applicable Phase III clinical trials to report results of valid analyses by sex/gender, race, and ethnicity in Clinicaltrials.gov. In general, applicable clinical trials are those that investigate therapeutics, biologics, and devices regulated by the U.S. Food and Drug Administration (FDA).²⁶ Such reporting is essential to improving health care and supporting scientists as they generate research questions, observe outcomes, and analyze data. The amended inclusion policy went into effect for all new competing grants and cooperative agreements awarded on or after December 13, 2017. The full impact of the amended policy on the reporting of research findings will not be seen until enough time has passed for the completion of large-scale clinical trials that began enrolling participants in the 2 years since the new policy went into effect.

Also, in alignment with the 21st Century Cures Act, the NIH implemented the Inclusion Across the Lifespan (IAL) Policy,²⁸ which expands the Inclusion of Children in Clinical Research Policy²⁹ such that it now includes individuals of all ages. The IAL Policy also requires that the age of enrollment of each participant in clinical research be collected and reported. This policy went into effect for all grant applications with due dates on or after January 25, 2019, as well as all solicitations for Research and Development contracts issued on or after that date.²⁸

The NIH is also enhancing the transparency and accountability with which it monitors the inclusion of participants in clinical research by sex/gender, race, and ethnicity. In response to the Government Accountability Office's recommendations,³⁰ and as part of 21st Century Cures Act implementation, NIH recently began providing its inclusion data on sex/gender, race, and ethnicity in a format that is disaggregated by Research, Condition, and Disease Categorization (RCDC) categories. The inclusion data may be viewed and downloaded on the new NIH RCDC Inclusion Statistics Report webpage.³¹

The ability to analyze NIH investments more granularly is helping to ensure that women are appropriately involved in clinical research in each research, condition, and disease category, including categories of particular concern historically (e.g., clinical trials in certain areas of cardiology,³² among others).

NIH Efforts on Multiple Fronts

The consideration of both sexes of animals and cells in preclinical research had generally not advanced at the same pace as the inclusion of women in clinical studies. In this context, NIH promulgated its SABV Policy as part of the fabric of advancing rigorous research and achieving optimal stewardship of taxpayer investments.

A little over 5 years ago, NIH first announced intentions to address insufficient consideration of SABV, described the rationale for policy development, and outlined steps of a multidimensional approach to implementation.³³ Soon thereafter, the NIH Office of Research on Women's Health (ORWH) began soliciting feedback on the concept from NIH stakeholders and the broader scientific community, beginning with a request for information on the consideration of SABV, released in September 2014.³⁴ The same month, ORWH established the Trans-NIH SABV Working Group, which was then co-chaired by the Director of ORWH and the Director of the NIH Office of Extramural Research and composed of senior staff of the NIH Institutes and Center (ICs) who are nominated by their respective IC directors. The Trans-NIH SABV Working Group was formed to support and inform development and implementation of the SABV Policy.

Moreover, in October 2014, ORWH convened a Methods and Techniques workshop that brought together key stakeholders to discuss methods, experimental design considerations, and statistical approaches to include male and female animals, cells, and tissues in preclinical research, analyze sex influences on outcomes, and consider gender as an additional influence on health and disease. One outcome of that workshop was an article published by Miller et al., which includes case studies demonstrating exemplar strategies for applying SABV to preclinical investigations.³⁵ Based on all the feedback received from NIH stakeholders and the broader scientific community through the aforementioned activities, ORWH has taken action on several fronts to advance the consideration of sex and gender in biomedical research.

For example, ORWH and the FDA have jointly supported the training of scientists across the biomedical research continuum with a series of online courses and resources on the biological basis of sex- and gender-related differences and their influences on health and disease. These courses have had good uptake and positive impact on a large number of participants.³⁶

The newest educational resource from ORWH and FDA's Office of Women's Health is called Bench-to-Bedside: Integrating Sex and Gender to Improve Human Health,³⁷ which is scheduled for release in 2020. This course contains modules that incorporate the application of SABV to various clinical topic areas, such as immunology, cardiovascular medicine, and pulmonology. The goal of this interprofessional educational course is for learners, including clinician researchers from various disciplines, to gain an understanding of the influences of sex and gender on health and disease that they can apply when conducting research and interpreting and applying evidence for clinical practice.

In addition, ORWH and the National Institute of General Medical Sciences have collaborated on the SABV Primer³⁷ to highlight the importance of considering SABV in preclinical research as well as across the entire biomedical research spectrum. The SABV Primer consists of four interactive modules titled: SABV and the Health of Women and Men; SABV and Experimental Design; SABV and Analyses; and SABV and Research Reporting. Also scheduled for release in 2020, this resource will help biomedical researchers, including academics and students, with the incorporation of SABV into the design of research studies, the preparation of NIH grant applications, and the education of the next generation of investigators.

Recognizing that a catalyst was needed to stimulate research considering sex and gender influences, ORWH launched the Sex/Gender Administrative Supplement program in 2013. Under this program, investigators with ongoing NIH-funded grants could apply for an Administrative Supplement to explore sex and gender influences within the scientific scope of their original award.

In 2014, the Common Fund, which is part of the NIH Office of the Director (OD) and supports some of the most innovative and integrative biomedical research funded by NIH, also adopted the Sex/Gender Administrative Supplement paradigm. Subsequently, ORWH and the Common Fund collaborated to convene the Sex As a Biological Variable Workshop in October 2017, at which NIH grantees from a variety of disciplines shared the approaches they had used to consider SABV in 16 different investigations across a wide range of topic areas. As of the close of fiscal year 2019, ORWH has invested $37 million to make over 350 Sex/Gender Administrative Supplement awards across 20 ICs and the OD.

ORWH also partnered with several ICs to co-fund the Specialized Centers of Research Excellence (SCORE) on Sex Differences program. Initiated in 2003 as the Specialized Centers of Research (SCOR) on Sex Differences program with support from FDA, this disease-agnostic program is the only NIH center program on sex differences. Funded centers model how biomedical scientists can apply SABV in their study of major medical conditions in an interdisciplinary and integrative way. The interdisciplinary collaborations exemplified by SCORs bridge basic and clinical research on sex differences and major medical conditions affecting women.

Now, as SCOREs, these centers are centers of excellence. The centers focus on training researchers in experimental design and analyses that consider sex and/or gender, meeting the career enhancement needs of translational scientists. Under the SCOR/E on Sex Differences Program, NIH has invested more than $160 million in centers at more than 25 institutions since 2003. SCOR/E awards have been made in collaboration with seven ICs in support of a broad diversity of topics in women's health and sex difference research, including chronic pain; addiction; female urinary tract and reproductive organs; infectious diseases and immunity; metabolic disorders; age-related cognitive decline; and mental health.

To enhance implementation of the SABV Policy, NIH has also made changes to its forms and processes and updated its guidance to applicants and peer reviewers.³⁸ There is no one-size-fits-all approach to evaluating science for SABV integration. NIH peer review guidance encourages assessment in the context of current knowledge of sex and gender influences in a given proposal's field of study, as well as in light of available methods and other relevant considerations.

A survey of NIH standing study section and special emphasis panel members, conducted independently by concerned members of the scientific community, found that a majority (54% and 58% in 2016 and 2017, respectively) thought the SABV Policy would improve rigor and reproducibility.³⁹ Respondents also thought that the proportion of grants appropriately addressing the SABV Policy had increased over time. Although a majority of respondents (55% and 61%, respectively) indicated that the consideration of SABV was consistently factored into the reviewers' scoring of applicants' “approaches,” open-ended comments revealed a variety of views on this aspect of the grant review process.³⁹ Further study is warranted to evaluate the uptake of the SABV Policy, which could help guide efforts to further enhance policy implementation.

The integration of SABV into biomedical research cannot rely solely on NIH policy, but also depends on a critical group of stakeholders—editors of biomedical journals, who are key gatekeepers of scientific knowledge. Editors of science journals have traditionally overlooked the consideration of sex and gender in selecting articles. Yet, one international standard—the Sex and Gender Equity in Research (SAGER) guidelines—promotes systematic reporting of sex and gender across biomedical research disciplines and helps authors and editors determine whether articles have appropriately considered these variables.⁴⁰

Several journals have adopted SAGER guidelines or similar guidelines^41,42 for reporting the sex of research animals and of cells cultured from animals in scientific publications.^15,43 Some journals have been more proactive on the consideration of SABV by encouraging submissions that include meaningful statistical comparisons between sexes⁴⁴ or requiring sex to be included in experimental design and analyses, whenever doing so is relevant to the scientific question being addressed.⁴⁵ Despite these laudable steps by certain journals, more work remains to be done by the science publishing industry, in general, to set higher expectations for the consideration of sex and gender in the conduct and reporting of biomedical research.

Application of SABV to the Science

ORWH created and chairs the NIH Scientific Interest Group (SIG) on Sex and Gender in Health and Disease (SGHD). This SIG aims to expand the understanding of NIH intramural scientists as well as researchers in the extramural scientific community about the range of ways that SABV applies across scientific disciplines. The SGHD SIG promotes the dissemination of research on sex and gender influences in health and disease, and it fosters potential interdisciplinary collaborations on these topics by bringing together interested scientists.

NIH ICs have also undertaken initiatives to underscore the importance of sex- and gender-based research and disseminate the results of such research. In July 2017, for example, the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) convened a 2-day workshop, titled Sex and the Kidneys: Sex Differences in Renal Disease. The workshop highlighted NIDDK's application of SABV to renal research. In addition, the workshop afforded the research community an opportunity to come together to share information about the role of sex in renal disease risk and etiology, the action of sex steroids on renal tissues, and the role of sex chromosomes in renal disease pathophysiology.

NIH ICs take a flexible approach to integrating sex and gender into biomedical research based on their disciplines' unique considerations and research methodologies, as well as the health conditions within their purviews. The Director of the National Institute on Alcohol Abuse and Alcoholism co-authored a review article on ways that addiction researchers can consider and investigate sex differences in preclinical animal models—offering a standard for the identification of such models and the transparent reporting of their limitations.⁴⁶

The National Institute on Aging's Interventions Testing Program examines various potential medications for their ability to extend lifespan and delay disease in male and female mice. Advances arising from this program illustrate how integrating SABV into preclinical research apprises investigators about the applicability of results across sexes and the existence of potential sex influences on research outcomes. Results informed in this manner comprise a valuable preclinical foundation of knowledge for the design and conduct of clinical investigations that are critical to bolstering the development of effective medications and confirming their safety for everyone. For example, researchers have found that 17-α estradiol and the antidiabetic drug, acarbose, each extend lifespan and improve glucose tolerance in male mice, but not female mice, and that alterations in the relevant metabolic pathways occurred in opposite directions in males and females, an effect that is influenced by gonadal hormones.⁴⁷

Trans-NIH research initiatives are also capitalizing on the power of SABV. In the Knockout Mouse Phenotyping Program (KOMP2), which is a major source of funding for the International Mouse Phenotyping Consortium (IMPC), researchers who applied SABV to analyses of knockout mouse strains, with a single gene deletion each, have opened new opportunities to improve the precision of animal models. One group of IMPC researchers found that 10% of all categorical traits and 57% of all continuously distributed traits in wild-type mice were significantly influenced by sex.⁹ These sex influences were found to be distributed across a wide spectrum of traits and research fields. Of the traits with a significant genotype effect, 13% and 18% of categorical and continuous phenotypes, respectively, were classified as sexually dimorphic.⁹

Focusing on metabolic phenotypes in over 2000 knockout mouse strains, another group of IMPC researchers found that there are pronounced sex differences in many genes and pathways associated with metabolic traits.⁴⁸ The researchers also found that some genes from different metabolic pathways cause comparable metabolic phenotypes in females and males.⁴⁸

Through its Genotype-Tissue Expression (GTEx) project, NIH has invested in the establishment of a data resource and tissue bank to build knowledge about the relationships between genetic variants and gene expression among different tissues and in different groups of people (e.g., males vs. females). The GTEx project is advancing our understanding of gene regulation mechanisms and how perturbations in gene expression relate to human diseases, which is foundational information for personalized medicine.

Using GTEx data, one group of researchers quantified large numbers of genes with sex-differential expression in tissues common to males and females, with the highest numbers of such genes found in breast mammary gland, skeletal muscle, adipose tissue, skin, brain anterior cingulate cortex, and heart left ventricle.⁸ Another group of GTEx researchers found that most autosomal genes with tissue-specific sex differences (TSSDs) showed discordant directions of sex differences across tissues, whereas genes with TSSDs on the X chromosome were more often concordant in terms of the direction of sex differences across tissues (i.e., consistently higher expression in one sex across many tissues).⁴⁹

NIH is trying to accelerate the integration of SABV into standard scientific practice as part of its efforts to achieve optimal stewardship of taxpayer investments throughout the research enterprise. Incorporating SABV into a research program or discipline pays dividends by advancing our understanding of how sex affects biology, health, and disease, as well as by spurring opportunities for improved medical practice.

Research from the SCOR/E program illustrates how applying SABV can result in new discoveries that are highly relevant to human health. A previously funded Center identified how maternal stress during pregnancy disrupts the placenta—transmitting adverse maternal experiences and environments to offspring and altering fetal brain development in a sex-dependent manner—and how this contributes to males' vulnerability to neurodevelopmental disorders.^50,51 The investigators have used insights from basic research that applied SABV to develop a phenotypically relevant mouse model that researchers can use to study male sensitivity to insults in utero as well as female neurodevelopmental resilience,⁵² with the promise of achieving a better understanding of the origins of sex biases in the prevalence and presentation of psychiatric disorders.

SABV: A Guiding Principle for Addressing Biomedical Research Challenges

Using SABV as a guiding principle throughout the research continuum can help address key issues affecting the biomedical research enterprise, including (1) the need to identify animal models that better reflect human diseases^46,53,54; (2) concerns about the increased risk of adverse events or reduced treatment effectiveness in women^55,56; and (3) the importance of developing diagnostic tests and criteria that are sex and gender aware,^57,58 among other issues. Addressing these challenges offers opportunities to improve biomedical research and lays the foundation for high-quality and safe medical care for everyone.

As we see it, clinical and translational research do not have a solid foundation unless SABV is applied seamlessly across all phases of research, including basic and preclinical studies. NIH's vision is for each stage of research to inform the subsequent ones about the influences of sex and gender, thereby improving scientific rigor, transparency, and return on taxpayer investments.

In working with animal models, for example, researchers studying the mechanisms underlying type 2 diabetes have applied SABV to reveal multiple, crucial discoveries about sex differences in this condition. This work has described the role of testosterone in diabetes and metabolic health⁵⁹ and identified estrogen as a protective factor in type 2 diabetes.^60,61

Furthermore, the higher risk of adverse drug events among women can be considered in the context of females' historical underrepresentation in clinical research. Of the 10 medications withdrawn from the U.S. market between 1997 and 2000, eight posed greater health risks for women than men.⁶² An analysis of materials submitted to the FDA in support of marketing applications for 36 drug approvals for cardiovascular disease (CVD) between 2005 and 2015 found that the mean proportion of women enrolled in these trials was 46%, but the overall range was wide (22%–81%).⁶³ Women were appropriately represented in clinical trials for some CVDs (e.g., atrial fibrillation and hypertension), but not others (e.g., heart failure, coronary artery disease, and acute coronary syndrome).⁶³

Integrating SABV into basic and preclinical research to identify potential differences in drug safety is far more efficient than discovering them during clinical trials or in postmarketing surveillance. Investments in preclinical research that consider SABV may also help avoid differential effectiveness outcomes—for example, poorer treatment outcomes among women for disulfiram, an FDA-approved medication for alcohol abuse that has been tested as a pharmacotherapy for cocaine abuse.⁶⁴

How Do We Get There?

A year ago, the NIH released Advancing Science for the Health of Women: The 2019–2023 Trans-NIH Strategic Plan for Women's Health Research.⁶⁵ The strategic plan incorporates the missions of the ICs with ORWH's mission to chart a course toward NIH's vision: Imagine a world in which the biomedical research enterprise thoroughly integrates sex and gender influences; every woman receives evidence-based disease prevention and treatment tailored to her own needs, circumstances, and goals; and women in science careers reach their full potential.⁶⁵

To complement its other efforts to incorporate SABV into the biomedical research enterprise, NIH recently released a funding opportunity announcement (FOA) titled The Intersection of Sex and Gender Influences on Health and Disease for research project grant (R01) applications.⁶⁶ This FOA is NIH's first investigator-initiated R01 on the influence and intersection of SGHD. The FOA encourages research across many scientific disciplines. Proposed investigations must include both sex- and gender-related variables and also address at least one of the five objectives from Strategic Goal 1 of the 2019–2023 Trans-NIH Strategic Plan for Women's Health Research, which is to advance rigorous research that is relevant to the health of women. The first year's applications were just received. Future application due dates are November 25, 2020, and November 26, 2021.

The integration of sex and gender considerations throughout the research enterprise and the health care advancements fueled by that enterprise require broad involvement in and end-to-end stewardship of this endeavor. We call on all NIH ICs and the NIH's partners to advance application of SABV to all areas of science, as appropriate, by adding sessions at meetings, through focused attention in FOAs as scientifically relevant, and in training the next generation of researchers. Individuals who lead other funding organizations and oversee the governance of research and clinical enterprises, those who head up regulatory agencies, and editors of peer-reviewed journals also should incorporate the consideration of SABV into their organizations' roles in the biomedical research enterprise.

Scientists themselves—whether working in academia, medicine, industry, or scientific associations and societies—have an important responsibility to champion the integration of SABV throughout biomedical research. Patients also have a role to play in asking their health care providers how well diagnostic tests and treatments work in people like themselves, thereby driving the demand for sex- and gender-based medicine. Only by making systematic attention to sex and gender part of business as usual can we advance the NIH mission of turning discovery into health for both women and men.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding has been received for this article.

References

Roth

, Cox

. Science isn't science if it isn't reproducible. Am J Pathol, 2015; 185:2–3.

National Institutes of Health. NOT-OD-15-103: Enhancing Reproducibility through Rigor and Transparency. 2015. Available at: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-15-103.html Accessed January 10, 2020 .

National Institutes of Health. NOT-OD-16-011: Implementing Rigor and Transparency in NIH & AHRQ Research Grant Applications. 2015. Available at: https://grants.nih.gov/grants/guide/notice-files/not-od-16-011.html Accessed January 10, 2020 .

Collins

, Tabak

. Policy: NIH plans to enhance reproducibility. Nature, 2014; 505:612–613.

National Institutes of Health. NOT-OD-15-102: Consideration of Sex as a Biological Variable in NIH-funded Research. 2015. Available at: https://grants.nih.gov/grants/guide/notice-files/not-od-15-102.html Accessed January 10, 2020 .

National Institutes of Health. Consideration of Sex as a Biological Variable in NIH-funded Research (additional guidance for NOT-OD-15-102). 2015. Available at: https://orwh.od.nih.gov/sites/orwh/files/docs/NOT-OD-15-102_Guidance.pdf Accessed January 10, 2020 .

Neigh

, Mitzelfelt

. Sex differences in physiology. San Diego: Academic Press, 2016:236.

Gershoni

, Pietrokovski

. The landscape of sex-differential transcriptome and its consequent selection in human adults. BMC Biol, 2017; 15:7.

Karp

, Mason

, Beaudet

, et al. Prevalence of sexual dimorphism in mammalian phenotypic traits. Nat Commun, 2017; 8:15475.

10.

Westergaard

, Moseley

, Hemmingsen Sørup

, Baldi

, Brunak

. Population-wide analysis of differences in disease progression patterns in men and women. Nat Commun, 2019; 10:666.

11.

Beery

, Zucker

. Sex bias in neuroscience and biomedical research. Neurosci Biobehav Rev, 2011; 35:565–572.

12.

Taylor

, Vallejo-Giraldo

, Schaible

, Zakeri

, Miller

. Reporting of sex as a variable in cardiovascular studies using cultured cells. Biol Sex Differ, 2011; 2:11.

13.

Shah

, McCormack

, Bradbury

. Do you know the sex of your cells?. Am J Physiol Cell Physiol, 2014; 306:C3–C18.

14.

Yoon

, Mansukhani

, Stubbs

, Helenowski

, Woodruff

, Kibbe

. Sex bias exists in basic science and translational surgical research. Surgery, 2014; 156:508–516.

15.

Potluri

, Engle

, Fink

, vom Steeg

, Klein

. Sex reporting in preclinical microbiological and immunological research. MBio, 2017; 8:e01868-01817.

16.

Will

, Proaño

, Thomas

, et al. Problems and progress regarding sex bias and omission in neuroscience research. eNeuro, 2017; 4:e0278-0217.2017.

17.

National Institutes of Health Advisory Committee on Research on Women's Health. Report of the Advisory Committee on Research on Women's Health, Fiscal Years 2015–2016. Rockville, MD: National Institutes of Health, 2017.

18.

National Institutes of Health Advisory Committee on Research on Women's Health. Report of the Advisory Committee on Research on Women's Health: Fiscal Years 2017–2018. Rockville, MD: National Institutes of Health, 2019.

19.

IOM (Institute of Medicine) Committee on Understanding the Biology of Sex and Gender Differences. Exploring the Biological Contributions to Human Health: Does Sex Matter?. Washington, DC: National Academy Press, 2001.

20.

Clayton

. Studying both sexes: A guiding principle for biomedicine. FASEB J, 2016; 30:519–524.

21.

Tannenbaum

, Ellis

, Eyssel

, Zou

, Schiebinger

. Sex and gender analysis improves science and engineering. Nature, 2019; 575:137–146.

22.

Geller

, Koch

, Roesch

, Filut

, Hallgren

, Carnes

. The more things change, the more they stay the same: A study to evaluate compliance with inclusion and assessment of women and minorities in randomized controlled trials. Acad Med, 2018; 93:630–635.

23.

Geller

, Goldstein Adams

, Carnes

. Adherence to federal guidelines for reporting of sex and race/ethnicity in clinical trials. J Womens Health (Larchmt), 2006; 15:1123–1131.

24.

Geller

, Koch

, Pellettieri

, Carnes

. Inclusion, analysis, and reporting of sex and race/ethnicity in clinical trials: Have we made progress?. J Womens Health (Larchmt), 2011; 20:315–320.

25.

Feldman

, Ammar

, Lo

, Trepman

, van Zuylen

, Etzioni

. Quantifying sex bias in clinical studies at scale with automated data extraction. JAMA Netw Open, 2019; 2:e196700.

26.

National Institutes of Health. NOT-OD-18-014: Amendment: NIH Policy and Guidelines on the Inclusion of Women and Minorities as Subjects in Clinical Research. 2017. Available at: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-18-014.html Accessed January 10, 2020 .

27.

National Institutes of Health. NOT-OD-02-001: NIH Policy and Guidelines on the Inclusion of Women and Minorities as Subjects in Clinical Research. 2001. Available at: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-02-001.html Accessed January 10, 2020 .

28.

National Institutes of Health. NOT-OD-18-116: Revision: NIH Policy and Guidelines on the Inclusion of Individuals Across the Lifespan as Participants in Research Involving Human Subjects. 2017. Available at: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-18-116.html Accessed January 10, 2020 .

29.

National Institutes of Health. NOT-98-024: NIH Policy and Guidelines on the Inclusion of Children as Participants in Research Involving Human Subjects. 1998. Available at: https://grants.nih.gov/grants/guide/notice-files/not98-024.html Accessed January 10, 2020 .

30.

U.S. Government Accountability Office. National Institutes of Health: Better Oversight Needed to Help Ensure Continued Progress Including Women in Health Research. Washington, D.C.: United States Government Accountability Office, 2015.

31.

National Institutes of Health. NIH RCDC Inclusion Statistics Report. 2019. Available at: https://report.nih.gov/risr/# Accessed November 19, 2019 .

32.

Clayton

, Arnegard

. Taking cardiology clinical trials to the next level: A call to action. Clin Cardiol, 2018; 41:179–184.

33.

Clayton

, Collins

. Policy: NIH to balance sex in cell and animal studies. Nature, 2014; 509:282–283.

34.

National Institutes of Health. NOT-OD-14-128: Request for Information (RFI): Consideration of Sex As a Biological Variable in Biomedical Research. 2014. Available at: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-14-128.html Accessed January 10, 2020 .

35.

Miller

, Marks

, Becker

, et al. Considering sex as a biological variable in preclinical research. FASEB J, 2017; 31:29–34.

36.

Plank-Bazinet

, Sampson

, Miller

, et al. The science of sex and gender in human health: Online courses to create a foundation for sex and gender accountability in biomedical research and treatment. Biol Sex Differ, 2016; 7:47.

37.

NIH Office of Research on Women's Health. Career Development & Interprofessional Education: E-Learning. 2019. Available at: https://orwh.od.nih.gov/career-development-education/e-learning Accessed January 10, 2020 .

38.

National Institutes of Health. Reviewer Guidance to Evaluate Sex as a Biological Variable (SABV). 2017. Available at: https://grants.nih.gov/grants/peer/guidelines_general/sabv_decision_tree_for_reviewers.pdf Accessed November 19, 2019 .

39.

Woitowich

, Woodruff

. Implementation of the NIH sex-inclusion policy: Attitudes and opinions of study section members. J Womens Health (Larchmt), 2019; 28:9–16.

40.

Heidari

, Babor

, De Castro

, Tort

, Curno

. Sex and gender equity in research: Rationale for the SAGER guidelines and recommended use. Res Integr Peer Rev, 2016; 1:2.

41.

IOM (Institute of Medicine). Sex-specific Reporting of Scientific. Research: A Workshop Summary. Washington, DC: The National Academies Press, 2012.

42.

Kilkenny

, Browne

, Cuthill

, Emerson

, Altman

. Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biol, 2010; 8:e1000412.

43.

Schiebinger

, Klinge

, Sánchez de Madariaga

, Paik

, Schraudner

, Stefanick

. Gendered Innovations in Science, Health & Medicine, Engineering and Environment: Sex and Gender Analysis Policies of Peer-Reviewed Journals. 2011–2019. Available at: https://genderedinnovations.stanford.edu/sex-and-gender-analysis-policies-peer-reviewed-journals.html Accessed November 4, 2019 .

44.

Prager

. Addressing sex as a biological variable. J Neurosci Res, 2017; 95:11.

45.

Docherty

, Stanford

, Panattieri

, et al. Sex: A change in our guidelines to authors to ensure that this is no longer an ignored experimental variable. Br J Pharmacol, 2019; 176; 4081–4086.

46.

Becker

, Koob

. Sex differences in animal models: Focus on addiction. Pharmacol Rev, 2016; 68:242–263.

47.

Garratt

, Bower

, Garcia

, Miller

. Sex differences in lifespan extension with acarbose and 17-α estradiol: Gonadal hormones underlie male-specific improvements in glucose tolerance and mTORC2 signaling. Aging Cell, 2017; 16:1256–1266.

48.

Rozman

, Rathkolb

, Oestereicher

, et al. Identification of genetic elements in metabolism by high-throughput mouse phenotyping. Nat Commun, 2018; 9:288.

49.

Kassam

, Wu

, Yang

, Visscher

, McRae

. Tissue-specific sex differences in human gene expression. Hum Mol Genet, 2019; 28:2976–2986.

50.

Howerton

, Bale

. Targeted placental deletion of OGT recapitulates the prenatal stress phenotype including hypothalamic mitochondrial dysfunction. Proc Natl Acad U S A, 2014; 111:9639–9644.

51.

Bronson

, Bale

. The placenta as a mediator of stress effects on neurodevelopmental reprogramming. Neuropsychopharmacology, 2016; 41:207–218.

52.

Nugent

, O'Donnell

, Epperson

, Bale

. Placental H3K27me3 establishes female resilience to prenatal insults. Nat Commun, 2018; 9:2555.

53.

Klauss

, Schorn

, Teller

, et al. Genetically induced vs. classical animal models of chronic pancreatitis: A critical comparison. FASEB J, 2018; 32:5778–5792.

54.

Kokras

, Dalla

. Sex differences in animal models of psychiatric disorders. Br J Pharmacol, 2014; 171:4595–4619.

55.

Parekh

, Fadiran

, Uhl

, Throckmorton

. Adverse effects in women: Implications for drug development and regulatory policies. Expert Rev Clin Pharmacol, 2011; 4:453–466.

56.

Carey

, Nader

, Chai

, Carreiro

, Griswold

, Boyle

. Drugs and medical devices: Adverse events and the impact on women's health. Clin Ther, 2017; 39:10–22.

57.

Petitto

, Esposito

, Sorrentino

, et al. Sex-specific echocardiographic reference values: The women's point of view. J Cardiovasc Med, 2018; 19:527–535.

58.

Lundström

, Mårland

, Kuja-Halkola

, et al. Assessing autism in females: The importance of a sex-specific comparison. Psychiatry Res, 2019; 282:112566.

59.

Morford

, Wu

, Mauvais-Jarvis

. The impact of androgen actions in neurons on metabolic health and disease. Mol Cell Endocrinol, 2018; 465:92–102.

60.

, Allard

, Alvarez-Mercado

, et al. Estrogens promote misfolded proinsulin degradation to protect insulin production and delay diabetes. Cell Rep, 2018; 24:181–196.

61.

Zhou

, Ribas

, Rajbhandari

, et al. Estrogen receptor α protects pancreatic β-cells from apoptosis by preserving mitochondrial function and suppressing endoplasmicreticulum stress. J Biol Chem, 2018; 293:4735–4751.

62.

U.S. General Accounting Office. Drug Safety: Most Drugs Withdrawn in Recent Years Had Greater Health Risks for Women. Washington, D.C.: United States General Accounting Office, 2001.

63.

Scott

, Unger

, Jenkins

, et al. Participation of women in clinical trials supporting FDA approval of cardiovascular drugs. J Am Coll Cardiol, 2018; 71:1960–1969.

64.

DeVito

, Babuscio

, Nich

, Ball

, Carroll

. Gender differences in clinical outcomes for cocaine dependence: Randomized clinical trials of behavioral therapy and disulfiram. Drug Alcohol Depend, 2014; 145:156–167.

65.

National Institutes of Health. Advancing Science for the Health of Women: The 2019–2023 Trans-NIH Strategic Plan for Women's Health Research (Available at: https://orwh.od.nih.gov/sites/orwh/files/docs/ORWH_Strategic_Plan_2019_02_21_19_V2_508C.pdf). Bethesda, MD: National Institutes of Health, 2019.

66.

National Institutes of Health. RFA-OD-19-029: The Intersection of Sex and Gender Influences on Health and Disease (R01 Clinical Trial Optional). 2019. Available at: https://grants.nih.gov/grants/guide/rfa-files/RFA-OD-19-029.html Accessed January 10, 2020 .