Educating the Scientific Workforce on Sex and Gender Considerations in Research: A National Scan of the Literature and Building Interdisciplinary Research Careers in Women's Health Programs

Abstract

Background:

This study was a national scan of education resources on integrating sex and gender considerations into research. The purpose was to assess capacity for educating researchers and to identify gaps, with implications for implementation of guidelines or mandates to consider sex and gender differences in research. Information sources were U.S. training programs in women's health and sex/gender difference research, Building Interdisciplinary Research Careers in Women's Health (BIRCWH), and published peer-reviewed biomedical literature.

Materials and Methods:

This descriptive study used multiple methods: a national survey and a comprehensive literature review. BIRCWH leaders responded to a survey regarding education on sex/gender difference research for BIRCWH scholars (response rate 100%, 20 of 20). A comprehensive literature review was conducted for 1993–2018.

Results:

Nearly half (45%) of BIRCWH institutions offered education on integrating sex or gender differences in clinical translational research; of those, roughly half (54%) offered in-person training and one-third (31%) offered content within existing for-credit courses. Respondents preferred online training (84%) to in-person offerings or reference materials (47% and 42% respectively). Published indexed literature on sex or gender differences has quadrupled since 1993, although growth in these publications remained flat in the most recent six years.

Conclusions:

Published resources to educate researchers on integrating sex and gender differences into medical research have increased, and BIRCWH programs connect scholars to national resources. Educational gaps remain due to limited access to curricula on applied research approaches, design, and methods for sex/gender difference research. BIRCWH programs desire curricula that are easily accessible online and asynchronously; sanctioned and supported by national thought leaders; linked to required training such as rigor and reproducibility; foster collaboration; and offer practical applications. Evidence-based, high-quality educational curricula and a dissemination plan are needed to enhance the adoption and integration of sex and gender into scientific endeavors.

Introduction

Major regulatory changes from the National Institutes of Health (NIH) over the last 25 years have generated significant movement toward integrating sex and gender considerations into medical research. In 1993, the NIH Revitalization Act required that women as well as men be included in clinical studies, and in 2014, NIH leaders called for sex differences to be addressed in basic and preclinical studies to enhance replicability, generalizability, and scientific impact,^1,2 culminating in a requirement that NIH grant proposals address the inclusion of sex as a biological variable (SABV), for studies with vertebrate animals and humans (NOT-OD-15-102; NOT-OD-15-103).

Gender differences must also be considered. Although the word “gender” is not explicitly contained within the term “SABV,” gender is clearly considered a key component; SABV is often used in colloquial language to connote the full spectrum of sex and gender difference research. In this way, SABV is consistent with international Sex and Gender Equity in Research (SAGER) guidelines,³ echoing international thought leaders who broadly address unbalanced participation of women and men in clinical research studies by incorporating consideration of sex and gender differences into all research.

The goal of integrating these considerations into all medical research is to generate efficacy, effectiveness, and safety evidence that is pertinent to all people, and specifically to avoid adverse outcomes in women as a result of underappreciated sex differences such as those found in physiology and metabolism. In addition, such outcomes can also result from gender differences that may lead to treatment disparities.

Education and training are critical to disseminate and implement sex and gender considerations in medical research as practiced by the scientific workforce, including early career researchers and experienced scientific reviewers. This kind of scientific workforce preparedness is low at this point, as institutional ethics committees, editorial boards, and review groups apply varying standards for integrating sex and gender into medical research. This overarching training gap must be filled with universal and accessible curricula that are effectively disseminated to all learners.^4,5

We sought to assess the status of the current national capacity to educate and train researchers pursuing careers in externally funded women's health as well as sex and gender difference research. We examined current practices and resources produced or curated by some of the key leading training programs focused in this scientific area, which are housed at major research universities across the country, that is, funded sites for NIH Building Interdisciplinary Research Careers in Women's Health (BIRCWH) K12 programs.⁶ We conducted a survey of all BIRCWH Principal Investigators nationally about current and planned education and training practices, and preferences for future resources.

We also conducted a literature review starting the year of the NIH Revitalization Act (1993) to assess subsequent growth in published and disseminated resources on sex- and gender-based medical research. We further searched among these articles for a focus on education and training (“education” generally denoting differences in degree-granting or credit-based learning, vs. “training” as noncredit continuing education), and specifically on research training to conduct sex and gender differences studies. This article reports findings from the national BIRCWH program survey as well as the comprehensive literature review, and makes recommendations for educating the scientific workforce about sex and gender considerations.

Materials and Methods

A national scan on resources for education/training (henceforth referred to as education) was conducted by obtaining survey responses from BIRCWH program sites, as they are federally funded national leaders in mentored research training in women's health and sex/gender difference research. In addition, because many university programs are local and not openly accessible, a literature review was conducted to assess the peer-reviewed information that would be available to people nationally and internationally outside of BIRCWH sites. Although there is a growing literature on sex/gender differences generally, with notable resources available on medical education and clinical assessment, we sought to identify content that would be more specifically used for research training.

Survey

A national survey was deployed in mid-2018 to Principal Investigators/Program Directors for BIRCWH sites. BIRCWH programs have been funded by the NIH Office for Research on Women's Health (ORWH) since 2000, and currently include 20 programs housed at research-based universities across the U.S. https://orwh.od.nih.gov/career-development/building-interdisciplinary-research-careers-womens-health-bircwh. These 20 programs served as the denominator for this survey. BIRCWH programs, through peer review, award institutional career development funding (K12) to early career female and male junior faculty researchers (called Scholars) with a goal of advancing careers in the area of women's health or sex-/gender-based difference research. All Scholars are required to have interdisciplinary mentoring teams.

Respondent program leaders' names and email contact information were obtained from the ORWH BIRCWH website, and an email was sent with an attached electronic survey asking for participation. Nonresponders were sent up to three email reminders from August to September 2018 for a final response rate of 100% (20/20). Respondent sites were as follows: University of California-Davis, University of Colorado, Duke University, Emory University, Brigham and Women's Hospital Harvard School of Medicine, Johns Hopkins University, University of Kentucky, Mayo Clinic College of Medicine and Science, McGee-Womens Research Institute University of Pittsburgh, University of Minnesota, University of North Carolina at Chapel Hill, Oregon Health and Science University, University of Pennsylvania, University of California-San Francisco, University of South Carolina, University of Texas Medical Branch, Tufts University, Tulane University, University of Utah, and Vanderbilt University.

The overall goal of the survey was to understand the landscape of education on sex and gender differences in biomedical and life science research at BIRCWH-funded institutions, whether produced locally or accessed from outside resources. The rationale for the survey was to identify overlaps, gaps, and opportunities for aiding dissemination and implementation of sex and gender equity in research. The survey comprised twelve items, including closed multiple choice and open-ended responses, and was estimated to take less than 10 minutes to complete (see Appendix A1 for survey).

Literature review

A comprehensive literature review using PubMed (pubmed.gov) and published search criteria was conducted.^7,8 PubMed is a free resource to access databases of citations in biomedical and life sciences field journal supported by the U.S. National Institutes of Health's National Library of Medicine (NIH/NLM) www.nlm.nih.gov/bsd/pubmed.html. Citations are indexed to MEDLINE and with MeSH^® subject headings, a hierarchical terminology for indexing and cataloging biomedical and life sciences publications www.nlm.nih.gov/mesh/meshhome.html. Published literature was reviewed for the years 1993–2018.

For the literature search, we combined free text and MeSH term searching PubMed sources. The search process was to first identify articles indexed in PubMed, which were flagged as addressing sex or gender differences. “Sex or Gender differences” search terms included the following: SABV, as well as sex and gender differences, for example, sex difference research, health status disparities, gender differences, gender difference research, gender bias, sex reporting, sex factor, sex distribution, sex characteristic, sex dimorphism, gender based, gender, sex, women, and disease in various configurations. The reported literature search used comprehensive search criteria posted online at the Laura W. Bush Institute for Women's Health.

Terms were combined using the Boolean search term “or” statement for broader searching. Within that article set, we then searched for articles addressing research design or analysis terms using terms labeled “Research,” or alternatively, teaching, training, or education-based articles using terms labeled “Education.” We searched our “Sex/Gender differences” articles using Boolean search term “and” with “Research” terms, and then with “Education” terms, and then with both “Research” and “Education” using the “and” Boolean search connector. We also reported articles indexed to “Sex/Gender difference,” but neither “Research” nor “Education.” Publication counts are reported over time and by category. (See Appendix Table A1 for search algorithm and example publications indexed to research and education, as well as a table for key web resources on sex and gender research and education, last updated January 28, 2020).

To cross-check the effectiveness of the above search criteria, we also implemented a more directive, snowball search strategy using articles that discussed the NIH SABV policy and articles they referenced. Articles were included if the primary focus was on gender differences (regardless of design) in health outcomes, gender bias, gender differences in preclinical research, gender differences in clinical research, gender differences in medicine, and gender differences in medical education. Study relevance was assessed using title, abstract, and full text content. Databases included PubMed, GenderMedDB, and Ovid MEDLINE. The NIH Office of Women's Health Research website and the Journal of Biology of Sex Differences website were manually searched. Articles were excluded if they were response/reply articles, opinions, commentaries, and articles not published in English.

Results

Among BIRCWH program respondents, one-quarter (25%, 5/20) require training on sex and gender difference research, while the majority (65%, 13/20) recommend Scholars identify specific relevant training that meets their individual needs. Among those that require training, one BIRCWH site asks Scholars to take the online NIH ORWH course on the Science of Sex and Gender in Human Health; other sites indicated that the topic arose informally in mentored training seminars on campus.

Figure 1 shows a division among the nearly half (45%) of BIRCWH institutions who offer education or training on sex- and gender-based difference research, with a small number (15%, 3/20) planning such offerings. Among those 13 sites who access SABV education, roughly half (54%, 7/13) offer in-person training, and another third (31%, 4/13) offer material either as part of existing for-credit coursework in study design or as online webinars (31%, 4/13). Open-ended responses were used to describe the nature of on-campus offerings, often described as leveraging the existing Clinical Translational Science Award training infrastructure, university-based centers for women's health research or health research and policy, or informally using an apprenticeship model through mentoring consultations.

FIG. 1.

Percent BIRCWH Program Institutions offering education/training on SABV/Sex/Gender difference research (one choice), and modality offered (multiple choices possible so percentages exceed 100%), 2018. BIRCWH, Building Interdisciplinary Research Careers in Women's Health; SABV, sex as a biological variable.

Among programs offering education on sex/gender, leaders indicated as many target learners for the majority of BIRCWH programming as applied (therefore totals exceed 100%). Target learners were reported to be early career faculty researchers (75%) and their mentors (50%), followed by medical students (UME, 25%) and practicing clinicians taking (CME, 25%), and finally for residents (GME, 17%) and community/general public (17%) (Fig. 2).

FIG. 2.

Percent BIRCWH Program Institutions offering education/training on SABV/Sex/Gender difference research by learner population, 2018. (Multiple choices possible so percentages exceed 100%).

In a list of options where respondents whose sites provided sex/gender education offerings (n = 12) indicated “as many as applied,” respondents endorsed topics covered in education, in decreasing order, as follows: clinical translational science/biomedical science (83%); evidence-based medicine, generalizability and subpopulations (75%); clinical teaching for patient presentation and diagnosis (50%), part of the larger body of personalized medicine (33%); and treatment effectiveness and safety (25%). In open-ended comments, respondents indicated that research design and analysis are the most common topics for existing education on sex and/or gender in basic, clinical, and outcomes research. Finally, some respondents emphasized that grantsmanship and professional development topics required training for advancing careers in women's health and sex/gender difference research.

Survey respondents described their preferences for the educational format or training modality with which sex/gender difference research are taught (Fig. 3). Nearly all respondent leaders (93% sites, not reported in Fig. 3) reported accessing online resources for training from the NIH ORWH, and therefore were aware of national, online resources (other online resources reported can be found in Appendix Tables A2 and A3). In Figure 3 (below), 42%–47% of respondents endorsed preferring in-person training at either conference gatherings or at their own institutions, or access to consensus reports or compilations such as special issues of journals. There was a clear contrast between online options—online trainings/webinars (84%) and massive open online courses (37%) using video were heavily endorsed as preferred formats versus less-used print formats such as textbooks/edited volumes and audio podcasts (16% each, respectively).

FIG. 3.

Percent BIRCWH Program Institutions expressed preference for format/modality for education/training on SABV/Sex/Gender difference research, 2018.

A comprehensive literature review supplemented the survey for a national scan on educational resources on women's health and sex/gender difference research. In Figure 4, publication frequency and time trends are reported in 5-year increments starting in 1993 and ending most recently in 2017. From 1993 to 1997, there were just over 4,000 peer-reviewed indexed publications in PubMed that addressed sex-/gender-based differences; of those, just over 300 address education or training and also research as keywords. In contrast, from 2013 to 2017, there were nearly 18,000 articles and >2,000 indexed as research education. The proportion of research education articles among the sex/gender articles increased >60% from 8% to 13%.

FIG. 4.

Frequency of Published, Indexed Articles on Sex or Gender and Research or Education over time in 5-year increments, 1993–2017.

The most recent 5-year period (2013–2017), plus 2018 (Fig. 5), shows fairly flat growth over 6 years in sex or gender differences published biomedical and life sciences articles, starting with 3,719 publications in 2013 and increasing to 4,208 in 2018. The annual proportion of articles that could be categorized as education, research, or research education in the areas of sex and/or gender was fairly stable over this time.

FIG. 5.

Frequency of Published, Indexed Articles on Sex or Gender and Research or Education, annually 2013–2018.

Discussion

This study was a national scan of education and training resources on integrating sex and/or gender considerations into medical research to assess the current capacity for educating the scientific workforce in these important areas. Conclusions were drawn from a national survey of BIRCWH program leaders, and from a comprehensive literature review for 1993–2018. The BIRCWH national survey showed that the BIRCWH Principal Investigators/Program Directors as a group were clearly dedicated to training and educating their early career Scholars and their mentors, and were connected to the NIH ORWH and their resources. They generally provided local, periodic, and occasional in-person educational opportunities relevant to sex and gender considerations at their institution; they also relied on shared national resources such as preconference training.

Although all BIRCWH programs support and mentor individual Scholars, only half of the programs required formal sex/gender research education for Scholars; the other half of programs did not require or provide such training, leaving a national training gap. The literature review showed a growth overall in peer-reviewed publications flagged for sex/gender differences over the past 20 years, yet the proportion of these articles dedicated to education generally, and to research education specifically, remained flat.

There were notable access barriers to effective training for women's health and sex/gender difference researchers nationally because some institutional training was only accessed as part of a for-credit course or was in person on an annual basis, thus limiting the ability for researchers to use the training when they need it. Respondent leaders stated preferences for webinars, video-based courses, and published compendia over in-person training, likely reflecting the need to scale the training for future Scholars, to leverage other sites and areas of expertise, and to make training asynchronous and therefore time independent.

BIRCWH program leaders expressed preferences for research curricula that are accessible and scalable, both online and asynchronous; standardized, sanctioned, and supported by national thought leaders; linked to mandated research training such as training about rigor and reproducibility; universal and multidisciplinary to emphasize collaborative opportunities; ideally able to foster connections to other sex and gender researchers for future collaborations; and being practical and applied, with example cases provided.

Institutional and published resources to educate the scientific workforce on integrating sex and gender differences into medical research have grown over time, and span the clinical translational spectrum from basic to preclinical to clinical to policy-based research, as well as spanning a wide variety of clinical domains from neuroscience and toxicology to metabolism and mental health. Likewise, the literature has grown and deepened in the areas of sex and gender bias and mitigation, SABV, as well as theoretical frameworks for understanding sex and gender differences.^1,9 Although there have been some standardized approaches to teaching sex and gender considerations in the clinical setting,^10,11 the same type of education for the research community has lagged, and agreed-upon competencies and markers of quality education are lacking. Major advances in measurement and methods for sex and gender difference research remain not well understood for people in the scientific workforce, who are most likely to evaluate and sanction these studies, that is, journal reviewers and editors, grant reviewers and program officers, mentors and graduate educators, and human subjects review committees.

Although this study included a complete national survey of all BIRCWH program leaders, questions were generally close ended, so were limited in scope of information provided. Therefore, what is yet unknown about sex/gender difference research education includes content of existing offerings, how competencies are defined, and specifically what topics are not yet addressed. This is a future direction for research in this area. In addition, it is necessary to determine the applied experience of the course directors, the methods content of trainings, and the applied fields being addressed in the trainings. Caution must be taken in generalizing survey results as respondents were limited to BIRCWH site leaders, and neither Scholars nor mentors were queried.

We and others have recognized that despite the growing global recognition for the need to consider sex and gender differences in medical research, there is a gap in available and accessible resources to educate the scientific workforce.¹² Creating the curricula is the first key step, but is not sufficient to fill the gap. There is also an implementation challenge to disseminating such education into practice. International collaborations that partner leading scientists and educators with thought leaders will encourage curricula to be universally recognized and adopted by leaders who have institutional levers such as required training for funded investigators or reviewers. Continuing to leverage education and training that touch all areas of science and at all stages of training and experience—such as rigor and reproducibility, responsible conduct of research, and institutional review—may be part of a strategy to deploy sex and gender considerations to mainstream science. It is also imperative to define educational competencies in this area and to define high-quality approaches.

Conclusions

BIRCWH programs represent a constellation of multidisciplinary research groups comprising national and international leaders focused in mentoring PhD and clinician scientists who wish to develop research careers in women's health and sex and/or gender differences. Thus, these are the groups who would be expected to lead efforts to create, implement, and disseminate educational curricula on integrating sex and gender into medical research. This leadership role has currently been taken on by the NIH ORWH to some extent as a “top-down” leadership engine approach, with the funder defining the goals. This has created the impetus for the existing focus and infrastructure and is highly valuable. However, in keeping with the leadership engine approach, the next step is for progress to be driven by the “bottom-up,” defined by the nationally distributed BIRCWH sites. In this way, change would be strengthened and supported from local implementers and national priority setters. It would be ideal to offer programmatic or project funding to incentivize scientific leaders to develop sex and gender education resources in their areas of expertise, and to forge collaborative and coordinated partnerships with other content experts, for standardized curriculum that can be made universally accessible and internationally adopted.

Another local approach to increasing sex and/or gender difference research is that all practitioners can contribute to the published literature and disseminate not only their findings but also their approaches and strategies that can be accessed by anyone in the field. Another opportunity is for the research community to learn from the medical education community about how to integrate sex and gender considerations into standing curricula. Only by implementing and disseminating both the knowledge and the practical application of integrating sex and gender into medical research will we see consistent and growing biomedical research, focused on sex and gender, which yields evidence-based health care and better health for all. Challenges remain with assessing the quality of educational offerings, defining competencies in sex and/or gender difference research, and uptake of this education for people in key roles such as researchers, mentors, reviewers, and editors.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

Appendix

References

Mazure

, Jones

. Twenty years and still counting: Including women as participants and studying sex and gender in biomedical research. BMC Womens Health, 2015; 15:94.

Clayton

, Collins

. Policy: NIH to balance sex in cell and animal studies. Nature, 2014; 509:282–283.

Heidari

, Babor

, De Castro

, Tort

, Curno

. Sex and Gender Equity in Research: Rationale for the SAGER guidelines and recommended use. Res Integr Peer Rev, 2016; 1:2.

McGregor

, Choo

. Gender-specific medicine: Yesterday's neglect, tomorrow's opportunities. Acad Emerg Med, 2012; 19:861–865.

Kling

, Rose

, Kransdorf

, Viggiano

, Miller

. Evaluation of sex- and gender-based medicine training in post-graduate medical education: A cross-sectional survey study. Biol Sex Differ, 2016; 7(Suppl 1):38.

Domino

, Bodurtha

, Nagel

, Leadership tBP. Interdisciplinary Research Career Development: Building Interdisciplinary Research Careers in Women's Health Program Best Practices. J Women's Health (Larchmt), 2011; 20:1587–1601.

Song

, Simonsen

, Wilson

, Jenkins

. Development of a PubMed based search tool for identifying sex and gender specific health literature. J Women's Health (Larchmt), 2016; 25:181–187.

Grant

, Booth

. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Info Libr J. 2009; 26:91–108.

Clayton

. Studying both sexes: A guiding principle for biomedicine. FASEB J, 2016; 30:519–524.

10.

Miller

, Rice

, Schiebinger

, et al. Embedding concepts of sex and gender health differences into medical curricula. J Women's Health (Larchmt), 2013; 22:194–202.

11.

Rojek

, Jenkins

. Improving medical education using a sex- and gender-based medicine lens. J Womens Health (Larchmt), 2016; 25:985–989.

12.

Regensteiner

, Libby

, Huxley

, Clayton

. Integrating sex and gender considerations in research: Educating the scientific workforce. Lancet Diabetes Endocrinol, 2019; 7:248–250.

13.

Weyers

, Vervoorts

, Dragano

, Engels

. The Gender Lens: Development of a learning aid to introduce gender medicine. GMS J Med Educ, 2017; 34, Doc17. DOI: 10.3205/zma001094

14.

Miller

. Why are sex and gender important to basic physiology and translational and individualized medicine?. Am J Physiol Heart Circ Physiol, 2014; 306:H781–H788.

15.

McGregor

, Madsen

, Clyne

. Foundations for a novel emergency medicine subspecialty: Sex, gender, and women's health. Acad Emerg Med, 2014; 21:1469–1477.

16.

Tingen

, Kim

, Wu

, Woodruff

. Sex and sensitivity: The continued need for sex-based biomedical research and implementation. Womens Health, 2010; 6:511–516.

17.

Tsouroufli

, Rees

, Monrouxe

, Sundaram

. Gender, identities and intersectionality in medical education research. Med Educ, 2011; 45:213–216.

18.

Stewart

, Kushner

, Gray

, Hart

. Promoting gender equity through health research: Impacts and insights from a Canadian initiative. Glob Health Promot, 2013; 20:25–38.

19.

Acharya

. Bridging the gap between evidence-based knowledge and clinical practice by avoiding sex-biased research and rediscovering the value of traditions in promoting health. Acta Obstet Gynecol Scand, 2015; 94:795–796.

20.

Ashurst

, McGregor

, Safdar

, et al. Emergency medicine gender-specific education. Acad Emerg Med, 2014; 21:1453–1458.