Mini-Residencies to Improve Care for Women Veterans: A Decade of Re-Educating Veterans Health Administration Primary Care Providers

Abstract

Background:

Many primary care providers (PCPs) in the Veterans Health Administration need updated clinical training in women's health. The objective was to design, implement, and evaluate a training program to increase participants' comfort with and provision of care to women Veterans, and foster practice changes in women's health care at their local institutions.

Methods:

The Women's Health Mini-Residency was developed as a multi-day training program, based on principles of adult learning, wherein knowledge gleaned through didactic presentations was solidified during small-group case study discussions and further enhanced by hands-on training and creation of a facility-specific action plan to improve women Veterans' care. Pre, post, and 6-month surveys assessed attendees' comfort with and provision of care to women. The 6-month survey also queried changes in practice, promulgation of program content, and action plan progress.

Results:

From 2008 to 2019, 2912 PCPs attended 26 programs. A total of 2423 (83.2%) completed pretraining and 2324 (79.3%) completed post-training surveys. The 6-month survey was sent to the 645 attendees from the first 14 programs; 297 (46.1%) responded. Comparison of pre-post responses indicated significant gains in comfort managing all 19 content areas. Six-month data showed some degradation, but comfort remained significantly improved from baseline. At 6 months, participants also reported increases in providing care to women, including performing more breast and pelvic examinations, dissemination of program content to colleagues, and progress on action plans.

Conclusions:

This interactive program appears to have been successful in improving PCPs' comfort in providing care for women Veterans and empowering them to implement institutional change.

Introduction

The Veterans Health Administration (VHA) created a Women Veterans Health Program to focus efforts on providing medical and psychosocial services to women in 1988 when women represented 4.4% of all U.S. Veterans. Today, women make up nearly 14%.¹ The majority of female Veterans from the wars in Afghanistan and Iraq are of childbearing age, a new cohort for many VHA primary care providers (PCPs). VHA recognized that these providers needed a program to update their competencies in women's health. In 2007, VHA charged a design team to develop a national educational program on women Veterans' health to be offered in multiple regions of the country. The goal was to train at least one PCP who was interested and clinically active in women's health care in each of the roughly 1200 VHA facilities.

The curriculum was to be standardized across sites in terms of major content, materials, and format, but feedback from each session would be integrated for improvements around the fixed base format. Success of the program would be assessed by pre-post training and 6-month surveys measuring self-reported comfort in caring for women, personally providing care for women, and institutional changes aimed at improving women Veterans' health care. This report describes the development process and 12 years of results for the Mini-Residency on Primary Health Care for Women Veterans.

Methods

Design team

The VHA Women Veterans Health Strategic Health Care Group selected a national leader in program development and women Veterans' health care to spearhead the program design and lead a team of national experts and opinion leaders in women Veteran's health and medical education. VHA also recruited a national training coordinator with experience in both health curriculum design and workshop development and implementation to operationalize delivery of a “mini-residency.” The team met over a period of 12 months.

Learners

The program was designed to be offered in diverse and easily accessible geographic locations. Invitations were extended to primary care physicians, nurse practitioners, and physician assistants in targeted geographic service areas.

Curriculum design: topic selection

Course content was based on review of existing women's health curricula, assessment of previous VHA-sponsored women's health workshops, results from a 2007 national educational needs assessment of VHA providers, and input from the assembled team of experts. The group initially identified eight areas by consensus as highly relevant to PCPs serving women Veterans of reproductive age: contraception, cervical cancer screening, sexually transmitted infections, abnormal uterine bleeding, chronic pelvic pain, breast mass management, post-deployment issues specific to women Veterans, and gynecologic (breast and pelvic) examinations. To address transforming the culture of an institution built to provide care for men, a lecture on institutional change was also included.

Course content was adjusted in response to newly identified learner needs, changes in women Veterans' demographics, current literature, new guidelines, and women's health innovations. Modifications included: (i) lectures on interpersonal violence, menopause, drug formulary changes relevant to women, and gynecologic emergencies; (ii) 2 hours of breast and pelvic exam practice with observation and feedback using manikins with various conditions; (iii) MammaCare session utilizing breast simulators to differentiate suspicious masses from normal nodular breast tissue (MammaCare Foundation, Gainesville, FL); (iv) woman Veteran's story of her deployment/reintegration experiences; and (v) military gear demonstration to illustrate its effect on the musculoskeletal system. Pharmacists were added as a resource for lectures, case discussions, and participant questions.

Curriculum design: format

To increase the likelihood of influencing participant knowledge, attitudes, and behaviors toward improving care for women Veterans, we reviewed multiple descriptive,^2,3 theoretical,^4
–6 and experimental^7
–9 writings on adult learning and continuing medical education. We rejected the traditional large group lecture-only format, as passive methods were generally ineffective in changing provider practice.^10
–12 Because no single teaching format had been found effective in all settings, we designed a multifaceted intervention^10

–15 using tenets from adult education to guide program development:

Creating a collaborative learning environment

Because learning new skills occur most effectively in environments where learners feel safe to acknowledge gaps in knowledge,^16

–20 are removed from distractions, and engage in social connections,²¹ the training occurred outside of the workplace.

Fostering motivation for changes in practice by making the course content relevant

The program employed multiple techniques to motivate participants and increase the likelihood of intentional behavioral change^6,22,23: (i) attendance costs were reimbursed; (ii) an introduction by a leader from VHA Office of Women's Health emphasized the program's importance to the VHA mission and attendees' responsibility to improve women Veteran care; (iii) participants identified deficiencies in knowledge through a precourse survey evaluating their comfort in managing issues covered during the training^{4,21,22,24,25}; and (iv) opportunities for networking and information exchange were provided.²⁶ A lecture on organizational change introduced factors that affect change and the plan-do-study-act quality improvement cycle.²⁷

Learners were then charged with the task of leading change by developing an action plan to improve care for women Veterans. Plan development was guided by a written template and participants from the same geographic locations worked collaboratively. Written commitments enable and reinforce intended behavior change^{7,30

–36} and collaboration fosters an inclusive learning environment.¹⁷

Facilitating active learning with an interactive program

Clear and unambiguous learning objectives for each topic directed the learning process.^24,37 Initial objectives were guided by a list of knowledge and skills compiled from a review of existing women's health curricula,^38

–41 and finalized by our expert team by consensus. The objectives evolved in response to program component modifications, learner needs, changes in the literature or guidelines, and women's health innovations (Table 1). Lecture and small group training materials were standardized⁴² and updated at least annually. Learners were provided with all materials.

Table 1.

Objectives for the Mini-Residency Activities

Welcome and introduction	1. Describe the growth and health care needs of the women Veteran population 2. Outline the expectations for WH-PCPs in providing care for women Veterans 3. Explain the role of VHA WHS for the provision of care to women Veterans 4. Discuss how to access VHAs Women's Health policies, resources, and programs to better help women Veterans
Abnormal uterine bleeding	1. Define abnormal uterine bleeding 2. Identify high acuity/risk presentations 3. Describe etiology and management of AUB 4. Discuss polycystic ovary syndrome
Breast issues	1. Explain the guidelines and practices for breast cancer screening and breast cancer risk assessment 2. Define the role of breast density in breast cancer risk and discuss the controversy surrounding management 3. Describe the appropriate steps for breast mass triage and management 4. Identify the causes and management of breast abnormalities in pregnancy and lactation
Chronic pelvic pain	1. Discuss the epidemiology of chronic pelvic pain disorders 2. Describe evaluation components, including triage 3. Identify the most common causes and symptoms 4. Review treatment options and evidence
Contraception	1. Describe major contraceptive issues for women 2. Explain the pharmacological effects of different types of contraceptives 3. Identify appropriate candidates for different methods of contraception 4. Educate patients about contraceptive methods, use, and side effects
Gynecologic emergencies in office primary care	1. Discuss the differential diagnosis and immediate management of women with bleeding in early pregnancy 2. Explain management options for nonpregnant women with acute heavy bleeding 3. Develop a differential diagnosis for reproductive age women with acute abdominal/pelvic pain 4. Identify gynecologic situations requiring urgent GYN consultation and/or hospitalization
Innovation diffusion and institutional change	1. Provide current information on “Hot Topics” in VA care 2. Explain steps to implement organizational change 3. Identify additional WH training opportunities
Interpersonal violence	1. Review interpersonal violence and its health outcomes 2. Discuss three types of interpersonal trauma, including newer research findings 3. Explain VHA policies and screening recommendations 4. Describe optimal management, emphasizing a patient-centered, trauma-informed, team-based approach
Menopause	1. Define menopause and perimenopause 2. Appropriately assess women presenting with menopause-like symptoms 3. Review common symptoms and discuss management options
Pelvic examination and cervical cancer screening	1. Explain the role of the pelvic examination—screening vs. diagnostic 2. Discuss the epidemiology and etiology of cervical cancer 3. Apply current cervical cancer screening guidelines 4. Describe how test results are reported 5. Manage abnormal test results 6. List indications and benefits of the HPV vaccine
Pharmacy and women's health in VA	1. Explain the national formulary process 2. Highlight pharmacy updates 3. Review frequently asked questions regarding pharmacy 4. Describe women's health clinical pharmacy services
Postdeployment issues for OEF/OIF/OND women Veterans	1. Describe the characteristics of OEF/OIF/OND women Veterans using VA health care 2. Identify the components of a good military history 3. Explain common health issues for OEF/OIF/OND women Veterans 4. Discuss methods to address physical and mental health care
Vaginitis and cervicitis, including sexually transmitted infections	1. Diagnose and manage common causes of vaginitis 2. Explain the components of a good sexual history 3. Discuss patient education for common vaginal infections and how to prevent them
Hands-on: breast and pelvic examination training with live gynecologic teaching associates and manikins	1. Perform a focused gynecologic history 2. Perform a female breast examination using currently recommended techniques 3. Demonstrate preparation and insertion of the vaginal speculum 4. Perform a bimanual pelvic examination 5. Identify normal and common abnormal pathologies of the breast and pelvis 6. Describe actions the provider and nurse can take to minimize patient anxiety
Hands-on: demo of women's health supplies/equipment	1. Identify and explain the use of common supplies and equipment utilized in women's health primary care
Hands-on: military gear demo	1. Discuss basic military gear and its weight 2. Reinforce its impact on women's musculoskeletal system
Hands-on: MammaCare training (optional activity)	1. Use palpation exercises to detect suspicious breast tumors

AUB, Abnormal Uterine Bleeding; HPV, Human papillomavirus; OEF/OIF/OND, Operation Enduring Freedom/Operation Iraqi Freedom/Operation New Dawn; PCP, primary care provider; VHA, Veteran's Health Administration; WH, women's health; WHS, Women's Health Services.

Participant interactions were facilitated by intermingling lectures and small-group activities.^13,43

–46 Seven of the 13 lectures had accompanying case studies. Case discussion rooms were arranged with four to five round tables of six to eight learners and one to two faculty facilitators. The cases modeled authentic patients and included real-life uncertainties in approaches, a technique that enhances learning over cases with more obvious answers.^47
–49

A key goal of the program was the development of basic skills that would enable participants to perform efficient and comprehensive breast and pelvic examinations using current evidence-based techniques. Sorted three-to-a-room, participants had 2 hours to work with a gynecological teaching associate (GTA) who was trained to demonstrate, review, and assess breast and pelvic examination skills using her body as the model.

To supplement the lecture content and GTA experience, participants also spent 20–30 minutes at tables showcasing a standardized collection of women's health supplies and equipment. Program faculty staffed the tables, providing information about the items and encouraging learners to handle them. Because only a limited number could attend at one time, participants received a customized guide of the entire program, delineating when hands-on activities would occur for their small group (Supplementary Appendix Table SA1).

Enhancing learning by practice with immediate feedback from trusted peers

Because physicians often prefer colleagues as sources of information^50
–52 and peer leaders can influence change,^{9,27,52

–57} we recruited national leaders on women's health, who were also experienced educators. Similar to their learners, these faculty were mainly general internists who practiced in community clinics and medical centers across the United States, but included gynecologists and advanced practice providers. This core of highly capable faculty contributed to the program's success and mitigated some of the need for faculty development. New faculty candidates were carefully screened and started as co-facilitators with experienced faculty members, who role modeled the teaching activities and provided feedback in an apprenticeship model.

Outcomes: self-evaluation of comfort and behavioral change

The mini-residency surveys were based on evaluations developed for a federally funded training program conducted at multiple academic institutions across the United States.⁵⁸ Several questions were added after a review of existing women's health curricular evaluations, and contributions from our expert team. The surveys followed Kirkpatrick's levels of program evaluation: (i) student reactions to the training; (ii) student increases in learning; (iii) student behavioral change and improvement; and (iv) result of student performance on the organization.⁵⁹ Participants evaluated their comfort with managing the training topics on a 5-point scale (1 = not comfortable to 5 = very comfortable) before and immediately after the program.

Learners who attended the first 3 years of mini-residencies also received a 6-month survey that queried comfort as well as (i) additional women's health training obtained; (ii) dissemination of knowledge learned to colleagues; (iii) changes in clinical practice attributable to mini-residency attendance; and (iv) progress on their change activity. Participants were also asked if they would recommend the mini-residency to colleagues and whether they would attend again. The follow-up survey was mailed twice to enhance the response rate.

During the first 2 years when the mini-residency model and generalizability were being closely evaluated, 2 exemptions were requested from local Institutional Review Boards for each of the 13 programs: 1 from the host VA Medical Center and 1 from the academic institution through which continuing medical education credits were obtained. All IRBs determined that, in accordance with federal regulations, the project did not constitute research.

Analysis

Main outcomes entailed changes in the average self-reported comfort levels across time. Our analytic approach used both paired t-tests and effect size estimates to run a priori contrasts of pre versus post, pre versus 6 month, and post versus 6 month data. Due to the large number of possible comparisons, results are presented not as null hypothesis tests, but as forest plots of standardized effects size differences with 95% confidence intervals. This provides an easier representation of where effects were occurring. All analyses were conducted using Stata version 16.1 (StataCorp, College Station, TX).

Results

Participants

From 2008 to 2019, this 3-day training program was successfully implemented 26 times in 13 locations across the United States. A total of 2912 PCPs attended, comprising at least 1 PCP from each of the 140 VA Health Care Systems, and from 844 of the 1074 (78.6%) VA community-based outpatient clinics. Of these 2912 attendees, 2398 (82.3%) provided sociodemographic and practice information queried on the precourse survey (Table 2). Eighty percent were female and half (51.7%) were in the 46–60 age range. More than half (56.1%) were physicians, 64.6% of whom were general internists and 35.4% family practitioners. The remaining attendees were nurse practitioners and physician assistants (21.7% each).

Table 2.

Participant Characteristics (n = 2398)

Sex
Female	1924 (80.2%)
Male	470 (19.6%)
Other or unspecified	4 (0.2%)
Age, years
20–35	224 (9.3%)
36–45	600 (25.0%)
46–60	1240 (51.7%)
60+	233 (9.7%)
Unspecified	101 (4.2%)
Training
Physician	1346 (56.1%)
Nurse practitioner	518 (21.7%)
Physician assistant	517 (21.7%)
Unspecified	17 (0.7%)
Physician specialty
General internal medicine	1001 (64.6%)
Family medicine	548 (35.4%)
Other or unspecified	276 (20.5%)
Practice
VA outpatient clinic	1305 (54.8%)
VA medical center	1076 (45.2%)
Full time	2239 (93.4%)
100% time in direct patient care	1217 (51.9%)
WH training in past 24 months	582 (24.3%)
WH importance to practice
Not at all	4 (0.1%)
Somewhat	131 (5.5%)
Fairly	319 (13.5%)
Moderately	514 (21.7%)
Very	1402 (59.2%)
Likely to improve skills in WH
Not at all	6 (0.3%)
Somewhat	66 (2.7%)
Fairly	168 (7.1%)
Moderately	514 (15.3%)
Very	1774 (74.6%)
Likely to use skills obtained from program
Not at all	5 (0.2%)
Somewhat	32 (1.4%)
Fairly	129 (5.4%)
Moderately	333 (14.0%)
Very	1879 (79.0%)

Participants were slightly more likely to practice in community outpatient clinics (54.8%) and nearly all worked full time (93.4%). Half (51.9%) dedicated 100% of their time to direct patient care and 24% had attended a seminar or conference on women's health in the previous 2 years. Eighty-one percent reported that the topic of women's health was moderately or very important to their current practice, 90% felt that they moderately or very likely could improve their skills in women's health, and 90% anticipated that they were moderately or very likely to use skills learned at the mini-residency in their practices.

Faculty

The mini-residency helped to develop a nationwide community of practice for program faculty. The current roster contains over 40 trained women's health experts. In 2012, VHA Women's Health Fellows began participating as co-faculty, thus fostering the next generation of women's health educators and leaders.

Outcomes: comfort in providing care to women

Of the 2912 participants, 2423 (83.2%) completed the preprogram survey and 2324 (79.3%) the post-program survey. The 6-month survey was sent to 645 participants from the first 3 years; 297 (46.1%) responded. Responses from the 2051 participants who completed both a presurvey and postsurvey indicated significant improvements in comfort for all issues taught. Figure 1 illustrates the changes in comfort by topic for all participants. The standardized mean difference (SMD), useful for reporting efficacy, is the size of the intervention effect relative to the variability. A positive SMD value indicates an improvement in the mean comfort rating.

FIG. 1.

Self-rated comfort in managing issues discussed during the mini-residency, all participants. 6mo, 6-month survey; IPV, Interpersonal Violence; MST, Military Sexual Trauma; Post, post-training survey; Pre, pretraining survey.

A negative SMD (seen for some 6 month postvalues) indicates a degradation in comfort. If the confidence interval crosses 0, changes in comfort are not significant. Comparisons of presurvey and postsurvey data indicate at least a half-point increase on the 5-point comfort scale for all 19 topics, with over a 1-point increase for 13 of them. Comparisons of the pre and 6-month responses show sustained improvements of at least 0.5 points on the scale (p ≤ 0.001) for all topics. Postsurvey versus 6-month responses, however, indicate degradation in comfort over the 6-month post-training period in all areas, except the four that involved hands-on activities: performing breast exams, performing pelvic exams, performing Pap tests, and identifying common breast and pelvic pathologies.

We examined comfort by the binary identified sex of the participant. Female participants showed degradation in comfort levels in all areas, except the four areas noted (Fig. 2). In contrast, comfort levels remained significantly higher than baseline for male participants in all areas, except managing atypical Pap tests and abnormal uterine bleeding (Fig. 3). Further analyses indicated that comfort responses were similar across practice sites from hospital-based clinics to much smaller community-based clinics.

FIG. 2.

Self-rated comfort in managing issues discussed during the mini-residency, female.

FIG. 3.

Self-rated comfort in managing issues discussed during the mini-residency, male participants.

Other data from the 6-month survey administered to the first 3 years of mini-residency attendees indicated that 28% of respondents had obtained additional women's health training, 68.7% were being consulted by colleagues regarding women's health issues, and 21% had promulgated mini-residency content by conducting training for other providers and nursing staff. Nearly all participants (98.6%) would recommend the mini-residency to colleagues and 87% would attend again; “no longer seeing women” was the most common reason for not re-attending.

Outcomes: self-reported behavioral change

A total of 234 of the 297 respondents (78.8%) who completed the 6-month survey provided information on changes in practice as a result of the mini-residency. Thirty-one percent reported newly engaging in breast and pelvic examinations or enhanced comfort and technical skills when providing them. Twenty percent reported either initiating contraceptive prescribing or improved contraception knowledge and discussion skills. Sixteen percent of comments related to cervical cancer prevention and detection, including initiation of Pap testing, improved specimen collection technique, increased screening rates, guideline-concordant screening intervals, better follow-up for abnormal results, use of liquid-based cytology, and improved human papillomavirus vaccination rates.

Additional changes in practice included workup and management for conditions such as abnormal uterine bleeding, chronic pelvic pain, vaginitis, and menopause (14%); enhanced awareness of postdeployment issues for women (12%); better screening and referral for military sexual trauma, intimate partner violence, and Post-Traumatic Stress Disorder (12%); improved patient education (11%); and organizational changes that involved physical space, materials, policies, protocols, reminders, and outreach related to women Veterans' health (4%).

Outcomes: organizational change

Fifty percent of participants completed their action plan to make a change to enhance women's care at their institution. An additional 21.6% reported some progress. Of the completed projects, the most frequent changes involved: obtaining women's health supplies and patient education materials for exam rooms (16%); adapting scheduling to include more women's health appointments or extend appointment duration for gynecologic exams (15%); performing rather than referring gynecologic exams (11%); stocking pharmacy items such as Thin Prep, Gardasil, and Plan B (11%); adding staff or increasing staff time to perform women's health care (7%); and establishing space for women such as a dedicated exam room with a nearby women's restroom (7%). The most common barriers to action plan completion included lack of time and support from facility administration, costs, pharmacy and laboratory “turf” issues, and resistance from other providers or nursing staff.

Discussion

In this article, we describe the rationale and process for developing the Mini-Residency on Primary Health Care for Women Veterans and results from this ongoing 12-year endeavor. This program was the VHA's first attempt to systematically blanket the United States with a training program that would increase comfort with women's health care, change provider behavior, and most innovatively enhance organizational capacity for change⁴⁴ down to the facility level. Our analysis of pre-post surveys suggests that this program was successful in meeting its goals of increasing PCPs' comfort with women's health care, increasing their engagement in the clinical care of women, and leading or catalyzing organizational changes in the provision of care to women Veterans.

Although we had a smaller sample with a lower response rate for the 6-month survey, we found that comfort levels remained above pretraining levels. However, comfort degraded over time, except in areas involving hands-on training where comfort remained at post-training levels. Self-reported comfort by male participants did not significantly degrade in most areas to the extent reported by female participants. We do not have an explanation for this. It is possible that female providers entered training with more experience caring for women patients or greater knowledge from their own personal experiences.

The demand for this women's health training program continues. When announced, registration typically fills within days. In recent years, registration efforts have focused on VA facilities that have not sent a provider to a mini-residency or facilities that need additional trained PCPs due to increasing numbers of women patients. Past participants may attend again in 3 years if last-minute registration cancellations occur.

This enhanced CME course, based on principles of adult learning, focused on improving participants' knowledge, skills, and behaviors, as well as empowering them to facilitate institution-level improvements in women Veterans' care. Unlike traditional CME courses, this program offers 6-month follow-up data on both behavioral change and institutional change. Both the design of this program and the 6-month outcomes help to differentiate it from more traditional CME courses where only immediate pre-post assessments are routinely collected.

Improvements resulting from this program have the potential to address women Veterans' reluctance to use VA health care due to perceptions that providers were not well-versed in women's health issues.⁶⁰ Available data indicate that providers who have been trained in gender-specific care both provide better quality of care to patients and are more highly rated by patients.^61

–64 Based on the success of this model for PCPs, similar training programs were created for primary care nurses (2011), PCP/nurse pairs (2014), emergency care provider/nurse pairs (2014), and emergency care nurses (2019). Women's Health Services offers small grants to VHA facilities who wish to convene any of these mini-residencies at the regional level using the standardized materials.

A traveling mini-residency for PCPs in rural facilities is currently underway as part of a partnership with the VHA Office of Rural Health. Finally, the model has been used to develop other VHA training programs, including the Mental Health Mini-Residency and the Mini-Residency in Musculoskeletal Health. Next steps are to (i) evaluate programs involving alternate methods of delivery; (ii) compare outcomes for learners from varying disciplines; (iii) objectively measure provider outcomes pre-post training (e.g., increases in pelvic examination numbers and contraceptive and menopausal hormone prescribing behavior); and (iv) investigate how to engage participants with follow-up training opportunities to freshen their knowledge.

This study has both strengths and limitations. The biggest strength is its size, duration, and high response rate for the presurvey and postsurvey. Over 2900 participants attended 26 training sessions over a 12-year period, making this a robust and sustainable training initiative. The educational interventions were grounded in evidence from the adult learning literature and the topics presented were carefully chosen to satisfy needs of VA providers who were newly charged with the care of women Veterans. The outcomes demonstrate an increase in comfort for managing all 19 issues taught, a change that was sustained for 6 months.

More importantly, the data show self-reported changes in providers' practice as a result of the mini-residency, including such services as adding breast and pelvic examinations to their practice or counseling on/prescribing contraceptives. An aspirational goal was to effect organizational change to try and modify a longstanding male-only culture of health care delivery in the VHA system. Over 70% of attendees who responded to the 6-month survey reported completing or making progress in implementing an action plan to improve women's health care delivery. This suggests that the program not only impacted individual provider behavior but also had a more diffuse impact on culture as well.

Limitations of this study include a small response rate for the 6-month survey, which limits the generalizability of results. In addition, all outcomes are self-reported and thus may not reflect actual practice, which would be a more robust outcome. Clinician's self-reported behavioral change has been found to be a reliable surrogate measure for objective behavioral outcomes,⁶⁵ but it remains unclear whether women Veterans actually received better care.

Finally, to address degradations in comfort and provide updated information, VA has implemented additional training for PCPs: (i) distribution of breast and pelvic examination simulation equipment to all medical centers with a web-based, CME-accredited component; (ii) online, CME-accredited versions of the mini-residency lectures; (iii) monthly women's health webinars that include updates of mini-residency topics as well as other issues; and (iv) grant funding and training materials to convene the national mini-residency at the regional level.

Conclusions

The Mini-Residency on Primary Health Care for Women Veterans utilized multiple evidence-based training methods, including lectures by national experts in women's health, small group case-based discussions facilitated by local women's health champions, hands-on breast and pelvic examination training sessions with live and simulated models, and creation of an action plan with site-specific achievable objectives and timelines.

Transforming the VHA into a system that provides comprehensive, quality care for women Veterans requires a change in organizational culture and practice. Analysis of this far-reaching educational endeavor suggests that it has been effective in improving the ability of individual providers to care for women Veterans, and in empowering attendees as agents of change to improve women Veterans' health care within their organizations.

Footnotes

Acknowledgments

The authors would like to acknowledge additional members of the mini-residency's original Planning Committee: Rachel Bonnema, MD, MS; Rebecca Gonzalez, MSN, ARNP; Gail Gunter-Hunt, ACSW, LCSW; Connie LaRosa, RN, MSA, CPHQ; Lisa Longo, PharmD, BCPS; and Ellen Yee, MD, MPH.

We are grateful to the following faculty for their invaluable contributions to the lecture, case study, or skills session material: Lygia Acaro, MSN, MHA, RB, BC; Suvarna Akki, MD; Randall Ball, MD; Deborah DiNardo, MD, MS; Natara Garovoy, PhD, MPH; Karen Goldstein, MD, MSPH; Lisa Hardman, DNP, RN, CDE; Elizabeth Jackson, MS; Carrie Kairys, DNP, FNP-BC; Christine Kolehmainen, MD, MS; Chasitie Levesque, Pharm-D; Kathleen McIntyre-Seltman, MD; Sherry Nordstrom, MD; Barbara Palmer, NP; Barbara Polak, MSN, RN; Lisa Roybal, FNP; Aimee Sanders, MD, MPH; and Amy Stevens, MD. And finally, we thank Magelli Talastas and Traci Turner for their inestimable program support.

Authors' Contributions

L.B.M., MS, contributed to the design and coordinated the delivery of the mini-residencies. She participated in drafting and revising this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

M.M., MD, MPH, contributed to the design and conduct of the mini-residency. She made critical revisions to this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

M.R.G., MD, MPH, contributed to the design and conduct of the mini-residency. She made critical revisions to this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

S.I., MD, contributed to the design and conduct of the mini-residency. She made critical revisions to this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

S.S., MD, MS, contributed to the design and conduct of the mini-residency. She made critical revisions to this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

C.S., MD, contributed to the design and conduct of the mini-residency. She made critical revisions to this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

R.B., PhD, contributed to the design, conduct, and interpretation of data analyses. He participated in drafting and revising this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results

L.V., MD, contributed to the design and conduct of the mini-residency. She made critical revisions to this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

S.H., MD, MS, contributed to the conduct of the mini-residency. She made critical revisions to this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

P.H., PhD, contributed to the design of the mini-residency. She made critical revisions to this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

M.C., MD, MS, contributed to the design and conduct of the mini-residency. She participated in drafting and revising this article's intellectual and technical content, provided final approval, and assumes responsibility and accountability for the results.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

Office of Women's Health in the Veterans Health Administration provided all funding for the mini-residency program, including faculty and participant expenses.

Supplementary Material

Supplementary Appendix Table SA1

References

National Center for Veterans Analysis and Statistics. August 5, 2020. Available at: https://www.va.gov/VETDATA/Veteran_Population.asp Accessed October 28, 2020 .

Lowe

, Aparicio

, Galbraith

, et al. The future of continuing medical education: Effectiveness of continuing medical education: American College of Chest Physicians evidence-based educational guidelines. Chest, 2009; 135:69S–75S.

Gruppen

, Simpson

, Searle

, Robins

, Irby

, Mullan

. Educational fellowship programs: Common themes and overarching issues. Acad Med, 2006; 81:990–994.

Knowles

. The adult learner, 6th ed. Burlington, MA: Elsevier, 2005.

Kaufman

. Applying educational theory in practice. ABC of learning and teaching in medicine. BMJ, 2003; 326:213–216.

Simpson

. A conceptual framework for transferring research to practice. J Subst Abuse Treat, 2002; 22:171–182.

Mazmanian

, Daffron

, Johnson

, Davis

, Kantrowitz

. Information about barriers to planned change: A randomized controlled trial involving continuing medical education lectures and commitment to change. Acad Med, 1998; 73:882–886.

Bryner

Jr . Learning as a function of lecture length. Fam Med, 1995; 27:379–382.

Lomas

, Enkin

, Anderson

, Hannah

, Vayda

, Singer

. Opinion leaders vs. audit and feedback to implement practice guidelines. Delivery after previous Cesarean section. JAMA, 1991; 265:2202–2207.

10.

Forsetlund

, Bjørndal

, Rashidian

, et al. Continuing education meetings and workshops: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev, 2009; 2009:CD003030.

11.

Marinopoulos

, Dorman

, Ratanawongsa

, et al. Effectiveness of continuing medical education. Evid Rep Technol Assess (Full Rep), 2007; 149:1–69.

12.

Davis

, Thomson

, Oxman

, Haynes

. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA, 1995; 274:700–705.

13.

Bloom

. Effects of continuing medical education on improving physician clinical care and patient health: A review of systematic reviews. Int J Technol Assess Health Care, 2005; 21:380–385.

14.

Grimshaw

, Shirran

, Thomas

, et al. Changing provider behavior: An overview of systematic reviews of interventions. Med Care, 2001; 39:II2–II45.

15.

Smith

WR.

Evidence for the effectiveness of techniques to change physician behavior. Chest 2000;118;8S–17S.

16.

Edmondson Amy. Psychological safety and learning behavior in work teams. Admin Sci Q, 1999; 44:350–383.

17.

Imel

. Inclusive adult learning environments. ERIC Digest #162. Columbus, OH: ERIC Clearinghouse on Adult, Career, and Vocational Education, 1995.

18.

Musselwhite

. Managing change: Styles, stages, and effective leadership for academic leaders. In: Johnson

, Hanna

, Olcott

Jr ., eds. Bridging the gap: Leadership, technology, and organizational change for university deans and chairpersons. Madison, WI: Atwood Publishing, 2003:55–74.

19.

Johnson

, Johnson

. Cooperative learning: Where we have been, where we are going. Coop Learn Coll Teach, 1993; 3:6–9.

20.

Schwartz

. Active, small group learning with a large group in a lecture theatre: A practical example. Med Teach, 1989; 11:81–86.

21.

Knox

. Helping adults learn. San Francisco: Jossey-Bass, 1986.

22.

Janis

, Mann

. Decision. making: A psychological analysis of conflict, choice, and commitment. New York: Free Press, 1977.

23.

Prochaska

, DiClemente

. The transtheoretical approach: Crossing the traditional boundaries of therapy. Melbourne, FL: Krieger, 1984.

24.

Grant

. Learning needs assessment: Assessing the need. BMJ, 2002; 324:156–159.

25.

Moon

. Reflection in learning and professional development: Theory and practice. London: Kogan Page, 1999.

26.

Tipping

, Donahue

, Hannah

. Value of unstructured time (breaks) during formal continuing medical education events. J Contin Educ Health Prof, 2001; 21:90–96.

27.

Deming

. The new economics. Cambridge, MA: MIT Press, 1993.

28.

Lowe

, Rappolt

, Jaglal

, MacDonald

. The role of reflection in implementing learning from continuing education into practice. J Contin Educ Health Prof, 2007; 27:143–148.

29.

Sheridan

. Rethinking andragogy: The case for collaborative learning in continuing higher education. J Contin Higher Educ, 1989; 37:2–6.

30.

Overton

, MacVicar

. Requesting a commitment to change: Conditions that produce behavioral or attitudinal commitment. J Contin Educ Health Prof, 2008; 28:60–66.

31.

Herscovitch

, Meyer

. Commitment to organizational change: Extension of a three-component model. J Appl Psychol, 2002; 87:474–487.

32.

Pereles

, Lockyer

, Hogan

, Gondocz

, Parboosingh

. Effectiveness of commitment contracts in facilitating change in continuing medical education interventions. J Contin Educ Health Prof, 1997; 17:27–31.

33.

Weiner

. Commitment in organizations: A normative view. Acad Manage Rev, 1982; 7:418–428.

34.

White

, Grzybowski

, Broudo

. Commitment to change instrument enhances program planning, implementation, and evaluation. J Contin Educ Health Prof, 2004; 24:153–162.

35.

Wakefield

. Commitment to change: Exploring its role in changing physician behavior through continuing education. J Contin Educ Health Prof, 2004; 24:197–204.

36.

Wakefield

, Herbert

, Maclure

, et al. Commitment to change statements can predict actual change in practice. J Contin Educ Health Prof, 2003; 23:81–93.

37.

Vermunt

. Metacognitive, cognitive and affective aspects of learning styles and strategies: A phenomenographic analysis. Higher Educ, 1996; 31:25–50.

38.

Association of Professors of Gynecology and Obstetrics

, GO Women's Healthcare Education

Office

. Women's health care competencies for medical students: Taking steps to include sex and gender differences in the curriculum. Crofton, MD: APGO, 2005.

39.

Cassel

, Blank

, Braunstein

, Burke

, Fryhofer

, Pinn

. What internists need to know: Core competencies in women's health. Am J Med, 1997; 102:507–512.

40.

Magrane

, McIntyre-Seltman

. Women's health care issues for medical students: An education proposal. Womens Health Issues, 1996; 6:183–191.

41.

National Academy on Women's Health Medical Education. Women's health in the curriculum: A resource guide for faculty: Undergraduate, residency, and continuing education. Philadelphia, PA: NAWHME, 1996.

42.

Rohrbach

, Grana

, Sussman

, Valente

. Type II translation: Transporting prevention interventions from research to real-world settings. Eval Health Prof, 2006; 29:302–333.

43.

Peloso

, Stakiw

. Small-group format for continuing medical education: A report from the field. J Contin Educ Health Prof, 2000; 20,:27–32.

44.

Casebeer

, Centor

, Kristofco

. Learning in large and small groups. In: Davis

, Barnes

, Fox

, eds. The continuing professional development of physicians: From research to practice. Chicago, IL: American Medical Association, 2003:169–190.

45.

Maltby

, Andrusyszyn

. The case study approach of teaching decision-making to post-diploma nurses. Nurse Educ Today, 1990; 10:415–419.

46.

Satterlee

, Eggers

, Grimes

. Effective medical education: Insights from the Cochrane Library. Obstet Gynecol Surv, 2008; 63:329–333.

47.

Bennett

, Harper

, Hedberg

. Designing real life cases to support authentic design activities. Aust J Educ Tech, 2002; 18:1–12.

48.

Irby

. Three exemplary models of case-based teaching. Acad Med, 1994; 69:947–953.

49.

Williams

. Putting case-based instruction into context: Examples from legal and medical education. J Learn Sci, 1992; 2:367–427.

50.

Cacioppo

, Petty

. Effects of message repetition on argument processing, recall, and persuasion. Basic Appl Soc Psych, 1989; 10:3–12.

51.

Haug

. Physicians' preferences for information sources: A meta-analytic study. Bull Med Libr Assoc, 1997; 85:223–232.

52.

Premi

. Individualized continuing medical education. In: Davis

, Fox

, eds. The physician as learner: Linking research to practice. Chicago: American Medical Association, 1994:201–216.

53.

Fox

, Mazmanian

, Putnam

. A theory of learning and change. In: Fox

, Mazmanian

, Putnam

, eds. Changing and learning in the lives of physicians. New York: Praeger Publishers, 1989:161–175.

54.

Cialdini

. Influence: The new psychology of modern persuasion. New York, NY: Quill, 1984.

55.

Valente

, Pumpuang

. Identifying opinion leaders to promote behavior change. Health Educ Behav, 2007; 34:881–896.

56.

Doumit

, Gattellari

, Grimshaw

, O’Brien

. Local opinion leaders: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev, 2007; 24:CD000125.

57.

Kelly

, St Lawrence

, Diaz

, et al. HIV risk behavior reduction following intervention with key opinion leaders of population: An experimental analysis. Am J Public Health, 1991; 81:168–171.

58.

National Institute on Alcohol Abuse and Alcoholism. A Medical Education Model for the Prevention and Treatment of Alcohol Use Disorders. Bethesda, MD: U.S. Department of Health and Social Services, National Institutes of Health, NIAAA, 1998.

59.

Kirkpatrick

. Great ideas revisited. Techniques for evaluating training programs. Revisiting Kirkpatrick's four level model. Training Dev, 1996; 50:54–59.

60.

Washington

, Yano

, Simon

, Sun

. To use or not to use. What influences why women veterans choose VA health care. J Gen Intern Med, 2006; 21:S11–S18.

61.

Bean-Mayberry

, Bastian

, Trentalange

, et al. Associations between provider designation and female-specific cancer screening in women Veterans. Med Care, 2015; 53(Suppl 1):S47–S54.

62.

Borrero

, Mor

, Zhao

, McNeil

, Ibrahim

, Hayes

. Contraceptive care in the VA health care system. Contraception, 2012; 85:580–588.

63.

Koenig

, Borrero

, Zhao

, Callegari

, Mor

, Sonalkar

. Factors associated with long-acting reversible contraception use among women Veterans in the ECUUN study. Contraception, 2019; 100:234–240.

64.

Reddy

, Pollack

, Asch

, Canamucio

, Werner

. The effect of primary care provider turnover on patient experience of care and ambulatory quality care. JAMA Intern Med, 2015; 175:1157–1162.

65.

Eccles

, Hrisos

, Francis

, et al. Do self-reported intentions predict clinicians' behavior: A systematic review. Implement Sci, 2006; 1:5908–5928.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB