An approach to improve the care of mid-life women through the implementation of a Women’s Health Assessment Tool/Clinical Decision Support toolkit

Abstract

The objectives of this study are to describe the implementation process of the Women’s Health Assessment Tool/Clinical Decision Support toolkit and summarize patients’ and clinicians’ perceptions of the toolkit. The Women’s Health Assessment Tool/Clinical Decision Support toolkit was piloted at three clinical sites over a 4-month period in Washington State to evaluate health outcomes among mid-life women. The implementation involved a multistep process and engagement of multiple stakeholders over 18 months. Two-thirds of patients (n = 76/110) and clinicians (n = 8/12) participating in pilot completed feedback surveys; five clinicians participated in qualitative interviews. Most patients felt more prepared for their annual visit (69.7%) and that quality of care improved (68.4%) while clinicians reported streamlined patient visits and improved communication with patients. The Women’s Health Assessment Tool/Clinical Decision Support toolkit offers a unique approach to introduce and address some of the key health issues that affect mid-life women.

Keywords

clinical decision support patient satisfaction primary care quality improvement women’s health Women’s Health Assessment Tool

Introduction

Women’s annual health assessments or “well-woman visits” have been found to aid disease detection, facilitate referrals to further screening, help relieve patients’ worries about health, and are considered particularly important for mid-life women aged 45–64 years.^1–4 However, the practical challenges of conducting annual health assessments within the primary care practice are notable. Extensive screening lists ultimately reduce time spent between clinicians and patients which negatively impacts the quality of care provided.^5,6 This is only compounded as women enter mid-life and have an increasing number of health issues including the menopause transition.^6–9

With the integration of electronic medical records (EMR) into routine clinical practice, new strategies can be implemented to facilitate patient screening and follow-up. One approach has been the implementation of online screening tools and clinical decision support (CDS) systems. The use of both components conjointly enables point-of-care risk assessment for clinicians and has been shown to improve delivery of care and patient outcomes.^10–13

However, despite the noted improvements to health outcomes, barriers exist in the implementation process. The inclusion of online screening tools and CDS systems are heavily reliant on the adaption of the instruments into routine practice, requiring the engagement of clinicians in the process where many clinicians are skeptical or reluctant to participate.^14–16 In addition, patient-completed components need to rely on patients’ access to and use of computers which may create participation barriers. Implementation also requires the involvement of many stakeholders at an organizational level, which can provide unique challenges such as effective communication and collaboration across stakeholders.^17–19 Finally, there is a dearth of information on the implementation of this electronic, patient-integrated approach in primary care and even less information available on such targeted care improvements among mid-life women.

Therefore, the Women’s Health Assessment Tool (WHAT)/CDS toolkit, a health assessment questionnaire coupled with a tool to support clinical decision, was developed and implemented at three clinical sites of an integrated delivery network (IDN). Specifically, the WHAT/CDS toolkit was developed to engage patients in their health and allow them to become active participants in the care they receive; to standardize the condition assessment process, create efficiencies, and promote communication between clinicians and patients; to improve the efficiency of the clinician office visit by tailoring to meet patients’ needs; and to improve the quality of the well-woman visit for mid-life women. The objectives of this article are to describe the implementation process and summarize the patients’ and clinicians’ perception about the utility, feasibility, and satisfaction with the WHAT/CDS toolkit.

Method

WHAT project design

The WHAT/CDS toolkit was implemented at three clinical sites (two primary care and one gynecology) of the IDN in Washington State through a multistep process. The process involved the engagement and coordination of multiple stakeholders over the course of 18 months. This project was reviewed and approved by the Multicare Health System Institutional Review Board (Study 13.14) of the participating IDN. The outcomes from the implementation of the WHAT/CDS toolkit have been described elsewhere.²⁰

Development of the WHAT/CDS toolkit

The WHAT/CDS toolkit is a two-component system with a patient-completed assessment tool developed through an iterative process. The first component of the toolkit, the WHAT questionnaire, is an electronic 35-item patient survey containing health-related questions and assesses health conditions relevant to women aged 45–64 years. The WHAT questionnaire was developed through a situational analysis and needs assessment to determine what symptoms and conditions were relevant for mid-life women. Cognitive interviews with 30 patients and 5 clinicians were then conducted to evaluate relevance, ease of use, and comprehensiveness of the questionnaire. The final questionnaire includes both existing measures and newly developed questions to evaluate prevalent health conditions such as urinary incontinence (UI), depression, vulvovaginal atrophy (VVA), and vasomotor symptoms (VMS). The existing measures used in the WHAT included the Patient Health Questionnaire-9 (PHQ-9),²¹ Generalized Anxiety Disorder-2 (GAD-2),²² Female Sexual Function Index-6 (FSFI-6),²³ and 3 Incontinence Questions (3-IQ).²⁴

The second component, the CDS toolkit, is an electronic data capture system built into the EMR to support the delivery of evidence-based care. The CDS toolkit was developed from existing SmartSets defined as condition-specific action-items relevant to a patient’s visit such as possible laboratory or diagnostics tests that were reviewed and modified by clinicians at participating medical facilities.

The toolkit captures relevant information from the patient’s EMR including prior history, labs, imaging studies, reports in the form of discrete data or links to the EMR. The toolkit also includes Best Practice Alerts (BPAs), which alert clinicians of additional questions to consider based on the woman’s responses to the WHAT questionnaire.

Implementation of the WHAT/CDS toolkit and associated process

The WHAT/CDS toolkit was implemented in two parallel stages (Figure 1). The implementation process involved collaboration between hospital management, information technology (IT) personnel, research staff, clinical staff, and clinical site clinicians and took place between June 2013 and December 2013. The first stage included the configuration and testing of the WHAT/CDS toolkit through the integrated delivery system (IDS) IT platform. Technical plans were drafted and reviewed by hospital IT and research staff for base configuration. After finalization of the WHAT/CDS toolkit content, all features of the system were integrated into the patient portal and IDS EMR system and were tested and refined by research staff and clinical site clinicians.

Figure 1.

WHAT/CDS Toolkit Integration Process.

The second stage of implementation involved the development of the project process design and training of all clinical staff to be involved with the patient implementation process. This included clinical support staff, nurses, front desk staff, hospital administrators, and clinicians. The trainings were group-specific (support staff and administrators: in-person versus clinicians: web-based) and were conducted in November 2013.

WHAT/CDS toolkit process

Eligible women were aged 45–64 years, existing patients from one of the three clinical sites, and registered MyChart (a patient portal into their EMR) users who were scheduled for an annual well-woman visit between 1 March and 31 July 2014. These eligible patients were sent an email 7 days in advance of their annual visit, notifying them to log into their MyChart account and view a message in their MyChart inbox. Reminders were sent to eligible patients 48 h prior to their visit if they had not logged in to complete the WHAT questionnaire.

The MyChart message invited patients to participate in the project and complete the WHAT questionnaire. The message included a brief description of the nature and objective of the project. Those who agreed to participate were directed to electronically review and sign an informed consent form and then complete the WHAT questionnaire. After completing the WHAT questionnaire, the results were directly uploaded to the patient’s EMR for review by their clinician and medical support staff prior to the patients’ visit.

During the well-woman visit, the SmartSets offered a list of relevant action steps associated with specific conditions and BPAs were triggered based on the responses to the WHAT questionnaire (Figure 2). All data collected through the WHAT/CDS toolkit were embedded in the patient’s EMR for clinicians to reference in future visits.

Figure 2.

WHAT Project Process.

Evaluation of the WHAT/CDS toolkit by patient and clinician

Following their well-woman visit, patients who completed in the WHAT questionnaire were given the opportunity to complete a brief 7-question paper survey about their perception of the WHAT in the waiting room. The patient’ post-visit survey was designed to assess the ease of use, quality of clinician visit, quality of interaction with clinician, better overall care, patient satisfaction, and willingness to reuse.

A total of 2 weeks prior to the completion of the project, clinicians were given opportunities to provide feedback through an online web-based survey and a one-on-one qualitative interview. The post-project clinician survey was a 10-question web-based survey that assessed the training, ease of use, efficiency, improved care, patient communication, utility, and willingness to reuse. Both patients’ post-visit and clinicians’ post-project surveys used a 7-point response scale and were developed based on a literature review of published healthcare quality improvement (QI) surveys.

Clinician qualitative interviews were conducted over the telephone. A trained interviewer asked a series of open-ended questions regarding clinicians’ perceptions of the WHAT/CDS toolkit, how the toolkit impacted patient care, areas of improvement in the process, and perceived value of the toolkit. Interviewers took detailed notes during the interviews; all interviews were audio-recorded with clinicians’ permission and subsequently transcribed. Each clinician interview took approximately 30 min to complete.

Data analysis

All descriptive analyses were conducted in a locked database and using SAS version 9.4.²⁵ Patient and clinician survey data were summarized using descriptive statistics (n, percentage, mean, standard deviation (SD), or range, as appropriate). All qualitative data were analyzed using a content analysis approach (based on notes, transcripts, and audio recordings). Transcripts and interviewer notes were systematically analyzed for feedback and information relevant to the key areas of interest identified in the interview guide.

Results

Over the 4-month recruitment period, 110 women (29.7% response rate) completed the WHAT questionnaire and 12 clinicians used the WHAT/CDS toolkit. The majority of women (n = 84, 76.4%) who completed the WHAT questionnaire did so after receiving a reminder email to complete the WHAT 48 h prior to their visit.

Mean age was 54.3 (SD = 5.9, range: 45–64) years and the majority of the women enrolled in the project were White (n = 99, 90.0%). A third of the women (n = 37, 33.6%) had at least one diagnosis of UI, VVA, depression, or VMS during the well-woman visit (Table 1), and the most common diagnosis among the four conditions was VMS (n = 16, 14.5%).

Table 1.

WHAT project diagnoses and CDS tool outcomes.

	Total sample (N = 110)
Diagnosis, n (%)^a
UI	5 (4.5%)
VVA	12 (10.9%)
Depression	10 (9.1%)
VMS	16 (14.5%)
Multiple diagnoses
0	73 (66.4%)
1	31 (28.2%)
2	6 (5.5%)
Best Practice Alert per patient^b
Number of patients with Best Practice Alerts (n)	21
Mean (SD)	5.3 (2.1)
Median	5.0
Range (min, max)	(2.0–10.0)
SmartSets accessed per patient^b
Number of patients accessed SmartSets (n)	22
Mean (SD)	1.6 (0.7)
Median	1.5
Range (min, max)	(1.0–3.0)
SmartSets accessed, n (%)^b
Anxiety	1 (0.9%)
Birth control	2 (1.8%)
Depression	5 (4.5%)
HRT	2 (1.8%)
Healthy behaviors	4 (3.6%)
Hot flashes	3 (2.7%)
Intimate relationships	1 (0.9%)
Kegel exercises	2 (1.8%)
Menopause	3 (2.7%)
Osteoporosis	3 (2.7%)
Sleep apnea	1 (0.9%)
Smoking cessation	1 (0.9%)
Tests	1 (0.9%)
UI education	1 (0.9%)
VVA education	3 (2.7%)

WHAT: Women’s Health Assessment Tool; CDS: Clinical Decision Support; HRT: hormone replacement therapy; UI: urinary incontinence; SD: standard deviation; VMS: vasomotor symptoms; VVA: vulvovaginal atrophy.

Diagnosis categories are not mutually exclusive.

This only includes patients who had at least one diagnosis in UI, VVA, Depression, or VMS (either newly or existing diagnosis).

Patients: perceptions on the WHAT toolkit

Two-thirds (n = 76) of the patients who completed the WHAT questionnaire also completed the post-visit survey. Women provided generally positive feedback about use of the WHAT questionnaire (Table 2). More than 90% of women “agreed” or “strongly agreed” that the survey was easy to understand (96.1%) and that the website was easy to use (93.4%). More than half “somewhat agreed” or “strongly agreed” that completing the WHAT questionnaire improved the care received from their clinician (68.4%), facilitated discussions with their clinician (69.8%), and helped them feel more prepared for their visit (69.7%). Approximately 20% of the women provided a neutral response to these questions. A little more than half of the women “somewhat agreed” or “strongly agreed” (56.6%) that they were more satisfied with services received, whereas 35.5% “neither agreed nor disagreed” with the statement. Most (71.1%) women “agreed” or “strongly agreed” that they would use the questionnaire again.

Table 2.

Post-visit patient survey (N = 76).

Questions	“Strongly disagree”	“Disagree”	“Somewhat disagree”	“Neither agree nor disagree”	“Somewhat agree”	“Agree”	“Strongly agree”	Missing	Mean (SD)
Easy to understand	0 (0.0%)	1 (1.3%)	1 (1.3%)	1 (1.3%)	0 (0.0%)	28 (36.8%)	45 (59.2%)	0 (0.0%)	2.5 (0.9)
Website easy to use	1 (1.3%)	0 (0.0%)	1 (1.3%)	0 (0.0%)	1 (1.3%)	28 (36.8%)	43 (56.6%)	2 (2.6%)	2.5 (0.9)
Felt more prepared for well visit	2 (2.6%)	2 (2.6%)	2 (2.6%)	16 (21.1%)	13 (17.1%)	17 (22.4%)	23 (30.3%)	1 (1.3%)	1.4 (1.5)
Helped discussion with clinician	4 (5.3%)	1 (1.3%)	2 (2.6%)	16 (21.1%)	5 (6.6%)	19 (25.0%)	29 (38.2%)	0 (0.0%)	1.5 (1.7)
Improved care received	3 (3.9%)	2 (2.6%)	1 (1.3%)	18 (23.7%)	7 (9.2%)	21 (27.6%)	24 (31.6%)	0 (0.0%)	1.4 (1.6)
More satisfied with services	2 (2.6%)	1 (1.3%)	1 (1.3%)	27 (35.5%)	6 (7.9%)	14 (18.4%)	23 (30.3%)	2 (2.6%)	1.3 (1.5)
Would use survey again	1 (1.3%)	1 (1.3%)	1 (1.3%)	16 (21.1%)	3 (3.9%)	27 (35.5%)	27 (35.5%)	0 (0.0%)	1.7 (1.4)

SD: standard deviation.

Raw score valued as follows: “strongly disagree” (–3), “disagree” (–2), “somewhat disagree” (–1), “neither agree nor disagree” (0), “somewhat agree” (1), “agree” (2), “strongly agree” (3).

Healthcare clinicians: perceptions on utility, feasibility, and impact of WHAT toolkit

The 12 participating clinicians used the SmartSets with 22 patients and used BPAs with 21 patients. A mean of 5.3 ± 2.1 BPAs were issued and 1.6 ± 0.7 SmartSets were used among patients in whom the respective tool was used. A total of 5 clinicians participated in the one-on-one interviews (Table 3) and 8 of the 12 clinicians completed the clinician post-project survey (Table 4).

Table 3.

Clinician qualitative feedback.

	n	Quote
Clinician feedback on the impact of the WHAT/CDS toolkit on patient care
Facilitated communication	3	It just opens the door for conversations though sometimes the issues that were brought up weren’t really problems. (Site 1; NP)
Helped bring up issues usually not addressed	3	So, maybe sometimes writing it down makes it a little, um, easier to, you know, kind of break the ice for lack of a better word and get their concern out there without them having to, you know, look at me face-to-face in the office and kind of address it which might be a little more, um, embarrassing for the patient. (Site 2; NP)
Helped prioritize issues and streamline the visit	4	Efficiency; and it helps me get right to the point and the patient feels valued. And they feel a part of their healthcare, uh, plan. (Site 1; NP)
Improved the ability to provide care	3	Yeah, it’s been good. Uh, I really—I really like it. And—and as a clinician in a busy day I don’t feel so isolated and I feel like I’m providing good care that I couldn’t give. But I’m getting them hooked with good care that’s available. And instead of being, oh that’s such a drag, or—or moving on to the next topic because as a clinician I’m feel–. (Site 1; NP)
Improved patient engagement in their healthcare needs	4	And I think when someone identifies themselves, what their issues are, they’re going to buy into it more. (Site 1; NP)
Clinician feedback on WHAT/CDS toolkit system, training, and continuation
Satisfied with the layout and functionality and usability of the system	3	Yeah, that was, you know, and our—the way we do our—we have electronic medical records and the way it was laid out, they have the patients do it and then have it available at the time of their visit electronically was very useful. That’s a good idea. So, I’m—I’m very much in favor of that. (Site 1; MD)
Satisfied with the information captured in the WHAT/CDS toolkit	4	I think that probably covers, um, some of probably the main concerns, um, for that particular group outside the normal kind of preventative things that we address with them. (Site 2; NP)
Satisfied with training provided on WHAT/CDS toolkit	3	Yeah. They, um, yeah the training was fine and—and they, you know. I didn’t have any issues with that. I think the goals were—were well-stated. (Site 1; MD)
Interested in continuation of the WHAT/CDS toolkit	3	Yes please sign me up, um, and it’s turned out to be better than I would’ve thought. And, you know, kind of my usual fears of, oh God. I—I hope I have time to do this, have been completely, um, eliminated because not only do I have time, but when I see now in my schedule that, um, I have two or three who’ve completed the questionnaire I go, oh thank God. (Site 1; NP)

WHAT/CDS toolkit: Women’s Health Assessment Test/Clinical Decision Support Tool; NP: Nurse Practitioner; MD: Medical Doctor.

Table 4.

Post-project clinician survey (N = 8).

Questions	“Strongly disagree”	“Disagree”	“Somewhat disagree”	“Neither agree nor disagree”	“Somewhat agree”	“Agree”	Mean (SD)
Sufficient training and support	2 (25.0%)	0 (0.0%)	2 (25.0%)	1 (12.5%)	0 (0.0%)	3 (37.5%)	−0.3 (2.1)
Easy to use	1 (12.5%)	2 (25.0%)	1 (12.5%)	2 (25.0%)	0 (0.0%)	2 (25.0%)	−0.5 (1.9)
Care was more efficient	1 (12.5%)	2 (25.0%)	0 (0.0%)	4 (50.0%)	1 (12.5%)	0 (0.0%)	−0.8 (1.4)
Ordered more targeted test or referrals	1 (12.5%)	2 (25.0%)	1 (12.5%)	2 (25.0%)	2 (25.0%)	0 (0.0%)	−0.8 (1.5)
Changed approach to health condition	1 (12.5%)	2 (25.0%)	3 (37.5%)	2 (25.0%)	0 (0.0%)	0 (0.0%)	−1.3 (1.0)
Helped make more targeted care decisions	1 (12.5%)	3 (37.5%)	1 (12.5%)	2 (25.0%)	1 (12.5%)	0 (0.0%)	−1.1 (1.4)
Patients were better prepared	1 (12.5%)	0 (0.0%)	1 (12.5%)	3 (37.5%)	2 (25.0%)	1 (12.5%)	0.0 (1.5)
Increased communication quality between clinician and patient	1 (12.5%)	1 (12.5%)	1 (12.5%)	2 (25.0%)	2 (25.0%)	1 (12.5%)	−0.3 (1.7)
Captured pertinent information	1 (12.5%)	0 (0.0%)	1 (12.5%)	1 (12.5%)	1 (12.5%)	4 (50.0%)	0.6 (1.8)
Would continue to use the survey	1 (12.5%)	3 (37.5%)	1 (12.5%)	2 (25.0%)	0 (0.0%)	1 (12.5%)	−1.0 (1.6)

SD: standard deviation.

Clinician interviews

Three nurse practitioners and two physicians participated in the one-on-one qualitative interviews (Table 3). Overall, the clinicians’ experience and views on the WHAT/CDS toolkit were positive. Specifically, the clinicians identified aspects of care that were improved as a result of the WHAT/CDS toolkit, including streamlining patient visits (n = 4), improving communication between clinician and patient (n = 4), allowing patients to feel more ownership over their health care (n = 4), and reminding clinicians of issues to address (n = 3).

By highlighting the most important issues, clinicians (n = 2) felt they were able to better meet their patients’ needs. Clinicians (n = 3) considered the toolkit to be well developed (i.e. layout, functionality, performance, usability), and easy to use. Three clinicians thought that the questionnaire covered important topics for the patients; two clinicians wanted to expand the scope and cover additional topics such as on lifestyle, diet, exercise, family history, and vaccination history.

All clinicians (n = 5) said that the most helpful features were allowing patients to prioritize their health conditions and receiving the responses to the WHAT questionnaire prior to the patients’ well-woman visit (n = 4). Clinicians suggested ways of improving the WHAT/CDS toolkit, such as adding a personalized note from the patient’s clinician in the initial message sent to patients or providing the questionnaire responses to the patients.

Clinician survey

More than half of the clinicians surveyed were neutral (“neither agree nor disagree”) or “somewhat agreed” that the WHAT/CDS toolkit made medical care delivery more efficient (n = 5). Clinicians “agreed” or “somewhat agreed” that the WHAT/CDS toolkit captured pertinent information for diagnosis and treatment of women’s health conditions (n = 5), helped patients be better prepared for their visit (n = 3), and increased the quality of patient–clinician communication (n = 3). Clinicians “somewhat disagreed” to “strongly disagreed” that the WHAT/CDS toolkit helped target patient decisions (n = 5), changed their approach to health conditions (n = 6), and would continue the WHAT/CDS toolkit (n = 5). Four of the eight clinicians provided responses ranging from “somewhat disagree” to “strongly disagree” for the remaining three questions (i.e. ease of use, increased tests or referrals, received sufficient training/support).

Discussion

Overall summary

The well-woman visit provides an opportunity for promoting the overall health of women as well as a platform in which overlooked and highly prevalent conditions can be addressed. This article aimed to describe the process of implementing an electronic, patient-integrated approach as well as the perceived benefits and difficulties in using this tool.

The WHAT/CDS toolkit was developed and implemented through a multistep process requiring time and resources from both external and internal partners as well as the active engagement of clinicians. The participation rate by women (30%) demonstrates successful implementation of the WHAT/CDS toolkit and supports the use of a home-administered web-based survey. The response rate also benefited greatly from the implementation of reminders close to the well-woman visit date, which suggests that this email reminder approach should be taken to achieve the desired patient response.

Overall, the majority of women found that the WHAT questionnaire was easy to use and facilitated their engagement in their visit and improved clinician–patient communication. Given that patient satisfaction with care is influenced by multiple factors such as a clinician’s interpersonal skills, time spent with the patient, and access to care,²⁶ the findings from the post-visit survey suggest that the WHAT/CDS toolkit improved aspects of the visit that are important to patient satisfaction. Additionally, patients’ willingness to complete the WHAT questionnaire again suggests the questionnaire was well accepted by patients.

In contrast, the clinicians’ survey responses to the WHAT/CDS toolkit were generally neutral or negative, which may be reflective of a larger issue with clinicians’ overall willingness to adopt new approaches to care. A literature review of clinicians’ perceptions on QI initiatives similar to the WHAT/CDS toolkit found that clinicians often have different definitions of QI and preconceived perceptions that these types of initiatives are ineffective, which results in reluctance to engage.¹⁵ This suggests earlier involvement of clinicians and continued engagement through the implementation process may be beneficial for future use.

The qualitative interviews, on the other hand, indicated that clinicians were generally positive about the functionality, comprehension, and usefulness of the WHAT/CDS toolkit. The WHAT/CDS toolkit was reported to help clinicians prioritize the patients’ needs and streamline the patient visit while improving clinician–patient communication. These results were reflective of studies by Wagholikar et al.²⁷ and Edelman et al.¹¹ who found that CDS assistance significantly reduced the time spent by clinicians for deciding on preventive services and chronic disease management while improving delivery of care and enhancing patient engagement.

The clinician interviews and survey highlighted the issues surrounding the complexity of clinical decision making and the range of factors in addition to treatment guidelines that influence clinical decisions.²⁸ While most clinicians felt that the WHAT/CDS toolkit captured pertinent information, they did not think the toolkit affected their clinical decision making. This is in contrast to research showing CDS systems to be effective instruments to improve adherence to clinical guidelines and improvements in clinical decision making.¹⁹ A systematic literature review of 148 studies showed increased odds of adherence to preventive care recommendations, ordering or completing recommendations, and prescribing therapies during patient visits when clinicians used a CDS system.²⁹ However, clinicians’ application and perception of integrating similar assessment tools in routine clinical practice is influenced by various views and attitudes including wanting to adhere to the medical model of health, professional inertia, and a belief that his or her own ad hoc judgment is more credible than standard health status assessments which ultimately may influence clinicians’ likelihood to attribute such improvements to these tools.^28,30–32 Additionally, for some conditions, such as VMS and urgency UI, no specific well-adopted best practices exist for clinicians; thus, multiple treatment considerations may be presented.

In general, the current research evaluating impacts on the use of patient screening and CDS systems as part of routine clinical practice suggests systems like the WHAT/CDS toolkit positively impact patient aspects of the care process.^30,33–35 However, longer term outcomes associated with the implementation of such systems have been difficult to assess. Findings from this study contribute to this small but growing body of published research on the implementation of these systems and aim to highlight implementation issues to benefit future integration of the WHAT/CDS toolkit in primary care.

Limitations

This project was conducted only at three clinical sites. Therefore, the findings cannot be generalized to other clinical sites or some healthcare systems. There was a general lack of diversity among patients with the majority of patients identifying as White. Implementation of a health survey completed by patients prior to their primary care visit/gynecology visit was new for this IDN and relied on women to log into their email or MyChart prior to their visit which may have impacted the participation rate. Only five clinicians participated in the qualitative interviews of which only one was from a primary care practice which may have a different perspective on the inclusion of the WHAT toolkit compared to clinicians from a gynecology practice. Finally, the long-term impact of the WHAT/CDS toolkit was not investigated and additional longitudinal research is needed.

Conclusion

The WHAT/CDS toolkit offers a unique approach to addressing some of the key health issues that affect women aged 45–65 years through the routine well-woman annual visits. Feedback on the usability and functionality of the WHAT/CDS from the women was positive. The toolkit facilitated greater engagement of the women in their health and enhanced the quality of the visit. Implementation of the WHAT/CDS toolkit should be viewed as one of many initiatives to enhance women’s satisfaction with care and health outcomes.

Summary points

Women’s annual health assessments or “well-woman visits” are considered particularly important for mid-life women aged 45–64 years and provide important opportunities for healthcare practitioners to screen for health concerns associated with aging and menopause.

The WHAT/CDS toolkit is a health assessment questionnaire coupled with a tool to support clinical decision.

The WHAT/CDS toolkit was developed to engage patients in their health care while standardizing the condition assessment process, and promote communication between clinicians and patients.

Implementing the WHAT/CDS toolkit into clinical practice within an integrated delivery system requires the engagement of all involved staff from information technology to the clinicians to foster a successful implementation.

Women provided generally positive feedback with approximately 70% of women reported that completing the WHAT questionnaire improved the care received from their clinician, facilitated discussions with their clinician, and helped them feel more prepared for their visit.

Qualitatively, clinicians reported that the WHAT/CDS toolkit helped streamline patient visits, improve clinician–patient communication, and enhanced patients’ ownership over their health care.

The WHAT/CDS toolkit offers a unique approach to introduce and address some of the key health issues that affect women aged 45–65 years through the routine well-woman annual visits.

Implementation of the WHAT/CDS toolkit should be viewed as one of many initiatives to enhance women’s satisfaction with care and health outcomes and contribute to the development of best practices for future implementations in primary care.

Footnotes

Acknowledgements

Formatting support for this article was provided by Kawthar Nakayima, BS. All authors reviewed and approved of the final article.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Karin Coyne and Anna Steenrod are employees of Evidera who were paid consultants by Pfizer in connection with this study and writing of this manuscript. David Gross, Canan Esinduy, Angela Kodsi, Lucy Abraham, Andrew Bushmakin, and Xuemei Luo are employees of Pfizer. Gayle Slifka was an employee of Pfizer at the time the study was conducted. Anna Araiza is a former employee of Atrium Staffing, who was a paid contractor to Pfizer for study management and writing of this manuscript. Terry Silvestrin is affiliated with MultiCare Tacoma Women’s Specialists in Tacoma, WA, USA.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by Pfizer, Inc.

References

Committee opinion No.534: well-woman visit. Obstet Gynecol 2012; 120(2 Pt 1): 421–424.

Barbieri

. Your age-based guide to comprehensive well-woman care. OBG Manag 2012; 24(10): 20.

Fitzgerald

Glynn

Davenport

. Well-woman visits: guidance and monitoring are key in this turning point for women’s health. Womens Health Issues 2015; 25(2): 89–90.

Harvard Health Publications. Why regular check-ups are still a good idea. An annual “well woman” visit has important health benefits—especially the doctor-patient relationship it fosters. Harv Womens Health Watch 2007; 14(9): 1–3.

Harris

Link

Tennstedt

. Care seeking and treatment for urinary incontinence in a diverse population. J Urol 2007; 177(2): 680–684.

Parish

Nappi

Krychman

. Impact of vulvovaginal health on postmenopausal women: a review of surveys on symptoms of vulvovaginal atrophy. Int J Womens Health 2013; 5: 437–447.

Bromberger

Kravitz

. Mood and menopause: findings from the Study of Women’s Health Across the Nation (SWAN) over 10 years. Obstet Gynecol Clin North Am 2011; 38(3): 609–625.

Dooley

Kenton

Cao

. Urinary incontinence prevalence: results from the National Health and Nutrition Examination Survey. J Urol 2008; 179(2): 656–661.

Gold

Colvin

Avis

. Longitudinal analysis of the association between vasomotor symptoms and race/ethnicity across the menopausal transition: study of women’s health across the nation. Am J Public Health 2006; 96(7): 1226–1235.

10.

Delbanco

Walker

Bell

. Inviting patients to read their doctors’ notes: a quasi-experimental study and a look ahead. Ann Intern Med 2012; 157(7): 461–470.

11.

Edelman

Lin

Doksum

. Evaluation of a novel electronic genetic screening and clinical decision support tool in prenatal clinical settings. Matern Child Health J 2014; 18(5): 1233–1245.

12.

Goldberg

Ralston

Hirsch

. Using an Internet comanagement module to improve the quality of chronic disease care. Jt Comm J Qual Saf 2003; 29(9): 443–451.

13.

Tamblyn

Ernst

Winslade

. Evaluating the impact of an integrated computer-based decision support with person-centered analytics for the management of asthma in primary care: a randomized controlled trial. J Am Med Inform Assoc. Epub ahead of print 10 February 2015. DOI: 10.1093/jamia/ocu009.

14.

Bradley

Griffin

. Staff perceptions of a Productive Community Services implementation: a qualitative interview study. Int J Nurs Stud 2015; 52(6): 1052–1063.

15.

Davies

Powell

Rushmer

. Universities of Dundee and St Andrews. Healthcare professionals’ views on clinician engagement in quality improvement: a literature review. London: The Health Foundation 2007.

16.

Davies

Powell

Rushmer

. Why don’t clinicians engage with quality improvement? J Health Serv Res Policy 2007; 12(3): 129–130.

17.

Austin

Ewing

Grande

. Organisational factors influencing uptake of a carer support needs assessment tool in palliative home care. BMJ Support Palliat Care 2015; 5(1): 115.

18.

Kok

Gurabardhi

Gottlieb

. Influencing organizations to promote health: applying stakeholder theory. Health Educ Behav 2015; 42(1 Suppl.): 123S–132S.

19.

Van Engen-Verheul

De Keizer

Van

Der

Veer

. Evaluating the effect of a web-based quality improvement system with feedback and outreach visits on guideline concordance in the field of cardiac rehabilitation: rationale and study protocol. Implement Sci 2014; 9(1): 780.

20.

Silvestrin

Steenrod

Coyne

. An approach to improve the care of mid-life women through the implementation of a Women’s Health Assessment Tool and clinical decision support toolkit. Womens Health 2016; 12(3): 313–323.

21.

Kroenke

Spitzer

Williams

. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001; 16(9): 606–613.

22.

Donker

Van Straten

Marks

. Quick and easy self-rating of Generalized Anxiety Disorder: validity of the Dutch web-based GAD-7, GAD-2 and GAD-SI. Psychiatry Res 2011; 188(1): 58–64.

23.

Isidori

Pozza

Esposito

. Development and validation of a 6-item version of the female sexual function index (FSFI) as a diagnostic tool for female sexual dysfunction. J Sex Med 2010; 7(3): 1139–1146.

24.

Brown

Bradley

Subak

. The sensitivity and specificity of a simple test to distinguish between urge and stress urinary incontinence. Ann Intern Med 2006; 144(10): 715–723.

25.

SAS Institute, Inc. SAS^® 9.4 software, 2013, http://support.sas.com/documentation/cdl/en/whatsdiff/66129/PDF/default/whatsdiff.pdf

26.

Renzi

Abeni

Picardi

. Factors associated with patient satisfaction with care among dermatological outpatients. Br J Dermatol 2001; 145(4): 617–623.

27.

Wagholikar

Hankey

Decker

. Evaluation of the effect of decision support on the efficiency of primary care providers in the outpatient practice. J Prim Care Community Health 2015; 6(1): 54–60.

28.

Hajjaj

Salek

Basra

. Nonclinical influences, beyond diagnosis and severity, on clinical decision making in dermatology: understanding the gap between guidelines and practice. Br J Dermatol 2010; 163(4): 789–799.

29.

Bright

Wong

Dhurjati

. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157(1): 29–43.

30.

Greenhalgh

Meadows

. The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract 1999; 5(4): 401–416.

31.

Katugampola

Hongbo

Finlay

. Clinical management decisions are related to the impact of psoriasis on patient-rated quality of life. Br J Dermatol 2005; 152(6): 1256–1262.

32.

McHorney

Earl Bricker

Jr . A qualitative study of patients’ and physicians’ views about practice-based functional health assessment. Med Care 2002; 40(11): 1113–1125.

33.

Boyce

Browne

. Does providing feedback on patient-reported outcomes to healthcare professionals result in better outcomes for patients? A systematic review. Qual Life Res 2013; 22(9): 2265–2278.

34.

Marshall

Haywood

Fitzpatrick

. Impact of patient-reported outcome measures on routine practice: a structured review. J Eval Clin Pract 2006; 12(5): 559–568.

35.

Valderas

Kotzeva

Espallargues

. The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res 2008; 17(2): 179–193.