Use of an Online Breast Cancer Risk Assessment and Patient Decision Aid in Primary Care Practices

Abstract

Background:

U.S. Preventive Services Task Force (USPSTF) recommendations for mammography screening, genetic counseling and testing for pathogenic BRCA1/2 mutations, and use of risk-reducing medications require assessment of breast cancer risk for clinical decision-making, but efficient methods for risk assessment in clinical practice are lacking.

Materials and Methods:

A cross-sectional study evaluating a web-based breast cancer risk assessment and decision aid (MammoScreen) was conducted in an academic general internal medicine clinic. All eligible women, 40–74 years of age without previous diagnosis of breast or ovarian cancer and who were enrolled in the Epic MyChart patient portal were invited. MammoScreen uptake and completion rates and consistency between breast cancer risk determined by MammoScreen and existing risk information in the Epic record were measured. Patient and physician experiences were summarized from interviews.

Results:

Of 448 invited participants, 339 (75.7%) read their MyChart invitation and 125 (36.9%) who read invitations enrolled in the study; 118 (94.4% of enrolled) completed MammoScreen. Twenty-one women were categorized as above-average risk from either MammoScreen data or the chart review and 7 (33.3%) were identified by both sources. Physicians and patients believed MammoScreen was easy to use and was helpful in identifying risks and facilitating shared decision-making.

Conclusions:

Breast cancer risk assessment and mammography screening decision support were efficiently implemented through a web-based tool for patients sent through an electronic patient portal. Integration of patient decision aids with risk algorithms in clinical practice may help support the implementation of USPSTF recommendations that include risk assessment and shared decision-making.

Introduction

The U.S. Preventive Services Task Force (USPSTF) recommends three breast cancer screening and preventive services based on evidence of benefits and harms.^1

–5 These include periodic mammography screening,^6,7 risk assessment and testing for pathogenic BRCA1/2 mutations,⁸ and use of risk-reducing medications for women at increased risk.⁹ Recommended services define standards of clinical care and are currently covered by most private and public health insurance plans with no copay or deductible charges under the Affordable Care Act¹⁰ and by Medicare, affecting >63 million women in the United States.¹¹

Effective implementation of these recommendations requires assessments of patients' breast cancer risks that differ for each preventive service. The USPSTF recommends biennial mammography screening for all women 50–74 years of age, and individualized screening for women 40–49 years of age based on a shared decision-making process that weighs individual risks and personal values regarding benefits and harms.^6,7 Referral for genetic counseling and BRCA1/2 mutation testing is based on familial cancer risk,⁸ and selection for risk-reducing medications is based on overall breast cancer risk.⁹ Despite the importance of risk assessment in guiding these screening and prevention decisions, there are currently no standard risk assessment methods for implementing the USPSTF's recommendations into routine clinical practice. General methods to estimate personal risk for breast cancer, such as the Breast Cancer Risk Assessment Tool or Gail model, perform poorly as predictors for individual women.⁸ Although methods to determine familial risk for breast cancer are generally fair to good predictors of BRCA1/2 mutation status,⁴ their effectiveness in routine clinical practice is not clear. As a result, clinical applications of the USPSTF recommendations often fall short of their potential.

Challenges of the clinical practice environment also limit effective implementation. Physicians lack time during patient visits to identify breast cancer risks and screening preferences that change over time. Patients are often not prepared to report family risk information, or to make decisions about screening because they lack knowledge on benefits and harms relevant to them. Information about risk may be incompletely documented in health records and not easily available at the point of care. In addition, patients face increasing direct-to-consumer pressure for newer imaging technologies and genetic testing, and may choose tests without consulting their physicians about its appropriateness or interpretation. For example, negative results from the 23andMe test for BRCA1/2 mutations may falsely assure patients who do not understand the limits of the test in estimating risk for breast cancer.¹²

To address these issues, we developed an interactive web-based risk assessment and clinical decision support tool, MammoScreen, to help women 40–74 years of age determine their personal risks for breast cancer, clarify their values and preferences, and consider screening decisions. This study builds on our previous work¹³ by implementing MammoScreen in an academic general internal medicine clinic through an electronic patient portal, and referring patients to appropriate services as indicated by their responses. The purpose of this pilot study was to determine the uptake, data accuracy, and patient and physician experiences using MammoScreen.

Materials and Methods

Development of MammoScreen

MammoScreen is a mobile web application that builds on a previously developed tool, Mammopad.¹³ Both tools help women understand their personal risks for breast cancer, priorities for screening, and screening options before discussing them with their physicians. They do not provide individual recommendations for mammography screening, genetic testing, or risk-reducing medications, but prepare women for discussions of these topics based on their responses. These tools adhere to the International Patient Decision Aids Standards for development and content and are updated as new evidence becomes available.¹⁴ The development process included interviews with patients and clinical experts to assess face validity, patient comprehension, and ease of use.^13,15

The original tool, Mammopad, was designed for women 40–49 years of age who were not screened with mammography in the previous year and without characteristics indicating increased risk for breast cancer. These include risk factors based on a systematic review of studies of women in their 40s¹⁶ (familial breast cancer syndrome, previous high-risk breast lesion on biopsy, and previous high exposure chest radiation), personal history of breast or ovarian cancer, or current breast symptoms (lump, discharge, and skin changes). Mammopad includes an assessment of familial breast cancer risk based on the Breast Cancer Genetics Referral Screening Tool (B-RST, version 2).^17,18 The B-RST is designed to guide referrals for genetic counseling to determine individual risks for pathogenic BRCA1/2 mutations and has demonstrated high accuracy in screening populations (area under the receiver operating characteristic curve [AUC] = 0.87).⁴ The B-RST result or presence of at least one of the individual risk factors described previously determine average versus above-average risk status. Mammopad intentionally does not include tools that assess overall breast cancer risk, such as the Gail model, because they are inaccurate in predicting individual risk (AUC = 0.55–0.65).⁵

Women eligible for screening advance through information modules on breast cancer and mammography containing images and an audio option. Breast cancer risk is described by text and pictographs.^13,15 A priority setting module guides users in identifying the benefits and harms of screening most important to them based on decisional conflict theory¹⁹ that recognizes the competing benefits and harms associated with each option.¹³ A customized report summarizes next steps, priorities, intentions for screening, and questions or concerns about screening options.

The updated tool (MammoScreen) was expanded to include age-specific information about screening for two age groups (40–49 and 50–74 years),^2
–4 Mammography screening decisions for women 50–74 years primarily concern whether to screen yearly or biannually. MammoScreen users receive tailored messages for follow-up care based on their specific responses (e.g., women reporting breast lumps are advised to see their physicians for timely evaluations). The internal medicine clinical team provided guidance on workflow logic, placement of MammoScreen data in the electronic health record (EHR), types of follow-up, and design of the tailored messages. For example, women identified with increased risks based on family history information were given a message to contact their physician and the clinic was also notified. Women who reported a previous breast biopsy but either did not know the result or reported the result as precancer or cancer were informed that this information was sent to their physician. All tailored messages were carefully reviewed by the clinic team and written at 8th grade reading level or lower. Figure 1 provides the decision logic underlying MammoScreen.

FIG. 1.

Decision logic of MammoScreen. Women begin by answering a series of personal health and family cancer history questions to identify increased risks for breast cancer. Answers to specific questions trigger tailored messages (hexagon) to the patient and clinical providers regarding appropriate follow-up. Women without increased risk advance to the decision aid. Ovals indicate questions collecting patient information and diamonds are questions that lead to additional questions only for patients answering yes.

Study design

In this cross-sectional pilot study, data were collected on breast cancer risk factors and breast symptoms reported by participants in MammoScreen and through chart review. Investigators interviewed patient and clinical team users about their experiences using MammoScreen. The study protocol was reviewed and approved by the Oregon Health & Science University Institutional Review Board (Protocol e7118) and the Information Security Management Group.

Settings, participants, and recruitment

Eligible women were established patients in a general internal medicine clinic at an academic medical center, age 40–74 years, English proficient, not previously diagnosed with breast or ovarian cancer, enrolled in Epic's portal, MyChart, and able to provide informed consent. All eligible participants from three physician practices were invited to use MammoScreen by a medical assistant between July 15, 2018 and December 31, 2018 through a MyChart invitation. Participants were sent a uniform resource identifier containing their encrypted medical record number (MRN). Once accessed by the participant, MammoScreen generated a unique record for the participant. Once their session was completed, MammoScreen sent an automated email to the research team indicating that a new participant report was available for review. An authorized research team member then accessed and downloaded the report through secure administrative console. The encrypted MRN on the report was then decrypted by a separate tool so the report could be uploaded to the participants' medical records in Epic by the research team, and then routed to the medical assistant for review. The medical assistant then reviewed the charts of patients who reported symptoms or who were above-average risk for appropriate follow-up. Participants were sent reminders through MyChart 2 weeks after the initial invitation and again at 1 month for noncompleters. All participants had unlimited access to MammoScreen and could access it as many times as they wished during the study period. The patients received no incentive for participation.

Chart review

To compare MammoScreen data with EHR information, a single investigator abstracted clinical data for all completers by searching several Epic tabs (pathology, imaging, and media); problem list; and history including personal medical history, family history, surgical history, and social history using the following keywords: Jewish, Ashkenazi, eastern European, family, radiation, gray (a unit of radiation), Gy, BRCA, genetic, biopsy, ovarian, oophorectomy, bilateral-salpingo-oophorectomy (BSO), breast, nipple, lump, galactorrhea, mastectomy. To check the accuracy of the abstracted data, a second investigator independently used the same technique to abstract information for 12 randomly selected participants, stratified by risk (7 above-average risk, including 4 with symptoms, and 5 with average risk). Among the 11 risk questions, the proportion of agreement between abstractors for identifying a risk factor was 100% (12/12 participants) on 9 questions, and 91.7% (11/12) on 2 questions.

Outcome variables and data sources/measurement

Primary outcomes included uptake and completion rates for using MammoScreen, measured as proportions of invited women who enrolled (consented to participate) and completed MammoScreen, respectively. In addition, breast cancer risk data reported in MammoScreen were compared with risk information documented in the EHR. Risk data included breast cancer risk category (above-average vs. average familial risk as determined using the B-RST^17,18) and presence of individual risk factors (previous high-risk breast lesion on biopsy, previous high-exposure chest radiation, and personal history of breast or ovarian cancer) or current unresolved physical breast symptom.

Secondary outcomes included patient and physician experiences with MammoScreen including times to complete the risk algorithm (interval between accessing the breast cancer risk questions and the first webpage of the decision aid) and decision aid (interval between accessing the first webpage of the decision aid and generation of the report). Patient and physician experiences were qualitatively evaluated from semi-structured interviews.

Statistical methods

Participant characteristics were compared between women who did not open the MyChart invitation, those who opened the invitation but did not proceed, and those enrolled in the study using Fisher's exact test or the analysis of variance model as appropriate. Uptake, completion rates, and agreement between MammoScreen and EHR risk data were quantified by proportions and 95% confidence intervals (CIs). Times to complete the breast cancer risk algorithm and the decision aid were summarized descriptively using median and interquartile range (IQR).

Qualitative methods

Eight randomly selected women participated in audio-recorded semi-structured interviews to evaluate their experiences using MammoScreen through My Chart. The decision aid itself has been formally evaluated in patients for understanding and use during its earlier development.^13,15 Interview participants included four in their 40s (three average risk and one with breast pain) and four in their 60s and early 70s (three average risk and one with family history of breast cancer). All clinical team members (three internal medicine physicians and one medical assistant) were also interviewed by the investigators. Two investigators independently reviewed audio recordings of the interviews, wrote summaries, and conducted thematic analysis using grounded theory methods. Final themes and conclusions resulted from consensus of the two investigators.

Results

Participant characteristics

A total of 448 women were invited through MyChart to participate. Of these, 339 (75.7%) opened the initial MyChart invitation or at least one of two reminders, and 125 (36.9% of those who opened at least one MyChart message) enrolled (completed the consent form). White women were more likely to open a MyChart invitation or a reminder (p = 0.001; Table 1).

Table 1.

Characteristics of Participants

	Patients who did not open MyChart invitation	Patients who opened MyChart invitation but not MammoScreen, “Opened”	Patients who consented and began MammoScreen, “Enrolled”	p^a	Patients who completed MammoScreen, “Completed”
	n = 109	n = 214	n = 125	p^a	All ages, n = 118
Mean age, years (SD)	57.1 (9.8)	58.2 (9.9)	60.0 (10.7)	0.083	59.7 (10.7)
Race, n (%)
White	88 (80.7)	191 (89.3)	120 (96.0)	0.001	113 (95.8)
Non-white	18 (16.5)	17 (7.9)	5 (4.0)		5 (4.2)
Unknown/declined	3 (2.8)	6 (2.8)	0		0
Hispanic, n (%)				0.469
Hispanic	5 (4.6)	6 (2.8)	2 (1.6)		2 (1.7)
Non-Hispanic	103 (94.5)	207 (96.7)	123 (98.4)		116 (98.3)
Unknown/declined	1 (0.9)	1 (0.5)	0		0
Insurance type, n (%)				0.226
Blue cross	19 (17.4)	17 (7.9)	11 (8.8)		11 (9.3)
Other Commercial	61 (56.0)	136 (63.6)	74 (59.2)		69 (58.5)
Managed care	20 (18.3)	32 (15.0)	22 (17.6)		22 (18.6)
Medicaid/Medicare	8 (7.3)	25 (11.7)	17 (13.6)		15 (12.7)
Other^b	1 (0.9)	4 (1.9)	1 (0.8)		1 (1.8)

The p-value is for testing the differences between patients who did not open MyChart invitation versus “Opened” versus “Enrolled.”

Tricare, Worker's Comp, or Agency/Grant.

SD, standard deviation.

Primary outcomes

Uptake and completion of MammoScreen

Of the 339 women who opened their MyChart invitation, 125 enrolled in the study (36.9%; 95% CI = 31.7%–42.3%), and 94.4% (95% CI = 88.8%–97.7%) of the enrolled participants completed MammoScreen (Fig. 2). Of 118 women who completed MammoScreen, 30 (25.4%) were 40–49 years of age (mean = 44.8, standard deviation [SD] = 2.9); 88 (74.6%) were 50–74 years of age (mean = 64.8, SD = 7.0); 113 (95.8%) were White, and 116 (98.3%) non-Hispanic, which is representative of the clinic population. Four (13.3% of 30) women 40–49 years of age, and 53 women (60.2% of 88) 50–74 years of age had a documented history of a mammogram in the previous 2 years recorded in the EHR.

FIG. 2.

Uptake of MammoScreen. This figure shows the number of women who opened their MyChart messages, enrolled in the study, and completed the MammoScreen study.

Match of risk data

Using MammoScreen, 47 (39.8%) completers reported a family member who had breast cancer diagnosed under the age of 50, 11 (9.3%) reported having a family member who had ovarian cancer, 8 (6.8%) reported a symptom of breast cancer, 27 (22.9%) reported a previous breast biopsy. Furthermore, 13 women were categorized as above-average risk based on reporting a personal history of a high-risk lesion of a breast biopsy, or family history of breast or ovarian cancer using the B-RST algorithm. Six women reported breast cancer-related symptoms.

From the chart review of the completers of MammoScreen, 39 (33.1%) women had a family member with breast cancer diagnosed under the age of 50, 1 (0.8%) had a family history of ovarian cancer, and 23 (19.5%) had previous breast biopsies. Fifteen women were categorized as above-average risk based on the B-RST algorithm used in MammoScreen. Overall, 21 women were categorized as above-average risk from either MammoScreen data or the chart review data, with overlap in 7 (33.3%; Table 2). Overall agreement between risk data in the EHR and MammoScreen was 88.1% (95% CI = 80.9%–93.4%), and kappa coefficient was 0.43 (0.19–0.68), indicating moderate agreement.

Table 2.

Risk Groups Based on Data from MammoScreen and Chart Review

	MammoScreen		Total, n (%)^a
	Above average risk, n (%)^a	Average risk, n (%)^a	Total, n (%)^a
Chart review
Above-average risk	7 (5.9)	8 (6.8)	15 (12.7)
Average risk	6 (5.1)	97 (82.2)	103 (87.3)
Total	13 (11.0)	105 (89.0)	118 (100)

Percentages = n/118.

Secondary outcomes

Completion times

Of 118 completers, 117 (99.2%) finished the risk assessment section of MammoScreen in one brief session (median = 3.8 minutes; IQR = 2.5–5.6 minutes; Table 3). Of 97 women without symptoms and categorized as average risk for breast cancer, 94 (96.9%) finished using the decision aid in one session (median = 2.3 minutes; IQR = 1.6–3.4 minutes).

Table 3.

Times to Complete Risk Algorithm and Decision Aid

Participants, women	Time to complete risk algorithm		Time to complete decision aid^a
Participants, women	n	Median (IQR)	n	Median (IQR)
All (age 40–74 years)	117	3.8 (2.5–5.6)	94	2.3 (1.6–3.4)
40–49 years	29	3.3 (2.5–4.4)	25	1.9 (1.3–2.2)
50–74 years	88	3.9 (2.7–5.8)	69	2.7 (1.7–4.1)

Participants at average risk for breast cancer and no symptoms.

IQR, interquartile range.

Patients' and clinical team's experiences

Eight randomly selected participants completed telephone interviews conducted by investigators. All participants agreed that MammoScreen is “highly intuitive and easy to navigate” and reported no changes in their screening intentions. Seven participants reported no technical challenges between MyChart and Mammoscreen and would recommend MammoScreen to other women who could be less knowledgeable or who need to decide about getting a mammogram. Three participants felt more informed about breast cancer screening and their options after using MammoScreen.

Three participating physicians and a medical assistant completed telephone interviews conducted by investigators. All three physicians believed that patients benefited from using MammoScreen. Two physicians suggested pairing use with an upcoming visit, and two believed MammoScreen helped identify patients at increased risk for breast cancer and patients with symptoms. Each physician reported different preferences for receiving results in the EHR, including notification for only patients with increased risk, intermittent reports with results from all patients, and direct entry of MammoScreen results into patients' medical records. The medical assistant would recommend MammoScreen for other clinics. She felt that MammoScreen is particularly useful for catching patients noncompliant with current screening recommendations and patients with symptoms who appreciated follow-up when contacted.

Discussion

This is the first study of implementation of a patient aid to assess risk and guide mammography screening decision-making using an electronic patient portal that reported patient uptake and also compared patient self-reported information with data stored in the EHR. More than 36% of patients who opened their MyChart invitations enrolled in the study and, of those, >94% completed MammoScreen. Previous research suggests that up to 37% of patients complete a requested health assessment sent by a patient portal,²⁰ although this estimate may be high because actual adoption of patient portals varies across organizations.²¹

Based on interviews, MammoScreen was well received by both patient and clinical team participants. MammoScreen differs from other risk and symptom assessment tools by providing tailored messages to the patients about important next steps. These messages were carefully crafted to engage patients, enhance clinical workflow for clinicians, and allow individual preference of notification and follow-up.

At present, electronic medical records are not structured for physicians to easily identify women with breast cancer risks, an essential component of the USPSTF recommendations. In this study, relevant information was found in different locations, often in an unstructured format. In addition, important risk information (e.g., family history, personal history of chest radiation) was not available.

Decision aids can support the shared decision-making process and implementation of the USPSTF breast cancer screening and prevention recommendations. Previously, several decision aids have been evaluated for women younger than age 50 or older than 74 years.^{22

–29} A randomized trial of a decision aid for women 39–48 of age that included breast cancer risk assessment reported improved knowledge, but no effect on the decision to begin screening.²¹ A systematic review of three randomized trials and three observational studies found that use of the aids compared with usual care reduced screening intention, but these studies did not evaluate actual screening.²⁹ Four of the six aids reviewed included a breast cancer risk assessment to identify risk as average or increased when considering screening mammography.²⁹

Limitations

This study was implemented in a patient population of active MyChart users in academic medical clinics enrolling a relatively small number of patients and only three physicians restricting the generalizability of the results. Although this study did not evaluate socioeconomic status or educational levels of users, the initial development of the aid included several rounds of testing with patients of varying educational levels. In addition, an earlier version of MammoScreen was previously evaluated with women in rural settings of varying educational levels and economic status.^13,15 In this study, we did not observe differences in user experiences or difficulties navigating between MyChart and MammoScreen.

This study was not designed to determine whether shared decision-making was achieved, whether screening intentions changed, and whether MammoScreen users received care that corresponded to their current health, values, and preferences. Finally, this study attracted a higher than expected rate of patients with family histories of breast cancer. It may be that women with increased risk were more likely to respond to invitations to use a breast cancer risk assessment tool than women in the general population. Results of this pilot study will be used to design and conduct a large-scale trial to address these issues. Future implementation efforts should expand access to patients who may encounter barriers with this approach, such as nonusers of patient portals and non-English speakers, for example.

Conclusions

Breast cancer risk assessment and mammography screening decision support were efficiently implemented through a web-based tool for patients in general internal medicine clinics using electronic portals to provide updated risk and patient history data to guide clinical decisions. Integration of patient decision aids with risk assessment algorithms in routine clinical practice may help support the implementation of USPSTF recommendations that include shared decision-making. This approach requires modification to improve access for nonusers of electronic portals.

Footnotes

Acknowledgments

The authors thank the OHSU General Internal Medicine Clinic team: Elizabeth Haney, MD, Reem Hasan, MD PhD, Abby Chesimet, Gray Winkler, and patient participants; Rachel Navarro for identifying eligible patients and designing an approved, secure information exchange between Epic and MammoScreen; Elizabeth Parker for managing the patient recruitment and Epic routing of MammoScreen reports, Madeline Mosscrop for drafting a diagram of MammoScreen decision logic, and Jane and John Beekman for editing.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, Knight Cancer Center, or OHSU.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This study was funded by an OHSU Knight Cancer Center Support Grant (P30-CA06953). Dr. Ivlev was supported by a U.S. National Library of Medicine Biomedical Informatics Training (grant no. T15LM007088) and by the Agency for Health care Research and Quality (grant no. K12HS026370). Funders had no role in the design or conduct of the study.

References

Nelson

, Fu

, Cantor

, Pappas

, Daeges

, Humphrey

. Effectiveness of breast cancer screening: Systematic review and meta-analysis to update the 2009 U.S. Preventive Services Task Force recommendation. Ann Intern Med, 2016; 164:244–255.

Nelson

, Pappas

, Cantor

, Griffin

, Daeges

, Humphrey

. Harms of breast cancer screening: Systematic review to update the 2009 U.S. Preventive Services Task Force recommendation. Ann Intern Med, 2016; 164:256–267.

Nelson

, O'Meara

, Kerlikowske

, Balch

, Miglioretti

. Factors associated with rates of false-positive and false-negative results from digital mammography screening: An analysis of registry data. Ann Intern Med, 2016; 164:226–235.

Nelson

, Pappas

, Cantor

, Haney

, Holmes

. Risk assessment, genetic counseling, and genetic testing for BRCA-related cancer in women: Updated evidence report and systematic review for the U.S. Preventive Services Task Force. JAMA, 2019; 322:666–685.

Nelson

, Fu

, Zakher

, Pappas

, McDonagh

. Medication use for the risk reduction of primary breast cancer in women: Updated evidence report and systematic review for the U.S. Preventive Services Task Force. JAMA, 2019; 322:868–886.

Final Recommendation Statement: Breast Cancer: Screening. U.S. Preventive Services Task Force. November 2016. Available at: www.uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/breast-cancer-screening1 Accessed July 17, 2019 .

Siu

. Screening for breast cancer: U.S. Preventive Services Task Force Recommendation Statement. Ann Intern Med, 2016; 164:279–296.

U.S. Preventive Services Task Force. Risk assessment, genetic counseling, and genetic testing for BRCA-related cancer: U.S. Preventive Services Task Force Recommendation Statement. JAMA, 2019; 322:652–665.

U.S. Preventive Services Task Force. Medication use to reduce risk of breast cancer: U.S. Preventive Services Task Force Recommendation Statement. JAMA, 2019; 322:857–867.

10.

U.S. Centers for Medicare & Medicaid Services. Preventive care benefits for women. Available at: www.healthcare.gov/preventive-care-women/ Accessed June 3, 2019 .

11.

Howden

, Meyer

. Age and Sex Composition: 2010. 2010 Census Briefs. US Census Bur. 2011. Available at: www.census.gov/prod/cen2010/briefs/c2010br-03.pdf Accessed July 17, 2019 .

12.

Gill

, Obley

, Prasad

. Direct-to-consumer genetic testing: The implications of the US FDA's first marketing authorization for BRCA mutation testing. JAMA, 2018; 319:2377–2378.

13.

Eden

, Scariati

, Klein

, et al. Mammography decision aid reduces decisional conflict for women in their forties considering screening. J Womens Health (Larchmt), 2015; 24:1013–1020.

14.

Volk

, Llewellyn-Thomas

, Stacey

, Elwyn

. Ten years of the International Patient Decision Aid Standards Collaboration: Evolution of the core dimensions for assessing the quality of patient decision aids. BMC Med Inform Decis Mak, 2013; 13:S1.

15.

Klein

, Watson

, Ash

, Eden

. Evaluation of risk communication in a mammography patient decision aid. Patient Educ Couns, 2016; 99:1240–1248.

16.

Nelson

, Zakher

, Cantor

, et al. Risk factors for breast cancer for women aged 40 to 49 years: A systematic review and meta-analysis. Ann Intern Med, 2012; 156:635–648.

17.

Bellcross

, Lemke

, Pape

, Tess

, Meisner

. Evaluation of a breast/ovarian cancer genetics referral screening tool in a mammography population. Genet Med, 2009; 11:783–789.

18.

Bellcross

. Further development and evaluation of a breast/ovarian cancer genetics referral screening tool. Genet Med, 2010; 12:240.

19.

Elwyn

, Stiel

, Durand

M-A

, Boivin

. The design of patient decision support interventions: Addressing the theory-practice gap. J Eval Clin Pract, 2011; 17:565–574.

20.

Wagner

, Schink

, Bass

, et al. Bringing PROMIS to practice: Brief and precise symptom screening in ambulatory cancer care. Cancer, 2015; 121:927–934.

21.

Baldwin

, Singh

, Sittig

, Giardina

. Patient portals and health apps: Pitfalls, promises, and what one might learn from the other. Healthcare (Amsterdam, Netherlands), 2017; 5:81–85.

22.

Schapira

, Hubbard

, Seitz

, et al. The impact of a risk-based breast cancer screening decision aid on initiation of mammography among younger women: Report of a randomized trial. MDM Policy Pract, 2019; 4:1–13.

23.

Hersch

, Barratt

, Jansen

, et al. Use of a decision aid including information on overdetection to support informed choice about breast cancer screening: A randomised controlled trial. Lancet, 2015; 385:1642–1652.

24.

Mathieu

, Barratt

, McGeechan

, et al. Helping women make choices about mammography screening: An online randomized trial of a decision aid for 40-year-old women. Patient Educ Couns, 2010; 81:63–72.

25.

Mathieu

, Barratt

, Davey

, McGeechan

, Howard

, Houssami

. Informed choice in mammography screening: A randomized trial of a decision aid for 70-year-old women. Arch Intern Med, 2007; 167:2039–2046.

26.

Schonberg

, Hamel

, Davis

, et al. Development and evaluation of a decision aid on mammography screening for women 75 years and older. JAMA Intern Med, 2014; 174:417.

27.

Scariati

, Nelson

, Watson

, Bedrick

, Eden

. Impact of a decision aid on reducing uncertainty: Pilot study of women in their 40s and screening mammography. BMC Med Inform Decis Mak, 2015; 15:89.

28.

Gibbons

. “The evaluation of a breast cancer screening decision aid in the community setting.” (2018). DNP ScholarlyProjects. 3. Available at: https://scholars.unh.edu/scholarly_projects/3 Accessed July 17, 2019 .

29.

Ivlev

, Hickman

, McDonagh

, Eden

. Use of patient decision aids increased younger women's reluctance to begin screening mammography: A systematic review and meta-analysis. J Gen Intern Med, 2017; 32:803–812.