Patient Perceptions of a Decision Support Tool for Men with Localized Prostate Cancer

Goal

This is a quality improvement project prior to consideration of implementation of a decision support tool in clinical practice, conducted between April and November 2019. The project was determined not to require institutional review board oversight by the Human Research Protection Program at Memorial Sloan Kettering Cancer Center. The goal of the project was to examine the ability of a Web-based decision support tool to deliver the benefits of SDM via a novel, innovative, and integrated technology to the patient. To do so, we solicited patients’ feedback and impressions of the Web-based decision support tool: whether the questions in the tool made sense to them, what they understood, how they felt about the presentation of the tool’s outputs, and what suggestions they would make for improvement.

Population and Setting

The population consisted of 21 patients presenting to a urology clinic following their diagnosis of localized prostate cancer of low or intermediate risk (Gleason score 3+3 = 6 [grade group 1] or 3+4 = 7 [grade group 2], prostate-specific antigen <20 ng/mL and clinical stage T1c-T2b) in a large metropolitan setting. The average age of the patient population in this setting is about 62 to 67 y,^12,13 and the racial distribution of male urology patients comprises 82% White, 8% Black, 5% Asian, and 6% other race. Approximately 6% are Hispanic. Purposive sampling was the primary strategy for recruitment, and a target sample size of 20 was chosen to reach content saturation for the project’s goal of assessing a patient’s perceptions and feelings while using a decision support tool when determining treatment options. Content saturation was defined as the point at which a topic was fully explored, with no new information emerging from subsequent interviews. ¹⁴

Decision Tool

The development and details of the Web-based decision aid have been described previously.^15,16 In brief, the tool was developed for patients with newly diagnosed localized prostate cancer with the goals of providing education about localized prostate cancer and treatment options, preference measurement, and personalized decision analysis, using adaptive conjoint analysis. It took participants about 15 to 20 min to walk through the tool. There is evidence that implementation of this decision tool preconsultation reduces decisional conflict and improves several measures of SDM, such as helping men feel better informed and more involved in the decision-making process. ¹⁵ In a randomized trial comparing the tool with an educational brochure about prostate cancer treatment, patients who completed the software-based preference assessment felt more certain about their treatment decisions and reported decreased levels of decisional conflict. ¹⁶

After completing the questions posed by the decision aid, the tool generates a patient personal profile based on values and preferences. This is achieved by conjoint analysis, which ranks preferences based on substitution (e.g., avoiding frequent urination > avoiding sexual dysfunction > avoiding a surgery requiring a hospital stay), in which the treatment(s) that match those preferences, together with tumor characteristics, will be ranked higher than other options using a “fit bar” (e.g., radiation therapy > brachytherapy > surgery > active surveillance). Treatments are ranked based on an expected value calculation that runs in the background using a decision analytic framework, with the quantified preferences providing the weights in the model. For conjoint analysis to accurately determine a patient’s preferences, it requires the questions be phrased for patients in a tradeoff manner. Patients were asked, “Which of these outcomes is more acceptable to you?” and then shown 2 groups of 2 to 3 health states. The patient is then asked to rate the strength of their preference on a 5-point scale. For instance, a patient might be asked to state a preference between, “After your treatment you quite frequently need to have a bowel movement with little notice. (Within six months you return to normal) AND You avoid an outpatient procedure. AND Your life expectancy is reduced by six months.” versus “Your bowel movement patterns remain the same. AND You need minor surgery, with no hospital stay required. AND You live out your expected lifespan.” The decision aid encompassed a total of 20 such questions.

Upon completion of the questions, the tool generates an individualized report displaying a values profile based on the patient’s values and preferences and lists the suggested treatment options. This report is then used during the consultation with the treating urologist.

Data Collection

Usability testing using cognitive interviews occurred in person in a clinic room. Cognitive interviewing is a qualitative approach that is used to assess the content validity of patient-facing materials. ¹⁷ Patients were introduced to the quality improvement project by their treating urologist, who explained the purpose and then left the room. Upon confirming verbal interest to participate in the project, patients were joined by 1 or 2 members of the evaluation team and given an electronic tablet to walk through the decision support tool. Interviews were conducted by 1 trained interviewer on the team, with or without another team member observing.

As per observational research methodology, jottings in the form of snippets of verbatim conversations were taken continuously by 1 or 2 observers during the interviews, serving as memory aids to document patient comments, ¹⁸ following a semi-structured interview script with open-ended questions regarding their overall experience of using the decision support tool in general (for example, “What features of the tool did you like the most/least?” and “Are there any questions you would like to add/remove?”) and its usefulness more specifically (for example, “Would you have found a decision aid like this useful when making your decision about treatment for prostate cancer and would you recommend this tool to a friend?”). Further probes (proactive and reactive) were used to elucidate emotional states experienced while using the tool (for example, the proactive probe: “Is there anything else that, in your opinion, would be important for this tool to ask, but was not included?” or the reactive conditional or emergent probes in which the interviewer would react to the participant’s behavior and responses during the think-aloud interview as the participant was reacting to the questions in the tool). Immediately after the conclusion of each interview, the jottings were written out as detailed field notes by the interviewer in a standardized interview summary report of patient feedback while referencing direct quotes that were jotted down. The structured interview report template consisted of the following labels: participant number, date, time of interview, observer’s post-interview thoughts, interview summary, and notable quotes from interview. The snippets of conversations were turned into detailed field notes after the interview. Interviews were not audio-recorded because of the observational methods used and to ensure anonymity of participants in this quality improvement project.

Analysis

We analyzed the summary reports using inductive thematic content analysis to examine themes and patterns of meaning within text data, using predefined codes derived from a subset of 5 interview scripts, then applied to all interview scripts. ¹⁹ Analysis of interviews was performed independently by 2 members of the project evaluation team. Both members were completely independent from the conception, development of aims, and conducting of interviews. Interview reports were reviewed independently by the 2 project team members and hand-coded to identify the code structure. Interviews were analyzed in groups of 5 to 6 patients, and the team members met for weekly consensus meetings during ongoing analysis to discuss and resolve any discrepancies in coding as well as discussed areas of convergence and divergence between participants and their responses. A summary consensus document was prepared for each meeting with key reference quotes included for each section. After coding and consensus discussion was complete, a secondary analysis was completed by combining the consensus documents and reviewing each section for common themes. Consistent with standard practice for qualitative interview analysis, feedback mentioned by at least 3 participants were included as a common theme. ²⁰ After the completion of 4 consensus meetings, all consensus documents were combined into a larger convergence document that was shared and discussed with the quality improvement project team.

Theme 1: Patients had a Negative Emotional Reaction to the Tool, Pointing out the Unnecessarily Negative Framing and Language Used

Patients felt overwhelmed by multiple different options being presented at a single time and what was felt as unnecessarily negative language. Some language used in tradeoff questions was perceived to induce negative emotional states over and above that which is inevitable when discussing a medical decision with important tradeoffs. One patient noted the “extreme language” of some of the questions, for example, the usage of words “permanent” and “no improvement over time.” Upon further questioning, he explained that the “maximizing life span and losing one year” questions were “fear-inducing.” Another patient confirmed this feeling: “The wording makes me nervous. It hits you on the nose and I was just diagnosed three weeks ago.”

The patients’ main negative reaction concerned the choice between symptoms. The language was thought to be scary and overwhelming, as it was a matter of choosing between “the lesser of two evils.” Patients disliked being presented only negative outcomes in relation to their care. One patient spontaneously stated, “These questions suck!” When asked for clarification, he referred to the tradeoff questions being difficult to think about and choose between, because he felt that 2 different outcomes would both “suck.” Several other patients shared the same experience. One patient said, “These are bad questions. They are almost like psychology questions. It’s like picking the [lesser] of two evils.”

Theme 2: Patients were Forced to Stop and Think about Values and Preferences while going through the Tool and Found this Process of Deliberation to be Useful

Patients reported that their knowledge of prostate cancer treatments and side effects increased after completing the decision aid. A patient who had come to the urology clinic for a second opinion said, “I felt like the tool helped recall bits and pieces of what I had already learned. It tested my knowledge.”

This recollection of knowledge went hand-in-hand with patients reporting that the tool prompted a deeper consideration about the decision-making process of their treatment options. When probed about this sentiment, one patient said, “It was good to think this through and put my mind in the place of the things that can happen.”

This deeper consideration led another patient to state, “It was useful for quality of life questions since doctors don’t always ask these questions or have the time during appointments.” The same patient went on to describe the usefulness of the experience of completing the tool to “get the mind going,” prior to an in-depth discussion with their clinician.

However, not all patients found the tool useful in terms of prompting consideration of values and preferences. One patient illustrated difficulties with needing to make a decision, saying that a tool like this “wouldn’t be for me” as he preferred to talk to people (doctor, family members, etc.) about his different options, rather than a tool. The patient thought the tool failed to consider the human experience of decision-making.

Theme 3: Patients were Confused by the Tool

While some patients found the process of deliberation to be useful, others were confused by the tool and had a more difficult time answering certain tradeoff questions.

Theme 4: Patients Tried to Discern the Intent of the Questions

By having to trade off between the listed health states and possible symptoms of each question, patients felt they needed to deconstruct the true meaning and purpose of the questions. With further probing, some patients reported that the intent of the tool felt unclear to them due to its overall design. This questioning of the intent was often presented as general confusion, and patients wondered whether the primary purpose of the tool was for education, for helping to make a treatment decision, or simply a discussion guide. Patients inquired curiously about the development and design of the tool. One patient wondered how this tool would apply to his choice of treatment, stating, “Will it matter?”

Building on patients’ assumed responsibility to infer the intent of the tool, one patient attempted to understand the meaning behind the questions and tease out whether specific side-effects presented pertained to certain treatments: “Oh, so these questions are sort of binary options? So, this is like surgery or radiation? This is definitely asking about surgery versus radiation.”

Theme 5: There was a Disconnect between Patients’ Negative Reactions while Using the Tool and the Contrasting General Satisfaction of the Final Values Profile

Upon completing the tool, the patient was shown their personal values profile. The project evaluation team member asked each patient, “Do you think that this values profile aligns with your values and preferences?” Intriguingly, while most patients had expressed confusion and skepticism while using the tool, almost all nevertheless answered positively, a simple “yes” or “this seems about right.” They felt that the tool was accurate in its elicitation of their values and preferences in relation to the various side effects from the treatment options available. The generation of a values profile was seen as useful information that allowed for further conversations to be initiated between patients, their providers, and caregivers. Many patients indicated that they thought this feature would be helpful and beneficial for future patients and said, “This [values profile] would be helpful for future patients.” That is, in terms of values and preference elicitation, patients expressed the tool had a general sense of accuracy, in relation to the various side effects from the treatment options available. One patient said, “It helped me identify my values, that I desire less side effects such as urinary, sexual, and bowel dysfunction that I don’t want to experience.”

Strengths and Limitations

Strengths of this project include the requirement that participants have the decision aid on hand, as opposed to having participants try to remember the aids. As described above, some studies interviewed participants about their experiences using the decision aid weeks after they had used them and sometimes even after they had already made a treatment decision. We believe that our approach allowed us to obtain more accurate assessments of participants’ true experience of using the decision aid. Moreover, many studies used quantitative surveys to ask participants about their experiences, which did not allow for many open-ended or spontaneous responses. In contrast, we used a qualitative interview design with the think-aloud interviews, which allowed participants to describe their experiences in their own words as they reviewed the decision aids for the first time. Our approach allowed us to obtain richer data by observing the participants’ process and deliberation throughout the qualitative interviews, as opposed to reading what participants had to say about the experience in a free text box on a questionnaire.

It should also be mentioned that the contemporary version of the tool has undergone edits and revisions since the time these patients were interviewed. There are now screens that introduce medical concepts prior to rating exercises, and the partial profile comparisons involving 3 separate outcomes—explicitly described as confusing by our participants—have been removed.

With regard to limitations, we note that field notes rather than audio recordings were taken; however, we were able to capture near-verbatim quotes by 2 interviewers taking continuous hand-written jottings of snippets of conversations and completing the interview report immediately after the interviews. We did not collect any sociodemographic, education, or health literacy information from participants other than the average age, race and ethnicity, and clinical tumor characteristics of the typical patient population in this setting. There was also a lack of a Spanish version of the tool; 1 participant’s daughter translated it for him, which is not the same experience as reading. Participants may also not have felt as open to sharing their thoughts and honest opinions about the decision aids to the project evaluation team; they might be more comfortable and willing to provide feedback in a more neutral setting to observers whom they did not connect with the tool.

We also observed that having the patients walk through the tool prompted spontaneous interaction with accompanying caregivers. Future studies are encouraged to further study the use of decision aids in increasing dyad interactions regarding important, preference-sensitive medical decisions.

Finally, we note that some of the concerns raised by patients, particularly about being faced with binary choices, reflect the framework of conjoint analysis, which is not a feature of many decision aids; thus, some of these concerns may not be generalizable to other tools. The tool tested in this study was a multicomponent intervention, and some features may be more desirable than others by patients.