The Development and Pilot Study of a Multiple Criteria Decision Analysis (MCDA) to Compare Patient and Provider Priorities around Amputation-Level Outcomes

Abstract

Background. Patients with chronic limb-threatening ischemia who are facing a lower-limb amputation often require a transmetatarsal amputation (TMA) or a transtibial amputation (TTA). A TMA preserves more of the patient’s limb and may provide better mobility but has a lower probability of primary wound healing relative to a TTA and may result in additional amputation surgeries. Understanding the differences in how patients and providers prioritize key outcomes may enhance the amputation decisional process. Purpose. To develop and pilot test a multiple criteria decision analysis (MCDA) tool to elicit patient values around amputation-level selection and compare those with provider perceptions of patient values. Methods. We conducted literature reviews to identify and measure the performance of criteria important to patients. Because the quantitative literature was sparse, we developed a Sheffield elicitation framework exercise to elicit criteria performance from subject matter experts. We piloted our MCDA among patients and providers to understand tool acceptability and preliminarily assess differences in patient and provider priorities. Results. Five criteria of importance were identified: ability to walk, healing after amputation surgery, rehabilitation intensity, limb length, and prosthetic/orthotic device ease. Patients and providers successfully completed the MCDA and identified challenges in doing so. We propose potential solutions to these challenges. The results of the pilot test suggest differences in patient and provider outcome priorities. Limitations. The pilot test study enrolled a small sample of providers and patients. Conclusions. We successfully implemented the pilot study to patients and providers, received helpful feedback, and identified solutions to improve the tool. Implications. Once modified, our MCDA tool will be suitable for wider rollout.

Highlights

Patients and providers have successfully completed our MCDA, and patients feel the MCDA may be useful in clinical practice.

We encountered several methodologic challenges and identified approaches to ease participant burden.

When data are sparse, using the Sheffield elicitation framework is helpful in creating a performance matrix, although patients relied largely on their amputation experiences to complete the exercise. Blinding the alternatives may help patients better understand the process.

Keywords

shared decision making multi-criteria decision analysis lower extremity amputation patients priorities providers priorities

Patients with chronic limb-threatening ischemia (CLTI) may require a lower-limb amputation to resolve the sequalae of diabetes and/or peripheral artery disease. In patients with localized forefoot disease, common amputation strategies include either a transmetatarsal amputation (TMA) or transtibial amputation (TTA).¹ The number of TMAs performed in the Veterans Health Administration tripled between 2005 and 2014; this is potentially driven by providers assuming that TMAs yield better mobility because the ankle joint is preserved.^2–4

Despite the increase in the number of TMA surgeries performed, there is insufficient evidence that TMAs provide a higher quality of life than TTAs.^5,6 Nearly a quarter of TMAs fail to heal properly, resulting in additional pain, a longer time to mobility restoration, and a risk of future amputation revisions.² Patients with a failed TMA amputation have reported negative impacts on their lives during the period of failed wound healing, including increased pain and fear. They have also expressed a need for more information during and immediately following their amputation.⁷ The Veterans Affairs and Department of Defense Lower Extremity Amputation Clinical Practice Guidelines recommend shared decision making for CLTI amputees whose clinical status suggests either a TMA or TTA. Despite this, patient involvement in the amputation-level decision process varies widely across providers.⁸ It is unclear whether providers and patients share in decision making through a discussion of patient priorities or whether providers make decisions based on their perception of patient priorities.

Multiple criteria decision analysis (MCDA) is a collection of methods based in decision theory. These methods aim to help individuals who are making a complex decision, such as amputation level, understand which outcomes in the decisional problem matter most to them.^9–11 MCDAs use a formal, explicit process to evaluate the priorities of an individual or group, which in turn improves the transparency and lowers the cognitive burden of a decision.¹¹ MCDAs are increasingly used to aid health care decision making because these decisions are complex, oftentimes with competing goals. MCDAs are currently being used in health technology assessment, clinical guideline development, portfolio decision analysis, and shared decision making.¹¹

Although the use of MCDAs in shared decision making is limited, MCDAs have been shown to improve patient-provider communication and help patients better understand their own priorities.¹² Existing MCDAs have been used to compare patient preferences with provider preferences for rehabilitation after surgery as well as assist patients in deciding on cancer screening and disease treatment courses.^12–16 These MCDAs frequently use pairwise comparison and rank order methods to assess preferences, as these methods yield a lower cognitive burden to patients than other methods, such as discrete choice experiments.^12,13,17,18 To date, no study has compared patient priorities and provider perceptions of patient priorities with respect to potential patient outcomes associated with differing amputation levels.

The aim of this pilot study is to develop an MCDA for a future, larger, clinical study to compare the priorities and preferences of patients with providers around amputation outcomes. This study is part of a larger research program that aims to facilitate shared decision making around amputation levels through the development of decision-making tools. Our goal is to develop an MCDA that 1) encompasses all patient values without a cumbersome number of criteria, 2) presents a low cognitive burden to patients, and 3) accurately represents clinical scenarios that patients face. This article reports the approach we used to develop and pilot test our MCDA and provides preliminary results that will be validated in our larger study.

Methods

MCDA Overview

We developed the MCDA according to the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) guidelines. The steps to develop an MCDA are as follows: 1) defining the decision problem and the alternatives, 2) identifying and structuring criteria important to the decision, 3) measuring the performance of the alternatives with respect to the selected criteria and placing them in a performance matrix, 4) scoring the alternatives, 5) weighting the criteria, 6) calculating aggregate scores, 7) dealing with uncertainty, and 8) reporting and examining the findings.¹¹ We describe our steps in detail below.

Step 1: Defining the decision problem and the alternatives

The decision problem in our MCDA is choosing an amputation level, and the alternatives are a TMA and a TTA.

Step 2: Identifying and structuring criteria important to the decision

We conducted a rapid review of the qualitative literature to identify and structure criteria important to patients when considering amputation and amputation level.^19,20 Marsh et al.²⁰ defines criteria as the factors on which the decision is evaluated. We used existing studies and input from clinical subject matter experts and librarians from the University of Washington to help define our search terms and search strategies.^6,21 We included articles in our review that 1) used qualitative research methods; 2) focused on patient values; 3) ascertained values from CLTI patients with a TTA, TMA, or partial foot amputation; 4) were conducted in a high-income country; and 5) were written in English. We reviewed the qualitative literature from January 2010 through June 2019 to identify criteria important to patients. We extracted the study purpose, sample characteristics, and main themes from each article that met our inclusion criteria (Appendix A). Two members of the research team coded each theme and aggregated the coded themes to identify overarching criteria that are important to patients after amputation. We then created definitions for the criteria using a synopsis of the coded themes. We structured the definitions so that the criteria met MCDA requirements for completeness, nonredundancy, nonoverlap, and preferential independence.^11,20 We modified the definitions to create a plain-language version for the patients and a clinical version for the providers. Both the patient and provider definitions were reviewed by the team to ensure that the definitions included the same constructs. Subject matter experts reviewed and revised the aggregation of criteria and definitions.

Step 3: Measuring the performance of the alternatives with respect to the selected criteria

We conducted a rapid review of the quantitative literature to measure the comparative performance of TMA and TTA for the criteria identified during the qualitative literature review. These criteria performances were then aggregated into a table. This table is often referred to as the performance matrix. The performance matrix is used to help participants easily compare the performance of each alternative during an MCDA.²⁰

We reviewed the quantitative literature from December 2015 to June 2019 to create the performance matrix. Because Dillon et al.⁷ conducted a systematic review of amputation outcomes through December 2015, we screened all citations within Dillon et al.⁷ for inclusion in our rapid review. This allowed our results to span a longer time horizon without duplicating work that had been previously published.

Articles met our inclusion criteria if the article 1) provided a comparison between a TTA and TMA, 2) used a randomized control trial or a cohort study design, 3) included only unilateral incident amputations caused by CLTI, and 4) presented an outcome of interest within 5 y postamputation. From this review, 5 articles met our inclusion criteria; these 5 articles provided quantitative data for only 1 of the criteria identified in the qualitative review (Appendix B).^4,22–25 We then conducted a modified Sheffield elicitation framework (SHELF) to inform our performance matrix because the quantitative literature yielded few comparative results for TMA and TTA for the criteria identified in the qualitative review.

Sheffield elicitation framework (SHELF)

SHELF is a method used to elicit expert opinion on probability distributions when the evidence base is sparse and has been used in prior studies to assess comparative outcomes.^26,27 To conduct SHELF, we framed each of our MCDA criteria as a probability (e.g., probability of healing) and designed our SHELF exercise in Microsoft Excel using the Roulette method.²⁷ In the Roulette method, we gave 100 “coins” to each participant and asked the participant to spend the coins across a decile distribution based how likely they believed the true value of the criteria was in each decile.

To ensure that the tool was understandable, we piloted our SHELF to a convenience sample of 5 clinical experts. A secondary goal of the SHELF pilot test was to receive additional feedback on our MCDA criteria and the criteria definitions. We piloted the SHELF tool to experts who are involved in amputee care, including individuals who practice physical medicine and rehabilitation (PM&R), physical therapy, vascular surgery, podiatry, and prosthetics. We conducted video-enabled interviews with providers who consented to our study using a talk-aloud framework and asked participants to share their thoughts and initial reactions to the SHELF tool. The feedback in the pilot test was consistent and required only minor changes to our criteria definitions with no major concerns regarding the tool structure or criteria.

We then deployed the SHELF tool to a larger convenience sample of clinical experts to obtain values for our performance matrix. Our sample included 8 providers in the following specialties: PM&R physician (1), physical therapist (1), vascular surgeons (2), podiatrists (2), and prosthetists (2). These provider specialties were chosen based on their integration with patient care during and immediately after amputation.

We conducted SHELF in 2 stages. First, a member of the study team walked through the SHELF exercise with each consenting participant via a video-enabled call. After all interviews were complete, we summed the responses in each decile to obtain an aggregated distribution. Second, we sent the aggregated distribution to each participant along with their original responses and allowed participants to revise their responses if they wished. We estimated a single point value for each criterion by taking a weighted average across the final distribution using the midpoint of each decile. We sent the point values to participants for final approval before creating our performance matrix.

Steps 4 and 5: Scoring the alternatives and weighting the criteria

To obtain preferences for each alternative, MCDAs must assess both the preferences for criteria and the performance of each alternative with respect to the criteria. Assessing preferences for criteria is referred to as the “weighting exercise,” and assessing the performance of each criterion is referred to as the “scoring exercise.”²⁰ These 2 steps can be conducted separately using what is called a compositional MCDA method (e.g., analytic hierarchy process [AHP]) or in a combined fashion using a decompositional MCDA method (e.g., discrete choice experiment).²⁰ To complicate things further, when using the compositional method, each step can be completed using a unique or the same MCDA method.

We chose to use AHP to conduct both scoring and weighting; thus, we conducted steps 4 and 5 in a combined fashion. We chose AHP for our MCDA because it uses pairwise comparisons in the scoring and weighting exercises and thereby imposes a lower burden of cognitive stress when compared with other MCDA methods that require participants to compare more than 2 criteria at a time.¹⁸ AHP uses eigenvectors to assess preferences for criteria and alternatives.²⁸ We used an existing software package, ExpertChoice, to develop our MCDA (Expert Choice, 2022, Arlington, VA, USA).

Step 6: Calculating aggregate scores

Consistent with Saaty’s AHP model, we created judgment matrices based on responses to the weighting and scoring exercises; the eigenvectors of these matrices represent criteria and amputation-level preferences.^28,29 We followed the existing literature and first aggregated to patient-level and provider-level comparisons by calculating the geometric mean across individual pairwise comparisons for each scoring and weighting question.^17,30 We then used these geometric means to create patient-level and provider-level weighting and scoring judgment matrices and estimated the eigenvector associated with the maximum eigenvalue for each matrix. The eigenvectors were normalized so each vector summed to 1. The normalized eigenvector from the weighting exercise represents the preferences for each criterion. The normalized eigenvectors from the scoring exercise were multiplied by the weighting exercise eigenvector to estimate final preferences for each amputation level. We then graphed patient and provider criteria-specific scores and amputation-level scores to understand differences in values.

We additionally estimated a measure of internal consistency for each participant’s judgments, referred to as the consistency ratio. The consistency ratio measures how consistent each participant’s judgments are as compared with a sample of purely random judgments. Perfectly consistent judgments have a consistency ratio of 0, and larger consistency ratios represent less consistent judgments.

Steps 7 and 8: Dealing with uncertainty and reporting and examination of findings

The goal of our study was to pilot test the MCDA to determine tool suitability. We did not conduct uncertainty analyses. Patient and provider preferences from the pilot test are reported in this article.

MCDA Pilot Test

We piloted our MCDA within the national Veterans Health Administration health system among a national sample of patients who had had a TMA or TTA and among a convenience sample of providers who interact with patients during or immediately after the amputation process. The goal of the pilot test was to ensure our MCDA was acceptable and easy to understand before we administered the MCDA in a larger research study to understand differences between patient preferences and provider beliefs of patient preferences. We created separate versions of the MCDA for patients and providers; these differed solely in the instructions provided and words used in the criteria definition. These components were altered in each version of the MCDA to make the MCDA more comprehensible to its respective population. Notably, the patient MCDA instructed patients to think about their personal values, whereas the provider MCDA instructed providers to think about what their patients value.

Because patients may not have access to the internet, we also adapted the patient MCDA into a paper format for easier administration. We then piloted our MCDA to patients and providers to better understand tool acceptability. Appendix C contains an example question from the patient and provider MCDA.

Patient pilot test

We used the Veterans Affairs Corporate Data Warehouse (CDW) to identify postamputation patients who had undergone a primary TMA or TTA attributable to CLTI within the past 3 y. We piloted the MCDA to postamputation patients because these patients have experienced the amputation process and the outcomes around their amputation level; therefore, they are better able to provide feedback about whether the MCDA encompassed important priorities and preferences than a patient who has not yet experienced an amputation.

We identified a convenience sample of patients from the CDW who had participated in another part of our larger study and met our inclusion criteria. We sent an information packet to these patients that included background information on our study and a paper version of our MCDA. We followed up with a phone call if patients did not respond to our recruitment packet.

We conducted a 30-min phone interview with each patient who agreed to participate and consented to be in the study. During the interview, we walked the patient through the MCDA and collected their responses to the exercise. To ensure patients fully understood the exercise, we began each interview with an example AHP question. The example asked the participant to choose between 2 car colors and explained how to rate their preferences using the AHP scale (Appendix D). We also asked open-ended supplemental interview questions to understand the patient perspectives of the MCDA exercise suitability for use in clinical practice. A single study staff member led the interview and took notes during the interview process. To maintain patient privacy, patient interviews were not recorded. Appendix E contains our supplemental interview questions.

Provider pilot test

To pilot the MCDA, we identified a convenience sample of PM&R physicians and podiatric surgeons at the Veterans Affairs who were familiar to study team members. We included providers who perform or consult on at least 5 amputations per year or care for at least 5 postamputation patients per year. We contacted providers by email, provided an introduction to our study, and asked if the providers were interested in participating. We followed up with each provider twice via email if the provider did not respond initially.

We conducted a 30-min video call interview with each provider who agreed to participate and consented to be in the study. Interviews were recorded with permission from the provider. All interviews were conducted with a single study staff member, who led the interview and took notes. At the end of the interview, the study staff reviewed the recording to ensure all relevant feedback was captured in the notes. Providers completed the MCDA online during the interview while sharing their screen and were encouraged to talk aloud as they completed the exercise. We also asked supplemental interview questions to understand provider perspectives of the suitability of the MCDA in clinical practice. Appendix F contains our supplemental interview questions. For all analyses, we used Stata (StataCorp, 2019, Stata Statistical Software: Release 16, StataCorp LLC, College Station, TX, USA). This study was funded by a Veterans Affairs Puget Sound research grant and approved by both the Veteran’s Affairs Institutional Review Board (IRBNet 16316080) and the University of Washington Institutional Review Board (ID: RNI000001596).

Results

MCDA Criteria

Our review of the qualitative literature yielded 254 themes from 18 publications.^21,31–47 These 18 publications explored varied research questions and had varied study time horizons that led to diverse themes in each article. Study questions varied from exploring the lived experiences of people who have undergone a lower-extremity amputation to describing changes in sexual functioning and sexual well-being among lower-extremity amputation patients. Study time horizons ranged from 1 wk to 2 y postamputation. The varied time horizons led to themes that spanned the full postamputation patient experience, leading themes in some articles to be time dependent on themes identified in other articles. Through multiple iterations of coding, we narrowed the themes to 5 final criteria (Table 1). These themes are the ability to walk after amputation, healing after surgery, rehabilitation program intensity, ease of the prosthetic or orthotic device, and limb length after amputation. We carefully crafted the criteria and their definitions to ensure the criteria met all requirements for a well-designed MCDA according to the ISPOR MCDA Emerging Good Practices Task Force reports.^11,20

Table 1

Multiple Criteria Decision Analysis Criteria, Definitions, and Corresponding Performance Matrix Developed from Qualitative Literature Review and the Sheffield Elicitation Framework (SHELF)

Patient Outcome	Patient Definition	Provider Definition	Performance Matrix (SHELF Results) (25th–75th Percentile Ranges)
Patient Outcome	Patient Definition	Provider Definition	TMA	TTA
Ability to walk	This is the ability to return to walking within the first year after amputation, either inside or outside the house without the help of someone else (whether or not canes, crutches or a walker are required).	This is the patient’s ability to return to walking within the first year after amputation, inside or outside the house without the help of someone else (whether or not canes, crutches or a walker are required).	75% chance of walking (55%–85%)	65% chance of walking (45%–75%)
Healing after amputation surgery	Healing after amputation surgery means a shorter time between the amputation surgery and being able to be fit with a prosthetic/orthotic device. If the incision doesn’t heal, oftentimes that means additional specialized wound care, hospitalizations, and possibly another amputation surgery at the same or higher level on your leg.	Primary healing of the amputation incision means healing after amputation within a reasonable time frame, while delayed or nonhealing implies the need for prolonged wound care, additional hospitalizations, with an associated delay in prosthetic/orthotic fitting and/or need for an additional amputation surgery at the same or higher level.	55% chance of healing (35%–65%)	75% chance of healing (65%–85%)
Low-intensity rehabilitation program	Rehabilitation is the process of increasing strength, balance, and endurance, as well as learning new skills to be as independent as possible. How many treatments you require, how long you need to be treated, and the time commitment involved differs between amputation levels. Rehabilitation usually requires visits to the hospital or clinic for therapy and may impose a burden to you and your family; rehabilitation takes time and is hard.	Rehabilitation is the work associated with undergoing a rehabilitation program after amputation to restore activities of daily living (ADLs) and mobility. This includes the time commitment, number of visits, the need to travel to a rehabilitation facility, or the time required in an inpatient rehabilitation program.	85% chance of a low-intensity rehabilitation program (55%–85%)	15% chance of a low-intensity rehabilitation program (5%–35%)
Ease of prosthetic/orthotic device use	The difficulty in being fit with a prosthetic/orthotic device, learning to use it, putting it on and taking it off on a day-to-day basis, and maintaining it. Some prosthetic/orthotic devices require more work, time, and energy, and some require less.	The degree to which learning to use the prosthesis/orthosis is burdensome, the ease of putting it on and taking it off, and the time and energy required for ongoing fitting and maintenance.	85% chance of ease of prosthetic/orthotic device (75%–95%)	55% chance of ease of prosthetic/orthotic device (35%–65%)
Limb preservation	Amputation surgery removes a part of your limb. Depending on your amputation level, more or less of your limb will be removed. How important saving as much of the limb as possible differs between people.	Amputation surgery removes a part of the limb. Depending on the amputation level, more or less of the limb will be removed.	92% of original limb preserved (85%–99%)	67% of original limb preserved (60%–74%)

MCDA Performance Matrix

The results from the SHELF exercise are displayed in Table 1. TMAs outperformed TTAs on all criteria except healing after surgery. TTAs had a 20-percentage-point higher chance of healing when compared with TMAs. In contrast, TMAs outperformed TTAs on ability to walk (10-percentage-point difference), rehabilitation program intensity (70-percentage-point difference), ease of prosthetic or orthotic device (30-percentage-point difference), and percentage of original limb preserved (25-percentage-point difference). Rehabilitation program intensity yielded the largest difference between the amputation levels; TMAs were expected to have an 85% chance of having a low-intensity rehabilitation burden, while TTAs were expected to have a 15% chance of a low-intensity rehabilitation burden.

MCDA Pilot Test

Key comments on MCDA

We recruited 6 patients and 3 providers to pilot the MCDA (Table 2). Table 3 delineates the feedback we received from patients and providers. Solutions to challenges faced are outlined below. Patients felt comfortable completing the MCDA and thought it would be helpful for preamputation patients and wished they would have been provided with the same information before their surgery. Providers were silent on these issues.

Table 2

Description of Patient and Provider Pilot Test Participants

Parameter	No. (%)
Patient participants
Male	6 (100)
Age, y
50–59	2 (33)
60–69	1 (17)
70–79	3 (50)
80+	0 (0)
Race
White	4 (67)
Black or African-American	2 (33)
Ethnicity
Non-Hispanic	6 (100)
Hispanic	0 (0)
Education
Partial high school	0 (0)
High school	2 (33)
Some college	4 (67)
College graduate	0 (0)
Provider participants
Gender
Male	2 (67)
Female	1 (33)
Surgical specialty
Physical medicine and rehabilitation	1 (33)
Podiatric surgeon	2 (67)
Average years of practice	17.7

Table 3

Feedback from Patients and Providers during the Multiple Criteria Decision Analysis (MCDA) Pilot Test

Theme	Patients (n = 6)	Providers (n = 3)
Comfort in completing the MCDA	Felt comfortable completing the MCDA and reported no major issues in completing the exercise. (6 patients) No patient expressed discomfort with completing the exercise.	No feedback provided
Utility of MCDA exercise	Believed that MCDA would be helpful for preamputation patients as the criteria in the exercise can help orient patients in what to expect after an amputation. (1 patient)	No feedback provided
Utility of performance matrix	Appreciated the performance matrix because it illustrated differences between amputation levels. The participant was not informed of key differences in TMA and TTA outcomes before their surgery and wished the performance matrix had been available to them preamputation. (1 patient)	No feedback provided
Wording of instructions	No concerns expressed (6 patients)	Commented that the instructions were too vague for the providers to understand the exercise (3 providers)
Wording of questions	No concerns expressed (6 patients)	Felt that the weighting section questions were hard to understand as the questions gave multiple instructions in 1 sentence (1 provider)
Performance matrix used in scoring exercise^a	Patient 1: Used his own personal experiences with amputations during the scoring process rather than the performance matrix, even after we repeatedly guided the participant back to the performance matrix. Based on his personal experiences, this patient felt that TTAs were superior to TMAs and selected TTAs for each scoring question even when the evidence in the performance matrix suggested TMAs outperformed TTAs (e.g., limb length). Patient 2: Struggled to understand the scoring exercise with the TMA/TTA labeling in the questions and performance matrix. This participant was confused that we were asking him about TTA amputations as well as TMA amputations given that he had only had a TMA. We needed to provide this participant with additional support in understanding the scoring exercise and guide him to use the performance matrix a few times during the scoring exercise.	Did not use the performance matrix during the scoring exercise. Rather, these providers used their clinical experiences and opinions to respond to the scoring questions. (3 providers)
Framing of rehabilitation burden criteria^a	Struggled to understand what was being asked in the rehabilitation burden scoring question due to the reverse scoring and ended up skipping that question. This participant was able to answer all other scoring questions. (1 patient)	Found the wording of the low-intensity rehabilitation program scoring to be cumbersome and not intuitive (2 providers)

TMA, transmetatarsal amputation; TTA, transtibial amputation.

Wording differed between patient and providers and was purposefully chosen to facilitate understanding by each type of participant.

Although patients did not comment on the wording of instructions or questions, providers commented that the instructions were too vague and too complex.

Proposed Solution: Split the questions into simplified subquestions to help participants break down into simpler steps the task being asked of them. In addition, add the example AHP question that we used in the patient MCDA to the provider MCDA.

Two patients called upon their own experiences with amputations rather than using the performance matrix when completing the scoring exercise. Similarly, providers used their clinical experiences and opinions to respond to the scoring questions.

Proposed Solution: Blinding the level of amputation in the scoring exercise in the instructions, performance matrix, and scoring questions so participants are required to review the performance matrix rather than rely on personal experiences.

A few patients and providers struggled with how to score the “low-intensity rehabilitation” criteria, as they felt that the question wording was cumbersome and not intuitive.

Proposed Solution: Rephrasing the criteria from “low-intensity rehabilitation burden” to “physical demands of rehabilitation program.” Within the scoring exercise, change the questions from: “Consider the chance that a patient needs a low-intensity rehabilitation program after each type of amputation” to “Consider the chance that a patient needs a rehabilitation program that is less physically demanding after each type of amputation.”

MCDA results

Patients valued healing after surgery the most (39.5%), followed closely by the ability to walk after amputation (33.8%; Figure 1). Ease of prosthetic or orthotic device use (14.1%), preserving limb length (8.1%), and low-intensity rehabilitation program (4.5%) were not highly valued by patients and did not contribute much to their preference for amputation level. Although patients preferred TMAs (score of 51.3%) over TTAs (48.8%), the scores for amputation levels were very close. Notably, 1 patient in our sample had a very high consistency ratio, and their responses heavily skewed our results; removing this patient’s results reversed the aggregated patient preferences for healing after surgery (36.6%) and ability to walk (41.4%). The average consistency ratio was 0.61 with the outlier included in the data and 0.32 with the outlier excluded.

Figure 1

Patient and Provider Results from the Analytic Hierarchy Process in the Pilot Test.

Although patients rated healing after surgery as most important to them, providers believed patients valued the ability to walk the most (score of 56.1%; Figure 1), followed by healing after surgery (14.2%), preserving limb length (11.5%), and ease of prosthetic or orthotic device use (7.8%). Providers also believed patients preferred TMAs (74.4%) nearly 4 times as much as TTAs (22.6%), which was a larger spread in amputation scores than observed in the patient results.

Discussion

We identified 5 key criteria for our MCDA through our qualitative literature review. These criteria included the ability to walk after amputation, healing after surgery, rehabilitation program intensity, ease of the prosthetic or orthotic device, and limb length after amputation. Because the quantitative literature were sparse, we developed a Sheffield elicitation framework exercise to elicit criteria performance from subject matter experts. TMAs outperformed TTAs on all but 1 criterion; TTAs had a higher probability of healing after surgery than TMAs did.

Patients and providers found the MCDA acceptable and usable. However, we identified several improvements we could make before conducting the larger study with an MCDA. These improvements include breaking the questions into more simplified subquestions, including our example AHP question in both the provider and patient instructions, blinding the level of amputation in the scoring questions and the performance matrix, and modifying the wording of our rehabilitation burden criteria. Even with these limitations, our preliminary results from the MCDA suggest differences in provider beliefs and patient preferences for criteria.

Although patient involvement in the amputation-level decision process varies by provider, this is the first study to date that has explored whether provider perceptions of patient priorities align with patient priorities after amputation.⁸ Because MCDA methods use explicit questions to decompose priorities, MCDA methods can help quantify any differences in patient and provider values.¹¹ We plan to use our MCDA framework in a future study to understand the differences between patient values and provider perceptions of patient values.

Prior literature has revealed that AHP is well accepted by patients, which was consistent with our experience: both patients and providers successfully completed our MCDA.⁴⁸ Moreover, our pilot test results are in accordance with previous findings that patient preferences and provider preferences for treatment may not always align.¹⁷ However, when we removed the patient from our sample who had a very high consistency ratio, patient and provider preferences were more closely aligned.

In developing the MCDA, we encountered several methodologic challenges and identified approaches to ease participant burden. First, although we found 20 studies that met our inclusion criteria for the qualitative literature review, the studies used a diverse set of methods. The diverse methods led to challenges in aggregating the themes from the qualitative review into independent, overarching criteria, particularly since many themes were time dependent on each other and therefore did not meet MCDA criteria for criteria independence. We carefully crafted our MCDA so that each criterion met the independence criteria. Second, depending on the topic, quantitative comparative data for MCDA criteria may be sparse, and researchers should consider using expert opinion to complete performance matrices. Even so, despite having a performance matrix at hand, patients largely relied on personal experiences to complete the exercise. In a future study, we will address this challenge by blinding the alternatives, which may help patients better understand the MCDA scoring process. We encountered limitations when conducting our study. First, our pilot test included a small sample of male patients and providers, which reduces the generalizability of our results. However, the goal of our pilot test was to understand the acceptability of our MCDA, identify challenges inherent in the process, and preliminarily compare the preferences of patients to those of providers. Our research is beneficial because it outlines a roadmap for creating a patient MCDA when existing data are sparse. Second, we administered the MCDA online to providers and over the phone to patients; the patients were viewing a paper version of the MCDA during the interview. This difference in administration may have influenced how patients and providers responded the MCDA, particularly since the online MCDA alerted providers to choice inconsistencies during the exercise. We were not able to inform patients of choice inconsistencies during their interview.

We successfully implemented our MCDA to patients and providers and found that they were comfortable completing the exercise. Although patients and providers identified challenges in completing the MCDA, we were able to identify solutions to those challenges. Future work will include administering this MCDA to a larger group of patients and conducting sensitivity analyses that compare patient values with provider values, patient values with provider estimates of patient values, and provider values with provider estimates of patient values. In addition, future work could explore using MCDAs in patient decision aids for amputation-level shared decision making.

Conclusion

Patients with CLTI who need to undergo a lower-limb amputation often face a tradeoff in potential outcomes related to different amputation levels. While TMAs preserve a greater portion of the limb, any benefits attributable to ankle joint preservation are contingent on the limb healing. Because patients may value amputation outcomes differently, integrating an MCDA within a shared decision-making context may help patients better understand their priorities around amputation. Patients and providers generally found our MCDA to be acceptable and easy to use. Findings from this pilot study and a future larger-scale study will provide useful information for amputation-level patient decision aids. MCDAs are still gaining traction in health care and show promise, particularly in the context of shared decision making.

Supplemental Material

sj-docx-1-mpp-10.1177_23814683221143765 – Supplemental material for The Development and Pilot Study of a Multiple Criteria Decision Analysis (MCDA) to Compare Patient and Provider Priorities around Amputation-Level Outcomes

Supplemental material, sj-docx-1-mpp-10.1177_23814683221143765 for The Development and Pilot Study of a Multiple Criteria Decision Analysis (MCDA) to Compare Patient and Provider Priorities around Amputation-Level Outcomes by Diana Poehler, Joseph Czerniecki, Daniel Norvell, Alison Henderson, James Dolan and Beth Devine in MDM Policy & Practice

Footnotes

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Veterans Administration, Rehabilitation Research and Development, grant No. 1 I01 RX002960-01. Financial support for this study was provided entirely by the US Department of Veterans Affairs, Office of Research and Development, Rehabilitation Research and Development, grant No. 1 I01 RX002919R-01. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report. The following authors are employed by the sponsor: Dr. Daniel Norvell and Dr. Alison Henderson.

Authors’ Note

The contents of this article do not represent the views of the US Department of Veterans Affairs or the United States Government. Methods and study materials are available to other researchers upon request.

ORCID iDs

Diana Poehler

James Dolan

Supplemental Material

Supplementary material for this article is available on the MDM Policy & Practice website at .

References

Mayfield

Reiber

Maynard

Czerniecki

Caps

Sangeorzan

BJ.

Trends in lower limb amputation in the Veterans Health Administration, 1989-1998. J Rehabil Res Dev. 2000;37:23–30.

Dillon

Fatone

Deliberations about the functional benefits and complications of partial foot amputation: do we pay heed to the purported benefits at the expense of minimizing complications?

Arch Phys Med Rehabil. 2013;94:1429–35.

Norvell

Czerniecki

JM.

Risks and risk factors for ipsilateral re-amputation in the first year following first major unilateral dysvascular amputation. Eur J Vasc Endovasc Surg. 2020;60:614–21.

Czerniecki

Thompson

Littman

et al. Predicting reamputation risk in patients undergoing lower extremity amputation due to the complications of peripheral artery disease and/or diabetes. Br J Surg. 2019;106:1026–34.

Anthony

Roberts

Modrall

et al. Transmetatarsal amputation: assessment of current selection criteria. Am J Sur. 2006;192:e8–11.

Dillon

Quigley

Fatone

Outcomes of dysvascular partial foot amputation and how these compare to transtibial amputation: a systematic review for the development of shared decision-making resources. Syst Rev. 2017;6:1–20.

Dillon

Anderson

Duke

Ozturk

Stuckey

The lived experience of sequential partial foot and transtibial amputation. Disabil Rehabil. 2020;42:2106–14.

Webster

Crunkhorn

Sall

Highsmith

Pruziner

Randolph

BJ.

Clinical practice guidelines for the rehabilitation of lower limb amputation: an update from the Department of Veterans Affairs and Department of Defense. Am J Phys Med Rehabil. 2019;98:820–9.

Belton

Stewart

Multiple Criteria Decision Analysis: An Integrated Approach. New York: Springer Science & Business Media; 2002.

10.

Keeney

Raiffa

Meyer

RF.

Decisions with Multiple Objectives: Preferences and Value Trade-offs. Cambridge (UK): Cambridge University Press; 1993.

11.

Thokala

Devlin

Marsh

et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. 2016;19:1–13.

12.

Dolan

Boohaker

Allison

Imperiale

TF.

Patients’ preferences and priorities regarding colorectal cancer screening. Med Decis Making. 2013;33:59–70.

13.

Dolan

Frisina

Randomized controlled trial of a patient decision aid for colorectal cancer screening. Med Decis Making. 2002;22:125–39.

14.

Kitamura

Decision-making process of patients with gynecological cancer regarding their cancer treatment choices using the analytic hierarchy process. Jpn J Nurs Sci. 2010;7:148–57.

15.

Mühlbacher

Juhnke

Kaczynski

Patients’ priorities in the treatment of neuroendocrine tumours: an analytical hierarchy process. Gesundheitswesen. 2015;78:651–7.

16.

Levy

Daly

Bergus

Dunkelberg

JC.

Comparison of patient preferences for fecal immunochemical test or colonoscopy using the analytic hierarchy process. BMC Health Serv Res. 2015;15:1–9.

17.

Hummel

Snoek

van Til

van Rossum

IJzerman

MJ.

A multicriteria decision analysis of augmentative treatment of upper limbs in persons with tetraplegia. J Rehabil Res Dev. 2005;42:635–44.

18.

IJzerman

van Til

Snoek

GJ.

Comparison of two multi-criteria decision techniques for eliciting treatment preferences in people with neurological disorders. Patient. 2008;1:265–72.

19.

Garritty

Gartlehner

Nussbaumer-Streit

et al. Cochrane rapid reviews methods group offers evidence-informed guidance to conduct rapid reviews. J Clin Epidemiol. 2021;130:13–22.

20.

Marsh

IJzerman

Thokala

et al.; ISPOR Task Force. Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. 2016;19:125–37.

21.

Suckow

Goodney

Nolan

et al. Domains that determine quality of life in vascular amputees. Ann Vasc Surg. 2015;29:722–30.

22.

Czerniecki

Turner

Williams

Hakimi

Norvell

DC.

The effect of rehabilitation in a comprehensive inpatient rehabilitation unit on mobility outcome after dysvascular lower extremity amputation. Arch Phys Med Rehabil. 2012;93:1384–91.

23.

Norvell

Thompson

Boyko

et al. Mortality prediction following non-traumatic amputation of the lower extremity. Br J Surg. 2019;106:879–88.

24.

Norvell

Turner

Williams

Hakimi

Czerniecki

JM.

Defining successful mobility after lower extremity amputation for complications of peripheral vascular disease and diabetes. J Vasc Surg. 2011;54:412–9.

25.

Norvell

Williams

Turner

Czerniecki

JM.

The development and validation of a novel outcome measure to quantify mobility in the dysvascular lower extremity amputee: the amputee single item mobility measure. Clin Rehabil. 2016;30:878–89.

26.

Dolan

Veazie

PJ.

Harnessing expert judgment to support clinical decisions when the evidence base is weak. Med Decis Making. 2019;39:74–9.

27.

Oakley

O’Hagan

SHELF: the Sheffield elicitation framework (version 4.0). School of Mathematics and Statistics, University of Sheffield, UK. 2010. Available from https://shelf.sites.sheffield.ac.uk: http://tonyohagancouk/shelf.

28.

Saaty

TL.

What is the analytic hierarchy process? In: Mitra

Greenberg

Lootsma

Rijkaert

Zimmermann

, eds. Mathematical Models for Decision Support. Berlin: Springer; 1988. p 109–21.

29.

Saaty

TL.

Decision-making with the AHP: why is the principal eigenvector necessary. Eur J Oper Res. 2003;145:85–91.

30.

Bernasconi

Choirat

Seri

Empirical properties of group preference aggregation methods employed in AHP: theory and evidence. Eur J Oper Res. 2014;232:584–92.

31.

Barg

Cronholm

Easley

et al. A qualitative study of the experience of lower extremity wounds and amputations among people with diabetes in Philadelphia. Wound Repair Regen. 2017;25:864–70.

32.

Couture

Caron

Desrosiers

Leisure activities following a lower limb amputation. Disabil Rehabil. 2010;32:57–64.

33.

Couture

Desrosiers

Caron

CD.

Cognitive appraisal and perceived benefits of dysvascular lower limb amputation: a longitudinal study. Arch Gerontol Geriatr. 2011;52:5–11.

34.

Day

Wadey

Strike

Living with limb loss: everyday experiences of “good” and “bad” days in people with lower limb amputation. Disabil Rehabil. 2019;41:2433–42.

35.

Delea

Buckley

Hanrahan

McGreal

Desmond

McHugh

Management of diabetic foot disease and amputation in the Irish health system: a qualitative study of patients’ attitudes and experiences with health services. BMC Health Serv Res. 2015;15:251.

36.

Dillon

Anderson

Duke

Ozturk

Stuckey

The lived experience of sequential partial foot and transtibial amputation. Disabil Rehabil. 2020;42(15):2106–14.

37.

Horne

Paul

Pain support for adults with a diabetes-related lower limb amputation: an empirical phenomenology study. Pain Manag Nurs. 2019;20:270–5.

38.

Ostler

Ellis-Hill

Donovan-Hall

Expectations of rehabilitation following lower limb amputation: a qualitative study. Disabil Rehabil. 2014;36:1169–75.

39.

Pedlow

Cormier

Provost

et al. Patient perspectives on information needs for amputation secondary to vascular surgery: what, when, why, and how much? J Vasc Nurs. 2014;32:88–98.

40.

Schaffalitzky

Gallagher

Maclachlan

Ryall

Understanding the benefits of prosthetic prescription: exploring the experiences of practitioners and lower limb prosthetic users. Disabil Rehabil. 2011;33:1314–23.

41.

Senra

Oliveira

Leal

Vieira

Beyond the body image: a qualitative study on how adults experience lower limb amputation. Clin Rehabil. 2012;26:180–91.

42.

Torbjornsson

Ottosson

Blomgren

Bostrom

Fagerdahl

AM.

The patient’s experience of amputation due to peripheral arterial disease. J Vasc Nurs. 2017;35:57–63.

43.

Trevelyan

Turner

Robinson

Perceptions of phantom limb pain in lower limb amputees and its effect on quality of life: a qualitative study. Br J Pain. 2016;10:70–7.

44.

van Twillert

Stuive

Geertzen

Postema

Lettinga

AT.

Functional performance, participation and autonomy after discharge from prosthetic rehabilitation: barriers, facilitators and outcomes. J Rehabil Med. 2014;46:915–23.

45.

Verschuren

Geertzen

Enzlin

Dijkstra

Dekker

People with lower limb amputation and their sexual functioning and sexual well-being. Disabil Rehabil. 2015;37:187–93.

46.

Ward Khan

O’Keeffe

Nolan

Stow

Davenport

. “Not a whole woman”: an interpretative phenomenological analysis of the lived experience of women’s body image and sexuality following amputation. Disabil Rehabil. 2019;43(2):1–11.

47.

Washington

Williams

AE.

An exploratory phenomenological study exploring the experiences of people with systemic disease who have undergone lower limb amputation and its impact on their psychological well-being. Prosthet Orthot Int. 2016;40:44–50.

48.

Dolan

JG.

Are patients capable of using the analytic hierarchy process and willing to use it to help make clinical decisions?

Med Decis Making. 1995;15:76–80.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.31 MB