Towards an individualized management strategy for patients with chronic venous disease: Results of a Delphi consensus

Abstract

Objective

To obtain consensus on management criteria for symptomatic patients with chronic venous disease (CVD; C2–C6) and superficial venous reflux.

Method

We used a Delphi method by means of 36 statements sent by email to experts in the field of phlebology across the world over the course of three rounds. The statements addressed criteria for different venous treatments in patients with different characteristics (e.g. extensive comorbidities, morbid obesity and peripheral arterial disease). If at least 70% of the ratings for a specific statement were between 6 and 9 (agreement) or between 1 and 3 (disagreement), experts’ consensus was reached.

Results

Twenty-five experts were invited to participate, of whom 24 accepted and completed all three rounds. Consensus was reached in 25/32 statements (78%). However, several statements addressing UGFS, single phlebectomies, patients with extensive comorbidities and morbid obesity remained equivocal.

Conclusion

Considerable consensus was reached within a group of experts but also some gaps in available research were highlighted.

Keywords

Varicose veins endovascular treatment echo-sclerotherapy

Introduction

Over the past decades, numerous effective treatments have been developed for patients with chronic venous disease (CVD). The increased availability of different treatment options has made the management of CVD more challenging. It has resulted in a large worldwide variation in management preferences for treating patients with CVD.^1,2 Management strategies are not only based on evidence from the literature but also on the physician’s own experience, availability (and costs) of the equipment and on national healthcare reimbursement systems. However, in clinical practice, management decisions should ideally be influenced mainly by a combination of patient characteristics, clinical findings and results of duplex ultrasound (DUS).^1,3,4

Although there are several well-established national and international guidelines,^5–8 their recommendations only focus on the treatment of a diseased population rather than an individual patient.⁹ Appropriate tools for management strategies incorporating the specific characteristics of an individual patient, as well as the clinical and DUS findings, are lacking in current clinical guidelines.^5–8 To bridge the gap between current clinical guidelines and individualized care, expert opinion methods such as a Delphi consensus¹⁰ are desired.

The aim of the present study was to achieve an international Delphi-based consensus on management criteria for patients presenting with venous symptoms, clinical signs of CVD, and superficial venous reflux confirmed by DUS.

Methods

Expert panel

Experts were selected from a group of physicians who already participated in our recently published worldwide survey on management strategies in patients with CVD (C2–C6).¹ Only those with at least one scientific publication in the field of phlebology in a peer-reviewed journal and at least 10 years of experience in treating patients with CVD were eligible for the present study. They had to be performing ultrasound-guided foam sclerotherapy (UGFS), endovenous thermal ablation (EVTA) and phlebectomies themselves. Two investigators (SKvdV and MGRDM) selected the expert panel. For this selection, they aimed at having a reasonable distribution between different specialties (vascular or general surgeons, dermatologists, angiologists, phlebologists) and continent or country of clinical practice. Based on the selection criteria, 25 eligible experts were invited by email. They were asked to judge on several statements in view of obtaining consensus over the course of three rounds, according to the Delphi method between March 2015 and November 2015 (Figure 1). In case of non-responding, the investigators sent two email reminders in each round.

Figure 1.

Selection of participants for the Delphi consensus.

Informed consent was not performed because no patients were involved in this study.

Delphi consensus procedure

The authors formulated the statements, using results from a worldwide survey regarding management strategies in patients with great saphenous vein (GSV) and tributary reflux.¹ The first round contained 30 statements addressing criteria for endovenous ablation (EVA) of the refluxing saphenous trunk (ST), including both thermal and non-thermal ablation techniques (mainly for simplicity), UGFS, high ligation and stripping (HL/S), single phlebectomies without treatment of the refluxing saphenous trunk referred to as ‘ambulatory selective varicose ablation under local anaesthesia’ (ASVAL)¹¹ and non-interventional measures (venotonic drugs and medical elastic compression stockings (MECS)). The statements contained scenarios with different patient characteristics, clinical class (C2–C6, according to the CEAP classification¹²) and DUS findings in patients with CVD of the lower limbs and superficial venous reflux in a ST. The terminology used for the statements is summarized in Table 1.

Table 1.

Definition of terminology used for the statements of the Delphi consensus.

Venous symptoms:	ache, pain, heaviness, tightness, feeling of swelling, nocturnal cramps, itching
C disease:	clinical class according to the CEAP classification¹²: C2: varicose veins; C3: edema; C4: skin changes (pigmentation, eczema, atrophie blanche, lipodermatosclerosis); C5: healed venous ulcer; C6: open venous ulcer
Saphenous trunk:	great saphenous vein, anterior accessory saphenous vein, posterior accessory saphenous vein, Giacomini vein, small saphenous vein
Varicose tributaries:	visible or palpable varicose veins in the subcutis
Reflux in a saphenous trunk:	abnormally reversed flow in a saphenous trunk, during > 500 ms at calf compression-release or Valsalva (the latter only for the SFJ) involving the terminal or preterminal valve of the SFJ or the SPJ
Terminal valve reflux:	reflux at the junction of the SFJ or SPJ
Segmental reflux:	reflux limited to a segment of a saphenous trunk, not involving the SFJ or SPJ
Saphenous diameter:	diameter measured in a tubular part of the refluxing vein segment, about 15 cm from the junction in standing position
Focal dilatation:	localized dilatation of the saphenous trunk less than 1.5–3 times the saphenous diameter above or below
Aneurysm:	dilatation of the saphenous vein more than three times the saphenous diameter above or below, or more than 20 mm (close to the SFJ or SPJ)
Morbid obesity:	BMI > 40 kg/m² or BMI > 35 kg/m² and experiencing obesity-related health conditions
Severe peripheral arterial
disease:	ankle brachial index < 0.6
Bridging:	oral anticoagulation is interrupted with bridging anticoagulation, using either heparin or low-molecular weight heparin, administered during the sub-therapeutic window

SFJ: saphenofemoral junction; SPJ: saphenopopliteal junction; BMI: body mass index.

Participants were asked to anonymously rate each statement on a 9-point scale, by ticking boxes with marks between 1 and 9, where a score of 1 denoted complete disagreement and a score of 9 indicating full agreement (Figure 2). If at least 70% of the ratings for a specific statement were between 7 and 9, it was concluded experts consented on agreeing with the statement. Vice versa, if ≥70% were between 1 and 3, it was concluded they consented on disagreeing with the statement. This cut-off level was based on previous literature.^13,14 Any other distribution of ratings was valued ‘equivocal’. Participants also had the opportunity to add a comment in the ‘remark(s)’ box accompanying each statement. All this information was used for preparing the subsequent round of the Delphi consensus.

Figure 2.

Example of a statement showing consensus on agreement (84% of marks between 7 and 9).

In the second round, experts received a full report of the first round, including a compilation of ratings for all statements of the first round, and all the remarks of their colleagues, provided anonymously. Then, they were asked to rate again those statements that had remained equivocal in the first round. Statements were reformulated and new statements were added based on the remarks of the experts in the previous round. For instance, in the first round, we had included older age as a patient characteristic, which might influence management. However, participating experts suggested age was not the main factor, but rather extensive comorbidities, and therefore the statement was adapted as such. Eventually, the second round contained 18 statements. In the second round, we also attached a summary of the present literature regarding each statement.

The same procedure was followed for the third (and last) round, which contained only nine statements to be rated. Again, participating experts could consider the results of the second round and the remarks of their colleagues.

In case of one or more missing items, the expert was contacted by email to retrieve the missing answer.

Results

Experts

In total, 24 out of 25 contacted experts accepted to participate: 15 vascular surgeons, two general surgeons, four dermatologists, two angiologists and one phlebologist from Europe (17/25), Australia (1/25), North-America (4/25) and Central- or South-America (3/25). Experts were working in private practice (13/25), general hospital (4/25) or university hospital (7/25). All these experts completed the first, second and third Delphi round (25/25). The majority of experts (16/25) had over 20 years of experience in treating patients with CVD.

Delphi consensus

After three rounds, consensus was reached in 25 out of 32 statements (78%; Table 2). All statements regarding indications for treatment with thermal or non-thermal EVA vs. treatment with HL/S reached high (>79%) consensus in favor of EVA, except for venous aneurysms. Of note, 96% of the experts agreed on that high ligation should not be added to EVA treatment. Similar level of agreement was reported about the use of MECS as preferred strategy in those patients with C4–C6 disease and reflux in an ST, who are not willing or contraindicated to undergo any intervention.

Table 2.

Results of Delphi consensus.

	Delphi rounds
Statements	1	2	3
EVA vs. HL/S
Nowadays, with the availability of endovenous treatments, HL/S is only rarely indicated.	84%
In case of saphenous reflux, EVA is indicated rather than HL/S, even if there is C4–C6 disease.	83%
In patients with venous symptoms and reflux in a saphenous trunk, EVA is indicated rather than HL/S, even in the presence of TV reflux.	83%
In patients with venous symptoms and reflux in a saphenous trunk, EVA is indicated rather than HL/S, even in the presence of a large (>10 mm) saphenous diameter.	79%
In patients with venous symptoms and reflux in a saphenous trunk, EVA is indicated rather than HL/S, even in the presence of one or more focal dilatations.	79%
Presence of a venous aneurysm (>20 mm) within 2 cm from the SFJ or SPJ is an indication for HL/S rather than EVA.
HL should not be added to patients being treated with EVA.	96%
UGFS
In the presence of C4-C6 disease in patients with reflux in a saphenous trunk > 4 mm in diameter, UGFS is a treatment option.^a,b			93%
In the presence of C4-C6 disease in patients with reflux in a saphenous trunk < 4 mm in diameter, UGFS is a valuable treatment option.*		80%
In the presence of C4-C6 disease in patients with reflux in a saphenous trunk < 4 mm in diameter and refluxing tributaries in a diseased skin area, UGFS of tributaries is preferred rather than phlebectomies.	^c	87%
In the presence of C4-C6 disease in patients with reflux in a saphenous trunk < 4 mm in diameter and refluxing tributaries, UGFS and phlebectomies of tributaries at a distance from the diseased skin area, are both valuable treatment options.	^c	92%
In patients with venous symptoms and reflux in a saphenous trunk, UGFS (without tumescent anesthesia) is a valuable treatment option, even in the presence of a large (>10 mm) saphenous diameter.^a		75%
In patients with venous symptoms and reflux in a saphenous trunk vein > 10 mm in diameter, where ablation is indicated, EVA is preferred rather than UGFS.	92%
If you have decided to ablate the saphenous trunk, in the presence of C2-C3 disease in patients with venous symptoms and reflux in a saphenous trunk vein < 4 mm in diameter, UGFS is preferred rather than EVA.^a,b
EVA
In the presence of C4–C6 disease in patients with reflux in a saphenous trunk < 4 mm in diameter, EVA is a valuable treatment option.	^c	71%
ASVAL
In the presence of C2–C3 disease in patients with venous symptoms, segmental reflux of a saphenous trunk < 4 mm in diameter and large refluxing tributaries, preservation of the saphenous trunk is indicated rather than its ablation.^a,b
ASVAL is not indicated in case of reflux in a saphenous trunk and C4-C6 disease.	83%
Non-interventional measure
In patients with venous symptoms, C2–C3 disease and reflux in a saphenous trunk and who are not willing to undergo any intervention or who are unfit for intervention because of extensive comorbidities, MECS should be considered.	88%
In patients with venous symptoms, C2–C3 disease and reflux in a saphenous trunk and who are not willing to undergo any intervention or who are unfit for intervention because of extensive comorbidities, venotonic drugs should be considered.		79%
In patients with C4–C6 disease and reflux in a saphenous trunk and who are not willing to undergo any intervention or who are unfit for intervention because of extensive comorbidities, MECS are indicated.	96%
Comorbidities
In patients with venous symptoms in addition to extensive comorbidities and reflux in a saphenous trunk, UGFS is indicated rather than EVA.^a
In patients with venous symptoms in addition to extensive comorbidities and reflux in a saphenous trunk, EVA or UGFS should only be considered in case of C4–C6 disease.^a
Morbid obesity
In patients with venous symptoms, reflux in a saphenous trunk and morbid obesity, EVA or UGFS are indicated rather than HL/S.	87%
In patients with venous symptoms, reflux in a saphenous trunk and morbid obesity, treatment should only be considered in case of C4–C6 disease.^b
In patients with venous symptoms, morbid obesity and reflux in a saphenous trunk, weight reduction is advised prior to any venous treatment.^b	^c
Anticoagulation
In patients with venous symptoms, reflux in a saphenous trunk and who are on chronic anticoagulants, EVA is indicated rather than HL/S.	92%
In patients with venous symptoms, reflux in a saphenous trunk < 4 mm diameter and who are on chronic anticoagulants, UGFS is indicated rather than HL/S.^a		91%
In patients with venous symptoms, reflux in a saphenous trunk and who are on chronic anticoagulants, EVA can be performed without bridging.	87%
In patients with venous symptoms, reflux in a saphenous trunk and who are on chronic anticoagulants, UGFS can be performed without bridging.	88%
In patients with venous symptoms, who are on chronic anticoagulants and who are scheduled for extensive phlebectomies, temporary discontinuation and anticoagulant treatment bridging is indicated.^b			88%
Severe peripheral arterial disease
In patients with C2–C3 disease, reflux in a saphenous trunk and severe peripheral arterial disease, it is preferable not to ablate the saphenous vein by means of EVA, UGFS or HL/S.	79%
In patients with C4–C6 disease, reflux in a saphenous trunk and severe peripheral arterial disease, EVA or UGFS of the refluxing saphenous vein may be considered.	71%

EVA: endovenous ablation (including thermal and non-thermal non-tumescent techniques, excluding UGFS); HL/S: high ligation with stripping; SFJ: saphenofemoral junction; SPJ: saphenopopliteal junction; UGFS: ultrasound-guided foam sclerotherapy; ASVAL: ‘ambulatory selective varicose ablation under local anaesthesia’ (= single phlebectomies without treating the saphenous trunk); MECS: medical elastic compression stockings.

Consensus on agreeing with the statement.

Consensus on disagreeing with the statement.

Equivocal (no complete agreement nor disagreement).

Statement was reformulated after first round.

Statement was reformulated after second round.

Statement was added after remarks of experts in the first round.

The experts’ opinion on several statements remained equivocal after three rounds. This was the case for statements addressing different indications for UGFS, ASVAL, extensive comorbidities and morbid obesity. The experts stated that it depended on the type and extent of the patient’s comorbidities and therefore no definitive conclusion on preferred management could be made. Regarding morbid obesity, the experts would definitely prefer EVA over HL/S, but there was no agreement on limiting treatment to those patients with skin changes (C4–C6). Moreover, based on their experience, the participants were convinced that advising weight reduction unfortunately was unsuccessful in the majority of obese patients.

Throughout the different rounds of the Delphi consensus, three statements were removed, two because of redundancy after reformulating other statements and one because of lack of relevance following the experts’ remarks (Supplementary Table 1).

Discussion

The present study showed that considerable consensus could be reached within a group of selected experts in the field of treatment of superficial venous disease, by means of a Delphi consensus procedure. This may help physicians to make supported and individualized choices for their patients with CVD (C2–C6).

Not surprisingly, experts’ recommendations regarding EVA were in line with current guidelines on the care of patients with varicose veins and associated CVD.^5–8 EVA was considered the preferred treatment for most symptomatic patients with CVD (C2–C6) and a refluxing ST > 4 mm in diameter. The expert panel did not recommend an alternative treatment for specific DUS findings such as terminal valve reflux, a large (>10 mm) ST diameter or the presence of one or more focal dilatations of the ST (Table 2). EVA has now globally become the preferred treatment for ST reflux as is reflected in our results. Although EVA (thermal or non-thermal) has largely replaced HL/S in many countries, it should be acknowledged that, whenever equipment for EVA is not available, HL/S (ideally using tumescent anesthesia, in agreement with recent guidelines⁶) remains a valuable treatment option. While patient-reported outcomes favor EVTA, long-term efficacy of HL/S and EVTA are very similar.^15,16

UGFS is usually considered as the second best option of the minimally invasive treatments for abolishing ST reflux.^5–8 In the present consensus approach, the experts considered UGFS to be just ‘a’ treatment option if ST diameter is ≥4 mm, and only agreed on categorizing it ‘a valuable treatment option’ for ST’s with small diameters < 4 mm. This discrepancy illustrates that ST diameter may be used as a relevant criterion to distinguish between different treatment options,¹ which is in line with the European guidelines forsclerotherapy.¹⁷

ASVAL is not yet a worldwide-accepted strategy, and this was reflected in the Delphi consensus. Contrarily to what the investigators had been suggesting, the experts could not agree on particular criteria for implementing ASVAL so far. This is somehow to be expected as only 42% of the experts had ever used ASVAL in their clinical practice (data not shown) and the evidence on the subject is still limited.^11,18,19 Future research should point out whether ASVAL can be applied in all patients with varicose veins or in a selected varicose vein population only.

At the end of the first round of the Delphi consensus, older age appeared to be less relevant than we initially had hypothesized and it was therefore converted into extensive comorbidities as suggested by the participants from the second round onwards. Indeed, clinical studies demonstrated that the feasibility and safety of EVTA and tolerability of UGFS were similar in patients older or younger than 75 years.^20,21 In contrast, studies regarding venous treatments in patients with (extensive) comorbidities are completely lacking. Although the investigators thought that minimizing the extent of procedures in patients with comorbidities and/or morbid obesity would be good clinical practice, experts did not reach consensus on this issue. This lack of consensus could partly be explained by the fact that many different combinations of comorbidities can be thought of in these cases and the extent of comorbidities was not defined in the statements. The experts seemed to be more unanimous regarding strategies in patients under chronic anticoagulant treatment and those with severe peripheral arterial disease.

Although we found considerable consensus about treatment strategies in different clinical classes and DUS findings, statements addressing patients’ characteristics remained largely equivocal in the Delphi consensus. This clearly highlights the lack of knowledge in the field of personalized medicine and implies that more research regarding patient characteristics, and how these are influencing treatment outcomes, is needed. In this way, future guidelines may include more tools for proper stratification of patients with CVD. Obviously, when defining individualized treatment strategies in patients with CVD, physicians should also consider patient’s preference, as well as the impact of the disease on Health-Related Quality of Life (HRQoL), the estimated risk of deterioration of CVD and local healthcare resources.

Limitations

This study has several limitations. The statements may have been incomplete, but this was minimized because they were based on a prior worldwide survey among 211 specialists in CVD.⁵ Also, this study focused on the treatment of patients with venous symptoms, clinical signs of CVD, and superficial venous reflux in the ST and tributaries, and therefore our results are not generalizable to all patients with CVD (e.g. patients with a pathological perforating vein nearby a venous ulcer or patients with groin recurrence). Selection bias among the eligible experts might have occurred because the majority of the participants were working in Western Europe. However, the response rate was almost perfect. In the absence of clear guidelines for a Delphi consensus, we chose the cut-off value of 70% as acceptable, because of the relatively mild impact of CVD on HRQoL, contrarily to decision-making in more life-threatening diseases in which a higher cut-off may be desired. In addition, there is no standard method for calculating sample sizes for a Delphi study, but there is evidence that expert panels of 20 can reach a valid consensus.²²

Conclusion

In conclusion, in the present investigation about management strategies in patients with symptoms and signs of CVD and superficial venous reflux, the experts agreed on most of the strategies and their opinions largely reflected international guidelines. Nevertheless, it appeared to be more difficult to reach consensus on the influence of certain patient-related characteristics. More research is needed in the field of personalized medicine, in order to further optimize phlebologic care for all our patients presenting with CVD.

Footnotes

Acknowledgements

The authors thank all experts who took part in the Delphi consultation: Lena Blomgren, Kornelia Böhler, Alberto Caggiati, Dumitru Casian, Alessandro Frullini, Kathleen Gibson, Jean-Luc Gillet, Manj Gohel, Evi Kalodiki, Mark Malouf, Uldis Maurins, Mark Meissner, Nick Morrison, Alfred Obermayer, Alvaro Orrego, Pierre Ouvry, Eberhard Rabe, Lars Rasmussen, Stefanie Reich-Schupke, Guillermo Rossi, Roberto Simkin, Ted van den Broek, Marc Vuylsteke, Steven Zimmet.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Contributorship

SKvdV designed study, Delphi consultation, analysed data, drafted paper. RRvdB designed study, drafted paper. OP designed study, critically reviewed paper. TN designed study, critically reviewed paper. MGRdM designed study, Delphi consultation, analysed data, drafted paper.

References

van der Velden

Pichot

van den Bos

et al.

Management strategies for patients with varicose veins (C2-C6): results of a worldwide survey. Eur J Vasc Endovasc Surg 2015; 49: 213–220.

Moore

Lane

Thapar

et al.

The European burden of primary varicose veins. Phlebology 2013; 28(Suppl 1): 141–147.

Chastanet

Pittaluga

. Influence of the competence of the sapheno-femoral junction on the mode of treatment of varicose veins by surgery. Phlebology 2014; 29(1 suppl): 61–65.

Pichot

De Maeseneer

. Treatment of varicose veins: does each technique have a formal indication? Perspect Vasc Surg Endovasc Ther 2011; 23: 250–254.

National Institute for Health and Care Excellence (NICE). Varicose veins in the legs: the diagnosis and management of varicose veins, www.nice.org.uk/cg168 (accessed 17 December 2015).

Wittens

Davies

Bækgaard

et al.

Editor's choice - management of chronic venous disease: clinical practice guidelines of the European Society for Vascular Surgery (ESVS). Eur J Vasc Endovasc Surg 2015; 49: 678–737.

Nicolaides

Kakkos

Eklöf

et al.

Management of chronic venous disorders of the lower limbs – guidelines according to scientific evidence. Int Angiol 2014; 33: 87–208.

Gloviczki

Comerota

Dalsing

et al.

The care of patients with varicose veins and associated chronic venous diseases: clinical practice guidelines of the Society for Vascular Surgery and the American Venous Forum. J Vasc Surg 2011; 53(Suppl. 5): 2S–48S.

Boudoulas

Leier

Geleris

et al.

The shortcomings of clinical practice guidelines. Cardiology 2015; 130: 187–200.

10.

Jones

Hunter

. Consensus methods for medical and health services research. BMJ 1995; 311: 376–380.

11.

Pittaluga

Chastanet

Locret

et al.

The effect of isolated phlebectomy on reflux and diameter of the great saphenous vein: a prospective study. Eur J Vasc Endovasc Surg 2010; 40: 122–128.

12.

Eklöf

Rutherford

Bergan

et al.

Revision of the CEAP classification for chronic venous disorders: consensus statement for the American Venous Forum International Ad Hoc Committee for Revision of the CEAP Classification, Helsingborg, Sweden. J Vasc Surg 2004; 40: 1248–1252.

13.

Elwyn

O'Connor

Stacey

et al.

Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ 2006; 333: 417–417.

14.

Palter

MacRae

Grantcharov

. Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology. Am J Surg 2011; 201: 251–259.

15.

van der Velden

Biemans

De Maeseneer

et al.

Five-year results of a randomized clinical trial of conventional surgery, endovenous laser ablation and ultrasound-guided foam sclerotherapy in patients with great saphenous varicose veins. Br J Surg 2015; 102: 1184–1194.

16.

Rasmussen

Lawaetz

Bjoern

et al.

Randomized clinical trial comparing endovenous laser ablation and stripping of the great saphenous vein with clinical and duplex outcome after 5 years. J Vasc Surg 2013; 58: 421–426.

17.

Rabe

Pannier

. for the Guideline Group. Indications, contraindications and performance: European guidelines for sclerotherapy in chronic venous disorders. Phlebology 2014; 29(1 suppl): 26–33.

18.

Biemans

van den Bos

Hollestein

et al.

The effect of single phlebectomies of a large varicose tributary on great saphenous vein reflux. J Vasc Surg: Venous and Lym Dis 2014; 2: 179–187.

19.

Pittaluga

Chastanet

Rea

et al.

Midterm results of the surgical treatment of varices by phlebectomy with conservation of a refluxing saphenous vein. J Vasc Surg 2009; 50: 107–118.

20.

Hamel-Desnos

Desnos

Allaert

et al.

Thermal ablation of saphenous veins is feasible and safe in patients older than 75 years: a prospective study (EVTA study). Phlebology 2015; 30: 525–532.

21.

Gillet JL, Hamel-Desnos C, Lausrecker M, et al. The consequences of aging on the population in the management of varicose veins by sclerotherapy. In: UIP chapter meeting: Seoul UIP, Seoul, Korea, 27–29 August 2015. Volume: International Angiology, August 2015;Suppl.1 to issue no. 4:92.

22.

Akins

Tolson

Cole

. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol 2005; 5: 37–37.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.18 MB