Quality assessment of breast cancer studies conducted with the Delphi technique

1. Introduction

Cancer incidence and mortality continue to rise worldwide. According to GLOBOCAN estimates, there were 19.3 million new cases and about 10 million deaths related to cancer in 2020. Breast cancer correspond to 11.7% of the incidence and 6.9% of the mortality for this same year [1].

While breast cancer is an area of growing research, it is also a heterogeneous and wide field. A study conducted by Eccles et al. found some critical research gaps in breast cancer; including the need for tailored screening approaches, the evaluation of optimal duration, sequencing and combinations of therapy, and molecular or genetic changes responsible for malignancy [2].

In instances where available evidence is insufficient, the Delphi technique may be employed for identifying problems, setting goals, or assessing priorities through expert consensus [3,4]. Regarding breast cancer, some gaps in knowledge may be addressed through this technique, for guiding decision-making processes and patient care [5]. The Delphi technique is a consensus method aiming to obtain statistical estimations from a qualitative approach, through an iterative process that leads to consensus within experts [6]. The main characteristics of the technique include iteration, anonymity, feedback, and consensus reaching. Iteration, the first characteristic, means that experts must emit their opinion more than once through various rounds. As for anonymity, this quality is important for assuring that none of the participants are aware of the answer given by another participant, thus decreasing the risk for bias. Feedback is a characteristic meant to take place after each round of the technique has been held. Feedback aims to position each participant in regards to the group. Highlighting significant contributions, both agreeing or discordant that may modify future answers. Finally, consensus must be reached, which is the main goal of using a Delphi technique [7,8].

A relevant issue to consider in the field of evidence-based medical care is the quality of scientific articles. As such, multiple tools have been designed for assessing the quality of different types of studies. For instance, the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) aims to assess diagnostic tools studies, the Risk of Bias 2 (RoB 2) gives a guide for considering bias in randomized trials, and the Equator Network provides tools for reporting different types of studies with quality and transparency [6,9–12]. Regarding the Delphi technique, the CREDES checklist (Guidance on Conducting and Reporting Delphi Studies) provides some criteria for authors of studies employing this method [11]. Our research group recently developed the Quali-D tool, meant to allow readers to assess for the quality of studies conducted with the Delphi technique (Fig. 1). This study aims to report on the quality of breast cancer studies using Delphi technique, assessed through the Quali-D tool.

OPEN IN VIEWER

Fig. 1.

Quali-D tool for the assessment of quality in studies using the Delphi technique.

OPEN IN VIEWER

Fig. 2.

Selection process and studies included for analysis.

3. Literature search and tool application

We performed a literature search on studies regarding breast cancer using the Delphi technique, using the following search strategies: (“Delphi Technique” [All Fields] AND “Breast neoplasms” [Majr]) for Medline; and ‘breast tumor’/exp/mj AND ‘delphi study’/exp for Embase. The only filter used, restricted our search to articles published in English, Spanish and French, with no limits for publication date. The searches were conducted on October 4th 2022.

The search yielded 149 results. All abstracts were reviewed and duplicate records were removed. After removing conference abstracts, unretrieved reports, studies not using the Delphi technique, protocols without final results and studies not reporting breast cancer exclusively, 72 studies were included for review (Fig. 2). The studies included for review were analyzed using the Quali-D tool (Fig. 1). This is a descriptive study, as such, results are expressed through frequencies and percentages.

4. Results

The studies about breast cancer using the Delphi technique included a wide array of topics. The most common were: quality indicators (9.73%), expressed needs (5.31%), screening (4.42%), event definitions (4.42%) and reconstructive surgery (3.54%), among others (Table 1). A full list of topics can be found in the supplementary material (Supplementary Table S1).

98.61% of the studies chose the Delphi technique due to lack of a more adequate method to solve the research question, the same number of studies summarized and presented results in a clear way. In 91.67% of the assessed studies, at least two rounds were conducted, and 86.11% of the studies described the methods for expert selection in a complete manner. On the other hand, only 4.17% of the studies disclosed conflicts of interest for both experts and authors, and 54.17% of the studies reported anonymous participation of experts or anonymization of their answers to other participants (Table 2). From the total analyzed studies, 40 (55.5%) were reported as modified Delphi studies. Of these, only 24 studies reported on the ways the authors had modified the original Delphi technique.

When analyzing the number of high-quality characteristics reported by studies, we found that 46 (63.89%) reported 7 or more characteristics, while 3 (4.17%) reported 3 or less high-quality characteristics (Table 3).

Regarding expert panels, most of the studies (68.06%) included only healthcare experts, seven studies (9.72%) included both patient representatives and healthcare experts, and six (9.72%) included healthcare and epidemiology experts. Only two studies (2.78%) had a panel composed solely by patient representatives (Table 4). We subanalyzed studies not reporting conflicts of interest for both the expert panel and authors. Of these, 73.52% reported the conflicts of interest for authors, while 27.69% did not report conflicts of interest of either the authors or the experts.

Our study found that the quality of articles conducted with the Delphi technique, regarding breast cancer, is overall good. Characteristics considered of high quality were reported in more than half of the studies. However, the anonymization of answers and the disclosure of conflicts of interest were the least reported characteristics in the analyzed studies. The studies considered different types of experts for their panels, but the greater proportion (68.06%) used only healthcare experts as members of their panel. A variety of topics were assessed through the Delphi technique, with quality indicators, expressed needs and screening among the most frequent. The Delphi technique was also employed for subjects such as radiotherapy techniques and treatment in men.

The least reported characteristics in the included studies were anonymization and conflict of interest disclosure. As for anonymization, our tool proposed to report on whether the panel members were blind as for who other experts were, or, when not possible, at least for their answers to be anonymized (i.e. other experts would not be able to know who each answer belonged to). We recognize that it is possible for studies to have correctly anonymized answers, but for authors to not have disclosed such process within the manuscript. Nonetheless, we consider that this aspect is crucial to the integrity of the Delphi technique, as previous reports have previously stated [12].

Only 4.17% of the studies disclosed conflicts of interest for both experts and authors. Conflicts of interest could be as mild as wholeheartedly forgetting to report an accepted meal, or tragedies such as the Tuskegee Experiment [13–16]. Therefore, we believe that conflicts of interest should be completely and profoundly reported for all participants in studies using the Delphi technique to avoid damaging the integrity of the study.

More than half of the studies were reported to have modified the Delphi technique. Of these, only a third reported on the ways the authors had modified the original methods. This information is valuable, since mild changes to the technique may be required for specific research questions, but a significant modification may lead to the study to become qualitative in nature, but outside the scope of a Delphi technique. This issue has been reported to be addressed through assessment of quality of other aspects within the study, with tools such as the one we herein propose [12].

Finally, a significant proportion of the studies did not report on why experts were considered as such. Expert opinion is the backbone of the Delphi technique, which allows for research questions to be answered in the absence of an adequate alternative method [17]. However, very scarce information exists regarding what an expert should be. We advise for authors to thoroughly report the reasons behind choosing a specific expert panel, as for readers to understand their opinions better.

Our study is the first to assess quality of breast cancer studies conducted with the Delphi technique. We found that the quality of studies conducted with this technique, regarding breast cancer is overall good. However, the limitations of each study must be considered when applying their results to clinical practice.