Abstract
Objectives:
The aim of this study was to highlight features of pragmatic real-world integrative oncology research by applying the PRagmatic Explanatory Continuum Indicator Summary (PRECIS-2) criteria to an ongoing integrative oncology clinical trial. The ongoing trial is a preference-based randomized comparative effectiveness trial of mindfulness-based cancer recovery (MBCR) versus t'ai chi/qigong (TCQ) for cancer survivors (the Mindfulness and T'ai Chi for Cancer Health [MATCH] study). The primary outcome of the MATCH study is distress, and secondary outcomes are quality of life, sleep disturbance, and physical functioning. The clinical trial is being undertaken at tertiary care cancer centers across two sites in Canada: Calgary (AB) and Toronto (ON), with a sample of 600 cancer survivors who have finished all cancer treatments and are distressed.
Methods and Results:
The MATCH trial was scored on the explanatory–pragmatic continuum for each of the nine domains of the PRECIS-2 criteria on a scale of 1–5, and was rated as more explanatory than pragmatic, despite initial design efforts being more pragmatic. Areas that were least pragmatic were methods of recruitment, follow-up, and intervention delivery. The more pragmatic areas were setting, outcomes, and data analysis.
Conclusions:
More efforts toward conducting pragmatic trials are needed in the field of integrative oncology, as cancer-care institutions and policy makers are looking for sustainable interventions within already established treatment models. The PRECIS-2 criteria can help researchers meet these goals in the planning stages of trial development.
Real-World Research in Integrative Oncology
T
This move toward pragmatism contrasts with traditional research designs adopted from conventional medical research such as the randomized controlled trial (RCT). RCTs are the gold-standard design used to determine treatment efficacy, addressing questions of whether the treatment is better than another option (often usual care) under tightly controlled conditions. 4 While these designs maximize internal validity and allow us to draw causal conclusions about the efficacy of treatments, they fall short when it comes to evaluating effectiveness in real-world settings. 3,5 If integrative oncology as a field wishes to influence models of care, it can be argued that researchers and clinicians need to consider effectiveness, uptake, and implementation of interventions, not only efficacy. 6
Defining Pragmatism
Research that aims to determine the effectiveness of interventions in real-world settings using real-world personnel and resources is often called “pragmatic.” However, pragmatism in research exists on a continuum; all studies are pragmatic to a certain degree, but this has not often been well defined. While some investigators describe pragmatism as an attitude toward trial design, others go further to define it as a characteristic of the trial itself, often employing a continuum with “explanatory” at one end and “pragmatic” at the other. 7,8
Explanatory trials, also known as efficacy trials, primarily aim to identify a causal relationship between an intervention and some outcome (psychological or physiologic), 7,9 and whether the intervention would work under ideal circumstances. In contrast, pragmatic trials usually involve a choice between two or more interventions (comparative effectiveness trials), are conducted in “normal practice” settings, and primarily measure the clinical effect of the interventions. 3,4,10 Explanatory trials have a more rigid protocol with little flexibility for deviation, are highly selective in their inclusion of participants, and the trial outcomes are often more relevant to the investigators than to the patients. Conversely, the protocols of pragmatic trials are more reflective of usual care, the trials have broad inclusion criteria for participants, and the outcomes are directly relevant to patients. 11 In addition, the knowledge gained from pragmatic trials can aid decision making in healthcare and public policy. 7 There are also possible downsides to pragmatic trials; for example due to increased heterogeneity in the real world, often sample sizes need to be larger than in explanatory trials. If the costs of implementing the trial include per-patient intervention costs, this can make these trials more expensive.
A trial is not usually wholly pragmatic or explanatory. 12 An RCT can at one end investigate whether the intervention would work in ideal conditions, such as a laboratory (high internal validity or highly explanatory), or if it would work in everyday practice or usual care settings (high external validity or highly pragmatic). 11 The intermediate stages on the continuum are indicative of trial designs that become less explanatory and more pragmatic from one end of the continuum to the other. 10 The exact position of a trial on this continuum has implications for the usefulness of the trial to a wide range of stakeholders such as researchers, funding bodies, patients, clinicians, and policy makers. 3
Pragmatic trials are typically easier to undertake in a routine clinical and healthcare setting than explanatory trials. Interventions that work in a pragmatic setting are also highly likely to work in a controlled or explanatory setting, 5 although the reverse is not always true. This means that a highly successful explanatory trial does not guarantee that the intervention will work in usual care settings. In addition, if an intervention fails to work in a highly explanatory trial, it is highly unlikely to work in a regular clinical setting. Hence, moving to more pragmatic designs at the outset can be more efficient by eliminating the intermediate step of determining efficacy, and moving directly to the harder test of effectiveness and implementation, often at a lesser cost.
Measuring Pragmatism
Helpfully, there has been a tool developed to guide researchers in both the design and evaluation of pragmatic trials. The PRECIS-2 (PRagmatic Explanatory Continuum Indicator Summary-2) is the second iteration of the original PRECIS tool developed collaboratively by clinicians, researchers, and policy makers. 8 It provides descriptions, a scoring system, and a pictorial representation of the pragmatic features of a trial. 13 The tool has nine dimensions for assessing the level of pragmatism in a clinical trial on the explanatory–pragmatic continuum, these are as follows: (1) Eligibility criteria; (2) Recruitment Path; (3) Setting; (4) Organization: expertise and resources needed to deliver the intervention; (5) Flexibility in delivery of the intervention; (6) Flexibility of Adherence of participants to intervention; (7) Follow-up of participants; (8) Primary Outcome and its relevance to the participants; and (9) Primary Analysis: How much of the data are included. 14 Each of these domains is scored on a scale of 1–5 from highly explanatory to highly pragmatic. See Table 1 for detailed descriptions and scoring criteria. The score achieved on this scale is indicative of the agreement between the clinical trial's design and its intended objective. If the trial was meant to be highly pragmatic, it should have higher scores on the tool. Generally, the tool should be used at the conception and design stage of a clinical trial, but it can be a useful exercise if done afterward to see where the trail fell short, or during an ongoing trial if your aim is to adapt the trial to be more or less pragmatic. With adaptive designs, there is latitude to make adjustments to elements such as the allocation ratio, the sample size, or inclusion/exclusion criteria depending on how the trial is going; some of these adaptations could be done to make the trial more pragmatic, but it is much harder to do this once the study is already underway.
The criteria for scoring are adapted from Loudon et al. 14
MBCR, mindfulness-based cancer recovery; TCQ, t'ai chi/qigong.
The best way to understand how the PRECIS-2 tool works is to apply it to an example. This team is currently conducting a study called the MATCH (Mindfulness and T'ai Chi for Cancer Health) trial. 15 In its conception and design, this trial was meant to be closer to the pragmatic/effectiveness end of the continuum than the explanatory/efficacious end; however, during the design phase the authors did not have the PRECIS-2 criteria in mind. In this article, they evaluated the trial design of the MATCH study against each of the dimensions of the PRECIS-2 tool.
Case Example: The MATCH Study
The MATCH study is an ongoing trial designed to be a pragmatic preference-based comparative effectiveness trial comparing two mind–body interventions, mindfulness-based cancer recovery (MBCR) and t'ai chi/qigong (TCQ), for the health of cancer survivors. 15 The authors are recruiting distressed cancer survivors (distress score ≥4 on the distress thermometer [0–10] with any type of cancer except brain, and at any stage except metastatic). Upon eligibility, each participant is given the choice of taking his or her preferred intervention (MBCR or TCQ) or being randomly assigned to one if he or she has no preference. In the preference arm, comparing participants who received MBCR and TCQ allows us to determine effectiveness under more real-world conditions where participants choose their interventions, whereas in the randomized arm this comparison allows us to determine causality of effects between interventions due to randomization, which creates equal groups at baseline under more controlled conditions. Comparing the two arms with each other will also allow us to isolate the effect of preference on outcomes within each intervention. Each preference and nonpreference arm also has a randomized waitlist control; thus, participants may start the intervention immediately or 3–4 months later. Participants in the immediate groups are assessed at baseline, postintervention, and 6 months follow-up, whereas those in the waitlist groups are assessed four times: at baseline, postwaiting, postintervention, and 6 months follow-up. The primary outcome is mood disturbance, self-reported by the participant. Secondary outcomes include quality of life, psychological function, sleep quality, fatigue, and physical functioning (strength, balance, flexibility, and speed). 15 At each assessment, a variety of questionnaires, physiologic tests (blood pressure, ECG), and physical tests are administered, and blood and saliva samples are collected for biomarker analysis (e.g., cortisol, cytokines, and telomeres).
Application of the PRECIS-2 to the MATCH Study
There are four steps involved in using the PRECIS-2 tool 14 :
Step 1: Understand the purpose of the trial. To begin, the investigators should be clear about whether they wish for the intervention to have an explanatory/efficacy or pragmatic/effectiveness approach.
Step 2: Consider the choice for the trial design elements (explanatory vs. pragmatic) for each of the nine domains of the PRECIS-2 tool, using the descriptions of each domain as provided in Table 1.
Step 3: Rate the choices made for each of the nine PRECIS-2 domains on a scale of 1–5 (highly explanatory to highly pragmatic). The score assigned to each domain should be justified through rationale.
Step 4: Review the resultant PRECIS-2 wheel that graphs the nine dimensions, and determine whether the trial design supports the overall aim of the trial as identified in step 1.
The steps described above are often iterative between steps 2 and 4, especially in the stage of initially designing a trial, so that if the wheel dips too far toward explanatory or pragmatic and is in contrast to the purpose of the trial as decided in Step 1, other choices can be made to better fit the purpose and fill out the wheel.
Raters were the trial principal investigator (n = 1) and other investigators (n = 3) working on this trial. Steps 3 and 4 of the procedure were applied until consensus was reached on scores.
Each of the nine PRECIS-2 domains received the following scores: Eligibility: 3; Recruitment: 2; Setting: 4; Organization: 3; Flexibility of delivery of intervention: 2; Flexibility of intervention adherence: 2; Follow-up: 1; Primary outcome: 5; and Analysis: 5. The rationales for the scores are presented in Table 1 along with the criteria used to score these domains. The PRECIS-2 “wheel” (Fig. 1) is a visual representation of the MATCH study on the explanatory–pragmatic continuum. Much of the wheel being nearer the hub, the trial takes more of an explanatory approach.

PRECIS-2 Scoring wheel for the MATCH study. This is a visual representation of pragmatism of the trial on the explanatory–pragmatic continuum. Most sections of the wheel are closer to the central hub, and only a small segment is closer to the rim of the PRECIS-2 wheel, indicating that the trial is more explanatory than pragmatic. MATCH, Mindfulness and T'ai Chi for Cancer Health; PRECIS-2, PRagmatic Explanatory Continuum Indicator Summary-2.
Implications: Pragmatism in MATCH
The results of the PRECIS-2 application to the MATCH study were somewhat surprising but very useful, as they showed much lower levels of pragmatism than hoped for when conceptualizing and designing the study. Four of the nine domains of the PRECIS-2 (recruitment, flexibility of intervention delivery, flexibility of adherence to intervention, and follow-up) stood out for being rated as more explanatory than pragmatic, while only three domains (setting, primary outcome, and analysis) qualified as more pragmatic. Two domains (eligibility and setting) were rated as equally explanatory or pragmatic. Limitations to this approach were that there were only four assessors of the criteria (all of whom were involved in designing or running the study), and no stakeholders or patient advisors were included.
As the MATCH study is an ongoing trial, the process of applying the PRECIS-2 allowed us to understand the elements of study design that could have been made more applicable to real-world settings. Given a chance to redesign the study to be more pragmatic, the authors would (1) recruit cancer survivors by following-up at clinics after they complete their last treatments by training the existing staff and relying more heavily on self-referral, and visiting community-based nonprofit cancer support agencies to distribute information. While this typically results in slower recruitment than sending personal invitation letters, the flow of patients into the programs using this method would give better estimates of actual organic interest levels; (2) reduce the frequency of assessment visits or make them easier and less time-consuming (e.g., shorter surveys, online completion or completion at the same time as attending classes, elimination of physiologic assessments and blood draws); (3) incorporate more measures relevant to patients and the healthcare system, such as return to work and healthcare utilization; (4) restructure the intervention programs to allow more flexibility in terms of delivering the modules of the intervention; (5) adapt parts of the program so they could be delivered online or remotely; and (6) allow patients to provide their preference for the timing of the mindfulness/t'ai chi intervention, which is delivered three times yearly. Such changes to the study design based on the PRECIS-2 domains would likely increase the applicability of the findings to a larger audience and ease the implementation of the programs to its stakeholders. One can see how valuable discussion of the PRECIS-2 domains would be during the earliest phases of study planning, when these decisions could easily be put into effect.
Although the authors were trying to be pragmatic in the design of the trial to better ease transition into program implementation, they fell into patterns of efficacy methodology that researchers are familiar with, and granting agencies will fund. The trial will still provide very valuable information about efficacy and effectiveness for these participants, and does have strengths in its relatively broad inclusion (almost all cancer sites, most stages, two geographical locations) and a wide range of outcome measures that will add to authors' knowledge of biomarkers of change. However, if the purpose of the trial is to segue into implementation and evaluate real-world program delivery, one could have made many more pragmatic design choices.
When Is Pragmatism Appropriate?
The authors have been arguing for the value of conducting pragmatic trials in integrative oncology settings as a way to improve implementation and uptake of evidence-based programs. However, the question of when a pragmatic trial is appropriate is not always straightforward. For example, within integrative oncology when investigating new or controversial treatments without strong Level 1 evidence (multiple large RCTs; meta-analyses of RCTs), tightly designed efficacy trials may still be more appropriate. Because integrative oncology treatments may undergo more scrutiny from the medical community and other critics, researchers have to adhere to stringent explanatory design requirements at early stages. For example, in acupuncture efficacy trials they carefully select sham treatments that allow blinding of participants and assessors, strictly monitor treatment fidelity, and use objective outcome measures where possible. To obtain funding and ethics approval, researchers may also have to be especially stringent, creating a tension between the desire to be pragmatic and demands from critics and funding bodies to be tightly controlled.
There is a role for pragmatism in areas where the evidence base is strong; for example, the authors recently conducted a pragmatic acupuncture trial comparing group with individual acupuncture for cancer pain. In this area, there is strong evidence supporting the efficacy of acupuncture for pain control in general, and positive evidence for cancer pain, so they investigated whether group application in a nonblinded study within a clinical cancer rehabilitation program would be as good as traditional individual treatments at a much lower cost. There was no sham or no-treatment control group, participants knew what they were getting, and the treatment was individualized to each client as would be the case in the community. These pragmatic design elements make it much more likely that the program will continue in this setting once the research study is complete.
Strategies to Increase Pragmatism
There is an increasing need to find ways to move integrative oncology interventions from research into practice. Integrative oncology researchers and clinicians now have a growing list of efficacious interventions, 16 but they are not routinely available or offered to many survivors who may benefit. One strategy is to conduct research that is more pragmatic, which should lead to easier implementation and sustainability. In the current structure of healthcare delivery, policy makers are looking for sustainability of interventions within already established treatment models. If clinical researchers can create and evaluate interventions in real-world settings using the staff and patient who are already there, chances of sustainability are greatly improved. The value of using the PRECIS-2 tool in this context can clearly be seen in this example, which illustrates how it provides valuable insight and may help preempt practical problems when applied in the stages of trial development.
Conclusions
To apply the PRECIS-2 tool effectively, a clear philosophical position on the issue of pragmatism versus efficacy must be first delineated. Following this, a discussion at the design stage is necessary between stakeholders who have knowledge of usual care for the population and problem of interest; details of the operational and technical aspects of the interventions; an understanding of the needs of the population to be served; and an understanding of the PRECIS-2 domains. Hence, the oft-heard call for patient engagement in research design and implementation, and the need to involve administrators, policy makers, and front-line service providers in these design decisions.
While creating research that is pragmatic is a significant departure from the training and experience many researchers possess and may seem difficult and even methodologically sloppy, the potential value in terms of broadening reach makes these attempts crucial if one wishes to succeed in the mission of improving the lives of cancer survivors through integrative oncology.
Footnotes
Acknowledgments
Dr. Carlson holds the Enbridge Research Chair in Psychosocial Oncology, cofunded by the Canadian Cancer Society Alberta/NWT Division and the Alberta Cancer Foundation, and a Canadian Institutes of Health Research (CIHR) Strategy for Patient-Oriented Research (SPOR) Innovative Clinical Trials Mentorship Chair, which funds the TRACTION (Training in Research and Clinical Trials in Integrative Oncology) program. Postdoctoral Fellows Dr. Oberoi and Dr. Subnis are supported by the CIHR SPOR TRACTION program. The MATCH study is funded by the Lotte & John Hecht Memorial Foundation.
Author Disclosure Statement
No competing financial interests exist for any of the authors.
