The PROTEUS-Trials Consortium: Optimizing the use of patient-reported outcomes in clinical trials

Abstract

Background:

The assessment of patient-reported outcomes in clinical trials has enormous potential to promote patient-centred care, but for this potential to be realized, the patient-reported outcomes must be captured effectively and communicated clearly. Over the past decade, methodologic tools have been developed to inform the design, analysis, reporting, and interpretation of patient-reported outcome data from clinical trials. We formed the PROTEUS-Trials Consortium (Patient-Reported Outcomes Tools: Engaging Users and Stakeholders) to disseminate and implement these methodologic tools.

Methods:

PROTEUS-Trials are engaging with patient, clinician, research, and regulatory stakeholders from 27 organizations in the United States, Canada, Australia, the United Kingdom, and Europe to develop both organization-specific and cross-cutting strategies for implementing and disseminating the methodologic tools. Guided by the Knowledge-to-Action framework, we conducted consortium-wide webinars and meetings, as well as individual calls with participating organizations, to develop a workplan, which we are currently executing.

Results:

Six methodologic tools serve as the foundation for PROTEUS-Trials dissemination and implementation efforts: the Standard Protocol Items: Recommendations for Interventional Trials-patient-reported outcome extension for writing protocols with patient-reported outcomes, the International Society for Quality of Life Research Minimum Standards for selecting a patient-reported outcome measure, Setting International Standards in Analysing Patient-Reported Outcomes and Quality of Life Endpoints Data Consortium recommendations for patient-reported outcome data analysis, the Consolidated Standards for Reporting of Trials-patient-reported outcome extension for reporting clinical trials with patient-reported outcomes, recommendations for the graphic display of patient-reported outcome data, and a Clinician’s Checklist for reading and using an article about patient-reported outcomes. The PROTEUS-Trials website (www.TheProteusConsortium.org) serves as a central repository for the methodologic tools and associated resources. To date, we have developed (1) a roadmap to visually display where each of the six methodologic tools applies along the clinical trial trajectory, (2) web tutorials that provide guidance on the methodologic tools at different levels of detail, (3) checklists to provide brief summaries of each tool’s recommendations, (4) a handbook to provide a self-guided approach to learning about the tools and recommendations, and (5) publications that address key topics related to patient-reported outcomes in clinical trials. We are also conducting organization-specific activities, including meetings, presentations, workshops, and webinars to publicize the existence of the methodologic tools and the PROTEUS-Trials resources. Work to develop communications strategies to ensure that PROTEUS-Trials reach key audiences with relevant information about patient-reported outcomes in clinical trials and PROTEUS-Trials is ongoing.

Discussion:

The PROTEUS-Trials Consortium aims to help researchers generate patient-reported outcome data from clinical trials to (1) enable investigators, regulators, and policy-makers to take the patient perspective into account when conducting research and making decisions; (2) help patients understand treatment options and make treatment decisions; and (3) inform clinicians’ discussions with patients regarding treatment options. In these ways, the PROTEUS Consortium promotes patient-centred research and care.

Keywords

Patient-reported outcomes clinical trials reporting methods protocols measure selection data visualization

Background

Patients, clinicians, regulators, and policy-makers value data on patient-reported outcomes (PROs), such as symptoms, functioning, and health-related quality of life, from clinical trials to inform decision-making.^1–8 To be most informative, the PRO methods need to be specified appropriately, the PRO endpoints measured effectively, the PRO data analysed properly, and the PRO results reported clearly to multiple stakeholders. However, there is evidence that these goals are frequently not met at the protocol, analysis, reporting, or application levels.^6,9–13

Methodologic tools have been developed to help clinical trialists optimize the capture and communication of PROs.^14–19 These tools, developed in partnership with patients and other stakeholders, provide guidance on designing the PRO aspects of clinical trials, collecting and analysing the PRO data, and interpreting and reporting the PRO findings. However, these tools require a coordinated, stakeholder-driven implementation and dissemination strategy to ensure their usefulness in practice.

In 2018, we formed the PROTEUS Consortium (Patient-Reported Outcomes Tools: Engaging Users & Stakeholders), first with a focus on advancing the use of PROs in clinical research (PROTEUS-Trials) and, subsequently, expanded to address the use of PROs in clinical practice (PROTEUS-Practice).²⁰ PROTEUS-Trials are partnering with stakeholders to promote the application of existing tools to optimize the capture and communication of PROs in clinical trials. The goal is to give researchers guidance to help them conduct their studies so that patients can make better-informed health decisions; regulators and policy-makers can take the patient perspective into account in their deliberations; and clinicians can have reliable, relevant, and interpretable PRO data to discuss treatment options with patients. Stated differently, the PROTEUS-Trials Consortium aims to promote patient-centred research and care at multiple levels.

Methods

The work of PROTEUS-Trials is guided by the Knowledge-to-Action framework, an implementation and dissemination, or knowledge translation, model with two parts.²¹ First is knowledge development, which creates knowledge tools and products through inquiry and synthesis. Second is the action cycle, which encourages dissemination of the knowledge tools by identifying knowledge gaps, adapting the knowledge to the local context, assessing facilitators and barriers to knowledge use, and implementing tailored interventions. The cycle continues with monitoring and sustaining knowledge use while evaluating outcomes.

In the case of PROTEUS-Trials, the knowledge tools are the existing methodologic tools for integrating PROs in clinical trials, mentioned above and described in more detail below.^14–19 Therefore, PROTEUS-Trials focuses primarily on the action cycle. First, the Principal Investigators (C.S. and M.B.), Project Manager (N.C.), and other Steering Committee members (M.K., B.B.R., A.B., M.C., E.T., and A.W.W.) who played key roles in developing the PRO methodologic tools assembled the consortium, engaging relevant patient, clinician, research, and regulatory groups. Second, a consortium kick-off webinar reviewed PROTEUS’ objectives and approach. Third, we held calls with representatives from each organization to obtain their input on knowledge translation strategies specific to their contexts and needs, and more generally. Fourth, we held an in-person meeting (June 2019) to develop and prioritize cross-cutting knowledge translation strategies. Fifth and ongoing, we are executing the organization-specific and cross-cutting knowledge translation strategies. The consortium has a cancer focus because some of the methodologic tools were developed in oncology contexts; however, the expectation is that this work is largely applicable across health conditions.

Results

Consortium formation

There are 27 organizations with participants in the PROTEUS-Trials Consortium (Table 1). Membership includes patient and clinician advocacy groups, government and regulatory agencies, clinical trial cooperative groups, organizations focused on research methods, and funding agencies.

Table 1.

Organizations with PROTEUS-Trials participants.^a

AcademyHealth

American Cancer Society

American Society of Clinical Oncology

American Society for Radiation Oncology

Australian Clinical Trials Alliance

Canadian Association of Radiation Oncology

Cancer Australia

Consolidated Standards for Reporting of Trials (CONSORT)

Critical Path Institute PRO Consortium

European Medicines Agency-Scientific Advice Working Party/Dutch Medicines Evaluation Board

European Organisation for the Research and Treatment of Cancer

Food and Drug Administration

Health Canada

Industry (GlaxoSmithKline)

International Society for Quality of Life Research

ISPOR

Journal editor perspective

Medicines and Healthcare Products Regulatory Agency

National Cancer Institute

National Cancer Research Institute

National Clinical Trials Network PRO representatives

National Coalition for Cancer Survivorship

National Institute for Health and Care Excellence

Oncology Nursing Society

Patient-Centered Outcomes Research Institute

Society for Clinical Trials

Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)

Participating in PROTEUS does not imply endorsement of any particular PRO tools or guidance documents.

Methodologic tools

The consortium’s knowledge translation activities focus on these six methodologic tools, which are summarized in Figure 1.

Figure 1.

Methodologic tools for implementing patient-reported outcomes (PROs) in clinical trials.

Guidelines for inclusion of PROs in clinical trial protocols: the Standard Protocol Items: Recommendations for Interventional Trials-PRO extension

The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)-PRO extension recommends best practices for writing the PRO aspects of randomized controlled trial protocols.¹⁴ It extends the 2013 SPIRIT guidance that identifies the minimum elements generally required in clinical trial protocols.²² The SPIRIT-PRO extension builds on the SPIRIT guidance by addressing the minimum elements related to PROs that should be included in clinical trial protocols. SPIRIT-PRO was developed through a Delphi process and consensus meeting, which was informed by a list of PRO-specific protocol items generated from a systematic review of existing guidance and a survey of international stakeholders. The resulting recommendations provide guidance on PRO issues relating to the trial rationale, objectives, eligibility criteria, concepts used to evaluate the intervention, timepoints for assessment, PRO instrument selection and measurement properties, data collection plan, translation to other languages, proxy completion, strategies to minimize missing data, and whether PRO data will be monitored during the study to inform clinical care.

International Society for Quality of Life Research minimum standards for PRO measures used in patient-centred outcomes and comparative effectiveness research

In 2013, the International Society for Quality of Life Research (ISOQOL) conducted a project to recommend minimum standards for PRO measures in patient-centred outcomes and comparative effectiveness research.¹⁵ To develop these standards, the ISOQOL Task Force reviewed existing guidelines for the selection of PRO measures and surveyed ISOQOL members to obtain their input on potential PRO standards. The final recommendations, which helped to inform the Patient-Centered Outcomes Research Institute’s methodology standards related to PROs,²³ address documentation of the conceptual and measurement model for the PRO measure; evidence for reliability and validity; interpretability of scores; translation quality; and patient and investigator burden.

Setting International Standards in Analysing PROs and Quality of Life Endpoints Data Consortium

The European Organisation for the Research and Treatment of Cancer (EORTC) formed the Setting International Standards in Analysing PROs and Quality of Life Endpoints Data (SISAQOL) Consortium to establish international standards in analysing PRO endpoint data.¹⁶ The consortium includes researchers, statisticians, clinician researchers, regulators, patients, and stakeholders with multidisciplinary expertise and international perspectives. The goal is to set standards that are methodologically rigorous, comprehensive, and practical. Initial SISAQOL recommendations focus on establishing the research objectives a priori, distinguishing between domains with confirmatory analyses (superiority, non-inferiority, and equivalence) versus those that are exploratory/descriptive. The recommendations also address specifying the assumptions, endpoints of interest, recommended statistical models, and handling missing data. The initial SISAQOL work is continuing through the SISAQOL-IMI (Innovative Medicines Initiative) Consortium.²⁴

Reporting of PROs in randomized trials: the Consolidated Standards of Reporting Trials-PRO extension

The CONSORT (Consolidated Standards of Reporting Trials) guidance provides recommendations for publications reporting clinical trial results.²⁵ In 2013, a PRO-specific extension was published, which includes recommendations for identifying the PRO as an outcome in the abstract, providing the background and rationale for PRO assessment, describing the PRO hypothesis, providing evidence of the PRO measure’s reliability and validity, detailing the mode of PRO completion, describing missing PRO data rates and statistical methods, reporting baseline PRO data and results for each domain and timepoint pre-specified for analysis, and discussing the PRO-specific limitations and generalizability.¹⁷

Recommendations for graphically displaying PRO data

A specific issue related to reporting PRO results is the best way to graphically report the data so that patients and clinicians can easily and accurately interpret the PRO findings.¹⁸ Patients and clinicians value the information provided by PRO results from clinical trials, but due to variations in how PRO measures are scored and scaled, and the different approaches to analysing them (e.g. modelling scores over time and proportions meeting a responder definition (i.e. improved/stable/worsened)), they have also reported challenges interpreting PRO results.⁶ To address these issues, a multi-phase research study explored approaches for displaying clinical trial PRO results graphically to identify formats that are interpreted most accurately and rated clearest.^26–30 These results informed the development of stakeholder-engaged, evidence-driven recommendations for how to display PRO data to promote understanding and use.¹⁸

Clinician’s Checklist for reading and using an article about PROs

As noted above, clinicians value the PRO information from clinical trials, but they also report challenges interpreting the findings so that they can use them in practice.⁶ Another tool developed to address this issue is the Clinician’s Checklist for reading and using an article about PROs.¹⁹ The checklist includes five questions: (1) Was the PRO assessment strategy appropriate? (2) Did they measure PRO effectively? (3) Should I believe the results? (4) Were the results placed in clinical context? (5) Do the results apply to my patients? For each question, the checklist provides a description of what to look for in the article, guiding clinicians who are not expert in PRO research on how to use PRO findings in their practice.

Workplan

Based on our consultations with the PROTEUS-Trials participating organizations individually, and then jointly at the in-person meeting, we developed a prioritized knowledge translation workplan, including the below activities.

Website

www.TheProteusConsortium.org includes back ground on PROTEUS-Trials, links to the six methodologic tools, and the resources developed to-date, including those listed below.

Roadmap

We created a visual display of the clinical trial continuum and where each of the six methodologic tools apply (Figure 2).

Figure 2.

PROTEUS-Trials roadmap.

Web tutorials

The PROTEUS-Trials Steering Committee members who developed the tools recorded a series of presentations on PROs, PROTEUS-Trials, and the PROTEUS-Trials tools. High-level presentations are directed at, for example, principal investigators who want to ensure that the researchers responsible for the PRO aspects follow best practices. Intermediate-level presentations provide sufficient detail for someone who is reviewing the PRO aspects of a trial (e.g. a manuscript peer-reviewer), but not someone implementing the recommendations directly. Advanced-level presentations are directed to the researchers responsible for implementing the recommendations themselves.

Checklists

A checklist, with citations, for each of the six tools helps users follow the guidance document recommendations without having to refer to lengthy, academic publications.

Handbook

To complement the web tutorials, a handbook provides a self-guided approach to learning about the tools and recommendations.

Publications

Several papers were prioritized to advance the PROTEUS-Trials objectives. For example, we conducted a project comparing different guidance documents’ recommendations for PRO measure selection.³¹ We also developed recommendations for grant applicants and reviewers regarding the key information to include/review in grant proposals.³² A paper that highlights some of the ‘greatest hits’ (focused on value-added) of PROs in clinical trials is forthcoming.

Meetings/presentations/workshops/webinars

Most of our organization-specific strategies involve meetings, presentations, workshops, and webinars to key groups. We have presented on PROTEUS on over 35 occasions to a variety of audiences, in-person and virtually. These presentations are a critical aspect of engaging with different stakeholders and reaching relevant audiences to disseminate information about the tools.

Communication strategy

We are developing communications strategies to further publicize our work, ensure we reach key audiences, and promote the consortium’s objectives.

Discussion

The assessment of PROs in clinical trials has enormous potential to promote patient-centred care, but for this potential to be realized, PRO data must be captured effectively and communicated clearly to diverse audiences. The PROTEUS-Trials Consortium aims to engage with stakeholders to develop organization-specific and cross-cutting strategies for implementing and disseminating methodologic tools for optimizing the use of PROs in clinical trials. It is always advisable to include a researcher with PRO expertise on the clinical trial team. This article summarizes the tools and resources available to these PRO experts, as well as others who engage with PROs at a higher level (e.g. peer-reviewers and principal investigators). Future work will evaluate the impact of PROTEUS-Trials on the capture and communication of PROs from clinical trials.

While the PROTEUS-Trials Consortium has focused on six core tools, users are encouraged to consider other guidance as applicable to their purpose and jurisdiction. For example, if planning a regulatory submission to the Food and Drug Administration, its PRO guidance would be most applicable.¹ Reassuringly, the PROTEUS paper comparing various guidance documents regarding PRO measure selection found general consistency in their recommendations.³¹

In summary, through the PROTEUS-Trials Consortium, we hope that researchers will be better able to generate PRO data from clinical trials to (1) enable investigators, regulators, and policy-makers to take the patient perspective into account when conducting research and making decisions; (2) help patients understand treatment options and make treatment decisions; and (3) inform clinicians’ discussions with patients regarding treatment options. In these ways, the PROTEUS Consortium aims to promote patient-centred research and care.

Footnotes

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: C.S., M.B., and N.C. report an unrestricted grant from Pfizer for a separate PROTEUS project, for which B.B.R., M.C., E.T., and A.W.W. also receive payments as Steering Committee members. C.S., A.W.W., and M.B. have had previous funding from Genentech outside of the submitted work. C.S. has received consulting fees from Janssen, via Health Outcomes Solutions. E.T. has received funding from Pfizer for a separate project. M.C. reports personal fees from Astellas, from Takeda, Glaukos, Merck, Daiichi Sankyo, CIS Oncology, Aparito Ltd., and GSK, and grants from Health Data Research UK, Wellcome Trust, Alan Turing Institute, Research England, and UK Research and Innovation outside the submitted work; M.C. is a National Institute for Health Research (NIHR) senior investigator and received funding from the NIHR Birmingham Biomedical Research Centre, NIHR Surgical Reconstruction and Microbiology Research Centre and NIHR ARC West Midlands at the University of Birmingham and University Hospitals Birmingham National Health Service Foundation Trust, Health Data Research UK, Innovate UK (part of UK Research and Innovation), Macmillan Cancer Support, and UCB Pharma. A.W.W. reports consulting fees from ViiV, Osmotica, Pfizer, GSK, Gilead, and Johnson & Johnson. A.B. is a member of the EORTC; he has research grants from BI, Pfizer, and BMS to support the EORTC. No other authors have conflicts to report. Participating in PROTEUS does not imply endorsement of any particular PRO tools or guidance documents. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: PROTEUS-Trials have received funding through an Engagement Award from the Patient-Centered Outcomes Research Institute (12565-JHU) and through an unrestricted grant from Genentech, a Roche Company. The funders had no role in the study design, collection, analysis, and interpretation of data, or writing of the report; there were no restrictions regarding publication.

ORCID iD

Claire Snyder

References

U.S.Food and Drug Administration. Guidance for industry: patient reported outcome measures: use in medical product development to support labeling claims. Federal Register 2009; 74(35): 65132–65133.

Acquadro

Berzon

Dubois

, et al. Incorporating the patient’s perspective into drug development and communication: an ad hoc task force report of the Patient-Reported Outcomes (PRO) Harmonization Group meeting at the Food and Drug Administration, February 16, 2001. Value Health 2003; 6(5): 522–531.

H-J

Ringash

Brundage

, et al. Added value of health-related quality of life measurement in cancer clinical trials: the experience of the NCIC CTG. Expert Rev Pharmacoecon Outcomes Res 2010; 10(2): 119–128.

Till

Osoba

Pater

, et al. Research on health-related quality of life: dissemination into practical applications. Qual Life Res 1994; 3(4): 279–283.

Lipscomb

Gotay

Snyder

, et al. Outcomes assessment in cancer: measures, methods, and applications. Cambridge: Cambridge University Press, 2005.

Brundage

Bass

Jolie

, et al. A knowledge translation challenge: clinical use of quality of life data from cancer clinical trials. Qual Life Res 2011; 20(7): 979–985.

Bezjak

Skeel

, et al. Oncologists’ use of quality of life information: results of a survey of Eastern Cooperative Oncology Group physicians. Qual Life Res 2001; 10(1): 1–13.

Kluetz

O’Connor

Soltys

. Incorporating the patient experience into regulatory decision making in the USA, Europe, and Canada. Lancet Oncol 2018; 19(5): e267–e274.

Kyte

Retzer

Ahmed

, et al. Systematic evaluation of patient-reported outcome protocol content and reporting in cancer trials. J Natl Cancer Inst Monogr 2019; 111(11): 1170–1178.

10.

Efficace

Fayers

Pusic

, et al. Quality of patient-reported outcome reporting across cancer randomized controlled trials according to the CONSORT patient-reported outcome extension: a pooled analysis of 557 trials. Cancer 2015; 121(18): 3335–3342.

11.

Mercieca-Bebber

Friedlander

Calvert

, et al. A systematic evaluation of compliance and reporting of patient-reported outcome endpoints in ovarian cancer randomised controlled trials: implications for generalisability and clinical practice. J Patient Rep Outcomes 2017; 1(1): 5.

12.

Bylicki

Gan

Joly

, et al. Poor patient-reported outcomes reporting according to CONSORT guidelines in randomized clinical trials evaluating systemic cancer therapy. Ann Oncol 2014; 26(1): 231–237.

13.

St Germain

Denicoff

Torres

, et al. Reporting of health-related quality of life endpoints in National Cancer Institute-supported cancer treatment trials. Cancer 2020; 126(11): 2687–2693.

14.

Calvert

Kyte

Mercieca-Bebber

, et al. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: the SPIRIT-PRO extension. JAMA 2018; 319: 483–494.

15.

Reeve

Wyrwich

, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res 2013; 22(8): 1889–1905.

16.

Coens

Dueck

, et al. International standards for the analysis of quality of life and patient reported outcomes endpoints in cancer randomised controlled trials: recommendations based on critical reviews of the literature and international multi-expert, multi-stakeholder collaborative process. Lancet Oncol 2020; 21: e83–96.

17.

Calvert

Blazeby

Altman

, et al. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA 2013; 309: 814–822.

18.

Snyder

Smith

Holzner

, et al. Making a picture worth a thousand numbers: recommendations for graphically displaying patient-reported outcomes data. Qual Life Res 2019; 28(2): 345–356.

19.

Bradford

Velanovich

, et al. Clinician’s checklist for reading and using an article about patient-reported outcomes. Mayo Clin Proc 2014; 89(5): 653–661.

20.

The PROTEUS Consortium: patient-reported outcomes tools – engaging users & stakeholders, www.TheProteusConsortium.org (accessed 30 July 2021).

21.

Straus

Tetroe

Graham

. Knowledge translation is the use of knowledge in health care decision making. J Clin Epidemiol 2011; 64: 6–10.

22.

Chan

Tetzlaff

Altman

, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med 2013; 158: 200–207.

23.

Patient-Centered Outcomes Research Institute. Methodology standards associated with patient centeredness, https://www.pcori.org/research-results/about-our-research/research-methodology/pcori-methodology-standards#Associated%20with%20Patient-Centeredness (accessed 30 July 2021).

24.

SISAQOL-IMI: setting international standards of patient-reported outcomes quality of life endpoints in cancer clinical trials – IMI, https://www.sisaqol-imi.org/ (accessed 21 December 2021).

25.

Schulz

Altman

Moher

, et al. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340: c332.

26.

Brundage

Smith

Little

, et al. Communicating patient-reported outcome scores using graphic formats: Results from a mixed methods evaluation. Qual Life Res 2015; 24(10): 2457–2472.

27.

Smith

Brundage

Tolbert

, et al. Engaging stakeholders to improve presentation of patient-reported outcomes data in clinical practice. Support Care Cancer 2016; 24(10): 4149–4157.

28.

Brundage

Blackford

Tolbert

, et al. Presenting comparative study PRO results to clinicians and researchers: beyond the eye of the beholder. Qual Life Res 2018; 27(1): 75–90.

29.

Tolbert

Brundage

Bantug

, et al. Picture this: presenting longitudinal patient-reported outcome research study results to patients. Med Decis Making 2018; 38(8): 994–1005.

30.

Tolbert

Brundage

Bantug

, et al. In proportion: approaches for displaying patient-reported outcome research study results as percentages responding to treatment. Qual Life Res 2019; 28(3): 609–620.

31.

Crossnohere

Brundage

Calvert

, et al. International guidance on the selection of patient-reported outcome measures in clinical trials: a review. Qual Life Res 2021; 30(1): 21–40.

32.

Snyder

Gilbert

Moher

, et al. Recommendations for including or reviewing patient-reported outcome endpoints in grant applications. BMJ 2021; 373: n1367.