Promoting informed health choices: the long and winding road

Abstract

Introduction

When health professionals intervene in the lives of others, their actions sometimes inadvertently do more harm than good. The same risk is true for legislators or individuals making health choices. Informed health choices can increase the probability that patients benefit, that is that the wanted intervention effects will outweigh unwanted effects. For this reason, all health policies and practices should be informed by the best available evidence of intervention effects.

In this commentary, we consider some of the challenges that confront those promoting the evaluation of the effects of health interventions to inform health choices. By ‘health choice’ we mean any action that individuals or groups can choose to take in the belief that it will protect or improve their health or the health of others. ‘Health interventions’ include everything from dietary, device, surgical or pharmaceutical interventions for individuals to policies for population health.

We are not trying to cover everything that goes into making informed choices in this commentary. We focus particularly on the evaluation of the effects of health interventions. People make health choices based largely on what they believe will happen because of their choices (the expected effects). For informed health choices, reliable information about the probability of those effects is essential, but not sufficient. Other types of research are also important, as is efficiency, equity, and inclusivity. However, it is not possible to make informed judgements about efficiency and equity without reliable information about the effects of a health choice.

Our interest in informed health choices began in the 1970s. It started with raising questions about the opinions of authorities. This led to collaborating to help address some of the challenges considered in this commentary. Brief accounts of how we came to question authority and to recognise the need to collaborate can be found in Supplemental Appendix 1.

This is an abridged version of a longer commentary published in the James Lind Library.¹ It is based on discussions we had over several months, our experience, and feedback from colleagues. In Supplemental Appendix 1, we address how our experience may have affected this commentary. This, together with the references we cite, may help those bothering to reflect on our opinions to decide what to believe or do.

People have recognised for hundreds of years some of the problems we address. Some illustrative examples are outlined in Figure 1. However, over the past five decades, awareness of these problems and efforts to address them have increased dramatically. Illustrative examples of this can be found in Supplemental Appendix 2. There has been astonishing progress towards making informed health choices a reality.

Figure 1.

Illustrative examples of promoting evaluation of the effects of health interventions to inform health choices dating back to the fifth century BCE.*

To get to where we are today has been a long and winding road, and we still have a long way to go. For example, in 1987, Cynthia Mulrow showed that review articles in major medical journals were not using systematic, explicit methods to reduce the risk of being misled by bias or the play of chance.⁴ There has since been an explosion in the production and publication of systematic reviews, but there is unnecessary and confusing duplication of effort.^5–8 Access to high-quality reviews is also limited, partly because of poor communication and pay walls.^9,10 For these and other reasons, many health choices still are not informed by high-quality, up-to-date systematic reviews.^7,8,11

The fundamental problem underlying poorly informed health choices is uncritical reliance on poorly informed opinions. Often, authorities (and others) express strong opinions about the effects of interventions without being explicit about the basis for their opinions, or cherry-picking evidence. Those opinions influence choices without people questioning their basis. In the current political environment, with increasing authoritarianism and massive amounts of misinformation, this is particularly important for all kinds of interventions, not just healthcare interventions.^12–14

Other problems using evaluations of the effects of health interventions to inform policies and practice include:

inadequate consideration of the harmful effects of intervention

biased evaluations of intervention effects

conflicts of interest

biased reporting of research

deficient reporting of research

deficient reviews

ineffective and inefficient peer review

failure to keep reviews up to date

unnecessary duplication of effort

misinformation

limited access to trustworthy information about the effects of policies and practices

gaps between evidence-based recommendations and professional practice

inadequately informed policymaking

lack of patient and public participation in decision-making.

Collaboration is needed to address all these challenges. Like the importance of thinking critically about opinions, collaboration and collective action are more important than ever, given the many challenges facing humanity. These include massive inequities, pandemics, antimicrobial resistance, unsustainable use of resources, and climate change.

Uncritical reliance on inadequately informed opinions

Researchers, doctors, pundits, influencers and other authorities often disagree about the effects of interventions. This may be because their opinions are not routinely informed by systematic reviews. People frequently consider: Who expressed the opinion? How strong is their opinion? And how much experience do they have? The answers to these questions do not always provide a trustworthy basis for judging the reliability of opinions.

Improving the extent to which health choices are informed requires enabling people to avoid uncritical reliance on inadequately informed opinions. This depends on developing, evaluating, and implementing effective interventions to enable people to question the basis for opinions about the effects of interventions.

Inadequate consideration of harmful effects of intervention

Almost all health interventions have unwanted effects that must be weighed against potential desired effects. These include individual treatments, but also public health interventions and educational interventions.^15,16 Poor reporting of harms makes this weighing up difficult. Findings that suggest benefits tend to be emphasised, while potential harms are downplayed or ignored.^17,18 Moreover, reliable evidence of harmful effects, especially long-term harms, often lags behind evidence of beneficial effects.^17,19,20

Added to these deficits, press releases for research reports are often designed to attract favourable media attention, and news reports of those studies do the same.²¹ Most news reports about health interventions mention at least one benefit, but fewer than half mention or adequately discuss harms.²² Patients and health professionals tend to overestimate the benefits and underestimate the harms of health interventions.^23–25

Improving the chances that wanted intervention effects outweigh unwanted effects will depend on researchers, health professionals, policymakers, patients and the public ensuring that harmful effects are adequately measured, reported and considered.

Biased evaluations of intervention effects

The use of randomised trials to evaluate the effects of interventions has increased dramatically.⁸ But many evaluations (both randomised and non-randomised) do not take adequate steps to reduce the risk of bias.^26,27

When there are important uncertainties about the effects of health interventions, they should ideally be evaluated in well-designed randomised trials. The trials need to be large enough to yield reliable estimates of effects on important outcomes. As the Australian economist and politician Andrew Leigh writes at the end of his book describing randomised trials of all types of interventions: ‘If we let our curiosity roam free, we might be surprised how much we can learn about the world, one coin toss at a time’.²⁸

Because resources are limited, it is vital to identify important uncertainties about the effects of interventions and then set priorities for evaluations that minimise the risk of bias. Collaboration among research funders, research institutions, researchers, policymakers, patients and the public can help do this.²⁹

Conflicts of interest

In addition to wanting to help people, people may have other interests in promoting a particular intervention, for example, to make money. They may promote an intervention by exaggerating its benefits, ignoring potential harms, cherry-picking which information is used, or making exaggerated or false claims. Conversely, people may object to an intervention for a range of reasons, such as cultural practices.

Most research on funding bias (conflicting interests of funders) and conflicts of interest of researchers and authors of reviews or guidelines is devoted to evaluating the prevalence, nature and effects of disclosing conflicts of interest.^30,31 While disclosure policies are ubiquitous, those policies are not consistently designed, implemented, or enforced. Disclosure alone is insufficient. It is not particularly effective in mitigating the undesirable consequences of conflicts of interest.³⁰ Effective strategies are needed to assess whether an interest constitutes a conflict of interest, and to better manage conflicts of interest.³² There is also a need to identify and reduce incentives that contribute to distorting the conduct, reporting and interpretation of research.^33,34

Biased reporting of research

Many evaluations of the effects of interventions are not published,^35–37 and outcomes are sometimes selectively reported in research reports.^37–41 Those outcomes that are published are more likely to report favourable results. Consequently, relying only on published reports sometimes results in overestimating the wanted effects of interventions and underestimating unwanted effects.

The International Committee of Medical Journal Editors (ICMJE) adopted a policy in 2004 to mitigate bias resulting from selective reporting of research.⁴² The policy requires, as a condition of consideration for publication, registration of trials in a trial registry. In 2006, the World Health Organization established the International Clinical Trials Registry Platform to link trial registries to ensure a single point of access and unambiguous identification of trials, and to establish standards for trial registries.⁴³ These and other initiatives have helped accurately document non-publication, but biased reporting of evaluations of interventions remains a problem. Better understanding and evaluation of solutions is needed. This will require the collaboration of research funders,⁴⁴ industry and regulators,^45–47 trial registries, journal editors,⁴² research institutions,⁴⁸ systematic review authors^46,49–51 and researchers.⁵²

Deficient reporting of research

For people to be able to critically appraise research, the research must be clearly and completely reported. Recognition of inadequacies in the reporting of research dates back at least 50 years.⁵³ In 1996, two groups working independently to address this problem collaborated to create the first Consolidated Standards of Reporting Trials (CONSORT) statement in 1996. The statement is a checklist of items that should be included in reports of randomised trials, which has been updated three times since 1996.⁵⁴ The original Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement for reporting protocols for randomised trials was published in 2013.⁵⁵ Major updates of the CONSORT and SPIRIT guidelines were published in five major medical journals in 2025.^54,55

Other deficiencies in reporting research have been recognised, and many other checklists for reporting research have been developed subsequently. This includes checklists to improve the reporting of descriptions of interventions,^56,57 systematic reviews⁵⁸ and guidelines.⁵⁹ EQUATOR (the Enhancing the QUAlity and Transparency Of health Research network) was officially launched in 2008 as a global collaborative initiative. Its objective is to improve the value and reliability of published health research by promoting transparent and accurate reporting and wider use of evidence-based reporting guidelines.⁶⁰

The first formal Outcome Measures in Rheumatology (OMERACT) conference was convened in 1992 to address deficiencies in the measurement and reporting of outcomes in randomised trials of interventions for people with autoimmune and musculoskeletal diseases.^61,62 In 2010, the Core Outcome Measures in Effectiveness Trials (COMET) initiative was launched.^63,64 It promotes the development and use of core outcome sets to improve the measurement and reporting of outcomes for trials in all specific areas of health or health care.

Reporting guidelines and core outcome sets have been widely disseminated and endorsed, but there remain important inadequacies in reporting research.^65–67 There is a wide range of strategies that research funders, editors and peer reviewers, academic and research institutions could currently use to improve adherence to reporting guidelines, but little evidence of the effects of those strategies.^68–70

Deficient reviews

Reviews that fail to use systematic methods may yield biased or imprecise estimates of the effects of interventions. Unsystematic searches mean good studies may not be found, and poor selection or appraisal of identified studies may lead to bias. Finally, the synthesis of the results of the included studies may be inadequate or inappropriate.⁷¹

Hundreds of systematic reviews are published every week now.^5,8 There are about 9500 Cochrane reviews, and over 500,000 systematic reviews in the Epistemonikos database.⁷² However, most reviews are still not systematic, and many systematic reviews have serious flaws.^6,7 While there have been slow improvements, there is a need for ongoing collaboration among organisations such as the Cochrane Collaboration, the Campbell Collaboration, and the International Collaboration for the Automation of Systematic Reviews to improve the efficient and effective production of systematic reviews.

Automation tools have the potential to improve the speed and efficiency of systematic review production, their accuracy, and to keep them up to date.⁷³ However, caution is needed as evaluations show mixed results with the potential to decrease, as well as increase, accuracy and only for some review steps.⁷⁴ Those tools that have been shown to be helpful remain underused. Improved and automated review methods can also help improve the quality of reviews. But the methods used in systematic reviews have become increasingly complex – sometimes too complex.⁷⁵ To paraphrase a quote attributed to Albert Einstein: ‘The methods used in systematic reviews should be made as simple as possible, but not simpler.’

Ineffective and inefficient peer review

Journals rely on peer review to ensure the quality of the research they publish, and decisions about which research proposals are funded rely heavily on peer review.^76–78 However, peer review is highly variable, inconsistent and often flawed.^77,79–82 There is little evidence of the effects of peer review on the quality of published research evidence,^83,84 and very little evidence on the effects of peer review of proposals for research funding.^76,77 For the most part, it is done by volunteers, few of whom have formal training, and they commonly miss major errors. For example, the British Medical Journal (BMJ) sent three papers, each of which had nine major methodological errors inserted, to about 600 peer reviewers.^84,85 On average, the peer reviewers detected about one-third of the errors in each paper. In addition, peer reviewers fail to detect or comment on spin.⁸⁶

A systematic review published in 2007 found only 28 comparative studies of the effects of processes in editorial peer review.⁸³ The same year, another systematic review found only 10 comparative studies of the effects of grant-giving peer review processes.⁷⁶ A more recent review found 83 studies of innovations to improve the effectiveness and efficiency of peer review of health research funding. The studies had important limitations, but many innovations appear promising and warrant further evaluation.⁷⁸

Strategies that work are needed to improve the effectiveness and efficiency of peer review.

Failure to keep reviews up to date

A landmark study by Eliot Antman and colleagues, published in 1992, compared the results of cumulative meta-analyses of treatments for myocardial infarction with the recommendations of clinical experts writing (unsystematic) review articles and textbook chapters.⁸⁷ They showed that research had continued long after robust estimates of treatment effects had accumulated, and that recommendations had overlooked strong, existing evidence from randomised trials, both of beneficial and of lethal effects of treatments.

An analysis of 50 reports including over 1500 cumulative meta-analyses of health intervention studies was published 22 years later, in 2014.⁸⁸ This analysis showed ‘that, had researchers assessed systematically what was already known, some beneficial and harmful effects of treatments could have been identified earlier and might have prevented the conduct of the new trials. This would have led to the earlier uptake of effective health and social care interventions in practice, less exposure of trial participants to less effective treatments, and reduced waste resulting from unjustified research.’

‘Living systematic reviews’ are continually updated analyses, incorporating relevant new evidence as it becomes available.⁸⁹ Although this has been described as a novel approach to updating systematic reviews, it is like the approach that was used by The Oxford Database of Perinatal Trials (ODPT), first published electronically in 1988. ODPT became the Cochrane Pregnancy and Childbirth Database and was the pilot for the Cochrane Database of Systematic Reviews.^90–92 What’s more recent is identifying a subset of reviews for which this approach is appropriate and the use of automation to assist with some systematic review tasks. These include searching, eligibility assessment, identification and retrieval of full-text reports, extraction of data, and risk of bias assessment.⁹³

The evolution of ‘living guideline recommendations’ is linked to that of ‘living systematic reviews’. Studies have documented how quickly practice recommendations need updating.^94–96 Collaboration between ‘living systematic reviews’ and ‘living guideline’ teams is needed,⁹⁷ and digital tools can facilitate keeping practice recommendations up to date.^98–100

Unnecessary duplication of effort

Plans for randomised trials should use systematic reviews to avoid conducting new trials that are unnecessary (and thus probably unethical).^101,102 Cumulative meta-analyses have shown that new trials might have been recognised as unjustified had a systematic review informed plans for new trials.^87,88

Many published systematic reviews duplicate other reviews on the same topic without adding anything important.^7,103,104 In addition, some duplicate reviews are discordant and confusing.¹⁰⁵ Some intentional replication of systematic reviews by different teams might be useful (WHO often commissions duplicate reviews for very important topics). However, for many topics, duplication is a waste of resources.

Pressure on academics to publish contributes to unnecessary duplication. Many institutions base promotion and tenure on publication quantity, rather than quality. When income and professional advancement depend on publication output, systematic review authors may choose to publish a review, even if it duplicates another review unnecessarily.

To help avoid unintended and unnecessary duplication of systematic reviews, review authors should register protocols for their reviews, and they should search for other systematic reviews and review protocols on the same topic before undertaking a new review.¹⁰⁶

Misinformation

Mass media are a source of health information for many people. Researchers have studied and criticised the quality of health information in the mass media for at least five decades. A systematic review of 44 studies of the quality of health news found that many news reports gave an unbalanced and oversimplified picture of the potential consequences of health interventions.²²

Over the past three decades, people have been increasingly using the internet and social media to seek and share health information.^13,107 Several systematic reviews have found high prevalences of poor-quality online health information and misinformation.^13,107–110

The creation and dissemination of trustworthy information about the effects of health interventions can help to mitigate the adverse effects of misinformation.^9,10,13 Fact-checking and automated detection of health misinformation could also help. However, there is limited evidence of the effects of these and other interventions.^13,111–113 Artificial Intelligence (AI) could potentially help to reduce inequalities in access to evidence-based health information by facilitating equitable access to trustworthy information.^114,115 However, AI can also generate and worsen the spread of misinformation.

Eliminating misinformation is a worthy but Sisyphean task. In addition, teaching people to think critically about health claims and choices (including when and where to find trustworthy information) is essential.

Limited access to trustworthy information about the effects of policies and practices

To make informed decisions about interventions, patients and the public, health professionals and policymakers need information about effects based on the best available research evidence.⁹ Those communicating evidence-based information about the effects of health interventions should make it easy for readers to quickly assess the relevance of the information. For each important outcome, they should help their target audience to understand the size of the beneficial and harmful effects and how sure we can be about those, presented in ways that avoid misleading. They should help their target audience to put information about the effects of interventions in context and to understand why the information is trustworthy.

There is an abundance of health information on the internet, but it is hard to find trustworthy information that is explicitly based on systematic reviews.^10,116 Patients and the public are unlikely to critically appraise the information that they find, and most are unlikely to understand the key concepts people need to understand to assess claims about the effects of health care.¹¹⁷

Clinicians frequently have questions about the care of patients in their practice.¹¹⁸ Roughly half of the questions are never pursued. This picture has been stable over time despite the broad availability of online evidence resources that can answer these questions. It may be difficult for health professionals to find the best available evidence due to time constraints, lack of access to user-friendly, up-to-date, evidence-based resources, or lack of the skills needed to find and appraise up-to-date, evidence-based recommendations or systematic reviews.

Evidence-based textbooks that are kept up to date can help. However, the costs of these resources are passed along to the consumer, limiting access for many health professionals. In addition, a plethora of cheap, low-quality imitations may further limit access. The label ‘evidence-based’ is sometimes applied to resources that simply reference the medical literature (and may not even do that), regardless of how old or unsystematic they may be.

Gaps between evidence-based recommendations and professional practice

Underuse of effective interventions and overuse of interventions that are more likely to cause harm than good are common.^119,120 The size of gaps between evidence-based recommendations and clinical practice varies widely.^121–125 For example, a systematic review of inappropriate practice in Canada found a median proportion of inappropriate care of 30%. Underuse was more frequent (median 44%) than overuse (median 14%).¹²¹ A review of studies in the U.S. found that about 50% of people received recommended preventive care.¹²² None of the studies reported a percentage of people receiving contraindicated preventive care. An average of 70% of patients received recommended acute care, and 30% received contraindicated acute care. For chronic conditions, 60% received recommended care and 20% received contraindicated care. A review of studies of quality of care in general practice in the UK, Australia, and New Zealand found that in almost all the included studies, care did not attain the standards set out in national guidelines or by researchers.¹²³

Over the past three decades, increased attention has been given to the need to narrow the gap between research and practice,^126–128 but important gaps remain. Many initiatives to close gaps are narrowly focused and short term. Hence, ongoing collaborative efforts are needed to continually reduce those gaps. This includes addressing barriers to improvements and evaluating the effects of implementation interventions.

Inadequately informed policymaking

Substantial sums of money are invested each year in public programmes and policies. For most of these, little is known about their effects, including whether public programmes and policies can fulfil their primary objectives. Furthermore, what is known is often not used to inform policy decisions.¹²⁹ Because public resources are limited, it is important to use them effectively, efficiently and equitably.

When making decisions about public programmes, good intentions and plausible theories are not enough. Research evidence, values, political considerations, and judgements are all needed for well-informed decisions. However, decisions are often made without systematic or transparent access and appraisal of relevant research evidence and without an adequate evaluation of intended and unintended effects of policies or programmes.

Evidence-informed policymaking aims to ensure that decision-making is well-informed by the best available research evidence.¹³⁰ It is characterised by systematic and transparent access to, and appraisal of, evidence as an input into the policymaking process. While the overall process of policymaking may not be systematic and transparent, within that, systematic processes should be used to ensure that relevant research is identified, appraised and used appropriately. To ensure that others can examine what research evidence was used to inform policy decisions, as well as the judgements made about the evidence and its implications, the processes should be transparent.

Four key principles underlie fair processes: reporting, reasonableness, revision and regulation.¹³¹ In a fair process, the reasons for a decision are transparent and clearly reported. There are no secret deals or hidden motives. The reasons and evidence used to justify a decision must be reasonable. That is, they must adhere to agreed-upon principles. If someone feels a decision is mistaken or unfair, there should be a way to challenge it, and the process should allow for revision if better reasons or evidence come up. To ensure that the process is fair, it must be regulated, and the regulations must be enforced.

Increased attention has been given to the need to evaluate the impacts of health policies and for using systematic reviews to inform health policy.^132–137 However, few governments have in place fair processes that ensure that policy decisions are informed by systematic reviews of relevant research. And few governments have processes that ensure that the effects of implementing policies are evaluated when there are important uncertainties.

Lack of patient and public participation in decision-making

Funders’ decisions about research priorities for evaluating the effects of interventions should reflect the needs of the end users: patients, carers and clinicians. There are marked differences between the types of treatments prioritised for evaluation by patients, carers and clinicians and those currently being evaluated by researchers.¹³⁸ Inclusive participation of patients and clinicians in priority setting processes could help to address this.²⁹

Patients’ decisions about their care should allow involvement to the extent they want. A reason for this is that people vary greatly in the importance they attribute to outcomes,^139,140 and patients may value the pros and cons of an intervention differently than their physician.¹⁴¹ Most patients prefer sharing decisions with their doctors.¹⁴² Yet, few health care providers consistently attempt to facilitate patient involvement, and even fewer adjust care to patient preferences.¹⁴³ Decision aids can help, but few decision aids are currently used in clinical practice.¹⁴⁴

Involving patients and the public in the development of clinical guidance has increased in recent decades. But there is little research to inform decisions about how to do this.^145–147

The International Conference on Primary Health Care, meeting in Alma-Ata in 1978, declared that people have the right and duty to participate individually and collectively in the planning and implementation of their health care.¹⁴⁸ In addition to being a democratic right, inclusive participation in deliberative and decision-making processes has the potential to improve the quality of the judgements and decisions that are made, build trust, improve adherence and help to ensure transparency and accountability.¹⁴⁹

Public engagement may also help to ensure that important uncertainties are identified and addressed.¹⁵⁰ However, participation without clear objectives may anger participants and fail to benefit the policymaking process or outcomes. Poorly planned and implemented participation can create mistrust, waste people’s time, and undermine future attempts to engage the public.¹⁵¹

Recognition of the importance of patient and public participation in research, systematic reviews, clinical practice guidelines, clinical decision-making, and health policymaking has grown in recent decades. However, participation is limited in practice and policy, and there is little evidence of the effects of strategies to facilitate participation.^152–163

The way forward

Addressing the multifaceted challenges in medicine, public health, research and science increasingly demands the pooling of diverse expertise, resources and perspectives. There has been increasing recognition of the power of focused collaborative endeavours, such as the Human Genome Project, the Cochrane and Campbell Collaborations, collaborations fostered by the Oxford Clinical Trials Service Unit, and other collaborations referred to in this commentary.¹⁶⁴ An analysis of nearly 20 million research articles and over two million patents found a clear global trend towards team science across all scientific disciplines.¹⁶⁵ Teams now also produce the exceptionally high-impact research, even where that distinction was once the domain of solo authors. Since the foundation of the World Health Organization in 1948, the world has experienced public health challenges that have required international collaborations to improve health around the world.¹⁶⁶

Although there have been dramatic improvements in the design, implementation and use of health research evidence to inform policy and practice, there are still problems that need to be addressed. In Table 1, we suggest key challenges that need to be addressed by research funders, publishers, academic institutions, healthcare providers, governments and collaborations that aim to promote informed health choices. More specific recommendations that overlap with these suggestions have been made elsewhere.^{27,67,101,167–171}

Table 1.

Key challenges for research funders, publishers, academic institutions, healthcare providers, governments and collaborations that aim to promote informed health choices.

Who	Key challenges that need to be addressed
Research funders	• Strengthen inclusive participation in setting priorities and peer review of grant applications • Adopt and enforce evidence-based research policies¹⁰² • Support evaluations of the effects of interventions to address the problems considered in this commentary
Publishers	• Improve the effectiveness and efficiency of journal peer review • Strengthen the identification and management of conflicts of interest • Assess the effects of registered reports,¹⁷² post-publication peer review,¹⁷³ and other publishing models³³ to improve the quality of and access to research evidence
Academic institutions	• Change how researchers are evaluated from rewarding quantity and competition to rewarding quality, relevance, and collaboration^33,34 • Reclaim publication from commercial publishers that are making large profits by using unpaid researchers and charging high fees³³ • Design, evaluate, and implement effective strategies to foster critical thinking from primary school through to higher education and beyond
Healthcare providers	Take responsibility and be accountable for implementing learning health systems to^174–176 • Reduce gaps between evidence-based recommendations and professional practice • Ensure patient and public participation in clinical and health service decisions • Conduct research to reduce important uncertainties about the effects of clinical and implementation interventions
Governments	• Take responsibility and be accountable for addressing the same key challenges as other research funders and healthcare providers • Support non-commercial, open access publication and dissemination of trustworthy information about the effects of policies and practices • Adopt and implement fair processes with inclusive public participation to ensure that ○ Policy decisions are informed by systematic reviews of relevant research and ○ The effects of implementing policies are evaluated when there are important uncertainties
Collaborations	Researchers, editors, health professionals, policymakers, patients and the public need to collaborate within and across organisations and networks to • Persuade research funders, publishers, academic institutions, healthcare providers and governments to take responsibility and be accountable for addressing the key challenges noted above • Prepare, update and disseminate high-quality systematic reviews and guidelines, and reduce unnecessary duplication of effort • Identify, prioritise and reduce important uncertainties about how to address the problems discussed in this commentary

In Figure 2, we suggest a framework linking together the way forward and the challenges we have discussed in this commentary with using evaluations of the effects of interventions to inform health choices.

Figure 2.

Framework for promoting evaluation of the effects of health interventions to inform health choices.

Conclusion

We have come a long way over the past four decades, but we still have a long way to go. Ongoing collaborative efforts are needed to ensure that the effects of interventions are evaluated when there are important uncertainties about their effects. Efforts are also needed to ensure that systematic reviews of the effects of interventions are used to inform policy and practice. Questioning authority and thinking critically about the basis for opinions has been at the heart of the progress made, and it is essential for continued progress. This includes questioning the basis for the opinions we have expressed in this commentary.

There is joy as well as power in collaboration. To paraphrase what Dave Sackett wrote in the preface to the first edition of the Cochrane Collaboration Handbook,¹⁷⁷ the emphasis on collaboration is not simply a sentimental comment on the ‘generosity of spirit’ of those who become involved (although this spirit certainly makes collaboration a pleasure). The shared will to collaborate is a precondition for efficient improvement in the use of research to inform policy and practice and ensuring that interventions are likely to do more good than harm.

In summary, ‘good intentions and plausible theories are insufficient for selecting policies and practices for protecting, promoting and restoring health. Humility and uncertainty are preconditions for unbiased assessments of the effects of the prescriptions and proscriptions of policy makers and practitioners’. ‘The interests of the public will be served more responsibly and ethically when research designed to reduce the likelihood that we will be misled by bias and the play of chance has become an expected element of professional and policy making practice, not an optional add-on’.¹⁷⁸

Supplemental Material

sj-docx-1-jrs-10.1177_01410768251384317 – Supplemental material for Promoting informed health choices: the long and winding road

Supplemental material, sj-docx-1-jrs-10.1177_01410768251384317 for Promoting informed health choices: the long and winding road by Andrew D Oxman, Iain Chalmers and Paul P Glasziou in Journal of the Royal Society of Medicine

Supplemental Material

sj-docx-2-jrs-10.1177_01410768251384317 – Supplemental material for Promoting informed health choices: the long and winding road

Supplemental material, sj-docx-2-jrs-10.1177_01410768251384317 for Promoting informed health choices: the long and winding road by Andrew D Oxman, Iain Chalmers and Paul P Glasziou in Journal of the Royal Society of Medicine

Footnotes

Acknowledgements

We are grateful to the following people for their comments on an earlier (and much worse) draft of this commentary: Lisa Bero, Alan Cassels, Mike Clarke, Kay Dickersin, Jeremy Grimshaw, Gordon Guyatt, Sally Hopewell, France Legare, David Naylor, and Jimmy Volmink. None of them should be held responsible for whatever we got wrong or left out after taking account of their feedback. We are especially grateful to Jan Chalmers for her assistance and tolerance.

Declarations

References

Oxman

Chalmers

Glasziou

. Promoting informed health choices: the long and winding road. JLL Bulletin: Commentaries on the history of treatment evaluation. 2025. See https://www.jameslindlibrary.org/articles/promoting-informed-health-choices

Oransky

William Silverman. Lancet 2005; 365: 116. DOI: 10.1016/s0140-6736(05)17689-2.

Chalmers

Eckhardt

Reynolds

Cigarroa

Jr. Deane

Reifenstein

, et al. The treatment of acute infectious hepatitis. Controlled studies of the effects of diet, rest, and physical reconditioning on the acute course of the disease and on the incidence of relapses and residual abnormalities. J Clin Invest 1955; 34(Part II): 1163–235. DOI: 10.1172/jci103164.

Mulrow

CD.

The medical review article: state of the science. Ann Intern Med 1987; 106(3): 485–488. DOI: 10.7326/0003-4819-106-3-485.

Hoffmann

Allers

Rombey

Helbach

Hoffmann

Mathes

, et al. Nearly 80 systematic reviews were published each day: observational study on trends in epidemiology and reporting over the years 2000-2019. J Clin Epidemiol 2021; 138: 1–11. DOI: 10.1016/j.jclinepi.2021.05.022.

Page

Shamseer

Altman

Tetzlaff

Sampson

Tricco

, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med 2016; 13: e1002028. DOI: 10.1371/journal.pmed.1002028.

Ioannidis

JP.

The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 2016; 94: 485–514. DOI: 10.1111/1468-0009.12210.

Bastian

Glasziou

Chalmers

Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?

PLoS Med 2010; 7: e1000326. DOI: 10.1371/journal.pmed.1000326.

Oxman

Glenton

Flottorp

Lewin

Rosenbaum

Fretheim

Development of a checklist for people communicating evidence-based information about the effects of healthcare interventions: a mixed methods study. BMJ Open 2020; 10: e036348. DOI: 10.1136/bmjopen-2019-036348.

10.

Oxman

Paulsen

EJ.

Who can you trust? A review of free online sources of ‘trustworthy’ information about treatment effects for patients and the public. BMC Med Inform Decis Mak 2019; 19: 35. DOI: 10.1186/s12911-019-0772-5.

11.

Clarke

Chalmers

Alderson

Hopewell

Reports of randomised control trials should begin and conclude with up-to-date systematic reviews of other relevant trials: a 25-year audit of the quality of trial reports. J R Soc Med 2024; 117: 212–216. DOI: 10.1177/01410768241258090.

12.

Nord

Lundstedt

Altman

Angiolillo

Borella

Fernandes

, et al. Democracy Report 2024: Democracy Winning and Losing at the Ballot. University of Gothenburg: V-Dem Institute; 2024. See https://v-dem.net/documents/44/v-dem_dr2024_highres.pdf (last checked September 2025).

13.

Swire-Thompson

Lazer

Public health and online misinformation: challenges and recommendations. Annu Rev Public Health 2020; 41: 433–451. DOI: 10.1146/annurev-publhealth-040119-094127.

14.

Aronson

Barends

Boruch

Brennan

Chalmers

Chislett

, et al. Key concepts for making informed choices. Nature 2019; 572: 303–306. DOI: 10.1038/d41586-019-02407-9.

15.

Stratil

Biallas

Movsisyan

Oliver

Rehfuess

EA.

Development of an overarching framework for anticipating and assessing adverse and other unintended consequences of public health interventions (CONSEQUENT): a best-fit framework synthesis. BMJ Public Health 2024; 2: e000209. DOI: 10.1136/bmjph-2023-000209.

16.

Oxman

Chesire

Mugisha

Ssenyonga

Ngatia

Nsangi

, et al. Development of a framework of potential adverse effects of interventions to improve critical thinking about health choices: a mixed methods study. F1000Res 2024; 13: 1303. DOI: 10.12688/f1000research.158042.1.

17.

Zorzela

Loke

Ioannidis

Golder

Santaguida

Altman

, et al. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ 2016; 352: i157. DOI: 10.1136/bmj.i157

18.

Bonell

Jamal

Melendez-Torres

Cummins

‘Dark logic’: theorising the harmful consequences of public health interventions. J Epidemiol Community Health 2015; 69(1): 95–98. DOI: 10.1136/jech-2014-204671.

19.

Cuervo

Clarke

Balancing benefits and harms in health care. BMJ. 2003; 327: 65–66. DOI: 10.1136/bmj.327.7406.65

20.

Chou

Aronson

Atkins

Ismaila

Santaguida

Smith

, et al. AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. J Clin Epidemiol. 2010; 63: 502–512. DOI: 10.1016/j.jclinepi.2008.06.007.

21.

Yavchitz

Boutron

Bafeta

Marroun

Charles

Mantz

, et al. Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med 2012; 9: e1001308. DOI: 10.1371/journal.pmed.1001308.

22.

Oxman

Larun

Pérez Gaxiola

Alsaid

Qasim

Rose

, et al. Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses. F1000Res 2022; 10: 433. DOI: 10.12688/f1000research.52894.2.

23.

Hoffmann

Del Mar

Clinicians’ expectations of the benefits and harms of treatments, screening, and tests: a systematic review. JAMA Intern Med 2017; 177(3): 407–419. DOI: 10.1001/jamainternmed.2016.8254.

24.

Hoffmann

Del Mar

Patients’ expectations of the benefits and harms of treatments, screening, and tests: a systematic review. JAMA Intern Med 2015; 175(2): 274–286. DOI: 10.1001/jamainternmed.2014.6016.

25.

Rejas Bueno

Bacaicoa López de Sabando

Sánchez Robles

. [Health professionals expectations’ about the benefit of regular primary care interventions]. Aten Primaria 2022; 54: 102235. DOI: 10.1016/j.aprim.2021.102235.

26.

Dechartres

Trinquart

Faber

Ravaud

Empirical evaluation of which trial characteristics are associated with treatment effect estimates. J Clin Epidemiol 2016; 77: 24–37. DOI: 10.1016/j.jclinepi.2016.04.005.

27.

Ioannidis

Greenland

Hlatky

Khoury

Macleod

Moher

, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 2014; 383: 166–175. DOI: 10.1016/s0140-6736(13)62227-8.

28.

Leigh

Randomistas: How radical researchers are changing our world. New Haven: Yale University Press, 2018. See https://www.google.no/books/edition/_/gOJgDwAAQBAJ?hl=en&sa=X&ved=2ahUKEwico4vIqr6NAxXLIxAIHZNkCaQQ7_IDegQIGhAG (last checked September 2025).

29.

The James Lind Alliance. James Lind Alliance Priority Setting Partnerships. See https://www.jla.nihr.ac.uk/ (last checked 4 June 2025).

30.

Graham

Grundy

Sharma

Edward

Barbour

Rousseau

, et al. Research on policy mechanisms to address funding bias and conflicts of interest in biomedical research: a scoping review. Res Integr Peer Rev 2025; 10: 6. DOI: 10.1186/s41073-025-00164-0.

31.

Korfitsen

Nejstgaard

Hróbjartsson

Boutron

Bero

Lundh

Peer reviewers’ conflicts of interest in biomedical research: scoping review. BMJ Evid Based Med 2025; 30: 104–117. DOI: 10.1136/bmjebm-2024-112967.

32.

Xun

Estill

Khabsa

Florez

Guyatt

Norris

, et al. Reporting conflicts of interest and funding in health care guidelines: The RIGHT-COI&F checklist. Ann Intern Med 2024; 177: 782–790. DOI: 10.7326/m23-3274.

33.

Trueblood

Allison

Field

Fishbach

Gaillard

SDM

Gigerenzer

, et al. The misalignment of incentives in academic publishing and implications for journal reform. Proc Natl Acad Sci USA 2025; 122: e2401231121. DOI: 10.1073/pnas.2401231121.

34.

Allen

KA.

Move beyond ‘publish or perish’ by measuring behaviours that benefit academia. Nature 2025; 638: 861. DOI: 10.1038/d41586-025-00563-9.

35.

Chapman

Shelton

Mahmood

Fitzgerald

Harrison

Bhangu

Discontinuation and non-publication of surgical randomised controlled trials: observational study. BMJ 2014; 349: g6870. DOI: 10.1136/bmj.g6870.

36.

Hopewell

Loudon

Clarke

Oxman

Dickersin

. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 2009(1): MR000006. DOI: 10.1002/14651858.mr000006.pub3.

37.

Dwan

Gamble

Williamson

Kirkham

JJ.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias – an updated review. PLoS One 2013; 8: e66844. DOI: 10.1371/journal.pone.0066844.

38.

Fleming

Koletsi

Dwan

Pandis

Outcome discrepancies and selective reporting: impacting the leading journals?

PLoS One 2015; 10: e0127495. DOI: 10.1371/journal.pone.0127495.

39.

Page

McKenzie

Kirkham

Dwan

Kramer

Green

, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev 2014; 10: MR000035. DOI: /10.1002/14651858.MR000035.pub2.

40.

Jones

Keil

Holland

Caughey

Platts-Mills

TF.

Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med 2015; 13: 282. DOI: 10.1186/s12916-015-0520-3

41.

Riveros

Dechartres

Perrodeau

Haneef

Boutron

Ravaud

Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals. PLoS Med. 2013;10(12):e1001566. DOI: 10.1371/journal.pmed.1001566

42.

De Angelis

Drazen

Frizelle

Haug

Hoey

Horton

, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Ann Intern Med. 2004;141(6):477-8. DOI: 10.7326/0003-4819-141-6-200409210-00109.

43.

World Health Organization. International Standards for Clinical Trial Registries – Version 3.0. Geneva: World Health Organization; 2018. See https://www.who.int/publications/i/item/international-standards-for-clinical-trial-registers (last checked September 2025).

44.

Goldacre

The WHO joint statement from funders on trials transparency. BMJ 2017; 357: j2816. DOI: 10.1136/bmj.j2816.

45.

Goldacre

Lane

Mahtani

Heneghan

Onakpoya

Bushfield

, et al. Pharmaceutical companies’ policies on access to trial data, results, and methods: audit study. BMJ 2017; 358: j3334. DOI: 10.1136/bmj.j3334.

46.

Baudard

Yavchitz

Ravaud

Perrodeau

Boutron

Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses. BMJ 2017; 356: j448. DOI: 10.1136/bmj.j448.

47.

Manheimer

Anderson

Survey of public information about ongoing clinical trials funded by industry: evaluation of completeness and accessibility. BMJ 2002; 325: 528–531. DOI: 10.1136/bmj.325.7363.528.

48.

DeVito

Morley

Goldacre

Barriers and best practices to improving clinical trials transparency at UK public research institutions: a qualitative interview study. Health Policy 2024; 142: 104991. DOI: 10.1016/j.healthpol.2024.104991.

49.

Page

Sterne

JAC

Boutron

Hróbjartsson

Kirkham

, et al. ROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysis. BMJ 2023; 383: e076754. DOI: 10.1136/bmj-2023-076754.

50.

Guyatt

Wang

Eachempati

Iorio

Murad

Hultcrantz

, et al. Core GRADE 4: rating certainty of evidence-risk of bias, publication bias, and reasons for rating up certainty. BMJ 2025; 389: e083864. DOI: 10.1136/bmj-2024-083864.

51.

Kirkham

Altman

Chan

Gamble

Dwan

Williamson

PR.

Outcome reporting bias in trials: a methodological approach for assessment and adjustment in systematic reviews. BMJ 2018; 362: k3802. DOI: 10.1136/bmj.k3802.

52.

Hohlfeld

Kredo

Clarke

A scoping review of activities intended to reduce publication bias in randomised trials. Syst Rev. 2024;13(1):310. DOI: 10.1186/s13643-024-02728-5.

53.

Begg

Cho

Eastwood

Horton

Moher

Olkin

, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996; 276: 637–639. DOI: 10.1001/jama.276.8.637.

54.

Hopewell

Chan

A-W

Collins

Hróbjartsson

Moher

Schulz

, et al. CONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials. BMJ 2025; 389: e081124. DOI: 10.1136/bmj-2024-081124.

55.

Hróbjartsson

Boutron

Hopewell

Moher

Schulz

Collins

, et al. SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials. BMJ 2025; 389: e081660. DOI: 10.1136/bmj-2024-081660.

56.

Hoffmann

Glasziou

Boutron

Milne

Perera

Moher

, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687. DOI: 10.1136/bmj.g1687

57.

Hoffmann

Oxman

Ioannidis

Moher

Lasserson

Tovey

, et al. Enhancing the usability of systematic reviews by improving the consideration and description of interventions. BMJ 2017; 358: j2998. DOI: 10.1136/bmj.j2998.

58.

Page

Moher

Bossuyt

Boutron

Hoffmann

Mulrow

, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 2021; 372: n160. DOI: 10.1136/bmj.n160.

59.

Chen

Yang

Marušic

Qaseem

Meerpohl

Flottorp

, et al. A reporting tool for practice guidelines in health care: The RIGHT Statement. Ann Intern Med 2017; 166(2): 128–132. DOI: 10.7326/m16-1565.

60.

Catalá-López

Alonso-Arroyo

Page

Castelló-Cogollos

Hutton

Ridao

, et al. A cross-sectional analysis identified co-authorship networks and scientific collaboration on reporting guidelines for health research. J Clin Epidemiol 2023; 157: 22–34. DOI: 10.1016/j.jclinepi.2023.02.001.

61.

Boers

Kirwan

Wells

Beaton

Gossec

d’Agostino

, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol 2014; 67(7): 745–53. DOI: 10.1016/j.jclinepi.2013.11.013.

62.

de Wit

Kirwan

Tugwell

Beaton

Boers

Brooks

, et al. Successful stepwise development of patient research partnership: 14 years’ experience of actions and consequences in outcome measures in rheumatology (OMERACT). Patient 2017; 10(2): 141–152. DOI: 10.1007/s40271-016-0198-4.

63.

Kirkham

Davis

Altman

Blazeby

Clarke

Tunis

, et al. Core Outcome Set-STAndards for Development: The COS-STAD recommendations. PLoS Med 2017; 14: e1002447. DOI: 10.1371/journal.pmed.1002447.

64.

Kirkham

Gorst

Altman

Blazeby

Clarke

Tunis

, et al. Core Outcome Set-STAndardised Protocol Items: the COS-STAP Statement. Trials 2019; 20: 116. DOI: 10.1186/s13063-019-3230-x.

65.

Turner

Shamseer

Altman

Weeks

Peters

Kober

, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev 2012; 11: Mr000030. DOI: 10.1002/14651858.mr000030.pub2.

66.

Samaan

Mbuagbaw

Kosa

Borg Debono

Dillenburg

Zhang

, et al. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc 2013; 6: 169–188. DOI: 10.2147/jmdh.s43952.

67.

Glasziou

Altman

Bossuyt

Boutron

Clarke

Julious

, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014; 383: 267–76. DOI: 10.1016/s0140-6736(13)62228-x.

68.

Blanco

Altman

Moher

Boutron

Kirkham

Cobo

Scoping review on interventions to improve adherence to reporting guidelines in health research. BMJ Open 2019; 9: e026589. DOI: 10.1136/bmjopen-2018-026589.

69.

Blanco

Schroter

Aldcroft

Moher

Boutron

Kirkham

, et al. Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial. BMJ Open 2020; 10: e036799. DOI: 10.1136/bmjopen-2020-036799.

70.

Nasser

Clarke

Chalmers

Brurberg

Nykvist

Lund

, et al. What are funders doing to minimise waste in research? Lancet 2017; 389: 1006–1007. DOI: 10.1016/s0140-6736(17)30657-8.

71.

Whiting

Savović

Higgins

Caldwell

Reeves

Shea

, et al. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016; 69: 225–234. DOI: 10.1016/j.jclinepi.2015.06.005.

72.

Epistemonikos. Updated report – Strategy for systematic reviews 2023. See https://www.epistemonikos.org/en/about_us/updated_report].

73.

Clark

Glasziou

Del Mar

Bannach-Brown

Stehlik

Scott

AM.

A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol. 2020;121:81-90. DOI: 10.1016/j.jclinepi.2020.01.008

74.

Clark

Barton

Albarqouni

Byambasuren

Jowsey

Keogh

, et al. Generative artificial intelligence use in evidence synthesis: a systematic review. Res Synth Methods. 2025: 1–19. DOI: 10.1017/rsm.2025.16.

75.

Guyatt

Hultcrantz

Agoritsas

Iorio

Vandvik

Montori

VM.

Why Core GRADE is needed: introduction to a new series in The BMJ. BMJ 2025; 389: e081902. DOI: 10.1136/bmj-2024-081902.

76.

Demicheli

Di Pietrantonj

Peer review for improving the quality of grant applications. Cochrane Database Syst Rev 2007; 2007: Mr000003. DOI: 10.1002/14651858.mr000003.pub2

77.

Guthrie

Ghiga

Wooding

What do we know about grant peer review in the health sciences?

F1000Res. 2018;6:1335. DOI: 10.12688/f1000research.11917.2.

78.

Shepherd

Frampton

Pickett

Wyatt

JC.

Peer review of health research funding proposals: A systematic map and systematic review of innovations for effectiveness and efficiency. PLoS One 2018; 13: e0196914. DOI: 10.1371/journal.pone.0196914.

79.

Smith

Peer review: a flawed process at the heart of science and journals. J R Soc Med 2006; 99: 178–182. DOI: 10.1258/jrsm.99.4.178.

80.

Smith

Classical peer review: an empty gun. Breast Cancer Res 2010; 12 (Suppl. 4): S13. DOI: 10.1186/bcr2742.

81.

Tamblyn

Girard

Qian

Hanley

Assessment of potential bias in research grant peer review in Canada. CMAJ. 2018; 190: E489–E99. DOI: 10.1503/cmaj.170901

82.

Abdoul

Perrey

Tubach

Amiel

Durand-Zaleski

Alberti

Non-financial conflicts of interest in academic grant evaluation: a qualitative study of multiple stakeholders in France. PLoS One 2012; 7: e35247. DOI: 10.1371/journal.pone.0035247.

83.

Jefferson

Rudin

Brodney Folse

Davidoff

Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database Syst Rev 2007; 2: MR000016. DOI: /10.1002/14651858.MR000016.pub3.

84.

Bruce

Chauvin

Trinquart

Ravaud

Boutron

Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis. BMC Med 2016; 14: 85. DOI: 10.1186/s12916-016-0631-5.

85.

Schroter

Black

Evans

Godlee

Osorio

Smith

What errors do peer reviewers detect, and does training improve their ability to detect them?

J R Soc Med 2008; 101: 507–514. DOI: 10.1258/jrsm.2008.080062.

86.

Lazarus

Haneef

Ravaud

Boutron

Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention. BMC Med Res Methodol 2015; 15: 85. DOI: 10.1186/s12874-015-0079-x.

87.

Antman

Lau

Kupelnick

Mosteller

Chalmers

TC.

A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 1992; 268: 240–248. DOI: 10.1001/jama.1992.03490020088036.

88.

Clarke

Brice

Chalmers

Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources. PLoS One 2014; 9(7): e102670. DOI: 10.1371/journal.pone.0102670.

89.

Elliott

Synnot

Turner

Simmonds

Akl

McDonald

, et al. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol 2017; 91: 23–30. DOI: 10.1016/j.jclinepi.2017.08.010.

90.

Chalmers

Hetherington

Newdick

Mutch

Grant

Enkin

, et al. The Oxford Database of Perinatal Trials: developing a register of published reports of controlled trials. Control Clin Trials 1986; 7: 306–324. DOI: 10.1016/0197-2456(86)90038-3.

91.

Chalmers

The Cochrane collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. Ann N Y Acad Sci 1993; 703: 156-63; discussion 63-5. DOI: 10.1111/j.1749-6632.1993.tb26345.x.

92.

Starr

Chalmers

Clarke

Oxman

AD.

The origins, evolution, and future of The Cochrane Database of Systematic Reviews. Int J Technol Assess Health Care 2009; 25(Suppl. 1): 182–195. DOI: 10.1017/s026646230909062x.

93.

Thomas

Noel-Storr

Marshall

Wallace

McDonald

Mavergames

, et al. Living systematic reviews: 2. Combining human and machine effort. J Clin Epidemiol 2017; 91: 31–37. DOI: 10.1016/j.jclinepi.2017.08.011.

94.

Shekelle

Ortiz

Rhodes

Morton

Eccles

Grimshaw

, et al. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA 2001; 286: 1461–1467. DOI: 10.1001/jama.286.12.1461.

95.

Neuman

Goldstein

Cirullo

Schwartz

JS.

Durability of class I American College of Cardiology/American Heart Association clinical practice guideline recommendations. JAMA 2014; 311: 2092–2100. DOI: 10.1001/jama.2014.4949.

96.

Martínez García

Sanabria

García Alvarez

Trujillo-Martín

Etxeandia-Ikobaltzeta

Kotzeva

, et al. The validity of recommendations from clinical guidelines: a survival analysis. Cmaj 2014; 186: 1211–1219. DOI: 10.1503/cmaj.140547.

97.

Akl

Meerpohl

Elliott

Kahale

Schünemann

HJ.

Living systematic reviews: 4. Living guideline recommendations. J Clin Epidemiol 2017; 91: 47–53. DOI: 10.1016/j.jclinepi.2017.08.009.

98.

Martínez García

Pardo-Hernández

Sanabria

Alonso-Coello

Continuous surveillance of a pregnancy clinical guideline: an early experience. Syst Rev 2017; 6: 143. DOI: 10.1186/s13643-017-0506-7.

99.

Vandvik

Brandt

Alonso-Coello

Treweek

Akl

Kristiansen

, et al. Creating clinical practice guidelines we can trust, use, and share: a new era is imminent. Chest 2013; 144: 381–389. DOI: 10.1378/chest.13-0746.

100.

Tendal

Vogel

McDonald

Norris

Cumpston

White

, et al. Weekly updates of national living evidence-based guidelines: methods for the Australian living guidelines for care of people with COVID-19. J Clin Epidemiol 2021; 131: 11–21. DOI: 10.1016/j.jclinepi.2020.11.005.

101.

Chalmers

Bracken

Djulbegovic

Garattini

Grant

Gülmezoglu

, et al. How to increase value and reduce waste when research priorities are set. Lancet 2014; 383: 156–165. DOI: 10.1016/s0140-6736(13)62229-1.

102.

Lund

Brunnhuber

Juhl

Robinson

Leenaars

Dorch

, et al. Towards evidence based research. BMJ 2016; 355: i5440. DOI: 10.1136/bmj.i5440.

103.

Siontis

Hernandez-Boussard

Ioannidis

JP.

Overlapping meta-analyses on the same topic: survey of published studies. BMJ 2013; 347: f4501. DOI: 10.1136/bmj.f4501.

104.

Naudet

Schuit

Ioannidis

JPA

. Overlapping network meta-analyses on the same topic: survey of published studies. Int J Epidemiol 2017; 46: 1999–2008. DOI: 10.1093/ije/dyx138.

105.

Jadad

Cook

Browman

GP.

A guide to interpreting discordant systematic reviews. CMAJ 1997; 156: 1411–1416. See https://pmc.ncbi.nlm.nih.gov/articles/PMC1227410/ (last checked September 2025).

106.

Stewart

Moher

Shekelle

Why prospective registration of systematic reviews makes sense. Syst Rev 2012; 1: 7. DOI: 10.1186/2046-4053-1-7.

107.

Suarez-Lledo

Alvarez-Galvez

Prevalence of health misinformation on social media: systematic review. J Med Internet Res 2021; 23: e17187. DOI: 10.2196/17187.

108.

Eysenbach

Powell

Kuss

ER.

Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA 2002; 287: 2691–700. DOI: 10.1001/jama.287.20.2691.

109.

Roundtree

Kallen

Lopez-Olivo

Kimmel

Skidmore

Ortiz

, et al. Poor reporting of search strategy and conflict of interest in over 250 narrative and systematic reviews of two biologic agents in arthritis: a systematic review. J Clin Epidemiol 2009; 62: 128–137. DOI: 10.1016/j.jclinepi.2008.08.003.

110.

Borges

Nascimento

Pizarro

Almeida

Azzopardi-Muscat

Gonçalves

Björklund

, et al. Infodemics and health misinformation: a systematic review of reviews. Bull World Health Organ 2022; 100: 544–561. DOI: 10.2471/blt.21.287654.

111.

Smith

Chen

Winner

Friedhoff

Wardle

A systematic review of Covid-19 misinformation interventions: Lessons learned. Health Aff 2023; 42: 1738–1746. DOI: 10.1377/hlthaff.2023.00717.

112.

Zhang

Zhou

Zhu

Have we found a solution for health misinformation? A ten-year systematic review of health misinformation literature 2013-2022. Int J Med Inform 2024; 188: 105478. DOI: 10.1016/j.ijmedinf.2024.105478

113.

Schlicht

Fernandez

Chulvi

Rosso

Automatic detection of health misinformation: a systematic review. J Ambient Intell Humaniz Comput 2023: 1–13. DOI: 10.1007/s12652-023-04619-4.

114.

Meyrowitsch

Jensen

Sørensen

Varga

TV.

AI chatbots and (mis)information in public health: impact on vulnerable communities. Front Public Health 2023; 11: 1226776. DOI: 10.3389/fpubh.2023.1226776.

115.

Beam

Darmono

Misinformation doesn’t have to get the last word: AI can support the truth, too. Harvard Public Health 2024. See https://harvardpublichealth.org/tech-innovation/how-ai-misinformation-could-impact-the-future-of-public-health/ (last checked September 2025).

116.

Glenton

Paulsen

Oxman

AD.

Portals to Wonderland: health portals lead to confusing information about the effects of health care. BMC Med Inform Decis Mak 2005; 5: 7. DOI: 10.1186/1472-6947-5-7.

117.

Dahlgren

Furuseth-Olsen

Rose

Oxman

AD.

The Norwegian public’s ability to assess treatment claims: results of a cross-sectional study of critical health literacy. F1000Research 2021; 9: 179. DOI: 10.12688/f1000research.21902.2.

118.

Del Fiol

Workman

Gorman

. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA Intern Med 2014; 174: 710–718. DOI: 10.1001/jamainternmed.2014.368.

119.

Brownlee

Chalkidou

Doust

Elshaug

Glasziou

Heath

, et al. Evidence for overuse of medical services around the world. Lancet 2017; 390: 156–168. DOI: 10.1016/S0140-6736(16)32585-5.

120.

Glasziou

Straus

Brownlee

Trevena

Dans

Guyatt

, et al. Evidence for underuse of effective medical services around the world. Lancet 2017; 390: 169–177. DOI: 10.1016/s0140-6736(16)30946-1.

121.

Squires

Cho-Young

Aloisio

Bell

Bornstein

Brien

, et al. Inappropriate use of clinical practices in Canada: a systematic review. CMAJ 2022; 194: E279–E96. DOI: 10.1503/cmaj.211416.

122.

Schuster

McGlynn

Brook

RH.

How good is the quality of health care in the United States?

Milbank Q 2005; 83: 843–895. DOI: 10.1111/j.1468-0009.2005.00403.x.

123.

Seddon

Marshall

Campbell

Roland

MO.

Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand. Qual Health Care 2001; 10: 152–158. DOI: 10.1136/qhc.0100152.

124.

McGlynn

Asch

Adams

Keesey

Hicks

DeCristofaro

, et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003; 348: 2635–2645. DOI: 10.1056/nejmsa022615.

125.

Grol

Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care 2001; 39(Suppl. 2): Ii46-54. DOI: 10.1097/00005650-200108002-00003.

126.

Balas

Boren

SA.

Managing Clinical Knowledge for Health Care Improvement. Yearb Med Inform 2000: 65–70. See https://www.thieme-connect.com/products/all/doi/10.1055/s-0038-1637943 (last checked September 2025).

127.

Morris

Wooding

Grant

The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 2011; 104: 510–520. DOI: 10.1258/jrsm.2011.110180.

128.

Djulbegovic

Guyatt

GH.

Progress in evidence-based medicine: a quarter century on. Lancet 2017; 390: 415–423. DOI: 10.1016/s0140-6736(16)31592-6.

129.

Kebebe

Oxman

Glenton

Blueprints for informed policy decisions: a review of laws and policies requiring routine evaluation. Oslo: Norwegian Knowledge Centre for the Health Services, 2012. See https://fhi.brage.unit.no/fhi-xmlui/bitstream/handle/11250/2477255/K_rapport_2012_16.pdf?sequence=1 (last checked September 2025).

130.

Oxman

Lavis

Lewin

Fretheim

SUPPORT Tools for evidence-informed health Policymaking (STP) 1: What is evidence-informed policymaking?

Health Res Policy Syst 2009; 7(Suppl. 1): S1. DOI: 10.1186/1478-4505-7-s1-s1.

131.

Daniels

Sabin

Limits to health care: fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philos Public Aff 1997; 26: 303–350. DOI: 10.1111/j.1088-4963.1997.tb00082.x.

132.

World Health Organization. Ministerial Summit on Health Research. In Fifty-eighth World Health Assembly: Resolutions and Decisions. Geneva: WHO, 2005:126–128. See https://apps.who.int/gb/ebwha/pdf_files/wha58-rec1/english/a58_2005_rec1-en.pdf

133.

Hanney

González-Block

MA.

Evidence-informed health policy: are we beginning to get there at last?

Health Res Policy Syst 2009; 7: 30. DOI: 10.1186/1478-4505-7-30.

134.

World Health Organization. Evidence, policy, impact. WHO guide for evidence-informed decision-making. Geneva: World Health Organization, 2021. https://www.who.int/publications/i/item/9789240039872

135.

World Health Organization. Evidence as a catalyst for policy and societal change: towards more equitable, resilient and sustainable global health. Meeting report of the WHO Global Evidence-to-Policy Summit, 15–17 November 2021. Geneva: World Health Organization; 2022. https://www.who.int/publications-detail-redirect/9789240052130

136.

World Health Organization. Evidence. Knowledge Translation. Impact: Closing evidence-to-policy gaps. Geneva: World Health Organization, 2022. See https://cdn.who.int/media/docs/default-source/evidence-to-policy-and-impact/evipnet–brochure-contents—27a—web.pdf? (last checked September 2025).

137.

Beane

Gingrich

Kerry

. How to take American health care from worst to first. New York Times, 24 October 2008. See https://www.nytimes.com/2008/10/24/opinion/24beane.html.

138.

Crowe

Fenton

Hall

Cowan

Chalmers

Patients’, clinicians’ and the research communities’ priorities for treatment research: there is an important mismatch. Res Involv Engagem 2015; 1: 2. DOI: 10.1186/s40900-015-0003-x.

139.

MacLean

Mulla

Akl

Jankowski

Vandvik

Ebrahim

, et al. Patient values and preferences in decision making for antithrombotic therapy: a systematic review: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012; 141(2 Suppl.): e1S–e23S. DOI: 10.1378/chest.11-2290.

140.

Oxman

Chalmers

Dahlgren

, Informed Health Choices Group. Consider how important each advantage and disadvantage is when weighing the pros and cons and making choices. In: Key Concepts for Informed Health Choices: a framework for enabling people to think critically about health claims (Version 2022). See https://www.informedhealthchoices.org/wp-content/uploads/2024/04/3.2c-Consider-how-important-each-advantage-and-disadvantage-is-when-weighing-the-pros-and-cons-and-making-choices.pdf (last checked September 2025).

141.

Alonso-Coello

Montori

Díaz

Devereaux

Mas

Diez

, et al. Values and preferences for oral antithrombotic therapy in patients with atrial fibrillation: physician and patient perspectives. Health Expect 2015; 18: 2318–2327. DOI: 10.1111/hex.12201.

142.

Chewning

Bylund

Shah

Arora

Gueguen

Makoul

Patient preferences for shared decisions: a systematic review. Patient Educ Couns 2012; 86: 9–18. DOI: 10.1016/j.pec.2011.02.004.

143.

Couët

Desroches

Robitaille

Vaillancourt

Leblanc

Turcotte

, et al. Assessments of the extent to which health-care providers involve patients in decision making: a systematic review of studies using the OPTION instrument. Health Expect 2015; 18: 542–561. DOI: 10.1111/hex.12054.

144.

Stacey

Lewis

Smith

Carley

Volk

Douglas

, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2024; 1: Cd001431. DOI: 10.1002/14651858.cd001431.pub6.

145.

Schunemann

Fretheim

Oxman

AD.

Improving the use of research evidence in guideline development: 10. Integrating values and consumer involvement. Health Res Policy Syst 2006; 4: 22. DOI: 10.1186/1478-4505-4-22.

146.

Rashid

Thomas

Shaw

Leng

Patient and public involvement in the development of healthcare guidance: an overview of current methods and future challenges. Patient 2017; 10: 277–282. DOI: 10.1007/s40271-016-0206-8

147.

Bryant

Scott

Greenwood

Thomas

Patient and public involvement in the development of clinical practice guidelines: a scoping review. BMJ Open 2022; 12(9): e055428. DOI: 10.1136/bmjopen-2021-055428.

148.

World Health Organization. Declaration of Alma-Ata. Geneva: World Health Organization, 1978. See https://apps.who.int/iris/handle/10665/347879 (last checked September 2025).

149.

Norheim

Abi-Rached

Bright

Bærøe

Ferraz

OLM

Gloppen

, et al. Difficult trade-offs in response to COVID-19: the case for open and inclusive decision making. Nat Med 2021; 27: 10–13. DOI: 10.1038/s41591-020-01204-6.

150.

Tan

Nagraj

Nasser

Sharma

Kuchenmüller

What do we know about evidence-informed priority setting processes to set population-level health-research agendas: an overview of reviews. Bull Natl Res Cent 2022; 46: 6. DOI: 10.1186/s42269-021-00687-8.

151.

Involve. People & Participation: How to put citizens at the heart of decision-making. London: Involve, 2005. https://www.involve.org.uk/resources/publications/practical-guidance/people-and-participation-how-put-citizens-heart-decision (last checked September 2025).

152.

Brett

Staniszewska

Mockford

Herron-Marx

Hughes

Tysall

, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect 2014; 17: 637–650. DOI: 10.1111/j.1369-7625.2012.00795.x.

153.

Concannon

Fuster

Saunders

Patel

Wong

Leslie

, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med 2014; 29: 1692–1701. DOI: 10.1007/s11606-014-2878-x.

154.

Domecq

Prutsky

Elraiyah

Wang

Nabhan

Shippee

, et al. Patient engagement in research: a systematic review. BMC Health Serv Res 2014; 14: 89. DOI: 10.1186/1472-6963-14-89.

155.

Pollock

Campbell

Struthers

Synnot

Nunn

Hill

, et al. Stakeholder involvement in systematic reviews: a scoping review. Syst Rev 2018; 7: 208. DOI: 10.1186/s13643-018-0852-0.

156.

Nilsen

Myrhaug

Johansen

Oliver

Oxman

AD.

Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev 2006; 3: CD004563. DOI: 10.1002/14651858.cd004563.pub2.

157.

Wiles

Kay

Luker

Worley

Austin

Ball

, et al. Consumer engagement in health care policy, research and services: a systematic review and meta-analysis of methods and effects. PLoS One 2022; 17(1): e0261808. DOI: 10.1371/journal.pone.0261808

158.

Abelson

Forest

P-G

Eyles

Smith

Martin

Gauvin

F-P.

Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Soc Sci Med 2003; 57: 239–251. DOI: 10.1016/S0277-9536(02)00343-X.

159.

Conklin

Morris

Nolte

What is the evidence base for public involvement in health-care policy?: results of a systematic scoping review. Health Expect 2015; 18(2): 153–165. DOI: 10.1111/hex.12038.

160.

Lloyd

Kenny

Hyett

Evaluating health service outcomes of public involvement in health service design in high-income countries: a systematic review. BMC Health Serv Res 2021; 21: 364. DOI: 10.1186/s12913-021-06319-1.

161.

Baumann

Reinhold

Brütt

AL.

Public and patient involvement in health policy decision-making on the health system level – a scoping review. Health Policy 2022; 126: 1023–1038. DOI: 10.1016/j.healthpol.2022.07.007.

162.

Aguilera

Donya

Vélez

Kapiriri

Abelson

Nouvet

, et al. Stakeholder participation in the COVID-19 pandemic preparedness and response plans: a synthesis of findings from 70 countries. Health Policy 2024; 142: 105013. DOI: 10.1016/j.healthpol.2024.105013.

163.

Munthe-Kaas

Oxman

von Lieres

Gloppen

Ohren

Public participation in decisions about measures to manage the COVID-19 pandemic: a systematic review. BMJ Glob Health 2024; 9: e014404. DOI: 10.1136/bmjgh-2023-014404

164.

Wang

Barabási

A-L.

The Science of Collaboration. In: The Science of Science. Cambridge: Cambridge University Press, 2021: 81–158. DOI: 10.1017/9781108610834.010.

165.

Wuchty

Jones

Uzzi

The increasing dominance of teams in production of knowledge. Science 2007; 316(5827): 1036–1039. DOI: 10.1126/science.1136099.

166.

World Health Organization. Public health milestones through the years 2023. See https://www.who.Int/campaigns/75-years-of-improving-public-health/milestones#year-1945 (last checked 6 June 2025).

167.

Chalmers

Glasziou

Avoidable waste in the production and reporting of research evidence. Lancet 2009; 374(9683): 86–89. DOI: 10.1016/s0140-6736(09)60329-9.

168.

Al-Shahi Salman

Beller

Kagan

Hemminki

Phillips

Savulescu

, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet 2014; 383(9912): 176–185. DOI: 10.1016/s0140-6736(13)62297-7.

169.

Chan

Song

Vickers

Jefferson

Dickersin

Gøtzsche

, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet 2014; 383(9913): 257–266. DOI: 10.1016/s0140-6736(13)62296-5.

170.

Elshaug

Rosenthal

Lavis

Brownlee

Schmidt

Nagpal

, et al. Levers for addressing medical underuse and overuse: achieving high-value health care. Lancet 2017; 390(10090): 191–202. DOI: 10.1016/s0140-6736(16)32586-7.

171.

Bradley

DeVito

Lloyd

Richards

Rombey

Wayant

, et al. Reducing bias and improving transparency in medical research: a critical overview of the problems, progress and suggested next steps. J R Soc Med 2020; 113(11): 433–443. DOI: 10.1177/0141076820956799

172.

Center for Open Science. Registered Reports: Peer review before results are known to align scientific values and practices. See https://www.cos.io/initiatives/registered-reports (last checked 6 June 2025).

173.

Davidson

Korfitsen

Riveros

Chaimani

Boutron

Post-publication peer review and the identification of methodological and reporting issues in COVID-19 trials: a qualitative study. BMJ Evid Based Med 2025; 30: 233–240. DOI: 10.1136/bmjebm-2024-113068.

174.

BOLDER Research Group. Better Outcomes through Learning, Data, Engagement, and Research (BOLDER) – a system for improving evidence and clinical practice in low and middle income countries. F1000Res 2016; 5: 693. DOI: 10.12688/f1000research.8392.1.

175.

English

Irimu

Agweyu

Gathara

Oliwa

Ayieko

, et al. Building learning health systems to accelerate research and improve outcomes of clinical care in low- and middle-income countries. PLoS Med 2016; 13: e1001991. DOI: 10.1371/journal.pmed.1001991.

176.

McDonald

Foley

Verheij

Braithwaite

Rubin

Harwood

, et al. Data to knowledge to improvement: creating the learning health system. BMJ 2024; 384: e076175. DOI: 10.1136/bmj-2023-076175.

177.

Sackett

Oxman

, eds. Cochrane collaboration handbook. 1st ed. Oxford: The Cochrane Collaboration, 1994.

178.

Chalmers

Trying to do more good than harm in policy and practice: the role of rigorous, transparent, up-to-date evaluations. Ann Am Acad Political Soc Sci 2003; 589(1): 22–40. DOI: 10.1177/0002716203254762.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB

0.04 MB