History of evidence-based research

What is evidence-based research?

In the 2000s, the need to explicitly consider existing research when justifying, planning and reporting new studies, as illustrated in the examples above, was formally named. The term evidence-based research was independently coined first by Karen Robinson in her PhD dissertation in 2009 and then by Hans Lund in an application for a PhD programme at Høgskolen i Bergen in 2013. Robinson described evidence-based research as ‘one way to reduce waste in the production and reporting of trials, through the initiation of trials that are needed to address outstanding questions and through the design of new trials in a way that maximizes the information gained. Investigators need to identify and consider prior studies to provide the ethical and scientific justification for why they started a clinical trial, and to determine the most appropriate design and methodological characteristics of that trial. The synthesis of prior studies is also needed to provide an informative discussion of the results of a trial by placing its results within the context of existing evidence’ (p. 123). ⁶

Later, Lund wrote: ‘Current evidence shows that several randomized controlled trials have been unnecessarily performed because a clear answer to the research question had already been established.^3,5,7 Just as evidence-based practice has identified gaps between research findings and clinical practice, the studies referred to above indicate a worrying gap between the recommended and the actual practice among researchers’. ⁸

The Evidence-Based Research Network (EBR Network, established in 2014, see below) adopted the definition of evidence-based research as ‘the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner’. ⁹

History of evidence-based research

The recognition of the need for research to identify, acknowledge and build upon earlier work is not new. David Wootton, in The Invention of Science, provides an example of Gilbert who published a book about magnets in 1600. In the first chapter, Gilbert presented a summary of all earlier books in order to convince the reader that he had made new discoveries about magnetism. ¹⁰ This thinking about science as cumulative is also emphasised by Sir Isaac Newton’s statement in a letter to Robert Hooke in 1676 that ‘If I have seen farther it is by standing on the shoulders of giants’ and by James Lind who presented all earlier attempts to heal scurvy in the first chapter of his book about his study of citrus fruits. ¹

Later, in the first issue of the New England Journal of Medicine in 1812, Warren wrote ‘[i]n our inquiries into any particular subject of medicine, our labours will generally be shortened and directed to their proper objects, by a knowledge of preceding discoveries’. ¹¹ Further, at the 54th meeting of the British Association for the Advancement of Science in Montreal in 1884, Lord Rayleigh stated: ‘If, as is sometimes supposed, science consisted in nothing but the laborious accumulation of facts, it would soon come to a standstill, crushed, as it were, under its own weight. . . The work which deserves, but I am afraid, does not always receive, the most credit is that in which discovery and explanation go hand in hand, in which not only are new facts presented, but their relation to old ones is pointed out’ (cited from Chalmers et al. ¹² ).

The interesting aspect here is that another key feature of science – discovery – also depends on the use of prior work. These two apparently conflicting features – science is cumulative, and science is discovery – are challenged in Lord Rayleigh’s comment and in Light and Pillemer’s introduction Summing Up – the science of reviewing research: ‘Why do scientists think that new research is better, or more insightful, or more powerful? The underlying assumption must be that new studies will incorporate and improve upon lessons learned from earlier work. Novelty in and of itself is shallow without links to the past. It is possible to evaluate an innovation only by comparison with its predecessors. For science to be cumulative, an intermediate step between past and future research is necessary: synthesis of existing evidence’. ¹³

To convey the existing body of evidence, researchers must cite relevant prior studies. Yet the selection of citations or references in a paper reporting a study is more often strategic, and preference-based, not systematic and transparent.^14,15 In 1965, Austin Bradford Hill laid out a structure for scientific articles that included ‘why did you start?’ and ‘what does it mean anyway?’ To provide scientific and ethical justification for new trials, researchers need to convey the current state of knowledge. In addition, to facilitate interpretation, researchers should indicate the contribution of their trial results to the existing evidence base. As Bradford Hill stated, ‘All scientific work is incomplete . . . [t]hat does not confer upon us a freedom to ignore the knowledge that we already have. . . . ¹⁶

Eugene Garfield, a pioneer in bibliometrics and founder of the company which developed the Science Citation Index, talked about bibliographic negligence in 1991: ‘For a long time, scientists and others have expressed the need for a “science court” – a panel that would, among other things, sit in judgment concerning matters of fraud, misconduct, and other transgressions by researchers. If such a court is ever established, I hope that cases of bibliographic negligence are among the issues that come under consideration – and I hope that proven cases of such negligence will be dealt with firmly. As important as the need for meting out punishment to willful perpetrators in this regard, however, is the need to instruct young researchers, preventively, on the ethics and etiquette involved in proper and complete referencing. Acknowledging prior research and intellectual debts is of crucial ethical importance’. ¹⁷ Around the same time, Thomas Chalmers and colleagues noted that failure to appropriately consider previous studies may limit the design and conduct of studies; a cause of bias they called ‘sloth’. ¹⁸

The development of EBR is similar to the development of evidence-based medicine. The ‘evidence-based medicine’ approach, developed in the 1980s and coined as a term in the early 1990s, promoted a new way to make clinical decisions, where a systematic search and synthesis of existing research is included in informing clinical decisions. In 1994, Cynthia D Mulrow, then Director of the San Antonio Cochrane Collaboration Center in USA and editor at Annals of Internal Medicine, wrote a paper called ‘Rationale for systematic reviews’. In her call for evidence-based medicine, Mulrow also expressed EBR concepts: ‘Researchers use the review to identify, justify, and refine hypotheses; recognise and avoid pitfalls of previous work; estimate sample sizes; and delineate important ancillary or adverse effects and covariates that warrant consideration in future studies’. ¹⁹

Around this time, the first version of CONSORT (Consolidated Standards of Reporting Trials), published in 1996, included guidance that reports of randomised trials ‘state general interpretation of the data in light of the totality of the available evidence’. ²⁰ Later versions of CONSORT included recommendations to use a systematic review of earlier, similar studies to justify the new study and to interpret its results. ²¹ The 2025 version retained the recommendation to provide, ideally, a systematic review of prior relevant research. ²² The explanation cites work by members of the EBR Network, which found that despite an increase in the proportion of reports of trials citing an available systematic review, over a quarter still failed to do so. ²³

In 1996, inspired by a new book Systematic Reviews, Julian Savulescu and colleagues wrote an article in BMJ called ‘Are research ethics committees behaving unethically? Some suggestions for improving performance and accountability’. ²⁴ They suggested that ‘[t]he performance and accountability of research ethics committees would be improved if they required those proposing research to present systematic reviews of relevant previous research in support of their applications’. ²⁴

Thus, the 1990s saw many calls to take advantage of the systematic and transparent syntheses of earlier studies to inform both clinical decisions and the justification and interpretation of a new study. However, audits on the five high-impact journals Annals of Internal Medicine, BMJ, JAMA, The Lancet and the New England Journal of Medicine from 1997 to 2022 found that more was needed to change the way new research was justified and the way new results were reported and interpreted. Over those 25 years, the authors performed a series of audits of randomised trials published in the month of May in 1997, 2001, 2005, 2009, 2012 and 2022. ²⁵ For each published report of a randomised trial, they read the introduction and discussion sections. For the introduction, they assessed whether a systematic review or references to prior trials were included and, for the discussion section, they assessed if there was a systematic attempt to set the new results in the context of existing trials. They concluded that while the numbers improved over the 25 years, reports of trials continue to inadequately consider existing trials and stated ‘[u]ntil this deficiency has been addressed by the research community, people are likely to continue to suffer and sometimes to die unnecessarily because of an unreliable evidence base for health care and additional research’. ²⁵

Building on that work by Clarke and colleagues, and on prior work such as the aprotinin example, ⁵ Robinson’s 2009 doctoral thesis asked the simple question: if clinicians should use earlier research to inform clinical decisions, why shouldn’t researchers do the same? Or more precisely, it should be standard that researchers planning a new study should be informed by a systematic and transparent synthesis of earlier similar studies to inform the justification and design of a new planned study. In her thesis, which focused on trials, Robinson suggested a term for this kind of thinking: ‘While the use of research synthesis to make evidence-informed decisions is now expected in health care, there is also a need for clinical trials to be conducted in a way that is evidence-based’. ⁶

Robinson’s doctoral thesis generated very worrisome results. She identified 227 systematic reviews related to healthcare interventions published in 2004, which included at least one meta-analysis where at least three included trials could have referred to earlier studies. The dataset contained 1523 original trials (published from 1963 to 2004) that could have cited on average 9.7 earlier similar trials. However, trials cited on average 1.9 prior relevant trials regardless of the number of possible prior trials to cite. ⁶ Furthermore, the proportion of trials cited was 21%, similar to the 20% in Fergusson and colleagues. ⁵

EBR is one way to reduce waste in the production and reporting of trials, through the initiation of trials that are needed to address outstanding questions and through the design of new trials in a way that maximises the information gained.^6,26 In 2011, Robinson and colleagues published a framework outlining how systematic reviews of previous similar studies can be used to identify and characterise evidence gaps, thereby informing the questions, design and methodological choices for new research. ²⁷ Earlier that year, the same authors used the term EBR in a paper describing methods to identify research gaps from guidelines. ²⁸

Around this time, Lund was exploring the use of systematic reviews in basic science to guide the development of clinical research. Studies suggested that the growing volume of research held the risk of increasingly reporting less meaningful observations. The goal was to use systematic reviews of biomechanical studies to generate new clinical research questions, particularly in the context of rehabilitating patients with chronic musculoskeletal conditions such as knee or hip osteoarthritis. While conducting systematic reviews of previous biomechanical studies and considering their application in formulating and prioritising new clinical research questions, Lund encountered a presentation by Iain Chalmers titled ‘The Scandalous Failure of Scientists to Cumulate Scientifically’ (presented at the Ninth World Congress on Health Information and Libraries, September 20, 2005). When Lund was appointed as a Guest Professor at Høgskolen i Bergen (HiB), later renamed the Western Norway University of Applied Sciences, in 2013, he proposed a new PhD programme incorporating what he referred to as ‘Evidence-Based Research’, inspired by that presentation. As HiB approved the inclusion of the EBR approach in the PhD programme application, Lund conducted an extensive literature search on EBR. He quickly realised that Robinson had already defined the concept of EBR in precisely the way they intended to implement it in the PhD programme.

On April 10, 2014, Robinson and Lund met at Johns Hopkins University and agreed to establish a network for EBR.

Evidence-Based Research Network

Subsequently, a meeting in Bergen established a network to promote the development and use of EBR. The following were invited attendees: Iain Chalmers (UK), Karen Robinson (USA), Donna Ciliska (Canada), Maureen Dobbins (Canada), Klara Brunnhuber (UK), Mona Nasser (UK), Matt Westmore (UK), Robin Christensen (Denmark), Carsten Juhl (Denmark), Gro Jamtvedt (Norway), Monica Nortvedt (Norway), Birgitte Espehaug (Norway), Hans Lund (Denmark/Norway), Kjetil Gundro Bruberg (Norway), Majbritt U. Johansen (Denmark), Mette Brandt Eriksen (Denmark), Thea Marie Drachen (Denmark) and Hanna Nykvist (Norway/Sweden). Paul Glasziou (Australia) and Malcolm Macleod (UK) participated virtually. Just before the meeting, Chalmers and Magne Nylenna published a commentary in The Lancet outlining the brief history and rationale for EBR and announcing the network’s establishment. ²⁹

During the meeting, a Steering Group was established, and two principles were formulated: (1) no new research studies without prior systematic reviews of existing evidence and (2) efficient production, updating and accessibility of systematic reviews. The following day, HiB and the Norwegian ‘Kunnskapssenteret’ arranged a public meeting to introduce and discuss the concept of EBR. Subsequently, the EBR Network published an introductory paper in the BMJ, ³⁰ that is now translated into nine different languages. This paper, known as the Bergen Statement on EBR, outlined the need for EBR and noted the role and responsibilities of researchers, funding agencies, research ethics committees and journal editors and reviewers. This paper made explicit the ways that EBR can address the factors set out in the Lancet series on increasing value and reducing waste.^{31 –36}

In the following years, the Steering Group met monthly, arranging dissemination of EBR through publications and presentations and preparing funding applications. One such application was for a COST (European Cooperation in Science and Technology) Action supporting grant for network building in the European Union. With support from Western Norway University of Applied Science, the EBR Network was awarded the grant in 2018.

For the following 5 years, the EBR Network focused on the new COST Action (CA17117) grant called ‘EVBRES’ (Evidence-Based Research). More than 36 European countries joined, and EVBRES was launched on October 17, 2018. Subsequently, more than 60 active participants from across Europe met and prepared courses, a handbook and several publications related to the conduct of EBR and efficient production, updating and dissemination of systematic reviews (see Appendix).

In 2021, the EBR Network published a three-paper series in the Journal of Clinical Epidemiology elaborating on the concept and application of EBR.^9,37,38 The concept had become more clearly aligned with the concept of evidence-based medicine in considering multiple factors in addition to an explicit consideration of prior research. As shown in Figure 1, EBR promotes a transparent and systematic synthesis of earlier similar studies and of end users’ perspectives. ⁹

OPEN IN VIEWER

Figure 1.

Elements of evidence-based research.

The EBR Network outlines needed research within three domains: (1) meta-research on the current practice of EBR, and consequences of not doing so, (2) development of methods to conduct EBR and (3) methods to implement and evaluate the implementation of EBR. The Network established a research interest group, GROVR (Group on Research on Value of Research), and research by members of the EBR Network, GROVR and EVBRES have found that researchers rarely use an EBR approach when justifying and designing a new study and when interpreting new results in the context of existing evidence.^{39 –42} As outlined in its research framework, the EBR Network endeavours to evolve from characterising current research practices and their consequences to developing methods for applying EBR, ultimately, assessing the impact of EBR itself.

Future of evidence-based research

During the fourth EBR Conference, two key priorities for the Network were discussed. First, EBR has primarily been developed within health sciences, but research in other scientific fields would also benefit from this approach. Thus, the Network is working to move beyond health. Second, the role of EBR in reducing waste and increasing the value of research may be even more critical where resources for research are more limited, such as in low- or middle- income countries. Furthermore, when researchers and research groups across the globe collaborate more closely, they can more effectively address the challenges of conducting valuable research in different contexts. Thus, the EBR Network is also focused on efforts beyond the ‘Global North’. The future of the Network, in addition to expanding to other scientific disciplines and ensuring global engagement, involves progressing from simply identifying problems and developing methods to assessing the impact of EBR.

To undertake research without systematically considering what has been done before is unethical, unscientific and wasteful. The EBR Network continues to raise awareness, conduct research, and engage stakeholders to ensure that new research is valuable and to achieve the vision of a world in which the cultural norm and expectation is that decisions about research are based on transparent and systematic use of evidence.