Abstract
Most of us can just about still recall frantic trips to libraries as medical students, in which we leafed through pages of medical texts in search of the ‘answer’ to the question, which, once found, was gleefully transcribed in to essay with minimal questions asked of it. While not necessarily a bad approach to tackling an undergrad essay question, what would happen if we applied that approach to answering a more clinical question, let’s say ‘would giving corticosteroids to this child with a sore throat make the sore throat get better faster’? The problem is that we might very well find many papers addressing such a question, and they might give different answers. Which do we believe then? Would accepting one answer without questioning it, be a justifiable basis for giving a prednisolone script to a 2 year old? This is where systematic reviews can be really helpful to us as jobbing clinicians.
The Cochrane library says that: A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias in order to produce more reliable findings that can be used to inform decision making.
The key steps in a systematic review are:
Defining the clinical question (this is usually done in a population, intervention, comparison, outcome (PICO) style) and agreeing eligibility criteria (what you will and won’t allow in your review) A comprehensive literature search to make sure that you’ve picked up every possibly relevant study Applying the eligibility criteria to the results of your literature search to get rid of any studies that do not meet them Quality assessing eligible studies and assessing bias (to decide what weight to give different studies) By this stage you now have a dataset that is both comprehensive and specific to your particular question Analyse results using meta-analysis if appropriate (see below) Preparing a summary of the review for publication
‘Meta-analysis’ is different to a ‘systematic review’. If the results of several studies are combined to give an overall statistic, this is a meta-analysis. They are often used to combine results of different trials that looked at the same thing. The idea is that by combining all the trial data (and therefore more data from more patients), you get a more precise estimate of the effect of an intervention. Many systematic reviews will contain a meta-analysis, but it is important to understand that not all will. This is because combining results from studies is not meaningful if the individual studies were too different, were of poor quality or measured different outcomes. It is important to understand that while a good meta-analysis of high-quality similar trials may be very useful, a meta-analysis of poor-quality data is a dangerous thing (because we may assume that it is correct). Results of meta-analyses are often represented pictorially in a forest plot.
Hopefully this column is teaching you to be more than a little questioning of ‘evidence’. A high-quality systematic review is a wonderfully useful thing. They inform many clinical guidelines and help our everyday clinical practice. A poorly conducted systematic review however is quite the opposite, and there are many ways in which a systematic review can be methodologically flawed. A necessary (but not sufficient) guide is that high-quality reviews should follow the internationally agreed PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) checklist for the conduct and reporting of reviews. Being able to critically appraise a review is an important skill – the Critical Appraisal Skills Programme (CASP) worksheets guide you through how to do this if you are interested (http://media.wix.com/ugd/dded87_a02ff2e3445f4952992d5a96ca562576.pdf).
For those of us with less time, but who really do want to apply evidence in our everyday practice, the Cochrane Library is a fantastic, easy-to-access collection of high-quality systematic reviews. Have a look at it when you get a chance, and consider having it open on your desktop at work – it is a fast and effective way of bringing evidence to your fingertips during a consultation (www.cochranelibrary.com/cochrane-database-of-systematic-reviews/index.html).