Abstract
Our future healthcare professionals need training in the principles and practice of evidence-based treatments. We are all inundated with publicity in the media and internet about new and allegedly more effective treatments but which, on closer examination, are based on poorly designed clinical trials that could not possibly support the stated claims. Medical and nursing/midwifery students and allied professions need to be able to assess whether and how far such diagnostic or therapeutic claims for a particular treatment are valid to guide their clinical decisions. The 12th Annual HealthWatch Student Prize Competition (which this year was funded by the Medico-Legal Society) aims to determine whether these students have the essential skills to critically appraise clinical trial protocols and assess the validity of research findings.
Method
Students are invited to appraise four one-page long hypothetical research protocols and to rank them on a scale of 1–4 (1 = best, 4 = worst) according to which one is most likely to provide a reliable answer to the stated aim of the trial. The protocols are designed to contain varying degrees of scientific, methodological and ethical flaws (e.g. no control group, or no informed consent), which the students are expected to identify and comment upon. They are required (in no more than 600 words) to explain their reasons for assigning their ratings and suggest ways in which the protocols could be improved. Their replies are then assessed against the preas-signed ranking from the judges. (Protocols of the current competitions can be viewed on the HealthWatch website.)
Administration
The competition is open to all medical and nursing/ midwifery students in the UK and is administered by Professor David Bender. He notifies all medical and nursing colleges early in the year, collects the entries, sends the anonymized entries to the judges and notifies all participants of the results. In addition, he and Professor Debra Bick personally contact staff at selected medical and nursing schools, encouraging them to bring the competition to the attention of their students.
Results
Most students reported they had heard about the competition via emails from their university, or via their university intranet/virtual learning environment. A total of 135 entries, including 21 from nurses/mid-wives, were received, representing the best response since the competition began 11 years ago. Though still disappointing, when compared to the total student body (there are approximately 6000 medical students in the UK), it is an encouraging trend and suggests greater awareness among students of the need for research scrutiny. Of the 114 entries from medical students, 88 (77%) had ranked the protocols in the correct order, while the corresponding figure for nurses/midwives is 13 (62%) out of 21. Thus, a total of 101 correctly-ranked entries (comprising 404 protocol-sections) were then subjected to detailed scrutiny by six judges, who were unaware of the students' identities or college affiliations. As in previous years, the judges paid particular attention to whether students identified protocol design weaknesses, such as absence of or inappropriate control group, absence of patient or assessor blinding and ethical issues. The six judges assessed the entries independently of each other with the aid of an 11-point check-list and then compared their results to reach agreement.
As well as an overall increase in the number of entries received, it was encouraging to note that, in both categories of students, a much higher proportion (77% of medical and 62% of nursing/midwifery students) had ranked the protocols in the correct order, compared with less than 50% in the 2011 competition. Nevertheless, when scrutinizing students' narratives given in support of their choice of ranking, it became obvious that many still have only a limited grasp of essential protocol design elements, as evidenced by approximately half the entries having been awarded only 22 or fewer points (of the maximum possible 44) on the 11-point check list.
The judges were more impressed by students who gave well expressed sound explanations as to why they considered some protocols inferior to others, rather than merely having ticked boxes in “clinical trial design tables” obtained via the internet and accompanied by only sparse comments which, on closer examination, suggested a lack of true understanding.
The short-list of the top ten entries from which the winners and runners-up were selected generally showed a good level of understanding, and a well-argued rationale for their choice of ranking. Their numerous valid suggestions on how the protocols could be improved indicated that they had thought carefully about design flaws. It was pleasing to have a better representation of nurses/midwives whose entries were of a higher standard than previously (62% had ranked the protocols in the correct order). Nurses and midwives need to understand the principles of good trial design as they are frequently the first professionals patients will ask for advice about (sometimes exaggerated) media reports about “the new wonder drug”.
The prizes were awarded by the president, Nick Ross, at the HealthWatch AGM on October 23 2012 at the Medical Society of London.
Footnotes
Acknowledgements
The HealthWatch committee thank all those involved in the organization and scientific input/assessments of this year's competition, specifically: David Bender, Debra Bick, Walli Bounds, Roger Fisken, Sally Gordon Boyd, William Lee, Anil Patel and Gillian Robinson. Without their dedication and generously given support, this important competition would not have been possible.
HealthWatch gratefully acknowledges the interest in, and financial support for, this important competition from the Medico-Legal Society (M-LS), and in earlier years from the AJAHMA Trust. The M-LS funding is now exhausted, but Professor John Garrow has generously offered to fund the 2013 competition.