Abstract
In the current climate of evidence-based medicine (EBM), health-care professionals are encouraged to base clinical decisions on sound scientific evidence, rather than on anecdotes or personal opinions. Are medical and nursing students sufficiently taught and so equipped to assess the quality?
In attempting to answer this question, HealthWatch has been running a “Student Prize Competition” for the past 10 years, and this report describes the 2011 competition.
Aim
The project is designed to determine whether medical and nursing students have acquired the skills needed to critically appraise clinical trial protocols and hence assess the validity of research findings. Often claims for effectiveness or safety of health-care treatments published in the medical and lay press and on the Internet are, on closer examination, based on poorly designed clinical trials that could not possibly support the stated claims. It is vitally important that our future doctors and nurses, who will base their advice and clinical decisions on research findings, develop sound judgements of which studies can be trusted and which are obviously flawed.
Method
Students are invited to appraise four one-page long hypothetical research protocols and to rank them on a scale of 1–4 (1=best, 4=worst) according to which one is most likely to provide a reliable answer to the stated aim of the trial. The protocols are designed so as to contain varying degrees of scientific, methodological and ethical flaws (e.g. no control group, or no informed consent), which the students are expected to identify and comment upon. They are required (in no more than 600 words) to explain their reasons for assigning their ratings and suggest ways in which the protocols could be improved. Their replies are then assessed against the preassigned ranking from the judges.
(Protocols of the current and previous competitions can be viewed on the HealthWatch website
Administration
The competition is open to all medical and nursing students in the UK and is administered by Professor David Bender. He notifies all medical and nursing colleges early in the year, collects the entries, sends the anonymized entries to the judges and notifies all participants of the results. In addition, he personally contacts staff at selected medical schools, encouraging them to bring the competition to the attention of their students.
Results
The majority of students reported having heard about the competition via the Student BMJ, emails from their university, or via their university virtual learning environment. A total of 130 entries, including five from nurses, were received, representing the best response since the competition began 10 years ago. This is still disappointing, when compared with the total student body (there are approximately 6000 medical students in the UK). Of the 130 entries, 62 (58 from medical, and 4 from nursing students) had ranked the protocols in the correct order, and these were then subjected to detailed scrutiny by the judges who were unaware of the students’ identities or college affiliations. As in previous years, the judges paid particular attention to see whether students identified protocol design weaknesses, such as absence of, or inappropriate control group, absence of patient or assessor blinding, and ethical issues. The judges assessed the entries independently of each other with the aid of a 12-point checklist and then compared their results to reach agreement.
While it was disappointing that fewer than half the entries ranked the protocols in the correct order, those that did, generally showed a good level of understanding of clinical trial design, and their numerous valid suggestions on how the protocols could be improved indicated that they had carefully thought about design flaws. The judges were more likely to be impressed by students who gave well-articulated sound explanations as to why they considered some protocols inferior to others, rather than merely having ticked boxes in “clinical trial design tables” obtained via the Internet and accompanied by only sparse comments which, on closer examination, suggested a lack of true understanding.
There was confusion in the mind of some students as to what constitutes valid patient consent, e.g. in the “obesity” protocol, some interpreted the headteachers’ agreement to their schools taking part, to mean valid consent, whereas in fact the specific consent of the parents should have been obtained. Nevertheless, the judges were pleased to note that, compared with earlier years, this year's students paid more attention to ethical aspects and more often commented correctly on absence of ethics committee clearance or valid consent clauses in specific protocols. This, together with their informed comments on good protocol design, is to be welcomed and suggests that medical and nursing colleges are now paying more attention to the teaching of clinical research methodologies.
As in earlier years, the poor response from nursing students (only five entries) is most disappointing, and active steps by our HealthWatch President and committee members to generate interest among the nursing community have so far not been successful. Possible further approaches are now being explored. Interestingly, of the five nurses who took part, four (80%) had placed the protocols in the correct order, one of whom gave sufficiently well-argued reasons for her choice to merit a “commendation”.
Prize presentation
The prizes will be awarded at the HealthWatch AGM by our President Nick Ross (see photo).
From left to right: Derek Ho (winner), Jennifer Johnson and Mark Loughrey (runners-up) of the HealthWatch Student prize sponsored by the Medico-Legal Society.
DECLARATIONS