Objective. To examine the impact of online evidence retrieval on clinicians’ decision-making confidence and to determine if this differs for experienced doctors and nurses.Methods. A sample of 44 doctors and 31 clinical nurse consultants (CNCs) answered 8 clinical scenarios (600 scenario answers) before and after the use of online evidence resources. Clinicians rated their confidence in scenario answers and in the evidence they found using the information system.Results. Prior to using online evidence, 37% of doctors and 18% of CNCs answered the scenarios correctly. These clinicians were more confident (56% very confident or confident) in their answers than those with incorrect (34%) answers. Doctors with incorrect answers prior to searching rated their confidence significantly higher than did nurses who were incorrect. After searching, both groups answered 50% of scenarios correctly. Clinicians with correct answers had greater confidence in the evidence found compared to those with incorrect answers. Doctors were more confident in evidence found confirming an initially correct answer than were nurses. More than 50% of clinicians who persisted with an incorrect answer after searching reported that they were confident or very confident in the evidence found. Clinicians who did not know scenario answers before searching placed equal confidence in evidence that led them to a correct or incorrect answer.Conclusions. The information obtained from an online evidence system influenced clinicians’ confidence in their answers to the clinical scenarios. The relationship between confidence in answers and correctness is complex. Both existing knowledge and professional role were mediating factors. The finding that many clinicians placed confidence in information that led them to incorrect answers warrants further investigation.
Gorman P, Ash J, Wykoff L. Can primary care physicians’ questions be answered using the medical literature?Bull Med Library Assoc. 1994;82:140-146.
2.
Hersh W, Hickam D. How well do physicians use electronic information retrieval systems? A framework for investigation and systematic review. JAMA. 1998;280:1347-1352.
3.
Hersh W, Crabtree K, Hickam D, et al.Factors associated with success in searching MEDLINE and applying evidence to answer clinical questions. JAMA. 2002;9:283-293.
4.
Haynes B.Of studies, syntheses, synopses, and systems: the “4S” evolution of services for finding current best evidence. Evid Based Med. 2001;6:36-38.
5.
Ward M, Gruppen L, Regher G. Measuring self assessment: current state of the art. Adv Health Sci Educ. 2002;7:63-80.
6.
Harvey N.Confidence in Judgement. Trends Cognit Sci. 1997;1:78-82.
7.
Kahneman D, Slovic P, Tversky A. Judgement under Uncertainty: Heuristics and Biases. New York: Cambridge University Press; 1982.
8.
Gigerenzer G, Hoffrage U, Kleinbolting H. Probabilistic mental models: a Brunwikian theory of confidence. Psychol Rev. 1991;98:506-528.
9.
Pallier G, Wilkinson R, Danthir V, et al.The role of individual differences in the accuracy of confidence judgements. J Gen Psychol. 2002;129:257-299.
10.
Ely JW, Osheroff J, Ebell M, et al.Analysis of questions asked by family doctors regarding patient care. Br Med J. 1999;319:358-361.
11.
Hersh W, Pentecost J, Hickam D. A task-oriented approach to information retrieval evaluation. J Am Soc Information Sci. 1996;47:50-56.
12.
Elstein A, Shulman L, Sprafka S. Medical Problem Solving: An Analysis of Clinical Reasoning. Cambridge (MA): Harvard University Press; 1978.
13.
Elstein A, Scharz A. Clinical problem solving and diagnostic decision-making: a selective review of the cognitive literature. Br Med J. 2002;324:729-732.
14.
Gosling A, Westbrook J. Factors associated with differences between professional groups’ use of point-of-care online evidence—a survey of 5,511 health professionals. Paper presented at the 11th National Health Informatics Conference and 12th National RACGP Computer Conference; 2003 Aug 10-12; Sydney, Australia.
15.
Gosling A, Westbrook J. Allied health professionals’ use of online evidence: a survey of 790 staff working in the Australian public hospital system. Int J Health Inform. 2004;73:391-401.
16.
Hersh W.Information Retrieval: A Health and Biomedical Perspective. New York: Spinger-Verlag; 2003.
17.
Westbrook J, Gosling A, Coiera E. Do clinicians use online evidence to support patient care? A study of 55,000 clinicians. J Am Med Inform Assoc. 2004;11:113-120.
18.
Hersh W, Crabtree M, Hickam D, Sacherek L, Rose L, Friedman C. Factors associated with successful answering of clinical questions using an information retrieval system. Bull Med Library Assoc. 2000;88:323-331.
19.
Thompson C.Clinical experience as evidence-based practice. J Adv Nurs. 2003;43:230-237.
20.
Stankov L.Complexity, metacognition and fluid intelligence. Intelligence. 2000;28:121-143.
