Comparing Methods of Learning Clinical Prediction from Case Simulations

Abstract

Feedback to physicians about how they use information in making judgments can improve the quality of their judgments, but questions remain about which types of feedback are most effective. The authors conducted a controlled study of feedback in 60 medical students learning to predict the risk of cardiovascular death based on the presence or absence of five risk factors. After a pretest of 40 cases abstracted from patient records, the students worked through 173 computer-simulated cases and a posttest of 40 patient cases. The students received no feedback, probability feedback (correct probability of cardiac death for each case), cognitive feedback (the correct cue weights compared with their own weights derived from the previous set of cases), or both types of feedback. Students who received probability feedback markedly improved both base rate calibration and discrimination. Those who received only cognitive feedback showed no improvement over control on any of the measures of learning. All subjects were highly consistent in their weightings. The superiority of probability feedback differed from previous findings that cognitive feedback was essential for mastery of multiple-cue-probability learning tasks. The information on cue-outcome relationships given by cognitive feedback may be more useful when these relationships are complex and the combining rule is not known, while the precise outcome information provided by probabilistic feedback is more useful when the combining rule is known and the cue- outcome relationships are straightforward. Thus, the optimal method of learning depends on the nature of the task. Key words: risk factors; computer simulations; heart disease. (Med Decis Making 1992;12:213-221)

Get full access to this article

View all access options for this article.

References

Poses RM , Cebul RD , Collins M. , Fager SS The accuracy of experienced physicians' probability estimates for patients with sore throats JAMA. 1985;254:925-9.

Christensen-Szalanski Jjj , Bushyhead JB

Physicians' use of probabilistic information in a real clinical setting

J Exp Psychol [Hum Percep]. 1981;7:928-35.

Dolan JG , Bordley DR , Mushlin AI

An evaluation of clinicians' subjective prior probability estimates

Med Decis Making. 1986;6:216-23.

Greibe J. , Bugge P. , Gjørup T. , Lauritzen T. , Bennevie O. , Wulff HR

Long-term prognosis of duodenal ulcer: follow-up study survey of doctors' estimates

Br Med J. 1977;2:1572-4.

Tape TG , Wigton RS. Medical students' and residents' estimates of cardiac risk Med Decis Making. 1989;9:170-5.

Goldberg LR

Simple models or simple processes? Some research on clinical judgments

Am Psychologist. 1968;23:483-96.

Brehmer B.

In one word: not from experience

Acta Psychol. 1980;45:223-41.

Klayman J. On the how and why (not) of learning from outcomes In Brehmer B, Joyce CRB. Human judgment: the SJT view. Amsterdam : North Holland, 1988.

Wigton RS , Poses RM , Collins M. , Cebul RD

Teaching old dogs new tricks: using cognitive feedback to improve physicians' diagnostic judgments on simulated cases

Acad Med. 1990;65 (suppl):S5-S6.

10.

Wigton RS , Patil KD , Hoellerich VL

The effect of feedback in learning clinical diagnosis

J Med Educ. 1986;61:816-22.

11.

Balzer WK , Doherty ME , O'Connor R. Jr.

Effects of cognitive feedback on performance

Psychol Bull. 1989; 106:410-33.

12.

Hoffman PJ , Earle TC , Slovic P.

Multidimensional functional learning (MFL) and some new conceptions of feedback

Organ Behav Hum Perform. 1981;27:75-102.

13.

Lindell MK

Cognitive and outcome feedback in multiple-cue probability learning tasks

J Exp Psychol: Human Learning and Memory. 1976;2:739-45.

14.

Hammond KR

Computer graphics as an aid to learning

Science. 1971;172:903-8.

15.

Hammond KR , Summers DA , Deane DH

Negative effects of outcome-feedback in multiple-cue probability learning

Organ Behav Hum Perform. 1973;9:30-4.

16.

Smith KW , McKinlay SM , Thorington BD

The validity of health risk appraisal instruments in assessing coronary heart disease risk

Am J Public Health. 1987;77:419-24.

17.

Abraham S. , Johnson CL , Caroll MD Total serum cholesterol levels of adults 18-74 years: United States, 1971-1974 Hyattsville, Md: National Center for Health Statistics, U.S. Department of Health, Education and Welfare; 1978;19. (Vital and Health Statistics; series 11, no. 205. DHEW publication no. [PHS] 78-1652).

18.

Roberts J. , Maurer K. Blood pressure levels of persons 6-74 years, United States, 1971-1974 Hyattsville, Md: National Center for Health Statistics, U.S. Department of Health, Education and Welfare;1977;45. (Vital and Health Statistics; series 11, no. 203. DHEW publication no. [HRA]78-1648).

19.

Bonham GS , Leaverton PE Use habits among adults of cigarettes, coffee, aspirin, and sleeping pills, United States, 1976 Hyattsville, Md: National Center for Health Statistics, U.S. Department of Health, Education and Welfare; 1977;45. (Vital and Health Statistics ; series 10, no. 131. DHEW publication no. [PHS] 80-1559).

20.

Najjar MF , Rowland M. Anthropometric reference data and prevalence of overweight, United States, 1976-80 Washington, DC: U.S. Government Printing Office, 1987;15. (Vital and Health Statistics. Series 11, nl. 238. DHHS publication no. [PHS] 87-1688).

21.

Hammond KR , Stewart TR Social judgment theory In Kaplan MF, Schwartz S, eds. Human judgment and decision processes. London: Academic Press, 1975; 271-312.

22.

SAS user's guide: Statistics, version 5 edition Cary, NC : Sas Institute Inc.; 1985.

23.

Brunswik E. The conceptual framework of psychology Chicago: University of Chicago Press, 1952.

24.

Hammond KR , Hursch CJ , Todd FJ

Analyzing the components of clinical inference

Psychol Rev. 1964;71:438-56.

25.

Tucker LR

A suggested alternative formulation in the developments by Hursch, Hammond, and Hursch, and by Hammond, Hursch, and Todd

Psychol Rev. 1964;71:528-30.

26.

Brehmer B. The development of social judgment theory In Brehmer B, Joyce CRB, eds. Human judgment: the SJT view. Amsterdam : North Holland, 1988.

27.

Stewart TR Judgment analysis: procedures In Brehmer B, Joyce CRB, eds. Human judgment: the SJT view. Amsterdam: North Holland, 1988.

28.

Murphy EA

Biostatistics in medicine

Baltimore: The Johns Hop-kins University Press; 1982:230-44.

29.

Deane DH , Hammond KR , Summers DA

Acquisition and application of knowledge in complex inference tasks

J Exp Psychol. 1972;92:20-6.

30.

Brehmer B. , Svensson C.

Learning to use functional rules in inference tasks

Scand J Psychol. 1976;17:313-9.

31.

Dawes RM , Corrigan B.

Linear models in decision making

Psychol Bull. 1974;81:95-106.

32.

Schmitt N. , Coyle B. , King L.

Feedback and task predictability as determinants of performance in multiple cue probability learning tasks

Organ Behav Hum Perform. 1976;16:388-402.

33.

Adelman L.

The influence of formal, substantive, and contextual task properties on the relative effectiveness of different forms of feedback in multiple-cue probability learning tasks

Organ Behav Hum Perform. 1981 ;27:423-42.

34.

Poses RM , Cebul RD , Wigton RS , Collins M.

Feedback on simulated cases to improve clinical judgment (abstr)

Med Decis Making. 1986;6:274.