The Development of an Online Practice-Based Evaluation Tool for Social Work

Abstract

Objective: This paper describes the development of a practice-based evaluation (PBE) tool that allows instructors to represent their student’s clinical performance in a way that is sufficiently authentic to resonate with both instructors and students, is psychometrically sound, and is feasible in the context of real practice. Method: A new online evaluation tool was designed to address several of the problems associated with previous methods of evaluation, and was tested on 190 field instructor—student pairs. Results: Results demonstrated feasibility of the tool, high acceptability from students and faculty, high internal consistency, and clearly reduced ceiling effect, when compared with a traditional competency-based evaluation (CBE) tool. It did, however, continue to result in a strong skew toward positive evaluation and did not increase the identification of students at risk. Conclusions: The online PBE tool demonstrates promise in redressing some of the evaluation issues posed by the previous CBE model of evaluation.

Keywords

competency practice evaluation field education clinical training

Get full access to this article

View all access options for this article.

References

Arkava, M. , & Brennan, C. (Eds.). (1976). Competency-based education for social work: Evaluation and curriculum issues. New York: Council on Social Work Education.

Bogo, M. , Regehr, C. , Hughes, J. , Power, R. , & Globerman, J. ( 2002). Evaluating a measure of student field performance. Journal of Social Work Education, 38(3), 385-401.

Bogo, M. , Regehr, C. , Power, R. , & Regehr, G. ( 2007). When values collide: Field instructors’ experiences of providing feedback and evaluating competency. The Clinical Supervisor, 26(1-2), 99-117.

Bogo, M. , Regehr, C. , Powers, R. , Hughes, J. , Woodford, M. , & Regehr, G. ( 2004). Towards a new approach for evaluating student field performance . Journal of Social Work Education, 40(3), 417-425.

Bogo, M. , & Vayda, E. ( 1998). The practice of field instruction in social work: Theory and process (2nd ed.). New York : Columbia University Press.

Bondy, K.N. , Jenkins, K. , Seymour, L. , Lancaster, R. , & Ishee, J. ( 1997). The development and testing of competency-focused psychiatric nursing clinical evaluation instrument. Archives of Psychiatric Nursing, XI(2), 66-73.

Brennan, E.C. ( 1982). Evaluation of field teaching and learning. In B. W. Sheafor & L. E. Jenkins (Eds.), Quality field instruction in social work . New York: Longman.

Gonsalvez, C. , & Freestone, J. ( 2007). Field supervisors’ assessments of trainee performance: Are they reliable and valid? Australian Psychologist, 42(1), 23-32.

10.

Govaerts, M. , van der Vleuten, C. , Schuwirth, T. , & Muijtjens, A. ( 2006). Broadening perspectives on clinical performance assessment: Re-thinking the nature of in-training assessment. Advances in Health Science Education, 12, 239-260.

11.

Hyland, T. ( 1995). Morality, work and employment: Towards a values dimension in vocational education and training. Journal of Moral Education , 24(4), 393-406.

12.

Muntz, Y. , Moorthy, K. , Bann, S. , Shah, J. , Ivanova, S. , & Drazi, S. ( 2004). Ceiling effect in technical skills of surgical residents . American Journal of Surgery, 188, 294-300.

13.

Regehr, C. , Regehr, G. , Leeson, J. , & Fusco, L. ( 2002). Setting priorities for learning in the field practicum: A comparative study of students and field instructors. Journal of Social Work Education, 38(1), 55-65.

14.

Regehr, G. , Hodges, B. , Tiberius, R. , & Lofchy, J. ( 1996). Measuring self-assessment skills: An innovative relative ranking method. Academic Medicine, 71(10supp), S52-S54.

15.

Regehr, G. , Regehr, C. , Power, R. , & Bogo, M. ( 2007). Can we build a better mousetrap? Improving measures of social work practice performance in the field. Journal of Social Work Education, 43(2), 327-344.

16.

Shrout, P.E. , & Fleiss, J.L. ( 1979). Intraclass correlations: Uses in assessing rater reliability . Psychological Bulletin, 2, 420-428.

17.

Siegel, B.S. , & Greenberg, L.W. (2000). Effective evaluation of residency education: How do we know it when we see it? Pediatrics , 105(4), 964-965.

18.

Sliwa, J.A. , & Kowalske, K.J. (2000). Assessing resident clinical competence. American Journal of Physical Medicine and Rehabilitation , 79, 468-473.

19.

Ward, M. , Gruppen, L. , Regehr, G. ( 2002). Research in self-assessment: Current state of the art . Advances in Health Sciences Education, 7, 63-80.

20.

Williams, B. ( 1997). University-based social work education: Quaint and dated ? Canadian Social Work Review, 14(1), 69-81.