Abstract
Objective:
Communication skills are important, but the evaluation of these skills in the medical curriculum remains inconsistent. The aim of this study was to explore the relationship between the communication skills scores of junior medical students following curricula with different module evaluations and the clinical interview performance of the same students in a psychiatry department in their fifth year.
Design:
Prospective evaluation of communication and clinical interview skills.
Setting:
Medical school and psychiatry department inpatient setting.
Method:
The communication skills scores of 208 medical students in their second year (between 2009 and 2011) and their performance in interviews with patients in a psychiatry department in their fifth year (between 2012 and 2014) were assessed. Initial assessment was undertaken on the basis of problem-based learning (PBL) performance, evaluation of performance during conversation with patients, a related oral presentation and written report, students’ self-rating and a peer rating. Extra marks were given if students completed an optional special report inspired by the curriculum. To assess the interview performance of fifth-year medical students, the mini-Clinical Evaluation Exercise (mini-CEX) was used, together with weekly case-based discussions and the observation of a clinical interview with a real patient.
Results:
Tutors’ evaluations of the junior medical students’ performance during role-play (ρ = .22, p = .001), its related reciprocal assessment (ρ = .18, p = .010) and an extra-effort task (ρ = .18, p = .009) as part of the communication skills curriculum were significantly correlated with the students’ later performance in a clinical interview with a patient.
Conclusion:
Observation of student interactions and motivations along with reciprocal peer evaluations offer better measures of communication skills in medical education than are oral presentations, self-ratings and conversations with real patients.
Introduction
Communication breakdown is a factor in most instances of medical error (Maughan et al., 2011; Stillman, 2008; Tallentire et al., 2015; Tran and Johnson, 2010). It has been proposed that communication skills training should be offered in every discipline (Levinson et al., 1997; The Joint Commission, 2013). In the undergraduate curricula of medical schools in Taiwan, relatively little time is allocated to communication skills compared with other aspects of the curriculum despite the importance of these skills. Generally, communication-related curricula are provided only for junior medical students, who have less clinical experience and more time for psychosocial learning than senior medical students do. However, it is well known that communication skills are important for problem solving. Medical students could learn more if such curricula were provided to senior medical students and/or postgraduate staff. However, only a small number of clinical departments provide comprehensive clinical interviewing skill training for medical students, such as in the Department of Psychiatry at National Cheng Kung University Hospital in Taiwan.
There is an important connection between communication skills curricula for junior medical students and their performance in clinical interviews in senior courses and postgraduate careers (Deveugele et al., 2005). However, few studies have investigated this relationship in detail (Essers et al., 2012; Wright et al., 2009). Moreover, medical communication skills can be acquired by general observation or modelling, even without formal feedback or assessment. Training programmes for communication skills have been attempted in many forms (Duffy et al., 2004), including through single lectures, series lectures, workshops and simulations (Hoffman et al., 2004; Madan et al., 1998; Roter et al., 1998; Wehbe-Janek et al., 2011). However, the methods used to evaluate communication skills curricula are inconsistent (Gillis et al., 2015).
Direct observation of interactions with either actual patients or standardised patients is the technique most often used for communication skills assessment. Some objective evaluations have used audio- or video recording or real-time observations, which are present in the most robust studies. Expert facilitators and blind coders have often used a previously validated checklist to evaluate the interactions. However, in these studies, there is great diversity among the checklists selected. The most commonly used checklists are author-developed, and most of them have not been validated. Although self-evaluation is also common, several authors have found it lacking in comparison to more objective forms of assessment. A standardised format remains unavailable, and there are no exact criteria for defining ideal communication skills.
The aim of this study therefore was to explore the relationship between the scores of junior students in different communication skills assessments and their performance in clinical interviews with patients in a department of psychiatry in their senior year.
Methods
Participants
The study population consisted of second-year undergraduate medical students at the National Cheng Kung University in Tainan, Taiwan, who attended a communication skills and practice course between 2009 and 2011. Three years later, in their fifth year, these same students were senior undergraduate medical students, at which point they attended a 6-week clerkship training course run by the Department of Psychiatry at the same university hospital. The Institutional Review Board of the National Cheng Kung University Hospital approved the study.
Scoring of communication skills and practice curricula for second-year medical students
This curriculum consisted of three parts. The first part comprised a lecture that discussed the key components of communication, including verbal and nonverbal communication. Lecturers included a surgeon, who shared his expertise on how to deliver bad news to patients or families, and a psychiatrist, who demonstrated communication skills and nonverbal communication by using film clips or real-time interaction with the audience. A nurse supervisor, who was recognised as ‘the mother of hospice palliative care in Taiwan’, was invited to give a talk on this theme and to share her personal struggle. The second part of the curriculum comprised six problem-based learning (PBL) sessions in which a tutor provided different scenarios involving communication dilemmas and asked the students to resolve them using role-play. The tutor then led a discussion on the scenarios. The third and final part of the curriculum entailed a bedside conversation in which the student was asked to communicate with a patient and his or her family for at least 1 hour. The student then wrote a report on this conversation. The patients had been informed and had given their permission before speaking with these junior students.
The assessment of students’ communication skills consisted of several elements. The first was a PBL tutor’s evaluation of students’ performance during PBL, role-play and small group discussion. The second comprised evaluation of an oral presentation and a report, together with a self-rating and a reciprocal peer’s rating of performance during group learning, and an evaluation of the student’s performance during the conversation with the patient. The final element was an optional task: the completion of a special report inspired by this curriculum and the recommended readings. Extra marks were given if students completed this task as students who were more motivated to invest in their communication skills seemed to demonstrate better performance when required to interview real patients.
Psychiatric interview performance of fifth-year medical students
When the medical students became seniors, a 6-week psychiatric clerkship took place. This training course included psychiatric lectures and training on inpatient and outpatient services and on community psychiatry. The mini-Clinical Evaluation Exercise (mini-CEX) was used to evaluate and improve students’ clinical interview performance during the clinical learning period. A psychiatrist who was a major in medical education in the psychiatric clerkship programme was responsible for leading weekly case-based discussions, during which each student’s competence in medical record-keeping, clinical assessment, investigations and referrals, treatment, follow-up and future planning was assessed along with his or her professionalism and overall clinical judgement. At the end of the training course, each student’s clinical skills were evaluated via an oral examination in which the student was required to interview one real patient with a mental disorder for 30 minutes. An examiner, who was a senior psychiatrist and the given student’s clinical teacher, observed the student’s performance and discussed it with the student before allocating a score.
All those who completed the above-mentioned assessment did not have access to the previous scores. It was especially important that we did not rely on observation by only one examiner. Residents used mini-CEX to evaluate the students. A psychiatrist who was a major in medical education in the psychiatric clerkship programme was responsible for the weekly case-based discussion. The clinical tutor conducted the oral examination, in which the student was required to interview a real patient with a mental disorder for 30 minutes. The overall assessment of senior students during their psychiatry rotation consisted of several parts including a paper–pencil test for lectures, with clinical bedside performance being assessed by supervisors on the basis of an interview performance with a real patient. In this study, only interview performance was used as a measure of communication competence, because we believe it to represent the most robust way of measuring clinical communication skills. We omitted the related data from this analysis.
Statistics
The scores for each assessment ranged from 0 to 100. Due to the skewness of the data, the Spearman’s rho correlation was used to examine the relationship between the scores of students in their second and fifth years. The data were analysed using the Statistical Package for Social Science, software version 17 (SPSS Inc., Chicago, Illinois, USA). The threshold for statistical significance was set at p < .01 (two-tailed) for the correction of multiple comparisons.
Results
A total of 208 second-year undergraduate medical students were recruited. Their mean (SD) age was 22.9 (1.7) years. Of these students, 71.6% (149/208) were male. Mean, standard deviation and range of scores are shown in Table 1. Communication skills as assessed earlier were significantly and positively correlated with the scores derived from students’ interviews with patients suffering from mental disorders in their senior years (p < .01, Table 1).
Spearman’s rho correlation between the second-year students’ communication skills scores and their psychiatric interview scores in their fifth year.
Discussion
Tutors’ evaluation of junior medical students’ performance during role-play and related group discussion was significantly correlated with their later performance in a clinical interview of a patient with a mental disorder. This result supports the findings of other researchers (Gillis et al., 2015), who reported that the observed interaction between a medical student and actual patients or standardised patients is a good method for evaluating communication skills.
Communication is multidimensional, and assessments need to include indistinct aspects such as the establishment of the physician–patient relationship, empathy and nonverbal interaction (Wehbe-Janek et al., 2011). Although communication has been recognised as a core competency for health professionals and medical students, evaluation of these skills varies between different curricula (Davis et al., 2006; Kalet et al., 1992). It has been reported that the most robust studies have used objective scoring using real-time observations or video recording, and have involved an expert to evaluate students’ performance (Gillis et al., 2015). However, it remains uncertain whether the communication skills training curricula in preclinical courses, compared to the curricula in the clinical training years, are relevant to students’ future clinical careers. Findings from this study indicate that the scoring of an observed interaction provides a good indicator of students’ later performance in communication-related activities.
The reciprocal peer assessment of students’ role-play performances and discussion also offers a good indicator of interview performance in senior years. Although the peers in this study were not professionals, evaluation by peers could therefore be helpful in the assessment of interview skills (Awasthi and Yadav, 2015; Cushing et al., 2011).
This study also investigated the significance of an optional report on recommended books. It is well known that motivation is one of the most important predictors of later academic performance (Holland, 2016). It is reasonable to assume that students who made more effort to read related books and write related reports were more motivated to learn communication skills, accounting perhaps for why their skill performance was better in the senior years.
Limitations
Several limitations should be noted. First, generalisation of these results should be undertaken with caution because higher communication skills scores do not necessarily indicate better physician–patient relationships in the later clinical practice of students. Some researchers (Davis and Lee, 2011; Ponton-Carss et al., 2011) report a trend of worsening communication skills with increasing seniority and suggest that this implies that communication skills are inherent, not taught. Second, reliability data were not available for different evaluators or senior psychiatrists in this study. Third, there was no comparison group. Therefore, it is difficult to make claims for one form of communication skills training, such as a lecture, being less effective than another. This study only demonstrates that some scoring methods are good predictors of later performance. Finally, the sample size was small, and the sample was limited to one medical school.
Footnotes
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research received funding (NCKUH-10606014) from the National Cheng Kung University Hospital. The authors are indebted to the research participants. They would also like to thank Yen-Hsiu Chen and Chien Ting Lin for their administrative support.