Student’s perspective of Residents as educators following introduction of Accreditation Council for Graduate Medical Education-International (ACGME-I) in SingHealth Residency

Abstract

Introduction:

Singapore’s postgraduate medical education underwent a transition in 2010, moving away from the House staff model to the American Residency model. Teaching was defined as a core competency for SingHealth Residency programs. This study sought to determine how this transition impacted medical students’ perception of Residents as teachers.

Methods:

A 19-item Likert-type questionnaire comprising three domains—(1) quality, (2) frequency and (3) engagement—was given to all Duke-NUS medical students at the end of each clerkship from the academic year (AY) 2009–10 through 2013–14. The first two academic years were classified as the pre-initiation stage of the Residency, and the latter three, the post-initiation stage. Students rated Residents they encountered in their most recent clerkship.

Results:

Student ratings have gradually improved, with the largest improvement occurring after the initiation of Residency, and after the introduction of Residents as Future Teachers courses. Student ratings in the post-initiation Residency years combined were significantly higher than the pre-initiation of Residency years combined, for all three domains. Analysis of individual items revealed significant improvement in student ratings on all items in all domains, except for the quality of teaching in performing procedural skills.

Conclusion:

Findings from this study indicate a positive shift in the teaching culture with the transition of Singapore’s postgraduate medical education from the House staff model to the Residency program.

Keywords

Residency ACGME-I Singapore Residents as educators

Introduction

In 2010, Singapore’s postgraduate medical education underwent a transition from the House staff model to the American Residency model, in collaboration with the Accreditation Council for Graduate Medical Education-International (ACGME-I).^1–5 This shift to the Residency model created changes in the roles and expectations of all junior doctors in training, with respect to the teaching of medical students.^6,7 “Resident” refers to junior doctors in training, including House staff and medical officers. In the House staff model, Residents were not expected to play a major role in the clinical instruction of medical students. By contrast, Residents are expected to play a significant role in the teaching and evaluation of medical students (as outlined by the Liaison Committee on Medical Education)⁸ in the American Residency model.

Singapore Health Services (SingHealth), one of the main sponsoring institutions for the Residency programs, felt teaching was an important competency. This led to Residents as Future Teachers courses being incorporated into their Residency training.

At Duke-NUS Medical School (DNUS), a US-style medical school in Singapore, students begin their clinical training in the second year. Clinical training for DNUS students is primarily conducted at SingHealth institutions. They are embedded in clinical teams to enrich and refine their clinical skills, and to learn to diagnose and manage core patient conditions.

Although the new residency system had been introduced, it was uncertain when, or if, the culture of teaching would change. Residency spots are not available for all graduates following medical school, which explains why some medical school graduates continue to enter the traditional House staff/Medical Officer system.

At this transitional stage, the House staff and American Residency programs exist concurrently.⁹ As such, medical students may be exposed to the two different systems, arising from interaction with Residents from the House staff model and the American Residency model. Hence, it is important to examine this transitional stage to determine whether a positive shift in teaching culture would result from it. In this study, we sought to determine the impact of this transition on the culture of teaching from the medical students’ perspective. We measured, among other things, the quality (1=poor, 5=excellent) and frequency (1=not at all, 5=very often) of teaching of core clinical skills, as well as their opinion of the Residents’ engagement with the students as teachers, before and after the transition to the Residency system (1=strongly disagree; 5=strongly agree).

We hypothesized that the culture of teaching would gradually develop, with medical students reporting an improvement in the quality and frequency of the teaching of clinical skills, as well as the Residents’ engagement with them as teachers. Understanding the Residents’ involvement in teaching would provide a baseline assessment for improving the Residents as Future Teachers training programs. This can provide insight to how the design of the program, which is intended to enhance teaching competency, can be improved.

Methods

Sites and subjects

All second-year DNUS students from the academic year (AY) 2009–10 through 2013–14 were asked to complete questionnaire at the end of each of the six core clerkships: Medicine, Obstetrics and Gynecology, Pediatrics, Psychiatry, Neurology and Surgery. We asked students to complete the same questionnaire after every clerkship because each Residency program is implemented differently. The findings would therefore only be representative and balanced if student responses were consistently collected each year.

During each rotation, Residents trained students in a hospital-based setting. Students were embedded in the hospital wards in their clerkship teams. Each team comprised faculty members, students and Residents. The goal of the survey was to discover whether the quality of interaction between Residents and students was changing, in the transition from the House staff model to the Residency model.

Questionnaire development

A 19-item questionnaire was developed by a panel of experts in medical education from DNUS. The first and second domains were derived from the Resident’s clerkship objectives to teach key clinical skills (i.e. interviewing patients and performing physical examinations).

Students were asked to rate the quality (1=poor, 5=excellent) and frequency (1=not at all, 5=very often) of being taught these skills. The third domain was based on the students’ interaction with the Residents, for example engagement with them—helping, encouraging, acting as a role model for them, etc. Students were asked to rate whether they felt Residents exhibited teaching behaviors such as offering encouragement, providing feedback and acting as a role model, etc (1=strongly disagree; 5=strongly agree; see Appendix for full survey).

The questionnaire items were then reviewed for their cultural applicability in the Singapore context by SingHealth Residency’s Designated Institutional Officer and several local faculties.

Distribution

Paper survey forms were distributed to the second-year students after they had completed the National Board of Medical Examiners Clinical Subject Examination at the end of each clerkship. Responses were anonymized.

As this project was initiated in the middle of AY 2009–10, the questionnaire was distributed after three rotations in that AY. We were able to obtain the responses of at least half the class for all six clerkships. Since students were rotated randomly through the six clerkships, we thought it would be beneficial to obtain the responses of at least half the class per clerkship, than none at all.

For the subsequent academic years, the questionnaire was distributed at the end of all clerkships, with the exception of AY 2010–11, where one rotation was excluded due to administrative difficulties in the data collection.

A total of 1208 surveys were administered to 264 students. Table 1 summarizes the number of survey administrations and the corresponding response rates.

Table 1.

Response rate for the questionnaire.

Academic year	Number of students	Number of times the survey is administered	Total number of surveys administered	Number of surveys returned	Percentage (%) returned
2009–10	45	3	135	103	76
2010–11	56	4	224	221	99
2011–12	54	5	270	256	95
2012–13	55	5	275	261	95
2013–14	54	6	304	285	94
Total	264	–	1,208	1,126	93

Note: Students underwent five rotations from the Academic Year (AY) 2009–10 to AY 2012–13, except for AY 2013–14. Students underwent six rotations instead of five because of a curriculum revision in 2013. As this project started in the middle of AY 2009–10, the questionnaire was only administered to three clerkships. For AY 2010–11, one rotation was excluded due to administrative difficulties in the data collection.

Analysis

The ACGME-I residency model in SingHealth was introduced in 2010. As first-year Residents are not expected to be active in teaching, AY 2009–10 and 2010–11 were classified as the pre-initiation Residency stage, while the latter three academic years, AY 2011–12, 2012–13 and 2013–14, were classified as the post-initiation Residency stage. Pre- and post-initiation of Residency data were compared using the Mann–Whitney U test. The domains of quality and frequency of teaching, and engagement were then further compared across different academic years using the Kruskal–Wallis test.

While all analyses were conducted using non-parametric tests, there was no substantive difference with the results of significance tests obtained via parametric tests. Therefore we reported the means of all ratings to aid interpretation. Following the administration of the survey, a Cronbach’s alpha was used to assess the internal consistency of the domains and the total survey. All statistical analyses were performed using SPSS Version 22.

This study was approved by the National University of Singapore Institutional Review Board.

Results

A total of 1126 surveys were returned with an average 93% response rate across the years. Table 1 summarizes the response rate for each academic year.

The Cronbach’s alpha values for the quality, frequency and engagement domains were 0.93, 0.94 and 0.95, respectively. The overall Cronbach’s alpha for the entire questionnaire was 0.97.

Figure 1 shows the mean ratings of each domain across five academic years. This showed an increasing trend across all three domains, with the significantly highest ratings recorded in AY 2013–14.

Figure 1.

Mean student ratings on quality, frequency and engagement domains across five academic years.

We found no significant difference in the ratings between AY 2009–10 and AY 2010–11 (the pre-initiation Residency stage). Similarly, there was little difference in ratings between AY 2011–12 and AY 2012–13 (the post-initiation Residency stage). However, there was significant increase in the student ratings for all three domains from AY 2012–13 to AY 2013–14 (p<0.05).

Overall, the student ratings in the pre-initiation stage of Residency years combined were significantly lower than the ratings of post-initiation Residency years combined. This covered all three domains: quality of teaching (3.02 versus 3.50, p<0.001), frequency of teaching (2.62 versus 3.07, p<0.001) and engagement (3.05 versus 3.58, p<0.001).

We further analyzed the survey at the individual item level, for all three domains. Figures 2 and 3 summarize the comparison of student ratings for quality, frequency of teaching and engagement in the pre- and post-initiation stages of the Residency.

Figure 2.

Mean student ratings on quality and frequency of teaching at pre- and post-initiation of ACGME-I residency program.

Figure 3.

Mean student ratings on engagement at pre- and post-initiation of ACGME-I residency program.

In terms of the quality of teaching, there was a significant increase in student ratings on all items when comparing the pre- and post-initiation stages of the Residency program (p<0.001), except for “performing technical procedures” (p=0.71). Ratings on frequency of teaching saw significant increase across all items in pre- and post-initiation stages (p<0.001 for all and p=0.03 for “performing technical procedures”). There was also significant increase in the ratings on engagement for all items (p<0.001 for all and p=0.002 for “discussing end-of-life issues”).

Limitations

This was a cross-sectional study of different cohorts of students. The changes recorded could be a reflection of difference in perception between cohorts, rather than changing student perception of Residents as teachers. In addition, many other changes within Singapore healthcare system could also have had an impact beyond the Residency itself. Furthermore, this study was restricted to students from one medical school. The survey called for students to generalize their entire experience of a clerkship.

Students might have worked with one or more Residents in their clerkships, some of whom may have been excellent teachers, and others less so, making it difficult for the students to generalize their response, potentially diluting the overall results.

We were unable to identify how many Residents were actually involved in teaching our medical students and how many students the Residents might have encountered during their rotation. It would be more useful had we included the perception of Residents and their views about teaching and their roles over the same time frame. We were, however, limited in our ability to gather that data.

Discussion

Medical students reported an overall increase in the quality and frequency of teaching since the transition to the American Residency model. In comparing specific survey items in the pre- and post-Residency stages, we observed a significant increase in ratings in all areas, except in the teaching of procedural skills. This shows there is some measure of improvement in Residents’ teaching ability, an indication we are moving in the right direction of installing Residents as Future Teachers.

Although the ratings for the quality of teaching in procedural skills did not see a significant increase, the ratings for this aspect of teaching were relatively high in both quality and frequency to begin with. Hence, this could be a skill that was already being regularly taught by Residents in the previous system.

Both quality and frequency showed an increasing trend, although mean frequencies tended to be lower than quality. This might suggest that while Residents are not yet heavily involved in teaching, students appreciate the teaching encounters that they have.

Students also perceived that an improvement in the engagement provided by Residents over the years in all areas, except in how the Residents perceive themselves as a physician. We know that Residents can serve as role models, hence the students’ perception of their impact is important to their professional growth, regardless of the amount of teaching offered by the Residents.

The first big jump in ratings occurred in AY 2011–12. This indicates strong possibility that the first batch of Residents entering their second year had more opportunities to teach and to interact with medical students.

In addition, AY 2011–12 was the first time DNUS students graduated into Residency. Having a cohort of Residents who understood the DNUS program and the struggles our students face in clerkships could also have had an impact on their ratings. _______. Our students may also have sought out friendly alumni for help.

Another jump occurred during post-initiation phase of the Residency, in AY 2013–14. Along with the growing number of Residents in more senior position, who may have more time and confidence in teaching, this change can also be attributed to the introduction of online modules in late AY 2012–13, which are designed to facilitate delivery of the Residents as Future Teachers program.

The Senior Residency and Chief Residency programs implemented in AY 2013–14 could have accounted for the increase in number of Residents involved in teaching. Furthermore, the Academic Medicine Education Institute (AM•EI),¹¹ a collaboration between DNUS and SingHealth to develop Academic Medicine, was established in AY 2012–13. AM•EI plays an important role in developing and delivering programs designed to enhance the learning environment for all learners. These programs are primarily catered towards Residents and faculty.

It has been reported that one-third of medical students’ knowledge and clinical education can be directly attributed to Residents’ teaching.^7,12 The role of Resident differs from faculty because of their close relationship with the medical students during training. Due to the clinical position of Residents, what they teach medical students complements what is taught by faculty.^13,14 This shift towards Residents taking on a teaching role, has, we believe, benefited learners. Medical students view Residents as being highly important to their learning experience, and perceive them as contributing to their knowledge and skills. Students are also likely to acquire professional values and behaviors from them.^15,16

Residents are therefore important role models for medical students.^17–21 However, Residents have to be given the time and training^22–23 to be able to teach. While we know that the student perception of Residents as teachers has changed, we need to understand why this has happened. To do so, we need to explore in greater depth the facilitators and inhibitors to teaching; we need to determine if this shift in perception is because of the student–Resident relationship, or if it is because of changes in Residents’ teaching ability.

Conclusions

The training of Residents in Singapore has undergone significant change since 2010, with the transition from the British House staff model to the American Residency model. The key change was Residents being required to play a major role in the education of medical students. Our hypothesis that residency would create a growing culture of teaching among the residents is backed by our survey findings that show medical students perceived improvement in the frequency and quality of teaching in all areas during this transition.

The results also confirmed the perceived impact Residents have on medical students’ impression of professional development in the field.

It is crucial for medical schools to work closely with the SingHealth Residency program directors to further develop and improve the Residents as Future Teachers training program, to foster effective teaching by Residents.

Future studies can further examine other possible reasons for the improved ratings. For instance, were there changes in the teaching culture for Residents? What were the factors that facilitated the Residents’ ability to teach? Changes in clinical loads, special incentives to teach, and an increase in teaching opportunities can well be factors to consider. Finally, it would be important to pin down what the actual changes were in the specific teaching competencies among Residents.

Footnotes

Appendix

Author Note

A portion of the results in this manuscript was previously presented as posters at the 8^th Asia Pacific Medical Education Conference and the Association for Medical Education in Europe’s Annual Conference 2012.

Declaration of conflicting interests

None declared.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

References

Huggan

Samarasekara

Archuleta

. The successful, rapid transition to a new model of graduate medical education in Singapore. Acad Med 2012; 87: 1268–1273.

Lim

. Residency programmes in singapore-challenges ahead. Ann Acad Med Singapore 2013; 42: 265–266.

Tagore

Min

BCS

Goh

. Specialist training in obstetrics and gynecology in Singapore: Transition to structured residency program. J Grad Med Educ 2012; 4: 272–275.

Khoo

Lahiri

Huggan

. When traditional model meets competencies in Singapore: Beyond conflict resolution. Ann Acad Med Singapore 2014; 43: 544–549.

Wong

Chong

Chng

. Postgraduate family medicine training in Singapore – a new way forward. Ann Acad Med Singapore 2012; 41: 221–226.

Carr

Deal

Dehmer

. Who teaches basic procedural skills: Student experience versus faculty opinion. J Surg Res 2012; 177: 196–200.

Bing-You

Sproul

. Medical students’ perceptions of themselves and residents as teachers. Med Teach 1992; 14: 133–138.

Liaison Committee on Medical Education. 2013. Functions and Structure of a Medical School. http://www.lcme.org/publications/functions2013june.pdf (accessed 14 January 2015).

Mahendran

Broekman

Wong

JCM

. The educational environment: Comparisons of the British and American postgraguate psychiatry training programmes in an Asian setting. Med Teach 2013; 35: 959–961.

10.

Karani

Fromme

Cayea

. How medical students learn from residents in the workplace: A qualitative study. Acad Med 2014; 89; 490–496.

11.

Krishnan

. Academic medicine: Vision to reality. Ann Acad Med Singapore 2013; 42: 2–4.

12.

Barrow

. Medical students’ opinions of the house officer as teacher. J Med Educ 1966; 41: 807–810.

13.

Bordage

. Elaborated knowledge: A key to successful diagnostic thinking. Acad Med 1994; 69: 883–885.

14.

Bordage

Lemieux

. Semantic structures and diagnostic thinking of experts and novices. Acad Med 1991; 66(Suppl): 70–72.

15.

Makama

Ameh

. Quality of teaching provided by surgical residents: An evaluation of the perception of medical students. Niger J Med 2011; 20: 341–344.

16.

Griffith

III Wilson

Haist

. Relationships of how well attending physicians teach to their students’ performances and residency choices. Acad Med 1997; 72(Suppl. 1): 118–120.

17.

Mann

Sutton

Frank

. Twelve tips for preparing residents as teachers. Med Teach 2007; 29: 301–306.

18.

Morrison

Shapiro

Harthill

. Resident doctors’ understanding of their roles as clinical teachers. Med Educ 2005; 39: 137–144.

19.

Remmen

Denekens

Scherpbier

. An evaluation study of the didactic quality of clerkships. Med Educ 2000; 34: 460–464.

20.

Henke

Ailawadi

. Attending, house officer, and medical student perceptions about teaching in the third-year medical school general surgery clerkship. J Am Coll Surg 2004; 199: 932–942.

21.

Whittaker

Jr Estes

Ash

. The value of resident teaching to improve student perceptions of surgery clerkships and surgical career choices. Am J Surg 2006; 191: 320–324.

22.

Hill

Barrow

. A systematic review of resident-as-teacher programmes. Med Educ 2009; 43: 1129–1140.

23.

Wachtel

Greenberg

Smith

. Residents as teachers: Residents’ perceptions before and after receiving instruction in clinical teaching. J Am Osteopath Assoc 2013; 113: 23–33.