A balancing act: The Supervisor of Training role in anaesthesia education

Abstract

In this qualitative study, we report how Supervisors of Training, educational supervisors overseeing the learning of anaesthesia trainees, experience their role in practice. Using purposive sampling, we interviewed Supervisors of Training from across Australia and New Zealand. The interviews began by asking ‘what do you see as your role as a Supervisor of Training?’ then explored the response in detail. Following the technique of thematic analysis, inductive analysis occurred as data were collected until we generated a thematic structure sufficient to address our research question after 19 interviews. In the first three of the four identified themes, Supervisors of Training perceived themselves as the fulcrum of the learning environment, ‘the something in between’. These three themes were: guiding and assessing trainees; identifying, supporting, and adjudicating trainee underperformance; and mediating trainees’ relationship with the hospital. Participants perceived themselves as a broker between trainees, their colleagues, their hospital, the Australian and New Zealand College of Anaesthetists and the community to varying degrees at different times. Negotiating these competing responsibilities required Supervisors of Training to manage multiple different relationships and entailed significant emotional work. Our fourth theme, scarcity, described the imbalance between these demands and the time and resources available. The complexity of the Supervisor of Training role and the tensions between these competing demands is underappreciated. Our findings would support strategies to mitigate the administrative load and share the decision-making burden of the role and to enhance the capability of Supervisors of Training by requiring formal training for the role.

Keywords

Attitude of health personnel qualitative research anesthesiology/education clinical competence curriculum education medical graduate learning

Introduction

All health professionals learn while working in clinical settings and this is true in anaesthesia. Commonly, a clinician is appointed as educational supervisor or lead to oversee the learning of trainees within a clinical department and ensure an appropriate trainee experience. As educational leads, they have a vital role in determining the educational environment in the clinical workplace.¹ In anaesthesia training in Australia and New Zealand, Supervisors of Training fulfil this educational supervisor role. In this study, we explore the views of Supervisors of Training on their role within the context of a large, bi-national anaesthesia training scheme responsible for over 1500 postgraduate anaesthesia trainees.

Educational supervisors within hospital settings such as Supervisors of Training are often appointed because of their interest in teaching; whether this is an adequate qualification for the role is questionable.^1,2 Various studies have highlighted the need for educational supervisors to have appropriate orientation,² protected time,³ leadership development⁴ and mentoring.⁵ Together, these studies suggest there is a risk that educational supervisors may be unprepared and their need for support to fulfil their role may go unrecognised.

Across Australia and New Zealand, there are currently 353 Supervisors of Training appointed by the Australian and New Zealand College of Anaesthetists (ANZCA) in 160 departments accredited for anaesthesia training. ANZCA accords them overall responsibility for anaesthesia training in their hospitals, although no educational credentials are required.⁶ They are expected to provide educational supervision for all trainees, ongoing training to their colleagues in teaching and assessment and to liaise with department leaders and hospitals to ensure there is adequate support for training.⁶ With the evolution of anaesthesia education, there has been an increase in the curricular requirements for trainees and hospitals, resulting in increasing demands upon educational supervisors. Although the ANZCA website⁶ describes what should happen in the Supervisor of Training role, summarised in Table 1, we do not know how this plays out in practice in the workplace. The difference between ‘work as imagined’ and ‘work as done’⁷ has been recognised as an important source of information for improving practice. Hence, our research question for this study is how do educational supervisors experience their role in practice? Illuminating these experiences will allow us to consider how best to optimise this critical role and thereby improve the quality of anaesthesia training.

Table 1:

Supervisor of Training responsibilities.

Role	Specific duties	Areas requiring Supervisor of Training oversight
Administrative	Appoint consultants to lead teaching in various aspects of the curriculum. Record all aspects of training in the Trainee Portfolio System.	Orientation of trainees. Monitoring of consultant staffing and workload changes that may impact on training. Monitoring availability of cases and procedures in their hospital and advise trainees on how to meet curriculum requirements. Advising current and potential trainees on training, registration, fees, exam dates and courses.
Trainee supervision and management	Undertake the in-training assessment process for trainees in the department, including reviews at the start and end of each placement. Make decisions on progression of trainees through stages of training. Verify completion of curriculum requirements by trainees. Assess and manage trainees experiencing difficulty, including managing and adjudicating the success of remediation.	Development of clinical placement plans by trainees. Oversee the training experience and achievements of each trainee within the department and work with other supervisors and tutors to ensure requirements are met.
Education	Approve courses attended by trainees as being appropriate for ANZCA training. Ensure trainee access to examination courses and leave. Participate in education courses and Supervisor of Training meetings in the training region. Access ANZCA training resources for Supervisors of Training.	Provision of tutorial programmes in their department. Education of WBA assessors, including ongoing training of new provisional Fellows and consultants within their department.
Relationship with Head of Department	Advise Head of Department regarding trainee duties, required supervision levels, rest and study time.

ANZCA: Australian and New Zealand College of Anaesthetists; WBA: Work place-based Assessment.Source: reproduced with permission from the Australian and New Zealand College of Anaesthetists.⁶

Methods

In this study, we have used data from interviews with Supervisors of Training collected as part of a larger project exploring assessment in anaesthesia education. We gained ethics approval from the Monash University Human Research Ethics Committee (reference: 2016000919) and the University of Auckland Human Participants Ethics Committee (reference: 017408).

To capture diverse views, we sampled purposefully, aiming for variation in participant characteristics that we anticipated might influence responses.⁸ We considered gender, geographic spread, rurality, experience and hospital size. ANZCA staff approached potential participants on our behalf to preserve their privacy.

The interviews started with a single broad question: what do you see as your role as a Supervisor of Training? We followed this open question with unscripted probes to extend the initial response and further elucidate participants’ understanding of their role.⁹ After this phase of the interview was complete we then moved on to questions that examined the assessment process to address our other research question; however, we also examined these responses for their elucidation of the Supervisor of Training role. Each interview was performed by a single interviewer, either the first author or the research assistant. Interviews were transcribed verbatim and ranged from 46 to 70 min long.

We used thematic analysis as described by Braun and Clark.^10,11 This methodology includes phases of familiarisation with the data, descriptive coding, generating themes and developing a thematic structure. Coding was facilitated using NVivo software (version 11, QSR International Pty Ltd, Melbourne, Australia.) The first author led the analysis but all authors contributed, allowing multiple perspectives to inform the findings. We continued interviewing until we judged the thematic structure sufficient to address the research questions. Themes related to the Supervisor of Training role were then separated to form the basis of this paper and further refined in discussions amongst the research team.

In qualitative research, researchers co-create the data with participants and its interpretation is informed by their unique perspectives.¹² As an ANZCA Supervisor of Training, the first author is an educational supervisor and hence has ‘insider’ status, which aided the development of rapport in the interviews but also sacrificed the potential advantage of naiveté.⁹ We managed the impact of this insider status on our data by having a research assistant with a non-medical background also conduct some of the interviews. Similarly, an insider perspective in analysis is recognised as valuable in ensuring findings are persuasive to the community from which participants are drawn.¹³ The diverse perspectives of the research team with their varied backgrounds in medical education and their constant involvement in the analysis were important in balancing the first author’s personal perspective and ensuring alignment between participants' views and our subsequent interpretations.

Results

We ceased interviewing after 19 interviews, when we judged the thematic structure sufficient to address the research questions, as described by Braun and Clark.¹⁰ We interviewed eight male and 11 female participants and the median experience in the role was four years (range one to 11 years). Four participants were from New Zealand, with at least one from each Australian state and territory. Four participants came from each of small metropolitan, large regional and small regional hospitals, whereas seven participants were from large metropolitan hospitals.

We identified four themes in our analysis of the Supervisor of Training role. In the first three themes, the Supervisors of Training described themselves as ‘the something in between’, balancing competing interests. We observed that participants perceived themselves as the fulcrum of the learning environment in their workplace. In this shared view of their role, they reported facilitating the key relationships between all those involved in training. They used many similar expressions such as ‘lynchpin’, ‘interface’, ‘go-to person’, or ‘go-between’ to describe this fundamental conception of their role. In maintaining their place as the fulcrum of the learning environment, we observed that Supervisors of Training needed to make constant adjustments and accommodations to balance competing interests and responsibilities; they perceived themselves to be representing trainees, colleagues, their hospital, ANZCA and the community to varying degrees at different times. Negotiating these competing responsibilities required managing multiple different relationships and entailed a significant amount of emotional work. Our fourth theme described the imbalance between the demands of these first three themes and the preparation and resources available to our participants.

The four themes we identified were:

Guiding and assessing trainees.

Identifying, supporting and adjudicating trainee underperformance.

Mediating trainees’ relationship with the hospital.

Scarcity: inadequate orientation, support and time.

1. Guiding and assessing trainees

The Supervisors of Training’s interaction with trainees entailed two potentially discordant functions: guidance and assessment. For the most part, Supervisors of Training saw themselves primarily as ‘a guide, assisting trainees to understand what they need to do and helping them navigate ANZCA training’ (Participant 3). Guiding trainees was seen as a continuous process that followed a rhythm of planning, interim and feedback interviews with trainees across their placement that provided the scaffolding for this assistance. There was a particular focus among Supervisors of Training on keeping trainees aware of the ANZCA training requirements and the need to record these in their electronic portfolio.

For Supervisors of Training, the electronic portfolio required ‘a very large amount of documentation and administrative stuff’, which was seen as necessary to ensure a ‘robust and clearly documented training scheme’ (Participant 6). However, this entailed a significant opportunity cost in that it decreased the time available for meaningful interaction with trainees: ‘I’m spending a significant amount of time in front of the computer going through paperwork’ (Participant 17). At the extreme, this created the perception the Supervisor of Training had become ‘essentially a bureaucrat: tick the boxes, make sure the boxes continue being ticked’ (Participant 1).

Despite the prominence of routine bureaucratic duties, Supervisors of Training reported their guidance extended beyond assistance with meeting administrative requirements. They saw themselves as integral to supporting trainees’ self-directed learning:

‘Helping them to develop skills in order to be competent anaesthetists at the end and guiding them to do that. Where they have weaknesses, just helping them to identify what those are and providing them with opportunities to assist them to improve in those areas.’ (Participant 3)

The guidance aspect of the role for some participants involved a pastoral component, providing ‘a port of call for welfare issues’ (Participant 11). Participants also reported providing career advice that gave a future orientation to their support for trainees, where they aimed to:

‘Look after a trainee through their journey, not just through their rotation with us.’ (Participant 16)

A competing yet equally fundamental aspect of the Supervisor of Training role for all participants was assessing trainee performance. Our participants saw themselves as collectively responsible for ensuring graduating specialist anaesthetists were appropriately qualified for their work. This collective responsibility was expressed both as a duty to the profession and the community, for example:

‘I have a responsibility to the public to ensure that (trainees) are of sufficient standard that they can do their job safely.’ (Participant 3)

Supervisors of Training recognised the conflict inherent in their dual responsibility for both assessing and guiding trainee learning:

‘We try and separate out the assessment role from the supportive role, but you end up doing both of course.’ (Participant 12)

Some participants thought it was inevitable that they would be seen as ‘gatekeepers’ (Participant 19) once a trainee’s performance was in question, whereas others were reluctant to be the ‘policeman’ (Participant 3). One way of managing this conflict reported by experienced Supervisors of Training was to ‘get the trainee to see that we care about your training’ (Participant 16) and emphasise their role in helping the trainee deal with questions about their performance ‘because we want you to improve, to become a better anaesthetist’ (Participant 16).

2. Identifying, supporting and adjudicating trainee underperformance

Participants reported that monitoring potential trainee underperformance was an important part of their role. Sometimes this involved the Supervisor of Training actively seeking reassurance that performance was adequate:

‘If your trainee is struggling, if you're a Supervisor of Training who is on their game you should be doing it (monitoring performance) as an ongoing process.’ (Participant 11)

Other times, the Supervisor of Training’s monitoring was more passive and reliant on colleagues or other staff flagging potential issues:

‘People will come and have a little chat and just let you know they have some concerns.’ (Participant 8)

The aim of this monitoring was generally reported as facilitating early remediation, to

‘Get the trainees as early as possible and using as many resources as we can.’ (Participant 16)

Participants also saw their Supervisor of Training role included responsibility for supporting trainees during remediation after they had detected underperformance. They described assisting trainees to address identified issues in various ways. In the case of one male trainee, for example, the Supervisor of Training reported they aimed to:

‘Give him the tools and help him out and roster him with people who’d give him a fair go.’ (Participant 15)

Some participants found the multifactorial and idiosyncratic nature of the issues that constitute poor performance a source of difficulty in developing expertise in remediation:

‘It is a very complex process … all the steps are different for different trainees and for different circumstances.’ (Participant 3)

Some participants reported their role also extended to monitoring and judging the success of remediation efforts:

‘It’s a matter of trying a number of different solutions depending on what the issue is and then adjusting them accordingly if you aren’t able to get an improvement.’ (Participant 3)

There was concern among participants that they could be accused of bullying when managing trainee performance issues and this influenced their interactions with trainees. One participant reported this had indeed occurred:

‘He’d had a go at us earlier saying we were bullying him when we told him that he needed to pull his socks up, so I guess we were a little bit in fear of what might happen. Yes, he’d written to our HR and our Head of Department about how there was a bullying culture amongst the Supervisors of Training when we told him that turning up late and not being prepared wasn’t good enough.’ (Participant 15)

If remediation ultimately failed then the stakes in decision-making were raised. Although recognising the ultimate decision to exit a trainee from the training programme was subject to official ANZCA processes, participants were conscious these processes were only initiated after their decision that remediation had been unsuccessful. Their role as sole decision-maker in these circumstances meant they were very reluctant to make this decision, even more so than when they reported the initial decision to hold the trainee back for additional clinical experience:

‘When you have had to intervene at multiple levels with no improvement, those decisions should not be left to one individual.’ (Participant 3)

Given the obvious consequences for the trainees involved, making adverse decisions on progression in this context was, perhaps unsurprisingly, stressful and seen as a considerable burden. Our participants generally managed this burden by engaging a continuum of support to share the responsibility for these decisions even though they were officially solely responsible.

‘I do find it personally stressful doing that, and so I try to share the burden.’ (Participant 12)

Our participants mentioned multiple other people as potential candidates to share this perceived burden. They relied on Clinical Directors, ANZCA state or national Education Officers, or their own mentors for support, whereas in larger centres with multiple Supervisors of Training they might come to a collective judgement:

‘We will decide together – the group of Supervisors of Training – whether it will be a yes, a borderline or a no.’ (Participant 14)

3. Mediating trainees’ relationship with the hospital

Participants described a role in mediating the relationship between the trainee and the hospital. This mediation sometimes included clarifying for trainees the expectations of the department for trainee work practices and ways in which trainees might contribute. More generally, participants reported acting as ‘an advocate for trainees for the quality of their training within the department’ (Participant 10). This involved representations on behalf of trainees to the department leadership to ensure appropriate rostering, support for exam preparation or access to clinical learning opportunities.

‘I want them to see me as someone who is a kind of a “go-between” between themselves and … roster makers or the director or whatever. I like them to think that I am the person they come to whenever they have a grievance.’ (Participant 7)

Supervisors of Training also discussed their role as an intermediary between the consultants and trainees. In this role, they coordinated formal teaching and encouraged consultants to fulfil their part in meeting trainees’ training requirements:

‘Acting as a liaison between consultants and trainees, in terms of things that we need to get done, including work place-based assessments, and providing education for the consultants to provide effective training for the registrars.’ (Participant 16)

Supervisors of Training also reported they sometimes needed to resolve interpersonal conflicts between individual trainees and consultants:

‘The real, the tricky work, is the problem solving … typically consultant-trainee personality clash. I have got to pick up the pieces and solve it.’ (Participant 4)

Another aspect of this intermediary role arose because some consultants were unwilling to provide critical feedback directly to trainees or in the workplace-based assessments, preferring instead to convey this information via the Supervisor of Training:

‘Sometimes a consultant might think negative things about the registrar, but they don’t bring them up with them, they just sort of feed back to us.’ (Participant 14)

On one occasion, a participant noted this delayed and compromised their attempt to provide remediation:

‘We had all this time to tell her, and nobody did … it would have been easier if someone had had the backbone to tell her before then, rather than leave it for me to do, for me to be the bad guy.’ (Participant 17)

4. Scarcity: inadequate orientation, support and time

Although the themes above illustrate multiple challenges in the Supervisor of Training role, we found this was not balanced by a commensurate preparation, support, or allocation of time. Some participants reported they knew little about the role before they started. The general perception was that Supervisors of Training are often unprepared for the job, for example:

‘Literally I was thrown into it and asked “Will you be the Supervisor of Training and if not then why not?!!” (laughing). I thought I would give it a shot and see what happens.’ (Participant 1)

Supervisors of Training in smaller centres, in particular, reported a lack of orientation to their role and felt they were at risk of isolation, with one participant resigning in frustration after 12 months:

‘It was a bit of a struggle actually to get your head around it … I didn’t have a proper handover… I really struggled to find out exactly what my role actually was.’ (Participant 2)

Support from within the department, both from colleagues and the Clinical Director, was important to participants. The ANZCA state or national Education Officer was also a source of support outside the local department. Larger departments often had multiple Supervisors of Training and this allowed for more orientation and support. Sitting in with an experienced Supervisor of Training and receiving mentoring from and mixing with other Supervisors of Training were seen to be important support strategies:

‘I feel for the people who don’t have that formal mentorship; I think it’s a useful tool for brainstorming, for getting advice, for having someone to sit in a conference with you with a trainee.’ (Participant 16)

Participants reported they were time poor and very busy, especially with the administrative requirements and routine performance management meetings.

‘Over time it has become less enjoyable. It takes up a lot of time and I find I do a lot of what I feel is secretarial work.’ (Participant 6)

Participants also reported other aspects of the role that can be time-intensive. Investigating potential poor performance, in particular, resulted in a significantly increased workload. Assisting trainees with poor performance or other difficulties, although acknowledged as a valuable service to the trainee and the community, was reported to take up large amounts of time and mental energy on top of an already full schedule. Participants also liaised with Clinical Directors and roster coordinators to enhance trainee learning and welfare. Some participants reported they have other tasks that extend from their Supervisor of Training role in their particular context, such as organising mentors for trainees, appointing trainees, organising tutorials and exam practice, or simulation sessions.

Discussion

The picture of Supervisors of Training our participants paint is much more complex than they thought it would be when they took on the role and more nuanced than the official guidance available to prospective or incumbent ANZCA Supervisors of Training. In our analysis, the Supervisors of Training found themselves central to the learning environment, balancing competing responsibilities and simultaneously maintaining relationships with multiple stakeholders with different interests to pave the way for trainee learning and progression (or otherwise). However, although the official role descriptions, or the ‘work as imagined’, outline the key tasks required, they do not capture the complexity of the ‘work as done’ described by our participants.⁷ Specifically, they do not explore the inherent contention between different aspects of the role or the divergent interests of the various stakeholders in the learning environment. There is a lack of guidance on the intersection of influences on trainee learning, how to best reconcile competing priorities, or which priority might need to be emphasised in a particular circumstance.

Our participants noted that the administrative requirements of their role were significant and had increased over time. They generally accepted that innovations in education and assessment aimed at improving the quality of training and ultimately the outcomes of care were a step forward. However, they identified that the subsequent bureaucratic burden lessened the time available for direct trainee contact, a significant opportunity cost that also detracted from their enjoyment of the role. Authors in other contexts have also reported that educational innovations have resulted in an underappreciated increased administrative requirement necessitating additional time and effort from educational leaders.^14,15 We think this increased education-based administration, with its adverse impact on educational supervisors, deserves greater recognition in the design and implementation of future educational innovations in the workplace. Within our participants’ context, we think ANZCA ought to advocate for increased time and administrative support for Supervisors of Training while seeking to simplify the administrative requirements where possible.

Our findings also emphasise the heavy responsibility Supervisors of Training carry for maintaining the standards of professional practice and protecting the community. The effort our participants go to in order to distribute this burden among their colleagues or others conveys an important message. We consider this sharing of responsibility an adaptive response to the overly arduous situation they find themselves in. Elsewhere in the medical education world, collective processes for decision-making on trainee progression have been formalised into clinical competency committees, which reflects the increasing recognition of the importance of performance assessment in the workplace and the shift to more programmatic approaches to assessment.^16,17 The institution of formalised group decision-making in ANZCA would also serve to ameliorate the burden on individual Supervisors of Training. In our local context, we would recommend ANZCA attend to the evolution of competency committees and explore how group decision-making might work in ANZCA training sites.

Our participants’ responses highlight the pivotal role educational supervisors such as Supervisors of Training have within the learning environment, a point noted by others.^1,2 This starkly contrasts with the lack of requirements for educational qualification or preparatory training before assuming the educational supervisor role,⁶ a situation that is not unique to ANZCA.¹ Commencing the role without any formal preparation seems less than ideal, given the complexity involved in supporting and assessing trainees, detecting and managing underperformance, and mediating relationships amongst multiple stakeholders. The data speak of the consequent strain for Supervisors of Training in the clinical environment, suggesting that Supervisors of Training ought to have an understanding of workplace-based learning and performance assessment and skills such as coaching learners and delivering bad news. This preparation could be provided either through formal qualifications in medical education or other practice-oriented training. According to the Australian Medical Council, ‘The education provider selects supervisors who have demonstrated appropriate capability for this role. It facilitates the training, support and professional development of supervisors’.¹⁸ Educational supervisor training and professional development is a responsibility of postgraduate specialist medical education providers such as ANZCA and our findings highlight the need for greater attention to this requirement.

Regehr has called for educational research to generate ‘rich understandings of the complex environments in which our collective problems are uniquely embedded’.¹⁹ This study contributes to our understanding of the broader collective problem of the challenges faced by those overseeing postgraduate specialist training in clinical departments^1–5 by specifically examining a particular context, the Supervisor of Training role in anaesthesia training in Australia and New Zealand. The extent to which the experiences of participants in this study mirror those in similar positions in other specialties or countries will vary. However, we suggest that many of the insights generated are related to the universal nature of educational supervision in clinical workplaces and will be of value to others when they contemplate the role of the educational supervisor in their own context.

Conclusion

Our main finding is that the Supervisor of Training role is the fulcrum of the learning environment and requires balancing competing responsibilities and the needs of diverse stakeholders. Based on the reports of our participants, the complexity of the Supervisor of Training role is not represented in the official descriptions of the role, and is unappreciated by novice Supervisors of Training. Formalising collective decision-making, using a mechanism such as a clinical competency committee, would go some way to meeting the needs of Supervisors of Training to share the burden of making progression decisions. Requiring all Supervisors of Training to have an appropriate educational qualification or specific training ought to help them manage the scope and complexity of their role in ensuring trainees receive high-quality anaesthesia education.

Footnotes

Acknowledgements

We would like to thank the Supervisors of Training who participated in this study and the ANZCA Clinical Trials Network for their assistance in recruitment.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: DC and JW hold voluntary positions on ANZCA Education Committees. Otherwise, all authors report no declarations of interest in this work.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study was supported by an untied grant from the ANZCA Research Foundation (Grant No: S16/043).

References

Malling

Scherpbier

Ringsted

What is the role of the consultant responsible for postgraduate education in the clinical department?

Med Teach 2007; 29: 471–477.

Haan

Zenni

West

, et al. Graduate medical education leadership development curriculum for program directors. J Grad Med Educ 2011; 3: 232–235.

Fleming

Brook

Herman

, et al. Recommended protected time for pediatric fellowship program directors: A needs assessment survey. Acad Pediatr 2016; 16: 415–418.

Bing-You

Wiltshire

Skolfield

Leadership development for program directors.

J Grad Med Educ 2010; 2: 502–504.

Norby

Karniski

Schmidt

, et al. Mentoring for subspecialty training program directors: An unrecognized, unmet need? J Grad Med Educ 2010; 2: 206–209.

Australian and New Zealand College of Anaesthetists. Handbook for Training v 1.9. Melbourne: Australian and New Zealand College of Anaesthetists, 2017.

Braithwaite

Changing how we think about healthcare improvement.

BMJ 2018; 361: k2014.

Patton

MQ.

Qualitative research & evaluation methods: Integrating theory and practice. 4th ed. Thousand Oaks, CA: SAGE, 2015.

Kvale

Brinkmann

InterViews: Learning the craft of qualitative research interviewing. 2nd ed. Thousand Oaks, CA: SAGE, 2009.

10.

Braun

Clarke

Terry

Thematic Analysis. In: Rohleder

Lyons

, (eds.). Qualitative Research in Clinical and Health Psychology. New York: Palgrave Macmillan, 2015, pp. 95–114.

11.

Braun

Clarke

Using thematic analysis in psychology.

Qualitative Research in Psychology. 2006; 3: 77–101.

12.

Koro-Ljungberg

A Social Constructionist Framing of the Research Interview. In: Holstein

Gubrium

, (eds.). Handbook of Constructionist Research. New York: The Guildford Press, 2008.

13.

Lincoln

Guba

EG.

The Constructivist Credo. Walnut Creek, CA: Left Coast Press, 2013.

14.

Dauphinee

Boulet

Norcini

JJ.

Considerations that will determine if competency-based assessment is a sustainable innovation. Adv Health Sci Educ Theory Pract 2019; 24: 413--421.

15.

Ellaway

Mackay

Lee

, et al. The impact of a national competency-based medical education initiative in family medicine. Acad Med 2018; 93: 1850--1857.

16.

Hauer

Chesluk

Iobst

, et al. Reviewing residents' competence: A qualitative study of the role of clinical competency committees in performance assessment. Acad Med 2015; 90: 1084–1092.

17.

Harris

Bhanji

Topps

, et al. Evolving concepts of assessment in a competency-based world. Med Teach 2017; 39: 603–608.

18.

Australian Medical Council. Standards for Assessment and Accreditation of Specialist Medical Programs and Professional Development Programs by the Australian Medical Council. Kingston, ACT: Australian Medical Council Limited, 2015.

19.

Regehr

It's NOT rocket science: Rethinking our metaphors for research in health professions education.

Med Educ 2010; 44: 31–39.