Abstract
Competency-based medical education has evolved over the past decades to include the Accreditation Council for Graduate Medical Education Accreditation System of resident evaluation based on the Milestones project. Entrustable professional activities represent another means to determine learner proficiency and evaluate educational outcomes in the workplace and training environment. The objective of this project was to develop entrustable professional activities for pathology graduate medical education encompassing primary anatomic and clinical pathology residency training. The Graduate Medical Education Committee of the College of American Pathologists met over the course of 2 years to identify and define entrustable professional activities for pathology graduate medical education. Nineteen entrustable professional activities were developed, including 7 for anatomic pathology, 4 for clinical pathology, and 8 that apply to both disciplines with 5 of these concerning laboratory management. The content defined for each entrustable professional activity includes the entrustable professional activity title, a description of the knowledge and skills required for competent performance, mapping to relevant Accreditation Council for Graduate Medical Education Milestone subcompetencies, and general assessment methods. Many critical activities that define the practice of pathology fit well within the entrustable professional activity model. The entrustable professional activities outlined by the Graduate Medical Education Committee are meant to provide an initial framework for the development of entrustable professional activity–related assessment and curricular tools for pathology residency training.
Keywords
Introduction
Competency-based medical education (CBME) has emerged throughout the world as the driving concept behind innovations in curricula for medical training spanning undergraduate to graduate medical education. The International CBME Collaborators define CBME as “an outcomes-based approach to the design, implementation, assessment, and evaluation of medical education programs, using an organizing framework of competencies.” 1(p.641) Unlike previous objective-driven education models, CBME places the emphasis for curriculum development and assessment on the desired outcome of the learner instead of the goals and objectives of the educational process. Competency-based medical education also melds concepts from various educational frameworks such as the “does” component of Miller’s pyramid 2 (Figure 1) for the assessment and the evaluation of learner progression in conjunction with adaptation of teaching style of the Dreyfus model (Figure 2). 3
Miller’s framework for clinical assessment (Miller’s pyramid). Adapted from Miller, 1990. 2
Dreyfus stages of learning. Adapted from Holmboe, Hawkins, Durning, 2008. 3
In the United States, CBME has evolved over the past decades to include the current system of formal resident evaluation required by the Accreditation Council for Graduate Medical Education (ACGME) in the form of outcome-based Milestones. The domains of competence employed in graduate medical education in the United States were first introduced by the ACGME and American Board of Medical Specialties in 1999 as part of the Outcomes Project and include the domains of patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. With this original requirement to educate residents based on competency domains, the ACGME initially relied on the education community to innovate and develop methods to teach using this competency-based framework. 4 Organizations, including the Institute of Medicine and the Medicare Payment Advisory Commission, felt that improvement in resident education based on this initial effort was insufficient and encouraged further innovation within the ACGME. 5 With the phased implementation of the Next Accreditation System (NAS) beginning in 2013, the ACGME sought to accredit programs “based on outcome” in the hopes of better preparing residents for practice and reducing the burden of the previous administrative structure. The NAS assigns the responsibility for maintaining a clinical learning environment to the designated institutional official at the training site, with clinical learning environment review inspections of the sponsoring institution scheduled approximately every 18 months. Instead of on-site individual program inspections every 1 to 5 years, programs are now assessed on the basis of biannual evaluation of resident progress on specialty-specific “Milestones,” annual scrutiny of ACGME faculty and resident survey data, and review of resident case logs. Self-study and program inspection site visits occur less frequently in the NAS, with intervals of up to 10 years between visits. 6 The development of specialty-specific “Milestones” was a significant step forward in advancing CBME in graduate medical education in the United States, but the 27 subcompetencies or “Milestones” in Pathology are not intended to serve as a comprehensive curriculum. Rather, they were developed as a broad view of the expected progression of trainees in a variety of domains. The subcompetencies and Milestones have specific limitations in a discipline like pathology, where trainees traditionally gain knowledge, skills, attitudes, and behaviors discontinuously in the many distinct rotation subdisciplines that comprise both anatomic pathology and clinical pathology. For example, the Milestone subcompetencies contain several assessments that rate a trainee across all of anatomic pathology or all of clinical pathology, when the trainee may have a relative strength in transfusion medicine and a relative weakness in clinical chemistry or a relative strength in surgical pathology and a relative weakness in cytopathology. The Milestone subcompetencies therefore leave room for more focused assessments of the progress of pathology trainees as they earn the trust of their instructors to accept the responsibility for independent practice in specific areas.
Other competency-based assessment models have been developed independent of the ACGME Milestones framework. Entrustable professional activities (EPAs) were first proposed by ten Cate in 2005 and represent an additional way to implement an outcome-based model of acquisition of observable knowledge, skills, and attitudes by a learner. 7 Per ten Cate, EPAs “are units of professional practice, defined as tasks or responsibilities to be entrusted to the unsupervised execution by a trainee once he or she has attained sufficient specific competence.” 8(p.157) Furthermore, EPAs should be activities that define the practice of a specialty. Specific attributes of EPAs as defined by ten Cate (Table 1) also require that they have a recognizable output and be executed within a specific time frame with observation and measurement of both the process and the outcome. 7 Based on a resident’s demonstrated level of competence, differing levels of supervision can be entrusted (Table 2), with the highest levels of entrustment allowing for distant or post hoc supervision and the ability of the resident to provide supervision to more junior trainees. 8
ten Cate’s Attributes of an Entrustable Professional Activity (EPA). 7
ten Cate’s Five Levels of Supervision for the Entrustable Professional Activity (EPA) Framework. 8
In the EPA model, EPAs can be mapped back to relevant individual subcompetencies of the ACGME Milestones framework, often spanning multiple core competencies, with proficiency across multiple subcompetencies required for acceptable performance of an EPA. Less complicated than the Milestones-based framework, EPA-based assessment tools should in theory be easier to implement in a clinical training setting. Entrustable professional activities rely on repetitive units of work familiar to physicians from their daily clinical activities. Therefore, the language and descriptions used in assessment tools can be less complex, avoiding cognitive overload. 9 In addition to serving as a tool for assessment and assignment of the appropriate level of supervision for individual residents, EPAs and other CBME-based tools may also eventually allow for a major shift in the structure of training programs. For instance, by developing well-defined performance levels of expected outcomes at the completion of training, programs may be able to transition from a time-dependent to outcome-dependent model tailored to the pace of achievement of the individual learner. 7
Much of the literature to date on EPAs has served to delineate the concept and establish EPAs for various specialties, including internal medicine, 10,11 family medicine, 12,13 pediatrics, 14 developmental-behavioral pediatrics, 15 geriatrics, 16 pulmonary and critical care, 17 hematology/oncology, 18 gastroenterology, 19 and radiology, 20 in the United States and abroad, and for undergraduate medical education. 21,22 In their recent publication in Human Pathology, Powell and Wallschlaeger provide examples of possible EPAs for pathology, give a detailed description of performance of an autopsy as an example of an EPA, and advocate for the development of EPAs for pathology on the national level. 23 To this end, the Graduate Medical Education Committee (GMEC) goal was to develop practical EPAs for pathology graduate medical education encompassing primary anatomic and clinical pathology residency training.
Materials and Methods
The GMEC met over approximately 2 years to explore the concepts set forth in CBME and to develop a preliminary list of EPAs for resident training in pathology. The GMEC’s charge includes identifying the impact of trends in medical education on the ability to effectively recruit and train pathologists throughout the continuum of medical education and to facilitate the exchange of information, tools, and resources across pathology training programs; therefore, delineation of EPAs for pathology training aligns well with the goals of the committee. As background, the GMEC consists of a cross section of the pathology community, including members from academic, private, and military practice environments, with expertise in many subspecialty areas of anatomic and clinical pathology, including neuropathology, hematopathology, pediatric pathology, autopsy pathology, gynecologic pathology, gastrointestinal pathology, transfusion medicine, and medical microbiology. Graduate Medical Education Committee membership also represents a variety of departmental and institutional administrative positions including residency program and fellowship directors, directors of undergraduate medical education, designated institutional officials, and department chairs. A resident member is also included in the committee. Several members of the working group who participated in drafting the Milestones for Pathology (SZP, MDP, MDB, RDH) were members of the GMEC during the EPA project.
The GMEC initially examined the feasibility of developing pathology-specific CBME content for residency training in November 2014 and, based on a literature review, narrowed its focus to EPAs in March 2015. After a subset of committee members drafted an initial list of EPAs for pathology, the full committee evaluated and refined this preliminary list. Members were assigned to develop content related to each EPA based on the “guidelines for full EPAs descriptions” and the “components of a fully described EPA” described by ten Cate and colleagues and also based on the example by Caverzagie et al in their EPAs for internal medicine. 8,10,24 Entrustable professional activities were then edited and refined in meetings in April, August, and November 2016.
Results
The GMEC identified and developed 19 EPAs for anatomic and clinical pathology training. Of these, 7 are within the scope of anatomic pathology practice, 4 are within the scope of clinical pathology practice, and 8 are generalizable to both disciplines, including 5 EPAs related to laboratory management. The content defined for each EPA includes: the EPA title; a description of the EPA, including required knowledge and skills needed for competent EPA performance; mapping of the EPA to relevant ACGME Milestone subcompetencies; possible assessment methods for evaluation of EPA performance.
The EPA titles are listed in Table 3, while the remainder of the content is included in a supplement to this article. Due to the individualized nature of pathology residency program curricula and the many ways in which EPAs could be used, the committee did not include correlation of supervision levels with specific stages of training or expiration of entrustment in the absence of observation, as these seemed most appropriately defined at the individual program level.
Entrustable Professional Activities (EPAs) for Pathology Graduate Medical Education.
Abbreviations: AP, anatomic pathology; CP, clinical pathology.
The anatomic pathology-specific EPAs related to surgical pathology include gross dissection of specimens, performance of intraoperative consultations and frozen sections, and generation of surgical pathology reports. Entrustable professional activities in cytology include performance of fine needle aspiration (FNA) procedures, performance of adequacy assessments, and composition of cytology reports. Finally, performance of the medical autopsy is also included. For clinical pathology-specific EPAs, tasks addressed include generation of interpretative reports for clinical laboratory testing (eg, flow cytometry immunophenotyping, antibody identification, body fluid analysis, peripheral blood and bone marrow diagnosis, etc). Management of patient care issues including resulting critical values and managing adverse transfusion reactions is described. A general EPA for “other” procedures is included that can be adapted to address bone marrow aspiration and biopsy or apheresis, among others. Entrustable professional activities common to both anatomic and clinical pathology cover guidance of preanalytical testing, support of interdisciplinary conferences, and provision of patient care consultations. Finally, EPAs encompassing laboratory management-specific tasks address test utilization, quality and safety, test or instrument evaluation and implementation, and laboratory accreditation inspections. The committee also identified obtaining informed consent as a critical EPA; however, this EPA has already been defined by the Association of American Medical Colleges (AAMC) in their Core EPAs for Entering Residency. 22
An example of the full content of 1 EPA that encompasses both anatomic and clinical pathology training, “Provide patient care consultations,” is given in Table 4. The description of the EPA and tasks involved in appropriately performing the activity is defined in the first section. For this EPA, a resident is expected to provide a verbal or written consultation in response to a patient care–related clinical provider inquiry. The knowledge and skills required to effectively complete this activity include appropriately defining the clinical question, obtaining background information including patient history, identifying relevant medical knowledge and laboratory procedural information, communicating results with adequate documentation of such, handing off of information, and following up on ongoing patient care and laboratory issues. This detailed definition of the activity to be performed provides novice and advanced learners alike with a clear outline of expectations to successfully complete the consultation. It also gives faculty distinct and concrete steps for evaluation and feedback for this particular activity as opposed to simply viewing the task as a whole. The second section of Table 4 relates the EPA back to the ACGME Milestones framework. This EPA includes subcompetencies from across the 6 ACGME core competencies and demonstrates how EPAs pull together knowledge and skills across the different domains of competency. The last section of Table 4 provides general ideas for assessment methods for the EPA. In the consultation example, direct observation of the resident, 360° evaluations from laboratory staff, review of on-call activities, or consultation documentation in a portfolio are all potential methods for assessment of the EPA.
Entrustable Professional Activity (EPA): Provide Patient Care Consultations (AP/CP).
Abbreviations: AP, anatomic pathology; CP, clinical pathology; PC, patient care; MK, medical knowledge; SBP, systems-based practice; PBLI, practice-based learning and improvement; PROF, professionalism; ICS, interpersonal and communication skills.
Discussion
The EPAs outlined by the GMEC are meant to provide an initial framework for the further development of EPA-related assessment and curricular tools for pathology residency training. Interestingly, many of the subcompetencies identified in the Pathology Milestones Project conceptually align with the EPA construct and are similar in content to EPAs identified by the GMEC (Table 5). 25 This overlap in scope between ACGME Pathology Milestones (or subcompetencies) versus EPAs does contribute to confusion surrounding the 2 frameworks. In general, core competencies refer to observable skills or abilities of an individual, while EPAs refer to a professional task that often requires skills from across a range of competencies. 10 The Milestones and EPAs differ in several important ways. Whereas Milestones are deliberately intended to serve as a fixed reference for measuring the progress of trainees in all programs of a discipline, EPAs are flexible and can be adapted to the specific needs of individual programs. Whereas Milestones subcompetencies are assigned to uniquely emphasize 1 core competency, EPAs are able to cross-reference multiple Milestones and therefore multiple core competencies. Figure 3 provides a visual example of how a specific EPA, “Provide patient care consultations” could be mapped to each ACGME core competency and associated subcompetencies. The content in the milestone levels provided in the ACGME subcompetencies could help further define observable knowledge, skills, and attitudes associated with each EPA beyond what is outlined in the “Descriptions and Tasks” of the EPA and could prove useful in driving curriculum development. Achievement of specific milestone levels could also be used to designate supervision levels for associated EPAs as described by ten Cate et al. 24
Accreditation Council for Graduate Medical Education (ACGME) Milestone Subcompetencies With Similarity to EPA Framework. 25
Abbreviations: AP, anatomic pathology; CP, clinical pathology; PC, patient care; MK, medical knowledge.
Sample entrustable professional activity (EPA) mapped to the Accreditation Council for Graduate Medical Education (ACGME) core competencies and relevant subcompetencies. Achievement of specific milestone levels within each subcompetency could also be used to designate supervision levels for associated EPAs.
There is no one way to prescriptively include EPAs in graduate medical education, and a growing number of studies demonstrate the versatility of the EPA model, which can be used for feedback, assessment, and curricular design. Entrustable professional activities offer a simple but powerful model for improving formative feedback for residents. Entrustable professional activities and CBME in general stress observational evaluation of the outcome of not only units of work but also the work process itself. In defining EPAs, the activity in question is broken down into knowledge, skills, and attitudes demonstrated with successful, competent performance of an activity. This description of the work process may provide guidance to the novice learner, which may otherwise only be available through trial and error. For the supervising faculty member, EPAs may highlight specific areas for instruction and coaching, beyond simple feedback on medical knowledge that may be otherwise overlooked by an expert practitioner. Such a shared model of expectations for both process and outcomes could lead to more efficient learning, which is critical in the age of duty hours, and to more open conversation and feedback centered on the learning process. To this end, Aylward and colleagues used an iterative process to develop a formative assessment tool for handoffs and were able to demonstrate improvement in resident performance over multiple observations in an internal medicine/internal medicine–pediatrics program. 26 Tools such as this provide a framework to move beyond generic feedback such as “good job,” “needs to read more,” or “enjoyed working with this resident” and give the evaluator a set of concrete knowledge, skills, and attitudes on which to comment for a particular activity to inform learner progress.
Entrustable professional activities can also offer a framework to implement curricular changes on a local level. For example, Hamburger and colleagues chose 1 EPA they felt was critical to their pediatric training and practice environment that of “Referral and Consultation.” 27 Through a comprehensive literature review and needs assessment using resident surveys and patient and clinician focus groups, they were able to develop granular curricular elements that mapped to competency domains. They also identified ways to deliver the curricular content, proposed a variety of methods for assessment of competency, and outlined the need for faculty development in the use of feedback tools. In this way, they transformed a process that had been dissatisfying for patients and providers alike that was not formally addressed as a curricular element into something that could be taught, observed, and improved upon by residents and faculty. At the program and institution level, a similar application could be considered for pathology, especially utilizing EPAs related to laboratory management. Adoption of EPA mastery into the curriculum surrounding tasks such as test utilization review or quality improvement could lead to novel educational content in teaching laboratory management, a more consistent experience for learners, and a constant pipeline of improvement initiatives for clinical laboratories associated with pathology residency training programs.
There are also multiple examples of EPAs being used to drive curricula at the national level. In 2014, the AAMC published 13 EPAs that graduating medical students entering residency should be able to perform unsupervised. 22 Since that time, the AAMC has also moved forward with a 5-year pilot study to develop curriculum, assessment, and faculty development tools to implement these EPAs with 10 participating institutions. Interest in this project was so great that over half of the 141 Liaison Committee on Medical Education (LCME)-accredited schools applied to take part in the study. 21 Numerous opportunities exist in pathology to use the EPA concept to help standardize training expectations nationally.
In addition to providing a framework for assessment tools and curriculum development, EPAs may also have additional benefits such as increasing expectations surrounding frequency of observation of resident performance and facilitating pass/fail decisions for residents given the standardization of assessment content. 28 However, CBME and the EPA framework also pose significant challenges to successful adoption in medical education. Klamen et al warn against repeating mistakes in competency-based models from other fields such as K-12 education and the military, including the overcomplication of the assessment process and the creation of time-consuming, inflexible training systems. 29 They and others also warn about the lack of consistent direct observation in medical education and the variability in rater observations as other potential pitfalls in the implementation of CBME-based initiatives. 28 -30 All of these factors may play a role in lack of faculty acceptance of CBME or failure of competency frameworks to achieve desired educational outcomes.
A specific challenge to the development and application of EPAs for pathology practice and training is that the scope of work performed by pathologists does not necessarily fit into the EPA framework. Many activities performed by pathologists, especially those with administrative responsibilities, are more longitudinal in nature, are administrative or supervisory, or occur with low frequency, limiting the opportunity for repeated performance and assessment of trainees. Examples of such activities include longitudinal test utilization or performance review, personnel management, participation in internal or external laboratory inspections, managing laboratory quality and safety programs, selection of new test platforms or laboratory information systems, application of informatics skills in laboratory management, and test verification or validation. However, the GMEC chose to include EPAs for many of these activities as they are critical to the practice of pathology. Residents do participate in components of many of these activities and perhaps are able to complete smaller discrete projects such as simple assay verifications or quality improvement projects. However, seldom do residents have the opportunity to complete complex, longitudinal projects, let alone with a frequency to allow for repeated observation. In these situations, documentation of participation likely acts as a surrogate marker for competence for many programs. Perhaps assessment tools based on EPAs could evaluate background knowledge, critical thinking skills, and situational leadership skills in lieu of repetition.
Another challenge to the EPA model for pathology is how to apply direct observation to the work of pathology trainees that is typically performed independently, such as microscopy. Although attending pathologists spend time “double-scoping” with residents during case sign-out or reviewing reports composed by residents on specific cases, they often rely upon a correct diagnosis as a marker for adequate evaluation of a case. A better understanding of the process novice learners go through in evaluating a case could provide an opportunity to teach a more efficient approach that is tailored to the individual resident. Using EPAs to focus on direct observation of resident work process may provide insight into inefficient work habits that lead to an inability to handle higher volumes and hinder a trainee’s ability to transition to independent practice.
A final challenge to the EPA model is how programs can integrate the concept of EPAs into their individual programs in a way that is meaningful to resident education, increases direct observation of resident performance, and does not overcomplicate the evaluation process or overburden faculty or program administration. The work presented here is only a preliminary list of EPAs focused on anatomic and clinical pathology training for the purposes of primary certification and should be expanded to include tools for assessment and curriculum development. Fellowships would also need to refine and add to the current list to reflect the more comprehensive treatment of some topics in fellowship training. Finally, pathology does not have a strong history of research on educational methods, and the small size of training programs makes evaluating and demonstrating the value of curriculum innovations or use of assessment tools difficult. Fostering a cooperative environment among programs, sharing innovative educational ideas openly, and encouraging cooperative research among institutions may be a way to ensure that adoption of CBME concepts such as EPAs is actually providing the intended educational benefits.
In conclusion, many of the critical activities that define the practice of pathology fit well within the EPA model, especially in regard to the procedural and diagnostic activities performed by pathologists. The EPAs presented by the GMEC are meant to provide an initial framework for the development of EPA-related assessment and curricular tools for pathology residency training. This innovation in the CBME movement encourages educators to be more observant and articulate teachers, to use desired outcomes to drive curriculum, and to recognize the learner’s progression toward mastery of an EPA, with the ability to adjust teaching and supervision strategies along the way. As such, EPAs may be one tool to address current challenges in pathology graduate medical education.
Footnotes
Author’s Note
For Dr. Brissette: The views and opinions expressed in this manuscript are those of the author and do not reflect the official policy or position of the Department of Army/Navy/Air Force, Department of Defense or the United States Government.
Acknowledgments
The authors would like to acknowledge Dr Gregory Blakey for reviewing the manuscript and supplemental materials.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding and support of the College of American Pathologists Graduate Medical Education Committee (CAP GMEC) is provided by the CAP in the form of educational staff support (authors KJ and SP), travel reimbursement for committee members (authors CBM, RED, RMC, RDH, MDP, MDB, DAG, PMR, DAC, CAR, AMR, and SZP), and funding for publication.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
