Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal

Abstract

Hematopathology fellowship education has grown in complexity as patient-centered treatment plans have come to depend on integration of clinical, morphologic, immunophenotypic, molecular, and cytogenetic variables. This complexity is in competition with the need for timely hematopathology care with stewardship of patient, laboratory, and societal resources. Accreditation Council for Graduate Medical Education Milestones provide a guidance document for hematopathology training, but fellows and their educators are in need of a simple framework that allows assessment and feedback of growth toward independent hematopathology practice. Entrustable professional activities provide one such framework, and herein, we provide proposed Hematopathology Fellowship Entrustable Professional Activities based on review of pertinent guidelines and literature, with multiple rounds of expert and stakeholder input utilizing a modified mini-Delphi approach. Ten core entrustable professional activities deemed essential for graduating hematopathology fellows were developed together with skills and knowledge statements, example scenarios, and corresponding Accreditation Council for Graduate Medical Education Milestones. Application of these entrustable professional activities in program design, fellow evaluation, and decisions regarding level of supervision is discussed with consideration of benefits and barriers to implementation. These entrustable professional activities may be used by hematopathology fellowship directors and faculty to provide fellows with timely constructive feedback, determine entrustment decisions, provide the Clinical Competency Committee with granular data to support Milestone evaluations, and provide insight into areas of potential improvement in fellowship training. Fellows will benefit from a clear roadmap to independent hematopathology practice with concrete and timely feedback.

Keywords

entrustable professional activities hematopathology fellowship graduate medical education entrustment milestones assessment

Introduction

Entrustable professional activities (EPAs) were formally conceptualized in 2005 by Olle ten Cate, who defined EPAs as “units of professional practice, defined as tasks or responsibilities to be entrusted to the unsupervised execution by a trainee once he or she has attained sufficient specific competence.”¹ The distinction between an EPA, whereby the work unit is described, and competency, whereby one’s ability is described, is critical: Evaluating a trainee’s overall abilities is subjective and prone to comparative-based judgments, whereas evaluating whether a task was performed adequately or not is a more objective decision that can be made in the context of a discrete, granular interaction at a specific point in time. For additional background on the development and utilization of EPAs in assessment, a “primer” by Olle ten Cate provides a straightforward summary.²

Entrustable professional activities have been developed and published for a variety of medical subspecialties, but there are relatively few published articles on EPAs for pathology training.^3
–5 In a paper from 2017, McCloskey et al published a list of 19 EPAs for pathology residents. While superficially there may appear to be overlap in the EPAs for pathology residents and the EPAs for hematopathology fellows, the skills and knowledge required for a hematopathology fellow are more specific to the subspecialty and commensurate with the advanced level of training. At the 2019 Society for Hematopathology Program Directors’ meeting, the Society for Hematopathology Education Committee (SH-EC) discussed the need to draft a list of EPAs specific to a 1-year ACGME-accredited hematopathology fellowship. While most hematopathology program directors (PDs) had heard of EPAs, very few were utilizing them in their programs. Most PDs expressed the need for a more standardized, objective method of assessment that complements current program requirements and assessment tools without adding additional administrative burden and oversight.

An ad hoc committee, the Hematopathology EPA Working Group (HEWG), was convened for the purposes of proposing hematopathology fellowship EPAs and assessing their utility and feasibility with the ultimate goal of generating resources, training, and education for programs and fellows to facilitate institutional implementation. The HEWG is comprised of hematopathologists who are directly involved in fellow and resident education, including members of the Education Committee of the Society for Hematopathology, as well as the current hematopathology fellowship ad hoc committee representative from the Association of Pathology Chairs (B.A.). Three working group members are also on the ACGME working group on the Hematopathology Milestones 2.0 revision (K.W., L.A.S., and J.T.-F.). One working group member is also a member of the College of American Pathologists (CAP) and Association for Pathology Chairs (APC) working group to test EPAs in the pathology residency setting (D.G.). We have also been fortunate to receive generous input from multiple subject matter experts (see Acknowledgments). Through its composition and consultation, HEWG coordinated our hematopathology EPA development efforts with a concurrent effort, led by Cindy McCloskey, of the College of American Pathologists and the APC to pilot test EPAs in the setting of pathology residency education. The HEWG also coordinated with concurrent updating of the Accreditation Council for Graduate Medical Education (ACGME) Milestones for Hematopathology Fellowship to version 2.0. Working group members are predominantly female, 7 to 21 years out of fellowship, and include one member who identifies as belonging to a group that is underrepresented in medicine. The committee sought to identify the most crucial tasks that were observable for a 1-year fellowship and were considered essential to successful independent practice, with a goal of 5 to 10 EPAs.

Methods

The HEWG used a multistep iterative mechanism to define proposed hematopathology EPAs (see Figure 1, flowchart). A modified mini-Delphi approach^6
–8 was used. The first mini-Delphi round consisted of input from the HEWG experts over web-based conference calls to define the scope and degree of granularity for hematopathology EPAs and to brainstorm and combine various aspects of hematopathology practice into a list of potential EPAs. The HEWG expert panel reviewed applicable literature, including resources on EPA development,^9
–11 proposed pathology EPAs,^4,5 and EPA experiences from other medical specialties^12

–15 We then reviewed ACGME Hematopathology Fellowship Milestones¹⁶ as well as ACGME Hematopathology Program Requirements¹⁷ for applicable entrustable activities and cross-checked them against our draft EPAs to provide content validity.

Figure 1.

Overview of the iterative process used to draft hematopathology fellowship EPAs. The “expert panel” is the HEWG. The Program Director survey was distributed through the Society for Hematopathology Fellowship Program Directors listserv. The Expert and Stakeholder survey was distributed through the same listserv and was also distributed to members of HEWG and of the Society for Hematopathology Executive Committee who were not on the Program Directors listserv, as well as a group of 15 diverse stakeholders including trainees, new-in-practice pathologists, private practice pathologists, academic hematopathologists who are not program directors, EPA experts, and a residency program director. ACGME indicates Accreditation Council for Graduate Medical Education; EPA, entrustable professional activity; HEWG, Hematopathology EPA Working Group.

Simultaneously with the initial mini-Delphi round of EPA generation, we designed and pilot tested an anonymous fact-finding survey to be administered to hematopathology fellowship PDs and associate program directors (APDs). The survey was designed to elicit PD and APD assessment, entrustment and remediation needs, and what entrustments they currently make (Supplementary Figure 1). The survey was deemed to be exempt by the Stanford University School of Medicine Institutional Review Board.

One hundred \eleven individuals representing all 87 hematopathology fellowships were sent the link to the PD survey via the Society for Hematopathology Program Directors listserv. Six HEWG members are hematopathology PDs and therefore received the survey link through the listserv. The survey contained both quantitative and qualitative components, including free-text questions, “What professional tasks and responsibilities do you give your fellows feedback on during fellowship?” and “Please specify what other graduated responsibilities you have for your fellows,” designed to elicit additional EPAs or components of EPAs that might not have been identified based on the initial rounds of background research and brainstorming. The HEWG utilized the survey questions in parallel with the draft EPAs to further modify the structure of the EPAs, particularly the distinction between EPAs and their knowledge and skills components. The first mini-Delphi round reached consensus on a list of 10 preliminary EPAs that are considered crucial to the practice of hematopathology.

After administration of the PD survey, the free-text response data from the survey were independently and manually coded with inductive qualitative methods¹⁸ by 2 HEWG members. The coding groups were compared to create a final list of feedback from the survey. In the second mini-Delphi round, HEWG members discussed the qualitative and quantitative survey results and reached a consensus agreement for further modifications to the 10 draft EPAs.

For the third mini-Delphi round, a second anonymous survey (Supplementary Figure 2) was designed, pilot tested, and administered to experts and stakeholders. The purpose of the survey was to elicit whether each draft EPA from the second mini-Delphi round captured a professional activity that is relevant and important to hematopathology practice and to solicit suggested changes or additions. The second survey consisted of draft EPAs with an example scenario; the knowledge and skills statements and milestone crosswalks were omitted for survey brevity. The survey was deemed to be exempt by the Stanford University School of Medicine Institutional Review Board. The survey data were reviewed by HEWG and used to inform additional revisions to the 10 draft EPAs.

In addition to 111 previously surveyed hematopathology PDs, APDs, and program coordinators previously sent the first survey link via the Society for Hematopathology listserv, the second anonymous survey was sent to 26 other experts and stakeholders for a total of 135 recipients. These included the additional 4 HEWG panel members who were not on the Program Directors listserv, 7 Society for Hematopathology Executive Committee members who were not on the Program Directors listserv, 2 trainees, 5 private practice pathologists, 3 new-in-practice pathologists, 1 residency PD, 2 academic hematopathologists who are not PDs, and 2 EPA experts. A third and final level of input was sought by email from the Society for Hematopathology Executive Committee, with edits made accordingly. The third mini-Delphi round resulted in consensus on the final list of proposed EPAs for hematopathology fellowship.

For both surveys, survey data were collected and managed using REDCap electronic data capture tools hosted at Stanford University.^19,20

Results

A summary of the methods and components to provide the content validity to draft the hematopathology EPAs is presented in Figure 1. An initial set of draft hematopathology EPAs were written by HEWG based on review of the EPA literature and the group’s experience as educators and clinicians. After reviewing published EPAs, we decided on a goal of producing a manageable 8 to 10 EPAs for the 1-year fellowship. In the first mini-Delphi round, an initial list of 16 possible hematopathology EPAs was generated; these were then condensed into 10 EPAs with associated knowledge and skills statements, and these were aligned with published AP/CP Residency EPAs⁴ as a framework for our EPA development (Supplementary Figure 3).

Results of Hematopathology Program Director Fact Finding Survey (Survey 1)

One hundred eleven individuals representing all 87 hematopathology fellowships were sent the link to the PD survey via a listserv email communication. Twenty-four began the survey, 3 were excluded due to attesting that they were not a PD or APD, and 4 who did attest to being a PD or APD did not answer any of the content questions; 17 self-identified PDs or APDs completed the survey. Regarding existing hematopathology fellowship assessment among the PD/APD respondents, the methods of rotation evaluations, faculty evaluations, and 360 evaluations were nearly universal (94%). A smaller percentage (65%) used written or oral examination and/or formal evaluations of presentation performance. Fewer than half used competency checklists/rubrics (41%), EPAs (24%), formal assessment of project performance (12%), or formal assessment of simulation performance (0%). Multiple methods of evaluation were common, with each respondent endorsing on average 4.5 methods of evaluation (range, 2-7).

In an attempt to elicit potential EPAs, we asked our respondents to identify professional tasks and responsibilities they give their fellows feedback on during fellowship, and of these, which they would consider EPAs. An impressive 72 free-text responses were provided; most but not all of the proposed tasks were considered by the respondents to be EPAs (see Supplementary Figure 4). We clustered these tasks and responsibilities and used them not only to update the list of draft EPAs but also to help generate associated examples and knowledge and skills statements.

This survey also provided an opportunity to solicit information regarding additional resources to help programs implement EPAs and self-rated ability to rapidly identify learner deficiencies, a crucial activity that may benefit from EPA evaluation. Of interest, when survey takers were asked to rate their ability to identify learner issues within the first quarter of the academic year, and then to rate their effectiveness at intervention and remediation for learners with identified challenges (Figure 2A), few rated themselves as excellent at identifying learner issues (24%) or very effective at remediating learners with identified challenges (17%). Both PDs and APD survey respondents indicated that they use existing feedback mechanisms to inform Clinical Competency Committee assessments (82%), formative evaluations (76%), graduation requirements and/or individualized education or remediation planning (65%), letters of recommendation (59%), rotation/curriculum structuring (47%), and length of training (18%).

Figure 2.

A survey was sent to hematopathology fellowship program directors and associate program directors through the Society for Hematopathology mailing list. A, Responses to the questions “Please rate your ability to identify learner competency issues within the first quarter of the academic year.” (black bars; n = 17) and “Please rate your program’s effectiveness at intervention/remediation for learners with identified challenges.” (white bars, n = 18). Answer options for the first question were “excellent,” “good,” “fair,” “poor,” and “no mechanism exists”; zero respondents chose “poor” and “no mechanism exists.” Answer options for the second question were “very effective,” “somewhat effective,” “somewhat ineffective,” “very ineffective,” and “challenges are not properly identified”; zero respondents chose “very ineffective” and “challenges are not properly identified.” B, Responses to the question “What graduated responsibilities (ie, entrustment decisions) do you have for your fellows? Please check all that apply.” N = 17. Options: full survey text in quotation marks, followed by abbreviated figure text in parentheses “independent call coverage” (call coverage); “independent triage of specimens for workup of a clinical disorder” (triage specimens); (Continued) “independent utilization of paraffin-embedded tissue (IHC, FISH, molecular, etc” (FFPE ancillary); “independent intraoperative consultation with surgical pathology” (frozen sections); “independent presentation at multidisciplinary conferences” (conferences); “independent clinical consultation regarding test selection/ordering” (test consult); “independent clinical consultation regarding test interpretation” (interpretative). C, Responses to the question: “What formal processes for transferring graduated responsibility do you use in your program (ie, the fellow may not independently take on the responsibility until it has been documented or discussed formally)? Please check all that apply.” (N = 17). Answer options were: “faculty evaluations,” “CCC assessment,” “certificate,” “we don’t have a formal process, responsibilities are assumed at a defined time point” (shortened to “time point” in bar graph), “we don’t have a formal process, it’s up to the individual attendings working with the fellow on services” (shortened to “attending-dependent” in bar graph), and “we don’t provide graduated responsibilities.” CCC indicates Clinical Competency Committee; FFPE, formalin-fixed paraffin-embedded; FISH, fluorescence in situ hybridization; IHC, immunohistochemistry.

Both PD and APD survey respondents indicated that the following EPA resources would be extremely useful: sample EPA forms (65%), crosswalks linking EPAs to ACGME Milestones (41%), a website with FAQs and information (35%), online training for faculty (35%), an online presentation for learners (29%), a physical meeting or conference (24%), email information distribution (18%), an online discussion board (12%), and a virtual meeting (12%). Respondents indicated the following informational content would be helpful: examples of how programs are evaluating EPAs (81%), faculty education regarding EPA implementation (81%), incorporation of EPAs into learner assessment (75%), introductory faculty education regarding EPAs (69%), introductory learner education regarding EPAs (62%), relation of EPA performance to milestone assessment (50%), learner education regarding implementing EPAs in their program (50%), clarification of what is an EPA (44%), and primary literature references (31%).

The survey also assessed graduated responsibilities (entrustment decisions) currently in place for fellows (Figure 2B) and asked about formal processes currently used for transferring graduated responsibility (Figure 2C). While all respondents endorsed providing graduated responsibility, about half indicated they do not have a formal process to assess the graduated responsibility.

Results of Expert Feedback Survey (Survey 2)

Of the 135 individuals who were emailed the second anonymous survey link, 35 recipients opened the survey, and 26 respondents completed some (1) or all (25) of the survey questions regarding EPAs. Of those who completed some or all of the questions regarding EPAs, 8 were hematopathology fellowship PDs or APDs. There was a broad range of time in practice: 2 respondents were trainees in pathology residency or fellowship, 5 were <5 years in practice, 8 were 5 to 10 years in practice, and 8 were >10 years in practice.

Survey results are included in Figure 3; most EPAs earned uniform or near-uniform endorsement from respondents. The EPA with the least endorsement was EPA 10 (perform bone marrow biopsy and aspirate), with 19 in agreement and 6 in disagreement that it “captures a professional activity that is relevant and important to hematopathology practice.” This is currently an ACGME Milestone requirement for hematopathology fellowship. Many respondents provided comments which were used by the committee in the final round of edits and to clarify wording of Milestones, Examples, and Knowledge and Skills statements. The revised draft EPAs were then emailed to the SH-EC and Executive Committee for a third and final round of feedback; minor comments were used for a final round of edits.

Figure 3.

The draft hematopathology fellowship EPAs along with a single representative example scenario was sent to a group of experts and stakeholders; for each listed draft EPA, the anonymous survey asked “Does this proposed EPA capture a professional activity that is relevant and important to hematopathology practice?” EPA indicates entrustable professional activity.

Finalized List of Proposed Hematopathology Fellowship EPAs and Example EPA-Based Formative Assessments

The finalized list of hematopathology fellowship EPAs is presented in Table 1. The full text of all 10 proposed hematopathology EPAs with example scenarios, knowledge and skills statements, and relevant milestones are available as Tables 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11.

Table 1.

Proposed Hematopathology Fellowship Entrustable Professional Activities.

1. Guide selection of diagnostic tests and triage and allocate specimens for ancillary studies.

2. Identify and communicate critical values and clinically urgent results.

3. Complete workup and diagnostic reporting of a simple hematolymphoid diagnosis.

4. Complete workup and diagnostic reporting of a complex or rare hematolymphoid diagnosis.

5. Select a flow immunophenotyping panel and compose an interpretive report.

6. Interpret hematology/coagulation tests and provide consultation.

7. Provide guidance on testing parameters and limitations for routine hematology, ancillary, or coagulation testing.

8. Present at interdisciplinary conferences and effectively communicate in a consultative role.

9. Maintain and improve quality of care on the hematopathology service and in the hematology laboratory.

10. Perform bone marrow aspiration and biopsy.

Table 2.

Hematopathology Entrustable Professional Activity 1: Guide Selection of Diagnostic Tests and Triage and Allocate Specimens for Ancillary Studies.

Skills

Ordering ancillary workup necessary for diagnosis and management for lymphoid and myeloid neoplasia and nonneoplastic causes of adenopathy and blood count abnormalities.

Stewardship of limited tissue.

Providing test utilization consultation, including intervention in inappropriate test ordering, and identifying potential areas of test overutilization.

Knowledge areas

Pathogenesis, clinical correlation, and prognostic significance; diagnostic and relevant clinical practice guidelines for hematolymphoid neoplasia, congenital, infectious, and other specific nonneoplastic entities.

Flow cytometry immunophenotyping panels, immunohistochemical stains, cytogenetic analysis, including karyotyping and fluorescence in situ hybridization (FISH) and molecular ancillary testing.

Example scenarios

Determine appropriate immunohistochemical stains to order when limited material available. Determine correct flow cytometry panel to perform when limited material available.

Triage cerebrospinal fluid or fine needle aspiration/core biopsy material to appropriate diagnostic assays, including morphology (cytologic or cell blocks), flow cytometry immunophenotyping, cytogenetic and molecular analysis.

Discuss inappropriate flow cytometry immunophenotyping orders with ordering provider.

ACGME Hematopathology Milestones version 2.0

Patient Care 1 and 4: Interdisciplinary consultation, specimen handling and triaging.

Medical Knowledge 5 and 6: Selection of molecular and cytogenetics testing and interpretation of reports, clinical reasoning in hematopathology and hematology.

Systems-Based Practice 3 and 5: Physician role in health care system, utilization.

Practice-Based Learning and Improvement 1 and 2: Evidence-based practice and scholarship, reflective practice and commitment to personal growth.

Professionalism 1, 2, and 3: Professional behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking.

Interpersonal and Communication Skills 2 and 3: Interprofessional and team communication, communication within health care systems.

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 3.

Hematopathology Entrustable Professional Activity 2: Identify and Communicate Critical Values and Clinically Urgent Results.

Skills

Rendering clinically urgent diagnoses triggered by microscopic or ancillary test review of any specimen type

Communication of critical values, including interdisciplinary communication

Documentation of clinically urgent communication

Knowledge areas

Disease mechanisms and clinical course of acute leukemias, consumptive coagulopathies, and systemic infections; morphologic/immunophenotypic recognition of these entities as applicable, and confirmatory testing for these entities as applicable

Concept and rationale of critical laboratory results

Principles and techniques of transition of care/handoffs

Example scenarios

Identify and communicate acute promyelocytic leukemia, thrombotic thrombocytopenic purpura, hemophagocytic lymphohistiocytosis, blasts in cerebrospinal fluid, organisms in cerebrospinal fluid

ACGME Hematopathology Milestones version 2.0

Patient Care 1: Interdisciplinary consultation

Medical Knowledge 6: Clinical reasoning in hematopathology and hematology

Systems-Based Practice 1 and 2: Patient safety and quality improvement (levels 1-3), systems navigation for patient-centered care

Practice-Based Learning and Improvement 2: Reflective practice and commitment to personal growth

Professionalism 1, 2 and 3: Professional Behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking

Interpersonal and Communication Skills 1, 2 and 3: Patient and family-centered communication, interprofessional and team communication, communication within health care systems

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 4.

Hematopathology Entrustable Professional Activity 3: Complete Workup and Diagnostic Reporting of a Simple Hematolymphoid Diagnosis.

Skills

Ordering appropriate initial and, if indicated, next round ancillary studies based on clinical setting and differential diagnosis. Integrating ancillary studies including immunohistochemistry, flow cytometry immunophenotyping, cytogenetics, fluorescence in situ hybridization, targeted molecular studies

Providing a preliminary report to the patient-facing team

Writing succinct and complete final report including any indicated synoptic reporting

Knowledge areas

Acute leukemias, classic myeloproliferative neoplasms, B-cell lymphomas, involving bone marrow aspirate and core, lymph node, extranodal tissue, peripheral blood, and body fluids

Indications for and interpretation of immunohistochemistry, flow cytometry immunophenotyping, cytogenetics, fluorescence in situ hybridization, and targeted molecular studies

Example scenarios

Order and interpret lymphoma staging studies with appropriate flow cytometry immunophenotyping, immunohistochemistry, and fluorescence in situ hybridization studies as indicated. Initial diagnosis of small B-cell lymphomas. Work up large B-cell lymphoma and classic Hodgkin lymphoma

Provide verbal and/or written preliminary reports to colleagues and to the patient-facing team, including discussion of level of uncertainty and significance of pending studies

ACGME Hematopathology Milestones version 2.0

Patient Care 1 and 2 (levels 1-3): Interdisciplinary consultation, reporting

Medical Knowledge 4, 5, and 6 (levels 1-3): Morphologic interpretation and diagnosis, selection of molecular and cytogenetics testing and interpretation of reports, clinical reasoning in hematopathology and hematology

Systems-Based Practice 3 and 5: Physician role in health care system, utilization (levels 1-3)

Practice-Based Learning and Improvement 1 and 2: Evidence-based practice and scholarship (levels 1-2), reflective practice and commitment to personal growth

Professionalism 1, 2, and 3: Professional behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking

Interpersonal and Communication Skills 2 and 3: Interprofessional and team communication, communication within health care systems

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 5.

Hematopathology Entrustable Professional Activity 4: Complete Workup and Diagnostic Reporting of a Complex or Rare Hematolymphoid Diagnosis.

Skills

Providing a preliminary report to the patient-facing team

Writing succinct and complete final report including any indicated synoptic reporting

Integrating relevant literature/references and/or expert consultation

Recognizing areas of diagnostic challenge where a definitive diagnosis cannot be reached with the given material and communicating effectively to the clinician

Knowledge areas

Flow cytometry immunophenotyping panels, immunohistochemical stains, cytogenetic analysis, including karyotyping and fluorescence in situ hybridization and molecular ancillary testing

Indications for and interpretation of next-generation sequencing panel testing, fluorescence in situ hybridization testing, and esoteric/sendout testing; techniques and technical limitations of ancillary studies

Example scenarios

Perform complete diagnostic workup of myelodysplastic syndromes, myelodysplastic/myeloproliferative neoplasms, T/NK cell lymphomas, gray zone lymphomas, histiocytic and dendritic cell neoplasms, and nodular lymphocyte predominant Hodgkin lymphoma

ACGME Hematopathology Milestones version 2.0

Patient Care 1 and 2 (levels 3-5): Interdisciplinary consultation, reporting

Medical Knowledge 4, 5 and 6 (levels 3-5): Morphologic interpretation and diagnosis, selection of molecular and cytogenetics testing and interpretation of reports, clinical reasoning in hematopathology and hematology

Systems-Based Practice 3 and 5: Physician role in health care system, utilization (levels 3-5)

Practice-Based Learning and Improvement 1 and 2: Evidence-based practice and scholarship (levels 3-5), reflective practice and commitment to personal growth

Professionalism 1, 2, and 3: Professional behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking

Interpersonal and Communication Skills 2 and 3: Interprofessional and team communication, communication within health care systems

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 6.

Hematopathology Entrustable Professional Activity 5: Select a Flow Immunophenotyping Panel and Compose an Interpretive Report.

Skills

Design/select appropriate flow cytometric initial and follow-up immunophenotyping panels based on clinical and/or morphologic information

Gate and analyze raw flow immunophenotyping data

Recognize common technical and gating errors in flow cytometry and know how to avoid them

Accurately describe and interpret a flow cytometric immunophenotype

Incorporate flow cytometry data into clinical and morphologic context

Knowledge areas

Flow cytometry techniques, including specimen processing and analysis, as well as pitfalls in analysis and interpretation

Example scenarios

Design/pick an appropriate flow cytometry panel for a low cellularity cerebrospinal fluid specimen in a patient with known lymphoma history

Recognize typical and atypical immunophenotypic patterns for chronic lymphocytic leukemia

Distinguish between hematogones (normal B-lineage precursors) and leukemic B-lymphoblasts

Assign lineage to an acute leukemia. Recognize reactive and neoplastic T antigen abnormalities

ACGME Hematopathology Milestones version 2.0

Patient Care 1 and 2: Interdisciplinary consultation, reporting

Medical Knowledge 3 and 6: Interpretation of flow cytometry, clinical reasoning in hematopathology and hematology

Practice-Based Learning and Improvement 1 and 2: Evidence-based practice and scholarship, reflective practice and commitment to personal growth

Professionalism 1, 2, and 3: Professional behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking

Interpersonal and Communication Skills 2 and 3: Interprofessional and team communication, communication within health care systems

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 7.

Hematopathology Entrustable Professional Activity 6: Interpret Hematology/Coagulation Tests and Provide Consultation.

Skills

Answer clinical questions regarding test selection (preanalytic) and patient results (postanalytic)

Provide interpretative report for hemoglobin analysis and other tests (eg, hypercoagulation panel)

Automated hematology analyzers, coagulation testing, red cell disorder testing, hemoglobin analyses

Knowledge areas

Algorithmic and/or panel testing approaches for evaluation of bleeding disorders and hypercoagulable states

Knowledge of congenital and acquired hemostatic disorders and their management

Categories of anticoagulant therapies, and the indications and laboratory methods of monitoring them

Clinical significance and methods of diagnosing common hemoglobinopathies and thalassemias

Example scenarios

Provide interpretation of hemoglobin electrophoresis analysis in the context of peripheral blood findings and clinical scenario

A diagnostic approach to the evaluation of hemolytic anemias, congenital or acquired. Work up von Willebrand disease, factor deficiencies, and inhibitors

Guide and interpret testing for lupus anticoagulant testing, protein C or S deficiency. Consult on management of heparin, direct thrombin inhibitors, anti-Xa inhibitors

ACGME Hematopathology Milestones version 2.0

Patient Care 1 and 2: Interdisciplinary consultation, reporting

Medical Knowledge 1, 2, and 6: Interpretation of hematology and coagulation testing, clinical reasoning in hematopathology and hematology

Systems Based Practice 3 and 5: Physician role in health care system, utilization

Practice-Based Learning and Improvement 1 and 2: Evidence-based practice and scholarship, reflective practice and commitment to personal growth

Professionalism 1, 2, and 3: Professional behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking

Interpersonal and Communication Skills 2 and 3: Interprofessional and team communication, communication within health care systems

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 8.

Hematopathology Entrustable Professional Activity 7: Provide Guidance on Testing Parameters and Limitations for Routine Hematology, Ancillary, or Coagulation Testing.

Skills

Provide guidance on appropriate testing based on the clinical question and specimen and test characteristics

Troubleshoot discrepant/unusual/unreportable results and provide recommendations to laboratory staff and/or ordering provider

Develop laboratory protocols/procedures for commonly occurring test issues

Knowledge areas

Appropriate indications, technical requirements and techniques for routine and special hematology testing, special testing, flow cytometry immunophenotyping, cytogenetics, fluorescence in situ hybridization, immunohistochemistry, single and panel molecular testing, coagulation studies

Example scenarios

Troubleshoot and provide guidance on effects of interfering substances (eg, elevated bilirubin, hyperlipidemia, cryoglobulins, anticoagulation medication) on automated hematology and coagulation testing

Evaluate automated hematology analyzer flagging criteria, manual differential/pathologist review criteria

Detect and resolve platelet clumping (pseudo-thrombocytopenia)

ACGME Hematopathology Milestones version 2.0

Patient Care 1 and 4: Interdisciplinary consultation, specimen handling and triaging

Medical Knowledge 4 and 5: Selection of molecular and cytogenetics testing and interpretation of reports, clinical reasoning in hematopathology and hematology

Systems-Based Practice 3 and 5: Physician role in health care system, utilization

Practice-Based Learning and Improvement 1 and 2: Evidence-based practice and scholarship, reflective practice and commitment to personal growth

Professionalism 1, 2, and 3: Professional behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking

Interpersonal and Communication Skills 2 and 3: Interprofessional and team communication, communication within health care systems

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 9.

Hematopathology Entrustable Professional Activity 8: Present at Interdisciplinary Conferences and Effectively Communicate in a Consultative Role.

Skills

Interdisciplinary communication and presentation skills

Able to state and support degree of confidence of diagnosis, and specify additional studies that could clarify the diagnosis

Knowledge areas

Pathologic features that inform staging, prognostication, or prediction of treatment response

Ongoing clinical trials at institution that may require additional ancillary studies or reporting of specific features

Example scenarios

Actively participate in multidisciplinary tumor boards

Present at morbidity and mortality conferences

ACGME Hematopathology Milestones version 2.0

Patient Care 1: Interdisciplinary consultation

Medical Knowledge 6: Clinical reasoning in hematopathology and hematology

Systems-Based Practice 2: Systems navigation for patient-centered care

Practice-Based Learning and Improvement 1 and 2: Evidence-based practice and scholarship, reflective practice and commitment to personal growth

Professionalism 1, 2, and 3: Professional behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking

Interpersonal and Communication Skills 1, 2 and 3: Patient and family-centered communication, interprofessional and team communication, communication within health care systems

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 10.

Hematopathology Entrustable Professional Activity 9: Maintain and Improve Quality of Care on the Hematopathology Service and in the Hematology Laboratory.

Skills

Identify and evaluate potential safety/quality issues in the hematopathology service/hematology laboratory and propose changes as needed

Maintain appropriate and up-to-date hematology laboratory testing menus, flow cytometry panels, immunohistochemical stains, and fluorescence in situ hybridization and molecular tests if appropriate (in-house and reference testing)

Apply root cause analysis and performance improvement tools (Lean, Six Sigma, Plan Do Study Act cycle) to the hematopathology service/hematology laboratory

Knowledge areas

Be aware of applicable hematology, anatomic pathology, and flow cytometry laboratory accreditation requirements

Stay up to date with new clinically relevant hematology tests and ancillary diagnostic tests

Example scenarios

Monitor send out test request patterns to prioritize new hematology test or molecular tests for in house validation

Identify a recurring slide quality issue, communicate the issue to the appropriate section supervisor, and provide feedback on whether changes to workflow are satisfactory

Evaluate, choose, and validate a new hematology laboratory or hematopathology service test, instrument, or assay, such as an immunohistochemistry assay or flow cytometry panel

Participate in proficiency testing and sign off on protocol changes in the hematology lab, such as coagulation test, flow cytometry, or immunohistochemistry

ACGME Hematopathology Milestones version 2.0

Patient Care 1: Interdisciplinary consultation

Medical Knowledge 6: Clinical reasoning in hematopathology and hematology

Systems Based Practice 1, 2, 3, and 4: Patient safety and quality improvement, systems navigation for patient-centered care, physician role in health care system, accreditation, compliance, and quality

Practice-Based Learning and Improvement 1 and 2: Evidence-based practice and scholarship, reflective practice and commitment to personal growth

Professionalism 1, 2, and 3: Professional behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking

Interpersonal and Communication Skills 1, 2, and 3: Patient and family-centered communication, interprofessional/team communication, communication within health care systems

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 11.

Hematopathology Entrustable Professional Activity 10: Perform Bone Marrow Aspiration and Biopsy.

Skills

Provide informed consent, administer local anesthetic, identify correct needle placement, sterile technique, identify bone spicules, document procedure

Collect and triage material for flow immunophenotyping, cytogenetic studies, and other studies in appropriate media as clinically indicated

Knowledge areas

Components of informed consent, collection requirements for ancillary testing, anatomy/landmarks of the posterior superior iliac crest, appropriate postprocedural care

Example scenarios

Diagnostic bone marrow biopsy and aspirate in a patient with unexplained cytopenias, with material sent for cytogenetics and flow immunophenotyping

ACGME Hematopathology Milestones version 2.0

Patient Care 3 and 4: Bone marrow aspiration and biopsy, specimen handling and triaging

Medical Knowledge 6 (levels 1-3): Clinical reasoning in hematopathology and hematology

Practice-Based Learning and Improvement 2: Reflective practice and commitment to personal growth

Professionalism 1, 2, and 3: Professional behavior and ethical principles, accountability and conscientiousness, self-awareness and help-seeking

Interpersonal and Communication Skills 1, 2, and 3: Patient- and family-centered communication, interprofessional and team communication, communication within health care systems

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Entrustable professional activity-based formative and summative assessment tools are a key component to successful implementation. While formative and summative assessments will need to be individually tailored for each program, Tables 12 and 13 provide example formative and summative assessments for EPA 1. The entrustment scale utilized is adapted from the Ottawa Surgical Competency Operating Room (O-SCORE) Scale: An Entrustability-Aligned Anchor Scale.²¹ Table 14 provides an overview of correspondences between the hematopathology fellowship EPAs and relevant milestones.

Table 12.

Example Formative Assessment for Hematopathology Entrustable Professional Activity 1: Guide Selection of Diagnostic Tests and Triage and Allocate Specimens for Ancillary Studies.

Check off observed skills or knowledge
1. Orders necessary and sufficient ancillary workup for diagnosis and clinical management (immunohistochemistry, flow cytometry, cytogenetics, molecular)
a. Recognizes which cases need initial ancillary testing
b. Recognizes which cases need add-on ancillary testing
c. Suggests specific ancillary testing needed
d. Independently orders appropriate and specific testing
Comments (eg, What concrete steps can the fellow take to improve this skill?)____________
2. Guides stewardship of limited tissue
a. Recognizes when tissue or sample is limited
b. Suggests priority list of ancillary testing for limited samples
c. Independently orders the most critical testing for limited samples
Comments____________
3. Manages appropriate test utilization
a. Recognizes suboptimal test ordering or overutilization
b. Consults or intervenes in individual cases with guidance
c. Independently consults or intervenes in individual cases
d. Recognizes and generates solutions for patterns of suboptimal test utilization
Comments____________

Table 13.

Example Summative Assessment for Hematopathology Entrustable Professional Activity (EPA) 1: Guide Selection of Diagnostic Tests and Triage and Allocate Specimens for Ancillary Studies.

1. Based on formative assessments, please rate the level of entrustment you suggest for this EPA

a. May observe only

b. May perform under direct in person supervision

c. May perform with indirect/occasional supervision

d. May perform independently with post hoc review

e. May perform independently and may supervise others

Based on formative evaluations and entrustment level for this EPA, assess level of key related milestones. Suggested levels or ranges are adapted from 2020 draft ACGME milestones but have not been suggested or approved by ACGME. Final ACGME Milestone levels should be assigned by the Clinical Competency Committee based on multiple inputs and with reference to the up-to-date ACGME guidance.

2. ACGME Milestone Patient Care 4: Specimen handling and triaging

Level 1: Describes specimen handling and processing requirements

Level 2: Designates specimen for indicated ancillary testing

Level 3: Prioritizes plentiful or limited specimen for indicated ancillary testing

Level 4: Independently prioritizes plentiful or limited specimen for ancillary testing

Level 5: Serves as a resource for specimen handling and triaging

3. ACGME Milestone Systems-Based Practice 5: Utilization

Level 1: Identifies general hematopathology work practices and workflow

Level 2: Explains rationale for utilization patterns in own practice setting

Level 3: Identifies opportunities to optimize utilization of pathology resources

Level 4: Initiates efforts to optimize utilization

Level 5: Completes a utilization review and implements change

4. ACGME Milestone Professionalism 2: Accountability and conscientiousness

Level 1: Responds promptly to instructions and prompts

Level 2: Takes ownership, timely and attentive to detail

Level 3: Recognized impediments and describes the impact on the team

Level 4: Anticipates and intervenes in impediments to team functioning

Level 5: Takes ownership of system outcomes and implements new strategies

Abbreviation: ACGME, Accreditation Council for Graduate Medical Education.

Table 14.

Correspondence of Hematopathology Entrustable Professional Activities (EPAs) to ACGME Hematopathology Milestones, Version 2.0.*

	EPA 1	EPA 2	EPA 3	EPA 4	EPA 5	EPA 6	EPA 7	EPA 8	EPA 9	EPA 10
PC1: Interdisciplinary Consult	x	x	x	x	x	x	x	x	x
PC2: Reporting			x	x	x	x
PC3: Bone Marrow Biopsy										x
PC4: Specimen Handling, Triage	x						x			x
MK1: Hematology						x
MK2: Coagulation						x
MK3: Flow Cytometry					x
MK4: Morphologic Diagnosis			x	x
MK5: Molecular, Cytogenetics	x		x	x			x
MK6: Clinical Reasoning	x	x	x	x	x	x	x	x	x	x
SBP1: Patient Safety and Quality		x							x
SBP2: Systems Navigation		x						x	x
SBP3: Physician Role	x		x	x		x	x		x
SBP4: Accreditation, Compliance									x
SBP5: Utilization	x		x	x		x	x
PBLI1: Evidence-Based Practice	x		x	x	x	x	x	x	x
PBLI2: Reflective Practice	x	x	x	x	x	x	x	x	x	x
PROF1: Professional Behavior	x	x	x	x	x	x	x	x	x	x
PROF2: Accountability	x	x	x	x	x	x	x	x	x	x
PROF3: Self-awareness	x	x	x	x	x	x	x	x	x	x
ICS1: Patient Communication		x						x	x	x
ICS2: Team Communication	x	x	x	x	x	x	x	x	x	x
ICS3: System Communication	x	x	x	x	x	x	x	x	x	x

Abbreviations: ACGME, Accreditation Council for Graduate Medical Education; EPA, entrustable professional activity; ICS, interpersonal and communication skills; MK, medical knowledge; PBLI, practice-based learning and improvement; PROF, professionalism; SBP, systems-based practice.

* ACGME Milestone names have been abbreviated to fit the table.

Discussion

The hematopathology community is deeply invested in training the next generation of hematopathologists as confident and capable physicians, and competence-based education has contributed positively toward meeting this goal. Evaluating competency, however, is fraught with drawbacks, including subjectivity in assessing performance, interpretation of competencies involving broad skills and/or intangible outcomes, and determining thresholds of competency for both graduation and remediation. Entrustable professional activities represent a subtle but important paradigm shift in evaluating performance by changing the focus from observing personal characteristics and behaviors to observing the performance of specific tasks. Observing a task is a discrete activity, with a beginning and end, and the observation can be carried out repeatedly over time by faculty, peers, and support staff. Our consortium, HEWG, has developed 10 core EPAs of hematopathologists with the input of current fellowship PDs and an expert panel of pathologists in a variety of practice settings and education specialists.

We sought input from hematopathology fellowship PDs and a variety of stakeholders through anonymous surveys. In order to preserve anonymity for PDs, we did not ask for the name of the institution of survey respondents, and we therefore cannot guarantee equitable representation regarding program size, geography, academic/research expectations, and other logistical and educational aspects of programs. Survey response rates were low, but those who completed the surveys were generous with their free-text comments, all of which were carefully considered in iteratively writing the EPAs and their associated knowledge and skills statements and example scenarios. While the majority of potential EPAs mentioned by PDs (Supplementary Figure 4) were included, some were either not specific to hematopathology, not a necessary component of independent hematopathology practice, or not amenable to observation-based assessment. For example, the ability to teach junior residents and medical students is a core skill acquired over the course of residency and fellowship training; 4 PDs mentioned providing fellows with feedback on this skill and stated that they considered this an EPA. Whether teaching should be considered a hematopathology EPA was discussed at length by HEWG; in the end, we decided not to include it as an EPA given that it (1) is difficult to observe a discrete instance of teaching and determine whether it was “successful” from the viewpoint of an observer and not a learner, (2) is not a necessary component of independent hematopathology practice, and (3) encompasses a broader category of medical practice culture beyond the purview of this educational initiative. The final proposed EPAs had near-uniform endorsement as capturing a professional activity that is relevant and important to hematopathology practice with the exception of EPA 10 (perform bone marrow biopsy and aspirate), which reached only 76% endorsement. Indeed, a recent survey of American Board of Pathology diplomats in hematopathology indicated that only 18% themselves perform bone marrow biopsies in practice.²² Nevertheless this is an ACGME Milestone requirement and thus is considered a core EPA for hematopathology fellowship.

When asked what proposed resources would be useful to them, PDs and APDs indicated greatest interest in sample EPA forms, crosswalks linking EPAs to ACGME Milestones, and a website with FAQs and information. A majority stated the following resources would be useful: examples of how programs are evaluating EPAs, faculty education regarding EPA implementation in their program, incorporation of EPAs into learner assessment, and faculty education regarding EPAs (introduction to the topic). The HEWG has addressed some of these needs by incorporating knowledge and skills statements, scenarios, and crosswalks to ACGME Milestones in the EPAs proposed here, as well as by providing example formative and summative assessment forms.

When asked about entrustment decisions, about half of PDs and APDs say “We don’t have a formal process”—choosing “it’s up to the individual attendings working with the fellow on services,” “responsibilities are assumed at a defined time point,” or both. This suggests a lack of data and/or consistent criteria to apply to an individual. With a goal of entrustment, the inability to achieve entrustment implies the learner has a deficiency that requires identification and remediation. The ability to identify learner issues early is key to successful intervention and remediation in a 1-year fellowship. Despite the use of multiple forms of evaluation (averaging 4-5 per program), <25% of surveyed PDs and APDs indicated their ability to identify learner issues within the first quarter was “excellent.” Likewise, the majority of PDs and APDs rated themselves as only “somewhat effective” at intervention and remediation for learners with identified challenges in the same survey. The data demonstrate the need to improve how we determine entrustment and recognize barriers to entrustment. We see EPAs as a pivotal tool to help inform entrustment decisions; if EPAs are carefully defined and related to knowledge and skills statements, they can also help identify learner issues early and provide a roadmap to learning/remediation plans.

In order to facilitate these objectives, we have provided a mechanism to assess EPAs via formative and summative assessment forms. Examples are provided for hematopathology EPA 1 (Tables 12 and 13). Formative and summative assessments of EPAs can provide fellows with tailored, standardized feedback to direct their learning by indicating which specific skills of the EPA have been mastered and which have not yet been mastered. In addition to alleviating the ambiguity of an open-ended “comment” evaluation, faculty can make evidence-based graduated responsibilities.

A model formative assessment for EPA 1 (guide selection of diagnostic tests and triage and allocate specimens for ancillary studies) is presented as Table 12. This form offers faculty a checklist of observed behaviors with respect to key skills encompassed in the EPA; in this example, skills include ordering a necessary and sufficient ancillary workup for diagnosis and clinical management, guiding stewardship of limited tissue, and managing appropriate test utilization. Each skill lists behaviors and also contains a prompt for granular feedback, which can be used to elaborate performance aspects needing attention to advance their practice or recognize mastery or exceptional strength. For example, an overzealous orderer of immunohistochemical studies may never bring an excisional biopsy to sign out that requires more workup (skill 1) but may struggle prioritizing which stains to order on a rapidly vanishing needle core where the original diagnostic objective may not be met (skill 2).

The formative assessment is meant to be used periodically “on the job,” for example, at the end of a week on service with the fellow.

Over time, when multiple formative assessments are available, the PD, rotation director, clinical competency committee member, and so on, can then synthesize the submitted formative assessments and create a summative evaluation for the EPA, such as that presented in Table 13. Question 1 in the summative assessment can be used to provide graduated responsibilities and to make entrustment decisions, such as signing off on having fellows order ancillary studies independently without faculty oversight for limited specimens. Remaining questions constitute proposed milestone levels for select milestones deemed most directly related to the EPA, with the recognition that the clinical competency committee takes additional and broader information into account when determining milestone achievements. Table 14 presents correspondences of Hematopathology Entrustable Professional Activities to ACGME Hematopathology Milestones, version 2.0; the multiple correspondences provide opportunities for structured EPA-based feedback for each milestone.

The connections between EPAs and the ACGME Milestones may be of particular interest to educators in hematopathology fellowship programs. As described by ten Cate, EPAs can serve to “bridge [the] gap between well-elaborated competency frameworks and clinical practice.”² Indeed, EPAs are tasks that require the successful application of multiple competencies. The matrix illustrating the relationships between our proposed EPAs and the ACGME Hematopathology Milestones 2.0 (Table 14) provides a compelling demonstration of the relevance of the proposed EPAs vis-à-vis the accepted competency framework in the field. Of note, some groups have utilized such “maps” between EPAs and milestones to translate direct observation of work activities via EPAs to more general milestone competency evaluations.¹⁴

Aggregate EPA-based program data may aid in rotation and curriculum structuring; fewer than half of PDs and APDs surveyed use existing forms of feedback to inform the structure of their rotations and educational content. This lack of alignment may predispose programs to recurring learner issues. An EPA-informed approach to program structure may provide needed balance to the workload-based approach that tends to dominate large, busy academic services.

Entrustable professional activities are concurrently being piloted for pathology residency education, and some EPAs, such as reporting of critical values and writing of a diagnostic report for a simple or complex specimen, are a component of both residency- and fellowship-level EPAs. It will be important to revisit the relevance of the fellowship-level EPAs if and when pathology residency EPAs have been widely adopted. However, unless mastery of EPAs is a residency program graduation requirement, there is no guarantee a fellow will have done so at the time of matriculation to fellowship. Assumption of competency creates patient safety risks for fellows, their training programs, and future places of employment. The skills and knowledge statements of the hematopathology fellowship EPAs are specific to the subspecialty and commensurate with the advanced level of training; they build on the skills demonstrated as part of the related pathology residency-level EPAs. For example, reporting a common and straightforward diagnosis such as chronic lymphocytic leukemia/small lymphocytic lymphoma in an excisional biopsy may be encompassed in a residency EPA; however, integration of clinically relevant ancillary studies such as fluorescence in situ hybridization or molecular studies in the same report is captured in the fellowship but not the residency-level EPA. Fellows already competent at the resident level for analogous EPAs might be eligible for earlier entrustment based on faculty observations at the fellowship level.

Entrustable professional activities are not a panacea to the challenges of modern medical education. Not every skill set can be easily observed in a year’s time, particularly in regard to the leadership and management of laboratories, staff, and various operational activities, many of which are critical to success but not included in the proposed EPAs. Both teaching and assessing these skills remain a largely unmet need in the ever-growing list of skills expected of new pathologists. Levels of case complexity have been superficially addressed in our EPAs; not every example scenario will be observed, and the determination of simple and complex is subjective. In these regards, the ACGME core competencies and their corresponding milestone evaluations remain essential global assessments of performance. The Clinical Competency Committee serves a crucial role in synthesizing all available information at its disposal, and it is hoped that EPA assessments will become a valuable addition to a toolkit whose contents can be simultaneously sparse and overwhelming. Programs, departments, and specialties that have adopted EPAs have followed diverse strategies, and implementation among hematopathology fellowship programs will likely also differ according to specific program needs and assessment philosophies. This working group has endeavored to be descriptive rather than prescriptive. As in other subspeciality areas, implementation of EPA-based assessment tools in hematopathology will be a subject for further study.

The challenge remains to find a niche for EPAs in the design of thoughtful programmatic improvement and evaluation. Program directors already face an overwhelming administrative burden and have expressed a desire for more concrete evaluation tools during discussions at the 2018 SH PDs meeting. It is our hope that rather than being an additional burden, EPA-based assessment forms will become a chosen tool to provide concrete fellow feedback and inform entrustment and clinical competency committee decision-making. Fellowship directors may choose to replace current evaluation forms in programs where the need for objective, actionable data remains unmet. It will be important to revisit the utility, validity, and relevance of these hematopathology fellowship EPAs as our profession evolves. The growing depth and breadth of our subspecialty practice necessitates the adoption of practical models of entrustment and an individualized approach to learning. The concept of an individualized learning approach afforded in part by EPAs begs many intriguing questions. Should a trainee’s schedule be modified, either in duration or content, to ensure they achieve independent practice of EPAs before matriculation? How can a program logistically accommodate trainees of different levels of entrustment? There are no easy solutions for these questions, but they speak to the essence of our charge as educators, whom the public has entrusted to cultivate future generations of physicians.

Supplemental Material

Supplemental Material, sj-pdf-1-apc-10.1177_2374289521990823 - Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal

Supplemental Material, sj-pdf-1-apc-10.1177_2374289521990823 for Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal by Kristie White, Julianne Qualtieri, Elizabeth L. Courville, Rose C. Beck, Bachir Alobeid, David R. Czuchlewski, Julie Teruya-Feldstein, Lorinda A. Soma, Sonam Prakash and Dita Gratzinger in Academic Pathology

Supplemental Material

Supplemental Material, sj-pdf-2-apc-10.1177_2374289521990823 - Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal

Supplemental Material, sj-pdf-2-apc-10.1177_2374289521990823 for Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal by Kristie White, Julianne Qualtieri, Elizabeth L. Courville, Rose C. Beck, Bachir Alobeid, David R. Czuchlewski, Julie Teruya-Feldstein, Lorinda A. Soma, Sonam Prakash and Dita Gratzinger in Academic Pathology

Supplemental Material

Supplemental Material, sj-pdf-3-apc-10.1177_2374289521990823 - Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal

Supplemental Material, sj-pdf-3-apc-10.1177_2374289521990823 for Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal by Kristie White, Julianne Qualtieri, Elizabeth L. Courville, Rose C. Beck, Bachir Alobeid, David R. Czuchlewski, Julie Teruya-Feldstein, Lorinda A. Soma, Sonam Prakash and Dita Gratzinger in Academic Pathology

Supplemental Material

Supplemental Material, sj-pdf-4-apc-10.1177_2374289521990823 - Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal

Supplemental Material, sj-pdf-4-apc-10.1177_2374289521990823 for Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal by Kristie White, Julianne Qualtieri, Elizabeth L. Courville, Rose C. Beck, Bachir Alobeid, David R. Czuchlewski, Julie Teruya-Feldstein, Lorinda A. Soma, Sonam Prakash and Dita Gratzinger in Academic Pathology

Footnotes

Authors’ Note

The Stanford REDCap platform (), which was used to administer the surveys, is developed and operated by Stanford Medicine Research IT team. The REDCap platform services at Stanford are subsidized by (1) Stanford School of Medicine Research Office and (2) the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through grant UL1 TR001085.

Acknowledgments

This effort could not have succeeded without generous input from the dozens of hematopathologists and recent fellowship graduates through senior pathologists, academic and private practice, who generously gave us their time through anonymous surveys. We would like to acknowledge Laura Edgar, EdD, CAE Vice President, Milestones Development, Accreditation Council for Graduate Medical Education (ACGME) and Sydney McLean, MHA, ACGME Milestones Project Manager, for their assistance and willingness to let us use the ACGME Hematopathology Fellowship Milestones version 2.0 as a reference while it was still a work in progress. We would like to thank Dr Cindy McCloskey for her input on scoping of hematopathology EPAs and helping align hematopathology EPAs with pathology residency EPAs under development. We would like to thank the Society for Hematopathology Educational and Executive committees for their expert input, particularly Dr Genevieve Crane and Dr Alexandra Kovach for their specific input.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Rose C. Beck

Dita Gratzinger

Supplemental Material

Supplemental material for this article is available online.

References

Cate

. Entrustability of professional activities and competency-based training. Med Educ. 2005;39:1176–1177. doi:10.1111/j.1365-2929.2005.02341.x

Cate

. A primer on entrustable professional activities. Korean J Med Educ. 2018;30:1–10. doi:10.3946/kjme.2018.76

O’Dowd

Lydon

O’Connor

Madden

Byrne

. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011-2018. Med Educ. 2019;53:234–249. doi:10.1111/medu.13792

McCloskey

Domen

Conran

, et al. Entrustable professional activities for pathology: recommendations from the College of American Pathologists Graduate Medical Education Committee. Acad Pathol. 2017;4. doi:10.1177/2374289517714283

Powell

Wallschlaeger

. Making sense of the milestones: entrustable professional activities for pathology. Hum Pathol. 2017;62:8–12. doi:10.1016/j.humpath.2016.12.027

Strasser

. Delphi method variants in information systems research: taxonomy development and application. Electron J Bus Res Methods. 2017;15:120–133.

Humphrey-Murto

Varpio

Gonsalves

Wood

. Using consensus group methods such as Delphi and Nominal Group in medical education research. Med Teach. 2017;39:14–19. doi:10.1080/0142159X.2017.1245856

Humphrey-Murto

Varpio

Wood

, et al. The use of the Delphi and other consensus group methods in medical education research: a review. Acad Med. 2017;92:1491–1498. doi:10.1097/ACM.0000000000001812

ten Cate

. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013;5:157–158. doi:10.4300/JGME-D-12-00380.1

10.

ten Cate

Chen

Hoff

Peters

Bok

van der Schaaf

Curriculum development for the workplace using Entrustable professional activities (EPAs): AMEE Guide No. 99. Med Teach. 2015;37:983–1002. doi:10.3109/0142159X.2015.1060308

11.

Shorey

Lau

Ang

. Entrustable professional activities in health care education: a scoping review. Med Educ. 2019;53:766–777. doi:10.1111/medu.13879

12.

Hauer

Soni

Cornett

, et al. Developing entrustable professional activities as the basis for assessment of competence in an internal medicine residency: a feasibility study. J Gen Intern Med. 2013;28:1110–1114. doi:10.1007/s11606-013-2372-x

13.

Caverzagie

Cooney

Hemmer

Berkowitz

. The development of entrustable professional activities for internal medicine residency training: a report from the Education Redesign Committee of the Alliance for Academic Internal Medicine. Acad Med. 2015;90:479–484. doi:10.1097/ACM.0000000000000564

14.

Warm

Held

Hellmann

, et al. Entrusting observable practice activities and milestones over the 36 months of an internal medicine residency. Acad Med. 2016;91:1398–1405. doi:10.1097/ACM.0000000000001292

15.

Hart

Franzen

Beeson

, et al. Integration of entrustable professional activities with the milestones for emergency medicine residents. West J Emerg Med. 2019;20:35–42. doi:10.5811/westjem.2018.11.38912

16.

ACGME Pathology Milestones. 2020. Accessed April 15, 2020. https://www.acgme.org/Specialties/Milestones/pfcatid/18/Pathology

17.

Pathology Program Requirements. 2020. Accessed June 3, 2020. https://www.acgme.org/Specialties/Program-Requirements-and-FAQs-and-Applications/pfcatid/18/Pathology

18.

Thomas

. A general inductive approach for analyzing qualitative evaluation data. Am J Eval. 2006;27:237–246. doi:10.1177/1098214005283748

19.

Harris

Taylor

Thielke

Payne

Gonzalez

Conde

. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–381. doi:10.1016/j.jbi.2008.08.010

20.

Harris

Taylor

Minor

, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. doi:10.1016/j.jbi.2019.103208

21.

Rekman

Gofton

Dudek

Gofton

Hamstra

. Entrustability scales: outlining their usefulness for competency-based clinical assessment. Acad Med. 2016;91:186–190. doi:10.1097/ACM.0000000000001045

22.

Swerdlow

Monaghan

Douglas

Johnson

. Harmonization of training, training requirements, board certification, and practice of hematopathology. Am J Clin Pathol. 2019;152:625–637. doi:10.1093/ajcp/aqz084

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.15 MB

0.00 MB

0.14 MB

0.17 MB

0.16 MB