Criteria Used by Endourology Society Fellowship Program Directors for the Selection and Evaluation of Fellows

Abstract

Introduction:

An understanding of which trainee factors Endourology Society (ES) fellowship program directors (PDs) value may help urology residents optimize their fellowship candidacy and help current fellows optimize their fellowship performance. In the present study, we evaluated which factors are most important to ES PDs in selecting fellowship candidates and evaluating current fellows.

Materials and Methods:

A survey was constructed using Research Electronic Data Capture (REDCap) software and emailed to PDs of fellowships registered with the ES. Fellowships in the ES include Endourology/kidney stone disease (ESD) programs, robotic-laparoscopic surgery programs (MIS), and combination programs. A Likert scale ranging from 1 (not important) to 5 (extremely important) was used to rank the criteria for selecting and evaluating fellows. The survey captured program demographics such as geographic region, program type, and program duration.

Results:

Of the 52 ES PDs, 35 (67%) responded. Respondents represented 8 (23%) ESD programs, 7 (20%) MIS programs, and 20 (57%) combination programs. Furthermore, 16 (46%) represented 1-year programs and 19 (54%) represented 2-year programs. The top-five factors for candidate selection were perceived fit of the applicant, perceived level of applicant interest, initial interview, personal emails from applicants' advocates, and personal phone calls from applicants' advocates. Advocacy on behalf of the applicant appeared to be especially important for 2-year fellowships. The top-five factors for the evaluation of fellows in training were patient interactions, professionalism, attitude/demeanor, operative skills, and interactions with ancillary staff. Research productivity was of greater importance for the evaluation and selection of 2-year fellows and ESD fellows.

Conclusions:

Applicants for ES fellowships should focus on the initial interview and recruiting supportive mentors to advocate for their applications, particularly for 2-year programs. Although PDs value both clinical skills and research productivity, research productivity was more important for 2-year programs and ESD programs. Further research into applicant perspectives on the fellowship application process is warranted.

Introduction

After completion of a urology residency training program, graduates may enter independent practice or elect to pursue additional specialized training via a fellowship. Urology fellowship disciplines include endourology/kidney stone disease (ESD), laparoscopic and robotic surgery (MIS), andrology/infertility, urologic oncology, pediatric urology, female pelvic medicine and reconstructive surgery, and genitourinary reconstructive urologic surgery. Typically, applications for fellowship are compiled and submitted early in the penultimate year of training. Although there are some minor differences, the applications for each discipline are similar and include letters of recommendation, a curriculum vitae, and an interview. In addition to providing letters of recommendation, applicants' advocates may make personal phone calls or write personal emails in support of the applicant. Despite the increasing popularity of urology fellowships,¹ little is known about which factors are most important to program directors (PDs) in selecting applicants.

Two common urology fellowships are ESD and MIS.² The majority of these fellowships are certified through the Endourology Society (ES). In total, there are three ES fellowship disciplines: ESD, MIS, and combination ESD/MIS. Each discipline has a 1-year track (officially called an “advanced clinical training program”) and a 2-year track (a formal “fellowship”), making for a total of six fellowship tracks.³ As with the other urology fellowships, the candidate attributes coveted by PDs for these fellowships are unknown. Furthermore, no studies have evaluated how fellows within these programs are evaluated by PDs.

A greater understanding of which trainee factors ES fellowship PDs value may help guide urology residents in optimizing their candidacy for fellowship and help current fellows optimize their fellowship performance. Thus, in the present study, we sought to study which factors are most important to ES PDs in selecting fellowship candidates and evaluating current fellows via a PDs survey. Given that most ES programs are in North America (52 of 71 programs in 2020 were in North America, of which 50 were based in the United States and two based in Canada), we limited the scope of our survey to North American ES programs. In addition to evaluating ES programs, we evaluate differences between the six different ES fellowship tracks. To our knowledge, this is the first such study for any urology fellowship.

Materials and Methods

Survey design

Our study was reviewed by our Institutional Review Board (IRB) and received an IRB exemption. We utilized our institution's Research Electronic Data Capture (REDCap) software to generate an anonymous survey for distribution to ES fellowship PDs. The main portion of the survey asked the following two questions: “How important are the following factors in your candidate selection criteria” and “How important are the following factors for the evaluation of your fellows.” A 1–5 Likert scale (1 = not important, 5 = extremely important) was utilized for responses.

Criteria included in the candidate selection question included written letters of recommendation, personal phone call(s) from the applicant's advocates, personal email(s) from applicant's advocate/mentors, initial interview, second look/site visit, research experience, number of publications, number of presentations at national meetings, perceived level of applicant interest, postvisit “thank you email,” geographic locations of the applicants residency training program, familiarity with the applicant's residency program, familiarity with the applicant's advocates, perceived quality of the applicant's residency training program, perceived fit of the applicant within your fellowship/institution, perceived quality of the applicant's medical school, perceived quality of the applicant's undergraduate institution (college), other advanced degrees apart from MD/DO, and the applicant's interest in pursuing academic urology. Criteria utilized in the current fellow evaluation question included overall perception of the fellow's research productivity, number of publications, number presentations at national meetings, fund of knowledge, operative skills, attitude/demeanor, interactions with ancillary staff, professionalism, and patient interactions.

In addition, our survey asked questions pertaining to the individual fellowship programs, including fellowship duration (1-year vs 2-year; programs with both tracks were considered 2-year), fellowship discipline (Endourology/kidney stone disease [ESD], robotic-laparoscopic surgery programs [MIS], or a combination), number of fellows per year, American Urological Association (AUA) section of the program, amount of dedicated research time, average number of applicants interview per year, how often fellows are evaluated, career trajectory of past graduates, and the existence of a restrictive covenant for graduates. We also inquired about the PDs' attitudes on internal institutional applicants and applicants from other local training programs (positive, negative, or neutral). Finally, we provided a free text space with the prompt “Do you have any additional comments.”

Survey distribution

Using the official ES website (www.endourology.org/fellowships), we obtained the email addresses for the PDs or associate PDs of all 52 North American programs. An email describing our study and the nature of the survey with a link to the survey was sent to each PDs or associate PDs (one email per program, associate PDs were only emailed if a PD's email address was unavailable). The survey was open for 1 month starting on January 30th, 2021.

Data analysis

Given the small overall number of ES fellowships, we anticipated that the ability to do meaningful statistical analyses would be limited, even with a high respondent rate. Accordingly, rather than perform discrete statistical tests, we analyzed our data in a descriptive manner. Centered bar charts showing the percentage of responses for each question for each of the respective Likert question sections were created with items listed in order of most agreement to least agreement. This analysis was performed for all respondents. In addition, density plots were created to compare responses to each of the Likert-rated questions, by length of fellowship program (1-year vs 2-year), and by subdiscipline of fellowship program (ESD vs MIS vs combo). This analysis only included respondents who provided complete data for every question. The survey results were plotted using the “Likert” package for R statistical software v3.6.2.^4,5

Results

Program demographics and responses to non-Likert questions

In total, 35/52 (67%) of ES PDs participated in the survey and 33/52 (63%) provided compete data for every question (not counting the free text “additional comments” question). Program demographics and responses to non-Likert questions are summarized in Table 1. The breakdown of respondents by program discipline is as follows: 8 ESD PDs (23%), 7 MIS PDs (20%), and 20-combination PDs (57%). The breakdown of respondents by year was 16 (46%) 1-year PDs and 19 (54%) 2-year PDs.

Table 1.

Program Demographics and Responses to Non-Likert Questions

	Program demographics	No. of programs (%)
Response rate	Total number of ES programs	52
	No. of programs that responded	35/52 (67%)

Section of North American AUA	New England	3/35 (8.6%)
	New York	4 (11.4%)
	Mid Atlantic	3 (8.6%)
	North Central	7 (20%)
	South Eastern	8 (22.9%)
	South Central	2 (5.7%)
	Western	8 (22.9%)
	North Eastern	0

Type of program	Endourology stone disease programs	8/35 (23%)
	Minimally invasive surgery programs	7/35 (20%)
	Combination programs	20/35 (38.4%)

No. of years in program	1-year programs	16/35 (45.7%)
	2-year programs	19/35 (54.3%)

No. of fellows per year	One fellow per year	30/35 (85.7%)
	Two fellows per year	4/35 (11.4%)
	More than four fellows per year	1/35 (2.9%)

Dedicated research block	No dedicated research block	26/35 (74.3%)
	1–6 months of research block	4/35 (11.4%)
	7–12 months of research block	5/35 (14.3%)

No. of candidates interviewed	1–3	3/35 (8.6%)
	3–6	7/35 (20%)
	6–9	16/35 (45.7%)
	10–12	5/35 (14.3%)
	12–15	2/35 (5.7%)
	15+	2/35 (5.7%)

No. of protected research days per week	0.5	10/35 (28.6%)
	1	14/35 (40%)
	1.5	4/35 (11.4%)
	2	1/35 (2.9%)
	2.5	2/35 (5.7%)
	3	1/35 (2.9%)

AUA = American Urological Association; ES = Endourology Society.

Programs from all North American AUA sections were represented except for the Northeast section. Of note, the Northeastern and Western sections of the AUA include Canadian provinces. The vast majority of respondents (30/35) had one fellow per year in their programs. In 26 programs (74%), the fellow did not have a dedicated research block, while in four programs the fellow had 1–6 months of dedicated research block-time, and in five programs the fellow had 7–12 months of dedicated research block-time. During clinical blocks, in most programs, fellows have either half a day (10 programs, 26%) or a full day (14 programs, 40%) of protected research time per week. Most programs interview between 4 and 12 applicants per year and no programs had restrictive covenants. Free text responses to the “additional comments” question are summarized in Supplementary Table S1.

Candidate selection

Overall, the top-five factors for candidate selection were “perceived fit of the applicant within the program/institution” (mean Likert score 4.45), “perceived level of interest from the applicant” (mean Likert score 4.41), initial interview (mean Likert score 4.42), written letters of recommendation (mean Likert score 4.22), and personal phone calls from applicants' advocates (mean Likert score 4.08). Notably, 91% found that “perceived fit of the applicant within your fellowship program/institution” was important for selecting an endourology fellow (Fig. 1).

FIG. 1.

Importance of criteria for fellowship candidate selection as rated by fellowship program directors. Color graphics are available online.

The importance of perceived fit was consistent between both 1- and 2-year programs, with 87.5% of 1-year programs and 94.7% of 2-year programs reporting perceived fit of the applicant to be important when selecting fellows (Fig. 2). Regarding differences between 1- and 2-year programs, written letters of recommendation for fellowship candidates were considered important or extremely important, for 68.75% of 1-year programs and 89.5% of 2-year programs. In addition, personal phone calls were rated as important or extremely important to 68.75% of 1-year PDs and 84.2% of 2-year PDs (Fig. 2).

FIG. 2.

Importance of criteria for fellowship candidate selection as rated by fellowship PDs, sorted by 1- and 2-year programs. PDs, program directors. Color graphics are available online.

The least important factors in fellow selection included geographic location of applicant's residency training program (mean Likert score 1.57), perceived quality of the applicant's undergraduate institution (mean Likert score 1.79), and other advanced degrees apart from MD/DO, MPH, PHD, and MBA (mean Likert score 2.48). The least important factors were perceived similarly by 1- and 2-year programs. A comparison of the factors affecting candidate selection between ES fellowship discipline (MIS vs ESD vs combo), is presented in Figure 3. ESD fellowships tended to place the greatest emphasis on academics and research as demonstrated by a greater degree of interest in applicants' research experience, academic career goals, and other degrees outside of MD/DO.

FIG. 3.

Importance of criteria for fellowship candidate selection as rated by different program disciplines. Color graphics are available online.

Fellow evaluation

Overall, the top five factors for the evaluation of fellows in training were patient interactions (mean Likert score 4.80), professionalism (mean Likert score 4.83), attitude/demeanor (mean Likert score 4.83), operative skills (mean Likert score 4.63), and interactions with ancillary staff (mean Likert score 4.63) (Fig. 4). Furthermore, 100% of respondents from both 1- and 2-year programs perceived patient interactions, professionalism, attitude, and operative skills as important (Fig. 5). However, the responses from 1- and 2-year programs differed with regard to how important they considered research productivity when evaluating fellows. Indeed, 74% of 2-year PDs considered number of publications important or extremely important for evaluating current fellows, compared to 38% of 1-year PDs.

FIG. 4.

Importance of criteria for evaluating current fellows, as rated by fellowship program directors. Color graphics are available online.

FIG. 5.

Importance of criteria for evaluating current fellows, as rated by fellowship PDs, sorted by 1- and 2-year programs. Color graphics are available online.

Similarly, 79% of the 2-year PDs rated overall perception of fellows' research productivity important or extremely important compared to only 54% of 1-year PDs. As with candidate selection criteria, ESD fellowship directors placed greater emphasis on research and academics in evaluating current fellows as demonstrated by greater Likert scores for number of publications, overall perception of research productivity, and number of presentations at national meetings compared to MIS fellowship directors (Fig. 6).

FIG. 6.

Importance of criteria for evaluating current fellows as rated by different program disciplines. Color graphics are available online.

Discussion

ES fellowships are competitive as demonstrated by a 77% match rate in the 2021 match.⁶ Yet, there is a paucity of literature regarding the characteristics PDs seek in their prospective fellows and current fellows. To this end, we have conducted the first study elucidating the criteria for evaluating fellowship candidates and current fellows by surveying ES fellowship PDs throughout North America. To our knowledge, this is the first survey of PDs preferences for any urology fellowship.

We found that the most important factors in selecting prospective fellows related to applicant interest, perceived “fit,” initial interview, and support from the applicant's advocates. This was similar in both 1- and 2-year fellowships. Notably, ESD fellowship directors placed greater emphasis on academic inclination and research productivity, suggesting that applicants for such fellowships should work to bolster their academic credentials throughout residency. Our findings are similar to that of Weissbart et al. who surveyed urology residency PDs and found urology references to be among the most important selection criteria.⁷ However, one key difference is that for urology residency selection, United States Medical Licensing Examination (USMLE) scores and medical school performance (class rank and grades) were important. Although urology residents sit for the annual AUA ISE and receive rotation evaluations, these metrics are not included in the current ES Fellowship application.

Other specialties have also assessed fellowship selection criteria. A survey of surgical oncology fellowship directors found the applicant interview to be the most important factor in selecting fellows.⁸ A study of pediatric emergency medicine fellowship directors found that recommendations from colleagues was the most important factor in granting an interview.⁹ A similar survey of hand surgery fellowship PDs also found letters of recommendation from colleagues to be the most important factor in selecting fellows.¹⁰ Our findings are consistent with other specialties as we found support from applicants' advocates, and initial interview to be among the most important criteria for evaluating applicants.

Our study also assessed ES fellowship PDs perspectives on evaluating current fellows. The most important criteria in evaluating current fellows are related to clinical care (e.g., patient interactions, professionalism, and operative skills). The least important factors in evaluating current fellows are related to research productivity (e.g., perception of research productivity, number of publications, and number presentations at national meetings). That is not to say that research productivity was not important; the vast majority of PDs rated research-related criteria for current fellows as somewhat important. Furthermore, ESD fellowship PDs tended to place more weight on research productivity. However, as would be expected with a clinical training program, clinical skills are the main barometer of trainee success.

Our study has several limitations that should be noted. Our sample size was small and thus precluded any meaningful statistical comparisons. This is an inherent feature of our study population as there are only 52 ES fellowship PDs; our respondent rate was quite high (67%). Accordingly, the differences noted on our subanalyses of program types are observational and may be due to random variance. Furthermore, it should be noted that PDs self-reporting has inherent subjective bias and may not reflect actual fellow selection and evaluation practices. Despite these limitations, we believe our study to be an important contribution to the literature given that it is the first study of fellowship director preferences for any urology fellowship and the study's high respondent rate.

In summary, urology residents interested in pursuing an ES fellowship should identify strong advocates, hone interview skills, and identify programs that are potential good “fits.” Furthermore, those interested in ESD fellowships should aim to bolster research productivity in residency. Ultimately, every story has two sides, and further research to evaluate the perspectives of endourology fellowship applicants and current fellows on the fellowship application process is warranted and is an important avenue for future research.

Conclusions

We have conducted the first study of fellowship PDs preferences regarding the evaluation of applicants to and current trainees in ES fellowships. This is the first such study of any urology fellowship. The most important factors in applicant selection related to applicant interest, perceived “fit,” initial interview, and support from the applicant's advocates. The most important factors in the evaluation of current fellows related to clinical practice. ESD PDs played particular emphasis on research productivity for both applicants and current fellows. Urology residents seeking to apply for an ES fellowship should identify strong advocates, practice interview skills, and aim to identify programs that are good “fits.” In addition, residents interested in ESD fellowships should engage in research early in residency to bolster research productivity. Further studies to gain insight in applicant and current fellow perspectives are warranted.

Footnotes

Authors' Contributions

K.G., J.A.K., B.G., and M.G. contributed to the design, the implementation of the research, and writing of the article. D.J.L. helped with the analysis of the results. A.S.S. and W.M.A. also helped with the writing of the article.

Author Disclosure Statement

Dr. Mantu Gupta is compensated for educational training for Cook Urological Incorporated, Boston Scientific Incorporated, Olympus Incorporated, Lumenis Incorporated, and Retrophin Incorporated. In addition, Dr. Gupta is the Editor in Chief for Video Urology. No other authors have any relevant conflicts of interest to disclose.

Funding Information

No funding was received for this article.

Supplementary Material

Supplementary Table S1

Abbreviations Used

References

American Urological Association. 2019 Urologists in Training: Residents and Fellows in the United States. 2020. https://www.auanet.org/research/research-resources/aua-census/census-results (Last accessed August 20, 2021 ).

Welk

, Kodama

, Macneily

. The newly graduated Canadian urologist: Over-trained and underemployed?. Can Urol Assoc J, 2013; 7:E10–E15.

Endourological Society. Requirements for Endourology Fellowship training. 2021. https://www.endourology.org/fellowships-alias/program-requirements (Last accessed August 20, 2021 ).

Bryer

, Speerschneider

. Likert: Analysis and Visualization Likert

Items

. R package version 1.3.5. https://CRAN.R-project.org/package=likert (Last accessed August 20, 2021 ).

R Core

Team

. A language and environment for statistical computing. R Foundation for Statistical

Computing

. Vienna

Austria

. 2021. https://www.R-project.org (Last accessed August 20, 2021 ).

American Urological Association. Match Statistics-2021 Endourology Fellowship Match. Match Statistics-2021 Endourology fellowship match. 2021. https://www.auanet.org/documents/education/specialty-match/Match%20Statistics%20-%202021%20Endourology%20Fellowship%20Match.pdf (Last accessed August 20, 2021 ).

Weissbart

, Stock

, Wein

. Program directors' criteria for selection into urology residency. Urology, 2015; 85:731–736.

Wach

, Ayabe

, Ruff

, et al. A Survey of the Complex General Surgical Oncology Fellowship Programs Regarding Applicant Selection and Rank. Ann Surg Oncol, 2019; 26:2675–2681.

Poirier

, Pruitt

. Factors used by pediatric emergency medicine program directors to select their fellows. Pediatr Emerg Care, 2003; 19:157–161.

10.

Nies

, Bollinger

, Cassidy

, Jebson

PJL

. Factors used by program directors to Select Hand Surgery Fellows. J Hand Surg, 2014; 39:2285–2288.e5.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB