Competencies for Repetitive Transcranial Magnetic Stimulation in Postgraduate Medical Education: Expert Consensus Using a Modified Delphi Process

Abstract

French

Background

Repetitive transcranial magnetic stimulation (rTMS) is recommended in Canadian guidelines as a first-line treatment for major depressive disorder. With the shift towards competency-based medical education, it remains unclear how to determine when a resident is considered competent in applying knowledge of rTMS to patient care. Given inconsistencies between postgraduate training programmes with regards to training requirements, defining competencies will improve the standard of care in rTMS delivery.

Objective

The goal of this study was to develop competencies for rTMS that can be implemented into a competency-based training curriculum in postgraduate training programmes.

Methods

A working group drafted competencies for postgraduate psychiatry trainees. Fourteen rTMS experts from across Canada were invited to participate in the modified Delphi process.

Results

Ten experts participated in all three rounds of the modified Delphi process. A total of 20 items reached a consensus. There was improvement in the Cronbach's alpha over the rounds of modified Delphi process (Cronbach's alpha increased from 0.554 to 0.824) suggesting improvement in internal consistency. The intraclass correlation coefficient (ICC) increased from 0.543 to 0.805 suggesting improved interrater agreement.

Conclusions

This modified Delphi process resulted in expert consensus on competencies to be acquired during postgraduate medical education programmes where a learner is training to become competent as a consultant and/or practitioner in rTMS treatment. This is a field that still requires development, and it is expected that as more evidence emerges the competencies will be further refined. These results will help the development of other curricula in interventional psychiatry.

Keywords

transcranial magnetic stimulation psychiatry competency by design medical education Delphi

Introduction

Transcranial magnetic stimulation (TMS) is a non-invasive neurostimulation procedure delivering magnetic fields to treat psychiatric and neurological disorders. Repetitive transcranial magnetic stimulation (rTMS) is currently recommended as a first-line treatment in the Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016 guidelines for major depressive disorder.¹ There is increasing interest and access to TMS internationally including USA, Brazil, Australia, Israel, Germany, Finland, Serbia and the United Kingdom.² The National Institute for Health and Care Excellence (NICE) guidelines and position statement in 2017 also supports the use of rTMS in the treatment of depression.³ As access to novel neurostimulation strategies is made available, clinicians need to become familiar with treatment options. At the time of this study, access to rTMS is increasing globally and in Canada, 3 out of 10 provinces (Alberta, Quebec and Saskatchewan) and Yukon have publicly funded rTMS interventions. Furthermore, the Health Technology Assessment Committees in British Columbia and Ontario have made formal endorsements for rTMS to be covered under provincial health plans. Once implemented, 90% of the Canadian population will have access to rTMS.⁴

In the Canadian context, psychiatry resident trainees are required to become competent regarding prescribing and delivering electroconvulsive therapy (ECT) as outlined by the Royal College of Physicians and Surgeons of Canada (RCPSC) training requirements.⁵ Similar learning objectives for ECT are outlined for the American Board of Psychiatry and Neurology (ABPN).⁶ As competencies have been developed for ECT, this ensures standardization of training for residents who are receiving teaching despite being at different universities and hospitals. In the new competency-based training curriculum developed by the RCPSC in 2020, there is a requirement to develop knowledge of neurostimulation strategies other than ECT, such as rTMS and deep brain stimulation.⁷ However, it remains unclear how to determine when a resident would be considered competent in being able to apply knowledge of rTMS to patient care. Currently, although there is exposure to rTMS in residency, inconsistencies exist between programmes with regards to training requirements.⁸ For those who wish to practice neurostimulation as part of clinical practice, training and competency standards are not yet defined. For example, although rTMS has been approved for use by the NICE guidelines, specific training requirements regarding indication and delivery have not yet been outlined by the Royal College of Psychiatrists in the United Kingdom in the recent 2022 curricula.⁹

Defining the milestones and competencies for rTMS is important as novel neurostimulation strategies become available and access improves across jurisdictions. Training regarding neurostimulation modalities should be a mandatory component of psychiatric residency. Standardizing training requirements across different training sites using a common list of competencies will be important as competency-based training curricula are implemented across training programmes.¹⁰ The list of competencies will need to differentiate between different stages of training, that is, foundations of discipline, core of discipline or transition to practice.¹¹

This study used a modified Delphi methodology to develop a list of competencies for psychiatry residents in training for rTMS. The Delphi methodology is a widely used technique to develop a consensus from a group of experts.^12,13 In this modified Delphi methodology, statements are first developed by a working group based on current neurostimulation curricula and available literature. The goal of this study is to develop milestones and competencies for rTMS which will be useful in developing a competency-based training curriculum in Canadian postgraduate psychiatry training programmes. Our work may facilitate similar efforts to develop formal training in interventional psychiatry in other jurisdictions.¹⁴

Methods

The development of the competencies for rTMS occurred through a modified Delphi process. This study was approved by the Ontario Shores Research Ethics Board (OS REB #21-015-D).

Development of Steering Committee

The Interventional Psychiatry Competency Diploma (IPCD) Steering Committee was formed at the University of Toronto in May 2021. A working group was led by Dr Amer M. Burhan (Physician-In-Chief and Research Chair at Ontario Shores Centre for Mental Health Sciences) and Dr Robyn Waxman (Medical Head of Brain Stimulation Clinic, Ontario Shores Centre for Mental Health Sciences). The objective of this working group was to define a set of competencies in the practice of rTMS relevant for residents training in general adult psychiatry, and advanced psychiatry trainees in neurostimulation necessary to become a competent consultant in rTMS.

Formulation of Survey

The working group included current rTMS practitioners (RW, AMB) affiliated with Ontario Shores and a senior psychiatry resident (KSPL) to bring the perspective of both current rTMS practitioners and trainees. Current rTMS training requirements were reviewed using the brain medicine fellow and interventional psychiatry curricula available in the United States^15–17 as well as recent work on competency-based education in neurostimulation.¹¹ Competencies were drafted that were relevant for both general adult psychiatry resident trainees and advanced psychiatry trainees in neurostimulation and this list was distributed to rTMS expert panelists for review. None of the members of the working group participated as an expert panelist. The survey items consisted of a questionnaire using a Likert scale ranging from 1 to 5. The survey items were divided into the stages of residency training as outlined by the RCPSC: foundations of training, core of training and transition to practice.¹⁸ The survey included opportunities for experts to enter free-text comments relevant to individual items and to enter suggestions for each statement.

rTMS Expert Panelist Recruitment

rTMS practitioner experts from across Canada were invited to participate in the modified Delphi process. Those who participated were subsequently invited to be co-authors of this manuscript if they agree to provide critical feedback on the drafts of this manuscript. Criteria used to define an “experts” included: practiced rTMS on a regular basis at an academic health sciences centre and have been doing so for at least the past 2 years and has contributed as a clinician educator and/or clinician-researcher in the field. This study included clinicians ranging from the early stages to late stages of their careers. Clinicians were recruited from a variety of sites differing in geographical locations across Canada. Emails were sent to members of the Ontario rTMS network who meet the above criteria to invite them to participate in this study.

Modified Delphi Process

Expert panelists communicated findings using Survey Monkey (https://www-surveymonkey-com-s.web.bisu.edu.cn). Results from the first round of the Delphi process were collected and summarized. For each round of the modified Delphi process, Cronbach's alpha and the intraclass correlation (ICC) were calculated for internal consistency and interrater reliability respectively. Items that had a level of agreement of less than 80% were modified or removed from the subsequent round of the modified Delphi process. Survey items were refined accordingly based on panelist feedback. The modified survey was approved by the working group and then sent to the expert panelists for a subsequent round of the modified Delphi process. A total of three rounds were pursued and has been suggested to be adequate in achieving consensus for a Delphi study.¹² This was set to allow for consensus building over multiple rounds of the survey while minimizing sample fatigue. The final list of survey items consisted of statements that had a rating of at least 80% approval by experts.

Statistical Analysis

Survey items were analysed using IBM SPSS Statistics version 26.¹⁹ Means and SDs of survey items were calculated with a summary of relevant expert feedback to expert panelists.

Cronbach’s alpha was used to assess internal consistency and reliability of survey items among experts.²⁰ Cronbach’s alpha assessed the extent to which items within the scale were intercorrelated for each round of the modified Delphi. Results of the Cronbach’s alpha were interpreted as follows: α ≥ 0.9, excellent; 0.9 > α ≥ 0.8, good; 0.8 > α ≥ 0.7, acceptable; 0.7 > α ≥ 0.6, questionable; 0.6 > α ≥ 0.5, poor; and α < 0.5, unacceptable.²¹

The two-way mixed effects ICC was used to calculate interrater reliability and agreement between experts for each round of the modified Delphi process. A two-way mixed effects model was used given that there are multiple raters making measurements on multiple items in each survey. The two-way ICC is therefore suitable to account for the variability between the multiple raters. ICC values less than 0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability and values greater than 0.90 indicate excellent reliability.²²

Results

Fourteen experts in rTMS were invited to participate in this study. Ten experts consented and participated in all three rounds of the modified Delphi process (Figure 1). The remaining four experts did not reply to the invitation. Table 1 describes the demographic characteristics of the experts. Experts on the panel have been practicing in psychiatry for an average of 23 years and practicing in rTMS for an average of 12 years. Three experts were female. Most experts were practicing in Ontario (N = 7), while others were practicing in Quebec (N = 1), Alberta (N = 1) and British Columbia (N = 1).

Figure 1.

Ten experts consented and participated in all three rounds of the modified Delphi process.

Table 1.

Demographic Characteristics of the Experts (N = 10).

Demographic	N or Mean (SD, Range)
Years of practice in psychiatry	23 (10, 13–41)
Years of practice in rTMS	12 (7, 3–26)
Female	3
Province of practice
- Quebec	1
- Ontario	7
- Alberta	1
- British Columbia	1

Note. rTMS = repetitive trasncranial magnetic stimulation.

There was a total of 21 items in the first formulation of the survey. Two new items were introduced for round 2 of the modified Delphi survey after feedback. These items were related to the mechanism of action of rTMS and identifying contraindications to rTMS. One item related to educating the interdisciplinary team was excluded for concerns about redundancy. Therefore, a total of 22 items were rated for round 2 and round 3. There was a total of 14 items that were reworded over the course of the two rounds. A total of 20 items reached a consensus and 2 items did not reach a consensus. Table 2 shows the statements presented to the expert panelists during round 3 of the modified Delphi survey with mean rating scores and percent agreement.

Table 2.

Statements Presented to the Expert Panelists During Round 3 of the Modified Delphi Survey with Their Mean Rating Scores and Percent Agreement (1 = Strongly Disagree, 5 = Strongly Agree).

Item	Description	Average scores (agreement)
Item	Description	Round 1	Round 2	Round 3
Foundations of discipline (advanced beginner) These competencies are expected for residents who are beginning advanced training in neurostimulation and rTMS. Residents will acquire broad-based competencies in the assessment and planning for patients undergoing neurostimulation.
1	Establish the clinical diagnosis with treatment history and rationale for rTMS.	4.7 (1.0)	4.6 (0.9)	5 (1.0)
2	Decide if rTMS is indicated and appropriate after balancing risks and benefits for the patient.	4.3 (0.8)	5 (1.0)	5 (1.0)
3	Understand the principles and mechanism of action of rTMS delivery in the context of depression and be able to provide relevant psychoeducation to a patient and/or their family members.	-	4.1 (0.7)	4.8 (1.0)
4	Inform the patient about potential benefits, risks and complications of rTMS	4.7 (1.0)	5 (1.0)	4.9 (1.0)
5	Inform patients about the process of rTMS (preparation, treatment and follow-up) and available rTMS resources in their community.	4.4 (0.9)	4.5 (0.9)	4.5 (0.9)
6	Select a patient for rTMS accounting for relative contraindications of rTMS (e.g., metallic implants, presence of scalp metal or deep brain stimulation electrodes).	-	4.8 (1.0)	4.7 (1.0)
7	Prescribe the rTMS protocol for the treatment of depression as per established treatment guidelines.	4.3 (0.9)	4.5 (0.9)	4.6 (1.0)
8	Recommend modifications to psychiatric and nonpsychiatric medications before, during and after rTMS	4.6 (1.0)	4.4 (0.9)	4.2 (0.8)
9	Communicate with referral source regarding appropriateness of rTMS	4.6 (1.0)	4.6 (0.9)	4.7 (1.0)
10	Educate allied health professionals in psychiatry services regarding potential benefits, risks and complications of rTMS.	4.1 (0.7)	4.1 (0.8)	4.4 (0.9)
Core of discipline (competent) These competencies are expected for residents who are developing competency and independence in neurostimulation and rTMS. Residents develop more advanced skills in the delivery of neurostimulation and rTMS in this stage.
11	Perform rTMS consultation to determine the appropriateness of rTMS with minimal complicated comorbidity	4.7 (1.0)	4.7 (1.0)	4.9 (1.0)
12	Plan a course of treatment of rTMS	4.6 (1.0)	4.3 (0.9)	4.8 (1.0)
13	Select appropriate rTMS stimulus parameters and coil placement.	4.7 (0.9)	4.5 (0.9)	4.7 (1.0)
14	Finding motor threshold for treatment.	4.6 (0.9)	4.3 (0.7)	4.2 (0.8)
15	Perform rTMS under direct supervision of rTMS consultant within a multidisciplinary and interprofessional team	4.4 (0.8)	4.4 (0.8)	4.4 (0.8)
16	Manage and triage immediate complications (e.g., seizures) arising from rTMS within the scope of practice of psychiatrists and appropriate to the clinical setting	4.7 (0.9)	4.2 (0.7)	4.7 (1.0)
17	Manage immediate side-effects (e.g., scalp pain, dizziness, etc…) arising from rTMS	4.8 (1.0)	4.6 (0.9)	4.8 (1.0)
18	Recommend relapse prevention strategies after an acute course of rTMS treatment (e.g., Medications, maintenance rTMS)	3.7 (0.7)	4.1 (0.8)	4.2 (0.7)
Transition to practice (advanced and proficient) These competencies are expected for residents transitioning to independent practice and are advanced clinicians in neurostimulation and rTMS. During this stage the trainee demonstrates readiness for autonomous practice.
19	Explore and discuss appropriateness of rTMS for off-label non-unipolar depression related diagnoses (e.g., OCD, PTSD)	4.0 (0.8)	4.1 (0.8)	4.3 (0.9)
20	Explore and discuss appropriateness of rTMS in special clinical populations with complex medical/psychiatric comorbidity	4.2 (0.8)	4.4 (0.8)	4.3 (0.9)
21	Provide education and scholarly contributions in rTMS (teaching, audit or research).	4.1 (0.8)	4.4 (0.9)	4.2 (0.7)
22	Advocate to promote uptake of rTMS in a range of clinical settings (e.g., grand rounds).	4.1 (0.8)	3.8 (0.6)	4.2 (0.8)

Note. rTMS = repetitive trasncranial magnetic stimulation.

There was improvement in Cronbach's alpha over the rounds of modified Delphi process. Cronbach's α = 0.554 in round 1 and improved up to Cronbach's α = 0.824 by round 3 suggesting improvement in internal consistency within the survey items. In addition, the ICC improved from ICC = 0.543 in round 1 to ICC = 0.805 by round 3 suggesting improved interrater agreement over the rounds of the modified Delphi process. Items that benefited from rephrasing included items related to managing complications of rTMS and items related to the technical aspects of delivering rTMS. Items, where there were difficulty in achieving consensus, were related to managing maintenance treatment after rTMS and how to deliver academic contributions related to rTMS.

Calculations of means, SDs and ranges of the expert panel's in response to each statement over the rounds of modified Delphi process show reduction of the SDs and ranges suggesting convergence and stability of agreement among expert panelists (Supplementary Table S1).

Discussion

This study developed and validated competencies for postgraduate medical education programmes where a learner is training to become competent as a consultant and/or practitioner in rTMS using the modified Delphi methodology. The utilization of the modified Delphi process allowed for quantitative results showing improvement in internal consistency and agreement over the rounds of Delphi as well as the opportunity to incorporate qualitative feedback from comments provided by expert panelists in subsequent rounds of the modified Delphi process.

Although broad learning objectives are previously defined in the fellowship programmes reviewed, this study proposes competencies that can be practically implemented into training programmes for rTMS. The implementation of competency-based medical education is expected to better track resident progression status than time-based training programmes as competencies can track both the amount and volume of training, especially with procedural-based tasks.²³ Therefore, a minimum amount of exposure is set to ensure residents are obtaining adequate amount of exposure to a procedure such as rTMS. Implementation of competency-based education also encourages increased direct observation of residents which will improve the quality of feedback provided to learners. Competencies were developed using a competency-by-design framework so that there is a graded level of responsibility in a progressive fashion. Residents in general adult psychiatry training programmes would be expected to successfully complete the competencies in the foundations of discipline stage outlined in Table 2, but advanced psychiatry trainees in neurostimulation would be expected to be competent in the competencies across all stages.

Over multiple rounds of the modified Delphi process, expert panelists expressed concerns about whether the objectives for those competencies can be met based on the limited amount of time allocated for training. For example, for statement 14 “Finding motor threshold for treatment”, experts felt there would not be enough time during training to become competent in the technical skills of rTMS. However, expert panelists felt that this is an important statement to include because motor threshold is the method by which rTMS dose is determined and side effects are managed; therefore, it is an essential competency to achieve that cannot be delegated to others involved in the care delivery such as rTMS technicians.

It is important to clarify that although these are the competencies agreed upon by rTMS experts, these statements may be flexibly interpreted by the learner and the evaluator. For example, statement 4 evaluates the competency to “Inform the patient about potential risks, benefits, and complications of rTMS”. The number of items that could be assessed are quite broad including scalp tenderness, pain, seizures, as well as competency in educating the patient and caregivers.^24,25 This is similar for statement 14, “Manage immediate complications (e.g., Seizure) arising from rTMS” where complications may include seizures and tachycardia.^25,26 It is expected that even though these may be rare complications arising from rTMS, learners should be able to manage and triage this as these complications can be life-threatening. In addition, with statement 15 “Manage immediate side-effects (e.g., Scalp pain) arising from rTMS”, side-effects of rTMS may include headache, syncope, dizziness, facial twitching and tearing.

These competencies would form the basis for the development of entrustable professional activities (EPAs) as per the RCPSC competence by design model. EPAs are the specific tasks that supervisors can observe and evaluate in trainees. One EPA may integrate multiple items from the list of competencies as demonstrated in Table 3 (“Assessment of Eligibility for rTMS”). The EPAs can therefore incorporate multiple CanMEDS roles such as medical expert, communicator, collaborator and leader. The implementation of these competencies as EPAs will likely require further review with psychiatry programmes before further implementation.

Table 3.

Example of an Entrustable Professional Activity (EPA) “Assessment of Eligibility for rTMS”.

	Intervention needed	Direction needed	Support needed	Autonomous	Excellence
	Performs some tasks. Requires considerable direct intervention/involvement	Performs most tasks, requires direct involvement/extensive guidance.	Performs most tasks autonomously. Requires minimal guidance.	Performs all tasks autonomously and competently. Appropriately asks for guidance.	Performs above expected level of training.
Establish the clinical diagnosis with treatment history and rationale for rTMS.
Decide if rTMS is indicated and appropriate after balancing risks and benefits for the patient.
Inform the patient about potential benefits, risks and complications of rTMS
Inform patients about the process of rTMS (preparation, treatment and follow-up) and available rTMS resources in their community.
Select a patient for rTMS accounting for relative contraindications of rTMS (e.g., metallic implants, presence of scalp metal or deep brain stimulation electrodes).
Recommend modifications to psychiatric and nonpsychiatric medications before, during and after rTMS
Communicate with referral source regarding appropriateness of rTMS

Note. rTMS = repetitive trasncranial magnetic stimulation.

There were two items that did not reach a consensus in the modified Delphi process. Item 18 was “Recommend relapse prevention strategies after an acute course of rTMS treatment (e.g., Medications, maintenance rTMS)”. Understandably, the concern for this statement was that evidence remains limited for how to sustain maintenance after an acute course of rTMS. In addition, training regarding relapse prevention is expected to have been completed through general adult psychiatry training. Item 21, “Provide education and scholarly contributions in rTMS (teaching, audit, or research).”, also did not reach a consensus. It was felt that this is not expected or required for all learners, but it is recommended that these opportunities be made available for those residents who wish to pursue this (e.g., Evaluating efficacy or clinical utility of accelerated rTMS protocols or theta burst stimulation).

This study was limited to clinicians who are practicing at academic centres and may be difficult to generalize to learners who are practicing in community-based settings. In addition, there have also been concerns raised about the implementation of competency-based medical education including increased assessment burden and logistical challenges including increased need for observation of learners.²⁷ Therefore, these factors should be considered when applying these competencies as the use of rTMS generalizes and becomes more readily available.

Conclusions

This study was a first attempt at reaching a consensus for postgraduate medical education programmes for residents in general adult psychiatry and for advanced trainees in neurostimulation and rTMS. This is a field that still requires development and research and it is expected, as more evidence emerges in the field of rTMS, that competencies will be updated to reflect new knowledge in the field (e.g., when there is evidence for rTMS in diagnoses other than unipolar depression or maintenance treatment in rTMS). These competencies will be useful in the development of other curricula such as training requirements for an Interventional Psychiatry Diploma.

Supplemental Material

sj-xlsx-1-cpa-10.1177_07067437231164571 - Supplemental material for Competencies for Repetitive Transcranial Magnetic Stimulation in Postgraduate Medical Education: Expert Consensus Using a Modified Delphi Process

Supplemental material, sj-xlsx-1-cpa-10.1177_07067437231164571 for Competencies for Repetitive Transcranial Magnetic Stimulation in Postgraduate Medical Education: Expert Consensus Using a Modified Delphi Process by Ka Sing Paris Lai, Robyn Waxman, Daniel M. Blumberger and Peter Giacobbe, Gary Hasey, Lisa McMurray, Roumen Milev, Lena Palaniyappan, Rajamannar Ramasubbu, Yuri E. Rybak, Tegan Sacevich, Fidel Vila-Rodriguez, Amer M. Burhan in The Canadian Journal of Psychiatry

Footnotes

Author’s Contribution

Lai KSP: conception and study design, data acquisition, analysis/interpretation of data, drafting manuscript, revision of manuscript and final approval of manuscript; Waxman R: conception and study design, data acquisition, analysis/interpretation of data, drafting manuscript, revision of manuscript and final approval of manuscript; Blumberger DM: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; Giacobbe P: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; Hasey G: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; McMurray L: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; Milev R: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; Palaniyappan L: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; Ramasubbu R: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; Rybak Y: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; Sacevich T: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; Vila-Rodriguez F: data acquisition, interpretation of data, revision of manuscript and final approval of manuscript; and Burhan AM: conception and study design, data acquisition, analysis/interpretation of data, drafting manuscript, revision of manuscript and final approval of manuscript

Data Availability Statement

Additional data can be accessed through .

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Ka Sing Paris Lai

Daniel M. Blumberger

Peter Giacobbe

Supplemental Material

Supplemental material for this article is available online.

References

Milev

, et al. Canadian Network for mood and anxiety treatments (CANMAT) 2016 clinical guidelines for the management of adults with Major depressive disorder: section 4. Neurostimulation treatments. Can J Psychiatry. 2016;61(9):561–75.

Brem

Lehto

SM.

Stuck between bench and bedside: why non-invasive brain stimulation is not accessible to depressed patients in Europe

. Front Hum Neurosci. 2017;11:39.

Psychiatrists R.C.o. Statement on Repetitive Transcranial Magnetic Stimulation for Depression . 2017; Available from: https://www.rcpsych.ac.uk/docs/default-source/about-us/who-we-are/ectcommittee-repetative-transcranial-magnetic-stimulation-statement-may18.pdf?sfvrsn=695e93be_2.

Canada

Table 17-10-0009-01 Population estimates, quarterly . 2022; Available from: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1710000901.

Canada R.C.o.P.a.S.o. Specialty Training Requirements in Psychiatry . 2015 [cited 5 April 2021]; Available from: https://www.royalcollege.ca/rcsite/documents/ibd/psychiatry_str_e.pdf.

Neurology A.B.o.P.a. American Board of Psychiatry & Neurology Psychiatry Core Competencies Outline . 2022; Available from: https://www.abpn.com/access-residency-info/residency-training-information/psychiatry/.

Canada R.C.o.P.a.S.o. Psychiatry Competencies . 2020 [cited 5April 2021]; Available from: https://www.royalcollege.ca/rcsite/documents/ibd/psychiatry-competencies-e.pdf.

Giacobbe

, et al.

Interventional psychiatry: an idea whose time has Come?

Can J Psychiatry. 2021;66(3):316-18.

Psychiatrists R.C.o. Curricula documents and resources . 2022; Available from: https://www.rcpsych.ac.uk/training/curricula-and-guidance/curricula-implementation/curricula-documents-and-resources.

10.

Canada R.C.o.P.a.S.o. Competence by design: reshaping Canadian medical education . [cited April 2021]; Available from: http://www.royalcollege.ca/rcsite/educational-initiatives/cbd-medical-education-ebook-e.

11.

Danilewitz

, et al.

Towards competency-based medical education in neurostimulation

. Acad Psychiatry. 2020;44(6):775-78.

12.

Hasson

Keeney

McKenna

. Research guidelines for the delphi survey technique. J Adv Nurs. 2000;32(4):1008-15.

13.

Trevelyan

Robinson

. Delphi methodology in health research: how to do it? Eur J Integr Med. 2015;7(4):423-28.

14.

Trapp

Williams

. The future of training and practice in neuromodulation: an interventional psychiatry perspective. Front Psychiatry. 2021;12:734487.

15.

Medicine S. Interventional Psychiatry Fellowship Program . 2021 [cited 2021]; Available from: https://med.stanford.edu/psychiatry/education/interventionalpsych.html.

16.

Florida U.o. Neuromodulation Fellowship . 2022 [cited 2022; Available from: https://psychiatry.ufl.edu/training/fellowships/neuromodulation-fellowship/.

17.

Hospital M.G. Interventional Neuropsychiatry & Neuromodulation Fellowship . 2021 [cited 2021]; Available from: https://www.massgeneral.org/psychiatry/education-and-training/interventional-neuropsychiatry-and-neuromodulation-fellowship.

18.

Canada R.C.o.P.a.S.o. Competence by Design . 2022; Available from: https://www.royalcollege.ca/rcsite/cbd/competence-by-design-cbd-e.

19.

Corporation I. IBM SPSS Statistics for Windows. Armonk, NY: IBM Corporation

;

2015.

20.

Bland

Altman

. Cronbach's alpha. Br Med J. 1997;314(7080):572.

21.

Tavakol

Dennick

. Making sense of cronbach's alpha. Int J Med Educ. 2011;2:53=55.

22.

Koo

. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-63.

23.

Potts

3rd . Assessment of competence: the accreditation council for graduate medical education/residency review committee perspective. Surg Clin North Am. 2016;96(1):15-24.

24.

Humaira

, et al.

A patient-oriented analysis of pain side effect: a step to improve the patient's experience during rTMS?

Brain Stimul. 2021;14(5):1147-53.

25.

Dobek

, et al.

Risk of seizures in transcranial magnetic stimulation: a clinical review to inform consent process focused on bupropion

. Neuropsychiatr Dis Treat. 2015;11:2975-87.

26.

Dobek

, et al.

Sinus tachycardia induced by methocarbamol and repetitive transcranial magnetic stimulation (rTMS)

. Brain Stimul. 2016;9(1):156-58.

27.

Mann

, et al.

Resident perceptions of competency-based medical education

. Can Med Educ J. 2020;11(5):e31–e43.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB