Development Methodology of the Novel Endoscopic Stone Treatment Step 2/A Training/Assessment Curriculum and a Roadmap on Developing Hands-on Training Curriculums in Future: An International Collaborative Work by European Association of Urology Sections

Abstract

Background:

Basic simulation training in endourology was established with the endoscopic stone treatment step 1 (EST-s1), which is now recognized worldwide for training and examination. Following on from EST-s1, the endoscopic stone treatment step 2 (EST-s2) was started by the European Association of Urology (EAU) sections.

Objective:

We describe the methodology used in the development of EST-s2 assessment curriculum.

Materials and Methods:

The “full-life cycle curriculum development” template was followed for curriculum development, focusing on intermediate training of EST protocol with complex endourologic tasks. A cognitive task analysis (CTA) was run in accordance with EAU Urolithiasis guidelines. The protocol and its details underwent a first consensus by Delphi method with EAU Urolithiasis Section experts in March 2017. Once the outcome and metrics were decided, curriculum development was carried out. Purpose-built stones were developed, and simulator system requirement was defined. Preliminary testing was done in European Urology Residents Education Programme 2019 and in phase five the protocol was finalized with full tutor instruction sheet.

Results:

The EST-s2/A curriculum development took 38 months and involved EAU Uro-technology and urolithiasis sections with coordination from the European School of Urology training group. Starting from the initial CTA, a 1277-word revision with preliminary task description was produced. Nine intermediate skills were identified and included in the final training protocol. The training content and session evaluations were carried out by 26 experts and 16 final year trainees, respectively. Although the experts agreed that EST-s2/A protocol was well structured (96%), covered the complex endourologic maneuvers (92%), and was useful to optimize and improve hands-on-training (HoT) sessions (92%), the overall evaluation was scored 4.25/5 by trainees.

Conclusion:

We describe the development methodology for intermediate EST curriculum, which also provides a roadmap on developing other HoT protocols in future.

Patients Summary:

In this report we described the development of the novel intermediate training curriculum for EST, called EST-s2, which took 3 years of collaborative work inside the EAU. This article is aimed to strengthen the standards in curriculum development and clearly describe the background of this new EAU official endourology protocol.

Introduction

In the past decade urologic training for practical skills has undergone a dramatic rearrangement and transformation. Since the first time European Basic Laparoscopic Urological Skills¹ was delivered in 2011, heavy standardization has been introduced to ensure high-quality teaching and assessment on a global scale. Between the most relevant innovations were the definition of the ideal hands-on training (HoT) methodology² and of the modular endoscopic HoT template³ divided into three steps: basic, intermediate, and advanced training. This novel three-step arrangement allowed to define how to deal with existing and future training protocols, giving a consequentiality and a sense to all of them in the training journey.

Endoscopic Stone Treatment step 1 (EST-s1) was conceived as the first part of the EST training system. EST-s1 required a 3-year development,⁴ involving three sections of the European Association of Urology (EAU). With its validation,⁵ it became the official EAU basic training protocol for stone treatment,⁶ reaching 40 countries through training or examination activities. Following the great enthusiasm that surrounded EST-s1 and to proceed with the delivery of the full modular system described (Fig. 1), in 2016 the development of EST-step 2 started and is today in its final phases before validation and widespread use.

FIG. 1.

Modular HoT template for EST educational system. EST = endoscopic stone treatment; HoT = hands-on training.

Objective

According to the latest concept inspired by Messick's framework of validity,^7,8 one of the main points of validation is the development process. Our aim was to describe the methodology followed to develop the Endoscopic Stone Treatment step 2 (EST-s2), the second step of the modular training/assessment curriculum for EST. Our goals included the development of a set of replicable standardized low-cost exercises, providing objective assessment and applicability to 60-minute HoT sessions.

Materials and Methods

The “full life cycle curriculum development” template (Table 1), described by Richard Satava⁹ and previously applied to EST-s1, was also followed in EST-s2. Following this process allowed us to produce our protocol in line with the Proficiency Based Progression training standards as described by Satava and Gallagher.⁹ The process starts by defining the outcomes and metrics and ends with the certification that planned outcomes were achieved. The protocol was developed since October 2016 and is now ready for validity analysis. The study was conducted without the need for funding as simulators and tools were provided for research purposes, free of charge.

Table 1.

Full Life Cycle Curriculum Development Template

WHAT	Outcomes and metrics	Curriculum development	Simulator development	Validation studies	Implement survey training certification	Issue certification
HOW	Consensus conference	Standard curriculum template	Engineering physical simulator	Standard validation template	Current procedures	Issue mandates and certificates
EST-s2	October 2016 CTA re-evaluation	March 2018 task design	July 2018 (AiF) task consensus June 2019 stone dev	September 2020 (EUREP) data collection	January 2021 (SET-UP) final tests	September 2021 (EAU21) first planned examinations
EST-s2	March 2017 (EULIS 2017) consensus step 2A/B	May 2018 (ESUT18) laser testing (regulations)	September 2019 (EUREP) task testing December 2019 (AiF) stone testing	December 2020 (AiF) data collection

AiF = art in flexible; CTA = cognitive task analysis; EAU = European Association of Urology; EST-s2 = endoscopic stone treatment step 2; EULIS = EAU Urolithiasis Section; EUREP = European Urology Residents Education Programme; SET-UP = Standardized Endoscopic Training in Uro Procedures.

Phase 1: Outcomes and metrics

In accordance with the Modular HoT system described by Veneziano and Hananel,³ the step 2 for intermediate training of the EST protocol keeps its focus on complex tasks: “the most challenging steps of full procedures.” Cognitive Task Analysis (CTA) was run by the Endourology and Stone Treatment group of the Young Academic Urologist Working Party for EST-s1 and re-evaluated in 2016 for EST-s2. It integrated all the information needed for the development of the step 2 protocol. Indeed, the CTA document deconstructed and analyzed the details of retrograde intrarenal surgery (RIRS), including simple maneuvers (step 1) and/or complex tasks (step 2). CTA was run in accordance with the EAU Urolithiasis guidelines¹⁰ and in parallel with a focused updated literature review, to avoid discrepancies.

Preoperative procedural continuous variables and completion details were analyzed in relation to each complex step of the technique: ureteral stone fragmenting and retrieval, basketing and relocation of lower pole stone, kidney stone dusting and inspection of the pelvicaliceal system after the treatment. CTA¹¹ was used during the process to set the equipment needed, the procedural steps, errors and goals, and the requirements for simulator development. This protocol and its details underwent a first consensus by Delphi method with 21 EAU Urolithiasis Section (EULIS) experts in March 2017. During this meeting the board unanimously decided to divide step 2 in two skill subsections: “EST-s2/A” for EST and “EST-s2/B” for percutaneous stone treatment.

During the following timeframe, tasks for EST-s2/A were preliminarily defined.

Phase 2: Curriculum development

The development of EST-s2/A protocol was carried out in strict accordance with the CTA (2016 revision) and was aimed for complex ureteroscopic skills training.

Training tasks were created from March 2018 to May 2018, with minor further modifications required after preliminary testing in July 2018 and September 2019.

After detailed evaluation of the CTA, the first set of exercises were as follows: task 1, Mid-ureteral laser/ballistic stone fragmentation and clearance; task 2, Laser fragmentation of interpolar-calix stone; task 3, Fragment relocation and retrieval; task 4, Stent placement. After a preliminary definition of the tasks, these underwent the first test during European Section of Uro-Technology (ESUT) meeting in May 2018. The aim was to understand applicability of the training curriculum to conferences and dry-laboratory events, especially in relation to usage of laser devices in nondedicated areas and to standardization of available stone models. This preliminary test involved expert tutors for feedback and laser-device companies to help strategize laser settings.

This test was crucial to define some regulatory rules to be applied to safely run the new protocol:

Protective glasses for both tutor and trainees involved

Boundary panels to delineate the training area for laser safety

Water-filled simulators to avoid excessive heat buildup

Standardized stones to ensure consistent and reliable results during training/assessment sessions.

Companies involved were then provided with the safety and regulatory information, to arrange the training sessions accordingly.

In July 2018 the full protocol was tested again, in accordance with the novel regulations, during the Art in Flexible (AiF) event in Berlin. On this occasion eight expert tutors with at least 4 years of teaching experience for the EAU were involved in the design of synthetic stones, which would be manufactured, specifically designed for EST-s2.

Phase 3: Simulator development

Stones

In June 2019 the development of dedicated purpose-built stones was started.

Following the requirements set after the consensus in July 2018, two different custom-made spherical stones were developed by Medics 3D (Turin, Italy) (Fig. 2):

FIG. 2.

Synthetic stones (A and B).

Stone A: 7 mm diameter, brown coating, hard consistency, designed for fragmentation in task 1.

Stone B: 1 cm diameter, yellow outer coating and internal 3 mm core, designed for dusting during task 3.

Stone B was designed with an internal core, to be released by dusting, and removed by basketing, to allow objective task assessment. Both stones were created by three-dimensional (3D) printing a resin mold, and then casting custom-made materials. The preliminary core of stone B was 3D printed using polylactic acid (PLA). Because of melting at lasing temperatures, the PLA cores were there replaced by aluminum 3 mm spheres, easy to find for online purchase. Testing of stones was performed in December 2019 with the aim of checking the feasibility of these customized products, in relation to the replicability of the tasks. Laser device used was a holmium 30 to 35 W laser system by Quanta System S.p.A. (Milan, Italy). The laser model chosen for tests was purposely selected to be of low power, to avoid settings that could not be replicable in other machines. Settings used for fragmentation were 0.6 to 0.8 J, 6 to 8 Hz, whereas those for dusting were 0.4 to 0.5 J, with frequency ranging from 15 to 20 Hz in a long-pulse mode, with 272 μm laser fibers. For the test, five stones of each model were placed inside a test tube filled with saline and then treated. Weight of the stone was measured before and after treatment with an electronic precision device. Laser fiber was fixed vertically over the stone on a custom-made support with the tip at 1 mm from the stone surface (Fig. 3).

FIG. 3.

Stone lasing test.

During the test, it was noted that the core placed inside stone B, developed in 3D-printed PLA, was not appropriate as the material often melted and deformed because of the high lasing temperature. After the test, it was replaced by a 3 mm metal sphere that resolved this problem.

Simulation system

In accordance with the task requirements, the simulator for EST-s2/A needed to be equipped with a navigable low-friction urethra, a 1:1 dimensioned, water-tight bladder, navigable low-friction ureter with operative access for easy stone placement and cleaning, navigable 1:1 scaled kidney with internal calices and easy access for stone placement. The system had to resist high operating temperatures and needed to be completely filled with water to allow proper working of laser units.

For development and testing the only simulator available on the market that met the aforementioned requirements was the Endo-Uro Trainer (Samed, Dresden, Germany), previously adopted as one of the options for the EST-s1 curriculum. An alternative to this model was provided by Cook Medical (Bloomington, IN), during the test phases in July 2018, along the AiF event by the European School of Urology (ESU).

Phase 4: Preliminary test

In September 2019 the whole preliminary protocol underwent the first full test during European Urology Residents Education Programme (EUREP) 2019. This test was used to check the applicability of the curriculum to a standard 60-minute HoT session, involving one tutor and two trainees in accordance with the previous EUREP training format.² The station was equipped with cystoscope, semirigid ureteroscope, flexible ureteroscope, guidewires, and access sheath by Olympus (Olympus Europa SE & CO. KG, Hamburg, Germany). The simulator used was an Endo-Uro Trainer (Samed), modified in accordance with the EST-s1 requirements.

The preliminary HoT step-2/A protocol was delivered 16 times on the provided station, with an expert dedicated tutor (minimum 4 years of teaching experience in similar protocols). Residents who previously achieved EST-s1 certification participated in it. Likert scale-based quality feedback questionnaires were collected from the participants upon informed consent, with focus on several aspects of this training session (Table 2): session duration, task selection, training session planning, expectations, and overall evaluation of the course. Scores ranged from 1 (poor) to 5 (very good).

Table 2.

Quality Feedback Questionnaire, Results

	Average	Standard Deviation
Session time	2.75	1.2
Exercises chosen	4.17	0.6
Providing a structured training session	4.81	0.5
Did it meet your expectations?	4.06	0.8
Overall impression of the course	4.25	0.7

The results were collected along with a detailed description and refinement of the four tasks. The tasks and metrics were further analyzed by a consensus meeting between 10 experts in education during the same event. Following the feedback provided, task list was slightly reconfigured as follows: task 1, Ureteral stone fragmenting; task 2, Fragment relocation; task 3, Kidney stone dusting; task 4, Inspection of the pelvicaliceal system. Ballistic stone fragmentation was excluded to prioritize laser treatment technologies, in consideration of their safety and wider use. Stent placement and other steps involving X-rays (Table 4, in bold italics) were taken out of the list becaseu of the issues related to their applicability to dry laboratories. Tutor:Trainee rate was also modified from 1:2 to 1:1, to provide more satisfactory teaching. Finally, to double check content and setup, the final task description and a related online survey was sent to 26 experts from EULIS and ESUT section who were not personally involved in the development process. Seven Likert-scale questions were designed, aimed to collect experts' opinion about whether the protocol was well structured and optimized for HoT sessions, whether these tasks covered the maneuvers of RIRS, which candidates could apply for this examination, if appropriate kit was available in their center for this protocol and whether they would adopt the protocol in their center (Table 3).

Table 3.

Experts' Opinion Regarding Endoscopic Stone Treatment Step 2

	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
The four EST-s2/A tasks cover the main complex maneuvers of a RIRS procedure	15	9	2	0	0
EST-s2/A is well structured and easy to follow	11	14	1	0	0
EST-s2/A is a useful tool to optimize and improve the quality of HoT sessions	14	10	2	0	0
I would recommend the use of EST-s2/A for trainees who already achieved EST-s1	17	8	1	0	0
EST-s2/A training should be allowed only after having passed EST-s1 examination	15	6	2	3	0
I would adopt EST-s2/A to teach intermediate skills to my residents	10	12	4	0	0
My university/hospital is already equipped with everything necessary to deliver EST-s2/A (simulator/instruments/devices)	9	6	4	6	1

HoT = hands-on-training; RIRS = retrograde intrarenal surgery.

Table 4.

Procedural Steps for Endoscopic Stone Treatment (Basic Skills in Bold, Intermediate Skills in Italics, Steps Needing X-Rays in Bold Italics)

Intermediate (step 2/A) HoT curriculum

CTA: procedural steps study

In bold: basic skills (already part of EST-s1)

In italics: intermediate skills (integrated in EST-s2/A)

In bold italics: steps needing X-ray use

Preoperatory phase (WHO checklist)

1. Check material-related details (equipment up to date, presence of ancillary equipment, laser settings, irrigation fluid)

Procedural phase (RIRS)

1. Assembling instruments and connecting tubes

2. Adjustment of light settings, focus camera, white balance

3. Instillation of lubricant into meatus and introduction of the cystoscope

4. Inspection bladder, including orientation, identification of orifices, and eventual bladder tumors (or any other abnormalities such as red patches and squamous metaplasia)

5. Insertion of ureteral catheter (appropriate insertion, safely through the ureteral orifice)

6. Retrograde pyelography

7. Ureteral and renal pelvicaliceal mapping (or identification of stones/strictures/filling defects)

8. X-ray-guided placement of the guidewire (safety guidewire, X-ray safety precautions)

9. Semirigid ureteroscopy (inspection of the ureter)

10a. In case of a ureteral stone, proceed with fragmentation (laser)

10b. In case of a renal stone, placement of the working guidewire (through the semirigid ureteroscope)

11. Placement of the access sheath under fluoroscopy guidance (choosing the optimal size and length)

12. Insertion of the flexible ureteroscope

13. Inspection of calices

14a. Insertion of the laser fiber (laser safety precautions, e.g., protective glasses)

14b. Introduction of biopsy forceps in case of suspected lesions

15. Stone fragmentation/dusting, and basketing as appropriate

16. Double check under fluoroscopy and endoscopy for residual fragments

17. Stent placement at the end of the procedure

Continuous variables procedural phase

1. Safe change of instruments during procedure

2. Orientation in the bladder and upper renal tract

3. Regulation of irrigation and emptying bladder

4. Maintenance of visibility

5. Intrarenal pressure awareness

6. Potential to cause injuries

7. Economy of movement

Completion phase

1. Documentation of stone-free status/residual fragments

2. Safe removal of instruments

3. Ensuring bladder emptying

4. Debriefing (check count materials and stone specimen/biopsy, discussion complications, and postoperative policy)

5. Registration (operating report, eventual pathology file, patient file, and financial registration)

EST-s1 = endoscopic stone treatment step 1.

Phase 5: Finalization of the protocol

After the aforementioned process a full tutor-instruction sheet was produced (Supplementary Appendix SA1), mentioning all the pass criteria to be met to complete each task.

Statistical analysis

Data were summarized as a mean and standard deviation or as absolute number, as appropriate. All statistical analyses were done by SPSS for Windows (ver. 22), IBM, Chicago, IL.

Results

The EST-s2/A curriculum development, from early data collection to validation, took a total of 38 months and involved EAU sections (ESUT, EULIS) with guidance and coordination from the ESU/ESUT training groups. Starting from the initial CTA, a 1277-word revision with preliminary task description was produced. Between the 17 procedural steps identified by the CTA for the RIRS procedure, 9 intermediate skills were identified and included in the final training protocol (Table 4). Cognitive contents were considered separately by the EULIS.

Training session evaluation

All 16 participants of the EST-s2/A test course at EUREP were provided with quality feedback questionnaires. Results were provided with Likert-based format, ranging from 1 (really poor) to 5 (very good). All trainees were fifth-year residents in urology and 80% of them were working in departments without any simulation tool available. Data collected are summarized in Table 2.

Standardized stone evaluation

Twenty stones of each kind were produced for our test, to analyze consistency of their physical properties, including dimensions and weight.

Stone A prototype's average weight before the test treatment was 0.3 g (±0.0), whereas average dimension was 7 mm (±0.0). Stone B prototype's average weight before the treatment was 0.6 g (±0.0), whereas average dimension was 10 mm (±0.0). In both cases the stones were treated in accordance with the suggested laser settings.

Training content evaluation

The final content evaluation questionnaire was filled out by 26 experts (EULIS and ESUT endourology subsection members), after a complete overview of the final task description. Results were provided with Likert-based format, ranging from 1 (strongly disagree) to 5 (strongly agree).

Data collected are summarized in Table 3.

Discussion

The new EST-s2 protocol is the second such structured and streamlined endourology curriculum simulation protocol. We defined simulator requirement, provided replicability of a synthetic stone model that ensured same physical properties for each single training/assessment session. Although the participants provided high scores to task selection, planning, and overall quality of the new protocol, the duration of the session was scored on average 2.75/5, which led to modify a tutor:trainee rate to 1:1, instead of 1:2.

In this article we show the various steps that contributed to this novel protocol and provide preliminary content validity, by sharing the final task description and collecting experts' opinion. Most experts involved agreed that EST-s2/A protocol covered the complex maneuvers of RIRS, the protocol was well structured and easy to follow. Twenty-five would recommend the use of EST-s2/A to those previously certified in EST-s1. This result expresses the importance of following the modular HoT template³ to ensure predictable and optimized training results. Unfortunately, although 85% of the experts interviewed would adopt EST-s2/A to teach intermediate stone treatment skills to their residents, only 58% had the necessary equipment in their center.

Strengths and weakness of our study

The strength of our study is the use of recognized methodology with a phased curriculum development. Along with the development of a new training guideline, the study provides clear information about the safety measures that needs to be adopted with using laser units. The article complements the online theoretical curriculum already developed by EULIS, including nontechnical skills and is the next step after the EST-s1 protocol. For the exercises we used custom materials instead of Begostone^12,13 as these could be standardized and manufactured as per training protocol. The price quotes per single stone was around 5€, which might be a limit to the spread of the protocol because of the high number of stones used during training sessions. However, we plan to conduct the studies on Begostones as it might help increase the availability and uptake of the curriculum and reduce the cost of manufacturing the stones. Another limitation could be found in the simulators used because of their cost. In this case, considering the development phase, this detail was not considered, but will be taken into account during the validation and optimization phases. At the same time, repetitions to reach proficiency will be studied, to provide detailed teaching strategies.

For complete endourology training, technical skills with a validated curriculum need to be supplemented with theoretical background, patient-specific information, and nontechnical skills,¹⁴ which can help in training and assessment of communication skills in high stress or emergency situations. Cognitive information about the full procedure were analyzed and collated in a dedicated theory module during the EST-s1 development phase by the educational group of EULIS. This protocol does not consider the steps that need radiation exposure.

Area of future research

EST-s2/A development represents the pathway of protocol design, development, and pilot testing in EAU-affiliated meetings. Further validation studies will be required to confirm it as the official second step of the EST training pathway. Contemporary work will include the development of the step s2/B percutaneous nephrolithotomy (PCNL) part. Once this is established, the full curriculum will need to be completed (step 3), including the full RIRS, and PCNL procedures, before proceeding to fellowships and advanced clinical experience involving real patients. Given the latest COVID-19 pandemic regulations on social distancing, the current protocol should be tested for suitability for tele-training delivery.

Conclusion

We describe the development methodology for intermediate EST, which also reinforces the roadmap on developing other HoT curriculums in future. The CTA, expert consensus input and adherence to guidelines allowed evidence-based training and assessment protocol development. Our results anticipate the validation phase that will assess whether the EST-s2/A protocol is feasible for global uptake.

Footnotes

Acknowledgments

The authors thank Medics 3D (Turin, Italy) for the free provision of standardized synthetic stones during the testing phases; EULIS training board, ESUT training group, and ESU training group; and the staff of the Grande Ospedale Metropolitano of Reggio Calabria, for recording the stone testing videos.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this study.

Supplementary Material

Supplementary Appendix SA1

Abbreviations Used

References

Brinkman

, Tjiam

, Schout

BMA

, et al. Results of the European basic laparoscopic urological skills examination. Eur Urol, 2014; 65:490–496.

Somani

, Van Cleynenbreugel

, Gozen

, et al. The European urology residents education programme hands-on training format: 4 years of hands-on training improvements from the European School of Urology. Eur Urol Focus, 2018. DOI: 10.1016/j.euf.2018.03.002.

Veneziano

, Hananel

Chapter 75. In: Smith A, Preminger G, Kavoussi L, Badlani G, Rastinehad A, eds. The Smith's Textbook of Endourology. Fourth Edition. Wiley, 2019. DOI: https://doi.org/10.1002/9781119245193.ch75.

Veneziano

, Ahmed

, Van Cleynenbreugel

, et al. Development methodology of the novel endoscopic stone treatment step 1 training/assessment curriculum: An International Collaborative Work by European Association of Urology Sections. J Endourol, 2017; 31:934–941.

Veneziano

, Ploumidis

, Proietti

, et al. Validation of the endoscopic stone treatment step 1 (EST – s1): A novel EAU training and assessment tool for basic endoscopic stone treatment skills—A collaborative work by ESU, ESUT and EULIS. World J Urol, 2019; 1. DOI: 10.1007/s00345-019-02736-4.

Veneziano

, Ploumidis

, Proietti

, et al. Evolution and uptake of the endoscopic stone treatment step 1 (EST-s1) protocol: Establishment, validation, and assessment in a Collaboration by the European School of Urology and the uro-technology and urolithiasis sections. Eur Urol, 2018; 74:401–402.

Messick

Validity of psychological assessment. Am Psychol, 1995. DOI: 10.1037/0003-066X.50.9.741

Goldenberg

, Lee

. Surgical education, simulation, and simulators—Updating the concept of validity. Curr Urol Rep, 2018; 19:52.

Satava

, Gallagher

. Next generation of procedural skills curriculum development: Proficiency-based progression. J Health Specialties, 2015; 3:198.

10.

Turk

, Knoll

, Petrik

, et al. EAU guidelines on urolithiasis. Eur Assoc Urol, 2018. DOI: 10.1002/wrcr.20090

11.

Salmon

, Stanton

, Gibbon

, Jenkins

, Walker

. Cognitive task analysis. In: Human Factors Methods and Sports Science. 2009. DOI: 10.1201/9781420072181-c4

12.

Liu

, Zhong

. BegoStone—A new stone phantom for shock wave lithotripsy research. J Acoust Soc Am, 2002; 112:1265–1268.

13.

Esch

, Simmons

, Sankin

, Cocks

, Preminger

, Zhong

. A simple method for fabricating artificial kidney stones of different physical properties. Urol Res, 2010; 38:315–319.

14.

Yule

, Flin

, Paterson-Brown

, Maran

, Rowley

. Development of a rating system for surgeons' non-technical skills. Med Educ, 2006; 40:1098–1104.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.15 MB