Over the past few decades, urological residency training has been changing due to evolving medical technology in combination with increased attention to patient safety issues and the need for efficient use of training time and money [1, 2]. The advent of the European Working Time Directive has led to limited training time, while competency needs to be preserved [3]. Consequently, training will have to become more structured and better assessed [3, 4]. Besides a changing focus toward competency-based training, interest in structured, non-patient-related (simulation) training of technical and non-technical skills (e.g., leadership, decision making, and situational awareness) has strongly increased [5–8]. Non-patient-related skills training offers important opportunities for residents, because the initial phase of the residents’ surgical learning curve is moved to a risk-free and time-independent environment [1, 4, 9].

Nowadays, simulation-based training is becoming increasingly accepted as a method to complement training in clinical practice [10, 11]. Still, residency programs struggle with the integration of simulation training into curricula due to issues as considerable costs, limited personnel, and resident working hour restrictions [12, 13]. Moreover, despite the fact that generally the validity of simulators has been proved, their effectiveness in a central training program is often lacking [14].

In some countries, simulation training is gradually integrated into urological curricula on a national scale. McDougall et al. [15] (USA) presented a template for a cognitive and basic skills curriculum to cover the 4 years of urological residency training based on weekly, year specific training sessions. The first experiences of residents and faculty were favorable, but the effectiveness of the curriculum on clinical performance was not yet confirmed. Another example is the national (UK) simulation curriculum presented by Shamim Khan et al. [16]. This curriculum includes training of technical and non-technical skills, and its feasibility, face, content, and construct validity were previously shown. Again the assessment of concurrent validity is subject to further study, and fortunately, an RCT regarding the effects of this curriculum on operative performance is on its way.

Although concurrent validity of central simulation programs for urological skills is still lacking, this has been confirmed in other specialties. In a recent study of Aghazadeh et al. [17], it was shown that training of simulated robotic skills has a significant relationship between simulated robotic performance and robotic clinical performance. Furthermore, in the field of gynecology, it was shown that participation in a simulation-based training curriculum for gynecologic laparoscopy (including cognitive, technical, and non-technical components) leads to a superior improvement in knowledge and technical performance in the operating room compared with conventional residency training [14].

A recent study on the current situation in the Netherlands revealed that structured practical skills training takes place in a minority of teaching hospitals, though skills laboratories are widely available and residents prefer to practice certain procedures in a non-patient-related setting first [18]. This has led to the development of the Dutch Urological Practical Skills (D-UPS) curriculum, which was implemented on a national scale between September and December 2014.

In this study, we aimed to answer the following research questions: ‘What is the experienced educational impact of the national implementation of the D-UPS curriculum?’ and ‘What are focus points for improvement of the D-UPS curriculum according to the participants?’

Methods

Design, content, and purpose of the D-UPS curriculum

The D-UPS curriculum is designed using the backward design principle of Wiggins and McTighe [19]. This simulation-based curriculum combines the acquisition and rehearsal of basic theoretical knowledge with practical training of basic urological skills and techniques. The first step in the development of each specific training module was a ‘training needs analysis’ (TNA), in which procedural steps were identified, potential pitfalls analyzed, and learning objectives defined [11, 20, 21]. Subsequently, a suitable simulator was selected (training media specification, TMS).

Important features of the D-UPS curriculum are: (1) training of technical and non-technical basic urological skills; (2) local hospital setting; (3) small groups; (4) use of peer teaching and expert supervision; and (5) yearly recurrence. Training modules can be performed at junior level or senior level and consist of an online theoretical part (theory, educational videos, pretest) and a practical part, in which procedural steps, pitfalls, and non-technical skills of the procedure are trained in a non-patient-related setting, under supervision of an experienced urologist. Residents and supervisors can register on the Web site www.traininginurology.com to then get full access to the theoretical part and the content of the modules. Figure 1 presents the general outline of the training modules. Further details on the development of the D-UPS curriculum and the assessment of its feasibility and acceptability have been described in a previous study [18].

Fig. 1
figure 1

General outline of the training modules of the D-UPS curriculum

Full size image

The overarching aim of the D-UPS curriculum is to shorten the patient-related learning curve of basic urological procedures by training these procedures in a risk-free simulation environment. The implementation of the D-UPS curriculum should ultimately lead to an improved patient safety, time efficiency in the OR, self-confidence of the residents, and uniformity of actions [18]. Furthermore, the D-UPS curriculum aims to offer senior residents’ training in their peer teaching skills, preparing them for their future role as educators.

Between September and December 2014, the curriculum was implemented on a national scale in the Netherlands. During this period, the first eight training modules were attended by junior and senior residents in the local hospital setting, namely ‘ultrasound of kidney and bladder,’ ‘ultrasound of prostate,’ ‘basic laparoscopy,’ ‘electrosurgery,’ ‘acute penile pathology,’ ‘mid-urethral sling (MUS),’ ‘transurethral resection of the prostate (TURP),’ and ‘flexible ureterorenoscopy (URS).’

Post-curriculum survey and individual module-specific feedback

The perceived educational impact of the D-UPS curriculum was assessed by means of a quantitative online survey and qualitative individual module-specific feedback. The quantitative online survey (www.surveymonkey.com) was designed and validated on its contents by a multidisciplinary team of experts in urology (BS, EK), an educationalist (JvM), and a healthcare safety expert (CW). In February 2015, the survey was sent to all residents (n = 63) and supervisors (n = 58) that participated in the curriculum. Upon initial non-reply, two reminders were sent by e-mail to maximize response rate. Anonymity and confidentiality were guaranteed to all participants.

Two similar questionnaires were used, one for residents and one for supervisors. The survey for residents consisted of 41 questions, divided into three sections. The first section focused on demographics and practical aspects of the D-UPS curriculum, e.g., logistics, supervision, and use of peer teaching. The second section queried residents about their motivation and general judgement of the modules. The last section comprised questions regarding the impact of the D-UPS curriculum on residents’ knowledge and their technical and non-technical skills. Important endpoints were value of knowledge training (studying theory and watching educational videos) for learning the procedure, and impact of training on (1) knowledge of materials and equipment, (2) ability to anticipate pitfalls and complications, and (3) effect on performance due to increased knowledge of procedural steps. A similar questionnaire was developed for the supervisors, consisting of 37 questions divided into the same three sections. This questionnaire contained three extra questions in the second section regarding the provision of information on the content and aim of the curriculum. Seven questions focusing on the impact of training for residents in particular were excluded in this version. Answer options included multiple choice, statements on a five-point Likert scale (1—disagree, 3—neutral, 5—agree; not attended), and answers to open-ended questions. A copy of the questionnaires can be obtained from the first author on readers’ request.

Written individual module-specific feedback was obtained directly after the training modules and consisted of a form inviting the participant to answer the questions: ‘What went well?’ and ‘What could be improved?’

Statistical analysis

To enhance the clarity and robustness of between-group tests, the score per module (five-point Likert scale) was dichotomized to a non-favorable/favorable score (0: score ≤ 3; 1: score = 4, or 5). For each participant, the resulting binary scores were aggregated (over the 8 modules) to a percentage favorable score (number of high scores/number of attended modules expressed as a percentage). Two-sample t tests were used to analyze differences between groups. Chi-square test was used to analyze differences in categorical variables. A p value of p < 0.05 was considered statistically significant. Analyses were performed using the software package IBM SPSS Statistics version 20.0.

Results

Demographics

The survey response was 95 % (n = 60) for residents (junior n = 30; senior n = 30) and 76 % (n = 44) for supervisors. The training modules were performed in 20 out of 26 national teaching hospitals, and all participating hospitals were represented in the response. The number of modules attended by each participant varied because in some hospitals not all modules were performed due to logistical issues. As a result, the total number of participants was found to range from 19 out of 60 for mid-urethral sling (MUS) to 54 out of 60 for ultrasound kidney and bladder. Regarding the qualitative module-specific feedback forms, a substantial number of completed forms (n = 173) were received.

Practical aspects

The mean number of participants per training module in the local hospitals was 4–6 persons, and the modules were supervised by a urologist. Seventy-three percent of the supervisors confirmed that they generally used peer teaching, and 85 % of residents and 85 % of supervisors considered this to be of additional value. Residents agreed that the local training setting (91 %) and the used simulation models (76 %) were mostly suitable.

Sixteen out of 20 teaching hospitals succeeded in creating a fixed schedule for the training modules, which was either 1 h every week, 1 h every 2 weeks, or a monthly afternoon. According to 77 % of residents, training modules proceeded according to plan. Although the initially proposed planning for the modules was 1 h every week, 61 % of the residents preferred a monthly afternoon of training.

Eighty-nine percent of supervisors (partially) agreed that they were well informed about the aim and content of the training modules. Nevertheless, 24 % of supervisors (14/58) did not register on the Web site, which was a precondition for obtaining full information on the content of the modules.

General judgement of the D-UPS curriculum

The majority of residents (85 %) and supervisors (90 %) considered themselves motivated for performing the training modules. Participants judged the D-UPS curriculum to be an important addition to current residency training (90 % of supervisors vs. 67 % of residents; p = 0.007). Figure 2 shows the general judgement per module for junior residents, senior residents, and supervisors separately. The participants’ aggregated general judgement of the eight modules was substantially favorable (percentage favorable score: 57 ± 4 %, M ± SE).

Fig. 2
figure 2

General judgement of the training modules. The score per module (five-point Likert scale, 1: not useful; 5: very useful) was dichotomized to a non-favorable/favorable score (0: score ≤ 3; 1: score = 4, or 5). The mean proportion of favorable score per group (junior, senior, supervisor) is shown as a percentage (mean ± standard error; shaded bar and boundary lines, respectively)

Full size image

Impact of the D-UPS curriculum on technical and non-technical skills

All residents confirmed that they (partially) studied the mandatory theory and 85 % (partially) watched the demonstration of the procedure in educational videos. The perceived value of this knowledge training on learning the procedure is shown in Fig. 3.

Fig. 3
figure 3

Value of knowledge training on learning the procedure. The score per module (five-point Likert scale, 1: no value; 5: highly valuable) was dichotomized to a non-favorable/favorable (0: score ≤ 3; 1: score = 4, or 5). The mean proportion of favorable score per group (junior, senior) is shown as a percentage (mean ± standard error; shaded bar and boundary lines, respectively). Educational videos were not yet available for the procedures acute penile pathology, MUS, TURP, and flexible URS

Full size image

Thirty-three percent of residents versus 55 % of supervisors (p = 0.04) confirmed that they had discussed the intended training level (junior/senior) before the start of the modules. Table 1 presents the perceived educational impact of the curriculum for junior and senior residents on knowledge of materials and equipment, ability to anticipate on pitfalls and complications, and effect on performance due to increased knowledge of procedural steps. On each of these three endpoints, junior residents graded the impact of the curriculum significantly higher than senior residents (two-sample t test).

Table 1 Educational impact of the D-UPS curriculum as perceived by junior and senior residents
Full size table

Finally, Table 2 shows a summary of the qualitative module-specific feedback including satisfactory and unsatisfactory aspects of the training modules.

Table 2 Individual module-specific feedback regarding satisfactory and unsatisfactory aspects of the training modules
Full size table

Discussion

The present study shows that the national implementation of the D-UPS curriculum was successful. Residents and supervisors considered the curriculum to be an important addition to current residency training and to create a uniform foundation of basic urological skills.

Structured scheduling and commitment of all participants is paramount for the successful implementation of a simulation-based skills curriculum [22]. The proposed planning for the training modules of the D-UPS curriculum was 1 h each week, similar to the planning of the 4-year curriculum presented by McDougall et al. [15]. However, the results of the present study revealed that two-third of residents (61 %) preferred a monthly afternoon of training. Moreover, ‘time constraints’ was an important suggestion for improvement arising from the qualitative module-specific feedback. Planning a single hour of training in the busy schedule of residents and urologists incurs the risk that the training will be delayed or even canceled, leading to a decrease in preparation, participation and quality of training. This is where theory collides with practical experience, as the principle of ‘distributed practice’ suggests that it would be better to frequently train for shorter periods instead of less frequently for longer periods [23]. Fortunately, despite the difficulties involved, 16 out of 20 hospitals succeeded in creating a fixed schedule for the obligatory training modules. Nevertheless, logistics remains a point of attention in the implementation process.

Besides logistic issues, also the accompanying cost of a skills curriculum is an important aspect that needs to be taken into account in the implementation process. For the D-UPS curriculum, the cost per module is €87.50 per resident. This money is funded by the Dutch Ministry of Education, who pay a certain amount of money to educational hospitals that is used for the education of residents. This contribution covers the overhead organization and enables the local hospitals to buy and maintain the models and equipment. Furthermore, the industry supports the D-UPS curriculum by providing the models and equipment that are used for the more advanced modules such as TURP and URS.

Residents and supervisors were highly motivated to participate in the training modules. The majority of supervisors felt adequately informed about the aim and the content of the training modules. Still, a substantial number (24 %) did not register on the Web site, which was necessary for obtaining full information on the content of the modules. Although supervisors indicated to be well informed, one could question their degree of preparation prior to the modules. Providing clear instructions to supervisors and emphasizing the importance of preparation remain an important point of attention that could result in better preparation and motivation and ultimately increase the quality of the training modules.

In the setup of the D-UPS curriculum, the same training modules are attended by all the residents (junior as well as senior) who work in a particular teaching hospital. This is to ensure that all residents train their basic skills using their own equipment and materials in their own environment. The modules can be performed at junior or senior level, depending on the overall level of the attending residents. This training level is discussed before the start of the module. It is aimed to further increase the level of training for senior residents by the use of peer teaching. Despite these measures, junior residents graded the educational impact of the modules higher than senior residents. Moreover, the majority of the modules (7/8) was judged to be ‘not useful’ by over 50 % of the senior residents. These results are disturbing and suggest that joint training of junior and senior residents lowers the overall training level such that the educational impact for senior residents is considerably reduced. Possibly, the available measures to increase the value for senior residents were not always optimally applied. Our results showed that peer teaching was not always used and the intended training level was frequently not discussed before the start of the modules. Nevertheless, increased efforts should be made to adjust the required level of performance to the experience level of the particular residents. This implies a more personalized approach by the integration of spiral learning, in which the same skill is taught at each level, but with increasing degrees of complexity and sophistication [24]. The periodic recycling of the same topics with progressively greater complexity continues exposure to certain topics or skills while facilitating deep understanding of the subject and promoting more intuitive handling of the tools in question. Although annual recurrence of the modules was incorporated in the design of the D-UPS curriculum, the integration of this spiral learning approach remains an important opportunity for curriculum optimization. An example of a spiral learning approach for the module ‘flexible URS’ is presented in Appendix 1 of Electronic Supplementary Material. Besides the integration of a spiral learning approach, an additional strategy that could increase the educational value for senior residents is the use of peer teaching [25]. Peer teaching was not yet used in all the modules and could be used more explicitly. Further study is needed to evaluate whether peer teaching and optimized integration of the spiral learning approach will sufficiently increase the educational value for senior residents, or whether separate modules for residents with varying levels of difficulty are desirable to further individualize the curriculum. Obviously, this would complicate the logistics of the curriculum implementation.

One could question the need for basic urological skills training for experienced, final-year residents or highly technically skilled residents. Interestingly, a recent study showed that a high percentage of unintended events occurred in basic urological procedures performed by junior as well as senior residents [26]. This indicates that ‘being a senior resident’ does not automatically exclude the need for basic urological skills training, and suggests the need for objective assessment of skills in the integration of a more personalized approach. In such an approach, the concept of entrusted professional activities (EPA) could be useful [27]. An EPA is an activity that a resident can be trusted to perform competently. EPA assessment enables supervisors to know when a resident can be trusted to carry out specific procedures with minimal or no supervision, in different stages of the training [27]. Consequently, residents that have demonstrated to master a certain skill at a certain level could proceed from junior, to senior level in the DUPS curriculum, or (when perfectly mastering the skill) be exempted from the module that focuses on that particular skill. Such a more personalized approach widens the scope for higher performing residents and would be in line with the current shift from time-based residency training, with a set number of training years, toward competency-based residency training, in which competency levels are defined that have to be met before a resident is allowed to perform a certain procedure independently [28].

The results of the current study will be applied for further improvement of the D-UPS curriculum. A paramount aspect in curriculum development was pointed out by Kern et al. [29]: ‘curriculum development does not usually proceed in sequence, one step at a time. Rather, it is a dynamic, interactive process that continues and the curriculum evolves, based on evaluation results, changes in resources, targeted learners, and the material requiring mastery.’ After the completion of this first national run of the D-UPS curriculum, ongoing efforts will go to its quality improvement. Focus points—as well as recommendations for design and integration of similar curricula in other specialties—include integration of the spiral learning approach into the curriculum and an increased attention to logistics. Worldwide, the D-UPS curriculum can be seen as a front-runner, since integrated simulation-based curricula of technical and non-technical skills are still scarce and their structured implementation remains a challenge [13, 15, 16]. A limitation of this study is the relatively small number of participants, which is inherent to a curriculum implemented in a small country as the Netherlands. Although the number of participants was relatively small, the response rate to the survey was high and comprised information of all the teaching hospitals where the curriculum was implemented. This ensures a realistic evaluation of the implementation of the D-UPS curriculum in clinical practice.

Conclusion

The simulation-based D-UPS curriculum has a high perceived educational impact. Residents and supervisors consider the curriculum to be an important addition to current residency training. Focus points for improvement of the D-UPS curriculum according to the participants include increased attention to logistics and integration of a spiral learning approach.