Introduction

In a recent international survey [1] of specialist training in adult intensive care medicine (ICM) across 41 countries we found substantial variations in specialty ownership, duration, format and methods of assessment. Within Europe alone there were 37 different training programmes with a minimum duration of ICM training required for recognition as a specialist which varied from 3 to 72 months (mode 24 months). Although there are similarities between curricula, there is evidently no international agreement about the ‘end-product’ of training in terms of the competencies expected of a specialist in ICM. This makes it more difficult to attain the European Union objective of free movement of professionals [2, 3], and complicates mutual recognition of qualifications. We therefore established an international partnership of training organisations under the aegis of the European Society of Intensive Care Medicine and part-funded by the European Commission. Our aim is to develop an internationally acceptable competency-based training programme in intensive care for Europe (CoBaTrICE) and other world regions, by using consensus techniques to develop minimum core competencies for specialists in ICM. This work has previously been presented in abstract form [4, 5, 6].

Methods

National and international organisations with an interest in or responsibility for training in ICM were invited to nominate expert clinicians as representative national coordinators in the project.

We used a modified Delphi process and a nominal group (NG) [7] to generate and rate the importance of core competencies in ICM, an approach recently used to identify undergraduate competencies in acute care [8] and previously used to identify professional roles [9, 10], curriculum content [11, 12, 13, 14, 15], desired outcomes of national training programmes [8, 16, 17, 18], and prioritisation of research topics in intensive care [19, 20]. The Delphi technique is used to gather and prioritise opinion from large numbers of expert contributors using an iterative process with feedback of individual and group ratings of each item; in its modified form ratings may be omitted and the iterations are limited in number. The nominal group technique uses a small number of people with a facilitator to mediate discussion, thus permitting consideration of concepts in depth. There were three phases (Fig. 1), as described below.

Fig. 1
figure 1

Consensus methodology: application of modified Delphi and nominal group techniques

Full size image

Phase I: generating ideas for competencies

We invited health care professionals and educators to suggest essential competencies for an ICM specialist via a dedicated website. The project was promoted by national coordinators using partnership websites, national and international conferences, and publications; stakeholders were also contacted directly by e-mail. An unlimited number of free-text contributions could be made via the website. A single open-question invited contributors to ‘tell us which competencies are essential for physicians specialising in Intensive Care Medicine’. An example was provided to assist contributors, who were also asked to identify their specialty, country and email address if they wished to participate in the second round of the Delphi. The website was available in six languages (Czech, English, Finnish, Hungarian, Polish and Spanish). All contributions were translated into English by national coordinators before analysis. An option to submit suggestions by email, by post or via national coordinators was also provided. Concurrently patients discharged from intensive care units (ICUs) and relatives of ICU in-patients were invited to participate in a structured survey in order to gain insights into the views of the ‘consumers’ of intensive care. The questionnaire comprised 21 items which could be divided into three categories: (a) medical knowledge and skills, (b) communication and interpersonal skills, (c) decision making. Questionnaires were distributed by local ICU representatives in eight European countries (encompassing northern, central western, central eastern and southern regions) and were returned by post for central analysis. Preliminary results from this survey have been reported [21]; detailed results will form a separate publication. Responses from this consumer survey were integrated with the contributions from the web-based Delphi.

Thirty-seven keywords were derived from ESICM and SCCM guidelines [22, 23, 24, 25, 26], English-language curricula and training guidelines of national ICM training programmes [27, 28, 29, 30, 31, 32] personal communications with national coordinators (for Belgium, Bulgaria, Croatia, Poland) and major textbooks of critical care medicine [33, 34, 35]. All Delphi contributions were categorised by an ICU research nurse (H.B.) using these pre-determined keywords. Multiple keywords were assigned if applicable. Common themes within each category were identified; these multiple contributions were then expressed as single statements by an experienced specialist in ICM (J.B.). The resultant materials were articulated in the form of short phrases, or competence stems, preceded by generic descriptors of level of expertise (derived from the Delphi material), which taken together produced the competence statements (see Appendix and Table 1).

Table 1 Constructing a competence statement
Full size table

Phase II: nominal group ratings

Members of the NG were professionals with expertise in ICM, and were representative of the first round Delphi respondents in terms of geographical location, profession, specialist background and experience. The group comprised 11 physicians, two of whom were trainees, and one nurse. NG members were asked to identify, in private, the minimum level of expertise (chosen from the generic descriptors) which they considered acceptable for a specialist at the end of ICM training for each competence stem. The resulting data (group mode and range, and personal selection) was made available to each member of the group during a subsequent plenary meeting. The NG met in plenary session, with discussion recorded by a research nurse and moderated by an intensivist who had prior experience in consensus techniques; neither participated in the rating process. To avoid fatigue, participants undertook some preparation before the meeting, spread the work over one and a half days and took regular breaks. Participants were permitted to alter the vocabulary used in the stems to improve clarity, if all members agreed. A two-step process was adopted: the group first discussed each item to agree the minimum level of expertise; then using this agreed level they rated it privately for importance using a five-point Likert scale (ranging from 1, unimportant, to 5, very important; Electronic Supplementary Material, ESM Fig. 1). It was agreed before the meeting that if consensus could not be achieved for level of expertise, the lowest level proposed by any NG member would be accepted as the default minimum. This ensured that all items could be rated for importance. The mode was used to analyse level of expertise, mean values were used to determine the group ratings for importance for each item, standard deviations as a measure of disagreement, and mean and standard error to examine the rating characteristics of the members of the NG. We determined in advance that statements rated 4 or higher by the NG would be included in the final set of core competencies; those rated 3 or higher might be included, subject to the comments from the ICM community during the second round Delphi, and statements rated lower than 3 would not be included.

Phase III: recirculation for comment and iterative review

The output from the NG was made available online for 1 month. All first round Delphi participants who provided an email address were contacted and invited to send free-text commentary. National representatives sought the views of their national training organisations and professional colleagues. At the end of this second round comments were analysed by the original reviewers. Modifications included combining common themes to remove discrepancies and reduce repetition, using three rather than four levels of expertise, re-ordering the statements to aid comprehension and categorising them by themed domains. The process of review was overseen by the steering committee and national coordinators, and finally by a specially convened editorial group with clinical and educational expertise, and which reflected both linguistic and cultural diversity.

Results

Phase I: generating ideas for competencies

A total of 536 respondents from 57 countries worldwide participated in the first round online Delphi. The majority of respondents were physicians; 76% were specialists in ICM, 10% trainees and 8.5% other specialists. The remaining respondents included nurses (3%), educators (1%), medical students (1%) and allied health professionals (0.5%). Most (70%) described their primary place of work as a university or university-affiliated hospital.

In total 5,241 suggestions for competencies were received during the 6-month period of data collection (ESM: Fig. 2). The majority (80%) were submitted in English, but suggestions were also made in Polish (8%), Spanish (3.5%), Hungarian (3.5%), Czech (3%), Finnish (1%), Italian (0.5%) and French (0.5%). The median number of suggestions per respondent was 10 (range 1–124).

Suggestions were submitted as single words, phrases or paragraphs with varying descriptive detail. When categorised, suggestions were allocated to all 37 pre-determined keywords; 1,419 (27.1%) suggestions required more than one keyword. The most frequent categories of suggestions were practical procedures (20%), professionalism (‘attitudes and behaviour’ and ‘communication’: 19.6%) and organ system failure/support (all sub-categories: 19.4%; Table 2), irrespective of country of submission or profession (ESM, Figs. 3, 4).

Table 2 Delphi suggestions by keyword category
Full size table

Following iterative content analysis of materials from the online Delphi and consumer questionnaire, all suggestions were encapsulated within 164 competence stems of varying specificity, and 5 behavioural themes. Items from the consumer survey could be linked to 76 of these; only two consumer items were not also represented in the Delphi material. Four levels of expertise (A–D) were identified in the Delphi material (Table 1).

Phase II: nominal group

Changes to stems

Three stems were removed as participants considered that they were already included within others (e.g. ‘intermittent positive pressure ventilation’ was already part of ‘invasive ventilatory support’). Three additional items were proposed and subsequently rated by the NG. Minor changes were made to the wording of 23 items in order to enhance clarity, for example, ‘bronchoalveolar lavage’ changed to ‘bronchoalveolar lavage in the intubated patient’ and the 5 behavioural themes were articulated as competence stems. At the end of this process 169 stems were rated by the NG.

Level of expertise

Complete consensus of minimum level of expertise was not achieved for any of the 164 competence stems by the members of the NG when making their private selection before the plenary meeting. Twenty stems achieved mode level A (‘knowledge of’), 14 mode level B (‘performs under supervision’), 86 level C ‘performs independently’) and 25 level D (‘supervises others’); 19 stems stimulated a bimodal response. During the meeting a minimum level of expertise was agreed for all 169 stems by all participants; in five instances this required the adoption of the default level (i.e. the minimum level proposed by a member of the group) rather than the group consensus. For the majority of stems the consensus level was the same as the mode of the selections made prior to the meeting. For 12 stems it was greater than the mode; all increased from level C to D. At the end of the plenary meeting 38 stems were assigned level A, 18 level B, 74 level C and 39 level D. The concept of the supervisory level of expertise (level D) stimulated the most debate.

Rating of importance

Once the level of expertise had been agreed, rating of importance was comparatively rapid. A mean rating of 4 or higher (important or very important) was achieved for 111 stems (66% of competence statements); between 3 and 4 (moderately important) for 50 stems (29%) and lower than 3 (minor importance or unimportant) for 8 stems (5%). There was a trend for greater agreement between raters for those competencies given high importance (ESM, Fig. 5). The mean ratings by each member of the NG for all competencies ranged from 3.65 to 4.55 (ESM, Fig. 6).

Phase III-Iteration

During the 4-week consultation period 73 detailed responses were received. These represented the views of both individuals and groups from 23 European countries and 6 in other world regions. More than half of the respondents had contributed to the first round Delphi. Comments were substantially favourable and constructive; they concerned changes in level of expertise, importance and relationship between competencies. There were no criticisms of the choice of competencies. Several additional topics were identified, most of which were either already captured within existing statements or could easily be incorporated through minor re-wording; two new statements were added to the final set of competencies. Proposals to downgrade level of expertise were counterbalanced by proposals to increase it.

Development of the final core competency set

All competencies rated greater than 3 were included in the set. Common themes were merged to reduce repetition and discrepancies, and accommodate local constraints (lack of specific equipment or therapies; ESM, Fig. 7). Low-rated competence statements were not included in the final selection, but themes considered to be important by the second round Delphi respondents were retained within the syllabus: for example, suprapubic catheterisation (rated low) was retained by amending ‘transurethral urinary catheterisation’ (rated high) to ‘urinary catheterisation’. Responding to Delphi comments, we simplified level of expertise by merging levels C and D (independent practice and supervising others). The requirement for trainees to become competent at supervising and delegating safely and teaching others was retained via specific competence statements. Competencies are thus expressed at the level of independent practice unless accompanied by the prefix ‘describes…’ (knowledge), or the suffix ‘…under supervision’ (supervised practice). The relationship between trainee supervision and level of expertise is presented schematically in Fig. 2. Following preliminary development of the syllabus, the editorial group made minor changes to the wording of the statements to improve comprehension, reduce repetition and make statements more inclusive in terms of multiple links to the syllabus. Three themes contained within existing competencies were extracted and reformulated as competence statements. All were then grouped into domains; generic elements within each domain were identified, and a descriptor created to contextualise the theme of the domain. The final competence set consists of 102 competence statements grouped in 12 domains (Table 3).

Fig. 2
figure 2

Relationship between supervision and minimum level of expertise

Full size image
Table 3 Domains, descriptors and competence statements
Full size table

Discussion

Curriculum development is more of an art than an exact science. We have used the combined experience and wisdom of a large number of ‘stakeholders’ in critical care worldwide, to develop a set of minimum core competencies to describe a specialist in ICM. It should be possible for these competencies to be acquired in any country with ICM services and training infrastructures and for them to be applicable across national borders and professional disciplines.

The emphasis given by Delphi respondents to the importance of professionalism is impressive. This was given a prominence equal to technical ability and demonstrates the value accorded by intensive care clinicians and consumers to attitudes and behaviours, particularly communication skills and self-regulation (‘governance’) and is consistent with increasing concern about the de-professionalism of physicians [36]. Patient safety also emerged as a priority area; it was not specifically identified as a keyword category from existing training programmes, but appeared so frequently in the Delphi material that it warranted presentation in a specific domain.

The NG provided an important forum for mature reflection, focussed discussion and prioritisation by front-line clinicians. A high level of group attention on the task was maintained by fostering an informal atmosphere within a disciplined framework. We found the two-step process of first determining level of expertise before rating importance of great value, since the latter judgements were so clearly influenced by the former. Discussion allowed us to accommodate local variances and to set an achievable safe standard. There was a necessary compromise between desirable training objectives and deliverable training opportunities in an international context. Aspects of training which were considered to be valuable but difficult to deliver universally were assigned a lower minimum level of expertise (e.g. paediatric competencies) to facilitate implementation. However, flexibility in application permits countries to set higher levels of expertise (or include additional competencies) if local factors make this necessary. The competence statements are a compromise between simplicity (‘the trainee is competent in all aspects of intensive care medicine’) and specificity. Detailed aspects of competence will be contained in the syllabus.

Supervision and levels of expertise are complementary aspects of training. We found it necessary to clarify these concepts. The minimum level of expertise can be equated with a maximum level of supervision which should be attained by the end of specialist training (Fig. 2). All trainees by definition are expected to work under specialist supervision, which may be direct or indirect [37] (see Appendix). However, training means a journey towards independent practice during which the trainer must determine the level of supervision required by trainees in relation to their expertise and the needs of the patients.

Our study has a number of potential limitations. Although over 5,000 suggestions were submitted, we may not have captured all relevant competencies. However, all pre-defined keyword categories were used, indicating that the suggestions largely reflected the major components of existing training programmes. The web site was available in only six European languages including English; this might have biased the participant sample, but the universal language of the European medical community is considered to be English. Fatigue or participant disillusionment could be a limitation of internet-based surveys, and might account in part for the disparity in the number of participants from different countries. Relatively few nurses, educators and allied health professionals contributed; there may be scope for the application of relevant competencies to other critical care training programmes (non-medical); however, such wider application would require further consultation with representatives of these allied professions. Finally, the existence of a consensus (the ‘tyranny of the majority’) does not necessarily mean that the ‘correct’ solution has been found; short of observing practice in every country, we may only learn of possible discrepancies when the core competencies are applied to national training programmes. Further iterative review will therefore be required once they are implemented.

Competencies are measurable outcomes of training, assessed in the workplace as knowledge, skills, attitudes and behaviours (see Appendix), which allow us to make judgements about a clinician's abilities (performance) in a transparent and reproducible manner. As core competencies could be interpreted in conceptually different ways according to national working practices [16], a detailed syllabus and assessment guidelines are required in order to promote standardisation. To assist trainers and trainees the next phases of the CoBaTrICE project will link these competencies to guidance on assessment and to online educational resources.