Point-of-care ultrasound (POCUS) is an important and noninvasive diagnostic tool that allows rapid assessment and guidance of patient care at the bedside. For instance, in the acute care setting (e.g., emergency medicine, critical care, anesthesiology), POCUS allows physicians to assess hemodynamically unstable patients in a binary (yes/no) fashion and answer multiple clinical questions to rule in/out important differential diagnoses that often have conflicting management goals. Point-of-care ultrasound has the additional benefit of 1) being highly sensitive; 2) not emitting ionizing radiation; 3) having lower associated costs compared with other diagnostic imaging modalities (e.g., computed tomography); 4) being immediately available at the bedside (thereby precluding the need to transfer unstable patients to the radiology department); and 5) allowing for serial real-time assessments to determine responsiveness to treatment and re-evaluation of the diagnosis in light of treatment response. When compared with physical examination, POCUS has improved accuracy in detecting cardiac1 and lung2 abnormalities, as well as free fluid in the abdomen.3 Such benefits, combined with increasing advocacy4,5 and adoption6,7 of POCUS in the perioperative setting, led the Royal College of Physicians and Surgeons of Canada (RCPSC) to recently include POCUS as a mandatory component of Canadian anesthesiology training.8 Currently, however, there is no national consensus regarding competencies of perioperative POCUS to guide curriculum development. Similarly, little consensus exists pertaining to the assessment of POCUS expertise/competence in clinical practice.5 While some progress has been made with respect to the development and validation of competency-based assessments in critical care for cardiac9 and thoracic10 POCUS, similar tools specific to anesthesiology are still lacking,5 leading to high variability in POCUS training among anesthesiology residency programs in Canada6 and elsewhere.11 To this end, a group of 55 anesthesiologists serving as POCUS leads at 12 Canadian universities recently published recommendations for training and performance in basic perioperative POCUS, providing a framework for the development of a standardized national POCUS curriculum in anesthesiology.12

Given the Competence by Design (CBD) paradigm recently adopted into Canadian medical education, the lack of a structured national POCUS curriculum5,6,7 represents a significant gap in the specialty of anesthesiology both for resident trainees and practicing anesthesiologists, highlighting the need for urgent development of standardized and validated methods for assessing perioperative POCUS competencies in anesthesia. Indeed, without standardized, validated tools, assessments of competence are of limited value. Additionally, there is a clear need to define a series of agreed-upon competencies that can be used nationally to better inform POCUS curricula development in anesthesiology programs. The present investigation, therefore, aimed to define residency competencies for proficiency in basic perioperative POCUS using the Delphi method with expert consensus at the national level.13 Notably, this study represents the first stage of a three-phase investigation and will serve as the basis for the development of a national standardized curriculum and for the creation of assessment tool(s) to measure these defined competencies (phase 2). Phase 3 will focus on the collection of validity evidence (i.e., related to content, response process, internal structure, etc.) from anesthesiology learners (residents and attending anesthesiologists) to support/refute the use of these assessment tool(s) as measures of POCUS competency using Messick’s unified theory of validation.14

Methods

We obtained ethical approval from the Queen’s University and Affiliated Teaching Hospital’s Research Ethics Board (Kingston, ON, Canada; TRAQ #6026771) to conduct a Delphi study15 to reach national consensus regarding competencies required for proficiency in basic perioperative POCUS. We circulated an initial set of competencies informed by the RCPSC's National Curriculum8 and other relevant literature to experts across Canada for ongoing feedback (via iterative rounds of Delphi survey) until consensus was reached.

Setting and participants

We recruited panellists using purposive sampling, as recommended for the Delphi method.16 Panellists were expert academic anesthesiologists who were identified (based upon coauthorship on previous relevant Canadian literature,12 as well as through direct contact with anesthesiology department chairs and residency program directors) as POCUS leads at each of the 17 Canadian medical schools. Notably, larger residency programs with multiple hospitals/training sites may have been represented by more than one panel member depending on the number of individuals recognized as POCUS leads at that academic centre. All participants provided informed consent.

Data collection and analysis

Data were collected between April 2022 and February 2023. We administered all surveys via the Qualtrics platform (www.qualtrics.com; Seattle, WA, USA) to 25 panellists representing all 17 Canadian anesthesiology residency programs. The Round 1 survey (Electronic Supplementary Material [ESM] eAppendix 1) contained 74 proposed competencies required for proficiency in perioperative POCUS as determined by local experts (G. B. M., R. A., S. M.). We derived such competencies primarily from the RCPSC’s National Curriculum8 and other relevant sources12,17 originating from a PubMed® search on POCUS Canadian competencies/curricula and assessment tools relevant to anesthesiology. The competencies spanned 19 clinical domains. The survey also included demographic (age, sex, specialty and/or subspecialty) and contextual (number of years in independent practice, number of years using/practicing POCUS, POCUS certifications or courses) information. For each competency, we asked panellists to rate their level of agreement (agree or disagree) with including the item in the final list of perioperative POCUS competencies, and/or indicate the need for revision (should be revised to …), with an opportunity to describe the revised statement for each competency. Additionally, we asked participants to identify any additional areas of expertise required that had not yet been captured and share any other comments about the competencies in each clinical domain.

We used the results from Round 1 to generate the survey for Round 2. In addition to modified POCUS-specific items based on qualitative feedback provided in Round 1, the survey for Round 2 also included a question about any assessment tools currently in place at the panellists’ centre (ESM eAppendix 2). Any items that did not meet the necessary agreement level were discarded. We repeated this process for a third round (ESM eAppendix 3), after which consensus was reached.

We analyzed the quantitative data descriptively in the form of frequency and percentage of participants for each competency. For each round, we discarded items with < 50% agreement; revised those with 50–79% agreement based upon the qualitative feedback provided; and maintained unrevised those items with ≥ 80% agreement. The research team reviewed and revised any/all competencies that did not meet the agreement level but were above the discard level. Additionally, the research team also reviewed the qualitative data (feedback provided through the open-ended items) to identify and develop any new competencies as necessary.

Results

Three rounds were required for this Delphi process. Twenty-five individuals participated in each of the three rounds, of whom 64% identified as male and 36% as female. The mean age was 41 yr (range, 33–60 yr). There was a range of clinical experience, from between two and 32 years in independent practice, with a mean of nine years. The number of years using or practicing POCUS also ranged from four to 12 years, with a mean of 7.7 years. We achieved a response rate of 100% for each round. Table 1 details the Delphi process with the POCUS competencies organized according to clinical domain. A full list of competencies for Round 1 is available in ESM eAppendix 1, Round 2 in ESM eAppendix 2, Round 3 in ESM eAppendix 3, and a final list in Table 2.

Table 1 Delphi process for defining anesthesiology residency point-of-care ultrasound competencies organized according to clinical domain
Full size table
Table 2 Final list of anesthesiology residency competencies for proficiency in basic perioperative point-of-care ultrasound using a Delphi method with expert consensus at the national level
Full size table

In summary, we initially identified 74 competencies spanning 19 clinical domains that were circulated for Round 1, at the end of which 56 were accepted unchanged, 15 were revised, and three were discarded; in addition, four new competencies were added. For Round 2, 19 competencies were circulated, after which 12 were accepted and seven revised and recirculated for Round 3. After all three rounds, 75 competencies were ultimately agreed upon (Table 2).

Discussion

As access to ultrasound machines and POCUS expertise grows, so does POCUS use in clinical practice by both attending physicians and trainees of all levels. As such, there has been strong advocacy4,5,18 for formally incorporating POCUS into residency training programs across multiple medical specialties, leading to the development of consensus statements of key POCUS competencies (as well as assessment tools) from numerous disciplines (e.g., internal medicine,19 emergency medicine,17,20 critical care medicine9,10,21) that have been early adopters both in Canada17,19 and elsewhere.9,10,20,21,22

Accordingly, the RCPSC has recently recognized focused cardiac and lung ultrasound as an essential skill expected of anesthesiologists entering practice in Canada.8 To ensure evidence-based use of POCUS and a standardized level of expertise of Canadian anesthesiology graduates, however, it is imperative to have nationally defined competencies upon which to base training curricula. The present article builds on pre-existing work done in the Canadian anesthesiology context12 and serves as the most up to date and comprehensive set of competencies relevant to anesthesiology training programs and expands on the POCUS domains outlined in the Royal College curriculum.8

One notable strength of the current investigation lies on the use of a validated Delphi process13 including a panel of POCUS experts from every anesthesiology training program across the country, thereby making the resulting competencies highly applicable to the Canadian context. The 100% response rate across all three rounds of questionnaires combined with the inclusion of every training centre in Canada may also increase buy-in and national implementation of these competencies going forward. In addition, given the broad nature of the competencies generated in this study (i.e., 75 competencies across 19 clinical domains), our results are more comprehensive than previous resources12 and can serve as a robust starting point for the development of a standardized national curriculum and the creation and validation of assessment tool(s) to test proficiency in basic perioperative POCUS.

Importantly, while the competencies described herein are intended to serve as a minimum standard for Canadian anesthesiology graduates, individual trainees and residency programs are encouraged to expand their POCUS skillset beyond this core set of competencies. For example, competencies (e.g., Doppler assessment of aortic valve gradients) that wound up being discarded for a variety of reasons (e.g., “too advanced for the purposes of this document”) during the Delphi iterations would, nevertheless, be valuable skills for the practice of anesthesiology and perioperative medicine. Notably, and in accordance with the formal Delphi process, we did not formally address some narrative feedback from the Delphi rounds in the final version of competencies once consensus was achieved. The majority of such comments expressed conflicting concerns that some of the expectations related to cardiac assessment were too advanced for the purposes of this manuscript. For instance, one expert commented that only qualitative assessment of stenotic and regurgitant valvular lesions using B-mode and colour Doppler should be included, whereas another one not only suggested the implementation of a quantitative assessment but also that there should be “… objective criteria for peak velocities and gradients associated with severe aortic stenosis and mitral stenosis” included in the competencies. Conversely, two experts felt that colour Doppler should not be used to identify stenotic valvular lesions because of the advanced nature of this technique, whereas another expert (and for similar reasons) suggested that mitral stenosis and identification of systolic anterior motion of the mitral valve should be excluded. Similarly, identification of tricuspid stenosis was deemed of low prevalence by one expert and, therefore, unnecessary as part of these competencies. All in all, while the narrative feedback provided by the expert panel was of great value, conflicting (and often isolated) comments did not generate enough priority to discard competencies and/or generate new ones, and possibly reflected the heterogeneity in use of POCUS (with differing expectations from learners) across various institutions. The consensus competencies generated herein could, therefore, contribute to closing this gap and making POCUS training/practice more homogeneous across Canadian academic institutions moving forward.

Current variability and time constraints among different residency programs may limit trainees’ clinical exposure to some pathology-related competencies. It is understandable (and expected), therefore, that not all residents would experience (in the clinical setting) every one of the competencies described herein. Nevertheless, theoretical/conceptual knowledge of all pathology-related competencies (particularly with regards to image acquisition and interpretation) should be mandatory, and may need to rely on alternative (e.g., simulation, videoclip-based modules, etc.) training modalities. For instance, while an extensive pulmonary embolism is a rare condition that many anesthesiologists may not experience during training, POCUS practitioners should be able to recognize the McConnell’s sign to facilitate diagnosis and management. This is akin to other rare clinical conditions (e.g., malignant hyperthermia) in anesthesiology where the theoretical knowledge (e.g., clinical presentation, diagnosis, management) is often “enforced” upon trainees through alternative (simulation education) teaching modalities. Similarly, we acknowledge that the 75 competencies proposed herein may not be all easily shown/achieved by a routine bedside POCUS exam; rather, some may need to rely on alternative teaching modalities to be mastered.

There are several limitations to this study. Since our expert panel consisted exclusively of academic anesthesiologists practicing in Canada, there is a potential lack of generalizability outside of the Canadian context, as well as to Canadian anesthesiologists practicing in community hospitals and rural areas. Likewise, it is unclear how these competencies would apply to attending anesthesiologists who entered practice prior to the widespread use of POCUS in perioperative medicine. Another important consideration is the likely inclusion of focused assessment with sonography in trauma, gastric ultrasound, and airway ultrasound in the next (updated) version of the RCPSC National Curriculum. As such, these guidelines will require iterative revisions and updates to increase generalizability and keep up with changing use of POCUS in anesthesia/perioperative practice.

The next stage of the present investigation will involve the development of assessment tools designed to measure POCUS proficiency among anesthesiology learners using the competencies outlined herein, to be implemented as part of the CBD assessments already in place in Canadian anesthesiology training programs. Once developed, validity evidence related to these assessment instruments will be generated using Messick’s unified framework of construct validity, in both a high-fidelity simulation environment as well as in the clinical context. The goal of this work is to create the first Canadian assessment tool(s) on perioperative POCUS based on national expert consensus.