Introduction

The Accreditation Council for Graduate Medical Education requires graduates to demonstrate proficiency in six broadly defined domains including medical knowledge and patient care in order to be considered competent to practice in their respective disciplines [1]. In graduate surgical education, there has been much discussion on how best to ensure that trainees uniformly attain proficiency in the medical knowledge and patient care competency domains. Much of the focus is centered on competency-based education, which involves defining a standard set of skills and knowledge that reflect the duties of a profession on which education is based [5, 6, 26]. One important advantage of competency-based education is that the curriculum components are uniformly required of all involved programs so that there are no gaps or variations in trainees’ experiences in exposure to critical skills [15]. However, current hand surgery education lacks essential components of the competency-based education model. For example, there are currently no set requirements for determining proficienc, and no benchmarks set for sufficient exposure to any skills or knowledge in hand surgery education. Given this lack of standard requirements for exposure to skills and knowledge, it is perhaps not surprising that Sears et al. recently reported that that hand surgery fellowship program directors from two of the three primary specialties involved in hand surgery education offered differing opinions on skills and knowledge components that should be emphasized as essential in hand surgery fellowship [11]. These differing opinions are likely manifest as variations in the educational components that comprise curricula across hand fellowship programs [11].

In July 2010, the American Association for Surgery of the Hand (ASSH) took a step towards establishing standard essential skills and knowledge for hand surgery education when it unveiled the Spectrum of the Hand and Upper Extremity curriculum, a comprehensive outline of knowledge items and topics to facilitate standardized hand surgery training [3]. Sears et al. published a recent study aimed at soliciting the input of hand fellowship program directors regarding knowledge topics and procedures essential for competency in hand surgery [10]. In that study, a majority of the program directors of orthopedic- and plastic surgery-accredited hand fellowship programs agreed on approximately 170 knowledge topics and procedures they deemed essential for competency in hand surgery [10]. As of now, however, there is no standard curriculum for hand surgery education and so it remains unclear what gaps and variability in exposure to essential skills and knowledge exist across hand fellowship programs. Hence, the purpose and an extension of this study is to examine the fellowship educational experience of hand surgeons to identify gaps and variability in exposure to skills and knowledge judged essential by a majority of accredited hand fellowship program directors. Additionally, we aimed to examine if differences in exposure are manifest in practice patterns. We hypothesize that there is differential exposure to essential knowledge topics and procedural skills depending on fellowship type.

Materials and Methods

We used a role delineation study (RDS) design for this study. Role delineation studies are surveys that obtain information from professionals about their educational experience, preparedness for practice, and the nature of their current practice [17, 25, 29]. Role delineation studies are a widely used method to understand trends and differences in professional education [9, 16, 32]. Figure 1 shows a general schema for role delineation study design and our study design. The University of Michigan Institutional Review Board determined the study exempt from review.

Fig. 1
figure 1

Schematic of this Role Delineation Study (RDS) survey. This schematic is widely used in the design of RDS surveys. *American Society for Surgery of the Hand, ** American Association for Hand Surgery

Full size image

Survey Instrument

From a recent survey in which over 80 % of accredited hand fellowship program directors (PDs) participated, we selected 72 knowledge topics and 98 surgical procedures that ≥50 % of PDs identified as essential to competency in hand surgery [10]. We used these topics and procedures organized into 17 thematic categories as survey items (ESM 1). Approximately, 80 % of the participants in that study were PDs of orthopedic surgery (OS) Residency Review Committee (RRC)-accredited fellowships. As a result, there was potential for bias regarding topics and procedures contained in our survey. However, the survey of PDs had predetermined cutoffs for consensus on items deemed essential. Hence, to maintain methodological rigor, we used the same cutoffs [11]. Moreover, using ≥50 % agreement as consensus for defining essential items ensured that a comprehensive range of topics and procedures were included in the survey. This method of survey instrument development is customary for RDS survey design [9, 16, 32].

Owing to the large number of items that met our criteria (170 in total), we decreased the survey burden on potential responders by dividing both the knowledge topics and procedures each into two lists of items yielding four survey versions. This practice is also common practice in conducting RDS surveys with a high burden of survey items [12, 19]. Each of the survey versions contained items in three segments. The first segment inquired about responders’ practice composition by seven anatomic domains including the hand, wrist, distal radius and ulna, forearm, elbow, shoulder, and brachial plexus (Fig. 2). It also inquired about their level of exposure to each anatomical domain during fellowship (Fig. 2). The second segment inquired about the level of exposure during fellowship to knowledge topics or procedures (Fig. 3a and b). Finally, the third segment inquired about demographic data. For each anatomical domain in the first segment and detailed knowledge topic or procedure item in the second segment, responders rated their level of exposure in fellowship as =1 if they thought they received enough exposure for proficiency, =2 if they thought they did not receive enough exposure for proficiency, and =3 if they received no exposure (Figs. 2 and 3a, b). We piloted all four instruments with five fellowship-trained hand surgeons and an expert in the development of psychometric instruments including the national Health and Retirement Survey [31]. We made adjustments to the survey instrument to ensure brevity, enhance clarity, and flow based on feedback.

Fig. 2
figure 2

Items on first segment of the survey. The items were related to anatomic domains of the hand and upper extremity

Full size image
Fig. 3
figure 3

a Example of a knowledge topic survey item. b Example of procedure survey item

Full size image

Dissemination

Our sampling frame was the membership of the ASSH and the American Association for Hand Surgery (AAHS). After approval from the leadership of both organizations, we obtained a list of electronic mail addresses of the membership. We divided the electronic mail addresses into four approximately equal groups and then sent one of the four surveys to each group. In all, we sent prenotification, invitation, and three reminder messages to nonresponders over the course of 8 weeks. Survey participants were not compensated.

Analyses

We excluded responders who practiced outside the USA and its territories, and then we calculated summary proportions for demographic items. In order to analyze and present the data concisely, we created scales consisting of related knowledge topic and procedure items based on clinical themes (ESM 2). We estimated Cronbach’s alpha statistics, a measure of internal consistency, for each scale (ESM 2) [8]. A group of items with a Cronbach’s statistic of ≥0.7 is generally accepted as having good reliability in measuring a common theme [8]. Items that had a Cornbach’s alpha <0.7 or had no variance in exposure ratings were analyzed individually.

We made comparisons based on the type of hand fellowship program attended by responders (OS vs. plastic surgery (PS) RRC-accredited fellowships). We excluded graduates of general surgery RRC-accredited fellowships from analytical comparison because there were too few of them among responders to yield reliable point estimates. For each anatomical domain (e.g., hand, wrist, etc.), we calculated percentages of each group who indicated adequate exposure for proficiency and the percentage that indicated no exposure. For each knowledge topic and procedure scale, we calculated the mean level of exposure during fellowship for each comparison group. We used two tailed t tests to compare mean exposure responses between the comparison groups.

Results

A total of 2,779 ASSH and AAHS members received invitations to participate in the surveys, and 582 responded with sufficient data for analysis. We excluded 20 responders who practiced outside the USA and its territories, yielding 562 observations including 443 OS RRC-accredited fellowship graduates and 58 PS RRC-accredited fellowship graduates. This made for a 21 % response rate. Participant characteristics are displayed in Table 1.

Table 1 Demographic characteristics of survey responders
Full size table

Anatomical Domains

The hand domain was the only one in which ≥90 % of graduates of PS RRC-accredited fellowships indicated receiving enough exposure for proficiency (Fig. 4). In five of the seven anatomical domains, including distal radius/ulna and forearm, >10 % of graduates of PS RRC-accredited fellowships indicated receiving no exposure (Fig. 5). Additionally, in four of seven domains including the forearm, >20 % of graduates of PS RRC-accredited fellowships indicated receiving no exposure (Fig. 5). In contrast, there were only two domains (shoulder and brachial plexus) in which ≥10 % of graduates of OS RRC-accredited fellowships indicated receiving no exposure. Table 2 shows the mean exposure ratings for each anatomical domain by comparison group.

Fig. 4
figure 4

Percentage of responders in each comparison group with enough exposure for proficiency in each anatomical domain during fellowship. *Distal radius/distal ulna

Full size image
Fig. 5
figure 5

Percentage of responders in each comparison group with no exposure in each anatomical domain during fellowship. *Distal radius/distal ulna

Full size image
Table 2 Exposure ratings for anatomical domains by Residency Review Committee fellowship accreditation
Full size table

Exposure Ratings

The 170 knowledge topic and procedure items were grouped into 79 scales. A scale rating of <1.5 indicates that majority of responders reported having enough exposure for proficiency for items within that scale and a value of ≥1.5 indicates that majority of responders reported not having enough exposure for proficiency for items within that scale.

In 17 (22 %) of the 79 scales, graduates of PS RRC-accredited fellowships reported aggregate exposure ratings of ≥1.5. These scales included procedures for carpal osteoarthritis, CRPP/ORIF/ex-fix for carpal/distal radius and ulna fractures, and intercarpal repairs/carpal fusions/wrist arthroscopy (Table 3). In contrast, graduates of OS RRC-accredited fellowships reported aggregate exposure ratings of ≥1.5 in 8 (10 %) scales including soft tissue defects reconstruction and procedures for thumb carpometacarpal (CMC) arthritis such as prosthetic arthroplasty (Table 3).

Table 3 Scales with exposure ratings ≥1.5 (exposure ratings ≥1.5 indicate the majority of responders (>50 %) reported receiving inadequate exposure to achieve proficiency) for graduates of plastic surgery (PS) and orthopedic surgery (OS) Residency Review Committee-accredited fellowships
Full size table

Furthermore, in 15 (19 %) of the 79 scales, graduates of PS RRC-accredited fellowships reported significantly poorer aggregate exposure ratings than graduates of OS RRC-accredited fellowships (Table 4). Examples of these scales include: fracture/dislocations/malunions/non-unions of the hand, carpus, distal radius/ulna and forearm, and treatments of carpal avascular necrosis including vascularized bone grafts/fusions/denervation (Table 4). There were no scales in which graduates of PS RRC-accredited fellowships reported aggregate exposure ratings significantly better than graduates of OS RRC-accredited fellowships.

Table 4 Comparison of exposure ratings for scales in which graduates of orthopedic surgery Residency Review Committee-accredited fellowships report significantly better exposure than graduates of plastic surgery RRC-accredited fellowships
Full size table

Frequency of Application in Clinical Practice

Graduates of PS RRC-accredited fellowships contributed ≥60 % to the total frequency tally in 12 (15 %) scales including microsurgery and soft tissue defect reconstruction (without free tissue transfer; Table 5). Graduates of OS RRC-accredited fellowships contributed ≥60 % to the total frequency tally in 33 (42 %) scales including scales such as CRPP/ORIF/ex-fix of carpal/distal radius and ulna and vascularized bone grafts/carpal fusion for avascular necrosis (Table 5). In other words, practice patterns in general appear to be related to the level of exposure gained to skills and knowledge topics during fellowship training.

Table 5 Proportion of contribution from each comparison group to the total frequency of performance of procedures and application of knowledge topics
Full size table

Discussion

In this national role delineation study, responses from US hand surgeons indicated that there is significant variation in exposure to different knowledge topics and procedures during hand surgery fellowship. Study participants, especially graduates of PS RRC-accredited fellowships, also indicated that there are a considerable number of knowledge topics and procedures in which a majority received what they perceived as inadequate exposure for proficiency or no exposure at all. Lastly, it appears that one consequence of the variations and gaps in exposure demonstrated in this study was that there was a trend, within groups of surgeons, towards focusing practice on areas in which they aggregately reported adequate exposure for proficiency.

These findings appear to augment existing literature on the potential consequences of differential exposure to skills and knowledge in hand surgery education. For example, Stern highlighted the differential pass rates on the Certificate of Added Qualifications (CAQ) examination between plastic, orthopedic, and general surgeons [28]. He noted that American Board of Plastic Surgery (ABPS) examinees had approximately tenfold higher failure rates than American Board of Orthopaedic Surgery (ABOS) examinees, likely due to substantial gaps in the exposure to the content of the certifying examination that ABPS examinees received [28]. Indeed, the results of this study that demonstrated that a majority of responders who are graduates of PS RRC-accredited fellowships reported inadequate exposure for proficiency in over a fifth of the knowledge and procedures scales perhaps provides part of the explanation for the higher CAQ examination failure rates. In essence, such significant variations and gaps in hand surgery education should be critically examined and remedied.

Studies from several surgical disciplines including general, urologic, and vascular surgery have likewise reported similar variations and gaps in exposure to essential knowledge and skills [7, 21, 27]. Educators in these disciplines have also reported similar consequences of such gaps including high failure rates on certification examinations [27]. However, there are active efforts being made across disciplines to address the variations among programs. One recurring concept in many of these efforts is the development of competency-based standard curricula [4, 18]. An example of the effort to develop a standardized curriculum is the work of the Surgical Council on Resident Education (SCORE) [4], in which stakeholders in general surgery education developed a list of conditions and procedures to form the basis of a standardized curriculum. One crucial advantage of such a curriculum is that the performance of programs in exposing their trainees to components of the standard curriculum can be evaluated [7, 13]. Gaps and variations in exposure are thus uncovered and can be corrected in order to have trainees meet the uniform standard of exposure to curricular components [7, 13]. In similar fashion, a standard curriculum for hand surgery education should stipulate that programs be evaluated on the adequacy of exposure their trainees receive to essential skills and knowledge.

However, the ability to achieve standardization of curricula and benchmarks for sufficient exposure to components of such standardized curricula is contingent on the leadership of hand surgery in the three involved specialties establishing prerequisite standards for entering hand fellowships, such as achieving better balance in exposure to hand-related knowledge during residency and determining the appropriate structure for education (e.g., fellowship models vs. early specialization models) in their respective disciplines in order to meet the requirements of a standardized curriculum [14, 30]. In disciplines similar to hand surgery in which the areas of clinical expertise encompass skills and knowledge common to more than one primary specialty, conjoint boards have been a repeatedly used approach for allowing professionals in such disciplines to set educational standards in concert with their respective primary boards [20, 24]. For example, the scope of emergency medicine covers assessment of conditions across several disciplines, medical and surgical alike. However, to achieve the responsibility for determining requirements for certification and designing a certification examination for their profession, a conjoint board was formed in 1979 in concert with seven supporting primary specialties including general and orthopedic surgery [24]. Another example demonstrating the advantages of autonomy gained by disciplines in establishing educational standards and structure is in vascular surgery. The Vascular Surgery Board (VSB) as a sub-board of the ABS succeeded in gaining primary responsibility for determining educational standards and structure in vascular surgery in 2005 and has since introduced an integrated model in vascular surgery with the underlying belief that this provides far more efficient exposure to skills and knowledge pertinent to competence in vascular surgery [18]. Omer’s observation that delegates to the Joint Committee for Surgery of the Hand in the process of determining the contents of the certification examination, could only reflect the very different majority decisions of their primary boards is perhaps emblematic of one of the critical challenges of designing a standardized curriculum and alternative structures for hand surgery education [22].

The most notable limitation in this study was the 21 % response rate, which raises concerns about nonresponse bias and external validity of study results. However, results from this study are in conformity with results from previously published studies on knowledge and skills similar to those examined in this study. For example, Payatakes et al. demonstrated that significantly more graduates of PS RRC-accredited fellowships performed digital replantations and the results from this study similarly demonstrated that >60 % of the median frequency tally for digital replantations were performed by graduates of PS RRC-accredited fellowships (Table 5) [23]. Moreover, Sears et al. demonstrated that program directors of OS RRC-accredited fellowships had significantly higher preference for considering knowledge of forearm fractures as essential to hand surgery competency and results from this study reflect this preference (Table 4) [3]. This conformity to known trends in published reports may be cautiously regarded as a measure of external validity as suggested by the American Association for Public Opinion Research, a national resource on survey research quality [2].

Addressing variations and gaps in hand surgery education can be facilitated by standardization of curricular components to which involved programs are obligated to provide trainees a benchmark level of exposure sufficient for achieving proficiency. Although there appears to be no single optimal path to achieving such standardization of curricula content, educators in several medical and surgical disciplines provide notable examples of steps that could be taken to attain the autonomy necessary to achieve such standardization [13, 20, 24]. It is time that there are consistent educational standards to ensure that the certification of a hand surgeon confirms comprehensive competencies to manage various components of clinical practice well articulated in a standardized curriculum.