Introduction

During the past few years, the surgical literature has seen a number of attempts to broaden the way surgical performance and outcomes are conceptualized [1, 2]. These approaches, often termed “system approaches,” suggest that surgical outcomes can be better understood when the methods used extend beyond the technical skills (e.g., motor co-ordination) [3–5] or nontechnical skills (e.g., decision-making) [6–8] of individual surgeons. In addition to such skills, systems approaches suggest that a better understanding is needed of the interactions between members of the surgical team and the interaction between members of the team and the surgical environment. Our focus in this study was on the operating room environment.

Empirical evidence on what the surgical environment is like and what its impact is on surgical processes and outcomes is scarce. This is rather surprising: behavioral science suggests that environment is a major determinant of the outcomes of human action (e.g., its chances of success or failure) [9, 10]. Recent attempts to fill this lacuna have focused on what may distract the surgical process in operating rooms. In a recent study performed by our group in a U.K. teaching hospital [11], we investigated the impact of distractions and interruptions on surgeons, anesthetists, and nurses during routine general surgical procedures. We recorded the frequency of the distractions and interruptions as well as their visible impact on the affected team-members in 50 general operations (open and laparoscopic). Among other distractions and interruptions, those related to communication, equipment, procedures, and the operative environment occurred most commonly and were most visibly disruptive. Focusing on communication, the most distracting communications were those related to (1) operating room equipment and provisions, (2) responses to queries about other patients, (3) on-going management of the operating list with the members of the operating room team, and (4) teaching that senior clinicians had to deliver as they were performing procedures [12]. Similar findings were obtained in urological surgery [13], whereas similar patterns of often interrupted clinical work have been found in specialties outside surgery, including intensive care units [14], emergency departments [15, 16] and trauma centers [17].

Studies such as these are useful in that they offer systematic, quantitative description of the work environment of a typical surgical team. Additionally, they allow systematic assessment of the relations between surgical environment and surgical outcomes and processes via the provision of replicable and quantifiable measures. However, observational assessments of the surgical environment tell a limited story. People differ in the way they perceive, filter, and evaluate environmental stimuli [9, 10]. In the context of surgery, recent observational studies indicate that operating room staff from different professional backgrounds have inaccurate perceptions of roles and working as a team with their colleagues from other specialities [18, 19]. Perhaps it is not surprising that communication failures occur in operating rooms [20]—often as a result of misunderstanding of colleagues’ roles and priorities [21]. Surgeons, as a professional group, feature heavily in these findings.

The study that we report investigates operating room professionals’ perceptions of the surgical environment and its impact on operating room staff performance. The study was intended to supplement existing evidence on the surgical environment from observational studies. We report the development of the “Disruptions in Surgery Index” (DiSI): a tool that captures self-perceptions of operating room staff regarding the disruptions that they and their colleagues have to deal with in the operating room. Our goal in developing the DiSI was to produce a tool that captures clinicians’ views consistently (reliable) and also captures distinct types of disruptions and not anything else (valid). We describe the development and contents of the tool, provide initial evidence on its reliability, and report preliminary findings from the first administration of DiSI.

Materials and methods

Participants

We collected data from all three subgroups of professionals that make up a typical operating room team. Sixteen surgeons, 26 nurses, and 20 anesthetists/operating departmental practitioners (in the United Kingdom, ODPs are operating room technicians trained to assist the anesthetists) completed the questionnaire at a time and place convenient to them.

Materials

DiSI was developed by using input from observational studies of the operating room environment that were concurrently being performed by our group [11–13], published evidence and anecdotes, and also senior clinicians. In DiSI, surgical disruptions are grouped into seven different types. Each disruption type is assessed with two or more specific items. Table 1 summarizes the types of disruptions and the focus of the individual items.

Table 1 Disruption in Surgery Index dimensions (initial structure and content)
Full size table

Content domain of each disruption type

  1. A.

    Individuals’ skill, performance, and personality: This relates to individual clinicians’ performance and personality. Anecdotal evidence suggests that personality characteristics affect teamwork in operating rooms [22].

  2. B.

    Operating room environment: This relates to the environmental conditions of an operating room and the distractions obtained in the observational studies (bleeps, phone calls, unavailable equipment, door openings, etc.) [11–13].

  3. C.

    Communication: Inclusion of communication as a separate disruption type was informed by the presence of communication as a major dimension in our conceptual and empirical modeling of surgical teamwork [23–25]. The specific content of this disruption type reflects the distracting communication exchanges that we obtained in an observational study [12], as well as language barriers.

  4. D.

    Coordination and situational awareness: Inclusion of this disruption type was informed by the presence of coordination/situation awareness in surgical teamwork models [23–25]. In addition, the chosen items reflect the findings of our observational studies (e.g., managing the operating list during the course of the day) [12], as well as anecdotal evidence (e.g., people being late/absent).

  5. E.

    Patient-related disruptions: This disruption touches on a major patient safety issue: the surgical team having access to full and accurate information on the surgical patient.

  6. F.

    Team cohesion: Team cohesion originates in the organizational behavior and teamwork in nonclinical contexts literature. It reflects individual team-members’ feeling part of and identifying with the team [26].

  7. G.

    Organizational disruptions: This disruption encapsulates some of the macro-management issues that affect working in an operating room, including the fact that the delivery of surgical services and teaching occur concurrently and under time pressure (because of staffing levels, waiting lists, etc.). Some of these issues appeared in our observational studies [12].

For each specific disruption type, the clinicians who participated in the study provided the following measures:

  1. (i)

    How often, on average, they observe a specific disruption in the operating room (percentage scale: 0 percent = disruption is never observed; 100 percent = disruption is always present).

  2. (ii)

    How much each disruption contributes to potential error (10-point scale: 0 = not at all; 9 = extremely).

  3. (iii)

    How much each disruption obstructs the achievement of the goals of the procedure (10-point scale: 0 = not at all; 9 = extremely).

Participants provided these ratings twice: once for themselves and once again for their colleagues in the operating room (resulting in 6 measures per disruption). For example, for the “tiredness” item of the individual issues disruption participants gave the following responses:

  1. (i)

    How often, on average, they and their colleagues are tired (2 frequency estimates)

  2. (ii)

    How much tiredness contributes to error in their own and their colleagues’ work (2 judgments)

  3. (iii)

    How much tiredness obstructs the achievement of the goals of the procedure for themselves and their colleagues (two judgments)

Statistical analyses

We performed reliability analyses (Cronbach α internal consistency coefficients). Cronbach α can range from 0 and 1; values from 0.7 to 0.9 are considered most desirable. Lengthier scales usually achieve higher α than shorter scales [27]. These analyses assess whether each disruption type has been scored consistently across participants.

We also submitted the data to analysis of variance (ANOVA). This analysis reveals differences in perceived disruptions across the three operating room subteams (surgical, anesthetic, and nursing), differences across the disruptions under investigation, and, finally, differences between the participants’ judgments for themselves and those that they furnished for their colleagues. Given the complexity of the ANOVA, we have simplified the presentation of our findings by removing excessive detail of the analysis from the Results. Interested readers can find a full description in the Appendix.

Results

Reliability analyses

Table 2 summarizes the Cronbach α coefficients for each one of the six measures (i.e., frequency estimates, error judgments, and obstruction judgments for self and others). Overall, Table 2 suggests acceptable reliability in our findings, because the majority of the coefficients fell within the required range. However, there were problems in the “team cohesion” and “organizational disruptions” across all three measures, and in the “communication” disruption in the measure of frequency.

Table 2 Reliability analyses (Cronbach alpha coefficients)
Full size table

For “team cohesion” and “organizational disruptions,” further analyses suggested that the reliability coefficients would improve if these two dimensions were combined. Therefore, we decided to condense them into a single “team and organizational issues” disruption type. The improved Cronbach α coefficients for the merged dimension can be seen in the bottom row of Table 2.

In “communication” only, the estimated frequencies exhibited poor consistency, whereas the other two measures (contribution to error and obstruction) achieved acceptable reliabilities. Despite this problem, we decided against removing this disruption, or merging it with any other. First, the problem only affected frequency estimates. These could be less robust than error contribution and obstruction judgments, although empirical testing is required to assess this. More importantly, existing evidence suggests that communication is a potent distraction in operating rooms and an area of work that can be improved [12, 20, 21], therefore, it makes sense to assess communication-related disruptions separately from other disruptions. Additional items, however, should be added to the “communication” dimension to improve reliability (this will be revisited in future studies).

Relative impact of disruptions

We computed total scores of disruption frequency, contribution to error, and obstruction for each disruption type by averaging the relevant items (Table 1). This calculation resulted in one set of three scores for one’s self and another set of three scores for one’s colleagues per disruption type (Table 3). The scores were submitted to ANOVA, with the following factors:

  • Group (surgical versus anesthetic versus nursing)

  • Disruption type (individual disruptions versus operating room environment versus communication versus coordination/situational awareness versus patient-related disruptions versus team/organizational disruptions)

  • Target of estimate/judgment (self versus others)

In the three subsections that follow, we report findings for disruption frequency, contribution to error, and obstruction separately.

Table 3 Means (standard deviations) of disruption scores across disruption types and self versus other focus
Full size table

Disruption frequency

Participants judged all disruptions to occur more frequently to their colleagues than to themselves (F(1, 38) = 12.41, p < 0.01). However, a significant interaction between target and disruption type (F(5, 190) = 4.43, p < 0.01) qualified this finding. Participants judged that others are more frequently affected than themselves by individual disruptions (p < 0.001), disruptions in the operating room environment (p < 0.001), and communication issues (p < 0.01) but not by any other disruption. In other words, the remaining disruptions (i.e., coordination/situational awareness, patient-related disruptions, and team/organizational disruptions) were judged to affect everyone in the operating room equally often.

The analysis also revealed differences across the three groups (F(2, 38) = 4.4, p < 0.05). Surgeons estimated lower frequencies for all disruptions (M surgeons  = 25%; SE = 5%) than nurses (M nurses  = 42%; SE = 4%; p < 0.01) or anesthetists/ODPs (M anesthetists/ODPs  = 37%; SE = 4%; p < 0.05).

Disruption contribution to error

Participants judged that, when disruptions occur they contribute more to error for their colleagues than for themselves (F(1, 36) = 6.20, p < 0.05). Moreover, the analysis revealed differences across disruption types regarding their contribution to potential error (F(5, 180) = 16.68, p < 0.001). Patient-related disruptions were judged as more serious contributors to error than all other disruptions (all pair-wise ps < 0.01) and communication issues were judged as less serious contributors than all disruptions except team/organizational disruptions (all pair-wise ps < 0.05). As in the previous analysis, a significant interaction between target and disruption type qualified both of the above findings (F(5, 180) = 4.79, p < 0.001). Participants judged others as more vulnerable than themselves to error resulting from individual issues (p < 0.001), disruptions in the operating room environment (p < 0.01) and from communication issues (p < 0.05). The remaining disruptions were judged as contributing equally to errors for everyone.

Goal obstruction by disruption

On the whole, patient-related disruptions were judged as more serious contributors to error than all other disruptions (F(5, 180) = 13.04, p < 0.001; all pair-wise ps < 0.01) and team/organizational disruptions were judged as less serious contributors than all disruptions except communication issues (all pair-wise ps < 0.05). As above, a significant target x disruption type interaction qualified this effect (F(5, 180) = 5.77, p < 0.001). The goals of the procedure were thought to be obstructed by individual disruptions (p < 0.01), by disruptions in the operating room environment (p < 0.05), and by communication problems (p < 0.05)—more for one’s colleagues than for one’s self. No such differences were obtained for the remaining disruptions (i.e., they were judged as obstructing equally the participants’ own and colleagues’ goals).

Discussion

The goal of the present research was to develop a tool that captures comprehensively perceived annoyances, interruptions, and disruptions in the surgical process. Based on empirical research that was concurrently being performed by our research group [11–13] existing evidence and anecdotal reports, and senior clinicians’ input, we developed the Disruptions in Surgery Index (DiSI). Administration of DiSI to a group of operating room professionals allowed us to investigate the relative perceived impact of different disruptions on surgical work in theatre.

An interesting initial observation is the high-frequency estimates of disruptions that we obtained. Across all professionals groups, estimates ranged between 25% (surgeons) and 42% (nurses), with the anesthetic group in-between (37%). Frequency estimates showed low variability across the different disruption types: they all ranged between 30% and 41%. These findings indicate that operating room staff witness and experience disruptions in between a third and just below half of the procedures that they actually perform—a rather high frequency by any standards. These findings are not inconsistent with those of earlier observational studies. Healey et al. [11] reported between 13 and 14 visibly distracting events per procedure, whereas Sevdalis et al. [12] reported between three and four instances of distracting communication per procedure. These findings provide converging evidence from self-report and observation that operating room staff face disruptions of various kinds on a frequent basis.

Interestingly, our participants exhibited a bias: they all shared the belief that some disruptions occur to and affect their colleagues more than themselves. Specifically, issues with individual clinicians, with the operating room environment, and with communication were judged to affect others more than one’s self. In contrast, disruptions relating to the coordination of work, or to the surgical patient, or, finally, to team and organizational issues were perceived as equally disrupting everyone’s work. A possible interpretation of this finding is that the disruptions that were thought to affect others more than one’s self are more related to individuals’ attitudes and performance than those thought to affect everyone equally. The specific content of the items used (Table 1) supports this explanation: being tired or short-tempered, being unable to concentrate when phones and bleeps go off during a procedure, or being affected by poor communication are likely to be taken more personally than the collective management of the operating room list, or the availability of patient results and notes, or hospital policies that affect the flow of work in an operating room. As a result, our participants may have been more willing to see such disruptions more in others’ behavior than in their own.

The surgeons in our sample reported significantly fewer disruptions than the anesthetic or nursing group. In other words, not only did the surgeons think that their colleagues experience on average more and more severe disruptions than themselves, but they also perceived fewer disruptions than other members of the operating room team. To some extent, this suggests surgical “bravado”: as a professional group, surgeons typically do not acknowledge disruptive stressors and their impact [28, 29]. Traditional surgical training, with its lack of emphasis on nontechnical skill, and the ensuing culture in the profession are likely to be at the route of such findings, although the link should be empirically explored. For our purposes, it is important to note that significantly discrepant views between operating room staff on what the everyday working environment is like (as well as in other issues, such as their respective roles, and teamwork) [18–21] suggest that any attempt to improve the surgical environment is likely to fail unless professionals from all specialties are consulted and their views understood. Simply put, it is likely that any change would be considered unnecessary unless a surgeon agrees that it reduces the negative impact of a disruption on his/her work. For this to happen, recognition of the disruption and its effect is a prerequisite.

A limitation of the study that we report stems from how the operating room staff was approached to complete DiSI. Because completed the questionnaire at a time and place convenient to them, the findings are subject to two potential biases. First, there are potential recency effects: if a participating surgeon happened to recall a recent case (e.g., a case just finished), which was particularly affected by a specific disruption, he would be likely to report higher frequency/severity of that disruption. Second, there also are potential availability effects: a past case unusually affected by numerous disruptions, if available in memory at the time of completion of the questionnaire, would probably lead a participating surgeon to report higher overall frequency/severity of disruptions, although the case might be unusual in this respect. Our results are potentially affected by such memory-driven biases from the recent past and also from each participant’s “bank” of experienced disruptions. However, we think that it is unlikely that all participants were biased in the same direction. The concordance of the present findings with findings from observational studies on disruption frequency (not subject to such biases) reinforces this point. Nonetheless, bias in questionnaire completion is a technical issue that researchers should be aware of.

From a conceptual point of view, DiSI is a useful new measure of perceived disruption of surgical work in that it explicitly models disruptions as a function of their frequency and their severity:

$$ {\text{DiSI score = Disruption Frequency }} \times {\text{ Disruption severity}} $$

This means that each disruption can be characterized for frequency and severity. A high DiSI score can be a function of an accumulation of many low-disruptive events (e.g., bleeps) or few really disruptive events (e.g., unavailable test result). The total score provides an indication of the total amount of disruption, which can then be examined for high-frequency/low-severity events, and/or low-frequency/high-severity events. (There will of course be cases where many highly disruptive events will accumulate, but these are likely to be relatively rare). Given the exploratory purpose of the work that we report, we did not perform an analysis of DiSI total scores; however, it can be done and then fed into computer simulation models that explain/predict the impact of disruptive environment on surgical processes and outcomes. For this purpose, further research should use DiSI in conjunction with technical/nontechnical skill, observational teamwork assessments, and other measures to obtain meaningful correlations between measures.