UK and US studies suggest that 10% of patients admitted to hospital suffer adverse events [1, 2]. Half of these adverse events are thought to be preventable with today’s standards of care. Such statistics are relevant to the outcome of surgical patients [3], in whom 30–50% of complications are thought to be preventable. To improve safety, health care is increasingly looking for guidance from other high-risk industries. In aviation, a recognition that failures in team communication are the most important cause of serious errors [4] has resulted in a shift in the emphasis of training to include the assessment of cognitive and social skills (or non-technical performance).

Despite the many advances in surgical technique and technology over the last 50 years, minimal consideration has been given to the analysis of team dynamics, interactions and behaviour in the operating theatre [5]. Parallels between operating theatres and other high-risk teamwork environments suggest that attention to non-technical skills in the operating room is justified [6]. Although the proposition is intuitive for many experienced clinicians, it has not yet been established that the quality of operating theatre teamwork has any bearing on surgical technical performance, and it is possible to argue that the effect of team communication on the quality or outcome of surgery may be unimportant in comparison to the skills of the operator.

However, the operating team is in itself complex, consisting of three disciplinary subteams, namely surgeons, anaesthetists and nurses. Each subteam functions as a unit, as well as comprising an essential part of the overall unit, and all have to communicate and coordinate properly for success. Indeed, the performance of one subteam may be influenced by or influence the others.

The formal analysis of non-technical skills in health care is relatively new, and it has therefore been convenient to adapt existing tools from the aviation industry rather than invent new ones [7]. In the airline industry, non-technical performance is assessed using the NOTECHS scoring system [8]. The adaptation of this tool for health care also provides an opportunity to assess the implications of non-technical or teamwork skills on surgical technical performance.

This work builds on methods developed for previous studies that examined technical and non-technical performance in surgery [9, 10], and aimed to evaluate the relationship between non-technical performance and technical performance in an index operation.

Method

Laparoscopic cholecystectomy was selected as an index operation as it is the standard operation for treatment of gallstone disease [11]. The operation was observed live by a research fellow with experience of clinical laparoscopic surgery. Observations were made during the operation from the time that patient was wheeled into the operating room to the time when the patient was moved off the table.

Ethical approval for the study was obtained from the Milton Keynes local research ethics committee (study no: 04/Q1603/35 amendment 2). Informed consent was obtained from the patient, and all members of the theatre staff for observation of each procedure.

Surgical technical performance

Surgical technical performance was assessed using the observation clinical human reliability assessment (OCHRA) tool, which has previously been developed and used for analysis of technical error during laparoscopic cholecsytectomy [12, 13]. The operation is divided into a series of nine steps, each consisting of subtasks (see Table 1). Each subtask was considered to have 11 possible outcomes: 1 correct and 10 generic forms of errors (see Table 2). We made two minor adaptations to the original form of the tool to accommodate standard practice in our institution, where Verres needle insufflation is not used, and operative cholangiogram use has declined dramatically due to the easy availability of preoperative MRCP.

Table 1. Nine OCHRA steps of laparoscopic cholecystectomy (with sub-tasks). Correct outcome and 10 generic forms of error for each subtask, shown in step 3.1. (Adapted from Joice et al., substituting open creation of pneumoperitoneum for Verres needle use and excluding steps relating to intraoperative cholangiogram).
Full size table
Table 2. definitions of NOTECHS categories and NOTECHS scoring system.
Full size table

Non-technical performance

The research fellow was trained in the use of the NOTECHS observation tool by a retired British Airways pilot (TD) with extensive experience in delivering airline teamwork training and previous observational experience of operating theatre environments. Familiarity with the NOTECHS system and reliability in its use was obtained by live parallel use in theatre in conjunction with the trainer, prior to commencement of the study. This process continued until the independent scores of the trainer and trainee were consistently in good agreement. Reliability of the NOTECHS observations was formally assessed on 10 of the cases in this study by a second observer, who was a human factors expert (KC), with no formal healthcare training, but with previous experience of observing in the operating theatre using the surgical NOTECHS system.

The surgical NOTECHS scoring system was adapted from a similar tool in aviation, and had previously been used to score theatre teams [9]. NOTECHS classifies non-technical skills into four dimensions: (1) leadership and management (LM), (2) teamwork and cooperation (TC), (3) problem-solving and decision-making (PD) and (4) situation awareness (SA) [14], with a score of between 1 and 4 given upon each dimension (Table 2). Each of the three subteams (surgeons, anaesthetists and nurses) was scored on each dimension for every operation. Overall subteam performance was taken as the sum of the dimension performances (out of 16). The overall team non-technical performance was calculated from the sum of the overall subteam performance scores (out of 48). Each overall team dimension performance was scored as the sum of all the subteam performances in that dimension (out of 12). Thus, a non-technical score was obtained in each dimension for the theatre team, and for each subteam of surgeons, anaesthetists and nurses. A high NOTECHS score reflects good communication, mutual support, coaching, pitfall discussion, goal setting, critical stage discussion, and an ability to recognise current and predict future surgical requirements or actions.

Statistics

Comparisons between subgroup scores were performed using Mann-Whitney U analysis. Reliability of the NOTECHS observations was calculated with Cronbach’s alpha. Assocation between NOTECHS scores and sub-scores and OCHRA observations was evaluated by Spearman’s rank correlation.

Results

A total of 28 patients were recruited for the study in one hospital. On two occasions, the research fellow was recruited to scrub in due to the absence of a surgical assistant and was therefore unable to complete observation on these cases, which observations from 26 elective laparoscopic cholecystectomies. There were no conversions to open procedures and no intraoperative cholangiograms were performed, and mean intraoperative duration was 75.4 ± 6.46 (95% CI).

Surgical technical performance

Between zero and six technical errors per operation were observed, with a mean of 2.62 ± 0.55 (95% CI). The number of technical errors in each operation can be seen in Figure 1, with the types of technical errors observed shown in Figure 2.

Figure 1.
figure 1

bar chart representing the frequency of number of TE per operation.

Full size image
Figure 2.
figure 2

The number of TE observed throughout the 26 cases. None of these errors resulted in identifiable harm to the patient, but some required corrective measures to be taken, e.g., suction/irrigation for perforation of the gallbladder. Step 2.1 (inspection of the abdomen) was repeatedly partially completed, possibly because it may be argued that complete inspection of the abdomen is an unnecessary step in the procedure. Steps 8.2 and 8.3 (inspecting for clips at the end of the procedure) were also often partially done, as they were deemed unnecessary by the surgeon.

Full size image

Non-technical performance

The mean team NOTECHS score was 35.5 ± 1.88 (95% CI) out of a maximum of 48. The mean subteam scores for surgeons, anaesthetists and nurses were 13.3, 11.4 and 10.8, respectively. The difference between surgeons and anaesthetists was significant (U = 197, p = 0.009) as was the difference between surgeons and nurses (U = 134, p < 0.001). There was no significant difference between the anaesthetists and nurses (U = 294, p = 0.422). The dimensions and performances of the teams and sub-teams are shown in Figure 3.

Figure 3.
figure 3

(a) bar chart showing subteam performances in each NOTECHS dimension, (b) bar chart showing mean team dimensions performances.

Full size image

Reliability of the NOTECHS observations

Cronbach’s alpha for the 10 dual observed cases was 0.880 for the total team score and thus there is good reliability for the NOTECHS observations between the surgically-trained and non-surgically-trained observer.

Relating non-technical performance to technical errors.

Analysis of the correlation between dimension scores and OCHRA outcomes showed a strong negative correlation between team situational awareness and technical errors (ρ = −0.505, p = 0.009, see Table 3), though the correlation between overall team NOTECHS score and technical error rate was weak (ρ = −0.16, p = 0.436). However, the strongest correlation was between technical errors and the situational awareness of the surgeon subteam (ρ = −0.718, p <0.001). The weak correlations with the other subteams suggest that the weaker correlation with team situation awareness is entirely the result of the situation awareness of the surgeon. Application of the Bonferroni correction for multiple comparisons suggests an appropriate level of significance of p < 0.003. Though it is important to avoid a type II error [15] this would add further weight to the view that the situation awareness of the surgeon’s subteam, rather than the situation awareness of the theatre team as a whole, is the important factors with respect to technical errors.

Table 3. showing correlation coefficients between NOTECHS dimensions performance (per team and subteam) and technical errors.
Full size table

Discussion

Situation awareness describes the perception, comprehension and prediction of all features of importance in a dynamic environment [14, 16], and thus reflects a collection of cognitive skills. In the present study, surgical error was closely correlated with the situation awareness of surgeon subteam, which supports the results of previous studies in identifying the importance of cognitive skills in avoiding, or contributing to, surgical errors. Though not established, a causal relationship is probable when considering that components of the OCHRA measure of technical errors also imply cognitive processes. The surgical focus of the OCHRA measurement system, and the differences in the demands of non-technical skills in the other two subteams may explain why there was no relationship between their non-technical performance and the technical performance of the surgeon. However, broader measures of intraoperative performance may yield different results, and the relationship between technical and non-technical skills, and the contributions made by subteams may vary between different types of surgery. OCHRA is the only currently available objective methodology for assessing the technical performance of laparoscopic cholecystectomy. It is necessarily arbitrary in the selection of items to be recorded, but it seems sensible and comprehensive in describing the essential steps of the operation and so has good face and content validity [17]. From the available published data it also appears to have good construct (predictive) and criterion validity, and excellent inter-observer reliability.

The reliability of the NOTECHS scoring systems appears good to date and suggests that NOTECHS and OCHRA can be used simultaneously. Ideas developed in aviation about the relationship between technical error and non-technical teamwork can only be applied to the operating theatre with considerable caution. The situation facing the commercial pilot is less variable than that facing the surgeon even in routine elective surgery. However, although the range of acceptable actions for commercial pilots is considerably more restricted than for surgeons, the principles of good surgical practice are well recognised in respect of laparoscopic cholecystectomy. Quality of performance can therefore be defined in a similar fashion for both professions, even if the freedom of action of the surgeon remains (for the moment) considerably greater.

There are a number of advantages of considering cognitive skills in terms of situation awareness. The notion of situation awareness is commonly used in other high-risk industries [18], and so identifying the value of situation awareness in healthcare will aid the transfer of knowledge and good practice. As a consequence, it is possible to suggest methods of training that might be employed to improve cognitive performance, and thus technical performance, though care must be taken to learn the right lessons from other industries.

A range of measures of situation awareness can be used to inform methods for evaluation purposes, including NOTECHS, the anaesthetists’ non-technical skills assessments (ANTS) [19], situation awareness rating technique (SART) and the situation awareness global assessment technique (SAGAT) [20], and further assessment of postoperative complications, cancellations of cases, delays in lists, returns to theatre, recall of surgeon/anaesthetist to recovery and clinical incidents is necessary to ascertain the influence of non-technical skills on clinically relevant outcomes. We are currently examining more cases in different operations, and adopting broader measures of intraoperative performance, including clinical outcome. By examining the relationship between the surgeon, the team, the patient, and the operating environment [21], we hope it will be possible to examine the mechanisms by which adverse events happen, and ultimately reduce their incidence.

Formal laparoscopic training is conducted through lectures, simulation, and in operating theatres [22], but focuses on specific psychomotors and the particular technical skills required for laparoscopy. Though this training often explicitly incorporates some elements of awareness, it may not encompass the full range of situation awareness skills. Many of these – such as the gaps in the knowledge of the other members of the team – may be far less explicit, but may be an underappreciated source of adverse events in the operative theatre [23]. Training in non-technical skills, and situation awareness in particular, might benefit laparoscopic skills such as viewing two-dimensional video images 2 m away from the operation site, limited feedback, smaller incisions and only tips of long instruments being visible. Situation awareness can also be improved by controlling external distractions [24], anticipation of future events [25], and appropriate use of all members of the team [26]. Assessment of non-technical performance in other operations will also be required to develop a clearer picture of the importance of these skills in surgical performance, and as part of these ongoing studies, the effects of a training course to improve these skills in the operating theatre are currently being evaluated.