Abstract
For over 30 years, aviation has conducted training courses to enhance team performance and improve safety involving simulation with observation and directed feedback. Participants’ performance is observed by trained and experienced observers who then provide feedback using behaviour-based evidence noted during the simulator exercise. More recently, in healthcare, operating theatre personnel have adopted simulator-based training (SBT), observation and feedback for learning and practice to reduce the potential for human errors and improve safety. Maritime and nuclear power also incorporate high-fidelity simulators and feedback in team training interventions including technical and non-technical skills. The design and development of drilling rig simulators means that drill crews can now practise and test out their decision-making and receive feedback from observers, with the aim of improving team non-technical skills and consequently reducing the potential for errors. This paper presents five principles gleaned from research and the experiences of both aviation and healthcare to be applied to the development of simulator-based exercising for drilling teams. The principles include: (a) developing learning objectives and expected performance standards; (b) training the team as a whole; (c) using a structured observation tool; (d) providing feedback during a structured debrief; (e) repeat the SBT regularly to enhance expertise and retain performance standards. It is anticipated that these principles can be generalised for simulator-based exercising to benefit team social and cognitive competences in other high-hazard or process industries.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
High-hazard industries, such as aviation and healthcare, rely heavily on effective functioning of teams primarily due to the complex, dynamic, and safety critical nature of their industries. It is not enough that these teams are made up of individual experts, but the team itself must be an expert team (Salas et al. 1997) in order to demonstrate high levels of not only technical performance but also team attitudes and behaviours to operate safely and adaptively to achieve their goals. An expert team is characterised, according to Salas et al. (2006), as having “a set of interdependent team members, each of whom possesses unique and expert-level knowledge, skills, and experience related to task performance, and who adapt, coordinate, and cooperate as a team, thereby producing sustainable, and repeatable team functioning at superior or at least near-optimal levels of performance” (p. 440).
Teams working on drilling rigs share similar characteristics to those in aviation and healthcare in that they operate in high-hazard environments where situations are dynamic, risky, uncertain, involve multiple operators, and where membership is not stable. Drilling teams are made up of skilled individuals who may or may not know each other and may not have worked together previously, moreover, team members can be geographically remote (Lauche et al. 2009). Key members of a drilling team, as illustrated in Fig. 1, include representatives from the operator, the contractor, and service companies (i.e. specialists who work within the team for varying periods of time dependent on the phase of operations). As Haavik (2011) states, cooperation between different disciplines and division of labour can lead to issues with differing goals, perspectives, and timescales.
These personnel may only meet once the drilling plan is ready to be operationalised and will have to rapidly form an effective team. This team will work together for the duration of the drilling programme which may last weeks or months. The emphasis traditionally for drilling teams has focused on technical competence of team members, with less recognition of the impact of non-technical skills. However, following analyses of accidents in the oil and gas industry, including the Macondo tragedy in April 2010 (Chief Counsel 2011), there is growing recognition of the influence of skills such as communication, situation awareness and decision-making on safety.
Dynamic situations require that teams, particularly “control crews”, are trained to react in an effective and timely manner. Control crews are defined by Waller et al. (2004) as highly trained teams responsible for monitoring complex systems, performing routine procedures, and quickly responding to non-routine situations. Team members need assurance that their colleagues know and understand their roles and responsibilities, and can use their skills and knowledge individually and collaboratively to intervene and recover control. Assurance can be developed over time as team members increase their familiarity with each other and improve personal working relationships. Where there is the potential for things to rapidly deteriorate from ‘normal’ conditions to ‘abnormal’ conditions, such as in aviation when severe weather occurs, in healthcare when a patient responds unpredictably during an operation, or on the drill floor when a routine task starts to go wrong due to equipment failures, there is even more reliance on the team to respond quickly and adaptively to manage the situation (Entin and Serfaty 1999).
High reliability industries have typically introduced training, such as crew resource management (CRM: Helmreich and Foushee 1993; Helmreich et al. 1999), to support and enhance team performance. This type of training frequently incorporates simulator-based exercises to allow team members to practise both their technical and non-technical skills and receive feedback on their performance. Non-technical skills, in the aviation environment, refer to “all pilots’ attitudes and behaviours in the cockpit not directly related to aircraft control, system management and standard operating procedures” (van Avermaete 1998, p. 4).
The experiences of aviation and healthcare in integrating simulator-based exercises into training designed to improve safety and performance can therefore provide guidance when developing simulator-based exercises on drill floor simulators. Technical advances in drilling simulator software and hardware now means that simulator-based exercises are increasingly available to develop both the technical and non-technical skills of drilling team members. To ensure that training interventions are suitable for team members to develop and enhance the necessary skills and knowledge to carry out their operations, detailed, standardised guidance is required.
1.1 Five principles for developing simulator-based exercises
Based on the experiences described above, predominantly gleaned from aviation and healthcare, the following five principles should be applied when designing training interventions to improve safety and effectiveness:
Principle 1
Develop learning objectives and expected performance standards.
A team task analysis should precede any training intervention to identify the task and team requirements. Simulator-based training interventions should be developed around specific learning objectives, with predetermined measurements for performance standards. Cues to stimulate the expected behaviours should be embedded into the scenarios to allow the expected behaviours to be practised and demonstrated.
Principle 2
Train the team as a whole (simulator-based team training).
Although much can be gained from training individuals’ non-technical skills, there are huge advantages to training the team as a whole during simulator-based exercises. To improve team effectiveness, teams should be trained as a whole to develop team knowledge, skills, and attitudes, improve coordination, foster shared mental models and accurate expectations of the requirements of the team, and encourage adaptability and flexibility.
Principle 3
Use a structured observation tool (behavioural marker framework).
A structured observation tool should be used in order to capture positive team behaviours as well as areas for improvement. A predetermined descriptive framework should define the behaviours of the key non-technical skills to be observed. The framework can be used as a basis of the training design, used to provide feedback, and used for self-reflection.
Principle 4
Provide feedback during a structured debrief.
Feedback should take place face-to-face immediately following a scenario-based simulator exercise, and highlight team behaviours observed during the exercise. The debrief should focus on the team processes, such as who said what, who was involved in making decisions, which roles were involved in coordinating activities, and so forth, rather than only focus on the outcome of the exercise.
Principle 5
Repeat the simulator-based training regularly to enhance expertise and retain performance standards.
Simulator-based exercises should not be expected to change team behaviours after one exposure. Exercises should be repeated on a regular basis, dependent on access to the simulator, demands on the team, and also how long the team has worked together. Newly formed teams will benefit greatly from sharing experiences on a simulator to develop Team KSAs. Longer-established teams can also take advantage of the opportunity to test out new responses, practise new procedures, or become familiar with new equipment or processes.
This paper explains the rationale supporting these proposed principles. The paper starts by outlining work undertaken by Salas and colleagues emphasising the importance of structured training for improving team performance. The key concepts of simulator-based training (SBT) and simulator-based team training (SBTT) are presented along with a description of SBT in high-hazard environments such as aviation and healthcare, and reported benefits in terms of team behaviours, attitudes, and cognition. The development and use of a behavioural framework to evaluate team performance and provide feedback based on observations of team performance during simulator-based exercises is then outlined. Finally, support for repeated exposure to SBT is provided.
2 Training interventions and team performance
Over the past 25 years, Salas and colleagues have undertaken extensive research into team training and the critical factors involved in the creation of high performing teams, examples of which are summarised in Table 1.
Salas et al. (1997) commented that training is not always effective in imparting crucial knowledge and skills, highlighting the lack of integration between training theory and training practice. What is required, therefore, are training strategies or guidelines for turning a team of experts into an expert team that includes aspects such as fostering shared mental models of the task and of other team member roles, training team members on teamwork skills (e.g. situation awareness, communication), and developing simulations that allow team members to experience different courses of action Salas et al. (1997). Team training, according to Salas and Cannon-Bowers (1997), comprises a set of tools and methods that, in combination with required [team-based] competencies and training objectives, forms an instructional strategy. The aim of team training is to improve team effectiveness by addressing both individual competencies and developing team competencies.
Team effectiveness depends on individuals attaining a set of competencies including knowledge, skills, and attitudesFootnote 1 (KSAs) (Salas et al. 2002, cited in Flin et al. 2008). Similarly, team competencies are the knowledge, skills and attitudes required by the team as an entity, i.e. Team KSAs, and are dependent on the type of team and the team’s task, that is whether the team is stable with few change-outs of personnel or not, and whether the team’s task is context-specific or variable (Paris et al. 2000). Training interventions therefore must match the requirements of the team, which are typically identified following a team’s competency (Cannon-Bowers et al. 1995) or task analysis (see Stanton et al. 2013 for further information on team task analysis techniques). A rigorous analysis of a team’s competencies and tasks ascertains the required Team KSAs which guide the instructional strategies and learning objectives of a training intervention designed to improve team effectiveness.
Delivery methods for team training, as described by Salas and Cannon-Bowers (1997) can be information-based (providing knowledge, facts, theories or concepts via lectures, presentations, etc.), demonstration-based (observation of videos or other operators), or practice-based (physical participation and feedback). Practice-based methods form the basis of SBT. However, simulator-based exercises in themselves are not enough to train the expected behaviours (see Salas et al. 1998; Assumption 2). What are also required are training strategies, learning objectives, and feedback from trained and competent observers (Salas et al. 2004). In particular, scenarios to be used for SBTT must incorporate events and/or cues to stimulate the practise and demonstration of the identified KSAs. Training structured around learning objectives, exercise design, performance measurement, and feedback has been shown to lead to improved team performance (Dwyer et al. 1999; Rosen et al. 2008; Salas et al. 2007, 2008). This underpins Principle 1 relating to the design of learning objectives and expected performance standards.
3 Simulator-based training (SBT)
Simulator-based training is defined as an instructional technique that accelerates expertise and skills development by providing active learner engagement, repetitive practice, variable scenario complexity, and performance measurement and feedback (Owen et al. 2006). SBT is characterised by “feedback, repetition, variations in degree of difficulty, use in a controlled environment and defined outcomes for measureable learning” (Issenberg et al. 2005, as cited in Bilotta et al. 2013, p. 4). As an experiential based learning technique, SBT involves active engagement and cognitive experiences based on realistic events (Fanning and Gaba 2007). Participants can apply their learning and experiences in a simulated environment, test out new responses and actions, and can respond adaptively to situational changes.
Simulators can range from low fidelity, such as paper and pen or computer games (Baker et al. 1993) through to high fidelity, virtual reality cockpit simulators (RAEs 2009), dependent on aspects such as cost-effectiveness, space, location, demand, and so forth. However, it is not the case that the higher the fidelity, the higher the opportunity for enhanced learning, rather it is the design of the training programme, based on training strategies as discussed above, that is crucial (Beaubien and Baker 2004). An effective SBT relies upon equipment fidelity, environment fidelity and psychological fidelity, with the latter being the most important for teamwork skills training. That is, that the simulation itself needs to closely match how tasks are performed in the real world; the simulation has to replicate the cognitive processes that would be experienced in the actual environment (Kozlowski and DeShon 2004). The potential of transfer of training from simulator-based exercises to reality is therefore increased.
SBT, in terms of both low to high-fidelity simulators, have been introduced in aviation and healthcare to enhance the skills, both technical and non-technical, of operators. The next section describes the experiences of both these sectors in terms of the benefits gained from the introduction of SBT in terms of improved safety and effectiveness.
3.1 Aviation
Crew resource management is a training intervention designed to support the development of interpersonal aspects of flight operations (Helmreich et al. 1999; Helmreich 2002). At the core of a CRM training programme are CRM skills, also known as non-technical skills. These skills are the cognitive, social and personal resource skills that complement technical skills, and contribute to safe and efficient task performance (Flin et al. 2008). CRM was introduced following the recognition that the majority of aviation accidents were attributable to human error, such as failures in coordination (Salas et al. 2004). Teamwork competencies are essential for aviation teams, particularly as these tend to be ad hoc teams in that the team members vary almost on a flight-by-flight basis.
In order to reduce the frequency of human errors, and to enhance non-technical skills, the aviation industry has typically integrated SBT into CRM training courses. SBT for aviation can range from low-fidelity simulation, such as role play or tabletop computer-based simulators (Baker et al. 1993), through to full-flight simulators (RAeS FSG 2009). Dependent on the task requirements, different levels of simulator fidelity can be used for specific objectives. For example, operations such as practising complex avionics or engine start procedures can be carried out on a low-fidelity simulator, whereas using a full-flight simulator allows aircrew to practise managing severe weather conditions, or even dealing with an engine failure, but in a safe environment. The intensity of SBT assists crew to handle challenging, but infrequent, situations (Bilotta et al. 2013).
The use of SBT is considered to have improved training standards and also contributed to improved aviation safety, according to Wise et al. (2010), through the introduction of CRM. As Bilotta et al. (2013) comment, SBT is now accepted by pilots (and mandated by aviation authorities) as a reliable and trustworthy educational tool (Cook et al. 1998).
Due, in part, to the benefits of SBT and non-technical skills training in improving safety, lessons learned from aviation have been used by other high-hazard industries. Healthcare, in particular, has adopted SBT for a range of settings including emergency medicine (Weaver et al. 2010), resuscitation (Ornato and Peberdy 2014), anaesthesia (Gaba 2010), obstetric crisis team training (Robertson et al. 2009) and interprofessional medical teams (Fung et al. 2015), as discussed below.
3.2 Healthcare
Toff (2010) comments that healthcare can learn much from the evolution of aviation safety, especially anaesthesia. He further suggests that the relevance of human factors in improving safety acknowledged in aviation was instrumental in the introduction of CRM-type training interventions being introduced in anaesthesia (Gaba et al. 2001). CRM training has also been introduced for surgeons leading to improvements in team coordination and reductions in errors (Guerlain et al. 2008). Comparing aviation and healthcare in terms of the use of simulation, Sir Liam Donaldson, Chief Medical Officer of England, cited in his 2009 annual report, ‘That when a person steps on a plane, their risk of dying in an air crash is 1 in 10 million. When a person is admitted into hospital, their risk of dying or being seriously harmed by a medical error is 1 in 300” (DOH 2009, p. 51). Sir Liam comments, amongst a number of key points, that surgeons trained on simulators make fewer errors and carry out technically more exact procedures (2009). Furthermore, simulators as highly realistic training environments are valuable for training and developing clinical teams, especially for providing behavioural debriefing. The report’s recommendations include that “The importance of human factors training to safe care should be widely communicated”.
Notwithstanding the Chief Medical Officer’s statement that simulation in the healthcare system still requires further expansion, the use of simulation has grown in the UK over the past 20 years. A number of general findings have been identified across the range of medical settings where SBT has been introduced relating to the structure and method of training, the importance of feedback and debriefing, facilitation, and the overall benefits of SBT.
It is worth noting that, similar to aviation, SBT in healthcare can be implemented on simulators with low physical fidelity (Weaver et al. 2010), or even on portable simulators (Paige et al. 2008), i.e. a transportable simulator that can be taken to different locations for training purposes. Low-fidelity simulations are often as useful as higher-fidelity simulations, as long as the psychological/cognitive fidelity of the tasks undertaken on the simulator is high (Bowers et al. 1992). Kneebone (2010) comments that, in healthcare, the relationship between clinical and simulator-based practice is a mutually dependent, two-way process, and that the simulation should mirror the essentials of a clinical setting but only needs to be ‘good enough’ to engage participants and achieve learning goals. Paige et al. (2008) found significant improvements in team competencies, such as role clarity, anticipatory response, cross-monitoring, and team cohesion and interaction, based on the use of a portable simulator.
3.3 Maritime and offshore oil and gas
In the maritime setting, specially designed simulators that realistically simulate complex conditions on board vessels have been used to provide experience and improve both technical and non-technical skills (Baldauf et al. 2012). Full-mission simulators are integral to the training of non-technical skills through CRM-type training courses specifically focused on learning attitudes, behaviours and cognition on board ship as practised in simulator-based exercises (Barnett et al. 2006; Hanzu-Pazara et al. 2008). Related to maritime transportation, SBT courses for offshore anchor handling vessels targeting teambuilding, leadership, and communication have shown positive influences for both individuals and groups (Havold et al. 2015).
Currently, in the oil and gas industry, simulators are predominantly used for technical training (Frink et al. 2004—coiled tubing; Veitch et al. 2008a—evacuation training; Veitch et al. 2008b—emergency response training) with little explicit emphasis on non-technical skills. Desktop simulators are used during assessment exercises for technical expertise by individuals for well-controlled accreditation training (IOGP 476). A high-fidelity power distribution simulator has also been designed and is currently being used for Responsible Electrical Person (REP) training (Moffat and Crichton 2015) for drilling rig electricians as well as electricians on production vessels. In this latter case, the focus is on both technical and non-technical skills of individuals resolving complex scenarios.
Non-technical skills training is increasingly being considered by the offshore oil and gas industry to enhance safety and performance (Energy Institute 2014). Guidance on the implementation of Well Operations Crew Resource Management has been published by the International Association of Oil and Gas Producers (IOGP) (WOCRM: IOGP 502) with the aim of providing learning objectives for CRM competencies/non-technical skills as well as guidance on training delivery and assessment. SBT is considered to be preferable, but not essential, for WOCRM training (IOGP 501).
3.4 Drilling
Given the predominance and experiences of simulation and SBT in aviation and healthcare, and the reported benefits in terms of the development of competence in both technical and non-technical skills, the drilling sector can learn much from these industries. The technical advances in building high-fidelity drill floor simulators provides the opportunity for drill teams to practise operations, especially high risk operations such as managing well control incidents, and to enhance the effectiveness of the team to become more flexible and adaptable in terms of their social and cognitive skills.
Drilling simulators can range from desktop well control simulators used by individuals during certification training to high-fidelity full-scale drill floor simulators designed for team training. High-fidelity drilling simulators mimic the layout of a drill floor, typically involving two cyber chairs for the roles of the driller and the assistant driller. The cyber chairs support electronic instrumentation for drilling processes such as setting pumps and rotary speeds, with data being displayed on human–machine interfaces (HMIs) attached to the chairs. Illustrations of virtual reality graphics of rig floor operations can also be presented to the team members. Additional drilling team roles can access the drilling data using separate HMIs displayed throughout the simulator suite. As drilling operation exercises are underway, team members communicate verbally while assessing situations and making decisions. Facilitators can introduce unexpected events or complexities into the scenario during the exercise to test out the individual’s and team’s knowledge and expertise.
Due to the relative stability of the team and the duration of time involved in drilling a well, or series of wells, there is a greater emphasis in drilling teams for teams to receive training and practise as a complete entity. This then leads to a greater requirement for SBTT.
4 Simulator-based team training (SBTT)
Simulator-based team training expands on the concepts of SBT to develop, practise, and enhance team competencies and KSAs. Whereas SBT provides active learner engagement, repetitive practice, and the ability to vary complexity in scenario-based exercises, along with directed feedback (Weaver et al. 2010), SBTT advances teamwork skills, such as team communication, cooperation and coordination, situation awareness, and shared mental models. Hamman (2004) comments that training programmes often focus on individual responsibilities but healthcare requires interdisciplinary teams, and therefore simulator-based interdisciplinary team training which crosses organisational divisions is required to allow communication, accountability, and teamwork, to be effective. SBTT has been used with a multidisciplinary healthcare team comprising surgeons, anaesthetists and nurses (Bilotta et al. 2013) where team members come together to work as a team. Teamwork training undertaken in an immersive simulator environment can expand traditional training leading to improved performance and reduced errors (Lateef 2009).
Reviewing the before, during and after effects of SBTT, Weaver et al. (2010) conclude that when bringing together a group of experts (e.g. clinicians), it is essential that the group knows how to coordinate, shares expertise, and is motivated to work as a team, in order to work safely and effectively. These authors have proposed recommendations to grow SBTT as a viable and efficient method to develop teamwork expertise in the US healthcare system. In this sense, SBTT requires training strategies, content, and methods, utilises either high- or low-fidelity simulation, and includes diagnostic feedback. Reductions in errors and performance improvements, as well as a significantly positive impact on teamwork behaviour in a medical setting, have been reported following SBTT compared to a control group (Morey et al. 2002; Shapiro et al. 2004). Similarly, Robertson et al. (2009) found positive changes in attitudes, perception of individual and team performance, and overall team performance following the introduction of an SBTT programme.
In sum, SBTT has been introduced in a variety of settings where teams must function safety and effectively. Such teams may comprise a number of team members, involve a variety of specialisms, and work in stressful situations, such as aviation and healthcare. Principle 2 is train the team as a whole (simulator-based team training).
5 Evaluating team performance: behavioural marker framework
The use of SBT and SBTT then raises the issue of the evaluation of performance and potential benefits directly linked to safe outcomes. A variety of methods have been developed to be used by observers to assess different aspects of team performance in complex systems (Stanton et al. 2013). These typically focus on technical skills and include Behavioural Observation Scales (BOS: Latham and Wexley 1977) and Behaviorally Anchored Rating Scales (BARS: Bernardin and Smith 1981). In order to address non-technical skills performance, however, behavioural marker frameworks are typically utilised (Flin and Martin 2001). Behavioural markers are defined as observable, non-technical behaviours that contribute to superior or substandard performance within a work environment (Klampfer et al. 2001; Carthey et al. 2003). The UK Civil Aviation Authority propose that it is necessary to assess the CRM [non-technical] skills of flight crew members from time to time (CAA 2006) and behavioural marker frameworks provide the basis for such an assessment. The purpose of an assessment is to give feedback to trainees, test skills in a competence assurance system, ascertain whether a CRM training programme has been effective, and audit the level of skill demonstrated in a work unit (Flin et al. 2008).
Associated initially with aviation CRM training programmes, descriptive behavioural marker frameworks have been developed as a means of defining and evaluating the behaviours relating to the key non-technical skills required by crew members. Examples from aviation and healthcare include Non-technical Skills (NOTECHS: van Avermaete and Kruijsen 1998), Anaesthetists Non-technical Skills (ANTS: Fletcher et al. 2003, 2004), and Non-technical Skills for Surgeons (NOTSS: Yule et al. 2008). It is acknowledged, however, that behavioural markers need to be validated and that raters using the systems must be adequately trained (Jepson et al. 2015).
Specific to team performance, an observational system, Observational Teamwork Assessment for Surgery (OTAS: Undre et al. 2006), has also been designed to provide a measure of teamwork on outcomes in the operating theatre. The OTAS system comprises a procedural task checklist for the patient, equipment and communication tasks, as well as ratings on team behaviour constructs such as communication, cooperation, coordination, shared-leadership and monitoring. Similarly, the University of Texas Behavioral Markers for Neonatal Resuscitation (UTBMNR: Thomas et al. 2004) is an audit form which aims to improve teamwork in healthcare to reduce and manage errors. A Team Behavioural Marker framework has also been developed and tested on drilling team behaviours during SBTT sessions (Crichton and Moffat 2015), with the four categories being Team Decision Making, Team Situation Awareness, Teamwork and Communication, and Team Workload and Stress Management.
Behavioural marker systems are often used both in real-life situations and during SBT sessions. The ultimate aim of the systems is to capture team performance behaviours, which will then be used as a basis for feedback during a debrief to improve performance. Principle 3 is to use a structured observation tool (behavioural marker framework) during simulator-based exercises.
6 Debrief and feedback
One aspect of SBT and SBTT almost universally deemed to be essential to improving performance is that of debriefing and feedback (Fanning and Gaba 2007). Debriefing is the critical phase of learning, as new information is given meaning (Barnett et al. 2006). Typically undertaken following a simulator-based exercise, a debrief offers constructive feedback to enhance learning and skill retention (Barnett et al. 2006; Bilotta et al. 2013; Havold et al. 2015). Ideally, a structured debrief should take place immediately following the termination of the training session (Weaver et al. 2010), and should reinforce lessons learned during the training as well as revisit the learning objectives (Beaubien and Baker 2004). The value of debriefing is highlighted by a finding that a 20–25% improvement in individual and team performance occurred following the introduction of debriefing (Tannenbaum and Cerasoli 2013). These authors propose four essential elements for a debrief including: active (vs. passive) self-learning, developmental (vs. administrative) intent, specific (vs. general) events, and multiple (vs. single) information sources.
Issenberg et al. (2005) propose that feedback is the most important feature of SBT medical education. In a review of SBTT in healthcare, Weaver et al. (2010) comment that trainees reflected on their performance in terms of both the outcome of the session and the process. The combination of both outcome and process feedback from sources such as self, peers, and facilitator assists with acquiring a more valid picture of performance. Reflecting on performance (either self-reflection by the trainee or facilitator-guided) should include what went well, what went wrong, why it went that way, and what can be learned from the experience.
In healthcare, debriefs are considered to be critical for improving patient safety as a part of training teams to successfully manage emergencies (Fanning and Gaba 2007). A debrief should be structured to ensure effective reflection and learning, and Lederman (1992) suggests seven elements for a debrief including the person conducting the debrief, the participants to be debriefed (who may indeed be the same in the case of self-reflected debriefing), an experience (such as a simulation exercise), the impact of the experience, recollection, report, and time. A tool for structured debriefings following SBTT, named TeamGAINS (Kolbe et al. 2013), has been designed and incorporates guided team self-correction, advocacy-inquiry, and systemic-constructivist techniques. Tests of this tool during SBT of clinical and behavioural skills for anaesthesia staff indicated a positive effect on psychological safety and leader inclusiveness following the debriefings.
Given the recognition of the importance of debriefs following SBT sessions to improve performance and behaviours, this raises the issue of the skills of an observer to provide effective feedback. Observers must be able to sensitively and constructively address non-technical skills, which can be more challenging than feeding back observations on technical skills. Based on experiences from aviation in providing non-technical skills training and debriefing, Dismukes and Smith (2000) describe the skills required for facilitation and debriefing. In healthcare, specific debriefing techniques for instructors have been identified (Rudolph et al. 2006). Fanning and Gaba (2007) to highlight the role of the facilitator in the debriefing process in that the facilitator should not act as a ‘teacher’ but rather should guide and direct the critical analysis of performance. Principle 4 is to provide feedback during a structured debrief.
7 Frequency of simulation training
In order to increase skills and knowledge, context-specific training opportunities must be experienced. Training is an ongoing process used to counteract the potential for skill degradation if little-used skills are not practised in situ. The development of expertise is the end result of individuals’ prolonged efforts to improve performance (Ericsson et al. 1993). While it is generally accepted that a training experience should not be a one-time activity (Beaubien and Baker 2004), one area of SBT and SBTT which still appears to remain unclear is that of how frequently training, especially using simulators, should occur (Shapiro et al. 2004). Kneebone (2005) suggests that one of the four key areas that underpin SBT is gaining technical proficiency through repeated practice. Such simulator-based repeated practice provides the opportunity to consolidate recently acquired skills within a defined curriculum. However, little research is available demonstrating the optimal frequency of recurrent training to retain performance levels.
Repetitive practice is a key feature in the use of high-fidelity simulations in medical education (Issenberg et al. 2005), but again there is variability in the regularity of programming of simulator-based exercises. High-hazard industries, such as aviation, nuclear power production, oil and gas industry, address competency assurance of safety critical roles, typically involving simulator-based exercises and observations; however, even in these settings there is little consistency in assessment timescales which can range from 6 months to 3 years (Flin 2005). In the UK, the Civil Aviation Authority (CAA) recommend that recurrent training in CRM should take place over a period not exceeding three years (CAA 2006). More research is required into how frequently simulator-based repeated practice should occur to maintain competence, bearing in mind that this will be affected by the type of operations and safety criticality of the individual’s and team’s tasks. Principle 5 is to repeat the SBT regularly to enhance expertise and retain performance standards.
8 Discussion
This paper reviews SBT and SBTT in order to identify the core principles that could be gleaned from other settings where SBT and SBTT has previously been implemented. Once identified, these principles should then be taken into account when developing training interventions for drilling teams. Aviation and healthcare comprise the two main industries where SBT and SBTT have been increasingly integrated into training interventions particularly to reinforce effective individual and team behaviours, and to improve safety. In both cases, simulation, whether low or high fidelity, is the medium that is used due to the opportunity to rehearse, test out, and reinforce decisions and actions.
In healthcare, there is a significant and growing recognition that SBT is educationally effective not only to develop technical skills but also to maximise safety and minimise risk, especially to patients (Ziv et al. 2003). Research by Riley et al. (2011) found an interdisciplinary team training programme in a medical setting using in situ simulation found significant improvements in non-technical skills leading to a 37% improvement in perinatal safety. Similarly, the impact of SBT and SBTT in aviation has been driven by its contribution to safety, as handling the aircraft during different conditions (e.g. severe weather, system failures, and unexpected situations) can be practised in a low risk environment.
A recognised aspect of planning and preparing for well operations in drilling is that of Drill the Well on Paper (DWOP) where relevant team members, representing the operator, the contractor and service companies, come together to review the drilling plan. Steps in the plan are examined, and methods are discussed to improve communication, performance, efficiency, and safety as well as risks and mitigations. A recent development is that of Drill the Well on the Simulator (DWOS) where more challenging sections of the drilling plan can be tested out on a simulator, and modifications can be made prior to operations commencing.
Analyses of accidents and near-misses during drilling operations have highlighted the impact of non-technical skills on safety and performance. In response, the drilling industry is now endorsing non-technical skills training for drilling teams. At this early stage in the development of non-technical skills training in oil and gas and the introduction of interventions incorporating simulator-based scenarios, such as CRM (IOGP 501; IOGP 502), lessons can be learned from the experiences of other high-hazard industries to the benefit of the individual, the team, and the industry, in terms of safety and performance especially for drilling teams. The five principles presented here provide guidance for designing and developing SBT, particularly for drilling teams, such as CRM/non-technical skills training courses, to enhance safe behaviours and performance.
References
Baker D, Prince C, Shrestha L, Oser R, Salas E (1993) Aviation computer games for crew resource management training. Int J Aviat Psychol 3:143–156
Baldauf M, Schröder-Hinrichsa J, Kataria A, Benedict K, Tuschling G (2012) Simulation-based team training for maritime safety and security. J Marit Res 9:3–10
Barnett M, Gatfield D, Pekcan C (2006) Non-technical skills: the vital ingredient in world maritime technology? In: Proceedings of the international conference on world maritime technology, London
Beaubien JM, Baker DP (2004) The use of simulation for training teamwork skills in health care: How low can you go? Qual Saf Health Care 13:i51–i56. doi:10.1136/qshc.2004.009845
Bernardin HJ, Smith PC (1981) A clarification of some issues regarding the development and use of behaviorally anchored ratings scales (BARS). J Appl Psychol 66:458–463. doi:10.1037/0021-9010.66.4.458
Bilotta FF, Werner SM, Bergese SD, Rosa G (2013) Impact and implementation of simulation-based training for safety. Sci World J. doi:10.1155/2013/652956
Bowers C, Salas E, Prince C, Brannick M (1992) Games teams play: a method for investigating team co-ordination and performance. Behav Res Methods Instrum Comput 24:503–506. doi:10.3758/BF03203594
CAA (2006) Crew Resource Management (CRM) training report. CAP737: guidance for flight crew. Civil Aviation Authority, London
Cannon-Bowers JA, Tannenbaum SI, Salas E, Volpe CE (1995) Defining competencies and establishing team training requirements. In: Guzzo R, Salas E (eds) Team effectiveness and decision making in organisations. Jossey-Bass, San Francisco, pp 333–380
Carthey J, de Leval MR, Wright DJ, Farewell VT, Reason JT (2003) Behavioural markers of surgical excellence. Saf Sci 41:409–425. doi:10.1016/S0925-7535(01)00076-5
Chief Counsel’s Report (2011) Macondo: the Gulf oil disaster: National Commission on the BP Deepwater Horizon Oil Spill and Offshore Drilling. purl.fdlp.gov/GPO/gpo4390. Accessed 1 Nov 2016
Cook RI, Woods DD, Miller C (1998) A tale of two stories: contrasting views of patient safety, report from a workshop on assembling the scientific basis for progress on patient safety, National Health Care Safety Council of the National Patient Safety Foundation at the AMA. http://c.ymcdn.com/sites/www.npsf.org/resource/collection/ABAB3CA8-4E0A-41C5-A480-6DE8B793536C/A-Tale-of-Two-Stories(1).pdf. Accessed 1 Nov 2016
Crichton M, Moffat S (2015) Developing behavioural markers for teams: A drilling team case study. Paper presented at the International Association of Drilling Contractors (IADC) human factors conference, Galveston
Dismukes K, Smith G (2000) Facilitation and debriefing in aviation training and operations. Ashgate, Aldershot
DOH (Department of Health) (2009) Annual report of the chief medical officer on the state of public health. Department of Health Publications, London
Dwyer D, Oser R, Salas E, Fowlkes J (1999) Performance measurement in distributed environments: initial results and implications for training. Mil Psychol 11:189–215. doi:10.1207/s15327876mp1102_4
Energy Institute (2014) Guidance on crew resource management (CRM) and non-technical skills training programmes, 1st edn. Report prepared by Energy Institute, London
Entin E, Serfaty D (1999) Adaptive team coordination. Hum Factors 41:312–325. doi:10.1518/001872099779591196
Ericsson KA, Krampe RT, Tesch-Roemer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100:363–406. doi:10.1037/0033-295X.100.3.363
Fanning RM, Gaba DM (2007) The role of debriefing in simulation-based learning. Simul Healthc 2:115–125. doi:10.1097/SIH.0b013e3180315539
Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R (2003) Anaesthetists’ non-technical skills (ANTS): evaluation of a behavioural marker system. Brit J Anaesth 90:580–588. doi:10.1093/bja/aeg112
Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R (2004) Rating non-technical skills: developing a behavioural marker system for use in anaesthesia. Cogn Technol Work 6:165. doi:10.1007/s10111-004-0158-y
Flin R (2005) Safe in their hands? Licensing and competence assurance for safety-critical roles in high risk industries. Report for the Department of Health, University of Aberdeen, Aberdeen
Flin R, Martin L (2001) Behavioral markers for crew resource management: a review of current practice. Int J Aviat Psychol 11:95–118. doi:10.1207/S15327108IJAP1101_6
Flin R, O’Connor P, Crichton M (2008) Safety at the sharp end: a guide to non-technical skills. Ashgate, Aldershot
Frink PJ, Leslie C, Wooten JE (2004) Transient simulator-based training of rig-site decision makers and rig crews for through-tubing coiled tubing underbalanced multilateral project. Paper presented at SPE/ICoTA coiled tubing conference and exhibition, Houston, 23–24 March
Fung L, Boet S, Bould MD, Qosa H, Perrier L, Tricco A, Tavares W, Reeves S (2015) Impact of crisis resource management simulation-based training for interprofessional and interdisciplinary teams: a systematic review. J Interprof Care 29:433–444. doi:10.3109/13561820.2015.1017555
Gaba DM (2010) Crisis resource management and teamwork training in anaesthesia. Brit J Anaesth 105:3–6. doi:10.1093/bja/aeq139
Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb BE (2001) Simulation-based training in anesthesia crisis resource management (ACRM): a decade of experience. Simul Gaming 32:175–193. doi:10.1177/104687810103200206
Guerlain S, Turrentine FE, Bauer DT, Calland F, Adams R (2008) Crew resource management training for surgeons: feasibility and impact. Cogn Technol Work 10:255. doi:10.1007/s10111-007-0091-y
Haavik TK (2011) Chasing shared understanding in drilling operations. Cogn Technol Work 13:281. doi:10.1007/s10111-010-0166-z
Hamman WR (2004) The complexity of team training: what we have learned from aviation and its applications to medicine. Qual Saf Health Care 13:72–79. doi:10.1136/qshc.2004.009910
Hanzu-Pazara R, Barsan E, Arsenie P, Chiotoroiu L, Raicu G (2008) Reducing of maritime accidents caused by human factors using simulators in training process. J Marit Res 5:3–18
Havold JI, Nistad S, Skir A, Odegard A (2015) The human factor and simulator training for offshore anchor handling operators. Saf Sci 75:136–145. doi:10.1016/j.ssci.2015.02.001
Helmreich RL (2002) Threat and error management: 6th generation CRM training. In: Proceedings of the first TEM workshop (ICAO), San Salvador, El Salvador, April 30, pp 1–14
Helmreich RL, Foushee JC (1993) Why crew resource management? Empirical and theoretical bases of human factors training in aviation. In: Wiener EL, Kanki BG, Helmreich RL (eds) Cockpit resource management. Academic Press, San Diego, pp 3–45
Helmreich RL, Merritt AC, Wilhelm JA (1999) The evolution of crew resource management training in commercial aviation. Int J Aviat Psychol 9:19–32. doi:10.1207/s15327108ijap0901_2
IOGP (2014a) Crew resource management for well operations teams: Report No 501. International Association of Oil and Gas Producers, London
IOGP (2014b) Guidelines for implementing Well Operations Crew Resource Management training: Report No 502. International Association of Oil and Gas Producers, London
IOGP (2016) Recommendations for enhancements to well control training, examination and certification. Report no 476 (2nd Rev), International Association of Oil and Gas Producers, London
Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL, Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 27:10–28. doi:10.1080/01421590500046924
Jepson RMHG, Ostergaard D, Dieckmann P (2015) Development of instruments for assessment of individuals’ and teams’ non-technical skills in healthcare: a critical review. Cogn Technol Work 17:63. doi:10.1007/s10111-014-0306-y
Klampfer B, Flin R, Helmreich R, Husler R, Sexton B, Fletcher G, Field P, Staender S, Lauche K, Dieckmann P, Amacher A (2001) Enhancing performance in high risk environments: recommendations for the use of behavioural markers. Report from the behavioural markers workshop, Zurich, 5–6 July. Daimler Benz Foundation, Berlin. Retrieved from http://hub.easa.europa.eu/crt/docs/viewcrdattachment/cid_16027/aid_237/fmd_3902d6107b8462a366188c5dc8a59fb1
Kneebone R (2005) Evaluating clinical simulations for learning procedural skills: a theory-based approach. Acad Med 80:549–553
Kneebone R (2010) Simulation, safety and surgery. Qual Saf Health Care 19:47–52. doi:10.1136/qshc.2010.042424
Kolbe M, Weiss M, Grote G, Knauth A, Dambach M, Spahn DR, Grande B (2013) TeamGAINS: a tool for structured debriefings for simulation-based team trainings. BMJ Qual Saf 22:541–553. doi:10.1136/bmjqs-2012-000917
Kozlowski SWJ, DeShon RP (2004) A psychological fidelity approach to simulation-based training: theory, research, and principles. In: Salas E, Elliott LR, Schflett SG, Coovert MD (eds) Scaled worlds: development, validation, and applications. Ashgate, Burlington, pp 75–99
Krathwohl DR, Bloom BS, Masia BB (1964) Taxonomy of educational objectives: the classification of educational goals. Handbook II: the affective domain. David McKay Company, New York
Lateef F (2009) Simulation-based learning: just like the real thing. J Emerg Trauma Shock 3:348–352
Latham GP, Wexley KN (1977) Behavioral observation scales for performance appraisal purposes. Pers Psychol 30:255–268
Lauche K, Crichton M, Bayerl PS (2009) Tactical decision games: Developing scenario-based training for decision-making in distributed teams. In: Proceedings of the 9th conference on naturalistic decision making, London
Lederman NG (1992) Students’ and teachers’ conceptions of the nature of science: a review of the research. J Res Sci Teach 29:331–359. doi:10.1002/tea.3660290404
Moffat S, Crichton MT (2015) Investigating non-technical skills through team behavioral markers in oil and gas simulation-based exercises. In: Proceedings of the 6th international conference on Applied Human Factors and Ergonomics (AHFE 2015), Las Vegas
Morey JC, Simon R, Jay GD, Wears RL, Salisbury M, Dukes KA, Berns SD (2002) Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res 37:1553–1581. doi:10.1111/1475-6773.01104
Ornato JP, Peberdy MA (2014) Applying lessons from commercial aviation safety and operations to resuscitation. Resuscitation 85:173–176. doi:10.1016/j.resuscitation.2013.10.029
Owen H, Mugford B, Follows V, Plummer JL (2006) Comparison of three simulation-based training methods for management of medical emergencies. Resuscitation 71:204–211. doi:10.1016/j.resuscitation.2006.04.007
Paige JT, Kozmenko V, Tang Y, Gururaja RP, Hilton CW, Cohn I Jr, Chauvin SW (2008) High-fidelity, simulation-based, interdisciplinary operating room team training at the point of care. Surgery. doi:10.1016/j.surg.2008.09.010
Paris CR, Salas E, Cannon-Bowers JA (2000) Teamwork in multi-person systems: a review and analysis. Ergonomics 43:1052–1075. doi:10.1080/00140130050084879
RAES (Royal Aeronautical Society Flight Simulation Group) (2009) The impact of flight simulation in aerospace. FSG (Flight Simulation Group) discussion paper. London. Retrieved from http://aerosociety.com/Assets/Docs/Publications/DiscussionPapers/The_impact_of_flight_simulation_in_aerospace.pdf
Riley W, Davis S, Miller K, Hansen H, Sainfort F, Sweet R (2011) Didactic and simulation nontechnical skills team training to improve perinatal patient outcomes in a community hospital. Jt Comm J Qual Patient Saf 37:357–364
Robertson B, Schumacher L, Gosman G, Kanfer R, Kelley M, DeVita M (2009) Simulation-based crisis team training for multidisciplinary obstetric providers. Simul Healthc 4:77–83. doi:10.1097/SIH.0b013e31819171cd
Rosen MA, Salas E, Wu TS, Silvestri S, Lazzara EH, Lyons R, Weaver SJ, King HB (2008) Promoting teamwork: an event-based approach to simulation-based teamwork training for emergency medicine residents. Acad Emerg Med 15:1190–1198. doi:10.1111/j.1553-2712.2008.00180.x
Rudolph J, Simon R, Dufresne R, Raemer D (2006) There’s no such thing as ‘nonjudgmental’ debriefing: a theory and method for good judgment. Simul Healthc 1:49–55
Salas E, Cannon-Bowers JA (1997) The anatomy of team training. In: Tobias L, Fletcher D (eds) Handbook on research in training. MacMillan, New York
Salas E, Dickinson T, Converse S, Tannenbaum S (1992) Toward an understanding of team performance and training. In: Swezey R, Salas E (eds) Teams: their training and performance. Ablex, New York
Salas E, Cannon-Bowers JA, Blickensderfer EL (1997a) Enhancing reciprocity between training theory and practice: principles, guidelines, and specifications. In: Ford JKVE, Kozlowski SWJ, Kraiger K, Salas E, Teachout MS (eds) Improving training effectiveness in work organizations. Psychology Press, New York
Salas E, Cannon-Bowers J, Johnston JH (1997b) How can you turn a team of experts into an expert team: emerging training strategies. In: Zsambok C, Klein G (eds) Naturalistic Decision Making. LEA, New York
Salas E, Bowers CA, Rhodenizer L (1998) It is not how much you have but how you use it: toward a rational use of simulation to support aviation training. Int J Aviat Psychol 8:197–208. doi:10.1207/s15327108ijap0803_2
Salas E, Cannon-Bowers J, Weaver J (2002) Command and control teams: principles for training and assessment. In: Flin R, Arbuthnot K (eds) Incident command: tales from the hot seat. Ashgate, Aldershot
Salas E, Burke CS, Stagl KC (2004) Developing teams and team leaders: strategies and principles. In: Demaree RG, Zaccaro SJ, Halpin SM (eds) Leader development for transforming organizations. Lawrence Erlbaum Associates, Mahwah, pp 325–358
Salas E, Rosen MA, Burke CS, Goodwin GF, Fiore S (2006) The making of a dream team: when expert teams do best. In: Ericsson KA, Charness N, Feltovich PJ, Hoffman RR (eds) The Cambridge handbook of expertise and expert performance. Cambridge University Press, New York, pp 439–453
Salas E, Nichols DR, Driskell JE (2007) Testing three team training strategies in intact teams: a meta-analysis. Small Group Res 38:471–488. doi:10.1177/1046496407304332
Salas E, DiazGranados D, Klein C, Burke CS, Stagl KC, Goodwin GF, Halpin SM (2008) Does team training improve team performance? A meta-analysis. Hum Factors J Hum Factors Ergon Soc 50:903–933. doi:10.1518/001872008X375009
Shapiro MJ, Morey JC, Small SD, Langford V, Kaylor CJ, Jagminas L, Suner S, Salisbury ML, Simon R, Jay GD (2004) Simulation based teamwork training for emergency department staff: Does it improve clinical team performance when added to an existing didactic teamwork curriculum? Qual Saf Health Care 13:417–421. doi:10.1136/qshc.2003.005447
Stanton NA, Salmon PM, Rafferty LA, Walker GH, Baber C, Jenkins DP (2013) Human factors methods: a practical guide for engineering and design, 2nd edn. Ashgate, Aldershot
Tannenbaum SI, Cerasoli CP (2013) Do team and individual debriefs enhance performance? A meta-analysis. Hum Factors 55:231–245. doi:10.1177/0018720812448394
Thomas EJ, Sexton JB, Helmreich RL (2004) Translating teamwork behaviours from aviation to healthcare: development of behavioural markers for neonatal resuscitation. Qual Saf Health Care 13:i57–i64. doi:10.1136/qshc.2004.009811
Toff NJ (2010) Human factors in anaesthesia: lessons from aviation. Brit J Anaesth 105:21–25. doi:10.1093/bja/aeq127
Undre W, Sevdalis N, Healey AN, Darzi A, Vincent CA (2006) Teamwork in the operating theatre: Cohesion or confusion? J Eval Clin Pract 12:182–189. doi:10.1111/j.1365-2753.2006.00614.x
van Avermaete JAG (1998) NOTECHS: non-technical skill evaluation in JAR-FCL. EC NOTECHS project final report: NLR-TP-98518, Amsterdam
van Avermaete JAG, Kruijsen EAC (1998) NOTECHS: The evaluation of non-technical skills of multi-pilot aircrew in relation to the JAR-FCL requirements. EC NOTECHS project final report: CR 98443, Amsterdam
Veitch B, Billard R, Patterson A (2008a) Evacuation training using immersive simulators. Paper presented at the eighteenth international offshore and polar engineering conference, Vancouver, Canada, 6–11 July. Retrieved from http://www.isope.org/publications/proceedings/ISOPE/ISOPE%202008/papers/I08SJH-01Veitc.pdf#page=1
Veitch B, Billard R, Patterson A (2008b) Emergency response training using simulators. Paper presented at offshore technology conference, Houston, 5–8 May
Waller MJ, Gupta N, Giambatista RC (2004) Effects of adaptive behaviors and shared mental models on control crew performance. Manag Sci 50:1534–1544. doi:10.1287/mnsc.1040.0210
Weaver SJ, Salas E, Lyons R, Lazzara EH, Rosen MA, DiazGranados D, Grim JG, Augenstein JS, Birnbach DJ, King H (2010) Simulation-based team training at the sharp end: a qualitative study of simulation-based team training design, implementation, and evaluation in healthcare. J Emerg Trauma Shock 3:369–377. doi:10.4103/0974-2700.70754
Wise JA, Hopkin VD, Garland DJ (2010) Handbook of aviation human factors. CRC Press LLC, Boca Raton
Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S (2008) Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 32:548–556. doi:10.1007/s00268-007-9320-z
Ziv A, Wolpe PR, Small SD, Glick S (2003) Simulation-based medical education: an ethical imperative. Acad Med 78:783–788. doi:10.1097/01.SIH.0000242724.08501.63
Acknowledgements
These principles for training have been developed in response to requests by simulator-based exercise facilitators for clearer guidance to improve training directed towards enhancing team effectiveness and safety. The authors would like to acknowledge the cooperation and support provided by drilling team members as well as simulator-based exercise facilitators during the development of this project to design the principles for training. The authors would particularly like to thank Lauren Crichton for her helpful comments and suggestions about the draft manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Crichton, M.T. From cockpit to operating theatre to drilling rig floor: five principles for improving safety using simulator-based exercises to enhance team cognition. Cogn Tech Work 19, 73–84 (2017). https://doi.org/10.1007/s10111-016-0396-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10111-016-0396-9