1 Introduction

The research field of computer-supported cooperative work (CSCW) has explicated performance in the workplace as participation in collaborative activities and the coordination of sociotechnical systems. Suitably, this paper will discuss strategies for utilising simulators in professional training, with a particular focus on coordinating and collaborative activities. In the following, we will sketch a framework for matching learning objectives with simulation requirements and assessment in simulator training designs. By suggesting three different foci for facilitating work-relevant training activities in simulators, we develop a framework that may function as a conceptual lens for considering and preparing training trajectories.

Simulators have a long history of facilitating learning activities for professionals who work in settings involving high costs and safety risks (Vincenzi et al. 2009). Simulators are used for developing professional expertise within domains, such as healthcare (Gaba and DeAnda 1988; Grantcharov et al. 2004; Issenberg et al. 1999), maritime navigation (Sellberg 2017; Øvergård et al. 2017; Rønningen and Øvergård 2017), aviation (Hutchins and Klausen 1996; Roth and Jornet 2015) and military operations (Oskarsson et al. 2010; Øvergård et al. 2005). Although simulations have emerged as a common training strategy, research on the relationship between simulation technology and learning opportunities remains ambiguous. As we will further discuss, the productive use of simulators is often ascribed to different kinds of learning activities for different objectives, and it often reveals different concepts of knowing. For coordinating and explicating this research potential for simulator training design, this article points out some key distinctions in prior research and advances three different research-based approaches to simulator training.

Many expectations for simulator training involve concerns over whether simulated experiences equal ‘the real thing’. Fidelity is a key concept describing a simulator’s resemblance to the actual work setting, and it is essential for configuring simulator training designs (Liu et al. 2009a; Dahlstrom et al. 2009). However, this terminology is often used ambiguously. In this article, a simulator is considered a ‘device that duplicates the essential features of a task situation and provides for direct human operation’ (Vincenzi et al. 2009, p. 426). Furthermore, a simulation is considered the representation of professional practice which emerges in the interaction between participants, the simulator and the context (Rystedt and Sjöblom 2012). When reviewing prior studies of simulator fidelity, we found it to be a key distinction whether the notion of fidelity was used to conceptualise the functions and level of immersiveness of the physical/technical environment or the accuracy of the interactionally constituted simulation. Scholars representing different theoretical backgrounds have emphasised different aspects of fidelity in the simulation. Moreover, the concept has epistemological underpinnings that blur its usefulness for designing and conducting training at a practical level.

Aiming to aid in the construction of simulator training designs, this paper presents a framework for configuring simulator fidelity with training objectives and assessment by advancing the following three central approaches:

  1. (1)

    One key approach emphasises physical and functional accuracy in simulation-based training, conceptualised as technical fidelity.

  2. (2)

    Another vein of research underscores the significance of environments that invoke a high level of cognitive and meta-cognitive activity, understood as the simulation’s activation of relevant problem-solving strategies, mental models and feelings, conceptualised as psychological fidelity.

  3. (3)

    Lastly, we emphasise a research vein that pursues simulations as a socio-technical environment for recreating precise coordination and collaborative patterns within a team, which we named interactional fidelity.

In the following chapters, we review the concept of fidelity and position the current contribution in relation to prior research before we elaborate on the methodology and background of the case study from a ship simulator training. In the subsequent parts of the article, we outline our tripartite conceptual model with empirical demonstrations from this case study. Finally, we discuss how this framework provides suggestions for simulator training design.

2 The role of fidelity in simulation-based training

In this chapter, we aim to unpack the notion of fidelity in simulation-based training and provide a foundation for a new framework that draws connections between fidelity and learning opportunities. A range of research shares the general assumption that simulator fidelity is related to learning opportunities. However, these assumptions rest on fairly different theoretical models of learning (Hontvedt 2014). In the early years of simulator training, a strong link was formed between a simulator’s general level of fidelity and the amount of learning (Liu et al. 2009a). Although such one-dimensional relationships between simulator fidelity and learning have been heavily criticised for supposing a simplistic view of learning as plain transmission, they still appear frequently (e.g. discussions in Liu et al. 2009b; Beaubien and Baker 2004; Dahlstrom et al. 2009). Later, there have been many studies from the perspective of organisational psychology, cognitive psychology and human factors research (e.g. Kozlowski and DeShon 2004; Øvergård et al. 2015), as well as a growing body of research from situated and sociocultural perspectives on learning (e.g. Hontvedt and Arnseth 2013; Øvergård et al. 2010; Sellberg 2018; Rystedt 2002).

Across these positions, the general conception of fidelity is often considered insufficient because a simulation often has high fidelity in some aspects and low fidelity in others. To compensate, specifying different types of fidelity has been a common strategy. Several conceptual frameworks for describing simulators already exist. For example, Rehmann (1995) reviewed the types of flight simulator fidelity and reported and organised more than 20 conceptualisations, such as equipment fidelity, environmental fidelity, psychological fidelity, task fidelity, physical fidelity and functional fidelity. However, Rehmann (1995) overview was primarily made to describe simulator affordances for flight deck human factors research. This is important because we find that the notion of fidelity differs between those who either use simulators as test beds for how different types of equipment affect task performance (Gould et al. 2009; Murai et al. 2009), for measuring simulator effectiveness, for example in terms of ‘action fidelity’—meaning similarity between performance in the simulator and the simulated system (Stoffregen et al. 2003), or for those who are concerned with the construction of simulations as learning environments, such as the current study. Because a learning environment may profit from other features than those of the actual work setting, such as room for providing instruction or metareflection, general terms such as ‘realism’ or ‘authenticity’, may not always be the most relevant measure for assessing simulators’ potential as tools for learning (Johnson 2008; Rystedt and Sjöblom 2012; Taylor 2011). For this purpose, we argue that the notion of fidelity could provide more detailed accounts.

We find the various sub-types of fidelity conveying a simulator’s physical characteristics useful for describing simulator technology. However, what fidelity is and where and how it is constituted are somewhat unclear. For example, physical fidelity may relate to the material characteristics of a simulator, whereas psychological fidelity often describes an active experience. Studies also relate fidelity to learning opportunities without connecting the technology to activity. For example, Hays and Singer (1989) pointed out that fidelity conceptualises how similar the simulation needs to be to the operational situation in order to maintain training efficiency, but that the notion of fidelity “should be restricted to descriptions of the required configuration of the training situation and not to be used when discussing behaviours” (p. 47); they also argued for the development of strict typologies of simulator fidelity that orient fidelity towards the simulator setting.

In contrast, the current study questions the benefit of separating the physical/technical simulator setting from the collaborative training activity when considering fidelity. Furthermore, because of this, the current framework does not take up the full range of prior research on simulator fidelity. This does not mean that we reject other conceptualisations but that we want to focus and refine some aspects particularly relevant to simulator training design.

Whether the concept of fidelity describes exactness and similarity between the physical and technical environments or the exactness and work relevance of the simulated activity is often unclear in the literature, so we propose a conceptual distinction: we distinguished between a simulator, which we consider to be a set of devices or systems that reproduce certain aspects of a real environment, and a simulation or simulating, which involves the active process of constituting a socio-technical environment for training. Our position is that fidelity can describe the technology whilst also addressing the precision and accuracy of the simulation as an activity. Nonetheless, we claim that if one seeks to develop the relationship between fidelity and learning, addressing the simulation as an activity is most pertinent.

This position is grounded in prior studies that demonstrate how learning outcomes from the use of technology should not be considered as programmed or inscribed in a technological environment itself; rather, they should be seen as situated and social achievements (Arnseth 2004; Dahlstrom et al. 2009; Dillenbourg et al. 2009; Heath and Luff 1996; Petraglia 1998; Silseth 2012). Considering learning as an interactionally constituted phenomenon, we argue that fidelity should not be conceived of as an isolated factor with a stable relationship with learning efficacy; rather, it should be viewed as a sense of precision in training that is constituted in an activity. As will be detailed in the further parts of this article, such a take on fidelity has consequences for conceptions of simulator training design.

3 Methodology

This article develops a novel conceptual framework by conducting a selected review of three distinct approaches to fidelity in simulation-based training and demonstrates the relevance of this framework in a study of professional maritime pilots who attended training at a Norwegian educational facility. The current chapter will outline this observational study and provide some background information on the use of simulators in maritime training.

Maritime training in Norway follows international standards for competency-based education, training and assessment, standards that allow professionals to demonstrate competence using simulators (International Maritime Organisation 2011). The particular data used in this paper comes from video observations of professional maritime pilots who used simulators to train on cruise ship manoeuvring in and out of the port of Oslo. Within the field of shipping, maritime pilots play a crucial role as local guides with extensive knowledge of the waters in which they are certified (International Maritime Organisation 1968). Maritime pilots are experienced seamen that have extensive knowledge of and experience with local waterways. They support vessels in entering and leaving these waterways to ensure safety and efficiency during the voyage through the piloting area (e.g. the Oslo fjord); it is mandatory for ships to have a qualified pilot on board while entering and leaving ports or other areas that require specialised local knowledge.

The observed training was conducted in a full mission ship simulator, which consists of a replica of a ship’s bridge placed in a cinema-like room with a 240-degree visual display (see Figure 1). Simulator sessions typically consisted of a short briefing before simulator sailing and debriefing. Briefings and debriefings were facilitated in a room dedicated to that purpose, which had a smart board that allowed replaying the movements of the exercise on an electronic map. The facility consists of five full mission simulators with differing degrees of immersion, which can be linked for participating in cooperative scenarios. The simulator allows for training in many different tasks and scenarios, such as anchoring, overtaking and teamwork. It also provides scenarios of critical situations involving equipment failure or loss of rudder control or propulsion, or – as in the case of this study – steering cruise ships in confined waters using Azipod propellers in high winds.

Figure 1
figure 1

The full mission ship simulator used in this study.

Full size image

Relevantly, prior studies in this specific simulator environment has shown how participants’ role-playing became an important part of creating the simulation of bridge management on a cruise ship experiencing engine failure (Hontvedt and Arnseth 2013). Another study of this specific simulator environment sheds light on the technical requirements for simulating ship navigation, and reveals the risk in simulator training of entering a mode of manipulating the simulator, rather than enacting professional work tasks (Hontvedt 2015). This suggests that the technological aspects of the simulator are fundamentally connected to instructional design. Generally, this simulator environment provides an immersive environment that is used to facilitate many different simulator training designs for both professionals and university students — but does not represent “best practice”. Rather, we consider that the simulation validity not as a static feature but as realised in practice (Johnson 2008). In such regard, this simulator environment represents a relevant case that illustrates common challenges and possibilities for creating simulator training designs.

3.1 Participants and data

For the course described in this article, 12 maritime pilots from several pilot stations participated over two training days. Maritime pilots usually achieve the rank of captain before acquiring further training and certification to serve as local guides for a specific area. To maintain expert competence, they regularly attend courses and training sessions beyond their initial qualification — such as the training sessions observed. The training objectives are set to meet the specific needs of the pilotage services, and relate to the overarching Convention on Standards of Training, Certification and Watchkeeping for Seafarers (STCW) competence requirements.

Each of the two training days consisted of a joint briefing before the simulated scenarios and a debriefing. Throughout the day, the instructors adjusted the scenario to include more wind or other external factors. The two training days in this study included approximately ten hours of training drawn from a larger set of videos used for both nautical students and professional maritime pilots training at the same simulator facility. The larger component of the material involved approximately 45 hours of training obtained during 11 training sessions over a two-year period (2010 to 2012). The video observations were conducted in a naturalistic setting, meaning that the training practices were observed without intentional interventions on the researcher’s part.

3.2 Data analysis

Interaction analysis was applied to investigate the pilots’ use of simulators. Interaction analysis is an empirical and video-based method used to study social interaction as it evolves through talk, non-verbal interactions and the use of artefacts and technologies among members of a community of practice (Derry et al. 2010; Jordan and Henderson 1995). In particular, interaction analysis places the sequential unfolding of interactional activities in centre of attention. In this study, interaction analysis allowed a focus on the situated constitution of the simulations and the coordinating and communicative practices of the bridge team. In the initial analysis it stood out how the simulator training afforded several different foci for the learning activities — which we considered potentially interesting to analyse and conceptualise. In the following we pursue these analytical interests as we present several vignettes of interaction, which were transcribed and analysed in detail.

3.3 Limitations

Due to the limited number of participants, this study’s estimation of how future training will develop should be regarded with caution. However, to a certain extent, analyses of specific situations and smaller corpuses of data can be considered generalisable because they indicate a range of possible activities within a specific sphere (Ercikan and Roth 2006, p. 15). Interaction analysis holds that artefacts and technologies set up a social field, such as simulated ship travel. Within this field, certain activities become very likely, some possible, and others improbable. Accordingly, this article presents possible training activities and ways of constructing work relevance in training in order to demonstrate the usefulness of a more precise notion of fidelity in creating simulator trainings.

4 A framework for simulator training design

In this chapter, we will explicate the framework for creating simulator training designs. We establish a conceptual framework that coordinates some important veins of prior research, and suggest a new category of fidelity related to situated perspectives on CSCW. This framework, presented in Table 1, highlights some important relationships between different types of learning objectives and the need for fidelity and assessment. Unlike prior research on fidelity, this framework connects fidelity requirements to specific forms of learning activities.

Table 1 Framework for simulator training design.
Full size table

Creating simulator training designs involves consideration of a range of factors, much more than those treated in the current framework. Examples are constructing relevant training scenarios that match the learners’ professional culture and considering the trainers´ role. Furthermore, the concrete learning objectives will vary between professional domains and involve types of expertise that require very specific organisation and support. Nonetheless, the benefit of this framework is that it links learning objectives to requirements for different types of fidelity and assessments, and it opens up for a better conception of the complexity in training. This may aid in the construction and orchestration of a training design and entail support for practitioners seeking to formulate a research-based practice. As we will explicate in the following chapters, the framework coordinates different theoretical approaches to learning. In the three veins of research on fidelity and learning, one may recognise concepts from broad theoretical domains with insights into learning and training in simulators, such as skills training (Flin et al. 2008), cognitive/constructivist theories of learning (Anderson 2009; Papert 1980) and situated perspectives on communication (Engeström and Middleton 1998; Goodwin 1995) and learning (Greeno 2006). Such a coordination of findings generated from different theoretical standpoints is not without controversy because they often operate with different metaphors and models for what learning is. However, we will demonstrate that the current framework is useful for the practical management of training by highlighting its potential implications for creating a simulator training design.

4.1 Technical fidelity

Professional expertise involves a range of skills and motor abilities that require training with high technical fidelity. In the framework proposed in this article, technical fidelity describes the degree of accuracy to which the technical and environmental cues are recreated by the simulator technology. Accuracy involves how much the simulation looks, sounds and feels like the operational environment in terms of the photorealism of visual displays, the visual and haptic characteristics of input devices and controllers as well as the sound in the simulator (Alexander et al. 2005; Liu et al. 2009a). Similar concepts include equipment fidelity (Rehmann 1995), and physical fidelity (Miller 1954).

Technical fidelity is crucial in many training designs because it facilitates training and provides technological support for the training activities. Typically, such training is focused at technical skills that are needed to manoeuvre a vessel, handle surgical equipment or operate other technical systems. Technical skills are often characterised in opposition to non-technical skills, which involve communication, teamwork and leadership (Flin et al. 2008). While technical skills and motor abilities are often integrated with complex social practices in situ (Goodwin 1994, 1995), it is common practice to train technical skills separately. In a range of professional domains, handling technological tools and quick responses to a changing environment are crucial skills that must be obtained through effective training. Skill learning is commonly achieved through repeated exposure to a certain task, and simulators have proven promising as tools for such training (Rose et al. 2000). From a technical skills training perspective, higher technical fidelity ensures less skill degradation and a higher probability for robust learning (Gupta and Cohen 2002).

For example, a recent empirical study illustrating the requirements for high technical fidelity examined the conditions for surviving a helicopter crash at sea (Taber 2013). A full-scale helicopter simulator was used to test conditions that enabled or hindered effective egress from a helicopter that was upside down and fully submerged in water. The findings showed that a specific skill practiced in a simulator may not be effective in a real helicopter if, for example, the actual helicopter windows or the seat harness release mechanism differs from those used in the training environment. The study pointed out that the design of the seats affected the passengers’ ability to egress, concluding with a recommendation for highly-detailed modelling and instruction during briefings as well as varied egress scenarios in training. In other words, the technical fidelity of the simulator needed to be very close to real-world conditions to ensure effective training.

In a range of simulator training designs, a certain degree of technical fidelity is required to complete specific tasks. Meanwhile, high technical fidelity may ensure a consistent and realistic feel and response to the tasks carried out. However, the level of technical fidelity needs to be closely attuned to the learning objective (Dahlstrom et al. 2009). Simulator skills training may function well through repetition priming, but non-technical factors, such as communication, leadership and teamwork, do not necessarily correlate with developing technical skills. Rather, they are complementary skills that also need to be offered during training to ensure safety and efficiency in the performance of work tasks (Flin et al. 2008).

Technical fidelity seems to be a worthwhile conception of fidelity in simulator representation. However, it is not the connection between the concept and characteristics of the simulator that is important; rather, it is the suitability of the simulated training scenario (including the simulator) with respect to the training objectives that matters. Some scenarios might require an immersive environment that provides a lifelike replication of surroundings and system movements (which would be necessary, e.g., during coastal ship navigation). However, for other training objectives, it might be advantageous to focus on a specific skill isolated from the complexities of the full-scale scenario (e.g. handling a needle in surgical training, isolated from the complexity of an operating room; Grantcharov et al. 2004).

4.1.1 Empirical example of technical fidelity in training

The following analytical example is the first of three cases from a study of professional maritime pilots. The training objective for the exercise was threefold: (1) to gain confidence in handling the increasingly common 360-degree manoeuvrable Azipod propellers; (2) to obtain experience in handling and predicting the effect of high winds on large ships entering and leaving port; and (3) to provide additional training and experience in communication and teamwork skills during close manoeuvring. Throughout the paper, we will carefully consider all three of these objectives and show how they involve different types of fidelity. First: to achieve proficiency in handling the Azipod propellers.

In the observed exercise, a high level of technological fidelity became crucial for trainees to develop proficiency handling Azipod controllers (objective 1). Commonly used on tugboats, ferries and cruise ships, Azipods are 360-degree manoeuvrable propellers that allow greater flexibility in steering. Even though pilots usually only have an advising function, training on the use of Azipod propulsion systems is considered useful for pilots because they should also have the ability to skilfully control vessels in the event of an emergency situation.

The simulator’s control levers for manipulation of Azipod propellers can be seen in Figure 2 on the left-hand side. Figure 2 also illustrates an example Azipod propeller on the right-hand side. The simulator allows for steering controllers identical to the ones placed on real ships, and the operator can see the effect of changing the orientation and power of the Azipods in three places. First, the operator can see the human-computer interface called “conning”, which shows information about propeller revolutions per minute, the thrust and orientation; second, an Electronic Chart Display and Information System (ECDIS) displays information on changes in the vessel’s position as a function of altered physical forces acting on the vessel. Lastly, movements of the vessel can also be spotted by looking “outside” through the bridge windows. To facilitate the learning objective of Azipod handling, the simulator requires a level of fidelity connected to knobs and levers, precision in electronic maps, a realistic rendering of the outside environment and software able to calculate and realistically replicate the effects of wind and thrusters on the ship’s movements.

Figure 2
figure 2

On ships, Azipods are handled with the same type of knobs as in the simulator. As shown in picture a, the knob adjusts both the effect and position for each Azipod propeller. Picture b shows an Azipod propellar and was retrieved from: http://en.wikipedia.org/wiki/Azipod. Picture a was taken in the studied simulator by authors, as were the other pictures in this article.

Full size image

The observations and the majority of the video data showed that the pilots actively navigated and tested the effects of wind on the simulated cruise ship. In this simulator training design, the trainers focused on entering and leaving port by the use of Azipod-propellers and therefore repeated the same scenario several times. This demonstrates how simulators provide plastic learning environments with opportunities that are not possible in actual work settings, such as repetitions and do-overs — or ‘freezing’ scenarios for close instruction. For the training design, this represents a trade-off between realism and focused skills training. This issue has also been discussed in the research literature; that in terms of creating conditions for learning, simulators may have certain advantages over real environments (Hollnagel 2011, pp. 80–81).

Accordingly, whether the level of fidelity is sufficient for effective training, depends on the learning objective. In this case, when focusing on Azipod-handling in close waters, this particular feature of the simulator is put to scrutiny. The Azipod-steering, movement on the electronic map and the visual display needs to be more precise than during other scenarios we observed. In this training, this became particularly visible as pilots detected and paid a great deal of attention to discrepancies between the electronic map and the visual outlook through the bridge windows. This issue had not come up in prior trainings, but became a problem here because the close-manoeuvring required higher fidelity than prior trainings in terms of the relationship between information presented on the ECDIS and the visual lookout. This issue demonstrated that the requirements for technical fidelity are closely linked to the training scenarios, and that “high fidelity” should not be conceived of as a stable characteristic in the simulator. See Hontvedt (2015) for a detailed analysis of this incident.

As the next sections display, the training also involved several other objectives that influenced the experience. From a skills training perspective alone, it might have been more effective to train the Azipod handling in a more isolated manner that would allow for more repetitions and for more varied conditions. This situation illustrates how simulator training design involves a trade-off between the need to focus training by isolating learning objectives and the desire to maintain a high level of ecological validity. Table 2 displays this specific training objective and the associated fidelity requirements and measures for assessment.

Table 2 Configuring simulator fidelity, learning objective I.
Full size table

4.2 Psychological fidelity

Being a professional involves in-depth understanding and an adaptive attitude toward work challenges, abilities that benefit from a high level of psychological fidelity in training. In the current framework, psychological fidelity relates to each individual and his or her problem-solving strategies; it also relates to the establishment and use of mental models. Notions of psychological fidelity have roots in cognitive psychology focusing on individual human capabilities and internal information processing (Reisberg 1997). We reserve use of the term psychological fidelity for the training of individual operators’ problem-solving strategies, the establishment of mental models and sense-making activities. Similar concepts include environment fidelity (Waller et al. 1998), functional fidelity (Moroney and Moroney 1998) and experiential fidelity — closely linked to the feeling of ‘presence’ (Stoffregen et al. 2003).

One can several veins of research which orients fidelity—often tacitly—towards the learner’s perception and cognitive engagement. However, this notion often invokes differing definitions, such as the operators’ perception of realism (Beaubien and Baker 2004) or accuracy in prompting psychological processes relevant to performance in real-world settings (Kozlowski and DeShon 2004).

Researchers have connected the idea of psychological or cognitive fidelity with how simulator training affects cognitive schemas and problem-solving strategies, “the extent to which the training environment prompts the essential underlying psychological processes relevant to key performance characteristics in the real-world setting” (Kozlowski and DeShon 2004, p. 4). Psychological fidelity thus involves the human’s ability to perform the cognitive aspects of work tasks. Examples of these cognitive aspects of work tasks include decision making (Klein and Zsambok 1997), situation awareness (Endsley 1995), problem solving (Rasmussen 1983) and sense making (Kurtz and Snowden 2003). In addition to the technical functions of the simulator, other aspects of the learning situation are also important in creating a high level of psychological fidelity, including simulator immersiveness, the designed task, guidance and other technological or social affordances in the environment.

For example, though they themselves do not use the term psychological fidelity, this orientation towards meta-cognitive activity and underlying principles for action may be recognised in Silvennoinen et al. (2012). Their study shows how surgical residents were trained on basic laparoscopic surgical skills using computer-based simulator training. Simulator training is commonly considered to be a suitable learning tool for these skills because laparoscopic surgery demands the mastery of various instruments and visuomotor skills before the surgeon can operate on patients. The article examines the use of a laparoscopic training simulator with high technical fidelity on surgical instrument handles and pedals for conducting procedures in three-dimensional virtual interfaces through specially-designed exercises. The study suggests that simulator training improved performance, but also that autonomous training with the simulator was not ideal; rather, the residents needed a certain level of content-based feedback and supervisor support during their training activities. Through these meta-reflective activities, the teams had to pay explicit attention to problem-solving strategies, consequently promoting simulator learning as an avenue for developing adaptive expertise.

This latter finding shows how simulators may create opportunities for modelling a problem space and allowing participants to reflect on different scenarios and choices of action. Several studies have focused on the way knowledge structures influence our engagement with the surrounding world, and the shaping of shared strategies for action. High psychological fidelity involves creating scenarios that both confront and expand on participants’ mental models. It also means facilitating productive debriefings and reflection sessions where participants are allowed to connect concrete experiences to stable rules and conceptualisations as well as successful strategies for developing expertise (Bransford et al. 2000). Accordingly, many findings related to psychological fidelity weigh the simulation’s ability to facilitate discussions and debriefings. Such studies have found that low-fidelity simulators often offer such affordances just as well as full mission simulators (Baker et al. 1997; Beaubien and Baker 2004).

4.2.1 Empirical example of psychological fidelity in training

An accompanying learning objective for our case example was to help the participants develop a deep understanding of how ships react to wind and Azipod steering. The pilots were explicitly introduced to a table with matrices of wind and power as well as strategies for steering. The simulator activity was designed to sequentially test how different weather conditions and steering strategies affected the ship, developing and reinforcing mental models of these dynamics. Trainees’ situated experiences were supported through guidance and frequent debriefings. The following two extracts portray the dynamics and complexities involved in meeting these objectives. The transcriptions are presented verbatim, with square brackets marking the start and end of overlapping speech, equal signs indicating the immediate ‘latching’ of successive talk and double brackets enclosing comments on context or delivery.

Extract 1 shows a case of two pilots discussing strategies for Azipod positioning when departing Oslo Harbour. Pilot 1 handles the thrusters, and Pilot 2 leans over to confer on alternative actions:

 

Pilot 1:

But how much difference is there between the ways these pods are

2

 

positioned? Obviously, a little bit with=

3

Pilot 2:

= Well, as they are positioned now, you will get some water from the

4

 

propeller over on the other side. From that one. But when you move

5

 

forward, you know, the thickest end should be positioned forward. And that

6

 

is at that arrow there. But [I] feel it could have been positioned as that

7

Pilot 1:

[yes]

In line 1 of the extract, Pilot 1 requests his colleague’s advice on how the Azipods are positioned. In line 3 of the excerpt, his colleague, instead of giving a straightforward assessment, expands the problem area that Pilot 1 is tackling by pointing out some of the factors that influence the choice of Azipod positioning. He directs attention to how the propellers push water away, and to how this action affects the other Azipods. After pointing out his concern, he claims that he feels they could have been placed like they are.

This extract demonstrates how the participants continuously evaluated the relationship between present actions and general rules and principles founded in theory and professional knowledge. They do not enact professional positions—such as captain, helmsman, etc.—in this extract. Instead, they discuss different choices of action for the pilots in training. In this case, it is relevant to view the participants’ specific (co)construction of the problem area as a sense of psychological fidelity. As will be illustrated in extract 2, debriefings often assist in these constructions of the subject matter.

In extract 2, the two groups jointly discuss issues from the exercise regarding preferred ways of positioning the Azipods. The instructor relates an example of thruster control from a towboat he has been on, using a replay of the exercise on an electronic map to aid the debriefing:

1

Instructor:

In many ways it’s just as good to use the outer pod, right? And pull that

2

 

astern, that you pull ((the ship)) out with that one ((illustrates the

3

 

positioning of the propellers with his hands)). Because then it comes a bit

4

 

(x). And that worked really well ((referring to a ship he had been on))

5

Pilot 1:

But if you are going to thrust that way, it’s the outer one that is used since

6

 

that has the most effect

7

Instructor:

Yes, and he ((the captain on the ship he had been on)) told that=

8

Pilot 1:

=because it ((the Azipod)) sucks and sucks and it gets water much easier

9

 

than the one that has to draw water past the other one

10

Instructor:

And then you’ll use the starboard one, placed near the pier just for control

11

 

((illustrates with his hands how to turn the knobs)). That was very elegant

12

 

and with a feel of control

13

Pilot 2:

But if you, in relation to this, has learned that you should use one in relation

14

 

to this and the other one across, at least at dockings and so on. Because if

15

 

you start like...splitting them, if you can split them, that is. But I don’t know;

16

 

have there been any studies, like experiments, on what actually is safest?

In line 1, the instructor uses a replay of the ship’s movements on a digital map to discuss different strategies for Azipod steering. He uses his hands to illustrate how the Azipods may be positioned and refers to a ship he had been visiting earlier in the year. In line 5, Pilot 1 adds to the instructor’s explanation by pointing out the way the Azipods affect each other by sucking water away. The instructor (line 10) adds to Pilot 1’s clarification by recommending the inner Azipod for adjustments because the outer propeller achieves more propulsion. As Pilot 1 and the instructor discuss the use of the Azipod in terms of ship control and efficacy, Pilot 2 raises a new issue in line 13. He repeats some strategies for manoeuvring before he calls attention to another general aspect of ship manoeuvring: safety.

This extract demonstrates how the pilots relate the actions in the simulator to general ideas and conceptions while also seeking rule-based or otherwise valid strategies for manoeuvring with Azipods. In these sequences, the participants seek general models for action to guide future actions, using a digital replay of the exercise to aid the debriefing (Figure 3). The instructor’s use of the replay also shows how different types of tools can aid debriefing.

Figure 3
figure 3

The instructor points at the ship’s movements on a replay of the exercise.

Full size image

The participants link the simulator experiences with their prior understandings of how wind and thruster control affect a ship’s movements. This connection between prior understandings and experiences in the simulator became an important theme in the debriefings, displaying how processes of connecting immediate experiences to general principles require guidance and debriefing. Such processes also necessitate critical reflection regarding whether the simulator sufficiently recreates the work setting. Psychological fidelity can be a helpful concept for describing the simulation’s accuracy and engagement in confronting participants’ mental models, problem-solving strategies and rule-based thinking. However, as shown in the following table, the facilitation of psychological fidelity may also infer technical fidelity. Extract 1 demonstrates how sometimes participants’ reflections regard participants’ direct experience in the simulator — made possible through technical tools — at other times the simulation facilitates a joint focus and discussions that move beyond the concrete experience. This was the case in extract 2, in which participants touch upon more advanced aspects of Azipod-handling than was directly experienced during the simulator session.

This specific training objective, its fidelity requirements and measures for assessment, may be presented as follows (Table 3):

Table 3 Configuring simulator fidelity, learning objective II.
Full size table

4.3 Interactional fidelity

Professional life involves an ability to participate in interactional patterns at work (Edwards 2010; Engeström and Middleton 1998). We introduce the notion of interactional fidelity to describe the accuracy and relevance of participant collaboration and enactment of work tasks in simulator training. While psychological fidelity is reserved for aspects that affect the individual operator, interactional fidelity emphasises the interactional patterns of a socio-technical system (Hutchins 1995). Hence, interactional fidelity involves collaborative and coordinating patterns between both human and material entities of the system. The notion of interactional fidelity is a neologism that is introduced for conceptualising such accuracy in the interactional patterns in a sociotechnical system.

The distributed and situated nature of professional work has been empirically investigated by prior studies, and work tasks have proven to be collaboratively and discursively coordinated (Goodwin 1994; Hutchins 1995). Suchman (1987) used the term situated action to describe how the performance of an action cannot be considered merely the completion of a pre-registered plan; it must also address situation-related contingencies and opportunities. Such a view on professional expertise emphasises how skills, knowledge and attitudes all are shaped and (re)constructed in situ. Training for such complex situations involves participation in relevant activity systems of work.

One study that illustrates high interactional fidelity in a simulation is that of Hutchins and Klausen (1996), who conducted a detailed investigation of interactions in a full-mission flight simulator and described the pilots’ collaborative work efforts with the terminology of distributed cognition. Without focusing on the simulation as a training activity per se, they nonetheless revealed the potential for credibly recreating situated patterns of work actions in a simulator. Researchers have also shown how simulators may provide support for participating in the activity systems of work, where the human actors interact with other entities of the system. As Hutchins and Klausen (1996) have demonstrated, the cognitive properties of the cockpit system are created by the pilots’ cognitive efforts, together with the physical properties of representational media.

Hutchins and Klausen show how a situated activity system can be simulated in a way that involves what we characterise as high interactional fidelity. However, interactional fidelity can also characterise accuracy in recreation of social interactions on a broader level. For example, Kneebone (2016) suggests that simulations can be used for bridging and aiding communication between different shareholders involved in medical practice, such as doctors, patients and family. Similarly, Hontvedt and Arnseth (2013) showed how a professional maritime pilot participated in a simulation together with nautical students and how this affected the collaborative organisation of the simulation.

Given this backdrop, creating complex work environments for learning requires sensitivity to how work settings are created and coordinated collaboratively using conceptual, cognitive and physical tools. Acknowledging action as situated in specific contexts has consequences for simulator-training designs. To ensure high interactional fidelity in simulator training, one should evaluate how the social activity of simulating targets the activity of the professional domain for which the participants are training. Consequently, to prepare trainees for participation in distributed work environments—such as a ship bridge, an operating room or an airline cockpit—simulator training should involve the collaborative and coordinating practices of work that are key to later professional action. Pursuing interactional fidelity may provide measures for organising training, linking simulated actions to the cooperative practices of professional life.

This notion does not suggest that a simulation can incorporate the full complexity of an actual situation. A simulation will mainly remain a training situation with opportunities and constraints different from those of a real work situation. Nevertheless, interactional fidelity may orient attention in training towards the cooperative and coordinating patterns that is not encompassed in prior conceptions of fidelity.

4.3.1 Empirical example of interactional fidelity in training

In the exercise, one overall learning objective was to strengthen the pilots’ teamwork abilities by having them simultaneously adopt unfamiliar roles and scenarios while providing a relatively rare opportunity to collaborate with peers—since pilots usually work individually.

In extract 3, we see three pilots leaving the port of Oslo and observe how they shift from the captain steering the ship by using the propeller-knobs to the helmsman operating the steering wheel in the centred position (See Figure 4 below).

1

Captain:

The thruster is going 50 % starboard

2

Pilot:

Stop thruster

3

Captain:

Stop thruster

4

Pilot:

We may increase speed a little bit

5

Captain:

Increasing speed

6

Pilot:

Is the helmsman ready?

7

Helmsman:

I am ready

8

Pilot:

Starboard twenty

9

Helmsman:

Starboard twenty

Figure 4
figure 4

During the extract, the captain, pilot and helmsman, were positioned typically as marked in the picture: in front of the main control board, next to the radio overseeing the same instruments as the captain and behind the mid-centred steering wheel. During close manoeuvring, the captain usually shifts to manually controlling the ship by the propeller-knobs him or herself, rather than having a helmsman at the centred steering wheel.

Full size image

In this extract, three pilots are filling the roles of captain, pilot and helmsman in the simulator ‘Bergen’. In line 1, the captain announces the thruster force, leading the pilot to request that he stop the thruster. In line 3, the captain confirms the thruster reduction by repeating the pilot’s request. During their briefing, pilots were encouraged by the instructors to adopt institutional roles and to practice ‘closed-loop communication’, meaning that the participant that receives orders or information repeats them out loud, displaying that the correct message has been received and will be carried out.

In line 4, the pilot requests an increase of speed, which is confirmed by the captain in line 5. This request displays the distributed nature of ship manoeuvring, as well as the importance of coordinating their actions. In line 6, the strict divisions of labour within the team are displayed when the captain requests the helmsman to take over the steering, transferring steering from the control table positioned in front of the cockpit to the mid-centred helm. This shows how the instructor’s advice on role-playing ‘closed-loop communication’ enables the coordination of tool-handling. Consequently, the technological aspects of the simulator are fundamentally connected to the social aspects of the simulation.

This extract shows how simulation practices are closely entwined with the maritime profession’s specific way of perceiving and enacting work tasks. It also emphasises that skills are deeply integrated with communication and coordinating actions. Skills may be taught successfully in an isolated manner, but participants’ ability to adapt and use the skills successfully within the socio-technical environment of a ship’s bridge is also of great importance for developing expertise.

Accordingly, the facilitation of one type of fidelity may also require other types of fidelity. Regarding the learning objective of obtaining teamwork and coordinating experience in docking manoeuvres, the fidelity requirements and measures for assessment may be displayed as follows (Table 4):

Table 4 Configuring simulator fidelity, learning objective III.
Full size table

5 Discussion

The point of departure for this article was its discussion of strategies for utilising simulators in professional training, with a particular focus on coordinating and collaborative activities. We identified fidelity as a key concept for describing the accuracy in which professional practice is simulated, and suggested that prior conceptualisations of fidelity in simulator training remain unclear in terms of their relationship to learning. By outlining a novel framework for creating simulator training designs, we explained how different training objectives can relate to simulator requirements and enable the trainer to draw on different learning perspectives as resources for designing and analysing simulator training.

Throughout the paper, we have provided the example of a maritime pilot training to illustrate how fidelity may be focused in accordance with training objectives. The ship pilot example demonstrated how the handling of the vessel and responses to changing environmental cues require a lifelike appearance for the visual, haptic and auditory aspects of these work tasks. Targeting specific knowledge forms or mental models requires the ability for meta-reflection and rule-based judgements, whilst communication or teamwork requires consideration of the patterns of coordination and participation in training sessions.

The three types of fidelity—technical, psychological and interactional—are not considered mutually exclusive. The presence of these three different types are a matter of degree, not a dichotomous present/not present distinction. To a certain extent, all three types are probably present in any given training session. We simply assert that the learning objective should be the factor determining the type of fidelity that receives the most weight in the training scenario and that facilitators of simulator training designs may profit from a variety of research.

However, a study’s theoretical approach guides the methodological foci, the vocabulary for reporting findings and the recommendations for future practice. Accordingly, there have been extensive discussions on the (in)commensurability of learning theories (e.g. Anderson et al. 1996; Greeno, 1997; Packer and Goicochea 2000; Sfard 1998). However, from our perspective, even if theoretical traditions can have incommensurable concerns, they might provide complementary suggestions to practitioners.

The learning objectives in professional training are often defined by institutional standards and formal classifications of learning outcomes. For example, within the maritime domain, the Convention on Standards of Training, Certification and Watchkeeping for Seafarers represents internationally agreed upon standards for competence (International Maritime Organisation 2011). Here, technical and non-technical skills are described and therefore need to be addressed systematically in training. However, such a distinction is not necessarily supported across theoretical domains. For instance, a situated learning approach differs from a cognitive psychology one in its view of learning and thinking as situated in social contexts, not in the individual, and in the rejection of the divide between mind and behaviour (Lave and Wenger 1991). This is an example of theoretical underpinning which needs to be handled pragmatically when creating training designs.

Furthermore, separating different types of skills and expertise is a much-used instructional strategy that makes automating and measuring different types of skills possible. For example, low-fidelity simulators simplify a system to highlight its key components and to allow professionals to isolate and cut out irrelevant parts of training. Although instructional strategies that target specific skills and training perfected in isolated environments have been proven successful in many cases, training practitioners must not underestimate how such skills and expertise could be used to coordinate the activities between different actors in the socio-technical system (Cooke et al. 2000). Focused training may provide good results, but in a situated perspective, the divide between communicative and instrumental action is not valid in situ where communication is increasingly intertwined in core productive processes (Engeström 2008, p. 22).

This is particularly relevant in CSCW settings. Through the notion of interactional fidelity, this framework puts more weight on participation in socio-technical systems as a focus in training, compared with previous research on fidelity and learning. This focus on interactional patterns is in line with what prior research has shown that many innovative and advanced attempts to provide technological support in the workplace fail not so much because of technological insufficiency but because of insensitivity to the ways in which individuals interact and collaborate in the workplace (Heath and Luff 1996).

Sensitivity towards the complexity in socio-technical systems does not necessarily mean that one should strive to create simulations that are exact copies or with the same degree of complexity as the actual workplace. It is the joint notion of the authors that learning in simulators represents a social practice that differs from that in work situations; therefore, striving to fully recreate the real thing is a dead end (Stoffregen et al. 2003). In our conception, a more nuance notion of fidelity enables focus on the accuracy in the re-creation of specific elements of the simulated environment, independent of whether these elements are technical or social, and provide a lens for assessing the configuration of work-relevant simulations.

5.1 Implications for simulator training design

Our tripartite conceptualisation of fidelity for training simulators has a number of practical implications for creating simulator training designs.

First, the framework for aligning learning objectives with simulator fidelity requirements and learning outcomes is presented as a practical table that easily enables trainers to make notes and conceptualise different types of simulator training designs. We believe that conceptualising such elements of simulator activities might increase the level of sensitivity to the learning processes that are initiated and enhance collaboration among participants. And provide an introduction to key approaches in the research field, and the benefits and limitations of the various veins of research.

Second, the conceptualisations focus attention on how well a training situation supports the achievement of learning objectives. Through the use of analytical examples, we have demonstrated how technological, psychological and interactional fidelity may provide useful foci for targeting accuracy and truthfulness in different entities of simulator training. Such facilitation of simulator training requires creating a productive balance between wishes for focused training and the maintenance of a high level of ecological validity in the learning process.

Third, we emphasise that the assessment should be aligned with the character of the task. Learning spans the acquisition of rather different types of skills and knowledge, such as behaviors, thought processes, memory traces or problem-solving strategies (Säljö 2003). For such assessment, the learning outcome may or may not depend on the simulator. For example, in behavioural and skills training, the accuracy of the tool used is essential to task performance. The ability for meta-reflection may easily be assessed in written or oral form in another setting; by contrast, technical and coordinating abilities depend on a simulator or a real work setting for assessment. In this way, organising assessment in simulator training design involves close analysis to which settings expertise is made visible.

6 Concluding remarks

Simulators have been proven effective in providing work-like environments for professional training. However, simulations have been challenging to model, to facilitate technologically and to assess. The framework developed through this article sheds light on the complexity of simulating professional work. It focuses on the learning objectives and considerations for the physical representation of the environment (technical fidelity), the activation of suitable cognitive processes and problem-solving strategies (psychological fidelity) and the enabling of successful teamwork skills (interactional fidelity). Together, these aid in the creation of specific learning designs in which participants can undertake different types of tasks, and underscore the importance of connecting these specific types of skills and knowledge to a larger body of expertise over time.

Simulation based learning environments provides an opportunity for further investigations into the complexities of work, as well as joint efforts in targeting such expertise in safe and adaptable training environments. Hence, rich conceptualisations of simulator fidelity may assist in the creation of reliable simulated environments for building professional expertise.