Keywords

1 Introduction

Command and control systems have many potential sources of faults, one of which is the human operator. Operators are a primary source of vulnerability in complex systems: for example, studies have shown that 66% of hull-loss accidents in commercial jet aircrafts [2] and 74% of fatal accidents in general aviation [1] are attributed to human error. However, there is relatively little work on categorizing the different types of operator faults than might contribute to the high frequency of operator errors.

One of the most influential taxonomies of faults was developed by Avizienis and colleagues [6]. It covers some aspects of the human operator and allows for a variety of operator faults, but does not provide a full treatment of the human-computer interaction loop (HCIL). In particular, Avizienis’s taxonomy does not address issues such as environmental causes for operator faults (e.g., turbulence that prevents a pilot from pressing a button), the different subsystems in a human (perceptual, cognitive, or motor function) that can cause faults, or the difference between failures within the operator (e.g., not having adequate muscular control to guide a vehicle) and failures that are caused by another person (e.g., someone shining a laser pointer into a pilot’s eyes).

Because faults stemming from the operator’s interaction with a complex system are common and often critical, it is important to better understand the nature and causes of these faults. In identifying this need, Sheikh Bahaei et al. [3,4,5] extended previous fault taxonomies with a focus on addressing the specific issues arising in augmented reality interaction. More precisely, [3] builds on top of human error taxonomies including Reason [50], Norman [51] or Rasmussen [52] and provides a human error taxonomy using feature diagrams from [53]. In this paper, we pursue a more general approach, expanding on Avizienis’s influential fault taxonomy to better characterize and explain operator faults, focusing on internal and external events that induce error states inside the operator. This approach contrasts with that of others – in particular, Avizienis’s work focused on faults that induce error states inside the system. We build on that taxonomy (widely used in dependable computing) and integrate an interactive systems engineering approach with the goal of improving dependable interactive systems.

We add categories in Avizienis’s System boundary dimension to include causes of operator error that are either internal to the operator or external; we add categories to the Phenomenological cause dimension to recognize new sources of faults including environmentally-induced operator faults and faults induced by other people; and we add a new dimension Human capability to separate out faults that occur in the operator’s perceptual, cognitive, and motor subsystems. These additions provide 24 types of operator faults, many of which have not been considered in previous work. Our expansion provides designers and researchers with new classes of potential faults that cover common and important real-world phenomena, and that improve understanding of how faults occur in the human-computer interaction loop. By showing where operator faults can arise, our work can improve the design of new interactive systems and lead to better evaluation of existing systems and diagnosis of accidents and incidents. We demonstrate that the classification makes it possible to position previous work in the field of HCI addressing fault tolerance, fault prevention, fault removal and fault forecasting.

The paper first provides an introduction to the human-computer interaction loop and the way operators interact with technological systems. Second, we present our expanded taxonomy of operator faults and describe the main structures and categories, with examples from aviation and other task domains. Third, we describe how existing HCI research fits into our framework, and fourth, we discuss amelioration strategies for the new fault categories, using the general approaches of fault removal, fault tolerance, fault prevention, and fault forecasting.

2 The Human-Computer Interaction Loop (HCIL)

The research field of Human-Computer Interaction (HCI) aims to build knowledge about humans interacting with computing systems. The field covers methods, techniques, and tools for designing and developing computing systems adapted to their users. Typical properties that are targeted by HCI research are usability [7], user experience [8], accessibility [9], and acceptance [10]. In order to reach these objectives, HCI promotes iterative user-centered design and development processes [11] that use variable-fidelity prototyping [33] and continuous feedback from real users [12].

These processes do not necessarily lead to robust computing systems: for example, cellphones are less dependable than fixed lines [13], but users may accept reduced dependability if it is accompanied by significant improvements in user experience. In the area of safety-critical systems, however, dependability cannot be compromised for user experience and usability, as people’s lives are at stake; in addition, in some domains such as aviation, certification authorities explicitly require a very high level of dependability (e.g., the certification specification requirements in [14]).

These requirements mean that designing interactive systems that are both dependable and usable implies making informed compromises. Making such compromises is not an easy task (as demonstrated in [15]) as it requires blending knowledge from several disciplines like HCI and dependable computing. The following section highlights the principles behind the engineering of interactive systems, providing a holistic view that incorporates the human with the computing system.

2.1 The Human-Computer Interaction Loop

Figure 1 presents an architectural view (from left to right) of the operator, the interactive command and control system, and the underlying system (e.g., an aircraft engine). This architecture is a simplified version of MIODMIT (Multiple Input and Output Devices and Multiple Interaction Techniques), a generic architecture for multimodal interactive systems [26] described in AADL [27]. Following the attribute dimensions of [6] we highlight (top right of Fig. 1) the hardware and software components, and show how the human operator interacts with them (thick dotted lines).

Fig. 1.
figure 1

Architecture of interactive systems with operator, hardware, & software components.

Full size image

As shown in the figure, interaction mainly takes place though the manipulation of input devices (e.g., keyboard or mouse) and the perception of information from the output devices (e.g., a computer screen or speaker). Another channel usually overlooked is the direct perception by the operator of information produced (usually as a side effect) of the underlying cyber-physical systems (e.g., noise or vibrations from an aircraft engine (represented by the lower dotted line in the figure)).

The top left of the Software section of the diagram corresponds to the interaction technique that uses information from the input devices. Interaction techniques have a tremendous impact on operator performance. Standard interaction techniques encompass complex mechanisms (e.g. modification of the cursor’s movement on the screen according to the acceleration of the physical mouse on the desk). This design space is of prime importance and HCI research has explored multiple possibilities for improving performance, such as enlarging the target area for selection on touch screens [29] and providing on-screen widgets to facilitate selection [28].

The right side of the Software section of the architecture corresponds to what is usually called interactive applications. This is where HCI methods such as task analysis are needed for building usable application that fit the operators’ work [30].

The left side of Fig. 1 represents the operator’s view. The drawing is based on work that models the human as an information processor [22], based on previous research in psychology. In that model, the human is presented as a system composed of three interconnected processors. The perceptive system senses information from the environment – primarily the visual, auditory, and tactile systems as these are more common when interacting with computers. The motor system allows operators to act on the real world. Target selection (a key interaction mechanism) has been deeply studied [32]; for example, Fitts’ Law provides a formula for predicting the time needed for an operator to select a target, based on its size and distance [31]. The cognitive system is in charge of processing information gathered by the perceptual system, storing that information in memory, analyzing the information and deciding on actions using the motor system. The sequential use of these systems (perceptive, cognitive and motoric) while interacting with computers is called the Human-Computer Interaction Loop (HCIL).

2.2 The Operator as a Service

If we consider the operator as a service provider to the interactive system (by manipulation of input devices for selecting commands and entering data) and a service consumer of information presented by means of the output devices, this service might exhibit failures, i.e., that the delivered service deviates from correct service (as introduced in [6], section 3.3.1, p. 18). While that paper [6] was focusing on faults that might trigger service failure on the software and hardware parts of systems, the taxonomy presented in Sect. 3 will identify faults that might trigger failures in the operator him- or herself by exploiting the human information processor decomposition.

A key abstraction in the HCIL is that of the match between the variance in the signal produced by either the user or the system (e.g., the variance in the user’s motor movements, or the brightness of the output display) and the tolerance for variance in the receiver of the information (e.g., the size of a target in the interface, or the user’s visual acuity). If the variance exceeds the tolerance, the operator might enter an error state. For example, the requirements for correct selection of a button on a touchscreen are that the variance in the movement of the finger is less than the extents of the button: if the button is 2 cm in diameter and the user has a 1 cm variance when aiming for the centre of the button, the button will be selected correctly; if the user has a 3 cm variance in their aiming motion, errors may arise. This variance element is key in the design of user interfaces and interaction techniques. If the button is the size of the entire screen, selection will be faster and the operator will be able to select even in severe turbulence; however, very little information will be presented, thus reducing the effectiveness and efficiency of the application. As described in the next section, various elements of the operator and the external environment can affect both the variance in the signals, and the tolerance for variance, in an operator-system interaction.

3 Taxonomy of Faults for the HCIL

Our taxonomy of operator faults expands on the framework of Avizienis and colleagues [6]. We use Avizienis as a foundation due to its widespread use and influence on the field. Other taxonomies have been introduced (such as Sheikh Bahaei et al.’s taxonomy of fault taxonomies [3]) that cover various aspects of operator error lacking in previous frameworks (such as faults that arise from augmented reality interaction [3]); however, previous work is primarily focused on specific areas rather than general limitations.

We expand the Avizienis framework in four ways. First, we extend the System boundary dimension to recognize that human faults can be induced in the operator from external causes. Second, we add new levels to the Phenomenological cause dimension to distinguish between faults arising 1) from the operator, 2) from another person, and 3) from the natural world (including the system itself). Third, we introduce the Human capability dimension to differentiate faults in the operator’s perceptual, cognitive, and motor abilities. Fourth, we add specific fault categories that derive from these dimensions. Figure 2 provides an overview of the taxonomy.

Fig. 2.
figure 2

Overview of the taxonomy of faults focusing on the HCI Loop.

Full size image

3.1 Changes and Additions to the Fault Dimensions

The Avizienis framework provides several dimensions that characterize faults in terms of when, where, and how they arise: at the highest levels, they distinguish between development and operational faults (Phase), faults that are internal or external to the system (System boundary), and faults that are natural or human made (Phenomenological cause). Although this structure allows for a wide range of fault types (including operator and environmental faults), it does not systematically categorize and describe the ways in which operator faults can occur.

In particular, the complex interactions between an operator and a system (i.e., the HCIL) have properties and characteristics that are separate from the operator alone or the system alone, and the HCIL can lead to many different types of faults that have many different underlying causes – some of which involve the fault being “induced” in the operator by outside forces. For example, an aircraft’s hard landing may arise from within the operator (e.g., a pilot’s early-stage Parkinson’s disease that reduces their muscular coordination), from another person (e.g., someone shining a laser pointer into the pilot’s eyes from the end of the runway), or from effects of the natural world (e.g., air turbulence that shakes a pilot’s arm as they try to press a button on the instrument panel). Although these three faults are very different in terms of implications for design, they would all be placed in the same category in the Avizienis framework (i.e., “Operational/External/Human-made/Non-malicious/Non-deliberate/Accidental” operator faults). To address this gap, we need to broaden the dimensions that characterize faults. In this paper we focus only on operational faults (leaving aside the development faults), and expand the dimensions of System boundary and Phenomenological cause.

We expand the System boundary dimension to add the HCIL as a conceptual location for faults that should be considered separately from Avizienis et al.’s categories of “internal to the system” and “external to the system.” We apply the idea of internal/external to divide HCIL-based faults into those that arise from inside the operator (see Fig. 3) and those that arise external to the operator (see Fig. 4).

Fig. 3.
figure 3

Focus of the taxonomy on Internal faults (from inside the operator) with examples.

Full size image
Fig. 4.
figure 4

Focus of the taxonomy on External faults (from outside the operator) with examples.

Full size image

We next identify new levels for the Phenomenological cause dimension that explicitly recognize that when an operator is unable to perform task actions correctly, the cause may be human-made or from the natural world. When the fault is internal, human-made implies that it is within the operator; when external, human-made implies the action of another person. We then create a new dimension – Human capability – to characterize the human processing subsystem where the fault is located (see discussion of the human information processor in the previous section). The HCIL requires three main kinds of human ability (perception, cognition, and motor control) and faults in any of these can lead the operator to reach an error state.

Finally, there are several other dimensions that play roles in our characterization of operator faults. We use the Objective (Malicious, Non-malicious), Intent (Deliberate, Non-deliberate), and Persistence (Persistent, Transitory) dimensions in a similar fashion to Avizienis and colleagues; however, the Capability dimension, which classifies faults as accidental or due to incompetence, focusses on the cause of the fault rather than its description. For this reason, we decided to leave it outside of our classification.

3.2 Operator Faults Arising Externally (From Outside the Operator)

Operator Faults Induced by the Natural World.

This category can only have Non-malicious and Non-deliberate faults, because the source of these faults is the natural world, which does not have objectives or intents. There are fault types that affect each of the three human capabilities (perception, cognition, motor control), and it is important to note that some types of environmental phenomena may arise from the system itself rather than from weather, sunshine, or terrain (e.g., vibration may come from an aircraft’s engines as well as from air turbulence). The main difference between system-based and non-system induced faults is in the operator’s ability to control the system to reduce the phenomena (e.g., reduce engine power to reduce vibration); we discuss this further in Sect. 4 below.

Environment-Induced Perceptual Faults are caused by natural-world phenomena that reduce the operator’s perception of the system. The primary senses of concern for interactive systems are sight, hearing, and touch. Example faults in this category include bright sunlight that “washes out” a display screen, reducing the operator’s ability to see and interpret visual objects; vibration from air turbulence or a rough road that reduces both the operator’s visual perception (e.g., tracking a moving object on a vibrating display) and tactile perception (e.g., receiving vibro-tactile alerts); or a noisy environment that reduces the operator’s ability to hear alert sounds.

Environment-Induced Cognitive Faults are caused by phenomena that reduce the operator’s cognitive capabilities – primarily memory and decision-making. Natural phenomena such as loud noises and bright flashing lights are known to cause problems for cognitive ability by saturating the brain’s communication channels [34]. In addition, environmental conditions such as a low-oxygen atmosphere can have severe effects on cognitive ability, memory [36], and peripheral perception [35]. This demonstrates that environmental faults may alter operator performance on all three capabilities, but dividing these into three is relevant as some faults only alter one capability.

Environment-Induced Motor Faults involve natural phenomena that reduce motor abilities including movement precision, strength, or endurance. For example, air turbulence reduces the accuracy of a pilot’s finger movement toward a touchscreen target; if the variance in the pilot’s finger movement exceeds the system’s tolerance (as determined by the size of the target) then failures in selecting targets can occur [23]. Similarly, reduced temperature can affect muscular control in an operator who needs to carry out precise hand movements or complex gestures [37].

Operator Faults Induced by Other People.

This category involves another person acting in such a way that the operator’s perceptual, cognitive, or motor abilities are compromised. Because the source is another human, these faults can vary in terms of Objective and Intent.

Other-Person-Induced Perceptual Faults are those in which another person’s actions compromise the operator’s sight, hearing, or touch. Malicious actions include, for example, shining a laser pointer into a pilot’s eyes (preventing them from seeing a display [38]), or making loud noises when an operator needs to hear an auditory signal. The degree to which the operator’s perception is compromised and the tolerance built into the HCIL will determine whether or not a failure can occur.

Non-malicious faults in this category can be either Deliberate or Non-deliberate. Non-deliberate actions are extremely common: these could involve a person inadvertently standing in front of the operator (and thus occluding a display screen) or talking loudly to the operator (and thus preventing them from hearing an auditory signal). Deliberate but non-malicious actions in this category are less frequent, but still possible: for example, a person could stick a post-it note on a display to cover an annoying flashing alert (i.e., deliberately reducing perception of the display) without realizing that the operator will not perceive future alerts.

Other-Person-Induced Cognitive Faults are those in which another person compromises the operator’s memory or decision-making ability. A Malicious and Deliberate action here could involve another person interrupting the operator to prevent proper decision-making. Non-malicious and Non-deliberate actions could include another person providing information to the operator at the exact time when the operator is trying to memorize something. Other types are less likely: e.g., it is unlikely that someone would deliberately compromise an operator’s cognitive abilities without malice.

Other-Person-Induced Motor Faults are those in which another person reduces the operator’s motor control. A Malicious and Deliberate action here could involve another person bumping the operator’s arm to prevent them from targeting precisely. Non-malicious and Non-deliberate actions could include another person placing objects on a desk that get in the operator’s way, or a child pulling on a parent’s arm while the parent is trying to drive a car. Alternatively, an operator might Deliberately but Non-maliciously block another person’s action (by grabbing their arm, for example) if they see that they are about to select an incorrect control (a fault would occur if the blocking results in the triggering of another incorrect control).

3.3 Operator Faults Arising Internally (From Inside the Operator)

This category involves faults that are not induced in the operator by external forces, but that arise from the operator him- or herself. These can still be categorized in terms of their effects on the operator’s perception, cognition, and motor abilities.

Operator-Made Faults.

Operator-made faults are those in which the operators compromise their own perceptual, cognitive, or motor abilities. These faults are most commonly Non-malicious, although rare cases involving malice are possible. Examples of Operator-Made Perceptual Faults include Non-malicious actions that are Deliberate (e.g., an operator not wearing their prescription glasses because of vanity, thereby reducing their visual acuity) or Non-deliberate (e.g., damaged hearing from listening to loud music). Malicious actions are rare (e.g., deliberate self-harm of the operator’s eyes or ears).

Examples of Non-malicious Operator-Made Cognitive Faults include Deliberate actions (e.g., an operator skipping system training because of laziness) and Non-deliberate actions (e.g., an operator drinking or using drugs on the job, or an operator forgetting to carry out a training module). Again, malice is rare in this category (e.g., purposefully choosing to skip training or take a drug to impair cognition). Operator-Made Motor Faults can be Non-malicious and Deliberate (e.g., an operator wearing gloves even though they know this reduces their ability to type) or Non-deliberate (e.g., having long fingernails that reduce touch accuracy on touchscreens). Malicious and Deliberate actions (again rare) could involve self-mutilation of the hands or fingers needed to operate a system.

Faults in the Operator Induced by the Natural World.

The natural world can also affect the operator’s capabilities through natural processes that are internal to the operator. Aging, disease, fatigue, and other elements of the human condition can have substantial effects on perception, cognition, and motor control. As the natural world is the source of these faults, they are all Non-malicious and Non-deliberate. Examples of Natural Perceptual Faults include reduction in color perception due to color-vision deficiency (commonly called color blindness) or reduction in visual acuity because of age-related presbyopia; reduction in auditory capability is commonly caused by age-related deafness. Examples of Natural Cognitive Faults include well-known cognitive biases (e.g., “loss aversion” [39] in which people prefer to avoid losses rather than achieve equivalent gains) as well as age- or disease-related dementia and memory loss. Examples of Natural Motor Faults include reduction in touch accuracy due to conditions such as Parkinson’s disease, or reduction in strength due to aging (e.g., the captain of Ethiopian flight ET302 requested the first officer to “Pull with me”, applying a force of up to 110 lbs on the control column [49]).

It is important to note that these biases are not separated out in Avizienis et al.’s classification, even though they are of different types (information overload, lack of meaning, need for fast action, and decisions about what to remember), are numerous [41], and have strong safety implications (e.g., attention tunneling in aviation [40]).

4 Analysis of Gaps for Improving Dependability of the HCIL

The taxonomy presented above introduces new concepts and classes for the domain of operator faults, and the specific characteristics of many of these categories suggest ways in which the faults can be avoided, ameliorated, or repaired. In this section we consider four common mechanisms for improving dependability – fault removal, fault tolerance, fault prevention, and fault forecasting – and apply them to the taxonomy.

Many of the strategies described in the sections below arise from our basic characterization of interaction between an operator and a system as a communication of information with certain variance and tolerance (Sect. 2.2). Improvements to dependability can therefore focus on either increasing tolerance or reducing variability. On the input side, tolerance could be increased by making touchscreen buttons larger and using a more stable selection action such as a long press instead of a tap; variability could be reduced by training the operator to brace their hand on the display bezel. For output, the operator’s visual acuity could be improved with corrective lenses, or the size and contrast of the text in an alert dialog could be increased to improve comprehensibility. While at the core of HCI discipline, systematically identifying design options with respect to the faults they address could lead to more dependable interactive systems.

4.1 Fault Prevention

Fault prevention involves preventing the introduction or occurrence of faults. In Avizienis et al., prevention of operational faults is not addressed (even though prevention of development faults is covered in section 5.1, p. 24). Prevention of operational faults (inside the operator) can be done by adapting input devices, output devices, interaction techniques and user interface so that they prevent faults from occurring. On the External side (Fig. 4) this can be done by removing interference from others, from the system, and from natural causes. Some solutions are beyond current technology (e.g. preventing turbulence in aircraft) or may add other problems (e.g., removing the first officer to reduce distraction would cause more problems when workload increases). On the Internal side (see Fig. 3), prevention can be accomplished through training (e.g. informing operators about cognitive biases and techniques for debiasing [44]) or through human augmentation (e.g. using night-vision goggles, although their use can induce new types of accidents [43]). As operator behavior is far from predictable, however, fault prevention techniques might fail and faults then have to be removed.

4.2 Fault Removal

Fault removal strategies attempt to reduce the number and severity of faults. The main type of fault removal for the HCIL is “preventive maintenance” which aims to uncover and remove faults before they cause errors (Avizienis, p. 28). However, different strategies will be needed in the different main categories of our taxonomy:

  • Internal/Operator-made faults arise from actions taken by the operator, and are therefore best removed through organizational strategies (e.g., better enforcement of training, increased concern for operator mental health, and better understanding of conditions in the workplace that could lead operators to act in an unsafe manner).

  • Internal/Natural faults must be removed by addressing the underlying natural cause. For example, faults caused by limits on ability due to aging or disease can be avoided both by treating conditions that are treatable (e.g., providing corrective lenses to operators who need them) and by accommodating reduced ability by increasing system tolerances (e.g., using brighter cockpit displays to accommodate the reduced night vision of an aging pilot population).

  • External/Human-made faults involve the actions of other people, and so removal strategies are more difficult to prescribe. Regulations that limit access to the operators’ workplace can assist with this category (e.g., not allowing the public near where operators are working, and ensuring that people who do have access are aware of the risks of interrupting or disrupting operators).

  • External/Natural faults involve natural-world phenomena inducing faults in the operator. Removal strategies can focus either on reducing the effects of likely phenomena or on improving the operator’s abilities during the phenomena. For example, the effects of turbulence could be mitigated by allowing pilots to fly to smoother air (reducing the phenomenon), or by teaching pilots to brace their hands while reaching for controls [18] or better control their movements during turbulence (improving operator abilities). Similarly, strategies could reduce sunlight or loud noise through filters or through technologies such as noise-cancelling headphones, or could increase the magnitude of the system’s visual or auditory signals.

4.3 Fault Tolerance

Fault tolerance is the delivery of a correct service despite the occurrence of faults, and has several elements that are relevant to the HCIL, including error detection, recovery, and error handling. First, error detection in a human-computer system will often involve the operator rather than the system – that is, it is often only the operator who can determine that an input was erroneous. The design of an interactive system can assist the operator using well-known HCI strategies such as providing sufficient feedback to the operator to help them detect errors (e.g., mode errors in aircraft automation [42]), or providing “reasonableness checks” on input.

Second, HCIL-based recovery and error handling (i.e., mechanisms that eliminate errors from the interactive system state) can be based on the idea that input and output are a kind of communication between the operator and the system that occurs in a noisy channel. Telecommunications theory uses the idea of adding redundancy in order to preserve the signal, and a similar approach can provide fault tolerance in the HCIL. For example, in an aircraft cockpit where turbulence causes touchscreen errors, the human-system communication channel could add redundancy through command repetition or explicit confirmation. As in telecommunications, however, adding redundancy reduces the throughput of the system, and as a result, operator actions will take longer. Therefore, an important design principle is that the degree of redundancy should be adaptively matched to the amount of “noise” in the channel. In the case of turbulence, sun glare, or ambient sound, this could be accomplished using environmental sensors and models of the effects of these phenomena on the operator. In the context of interactive cockpits, self-checking interactive components have been proposed that migrate checking from the flight crew to a software component – a study of this system showed that dependability increased without degrading operator throughput [20].

4.4 Fault Forecasting

Predictive models are another type of fault forecasting that is critical to some of the strategies described above. These models provide a prediction of the likelihood that a physical phenomenon such as noise, sunlight, or turbulence will affect operator actions or perception; these predictions are critical because of the possibility of adapting the system to the magnitude or severity of the current phenomena. In addition, if techniques such as adding redundancy are used (see Sect. 4.2), models can help avoid situations where the system asks for more confirmation or repetition than is required by the environmental conditions. As an example, recruitment procedures of operators aim at detecting operators’ capabilities in order to reduce likelihood of failures [45].

Experimental psychology is also an evolving field and previous knowledge can be overturned by new studies. For example, Wason [46] describes a study where a large majority were not able to deduce information correctly (philosophyexperiments.com//wason). A more recent study [47] showed that the abstract nature of the task was the limiting factor; concrete presentation of the same information removed the difficulty.

5 Discussion and Conclusion

We analysed our taxonomy in terms of the requirements identified by Hansman [54]: acceptability, completeness, standard terminology, determinism, mutual exclusiveness, repeatability, and unambiguity. We meet several of these requirements by using well-accepted foundations (i.e., the Avizienis framework and the standard HCI model of the human information processor); this improves acceptability, facilitates completeness (although further sub-divisions are possible), uses standard terminology that is familiar to researchers, and provides clear classification structures (determinism). We partially meet the requirements of mutual exclusiveness, repeatability, and unambiguity: the divisions in the taxonomy are clearly separated, but because many tasks involve multiple human capabilities, a phenomenon could affect multiple categories (e.g., turbulence can affect both perception and motor action); therefore, users of the taxonomy will need to separately consider effects on different human capabilities. In addition, it is often difficult to ascertain people’s internal states (i.e., deliberateness and maliciousness may not be knowable). Finally, the usefulness of the taxonomy is in providing new ways to think about operator faults, which can lead to better analysis of incidents and improved designs. However, usefulness must be further determined as the taxonomy is used by the research and practitioner communities.

Although several taxonomies exist that cover different aspects of operator failures, these have not comprehensively explored the many ways in which operators fail while interacting with an interactive human-computer system. We expanded on Avizienis et al.’s fault taxonomy [6] to better characterize and explain operator faults, focusing on internal and external faults that induce error states inside the operator. Our new taxonomy explicitly recognizes that operators can be induced into error states, and separates out faults in the operator’s perceptual, cognitive, and motor subsystems. These additions provided 24 types of operator faults that expand on the coverage of previous taxonomies. The framework highlights the fact that the some research contributions are able to address one type of fault (e.g., stabilizing touch interaction by bracing the hand on the display [48]) while triggering another type of fault (e.g., bracing with the hand on the display may cause other faults if the hand occludes the display content).

Our work provides new opportunities for future research. First, we will refine and validate the taxonomy by classifying existing incidents in consultation with domain experts and practitioners. Second, we will develop new adaptive fault-removal techniques for different environmental conditions such as ambient noise, vibration, and glare. Third, we will look more deeply into some of the fault categories by developing formal models of the operator’s actions in the HCIL, and will further develop the idea of human-system interaction as communication in a noisy channel that can be improved through redundancy. Overall, our new taxonomy provides researchers and designers with a broad understanding of how and where operator faults can arise, and can improve the design of new interactive systems in complex environments. Our classification is able to integrate previous work in multiple domains such as medicine, psychology, and HCI, all of which contribute to the dependability of interactive systems.