Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The modern use of the term automation can be traced to a 1952 Scientific American article; today it is widely employed to define the interaction of humans with machines. Automation (machines) may be electrical, mechanical, require interaction with computers, involve informatics variables, or possible relate to parameters in the environment. As noted [17.1], the first actual automation was the mechanization of manual labor during the Industrial Revolution. As machines became increasingly useful in reducing the drudgery and danger of manual labor tasks, questions begin to arise concerning how best to proportion tasks between humans and machines. Present applications still address the delineation of tasks between humans and machines [17.2], where, e.g., in chemistry and laboratory tasks, the rule of thumb is to use automation to eliminate much of the 3-D tasks (dull, dirty, and dangerous). In an effort to be more quantitative in the allocation of work and responsibility between humans and machines, Fitts [17.3] proposed a list to identify tasks that are better performed by humans or machines (Table 17.1).

Table 17.1 Fittsʼ list [17.3]
Full size table

This list initially raised some concern that humans and machines were being considered equivalent in some sense and that human could easily be replaced by a mechanical counterpart. However, the important point that Fitts raised was that we should consider some proper allocation of tasks between humans and machines (functional allocation [17.4,5]) which may be very specific to the skill sets of the human and those the machine may possess. This task-sharing concept has been discussed by numerous authors [17.6]. Ideas of this type have nowadays been generalized into how humans interact with computers, the Internet, and a host of other modern apparatus. It should be clarified, however, that present-day thinking has now moved away from this early list concept [17.7].

Some Basics of Human Interaction with Automation

In an attempt to be more objective in delineating the interaction of humans with machines, Parasuraman et al. [17.8] defined a simple four-stage model of human information processing interacting with computers with the various levels of automation possible delineated in Table 17.2. Note how the various degrees of automation affect decision and action selection. This list differs from other lists generated, e.g., using the concept of supervisory control [17.9, p. 26] or on a scale of degrees of automation [17.1, p. 62], but are closely related. As the human gradually allocates more work and responsibility to the machine, the humanʼs role then becomes as the supervisor of the mechanical process. Thus the level of automation selected in Table 17.2 also defines the roles and duties of the human acting in the position of supervisor. With these basics in mind, it is appropriate to examine a brief sample of modern application areas involving humans dealing with automation. After these applications are discussed, the current and most pertinent issues concerning how humans interact with automation will be brought to light.

Table 17.2 Levels of automation of decision and action selection by the computer
Full size table

Various Application Areas

Besides the early work by Fitts, automation with humans was also viewed as a topic of concern in the automatic control literature. In the early 1960s, Grabbe et al. [17.10] viewed the human operator as a component of an electrical servomechanism system. Many advantages were discovered at that time in terms of replacing some human function via automated means; for example, a machine does not have the same temperature requirements as humans, performance advantages may result (improved speed, strength, information processing, power, etc.), the economics of operation are significantly different, fatigue is not an issue, and the accuracy and repeatability of a response may have reduced variability [17.1, p. 163]. In more recent applications (for example, [17.11]) the issues of automation and humans extend into the realm of controlling multiple unmanned air vehicle systems. Such complex (unmanned) systems have the additional advantage that the aircraft does not need a life-support system (absence of oxygen supply, temperature, pressure control or even the requirement for a transparent windshield) if no humans are onboard. Hence the overall aircraft has lower weight, less expense, and improved reliability. Also there are political advantages in the situation of the aircraft being shot down. In this case, there would not be people involved in possible hostage situations. The mitigation of the political cost nowadays is so important that modern military systems see significant advantages in becoming increasingly autonomous (lacking a human onboard).

Concurrent with military applications is the desire to study automation in air-traffic control and issues in cockpit automation [17.12], which have been topics of wide interest [17.13,14,15]. In air-traffic control, software can help predict an aircraftʼs future position, including wind data, and help reduce near collisions. This form of predictive assistance has to be accepted by the air-traffic controller and should have low levels of uncertainty. Billings [17.13] lists seven principles of human-centered automation in the application domain of air-traffic control. The human-centered (user-centered) design is widely popularized as the proper approach to integrate humans with machines [17.14,16,17]. Automation also extends to the office and other workspace situations [17.18] and to every venue where people have to perform jobs.

Modern applications are also implicitly related to the revolution in information technology (IT) [17.19] noting that Fittsʼ list was preceded by Watsonʼs (IBMʼs founder in the 1930s) concept that Machines should work. People should think. In the early stages of IT, Licklider envisioned (from the iron, mainframe computer) the potential for man–computer symbiosis [17.20], interacting directly with the brain. IT concepts also prevail in the military. Rouse and Boff [17.21] equate military success via the IT analogies whereas bits and bytes have become strategically equated with bombs and bullets and advances in networks and communications technologies which have dramatically changed the nature of conflict. A brief list of some current applications are now reviewed to give a flavor of some modern areas that are presently wrestling with future issues of human interaction with automation.

Agriculture Applications

Agriculture applications have been ubiquitous for hundreds of years as humans have interacted with various mechanical devices to improve the production of food [17.22,23]. The advent of modern farm equipment has been fundamental to reducing some of the drudgery in this occupation. The term shared control occurs in situations where the display of the level of automation is rendered [17.9]. In related fields such as fish farming (aquaculture), there are mechanized devices that significantly improve production efficiency. Almost completely automated systems that segment fish, measure them, and process them for the food industry have been described [17.24]. In all cases examples prevail on humans dealing with various levels of automation to improve the quality and quantity of agriculture goods produced. Chapter 63 discusses in more detail automation in agriculture.

Communications Applications

The cellphone and other mobile remote devices have significantly changed how people deal with communications in todayʼs world. The concept of Bluetooth [17.25] explores an important means of obviating some of the problems induced when humans have to deal with these communication devices. The Bluetooth idea is that the system will recognize the user when in reasonably close proximity and readjust its settings and parameters so as to yield seamless integration to the specific human operator in question. This helps mitigate the human–automation interaction problems by programming the device to be tailored to the userʼs specifications. Other mobile devices include remote controls that are associated with all types of new household and other communication devices [17.26]. Chapter 13 discusses in more detail communication in automation including networking and wireless.

Inspection Systems Applications

In [17.27], they discuss 100% automated inspection systems rather than having humans sample a process, e.g., in an assembly-line task. Prevailing thinking is that humans still outperform machines in most attribute-inspection tasks. In the cited evaluation, three types of inspection systems were considered, with various levels of automation and interaction with the human operator. For vision tasks [17.28] pattern irregularity is key to identification of a vector of features that may be untoward in the inspection process.

Manufacturing Applications

Applications in automation are well known to be complex [17.29], such as in factories where production and manufacturing provide a venue to study these issues. In [17.30] a human-centered approach is presented for several new technologies. Some of the negative issues of the use of automation discussed are increased need for worker training, concerns of reliability, maintenance, and upgrades of software issues, etc. In [17.31], manufacturing issues are discussed within the concept of the Internet and how distributed manufacturing can be controlled and managed via this paradigm. More and more modern systems are viewed within this framework of a machine with which the human has to deal by interfacing via a computer terminal connected to a network involving a number of distributed users. Chapters 49, 50, 51 presents different aspects of manufacturing automation.

Medical and Diagnostic Applications

Automation in medicine is pervasive. In the area of anesthesiology, it is analogous to piloting a commercial aircraft, (“hours of boredom interspersed by moments of terror” [17.32]). Medical applications (both treatment and diagnosis) now abound where the care-giver may have to be remote from the actual site [17.33]. An important example occurs in modern robotic heart surgery where it is now only required to make two small incisions in the patientʼs chest for the robotic end-effectors. This minimally invasive insult to the patient results in reduced recovery time of less than 1 week compared with typically 7 weeks for open heart surgery without using the robotic system. As noted by the surgeon, the greatest advantage gained is [17.34]:

Without the robot, we must make a large chest incision. The only practical reason for this action is because of the `size of the surgeonʼs handsʼ. However, using the robot now obviates the need for the large chest incision.

The accompanying reduction of medical expenses, decreased risk of infection, and faster recovery time are important advantages gained. The automation in this case is the robotic system. Again, as with Fittsʼ list, certain tasks of the surgery should be delegated to the robot, yet the other critically (medically) important tasks must be under the control and responsibility of the doctor. From a diagnostics perspective, the concept of remote diagnostic methods are explored [17.35]. As in the medical application, the operator (supervisor) must have an improved sense of presence about an environment remote from his immediate viewpoint and has to deal with a number of reduced control actions. Also, it is necessary to attempt to monitor and remotely diagnose situations when full information may not be typically available. Thus automation may have the disadvantage of limiting the quality of information received by the operator. Chapters 77, 78, 80, 82 provide more insights into automation in medical and healthcare systems.

Robotic Applications

Robotic devices, when they interact with humans, provide a rich venue involving problems of safety, task delegation, authority, and a host of other issues. In [17.36] an application is discussed which stresses the concept of learning from humans. This involves teaching the robotic device certain motion trajectories that emulate successful applications gleaned from humans performing similar tasks. In robotic training, it is common to use these teach pendant paradigms, in which the robotic path trajectory is stored in a computer after having a human move the end-effector through an appropriate motion field environment. In [17.37], the term biomimicking motion is commonly employed for service robots that directly interact with humans. Such assistive aids are commonly used, for example, in a hospital setting, where a robotic helper will facilitate transfer of patients, removing this task from nurses or other care-givers who are normally burdened with this responsibility. Chapter 21 provides an insightful discussion on industrial robots.

Teaching Applications

In recent years, there has been an explosive growth of teaching at universities involving long-distance learning classes. Many colleges and professional organizations now offer online courses with the advantage to the student of having the freedom to take classes at any time during the day, mitigating conflicts, travel, and other costs [17.38]. The subject areas abound, including pharmacy [17.39], software engineering [17.40], and for students from industry in various fields [17.41]. The problem of humans interacting with automation that occurs is when the student has to take a class and deal directly with a computer display and a possibly not so user-friendly web-based system. Having the ability to interrelate (student with professor) has now changed and there is a tradeoff space that occurs between the convenience of not having to attend class versus the loss of ability to fully interact, such as in a classroom setting. In [17.42], a variety of multimedia methods are examined to understand and help obviate the loss of full interactive ability that occurs in the classroom. The issue pertinent to this example is that taking the course online (a form of automation) has numerous advantages but incurs a cost of introducing certain constraints into the human–computer interaction.

These examples represent a small sample of present interactions of humans with automation. Derived from these and other applications, projections for some present and future issues that are currently of concern will be discussed in further detail in the next section.

Modern Key Issues to Consider as Humans Interact with Automation

Trust in Automation

Historically, as automation became more prevalent, concern was initially raised about the changing roles of human operators and automation. Muir [17.43] performed a literature review on trust in automated systems since there were alternative theories of trust at that time. More modern thinking clarifies these issues via a definition [17.44]:

Automation is often problematic because people fail to rely upon it appropriately. Because people respond to technology socially, trust influences reliance on automation. In particular, trust guides reliance when complexity and unanticipated situations make a complete understanding of the automation impractical. People tend to rely on automation they trust and tend to reject automation they do not.

Taking levels of trust by pilots as an example, it was found [17.45] that, when trust in the automated system is too low and an alarm is presented, pilots spend extra time verifying the problem or will ignore the alarm entirely. These monitoring problems are found in systems with a high propensity for false alarms, which leads to a reduced level of trust in the automation [17.46]. From the other extreme, if too much trust is placed on the automation, a false sense of security results and other forms of data are discounted, much to the peril of the pilot. From a trust perspective, Wickens and colleagues tested heterogeneous and homogeneous crews based on flight experience [17.47] and found little difference in flying proficiency for various levels of automation. However, the homogeneous crews obtained increased benefit from automation, which may be due to, and interpreted in terms of, having a different authority gradient. In considering the level of trust (overtrust or undertrust), Lee and Moray [17.48] later noted that, as more and more false alarms occur, the operator will decrease their level of trust accordingly even if the automation is adapted. It is as if the decision-aiding system must first prove itself before trust is developed. In more recent work [17.49], a quantitative approach to the trust issue is discussed, showing that overall trust in a system has a significantly inverse relationship with the uncertainty of a system. Three levels of uncertainty were examined using National Institute of Standards (NIST) guidelines, and users rated their trust at each level through questionnaires. The bottom line was that performance of a hybrid system can be improved by decreasing uncertainty.

Cost of Automation

As mentioned previously, cost saving is one of the significant advantages of the use of automation, e.g., in an unmanned air vehicle, for which the removal of the requirement to have a life-support system makes such devices economically advantageous over alternatives. In [17.50] low-cost/cost-effective automation is discussed. By delegating some of the task responsibilities to the instrumentation, the overall system has an improved response. In a cost-effective sense, this is a better design. In [17.51] the application of a fuzzy-logic adaptive Kalman filter which has the ability to provide improved real-time control is discussed. This allows more intelligence at the sensor level and unloads the control requirements at the operator level. The costs of such devices thus drops significantly and they become easier to operate. See Chap. 41 for more details on cost-oriented automation.

Adaptive Versus Nonadaptive Automation

One could argue that, if automation could adapt, it could optimally couple the level of operator engagement to the level of mechanism [17.52]. To implement such an approach, one may engage biopyschometric measures for when to trigger the level of automation. This would seem to make the level of automation related to the workload on the human operator. One can view this concept as adaptive aiding [17.53]. For workload consideration, the well-known National Aeronautics and Space Administration (NASA) task load index (TLX) scale provides some of these objective measures of workload [17.54]. As a more recent example, in [17.55], adaptive function allocation as the dynamic control of tasks which involves responsibility and authority shifts between humans and machines is addressed. How this affects operator performance and workload is analyzed. In [17.56] the emphasis focuses on human-centered design for adaptive automation. The human is viewed as an active information processor in complex system control loops and supports situational awareness and effective performance. The issues of workload overload and situational awareness are researched.

Safety in Automation

Safety is a two-edged sword in the interaction of humans with automation. The human public demands safety in automation. As mentioned in regard to robotic surgery applications, the patient has a significantly reduced recovery time with the use of a robotic device, however, all such machines may fail at some time. When humans interact with robotic devices, they are always at risk and there are a number of documented cases in which humans have been killed by being close to a robotic device. In factories, safety when humans interact with machines has always been a key concern. In [17.57], a way to approach the safety assessment in a factory in terms of a safety matrix is introduced. This differs from the typical probability method. The elements of the matrix can be integrated to provide a quantitative safety scale. For traffic safety [17.58] human-centered automation is a key component to a successful system and is recommended to be multilayered. The prevailing philosophy is that it is acceptable for a human to give up some authority in return for a reduction in some mundane drudgery.

Remote control presents an excellent case for safety and the benefit of automation. Keeping the human out of harmʼs way through teleoperation and other means but providing a machine interface allows many more human interactions with external environments which may be radioactive, chemically adverse or have other dangers. See Chap. 39 for a detailed discussion on safety warnings for automation.

Authority in Automation

The problem of trading off authority between the human and computer is unsettling and risky, for example, in a transportation system, such as a car. Events that are unplanned or produce unexpectancy may degrade performance. Familiarity with antibrake systems can be a lifesaver for the novice driver first experiencing a skidding event on an icy or wet road. However, this could also work to the detriment of the driver in other situations. As mentioned earlier, automation has a tradeoff space with respect to authority. In Table 17.2 it is noted that, the higher the level of automation, the greater the loss of authority. With more automation, the effort from the operator is proportionally reduced; however, with loss of authority, the risk of a catastrophic disaster increases. This tradeoff space is always under debate. In [17.59] it is shown that some flexibility can be maintained. Applications where the tradeoff space exists between automation and authority include aircraft, nuclear power plants, etc., Billings [17.13] calls for the operator to maintain an active role in a systemʼs operation regardless of whether the automation might be able to perform the particular function in question better than the operator. Norman [17.60] stresses the need for feedback to be provided to the operator, and this has been overlooked in many recent systems.

Performance of Automation Systems

There are many ways to evaluate human–machine system performance. Based on the signal detection theory framework, various methods have been introduced [17.61,62,63]. The concept of likelihood displays shows certain types of statistical optimality in reducing type 1 and type 2 errors and provides a powerful approach to measure the efficacy of any human–machine interaction. Other popular methods to quantify human–machine performance include the information-theoretic models [17.64] with measures such as baud rate, reaction time, accuracy, etc. Performance measurement is always complex since it is well known that three human attributes interact [17.65] to produce a desired result. The requisite attributes include skill, and rule- and knowledge-based behaviors of the human operator.

When Should the Human Override the Automation?

One way to deal with the adaptive nature of automation is to consider when the human should intervene [17.66]. It is well known that automation may not work to the advantage of the operator in certain situations [17.67]. Rules can be obtained, e.g., it is well known that it is easier to modify the automation rather than modify the human. A case in point is when high-workload situations may require the automation level to increase to alleviate some of the stress accumulated by the human in a critical task [17.68] in situations such as flight-deck management.

Social Issues and Automation

There are a number of issues of the use of automation and social acceptance. In [17.69] the discussion centers on robots that are socially acceptable. There must be a balance between a design which is human-centered versus the alternative of being more socially acceptable. Tradeoff spaces exist in which these designs have to be evaluated as to their potential efficacy versus agreement in the venue in which they were designed. Also humans are hedonistic in their actions, that is, they tend to favor decisions and actions that benefit themselves or their favored parochial classes. Machines, on the other hand, may not have to deal with these biases. Another major point deals with the disadvantages of automation in terms of replacing human workers. If a human is replaced, alienation may result [17.70], resulting in a disservice to society. Not only do people become unemployed, but they also suffer from loss of identity and reduced self-esteem. Also alienation allows people to abandon responsibility for their own actions, which can lead to reckless behavior. As evidence of this effect, the Luddites in 19th century England smashed the knitting machines that had destroyed their livelihood. Turning anger against an automation object offers, at best, only temporary relief of a long-term problem.

Future Directions of Defining Human–Machine Interactions

It is seen that the following parameters strongly affect how the level of automation should be modified with respect to humans:

  • Trust

  • Social acceptance

  • Authority

  • Safety

  • Possible unplanned unexpectancy with adaptive automation

  • Application-specific performance that may be gained

Conclusions

Modern application still wrestle with the benefit and degree of automation that may be appropriate for a task [17.71]. Emerging trends include how to define those jobs that are dangerous, mundane, and simple where there are benefits of automation. The challenges include defining a multidimensional tradeoff space that must take into consideration what would work best in a mission-effectiveness sense as humans have to deal with constantly changing machines that obviate some of the danger and drudgery of certain jobs. For further discussion of the human role in automation refer to the chapters on Human Factors in Automation Design Chap. 25 and Integrated Human and Automation Systems, Including Automation Usability, Human Interaction and Work Design in (Semi)-Automated Systems Chap. 34 later in this Handbook.