Keywords

1 Introduction

Due to an exponentially growing increase in hardware performance, qualitative new technical capabilities have emerged [1]. This is because the performance enhancement makes it possible for technology to process a huge amount of data in a self-learning manner. With this - and this is a very important qualitative leap - the problem referred to as Polanyi’s paradox can be overcome [2]. Polanyi’s paradox states that people know more than they can say. What people can express is their explicable knowledge but not their tacit knowledge. Since in conventional automation technology has to be programmed by humans using rules (or algorithms), the non-explicable tacit knowledge is a barrier to what can be automated. As a consequence, the emerging technical ability to learn by itself, allows to automate tasks that were not automatable before. In terms of a qualitative leap, two new technical capabilities result: Pattern recognition and autonomy.

Pattern Recognition.

Pattern recognition through machine learning is a substantial advantage with regard to decision support systems [1]. Conventional expert systems (e.g. the general problem solver) were based on programmed rules. On the one hand, however, they turned out to be very error-prone and, on the other hand, their generalizability, i.e. their applicability to other situations, is very limited. The reason for this is that the human experts who provide the rules for conventional expert systems possess a great deal of tacit knowledge that is not transferred into explicit rules. In contrast, data-based pattern-recognition by machine learning allows for decision support systems that recognize rules themselves and hence are independent from explicable human knowledge. However, a new problem arises out of this capability, which is a kind of a reversed Polanyi’s paradox: By self-learning, the technology for its part builds knowledge, which it can no longer communicate to humans (cf. below: explainable artificial intelligence).

Autonomy.

Machine learning is also a key technology for autonomous systems. A system is autonomous if it can achieve a given goal independently and adapted to its situation without human intervention and without detailed programming for a situation [3]. Such systems are capable of mastering complex data processing and automatic object identification right up to the creation of a digital representation of reality that is sufficiently accurate for successfully pursuing objectives in dynamic contexts [2]. This enables self-driving cars, for example. The benefit is not just in the automation. Rather, autonomous systems may also be resource-saving. If cars drive autonomously, i.e. if they are not dependent on a person driving them, it makes no sense to leave them standing around in parking lots waiting for human drivers. The use of mobility services will then be more economical than owning a car, which will reduce the number of vehicles required [3].

Human-Technology Relation.

With pattern recognition and autonomy as two new technical capabilities, the relation between humans and machines changes. For decades Fitt’s maba-maba (“men-are-better-at”, “machines-are-better-at”) lists [4] validly described qualitative differences of humans and machines. As a main difference the human’s ability to cope with ill-defined problems in contrast to the machine’s superiority in handling well-defined problems was emphasized. On the basis of this difference, concepts for human-machine function allocation were developed that considered humans and machines as complementary (e.g. [5]). The main assumption of complementarity was that humans and machines have different strengths and weaknesses and may - if smartly combined - foster each others’ strengths and compensate for each others’ weaknesses [6]. With the capability of pattern recognition and autonomy technology has become able to handle ill-defined problems as well. As a consequence, the assumption of human machine complementarity is questioned.

This paper reflects on this question. It takes the position that despite powerful technical capabilities to handle ill-defined problems, humans and machines are still qualitatively different and complementary. Furthermore, it questions approaches that conceptualize levels of technical autonomy and suggests that rather levels of progressive intensity of human-technology teaming need to be elaborated.

2 Limits of Human-Technology Bi-polarity Thinking

It is very common to operationalize levels of human-machine interaction as levels of increasing automation. Sheridan [7] presented a scale with manual process control (The computer offers no assistance: The human does it all” (p. 207)) on the one side of the dimension and full automation (“The computer selects, acts, and ignores the human” (p. 207)) on the other side. Today, corresponding concepts specify levels of technical autonomy ranging from no autonomy to full autonomy (e.g. [8]). Concepts like these are bi-polar with human control decreasing continually from full control to no control. From a human factors perspective there are at least two major challenges associated with it.

First, levels of technical semi-autonomy may create unaccomplishable tasks for humans. The main problem is that humans need to monitor technical performance [9], which at least causes problems like monotony and fatigue. Even worse, humans may lack capabilities when required to supervise a technology that was designed to act quicker and take more factors into account than humans are able to (Bainbridge’s Ironies of Automation [10]). Other negative effects of automation on humans are described elsewhere (e.g. [11, 12]). These include for example over-confidence and under-confidence in technology as well as misjudgment of process states, inadequate situation awareness, demotivation or loss of skills and experiences as a result of automation.

Second, it is fundamentally questioned whether full technical autonomy is possible at all [13,14,15]. Main arguments are twofold: (i) autonomy is not a property of a technology but rather a capability emerging in concrete situations by the interaction of technology, task and specific circumstances. Hence, the same technology may act autonomously in certain situations and fail in others. (ii) In the real world there are no isolated tasks. Rather, all activities need to be coordinated with those of other actors, be this technical devices or humans. Hence, autonomy, i.e. the ability to perform a task alone, but lacking the ability to coordinate with others, is not enough. Therefore, Bradshaw et al. conclude that rather than working on increased technical autonomy, solutions regarding human-technology teaming need to be elaborated [14].

The following section reflects on human-machine complementarity considering new technical capabilities as described above (i.e. pattern recognition and autonomy allowing the machine to handle ill-defined problems independent from humans) in order to better understand what human-technology teaming might look like.

3 Humans and Technology Still Complementary

Although machine learning and autonomy allows technologies to invade the human domain of coping with ill-defined problems, the emerging technical capabilities still are bounded. Some of these restrictions may be due to current technical immaturity and hence may improve in future. However, others are more fundamental, reflecting cardinal differences between humans and machines.

Mitchell describes shortcomings of machine learning algorithms [1]. For neuronal network based learning, these are mainly biases in decision models the algorithm learns from biased training data or by overfitting to training data sets. Furthermore, decision models are impenetrable for humans and hence create a “dark secret”. For reinforcement learning algorithms she mainly questions the ecological validity of the algorithm’s learnings. This because such learning normally does not happen in real world settings but in simulations. Furthermore, in complex real-world settings it is not always clear what a success is and accordingly what actually should be reinforced.

Technical autonomy as well shows numerous shortcomings. Russel for example points to the fact, that autonomous systems aim at achieving a programmed objective without embedding corresponding actions into a greater value system [16]. Chen argues that such embedding is not computable, since compliance to legislation and ethics is always based on social negotiation processes that are and need to be impugnable [17]. Other authors question whether technical systems are able to take decisions at all since decision-making always requires sense-making [1, 13, 18].

Of course, humans are not perfect either. “To err is human” as already was known in the ancient world. However, it seems that a cardinal difference between humans and machines is human understanding and sense-making. This requires common sense, background knowledge, theories of mind, and the like (e.g. [1]). For clinical decision support systems (CDS) for example, technically generated decision suggestion usually require an interpretation that takes the context into account [19]. Thereby not only contextual knowledge of human experts needs to be integrated into decision-making, but also, for example, the needs of the patients concerned. Already Floridi has pointed out that machines can calculate much better than humans. Humans, on the other hand, can think [20]. Calculating and thinking are not the same, however.

It can be concluded that humans and machines despite new technical capabilities as described above, are still qualitatively different and complementary. Based on this assumption, the following section conceptualizes different intensity levels regarding human-technology teaming.

4 Progressive Intensity of Human-Technology Teaming

The scale of different human-technology teaming levels presented here, reflects the qualitative differences of humans and machines as just described. Hence, it is based on the assumption that humans and machines show qualitative different strengths and weaknesses. Furthermore, it adapts a complementary design approach [6] and thus aims for a human-technology combination allowing for mutual compensation of weaknesses as well as for mutual reinforcement of strengths.

Consequently, the proposed scale does not range from human control to full automation or from no technical autonomy to full technical autonomy. Scales like these conceptualize a bi-polarity between humans and machines, which are rather competing regarding control than collaborating. In contrast, for describing human-machine teaming we suggest a scale with levels of an increasingly intense human-machine collaboration. A scale like that is rather orthogonal to bi-polar conceptualizations of increasing technical autonomy.

Basically, it encompasses two aspects. On the one hand, with regard to the relation of humans and machines, a distinction is made between “automate” and “informate” [21]. While the former aims at replacing humans by technology, the latter aims at complementing humans with technology. On the other hand, mutual complementation of humans and technology can be increasingly integrated. These two aspects result in the following levels of increasing collaboration intensity in human-machine teaming (cf. [22]):

  • Level 0: Automate: Technology works independently from humans.

  • Level 1: Informate: Technology supports people in making better decisions by providing information for better decisions.

  • Level 2: Interact: People and technology influence each other, for example by learning from each other.

  • Level 3: Collaborate: People and technology work together synergistically by coordinating each other and making decisions together.

Following, these levels are outlined in more detail.

Level 0: Automate.

Since automation aims at replacing human skills and human effort by technology [21], there is no human machine interaction required - at least in theory. In practice, however, no technology is fully independent from humans (yet). If there are no humans required on the operational level, then at least on the levels of maintenance or engineering [23]. Hence, automation is level 0 on our scale since human-machine teaming is not really envisioned. Human factors related problems arising from this level have been mentioned above.

Level 1: Informate.

On this level, people set goals and are supported by technology in achieving them. Consequently, technology is applied to increase human performance, which is also known as augmented intelligence or augmented cognition [24, 25]. With the support of technology, humans should be able to make more precise decisions [26] or be supported with regard to their susceptibility to errors [27]. Prerequisite for augmenting the humans’ cognitive performance is that people can understand and interpret technically generated suggestions. Consequently, the need for explainable artificial intelligence (XAI) is postulated [28]. However, final decision-making and hence taking of responsibility is with the human.

Kirste points out that interaction of humans and technology requires that technology is accepted by humans [25]. The prerequisite for this is confidence in technology. However, confidence should not be a blind one, but rather develops through understanding and explainability. When people, as a result of overconfidence, accept suggestions from decision support systems without checking them or looking for additional information automation complacency results [11]. It is therefore recommended to avoid features in the design of technology that makes it appear more competent or more convincing, as this could lead to overconfidence [29].

Level 2: Interact.

This level includes level 1. In addition, technology not only provides humans with understandable information, but humans and technology actively influence each other, e.g. by mutual learning [28]. Kirste describes specific ways of mutual learning [25]. He differentiates between (a) interactive learning, in which people can influence the machine learning’s model formation before or after the learning phase, (b) visual analytics, in which large amounts of data are visualized using methods of machine learning in such a way that people can gain knowledge, and (c) dimension reduction, in which humans reduce the parameters that the technology has identified in a self-learning manner. Also, Schmid and Finzel describe a form of interactive, mutual learning by combining sub-symbolic black box methods of machine learning with symbolic white box methods that are understandable to humans [30]. This allows the humans to continually influence the technology.

On the other hand, technology can explicitly influence humans and hence support human learning if it communicates proactively with people. It may for example provide humans with information even if they are not (yet) looking for it [31]. This may also mean that the machines need to be able to understand the humans’ intentions and needs [32, 33].

Level 3: Collaborate.

This level represents full human technology-teaming. Based on their complementary differences, humans and machines work together in a synergetic way in order to achieve common objectives. Doing so, they coordinate their activities and take decisions together. Both are (still) utopian. Regarding mutual coordination of activities, on the one hand Bradshaw et al. assume that technology is not (yet) able to coordinate its actions with humans [14]. However, recent insights in human-robot interaction may contribute to further development (e.g. [34]).

Joint decision-making on the other hand requires joint responsibility-taking. Corresponding concepts are still to be developed. Chen suggests a concept with checks and balances on different decision-making levels. Thereby he differentiates between frontend and backend processes [17]. At the frontend there are functions of execution, where speed and accuracy are important (e.g. a self-driving car taking an emergency break). This is why the machines should be in the lead here. However, the machine acts according to rules defined by the human in backend processes, which encompass legal, ethical, and regulatory functions. Schmidt and Herrmann suggest an “intervention user interface” that allows the human to intervene into the autonomous processes at any time [35]. These interventions allow the human to change the predefined course of a process and have the following three characteristics: (a) interventions are unplanned and occur exceptionally, (b) interventions can be initialized quickly and have an immediate effect, and (c) interventions have a feedback function and help to improve the autonomous behavior of the technical system.

However, despite progress regarding mutual coordination and joint decision-making, the problem of human-machine collaboration on level 3 is not (yet) fully solved. Especially regarding decisions that need to be taken quickly at the frontend, humans and machines rather are taking partial decisions separately. Consequently, there is a partial loss of control for humans. According to Grote, systems should be designed in such a way that such loss is kept as small as possible and made transparent to the human [23]. Waefler suggests that human loss of control might be compensated by social resilience. However, corresponding sociotechnical system design concepts are still to be developed [22].

5 Conclusion

This paper starts with a description of new technical capabilities, which mainly are pattern recognition in complex data sets and autonomy. It argues that these capabilities allow technology to take over activities that hitherto belonged to the human domain. More specifically, this refers mainly to the human capability to cope with ill-defined problems. As technology invades this human capability, the question is how to best combine humans and technology. Corresponding conceptualizations suggest a bi-polar dimension of increasing technical autonomy and hence of decreasing human control. Such, humans and machines are considered competitors rather than team players. As a consequence, this approach is not really suitable for conceptualizing human-technology teaming.

Instead, we suggest to take an orthogonal approach to better understand true human-technology teaming. Rather than competing for control, humans and machines are envisioned to progressively intensify collaboration. This is based on the assumption that humans and technology remain complementary although technology shows new capabilities that used to be exclusively part of the human domain. Significant remaining differences of humans and machine are for example sense-making, understanding but also negotiating where there are no clear solutions.

We suggest three levels of progressive intensity in human-technology teaming: (1) informate, (2) interact, and (3) collaborate. On levels 1 and 2 promising results can be found in literature (e.g. XAI or mutual learning). However, level 3 - i.e. on the level of most intensive human-technology teaming - still requires research especially regarding possibilities to compensate for loss of human control.