1 Introduction

Within the next few decades, autonomous machines will enter our lives not as passive devices but as autonomous agents in unprecedented numbers and roles (Ford 2015; Berg 2016; Bloem et al. 2014; Boyle 2016; Brown 2016; Clifford 2017; Cookson 2016; Krzanowski et al. 2016; Pew Research Center 2014; Schwab 2016, 2017; Sullins 2011). The view that this technology will be all gain and no pain, supported by some,Footnote 1 is hardly justified by the historical record and current experience (see Heron and Belfort 2015). A more cautious approach, such as that suggested by Ford (2015), Gray (2007), Yampolskiy (2012a, b) and Kaczynski (1995), is preferable.

The question, then, is the following: How can we be sure that autonomous machines will not harm us and will behave as “ethical” agents?Footnote 2 To address this problem we need a test that would verify the autonomous machine’s aptitude to act in a safe way, i.e. in some sense to act ethically. We will call such a test the machine safety test (MST).Footnote 3

2 Key Terms

To avoid any misinterpretations, let us define certain key terms used in this chapter. These are ethics, ethical agent, machine ethics, autonomous machines and autonomous ethical agents.

Ethics, in the most general terms, is a set of prescriptions or rules about how to live a good and rewarding life as an individual and as a member of society (Bourke 2008; Vardy and Grosch 1999; MacIntyre 1998). Such a concept of ethics may be reduced, as is often the case, to a set of rules with a yes/no answer, specifying what to do (Beavers 2011). But ethics is more than just rules. It (implicitly or explicitly) requires free will, consciousness, a concept of good and bad, virtue and values, a concept of self, an understanding of responsibility (of “ought” and “ought not”) (MacIntyre 1998; Veach 1973; Sandel 2010) and a good comprehension of the reality around us. A lot of deep metaphysics (free will, a concept of good, a concept of self, etc.) is involved in the concept of ethics. Dispensing with metaphysics leaves ethical statements groundless.

Can then such a deep ethics be computed (in the Church–Turing sense), given that metaphysics is not mathematical? Ethical rule-based on Hobbesian, Kantian, utilitarian or other ethical schools can be to some extent translated into a computer algorithm and made “computable”. But then all “metaphysical” dimensions of the ethical actor are “lost in translation”. If a machine is programmed according to “translated” rules, one may claim that it possesses ethical qualities or that it is an ethical machine (Anderson and Anderson 2010). But this ethics would be a special type of ethics, not ethics in the deep, metaphysical sense. Ethics in a deep sense (like metaphysics) is non-computable, and we do not have any other meaning of “computable” that could rescue “computerized ethics” from its shallows (Yampolskiy 2012a, b; Krzanowski et al. 2016)Footnote 4 (see Turner 2016 for a definition of computation). The ethical rules translated into a machine format constitute what we will call machine ethics as m-ethics.Footnote 5

An ethical agent is an individual (artefact or natural) acting according to ethical rules. In the context of our discussion, we may call an autonomous machine an ethical agent understanding that we mean here ethics as m-ethics, or a set of behavioural rules directing the behaviour of an autonomous machine. Nothing else. Thus, it is misleading to talk about “moral machines”, “ethical machines” or the likes. Too generous use of these terms will only confuse the problems we face with autonomous machines (by attributing to them capacities they cannot have); we need to constantly remind ourselves that the subject of our discussion is autonomous machines with implemented m-ethical software or system.

Autonomous machines (e.g. Floreano et al. 1998; Patrick et al. 2008; Ni and Leug 2016) are machines that act in the environment without direct command by or involvement with humans. Autonomous machines, which implement m-ethics rules and thus display ethical-like behaviours, are autonomous ethical machines.

3 Where We Are with MST

So far we have only a few proposals of such a test. These are:

  1. 1.

    Moral Turing test (MTT)

  2. 2.

    Turing triage test (TTT)

  3. 3.

    Ethical competence test (ECT)

The moral Turing test (MTT) proposed by Allen et al. (2000) is similar to the “imitation game” proposed by Turing (1950). In the original Turing formulation of the “imitation game”, machine “intelligence” is assessed by a panel of judges based on a series of responses to questions posed to a machine and a human subject. If, based on the answers given, the judges cannot distinguish a machine from a human, then the machine has passed the test, but what exactly it means is open to interpretations as pointed out for example by Oppy and Dove (2016). The MTT would be run in a similar way, but with the key difference that the questions would be of moral import. Allen, Vermer and Zinser recognized the multifarious nature of ethical problems that would beset such a test and thus recognized its potential limitations. The problem with the MTT, however, may lie elsewhere. The Turing test has not lived up to its promise or its author’s intentions, and there is no consensus as to what the TT is actually testing or attempting to demonstrate (Turing 1950; Oppy and Dove 2016; Saygin et al. 2000). Thus, if the TT is not clear regarding its meaning,Footnote 6 on what grounds can it be extended to ethical problems with any expectation of success?

Another proposal to apply the TT to an ethical machine test is the Turing triage test (TTT) described by Sparrow (2004, 2014). In the test, the AI-based machine (robot, autonomous artificial agent), following an electrical shortage in a hospital, must choose between (1) turning itself off (equivalent to a human suicide in some respects) and saving a patient and (2) saving itself and allowing a patient to die. Sparrow claims that:

My thesis, then, is that machines will have achieved the moral status of persons when this second choice has the same character as the first one. That is, when it is a moral dilemma of roughly the same difficulty. For the second decision to be a dilemma it must be that there are good grounds for making it either way. It must be the case therefore that it is sometimes legitimate to choose to preserve the existence of the machine over the life of the human being. These two scenarios, along with the question of whether the second has the same character as the first, make up the Turing Triage Test (Sparrow 2004).

Sparrow himself suggests that a machine could never pass the TTT; thus, a machine could never achieve the moral status of a person. Apart from the fact that the TTT has little to do with the original TT (only that a test is applied to a computing machine), it is within that class of abstract ethical problems that includes the notorious trolley problem, terror bomber, strategic bomber or similar (keeping in mind the differences). The “imaginary ethical problems”(also known as thought experiments) have been devised not to provide a test of general ethical abilitiesFootnote 7 but to expose the multifarious nature of ethics. Thus, solving of the Turing triage problem will not translate itself into solving a problem of mariners stranded at sea (Sandel 2010); similarly, an ability to excel in strategic military games does not guarantee success in leading a real battle (as examples from history testify). Complex abstract ethical puzzles tell us little about the solutions to practical ethical problems (Dancy 1985; Szabó 2000; Elster 2011; Lehtonen 2012; Cathcart 2013).

A different proposal, not based on the Turing test, came from Moor (2006). Moor suggested the ethical competence test (ECP), which would assess the ethical aptitude of the computing agent using hypothetical situational scenarios (but not of the “imaginary ethical paradoxes” type). Such scenarios often do not have yes/no options, but instead use less or more favourable ones. The responses of an artificial agent to such scenarios would be compared with the choices made by humans in the comparable situations. Moor also introduced a requirement that the ethical robot provide a defensible and convincing justification for its decision. As he writes: “If a robot could give persuasive justifications for ethical decisions comparable to or better than those of good human ethical decision-makers, then the robot’s competence would be inductively established for that area of ethical decision-making” (Moor 2009). As he points out, ethical tests should be situation-dependent as an automated agent may be competent in some situations and yet not in others. In certain situations, Moor points out, computer agents, because of their huge information processing powers, can make better (and faster) decisions than human agents regarding, for example, the allocation of scarce resources or scheduling the delivery of supplies in the event of catastrophic situations to avoid waste. But these decisions would qualify rather as better or optimal managerial decisions, rather than as ethical ones per se. Moor’s proposal warrants attention as it acknowledges the complexity of ethical decisions, their dependence on situational context and the need to view an artificial agent not as a moral black box but, at the very least, as a grey one.

Summing up current efforts on testing of m-ethics, we can say that no conclusive proposal is on the table. We do not have a test that could serve as the MST of the general m-ethical capabilities of an artificial agent, nor do we have (implemented, proposed or conceptualized) a testing methodology to construct such a test. As well, we do not know what it would take to test an autonomous ethical agent for its ethical (m-ethical) probity.

4 Machine Safety Test: What It Should Be?

We take a safe approach to the MST (advocated by Yampolskiy 2012a, b; Moor 2009) and formulate three assumptions that underline the definition of the MST:

  1. 1.

    The inherent complexity of ethics renders it incomputable in the TM sense.

  2. 2.

    Computing machines cannot be ethical in the way people are.

  3. 3.

    Computing machines may play the ethical game. It just means that autonomous machines/agents may be made to act amicably towards us in all foreseeable situations and they should be safe to be around.

5 Claim: What We Are Testing

We need to ask: What are the objectives of the MST? Are we testing whether autonomous machines behave like humans or like ethical artificial agents with m-ethics capabilities? The first case is impossible to achieve considering our definition of m-ethics. The machine safety test is NOT the test verifying the general “moral” or ethical aptitude of a machine. It means that we do not test the machine’s “equivalence” to a human agent, and it also means that we do not test a moral aptitude in specific circumstances.

The machine safety test is not designed to prove that machines have reached the same level of moral standing as people or have reached the level of autonomy that endows them with “moral personality” and makes them responsible for what they do.Footnote 8 The objective of the test is only to verify that:

  1. 1.

    The product we develop, i.e. an autonomous agent, a machine, is “safe” to be around people in general circumstances.

  2. 2.

    The propensity of the autonomous machine to do harm, by “intention”, design or neglect, is limited to as narrow a margin as reasonably possible. However, it seems that the possibility of doing harm cannot be completely eliminated.

Of course, terms such as “general circumstances”, “safe” and “a narrow margin” can be interpreted in many ways. Here, I use common understandings of these terms, accepting that it may require further elaboration. It is possible that soon we will have to create a new dictionary of ethical terms that would correctly represent machine ethics, as current ethical terms may not be sufficient to describe the complex human–machine interactions.

6 What Should the Test Be?

It seems that the MST should not be theoretical or primarily theoretical (theoretical meaning involving only ethical reasoning verified in a dialogue or a conversation—free or structured). Theoretical questioning is not a good test of moral aptitude. As we know, behind the clever and reasonable answers there may be nothing of substance but software, nothing of substance in an ethical sense, as the experience with chat bots (and politicians) teaches us. And besides, what kinds of questions would we ask? Certainly, any type of standard personality test or tests that try to gauge a person’s sanity should be ruled out as the responses to them can be easily programmed and reproduced by a computing machine without even the smallest ethical insight. Imaginary ethical cases as proposed in the TTT are not suited to this function, as explained earlier. Yet, there may still be some use for the verbal verification of the moral standing. Such a verbal test in the form of an interview (qualifying conversation?) may be used to understand the ethical reasoning of a machine (correctness of the software implementation of ethical capacities?), and it may be more conducive to the purpose of the MST than a test requiring specific answers. The requirement that an autonomous ethical agent be able to explain itself stipulates a requirement that it be implemented as a white box. The problem as to how we would gauge the results of such an interview remains an open question.

The bulk of the MST test should be an ability to make just decisions in specific life situations. Making just decisions in a real-life context, not an abstract ability to assign “right” or “wrong” labels to abstract situations, seems to be at the core of ethics.Footnote 9 Of course, to act as an ethical agent in life situations an agent will have to possess considerable abstract knowledge of ethics. But we would rather require that an autonomous machine makes ethical choices in concrete situations rather than be able to respond to complex ethical questions or solve imaginary cases.

We may compare the MST to the skipper patent test or an airline pilot test ceteris paribus. These tests include theoretical and practical components. Learning or testing for a pilot or a skipper begins with theoretical tests of basic technical knowledge, before progressing through training on the flight simulators and then flights with an instructor. Finally, these tests also include a period of apprenticeship. In the case of an airline pilot, the pilot-to-be must fly in a junior position for a certain number of hours, before he or she can be recognized as a pilotFootnote 10 with the licence to undertake solo flights, likewise with a skipper permit. In the case of the MST, the tests are even more complex than for a specific job function, as they will be testing more general situations.

Thus, it seems that the proper test of the artificial moral agent should consist of a theoretical part, a series of practical problems of varying scope and difficulty, progressing from staged scenarios through to gradually less controlled situations and ending up with completely uncontrolled life situations, and (maybe) include a period of apprenticeship during which we verify an agent’s capacity to think and act morally in real-life situations (we would call such situations “open-ended”).

Thus, in summing up the discussion, the MST should include the following components:

  1. 1.

    Theoretical verification of ethical aptitudes and reasoning—possibly a qualifying interview rather than a Q&A session plus a white box option.

  2. 2.

    A situational test or series of tests, in which an artificial agent makes autonomous decisions in the fully life-like (controlled or not) environment. The tests may have a different scope and increasing complexity and include:

    • Staged tests

    • Controlled life situations

    • Open-ended situations

  3. 3.

    A period of apprenticeship in which an artificial agent acts in the real conditions under close supervision.

7 Use Case Framework for the MST

Due to the generality of the MST test, only the high-level framework of the UC could be provided. This would consist, in the proper sequence, of four stages:

  1. 1.

    Interview and discussion that would verify understanding of m-ethical rules and m-ethical reasoning using imaginary ethical cases. The tests should be performed under a white box paradigm; i.e. the tested system should be able to explain its decisions

  2. 2.

    Staged situational tests that would verify an ability of a tested system to respond to complex (arranged) situations. These tests may be similar to those used on human subjects such as Milgram experiment (Milgram 1963), Phone booth and Dime experiment (Doris 2002), Stanford Prison Experiment (2014) or Cornell experiment (Doris 2002).

  3. 3.

    Situational tests including controlled and open-ended life situations that would verify an ability of a tested system to respond to complex life situations. In this case, any real-life situation of substantial ethical import could be used, in particular situations prone to dilemmas and conflicts.

  4. 4.

    Apprenticeship, which would test an autonomous machine’s ability to act without supervision in real-life environment by participation in real situations.

The white box paradigm, as it was pointed out in the Stage 1, applies to all four stages of testing.

8 Operational Concerns

It is not obvious how the MST should be implemented. It obviously requires a machine capable of human-like functioning. A hardware-embedded software would be incapable of situational tests or apprenticeship without significantly compromising the MST framework. Thus, such devices by definition would not qualify as ethical agents and would not be subjected to the MST.

The learning period for an ethical agent would be long; yet because of the nature of computer technology, it may be that only selected exemplars of robots would be tested and the gains in ethical aptitude may be shared by appropriate updates within the compatible class of machines. Thus, there is no need to test every machine; only selected units should be tested, and the experience would be passed onto other agents.

The learning process for autonomous robots is not well defined. How the autonomous system would learn the proper responses to complex situations and how these responses would be integrated into their m-ethical data base is not clear. This should be another area of research.

It seems also that computing technology would allow the ethical experience to be “inherited” from generation to generation of ethical machines, provided that ethical norms stay unchanged. Thus, the ethical testing would not have to be done ab ovo with each new version of machines, something that we humans cannot avoid.

9 Review and Summary

It seems that the MST should include several testing venues, leaning mostly towards solving practical life situations. Such complex tests would verify the ability to make ethical decisions in the presence of conflicting cognitive stimuli, conflicting values and time pressure. Table 10.1 below shows possible components of such a test. The elements of the MST are arranged from the most elementary (Level I) and as such of lower importance to the most critical and complex (Level IV) in the rising degree of importance.

Table 10.1 Proposed structure and components of the machine safety test
Full size table

We should ask the question whether every autonomous agent should pass all of these test levels, or maybe, we could accept different levels of “robot ethics” and accept after Seibt (2017) “partial realizations” as applied to the MST, depending on the robot design?

10 How Would We Evaluate Results?

How would we know that the machine passed the test? One option is to have a panel of judges to review the results of the test and develop test-passing criteria as in the Turing proposal. Should we also accept the Turing criterion of a 70% pass score? If we do, what would it mean to have a 70% ethical agent? Or, would we accept a 70% ethical machine to be among us? It is easier, it seems, to use a 70% pass score to judge that a machine functions reasonably (this was a Turing proposal), but not whether it has a 70% moral aptitude. It seems that any number, short of 100%, as the criterion of acceptance of the MET results would be, in this case, an arguable qualification. But how are we to judge situational tests?

We need to admit that we are not sure how the MST should be graded and what it would mean for the autonomous agent to pass/fail the test.

Perhaps instead of a numerical score, we ought to assign some qualitative “moral” standing to ethical machines. Moor (2009) proposed four classes of ethical robots, or as he calls them—ethical impact agents. These are unethical agents, implicit ethical agents, explicit ethical agents and full ethical agents. These are interesting classifications of hardware–software constructs. However, the four classes are too crude to address the ethical capacity of autonomous robots required by the MST. The best we can say is that these classes mark the points on the spectrum of ethical aptitude from inert objects to human agents, but the scale by nature admits fuzzyfied, not crisp, classes.

Some suggestions as to the gradation of ethical abilities may come from HRI research. Seibt (2017) proposes in the context of human–robot interactions “five notions of simulation or partial realization, formally defined in terms of relationships between process systems (approximating, displaying, mimicking, imitating, and replicating)”. With the MST assumption that ethical machines “play an ethical imitation game” (not in the Turing sense of the game) and do not replicate human ethical abilities, such a classification may help us in understanding and classifying the MST results.

11 Parting Comments

We cannot exclude the possibility that the meaning of ethics or morality will evolve to the point that in the future ethical or moral principles attributed to humanity would be attributable to machines, robots, software or the like. Meanings of the words do evolve. Yet it is and will be important to make sure that now and in the future “machine ethics” means behavioural rules for machines, or machine safety specifications, not ethics in the human context. And the MST is supposed to test just this, not the presence of some kind of metaphysical moral fibre in hardware or software.

It seems that one of the barriers in the conceptualization of the MST is that ethical agents are perceived as “computers” or software bundles, in the same way as Turing conceptualized the Turing test subject (which is why we have TTT and MTT proposals). The ethical agents will be machines that act, move, interact with us in physical, not only mental, space. A small taste of this is offered by self-driving cars, which are essentially tested as the MST test is structured including software development, driving in a controlled environment,Footnote 11 driving with a supervisor and autonomous drivingFootnote 12 (Stillgoe 2017a; Hern 2017; Balch 2017). These are essentially four stages of the MST. In the context of self-driving cars, these tests are called social learning (Stillgoe 2017b).

It is rather difficult to imagine that machines will have the same complex of values that people have and thus the same responsibilities towards us. Thus, the MST will verify not how close computing machines come to us, but rather how close they come to our expectations about safe, autonomous machines. One must consider that our expectations regarding autonomous machines will evolve. Another challenge will be posed by the fact that the machine technology has global reach, while m-ethics (as any ethics) is quite often local. Thus, training of “ethical” machines will have to keep pace with these changes; otherwise we may risk meeting on our streets autonomous agents with behavioural propensities of cavemen.

And above all, we need to constantly keep in the mind the fact that we are training or developing machines to act safely, to not to harm us—this is the essence of m-ethics. If, by some fit of imagination, we call it ethical training so be it, as long as we are aware of the difference—thus, no moral robots, no ethical robots, just safely operating autonomous machines.

If history teaches us anything, in this case it may indicate that the ethics of autonomous artificial agents may go the same way as Internet security or software in general: just as software companies do not take responsibility for damage caused by their faulty software, so they will shed the responsibility for the transgressions of their faulty ethical agents. Thus, willingly or not, we may have to learn how to live with Microsoft-Windows-quality ethical machines.Footnote 13 Because is there any reason why the future should be any different? It rarely is; it just presents itself in different technology.