1 Introduction

Recent developments and applications of machine learning in legal, social and biomedical contexts have raised questions about ethical implications. Even if the algorithms used in such applications are accurate (that is, they capture the main relevant features that distinguish one group from another) how can we assure that they are fair? For example several researchers as well as activists have alerted that COMPAS, a computer program that judges are using in the United States to help them decide whether to give a prisoner a parole, may generate racially biased decisions because black population are over-represented in the data sample used to train the algorithm.

In this paper, we will use key concepts from Dreyfus’ legacy such as his view on skill acquisition, moral intuition or readiness-to-hand as the way humans usually undertake cognitive tasks to face moral decisions. We will draw from his phenomenological account of ethical expertise (Dreyfus and Dreyfus 1990, 1991, 2004) to establish the abilities that an algorithm should include to make ethical inferences. We need such criteria to avoid designing intelligent systems that appear to be prima facie fair but do not meet the standards of (human) experience-based moral competence.

In the first section we will argue the importance of ethical expertise in AI. In the second section, against emotivist or eliminativist accounts, we will defend that ethical expertise does exist and can be philosophically analysed. In the third section we will present Dreyfus theory of ethical expertise. The fourth section will be devoted to explain the limits of Dreyfus theory. In the fifth section we will discuss the difficulties that AI may face now to implement such an ethical expertise system and whether they might be solved in the near future, presenting our conclusion in the last section.

2 Ethical expertise and AI

Artificial intelligence (AI) is part of our lives, and in some areas, computer programs are starting to take decisions that can have great impact in human lives. Software is routinely used to decide whether a person is accepted in a university, whether she will be able to return a credit or he will be released on parole. We expect that those algorithms will take accurate decisions, based on the available evidence. But accuracy is not enough. We also expect that the decisions taken are fair that they capture basic ethical insights.

There is some discussion about future scenarios in which taking fair decisions are key, like autonomous driving. In case of accidents, we expect that those cars will take responsible decisions. Consider the following example from Lin (2016).

Imagine in some distant future, your autonomous car encounters this terrible choice: it must either swerve left and strike an 8-year old girl, or swerve right and strike an 80-year old grandmother. Given the car’s velocity, either victim would surely be killed on impact. If you do not swerve, both victims will be struck and killed; so there is good reason to think that you ought to swerve one way or another. But what would be the ethically correct decision? If you were programming the self-driving car, how would you instruct it to behave if it ever encountered such a case, as rare as it may be?

Another widely discussed future scenario is superintelligence and how all human life could be in danger if we cannot give some ethical protocols to AI general intelligence, see Muehlhauser and Helm (2012), Bostrom (2014), Bostrom and Yudkowsky (2014), Davis (2015) among others.

However, we do not need to describe possible future scenarios to consider a good idea to implement some ethical knowledge in AI. Consider COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) an actual software that judges use in hundreds of courts in the USA to decide whether a defendant is going to reoffend based on his or her background, former offenses, the type of crime, and so on (Skeem and Eno Louden 2007). According to Angwin et al. (2016) COMPAS software tends to wrongly flag as a potential reoffender a black person almost twice the rate as white people (45–24%). This is not the result of the developers of COMPAS being racists, but to the fact that, in the US, there are a way more black people in prison that white people. Because COMPAS tries to find patterns based on examples, it is understandable that it takes the shortcut of giving some races a higher probability to reoffend than others. However, by doing so, the software does not take into account that there are many other possible causes for this overrepresentation that are qua data wise less well formalized and simplistically focuses on race, which is part of the traditional classification scheme for offenders and of which there are a lot of data available. Evidently, the algorithm does not choose to focus on race beforehand, the race-centric pattern emerges during the learning. Race is not a bio-statistical fact, it is a normative class in which inmates are classified, without this class the system would not have the data to do its learning and the pattern of race would never emerge as such.

And COMPAS is just an example of a whole class of software labelled risk assessment algorithms which are used by judges and police to decide where in a city a crime is more likely to be committed, whether a person may participate in a violent crime or not, and even to settle a criminal sentence (Koerth-Baker 2016; Berry-Jester et al. 2015). This type of software is being widely used in the US right now, mainly due to the terrorist attacks of 9/11. After them there was a surge in surveillance, surge that leads to a big inversion in automatic recognition and prediction systems (see Wood et al. 2003). Zureik (2004) identified 17 different bills introduced in the US only several weeks after the attacks to develop such development turning a marginal industry into a main player in the security business.

Other countries are also keen to develop such computerized systems, though. The Indian government wants to turn the Aadhaar card (a biometric card) as the country de facto identity card, and is also developing a central monitoring system of telematic communications, including video-conferences, phone calls, text messages or WhatsApp (Sethi et al. 2013).

Another problematic area from an ethical point of view is face recognition. Consider the complex algorithms behind the Cromatica System of face recognition in the London Underground. As Introna and Wood (2002) argue, what they are doing is far beyond our concept of face recognition, it is a black box open to biases. It has also been established how most face recognition systems are trained using pictures of Caucasian individuals and, therefore, tend to fail systematically when trying to recognize people with darker skin (Skirpan and Yeh 2017; Buolamwini 2016) and how that may lead to unexpected discrimination. Another peculiar case of malfunctioning of face recognition happened in New Zealand when an automatic system rejected a passport photo of a person of Asian origin stating that he had his eyes “closed”. That person finally got the picture approved and his passport thanks to human intervention (Reuters 2016).

Then we have what Newman (2014) calls “algorithmic profile” the process of mixing data from diverse sources to include or exclude an individual from a specific category. Users are mostly unaware that data about them are being gathered and that they are being labelled, and some of those labels can generate unjust situations. Data brokers sell sets of identities of addresses of domestic shelters, rape victims, sufferers of addictions of genetic diseases or addictions, among others (Lyon 2015).

Credit card companies around the world obtain huge amounts of personal data from their costumers, and how much credit one customer get is usually done automatically using an algorithm. As Hurley and Adebayo (2016) describe, a man found his credit rating reduced from $10,800 to $3800 in 2008 because American Express determined that ‘other customers who ha[d] used their card at establishments where [he] recently shopped have a poor repayment history with American Express’.

Gender discrimination can also be generated by biased, unfair algorithms. In Datta and Tschantz (2015) it is described how Google ads of job offers differ whether the user was a man or a woman, men getting ads for higher paying jobs more often. The reason why this happened is not clear, as the researchers did not have access neither to Google’s algorithm nor to a description of how the ads were targeted. However, it is easy to infer that a machine learning algorithm taught with real-world job offers where women get less pay would repeat this unfairness in its decision system.

Finally, O’Neil (2016) develops a global stance describing how algorithms can make unfair decisions. A significant and fine point is how biased feedback loops can be created. One example is a crime-prediction software named PredPol used by the Pennsylvania police. Because the model included antisocial behavior such as vagrancy, the model sent police to areas where vagrancy was more flagrant. So policemen took care of such activities raising the “criminal activity” there, so the algorithm assigned even a greater probability of criminal actions there. Similar situations arise with the use of Compstat by the NY Police and Hunchlab by Philadelphia police.

Therefore, finding a way to include some ethical expertise in computers is not only a relevant issue for future AI developments, such as autonomous driving or general artificial intelligence, but a pressing concern to revise and improve current software that helps human to make decisions.

3 The case for ethical expertise

First of all, we need to establish that there is something actual under the label “ethical expertise”. By “ethical expertise” we mean the possession of ethical and moral knowledge, and the ability to use them to solve an ethical conundrum in a proficient way.

Some critics like Ayer (1937), Cowley (2005) or Pinker (2008) argue that ethical statements are just an expression of human emotions, not that different from our expressions on whether snails are a tasty food or something disgusting to eat. According to this view, ethics is just a matter of people expressing their own choices, so there can be no room for ethical expertise.

Other authors, such as Scofield (1993) argue that there can be expertise only if there is an objective truth and general agreement among experts based on that truth. Because there is no general agreement in ethical questions, there cannot be ethical expertise.

Lack of consensus in ethical judgements is not a relevant argument. As Varelius (2007) has argued, there is no such consensus in legal matters, for example, but nobody argues that there are no legal experts able to use their knowledge to help analyzing legal questions.

Emotivism—the statement that ethical utterances are just the expression of emotions—can be interpreted in several ways. The fact that some moral statements can be just a plain expression of emotions—as argued in Pinker (2008)—does not imply that all ethical knowledge has to be reduced in the same way. As Gibbard (1991) or Appiah (2008) has consistently argued, the fact that some of our ethical intuitions are grounded on basic emotions does not imply that ethical judgments are subjective; it only shows that they are intrinsically tied to basic traits of human nature.

How do we develop ethical expertise? What are the grounds for such knowledge? We will present Dreyfus’s account in the following section.

4 Dreyfus’ account of ethical expertise

Dreyfus and Dreyfus (1991) proposed a phenomenological description of five stages in the development of expertise. This model describes a progression from acting consciously on the basis of context-free and general rules at the novice level, to the application of situational rules at the proficient performer. At the expert level, behavior flows naturally as the actions fit the demands of the situation without analytical reasoning. When experts deal with typical situations, they are not ‘making’ decisions but carrying out actions that are likely to be successful. Dreyfus emphasizes the holistic nature of expert behavior, in which problem, goal, plan and decisions are appraised simultaneously. We will briefly lay out the model below:

  • Novice At this level the person shows a total adherence to taught general rules and plans; there is no contextual or situational perception that affects decision making.

  • Advanced beginner The person behaves according to general rules but begins to apply them to related conditions at her discretion; this behavior requires the identification of situational elements or aspects.

  • Competence At this level the person senses that the amount of general rules becomes excessive and begins to apply organizing principles to assess the information by its relevance to a longer term goal. There is conscious, deliberate planning and standardized and routinized procedures.

  • Proficiency The proficient performer sees the situation holistically rather than in terms of aspects, detecting without conscious effort what is most important and what is going on in a given situation, the proficient person uses analytical decision making and general rules but the general rules and principles are adapted according to the situation.

  • Expertise The expert does not rely on rules but uses intuition to make decisions. In typical situations, planning and acting as well as diagnosing the situation are performed without analytical calculation. At this level, experts are not actually making decisions or solving problems, ‘what must be done, simply is done’. Analysis is only performed during novel situations or when anomalies are detected.

Dreyfus typically uses chess or car driving as examples of expert behavior, but Dreyfus and Dreyfus (1991) and Dreyfus and Dreyfus (2004) consider ethical comportment also an expertise and expect it to show a similar developmental pattern as reflected in the model. The beginner ethical expert would learn some principles and maxims and use them regardless the context. She would gradually move towards the highest stage in which rules and principles are left behind and ethical answers are intuitive, holistic and spontaneous. So a beginner would never lie as the maxim tells us, but an expert would lie or tell the truth according to the situation. Of course, expertise in chess is different than expertise in ethics, in chess we could always tell who has won a match, but what is a successful ethical behavior? Dreyfus takes an openly Aristotelian view: ‘what is best is not evident except to the good man’ (Aristotle as cited in Dreyfus and Dreyfus 1991). And what is good is learned in practice; one becomes an expert by being exposed and by experiencing the same types of situations as those endured by experts. Also, one must also be able to sense satisfaction or regret at the outcomes of one’s action, a sense of regret or satisfaction that must be somehow shared or endorsed by other experts.

5 Limits of Dreyfus’ account

As we have seen in the previous section, an ethical expert develops her expertise in a progressive way, starting with basic declarative concepts, such as “you shall not lie” or “it is bad to take other people’s possessions without their consent”, which need to be analyzed in a conceptual manner as well, like the person that is a chess beginner and has to remind herself constantly to check whether the figure that her opponent moved is attacking one of her pieces.

After some time of practice, those rules are internalized and not lying or not stealing are second nature to us, like the chess master that can see checkmate in three, just by dropping a casual look at a chessboard.

However, if we consider empirically how soft skills develop and how Dreyfus uses basic Heideggerian concepts from Being and Time then we can consider another possible mechanism. We are presented with a moral conception of human actions since we are born, and we get a lot of examples and experiences of ethical and unethical roles in fairy tales, myths, cartoons, arts and entertainment.

Before we are formally introduced to the concepts of lying, stealing, treason, jealousy and so on we have been exposed to lots of experiences and examples of such behaviors, so we already have a pre-reflective layout which helps us to understand such concepts. This is quite different from other expert domains such a chess or computing programming in which first introductions tend to be conceptual. For a more extended discussion on this subject, see for example, Chrisley (1995) or DeSouza (2013).

It should be noted that this second mechanism does not imply any calls to irrationality. For both Heidegger and Dreyfus being fair, doing the right thing is not ultimately based in discursive thinking, but in a intuitive, contextual understanding of our surroundings, namely in what Dreyfus and Dreyfus (1986) call arational behaviour: ‘action without conscious analytic decomposition and recombination’. Dreyfus’ phenomenology of moral expertise has a particular emphasis placed on the linkage between knowledge and context, and as we have indicated in Sect. 4, Dreyfus defends a rational, discursive base in our beginning as moral agents, when our ethical knowledge is linked to discursive thinking, so there is no Kierkegaardian “leap of faith” either.

6 Can we build an artificial ethical expert?

If we carefully consider Dreyfus’ legacy, we can realize that we are still far away from an artificial system able to take fair decisions. As we have seen in Sect. 4, a pure declarative system, such as symbolic, GOF AI is not enough. Ethical rules are internalized in humans, and they arise depending on the context, and the relationship that the subject has with it. Moreover, according to Dreyfus, in familiar situations, however problematic, the expert contextual intuitive response is superior to judgment based on detached, abstract rules and principles.

Therefore, according to Dreyfus, those systems proposed in Goodall (2014), Hevelke and Nida-Rümelin (2015), Bonnefon et al. (2016) that rely on a formalization of the trolley problem (Foot 1967; Thomson 1976) to give an ethical framework to autonomous cars are not enough, as they are purely declarative systems, and they should be systems based on pre-reflective rules able to make sense of the surroundings. Returning to the example quoted in Sect. 2 from Lin (2016) we want a system in which there is a big qualitative difference between losing a human life and damage to a car, and not just a rule based system in which the price of repairing a Tesla might weight more than the insurance to pay for hitting a small child.

A pure machine learning approach would not work either. A neural network can be used to spot correlations and find patterns that are relevant to decide whether a person is going to return a credit or whether he is going to reoffend. However, such a protocol does not assure us that the system is going to be fair. Dreyfus and Dreyfus (1991) argue that “[w]hat one chooses to investigate as the relevant phenomena will prejudice from the start where one stands on these important issues”, we are not saying that the algorithms are biased per se, we just want to point out that existing biases in society, e.g. racial ones, might end up producing unintended unfair results. If a discrimination is already happening in a society, the system will just include it in its pattern recognition system.

Let us consider the discussion we opened in Sect. 2 about the nature of ethical expertise, and how ethical judgments are grounded in basic human emotions. Then we should consider the possibility than to have an AI able to make fair judgments it should also have some sort of artificial emotion repertory implemented (Arkin and Ulam 2009; Vallverdú 2011).

An artificial expert in ethics that meets Dreyfus requirements needs a pre-reflective system that has some autopoiesis, the ability to make sense of the surroundings and generate context-based judgements of the ethical implications of a situation, that are not just rule based.

For Dreyfus, ethical expertise is intuitive, immediate and spontaneous. Constant careful rational analysis would be a regression from ethical expert involvement into lower stages of ethical expertise. Still, expert deliberative judgment is important, especially when expert intuition breaks down, in a problematic and novel situation. However, expert deliberation it is not a ‘self-sufficient mental activity that can dispense with intuition. It is based upon intuition’ (Dreyfus and Dreyfus 1991). In a problematic but familiar situation, falling back on principles in a detached way produces inferior responses than when the expert ethical decision maker deliberates about the appropriateness of her intuitions while remaining involved in the situation.

To have an AI able to become an ethical expert we will need:

  • Ethical declarative concepts which the system can present to justify higher order decisions.

  • Some sort of autonomy to able to make sense of the surroundings and generate ability for autopoiesis. An artificial emotion module is a sound candidate to achieve that.

  • A pattern recognition system based on some machine learning paradigm that can capture common pre-reflective ethical judgments that are the basis of ethical expertise.

  • A framing and reframing mechanism that can define situations and its constitutive elements in a dynamic way and from multiple perspectives, and describe the importance of these elements and their mutual relationships.

This is not an easy task, but nothing in the actual development of AI implies that any of the items above cannot be reached, so we can be confident that, as AI develops, 1 day we could be able to implement some ability to take fair decisions in computers.

Let us consider how this could be implemented in the case of biased risk assessment algorithms, like COMPAS, its inner working precisely described in Freeman (2016):

First of all, we need to see how a machine learning environment would not be able to spot the unfairness. Based on examples in the past, which are biased against Black people, the system will continue repeating the same bias, so even if we use future results to further train the system, the bias will continue, following a biased feedback loop like the ones described in Sect. 2.

A pure declarative, discursive system would have a hard time making a decision either. From one point of view it could spot inequality as the percentage of Black people without parole are greater than the one of Caucasians, but other principles such acceptance of judge decisions, and statistical data about when crimes are committed and by whom would make the problem non-solvable.

So we need to add other criteria. Let ask ourselves, what lead to researchers to suspect about COMPAS, facial recognition software biases or gender discrimination in Google ads? It was the feeling that something was not right. There was something in that denial of parole or the amount offered to that person that was not fair. This is how the autonomy, the autopoiesis we described above, will be able to start the process.

Then we need to recover what we know about social, political and cultural context, the prereflective ability to make judgements that knows how inner cities and ghettos are, that knows how women are still discriminated at work, that understands the vicious loops of poverty and can go beyond what has happened in the past, realize that it is not fair and propose alternatives to limit and finally remove the biases. This is not easy, no rule-based system will be able to make a clear cut distinction between a fair and unfair parole, or a fair or unfair job offer. We need to switch perspectives, understand all the contextual implications and how they relate to the targeted individual.

All of this implies that, to have algorithms capable to make fair decisions, better algorithms and learning environments are needed, so they are able to generate process akin to general intelligence, and not just pattern matching from past examples. It also implies that such algorithms besides the information they have about the specific case, like the former criminal history of the person, the type of crime and where that person lives, it needs a lot more about social and cultural conditions, discriminative practices, socio-economic conditions, and all the prereflective knowledge we humans have about the world that leads to suspect whether a person is victim of a discrimination due to his/her race, gender, sexual orientation and so on. In a sense, they need to know the difference between what it is now, and what it should be, which can never be deduced from past examples.

7 Conclusions

We cannot expect an artificial expert on ethics in the near future. However, the need to evaluate whether AI systems are taking ethical decisions is mandatory, as plenty of software is already being used in situations that have ethical implications for human beings. This need will grow as autonomous cars start to move around freely, and will be a must if we decide to embark in the project of superintelligences. So, in the meantime, while we start to develop AI with the ability to produce ethical judgements, we need committees of human experts that can examine a given algorithm and decide whether it is free of biases and can produce fair decisions, or not. To do so, algorithms used in situations in which a fair judgement is needed must be open and transparent, both code and training examples. Such committees have to include experts in computer ethics as well as computer scientists that can make sense of the argument, but, as Dreyfus pointed out, we also need common citizens that master the non declarative, pre-reflective ultimate nature of ethical judgements and give that extra dimension to assure that we are having not only accurate, but also a fair software.