Keywords

12.1 Introduction

The development of medicine in our days is profoundly driven by the application of ever new technologies. Currently, two classes of novel technologies are at the brink of shaping the medicine of the future: artificial intelligence (AI) and neurotechnologies that directly interact with the human brain, in particular brain-computer interface (BCI). The potential of these technologies is vast, but so are also their risks and ethical implications. In this chapter, I would like to draw attention to some of these ethical implications that BCI and AI have on the practice of medicine and health care today.

Before embarking on this task, I would like to make three important preliminary remarks: (1) AI and BCI are, of course, two distinct technologies with different ethical implications. However, invasive neurotechnologies, in particular BCI, increasingly incorporate AI and deep learning so that the former is not fully comprehensible without the latter. (2) The implications of these new technologies go far beyond ethical issues, they touch on legal, social, philosophical, economic, political, and other dimensions of human life. To be coherent with the focus of this book, however, I will focus on the ethical implications, in particular on research ethics and clinical ethics, while being aware that the ethical dimension overlaps with these other dimensions. (3) Given the breathtaking pace of these technological inventions and their broad potential uses I can only proceed in a cursory way in this chapter, selecting some of the most remarkable forms of technology use and some of the most salient ethical issues arising thereof.

12.2 The Importance of Neuropsychiatric Disorders for the Development of Health Care Technology

Disorders of the nervous system are among the most common diseases worldwide, and their frequency is rising due to global aging [1]. They contribute most significantly to the global burden of disability. Recent advances in early (even predictive) diagnosis and in disease-modifying therapies both result in even longer chronicity and a higher prevalence of such disorders as Alzheimer’s disease, Parkinson’s disease, or multiple sclerosis.

One of the characteristics of nervous system disorders is that they often have “negative” rather than “positive” symptoms, loss of function rather than unpleasant sensations like pain, nausea or dyspnea. Patients lose basic functions of their everyday life, such as walking, using their hands, practicing hygiene, speaking, or swallowing. The capacities that people tend to lose with nervous system disorders are commonly capacities that are highly valued in our society: cognition, communication, personal autonomy, responsibility, biographical life planning, and social interaction. This contrast explains much of the enormous impact that neuropsychiatric diseases have on individuals, families, and society. It is no surprise that dementia is tending to replace cancer as the most feared illness [2]. These fears create a very strong emotional incentive and a societally compelling reason to conduct research on nervous system disorders.

One particular disease that is among the most feared (and that often nurtures wishes to hasten death [3]) is amyotrophic lateral sclerosis (ALS), also called motor neuron disease. This disorder is caused by a continuously progressive degeneration of the two successive motor neurons in the central nervous system, leading to a gradual weakening of all skeletal muscles of the body. Thus, all movements that use the extremities, the body trunk, but also facial muscles are affected. Patients initially note a clumsy hand or a weak leg; in a subset of patients the first symptoms are slurred speech and swallowing problems due to facial, oropharyngeal and neck muscle weakness (Box 12.1).

Box 12.1: A Fictitious case example

Fred was a 58-year-old farmer from Bavaria, Germany, and a passionate amateur soccer player. One day he noted difficulties while handling the gear lever of his tractor and when opening the door of his house with his keys. Over the next weeks, both of his hands became more and more clumsy and weak. The diagnostic workup at his primary care physician was inconclusive, so he was sent to an orthopedic physician who presumed the diagnosis of a cervical spine injury and operated on him. The operation, however, did not change anything for the better and it took another 5 months until a neurologist finally established the diagnosis of amyotrophic lateral sclerosis. At that time, Fred already had some degree of paralysis in all four limbs. Shortly after, his speech became slurred and his family and friends suspected him to be an alcoholic, which is why he increasingly withdrew from social contacts and became very isolated. When he finally lost his ability to express himself verbally, he used a tablet computer to communicate, but even this became more and more cumbersome for him. One day, when he was surfing the Internet, he read about a new study on a device named brain-computer interface, which allows paralyzed persons to speak, move, and be autonomous again. He immediately seized this opportunity and registered for the study, dreaming of a new life despite his illness and disability…

There are, to date, no therapeutic options that have the potential to cure ALS or halt its progression. The fact that a new and extremely expensive drug was approved after showing modest effectiveness in a subset of patients in a small and short-term pilot trial underscores the desperate need for a therapy to treat ALS [4]. In most cases, the disease inexorably leads to death within 3–5 years. Patients’ lives can be sustained by mechanical ventilation, but at the same time the disease progresses and often leads to a locked-in state (LIS) that is characterized by a relatively intact mind that is locked in a completely paralyzed body, except for some eye and eyelid movements, which, however, may eventually also get lost (“complete LIS,” CLIS).

Many health care professionals and researchers consider this to be a state that they would not want to live in and for which they would prefer forgoing life-sustaining measures [5]. What the patients in CLIS themselves think is unknown to us. A study among LIS patients who could still communicate reported that some maintained a high quality of life, while others felt miserable and preferred euthanasia [6]. We do not know what happens when LIS patients’ last communication channel closes.

It is obvious that there is a very strong incentive to treat or mitigate the state of CLIS. Moreover, scientific and human curiosity drives our quest to know what it is like to live in such a state, without any possibility of expression and communication.

A technology that has the potential to be this desired window into CLIS is BCI. In the context of this book, it is not necessary to include a lengthy introduction into BCI [7]. It suffices to say that BCI is defined here as a technology that joins the human brain and a computer, thereby enabling a person to directly influence the environment via his or her own brain activity, without using the body’s own motor system.

12.3 Ethical Implications of Medical Research Using Brain-Computer Interface

The ethical implications of medical research on BCIs is clearly exemplified by a recent case of alleged misconduct that happened in Germany. In 2017, the open-access peer-reviewed scientific journal “PLoS Biology” published an original article authored by Ujwal Chaudhary and colleagues, with the well-known neuropsychologist Niels Birbaumer as last author [8, 9]. The international author team (in addition to the German researchers there was also a scientist from the US National Institute of Neurological Disorders and Stroke, as well as scientists from China and Italy) was funded by a multitude of renowned funding agencies; among them the most reputable state funding agencies in Germany. Their paper, entitled “Brain-Computer Interface-Based Communication in the Completely Locked-in State,” promised a major breakthrough in BCI research: for the first time in history, the authors contended to have successfully established communication with patients in CLIS.

Their publication disclosed the results of a case series of four patients that were reported to be in a CLIS. Using functional near-infrared spectroscopy (fNIRS), the researchers reported having decoded frontotemporal brain activity allowing them to distinguish between the mentalization of “yes” and “no” in response to orally presented questions of personal relevance (like the sentence “You were born in Berlin”). Three of the four patients were also asked so-called open questions about their attitudes towards their present situation and their life in general, yet the article was not very specific about these questions mentioning only that each patient was asked 40 open questions, such as whether they love their life or whether they feel sad.

The main summarizing statement by the authors was the following bold proposition:

Even after extended CLIS in ALS spanning months and years, reliable, meaningful communication using questions requiring a mental affirmative (yes) or negative, rejecting (no) answer is possible with fNIRS-BCI [8].

Regarding the open questions, they affirmed:

Patients F, G, and B answered open questions containing quality of life estimation repeatedly with a “yes” response, indicating a positive attitude towards the present situation and towards life in general (…) [8].

Strangely, the authors admitted that they did not have a plausible physiological explanation as to why the fNIRS responses were different between presumed “yes” and “no” responses.

Chaudhary et al. were aware that their experiments had an existential impact on the family members of the patients: “Family members of all four patients experienced substantial relief and continue to use the system” [8]. Interestingly, family members were always present during the experiments. The authors even mentioned that they never officially screened or recruited these patients, but it was the family members who approached the senior researcher and asked to participate in these experiments.

They were also aware of the ethical implications of their study:

Still, we have to remain cautious about our judgements to open questions’ answers, particularly if it comes to quality of life and psychological changes of CLIS patients. In view of the gravity of the subject matter (i.e., establishing communication with nonverbal, completely paralyzed persons with preserved cognition), a call for replication of the current results by other investigators would be welcome [8].

This call for replication was repeated in the abstract of the article.

The media response to this article was overwhelming. Around the world, newspapers and news agencies reported about these experiments in an enthusiastic manner. “Decoding the thoughts of patients who can’t even blink,” was the CNN headline [10], “The Telegraph” used the headline “Locked-in patients tell doctors they are ‘happy’ after computer reading thoughts” [11], and the MIT Technology Review heralded the study as “Reached mind via a Mind-Reading Device, Deeply Paralyzed Patients Say They Want to Live” [12]. The scientific publication resonated with a long-held dream of mankind, the dream of mind-reading and of controlling the environment solely with thoughts [13, 14].

The researchers themselves appeared in the lay media presenting and explaining their research results, primarily the senior author Niels Birbaumer. In a long verbatim interview with the Germany weekly newspaper for intellectuals “Die ZEIT,” he not only specified how the experiments were conducted and what the results mean, but also gave insights into his own personal motivations and attitudes [15] (Box 12.2).

Box 12.2: Extracts of a verbatim interview with Niels Birbaumer (Die ZEIT), translated by the author (RJJ) [15]

About his motivation to do this kind of research:

I am terrified about the situation of these people (…) No one makes any effort to do experiments with these unattractive patients (…) We do research and publish for 20 years. The result? Nothing (…) How many disappointments along the way! Without a few positive results now and then, I would have resigned long ago.

About the measurement of brain activity:

We pose the same question several times and the computer averages the yes/no responses (…) When the brain waves slow down, the computer does not count the answer (…) The computer sums up yes/no responses and calculates whether they correspond to the expected answers.

About the open questions:

Once we asked a patient whether his daughter would be allowed to marry her boyfriend. The answer, in 9 out of 10 cases, was no. Asked whether they are satisfied with their life, all responded yes.

About the continued BCI use after the end of the study:

One patient’s family is using it regularly (…) We have already litigated together with the family of a patient (…) the court has decided that the € 50,000–70,000 device for locked-in patients has to be paid by the health insurance.

About the significance of their own results:

I now let other groups replicate the results. It would not be trustworthy if always the same researchers would do that.

Remarkably, Birbaumer mentioned that their results have enormous practical consequences for these families. One family obviously wanted to continue using the BCI technology after the trial and had to litigate with the insurance company to receive a reimbursement of € 50,000–70,000, which is how much the device seems to cost, and the researchers appeared to be engaged in that litigation.

Birbaumer also reiterated the invitation from the journal article for colleagues to replicate the study. In fact, another researcher from the University of Tübingen, Germany, Martin Spüler, took this call seriously: while he did not replicate the whole study (an identical replication would be difficult for such a clinical case series), he took the published raw data and replicated the statistical analyses. His comment was published in the same journal as the original study several months later [16]. He claims to have used two different statistical models to analyze the data and did not find any significant difference in the fNIRS response between yes/no questions. He also exposed some methodological flaws of the original study. He basically asserted that the authors used a statistical calculation method that always leads to a significant answer, whatever the data may be.

In the same issue of the journal, the study authors refuted the critique by Spüler [17] and a researcher from England wrote a commentary that ended with a diplomatically worded criticism of the original publication:

BCI research is interdisciplinary and is at the intersection of natural science, social science, engineering science, and medicine. Clear and simple communication is essential. Lack of detail can lead to confusion. Confirmation bias has an influence on the interpretation of results (…) To enhance clarity of communication, reports should (i) be written in simple language; (ii) methods should be clear, precise (…) and (iii) interpretation of results should be objective and realistic—in itself a hard task [18].

The whole issue went public through a newspaper report by scientific journalists in the renowned German daily newspaper “Süddeutsche Zeitung Magazin” [19], just a few days after the publication of Spüler’s criticism. The report is a journalistic masterpiece under the heading of “Wunschdenken,” in English “Wishful thinking.” The three journalists claim to have talked to the persons involved and other expert researchers in the field, stating that experts who prefer to remain anonymous confirm the doubts about the quality of the study. Moreover, they have shed light on attempts of the study authors to silence their critical colleague. In fact, Spüler was fired by the University of Tübingen, the same university where the first and last study authors worked.

The press coverage, however, stirred international attention on this case [20] and prompted the University of Tübingen and the main funding agency to initiate investigations. A commission at the university confirmed the doubts about the scientific validity of this study [21, 22]. The main criticisms were (a) a highly selective choice of data for the publication (not publishing all data), (b) a lack of transparency concerning some data and the statistical methods, (c) missing data for results that were published in the article. The university asked the journal to withdraw the paper, which has eventually been done in December 2019, and informed the involved patients and insurance companies. Birbaumer himself criticized the university commission for lack of expertise. Later, the commission of the German national funding organization DFG confirmed this position and labeled the study as scientific fraud. It has banned the authors from applying and reviewing for the DFG for 5 years and has backed the withdrawal of the publications related to this study.

From a research ethics perspective, several issues are remarkable. First of all, this study underscores the fact that scientific research is never morally neutral. It is always embedded in a context of value-laden motivations of researchers, funders, and research subjects (or their families). These underlying moral assumptions and attitudes shape the choice of a research domain, the definition of study objectives, the selection of methods, and the interpretation of results. This case shows that it would be naïve to entertain the idea of a morally neutral, purely factual science. Rather, researchers should be aware of this moral dimension of their work and reflect on it in a self-critical and transparent way.

Second, this research example underlines that the psychology of the researcher may not always match the logic of research. While the latter should be the quest for falsification of a hypothesis (trying to disprove it), researchers are usually rather motivated by a strong conviction or even belief in their hypothesis, pushing them to try to prove the hypothesis at all costs. This attitude may lead to confirmation bias, influencing study design, methodology, and result interpretation so as to confirm the hypothesis. In fact, the highly competitive and commercialized research system favors this attitude: researchers have to deliver “positive” results to yield high-impact publications, funding, and attention in the scientific community and the public, necessary ingredients both for personal careers and the survival of research teams and centers.

A third implication of this case is the troubling observation that the rational criticism of study methods and results may be regarded by researchers as “whistle-blowing,” whereas in fact it should be appreciated as the very essence and heart of scientific research. If critical thinking and expression become endangered, it will be the end of the kind of modern scientific research that the enlightenment has brought about and that proved to be so productive and successful. Small scientific communities working on rare diseases with very few patients might be particularly prone to this kind of danger, as all researchers in the field know each other and depend on each other for peer review of publications and funding applications.

A fourth implication concerns the technology more specifically. The controversy around this case is partly due to the complexity of the applied technology and the difficulty to retrace and understand the computer algorithms that were involved. In fact, as the researchers used a type of AI called deep learning algorithms, its characteristic is that it constantly changes its own method of data analysis in an automated way. In current deep learning devices, these automated adjustments may occur in a black box and may not be observable or retraceable one by one. In other words: using deep learning may confront us with research results whose scientific validity we may not be able to prove nor disprove. Did the BCI in the current case really detect true “yes” and “no” thoughts by the patients or was this result an artifact created by deep learning AI? The case may thus herald a major problem of AI that we will encounter more and more often with the increasing use of AI in the future.

Fifth, the case is troubling because it calls into question the role of research ethics committees. The article mentions that the internal review board of the University of Tübingen approved the experiments [8]. Did this board discuss the ethical implications of deep learning algorithms? How could it approve that such experimental technology was used to ask existential value-laden questions about the patients’ will to live? In practice, these committees are often not equipped to identify and deliberate ethical questions, but rather limit themselves to applying checklists on methodology, informed consent procedures, and data protection.

Sixth, one of the ethical questions that such a committee should discuss is that of surrogate consent in studies with locked-in patients. Clearly, patients in a CLIS do not have the legal capacity to consent themselves to the study because functional communication is a prerequisite for this capacity [23]. According to the Declaration of Helsinki, which is the internationally accepted code of ethics on research with human subjects, experiments on these highly vulnerable patients are possible, but they have to respect additional safeguards [24]. Commonly, these studies should have the potential of a direct benefit to the study participant, e.g., an effective treatment of disease symptoms. Was this the case here? Was there a direct benefit from answering the yes/no questions? Or may these questions, at least in some patients, actually increase their awareness of their own suffering? Of course, one could argue that a well-functioning BCI communication device would be a huge advantage for patients in CLIS, but only once this technology is advanced enough to allow this kind of well-functioning, fine-grained communication. Thus, the study participants would only have what is called an indirect benefit: they, or other patients in a similar clinical situation, will benefit in the future, when the technology is ripe for application. Studies on subjects without decisional capacity that contain the potential for an indirect benefit, only however, may not have more than minimal harm and minimal burden according to the Declaration of Helsinki [24]. Whether undergoing fNIRS and the other investigations of the study satisfy the criterion of minimal burden is debatable.

The seventh and last point about the ethical implications of this study concerns the families of the involved patients. It is evident that families of such highly vulnerable, dependent patients who cannot act, decide, and communicate by themselves are highly involved in everyday care and decision-making. They often place very high hopes in technology, making them prone to becoming the victims of false promises and subsequent frustration [25]. When families contact researchers in order to enroll their patients into studies, as was the case here, this amounts to a self-selection of families with extremely high hopes, and researchers have to be very careful to avoid nourishing these exaggerated hopes.

In today’s clinical reality, there is still not a single BCI application that has entered into routine health care. The most promising fields of medical BCI application in the near future are neurorehabilitation (restoring damaged motoneuronal pathways by pairing movement volition with external activation of the muscles or peripheral nerves), neurofeedback (training attention and mental focus using BCI tasks), and neuroprosthetics (controlling an arm or leg prostheses or even wheelchairs) [26,27,28]. As we have seen in the research project on patients in CLIS discussed above, most of these BCI applications will incorporate deep learning algorithms and AI that allow the BCI technology to learn from the person, adapt to her individuality and make movements and actions faster, more efficient and less effortful. Therefore, it is paramount to reflect as early as possible on the potential ethical and anthropological consequences and implications of AI in clinical care.

12.4 Clinical Use of Artificial Intelligence: Ethical and Anthropological Implications

BCI are one example of the use of AI in medicine. As in society in general, there are numerous potential applications for AI in health care. But in contrast to our everyday life, where we already use AI in smartphones, voice recognition devices, internet search engines, social media platforms, public transportation and many other domains, AI is still in its infancy in health care—even if the potential is vast. The most promising health care uses of AI can be seen in diagnostic and prognostic assessment, where large amounts of information have to be processed in order to increase the precision and validity of diagnostic or prognostic statements [29]. I will therefore first show some prime examples of diagnostic and prognostic AI uses and discuss their ethical implications afterwards.

Medical diagnosis relies more and more on imaging techniques and the visual recognition of pathological patterns. Typical examples are in dermatology, radiology, and endoscopy. A recent study has shown that a so-called neural network using deep learning paradigms could be trained so that its diagnostic performance in recognizing melanoma is actually superior to the performance of skilled dermatologists [30]. This kind of visual recognition may also function for diseases of the inner organs, and even the brain, that are associated with subtle changes in appearance: Fetal Alcohol-Spectrum Disorders could be reliably detected based on facial features by computer algorithms [31]. Moreover, AI can be a potent help in differential diagnosis, calculating the probabilities of various likely diagnoses based on a multitude of patient data [32].

In medical prognostication, physicians usually rely on validated scores, as well as on their intuitions that are ideally informed by long professional experience and many patient cases. Yet, this experience-driven knowledge is impossible with regard to rare diseases and hardly possible for junior physicians. Moreover, intuitions are prone to a host of psychological biases [33]. Thus, AI has the potential to significantly improve the accuracy of medical prognostication. As an example, an AI algorithm managed to predict survival rates based on microscopic pathology images in patients with non-small cell lung cancer [34]. One of the rarer studies where AI was used in medical treatment found that an AI-based chatbot that performed an automated form of an online cognitive behavioral therapy effectively reduced symptoms of anxiety and depression in young patients [35].

These few examples may already suffice to sketch the broad and diverse array of effective AI application in medicine, including neurology, coupled with BCI and other neurotechnology. Without exaggeration, one could expect a profound transformation of health care within a few decades. Diagnosis and prognosis will become much more precise and reliable. Disease entities will multiply as AI will help to differentiate between nuances of different diagnostic patterns as well as between different disease courses and responses to treatment. Moreover, refined diagnosis will also mean earlier diagnosis: AI may help to detect extremely subtle signs of diseases in a pre-symptomatic stage, increasing the prevalence of diseases in the population and contributing to a general trend of pathologizing and medicalizing our societies. The novel paradigm of medicine will be predictive medicine that can powerfully and precisely predict diseases, their course and symptoms, response to therapy, survival times, and many other characteristics—without necessarily having more to offer in terms of cure and effective treatment. The challenge for the patient is evident: How should he or she react to this novel kind of predictive knowledge? Will it restrain the degrees of individual liberty (at least subjectively perceived liberty) or even their autonomy? Will it stir a new kind of existential angst and lower quality of life or will we find ways to use it to increase quality of life? The main challenge will be to find positive, fruitful ways to deal with this new predictive knowledge in medicine.

The increasing use of AI in medicine will also challenge the patient–physician relationship, at least in three ways: first, it will reinforce the current trend of a growing interpersonal distance between the patient and the physician that already started with the invention of the stethoscope, deepened with radiology and is about to become even more distant by telemedicine. If the computer can diagnose, prognosticate and even treat better than a physician can, the latter will not need to go to the bedside; his place will only be at the computer, entering data, supervising the analysis, and interpreting the results. Ultimately, highly effective AI medicine could allow self-diagnosis and self-treatment from A to Z: the patient (or family members) could enter information and specimens, allow cameras and sensors to obtain data, receive an accurate diagnosis in the blink of an eye and get the appropriate medication a few minutes later by drone directly to his home. For a range of diseases and clinical situations, the involvement of a physician may not be necessary anymore. The ethical problem, however, is not only that the patient will become more and more isolated and lonely.

The dialogue between the patient and the physician that is so central to the acts of responsibility—defining a medical indication and obtaining informed consent—will be threatened. As the philosopher Emmanuel Levinas emphasized, the personal direct encounter of another human being (looking another person in the face) is what makes us responsible for our actions [36]. In this line of thinking, in AI medicine we cannot assume human responsibility, in a rich moral sense, in the same way as we can do when we have personal contact with the other person.

Second, AI medicine may lead to a desynchronization between the subjective time dimension of the ill person and the professional time dimension of medicine. The speed of AI algorithms is breathtaking and surpasses that of traditional medical diagnosis, prognosis, and treatment by far. Within a few seconds, a high-performing algorithm could find the correct diagnosis and define the treatment plan. It is questionable whether this saving of time really translates into valuable time for the persons involved, both the patient, the family, and the health care professionals [37]. Patients usually need many days, weeks, or even months to understand a diagnosis, to cope with a new life situation, and to make important treatment decisions. They also need other human beings with narrative capacity, who are able to understand their personal story, make sense of the development and course of a disease, and help to narrate an illness story that will be embedded into the life story of the patient [38].

Third, AI medicine has the potential to depersonalize health care in a profound way. Computer algorithms may have lots of advantages over physicians, but their inevitable shortcoming is that they always treat patients as instances of a collectivity, as statistical cases, and never as unique individual persons. This is inherent in their functioning. In fact, they do not understand the concept of a person because they themselves are not persons and lack human subjectivity. Yet, intersubjective relations are vital for human beings, especially for vulnerable persons with illnesses. In fact, it is these intersubjective encounters that mutually constitute our personhood. According to the German philosopher Axel Honneth, an individual cannot conceive himself as a genuine person without recognition by other individuals [39].

In summary, we have seen that AI may have profound consequences and challenges for health care. These challenges pertain to any kind of AI use in health care. Yet, if AI is applied to neurointerventions like BCI, the ethical problems of both areas converge and become even more troublesome. There is an inherent tendency in neurointerventions to use AI because they need to match the complexity of the human brain and this is best done by using deep learning algorithms that are based on artificial neuronal networks. I would even hypothesize that any new form of neurotechnological brain intervention that will be applied in humans will heavily rely on deep learning and AI. Yet, the inevitable combination of neurointerventional technology and AI for medicine potentiates the ethical problems. As I have shown, despite its remarkable performance, AI is in many respects alien to human needs and human characteristics: it cannot be take responsibility, it does not have emotions, it operates in a different temporal dimension, it follows a logic of depersonalization. When AI technology with these features is closely linked or even integrated into the human brain, this threatens what we call human nature, our human identity. The paradigm of the new, AI-based machine may merge with humans in such a way that what results could indeed be a novel kind of being, a cyborg that is not simply a combination, half human half computer, but a completely new entity in which human and technological parts may be indistinguishable. Thus, the fundamental distinction between human beings and technological artifacts might disappear in this new kind of entities.