Introduction

“We are drowning in information, while starving for wisdom. The world henceforth will be run by synthesizers, people able to put together the right information at the right time, think critically about it, and make important choices wisely.” — (Wilson 1998, 269)

“The objectives of the human race have not been reconsidered in light of the science of the past few hundred years.” — Gerald Feinberg (Snow 1969)

Defying the orders of Zeus, Prometheus stole the sacred fire from Mount Olympus and gave it to mankind, allowing fire to pave the way to technology. Prometheus was the perpetrator of human “progress” against the forces of nature. Similarly, in Marry Shelley’s 1818 magnum opus Frankenstein (initially subtitled The Modern Prometheus), Victor Frankenstein ventures into what should only belong to the gods—immortality—by working on inanimate body parts to eventually craft and bring into existence a “monster” with no name. To the astute reader, Frankenstein’s monster seems to be Shelley’s depiction of the Industrial Revolution (transition from predominantly agricultural societies to modernized and technology-based ones), including the ensuing fears and apprehensions that humans have towards the rampant progression of science and technology.

In this article, we discuss the fourth industrial revolution (Lavopa and Delera 2021), through a focus on the core of this phenomenon, artificial Intelligence (AI). We outline what the fourth industrial revolution (4IR) is, its basis around AI, and how this infiltrates human lives and society, akin to a transcendence. Our focus is on AI in medicine and medical education, and specifically the potential dangers of AI, ethical concerns at all levels of its development and implementation, and potential solutions to such dangers. While we acknowledge the importance and potential of AI, we also want to highlight the importance of having cautious reservations before plunging straight into the exciting world of the future, one which we long have heard of only in science fiction movies.

Discussion

The Fourth Industrial Revolution

Amid such exciting developments, one wonders to what extent humans in general, and healthcare workers in particular, can rely on these so-called “smart machines” in everyday life tasks. Can they be trusted to take on the roles and duties of humans, to do that which requires more than just running algorithms, like the integration of critical thinking and empathy in decision making?

AI, Medicine, and Health Care

The medical profession has often been regarded as a “sacred” field, due to its dealing with disease—the great equalizer of the wealthy and poor, the empowered and disadvantaged. As Pellegrino noted, medicine is “the most humane of sciences, the most empirical of arts, and the most scientific of humanities” (Kollmer 2019). The complexities of diseases require the flexibility of the human mind to analyze, diagnose, and treat. This field has been augmented and strengthened by technology, but it is also being infiltrated by, and merged with AI and machines. For example, a large proportion of processing happens through computers with physicians analyzing only the outputs. With greater pressure to see more patients per unit time, doctors rely more heavily on these machines, at times even at the expense of physical diagnosis and examination. With increased encouragement to merge with AI, we are beginning to see areas where the line between physicians and machines is becoming blurred, with tremendous impact on the profession of medicine as we know it.

A major pillar of medicine and medical ethics is the principle of justice, behind which the concept of equity stands. According to Karan et al. (2016), “The principle of justice in medical ethics refers to a fair and equitable distribution of health resources”. The integration of AI into clinical medicine could both serve to increase or undermine justice in this sense, depending on its applications, hence the crucial role of governance especially in relation to AI integration in LMICs. This risk is especially evident in the fields of medicine and medical education, as “ many LMICs still lack the capacity or resources to develop comprehensive digital health strategies and to implement digital health interventions.” (Ho and Malpani 2022).

Whereas some uses of AI are as tools that help develop drugs and diagnoses, others show the merging of human and machine, in which AI performs some roles of physicians. Some examples where we see such applications in LMICs are highlighted in the recent USAID (n.d.) report on AI in Global Health. Whereas the report provides extensive examples of how AI integration into healthcare is especially beneficial to LMICs, we will consider two prominent examples where such tools almost completely fill in for the lack of adequate healthcare providers and systems. The first example highlighted is a frontline healthcare worker in a remote village in Western Kenya, where the nearest hospital is more than 2 hours away and the nearest major city is 6 hours away (USAID n.d.). Through utilizing a smartphone and AI-enabled applications, the healthcare worker is able to enter patient symptoms and signs, demographics, and pertinent history. Machine learning algorithms then run said inputs through databases on which the system was trained to then recommend diagnoses, treatments, and improved care. Such benefits enable even a low-level of training to provide at least life-saving medical attention where they may otherwise not exist (USAID n.d.).

Another example where we see AI truly being integrated into medicine and particularly the clinician’s role in decision making is in ML algorithms that are integrated into magnetic resonance imaging to reduce scan time, and reconstruct scan “slices” into complete, structurally aligned readable results (FDA 2023; USAID n.d.). Such software enhances the efficiency of radiologists and radiologic technicians, while also reducing the burden of long MRI scans and scarcity of devices. An application in such clinician decision support in LMICs is highlighted in USAID’s (n.d.) report, where a radiologist in Ecuador struggles to screen hundreds of women presenting to her breast cancer clinic. Through the integration of an AI-enabled tool that “reads” scans and highlights suspicious lesions while also providing a percent certainty of its decision, the radiologist is able to accelerate the number of patients she sees threefold to fivefold per day, with increased certainty of diagnoses.

AI undoubtedly holds immense potential to facilitate medical revolutions and improve lives and living standards by integrating with medicine in the academic, industrial, and clinical settings, especially in LMICs.

AI in Medical Education

With AI gaining such a pivotal role in medicine and healthcare, the need to integrate it into the medical curriculum and training of future healthcare workers is a necessity. Several advantages of the use of AI in learning are immediate feedback, identifying and responding to gaps in students’ knowledge, and lower rates of information overload (Wartman and Combs 2019). AI is mostly used in undergraduate medical education due to the presence of a structured curriculum that allows its integration into anatomy and physiology simulations. AI is also used in the assessment of learners by automated clinical competency essay scoring, student case summary grading, and evaluation of basic laparoscopic skills (Alonso-Fernández et al. 2018; Kintsch 2002; Latifi et al. 2016).

Notwithstanding the need for a structured curriculum to act as a knowledge base for AI programs; concerns regarding the quality of feedback remain due to requiring a system equipped with expert domain knowledge for contextually driven education. The main barrier for AI use in assessments is lack of digitization which prevents fulfilling the data pool requirements to develop AI-based systems. This limitation is much more pronounced in the educational institutions that use mainly non-digital tools for different domains including teaching, assessment, and patient recordkeeping. The summative and sensitive nature of medical professionals’ assessments, security issues regarding communication and potential malfunctions, or improper coding of AI systems leading to wrong results, further limits the use of AI as systems of assessment. Additionally, AI cannot assess empathy, care, communication skills, and interpersonal skills of medical students, all of which become more pertinent with the introduction of AI in medicine. These competencies are essential to foster trust between the physician and patient to facilitate successful medical care (Shaya et al. 2019; Al-Habbal and Arawi 2020). A balance between the teaching that students receive from healthcare professionals and from machines is needed as there is no substitute for human touch in medicine (Imran and Jawaid 2020). Still, assessment in medical education may be augmented by the use of adaptive and programmatic evaluations. In adaptive assessments, the difficulty of questions is tailored according to the individuals’ responses to earlier questions to optimize learning outcomes. AI has infrequently been used for curriculum revision in medical education despite evidence of the advantages of artificial neural networks in providing an overview of the effectiveness of curricula and students’ satisfaction with the program (Imran and Jawaid 2020).

Additionally, medical schools may lack the faculty expertise required to teach AI in medicine content, considering that it is a novel field. This can be solved by increasing interfaculty collaborations between healthcare experts, engineering, and computer sciences faculties to ensure that AI tools complement medical teaching roles rather than replace teachers. Use of AI tools to teach ECG and echography reading skills, along with overviews of how algorithms are developed, can allow greater appreciation of AI in decision making and predicting probabilities. Programs have been designed to increase the awareness of and highlight the importance of AI in medical education, such as the 6-month course “Certificate in Online and Distance Education” (CODE), in which participants focus on how AI in relation to learning management systems (LMS) is helpful for student learning (Imran and Jawaid 2020). Including guided seminars and courses on biostatistics, digital health literacy, AI ethics, and engineering technologies will help students and physicians be better equipped for the fast-changing medical landscape. Thus, producing future physicians who will be more competent, inventive, and compassionate in the medicine of tomorrow (Frommeyer et al. 2023).

The American Medical Association reports that the current state of AI curricula is lacking, with only select medical schools reporting AI initiatives, mainly concentrated in the most developed nations. When considering developing nations, particularly the Global South, medical curricula are almost copied verbatim, or at least in the largest part, from notable institutions in the West that do not have to deal with problems that this part of the world often faces. We live in a world ravaged by different kinds of inadequate surveillance and response systems, shattered infrastructure, and crumbling health systems amidst frequent destabilizing violence. As such, we need to be taught the importance of having low cost and low maintenance medical equipment which are extremely useful in crises. We are taught how to preserve an X-ray machine, CTs, and MRI scanners, but are not taught how, for example, to manipulate available resources to match the need. Including AI in education but failing to train in the most basic medical maneuvers amounts to incomplete training to say the least. As noted by Fanon (1961, 159), education is “opening their minds, awakening them, and allowing the birth of their intelligence” which is exactly what this part of the world needs.

Towards a Frankenstein Syndrome: Ethical Concerns of AI

Ethical Concerns in AI Applications

Machine learning experts played a role in developing a system that allowed renowned astrophysicist Stephen Hawking to speak. To Hawking and to all of us who heard and read his books, this was akin to a miracle. Yet, in an interview aired on BBC on 14 December, he noted that while the initial forms of AI built so far have been quite useful, there are concerns about the outcomes of designing something that can be almost equal to, or surpass, humans “who are limited by slow biological evolution, couldn't compete and would be superseded.” In her Davos speech, British former PM May promised safety and ethics in AI development, but could she or anyone else really live up to such a promise? The fact is that AI is fraught with ethical concerns and controversies of which we will mention a few. A recent Open Letter from the Future of Life Institute called for a 6 month halt on any further development of AI systems more advanced than GPT-4, the latest language model hosted by OpenAI. The letter stated that “Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable” (Russell et al. 2023).

Misuse of AI

Indeed, while AI has the potential for great benefit through its main purpose of helping people lead longer and flourishing lives, it has the capacity to be misused or abused for evil and harm. Examples of this include covert surveillance systems used by authoritarian regimes to oppress dissents, militarized killer robots, and discrimination and bias in the data fed to the programs in training, as this is translated to discrimination in AI decision making. Other concerns include AI leading to lack of privacy, transparency, misuse of personal data, and potential loss of informed consent as data can be collected secretively; a loss of human contact and empathy, and a lack of accountability and liability.

Privacy Concerns and Accountability

With millions of lines of code in each “node” of an AI’s “brain,” it is currently impossible to know what values are inculcated in the software and how algorithms reach decisions. Thus, ensuring transparency and privacy is crucial because we cannot understand “why” or “how” the machine is taking the decisions it is taking mainly because of the decentralized processing of information within the network: no single “node” contains fixed information, but the entirety of the network forms the decisions, and a piece of information cannot be localized to a node. If said AI algorithms make a mistake, who is held accountable? Some thinkers wondered whether AI might someday become conscious and if so, would they earn personhood? If this is achieved, can said “persons” be held accountable for their decisions, or will responsibility still lie with their designers? Technical safety is thus another major concern, as writing very detailed contracts that limit liability might legally reduce a manufacturer’s accountability. However, from a moral perspective, the contract can be seen as an unethical scheme to avoid full responsibility. In general, the more powerful a system or entity is, the more transparent it should be, and the weaker it is, the more privacy it is entitled to (Green 2020).

ChatGPT and Large Language Models (LLM)

A large language model (LLM) is a type of AI algorithm utilizing deep learning techniques and massively large data sets to understand, summarize, generate, and predict new content. It is designed to sound “human and coherent” but not necessarily generate accurate or true claims (Kerner 2023). A series of prominent cases of ChatGPT misuse have prompted the Europol to issue a statement about the risks of developing such systems and releasing them to the public. “ChatGPT's ability to draft highly realistic text makes it a useful tool for phishing purposes,” Europol said. With its ability to reproduce language patterns to impersonate the style of speech of specific individuals or groups, the chatbot could be used by criminals to “target victims by allowing users to generate and spread messages reflecting a specific narrative with relatively little effort,” the EU enforcement agency said (Foo 2023). Additionally, the software has frequently been seen citing articles that do not exist as a backing for its claims, which is dangerously frightening given the already rampant pandemic of misinformation that is enabled by the web. If we are facing such critical problems and concerns with a relatively simple software like ChatGPT—a language processing model—imagine the massive risks that many times more advanced medical AI may pose in computer vision and diagnostic systems, in a medical sphere no less.

AI Dependency and Reliability

A dependency on AI is a dangerous phenomenon, especially in the medical field since AI requires electricity and internet connection/hosting networks. Hijacking or malfunction of such systems poses a serious threat (Green 2020). There is also an important shortcoming of AI design that is less publicly discussed and is crucial to any application of AI in safety, transport, identification, and especially medical diagnostic systems. This is the concept of AI brittleness—the limitation of AI where it is unable to generalize a set of provided information to accurately decide upon a circumstance, image, or a novel scenario. An example is the inability to identify the number “9” in different persons’ handwriting, and inability to identify a skin lesion on persons with different shades of skin, even if the shades differ on the pixel level. This is because computer vision—the “eyes” of AI—is a color analysis of pixels with pattern detection to decide upon the nature of the object from previously encountered data (Cummings 2020). This phenomenon increases the likelihood that a tried and trusted AI may make mistakes, and due to its reputation of accuracy, skews the decision of less-experienced professionals. Indeed, a recent study showed that the use of an advanced AI in dermatological neoplasm identification was significantly efficient only in less-experienced trainees, such as early residents (Han et al. 2022).

This is only the tip of the iceberg on a bigger problem that integration of AI in clinical medicine may pose, the worsening imbalance of power that such technologies will create. While prices may decrease due to lowered cost of production, those who control and host the AI development and operations will also likely make more money and have more influence. Besides, the contention of control, hosting, and use of data and systems will lead to negative impacts on vulnerable groups, bias insurance and healthcare systems, and reduce freedom of information. Such violation of the fundamental human rights of end users will widen the gap between the so-called developing countries and the so-called developed countries with AI surfacing as a new colonial project with massive impacts on world power balance (Stahl 2021; Green 2020).

Socio-economic Issues related to AI

There are additional and unique concerns for the use of AI in healthcare and medical education in countries currently labeled as “third world” and developing countries where financial and import restrictions lead to shortage of medicine and basic supplies. Matters become even graver when it comes to countries in protracted conflict with a frail infrastructure, shortage of staff, lack of basic surgical and medical tools (the Gaza Strip for example suffered a lack of stethoscopes even before the siege), shortage of beds, etc., making some medical diagnostics and operations a Herculean task. Shortage in equipment sterilizers, adequate supplies of suction pumps, supplies of sterile gloves, extension tubing, three way stop-cocks, large syringes, linens and anesthetic agents, monitoring devices, and X Ray machines are only a few of the shortages that physicians face in the Global South. Consequently, although AI is around the corner and healthcare workers need to be trained in this area, it remains a fact that one should also reflect on the (additional) nature of the medical education that needs to be acquired in areas of conflict. We graduate health professionals who are unprepared for the most pressing of diseases, casualties, and surgeries they cannot deal with in times of conflict or scarcity because the medical curriculum does not teach them how to handle such situations.

Disproportionate Climate Impact

A quick look at the institutions integrating AI into their curricula reveals a gap that disproportionately affects developing nations, widening the digital divide. While AI carries the potential to bear much benefit, it is very expensive to research and develop AI software with data hungry neural networks often requiring millions of examples to learn from. This requires a complex infrastructure of data storage and modern computing hardware that consume massive energy and release huge amounts of carbon emissions. Thus, making it unaffordable for developing nations, and exacerbating the disproportionate effects of climate change on these lower-economically abled nations (Chatterjee and Dethlefs 2022).

While AI implementation is theoretically beneficial for developing nations, it must be considered in appropriate contexts. Majority of the Global South is witnessing wars, droughts, food shortages, and lack of political stability; thus, there are various more pressing issues to be tackled prior to investment in AI technologies in medical education and in industries. This further increases disparities between healthcare centers in advantaged vs. non-advantaged countries in access to healthcare and in residency program opportunities when applicants want to match from a country where AI is not implemented. Such gaps limit social and work mobility between nations, as students are trained on different techniques and devices, an especially relevant factor in medical education, since most procedures are standardized globally. Less advantaged nations will need to outsource not only the hardware and software from richer nations where the AI programs are developed, but they will consistently need to import expertise and repair for these instruments, due to lack of underlying infrastructure and training.

Contextual Bias in Development

Until now, the implementation of AI has been largely focused on high-income settings and socioeconomic groups with high digital literacy. Furthermore, where AI-based tools are developed for low-resource settings, they may not be applicable or appropriate given a HIC-centered approach to development and implementation. Teaching of AI development needs to be provided to those who are embedded within the systems that will use it, thereby ensuring the local applicability of these tools. According to the data, a good place to start is for healthcare providers in LMICs to be provided the foundational technology to fix and repair AI systems in the future. Next, they should be offered the education and resources to strengthen AI literacy and to support advances in the implementation of AI-based tools in their healthcare systems (Ejaz et al. 2022).

Where do we Go from Here?

While we do not discourage the use of AI, we call for ensuring that to the most possible extent, appropriate, safe, ethical, and fair AI is developed including safeguards to thwart the abuse of AI and protect against the above-mentioned concerns. The globally flourishing interest in machine learning and artificial intelligence and resulting accelerated development and applications have prompted many regulatory organizations to issue both warnings and call for increased guidance moving forward. The WHO recently released an “AI Ethics and Governance” document that encompasses AI applications in various fields, including healthcare. Said document raises many relevant considerations and calls for the creation of more robust and detailed guidance frameworks and regulatory systems to ensure the safe and ethical implementation of artificial intelligence in healthcare settings. While it puts forth the considerations relevant to such applications, it does not provide an explicit framework, rather relegating the task to “health regulatory authorities responsible for ensuring the safety, efficacy and appropriate use of technologies for health care and therapeutic development” (WHO 2021). On the other hand, McCradden et al. (2022) put forth a suggested ethical framework to govern clinical translation of machine learning applications through drawing parallels to the very same human rights protections offered in human subject research. As Ho and Malpani (2022) pointed out in their commentary on the subject, “traditional paradigms relating to ethical rigor and the protection of human participants could be adapted to respond to rapid developments in ML healthcare” (Ho and Malpani 2022). Such global frameworks and ethical rigor are especially crucial in development of LMIC applications of AI in healthcare, as there may be reduced human rights protections to begin with, reduced digital literacy rates, and a general lack of expertise for the local oversight of said protections in ML development. The main concern is that whatever governance currently exists remains to the advantage/rooted in the values of affluent countries where majority of cutting-edge development is happening. Thus, in order to ensure such protections against the concerns raised in this paper are fulfilled, we suggest the following:

Regulatory

  • AI development teams must include a global panel of ethicists who collaborate with corporate decision makers and software developers to develop a code of AI ethics that lays out how various issues will be handled in low-resource settings. As such, global expertise should be united under one umbrella such as the WHO, in the hope of addressing these issues in a systematic and regulated manner at the developmental stage.

  • The establishment of an AI review board that regularly addresses corporate ethical questions and assesses the development and operation of such systems at every step where concerns may arise, particularly in cases of adapting applications to LMIC settings.

  • Having AI audit trails showing how various (known) coding decisions are made will protect against ambiguity of liability and allow easy rectification of errors.

Training

  • Implementation of AI ethics training programs allows staff to operationalize ethical considerations in their daily work and provides a means for remediation when AI solutions inflict harm or damages on people or organizations (West 2018).

  • AI training must be developed also with oversight from a global panel to ensure that providers understand and maintain data protection requirements, patient privacy, and confidentiality, as well as minimize biased sampling and to counteract epistemic injustice. This includes AI developers’ provision of transparent communication and actions addressing these endeavors.

  • Further collaboration among interprofessional educators will be necessary to clarify the additional competencies needed and refinement of historical competencies considering increasing penetration of AI in healthcare (Lomis et al. 2021).

Programming

  • Increasing the implementation of uncertainty quantification in the design of AI systems. This recent development in the field of neural networking and computer vision allows the system to not only suggest a decision but also tell the operator how certain it is of that decision. This minimizes AI brittleness, bias, and skewed conclusions by warning the operator of low certainty decisions, thus increasing caution in accepting outputs (Dera et al. 2019; Chua et al. 2023). This improves both reliability and accuracy, particularly in settings where there may be scarcity of experienced radiologists or advanced training, such as LMICs (Cifci 2023). Additionally, such uncertainty quantifications afford a higher level of explainability and accountability in LMIC settings where decisions (ethical or clinical) may need to be viewed through a regionally and culturally-relativistic lens.

Conclusion

With the recent advances in computer science and informatics, AI is becoming part of modern healthcare. AI algorithms and other applications driven by AI are being used to assist healthcare workers in clinics and in research. Notwithstanding, AI can be both a blessing and a curse and potentially a double-edged sword if not carefully designed and wielded. While it holds potential to benefit humans particularly in healthcare by assisting in both clinical and administrative processes, easing the lives of patients and healthcare workers alike (Daley 2021). The misuse of AI also holds potential for harm through impact of biases, unemployment, breaches of privacy, and lack of accountability (Stahl 2021). Not to mention inability to handle stringent sensitive medical operations in areas of conflict and disasters (South Sudan, Yemen, Afghanistan, Somalia to mention but a few). Ethicists have an important role in ensuring that AI is implemented in accordance with ethical standards to minimize risk of harm, while allowing maximization of benefit (West 2018), as well as to play the role of a gadfly and awaken the AI blind enthusiasts from their slumbers. This role includes participation at every level of development from design to coding, implementation, operation, and remediation. Ethicists are to be involved in creating a strict code of AI ethics, and a global advisory or supervisory board to monitor and regulate all activities in collaboration with other major stakeholders. Frameworks for the development of AI through research have been suggested by the WHO and McCradden et al. (2022). Similar regulation can be designed and put in place to ensure legal protections and ethical protections not only through the developmental phase of AI applications for healthcare but also in implementation, and successive conflicts or issues that may arise. The question remains, who will ensure that ethics and ethicists are involved in AI development? Will these considerations be shunned for sake of profit and convenience? Will ethicists be able to sway and guide development of AI for the benefit of all humans, equally? These are questions that only time can answer, but with greater advocacy and public awareness, we can ensure a future that includes AI, while protections for humans are afforded. Hawking, a person whose life and reputation rested on AI, reminds us that when the AI revolution “eventually does occur, it is likely to be either the best or worst thing ever to happen to humanity, so there’s huge value in getting it right” (Griffin 2015). The goal of AI must shift from creating pure undirected AI to creating beneficial intelligence. Figuring out how to do this will take decades of research, so we must begin this today rather than the night before the first strong AI is switched on (Griffin 2015). Although Prometheus “wished no living thing to suffer pain,” he was punished for transgressing and every day an eagle would eat part of his liver only for it to grow again and endure the punishment for eternity.