AI for Cyberbiosecurity in Water Systems—A Survey

Abstract

The use of Artificial Intelligence (AI) is growing in areas where decisions and consequences have high-stakes such as larger scale software, critical infrastructure, and real-time systems. This transition in recent years has been accompanied by the growth of research in AI assurance in fields such as ethical, explainable, and trustworthy AI. In this work, we survey the literature to find the state of AI assurance for cyberbiosecurity systems as they exist now, particularly for water and agricultural supply systems; future directions are also presented. We focus on papers at the intersection of cyberbiosecurity, AI assurance, and water/agricultural supply systems, discuss how assurance techniques improve these systems, and provide pointers for future research into the application of AI for the cyberbiosecurity field. Current cyberbiosecurity solutions do not focus much on AI, but existing AI solutions for water supply and cyber or Cyber-Physical Systems (CPS) exist and can be applied to benefit cyberbiosecurity. The inclusion of AI assurances help alleviate issues of applying AI to high-stakes human-centered infrastructure.

1 Introduction

The deployment of AI is outpacing the adoption of assurances that commit to its responsible use as policies and regulations lag behind. Assurances validate AI systems to assess the risk of failure, misuse, and even abuse, helping establish the trust needed for the adoption of AI. The risks of AI in infrastructure (e.g., agricultural supply chains, biological systems, and water supply systems) are significant, potentially affecting millions of citizens and resulting in loss of life, well-being, and economic opportunity.

For example, take a city-wide water distribution system that pumps in water from a reservoir and ensures every citizen has equal access to drinkable water. Imagine the city adopts an AI system that predicts demand and supplies regions of the system as needed. The system works fine to start, but years later it is not properly validated after new pumps are installed, so the sensor data changes and no longer predicts accurately. As a result there are large swaths of the city that are no longer receiving drinking water because the system forecasts are off. Or, maybe the system was trained with bias data because poorer neighborhoods had less data collected, so the system favors keeping the water supply greater for affluent regions resulting in poorer regions having intermittent supply issues.

For this water supply AI system to work properly assurances must validate outcomes are correct, fair, and that users can understand why the system has made its decisions. These concepts form the basis of AI assurance, which details the broad ways of verifying and validating AI systems, much the same way that traditional programming software (i.e., not machine learning) is verified and validated during its development process [1]. AI assurance applied during development would help avoid the mentioned issues of robustness and bias.

Water supply systems are a form of CPS, as physical sensors, pumps, and tanks act as data collectors to track the flow of water and relay data to a central computer. This data processing exposes the water supply to cyber-attacks. Additionally, water supply systems are part of the bioeconomy (the supply chain infrastructure that is tied to critical commodities like food, water, and medicine) meaning any impact to the system can have an effect on the livelihood of thousands or millions of people. The imagined water supply AI not only ensures proper water distribution, but there are additional security concerns, moving it into the relatively new realm of cyberbiosecurity, which is a discipline at the intersection of life science and information technology (IT) [2]. Cyberbiosecurity is defined in greater detail in Sect. 1.1.

Existing cyberbiosecurity research mostly focuses on the IT side of biology, or cybersecurity for biology labs and databases is a succinct way to put it. The cyberbiosecurity field, however, is lacking much research in applied AI for supply chain infrastructure, as most papers only identify vulnerabilities and propose high-level frameworks for addressing them. Our goal for this survey is to find papers at the intersections of cyberbiosecurity, AI assurance, and water and food supply systems and connect that to the bioeconomy. Our work searches for and discusses the applications of AI assurance to existing solutions within the cybersecurity and CPS to help ensure the proper function of cyberbiosecurity-related systems.

1.1 Relevant Terminology and Definitions

Proper use of AI assurances verifies and validates the outputs of those systems, convincing users that they are reliable. AI assurance codifies the process, so when changes occur to the water supply system, validation can be re-run to satisfy the AI is working properly or needs to be retrained. Definitions are intentionally broad in order to apply them to a wider range of applications. From Batarseh et al. [1], AI assurance is defined as:

A process that is applied at all stages of the AI engineering lifecycle ensuring that any intelligent system is producing outcomes that are valid, verified, data-driven, trustworthy and explainable to a layman, ethical in the context of its deployment, unbiased in its learning, and fair to its users.

The importance of AI assurance is that it applies a process to all stages of the AI lifecycle, from the start of development all the way through deployment. Assurances are not merely tests of AI to check some boxes that it is okay to use. In order to trust the AI is working properly engineers need to validate it meets all the criteria of assurance:

• Ethical—the AI system can make “right” decisions that benefit the people impacted and not just the people in power of the technology [3].

• Fair—the AI system makes decisions without considering demographics, backgrounds, affiliations, or individual preferences (i.e., does not inherently value some citizens over others).

• Safe—the AI system ensures the life and well-being of those who are using it and impacted by it.

• Explainable—the AI system can explain, or be interpreted, to understand why it came to a decision or how the algorithm works.

• Secure—the AI system can prevent or mitigate attacks or other threats to the proper operation of the system.

• Trustworthy—users have confidence the AI system works properly.

For infrastructure systems in the bioeconomy, AI must be ethical to make the right decisions, safe to protect users it potentially impacts, explainable so humans can understand it, fair in the decisions it makes, trustworthy so we have confidence in its abilities, and secure to prevent cyber-attacks and threats.

The bioeconomy refers to the sector of the economy that relates to research or innovations in the life and biological sciences and fields related to biotechnology [2, 4,5,6]. This sector grows as progress continues in technology relating to computing and information sciences [7], including most crop production, especially as big data, AI, and machine learning become more involved for enhancing land use and water management via precision farming [4]. As the bioeconomy grows, cyber threats against it increase and require mitigation to safeguard investments in the bioeconomy [8].

Richardson et al. [2] described cyberbiosecurity as the intersection of IT and life sciences, but Duncan et al. [9] specified it further as the intersection of cybersecurity, cyber-physical security, and biosecurity. Each discipline with its own existing challenges and new vulnerabilities appearing where they overlap.

By its nature, cyberbiosecurity is grounded in IT and with that brings the risk of cyber-attacks. This is the traditional realm of cybersecurity, or the shielding of computer networks and information from damage, exploitation, and unauthorized use [10,11,12,13,14,15]. Linking any computer system to a network increases risk. This is compounded in the bioeconomy as more remote monitoring and controlling is added to existing physical infrastructure, because of this interaction of cyber and physical the security needs “safety and reliability requirements qualitatively different from those in general-purpose computing.” [16]. A CPS integrates digital computing and physical processes, where a network monitors and controls a physical system via sensors and actuators, to interact with the real world [16, 17]. Communication and networking multiple devices is important because the components are often disparate and there is a back and forth of physical processes affecting the computer and vice versa, but this opens new vulnerabilities [16, 17].

The third aspect of cyberbiosecurity moves fully into the physical space for securing biological systems. Biosecurity is the protection of any form of life from the threat of disease and pests, including the protection of agriculture and food, or simply put the “re-branding of the centuries-old battle with disease” [18,19,20]. This includes threats that are natural, such as livestock and crop diseases, or intentional attacks, such as the deliberate use of smallpox and anthrax weapons [18]. The incorporation of biosecurity in the realm of cybersecurity and cyber-physical security is what sets cyberbiosecurity apart.

Traditional cyber-attacks are not necessary to impact biological systems, because there are physical, biological interactions outside the computer systems. We need to ensure that the biological aspects are operating properly, be it from natural causes (diseases, pests, etc.) or intentional cyber and physical attacks. There are three layers of interactions to protect: the cyber, the interactions of cyber and physical, and the biological.

Included in these biological systems are water supply systems, which can refer to distribution, treatment, agricultural, or storm water systems. Distribution systems control the transport and delivery of water through a network of pipes and pumps to ensure consistent supply, they are focused on the logistics of water transportation and storage. Treatment systems take raw or wastewater, unsafe for humans or the environment, and through a series of chemical and biological processing, filtering, and sanitizing produce either safe drinking water or water that can be released into the environment. Agricultural water systems focus on the distribution of water to crops and livestock. Unlike distribution systems, this water does not have to be safe for human drinking, but it must ensure the production of food for human use. This also closely ties agricultural water systems to food supply systems. Finally, storm water systems deal with the drainage of runoff water to prevent flooding or contamination of other water systems from the pollutants that it picks up.

These systems allow for the automation of critical infrastructure by adding more technology for monitoring and controlling human and agricultural water use. These water and food systems are not only cyber-physical but also biological as well. Their proper functioning is required for human livelihood, either through the supply of safe water or the growth of adequate food supplies. Water and food systems are cyber-physical and bio-infrastructure systems that are open to attacks (cyber and physical) and anomalies (such as maintenance issues, severe weather, sensor or equipment breakdowns).

Going back to our hypothetical city-wide water distribution system. If it were attacked by a bad actor who wanted to poison the water, they could give commands to add too much of a chemical or too little of a cleaning agent that would result in undrinkable water. In fact, there was an attack in 2021 on a Tampa, Florida water supply system where attackers increased the levels of lye in the water by 110 times before they were stopped [21]. We discuss this example further in Sect. 5.4, but it serves as a great example of the cyberbiosecurity threats to water supply systems. Threats can combine unauthorized access of computer systems to control physical processes; in the Tampa case, the lye controllers pose a biological threat to everyone that relies on the system for safe drinking water. The next section introduces the inclusion and exclusion criteria of the papers surveyed.

1.2 Description of Included Articles

In this survey, we used multiple online repositories and research paper search engines to find relevant papers on the topics of cyberbiosecurity, AI assurance, and water supply systems. Our focus was to find peer-reviewed papers at the intersection of two or more topics. We include papers from journals, conference proceedings, dissertations, books and book chapters, and industry white papers published from 2000 through April 2022. A complete repository of papers included in this study can be found here: https://github.com/AI-VTRC/CyberbiosecuritySurveyPaper.

Key search terms included the following to find papers:

• Cyberbiosecurity; Cyber-Biosecurity; Biocybersecurity; Bio-Cybersecurity

• Water Supply System; Water Distribution System; Water Treatment System; Water System

• AI Assurance (see assurance list in Sect. 1.1)

• Artificial Intelligence

Because cyberbiosecurity is a new research field, we kept search criteria as broad as possible to include enough papers for a survey. Some focus on the medical fields, but we tried to find relevant discussions that could apply to AI assurance or water supply systems as much as possible. Some focus just on the concept of cyberbiosecurity in general, but we focus on how best to apply the concept to AI assurance and water supply systems.

2 Survey Landscape

The papers surveyed for this research included publications between 2000 and 2022 (as of April 2022), but most are from 2016 onward. Figure 1 shows a histogram by publication year, and until 2016 there was not more than three publications per year that covered cyberbiosecurity, water systems, and AI assurance. There is a steady trend upward for the count of publications, and as cyberbiosecurity and AI assurance research continues to grow we expect the number of publications to continue to grow each year.

Fig. 1

Count of the number of publications by year that were used in this survey

Figure 2 shows the breakdown of publications by cyberbiosecurity and water sectors. Publications on water systems had a low but steady trend from the early 2000s until about 2017 when they increased and held since. The year 2017 was also when the cyberbiosecurity term started showing in the scientific literature, and there is a sharp peak in 2019 before cyberbiosecurity publications return to a more steady pace.

Fig. 2

The count of publications by year for the sectors of cyberbiosecurity and water supply (either water treatment or water distribution) systems. Papers are not confined to a single sector, and some are counted both as cyberbiosecurity and water supply papers. Most papers published since 2012, so older publications omitted from this figure

We break down the AI assurance publications by assurance pillars in Fig. 3. Here, a majority of the papers deal with safe and trustworthy AI, especially just before the term of cyberbiosecurity starts showing in 2017. As AI becomes more popular, especially with deep learning (since 2015), we see an increase in publications for all the pillars of AI assurance.

Fig. 3

The count of publications by year for the pillars of AI assurance. Papers are not confined to a single pillar, and some are counted for multiple. Most papers published since 2012, so older publications omitted from this figure

Figure 4 shows a citation graph we created using Citation Gecko.¹ The yellow nodes are surveyed papers, gray nodes are other papers which cite our surveyed papers, and edges (lines that connect the nodes) are the citation link between two papers. The cyberbiosecurity literature is relatively disjointed from the literature on water supply systems and attack/anomaly detection. Most of the AI assurance papers remain independent in this view from each other and other sectors, with the exception of some trustworthy AI papers that form a small network. This graph shows the relative separation of the cyberbiosecurity literature from water system security and attack/anomaly detection (which includes secure AI). There is one citation chain from cyberbiosecurity to water system security via Mueller [22], Schmale III et al. [23], Moyer et al. [24], and Housh and Ohar [25]. (note that Moyer et al. [24] is the oldest link in that chain.)

Fig. 4

Connected citation graph of the papers survey for this work. Yellow nodes are surveyed papers, gray nodes are other cited papers, and edges represent a citation between two papers. The cyberbiosecurity literature is relatively disjointed from the literature on water supply systems, AI assurance, and attack/anomaly detection. Graph generated using and courtesy of CitationGecko https://www.citationgecko.com/

3 AI Assurances for Cyberbiosecurity

In the introduction section, we described cyberbiosecurity as the intersection of life sciences and IT, and to be a little more specific it is the intersection of cybersecurity, cyber-physical security, and biosecurity [2, 9]. One of the best definitions we found is from Murch and DiEuliis [26], who defined cyberbiosecurity as the

understanding [of] the vulnerabilities to unwanted surveillance, intrusions, and malicious and harmful activities which can occur within or at the interfaces of commingled life and medical sciences, cyber, cyber-physical, supply chain and infrastructure systems, and developing and instituting measures to prevent, protect against, mitigate, investigate and attribute such threats as it pertains to security, competitiveness, and resilience. (emphasis ours).

It is the vulnerabilities at the intersections of these cyber, physical, and biological systems that make cyberbiosecurity what it is, complex interactions between machines and biology that are open to disruption. This interaction creates unique vulnerabilities open to biological systems that make detection, attribution, and mitigation difficult in a timely manner [27]. Bernal et al. [28] recreated a Distributed Denial-of-Service (DDoS) attack using bacteria “engineered to act as biosensors” in a novel cyberbioattack, demonstrating the unique risks of the field and that traditional cybersecurity measures are not always adequate for cyberbiosecurity applications. The literature addresses these issues with a widespread call for action and collaboration—“We call for analyses and publications to fully scope cyberbiosecurity and identify a comprehensive strategy to establish the discipline’s goals and objectives” [2] and others, as called out by [29] and seen in [26].

The purpose of our survey is to find how cyberbiosecurity intersects with AI assurance; there are applications that go beyond applying security to biological applications, and here we are interested in answering the question: what makes cyberbiosecurity different than cybersecurity for biology? It is the assurances a cyberbiosecurity system brings to the continuing function of the bioeconomy and relevant infrastructure. This is summed up well in the paper from Schmale III et al. [23], and while cyberbiosecurity is only mentioned briefly, the goal of the water supply system discussed is to ensure the safety of the drinking water from naturally occurring harmful algal blooms and cyber-attacks. Cyberbiosecurity “models must capture the physical dynamics of the system as well as the cyber-interconnections” [23].

Cyberbiosecurity systems that deal with supply chain and infrastructure systems have, or the potential to have, large impacts on the livelihood of people who rely on the system. All the residents of a city rely on its water distribution system to bring them water for drinking, cooking, and cleaning. A break down is not merely inconvenient but could be life-threatening, especially if the system is down for a long time or the water is contaminated. Even if AI is not considered for a cyberbiosecurity system, assurances are important to what cyberbiosecurity attempts to accomplish. AI brings an opportunity to add security or corrective actions in the event of any issues, and AI assurances validate their use for cyberbiosecurity applications. The end goal of any assurance (AI or not) is validating and verifying a system is working properly, so people have trust and adopt that system for use.

Turning back to the example of a water distribution in a city, suppose an AI monitors the system for cyber-attacks or natural anomalies (e.g., low levels from draught, bacterial growth, broken equipment, etc.) and takes corrective actions. If the hypothetical water distribution AI meets all the criteria listed in Sect. 1.1, then there is assurance that it behaves in a way that benefits everyone it impacts (people in the city who rely on the system providing drinkable water on demand) and minimizes unintended consequences. There is also some assurance the AI mitigates issues or threats to the system that would endanger city residents.

All these AI assurances are relevant to cyberbiosecurity, especially the secure assurance because the objective of cyberbiosecurity is “understanding the vulnerabilities” and developing “measures to prevent, protect against, mitigate, investigate and attribute such threats as it pertains to security…” [26]. There is also the human side of cyberbiosecurity, Perakslis [30] included the field in their list of public interest technologies, which are technologies that focus on public good. Further emphasizing the need for assurances to validate any AI systems involved with cyberbiosecurity and help promote their adoption in cyberbiosecurity. AI systems need to be trustworthy and explainable so people want to use them knowing they can rely on them to operate correctly, and because cyberbiosecurity systems focus on biological systems, safety is a big issue in order to ensure people impacted are not threatened by AI making a wrong decision. Ethics and fairness are a large part of the safety assurance too, as AI needs to ensure it does not favor some people over others, that it is not designed to favor its developers and investors over everyone else. Ethics and fairness are ensuring equal safety for everyone impacted.

4 AI Assurances for Open-Source Water Supply Testbeds

Open-source information engages more researchers allowing them to build better tools, frameworks, and operational systems such as Git, PyTorch, or Linux. Similarly, open-source testbeds allow the community to contribute, propose, test, and improve upon ideas. Lack of real-world water and CPS datasets prevented significant research in security of these systems [31]. Data from real facilities cannot be shared for both security concerns and lack of accurate ground truth, so the availability of reliable, open-source water testbeds is critical for research. Open-source datasets also allow hands-on experience and training scenarios needed for collaboration and understanding the security requirements of these systems [32].

Assurances for water systems closely match those of cyberbiosecurity systems discussed in Sect. 3. The two major assurances are the safety of the water quality and the security of the system’s operations. Explainability is another key assurance for water systems, so we can understand how the water and AI systems operate in order to ensure consistent and safe water supplies. This emphasizes the importance of open-source datasets to help the AI research community better understand the operation of water systems and develop explainable and interpretable AI that is open to the water industry. Here we present some open-source water distribution and treatment system (as defined in Sect. 1.1) testbeds available to researchers across the world [33].

4.1 Secure Water Treatment (SWaT) Dataset

SWaT is a scaled down water treatment plant with real cyber and physical equipment to investigate cybersecurity research, which started in 2015 by Singapore University of Technology and Design [31]. The testbed consists of a six-stage water treatment process with modern-day components. The data collected from the testbed consists of eleven days of continuous operation, including seven days’ worth of data under normal operation and four days’ worth of data under attack. All network traffic, sensor, and actuator data was stored in the database.

4.2 Water Distribution (WADI) Dataset

Due to the success of the SWaT testbed, Singapore University of Technology and Design launched WADI in 2016 as an extension of SWaT to form a complete water treatment, storage, and distribution system [34]. Similar to SWaT, data collected for the WADI testbed consists of sixteen days of continuous operation, including fourteen days’ worth of data under normal operation and three days with attack scenarios. All network traffic, sensor, and actuator data were collected.

4.3 Battle of the Attack Detection Algorithms (BATADAL) Dataset

The BATADAL dataset is not based on real-world data, though it is considered realistic since it was constructed using the de facto standard simulation tool for water distribution system modeling, namely the open-source Matlab software package EPANET [35]. EPANET is a Windows based software application for simulating and representing water distribution systems used world-wide by engineers and researches to design new water infrastructure, update existing water systems, and develop more efficient solutions to solve water quality problems. The BATADAL dataset was constructed for a competition to compare the performance of algorithms for the detection of cyber-attacks on water distribution systems. BATADAL simulates a fictional C-Town water distribution network, first introduced for the Battle of the Water Calibration Networks by Ostfeld et al. [36]. C-Town is based on a real-world, medium-size network which contains 388 nodes, 429 pipes, 7 tanks, 11 pumps, and one actionable valve.

4.4 Modbus Penetration Testing Framework (Smod) Dataset

Laso et al. [37] created the Smod dataset was produced in 2017 to investigate how data and information quality estimation can detect anomalies and malicious acts in a CPS. The data were acquired using a cyber-physical subsystem consisting of liquid fuel or water containers, along with its automated control and data acquisition infrastructure. The data consist of temporal series representing five operational scenarios—normal, anomalies, breakdown, sabotages, and cyber-attacks—corresponding to fifteen different situations. To acquire the data, Laso et al. [37] used two tanks of different volumes for storage, one ultrasound depth sensor, four discrete sensors, and two pumps.

4.5 Digital Hydraulic Simulation (DHALSIM) Framework

DHALSIM is an upgraded framework of the BATADAL Framework, which uses the Water Network Tool for Resilience (WNTR) EPANET wrapper to simulate the behavior of the water distribution systems [38]. DHALSIM uses Mininet and MiniCPS to emulate the behavior of the Industrial Control System (ICS) controlling a water distribution system. This means that in addition to physical data, DHALSIM also provides network captures of the Programmable Logic Controller (PLCs), Supervisory Control And Data Acquisition (SCADA) server, and other network and industrial devices present in the system. Similar to BATADAL, DHALSIM can be integrated into a C-Town Network, using a Mininet network that connects the C-Town PLCs and SCADA servers through Local and Wide Area Networks (LANs and WANs). In DHALSIM, each ICS equipment is a Mininet node running a script that represents the behavior of such equipment. In the C-Town network PLCs have private Internet Protocol (IP) addresses and NAT and port forwarding is used to connect the LANs.

4.6 Datasets Comparison

Figure 5 compares the number of total citations (labeled “General Citations”) to the number of cyberbiosecurity citations (labeled “Cyberbiosecurity Citations”) for the five datasets above. We obtained the number of cyberbiosecurity citations and general citations by counting the numbers of papers citing these datasets in our survey and by the count of citations from Google Scholar, respectively. We see the SWaT dataset is used the most, while DHALSIM dataset is used the least in both types of citations. This difference could be explained due to the early deployment of the SWaT dataset and the continuing collection and publishing of more data to that dataset by the University of Singapore in the years since its initial release. Although SMOD, BATADAL, and WADI are all water distribution systems published in 2017, the SMOD dataset is used significantly less. This could be explained by the scale of the datasets, specifically, both BATADAL and WADI simulate water distribution systems of large towns with multiple sensors, nodes, pipes, and a large recording time. On the other hand, SMOD only simulates a two-tank system, although SMOD is focused on different attack and anomaly scenarios than BATADAL and WADI. This shows that the research community prefers a dataset that can simulate a large scale, high quality real-world water distribution systems (WADI and BATADAL) and water treatment plant (SWaT) as benchmarks for model development.

Fig. 5

Comparison between five open-sources water datasets in term of data usage

5 AI Assurance Pillars

AI offers both opportunity and risk to cyberbiosecurity systems. It has the potential to detect and mitigate cybersecurity threats [2, 39,40,41,42], but at the same time offers an avenue for attacks [43,44,45], such as “poison” and “evasion” attacks on data or “inversion” attacks on AI models [43]. The current state of the cyberbiosecurity literature, however, focuses more on creating awareness and calls for collaboration to mitigate security threats rather than discussing the direct use of AI or AI assurance.

This supposition is not uniform, as Reed and Dunaway [40] praised the use of AI to “assist decision making… through the identification of cyberbiosecurity vulnerabilities and by providing recommendations for their elimination and/or mitigation.” AI already brings a lot of benefit to the field of cyber and cyber-physical security, so the extension to cyberbiosecurity seems inevitable. However, with different physical, biological, and safety considerations required for cyberbiosecurity, there are no guarantees of success. This is where AI assurances come in to play a role, as they can help validate AI systems function as intended and aid in the responsible adoption of AI for the field of biology [1, 2, 46].

The multifaceted issues and solutions cyberbiosecurity systems face require interdisciplinary teams [47]. Solutions, therefore, cannot only be technical but require just as much of a human element [2, 47,48,49], and this is a more common topic in the surveyed papers than direct mentions of AI for cyberbiosecurity.

Assurances aid the adoption of AI by evaluating them for the benefit of humans and not because they make a solution more efficient, cheaper, or faster. The pillars of assurance are ethical, fair, safe, secure, explainable, and trustworthy. With the exception of secure, they are completely human focused. Clark et al. [48] claimed that cyber-defense is comprised of three aspects: technology, people, and physical protection and that these applications rely on people merging their knowledge rather than solely relying on automation. AI assurance is the way of merging the technological solutions of AI with the human values of the people within the cyberbiosecurity ecosystem. Aguilar et al. [49] argued a more holistic approach is required to solve the issues with the bioeconomy, one that includes “science, technology, economy, environmental issues, rural and industrial development, regulatory processes and social sciences.”

5.1 Ethical and Fair AI

The most important question we can ask about AI is whether it works as intended or not. If not, how bad can the results be? And what kind of measures can we take in case of such a failure? In March of 2018, “an autonomous car operated by Uber—and with an emergency backup driver behind the wheel—struck and killed a woman on a street in Tempe, Arizona. It was believed to be the first pedestrian death associated with self-driving technology” [50]. This incident is a crucial example of when AI fails to make a safe decision. Although writing detailed contracts can legally reduce a manufacturer’s liability, it might be morally unethical for the company to avoid legitimate liability.

With the growth of AI there are ethical and legal concerns regarding technology in areas, including how we can eliminate AI biases, ensure privacy, facilitate safety, and much more. AI should be made trustworthy, should be created and used with “an ethical purpose,” and created to do good in society, but there are lots of questions that come up with AI and robots, such as if we “[assume that] the robots cannot be morally responsible—who will be responsible?” [51]. Furthermore, AI is already used in automated decision-making, and in high-stakes scenarios their decisions can be impactful. One issue with algorithmic decisions is bias, which can be “cognitive biases of programmers,” “unrepresentative datasets used for training,” or “bias in the data used to make the decision” [51]. It is just as important to start with ethical considerations before AI is designed, let alone deployed, to ensure it is making fair and ethical decisions [51].

The concerns of inclusive, equitable, and correct decisions from AI are not solely left to industry, in fact it is gaining more ground in research from large tech companies and academics. The ambiguity of “fairer” decision-making systems, however, leaves fair AI as a broad open ended question without a real solution. Besides defining what “fair” means, researchers must deal with how to train systems for fair decisions or the fact that systems made fairer for one group can result in bias against another.

One of the most common reasons for biased results is the under-representation of certain groups within a dataset. Increasing the representation of that group, for example, oversampling a certain demographic in certain areas predominantly held another, may be a solution to rectifying the data. When it is not possible to modify or edit data, the objectivity of the decision-making process can be resolved by adjusting the AI algorithm. For algorithms that learn from discriminatory practices it is possible to change the internal weights in a way that makes decisions more neutral. It is also possible to modify the decisions of AI algorithms directly to create more equitable outcomes.

In some instances, it is not the lack of representation, but rather, the over-representation on certain groups that can created biased results. In such fairness related cases, openness in the development and deployment of AI is required [52,53,54,55].

In short, it is possible for AI technologies to be more equitable, but this requires the cooperation of different stakeholders and a lot of work. Arnold et al. [56] pointed out the importance of ethical decision-making while raising critical questions for every AI developer. The authors also refer to relevant answers for these questions from the literature, making this article serve as a guidebook for comprehensive AI assurance deployment.

Laplante et al. [57] investigated the causes that lead to unethical AI and its potential results. The authors saw the main reason as unbalanced or underrepresented data. [57] also emphasized the importance of ethical considerations for AI over its importance for classical software.

Zicari et al. [58] provided a framework to assess the trustworthiness of AI systems. The parameters the authors investigated include, but are not limited to, ethical and fair AI. The article provided a lifecycle to ensure ethics in AI decision-making. The authors emphasized the required absence of conflict for a reasonable assessment of ethical AI.

Grady et al. [59] proposed an epistemic, ethical analysis framework; as the name suggests, the authors proposed ways to detect and analyze ethical issues in cyber-physical infrastructures including, but not limited to, water treatment and distribution systems. The article investigated the importance of ethical decision-making and the roots of the problems in this topic.

Freeman et al. [46] proposed a framework to investigate AI using AI assurance metrics. The authors brought together many AI measures on common ground in this work, challenged the readers, and provided answers to these AI assurance problems.

Calvo et al. [60] investigated the algorithmic, environmental, and human impact assessment of AI systems. They proposed a measurement algorithm called Human Impact Assessment for Technology (HIAT) and discussed ways to build trust into the algorithm using this method.

5.2 Safe AI

One goal of cyberbiosecurity is ensuring the safety and well-being of those impacted by the system. This stems from the biosecurity aspect of the field [61] but naturally extends to any form of safety ensured by systems like water and food supply chains (and agriculture [62] as an aspect of these supply changes). The goal of the safe AI assurance is for AI to guarantee some level of safety to ensure the life and well-being of anyone impacted by the AI. These two forms merge to, as Mueller [22] described cyberbiosecurity, develop, validate, and implement safety measures.

Physical consequences, including harm to humans, are what separates cyberbiosecurity from most forms of technological security. Walsh and Streilein [43] pointed out that “a successful cyber intrusion within the bioeconomy may yield a result that causes physical harm, something generally associated with biosafety and biosecurity but not cybersecurity.” Any interference with the bioeconomy has potential to harm, and while Walsh and Streilein [43] focused on illicit interference, this extends to unintentional interference as well. It is the ability for any cyberbiosecurity system to cause physical harm, intentional or otherwise, that safe AI and safety assurances need fortifying.

Water and food supply systems are a prime example of a cyberbiosecurity systems where safety is a priority. Quality and supply from the system impact everyone in a service region, and both are affected by natural anomalies (algal blooms, weather, draughts, and floods) or cyber-attacks. Water supply systems require constant monitoring and threat mitigating to ensure safety of the water quality and supply [23, 63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79]. On the other hand, food supply relies less on technological innovations, whereas water systems have standardized the use of SCADA systems [48], food supply and agriculture have seen a more limited and hesitant adoption of technology, especially for small-scale farmers [9]. A more standardized approach to tech adoption helps by “securely sharing and interpreting data across sectors and identifying cyberbiosecurity risks,” ultimately improving food supply chains by designing “agricultural and food systems to better meet consumers’ need and protection of life science data” [80]. Data privacy is also a concern any time personal health information may be involved with genomic databases with the potential for cyber-attacks on lab automation [81, 82].

We found in the literature that water and wastewater sectors vary greatly in size, complexity, organization, security protocols, available resources, and even in imposed regulations [47, 48]. While the end goal of each water system is to supply clean water on demand, the approach each system takes is unique and requires different considerations, including adopting security measures specific to their organization [48]. This means that each system needs to take unique considerations to ensure to the quality of the water and consistency of the supply, posing a challenge to the field as a whole because standardized approaches to safety cannot be developed or relied on for all situations.

The bioeconomy, too, consists of large and complex systems that intertwine and connect, and it “harbors unique features that have to be more critically assessed for their potential to unintentionally cause harm to human health or environment” [22]. Water systems supply water to farms that impact agricultural production which in turn impacts food supplies to retails (grocery stores), prices, and the ag-economy. Any hiccup along the way can have unforeseen consequences. The complexity, however, makes it difficult for any one person, or even organization, to understand what consequences their actions have. This means that changes for the sake of mitigating external threats could lead to unintended consequences [39]. Cyberbiosecurity cannot focus solely on cybersecurity and attack detection or, as mentioned in the previous section, on monitoring natural phenomena as interference. We need to implement assurances to guarantee the safety of a system (e.g., quality of water or food for human consumption) at all times.

AI and other emerging technologies’ reliance on data provides both benefit and potential harm. The concern of unintentional errors can arise in the data used for Safe AI. Caswell et al. [83] pointed out the potential issues of errors in biological databases, but the concern is applicable to any data-driven analysis in cyberbiosecurity. While referring to synthetic biology, Li et al. [84] emphasized that unintentional risks can lead to food scarcity despite the efforts of biosafety and biosecurity to provide more. Similar concerns for unintended consequences of dealing with biological data have been expressed in [84, 85]. As these technologies are implemented more into cyberbiosecurity systems (such as precision agriculture) more emphasis needs to be placed on quality assurance of the data and safety assurances for the final product.

5.3 Explainable AI

In the introduction section we defined explainable AI as AI that can “explain, or be interpreted, to understand why it came to a decision or how the algorithm works.” Here, we expand this to include cyberbiosecurity systems in general because that is the environment the AI system operates in, the AI’s behavior is dependent on the larger system, and the end user needs to understand both in order to operate the system correctly. Even if a cyberbiosecurity system does not incorporate AI, human understanding is crucial to its operation. Therefore, we expand the definition of explainability to include “the process of making complex systems human intelligible.”

The literature surveyed often mentions the lack of training, understanding, and even awareness of cyberbiosecurity and cybersecurity risks as a vulnerability. This means a lack of knowledge and human understanding of threats, how to recognize them, and what to do about them is one of the biggest hurdles for the cyberbiosecurity field to overcome. Accordingly, a framework for making these complex systems understandable in order to avoid and mitigate risks is recommended. However, even in the biotechnology and cybersecurity realms “cyberbiosecurity is not well-known or understood” [86] and there is “a failure to recognize vulnerabilities” [40]. This lack of awareness is detrimental because cyberbiosecurity relies on understanding the vulnerabilities, threats, and risks to mitigate impacts [22, 26]. Even with the conventional cybersecurity approach, a “good cybersecurity plan is understanding the threat and establishing cybersecurity governance protocols” [47]. The mentioned approaches are not fully implemented or are done so inadequately resulting in “the failure of individuals to identify and address cybersecurity vulnerabilities” in cyberbiosecurity systems [40].

Part of this lack of awareness is from lack of education or training available in cyberbiosecurity [87]. Drape et al. [29] surveyed researchers from the agricultural sector attending a cyberbiosecurity workshop and found that no participants had cybersecurity training or resources, and attendees were uncertain about obtaining training or implementing solutions. Despite the research going into cyberbiosecurity vulnerabilities, there is no “one size fits all” solution, the difference in educational resources for agricultural security varies from county to county in the USA [29]. It is no stretch of the imagination to see that disparities exist country to country for agriculture, water supply, and food supply chains. These sectors are critical everywhere around the world, but the resources for cyberbiosecurity are not equally distributed, so a solution needs to be general and easy to implement and maintain. Authors in Duncan et al. [88], by focusing on the US food supply chain, stated that “this gap in education and training increases risks to the domestic [U.S.] food supply chain and the ultimate mission of securing the U.S. and global food supply.”

Lack of understanding is a significant risk for any cyberbiosecurity system, but especially for small farms where available knowledge and resources are less than large infrastructure organizations (e.g., utility companies, and industrial farms). More needs to be done to explain cyberbiosecurity as a concept and raise awareness of the vulnerabilities it creates. Richardson et al. [2] point out that as agricultural becomes more reliant cyber-enabled systems the security of these systems is “unclear from a cyberbiosecurity perspective.” This is at the same time that technology is increasingly incorporated into water supply and food supply systems, creating similar vulnerabilities [9, 34, 43, 48, 89,90,91]. Although, Reed and Dunaway [40] were optimistic that technology would bring solutions without any vulnerabilities.

As the size of an organization increases (e.g., industrial farms, utility water supplies, and the bioeconomy) so does complexity and difficulty in understanding how the system operates. Lack of understanding of minute details and interconnectedness are a vulnerability, as even changes to mitigate external threats can lead to unintended consequences [39]. Imagine updating security software and a bug prevents water tanks in a system form relaying fill levels to the central control. More effort needs to be placed on understanding how the system actually operates and how best to explain that operation to the people it matters most.

This approach needs to be done on a case by case basis, as the variability in each individual systems differs. Germano [47] and Clark et al. [48] both point out that differences among organizations and utilities in the water and wastewater sectors include size (employee count and water processed), management, available resources, regulatory oversight, and even security protocols. These differences make a unified approach to cyberbiosecurity in the water sector unfeasible, as each organization or utility needs to build their own approach to match their unique operation and threats. The water distribution system for a large city is going to vary in size, available resources, and security measures from that of a small rural county. This disparity exists in the other sectors of the cyberbiosecurity as well, no two farms, food supply chains, or any other large-scale infrastructure are going to be the same as the issues each one deals with greatly varies. Understanding the needs and shortcomings of each system is critical for cyberbiosecurity.

Awareness of threats and how cyberbiosecurity systems operates is a form of threat mitigation, and several papers make the case for simply making people aware of the risks [26, 44, 45, 47, 92, 93]. Even something as simple as “understanding the threat and establishing cybersecurity governance protocols” is all it can take to protect these systems [47]. That said, understanding these complex systems is no trivial tasks. Both cyberbiosecurity and AI can benefit from the explainability assurance to make them human intelligible. Explainable AI systems are easier to understand how they operate and therefore understand what might negatively impact the system cyberbiosecurity systems, on the other hand, could be explained via machine learning techniques like clustering or even learning a Directed Acyclic Graph (DAG) of the data like Lin et al. [94] did for the SWaT dataset.

The next step for building understanding of cyberbiosecurity systems is through education and training. Richardson et al. [87] call for a standardization of the training process, in the same manner as biosafety and cybersecurity, through credentialing. They also called for integrating training into existing programs or relying on existing programs, as did [29], while others merely made a call for increasing education and awareness [95]. Another theme that emerged in the literature was a need for training across sectors in the water and agricultural industries, so employers training employees [45, 47], cross-sector training [80, 96, 97], government or university curated resources and training, both formal and informal [48, 88, 97], and even war-gaming [98].

5.4 Secure AI

Undoubtedly, one of the most important factors in ensuring the security of water distribution systems is to detect anomalies that may occur in these systems or malicious attacks that may come from adversaries. Water treatment and distribution systems have been increasingly targeted by cyber-physical attacks in recent years [99]. This is partially due to the expansion of the Internet of Things (IoT) and proliferation of AI increasing the digitization of the decision-making processes and creating an adversarial attack opportunity following recent development in the machine learning field, which led to black-box adversarial methods that work well even with limited information [100].

The Kemuri Water Company (KWC) [101] attack in 2016 is a very important example of the risk these national infrastructures are under. The attack has resulted in more than 2.5 million records stolen, but more importantly, the attackers were able to change control data to manipulate the water supplied to the area. The attacks were halted before any public health damage occurred, nonetheless, it showed how vulnerable these infrastructures are and how important it is to ensure their safety.

Another recent, important incident was the Florida Water Supply hack in 2021 [21]. In this malicious attack, the hacker was able to gain remote access to the PLC (Programmable Logic Controller) unit that controls the sodium hydroxide level (also known as lye) of the water supplied to more than 15,000 residents in Tampa, Florida. The hacker was able to increase the amount of sodium hydroxide content of the water by 110 fold. Fortunately, the attack was mitigated before the poisonous levels of chemical diffused into the distribution network.

Both of these incidents show how important it is to detect any anomaly or malicious attacks early to mitigate, or hopefully prevent, any damage. Taormina et al. [35] investigated the vulnerabilities of these critical infrastructures in-depth in their research.

Pasqualetti et al. [102] investigated the detection and identification of CPS attacks from two different perspectives in their 2013 paper. They categorized the monitoring limitations from “graph-theoretic” and “system-theoretic” while proposing a mathematical framework for the problem’s solution. The framework they proposed considers the CPS as a linear time-invariant descriptor system. They then defined a comprehensive set of assumptions and equation systems to measure and detect the corrupted signals in the system. They have also made a theoretical quantification of the limitations of both monitoring approaches to determine undetectable and unidentifiable attacks boundaries. Their paper is also one of the earliest attempts to formally describe the attack detection against CPSs and in this sense, its importance in the field is substantial.

Machine learning is a powerful and important tool for ensuring cyber-physical security. It is not surprising to see deep learning, more specifically Long-Short Term Memory Recurrent Neural Networks (LSTM-RNN), as efficient solutions to a problem with a time-dependent and high sequential relations such as attack detection [103]. Goh et al. [104] used the SWaT dataset [105] as a small-scale representation of a water treatment plant to detect anomalies and identify the sensors affected by this anomaly. They proposed to use the Cumulative Sum (CUSUM) method to mitigate the effects of an extremely unbalanced distribution of positive and negative classes (millions of negative samples to only thousands of positive samples with a sequential dataset). The SWaT dataset is a comprehensive and very important dataset for cyber-physical security research and the contributions of the authors and supporting organizations to the field should not be left unacknowledged.

Inoue et al. [106] applied another deep learning approach in their 2017 paper. The authors used a Deep Neural Network (DNN) to evaluate the Support Vector Machine (SVM) method’s performance for anomaly detection problems. The paper also made a side-by-side comparison of the two models while discussing their advantages and disadvantages. Unlike Goh et al. [104], the authors did not address the data imbalance in the paper. The researchers used the SWaT dataset and the simulation to test the models.

BATADAL is a planning and management competition for Water Infrastructures and it takes place as part of the Water Distribution Systems Analysis Symposium. This competition presents an imaginary C-Town as a water distribution network dataset to detect the real-life size and real-time, simulated data from this town (SCADA) [107]. The paper includes seven well-performing solutions to the problem on this dataset from the competitors. Others (Aghashahi et al. [108]) used a two-stage approach to solve the anomaly detection problem. In the first stage they make a feature extraction, and in the second stage they use a supervised classification method, Random Forests, to detect attack instances.

Brentan et al. [109] proposed a statistical approach to the problem. They used the sectioned nature of the problem environment and trained Recurrent Neural Networks (RNNs) to learn each district’s normal behaviors and then calculated the deviation from these expected normals to measure the anomaly levels on the system.

Chandy et al. [110] used a similar two-staged approach to Aghashahi et al. [108]. Chandy et al. [110], however, first make a detection of the anomaly and then confirm or reject this detection is with a second model, a Convolutional Neural Network (CNN) Auto-Encoder, by calculating reconstruction probabilities.

Giacomoni et al. [111] proposed another two-stage approach. In the first stage, the authors created a set of rules and calculated the integrity of the rules for each instance. In the second stage, they analyzed the dataset to calculate certain thresholds of normalcy. They also proposed using Principal Component Analysis (PCA) and convex optimization routine to perform this analysis [112].

Abokifa et al. [113] proposed a three-stage model and they classified different types of attacks on each stage of the process. In the first stage, the authors used statistical methods to detect local outlier events. In the second stage, they introduced a neural network to the process to detect operational outliers. In the third stage, they focused on the global scope to detect events that might affect more than one aspect of the system with PCA.

Pasha et al. [114] introduced another three-stage method for anomaly detection. The first stage checked the consistency of the underlying rules of the water distribution system. The second stage checked each component for behavioral patterns to see if the system is following the normal patterns it is supposed to do. If any anomaly is detected in the first two stages, the third stage confirms the detections by comparing the estimations of the system made by the method.

Housh and Ohar [25] used EPANET to create a simulation of a water distribution system’s behavior to calculate the difference between the SCADA and the expected values from the simulation to detect and locate anomalies in the systems. Housh and Ohar [115] also used a similar approach to detect contamination attacks against water distribution systems with successful results.

Taormina et al. [107] have comparatively investigated all these proposed approaches, and many more, are discussed along with the advantages and disadvantages of the models. Even though the methods are very diverse, one common factor should not be unnoticed: each of the major competitors followed a direction of first discovering underlying behavioral principles of the system in some manner and then proposed ways to measure the diversion from these principles in anomalous scenarios.

The BATADAL competition provides immense contributions to the cyber-physical security field by providing a great dataset to the researchers as well as creating a valuable comparative environment for all the approaches to provide assurances methods for cyber-physical security [107], an approach (competitions) that proved successful in other areas of AI. Kravchik and Shabtai [116] investigated the attack detection problem from an ICS perspective in their 2018 paper. They used the SWaT dataset to train CNN and Long-Short Term Memory (LSTM) models to compare their effectiveness to detect anomalies. The experimental results showed that 1D CNNs can outperform RNN and LSTMs in more complex multivariate tasks.

Umer et al. [117] investigated attack detection from a distributed system. In their work, they separated the endeavor into two categories: “design-centric” and “data-centric,” while proposing a model for each category. The research used the SWaT dataset [105] as a small-scale representation of a water treatment plant. The methods they proposed utilize Association Rule Mining (ARM). They also compare the advantages and disadvantages of the two approaches proposed in the paper.

Junejo and Goh [118] proposed a behavior-based machine learning approach for the detection and classification of cyber-physical attacks. Their approach promised a low false-positive rate, which some of the other approaches discussed earlier suffer from, and still provided high recall and precision. They used the SWaT dataset to evaluate the effectiveness of nine different algorithms from supervised machine learning literature ranging from Bayesian networks, naive Bayes, logistic regression, neural networks, SVM, and more while making comparisons between models for advantages and disadvantages.

Adepu and Mathur [119] proposed a Single-Stage Multi-Point (SSMP) type of attack with a distributed detection method. Even though they focus on single-stage attacks in their paper, the authors noted that they found it more effective to detect this kind of attack using the information from neighboring stages. The researchers used the SCADA dataset to create two invariants: State-Dependent (SD) and State-Agnostic (SA). Later the authors combined both invariants to create a more efficient tool for distributed detection problems.

In another paper, by Adepu and Mathur [120] authors used the SWaT dataset to investigate ways to improve cyber-physical security and attack detection problems by asking the following questions: “What attacker and attack models should be used to understand the behavior of a CPS?”, “How do cyber-attacks impact a specific CPS with respect to the number of actuators affected, state of a CPS when the attack is launched, and duration of the attack?”, and “Given the response of a CPS to one or more cyber-attacks, how does one design attack detection mechanisms using the physical properties of the system?”. While trying to answer these questions with experimental results the authors disclaimed the generalizability of their findings and stated that this research only targets the SWaT testbed.

This disclaimer shows a very important direction that requires more attention in the field, which is the generalization of the proposed methods since almost all of the methods we discussed so far require prior knowledge of the attack samples to be effective in the first place. The need for generalizability of the proposed approach is the utmost importance since solutions cannot wait until the attacks happen on the real systems to collect the necessary data to train the models.

Adepu and Mathur [121] must have seen this problem as well, as they tried to address it in their next work with a case study of their earlier distributed attack detection proposal [119]. Adepu and Mathur replicated real-life scenarios to test their improved attack detection mechanism and shared their findings with the strength and weaknesses of the model with an in-depth discussion [121].

As we pointed out earlier, fast adoption of automation and networking technology does not come without drawbacks. Al-Abassi et al. [122] tried to remark these issues and address the vulnerabilities created by another attack detection method while promising generalizability on the way. The researchers propose a combined model of DNN and Decision Tree with results that outperformed most of the conventional machine learning models including DNN and Random Forest. The authors also addressed the imbalanced class distribution and effective performance of the proposed approach with experimental results.

5.5 Trustworthy AI

AI is used in an increasing number of different systems, for example, autonomous vehicles, search engines, recommendation systems, medical imaging [123], public health [124], and others. It appears well-developed, yet there are still a lot of issues that need to be addressed and discussed, especially when it comes to the question can AI be trusted in “these scenarios that have life-critical consequences?” [125]. The foundation of societies, economies, and sustainable development is based on trust. If there is no trust the whole societal system would not grow or be stable [126, 127], and the same applies to cyberbiosecurity applications. Inderwildi et al. [128] discussed the impact of intelligent CPSs in energy provision and gave policy recommendations to lower potential risks. The same applies to AI systems, the idea of trustworthy AI is to build trust between users, developers, and the system itself [129].

Trust is a concept that is difficult to build, and trust in AI is even harder to address. The “black-box” characteristic is one of the most important reasons of mistrusting AI [130]. It is hard to build trust without knowing why the system makes its decision. We need to be able to explain the results, and this leads to the importance of explainable AI (see Sect. 5.3). Another situation where trust in AI faces scrutiny is ethical decisions, such as the trolley problem. What is the priority that the system should follow? Are there any guidelines to follow? There are so many different questions to address in order to build trust.

In recent years, a significant amount of research on trustworthy AI has been conducted in different academic and industry areas (see Fig. 3). Each study focused on different aspects of trustworthy AI, for example, [131] focused on government guidelines, which advise how to establish a trustworthy AI system through rules and regulations, and other studies focused on the computational aspect of achieving trustworthy AI [132,133,134,135,136,137]. Most of the research agrees that trustworthy AI systems should include a set of properties: reliability, safety, security, privacy, availability, usability and can be extended to the following dimensions: accuracy, robustness, fairness, accountability, transparency, interpretability/explainability, and ethics [56, 125, 126, 129, 131,132,133, 138,139,140,141].

Trust is a complicated concept that combines numerous factors, and different researchers from various backgrounds would also see trustworthy AI from a diverging perspectives. Liu et al. [132] defined trustworthy AI from three perspectives: technical, user, and social. The system should focus on accuracy, robustness, and explainability from a technical perspective; while it should focus on availability, usability, safety, privacy, and autonomy from the user’s perspective. Whereas from the social perspective, there should be a guideline or regulation regarding legality, ethics, fairness, accountability, and environmental-friendliness. To have more clear guidelines for accomplishing trustworthy AI, the EU established the High-Level Expert Group (HLEG) to provide ethical guidelines, not just principles to follow but also concrete operational steps that allow an AI developer to examine when building and deploying an AI system [131]. Zicari et al. [58] proposed a state-of-the-art process to evaluate the trustworthy AI based on applied ethics called “Z-Inspection,” which is also first process in practice that HLEG defined to evaluate the trustworthiness of AI. Z-Inspection consists of three processes: set-up, access, and resolve, and each phase breaks down into different aspects to examine whether the AI systems are trustworthy.

Toreini et al. [133] pointed out that there are various AI policy frameworks to follow from different nations and organizations, and categorize those objectives into eight qualities: privacy, accountability, safety & security, transparency & explainability, fairness & nondiscrimination, human control of technology, professional responsibility and promotion of human values. They further mapped these eight qualities with four principles, including fairness, explainability, auditability, and safety. The authors separate two main technologies of trustworthiness: Data-Centric Trustworthiness and Model-Centric Trustworthiness.

Liu et al. [132] stated “Trustworthy AI are programs and systems built to solve problems like a human, which bring benefits and convenience to people with no threat or risk of harm.” They focused on six dimensions in achieving trustworthy AI including safety & robustness, nondiscrimination & fairness, explainability, privacy, accountability & auditability, and environmental well-being. Instead of focusing on policy framework or guidelines, they worked on specific computational solutions for each dimension for realizing trustworthy AI.

Li et al. [138] mentioned AI practitioners, including researchers and developers, should focus on pursuing system performance as the main goal, whereas this is not sufficient to reflect the trustworthiness of an AI system. Therefore, they proposed a methodology that takes the entire lifecycle of AI systems into consideration, from data management to model development, deployment, and all the way to monitoring and governance. For the future research direction, while adopting this systematic approach, there are side-effects due to increased learning time and slowed development by using this new approach.

We mentioned that the trustworthiness of AI is essential when it comes to AI systems related to life-critical consequences. There were incidences where critical CPSs came under attack [142] and affected the overall trust in CPSs. For example, an attack happened on a water treatment plant in Florida in 2021 and the level of sodium hydroxide in the water supply was increased over 100 times higher than usual [143]. There were also numerous cyber-attacks on Israel’s water system in 2020 [144]. That exposes how vulnerable those CPSs are and the importance of the security of those systems [145,146,147,148,149,150,151,152,153,154,155]. There has been no lack of related research done in the area of anomaly detection in water system or its security challenges using machine learning methods [33, 107, 116, 156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190], statistical methods [191,192,193,194,195,196,197,198], or other tangential methods [106, 199,200,201,202,203,204,205,206,207,208,209,210,211,212,213].

Wang et al. [214] applied probabilistic model learning to probabilistically validate a real-world CPS. MR and Mathur [215] proposed “AICrit” to effectively detect anomalies in real-time with low false alarms. Another factor contributing to the complication of evaluating trustworthiness is that most of the research or review that discusses how to achieve trustworthy AI focuses more on social science topics, such as ethics and policy [59, 139]. Most of the frameworks or guidelines they proposed, however, do focus on the human factor. Uslu et al. [216] proposed a decision-making framework to manage Food-Energy-Water (FEW) resources. While developing the optimal solutions under different scenarios, they included humans in the framework to make the solutions more trustworthy. They introduced two new metrics, trust sensitivity and trust pressure, in the framework and used a game-theoretical tool to explore the relationship between trust sensitivity and the distance of community-desirable solutions.

6 Discussion

6.1 Attack Detection Models for Water Systems

Cyberbiosecurity attack/anomaly detection research in the literature mainly focused on three datasets SWaT, WADI, and BATADAL which have been introduced in Sect. 4. These three datasets have become field leading benchmarks. As a part of the survey, we have created tables for each dataset. In order to make a fair comparison, we have used the most commonly reported statistical metrics to rank models proposed by researchers for attack/anomaly detection problem. For SWaT (Table 1) and WADI (Table 2) datasets it was F-Score (also known as F-measure, more specifically F₁ score) and for BATADAL (Table 3) we have used S score defined by Aghashahi et al. [108] and listed S_TTD (Time Taken for Detection) as well. For each dataset, state of the art over the years has been marked with bold fonts on Tables 1, 2, 3.

Table 1 SWaT F1-Scores^a

Table 2 WADI F1-Scores^a

Table 3 BATADAL S Scores^a

Throughout the years efficiency of the neural network based models have drastically increased over numerous problems and attack/detection is one of them as well. Looking at highest ranked models on the SWaT F1-Scores Table 1, it can be seen that deep learning had a huge impact on the problem and following the success of Inoue et al. [106] with One-class SVM, in last 4 years breakthroughs were achieved using Deep Learning models Kravchik and Shabtai [116], Li et al. [219] and Ayas and Ayas [63]. This dominance can further be verified with the successful state of the art models developed by Goh et al. [104] and Xu et al. [218], once again using DNN models.

When it comes to the BATADAL dataset the picture slightly changes. Neural Network based models are still very effective on solving attack detection problem with BATADAL as well but they are not as dominant as they are with the other two datasets. Various types of approaches to the problem from many researchers provide a great understanding of the chaotic nature of data-driven problems on large physical systems. Dynamical essence of these systems requires researchers to approach the problem from many angles to ensure the models they would create to be trustworthy and secure. Some of the most successful researches to achieve these feats were, Abokifa et al. [113], Housh and Ohar [25] and Brentan et al. [230] as the state of the art holders.

6.2 Assessing the Cyberbiosecurity Literature

In this section, we discuss cyberbiosecurity further because it is a new discipline and there are different takes on exactly what it is. Unfortunately, most of the literature writes about cyberbiosecurity in a manner similar to cybersecurity for biological applications [8, 39, 81, 84, 90, 92, 95, 235,236,237,238,239].

This is not a fault, the focus of cyberbiosecurity is biology or related applications; however, most of the literature does not adequately define what sets cyberbiosecurity apart from IT or Computer Science in the life sciences. Gillum et al. [97] expressed a similar concern with the issues in the term “biosecurity,” established fourteen years prior to their work. Multiple papers in the literature call for action or collaboration—“We call for analyses and publications to fully scope cyberbiosecurity and identify a comprehensive strategy to establish the discipline’s goals and objectives” [2] and others, as called out by Drape et al. [29] and seen in Murch and DiEuliis [26]. This call from Richardson et al. [2] makes it seem like the field is still in the early planning stages, but this is not entirely true as there are papers that focus on concrete examples, lie case studies, surveys, and even one where the authors initiated an attack on a synthetic DNA supply chain that went undetected [29, 80, 86, 93, 97, 238].

Cyberbiosecurity systems are rooted in the physical sciences, but they can include pure information systems like databases for pathogens, genomics data, and land use data [4, 44, 83, 235]. We focus, however, on the physical supply chains and infrastructure, specifically water and food supply systems. Here, cyberbiosecurity secures supply through “the design of digital strategies, business models, technologies, standards and regulations” [240]. This does not exclude systems that rely on data, as even food systems depend on sharing and gathering insights from data. For example, in Duncan et al. [80] the authors discuss the need for sharing and protecting data to “design promising agricultural and food systems to better meet consumers’ need.” Data is just as much a part of physical systems.

Water systems are open to both natural anomalies and intentional attacks, something highlighted by Schmale III et al. [23], in their paper on a water supply system that is subject to harmful algal blooms, remote monitoring and control are incorporated to help ensure the water stays safe for drinking. However, this opens the system up to cyber-attacks, so cyberbiosecurity measures need to be taken to monitor and mitigate both sources of issues to ensure the safety of the water.

These systems are complex and multifaceted, which makes protections harder to implement and formalize, and this sentiment is highlighted in Duncan et al. [9] where the authors state current protections are not enough and “do not broadly exist across the food and agricultural system,” and the “conversation on cyber security on the U.S. food and agricultural system (cyberbiosecurity) is incomplete and disjointed.” There is a critical need to better incorporate cyberbiosecurity into the water and food supply chain infrastructures. Something easier said than done as these systems have multiple layers of weaknesses at the software level, the interface of cyber and physical, and the biological level. A sentiment that was expressed in Farbiash and Puzis [238] for the synthetic DNA supply chain, as those authors demonstrated an attack can bypass cybersecurity and biosecurity screenings to generate an attack based on gene editing in the synthetic data. In Bernal et al. [28], the work presented used bacteria in a DDoS style attack to demonstrate the unique risks to cyberbiosecurity that traditional cybersecurity measures cannot accommodate. These papers highlight the fact that there are biological exploits available to cyberbiosecurity systems an attacker can use without ever having physical access to a system. The multifaceted supply chains allow for multifaceted attacks that can slip through the cracks of traditional cybersecurity and biosecurity efforts.

6.3 Adoption of AI Assurance for Cyberbiosecurity

The goal of AI assurance is to mitigate any potential drawbacks or failures of AI in high-stakes applications. Assurance is a way of validating AI operates in a human-centered manner, and likewise the goals of cyberbiosecurity are to protect people from biological threats in many forms, they just happen to focus on cyber-systems and CPSs specifically. Despite this alignment of goals, we see little direct connections between cyberbiosecurity and AI in the surveyed papers (see the separation of cyberbiosecurity from the other papers in Fig. 4). There are, however, a handful of cyberbiosecurity papers we found that do overlap in topic with AI assurance, even if there is no connection via citations. Most of these papers deal with trustworthiness and safety [8, 28, 84, 241], and in fact these are also the most common assurances in the literature (see Fig. 3). Two of these papers also focus on fairness [84, 241], a little more surprising because fair AI was the least common assurance we found (again, see Fig. 3). There is one paper that focuses on explainability, specifically data and model transparency, in cyberbiosecurity [44], and how explainability ties more to security. The last paper focuses solely on trustworthiness in cyberbiosecurity [242].

Safety is a key AI assurance pillar (see Sect. 5.2), followed closely by trustworthiness (Sect. 5.5), that applies to cyberbiosecurity. The efforts of all the others are done in order to ensure the safety of the system or in the trust that the system operates in a safe manner. Ethical and fair AI (Sect. 5.1) ensures the AI system makes decisions that are correct and benefit everyone impacted equally, letting users trust that the AI makes safe decisions. Explainability (Sect. 5.3) gives us understanding of how the system operates and why it makes the decisions it does, letting users trust that the AI operates as it should to ensure the safety of those impacted. Secure AI (Sect. 5.4) ensures that if problems arise (anomalies or attacks) that the AI can handle them, either by correcting or mitigating negative effects, letting users trust that the AI system negates or limits possible harm to those impacted. Everything is done so we can trust the safety of the system.

Safety in cyberbiosecurity is mostly concerned with biosafety, or the protection from biological threats. We believe there should be more focus in the literature on food and water safety from a cyberbiosecurity perspective, especially as more technology is adopted in the water and agriculture sectors. However, there are some existing safety measures that can be adopted, like the Hazard Analysis and Critical Control Points (HACCP) for food safety and management which could be used as a starting point for safety assurances [9, 88].

Policy and regulations need to be part of the cyberbiosecurity solution, in part for the need of creating standard practices and metrics across the whole bioeconomy, and in part because cyberbiosecurity threats pose national and international security risks [243]. Cyberbiosecurity should be part of the national strategy for cybersecurity, part of the “Defend Forward” idealogy of national security [244]. This approach, however, requires the need for understanding the cyberbiosecurity field to create regulation and policy for federal agencies, something which is still lacking as “cyberbiosecurity roles, practices and metrics have not been defined and federal agencies appear uncertain regarding how to proceed” [93, 245].

The current state of the cyberbiosecurity literature focuses more on creating systems of awareness or best practices for mitigating security or safety threats, and there is little direct discussion on using explainable AI for cyberbiosecurity. Explainable AI lacks discourse in the cyberbiosecurity literature but is discussed frequently in the medical AI domain, where the goal is to create trust in AI in order to facilitate adoption by medical practitioners and to create transparency and traceability in the decisions made by the AI [246]. Explainable AI also allows for the combination of an interpretable, knowledge-based approach with that of an efficient neural based approach [247]. This means explainable AI is a way of augmenting human understanding of a problem when it uses models designed for human comprehension.

The augmentation of human intelligent via explainable AI feels like a particularly fitting application of AI for cyberbiosecurity. There is still more challenges to be addressed in the domain of explainable AI to show applicability in real-world deployments [246]; however, it does offer a lot of promise in applications where decisions are high-stakes, such as critical infrastructure including agricultural, food, and water supply chains. Richardson et al. [2] called for the implementation of “frameworks to facilitate responsible application of AI techniques to biology” and explainable AI is one way to do so.

This is particularly important to cyberbiosecurity and parts of the bioeconomy, where the sheer size and complexity of systems creates the potential for unintentional harm when trying to mitigate threats [22, 39]. Training and education of these systems (AI or otherwise) become a form of ensuring the continued safe operation of these complex systems. Training and education are also a form of creating awareness of threat mitigation to help ensure security. This is a common theme in the cyberbiosecurity literature [26, 29, 44, 45, 47, 80, 87, 88, 92, 95, 97].

All the pillars eventually boil down to ensuring trust that AI and cyberbiosecurity systems operate as intended. Section 5.5 discussed the connection of AI assurance to trustworthy AI. Society and the bioeconomy, in general, are built on trust, and if we do not trust them we will not use or participate in their activities. The same goes for AI in cyberbiosecurity, trust needs to be built so operators and all parties involved use them.

Developments in AI for cybersecurity and cyber-physical security could protect water, food, or other supply chains from intentional interference, while developments in AI for anomaly detection could protect the supply from natural phenomena [23, 25, 94, 102, 104, 106, 114, 119, 121, 225, 248,249,250,251,252,253,254]. Despite a clear alignment of incentives, there is not much direct overlap between these approaches in the cyberbiosecurity literature (see the separation of between cyberbiosecurity and attack/anomaly detection in Fig. 4). We conclude that although more of the cyberbiosecurity papers clearly make a call for action [2, 26, 29, 255], there is at best merely a brief attempt over existing solutions like the National Institute of Standards and Technology (NIST) cybersecurity framework [43, 47, 95, 256]. The safety and continuing function of any and all systems in the bioeconomy are important but “currently protections are minimal and do not broadly exist across the food and agricultural system” [9].

6.4 Merging the Water Security and Cyberbiosecurity Fields

Similar to AI assurance, there is not a large direct link in the literature between cyberbiosecurity and water systems. There is one series of links from cyberbiosecurity to water systems via Mueller [22], Schmale III et al. [23], Moyer et al. [24], and Housh and Ohar [25]. When we broadened our definition of cyberbiosecurity a little more from the literature we see a broader connection of papers that link the topic with water supply systems [6, 9, 23, 47, 48, 257]. What is also interesting to note is that none of these papers uses the open-source datasets we discussed in Sect. 4, instead these papers focus on broad topics of water within the food and agriculture sector [6, 9, 257] or the security of water sources [23, 47, 48]. Most of the water supply-related papers deal with security and attack/anomaly detection, aligning them more with AI assurance, but we feel they apply just as much to cyberbiosecurity as well.

There is not much existing cyber or cyber-physical security knowledge within the cyberbiosecurity field [2, 8, 29, 45, 86,87,88, 97]. This makes the openness of water supply testbeds and AI research critical, as these technologies can be developed and tested open-source in view of researchers focusing on cyberbiosecurity. More emphasis of the cyberbiosecurity research should be placed on using the open-source water testbeds from Sect. 4. This is the only way that water security (as a form of cyberbiosecurity) research can be performed using relevant data, and it also allows for training and hands-on experience, something a large portion of the literature called for [26, 29, 44, 45, 47, 80, 87, 88, 92, 95, 97]. This development of human understanding of cyberbiosecurity and water systems is a form of explainability and it significantly benefits from open-source data on how these systems operate.

6.5 Recommendations and Future Direction

Much of the work regarding AI assurance and cyberbiosecurity occurred in the last few years and developed separately. Figure 4 shows one link connecting cyberbiosecurity to water systems, which is then tied to the large web of anomaly and attack detection papers. Cyberbiosecurity research, however, still has a long way to meet its goal of wider adoption, and while we cannot speak for all possible sources of cross-collaboration, the expansion of cyberbiosecurity into the domains of water supply systems and AI assurance is wide open for future research.

Continuing the thread of expanding the research outside its immediate domain, cyberbiosecurity has a lot to gain from embracing open-source water supply testbeds. For one, the domain of water security is directly applicable to cyberbiosecurity, despite not making up much of the research. The literature mostly focuses on biology applications, but this feels narrow and collaborating with the established field of water security would be a great way to apply all those lessons learned to cyberbiosecurity. Many of the papers in the cyberbiosecurity literature call for more training, education, and hands-on experience. Open-source testbeds are ideal for developing resources for training and education, as well as developing new research into secure AI and other forms of AI assurance.

The goals of assurance are to validate AI aligns with the values of users impacted by an AI system, and likewise the goal of cyberbiosecurity is to protect users and citizens impacted by a biological system. AI has been instrumental in multiple agricultural applications [258,259,260] and offers many solutions to the threats of cyberbiosecurity but also includes several downsides; assurance nonetheless offers a way to apply AI to maximize its benefits while mitigating potential pitfalls. AI assurance should also be broadened to focus on the entirety of the system AI is deployed in, not just the assurance of the AI itself. For example, both applying AI to ensure the safety of drinking water via water quality monitoring and applying evaluation procedures to ensure the AI is operating properly are forms of assurance. In short, the cyberbiosecurity field should adopt AI measures to meet its goals and use AI assurance to validate both the AI employed is working properly and that the larger system the AI is used in is also operating properly.

7 Conclusions

In this survey, we investigated academic papers at the intersection of AI assurance, cyberbiosecurity, water and food supply systems. We assessed the application, both current and potential, of AI assurance to problems in cyberbiosecurity, specifically focusing on water and food supply systems. The survey focused on journal articles, conference proceedings, dissertations, books and book chapters, and industry white papers published from 2000 to April 2022 and at the intersection of two or more of the mentioned sectors.

A survey landscape (Sect. 2) was performed for an overview of the literature, showing most of the papers included were published since 2016, as researchers started applying AI more broadly and investigating AI assurance. Soon after in 2017, the field of cyberbiosecurity had traction and more water supply system papers were published. The increase in water supply papers since 2016 seems in part due to the start of open-source testbeds (SWaT in 2015, WADI in 2016, BATADAL in 2016, Smod in 2017, and DHALSIM in 2020), and because we specifically focused on papers that intersected with AI and cyberbiosecurity fields, both of which have seen sharp increases in the past few years. Although, looking at Fig. 4, we see there is little connection between the literature of cyberbiosecurity with the other sectors. We discussed how the papers covering these topics connected and how AI assurances apply in these fields, followed by our recommendations for future directions.

In the previous sections, we discussed the six pillars of AI assurance [1], the importance of each pillar, and the effects of the papers surveyed on water distribution systems and their applications. Figure 3, however, shows this distribution is not uniform. The pillars of Ethical AI and Fair AI were neglected, while the importance of these aspects kept growing over the last several years. This shows a great gap and opportunity for research in Ethical and Fair AI for agricultural and water systems.

We found less collaboration among the fields of AI assurance, cyberbiosecurity, and water or food supply systems than we initially expected. Figure 4 shows this disjoint well, and the literature for cyberbiosecurity does not directly discuss AI much, let alone AI assurance. The cyberbiosecurity definition should adapt a little more, as it feels too focused on cybersecurity for the life sciences. There is some acknowledgement that the current literature is not broad enough [9], especially when there are biological processes that can be exploited [28, 238].

Further research should emphasize collaboration across sectors and the use of open-source datasets and testbeds. The call for collaboration already exists with the cyberbiosecurity field, and one of our proposed solutions to that is publishing open-source datasets online. These open the field to broader research and hands-on training and experience, both of which have been expressed as needs for the cyberbiosecurity field. There are unique challenges, though these require expertise from biology, CPSs, and other domain specific knowledge for a desired application.

Lastly, we recommend that the cyberbiosecurity field adopts AI and AI assurances practices for better security while maintaining safe and trustworthy operations of these complex biological systems. There has been a lot of prior research applying AI for cybersecurity, and this would be a natural extension to incorporate into cyberbiosecurity. AI also offers more robust monitoring and an ability to make corrective actions, but this is not without issue as AI creates new vulnerabilities or failure modes. AI assurance can help mitigate these and help ensure the proper function of the overall cyberbiosecurity system.