Intersections of Living and Machine Agencies: Possibilities for Creative AI

Abstract

This chapter looks at artists’ experimentations with linkages between intelligent computational systems and non-human living organisms. These unusual hybrid systems showcase models for how we can bridge heterogeneous lifeworlds. In doing so, they evince all kinds of heretofore unimagined possibilities for mutual understanding and influence, which may give us new perspectives on AI-non-human alterities and may serve to question the anthropocentric divisions between humans, human technology and the more than human world. At the same time, this approach may point toward a model of art-making, where encounters between living organisms and intelligent machines can serve not only as vectors of novelty and unexpected variety, but also as a step toward developing a system of ideas focused on showcasing alternative possibilities of human–machine non-human relations.

1 Introduction

In The Three Ecologies, French philosopher Félix Guattari argues for more subjectivity in science (Guattari 2000). In extending the definition of ecology to encompass social relations and human subjectivity as well as environmental concerns, Guattari states that the boundaries between nature and technology need to be collapsed if we are to properly address the ecological crisis. Learning to think “transversally” or across disciplines is a crucial step toward the goal of developing the alternative ontologies, epistemologies and social relations necessary for ecological sustainability. Guattari’s ideas draw significantly from those of cyberneticist Gregory Bateson who argued it is not merely our technologies which are unsustainable but our ways of thinking (Bateson 2000). However, I believe that if we want to properly account for subjectivity in science and explore alternative epistemological models, we would do well to look at the work and ideas of cyberneticist Gordon Pask.

Although a full accounting of Pask’s work is out of the scope if this chapter, what I can briefly discuss here is Pask’s notion of the “participant observer” and how it contrasts with the more traditional “scientific observer” (Pask 1958). Pask pointed out the marked difference between a scientific observer, who minimizes interaction with an observed system and a participant observer who maximizes it. He proposed that when we build and try to understand complex systems, we approach them as a natural historian would: through our interactions with them (Pask 1960). Much of mainstream science considers observer interaction as a source of confounding variables and thus as something to be avoided. For Pask, however, constructing devices that adapt to environmental conditions and interacting with them can add to our knowledge in ways the traditional scientific experimental method cannot. As Andrew Pickering notes in his analysis of Pask’s work, there is an original, inchoate philosophy of science embedded in Pask’s model (Pickering 2010).

What does the is mean for the arts, and specifically art and AI? Many contemporary artists who work with digital and emerging technologies have employed living organisms and explored ecological themes in their work in recent years, each of them exploring unique aspects of non-human life and their relevance to contemporary science, technology and art. Much in the vein of Pask and Guattari, these artists bring these ideas of scientific subjectivity and observer interaction into their work. Likewise, artists have been employing AI and A-life techniques in their work for over 50 years, yielding a rich and diverse set of artworks in that time. There is an inherent strangeness and ambiguity to these technological systems. They behave quite unlike any technological systems we are accustomed to, often exhibiting autonomy, life-like behavior and at least the appearance of intelligence or sentience.

For this chapter, I will discuss the intersection of these two kinds of artistic practice. There has been an increasing artistic experimentation over the last several years with linkages between computational systems (often intelligent ones) and non-human living organisms. Although sometimes falling under the umbrella of biological or ecological art, I want to argue for its distinction as a unique genre, that lies at the intersection of bio-art, eco-art and Creative AI. These works feature encounters and interactions between living organisms and intelligent computational systems, often employing agent-based and/or machine learning methods. Here, the autonomous agents respond to actions and behaviors of the living organisms and produce some sort of output related to their learnings and interpretations, which they often use to respond in a manner that feeds back into the living organism’s environment, influencing its behavior in some way.

Thematically, these works often examine the implications that arise from encounters between human technology and the environment, while simultaneously raising the ethical and ontological status of the organisms they use. They bring into question the rational clarity of the classical ideal of a dualist ontology that separates people and things, throwing into high relief, a recognition that matter and non-human life are not passive and inert but are lively and dynamic, with agency and lifeworlds of their own. In doing so, they showcase the nascent elements of this distinct aesthetic paradigm and its possibilities for Creative AI.

2 Speculative of Models of Shared Non-Human Machine Agency in the Arts

The recognition of sophisticated information processing capacities in prokaryotic cells represents another step away from the anthropocentric view of the universe that dominated pre-scientific thinking. Not only are we no longer at the physical center of the universe; our status as the only sentient beings on the planet is dissolving as we learn more about how smart even the smallest living cells can be. (Shapiro 2007)

The idea of microbial cognition and intelligence has been gaining purchase in the life sciences (Shapiro 2007; Lyon 2015). Many of us have probably heard of the amazing learning and decision-making capabilities of the slime mold Physarum polycephalum (Tero et al. 2010; Tsuda 2009) or the collective social motility and pattern-forming of numerous bacterial species (Ben-Jacob 1997; Ben-Jacob, Cohen, and Levine 2000) (along with their sophisticated communication methods such as quorum sensing). More and more notions of cognition, collective intelligence, communication, and creativity in a number of “primitive species” are being more seriously considered, significantly challenging how these concepts are constructed (for example as requiring brains or neurons). This is a more complex adaptive systems of view of life and intelligence, with more of a focus on adaptive relationships with the environment. There is an emerging ontology here that is based upon a recognition of the agency of non-humans, expectations of complexity and ambiguity, self-organization and co-emergent interplay between all kinds of agents: human, non-human and computational. What I would like to focus on here is on how artists are building artifacts and systems that act out a kind of model of this unknowable and co-emergent world.

So in what ways are artists working with “primitive” living organisms to create experiences that suggest alternative ontological visions for understanding and acting in the more than human world. And in doing so, how are they laying the groundwork for a new kind of art-science? And what are the implications for Creative AI?

Many artworks in this emerging art-science field feature technological interfaces with non-human organisms, often involving some sort of interface with an electronic and/or AI-powered computational system. For example, the art collective Interspecifics have been constructing these types of organic-computational interfaces in various works that feature speculative attempts at bacterial interaction and communication as a source of sonic and visual variety. A recent work, Speculative Communications (2017–18) (Interspecifics 2017), features an AI-powered microscope that observes and learns from cultures of the bacteria Bacillus circulans. The tracking and analysis of the bacteria’s growth and swarming behavior is then used a generative sound score. In micro-rhythms (2016) (Interspecifics 2016), another bacteria-focused work, the group utilized microbial fuel cell (MFC) technology (Logan 2008) along with machine learning to create a generative sound and light composition.

Here, voltages released from anaerobic bacteria in the MFCs are used to trigger lights which are then read by a computer vision and machine learning system that groups the light patterns into different categories or clusters. The appearance and repetition of these clusters are then used as modulation sources for a sound synthesis and spatialization system. The group’s most recent project (currently in development at SETI), titled Codex Virtualis (Interspecifics 2021), proposes the use of deep learning generative models and cellular automata to create speculative hybrid bacterial-AI organisms.

Similarly, Kuai Shen Auson’s 0 h!m1gas (2010–13) (Auson 2012) explores the human-ant relationship. This piece uses computer vision to track the movement of leafcutter ants and piezo sensors to detect vibrations that the ants produce via their stridulation behavior. The collected data is then used to control the movement of a pair of turntables, and the sound of which is amplified. The movement and behaviors of the ant colony thus emerge as a soundscape of scratching effects, an obvious reference to the ants’ stridulating behavior.

In Michael Sedbon’s CMD (2019) (Sedbon 2019) (Fig. 1), bioreactors of photosynthetic bacteria can claim access to light thanks to credits earned for their oxygen production. They can sell it in a market, the rules of which are optimized through a genetic algorithm. The system tests different populations of financial systems on these two sets of cyanobacteria. Andie Gracie’s Autoinducer_Ph-1 (Gracie 2006) features bacterial cultures (Anabaena azollae) that interact with simulated bacteria powered by an AI model. Digitized stimuli produced by the real bacterial cultures are interpreted by the AI, which in turn dictates the supply of air, heat and light provided to the organic cultures. The piece borrows from a traditional Southeast Asian rice cultivation technique. The outcomes of this complex relationship (parasitic vs symbiotic) determine the behaviors of a robotic rice farming system that cares for and manages the rice growing.

Fig. 1

CMD (2019), Michael Sedbon

Whether fanciful and provocative or strange and mysterious, these works provide a new lens to view conceptions of life, the environment and non-human “otherness” overall. While all very different, I believe they share at least three traits in common.

• First (and most obviously), they feature biocybernetics machine-organism interactions.

• Second, they all eschew the “look but don’t touch” approach to environmentalism—trading in traditional conservation approaches (consume less, recycle, etc.) for active engagement and literal contact with living systems and the environment.

• Third, they explore speculative ontological and epistemological models with regard to the non-human world (or at least the “primitive” species such as bacteria and ants). In essence, they adopt a “Paskian” approach, interacting directly with organisms to learn from them and generate knowledge about them. In other words, adding an element of scientific subjectivity.

3 PlantConnect and Biopoiesis

To supplement this theoretical discussion, I will also discuss two projects of my own:

• First Biopoiesis (Castellanos 2018; Castellanos and Barnes 2018), a cybernetic art project that explores the relationships between structure, matter and self-organization. Based upon Pask’s experiments with the construction of electrochemical assemblages that were capable of growing their own sensors (a kind of analog AI), I will discuss the design and construction of the system and explore the relevance of Pask’s electrochemical work to the arts. I also put forth the notion of a “philosophy of open-ended ambiguity” embedded within this work and discuss its resonance with the arts.

• Second, PlantConnect (Castellanos 2020) (Fig. 3), an exploration of human-plant interaction via the human act of breathing, the bioelectrical and photosynthetic activity of plants and computational intelligence to bring the two together. The system measures the photosynthetic and bioelectrical activity from an array of plant microbial fuel cells (P-MFCs) and translates them into light and sound patterns using computer vision and machine learning. Part of larger investigations into alternative models for the creation of shared experiences and understanding with the natural world, the project explores complexity and emergent phenomena by harnessing the material agency of non-human organisms (plants and bacteria in this case) and the capacity of emerging technologies as mediums for information transmission, communication and interconnectedness between the human and non-human. PlantConnect is a collaboration with Bello.

PlantConnect explores human-plant interaction via the human act of breathing; the bioelectrical and photosynthetic activity of plants and computational intelligence to bring the two together. The system measures the photosynthetic and bioelectrical activity from an array of plant microbial fuel cells (P-MFCs) and translates them into light and sound patterns using machine learning. The primary mode of participant interaction with the system is via breath. When a participant blows or whistles into a CO₂ sensor located within the array of plants, it triggers an array of 16 grow lights that are directed at the plants and thus contribute to their photosynthesis. The photosynthesis levels are obtained from small measuring chambers containing CO₂ sensors attached to each plant. In addition to an instantaneous audible response to the decreasing CO₂ levels caused by the increased photosynthesis, these photosynthesis levels are translated into interpolation parameters for the virtual sound instruments and spatialization module of the system. Meanwhile the voltage signals from the P-MFCs are amplified so they can be read by a standard microcontroller. These signals are then analyzed to find the minimum and maximum voltage values, which are used to generate a set of adaptive thresholds that are sent in binary code to the light array. These thresholds determine the on/off patterns of the lights when they are triggered by human breath/CO₂. Using a blob detection algorithm, the system detects the on/off state of the lights in the light array as well as the general shape produced by the lights, relative to the background. This data is then sent to a clustering algorithm (a form of unsupervised machine learning). This algorithm recognizes similarities and differences in the repeating light patterns and classifies them into groups or clusters. Essentially performing rudimentary pattern recognition. This data is then sent to a Max/MSP application via OSC/UDP messages that control a set of virtual instruments and a spatialization module within the Max/MSP environment. In this way, the machine learning algorithm—and by extension the plants—selects instruments and alters their amplitude, duration, frequency and spectral parameters. They also select a spatialization state (Fig. 2).

Fig. 2

PlantConnect (2019)

Fig. 3

A typical setup: an array of 13 electrodes placed in the stannous chloride-ethanol solution

In PlantConnect, bioelectricity, light, sound, CO₂, photosynthesis and computational intelligence form a circuit that enhances informational linkages between human, plant, bacteria and the physical environment, enabling a mode of interaction that is experienced not just as a technologically enabled act of translation but as an embodied flow of information.

Biopoiesis (Fig. 3) is a series of experiments exploring the relationships between structure, matter and self-organization, in what might be described as a computational "primordial soup." This work builds on Gordon Pask’s research into electrochemical control systems that could adapt to certain aspects of their environment (Pask 1960). A collaboration with neuroscientist Steven Barnes, these experiments explore the artistic potential of Paskian-like systems while also examining the interactive and computational possibilities of natural processes to serve as an alternative to the commonplace digital forms of computation, which might help (re)establish a dialog between cybernetics, mainstream science and the arts. The piece entails the construction of several simple computational devices that are all based upon the process of electrochemical deposition: When electrical current is passed through a metallic ion solution (e.g., ferrous sulfate, stannous chloride), metal is deposited on the electrode that is the source of electrons (i.e., the cathode). In our experiments, information (in the form of an electrical current) is fed to a chamber filled with a solution of stannous chloride and ethanol via an array of electrodes (see Fig. 3, below). The resultant electrochemical reaction includes the growth and/or dissolution of metallic dendritic threads in the metallic ion solution, leading to a dynamic pattern of complex electrical and physical growth activity across the entire system. The dendrites are fluid and unstable, bifurcating and dissolving in seemingly unpredictable ways. Thread bifurcation and dissolution, in turn, lead to resistance changes that modify the flow of information (current) through the network. If a subset of electrodes in the electrochemical solution receives input from an environmental sensor (or via some other method), and the electrochemical output can affect that sensor (or otherwise influence the growth of threads), then the network may move toward a dynamic equilibrium with its environment. The dendritic network also carries a decremental memory trace of its previous activities: When the environment changes, the system is perturbed but not immediately reset. Thus, the prior activity and configuration of the system affect how it handles a change in its environment. It can thus learn from its interactions. Furthermore, the system can be trained by providing reinforcement for certain sorts of conductance changes that are produced in response to a particular environmental perturbation.¹

Biopoiesis and Pask’s electrochemical assemblages both serve to redirect our attention to the very material forms of the works and how they add a certain dimension of materiality and sensuous presence that is often lacking in digital and even robotic works. Some of these works display at least a hint of a certain kind of agency that can only come from these non-symbolic (i.e., non-digital), material forms grounded in processes of organic or quasi-organic growth. For many years now, artists have experimented with different mediums, techniques and locations without knowing exactly what the results would be. Thus, to an artist, Pask’s approach might seem familiar and not that different from certain other artistic modes of experimentation. Pickering notes how in Pask work (and that of Stafford Beer, his sometime collaborator) there is a belief in the agency and variability of matter. He notes how rather than marshaling (or dominating) “inert lumps of matter” (as the building of computers and industrial machinery entails), there are attempts to couple this variability to human concerns (Pickering 2010, 236). A Paskian electrochemical system such as Biopoiesis encourages us to view the world as full of co-emergent, co-evolving systems too complex to be fully apprehended or objectively explained. A world that is in a perpetual state of becoming, characterized and brought forth via emergent relations of complexity that adumbrate an experience of the world that we characterize as open-endedly ambiguous. In other words, what we as artists who employ sophisticated technology in our work can learn from this Paskian philosophy is, in a sense what we already know.

4 Toward a Framework for Understanding Living and Machine Agencies in the Arts

These works are but a small sampling of a larger trend in the arts of increasingly hybridized practices and research agendas that necessarily require new ontological models for their analysis, critique and understanding. It is important to recognize that these works are created within a context infused with notions of art as research and art as experiment. This emerging paradigm—which can be said to foreground unpredictable emergence, rather than the rational clarity of a narrow and precise cause and effect—does not fit very well into the paradigm of most modern science, but it does fit quite well within the Paskian “cybernetic method” of maximizing interaction with an observed system to attain knowledge as a participant observer. This stands in stark contrast to the traditional scientific method, where interaction is minimized, as it is a potential source of confounding variables. I believe this model presents possibilities for new zones of negotiation and reciprocity between humans, non-humans and intelligent machines, as well as a reexamination of how art and science can come together. It offers a vision of the world as one that is filled with co-emergent, autonomous agents in reciprocal interplay—a world in a perpetual state of becoming, whose relations are too complex to fully apprehended expect through interactions and complex relations of alterity.

Beyond showcasing a new kind of art-science that embraces unknowable complexity, what more can we tease out of these works? What characteristics do they share that give rise to their performative ontologies? As no coherent analytical framework currently exists for understanding artworks that feature encounters between machine and non-human agencies, I present here a few key points of a provisional analytical model, which posits a concentration across four key areas (presented in pairs):

• Agency, the ability to act in the world and exert influence, is innately bound up with Autonomy, the ability to govern one’s own interactions with the world. Autopoietic theory shows us that it is precisely the operational closure of the organism that gives it its autonomy and ensures the development of its own unique form of structural coupling to its environment (Maturana and Varela 1980). As Varela and Bourgine note, autonomy refers to a system’s ability to assert itself and to “bring forth a world” for itself (Varela and Bourgine 1992). These works give us a view of intelligent machine-biological systems as a performance of agency, where autonomy is seen as arising from situated, contingent and perhaps most importantly (and a bit counterintuitively) collective interactions with the world (environment, other systems, etc.). These pieces emphasize the ontological nature of autonomous systems. Their capacity to simply be, “to assert their existence” and—through their interactions with their environment—“shape a world into significance” (Varela and Bourgine 1992). As such, they are good examples for how to think of autonomy as shaped through engagements with an “other(s)” (be they machines or other organisms).

• Adaptation is an adjustment of internal and external relations, and a change in internal structure to better perform in an environment. This is bound up with Emergence. The latter is notoriously difficult concept to define. Indeed, its ambiguous and subjective nature is part of what gives the concept its appeal. Although there are highly formalized, computationally focused conceptions of emergence from fields such as computer science (Dessalles, Müller, and Phan 2007), in the context of art and Creative AI, neocybernetic models of emergence as observer-dependent and tied to our knowledge of the material/physical system and its internal/external adaptive relations (which is always incomplete) offer a more suitable definition that accounts for subjectivity—while not being overly reliant on rules and formalisms (Cariani 1992).

Analyzing these works through this lens may help us apprehend and leverage this incomplete knowledge by accepting the world as “exceedingly complex” and ultimately unknowable (Pickering 2010, 223). It thus allows us to understand our interactions as experiments or what Pickering calls “dance[s] of agency” (Pickering 2010). Or as Gordon Pask put it: “a participant observer decides upon a move which will modify the assemblage and, in general, will favor his interaction with it” (Pask 1958, 173). This approach of favoring alterity relations may open up window into how Creative AI can be leveraged to explore the “otherness” of other creatures in a way that a detached “view from nowhere” of traditional science (Nagel) may miss (Nagel 1989).

5 Conclusion

Understood within the theoretical context outlined above, and with the examples, we can begin to see that by creating these strange types of techno-ecological systems that can bridge heterogeneous lifeworlds, all kinds of heretofore unimagined possibilities for mutual understanding and influence emerge, which may give us new perspectives on AI-non-human alterities and may serve to question the anthropocentric divisions between humans, human technology and the more than human world, while also pointing toward a model of art-making where encounters between living organisms and intelligent machines can serve not only as vectors of novelty and unexpected variety, but also as a step toward developing a system of ideas focused on showcasing alternative possibilities of human–machine non-human relations.

Their works thematize and harness complexity, symbiosis and technological reciprocity, transformation and renewal between the human and non-human worlds. They disrupt nature/culture boundaries in ways a disciplined scientific method cannot, and in doing so, outline possible futures and alternative methods of relating our technologies to our environment.

By taking a non-anthropocentric approach, these art as experiments highlight the possibilities of creative partnerships between humans, computational systems and living organisms by reframing biological systems as active, intelligent agents capable of perception, cognition and sentience, rather than passive materials or design elements.

Furthermore, by developing intelligent computational environments that increase non-human participation and set-up states of constructive mutual influence between non-human organisms, humans and machines may make us better attuned to wider cultural shifts regarding the importance of non-human organisms and the larger environment, which may in turn lead to research that has important positive effects on our collective view of interspecies relationships. This kind of conceptual expansion is just one example of how perspectives from the experimental digital art field can contribute to Creative AI research by creating novel multispecies experiences that create meaning and encourage critical reflection.

While simultaneously drawing from established scientific tools and methods, artists working at this intersection of machine and biological agency are subverting radically—through the actual products they produce (i.e., the artworks)—the ontological premises of these very fields. The inherent tensions and challenges that artists must confront when working in this area (and art and science more broadly) also simultaneously offer opportunities for disruption and opening up of new perspectives. In so doing, they point to new avenues in Creative AI research and more broadly, the beginning elements of an alternative art-science formation that embraces complexity, ambiguity and unknowability.