Machines That Make Art

Abstract

Robots can make art. Based on simple rules and stigmergy it is possible to produce unique artworks that are at least partially independent from the human that triggers the process. I have coined it a “New kind of Art”.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

For more than a decade I have been working with machines able to create their own art works. Such a statement raises several questions from which the definition of art, as an exclusive human skill, is the most evident. Is it really art what these machines do? Or, as common sense claims, machines can only make something that looks like art because a human builds them, programs them and hits the on button. Hence the art made in such a fashion is still essentially human or, at best, the product of a man/machine symbiotic relation.

If we are less anthropocentric we may however recognize a certain degree of autonomy in creative machines. They can do things that are not programmed and/or result from an internal information gathering device. On the other hand if we accept the existence of such a thing as ‘artificial intelligence’, i.e. the intelligence of machines, why not recognize the possibility of ‘artificial creativity’, i.e. the art of machines?

As an artist I have state that robots can produce a kind of creativity that although triggered by a human and rooted in a symbiotic partnership may along the process generate novelty.

Robots are machines able to interact in the real world with humans, other machines and the environment. Their degree of autonomy varies considerable and can be measure in many ways such as intelligence, behavior, mobility or/and energy sustainability. Robots also diverge in the type of interaction that they can perform. Some depend entirely on some kind of human control, fitness or predetermined behavior, while others are able to evaluate situations on their own and determine possible reactions. The late are the ones I am interested.

I will demonstrate that based on simple rules and emergent behavior robots can create pictorial compositions that are not predetermined.

Intro

Mankind has been intrigued by the possibility of building artificial creatures. For the ancient Greeks this possibility was provided by techné, the procedure that Aristotle conceived to create what nature finds impossible to achieve. Hence, under this view, techné sets itself up between nature and humanity as a creative mediation.

This was the path taken by Norbert Wiener as he opened up the cybernetic perspective, viewed as the unified study of organisms and machines [1]. One line of development linked to this approach gave rise to the familiar humanoid robot, inspired by the von Neumannian self-replicating automata and based on the top-down attitude of the earliest Artificial Intelligence [2]. A much more interesting trend, also stemming from the seminal work of Wiener but intended to ‘take the human factor out of the loop’, emerged in the mid-1940s with William Grey Walter, who proposed turtle-like robots that exhibit complex social behavior. This was the starting point for a new behavior-based robotics, abolishing the need for cognition as mediation between perception and plans for action.

This line of research was pursued in the 1980s by Rodney Brooks [3], who began building six legged insect-like robots at MIT. This new generation of robots was based on Brooks’ ‘Subsumption Architecture’, which describes the agent as composed of functionality distinct control levels under a layered approach. The addition of new layers doesn’t imply changes in the already existing layers. The aforementioned control levels then act in the environment without supervision by a centralized control or action planning centre. Also, no shared representation or any low bandwidth communication system is needed.

The most important concept in Brooks’ reactive robots is ‘situatedness’, which means that the robot’s behavior refers directly to the parameters sensed in the world, rather than using inner representations. Linked to this concept is the ‘embodiment’ feature, which corresponds to the fact that each ‘robot as a physical body and experiences the world directly through the influence of the world in that body’.

The idea of collective robotics appeared in the 1990s from the convergence of the above described Brooks’ architecture with a variety of bio-inspired algorithms, focused on new programming tools for solving distributed problems. These bio-inspired algorithms stemmed from the work of Christopher Langton, who launched a new avenue of research in AI denoted Artificial Life that allows us to break our accidental limitations to carbon-based life to explore non-biological forms of life [4].

The well known collective behavior of ants, bees and other eusocial insects provided the paradigm for the swarm intelligence approach of a Life. This bottom-up course is based on the assumption that systems composed of a group of simple agents can give rise to complex behavior, which depends only on the interaction between those agents and the environment. Such an interaction may occur when the environment itself is the communication medium and some form of decentralized self-organized pattern emerges without being planned by any exterior agency.

Stigmergy

Based on ants and other social insect’s studies [5], I have tried to reproduce artificially a similar emergent behavior in a robot swarm. These insects communicate among themselves through chemical messages, the pheromones, with which they produce certain patterns of collective behavior, like follow a trail, clean up, repair and build nests, defense and attack or territory conquest. Despite pheromone not being the exclusive way of communication among these insects—the touch of antennas in ants or the dance in bees are equally important, pheromone language produces complex cognition via bottom-up procedures. Pheromone expression is dynamic, making use of increments and decrements, positive and negative feedbacks. Messages are amplified when pheromone is reinforced, and lose ‘meaning’ when breeze disperses it. It is also an indirect form of communication, coined stigmergy by Grassé [6], from the Greek stigma/sign and ergon/action. Between the individual who places the message and the one who is stimulated by it, there is no proximity or direct relation (Fig. 1).

Fig. 1

Artsbot robot, 2003

Following these principles and with the aid of an algorithm, coined Ant System, developed by Marco Dorigo in 1992 in his Ph.D. thesis [7], I have replaced pheromone by color in my first ant-robots (2001). The marks left by one robot triggers a pictorial action on other robots. Through this apparent random mechanism abstract paintings are generated, which reveal well defined shapes and patterns. These robots create abstract paintings that seem at first sight just random doodles, but after some observation color clusters and patterns become patent. Through the recognition of the color marks left by a robot, the others react to it reinforcing certain color spots. The process is thus everything but arbitrary. As far as I know, ArtSBot (Art Swarm Robots) [8] was the first art project to use emergent organization for developing robot creativity. Every previous experiment focused exclusively on randomness or sometimes on target strategies leading the machines to fulfill a predetermined program created by the human artist. On the contrary, ArtSBot was meant to put into practice the utmost possible machine autonomy, aimed at producing original paintings. In operational terms, ArtSBot consists of a series of small ‘turtle’ type robots, equipped with felt pens and sensors. With these ‘eyes’ the robots seeks color, determine if it is hot or cold, choose the corresponding pen and strengthen it by a constant or variable trace. To begin the process, when the canvas is still blank, the robots leave here and there a small spot of color driven by chance. Based on these simple rules, unique paintings are produced: from a random background stands out a well defined composition with intense shapes of color. In other words, initial randomness generates ‘order’. The process is emergent and based on the properties of stigmergy (Fig. 2).

Fig. 2

Artsbot painting, 100304, 2004, ink on canvas, 190 x 160 cm

Machine Creativity

The artistic product of these robots is unique. In the same way that somebody who writes a book cannot be considered as a mere instrument of his primary school teacher, robots cannot be seen as simple instruments of the artist that conceived and programmed them. There is an effective incorporation of new and non predetermined information in the process, which cannot be called anything but creativity. It is true that consciousness is lacking to this creativity. But if we look at the history of modern art, it is obvious that, for example, surrealism tried to produce art works exactly in these same terms. The ‘pure psychic automatism’, the quintessential definition of the movement itself, appeared as a spontaneous, non-conscious and without any aesthetic or moral intention technique. In the first Surrealist Manifesto André Breton (1924) defined the concept in this way: ‘Pure psychic automatism by which it is intended to express, either verbally or in writing, the true function of thought. Thought dictated in the absence of all control exerted by reason, and outside all aesthetic or moral preoccupations.’ [9]. In the field of the visual arts, Jackson Pollock was the artist that better fulfills this intention by splashing ink onto the canvas with the purpose of representing nothing but the action itself. This was coined Action Painting, as it is well-known. Perhaps, because of that, the first paintings from my robots are, aesthetically, so similar to the ones of Pollock or André Masson, another important automatism based painter. In his surrealist period, this artist tried frequently to prompt a low conscious state by going hungry, not sleeping or taking drugs, so that he could release himself from any rational control and therefore letting emerge what at the time, in the path of Freud, was called the subconscious. The absence of conscience, external control or pre-determination, allow these painting robots to engender creativity in its pure state, without any representational, aesthetic or moral intention.

Artsbot

Artsbot (Art Swarm Robots), created in 2003, can be described as a set of robots able to interact with which other through the environment (Fig. 3).

Fig. 3

Swarm of Artsbot robots, 2003

The basic architecture of each robot contains three components: the sensors, the controller and the actuators. The sensors receive signals from the environment, which are processed by the microcontroller in order to command the actuators.

The sensors are of two kinds: those that receive the signal from the key environmental variable chosen, which is color, and those that perceive the proximity of obstacles.

Each color sensor is composed by one LED (Light Emitting Diode) for each RGB color plus a fourth LED directed to White. The function of each LED is to measure the intensity of reflected light.

Proximity sensors are in a number of four located in the robot’s front. They consist of an IR emitter/receptor that produces a signal which is proportional to the distance from a white surface. Hence, the bounding barriers of the terrarium where robots evolve must be white. Since solar light may interfere with the sensors, robots should function in an artificial light setting. The range of distances perceived by this type of sensors is 1–15 cm.

The controller is an on-board PIC 16F876 from Microchip, which reads signals from sensors, processes them according to a program and transmits the result to the actuators. The program is uploaded into the robot’s chip, prior to each run, through the serial interface of a PC. This program is developed based on the PC graphic interface, consisting of a flowchart where test blocks for sensors and actuators are combined according to a certain sequence that can obviously be changed whenever wanted. Each test block compares a given variable with a previously defined control parameter and executes an ‘IF…THEN’ rule.

The actuators consist of two servomotors producing movement by differential traction based on velocity control and one servomotor for manipulating the two pens that execute the action of painting. The latter is commanded by a signal analogous to the one sent to traction motors but, in this case, an angular position control is used.

The ‘warm’ colors corresponds to an intensity <128, encompassing yellow, red and green, whilst ‘cold’ covers blue, violet and rose.

The chassis consists of an oval 20 × 15 cm platform, moved by 3 wheels and carrying two pens. Each robot is 12.5 cm tall and weights 750 g. Their life-time endowed with the 8 AA type batteries is 4–5 h.

Prior to launching any collective experiment, the following procedure is done:

• Parameterization of the control program in the graphic interface with the same values, compilation and transmission for each robot.

• Calibration of all sensors of each robot in the programming interface.

• Provision of fresh batteries for each robot.

This procedure guarantees that all robots have the same individual behavior, in order to meet the non-hierarchy requirement. Autonomy and self-organization are other preconditions assured by this procedure. In regard to how stigmergy is achieved in the experiment, it is worth noting that robots interact only via the environment. In fact, they avoid each other through the effect of the proximity sensors and ‘communicate’ only through the trail left in the canvas by a previous passage. Given that this signal is amplified through the positive feedback mechanism and that no ‘fitness’ function is included in the process, the problem arises of how to stop the experiment. If the battery power was infinite, the canvas would be completely full after a certain time. Hence, in the Artsbot project an exterior stopping criterion must be applied. The more ‘natural’ criterion is the familiar attitude of the human painter, when he stands back from the canvas and realizes that the painting ‘works’. The other, less discretionary is when batteries run out of power.

The experiment performed in the same conditions is driven by the following rules (introduced by a trial-and-error parameterization of the programming interface):

• If any of the proximity sensors detect an obstacle nearer than 10 cm, the robot turns to opposite side of that sensor.

If no obstacle is detected:

• If both RGB sensors read a color, then the pen whose color corresponds to the same range as the average intensities is activated and the robot goes ahead.

• If the left RGB sensor reads a color and the right reads white, then the pen whose color corresponds to the same range as the average intensities is activated and the robot turns left.

• If the right RGB sensor reads a color and the left reads white, then the pen whose color corresponds to the same range as the average intensities is activated and the robot turns right.

• If both RGB sensors read white, then the random module is fired and a pen is activated with the probability of 2/256 and the robot, painting or not, goes ahead.

The results of the experiment are prone to pass the Turing Test for intelligent machines. In fact, it is not possible to discriminate the paintings from human hand made art (Fig. 4).

Fig. 4

Artsbot painting, 260204, 2004, ink on canvas, 190 x 160 cm

The case to be made by the proposed approach is that creativity emerges in the set of robots as a consequence of self-organization, driven by their interaction with the environment. Actually, the random walk of each robot is only interrupted by the ‘appeal’ of a certain color spot, trace or patch that was previously left in the canvas by another robot. Given that the robot only ‘sees’ a limited region of the canvas, if no color is detected in that region, it follows its way, putting down a mark of its passage only in the case that its random number generator produces a value that exceeds a given threshold. In statistics language, each one of the outcomes of the experiment is regarded as the realization of a Random Function (RF), i.e., as a Regionalized Variable (RV). The RF is defined as the infinite set of dependent random variables Z(u), one for each location u in a certain area A. In this case, the area A is canvas, and the random variable is discrete, taking only three nominal color values—‘Warm’, ‘Cold’ and ‘White’. The underlying feedback process leads to the spatial dependency of the random variables and explains why clusters are usually formed in most of the RF instances. These instances are the mapping of the RV onto the canvas, depicting its hybrid structural/random constitutive fundamental nature.

The collective behavior of the set of robots evolving in a canvas (the terrarium that limits the space of the experience), is governed by the gradual increase of the deviation-amplifying feedback mechanism, and the progressive decrease of the random action, until the latter is practically completely eliminated. During the process the robots show an evident behavior change as the result of the ‘appeal’ of color, triggering a kind of ‘excitement’ not observed during the initial phase characterized by the random walk.

This is due to the stigmergy interaction between the robots, where one robot in fact reacts to what other robots have done. As referred before according to Grassé [6], stigmergy is the production of certain behaviors in agents as a consequence of the effects produced in the local environment by a previous action of other agents.

Thus, the collective behavior of the robots is based on randomness and stigmergy.

Man/Machine

From the results of this experiment, one can draw the concept of the thing that feels, the thing that plays, and, a fortiori, the thing—the group of robots—that interacts with the environment in an arty way. This line of thought can be derived from the original idea of Asger Jorn [10] that individual creativity cannot be explained purely in terms of psychic phenomena. In his critique of Breton’s surrealism, Jorn made the point that explication is itself a physical act which materializes thought, and so psychic automatism is closely joined to physical automatism. What is overwhelming is that this attitude corresponds to the approach developed by Rodney Brooks [11] in the field of robotics. Conversely, it is worth noting how Brooks’s approach influenced computer-based art in its ‘materialization’ aspect. In fact, the MIT researcher considers that human nature can be seen to possess the essential characteristics of a machine, even though this idea is usually rejected instinctively by our putative uniqueness, stemming from some kind of ‘specialness’.

In spite of its specific character, the proposed art-making mechanism shares obviously some characteristics with a large range of creative activities.

In first place, when the urban science context is called upon, the way robots evolve evokes irresistibly situationists’s dérive [12], a haphazard drift in a city performed since the 1950s by any group of individuals in compliance with their psychogeographic emotional penchants. Indeed, the positive feedback mechanism may be seen as the drive for revisiting certain spots of the city, which were considered particularly appealing in former passages. In addition, both in the dérive and in the robots’ pseudo random walk, there is always place for the surprise that is the core of art (and of the aforementioned collective art form developed by situationists by viewing their strolls as an aesthetic experience). Also, the ‘emogram’, a map of emotive impressions produced by the participants in the dérive, is the analogue, in urban psychogeographic terms, of the final artwork produced by the robots.

Another way of looking at this experiment is inspired by the surrealists’ cadavre exquis. This ‘game’ involved a group of persons that contributed to the eventual collective artwork of which they only knew, until the final outcome, their individual part. When one of the players finishes his contribution, the sheet of paper upon which he had drawn is folded, in order to prevent the next player from seeing the previous composition, except in a small part, which is the starting point for his input to the collective artwork. Similarly, in our experiment, each robot does not have the ‘general picture’; it ‘must’ rely on the clue left by a previous passage of another robot.

The positive feedback, coupled with a hint of randomness, produces novelty by unexpected change in the spatial arrangement of traces in the canvas. Since no predefined plan commands the global behavior of the group of robots, this experiment can be interpreted at the light of Lefebvre’s [13] idea that ‘Topos is prior to logos’.

Aesthetic creation is defined in this context as set of transformative rules that claims for a vital examination of all stages of the aesthetical production/consumption process, instead of overrating the output (as used to come about when art was considered as a ‘matter of taste’).

In the scope of the experiment presented here, it can be stated that if an idea becomes a machine that makes the art [14], then there is no point in imitating Nature, but to perceive the ‘beauty of the idea’. If a self-referential conceptual art that does not care for objects is to be made, then the point is to simulate those artificial features of life (as it could be) that are driven by creativity. And creativity is not the capacity of arranging objects and forms, it is the invention of new laws on that arrangement.

Modern and contemporary art distinctive features are ‘magnificence and unusefulness’ as stressed by Fernando Pessoa referring to his own masterpiece ‘The book of disquiet’, and confirmed by the main artistic tendencies of the 20th century. In the art of our time the conceptual prevails over the formal, the context over the object manufacture and the process over the outcome.

In consequence, if art is to be produced by robots no teleology of any kind may be allowed. Accordingly, all the goal-directed characteristics present in the industrial-military and entertainment domains of robotics must be carefully avoided. Also bio-inspired algorithms that have any flavor of ‘fitness’ in neo-Darwinian terms or any kind of pre-determined aesthetical output must be regarded as of limited and contradictory significance.

Art produced by autonomous robots cannot be seen as a mere tool or device for human pre-determined aesthetical purpose, although it may constitute a singular aesthetical experience. The unmanned characteristic of such a kind of art must be translated in the definitive overcoming of the anthropocentric prejudice that still dominates Western thought. In short, a true robotic art must be the matter of robots themselves.

As opposed to ‘traditional’ artworks, the constructing of the painting by the collective set of robots can be followed step-by-step by the viewer. Hence, successive phases of the art-making process can be differentiated.

Instead of trying to ‘tell a story’ by assigning ‘movement’ or ‘sequence’ to a preset spatial image, the proposed approach shows in real time the picture construction, relating each stage of the process with the conditions under which the set of robots is evolving.

Even though the same parameters are given to the program commanding the behavior of the set of robots, the instances produced are always different from each other, leading to features like novelty and surprise, which are at the core of contemporary art.

From the viewer’s perspective, the main difference from the usual artistic practice is that he/she witnesses the process of making it, following the shift from one chaotic attractor to another. Though finalized paintings are kept as the memory of an exhilarating event, the true aesthetical experience lies in the dynamics of picture construction as shared, distributed and collaborative man/machine creativity. At any given moment, the configuration presented in the canvas fires a certain gestalt in the viewer, in accordance with his/her past experience, background and penchant (a correspondence may be established between the exterior color pattern and its inner image, as interpreted by the viewer’s brain).

The propensity for pattern recognition, embedded in the human perception apparatus, produces in such a dynamic construction a kind of hypnotic effect that drives the viewer to stay focusing on the picture’s progress. A similar kind of effect is observed when one looks at sea waves or fireplaces. However, a moment comes when the viewer feels that the painting is ‘just right’ and stops the process. Such a gesture can be defined as a moment of aesthetical awareness.

Autonomous robots able to produce their own art based on simple rules, randomness and stigmergy represent for the human viewer the opportunity to understand life and aesthetics beyond the anthropocentric paradigm and the mystifying separations it generates.

If robots can make art, humans can envision a global consciousness based on co-operative and distributed creativity, with no distinction between human beings, life forms and machines.

RAP

RAP (Robotic Action Painter), created in 2006 for the Museum of Natural History in New York, is an individual robot artist and not a swarm, but makes use of the same composition methods based on stigmergy and emergence. Additionally it is able to determine, by its own means, the moment in which the painting is finished. Previous versions didn’t have this capacity being conditioned by battery discharge or my will to stop the process. The wrapping up decision is taken based on the information that the robot gathers directly from the painting, what produces a considerable variation of time and form. RAP can decide that the work is complete after a relatively short while (entailing accordingly a low pictorial expression) or can extend the picture construction for a quite long period, making it much more dense and complex. The ‘secret’ of this behavior is in the significant change of the sensors, which passed from two to nine ‘eyes’, allowing now the reading of local patterns, in addition to color spots. RAP is also my first robot to sign its works (Fig. 5).

Fig. 5

RAP robot, 2006

RAP is equipped with a grid of 3 × 3 color detection sensors, eight obstacle avoidance sensors, a compass, a microcontroller and a set of actuators for locomotion and pen manipulation. The microcontroller is an onboard chip, to which the program that contains the basic rules is uploaded through a PC serial interface.

The algorithm that underlies the program uploaded into RAP’s microcontroller induces basically two kinds of behavior: the random behavior that initializes the process by activating a pen, based on a small probability, whenever the color sensors read white; and the positive feedback behavior that reinforces the color detected by the sensors, activating the matching color pen. These two distinct behaviors are described as modes, the Random Mode and the Color Mode. In the random mode RAP searches for color. If a sufficient amount is not found RAP activates here and there, randomly, a pen stroke choosing also randomly the color and the line configuration. The shape, orientation and extent of these initial lines are determined by the robot based on a random seed acquire from its relative position in the space. This is done with the data retrieved by the onboard compass. In this way RAP’s random generator can be described as real random and not pseudorandom.

When a certain amount of color is detected the robot stops the random behavior and changes to color mode. In this phase RAP only reacts to the spots where a certain amount of color is found, reinforcing it with the same tone.

After a while a discrete pattern emerges, where from a general random background a well defined composition can be recognized.

In order to determine when the painting is finished RAP makes use of a grid of 3 × 3 RGB sensors. If a certain (generative) pattern is found the robot ‘considers’ the work to be done, moves to the down right corner and signs (Fig. 6).

Fig. 6

RAP painting, 200906, 2006, ink on canvas, 90 x 120 cm

RAP creates artworks based on its own assessment of the world. At any given moment the robot ‘knows’ its situation and acts accordingly. It scans constantly the canvas for data retrieving. It uses its relative position in the space as a real random generator. It builds gradually a composition based on emergent properties. It decides what to do and when to do it. It finishes the process using its particular ‘sense of rightness’.

Although the human contribution in building the machine and feeding it with some basic rules is still significant, the essential aspects of RAP’s creativity stems from the information that the robot gathers by its own means from the environment. In this sense RAP’s art must be seen as a unique creation independent of the human artist that was at the origin of the process.

A New Kind of Art

My painting robots were created to paint. Not my paintings but their own paintings. The essential of their creations stems from the machine own interpretation of the world and not from its human description. No previous plan, fitness, aesthetic taste or artistic model is induced. These robots are machines dedicated to their art.

Such an endeavour addresses some of the most critical ideas on art, robotics and artificial intelligence. Today we understand intelligence as a basic feedback mechanism. If a system, any system, is able to respond to a certain stimulus in a way that it changes itself or its environment we can say that some sort of intelligence is present. ‘Sheer’ intelligence is therefore something that doesn’t need to refer to any kind of purpose, target or quantification. It may plainly be an interactive mechanism of any kind, with no other objective than to process information and to react in accordance to available output capabilities.

Hence and although my starting point was bioinspiration, in particular modeling social insect’s emergent behavior, the idea was to construct machines able to generate a new kind of art with a minimum of fitness constraints, optimization parameters or real life simulation. It is the simple mechanism of feedback and stigmergy that is at work here.

These artistic robots are singular beings, with a particular form of intelligence and a kind of creativity of their own. They do art as other species build nests, change habitats or create social affiliations. But since we, humans, are for the time being the only pensive observers, the relation between machine art and human aesthetics principles is here the central issue. Many people like the robot paintings, probably because we seem to gladly embrace fractal and chaotic structures. But, more than shapes and colors, what some of us really appreciate in this idea and its associated process, is the fact that it questions some of our most strong cultural convictions as it was supposed art to be an exclusive matter of mankind. In this sense, my robot paintings are a provocative conceptual art that problematizes the boundaries of art as we know it (Fig. 7).

Fig. 7

RAP painting, 230807, 2007, ink on canvas, 130 x 180 cm