AI, Creativity, and Art

Abstract

The chapter reviews a personal history of research and art practice in which AI systems played a significant part. The key observation was that interacting with such systems provided support for creative thinking. This history is told in relation to a series of meetings held on Heron Island in Australia in which computational creativity was explored. Meetings were held over a 30-year period, the last one in 2019, where an earlier version of this chapter was presented. The chapter reports on those meetings and on the advances that have taken place since 1989.

1 Introduction

The chapter presents a personal history of work on computation and creativity, placing it in a broader context. It reviews a set of contributions to meetings about computation and creativity held on the Great Barrier Reef Island, Heron, starting in 1989 and concluding in 2019, where an earlier version of this chapter was presented. The chapter briefly states the key arguments that I made at the earlier Heron Island meetings and reviews the progress of that thinking, including a discussion of how the ideas developed in relation and the debates held on the island. A central concern has been to take a human-centered perspective on the value of computational and cognitive models. This goes back to a research agenda first announced in a joint chapter at a 1970 Computer Graphics conference. The title was “the creative process where the artist is amplified or superseded by the computer”, where the two alternatives provided defined the end points of the field to be investigated. There has been considerable progress in creative artificial intelligence applications and the cognitive modelling of creativity since 2005 and the chapter looks at these earlier arguments in the context of that progress, re-assessing them in light of our understandings of 2019.

2 The Earlier Heron Arguments

In this section, I briefly restate the key arguments that I made in my contributions to the earlier Heron Island meetings. I will do this chronologically without reflecting on those arguments, which will be done later in this chapter.

My chapter in the first Heron Island meeting (Edmonds 1993) was titled “Knowledge-Based Systems for Creativity”. The key claim was that knowledge-based systems offered new mechanisms for the support of human creativity. The term “knowledge-based systems” refer to AI and at that time was generally used to refer to symbolic AI systems as against the connectionist systems that are so popular today. This argument was supported by two very different case studies. One was from my own computer-based art practice, and the other was of a speech scientist investigating speech recognition. The first example was concerned with making time-based generative artworks. That is, artworks that change over time as defined by an internal set of rules specified by the artist. I will return to this domain slightly later in this section. Gerhard Fischer and Robert Coyne, with Eswaran Subaran, also advocated the study and development of creativity support systems in the 1989 meeting (Fischer 1993; Coyne and Subrahmanian 1993).

In the second meeting I presented a paper jointly written with Basel Soufi (Edmonds and Soufi 1992) and inspired by Bill Mitchell’s contribution to the first conference (Mitchell 1993). Mitchell had pointed out that the recognition of emergent shapes in drawings was often a key factor in the creative discovery of what might be. The paper investigated the computational implications for support systems and showed how they might be implemented. An important conclusion from this investigation was that models of creativity need to be open systems, not closed ones that are essentially Turing machines.

At the 1995 Heron conference I presented another paper by Basel Soufi and myself in which we went into more detail on the cognitive issues of emergence and their implications for our computational models (Soufi and Edmonds 1995). It was shown that, in order to support the creative practitioner’s interaction with a support system, different mechanisms need to work in parallel so that emergence can be fully handled.

Kelvin Clibbon and I also produced a paper for the 1995 meeting that looked at another important aspect of creative design. The paper was titled “Strategic Knowledge in Computational Models of Creative Design” (Clibbon and Edmonds 1995). The key problem addressed was to deal with the fact that creative design cannot be seen as problem solving within a well-defined problem space. that is normally known as “routine” or “variant design”. We took an approach that used formal logic as the basis of the computational modeling and showed how the use and modification of strategies could be incorporated by extending first order logic to include meta-rules representing strategic knowledge. We argued that an explicit multi-layered representation of design knowledge within the computational support system was important in giving the designer the freedom that they need for creative exploration.

In 1998, I co-wrote a paper with Linda Candy (Edmonds and Candy 1999). We returned to the concerns of my 1989 Heron paper, the influence on human cognition of interacting with a computational system. In this study, we observed an artist working collaboratively with a VR specialist in order to build virtual sculptures. The key observation was that the interaction with the technology had a significant influence on the direction of the artist’s thinking. Following the computer-based work, he went on to make paintings (by hand) that he only conceived of as a result of the computational experience. A new direction for his painting emerged during the interactions.

In 2001, I contributed to a paper presented by Linda Candy, “Model-ling Creative Collaboration: Studies in Digital Art” [an expanded version was later published (Candy and Edmonds 2002)]. In this paper we added to the regular Heron topics by considering collaborative creativity, which we were studying in the context of digital art and had found to be very significant. An initial report was given of findings about the various flavors of collaboration that we observed, together with the identification of success factors for good practice.

It was also in 2001 that I presented a paper, written with John Dixon, that returned to computational support for making generative art (Edmonds and Dixon 2001). As I had argued on Heron Island in 1992, a valid and useful model of the creative process has to be an open system. In my personal exploration of the implications of the computer for art practice, first announce in 1970 (Cornock and Edmonds 1970) and discussed more fully below, I was very interested in computer-based artworks that were themselves open systems. In this paper we showed how the logic of a closed generative artwork could be extended by adding inter-relationships between the rules and external events. In the examples discussed, those events were movements by observers detected and analyzed through video capture. By making such artworks, we postulated that “the “significant trigger for creative thought” in the artist, reported in the 1989 Heron Island meeting, might here be provided to everyone”.

It seemed natural to move on to consider the nature of the experiences that the artist, designer and everyone else have when interacting with these various computational creativity models and support systems. So, in the 2005 Heron meeting, I presented a paper, written jointly with Lizzie Muller, that investigated human creative engagement in such contexts (Edmonds and Muller 2005). The motivation was largely to come to a better understanding of the relevant human experiences and engagement in order to better inform the design of these systems, whether they were design support environments, artworks, scientific aids, or anything else. Based on field studies conducted in the Powerhouse Museum, Sydney, we proposed an initial model of creative engagement, and an approach to further study, that began to provide the answers required.

3 Reflections on Personal Heron Contributions

None of the papers that I presented at Heron provided definitive answers or completed studies. That was never appropriate. They each defined a step in the development of the thinking that my colleagues and I were gripped by. Most importantly, the meetings between 1989 and 2005 provided a forum for debate, for the refinement of concepts and for the birth of new ideas. Thus, in each case, there are later papers published elsewhere that elaborate on the ideas, but these Heron papers typically represent my first public testing of them. In this section, I will provide an overview of the trajectory that provides a basis for the next part of this paper, where I look at what has happened since 2005. First, though, it is helpful to step back and describe certain earlier publications that form an important background to this work.

3.1 Context for Heron

At the 1970 computer graphics conference, CG70, held at Brunel University, UK, Stroud Cornock and I presented a paper with the title, “the creative process where the artist is amplified or superseded by the computer” [later published in Leonardo (Cornock and Edmonds 1973)]. As the title implies, a key concern was to consider whether the computer would become the creative practitioner, the artist, or whether the computer would amplify the artist’s capabilities. One prediction that we made was that a major development would be computer-based interactive art and we identified several models of different interactive scenarios that could apply. We also showed an example interactive artwork. It took another decade before we had personal computers and all the opportunities that they brought with them, but the basics were clear enough then. In brief, the conclusion was that the artist, and by implication the designer as well as other creative practitioners, would be amplified. This meant, however, that much research was needed into how to build effective human–computer (or man–machine as they were termed then) systems.

I immediately started working on human–computer interaction and chose design, and architecture in particular, as the domain within which to research. The very early papers, following Cornock and Edmonds (1973), made a number of points that framed the work that I have discussed on Heron. In a 1972 paper with Christine Daniels and Martin Humphrey (Edmonds et al. 1972), we presented a simple demonstration support system for designers that concentrated on helping them define the problem as well as solve it. This was important because, from my experience within art practice and in talking with designers, I saw problem specification as a key part of the creative process and that would have to be supported by any helpful system.

I then worked with the architect John Lee, developing ideas about how computer systems might successfully support creative design. For example, in 1974 we presented a paper at the EUROCOMP online computing systems conference with the title “an appraisal of some problems of achieving fluid man–machine interaction” (Edmonds and Lee 1974). This was a study made in the context of computer aided architectural design. We argued that it was important to match the structure of the computer system to the human processes that it was to support. We observed the consequential need for flexibility, the provision of backtracking the opportunity to change one’s mind. We showed an approach to dealing with this problem and elaborated on the software and human–computer interaction issues the following year (Edmonds and Lee 1975).

When I came to the first Heron Island meeting, therefore, I took it that a significant contribution that computational models of creativity could make was within interactive creativity support systems. I also saw that such interactive systems should handle poorly specified problems—in fact help to formulate the problem—and should handle the process flexibly, allowing the creative user to develop their thinking and understanding as time progressed. With those starting points, I can now summarize the trajectory of my Heron Island contributions from 1989 to 2005.

3.2 The Heron Trajectory

I began by showing how interacting with a knowledge-based system, AI, could be a significant trigger for creative thought. I showed how, given the right computational system and the right user interface, the creative practitioner’s endeavors could be amplified. To achieve this, it was clearly necessary to understand the human side of human–computer interaction as well as the computational one.

Then, considering such interactive systems it was clear that the human ability to discover emergent shapes and properties was critical in creative thinking, so it needed computational support. Doing this requires relatively deep technical decisions to be appropriately taken. As part of this investigation, it also became clear that any adequate model of creativity had to be interactive, an open system.

Later, the need to also model and interact with knowledge about design strategies became clear and so I presented ideas about strategic knowledge, showing how multi-layered logics could be used for this purpose.

Then, we reported on studies that we had been undertaking of collaborative creativity. In practice, it seemed that much design, and art in the digital domain, was conducted in some form of collaboration or other and the characteristics of these collaborations needed to be investigated.

Returning to the technical aspects of building interactive creative systems, I next demonstrated how a closed generative art system, built using logic, could be made open and so interactive. In the example used, image analysis of video captured from the artwork’s environment was used to modify parameters in the generative rules being followed. Thus, I demonstrated formal systems that were used to make interactive generative art. See Fig. 1.

Fig. 1

Interactive video construct at Kettles Yard, Cambridge, 2001

The 2005 meeting changed the name from “Computational Models of Creative Design” to “Computational and Cognitive Models of Creative Design”, so it was particularly appropriate that the concern I always had for the human, cognitive, side of the problems being addressed became the prime focus of the paper presented then. It reported empirical work that was leading toward a cognitive model of creative engagement with interactive, computationally driven, art systems.

Taking all of the above contributions together, we can see a core argument concerning computational and cognitive models of design. I take a very broad view of the scope of design, in relation to this work. In particular, I take it to include art, while recognizing that the focus of design might normally be different to that of design. In the last 100 years, since the founding of the Bauhaus, it has been particularly common to see these two fields as at least firmly overlapping.

The core argument, then, is as follows:

An adequate computational model of creative design needs to be an open system. It is important to consider both the internal characteristics of the computation and the characteristics of what that system interacts with. An interesting application of such models is in creativity support systems, and hence, it makes sense to look at computational and cognitive models of creative design together. Emergence and strategic knowledge both need to be addressed as part of this work, and finally, if their application is to be a practical value, collaborative creativity must also be better understood.

4 What Progress Has Been Made?

4.1 The Computation and Creativity Community

Just after the second Heron Island meeting Linda Candy and I started the Creativity and Cognition conference series. This was inspired by the first Heron meeting and intended to complement it by having a focus on the human dimension of the computational support for creativity, including creative practice. Because of the interest in practice, an art exhibition has been a normal feature. Later in that decade the series was adopted by the ACM Special Interest Group on Computer–Human Interaction, who continue to run it, most recently in Venice this year (C&C 2022).

The Heron meetings would seem to have also inspired other initiatives, more focused on the computational modelling aspect. For example, in 2009, a Dagstuhl Seminar was held on “Computational Creativity: An Interdisciplinary Approach” (Dagstuhl 2009) and the next year a series of conferences on the subject began and a society was formed, the Association for Computational Creativity, which runs regular conferences, most recently in 2021 in Mexico City (ICCC 2021).

There is no doubt that the Heron Island innovation has led to at least two ongoing and overlapping community initiatives.

4.2 Developments of My Heron Argument

It is encouraging to see that, since 1989 (indeed since 2005) there has been a notable growth in the literature about the Heron Island topics. A full survey is beyond the scope of this chapter, so I will just refer to a few notable books and the latest conferences in the two series mentioned above.

As a background point, it is important to note the undeniable resurgence of artificial intelligence in recent years, both in terms of public interest and success in applications. It is also important to notice that this has been largely driven by progress in connectionist AI, rather than in the symbolic AI that was more commonly discussed at the earlier Heron meetings. What does this mean? Does it imply the need for a re-think? Richard Coyne, Sidney Newton and Fay Sudweeks actually discussed these questions at the first Heron Island conference (Coyne et al. 1993). Their conclusion was that objections to connectionist modeling of creativity did not stand up and that it had significant potential. I will comment on this issue before a more specific look at what has happened in relation to the points in my argument. The issues, then, are:

• Connectionist models

• Interaction

• Creativity support

• Emergence

• Strategic knowledge

• Collaborative creativity

• Experience.

4.2.1 Connectionist Models of Creativity

Of the objections to connectionist models that Coyne et al. covered, the one that is now most widely discussed and seen as problematic is the obscurity of such models: our inability to question them as to why they have generated a particular result. This is often seen as particularly concerning when we suspect an inappropriate bias in the data set that the system has been exposed to (the training set as it is often termed). Examples of serious problems are frequently reported, for example in facial recognition applications (USA Today 2015; Wired 2019).

Whichever approach one takes to our subject, building autonomous models of creativity or building creativity support tools, and the inability to question the model is a problem. If the model is autonomous and connectionist, it may be able to generate a creative outcome, but what else can it tell us? It will not be a very valuable model, in the sense of a theory about that creative act, because we cannot inspect the decision-making that led to it. If it is a connectionist support tool, then if it is just throwing up proposals for the user to evaluate, it may well be helpful. If it is being used for pattern recognition, then it will be as useful as the training set is unbiased. However, if the user needs to interact with the knowledge in the system, as for example was done in my 1989 paper (Edmonds 1993), then it is as useless today as it was then. I would argue that the recent great advances in connectionist AI do not offer a revolution or even a notable opportunity in our area of the computational and cognitive modeling of creative design.

4.2.2 Interactive Systems

The discussion about creative models and open systems has continued. The most significant contributions have probably been philosophical. In her book “Creativity and Art” (Boden 2010), Margaret Boden considered autonomy, integrity, and authenticity in relation to computer art as well as posing the question “is metabolism necessary”?. In each of these cases she implicitly contributed to our open/closed system debate. Her arguments are too long and complex to summarize here, but I will cover the important context in which she makes them. Boden lists three different ways in which we can be surprised. They are, briefly:

• 1. Combinational: the generation of unfamiliar combinations of familiar ideas.

• 2. Exploratory: finding new ideas in a known conceptual space.

• 3. Transformational: changing or extending a known conceptual space.

It is the third category that we would normally consider creative, rather than just innovative, although this is a matter of definition rather than fact so not everybody will agree. It is also the case that Boden is not taking much interest in surprises that have no relation to anything that we know already, so she dismisses “…the undisciplined outpourings of a schizophrenic’s “word-salad”—which, despite being unpredictable and occasionally suggestive… is not in itself an exercise in creative thinking”.

By definition, if a computer program is a closed system, it can only reach states within the conceptual space defined by that system. Only by interacting with some other system(s) can the space be changed. Hence, from this perspective, transformational creativity can only be exhibited by an open system. The question of whether a computer program, rather than a human being, could be said to be creative is something else that Boden discusses, but this is beyond my scope. In the Heron context we would at most argue that such a program modeled human creativity.

4.2.3 Creativity Support Systems

Just before the 2005 Heron meeting, in June of that year, a workshop, sponsored by the U.S. National Science Foundation, met in Washington to discuss creativity support tools. They generated a white paper that set out a research agenda as well as various carefully debated propositions about how such support systems should be developed, evaluated and deployed (Shneiderman et al. 2006). Partly as a result of that initiative there has been a widespread growth of work in the area, much too large to survey here. Suffice to say that we now have an active research community working on computational systems that support human creativity. Much of that work addresses the support of “everyday” creativity rather than the work of expert artists, designers, scientists, etc., but a significant proportion is addressing the kind of problems that we debated on Heron. For a recent review, see for example Frich et al. (2019).

4.2.4 Emergence

Emergence was discussed in a range of Heron papers and is often considered in creativity research. One notable example is the book on Emergence in Interactive Art by Seevinck (2017). Although the examples described in this book are artworks, the principles investigated are more general. The research can be seen in the context of creativity support where the interest is in “…facilitating emergence in creative practice as well as characterizing it in people’s interaction …”.

Seevinck’s results lead her to list certain mechanisms as particularly important in the building of this kind of support system:

• 1. Structural transparency: giving the user access to the system’s underlying mechanisms

• 2. Priming: exposing the user to examples prior to engaging them in the interactions that might lead to emergence

• 3. Combined directly responsive and influencing interactions: combining action-response with (possibly delayed) interaction over long periods (Edmonds 2007a).

She also points out that, at least in her work, it was important to make empirical studies of the interactions so as to get a handle on the thinking going on in the heads of her users.

4.2.5 Strategic Knowledge

I have worked on strategic knowledge myself, making significant use of the concepts in a range of interactive artworks. The theoretical foundation was described in a paper already mentioned (Edmonds 2007a). In that paper a systems view of interaction is taken and contrasted with an action/response model. A refined view of such interactions is proposed in which artwork and audience are said to influence one another. All interaction involves an exchange but need not necessarily leads to a significant change in behavior: at least not at that time.

In the case of my application of the idea to art works, I created the Shaping Form series of interactive pieces (Edmonds 2007b, 2017). In the Shaping Form works, images are generated using rules that determine the colors, the patterns and the timing. Figure 2 shows a moment in the interaction with an installation version, Shaping Space.

Fig. 2

Shaping Space at the Site Gallery, Sheffield, 2012

The strategic knowledge employed in these works is codified in meta-rules that can modify the basic rules that run the system. Interaction aside, a rule-based computational system produces a never-ending stream of graphics, with an equally changing timing pattern. However, these are interactive generative works that evolve by the influence of the environment around them, as detected by image analysis of data from a Webcam. Movement in front of each work is detected and leads to continual changes to the rules being used. The data produced from the image analysis are used by meta-rules to modify the generative system as well as to cause an occasional direct response. People can readily detect these immediate responses to movement but the changes over time are only apparent when there is more prolonged, although not necessarily continuous, contact. A first viewing followed by one several months later will reveal noticeable developments in the colors, the timings, and the patterns. The Shaping Form works use meta logic to implement, or one might say model, the systems-based “influence” form of interaction mentioned above, contracting with an “action-response” one as is commonly used, for example, in computer games.

4.2.6 Collaborative Creativity

The work that Linda Candy and I introduced in 2001 (Candy and Edmonds 2002) has been expanded and reported in various ways, see for example Mamykina et al. (2002) and, in particular, in the book “Explorations in Art and Technology”, now in its second edition (Candy et al. 2018). In that book, as well as reporting on the empirical research on collaboration in creative computational systems, a range of practitioners report on their experiences of making computer-based art systems in collaborative teams. The results of the studies reported in the first edition (and briefly announced on Heron) identified various success factors:

• Shared language: the need to evolve shared terminology

• Common understanding: of intentions and visions

• Open discussion: facilitating free “what-if” conversations

• Establish relationships: giving time for the team to develop, for example feeling free to make mistakes and recover.

These were found to largely stand but the successful work of described by collaborative teams, such as Christa Sommerer and Laurent Minonneau (Candy et al. 2018: 363–370) Andrew Johnston and Andrew Bluff with Stalker (Candy et al. 2018: 341–352), Anthony Rowe and his Squidsoup team (Candy et al. 2018: 333–340), etc., gave a much stronger argument for the importance of collaboration in much creative work in the area.

4.2.7 Experience

The study of creative experience has demanded the development of appropriate research methods. Most notably, these have to be methods that can be applied in the field, in real rather than laboratory situations. Our interest is not in what experiences people have in artificial situations, we need to know what happens in real contexts. One approach to this problem was to conduct investigations in a dedicated space within a museum. The space was known as Beta_Space and was part of the permanent exhibition area of the Power House Museum, Sydney. Much of the work has been described in detail in a book (Candy and Edmonds 2011). As a result of conducting many studies of creative engagement with interactive art systems in Beta_Space, Zafer Bilda developed a model of engagement that showed, in particular, how the nature of that engagement changed over time (Candy and Edmonds 2011:163–181) See Fig. 3. The important lesson is to see that experience changes the form of engagement and, consequently, the design criteria that work for instant engagement will be different to those that work for long term, even life long, engagement.

Fig. 3

Bilda’s model of creative engagement

4.3 Where Are We Now?

Looking at the most recent relevant conference series mentioned above (C&C 2022; Heron 2019; ICCC 2021) we see that the issues covered in my above “Heron argument” are still live in these communities. Collaboration is quite a significant topic, often in terms of “co-creation” and the nature of sharing. Interaction (using an open system) is everywhere with explicit studies of computer support systems and a nice clear observation, drawing upon Lubart (2005), that what might at times be seen as a “failed AI system” is, in fact, a successful human–computer interactive system. Having the human in the loop is not such a bad thing, a point that goes back to my first contribution on Heron (Edmonds 1993).

As to my own current position, apart from the points made above, I have argued strongly that rather than always starting with the science and observing the creative arts—and design—there is much to be gained by starting with the increasingly impressive volume of research within those arts to see what science can learn. I have argued this in a recent book, by taking a selection of research projects in interactive art and showing how they offer human–computer interaction a great deal (Edmonds 2018). The lesson here is equally valid if other creative arts are considered in light of what can be learnt by other scientific disciplines, such as computational creativity.

In a more extensive study, I have worked with Margaret Boden on a book that covers the history, philosophy and practice of generative art: art which focuses on the writing of code as a key medium (Boden and Edmonds 2019). Following on from the point made in the previous paragraph, a non-trivial contribution of the book is a set of interviews with artists who use code, artists for whom computational models in effect are central to their work. It is impossible to summarize the book in this chapter, but I will mention two points that seem to me to be significant in relation to our research into computation and creativity or, as I prefer to think about it, computation for creativity.

Margaret Boden and I spend some time placing computational art in its broader context, both philosophically and historically. We show how the computer, software and computational modelling are being used to support and enable creativity, in many forms, in ways that have a strong relationship with what was done before the computer existed. Of course, computation offers something new and specially but understanding just what that is can only be done by seeing it in its true perspective. The fact, for example, that very many computer-based artists only show their work in computer art contexts, rather than in mainstream exhibitions, and in so doing risk marginalizing their work. More significantly, the risk is that the evaluation criteria and the critical apparatus that we use in considering art and the art making process might be by-passed in relation to the computational arts.

The second point from the book is the change in attitude and perception observable in different age groups. The interviewees range from pioneers who started using computers in the 1960s to artists who regularly programmed computers well before they discovered art. This second category is “digital natives”, and it was clear that they see code and computation quite differently to their elder colleagues. For example, Alex McLean, a live coding artist, said “code is the most suitable way of thinking about music”. For many current creative practitioners, code and computational models are an integral part of their work and something that they can hardly imagine not using. We need to understand the creative thinking, the cognitive processes, of digital natives and that means making new studies, not based on the work of the established pioneers. The best route for pushing this research forward is, I suggest, to interrogate that thinking from the inside. The best approach that we have available for this has to be based on the creative reflective practice (Candy 2019), where the creative practitioner themselves articulates the processes of their thinking.

5 Conclusion

The Heron Island meeting, from 1989 to 2005, provided an exceptional forum for proposing, debating, and innovating ideas about computational and cognitive modeling in creative design, including the arts. In itself, the conception of the series was creative and, as I have briefly indicated, it has led to many new paths of research and application. The developments in technology and in research since 2005 have only made that earlier work more relevant. In my discussion above, as well as reviewing my own “Heron” thinking, I have pointed to some examples of where we have moved on and where we are going. Looking overall at this work, it seems that the human element, the cognitive, and the computational/cognitive interactions have proven most interesting and most promising. A start has been made, but as I see it, the list of interesting questions has only grown.