Dynamic Concept Spaces in Computational Creativity for Music

Abstract

I argue for a formal specification as a working understanding of ‘computational creativity’. Geraint A. Wiggins proposed a formalised framework for ‘computational creativity’, based on Margaret Boden’s view of ‘creativity’ defined as searches in concept spaces. I argue that the epistemological basis for delineated ‘concept spaces’ is problematic: instead of Wiggins’s bounded types or sets, such theoretical spaces can represent traces of creative output. To address this problem, I propose a revised specification which includes dynamic concept spaces, along with formalisations of memory and motivations, which allow iteration in a time-based framework that can be aligned with learning models (e.g., John Dewey’s experiential model). This supports the view of computational creativity as product of a learning process. My critical revision of the framework, applied to the case of computer systems that improvise music, achieves a more detailed specification and better understanding of potentials in computational creativity.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

So far, there is no known definitive description of what computational creativity might be; to improve that end I argue for a formal specification as a working understanding of computational creativity for music. My working understanding supports an analytical view of machines that improvise co-creatively with humans, and the specification can also serve as a generative tool for development of new improvising systems (as in (Mogensen 2017b)).

A computational creativity is not necessarily in the same category as human creativity and comparing these two ‘creativities’ may well, in logic, be a category mistake. Kinds of what we call creativity may have in common what Wittgenstein called ‘family resemblances’, and so I take the creativity concept family as a term covering possible ‘creativities’ that exhibit both similarities and differences. The vaguely defined ‘human creativity’ serves heuristically as prototype for the creativity concept family only to the extent that I use terms derived from ideas about human creativity to name and to guide the conceptualisations of my proposed components in the specification for computational creativity, no identity between human creativity and computational creativity is implied.

I take as given that anything that a current digital computer (or a Universal Turing Machine¹) can do, can be represented in a formal specification. Therefore, if a computer can in some way be programmed to perform creatively, in other words produce a kind of ‘creativity’ and become a member of the creativity concept family, then such a creativity must be definable as a formal specification of ‘computational creativity’. Developing a more detailed formal specification for computational creativity is an essential step towards understanding the potentials of such technology; and such specification can additionally serve as a guide for developing more capable implementations that can interact constructively with human priorities.

Creativity is often referred to as consisting of some creative process, whereas I argue for understanding creativity as determined by product achieved by a learning process, so that creativity itself is not a process but instead is a product (echoing Glickman (1976)). In support of this view of creativity I argue that the formal specification allows alignment with learning models (e.g., John Dewey’s experiential model (Dewey 1938), (Kolb 2015)).

I base my formal specification on my reworking, in effect replacement, of Wiggins’s (2006a) formal framework, which in turn was based on Boden’s (2004) conception of ‘creativity’ as searches in concept spaces. In order to allow the alignment of the specification with the experiential learning model as mentioned, I argue that the epistemological delineation of ‘concept spaces’, in the Wiggins/Boden framework, is problematic: instead of bounded types or sets (that imply a rather static character), such theoretical spaces should more properly represent traces of creative output.² These emergent traces are much better represented by dynamic concept spaces. I examine my revised specification in the context of computers that co-creatively improvise music together with human performers.³

2 A Working Specification for Computational Creativity

My working specification for computational creativity, in Z-style notation,⁴ views creativity as searches in conceptual spaces. In my initial adaptation of Wiggins’s framework I summarise Wiggins’s Axioms in Fig. 1 and his approach to determining ‘creative output’ in Fig. 2 (from (Mogensen 2017b) and (Wiggins 2006a, pp. 451–453)). In Fig. 1 the declarations are interpreted as follows: $\mathscr {C}$ is a concept space of type $\varSigma$ in the universe of possible concepts $\mathscr {U}$. C is a concept type and $c^{1},c^{2}$ are instances of C and $\top$ is the empty concept, all of which may be within a concept space $\mathscr {C}$. In Wiggins’s formalism “creativity” is seen as searches in a conceptual space ($\mathscr {C}$), which is a subset of the universe of possible concepts ($\mathscr {U}$).⁵

Wiggins proposed an approach to evaluating concepts, discovered through the searches, which is summarised in Fig. 2: a Language ($\mathscr {L}$) gives the basis for a Search strategy ($\mathscr {T}$) and Constraints ($\mathscr {R}$) on the conceptual space ($\mathscr {C}$), along with Evaluation criteria ($\mathscr {E}$), that are related to form part of the input to a decision function which consists of an interpreter $\mathscr {\langle \langle . , . , . \rangle \rangle }$ and an evaluator [[ . ]].

Fig. 1.

My schema of Wiggins’s four Axioms.

I have previously (Mogensen 2017a) modified the specification by adding Intrinsic Motivations and Extrinsic Motivations⁶ (see Figs. 3 and 4) based on information theoretic types proposed in Oudeyer and Kaplan’s typology of computational models of motivations, which combines psychological concepts with generalisations of robot implementations (Oudeyer and Kaplan 2007, pp. 4–5). The formalised representations of intrinsic motivation can indicate a combination of motivations that can described as in the schema in Fig. 3.⁷ Four types of motivation components are included: 1. $r_{l} : Attraction \; to \; novelty$; 2. $r_{m} : Information \; gain$; 3. $r_{n} : Pleasure\; of \; surprise$; 4. $r_{o} : Comfort\; of\; the$ familiar. These four components are described as probability-based computations⁸ that operate on an experienced concept ($c_{k}(t)$) in relation to the known part of the concept space at the time ($\mathscr {C}(t)$).

Fig. 2.

My summary of Wiggins’s ‘[e]valuating members of the conceptual space’ with the empty concept as a starting point.

Fig. 3.

My adaptation of some types from the Oudeyer/Kaplan formal intrinsic motivation typology.

Fig. 4.

My adaptation of reward structures from the Oudeyer/Kaplan typology for extrinsic motivation.

I proposed that extrinsic motivations can be formalised in a similar way, although with a focus on external input as shown in Fig. 4. The four motivation components are similar to those of the intrinsic motivations, except that for extrinsic motivations ($\mathscr {M}_{2}$) the probability-based computations operate on an external source of sensory input ($M_{k}(t)$) in relation to the known part of the concept space at the time ($\mathscr {C}(t)$).

My four choices of the formalisations of motivations ($r^{l}, r^{m}, r^{n}, r^{o}$) are only part of the Oudeyer/Kaplan intrinsic motivation typology and it may be useful to explore other types and hence other concepts of motivations in the framework, but I leave this for future research. The four formalised motivation types are based on human psychology and so would seem to contradict my proposal in the Introduction that human and computational creativity are different categories. However, I argue that using theories of human motivation as the basis for computational models does not mean that these are of the same categories, but rather that the computational motivation models reference human motivation in order to guide conceptualisation.

$$\begin{aligned} Memory: \mathscr {W}(t) = \bigcup ^{t-1}_{p=1} \Big ( Q(p) \cdot [[ \mathscr {E} ]] \Big ( \langle \langle \mathscr {R} , \mathscr {T} , \mathscr {E} \rangle \rangle \big ( c(p) \big )\Big ) \Big ). \end{aligned}$$

(1)

This formalisation required a more explicit Memory representation, as discussed in (Mogensen 2017b), which is defined as $\mathscr {W}(t)$ in expression 1 and reappears in Fig. 5 in my version of the framework. $\mathscr {W}(t)$ is a memory of past evaluations at time t: it is the set of past results of Wiggins’s evaluator functions. Each element of the memory (subset of past interpreter function outputs) may be attenuated by some time-dependent effect which I indicate as Q.

Fig. 5.

My revised version of the Creative Output formalisation.

My revised Creative Output formalisation is shown in Fig. 5 (Mogensen 2017a, p. 8), which can be summarised as follows: the interpreter function uses constraints to interpret changes in intrinsic ($\mathscr {M}_{1}$) and extrinsic ($\mathscr {M}_{2}$) motivations as well as the current concept space ($\mathscr {C}(t)$) and accumulated memory ($\mathscr {W}$). This interpretation is processed by the evaluator function to give the Creative Output.⁹

3 Concept Space Morphology

With my specification we can begin to examine the possibility that concept spaces ($\mathscr {C}$) are not the delineated types ($\varSigma$) that seem to be used in the Wiggins/Boden framework; rather, concept spaces are dynamic and can represent emergent qualities of the traces of creative output, and the structure over time of these traces is generated from the experiences of the agents that operate on and within them. In Fig. 6 I have formalised a view of dynamic concept spaces: changes in constraints $\varDelta \mathscr {R}(t)$, search strategy $\varDelta \mathscr {T}(t)$ and value definitions $\varDelta \mathscr {E}(t)$ are functions of memory $\mathscr {W}(t-1)$ and motivations ($\mathscr {M}_{1}(t-1), \mathscr {M}_{2}(t-1)$). The change of concept space at time t ($\varDelta \mathscr {C}(t)$) is, in turn, a function of the changes of constraints $\mathscr {R}(t)$, search strategy $\mathscr {T}(t)$ and value definition $\mathscr {E}(t)$ as well as the latest concept c(t) and the concept space previously perceived $\mathscr {C}(t-1)$.

Fig. 6.

A view of Concept Space morphology.

This morphology of the concept space is examined from the agent perspective, since it is generated from inputs that include memory and motivations. So here the concept space is not an ideal space encompassing all possibilities in a particular domain, rather it is a dynamic space of possibilities as perceived by an agent which may or may not correspond to a particular idealised domain. This distinction is the key to refining this part of the formalism. To define an ideal domain-based concept space would require omniscience, knowledge of the entire universe of possible concepts ($\mathscr {U}$) which is obviously not accessible; instead, we might postulate that a dynamic possibility space ($\mathscr {C}$) may be on a trajectory towards a possible ideal domain ($\varSigma$) in the universe ($\mathscr {U}$), while completion of this trajectory seems unlikely to be a reachable goal.

I propose the dynamic concept space as a generated space, where the space at time t is defined as a function of constraints, search strategy and value definition moderated by memory, as shown in expression 2. This definition is then equal to the last predicate in the specification in Fig. 6.

$$\begin{aligned} \mathscr {C}(t) : f \big ( \mathscr {R}(t), \mathscr {T}(t), \mathscr {E}(t), \mathscr {W}(t-1) \big ) \nonumber \\ = \mathscr {C}(t-1) \cdot \varDelta \mathscr {C}(t). \qquad \qquad \quad \end{aligned}$$

(2)

Wiggins and Boden distinguish between ‘exploratory creativity’ and ‘transformational creativity’. When a concept space is changed by the agent through the action of searching, in other words when there is a morphology of the concept space, then the Boden/Wiggins distinction between transformational and exploratory creativity seems to break down. Instead of being separate categories, exploratory creativity does transform the concept space and transformation of the concept space is the result of exploratory action.

Consequent to the dissolution of the Boden/Wiggins distinction between transformational and exploratory creativity is that the Axioms from Fig. 1 can be simplified and redefined as shown in Fig. 7: we retain $\mathscr {U}$ as the universe of possible concept types C and we want to be able to differentiate individual points $(c_{1}, c_{2})$ in the concept universe. Wiggins’s empty concept $\top$, which represents nothing but which Wiggins used to initiate the search process (see Fig. 2), can be omitted, since we use intrinsic motivation $\mathscr {M}_{1}$ as a driver of Creative Output even if memory $\mathscr {W}$ is empty and regardless of whether there is any extrinsic motivation $\mathscr {M}_{2}$ (see Fig. 5). The declaration of $\mathscr {C}: Concept \; Space$ is no longer axiomatic since we define it in Fig. 6. Also, we no longer need the axiomatic expression that a concept space is a subset of the universe ($\forall \mathscr {C} \vert \mathscr {C} \subseteq \mathscr {U}$) since it is conceivable that a $\mathscr {C}$ could become identical to $\mathscr {U}$, although this is only as a limiting case since it would mean omniscient knowledge of the universe.

Fig. 7.

The simplified set of Axioms for the specification including concept space morphology.

Wiggins required the third proposition in Fig. 1 because ‘for transformational creativity to be meaningful, all conceptual spaces, $\mathscr {C}$, are required to be non-strict subsets of $\mathscr {U}$’ (Wiggins 2006a, p. 452). However, as mentioned above, in this new specification for computational creativity the idea of ‘transformation creativity’ as distinct from ‘exploratory creativity’ is no longer meaningful: instead, with dynamically generated concept spaces, exploratory creativity may be said to be transformational of the concept space as expressed in the morphology of the concept space over time. The resulting axiomatic expression for my specification in Fig. 7 simply expresses that we can differentiate between some different concepts in the universe of possible concepts.

According to Wiggins, Boden views transformational creativity as changes in $\mathscr {R}$, in other words, as changes in the constraints on the concept space. Wiggins proposes a view of a transformational creative system ‘as an exploratory creative system working at the meta-level of representation’ (Wiggins 2006a, p. 455). At this ‘meta-level’ Wiggins uses his valuing function $[[\mathscr {E}]]$ as a method for determining what impact an explored concept c(t) has on the current concept space $\mathscr {C}(t)$. However, using a dynamic, generative concept space, any explored c(t) will change the concept space $\mathscr {C}$ regardless of the results of using it as input to an evaluation function. This seems to be an acceptable feature of the common conception of creativity: any explored possibility becomes part of memory, and so part of the concept space, regardless of whether it is valued at a given time or not. Anecdotally: when teaching music composition and creative use of music technology at Birmingham Conservatoire I often emphasise that any compositional choice that is considered for, but isn’t applied in a particular musical work becomes part of the space of compositional choices available for another composition later on. In other words, the musical ‘object’ produced represents a subset of the dynamic concept space.

Fig. 8.

Overview of the framework.

Figure 8 gives an informal overview of the present version of the framework where Memory — $\mathscr {W}$ — Intrinsic and Extrinsic Motivations — $\mathscr {M}_{1}$ and $\mathscr {M}_{2}$ — and the current Musical ‘object’ — c(t) — are inputs to the Evaluator(interpreter) function: $[[ . ]] \big ( \langle \langle .,.,. \rangle \rangle ( .,.,.,. )\big )$ in Fig. 5. The Evaluator(interpreter) function results in Creative output (Fig. 5), and this in turn becomes the next Musical ‘object’. The output of the Evaluator(interpreter) function modifies the Dynamic concept space $\mathscr {C}$. The Dynamic concept space is the basis for Memory in my version of the framework. The components, aside from the Musical ‘object’, form the Computational Creativity. I expand the framework to include a wider context in another article (Mogensen 2018).

4 Aligning Concept Space Morphology with an Experiential Learning Model

Returning to Fig. 5 and Expression 2 the specification might appear to indicate some circularity in the system: 1. the concept space is dependent on constraints, strategy, value and memory; 2. memory is dependent on application of constrains, strategy, value; 3. constrains, strategy, value are dependent on memory of the concept space. But that is a misinterpretation: given discrete time t the equation should be interpreted as a process of discrete iterations, and so the formalism can be aligned with learning models. As an example I align the specification with John Dewey’s experiential learning model (Dewey 1938).

Dewey’s model of experiential learning, in other words his ‘formation of purposes’ in the case of learning music, can be understood as four steps that are cyclically reiterated: 1. ‘Impulse’ (the desire to play or create); 2. ‘Observation’ (listening to uses of techniques and ideas); 3. ‘Knowledge’ (analytical insights and embodied cognitive practice); 4. ‘Judgement’ (critical evaluation to make choices which will guide the next ‘Impulse’) (Kolb 2015, pp. 33–34) (Kolb 1984, pp. 22–23) (Dewey 1938, p. 69). This iterative process is illustrated in the diagram in Fig. 9, adapted from Kolb’s (2015) interpretation of Dewey.

Fig. 9.

Dewey’s model of experiential learning with iterations leading to ‘Purpose’, where I: Impulse, O: Observation, K: Knowledge, J: Judgement, and t represents time.

I propose to align these four steps with components of the formal model so that I represent Experiential Learning as generative recursion, shown in Fig. 10. In this interpretation, the Kolb/Dewey Impulse is represented by intrinsic motivation $\mathscr {M}_{1}$; Observation is represented by extrinsic motivation $\mathscr {M}_{2}$; Knowledge is the dynamic concept space $\mathscr {C}$; and Judgement is the Creative Output. Dewey’s ‘Purpose’, as a goal of the learning process, may be an artefact output that is considered ‘complete’ in some aesthetic or poietic¹⁰ sense. In the case of improvised music, the ‘purpose’ may be the completion of a performance; the Judgements (or Creative Outputs) of the generative recursion correspond to the playing of the music; the dynamic concept space is the musical performance, which is here represented in a discrete time sequence $[0,..,t, t+1, ..]$.

Fig. 10.

Experiential Learning as generative recursion.

As a consequence of the expression in Fig. 10 the generative recursion of this computational creativity specification can be understood as an experiential learning process. If the Wiggins/Boden’s ‘searching’ in the universe of possible concepts is a learning process then the ‘creativity’ of the system is expressed in the emergent traces that are the Creative Output of this learning process.¹¹ This resonates with the philosophical argument made by Jack Glickman (1976, pp. 130–131) on the concept of creativity in the arts: that speaking of “‘creative process”... is the wrong way to go about characterizing creativity, [instead] one must attend to the artistic product rather than to the process’. So I propose that creativity is not a process itself but is rather an artefact that may emerge from a learning process.¹²

According to Kolb, there is a ‘dialectic... between the impulse that gives ideas their “moving force” and reason that gives desire its direction’ in Dewey’s model (Kolb 2015, p. 40). Applied in the formal model this may translate into a relation between intrinsic motivation $\mathscr {M}_{1}$ and Creative Output, aligned with Impulse and Judgement (expression 3).

$$\begin{aligned} \mathscr {M}_{1} \longleftrightarrow Creative Output \thickapprox Impulse \longleftrightarrow Judgement \end{aligned}$$

(3)

Kolb’s ‘most current statement [of] experiential learning theory is described as a dynamic view of learning based on a learning cycle driven by the resolution of the dual dialectics of action/reflection and experience/abstraction’ (Kolb 2015, pp. 50–51) and his working definition of learning is that experiential ‘[l]earning is the process whereby knowledge is created through the transformation of experience’ (Kolb 2015, p. 49).¹³ Within the formal framework, these two dialectic relations can be understood as shown in expressions 4 and 5. We can say that reflection is evident in the change of concept space ($\varDelta \mathscr {C}(t)$) which is in a dialectic relation with Creative Output. The external input ($M_{k}$), whether cognitive or computational, may be considered as ‘experience’ which is in a dialectic relation with the concept space abstraction ($\mathscr {C}$). Further investigation of these relations is beyond the present scope and are reserved for future work.

$$\begin{aligned} Creative Output \longleftrightarrow \varDelta \mathscr {C}(t) \thickapprox action \longleftrightarrow reflection \end{aligned}$$

(4)

$$\begin{aligned} M_{k} \longleftrightarrow \mathscr {C}(t) \thickapprox experience \longleftrightarrow abstraction \end{aligned}$$

(5)

5 Conclusion

The presented development of the formal specification and understanding of its meaning opens up new possibilities for developing computational creativity. In much current Artificial Intelligence work the goal of a search algorithm is usually to find optimal solutions to search problems. In music, improvised music in particular, a focus on searching for optimal solutions to a ‘problem’ may be a category mistake. In other words the question, whether an optimal music has been achieved seems to be a misleading question. Instead one should ask what has been the value of the aesthetic experience of the music, and also has the learning process, that aligns with the making of the music, been productive of a transformed experience? In a creative system for improvising music there is no imperative to find an ‘optimal’ solution, since the morphology of the search itself can constitute a musical ‘solution’, a trace of a learning process, which counts as a valuable contribution to an aesthetic event. In this specification the generative search in the possibility space is a ‘solution’ to the improvisational performance ‘problem’.