Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

4.1 Introduction

This chapter reflects on the ongoing application and development of practice-based research methods in the domain of live music and dance performance. The ‘practice’ here involves the collaborative development of interactive systems which respond to performers’ actions and produce real-time sounds and visuals. These are visible to both audience and performer, and the intention is to facilitate a creative dialogue between performer and system by providing a rich and stimulating environment for improvisation.

Engaging in practice-based research demands careful consideration of the relationship between practice and research. A core question that needs to be considered at the outset is: who is this research for? Ontological and epistemological positions flow from this. In the case of the work described here, the creative practice is focused on the design of interactive systems for use in live performance. The ‘audience’ for this research is artists and designers who are engaged in similar work. Thus any techniques, findings, theories or observations that arise from this work are evaluated in relation to creative practice and the relevance, utility and/or impact they have for those people.

Following from this, a key concern has been to keep the evaluation (a term used in the broadest possible sense) of the interactive systems grounded in creative practice. Just as Glaser and Strauss (1967) sought to develop an approach to theory development in sociology, which was intimately linked with the words and actions of people in the situations under study, we pursue a number of strategies to keep creative practice and associated research closely tied to the concerns of practicing performers. These include:

  • Working with experienced and high-calibre artists.

  • Iterative development in close collaboration with artists.

  • Meaningful examination of the impact of interactive systems on the creative practice and experiences of performers.

  • Engaging performers in reflection on all aspects of the work, usually in interviews.

  • Analysing data gathered during interviews as a final reflective step to generate theory linked to practice.

4.2 Background: Practice Based Research

Since 2004, we have developed a series of interactive performance works which use what could arguably be termed ‘natural user interfaces’ or ‘reality-based interaction’ (Jacob et al. 2008). The term ‘natural’ is potentially controversial (Norman 2010). Here it is used specifically to refer to interfaces based on simulations of physical systems: i.e. interfaces which respond to user gestures in ways which are intuitively understandable because they are based on the laws of Newtonian physics. In practice, this means that performers’ gestures (directly from the body or via a musical instrument) influence what appear to be physical objects projected on a large screen. In response to the performers’ gestures, the objects move and, at times, create sounds in ways that are physically plausible.

There are a number of advantages to using this strategy. The primary one from the point of view of someone designing for expressive musical or physical performance is that it supports the creation of systems which are intuitively understandable and controllable, while simultaneously exhibiting rich, complex and nuanced behaviours. Works which use this basic approach include Partial Reflections and Touching Dialogue (Johnston et al. 2008, 2009; Johnston 2011). More recently, the interactive systems developed for the Stalker Theatre production, Encoded, have refined and extended this technique.

These works were developed as part of a practice-based research project conducted at the Creativity and Cognition Studios at the University of Technology Sydney. The ‘practice’ in this case is the collaborative creation of interactive systems for live performance. The ‘research’ involves reflection on the creative process and careful examination of the experiences of performers with these systems.

This has three main aims:

  • evaluation of the systems in terms of relevant design criteria;

  • examination of performers’ experiences with the systems; and

  • examination of performers’ practice in relation to the interactive systems.

Outcomes of this process can include design criteria based on artistic practice, theories relating characteristics of the interactive systems to performer experience and creative practice, documentation of artists’ creative practice in relation to the new systems, and the works themselves (Edmonds and Candy 2010).

4.2.1 Encoded

Encoded is a large-scale dance work which premiered in November 2012. Encoded explores how notions of digitised space alter our perceptions of physical space. By using a combination of large and small-scale interactive projections onto the performance space and the dancers themselves, Encoded attempts to blur the boundaries between physical space and digital space.

A core concern with this work was how to realise the interaction between performers and the digital elements of the environment. It would certainly be possible to simply consider the physical performance environment and the dancers’ bodies simply as ‘surfaces’ upon which various pre-prepared images and videos could be projected but in some ways this would seem to reinforce the boundaries between the physical and the digital rather than provide an opportunity to explore them.

The approach that was developed is closely related to the Partial Reflections and Touching Dialogue works described above, in that a simulated physical system is used as a mediating layer between the physical gestures of performers and visuals and sounds produced by the computer. However, rather than using a simulation based on solid objects which are linked together, Encoded uses simulated fluid (Fig. 4.1), based on a heavily modified version of the excellent MSAFluid simulation by Mehmet Atken.Footnote 1 Figures 4.1 and 4.2 show the system in action. Video of a recent performance can be seen at: http://vimeo.com/55150853.

Fig. 4.1
figure 1

Moving particles from the fluid simulation are projected upon the performer. The performer’s movements ‘stir’ the fluid that flows over and around their body (Photograph reproduced with kind permission of Matthew Syres)

Full size image
Fig. 4.2
figure 2

Rick Everett and Lee-Anne Litton of Stalker Theatre perform with the Encoded system. The visual appearance and behaviour of the fluid simulation at the core of the system can be changed significantly in real-time (Photograph reproduced with kind permission of Matthew Syres)

Full size image

The intention is that the appearance and behaviour of the software-simulated fluid will be intuitively understandable for both performers and audience, yet complex enough to facilitate conversational interactions.

4.3 Strategies for Practice-Based Research

A number of strategies for keeping research ‘in tune’ with creative practice have evolved during the creation of the works described above. This section presents these strategies and places them in the context of a framework for practice-based research.

4.3.1 Work with Experienced, High-Calibre Artists

The works described here involved close collaboration with performers who are already have a high degree of proficiency in what might be termed ‘non-digital’ disciplines: music (played on acoustic instruments), dance and physical theatre. These are professional musicians, dancers and choreographers who are highly experienced and well regarded in their fields.

There are a number of reasons for wanting to work with artists of this calibre. The primary one is that they are inspiring to work with and able to present the works that we develop in the best possible light. Beyond this though, in general these artists are also articulate about their experiences with new interactive systems and its impact on their practice. This might be surprising, as it might be expected that practising artists would primarily be concerned with performing and not necessarily interested in theorising about the process. However, perhaps because the interactive systems that we work with have such direct and obvious impacts upon the experiences of performers, and because these kinds of systems are still relatively new, the performers appear to have little difficulty talking about their experiences and how the use of the systems impacts upon their creative processes. Another reason for this, perhaps, is that we use the language of performance to talk about their experiences. This is particularly important during more formal evaluations of the systems. Crabtree argues that ethnographic researchers need to have (or at least develop) ‘adequate mastery’ of their domain of study, in order that they, “can recognize as members recognize what is going on in the phenomenal field of practical action under study and how it is getting done” (Crabtree 2003, p. 81, italics in original). This implies that, when evaluating systems designed for music or dance performance, it is important that those conducting the evaluation understand the creative domain and are able to converse with the performers in that language. The focus of the conversations should be on the experiences of the performer and their creative strategies, using the language of performance, and not technical, computer-related terms.

We have found that experienced performers are willing to explore the boundaries of their practice and consequently open to the possibility of using interactive systems in creative ways. Even performers working in what might naively be considered ‘conservative’ environments of symphony or opera orchestras, are often extremely engaged with new music and approaches to performance.

4.3.2 Iterative Development in Close Collaboration with Performers

Working closely with performers to develop a new work is a strategy that helps maintain the connection between the interactive systems that are developed and the creative interests of the artists. Finding an appropriate balance between working with a performer and working for them can be difficult. By this it is not implied that performers seek to dominate the relationship, although this can of course occur. It is perhaps more often the case that software developers want to be dominated! Software development culture and methods have traditionally encouraged a kind of ‘gun for hire’ mentality amongst software developers leading to a, “just tell me what to do and I’ll do it” attitude. In this view, the artist is the ‘customer’ who specifies (in detail, in advance) exactly what the computer system should do, and the software developer is the ‘technician’, solely responsible for solving all the associated technical problems and creating software which does exactly what the customer specified. As has long been acknowledged in the participatory design and agile software development literature, this approach rarely leads to satisfactory outcomes.

By definition, genuine collaboration involves openness, frequent feedback and a willingness to change direction. To some extent this openness to change is in conflict with traditional software development methods, which are predicated on the notion that the design of the software should be fixed in advance and any changes minimised. Agile software development methods have, arguably, largely addressed this issue, insisting on frequent informal communications between developer and users, and on a culture of ‘embracing change’ (Beck 1999).

Our experience suggests that creative collaborations work best when the relationship is one of ‘full partnership’ (Candy and Edmonds 2002). These are situations where the software developer is fully engaged in the creative process, responds to ideas, is willing to compromise when necessary, but is also prepared to argue against compromise if they feel this is warranted.

4.3.3 Examine and Document the Impact of New Interactive Systems on Performers and Their Performances

The relationship between practice and research in practice-based research remains a point of contention. Frayling (1993) has argued for three categories of research in the area of art and design: research into art and design, research through art and design and research for art and design. While these categories are contentious, especially in relation to the status of research for art, they help to situate the research described in this chapter.

In Frayling’s terms, research into art and design involves the examination of aesthetics, history, perception and theoretical perspectives. Research through art and design is concerned with the exploration of new materials, new applications of existing materials and reflection on creative practice. Finally, research for art and design consists of research targeted at the creation of a specific work, or perhaps a series of works. Frayling describes this as research,

where the thinking is, so to speak, embodied in the artefact, where the goal is not primarily communicable knowledge in the sense of verbal communication, but in the sense of visual or iconic or imagistic communication. (Frayling 1993, emphasis in original)

The research described in this chapter draws on all three categories, but the focus of this chapter is primarily on the first two. Because the work involves the creation of artworks there is, of course, a significant amount of research for art and design, involving exploration of themes and the gathering and consideration of materials. The outcomes from these activities are incorporated into the final work, but often in ways which are hard to precisely identify or explain in words. However, this kind of research is critical if the work is to be effective as an artwork, and without it the research into and research through art and design will be significantly compromised.

While research for the artwork is necessary and important, the comparatively recent use of digital technologies in live performance, and their continued rapid development, means there is an opportunity for broader contributions beyond the artwork itself. First, there is of course the design and application of the technologies themselves. Many digital artists develop new tools and/or technologies as part of their creative practice. Where these are novel in themselves, or used in novel ways, this can of course be a contribution to the field. In Frayling’s terms, this is research through art and design.

What is of particular interest here though is the opportunity that these new applications of technology provide to examine creative practice. The use of interactive technologies in dance, for example, has an impact on every aspect of the work, touching on the practice of performers, choreographers and directors as well as lighting, costume and set designers. In a sense, there would be little point in developing the systems in the first place if they did not disrupt – hopefully productively – existing approaches.

This is research into art and design, which here involves careful evaluation of the digital systems which are developed and used in performance, but also examination of creative practice in relation to those systems. All creative artists reflect upon and evaluate their work in order to learn and develop their personal aesthetic and abilities. The aim here though is to develop theories and techniques which are more broadly applicable, or at least of interest to others working in this domain. These theories are essentially ‘middle-range’ theories (Merton 1957), in that they are based on data gathered from interviews and observations, as opposed to more abstract ‘grand theories’ less concerned with empirical evidence.

It has been our experience that while working closely with performers during development leads to the development of effective “theories-in-use” (Schön 1983), more formal studies help make these theories more explicit. The form of the studies can include interviews with the performers who are involved in the work as well as more experimental studies in which performers who are not familiar with the interactive systems experiment with them and are interviewed about their experiences.

The performers involved here have generally had less interest in the formal studies than in the immediately practical concerns of creating works and putting on a show. However, it is often the case that beyond the higher-level theories which emerge from these studies, artists often do receive immediate practical benefits. The process of sitting down for an hour or longer and talking in depth about their creative practice and the interactive systems which have been developed often leads to new insights for the interviewee as well as the interviewer.

A final benefit of more formal studies is that the interviews document the work of the performers concerned and the artistic concerns which drive their work. As video technology becomes increasingly sophisticated and ubiquitous, artists are becoming increasingly adept at documenting their performances and artworks. However, it is less common to document performers’ perspectives on their performances and the motivations behind them. Given that these creative concerns are likely to change over time, there is value in documenting these more ephemeral concepts as well as their physical/technical manifestations.

4.3.4 Analysing Data Gathered During Interviews as a Final Reflective Step to Generate Theory Linked to Practice

Interviews and video data gathered during user studies are a rich source of data, and it can be a challenge to make sense of this. Analysis of this data is a final opportunity to reflect on the interactive systems and their impact (or lack of it) on the performers’ practice.

For the work described here, the grounded theory methods (Glaser and Strauss 1967; Glaser 1978) were used to help us take full account of everything that performers say or do during the studies. These methods are essentially extremely simple, but time-consuming, techniques which enable the minute examination of interviews and facilitate the construction of theories ‘grounded’ in this data.

From a purely pragmatic point of view, the methods also help ensure that the researcher genuinely takes account of what the interviewees are saying and is not blinded by her or his own pre-conceived notions. Having said this, there is of course nothing in grounded theory methods which guarantees objectivity. The methods are ultimately interpretive and do not aim to produce objective, generalisable findings.

4.4 Practice-Based Research and Evaluation

Linda Candy asks in Chap. 3 (“Evaluation and Experience in Art”) (2014) the question: “Why is evaluation in the context of art seen as a problem?” She argues that, often, evaluation is perceived as:

  • taking more time and effort than the results warrant;

  • conflicting with creativity, because it risks reducing creative work to a process of completing checklists; and/or

  • ignoring important qualities of artworks and people’s experience of art because they are difficult to measure.

Our experience has been that while these risks are real, evaluation in creative work can lead to insights and improved understanding of design and creative processes. As Candy points out, artists already embrace evaluation implicitly in as part of every artistic decision that they make. Even artists such as John Cage, who embrace chance operations in order to reduce or remove the influences of their personal preferences in music, choose ways of linking randomness to sonic events. As part of this process they must necessarily consider alternate links, evaluate them in some way, and select one for a particular work.

Given these evaluative processes at the heart of creative practice, it is probable that the biggest barrier to acceptance of broader types of evaluation in the field is the term ‘evaluation’ itself. Some researchers are proposing new ways of thinking about evaluation in the context of systems which have uses that are open to a range of interpretations. Sengers and Gaver (2006), for example, argue that interaction designers are becoming less concerned with designing software which unambiguously conveys and supports a clearly defined ‘purpose’. They propose that HCI needs to support interactions in which users may have multiple interpretations of what a system is for and how it works. ‘Evaluation’ in this context goes beyond identifying whether users’ interpretations of a system’s purpose and behaviour matches the designer’s anticipated interpretation. Rather,

evaluation shifts from determining whether an authoritative interpretation was successfully communicated to identifying, coordinating, stimulating, and analyzing processes of (evaluative) interpretation in practice (Sengers and Gaver 2006, p. 105)

This approach suggests we move beyond ‘evaluating’ our designs, and use examination of users’ experiences to support reflection on both interactive system design and the nature of the activities they afford. That is, we move beyond evaluating how effective our designs are at supporting creative expression, for example, and instead use them as provocative prototypes (Mogensen 1992), which stimulate examination of the nature of expression itself - as it occurs in a particular cultural context.

With this in mind, we see two ‘traps’ for artists and researchers in relation to evaluation:

  • Focusing on the artefact and neglecting the goals, behaviour and experiences of those who use it – and how these may differ from what was anticipated; and

  • Premature commitment to, and evaluation against, particular design criteria.

Because our work is primarily concerned with developing interactive systems for use in performance, we are careful to ensure that as well as evaluating the systems, we more broadly examine the full context of use. The term ‘examine’ here is deliberately chosen to indicate our intention to look beyond whether or not the system has met the design criteria we established, and instead see the system as a kind of ‘probe’ which may disrupt performers’ habits. These disruptions can highlight habits which may not have otherwise been readily apparent, and can help throw performers’ creative practices into sharp relief.

Framing these studies as examination, as opposed to evaluation, of the experiences of performers in the context of using a new interactive system is intended to keep the scope of investigation broad and allow us to discover any new approaches, conceptions and techniques which performers may develop. Keeping examinations of creative practice broad and open helps keep design criteria malleable. During creative development, the design criteria for our interactive systems are in flux. It is often the case that an element of the system intended for a particular use was repurposed. The improvisatory nature of the workshops which explored the Encoded interactive systems was deliberately fostered in order to continually re-examine their purpose and characteristics. Committing to a set of design criteria too early in the creative process, and evaluating the systems in those terms, would have risked shutting down options before they were even considered.

4.5 Performers’ Experiences with Encoded

The work Encoded was the end result of collaboration with Stalker Theatre, a dance/physical theatre company based in Sydney, Australia. As outlined previously, the work involved developing interactive systems in which the body movements of performers ‘stirred’ large projections based on computer-simulated fluid.

The creative development of the interactive systems for Encoded drew on theories of interaction and mapping (the linking of physical gestures to computer-generated audio visuals) to attempt to create an environment which encouraged conversational interaction (Johnston et al. 2008; Johnston 2011).

It is important to stress that ‘conversation’ in this context does not refer only to interactions directly analogous to human spoken conversation. In human-human conversation there are comparatively long time periods between one person saying something and the other person responding. The interactive conversation which we were aiming for in Encoded involved less structured turn-taking and much faster and more immediate feedback. As one of the interviewees observed, the interaction style is more akin to that experienced by practitioners of ‘contact improvisation’ (Paxton 1975) in which two dancers improvise while maintaining close body contact at all times. While one or the other dancer may take the initiative at particular times, the presence of direct physical feedback enables rapid and subtle communication and shifts in the balance between performers.

To explore the experiences of the artists involved with the Encoded systems, detailed observations of the rehearsal and development process were conducted and a series of interviews undertaken at different stages of the approximately 18 month development period.

Analysis of the interview and observation data lead to the identification of a number of key themes. These included:

  • The environment for interaction;

  • The process of assembling ‘components’ of the show into a coherent whole;

  • A degree of separation between the choreography and interactive system; and

  • A trajectory of creative development which began with improvisation and ended with composition.

4.5.1 The Environment for Interaction

Even though Encoded was a show which made extensive use of interactive systems, the existing and well-established techniques used in staged performances often work to prevent performers themselves feeling that there is much interaction going on. From a purely practical point of view, the fact that dancers are working in a space which is lit by powerful stage lights, as well as high-power projectors, means that their vision is significantly impaired.

It’s very hard [laughs] to interact when so often the, um, ability to see is compromised, whether it’s by lights in the eyes or projectors in your face… (Performer 2)

For stage performers this of course is not uncommon. The implications for designers of interactive systems for live performance though, are significant. If there is a desire for interactive systems to be used instrumentally or conversationally, then it will almost certainly be necessary to present the work in non-conventional settings.

Apart from lighting, there are also practical problems with positioning the audience in relation to the performers and the projections. In order for the audience to see both, the Encoded audience was placed in a more-or-less conventional position looking onto the dance floor with projections on the wall behind the performers. This meant that if performers were to meaningfully interact with the projections then their attention would need to be taken away from the audience, at least to some degree.

Um, I think in this performance, um, if you have, it’s funny I think because my back was so often to the visuals on the wall behind me… or you know or my performance presence is so often projected in the direction of the audience that I don’t know how often my attention was really drawn back to the visuals. (Performer 2)

4.5.2 The Process of Assembling ‘Components’ into a Coherent Whole

In general, the performers in Encoded saw themselves as part of a larger whole that was assembled by the director and choreographer. Partly, this was due to the scale and architectural style of the performance space (a large nineteenth century railway carriage works, repurposed as an arts venue), and the correspondingly large scale of the computer projections. This made it more difficult for the performers to get a sense of how their actions fitted in to the work as a whole.

This process was a familiar one for performers, as the quote below illustrates:

…it’s often the case in performance and dance performance that the dancers are…performing the steps but it’s this kind of grand vision that’s creating the work, that’s creating the story or the artistry. Maybe the dancers don’t need to know what they’re dancing about but if they dance the steps then the audience will still understand the story. (Performer 4)

While the performers were familiar with this way of working and were comfortable with it, it was another factor which led to a lack of direct engagement with the interactive systems during the later part of the rehearsal period and during performances.

4.5.3 Separation Between Choreography and Interactive System

The fact that Encoded was the first work by Stalker Theatre which made use interactive systems, and that these systems were built specifically for this show, meant that a significant amount of time was spent by the director in setting up the technology, auditioning various settings and pre-sets and generally tweaking the system. This meant that where there would usually be one person acting as director/choreographer, a dedicated choreographer was brought in to work specifically on dance movements so the director had the time and capacity to maintain control of the overall work.

In the case of Encoded this led to a reduction in the amount of improvisation from the performers, as the development of the technical systems, under the supervision of the director, and the development of the choreography, under the supervision of the choreographer, were often occurring in parallel.

This is not to say that the choreography and interactive systems developed in isolation – in fact there were frequent sessions in which performers were able to play with the interactive systems, which resulted in further refinement of both the choreography and the systems themselves – but performers and choreographer acknowledged that the interaction was more limited than it could have been.

So I think if we’d had a time, if we’d had time we could have done something more interactive… the wall didn’t really drive us it just kind of decorated us, what we were doing. …I think we could have done a lot more with that but I think the time constraints [limited us].

I looked at it [the interactive system] as another theatrical layer on top of what we were already doing. (Choreographer)

4.5.4 A Trajectory of Creative Development: From Improvisation to Composition

During earlier workshops, the separation between choreography and digital system was less apparent. In workshops and earlier performances, the performers were often working with much smaller scale projections – more in line with the size of the human body. As the workshops became more focused on producing a final, polished work the emphasis shifted from playful, improvised interaction to a far more tightly choreographed performance with tightly controlled transitions between interactive and visual states.

I enjoy the whole process but I definitely enjoyed the earlier stages, the play stages of where you really get to find ideas and really play with them… Towards the end when you just lock in things it’s a bit tedious. (Performer 1)

The process for Encoded, probably typical of many creative projects, began with improvisations, which were reviewed and reflected upon, gradually codified and finally assembled into a final composition.

After initial improvisations and discussion took place, videos of performances were reviewed and successful elements were identified and usually given a name. The digital artists then saved successful states of the interactive systems into pre-sets with these names. The director and choreographer would refine the movements which seemed effective with those pre-sets. Finally, movement sets and interactive system pre-sets were arranged and assembled into the final show.

While some performers lamented the reduction in playful interaction which occurred in order to produce a polished work, there was also a sense that as the final choreography and interaction states became more familiar, the was scope for some of the early playfulness and connection with the interactive system to return.

Then it flips and towards the end of the shows when you start embodying what you’ve made and really trying to find the connection with everything, it becomes fun again because you’re finding your own creative path in the framework that’s being built. (Performer 1)

4.6 Conclusions

The approach to practice-based research in live performance that has been outlined here attempts to maintain strong links between professional creative practice and research. In this kind of work there are a number of points where technology design and research can become separated from the concern of practitioners. A commitment to full creative partnerships, drawing on agile, iterative design approaches, is required to ensure that separation does not occur as a result of poor design. Research findings which result from unsuccessful collaborations can still be valuable, but it is unlikely that a badly designed interactive system will have a significant effect on the practice of performers, as they are likely to simply ignore it, or draw on the skills of their craft to work around its limitations.

In order to ensure that findings from research remain aligned with creative practice, we have found that careful observation and reflection, usually involving interviews and in-depth qualitative analysis, can result in findings which are grounded in the concerns of practitioners. The observations from the study of performers’ experiences with Encoded shows how findings from these kinds of studies can provide insight into the relationships between interactive systems, performers and the broader creative process. These methods do not guarantee that research remains in tune with practice, but the strategies of collaboration, iterative development, reflection and evaluation are all applied in order to ensure that the development of digital systems, their use in performance and the findings which result from careful examination of the impact they have on creative practice, remain aligned with the concerns of practicing artists.