Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

2.1 Introduction

In interaction studies, there is a fundamental assumption that all else being equal, systems should be as easy as possible to use. This focus is evident in the literature. Nielsen’s (2003) list of five components of usability (learnability, efficiency, memorability, errors, and satisfaction) uses the terms “easy” and “easily” three times in five short sentences. It is good to remember both halves of the phrase attributed to Einstein (though apparently only a paraphrase): everything should be as simple as possible, but no simpler. There are cases where other goals must take priority at the expense of ease of use, and music interaction (the interactions between humans and tools in the domain of music) seems to be one of them. So what makes music interaction different?

We can begin with language. The term “user”, prevalent in the language of interaction studies, is a bad fit in music. It contains an implicit assumption that the computer is viewed as an inanimate object, in which the relationship of the computer to the user is that of a tool (Karlstrom 2007). Music systems occupy a spectrum of autonomy including what Rowe (2001) calls the “instrument paradigm” and the “player paradigm”. In the player paradigm the computer is viewed as an agent, and the human is better described as an interactor than a “user”. Even in the instrument paradigm the term “user” strikes some discord. One does not “use” an instrument to accomplish some ultimate goal: one plays it, and often that is the only goal. As Tanaka (2000) says, an instrument is not a utilitarian tool, which only needs to be easy to use in a specific context and whose development need only be characterised by ever greater efficiency. Instead, “What might be considered imperfections or limitations from the perspective of tool design often contribute to a ‘personality’ of a musical instrument” (Tanaka 2000). Indeed, efficiency is less important than engagement, a term which brings to mind the concept of flow (Csikszentmihalyi 1991; for more see Sect. 2.3).

Engaging, consuming, flow-like activities such as music are characterised by being at an appropriate level of difficulty: not too difficult, and not too easy. Often an activity which remains engaging in the long term does so at the expense of being rather painful to a beginner—in other words there is an important trade-off between ease of learning and long-term power and flexibility (Gentner and Nielsen 1996). Instrumental performance and practice, recording, mixing and production, live-coding and turntabling, the study of theory and notation—all of these are activities which take place in sessions that can last for hours and are mastered over years. Therefore the best interfaces for these tasks tend to fall towards the long-term power end of the trade-off.

One of the most characteristic aspects of music interaction is the extent to which skilled musicians become one with their instruments. Leman (2008) identifies the importance of this transparency: “Transparent technology should […] give a feeling of non-mediation, a feeling that the mediation technology ‘disappears’ when it is used” (Leman 2008: 2). This feeling is important to instrumentalists as artists and to skilled use of tools and systems in general.

Hand-in-hand with transparency goes the crucial concept of embodied cognition. Embodied cognition is a view of perception in which perception and action are inextricably linked (Wilson 2002). Leman (2008) argues that musical experience involves embodied cognition, rather than symbolic mental processing, even in the case of passive listening. On the other hand, Hunt and Hermann (2011) emphasise the divergence of experience between the player, inside the control loop, and the listener outside it.

Paine (2009) proposes that “the issue of embodied knowledge is vital in both the learning and teaching of musical performance skills and the relationship the musician has to their instrument”. He suggests that the capacity for an instrument (in the hands of an experienced player) to disappear from consciousness transforms it into “a conduit for expression rather than an object in its own right”. A musical tool which encourages internalisation of its concepts (van Nimwegen et al. 2004) seems essential for fluid, real-time use.

Armstrong (2006) suggests that the “prevailing guiding metaphors of […] HCI are at odds with the embodied/enactive approach”. Within interaction design, two subfields that do embrace the embodied perspective are haptics (Gillespie and O’Modrain 2011) and tangible interfaces (Hornecker 2011), both of which have frequently been used in music interaction design (Jordà et al. 2007).

Another distinction between music interaction and more general interaction studies is made explicit by Stowell and McLean (2013) in this volume: they say that applying typical experimental HCI techniques to musical tasks is in some ways useful, but “the experimental setups are so highly reduced as to be unmusical, leading to concerns about the validity of the test. Further, such approaches do not provide for creative interactions between human and machine.” Music interaction, it seems, must be studied in its native environment. More broadly, the language of experience design is perhaps more appropriate than that of usability for discussing music interaction. Experience design privileges consideration of the holistic experience of the interaction, which by nature is longitudinal, and must incorporate temporal changes in the human due to the interaction—see Sect. 2.3.

In order to do productive research in music interaction, it is necessary to correctly specify our goals. In many cases they do coincide with the typical goals of interaction studies, including the elimination of unnecessary difficulty. In others it is better to identify the aspects where ease of use should not be made a priority. In this chapter, then, we consider where and why music interaction should be difficult. Our goal is not a set of definitive findings but a framing of the questions and distinctions. Our scope includes all types of creative music interaction, including instrumental performance, virtual instruments and effects, laptop performance, turntabling and similar, notation and sequencing tasks, and production.

The remainder of this chapter is laid out as follows. In Sect. 2.2, a simple framework of multiple types of difficulty is set out. The learning curve model of time-varying difficulty, crucial to understanding long-term activities typical of music interaction, is described in Sect. 2.3. In Sect. 2.4, the sometimes counter-intuitive advantages of difficulty in music interaction are categorised. Section 2.5 describes aspects of music interaction where difficulty is genuinely undesirable and unnecessary, corresponding to areas where HCI and interaction design have the opportunity to contribute. Section 2.6 concludes.

2.2 Dimensions of Difficulty

Difficulty is not a one-dimensional variable. Various real and virtual instruments, software interfaces, and music hardware all exhibit their own characteristic types of difficulty, with different interfaces making some things easier and some things more difficult. Sometimes, there is a trade-off between these factors. In this section some dimensions of difficulty are categorised.

2.2.1 Physical Difficulty

Most computer software requires of users a minimal set of physical abilities: typing, pointing and clicking with the mouse, and looking at the screen. The same is true of music software in general, and studio hardware adds little to this set. However musical instruments can require a lot more. Pianists require at least an octave span in each hand. Stringed instruments require finger strength and, in early stages, some endurance of pain in the fingertips. Wind instruments can require great physical effort to produce the required air pressure, while holding a long note also requires physical effort and practise. The intense physical demands of rock drumming have been demonstrated by Smith et al. (2008). Long-term practice of instruments can lead to muscle and nerve injuries. In contrast, non-contact instruments such as the theremin and Sound Spheres (Hughes et al. 2011) make minimal physical demands.

Physical inconvenience can also be relevant, ranging from immobile equipment such as studios and church organs, to highly inconvenient equipment such as the double bass, down to pocket-size smartphone instruments and even to “disappearing” equipment (Kock and Bouwer 2011).

2.2.2 Difficulty of Dexterity and Coordination

All instruments require some dexterity and coordination. Many require the fingers, the hands, or the limbs to do different things at the same time. Often, there is a disassociation between the choice of notes and the control of timing and expressiveness. Some instruments require additional tools to be used as extensions of the body (or instrument), such as drum sticks or the bow for string instruments.

On string instruments such as guitar and electric bass, the presence of frets supports playing in tune, which is much harder to learn on a fretless instrument, such as violin or double bass. However, fretted instruments do not allow for the same flexibility in intonation and expressiveness (e.g., vibrato) as fretless instruments. In the case of guitar, this lack has been addressed by the use of additional devices such as the bottleneck slide and the tremolo bar (allowing vibrato to go down as well as up).

An analogous distinction can be made in wind instruments, where the trombone has one telescopic slide, instead of valves to control pitch. Circular breathing, required for some wind instruments, seems to present both a physical and a coordination problem.

Mixing on studio hardware requires coordination in terms of handling various controls in a timely manner, but many of these actions can nowadays be recorded and coordinated automatically by computer controlled systems. Interactive music software seldom requires much dexterity or coordination.

2.2.3 Difficulty Between Imagination and Realisation

A chief difficulty in the tasks of mixing and mastering of recordings is that of identifying the required changes, for example noticing an undesirable compression effect or an improvement that could be made to an equalisation setting. When the required change has been identified, making that change is often trivial. Although the main requirement is an internal “auditory imagination”, a good interface can help, for example by making A/B comparison convenient or by showing a spectrogram visualisation.

2.2.4 Nonlinearities, Discontinuities and Interactions in Control

The tin whistle’s scale is linear: within an octave, each higher note just requires the removal of the next finger. In contrast, the recorder’s scale has nonlinearities in which previously-removed fingers must be replaced for later notes. There is also a discontinuity in both, and in many wind instruments, when overblowing is required to produce higher octaves. Extreme examples of nonlinearity include the rackett, a Renaissance double-reed instrument with unusually complex fingering, and the button accordion, which features a two-dimensional layout of controls for each hand, and in some cases can be bisonoric (“push” and “pull” notes are distinct).

Nonlinearities and discontinuities are also common in synthesizer parameters. Interactions between parameters also cause problems (Seago 2013, in this volume). Much research into timbre control is aimed at reducing unnecessary nonlinearities, discontinuities, and interactions (e.g. Hughes et al. 2011).

2.2.5 Polyphony, Multiple Streams and Multiple Paths

It would be problematic to state that polyphonic instruments are more difficult than monophonic ones, since a fair comparison would require aspects other than polyphony to be equalised between the instruments, a condition that can rarely be achieved in practise. However it seems fair to state that playing a monophonic melody on a polyphonic instrument is easier than playing multiple lines simultaneously. Playing a pseudo-polyphonic piece such as a Bach Partita on a “mostly monophonic” instrument like the violin requires the performer not only to handle the multiple streams of music but to differentiate between them through dynamics and articulation. Live computer performers, turntablists, and studio mixers often have to handle multiple streams of music simultaneously, again imposing a larger cognitive burden (see Stowell and McLean 2013, in this volume). Some instruments and equipment allows any given action to be performed in multiple different ways, the simplest example being the choice of guitar string for a given pitch. This flexibility can be both an advantage and a disadvantage.

2.2.6 Difficulty of Repertoire

The violin and viola are very similar instruments, but because of the greater number of compositions written for violin, and the generally higher demands imposed on playing skills, the violin can be said to be more difficult in terms of repertoire.

2.2.7 Tuning Systems and Graphical Layout

In the case of traditional instruments, the tuning system embedded in the instrument’s design and graphical layout determines to an important degree how players conceptualize the interaction with their instrument.

On a piano, the notes in the scale of C are easily recognizable and playable as they correspond to the white keys on the instrument. Transposing a piece to another key makes the black keys necessary, and therefore changes the spatial pattern of keys to be played and the pattern of finger movements required. On a guitar, on the other hand, it is often possible to transpose a tune to another key by moving all the notes up or down the neck. Determining whether there are any flat or sharp notes being played is much easier on a piano than on a guitar, however. These are examples of how the graphical layout of notes on a musical instrument offers “representational guidance” (Suthers 2001) by facilitating the expression and inspection of certain kinds of information rather than other kinds.

Several string instruments (e.g., violin, cello, mandolin) have four strings tuned in fifths (e.g., G, D, A, E, from low to high, on the violin). This allows a player to reach a large range of notes to be played in a single position. On the double bass (and bass guitar), the four strings are tuned in fourths (E, A, D, G) instead. The greater dimensions of the bass do not allow the same range of notes to be played in one position as smaller instruments, so this difference in tuning is practical, but it also makes it difficult to transfer experience in playing one instrument to another. On a guitar, all of the six strings (E, A, D, G, B, E) are tuned in fourths, except one (the B-string). This is again practical especially for chord shapes, but makes scales which cross the G and B strings inconsistent. To make matters even more complicated conceptually, some musicians tune their instrument in alternative ways to make a particular tune easier to play, or to better accommodate their playing style (e.g., slide guitarists often tune in open tunings, such as D, G, D, G, B, D, to allow playing chord shapes directly with the slide).

In summary, considerations of physical playability can conflict with that of conceptual systematicity. Physics also constrains the set of possible tuning systems, especially in wind instruments. The variety of musical instruments and tuning systems in use today is a result of cultural history, showing that people are quite flexible in the relationships they recognise between musical qualities and physical ones.

2.2.8 Conceptual Difficulty

As explained in Sect. 2.2.7, the design of a music interaction system may support some conceptual activities and prevent or limit others. When we turn to other music interaction systems, other forms of conceptual difficulty arise. Sequencer programs often contain a wealth of functionality and options which may be difficult to find or remember. Mathematically-oriented synthesizer algorithms lead to large numbers of numerical control parameters which do not correspond to most users’ musical intuitions. Programming languages used in livecoding are another example of conceptual difficulty (see Stowell and McLean 2013, this volume).

2.3 Learning Curves: Difficulty Over Time

As previously outlined, all musical instruments, equipment, and software present the user with various types of difficulty in varying degrees. Such difficulties are not static, but dynamic. Time-varying aspects of difficulty are discussed in this section.

A natural model originating in the psychology literature is the “learning curve” (Bills 1934; Ritter and Schooler 2002). The learning curve is a task or activity’s characteristic pattern of difficulty versus time. Steep growth in the curve indicates slow progress. Note that difficulty does not always increase with time.

Several aspects of difficulty in music interaction can be described in terms of learning curves. For example, Wallis et al. (2013) in this volume discuss the idea of “maximum potential complexity”, saying: “there is such potential complexity in music that no individual can be said to fully master any instrument”. For non-trivial instruments, the learning curve remains non-zero indefinitely. Virtuosi push the limits of whatever instrument they choose. There is always room to grow, and if an instrument seems relatively easy in some respect, it encourages a more difficult repertoire.

The learning curve is useful in describing the difficulties experienced by musical beginners. Instruments such as piano with which musical sounds can be produced immediately, by default, have less of a “learning hump” (i.e. high values for difficulty at the very start of the curve) than instruments such as woodwind in which early attempts can produce distinctly non-musical sounds. Fretted instruments avoid one of the major early learning humps associated with free-pitch instruments such as the violin. Polyphony is another early learning hump avoided by some instruments. In contrast, “instruments such as diatonic harmonica are not hard to take up initially, but have large challenge jumps corresponding with times when advanced techniques such as note-bending must be learned” (Wallis et al. 2013, this volume). Hence it is essential to model these difficulties as time-varying.

It seems natural to think of the “ideal” learning curve as being initially low with gradual increase in difficulty over time. Such a curve would have the advantages of not discouraging beginners, rewarding effort, and remaining non-trivial indefinitely. A key concept is that of flow (Nakamura and Csikszentmihalyi 2002). Being in flow means “the subjective experience of engaging just-manageable challenges by tackling a series of goals, continuously processing feedback about progress, and adjusting action based on this feedback.” In these circumstances, people sometimes experience enjoyment of the activity for its own sake; loss of self-consciousness; focussed concentration; and a sense of control. It is regarded as a highly positive experience, both in terms of enjoyment and in terms of effective learning or productivity. Both too-easy and too-difficult tasks can break flow, so the ideal learning curve again exhibits a gradual increase in difficulty.

Van Nimwegen et al. (2004) distinguish between internalised and externalised learning. They show that systems which require the user to internalise knowledge (as opposed to relying on external cues) present greater initial difficulty, but are more robust to disruption. Internalised knowledge leads to greater independence, “better knowledge”, and long-term retention. It seems essential to fluid, real-time use of any system.

Teachers, instructors, or mentors often play a role in stimulating development and motivation by selecting material that is just within reach. As described by Hedegaard (2005), Vygotsky’s concept of the zone of proximal development has been influential in informing instructional planning. This refers to the difference between the tasks that a learner is capable of with and without expert guidance (Hedegaard 2005).

2.4 Where Music Interaction Must Be Difficult

In many cases, what difficulty exists in music interaction is not easily eliminated because it is inextricably linked with some property seen as desirable. Some such properties are set out in this section.

2.4.1 Open-Endedness and Long-Term Engagement

For many musicians, composing music or playing an instrument is an end in itself, not a means to an end (Swift 2013, in this volume). An appropriate degree of difficulty seems to be inextricably linked to the motivation for such autotelic activities (Wallis et al. 2013, in this volume; Nakamura and Csikszentmihalyi 2002). Satisfaction in musical activities derives partly from accomplishing goals of appropriate difficulty. There is a great contrast with the tools and interfaces typically studied in usability research, where the aim is usually to “get the job done” as quickly and efficiently as possible. When the end-goal of an activity is for the sake of enjoyment of the activity itself, a suitable level of difficulty becomes acceptable and even beneficial.

A second component of the motivation for autotelic activities, in general, is the potential for long-term engagement. A skill which can be mastered in the short term may be of interest as a means to an end, but less so for its own sake. Long-term engagement with a skill such as composition or performance is possible because they are unlimited both in the possibilities and the challenge they can offer. Most musical instruments have large and varied repertoires, and with the possible exception of a few trivial instruments, no player can be said to have ever fully mastered an instrument.

It is interesting to think about cases in which music interaction is limited. Popular video games like Guitar Hero and Rock Band (Harmonix Music Systems 2005, 2007) offer a good case study. Such games are very popular, and anecdotal evidence suggests that they can kindle an interest in music and musical performance in non-musicians. They seem to promote long-term engagement to a surprising degree. Research suggests that playing these games “feels like” making music “because the affective experience of making music is so bound up with embodied performance”, which includes elements of theatrical play and social interaction (Miller 2009). However, a real disadvantage is that these games seem to be dead ends. The “guitar” controller is not intended for independent musical control, since its controls depend on game context. There is a discontinuity between the game controllers and true guitars. In contrast, drumming games usually feature a true MIDI drum kit as the controller, albeit a cut-down one. Real drumming skills transfer directly to such a kit and vice versa. Therefore there is the potential for a beginner in such a game to experience unlimited growth by switching seamlessly to a real drum kit when the game is outgrown.

It is easy to imagine that such a process might become a fundamental part of a musician’s self-narrative and self-image. One’s self-narrative as an instrumentalist includes experiences such as early memories of an instrument, first skilled performance witnessed, first ownership, and so on. One’s self-image is an important aspect of autotelic activities, including components such as one’s sense of autonomy, of dedication, and of authenticity. An instrumentalist is not only a player of music, but also a fan who has witnessed virtuosic performances, and may regard favourite players as role models. As such, there is little possibility of “blaming the tools”: instead there is a strong positive example to help the beginner to work through an initially steep learning curve. One’s relationship with the instrument might be strengthened by a feeling of having been together “through thick and thin”: again, difficulty contributes to long-term engagement. In contrast to typical HCI, errors in performance can even lead to new interpretations and goals, for example in instrumental improvisation.

A final point in this connection is that our aesthetic senses seem tuned to detect and enjoy open-endedness. A sense of mystery has been shown to appeal broadly to all humans, in both real landscapes and landscape art (Thornhill 2003). It is the idea that one can see far enough ahead to know that there is more beyond. The appeal of these features has been explained in evolutionary terms. We are naturally curious creatures. “Mystery is the promise of more useful information, and the mental inference of mystery draws us into the scenario for more information gathering” (Thornhill 2003). It has thus been seen as a fundamental root of our visual, and possibly also cross-domain aesthetics (Ruso et al. 2003). Open-ended activities including musical composition and the learning of instruments may trigger this evolutionary sense of enjoyable mystery in the same way.

2.4.2 Expressivity, Creativity and Flexibility

Ease of learning in interfaces tends to go along with being locked-down, inflexible, inexpressive, or non-amenable to creative, unexpected use. It trades off against the long-term power of the interface, as described in Sect. 2.1.

A common technique in HCI is to hide rarely-used controls in menus or configuration panels (Nielsen 2006). Contrast this with the interface of a typical mixing desk. Such an interface is clearly not optimized for the beginner’s ease of use. It requires an assumption that studio engineers are willing to take time to learn the interface well. It is motivated partly by the consideration that a mixing desk must be used in real-time. Menus are useful for organizing and hiding controls, but take time to navigate. For intense, real-time tasks, there is a benefit to having every single control and indicator immediately available to the eye and hand. In this, mixing desks may have something more in common with airplane cockpits than with office software.

Beginners sometimes find western music notation a gratuitous obstacle. Since the beginner starts by mentally translating every note from the staff to a note-name and thence to a physical action, it seems natural to do away with the staff and simply use note-names (C E G C’) or solfège (do mi so do’) as notation. However musical notation has been optimized for a different scenario. It makes patterns and motifs in the music visually obvious, so that an experienced musician can read and play a semi-familiar piece live. This requires a type of gestalt comprehension of perhaps multiple bars at a glance that could not come from text.

The trade-off is again evident in sequencer software. Programs such as GarageBand (Apple 2011), a simplified sequencer, are appealing and relatively easy to use for beginners. Seasoned users might find that it lacks options or that some tasks are relatively cumbersome or slow. In contrast, a “tracker-style” sequencer interface such as Jeskola Buzz (Tammelin 1999) is intimidating to beginners, but seen as irreplaceable by its more experienced fans. Advanced users who prefer such interfaces tend to take advantage of the relatively high-capacity keyboard input channel, as opposed to the low-capacity mouse channel.

The type of creativity which drives some saxophone players (for example) to use malformed notes, squeaks and wheezes as part of their music cannot be predicted or allowed for. Think of the artist who sets out to produce a painting but ends up with a sculpture. Hofstadter (1979) uses the phrase “jumping out of the system” for such creativity. If a system is rigid, then patching it to allow a particular, anticipated jump might only make it more rigid for other, unanticipated ones.

2.4.3 Demonstrating Virtuosity

In musical performance, demonstrating virtuosity may be central to the intended experience (Wagner 1830). Virtuosity cannot exist without difficulty. Virtuosity must also be identifiable as such by the audience. Wallis et al. (2013) in this volume argue that demonstrations of skill are an important motivating factor for people to learn and play musical instruments. Levitin (2006) suggests that musical ability, like any craft, carries a level of prestige associated with the commitment of time and energy required to attain mastery.

In order to attain mastery over an instrument, it is helpful if the design of the instrument remains relatively stable. Massey points out that “constant design changes make truly skilled use impossible” (2003). This sets up an interesting contrast between traditional acoustic instruments, and digital music interfaces. Where an instrument such as a violin has a rich cultural history of continued use and exploration, software interfaces such as a Pure Data patch may be created for a single performance. Whilst there is certainly skill associated with creating a Pure Data patch, this skill may not be evident to the audience.

The issue of demonstrating virtuosity is of particular concern in a digitally mediated context, such as laptop performance, where the disconnect between physical gestures and sonic results obscures the skill of the performer, resulting in a loss of perceived performativity (Stuart 2003) and authenticity (Paine 2009). Collins comments: “Unfortunately, much of the complexity of these real-time systems [generative, etc.] is lost on a potential audience, excepting those few connoisseurs who sneak round the back to check the laptop screen. An artist using powerful software like SuperCollider or Pure Data cannot be readily distinguished from someone checking their e-mail whilst DJing with iTunes” (Collins 2003: 1). Artists in such cases may feel unappreciated, and the audience short-changed.

In live-coding (Stowell and McLean 2013, this volume) program code will often be projected for an audience to see. While the audience may not understand the code per se, live coders argue that the audience can nevertheless recognise virtuosity in this manner (TOPLAP 2011). Other methods of informing and indirectly educating the audience may also have a role to play.

2.4.4 Communicating Effort and Emotion

As remarked above, the complexity and skill of a laptop performance is often invisible to the audience. A related problem concerns physical effort. Audiences have been trained to associate physical effort in performances with emotional intensity and commitment to the music. Physical effort intensifies the experience of the audience. This is true even in the refined setting of the symphony orchestra, where the audience sit quietly but the players and the conductor may work up a real sweat. Indeed a more general association between physical effort and commitment is the inevitable consequence of an evolutionary environment in which important tasks were physical and social actors required the ability to judge each others’ contribution to a shared task, and the relative difficulty of different tasks. In the case of musical performance, the specific problem is that although virtual instruments and laptops have removed the need for the player to input significant energy to produce an intense performance, the audience’s desire for commitment and authenticity on the part of the performer remains. Thus even a painful struggle on the part of the performer is not entirely negative: “One has to suffer a bit while playing” (Krefeld and Waisvisz 1990). The audience may also require cues from the performers to help recognise, for example, a particularly intense or emotional section of the music. “[T]he physical effort you make is what is perceived by listeners as the cause and manifestation of the musical tension of the work” (Krefeld and Waisvisz 1990). It is awkward or impossible to communicate such intensity via a laptop instrument which requires only typing, or a smartphone application whose control surface is just a few square inches, or a virtual instrument which requires no physical contact whatsoever.

2.5 Where Music Interaction Could Be Easier

Despite the arguments put forward above, there remain aspects of music interaction which do not benefit from difficulty. These are areas where interaction studies have made contributions to music interaction, or have the opportunity to do so.

2.5.1 Transient and Frivolous Music

Many musical activities are primarily intended to be transient and/or frivolous. Good examples include Guitar Hero and Rock Band (Harmonix Music Systems 2005, 2007) and similar games, and (according to musical tastes) “air guitar” and “air conducting”. In the latter there is no aim of demonstrating virtuosity or performing, simply the enjoyment of pretending to play along. Some instruments seem particularly associated with frivolous playing, e.g. the ukulele is seen as easy to pick up and play while doing something else, such as watching television. Many virtual instruments, smartphone instruments (e.g. Smule 2008) and generative methods (e.g. Earslap 2011) may also be described as close to being toys. This is no criticism, merely recognition that in such cases, the aim is to allow the user some type of expressivity without difficulty. In the ideal case, the user will get something good, but something different, no matter what he or she does. A simple technique is to map all gestures to diatonic scales, to avoid many obviously wrong notes. “Bloom” (Eno and Chilvers 2009) not only constrains the user’s inputs to a diatonic scale, but it allows the user to do nothing at all and still obtain interesting music. On the other hand, it prevents fine-grained control of the music.

2.5.2 Peripheral and Technical Tasks

Many musical tasks can be regarded as being of a technical nature, or as inessential to the artistic process. Removing the burden of these tasks would help musicians to concentrate on the essentials. As a rather trivial example, tuning up is not a core task, but it presents a minor difficulty to some players. Electric guitars with software-based modification of tuning (Line 6 Variax guitar, introduced in 2003), and automatic mechanical tuning systems (AxCent Tuning Systems 2013 and Gibson’s robot guitar, introduced in 2007) are already available.

Many musicians do not want to spend time creating new timbres. For them, FM or granular synthesis parameters, for example, with their technical names and many nonlinearities, discontinuities and interactions, are an obstacle and a distraction from the core goal. In these cases it is useful to provide a good set of presets. There is also motivation for easy-to-use, simplified interfaces based on ideas like iterative search (see Seago 2013, this volume; McDermott et al. 2007).

Many musicians avoid mixing and mastering of their recordings. The vast majority of professionals out-source these tasks. Among amateurs, a shortcut like the mythical “soundgoodizer” would be very popular. Only a few musicians acquire the expertise to use production as an expressive tool. The types of tools best suited to these two groups differ in obvious ways. Similar remarks apply to live sound engineering and to some sequencers and hard-disk recorders.

2.5.3 Learning Humps and Layered Affordance

One obvious disadvantage of difficulty is its toll on beginners: they struggle, suffer from wrong notes and inflict them on those in their vicinity, and sometimes give up. Among fourth-grade students, the perceived difficulty of an instrument is the main factor in deciding not to play it (Delzell and Leppla 1992). Musical pedagogy is concerned with ways of helping beginners past these “learning humps”. A music curriculum for young children (NCCA 2011) suggests using simplified scales such as the pentatonic (p. 100), simplified notation such as “stick notation” for rhythms (p. 93), and “stepping-stone” instruments such as the recorder (p. 106).

The same is true in research into new music interfaces. A good example is the “Haptic Bracelets” (Bouwer et al. 2013a, this volume). A haptic learning system helps to make building-block skills easier to acquire, reducing an initial learning hump. Later “the training wheels come off”, and the user’s ultimate goal is to reach an unaided mastery. The violin training system of Johnson et al. (2011) helps players learn good bowing technique initially, using real-time feedback, and importantly provides a constant guard against regression to bad technique. In the field of harmony, regarded as difficult due to its many abstract concepts and rules, employing embodied cognition to make the subject matter more concrete and engaging seems promising, as in “Song Walker Harmony Space” (Bouwer et al. 2013b, this volume).

Such examples demonstrate that in some areas the right interfaces and systems can help beginners without detracting from the positive aspects of difficulty. Perhaps further examples are possible. Imagine a “beginners’ violin” which restricted notes to a well-tuned pentatonic scale for the first year, allowing the beginner to learn basic skills like stance and bowing in a relatively painless way. This restriction could be lifted gradually, to allow a full diatonic and then chromatic scale, and eventually also the in-between pitches needed for vibrato and slurs. Such an instrument would grow with the player. Crucially, such an instrument would not represent a dead-end, since with the lifting of restrictions the player would have unlimited room for growth. The learning curve for such an instrument would be far less intimidating than that for a standard violin. A partial implementation might require little more on the technical side than an electric violin, “auto-tune” software, and headphones.

There are two possibilities for controlling the gradual increase in difficulty which would be needed for such a hypothetical instrument or interface. In some cases the player could choose when to lift the restrictions and increase difficulty. This is layered affordance. An alternative is to have the system use heuristic learning methods to judge when the player or user is ready for a more complex, difficult, or open interface. This intriguing possibility might be termed adaptive affordance. Compare the ideas of “progressive” and “staged disclosure” (Jones 1989; Nielsen 2006).

Some existing systems use variations on this idea. A good example is the Continuator, or musical flow machine, of Pachet (2003). It is intended to produce “flow” experiences in the user by adapting the complexity of its musical interactions to the user’s skill. Another example is the Jambot (Gifford 2013, this volume), which provides real-time rhythmic accompaniment to a human musician, and aims to produce a level of rhythmic complexity that is complementary to the rhythmic complexity of the human performer. Computer game designers also understand that to keep players engaged the game must be easy to start, and increase in difficulty as the player becomes more skilled. Computer games, including music-oriented ones such as Guitar Hero and Rock Band (Harmonix Music Systems 2005, 2007), implement this behaviour through the use of “levels” of difficulty. The level is self monitoring, in that mastery of a given level is the trigger for the next level of difficulty.

2.5.4 Instruction and Meta-Cognition

Musical training is a field where one-on-one instruction and apprenticeship is very common, and often seen as important for musical growth. Intelligent tutoring systems and intelligent learning environments allow, to some degree, computer-based personalized instruction, which offers potential for applications in music, including score-reading, improvisation and composition tasks (Brandao et al. 1999; Holland 2000). Cook (1998) has emphasized the importance of dialogue in open-ended creative domains such as music, and studied interactions between music teachers and students related to motivation, creativity and critical thinking. His framework for knowledge mentoring can help recognize opportunities for instructional planning to stimulate meta-cognitive activities. Another important factor in acquiring musical skills, besides the amount of practice, is the method of practice. In studies comparing musicians of varying skill level, advanced musicians have been found to use more complex, more abstract, and more flexible practising strategies than less skilled musicians (Gruson 1988). Combining work in this area with learning environments could lead to technology that supports musicians in improving their practising strategies.

2.6 Conclusions

It is fascinating to analyse music interaction using the methods of HCI and interaction design. It tells us something about music interaction, but perhaps also something about the methods of study. The following gedankenexperiment makes the point well: “If our field [interaction design] had existed at the time that these musical instruments [accordions and others] were evolving, would we have told them [their designers] to toss the design in the trashcan as TOO COMPLEX for any users to master?” (Boese 2006). It illustrates an important distinction. The fields of HCI and interaction design are not wrong in their assumptions and findings that users sometimes find interfaces frustrating, and that productivity can be improved through good design. However the very vocabulary being used here—users, frustration, and productivity—seems ill-suited to describe music interaction. Users are musicians. Productivity can’t be measured. Frustration is part of a musician’s growth. A musician who learns and plays for love of music is in a very different mind-set from that of a software user, impatient to carry out a task.

Within the broader domain of music interaction, this chapter has been focussed on difficulty. A simple taxonomy of dimensions of difficulty and the learning-curve model have been described. Various aspects of music interaction have been set out in which difficulty is counter-intuitively positive, or linked to positive features. In these cases, it is clear that typical interaction studies techniques should not be applied blindly with the aim of banishing difficulty. The “user-friendly” interfaces likely to result might turn out to be uninteresting to musicians.

Some aspects of music interaction have also been described in which difficulty is, on the other hand, negative and unnecessary. In these cases there are opportunities for interaction studies to be applied and to make clear improvements to existing systems. One can speculate that musical instruments, and in particular their capacity for long-term engagement, flexibility and expressivity could serve as a model for new musical systems: making new musical interfaces with these qualities is a challenge and an opportunity.

In all cases, it seems to be essential to recognise music as a distinct set of activities with distinctive goals and mindsets. Music interaction must be studied in its native environment. The contributions to this book, many already mentioned in this chapter, approach music interaction using HCI and interaction design methods, but informed by experience and insight into music and musicians.