Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

4.1 Introduction

At the highest level of abstraction, the principal objective for using sounds in interactive technologies is usually to communicate information to users in the form of auditory messages. The specific incentives for using auditory messages can vary widely, but some examples include:

  • improving usability by providing users with auditory feedback to indicate a successfully completed interaction. For example, in mobile phone interfaces sounds can inform users that they have clicked on an icon (Brewster 2002).

  • presenting data in an auditory format to aid domain-specific understanding and interpretation. For example, with seismic data (Quinn and Meeker 2001) or algorithms (Brown and Hershberger 1992).

  • alerting users to the occurrence of a particular event. For example, alarms on medical devices (Sanderson 2006).

  • making graphical interfaces (Edwards 1989) and data (Mansur et al. 1985) accessible to people with visual disabilities.

Although the scope of information portrayed audibly to users varies, the clear objective is to communicate information. We suggest that additional objectives can be realized by exploiting the auditory mode in different ways. There is the potential for sounds to be incorporated into interfaces in order to affect how people act when using a particular technology (i.e. the user’s behaviour) as well as affecting their experiences, including felt emotions and mood. To affect users through the auditory mode, a type of audio source that has been somewhat overlooked in human-computer interaction (HCI) can be used, that of music.

This chapter describes how other fields already exploit music for its affective qualities, with both behavioural and experiential motivations. In addition, the limited amount of research in the HCI area relating to affective musical interactions is discussed, including our previous work involving the Ambient Music Email (AME) extension to email clients. Inspired by the work from other fields, a research proposal for investigating affective musical interaction is presented. In particular, considering what aspects of user experience and behaviour might be affected with music, how we can measure the impact of music on users and which musical parameters can be manipulated.

4.2 Audio Interaction

Communicating information to users has been a predominant feature of HCI research since the field’s inception. Early interfaces conveyed information to users in the form of on-screen text alongside information that could be gathered audibly, for example, disc write sounds (Brewster 2003). The invention of the Graphical User Interface (GUI) permitted alternative methods to communicate information to the user visually, for example by using icons and colour. Relatively recent improvements in sound card capabilities have enabled much more detailed and precise information to be communicated audibly. This section describes how non-speech audio sounds and music have, historically, been used to communicate messages to users.

4.2.1 Non-speech Audio Interaction

Typically, auditory HCI research has concentrated on using three types of sounds; speech, sound effects e.g. Auditory Icons (Gaver 1986), and pitched tones e.g. Earcons (Blattner et al. 1989). With the latter non-speech audio types, the sounds have associated meanings that the user must decipher to understand the message that is being portrayed.

To communicate information quickly, non-speech audio sounds are often short in length. Although sometimes these sounds form part of an audio family or style, they can be quite disconnected, as they are typically distinct, separate entities. The original Earcon design proposal states that the motives employed must be kept short so that they do not resemble musical tunes which, when played repeatedly throughout the day, might irritate users (Blattner et al. 1989). Generally, the potential to annoy users with auditory interfaces has been the major aesthetic consideration taken by researchers. Typically, the intensity levels (or volume) of the sounds incorporated in audio interfaces is chosen with care, as the primary cause of irritation among users is thought to be sounds that are too loud (Brewster 1998).

Assigning meaning to individual or small groups of sounds is a compartmentalized, reductionist approach. The focus is on the messages communicated by the individual constituent sounds, rather than by the broader soundscape. With the exception of Blattner et al.’s (1989) concern about repeating Earcons, it is the annoyance caused by individual sounds rather than the impact of the wider soundscape that is usually considered.

4.2.2 Music Interaction

As auditory HCI research has evolved into using music as an additional form of non-speech audio the concentration on message communication has continued, although a more holistic approach is adopted. Music has been employed in interfaces to help users identify programming bugs (Vickers and Alty 2002), understand algorithms (Alty et al. 1997) and navigate unfamiliar locations (Jones et al. 2008). These examples of the use of music in an HCI context retain the primary objective of communicating auditory messages to users via the auditory channel. However, there are a few examples of HCI-related research exploiting the affectivity of music, described in Sect. 4.4 of this chapter.

Beyond a technological context, music is becoming ubiquitous in modern society, but not solely as a means of communication. People spend a large proportion of their time listening to music, whether this is focused, attentive listening (e.g. listening to an album to relax or going to a concert), accompanied listening where music is purposefully chosen to accompany an activity (e.g. choosing to listen to music on an mp3 player while exercising) or incidental listening (e.g. hearing background music in shops). Actively listening to music, or incidentally hearing music, rarely has the purpose of communicating a message; music is played for enjoyment or to provide a particular atmosphere. Given that people tend to listen to music for reasons other than receiving information, should auditory HCI research give more attention to alternative motivations when including music within interactive technologies?

4.3 Music as an Affective Medium

There is a growing body of evidence, from a wide variety of research fields, supporting the assertion that music has properties that can affect people, some of which is described here. Empirical research investigating the impact of music on consumers has been conducted in shops, supermarkets and restaurants, with a particular focus on music’s influence on peoples’ spending behaviour, with the ultimate aim of improving profitability. Additionally, empirical work has been performed in gyms where the objective was to investigate how music can improve sporting performance. In an education context, music has also been shown to impact on students’ performance. Furthermore, a branch of Music Psychology is specifically dedicated to research examining the relationship between music and emotion; investigating whether music can change how someone feels or whether people are simply able to perceive emotions expressed by a piece of music (see Juslin and Sloboda 2010).

It should be noted however, that the research areas that incorporate investigating the effects of music in their remit are not limited to those described in this chapter. For example, music is integrated into films and television programs purely for its experiential characteristics. Whilst the justification for music inclusion in this context is often based purely on the belief that music enhances the screen watching experience, there have also been some examples of empirical investigations towards understanding how music impacts on film plots (e.g. see Bullerjahn and Güldenring 1994).

4.3.1 Shopping

An objective of marketing researchers is to identify different tactics for increasing a company’s profitability. In a shopping context, this includes discovering which “atmospherics” (Kotler 1973) encourage consumers to spend more money. Marketing researchers have been investigating the affect that music has on purchasing behaviour since the 1980s; it is no coincidence that shoppers nowadays often browse and make purchases with a background musical accompaniment.

Music has been shown to impact on shoppers’ behaviour in a variety of ways. The tempo of the background musical accompaniment has a significant effect on the speed that consumers walk around a supermarket and the amount of money that they spend (Milliman 1982). In this study, Milliman found that slow tempo music makes people walk significantly slower around the supermarket. These consumers spent significantly more time and more money than those shopping with a fast tempo musical accompaniment. Does this result extend to an HCI context? Would the tempo of background music affect the speed that people browse when online shopping and thus, the amount of money that they spend?

The genre of background music has also been shown to affect the amount of money that people spend (Areni and Kim 1993) in a wine store. The real life purchasing behaviour of shoppers over a 2-month period was compared in two conditions, with background Classical or Top-Forty music. The results showed that a Classical music accompaniment caused the shoppers to spend significantly more money as they purchased more expensive products, though the volume of sales was comparable across both musical genres. Does this result extend to online shopping? For example, when purchasing furniture are people more likely to buy the expensive items if their browsing activity is accompanied by Classical music?

Another wine-purchasing study investigated how the nationality of the music being played in the wine aisle of a supermarket affected the type of wine that was purchased (North et al. 1999). Shoppers bought significantly more French wine when the music in the wine aisle was French and significantly more German wine when the accompanying music was German. This study was performed in a UK supermarket so was not confounded due to the country of investigation. Of further interest is the fact that the shoppers did not believe their wine purchases were influenced by the music they heard, though they did often accept that the French music made them think of France and vice versa. The authors state that “the finding is consistent with the notion that music can prime related knowledge and the selection of certain products if they fit with that knowledge” (North et al. 1999).

In an HCI environment, this result could suggest that if a particular online retailer wanted to encourage sales of, for example, a particular film then playing the theme tune, even if it was not an overtly familiar theme tune, may prime the customers into purchasing that film. It is worth noting that the methods adopted by these studies involved collating data from genuine shoppers that were spending their own money when purchasing real products from physical shops. As such, these studies have very strong ecological validity.

4.3.2 Restaurant Dining

Researchers have also investigated the influence of music on diners in restaurants. Music preference is a key indicator of how long people will remain in a restaurant although the tempo of the music was not found to have any effect (Caldwell and Hibbert 2002), contrasting with a study by Milliman (1986) where music tempo was shown to impact on the speed of dining. There is no technological equivalent of dining in a restaurant, although ordering food can be conducted online. Nevertheless, the dining context has presented yet more studies that consider tempo as a key element of music that can change behaviour as well as demonstrating that preference affects how long someone engages in a particular activity. This latter result may extend to HCI. If someone is performing a tedious task involving interactive technologies, accompanying this activity with music that they enjoy may subconsciously encourage them to continue the task for longer.

4.3.3 Gambling

As in shops, music is an integral element of the real world gambling experience. An observational study in amusement arcades showed that perceptions of the clientele’s musical preferences determined the type of music played in amusement arcades (Griffiths and Parke 2005). In areas of the arcade that were frequented predominantly by older, female players Easy Listening music was played. While in the areas and at times when the clientele was chiefly young men the background music was in a Rock or Dance style. Finally, the area of the arcade that catered for teenagers mainly played Pop and Dance music in the afternoons. This study was observational rather than empirical, but demonstrated that music is customary in the real world gambling environment. Despite the pervasiveness of music within casinos and amusement arcades there has been limited empirical attention focused on the affect of music on the clientele. Again, it appears that music forms an integral element of the gambling experience due to the belief that music increases profitability, rather than based on solid, scientific evidence.

4.3.4 Sport

Music can also have a positive impact on athletic performance. Researchers have played music to athletes both prior to (Bishop et al. 2007) and during (Waterhouse et al. 2010; Simpson and Karageorghis 2006; Edworthy and Waring 2006) sporting activities. These studies are both qualitative and quantitative in approach, but tend to focus on measuring changes in behaviour rather than experience (although Edworthy and Waring’s (2006) study does take subjective measures of Affect as well). It is unclear in the pre-performance studies if the athlete is engaged in another activity whilst listening or if they are focusing solely on the music. However, in the duration studies the music accompanies a specific sporting activity, whether this is cycling (Waterhouse et al. 2010), sprinting 400 m (Simpson and Karageorghis 2006) or walking/running on a treadmill (Edworthy and Waring 2006). As such, the music definitely acts as an accompaniment to another activity. There is evidence to suggest that sporting performance may be improved because the accompanying music acts as a distractor from discomfort (Waterhouse et al. 2010; Edworthy and Waring 2006). It is arguable that the focus of the athlete’s attention is quite strongly on the music and not the activity. Again, this leads us to consider whether a users’ sense of discomfort can be negated using music.

4.3.5 Education

Education is another key area where there has been much research focusing on the impact of music on students. A study by Thompson et al. (2011) focused on the affect of music tempo and intensity on reading comprehension. This study found that fast tempo, high intensity background instrumental music had a significant negative effect on reading comprehension, while slow tempo music had no significant effect in either high or low intensity conditions.

4.3.6 Music Psychology

Music psychologists have researched if and how music can elicit an emotional response in listeners for some time. There is some disagreement within the field as to whether listeners can perceive emotions from music (the cognitivist view) or if music can actually change listeners’ felt emotions (the emotivist view) (Krumhansl 1997). Given the differing music emotion theories it is important to distinguish between felt emotions and perceived emotions when discussing emotion and music. Our research takes the latter, emotivist viewpoint, which is supported by a growing body of empirical evidence (Livingstone et al. 2007). Extending this viewpoint to HCI, we ask if music can be included in interfaces to positively enhance users’ felt emotions, especially in stressful or boring situations?

The traditional approach of music psychologists when studying how music affects emotions in listeners is to conduct a laboratory-based study where music is played to participants and emotional reactions measured. The measurement techniques employed in these laboratory studies range from subjective measures such as self-reporting scales (e.g. Gfeller et al. 1991; Lychner 1998) to objective psychophysiological measurement techniques (Hodges 2010) including neuroimaging scans (Gosselin et al. 2006), heart rate and facial activity (Lundqvist et al. 2009).

Conducting experiments in a laboratory is a typical, valid psychological approach to a research question. Nonetheless, this does not really represent the true nature of most interactions that people have with music. Some music psychologists have extended their approach by considering how situational factors influence musically affected felt emotions. Someone’s reaction will normally differ depending on the situation they find themselves in. For example, the experience when listening to music in a concert is very different to the experience of hearing the same song played over a shop’s PA system (Sloboda and Juslin 2010). The discrepancy in experience is due to hearing music in an ordinary, everyday environment in comparison to hearing it on a special occasion (Sloboda 2010).

One method of overcoming the limitation of the focused listening approach adopted in laboratory experiments is to employ the Experience Sampling Method (ESM) (Larson and Csikszentmihalyi 1983). Sloboda and O’Neill (2001) and Sloboda (2010) used the ESM to investigate how people listened to music in their everyday lives. The participants in this study carried an electronic pager with them during waking hours for a week. At random intervals, once in every 2-h period, they received a page instructing them to answer questions about their current experience as soon as possible. Music was experienced in 44% of episodes but only 2% of those music experiences involved participants actively listening to music as opposed to the participants hearing music while undertaking another activity. This result reveals that focused attention on music is atypical of most listening situations. Instead, it is far more common for musical experiences to include passive hearing of music in the background with selective attention when needed.

When considering the best methods for evaluating interfaces with an affective musical component, laboratory-based experiments using subjective and objective psychophysiological measures should be considered, particularly with early-stage experiments. Later on, other appropriate methods can be adopted, including diary-studies and surveys, particularly when identifying areas that may benefit from affective musical interactions. Depending on the type of interface developed, the ESM may also be an appropriate method.

4.4 Music as an Affective Medium in HCI

The previous section described research from wide-ranging subject areas regarding the exploitation of music’s affectivity. While the major focus for using music in HCI has been on communication, there are a few instances where an affective objective has been adopted. This section provides details relating to some of the previous work that has been conducted involving affective musical interactions, within the HCI setting.

4.4.1 Computer Gaming

One domain where we might expect auditory interaction’s research focus to extend away from message communication is computer gaming. This is an area that is purely experiential; computer games designers already exploit music, sound effects and speech to create the optimum game playing experience.

Researchers have begun considering the impact that music has on gamers’ levels of immersion. A study by Sanders and Cairns (2010) identified that music preference i.e. whether the gamer enjoyed the music or not, significantly impacted on gamers’ immersion levels. The aim of the original study was to use music as a means of manipulating immersion in an investigation of the relationship between immersion and time perception. However, the authors found that the initial choice of music was not liked by the participants and therefore had a negative affect on immersion, to the extent that in the non-music condition the participants actually became more immersed. Repetition of their experiment with a different choice of music, that the participants enjoyed, had a positive affect on immersion.

Another computer gaming study took an objective measurement approach when investigating the physiological stress response due to built-in game music (Hébert et al. 2005). In this experiment, the participants played a first person shooter game (Quake III Arena, ID Software, 1999) in either the silence condition (no music or sound effects) or the music condition (built-in “pop-techno style” music only, with no sound effects). Saliva samples were taken at intervals after playing the game. Analysis of the samples revealed that the that cortisol levels, an indicator of stress, were significantly higher in the music group 15 min after completion of game playing “when cortisol levels are assumed to reflect the stress induced by the game” (Hébert et al. 2005).

Further research in a computer gaming context has investigated the affect that personal music preference has on driving game performance and enjoyment (Cassidy and MacDonald 2010). In situations were the participants self-selected the accompanying music for the driving game they enjoyed the experience more, whilst performance and experience diminished when the experimenter selected the music. This demonstrates that music preference is a key experiential factor, and when trying to positively influence users’ experiences, options for preference should be incorporated within the interface.

Given that computer gaming is a purely experiential activity and music is an integral element of the majority of computer games, the scarcity of research regarding the impact of music on computer gamers is somewhat unexpected. Although the field is emerging, it appears that music is incorporated in computer games on artistic merit and the anecdotal notion that music improves game-playing experience, rather than as a result of scientific verification of a hypothesis.

4.4.2 Typing Speed and Accuracy

One of the only examples of research investigating how music affects users’ behaviour in an HCI environment dates back to 1931. The impact of Jazz and Dirge music on a person’s typing speed and accuracy was evaluated alongside a control condition with no musical accompaniment (Jensen 1931). This study found that the speed of typing was significantly slower in the Dirge music condition. Further, while the numbers of errors in the Dirge and Silence conditions were comparable, Jazz music had a demonstrable impact on typing accuracy. The authors warn that this leaves “no doubt as to the seriousness of the influence of jazz music on typing, so far as errors are concerned” (Jensen 1931).

Nowadays, the relationship that people have with music is ubiquitous; music forms an ever-present part of daily life, considerably more so than it did in the 1930s. Given our familiarity with completing tasks while listening to music, including typing, would these results still stand today? Jensen does not pose any explanations for the discrepancy between the effects of Jazz and Dirge music on typing accuracy. However, we speculate that there are two possible explanations. Firstly, that it may have been due to the tempo of the Jazz music, which presumably will have had a faster tempo than the Dirge music. Secondly, it is likely that the Jazz music contained syncopated rhythms that may account for the increase in error rate. At present, these explanations are just supposition, but it would certainly be interesting to investigate this further in a modern setting, to see if the result still stands today.

4.4.3 Online Gambling

The influence of music on online gambling behaviour has recently been the focus of a couple of empirical studies. The effect of background music tempo on gambling behaviour when playing online roulette has been investigated in two studies (Dixon et al. 2007; Spenwyn et al. 2010). The risk of the bet (i.e. the amount of money spent) and the speed at which bets were placed were recorded for no music, slow music and fast tempo music (Dixon et al. 2007). The definitions of slow and fast tempo come from Milliman’s (1982) supermarket study (where slow tempo is less than 72 beats per minute (bpm) and fast tempo is greater than 94 bpm). The results showed that the music’s tempo had no affect on risk-taking behaviour. Although, the speed at which people placed bets was significantly higher in the fast tempo condition. A similar study by Spenwyn et al. (2010) concluded similar results. Here the authors speculated that the relationship between tempo and speed of bets is due to the increased arousal felt by participants in the fast tempo music condition. The authors also propose that online gambling websites should have an option to turn off musical accompaniment as players are more likely to become addicted with fast tempo musical accompaniment as the time for contemplation between bets is reduced (Spenwyn et al. 2010). Nevertheless, they also acknowledge that some websites or casinos may wish to profiteer by ensuring that the tempo of any music playing is fast, thus encouraging faster betting with less time for someone to consider the consequences of placing a bet.

Given the limited empirical attention given to the impact of music on gambling behaviour in real world casinos and arcades it is interesting that the virtual world equivalent has received considerably more empirical attention. This contrasts with the shopping situation where there has been much research in a physical context with little, if any, in the corresponding online environment. Perhaps this is due to the perception that as online gambling is an entertainment activity background music is more acceptable. The results from the online gambling research show that even in a laboratory-based gambling context music has a substantial impact on the players’ behaviour. The participants in these studies did not place bets with their own money, therefore conceding no risk, neither could they win any real money from the gambling activity. Although the ecological validity of studies conducted in this manner is reduced, significant outcomes can still be achieved, verifying the acceptability of the method.

4.4.4 Virtual Learning Environments

In an Education context, Immersive Virtual Worlds have been investigated to verify if they are an appropriate medium for learning (Richards et al. 2008). Here, researchers considered how music from computer game soundtracks affects learning, specifically remembering facts. This research found that, for one particular musical stimulus, the number of accurately memorised facts was significantly higher. The authors suggest that this piece of music may have been more congruent with the material being taught, hence the improvement in fact recall with this particular background music (Richards et al. 2008).

4.4.5 Email Management

One of the first pieces of empirical research investigating how music can affect someone’s emotional experiences, rather than behaviour, was performed by the authors of this chapter (Bramwell-Dicks 2010; Bramwell-Dicks et al. 2011). A Wizard-of-Oz prototype for an Ambient Music Email (AME) extension to email clients was developed. The AME prototype played continuous background Ambient music and when new emails arrived in a monitored email account, a musical notification phrase (a chromatic scale) played over the top. The objective for the AME was to positively influence user’s felt emotions by exploiting the affective properties of music.

The emotion altering potential of the AME was evaluated in a laboratory setting. Participants audibly monitored an email account whilst performing an occupying task. The email account was monitored under two conditions (i) using the AME prototype and (ii) using a standard email monitoring application. At pseudo-random intervals the email account received a new email prompting the participants to complete an online survey to assess their felt emotions.

The results showed that music did have a significant impact on how the participant’s felt emotions changed over time. In both conditions there was a drop-off in the positive felt emotions during the experiment, possibly as they became bored. However, the size of this drop-off was significantly smaller in the music condition than the non-music condition. In other words, the AME kept the participants’ positive felt emotions higher over the duration of the experiment. This was a positive result that adds weight to the argument that musical interfaces can be used to positively influence felt emotions. Contrastingly, the AME’s impact on the participants’ negative felt emotions had somewhat surprising results. Over time, in both conditions, the negative emotions increased; this is not surprising given the element of boredom in their task. In the AME condition, however, the size of increase in negative emotions was larger than in the non-music condition. Therefore, the AME increased negative felt emotions over time, which was an unexpected result, especially given the improvement in positive felt emotions.

A few explanations for this increase in negativity were offered (Bramwell-Dicks 2010). Further listening to the Ambient music used within the AME revealed that it contained repeated instances of discords that did not resolve. A discord is “a chord which is restless, jarring to the ear, requiring to be resolved in a particular way if its presence is to be justified by the ear” (Kennedy and Kennedy 2007). In the theoretical framework of underlying mechanisms that evoke emotions in music listeners, Juslin and Västfjäll (2008) include “expectancy violations” as a contributing factor. As such, the music’s characteristic of repeated instances of unresolving dissonance may be a causal factor that increased the strength of negative emotions felt by the participants. Though, this explanation requires further investigation before it can be verified.

Additionally, there were some methodological issues that may have added stress to the participants. The musical notification phrase that alerted the participants to the presence of an unread email in the account may have been too difficult for the participants to identify. A preliminary experiment was conducted to help choose the most appropriate notification phrase i.e. one that was easy to identify whilst also combining pleasantly with the background music. Participants in the preliminary study listened to the background Ambient music whilst performing the same occupying task used in the later AME study. The participants had to tell the experimenter whenever they heard a musical notification phrase over the top of the background music. This process was repeated for a number of potential notification phrases. The method employed in this preliminary study was very similar to that used in the AME study with one key difference – the time between instances of the notification phrase being played in the AME study varied between 1 and 7 min, while in the preliminary experiment there was only 15 s between notifications. Therefore, in the AME study the participants spent 20 min in each condition, but this was reduced to less than 2 min in the preliminary study, for each of the potential notification phrases.

The results from the preliminary study showed that all of the potential notification phrases were easy to identify, and the most pleasant ones were those that had not been pitch-shifted (or transposed). As such, it was felt that the chromatic scale notification phrase should be easy for participants to identify when audibly monitoring an email account. However, the experimenter observed that some of the AME study participants actually found it relatively hard to identify the musical notification phrase. It appeared that the participants had to split their attention between the occupying task and actively listening out for the notification phrase, rather than focusing their attention on the occupying task and passively hearing the notification phrase. The extra cognitive load required in splitting attention between the occupying task and active listening may have added some stress to the participants, thus affecting their negative emotions. In the preliminary experiment the participants were not engaged in the occupying task for sufficiently long periods of time between notifications to become engrossed in the task. In direct comparison, in the non-music condition of the AME study it was very straightforward for participants to identify the audible notification phrase, as there were no other sounds in the room at the time. Therefore, they were able to focus their full cognitive attention on the occupying task without a need to split their attention.

As a result, any future research using a musical interface that requires the user to distinguish features of the music to interpret the information being communicated will need preliminary studies that are carefully designed in such a way that they closely mirror the precise environment of the later experiments. In this example, should the preliminary study have been longer, with the time between instances of the notification expanded to match that employed in the AME study, the participants may have had to split their attention to identify the notification phrase and would therefore have reported it as slightly harder to identify. This is an important lesson for verifying the appropriateness of the sound design in any music interface that integrates alerts alongside background music.

4.5 Research Proposition

The aims for our research involving musically affective interactions are to identify if and how music can be included in interface designs to positively affect user experience and behaviour in a broad range of areas. The research will focus on the following questions. When music is integrated within an interactive technology:

  • how are the users’ experiences and behaviour affected?

  • what features of the music affects users’ experiences and behaviour?

  • what features of the users’ engagement with the music affects the users’ experiences or behaviour?

The first challenge is to refine these broad statements by identifying what elements of experience and behaviour we hope to affect with music (i.e. the dependent variables) and what musical features are to be manipulated (i.e. the independent variables).

4.5.1 Dependent Variables

There are a vast number of dependent variables that might be appropriate to research in this context. For example, does music affect accuracy when completing repetitive, boring tasks such as data entry? Can musical interfaces make stressful situations become more pleasant? Or can musical interfaces make mundane tasks more enjoyable? Generally speaking, behavioural characteristics can be measured objectively using quantitative methods, such as comparing time taken to complete tasks. While experience variables can either be measured subjectively by asking how the participant feels or objectively by taking measures of physiological responses e.g. heart rate.

4.5.2 Independent Variables

The independent variable may simply be two conditions, music versus non-music. Alternatively, the independent variables may focus attention on particular parameters of the music. There are numerous elements of music that can be altered as independent variables. These elements include tempo, pitch range, key, modality, dynamics, and rhythmic properties e.g. syncopation and whether the piece is lyrical or instrumental. Additionally, stylistic elements such as genre and instrumentation could be manipulated. Otherwise, properties of the listener’s engagement with the music can also be manipulated. For example, do they like or dislike the music? Is it familiar? Is the style of music one that they regularly listen to?

4.6 Potential Research Avenues

Music has the ability to affect how people behave and, arguably, how they feel. As such, there is clear potential for integrating music within technological interfaces to positively affect users’ experiences and behaviour. However, there is clearly also the potential for music to be incorporated in such a way that it causes negative experiences or has an exploitative impact on behaviour, particular with regard to consumers. At present, the idea of music integration for affective interaction may seem novel with no potential for mass adoption. Nevertheless, we argue that if music can be incorporated so that it improves behaviour and experiences then there is the potential for affective musical interfaces to become a typical feature of technology.

For example, maintaining and monitoring email accounts has become a vastly stressful experience for many people due to the wealth of emails sent and received on a daily basis (Shiels 2010). If the interface can positively affect the feelings of stressed email users by incorporating affective elements, including music, then there is the potential for the interface to be widely adopted. Additionally, if typing behaviour is more accurate and faster with a particular musical accompaniment then perhaps there is an argument that secretaries should be listening to an affective musical soundtrack when undertaking their dictation.

The online gambling research discussed previously demonstrates that even with no monetary risk or gain for the participants the influence of music on their behaviour was significant. Therefore, it seems fairly surprising that the marketing research regarding tempo and music genre has not been repeated in an online shopping context. Perhaps this is due to the concern that music does not align with an online shopping setting and could potentially annoy users. Nevertheless, given the prevalence for music in physical shops, it is a perhaps surprising that online shops do not incorporate music within their virtual shopping experience to endorse their branding and to affect purchasing behaviour online.

Research investigating the impact of music on peoples’ behaviour and experiences will always be constrained to a particular domain, some of which, at face value, may seem more appropriate than others. However, if it is shown that music can have a positive affect on people in one domain, it can be argued that music may also have the potential to positively affect people in other, seemingly less appropriate contexts. For example, the AME project incorporated music into an interface that runs constantly in the background; monitoring email is not normally someone’s primary attentive focus. In this case, positive emotions were enhanced, demonstrating that music can have a positive emotional impact on users in their primary attentive task as well as monitoring email, though future experiments will need to verify this.

The HCI field has been slow to incorporate music within interfaces, particularly with the aim of positively affecting users. However, as exhibited by this chapter, many other areas have already investigated the affective impact that music has on people. These fields regularly exploit this affective characteristic of music to positively influence behaviour and experiences. This chapter has, hopefully, demonstrated that there is great potential for future research in the area of affective musical interaction within an HCI context.