Keywords

1 Introduction

In this paper, we argue that brightness and colour animations are useful to novice users as feedback for sound zone systems. A sound zone system is a speaker-based solution to create delimited areas of sound in a room. There is a clear boundary for sound, and users’ proximity to a sound zone inversely impacts the amplitude of the sound. This allows for multiple persons to share a room while listening to different music tracks or other sounds without using headphones, thereby enabling social interactions and higher awareness of the surroundings. Sound zone systems have reached a state where they are considered useful for various settings, including car cabins [7], hospital wards [33], and homes [22]. Even though limitations still exist for constructing sound zone systems, they are on their way to becoming commercially available. Given the imminent availability to consumers, ensuring suitable feedback mechanisms to the sound zone users poses important challenges for human-computer interaction.

Our work is anchored in ethnographic research on social and subjective experiences of sound in homes [22]. One challenge relating to feedback and control of sound zones is that the sound inhabits a physical space in a way that is significantly different when compared to other sound systems, e.g., headphones or current standard home speaker systems. Offering more background into this, Lundgaard et al. [23] position sound zones as one type of technological intervention into soundscapes, unique for their spatial properties that headphones and standard speaker systems do not have. Since sound zones are inherently intangible and invisible, their spatial properties are not obvious. This necessitates the development of interfaces and displays to provide relevant feedback about the new properties that sound zones introduce, including size and overlaps. For novice users, forming an understanding of overlaps is important. Overlaps can happen when adjusting the volume, and thereby size, of a sound zone. Previously [17], we investigated direct and indirect representations of the size and location of sound zones, comparing a “spot-light” projection on the ceiling to an LED strip on the wall. This study showed potentials for exploring indirect mappings further, leading us to investigate the use of colour and brightness as part of small objects on a table (see Fig. 1). Early research has shown that subjective estimations of brightness and loudness stimuli were analogous [26]. We extend this by investigating how light and sound are experienced together.

Sound properties can be visualised in different ways, both for artistic purposes [28] and for sound design [27]. Lima et al. conducted a survey on visualisation techniques of musical features, showing that colour is used extensively for tone, frequency, and harmonic structure [16]. Here, we are interested in using light for visualising information in the novel context of sound zone systems. We consider light as an opportunity for feedback about the spatial properties of sound zones. Previous research shows that light affects people’s perception of physical properties of objects and spaces, such as dimensions and weight [21]. While light and sound waves are different, they also share common traits (e.g., high intensity at the source and a gradual drop-off out into a physical space) that we suggest are useful for visualising complex spatial behaviour of sound.

Fig. 1.
figure 1

Controllers with LED strips.

Full size image

In this paper, we present two consecutive studies, investigating the use of colour and brightness as feedback for sound zone volume, size, and overlaps. We ask the research question: How can feedback about sound zone volume, size, and overlaps be provided using light? In Study 1, we investigated colour and brightness as feedback about volume and size of one sound zone. In Study 2, we focused on sound zone overlaps. Novice users experienced animated light patterns and were asked how well each pattern provided information about overlaps, size increase and decrease, and how well it notified them of these issues. We also gathered qualitative insights into how novice users experienced sound zone behaviour in relation to particular light feedback. With this research, we provide insights that are useful in feedback design for sound zone systems in particular. This can also be extended to other sound systems with similar spatial properties.

2 Related Work

2.1 Personal Sound Zones

As part of a broader paradigm shift from channel-based to object-based audio reproduction [12], there is a potential for users to control new aspects of their soundscape. Spatial sound has become a subject of investigations in HCI, e.g., for connecting auditory and biomechanical cues [30] and for designing auditory displays where visual displays are inappropriate [34]. One type of system in the object-based paradigm is sound zones. The vision behind sound zone systems is to provide different users in the same shared physical space with personal listening experiences without wearing headphones [3]. This can be useful in many situations, such as shared office spaces, car cabins [6], and homes [22], the latter of which is expanded below. Sound zones can be implemented in different ways. Examples of commercial products are the directional speakers Audio Spotlight by Holosonics, Sennheiser AudioBeam or Soundlazer that use ultrasound. In this case, the ultrasonic waves can only be heard when they collide. Limitations to these systems exist such as their inability to reproduce low-frequency sound. Another method for creating sound zones is by using a combination of microphones and loudspeakers. Microphones capture the sound pressure of audio signals directed at predefined areas, and those measurements can then be used to create filters that enhance the acoustic contrast between the zones [7], i.e., the decibel difference of an audio signal between the area where a user wants to hear it and the areas in which users do not want to hear it. To help achieve this, Rämö et al. present a model which shows that if the acoustic contrast between two or more sound zones is too low, users become distracted by the chaotic mixing of sounds [29].

Previous research provides an overview of the interaction design challenges for domestic sound zones and approaches to addressing them [23]. Sound zone systems are intangible and invisible, making it difficult, especially for novice users, to understand their behaviour in comparison to experiences they have with other sound systems. This behaviour is further complicated by different implementation methods for sound zone systems typically resulting in different behaviours. Current methods result in low acoustic contrast when sound zones are close to each other or when the volume of one sound zone is much higher than another sound zone in the same room. We refer to this as overlaps.

A study on domestic situations for sound zones points to different social aspects of using sound zone systems [22]. For example, one use situation includes persons who engage in their own sound-producing activities but stay within the same room. Here, they are not actively engaged in social activities which also means that they may not be aware of how their own sound is affecting the others. In another situation, persons may want to listen to the same sound from a television but have different preferences for sound properties such as the volume. This means that the persons might be actively engaged with each other, changing their awareness of how their own sound adjustments affect the other. The research by Lundgaard et al. [23] includes descriptions of design concepts to visualise sound zones. These visualisations particularly focus on projections on floors or ceilings. Jacobsen et al. [17] recently presented an experiment evaluating the precision with which users can position themselves and sound zones from either ceiling projections, a relative wall display, or no visualisation, respectively. They show that visualisations aid users significantly in positioning tasks. Similarly, Johansen et al. [18] conducted an elicitation study of shape-changing visualisations that showed the effect different visualisations have on user experiences of and expectations for sound zone behaviour. These prior studies suggest that visualisations play an important role in sound zone interaction. All of them, however, rely on users having to map a 2-dimensional visualisation with a relative position to the 3-dimensional space within which the sound zones exist.

2.2 Light as Feedback

Using light for visualising information is not novel outside the context of sound zone systems to, e.g., show states such as on or off. A common way to do this is through point lights where one small LED lights up. This can be varied at different brightness levels and in different colours. Previous research investigates how the vocabulary for light can be expanded with more expressions that still make sense to users [14]. This includes categories for notification, active state, low energy state, and turning on. In relation to using expressive lights in robots, research shows that participants’ interpretation is dependent on the situation [1].

Many conventions and culturally inherent meanings are attached to colours. Early research shows that colour affects the perceived size of an object [35]. Similarly, dark coloured objects tend to be perceived as heavier than light coloured objects when the objects are only observed and not touched [32]. Löffler et al. investigate how these psychological traits can be combined with metaphors to convey meaning through colour in tangible user interfaces [21]. Their study shows that there can be other uses for colour that not only come from culturally derived symbolic meaning, but are instead based on sensorimotor experiences independent from language and culture. In terms of brightness, previous research within architecture shows an effect of light on perceived size and dimensions of physical spaces [20, 24]. The research shows that increased brightness is related to increased perceived size of a room. Furthermore, different light patterns affect perception of dimensions. For example, vertical lights projected onto a church resulted in perceptions of a taller church compared to no projections. Impressions of these spatial properties is shown by a study by Lindh et al. to relate to the gaze of participants [20]. They found that participants tended to direct their gaze towards areas with high brightness.

In previous research and art, sound has been visualised in many ways. This includes 2D visualisations of sound in 3D space, e.g., for the design of GUIs [4, 13]. Another example includes using opacity to represent sound volume [28] with a broad focus on generation of spatial forms from music. McGregor et al. developed visualisations of soundscape elements to assist interaction designers in capturing data about and identifying problems within a soundscape [27]. Different aspects of a sound were represented with particular visual features. Dynamics was represented with different sizes where small is soft, and big is loud. To represent this spectrally, they used hues where blue is low, and red is high. Spatiality was represented as a 2D position of depth and panning. In some cases, sound is visualised in a more direct way. For example, an art installation for the 2013 GLOW festival featured a large, dark room with projectors on the ceiling [31]. When a member of the audience produced a sound such as a whistle, the sound waves would be projected onto the floor. Given these possibilities of communicating with light, we set out to investigate how light is experienced together with sound zones, and what information can be communicated to users about specific properties of sound zones.

3 Experimental Setup

We conducted two studies of the use of colour and brightness for sound zone feedback. The second study was planned in response to the findings from the first to further investigate light pattern feedback when two sound zones overlap. The two studies use a highly similar experimental setup, illustrated in Fig. 2. The soundbar was constructed from a 20-channel speaker array positioned in two rows, with a displacement of half a speaker. This was to ensure the shortest distance from centre to centre as that impacts the maximum frequency possible to use. The speaker is a 3\(''\) full-range unit from Tymphany, covering a range from 100 Hz to 8 kHz. The sound zones centres were positioned at a distance of 4.8 m from each other and 2.47 m from the soundbar. This enabled us to create a total separation between the sound zones greater than 15 dB. The setup mimics a situation where the participant shares a room, e.g., a living room or an office, with another person, and want to listen to different music tracks without disturbing each other, the type of situation termed “social-connected” by Lundgaard and Nielsen [22]. In this situation, users are engaged in different activities but still desire to be physically close to each other. Since listening to sound is a secondary activity for both users, we wanted to make the interaction with the sound zone familiar through the use of a physical volume controller.

Colour can be described according to the HSB (Hue, Saturation, Brightness) model where hue is the “attribute of a visual sensation according to which an area appears to be similar to one of the perceived colors”, brightness is when that area “appears to emit more or less light”, and saturation is the assessment of the area’s colourfulness “in proportion to its brightness” [10], which all impact the perception of physical properties differently. As a first step, we focused on hue, henceforth referred to as colour, and brightness, independently, where a larger amount of prior research can be relied upon. This includes the animated patterns defined by [14] and the findings on brightness for object perceptions by [20, 24]. In contrast to previous research that proposes light projections, wall displays, and shape-changing displays as potential approaches [17, 18, 23], we investigate if integrating the feedback directly as light onto the control device enable users to understand the spatial properties of sound zones. For this purpose, we designed two sound zone volume controllers with an LED strip that can display light in various brightness and colour settings. The motivation behind the specific design choices for our experiments was to enable the study of different patterns of colour and brightness in order to learn about their individual strengths and limitations. We chose to augment physical volume controllers as opposed to constructing a smartphone application because users’ smartphones are not always visually accessible, especially when engaged in activities such as reading or watching television. While volume controllers can also be occluded by, e.g., furniture and people, this will more rarely be the case in comparison to smartphones. We invite readers to watch the supplementary video, linked in Appendix 4, for further description of the envisioned setup. Each controller is constructed of two 3D printed cylinders that are connected through a 10K rotary potentiometer. Outside the bottom cylinder, we attached a NeoPixel strip that can display light through 11 LEDs. Since the study included light transitions, which can affect how specific colours are perceived, we used a gamma correction function as specified in the NeoPixel library. The potentiometer and LED-strip were connected to an Adafruit Feather HUZZAH ESP8266 board. To ensure a non-painful and safe experience for participants, the maximum loudness in the room was 75 dB.

Fig. 2.
figure 2

The setup for the studies consisted of one or two volume controllers, a laptop and a soundbar. The volume controllers communicated with the laptop through UDP packets which were processed and resulting in a final output from the soundbar.

Full size image

4 Study 1: Volume and Size of One Sound Zone

The purpose of Study 1 was to investigate the research question: How can colour and brightness be used as feedback about volume and size of one sound zone? We conducted a mixed methods evaluation, including a quantitative investigation into the perceptions of users on colour and brightness as a mechanism for feedback in sound zones, and interviews about participants’ understanding of what they saw and heard. The study was approved by the ethics board of the institution. Even though the size of a sound zone can be set independently of its volume, these properties can be experienced as related, because the sound from a louder zone can be heard in a wider area. With a focus on users’ experiences in this research, we view sound zone size and volume as dependent characteristics. We hypothesised that high brightness would be related to loud sounds and low brightness to quiet sounds. Furthermore, we expected that colours could be used for volume in a systematic way, meaning that participants would organise them according to wavelength (e.g. in the order of blue, green, yellow, and red).

4.1 Method

Participants. Due to the novelty of sound zone systems, we recruited novice users with no prior experience with nor knowledge of sound zone systems. Recruitment was done through social media. We recruited 27 participants (12 female and 15 male) within an age range of 23 to 48 (M = 33.2). We screened participants for hearing impairments, colour blindness (including Tritanopia), and synesthesia related to sound and vision. All participants signed an informed consent form and were paid $24.

Setup. The study took place in a lab with two active sound zones (see Fig. 2). Participants were asked to stand inside the left sound zone. Two different pop-rock music tracks (by The Killers and Muse) were played, one in each zone.

4.2 Procedure

Participants were introduced to the sound zone system by walking around the room for 1 to 2 min. We did not provide a verbal explanation of the system in order for participants to only rely on their auditory and visual impressions. The facilitator instructed them to walk into each sound zone and listen to the sound within both of them as well as the sound around the zones. Participants were then informed that the aim of the prototype was to display feedback about their own sound zone and the room surrounding it. They were then asked to adjust the volume of their own sound zone from minimum to maximum and, again, walk into both sound zones and listen to the sounds. This also served as a manipulation check to ensure that they noticed a difference in volume of their own sound zone. Each trial then consisted of two phases as shown in Fig. 3), with feedback displayed on the participant’s controller.

Fig. 3.
figure 3

Left: Colour and brightness settings for phase A. Right: Colour transitions for phase B. Each line shows which colour is displayed from minimum to maximum volume.

Full size image

Phase A: Colour and Brightness. First, four colour settings were displayed in a randomized order. Then, two brightness settings were displayed. We chose the blue colour for displaying brightness settings, because it is the brightest, perceptually [9]. For each colour and brightness setting, participants were asked to rate how loud they would expect their sound zone to be on a scale from 0 to 100 (0 being silence, and 100 being maximum volume). They were then given a map of the room in which the centers of the two sound zones were marked. The facilitator asked each participant to draw a circular area within which they would expect to hear their music track without being disturbed by the other track. After all variations had been displayed, participants were given the opportunity to correct responses and drawings. Phase A was conducted with no sound in order to remove the possibility that participants would provide answers based on what they heard rather than the light.

Phase B: Colour Transitions. We designed three colour transitions that were displayed when participants adjusted the volume of their sound zone. Based on the hypothesis that colours would be systematically organized according to wavelength, we assigned blue to low volume and red to high volume. Participants were instructed to adjust the volume of their sound zone from minimum to maximum and focus on the colour and sound. The colour transition was initiated when the volume of one sound zone was predicted to disturb another sound zone. This was based on the distraction model developed in previous research [29]. The facilitator then asked each participant to describe what each colour threshold signified to them. Additionally, the facilitator asked participants to describe how they believed the adjustment had affected the other sound zone. Participants were exposed to all three versions in randomised order.

4.3 Analysis

Participants’ assessments of sound zone size and volume in relation to colour were analyzed using a repeated measures ANOVA to compare responses across all colour settings (blue, green, yellow, and red). This method is appropriate for continuous numerical response data with one variable when the distribution of responses is approximately normally distributed. If the null-hypothesis (no statistical significance exists between the independent variable, i.e., colours) could be rejected with an alpha value below .05, we conducted a pairwise post hoc analysis with Bonferroni correction (to correct for multiple t-tests) to determine which pairs were significantly different. For responses regarding brightness for sound zone size and volume (low brightness and high brightness), we conducted t-tests to determine significantly different pairs (p < .05). For significantly different pairs, we report effect sizes calculated using the Cohen’s d formula. We annotated the interviews and conducted a thematic analysis [8] in order to investigate experiences of the light. We used an open-coding process, identifying and developing themes in two iterations. In the first iteration, 13 themes were identified. Through the second iteration, these themes were consolidated to five regarding (1) types of transitions, (2) illusions of sound behaviour, (3) conventions, (4) experiences of how the sound distributes in the space, and (5) synchronisation of sound zones.

4.4 Findings

Colour and Brightness for Volume and Size. We found that participants related low brightness to low volume and small size, and high brightness to high volume and large size. The mean volume responses were 12.7 for low brightness and 79.5 for high brightness. The difference between these responses was statistically significant (F\(_{1,52}\) = 207.81, p < .01), effect size 4.329 (indicating a large effect). 24 participants (89%) drew differently sized zones in relation to different brightness values. The difference between low and high brightness in relation to size was also statistically significant (F\(_{1,52}\) = 63.18, p < .01), effect size 2.175 (indicating a large effect). The mean sound zone size responses were 3.5 cm for low brightness and 8.6 cm for high brightness on a map that measured 16\(\,\times \,\)14.5 cm. For colour, we found a significant difference in participants’ volume responses (F\(_{1,104}\) = 6.20, p < .01) which was due solely to the difference between blue (mean = 37) and red (mean = 67), effect size 0.973 (indicating a large effect), as well as green (mean = 42) and red, effect size 0.876 (indicating a large effect), both p < .05. However, we found that 24 participants (89%) ranked colours in similar ways. 15 participants (56%) placed red at the high end of their volume responses, followed by yellow, green, and blue. 7 participants (26%) placed the colours in the reverse order.

Instant Versus Fading Transitions. 15 participants described that a colour fade better represented the behaviour of the sound as compared to instant transitions. Some expanded that instant transitions were experienced as volume levels whereas a fading transition supported what they heard and felt on the volume controller. In other words, the fading transition matched the visual sense to the auditory and haptic senses. This was also supported by some participants being surprised when the colour changed instantly. Furthermore, even though we used the same colours for all transitions, participants experienced a bigger volume change for the instant transition as compared to the fade. Some participants were confused that the brightness did not increase with the volume since they believe that would have enhanced the representation of the sound behaviour.

Sound Zone Illusions. In several cases, participants described non-existing sound behaviour. One participant described that the volume did not change noticeably within each colour range, but it changed noticeably when the colour changed. Some participants experienced that the sound responded differently between the instant colour transition and the fading colour transition. For example, one described that the volume level increased abruptly when the colour changed instantly, but it increased gradually when the colour faded. One participant explained that the fading colour transition was less noticeable and therefore, he was aware of the sound in a different way compared to the instant colour transition. Six participants believed that the zones switched places when the colour changed on the controller. For most of these cases, participants believed that the volume of the zones was constant, and they instead controlled the position of the sound zones with the controller. Some participants mentioned this in relation to the instant colour transition but not the fading transition.

Colour Conventions. As expected, participants relied on experiences with colours as feedback about sound properties, but not all relied on the same conventions. 12 participants mentioned that they perceived red as a warning, and some specifically experienced it as a signal about potential hearing damage. Relating the colour transitions to conventions in sound displays only applied to the instant colour transitions and not the fading transition. We found that even though some participants rated blue and green as high volume colours, several of these participants changed this view quickly to match our settings where red is related to high volume. In some cases, participants mentioned that they wanted to modify their responses according to this newly established system. The purple colour in the ‘colour threshold’ transition was only displayed within a small volume range. Some described that this allowed them to more easily adjust their own sound zone to a volume level equal to that of the other sound zone.

A Bubble Effect. Seven participants believed that adjusting the volume of one sound zone would not affect the other zone, as if the sound was contained inside a closed bubble. We did not anticipate this response, because of the initial introduction to the sound zones. During the introduction, only one participant reported that he could not hear both music tracks in one zone with the other on maximum volume. Some participants explained that they forgot about the other sound zone when they could not hear it. In one case, a participant described that the fading colour transition made it easier to forget the other zone. Seven participants believed that the sound zones were acoustically isolated from each other regardless of volume levels. Furthermore, our findings show that some participants relied on the displayed light for knowing whether or not sound zones could overlap. For example, one participant believed that ‘colour threshold’ transitions signified disturbing another sound zone or not, while the fading colour transition would lead to increased volume but no disturbance.

Synchronisation and Movement of Sound Zones. Eight participants believed that the volume levels of the two sound zones were synchronised. We found that participants described this synchronisation in different ways. Three participants believed that increasing volume in one zone would lead to decreased volume in the other zone. Another participant believed she was adjusting the volume of both zones simultaneously. Finally, four believed that the purple colour signified an equilibrium between zones where they were at equal volume levels.

5 Study 2: Overlaps Between Two Sound Zones

For the second study, we investigated the research question: How can animated light patterns be used as feedback for showing that sound zones overlap? To address this, we investigated how patterns compare with each other in terms of supporting users’ understanding of sound zone overlaps and for notification. This study was also approved by the ethics board of the institution.

5.1 Method

Participants. We followed the same procedure for recruiting participants as in Study 1. Since overlapping sound zones can only occur when multiple sound zones are active, and social aspects of the experience could affect the experience of certain patterns, we asked participants to bring a friend. No participants from the first study participated in the second. None reported hearing impairments, colour blindness (including Tritanopia), or synesthesia related to sound and vision, and none had prior experience with sound zone systems. We recruited 36 participants (22 female and 14 male) aged 19 to 28 (M = 22.8). Similar to Study 1, all signed an informed consent form and were paid US$24.

Setup. Study 2 used the same setup as Study 1 with two active sound zones (see Fig. 2). For each trial, we asked the participant and their friend to sit in their own sound zones. Both participant and friend had a controller to adjust the volume of their own sound zone. Participants were asked to respond to questions about the light patterns on a tablet placed in front of them. They took turns being an active and passive user. The active user turns up the volume of their sound zone, and the passive user just listens to the music.

Feedback Settings. We used either white on both controllers or red or blue for each controller. When increasing the volume of one sound zone to the point where the sound zones overlapped, an animated light pattern was displayed on the affected sound zone. See Fig. 4 for the patterns. In study 1, we learned that displaying feedback on participants’ own controllers made them less aware of, or completely forget, the other sound zone. It could even emphasise the focus on that person’s own sound zone. Based on that, we displayed the patterns on the passive participant’s controller. We hypothesised that this would serve our goal of showing how a volume increase by the active participant would affect the passive participant’s sound zone, i.e., the fact that the active participant’s sound zone overlaps with the passive participant’s sound zone.

We used six patterns in two categories, ‘Instant’ and ‘Gradual’. We included instant patterns, ‘beacon’, ‘bright flash’, and ‘blink increasing’, as Harrison et al. showed that their participants related these patterns to particular notification states [14] and effectively grabbed their attention. This would be especially useful for overlapping sound zones, since a user adjusting their volume cannot otherwise hear or see overlaps with other sound zones. Instant patterns might be useful for bringing users’ attention to these other sound zones and prompt them to adjust the sound again to avoid disturbing other users. To complement the study of these instant patterns, we included Harrison et al.’s gradual patterns [14], ‘staircase blink’, ‘gradual build’, and ‘lighthouse’, which we believed could give a different understanding of how sound zones overlap, based on the research by Löffler et al. [21]. Where instant patterns might notify users well, we aimed to investigate whether the gradual patterns give users a more realistic imitation of how the sound from one sound zone interferes with another sound zone. Our investigation relied on animations shown on the entire LED strips as opposed to parts of the LED strips. As such, this is a first step to identify whether or not light animations can be utilised for sound zone feedback. Utilising the spatial features of the LED strips is a next relevant step.

5.2 Procedure

Participants were first allowed to explore the room to create an initial experience of the sound zones. They then received a short introduction to the prototype. This included a description of the use situation for the prototype (i.e., “You are in a shared living room with a friend, and you are listening to different music tracks without wanting to disturb each other”). For half of the trial, participants took an active role by increasing the volume of their sound zone and focusing on the light on the other volume controller. For the other half, they took a passive role and watched their own controller while the friend increased the volume of their own sound zone. We counterbalanced whether they started the trial as active or passive. After up to 1 min of watching a pattern, participants were asked to rate it in a Likert-item format from ‘Strongly disagree’ to ‘Strongly agree’ according to how well it provided information in the form of six statements about the sound zones. We incorporated recommendations from previous research, including a mid-point option (‘Neither agree nor disagree’) [5], using a 7-point scale [11], and labelling all options on the scale [5]. Participants could provide additional comments. The six statements were phrased to investigate sound zone overlaps, volume, and size: “The sound zones are overlapping”, “My sound zone is increasing in size”, “The other sound zone is decreasing in size”, “The person in the other sound zone can hear my music”, “I am being warned”, and “I should adjust the volume of my sound zone”. The order of the patterns and statements were randomized for each pattern. At the end of the trial, we conducted a small follow-up interview asking participants to elaborate on their answers and describe how the light was experienced with the sound.

Fig. 4.
figure 4

Brightness and colour patterns for study 2 categorized as ‘Instant’ and ‘Gradual’. For brightness patterns, the height of the line signifies the intensity of the brightness (from minimum to maximum). For colour patterns, the height is related to the colour, and the default state of the volume controllers were blue (bottom) and red (top). Here, the passive user’s controller is blue, and the active user’s controller is red. (Color figure online)

Full size image

5.3 Analysis

Similar to Study 1, we performed a statistical analysis on the quantitative data followed by a reflective thematic analysis with open coding of the follow-up interviews. Methods for analysing Likert items are widely debated [5]. The debate is mainly centered on interpreting the responses as either ordinal or interval data. For this study, we followed an ordinal interpretation and therefore conducted a non-parametric analysis. We conducted a related-samples Friedman’s two-way analysis of variance by ranks (using an alpha level of 0.05 as cutoff for significance) because of its suitability for analyzing non-parametric ordinal data with repeated measures. To determine which pairs were significantly different, we conducted a pairwise Dunn test. Finally, we conducted a Kendall’s W test which indicates the effect size as a value between 0 and 1, representing the agreement between participants. We used Cohen’s effect size interpretation guidelines of 0.1 − < 0.3 (small effect), 0.3 − < 0.5 (moderate effect), and > \(=\) 0.5 (strong effect). The thematic analysis was conducted in two iterations by coding of transcriptions of the interviews. In the first iteration, six themes emerged concerning (1) colours mixing, (2) gradual transitions as overlaps, (3) associations for the colours and patterns, (4) ability to perceive light patterns according to the participant’s role, (5) ability to act on overlaps based on the participant’s role, and (6) feeling powerless when passive. In the second iteration, these themes were consolidated within the five themes presented together with the quantitative findings below.

5.4 Findings

An overview of the results of the analysis for each statement described above can be seen in Fig. 7 in Appendix 2. Boxplots showing the differences in ratings can be seen in Appendix 3. Our analysis indicates that the patterns perform statistically differently for five of the information statements but not the statement “The other sound zone is decreasing in size”. We found that the Beacon (BA) and Blink Increasing patterns (BC) perform best for all three categories. The follow-up interviews conducted at the end of each trial offer qualitative insights into why certain patterns performed better than others. These insights sometimes contradicted the quantitative findings. In the following, we group our findings according to categories of the information statements for which two fit into each category.

Overlapping Sound Zones. The two information statements related to overlaps were “The sound zones are overlapping” and “The person in the other sound zone can hear my music”. For information about overlapping sound zones, W \(=\) .124, indicating a small effect, we found that patterns with two colours performed better than patterns in white for all except when comparing the Bright Flash in white (AB) to the Gradual Build in two colours (BE) where the Bright Flash performed significantly better. For example, the Bright Flash pattern performed better in two colours (BB) than in white (AB) for the statement that the other person can hear the participant’s music, W \(=\) .140, indicating a small effect. We also found that instant patterns perform better than gradual patterns for both white and two colours.

Some participants experienced the patterns and colours as a metaphor for the sound zones. For some, this was the case for gradual patterns which were described as more realistic than instant patterns. One participant explained that the transition towards an overlap was fluid. Related to that, others mentioned the colours mixing as a supportive metaphor for the music tracks mixing for both instant and gradual light patterns. For one participant who put the instant patterns highest, the colours signified what caused the blinking pattern to occur: “I think the fact that I was red and that was blue added something. Then I could see that my colour was blinking on the blue one which indicated that mine was disturbing the blue one. It was also visible without colour, but I think it made more sense that my colour was blinking on it.” (P11). Similarly, another participant described that the instant patterns better supported her in seeing the overlap between zones. She explained that the high-frequency shifting between two colours indicated that both sounds were present in the same area of the room. A third described a similar experience: “I think that with the [gradual patterns], they were too slow which made you think: Is it happening now? Oh, I guess it is. I could better understand it with shifts of a higher pace” (P23). One participant expanded on his experience of assessing how much each music track could be heard in the other sound zone: “It was hard that thing about, is this loud enough that they can hear? Because I can’t hear their music. It was tricky to figure out how much my sound affected them” (P19).

Sound Zone Size. The two information statements related to sound zone size were “My sound zone is increasing in size” and “The other sound zone is decreasing in size”. For the first statement, W \(=\) .059, indicating a small effect, we found that the Blink Increasing pattern in colour (BC) performed significantly better than the Lighthouse pattern in white (AF) and colour (BF) as well as the Bright Flash in white (AB) and Gradual Build in colour (CE). This was the most significant difference. The findings for these patterns followed a similar direction as above where instant patterns perform better than gradual patterns. When asked to clarify their preference for Blink Increasing, participants underlined the fact that it featured blinks at an increasing frequency. No patterns performed significantly better than others for informing about the impacted sound zone decreasing in size. Participants related the feedback to the sound zone they were modifying instead of the affected sound zone. One participant noted: “I thought it was difficult regarding whether or not they got bigger or smaller. I don’t really think the light indicated that to me in any way” (P36).

Notification Level. For the statements “I am being warned” and “I should adjust the volume of my sound zone”, we expected that results would align with previous research [14], from which we defined the instant patterns. As expected, the notification level was rated significantly higher for patterns with colours compared to patterns only using brightness in white colour, W \(=\) .364, indicating a moderate effect. This was especially true for the Beacon (AA, BA) and Blink Increasing (AC, BC) patterns that blinked at a higher frequency. For a call to action on adjusting their volume, participants put the Beacon pattern in colour (BA) statistically higher than other patterns (except for the Bright Flash (BB) and Blink Increasing (BC)), W \(=\) .190, indicating small effect. The Beacon pattern in white (AA) performed statistically better than the Lighthouse patterns in white (AF) and colours (BF). This indicates a tendency that, even though colours are good for alerting participants, short blinks generally signal a call to action for participants more than fading patterns, regardless of colour. Participants commented that the gradual patterns, particularly the Gradual Build (AE, BE) and Lighthouse (AF, BF) patterns did not attract their attention or make a call to action as much as other patterns. One participant described that the patterns were easier to separate from the colours when passive. In contrast, when she was in the active role, she noticed the colours more. Several participants described the patterns as more intrusive when passive. While instant patterns in colour performed best, these were also described as most intrusive.

Passive or Active Role. We did not find any statistically significant differences in how participants responded to the light patterns when comparing responses from when they were passive to when they were active. However, the thematic analysis showed that some participants experienced the feedback differently because, in the passive role, they could hear the overlap, and in the active role, they could only see it. One participant expanded on this, saying that in the active role, he could see there was an overlap but he did not know the extent to which he was affecting the other sound zone. Sitting far away from the light pattern in the active role, some participants noted that it was more difficult to see the pattern. Some elaborated, explaining that the colours can help make the pattern more clear. In the passive role, however, the patterns were more easily distinguished from each other. Some related the meaning of a pattern to whether or not they were acting, and others related to what they heard (their own volume getting louder or hearing the other music track overtaking their sound zone).

Light Patterns and Music. Some participants mentioned that they looked for whether or not the light patterns matched the tempo or beat of the music. For some, the gradual patterns reportedly seemed like a music feature, e.g., following the beat, whereas the instant patterns more clearly did not align with the music. Because the instant patterns did not appear to align with the music, participants mentioned that they seemed less like a visualisation of the music and therefore related to sound zone properties. One participant experienced that the blinking patterns began at different volume levels. This aligns with other participants commenting that the instant patterns better indicated the precise volume level at which the overlap occurred. There was no clear indication of preference when asked to choose between patterns in colour and in white. However, when asked in the follow-up interviews, participants preferring white patterns explained that these were more aesthetically pleasing and less intrusive for a home situation.

6 Discussion

6.1 Brightness Mimics Sound Zone Size

The goal of the first study was to investigate light as feedback about the volume and size of one sound zone. We specifically focused on colour, brightness, and animated light patterns. Our findings extend previous research in the effect of light on spatial experience to show that those spatial experiences can also be used for supporting control of sound zones. Where previous research investigated the effect of light on perception of room and object sizes [20, 21, 24], we have used light as a visual overlay on an otherwise invisible experience. Our results show that brightness can be used as an indicator for sound zone volume and size. From the interviews, we deduce that this is due to participants’ experience of the light behaviour as something that mimics sound behaviour. Interpreting these results, we note that while the controllers have a fixed maximum brightness, this is not initially known by novice users. While our findings suggest that brightness is useful, this has two main implications for further investigations. First, we expect that there is a learning curve in becoming familiar with the feedback design which would be relevant to explore in further research. Second, the experience of brightness will change according to the physical environment. This will become especially relevant to investigate as sound zone systems move outside the lab setting and into homes, hospitals, and other domains. Customisation of the feedback to fit personal preferences and the characteristics of the physical environment is part of the next steps in this research.

6.2 Instant Light Patterns Mimic Overlapping Sound Zones

Study 2 investigated animated light patterns for feedback on overlapping sound zones. The findings revealed participants’ understanding of what constitutes an overlap between sound zones. The fact that the instant patterns rank highest for all three information categories can have several reasons. First, we confirm previous work on the effectiveness of instant patterns for notifications [14]. Second, we expand on this work by showing that these patterns can also be used in the specific context of sound zone systems for showing that sound zones overlap—especially for patterns in colour that, according to some participants, provide a metaphor for the two music tracks mixing. This interpretation could be dependent on the situation as pointed out in previous research [1]. We acknowledge that the statements on increasing and decreasing sound zones might have skewed participants into a particular understanding of the sound zone behaviour. The fact that no significant differences were found for the statement “The other sound zone is decreasing in size” but were found for all other statements indicates that no animations visualised a decrease in the size of the other sound zone. Future research could include a broader range of statements to support or contend this.

The findings for notification levels are more general and not only applicable to sound zone systems. This means that they do not necessarily carry particular information about sound zones, such as whether or not the zones are overlapping. Further research could investigate whether or not novice users expect one sound zone to decrease in size when affected by another sound zone. Since participants felt affected by a pattern when in a passive role, further research could also investigate the case where feedback is only displayed on the side of the controller that is visible to the active user as feedback can otherwise be confusing and intrusive. This would also be relevant to explore outside lab setups to allow for further insights into how the light is experienced in situations in homes where it should not intrude on users’ primary activities, as stated by Hazlewood et al. [15]. Our findings point to two distinct interaction paradigms for sound zone interaction, either emphasising the behaviour of sound through gradual patterns or the more abstract notion of sound zone bubbles through instant patterns. Based on the follow-up interviews, our findings indicate that choosing either interaction paradigm leads to different subjective conceptualisations of how the sound zone affects the rest of the physical space. This will also affect the utilisation of the spatial features of the LED strips. To emphasise sound behaviour, a design could include sound direction. On the other hand, notification patterns could be useful to provide differential information to active and passive users.

6.3 Metaphors Confused with Symbolism

We included colours (blue, green, yellow, and red) in Study 1 to explore whether these colours could represent characteristics of sound zone volume and size through a similar affordance as brightness. However, while brightness and animated light patterns have shown to be useful, our findings confirm that colours in relation to sound have different symbolic meanings to users. This suggests that colour cannot be used in the same way as brightness, where users rely on tacit knowledge about object properties [21]. This does not necessarily mean that colour cannot be used in displays for sound zone feedback, but they do not reliably carry information about spatial properties themselves. Instead, colour can be used in animated light patterns that mimic sound behaviour, such as transitions between colours. A possible future avenue is to provide a colour “identity” for each sound zone which can be, e.g., chosen by the user. This will be especially useful in situations with more than two users. For Study 1, we found that while participants typically ordered the four colours in a sequential way (red, yellow, green, and blue – or mirrored), it also led to confusion among some participants. Because the aim is to show volume intervals, a future research avenue could be to explore other colour mappings such as sequential color mappings with single colours [25] or with inspiration from the use of nodes by Bastian and Heymann where colours reflect proximity to other nodes [2]. This could also extend Löffler et al.’s prior research [21] by investigating the effect of the saturation parameter on perception of sound zone properties. Drawing inspiration from previous research on the effect of light on perception of spatial properties of space and objects [20, 21] for Study 2, we expected that gradual light animation patterns would provide participants with realistic information about sound zones overlapping. However, while many participants understood and appreciated the metaphor offered by these patterns, our analysis indicates that they do not allow participants to easily adjust the volume of their sound zone with reassurance that they are not disturbing someone in a different sound zone. A key takeaway, therefore, is that light patterns for sound zones should have distinguishable contrasts to support understanding of overlaps.

7 Conclusion

Sound zones are invisible and thereby challenging to understand. We tackle this design challenge by using brightness and colour for feedback about sound zone volume, size, and overlaps based on prior research showing a relation between variations of light and experience of object properties. We confirm that brightness can be used to inform users about both sound zone size and volume. We also show that instant patterns in colour are useful for overlaps. Future research could investigate other light settings and designs, e.g., by including saturation as a parameter for adjusting the light. Another opportunity is to utilise the spatial features of the volume controller to, e.g., show the direction from which the sound is coming. Other avenues include using light on other interfaces and varying what light is visible to each user in a multi-user setup. Second, future research could investigate different situations where light is used as feedback for sound zones to clarify the effect the situation has on the user experience.