Abstract
This paper discusses interaction design for interactive sonification and visualisation of data in multi-touch contexts. Interaction design for data analysis is becoming increasingly important as data becomes more openly available. We discuss how navigation issues such as zooming, selection, arrangement and playback of data relate to both the auditory and visual modality in different ways, and how they may be linked through the modality of touch and gestural interaction. For this purpose we introduce a user interface for exploring and interacting with representations of time-series data simultaneously in both the visual and auditory modalities.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Adcock M, Barrass S (2004) Cultivating design patterns for auditory displays. In: Proceedings of the 10th international conference on auditory display, Sydney
Burkhard R et al. (2007) Visualisation summit 2007: Ten research goals for 2010. Inform Visual 6(3):169–188
Aldersey-Williams H, Hall P, Sargent T, Antonelli P (2008) Design and the elastic mind. The Museum of Modern Art, New York
Anderson E (1935) The irises of the gaspe peninsula. Bull Am Iris Soc 59:2–5
Barrass S (2003) Sonification design patterns. In: Proceedings of the 9th international conference on auditory display, Boston
Barrass S, Schaffert N, Barrass T (2010) Probing preferences between six designs of interactive sonifications for recreational sports, health and fitness. In: Proceedings of ISon 2010, 3rd interactive sonification workshop, Stockholm, Sweden
Beilharz K, Ferguson S (2009) An interface design for aesthetic and interactive sonification. In: Proceedings of the 15th international conference on auditory display, Copenhagen, Denmark
Bertelsen O, Pold S (2004) Criticism as an approach to interface aesthetics. In: Nordic conference on human-computer interaction, Tampere, Finland
Bovermann T, Groten J, de Campo A, Eckel G (2007) Juggling sounds. In: International workshop on interactive sonification, York, UK
Bovermann T, Hermann T, Ritter H (2006) Tangible data scanning sonification model. In: Proceedings of the 12th international conference on auditory display, London, UK
Chen C (2005) Top 10 unsolved information visualisation problems. IEEE Comput Graph Appl 25(4):12–16
Collins A, Apted T, Kay J (2007) Tabletop file system access: associative and hierarchical approaches. In: IEEE international workshop on horizontal interactive human-computer systems (TABLETOP’07), pp 113–120
Collins A, Kay J (2008) Collaborative personal information management with shared, interactive tabletops. In: CHI 2008, Florence, Italy
Cowie A (2010) Collaborative concept mapping at an interactive tabletop. PhD thesis
Ferguson S (2009) Exploratory sound analysis: statistical sonifications for the investigation of sound. PhD thesis
Flowers JH, Buhman DC, Turnage KD (1997) Cross-modal equivalence of visual and auditory scatterplots for exploring bivariate data samples. Hum Factors 39(3):341–351
Fry B (2004) Computational information design. PhD thesis
Fry B (2008) Nature hapmap cover. Nature 437(7063):1 (cover)
Geiselhart K (2010) Towards global governance—interactive technologies and global accountability measures. Public Commun Rev 1(1):37–46
Hermann T (2002) Sonification for exploratory data analysis. PhD thesis
Hermann T (2008) Taxonomy and definitions for sonification and auditory display. In: Proceedings of the 2008 international conference on auditory display, Paris, France
Hermann T, Bovermann T, Riedenklau E, Ritter H (2007) Tangible computing for interactive sonification of multivariate data. In: International workshop on interactive sonification, York, UK
Hermann T, Niehus C, Ritter H (2003) Interactive visualisation and sonification for monitoring complex processes. In: Proceedings of the 9th international conference on auditory display, Boston, USA
Hochenbaum J, Vallis O (2009) Bricktable: a musical tangible multi-touch interface. In: Proceedings of Berlin open conference 09, Berlin, Germany
Hochenbaum J, Vallis O, Diakopoulos D, Murphy J, Kapur A (2010) Designing expressive musical interfaces for tabletop surfaces. In: Proceedings of the 2010 international conference on new interfaces for musical expression, Sydney, Australia
Hunt A, Hermann T (2004) The importance of interaction in sonification. In: Proceedings of the 10th international conference on auditory display, Sydney, Australia
Judelman G (2004) Aesthetics and inspiration for visualisation design: Bridging the gap between art and science. In: International conference on information visualisation (IV’04), London, England
Kosara R (2007) Visualisation criticism—the missing link between information visualisation and art. In: International conference on information visualisation (IV’07), Zurich, Switzerland
Kramer G (1993) Auditory display: sonification, audification, and auditory interfaces, Perseus, New York
Kramer G, Walker BN, Bonebright T, Cook PR, Flowers JH, Miner N, Neuhoff JG (1997) Sonification report: status of the field and research agenda. Tech rep, National Science Foundation
Mayo E, Steinberg T (2010) The power of information: an independent review
Nees M, Walker B (2006) Relative intensity of auditory context for auditory graph design. In: 12th international conference on auditory display, London, UK, pp 95–98
Pousman Z, Stasko J, Mateas M (2007) Casual information visualisation: depictions of data in everyday life. IEEE Trans Vis Comput Graph 13(6):1145–1152
Sack W (2006) Aesthetics of information visualisation. In: Paul C, Vesna V, Lovejoy M (eds) Context providers. MIT Press, Boston
Scha R, Bod R (1993) Computational esthetics. Inf Infbeleid 11(1):54–63
Shneiderman B (1996) The eyes have it: a task by data type taxonomy for information visualisations. In: IEEE symposium on visual languages, Boulder, CO, USA
Song H, Beilharz K (2007) Concurrent auditory stream discrimination in auditory graphing. J Comput 3(1):79–87
Tufte ER (1992) Envisioning information. Graphics Press, Cheshire
Tünnermann R, Hermann T (2009) Multi-touch interactions for model-based sonification. In: Proceedings of the 15th international conference on auditory display, Copenhagen, Denmark
Moere Vande A (2005) Form follows data: the symbiosis between design and information visualisation. In: International conference on computer-aided architectural design (CAADfutures’05). OKK Verlag, Vienna
Vesna V (2007) Seeing the world in a grain of sand: the database aesthetics of everything. In: Vesna V (ed) Database aesthetics—art in the age of information overflow, electronic mediations. University of Minnesota Press, Minneapolis
Viegas F, Wattenberg M (2007) Artistic data visualisation: beyond visual analytic. In: Online communities and social computing. Lecture notes in computer science, vol 4564(15). Springer, Berlin, pp 182–191
Walker BN, Nees MA (2005) An agenda for research and development of multimodal graphs. In: Proceedings of the 11th international conference on auditory display, Limerick, Ireland
Wilkinson L (2005) The grammar of graphics, 2nd edn. Springer, Berlin
Zacks J, Levy E, Tverksy B, Schiano D (1998) Reading bar graphs: effects of extraneous depth cues and graphical context. J Exp Psychol, Appl 4(2):119–138
Zacks J, Levy E, Tversky B, Schiano D (2001) Graphs in print. In: Olivier P, Anderson M, Meyer B (eds) Diagrammatic representation and reasoning. Springer, London
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ferguson, S., Beilharz, K. & Calò, C.A. Navigation of interactive sonifications and visualisations of time-series data using multi-touch computing. J Multimodal User Interfaces 5, 97–109 (2012). https://doi.org/10.1007/s12193-011-0075-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12193-011-0075-3