Keywords

1 Introduction

Multitouch interfaces are a key element of the interaction design in Resonanzen, a collaborative cultural heritage research project aiming to build an interactive archive and exhibition of French post-war architecture in the Franco-German border region [12]. Its title (“Resonances: The Long Waves of Utopia”) is inspired by one of the signature buildings explored in the course of the research process, the now-defunct long-wave radio station of the French private broadcasting service Europe 1 built in the 1950s whose coverage reached across Western Europe, the Mediterranean, and Northern Africa [13]. This focus on broadcasting technology acknowledges the central role of communication architectures in the exploration of cultural heritage and frames an interaction design process that aims to make this cultural heritage tangible. Interactive experiences have been deployed and explored under various lenses, particularly in cultural heritage settings [9]. Multi-user tabletop installations in particular have been examined to identify how interactivity can enhance information visualization [4, 8], visitor engagement [5, 6] and what impact the design of the user interface on aesthetic and informative qualities may have [1,2,3, 7]. It has been shown that such installations have the potential to be inspiring and evocative, fostering collaboration and social engagement [2, 3]. But it is essential to design the interface carefully, to let it disperse and allow meaningful interaction with the actual contents [1, 3]. Our approach to use a multitouch interface to involve users in the curatorial process of categorising and exploring archived data is an effort to connect elements of interactive documentaries, the digitalisation of cultural data (as well as ways to display and use them in new and innovative ways) and user-driven design methods. In our first testing period, we encourage users to immerse themselves in a vast, deeply linked and comprehensively tagged pool of items on post-war architecture and their creators, effectively letting them curate their experience and own narrative of their journey through a multifaceted topic in a time and location charged with social, economic and political tension. The qualitative feedback well receive will be used to strengthen and reiterate both the interface and the way users can create their own narratives and share them with other visitors.

2 Technology

2.1 User Interface Design

The User Interface follows a minimalist, content-focused approach. A large map builds the foundation of our experience, with new windows created upon most interactions. The intuitive touch-actions (moving, scaling, rotating etc.) prevent this from becoming overwhelming, while encouraging users to share windows and newly found data with each other (Fig. 1).

Fig. 1.
figure 1

A mockup for the UI design. A history stack on the left, broad map with detailed maps and object information on top of it. Single elements are capable of common touch-interactions (moving, rotating, zooming etc.)

Full size image

2.2 User Experience Flow

A big, custom-drawn map is the entry point of our interface. It’s a geographically non-accurate map, featuring markers of the architecture and objects of the exhibition which are located all across the Franco-German border region. By touching an area and/or object of interest, a new window opens for the user, depicting a geographically accurate map of the touched point and its surrounding area. Here again are markers for the architecture. Touching one of the markers opens another window on top of the map, and this window shows information about the selected building/architectural object. We display a big, non-uniform variety of data for each building, depending on the amount of material available for the object in question. The material is primarily images, but also a text-based description with accompanying metadata like year of building, the architect and other available media. Other media includes 3D-models, 360 panoramic images and videos. They can be looked at and interacted with depending on their format, meaning users can look around in 360 degree material, rotate and zoom in on 3D-files and watch videos. Each of those special media types open in their own respective window, ready to be shared with other users. By touching tags and architects, the user can navigate between similar buildings and other works of the same person, creating their own individual route through the cultural objects, learning about their historic context and the persons behind them. The ability of the multitouch table allows a multitude of users to interact with our interface simultaneously and encourages intuitive sharing of their findings, telling each other about the things they learned and hopefully discussing their different takes on the different aspects of the buildings.

2.3 Technical Implementation

Media We Need to Consider. Before we could decide on a structural approach for our implementation, we had to define all the different types of media our interface should be able to handle. Our main medium are images, but we also have to support text, tags, geo coordinates, videos, dates, 3d files and models, CAD files, ground plans and 360 panoramas and videos. The tagging and categorisation is a big part and important part for this, because our approach, to let the user decide on what tags and information they continue their journey through the post-war architecture, relied on having a lot of accurate, precise tags for each building, ranging from the architect and type of building (eg. church, school, government buildings) to their respective locations, cost and enthusiasm among the general population they received.

The Medium. Our interfaces leverages the possibilities of a 65 inch multitouch table. It supports up to 50 concurrent touches and enables users to use intuitive gestures and mechanics to freely move through the displayed content. The big size allows multiple users to operate the interface without getting in each others ways.

Backend. The backend we developed allows for easy uploading and managing of objects, their media and relationship with architects and other objects. It is built with Spring BootFootnote 1, an opinionated take on the Spring frameworkFootnote 2. Spring Boot enables us to rapidly iterate the backend as new needs develop, allowing us to stay agile with new types of media and their requirements.

We also use different services like CloudinaryFootnote 3, automatically optimizing the images we receive from our partners, increasing the performance and efficiency of our interface and application as a whole.

Frontend. Our frontend is browser-based and uses non-proprietary, open source software wherever possible. Our browser-based approach allows us to run it on any pc with a web-browser (or even as a cross-platform, standalone application if combined with a wrapper like ElectronFootnote 4. It is built on the popular React-library, an open-source, high-performance JavaScript library with a small footprint. We also use a healthy range of plugins to support our different types of media, such as:

Leaflet.Footnote 5 This open-source JavaScript library runs the foundation of our interface: The different maps we display. Leaflet allows us to create maps with custom coordinate systems, images and markers. With its highly customisable set of features and open-source nature, Leaflet is a perfect fit for our project.

Panellum.Footnote 6 “A Lightweight Panorama Viewer for the Web”. Another open-source library. This library enables us to display 360 material like images, videos and even virtual tours of our objects. Users can touch to zoom, pan and rotate images with native, intuitive controls.

three.js.Footnote 7 For our 3D models and CAD files, we use three.js, an open-source library for abstracting WebGL, shaders and other 3D specific nuts and bolts into JavaScript. It allows us to serve the 3D and CAD files with great performance and controls, enabling the user to rotate, zoom and switch between models and plans on the fly.

Putting it all Together. In the end, our frontend needs to communicate with our backend. In our pilot project, this happens by running a local webserver (the backend) on our PC. This is the same PC that runs the multitouch interface, so the data from backend is supplied from the same PC running the frontend, which means virtually no latency and a very high performance of both loading and displaying interface and requested data. However, our web-based approach allows us to outsource the backend to a server/infrastructure of our choosing, which would then enable us to run the same (or an expanded and/or more intricately linked) interface from multiple machines.

2.4 Iterative Approach Based on Qualitative Feedback

The main focus of our research is the development of an cross-generational approach to co-authored storytelling that is workable in a “live” exhibition. Building on previous archive-based exhibition projects [14, 16], our goal is to generate visual stories to deepen engagement with cultural heritage items whose historical significance is much better understood through user-driven contextualization.

The involvement of users in a a co-creative interface design process usually ends prior to its deployment in an exhibition process. In our case, we aim to involve users in an iterative design process throughout the exhibition since such a real-life experience offers a much more comprehensive co-creation setting than an isolated workshop. However, this requires adjustments to conventional user testing methods. We will begin by designing a workshop series that combines guided exhibition visits with on-site documentation by users of their broader museum experience to better understand how use of the archive station is framed (and affected by) the awareness of other exhibition modules and the overall interaction design of the exhibition space.

3 Perspectives

3.1 End of Project

While the exhibition ends in 2018, its archival components are mobile and can easily be integrated into other exhibition contexts. We will explore the possibility of involving users in other sites (and countries) in a follow-up process that allows us to iterate our co-creation approach to interface design.

3.2 Vision

While our work on the project in context with the exhibition nears completion, we already have both tangible and more distant goals and possibilities in mind. For one goal, we want to strengthen the storytelling aspect, the user as co-author and co-creator, even further.

User-Driven Memory Mapping. One approach is a form of “memory mapping”. We track and save the way users engage with the cultural heritage data, linking subjective individual memories with the“objective” meaning of said data, effectively translating memory maps into contextual metadata. A key concern for this would be a simple yet effective component for our multitouch interface, enabling simple export of data. Connecting this with an already established software and/or API (eg. digiCultFootnote 8 would omit the need of backend functionality that rivals or duplicates already existing features of collections management software.

Expansion by Contextualisation. The way our users interact and share their “findings” and contextualise them lends itself very well to crowd-source otherwise invisible connections between different cultural heritage data. This would allow cultural heritage collection platforms like europeanaFootnote 9 to increase their impact, not by (as usually assumed and realized) the amount of data available but by using the individual contextualisation and connections users can supply, effectively turning users into active co-creators of cultural heritage archive infrastructures. This could strengthen the impact of such platforms by a large margin, because as seen in our first tests in previous exhibitions users are far more likely to engage with cultural heritage if they have a chance to build their own narrative and explore archival content on their own terms instead of being exposed to it by narration or reading alone.

The Future of Our Interface. The technical implementation of our interface allows for very flexible outputs. This means we are by no means limited to the multitouch interface, which would rather serve as (one) interaction point for our users. Content could then be mapped to different outputs, such as displays, audio speakers or even transmedia installations. We also have the option to develop a plugin for exporting collected user data into a machine readable format, eg. for feeding it back into other collection management platforms like digiCULT [19] or Europeana. The flexible output could also be used to further enable users to share their narrative and take on the available data with other visitors, saving it for review or displaying it prominently in the exhibition. This would build on previous iterations where we created a multi-touch interface to show user-generated playlists in a museum [15]. We also have the possibility to build on the “user as co-curator” perspective by letting users save content interesting to them and then generating a summary of their findings and experience, creating an individually tailored exhibition experience. Every user would therefore experience the exhibition based on their own interests and receives a responsive narrative, their own “passage through the archive”. This would encourage aesthetic education, an important aspect of working in the field of cultural impact.

Our Vision for Cultural Heritage. Our goal is not to “school” people with the past, but providing them with a window into history and our rich cultural heritage. We believe it is important to accessibly archive cultural heritage and knowledge, because we do not know what will be of interest and importance to people looking back onto culture in ten, a hundred or even a thousand years, both as individuals and society as a whole. Our focus on greater user involvement on curatorial processes reflects a broader trend across the cultural heritage field [18]. We also strongly support open data approaches in cultural heritage and hope to share archival content with as few restrictions for reuse as possible [17]. Our multitouch concept offers an open, expandable approach how to create an interface for a huge, diverse amount of data for users who can use this interface to find and facilitate information from this set of data relevant to them, enabling them to create their own, individual connections and approaches to a cultural heritage we share as society.