1 Introduction

Museums are dense with objects and rich in information concerning these objects, and therefore can be overwhelming to visitors, as they offer much more than the visitor can absorb in the limited time of a visit (Falk 2009). Moreover, the Cultural Heritage (CH) experience at museums is not limited to the time spent in front of the exhibits. It begins before the actual on-site visit and continues with memories and reflections after the visit (Falk 2009; Walhimer 2011). CH Tourism can benefit substantially from information and communication technology (ICT) developments targeted at it. Current technological developments such as widely available smart phones, Google glass (and similar devices), car-based computerized systems and the concept of internet of things (IoT) bring us closer to living in ubiquitous computing environments where computerized services are offered continuously to users (Weiser 1991). A primary challenge, therefore, is to understand how to use novel ICT to enhance the visit experience by extending its boundaries to include: (1) a phase prior to the visit, used for planning, (2) a phase during the visit, helping visitors access the exhibits of most interest to them and relevant information associated with those exhibits, and (3) a follow-up phase, with post visit memories and reflections (as already showed by Marty (2007) who studied the use of museum websites before and after the visit and found out that online visitors frequently use museum websites to complement their visits to physical museums). Furthermore, all this needs to be done while taking into account that visitors have their own personal interests and that the visit to a CH site is primarily a leisure, social activity (Falk 2009). Making this more complex is the fact that concurrently there are a growing number of different mobile applications aimed at supporting visitors, mostly on-site.

How can technology support the requirements mentioned above coherently and continuously? It may become a virtual companion by providing an artificial agent that realizes most of the characteristics of the classical human museum visitor’s guide that serves as a mentor, pathfinder, social mediator, culture broker, and leader to the museum visitors (Cohen 1985). Moreover, technology should enable visitors to plan their visit and get familiar with the museum before the visit starts. On-site, it needs to support navigation so visitors can easily find their way in the site, provide them with personalized information, and, like a human guide, eventually support not only individuals but also groups, as visitors often come to CH sites as groups (Bitgood and Shettel 1996; McManus 1991). After the visit, it should enable them to reflect on their experience, and if they were with others, possibly allow discussion and consolidation of what they have learned, so that the experience can be a basis for further CH exploration (Falk 2009).

According to Falk (2009) visitors have their own identity when they come to the museum that may change within and between visits. Technology can adapt to these identity changes and accommodate them. The area of User Modeling offers tools for monitoring and maintaining a model of the visitor, enabling such adaptation over time. Yet what is needed, as a further step, is a broader view: one that integrates the visit with its bordering phases, immediately before and after and possibly subsequent visits as well. Ultimately what visitors wish from the CH experience is that it will be pleasant, memorable, conducive to learning, and that pragmatically, it will lead to further interest, elaboration and additional experiences (Marty 2007; Falk 2009; Sheng and Chen 2012).

Even though these ideas are not new, so far, in spite of the vast amount of research about applying novel ICT in museums, as shown in recent surveys (Kenteris et al. 2011; Ardissono et al. 2012), there were only few attempts to suggest a generic technological solution that goes beyond the single, onsite visit. The challenge is exacerbated by todays’ mobile technologies and variety of CH-related mobile applications, each of them aimed at providing specific on-site support to its users. The main goal of most of the research conducted so far in applying novel ICT in CH was to explore the potential of state of the art technology for enhancing the on-site experience of visitors. Moreover, it focused mainly on specific aspects, including information delivery, context awareness, navigation, personalization and interaction modalities. A few salient examples that tried to address pre-visit planning are CHIP (Wang et al. 2008); LoL@ (Umlauft et al. 2003) and PIL (Belinky et al. 2012). As for post visit summaries, among the few prototypes, it is worth mentioning CyberGuide (Abowd et al. 1997); Lol@ (Umlauft et al. 2003); PEACH (Stock et al. 2007); and PIL (Kuflik et al. 2011). However, none of the above-mentioned prototypes suggested an integrative framework that can be applied for integrating the three visit phases in general, beyond the specific setting it was demonstrated in.

Following the above, our main research challenge was to suggest and demonstrate a generic framework that could be used for developing future museum (and other CH sites) visitors’ guides, linking the pre, during and post visit phases, extending the boundaries of the museum visit, as has been advocated by museum researchers (Falk 2009; Walhimer 2011). Applying our framework, a visitor will be able to (1) plan a museum visit by looking at the museum website before the visit (mobile or desktop), (2) visit the cultural site following the plan and then (3) reflect on the visit, either while still there or at a later time, keeping a visit summary which may initiate further exploration and be used for planning additional visits. We propose a holistic framework, integrating the three visit phases, based on experience gained during the development and experimentation of our own prototypes as well as on lessons learned from others’ prior works. The proposed approach is important when a visitor returns to the same museum to further explore it, or when past experiences are to be reused during the current visit. In these cases the pre-visit phase would not start from scratch, and could exploit a pre-existing user model, adapted to the new context. Considering past experiences has become even more important with the advent of smartphones and Apps that provide stand alone, disconnected opportunities for supporting onsite CH experience. We validate our framework by demonstrating it using the same common technology for each phase and show its feasibility in a realistic setting of a deployed museum visitors’ guide system.

We believe that the combination of mobile, context aware technology and cloud computing opens new opportunities for a large variety of CH related applications for each and every site and at the same time opens new opportunities for integration of pieces of information gathered from different sources for creating a coherent, ongoing visitor experience. Todays’ technologies provide the needed tools and the needed infrastructure while our suggested framework enables to bridge the gaps between the visit phases and beyond.

The rest of the paper is structured as follows: we survey related work, after which we present a framework and an architectural frameworkFootnote 1 for an end-to-end solution of the pre-, during and post- visit. Next we provide a concrete example of how the framework can be implemented, followed by a description of a case study applying the framework, with examples of several applications built on top of this infrastructure that use this framework. We then discuss implications, conclusions and possible future research directions.

2 Related work

Over the years, it has been notable how novel ICT has made the CH domain and the museum context a favored sector for exploring new frontiers, and thereby providing new ways to enhance and personalize the museum visit experience. Already in the early 1990s systems like Alfresco (Stock et al. 1993) and ILEX (Oberlander et al. 1998) introduced advanced techniques for exploring CH. Mobile technology was quickly adopted and experimented in the museum, as surveyed by Tallon and Walker (2008) as well as in tourism in general (Kenteris et al. 2011; Tallon 2012). In recent years, with the advent of smartphones, a large variety of CH related Apps appeared, aiming mainly at supporting the visitor onsite by applying novel, state of the art ICT including location awareness, augmented and virtual reality and more. A new interesting development is tangible interfaces, which allow for the embedding of ICT in non-traditionally IT-related objects (Petrelli et al. 2013). These trends continue to grow as the concept of Internet of Things (IoT) is now becoming a feasible reality (Atzori et al. 2010).

2.1 Supporting “Pre” and “Post” visit scenarios

Marty (2007) already studied the use of museum websites before and after the visit and found out that online visitors frequently use museum websites to complement their visits to physical museums. However, only few applications in the past looked beyond supporting the visitor on-site. Cyberguide (Abowd et al. 1997) is an example of a research prototype that briefly pointed out the need for “post visit” support, to let visitors see where they visited, but did not implement such support. Stock (2001) introduced intelligent natural language-based interfaces as an advanced research theme for the three phases of the visit. Lol@ (Umlauft et al. 2003), was one of a few applications that tried to extend the visit experience beyond the on-site visit. The prototype system supported three usage scenarios (for a tour to a city):

  1. 1.

    “Hotel room scenario” that enabled the user using a smartphone to look at various points of interest in the city, to get some information about them, and plan a tour (but no personal tour is saved for future use).

  2. 2.

    “Walking through the city scenario” where the visitor, using the smartphone explores the city following a pre-defined tour. The system can help the visitors find their way to points of interest, if positioning is turned on. The visitor can also record the experience in a visit diary.

  3. 3.

    “Accessing information from the desktop” where the visitor is able to access the visit diary (list of visited points of interest) after the visit, using the smartphone or downloading it to a PC.

The authors also recognize the need for personalization and refer to this option as future work. All in all, their system successfully demonstrated the feasibility of using mobile phones of that time as mobile tour guides, with the “pre” and “post” activities demonstrating the potential to go beyond the on-site visit. However, this was a proof of concept prototype, where the ability to use a smart phone for these three phases and also enable the use of a desktop for the post-visit scenario was demonstrated. No general/standard architecture was suggested, primarily since the technology of that time did not offer appropriate solutions.

Semper and Spasojevic (2002) describe the Guidebook, a pioneering mobile guide at the Exploratorium in San Francisco. In their description they refer to the pre-visit, on-site and post-visit phases and claim that novel ICT can support the extension of the visit to these phases, enriching the overall experience. However, the guidebook was a proof-of-concept prototype, intended mainly to demonstrate the potential of mobile technology for enhancing the museum visit experience onsite while the before and after phases were not demonstrated or explored.

Besides LoL@, there were a few additional systems that addressed the “pre” and/or “post” visit scenario. PEACH (Stock et al. 2007) enabled the visitor to choose a virtual character to accompany the visitor during the visit to an historic site with frescoed walls. There were two possible characters—a painter and a lady. Selection of a character implicitly defined the visitor’s interests—artistic or social, and bootstrapped a user model that was then updated and used throughout the visit to personalize information delivery. While PEACH did not provide a planning option per se, this option can still be considered as a “pre visit” activity that resembles what is suggested by Falk (2009), since the visitor, before starting the visit, has to reason about how information will be presented. Regarding “post visit” activity, PEACH demonstrated the ability to generate a personalized visit summary, describing what the visitor have seen and seemed to have been interested in, and suggest future activities that match the visitor’s interest. Moreover, there was a possibility to export the user model that was constructed during the visit for future use.

Following in the footsteps of PEACH, the PIL project (Kuflik et al. 2011), also demonstrated the ability to integrate the pre, during and post visit phases, by providing a simulated tour planner that created a user model that later on was used to bootstrap a personal user model for the on-site visit. At the end of the visit, a visit log enabled the visitors to re-visit the visit together. The PIL experience will be detailed later, as it is the basis for our framework.

There are quite a few systems that enable their users to plan a tour, but only a few carry this plan to the site itself, as suggested by CHIP (Wang et al. 2008). Using the CHIP demonstrator, the user is able to visit the Rijksmuseum online from the desktop and while interacting with the system build a user profile representing the user’s interests and then construct a personalized tour that can later be used on-site by the visitor using a mobile client.

Regarding tourism in general, quite a few systems enable their users to plan (and keep) a tour plan, as does Trip@advice (Ricci and Werthner 2001) or ETP (Dunstall et al. 2003), as well as many more commercial systems.

The above-mentioned systems tried to extend the on-site visit experience to the “pre-visit” or “post-visit” and demonstrated (or noted) the potential of enriching the visit experience beyond the on-site visit. While all were proof-of-concept demonstrators that used current state of the art technology, none of them led directly to the development of a system that was deployed in practice, mainly due to the limitations of the technology at that time. Furthermore, none presented an infrastructure that was designed to support an end-to-end solution for the “pre-visit” “during” or “post-visit” scenarios. Current technological evolution helps in bridging this gap, as demonstrated in CHIP where semantic web technologies provide the basic mechanisms for domain knowledge representation and usage in the different scenarios. Still, the core of the CHIP system is the semantically rich domain knowledge that needs to be constructed manually and ad-hoc developed applications that may use user models created online for an outdoor visit to a different site.

2.2 CH support in the era of ubiquitous/pervasive computing and the cloud

Smartphones and Apps are continuously growing in numbers (Apple recently published that their App store contains more than 1,200,000 Apps (Costello 2013), downloaded at a rate of about 48.6 million Apps per day (Dediu 2012). With this mobile technology, accompanied by cloud computing and IoT, we are approaching the era of ubiquitous computing, where the computers disappear and services are seamlessly available to the users, as Weiser (1991) predicted “they weave themselves into the fabric of everyday life until they are indistinguishable from it”. If we consider these recent technological developments we already see a wide range of CH related applications developed while taking advantage of these new technologies. Boiano et al. (2012) recently suggested guidelines for developing mobile Apps for cultural tourism. However, as before, they focused only on the on-site scenario when discussing the selection of an approach (Native App vs. web based service), platform, content and design of the user interface.

Context awareness and especially location awareness were among the first issues addressed by mobile application for CH (Ardissono et al. 2012). Nowadays, while the outdoor positioning problem is primarily solved by GPS which is commonly used for location aware CH information delivery (for instance Van Aart et al. 2010), the problem of indoor positioning still exists. Several attempts are made to overcome the indoor positioning problem, like the work of Angelaccio et al. (2012) using NFC (near field communication) for positioning; the work of Kuflik et al. (2012) that used ZigBee technology for indoor positioning as a trigger to drive context aware information delivery; and the work of Giemza (2013) that used QR-code reading as location indication.

Among the most notable recent developments in applying context aware mobile ICT to CH we can see a large number of mobile virtual/augmented reality applications that take advantage of the high quality cameras of mobile devices, combined with their strong computing power, positioning and communication bandwidth (see for instance Choudary et al. 2009; Haugstvedt and Krogstie 2012; Mendes et al. 2012).

Cloud computing, that suggests moving storage of data and computations to the “cloud” making software applications/services that can be accessed anytime anywhere (Armbrust et al. 2010), is another new ICT development that impacts CH. Museums and other CH institutions are considering moving their digitized collections and the information they offer to visitors to the cloud, making it more accessible while easy to maintain. One example is CH as part of smart cities (Schaffers et al. 2011). Another example is the work of Coralini et al. (2014) on the Parsjad project that focused on enabling online access to CH resources in the cloud. It is interesting to note, that cloud computing for CH is considered a step in centralization of CH information, as noted by (Snickars 2009), which contradicts the decentralization represented by the appearance of mobile Apps described earlier and the IoT described next.

Among the most recent developments that bring us closer to ubiquitous computing is the IoT. As described by Atzori et al. (2010), IoT is the pervasive presence of a variety of things or objects which, through unique addressing schemes, are able to interact with each other and cooperate with their neighbors to reach common goals. Turning to CH, these may include objects or exhibits in CH sites that talk to us, deliver information and tell us their stories. An initial step toward this idea is the work of Angelaccio et al. (2012) where the use of NFCFootnote 2 (Near Field Communication) for positioning and WiFi communication technology enabled the visitor to view a web page that describes the object on the mobile device’s screen. Another example it the “Talking Museum” (Amato et al. 2013), where the project exploits the IoT idea in order to make objects of a museum exhibition able to “talk” during the users’ visit and automatically tell their story using multimedia facilities. Using Bluetooth technology, their system is able to sense the surrounding area for detecting user devices’ presence. Once a device has been detected, a multimedia story of the closest museum objects is delivered to the related user and recommendation techniques drive users towards other objects of possible interest. The above systems combine centralized mechanisms with location aware information delivery where the objects are only used for location detection. However, current technology enables the objects themselves to store information, reason and deliver it on their own.

It is worth noting that we see what seem to be two contradicting trends. On the one hand, an abundance of stand-alone CH Apps developed for mobile devices and on the other hand centralization of CH resources in the cloud.

2.3 Development platforms used for CH applications

The step from prototypes to robust and high quality systems to be deployed and used on-site with real visitors is not a simple one. Development technologies continue to evolve and yield better means for developers and researchers to address this challenge while new technologies are continuously explored, experimented and evaluated. Recently, Gavalas and Economou (2011) published a survey analyzing development platforms for mobile applications. They analyzed the pros and cons of four development platforms, including Java Micro Edition (J2ME), .Net CF, Flash Lite and Android. This analysis is important for prototypes’ development as well as real systems’ development. Their analysis with respect to portability, functionality, performance and development speed showed that in some cases J2ME outperforms the other platforms. Kenteris et al. (2009) used J2ME for demonstrating the ability to automate the creation of a “mobile tourism” application. The user of the system visits a website and selects information of interest. The selected content is used to build an application that is downloaded to the user’s mobile device. The downloaded file is then executed by the client software. In addition to the downloaded information the user may subscribe to a “push” service where new information is delivered to the mobile device once it becomes available. In this application, the authors used Java and XML for implementing both the web based and the mobile application, taking advantage of Java’s platform-independence feature. According to the authors, J2ME offers an ideal platform for the development of full-fledged, interactive and portable applications tailored for resource constrained mobile devices. Contrarily to the above recommendation, we selected Google Web Toolkit (GWT) for reasons that will be explained later.

The TourML & TAP (IMLS grant 2009) are “toolkits supporting museum mobile experiences”. The project that developed them seeks to create mobile standards and to develop open-source tools and specifications for building, sharing and preserving mobile tours that can be used by museums of all types and sizes to create and deploy their own mobile experiences. Unlike the previously described research prototypes, this project aims at standardization of infrastructure, easing the process of tour creation in general, but at the moment only for the on-site scenario.

Another aspect of a developmental framework is an infrastructure supporting the creation of tour guide applications, regardless of the specific content. The goal of Höpken et al. (2010) was to develop a framework for mobile applications in tourism, enabling flexible implementation of adaptive, context-aware tourism applications. The framework provides approaches for user interface adaptation, content adaptation (recommendation), and interaction modality adaptation. Even though the framework seems to have the core components that may be needed also for supporting pre and post visit scenarios, these aspects were not addressed by the researchers.

3 Framework for an extended cultural heritage experience

As noted, one’s current CH experience is built on past experiences and impacts future ones. Hence technological support for CH should evolve from supporting episodes of an individual visit, as has been done so far, to a more holistic view with specific attention to personalization.

In practice, the interaction with the CH site may start when the visitor plans the visit—the visitor learns about what the site has to offer and decides what to see and how much time to dedicate to the visit. In this phase, the CH site model (see Fig. 1) may be used for enabling the visitor to explore what the site has to offer. The user model may help the system in tailoring the information to the user—if such a model already exists at the web site (returning user) or if it can be imported from an external source. In this phase, the use of standard semantic Web technologies (as suggested by the CHIP project) and/or user modeling ontology like GUMO (Heckmann et al. 2005), may enable easy access and use of the information created by different applications. In the case where no personal information is available, a user model may be bootstrapped based on the user’s behavior, as suggested by Belinky et al. (2012) or Wang et al. (2008). At the end of the planning process, using both the CH site model and according to the user model, a tour may be planned and saved. Once on-site, the individual plan may be executed or modified to be integrated with other individual plans when a group visits the museum and meets there. Alternatively, planning may take place on-site, just prior to the start of the visit (Belinky et al. 2012), again, using the common CH site model and the user model of the visitor. It is worth noting that it is not required that the visitor follow the planned tour, but rather we assume that the visitor may deviate from the planned tour and then, possibly return to the planned tour or even change it on the fly. While onsite, individuals and groups can be supported by a variety of services, including personalized information delivery offered in the right time, taking into account individuals as well as the whole group. During the visit events are recorded continuously and stored in a visit model and used for reasoning and updating the user model, thus enable to continuously infer visitors’ interests in exhibits and presentations. The accumulated on-site experience, represented in the user model, can also be used for suggesting additional relevant information for further learning and/or additional relevant CH sites to be visited by reasoning on topics of interest as reflected during the visit. It can also be used during and at the end of the visit for triggering discussion about the visit (like the “museum café” scenario—as suggested by Kuflik et al. (2011), where a group of visitors sits at the museum café and re-visits the visit together) and/or for creating a visit summary. Furthermore, the visit experience can be shared with friends using common social networks or simply kept as a personal visit summary (Stock et al. 2007). In addition the visit model can be used to update the museum model to reflect attributes such as popularity of the site and other social aspects.

Fig. 1
figure 1

The ongoing cycle of CH visit experience

Full size image

The major components (models, see Fig. 1) needed for supporting such ongoing CH experiences are: (1) a CH site model containing all information about the site that provides the content used in the three different phases, (2) a user (visitor) model that is available to the different systems using a standard communication mechanism (for instance, using UserML and GUMO), (3) a dynamic visit model that records all events happening during the visit and interpreting events into related activities. In addition we require the following abilities (services): (1) the ability to visit/plan the tour online using the CH-related information supported by the personal information, (2) the ability to transfer the planned tour and the user model to the museum (for instance using the museum website or the user’s mobile device), (3) on-site personalized support using the tour plan and the user model, and finally (4) the ability to generate a post-visit summary using the CH site information and the visit model, and updating the user model (again, using UserML and GUMO).

It is worth noting that while the site model and the user model are relatively static, the visit model is being built from scratch during the visit, for every visit. Also, the three principal components are independent even though they are associated. The user model may reside anywhere in the cloud and/or on the user’s mobile device or CH server (less likely though). It needs to be able to interact with the CH application and provide it with the relevant personal information and update the model once feedback is received from the application. Hence a standard information exchange protocol needs to be adopted between CH applications and the user model (e.g., UserML; Heckmann and Krueger 2003). The same is true for the visit model, as it is being built during the visit phases by the application; again, some kind of standard may be needed in order to enable future reuse and sharing (for instance we propose VisitML).

The visit model references the site model, as it relies on information that is available there. The site model can also reside in the cloud or in the CH website or be a part of a mobile application, but it must provide the CH site data, hence be managed by the museum staff (or on their behalf). Moreover, while the site model is one unique model per site, it does not imply that information about individual objects must be stored centrally—each and every object may have its own information, stored locally or remotely and delivered by the object or in its behalf during interaction. An appropriate base for this standard could be TourML which links stops (places at the site) with assets (multi-media objects which provide information related to the stop (see the class diagram in Fig. 3, top).

As far as technological support for the above mentioned framework, Semantic Web technologies already provide a standard infrastructure for the representation of CH artifacts as well as user models, while software engineering provides standard design patterns and development tools. As noted earlier, various components of the above suggested framework were already demonstrated in different applications, including LoL@, PEACH, PIL, CHIP, GUIDE, to list a few. In addition, technological infrastructure issues were addressed also by CHIP, myMytileneCity (Kenteris et al. 2009), TAL and TourML city projects. However, all these components need to be integrated into one coherent framework, from the visit planning until post-visit summary, in order to provide a better and more holistic experience to the visitor. Current cloud computing technology provides unlimited storage and continuous access to information by providing the necessary infrastructure while the idea of IoT provides another mechanism for distributed interaction and delivery of information.

4 Framework architecture

This section describes the design that implements the requirements described in Sect. 3 and demonstrates the framework architecture for supporting the three different phases in an integrated manner—the online pre-visit planning, the mobile on-site visit, and the online post visit experience (it should be noted that each of the pre and of the post visit scenarios can be interleaved with the on-site visit, as visitors may re-plan and re-visit the museum during the on-site visit).

The following hierarchy of components support and provide the basis for our framework architecture: models, which are collections of the representations of the objects contained in the museum environment, services, which utilize the models, clients, which in turn, interact with the services to provide various applications. Later (in Sect. 5), we describe the computing platform, which allows us to connect the above parts.

4.1 Models

The models constitute three basic shared building blocks of the framework and formalize the Visitor (or User) Site and Visit models that were mentioned earlier (we do not presently delve deeply into the details of the models for the sake of brevity). The Visitor model (Fig. 2) describes the individual visitor and groups of users and contains their user models. It contains a user profile consisting of demographics, preferences (using GUMO tuples as a standard), visitor context [based on Raptis et al. (2005) and Wigelius and Väätäjä (2009)] and a list of references to visit history objects. In the “pre” phase, it is used to enter demographics about the user, plan a tour (which is stored as a visit object), and build an initial profile (top right part of Fig. 2). “During” the museum visit, the Visitor model is used to make recommendations and guide the user along the planned tour, while the Visit model stores time-tagged information concerning the visit events. As noted, in the “post” phase the preferences along with the visit history information are used for presentation of visit information to the user. The Site model (which corresponds to a Tour in TourML terminology) describes the museum and contains objects which have a location and connections to other places. The non-strict hierarchy of the Museum object includes: yards, buildings (only 1 in our implementation), floors, rooms, gateways (which can be entrances, exits, or bidirectional), exhibition areas, cases, shelves, and exhibits. The “Exhibition areas” object contains the various different types of exhibits. The “Connection” is a description of an “edge” which connects two stops. When all the connections are taken together they form a graph of the possible paths through the stops. The connections can be physical (to aid in navigation) or semantic (to aid in recommendations of what to see). Continuing with the Site model (Fig. 3, top), “Assets” consists of various multimedia (video, audio, HTML) objects and their attributes which can be presented to the user (corresponds to Assets in the TourML sense). The Visit model consists of “Event” which tracks single “point in time” incidents related to the visit and “Activity” which contains historical information concerning the visit over a period of time (e.g. exhibits and presentations viewed, time spent near exhibits, etc.) and evaluations/feedback provided by a visitor during a visit and a reference to a planned “tour” (if any).

Fig. 2
figure 2

Class diagram of the visitor model

Full size image
Fig. 3
figure 3

Class diagram of site model (top) and visit model (bottom)

Full size image

Having a common model across all three phases of the visit prevents duplication and inconsistency, and enables rapid building of new applications. Table 1 presents a brief overview of the categories and top level objects in the models and corresponds to the UML class diagrams in Figs. 2 and 3.

Table 1 Model objects according to categories
Full size table

4.2 Services

A set of services was built upon the model and use the model to provide a high level functionality that can be combined together and reused to form different applications (clients). These services can be used in a variety of different phases (pre, on-site, post) of the museum visit. Table 2 includes a list of available services in our implementation. To illustrate their use of the model, let us consider the Navigation service. During the on-site phase, the Exhibit Areas serve as source and destination points in a route, for the navigation service that calculates the possible paths between the source and destination. In addition, from the Visitor object, the service takes the preference for an accessible or non-accessible route, and uses it as a factor in path calculations. This navigation service can be used also by the Tour Generator service, during the pre-phase to order a set of exhibition areas in a manner which is logical for traversal. Services, while reusable, can be expanded and modified for specific needs.

Table 2 Representative framework services
Full size table

4.3 Example web applications

Each web application in our framework is built from a single client (user interface, application logic) and a number of services. The applications (e.g., Mobile Museum Visitor’s Guide, or Visit Planner) can be built around the services listed above for use during the various phases of the museum visits (for a partial list of possible applications see Table 3). For an example of how an application is composed of services we examine the mobile guide application (which is composed of nearly all the services): It uses the “User Group Manager” service to authenticate the visitor and gathers information about the visitor from the “User Profile Infobase” service which is used as input to the various recommendation services. The visitor receives a planned itinerary from the “Tour Generator” service (either pre-planned or one of the standard available tours). As the visitor moves along in the museum his/her position is determined by the “Location Based” service according to advice given by the “Navigation” service. When the visitor arrives at an exhibit a choice of presentations ranked by both the “Presentation Recommender” service and the “Social Presentation Recommender” Service are suggested. The visitor then can pick a presentation to view, which is contained in the “Museum InfoBase” service. It is easy to see how these applications use the services described above. The services can be composed and used in other scenarios as well, like for instance the post visit summary generation. While applications are reusable (just use different site data), they may require customization at different levels. For example at the simplest level, one can change the “look and feel” to brand the application to a particular museum. At a higher level, it might be desirable to change application logic or offer additional or modified functionality based on addition services.

Table 3 Example web application
Full size table

5 A case study of applying the framework in practice

In this section we demonstrate the feasibility of our framework by reporting on several research prototypes that were built upon the framework’s infrastructure. We show how the same infrastructure can be used to build interlinked applications for all three visit phases. This is followed by technical implementation details that may be useful to practitioners. The PIL project is a research project focusing on exploring the possibility to use novel ICT for enhancing the museum visit experience. Its initial phases are described in (Kuflik et al. 2011) and more advanced phases in (Lanir et al. 2011; Kuflik et al. 2012). In the framework of the project, a research prototype was converted into a working museum visitors’ guide system, applying the framework presented earlier. The system is used on a daily basis by visitors to the Hecht museum,Footnote 3 a small size museum,Footnote 4 located on the campus of the University of Haifa, which contains both archeological and art exhibits (Lanir et al. 2010). It has been developed following user-centered design principles and was evaluated in user studies and field studies as detailed in (Kuflik et al. 2011; Lanir et al. 2013a). The user interface design was kept while the system itself has been developed from scratch using the suggested framework. The mobile guide is the core component around which the suggested framework has been developed and demonstrated. It is further described in the on-site support section.

5.1 Visit planning

The Web-based pre-visit planning system (the museum trip planner, see Table 3) (Belinky et al. 2012), enables the visitor to plan the visit ahead of time, by taking a virtual tour of the museum and learning about the exhibits, examining objects of interest and selecting those that will be included in the tour. As part of the process, the system informs the user about the estimated duration of the tour and creates a path through the museum, so the visitor will pass through all the points of interest selected for the tour (using the tour generator service, see Table 2). In addition, the visitor is questioned about specific interests related to the museum exhibits, for use in bootstrapping a user model for the tour that will be used for recommending multimedia presentations at the different locations. Considering our suggested framework, the above process is reflected in the visitor’s model—the planned tour becomes part of the model and the visitor’s preferences become part of the user model, both will be used on-site as we see in the on-site application (described below).

The user first views a floor plan of the museum, where general exhibition areas are marked. The visitor may select a room map or image (see Fig. 4), where all points of interest in this room are marked. Within the room the user may read about specific objects (additional screen). The user can select relevant points of interest, state their priority, and include them in a tour plan. Once the user is finished, the system constructs a personal tour using the tour generator service and keeps it on the system’s database, to be used later on-site. In practice, the tour plan may be saved in the cloud or mailed to the user or downloaded to the user’s mobile device.

Fig. 4
figure 4

Pre-visit planning system. Room level view map (left) and an aerial image (right)

Full size image

When the user arrives at the museum as a visitor, the tour is available for the visitor to follow. This planning capability may be available in principle also on-site—when the visitor is waiting in line to purchase tickets. In case of a group of visitors, the individual plans previously constructed may become an input for a group planning activity, where the group may view and integrate the individual plans over a large screen (Fig. 5) while using a mobile device as input and control device. The plan may be revisited and modified during the visit itself.

Fig. 5
figure 5

Group planning onsite. Individual plans on top and the resulting group plan at the bottom

Full size image

A second pre-visit option allows visitors to choose, in the planning phase either at home or on-site, a pre-set tour of the museum. Based on four visitor types defined by Falk (2009), we defined a number of stereotypical tours, with fixed itineraries.

The pre-visit planning system was evaluated in a small-scale user study, and was also linked to the on-site visit as the participants actually performed their planned tour (Belinky et al. 2012). Thirty six participants, divided into 12 groups, experimented planning a visit in two phases. The groups consisted of students aged 19–29 (17 male, 19 female) who were paid for their time.

The first phase included an individual online visit planning session using a desktop computer, while the second phase was a mutual group planning session onsite, where the group met and mutually integrated the individual plans into a group tour (following the planning phase, the groups conducted a real museum visit where the on-site navigation support was evaluated—see next section). The results of the pre-visit planning showed that participants enjoyed the planning process, both in the pre-planning application and in the on-site visit planning one. Participants felt that on-site planning made the museum visit experience more fun and enhanced the visit experience. Several participants stated that the option to prepare the visit ahead of time from home creates anticipation towards the actual visit in the museum and stimulates the interest to see the actual exhibits. Participants also felt that the on-site planning application is useful and easy to use, and generally they liked the proposed technology. They also liked interacting with the handheld devices and felt that it was comfortable. Furthermore, participants felt that the planning system saves time by enabling more efficient planning especially in case of a group visit. They stated that they would use such a system when they would not have much time for the visit and wished to have a more focused visit.

An additional system, not yet evaluated, allows users to choose plans based on their social network (spots their friends on Facebook™ liked) or a tour based on the most popular sites if there is not enough social network data.

5.2 Onsite support

The mobile guide used on-site is location aware. Its positioning is based on proximity detection (Fig. 6 illustrates the system’s components). Fixed Beacons are placed in points of interest in the museum and the visitors are carrying mobile devices called “Blinds”. The visitor wearing a Blind and holding a hand-held device (an iPod) is free to walk around in the museum.

Fig. 6
figure 6

PIL positioning architecture

Full size image

Once a visitor is detected at a point of interest (the Blind detects the Beacon and reports this event to the server), the system presents the user with a selection of nearby objects on the handheld device (Fig. 7, left). The user then selects a specific object of interest (among those marked by yellow rectangles) which prompts a list of questions (Fig. 7, middle). At this point, using recommendation service, the system suggests and recommends the visitor presentations to view in the form of a five-star scale, which in our framework, is based on a service that uses the user model part of the visitor model that was initially built during the planning phase and which is continuously updated during the visit. Once the visitor selects a question of interest, a 1-min multimedia presentation is played, providing an answer to that question (Fig. 7, right). Once the visitor views a presentation, a feedback is requested on a five point Likert scale.

Fig. 7
figure 7

Mobile visitor’s guide screenshots

Full size image

As mentioned above, when the visitor arrives at the museum, she/he can follow a personalized pre-defined tour, where the system guides the visitor through the museum using landmark-based navigation. The visitor is able to view a museum floor map and to zoom in and out and/or request directions to any point of interest, as needed. Since visitors may come to the museum in small groups and these groups sometime split when visitors follow their individual paths, the system enables context-aware communication between group members—visitors can see where their group members are on the museum map and they may leave virtual post-it messages and send immediate messages to other group members (for more details see Wecker et al. 2011).

The mobile system was successfully deployed at the museum at the end of 2010 and has since then been available to visitors free of charge, in three languages (Hebrew, Arabic and English). The system has been evaluated (and continues to be) in various user and field studies that show overall that the visitors enjoy to use it and feel that it contributes to the complete visit experience (for further details see Kuflik et al. 2011; Lanir et al. 2011, 2013a; Kuflik et al. 2013; Dim and Kuflik 2013). Even though the museum is relatively small and not a busy one, several hundreds of visitors’ records and questionnaires were gathered so far; enabling us to conclude that the use of a mobile guide significantly changed the way visitors behave in the museum (Lanir et al. 2013a). Some of the changes, such as the increased time spent in the museum, can be considered beneficial, while other changes, such as the increased attention of electronically interpreted exhibits, need to be carefully considered by museum curators regarding how this change will affect the visitor’s entire experience. Additionally we presented empirical evidence to support the notion that an electronic museum guide detaches the visitor from his or her group.

5.3 Post visit support

During the visit, all visit events are recorded in the visit model, including places visited, presentations viewed, feedback and more. These records are used for keeping the visitor in contact with the museum in various ways as detailed below. The visit history/log and the user model may be saved and exported for future use locally or elsewhere (e.g. in another site).

Several applications for post visit activities were explored and demonstrated. One option to re-experience the visit is the museum café scenario, where visitors may sit at the museum café and share and discuss their individual visit with their group members—looking at a larger display where the group data is presented (Fig. 8). It is worth noting that this scenario is applicable also to breaks during the visit and not necessarily just to post visit scenario. This was implemented in a previous version of the project (Kuflik et al. 2011).

Fig. 8
figure 8

An image of a visited exhibit with stick figures representing group members and positions visited (right), and a specific position at that exhibit with presentations viewed (left)

Full size image

An extension of this idea, allows the visitor to re-experience the visit at home by visualizing his or her visit path. Using a Web interface (that may be accessed through the museum website), the visitors can view and playback their visit path in the museum, see which exhibits they have seen (and which ones they missed), and re-view the presentations that they saw at the museum. Thus, the visitors can continue their learning and museum experience at home, possibly going over information that they did not understand or that they want to go over again. Furthermore, visitors can see which locations they visited and which ones they missed, and thus plan another visit to the museum.

Another application is a post-visit video summary (Lanir et al. 2013b). We built an application that automatically constructs a high-definition video summary of the visit using the visit logs by integrating snippets related to visitors’ path and visited exhibits. The video first provides a general introduction to the museum. Then, for each exhibition that the visitor attended, the video shows an introduction snippet on the exhibition, followed by video snippets on the exhibits that the visitor attended, as well as a summary of the presentations that the visitor viewed. At the end of the video, exhibitions that the visitor has not seen are shortly mentioned. Finally, the video shows statistical information about the visit (e.g.: “You saw 10 presentations in 5 exhibits”).

Finally, a text-based personalized visit summary report, where using the visit log, the system reasons about the visit, summarizes the most interesting aspects of the visit and suggests future activities following the visitors inferred interests, as learned during the visit is also possible. This was already implemented in the PEACH project (Stock et al. 2007), a predecessor of the current PIL and is being implemented within the PIL project presently, using the generic framework architecture.

Considering our framework, each of the post visit summaries may be an independent service that makes use of the visit model in order to provide the specific summary. This example demonstrates the flexibility of the approach where a variety of services use the same model for creating similar but different summaries. It should be noted that while these post-visit summaries were demonstrated, they were not yet evaluated by visitors.

5.4 Technical implementation details

5.4.1 Application architecture

We adopted the Client/Server application architecture, with the client side using the Model-Presenter-View design patternFootnote 5 described in Fig. 9 [the idea of applications consisting of various services is similar to a Service Oriented ArchitectureFootnote 6 (SOA)], as the basis for our individual application architecture. This architecture helps to isolate the view from the model details and hence enhances reuse and interoperability.

Fig. 9
figure 9

Client/server and model view presenter (Zhang and Luo 2010) application architecture

Full size image

To illustrate the flow, we present two examples. In the first, the visitor presses a button in the View (User Interface) to request information about an exhibit; this generates an event which is handled by the Presenter (Interface logic). The Presenter either fulfills the request using a call to the local cache of the model or sends an event to the application Controller (Application logic) to make a remote procedure call (RPC)Footnote 7 to a service to get the information needed to get the request. On the server, it may call a number of services who use the “Model” to provide the information and return it to the application Controller asynchronously, who then either creates or calls the appropriate Presenter. The Presenter then calls methods of the View to show the requested information. In a second example, the application Controller periodically requests events from the serverFootnote 8 (and can be answered when for instance the visitor has been identified in a new location by a positioning service). The information is returned as part of the RPC request for events. The application Controller, using the local cache of the model determines if the application is in a state to change (e.g. is it possible to interrupt the visitor), if so it calls the appropriate Presenter to call the corresponding View to show the new position (e.g. letting the visitor know about nearby points of interest).

5.4.2 Computing platform—Google Web Toolkit (GWT)Footnote 9

For developing our current version of the system and enabling a unified approach for the pre, during and post visit scenarios, we selected the GWT infrastructure. It is an Asynchronous JavaScript and XML (AJAX) web-based frameworkFootnote 10 for constructing web-applications in Java (Farrell and Nezlek 2007; Hanson and Tacy 2007). It allows platform free development of a web-based application that can run on a desktop as well as on a mobile device, a fact that eases and streamlines the development. GWT allows structured building of Web Applications (builds are optimized per browser) that are automatically updated from the Web (reduces deployment problem). It uses Java for both client and server [on the client side, the Java source code is compiled to JavaScript with optimization for different browser user agents (permutations)] and can be run on the cloud using Google Application Engine. It has a well-developed integrated development environment for testing (including performance) connected to Eclipse,Footnote 11 can be used with the Eclipse Modeling Framework and Graphic Modeling Framework for model based development (Meliá et al. 2010), and supports HTML55 and CSS3.Footnote 12

The advantage of using Java on both the client and server was critical for us, since it allowed us to choose where to deploy objects. For example, we experimented with putting the navigation services on the server vs. downloading the info to the client. The client side of the application has the option of being thick or thin, depending on the size of the model cache. In the museum guide, for example, we tried to minimize communication and have the guide be able to work despite communication disconnects, thus for the mobile museum guide we used a fat client by caching the relevant parts of the model (primarily the whole Museum object hierarchy) on the client. By downloading the museum InfoBase onto the client, we significantly reduced the number of RPCs (a major factor in performance) and increased stability (less vulnerability to temporary glitches in communication). For applications used by visitors at home we wanted to minimize application size, both for performance and security reasons hence we used thin clients. The GWT testing tools and optimizations per browser allowed us to build applications that were equivalent to native graphical user interface applications in performance and complexity, yet still maintain platform independence. Another important advantage is that as a default, to the developer, GWT communicates as if it was an asynchronous RPC protocol. Communication between a Java program and the GWT servlet is done through the SyncProxy protocol.Footnote 13 This allows us to support a number of interacting servers (distributed on their own Java Virtual Machine running various framework services.

GWT has its disadvantages though, it has a long learning curve. Java developers can deploy with AjaxTagsFootnote 14 (or other JSPFootnote 15 tag libraries that wrap AJAX functionality) in just a few minutes, whereas it takes much longer to get anything running with GWT. It has a nonstandard approach to integrate JavaScript. JavaScript is never directly contained in the HTML document. Instead, JavaScript Native Interface (JSNI)Footnote 16 is used to wrap JavaScript in Java. This is very powerful in the long run, but difficult to adjust to in the beginning. Most AJAX environments run JavaScript on the client and have an option to do so for the server-side. GWT is based entirely around Java. It follows a different approach with a fundamentally different strategy than other AJAX environments. In our context, a primary disadvantage of a GWT Web Application is the lack of an option to push messages from the server to the client; we overcame this using long polling. Other solutions include newer technologies such as Web Sockets (Lubbers et al. 2011) which may provide a more standardized solution to this problem. However, in summation, the above disadvantages were not critical in our context and were outweighed by the advantages described earlier.

6 Discussion, conclusions and future work

The suggested framework follows the recommendation of museum researchers such as Falk (2009), in extending the visit experience beyond the on-site visit, to integrate the pre-visit, on-site and post-visit experiences into one holistic experience. We started by defining the requirements for such a framework in Sect. 3, then, we suggested a framework architecture that enables the development of CH guide systems linking the three visit phases in Sect. 4. This was followed by Sect. 5 where we described several applications we developed that demonstrate the framework’s usefulness in an implementation of a real system, supporting all three visit phases using a common, state of the art GWT web-technology. Finally, we gave implementation specifics in order to aid future developers. As we showed, the suggested framework enabled us to demonstrate the idea of a continuous CH experience in a real life setting, in a small museum. However, considering our settings, a few questions need to be answered: how general is the small scale proof of concept? Is it scalable? Will it support foreseeable ICT for CH development? We discuss these questions next.

The suggested framework is generic. It does not contain any limiting component, as it includes generic user, visit and site models that can be used with any CH site. The site model can be adapted to any CH site and is compliant with the generic TourML tour planning/visiting language. The same is true for the visit model as it is used to store time tagged events that occurred during the visit. Finally, we have the user model, which may reside anywhere—on the user’s device, in the cloud or elsewhere. However, these three components need to be able to communicate in order to share the visitor’s personal information and to deliver personalized, context aware services. Hence, they need to use an agreed communication protocol (one option may be the UserML suggested earlier). However, like any standard, the suggested framework requires adoption by CH application developers. This may be a challenge with the growing number of different mobile CH Apps being developed for various CH sites. Still, like in many other areas, we believe that standards are needed for collaboration between applications and, keeping the visitor interests uppermost, such standardization is essential.

As far as scalability, assuming such a system is web-based, the services (mainly personalized, context aware information delivery) are limited only by the server and network bandwidth. Given the fact that visitor access information when they are approaching or near specific objects and that the information is relatively limited as visitors spend little time listening/watching presentations, the ability of a web-based system to support tens and hundreds and even thousands of visitors at the same time does not seem to be a major problem. One possible solution for bandwidth limitation may be to download the expected content once, at the onset of the visit while interaction during the visit will focus on reasoning and recommendation of what to select for presentation at any given moment/location. Other solutions involve streaming multi-media to avoid latency. As such, scalability does not seem to be a major issue.

A challenging issue is the suitability of the suggested framework to support future ICT, mainly in the area of ubiquitous computing and the evolving Internet of Things that starts to appear in CH applications, as being experimented in meSch (Petrelli et al. 2013). As we showed earlier, the migration of CH to the cloud and using cloud technologies for serving visitors may ease the adoption of standards. The Europeana project is one example where state of the art ICT are applied for enabling sharing of vast amounts of CH material and making it available to everybody in digital form (Koers et al. 2012). A key concept in Europeana is standardization of the data model. Within the framework of Europeana there is an acknowledgement of the potential impact of future ICT, among them IoT, that in the near future will allow for remote management of the objects. If we consider our suggested framework, it is compatible with the concept of IoT. The models do not have to change. Assuming the individual objects at a CH site “talk” to the visitor, a common communication protocol is needed. This is true regardless of the specific infrastructure a system used. Hence, our suggested approach is suitable in the sense that we suggest a standard way of interaction, following and extending the already existing TourML language.

It is worth noting that the suggested framework has additional important benefits for the museum, as an integration of accumulated post visit experiences may visualized for the museum staff that may learn about holding power and attraction power of exhibits as well as on visitors’ circulation (Lanir et al. 2014).

To conclude, the suggested framework architecture can be replicated and deployed in other museums and in other CH settings, not limiting its users to any specific technology or architecture. Moreover, we plan to make the main building blocks freely available to researchers and museum practitioners interested in using them for building their own system. In this way we hope to enable researchers to focus on their specific research interests without the need to develop a complete system every time, hence allow them to get meaningful results quickly. We also hope to encourage researchers to contribute and enhance this infrastructure and make it a common research infrastructure that will continuously improve. We believe that as community efforts and projects such as TourML, that address not only content preparation but also building blocks for applications, this idea becomes a viable reality easily used by any cultural heritage site and visitor alike.

In addition to the successful demonstration of the concept, one can view this work as a step towards linking individual CH experiences into a lifelong experience supported by current technology. A cultural heritage experience should eventually be reflected in subsequent occasions: for elaboration, for exploring further material and for recalling what was experienced in new related situations. This will require not only further technological advancement but also insight on the cognitive aspects of the visit and of the subsequent phases that reconnect a visit to previous visitor experiences.