Keywords

1 Introduction

Advances in interactive 3D technology in the last decades led to a variety of new and promising approaches for representing and disseminating cultural heritage, generally termed as ‘virtual heritage’ [1]. These applications incorporate a number of distinguishing characteristics that include high-quality visualization of digital content, real-time simulation of realistic or imaginary environments, natural and intuitive user interactions, and single- or multi-user embodiment in the 3D space through animated avatars. Applications of this kind not only allow people to closely observe cultural heritage artifacts that may be difficult for them to approach physically, for reasons of distance, cost or accessibility, but they can also serve as motivating means to supplement people’s knowledge and increase their interest in culture.

The majority of virtual heritage applications place more emphasis in the detailed presentation of buildings and artifacts, compared to other, intangible aspects of cultural heritage. This is somehow expected, because in the area of cultural heritage there is great interest in the form and architecture of past constructions. Nevertheless, a high-quality 3D visualization is not alone adequate to ensure that the user experience will be as engaging and fruitful as expected. These reconstructions need to be experienced together with a number of other, intangible aspects of the related historic period and culture in order to be immersed in the cultural context [2]. E.g. visitors might be interested in observing aspects of daily life, activities and rituals taking place in the buildings, typical usage of the artifacts being presented, related stories and events, etc.

The demonstration of such features in virtual heritage applications requires the introduction of digital characters that can participate and act in the simulated environment. These characters can be either controlled by humans (human avatars), or controlled by a computer program, with varying levels of adaptability to the environment or human user input [3, 4]. Furthermore, the use of techniques from the field of Artificial Intelligence, such as Intelligent Agents and Natural Language Processing, can enhance computer-controlled characters with features such as planning, personality and emotions, and conversing with users [5]. Virtual heritage applications have recently started to take advantage of the affordances of digital characters, and have incorporated them in the representation environment.

In this paper we present our work towards the use of multi-user 3D environments enhanced with digital characters for the presentation of intangible aspects of cultural heritage. We have developed a generic platform based on the OpenSimulator virtual world, which enables the creation of interactive scenarios and stories using programmable agents and ‘smart objects’, i.e. 3D objects with interactive behavior. The agents embodied as digital characters can move in the environment and hold conversations with users, whilst both users and agents can interact with the smart objects in the context of a larger story or quest. As a first application example, we have constructed a part of the ancient agora of classical Athens, and prepared an interactive scenario, in which a visitor and his companions want to make an offering to the gods. We present the design and implementation of the platform, the setup of the environment and the interactive scenario, as well as a first qualitative user evaluation of the environment. The evaluation results indicate that users may easily learn about intangible aspects of culture, explore and interact with virtual agents in virtual worlds for virtual heritage.

2 Related Work

Digital characters have been introduced in virtual heritage applications in a variety of ways. The most obvious approach is to have them operate as ‘animated props’. Such characters usually execute pre-scripted animation sequences that are sometimes parameterized for greater diversity, and their operation is independent from any changes in the environment or user activity. Their primary contribution in cultural heritage applications is to demonstrate the appearance and typical activities of indicative people in the place and time of reference.

More interesting and dynamic character behavior can be achieved with the use of virtual crowds. In this case, characters move and act collectively imitating the behavior of real human crowds. In most virtual crowd systems, individual agents are automatically generated based on generic rules that define their appearance and properties, and they are assigned a role from a predefined set. E.g., in the work of Maïm et al. [6] a real-time simulation environment presenting a reconstructed district of ancient Pompeii has been populated with a crowd of virtual Romans.

A different utilization of digital characters with more essential contribution to the user experience is to have them operate as virtual guides. In that case, the characters have the additional ability to communicate with the users, and their goal is make the experience livelier and pleasant for the visitors by presenting places, objects or related stories to them. There are plenty of virtual guide implementations in virtual museums and reconstructed cultural sites, which vary in terms of the means of communication and the adaptability of the presentation, e.g. [7].

In an attempt to integrate the affordances of the previous approaches, Panjoli et al. [8] have proposed a framework for digital characters in virtual heritage applications that consists of three distinct levels of interaction, depending on their distance from the user. The first level is the living background, where the character operates as part of the virtual crowd and facilitates the user immersion in the environment. Characters that are closer to the user switch to the interaction level; they pay attention to the user and allow for some basic interactions with him. The final level is the dialogue, where the user and character interact in natural language using text or speech. A first implementation of this framework is found in the Roma Nova project.

Finally, intelligent agent approaches can be used to offer a long-term, goal-oriented operation of characters in cultural heritage applications. Intelligent agent architectures such as the BDI (Beliefs – Desires – Intentions) approach can be used to implement digital characters with the ability to accumulate new knowledge about the environment using the input received by their sensors, to prioritize their next tasks according to their long-term goals, and to plan a sequence of actions to achieve the desired results. This approach leads to a more elaborate behavior that is not as predictive and repeatable as in the case of following pre-scripted orders. Approaches based on intelligent agents may possibly lead to the implementation of the recently proposed concept of cultural agents [9], which are digital characters that can select or recognize correct cultural behaviors and transmit cultural knowledge, and are considered more appropriate for virtual heritage applications.

The City of Uruk [10] is a virtual heritage application that uses intelligent agents in a virtual world to present daily life in an ancient city. The characters have been built based on a BDI architecture and follow a daily routine that involves movement, interaction with objects and communication with other characters. The agents’ actions are shaped by their beliefs about the environment. They can follow pre-scripted plans in order to perform some standard activities, and they can also update their goals and generate dynamic plans as a result of certain changes in the environment. Finally, they have the ability to communicate with human visitors using natural language. They can talk about their current goal and planned actions, and they can also present information about the surrounding objects and environment. In a study aimed to validate its learning effectiveness [11] the application yielded positive results regarding student performance.

Our work also focuses in the use of intelligent agent approaches for virtual heritage applications, emphasizing in the presentation of intangible aspects of cultural heritage, such as daily life activities and rituals. While the majority of similar applications created so far are single-user, we wanted to explore the suitability of this approach in multi-user environments and for that purpose, we designed and tested a collaborative scenario in which users can work in groups in order to explore and learn about a specific ritual. Additionally, given that there are only a few evaluations of multi-agent virtual heritage applications in the literature, we decided to set up a user evaluation of our environment to gain some useful insights on the suitability of this approach for the dissemination of cultural heritage.

3 A Platform for Multi-agent Virtual Heritage Applications

We have developed a platform for cultural heritage applications in virtual worlds that includes multiple autonomous agents moving and interacting with the users and the environment. The platform provides a set of high-level tools and reusable components that aim to assist developers, and allows the execution of complex virtual heritage scenarios that may include features such as:

  • demonstration of activities, habits and rituals of ancient cultures,

  • agents communicating with users and presenting locations, elements or activities of interest, and

  • scenarios requiring user participation, such as interactive stories and quests.

The architecture of our platform (Fig. 1) follows a three-tier client-server model based on OpenSimulator, an open source alternative to the virtual world of Second Life. Users join the application using a compatible browser that connects to the OpenSimulator server, and they are embodied as avatars in the representation space, which also includes the embodied agents (named NPCs – non-player characters in OpenSimulator terminology). The interactive behavior of the environment is orchestrated by a special object named Interface Unit that mediates between the virtual world and a multi-agent simulation environment. The latter is an external application implemented in Java, which handles the agents’ behavior and conversations, using an abstraction of the actual world and its contents. Thus, the agent operation takes place in two parallel layers: the low-level execution layer of the virtual world, in which the NPCs move and interact with the objects and users of the environment, and the high-level layer of the multi-agent simulation, in which the agents update their beliefs, prioritize their goals, and decide about their next actions.

Fig. 1.
figure 1

System architecture of the multi-agent platform.

Full size image

The elements of the virtual world that have an active role in the platform are the Interface Unit, the NPC Controllers, the user HUDs and the smart objects. The functionality of the Interface Unit is twofold: it constantly updates the multi-agent simulation with any changes happening in the 3D environment, and it also triggers actions related to the active objects of the virtual world. For the first function, it makes use of a specially built extension to the OpenSimulator server (Region Module), which monitors the placement of all entities of the environment and identifies any changes. For the second function, it communicates with specific scripted objects in the 3D environment. Specifically, digital characters have an attached object named NPC Controller, which controls their actions, user avatars have an attached Heads-up Display (HUD) object that triggers specific actions and presents information, and smart objects are can also receive messages from the Interface Unit and adapt their behavior accordingly.

The multi-agent simulation environment controls and monitors the behavior of the agents that participate as NPCs in the application. The operation of each agent is driven by a respective Agent Controller, which updates its beliefs, takes any required decisions and executes the current plan. The agents’ sensory input is based on an object repository that stores basic geometric information (size, position and rotation) of all elements that actively participate in the simulation, i.e. agents, users and selected objects. The repository is constantly updated by the Interface Object to reflect the active status of the 3D world. The multi-agent environment is equipped with a dialog engine based on AIML (Artificial Intelligence Markup Language [12]) for the agent-user communication.

The agents’ behavior in the environment is controlled by pre-scripted plans, which are assigned to them during the lifecycle of the application. Each plan is a composite program that results to the execution of a sequence of actions taking into account the perceived status of the environment. The plans are described using a dedicated high-level scripting language, the Plan Definition Language (PDL). Each plan defined in this language has a unique name and an arbitrary number of argument variables. The plan implementation is encoded in an imperative programming manner that allows sequential, conditional and iterative execution of commands, based on the definition of PDL. The supported commands fall into the following categories:

  • actions that are executed by the NPC in the virtual world (locomotion, animation, interaction with smart object),

  • “internal” actions that affect the internal values of the agent,

  • message passing between agents,

  • dialogs with users,

  • memory processes (store/retrieve temporary information), and

  • perception of specific elements based on criteria.

Finally, the execution of a new plan can be called during the plan implementation, thus leading to more complex plan definitions.

Dialogs between users and agents can start either with agent or with user initiative. The second is possible if the agent is available for discussion and the user clicks on its controller to start a dialog with it. In both cases, the agent part of the dialog is driven by the dialog engine. Designers can create a number of AIML bots, each of which is based on its own collection of AIML files that define how the agent responds to user input. We have extended the AIML language with custom tags in order to embed dynamic information in the dialog and to trigger new agent actions during the discussion.

The interactive behavior of the smart objects is also programmable. A simple scripting language defines the actions that they support and their implementation. Typical such actions include to take an object, to give it to another user, to place it on a designated surface, etc. Users trigger these actions through menus that appear when clicking on an object, whilst NPCs can send messages to the object to trigger the same actions.

4 An Interactive Scenario in the Ancient Agora of Athens

The scenario selected for this study takes place in the Ancient Agora of Athens. It adopts an exploratory learning stance [13], where the users have to uncover critical information themselves, and collaborate to fulfil the quest. We have created in OpenSimulator a reconstruction of the central part of the agora around the second half of the 5th Century BC, modeled in real-world scale according to measurements from the excavations of the American School of Athens [14] (Fig. 2a). The scenario aims to acquaint users with the multiple functions and locations in the agora and provide knowledge about ancient Greek worship practices. The story revolves around one of the user avatars, in the role of a visitor that arrives for the first time at the agora, accompanied by two friends, wishing to secure the gods’ favor to have a rich harvest at the end of the farming season. Being unfamiliar with the place and local customs, the three visitors cooperate in order to identify locations and persons of interest for their cause, as well as gather information about the proper way of worship.

Fig. 2.
figure 2

a. Construction based on the ground plans, b. Users visiting a wine seller, c. Users and priest outside the temple of Hephaestus, d. Performing a ritual in the altar of the 12 Gods

Full size image

Two roaming citizens (NPCs) serve as guides, providing information about different deities and indicating that the main character needs to secure the blessing of Demeter, the protector goddess of agriculture. For this, the main character needs to visit one of the two temples located in the agora. In order to pay respect to the god and request assistance, the main character has to perform a libation and in particular a ‘sponde’, that involves offering watered wine to the goddess, to be poured over the altar while reciting a prayer. The main character must now gather the necessary artifacts to perform the ritual, by visiting the vendors’ stalls, in particular a pottery seller to buy a proper container and a wine seller to buy the proper kind of wine. This task can be delegated to his friends.

By visiting the wine seller’s stall, users can get information on the different kinds of wine and the uses of each kind in everyday life and then choose the correct one for the purpose. At the pottery seller’s stall, users can get acquainted with the different kinds of pottery, the features distinguishing one from another as well as the particular functions of each type of pottery, once again choosing the proper one to perform the ritual.

The priest at the temple is the master of ceremony, monitoring the main character’s actions and sayings to ensure that the ritual is followed step by step. One of his aides, the ‘neokoros’, is assigned support tasks but most important he can further break down the ritual into steps that the main character has to follow in order for the ritual to be successfully completed. Another aide is assigned the task to play music using a double flute, while the ritual takes place. The main character is first required to cleanse himself by using the ‘perirranterion’, a washing basin located outside the altar. Then, the prayer must be recited in the proper order and the offering to the goddess must be given. This is done by slowly pouring wine from the ‘oenochoe’, the container the wine is brought into, to the ‘phiale’, a ceremonial cup provided by the priest. Some of the wine from the phiale has to be poured on the altar for the god, while the rest is consumed by the main character and his friends, when the prayer has been recited. This signals the closure of the ritual and the visitors may now depart the temple and the agora.

The scenario uses twelve NPCs and a wide variety of smart objects, including various pottery types, the washing basin, the flute, etc. Besides the musician, all other NPCs are able to hold conversations with the users that are related to their role in the scenario. Screenshots of the development and execution of the scenario are shown in Fig. 2b, c and d.

5 User Evaluation

We performed a qualitative user evaluation of the environment in which 12 users in four groups of three completed the above scenario in the computer lab. The goals of the evaluation were to: (a) identify if it would be possible for users to find out information about the ritual on the basis of exploration of the environment and natural language dialogues with avatars, (b) assess their user experience and identify potential areas of improvement. The only information provided to users was to talk to avatars to find out more about the offering. The tasks to be identified and performed were: to buy a container, to fill it with proper wine, to go outside the temple and cleanse, to learn about the ritual from the ‘neokoros’, and to properly perform the ritual with the aid of the priest. All participants were students of design engineering (av. age: 21.38, st.dev.: 1.38); six had previous experience with virtual worlds and relevant games. During each group session, an evaluator was observing ‘over the shoulder’ and took notes of utterances and behaviors. At the end of each group session, a retrospective think-aloud session occurred, in which users went through their acts and reported on their experiences. Furthermore, chat logs were reviewed for clarifications and validation of observations.

Overall, the results of the evaluation were very encouraging:

  • All user groups managed to make the offering to the Gods fairly fast.

  • All users tended to explore the world and their capabilities with enthusiasm.

  • Many users were impressed by the environment and they spontaneously made various positive comments, e.g.: “nice avatar clothes”; “nice buildings; are they exact copies?”; “it feels great that I can talk to an avatar in physical language”.

  • At the end, all groups orally repeated the process of making a libation to the Gods.

In addition, a total of 43 UX issues (Table 1) were identified during group interactions. Most issues may be characterized as minor or easy to correct, like for example that “some avatars provide incomplete information” (this was a design decision to promote exploratory learning), or that “too much information into avatar utterances, which disappear fast from chat window.” Others require further design and testing, like bugs.

Table 1. User experience issues, types and occurrences.
Full size table

6 Conclusions

We have presented a multi-agent platform based on virtual worlds for cultural heritage applications, as well as the design and user evaluation of an interactive scenario using this platform. The platform has been designed so as to be able to support a wide variety of environments and scenarios, given that the behavior of the agents and objects can be scripted and the dialogs can be adapted accordingly. Its operation does not have to be restricted to scenarios such as the one presented in this paper, where the users are participating in a pre-scribed story. It could also allow for a free exploration of the environment, where users can observe activities performed by NPCs and discuss with them to learn more about their life and customs. Alternatively, NPCs could assign quests to visitors and reward them with giving them improved clothes or access to extra content.

The preliminary evaluation results of the environment and scenario are encouraging, since that all users managed to timely finish the task, they had a positive experience, and they learned the key points of the ritual. The vast majority of identified UX issues were minor and will be further considered for improvements. We believe that the multi-user and collaborative nature of the tasks also had a positive impact of the experience, and thus it is important that the adoption of multi-user persistent virtual worlds as a platform for the dissemination of cultural heritage is further explored. Finally, digital characters that can interact with users and the environment seems to be an appropriate basis for dissemination of intangible heritage in interactive 3D environments.