Keywords

1 Introduction

Information and communication technologies (ICTs) offer powerful tools in the cultural heritage sector. Cultural organizations are increasingly embracing emerging digital technologies to add interactive elements to their exhibitions [10]. The value of digital tools for use and interpretation of heritage assets is now well established as is their value in the cultural heritage education [14, 15]. However, it is important to highlight that the use of technology in cultural heritage must be always at service of the cultural message; thus it is crucial to pay attention to the scientific data carried by the cultural assets [16]. When correctly implemented, ICTs may provide a wider access to the cultural heritage, not only from a spatial point of view, but also in terms of enlarging the potential audience. Therefore, ICTs may contribute to stimulating people to recognize the importance of the past and the history, encouraging appreciation of valuing the cultural heritage education and arts, key factors in development of knowledge society and creativity ability [20].

Technologies such as augmented realty, virtual reality and games, for their intrinsic visual and interactive nature, may be exceptionally suitable to enhance the enjoyment and understanding of cultural heritage and, consequently, they may be used as powerful learning tools. In particular, several studies have already showed the positive effects of gamification and serious games in learning [3, 5, 8, 9, 17, 19]. Gamification is the use of game design elements in non-gaming contexts to increase the user’s engagement and experience [6]. Games may encourage learning activities by enhancing engagement and by building challenges to achieve an intended learning objective. Games that have as primary goal education rather than entertainment are referred to as education-oriented games or more generally serious games [12].

The popularity of video games, especially among young people, makes them a powerful medium for educational purposes. Furthermore, the main strengths of serious games such as visual expression of information, interactivity, entertainment, and engagement make them compelling for the cultural heritage valorization, communication and user’s engagement [1]. Unlike other compelling multimedia technologies such as virtual reality, augmented reality, audio guide, which offer the opportunity to better understand and enjoy cultural assets, but can lack the active learning mechanism, computer games are able to provide entertainment and captivating experience. For this reason, the use of serious games in cultural sector stands out when the learning purpose is crucial [13].

In this work, we propose new tools to improve the interest of students in cultural heritage and enhance their learning process. We describe the design, development and use of a game, which aims to raise the engagement and learning process of primary students during educational workshops at the Caracciolo Castle in the town of Sammichele di Bari (Bari, Italy). Although it is mainly addressed to the workshop’s students, the exhibition is open to all visitors of the castle that can play and enjoy the activity. The game, thanks to a tangible user interface, combines manual activities and visual information in computer graphics. The system utilizes computer vision techniques and computer graphics. We provide a detailed description of both the design of the system and the technologies involved. Following, we describe the possible implications.

2 Learning About Local Heritage

2.1 The Main Project: Discovering the Territory of Old Peucetia

The project described in this paper concerns the use of a tangible user interface game to learn about a cultural heritage, specifically about the castle Caracciolo in Sammichele di Bari in Apulia (Italy). It is a part of a bigger project aiming to promote the local heritage of old Peucetia in Apulia and to encourage children and primary school students to learn about these heritage assets.

The territory of the old Peucetia is in the region around Bari in Apulia among the municipalities of Gioia del Colle, Acquaviva delle Fonti, Casammassima, Sammichele di Bari. The name “Peucetia” is derived from its ancient inhabitants, the Peuceti, a lapygian tribes that occupied the area before the Roman conquest (Fig. 1).

Fig. 1
figure 1

The area of old Peucetia

Full size image

The project involved a network of educational workshops and activities that have been organized in the historical buildings of towns of old Peucetia in order to let students of primary school discover the history and the environment of the region. The workshops rely on multimedia installations and interactive tools, which were designed, developed and installed by our team (Fig. 2). Through simulations and activities, students and general visitors of the exhibitions can immerse themselves in the ancient times, and discover the history of their region. The serious game described in this paper is the instrument in support of the educational workshop held in the castle of Sammichele.

Fig. 2
figure 2

Multimedia installations

Full size image

2.2 Building the Castle of Sammichele Di Bari

Construction of the castle “Caracciolo” in Sammichele di Bari (Fig. 3) started from the “Centurione Tower”, built at the end of the 15th century. The tower became important for the population as a small fortification in the territory. Around it, a small rural village developed, origin of Sammichele di Bari. Between the 16th and 18th century, the castle was subjected to several events, which determined the development of the castle. In the 16th century, it was owned by the Genoese banker Heronimo Centurione, in 1606 by the Jewish-Portuguese Michele Vaaz, who gave the name “San Michele” to the town, while in 1675 it was owned by the baron Antonio De Ponte. The dukes Caracciolo took possession of it in 1797 and then the restoration works were assigned to the architect Amenduni in the second half of the 19th century, who rebuilt the east facade covering it with bossage, widening the openings on the ground floor and changing the three windows of the first floor into neo-Gothic mullioned windows.

Fig. 3
figure 3

Caracciolo castle

Full size image

The educational workshop held inside the castle of Sammichele di Bari aims at letting the children discover its structure and its history, starting from an examination and construction of a model of the medieval castle. The workshop is supported by a serious game with a tangible user interface. In order to complete the game, it is necessary to physically solve a puzzle; each time the right element of the puzzle is inserted in the right spot, a part of the digital castle modelled in 3D takes shape on the screen. When a task is completed, the avatar of the castellan appears and interacts with the students to increase their engagement. The scene of a medieval village has been designed in order to raise the enchantment of the activity.

A camera located under the table on which the students introduce the items of the puzzle recognizes the elements inserted and confronts their position with the supposed one. If the spot is correct, the digital part of the game triggers the computer based feedback actions, which is the construction of the corresponding part of the castle and the virtual participation of the castellan.

In order to make the system works we have designed a tangible user interface based on a wooden box, which serves as the table of the game, into which 3d printed puzzle pieces can be inserted (Fig. 4).

Fig. 4
figure 4

Tangible user interface

Full size image

3 Developing the Game

The design and development of the game followed several steps. One part of the work was related to the scene design, another part was related the tangible user interface.

3.1 Scene Design

Like in any game, the goal of the game design is to give players an immersive experience. Immersion is the subjective suspension of disbelief that let players to be absorbed by the game theme. Higher level of immersion is related to higher level of enjoyment and engagement [4]. In our project the focus of the activity was the castle, thus, we needed to design a scene consistent with a medieval landscape, but also enchanting for the players of the game. At the same time, the digital reconstruction of the castle had to be factual with the real construction stages. Therefore, we focused on the authenticity of the castle’s digital model and on the charm of the global scene.

The modelling of the castle followed a specific workflow. First, we analyzed the source material such as historical literature and archived technical drawings; then, we realized a photographic campaign and a technical relief of the outdoor prospect of the castle. After recovering technical and historical documents about the castle, we worked on the building of the 3D data, proposing the castle to its early history.

The first step to build the model was to import the planimetry of the castle on the software platform (technical drawing), obtaining the orthographic views of the architecture. By using the software tool “poly-line” the base or the floor of the structure were built in 2D. The work continued modelling the perimetral walls the façade and the castle towers.

To facilitate the game dynamics, the modeled elements were not grouped in just one geometry but they were kept as separated elements. Completed the architecture 3D model, the geometries have been characterized in terms of materials and textures. Thanks to the dedicated photographic campaign resulting from the first step of the activity, we assigned for each surface a specific texture, returning how the castle should have been in the 17th century (Fig. 5).

Fig. 5
figure 5

3D model of the castle

Full size image

Once we modelled the real castle, we placed it in an environment that does not represent the real surroundings around Sammichele di Bari, but a fictional medieval ambiance to improve the enthralling aspect of the experience.

At the beginning of the game, the castle is not present in the scene; it takes shape piece by piece during the game progress until the castle is completed. Two different sceneries are loaded runtime according to the time of the day, one in the daytime (Fig. 6), and one at night (Fig. 7).

Fig. 6
figure 6

Daytime landscape with and without the castle

Full size image
Fig. 7
figure 7

Night landscape with and without the castle

Full size image

3.2 The Tangible User Interface

A Tangible User Interface relies on physical objects, instruments, and surfaces as physical interfaces to interact with digital information [18]. It couples physical representation with digital representations. In our work, we have built a box, which serves as the table for the game, where users need to insert the right element in the right spot in order to interact with the digital scene. The setup system is based on a wooden box with a camera and illumination source inside (Fig. 8).

Fig. 8
figure 8

Runtime system

Full size image

In this setup, the camera is located under the board in which users introduce the items of puzzle. It sends the video stream to the game application, which analyses the stream using computer vision techniques and recognizes the elements inserted. The interaction with the game occurs according to this analysis.

In order to make the recognition of the physical object possible, we placed fiducial markers under the puzzle pieces. Fiducial markers are images placed in the environment and detectable by a video camera by comparing their similarities with their pre-stored templates [7]. They are widely used in Augmented Reality applications to calculate their position and determine the camera pose, in order to mix virtual objects with real environment according the viewer perspective [2]. In the implementation of the computer vision algorithms, we used ARToolkit, an open source computer tracking library.Footnote 1

In the design of the tangible user interface we needed to take into account both the constrains related to the marker recognition and the ergonomics of the system. In order to contain all the pieces of the puzzle, the area of the board could not be smaller than 70 × 70 cm2, and, since both children and adults can play the game, we needed to put the board at a useable and conformable height for both. Thus, we built the wooden box of 70 × 70 × 85 cm3, and we placed the camera inside it. The ability to recognize the markers is affected by the position of the camera. A marker too far from the camera is unlikely to be detected. Hence, given the field of view of the camera, we inserted it at 75 cm from the top, the shortest distance able to frame all the board area (Fig. 9).

Fig. 9
figure 9

Box structure

Full size image

Another key factor affecting the detection of markers is their illumination. According to a study on the design and tracking of ARToolkit markers [11], the best illumination conditions are reached with a light intensity in the range of 55FC 2.8 KFC, and with a light-marker-camera angle of 90°, which we have achieved by inserting a led strip around the top part of the box (Fig. 10).

Fig. 10
figure 10

Box lighting

Full size image

Following the described specifications, we were able to build a reliable and affordable tangible user interface system (Fig. 11).

Fig. 11
figure 11

System at work

Full size image

4 Conclusion

Gamification and serious games offer significant new possibilities in cultural heritage education, encouraging learning activities by increasing engagement of students and by creating designed challenges. Using modern consumer hardware in combination with techniques derived from computer graphics, computer vision and human computer interaction broadens myriad of opportunities in interaction and appreciation of cultural assets.

In our project, we used computer vision and computer graphics techniques to build a tangible user interface game, which relies on gamification techniques as support for learning about cultural heritage. We focused on the technical and ergonomic aspects to achieve an affordable and reliable system.

Teachers remarked increased motivation and engagement of their students with the use of the game during the educational workshop. However further tests are needed to better understand what game mechanics should be implemented in order to achieve higher learning levels.

As other fields, ICTs are effective in cultural heritage education only when they are not used for their own sake, but keeping in mind the final aim of their use. Therefore, the design of serious games requires iterative collaboration of various experts with specific competences and skills.

Our aim was to design, build and test an affordable tangible user interface game to demonstrate the potential of serious game in cultural heritage education. Further research is needed to reach higher learning levels and find good practices to set as a standard for specific learning tasks.