Keywords

1 Introduction

Ecomuseum is a kind of museum in concept. The concept is to utilize real sites or objects of culture, history or nature as subjects for visiting. Because the subjects are not located in indoor venues, it is an important issue about how to process an appropriate exhibit interpretation.

This research proposes to utilize augmented reality (AR) technology and to study how to integrate appropriate interaction design can help get better experience in such ecomuseums. Augmented reality is a technology attaching computer generated images and information on the real world. It is different from virtual reality (VR) for the fully immersion in the virtual world. AR users can see the original real world, and also can see supporting information images or virtual objects, that help provide related information and interaction of the real world and objects. With a perspective of interaction design, this research considers the form of an interpretative game with location-based AR to combine the benefits of field interpretation and game tasks.

This research took Beitou area in Taipei city as the sample of ecomuseum, to develop the contents of the game. We developed a prototype of interpretative game for ecomuseums with location-based AR technology. We tested the prototypes on the real site with fewer participants. Guidelines are concluded to make the final model of the on-site interpretative game.

2 Literature Review

2.1 Interaction Design of Location-Based AR

Location-based service is a kind of information service systems combined with GPS and cellular communication technologies to provide digital map, navigation, location-based reminder, location-based personal AD, automatic weather forecast, etc. Location-based games are becoming a popular genre of mobile games with location-based service. For example, Pokémon GO is one of the most popular location-based game in recent years. Players could see virtual objects or roles near themselves on the game map by their real-world positions, and they could interact with such objects. In the stage of monster chasing, it is also possible to activate the AR effect to see the realistic scene with virtual monsters’ appearance at a real place.

Augmented Reality (AR) is a kind of technology with computer-generated image attached to the real world. It is very different from Virtual Reality (VR) that is fully emerged in the virtual world. The most scenes or backgrounds that AR users see are parts of the real physical world. In an AR environment, they can also see informative images or virtual objects providing assistive information and interactive mechanism of the real environment and objects [8]. Azuma [1] proposed AR as systems with the three characteristics: (1) Combines real and virtual, (2) Interactive in real time, (3) Registered in 3-D, to avoid limiting AR to specific technologies, such as head-mounted displays or other devices. He thought AR is a specific example of what Brooks [4] calls Intelligence Amplification that is defined as using the computer as a tool to make a task easier for a human to perform.

By the classification of Milgram & Kishino [9], the possibility of spatial com-position from real environment to virtual environment is named as Reality-Virtuality Continuum (see Fig. 1). In the continuum, the left is fully real environment and the right is fully virtual environment. Between the two ends there are kinds of Mixed Reality (MR) with combinations of different ratios of real and virtual environments. Real environment with some virtual objects is named as Augmented Reality (AR), and virtual environment with some real objects is named as Augmented Virtuality (AV).

Fig. 1.
figure 1

(Source: Milgram and Kishino 1994).

Reality-virtuality (RV) continuum

Full size image

The AR combined with location-based service is Location-based AR. It uses the user location and geographic information system to attach the virtual objects and information to the real environment. The location of the user can be measured with GPS, accelerometer, compass or other sensors. Because the virtual objects can be attached to the locations of real sites, it is easy to connect the real world and its related information with visual means. For the assistive information provided in real time, AR is a suitable technology for site interpretation or vehicle navigation.

Haahr [5] compared several kinds of location-based games, and found that some games like Pokémon GO emphasized the immersion of the challenges. It is very different from the presence of traditional cultural interpretation. The players may focus on the challenges of the game, but not on the experience of cultural resources. Therefore, Haahr [5] suggested more connections to physical space and more interactions of virtual objects related to the site in such cultural location-based games.

The interaction in AR is most about to interact with virtual objects to acquire information or action instruction, and to have fun in interaction itself. Virtual objects mostly show in the space, and can be triggered to activate reactive methods and feedbacks. The interactive mode of virtual objects in AR matches the microinteraction defined by Shaffer [11]. The structure of design and analysis can be classified as four parts including trigger, rules, feedback, and loops & modes. Trigger initiates the interaction; rules decide how to react; feedback provides perceptible clues that users can confirm; and loops & modes guide how to end the interaction and return to initial status and mode. The triggers include manual trigger and system trigger. Manual trigger happens by users’ intension. System trigger happens while a specific system event appears. Various system events can be used as system trigger, especially location-related ones, which are useful for this research.

2.2 Interpretation

Interpretation is multiple methods and media of demonstration and explanation for visitors to understand the themes and contents of exhibits. Tilden [14] provided a definition of interpretation as follows:

An educational activity which aims to reveal meanings and relationships through the use of original objects, by firsthand experience, and by illustrative media, rather than simply to communicate factual information (p. 33).

Beck & Cable [2] took interpretation as a procedure of transmitting information and inspiration, to promote the understanding, appreciation, and protection to cultural and natural heritage.

Experiential learning (or experience-based learning) is a way of learning through experience, especially learning by reflection in operation. Kolb [7] proposed the experiential learning theory that defines experience learning as the process of knowledge generation through the conversion of experience. Kolb’s experiential learning model (ELM) is divided into four stages, from individual concrete experience, reflective observation, abstract conceptualization, and finally to verification of the concept (active experimentation). Knowledge is obtained by summing up experience from actual situations, and promoted from verification and application.

Tilden [14] proposed the two concepts that the interpreter should think about. One is that the statement must go beyond the facts to inspire more important meanings behind the facts. The other is that the explanation must make full use of human curiosity to enrich and enhance human intelligence and mind. This means that interpretation can use people’s instinct of curiosity for novelty, to provide multiple experiential environments to trigger learning motivation, and to open up potentially important meanings.

The interpretative media is the medium that carries the content of the message. Sharpe [13] divides the interpretative media into two categories: 1. staff interpretation: the use of personnel to explain directly to tourists. And 2. non-staff interpretation: using a variety of facilities to explain the subject matter.

2.3 Game-Based Learning

Game-based learning (GBL) is a learning method that uses games to achieve specific learning purposes. The game-based learning using digital media is called digital game-based learning (DGBL). There are researches on the effectiveness of digital game-based learning [3, 10, 15].

It is difficult to determine whether the museum visitors have learned something. Regardless of whether visitors pay attention to and understands the theme information, the entertainment effect can still be achieved. Screven [12] proposed two modes of museum participation: passive participation and interactive participation. Passive participation plays only the role of the exhibit initiator. The visitors can see the dynamic demonstration, but has no choice. Interactive participation is to encourage visitors to find the answer from the exhibit to achieve the best cognitive learning effect before making a decision. Interactive participation is a better way of participation in education and entertainment.

About digital games on real site, Hwang, Tsai and Yang [6] discussed the location-based mobile learning system on the topic of context-aware ubiquitous learning environment. They suggested the learning system should be environment-aware, that is, the state of the environment can be detected before the system can perform learning activities. And the learning system should provide personalized support at appropriate paths, locations, and times that based on the learner’s personal, environmental factors and learning process.

3 Method

This research utilizes a qualitative method to collect and analyze data from the field test of the game prototype.

3.1 Prototype Design of Location-Based AR Game

From the history of Beitou area, we adapted the story of a Japanese, Hirada Gengo, the owner of the first hot spring hotel in Beitou. The setting is that four game players of a team play roles as brothers and sisters in the fiction story. They have to walk around to find clues in the Beitou hot spring area to re-construct the story of the friendship between their grandfather and Mr. Hirada Gengo. Nine sites were selected as game spots for them to visit. They can use their personal mobile phone to see and interact with AR virtual objects to get information of the sites and the tasks to do. Instructions will be given by the mobile phone to guide to complete their mission.

First, at the starting place they read a letter from their grandfather. In the letter, there are some important words missing like a puzzle and they are encourage to find clues in the Beitou hot spring area. A game map that show nine sites in the area is also attached in the letter. The team members play their roles as the Leader, the Navigator, the Recorder, and the Adventurer. Second, they go out to the hot spring area to visit the sites to find the clues. They can use their mobile phone to visit a web site that we provide at the starting place via QR code to search the virtual objects at the sites. The Navigator follows the guide in the web site to lead the players to assigned sites with the game map. When their approach a site, they will see two virtual objects appear. When the Adventurer interacts with the objects, information or a task will be showed. A task is a question that they must find the answer at the site or an action that has to be done at the site. After completing the task they can get a code. The Recorder have to input the result of the task to a LINE chatbot, and it will feed back the name of the following site to visit. Third, the players visits all nine sites to complete the tasks, and they will arrive at the ninth site, Tiangouan Historical site, the first hot spring hotel in Beitou. The players will find the final missing words to finish the mission, and the game ends here.

The interface of web AR included a cursor and two kinds of virtual objects. The cursor is set at the center of the screen. When a player moves his mobile phone, the object can be aimed and hovered with the cursor. One kind of virtual objects is for site information. Then the site information will be triggered to show a window that contain text and image about the site (see Fig. 2). One kind of virtual objects is for site task. Similarly, the site task will be triggered to show a window that contain text of a multiple-choice question or an action to perform (see Fig. 3).

Fig. 2.
figure 2

An image virtual object as a trigger of site information

Full size image
Fig. 3.
figure 3

A 3D star virtual object as a trigger of a site task

Full size image

3.2 Data Collection and Analysis

User Field Test.

The participants forms a team to play the prototype of the game at real site, around the Beitou hot spring area. It’s about one and half hours to complete the mission of the game.

Observation.

During the game playing, the researchers observe the discussions and actions of the participants. Especially mistakes and doubts should be noted carefully.

Interview: After game ends, the participants of a team are interviewed together. Questions are focused on the most impressive parts, understanding of the design, fun to play, learning, satisfaction, problems and suggestions, and any opinion or idea.

Questionnaire Test.

Pre-test of SUS (System Usability Scale) and IMI (Intrinsic Motivation Inventory) for the game usability are conducted. Just for test of filling questionnaires in future product use, not for formal quantitative analysis in this research.

Qualitative Analysis.

Qualitative data are collected and analyzed with a QDA (Qualitative Data Analysis) application. Coding and category are made to conclude the axial concepts and the structure of this research.

3.3 Test Site and Participants

This research takes Beitou area in northern Taiwan as the sample site of ecomuseum. In Beitou, many cultural and historic sites are very worth visiting. To develop the contents of the interpretative game, nine important sites near the hot spring area are selected to form an appropriate visiting path (see Fig. 4).

Four participants (three female and one male) were invited in this prototype test. Only one was familiar with Beitou, but not with hot spring area. All participants were college students.

Fig. 4.
figure 4

The game map that shows the Beitou hot spring area

Full size image

3.4 Devices and Technologies

Devices.

(a) Notebook computer: for programming and user interface design, (b) Mobile phone with GPS and Internet connection: for game playing to get interactive guides and tasks in location-based AR, (c) Camera: for site information design and prototype test observation.

Technologies.

(a) Virtual web host: a host service to provide interactive guides and tasks of the game, and web AR service platform, (b) Web AR: AR.js, a JavaScript framework for AR, integrated with Three.js, A-Frame.js, ARtoolkit tools, (c) Location-based Web AR: GeoAR.js, a JavaScript framework for Location-based Web AR, (d) LINE chatbot: a chatbot service operated by LINE. Much people in Taiwan own a LINE account, a text and media message service integrated with multi functions.

4 Results and Discussion

4.1 Results

Learnability. Experience and Pleasure to Find the Correct Answer at the Site. Players thought the experience is very impressive if they can find the correct answer from the clues at the site. For example, one question is “Why were the Tiger Windows of New Beitou Train Station added from three to four?” The players cannot answer the question from the appearance of the station. But when they entered the station to see the exhibition about the station history, they searched for related information in it and happily found the answer in a chart. They got a very delightful experience to know something through clue searching at the site.

How Tools Help Experience. It’s possible that new tools attract players to use for fresh experience. Thus, players found they can get more fun and deeper experience from the interaction with tasks and the various sites themselves. They could walk through the path and touch the real sites and objects physically. The design of tools needs to emphasize on the smoothness of use, but not on how to attract too much attention.

More Clear Understanding of the Sites and Space. Players found it’s easier to memorize the information of the sites naturally after playing the game. They could understand the spatial structure well among the sites. The Navigator (U2) learned the spatial knowledge with intense use of the map and continuous comparison between the map and the environment. It’s possible that situated learning made the players to learn the spatial knowledge with more connections of multi experience.

Scale of Interaction. Some players liked to solve the puzzles with standard solutions. But also, some players (like U3) liked to answer open questions. Open questions could lead to more thinking and physical actions to form more creative answers or works. Most players liked to have chance to observe and interact with the environment by themselves.

Satisfaction and Pleasure.

Team Roles and Tasks. The roles setting made the players more situated. They found they would like to focus on their parts in site tasks. They could help each other to complete the mission of the game. They had some responsibility to share information and thought without too much effort. The roles setting let them enjoy the pleasure of team cooperation.

Benefit for Learning Context. For casual visiting, a player (U3) thought the game maybe not so interesting or exciting for fun. But for learning context, like outdoor learning of school children, it could be very suitable to learn as in a museum.

Willing to Recommend Others to Play. After playing the game, the players had the willing to recommend others to play the game, especially for newcomers of Beitou area. With the help of the game playing, visitors should enjoy to understand the historical area much more.

Storytelling.

Connections among Tasks and the Place. A good and clear story can help build the connections among tasks and the place. The players liked the story behind the game. It could help the players situated in the historical context. They enjoyed to play roles to complete the mission to visit an old friend, given by the ancestor in the story.

Playing and the Experience Acquired in the Story. The game told the story by the arrangement of site visiting. The players thought they got the experience of engagement in the story. The fun and experience came from the storytelling of the game.

Story as Context. Some players noticed that the story is good to understand and to play the roles, but it could run through the game even more. Perhaps the introduction session and the user interface could be re-design to utilize more elements of the story to form a nostalgia mood.

Order of Tasks. The order of the game was arranged by a path around the hot spring area. Some played thought it was not so close to the storyline. It is possible that the story was told according the order of the sites, but it was not so compatible with a good story structure. For better experience, the story might need modification to get a better story structure.

User Interface.

Clear Presentation of Tasks. Some targets of the tasks were ambiguous. When the players go they t such tasks, they would use their imagination to guess how to process. Because such tasks had no standard answer, the players were not sure whether they had completed the tasks or not.

Need of Sites Information for Game Playing. The information provided from the virtual objects is some basic knowledge about the sites. It was not designed to play a necessary part in the tasks. Players found they can skip it to take the following new task more quickly. For the learning goal of ecomuseum interpretation, it is not an expected phenomenon. The information presentation could be designed to come up first when the objects are triggered. The information itself may also play a part in storytelling and task executions.

Gap Between Guide and Execution of Tasks. When the guides is not clear enough, the players continued to play it, but toward another direction the designers didn’t expect. It is a gap between designers’ and users’ mental models. We might need a mechanism to correct the mistake and let players back to the designed path.

Technology.

Precision of Positioning. Because of the poor GPS signal, the positioning of the location-based AR didn’t work well. We cannot count on the function fully to see the virtual objects on the precise positions of the sites. The virtual objects were put at right coordinates, but the position of the mobile phone might be set to wrong position. So a virtual object on the mobile phone’s screen could not been displayed well at the correct distance and direction. Our temporary solution was to expand the range of trigger to show the objects, although some objects still could not been shown correctly.

Power Consumption of Web AR. The web AR in our game worked well. But the web browser use the Internet and camera consumes much power. The battery would drain quickly. So the players were noticed not to use the web AR all the time. They could rely on the Navigator’s guide to approach the sites and then take out the mobile phone playing web AR to get the information and tasks.

4.2 Discussion

Orientation of Tasks.

The tasks design of the game could be a continuum between puzzle-based and experience-based. The original intension to introduce a game into ecomuseum interpretation is to get deeper experience by multi perception received in the tasks. But the tasks may not be interesting enough. Some players’ expectation about a game is to solve a puzzle or conquer a enemy to get clues or enter the next level. In this research we try to find a balance in the continuum. A background story was created to form the path to walk through. The task design is focused on how to get personal experience from puzzle-solving to satisfy the both needs.

Randomness and Customization of Game.

In this game prototype, we used only fixed sites, tasks and paths. It’s possible to create more sites and related information and tasks. We can try different combinations of the elements, or even the background story. Thus, the game can be played by same played again and again.

Natural Conversation.

Personality of chatbot. The dialogues of the chatbot were set as necessary information. One player suggested the dialogues could be more like human conversation. We think that even without advanced AI technology, the dialogues can be designed like ones spoken by a role in the story. Chatbot can perform better like a human.

Sole Player.

One player suggested that we should think about the condition of sole player without team members. It means the map, web AR, and dialogue window of chatbot need to be integrated in one system or user interface. It’s possible but due to the limitation of the screen size, the functions can only be displayed by different tab, mode, or overlay window.

5 Conclusion and Suggestions

5.1 Conclusions

Learnability.

Experience and pleasure to find the correct answer at the site is the goal and key feature of the interpretative game. In the test of the game prototype, it helped the players get rich experience and pleasure.

Satisfaction.

The roles setting made the players more situated and enjoy the pleasure of team cooperation. They had the willing to recommend others to play the game, especially for newcomers of Beitou area.

Storytelling.

A good and clear story can help build the connections among tasks and the place. The players thought they got the experience of engagement in the story. The fun and experience came from the storytelling of the game.

User Interface.

Is a clear presentation of tasks necessary or not for a game? We found some targets of the tasks were ambiguous. The players would use their imagination to guess how to process. It could result in unexpectable mistakes of site understanding.

Technology.

In the game prototype test, the positioning of the location-based AR didn’t work well for the GPS performance of the mobile phones. It cannot be guaranteed to see the virtual objects on the precise positions of the sites. Our temporary solution was to expand the range of trigger to show the objects.

5.2 Suggestions

One Object per Site.

For the learning goal of ecomuseum interpretation, the virtual objects should show information first then the tasks. By combining the two buttons of information and task, we need only one virtual object per site. The information presentation can be re-designed to come up when the objects are triggered.

Demo Playing as Introduction.

To prevent misunderstanding of the interface and tasks, a demo playing of operation and task execution would be necessary. It should be an important part of the game introduction.

Smaller Range for Better GPS Precision.

Consider use a smaller range for better GPS precision of mobile phone, especially for new ones.