Enhancing values through virtuality for intelligent artifacts that influence human attitude and behavior

Abstract

Embodied interaction technologies allow us to enhance physical artifacts surrounding us by adding an information layer to the artifacts. The information layer that we call virtual forms presents dynamically generated visual information representing virtual objects and creatures that influence human attitude and behavior. The focus of our research is to develop intelligent artifacts enhanced with virtual forms that influence human attitude and behavior. To suggest some ways to develop such artifacts that harmoniously integrate virtual forms into them, based on our experience with three case studies presented in the paper, we propose a value-based analysis framework, which allows us to discuss and consider some good-design implications for the design of the enhanced intelligent artifacts. We also present design implications to apply the value-based analysis framework to analyze and enhance one of intelligent artifact. Finally, our experience suggests that incorporating fictionality is a promising direction for the designing of intelligent artifacts with ideological messages intended to influence people’s attitude and behavior.

1 Introduction

Our daily life becomes increasingly virtual as surrounding artifacts become more intelligent [22, 24, 36, 43, 49, 65, 74]. Our definition of virtuality is something that does not really exist, but that affects our daily life as if it existed. Baudrillard explains our consumption behavior as consuming symbols associated with things, not the things themselves [3]. Because symbolization of things will be accelerated by embedding computers in our life by incorporating virtuality, our virtual consumption will progress rapidly. Currently, products’ quality is not the most important reason for many of us to buy the products. For example, new furniture and fashion goods attract us every year, but they do not become commodities sold at cheaper prices over time. Focusing on creating advances in customer value can make competition irrelevant by opening entirely new markets [35]. Additionally, a significant value offering for users distinguishes breakthrough products from their competitive followers [7]. In particular, prices for such high-value products are kept high by offering a sense of preciousness. Digital technologies have been effective in making digital objects common commodities and, consequently, lowering their prices. However, the same technologies are also effective for adding value to products and services by incorporating virtuality.

Embodied interaction technologies make it possible to enhance our physical artifacts with virtuality. The various displays and projectors already embedded into artifacts allow us to attach an information layer to them. The information layer shows dynamically generated visual expressions representing virtual objects and creatures that provoke a user to associate additional values with the artifacts and that enables him/her to consider the artifacts to be more attractive [49, 74]. Typically, projecting some information on a physical artifact or adding a display to show visual expressions offers computational visual forms on the existing artifacts. We call the information layer that incorporates virtuality into physical artifacts virtual forms. Technologies now have become mature enough to realize virtual forms. Sensors retrieve information about the real world, and virtual forms reflect what happens in the real world, offering more value to a user. For example, AwareMirror [21] adds an information layer to a traditional mirror. The information layer helps a user’s decision making or influences his/her attitude and behavior. It is also important to maintain the functionality of the existing artifacts even when virtual forms are introduced,. Using virtual forms is a promising way to enhance the artifacts that surround us and to make our daily life richer and more enjoyable because increasing daily pleasure is one of the most important social issues to be considered in our society.

Our focus is to design and analysis of intelligent artifacts augmented with virtual forms that influence human attitudes and behavior by incorporating virtuality in virtual forms. Some concepts from games have proven useful in designing for non-entertainment purposes and to alter human attitude and behavior. Serious games are games used effectively to increase specific domain knowledge and skills [54]. For example, defense, medical, scientific exploration, and emergency management are typical domains that widely adopt serious games. The effect of a serious game is to increase players’ intrinsic motivation and self-efficacy by increasing their knowledge and skills. Games for Change and Persuasive Games are intended to change people’s attitude and behavior.¹ Newsgame embeds ideological messages in video games, making people deeply consider essential, serious issues in our society [5]. Gamification recently became a popular concept for motivating daily and business activities [11]. The concept is to use game mechanisms to encourage human activities in daily life and business. These above games have been developed effective virtuality to influence people and we like to suggest some ways to develop intelligent artifacts that harmoniously integrate virtual forms into them by enhacing their values.

We call physical intelligent artifacts enhanced with virtual forms that influence human attitude and behavior and offer ludic interaction with people through virtual forms “DPHG artifacts,” where “DPHG” stands for “Digital-Physical Hybrid Gameful.” Mona Lisa Bookshelf [49], Augmented Calligraphy [74] and Sentient Cradle [36] are examples of DPHG artifacts. Mona Lisa Bookshelf reflects the current situation of a bookshelf as a Mona Lisa picture. Augmented Caligraphy adds playful visual and sound feedback to a paper for calligraphy. Sentient Cradle notifies a receipt of a phone call as the movement of a doll. Although DPHG artifacts offer a variety of promising directions, how to successfully and harmoniously design a virtualized real world with DPHG artifacts remains an important and challenging issue. As shown in [1, 14], the authors claim that a better analysis framework can be developed through experience-developing case studies. To suggest some clues for solving this problem, we describe three case studies and introduce a framework for the design and analysis of DPHG artifacts in this paper. We also present some useful design implications from our experiences with them that enhances their values through virtuality.

We present two issues for developing DPHG artifacts. The first issue is to propose a value-based analysis framework, an analysis tool for developing better DPHG artifacts. Using values has recently been recognized as an important design approach for developing desirable information services [6, 8, 19]. We extracted six values from our experience with developing three case studies and show evidence of the effectiveness of values from psychology, sociology, and design research. Values are especially useful when multiple persons analyze DPHG artifacts. As shown in the experiments presented in this paper, values can be considered to be frames for discussing and gathering opinions from different users; these opinions will be able to be effectively used to enhance DPHG artifacts to satisfy more users because different people with different personalities may have different preferences regarding the artifacts as shown in [57].

The second issue is adding fictionality to DPHG artifacts. Fictionality represents something that cannot exist in the real world but that enables virtual forms to express more intuitive meaning than can something that exists in the real world. Fictionality facilitates embedding ideological messages in virtual forms to influence human attitude and behavior [58, 60, 61] because more vivid and exaggerated expressions that are physically impossible can be used. From our experience with adding fictionality in our case study, we suggest that a concept named transmedia storytelling [10, 55] is a promising infrastructure for seamlessly integrating fictionality into DPHG artifacts. We also present some design implications for incorporating fictionality into the real world and how the proposed values are used for developing DPHG artifacts that influence human attitude and behavior and how ideological messages are included.

This paper is organized as follows. Section 2 presents several studies related to our proposals. Brief overviews of three case studies of DPHG artifacts—Augmented TCG, Augmented Go, and Virtual Aquarium— that incorporate virtual forms to enhance physical artifact are presented in Section 3. In Section 4, we describe the value-based analysis framework, which defines six values extracted from our experience with the case studies, and we show how these values are used in the three case studies. Section 5 presents how one of the case studies, Augmented TCG, is analyzed through the value-based analysis framework as an example to show how to apply the framework to analyze the values of DPHG artifacts. We present some issues to incorporate fictionality through transmedia storytelling in Section 6. Finally, Section 7 concludes the paper and suggests future directions.

2 Related work

Value-sensitive design (VSD) integrates ethics and design [19]. The design methodology emphasizes the values of direct and indirect stakeholders and accounts for human values throughout the design process; it is influenced by a participatory design experience. Worth-centered design (WCD) [8] claims that human computer interaction should incorporate the concept of values as a design goal. A priori usability evaluation and context fit cannot distinguish between tolerable design problems and problems that have a major impact. WCD moves the focus from context of use to the context of impact, meaning that understanding outcomes is more important than how to achieve the outcomes. Boztepe also proposes four values for designing products that are more attractive: utility value, social significance value, emotional value and spiritual value [6].

The values proposed in this paper are extracted from a semiotic point of view [38]. Adding values through virtual forms changes the meaning of intelligent artifacts to influence human attitude and behavior and increases human intrinsic motivation. Because the proposed values focus on changing the meaning of intelligent artifacts, working with a set of values is simpler than the methods of past approaches. This facilitates applying the proposed values to gathering various opinions from participants in the analysis of the intelligent artifacts and to use the opinions to enhance the products to satisfy more users.

There are several past approaches to developing tools to capture user experience. IDEO Method Cards [29] is a collection of 51 cards expressing various ways that design teams can understand the people who use products and services designed by them. They are used to make a number of different methods accessible to all members of a design team. They explain how and when the methods are best used and demonstrate how they have been applied to actual design projects. PLEX Cards [2] is a collection of 22 cards, plus two related idea generation techniques: PLEX Brainstorming and PLEX Scenario. The cards were created to communicate the 22 categories of the Playful Experiences (PLEX) framework to designers, researchers and other stakeholders who wish to design for playfulness. Each stakeholder interprets values from different points of view. The design with intent toolkit [42] helps a designer with designs intended to influence or result in specific user behavior. The toolkit contains 101 cards classified into eight categories: architecture, error proofing, interaction, ludic, perceptual, cognitive, Machiavellian, and security. Each category offers a useful design pattern to lead a designer to a better design. Because these tools offer additional visual and verbal information in each classified category, they are helpful for analyzing products by broadening horizons of participants who participate in the analysis. These above approaches suggest a promising direction to develop a tool for the proposed values to make the analysis of DPHG artifacts more effective,

Maslow claims that human motivation is based on a hierarchy of needs [44]. In Maslow’s hierarchy, basic needs are physiological needs such as for food, etc.. Other need categories include safety, attachment, esteem, cognitive and aesthetic. At the highest level, when all other needs are satisfied, we begin to satisfy self-actualization needs. Because a user imputes value to a product when he/she satisfies his/her needs, satisfying needs is closely related to defining values. Jordan proposes four pleasures: physio-pleasure, psycho-pleasure, socio-pleasure and ideo-pleasure [32]. Nakajima et al. proposes five incentives: physical, psychological, social, economic, and ideological [49]. Because these proposals offer classifications of human desires, they are useful in directly affecting human behavior. On the other hand, our approach focuses on designing DPHG artifacts that influence human attitude and behavior.

In [51], Pine et al. claim that what people actually desire is not products but the experience that products provide. Experience emerges from the product and the user’s interaction. A user’s activity involving a product engages the user’s experience with the product. Designing user experience links the viewpoint of usability with the notion of a user’s emotional and contextual needs [26, 75]. Traditionally, designing computer systems considers only quantitative metrics to improve the utility of products and services, but these studies in user experience show that capturing user experience offers more value in terms of products and services. User experience analysis can also be performed based on the value studies, but the value studies offer better tools for designing user experience.

Participatory design [16] is an approach originating in the Scandinavian design community. This design approach attempts to actively involve all stakeholders in the design process to ensure that the product design meets their needs. Identifying stakeholder values is essential in participatory design. A frame in social theory consists of a schema of interpretation, which is a collection of anecdotes and stereotypes [23]. People construct a set of mental filters through biological, emotional, economic and cultural influences. The choices they make are influenced by their creation of a frame. Framing can affect the outcome of a choice problem. The framing effect, a cognitive bias, implies that presenting the same option in different formats can alter a person’s decision [34]. Each person whose personality and culture are different from other persons may have different values. Participatory design and frame analysis are appropriate foundations for analyzing values for DPHG artifacts in the value-based analysis framework.

Persuasive technologies are information technologies that change human attitude and behavior [17]. These technologies are based on several psychology models, and several case studies have been developed. Associated user studies show that persuasive technologies are effective in altering a user’s attitude and behavior. For example, UbiFit Garden is a mobile application that changes a user’s physical activity by using positive feedback if the user’s behavior is desirable [9]. Nakajima et al. reported four case studies that adopt both positive and negative incentives to control a user’s behavior [49]. Appropriate feedback is chosen according to a user’s current situation, which is acquired by sensors attached to physical artifacts. Several problems with current persuasive technologies were recently presented [27]. This paper shows some insights into behavior science in cases when human computer interaction research develops services where persuasiveness is an important factor. Our proposed values can be used to analyze physical products that use persuasive technologies.

Gamification is “the use of game design elements in non-game contexts” [11] and starts to be used in various social media services and other information services. Most of current gamification-based services are based on adopting virtual rewards for some activity. For example, points for being a loyal customer, leaderboards to encourage competition, badges for visiting certain types of locations and achievements for reaching final goals. Most of current studies on gamification did not take into account the meaning of virtual rewards and concluded that the effect to introduce gamication is not significant [76]. However, the meaning of virtual rewards is essential whether players enjoy the rewards or not. Our approach can be used to enhance the values on the rewards, and expand the possibility of the current gamification researches.

Design activism [20] proposes several methodologies for changing the behavior of groups. A social media system is described that explicitly offers mission concepts to make people activist toward a sustainable society [59, 62]. The authors of [66] claim that incorporating both virtuality and fictionality are effective in gamifying human activities in social media. In particular, adding fictionality offers the possibility of increasing people’s self-efficacy by encouraging their activities. Virtual forms will be employable to enhance the design of social media through virtuality and fictionality. Adding a popular virtual character and item used in a fictional story is a useful approach to incorporating ideological messages in the real world without spending more time to understand the content in the fictional story [58]. The approach offers a new, easy way to incorporate fictionality in the real world. This approach can be adopted in the approach proposed in this paper.

Several techniques to analyze fictionality incorporated in the real world are proposed in [60]. Additionally, some design patterns are proposed to impose reality on the incorporated fictionality in the real world. [61] These results can be used to design fictionality incorporated into intelligent artifacts to complement the proposal presented in this paper. In [64], the authors claim that using multiple incentives is essential to motivate more people to participate in social media activities. Thus, adding multiple values to physical artifacts offers the same effect to satisfy more people.

3 Case studies of DPHG artifacts incorporating virtual forms

This section presents brief overviews of three case studies of DPHG artifacts that incorporate virtual forms to make them more attractive. These case studies have been developed by our project; we have extensive experience with them and conducted their user studies. In these DPHG artifacts, virtual forms are embedded in a different way. In addition, because each case study introduces virtuality from various aspects, they are appropriate for demonstrating extraction of values from our experience with DPHG artifacts.

3.1 Digital-physical hybrid artifacts: Augmented TCG

The trading card game (TCG) is a turn-based game employing paper cards and is typically played in a one-on-one manner. TCG combines the collectability of trading cards with strategic game play. A player chooses his/her favorite cards to construct his/her own original deck, which consists of multiple cards. Typically, a player purchases a starter set containing a playable deck of physical cards and a manual that includes an explanation of the rules and the mechanics of the game in an introductory fashion.

In TCG, each player puts some cards on his/her table to battle each other. One of the biggest problems faced by any new TCG player is the need for an opponent to truly engage in the game play because it is extremely unusual for any TCG to support a solitaire mode. Players usually begin playing with a friend at a particular location such as a hobby game store offering organized gaming opportunities with a tutorial component or via an online portal.

Computer-based TCGs are also becoming popular. In our project, we made a comparison between the real TCG² and its virtual counterpart on Nintendo DS. An important conclusion resulting from that comparison is that the computer-based TCG lacks many realities offered by the real TCG [57]. For example, the sense of physical paper cards is essential for many TCG players because building and completing collections of cards is a significant source of pleasure for them. Additionally, the computer-based TCG implies some communication limitations because it allows a player neither eye-to-eye contact nor the ability to look at or chat with the opponent.

As described above, although most current computer-based TCGs lack the realities of the real TCG, we claim that general available computing technologies may help recover these lost realities and encourage and attract players to enjoy the computer-based TCG in a way very similar to the real TCG. On the one hand, the physical tangibility of TCG cards is essential for maintaining the pleasure of TCG players. On the other hand, adding special effects through virtual forms might even increase the excitement and the enjoyment of the game beyond those of the real one.

Augmented TCG, a digital-physical hybrid artifact, enhances the remote trading card play of two persons [65]. As shown in Fig. 1, the two players are located in different places. Each player’s cards in his/her duel field on the table in front of him/her are captured by a camera and projected on the opponent player’s table. The approach can combine the advantages of the tangibility of physical TCG cards and the merits of virtuality incorporated by virtual forms. For example, it is easy to add special effects on the cards through virtuality, and this becomes big merits to introduce virtuality in the real world.

Fig. 1

Augmented TCG system

Each player is represented by the 3D model of a virtual character used in popular animations and games in Augmented TCG; this character is shown to the opponent player. In the current implementation, MikuMikuDance ³ is used to show the 3D models of the virtual characters. MikuMikuDance is free software for creating 3D movies by using virtual characters. The virtual character is controlled using MS Kinect, ⁴ and its movement is synchronized with the movement of the opponent player by inserting a model that uses OpenNI ⁵ in MikuMikuDance.

Furthermore, while playing the game, another virtual character, which has been depicted on one of the player’s cards in advance, appears on a small display near the player once that card is drawn out of the deck and supports and encourages him/her to win the game until the end of the game. The two players can communicate with each other via Skype if desired; thus, it is possible for them to introduce each other directly instead of using virtual characters.

Figure 2 shows the current prototype configuration of Augmented TCG. On a large display, which is the first virtual form of Augmented TCG, a virtual character, whose movement is synchronized with the movement of a person who imitates the opponent player, is shown. A camera is setup behind the small display near the participant and captures the image of the cards. The opponent player’s cards are projected on the table by a projector, which is the second virtual form. A small display is the third virtual form; it shows some special effects that help a player’s play decisions and that encourage the player. A typical special effect is to emphasize the power and elegance of a player’s favorite cards.

Fig. 2

Current prototype configuration

3.2 Playful augmented training systems: Augmented Go

Go is a traditional board game for two players. The goal is to occupy a larger area of the board than the other player. Black and white stones are used to control the territory; a board with a grid of 19 × 19 lines is used as the game field. The rules of Go are relatively simple, but the underlying strategies are extremely complex and rich. As in chess and Reversi, numerous strategies have been invented to reduce the complexity, but studying these strategies requires the player to actually understand the strategic concepts. Thus, it takes a long time for a beginner to play well with an experienced player and to feel pleasure during the play. Augmented Go, which is a playful augmented training systems, supports several gaming modes to play a game [31]. The goal is to offer useful information to beginners without extra interactions and intrusive devices, as shown in Fig. 3. A virtual form in Augmented Go is superimposed onto the real Go board. Proactive feedback information is offered visually by superimposing guidance information onto the Go board via projector. A web camera connected to a personal computer is used to detect the position of each Go stone.

Fig. 3

Augmented Go system

The DPHG artifact supports several gaming modes. As shown in Fig. 4a, players can interact with the DPHG artifact by placing Go stones on a menu that is projected onto a board. Here, we briefly explain some of the modes and how players interact with Augmented Go.

Fig. 4

Proactive information feedback in the Augmented Go system

Normal play mode

The normal play mode is the basic form of the Go augmentation. In this mode, two players play Go as usual, but useful information is projected on the board to help beginners recognize the situation and make better decisions. The rules of Go are simple, but the vast number of possible moves in each turn makes it hard for beginners to make decisions. Moreover, on the large 19 × 19 board, beginners tend to concentrate on localized fighting in a narrow region and lose the big picture. It is difficult to recognize invaded areas because an invasion process progresses gradually as new stones are put on the board. Recognizing the links between the Go stones is important to choosing good offense and defense strategies, but doing so requires some skill. Moreover, the normal play mode visualizes the strength of links between the Go stones. As shown in Fig. 4b, same-colored stones are connected with lines. If a dangerous situation occurs somewhere on the board, an alert appears warning the players to avoid losing the area. The sequence of stone moves is also recorded in the database, which facilitates replaying the game for self-training. Replaying allows us to review and analyze the play by later projecting the stones on the board.

Tsumego mode

Tsumego is a type of exercise where the player is given a game board situation. The aim is to find the best sequence of stone placements in a given board situation. In this mode, the positions of the stones are visualized on the board. Players can try different moves by placing stones on the board, with results and comments explaining important points displayed as visual feedback (Fig. 4c). The Tsumego mode prepares questions for a player with different skill levels, and the level of difficulty can be selected in the menu (Fig. 4d).

3.3 Persuasive ambient mirror: virtual aquarium

Virtual Aquarium is a persuasive ambient mirror that has the objective of improving users’ dental hygiene by promoting correct tooth brushing habits [49]. It is set up in the lavatory, where it turns a mirror into a simulated aquarium as shown in Fig. 5. A virtual form in Virtual Aquarium represents an aquarium located in the lavatory, and the form reflects a user’s tooth brushing behavior. Fish living in the aquarium are affected by the users’ tooth brushing activity. If users brush their teeth properly, the fish prosper and procreate. If not, the fish become unhealthy and may even perish.

Fig. 5

Virtual aquarium

In this DPHG artifact, we use a 3-axis accelerometer sensor that is attached to each toothbrush in a household. A user brushes his/her teeth in front of the virtual aquarium using the sensor-equipped toothbrush. Tooth brushing patterns are recognized by analyzing the acceleration data. The toothbrush is able to “observe” passively how the user brushes his/her teeth; this is the only interaction needed to use this DPHG artifact.

As shown in Fig. 6, when a user begins to brush his/her teeth, a scrubber inside the aquarium starts cleaning the algae off the aquarium’s wall. At the same time, a set of fish associated with the user starts moving in the aquarium in a playful manner. When the user has brushed his/her teeth for a sufficient period, the scrubber finishes cleaning and the fish dance becomes even more elegant. When the user finishes brushing, the fish end their dance and resume their normal activities. Both the activities of the fish and the movement of the scrubber are designed to give the user hints regarding the correct method of tooth brushing. However, if the user does not brush his/her teeth sufficiently, the aquarium becomes dirty and the fish in the aquarium become sick. The feedback information is returned immediately according to the movement of the user’s toothbrush.

Fig. 6

Proactive information feedback in virtual aquarium

The health of the fish is visibly affected by how clean the aquarium is. If the user neglects to brush his/her teeth properly, the health of the fish worsens. In contrast, faithful brushing may result in the fish laying eggs as shown in the right pictures of Fig. 6. At first, the eggs are not very likely to hatch. If the user continues to brush consistently for a number of days in a row, the incubation ratio increases. This way, the long-term accumulated feedback gives clues to the correct behavior and attempts to maintain motivation over a longer period.

When designing this DPHG artifact, we considered the association between a user’s healthy lifestyle and the cleanliness of the aquarium. Our design takes into account the fact that people usually feel empathy for virtual creatures [53]. In our daily life, a mirror reflects our appearance and allows us to know whether we look good or not, whether our makeup and clothes fit or not and so on; it has the power to make what is invisible about us visible. Virtual Aquarium is a new type of mirror that reflects a user’s current state, encourages him/her to change his/her behavior and motivates desirable lifestyle.

4 Value-based analysis framework for analyzing digital-physical hybrid gameful artifacts

In this section, we extract six values based on our experience with the design and use of the three case studies described in the previous section. The framework containing the values is called the value-based analysis framework. We mainly focus on the experience that provides us some understanding of how to integrate virtual forms into physical artifacts; we increase the artifacts’ values without losing the user’s reality, even when the artifacts incorporate virtuality. This section also presents some evidence from psychology, sociology and design studies to validate the effectiveness of the proposed values.

4.1 Semiotics of virtual forms

The meaning of the virtual forms should be easily understood by a user. Designing them is the process of defining the meaning of artifacts [38]. That meaning should be understood by a user with little additional training or information; i.e., the meaning should be defined in our daily culture and should not be ambiguous because ambiguity may lead a user to an unexpected interpretation of the virtual form [70]. When additional instructions on the usage of the virtual forms are needed, they should be sufficiently simple that the user can understand them completely with little effort. It is unwise to assume that the user will read a manual. One solution to this problem is to use metaphors. Understanding a metaphor relies on the user’s prior knowledge. If the user has been acquainted with similar information in the past, the user can learn the meaning of a new virtual form through usage of an appropriate metaphor. A metaphor does not require too much information to make a better decision. For example, as described above in Virtual Aquarium, the cleanliness of the aquarium is a metaphor for the cleanliness of the user’s teeth. Additionally, Augmented TCG’s special effects surrounding the battlefield become metaphors to show the will and strength of the virtual characters drawn on the trading cards. Another solution is to use affordance. In Augmented Go, a player chooses a command to the system by putting a Go stone on a circle projected on the Go board. In this case, accurately identifying commands is important because a user cannot distinguish misunderstanding of the offered affordance from an inability to recognize commands. The feedforward information offered to a user teaches how he/she should behave with artifacts [12, 42]. If the feedforward information can offer the correct meaning of the artifacts, then the user can use them correctly.

Users, however, sometimes misunderstand the meaning of the virtual forms, and this is one of the dangers of relying on metaphors and affordance. Users may find unintended meanings in a virtual form. For example, if a supposedly unattractive picture is used to discourage undesirable behavior, that picture may actually have the opposite effect on an avant-garde or ironic art consumer. The way a picture is understood by users strongly depends on the cultures and personalities of the users. It is not easy for a designer to attach a single meaning to a specific expression for all people. The interpretation of the expression could be left to the user. This open interpretation [69] allows the user to feel pleasure or positive surprise on one hand, but on the other hand, it is not easy to predict the effect of the interpretation by the user in a controllable way.

Presentation of the information according to a user’s current attitude and behavior is a key issue for representing the values described in this section. The information may appear in an ambient way. For example, as shown in this section, a metaphor is a useful tool for that purpose. Visual information representing a metaphor should be tangibly manipulated to present more detailed information [30]. One solution is to offer a visual expression that offers an affordance presenting more detailed information by opening the visual expression.⁶ After a user chooses a visual expression, detailed and concrete information appears on a visual form for the user to make a better decision. This style of information design is effective in showing information when designing DPHG artifacts for naturally integrating information into the artifacts. A skillful user chooses several necessary abstract pieces of information and opens the information to show information that is more detailed.

4.2 Six values extracted from the three case studies

The six values introduced in this section as shown in Fig. 7 are evolved from the pleasure framework proposed in [32] and the incentive framework presented in [49]. We enhanced these frameworks based on our experience with the case studies and from the semiotic aspect [38] to focus on the meaning of DPHG artifacts when virtuality is introduced.

Fig. 7

Six values

Empathic value

Using an empathetic form is an effective way to evoke a user’s emotions. Empathy is a strong social incentive to feel values on a virtual form. Empathy engages the user to feel close to the empathetic form. Virtual pets are a typical example of an empathic form; they are very popular in many online services. Social robot pets also make our daily life happier. The pets evoke the user’s empathetic emotion and encourage him/her to change undesirable behavior caused by negative emotions. The emotional impact is very effective in making the user keep his/her desirable habits. One interesting theoretical result is the media equation [53]. A user feels empathy for even non-living things such as a personal computer. This result shows that there is a possibility of using various expressions or products that do not represent living or animated characters. When the user considers that something expressed by information services has a personality, the user feels empathy. The product attachment theory [56] explains why people like customized things more than uncustomized things. If the personality fits the user’s personality or he/she feels altruism toward the personality, he/she feels a close relationship with customized things. Therefore, designing a good and appropriate personality is an important topic when designing DPHG artifacts. In Virtual Aquarium, fish in the virtual aquarium evoke emotion in the user. In this case, an important design issue is to synchronize the current situation of the virtual fish with the user’s tooth brushing practice. In Augmented TCG, the virtual character representing the opponent player has a strong impact on the player’s feelings during the game play. If a player has a good feeling about the virtual character, the player tends to enjoy the game play sincerely, even if he/she does not know the real opponent well.

Physical artifacts tend to be used for a longer time if the user feels empathy for them [56]. It is, however, not easy to offer empathetic experiences to users. One promising way to solve the problem is to make it possible for a user to customize his/her own experience. For example, decorating a mobile telephone is very typical for Japanese people, making it unique and differentiating it from others’ telephones. It is also very typical for people to customize their avatars in online games by changing their avatars’ accessories, clothes, shoes, and hairstyles, even if doing so involves paying extra money.

Persuasive value

An effective and easy way to alter a user’s attitude and behavior is to offer proper feedback information according to the user’s current situation [17]. When the user’s behavior is the desirable one, a positive expression is returned as feedback, but if he/she behaves in an undesired manner, a negative expression is returned as feedback. However, the feedback information needs to be synchronized with the users’ real world situation to make them feel a sense of reality. For example, in Virtual Aquarium, the cleanness of a user’s teeth is synchronized with the cleanliness of the virtual aquarium.

The transtheoretical model defines a five-stage process involving the progress to change a user’s undesirable behavior [52]. The model is useful to incorporate the persuasive value in DPHG artifacts and to construct a user’s intrinsic motivation by noticing the importance of the ideological value incorporated in DPHG artifacts. The returned feedback information needs to be changed according to the current stage of the user’s thinking, skills and knowledge. In earlier stages, the user prefers emotional reinforcement not to give up his/her current efforts. On the other hand, for the user who is in a nearly final stage, enough information to make a better decision through rational thinking is more suitable. Designing emotional incentives is an important aspect of offering persuasive value on a virtual form. Positive stimuli are effective in early stages, but in later stages, negative stimuli are desirable because thinking rationally is difficult when in a positive mood; users tend to think more rationally when they are in a negative mood [68].

Virtual Aquarium provides positive stimuli when the user’s current behavior is desirable, but a negative stimulus is returned when he/she behaves undesirably. One important finding is that a negative stimulus alone is not effective because the user becomes rational, and he/she considers the effectiveness of his/her behavior [49]. He/she needs enough information to think about the importance of the activity in a rational way. When rational decision-making is important, it is desirable not to evoke a user’s positive emotion too much. It may lead to heuristic thinking that leads to a wrong decision. Similarly, in Augmented Go, the projected information on the real Go board is useful to make a better decision, but it does not an offer emotional stimulus to encourage a beginner to play the game by encouraging a user’s rational thinking.

This fact is important when designing a DPHG artifact for a game. A game usually evokes a user’s deep emotion, but it may not be better to win the game without rational thinking.

A beginner may not have an interest in continuing to use DPHG artifacts over a period of time. In particular, if the activity to use the DPHG artifact requires additional effort, it is not easy to continue the activity until better habits are built because curiosity does not motivate as much as does the exertion of extra effort demotivates. Even when offering enough stimuli, a user may get bored after the same stimulus is received over time because the stimulus itself cannot be strengthened infinitely. This is one reason why most people quickly give up many interesting activities. Thus, it is important to offer extrinsic motivation like joy, comfort or reward in the early stages of an endeavor. For example, Virtual Aquarium offers a positive emotional stimulus to a user to continue to brush his/her teeth. Similarly, Augmented TCG offers special effects on the trading cards to encourage joyful play.

In the next stage, self-efficacy is the key to success. A user feels self-efficacy when he/she has enough confidence to continue the target activity. The confidence comes from the evidence that he/she has the ability to do the activity well; therefore, the user needs to be offered information showing such evidence. To move to a later stage, the user needs to perceive self-efficacy to increase the intrinsic motivation. However, because positive thinking is necessary to grow the user’s self-efficacy [18], it is essential not to offer too many negative stimuli to maintain a user’s positive thinking and to offer enough information to think rationally enough to notice the importance of the ideological messages incorporated in the DPHG artifacts.

Informative value

The informative value is useful for offering information to a user for him/her to make a better decision. When using the persuasive value, a user usually makes his/her choice unconsciously, but the informative value enables a user to feel his/her decision explicitly. If users believe that they make a decision by themselves, they usually do not try to change their decisions; this effect is useful for increasing their intrinsic motivation.

Proper decision-making support is very important [34, 72]; thus, the informative value should be incorporated in various future DPHG artifacts using virtual forms. In Augmented Go, extra information is projected on the physical GO board. The user can still use normal stones and a board without attaching any artificial objects such as visual tags. Additionally, the user does not need to be equipped with special devices such as a head-mounted display. Projecting information directly on the GO board is useful for avoiding the fragmentation of a user’s attention. While doing Tsumego, a user needs to look at a book. This forces users to look at the GO board and a book alternately, preventing them from concentrating on learning the strategies of the GO play.

For designing informative value, it is important to consider how much information is necessary to make better decisions [73]. If hidden information becomes explicitly visible for a user, his/her decision-making will become more rational, counteracting biases that may creep into his/her decision-making processes. For attaching informative value, how to offer the nudge, which becomes hints to make a user’s rational decisions, is an important design decision [72]. If proper information is not given, a user may become more confused; in such a case, making a rational decision is difficult.

For example, if there are too many choices, a user tends not to choose at all [67]. A large amount of information also requires heavy cognitive effort, and so it is important not to give too much information when asking for a decision. Additionally, too much information is neither effective nor helpful for assisting a user to think rationally. In some cases, heuristics are dangerous and lead to mistakes in decision-making [34]; the bias in heuristic thinking may cause the user to make a wrong decision. Heuristics, however, are necessary to make better decisions from among many choices within a reasonable time; the amount of information should be carefully designed for better decision making.

Smart artifacts [22, 36] have been developed to embed computing technologies into daily artifacts such as chairs and clothes. The key technical issue for developing this technology is the ability to sense immersively our daily environment by using multiple smart artifacts and to offer context-aware information to people [37]. The smart artifacts are deployed everywhere in our environment and connected by a network. Tangible bits [30] and slow technologies [24] are technologies that materialize information in the virtual world. In particular, slow technologies that offer information are naturally harmonized in the real world. These approaches are appropriate to offer informative values to the DPHG artifacts because the information is seamlessly integrated into intelligent artifacts.

Economic value

Not surprisingly, we also found that economic values are a powerful technique for motivating people to influence their attitude and behavior. An economic value is a tangible reward that users consider valuable, but it is not necessarily actual money or goods. In online games, millions of players work hard to obtain rare and valuable virtual goods, and these players even trade these goods for real money at a rate of three billion dollars per year [41].

Lendonvirta proposed three attributes that make virtual items valuable in a game [40]. The first attribute is a functional attribute consisting of two categories: performance and functionality. Performance is the skill to play a game well, and functionality of the equipment increases the possibility of winning the game. The second attribute is the hedonic attribute. This attribute consists of six categories: visual appearance and sounds, background fiction, provenance, customizability, cultural references, and branding. The hedonic attribute offers value to satisfy a user’s emotional desire. The third attribute is the social attribute. This attribute consists of one category: rarity. This value is strongly associated with the ability to distinguish a group of owners from non-owners. The above attributes are made effective by providing economic value to virtual forms by making the items with the attributes shown in the forms exchangeable with other people.

Adding economic values in our case studies is a very important issue. In Virtual Aquarium, we can buy fish and plants for the aquarium. If a user becomes sick, he/she may not brush his/her teeth properly; this would make the fish ill. The user may feel helplessness and hopelessness about using Virtual Aquarium if there is no way to solve such problems. If a user can use virtual currency and buy medicine for the fish, it will motivate him/her to continue using Virtual Aquarium. In Augmented Go, it is useful for a user to buy more Tsumego patterns to improve his/her skills or to buy new software to analyze his/her current play in detail and to advise him/her better ways to play in future Go games. In Augmented TCG, a player can buy a new trading card and strengthen his/her current card deck. It is also possible to buy a new virtual character and a new pattern to add special effects to the virtual trading cards.

There are different virtual currency systems in many online services. In each service, a different virtual currency is defined. For example, in [59], aging money that gradually reduces in value if a user keeps it in his/her wallet is adopted to encourage human activities in social media. One of the most interesting issues is the exchange of virtual and real currencies. The exchange rate between the virtual and real currencies may change according to the value of the virtual currency. This means that the value of the virtual currency decreases if trust in the virtual currency decreases [39]. We believe that economic value will change the understanding of and the attitude toward virtual and real currency and will be useful for the design of a money system that uses virtual currency in a more effective way.

Although economic value is a powerful tool for motivating desirable behavior, it may lead to unpredictable results if not used carefully [34]. One additional effect that could be utilized is reciprocity, or the desire to reciprocate gifts and favors received from others. Virtual gifts are frequently exchanged in online games, strengthening the relationships between users.

Aesthetic value

Aesthetics is an important concept for making artifacts more attractive. Aesthetics is a branch of philosophy dealing with the nature of art, beauty, and taste and with the creation and appreciation of beauty. For example, Japanese traditional folk craft represents the aesthetic value [25], and this value is always very important to improving our quality of life. In the proposed framework, our discussion is based on a pragmatic approach to aesthetics represented by Dewey [13]. Dewey insists that art and the aesthetic cannot be understood without full appreciation of their socio-historical aspects. For example, the chair is not aesthetic in itself but rather the aesthetic chair is a result of the socio-historical appreciation of the material, and the shapes. The meaning is socially constructed, thus the meaning is changed in the social-historical perspective. Our ability to engage in an aesthetic experience is based on our social context, manifested in a personal physical and intellectual experience. The approach is suitable for our conceptualization to discuss values by multiple participants.

In Virtual Aquarium, because an aquarium is installed in the lavatory at home, it should not spoil the appearance of the lavatory. In particular, if DPHG artifacts are used in our daily life, the aesthetic value becomes essential for acceptance of the artifacts in our daily environment. Virtual forms offer a new possibility for developing intelligent artifacts because their appearance can be dynamically changed according to the current situation. Future clothes will contain types of virtual forms and change their appearance [15]. Chalayan, ⁷ a fashion designer, has explored the issue of harmonizing art and fashion by embedding technologies, including retractable skirts, half-dome lighted hats and a literal showstopper of a dress. Bogost claims that procedural rhetoric offers a powerful persuasion effect [5]. Thus, the approach of embedding procedurality into the aesthetic value can also be used to increase the persuasive value rather than solely the aesthetic value. Dunne has developed aesthetic electrical products and has shown how technologies can enhance DPHG artifacts in the future [14].

From the point of view of interaction design, virtual forms will enhance the interactivity of physical intelligent artifacts. The aesthetic value also plays an important role when designing interaction [50]. In Augmented Go and Augmented TCG, the interaction between human and virtual forms needs to be playful to enhance the player’s experience. The playful aspects are strongly related to the aesthetic interaction offered in the DPHG artifacts. The tangible aspects of the DPHG artifacts are also important to increasing the aesthetic value in the interaction between humans and the DPHG artifacts because people usually impute more values to authentic artifacts than to virtual artifacts. For example, in Augmented Go, the sound when putting a Go stone on the real Go board enhances its aesthetic interaction. Additionally, in Augmented TCG, the tangibility of trading cards increases a desire to collect cards because a sense of the ownership of the cards encourages people to preserve the cards more carefully [57]. This aspect increases the aesthetic interaction with the trading cards.

Ideological value

What is here referred to as an ideological value is the notion of influencing users’ behaviors through influencing their attitudes and values; in other words, educating the users on a deeper level. Attitudes and values influence users’ behavior in the long term. The ideological value makes it possible to motivate the user by himself/herself. A user with higher ideological value has a belief called self-efficacy that makes him/her believe that he is able to achieve his/her goal. In our current case studies, we choose simple metaphors that could easily be understood by the user, but the metaphors have a shallow effect on the understanding of the importance of maintaining a desirable lifestyle. The ideological value is brought about by intellectual stimulation. For maintaining desirable behavior, it is important that the user is aware of the importance of the desirable behavior. The association between the effect of desirable behavior and the virtual form offered to the user as feedback is effective intellectual stimulation.

The virtual form that has an ideological value may also include a user’s dreams or expectations for the future. In particular, an art form is a useful style to express human’s hopes or expectations in an ambient way. A virtual character may speak some words that remind a player of his/her future hopes or expectations. This may be useful to help a player mature through the game play. Additionally, virtual forms can be drawn surrounding virtual trading cards or a virtual character, and they may include information about the importance of recent serious social problems such as sustainability and human wellbeing. This makes it possible to learn about these important issues during game play by incorporating these social problems in the game design. Additionally, a special effect in Augmented TCG can represent the wish of a character in a trading card; this effect can heavily influence a player’s current motivation. Using an art form is a useful tool to give meaning to virtual forms. We believe that artistic ways of thinking will help create digital-physical hybrid designs that offer more stimulating experiences while prompting consideration of the importance of a desirable future. For example, contemporary conceptual art uses complex metaphors to provoke deep reflections on issues such as sustainability and peace in the world.

The ideological messages can be used to alter human attitude and behavior [59, 62]. However, only incorporating the ideological messages does not effectively influence people. In particular, to influence people to tackle serious social problems, fictional stories are incorporated into the real world. Many Japanese fictional animation and game stories typically contain serious ideological messages that make our daily life more desirable. It is promising to use these stories in non entertaiment services or in intelligent artifacts, but there are a few studies to use them to enhance the future possibility of our daily life. Additionally, a user’s intrinsic motivation should be guided toward his/her attitude and behavior changes.

4.3 Proposed values and participatory design

Each stakeholder defines his/her own frame to feel values about products and services based on his/her requirements. This is why participatory design [16] is important for capturing different stakeholders’ requirements. Additionally, each user has a different personality; he/she defines his/her frame based on that personality. As shown in [56], a user who has a different personality usually chooses a different favorite value to make himself/herself experience products and services more empathetically. Thus, values discussed in this paper are useful for defining a frame to characterize activities with DPHG artifacts. A user can recognize how he/she feels each of the proposed values depending on the current frame associated with a DPHG artifact. The values could be changed by incrementally adding virtual forms to some DPHG artifacts. Thus, a participatory design approach to dynamically adding and changing virtual forms on DPHG artifacts based on each user’s personality or each stakeholder’s requirements offers the possibility of assigning various values to products and services in a more systematic way without heavily modifying the infrastructure. However, the possibility is increased that the products and services are satisfy many people with different personalities.

5 Improving Augmented TCG with the value-based analysis framework

In this section, we present an improvement of Augmented TCG based on the value-based analysis framework. The six values can be used to analyze the pitfalls of the current design and to suggest improvements in the design for satisfying more people. One important advantage of using the value-based analysis framework is to offer different frames to analyze the current design based on the proposed six values. We can collect a variety of opinions from the analysis and the use of them to expand the current design space. This section shows an example of applying the proposed framework for analyzing the possible design space of DPHG artifatcs.

In the current version of Augmented TCG, we adopt the Yu-Gi-Oh! TCG because it is popular with many young Japanese adults. Additionally, it is easy to cover and exploit all values because the Yu-Gi-Oh! TCG offers various sources of pleasure for people who have different preferences [57]. Thus, this is suitable for discussing how values affect players’ behavior. The Yu-Gi-Oh! TCG is a trading card game based on the Duel Monsters game portrayed in the popular Yu-Gi-Oh! comic. The battle between the players is called duel, and a table to put a player’s cards in front of the player is called the duel field. Yu-Gi-Oh! cards are categorized into three types: Monster, Spell and Trap cards. A Yu-Gi-Oh! TCG player structures his/her own original deck by selecting his/her favorite cards from the several thousand Yu-Gi-Oh! cards currently available. This leads to each user having his/her own unique and original deck that reflects his/her own personality and taste. In Japan, trading card games such as the Yu-Gi-Oh! TCG are very popular among not only children but also young adults.

The Yu-Gi-Oh! TCG has a number of sources of pleasure in addition to playing the game, including completing collections of cards, structuring decks, communicating with other players, trading, battling and forming associations with the Yu-Gi-Oh! television animations and comics.⁸ In the Yu-Gi-Oh! animation story, Yugi and Kaiba are the main heroes and opponent players. Yugi is always surrounded by many friends, and his success is the result of his strong bonds with his friends, who love the trading card game. Kaiba is a lonelier hero who always seeks strength in the game but does not accept help from others, even in a critical situation. In the story, however, he eventually learns the importance of friendship. Most young boys want to follow one of these two characters because of their typical, attractive and ideal personalities. As described above, the Yu-Gi-Oh! TCG also offers TV animations showing some ideological messages such as friendship and justice; this is a good case study to discuss human attitude and behavior change through the ideological value.

In this user study, we recruited five male and one female participant for our experiments. They all performed the duels in the experiments against one of the authors of this paper, who has a deep knowledge of the TCG and could lead and control the experiment so that all the participants played the game under the same conditions. The participants had more than 3 years’ experience with the Yu-Gi-Oh! TCG, and they knew the characters in the animation stories very well. They were 21–22 year old university students. The generation of the participants is especially familiar with Yu-Gi-Oh! TCG. Additionally, most active players are male players. Therefore, the selection of participants is reasonable for producing useful insights, and their opinions show how our approach is promising from an expert’s point of view. Before the experiments, players could not talk to each other. None of them knew about Augmented TCG. Additionally, they were told how the rules of the game were simplified right before the experiment. During the experiments, each participant plays a different duel against an expert player who is one of the authors, has more than 10 years’ experience with the Yu-Gi-Oh! TCG, and has several thousand Yu-Gi-Oh! trading cards for analyzing their respective values. We observed the participant’s duels and conducted interviews with him/her after the duels based on the contextual inquiry method [4]. All of the experiments were recorded, and all dialogs in the experiment were transcribed to facilitate analysis of the dialogs.

For the experiments, as described in this paper, the rules of the game were simplified to make the duels shorter, and special predefined decks of cards were prepared to facilitate the analysis of different values. The decks were prepared by the expert player who later played a duel against each participant. The expert is also familiar with the TCG animation story, and knows well how each character structures the deck and uses the cards in the animation. Therefore, for each possible virtual character to be chosen by a participant to represent him/her in the game, a suitable deck consistent with the animation story situation was prepared.

Empathetic value

In the experiment, we use a popular empathetic virtual character as an opponent player. As described in the original design, the behavior of the virtual character is synchronized with the opponent player. The virtual character representing a player provokes his/her empathy and motivates him/her to play and enjoy the game more if that is one of the characters used in his/her favorite games or animations. Currently, Link from The Legend of Zelda ⁹ is chosen as a favorite character for the players. Link is the main character in the Legend of Zelda. He is a chosen man to save his world and bravely stand against a lot of challenges. His strength, cool and bravery are primary factors of his popularity. The reason to choose Link as the character in our experiment is that we would like to investigate how a popular and heroic character from another unrelated-to-TCG story affects the thinking of a player. In the experiment, all the participants knew Link well; thus, it is easy to discuss how players’ preference for a character affects players’ behavior. In particular, for some young players in Japan, favorite virtual characters are like close friends, and so they like to play with the character for a long time.

The results of an experiment to evaluate the improvement were completely different depending on whether the participants liked the character or not. If the participants were not interested in Link, they usually did not care about the presence of Link, but if Link was their favorite character, then they found playing the game against Link more enjoyable. One of the male participants also told us, “If the character is a pretty girl, I may be more excited to play the game.” Additionally, a female participant told us, “I feel that Link is my boyfriend, so playing against him increases my pleasure and positivity.” Most of participants like to play against Link, but it is desirable to choose other favorite popular virtual characters for themselves to encourage their motivation to play the game.

Persuasive value

In the experiment, a small display shows a virtual character that is illustrated on one of the cards in the player’s deck and that encourages the player to win according to the duel’s current situation, as shown in Fig. 8. In TCGs, trading cards are collected by each player by expending significant effort. Therefore, the player feels that his/her cards are very precious, and empathy with the characters depicted on the cards is easily initiated. Therefore, encouragement by such a character becomes a strong incentive for a player to win the game.

Fig. 8

Encouraging a player by a virtual character

We have designed a new TCG card that depicts Dead Master from Black★Rock Shooter ¹⁰ as a character for this experiment, and the card is included in the deck of a participant. We feel that this character does not conflict with or violate the atmosphere of Yu-Gi-Oh!. We choose the character because we need a character that can be easily identified by all participants, and the character has some features that clearly distinguish him from other characters. The use of Dead Master investigates the effect of the association between a card and the character to encourage players. Black★Rock Shooter has two worlds. Dead Master is an enemy of Black★Rock Shooter in another dimension world, but in the daily world the two are very close friends. This becomes a persuasive message conveying the meaning that players need to keep and develop their friendship even if they fight seriously in a game.

In the experiment, a special deck was structured in advance for each participant to control carefully the situation of his/her duels. The deck’s contents depended on the character with which the participant chose to play. Then, in the duel, the participant always removed the card depicting Dead Master from the deck at the beginning of the game. Once that card had been drawn out, a small display next to the player displayed Dead Master until the end of the duel. Dead Master supported and encouraged the player during the game by using encouraging body gestures.

After the experiment, one of the participants said, “It is desirable that the card depicting Dead Master does not lose to the attack of the opponent player.” However, another player who was not interested in the character told us, “It is more enjoyable if the participant’s favorite character encourages him.” One of the other participants said, “I feel that the character does not encourage me enough using only gestures. It is better that the character talks or advises me.” He also told us, “It is desirable that the character behaves like a cheerleader.” Dead Master is a serious character, and so if that character behaves like a cheerleader, some players who know the animation story of Black★Rock Shooter may feel the unreality due to the loss of consistency with the story. Additionally, another participant told us, “The encouragement should be like the one in the animation story.” Most participants said, “The presence of the character increases the pleasure, but it is hard to consider winning the game just from that encouragement.” The participants’ comments showed that they were quite aware that the character depicted on one of their cards appeared on the small display without them being informed in advance about this feature of the system, but the character’s encouragement needs to be stronger and to be changed according to the situation in each participant’s play.

Informative value

In the improvement, detailed information about the card that the opponent player currently holds in his/her hand is shown on a small display next to a player, as shown in Fig. 9. Information about the strength level shown in the card is hard to see during the game, and so showing detailed information about the cards near a player is useful to support better decision making and strategy choices by the player. The informative value also encourages a player not to give up the game. In a typical setting of a duel, it might be hard to clearly see and understand the characters on cards that are placed in a duel field and thus more difficult for a player to make a correct decision. In particular, if the opponent player’s cards are projected on the table, the low resolution of the projected cards becomes a serious issue. That is why, in our setup, we show the card drawn by the opponent on a small display near a player. One participant said, “If more hints to choose a card in my deck are shown, it is helpful to make a better decision.” Another participant told us, “If the textual information in a card is represented as visual information, a player’s cognitive overload is decreased and [this] makes it possible to make a better decision.” Additonally, one of participants said “If the card shown in a small display expresses special effects, the effects make me more exiting.”. These hints or effects are usually adopted in computer-based Yu-Gi-Oh! TCG that uses virtual cards. The approach can be implemented in the current version of Augmented TCG even if physical paper cards are used.

Fig. 9

Showing a card in a small display

From the experiment, we found that the usefulness of the approach depends on the players. Players who usually want to win a dual and know the trading cards very well can recall the detailed information of the cards merely by looking at the image of the trading cards. For these players, special effects or hints to teach them how to duel better are more appropriate. On the other hand, players who know only their own cards well but do not know other players’ cards feel that showing the projected cards on the small display is very useful for them to play their duel better. Therefore, the offered information as the informative value needs to be customized according to players’ personalities and situations for increasing a player’s activities in his/her play.

Aesthetic value

This value is already considered in the original design. Here, we discuss how the original design satisfies the values. Collecting cards and constructing decks are important elements that increase the pleasure of playing TCGs for a TCG player [57]. From the interviews with the participants, we found that players usually have two main motivations to collect cards. The first motivation is the will to collect beautiful and rare cards, and the second motivation is to collect cards that are important for the player’s strategy to play the game. In Japan, there are still many types of trading cards, and the rare cards are sold at online auctions for high prices. Some trading cards depict characters from animation stories. In this case, the same character is depicted on several different cards, and the aesthetics of the cards are important for collecting the cards; thus, very serious players may collect all the cards depicting their favorite characters.

The natural interaction is also related to aesthetic value. The aesthetic interaction can be achieved when a player feels the interaction is natural [50]. They did not feel a sense of incongruity regarding the projection of the opponent player’s trading cards, and extra instructions are not necessary for using Augmented TCG. They also enjoyed playing against a virtual character without feeling a sense of incongruity.

From the interviews with the participants, we found that using virtual cards represented in a computer version of TCG such as the one on the Nintendo DS significantly degrades the pleasure a player experiences when playing a game. The above discussion shows that the tangibility of trading cards and the augmented reality approach in the current version of Augmented TCG contributes to maintaining the aesthetic value and increasing a player’s motivation in the game.

Economic value

In Augmented TCG, a player can use the physical paper cards in his/her TCG play. On the other hand, in the online version of TCGs, cards are digitally represented in a virtual world. The player cannot touch the cards in the virtual world directly, and so he/she does not feel a strong sense of ownership of the cards. Using the physical paper cards is essential for increasing the sense of ownership and thus increasing the motivation to collect more favored cards and enjoy the game more. In particular, the rarity of the cards becomes a strong incentive to collect and own the favored cards. Rarity is an important economic value to make users feel empathy for virtual items [40]. The value makes it meaningful to collect trading cards because the owner of the rare card has a feeling that the card is unique. This means that the economic value increases the motivation of a player because he/she believe that his/her cards are valuable.

From the interviews with the participants, we found that two issues are important in terms of the economic value. The first issue is that selling new trading cards is essential to encourage the collection of more cards. If there are no new cards, a player may forget to interact with his/her own cards. This finally would make him/her bored with collecting cards and playing the TCG game. The second issue is the tangibility of the trading cards. It is not easy to feel a sense of ownership for digital cards because bits cannot presently replace atoms. The reality of digital items is an important aspect of increasing the economic value of virtual forms. The above discussion shows that the original design to emphasize the tangibility of trading cards contributes to maintaining the economic value.

Ideological value

In the improved version of Augmented TCG, a player can also choose one of two virtual characters: Yugi or Kaiba. As described before, these characters represent some ideological concepts such as friendship and justice in their background story. Therefore, playing against Yugi and Kaiba reminds players of the importance of such ideological factors because the players know the stories behind the characters. The extrinsic motivation established by the other values encourages enjoying a game, and the ideological value teaches a player the importance of friendship for really enjoying the game. The value leads to self-efficacy, the improvement of a player’s gaming skills with his/her friends’ cooperation and support.

After the play, we interviewed the participants about their impressions of the virtual character representing their opponent. One of them said, “I could feel I am playing against Yugi, but the Yugi used in the experiment does not offer enough reality.” The movement of the character was sometimes not like the real movement of Yugi as in the animation story. He also said, “I will definitely enjoy the game against Yugi more and would like to win the game if the movement is more realistic.” Another participant said “The face expression of the character is poor and it is a very important issue when playing a game against a real person.” Additionally, one of the players told us, “The voice should be the same as the actor’s voice of the character in the animation story.” If the opponent player was a female, some participants felt strange because both Yugi and Kaiba were male characters. These comments suggest that the reality is an important metric to incoporate the story in DPHG artifacts.

All the participants in the experiment could easily recall the Yu-Gi-Oh! animation story during their play. The Yu-Gi-Oh! animation story contains some ideological messages as shown above, but the current story may not be powerful enough to remind players of its ideological ideas during their play because the growth of the main hero due to the competition and cooperation among friends is rather implicit in the story. The story is, however, able to sufficiently increase positive thinking when playing the game and to increase self-efficacy enough to win the game. We believe that the effect is valuable to influence human attitude and behavior. In our daily life, people may not feel enough self-efficacy to change undesirable attitudes and behavior because our daily life becomes more and more complicated, and we do not have enough time to consider the importance of desirable behavior. We also need to consider how the representation form of the stories affects the conveying of the ideological ideas in the near future. In Japan, the same story is represented in different forms such as animation, comics, games, and novels. These aspects of the current popular fictional stories suggest a new way to incorporate fictionality in the real world as described in the next section.

6 Incorporating fictionality through transmedia storytelling

As described above, the Yu-Gi-Oh! TCG has a close relationship with the animation story. This offers an interesting discussion regarding use of the proposed values. From our experience with the experiment, we suggest that incorporating fictionality into the real world through transmedia storytelling is a promising approach.

6.1 Fictional stories and ideological value

The story already plays an important role in product advertisements because it increases the appeal of the target product [45]. Using fictionality to incorporate stories makes our experience richer because the stories can translate the meaning of DPHG artifacts more easily. Fictional stories are particularly useful tools for enhancing our daily experience to increase our buying impulse [61]. Fictional stories can represent stories that do not exist in the real world or stories from the future. The stories can flexibly offer us a broad range of information using non-existent artifacts such as magic or alternate history. It is easy to embed ideological messages in these stories and make it possible for them to teach about various social issues. They are also useful for teaching how to use various daily artifacts and to encourage us to acquire a desirable habit.

We incorporate virtual characters appearing in animation and game stories into Augmented TCG to increase values that a player feels. Our experience with incorporating fictionality in Augmented TCG shows a positive effect on a player because the fictional story used to improve Augmented TCG defines the meaning of values added to virtuality introduced by virtual forms. The fictional stories contain ideological messages that make us aware of important social issues in daily, real life [60, 63]. The effect offers an interesting future possibility to enhance DPHG artifacts.

6.2 Incorporating fictionality through transmedia storytelling

Transmedia storytelling is the technique of telling a single story or story experience across multiple platforms and formats using current digital technologies [10, 55]. It allows fictional stories to be fragmented into several different media distributed in the real world. In particular, virtual forms are considered a desirable infrastructure to incorporate fictional stories represented as transmedia storytelling into the real world. Alternate Reality Games (ARG) are becoming popular to represent transmedia storytelling where stories are delivered across multiple media [46, 71]. Enhancing games played in the real world such as TCG with fictional stories is a promising direction for designing a new form of transmedia.

One of important insight is that there is a possibility for using a virtual character as a metaphor that recalls the story of the character in the player’s mind when the player is gaming. In this way, the story may convey a leitmotif containing ideological concepts such as the importance of friendship, honesty, justice, thoughtfulness and so on [58, 60]. This approach would have the power to alter the player’s attitude and behavior. When playing with a virtual character from animations and a game story, the player also tries to mimic the character’s behavior in the animation story. This can be a very useful and successful approach to teach players how to improve their gaming skills. If players follow the skillful character’s way of playing in the story, then they can learn new skills and techniques from that character’s experience in the animation.

In the current Augmented TCG, the Yu-Gi-Oh! animation story is incorporated, but the animation story is not explicitly shown during the play. A player needs to recall for himself/herself the story during his/her play. Tighter integration of the game play and the animation story offers the possibility of designing new transmedia storytelling. For example, as shown in the experience on the persuasive value, the current approach is weak in encouraging a player. He or she needs a reason to play a duel. Additionally, as shown in the experiment on the ideological value, a player needs more information to be reminded of the embedded ideological messages. Virtual forms enables presenting a part of a story inside as transmedia storytelling and give information during play to help a player understand well the persuasive value and ideological value embedded in Augmented TCG. Additionally, a virtual character in a fictional world appears in the real world. In the near future, 3D holographic displays may realize the approach in a better way.¹¹ Thus, tight integration between a fictional story and the real world becomes possible by showing a part of a story in a small display near a player as the informative value in Augmented TCG. This approach makes a stronger association between a story in the fictional world and the real world through a virtual character than does the current approach, and the boundary between the two worlds becomes more blurred.

In video games, players enjoy fictional worlds through real world interaction [33]. Game studies scholars introduced the term magic circle, where a player plays a game inside a magic circle that is not recognized in the real world.¹² However, incorporating fictional worlds in the real world needs to be indistinct both inside and outside of the magic circle. Virtual forms will be interfaces between real and fictional worlds, and we believe that transmedia storytelling is a promising way to design those interfaces. Additionally, several techniques for designing Pervasive Games can be used to incorporate transmedia storytelling in the real world by blurring the boundaries between real and fictional worlds in terms of time, space and social [47]. However, if the boundary between the real world and fictional world blurs, rules in the fictional world should be obeyed as are rules in the real world; this may significantly decrease pleasure in the fictional world. For example, typical ethical constraints in the real world may need to be satisfied even in the fictional world.

6.3 Achieving reality when incorporating fictional stories

Considering how to obtain and maintain the reality of DPHG artifacts is a very important issue. Even if fictional stories are incorporated, the virtual forms should convey their realities to users. In particular, if a fictional story is embedded in DPHG artifacts, the reality in the story shown on the virtual forms is essential. Several design patterns have been proposed to discuss the reality in the incorporated fictionality [61]. If a user is involved in a fictional story that uses DPHG artifacts, his/her belief that a part of the story exists in the real world is essential. In particular, if a typical artifact that is used in the story appears in the real world and the user is reminded of the story by using the artifacts, the artifact is a promising source of media for enjoying the story. For example, in live action role-playing [47], a player plays a specific role from a fictional story. He or she wears special clothes to represent the role and uses physical artifacts that the person in the role typically uses. DPHG artifacts enable us to influence the story when we wear the clothes and use the artifacts. This influence increases the feeling that the player is involved in the fictional story.

From the experiments described in Section 5, we have also found that the reality of the characters, such as facial expressions, movement and behavior, is essential to increase the enjoyment of the game. Reality is an important criterion to evaluate a design. In our case, the virtual character’s behavior should be consistent with the character’s behavior in the animation story; this is important for feeling the reality in the game. For example, cards that are not used by the character in the animation also should not be used in the game, and the movement of the virtual character should be consistent with its typical movement in the animation. The number of cards owned by the character should be consistent with the real game situation as well. Too-realistic expressions, however, may cause the uncanny valley problem [48] if we perceive that the achieved reality is not completely realistic.

There are three issues to be considered in terms of the reality in this case. First, a proper reason why items and characters exist in our world is needed to increase the reality of a user’s experience. In Augmented TCG, the appearance of Kaiba and Yugi as opponent players is not so unrealistic when playing Yu-Gi-Oh! TCG. Second, the behavior of items and characters should be consistent with their behavior in the fictional world. In particular, if a user is familiar with the fictional story, from our experience with Augmented TCG, this issue is very important. Third, the virtual form should be natural, similar to traditional real materials. However, the virtual form may exaggerate the effect in the real world, which might make the DPHG artifact with the virtual form even more exciting and attractive than the original one. A feeling of reality is important so that a user continues to use a DPHG artifact.

There is also an alternative way to increase the reality of the fictional world: adding interaction in the real world. If there is an interaction between a user and a fictional story, he/she can create more engagement in the story and the interactivity increases a user’s autonomy that enhances his/her intrinsic motivation. This is why we usually have more engagement in gaming than watching animations. Users feel the reality of the fictional story because their intervention has a strong influence on the story. If the reality is lost, a user’s experience is just like watching a story that is unrelated to the user’s daily life. Then, the relationship with the real world is also lost, and the user’s engagement is decreased.

6.4 Enhanced value-based analysis framework for increasing a User’s intrinsic motivation

The fictional story offered with a DPHG artifact makes it possible to influence a user’s attitude and behavior. A fictional story represented as transmedia storytelling can first teach us how the DPHG artifact is effective in our daily life. Then, the ideological message in the story also makes the user believe that changing his/her attitude and behavior is essential to achieve an ideological goal that will lead to overcoming some serious social problems. Finally, the positivity in the story increases the user’s self-efficacy to overcome the problems.¹³ Then, a user’s intrinsic motivation is established.

For considering the above issues, we need an enhanced analysis framework to show a guideline to design DPHG artifacts as transmedia storytelling. Figure 10 shows our basic approach to using the proposed six values to increase people’s intrinsic motivation and to make them think positively when incorporating a fictional story into DPHG artifacts. The empathetic, economic, and aesthetic values offer people extrinsic motivation, and the informative value shows a reason to change a human’s attitude or tips and tactics for making a better decision. These values are incorporated into virtual forms, and specific virtual forms contain corresponding pieces of stories, which a user feels to be valuable.

Fig. 10

Enhanced value-based analysis framework

The enhanced framework is based on the transtheoretical model, which offers a technique to increase the persuasive value described in Section 4.2. The informative, empathetic, economic and aesthetic values are used as tools to alter a user’s current behavior by reminding him/her of the importance of changing his/her behavior, which is narrated in a fictional story, and encouraging this change at an early stage in the transtheoretical model. On the other hand, the ideological value makes people’s dreams and expectations explicit and teaches them how changing a user’s attitude realizes his/her dream and how the attitude change is important for him/her. Additionally, the persuasive value leads the user to believe that he or she can perform the ideological activities in the real world if the story’s fictionality becomes a part of the real world through virtual forms. These two values are used to increase the intrinsic motivation to alter the user’s attitude in the later stage of the transtheoretical model. If a user knows a story well, presenting the other four values can be used to show metaphors of ideological messages to speed up the persuasive process more quickly than using traditional approaches.

In the later stage, the persuasive value changes the extrinsic motivation of the four values to an intrinsic motivation. For example, playing against a favorite virtual character becomes a player’s extrinsic motivation. However, while understanding the background story of the character well, empathy for the character also becomes an intrinsic motivation because the user can understand how the character has tried to realize the ideological message in the fictional story. Additionally, the tangibility of a trading card as the economic value becomes a player’s extrinsic motivation. While collecting trading cards as in a fictional story, a player increases his/her self-esteem by constructing his/her own personalized and unique card deck; this becomes his/her intrinsic motivation because the process also constructs his/her firm personality.

7 Conclusion and future directions

This paper presented three case studies of DPHG artifacts that add virtual forms to enhance physical intelligent artifacts. From our experience with the design and use of the case studies, we proposed the value-based analysis framework and extracted six values from experience with the case studies. The framework enables a designer to help design and analyze experience to make interaction with intelligent artifacts more ludic. We analyzed the enhancement of Augmented TCG with the six proposed values as an example, showing how to apply the framework to improve the design through participants’ collected opinions. Finally, we presented an approach to incorporating fictionality through transmedia storytelling from experience by enhancing Augmented TCG with the proposed values. One important advantage to use the value-based analysis framework is offer different frames based on the proposed six values to analyze the current design of intelligent artifacts based on participatory design. The paper presented design implications that would be useful for the development of future intelligent artifacts that incorporate virtuality through virtual forms. Since this kind of multi-disciplinary research is not that popular, we expect that the insights from this paper would activate discussions and possibilities in the area of the multi-disciplinary research which is important for making our everyday life more rich and enjoyable. In the next step, we need to validate the proposed values by designing more case studies and further showing the effectiveness of incorporating fictionality for increasing the effect of ideological messages.

The design implications presented in the paper are also helpful, relating to the design of various information services or products that make us feel more valuable. The insights presented in this paper should assist in the development of future digital-physical hybrid worlds, which will make future smart cities more enjoyable and gameful [63]. The aspect is important because the traditional approach typically focuses on efficiency issues such as energy management or traffic management in our daily life. Our approach allows us to consider how to make our daily lifestyle more enjoyable and happier by incorporating virtuality and fictionality in the real world.