Can an Advanced AI Provide the Same Level of Enjoyment as Playing with Human Beings?

Abstract

In the field of game AI, it has become possible to utilize characters whose movements are similar to characters controlled by humans. This raises the question of whether playing against AI will be as interesting as playing against humans. In this paper, we attempt to answer this question by investigating the psychological changes that occur when a player is told he or she will be playing against an AI opponent when it is actually a human opponent, in order to reproduce the state in which AI has evolved. We developed a competitive batting game and conducted an experiment in which participants were asked to play two games: one in a virtual AI condition and the other in a human condition. The results showed that the “I thought about reading my opponent’s pitch distribution (location and velocity of pitches” was significantly higher in the human condition. This suggests that human players enjoy trying to guess the opponent’s thoughts, indicating that playing against AI may not be suitable for psychological games.

1 Introduction

In the field of game AI, it has become possible to utilize characters whose movements are similar to characters controlled by humans [1,2,3,4]. However, the competitive element is an important factor in the enjoyment of playing games [5], and according to research by Kubo et al., it is generally considered more interesting when the opponent is a person [6]. In their study, ten university students played games under two conditions: a single condition in which they played against a computer, and a competitive condition in which they were told they were playing against a person but actually played against the computer. The results showed that the excitement, fun, and enjoyment were significantly higher in the competitive condition than in the standalone condition. These findings indicate that the belief that one is playing against others influences subjective enjoyment. In this paper, unlike prior studies, we tell players that they will play a game against a computer. In reality, however, we investigate the psychological changes that occur when playing a game against a person. In the experiment, we used a batting game that we originally developed to reproduce the state in which the computer has evolved, and attempted to answer the question of whether playing against an evolved computer is as interesting as playing against a human opponent.

2 System

Figure 1 shows the competitive batting game used in the experiment. The high score, points, hits, home runs, remaining balls and distance are displayed in the upper-left corner of the screen. Both the number of ideas and originality are known to be higher when given difficult goals compared to easy goals [7], so the high score is not an actual number but rather a number that is difficult to exceed.

Fig. 1.

Initial game screen.

Fig. 2.

Game screen during play.

The game rules consist of hitting the pitcher’s pitches and scoring points from a total of 15 pitches. One point is scored for a hit and three points for a home run. The pitcher throws two types of pitches: fast and slow. When a pitch is judged to be a ball, the number of remaining pitches is not reduced. Figure 2 shows the screen transition during play. While the left mouse button is pressed, the cursor of the bat is displayed, and moving the mouse also moves the cursor. The bat can be swung by releasing the left button. In the experiment, the experimenter played the pitcher and the participants played the batter.

3 Experiment

3.1 Experimental Procedure

The experimenter and participant sat facing each other across computer monitors, and the participant could not see the experimenter’s movements. When the opponent is of a high level, frustration is more likely to prevail than fun, suggesting that the skill level of the computer opponent should be roughly equivalent to that of the human player if game enjoyment is a priority [8]. Therefore, in our game, the opponents (experimenters) consciously allocated pitches (throwing locations and two types of pitch speeds) so that the participants could enjoy the game.

The participants were 20 university students (19 men and one woman). They were asked to play two games: one in the “virtual AI condition” in which they were told to play against a computer opponent but actually played against a person, and one in the “human condition” in which they were told to play a game against a person and in fact played against a human opponent. After the games, we administered a questionnaire to investigate the psychological changes. Our goal was to determine whether playing against a computer was as interesting as playing against a person. The experimental flow was as follows.

• 1. We explained the game operation and rules.

• 2. Participants were allowed to practice up to 30 swings.

• 3.1. After playing in the virtual AI condition, we administered a questionnaire.

• 3.2. After playing in the human condition, we administered a questionnaire.

• 4. We asked participants for feedback on the experiment.

Note that the order in which steps 3.1. and 3.2. were performed was assigned according to the counterbalance.

Table 1 shows the content of the questionnaire, which consisted of 11 questions: three on “dissatisfaction,” one on “concentration,” three on “enthusiasm,” one on “enjoyment,” and three on “difficulty,” referring to a prior study [6]. Responses were given using a 7-point scale (1: very much disagree to 7: very much agree). The questionnaire was administered twice, once after each condition. After the experiment, we asked the participants for their impressions of the game.

Table 1. Questionnaire items.

3.2 Results and Discussion

Figure 3 shows the results of the questionnaire. A Wilcoxon signed-rank test on the obtained results revealed that the value of the enthusiasm question, “I thought about reading my opponent’s pitch distribution (location and velocity of pitches),” was significantly higher in the human condition ($p = 0.016$). On the other hand, no significant differences were observed for the other questions (dissatisfaction, concentration, enjoyment, and difficulty).

Fig. 3.

Questionnaire results.

Since there were no significant differences in many of the items, we can infer that the participants enjoyed the virtual AI condition to the same extent as the human condition. This is presumably due to the fact that they had a psychological desire to score high points from the computer, as evidenced by responses such as “I had a great desire to increase my score when playing against the computer.” At the same, it seemed that participants enjoyed trying to guess their opponents’ ideas in the human condition based on the results of the questionnaire. In other words, in the virtual AI condition, the players found it interesting to improve their scores, while in the human condition, they found it interesting to read the psychology of their opponents. These findings suggest that playing against AI may not be suitable for psychological games such as batting games.

4 Conclusion

In this paper, we investigated whether playing against a computer was as interesting as playing against a person by examining the psychological changes that occur when a player is told that he or she will play a game against a computer but actually plays against a person. Our findings showed that the players found interesting in improving their scores in the virtual AI condition and in reading the psychology of their opponents in the human condition. As the values for reading the opponent’s mind were significantly higher in the human condition, playing against an AI may not be suitable for psychological games. In future work, we will investigate whether changes can be observed by varying the strength of the player and the opponent. In addition, we will compare games with and without psychological warfare to explore the range of possible AI applications in the game field.