Keywords

1 Introduction

Theater activities are currently actively carried out as a part of arts. Theater organizations create products by director, actors and other many staffs through roughly divide in three practice steps, “Reading Script”, “Standing Practice”, and “Stage Rehearsal”. Important factors of production coached by director are actor’s standing position, head direction, movement, theatrical elocution, and action timing. actors acquire performance through practice with other actors and be coached by director. However, theater organizations cannot practice anytime because many of them does not have their own practice space and most actors have another job. In this paper, we propose “Digital-Script” and theater practice support system. Digital-Script contains important information of production. Theatrical practice system enables actors to practice with awareness of production and other actors in the situation that director or a part of actors cannot gather by using Digital-Script. Practice system supports practice by showing virtual actor that plays absent actor’s role in monocular Head Mounted Display(HMD), and coaching automatically by detecting and comparing actor’s movement with information contained in Digital-Script.

The rest of this paper is organized as follows: in Sect. 2, we describe theater activities, the work related to them. In Sect. 3, we explain the issues for theater activities and the requirements for solving them. Section 4, provides details of our proposal, and the evaluation experiment is explained in Sect. 5. Finally, in Sect. 6, we present our conclusion.

2 Theater Activities

Theater organizations create products by director, actors and other many staffs through roughly divide in three practice steps, “Read Scripts”, “Standing Practice”, and “Stage Rehearsal”. “Standing Practice” is very important especially in three steps, and accounts for most of the period of the activity [1], so the quality of the product rely on standing practice. Standing practice is rarely done through to the end but mostly done several scenes repeatedly in a day and then director coaches. Table 1 shows the results of survey on efforts for theater quality improvement. It can be seen from Table 1 that many theater organizations encourage the voluntary practice of actors in order to improve the quality of theaters. Voluntary practice is very important and related deeply for the improvement of theater quality. In voluntary practice, learn by heart script or role making are mainly done and it is difficult for actors to practice standing position, head direction and timing of action because scripts only written in character.

Table 1. Activities of theater groups to enhance the quality of a theater [2]
Full size table

2.1 Related Work

Studies that is about artistic activities are widely carried out in the past. Singh et al. enables dancers to practice independently with awareness of the instruction by enabling choreographers to add annotations to a video that records dancers’ performance [3]. Oshita et al. made authoring system for video teaching materials about Noh play (Japanese traditional) [4]. They divided Noh motions in three elements (choreography, speech and locomotion) and visualize each element. Gandy et al. uses HMD for theater simulation. By using HMD, user can check stage environment from first-person view point but user cannot perceive the surroundings [5]. CAVE system that is omnidirectional display sometimes be used for study about theater production. Steptoe et al. use CAVE system for remote theatrical practice. By using CAVE system, user can perceive distance sense and positional relation [6]. Jacobson et al. and Cavazza et al. also uses CAVE system for theatrical practice [7, 8]. CAVE system enables users to practice in virtual stage environment that is in real space. Omnidirectional display enables users to perceive stage environment from first-person view point same as HMD but the facility of the system is costly, so it is not easy to use. Rijsselbergen et al. and Zhang analyzes scripts by using natural language processing and visualizes the scripts which analyzed. There are some studies about creating scripts [9, 10]. Szilas et al. made their original programming language to create scripts [11]. Hong et al. built interface that is used for creating scripts by using XML format [12]. Horiuchi et al. used tabletop interface to plan theatrical productions [13].

There are some studies on actors’ voluntary practice [14, 15]. These studies focus on important factors of production and enables actors to practice with awareness of these factors.

3 Support for Theatrical Creation

In this study, we propose “Digital-Script” which differ from usual script written only in characters contains actor’s standing position on the stage, head direction that the actor should look and time information that actor should act. By using Digital-Script, we enables actors to practice with awareness of production and other actors that is difficult for practice using usual script.

3.1 Digital-Script

Factors of Production. Theatrical production has technical factor and actor’s performance factors. Technical factors are such as sound effect, lighting effect, stage carpenter and stage design. When we focus on actor’s performance factors, standing position, head direction and timing of actions are very important [16]. These factors determine the sight from audience and the flow of the theater. However, it is difficult for actors to practice in the situation that actors face each other or move while looking at other actor and cannot be coached on condition that the director or a part of actors is absent. Therefore, reflecting actor’s performance factors to scripts is very important for theatrical practice that carried out by using a script.

Requirements for Scripts. A script is what represents story of the theater. It is difficult to perceive spatial environment such actors’ standing position that changes with progress of a story because usual script is only written in character. In addition, if it is the famous script, such as a masterpiece, it also be performed many times by other theater organizations. If a script is performed by other theater organizations, productions are different each other because the staff who create the theater are different. Therefore, it is very difficult for actors to perceive the production that the director images. We propose “Digital-Script” that contains spatial information and time information. Spatial information are such as “where the actors are standing” and time information are such as “where the actors are looking”. Spatial information is useful for actors to practice distance sense with other actors. Time information is useful for actors to practice the timing of actions. These two kinds of information represent production and important for practice. Digital-Script enables actors to share images of the theater by being shared. By adding production to script, actors who cannot be coached by the director because of economic, time and spatial constraints can perceive production. Further, since it is possible to make modifications to the production unlike such as recorded video, it is also possible to use information in the way appropriate for the production.

Requirements for Practice. Many theater organizations encourage actor’s voluntary practice in order to improve the quality of play because of the cost of practice space and time. Therefore, by reproducing the coaching or act of the absent staff, we support actors’ voluntary practice on the environment that a director or a actor is absent from the practice. The way to support voluntary practice in the situation that a director or a part of actors is absence is as follows.

  • Absent actor: It is difficult to practice such as facing each other in the situation that the partner actor is absence because it cannot be perceived that where the partner is looking at or where the partner actor is. Therefore, we support actors’ practice in the situation that a part of actors is absence by showing absent actor.

  • Coaching: We support practice with awareness of production in a situation that the director is absence by focusing on actor’s standing position and head direction. Standing position and head direction is very important for theater creation because these factors represent relationship between actors and determines theater progression. We focus on productions especially in spatial and time production and not mention theatrical elocutions or physical expressions. This is against that it is very difficult for directors to show what they image in an exactly way about such theatrical elocutions and physical expressions, and the purpose of voluntary practice is to acquire large scale performance such as standing position.

4 Digital-Script and Theatrical Practice System

We propose theatrical practice system. In this system, it is assumed a situation that is as follows.

  • Scripts that three actors appear:

    We assumed a script for three performers and only two actors can join a practice. The script contains performances such as actors facing each other or move. We assumed this number of performers, because it is considered this system to be able to apply to other scripts which for more performers by increasing devices or detections if we can achieve in this number setting.

  • Practice space: We assume practice space that about \(4 \times 4\) meters wide and nothing in it. actors practice in the space. This area is derived from device but same as number of performers, if we can achieve proposal in this setting it is considered that this system is thought to be able to apply large area by improving device capacity.

  • Production in practice: We do not deal with theatrical elocution, physical expressions and facial expressions because it is very difficult to express what the director wants in the exact way and we focus on large scale production. For the first step of practice, actors acquire standing position and head direction where they should look at.

4.1 Digital-Script Databse

Production information are managed in database implemented by MySQL. Figure 1 shows configuration of the Digital-Script database. Information stored in database in scene units. Scene data contains IDs, title of the script, title of the scene and actor data. actor data contains MOVE data that is about actor’s position, SAY data that is about actor’s speech and SEE data that is about head direction. Each of the data is as follows.

  • MOVE data: MOVE data are about actor’s standing position. It means actor’s move to (x, y, z) coordinates during elapsed time (the origin is the begging of the scene) is \(t_{1}\) sec to \(t_{2}\) sec.

  • SEE data: SEE data are about actor’s head direction where the actor should look at. It means that the actor looks at the designated direction when the elapsed time get t sec. The direction where the actor should look at is designated by (x, y, z) coordinates.

  • SAY data: SAY data are about actors’ speech. It means that the actor utterance speech T when the elapsed time gets t sec.

Fig. 1.
figure 1

Component of Digital-Script Database

Full size image

4.2 Theatrical Practice System

We propose theaterical practice support system in the situation that a part of actors or the director cannot gather.

Approach. We support theatrical practice by supplementing the role of an actor or a director who is absent from practice. In this study, we assume that two actors join the practice, so it is necessary for those two actors to be able to recognize the partner actor who joins the practice and the actor who is absent from practice. Further, there is a problem that actors cannot to be coached. Therefore, the function to supplement absent actor’s role and director’s role is needed for theatrical system. Practice system supplements the role of absent actor by showing the actor who cannot gather while checking the actor in the space. User can check virtual actor that perform instead of absent actor with real actor by using monocular HMD. We realize theatrical practice with awareness of other actors by showing virtual actor. In addition, the system enables actors to practice with awareness of production by coaching them automatically along the production. The role of real space and virtual space is as follows.

  • Real space: The space actors are present. Only the actors who join the practice perform in Real Space.

  • Virtual space: Virtual space is the virtual environment built on the system. Virtual actor performs move and change head direction along with production information in database instead of absent actor. The standing position and point of view links user’s position and head direction. The two actors gathering the practice are not showed in virtual space.

The actor who joins the practice recognizes real space in the eye that does not equip monocular HMD and virtual space through monocular HMD, so it is possible to recognize real actor and virtual actor simultaneously. In addition, the view point of the virtual space links the user’s position and head direction in real space constantly. Users can check partner’s performance and virtual actor’s performance from any place as if the virtual actor really be there.

4.3 Implementation

Theatrical practice system is coded in C# and implemented in Unity that is often used for building 3D games. Virtual stage and virtual actor are made in Unity and showed in monocular HMD, and the view point in the virtual space moves according to user’s movement. By using monocular HMD equipped with a gyro sensor and Microsoft Kinect that can detect user’s position, user can view virtual space from the view that links their own movement. In the following, we describe the method of supporting practice, then describe system configuration.

Detection of Actor’ Movement. The system detects user’s standing position and head direction. The detected information is used for controlling view point camera what is in the virtual space. Figure 2 shows system configuration.

Method of Showing Virtual Space. In the virtual space, a virtual actor is shown and perform along with Digital-Script information. In virtual space, there is a view point camera and it moves virtual space linked with actor’s real space movement. User can check virtual space intuitively because of the view point movement that links with actor’s movement.

  • Position: Kinect detects actor’s standing position. Kinect can detect many body parts coordination but we only use head position and reflect the detected data to view point because it is enough for controlling view point camera. Head position is detected in three dimensions and reflect them constantly to view point. Therefore, actors can check virtual space from anywhere in real time.

  • Direction: We use Vuzix M100 that is monocular HMD. The HMD equipped with gyro sensor. We get actor’s head direction and reflect it to the view point direction. It is possible for users to see virtual space intuitively by reflecting their head direction constantly to view point.

Fig. 2.
figure 2

System configuration

Full size image

4.4 Screen Displayed on HMD

Figure 3 shows what is displayed on HMD. Virtual actor, elapsed time and actor’s speech is shown on the screen. Virtual actor is 3D model and performs such action like move or change head direction along with production in the database. Elapsed time is shown in second unit and constantly progress while the system is being played. actor’s speech is colored and the color matches the virtual actor model’s body color, so user can distinguish who should say the speech.

Fig. 3.
figure 3

Screen displayed on monocular HMD

Full size image

Practice Method Using the Theatrical Practice System. User equips the monocular HMD and starts the system. Figure 4 shows an image from user view point. The performance along with the theater flow in the Digital-Script database starts in the virtual space automatically when the system starts. actors start performance such as move or change head direction along with the play progresses in the virtual space. While actors are performing, the system detects their movement constantly. If user do different performance from the production in the Digital-Script, for example the user should move to right edge of the stage but he moves to center of the stage, then the theater progression in the virtual space stops automatically. When user did wrong performance the time progression stops and then user can recognize that he did wrong performance and then the system gives coaching about user’s standing position or head direction. User acquire performance along with the production in Digital-Script database by modifying their performance through being coach by the system. Theater progression restarts if the user can satisfy the production in the Digital-Script database. Thus, the user practice and correct their performance repeatedly, he acquires performance skill that satisfies the production.

Fig. 4.
figure 4

View image from user view point

Full size image

Coaching by the Theatrical Practice System. Standing position and head direction are coached by the system. Standing position is detected constantly by using Kinect, and detected position is compared with the value in the Digital-Script database. If the difference between detected position and the position in the database exceeds 25 cm, the system coaches the user on the monocular HMD. Figure 5 shows how the system gives coaching to the user about standing position. The yellow marker shown in Fig. 5 is the correct standing position of the user. The user can acquire correct position by moving to the marker in the real space. After user moves to the position indicated by the maker, coaching ends and the yellow marker disappears. Head direction is detected constantly by monocular HMD and compared with the value in the Digital-Script database. If the difference between detected head direction and the direction in the database exceeds 10degrees, the system coaches user on the monocular HMD. Figure 5 shows how the system gives coaching to actors about head direction. The user is coached about his head direction by four patterns “Loot to the Right”, “Look to the Left”, “Look Up” and “Look Under”. User correct his head direction in accordance with the coaching by the system. While user is looking wrong direction, the coaching is shown. If user can look right direction, the coaching that is about head direction shown on the screen disappears. Thus, user correct his head direction and acquire right performance with awareness of production. The user can correct their performance when he did wrong performance and acquire right performance in the body.

Fig. 5.
figure 5

Displayed Screen when user need coaching

Full size image

5 Evaluation and Discussion

We evaluate efficacy of theatrical practice system by measuring the accuracy of actor’s performance after practice in the situation that an actor and a director are absence.

5.1 Description

Six students participated this experiment. Subjects are divided into two groups, A and B. Each groups practice using theatrical practice system or visualization system. Two scripts used in this experiment is about one minute long. Two people, a subject and a experimenter participate in each practice and it is assumed that the experimenter can perform perfectly. We evaluate accuracy of subjects’ performance by measuring the error about standing position and head direction by comparing their performance with the data in the Digital-Script database. We prepared two scripts for this evaluation. Each script is about one minute long and contains six instructions, three instructions about standing position and three instructions about head direction. Table 2 show two subject group and conditions of practice.

Table 2. Subject Group and Practice Condition
Full size table

Comparison Item. In this experiment, subjects practice in two environments. Two environments are as follows.

  • Using theatrical practice system(system-method): Subjects practice performance by using Theatrical Practice System. Only subject equips monocular HMD.

  • Using application that visualizes Digital-Script(visualization-method): Subject uses application that visualizes Digital-Script. Figure 6 shows application screen that visualizes Digital-Script. Standing position, head direction and elapsed time are shown on the application screen. Virtual theater progresses on the simple virtual stage along with production in the Digital-Script. User’s speeches are colored and the its color matches the virtual actor model’s body color, so user can distinguish whose speech that is. User’s head direction is represented by virtual actor model’s nose direction. The subject practices by using this application freely. The subject can control the application with start, stop and reset functions.

Fig. 6.
figure 6

Visualization application display

Full size image

Procedure. Each subject practices twice on two conditions shown in Table 2. Experimental procedure is as follows.

  1. 1.

    Subject practice for ten minutes with experimenter. At this time, experimenter performs perfectly but does not coach subject.

  2. 2.

    After practice, the subject and the experimenter demonstrate. While this demonstration, subject equips monocular HMD to detect head direction but does not show anything on the screen. We use the data detected in this flow to measure accuracy of subject’s performance.

  3. 3.

    The subject does a same procedure described above on another condition.

5.2 Evaluation Item

We evaluated about two points as follows.

  • Accuracy of subjects’ standing position

    We measured the differences between detected subject’s standing position and the standing position value in the Digital-Script database.

  • Accuracy of subjects’ head direction

    We measured the differences between detected subject’s head directin and the value in the Digital-Script database.

5.3 Results and Discussion

Results are obtained by calculating average about three instructions contained in each script. Result of each script (Tables 3 and 4) and total result (Table 5) is below.

Table 3. ScriptA result
Full size table
Table 4. ScriptB result
Full size table
Table 5. Total result
Full size table

It can be seen from Tables 3 and 4 that the errors of the subjects who used visualization-method is larger than that of subjects who used system-method in both A and B scripts. It can be thought from this result that subjects’ error who used system-method is smaller than that of subjects’ who used visualization-method about other scripts. It can be seen from Tables 3 and 4 that the errors of the subjects who used visualization-method is larger than that of subjects who used system-method about head direction. It can be thought that it is difficult for the subjects to acquire correct head direction by using visualized script. In addition, if it is possible for users to check rough direction, they could not correct details because they could not see absent actor. It can be seen from Tables 3 and 4 that the errors of the subjects who used system-method is smaller than that of subjects who used visualization-method about head direction. It can be thought that the subjects who used system-method could correct details about head direction because they could see virtual actor who performs instead of absent actor. In addition, subjects who used system-method could be coached by the system and they could correct their performance at the time they did wrong performance. It can be seen from Tables 3 and 4 that the errors of the subjects who used visualization-method is larger than that of subjects who used system-method about standing position. It can be thought that it is difficult for the subjects to acquire correct standing position in the real space by using visualized script. If it is possible for users to check rough position, they could not correct details because they could not see absent actor. Also, it can be thought that they could not perceive correct standing position in real space from visualized script. It can be seen from Tables 3 and 4 that the errors of the subjects who used system-method is smaller than that of subjects who used visualization-method about standing position. It can be thought that the subjects who used system-method could perceive distance sense with absent actor because they could see virtual actor from first-person viewpoint. In addition, subjects who used system-method could be coached by the system and they could correct their performance at the time they did wrong performance.

6 Conclusion

Theater organizations create theater through practice repeatedly for long term. All actors gather same place and make a practice based on the coaching of director to acquire performance with awareness of production and other actors. However, the amount of practice time that a director and all actors gather same place is limited. In this study, we built Digital-Script and Theater Practice System. Theater Practice System focuses on the roles of an actor and a director who is absence from practice. The system shows virtual actor in monocular HMD and gives coaching to a user. It is possible for actors to practice with awareness of standing position and head direction by using the system. We evaluated our system by measuring accuracy of subjects’ standing position and head direction. We confirmed that users who practice with our system tend to acquire performance along with the production in Digital-Script.