Keywords

1 Introduction

At present, cooperative work occurs on face to face and remote bases. Cooperative work can be said to be of three types (Fig. 1) -“face to face type”, “remote type”, and “mixed type”. The “face to face type” is the case in which all participants gather at the same location. Here, all participants’ operation can be seen clearly and therefore, there is no problem with their acknowledgement. The “remote type” is the case in which all participants are in different locations. Here too, all participants’ operations are acknowledged. The “mixed type” is the case in which some participants work from remote environments. Contribution acknowledgement is a problem in this case because members in the shared space are rarely aware whether remote participants have contributed to the work. For example, when the remote participants move an object on a sharing screen, it appears at the new location to the participants in the shared space. Because of this difference of the recognition, it is a possible that they will be left behind in work or communication that occurs during work on the remote type, especially the mixed type. In this study, we propose an emphasized pseudo expression that shows the remote participants’ contributions (moving objects) to those in the shared space. This expression uses a symbol to represent the individuals who perform operations when they perform operations; thus enhancing the pseudo expression. In this paper, we aim at solving the problem they will be left behind in work or communication, by improving the presence and acknowledgement of remote participants and their contributions, respectively, by using the emphasized pseudo expression when they operate.

Fig. 1.
figure 1

Different categories of a cooperative work

Full size image

2 Related Work

Higuchi et al. showed that the visualization of the job instructor’s gaze position in remote collaborative work helps improve the work support efficiency and decreases the number of failures [1]. Yamamoto et al. showed the improvement in the efficiency of remote work by presenting body motion using a head mount display [2]. In this study, it is different in that it supports each person’s work and not work instruction.

In a study on the recognition of collaborative work by remote participants, Suzuki et al. developed a communication support system using robots [3]. Doucette et al. used the user’s own arms in various ways to improve presence [4], Suzuki et al. investigated whether the user’s task execution could be improved with the presence of a physicalized agent in the form of a silhouette [5]. In a communication system, body area networksFootnote 1 is a focus [6, 7] and Varga et al. developed an application using a body channel communication [8]. Their application used not only wireless devices, but also floor type devices, which are placed on the floor, and deferred type devices, which are operated via touch with hands. In this study, it is different in that it aims at the presence improvement using the pseudo object which can identify an individual instead of using physical information. In the study on social telepresence using partial materialization of a body image, Onishi et al. proposed the reinforcement of telepresence by materializing the boundary of the display part of the body [9]. In this study, we aim at improving the recognition of the presence of remote participants and their contribution in “mixed type” cooperative environments.

3 The Proposed Emphasized Pseudo Expression

We describe the proposed method in this section. This method assumes that there is collaborative work when there are participants sharing the same space and also remote participants. Moreover, in supporting cooperative works, there are some systems with using card type information [10, 11], then it is assumed that participants use a card (displayed on the screen) on which information is written to present their opinions as a task. In this method, when the remote site sides are collaborating with the shared space side, it uses to delay by the expression that can distinguish who operated. As a tool that distinguish who operated, we can give USE-togetherFootnote 2, remote collaborative tool, has remote collaborator’s cursors have his/her name around the arrow icon. However, in this case, we display the face of the operator.

This expression displays as follows.

(1) :

Click on the card to be moved on the remote device.

(2) :

Determine the destination of the card and click on the remote device.

(3) :

The device ID, the card to be moved, and the information of the required destination is sent to the device on the shared space side.

(4) :

The use the information received by the device on the shared space side is as such: the object to be represented is placed on the moved card, and the object is moved linearly to the required destination.

Fig. 2.
figure 2

Use of the proposed emphasized pseudo expression

Full size image
Fig. 3.
figure 3

Experiment construction in the shared space

Full size image

The card operated by the remote participant cannot be manipulated until the state overwriting, the highlighting pseudo expression, is finished functioning. Moreover, when an operation is performed on the shared space side, the pseudo expression does not occur on the remote sides, even when the state is overwritten in the operation on the remote side. Figure 2 shows the state on the shared space side being overwritten after remote side operation. In Fig. 2, the figure on the left shows how the operator moves the card in the direction of the arrow from the upper right, and the figure on the right shows the pseudo expression on all devices other than that of the operator after the operation. It shows how the card moves in a straight line.

Fig. 4.
figure 4

A screen of work on the experiment. *1: In English, two pairs of sayings have the same meaning, but in Japanese, the proverbs mean are different phrasing. *2: When an existing English saying could not be found for a Japanese saying, the English saying was directly translated from Japanese.

Full size image

4 Experiment

We evaluated the performance of the proposed expression method. Figure 3 shows the experimental construction in the shared space. In this experimental setup in the shared space, the screen of the PC (Fig. 3-(a)) was projected by a projector (Fig. 3-(c)), and a dedicated pen (Fig. 3-(b)) was used the operation. The subjects of this experiment divided 20 cards, which had proverbsFootnote 3 written on them and which were produced by the system with and without emphasized pseudo expression. Figure 4 shows the screen displaying the work to be carried out. In this work, a participant moves pairs of cards that seem to have the same meaning to the same colored areas. The time allotted for this work was 10 min, but if the work did not finish within 10 min, it was stopped. However, all groups finished the work in 10 min. Additionally, before the actual work, in order to get familiar with the operation of the pen on the shared space side, they worked on dividing 20 cards with numbers from 1 to 20 into the same pair at the end of the digit. Here, numbers 0 to 9 were written in the move destination areas, and the number written in the area and the digit on the card were to be matched. In this experiment, three persons in the shared space and one person working remotely were assumed to be of the same sex, and the experiment on the shared space side was performed with 12 persons in total, with four groups of three persons. In consideration of the order effect, two groups worked from the side with emphasized pseudo expression first, and the remaining two groups worked from the section without emphasized pseudo expression. After the work, questionnaires were given to the participants in the shared space. The number of people operating with the pen (Fig. 3-(b)) was not changed on the shared space side. Figure 5 shows the experiment in each position. Figure 5(1) is a scene of the remote space. The participant used a PC in the experiment. Figure 5(2) is a scene of the shared space. The participants did the work looking at the screen projected from the projector, and the participant in the front in the figure used the operating pen.

Fig. 5.
figure 5

Scenes of the experiment

Full size image

5 Experimental Results and Discussion

5.1 Recognizing Task Participation in Remote Participants

Table 1 shows the results of the questionnaire on the recognition of participation in the activity by remote participants. In the 5-step evaluation of the Likert scale, for the question “I felt that the remote person participated in the work”, those using emphasized pseudo expression all answered “strongly agree” or “agree” and the median and the mode were 4. Those participating without the emphasized pseudo expression comprised of 2 people answering “strongly agree”, 8 people answered “agree”, 2 people answered “neither agree nor disagree”, and the median and the mode were 4. In the free-form description, people who used the emphasized pseudo expression had the opinions that “The face appeared when the card moved and it was easy to understand”, “I knew who moved the card”; the emphasized pseudo expression for those who did not use it had the opinion that “I knew it was operated on because it moved quickly, but I had no sense of presence.” From the results of the experiment, it was shown that the emphasized pseudo expression has the potential to help the recognition of remote participants, but there was no significant difference in recognition from the case of non-expression.

5.2 Recognizing the Number of Card Movements

We discuss the results in terms of the difference in how much the shared space side recognizes the contribution of the remote participant, depending on the presence or absence of the emphasized pseudo expression for each group. In the questionnaire, we asked whether there was an expression or not, whether they noticed the operation of a remote participant, and if they did not notice it, we counted the number as 0.

Figure 6(a) shows the difference between using emphasized pseudo expression and the average is −1.67 times (SD \(=\) 0.47), and the difference without emphasized pseudo expression is −4.67 times (SD \(=\) 0.47) on average. In both cases, the participants in the shared space were aware that the remote person operated the card, but the difference in the average value in this group was due to the absence of the remote participant. It is thought that the difference became extensive because the number of times the group moved was 8 times, but they said “I saw the face of the remote area participant”. So we showed an effect that makes it easier to recognize the operation of remote participants.

Table 1. Questionnaire on recognition of participation in work by remote participants (5-step evaluation)
Full size table

Figure 6(b) shows the difference between using the emphasized pseudo expression and the average is −0.33 times (SD \(=\) 0.47), and the difference without emphasized pseudo expression is −1.0 times (SD \(=\) 0.82) on average. In both cases, participants in the shared space were aware that remote person operated the cards. In fact, this group found it easy to recognize the operation of remote participants, additionally from the viewpoint of the card moving in the emphasized pseudo expression, no difference was found between the presence and absence of expressions as numerical values.

Fig. 6.
figure 6

Difference between the number of operations and the number of recognitions (Case of the experiment: (EPE: The case with emphasized pseudo expression, Nothing: The case without emphasized pseudo expression))

Full size image

Figure 6(c) shows the difference between using the emphasized pseudo expression and average is −1.33 times (SD \(=\) 0.94), and the difference without emphasized pseudo expression is −1.0 times (SD \(=\) 0) on average. It was thought that there was no difference in recognition because recognition was dependent on the presence or absence of expression and the emphasis was on the work regardless of the presence or absence of the emphasized pseudo expression.

Figure 6(d) shows the difference between using the emphasized pseudo expression and the average is −1.0 times (SD \(=\) 0), and the difference without emphasized pseudo expression is −1.33 times (SD \(=\) 0.47) on average. In both cases, participants in the shared space were aware that a remote person operated the cards. In this group, we showed an effect that makes it easier to recognize the operation of remote participants, but no difference was found between the presence and absence of expressions as numerical values. From these results, it can be seen that the emphasized pseudo expression can help making it simple to recognize the remote participant’s operation, but there was no significant difference in recognition from the case of non-expression.

5.3 Discussion the System

In the description of the system, there were opinions such as “The face was displayed and the track of the card was easy to understand” and “There was an impact”, but there were also opinions that required improvement such as “I felt that it would be nice to know the position when showing the location to the other party with the command language”, “I cannot tell which one to move because the preliminary movement to move the card is not transmitted”. With regard to the opinion, “I felt that it would be better to know the position when showing the place in the directive language”, in the shared space, the card was pointed during the experiment; however, it was not clear which point was assigned to the remote side. From these results, it seems that using the face of the individual for the emphasized pseudo expression contributes to help the recognition of their presence and contribution. However, by not using the expression at the time of preliminary operation, there was confusion on who was moving because the expression for indicating which card was being moved has not been made, and the instruction in the conversation also mentioned the problem of what was being pointed to; these issues need to be corrected.

6 Conclusions

In this paper, we proposed an emphasized pseudo expression aiming to improve the recognition of the presence and contribution of remote participants. This expression is to display the movement (card movement) by the remote participant on the shared space side. The result of the comparison experiment with the use or not of this expression, showed the following:

  1. (1)

    The emphasized pseudo expression can help the recognition of remote participants, but there was no significant difference in recognition from the case of non-expression.

  2. (2)

    The count of the remote participants’ operation does not have the condition of the cognition in the shared space if emphasized pseudo expression has.

Moreover, it became clear that the following points are problems.

  1. (1)

    We need to add the preliminary movement of the card in the emphasized pseudo expression.

  2. (2)

    We need to clarify which card is pointed to by the directive.

In the future, as a new form of emphasized pseudo expression, we plan to implement a display function to make the image intrude on the work screen using speech and experiment using it.