Keywords

1 Introduction

In this paper, we design a DiAna-AD (dialog analysis for adjusting duration) system that adjusts the length of discussions in face-to-face environments.

A discussion is considered one of the essential activities by which decisions are made and opinions shared. It can be face-to-face, online, or text-based. Moreover, with advances in technology, online discussions can be performed in real-time using multimedia [1]. This paper aims to improve the discussion skills of participants. Both direct and indirect methods are considered to be effective in improving participants’ discussion skills. The direct method possesses artificial processes of discussion to affect the improvement of participants’ skills directly. On the other hand, in the indirect method, participants must improve their discussion skills by repeatedly participating in ordinary conversations. Furthermore, in this method, the participants are required to be willing to improve their discussion skills. The approach of this research is based on the indirect method. Besides, participants’ satisfaction with a discussion is critical to enabling them to maintain a willingness to discuss. Therefore, in this paper, we propose a system that improves participants’ satisfaction with discussions.

In face-to-face discussions, the duration of discussions is determined in advance based on the experiences of the teachers; however, this fixed duration may be excessive or insufficient. With a short time of discussion, it is difficult for participants to discuss adequately; on the other hand, they feel bored when the assigned discussion time is excessively much. Thus, assigning an appropriate duration of discussions may improve participants’ satisfaction.

2 Related Work

Research aimed at improving the conditions of discussion have been previously carried out. One such example is a study [2] on enabling feedback from the teacher to students after a discussion was conducted. In the study, the authors utilized a rubric to clarify evaluation items. In [3], enrolling in Massive Open Online Courses was considered a means to acquire discussion skills. In the research, students share ideas using a discussion forum that is an asynchronous text-based discussion environment. It is notable that these researches aim to improve not the discussion skills but the condition of discussion.

Research on a non-speech section of pair conversation was conducted in [4], and it was found that non-speech sections are not suitable for those who feel weakness in their ability to talk. Furthermore, it was recommended that at the time of the non-speech section, discussions should be by topic presentation. There is a conventional approach to eliminating inactive discussion. However, methods for changing topics do not apply to a discussion with a fixed topic.

Furthermore, in [5], the authors aim to emphasize and maintain the relevance of face-to-face communication. By utilizing a touch-shake device to enable physical contact, they achieved high-quality communication. The device uses light and sound in response to physical contact. In this paper, we aim to improve the quality of discussion with a different approach.

3 Adjusting Duration During Face-to-Face Discussion

3.1 Dialog Analysis Based on Nonlinguistic Acoustic Information

Communication comprises transfer of verbal and non-verbal information. Verbal information refers to the understanding of the meaning of an utterance. In [6], verbal information was employed to create a corpus from transcribed dialogs and to analyze emotions. Non-verbal information includes facial expression and gaze, voice height, tone of voice, and gestures. In [7], the authors utilized non-verbal information to investigate gaze patterns using an eye tracker. From verbal information, it is possible to analyze the contents of the discussions. However, this analysis is manually performed by human observations. On the other hand, it is possible to analyze the attitude of participants and the intention of an utterance from non-verbal information. However, several types of non-verbal information require equipment specially prepared for collecting the information. In [8], a system that detects dialog breakdown in chat dialog from nonlinguistic acoustic information is proposed. However, the DiAna-AD system presented in this paper uses nonlinguistic acoustic information collected by wearable devices for analyzing discussions.

3.2 Nonlinguistic Acoustic Features Used in DiAna-AD System

In this subsection, we introduce the following features to estimate the situation of a discussion: time percentage of an utterance, percentage of silent time, and coefficient of speech overlap. These values are calculated from only nonlinguistic acoustic information per unit time.

Time Percentage of Utterance.

This value is the utterance time of a participant per unit time, and it is calculated for each participant in a group discussion. The value is given in the range from 0% to 100%. The degree of participation of each participant in the group discussion is obtained by checking the transition of this value.

Percentage of Silent Time.

This value is the time that nobody speaks per unit time, and it is calculated for a group. The value is given in the ranges from 0% to 100%. The degree of stagnation of the group in the discussion is obtained by checking the transition of this value.

Coefficient of Speech Overlap.

This value is the sum of the total time percentage of the utterance of all the participants in a group and the percentage of silent time of the group on discussion, and it is calculated for a group. Besides, this value is given in the range from 1 to the number of participants in the group. Furthermore, if there is no overlap of speech, the coefficient of speech overlap is 1. The degree of activeness of the group on the discussion is known by checking the transition of this value.

3.3 Architecture of DiAna-AD System

The DiAna-AD system consists of wearable devices and an aggregation server.

The wearable devices are Raspberry Pi 3B+ attached to a unidirectional USB microphone which collects participants’ utterances. The aggregation server is a laptop that controls the wearable devices and calculates nonlinguistic acoustic features from collected data in real-time. The DiAna-AD system decides the end of the discussion based on the features discussed in Subsect. 3.2. An end notification is sent by a notification device which is made up of Raspberry Pi 3B+ and a speaker. The wearable devices, the aggregation server, and the notification device are connected using Wi-Fi, while data is sent via UDP connection.

The data flow of DiAna-AD system is as follows: first, each of the participants wears the headset of the wearable device. When the discussion begins, the wearable devices start sending data to the aggregation server. During the discussion, only the volume values are extracted from the voice input of the unidirectional microphones. The server associates the received values of IP addresses, the volume values, and the timestamps. Then, the utterances of each participant are determined with the associated data.

The features are calculated every 20 s. Moreover, the percentage of silent time and the coefficient of speech overlap are used to decide the end of the discussion. The termination of the discussion is notified by the system when it continues for more than one minute that the percentage of silent time exceeds 40% and the coefficient of speech overlap becomes less than 1.07. These thresholds are examined in the experiment to improve discussion satisfaction. Figure 1 shows the diagram of DiAna-AD system while Fig. 2 shows the dataflow of the system.

Fig. 1.
figure 1

Diagram of DiAna-AD system

Full size image
Fig. 2.
figure 2

Dataflow of DiAna-AD system

Full size image

4 Experiment of Satisfaction

4.1 Procedure of Experiment

In this research, we conducted an experiment to validate the appropriateness of the criteria of termination by the DiAna-AD system. To this end, we investigated the relationship between the duration of discussions and the personal satisfaction of participants.

The subjects of the experiment were fifteen students from a science and engineering university. Five groups of three students were formed. The task of the experiment was a discussion for ten minutes. During the discussion, the subjects wore the wearable devices to record utterances. The agenda of the discussion was: “What is the point you want to be improved at your university?”

During the discussion, the subjects were asked to answer questionnaires every three minutes. The questionnaires had a “yes” or “no” voting formula for the question “Do you want to continue talking?” The subjects answered the same question using ballot boxes for each item every three minutes. After the discussion, we conducted a subjective evaluation questionnaire about the discussion. Figure 3 shows the setting of the experiment.

Fig. 3.
figure 3

Discussion using DiAna-AD system

Full size image

4.2 Result of Experiment

The result of the questionnaire that had to be answered every three minutes during the discussion, the answers to the question “Do you want to continue talking?” will gradually change from “yes” to dominantly “no” as the discussion progress; the ratio of the answers “yes” in all the groups is 0.93 in the first three minutes, 0.73 in the first six minutes, and 0.40 in the first nine minutes. It is shown that a tendency that the participant will gradually become reluctant to talk as time passes.

A correlation is found between the results of the questions “Did you want to shorten or extend the duration time?” and “How satisfied are you with the discussion?” which are items of the questionnaire after the discussion (r = 0.79, p < 0.001). It was found that the duration of discussions influences the satisfaction of participants.

Also, a correlation was found between the result of the question “How satisfied are you with the discussion?” and the coefficient of speech overlap (r = 0.65, p < 0.01).

4.3 Criteria for Termination

Experimental results of participants’ satisfaction with a discussion revealed a strong tendency of decrease in speech motivation within the first six to nine minutes of the discussion. In the first nine minutes’ vote, it was only in one group that all the subjects answered “yes” to the question “Do you want to continue talking?” Thus, the group did not have to shorten the assigned duration as the discussion was active.

We analyzed the percentage of silent time and the coefficient of speech overlap from the first six to nine minutes of the other four groups. Based on the analysis result, 40% is decided as the criteria of the percentage of silent time for termination because the deviation value of the groups is 40. The ratio of 1.07 is decided as the criteria of the coefficient of speech overlap because the median value of the groups is 1.07. The rising of the percentage of silent time indicates strong stagnation of discussions, while a decline in the coefficient of speech overlap indicates strong inactivity of the discussion group. Therefore, as the criteria for termination, the percentage of silent time exceeding 40% and the coefficient of speech overlap less than 1.07 are adopted.

The rule to notify termination is as follows: once a discussion meets both the criteria and the condition lasts one minute, a termination notification is sent.

5 An Experiment on DiAna-AD System

5.1 Procedure of Experiment

In this subsection, we conducted an experiment with the DiAna-AD system to evaluate the effectiveness of the system to improve participants’ satisfaction with a discussion.

The subjects of the experiment were fifteen students of the science and engineering university mentioned earlier; however, this set of students were different from those used in the experiment on satisfaction. We formed five groups of three subjects each. Two sessions of discussion were conducted as the task of the experiment. The duration of the discussion varies; a minimum of seven minutes and maximum 13 min. We asked the subjects to remove any time-measuring devices in their possession such as watches and smartphones. During the discussion, the subjects wore the wearable devices to record utterances. The topic of the first discussion is “What is the point you want improved at your university?” and that of the second discussion is “How do you involve children in learning programming?”

The DiAna-AD system is applied to both the first and second discussions. Additionally, the groups are notified about termination from the system respectively based on the criteria of termination. The subjects are required to wrap up the ongoing discussion once an end notification is indicated by the notification device.

We set time limitations to shorten or extend the duration of discussions. Even if the group is inactive from the beginning of the discussion, a certain time should be spent for the discussion to occur successfully. On the other hand, even if the discussion is continuously active, it must be wrapped up in a certain time.

After both discussions, we carried out a subjective evaluation questionnaire about the discussion. The questions in the subjective evaluation were the same as asked in the experiment of satisfaction.

Furthermore, we showed the subjects the duration times of each discussion after they answered the questionnaire of the second discussion. The mechanism of adjusting the time according to the activity of discussion by the system was also shown to the subjects. Finally, a questionnaire and an interview on the evaluation of the DiAna-AD system was answered by the subjects.

5.2 Result of the Experiment

We designated the five groups as G1 to G5. Table 1 shows the duration time of the discussions using the DiAna-AD system.

Table 1. The duration time of the discussions
Full size table

The system detected that the discussions of G1 to G4 were active and adjusted the duration time of discussions to the maximum. Table 2 shows the answers to the question “Do you want to shorten or extend the discussion time?”; it verifies that the adjustment was appropriate. The scores in the table are as follows: 1 means “want to shorten strongly,” 2 means “want to shorten,” 3 means “moderate,” 4 means “want to extend,” and 5 means “want to extend strongly.” Each answer is the result collected from three subjects in each group.

Table 2. The answer to the question “Do you want to shorten or extend the discussion time?”
Full size table

12 out of the 30 answers are “want to shorten” the discussion where the duration time was adjusted by the DiAna-AD system. 12 answers are “moderate,” and six answers are “want to extend”. Although the discussions are already extended, the feeling of the subjects differed in extending or shortening the duration time of the active discussions.

Tables 3 and 4 show the number of times that the termination criteria were met every 20 s during the discussion. It indicates that the end notification was made when the value in the brackets is three or more between seven minutes and 13 min. In this experiment, all the groups met the criteria multiple times, but the values are less than three. Several groups met the continuing criteria of three times in less than seven minutes; however, the discussion kept going.

Table 3. Classification of the end notification in the first discussion.
Full size table
Table 4. Classification of the end notification in the second discussion.
Full size table

Table 5 shows the voting results of the first discussion on the question “Do you want to continue talking?” It indicates that the willingness to speak decreased in the vote of the first nine minutes. However, it also revealed that the willingness was recovered in the vote of the first 12 min in several groups.

Table 5. The answers to the question “Do you want to continue talking?” in the first discussion (“Yes”/“No”)
Full size table

In the questionnaire on the evaluation of the DiAna-AD system, the question “Are you satisfied with the system’s decision?” was asked. The scores in the question are as follows: 1 means “not satisfied,” 2 means “partly not satisfied,” 3 means “moderate,” 4 means “partly satisfied,” and 5 means “satisfied.” The average score of the question is 4.27, and the standard deviation is 0.85. 13 out of 15 subjects answered “satisfied” to the decision by the DiAna-AD system. One subject answered “not satisfied.” However, it was found that the subjects in general accepted the decision criteria of the system.

5.3 Discussion

This research aimed at improving participants’ satisfaction in a discussion by adjusting the duration time of the discussion. Therefore, in the experiment of the proposed system, we set the same agenda of the experiment on satisfaction to the first discussion. The degree of satisfaction with a discussion was asked from 0% to 100% in 10% increments. Regarding the experiment on satisfaction, the average degree of satisfaction was 70%, and the standard deviation is 21%. On the other hand, for the experiment of system, the average was 65.3% and the standard deviation was 15.9%.

In addition, the average score for the question “Do you want to shorten or extend the discussion time?” in the experiment on satisfaction was 3.20 and the standard deviation was 1.05. Additionally, the average score in the experiment of the system was 2.67 and the standard deviation was 1.01. A correlation was found between the degree of satisfaction in a discussion and the answer to the question “Do you want to shorten or extend the discussion time?” of the first discussion in the experiment of the system (r = 0.61, p < 0.05).

From the result of the interview, it can be inferred that it is painful to keep participating in an inactive discussion. Hence, it is considered necessary to reduce the duration time of inactive discussions. On the other hand, there is a widespread opinion that “it was not good that the discussion time was excessively extended” by the subject who answered “not satisfied” to the decision of the system. Moreover, it is crucial to consider the degree of extended duration time of active discussions.

6 Conclusion and Future Work

In this research, we designed a DiAna-AD system that adjusts the duration time of discussions in face-to-face environments. The system aims to increase participants’ satisfaction in a discussion by adjusting the duration time based on the discussion. The system utilizes the nonlinguistic acoustic information obtained from participants. We investigated the influence of the system on the participants. The experimental result of the proposed system revealed that there are no significant differences in the satisfaction of the participants. However, there is an effective adjustment of the duration time of inactive discussions.

In future work, we intend to review the degree to which duration time needs to be adjusted in active discussion. Finally, we shall also attempt to contribute to increasing participants’ motivation to repeat discussions to improve discussion skills.