1 Introduction

In a traditional group meeting, group members working in close physical proximity have access to a large amount of information about one another. This information is obtained directly through communication and indirectly through observation of shared artifacts (Beaudouin-Lafon 1994). Computers are being used more frequently to assist in cooperative tasks (Ellis et al. 1991). When people collaborate via computer mediation, this information and the opportunity to access it are reduced. Increasing the amount of information about group availability in a computer mediated collaborative support system should increase the group’s ability to complete a task or perform higher quality work on a group task. When people work together, they share a task on which they cooperate, based on one or more artifacts, and a social context. Supporting an awareness of the activity of other members in a collaboration team and of changes in the shared work materials is very important in collaborative work systems (Brown and Duguid 1994). Moran and Anderson suggest that awareness information is important because it provides information about availability in an indirect way (Moran and Anderson 1990). People can infer information about disposition and commitment from interactions in the work environment (Spring and Vathanophas 2003). Generally, people pay attention to the activities of others. The presence and behavior of other people helps to define the meaning of situations for an individual and can have an influential effect on his or her behavior, attitudes, and feelings during interactions in group-work situations (McGrath 1984).

A group decision support system (GDSS) has been defined as “an integrated computer-based system to facilitate the solution of unstructured or semi-structured tasks by a group that has joint responsibility for performing the specific task” (DeSanctis and Gallupe 1984). Power and Kaparthi (2002) define a web-based decision support system as a DSS built with web technologies so that the DSS users can access it with web browsers deployed on corporate intranets to support internal business processes or they can be integrated into public corporate web sites to enhance services for trading partners. The goal of a web-based GDSS is to achieve a final group decision with a high level of quality and an effective consensus of needs. This research outlines issues and opportunities related to the use of group awareness tools to improve collaboration in online collaborative work environments by providing information about activity, availability, and contributions of group members, particularly as it relates to the state of the artifacts. The awareness information is presented to group members when they work together on given group tasks. We report results of a controlled experiment that examine how group awareness information affects various aspects in light of a web-based collaborative system. Our primary goal for the study was to make group awareness information available on the periphery, to investigate the impact of group awareness tools in supporting a web-based collaborative authoring task and to examine the linkages among the collaborative situation, tools, people, task, and the nature of distributed collaborative work in the experimental settings. We expect that the study will help guide possible alternative trajectories for developing the concepts of group awareness information in a web-based GDSS, building the new features of awareness tools in a web-based GDSS, and finding out the ways to accomplish several attributes in an online collaborative work system.

The remainder of the paper is organized as follows: Section 2 presents some additional detail about various aspects of awareness information and types of awareness. Section 3 addresses related work. Section 4 describes the design and implementation of group awareness information in a web-based GDSS in more detail. Section 5 describes the methodology used in the experiment that was performed. Section 6 describes the experimental study and the results. Section 7 presents a conclusion.

2 Types of Awareness

In the context of CSCW systems, a group can be seen as a number of individuals who interact directly or through shared artifacts and who perceive themselves as a group. Group awareness, therefore, can be broadly defined as consciousness and information of various aspects of the group and its members (Gross and Traunmueller 1995). Group awareness is available via technical support in situations of computer-mediated communication or computer-supported collaborative learning. Gutwin and Greenberg (1995) define synchronous group awareness as the up-to-the-minute knowledge of other people’s activities that is required for an individual group member to coordinate and complete their part of a group task. They also provide four types of awareness in group work. Informal awareness is a general knowledge of who is around and what these persons are doing, and what they are going to do. Informal awareness is a prerequisite for spontaneous interaction. They are the kinds of things that people know when they work together in a face-to-face environment. This kind of awareness can facilitate casual interaction (Gutwin et al. 1996a, b). Social awareness is awareness about the social situation of the group members, i.e., awareness about what they are doing, what they are talking about to others, if they can be disturbed, whether another person is paying attention, their emotional state, or their level of interest, etc. Social awareness is maintained through conversational cues such as through non-verbal cues like eye contact, facial expression, and body language. Social awareness is sometimes described as a sense of being aware of who is available for collaboration (Carroll et al. 2003). Group-structural awareness relates to the knowledge about people’s roles and responsibilities, their positions on an issue, their status, and the state of various group processes. Workspace awareness is the collection of up-to-the minute knowledge a person holds about the stage of another’s interaction with the workspace. It is information about other participants’ interactions with the shared space and the artifacts it contains. Workspace awareness emphasizes the role of the workspace in a collaborative activity. Ishii et al. (1994) emphasize the need to bridge the gap between social and workspace awareness by proposing gaze awareness which they define as “the ability to monitor the direction of a partner’s gaze and thus his or her focus of attention”. As such, gaze awareness is very similar to the social awareness mentioned above. Sohlenkamp and Chwelos (1994) distinguish between asynchronous and synchronous awareness. Concerning asynchronous awareness they claim that “users should be able to determine when shared artifacts have been changed by others, determine how those artifacts have changed, and determine when and where others have left messages for them”. It sometimes refers to change awareness or asynchronous change awareness of artifacts, which is defined as the ability of individuals to track the asynchronous changes made to a collaborative document or graphical workspace by other participants over time (Tam and Greenberg 2005). These types of information refer to the workspace awareness mentioned above. Concerning synchronous awareness they claim that “users should be able to obtain some idea of what co-workers are doing, ascertain a co-worker’s availability for contact, control their own level of availability, control information about themselves which is broadcast to others, know when shared documents are in use by others, and know exactly what others are doing in the course of a shared editing session”. These types of information refer to workspace as well as informal awareness mentioned above. Prinz (1999) distinguishes between “task-oriented” and “social” awareness. Task-oriented awareness is “focused on activities performed to achieve a specific shared task” and social awareness refers to “information about the presence and activities of people in a shared environment”. Gross et al. (2005) claim that task-oriented awareness refers to workspace awareness, and social awareness refers to informal awareness.

In the social sciences, empirical studies of human behavior have revealed several aspects of awareness. Group awareness has been defined as “a specific set of behaviors as characteristics of intimate, primary groups and maintains that these behaviors will occur more often in those groups that have attained an enhanced level of (the group’s) self-awareness” (Barker 1991). Group awareness also refers to information received by members of a group about the other group members, about mutually employed objects, and about current group processes, in order to efficiently carry out certain tasks (Mullen et al. 2003). Task-specific awareness of the work process can be “demonstrated by the adequate description of the used strategies (consciously monitoring and regulating these strategies), and by detailed reports on the difficulties in understanding” the task (Etelaepelto 1993). This aspect has been further investigated in the domain of education. Situation awareness is defined as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (Arvaja et al. 2002). Situation awareness has been analyzed for tasks that require integration of complex and dynamic multi-sensory information arrays. Perception, information processing, decision-making, memory, learning and performance of actions have important consequences for situation awareness (Angiolillo et al. 1993). The concept of activity awareness subsumes situation awareness, defined informally as “knowing what is going on around you” (Endsley 2000). Activity awareness is the awareness of project related work that supports group performance in complex tasks. Objective self-awareness refers to the process of taking oneself as the focus of one’s own attention, or becoming aware of oneself. It can be situation induced by the presence of an audience, a mirror or a video camera. Self awareness in conjunction with some salient behavioral standard may lead the individual to begin a matching standard process (Mullen and Goethals 1987).

This paper has considered both the notion of awareness in CSCW and the social sciences for the design and implementation of an application, group awareness information, and a process for group decision making. The activity values are designed based on using the concepts of group awareness, situation awareness, and activity awareness. The availability values are designed based on using the notion of social awareness. The commitment/disposition values are designed and measured by using the concept of situation awareness, group awareness, and group-structural awareness.

3 Related Work

Collaborative systems utilize a range of techniques that vary from symbolic systems through media spaces to support distributed work groups through access to information that supports generating awareness (Dourish and Bly 1992). In this section, only the work related to symbolic systems, group awareness tools, and similar types of an online collaborative system are described because they are the areas related to the work reported in this paper. Buder and Bodemer (2008) present the development of augmented group awareness tools that take mutual user ratings of their online discussion contributions as input, aggregate these data, and visually feed the data back to the members in real time, thereby informing participants about how the group as a whole perceives their contributions. Their group awareness tool was experimentally tested in a CSCL scenario using online controversies about a physics domain. The result based on their experimental study revealed that groups using an augmented group awareness tool showed higher performance in terms of group decisions and individual correctness than unsupported discussion groups. Kimmerle and Cress (2008) address the use of a group awareness tool to provide social awareness information by presenting personal information as well as photographs of the involved team members. They studied the role of group awareness and self-presentation within the information-exchange dilemma situation. They studied three different group-awareness conditions in a dilemma situation and investigated the personality variable of protective self-presentation (PSP). They observed the way in which people with high and low scores in PSP reacted to the group awareness information provided (Kimmerle and Cress 2008; Kimmerle et al. 2007). Dehler, et al. provide collaborative learners with a group knowledge awareness tool that presents visualized information about a partner’s as well as the learner’s own knowledge (Dehler et al. 2009, 2010). The knowledge awareness visualization provided learners with self-assessed knowledge, which was displayed next to the topic list separately for each dyad member. They use a symbolic representation, small green boxes and white boxes, next to each paragraph to represent knowledge and deficits, respectively. Each paragraph that participants collaboratively work on is accompanied by two boxes, with each being either white or green. The combination of these two boxes resulted in paragraphs of shared knowledge (two green boxes), shared deficit (two white boxes), complementary partner knowledge (participant box white and partner box green), and complementary participant knowledge (participant box green and partner box white). Their study results showed that the GKA tool could guide learners in their collaboration and, more specifically, in designing their communicative acts. Engelmann et al. (2009) present a specific group awareness approach for CSCL settings, namely knowledge awareness. They used knowledge awareness tools to visualize context information in a salient way that enables learners to easily detect and utilize it. All knowledge awareness tools that they used to visualize context information or knowledge awareness information were found to be effective, i.e., all tools could foster knowledge awareness and led to increased learning performance. Spring and Vathanophas (2003) addresses the use of a social awareness tool to improve collaboration by providing information about contributions and activity of team members, particularly as it relates to the state of the artifacts. The tool captures how active people are, what they are currently doing, and what is happening to the shared information space. They present an awareness tool in CASCADE (Computer Augmented Support Collaborative Authoring and Document Editing). CASCADE was designed to allow groups of people to work together on documents. One goal of CASCADE was to reduce the cognitive overhead in the authoring of structured documents by employing a variety of information streams, augmented processes and software tools. The awareness information presented is generated from user activity and tied to individual documents. It is constantly updated based on the activity of group members. In conclusion, they report that the use of an awareness tool decreases the quality of the work effort and the number of group member communications. The results contradict some of the theoretical predictions. Jongsawat and Premchaiswadi (2009) present a prototype of group awareness information in a web-based GDSS. They proposed and designed three factors that group members use in assessing others socially in an online collaborative work situation. These three factors consist of activity, availability, and commitment/disposition information. They proposed an experimental design for online collaborative work in order to study how group awareness information impacts the quality of the work effort on a given task, group decision making by the members in the same group and different groups, the communications among group members in the completion of an online collaborative authoring task, the cohesiveness among group members in a web-based GDSS, and the commitment/disposition to engage each member of the group (Jongsawat and Premchaiswadi 2010, 2011).

4 Implementation of Group Awareness Information in Web-Based GDSS

We conducted two pilot studies before we did the experiments. In the first pilot study, we began by asking four lecturers, two from the Department of Computer Engineering and another two from the Graduate School of Information Technology at the SIAM University, for help in developing some operational definitions for group awareness information to be used in a web-based GDSS and as well as suggestions for investigating some factors that were consistent with capturing awareness information from a user and presenting such information to a user during group work. Our group of lecturers participated in a brainstorming session, which lasted for three working days. During this period of time, they discussed the concepts and definitions for group awareness information used in an environment for a web-based GDSS, listing those relevant factors that corresponded with group awareness (by considering several factors used in the literature and then thinking of new factors) and giving these factors the proper definitions used in the web-based GDSS area. They also discussed the possibility of using these factors, and assigning a score to the factors using a Likert scale of 1–5 (not appropriate, less appropriate, fair, appropriate, and most appropriate). As a result of this study, we defined the operational definitions for the terms and selected the top seven factors representing awareness information for the group (Jongsawat and Premchaiswadi 2009). They consisted of activity, availability, commitment and disposition, agreement and disagreement, brightness, self-presentation, and feedback. Nevertheless, a group of lecturers refined all these factors once again and selected only the top three factors for the next stage because they emphasized that too much information might distract participants and might sometimes yield incorrect results for translation.

In the second pilot study, the same group of lecturers and one additional professor from the faculty of social sciences participated in this study. They participated on a weekday for brainstorming and investigating the parameters of each factor. They were asked for arguments about the appropriateness and possibility for using the parameters and consideration of symbols for representing each factor as a visual display. Several questions arose, “Which factors should be presented to a user as a visual display?”, “Would numbers, graphics, or images be preferred for representing each factor?”, “Given images, what types of images were best for presentation and would color, contrast, or resolution best show each factor?”, “Would the participants’ privacy be violated if we used pictures of themselves for representing their thoughts and behaviors during group work?”, etc. Furthermore, the lecturers were asked to consider other important issues such as a participant’s privacy, possibility of success in the use of the selected parameters, possibility for the creation, etc. After we received the final approach for measuring each factor, we recruited twenty graduate students to evaluate the use of the selected parameters. Before the evaluation process began, our lecturers explained all the subjects and the evaluation criteria to them in more detail, what we wanted them to do, the purpose of evaluating the parameters, and finally allowed them to ask questions about unclear issues in order to make sure that they clearly understood all aspects of the proposal, definition of terms, and the actual meaning of each criterion for evaluation. There were five main issues asked related to each factor, including parameters and its symbolic representation in the questionnaire: (1) easily understood, (2) appropriateness for use, (3) correctness, (4) symbolic representation, and (5) privacy using a Likert scale of 1–5 (very low, low, medium, high, and very high). The last part of the questionnaire contained several open-ended questions to let them relate comments on the issues. The results were presented in a previous study (Jongsawat and Premchaiswadi 2009, 2010). The results and comments obtained from participants were taken into consideration for adjusting some parameter values and their weight, using symbols for representation, and refining the awareness formulas. In conclusion, activity and availability were selected as two factors for transforming the captured data into visual data. They appear as a visual display representing cumulative data and changes accordingly when the group begins working together until the end of the experiment. The last factor, commitment/disposition information, is captured during collaborative work and summarized when they finish group work.

The two pilot studies then sought to examine how each group awareness factor would be selected and formed, what kinds of group awareness information would be present or absent for the user during group work in a web-based GDSS, how it would impact the completion of a collaborative authoring task, and how it would impact the members of the group with the process.

4.1 Three Factors Group Awareness Information

4.1.1 Activity Information

Activity Information is information about how actively each group member or group is working. It refers to the level of each group member or group’s activeness. The activity tool captures how active group members are during periods of group work. The activity values are designed based on using the concepts of activity, group awareness, and situation awareness. They are cumulatively measured through the direct and indirect communications among participants and lecturers and through artifacts, which are based on both qualitative and quantitative measures. In this research study, there are two types of activities that are measured and presented in the experiments. First, individual activity is measured through each individual activity or activeness during the group work task. Second, group activity is measured through the cumulative activity of all members in the group. The formulas are shown in (1) and (2).

$$\begin{aligned}&\displaystyle \text{ Individual } \text{ Activity } = \text{ LSavg } + \text{ SSavg }&\end{aligned}$$
(1)
$$\begin{aligned}&\displaystyle \text{ Group } \text{ Activity } \!\!=\!\! \mathop \sum _{\mathrm{i}=1}^\mathrm{n} \frac{\left( {\text{ LSavg }_\mathrm{i}} \right) \left( {\text{ W }_\mathrm{i}} \right) *100}{\text{ n }}\!\!+\!\!\mathop \sum _{\mathrm{i}=1}^\mathrm{m} \frac{\left( {\text{ SSavg }_\mathrm{j} } \right) \left( {\text{ W }_\mathrm{i}}\right) *100}{\text{ m }}&\end{aligned}$$
(2)

Note that LSavg (lecture score) is the average score of each group member that is evaluated by the lecturers. SSavg (student score) is the average score of each group member that is evaluated by other group members who are working together in the same group. Each individual is not allowed to assign a score to him/herself. n is the number of lecturers who assign the scores to each group member and m is the number of students in a group who assign the score to other group members. \(\text{ W }_\mathrm{i}\) is a uniform distribution of the weights. For example, if the number of members in a group is 5 then \(\text{ W }_{1}\) is equal to 0.2, which is also equal to \(\text{ W }_{2},\,\text{ W }_{3},\,\text{ W }_{4}\), and \(\text{ W }_{5}\). For the lecturers’ evaluation, there are three issues; (1) the quality of a task produced by each group member, (2) the work effort produced by each group member, and (3) the participation for each individual in solving a group task. The lecturers are asked to evaluate each member using these three issues. For the group members’ evaluation, there is only one issue; activity evaluation (e.g., “What score would you give to group member 1 for their activity (activeness) level?”). Each group member is asked to assign a score to other group members in the same group and lecturers assign a score to all students (group members) at the beginning and midpoint of each phase of NGT sessions in the experiment. Both of them use a scale of 1 (being lowest score: very sluggish) to 10 (being highest score: the most active) for an evaluation. In this research, the number of lectures in both experiments is two (n = 2) and the number of group members in each group in both experiments is five (m = 5).

4.1.2 Availability Information

Availability Information is information about who is around and available for others in the group at any point in time. It is designed based on using the notion of social awareness. It is operationally defined as the number of minute(s) connected divided by the expected number of minutes connected (determined per task) + a rating of the delta of availability + a self assessment rating of how busy each group member is + a self assessment rating of how busy each group member is + a degree of activity. The delta factor indicates whether group members are becoming more or less available over a set of time periods. That is, it indicates whether availability is increasing or decreasing. The formulas are shown in (3) and (4).

$$\begin{aligned} \text{ Availability }&= \frac{\text{ MinConnected }}{\text{ ExpMinConnected }} +\Delta \text{ P }+\text{ CLR }+\text{ ACTdeg }\end{aligned}$$
(3)
$$\begin{aligned} \Delta \text{ P }&= \frac{\text{ p }_\mathrm{n} }{\frac{\text{ p }_\mathrm{i} +\cdots +\text{ p }_\mathrm{{n-2}} +\text{ p }_\mathrm{{n-1}} +\text{ p }_\mathrm{n} }{(\text{ n }-1)}} \end{aligned}$$
(4)

Note that MinConnected is the number of minute(s) that a group member connects to the system to complete a task over a given period of time. It is not recorded if the data are not updated or changed within a given period of time. ExpMinConnected is the number of minute(s) that a group member is expected to spend on a task over a given period of time. Current Life Rating (CLR) is a self assessment. Group members rate their current life using a Likert scale of 1–5 (very busy, busy, normal, comfortable, and very comfortable), respectively. ExpMinConnected and CLR are obtained from individuals who fill out a pre-experiment questionnaire that asks about these two variables. \(\Delta \text{ P }\) is the number of minutes connected in time n over the number of minutes connected in any period of time from i to n for a particular user where \(\text{ P }_\mathrm{i}\) is the number of minutes connected in time i (starting period); \(\text{ P }_\mathrm{n-2}\) is the number of minutes connected in time \(\text{ n }-2\); and \(\text{ P }_\mathrm{n}\) is the number of minutes connected in time n (ending period). ACTdeg is the degree of activity. It is obtained from the activity calculation shown in the previous formula using a scale of 1–5 (not active, less active, fair, active, and very active), respectively. For example, group member 5 receives the highest activity score so that he or she will receive a degree of activity score of five, ACTdeg = 5. We accredit the group member who receives a higher score of activity value as a person who is more available to the others. The smiley icons with graphical representation of each individual are derived in the same way as described for activity information. If the availability value is low, it means that a group member is less available for other members because he or she probably has a lot of things to do at the moment and has no time for other tasks.

4.1.3 Commitment/Disposition

Commitment/Disposition is defined as information about how willing a group member is to do more work, which includes how he or she views the task, group work, or if the members in a group work positively or negatively. How much commitment/disposition does each group member have for accomplishing a group task? Based on the CSCW’ point of view, the commitment/disposition value is designed and measured by using the concept of group-structural awareness. From a social sciences’ point of view, they are designed based on using the concepts of group awareness and situation awareness. The formulas are shown in (5) and (6).

$$\begin{aligned} \text{ Commitment }/\text{ Disposition }&= \text{ GA }+\text{ NoOfComment }\left( \text{ s } \right) +\text{ NoOfResponse }\left( \text{ s } \right) \end{aligned}$$
(5)
$$\begin{aligned} \mathbf{GA}&= \sum \limits _\mathrm{i=1}^\mathrm{n}\frac{(\text{ GS }_{1}) {(\text{ W }_{1})*100}}{\text{ n }} \end{aligned}$$
(6)

Note that GA is Group Assessment. It is calculated as shown in (6). NoOfComment(s) is the number of comments or suggestions that a group member generates to other members during a working period through a comment tool or chat room. NoOfResponse(s) is number of responses, or answers that a group member posts to other members using the chat room or comment tool during a working period. GS is a group score that is generated by each group member by rating his or her group for commitment/disposition for engagement using a scale from 1 to 5 (lowest, low, medium, high, and highest). \(\text{ W }_{\mathrm{i}}\) is a uniform distribution of the weights. For example, if the number of members in a group is 5 then W\(_{1}\) is equal to 0.2, which is also equal to \(\text{ W }_{2},\,\text{ W }_{3},\,\text{ W }_{4}\), and \(\text{ W }_{5}\). The value n is the number of members in group. There are questions in the pre- and post-questionnaires about how users feel toward each section of a given task, how they feel about other group members, and so on. The irrelevant comments and responses to the task are eliminated by a lecturer; e.g., they are not counted. If a group member generates more comments and responses to others and/or has a positive attitude towards the task or other members in the group, he or she tends to have a high commitment for accomplishing a group task.

This information endeavors to capture the feelings of a person toward other members, the behavior of the person, a particular content aspect in the work space, and how much a person contributes to a group task using the number of comments and responses in group work, which have been made, by the members in the same group. This information is not shown as a visual display and is unknown to the users during group work. It is summarized at the conclusion of the work.

4.2 Screenshots of Activity and Availability Information

The formulas attempt to capture some sense of how active and how engaged, an individual is with respect to group work in a web-based group decision support environment. Figure 1 shows the member activity as a visual display when members click on the member activity’s link. The higher the activity value of a group member, the more active he or she has been doing his or her work. In this research, based on the results obtained from the previous two pilot studies, we avoided using score numbers, statements, or a member’s picture reflecting their activity or availability information because bias among group members may occur and yield negative results. We decided to use small icons with percentages as suggested by the lecturers and voted by the participants to present their awareness information. For the percentage calculation process, the highest score in a group is set to 100 % and the lower score is compared to the highest score. For example, in group I, if member 5 receives the highest score (8.83 out of 10) and member 4 receives a score of 4.84 out of 10, it means that member 5 receives 100 % (most active member in group I) and a member 4 obtains 50.74 % (50.74 % active compared to member 5). The program selects the smiley icons with graphical percentage as calculated and represents the status of each individual according to the rank they received. In this manner, we anticipate that the bias among group members should be relieved and the group members with lower scores would intuitively like to pursue or overcome the other group members who have higher scores. It means that they should pay attention and do more work in order to pursue other members who receive a higher score. Figure 2 shows the group activity as a visual display when members click on the group activity’s link.

Fig. 1
figure 1

The visual display of member activity in web-based GDSS

Full size image
Fig. 2
figure 2

The visual display of group activity in web-based GDSS for all six groups

Full size image

Figure 3 shows the member availability as a visual display when members click on any member’s activity’s link.

Fig. 3
figure 3

The visual display of member availability in web-based GDSS

Full size image

4.3 Web-based Group Decision Support System

A web-based GDSS, in this paper, is designed and implemented by following the Nominal Group Technique (NGT). The NGT is a structured decision making technique widely used both in industry and academia as a tool to aid in planning and decision-making processes. Paulus’ cognitive theory of group creativity (Paulus et al. 2000) suggests conditions under which cognitive stimulation can be observed in groups. The theory suggests that sharing ideas within a group stimulates additional generation and association of ideas. The NGT provides an advantageous environment to stimulate creativity since it allows for the silent generation of ideas, which then are shared with the group (Pilar et al. 2006). The technique has been recognized as a way to equalize participation, tolerate conflicting ideas and stimulate the generation of ideas by sharing them with the group (Delbecq et al. 1975). When groups generate ideas using brainstorming, the competing demands of generating their own ideas and also processing the ideas of the others has been observed to divide the participants’ attention. Paulus and Yang (2004) recognize that if, after brainstorming, individuals are provided with an opportunity to generate additional ideas on their own; the impact of cognitive stimulation may become more evident. The NGT seems to provide the appropriate environment for cognitive stimulation while encouraging creativity. The NGT session in a web-based GDSS consists of the following phases:

  1. (1)

    presentation of the task statement

  2. (2)

    fill out a pre-experiment questionnaire

  3. (3)

    idea generation

  4. (4)

    brainstorming

  5. (5)

    round robin (optional in this research)

  6. (6)

    clarification and preparation of each idea/solution

  7. (7)

    scoring and ranking all ideas/solutions

  8. (8)

    final discussion

  9. (9)

    fill out a post-experiment questionnaire

In the web-based GDSS used for this experiment, a script was created so that all sessions would be as consistent as possible. The NGT was primarily designed based on using the concepts of both a cooperation process (phases: 3 and 6) and a collaborative process (phases: 4 and 8). In cooperation, the learning is done by individuals, who then contribute their individual results and present the collection of individual results as their group product. Learning in cooperative groups is viewed as something that takes place individually (Roschelle and Teasley 2010). In cooperation, partners split the work, solve sub-tasks individually and then assemble the partial results into the final output. In collaboration, partners do the work together (Engelmann et al. 2009). Collaboration is a process by which individuals negotiate and share meanings relevant to the problem-solving task at hand. Collaboration is a coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem. A GDSS web site was specifically designed to conduct the NGT group work through a computer (web browser). Process support for the participants was provided through the site by instantaneous instructions on each step of the process. The web site has three main features, a built-in pre-session questionnaire, working area session and tools (edit/ ranking pages, comment/ communication tools), and a built in post-session questionnaires. Initially, group members are asked to complete the pre-experiment questionnaire when they enter the web site. Once the questionnaire is completed, they are directed onto the second part, which is a working area session, starting with instructions for group work. The group facilitator (lecturer) explains the instructions and demonstrates the application in the classroom. Next, the decision process begins with the idea generation phase. During this phase, they individually login to their session with the given username and password and spend their time generating ideas to perform their given tasks on an individual web page. Next, they enter the brainstorming phase. In this phase, they are allowed to communicate with each other by using a chat room (interaction screen) provided for each group. They can ask questions and discuss ideas with other members of the same group. They are asked to enter the other members’ workspaces in order to collaborate and help each other improve each individual’s solutions, which is becoming more or less an important part of the final solution for the group. Each group member can also use a comment tool for sending comments or recommendations to other members in the same group. Next, they enter the clarification and preparation for each idea/solution phase. Each individual gets back to work in their own workspace to develop their own solutions to a task by considering the comments and recommendations obtained from the others. All members have time to refine their solutions for a task by working in their own session such as the option to clarify, reword, add or group the contents or ideas if needed. During this phase, every member spends most of the time trying to improve their individual solutions. The comment tool and chat room are not available during this phase. Then, each member has to select his/her preferred solutions to be scored by the other group members in the same group. Next, they are directed to the scoring and ranking phase. They are asked to provide a score for the other group member’s listed solutions. Each group member cannot assign a score to himself/herself. He or she does not know the score the others assigned to his or her final solutions until the end of this phase. The total score for each individual in a group is calculated based on the sum of the scores he or she receives from other members in the same group. An online evaluation form is provided to each individual. Each item on the form uses a scale of 1–10 (one being worst and ten being the best). Next, they are directed to the final discussion phase. The aim of this phase is to investigate the final solution for the group. In the experiment, the task statement for each individual and the group is the same. It means that each group member performs the same task and finally after the vote they may agree that the best solution (highest score) for the task should sometimes be used to represent the final solution for the group. However, it is not necessary for the solutions with the highest score to be selected as the group solution. In this phase, the group members are asked to discuss with each other which parts of the solutions of each member should be included into the final group solution. For example, 60 % of the final group solution may come from the best solution and the rest may come from the other solutions obtained from other group members. They are asked to discuss and reach a compromise on all these matters. After all group members agree with the final group solution, a group leader posts the final solution on the final group’s solution workspace that is provided for each group. After that the group’s solution is then sent to three judges to be scored after finishing the experiment. Finally, participants fill out the post-session questionnaire. Pre- and post session questionnaires are used to capture some dependent variables from individuals. We obtained some variables from the pre-session questionnaires such as current life rating (CLR) and expected minutes connected in order to generate availability information. We used some variables from the pre- and post-session questionnaires for testing hypotheses. The duration of each session in the experiments is shown in Table 1. Note that we repeated the experiment 2 times (1st experiment: morning session and 2nd experiment: afternoon session; using different groups of participant).

Table 1 NGT session in a controlled experiment
Full size table

5 Methodology

A controlled experiment was undertaken to examine the effects of the use of a web-based group decision making technique with and without group awareness information. The main purpose of this study was to analyze the following factors in a web-based GDSS with and without group awareness information: the quality of the work effort and a given task, group decision making by the members in the same group and different groups, the communications between group members for the completion of an online collaborative authoring task, the cohesiveness among group members in a web-based GDSS, and the composition/disposition of engagement for social group work and to make the conclusions from the results obtained from the experimental settings.

5.1 Participants

The controlled experiment: Sixty undergraduate computer engineering students, who took and passed a database course in a previous semester, were recruited to participate in the two experiments. The two experiments were started and finished within one day. We used the same set of sequential steps provided for the two experiments. This was meant to insure that the two experiments were identical. The first experiment started at 8.00 a.m. and finished at 12.00 p.m. and the second experiment started at 13.00 p.m. and finished at 17.00 p.m. Students were randomly formed into two groups (group A: morning experimental session and group B: afternoon experimental session) of thirty in each group. In each experiment, subjects were randomly assigned to sub-groups with five members in each group. We formed six sub-groups. During group work, all members in the first three groups (group I, II, and III) could see group awareness information and the members in the last three groups (group IV, V, and VI) could not see any group awareness information. Demographic data were collected on age, gender, and the background of the participants. Specifically, age ranged from 19 to 24 years. With respect to gender distribution, 34 % of the participants were female and 66 % male.

During group work, all group members with and without group awareness information performed the same task, provided by the group lecturer, in each phase and used the same communication tools, such as a comment tool and a chat room. The difference between the two groups was that the members in the first three groups could see activity and availability information as a visual display during group work but the members in the last three groups performed their tasks without any awareness information.

5.2 Materials

Participants in the experiment used their own personal computer with a standard web browser to perform given task in a web-based GDSS. The group facilitator and participants mainly communicated each other via web board systems. We provided the tools on the web-based GDSS site such as an online chat room and a comment system to the participants for communication during the experiment. The web-based GDSS application was developed using a wiki-style, WikkaWiki (http://wikkawiki.org).

5.3 Hypotheses

5.3.1 Group awareness information and the quality of the work effort and a given task (quality task result)

Our expectation was that group awareness information (activity/availability) should stimulate the group members to generate more ideas, comments, recommendations, communications, and finally produce a better quality task result than the group members without group awareness information. The mean quality score on a task produced by group members with group awareness information should be higher than the mean score on a task produced by the group members without group awareness information.

Question 1

Do group members achieve a higher quality score for a given task in a web-based GDSS with group awareness information than a group without group awareness information?

  • \(\text{ H }_{0}\): There will be no significant difference between the mean quality score of a task solution produced by groups working with and without group awareness information in a web-based GDSS. (\(\text{ H }_{0}: {{\upmu }_{\text{ with } \text{ group } \text{ awareness }}}={\upmu }_{\text{ without } \text{ group } \text{ awareness }})\)

  • \(\text{ H }_{1}\): The mean quality score of a task solution produced by groups working with group awareness information will be significantly higher than the mean score of a task solution produced by groups working without group awareness information. (\(\text{ H }_{1}: {\upmu _{\text{ with } \text{ group } \text{ awareness }}} > \upmu _{\text{ without } \text{ group } \text{ awareness }})\)

5.3.2 Group awareness information and group decision making by members in the same group

We anticipated that the group awareness information (activity/availability) should affect group decision making by members in the same group. The group members with higher activity and availability rates/scores should obtain a higher voting score for their individual task or solutions from the other members in the same group. The results of this research question were summarized based on using the data obtained from the first three groups (with group awareness) only.

Question 2

Does group awareness information effect group decision making by the members in the same group?

5.3.3 Group awareness information and group decision making by members in different groups

We also anticipated that group awareness information (group activity, see Fig. 2) representing the group would affect the members in different groups in the sense of competing with other groups that are more active or obtain a higher activity value than they do. Group activity information can be seen by all members from all six groups.

Question 3

Does group awareness information (group activity) affect the members in different groups in the sense of competing with or overcoming other groups that are more active or obtain a higher activity value than they do for the same given task?

  • \(\text{ H }_{0}\): There will be no significant difference between the mean score of the two questions (in a post-experiment questionnaire) produced by each group member with and without group awareness information in web-based GDSS. (\(\text{ H }_{0}: {\upmu }_{\text{ with } \text{ group } \text{ awareness }}={\upmu }_{\text{ without } \text{ group } \text{ awareness }})\)

  • \(\text{ H }_{1}\): The mean score of the two questions produced by each group member with group awareness information will be significantly higher than the mean score of the two questions (in a post-experiment questionnaire) produced by each group member without group awareness information. (\(\text{ H }_{1}: {\upmu }_{\text{ with } \text{ group } \text{ awareness }}> \upmu _{\text{ without } \text{ group } \text{ awareness }})\)

5.3.4 Group awareness information and the number of communications

The group awareness information (activity/availability) should stimulate group members to produce a higher number of communications in the same group. The number of communications generated is expected to be increased because of group awareness information.

Question 4

Do group members produce a higher number of communications in a web-based GDSS with group awareness information than they do without group awareness information?

  • \(\text{ H }_{0}\): There will be no significant difference between the mean number of communications occurring among group members with and without group awareness information for solving a given task in a web-based GDSS. (\(\text{ H }_{0}: \upmu _{\text{ with } \text{ group } \text{ awareness }}=\upmu _{\text{ without } \text{ group } \text{ awareness }})\)

  • \(\text{ H }_{1}\): The mean number of communications occurring among group members with group awareness information will be significantly higher than the mean number of communications occurring among group members without group awareness information. (\(\text{ H }_{1}: \upmu _{\text{ with } \text{ group } \text{ awareness }}> \upmu _{\text{ without } \text{ group } \text{ awareness }})\)

5.3.5 Group awareness information and the cohesiveness among group members

The cohesiveness among group members is expected to be higher because of group awareness information (activity/availability). The group awareness information should stimulate group members to produce higher cohesiveness in a group. When the number of communications in a group increases, the cohesiveness among group members should also increase.

Question 5

Do group members achieve a higher cohesiveness among themselves for a given task in a web-based GDSS with group awareness information than they do without group awareness information?

  • \(\text{ H }_{0}\): There will be no significant difference between the mean value for cohesiveness occurring among group members with and without group awareness information for solving a given task in web-based GDSS. (\(\text{ H }_{0}: \upmu _{\text{ with } \text{ group } \text{ awareness }}=\upmu _{\text{ without } \text{ group } \text{ awareness }})\)

  • \(\text{ H }_{1}\): The mean value for cohesiveness occurring among group members with group awareness information will be significantly higher than the mean value for cohesiveness occurring among group members without group awareness information. (\(\text{ H }_{1}: \upmu _{\text{ with } \text{ group } \text{ awareness }}>\upmu _{\text{ without } \text{ group } \text{ awareness }})\)

5.3.6 Group awareness information and the commitment/disposition of engagement of each member

Lastly, we expected the commitment/disposition for the engagement of each member of a group to be higher because of group awareness information (activity/availability), a group task, and working collaboratively with others. They should stimulate individuals to have a positive attitude toward the task or other members in the group. Finally, they tend to have a higher commitment for accomplishing a group task and should be willing to do more work.

Question 6

Do group members achieve a higher commitment/disposition of engagement for accomplishing a group task in a web-based GDSS with group awareness information than they do without group awareness information?

  • \(\text{ H }_{0}\): There will be no significant difference between the mean value of commitment/disposition occurring among group members with and without group awareness information for solving a given task in a web-based GDSS. (\(\text{ H }_{0}: \upmu _{\text{ with } \text{ group } \text{ awareness } }=\upmu _{\text{ without } \text{ group } \text{ awareness }})\)

  • \(\text{ H }_{1}\): The mean value of commitment/disposition occurring among group members with group awareness information will be significantly higher than the mean value of commitment/disposition occurring among group members without group awareness information. (\(\text{ H }_{1}: \upmu _{\text{ with } \text{ group } \text{ awareness }}> \upmu _{\text{ without } \text{ group } \text{ awareness }})\)

In the controlled experimental setting, we recognized that relatively little social awareness information would be accumulated over the four hours allocated for the experiment. Group awareness information might not have much influence on group members because of the time constraint for each session. Generally, perceptions of other individuals are developed over a period of weeks and months. However, the controlled experiment was challenging in the sense of how group awareness information would impact the issues under the controlled conditions.

5.4 The Experimental Task Statement and Variables

Each group/member is asked to define the problem statements, functional and nonfunctional requirements, and to develop the first level of an entity relationship diagram for the given group task (A case study for the investment fund management system).

The dependent variables in this experiment were: the number of ideas/solutions generated by each group member, the level of quality of the given task solutions, perceived level of group decision making with and without group awareness information, number of communications among members of the same group, the amount of cohesiveness among group members, and the composition/disposition value for engagement.

We measured the number of ideas/solutions generated by each group member using individual web page files during group work. We measured the level of quality of the given task by using the average score from the three judges. The level of group decision making with and without group awareness information was measured by the average voting score that each group member assigned to the other group member’s tasks and the score obtained from the questions in a post-experiment questionnaire. The number of communications among members of the same group was measured using the log files of the comment pages and the chat room. The amount of cohesiveness among group members was measured based on the number of communications among members and the score obtained from the questions in a post-experiment questionnaire. The composition/disposition value was measured based on the score obtained from the questions in pre and post-experiment questionnaires. The independent variable was the NGT session in a web-based GDSS, which consisted of two types: with and without group awareness information.

6 The Results

The results of the study are as follows. They are summarized in Table 2.

Table 2 Results for research question 1–6
Full size table

Question 1: The quality of group solutions was evaluated by using three independent reviews conducted by outside judges.

The result for research question 1 was statistically significant and we concluded that the group members with group awareness information achieve a higher quality work for a given task in a web-based GDSS than those without group awareness information.

Question 2: The results for research question 2 were made using two different approaches. Both approaches were taken into consideration in arriving at a conclusion for this research question. Especially, we used the results from the second approach to confirm the findings in the first approach. We investigated the first approach based on (1) the relationship between the activity scores and the total score for the task solutions and (2) the relationship between the availability scores and the total score for the task solutions. The total score for the task solutions of each individual was calculated by the sum of the scores he or she received from other members in the same group. Pearson’s correlation was used to reflect the degree of linear relationship between the two variables for each group in the first approach. In the second approach, we determined a conclusion from the two questions in the post-experiment questionnaire, namely, “Do the activity rates/scores effect your decision making for giving a score to the other member’s solutions in the same group? (1)” and “Do the availability rates/scores effect your decision making for giving a score to the other member’s solutions in the same group? (2)” using a scale of 1–5 (1 being lowest impact and 5 highest impact). These two questions in the post-experiment questionnaire were available for groups I, II, and III (groups with awareness information) only.

For activity information, 4 groups out of 6 (Reject \(\text{ H }_{0})\) showed that there was a linear correlation or association between the activity values and the total scores for the task solutions that they received from other members in the same group in the first approach and the mean value obtained from question (1) was higher than the midpoint (\(3.34 > 2.5\)) in the second approach. The results from the two approaches were relevant to each other. The conclusion was that activity information evidently affected group decision making by members in the same group.

For availability information, 5 groups out of 6 (Not reject \(\text{ H }_{0})\) showed that there was no linear correlation or association between the availability values and the total scores for the task solutions and the mean value of question (2) was slightly lower than the midpoint (\(2.33 < 2.5\)). The conclusion was that availability information used in the experiments did not affect group decision making by the members in the same group.

Question 3: The conclusion for research question 3 was made from the two questions in the post-experiment questionnaire, namely, “Does group awareness information representing the group have an impact on you during group work? (1)” and “Would you like to increase your effort to solve the given task if you see information that other groups are more active? (2)” using a scale of 1–5 (1 being lowest impact and 5 highest impact).

The conclusion was that group awareness information representing the group (group activity) did not have an impact on himself/herself during group work. Group activity did not stimulate group members to increase their effort to solve the given task when they saw information that other groups were more active.

Question 4: The definition of communication in this research project is defined as the number of activities or interactions such as posting a comment/recommendation and asking/answering questions using the provided comment tool and chat room that the group members used to respond to each other. Research question 4 was investigated using two approaches. We captured the number of communications during group work in the first approach. Irrelevant information such as greeting messages was not counted. In the second approach, we used the question in the post-experiment questionnaire, namely, “Does group awareness information have an impact on you to communicate more with other members during group work?”. We prepared this question for groups working with group awareness information only.

The results from the two approaches were relevant to each other. The conclusion was that members produced a higher number of communications in a web-based GDSS with group awareness information than they do without group awareness information.

Question 5: The results for this research question were made from the pre and post-experiment questionnaires. The questions, namely, “Do you like to work with other members in a group to solve a given task? (1)” and “Do you feel a positive attitude towards the task or other members in the group when you work together? (2)” were asked in both pre and post-questionnaire. We calculated the average scores from pre and post experiment questionnaires.

Group awareness information did not affect the cohesiveness among group members using the web-based GDSS even when the number of communications in groups with group awareness information was higher than in groups working without group awareness information. The group members with group awareness information tend to communicate more but they do not develop much cohesiveness or interaction among themselves.

Question 6: We determined the results for the last research question using two approaches. First, we used the formula defined in Eqs. (5) and (6) to calculate the commitment/disposition value. Second, we used the question in the post-experiment questionnaire, namely, “How much of a rating score do you give to the group for commitment/disposition for engagement in order to accomplish a group task?”.

The results from the two approaches were relevant to each other. The conclusion was significant in that the groups with group awareness information had a higher commitment/disposition for engagement than in groups working without group awareness information.

A summary of the hypothesis test and an overview of the results obtained from experimental settings are shown in Tables 3 and 4, respectively.

Table 3 A summary of the hypothesis test for all research questions
Full size table
Table 4 An overview of the results obtained from experimental settings
Full size table

7 Conclusions

Group awareness information had a positive influence on improving the work effort on a given task and the quality of collaborative work. Based on the results from the statistical tests, the activity information as a visual display had a positive influence on group members’ decisions who collaboratively worked in the same group. On the other hand, the availability information had no influence on this issue. We found limited support for the use of group awareness information (group activity) to stimulate group members, who worked in different groups, to increase their own effort or their contribution to solve their given group task. Group awareness information did not affect group members in developing much cohesiveness or interaction among themselves. Group awareness information did have an impact on group members to communicate more with other members during group work. It did not have an influence on the group members’ willingness to do more work or to have a commitment with the others to solve a given group task but it had and impact on commitment/disposition for engagement.

In this experiment, the activity values are specifically designed based on using the concepts of group awareness, activity awareness, and situation awareness. Availability information is designed based on using the notion of social awareness. The commitment/disposition values are designed and measured by using the concept of group-structural awareness. Therefore, we can infer that group awareness, activity awareness and situation awareness had a positive influence on (1) the quality of the work effort on a given task, (2) group decision making by the members in the same group, and (4) cohesiveness among team members. Social awareness had a positive influence on (1) the quality of the work effort on a given task, and (4) communications between group members. Group-structural awareness had a positive influence on commitment/ disposition of engagement for social group work.

One main conclusion of the findings is that group awareness information did not simply provide more information to participants, it also changed how they behaved and reacted to the awareness information during group work through working on shared artifacts. The interaction occurring among group members becomes important. Based on the statistical tests from the experimental settings, there were statistically significant results showing that group awareness information had an influence on several issues such as an individual’s level of participation/effort and individual decision making to accomplish a given task, the number of communications among group members, and the composition/disposition of engagement for group work in web-based group decision system. The higher the quantity and quality of these factors, groups were able to build a good team atmosphere during collaborative group work, enhance individual work performance, and produce high quality results for a group task. The purpose of conducting the experimental settings was not only to finding out more about the influence of group awareness information on the issues, but also to shed light on learning mechanisms in relation to cooperation and collaboration in a web-based group decision support system. The impact of group awareness information on issues that we found in this study would help us investigate ways to measure members’ status or behavior during group work and provide us useful guidelines about how to use a symbolic representation for better representing group awareness information. In the next study, we plan to develop tools to measure individual knowledge and represent his/her knowledge through visualized information for others to observe. Group members can also modify their own knowledge representations through shared artifacts during the collaboration. Group awareness information will additionally be measured not only by capturing members’ behavior but also be used as a shared artifact for all group members as well. These kinds of capabilities can possibly lead the members in a group to collaboratively work more efficiently and to produce excellent-quality results in the final stage.