Keywords

1 Introduction

Enterprise modeling (EM) is a powerful way of capturing important information about a company, such as structures, processes and dependencies. It enables a company to identify problems and their causes as well as potential for change. Finally, it helps to prepare for and implement such changes [1]. EM is, however, not only about mapping processes and structures. In the modeling method 4EM [1], it is suggested that EM should start with basic models comprising general goals and problems a company might have. This resembles a brainstorming task involving collecting and capturing knowledge and ideas.

When creating an enterprise model, usually comprising several intertwined sub-models, it is necessary to involve all stakeholders. Participative EM suggests that the modeling be performed by the stakeholders themselves with the support of facilitators representing the experts of the modeling method [2]. The stakeholders are domain experts that directly provide the necessary knowledge in the modeling process.

This paper focuses on tools which may support teams of stakeholders particularly in the area of modeling tasks where brainstorming activities are required. Multi-touch tables appear to be a useful tool for such modeling tasks. In contrast to modeling at a whiteboard or with pen and paper, content can be easily changed and deleted, and, what is of greatest advantage, models can be saved digitally, shared and reused at any time. The differences in handling this tool may also cause differences in the way a group works together. In this paper, a study is presented which examines the influence of tool on the way groups work together. We compared a multi-touch table (MTT) with a whiteboard, the latter representing a traditional tool. We focused on the following research questions: (1) How and to what extent do single team members contribute to the modeling task? This should also show how evenly distributed individual contributions are, depending on the tool. (2) Are there different working styles depending on the tool? The latter question concerns aspects such as task division and coordination. (3) Are there any differences in team performance depending on the tool? The goal of this research is to find out whether the MTT already provides advantages that must be taken into consideration when deciding on a modeling tool. Furthermore, we wanted to look for hints on working styles of whiteboard users that might be transferred to the MTT, e.g. by aspects of function and design of modeling software for the MTT.

In the next section, some background on group work will be presented showing relations to participative EM. Moreover, research on MTT will be described briefly. In the third section, the method of the study is presented, followed by results in Sect. 4. The last section concludes the paper with a discussion of the results, including limitations of the study and implications for future research.

2 State of the Art

2.1 Group Work and Participative Enterprise Modeling

The main reason why we work in groups is that we hope to perform better by gathering performance and ability of several individuals. However, group work may bring both increase and decrease in performance with regard to motivation, individual skills and coordination (see e.g. [3] for more information). The mere presence of other persons can motivate someone to put more effort in a task. Furthermore, in some teams, persons compensate weaker performances of their teammates [3, 4]. On the other hand, persons might be less motivated because they do not see the concrete value of their contribution in the team effort [5]. With regard to individual skill, performance decrease might occur because teammates interrupt the flow of ideas of a person by keeping on voicing their own ideas. However, being inspired by others’ ideas may also lead to new ideas and thus a performance increase [3]. Group work, of course, also requires more effort on coordination the larger a group is, e.g., there has to be an agreement on who is allowed to talk at a certain moment. Lamm and Trommsdorff [6] showed that people produced less ideas in a brainstorming task when working in a group than when working individually.

Participative EM represents classic group work which is just as prone to the above-mentioned effects. It involves a variety of activities which the members of the group must perform. In this paper, we focus particularly on tasks of EM which involve gathering knowledge and ideas in terms of brainstorming. Performed in a team, such activities require a significant effort on coordinating the contributions of all members. Secondly, situations may occur when one of several alternatives has to be chosen by the group. If a person is dominating this decision process because of their knowledge or intelligence, this might lead to good overall performance. However, if such dominance is based on characteristics such as the rank of a group member, this might deteriorate results and lead to a decrease in the motivation of other team members to voice their ideas. Especially when tasks are at hand where there is no complete knowledge and the group has to base decisions on guesses, more extroverted persons may possibly dominate discussions at the expense of the result’s quality (see [1] for more information on EM, and [7] for information on task types). This paper will scrutinize EM especially from this perspective of group work and its challenges concerning individual participation, group performance and coordination.

2.2 Studies on Multi-touch Use

There have been several studies dealing with MTTs in general. Especially in educational context, advantages of these devices are underlined. They allow sketching ideas that can be easily changed or erased from screen, thus being less fixed and restricting than notes on paper [8, 9]. On the other hand, studies report that input via touch keyboard is more laborious and time-consuming [10]. Several studies compared MTTs with other tools assuming an influence of the tool on collaboration. Setting a brainstorming task, Buisine et al. [11] discovered that users of MTTs contributed less verbally and in gestures than users of a table covered with paper, but more than users of a flip chart. They hypothesize that the novel medium is distracting and thus restrains collaboration. Basheri and Burd [12] observed closer collaboration of teams using a MTT compared to pen and paper. When comparing PC and MTT for UML modeling, Basheri et al. [13] found the team members’ contributions in terms of modeling more evenly balanced and the collaboration to be closer at the MTT. Rogers et al. [14] considered laptop and MTT with and without tangible objects where the laptop turned out to cause less evenly distributed verbal contributions. All in all, MTTs seem to be promising for the purpose of participative EM. However, several other factors play a role in this, such as the orientation of the medium [15] and the task at hand [11] which may vary in EM.

3 Method

3.1 Experimental Design

As the influence of the tool on group work was to be explored, the following experimental design was applied. The independent variable was represented by the tool which was either whiteboard or MTT. It was decided to use a between-subjects design. In a within-subjects design, teams would have to work with both tools one after the other. On the one hand, this would have allowed direct comparisons where variables such as personal traits and modeling experience would have been kept constant for both treatments. However, the learning effect was assumed to have a more severe influence; i.e. groups would have developed their team roles and work strategy while using the first tool, and then would have continued with the second tool based on their recent experience. Following a between-subjects design, an EM task was to be solved in teams of three either on the whiteboard or the MTT. The team size of three was chosen due to the limited size of the media and because Nerdinger et al. [16] claims that group effects are to be encountered with only a team size of at least three.

The dependent variables corresponded to the participation of the group members, perceived team coordination and organization. The group members’ participation was measured based on their contributions in terms of talking and modeling including activities such as writing, drawing and moving elements on the respective medium, e.g. cards on the whiteboard. Participation was assessed via observation using video recordings of the modeling sessions. Perceived coordination and organization among team members were assessed via individual interviews. The participants were asked whether certain team members were responsible for or often took on certain tasks and how such task divisions arose. Moreover they were asked to describe how their group approached the task, with a special focus on modeling activities. Furthermore, group performance was measured by considering the complexity of the final solution in terms of number of components and relations drawn. As the participants had to solve a very open task comprising brainstorming activities, we did not assess quality aspects related to the content, e.g. semantic quality [17]. The team members were asked to take the perspective of entrepreneurs and collect ideas instead of mapping concrete knowledge. It was up to them on which aspects of the task they wanted to concentrate. To include the aspect of model quality when analyzing the modeling process we, however, examined to what extent the participants used the repertoire of modeling components offered by the notation, adapting the approach of [18] to our issue.

3.2 Procedure

The study took place at a laboratory of the computer science department at the University of Rostock. The participants were recruited by personal request. The assignment of persons to groups was organized taking the participants’ personal schedules into consideration. The groups were assigned to the tool randomly. The EM task the participants were expected to perform referred to the modeling of goals and problems of a pizza delivery service, an application field the participants would probably connect with. Moreover, goal and problem models belong to the most basic enterprise models in the 4EM method suggested by Sandkuhl et al. [1]. They should be easy to apply even for less experienced modelers. It was made sure that at least one team member had at least some experience with the 4EM notation of goal models. If the participants’ time schedule allowed it, a face-to-face tutorial was provided before the study in a separate meeting.

At the beginning of each trial, the three participants of a group and the investigator met in the laboratory. Beside the investigator, at least one person in charge of the technical equipment was present, but stayed in the background. In some cases the participants had not met before such that they had to be introduced to each other. Refreshments were provided to create a relaxed atmosphere. Each participant received a handout containing the modeling task and a short reference of the 4EM notation for goal models. They were explicitly asked to work together on the task. In case the group had to work with the MTT, an introduction to the user interface of the self-developed modeling software was given (see right-hand screenshot in Fig. 1). The software provides an editor for 4EM goal and problem modeling enabling several users to model at the same time. Menus can be opened at every spot of the canvas to create components such as goals and problems. All components can be moved and rotated at will. If a user wants to add a description to a component, a keyboard will pop up right below the component. Thus, users do not have to share a keyboard; multiple keyboards allow parallel editing. Each keyboard’s position depends on the position of the component they are appended to. If the component is moved, the keyboard will follow. Components can be linked by drawing arrows. These component relations can be further described by selecting one of the predefined annotations, e.g. “hinders” for a problem hindering a goal. We did not introduce a facilitator because one purpose of the study was to provide insights into natural working behaviors that may help facilitators in chairing modeling sessions. That is why the teams where required to comprise at least one member experienced in the notation.

Fig. 1
figure 1

Marking and annotating activity units in the video recordings with ELAN [19]

Full size image

After the participants had finished reading and remaining questions were answered by the investigator, they started to work on the task. Three groups used a MTT (size: 1210 × 680 mm) as can be seen in Fig. 1, the other three groups used a whiteboard (size: 2000 × 1000 mm). The whiteboard groups were additionally equipped with colored cards, magnets to pin cards to the board, and pens. A time limit of 30 min was set for the task. The modeling sessions were video and audio-recorded using two cameras, one installed at the ceiling and another one standing on a tripod. After the task was finished, interviews were conducted with each participant in parallel in separate rooms. The procedure was concluded by letting each participant fill out a questionnaire to capture demographics, experience with modeling notations and use of MTT.

3.3 Sample

Overall, 18 persons took part in the study, i.e. three teams using the whiteboard and three teams using the MTT. They were all students of business information systems or computer science. Three participants were female, two of them joined one team that was assigned to the MTT, the third woman joined a team working with a whiteboard. In the whiteboard group, participants were 24.9 years old on average (σ = 2.2, max = 28, min = 22), in the MTT group, participants were 23.6 years old on average (σ = 1.9, max = 28, min = 22). The participants came from three different nations. There was one completely Russian team, one completely Indian team and a mixed team of two German students and one Indian student. In two of the teams, the members knew each other already, in one team only two members knew each other before. The whiteboard group comprised two completely German teams whose members all knew each other before, and one mixed team of all three nations whose members had not met before. The level of experience of the 4EM notation, measured with a 5-point scale with 1 representing no experience, was at 2 on average for the MTT group (σ = 1, max = 3, min = 1) and at 3.7 on average for the whiteboard group (σ = 0.6, max = 4, min = 3). On a 5-point scale, the participants of the MTT group estimated their experience with MTTs at an average value of 1.7 with 1 representing no experience (σ = 1.3, max = 5, min = 1).

3.4 Data Evaluation Methods

The data from the questionnaires were statistically evaluated using the software SPSS. The interviews were transcribed, and qualitative content analysis according to Mayring [20] was applied, supported by the Software MaxQDA. First, the interviews were scanned for statements about coordination among team members and the organization of their work. The resulting coding units were inductively ordered into categories.

For the evaluation of the video recordings, the software ELAN was used [19]. To determine all time units of talking and modeling of each single person, the respective sections had to be marked on a timeline in the software as Fig. 2 shows. Furthermore, the time units of modeling were annotated with a detailed description of the according activity. In a subsequent step, the activities were again categorized, leading to major activities such as creating a new component (e.g. goal), moving a modeling component over the screen of the MTT or the whiteboard, respectively, writing or drawing relations between components. Each activity was linked with a time stamp such that talking and modeling behavior of each participant in the course of the whole session could be depicted and analyzed. Special focus was put on particular events in the modeling referring to the creation of content (e.g. create component, write, draw relation, pin component). The team members’ contributions were determined by considering individual speaking and modeling time in relation to speaking and modeling time of the whole team. That way we could also evaluate on team level how many components and relations were drawn, and how many component types of the eight offered types had been used.

Fig. 2
figure 2

The time proportions spent by the members (P1–P3) of each team on speaking, modeling as well as speaking and modeling in sum; MTT = multi-touch table, WB = whiteboard

Full size image

4 Results

4.1 Participation

Figure 2 shows the time proportions spent on talking and modeling in separate, and on talking and modeling on the whole for each member of each team. For all members in all groups it was captured when they contributed to the modeling work in a creative way in terms of adding content, comprising the creation of components (e.g. goal), writing, drawing relations between components and pinning components to the whiteboard. For the latter, there is no corresponding event on the MTT. Figure 3 shows the occurrence of these events in all teams in the course of the modeling session (30 min).

Fig. 3
figure 3

Modeling events where content was created by the participants in the course of the 30 min modeling session

Full size image

4.2 Coordination and Organization

Based on statements on team coordination and organization occurring in the interviews, six major categories arose which deal with (I) the subjects the team members explicitly agreed on, (II) statements that collective approval was part of the proceeding, (III) the existence of task divisions and if so, which form they took, (IV) reasons for certain task distributions, (V) parallel working, and (VI) awareness of the teammates’ activities during the task.

  • Subjects of explicit agreement. One participant of a team working with a MTT and one participant of a whiteboard group mentioned that the team explicitly agreed on how to start working. E.g. the latter said, “We soon agreed that we would start with goals and then move to problems and then the rest, like adding constraints and opportunities to the model …” (6,3,1,39, translated from German).Footnote 1 During the task, some teams consulted about how they would further proceed, as was mentioned by four interviewees from two MTT groups and two whiteboard groups. One participant who had worked with a whiteboard stated that the team had to agree on the level of abstraction with which they approached the task. Three participants from two MTT groups and one from a whiteboard group mentioned that they explicitly agreed on how or when to use the tool. E.g., “They directly wanted to do it on the software, but I convinced them, consoled them like it’s better list it first and just go there and put it on” (3,1,1,11).

  • Getting collective approval. Two participants of a MTT group mentioned that during their work they constantly searched for a common agreement, e.g. “Before we did anything, we always said first: we can do that” (4,3,1,37, translated from German). In each whiteboard group, there was always one person who described a similar behavior.

  • Division of tasks. In every group, except for one whiteboard group, at least one team member stated that there was no fixed division of tasks. In most cases, the statements refer to modeling activities, e.g. one participant of a MTT group said: “after that I was writing something, she was writing something, XXX was writing something” (1,3,1,1). In one whiteboard group, one participant said, “No, everyone everything together practically” (6,3,1,41, translated from German) referring to content-related contributions, while his teammate said they had fixed roles with regard to the modeling (“We did not vary. We always had the same roles really”, 6,1,1,11, translated from German). Two of those team members described this task division by modeling activity in detail, e.g., “We had this division of tasks, one wrote—meaning on the cards. I pinned them to the thing [whiteboard] and wrote on it. And one commented” (6,1,1,9, translated from German). Some groups, however, used a content-related division of tasks. One whiteboard group divided the task according to component types, as described by all team members, e.g., “… one captures goals, the other one problems, and the other one already opportunities or threats or whatever” (2,1,1,9, translated from German). There were two statements from two MTT groups and two statements from one whiteboard group that team members were responsible for certain topic areas, often corresponding to main goals.

  • How task distributions arose. With regard to the reasons why and how certain task distributions arose, some single statements could be found in the interviews. In one MTT group, a participant said that a teammate was mainly drawing the model because this person had more experience with the 4EM notation (“I think that she had the project connected with 4EM and she is more experienced in this case, so we decided that she is right person to do it”, 1,3,3,3). One participant who had worked with the whiteboard stated that he “was a bit busy writing” (6,2,1,7, translated from German), so he could not fully contribute to the discussion all the time. Another participant explained that the team members knew each other before such that everyone already had their role in the team (2,1,1,13). In one MTT group, as stated by two of its members, it was seemingly usual that the person who had the idea also wrote it down. Tasks were also distributed by request or enquiry. In one MTT group, all team members describe that teammates had been asked to do something. Two participants from two whiteboard groups described a similar behavior, e.g., “On enquiry and response. Meaning, ‘Do you want to write?’, ‘Yes’, and that was it.” (6,3,1,35, translated from German).

In all MTT groups, participants mentioned that the team members’ position was a reason why certain persons turned out to be responsible for certain modeling activities. According to one participant, the orientation of the screen played a role (“… we staying the different sides and the first orientation, the right orientation were only for one person, so XXX stayed in the right position and we stayed in the other sides”, 1,3,3,3). Two participants said that proximity was another reason (“I was doing, because I was […] right side and the creating menu was the right side. So every time I was doing this thing.”, 3,3,1,16). One of these participants also considered space as important (“… because he always had a lot of space up there in his corner …”, 4,3,1,65, translated from German). The participant further explained that the keyboards, which were automatically attached to every component as soon as the edit mode was opened, and the menu to create components, of which several instances could be opened everywhere on the screen, restricted the team’s space.

Some participants described that the distribution of tasks in their team arose out of its own. One participant from one MTT group and four participants from all three whiteboard groups said that the task distribution emerged without any apparent reason, e.g., “It just happens” (5,2,1,27).

  • Parallel working. Two participants from two MTT groups mentioned that they worked in parallel. One of them explained further that parallel working concerned modeling activities in particular and that they did not choose to model in parallel at the beginning, but they changed to this mode in the course of the task. This participant and another one from the second aforementioned MTT group also describe that they had been discussing while teammates were modeling at the same time. However, some participants also said that they did not work in parallel. A participant from a whiteboard group even said “when somebody was writing, there was mostly silence during that time” (6,1,1,23, translated from German). One MTT group seemed to have had some trouble with the software such that it appeared to them that parallel working was not supported, as mentioned by one member (“Actually, we’ve tried in the beginning to do at the same time, but the system was not that cooperative for that. We thought of like, we’ll do one by one. When one was completed with the goal, and the second person will start at goaling”, 4,1,1,25). The other two team members stated that they changed to non-parallel working particularly later in the process, e.g., “because later at the end when…, there was mostly just one working …” (4,2,1,17, translated from German).

  • Awareness. Two participants of one MTT group mentioned that the MTT helped them to keep an overview of what was going on during the task, e.g., “Once again wrote something, then you looked. What did the other one do? Ah yes, okay, then you can still write that …” (4,2,1,11, translated from German). However, two participants from two whiteboard groups said that there had been a moment where they lost track of what was done by the others, e.g., “where I was concentrated on this and I lost track of what XXX and YYY were doing at the other corner of the whiteboard, what lines they were drawing. And then I confined myself to my right side of the board” (2,3,3,3, translated from German).

4.3 Group Performance

Table 1 describes the complexity of the goal models the groups created including the number of components, among them decomposition elements (and, or), the number of relations in general and relations that have been annotated, e.g., a problem may hinder a goal. The last line represents the percentage of component types that had been used based on a repertoire of eight different components offered in the 4EM notation.

Table 1 Parameters describing the complexity of the respective groups’ final model
Full size table

5 Discussion

5.1 Summary and Interpretation

A study was presented that compared the use of MTT and whiteboard in the context of participative EM. First, verbal contributions and modeling contributions of the members within each group were analyzed. Addressing our first research question, Fig. 2 gives hint on more evenly distributed participation as the respective charts appear more compact for group MTT 2 and MTT 3. For MTT 2, the participants’ origin, India, might have played a role in their team work. Some team members often repeated what another one had said as a sign of confirmation, possibly reflecting their desire for consensus. However, looking at modeling events triggered by the individual team members in Fig. 3, the distribution of modeling activities appears less balanced. It seems that with regard to really creating content, one team member was particularly dominating. Experience with the modeling notation might also have influenced the distribution of individual contributions. In team MTT 1, there is mainly one person performing the modeling activities as this team member is the only one with sound experience in the modeling notation. A second team member, however, is dominating in the discussion whereas the third team member has a low proportion of verbal and modeling contributions. Although WB 1 and WB 3 comprised experienced modelers, they showed similarly uneven activity distributions. This gives hint that the tool might have been a stronger influencing factor than experience. Nevertheless, it might be the case that these persons had undergone similar collaborative tasks together before as the members of WB 1 and WB 3 knew each other before. This might have caused that these groups skipped the phase of team forming or at least found their roles more quickly than the others had. Taking a look at WB 2 in Fig. 2 though, we again see unevenly distributed contributions although team members did not know each other before. This still leaves the tool as important influencing factor. All in all, however, the differences between the team members’ contributions on whiteboard or MTT are not clear enough to lead to definite conclusions. At least in this study, it seems that the MTT promoted more balanced contributions by the team members.

Addressing the second research question on collaboration styles, the most interesting findings concern parallel working, awareness, and reasons for task assignments. The MTT seems to stimulate parallel working in terms of parallel modeling, but even more so in terms of discussing and modeling in parallel. Probably owing to time pressure at the last phase of the session, teams decided for parallel working. Only one team abandoned trying to work in parallel at the MTT. As the software was just a prototypical implementation, its interface might not have responded in the way the users had expected and thus might have made the impression that parallel working was not supported. This shows that parallel working and the challenges connected with it must be especially considered when designing and implementing user interfaces of a MTT modeling editor.

According to the interviews, MTT users might be more easily aware of what their teammates are working on. The horizontal work surface might be a reason for this as claimed by Rogers and Lindley [15]. Two participants from two different whiteboard teams mentioned that there had been a moment during the modeling session when they lost track of what the others were doing. These findings support statements by Muller-Tomfelde et al. [21] and Buisine et al. [11] that vertical work surfaces are less supportive for collaboration than horizontal. One might also assume that the restricted size of the working surface, which seems at first as a disadvantage of the tool, might also turn out to be an advantage as it is easier to keep an overview of the whole model than on the larger whiteboard.

When asked why certain persons took on a certain task, members of all MTT teams named the position at the MTT as one reason. When a person stood in opposite to the orientation of the user interface, rotating the elements meant additional effort. Still, it was observed that such MTT users were nevertheless engaged in modeling. Another participant reported that proximity implied responsibility, e.g. somebody standing near the main menu became responsible for creating new components. This is similar to the findings of Scott and Carpendale [22] and Ryall et al.[23] stating that the farther away a MTT user is from a part of the interface the less responsibility he or she feels for it. Moreover, it is possible that teams developed a certain routine in terms of a mental set, also known as cognitive fixedness [24]. Menus could be opened at every point on the surface, so other users would have also been able to open a menu and create new components. Due to a mental set, the team members might have stuck to a strategy they have found to be successful once. However, the interviews reveal that space also played a role in this, and thus space management is another challenge when designing the user interface. Team members took on a certain task because they had more space on the working surface where they stood. As a consequence, they were reluctant to open more menus that would take more space.

Concerning explicit arrangements, there do not seem to be many differences between MTT und whiteboard users, as far as can be concluded from the interviews, e.g. on how to start, or how to proceed in the middle of the task. The question of how and when to use the modeling tool seemed to be more explicit for MTT users (mentioned by three users in two groups) than for whiteboard users (mentioned by one user only). Due to the MTT’s novelty, users might be more aware of the tool and handling it, and consequently they might be more conscious and careful in the way they use the tool.

In addition to the above-mentioned differences between the tools, general ways of working that may arise within teams engaged in participative modeling could be observed which seem worth mentioning here.

According to the interviews, teams MTT 2, MTT 3 and WB 2 divided their work among their members based on topics, which is reflected in Fig. 3 by the occurrence of all kinds of modeling events for all persons, although not always evenly distributed among team members, and during the whole modeling session without any systematic order. Two members of team MTT 3 mentioned that if a team member presented an idea, this person would also model the according content. This led to more even distributions of creative modeling activities than in teams MTT 2 and WB 2.

The depictions of modeling events are especially characteristic for teams WB 1 and WB 3. In team WB 3, a division of modeling activities arose such that one person wrote on the cards representing model components such as goals and problems, a second person pinned these cards to the whiteboard and drew and annotated relations between the components, and the third person contributed mainly by discussion. On rare occasions, the third team member picked a new card from one of the card stacks only to hand it over to the teammate who had become responsible for writing on the cards, or he pinned a card to the whiteboard, but mostly left this job again to the other teammate who had meanwhile turned out to be responsible for that job. This is underlined by one team member’s statement in the interview, describing their roles as fixed. The described behavior might have been caused by the team members modeling experience and by knowing each other. Another reason, however, might also be some kind of cognitive fixedness as observed with the MTT.

Team WB 1, the other team that showed a very significant working behavior, decided for a division of work based on component types; i.e. goals, problems, constraints etc. Thus, all kinds of modeling events can be seen for all team members in Fig. 3. The time sequence in which these events occur is, however, most special. According to the interviews, there must have been an agreement to first gather the model components including their descriptions. Then there must have been a moment when the team decided to pin all the cards to the whiteboard and start drawing and annotating relations between the components. Figure 3 clearly reflects this time sequence. A similar procedure was chosen by team MTT 2, but the working behavior looks different in Fig. 3 due to the different tool used. While the members of a whiteboard team could work with and write on the cards independent of the whiteboard, the MTT users must look for other means of sketching ideas. Team MTT 2 decided to take notes on a sheet of paper before they started to model on the MTT. This is reflected by a longer delay of modeling activities for team MTT 2 at the beginning of the session compared to all other teams. The working behavior implies that MTT users should be provided with a possibility of sketching ideas apart from the MTT, whether by means of paper and pen or technologically supported must be discussed and further examined.

When considering team performance (third research question), the final whiteboard models tended to be slightly more complex. However, a reason for this might be that two of the whiteboard teams, WB 1 and WB 3, already knew each other before, as already mentioned above. One participant said they already knew their roles in the team. Moreover, both of these teams had sound experience in the modeling notation they had learned in compulsory courses at the university. Another reason why MTT models were relatively smaller may be the restricted space. This again points us to the challenge of how to accommodate large models on the working surface. When looking at the percentage of component types used, it seems that experience was not most important as WB 3 seemed to exhaust the component repertoire deliberately less than WB 1. Furthermore, it seems that with the MTT, less component types were used. We may speculate that the tool itself was distracting the participants from the possibilities of the notation. In both treatment groups one model was created containing no annotations for the relations between components. So, the tool did not seem to be especially influential with respect to this aspect.

On the whole, the MTT appeared to be equally suited for the task we presented in the study as the whiteboard. This is confirmed by analyses of perceived usefulness, perceived ease of use and perceived enjoyment which did not show significant differences between both tools [25]. These constructs are considered major factors, determining whether a technology will be accepted or not [26]. The analyses presented here give hint on future challenges when dealing with more complex tasks performed in teams with a MTT. These may, for example, concern the shaping of software for the MTT, e.g. with regard to space management and promoting parallel working while keeping team members’ awareness.

5.2 Limitations

When interpreting the findings of this study, certain limitations have to be taken into consideration. The study provides a detailed, mainly qualitative analysis of work behavior in modeling teams. Differences in the participants’ culture and their experience in the modeling language complicated the comparison of working styles with both tools. However, it was assumed that with a within-subjects design, a learning effect would have been created within the teams which would have distorted the data even more.

It was meant to explore possible differences in collaboration depending on the modeling tool used. That is why a small sample size of 18 persons assigned to six groups was considered as sufficient. For testing hypotheses and drawing generalizable conclusions, further studies with greater sample sizes are needed. Moreover, more representative samples comprising participants, preferably practitioners, from different domains would be desirable. The behavior of the participants may of course be influenced by their feeling of being observed which can never be prevented completely. It was expected that the feeling of being observed, particularly via cameras, would be weakened overtime while concentrating on solving the task. To explore different ways of working that arise naturally, possibly induced by the different tools, roles, such as domain expert and facilitator, had not been predefined. Future studies may further explore especially the role of facilitator and its tasks. In addition, further kinds of models beside goal models must be examined since they might each require a different extent of creative, knowledge-based, problem-solving activities etc.

Finally, when examining the MTT as a tool, the software that runs on the tool is a crucial factor of technology acceptance. In a study like this, we cannot really separate tool and software. A solution would have been to include several software products to distinguish effects caused by tool and software. This would, of course, have led to a need for a significantly greater sample size and an immense effort in data evaluation exceeding the benefit we expected from this exploratory study.

All in all, however, the study fulfilled its purpose of giving valuable insight into the procedure of participative EM. The next paragraph will give hint on next steps, particularly with regard to research in this area.

5.3 Implications

The study has shown that a MTT is a tool well-suited for participative tasks comparable to goal and problem modeling. With MTTs, all stakeholders can potentially participate and even work in parallel. Moreover, awareness of other teammates’ activities is higher on the horizontal work surface. For both MTT and whiteboard, we discovered several ways of how teams organize themselves to solve a modeling task. However, which of these work styles is most efficient and convenient for teams? Is it really necessary to strive for most balanced proportions of contributions, e.g. by having every team member talk and model to the same extent? Is it advisable to let everybody model their own ideas to give every team member a chance to equally contribute both to discussion and modeling? Should modeling sessions be organized in phases of collecting ideas and then formalizing these in models? Further research is needed on the influence of working styles on desired outcomes. These would lead to suggestions on how a facilitator should chair a modeling session and when interventions are advisable and when they become counterproductive. Moreover, we intend to improve the modeling software aiming at preventing functional fixedness, better space management and supporting awareness. Although there has been research on the topic of awareness for several years in the area of computer supported work (see e.g. [27, 28]), there is still a need for more knowledge on how to design shared workspaces to promote awareness.

Finally, the desired outcomes are manifold. Complexity and quality of a model represent typical outcome variables. Nevertheless, subjective perceptions such as the team members’ satisfaction with the modeling process, their acceptance of and commitment with the models also determine the success of an EM project. Thus, we will focus our future research especially on this area.