1 Introduction

A critical success factor for any community (at work and outside work) is the extent to which it can coordinate itself to communicate and achieve common goals: in other words, to collaborate (Patel et al. 2012). Therefore, when we talk about collaboration itself, we are referring to problem-solving with a group of people with different skillsets (Hennessy and Murphy 1999). The advantages that can be gained from the good collaboration will vary according to the type of community, but the benefits can include: increased benefits through sharing expertise; reduction in costs through sharing best practices; improved decision-making through sharing insights and knowledge; innovation through sharing ideas; and an improved ability to pursue goals (Hansen and Nohria 2004). Taking this into account, collaborative work was defined as the joint action of people working toward the same final goal (Lucero 2003). People and teams engage in collaborative work processes in order to complete tasks, meet defined goals where team task performance is as critical as collaborative performance (Patel et al. 2012). When people talk about collaborative work, they mean more than just completing a task; however, they mean the work that comes from people working together effectively (Perrault et al. 2011). That effectiveness comes from supporting each other, communicating well, and sharing elements that will lay the foundations to achieve true collaboration, a non-easy task (Assbeihat 2016). In the same way, there is a surprising lack of a clear understanding of what it is to collaborate, how to collaborate, and of how to best support and improve collaborative working (Patel et al. 2012). In this sense, in order for collaborative work to take place, there must be reciprocity among group members, which requires each one of them to make their duly argued contributions. These, at the same time, should be treated in the group in a critical and constructive way (Stigliano and Gentile 2006). As a consequence, the information must be available to everyone; they must understand and speak the same language and have successful communication (Clark and Brennan 1991). Since the process of achieving consensus is not only one of the agreements or disagreements, but also a consensus of arguments (Innes and Booher 1999). Therefore, the final product of the group cannot be the sum of the individual contributions; instead, collaborative work will be carried out when such a product includes negotiation, cohesion, communication, and therefore the understanding reached by the group (Jonassen and Kwon 2001). A pillar to have good negotiation, cohesion, and communication within a group so therefore it improves the collaboration is shared understanding (Bittner and Leimeister 2013), a cognitive aspect that ensures that the team works effectively and efficiently towards a common goal (DeFranco et al. 2011), which refers to the degree to which people concur on the interpretation of the concepts, when sharing a perspective (mutual agreement) or it can act in a coordinated manner (Van den Bossche et al. 2011), since group members may be using the same words for different concepts or words for the same concepts without realizing (de Vreede et al. 2009). These differences can interfere with the productivity of collaborative work if they are not early clarified (Mohammed et al. 2010). With this, it can be inferred that for collaboration to happen, there must be a shared understanding of the problem that is being solved. For this reason, the shared understanding of the task is an important determinant for the performance of collaborative groups (Langan‐Fox et al. 2004), (Mathieu et al. 2000). Therefore, in order to collaborate effectively and efficiently, it is necessary to help groups converge on a shared understanding of the task (Bittner and Leimeister 2013).

Consequently, this paper presents the validation of a process for the shared understanding construction in a problem-solving activity. Specifically, the validation consisted of conducting an experiment to statistically contrast whether with the use of the process it is possible to achieve the shared understanding construction when the participants solved a problem related to software process lines. From the statistical analysis, it could be determined that the process is feasible and partly useful. However, aspects to improve were identified, such as the reduction of the cognitive load that the process involved in its use, and also the incorporation of elements to monitor and assist in the shared understanding construction in such a way that it is maintained throughout the development of the activity.

This paper is structured as follows: Sect. 2 describes related works, Sect. 3 contains a description of the proposed process, Sect. 4 describes the validation through an experiment, the results, and its analysis. Finally, Sect. 5 has conclusions, future work, theoretical and practical contributions.

2 Related works

Smart in (2011) determines that the process of measuring emerges as the analysis of shared abilities and decides what kind of understanding we are interested in defining, showing the capacities that arise in each person. Also, he proposed to use a multi-agent simulation technique in which the extent of shared understanding will be indicated by the commonality of belief states across multiple (synthetic) agents. On the other hand, White and Gunstone (1992), describe a range of measurement techniques, primarily to be used in educational contexts, as the use of conceptual mappings, relational diagrams, and word association tests. Likewise, as defined by Sieck et al. (2010), the mental models can be used to index an individual’s understanding of some domain and the similarity might provide a measure of shared understanding. On their part, Bates et al. (2014) developed and validated the Patient Knowledge Assessment tool questionnaire that measured a shared clinical understanding of pediatric cardiology patients. And Rosenman et al. (2018) worked with interprofessional emergency medical teams, where they measured the shared understanding through team perception and a team leader effectiveness measurement. For her part, Bullard (2019) created a new theoretical model that used the constructs use of communication media, mode of interaction and team diversity to ascertain the influence shared understanding in virtual teams, also, examined the relationship between shared understanding and team performance. A developed, web-based survey measured the shared understanding and team performance in virtual environments. On the other hand, Sinval et al. (Sinval, y otros, 2020) developed a new shared mental model measure, specifically designed for the refereeing context, determing that the concept of shared mental models refers to the shared understanding among team members. The proposed version of the referee shared mental models measure (RSMMM) has 13 items that are reflected on a single factor structure, presenting good validity evidence both based on the internal structure and based on relations to other variables (presenting positive associations with team work engagement, team adaptive performance, and team effectiveness).

On the contrary, there are works about collaborative problem solving (CPS) as: Edem (Quashigah, 2017) examines the occurrences of the target group of CPS activities, as well as individual contributions. Roschelle and Teasley (1995) focus on the processes involved in the collaboration, where they concluded that the students used language and action to overcome impasses in shared understanding in order to coordinate their activity. Barron (2000) identified 3 dimensions in the interactive processes among the group such as the mutuality of exchanges, the achievement of joint attentional engagement, and the alignment of the goals. Häkkinen et al. (2017) presented their pedagogical framework for the twenty-first-century learning practices, among those that are collaborative problem-solving skills and strategic learning skills. Graesser et al. (2018) developed a CPS assessment of student skills and knowledge, by crossing three major CPS competencies with four problem-solving processes. The CPS competencies are (1) establishing and maintaining shared understanding, (2) taking appropriate action, and (3) establishing and maintaining team organization.

On the other hand, according to empirical studies in shared understanding, in (Ingrid et al. 2002) a project called international networked teams for engineering design (INTEnD) was developed, which aimed to investigate geographically dispersed engineering teams; an empirical study was developed to specifically inquire about group learning and shared understanding in a globally distributed engineering team. Likewise, (Humayun and Gang 2013) investigated the role that shared understanding plays in eliciting requirements within the framework of a global software development (GSD). As a part of the work, an academic experiment with two geographically distributed groups of students was defined, in which conceptual mapping exercises were carried out in which the compression patterns of the groups were observed; As a result, it was obtained that a clear organizational structure with communicative responsibilities improves the shared understanding in elicitation of requirements in GSD projects. On their part (Jentsch et al. 2014) developed an experiment with students and a pilot field study with professionals in which a content validity survey instrument is used to measure the shared understanding of companies with their IT units. Additionally, in (Rosenkranz et al. 2016) an empirical study is carried out in a software development company in which a survey and a series of semi-structured interviews are used to investigate how the distribution of a team influences the success of the project using a shared understanding approach. It was found that in this context the level of distribution of the team does not significantly influence the shared understanding of the success of the project. Similarly, in (Dossick et al. 2017) they develop an empirical study in which photo elicitation techniques are explored in combination with ethnography to evaluate shared understanding in multidisciplinary building design teams. It is a study developed in an interdisciplinary academic environment in which the interactions and visualizations created and used by different students were studied to learn and develop integration skills.

The previous works show different perspectives related to shared understanding, collaboration for problem-solving, and some empirical studies. These works have different visions and strategies on how to measure shared understanding and how to solve problems through collaboration, but none defines an experiment in which a process for shared understanding construction in a problem-solving activity is validated, by being this the main contribution of this paper.

3 Description of the proposed process

For the construction of the process, the definition of process as an ordered step sequence was taken from the field of software engineering with some kind of logic that focuses on achieving some specific results (Humphrey 1989), in addition to following the collaboration engineering design approach (Kolfschoten and De Vreede 2007), which addresses the challenge of designing and deploying collaborative work practices for high-value recurring tasks and transferring them to practitioners to execute them without the ongoing support from a collaboration expert (de Vreede et al. 2009). Furthermore, to model the process, the conventions based on the elements proposed by SPEM 2.0 were used (OMG 2007).

Considering this, the proposed process contains a set of phases, activities, tasks, roles, and work product that can be input or output, activities, tasks, roles, and work product in which input or output might be found. The structure of the process was defined taking into account the Pre-Process, Process, and Post-Process phases established in (Collazos et al. 2014), which were updated and adapted. The Pre-Process phase begins with the design and specification of the activity, phase that is executed by the activity coordinator; the Process phase is the one where the groups are formed to execute the collaboration activity in order to achieve the objectives, an activity that was designed in the previous phase, and where the activity coordinator is in charge of monitoring the prior activity and seeking that the groups achieve what it is expected. And finally, the Post-Process phase where the activity coordinator reviews the resolution of the problem, the objective achievement proposed by the activity, and the performance of the participants.

In this work, only the Process phase is detailed, since in this stage the construction of shared understanding can be materialized. In this phase, four activities were defined: organization, shared understanding, collaborative activity, and collaborative knowledge building. In addition, in this phase, only the shared understanding activity is detailed since its construction follows an iterative and incremental approach where portions of the process are proposed, progressively evaluated, and improved. With the shared understanding activity, it is intended that the group members agree on what the problem is, what they must solve in the collaborative activity before starting its development, this activity is formed by the tasks of, Tacit Pre-understanding where each group member acquires an individual understanding of the subject; Construction in which one of the group members exposes his/her ideas and the others actively listen to them; Collaborative Construction in which the original ideas are refined, built or modified, and finally the Constructive Conflict, where the differences of interpretation between the group members are dealt with through arguments and clarifications. These last three tasks were taken from the research of Van den Bossche et al. (2011) who examined a model of team learning behaviors, which we adapted in our research. The necessary steps to be executed were defined for each of these tasks.

4 Validation

One of the existing methods in software engineering to carry out research is the empirical method where a model is proposed and evaluated through empirical studies, for example, case studies or experiments (Robert L 1994). In this sense, we used the experiment, which are valuable tools for all software engineers who are involved in evaluating and choosing among different methods, techniques, languages, processes, and tools. Experiments are important to test the hypothesis and in particular the predictive ability of the hypothesis (Wohlin et al. 2012).

For this reason, in order to validate, the feasibility and usefulness of the process for the shared understanding construction in a problem-solving activity, an experiment was developed, part of which was presented in (Agredo-Delgado et al. 2020), but in this paper, it is shown in more detail.

To carry out the design, execution, and analysis of the results of the experiment, the steps defined in the guide presented in (Wohlin et al. 2012) were followed.

4.1 Experiment context and objective

The experiment was carried out in a university environment in which, thanks to the support of each of the participating universities, there was access to students from the systems engineering program, from the subject "Software Engineering". These students were selected because it was necessary to have knowledge in process Lines, the topic of collaborative activity, this in order to solve the indicated problem, for this the participants were: 45 students from Universidad de la Matanza—UM (Argentina), where the proposed process was applied, and 15 students from Universidad Nacional de la Plata—UP (Argentina) which was a contrast group where the proposed process was not applied. The problem-solving activity consisted of each group assuming that they were part of the engineering team process of a company, where they had to establish the software development processes that best adapt and support the projects in the company. To solve the problem, they had to follow an execution guide, where information about the projects and processes is defined, and with this they might determine the scope of each one of them.

The goal of the experiment was to inquire about the feasibility and utility of the proposed process for the shared understanding construction in a problem-solving activity. For this, the following research question was stated: how feasible and useful is this proposed process?

It is important to clarify that in order to validate the construct before it should be implemented in practice, the process was subjected to several revisions, in the first review, two members from the Universidad del Cauca and a member of the Universidad de la Matanza participated, with knowledge in the subject of the activity and in the definition of software processes, in another review, we conducted a focus group with two experts on group work and collaboration engineering. The objective of these reviews was to determine those elements of the process that were missing or needed to be improved according to the consideration of the experts who analyzed it.

4.2 Experiment hypothesis

Considering the experiment goal, it is intended to evaluate the following hypotheses:

  • The proposed process is feasible for the construction of shared understanding in a problem-solving activity.

  • The proposed process is useful for achieving the objectives of a problem-solving activity.

In Table 1, in order to refine the previous hypotheses, the following specific hypotheses were raised and their respective variables:

Table 1 Specific hypotheses with its respective variables
Full size table

4.3 Experiment design

Table 2 summarizes the activities designed for the experiment development and specifies the activities duration and the support instruments that were used for its development.

Table 2 Experimentation activities
Full size table

4.3.1 Validity threats

Construct validity: the shared understanding construction was observed and measured by the perceptions of the participants, but the constructs underlying these behaviors are still unknown. In order to minimize the subjectivity in the support instruments for the information collection, these underwent validations by expert personnel with knowledge in process definition, collaboration, thematic of collaborative activity and collaborative work, as shown in Sect. 4.1, where the validations allowed to improve and complete the elements of the process before the experiment was executed.

Internal validity: the invested time for the execution of the study is extensive, and very long sessions are needed where participants in the final stages of the experiment may perceive fatigue that may influence the results. To try to mitigate this threat amid experimentation, the participants took a break without communicating with each other.

External validity: the activity that they had to do with the participants was about the scope definition in software process lines, this subject is barely known by university students. This was tried to be mitigated by doing some training in order to contextualize this subject.

4.4 Execution of the experiment

The UM groups applied all the activities, tasks, and steps specified in the process, making use of the planned tools for their support. They used MEPAC to guide the Pre-Process and Post-Process, Collab for the formation of groups in the Process and the different formats designed to execute the collaborative activity, of previously designed process lines, and the necessary formats to collect the information that allowed the analysis of the provided data. On their part, the UP groups carried out the same designed collaborative activity, with the formation of random groups and simply among the groups they gave a solution to such activity without following the proposed process. Table 3 shows the time invested in each of the activities by UM and UP groups.

Table 3 Time invested in each activity
Full size table

4.4.1 Results

After the execution of the activities designed for the experiment, different results were generated, among those obtained from observation and from statistical calculations. From the observation, it was identified that groups that obtained poor results (in terms of final product grades) were those that did not generate internal discussions to resolve doubts, did not do the assigned role, and did not have the disposition to work in groups. It was also observed that following the complete process from the beginning was exhausting by the participants and that this generated a lack of commitment for the rest of the activity, due to its high cognitive load.

On the other hand, in the experiment, we used a control group that did not use the (UP) process and a group that did (UM), to ensure that the differences in the final results were not only observed but statistically significant using the student's T-distribution (Neave 2002), which allowed to validate the specific hypotheses shown in Table 1. Depending on the information to be analyzed, there are three types of test (a) T-test for means of two paired samples, this means data that comes from the same people, that is, the comparison of the experimental group BEFORE and AFTER a stimulus (in our case the stimulus is the application of the process), the following two types of T-test, use data that come from two different groups where one received the stimulus (in our case, the use of the process) and the other did not: (b) T-test for two samples with equal variances (c) T-test for two samples with unequal variances.

For the type of T-tests, the values that were used to make the calculation were: reliability level = 95%, significance level = 5%, critical value in two-tailed, observations or cases = 9 for the T-tests type (a) and 9 (UM), 3 (UP) for T-tests type (b) and (c), degrees of freedom = 8 for T-tests type (a) and 10 for T-tests type(b) and (c).

For the T-tests type (b) and (c), initially, it was necessary to determine if the variances of the values were equal or unequal. For this, we used the Fisher test (Freeman and Julious 2007).

The following considerations for the acceptance or rejection of the null hypothesis was considered for the 3 types of tests:

  • If P-value or F-Value≤ significance level, the null hypothesis is rejected.

  • If P-value or F-Value > significance level, the null hypothesis is accepted.

Considering the specific hypotheses, its respective variables, and the obtained values, after applying the statistical analysis, the following results shown in Table 4 were generated. The hypotheses with the subscript a were considered, as alternative hypotheses and the hypotheses with subscript 0 as null.

Table 4 Results for each specific hypothesis
Full size table

It is important to bear in mind that: in Table 4 different scales are used because they are different types of measures analyzed. The descriptions of both individual and group understandings were rated by the activity coordinator from 0 to 5 (0 for those descriptions of understanding that did not correspond to what they had to do in the activity, 5 for those descriptions that correctly wrote the activity) because they are rating scales used by the researchers, the other scales are based on previous studies that have used the analyzed questions with their respective scales as shown in the references from Table 2.

4.4.2 Discussion of the results

According to the observation, while the activity was being carried out, it was identified that the participants of the UM groups, while having the description of the process, had better execution of the activity, because they had a route map to reach the objective. On the contrary, in the UP groups, it was observed that their behavior in the execution of the activity was quite chaotic as they did not have clear guidelines to follow. Furthermore, it was possible to identify that to obtain better results, there must be an interest by each participant to interact with his/her classmates, and give the necessary contributions to achieve the objectives. However, it could also be observed that using the complete process generates a high cognitive load since it is a process that contains many steps.

From the statistical results it was able to determine that:

  • H.1.1.2a can be accepted. In this way, it can be said that there is a significant difference between the grade’s averages of the individual descriptions, compared to the group ones. According to what was previously said, it can be inferred that use of the process improved the understanding of participants.

  • H.1.1.4a can be accepted. In this way, it can be said that there is a significant difference in the averages of the grades given to the group descriptions between the UM and UP groups. Concerning the prior information, it can be inferred that use of the process can generate a better group understanding.

  • H.1.2.2a can be accepted. In this way, it can be said that in the UM groups, the average of the understanding of the individual descriptions of their colleagues is 81.6%. According to what was previously said, it can be inferred that there are no significant differences in understanding before interacting with the group because all participants may misunderstand the concepts and have the same mistakes.

  • H.1.2.4a can be accepted. In this way, it can be said that in the UM groups, the average of the opinion of the individual descriptions of their colleagues is 73.9%. Concerning the prior information, it can be inferred that there are no significant discrepancies before performing a group discussion due to the fact that all participants may have the same doubts or the same mistakes.

  • H.1.3.2a can be accepted. In this way, it can be said that for UM groups, there is a significant difference in the perception of the participants of their homogeneous understanding before and after the use of the process. With respect to the above, it can be inferred that the use of the process allows for a homogeneous understanding of what to do.

  • H.1.3.4a can be accepted. In this way, it can be said that for UM groups there is a statistically significant difference in the average of obtained results from differences in individual knowledge versus group knowledge. With respect to the above, it can be inferred that with the use of the proposed process there are no discrepancies of each participant with regard to the group understanding, that is, at the end of the use of the process, all know what to do and agree with what has been defined by the group.

  • H.1.3.6a can be accepted. With the UP group, the perception of homogeneous understanding was researched, only at the end, after creating a group description. In this way, it can be said that there is a statistically significant difference in the average of obtained results from the homogeneous understanding between the UM and UP groups. With respect to the above, it can be inferred that using the process allows for a homogeneous understanding among the participants.

  • H.1.3.8a can be accepted. With the UP group, the perception of discrepancy was researched, only at the end, after creating a group description. In this way, it can be said that there is a statistically significant difference in the average of obtained results from differences in individual knowledge versus group knowledge, between the UM and UP groups. With respect to the above, it can be inferred that using the process does not generate knowledge discrepancies with group members.

  • H.1.4.2a, H1.4.4a, and H1.4.6a can be accepted. In this way, it can be said that there is a statistically significant difference between the perception results of the UM and UP groups, with respect to the tasks of a shared understanding (Construction, Collaborative Construction, and Constructive Conflict). With respect to the above, it can be inferred that the MU groups generated better results and these tasks were better fulfilled among the participants, due to the used process.

  • H.2.1.2a can be accepted. At the end of the problem-solving activity, a final artifact was generated that was qualified by the coordinator, in this way, it can be said that there is a statistically significant difference in the grades between the UP and UM groups. With respect to the above, it can be inferred that the use of the process generates end products of the activity with better quality levels.

  • H.2.2.2a can be accepted. In this way, it can be said that there is a statistically significant difference in the number of questions posed to the coordinator between the UM and UP groups, questions to solve doubts or concerns about the activity. With respect to the above, it can be inferred that this difference is due to the use of the process since it allowed to solve these questions internally.

  • H.2.3.2a can be accepted. In this way, according to the results, it can be said that there is a statistically significant difference in the average of obtained results from perceived satisfaction by the participants about the attainment of the objectives between the UM and UP groups. With respect to the above, it can be inferred that the process allowed to obtain better satisfaction with the achievement of the proposed objectives by the activity.

  • H.2.4.10 can be accepted. In this way, also according to the results, it can be said that there is no statistically significant difference in the average of obtained results from perceived satisfaction by the participants about process items between the UM and UP groups. With respect to the above, it can be inferred that for the participants it is a process that has many steps, that its application takes a long time and therefore generates a high cognitive load.

  • H.2.4.30 can be accepted. In this way, also according to the results, it can be said that there is no statistically significant difference in the average of obtained results from perceived satisfaction by the participants about activity outcomes between the UM and UP groups. With respect to the above, it can be inferred that that according to the perception of the participants, this is an extensive process, and therefore, the results of the activity may not be the best.

5 Conclusions and future work

This work validates the feasibility and usefulness of a process for building shared understanding. For this, an experiment was developed with students from two universities. According to the validation of hypotheses H.1.1, H.1.2, H.1.3, and H.1.4 regarding the feasibility of the process, it can be said that the process is viable to build a shared understanding in a problem-solving activity. According to the validation of H.2.1, H.2.2, H.2.3, and H.2.4, it can be concluded that the process is partly useful to achieve the objectives of the problem-solving activity, but it cannot be ensured that the process improves the perception of satisfaction of the participants with their elements and with the results of the activity. Considering that the process is perceived as feasible and partly useful, it can be inferred that although good results are obtained, there is a high cognitive load that must be improved.

In addition, the contribution to collaboration engineering practice is a validated process proposal through an experiment research study, that can be used by designers of collaborative work practices to systematically and repeatedly induce the development of shared understanding in heterogeneous groups. As shared understanding has been identified as crucial for collaboration success in heterogeneous groups, the process presented may foster better group interactions and thus better results.

While we used existing measurement items for shared understanding combined with observation, a need is revealed for more advanced measurement instruments that allow all categories of shared understanding to be identified, in addition to the need to include monitoring and assistance mechanisms that allow to maintain it during the development of the activity, since when it is achieved, this can also be lost in the process. In the same way, although the results of this study are promising, we identify as future work the need for further research of mechanisms leading to shared understanding, at better understanding the phenomena, its antecedents, and effects, thus generating more opportunities for developing techniques that allow to leverage its benefits for effective group work. Considering that, the process should become lighter so that the cognitive load might be avoided.

5.1 Theoretical and practical contributions

Below, the theoretical and practical contributions are found, both from the carried-out research and from this paper.

The theoretical contribution of the research is reflected in the characterization and materialization of the process in which different elements of the process are conceptualized, related, collected, and proposed, such as: instruments, components, and tools involved and necessary in the construction of shared understanding in problem-solving activities.

The contribution practice of research is reflected in providing a formal and enriched process with steps, tasks, activities, workflows, guides, work products, and roles, a process that when executed supports the construction of shared understanding in resolution activities from problems.

The practical contribution of the article is to provide the planning, execution, and analysis of the results of an experiment to validate the feasibility and usefulness of a process for the construction of shared understanding in problem-solving activities. In addition to providing empirical evidence of the use of the process in a real academic context that served as the basis for its validation.

The theoretical contribution of the paper is achieved from the critical analysis of the found related works, where it is shown that there is no empirical evidence that validates a process for the construction of shared understanding in problem-solving activities, in addition to the obtained results in the experiment where it is inferred that the process is feasible and useful. With respect to the above, this is based on the conception and materialization of the process in which different elements of the process are conceptualized, related, collected, and proposed, such as: instruments, components, and tools involved and necessary in the construction of shared understanding in problem-solving activities.