Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

Design is sometime described as “a problem of resolving tension between what is needed and what can be done” [1]. During the engineering design process, the design team translates the end user needs into a set of product specifications, a measurable detail of what the product has to do in order to satisfy them. In this paper, the word “need” is used in the same sense as Ulrich and Eppinger [2]. That is to label any attribute of a potential product that is desired by the end user, for whom the product is designed. Ulrich and Eppinger [2] state that “product specifications do not tell the team how to address the customer needs, but they do represent an unambiguous agreement on what the team will attempt to achieve in order to satisfy the customer needs”. In order to define the product specifications, functional analysis might be used [3]. Functional analysis builds a standard language to enable designers to share their viewpoints about needs and constraints. In functional analysis, the functional requirements refer to the needs and constraints. Product specifications are derived from the list of the defined functional requirements.

However, in the early stages of the new product design process, the end users are not always defined sufficiently well enough to clearly identify their needs, or for them to be involved in the design process. Moreover, at the early stages of the design process, the final product does not yet exist. What do exist are the intermediary objects which help designers to represent, manipulate and translate the product idea on which they work; such as sketches, diagrams, written specifications etc. [4, 5]. Even for contract projects it may be difficult to gain access to a client who is busy or located geographically at distance. Additionally, the designers generally work under time pressure, which makes it difficult to access end users to get data or integrate them to the design process. Furthermore, researchers, who undertake market and user research, are not typically the design actors, and the results very often comprise ambiguities, uncertainties and gaps that the designers have to manage.

Consequently, in the case that the information about the users is not available at the right time or difficult to understand or to remember, each design actor may interpret the end user needs differently and become sensitive to different product constraints. This lack of shared understanding of end users and of their needs, between design actors, may cause difficulties in defining product specifications and cause non-convergent design processes [6]. To overcome this, support methods might be used in order to define the end users and their needs in order to improve shared understanding of functional requirements between design actors. However, the appropriateness and effectiveness of the various methods is unknown.

In the literature, personas [7] and scenarios [8] are used in order to define end users and their needs. Their usage is becoming more and more popular, especially in Software and System Engineering and Human Computer Interaction (HCI) [9]. Our hypothesis is that scenarios and personas can be used to develop and improve shared understanding of functional requirements between co-designers. In order to test this hypothesis, an empirical study has been undertaken in a laboratory context. This paper focuses on the protocol of the study and discusses our ability to find indicators of share understanding.

The paper is organized as follows. In the next section the theoretical background of the research is explained in detail. Section three discusses how the empirical study was designed and conducted. Indicators of shared understanding are discussed in the last section.

Shared Understanding of Requirement

Cooper and Kleinschmidt [10] underline that, for the product success, during the product definition phase there must be an agreement on: (1) the target market, (2) the customers’ needs, wants, and preferences (3) the product concept and (4) product’s attributes, features, specifications, and requirements. However, it is an inevitable natural occurrence that there are different points of view between design actors in the way they interpret the users and their needs. Then, the design actors have to clarify their views and build a shared understanding.

In a design process, design actors bring with them their own beliefs, responsibilities, language, interests, jargon, and knowledge to the design team. As a result of this, each actor might see the design object differently within different perspectives. This is described as object world by Bucciarelli [11], personnel mental model by Badke-Schaub et al. [12] or as perspective, heuristics and interpretation by Page in his model of diversity [13]. All of them referring to distinct theoretical background but addressing the differences in which the members of a team can see the world. Bucciarelli’s work comes from a social background, and the term ‘object world’ describes the assemblage of social, technical and symbolic components that make design possible within specific engineering domains. The Mental models “are simplification of the world” and “internal working models of the world” [12]. Thus, a mental model includes the three components of the diversity model of Page: perspective, heuristics and interpretation. In Page model, perspective has then a restrictive definition as “a map from reality to an internal language, such that each distinct object situation, problem or event gets mapped to a unique word” [13, p. 31]. Even if diversity and existence of diverse perspectives can be expected for creative problem solving, it is admitted that teams should “share at least some aspect about the task and the team” [12]. Badke-Schaub for example claims that “there are strong indices that shared mental models have an impact on creative problem-solving” (p. 10). Moreover, the importance of shared understanding in the performance of a design team had been highlighted in design research (see [14] for an updated literature review). The necessity of constructing common ground within the team to facilitate collaboration is acknowledged in the literature. Detienne [15] highlights the importance of creating negotiation mechanisms and grounding activity in order to manage the multiple perspectives in design groups.

The notion of common ground represents the knowledge that actors have in common and their awareness of this uniformity. Clark and Brennan [16] state that effective communication requires grounding activity. The grounding activity helps design actors to co-create the shared representation of the current situation of the problem, solutions. In a psychological perspective, authors refer to construction of team mental models, built on the communication based on individual mental model [12].

This process of building a common ground is also referred to as a framing cycle [6] or as managing multiple perspectives by Detienne [15]. The frames are defined as structures of belief, perception and appreciation that guide one’s way of viewing and attempting to solve it. The framing cycle consists of making individuals’ perspectives explicit, making conflicts salient, and building a common ground. In the following work, this construction of a shared understanding is detailed into two sub-sections: (1) perspective clarification, where the perspectives are explicated and (2) convergence: which is building a common frame.

Perspective Clarification

As mentioned above, it is important that design actors externalize and communicate their frames. In the literature, the creation of common ground is reasoned to improve the effectiveness of communication. Stumpf and McDonnell [17] claim that: “the team’s interaction to share frames provides a legitimate indication of the quality of team processes”. Visser [18] also underlines the importance of creating a common ground during the co-designing activity, with her words: “It is then essential that designers, who each also have their personal perspective, establish a ‘common ground’”. Then, the design actors can create shared representations, which “concern agreements, especially on the definition of tasks, states of the design, references of central notions, and weights of criteria and constraints”.

Different mediums of communication may be used for accomplishing this purpose, such as, conversation or sketching. For example, in conversation, the aim is to ensure that what has been said has been also understood. Creation of a common vocabulary can greatly improve perspective clarification within the team [19]. Conklin et al. [20] argue that shared displays are also beneficial to clarify the disagreements in a group: “When ideas and concerns are mediated via a shared display, challenges to positions assume a more neutral, less personal tone. It helps participants clarify the nature of their disagreement”. The roles of mediations in this perspective clarification has been shown in many studies [4, 11, 21–24].

Convergence

It is probably inevitable that there are disagreements within a design team. Positively, the divergence of opinions can stimulate creative ideas and solutions to the problems [25]. The diversity can enriched the results of the team [13]. The task conflicts can enforce team members to realize deeper analysis, which can increase learning and development of new and creative insights, and lead team to be more creative [26]. However, when the conflicts are not managed effectively, they can slow decision-making and keep members away from concentrating on the real task. They can also increase tension between team members, and cause interpersonal conflicts that can be detrimental to the creativity process [25]. Thus, a design team, which desires to reach an acceptable conclusion to their design task, has to find ways of resolving, or perhaps avoiding their conflicts [27].

As a traditional point of view, generally accepted opinions are chosen through negotiation. The notion of negotiation describes the way that the design actors reach agreement, which is based on argumentation [15]. With argumentation, the designers try to “convince themselves and their peers of the sense and validity of a particular solution, or of the necessity to respect a particular constraint related to the problem” [28]. As Détienne mentions, negotiation does not force a person to accept an argument but the conversation, which covers the arguments for and against a frame, makes it possible to get an agreement. The measure of agreement is quite difficult. In [12], authors claim that “the measurement of team mental models should reveal the degree of convergence among team members”. They explore propositions of the literature to access and measure mental models. External representations, interviews, team observation and graphics are used to access to mental models. In the next section, we will detail how we propose to use external representation to find indicators of convergence and perspective clarifications without tracking complete mental model, following the works on intermediary objects a mediation of design activity and the roles of objects as external representations of future product or design problems [21].

Scenarios for Shared Understanding of Functional Requirements

Despite their popularity, there is no common definition of what the term “scenario” means, their use also varies widely in different design contexts. In this paper, the term scenario is used in the same sense with Carroll [8], stories about people and their activities. Each scenario includes the setting, agents/actors who have specific goals/objectives and sequences of action and events [8]. In the early stages of the design process, talking about the end users and their actual activities allow designers to elaborate the requirements, analyze and prioritize them. They also guide the projected scenarios, which explain the future activities, after the creation of the new product. In that way the designers evoke new views on defined needs and define new ones. In other words, scenarios are used to help designers to focus on end users and their activities and how these activities may be changed because of a new design. They serve as a communication tool between designers.

In the literature, even if the focus is on end users and their needs, some researchers prefer to use vague definitions of end users while building scenarios. For example, in Carroll’s scenarios [8], we do not see the detailed description of the users; generally just a name or the job description. However, Cooper [7] argues that by focusing on the behaviors and the goals of specific end users, the designers can satisfy a particular class of users with similar goals. Cooper proposes the utilization of personas-representative user archetypes-, to provoke common sense of categories of end users. Personas are fictional people who have names, details, and goals. They may be presented in their working and/or living environments and tied to particular activities that they are practicing. Cooper’s “goal-directed design” focuses the design effort for achieving persona goals, which covers the goals of the target market. Cooper points out that, once personas have been created then scenarios can be constructed around them. They are used to improve the power of scenarios. Pruitt and Grudin [29] argue that scenarios are less engaging and difficult to memorize when not built on personas. They also mention that personas help to prioritize functions for a product development cycle and facilitate decision-making process [29]. On this basis, we formulate the hypothesis that scenario and persona usage might encourage a shared understanding of functional requirements within a design team. In order to test this hypothesis two questions are posed:

  1. 1.

    How to test if the design actors converge through a shared understanding of the requirements during a design meeting?

  2. 2.

    How to evaluate if the scenarios and personas are effective in creating shared understanding between design actors?

The following section will present the empirical study we had carried out to answer this questions.

Design of the Empirical Study

Design Situation Observations

Video recording is often used in order to observe and understand the design activity [30]. Audio and video captures make it possible to ensure reliable analysis. Hicks et al. [31] proposes a process model in order to realize a structured observation. This is an iterative approach that involves five main phases: (1) Monitor, (2) Capture, (3) Analyze, (4) Prepare, and (5) Intervene. In the monitoring phase the researchers define what will be monitored during the design activity: the actors, their interaction, the objects, etc. The technology and the tools that will be used for monitoring are also prepared in this phase. The inputs, outputs, content and relationships between activities and interactions are then captured in the second phase. In the third phase the data is analyzed and interpreted. The last two phases are respectively the preparation of new tools or methods that will have the impact on the activity and ensuring that those interventions are beneficial.

It is admitted that each situation of design is unique and context embedded. Comparing and analysing design situations can be difficult. Visser tends to highlight the generic characteristics of design and different forms of design that appears in different design situations [32]. Hicks et al. also recommend a fine definition of the design situation to perform observation. According to Prudhomme et al.’s model [28], a design situation contains four main elements: task, actor, object and environment. A design task expresses a goal and the conditions in which work should be realized, whereas the design object, or the product is the entity on which designers work. The design actors are the people who are involved into design process. Finally the environment element is described by the industry, the available technical means and the project organization. This model gives a macroscopic view of a design situation. By taking this model as a reference, the relevant considerations that have to be addressed in an observational research can be defined. In this research we realized an empirical study in a laboratory environment, which is based on the Hicks et al.’s process model. The detail of this study is presented in the next section.

Framework of the Empirical Study

The empirical study is built in different steps, with two groups of designers. A control group, which won’t use the scenarios and personas as a method during the design meeting, was used. Comparing the results of the control group (referred as group A) and experiment group (referred as group B) might help us to evaluate the effectiveness of scenarios and personas.

The main design task is completed by individual tasks that allow extracting individual external representation of requirements. Consequently, three main steps were defined for the empirical study:

  1. Step 1:

    An individual step in which each participant builds his own representation of the product specifications.

  2. Step 2:

    The design meeting step: during this stage the subjects elicit the functional requirements collectively. While the experiment group is asked to use scenarios and personas, the control group is free in the choice of a working method.

  3. Step 3:

    A second individual step to make explicit the participants’ representations of the product after the group meeting. The same representation media than in the first step has to be used to make the comparison easier.

In addition, because the participants have to be prepared for the study, a preliminary training phase is required. In these preliminary steps, the participants are trained to the tools and methods that they will use during the study.

A set of 4 experiments had been run in French and in English. Some of the experiments were conducted by other researchers to validate the replicability of the protocol. Not all of these experiments have been analysed today, and in this paper we focus on one of the experiment realized in U.K. For this reason we can consider this paper more than a case study as it is studying only one experiment even if this case is part of a larger experimental protocol.

The Design Situation

Actors

In this paper, the design actors are the subjects who participated in the study. Because our focus is to analyze the collective activity, we had to use more than one participant. Studies showed that in functional analysis teams with more than 5–6 people, tend to be divided into small informal groups with only a core of 3–4 people doing real work [33]. So, we decided to use 4 participants in each of the two groups (group A and B). The composition of the groups was configured to be as similar as possible using PhD students and postdoctoral research engineers with engineering degrees and similar levels of experience. All participants were volunteers and were not remunerated. They were not informed about the research question. They were told their collective activity would be observed and recorded (video and audio) as a part of the study. However, after the experiment a presentation was given to explain the research context and answer to the participant’ questions.

Task

The focused domain of the research is industrial design. We chose to construct a design meeting typical of the early stages of a new product design process, during which design actors elicit functional requirements of the new product. It was also decide to limit the duration of this meeting up to 90 min.

Before the design meeting, the participants are given some time to think individually about the product idea. They are asked to represent the product idea in the form of a 5W table (When, Why, Who, What and Where), which provides information regarding their individual perspectives about the product specifications. The question How was eliminated from the original 5W1H approach because it could possibly focus the subjects on the technical possibilities, hence limiting their perspectives. However, the aim of this step was to focus subjects on generating alternative solutions not creating a specific one (divergence). The duration of this was fixed to 20 min. Again, after the design meeting the participants are asked to fulfill a 5W table in order to see if the discussions changed their individual perspective about the product specifications.

During the design meeting, they are asked to elicit the functional requirements collectively in the form of a Function-Criteria-Level (FCL) table. As mentioned before, while the group A was free to use any method(s) they felt appropriate for defining the functional requirements, the group B was required to use the scenarios and personas.

Object

The product to be worked on was a “digital calendar”. The product idea was chosen from an open innovation web platform. This site allows its visitors to submit new product ideas, to commit arguments, or to make commentaries about product ideas. We had three reasons for choosing this design object: (1) We had the opportunity to analyze from the platform the discussions between the various contributors to this product idea, which gave us a possible list of requirements, making it possible to test the acceptability of the experiment; (2) As subjects have a very limited time for achieving the design task, materials have to be simplified. So, we have chosen a product idea for which the participants may feel familiar with and contribute to easily. Nevertheless, we checked that all our subjects were naïve regarding this product idea; (3) The idea was pointed out as the most popular one on the site, so that we think that it can be interesting for the subjects to work on it.

Environment

As mentioned in The Design Situation, the environment element is described by the industry, the available technology and project organization. In this research, because the study was realized in a laboratory layout, the industry was not considered.

The available technology for the subjects during the design meeting was limited with the supplied facilities. During the before and after steps of the design meeting, in order to realize their individual tasks, each subject was provided with a computer. The previous research on sketches shows that they play an important role in design process. As Ferguson [34] states: Many features and the qualities of the objects that a technologist thinks about cannot be reduced to unambiguous verbal descriptions: therefore, they are dealt with in the mind by a visual, nonverbal process. Thus, the subjects were also supplied with some draft papers and pens in each step, in order to allow them to sketch or write freely.

During the design meeting, both of the groups were provided with a computer for completing FCL table. Group A was also provided with a whiteboard and board markers that they might use to apply their methods, while as the group B was supplied with another computer in order to create the personas and scenarios in PowerPoint format. Because they had a limited time, group B was asked to use media which is easy to create and manipulate such as text or storyboards. Thus, they were also supplied with a set of pictures selected randomly from Google’s image library (which were rooms of a house, an office and a selection of faces) that might be used for scenario and persona creation.

In terms of the project organization, within each group, one of the subjects was proposed as the manager of the design meeting according to his/her previous experience of managing. His/her role was to manage the time, ensure that the tasks would be realized and organize the relationship between the subjects. The choice of a manager may have positive or negative effects, which is not within the scope of this paper. Otherwise, all the subjects had all equal rights during the meeting. The subjects were trained before the experiment with the aid of pre-prepared material. This included a document containing the explanation of the tools and methods (in additional to the information supplied to group A, the group B was informed about scenarios and personas), and examples of their usage. A formation document was also prepared for the manager in order to explain his/her responsibilities. Both of the documents were sent to subjects three days before the experiment via e-mail.

Observation Protocol

Monitor

The design meetings of the subjects were video and audio taped. Therefore, an observatory room was prepared, equipped with video and audio recording facilities. A voice recorder (placed on the table) and three movie cameras recorded the design activity (see Fig. 1). The movie cameras were installed to capture different views: a close view of the subjects when sitting at the table, the whiteboard and overhead view of the table. They were fixed and were not moved or repositioned during the session for not disturbing the participants. The experimenter and the recording equipment were situated in a neighboring room that the participants could not see. For group B, the same experiment layout was used with the difference that the movie camera recording the whiteboard was removed.

Fig. 1
figure 1

Observatory room view and 4-PIP (group A)

Full size image

Capture

The outputs of the each step were captured for the analysis phase. The same process was also followed for group B with the exception described below in step 2. The different steps of the empirical study are captured as follows:

  1. Step 1:

    (20 min) The 5W tables, the sketches, and the rough drafts are captured in this step.

  2. Step 2:

    (1 h 30 min) During the meeting the three camera views and the computer screen are recorded and mixed into one 4-PIP (four pictures in picture) combined view (see Fig. 1). A time stamp of the date, the time in hours, minutes and seconds is included in the video image. Group B’s 4-PIP combined view contains two computer screens (one for the FCM table and one for the scenarios and personas’ computer) and two camera views. For group B we also capture the created files for defining scenarios and personas. All potential documents produced during this step are also captured.

  3. Step 3:

    This step consists in three sub-steps:

    1. Step 3.1:

      (5 min) In this step the subjects are asked to rank the 5 most important requirements from the FCM table that they collectively created before. The aim was to know if the subjects of a same group would assign the same importance to the defined requirements. The individual ranking tables are captured.

    2. Step 3.2:

      (10 min) New vision of the problem: new 5W tables completed individually by the participants are captured.

    3. Step 3.3:

      (10 min) In this last sub-step the structured open question interviews were realised with the participants. There are three reasons for conducting these interviews: (1) to check if the participant is in agreement with the group results—i.e. to gather subjects’ individual perspectives on the FCM table completed collectively; (2) to understand the argumentation behind their ranking table; (3) to get some comments and critics on the design tools and methods used during the meeting. The interviews were audio recorded for a later analysis.

Analyze

This step of the process model defines how the data captured during the three main steps of our empirical protocol are analysed. This analysis is performed in two steps: evaluating the validity of the data, and answering to the research question.

The validity of the data is pointed out by Bryman [35] as a key issue in social research. The main types of validity are:

  • The internal validity: which is concerned with the causal relationship between the variables and the gathered results. In this research, in order ensure the internal validity, we have to be sure that the control group did not also use the scenarios and personas as a method.

  • The measurement validity: the measures gathered from the analyzing method have to be verified. In our study, the measures are often the result of a coding process of the raw data (video, transcripts, design deliverables, etc. To control the validity of this coding process, a double-coding is systematically performed, and the coding results of each coder are compared using the Cohen’s Kappa Calculations. Cohen’s kappa coefficient is a statistical measure used to quantify agreement between two raters for categorical items [36].

  • The external and ecological validity: the external validity is concerned with the question of whether the results of a study can be generalized beyond the specific research context. On the other hand, the ecological validity of the study is concerned with the questions if the findings are applicable to people’s everyday settings.

Answering to the research question is here based on a series of indicators that we defined in order to analyze the captured data. Some of these indicators are quantitative and some others are more qualitative. Some of them are based on an internal analysis, requiring an in depth analysis of the whole corpus of the meeting. Some other indicators stay on an external analysis and are only based on the before and after-meeting deliverables.

These different indicators are presented in the following section.

Problem of Understanding Between the Design Actors

Convergence

As a first step, we investigate whether scenarios and personas have an impact on the convergence of the participants to a common perspective in terms of the requirements. The level of convergence in each group is evaluated through three indicators in order to make comparisons.

Indicator 1: Similarity of the Ranking Lists

This indicator is defined using the Spearman’s rho \( \left( \rho \right) \) coefficient [37]. This coefficient is used to observe whether the participants agree to each other’s view, as far as the importance of the functional requirements are concerned. The calculated value of Spearman’s rho varies from −1 to +1 and makes it possible to compare two by two the ranking lists of each participant. A −1 Spearman’s rho means a perfect negative correlation, while a +1 Spearman’s rho means a perfect positive one. A correlation of zero means that there is no relationship between the two variables.

Table 1 shows the results of these calculations for both groups. The last row gives the interpreted strength of the correlation following Landis and Koch’s standards [38].

Table 1 Spearman’s rho for each pair of participants and group average correlation
Full size table

Thus, in group B which used the scenarios and persona approach, the ranking lists of the four participants are clearly more coherent than in group A. However, this result has to be cautiously interpreted since the total number of functions listed by each group is small (15 for group A and 12 for group B). This means that the probability that the ranking lists are coherent by chance is high.

Indicator 2: Convergence of the 5W Tables

This indicator is based on the analysis of the individual 5W tables made by the participants before and after the meeting. The idea is to see whether their individual representation of the product converged to a common one. In addition, comparing the 5W tables before the meeting makes it possible to see if the participants already had common perspectives.

Since a similar idea might be expressed in different words by two participants, a coding schema was defined in order to associate a unique identifier to each of the different ideas in the tables. The coding of each table was carried out by two different coders in order to check the reliability of the schema. This double coding allows the calculation of Cohen’s Kappa index [38] which measures the level of agreement between the coders. Table 2 shows the results of this calculation. These values are generally interpreted as a moderate (0.41–0.60) or substantial (0.61–0.80) agreement, making it possible to analyse the results of the coding.

Table 2 Cohen’s Kappa index for the codings of the 5W tables
Full size table

These results are presented Table 3. In both groups, we observe a decreasing total number of ideas that could be a sign of convergence of the two groups after the meeting. The rate number of ideas shared by all the participants in the group B is higher than in group A; Even if this rate was already higher before the meeting for this group B, the increasing is significant.

Table 3 Number of ideas and sharing of these ideas before and after the meeting
Full size table

This second indicator tends to show the potential impact of scenarios and persona on the convergence of the 5W tables. This result has to be confirmed by other experiments.

Indicator 3: Convergence During Verbal Communication

This qualitative indicator is based on analysing the group discussions to see whether the scenarios and personas had an influence on the way that the requirements where discussed.

In group B, personas create a common reference, each participant being aware of whom they are talking about. For example, in the 55th min, participant 3 says “[…] because when her daughter’s using it, she will be there to support her child as well.” At this point, everybody knows exactly who the daughter is since she was previously defined as a persona.

Moreover, the participants used personas as a medium to communicate their viewpoints. In other words, they made reference to personas, while presenting their arguments. For example, to support his point of view that the product has to be portable, in the 40th min of the meeting, participant 1 refers to the persona Emily and creates a fragment scenario around her: “I think one of the criteria is portable, isn’t it? So, she can take it into garden and play with her teddy.”

They also evaluated requirements and make decisions by referring to the personas. For example, in the 12th min of the meeting, participant 1 eliminates the requirement “reminding the bills” by referring to a specific persona: “I think one of the usefulness of the calendar is to see when the bills are due and stuff like that. […] But he is not going to be using those sorts of things.”

Moreover, usage of personas helped the participants to identify conflicting requirements. In other words, they realised that a requirement, which can be essential for a persona might be disturbing for another one. For example, while discussing about “sharing the personal planning with the other users of the calendar”, participant 1 says referring to the son Clayton: “[…] He wouldn’t use it then. If he knows that mother can see everything.” A total of nine proposed requirements were eliminated in similar ways, because they were not appropriated to one or more personas. The participants clarified and also strengthened their arguments based on the personas’ characteristics and on the scenarios created around them. Due to the fact that all the participants built personas collectively, this created common references and negotiation process was easier to take decisions.

In group A, we did not notice such constructions. The participants did not discuss about, who are the real users of the product. Moreover, there is no elimination of the requirements. The group just focused on listing the functional requirements. These primary observations, exhibited here by selected quotes, require deeper investigations for extracting quantitative elements from the observation. Metrics should also be found for this third indicator.

To summarize on the convergence issue, the first two indicators do not lead to a cut-and-dried conclusion, giving contradictory results, of at best showing slight differences which cannot be reliable due to poor-sized samples. On the other hand, the qualitative analysis of the transcript clearly shows a positive impact of the scenario and persona on the convergence of the participant to a common perspective.

Perspective Clarification

Indicator 4: Mutual Awareness

Because the participants can have a good mutual understanding without necessary sharing a single vision of the design, we also considered the potential awareness of their agreements/disagreements as an indicator. The latter is based on analysing the post-meeting interviews of the participants, and more particularly their answers to the questions of whether in their point of view, their ranking of the functions would be shared by the group. In the case that their answer was negative, they were asked to comment on the differences that could exist.

Table 4 presents their answers to these questions. In group A, two participants think that the ranking lists will be different, which is quite true (see Table 2). Participant 3 is undecided. Only participant 4 estimates that the group’s ranking list are similar, except the one of participant 3. In contrast with his assumption, according to Table 2, participant 4 has a substantial positive correlation with participant 3. In group B, they commonly imagine that their lists are similar, at least the first three functions. Analysing their lists shows that two participants have a common list, but not the same ranking. It comes also that two participants have the same first three functions in their list. Moreover, two participants claim that functions F4, F6 and F1 are the fundamental ones and should be listed by everyone, which is quite true. On the other hand, participant 1 believes that his ranking order is different from other group members’ one. He is partially wrong in the sense that he has an almost perfect agreement with participant 2 (ρ = 0.98). The differences between the two groups are too narrow to make a clear distinction. We consider that in both groups the participants were similarly aware and unaware of their disagreements and agreements.

Table 4 Comments of the participants about their ranking of the functions
Full size table

Conclusion and Next Steps

The results presented here are from the analysis of one experiment while we conducted four of them. This paper can be considered as a preliminary case study, even if we developed a replicable and reliable protocol to estimate the impact of scenario and persona on the shared understanding of functional requirements. A description of the design situation was proposed to facilitate the replicability and double coding was used to ensure the validation of the analysis. We proposed indicators to measure the share understanding of design actors within the team involved. Two Indicators are based on external representations produced by the designers, two indicators are based on conversation analysis and post meeting interviews.

The study shows that all the groups have difficulties to converge to a common ranking of the functions identified in the meeting. The correlations calculated, even when positive, are still moderate according to correlation coefficient. Our analysis shows that the Scenario and Persona usage influence positively the correlation. But the 5W analysis doesn’t show a strong, unique and common description of the product to be designed. Even if the group agreed to a list of requirements in the FCM tables jointly produced, the individual rankings and the individual representations of the problems mediated by the 5W sheet, remain different. Similarly, the study shows that the groups have difficulties to clarify their perspectives. The groups are not especially aware of their agreements and disagreements.

As a limitation of the analysis, we can mention that the study doesn’t allow to definitely concluding on the positive impact of scenario and persona on the shared understanding. The study should be expended, enlarging the number of situations observed to confirm the observations. The external indicators proposed to measure shared understanding have to be completed by the analysis of the discussions during the design meeting. The in-depth study of the verbal interaction is not presented here, but some preliminary results of the conversation analysis show qualitative differences between group A and group B. The Scenario and persona are used as argumentation and for negotiation. The persona serves in the elimination of some requirements. Thus the study shows also a significant increasing of the number of functions discussed during the meeting. We also observed that, in that phase of requirements elicitation, fragments of scenarios are always used by designers. But changes are observed from designer centered scenario based on designers’ experience to scenario involving personas. Thus, other impacts of scenario and persona are highlighted by the study, but their complete analysis requires new indicators.