Keywords

1 Introduction

The end-users influence how and in which contexts a software system will be used. Therefore, it is crucial to understand the context of use from the beginning, such as end-user goals, tasks, and characteristics of the environment [9]. Moreover, an increasing number of end-user groups has an impact on the number of contexts of use of said software systems [1]. However, user requirements as well as the contexts of use are often not sufficiently considered when designing the software [5, 12]. This is aggravated by the fact that the context of use of a software might change. For example, since no one can foresee future contexts of use [5], or due to unexpected external factors such as the pandemic situation the world faces these days.

Nevertheless, there are also cases in which the software system is known to be used in various contexts due to a large user group [8]. If this is known (or likely to be true) at the beginning of the development phase, taking this knowledge into account can improve the applicability of the software in the end, and it can reduce the necessity of future adjustments to different contexts.

As workshops have been proven to be meaningful for the requirements elicitation [7], we assume that workshops can also help identifying different contexts of use during the requirements elicitation phase. In addition, workshops are also an appropriate means of identifying (further) stakeholders. Following this line of thought we have conducted a workshop in the context of the cluster of excellence PhoenixDFootnote 1 in which optical systems, design and simulation tools are merged with production technologies. One of our tasks in this cluster of excellence is to develop a software that allows to connect different contributors in two means [8]: (1) to interact and collaborate with each other, and (2) to exchange different kind of data and knowledge.

In this paper in hand, we present our experience conducting an online workshop during the pandemic. The goal of this workshop was to elicit requirements from the crowd of different end users in order to identify as many contexts of use as possible.

Outline. The rest of the paper is structured as follows: Sect. 2 presents related work. In Sect. 3 we present the used methodology including the workshop structure. Section 4 summarizes our results which we discuss in Sect. 5. We conclude our paper in Sect. 6.

2 Related Work

The idea of using workshops to collect ideas is not new in scientific contexts. Ørngreen and Levinsen [10] discuss workshops as research methodology. They analyze five studies on teaching through video conferencing and argue that workshops are suited for domains that concern prospective subjects. The elicitation of requirements is one such domain, as the discussed project is not fully defined yet. A number of papers present workshop designs for such requirements elicitation processes.

Scherr et al. [11] evaluate the use of Amazons Echo Show device by elderly people through multiple different methods. An initial interview was conducted following the first interactions of participants with the device, before intermediate interviews were performed every three months. Additionally, the researchers organized group meetings akin to workshops. Due to constraints related to the Covid-19 pandemic the intermediate interviews were conducted using video calls at a later stage of the experiment. The three methods lead to clear insights into the relationship of elderly users and the installed Alexa devices as well as information on requirements specific to their context of use.

An experience report by Haase and Herrmann [4] presents a workshop design for requirements engineering purposes in virtual settings. Their workshop makes use of the 6-3-5 method as well as the six hats of de Bono [2]. A separate prioritization workshop was then conducted to prioritize the gathered requirements using the Hundred Dollar TestFootnote 2 as well as the Kano model [6].

Edwards and Sridhar [3] conducted an exploratory research study with 24 virtual teams consisting of university students based in Canada and India. Each team was paired with another group from the other university and tasked with the definition of requirements regarding a business information system. After the project phase participants were asked to fill out a survey. The researchers found indications for the importance of a good structure and ease of use of technologies regarding the efficiency, effectiveness and quality of virtual team projects.

In this paper we present a design for creative requirements elicitation workshops in the context of a virtual meeting that strives to identify different contexts of use as a by-product of the requirements elicitation phase. With in person meetings not being feasible due to Covid-19 restrictions, well designed processes and a thoughtful selection of supporting tools are required to increase the chance of a successful and satisfying workshop. Our design looks to identify further relevant stakeholders and to elicit requirements from end-users with different contexts of use (e.g., lab data exchange, knowledge exchange, data exchanges between machines, etc.) participating in an online meeting.

3 Methodology

We conducted the research presented in this paper in the context of the cluster of excellence PhoenixD. The main objective of PhoenixD is the creation of optical devices for different contexts of use. Several researchers from different universities (mainly Hannover and Braunschweig), as well as the Laser Zentrum Hannover and the Max Planck Institute for Gravitational Physics are involved in the project.

3.1 Case Project

The Software Engineering Group of the Leibniz University Hannover is also involved in the cluster of excellence. The large team of researchers and practitioners needs to exchange different kinds of data and knowledge. Working on such a project requires a close collaboration, an interactive exchange of information, and especially a well structured requirements engineering process. The challenge here is not only to design a software system for different contexts of use (e.g., data exchange, collaboration, different requirements of the various scientific disciplines) that is tailored to the needs of many end-users, but above all to create an infrastructure for data exchanges within the cluster [8]. However, the contexts of use can be subsequently divided into several other contexts of use, which are not completely known. Therefore, this software system is a meaningful case project to which we applied our idea of using workshops with creativity techniques to identify as many different contexts of use as possible as a by-product of the requirements elicitation process.

3.2 Objective of the Workshop

The workshop aimed to collect requirements from as many end-user groups as possible, as well as to identify different contexts of use to allow for a widespread use of the software afterwards. Therefore, the workshop aimed at discussing the data exchange in the research project, as well as at answering two questions:

  1. 1.

    What kind of data needs to be exchanged, i.e., what defines the data exchange?

  2. 2.

    What is needed for the data exchange, e.g., on the infrastructure level?

3.3 Workshop Agenda

The workshop lasted one morning and consisted of several parts which are described in the following. Table 1 presents an overview of the different parts of the workshop.

Table 1. Overview of the workshop structure
Full size table

3.4 Creativitiy Techniques Used

We opted for two different creativity techniques (Creative Task No. 1 and No. 2 in Table 1) to extract as many different requirements as possible - both must-have and nice-to-have requirements. We started with the 6-3-5 method: a brainstorming technique in which six participants write down three ideas and then develop existing ideas or write down three more ideas five times. In contrast to the classic brainstorming, ideas are not collected in a central place like a flip chart. Instead, each participant is given a single sheet of paper to write on. After each iteration the sheets are passed around the table, which means that each participant contributes to every sheet once. The method thereby allows every participant to present their most important ideas while also letting them contribute to other lines of thought. To simulate the 6-3-5 method in a virtual setting, we set up virtual sheets of paper that were accessible through static URLs. For this, we used an instance of OnlyOfficeFootnote 3 hosted by our university. After each iteration we provided participants with the URL of the next sheet.

The second creativity technique used was the Walt-Disney method. Each participant runs through three roles: Dreamer, Realist, and Critic. Within these roles they are asked to voice hopes, general comments and concerns. In the Walt-Disney method, all participants work simultaneously. Results are collected in a central place and written down by the moderators. For the virtual setting of the presented workshop, we used an online whiteboard.

We chose the 6-3-5 method at the beginning because this creativity technique is well suited for finding ideas in connection with concrete questions. The subsequent Walt-Disney method is already well suited for smaller groups. It also serves to generate ideas and refine them. We used the Walt-Disney method to refine the results of the 6-3-5 method. Note that participants in this workshop had very little to no previous experience in using creativity techniques.

3.5 Data Collection

In the beginning of the workshop, the participants were asked for their agreement to record the workshop. We guaranteed to only use internal university servers for the data storage. As every single participant agreed to the data collection, we video-recorded the workshop.

In addition to these video-recordings, we used notes made during the workshop and - due to the pandemic situation - the online-setting of the workshop allowed to collect data in a digital format using a whiteboard and digital paper sheets.

3.6 Conducting the Workshop

We conducted the workshop as described above on Feb 24, 2021. Due to the pandemic situation, the workshop took place entirely online using BigBlueButtonFootnote 4 for the workshop itself and MuralFootnote 5 as the whiteboard. In total, six researchers from five institutions participated in the workshop.

From the author team of this paper, two researchers conducted and moderated the workshop. In the background, two other researchers (who did not participate in the workshop) helped by clustering and processing the results. After having started the workshop with a welcome as presented in Table 1, we continued with the 6-3-5 method regarding the questions presented in Sect. 3.2. Hereafter, the data was processed and clustered (while the participants could refresh in a coffee break).

Subsequently, participants were given 15 minutes to read the clustered results of the 6-3-5 method and prioritize the aspects that were most important to them. For this, each participant had a total of 3 votes, without prioritization within their own votes. That is, a participant could also have rated one item with three votes, even though this did not happen. The voting was conducted across the clusters. After the 15 min had elapsed, the aspects that had just been rated were sorted in descending order of their rating.

Based on this rated data, the Walt-Disney method was conducted. To do this, all participants as a group took on one of the three roles in turn and advanced the vision of data exchange. Finally, each participant was asked to formulate a personal conclusion about the insights and points of the workshop that were most important to them.

4 Results and Interpretation

Using the results from the 6-3-5 method, a total of 63 ideas were collected. Afterwards, we assigned these ideas to the following clusters: Technical Solution, Types of Data, Collaboration Partners, Problems so far, Restrictions, and Types of Exchange. In the subsequent prioritization by participants, 12 ideas were highlighted and deemed particularly important. The three highest rated ideas were used as the basis for the Walt Disney method. When adopting the role of the dreamer, participants produced eight refinements of ideas which were analyzed in terms of their feasibility by the role of the realist. In the role of the critic, participants examined their proposals constructively and critically.

From the results of both methods, we were finally able to identify the following first contexts of use: Data Diversity, Unified Data Annotations, Long Term Use of the Data, Devices and Platform, Data Exchange, and Data Authority. The context of use Data Diversity includes the consideration of different types of data, e.g. lab data, simulation data, measurement data, data for production as well as specific parameter information or knowledge of persons involved. Unified Data Annotations refers to a unified approach to annotating data. It is important that the data is searchable and easy to find. Simultaneously, it is critical that the source of the data and the data itself are comprehensible over the long term. This is to be achieved by designing a uniform concept for embedding metadata. The latter aspect connects directly to the context of use Long Term Use of the Data. Until now, data has often only been usable in combination with an analog lab book. This will no longer be necessary in the future, as the data, including all dependencies, will be available digitally. In addition, data must also be understood by subsequent researchers, and a link from raw data to the corresponding publications must be ensured. Another context of use was identified as Devices and Platform. The system to be developed, or the emerging platform, must adapt to different end devices. This means, for example, that only certain information is provided depending on the performance of an end device. In order to enable data exchange between different researchers and disciplines, various aspects such as conversion of units, filtering of data, transmission of data, etc. have to be considered. These aspects belong to the context of use Data Exchange. The final context of use identified in the workshop was named Data Authority. It is intended to ensure that there is a superordinate instance that monitors the efforts or constraints surrounding data exchange within the cluster of excellence and ensures that operations are as error-free as possible.

In addition to the aforementioned results of the workshop, we were also able to identify other stakeholder groups for future iterations of the workshop. The final feedback from all participants was very positive. They expressed that the creativity techniques used during the workshop made it easier to envision the goals and requirements around the planned data management system. Based on this feedback we plan to conduct further editions of the workshop.

5 Discussion

Based on the insights we gained during the workshop, we can point to some advantages and disadvantages of such kinds of workshops both related to the workshop itself as well as to the online-setting due to the ongoing pandemic. We present these insights in the following and discuss their limitations.

5.1 Advantages and Disadvantages Due to the Pandemic

The workshop was conducted in the beginning of 2021, when there was the worldwide Corona pandemic. This required some adjustments compared to a non-pandemic situation.

Of course, an online workshop has disadvantages compared to a workshop in person. For example, interactions are way more difficult, in particular, if they are bi-directional and do not affect the whole group. In addition, the quality of the workshop and, consequently, of the results, depend on the technical equipment of the participants. For example, a low bandwidth complicates interactions and communication, and may lead to misunderstandings or lost information.

Nevertheless, according to the experiences we made during the workshop, meeting virtually also has some advantages. For example, results can be easily collected on a whiteboard that is accessible for all participants at the same time, and everybody can contribute to the data collection (without the problem of unreadable handwriting on a physical whiteboard). In addition, it appears to be easier to motivate people to participate in the workshop as they only need to be available for the real workshop time (and not, e.g., for traveling). This also facilitates to find appointments where as many participants as possible can join the workshop.

Summarizing, despite the fact that the pandemic shall end as soon as possible, we all - researchers and practitioners - can benefit to some extent from the experiences made when meeting virtually. There will be cases, when having such an online workshop is more practical even when the pandemic is over.

5.2 Take-Aways for Researchers

Given the experiences we made and the advantages and disadvantages we faced due to the online setting of our workshop, we have four take-aways that will influence our future research:

1) Creativity workshops with end-users are a suitable method to find different contexts of use. Considering the number of requirements we elicited during the workshop, we would encourage other researchers, requirements engineers, etc. to conduct such a workshop when different contexts of use should be taken into account. During the workshop, we got to know different viewpoints at the same time, and participants were able to extend or correct statements made by others.

2) Include creativity techniques in online settings. Creativity techniques are at the core of most workshops as they allow for interactions. Interactions are also possible in online-settings, but they need to be planned. Simply using creativity techniques from “normal” workshops often does not work in online-settings. Nevertheless, there are possibilities to increase the amount of interactions in online workshops.

3) Think about creativity techniques when deciding which tool to use. The selection of tools for an online-workshop should be thoughtful. However, the chosen creativity techniques have an influence on the choice of the tools to be used: Is there a need for a whiteboard? What about breakout rooms in which smaller groups of participants can converse without interruption? Surveys? On-the-fly video recording? Besides the requirements of the concrete workshop, it is meaningful to look at tools the participants are familiar with. The effort for the participants to join the workshop should be as small as possible.

4) Think twice whether a workshop in person is necessary. Given the advantages described above, it is meaningful to ask whether a workshop would benefit more from a meeting in person or from more participants being able to join. We expect that a meeting in person is not always the best choice.

5.3 Limitations

The insights we present in this paper emerge from only one workshop with six participants. Therefore, our results need to be taken with care and must not be over-generalized. Nevertheless, the results of the workshop are promising and motivate the further use of workshops to elicit requirements for different contexts of use from a crowd of end-users. We plan to expand the single workshop into a series of workshops throughout the software system development process to continuously engage end-users and collect as many insights and ideas as possible.

6 Conclusion

Workshops are frequently used during the requirements elicitation phase. This is a meaningful point in time to identify different contexts of use as this can help facilitate the further process and might help to improve the acceptance as well as the usability of a system. In this paper, we present the results of such a workshop conducted in the beginning of a software project of which we knew in advance that there will be a wide variety in the contexts of use.

Using two creativity techniques, namely the 6-3-5 method and the Walt-Disney method, we identified different contexts of use with only six participants from a large end-user group participating in the workshop.

Besides, we also experienced advantages and disadvantages when this kind of workshop needs to take place online, e.g., as in our case, due to an ongoing pandemic.

Despite the limited generalizability and credibility of our results, the insights we gained during the workshop motivate future work striving to improve the workshop structure to identify as many contexts of use as possible before starting the development phase of a software project.