Keywords

1 Creative Problem Solving

Many activities in software engineering (SE) are related to creative problem solving (CPS) [10]. Requirements engineering (RE) and user experience (UX) rely on an increasing number of methods that involve interdisciplinary teams engaging in collaborative face-to-face activities [6]. Design thinking, design sprints and innovation workshops (IWs) [4, 11] provide methodologies for solving business and design problems alike [3]. These involve many different methods (e.g., affinity diagrams, card sorting, brainstorming, brainwriting, storyboarding, low-fidelity prototypes, etc.) centered around face-to-face communication and the use of analog materials. Their success lies in their ease of application, as a team of professionals from disciplines such as RE, UX, and design can follow a structured approach to solving complex problems. IWs help to understand challenges, user needs, and requirements. One aspect that we think deserves more attention from both research and practice is the integration of these methods’ results into subsequent SE, RE, and UX activities. Karras et al. mention that written requirements specifications lack communication richness and effectiveness, and propose the use of videos in RE [7], especially for communicating project visions. Ideas found in IW an to be used in subsequent SE activities, are often expressed as concepts or goals, rather than concrete requirements, and need to be transformed [12]. Barrios et al. note that, in order to be capitalized on, results from IW need to be formalized and conserved [2].

2 Problem

In our business practice, we have conducted more than 40 IWs with different clients from research and industry in various domains, and have observed that many organizations struggle with actually implementing solutions found in CPS activities in later project stages. Actually utilizing the results in subsequent activities highly depends on whether they have been documented in a suitable manner. Typically, results created in IWs are built on initial hypotheses (e.g., problem statements, user needs, solution approaches) that need to be verified later on. Potential solutions need to be tested for their feasibility and applicability. Hence, appropriate documentation of workshop results and its availability is crucial for facilitating later processing and later application.

During an IW, participants create a large number of different artifacts. Within a typical two-day workshop, interdisciplinary teams compile a list of challenges (typically 30–60), analyze some of these challenges in detail (3–9), come up with many different initial solution ideas (approx. 600), select a subset of promising ideas, create storyboards or low-fidelity prototypes, and assess the solutions through presentations and discussions [1]. During face-to-face group activities, participants discuss the problems, ideas, and solutions, form mental models, bring in their own professional experience and background knowledge, and reason about the inclusion and exclusion of aspects found during and before the workshop. Even though many results are written down on sticky notes, paper cards, flipchart paper, and whiteboard walls, we observe that these often lack detail and only serve as mental anchors during the IW. Participants are busy following the creative process. Writing down details of their discussions that are not of immediate use slows down their thought processes, hence these are usually omitted.

Memories of details known during an IW fade with the time passing until its results are picked up or until subsequent activities are to be performed. One obvious means of documenting IWs is to take pictures of any results created, but these can only cover what is actually written down. Dedicating one participant to the documentation pulls her out of the group activity. Having a separate person (e.g., a co-moderator) doing the documentation can impose feelings of being observed or monitored. Audio and video recordings would provide the most detailed form of documentation, but a typical two-day IW with three sub-groups leads to about 42 h of recorded material. In order to make it usable, it has to be processed after the IW, either manually, increasing the cost (in person-hours) of the workshop by a large amount, or with the help of automation [8].

A more effective way of documenting IW results would enable them to be used more efficiently in subsequent activities, and in turn increase the applicability of IWs in RE. Understanding the requirements of the subsequent activities using these outcomes allows making targeted improvements to the way IW are documented. To collect initial evidence in support of this idea, we conducted an expert workshop.

3 Method

A workshop with usability and UX experts was held in which they discussed and analyzed the challenge and came up with initial solution ideas by applying CPS methods themselves. The workshop took place at the 2019 “Mensch und Computer (MuC)” conference held in Hamburg, Germany, and was part of the Usability Professionals (UP) track. Twenty-eight experts (23 female, 5 male) and two moderators were present throughout the 90-min workshop. One expert (female) joined later. Twenty-one disclosed their affiliation with a professional organization or company, two with a research institute or university, and six did not disclose their affiliation.

Session 1: At first, the problem of incomplete documentation was presented to the attendants in order to establish a common understanding of the goal of this expert workshop. In a twenty-minute presentation, we presented our typical approach to structuring two-day creativity processes [1, 4]. We used the photo minutes of an example IW held in October 2018 about a ridesharing solution in small communities, and showed images from the photo minutes for each phase of the IW. During the ridesharing IW, we had applied creativity methods to explore the problem space, analyze details of high-priority problems, come up with a large number of ideas, build solution scenarios, reiterate them with transformational methods, and conclude with prototyped solutions. The experts were able to understand the structure and creativity methods applied in the ridesharing example through the images shown and the explanations given. For their own reference and to support further discussion, a hand-out was prepared. The actual results of the ridesharing example were not explained in detail, as we wanted the experts to express their own experiences and knowledge rather than discuss our example. We highlighted the documentation problem and concluded the presentation with the key takeaways: 1. Pictures of IW results only show content that is written down, drafted, drawn, or built. 2. Pictures do not show artifacts that are not used or are deemed unusable for the IW topic. 3. Photo minutes cannot convey discussions between participants that may lead to important decisions; only their outcomes. 4. The full context of an IW cannot be reflected completely in photo minutes, as it also includes the background and knowledge of the participants, often embedded into an organizational body of knowledge. Session 2: Directly after the presentation, we had the participants reflect on the presentation and share their own knowledge. The experts could contribute their own experiences with either CPS methods or with documentation of their outcomes. This was done to ensure that the presented problems were understood by all participants. For Session 3, the experts were randomly divided into five groups. Each group was assigned one step of the presented creative process. The experts were given the task to 1. discuss which creativity methods they typically use in their group’s respective step, 2. write down a short summary of how the methods are performed, 3. analyze the types of results the method typically produces, 4. discuss in which activities after the creative process the results are typically used, and finally 5. what problems arise during later usage. The goal of this session was twofold: On the one hand, it should allow all the experts in a group to understand how they all apply creativity methods and form a rapport. On the other hand, we confronted them with the challenge that results of creative processes are used in later activities and the related assumption that this is difficult due to documentation problems. This implicitly includes our claim that proper documentation of IWs is important, as it allows their results to be used in later activities. The experts analyzed nine different CPS methods. Session 4 was concerned with finding possible solution approaches to the challenges identified in the preceding session, which was again done in the subgroups. We allowed the experts to follow any ideation strategy they deemed suitable. Twenty problems (16 distinctive ones) with using the results of creative processes were identified and written down by the experts (e.g., “insights are not transferred”, “other ideas are lost”, “assumptions, reasons, decisions are lost due to swarm intelligence”). Session 5: The expert workshop concluded with a group discussion between all participants, allowing them to share their findings and elaborate on the problems of documentation, respectively the use of results for later activities.

4 Initial Results

The notion that there is a challenge with documentation in CPS activities was shared by all workshop participants. The experts agreed with our idea that later activities determine the requirements for the documentation. One expert group analyzed the brainstorming method to collect problems. Osborn’s rules for brainstorming lead to a large number of results that are neither judged nor relate exclusively to the initial challenge, but are often based on associations that participants follow. The advantage of this approach is that it allows arriving at findings and insights that might have been overlooked or never uttered due to social pressure. But this leads to disadvantages for the documentation. The reasoning behind a single note is not part of it, hence it is lost once the participant forgets it. Additionally, the documentation can get unwieldy due to the large number of different notes, at varying levels of readability. The “Moonshot” or “Think Big” method [5] was analyzed by another group of experts. This method is used to work on product strategy and roadmaps in order to determine long-term goals. Many ideas for potential product features are created, of which only few are further elaborated. Assumptions and decisions made by the participants are not documented well through the “Moonshot” method itself. According to the experts, this happens due to the effects of swarm intelligence: During an IW, assumptions and reasons for decisions are shared, hence not written down. The results of the “Affinity Diagram” method [14] are used in conception, UX design, and implementation, according to the experts. They mentioned problems when using the results in later activities: The method builds empathy with the user, which degrades after an IW. Participants gain insights into the problem space, especially the user’s needs, which are lost due to not being documented well, leading to the potential risk of implementing improper solutions. The “Crazy Eights” is a method [9] that helps to quickly come up with variants of ideas. Within eight minutes, participants draw or describe eight alterations of an initial idea that can be used for comparing solutions and assessing feasibility. The results are low-fidelity due to time constraints and missing descriptions, which makes it hard to use them in later activities.

One solution that might spring to mind is the use of a specific room for groups over the course of working on a topic or project, where all results can stay visible for an extended period of time, typically several weeks, so people do not lose track of any spatial interrelations formed in their mind. Notes can be rearranged to improve readability. Assumptions stay visible until rejected or confirmed. Such spaces can be referred to as “creativity rooms” [13] and provide a good context for projects incorporating CPS methods, but they are seldom available. Only one of the 29 participants has permanent access to such a room for their work. All others need to clean and remove all results from the physical collaboration space after an IW.

5 Further Research Plan

The initial results obtained from the expert workshop support our idea that the documentation of CPS activities, especially IWs, needs improvement. Though the expert workshop provided some insights into the problem of CPS documentation, the initial ideas and proposed solutions are not sufficient for solving the challenges of IW documentation. However, the experts’ first insights into the problems of IW documentation motivate further research to fully understand the challenges and to come up with adequate solutions. The experts came up with an initial set of problems regarding the use of IWs results in later activities. This indicates that IW results do indeed need to be made available in a suitable manner for subsequent SE activities. A better understanding of which activities require input from IW will lead to a clearer scope of relevant subsequent activities. Analyzing different approaches to SE, RE, and UX processes will provide a comprehensive list of activities performed. We plan to elaborate on the analysis of how individual results of IWs can be used best in later activities from different angles.

To understand how the documentation of IW results can be improved, it would be beneficial to understand how it is used. Each subsequent activity should be analyzed in terms of the individual actions performed and the types of input required, such as information about the system to be built, the maturity of the requirements, or user needs. The input types then need to be categorized and condensed in order to be matched with the actual output of IWs. Not all information needs of later activities should be fulfilled by IWs, hence an understanding of result types is also necessary.

A large set of documentations on CPS activities and IW results should be obtained (e.g., existing photo minutes). If available, the documents used for planning the IWs will provide insights into the utilized methods and additional semantic information that might be useful for enriching available documentations. They should be analyzed and the output should be categorized by the type of output created (e.g., problems, ideas, scenarios, prototypes). According to our experience, different methods will provide the same type of output, even though the physical form of how the output is represented differs. On the other hand, one method might produce several types of output, either implicitly (e.g., assumptions uttered during discussions among CPS method participants) or explicitly (e.g., a concrete scenario). The output types should then be matched with the input types of subsequent activities, leading to a subset of IW result types that actually need to be preserved. For these, existing and novel approaches to documentation should be applied and evaluated. Not all means of documentation might be applicable.

Creative methods for groups collaborating face-to-face impose their own restrictions on possible means of documentation. They should not hinder the flow of ideas by overburdening the participants of IWs, neither by forcing them through seemingly unrelated activities nor by adding a feeling of being under surveillance. These constraints should be identified through literature research as well as experimental setups. Methods from IWs can be performed with static challenges and varying types of documentation (e.g., automatically analyzed audio recordings, team members facilitating the documentation, photo minutes). The quantity and quality of the results produced should provide an indication of problems arising from incorporating documentation into the CPS method. Possible documentation methods could be tailored specifically to the input needs of SE, RE, and UX processes, adapted to the given outputs, and incorporate the constraints of CPS methods. We envision different documentation approaches for different methods, which the facilitator will have to choose from, leading to better incorporation of IW results into later SE, RE, and UX activities.