1 Scenario Development for Next Generation Manufacturing

Technological developments can have an impact on firms and society. This impact is often unpredictable, creating a need to manage the involved uncertainty (IPTS Economists Group et al., 2002). A common approach to managing uncertainty is engaging in forecasting projects which involve the generation of future scenarios that outline influencing factors and trends (Gausemeier et al., 1998). The main goal of such forecasting efforts is to anticipate the future (Saritas & Oner, 2004) and to serve as a basis for long-term planning (Courtney et al., 1997).

Similarly, the implications for Next Generation Manufacturing are unclear given the high uncertainty of technological development involved. We provide an approach to developing scenarios for future outcomes of Next Generation Manufacturing which can enable strategic planning by firms and future research. Our research is guided by one core question: how will digital shadows influence manufacturing firms from the perspective of employees, managers, firms, and society?

Scenario development is done using technological forecasting methods, which have a long tradition in strategic technology management. One can distinguish three categories of forecasting methods. First, exploratory methods project current technological progress into the future, for example, trend exploration or bibliometric analyses (Cho & Daim, 2013). Second, normative methods, such as multi-criteria decision models or morphological analyses, illustrate the path to a desired future (Roberts, 1969). Third, combined methods integrate both approaches, such as the Delphi method (TFAMW Group, 2004). As our aim is to assess the probability and impact of early-stage technology on a diverse set of stakeholder groups, we followed common practice and drew on the Delphi method, an expert-based assessment (Landeta, 2006).

The Delphi method is a structured scientific approach to organizing and structuring an expert discussion in order to gain insights (Beiderbeck et al., 2021). Its purpose is to derive a reliable consensus about future developments by structuring complex opinions from various stakeholders (Kameoka et al., 2004; Linstone & Turoff, 2002; Rauch, 1979). It is considered a “judgmental forecasting procedure” (von der Gracht & Darkow, 2010), is constructed in an interactive multi-stage format, and is conducted anonymously and in written form. Here, experts assess statements about the future, so-called projections. Given the complexity of the problems, it is crucial to incorporate diverse perspectives in terms of both the set of projections and the selection of experts (Linstone, 1981). In addition, Saritas and Oner (2004) suggest including comments by the experts explaining the reasoning for their quantitative estimates.

Our scenario development is built on a multi-round, real-time Delphi survey with 35 experts from industry and academia who evaluate a set of 24 projections. In the following, we outline our process for conducting the Delphi survey. We applied a platform framework, adapted from Gawer (2014), distinguishing four dimensions: governance (e.g., open forms of collaboration), organization (e.g., boundaries and decision-making), capabilities (e.g., hybrid intelligence), and interfaces (e.g., open APIs and human-machine interfaces). In addition, we added a section on resilience drivers to framework for the development of the projections and the scenarios. Conventional Delphi surveys face criticism regarding a failure to translate their findings into actionable results due to being a time-consuming process (Gnatzy et al., 2011) with high drop-out rates (Keller & von der Gracht, 2014). Thus, we used a novel real-time Delphi approach, as described by Gordon and Pease (2006) and improved by Gnatzy et al. (2011). In this approach, experts evaluate the projections through an interactive online interface that provides instant feedback in the form of the other experts’ assessments and allows the participants to engage in discussion and potentially to adjust their estimations. As well as ensuring anonymity, the internet-based approach is more efficient and accessible, thus reducing drop-out rates and increasing the accuracy of the results (Gnatzy et al., 2011). A sample real-time Delphi survey can be found in Jiang et al. (2017).

2 Real-Time Delphi Process

Strict adherence to a rigorous process is key to ensuring the reliability and validity of a Delphi survey (Hasson & Keeney, 2011). We followed a four-step process to deliver this, as suggested by von der Gracht and Darkow (2010): first, we developed our Delphi projections; second, we selected a panel of experts; third, we conducted the Delphi survey; and finally, we developed future scenarios (see Fig. 1). We will provide a detailed overview and discuss the results of each step in the subsequent sections.

Fig. 1
A diagram, starting from top to bottom has inputs: Develop projections, Select panel experts, Conduct survey and Develop further scenarios. Each input has its respective information on the right side.

Real-time Delphi process

Full size image

2.1 Step 1: Develop Delphi Projections

First, we developed a set of projections for Next Generation Manufacturing. In line with previous Delphi studies (e.g., Jiang et al., 2017; von der Gracht & Darkow, 2010), we chose 2030 as the projection horizon for our scenarios, giving a 10-year timeframe. To address the required diversity of perspectives, we used an adaptation of Gawer’s (2014) framework to structure the formulation of our projections, as outlined previously. We conducted workshops with 27 experts from the fields of computer science, engineering, management, and social sciences to develop projections. Workshop participants did not participate in the survey. In addition, we used literature research to triangulate the workshop results (Gausemeier et al., 1998). As a result, we identified an initial set of 76 projections. We clustered similar projections to rule out redundancy and ensure an equal level of detail. Hence, we reduced the number of projections to 45. To ensure that we gained valid results without causing research fatigue and to guarantee that we covered all relevant topics within our framework dimensions, we went back to our workshop participants to discuss the reduced set of projections. After this second evaluation, we were able to dramatically reduce the number of projections again, to 24. In addition to the number of projections, we paid special attention to their quality and comprehensibility. Short, unequivocal, and precise wording is key to avoid any ambiguity which would impact the quality of the outcome (Mićić, 2007). Figure 2 illustrates the process of developing the projections.

Fig. 2
A diagram has 3 projection counts, from left to right, which are 76, 45, and 24 projections, which leads to the final set for the Delphi survey. Each projection has a detail or action plan bulleted below.

Projection development funnel

Full size image

We conducted a pre-test with 13 experts from industry and research to ensure content reliability as well as face validity. The pre-tested set of 24 projections then underwent a final editing round before being presented to the panel experts using the internet-based real-time Delphi tool developed by Gnatzy et al. (2011).

2.2 Step 2: Select Panel Experts

In a second step, we composed a panel of experts by identifying, selecting, and recruiting relevant actors in the field of manufacturing, in particular digital manufacturing (Gordon & Pease, 2006). Panel sizes in Delphi studies vary and depend on the scope of the study, required heterogeneity, and availability (Loo, 2002). In previous Delphi studies, the panel size has ranged from 10 to 60 participants (e.g., Beiderbeck et al., 2021; Gordon & Helmer, 1964; Jiang et al., 2017). Our target panel size was in the middle of this range, as digital shadows involve a heterogeneous actor structure but are still an emerging field with a limited number of available experts.

We identified potential experts by tapping into the network of all the workshop participants, as well as by searching in professional social networks such as LinkedIn. We reached out to this initial set of experts and asked them to refer experts who are more knowledgeable than they themselves, following an approach known as a pyramiding search (von Hippel et al., 2009). Selection criteria included technical expertise, publications in the relevant field, and profession. We evaluated the experts according to their corporate function, company’s stake in the technology, or previous publications and adapted the initial set to include a diverse group of experts. Our final panel contained 35 experts, including different stakeholders from industry (23) and academia (12) across a global range of nationalities (8). Table 1 provides an overview of our final expert panel. To our knowledge, our panel is one of the most comprehensive used in a study on digital shadows.

Table 1 Expert panel
Full size table

2.3 Step 3: Conduct Survey

For the Delphi survey, we used the real-time survey software developed by Gnatzy et al. (2011). The participating experts were presented with one projection at a time to reduce information overload. First, we asked the experts to provide their estimates on the probability of occurrence and the impact on firms of the projection in the year 2030. Probability of occurrence was measured in percent (0–100%), while firm impact was measured on a 5-point Likert scale (1 = very low to 5 = very high). In addition, we invited the experts to provide qualitative comments explaining the reasoning for their estimates. We were able to collect a large amount of qualitative data, with 629 comments. This indicates the commitment of the participants as well as their relevant expertise.

After the experts had provided their initial estimates and reasoning for a given projection, the next page presented the intermediate results (mean, standard deviation, interquartile range), as well as the anonymized arguments of the other experts for this projection. In line with the aim of the Delphi method to reach a consensus, the experts were prompted to reevaluate their estimated probability of occurrence and firm impact. In addition, they could engage in a discussion anonymously by responding to other experts’ comments. Thereby, we strengthened the data’s validity (von der Gracht & Darkow, 2010).

2.4 Step 4: Develop Future Scenarios

In a final step, we used the Delphi results to derive future scenarios regarding the probability of occurrence and impact on firms of digital shadows within Next Generation Manufacturing in 2030. For this, we first analyzed the quantitative results of the Delphi survey by calculating the mean, standard deviation, interquartile range, and outliers for each projection. To identify whether a consensus was reached among our expert panel for a projection, we used the interquartile range, which measures the difference between the upper and lower quartiles (Sekaran & Bougie, 2013). In line with previous Delphi studies (e.g., Jiang et al., 2017; von der Gracht & Darkow, 2010), we considered each projection with an interquartile range equal to or less than 2.0, indicating a low dispersion from the median, as having a consensus. Consensus was measured for the probability of occurrence as we developed our future scenarios using the most probable scenarios, even if there was a higher dispersion among the firm impact values.

After establishing the quantitative baseline for each projection, we described the results for each projection separately. In addition to the quantitative estimates, we analyzed the qualitative comments. They provided a richer understanding of and reasoning with which to interpret the quantitative estimates. For this, we took the results back to the initial workshop group that had developed the projections in the first place. Workshop participants clustered around projections they were particularly knowledgeable about. In this way, we were not only able to reflect a diversity of perspectives in the survey, but we were also able to incorporate heterogeneous perspectives in our interpretation of the results by including interdisciplinary research teams in the interpretation process. The quantitative results provided the basis for our analysis. The qualitative comments were coded and aggregated to broader themes and served as complementary data. We then followed a clear structure, describing the results for each projection, providing use cases from industry and academia, and outlining implications for policy makers, firms, and individuals (managers, employees).

Finally, we developed future scenarios by clustering selected projections, as suggested by von der Gracht and Darkow (2010). We followed the previously introduced framework adopted from Gawer (2014) and developed the most probable scenarios for each dimension according to the aggregated statistics from the survey (for the aggregated statistics, see in chapter “Big Picture of Next Generation Manufacturing”; and for a synthesis of the results, see chapter “Governance Structures in Next Generation Manufacturing”).

3 Summary

Managing the uncertainty resulting from technological developments is paramount to prepare for potential scenarios. By using an innovative real-time Delphi approach in the context of Next Generation Manufacturing, we extend this established tool of strategic technology management from predicting technological developments and their impact on firms and society (Courtney et al., 1997) to a providing a strategic guide for decision-makers in times of high uncertainty. Our study thus serves as a template for further applications of forecasting studies in interdisciplinary settings with high degrees of technical uncertainty.