Keywords

1 Introduction

The design of information systems (IS) has been an important topic within the IS research community for many years [1,2,3]. Studies in this domain have improved means for organizations to confront challenging issues, such as managing diffuse knowledge processes [2], aligning individual and organizational competencies [4], or exploiting potentials of secondary design in emergent IS [5]. Such design-oriented studies typically aim to generate prescriptive knowledge on how organizations design IS, while the development of explanatory or predictive knowledge might assume a secondary role [6, 7]. Therefore, design-oriented works are relatively well geared for contributions to solving important problems in organizational practice [8,9,10].

Research on how to design artifacts has indeed become such an important topic in IS that it is now widely acknowledged as the ‘Design Science (DS) paradigm’ [11]. It has grown significantly in volume throughout the last decades, comprising different streams of literature that we summarize by using the umbrella term ‘design-oriented research’ (DOR). The main aim in DOR is to understand how effective artifacts can be designed and how design-oriented knowledge can be utilized for theorizing [12]. To this end, DOR usually draws on design processes that unfold over different stages such as problem identification, development and evaluation of artifacts [13,14,15]. In this context, evaluation of the utility of artifacts is typically central [16,17,18]. Yet, this has led to the criticism that DOR is too narrowly concerned with designing and evaluating artifacts and circumventing questions about generalizable knowledge [19, 20]. Therefore, while there is a wellspring of work on how to design and evaluate the quality and utility of artifacts, little is known about how to develop and communicate knowledge contributions (KC). This is problematic since one important aspect of KC in DOR comprises gradual abstraction of knowledge about particular instantiations into more general ‘design principles’ and ‘design theories’ [12]. Yet, this abstraction demands careful attention to how researchers collected and used empirical data when interacting with organizations and how this affected formulation of ‘design principles’ or ‘design theories.’

In this paper, we constructively engage with the aforementioned challenges and discuss how usage of empirical data within a DOR project affects KC. In this regard, we propose that an increasing importance of theorizing from DOR [21, 22] demands critical engagement with procedures for data collection and analysis, which are carried out as part of DOR. As DOR projects are typically seen as processes where researchers enact multiple cycles and stages, we promote that reliance on procedures for the analysis of process-data, which are known from innovation research [23], increases potentials to clearly communicate how KC were formulated. In this spirit, we follow recent calls to extend use of these particular procedures within DOR [24]. Mandviwalla [24] recently stressed that techniques for analyzing process-data could be fruitfully used to build design theories. By extension, we propose that use of these methods is not limited to develop design theory but also to other types of KC. Our first research question is thus (RQ1): How do qualitative process methods help to develop knowledge contributions in DOR?

Through engagement with this question, we gradually discovered a second related issue that demands careful consideration, i.e., what ‘knowledge contributions’ really are. Gregor and Hevner [12] have provided first insights by introducing a differentiation of three levels of generalizability of KC. They also propose a rather generic typology of KC, which considers a relevant but not complete selection of KC. For instance, it does not cover some types of KC, such as the ones depicted from empirical data. Accordingly, we ask (RQ2): How do we present and communicate ‘knowledge contributions’ created in DOR?

By taking Gregor and Hevner [12] as a starting point, we elaborate on their contribution by further detailing what KC are and how they can be developed during DOR projects. Our study, thus, offers two main contributions. On the one hand, we offer a framework, called PDSA (Prescriptive, Descriptive, Situated, Abstract), that is conducive to communicate and capture the dynamic evolution of novel knowledge in DOR projects over time. Furthermore, we contribute by showing that techniques for analysis of qualitative process methods [23] have significant potentials to inform the development of cumulative KC, especially when DOR covers empirical cases.

We proceed by a brief review of the fundamentals in DOR, followed by a presentation of dominant procedural models to carry out DOR. This sets the basis to define the problem, which addresses the lack of prescriptions on how to develop and communicate KC from DOR. Subsequently, we introduce the PDSA framework to support the communication of KC and show how techniques for analysis of qualitative process data supports the development of KC in DOR. We indicate the potentials of this idea through an illustrative case. In closing, we discuss our contributions, limitations and further research opportunities.

2 Relevant Literature on Design-Oriented Research (DOR)

Design-oriented research within IS can broadly be seen as a problem solving paradigm that aims to extend human and organizational capabilities by developing artifacts [11]. Thus, development of IS artifacts is of central concern to DOR [3, 12, 20, 25], which has its intellectual roots in engineering and architecture [3, 24, 25]. Various perspectives from behavioral sciences have been used in DOR over time (see for example [2, 5, 26]). While this undoubtedly increased the prominence of DOR within the IS discipline [18, 20, 27], the broad label ‘design-oriented research’ has also become sub-divided into different branches of literature that, while mostly similar, differ in details [12, 25, 27]. First, ‘design research’ refers to constructing artifacts in order to solve a specific class of problems [22, 27]. Second, ‘design science’ in the narrower sense is concerned with general rigor standards for conducting research projects [20]. Thus, design science aims at “explicitly organized, rational and wholly systematic approach[es] to design” [28] (p. 53) of IS artifacts. Third, ‘design theory’ refers to theorizations of knowledge about how specific classes of artifacts should be designed [3, 7, 24]. This means that design theories put a strong emphasis on how design-oriented knowledge can be formalized and made subject to replication [3, 12, 24].

2.1 The Roles of Artifacts and Knowledge Contributions in DOR

The outputs of DOR encompass IS artifacts and KC. Recently, KC in the form of design theories have become increasingly important [12]. In contrast, earlier works highlighted that contributions of DOR largely comprise ‘design artifacts’ [29] like constructs, models, methods, and instantiations [7, 30, 31]. Even though many of these artifacts carry certain degrees of abstraction [25, 31], scholars in DOR expressed their concern that it is sometimes hard to identify abstract knowledge contributions which arise from a DOR project [12]. In this context, Gregor and Hevner [12] categorized DOR contributions by their level of abstractness. Instantiations can be seen as most concrete and particular contributions (‘Level 1’), ‘Level 2’ contributions comprise abstractions of a ‘mid-range’, such as design principles [32, 33]. They reach beyond a particular application context, but are themselves insufficient to be seen as ‘design theories’ [24, 26]. Finally, design theories would be the most general and abstract contribution of DOR (‘Level 3’). Table 1 reviews these contributions.

Table 1. Examples of contributions from DOR according to their level of abstractness
Full size table

The introduction of these levels ‘1–3’ coincides with a general concern to develop theory through DOR [24, 34,35,36]. The reason is that the levels are cumulative, i.e. level 1 contributions can be developed into level 2 contributions, which may be the basis for building level 3 contributions [24, 26, 37,38,39]. Accumulation is a key idea in this regard because moving from level 1 to level 3 will unlikely be possible within a single research project or one paper [24]. Instead, systematic design theory-building is likely to be a process that emerges across publications of different scholars interested in related phenomena [3]. Therefore, if DOR is to exploit these potentials for systematic KC development, it needs a toolkit to explicate how knowledge was developed as well as a clear way to communicate it so that succeeding studies can carry on in a systematic way. Next, we review procedure models in DOR to assess whether and how they incorporate such thinking.

2.2 Procedure Models in DOR

In this section, some of the commonly accepted procedure models for DOR are explored with the goal to highlight how they address KC. The model of Hevner et al. [11] aims to support the understanding, execution and evaluation of DOR. This particular framework was later revised as a model comprising three cycles [14]: (i) the “relevance cycle” draws on business’ needs and introduces the artifact into the application domain, (ii) the “design cycle” comprises artifact building and evaluation and, (iii) the “rigor cycle” receives applicable knowledge as input and adds contributions to the knowledge base as output. Hevner et al. [11] also emphasized guidelines for creation of useful artifacts. They highlight research contributions in the form of designed artifacts, foundations or methodologies as well as the importance to communicate results.

Peffers et al. [15] proposed a methodology within steps from problem identification to the development of a solution, demonstration, evaluation and communication. The authors illustrated the latter as a part of an iterative DOR process, however it is not further explained how ‘communication’ informs further design iterations. Nunamaker et al. [13] developed an iterative and prototypical process for system development in IS, which covers fives phases: construct of a conceptual framework, development of a system architecture, analysis and design of the system, building of a (prototypical) system, and observing and evaluating the system.

Mandviwalla [24] developed a set of processes to support the development of design theory. Goals, kernel theory and existing artifacts inform the prototyping cycle, which includes a concurrent iteration of design, evaluation, and appropriation/generation. Last, Sein et al. [26] proposed a method called ‘action design research’, which treats the artifact creation and evaluation as inseparable and interrelated activities, differently from traditional DOR, which separates artifact building and evaluation. Action design research addresses four inter-related stages: problem formulation, building, intervention and evaluation, reflection and learning and formalization of learning.

Even though KC play a role for these procedure models, they provide limited prescriptions on how to develop or communicate them. For example, the importance of knowledge contributions is echoed in the method of Sein et al. [26] within “formalization of the learning”; within Hevner et al. [11], in “Guideline 4: Knowledge Contributions”; and in studies that stressed (partial) design theories as outcomes of DOR [12, 24], including theory building or refinement [13].

3 Towards Development and Communication of KC

3.1 Knowledge Contribution in DOR

Claims on how to develop and communicate knowledge contributions demand a clarification of what knowledge represents in lieu of epistemological and ontological considerations. The epistemological position of this study sees DOR knowledge contributions as “knowing by making” [39] (p. 4). Thus, knowledge embraces creation of artifacts as well as understanding the more abstract idea that guided the design of the artifact, regardless if this is theory building or testing. Ontology addresses nature and components of theory [7]. Our ontological position coincides with DOR (see [7, 37, 40]), which separates theory from understanding of individuals through Popper’s [41] three worlds classification. He discerns an objective/material (1), from a subjective/mental (2), and an abstract world (3). The latter embraces human-made entities, e.g. language, theories, models, and constructs. According to this view, artifacts instantiations are part of world 1, abstract knowledge is part of world 3 and ideas and experiences of design science researchers belong to world 2 [12].

3.2 Types of Knowledge Contribution in DOR

In order to structure and understand better KC from DOR, we considered publications which addressed topics such as design knowledge, theory development or theorizing in DOR as well as knowledge contribution itself [7, 12, 21, 32, 40, 42,43,44]. Additionally, we investigated DOR procedure models, as presented in Sect. 2.2, and possible KC mentioned by them. In the end, a set of types of KC were identified, as exemplified in Table 2.

Table 2. Examples of knowledge contributions from DOR identified on the review
Full size table

The authors of this paper worked together in a set of discussion rounds to clarify the similarities and differences of the identified KC. Relying on Gregor and Hevner [12], our first insight was to classify KC according to descriptive and prescriptive knowledge. While this was a feasible alternative, we also realized that some of the KC differ according to its level of abstractness, i.e. some KC are context-related or data-driven while others are more abstracts or theory-driven, e.g. theories [45, 46]. Therefore, we propose four dimensions for a KC typology: descriptive, prescriptive, situational and abstract.

Descriptive vs. Prescriptive Knowledge.

Based on Aristotle’s terms “episteme” and “techne”, Gregor and Hevner [12] suggest to classify knowledge as descriptive and prescriptive. Descriptive knowledge describes natural, artificial and human phenomena as well as relationships among them. By classifying, observing, measuring and cataloging, these descriptions can be made accessible to the human mind [47]. Prescriptive knowledge addresses artifacts created to improve reality. Gregor and Hevner [12] have added design theories arguing that they are formed from prescriptive knowledge that can also include other types of knowledge. In this sense, while prescriptive research focuses on improvement through the “how” knowledge, descriptive research focus on understanding via the “what” knowledge [12, 29, 36]. Yet, descriptive knowledge might evolve into prescriptive knowledge, e.g., when explanatory statements are combined with goals into prescriptive statements [48] or when little is known about the phenomena and classification schema or taxonomies [7] prompt future research.

Situational vs. Abstract Knowledge.

Goldkuhl and Lind [45] proposed to classify DOR-related knowledge as abstract versus situational. Abstract knowledge refers to general knowledge enhancing understanding phenomena so that this knowledge can be used as foundation for DOR in a variety of contexts. Situational knowledge refers to specific knowledge generated in specific contexts and produced during empirical design practice. In this sense, a set of data about single facts are generated by not yet considered theory, although they might be foundations for future theories [7]. Against this background, situational outcomes are more empirical outcomes or exploratory results. Abstract knowledge embraces design theories but also other knowledge contributions developed throughout iterative cycles of (i) generation and validation of knowledge and (ii) between different types of knowledge sources, such as empirical data, design theory, other knowledge and theories [17, 45]. Abstract knowledge can be extracted from as well as empirically grounded in situational knowledge and adapted to be applied in situational contexts, which might lead to modification of the abstract knowledge. On the other hand, situational knowledge is grounded in abstract knowledge and can also evolve into generalized abstract knowledge [45].

Both attempts to classify knowledge, i.e. the descriptive vs. prescriptive as well as the situational vs. abstract dichotomies, share much in spirit yet differ in important ways. In terms of similarity, both ideas allow that a focal project uses and generates both types of knowledge per respective dichotomy and that cumulative research helps to develop one type of knowledge out of the other. In terms of differences, the dichotomies refer to different claims. Where prescriptive (P) vs. descriptive (D) addresses the knowledge base on DOR [7], situational (S) vs. abstract (A) is more concerned with the knowledge reach (design knowledge of a specific context or more generalizable) but less with whether that is prescriptive or descriptive. In this sense, synthesis of both views helps to systematically understand and classify knowledge generated in DOR, therefore supporting to answer our second research question.

After defining dimensions for a KC typology, we separately placed the set of identified KC into the typology to be sure that the four dimensions could comprise all types properly. In another round of discussion, we compared the individual classifications and discussed them until we found a common decision. Figure 1 presents the result of this discussion by illustrating examples of different knowledge contribution types classified according to the KC typology.

Fig. 1.
figure 1

Typology of KC from DOR

Full size image

3.3 Communication of Knowledge Contributions: The PDSA Framework

As KC in DOR emerge throughout a research process, it is important to consider time when classifying and presenting them. To this end, we introduce a framework drawing on the aforementioned typology and incorporate the time dimension, represented by the phases of a DOR project. Figure 2 shows the framework.

Fig. 2.
figure 2

PDSA framework

Full size image

The procedure models for DOR cover different number of phases and labelling. For Peffers et al. [15], phase 1 is “problem identification and motivation”, phase 2 “definition of the objectives for a solution”, phase 3 “design and development”, phase 4 “demonstration”, phase 5 “evaluation” and phase 6 “communication”. For Sein et al. [26], phase 1 is “problem formulation”, phase 2 “building, intervention and evaluation”, phase 3 “reflection and learning” and phase 4 “formalization of learning”. Therefore, the three phases showed in Fig. 2 have illustrative purposes.

Figure 2 provides a backdrop to propose means to communicate KC in DOR. As scholars generally see DOR as a process [13,14,15, 24, 26], KC likely emerge through phases like those in Fig. 2. Moreover, as they emerge through these phases, they likely fall into different quadrants over time. Providing transparency about these dynamics, we believe, is central to clearly communicate KC in DOR and support the reuse and accumulation of knowledge over time.

4 Development of Knowledge Contributions from Empirical Data: Potentials of Qualitative Process Methods

In this section, we draw on templates for analyzing process data in order to suggest a frame of reference for how DOR researchers could develop KC especially in DOR covering empirical cases. In doing so, we draw on qualitative process methods [23, 49], prominent in behavioral IS and management studies [50, 51] but comparatively under-utilized in DOR [52]. By doing that, we follow the suggestion of Mandviwalla [24], who proposed the use of these methods to develop design theory. Design theory not only represents the prominent KC generated in DOR, but may also represent the final outcome of knowledge accumulation steps throughout several projects and publications. Our intention is to show that such methods do not only help to develop design theory, but also other types of KC.

Process methods are methods to analyze data that has been collected over a series of events [23, 49, 53,54,55]. Phases of DOR procedure models (see above) include many potential events like ‘formalization’ and ‘evaluation’ [26]. Therefore, systematically collecting and analyzing process data can help to increase data quality and, hence, the overall rigor of the resulting KC. A general template for rigorous use of process methods, proposed by Langley [23], can be used to develop KC in DOR. The seven data analysis strategies are Grounded Theory, Alternative Templates, Narrative, Visual Mapping, Temporal Bracketing, Quantification and Synthesis.

Several strategies can be seen as “sources for concepts” [23] (p. 707) because they allow researchers to become grounded in the empirical phenomenon and to begin theorizing from it. Langley [23] proposed that two strategies would be helpful in this regard: (i) a grounded theory strategy as well as (ii) alternative templates strategy. (i) Grounded theory allows the systematic and transparent development of conceptual categories from the empirical data. The alternative templates strategy (ii) is more deductive in that it proposes to use different pre-existing theoretical premises to explain data and assess which premise performs best [23]. As such, these two strategies have a lot in common with the process of formalizing problems in DOR.

Even though they did not use this language, Giessmann and Legner [56] seemed close to the grounded theory approach since they engaged with the field to formalize the design problem as prescribed by Sein et al. [26]. In contrast, Peters et al. [57] surveyed existing literature, i.e. theoretical templates, to justify their solution. Accordingly, the source chosen to formalize or ground a problem in DOR depends on the individual study. This coincides with Gregor and Hevner’s [12] proposal that different types of problems imply KC that differ in scope. However, how to assess that scope is a relatively under-developed in DOR. Hence, reliance on a more standardized procedure could help to increase validity and transparency in qualitative DOR.

The organization of data can be seen as a crucial step in DOR. With organizing data, Langley [23] refers to means of “descriptively representing process data in a systematic organized form.” We believe that such systematic engagement with the data is important when researchers build artifacts, intervene in the field and evaluate outcomes of this intervention. At this stage, numerous encounters with the field happen, and empirical data gathered in this encounters affect the formulation of KC [17, 20, 36]. Thus, a transparent and organized way to report on the development of the processes of building, intervening and evaluating could help external audiences to trace how immersion with the field affected KC. Two other strategies could help here: (iii) a narrative strategy as well as a (iv) visual mapping strategy [23].

The narrative strategy (iii) comprises writing a detailed narrative about the research process to provide numerous contextual details about how a DOR process unfolded, putting more focus on the situational knowledge. This level of detail can help to disentangle which encounters affected the formulation of KC in a highly granular manner. The visual mapping strategy (iv) is more reductionist. While narratives capture many details in words, visual maps are abstract representations of the building, intervention and evaluation processes that took place. Such maps should include clear denominations, i.e. “arrows and boxes”, effects of one element on another (positive/negative) as well as brief descriptions of the involved elements. Both strategies could also be used in high and low n studies. For n > 1, it seems possible to compare narratives and visual maps across cases in order to search for regularities. For n = 1 narratives and maps could be compared across design cycles in the form of within-case analysis. Practically, narratives and maps could be made available as research supplements, which would increase transparency over the process of developing KC.

Because increasing attention has been paid to formalizing KC and making them replicable and testable [3, 24], it is important to understand how qualitative methods can serve this purpose. In this context, three of Langley’s [23] strategies can be helpful: (v) temporal bracketing, (vi) quantification as well as (vii) synthesis. Temporal bracketing (v) means to structure a DOR process into distinct phases, which arise due to a “certain continuity in the activities within each period and there are certain discontinuities at its frontiers” [23] (p. 703). Temporal bracketing is in a sense evident in DOR procedure models since labels such “building, intervention, evaluation” [23] (p. 559) are used to structure the process. This provides a significant opportunity for replicating DOR studies because if researchers document how each phase affected the development of the KC (for example by visual maps), other researchers could replicate studies or modify them by bringing them to other contexts or by holding certain factors constant while variating others. This may not be a ‘hard control’ in the statistical sense but nonetheless an insightful inquiry into the maturity of KC that allows to assess whether these are ‘design theories’ or ‘design principles’ [12].

The quantification strategy (vi) fosters quantitative analysis of the data and, hence, formalization of design principles as hypotheses that enable testing and replication. Specifically, this strategy involves systematically coding the data, for example visual maps or narratives, according to sets of pre-defined codes, which could be results of the formalization phase. For process theorists [23, 49, 53,54,55], one important coding in this regard is to capture whether intended changes in each phase of the process really occurred. This ties in nicely with high-level KC, i.e. design theory, because theoretical predispositions about why a design should work can be coded as well as whether the design really had such outcomes. This involves tests of the design propositions and thus yields more robust propositions as outcomes. Therefore, this is a means to formalize qualitative DOR and make it conducive to replication.

The last strategy, synthesis (vii), is related to quantification, perhaps most reductionist and least suited for low n case. Synthesis “attempts to construct global measures from the detailed event data” [23] (p. 704). The main goal of this approach is to identify larger regularities across processes that allows the formulation of a more predictive theory in the statistical sense. Thus, this approach aims to synthesize qualitative process data into a more abstract statement on how certain independent variables affect dependent variables. This could be done if sufficient information is available (like narratives or visual from multiple cases). For this reason, this approach is the most suitable in terms of making a qualitative design theory generalizable.

Through the description of these strategies, we attempted to unpack that qualitative process methods [23] have potentials to inform how DOR develops KC. Next, in order to illustrate how the PDSA framework can communicate DOR knowledge as well as how qualitative processes methods can be used to develop KC from empirical data, we present an illustrative case.

5 Development and Communication of KC: Illustrative Case

To illustrate our contributions, a published DOR by Ebel et al. [58] was chosen after a literature search on development of design theories from empirical cases. Search of relevant literature was done using the template by von Brocke et al. [59] in the journals listed in the AIS’ “Basket of Eight”. Because of space constrains, we only mention the 11 papers selected: [5, 21, 34, 35, 56,57,58, 60,61,62,63]. Ebel et al. [58] is an interesting example of empirical data use in different phases of a DOR project as well as for the communication of KC in the “formalization of learning” phase.

In order to illustrate the use of the framework in this case, each of the DOR phases are briefly explained and a number is given for the sequence of activities carried out within these phases. Subsequently, in order to understand the development of KC from empirical data, an analysis of the situated knowledge is done with the goals to present how deep this knowledge was addressed, how it was presented and similarities and opportunities of developing it according to Langley’s [23] strategies.

5.1 Communication of KC: Application of the PDSA Framework

Drawing on action design research, Ebel et al. [58] developed a solution for systematically designing business models based on theoretical and empirical knowledge about business models. Figure 3 shows the application of the PDSA framework to their study. Note that despite the fact that Sein et al. [26] suggest to carry out the phases development and evaluation together, we separate them for illustrative purposes, i.e. to better represent the time sequence of activities and the KC emerging from them.

Fig. 3.
figure 3

DOR from Ebel et al. [54] applied in the PDSA framework

Full size image

Problem identification was done by reviewing the existing product portfolio of a specific company (1) and investigating literature. Both led to the formulation of a set of processes relevant to develop and manage business models (2) as well as identification of gaps in the literature (3). Aiming to assess the processes they developed as well as to contribute to the literature, interviews with experts were performed using semi-structured questionnaires (4). Analysis of this data occurred in three phases: immersion, reduction and interpretation. During immersion, data were transcribed and analyzed. In the reduction phase, data was reduced to what was considered relevant to the research. To reach that, a coding scheme and codebook were created in order to enable the rearrangement of data into meaningful categories. The process of creating the codes are explained in detail, however the data itself was not presented. During interpretation, codes were then used to reassemble data in a coherently and concisely. From the analysis, the authors stated they could confirm the processes they developed in (2).

In order to build their artifact, kernel theories used to solve similar problems were investigated (5). Next steps concerned the creation of an alpha-version (6) and its evaluation within an organizational setting (7) drawing on 27 test users. With the aim of evaluating the usability of the artefact, the Questionnaire for User Interaction Satisfaction (QUIS) was used and the data analyzed through an independent-samples t-test (M > 5). Some insights of the evaluation are given: “… a major weakness of the artifacts is that the used terminology does not relate well to the work situation…”; “[t]he testers also criticized the system as being too dull (…) and too rigid to cope with their needs” (p. 17). When reporting the refinement of the functionalities, the authors described how they improved the tool (8) according to the weaknesses pointed out during the first evaluation. The second evaluation (9) aimed at assessing the tool efficacy. To this end, six project teams were formed with the goal of developing business models with the tool. Based on literature, six dimensions (novelty, originality, feasibility, acceptability, effectiveness and elaboration) were used to assess the results.

In “formalization of learning”, two aspects of the framework that extend the existing literature were pointed out: the shared material sections and the community Section (10). While the former includes guides on how people from outside the project can contribute to the design of the business model, the first provides training material to support the development of business models. Finally, by stating, “the artefact itself produces knowledge as constructs and instantiations that may or may not lead to the level of abstraction that constitutes a design theory” (p. 26), the authors concluded that they created three major contributions to the knowledge base for the specific problem domain. A conceptual classification (2), which is “…descriptive knowledge in the problem domain” (p. 26), additional descriptive knowledge that extends current literature (10) and the development of a framework to create business models, a nascent design theory (8), cover these contributions.

5.2 Development of KC from Empirical Data: Qualitative Process Methods

Despite the sophisticated use of data in DOR, it seems sometimes difficult to build on earlier studies because little is known about how data was used. In terms of this study in particular, situated knowledge is generated in problem identification (1), through expert interviews (4), as well as evaluation (7, 9) and it is here were use of Langley’s [23] strategies could amplify transparency. For example, details of the problem identification (1) can be made more widely known through either using grounded theory or alternative templates. Choice between the two likely varies with maturity of design theorizing in a particular domain, as more mature domains are likely to offer more firmly established templates to identify and analyze problems. In case of developing ‘nascent’ design theories, grounded theory is likely helpful as ‘nascence’ of design theories indicates that they were developed in partially unknown contexts that become more known through the nascent design theory.

Similarly, regarding evaluation with experts (4), while this particular study explains the coding process, it does provide little insights on data itself. While insights gathered from interviews are presented, they lack links to the raw data. It is interesting that provision of such links seems key for papers in behavioral IS, which implies that it should also be seen as such in the DOR context. To facilitate the formulation of these links, we propose the use of narratives or visual mapping. Narratives may accrue more to cases that began with grounded theory while visual mapping may facilitate to show how codes, which were developed from literature [58], can be related to each other on the basis of the analysis of empirical material. This could help to show how comments made in interviews converged towards KC. Concerning evaluation (7; 9), a statistical analysis is presented, which suggests that linking earlier, more qualitative insights to the ‘synthesis’ strategy [23] could be helpful to also use mixed-methods in DOR.

6 Discussion

Within this article, we highlighted the role of designing and communicating KC in DOR, a current gap in the literature [12, 17, 37]. Our first contribution is the PDSA framework, suitable to systematically communicate KC from the entire DOR process. PDSA goes beyond existing research by synthesizing somewhat isolated understandings of knowledge contributions in DOR (e.g. [12, 45]) and by providing a backdrop to map and understand how KC emerge over time. In addition, by considering the time dimension, the framework can be used complementary to different existing procedure models in DOR. The second contribution of this study is to offer suggestions on how to develop KC from empirical data. To this end, we leveraged the techniques for analyzing qualitative process data proposed by Langley [23] and pointed out that these can be used to report how empirical data was collected and analyzed. Through a somewhat more formalized approach to justify KC generated from empirical data, we hope to offer a toolkit that enables researchers to more easily explicate and document what they did, how they did it, and what the limitations of these approaches were. In turn, this will enable audiences to better assess the rigor of KC emerging from DOR.

While hopefully thought provoking, our work is not without limitations. First, our work has not been formally evaluated, a limitation that needs to be overcome in the future. In addition, only one application of the framework is presented, despite the fact that we applied it in several of the papers selected from the literature review. Therefore, we see multiple options for future research. By increasing the number of illustrative cases to which the framework is applied, we can find associations between types of KC in different phases of DOR. For instance, we can depict how descriptive and situational knowledge of one phase is linked to abstract and prescriptive knowledge in another. Therefore, the framework may be helpful in specifying the role, validity, and boundary conditions of these associations. Furthermore, it may enable us to explore patterns for KC developed, methods applied, and quality standards in their evaluation in accordance with different research aims, e.g. varying artifacts.