Keywords

1 Introduction

Knowledge contribution (KC) is a measure for assessing the value of individual research projects, as well as the overall direction taken by an academic discipline [1,2,3]. Knowledge contribution is often insufficiently articulated by authors resulting in lower potential acceptance of a research manuscript, and underappreciation of the value of a manuscript once published. Expressing the knowledge contribution of a new design is inherently challenging [4,5,6]. In fact, a meta-analysis of Design Science Research (DSR) research efforts reports that less than half of DSR articles effectively report knowledge contribution, showing only a modest increase over time [7]. Gregor and Hevner [8] explicitly address the challenges of presenting DSR for maximum impact, by analyzing 13 articles published in MISQ ([8] pg 348, Table 2), placing particular emphasis on expressing the contributions of artifact abstraction, and positioning such contributions within a formal and structured DSR Knowledge Contribution framework (KCF, Fig. 1). They focus their efforts on the “role of the IT artifact as a basis for appreciating levels of artifact abstractions that may be DSR contributions” ([8] pg 339). It is the focus on singular IT artifact knowledge contribution that we address in this paper. Gregor and Hevner footnote that they use the term ‘IT artifact’ as synonymous with ‘IS artifact’, and there is nothing in their presentation that precludes the use of their framework to map multiple artifacts. DSR projects are recognized as complex and can offer multiple varied contributions to design knowledge [9]. Yet the original analysis of 13 articles addresses a singular one-dimensional artifact and contribution from each paper. While some have questioned the efficacy and completeness of the framework [10], we believe that taking an expanded view of ‘IT artifact’, and graphically presenting knowledge contribution, will lead to a more effective use of KCF uncovering more and varied contributions, to the benefit of the field.

Academic discourse in the IS field has long considered the ‘information technology (IT) artifact’ as the atomic-level item of interest in DSR. This approach has been challenged by Lee, Thomas, and Baskerville [11] who advocate unpacking the IT artifact into a separate ‘information artifact’, ‘technology artifact’, and ‘social artifact’ which, when taken together, comprise an ‘information system (IS) artifact’, a position hotly debated [12]. We contribute to this debate by taking the ideas espoused by Lee et al. and using them to expand the DSR Knowledge Contribution framework of Gregor and Hevner [8]. In doing so we achieve two significant outcomes: a compelling empirically-grounded argument in favor of the IS-artifact approach; and the introduction of Knowledge Contribution Diagrams (KCD) which adds significant expressiveness to the presentation of DSR. After a brief review of the two primary building blocks for our work – the KCF and the IS-artifact approach, we introduce KCD through an example from Cascavilla et al. [13] where the technique is first demonstrated. This is followed by analysis of four DSR articles - two revisited from the set analyzed by Gregor and Hevner in their original work, and two from recent DESRIST proceedings. A series of examples drawn from the article set illustrates the additional expressive power of the approach.

Fig. 1.
figure 1

The DSR knowledge contribution framework (from [8])

Full size image

1.1 The Knowledge Contribution Framework

The KCF is meant to assess contribution relative to x:problem maturity (also referred to as application domain maturity) and y:solution maturity. Scaling regions of high and low for each axis gives the four quadrants of: Routine design (high, high) applying known solutions to known problems; Exaptation (low, high) extending known solutions to new problems; Improvement (high, low) developing new solutions to known problems; and Invention (low, low) inventing new solutions for new problems. Of the original thirteen articles analyzed, 10 (77%) were classified as improvement, 3 (23%) as exaptation, and none as either routine design or invention. At first this may seem surprising as one would expect DSR work to regularly report some form of invention and incorporate, to a lesser extent, routine design. However, when we look at the component breakdown, as will be shown later, we see a very different picture.

1.2 The IS-Artifact Approach

There has been considerable debate around the question of artifacts and their centrality to DSR. Much of DSR work describes information technology (IT) artifacts, which Orlikowski and Iacono [14, 15] define as “bundles of material and cultural properties packaged in some socially recognisable form such as hardware and/or software”. However, not all agree that the “bundled” IT artifact should be the focal point of information systems research in general and DSR in particular. Lyytinen and King [16] note that IT artifacts do not deliver value in their own right and must be viewed in the context of a system. Schwartz [17] advocates the decomposition of IT artifacts into several distinct yet interconnected artifacts. Lee Thomas, and Baskerville [11] suggest a multi-artifact view when approaching DSR, arguing that the IT artifact is just one element within a broader Information Systems artifact, which should be viewed as a construct incorporating information, social, and technology artifacts – and must be addressed as such in DSR. They define a framework comprised of three major elements (not to the exclusion of other potential elements, as they discuss) as follows: (1) A Social artifact is an artifact embodying relationships or interactions among multiple individuals; (2) An Information artifact is an instantiation of information produced by a human participant either directly (as their own creative output) or indirectly (through an individual’s invocation of a software program or other automated information production process); (3) A Technology artifact is a human-created tool used to solve a human-defined or perceived problem. All three interact within a broader systems framework achieving a result that is greater than the sum of its parts, comprising the IS artifact. But as Lee et al. ([11], pg 6) state “An examination of the larger system of which any IT artifact is necessarily a part quickly expands the focus from IT artifacts to include artifacts that are not IT and artifacts that are created by people who are not IT designers. We may conceptualize these different artifacts as enabling, interacting with and even transforming one another.” By following their analytical approach, the precise contribution of each element in DSR comes into sharper focus. For a discussion of the applicability and value of the IS-artifact framework the reader is referred to [11, 13]. We will refer to the approach defined by Lee, Thomas, and Baskerville as the IS-artifact framework. We give a motivating example illustrating the use of three artifact types in the IS-artifact framework, then extend this to diverse DSR knowledge contributions such as design principles [18], design theory [19], and a potpourri of others [7]. In this manner we add component level analysis to the existing systems level analysis.

1.3 Motivating Example

The first use of an enhanced graphical KCD to present an information system artifact appears in [13], which we repeat here by way of introduction. That work describes an IS-artifact that consists of five distinct modules as part of an information system artifact to Detect Unintentional Information Leakage (DUIL). In their DSR analysis of the DUIL artifact the authors examine knowledge contribution by applying the KCF which, accordingly, classifies their artifact in the improvement quadrant owing to low problem maturity and high solution maturity of the overall system. However, the authors go a step further by breaking down the overall IS-artifact into five component artifacts following the IS-artifact framework. The result is two information artifacts - articles of interest (AoI); comments of interest (CoI); two technology artifacts - network analysis (Egonet); and visualization (Viz); and one social artifact - users of interest (UoI). Each of these artifacts were then assessed for knowledge contribution according to KCF and assigned their own symbology as shown in Fig. 2. This allowed the authors to present and discuss the specific contributions of each module, showing some elements could be considered routine design and others exaptation, leading to a higher level artifact exhibiting improvement, bordering on invention. This six-fold contribution analysis of what would have been a single DSR artifact stands in sharp contradistinction from the currently accepted single contribution approach.

Fig. 2.
figure 2

KCD for “The insider on the outside: a novel system for the detection of information leakers in social networks” [13]

Full size image

2 Knowledge Contribution Diagrams

We define a Knowledge Contribution Diagram as a graphical representation of one or more design science artifacts on a knowledge contribution grid to present the contribution of each artifact in terms of problem maturity and solution maturity. A KCD template consists of a 2 × 2 grid on the axes of problem maturity and solution maturity alongside a palette presenting a library of information technology-related artifacts that can be drawn upon to complete the diagram. The choice of artifacts to include in the palette is wholly subjective and extensible. Following Lee et al. [11] our initial KCD template included only information system, technology, information, and social artifacts; however given the diversity of knowledge contribution found in DSR articles, we have extended this to include icons for policy, guideline, design theory, and design principle [7]. To simplify our textual discussion of the graphic-analytical framework we refer to the artifacts by acronym as per Table 1.

Table 1. Artifacts and icons
Full size table

3 Methodology and Data

Using a purposeful sampling approach [20, 21] we conducted qualitative text analysis of four DSR articles listed below, two from the original set discussed in Gregor and Hevner and two from DESRIST proceedings. According to Patton, “The logic and power of purposeful sampling lie in selecting information-rich cases for study in depth. Information-rich cases are those from which one can learn a great deal about issues of central importance to the purpose of the inquiry, thus the term purposeful sampling. Studying information-rich cases yields insights and in-depth understanding rather than empirical generalizations ([20, 21] p. 230).” We followed the content extraction process detailed by [7] to identify and extract information such as a research summary, artifact description, and any knowledge contribution declared by the authors. The use of key quotes from the target manuscripts to identify knowledge contributions follows the approach of [8].

The following four articles were selected:

  1. 1.

    Abbasi, A., Zhang, Z., Zimbra, D., Chen, H., and Nunamaker, J., Detecting Fake Websites: The Contribution of Statistical Learning Theory, MISQ, 2010 [22].

  2. 2.

    Adipat, B., Zhang, D., and Zhou, L., The Effects of Tree-View Based Presentation Adaptation on Mobile Web Browsing, MISQ, 2011 [23].

  3. 3.

    Hönigsberg, S., A Platform for Value Co-creation in SME Networks, DESRIST, 2020 [24].

  4. 4.

    Brahma, A., Chatterjee, S., & Li, Y., Designing a Machine Learning Model to Predict Cardiovascular Disease Without Any Blood Test, DESRIST 2019, [25].

4 Results

Content analysis of the four DSR articles yielded 15 identifiable sub-artifacts with independent knowledge contribution (MAX = 5; MIN = 3; AVG = 3.75). Seven technology artifacts, three information artifacts, two guidelines, one theory, one principle, and one social, were identified. Knowledge contribution was spread across all four quadrants: six exaptation, four improvement. three invention, two routine design. Detailed analysis of each article in a table documenting component decomposition and knowledge contributions, along with the resulting KCD, follows.

4.1 Case 1

In this case, the identification of two technology artifacts and one information artifact gives us the first indication that there are component-level aspects to this project that could be of interest to the reader and there is potential contribution beyond that found at the systems level (Table 2 and Fig. 3).

Table 2. Analysis of “Detecting Fake Websites: The Contribution of Statistical Learning Theory” [22]
Full size table
Fig. 3.
figure 3

KCD for “A fake website detection system” [22]

Full size image

4.2 Case 2

This case presents the opportunity to identify both theory and guideline artifacts, the former contributing an invention and the latter an exaptation. In addition, there are three technology artifacts, two of which provide exaptation and one an improvement (Table 3 and Fig. 4).

Table 3. Analysis of “The Effects of Tree-View Based Presentation Adaptation on Mobile Web Browsing” [23]
Full size table
Fig. 4.
figure 4

KCD for “Tree-View Based Presentation Adaptation on Mobile Web Browsing” [23]

Full size image

4.3 Case 3

In this case, the overall system is described as an exaptation KC, yet the breakdown shows exaptation contributed by the Service Configuration and Analytics sub-artifacts. Here we see the first example of identifying and reporting knowledge contribution of a principle artifact, which can be mapped to the invention quadrant (Table 4 and Fig. 5).

Table 4. Analysis of “A Platform for Value Co-creation in SME Networks” [24]
Full size table
Fig. 5.
figure 5

KCD for “A Platform for Value Co-creation in SME Networks” [24]

Full size image

4.4 Case 4

In this case we see a pattern emerge with no exaptation and no routine design - all KC is found concentrated in the improvement and invention quadrants. While our study is limited to identification of KC and does not attempt to use KCD to measure knowledge contribution value, we hypothesize that projects populating these upper quadrants may have higher overall value - an important focus of future research (Table 5 and Fig. 6).

Table 5. Analysis of “Designing a Machine Learning Model to Predict Cardiovascular Disease Without Any Blood Test” [25]
Full size table
Fig. 6.
figure 6

KCD for “Designing a Machine Learning Model to Predict Cardiovascular Disease Without Any Blood Test [25]

Full size image

5 Discussion

Knowledge contribution of DSR is not one-dimensional and need not be distilled to a single artifact when undertaking a complex project. By drawing attention to the KC of each component of their designs, authors can provide valuable insights relevant to a wider range of readers. In espousing this view we knowingly diverge from the view that IS artifacts are systems requiring a holistic view of their evaluation [26]. Further, they can compare the novelty of each component in comparison to other components, and the overall ‘pareto’-like novelty of the entire system. Summary comparison of the knowledge contributions following our approach (Table 6) shows a threefold increase in identified exaptation and invention, a fourfold increase in identified improvement, and 2 incidents of routine design where previously there was none. Knowledge contribution can also be used as a basis for comparison between different systems. By comparison, we do not refer to a quantitative ‘score’ but rather to the qualitative, multi-dimensional comparison that considers both novelties, the integration of routine-design components as part of a complex design, the level and deviation of exaptation and improvement, etc. For KCD to be used as a comparison method, an objective quantification of problem and solution maturity should be developed. We plan to develop a set of objective questions that together ‘place’ a component within each quadrant of the KCD. To do so, we will recruit multiple domain experts that will be asked a set of objective questions about a component in an artifact (e.g., ‘are you familiar with the method used?’), and then to rate the two-dimensional maturity level of the component. We will then learn the relationship between the objective questions and the experts’ mean perceived maturity. We believe this will enable a more precise placement of each artifact within its quadrant which will be an improvement over the current subjective positioning. Our work has shown representation of knowledge contribution from 8 different artifact types. Dwivedi et al. [7] identify 9 different forms of knowledge contribution in DSR research, ranging from guideline to generative mechanism. Each of these can potentially be incorporated in a KCD. Researchers use a plethora of artifact types, each with its expressive ability and contribution to the art. We see no reason that the valuable contributions of each artifact within a complex system should be lost in the interest of singular system-centric contributions. Graphical representations have been employed for other DSR uses such as project planning and characterization [9], and creating Design Knowledge Maps [27]. Our work builds on the spirit of Design Knowledge Maps to position DSR contributions in problem and solution spaces, by providing guidance on how to use KCD in presenting DSR results.

Table 6. Knowledge contribution at original system level and subsequent component level post KCD analysis
Full size table

5.1 Practical Implications

There are three main practical implications of this work. First, manuscripts presenting DSR can and should use the KCD template to help elucidate and amplify the knowledge contribution of their work. Second, our approach of identifying and presenting multiple forms of knowledge contribution of complex DSR projects can raise the perceived and actual impact of DSR. Finally, researchers who identify additional forms of artifact can extend the KCD approach to incorporate new research insights based on these artifacts.

5.2 Limitations and Future Research

Our analysis is based on post-publication interpretation of articles describing DSR which may not coincide with the original intent and/or interpretation of the authors of those articles. The articles analyzed were purposefully sampled to demonstrate the proposed technique and do not provide a random or representative sample. We note that while in most cases, a single Information System artifact will appear on the KCD, some DSR investigates systems of systems (e.g. [28]) in which case multiple instantiation of IS artifacts would be placed on the KCD. In such cases the component artifacts of each IS would need to be identified as connected to the correct IS, possible by the use of color or framing. Extending KCD for systems of systems should be the subject of future research. While this study has focused on evaluation and presentation of DSR research to enhance knowledge contribution, the component-level analysis of DSR knowledge contributions can be useful in the process of envisioning, designing, or executing a DSR project. Applying KCD at earlier stages of the DSR process may yield great benefits to the challenges faced by DSR and this potential should be explored.

6 Conclusions

Viewing Gregor and Hevner’s knowledge contribution framework in juxtaposition with Lee et al. information system artifact decomposition adds depth to knowledge contribution analysis. Visually intuitive Knowledge Contribution Diagrams can improve presentation of DSR knowledge contribution, elucidating multiple knowledge contributions of a single DSR initiative, leading to greater impact. KCD are extensible for new design artifact types, adding further expressiveness. We believe that our technique to express multi-layered DSR knowledge contribution in graphical form can give a much needed boost to DSR research reporting of all kinds, and should also be investigated for possible uses in earlier stages of DSR project planning and execution. We hope that the explanations presented and examples given can help guide researchers in effectively presenting their DSR knowledge contribution leading to fuller appreciation of potential impact. The KCD template used to produce the graphs in this manuscript is available at http://bit.ly/3cdt3fI.