Keywords

1 Introduction

Contemporary research has argued that an appropriate outcome from design science efforts is knowledge in the form of ‘design principles’ [1, 2]. Design principles capture knowledge about the creation of other instances of artifacts belonging to the same class [2] and are derived as the researcher moves from the specific instance to the more generic level. An implicit assumption in this argument is that design principles are the right form of abstraction that would allow the community to accumulate knowledge that may then be useful to both, the next cycle of research as well as for design practice.

Unlike other modes of research (e.g., positivist or interpretive), design science inherently builds on the assumption that designing and intervening in authentic settings can generate new knowledge [2] that would be of benefit to the next generation of practitioners. The assumption essentially makes an end-run around the argument for ‘making the research relevant’ by tackling problems that practitioners have encountered or may encounter. The intent of the design science researcher is, then, unabashedly one of generating knowledge that will aid the next generation of practitioners who may need to address similar problems [3].

As design science researchers, we expect that practitioners will then, use outcomes of our work. An implicit assumption we make here is that they will simply “follow the recipe” we give them. Unfortunately, much prior work tells us that such use (reuse) is not a natural adjunct to creative problem solving. Scholars describe phenomena such as “NIH” (not-invented-here) that have hampered use of prior knowledge in new situations [4]. The expectation that the design knowledge we generate and codify in the form of design principles “will simply be used in new situations” by the practitioners may, therefore, be naïve.

To understand whether and how practitioners will use design principles, a number of conditions need investigation. For example, there are several other forms of design knowledge, e.g., scenarios [5], patterns [6], or heuristics [7]. Each may be more or less useful depending upon the problem context [8]. The designer may bring little or substantial design experience or domain knowledge to the design situation, which will further influence how she will use prior design knowledge such as design principles. Insights from studies of designers and programmers tell us that the breakdowns [9] that occur in design efforts can trigger the search for such knowledge. Empirical studies of design [10, 11] also suggest that the messy nature of design processes can prevent the injection of external sources, including codified prior knowledge. And our understanding of design as a non-routine activity [12] warns us that design, particularly in complex settings, tends to be a situated activity [13] that must respect the knowledge brought to the endeavor by both, users and designers [14] who must then translate and integrate information from a number of sources, one of which happens to be the design principles. We thus ask:

How might (how do) designers use design principles (as a form of design knowledge) in new design contexts?

We believe that a number of designer behaviors are likely to manifest in such an examination. These include searching for sources of information [15], structuring the problem space [16], moving from the problem space to the solution space [16], evaluating the design alternatives, and so on. Each has been examined extensively in prior research. Our interest is not to examine these activities in themselves. Instead, our particular focus is on understanding how designers use design principles as they engage in the design process, which may include the above activities. In particular, we are interested in the processes of knowledge access, knowledge translation, and knowledge application that designers engage in—with a focus on design principles as the form of knowledge. Such knowledge is crucial as the design science research community attempts to develop a cumulative body of design knowledge.

We proceed as follows. In the next section, we describe how design knowledge is used in the practice of design, thereby focusing on relevant knowledge components and the understanding of design as a contextual activity. Next, we describe our research design. The analysis section ensues, and we conclude with a discussion of important implications for both research and practice and provide an outlook.

2 Using Design Knowledge in the Practice of Design

Experts and scholars argue and agree that design can never be decontextualized. Scholars in many research streams also agree, starting from Rittell and Weber [17], that design remains a wicked problem that starts with a user concern with no obvious answer, that is, it “is never a process that begins from scratch: to design is always to redesign. There is always something that exists first as a given, as an issue, as a problem” [18, p. 5]. The contextual nature of design poses a challenge for the creation and use of codified design knowledge (e.g., in the form of design principles), and thus for the development of a cumulative body of design knowledge created by design researchers. Codified design knowledge can be represented in various forms such as design patterns [e.g., 19, 20], technological rules [e.g., 21], analysis patterns [e.g., 22], and design principles [e.g., 2, 23]. In this paper, we focus on design principles, defined as “knowledge about creating other instances of artifacts that belong to the same class” [2, p. 39].

The definition suggests that design principles must be codified in some way and must be applicable to a class of problems. We may expect that information systems (IS) practitioners would use such design principles to produce reliable outcomes. On the other hand, the design of IT artifacts is thought to be ongoing and complex, shaped by their organizational context during development and use [2]. As a result, design principles must be understood in relation to the (often novel) contexts in which they are used. To understand how design knowledge is used in design practice and what makes design knowledge usable in practice, we thus need to get at the inherent tension between the formulation design principles that follows a nomothetic approach about how to design a class of things and their idiosyncratic use in highly contextual design practice. We can decompose this tension into two important aspects: (1) design is an activity that involves use of both codified and non-codified (un-codifiable) knowledge and (2) design is an activity that is repeatedly carried out across contexts and time to address similar, but still context-dependent and novel, problems.

2.1 Codified and Non-codified Knowledge in Design

Prior work highlights the epistemological differences in the spectrum of design knowledge, ranging from the more tangible or explicit knowledge to the more intangible or tacit [24–26]. Design Principles (in so far as they are considered a form of design knowledge) represent knowledge that is codified, explicit knowledge, readily accessible as prescriptive statements. Prior to the advances in design science research, such codified knowledge might have resulted from the externalization of tacit knowledge gained from design experience, derived from the combination of existent explicit design knowledge, or both. The design science research community is now engaged in explicating such principles from their efforts to generate IT artifacts in novel situations. Still, the caveats from prior work apply. Not everything can be explicated, “we can know more than we can tell” [27, p. 4], and “a wholly explicit knowledge is unthinkable” [p. 144]. This view suggests that there are limits on what can be expressed through design principles—and implicitly argues that not everything required for design practice can, or should, be explicated.

Using this position as a starting point, we may argue that when designers make use of design principles, they convert explicit knowledge (i.e., prescriptive statements that are shared in the form of design principles) into practice. This conversion is heavily influenced by designers’ tacit knowledge base—both, about the new context as well as the class of problems for which the design principles have been explicated [28]. One approach to better understand the relationship between tacit and explicit knowledge is to use labels such as “know-how,” “know-why,” and “know-what” [29]. Although we know that these categories are not mutually exclusive, there are differences. Garud [29], elaborates these as follows: “in the context of technological systems, knowledge represents an understanding of the principles that underlie their functioning, processes employed to create them, and the uses that these technological systems serve” [p. 83]. Simply put, know-how is usually acquired through learning by doing, know-why through learning by studying, and know-what through learning by using. We suspect that these three categories come together as designers make use of design principles. We anticipate that understanding the interactions of these components in the process of design will, in turn, help us better understand what should (and more importantly, should not) be captured in design principles.

2.2 Idiosyncrasy and Generalization in Design

Although design is a contextual activity [e.g., 18, 30], design principles are about a class of problems [1] and are thus intended for application across contexts and across time. These contexts are different even though they might share certain boundary conditions. Design principles, thus, carry multiple possible meanings as they are interpreted differently in different contexts according to the need and purpose. This characteristic is crucial in understanding how design principles are used in different contexts. In essence, utilizing design principles means that designers must write their own version of those principles—the knowledge contained in these design principles is not “cast in concrete” [31, p. 289].

Understanding the use of design principles thus requires us to investigate the activities that occur when designers “write their own version” as they use design principles in various design contexts. This phenomenon is related to the degree of novelty as well as the complexity of the design situation in knowledge storage, retrieval, and transformation [32, 33]. So, in low complexity situations (such as deterministic change of non-human task through a more efficient algorithm) knowledge might simply be transferred, whereas higher level complexity situations (such that impact human tasks) might require that designers translate or transform knowledge [33]. In more novel situations, the focus shifts from one of efficiently using knowledge to one of effectively creating knowledge [32, p. 1192].

3 Research Approach

To explore how designers make use of design principles in the early phase of the process of designing artifacts, we captured and analyzed their thought processes during as they attempted to use a certain set of design principles in a new context. This allowed us to gain insights into how they use design principles during the process of design. Contrary to the common self-report method, which is dependent upon vagaries of recall and interpretation [34], we collected data while the designers were engaged in active use of design principles, i.e., as they attempted to design a representation of a proposed IS artifact. As the participants thought aloud, their words were recorded.

3.1 Collecting and Analyzing Think-Aloud Protocols

Think-aloud protocol analysis places an emphasis on “externalizing covert thinking without altering it” [34, p. 180]. This means that reactive thinking (describing and explaining—and therefore transforming—thoughts) is not encouraged. The method is based on the assumption that thinking is a sequential process and can therefore be represented as a sequence of thoughts [34, 35]. The closest connection between thinking and verbal reports (and therefore, the least altered externalized thinking) can be established by asking participants to think aloud while completing tasks [34]. This method of eliciting verbalized thoughts of designers while designing has been widely used in studies related to problem solving and cognitive analysis of design activity in various design fields, from architecture [e.g., 36, 37] to usability studies (for a comprehensive review see Cross [30, 37]). Table 1 summarizes key elements of the work we carried out to collect the think-aloud protocols.

Table 1. Collecting think-aloud protocols
Full size table

The method we used aimed to elicit a verbal protocol [38] from each individual designer (i.e., excluding collaboration) as they made use of design principles in response to a problem scenario. Each session involved one participant and one researcher (in the case of our pilot, two), as described in Table 1. The data collection effort was divided into the four phases: preparation, training, design task, and debriefing. During the preparation phase each participant was formally asked for her/his consent to participate in this study. However, the researcher did not provide any information regarding the actual research objectives. The setting chosen for the experiment provided sufficient comfort and tranquility for the participant to feel at ease during the session, because it required them to think (talk) aloud. Participants were asked for their agreement to audio recording the session. Finally, participants were requested to give a brief description of their professional background, expertise, and experience in designing software or IS. In the training phase the participants were guided through a round of exercises for verbalizing their thoughts. When the researcher was satisfied with how the participant thought aloud, they moved to the design task. Here, the printed instruction including the problem formulation and the set of design principles were shared with the participants. The participants were also provided with a sketchbook and drawing/writing instruments. Before beginning the design task, participants were reminded to verbalize their thoughts, and encouraged to do so as a way of thinking instead of trying to describe or explain what they were doing, i.e., they were requested to focus on the design task itself. The researcher read the printed instructions and asked the participant to read both problem formulation and design principles aloud. A debriefing phase followed at the end of the design session. Participants were informed about the general aim of this study and were again asked for their continued consent, and further correspondence, should questions arise during analysis.

3.2 Design Principles and Scenario

The participants were provided with a scenario that was hypothetical, yet grounded in an authentic design science research project [39]. This ensured that the scenario was similar to what one might encounter in practice. They were also given a set of design principles. Table 2 provides an overview of the scenario and

Table 2. Problem scenario for the design session
Full size table

Table 3 shows two example design principles along with a short description.

Table 3. Design principles
Full size table

3.3 Participants

We collected data from three participants, all IT/IS designers with more than ten years of experience. The session with the first participant served as a pilot session and we thus excluded the resulting protocol from our analysis. Still, the pilot session assisted us in improving the clarity of instructions and the readability of both problem formulation and design principles. All sessions were conducted in German. The instructions, problem formulation, and design principles were made available in both German and English. The sessions were, however, conducted in German because we wished to allow the participants structure and describe their thoughts in the language they were most comfortable with, in spite of their proficiency in English.

The profile of the two participants (excluding the pilot) was as follows. The first participant (P) has a background in software design and engineering, but currently works as a business project manager in a large financial institution. P indicated the necessity to understand the logics of both business analysis and systems development in order to perform well in his function. More often than not, this participant must be able to translate the needs and implications from one side to another and vice versa. The second participant (Q) began his career in a software engineering company before eventually started up his own company in the same field. Being one of the leaders in the company, Q is also responsible for maintaining client relationships, and his expertise includes understanding client needs as well as explaining complex software systems in a simple yet intelligible manner.

3.4 Data Analysis

The recordings of participants’ spoken thoughts were transcribed verbatim. Due to the exploratory nature of this study, we aimed to achieve the closest textual representation of the recorded audio—including indication of pause, stutter, murmur, and other unintelligible or incomplete words. Through our data analysis, we intended to explore the observed behaviors of designers as they tried to use design principles in a new, given context. In order to remain open, we thus coded the data in the spirit of open coding [40] to identify salient categories related to the use of design principles. We omitted those segments that were unrelated to making use of the design principles, for instance, training segments and instruction segments.

4 Findings

From our analysis, a number of categories emerged that highlight how designers use design principles. Table 4 provides an overview of these categories along with a brief description.

Table 4. How designers use design principles
Full size table

Next, we describe the categories and illustrate them with quotes from the data. Due to constraints on space, we provide only illustrative quotes. Additional details such as frequency or sequencing of our observations are not shown.

4.1 Interpreting Scope and Content

Our data suggests that designers interpret design principles and create meaning under consideration of the application context. That is, the data suggests that different designers apply the same design principle differently, even when given the same scenario. Participant P, for instance, said with regard to one of the principles:

…so that we can update the data again before the discussion… so timely beforehand still… That’s what I think…that’s what I make out of “plan for and ensure accurate and current content.”

When interpreting the same principle, participant Q mentioned:

Contents that are accurate and actual… that’s… based on my experience always difficult, to maintain the data. It’s a big topic. Here it can be done through… partly through… user guidance, so that the users tell us all kinds of data they need and so that the data can always be updated.

In this statement it becomes also noticeable that in interpreting design principles, designers further draw on their personal background and experience. The interpretation of design principles is deliberative and conscious, and designers indeed write their own version of those principles. The need to interpret design principles also becomes evident in the language the designers used when thinking loudly, for instance, Participant P made repeated use of the phrase “it means…”:

…it means, the templates and later the decisions… and everything must be an integral part…

Collectively, these quotes suggest the following. As designers use the design principles, they exercise certain degrees of freedom. A generous interpretation of our data suggests that design principles do not prevent such degrees of freedom; instead, they channel the designers’ creative actions towards a narrower band of creativity. This is further elaborated by the categories to which we turn next.

4.2 Matching with Problem Space

Designers match design principles with the problem space. This appears to involve two conscious acts. First, they weigh the design principles under consideration of their domain knowledge [41]. In doing so, they map the abstract form of the design principle against their understanding of the problem domain. Second, they attempt to match an implicitly instantiated form of the design principle against the specific situation at hand. Because design principles are an abstraction, this second act seems to require more effort. Consider, for example, the following statement from Q, that points to both:

… data security can be a discussion, since it’s about medical data. […] the problem lies a bit in the enterprise that manages it and where the data is stored, in which country. (Q)

Here, Q points out that the system design needs to consider aspects of healthcare regulations in terms of legal requirements of the organization or country, thus clearly relating the discussion to his/her domain expertise. Participant P makes similar moves in noticing the time pressure while discussing and making decisions about each patient when matching that aspect of the problem space against another design principle:

It means someone must prepare the case-relevant knowledge…what I’d now implicitly assume is…that each of the cases comes from each of the experts, probably it’s still…prepared by each of their employees. Otherwise you can never ever discuss everything in five to ten minutes. (P)

However, we observed fewer traces in the data showing such engagement from the participants—in our case, expert designers—with the problem space than with the solution space. This observation is consistent with some of the findings of novice versus expert studies [e.g., 42] and relative focus of designers in both problem and design spaces [43].

4.3 Guesstimating Missing Information

The developers we studied perceived the design principles as lacking detail and providing certain degrees of freedom. This behavior thus logically follows from the interpretation of scope and content. As designers guesstimate this missing information, design principles appear to ‘come alive.’ In effect, the designers write their own version of those principles closely matched against the situation under consideration. In other words, guesstimating becomes an important process where designers act creatively within the boundaries provided by the design principles. For instance, participant P noted:

… but many details are still open, which I must then still pick up.

and

I need then more detailed information so that I can design the system completely and eventually I will have to go one step back…

It thus becomes noticeable that, in order to design an actual system, more information is required than is provided by the design principles. It is perhaps not surprising that designers have to make assumptions if the design principles are the only formal source of reference. Participant P said:

…what I’d now implicitly assume is…that each of the cases comes from each of the experts, probably it’s still…prepared by each of their employees. Otherwise you can never ever discuss everything in five to ten minutes. (P)

P also noted:

what’s not stated in any design principle here…but I take it for granted, that one can’t simply access the patient data at any time, but they are released…just in time before the discussion… those are private anyway…

The participant thus uses examples or assumptions from his professional experience (i.e., drawing on design and domain knowledge) and injects those into the description of the artifact/solution domain. Finally, it was expressed that the same design principle may have different meanings, from the participant’s point of view:

it means that…well this principle may also mean that we… it can have double meanings… on the one hand it means that one completely allows the access to… (P)

This conscious effort continues on the part of the designers. Not only do they interpret the design principles and deal with the lack of detail, but they also engage in translating the principles into concrete possibilities by considering the solution space.

4.4 Projecting into Solution Space

Both subjects showed a clear tendency to translate the design principles into what we might call “requirements,” that is, physical and functional needs that a particular design, product, or process must be able to perform. This included both: (a) specific functions and their technical implementation details, and (b) envisioned user roles. At this, the degrees of freedom turned out to be important, as the participants identified various potential implementations, for instance:

…so it means everyone has a laptop or something like that, look at the date on his own, and for that the whole group an expert would…ok it’s then… now I’d just say it like this…through…a… power point or similar presentation, where maybe keywords are presented and the facts revealed, and then each can still… (P)

This designer mentions the availability of a laptop computer or the use of Power Point as potential, appropriate means to allow for multiple presentation modalities (as stated in one of the principles). Not only did the participants speculate about potential implementations, but also about the quality of the system. For instance:

It means also, that it needs to be easy to use, or that… that the UX…the user experience and design should be relatively simple, because the one it, yeah, broad things… or it has many different users… (Q)

Participant Q highlights the need to design a system that is easy to use [44] because its users will be doctors, who are non-IT experts. This was an interesting observation as none of the design principles expressly states the need to develop an easy-to-use system. Similarly, Q talks about “usability,” while considering suggestions from another principle (in this case, “principle one”):

…principle one… so, that has much to do with usability, so a usability… problem… so that it becomes an integral part… it happens often that a user… or if the system should be user friendly…

4.5 Implanting into the Design Process

Both participants highlighted that design principles need to be embedded into a more comprehensive design process. The design process they were implicitly considering seemed to involve stages such as user observations, prototyping, and/or testing, for instance:

Afterwards we can actually start with a simple mock up or a click-prototype, that’s how we call it, which has the real logic… or that can specify and demonstrate the user experience, how the sequence may look like with the mock up simply… or a few cases, those cases can be invented, and to review the workflow with some doctors. (Q).

Similarly, the same participant elaborates with regard to one of the principles, how users might be included in the design process:

Here it can be done through… partly through… user guidance, so that the users tell us all kinds of data they need and so that the data can always be updated… it can be done through a user guidance in asking questions.

These articulations appeared to indicate that the designers placed the design principles in a web of action that they considered appropriate to deal with the context—regardless of whether such guidance was available within the set of design principles.

5 Discussion and Implications

The data we have presented and the observations tied to this data suggest a number of possibilities. First, with regard to interpreting scope and content, our study raises important questions about whether and how boundary conditions of design principles may be specified. In our example case, the boundary conditions were provided by the scenario, but the design principles themselves did not contain any explicit boundary conditions. Prior literature on abstract design knowledge has highlighted the importance of boundary conditions, or scope, when developing such knowledge [3, 45]. Consequently, there is need to investigate if, and how, boundary conditions and scope might be considered in the development and formulation of design principles. Such knowledge becomes important as designers evaluate the applicability of design principles in specific contexts.

Second, matching the design principles with the problem space surfaces as an important, yet intricate, step in applying design principles. By their very nature, design principles are abstract. Matching them with the problem space requires the designer to act creatively by applying both her expertise and domain knowledge [41] in particular. In our case, the participants were not experts in the field of healthcare, and did not have the opportunity to rely on such domain expertise. The design principles appeared to fill some of this lack of expertise by providing prescriptive guidelines, which the designers then seemed to weigh against their own knowledge-base.

Third, with regards to guesstimating, our study indicates that the formulation of design principles might lead to ambiguity and various degrees of freedom. Still, there are good reasons to argue that such freedom is required to allow for creativity and innovation in the design process. Design principles are not a simple set of instructions to be followed blindly. In fact, they do not contain procedural knowledge at such level of detail. To understand and enact design principles, designers must possess professional knowledge such as design skills and domain knowledge. It is our contention that it will be important to further investigate the appropriate levels of detail and abstraction that design principles must contain. These decisions would be important as the design science research community engages in the larger goal of (a) developing a cumulative, reusable body of knowledge without (b) compromising creativity and innovation.

Fourth, our analysis highlights that a key challenge is seen in translating design principles into more concrete requirements of form and function, as the participants in our sample struggled to understand ‘what exactly’ some of the design principles meant, and how they might be converted into something tangible like a concrete instantiation [46]. Coupled with the previous observation, this struggle points to the question of ‘what exactly is enough’ specification within a design principle.

Fifth, it became noticeable from the data we collected that designers naturally place the design principles into a design process. One promising approach to further study the use of design principles in practice might thus be through the lens of various software development approaches such as waterfall, spiral, or agile methods [47]. Further, it might be useful to differentiate between different types of design principles, for instance, design principles of form and function and design process principles [45], and expressly consider the stage at which the different design principles are intended to be useful. Such insights will further improve our understanding of how design principles should be specified.

Finally, we hope that our initial effort to understand how designers make use of design principles will be followed by comparative studies that include other forms or representations of design knowledge. Such efforts will contribute to the cumulative body of knowledge about the design of design principles, design knowledge reuse, and might have important implications for the overall design process.

6 Next Steps

This paper continues the ongoing discourse in DSR about studying designers’ activity, i.e., “studying design” [48] or “ethnography to design” [49] as a mode of research. The study we have reported is, however, not yet another study of design processes. Instead, our effort is to consciously bring together the two streams of “doing design research” and “studying design” as argued in [42]. By doing so, we hope to make two contributions. The first is about the design processes themselves, based on our investigation of the (re) use of a new form of design knowledge (design principles). However, as pointed out earlier, our effort is not to examine the design process in and of itself—but rather, focus on how it incorporates design principles. This is our interest because we wish to implicitly test the assertion that design principles are an appropriate knowledge form that design science researchers should develop. The second contribution is to the growing body of writing about design principles. By understanding how future designers use design principles in actual design situations, we hope that we can feed back to design science researchers who can then use the lessons learned to formulate more effective and useful design principles. We hope that the work reported will start this dialog.