Design Science Research Problems … Where Do They Come From?

Abstract

Effective and impactful design science research requires appropriate research conduct, and an appropriate research problem. Scholars in the DSR community continue to clarify the foundations for appropriate research conduct. In contrast, few guidelines or guardrails have been proposed to identify and develop research problems. Without such guidance, DSR efforts run the risk of pursuing problems that are either un-important or not-well-formulated. In this paper, I draw on writings beyond the DSR community to develop considerations that DSR scholars can use to identify and develop research problems, acknowledging their evolving ontological status. The paper describes an approach, articulates these considerations, and develops arguments that can help the identification and development of research problems for DSR efforts.

1 Motivation

Since its recognition as a distinct research genre within the IS discipline (Hevner et al. 2004), several scholars in the design science research (DSR) community have explored a number of fundamental questions. These have included explicating and distinguishing it from behavioral science (Hevner et al. 2004), approaches to conducting DSR (Vaishnavi and Kuechler 2008; Pfeffers et al. 2007); methods for pursuing DSR with authentic collaboration (Sein et al. 2011); clarifications to the nature of design principles (Chandra et al. 2015; Purao et al. 2020), and the role of theory and theorizing (Gregor and Jones 2007; Iivari 2020; Lukyanenko and Parsons 2020). Through these contributions, the DSR community is starting to signal a consensus about what constitutes design science research (Baskerville 2008), different genres of DSR (Kaul et al. 2015), and possibilities for accumulation of knowledge (Rothe et al. 2020). These efforts are useful to DSR scholars because they provide the infrastructure necessary for research conduct, and well-reasoned arguments to defend and justify choices (similar to other research genres such as case studies (Lee 1989), critical research (Myers and Klein 2011), and qualitative inquiry (Sarker et al. 2013)).

Such norms for research conduct are necessary (Parsons 2015) but may not be sufficient for effective and impactful design science research. That goal requires another important ingredient: research-worthy problems (Weber 2003). This concern is particularly important for DSR because design science scholars strive to produce prescriptive knowledge about the design of a class of future IT artifacts. Other professional fields and academic disciplines with a focus on the future have struggled with similar concerns, e.g. computer science (Brooks 1987; Regli 2017) and engineering design (Mosyjowski et al. 2017) as well as science and technology studies (Laudan 1984). If these concerns are not addressed, the promise of relevance and impact from DSR remains at the risk of remaining merely an ideology (Chatterjee 2000). Within the IS field, senior scholars have reflected on such concerns, sometimes describing these as Type III errors (Rai 2017). Within the DSR community as well, a nascent body of work is emerging with reflections and empirical investigations about problems and problematizing (Nielsen 2020, Maedche et el. 2019; Thuan et al. 2019).

The concerns – pursuing the right problems, framing them appropriately, and developing these in a manner that allows research contributions as well as broader impacts – are also of personal interest to all researchers. Effective responses to these concerns can influence speed of research completion (Pries-Heje et al. 2014), determine the type and quality of job opportunities, guide one’s research trajectory, and shape one’s academic reputation (Jensen 2013). Few sources provide help in this regard beyond broad recommendations, which are often aimed at doctoral students (Horan 2009; Luse 2012). However, researchers face these challenges throughout the career; and the skills needed to make these choices remain scarce and valuable (Jensen 2013).

In spite of this perceived importance of addressing important problems, it remains difficult for DSR scholars to figure out how to get there. Within the DSR community, “the problem of the problem” has been acknowledged (Nielsen 2020), efforts to structure the problem space have been proposed (Maedche et al. 2019), and reflective accounts are starting to appear (Twomey et al. 2020). Together, these efforts are providing DSR scholars new perspectives to explore “the problem of the problem.” Our intent in this paper is to suggest a broader lens that includes not just problem framing, formulation and development (Jones and Venable 2020; Nielsen 2020’ Twomey et al. 2020) but also initial problem identification and awareness. In Sect. 2, we review the core concern and how it has been addressed elsewhere. Section 3 develops key considerations for problem identification and formulation across four phases with particular attention to DSR scholarship. In Sect. 4, we summarize the challenges and offer a hopeful message to pursue more effective and impactful design science research.

2 Background

The core concern – how to choose a research problem – is particularly difficult to explore because of three reasons. First, this phase of research is rarely reported¹. The stylized reporting in conference and journal publications often provides the motivation by citing industry or societal statistics but does not reveal how the research idea was generated. Second, researchers appear to attribute this phase to individual creativity and brilliance that remains hard to describe. Apprenticeship models, commonplace in doctoral programs, further emphasize this perspective. Third, the vision of the individual inventor persists in spite of the shift to partnering (see, e.g. (Sein et al. 2011)) and the recognition of collaborations (Schneiderman 2016). In spite of these obstacles, several scholars have provided thoughtful reflections about this phase. An example is Jensen (2013) who puts forward, concisely, three principles for a worthy research problem: novelty, significance², and tractability. The empirical investigation by Barr (1984) supports this articulation. Table 1 summarizes.

Table 1. Key dimensions to assess research-worthiness

Novelty.

Sometimes described as originality, the interpretation of this dimension can vary. Barr (1984) describes this as “the integration of ideas that lead to a new or novel extension along a research path” (Chemistry) or “using a new framework or perspective in addressing a piece of literature or literary area” (English) or “use of a new theoretical perspective or new combination of perspectives as a framework for observing some group or system or use of a [new] methodology in studying a set of data” (Political Science and Sociology). In the IS discipline, Rai (2017) points out that answers that are “derivative to current understanding,” or “taking what is well known and reiterating it in a different context” does not meet this burden³. Embedded in this idea is a consideration of “the size of the inventive step⁴” (Jensen 2013).

Significance.

Sometimes described as importance, this dimension points out that the outcomes one produces “must be of interest (and relevance) to the profession or for the welfare of society” (Jensen 2013), but it is often tough to answer the question ‘why should anyone care’ [ibid]. Barr (1984) describes significance in terms of “contribution to knowledge” beyond what is already known. This may include “indicating that previous research was of little or no value” (English) or “both field and practical significance” (Political Science and Sociology). Note that these descriptions allude to the outcomes of research instead of a focus on choosing a research problem.

Tractability.

Sometimes described as solvability, this dimension refers to the ability to devise solutions to the research problem. If the problem is too large (scope), mere scale may make it un-solvable with available techniques or data within reasonable resource constraints (see also Rai (2017)). If the problem is addressed in a particular manner the researcher may encounter an intractable solution space compared to another research direction, which may be more efficient (and improve the probability of finding a solution) (Regli 2017). These dimensions are useful to assess whether a research problem is worthy. However, the first two (novelty and significance) are often easier to assess only in retrospect (e.g. by reviewers and readers). The third (tractability) may be considered by the researcher as s/he embarks on the research effort. Additional considerations are, therefore, necessary for choosing worthy research problems. The investigation by Mosyjowski et al. (2017), prescriptive suggestions by Luse et al. (2012), and the empirical investigation by Barr (1984) in diverse disciplines (Chemistry, English, Political science, and Sociology); along with the investigation by Thuan et al. (2021) suggests these elements. Table 2 summarizes.

Table 2. Additional criteria that influence selection of research problems

We may argue that finding a worthy research problem for DSR is no different from other disciplines (see Tables 1 and 2). However, many DSR scholars (e.g. Hevner et al. (2004)) have argued that the DSR paradigm is different from behavioral science. This distinction makes it incumbent upon us to explore whether these criteria and elements are adequate for DSR scholars. Consider, for example, how behavioral science researchers identify research problems by systematically examining prior work (Webster and Watson 2002) to identify gaps (Sandberg and Alvesson 2011) that can point to opportunities for further research. Contrast this with DSR. As Brooks (1996) describes: “hitching our research to someone else’s driving problems, and solving those problems on the owners’ terms, leads us to richer … research.” These ideas have been developed within the DSR community by Sein et al. (2011) who emphasize the important of collaboration. This distinction between discipline-generated and practice-generated problems (Welke 1998; Rai 2017) is the first indication that problem formulation may require new considerations for DSR. DSR scholars (Nielsen and Persson 2016; Nielsen 2020; Maedche et al. 2019; Twoney et al. 2020, Jones and Venable 2020) acknowledge this distinction as they address “the problem of the problem.” I build on these to suggest a broader lens that includes problem identification and problem formulation.

3 How to Identify and Develop Research Problems for DSR

My proposal consists of an approach to identify and develop research problems for DSR. It is reminiscent of other research approaches (Saunders et al. 2007; Creswell 1994) with phases such as (a) topic selection, and (b) topic development, and builds on contemporary work about problem formulation such as: the reflective account from Twomey et al. (2020); the efforts from Maedche et al. (2019) to structure the problem space; Venable et al.’s (2017) checklist of business needs; and Nielsen’s (2020) account of problematization that acknowledges that a problem ‘is not just given as though it exists objectively’ [ibid]. The approach I suggest consists of interlocking phases that start with initial awareness of the problem (Kuechler and Vaishnavi 2008) but continue ongoing engagement with problem formulation (Sein et al. 2011; Rittel and Webber 1973; Nielsen 2020). Figure 1 summarizes the approach that I elaborate next.

Fig. 1.

An approach to develop research problems for DSR

3.1 Choosing a Design Perspective

The first stage in this journey is to choose a design perspective (as opposed to, say, an empirical perspective) when faced with a problem situation⁵. Brooks (1996) describes it as a ‘toolsmith’ perspective, where researchers take their inspiration by “partner[ing] with those who will use our tools” [ibid]. This ensures that we (a) aim at relevant, not toy-scale problems, (b) remain honest about success and failure, and (c) face the whole problem, not just the parts easily amenable to analysis [ibid]. Echoes of these ideas are found in ADR (Sein et al. 2011), which emphasizes partnering with external stakeholders to identify a specific (instead of abstract) problem (see, e.g. (Lee et al. 2008)).

However, the need for a design perspective is not always apparent nor necessary in all situations. Consider the case reported by Purao and Karunakaran (2020), where initial awareness of the problem situation was articulated as ‘the need to better support complex knowledge-work in organizations.’ Developing a concise statement of the problem situation, however, required them to invest considerable effort; and eventually lead them to identify different aspects such as (i) what artifacts do actors use to support knowledge work, (ii) what strategies do organizations use to develop the artifacts, and (iii) how can knowledge embedded in these artifacts be extracted and managed. The first two require empirical investigations; the third requires a design perspective.

A corollary to the above is to ensure that the problem specification respects both, the specific problem faced by the external partners as well as the more general version of the problem. Rai (2017) (citing (Weber 2003)) describes an emphasis on the former at the expense of the latter as myopic⁶, where researchers “formulate the problem with a sole focus on an immediate practical problem … but do not evaluate how the problem relates to a more generic, archetypal problem.” This distinction between a problem instance and a class of problems has been acknowledged in the DSR community (Kuechler and Vaishnavi 2008) as well as elsewhere acknowledging the importance of partnering for grounding the problem (Van de Ven 2007), while emphasizing the need to generate and respond to a more general version of the problem. We note that for the DSR scholar, it is often important to address the idiosyncratic details of the problem, a theme acknowledged by Majchrzak et al. (2016) who emphasize the need to acknowledge a theory of the problem as an important part of the contribution. These challenges (outlined above) persist and resurface through the other phases.

3.2 Identifying a Problem

Continuing the ideas outlined so far, the source of the research problem for DSR efforts is not likely to be an effort at gap-finding (Alvesson and Sandberg 2013). Instead, the impetus is likely to be a situation perceived in practice. This distinction, described as discipline-generated research vs. practice-generated research (Welke 1997; see also Rai 2017) is evident in a recent empirical investigation of the source of research questions for DSR. Thuan et al. (2021) find that problems from practice accounted for 51 out of 63 conference papers, 18 out of 21 papers from MIS Quarterly, and 11 out of 20 dissertations (in total, 77% of the 104 research manuscripts they examined). Based on the results, they find overwhelming evidence that compared to gap-finding (or problematization) (Alvesson and Sandberg 2013) the dominant mode for DSR scholarship remains problem-solving (Pries-Heje and Baskerville 2008), where the researchers attempt to address a practical and/or knowledge problem by creating IS artifacts.

More detailed investigations of sources of technological problems and how inventions are generated are the domain of scholars in the fields of science and technology studies (STS) and research policy. Here, I draw on Laudan (1984) to suggest a taxonomy (albeit incomplete) for types of technological problems that DSR scholars can draw upon. For example, a problem situation may be described in terms of an imminent problem that would lead to the potential failure of a technology (presumptive anomaly). As another example, a problem situation may be described as one that requires some technology layers or components to work effectively with other, currently incompatible layers or components (technological imbalances). Table 3 summarizes.

Table 3. Types of technological problems

The typology suggests one possibility for moving to a class (a technological problem type). Although it is difficult to provide such a taxonomy of problem types for the larger socio-technical problem; it is possible that the DSR scholars would describe the problem they are dealing with using phrases at different levels of abstraction (e.g. decision support, knowledge management, therapy compliance and others) to signal the readers the specific stream(s) of work that define the problem space, and avoid falling prey to a myopic problem formulation (Rai 2017) described earlier.

A final consideration during this stage is to ensure that the problem is not one that “can be readily structured and solved by applying extant knowledge from IS or from other disciplines” (Rai 2017). For DSR scholars, such problems are described as routine design that requires “the application of existing knowledge” as opposed to innovative design, where the solution will produce “a contribution to the … knowledge base of foundations and methodologies” (Hevner et al. 2004).

3.3 Framing and Formulating the Problem

The research problem identified would need to be narrowed down with a particular focus, as well as developed with support from ongoing work, similar to the idea of a ‘rigor cycle’ (Hevner 2007). The apparatus suggested by Maedche et al. (2019) can be useful here to structure the problem space with concepts such as needs, goals, requirements, stakeholder and artifact. This stage is also important because the researcher must further explore how to place the practice-generated problem in one or more prior research streams, and explore the ‘research front,’ reflecting current research thinking in relation to the chosen research focus (Horan 2009). Here, the idea of a ‘research front’ encompasses an examination of what has been tried before, addressing known obstacles to progress, and using prior work as a catalyst for refining the research idea (Barr 1984). By clearly articulating the research front, the researcher can ensure that the planned research will produce new knowledge beyond what is already known.

Another consideration during this phase is to ensure that the research problem matters. Researchers are often drawn to problems they consider interesting⁷. Beyond personal interest is the realm of important and significant, i.e., “problems where the answers will matter in important ways” (Rai 2017). By mapping the research interest to larger business or societal challenges, the researcher can ensure that the problem is relevant (Hevner 2007). Horan (2009) describes several suggestions, e.g. identifying the audience, developing rationale, and other that can be useful towards this end. Ideas such as ‘grand challenges’ identified by different disciplines, government agencies and research groups⁸ can add to this significance. The researcher can use ‘backward reasoning’ to explore how their chosen research focus can contribute to the grand challenges. Without this effort, the researchers may be swayed by the ‘streetlight effect’ (Rai 2017), driven by available datasets or tools. Finally, the researchers must decide problem scope with the so-called Goldilocks principle, deciding what to emphasize and what to place in the background (Rai 2017). Venable et al.’s (2017) checklist of business needs provides one possible approach that DSR scholars can use for this purpose.

A related consideration during this phase is to establish that the problem is solvable. Unlike empirical research, where researchers are encouraged to: “choose topics where the result is interesting no matter what answer you get” (Barrett 2013); design science scholarship requires that the researcher would repeatedly develop and evaluate the artifact until positive results are achieved (Thuan et al. 2021). A part of this effort may include characterizing the expected outcomes with the typology suggested by Gregor and Hevner (2013), where a move from improvement to exaptation to invention would indicate increasing levels of difficulty. Another possibility to explore solvability ex-ante would be to follow the Heilmeier questions⁹, which include queries such as: (a) what is new in the approach and why the researcher thinks it will be successful, (b) what are the risks and the payoffs, and (c) what are the midterm and final exams to check for success. In their apparent simplicity, they allow the to reflect on the solvability of the research problem (Shapiro 1994).

This phase can lead to the construction of initial versions of research questions. Their nature and format will be different from behavioral and natural sciences (which may focus more on causality and explanation). As Thuan et al. (2021) describe, the research questions may take forms such as: “how can we develop X?” or “how can we develop X to resolve Y?”, i.e., they will not posit conjectures for falsification.

3.4 Representing the Problem with Design Options

The “problem of the problem,” however, continues because design involves re-design, which in turn, requires refining the problem. Hevner et al. (2004) capture this as: “knowledge and understanding of a problem domain and its solution are achieved in the building and application of the designed artifact.” Purao (2013) uses the phrase “evolutionary ontology” to characterize the problem, describing it as the following: “as … the artifact begins to take shape … [it] … influences the researcher’s stance towards the problem. It ceases to be independent of the researchers’ efforts. Instead, it is interpreted in conjunction with the properties of the artifact.” These descriptions emphasize a key characteristic of all design problems (including problems considered by DSR scholars): they are “wicked problems” that defy “a definitive formulation” (Rittel and Webber 1973, p. 161). For example, approaches to design science research describe phases such as problem awareness (Kuechler and Vaishnavi 2008, Figure 3), problem identification and motivation (Peffers et al. 2007, Figure 1, p. 54), and problem formulation (Sein et al. 2011, Figure 1, p. 41) and even acknowledge the need to return to the problem formulation stage as the research unfolds. However, they do not suggest pathways or specific ways to do this. In the absence of these, DSR scholars are tempted to rely on a more linear process (see, e.g. Feine et al. (2020)).

Returning to the original conceptualization of wicked problems (Rittel and Webber 1973, 1984), and its contemporary descriptions (Sweeting 2018), I note that an attempt to solve a wicked problem creates new problems. In other words, instead of moving from problem to solution, the “process of formulating the problem and of conceiving a solution (or re-solution) are identical” (Rittel and Webber 1973, p. 161). It is, therefore, important to recognize that problem formulation cannot stop after initial awareness; instead, the DSR efforts lead to continuing problem re-definition. The account by Twomey et al. (2020) provides a contemporary narrative that reflects similar ideas. Simon (1988) suggests a more specific perspective by describing problem-solving as a “change in representation¹⁰” to “make evident what was previously true but obscure,” i.e., problem-solving involves representing the problem in such a way that it “makes the solution transparent” [ibid]. He acknowledges that even if this is considered an exaggerated view, different representations can provide not only a path towards a solution but also a greater understanding of the problem itself. In the absence of a taxonomy of representation for socio-technical problems, the argument I advance here merely emphasizes the role of representation in continued problem (re-)formulation.

Devising and evaluating design options is, therefore, an important contributor to developing the problem¹¹. As Regli (2017) points out, outmoded or inappropriate representations remain obstacles to progress. Partnering with domain experts (similar to the arguments in Sein et al. (2011)) can provide opportunities to DSR scholars to develop novel representations and digital abstractions that can transform the outcomes.

4 Discussion and Next Steps

The concerns I have explored in this paper are difficult, partly because many of us may believe we already know the answer, and partly because it remains difficult to peel back the layers of uncertainty that DSR scholars face as they engage in DSR efforts. Although it is recognized that DSR is inherently different from behavioral science (Hevner et al. 2004), scholars engaged in DSR have sometimes remained trapped in the expectations they have inherited, e.g. “clearly articulate your research questions before starting the research project.” Contemporary investigations (Nielsen 2020; Twomey et al. 2020) have started to explore these differences, drawing on specific DSR (and allied) methods as well as prior work related to information systems design (Lanzara and Matthiassen 1985; Schön 1983). Others, such as Maedche et al. (2019) and Jones and Venable (2020) have proposed new structuring devices for problem formulation.

My investigation here has drawn upon a different set of foundations, such as research methodologies in other disciplines (Regli 2017), work related to design foundations (Rittell and Weber 1984), and foundational work in science and technology studies (Laudan 1984). The approach I have outlined, therefore, addresses a set of concerns that overlaps with contemporary investigations in the DSR community but points to new opportunities for investigation. Table 4 summarizes these.

Table 4. Identifying and developing research problems for DSR: key challenges

I hope that the DSR scholars will find the echoes of their struggles within the challenges outlined, and that we can continue this dialog to identify (choose) and develop (formulate) research problems towards more impactful outcomes.