1 Research Questions

A doctoral program takes a long time. The Australian PhD program takes three years or more, with the average being more than four years. German PhD programs often take between three and five years, and sometimes longer. Doctoral programs in other countries can take even longer—up to seven years in some North American programs. While the length of the program varies, they all start similarly with two challenges of doctoral study:

  1. (1)

    You will have to read the available literature to learn about methods and theories. This part is nicely structured as there is a set of useful books and papers, and there are typically good classes, workshops, and tutorials to attend. While this challenge should not be underestimated, the process of addressing it is usually well structured and typically has considerable guidance available. This structured challenge comes with the expectation that you will master the relevant methods and theories and be able to apply them appropriately. This challenge is relatively domain-agnostic in that it is at least to some extent independent of your field of study or phenomena of interest. You simply build the capacity to use a research method or set of methods well and to understand a particular set of theories, so this first challenge pertains mostly to how you perform research.

  2. (2)

    You will have to formulate and develop your own research questions and propose a plan to address them. This challenge is much more difficult than the first challenge largely because it is not as structured but is undefined and highly contextual. Formulating and understanding research questions is a domain-specific challenge that pertains closely and directly to what you want to research.

The difference between these two challenges is important. One of the great scholars in information systems (IS) put it much better than I can (Weber, 2003, pp. iii–iv):

I believe that the choice of research problem—choosing the phenomena we wish to explain or predict—is the most important decision we make as a researcher. We can learn research method. Albeit with greater difficulty, we can also learn theory-building skills. With some tutoring and experience, we can also learn to carve out large numbers of problems that we might research. Unfortunately, teasing out deep, substantive research problems is another matter. It remains a dark art.

The point is that a good doctoral study starts and ends with the right research questions. Finding and specifying them—not even answering them—is not easy. I have grouped some of the problems that are related to the formulation of research questions into five categories:

  1. 1.

    The “monologuing” problem: students with this problem cannot tell what research question they are tackling unless they engage in a five-minute monologue. If it takes an extended monologue just to explain the question they seek to answer, they typically do not understand the research question well enough to articulate it concisely. A good research question is simply a short question. Needing to elaborate for five minutes on this question or to explain its components indicates that they have not grasped the essence of the problem or discovered how to scope it in a way that it can be distinguished easily from related phenomena or problems.

  2. 2.

    The “so what” problem: in these cases, students can state a research question but not why it would matter to anyone. The relevance of a problem is particularly important in applied fields of research like information systems, where we seek to solve a specific problem or provide innovative solutions to issues that affect an individual, group, or society, such as how new technology shapes the work practices of employees. You may have a so-what problem if you have trouble justifying its practical merit, that is, why it is worth asking the question.

  3. 3.

    The “solving-the-world” problem: students who have this problem can state a research question that has value, but it cannot be answered given the students’ resource constraints. For example, in most PhD programs, your resources are basically limited to you. Your timeline may also be constrained either because the program’s regulations stipulate completion after so many years or because funding runs out. Students often have two to three years to complete their research, and a topic that is too broad (“The cure for cancer lies in the Amazon jungle”) has no chance of completion given the available time and resources.

  4. 4.

    The “insolvability” problem: your research question simply cannot be answered meaningfully because of a logical problem in the question, because the information needed to answer the question cannot be logically or legally obtained, or because the answer is so hard to obtain that feasibility of the research in the constraints is not possible. A good example that I often give is that of longitudinal studies, in which research is meant to examine the evolution of phenomena into the future. Such research, by definition, must be carried out over many years. For instance, studying the impact of early childhood abuse on the development of social morale in adolescence would require examining individuals in a sample over many years, and most PhD programs must be completed in much less time.

  5. 5.

    The “multitude” problem: students who have this common problem ask (too) many questions instead of one. I always tell my students that a good study sets out to answer one question with one answer—maybe two if they are ambitious. Nothing is gained by setting out to answer six questions as most of them will fall into five inappropriate categories:

  • Obvious questions: “Are there challenges in using information technology?” Of course there are. Obvious questions have answers to which everyone would agree.

  • Irrelevant questions: “What is the influence of weather on the salaries of technology professionals?” There is no reason to believe that there is any influence whatsoever.

  • Absurd questions: “Is the earth flat after all?” Absurd questions have answers to which everyone would agree, although I read that almost two percent of people still believe the earth is flat (Branch & Foster, 2018).

  • Definitional questions: “Is technology conflict characterised by disagreement?” The answer is simply a matter of creating a concept that says it does. A definition is a form of description, not research.

  • Affirmation questions: “Can a decision-support tool be developed to facilitate decision-making for senior retail executives?” Yes.

Use these categories as a “black list” (an exclusion list) to ensure that your proposed research questions do not fit any of these problems, and if they do, go back and revise them. Alternatively, use the questions above as a checklist of what not to do when creating your research questions in the first place.

A research question should be a key statement that identifies the phenomenon to be studied. The research question(s) is/are the fundamental cornerstone around which your whole doctoral research revolves and evolves. For you, your supervisory team, and other stakeholders (external examiners and reviewers, for example), the research question provides the frame that brackets your whole investigation and its representation in the final thesis. We set out in our study to answer a particular question, and when we find an answer, we turn back to the question to determine whether we have sufficiently and comprehensively answered the question. A number of guiding questions can help you find a good research question:

  • Do you know in which field of research your research questions reside?

  • Do you have a firm understanding of the body of knowledge (the domain’s literature) in that field?

  • What are important open research questions or unsolved problems in the field that scientists agree on?

  • What areas need further exploration?

  • Could your study fill an important gap in knowledge? Could it help clarify a problem we are having with the knowledge already available?

  • How much research has already been conducted on this topic?

  • Has your proposed study been done before? If so, is there room for improvement or expansion?

  • Is the timing right for the question to be answered? Is it a sustainable and important topic, or is it currently a fad but risks becoming obsolete?

  • Who would care about obtaining an answer to the question? What is the potential impact of the research you are proposing? What is the benefit of answering your research question? Who will it help, and how will it help them?

  • Will your proposed study have a significant impact on the field?

  • Is your proposed research question based only on the fact that you think it is “interesting” but nothing else?

Using these criteria and guidelines can help you formulate a good research question, but three key components should also be considered: motivation, specification, and justification

1.1 Motivation of the Research Question

A good research question does not fall out of thin air. It is the logical, necessary, and inevitable conclusion to a set of arguments that find (1) an important problem domain with (2) an important phenomenon that deserves attention from the research community and that relates to (3) an important problem with the available knowledge about this type of phenomenon. A motivation is not necessarily extensive, as long as it addresses these three points. A simple example is the following chain of arguments. The argument that reveals the important problem domain is as follows:

Organisations invest heavily in new information technology to seek benefits from these investments.

In this statement, we learn about a problem domain: investments into IT and benefit realisation from IT. This is an important problem domain for businesses because it involves money. As a tip, motivating research problems by citing data that confirm the quantifiable value associated with the problem (dollars spent, money lost, for example) might be valuable. Within this domain, we then learn about one important phenomenon:

Many of these benefits never materialise because employees do not use the technologies.

Here we drill down to a particularly important phenomenon of interest, individuals’ rejection of IT (i.e., their unwillingness to use IT), which narrows the problem domain down and will be useful in focussing the research later on. Finally, we learn about an important problem in the available knowledge that relates to this phenomenon:

The literature to date has studied only why individuals accept new technologies but not why they reject them.

Here we make a statement about the current body of knowledge. The problem with the body of knowledge we have (i.e., the literature available to date) is that it has a gap—that is, we do not know enough yet to address the specific phenomenon in the problem domain. For example, as the Covid-19 pandemic spread across the globe in 2020, at the beginning we had very little knowledge about the virus, its infection rates, and its possible cures or vaccinations. The problem then was a gap of knowledge.

A “gap” in knowledge is a typical problem with the available knowledge, but it is not necessarily the best or only problem (Alvesson & Sandberg, 2011). For example, we could develop a set of arguments to strengthen the proposition that our theories on technology acceptance (e.g., Venkatesh et al., 2003) fail to predict the opposite, rejection of technology, conclusively. Other typical problems with the knowledge could be that they have yielded inconsistent results: study A suggests one thing and study B suggests an opposite thing, so which is correct? That is a problem. Another problem might be that the knowledge to date rests on assumptions that are no longer current or realistic, such as when studies assume that findings for a sample of men must also apply to women.

Having introduced an important problem domain with a specific phenomenon and a substantial problem with the knowledge to date allows us to formulate a research question as the logical conclusion to these arguments:

Why do people reject new information technology?

This question is one that Centefelli and Schwarz (2011) set out to answer in their paper, “Identifying and Testing the Inhibitors of Technology Usage Intentions.” They do a good job in motivating why research on technology rejection is important, why the question of individual unwillingness to use technology is an important specific phenomenon, and why an answer to their research question helps alleviate a problem with the available literature.

1.2 Specification of the Research Question

With appropriate motivation, a good research question can be precisely defined. Research questions are typically one of two types based on the issues they address:

  1. 1.

    “What,” “who,” and “where” questions tend to focus on issues we seek to explore or describe because little knowledge exists about them.

  2. 2.

    “How” and “why” questions are explanatory as they seek to answer questions about the causal mechanisms that are at work in a particular phenomenon.

Exploratory and descriptive questions (type 1) are different kinds of research questions from explanatory questions (type 2). Type 1 questions seek to learn what the situation of a phenomenon looks like. We ask these questions about phenomena that are new to the world. For example, at the onset of the Covid-19 pandemic, the world realised that there was a new virus. The first step was then to find out everything about it: what it looks like, what it does, what its genetic structure is, where it occurs, who can be infected by it, and so forth. We are not explaining (or curing) anything at this stage; we are exploring the virus in order to describe it fully.

Type 2 questions then seek to explain the cause and effect mechanisms behind why and how something works. For example, we wanted to find out how the Covid-19 virus infects people so we could devise treatments and vaccinations that hinder the mechanism by which the virus infects people.

Often, type 2 questions temporally succeed type 1 questions as it is difficult to explain a phenomenon without first systematically exploring and describing it. For example, one of the most significant scientific breakthroughs of all time was the description of the double-helix structure of a deoxyribonucleic acid (DNA) molecule. Watson and Crick described it in 1953, and this description gave rise to modern molecular biology, which explains how genes control the chemical processes in cells. The double helix also paved the way for new and powerful scientific techniques, such as recombinant DNA research, genetic engineering, rapid gene sequencing, and monoclonal antibodies. A description is fundamental to being able to build explanations.

As you may have gleaned from the differences between description and explanation in the two types of questions, research questions also unveil clues about how the research question can be answered. The research processes for exploration and description differ from those for explanation. In other words, certain types of questions favour certain research methods.

Developing the research question as your problem statement is probably one of the most important steps in your doctoral study, so you should be both patient and flexible in doing so. Give yourself time to revise questions as your knowledge and experience grow over time, and resist being rigid and fixed on one type of problem if you realise that your doctoral study takes you down a path where what you do is not quite what you originally set out to do.

If you struggle with this important task, you may find that guidelines for structuring research questions hierarchically can be beneficial. For example, a common approach to hierarchical structuring begins with a managerial question before moving on to research and investigative questions (Cooper & Emory, 1991):

  1. 1.

    The managerial question states the driving question of the study: who the interested audience is and why the question is important to them.

  2. 2.

    The research question captures the general purpose of the study. It is derived from the managerial question and translates it into a research problem.

  3. 3.

    The investigative question then identifies what must be answered to address the research question.

1.3 Justification of the Research Question

As a final step, a good research question comes together with a convincing argumentation for why the particular problem is significant and deserves our attention. Thus, for each question that indicates a problem with knowledge in a particular area (our problem statement), we should be able to offer an argument regarding why that particular focus, aspect, or question deserves our focus. Think of the research question formulation as a top-down process in which you narrow down a particular domain or field of investigation to bring it to one specific question in the domain. In the motivation process, you argue why the domain should be examined, and you identify a phenomenon in the domain that warrants attention. In the specification of the research question, you define one particular aspect of some phenomenon that you propose to examine from a particular angle. In this final step, you offer arguments regarding why the particular focus demonstrated in the research question(s) is warranted.

After specifying your research questions, it can be useful to reflect on them. The following criteria can be applied to evaluate the quality of your research questions and assess critically your set of research questions:

  • Feasiblity: Are adequate subjects, technical expertise, time, and money available, and is the scope manageable?

  • Interestingness: Are you confident that you can maintain an interest in the topic and maintain your motivation to study it for several years?

  • Novelty: Will an answer to the research question confirm or refute previous findings or provide new findings?

  • Ethicality: Will pursuing and answering the question violate ethical principles for the conduct of research or put the safety of the investigators or subjects at risk?

  • Relevance: Will both the question and its answer(s) inform scientific knowledge, industry practice, and/or future research directions?

2 Research Design

Once your research question is well specified, the next challenge is to craft a plan of action to answer the question: a research design . A well-specified question will suggest the most appropriate course of study for answering the question.

A research design is a blueprint for the collection, measurement, and analysis of the data used to answer the stated research question. It should be economical and reflect complex research planning decisions that require compromises and trade-offs among the demands of resources, time, quality, and data access. But before discussing different types of research designs, let us explore the core methods of intellectual reasoning that form the basis of all types of research designs: induction, deduction, and abduction.

2.1 Intellectual Reasoning: Induction, Deduction, and Abduction

There is no series of events that commonly and consistently unfolds in a scientific process and would, therefore, be common to all research designs . However, all research designs help researchers engage in systematic forms of intellectual reasoning to generate knowledge from data or test knowledge against data. The aim of science is to advance human knowledge through such approaches as extension and intension (Kaplan, 1998/1964). Extension in this context refers to exploring new areas by applying existing knowledge in one area to other areas. Intension refers to seeking more complete knowledge in a single area. In other words, we can go outside to build new knowledge, or we can go deeper into the knowledge we have already accumulated.

This distinction is important because these two modes of advancing knowledge rest on different forms of intellectual reasoning: knowledge growth by extension relies on inductive reasoning, which is used to explore new knowledge, whereas knowledge growth by intension relies on deductive reasoning, which is used to refine and test existing knowledge.

Induction is a form of logical reasoning that involves inferring a general conclusion from a set of specific facts or observations. By developing patterns and commonalities, inductive reasoning allows tentative hypotheses and propositions to be formulated that declare general conclusions or theories. In other words, induction is used to infer theoretical concepts and patterns from observed data or known facts to generate new knowledge by proceeding from particulars to generals. A simple example of induction is as follows:

  • Every life form we know of depends on liquid water to exist.

  • Therefore, all life, including that we do not know of, depends on liquid water to exist.

Inductive arguments can be weak or strong. The induction “I always hang pictures on nails. Therefore: All pictures hang from nails” is an example of a weak induction because the observation is too limited to lead to such a broad generalisation.

The problem with induction is that inductive arguments cannot be proven or justified, only supported or not supported. Still, inductions are an accepted and often useful pathway for constructing explanations or hypotheses because conclusions are offered based on educated predictions. Case study research is a good example of research involving inductive reasoning because this kind of research collects observations that form the basis of educated speculations, which could form the basis of a more general theory. Inductive reasoning typically goes as follows: “I studied phenomenon X in Y number of cases and I have always found the particular relationship or phenomena Z to be at work. Hence, the evidence collected in my observation leads me to formulate the tentative proposition that Z is related to X in this or that way.”

Deduction is a form of logical reasoning that involves deriving arguments as logical consequences of a set of more general premises. It involves deducing a conclusion from a general premise (i.e., a known theory) to a specific instance (i.e., an observation).

Deduction is commonly used to predict the results of hypotheses or propositions, an approach to science called the hypothetico-deductive model (Mertens & Recker, 2020). That is, to predict what observations one might make if an inquiry is conducted, a hypothesis is treated as a premise, and from it some not obvious conclusions are logically derived, tested, and revised if necessary. In other words, through deductive reasoning, we attempt to show that a conclusion necessarily follows from a set of premises or hypotheses.

Deduction can be seen as an attempt to test concepts and patterns known from theory using new empirical data. You may already recognise that the principle of falsification is core to deductive reasoning as by deriving hypotheses from general knowledge and testing them in new settings, we can demonstrate a theory to be wrong or right in those settings, but we can never prove it. A simple example of deduction is as follows:

  • All men are mortal.

  • Socrates is a man.

  • Therefore, Socrates is mortal.

Similar to induction, deduction has potential drawbacks, the most obvious of which are related to deductive soundness and validity. Consider this deduction:

  • Only quarterbacks eat steak.

  • John eats steak.

  • Therefore, John is a quarterback.

We can see here that the deductive reasoning as it is applied is logically sound (the conclusion that John is a quarterback), but we don’t know whether the final statement is valid because the premise “Only quarterbacks eat steak” is not true. In other words, we can deductively reason in good faith but still end up with incorrect conclusions when the premise may not be valid (other people like eating steak, too).

At this point, you may recognise that induction and deduction play a role in many scientific processes, from setting up a plan to collecting data to the intellectual challenges of developing a new theory. However, neither is sufficient or complete. Mechanically executing induction or deduction cannot guarantee that knowledge will be advanced or new theory generated.

Abduction, a third form of intellectual reasoning, is the process of making sense of an observation by drawing inferences about the best possible explanation. The key point here is that abduction is not a process of inference or deduction but a trial-and-error search for a satisfactory explanation for an observed consequence after the fact. Some people refer to abduction as a form of educated or informed guessing (Peirce, 1958).

One key difference between abduction and induction or deduction is that abduction involves a creative process rather than a logical process. It is an operation geared towards the discovery of entirely new ideas (e.g., a new theory) rather than a mode of justification (through deduction) or formal inference (through induction). One could see abduction as a form of intellectual reasoning that is more concerned with discovery or design than rationalisation or validation.

Table 3.1 compares the three forms of intellectual reasoning using an example adapted from Fischer and Gregor’s (2020) discussion of the topic.

Table 3.1 Induction, deduction, and abduction
Full size table

2.2 Exploration, Rationalisation, and Validation

No research process relies on induction, deduction, or abduction in isolation. Consciously or not, we usually use multiple, if not all, of these strategies to explore and reason about phenomena, facts, and assumptions when we generate new knowledge. All of these modes of reasoning have some advantages and some disadvantages. Induction is useful for developing theories from observations and other singular facts and evidence, but it is insufficient for demonstrating the validity of any emergent theory. Deduction in itself can be used to test theories using some or many individual cases, but it must rely on a robust set of premises to begin with. Abduction may lead to entire new ideas or breakthroughs, but it rests on careful observation of facts and an understanding of existing rules and assumptions as a point of departure.

In any of the three forms of reasoning, observation plays a key role. Observation concerns the systematic discovery of things or phenomena encountered in common experience. It supports attempts to understand these observable things by discovering a systematic or structural order in them. Much of scientific work relies on observation. Many scientific advances began with scientists exploring and documenting previously unknown or understudied phenomena and scoping their magnitude, extent, or boundaries.

As part of a scientific study, observations must be made in adherence with the principles of scientific inquiry, so observations must be precise, reliable, and independent. An observation is conducted to gain an initial understanding of a phenomenon in its context and to generate ideas about the phenomenon or its relationships to other phenomena that may lead to the formulation of speculative propositions or theories of explanation.

Sound research designs often employ combinations of strategies that rely on induction, deduction, and/or abduction to achieve a meaningful mix of exploration , which generates an initial understanding and description of a phenomenon; rationalisation , which we use to begin to make sense of the puzzle or problem; and validation , which subjects our emergent or developed theory to rigorous examination and testing.

Exploration, rationalisation, and validation do not necessarily follow each other in a defined linear or temporal manner. Good research typically moves back and forth among them, as shown in Fig. 3.1.

Fig. 3.1
figure 1

Exploration, rationalisation, and validation in research design

Full size image

The exploration of a phenomenon can provide a basis for rationalising about it. For instance, using observations, we can rationalise about a solution to a problem, begin to explain a behaviour, or begin abduction to generate likely explanations or solutions. The rationalisation process can result in the need for further exploration (as shown in Fig. 3.1 by the arrow moving from rationalisation back to exploration). We may find that explaining a particular behaviour requires that we collect additional observations about other behaviours that we did not identify as relevant to our initial exploration. The interplay between rationalisation and exploration can also provide a set of initial evidence against which we can test the outcomes of our rationalisation process or evaluate a set of tentative propositions between constructs that capture a phenomenon. The rationalisation should be valid in light of any observations that we collected.

Once a rationalisation, in which tentative general propositions are created through inductive reasoning from observations or through abduction, has been made, we can proceed to validation, where we develop testable hypotheses or propositions for a particular context or scenario from our more general theory. These hypotheses can be subjected to further empirical tests using new or existing cases. The results or evidence collected may suggest that we revise our rationalisation (moving from validation back to rationalisation), which could involve abduction. For example, our validation may find an observable mechanism that speaks against the logic of our propositions, requiring us to make an educated guess about why it happened or how these observations can otherwise be explained.

Validation often also employs findings from exploration as a basis by defining the sample frame, the target population, or other contextual parameters that initial observations suggested were important. Likewise, the results might prompt a further exploration of the phenomenon, so we move from validation to exploration. For instance, observations might be collected that can be used to refine results, like those that are, at least on the surface, contradictory to the developed theory.

To illustrate this view of research designs, consider Fig. 3.2 and Table 3.2, which list elements of my own doctoral research program and how they relate to exploration, rationalisation, and validation. Study 1 reported on observations gathered through a survey, a type of exploration of the phenomenon, the adoption of the Business Process Model and Notation (BPMN) standard. Study 2 involved rationalising the phenomenon analytically through a deductive comparison of the standard against alternatives. Study 3 contained elements of deductive validation (empirical testing of propositions) and inductive exploration (identification of new, unexpected findings) on the basis of qualitative research (interviews). Study 4 was a typical deductive form of the empirical validation of two propositions that were based on quantitative survey data. The entire research program was later published in a book covering all four studies. Abduction did not feature prominently in my PhD research. I worked deductively for large parts of the research and inductively to some extent. I did not create a breakthrough idea or an entirely new discovery.

Fig. 3.2
figure 2

Elements of exploration, rationalisation, and validation in my doctoral research design

Full size image
Table 3.2 Studies in my doctoral research design
Full size table

2.3 Choosing a Research Design

A good research design typically includes work that combines induction, deduction, and abduction. You can use these three themes to reflect on whether your research design involves an appropriate way to rationalise, validate, or explore.

Fig. 3.1 also shows that a research design is the outcome of a process with many choices, none of which are trivial. Their implications regarding the potential of your research and its likely outcomes are significant. Table 3.3 summarises some of the important design decision parameters beyond the focus on the mode of intellectual reasoning alone. By no means are these decisions binary (either-or) in nature; rather, they are key points along a continuum of choices.

Table 3.3 Research design decisions (adapted from Cooper & Emory, 1991)
Full size table

The rows in Table 3.3 describe the spectra by which we can examine our research design choices:

  • The “aim” spectrum between exploration and explanation: Is the overall aim of the research more about exploration or explanation or somewhere on the continuum between them? For example, perhaps the aim of a research project is not only exploratory observation but also explanatory in determining an intervention. Intervention can, for example, help an organisation with its struggle to introduce new digital technologies and avoid issues like rejection and decreased performance.

  • The “method” spectrum between qualitative and quantitative modes of scientific inquiry: Which sets of procedures and techniques will allow obtaining the best possible answer given the research aim? Will the methods involved rely more on qualitative, quantitative, or some other data?

  • The “boundary” spectrum between case related and statistical: your research might be limited to a particular case (such as the case of an organisation or an individual) or by statistical properties (such as the required sample size for a survey or experiment). Variations along the spectrum might include research on several cases at one point in time or one case over time.

  • The “setting” spectrum between field and laboratory: Where will the location of your research mainly be? Will it be in the field—in the real world, within an organisation or community? Or will it involve participants, such as students, that are recruited to undertake experiments or other data collection efforts in controlled, simulated environments (i.e., a laboratory)?

  • The “timing” spectrum between cross-sectional and longitudinal: the focus of the “timing” might be one case over time (longitudinal) versus several cases at one point in time (cross-sectional) or somewhere in between.

  • The “outcomes” spectrum between descriptive and causal: your research might focus on descriptions of a previously undiscovered phenomenon or on the discovery of certain causal mechanisms that explain why a phenomenon manifests the way it does. The focus might also be on both outcomes.

  • The “ambition” spectrum between understanding and designing: the goal of your research might be to analyse the root causes or inner mechanics of a problem or situation, or to design a solution or a novel artefact, or some combination of both.

At this stage, the researcher should go back to the research questions to determine how to make a selection alongside these different dimensions of choice. The key benchmark against which your research design must be aligned is the problem statement as specified in the research question(s), so the research design must match logically the research question, not the other way around. The research question determines to a large extent the choices one makes in selecting a research design. It dictates, for example, whether a more qualitative, explorative inquiry is warranted or a more quantitative, statistical one.

This point is important because one key downfall of doctoral students that I have come to encounter is the “I do research method X syndrome.” When I ask students about their research, they often say things like “I’m doing case study research,” “I’m doing design science,” or “I’m doing an experiment.” In all these cases, the students are focussing on one aspect of their research design—the choice of method—which they have often chosen independently of the projected research outcome or the research question they seek to answer.

Finally, one of the key choices in research design relates to the research methodology . We will discuss this challenge in more detail in Sect. 3.3, but let me just say here that, while the research methodology is critical, the research design must account for several other considerations as well. The most important ones have to do with data, risks, theory, feasibility, and instrumentation:

  • Data: What type of data is required? What type of data is or might be available? Where and how can I collect observations or other forms of evidence? How will I sample the data I gather?

  • Risks: What are the dangers associated with executing the research design? For example, how likely is it that a case organisation will not be available for study? What strategies are available to minimise these risks?

  • Theory: How much literature concerning the phenomena I am interested in is available? What are the problems with the knowledge base, and what form do they take? What findings have been produced that might have an effect on my work and influence the choices in my research design?

  • Feasibility: Can my research design be executed within the constraints associated with a doctoral study, such as time limitations, resource limitations, funding, experience, and geographic boundaries?

  • Instrumentation: How will my concepts of interest manifest in reality? How can my constructs be measured? Will my operationalisations be appropriate given the choice of research methodology and the set of available data?

In selecting a research design, you may be able to evaluate your progress by determining whether you have appropriate answers to the questions above and have maintained alignment with the type of research problem that is specified in your research question. The alignment does not have to be unidirectional (from the question to the design); in fact, most research questions are tweaked and altered over time to reflect an updated research design, although research questions should retain their prominence over the research design. Since research sets out to answer an important question, it is not appropriate to find an answer through your research design and then devise a question that fits the answer.

In making decisions about their research designs, students and their supervisors need to work together to select a research design that they feel comfortable with and with which they have experience. This is not to say that new research designs should not be pursued, but many problems I have seen originated largely from the fact that neither the student nor the supervisory team had any experience with the research design they chose. Executing such studies then becomes unnecessarily difficult because of limited resources for meaningful feedback based on experience.

3 Research Methodology

One critical element in the development of a research design is the selection of a research methodology. Many scholars even argue that the research methodology is the most important design choice in the research process.

Research methodology is a term that describes the strategy of inquiry used to answer a research question. Creswell (2009) states that strategies of inquiry are “types of qualitative, quantitative and mixed methods designs that provide specific direction for procedures in a research design.” I would add design science and computational methods to Creswell’s list of strategies of inquiry, both of which are recent advances in research methodology:

  • Quantitative strategies are procedures that feature research methods like experiments and surveys. Quantitative strategies are characterised by an emphasis on quantitative data (a focus on “numbers”).

  • Qualitative strategies are procedures that feature research methods like case study, ethnography, and phenomenology. Qualitative strategies are characterised by an emphasis on qualitative data (a focus on “words”).

  • Mixed methods are procedures that feature a combination of qualitative and quantitative strategies in either sequential or concurrent fashion (a focus on both “words” and “numbers”).

  • Design science methods are procedures that feature methods for building and evaluating novel artefacts like new models, methods, and systems as the outcome of a research process. Design science methods are characterised by an emphasis on the construction of the artefact and demonstration of its utility in solving an organisational problem (a focus on “artefacts”).

  • Computational methods are procedures for data visualisation and pattern identification that rely on software to analyse digital trace data automatically for the purposes of classification, description, or theory generation. Computational methods are characterised by an emphasis on the digital records of activities and events captured and stored through digital information and communication technologies (a focus on “digital traces”).

Setting aside the mixed-method strategy for a moment (because it combines characteristics of two or more strategies of inquiry), we can differentiate qualitative, quantitative, design science, or computational strategies using the methodological requirements imposed by the research questions or elements of the research design. These requirements are summarised in Table 3.4.

Table 3.4 Differences in research strategies
Full size table

Controllability refers to the extent to which events that occur during a study are under the researcher’s control. In a qualitative inquiry, where the researcher often enters an organisation to observe behaviours, processes, or events, control over what happens is low compared to quantitative inquiries like surveys and experiments, where control is exerted through the operationalisation of a measurement instrument that defines what will be measured and how. In design science research, control over progress and effects is typically in the hands of the person who is designing the artefact, the researcher.

Deducibility refers to the extent to which the strategy allows for deductive reasoning. Through the emphasis is on quantitative data, quantitative strategies allow for strong deductive reasoning through statistical or other quantifiable conclusions, whereas deducibility is typically limited when doing qualitative inquiries such as single-case research or ethnography. Deducibility is often low in design science research because of the challenge involved in embedding hypothesis testing into the design of an artefact.

Repeatability refers to the extent to which the findings are reliable in the sense that the research procedures can be repeated with similar results. This requirement is easier to meet in quantitative inquiries, where instruments of measurement tend to be precisely defined. Repeatability is high in design science research because the artefact is typically designed to be stable. Repeatability is also often high in research that uses computational methods because the digital trace data can be used to run and rerun our algorithms for analysis and discovery any number of times.

Generalisability refers to the extent to which the findings and observations can be generalised beyond the data that are examined. Quantitative inquires, especially surveys, tend to provide greater generalisability beyond the sample than qualitative inquiries do because the latter are more deeply immersed in the context of the inquiry. Design science and computational methods are typically low in generalisability: design science creates situated artefacts for the most part that are meant to work for a particular problem in a particular setting. Likewise, the data basis for computational methods comes from a particular set of digital records (and only those records that are digitally available), so they typically do not allow broader conclusions beyond the boundaries of the data itself.

Explorability refers to the extent to which a research strategy encourages or enables the discovery of previously unknown or unconsidered observations or findings. Explorability is typically built into qualitative inquiries through an emphasis on broad and open measurements, while quantitative inquires, with their precise and formalised measurements, are more limited in terms of exploring beyond the focus of the study. Explorability can be an attribute of some artefact designs, but it is not a key requirement for creating novel artefact designs. Explorability is often high in computational methods because the key area of application is the use of automated algorithms to discover patterns, themes, and associations in data that are typically too large for manual analysis and interpretation.

Complexity refers to the extent to which a research design leads to comprehensive, exhaustive, and multifaceted contributions to knowledge. Quantitative inquiries are characterised by selected, precisely defined measurements of phenomena, whereas qualitative inquiry’s broader and more open data-collection procedures allow for more findings and contributions. Complexity in design science research depends heavily on the type of artefact, but it is often a key characteristic of the design since all simple artefacts have already been discovered and designed. Complexity in computation methods is also medium to high because of the nature or format of the digital trace data that are the subject of inquiry or the complexities of the automated methods and algorithms—or both.

Mixed-method designs are not mentioned in Table 3.4 because, as their name suggests, their key characteristic is that they combine, usually in a favourable way, different methods with the view to meeting several or all of the requirements.

Most often, mixed-method designs involve a combination of some qualitative and some quantitative methods. In such a situation, depending on the choice of research methods to be combined, the overall methodology may lean towards either purely qualitative or purely quantitative inquiries. A valuable mixed-method design is one that combines strong qualitative characteristics, like explorability and complexity, with strong quantitative characteristics, like generalisability and deducibility. This way, requirements that one method may not fully meet can be compensated by the other. Figure 3.3 groups several research methods in each of the two major strategies of inquiry and illustrates the extent to which different methods can meet desired characteristics of quantitative or qualitative strategies.

Fig. 3.3
figure 3

Qualitative and quantitative methodologies

Full size image

There are different reasons for choosing a mixed-method research design (Venkatesh et al., 2013). For example, the rationale for choosing a mixed-method strategy of inquiry can be to compensate for the limitations of one method (e.g., a survey) with the advantages of another method (e.g., a case study). Other reasons are to ensure a more complete observation of a phenomenon or to test the propositions that emerge inductively from one method, such as a case study, deductively through another method, such as a survey (Gable, 1994) or experiment (Chatman & Flynn, 2005). This type of mixed-method design is called developmental because it involves testing the results of one method by means of another method.

Selecting an appropriate strategy of inquiry to determine the research methodology is critical to the success of any research project and must be driven by the research question and by the current state of knowledge in the area of study. For example, an emerging problem domain or topic area is often initially explored through predominantly qualitative strategies because these strategies score highly on the characteristics of explorability and complexity and in turn allow researchers to develop a first comprehensive picture of that area and to identify and describe important concepts or phenomena within it. Conversely, as research in an area matures over time, the share of studies that use quantitative methodologies typically increases, building on more precisely defined concepts and exploring in a more controlled manner the studied behaviours, processes, events, or other phenomena, with a view to deducing generalisable findings. This is because demands for controllability and repeatability, for example, increase as more and more research in an area is being done.

Selecting a design science methodology indicates an entirely different strategy as the aim of design science research is not exploration, description, or explanation but instead intervention and problem-solving by constructing novel artefacts as solutions to problems. Even so, design science can be combined with or complemented by qualitative or quantitative research.

Computational methods, a new methodological approach to research, rely on automated collection and analysis of digital trace data. This strategy is new because such methods rely on the wide availability of large samples of digital records. Records of events and actions were once restricted to a computer system in a particular organisation (e.g., a transaction system in a bank), but so many business and social activities now involve digital technology that vast quantities of digital records about actions and events that occur in our lives are becoming available to researchers. Advances in the automated analysis of such data have led to algorithms that identify sequences, match patterns, mine associations, classify data into clusters or categories, and extract semantics from text, audio, and video. This field of inquiry is subject to substantial ongoing research and development itself, and it will be exciting to follow these developments and their implications for scientific activities, like discovery, theory generation, and hypothesis testing.

Personally, I like doctoral research programs that feature a combination of research methods. For example, when a subject area is not well understood, qualitative methods may be used to collect observations in an effort to build theory and testable hypotheses, after which such theory may be tested by means of deductive reasoning using quantitative methods like surveys and experiments. It should be noted, however, that such mixed-method approaches increase the learning burden for doctoral students—they need to learn not one but several methods and also understand ways to combine them appropriately. I should also point out that many excellent theses have been completed on the basis of one strategy alone.

4 The Role of Literature in the Research Process

A colleague of mine always tells his students: “you should read before you write.” There is a lot of truth in this advice. One of the key tasks in your effort to contribute to knowledge during your PhD journey will be to acquire knowledge. You need to know the current state of the knowledge in your area of study (codified in the literature) so you can demonstrate how you can contribute to it by adding new knowledge to the literature. Earlier in this book, we have discussed that you need to acquire at least three types of knowledge before you can even start your research:

  1. (1)

    Knowledge about the domain and topic of interest that relate to your chosen phenomena

  2. (2)

    Knowledge about relevant theories and available evidence that help you frame questions and phenomena, and

  3. (3)

    Knowledge about relevant research methods that you can apply to develop new knowledge, build innovative artefacts, or articulate new questions

Knowledge in the scholarly field is available predominantly in the form of articles and books because writings have historically been the most persistent and replicable medium in which to store and distribute knowledge. We build a cumulative tradition of knowledge by adding our own research to this repository of knowledge in the form of published books, book chapters, journal articles, and papers in conference proceedings, among other vehicles.

Doing research demands that you spend a significant amount of time and effort to acquire and critically appraise this cumulative knowledge base, constantly, repeatedly, continuously. You will not be able to add substantially to the body of knowledge without knowing where that body of knowledge currently stands, and because research is being done and published every day, the body of knowledge is not static; it moves and grows every day. A researcher has to be a good and avid reader.

I mention this role of the literature in the midst of discussing the research design process because it is already at this stage that a firm understanding of the body of knowledge is essential to many of the decisions regarding the planning stages of research:

  • The literature informs the extent, type, and nature of potential research problems because the research problem only exists if a problem is identifiable in the literature, and we can find that problem only if we have cumulative knowledge about it.

  • The literature informs where gaps of knowledge are about a particular phenomenon or question and where other problems with the extant knowledge are (e.g., inconsistency, false assumptions, inconclusiveness). Critically appraising literature helps in identifying an important academic research question, that is, one that deserves scholarly attention because its answer will contribute to the body of knowledge.

  • The literature informs the extent to which current theories can explain the particularities of a phenomenon or problem and where they fail to do so adequately.

  • The literature contains strategies and methodologies that have been used to research the same phenomena or problem and similar phenomena or problems.

  • The literature contains theories that can be used to frame an investigation.

  • The literature contains the current body of knowledge about research methodologies that are available.

A firm understanding of the literature is critical to the design and conduct of a doctoral study. Often, to beginning students, the list of readings appears large—maybe overwhelming. And even when you start with what looks like a long reading list (say, 50 papers), rest assured that the volume over time will easily be doubled, quadrupled, or more than that. Believe me: any “reading list” you may come across will be nowhere as long as the list of readings you will have consumed by the end of your journey. Therefore, start reading straight away, that is, from the start of your journey. Many doctoral students overlook parts of the literature because they underestimate how important the literature is to their ability to formulate sound research questions and research methodologies. As a rule of thumb, you can never read too much, but it is easy not to read enough.

In consuming the literature, it helps to follow a process of read-think-interpret. Reading a paper or book does not always mean consuming every sentence from start to finish as not all of the literature or even all sections of a piece of reading material will be relevant to your area of interest. Therefore, after you read, think about the piece’s relevance and interpret it based on your assessment of its relevance to you, that is, evaluate whether there are useful ideas, theories, concepts, methods, or findings in a reading that you should investigate in more depth. In following this process of read-think-interpret, the following questions can guide you:

  • What is the reading’s core contribution with respect to the contemporary practice in your field of inquiry?

  • How does it relate to other articles and practices?

  • Does it espouse a theoretical or methodological perspective that would be useful in studying the phenomena that are of interest to you? And why is this, or is this not the case?

  • How may it influence your own thinking of the field?

  • How do you think the reading influenced the body of knowledge in the field at the time it was published?

Following these guiding questions should benefit you in your quest to master the body of knowledge as it pertains to your research. In my own reading, I take several passes of read-think-interpret: I scan new papers when I come across them to develop an impression—Is this reading relevant to me? Do I find the idea, theory, evidence, data, method, or phenomena useful or interesting? My goal at this stage is to build passing knowledge about the paper, so when at some stage it becomes relevant to my research, I will remember it and engage in a second, more thorough reading that is more detailed and deliberate. In this second pass, I read a paper to determine exactly what was done and how: I read the title and the abstract, scan the introduction, move to findings and discussion, and in some cases go back to methods or implications. I read background sections only when I really need to.

Beyond my suggestions, there is ample guidance available for doing literature reviews using more or less structured processes. I suggest some “readings about reading.” The information systems discipline also has excellent web resources that are dedicated to the literature about theories and methodologies.

For example, in terms of theories , Larsen and Eargle (2015) created a wiki with summarised information on theories that are widely used in information systems research. At http://is.theorizeit.org/, you can explore theories by name and type of application, along with details about the theory, some examples of papers that use the theory, and links to related information.

IS scholars have also crafted online resources as a starting point to learn about and apply research methodologies . The most prominent of these resources are the following:

  • The AIS World Section on Quantitative, Positivist Research Methods in Information Systems (http://www.janrecker.com/quantitative-research-in-information-systems/): maintains an introductory material to address the needs of quantitative researchers in information systems, whether seasoned veterans or those just beginning to learn to use these methods

  • The AIS World Section on Qualitative Research in Information Systems (http://www.qual.auckland.ac.nz/): provides useful information on the conduct, evaluation, and publication of qualitative research in information systems

  • The AIS World Section on Design Research in Information Systems (http://desrist.org/design-research-in-information-systems/): provides useful information on understanding, conducting, evaluating, and publishing design science research

I am not aware of any online resource for computational methods that is comparable to these resources, but such a resource would be welcome, and I am expecting that someone will develop such a resource soon.

The literature about problem domains varies depending on the domain’s type, nature, and maturity. Many domains have their own outlets of coverage through practitioner communities, blogs, forums, journals, and so forth (e.g., CIO magazine). It is useful to explore the range of literature that connects various communities of technology practices. This literature is often valuable in, for instance, justifying the relevance and significance of a study. Good examples of such literature include market research reports by institutions like Gartner, Forrester, WinterResearch, and other technology-related research agencies.

A final suggestion is to be prepared to expand the scope of your reading. Many good papers on methods, domains, theories, and other aspects of the research process are published in outlets that you may not necessarily associate with the information systems field but that belong to some of its reference disciplines (e.g., management, computer science, organisational science, psychology). These fields have top journals whose papers are often useful. Expanding your literature search outside of your immediate domain can help you discover theories that are used in other research fields, see how methods are applied in other research, and learn how other scholars in other fields frame, examine, and solve real-world problems. Considering these adjunct fields of literature, a journey can be a source of inspiration and can even be a tipping point in your journey.

5 Further Reading

Finding a research question is an unstructured, domain-specific problem, so it can be difficult to find guidance in the literature. One article that can inspire your thinking is Weber’s (2003) editorial, “The Problem of the Problem.” Editors and senior scholars in other disciplines and journals have published similar pieces, such as in the Academy of Management Journal (Grant & Pollock, 2011). A particularly useful strategy in finding research questions is problematisation (Alvesson & Sandberg, 2011), a structured approach to finding problems like gaps, inconsistencies, and inconclusiveness in the literature, which then serves as the starting point for adding to the body of knowledge.

As for creating a pathway for answering a research question, many books are dedicated to research designs. One good example is Creswell (2009), which discusses research design choices from the perspective of favoured research methods. Leedy and Ormrod (2001) also include criteria for judging the quality of research designs.

The paper by Lee (1991) is an excellent resource on how to build research designs that combine what the author calls subjective, interpretive, and objective understanding. While these terms are different from mine (exploration, rationalisation, and validation) and the paper’s focus is much on the interplay between research paradigms (interpretivism versus positivism) in use at these levels, much of the argumentation and conclusions are similar.

Concerning the selection of a research methodology, ample readings are available that summarise research strategies, such as the 19 strategies Wolcott (2001) discusses and Creswell (2009), mentioned above. We discuss the literature on design science in Chap. 5, but if you are interested in the growing stream of research on computational strategies, Berente et al. (2019), Lazer et al. (2009), and Pentland et al. (2020) are good starting points.

Information about literature reviews and search strategies are widely available, varying in their aims, approaches, systematicity, and structure. Wonderful overviews of the types of literature reviews and their aims and approaches are Paré et al. (2015) and Templier and Paré (2015). Several other good resources include the following:

  • Webster and Watson’s (2002) guide to writing a literature review

  • The relevant section in Leedy and Ormrod’s (2001) book on research planning and design

  • The essays on conducting research reviews in Cooper (1982) and Strange & Strange (1972)

  • For inspiration and as good examples for how literature reviews can be conducted and lead to new knowledge, all articles in the theory and review sections of the information systems discipline’s top academic journal, MIS Quarterly.