Obviously, all research has to be relevant. But to whom and in what way? The very definition remains elusive and the concept is not only politically laden but often so ambiguous that discussions of relevance seem to regress in many cases to mere rhetoric. Current editorial policies do not provide adequate guidelines for relevance that are sufficiently explicit and operational; guidelines on how to think about relevance and how to establish the relevance of one’s research are equally obscure. My goal in this essay is to explore the concept and how we could move toward more informed and constructive discussion of relevance of academic operations management (OM) research.

That OM research should be managerially relevant is a more or less institutionalized principle. In what can be considered a fairly typical editorial policy statement, the Decision Sciences Journal (DSJ) calls for “the potential to substantially impact either decision making theory or industry practice” (editorial policy at the publisher’s web site, 02/05/2009). This new journal, Operations Management Research: Advancing Practice Through Theory, highlights relevance even in its name. I for one think this is a welcomed and much-needed addition to the collection of OM journals, but there are a number of challenging research policy questions that have not in my view been addressed, and that must be addressed for OMR fully to realize its potential and to avoid sending mixed messages about what relevance is and what it is not.

I want to start by reminding that not all academic management journals emphasize managerial relevance, the Administrative Science Quarterly (ASQ) being perhaps the best example. Instead, these journals embrace academic freedom to its fullest, including maximal autonomy of researchers to engage in research they find compelling. In ASQ, the value of one’s research does not depend on what managers think of it; the focus is squarely on theoretical and empirical contributions. In my view, there is absolutely nothing wrong with such editorial policies and consequently, the goal of this essay is not to promote relevance. Instead, I take relevance as a given and examine its implications to scientific practice and research policy. But I want to remind the reader that depending on one’s knowledge interests and research goals, managerial relevance may in some cases be, well, simply irrelevant to one’s inquiry.

1 Diverse facets of relevance

Relevance is ambiguous, because it has more to do with values, perceptions and interpretations than facts. All attempts at a brief definition would rob the concept of the depth and the richness of its meaning. Further, both researchers and practitioners can be uncertain or ambivalent about what ultimately is relevant and what is not (Bazerman et al. 1998, March 1994); and I have not even started talking about organizations consisting of individuals and coalitions with diverging and incommensurate interests. An honest and impartial look at organizations has revealed that organizations simply do not have goals (Cyert and March 1992). What is relevant and compelling for top management may be irrelevant and wholly unimpressive to middle managers or workers. In the academic world in turn, anyone who has submitted a manuscript to a journal review process will confirm the well-known fact that academic journals exhibit painfully low inter-rater reliabilities of peer reviews (Starbuck 2006).

I will briefly examine various facets of relevance in OM research by juxtaposing (1) exploration versus exploitation research, (2) paradigmatic versus practical research interest, and (3) formal versus substantive theory.

2 Exploration vs. exploitation research

In OM, there is considerable bias toward what I will label exploitation research (as opposed to exploration research). The terms exploration and exploitation are adopted from March (1991), who argued that the central challenge in organizational learning is to strike a balance between “the exploration of new possibilities and the exploitation of old certainties” (p. 71). In most OM research—and academic management research in general (see Winter 2006)—we focus on how firms can become more effective given the extant collection of resources and technologies. This becomes obvious if we look at a typical OM survey or case study and the kinds of questions we ask; we evaluate something that already exists.

Focus on what already exists is related to our ontological position of scientific realism and for the most part, objectivism, that is, inter-subjective evaluation based on impartial observation and analysis of data. We take the existence of the phenomenon of interest as prerequisite and assume it our task as scientists to understand relatively well-defined and observable, though complex, empirical phenomena. Researchers and journal editors, not managers, get to decide which phenomena deserve our attention. We may be able to offer managerial relevance, but this is what I would call exploitation relevance in that we try to help managers make better use of existing resources and technologies.

What if the phenomenon is not “out there” in an ontological realist sense? What if it is just an idea or at best an ill-structured problem? What if we wanted to be relevant to those who seek to explore new frontiers by developing new technologies and new products, the organizational entrepreneurs? Should we seek exploration relevance by seeking solutions to ill-structured problems? There is a rich tradition in engineering and architecture called design science, which focuses specifically on tackling ill-structured problems. A few exceptions aside, design science has yet to find its way to OM research (Holmström et al. 2009). At present, relevance tends to be biased toward exploitation relevance and incremental improvement of practice at best.

3 Paradigmatic vs. practical research interest

Relevance is always linked to our knowledge interest. The knowledge interest in the vast majority of OM research is paradigmatic. By paradigmatic I mean that we have a number of theoretical (i.e., paradigmatic) discourses to which individual researchers and research projects seek to contribute. Within these discourses and research streams, we seek new ways of explaining empirical phenomena by developing and testing theoretical propositions. Paradigms typically have fairly well-established rules regarding the kinds of methods and data that are acceptable. We also often seek prediction, although not so much prediction of the future as prediction of the unknown—this distinction is important. Prediction of the future is not only unreasonable as an expectation, it is also unenforceable as an evaluation criterion. Prediction of the unknown is in turn both reasonable and enforceable. Predicting the sign of a coefficient in a regression equation in a sample about to be analyzed is an example of prediction in the context of what is known as hypothetico-deductive research (Hempel 1965).

Paradigmatic research can lead to what I label here cognitive relevance. Theories and arguments about focused factories, drivers of performance, product-process matrices, and the like, may of course be relevant to managers, but I submit that the relevance is more in the category of “giving managers something to think about.” They may lead to interesting dialogue between researchers and managers, but these are more cerebral activities and thought exercises than prescriptions in the sense that we as researchers would (or could) tell the managers what they should do in a given situation—hence, cognitive relevance. I have myself studied the focused factory concept in many different research projects, but not once have I felt qualified to provide normative advice; whenever I think I am smart enough to offer prescription, the manager brings up a contingency factor I have failed to consider. But this does not preclude relevance. I am sure many of us know managers who are primarily interested in scientists giving them an independent, disinterested observer’s view of their current understanding. They do not expect us to make decisions for them, or even to give recommendations for action—they know better than that. Instead, they seek intellectual challenge, and research based on a cognitive knowledge interest can be a valuable source of such challenge—relevant but not prescriptive. Dialogue is the essence of relevance.

There is, however, an element of relevance—which I label practical relevance—that extends beyond dialogue and cerebral activity. Some researchers assume an active role in affecting, not just evaluating, the phenomenon of interest. Of course, in order to be practically relevant in this sense, one must thoroughly understand the context, because the focus must be on making interventions, observing their both intended and unintended consequences, and ultimately producing knowledge, perhaps even a technology that solves a problem in a specific context. In such research, the knowledge interest is practical as opposed to cognitive or paradigmatic. Because action research can “both illuminate what exists and inform fundamental change” (Argyris et al. 1985, p. 4), practical knowledge interest is not the same as explorative research. Those with a practical knowledge interest may well be interested in improving the use of existing resources, not exploration of new possibilities. Practical relevance is not, however, a retrofitting activity; it is something that must be considered both at the outset and throughout the research program.

4 Formal vs. substantive theory

Empirical OM research, by definition, always takes place in a specific context. One’s knowledge interest can, however, be either contextual or acontextual. The goal can be to develop either formal theory that is abstracted from context, or substantive theory that is fundamentally context dependent in that it is a theory about the context; the formal-versus-substantive distinction is adopted here from Glaser and Strauss (1967, pp. 32–35). An excellent example of formal theory is the contingency theory of organizations (Donaldson 2001), which consists of a set of more or less precisely defined abstract concepts (e.g., size, task interdependence, centralization, formalization) and propositional statements that link these concepts to one another; increasing size leads to a decentralized organizational structure. These propositions are formal theory in that we do not need an empirical context of application to appreciate them. Contingency theory is also a good example of a normal-science paradigm in organizational research (Donaldson 1996). Examples of substantive theories given by Glaser and Strauss (p. 32) are theories of patient care and professional education. An excellent example of substantive OM theory is the special issue of Journal of Operations Management (vol. 24, no. 3), which was explicitly devoted to the process industries. Theoretical insight in many of the special issue articles was fundamentally tied to the process industry context in that the arguments presented in these articles cannot be understood without considering the context.

Should we encourage development of substantive theories into more generalizable formal theories? Glaser and Strauss indeed suggested that many formal theories emerge from substantive theory. It seems that the current emphasis in OM is indeed to encourage development of formal theory, which is consistent with the predominantly cognitive and paradigmatic research interest emphasized in the major OM journals. From the point of view of relevance, however, encouraging formal theory as opposed to substantive has important implications.

Let me introduce a colleague who has, in close collaboration with a large metropolitan hospital, developed a substantive theory of managing operations in orthopedic surgery; not the details of the actual surgical procedure, of course, rather the management of the process in general. Should he in the future (a) strengthen practical relevance by engaging in further research in the same context, collecting more data, analyzing it and ultimately, refining his substantive theory in an attempt to make successful interventions to improve practice; or (b) strengthen the cognitive research interest by formalizing his substantive theory so that he might be able to contribute to a formal theoretical discourse? Perhaps he has gained formalizable theoretical insight on the constraints that highly institutionalized environments place on the design of operational systems. Now, current editorial policies in many OM journals tend to encourage (b) over (a), whereas practitioners would obviously favor (a) over (b). Should we embrace (a) as an option as well, as a valuable scientific effort? Our time as researchers is limited, and we likely have to choose between doing (a) or (b). These choices may be strongly driven by editorial policy. Reflecting upon choices I have made in my career, I have chosen (b) over (a) about 90% of the time. I am not entirely convinced this percentage is exactly what it should be, but it does seem that in order to become a tenured professor in OM, this percentage must be fairly high.

5 Is being relevant possible in the first place?

At this point I am going to challenge the premise and ask: Is managerial relevance even possible? The question is far from trivial. Van de Ven and Johnson (2006) presented different views of the link between theory and practice. The most challenging view in terms of the relevance question was the view that theory and practice are indeed “distinct kinds of knowledge, [reflecting] a different ontology (truth claim) and epistemology (method) for addressing different questions” (pp. 802–803). Perhaps trying to establish relevance by “translating” one’s results into managerial language is like trying to squeeze apple juice out of oranges. If theory and practice are incommensurate, then relevance is not a question of knowledge transfer but knowledge production. Are we misguidedly pretending that something produced for one purpose can simply be “translated” to serve another purpose?

I am going to be provocative and suggest that if discussing managerial relevance in a manuscript is limited to the author’s own conjecture and advice to the imaginary manager in section 4.2 entitled “Managerial Implications,” we might be better off without it. Conjecture is outside the scope of scientific inquiry. It is unlikely that a researcher with a purely cognitive and paradigmatic research interest will be able to offer anything of value in terms of managerial relevance to the reader, let alone the practitioner. Plus, I am afraid no one is listening: very few managers read academic journals, therefore, who exactly is the audience for the section “4.2 Managerial Implications”?

Some authors have argued that the practical value of one’s research can be determined by a “market test” of sorts (e.g., Kasanen et al. 1993): if managers “buy it,” it is relevant. Using this criterion, we would probably think of, say, the Balanced Scorecard as relevant—and it is. But causality works from rigor to relevance: BSC sells and is relevant because it is rigorous. Astrology, feng shui and pornography sell, too, but scientific rigor is not the first thing that comes to mind as the explanation. If we use the market test for relevance, we must view it as at best a necessary but not sufficient condition. In order to assess relevance, we must understand the antecedent; in scientific work this antecedent is always rigor.

One more time, for the record: doing relevant research does not mean that rigor can be compromised; all scientific work must be rigorous, period. Depending on one’s research interest, however, rigor can mean different things. In paradigmatically oriented research where formal theory is often central, being rigorous means explicit demonstration of a theoretical contribution to the existing theoretical discourse and strict adherence to the rules and conventions of the paradigm. This naturally calls for thorough mastery of the extant literature and understanding of theoretical paradigms and discourses. In practically oriented exploration research in contrast, being rigorous calls for intimate knowledge of the empirical context. In action research and design science in particular, being paradigmatically (theoretically) rigorous is borderline irrelevant; what is important is uncompromising commitment to making a successful intervention that solves the real-life problem. Toward these ends, limiting oneself to a specific theoretical or paradigmatic discourse and tools legitimated by the paradigm falls prey to the “given-a-hammer-all-you-see-is-nails” syndrome. Real-life problems do not map onto paradigmatic domains, even though we are often compelled to label something “a marketing problem,” “an information problem,” or “an HR problem” (e.g., Simon 1997). Problem representations are obviously essential, but straight-jacketing problems to fit existing paradigms is like trying to play a new game with the old rules (e.g., Winter 2006). Design scientists in particular know how important it is to be able to navigate multiple domains of expertise (Holmström et al. 2009).

To be sure, if no one out there is interested in what I do, I am by definition not engaging in managerially relevant research. The idea that someone out there buying my research results constitutes evidence of relevance is not, however, a corollary that follows. Relevance is not about convincing a manager to spend $60 on a book I wrote or about getting them to say nice things about my work. Relevance is about creating an explicit mutual understanding between the researcher and the practitioner: “The [relevance] challenge cannot be defined simply as getting practitioners to value and incorporate what academics learn… Research is more likely to be seen as useful if there are opportunities for researchers and [practitioners] to take each others’ perspectives and to jointly participate in interpreting the results” (Mohrman et al. 2001, p. 357).

Last, we might ask if there is a correlation between research design and relevance. Are, say, case studies more likely to exhibit relevance than hypothetico-deductive large-sample survey research? Now, while there may be a correlation, it is probably not a strong one, and it is far from definitional. The fact that one studies just one or a few companies in a specific context does not mean one seeks practical relevance. Conversely, doing a large-scale statistical survey does not preclude relevance. A great example of the former is the Adler et al. (1999) study of the NUMMI plant; a highly contextualized in-depth case study, where the primary contribution is formal theory of organization design. A good example of the latter, in turn, is Schroeder and Flynn (2001), a managerially oriented book on manufacturing management. The insights offered in the book are based on a large-scale multi-industry-multi-country survey of manufacturing organizations. The research program offered, however, individual companies an opportunity to benchmark themselves with manufacturers in other countries. Indeed, companies in many countries paid to participate in the project. In sum, relevance does not derive from the research design but from the knowledge interest.

6 Implications to editorial policy

Let me in conclusion ask questions about editorial policy that may help us engage in a constructive discussion of relevance. I encourage everyone who writes and evaluates manuscripts to think about these questions. As in any collective effort, nothing is more fundamental than defining the rules of the game. The rules of the game when it comes to managerial relevance of our research are in my view seriously incomplete. I hope the following questions can lead to improved rules of the game.

  1. 1.

    What are the implications of editorial policy to practical versus cognitive research interests? Should research with an expressly practical research interest (e.g., design science and action research) be considered legitimate scientific work? Do articles with an exclusively cognitive and paradigmatic research interest have to have a clear impact on practice? Who is the judge of this impact? Should we exhibit more “editorial patience” toward those tackling the really challenging and forward-looking questions?

  2. 2.

    What is the balance between emphasis on exploration versus exploitation research? Is the researcher’s role limited to evaluation, or can a researcher take an active role in actually shaping empirical phenomena? What kind of advocacy is acceptable for an OM scientist?

  3. 3.

    What are the implications of research policy to the substantive versus formal theory balance? Do we (and should we) encourage a balanced proportion of each? If substantive theories are encouraged, do we in our editorial policies wish to specify preferred substantive domains?

Many editorial policies offer a number of implicit answers to these questions, but I think we would be better served if the answers were made explicit and operational. In my view, current editorial policies of OM journals tend to steer research toward evaluative, backward-looking research and well-established research questions. Are we as a scientific community where we want to be? The fact that we have a new journal entitled Operations Management Research: Advancing Practice Through Theory suggests to me that perhaps we are not. The question then becomes: How do we realize OMR’s goal of advancing practice? What will be different about the manuscripts that are published in OMR compared to others and what implications does this have to the peer-review process of OMR? What do I as an author need to consider before submitting a manuscript? Should we encourage, for instance, co-authoring with practitioners (e.g., de Treville et al. 2008)?

Ultimately, relevance is an empirical question and no amount of rule-setting, reflection, and policy-making within the scientific community is going to give us the final answer. Relevance is not factual or codifiable. Instead, it is established—much like all scientific knowledge—in a process of social construction (Astley 1985). If we really want adequately to tackle relevance, we must recognize that relevance can emerge only from cooperation and dialogue with the practitioner. The toughest challenge for the peer-review process, in turn, is probably this: relevance is not as much a characteristic of an individual research article or a particular finding as it is a characteristic of an entire research program. Trying to assess the relevance of an individual manuscript may well be just as hard as trying to judge a book by its cover.