Wicked Problems and Genuine Uncertainty

Abstract

In the early 1970s, Horst Rittel, a design theorist and urban planner at the University of Berkley in California, developed the concept of wicked problems. These are complex societal, organisational and policy planning problems that are difficult to define and structure properly because they are continually evolving and mutating in a dynamic social context. This chapter gives a short historical and theoretical background to the notion of wicked problems, relates it to the concept of genuine uncertainty and shows how and why GMA is attuned to dealing with these types of problems.

Some of the text in this section is taken from two earlier sources, including an address to the Royal Institute of Technology in Stockholm, and a paper presented to SRA (Society for Risk Analysis) Conference in Paris, 2004: “Modelling Society’s Capacity to Manage Extraordinary Events”.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

3.1 Background

In 1973, Horst Rittel and Melvin Webber, both urban planners at the University of Berkley in California, wrote an article for Policy Sciences with an astounding title: Dilemmas in a General Theory of Planning. In this article, the authors observed that there is a whole realm of social planning problems that cannot be successfully treated with traditional linear, analytical approaches. They called these wicked problems, in contrast to tame problems.

Just a year later, in his book “Redesigning the Future”, Ackoff (1974) essentially put forward the same concept, although in less detail, which he called messes or unstructured reality – and which later became known as “social messes” (Horn 2001).

The best way to appreciate the nature of wicked problems is to compare them to tame problems (comp. Conklin, 2001). A tame problem

• Has a relatively well-defined and stable problem statement.

• Has a definite stopping point, i.e. we know when the solution or a solution is reached.

• Has a solution which can be objectively evaluated as being right or wrong.

• Belongs to a class of similar problems which can be solved in a similar manner.

• Has solutions which can be tried and abandoned.

The definitive model for a tame problem is a relatively well-defined engineering problem: building a bridge, designing a circuit, putting a person on the moon. Such problems are not necessarily “simple” – they can be very complicated indeed. However, their very property of being well defined and stable inherently defines a range of solution concepts. Also, their stability (at least in the relatively short-term) means that they are essentially the same problem today as they were yesterday, and will be more or less the same problem tomorrow.

Wicked problems (WPs) are completely different. WPs are ill-defined, ambiguous and associated with strong moral, political and professional issues. They are subjective and strongly stakeholder dependent: there is often little consensus about what the problem actually is, let alone how to resolve it. Above all, WPs won’t keep still: they are sets of complex, interacting issues evolving in a dynamic social context. Often, new forms of wicked problems emerge as a result of trying to understand and solve one of them.

The most obvious wicked problems are complex, long-term social and organisational planning and policy problems. Examples:

• How should we fight the “War on Terrorism?”

• How do we get democracies to emerge from authoritarian regimes?

• What is a good national immigration policy?

• How should scientific and technological development be governed?

• What should we do to deal with crime and violence in our schools?

• How should our organisation develop in the face of an increasingly uncertain future?

“The classical systems approach … is based on the assumption that a planning project can be organized into distinct phases: ‘understand the problems’, ‘gather information,’ ‘synthesize information and wait for the creative leap,’ ‘work out solutions’ and the like. For wicked problems, however, this type of scheme does not work. One cannot understand the problem without knowing about its context; one cannot meaningfully search for information without the orientation of a solution concept; one cannot first understand, then solve”. (Rittel & Webber, 1973, p. 161) (Emphasis added.)

Conklin (2001) represents the difference in tackling tame and wicked problems with the following diagrams (Figs. 3.1 and 3.2).

Fig. 3.1

Traditional wisdom for solving complex problems: the “waterfall” (Conklin, 2001)

Fig. 3.2

The iterative pattern of working with, and understanding, a wicked problem (Designer/subject process – grey zig–zags) (Conklin, 2001)

Conklin continues:

“… there are two common organizational coping mechanisms that are routinely applied to wicked problems: studying the problem, and taming it.

While studying a novel and complex problem is natural and important, it is an approach that will run out of gas quickly if the problem is wicked. Pure study amounts to procrastination, because little can be learned about a wicked problem by objective data gathering and analysis. Wicked problems demand an opportunity-driven approach; they require making decisions, doing experiments, launching pilot pro-grams, testing prototypes, and so on. Study alone leads to more study, and results in the condition known as ‘analysis paralysis,’ a Catch 22 in which we can’t take action until we have more information, but we can’t get more information until someone takes action. …

… attempting to tame a wicked problem, while appealing in the short run, fails in the long run. The wicked problem simply reasserts itself, perhaps in a different guise, as if nothing had been done. Or, worse, sometimes the tame solution exacerbates the problem”. (Conklin, 2001, p. 10f.)

Of course, problems are wicked and tame only a potiori. In practice there is a sort of gliding scale between tameness and wickedness. However, there is a set of pretty clear criteria for judging the degree of wickedness (so to speak) associated with complex social and organisational planning problems. We will look at Rittel and Webber’s ten criteria shortly.

Dilemmas in a General Theory of Planning has been described as a landmark article. I do not disagree with this. When I first discovered it in the early 1990s I was totally fascinated by the issues which it touched upon and by the provocative way in which the authors presented their ideas. Clearly, the authors were challenging us to test our preconceived (or, more correctly, our academically crammed) ideas about the nature of social planning and what we actually mean by problems and solutions.

However, the “problem” that the concept of wicked problems addresses did not suddenly emerge in the late 1960s and early 1970s. WPs are about people, stakeholders, vested interests and politics. As such, they are as old as human society itself. So, if WPs are everywhere and all-the-time, then why is Rittel &Webber’s article referred to as “landmark”, and why did at least two separate U.S. based research groups (R&W on the one hand, and Russel Ackoff on the other) start fussing about all this at about the same time.¹

During a presentation of GMA in the late 1990s, I naively put this question to a group of U.S. security specialists. They practically fell off their chairs with laughter! What was going on? The Vietnam War was going on, along with the “War on Poverty” and the “War on Drugs”. All of these “wars” were essentially managed (badly) like huge engineering projects, and all ultimately bogged down or went seriously wrong. Also “going on”, as a reaction to government policy (or lack thereof), were the Civil Rights Movement, the anti-war demonstrations and the general baby boomer “revolution”.

It is no wonder that academics representing social planners and policy professionals sought a new awareness and new modes of explanation. As Rittel & Webber put it in 1973 (I paraphrase): The “publics” are not going to take it any longer, and planners are going to be held accountable for what they do, or don’t do. (Wouldn’t that have been nice?)

So what is the problem that the term “wicked problem” addresses?

The common sense approach to WPs is fairly straight forward: As stated above, WPs are about people and politics; they are subjective problems. Everything that has to do with people and society is ultimately subjective. People think, reflect, get mad and react. And whatever some philosophers, psychologists or neurobiologists might say about the notion of “free will”, most people think that they have it, and, on occasion, act accordingly.

Above all, WPs are about people as stakeholders. Stakeholders – competing and cooperating, vying for position, and willing to re-think and change their positions when it suits them – are the epitome of wicked problems. This is why such problems do not have stable problem formulations; do not have pre-defined solution concepts; and why their course of development cannot be predicted.

If you think that all of this is pretty straight forward or self-evident, you are right, and you can skip the rest of this chapter. You don’t need it.

Going beyond this “common sense” approach, there is any number of epistemological-ontological issues lurking here. These concern, for instance, the problem of freedom vs. necessity, the distinction between causality and determinism and the relationship between being and becoming. However, this is not a philosophical tract and I do not have the intellectual wherewithal to go down this path again. (I wrote my doctoral thesis on this subject 30 years ago and still get a migraine when I think about it.)

Instead, I want to limit the discussion to two practical methodological issues. These are:

• The etic and the emic approaches to social/cultural research.

• The issue of genuine uncertainty.

3.2 The Etic vs. the Emic

The two main pillars of social/cultural research are the so-called etic and emic approaches. These terms were coined by the linguist Kenneth Pike in the early 1950s (Pike, 1954).² However, the concepts they represent were hardly new at that time. The same essential concepts were expressed by Philo of Alexandria in 50 BCE (Wright, 2009), by G. W. Leibniz’ (1646–1715) in the relationship between efficient and final causes (Ritchey, 1983), and by Marx’ (often misappropriated) dialectical method – i.e. the continual developmental relationship between the (“objective”) material world and (“subjective”) human apperception of the world.

In studying social systems, the etic perspective relies on extrinsic concepts and categories that have meaning for scientific observers (e.g. per capita energy consumption, the spread of an epidemic or crime statistics). Professionals (“scientists”) studying these phenomena are the primary judges of the validity of an etic account. For simplicity sake, call it the objective account, in the sense that something is being studied from the outside, as an object.

The emic perspective focuses on intrinsic cultural distinctions, perceptions and motivations that are meaningful to the members of a given society or group. The group members (of an organisation or culture) are the judges of the validity of an emic account. Again, for simplicity, call it the subjective account.

Now, the question arises: is one of these accounts “truer” than the other? Is one of them more fundamental or more reliable? Which account will best give us the information we need in order to understand what is going on in society?

Cultural materialism is a research orientation in social anthropology that makes the etic-emic distinction, but tends to see the etic as primary and the emic as a complementary (but relatively passive) explanatory. (Really extreme materialists – rare these days – see the emic context of mind and consciousness as an epiphenomenon – i.e. something that arises out of objective, physical world processes, but which has no actual, reciprocal causal effect on the world; i.e. “mind is an illusion”). The relative dominance of the etic account is understandable if your principle aim is to study and describe something “objectively”, i.e. without disturbing it. However, for policy analysis and for decision making such an approach is deluded: you will never begin to understand a society (or any living system) until you start poking it with a sharp stick to see how it reacts.

Clearly, if your aim is not just to study and describe a social system (a population, an organisation or an institution), but to do something with it – i.e. to intervene, to change, to develop – then the emic account is not simply a complementary explanatory. It becomes an all important context for understanding what is actually going on. More correctly, it is the interaction between the etic (objective) and the emic (subjective) – as fully equal and efficient causal contexts – that drives the development of human social systems in an open-ended manner. And this interaction is characterised by something called genuine uncertainty.

3.3 Genuine Uncertainty (GU)

People often confuse the notions of risk and uncertainty. However, these are completely different animals. The person who is usually credited with establishing the distinction between these two concepts (at least in the field of economics) is Frank Knight in his work “Risk, Uncertainty, and Profit” (1921):

“… Uncertainty must be taken in a sense radically distinct from the familiar notion of Risk, from which it has never been properly separated. … The essential fact is that “risk” means in some cases a quantity susceptible of measurement, while at other times it is something distinctly not of this character; and there are far-reaching and crucial differences in the bearings of the phenomena depending on which of the two is really present and operating. … It will appear that a measurable uncertainty, or “risk” proper, as we shall use the term, is so far different from an un-measurable one that it is not in effect an uncertainty at all”. (Emphasis added).

Risk is defined as a type of uncertainty based on a well grounded (quantitative) probability. Formally, Risk = (the probability that some event will occur) x (the consequences if it does occur). Where probability is relevant, one can calculate risk. And if risk has a well grounded probability then, as Knight says, there is, in effect, no uncertainty at all.

Genuine uncertainty, on the other hand, embodies processes and outcomes which cannot be ascribed (well grounded) probabilities. Thus in the world of social, political, organisational and ideological systems (i.e. in working with “subjective problems”), what is often called risk analysis is not about risk at all, but is properly a matter of genuine uncertainty.

Needless to say, banks, insurance companies, stockbrokers and other serious gambling establishments love risk and hate uncertainty. This is exemplified by insurance companies essentially being willing to insure anything to which they can assign a well-grounded probability while avoiding anything that they cannot (ironically called “Acts of God”).

It gets worse: besides not being amendable to quantification, processes involving GU do not necessarily have specified outcome spaces.

Suppose someone asks me: “What will the population of Sweden be in the year 2500?” (assuming that what we call “Sweden” is still around). My answer is: “I haven’t got the foggiest!” It could be anything – almost. However, there is one thing I do know with great certainty: it will be a number between 0 and (let’s say) 100 billion. That is to say, I don’t know the number, but I know what the “outcome space” looks like. It is the counting manifold representing zero and the positive integers. The outcome space is fully specified.

Now, suppose someone asks me: “What new scientific discoveries will be made in the next 500 years?” Again, my answer is: “I don’t have the foggiest!” But this time, I don’t even know how to categorise the possibilities. I have no certain knowledge about the “outcome space” (i.e. where to place such unknown unknowns in relation to what we know today) since this space does not exist yet. This type of GU involves an open-ended set of possibilities, in which the “outcome space” is unspecified.

Unspecified uncertainty is especially relevant when working with long-term future developments. This type of uncertainty is inherently ineradicable – you cannot get rid of it by trying to obtain more information about it, because the information needed to reduce it simply isn’t there (yet).

Finally there is the issue of so-called agonistic uncertainty. The word-stem agon comes from (classical) Greek and means a “contest” or (mental) “struggle” (compare the words agony and antagonistic). Agonistic uncertainty has to do with competing and cooperating actors, wills and intensions (think of the stock market). In terms of a complex self-referential system, ³ it refers to a network of conscious agents (e.g. individuals, organisations, institutions, nations) acting concurrently and reacting to each other. The behaviour of the system is emergent as opposed to predetermined (i.e. it produces surprises), and its development is unpredictable.

To sum up, the genuine uncertainty inherent in WPs is characterised by three (intertwined) properties that defy prediction:

• It cannot be ascribed a (well-grounded) probability (therefore you cannot predict the “odds” of certain things happening).

• It does not have a well-defined or complete outcome space, but is full of “unknown unknowns” and emergent processes (so you cannot even predict what might happen).

• It involves subjective, self-referential behaviour (which means that meta-actors can consciously decide to do unexpected, surprising things).

3.4 Ten Criteria for Wicked Problems

It is instructive to look at the original criteria put forward by Rittel and Webber to characterise WPs. (It has been pointed out that some of these criteria are closely related or have a high degree of overlap, and that they should therefore be condensed into four or five more general criteria. I think that this is a mistake, and that we should treat these criteria as 10 heuristic perspectives which will help us better understand the nature of complex social planning problems.)

• 1. There is no definite formulation of a wicked problem.

  “The information needed to understand the problem depends upon one’s idea for solving it. This is to say: in order to describe a wicked problem in sufficient detail, one has to develop an exhaustive inventory for all the conceivable solutions ahead of time”. [This seemingly incredible criterion is in fact treatable. See below.]

• 2. Wicked problems have no stopping rules.

  In solving a tame problem, “… the problem-solver knows when he has done his job. There are criteria that tell when the solution or a solution has been found”. With wicked problems you never come to a “final”, “complete” or “fully correct” solution – since you have no objective criteria for such. The problem is continually evolving and mutating. You stop when you run out of resources, when a result is subjectively deemed “good enough” or when we feel “we’ve done what we can…”

• 3. Solutions to wicked problems are not true-or-false, but better or worse.

  The criteria for judging the validity of a “solution” to a wicked problem are strongly stakeholder dependent. However, the judgments of different stakeholders …“are likely to differ widely to accord with their group or personal interests, their special value-sets, and their ideological predilections”. Different stakeholders see different “solutions” as simply better or worse.

• 4. There is no immediate and no ultimate test of a solution to a wicked problem.

  “… any solution, after being implemented, will generate waves of consequences over an extended – virtually an unbounded – period of time. Moreover, the next day’s consequences of the solution may yield utterly undesirable repercussions which outweigh the intended advantages or the advantages accomplished hitherto”.

• 5. Every solution to a wicked problem is a “one-shot operation”; because there is no opportunity to learn by trial-and-error, every attempt counts significantly.

  “… every implemented solution is consequential. It leaves “traces” that cannot be undone … And every attempt to reverse a decision or correct for the undesired consequences poses yet another set of wicked problems …”.

• 6. Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan.

  “There are no criteria which enable one to prove that all the solutions to a wicked problem have been identified and considered. It may happen that no solution is found, owing to logical inconsistencies in the ‘picture’ of the problem”.

• 7. Every wicked problem is essentially unique.

  “There are no classes of wicked problems in the sense that the principles of solution can be developed to fit all members of that class”. …Also, …“Part of the art of dealing with wicked problems is the art of not knowing too early which type of solution to apply”.

• 8. Every wicked problem can be considered to be a symptom of another [wicked] problem.

  Also, many internal aspects of a wicked problem can be considered to be symptoms of other internal aspects of the same problem. A good deal of mutual and circular causality is involved, and the problem has many causal levels to consider. Complex judgements are required in order to determine an appropriate level of abstraction needed to define the problem.

• 9. The causes of a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem’s resolution.

  “There is no rule or procedure to determine the ‘correct’ explanation or combination of [explanations for a wicked problem]. The reason is that in dealing with wicked problems there are several more ways of refuting a hypothesis than there are permissible in the [e.g. physical] sciences”.

• 10. [With wicked problems,] the planner has no right to be wrong.

  In “hard” science, the researcher is allowed to make hypotheses that are later refuted. Indeed, it is just such hypothesis generation that is a primary motive force behind scientific development (Ritchey, 1991). Thus one is not penalised for making hypothesis that turn out to be wrong. “In the world of … wicked problems no such immunity is tolerated. Here the aim is not to find the truth, but to improve some characteristic of the world where people live. Planners are liable for the consequences of the actions they generate …”

3.5 Tackling Wicked Problems with General Morphological Analysis

How, then, does one tackle wicked problems? Some 20 years after Rittel & Webber wrote their article, Rosenhead (1996), of the London School of Economics, presented the following criteria for dealing with complex social planning problems – criteria were clearly influenced by the ideas presented by Rittle, Webber and Ackoff:

• Accommodate multiple alternative perspectives rather than prescribe single solutions

• Function through group interaction and iteration rather than back office calculations

• Generate ownership of the problem formulation through transparency

• Facilitate a graphical (visual) representation for the systematic, group exploration of a solution space

• Focus on relationships between discrete alternatives rather than continuous variables

• Concentrate on possibility rather than probability

As I will attempt to argue in the coming sections of this book, group facilitated GMA is fully attuned to these criteria. As a preview, let us take some of Rittel &Webber’s criteria for WPs and see how GMA stacks up.

Criterion #1

… in order to describe a wicked problem in sufficient detail, one has to develop an exhaustive inventory for all the conceivable solutions ahead of time.

Done properly, GMA results in an inference model which strives to represent the total problem space, and as many of the potential solutions to the given problem complex as possible. This goes a long way in satisfying this seemingly incredible criterion. The idea is to “play” with the inference model in order to allow stakeholders to better understand the problem space and the possible consequences of alternative decisions. We literally build up an inventory of all possible solutions, before we begin to fully understand the problem.

Criterion #3

Different stakeholders …“are likely to differ widely to accord with their group or personal interests, their special value-sets, and their ideological predilections”.

The process of creating morphological inference models through facilitated group workshops is as important as the end-product – i.e. the model itself. As many stakeholders as possible should be engaged in the work, in order to create a common terminology, common problem concept and common modelling framework. Principal stakeholders and subject specialists should therefore be brought together in a series of workshops to collectively (1) structure as much of the problem space as possible, (2) synthesize solution spaces, (3) explore multiple solutions on the basis of different drivers and interests and (4) analyse stakeholder structures. The different stakeholders do not have to agree on a single, common solution, but must be encouraged to understand each other’s positions and contexts.

This last point is crucial. Consensus means “general agreement or concord” within a group. Facilitators usually differentiate between first-order and second-order consensus. The normal first-order form is that of gaining a common standpoint or agreeing upon a common solution. This is seldom the case with stakeholder groups working with wicked problems. So-called second order consensus is when stakeholders in a group learn to accept each other’s specific stakeholder positions – on the basis of understanding the reasons for these positions. (This is called “position analysis” in Swedish, and is a discipline in itself.)

Criterion #7

…part of the art of dealing with wicked problems is the art of not knowing too early which type of solution to apply.

In GMA we call this “remaining in the mess”, i.e. keeping one’s options open long enough to explore as many relationships in the problem topology as possible, before starting to formulate solutions. This can be a frustrating process for inveterate “problem solvers”, but is an absolutely necessary procedure when modelling wicked problems.

Criterion #8

Every wicked problem can be considered to be a symptom of another [wicked] problem.

With a morphological inference model, one can treat any particular parameter or “issue” as the starting point or “independent” variable. This allows one to change perspectives and treat different issues as both causes (drivers) and effects (or symptoms). Everything is connected, which is what both wicked problems and GMA is about.

Criterion #10

[With wicked problems,] the planner has no right to be wrong.

Not only should planners be part of the GMA modelling and shaping process, but also the potential “consumers” or “victims” of said planning. People do not like to be “planned at”, without having something to say about it. Representative “consumers” of the planning project must absolutely be made part of the planning process itself. GMA allows for – almost insists upon – this type of participation.

Finally, as is the case with all modelling methods dealing with complex social planning problems, there is always the garbage in–garbage out problem. However, even here group facilitated GMA has some clear advantages. It expressly provides for a good deal of in-built “garbage detection”, since poorly defined parameters and incomplete ranges of conditions are immediately revealed when one begins the task of cross-consistency assessment. These assessments simply cannot be made until the morphological field is well defined and the working group is in agreement about what these definitions mean. This type of garbage detection is extremely important when working with wicked problems and social messes.