1 The Task Environment Changes Behaviour

We watch an ant make his laborious way across a wind- and wave-molded beach. He moves ahead, angles to the right to ease his climb up a steep dunelet, detours around a pebble, stops for a moment to exchange information with a compatriot. Thus he makes his weaving, halting way back to his home....

He has a general sense of where home lies, but he cannot foresee all the obstacles between. He must adapt his course repeatedly to the difficulties he encounters and often detour uncrossable barriers. His horizons are very close, so that he deals with each obstacle as he comes to it; he probes for ways around or over it, without much thought for future obstacles. It is easy to trap him into deep detours. Viewed as a geometric figure, the ant’s path is irregular, complex, hard to describe. But its complexity is really a complexity in the surface of the beach, not a complexity in the ant....

An ant, viewed as a behaving system, is quite simple. The apparent complexity of its behaviour over time is largely a reflection of the complexity of the environment in which it finds itself.

—Herbert Simon, Sciences of the Artificial, p. 63–64.

People are, of course, not ants. Cognitively, they are much more complex. As he argued, Simon’s essential analogy still applies: observing people engaged in cognitive work yields, first, information about their task environment, and only second, information about their cognitive structure and abilities.

2 CAD Changes the Task Environment

An immediate implication for design is that design media, being a major part of the task environment, strongly impact designer action and behaviour. Yet the literature largely lacks accounts of such impact. Bhavani et al. (1999) argue that designers need specific training in the strategic use of design media in order to reify the capabilities inherent in such media. Flemming et al. (1997) argue that interaction metaphors inherited from past practice with manual drawing (in their case, the T-square metaphor) may hinder users from finding and using commands that have no analogue in manual techniques. In terms of task environment, the behaviour observed is partly inherited from prior media and partly influenced by the digital media commands and structure. Bilda and Demirkan (2003) explain the differences between manual and digital media in terms of media differences (CAD does not provide doodling, diagramming and pencil gesture), between the relative state of development of the media (sketching is mature, CAD operations are primitive), and in subject relative experience with the two media forms (all subjects were nearly novice CAD users). There is thus little theory that helps predict how change to design media results in change to design. Further, for a central part of design work, that is, exploring for alternatives (“search in a problem space” in Simon’s terms), both exemplary systems and guiding theory are in particularly short supply.

We are concerned here with designers using alternatives in their work, particularly with creating new tools that help designers create and understand alternatives. If Simon’s ant analogy holds, we should be able to observe different patterns in using alternatives across design media. The most widely accepted general pattern for expert designers using alternatives describes a general-to-concrete hierarchy of design problems with a predominant breadth-first exploration at each level of the hierarchy, followed by a depth-first exploration of one or more chosen alternatives. This breadth then depth pattern may be repeated at any hierarchical level. This process goes by several names, for example, Akin’s (2001) depth-and-breadth-first search and Fricke’s (1996) stepwise design strategy and balanced search tactic. Few studies, notably Fricke (1996), though, have been conducted over time frames sufficient to reveal this pattern being played out through a design process. For example, Akin (1986) describes a study of a short time scale suitable for protocol analysis. Smith and Tjandra (1998) describe exercises of “a few hours”. Goldschmidt (1991) studied episodes of “one to two hours”. Further, the design media used in most studies are manual.

3 Working with Alternatives

It is beyond argument that designers work with alternatives.

Every idea that is a true idea has a form, and is capable of many forms. The variety of forms of which it is capable determines the value of the idea.

—Frank Lloyd Wright

As Wright implies, exploring a space of possibilities is central, indeed essential, to many kinds of complex work. Accounts of such exploration have occurred in the literature for a long time, with perhaps the first thorough systematic treatment being that of Newell and Simon’s Human Information Processing System (HIPS) (Newell and Simon 1972). HIPS describes human problem-solving action as being search in a problem space, largely constrained by a task environment. In turn, the task environment almost always includes external media with which people store problem configurations. In design, HIPS became a principal research concept, around which arose accounts of designer action (Akin 1986; Cross 2004, 2001, 2008), the use of heuristic search as a computational strategy for solving design problems (Eastman 1973; Pearl 1984; Heisserman 1994; Flemming et al. 1992), and direct use as a concept and object in systems that aim to support design work (Chandrasekaran 1990; Woodbury and Burrow 2006). The idea of a space of designs figures large in the significant literature on shape and spatial grammars. In visual analytics, the term analysis of competing hypotheses describes a key phase in analytic workflows (Heuer 1999). The first step in such an analysis is to identify all potential hypotheses, though this is one of the least specific aspects of the method. Relatively recently, the computer science community, and HCI in particular, has published a number of perspectives, system and evaluations. For example, Shneiderman et al.’s (2000, 2006) argument for supporting exploration and providing rich history-keeping appears to have provided key direction in this area. Each field has its own terminology and authors such as Lunzer and Hornbæk Lunzer and Hornb (2008) have coined now-accepted terms such as subjunctive interfaces. In the face of a Tower-of-Babel-like profusion of terms, we adopt the simple and, hopefully, neutral term alternatives to encompass the entire complex. When we write “alternatives” we refer to the objects representing designs in the general enterprise of exploring a space of possibilities.

While there is a significant literature confirming the usefulness of supporting alternatives in computer-aided design systems, there is yet but a small set of distinct ideas on how to do so. In systems used on a regular basis, support for alternatives may exist, but is always a secondary feature, typically aimed at making a small number of variations of a design. What has been called the Single State Document Model (Terry and Mynatt 2002) dominates CAD interface designs.

The Single State Document Model requires a document to be in one, and only one, state at any particular time, thereby imposing a serial, linear progression through a task that is at odds with the “messy”, highly iterative creative process.

—Terry and Mynatt (2002).

Thus, how people work (designing with and through alternatives) clashes with the media they use (based on the Single State Document Model). It follows that discovering and devising ways to support alternatives is an appropriate goal.

4 What is an Alternative?

But what is an alternative? Is the idea of an alternative constant across tools?

The diversity of the literature on supporting alternatives suggests that constructs to recognize or measure an alternative vary across design media. Theory concurs—a given task environment makes some things easy and others hard, and we can expect to see people employ the “easy” more frequently than the “hard”. As a practical matter, let us consider only those design alternatives that appear in an external medium—leaving out those inaccessible and empirically doubtful ones that are “in the head”. Then, an alternative is a “mark” on a physical medium or a symbol structure in a digital medium. Consider a few examples.

Figure 1 shows a sketch study “Study for the Trivulzio Equestrian Monument,” made by Leonardo da Vinci in 1508–10, with a total of four different revisions of the sketch, all drawn by the artist on a single sheet of paper (da Vinci 1508). Here, an alternative is, arguably, a composite of several sketch fragments, and the entire sketch “contains” or connotes multiple alternatives. It is likely more plausible to interpret the contents of the multiple sketch fragments less as specific proposals for the positions of heads and legs as more as suggestions for exploration as described by Buxton (2007, pp. 111–120). Nonetheless, we see here sets of superimposed sketch fragments juxtaposed with other such sets within a single overall composition. Clearly, this represents a distinct and media-dependent technique for representing alternatives.

Fig. 1
figure 1

Examples of alternatives “in the wild” in the field of visual arts: “Study for the Trivulzio Equestrian Monument” (da Vinci 1508) by Leonardo da Vinci. Different parts of this sketch feature multiple alternative outlines, with different positions and orientations (see the cutouts on the left side of the figure, from top to bottom): (1) different rider’s head positions, (2) multiple positions of the horse’s hind legs, and (3) multiple positions of the horse’s front legs

Full size image

Figure 2 shows, in a single sheet, a so-called “ideation study” for running shoes, with multiple options. Such pages can be produced in a very short time by skilled designers. In this example, alternatives are spatially juxtaposed, likely for the purpose of suggesting comparisons and recombination of features into new versions. Interspersed with overall sketches are details that elaborate in particular features of one or more alternatives. Though seldom reproduced in publications, designers also often use multiple layers of transparent paper to superimpose alternatives.

Fig. 2
figure 2

Reproduced with permission

A page of sketches exploring concepts for running shoes. Marc van Tichelen, designer.

Full size image

Cross (2011, page 17) and Lloyd and Snelders (2003) show a series of hand-drawn sketches on a single sheet of paper by Philippe Starck, produced in designing the “Juicy Salif” citrus squeezer design. The designer arrived at the final product design by working his way through a sequence of about thirty related sketches and ideas (the lemon, the squid, and the 60’s rocket/spaceship). Our main reason to include this example is that it can be interpreted as successive refinements of a single idea that was present from the outset. One idea emerged later: the spaceship. The sketch can thus be taken as a counter-argument against alternatives as is merely demonstrates design refinement. We offer two observations to this counter-argument. First, Starck devised the final design while being able to view and compare several alternatives at once, and then recombine elements of several promising designs into the final (and commercially successful) product design. Juxtaposition is relevant even in a process of successive refinement. Second, design refinement is a limit form of using alternatives. That decisions are progressively made does not obviate the use of (or need for) multiple alternatives in the supporting media. (Remember, we use the term “alternatives” in a deliberately inclusive sense to refer to distinct representations used in design.)

Digital media, not surprisingly, provides different examples of alternatives. Designers using systems bound by the Single State Document Model employ well-known adaptations such as using multiple versions of files, storing partial alternatives on separate layers or cutting and pasting entire alternatives into an overall spatial scene. When using representations that explicitly model multiple states, the differences become both richer and more nuanced. We include two archetypal examples here, from generative and parametric design, respectively.

The label “generative design” applies to a broad range of ideas and techniques, from which we highlight spatial grammar (we include both shape grammar and the wider class of grammars defined over spatial representations other than shapes).

Spatial grammars require a rule set and a starting shape, and produce a language of designs by recursively applying rules beginning from the starting point. Chains of such rule applications are called derivations or derivation sequences. Spatial grammars thus produce a space of representations, linked into derivations through rule applications. Both the language of designs and all interim productions (called sentential forms) are candidates for being considered as alternatives. All of these devices: rules, applications, derivations and (subsets) of the grammar language may be used to explain a particular shape grammar (as shown in (Koning and Eizenberg 1981, 2019). For instance, Figure 3 shows members of the language of designs implied by John Portman’s own house.

Fig. 3
figure 3

Figure reproduced with permission from Ligler and Economou (2019)

Selected members from the language of designs of a grammar based on John Portman’s own house.

Full size image

Parametric models produce variations through changing model inputs. Figure 4 shows that models so produced can be visually diverse, even though produced by “mere” parametric change. Arrays such as that shown in Figure 4 have a regular part of architectural presentations and publications in recent years.

Fig. 4
figure 4

Used with permission from Shireen et al. (2017)

Array of parametric alternatives produced from the same model.

Full size image

Both of these digital examples share the property that alternatives are somehow marked or signified by designer intention (as they must be in the manual examples because the designer made the effort to commit each alternative to “paper)”. In spatial grammar, it is typical to call out specific members of the language of a grammar as somehow representative or typical of the whole. It is quite uncommon to have a meaningful grammar with only a few members of its language. Parametric models can produce very large numbers of variations, not each of which is a meaningful alternative. Though the representation provides for these large numbers, interfaces mostly remain in the Single State Document Model. Designers are typically reduced to using ad-hoc devices for alternatives similar to those noted above.

We conclude that an alternative is something produced in a design representation and about which a designer cares enough to mark in some way. This latter point will certainly be critical in interface design.

5 Supporting Design Alternatives

Each tool provides its own means to record alternatives, for example, sketches employ superposition, copy and modify, rapid sketch creation; conventional CAD provides aggregation and manipulation commands; and parametric CAD provides model editing and parametric variation.

Once a tool is mastered, we should expect designers to do different things with each tool. Indeed, we should be surprised if actions are comparable across tools. For instance, there is no easy analogue to a parametric variation in a sketch, nor does parametric modelling support the rapid, plentiful ambiguity of sketches in Buxton’s sense (2007). It may even be that comparable actions across tools are the result of tool designers pursuing a skeuomorphic design strategy, for instance, the reproduction of sketching affordances that was a goal in many 1980s to 1990s CAD papers and even more recently in computer graphics Bae et al. (2009). Nor should we expect the notion of an alternative to be invariant across tools. For example, parametric variations may seem trivial as alternatives, but such variations of a model can yield dramatically different outcomes.

This expectation of difference flows into research. Given different tool affordances and capabilities, creating good constructs for the concept of a design alternative may prove elusive. While the simple act of moving a slider may generate hundreds of parametric variations in a few seconds, it may seem odd to put these on the same conceptual footing as a separately constructed pencil sketch of a design alternative. Or is it? Experts in parametric tools regularly employ a deferment strategy (Woodbury 2010, p. 43) to construct models that allow them to explore alternatives later. Thus, seemingly simple variations may be part of an explicit exploration strategy. We can expect that the signal for a design alternative will vary widely across design media.

Thus, researchers and tool developers in design alternatives face a dilemma. New and old tools do widely (and hopefully, wildly) different things. Returning to Simon’s ant, different tools form different task environments and thus may (and should, if tool developers succeed) change designer behaviour radically from tool to tool. One horn of the dilemma attempts to compare tool use across manual and digital media, for example, (Bilda and Demirkan 2003), arriving at conclusions that the media vary, but raising serious internal validity issues. The other horn of the dilemma abandons comparison between old and new, making it difficult to argue that progress is being made.

These deep issues of research validation aside, what are potentially useful system features for supporting design alternatives?

General frameworks and descriptive accounts give a basis for understanding alternatives in design, but our interest lies in empowering designers with new kinds of tools. Research on this issue proceeds by cases: exemplary systems, their evaluations and analyses based upon them. And the action is in the domains: a system has to do something specific, for instance, programming, data analysis or building design. This focus on domains may explain why there is no single literature on alternatives—publications are spread over disciplines with limited cross references. A major problem with domains is that they tend to invent lenses, that is, particular interface designs that persist across multiple cases. These lenses focus effort on a few interface ideas and features, largely excluding others from consideration. Thus, the alternatives literature largely reports on a few basic designs. As I have argued (Woodbury 2016), lenses strongly channel system designs to consider only limited aspects of the entire problem. Thus, one should largely look for ideas that transcend or abstract the lens from which they come. I have identified six principal lenses into which almost every reported system falls: grammar, history, version, representation, task and search. For example, the grammar lens takes the intellectual structure of a generative grammar as a machine over which to build an interface. It uses ideas such as rule, derivation and derivation path to structure the interface. I use lenses largely as a filter, to look for system features that either (or both) depart from the lens or occur across several lenses. I take these features as candidates for further design and development. At a primitive level, they include juxtaposition (putting alternatives side-by-side), superposition (combining alternatives into single views), rapid serial juxtaposition (commonly used in choice interfaces in digital games), abstracting into charts and graphs (from which can be built more complex tools such as small multiples and multi-dimensional Pareto charts), and semantic zoom (from glyph to full CAD model). Larger, composite features emerge. Parallel editing enables single editing actions to affect multiple alternatives (Zaman et al. 2015). Exploring a local search space supports reuse of past design decisions (Lunzer 1994; Woodbury et al. 2000; Woodbury 2010; Zaman et al. 2015). User-defined and -controlled collections of alternatives are a chief feature of our recent prototypes (Sanchez et al. 2012; Kolarić et al. 2014; Zaman et al. 2015; Woodbury et al. 2017; Kolarić et al. 2017; Mohiuddin et al. 2017). Recent work (Shireen et al. 2017) shows that users need and invent such collections when tasked with understanding and organizing large numbers of alternatives.

All of these ideas stand apart from the most frequent approach in the literature for using digital alternatives in design. Papers taking this stance employ a heuristic search algorithm to sample a typically informally specified design space and argue that the outputs of such a process are useful in design. There are many such papers, using, for example, evolutionary algorithms, simulated annealing, Pareto optimization, tabu search, dispersion sampling and (more recently) generative adversarial networks. I label research of this type as appealing to an oracle. In computer science, an oracle machine is a Turing machine augmented by a black box, an oracle, able to solve a decision problem in a single operation. In mythology, an oracle can divine the future, typically by appeal to a higher power. To appeal to an oracle is to accept what the oracle produces as useful and not to inquire about how it does its work. From a research perspective, oracles are attractive as low-hanging fruit: undertaking such a project follows an established path, can be done with modest effort and yields a demonstrable result. As research contributions though, two problems emerge. The first is that such works are incremental: they elaborate one particular approach to using alternatives in design, whereas the domain may need new approaches. The second problem is that the approach itself is flawed: it does not support design tasks as they are done in the wild.

Why not? I offer two arguments. The first is that designers design—they devise things to achieve goals. In the task of designing, a design’s role is to be critiqued and re-worked in response. Schön’s (1983) account of the reflective practitioner argues this invariant thoroughly. Thus, the almost inevitable fate of an oracular result is to be subjected to yet more design work. Bradner et al. state this well in their study of designers using optimization systems.

Professionals reported that the computed optimum was often used as the starting point for design exploration, not the end product.

—Bradner et al. (2014).

The second argument is that designers explain their work. In such explanations agency is important. “This is what we thought and did” presents a more credible argument than “This is what the algorithm told us”. Although facetiously expressed, the way designers explain and understand (explain to themselves) designs involves having worked with and through the design’s ideas over time. Both of these arguments imply a strong need for basic and practical knowledge of designing interactions and systems supporting design alternatives.

6 How Malleable Are Task Environments?

Clearly, changes in tools can have variable effects on task environments. A refinement of an existing tool is less likely to induce a major change in designer action than did, for instance, the introduction of data flow visual programming in parametric design. Are there limits to changing the task environment? For instance, if appealing to an oracle can produce appropriate designs, then designers will no longer search in a problem space; rather, they will consult an oracle and select from the results. It would appear that tools may be able to change the fundamental nature of human problem solving and thus design. I would suggest that there are limits to such change, and further, that there is a hierarchy of task environment features that is hard to change at the bottom and more easy at the top. In other words, there are both variants and invariants in the situation. But what are these? One source for such is the design situation itself, particularly the size of design space, the structure of design problems and the role of knowledge.

Design spaces are VAST (in Dennett’s terms Dennett (1995)), that is, they will defeat any attempt at definitive enumeration. As a consequence, designers (and oracles) satisfice rather than optimize Simon (1956). A likely invariant that arises is a need to record, organize and visualize multiple potentially satisficing solutions as they arise. Adding to this need for multiplicity is that designs are seldom judged against a single criterion; thus, a design medium should be able to compare alternatives with multiple criteria. Likely a major invariant is that satisficing and multiple criteria act in concert to demand that oracles be wrapped in direct interaction. Since designers seldom take oracular results as final, whatever an oracle produces must be subject to the same interaction as any other design.

The word “design” labels many processes and thus structures across many domains. At the risk of implying a false spectrum, a design problem may be sufficiently well specified that hill-climbing strategies against fixed goals will reach satisficing solutions. Another design problem may be ill-structured or wicked; it may defy full or precise definition as the problem varies depending on the solution proposed. Indeed, there exists a significant literature that characterizes design problems by type; it suffices here to note that design processes and thus useful tools differ across problems and design domains. We should not expect a universal nor small set of operators for creating and visualizing design alternatives.

Designers use several forms of knowledge varying from formal or articulate (able to be explicitly specified) to tacit (incompletely codified, often embodied, known through experience). While formal knowledge can be directly codified in a design medium (for example, output parameters in a parametric model), tacit knowledge must be recognized. A task environment for alternatives must thus support multiple coordinated views, some expressing formal design knowledge and others providing general views through which tacit concepts, patterns and features can be seen.

The design medium itself provides variants in task environments. For example, manual sketching, image processing systems and parametric models are all useful design media, but each suggests and supports very different operations and visualizations. We should not expect that what works in one medium to transfer much to any other. Rather those who seek to support alternatives in a particular design medium would do well to immerse themselves in what designers actually do with that medium, singly and in concert with other media.

In summary, we know designers work with alternatives for which current systems provide impoverished support. We should expect such support to be specific to the design media (e.g., parametric models) being supported, and to be used by designers in ways both specific to the media and unexpected. While several system features seem promising and some have been reasonably well tested, the domain is young. There is much still to be learned about interactive design space exploration.