Keywords

Some early psychologists took the position that only external, observable behaviour could be measured and hence inform models and theories of learning and behaviour. In its most radical presentation (e.g., Skinner 1935; Watson 1913) the behaviourist perspective rejected the idea that human beings are autonomous agents and instead cast them as biological organisms whose behaviour was entirely driven by external stimuli and measurable environmental contingencies. By way of consequence, the behaviourists’ epistemological approach was to alter the environment to observe the consequence in behaviour and uncover stimulus-response relationships. The richness of human capabilities in general, however, soon made this position untenable. Some scholars retained the methodological tenets of behaviourism but rejected the idea that behavior was always purposeless and reducible to automatic responses to environmental cues (e.g., Tolman 1932, 1948). Others studied human memory (Bartlett 1995/1932), control systems (Craik 1948) or attention (Broadbent 1957) and pioneered a theoretical vocabulary that shaped the development of cognitive science in the second half of the 20th Century. In the words of George Miller, one of the early contributors to the so-called “cognitive revolution ”: “The grammatical rules that govern phrases and sentences are not behaviour” (Miller 2003, p. 141). In the work that followed, it was understood and accepted that the workings of the human mind could not be reduced to mere associations between events taking place “outside” the mind and behavioural responses in reaction to such events. Stimuli were recast as information inputs , and responses as the output of the mind, which itself took the centre stage as the processor of information. As cognitive psychologists sought to break in the mind’s “black box” they effectively took over from behaviourists, with a strong implicit assumption about how cognition enfolds: namely, people receive information , process it, and produce a response such as an answer to a problem, a judgement, or a choice.

Somehow, however, as the behaviourist perspective was overthrown, with it went the proverbial baby. By focusing on the workings of the mind, sandwiched between the external stimulus and the resulting behaviour (Hurley 2001), cognitivists put a disproportionate emphasis on the head, with its mental processes, operations and computations at the expense of behaviour and its “situatedness”. Cognition is studied in barren environments, where passive thinking subjects are conceived as mere consumers of information devoid of arms or hands. The issue with this approach is perhaps not immediately obvious: if a cognitive psychologist assumes that cognition is nothing but a mental process, then requiring her research participants to rely almost solely on their mind to carry out cognitive tasks seems reasonable enough. This state of affairs is further reinforced by the assumption that adult cognitive operations are ultimately and necessarily formal, involving operations on mental propositions rather than the messy physicality of the world (Inhelder and Piaget 2013). Effectively, while behaviourists reduced the workings of the mind associations between stimuli and responses, cognitivists are reducing them to mental operations. Both conceptions are wanting. Beyond adolescence, requiring anything other than your head to think is thought of as lazy, cheating, or at the very least appears to be a sign of poor cognitive functioning, mental disability or cognitive aging. By contrast, in the remainder of this chapter, we show that we could better understand how people actually think by allowing them to manipulate information both in their mind through mental processes and in their immediate environment through hands-on manipulations. Such manipulations, we contend, make a difference to the way people think. We present the Systemic Thinking Model (SysTM, adapted from Vallée-Tourangeau et al. 2015b), which builds upon the classical information processing model of cognition to account for this difference. We also introduce two new concepts: the concept of cognitive interactivity to refer to the emergence of cognitive events from the meshing of mental processing with the transformative actions of a thinking agent on her immediate environment; and the concept of affordance pool to refer to a short term storage of action possibilities in working memory that we conceive as sitting alongside the classical visuo-spatial sketchpad for imagery and the phonological loop for sounds. We conclude by illustrating how SysTM can be used to derive new predictions and guide the study of cognitive interactivity.

The Classical Information-Processing Model of Cognition

The classical information -processing model (Baddeley 2012; Wickens and Carswell 2012) represents cognition as originating from a series of input -processing-output events: We see or hear, we think, we respond. Figure 7.1 summarises this model (adapted from Baddeley 2012). According to this model, information flows from the environment to the mind as it is perceived and temporarily stored in one of two limited capacity systems: speech and sound information is stored in a phonological loop module through vocal or subvocal rehearsal whereas visuospatial information is stored in a visual-spatial sketchpad. While it is held in these modules, information is manipulated in an episodic buffer under the control of a central executive module. The central executive module is responsible for focusing attention , dividing attention between stimuli, and switching between tasks (Baddeley 1996, 2012). The episodic buffer is assumed to act as a buffer store, where a limited number of multidimensional information chunks are consciously combined from the phonological loop, the visuospatial sketchpad, and long term memory storage. This model assumes two “flows” of information. In the bottom-up flow, immediate stimuli present in the environment populate the phonological loop and the visuospatial sketchpad. In the top-down flow, information contained in semantic and episodic long term memory shapes the information stored, represented or processed. In all cases, information and mental processing all converge in the inner episodic buffer, from which the final response or cognitive outcome is produced. Taken together, these four modules make up the working memory system which is often cast as the cornerstone for complex cognition such as people’s ability to reason with novel information, also known as fluid intelligence (Kane et al. 2005).

Fig. 7.1
figure 1

The classical information-processing model (adapted from Baddeley 2012)

Full size image

This view of cognition and mental processing assumes a deductive flow of information in cognition where actions and behaviours (e.g., motor and verbal responses) are the end product of the application of intuitive or deliberative cognitive processes to the mental representation of an initial sensory input . This theoretical assumption is implicit in the methodology typically used to study thinking and decision-making. In the gambling paradigm, for example, a thinking agent will first perceive a stimulus, most likely in the form of a written text summarising two options: “Imagine you can choose between two lotteries: Lottery A gives you a 1% chance of winning $320 and a 99% chance of winning nothing whereas Lottery B gives you a sure gain of $3. Which lottery do you want to play?” (Adapted from Hertwig et al. 2004). According to Expected Utility Theory (EUT), a rational decision-maker should prefer the risky option (Lottery A) as its expected value ($3.2) is greater than that of the sure option ($3 in Lottery B). The classical information processing view describes the cognitive process by which human agents may reach their decision. This process begins with the mental representation of the different options in a more or less accurate fashion in the thinker’s mind. For example, Kahneman and Tversky (1979) proposed that the preliminary cognitive processing of a choice problem will result in a simplified mental representation of the prospects, following editing operations such as rounding probabilities or outcomes. Next, thinking agents are assumed to apply evaluation operations to this internal representation as they mentally compute the weighted probabilities of gains and losses to compare the expected values of the prospects. Their final choice is assumed to be the option with the highest subjective expected value.

Within such a framework, good cognitive outcomes are attributed to adequate representations, accurate individual knowledge in long term memory , appropriate cognitive and attentional resources and motivational levels. Conversely, breakdown in performance is assumed to arise from inadequate mental representations, shortcomings in knowledge, cognitive busyness or depleted cognitive resources , and low motivational levels. Situational influences such as contextual aspects including the content of the problem (e.g., a gamble situated in a medical context where lives rather than money are at stake) or the perspective of the decision-maker (e.g., a patient deciding for himself or a doctor deciding for her patient) are acknowledged but only to the extent that they can affect the mental representation and mental processing of the problem (Wagenaar et al. 1988). Seemingly departing from this internalist conception of cognitive processing , some researchers have recently argued that although monetary gambles may share the properties of many real-world decisions, the way they are presented (as verbal descriptions stating all outcomes and their probabilities) may not reflect everyday life decisions where risks are not explicitly tabulated but instead need to be estimated from people’s past experience. Deciding from experience rather than from description was found to affect final outcomes, as demonstrated by the robust description-experience gap effect (Hertwig and Erev 2009). Decisions from description are based on a written summary similar to the lottery example presented above. By contrast, decisions from experience are based on the sequential experience of uncertain outcomes. In an experiential setting, people first sample through each lottery outcomes instead of being presented with a descriptive summary of the lotteries. In Lottery A, they would experience winning nothing on 99% of the trials and winning $320 on 1% of the trials. In Lottery B, they would experience winning $3 on each trial. The description-experience gap results in a reversal of risk preferences. When choosing from a description, decision-makers prefer the risky option even if its outcome has a low probability of occurring, seemingly overweighting the probability of a rare but desirable event. Conversely, when choosing from experience, ceteris paribus, they prefer the safe option, thus seemingly underweighting the probability of a rare but desirable event in the risky prospect.

Despite the fact that thinkers appears more active in decisions from experience (in as much as they are in control of how much information they acquire), researchers still rely on the classical analysis of information processing to account for the description-experience gap. Thus, in decisions from experience, the mind is assumed to continuously update its representation of the probability value, experience after experience. The description-experience gap is thereafter assumed to orginate in a defective mental processing of probabilities informing the final choice in decisions from experience. Whether these representations are distorted due to the limited sampling of rare events, overweighting of late observations, or both, remains a debated issue (Hertwig et al. 2004; Hertwig and Erev 2009).

Besides representational issues, errors of judgement are also attributed to poor motivation to engage in effortful mental processing (Kool et al. 2010). Consider, for example, the bat and ball problem puzzle (Frederick 2005):

A bat and ball cost $1.10.

The bat costs one dollar more than the ball.

How much does the ball cost?

This problem invites a quick answer (10¢) which is incorrect (the bat would cost $1 more—that’s $1.10—and both would cost $1.20, not $1.10). The correct answer is 5¢. Kahneman (2012) writes “people who say 10¢ appear to be ardent followers of the law of least effort. People who avoid that answer appear to have active minds.” … “Failing these mini tests appears to be, at least to some extent, a matter of insufficient motivation, not trying hard enough … Those who avoid the sin of intellectual sloth could be called ‘engaged.’ They are more alert, more intellectually active, less willing to be satisfied with superficially attractive answers, more skeptical about their intuitions ” (p. 45). To sum up, these examples illustrate how reasoning and decision-making performance is attributed to the characteristics of the thinking subject, the quality of her mental representation of the problem and the quality of her mental processing of this representation. In the next section, we propose an alternative perspective where performance is, instead, conceived as an emergent feature of a distributed cognitive system (Hollan et al. 2000).

The Classical Information-Processing Model Is an Inadequate Model of Thinking and Deciding

The classical information -processing model is a very compelling account which is supported by a variety of classic empirical phenomena. For example, people have more difficulty remembering series of items in the appropriate order when they “sound alike” as in man, mad, can, cap, map compared to series of items which have different speech sounds as in pen, day, cow, bar, rig; a phenomenon called the phonological similarity effect (Baddeley 1966). There is also evidence to show that people’s recall of digits presented visually can be hindered when they simultaneously have to ignore irrelevant speech sounds (Salamé and Baddeley 1986). Altogether, these findings point to the fact that memory traces are maintained through (subvocal) speech rehearsal. This mechanism is further evidenced by studies showing that when speech rehearsal is actively prevented through articulatory suppression (e.g., by asking participants to repeat “the” aloud while performing a task), the phonological similarity effect and the irrelevant speech effect disappear. In those instances, performance is simply diminished, whether or not items have similar speech sounds and whether or not irrelevant speech sounds can be heard in the background (Baddeley et al. 1975).

These processes impact cognitive outcomes beyond their impact on memory performance. For example, articulatory suppression is known to also impede mental arithmetic performance, especially when participants use counting strategies (Hecht 2002). Similarly, spatial reasoning ability is related to working memory visuospatial span, suggesting that the visuospatial sketchpad is involved in visuospatial reasoning tasks testing spatial visualisation. For example, in the dot-matrix task, participants are asked to remember a dot location in a grid while verifying the accuracy of equations. Performance on this task is strongly correlated with performance on the Paper Folding Test, a spatial visualisation test where participants are asked to mentally fold a piece of paper, imagine a hole punched through the folded paper, and infer what the unfolded paper would look like by selecting one of five alternatives (Miyake et al. 2001).

So far, these results suggest that mental processes (such as rote memorisation, mental arithmetic, or mental visuospatial reasoning) are depending on mental resources. However, the classical assumption that responses are inferred or deduced from mental processing is implicitly bounded by methodological individualism (Weber 1978/1922). This doctrine assumes that behaviour necessarily originates from the intentional states that motivate individual agents. This assumption in turn imposes a top-down hierarchy where individuals’ thoughts precede their behaviour (Knappett 2005). If the individual agent is the ontological locus of cognition, then cognitive performance must be a reflection of the agent’s mental capacities and abilities (Malafouris 2013). As we mentioned earlier, context is assumed to (e.g., Wagenaar et al. 1988) but only to the extent that different contexts may cue different representations and call attention to different information-processing strategies. Methodological individualism also constraints theoretical accounts which are putting a stronger emphasis on the role of the environment in thinking . Thus, the ecological rationality approach to cognition, for example, posits the cognitive machinery of the mind is best understood by examining how the mind exploits its immediate environment (Brighton and Todd 2009). As such, the ecological rationality approach falls well within the remits of the classical information processing model as it views cognition as emerging from mental computational mechanisms , albeit ones that have evolved to use so-called “natural” information inputs such as natural frequencies (e.g., Gigerenzer and Hoffrage 1995).

In line with the implicit methodological individualism that underpins the classical information-processing model, the research procedures commonly used to study complex cognition typically place participants in a passive information acquisition role where an experimenter controls the information they receive through auditory or visual channels. Experimenters then examine how a change in stimuli impacts reaction time or performance. Whereas many cognitive phenomena can and have been studied from this perspective, we contend that it offers a procrustean framework for studying complex cognition. In typical cognitive psychology experiments, stimuli are presented with paper-and-pencil questionnaires or computer display screens. This procedure allows experimenters to exert stringent control on how information is presented to participants. This strict control, however, obscures the potentially constitutive role of action and of the immediate environment in thinking and deciding. Since participants are, de facto, barred from handling, manipulating, and rearranging the information presented to them, methodological individualism reduces people to passive information processors who are modelled as if they typically remember, think, reason, solve problems and make decisions with their hands, as it were, tied behind their back and their eyes closed (see Hutchins 1995, Chap. 9 for a similar argument).

In other words, the classical information-processing model is adequate as long as one seeks to account for cognition arising from an information processing pathway where a unique final action , response, or behaviour is deduced from the processing of a mental representation . An important shortcoming of this model, however, is that it precludes the conception of the thinking process as evolving through a series of actions which will inform and transform a concurrent mental processing of the task information . The classical information-processing model also lacks a different kind of information processing loop, namely one where the next action, response, or behaviour is induced from the action possibilities offered by the immediate environment rather than deduced from mental processing. Yet, as we will discuss in the next section, there is accumulating evidence to show that thinkers sometimes act before they think when they can interact with their immediate environment. Far from being mere noise, those unplanned actions can transform mental processing and augment cognitive performance .

Rehabilitating the Hands: The Role of Cognitive Interactivity in Thinking and Deciding

People and things mutually influence each others. The key to overcome the vintage divide between the cognitivist and behaviourist conception of human behaviour is to move away from the mind-as-processor versus behaviour-as-reaction divide and instead focus on the nexus between mind and behaviour, which we call cognitive interactivity. Cognitive interactivity refers to the meshed network of reciprocal causations between an agent’s mental processing and the transformative actions she applies to her immediate environment to achieve a cognitive result (see also Steffensen 2013; Steffensen et al. 2016). To embrace the central role of cognitive interactivity in cognition is to recognise that cognitive and behavioural outputs emerge, not from mental processing alone, but from the interweaving of mental and behavioural processes. The extent of such interactivity, however, depends on the affordances or action possibilities inherent in a given environment or situation. In interactive environments, thinkers become thinking agents, free to manipulate and fiddle with the information available in front of them. Interactivity results in physical changes in the environment in a way that best suits agents’ thinking needs and flow . This, in turn, appears to facilitate information processing , and in many instances, to improve performance .

Different task settings will vary in the level of cognitive interactivity they foster. Cognitive interactivity may be nonexistent, limited to unobtrusive gestures , or unbounded. It is non-existent in the vast majority of experimental cognitive paradigms, either because agents do not feel entitled to impact their environment or because the environment is static and immutable (or both). In those instances, cognition is reduced to its barest form and limited to mental information processing. This type of cognition typically occurs in cognitive neuroscience research where participants are instructed not to scratch their head, swallow, open their mouth, yawn, inhale deeply, or shift their posture as any of these would produce spurious but irremediable noise in the neuroimaging data (e.g., see http://psychology.msu.edu/liulab/subj_info.html). Next, at the minimal level of interactivity, agents do not act to change their immediate environment, but they do act to support their own thinking . This is evidenced by the role of gesturing in memory retrieval and thinking (Goldin-Meadow 1999; Novack and Goldin-Meadow 2015). When free to gesture, individuals speak more fluently when their speech describes spatial elements, suggesting that gesturing can facilitate access to people’s spatial lexicon (Rauscher et al. 1996). Gesturing can also lighten the cognitive load: individuals who were not allowed to gesture while explaining how they solved a mathematical problem also exhibited poorer recall on an interfering memory task compared to individuals who were allowed to gesture. This suggests that gesturing offloads some of the cognitive costs involved in the spoken task (Goldin-Meadow et al. 2001). In these instances, however, while gestures may offer a scaffolding support for thinking and deciding, they only represent a proto-level of cognitive interactivity .Footnote 1 The next level involves cognitive interactivity proper. When people can act upon and transform their immediate environment, their performance leaps up. A classic example is the finding that expert Tetris players begin rotating zoids so early (before 100 ms have elapsed) that they cannot reliably guess their shape (Kirsh and Maglio 1994). Such early rotations are also more frequent than what one would expect by chance alone so they are not random but they occur too early to be the enaction of a mental plan. Instead they contribute to performance by saving mental rotation effort and facilitating the identification of the zoid’s type as well as the process of matching its contour with the existing mass (Kirsh and Maglio 1994).

Research from our own lab has also accumulated evidence that agents ’ performance improves if they can perform actions while attempting to solve a problem. For example, we presented participants with a Bayesian inference problem, a complex statistical reasoning task requiring them to revise the probability that a hypothesis is true, in the light of new data (Vallée-Tourangeau et al. 2015b). Half of the participants were presented with a standard pen-and-paper task where they could offload the content of their thought on paper but could not perform any action on the information content itself to support their thinking . The remaining half were presented with the same problem accompanied by a stack of playing cards, with each card representing some of the statistical information in the problem. In this setting, information was no longer frozen in a verbal description. It was distributed across the cards, thus offering participants the opportunity to mold their own external information layout as they progressed through the task. This always led to improved performance , above and beyond simply visualising the information through the cards: performance improved dramatically when participants were allowed to manipulate the cards and took it upon themselves to spend time actively changing the information layout by sorting and rearranging cards (Vallée-Tourangeau et al. 2015b). In other studies, we found that increasing manipulability of the environment facilitated insight (Vallée-Tourangeau et al. 2015a; Weller et al. 2011; see also Fioratou and Cowley 2009), improved efficiency in mental arithmetic tasks (Vallée-Tourangeau 2013), facilitated learning in transformation problems (Guthrie et al. 2015), and enhanced word production in a Scrabble-like task (Vallée-Tourangeau and Wrightman 2010; see also Fleming and Maglio 2015; Maglio et al. 1999). These findings highlight the importance of cognitive interactivity: to understand cognition, that is, how cognitive events emerge, we need not only to understand how agents mentally process and represent problems but also how the actions they perform blend with their mental processes while they think the problem through.

SysTM: A Systemic Thinking Model of Cognition

The classical model of information-processing is ill-equipped to explain how higher cognitive results may emerge from cognitive interactivity . To account for, and further study, the role of cognitive interactivity on various cognitive operations such as mental arithmetic, insight problem -solving, or decision-making, we propose a dual-flow systemic model of cognition , where cognitive results arise from one of two processing loops: a deductive and an inductive loop (see Fig. 7.2). We characterize our thinking model as systemic to underscore our view that cognition emerges from a complex set of entities (human and non-human) that form an interconnected network of reciprocal causations. SysTM is intended as a framework for understanding and studying how information processing may be distributed across mental and material structures when an individual agent engages in a thinking task over a relatively short timeframe.Footnote 2 In SysTM, the human-environment interface separating the mental from the physical has been purposely removed to signal that the physical processing and the physical apparatus become an integral part of the cognitive substrate from which new thoughts and new actions may emerge.

Fig. 7.2
figure 2

The Systemic Thinking Model (SysTM)

Full size image

The Deductive Processing Loop

In the deductive processing loop, the next action or response is deduced from the mental processing of a representation , akin to what is generally assumed in the classical information processing model (see Fig. 7.1). A key difference between this former model and SysTM is that, unlike the classical view, SysTM does not assume that this process follows a linear pathway from the initial stimulus to an intermediate stage of mental processing before reaching the final response. Instead SysTM assumes that processing evolves through a closed loop that includes both mental and physical processing of the information available. This looping implies that, rather than being temporarily stored, the perception of a stimulus activates sets of cues held in long-term memory (Ericsson and Kintsch 1995). Verbal cues becomes activated through the phonological loop and visuo-spatial cues, through the visuo-spatial sketchpad. SysTM also features an additional short-term activation component, the affordance pool, where motor-action sequences are temporarily activated. In a deductive loop, the activated cue sets shape a mental representation which is processed in the episodic buffer and ultimately directs the physical processing of the stimulus, provided that the environment is amenable to physical processing and that thinkers are empowered to act upon the affordances of their immediate environment. When engaged in a deductive processing loop, people execute planned motor actions on the material presentation of the stimulus. These actions are constitutive of thinking because they contribute to transforming the immediate perceptual field when they result in an alteration of the physical apparatus in their immediate environment. This altered material presentation offers new percepts to the mind , potentially attracting attention to new features of the informational landscape , thus reshaping mental representations and offering further opportunities for representation updating and mental processing, followed by further physical processing and so on.

The Inductive Processing Loop

SysTM does not assume that all information processing takes place in private thoughts but instead posits that some information processing can be offloaded onto the thinking agent’s immediate environment when it features a manipulable information layout. In the deductive processing loop described above, such physical processing results from the execution of a mental plan formed through mental representing and processing. This processing pathway, however, is not sufficient to account for the emergence of cognitive results in interactive environments: not all actions need be carefully planned before they are executed. Agents may also act without a plan in their search for a fruitful physical information layout. To account for this type of external information processing, we propose that thinking agents’ physical processing will at times be driven by a direct perception of action possibilities.

Several scholars have pointed out that stimuli may guide action in the absence of top-down (mental) categorization (Baber et al. 2014; Gibson 1979/1986; Greeno 1994; Norman 2002; Withagen and Chemero 2012). Some actions arise as individuals “follow materials” in a spatio-temporal trajectory (Ingold 2010; Vallée-Tourangeau and Vallée-Tourangeau 2014). Whereas the concept of direct perception has been proposed and defended before (Gibson 1979/1986; Wilson and Golonka 2013), it has yet to be integrated in a general cognitive framework. The doctrine that cognitive events may arise without mental processing of an inner representation is often summarily dismissed as a mere (behaviourist) heresy by cognitive psychologists interested in studying higher cognition (e.g., Fodor and Pylyshyn 1981; Gyr 1972) and cast as ill-suited to account for human higher cognitive processes (e.g., Shapiro 2011).

The issue, we contend, lies in the fact that proponents of the radical embodiement hypothesis often pitch direct perception as an alternative to traditional cognitive explanations and as an account of how perception works, in any situation (e.g., Wilson and Golonka 2013). It needs not be so. Freeing ourselves from the constraints of methodological individualism, we propose that actions are not always informed by mental processing and do not always result from a mental plan. Actions may also inform thinking before a plan is made or even before the informational content of a perceptual input is mentally represented and processed. The challenge in accounting for this possibility is to reconcile the view that perception nevertheless acts as informational input to the thinking agent while this input is neither mentally represented nor mentally processed.

The key to overcome this conceptual challenge lies in Norman’s (2013) notion of “perceivable affordances”. We conceive perceivable affordances as unmediated perceptions which inform the activities or actions that are possible within the individual’s immediate environment . This concept is readily accommodated by the affordance pool, SysTM’s third working memory component. Thus, perceived action possibilities temporarily activated in the affordance pool may compel the thinking agent to engage in an inductive processing loop, bypassing the need for representation and mental processing (see Fig. 7.2). People who are familiar with playing cards, for example, may immediately perceive that the cards afford picking up and sorting. They may engage in such physical processing without a mental plan of action. Similarly, the material presentation of the physical apparatus in the immediate environment may render a particular affordance highly salient for the thinking agent , in which case the affordance may be brought to light through the perceptual field in a bottom-up fashion. This implies that the direct perception of a given affordance may be influenced by past experiences. To account for this possibility, SysTM posits that motor-action sequences retrieved from procedural long-term memory may impact the affordances perceived in a given task, akin to the top-down mechanisms by which information retrieved from semantic and episodic long term memory might impact the information available in the phonological loop and visuospatial sketchpad components of working memory. Thus, when the thinking agent engages in an inductive processing loop, her actions are enacted via a direct path from the central executive to the motor executive, and eventually informed by information retrieved from procedural long-term memory (see Fig. 7.2).

Note that there is a conceptual ambiguity in the scope of affordances . For Gibson (1977, 1979/1986; see also Greeno 1994), affordances encompass all latent actions possibilities available in the environment , independently of the agent’s ability to perceive them; we call these latent affordances. For Norman (2013), the critical affordances are those that are visible to the agent and affordances become visible in the presence of a perceptible sign or signal indicating what can be done; we call these perceivable affordances. In both conceptions, the ontology of affordances is unarguably relational: action possibilities depend on the relationship between physical or digital artefacts and agents’ capabilities to interact with such artefacts. For example, a ball affords kicking as long as its size and weight are commensurate with the agent’s ability to kick it. To account for what a thinking agent may do in a given setting, however, we need to focus on those affordances that become visible to the agent. Still, Norman’s conception of perceivable affordances as depending only on the presence of appropriate signifiers in the immediate environment of the agent is wanting for our purpose. The process by which those signifiers are understood or translated as action possibilities remains under-specified. It conceals the potential top-down role of the agent’s procedural knowledge , reflecting her behavioural repertoire, and what action possibilities may or may not be perceived. The affordance pool provides a route to address these issues by offering the means to specify the process by which affordances become visible to the agent .

Cognitive Interactivity

Taken together, the concepts of deductive and inductive processing loops provide a new framework to study how cognitive events may arise from cognitive interactivity where a thinking agent coordinates mental and physical resources to support her hypothesis testing, problem-solving or decision-making. As such, the systemic thinking model (SysTM) opens up a new agenda for research into higher cognition that transcends old debates: a key objective of the systemic approach is to understand how cognitive interactivity may produce cognitive events where the situated agent-environment ecosystem achieves a cognitive result (see also Steffensen et al. 2016). The systemic perspective presupposes that cognitive interactivity operates in a non-linear, but temporally situated fashion. Thinking and deciding need not to always follow a purely deductive processing pathway, as implied by the classical information processing model whereby perception precedes mental representation and mental processing which in turn directs physical processing. Conversely, thinking and deciding cannot be accounted for by a radical embodiment perspective where behaviour always emerges from the coupling of bodies with their specific environment without recourse to internal control structures including mental representation or mental planning (Wilson and Golonka 2013). Instead of pitching both frameworks against each other, our model reconciles these approaches by allowing either type of processing to take place through the spatio-temporal trajectory of cognitive interactivity. An agent may plan an action on her immediate environment before enacting it and the resulting change in the informational landscape may afford another action that does not require a plan. In other words, cognitive events may arise through mental or physical processing or both as the cognitive agent engages in a series of inductive and deductive loops in any given order.

Besides providing a new theoretical framework for understanding how cognitive results may emerge from information processing, the systemic thinking model (SysTM) can also serve as a guiding framework for research seeking to further our understanding of the complementary roles of mental aptitudes and environmental affordances in cognition . Mental aptitudes can be defined as the cognitive operations which are possible within the agent’s mind . These may be stable (i.e., cognitive capacity such as working memory span, cognitive tools such as mental scripts and schemas ) or transient (e.g., motivational and affective states). Mental aptitudes have long been studied by cognitive psychologists. As we have seen above, however, they have mostly been studied in dire settings stripped of most, if not all, environment affordances, thus reducing agents to thinkers paralysed by the dictates of methodological individualism, in the image of Rodin’s Le Penseur. By contrast, the systemic thinking model (SysTM) highlights the need for studying how cognition emerges when agents can make full use of their mind and hands in settings rich of environmental affordances . Environmental affordances may be physical since actions possibilities depend on the manipulability of the physical informational layout, as well as social, since norms or vicarious influences may also constrain or foster action possibilities.

Different mental representations and mental aptitudes make different environmental affordances salient. Identified environmental affordances, in turn, govern the physical actions that the agent will implement to transform the material presentation of the information . This physical transformation will inform the mental re-representation of the task and guide future mental operations and physical actions, and so on. As Kirsh (2010, p. 441) puts it, such dynamic cognitive interactivity “allow us to think the previously unthinkable.” Not only does cognitive interactivity saves memory and cognitive resources , but it also increases both the effectiveness and efficiency of cognitive processes . These processes become more effective as cognitive interactivity allows for more precise computations. They become more efficient as cognitive interactivity reduces errors and increases processing speed. In other words, adopting a systemic view of thinking calls for a re-assessment of the executive functions of working memory capacity such as attention allocation and switching. In a distributed cognitive system , executive functioning is no longer bound by the cognitive aptitudes of a thinking subject. Instead it is defined by an extended processing capacity that includes both the mental processing capabilities of a thinking agent and her physical processing abilities underpinned by the affordances she perceives in her immediate environment .

Key issues include better understanding of how environmental affordances are perceived and acted upon, both when behaviour results from careful mental planning and when it arises from direct perception. SysTM and a commitment to engineering thinking and deciding in interactive laboratory environments has the potential to usher in data that will cast a different light on models of problem solving [e.g., Ohlsson’s (2011) redistribution theory; Weisberg’s (2015) integrated framework], the role of working memory—and IQ —in thinking and deciding (e.g., Davidson 1995; Stanovich and West 1999), and on the cognitive abilities of the reasoners themselves (such as Bayesian reasoning, Vallée-Tourangeau et al. 2015a).

Qualitative analyses of agents solving problems in interactive environments along with verbal protocols and eye-tracking data may reveal the extent to which actions reflect the implementation of a plan and which don’t. Likewise, much remains to be learnt about the relative proportions of different types of actions along different segments of the spatio-temporal trajectory that leads to a cognitive result. For example, Weisberg’s (2015), Fleck and Weisberg (2013) recent integrated framework on insight problem solving is based on experimental data generated from a series of insight problems, some of which are presented with manipulable artefacts , some not. Participants’ performance is substantially different with interactivity : restructuring is much more likely (Vallée-Tourangeau 2014). In turn, measures of working memory capacity in participants working on a difficult insight problem (the 17 animal problem, adapted from Metcalfe and Weibe 1987), predict none of the variance in participants’ performance. Rather, the level of interactivity afforded by the problem environment alone explains whether participants can solve this problem (Vallée-Tourangeau et al. 2016).

One prediction from SysTM is that loading internal components of the agent’s working memory would be particularly detrimental to task performance in conditions where physical processing is limited but not where the environment is rich of affordances . Mental arithmetic, for example, is known to involve the phonological loop to store intermediate values and the episodic buffer to carry out operations (Fürst and Hitch 2000). SysTM predicts that as the phonological loop is overloaded, the thinking agent may switch to an inductive processing pathway and store intermediate values in the physical world instead, provided that the environment affords such a switch.

We tested this prediction in a recent experiment. Participants completed a series of additions involving 11 single digit numbers. Although the task certainly does not exceed the mental arithmetic skills of numerate undergraduate participants, requiring them to complete it with their hands down neverthelsess resulted in systematic calculation errors. Errors rates are more dramatic when participants must also engage in articulatory suppression, repeating ‘the’ continuously as they work on the sums. However, increasing interactivity by presenting the task as sets of manipulable number tokens reduces both the impact of articulatory suppression and the magnitude of calculation errors (Vallée-Tourangeau et al. 2016). From the SysTM perspective, the working memory of the system configured by the coupling of an agent with numbered tokens and a flat surface to (re)arrange them, can better absorb the internal resource depletion caused by articulatory suppression. Equally interesting, independent measures of the participants’ level of mathematics anxiety revealed that the impact of suppression on calculation error was significantly moderated by maths anxiety—the higher the level of maths anxiety, the larger the impact of suppression—but only in the task condition that did not afford interactivity; in the condition where participants could manipulate the physical presentation of the sum, maths anxiety did not predict performance . This suggests that the cognitive resources of the system, rather than those of the agent alone, can augment arithmetic performance in the low interactivity condition, even among participants who are particularly prone to errors because of anxious thoughts about maths.

Future research could make use of SysTM to further explore the determinants of productive cognitive interactivity. For example, in our study of interactive Bayesian reasoning we found that only actions that involve a restructuration of the information subsequently promoted successful performance whereas actions that made minimal changes to the perceptual layout were ineffective in fostering a path to solution. Likewise, the characteristics of the affordance pool which we introduced to account for unplanned actions in thinking remain to be specified. We have found that loading the phonological loop impedes mental processing but not physical processing. It remains to be established whether loading the affordance pool (e.g., by asking people to press a pedal repeatedly while solving an interactive mental arithmetic task) would have a similar detrimental effect on performance in the presence of tokens.

Concluding Remarks

In this chapter, we reviewed the classical information processing model and argued that it offered a limited and limiting view of human cognition , bounded by methodological individualism. Next we reviewed empirical evidence pointing to the crucial role interactivity can play in explaining how individuals think. We proposed a new model of information processing —SysTM—aimed at addressing the shortcomings of the classical information processing model and to offer a framework for studying cognition that is free of the constraints of methodological individualism. At the core of SysTM lies the concept of cognitive interactivity, where cognition is conceived as emerging from the close coupling of mental aptitudes and environmental affordances . Finally, we derived specific predictions from SysTM to illustrate how this model could provide a guiding framework for studying cognitive interactivity in the future.

SysTM aims to account for higher cognitive operations including how an agent makes inferences, solves problems and makes decisions. It conceives thinking as a cognitive process that evolves in time and space and results in a new cognitive event (i.e., a new inference, a solution to a problem, a choice). As such, it provides a framework for studying one kind of distribution of cognitive process, namely that where mental and material structures are coordinated by an agent thinking on her own during a relatively short time episode. SysTM also offers a platform for overcoming the theoretical stalemate created by seemingly antagonistic views of cognition : namely the classical cognitivist view and the radical embodied view. Proponents of the former view argue that cognitive events emerge solely from mediated perception and computational processes that are executed mentally. Proponents of the radical embodied cognition view argue, instead, that cognition originates from direct perception and physical coordination. Instead of taking position for one of these qualitatively incompatible views of cognition, SysTM conceives mediated perception, direct perception, mental computations , and physical coordination of external information structures as different kind of processes, which all form part of the thinking agent’s arsenal for addressing a cognitive task.

On a related note, in SysTM, attention control is conceived as a central executive process rather than a skill supporting the fine-grained sensorimotor coordination which can be observed in expert tool use (Baber et al. 2014). Again, these two conceptions of attention are not incompatible if we consider that expert tool use, on the one hand, and the type of cognitive tasks we have reviewed in this chapter, on the other hand, sit at the opposite end of a spectrum of cognitive activities people may engage in. At one end of the spectrum, we find learnt procedural routines and low levels of cognitive challenges. Skilled practice falls within that category. In expert tool use, attention will be primarily driven by the perception of affordances and SysTM posits that information processing will therefore primarily loop inductively by bypassing the mental representation and processing stage, and instead proceeds through a series of action -perception cycles. As mentioned earlier, the type of affordances cued in the affordance pool may be shaped by procedural long-term memory in a top-down fashion. So expertise could be reflected in a richer and perhaps more elaborate motor-action sequences readily activated, such as when the expert blacksmith engages in his or her craft (Wynn and Coolidge 2014). Simple cognitive tasks (e.g., “2 + 2 = ?”) also fall within the category of learnt procedural routines that don’t represent a cognitive challenge as such. Here, attention will be primarily driven by the perception of speech and sounds information via the phonological loop and SysTM posits this will be enough to retrieve the answer from long-term memory, thus also bypassing the mental representation and processing stage. For example, “2 + 2” is such a frequently encountered linguistic (verbal and written) expression that it no longer needs to be analysed in terms of its elements, but instead cues a direct association with another linguistic element, namely “4”, that is reflexively produced like a conditioned response. At the other end of the spectrum, we find cognitive tasks that are both highly novel and highly challenging. Non-routine reasoning , problem-solving and deciding are typical examples of such cognitive tasks. When faced with such a task, SysTM posits that the thinking agent will engage in an “unplanned cognitive trajectory ” (Steffensen 2013) that will involve a series of inductive and deductive processing loops. Whereas each cognitive trajectory may be unique, the balance between inductive and deductive processing loops should depend on the affordance landscape in the thinking agent’s immediate environment (i.e., the Gibsonian latent affordances), the possible actions perceived by the agent (i.e., the Normanian perceivable affordances), her ability to process and plan her actions mentally, as well as her capacity to exert control on where to focus her attention .

The systemic thinking model (SysTM) substitutes methodological individualism for cognitive interactivism and cognitive interactivity is conceived as the core component of human cognition as it naturally unfolds, whether it is strategic or opportunistic in capitalising on fruitful but unpremeditated human-environment interactions. Under the cognitive interactivism assumption, thinking and decision-making are shaped by an interaction of inductive and deductive information processes that take place both internally, in the agent’s mind , and externally, in her immediate environment: the material presentation of the initial informational landscape constrains people’s representation of the task at hand. As mentioned before, SysTM was developped to model how higher cognition may be distributed across an agent and her immediate environment . In its current form, it is not fit to account for how cognition may be distributed across the members of a social group or across larger time episodes that would allow for the stabilization of knowledge and practice (Hutchins 2001), but future research may explore how it may be developed into a useful framework for studying those other kinds of distribued cognition as well. By putting cognitive interactivity at its core, SysTM not only offers the means to reconcile the vestigial chasm between a behaviourist approach (or the more recent radically embodied approach) and a cognitivist approach to understanding behaviour but also offers new avenues for investigating how people engage in higher cognitive processes .