Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The past decade has seen major advances in cognitive, affective, and social neuroscience that have the potential to revolutionize educational theories about learning, especially in technology-rich environments. In this chapter, we lay out two general, complementary findings that have emerged from neuroscience research on emotion and social processing, with the goal of beginning a dialog about the meaning of these findings for the design of emotionally responsive learning technologies. First, emotion and cognition are intertwined, and involve interplay between the body and mind. Second, social processing and learning happen in part by internalizing our subjective interpretations of other people’s beliefs, goals, feelings and actions, and vicariously experiencing these in some ways as if they were our own. Together, these two results from neuroscience could have important implications for the design of technologies for learning and teaching, because they suggest that (1) social emotions and learning are intimately subjective processes, heavily influenced by social and cultural experience and individual predispositions and preferences; (2) affective responses involve a dynamic interplay between bottom-up and top-down processing. We conclude with a prospective discussion of the implications for affective computational systems design.

Humans and Computers Interacting: Reframing the Digital Learning Experience as a Social Encounter

We begin with a familiar scenario: a group of high school students are sitting in a computer classroom. Some are slumped over their desks or staring aimlessly out of the window. Others, though, appear to be highly engaged in the task, working in pairs or alone and obviously absorbed in the digital environment. What accounts for the differences between these groups? How is it that some students may find a digital learning environment engaging and useful, while others may wonder, “why am I doing this?”

Both social-affective neuroscientists and learning technology designers are interested in scenarios such as this one, and in explaining the motivation and learning differences between the two groups of students. However, while the neuroscientists would focus on the question of how neural systems enable some students to experience the digital classroom as a motivating environment and how both perception and learning are altered as a result, learning technology designers would focus on the tools and setup that characterize the digital environment as their starting point, asking “what technology designs promote more efficient and effective learning?”

In this chapter, we argue for a different, complementary approach – one that advocates that social-affective neuroscientists and digital learning designers meet in the middle. In this productive middle ground, we suggest, a new question emerges: “How could digital learning environments be designed more effectively if we were to consider digital learning as happening through a dynamic interaction between the person and the computer?” In this view, the use of a computer learning technology by a person would be akin to a social encounter between a mind and a machine. While there is a long tradition of studying mind–machine interfaces, our hope is that framing the problem in terms of the neurobiology of human social emotion may give technology designers a new perspective into their craft, by paving the way for a dialog with affective and social neuroscientists about what we can expect from social humans when they interact with each other or, by extension, when they interact with silicon.

Embodied Brains, Social Minds: The Neurobiology of Being Human

Think back to the atrocities committed on 9/11/2001. How do we know these actions were wrong? And why do most Americans have such a difficult time understanding how the terrorists were able to carry out these plans? To decide these things, we automatically, albeit many times nonconsciously, imagine how the passengers on those planes must have felt, empathically experiencing both what they were thinking about and their emotions around these thoughts by imagining ourselves in the fateful plane. For many, just thinking of the images of planes hitting buildings induces a fearful mindset with all its physiological manifestations, like a racing heart and anxious thoughts. By contrast, we have difficulty empathizing with the terrorists who brought down the planes, because the values, morals, and emotions that motivated these men are so different from our own.

Recent advances in methodologies such as brain imaging have led to unprecedented explorations into the neuroscientific bases of such social processing, affective responding, and their relation to learning, and have shed new light on their workings. These new discoveries link body and mind, self and other, in ways that call into question the traditional dissection of the mind and the brain into modality and domain-specific modules, underlain by unique and nonoverlapping physiological and brain responses. In demonstrating the functional overlap between low-level systems for physiological regulation and somatosensation with systems involved in the most complex of mental states (Immordino-Yang, McColl, Damasio, & Damasio, 2009), these discoveries dissolve traditional boundaries between nature and nurture in development (Immordino-Yang & Fischer, 2009). They suggest instead that complex social and emotional processing co-opt and specialize regions originally evolved for more primitive functions, such as homeostatic regulation, consciousness regulation, and the feeling of the body (Immordino-Yang, Chiao, & Fiske, 2010). Further, these findings underscore the importance of emotion in “rational” learning and decision-making in both social and nonsocial contexts (Damasio, 2005; Haidt, 2001; Immordino-Yang & Damasio, 2007), demonstrating the primacy of evaluative, reward-based and pain-based processing to learning, and our human propensity toward subjective, social thinking.

These new discoveries stand in contrast to traditional Western views of the mind and body, such as that of Descartes, that divorced high-level, rational thought from what were thought of as the basal, emotional, instinctual processes of the body (Damasio, 1994/2005). Far from divorcing emotions from thinking, the new research collectively suggests that emotions, such as anger, fear, happiness and sadness, are cognitive and physiological processes that involve both the body and mind (Barrett, 2009; Damasio, 1994/2005; Damasio et al., 2000). As such, emotions utilize brain systems for body regulation (e.g., for blood pressure, heart rate, respiration, digestion) and sensation (e.g., for physical pain or pleasure, for stomach ache). They also influence brain systems for cognition, changing thought in characteristic ways – from the desire to seek revenge in anger, to the search for escape in fear, to the receptive openness to others in happiness, to the ruminating on lost people, opportunities or belongings in sadness. In each case, the emotion can be played out on the face and body, a process that is felt via neural systems for sensing and regulating the body. And in each case, these feelings interact with other thoughts to change the mind in characteristic ways, and to help people learn from their experiences.

Further, educators have long known that thinking and learning, as simultaneously cognitive and emotional processes, are not carried out in a vacuum, but in social and cultural contexts (Fischer & Bidell, 2006). A major part of how people make decisions has to do with their past social experiences, reputation, and cultural history. Now, social neuroscience is revealing some of the basic biological mechanisms by which social learning takes place (Frith & Frith, 2007; Mitchell, 2008). According to current evidence, social processing and learning generally involve internalizing one’s own subjective interpretations of other people’s feelings and actions (Uddin, Iacoboni, Lange, & Keenan, 2007). We perceive and understand other people’s feelings and actions in relation to our own beliefs and goals, and vicariously experience these feelings and actions using some of the same brain systems that would be invoked if the feelings and actions were our own (Immordino-Yang, 2008). Just as affective neuroscientific evidence links our bodies and minds in processes of emotion, social neuroscientific evidence links our own selves to the understanding of other people.

For example, it is now known that the key brain systems involved in the direct sensation of physical pain, especially systems for the sensation of the gut and viscera (e.g., during stomach ache or cigarette craving), are also involved in the feeling of one’s own social or psychological pain (Decety & Chaminade, 2003; Eisenberger & Lieberman, 2004; Panksepp, 2005), as well as in the feeling of social emotions about another person’s psychologically or physically painful, or admirable, circumstances (Immordino-Yang et al., 2009). Put simply, the poets had it right all along: feeling emotions about other people, including in moral contexts for judgments of fairness, virtue, and reciprocity, involve the brain systems responsible for “gut feelings” like stomach ache (Greene, Sommerville, Nystrom, Darley, & Cohen, 2001; Lieberman & Eisenberger, 2009), and systems that are responsible for the construction and awareness of one’s own consciousness (i.e., the experience of “self”; Damasio, 2005; Moll, de Oliveira-Souza, & Zahn, 2008). Overall, affective neuroscience, together with psychology, are documenting the myriad ways in which the body and mind are interdependent during emotion, and therefore the myriad ways in which emotions organize (and bias) reasoning, judgments of self and others, and retrieval of memories during learning (Immordino-Yang & Damasio, 2007).

Related to this, the physiology of the social emotions that govern our interpersonal relationships and moral sense appears to involve dynamic interactions between neural systems for bodily sensation and awareness – the same systems that are known to be involved in the feeling of basic emotions like anger, fear, and disgust – and systems that support other aspects of cognition and emotion regulation, including regions involved in episodic memory retrieval and perspective-taking in relation to the self (Harrison, Gray, Gianaros, & Critchley, 2010; Zaki, Ochsner, Hanellin, Wager, & Mackey, 2007). During complex social emotions like admiration and compassion, for example, neural regions associated with memory and social cognitive functions appear to be functionally interconnected, or “talking,” with neural systems involved in somatosensation for the internal, visceral body, and systems involved in consciousness regulation (i.e., brainstem systems responsible for sleep–wake cycles, arousal, etc.), in patterns that reflect not only involvement in the induction or onset of the emotion, but in its maintenance and experiential aspects as well. The cross talk between these neural systems suggests that social emotions endure, guiding our decisions, ongoing engagement, and learning. Moreover, the data suggest that these emotions may get their motivational power through coordinating neural mechanisms responsible for complex computations and knowledge with mechanisms that facilitate retrieval of our own personal history, all the while ­colored by reactions played out on homeostatic regulatory systems that, in the most basic sense, keep our bodies alive and our minds attentive.

Information Processing in Humans and Computers: Top-Down, Bottom-Up, and the Fundamental Importance of Human Subjectivity

Let us begin this section with a simple question: why are you, the reader, interested in neuroscientific perspectives on the design of digital learning environments? Of all the possible range of intelligent behaviors available to you, from planting a garden to playing a piano sonata to drinking a coffee with friends, you chose to spend energy on thinking about ideas and evidence pertinent to this topic at this moment. Why?

We suspect that, although this obvious question may initially puzzle you, it would then compel you to respond to the effect that you feel this topic is useful, engaging and warrants attention, that designing better digital learning technologies will help learners and may gain you recognition and notoriety in the process, that you take pleasure in working on this problem, or a myriad of other possible answers in the same vein. And your answers would reveal a central and common misconception in understanding learning: that rational, logical intelligence is somehow separable or independent from emotion, and from subjective, self-relevant goals.

Human cognition, or the faculties for processing information, applying knowledge, and making decisions, differs greatly from the way information is represented and processed by computers. Most importantly, human information processing is driven by subjective and culturally founded values. Building from what we saw in the previous section, these values are instantiated – they come to organize our behavior – through dynamic interplays between complex thought and knowledge, and generally nonconscious, low level physiological reactions that shape our feelings and behavior and motivate us toward particular forms of engagement. Put another way, we humans are capable of both top-down and bottom-up strategies of attending and information processing; our cognition involves decomposing or breaking information into its composite parts, as well piecing together and integrating information into more complex representations (Immordino-Yang & Fischer, 2009). What is more, because these processes happen in accordance with prior learning and expectations, both top-down and bottom-up processing are organized by our desires, needs, and goals, sometimes conscious and sometimes not. As biological beings, a central part of explaining how we do things lies in explaining why we do them.

To see what we mean, let us return to the neurobiological evidence presented above, concerning the relationship between the body and the mind. If the feeling of the body (or simulated body) during emotion can shape the way we think, which ample evidence suggests that it can, this shaping would happen via the sensing of the body, or via perception. However, such sensations are not merely recorded in a value-neutral or objective way. All sensations are not of equal importance. Rather, sensations are assigned valence, starting with pleasure and pain and growing from there in complexity. Even the simple visual perception of objects or situations in the environment is understood in terms of its propensity to cause harm or good in relation to the current situation and context. In turn, we respond accordingly to maximize good and avoid harm, as we subjectively perceive and understand the consequences. Depending on the context, these responses can relate to our wellbeing in a basic survival sense, or in a more evolutionarily evolved, sociocultural sense.

Taken together, these appraisals, values and sensations lead to emotion, which supports and drives what we traditionally call cognition. Quite literally, and as the term “emotion” suggests, we are “moved by” the valences we assign to perceptions (or simulated perceptions); and in this way our perceptions and simulated perceptions “motivate” us to behave in meaningful ways (Immordino-Yang & Sylvan, 2010). Although a purely cognitive account of information processing describes perfectly the computations that govern artificial intelligence and embodiment (in the form of mobile robots’ behavior), from our perspective this represents a fundamental rift between artificial and biological intelligence that must be dealt with in the design of interfaces that facilitate useful interactions between the two.

From Me to It and It to Me: Applying Principles from Affective and Social Neuroscience to Design Better Learning Technologies

Humans are born with the propensity to impose order, to classify and organize our environment in accordance with our individual ways of theorizing about and acting in the world. The content and order of these theories and actions is the result of interaction between biological, social and cultural life experiences. As children develop, they encounter new experiences that shape and reshape existing neural networks and schemas, and impact their cognitive, social, and emotional development. Because of this, the hard-wired patterns of neural connectivity that underlie innate functional modules, such as those that facilitate social evaluation, are dynamically sculpted by social and cultural experiences as they are subjectively perceived and emotionally “felt.” In short, our personal experiences through development provide a platform on which to understand and relate to the thoughts and actions of other people.

But what if our social companion is not an acculturated, sentient, subjectively evaluating biological being, but instead a computer? How, then, can our past experiences and cultural knowledge help us to predict our computer companion’s actions, to understand its purpose, to collaborate on problem solving? Normally, the design of digital environments focuses on how computers can most effectively accommodate humans, and adapt to their needs and situations. But what about considering the complementary process – how humans adapt to computers – in designing the digital environment? New advances in social and affective neuroscience are making increasingly clear that humans use subjective, emotional processing to think and to learn, and that they use emotional and social processing to adapt to the current ­context and accommodate their social partners. Given the various forms of evidence that humans naturally anthropomorphize computers, a better understanding of the socio-emotional nature of computer users’ struggles to adapt to the digital context might afford a new vantage point from which to predict, and eventually influence, human users’ adaptations and learning.

In a learning environment such as a traditional classroom, each student brings her unique goals, knowledge and decisions that have been shaped by her social and cognitive experiences and that she must learn to use empathically to understand the teacher’s actions, whether the teacher is a person or a computer. For example, to learn how to build a model using a computer, the student must first understand the goal of the exercise, be able to relate this goal to her own skills and memories, and be able to translate her skills into commands that describe the procedures of the computer. Using computers and other technologies to learn and perform tasks presents the student with the challenge of mentally discerning and reconstructing actions with often times invisible goals and procedures. Not only do these processes depend upon knowledge of how computers work, they vary with the student’s subjective, emotional and personal history, and with her present interests and goals.

Here we suggest that perhaps one of the main difficulties that humans (and especially computer novices) have with computer interfaces is that the humans have trouble anticipating and understanding what the computer will do and why – in effect, because we have never lived as a computer, we have trouble “empathizing” with them and sharing their processing state, the way we would naturally strive to do with another person. If this is the case, perhaps rather than striving to build computer interfaces that seem as human-like and emotionally competent as possible, we should aim instead to make the programs and interfaces as transparent as possible. This does not mean that the technical information that makes the computer run would necessarily be available, but that the goals and the motivations of the digital environment would be readily apparent. A learner using the digital environment would understand what the program is good for, what the learning goal is, and therefore how best to engage with the computer without frustration or boredom.

Related to this, because computers do not have emotion, why not find ways that the human user can supply the emotion-relevant features to the human–computer interaction by giving the person some control over the critical aspects of how the interface and environment look, feel, and behave? A vast body of literature in education implicates “locus of control” as an important consideration when helping students in higher education environments to perform better (Dweck, 1999; Pekrun, 2011). That is, when students perceive that they have intrinsic control over the content, context, and pace of their learning, they begin to believe that they can be successful, and they invest more personal effort toward the academic task. Drawing from this, it seems crucial for learning technologies to be designed such that they do not give the students using them a sense of reliance or dependence on the machine, but instead foster a sense of agency that empowers the student to master skills that he could not have managed without computerized assistance. Engaging the student in an interaction rather than in a unidirectional manipulation by one conversational partner or the other (where either the person or the machine drive), students may be more likely to productively interact with the digital learning environment and to use it to facilitate performance (see also D’Mello, Lehman, & Graesser, 2011).

From Social Interactions to Digital Media for Learning

We began our chapter with a scenario involving students interacting with digital media, and asked why some students may be engaged with the activity, while others may be bored and listless. How can this question be informed by the above discussions on the embodiment of emotion, the interdependence of the body and mind, and the involvement of self-related processing in social emotions and motivation?

Affective and social neuroscience findings are suggesting that emotion and cognition, body and mind, work together in students of all ages. People behave in accordance with subjective goals and interests, built up over a lifetime of living and acting in a social and emotional world. By contrast, the values, judgments and calculations made by computers follow from the data, algorithms, and system constraints that their programmers choose to give them. Because the parameters governing these calculations are decided beforehand and are mainly invisible to the novice human user, many people may have trouble understanding and predicting the computer’s actions. In effect, they may have trouble “empathizing” – and therefore become frustrated and disengaged. For the actions and responses of the digital interface to be perceived as useful and productive, and for novice learners to effectively engage the digital learning environment as a collaborative partner, digital media designers might consider ways to make human–computer exchanges more akin to good social encounters: the goals should be transparent, the computer partner’s actions should be predictable and related to the subjective needs of the human learner, and each partner in the exchange should have an appropriate share of the control.