Keywords

1 Introduction

All technologies are for people to use to improve the quality of their everyday lives (Saariluoma et al. 2016). Thus, it makes sense to consider how emotions are involved in human–technology interaction (HTI). Emotions define the value of objects for individual people; the quality of life is closely linked to emotional harmony. Therefore, emotions represent a central topic in investigating and designing technical artefacts and technologies.

The study of emotions mainly belongs to psychology; thus, they are psychological phenomena. User psychology is a scientific discourse that investigates the emotions associated with HTI in order to explain (and therefore to understand) why people relate to technologies as they do (Saariluoma and Jokinen 2014). Emotions explain why people do not like to use some types of technology, such as washing machines. For instance, these machines may be ugly, noisy, smelly, unsafe or unreliable. All of these factors generate emotions in users’ minds, which in turn explain people’s reluctance to use the product.

Technologies may also produce important positive features such as fame, goodwill, brand and positive status. Users feel good about employing technologies that fit with their values, self-image or lifestyle. It is valuable for designers to understand how people emotionally experience their products by studying user psychology, which explicates the emotional experiences associated with a technology or technical artefact.

Psychologically, emotions are phenomena that are very intimately linked to motives. They define why people do what they do in a psychological sense. Therefore, in positive emotional states people pursue certain goals and products, and in negative states they try to avoid them. Thus, emotions can explain many aspects of human motivation.

Liking and disliking are not marketing or service design issues; they are modern psychology issues. They are considered important in a number of well-known design paradigms such as Kansei engineering, affective ergonomics, emotional usability and user experience research (Brave and Nass 2009; Hassenzahl 2011; Helander and Khalid 2006; Nagamashi 2011; Norman 2004). The term dynamic user psychology refers to the psychology of emotions and motives, which are linked through liking and disliking (Brave and Nass 2009; Hassenzahl 2011; Nagamashi 2011; Norman, 2004; Neisser 1967).

The importance of emotions should not be underestimated. Emotional questions not only concern individual beauty and pleasure; they can affect large numbers of people, and thus the whole society. For example, the Three Mile Island (1979), Chernobyl (1986), Fosmark (2006) and Fukushima (2011) accidents made people feel unsafe; many individuals lost their trust in nuclear power as a result (Friedman 2011; Saariluoma et al. 2016).

Likes and dislikes explain much of why clients accept and adopt some new technologies. The emotional and motivational dimensions of the human mind are central, and they are different from cognition in both their content and their neural representations (Neisser 1967).

Dynamic psychology has traditionally arisen from the clinical personality theories of Freud (1917/2000). However, there is no reason to limit the use of the term to psychoanalysis. In modern psychology, the most relevant paradigm of dynamic psychology is perhaps positive psychology. This paradigm analyses issues such as well-being, contentment, satisfaction, happiness and the flow of experience, which have proven to be important in many contexts of user experience (Seligman and Csikszentmihalyi 2000). Thus dynamic psychology involves the mental forces explaining why people act as they do.

2 Questions of Emotional User Psychology

User psychology introduces typical design questions, such as

  • Do certain colours excite people?

  • Does the brand of my watch communicate a good image?

  • Do people feel brand loyalty?

  • Do I feel that it is safe enough to smoke?

  • Is the product offensive?

  • Do children like their toys?

  • Is buying this product moral?

  • Is a particular use of a technical artefact ethical?

Emotional interaction research seeks to determine how objects are emotionally experienced, as well as the rational grounds for solving emotional design issues. This process involves investigating general psychological phenomena and using this knowledge to find solutions to technological problems. Following this line of thinking, it is possible to discuss the role of the psychology of dynamic interaction on a scientific level.

A core problem associated with evaluating the user psychology of emotional interactions is analyzing the emotional processes involved in interactions. Designers have to be able to explicate which emotions are relevant—and in what ways—when people interact with technologies. Good analytical practices are necessary to provide clear information about how the design prototypes should be improved.

Saariluoma and Jokinen (2014) presented an example of emotional analysis. They applied a type of psycho-semantic analysis that has been used in Kansei engineering (Nagamashi 2011). Saariluoma and Jokinen (2014) used a Likert-type survey in which subjects’ basic emotions were used as dimensions. They employed multidimensional scaling to identify an emotional dichotomy of competence vs. frustration: if people were able to use a technology they felt competent, but if they failed they became frustrated.

3 Emotions and the Mind

Emotions form a fundamental mental process in the human mind (Ekman 1999; Frijda 1986; LeDoux 1998; Rolls 2000). They serve as an action control system. Since emotions are always present in human information processing, they also represent a fundamental aspect of HTI. Progress on these issues is essential to the development of the field. User psychology should formulate good methods and concepts of emotional analysis for HTI and reliable practices to apply these methods in the product development process.

Emotions play a valuable role in explaining people’s behaviours (Ekman 1999; Frijda 1986, 1988, 2007; Oatley 2006). Evolutionarily, emotions establish a primitive neural system, and therefore the main characteristics of many human emotions can be found in other animals as well (Darwin 1872/1999 Panksepp 1998). Emotional processing is mainly centred in the subcortical areas in the brain, which are typical to evolutionarily early systems (McClean 1990; Rolls 2000). However, emotional processes are fundamental in controlling many behavioural processes. Cognition can explain why something is worth fearing but not fear itself.

Emotional systems operate holistically, and feature equally important physiological, psychological and social characteristics. There are numerous ample reviews and overviews of emotional processes, thus, in this book, we do not review basic theories of emotions in detail. If interested in further reading, we recommend that readers refer to following literature (Ekman 1999; Frijda 1986, 1988; Oatley 2006; Panksepp 1998; Power and Dalgleish 1997; Rolls 2000; Lazarus and Lazarus 1994). In any case, human–technology interaction research should examine emotions holistically.

A ready example of a long line of holistic thinking of emotions is hierarchical affect theory (Laros et al. 2005; Watson and Stanton 2017). In this theory, it is essential to divide emotions into groups on the grounds of positivity vs. negativity. Thus, the theory raises the issue of conceptual dimensions of hierarchy which are essential in defining emotions. In this chapter, the analysis of conceptual dimensions will be further discussed and elaborated.

Emotional state—how people feel at a certain moment in time—is a foundational concept in any analysis of the emotional mind. It illustrates the emotional aspect of a prevailing mental representation. Emotional states are thus aspects of mental states. They bring additional knowledge regarding the mental content of representations. These aspects must be explicated one at a time to understand how emotions operate with mental representations. Emotional states have four main basic dimensions.

The first is the intensity of emotions, which is called arousal (Kahnemann 1973; Power and Dalgleish 1997). There is a graded structure typical to all human emotions, and the intensity or level of arousal is used to express how emotions vary. Frustration and hate, for example, are emotional states towards an object, person or state of affairs (Norman 2004). Irritation can be seen as a less intense version of the same emotion.

The level of human performance depends on the intensity of the prevailing emotional state (James 1890; Kahnemann 1973). When arousal is very low, the human performance is not intense. When arousal increases, the performance capacity also increases. However, after a certain point, increasing arousal no longer improves the level of performance, which starts to weaken. When frightened, for example, people seldom act rationally because their cognitive capacity has diminished. Understanding this principle, called the Yerkes–Dodson law, and other aspects of arousal support the design of technical artefacts and systems that are meant to be used in critical situations or under pressure (Kahnemann 1973).

The second dimension of an emotional state is the extent of an emotion. A person’s response to a certain incident may be very short, such as a rapid reaction to a surprise (Power and Dalgleish 1997). In technology, design surprise is sometimes referred to as the wow effect. An emotional state can also last for weeks, as in moods or attitudes (Power and Dalgleish 1997). A good example of a long-lasting emotional response tendency is temperament, which forms in infancy and early childhood.

The third property of emotional states is their content. For instance, joy is a positive emotion and sorrow is its negative counterpart. Consequently, technical artefacts may include different aspects of emotional content, for example, when considering the feelings aroused by the design of a product. In this kind of analysis, two new dimensions of human emotion become relevant: emotional valence illustrates the negativity or positivity of the emotion, and emotional theme refers to its general content (Lazarus 1991; Saariluoma and Jokinen 2014). Emotions can be positive or negative (Lazarus and Lazarus 1994; Spinoza 1675/1955). Emotional valence is strongly connected to the pleasantness or unpleasantness of the emotional state, and thus valence is essential in defining the desirability of emotional contacts. A positive emotional contact is an important design goal in HTI (Jordan 2000; Hassenzahl 2011). Emotional theme refers to what the emotion is (e.g. joy, disgust). They are activated by different circumstances and mental states. Thus joy refers to a feeling of happiness and disgust to a distasteful or unpleasant feeling. It can be related to smells and foods, but it can also refer to experiencing social habits, for example.

An example may clarify the difference between valence and theme. The feeling of joy associated with using technologies usually embodies positive emotions such as wellness, commitment and positivity. Trust is also a positive emotion that can refer, for example, to human reliance on a given technology or its users. Thus, these two emotions—joy and trust—have the same positive valence but different themes. Hence, the theme defines the content of emotional states in a more sophisticated manner than valence.

Emotions determine the subjective meaning of a situation to an individual; they are closely connected to human action (Frijda 1986, 1988). They indicate the personal meaning of the target and the possible actions that people might take (Frijda 1986, 1988; Lazarus and Lazarus 1994). The feeling of fear, for example, makes people flee, whereas curiosity often results in approaching a target. Because these emotional characteristics are built into HTI processes, emotions are a critical element of situational representations.

Emotional responses to things or incidents are not static by nature; they change over time and continue to develop over a person’s lifetime (Lazarus and Lazarus 1994; Power and Dalgleish 1997). For instance, people who reacted hastily or aggressively to certain incidents in their youth may behave moderately and calmly in the same situations when they are more mature. This process of emotional development is called emotional learning.

Emotional learning processes change the content of emotions stored in the memory—so-called emotional schemas—which people use when they select information to retain during the perception process and build memory representations. Emotional learning is an important aspect of people’s relationship to technologies. A user who once regarded mobile services as redundant may later become an advocate of this kind of technology after having learned to use and understand its practical value. In this case, a change in an emotional meaning is explained by a change in the content of emotional schemas that are used to construct emotional states through apperception.

4 Cognition and Emotions

Cognition refers to how people process information. It refers to perceiving, attending, languages and thinking. These processes create mental representations and illustrate to people how things are in the world. Of course, how people cognitively represent the prevailing situation around them affects how they emotionally encode a situation. The process that associates emotions with cognition is called appraisal (Frijda 1986, 1988, 2007).

If parents see a lion approach their child, they normally feel fear, which is an emotion rather than a cognitive phenomenon. If the lion is in a cage, they may find the situation amusing. Cognitive analysis helps people understand whether to be afraid in a particular situation.

Emotional states are not randomly activated. Their activation is based on an individual’s understanding of the prevailing state of affairs. If the situation is cognitively risky, this stimulates danger-related emotions such as excitement, fear and courage. Emotional representations are constructed in the human mind, based on cognitive content (Frijda 2007).

Appraisal is a core process in the psychology of emotions (Frijda 1986, 1988, 2007). It forms the emotional dimension of mental representations, which defines the value of a situation to the person experiencing it (Frijda 1986). Emotions associated with technologies must be empirically defined in HTI. People are individuals: one user may feel anger while another may feel guilt in the same situation. Emotional states also depend on the context, but there is no conceptual way to study HTI situations. It is therefore essential to empirically define the emotions associated with interaction use cases.

Linking cognitions and emotions is a central issue in HTI design. If people cannot reach their goals using the technical artefacts they have at hand, they become frustrated (Saariluoma and Jokinen 2014). The overall appearance and colours of a technology product affect the mind, as do pictures and ‘gestalts’ (Norman 2004). The mere image of the product can be emotionally very important, which is why considerable attention is paid to semiotics and art design. The associations of pleasure and displeasure are vital to a product’s success.

In the appraisal process, human cognitive representations are connected to active emotional states. If users do not think they can learn to use an e-learning technology, they become frustrated (Juutinen and Saariluoma 2007). As a consequence, they will have generally poor experiences. Thus, cognitive assessments can generate emotional frustration regarding the whole action.

These dimensions are not always rationally connected, as people may emotionally represent situations in an inadequate manner; in the case of technology usage, they may even misrepresent the technologies or their uses both cognitively and emotionally. This may shape an individual’s feeling of self-efficacy—that is, their confidence in their ability to successfully perform a task (Bandura 1997). Mistaken beliefs about one’s own incapacity may even lead to a self-fulfilling misconception.

Repeated failures create a negative atmosphere and lower self-efficacy, whereas success in using technology improves self-efficacy and generates positive feelings and pride. This makes people more willing to accept, use and train to use new technologies to achieve their goals (Juutinen and Saariluoma 2007). The example illustrates how appraisal is significant in indicating personal meanings of technologies to people, how people differentiate their emotions, as well as their cognitive and physiological behavioural responses. Thus appraisal connects cognitive evaluation with emotions and actual human actions. Individual preferences and action choices are constructed based on representations made during the appraisal process.

The challenge of appraisal-based HTI research is defining what kinds of cognitions activate certain kinds of emotional states. When people interact with a technology, they create human mental representations and corresponding emotional representations. These cognitive and emotional representations control human actions and define whether people like and accept a technology. Thus even a small emotionally misplaced detail in a user interface can easily decrease the value of the technology in people’s minds.

5 From Emotions to Ethics

Interacting with technologies not only means that people can—and will—use technical artefacts. It is also important to ask how and why they use them (Saariluoma et al. 2016). Since ethics and emotions are related, it is important to discuss technology ethics in the context of emotional interactions.

Technologies are mostly ethically neutral. While kitchen knives are normally used to prepare food, they have also been used to kill people. Thus the focus when investigating ethical interactions with technologies is how a technology is used. It is important to think carefully about the ethical issues associated with using technologies, for instance, who has the right to own military class weapons, or should nuclear and coal power be abandoned. A special problem is artificial intelligence (AI) and autonomous technologies, because they have great performance capacities and have been ethically controlled to some degree.

Technology ethics focuses on how particular technologies can be used for good or bad. The role of emotional research in ethics involves defining what is good and bad. This is an emotional process in the human mind (Ayer 1936; Hume 1738/1968). People experience different situations and classify them as good or bad. The routes to these situations and the tools used to respond to them can thus be classified as either positive or negative. Through social discourse, people create the norms of social life (Habermas 2018).

Thus the ways in which people use technologies, the emotional experiences of consequences and social discourses constitute the norms for how to use technologies. Understanding emotions is a key point in this process: human cognitions tell people how things are, but emotions decide how they should be.

6 Emotions in HTI Design Discourses

There are some common concepts and discourses in analyzing emotions in interactions with technologies (Hassenzahl 2011; Helander and Khalid 2006; Nagamashi 2011; Norman 2004; Jordan 2000). Some of them discuss emotional features in products that activate users’ emotional states, while others are interested in what takes place in users’ minds and what the main characteristics of technology-relevant emotional states are.

Emotions have always been important to consider when designing technologies. Stone age objects often have features that cannot be explained on the basis of their use and usability only. They have unnecessary decorations, for example. Indeed, Plato paid specific attention in the tenth book of state to the beauty and purposefulness of technical objects such as flutes (Platon 601c–d).

In the twentieth century, a major opening to emotion-based thinking was Kansei engineering (Nagamashi 2011). Other paradigms include emotional and affective usability research (Helander and Khalid 2006; Norman 2004), user experience research (Hassenzahl 2011), affective design, pleasurable design (Jordan 2000) and entertainment design (Rauterberg 2010). These intimately linked scientific paradigms have illustrated the relevance of emotions in technology design (Helander and Khalid 2006).

Emotional interaction design is a generally recognized design discourse (Brave and Nass 2009; Hasenzahl 2011; Norman 2004, Saariluoma et al. 2016). Considerable work has been invested in finding ways to design products with excellent emotional interaction properties (Nagamashi 2011). A good explanation of the importance of emotional products is its ability to conquer consumer markets.

Brands, for example, are used to create particular emotional states (Stenros 2005). An example of a paradigm that renovates traditional art design thinking is Scandinavian design. Its minimalist, functional and practical forms are seen to reflect people’s relationship with the wild. These forms can be found in the works of designers such as Arne Jacobsen, Poul Henningsen and Alvar Aalto. The pursuit of the ideals of functionality and practicality can also be seen in the designs of information technology.

Emotional interaction does not concern only the immediate use of a technology. Technologies should also respond to emotions that are relevant for human life, such as trust and confidence. People should be able to trust that a given technology operates as expected, and that it will not create or exacerbate any practical or ethical problems. For example, when AI systems fail to be safe, this will decrease audience trust in technology, which may slow the development speed of the whole field. Negative trust in a particular technology can easily lead to technophobia (Brosnan 2002). For example, poor usability may cause stress. Technophobia can disappear as a consequence of direct or indirect positive emotional experiences.

Emotions are present all the time. Even neutral states have emotional values, and their influence is widespread in HTI. Much of human behaviour must be explained in emotional terms. Techno-dependencies provide an extreme example of emotion-related behaviour. Many people become game dependent as a result of playing games or surfing the net. Internet dependency is classified as a psychiatric illness that requires professional intervention. In South Korea, it has been assessed as one of the most serious public health issues, as it has been claimed to be connected to, for example, childhood obesity.

Knowledge of the nature of people’s relevant emotional states can be used to generate ethically responsive design goals for technologies, such as personal health monitoring systems. The field of emotional user psychology opens up an extensive set of design questions and challenges that must be seriously investigated.

7 Explanatory Emotional Design

The main goal of user psychology is to analyze, explain and design human interaction phenomena in concepts of psychological knowledge and methodology. Emotional user psychology investigates interaction phenomena, which can be analyzed and explained on the grounds of the psychological knowledge we have about human emotions. Of course, through the appraisal process relevant cognitive analyses can also be applied to solve users’ psychological problems.

Explaining and analyzing are the ideal functions of user psychological knowledge in technological design. Analysis enables designers to conceptualize the problem correctly, while explaining helps them answer ‘why’ questions. For instance, why an aircraft blew up or a bridge collapsed are typical questions in technical investigations.

Explaining is thus a common activity in technical design thinking. For example, there is a simple technical explanation based on natural laws why a car radiator breaks in sub-zero temperatures: the water in the radiator turns to ice, and the force of this expansion is too strong for the radiator to withstand (Hempel 1965). This problem can be fixed by using glycol to change the freezing point. Thus the explanation is linked to the problem, and solving it directs the designer towards a concrete solution. Explaining, converging and predicting form a chain of design thought.

There is a fundamental difference between traditional engineering and current HTI design practices. HTI design processes are generally dominated by intuitive procedures such as the free and creative generation of ideas, visioning and user testing. In these, explanatory practices typical to natural science have only a minor role or are absent, although they have the potential to place HTI design on a more solid foundation (Saariluoma 2005; Saariluoma and Jokinen 2014; Saariluoma and Oulasvirta 2010).

Explanations of user psychology should be based on psychology. The main challenge is to unify design problems, psychological methods and theories, and explanatory grounds in a sense-making manner.

Figure 2.1 shows that a designer binds the interaction problem and relevant scientific information together to generate a solution. Each design thought sequence of this type can be seen as a separate explanatory framework, but scientifically grounded design processes are generally characterized by this schema.

Fig. 2.1
figure 1

General explanatory framework

Full size image

The core concept of the explanatory process is coherence. This means that the explanandum (the phenomenon to be explained) and the explanans (the explaining phenomenon) should be coherent. For example, the superiority of good brands over unknown ones can be analyzed in terms of trust and evidence of past experiences. The structure of the explanatory argumentation is similar to Hempel’s radiator illustration.

Even in ethical design and responsible research, innovation processes and emotions are vital. Good and bad feelings help people decide whether a situation is worth pursuing. Ethical rules are derived from such analysis via social discourse, which helps explain social and legal norms. The denial of drugs, a kind of technology, is based on their unpleasant long-term consequences. The emotional states associated with these consequences explain why the related laws are so restrictive.

Psychology and design both have theoretical and practical features. In current design thinking, practicality can only be found by unifying science and design. Unification is the key to understanding the function of scientific knowledge in design thinking. It is well known that science and design have multiple connections, and that they are at their best when they are combined. However, precisely defining the connections between the two ways of thinking requires extending the circle of core issues.

Emotions are an important aspect in studies of the human mind; they also have a role in HTI design (Power and Dalgleish 1997; Norman 2004; Saariluoma et al. 2016). Psychologists and sociologists have made considerable progress in studying human emotions (Frijda 1986; Power and Dalgleish 1997). They are interested, for example, in the kinds of reactions that emotional words might inspire in people. Designers, for their part, incorporate emotional features into their systems to make them more appealing and more sellable (Norman 2004).

This chapter argues that emotions should form one of the main grounds in designing technologies. There are phenomena that can be investigated, analyzed and explained based on current knowledge about human emotions. However, when researchers and designers appreciate the importance of basic human research concepts in understanding HTI, it will be possible to conscientiously focus on these themes and make progress.