Keywords

1 Introduction

Fictional and speculative design scenarios can precede scientific and technological developments. I take this position to be conceptually and methodologically more plausible than presumed within disciplinary boundaries and methods as merely inspiring indication. In the following perspective on Holonovel as interactive and intelligent (new media) system based on narrative computation, a framework for understanding the underlying principles of human-level narrative comprehension is drawn. It is argued that narrative should be used as interface for human–computer interaction relying on humans’ practices that often make sense of the world in narrative terms. This leads to questions on narrative as sense-making of the world—how perception and cognition interact with the world, on one hand, and what constitutes narrative intelligence in a computed (artificial) system through which our experiences are mediated. In the paper, theoretical framework of narrative skills and their relation to cognition is considered as direction for research and design of computational mediated experiences and for re-conceptualizing modes of interaction and how we use tools and interfaces.

Rather than putting all creative or economical efforts into optimizing technological advancements, whether in fictional or scientific scenarios, we can try to identify new approaches to interaction and narrative as our engagement with the world that has always co-existed with the tools and technologies and try to understand “how models of thought are intimately tied to the available models of computation” [1].

In this line of investigation, one area of technological research that has been open to approaches other than computer sciences and that uses ideas from arts and humanities in actual developments is the field of Narrative Intelligence:

Narrative intelligence refers to research into human narratives and story-telling, as well as the development of software or robotic agents that either support human story-telling or are themselves story-tellers and/or story-listeners [2].

Narrative Intelligence, that is, the narrative organization of experience in the domain of artificial (computational) systems is a work that is inherently interdisciplinary and in its history, relied on practices and methodologies from arts, psychology, drama, literary and cultural studies. Even though language is often considered as uniquely human characteristic and a way of communicating experience and shaping the mind, and therefore applications of narrative have been centered on this modality, non-verbal or implicit aspects such as our embodiment and situatedness occupy research in HCI and narrative interfaces as well. Following the insights on narrative from psychology, it is argued that artificial (computational) systems will be more understandable with narrative presentation especially because humans use narrative for understanding intentional behavior [3].

If Holonovel concept could serve as a reflection on our current and future technologically intertwined mind with the world, then we can relate this condition to the idea that technology is anthropologically constitutive [4] and that humanity cannot be understood without its technological dimension. This paves the way to conceptualizing our engagement with the world through the relationship between an organism and environment, and further between a user and the tool.

The key ideas on narrative intelligence in the paper that are going to be assessed from the perspective of “enaction” with the world rather that “computation”, revolve around intelligence being “determined by the dynamics of interaction with the world” [1] and around computational narrative intelligence being “as much about human-computer interaction as it is about solving hard artificial intelligence problems” [5].

2 Interfacing with Reality

In the context of new media and technologically mediated experiences, much has been debated on the concepts of immersion, illusion of non-mediation or presence—the experience of “being there” in a simulated or mediated environment. The aim and scope of this paper is not directed towards conceptually clarifying these phenomena for the purposes of possible or imagined Holonovel environment as simulated and interactive environment. Rather, the framework that is presented here in relation to our interaction with technologically mediated environment departs from describing modes of representation in relation to experience that presuppose some “suspension of disbelief” needed to establish engagement. These discourses usually related to virtual environments and immersion tend to exhibit somewhat reductive conceptual position centered on transportation into another world or dimension where experience of altered reality is happening. Moreover, these kinds of concepts on participation in media environments or fictional settings additionally force divisions and modalities to perception as well as cognition that does not do justice to the complexity of experience (e.g., real vs virtual, belief vs disbelief, immersive vs non-immersive, low vs high order, creator vs user etc.).

Popular demands for immersion, whether defined in weak sense, for example in popular culture and marketing, or more rigorously specified for engineering purposes, tend to oppose traditional “old” audio-visual media as non-immersive due to the mode of presentation (no goggles, sensors, etc.). However, this seems to echo the same discourse found in “old” media experiences and related film studies—the idea or demand for the seamlessness of experience or illusion of non-mediation in a way that spectator should be tricked into the process and believe in the (re)presentation of reality without noticing its mechanism. In “old” media such as film, this was typically seen through the technological “apparatus” of film that selectively represents the reality through camera projection and seamless editing. This places the spectator into a passive position of meaning-making. Furthermore, it “sutures” the spectator, in a way that the spectator is stitched seamlessly into the ideological apparatus of the medium without noticing the mechanism behind it or being able to resist it (Marxist, psychoanalytic, structuralist film theories). In “new” immersive media, this stitching is performed with seamless interface or mediation transparency with technological tools while still preserving the passive position of the spectator: “Immersion in a virtual world is viewed by most theorists of postmodernism as a passive subjection to the authority of the world-designer—a subjection exemplified by the entrapment of tourists in the self-enclosed virtual realities of theme parks or vacation resorts (where the visitor’s only freedom is the freedom to use his credit card)” [6].

Even in the most recent empirical studies that examine our neurocognitive engagement with audio-visual media or films in particular (neurocinematics) and previously in the theoretical framework accounting for our biology and evolutionary makeup in dealing with media tools (cognitive film studies), the perspective is still the one of a passive recipient of media input with predictable behavioral output. In other words, it builds the “Discourse of Control” over spectators’ minds with techniques that predict neural or cognitive responses to (film) stimuli [7] and according to which effectiveness of media tools is measured. Relying on such discourses of spectators’ engagements, our mediated experiences are described and regulated in modes of representation assuming passive subjection rather than modes of interaction with the world.

Departing from conceptualization of engagement in mediated environments as input-output processes, how can we describe or design our coupling with the environment and technological tools with interfaces that support interactive dimension of our embodied, social and cognitive life?

3 Narrative as Interaction Tool (Enacting the World)

Rather than defining our engagement with the environment (real or fictional) as modes of representations of the world, the approach that emphasizes interdependence between action and perception can be described through the idea of “enaction” [8].

3.1 Enaction

Therefore, instead of taking the perspective of a spectator/user/agent presented with a world, enactive relation suggests that “a living organism enacts the world it lives in; its effective, embodied action in the world actually constitutes its perception and thereby grounds its cognition” [9]. These embodied and situated notions about the structure of our engagement with the world refer to enactivism—a novel approach to cognitive sciences that departs from previous computational modes of cognition exemplified in the metaphor of the brain as a computer processing machine. Enactivism sees perception, cognition and action as: “three facets of this single process of adaptively coping with the world (what the enactive literature refers to as “sense-making”) rather than being distinguishable links in a chain of processes that begin with “input” at the sensory surfaces and end with “output” at the muscles” [10].

The main idea behind agent’s engagement with its environment in enactive approach to cognition and perception is that perception consists of perceptually guided action and that cognitive structures emerge from the recurrent sensorimotor patterns that enable action to be perceptually guided [8]. Enactive approach emphasizes perceptually guided action and engagement that depends on how the organism perceives the world rather than the world being described from the outside for an organism to interact with. Thus, enactive account of the properties of the world and interaction between agent and environment leaves more space for examining this relation from the perspective of agent’s possibilities for action rather than describing the world as represented or modelled and then expected to be navigated.

While enactivist program has been so far widely accepted for the view on perception as embodied action (in terms of “lower level” cognition, perception, bodily action), the main criticism towards enactivist program questions its possibility to explain “higher order” cognition—thought, reasoning, planning, problem solving. As many recent research developments within the enactive program discuss questions related to consciousness, language, social concepts, mathematics etc. [9], new and distinctive articulations and methodologies apply enactive approach to complex forms of cognitions and reject sharp distinctions between “lower-level” embodiment and “higher-level” cognition.

Departing from the views on perception as a “mode of presentation” of a particular stimulus, as different sensorial modalities that represent the world, enactive account defines perception as “modes of action” where perception can be understood as “structured by the intentional actions of the agent” [10]. Thus, cognition is not added to perception after the fact, it is not an additional inferential process but it is inherent in the process of perception itself [10].

Understanding perception and cognition without hierarchy but as a unified process that does not begin with “input” of information and end with “output” of the representation of the world brings forward the idea of “sense-making” as active participation in generation of meaning between organism and its environment.

3.2 Narrative Interactivity

A particular structuring of our experiences in the world, our perceptions and cognitions, relates to our capacity to organize experience in a narrative way and to narrate this experience and interactions with the world and others.

In regard to human–tool interaction or human–human interaction in mediated forms, it is expected that humans will deal with interaction in a way that makes narrative sense and that technology (software and robotic agents) that fail to optimize human–tool relationship in a fluid way lack narrative intelligence [11]. Aspects that underlie narrative engagements and pose main questions for modelling this interaction in computationally mediated environments relate to implicit understanding in communication (human-level narrative comprehension) and the way in which narrative constitutes a cognitive tool for situated understanding [12].

Following enactive view on “sense-making”, the notion of narrative and its relation to cognition can be understood differently from its representational mode of reality or as higher order reflective activity bound to language use.

The following perspectives can help clarify narrative as interactive process that has constitutive role in reality and further help to conceptualize narrative as both embodied and cognitive tool that situates our interactions whether in direct or mediated engagements. This provides the point at which the possible computational narrative system and the user make contact.

In enactive view on literary narratives that follows from research on social cognition, narrative is defined as “interactional process of co-constructing a story-world with a narrator” [13]. In social cognition, the interactive nature of sense-making is exemplified through the process of participatory sense-making: “the coordination of intentional activity in interaction” [14].

This social aspect of cognition identifies narrative as means of engaging with other minds. However, depending on approaches to cognition, there are opposing perspectives on narrative understanding of other minds. One approach is supporting previously mentioned cognitive computationalist approaches that see the world represented as internal model (inferentialist), where “narrative fabulation is the cultural expression of innate Theory of Mind (ToM) abilities, in that it seems to require a pre-existing ability to theorize or simulate the ways in which the mental states of others dispose them to act the way they do” [15]. In the opposite perspective, “enactivists argue for narrative as the origin of our ability to engage with other minds. That is, narratives are volunteered as a cultural repository of explanatory precedents, background knowledge, interactional schemas and dispositional primers that we actively use to understand why others act the way they do” [15].

Recognizing narrative as enactive interactional process and breaking from both structuralist and cognitive study of narratives towards embodying and situating narrative engagement, narrative can be understood as something akin to the “rhythmic entrainment” of music that “implies that biological and cultural values are inextricably bound up with the emotional and cognitive impact of narrative itself” [16].

Non-verbal and physical strategies of human ancestors point to evolutionary origin of communicating in narrative format and are believed to co-evolve with social dynamics as The Narrative Intelligence Hypothesis [17] suggests.

Origins of narrative are identified also from its earliest expression in movement, “the innate sensorimotor intelligence of a hypermobile human body” [18] where embodied activity is seen as having inherent narrative structure.

Finally, our capacity for understanding actions in terms of reason (our folk psychology) can be defined as narrative practice [19, 20]—a kind of skillful competence that does not require conscious metalizing but depends upon having special training with narratives as humans do. The Narrative Practice Hypothesis by Hutto [19, 20] conveys of these practices in terms of embodied action or expressive behavior and further extends them to shared engagement with situations in the world. It suggests that another person’s intentions are not hidden beliefs and desires but can be explicitly expressed in certain embodied practices that are emotional, sensory-motor, perceptual, and nonconceptual. These practices lie in embodied action or expressive behavior (person’s bodily movements, facial gestures, eye direction, and so on). This is primary, immediate, non-mentalizing mode of interaction as a way of understanding others intentions and feelings in their embodied and situated component. Furthermore, they are always perceived in relation to ours or other people’s goals, intentions, or possible actions. As The Narrative Practice Hypothesis further proposes—to understand intentional action, it would demand more than simply knowing which beliefs and desires constitute actions but it would require to situate and evaluate reasons in wider contexts (in terms of cultural norms or person’s history or values) [19, 20].

4 Conclusion

Common features of current immersive and interactive media, typically virtual environments, still rely on the basic premise of the world that is there to be navigated by which they presuppose representation or a model of the world. Traditional approaches in designing such systems still support views that rely on representation of reality by trying to simulate physical reality: “For instance, engineers try to perfect the underlying physics model and to increase the resolution of the display while at the same time having to make sure that this additional complexity does not negatively impact the responsiveness of the system. This strategy may make sense from a classical cognitivist theory of mind, which holds that perception is about creating detailed internal world models. But it appears as misguided from the perspective of embodied and situated robotics, which emphasize interaction rather than representation. Similarly, we now have extensive empirical evidence that supports an enactive account of perception, which holds that our sense of experiencing a highly detailed world does not depend on a highly detailed inner model, but is enabled by our practical know-how of regulating sensorimotor dependencies” [21].

The concept of Narrative Intelligence in designing new interfaces for human–computer interaction could benefit from enactive account on interaction in its understanding of implicitness (embodiment and situatedness) in communication or generation of meaning. Instead of inferential or representational processes, enactive narrative engagement is seen as dynamical process of participatory sense-making and embodied interaction through which a meaning is generated.